2024-04-25T11:14:11.633 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T11:14:11.634 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/damage} --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135 --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/orig.config.yaml 2024-04-25T11:14:11.682 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:14:11.774 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/damage} email: pdonnell@redhat.com first_in_suite: false job_id: '7673135' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds - MDS abort because newly corrupt dentry sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnRa7wX4JPdnmhx6mqM2K0CI7CJKm/oKVfTMnJlEeI9hGPzJCwffaZS0YEwdPeKlZ+ZxyiU5I3NCcetbZngXq0= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEf7cTDSBvuXWGk+YVTL9kClpiBtnl29eEgFtkLLFGlvZA5yYftciQnVOnYRweqQ6NQax72C3f7M/xO7E+9ntJk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T11:14:11.774 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T11:14:11.774 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T11:14:11.775 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T11:14:11.776 INFO:teuthology.task.internal:Checking packages... 2024-04-25T11:14:11.800 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T11:14:11.800 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T11:14:11.800 INFO:teuthology.packaging:ref: None 2024-04-25T11:14:11.800 INFO:teuthology.packaging:tag: None 2024-04-25T11:14:11.800 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T11:14:11.800 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:14:11.800 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T11:14:11.950 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T11:14:11.951 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T11:14:11.958 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T11:14:11.958 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T11:14:11.966 INFO:teuthology.task.internal:Saving configuration 2024-04-25T11:14:11.984 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T11:14:11.991 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:14:12.014 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:08:59.056012', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnRa7wX4JPdnmhx6mqM2K0CI7CJKm/oKVfTMnJlEeI9hGPzJCwffaZS0YEwdPeKlZ+ZxyiU5I3NCcetbZngXq0='} 2024-04-25T11:14:12.039 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:08:59.057672', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEf7cTDSBvuXWGk+YVTL9kClpiBtnl29eEgFtkLLFGlvZA5yYftciQnVOnYRweqQ6NQax72C3f7M/xO7E+9ntJk='} 2024-04-25T11:14:12.039 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T11:14:12.045 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T11:14:12.045 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T11:14:12.045 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T11:14:12.123 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5393c72f70>, signals=[15]) 2024-04-25T11:14:12.123 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T11:14:12.131 INFO:teuthology.task.internal:Opening connections... 2024-04-25T11:14:12.131 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:14:12.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:14:12.211 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:14:12.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:14:12.284 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T11:14:12.291 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-04-25T11:14:12.309 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-04-25T11:14:12.310 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-04-25T11:14:12.368 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-04-25T11:14:12.368 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-04-25T11:14:12.368 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-04-25T11:14:12.368 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:14:12.368 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-04-25T11:14:12.368 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:14:12.368 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:14:12.369 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-04-25T11:14:12.369 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:14:12.369 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:14:12.369 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:14:12.369 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:14:12.369 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:14:12.369 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:14:12.370 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-04-25T11:14:12.396 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-25T11:14:12.414 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-25T11:14:12.414 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:14:12.471 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:14:12.472 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:14:12.472 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:14:12.472 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:14:12.472 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-25T11:14:12.495 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T11:14:12.505 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T11:14:12.513 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T11:14:12.513 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:14:12.515 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:14:12.531 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T11:14:12.538 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T11:14:12.538 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:14:12.573 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:14:12.693 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T11:14:12.749 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T11:14:12.765 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T11:14:12.765 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T11:14:12.765 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T11:14:12.766 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:14:12.766 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:14:12.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:14:12.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T11:14:12.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:14:12.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T11:14:12.963 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T11:14:12.963 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T11:14:12.963 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-25T11:14:12.981 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:14:12.981 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T11:14:12.981 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T11:14:12.982 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum upgrade -y linux-firmware 2024-04-25T11:14:12.986 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T11:14:12.987 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T11:14:12.987 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-25T11:14:13.005 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:14:13.005 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T11:14:13.006 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T11:14:13.006 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum upgrade -y linux-firmware 2024-04-25T11:14:15.815 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-04-25T11:14:16.638 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-04-25T11:14:18.416 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-25T11:14:19.589 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-25T11:14:22.459 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T11:14:24.394 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-25T11:14:25.552 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T11:14:29.073 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 1.1 MB/s | 6.2 MB 00:05 2024-04-25T11:14:30.702 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T11:14:32.734 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T11:14:32.735 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-04-25T11:14:32.735 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T11:14:32.834 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T11:14:32.834 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T11:14:33.081 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T11:14:33.082 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T11:14:33.084 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2024-04-25T11:14:35.774 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 3.0 kB/s | 16 kB 00:05 2024-04-25T11:14:45.187 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25 11:14:45 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T11:14:45.188 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2024-04-25 11:14:45-- 2024-04-25T11:14:45.188 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 12s 2024-04-25T11:14:45.188 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-25T11:14:45.191 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T11:14:45.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T11:14:47.351 INFO:teuthology.orchestra.run.smithi169.stdout:Verifying... ######################################## 2024-04-25T11:14:47.486 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing... ######################################## 2024-04-25T11:14:47.598 INFO:teuthology.orchestra.run.smithi169.stdout:Updating / installing... 2024-04-25T11:14:55.569 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T11:14:56.034 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-04-25T11:15:01.315 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-25T11:15:03.498 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T11:15:03.499 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-04-25T11:15:03.499 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T11:15:03.608 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T11:15:03.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T11:15:03.757 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T11:15:03.757 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T11:15:03.760 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi019.front.sepia.ceph.com to complete... 2024-04-25T11:15:15.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25 11:15:15 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T11:15:15.876 INFO:teuthology.orchestra.run.smithi019.stderr:FINISHED --2024-04-25 11:15:15-- 2024-04-25T11:15:15.877 INFO:teuthology.orchestra.run.smithi019.stderr:Total wall clock time: 12s 2024-04-25T11:15:15.877 INFO:teuthology.orchestra.run.smithi019.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-25T11:15:15.880 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T11:15:15.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T11:15:18.129 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying... ######################################## 2024-04-25T11:15:18.244 INFO:teuthology.orchestra.run.smithi019.stdout:Preparing... ######################################## 2024-04-25T11:15:18.351 INFO:teuthology.orchestra.run.smithi019.stdout:Updating / installing... 2024-04-25T11:15:26.673 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T11:15:34.944 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T11:15:34.945 DEBUG:teuthology.orchestra.run.smithi169:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T11:15:35.021 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:35.021 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:35.021 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:35.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T11:15:35.021 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T11:15:35.022 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T11:15:35.023 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T11:15:35.024 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T11:15:35.025 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T11:15:35.026 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T11:15:35.027 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T11:15:35.028 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T11:15:35.029 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T11:15:35.030 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T11:15:35.031 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T11:15:35.032 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T11:15:35.033 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T11:15:35.034 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T11:15:35.035 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T11:15:35.036 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T11:15:35.037 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T11:15:35.038 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T11:15:35.039 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T11:15:35.040 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T11:15:35.041 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T11:15:35.042 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T11:15:35.043 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T11:15:35.044 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T11:15:35.045 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T11:15:35.046 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T11:15:35.047 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T11:15:35.048 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T11:15:35.049 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T11:15:35.050 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T11:15:35.051 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T11:15:35.052 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T11:15:35.053 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T11:15:35.054 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T11:15:35.055 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T11:15:35.056 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T11:15:35.057 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T11:15:35.058 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T11:15:35.059 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T11:15:35.060 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T11:15:35.061 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T11:15:35.062 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T11:15:35.063 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T11:15:35.064 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T11:15:35.065 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T11:15:35.066 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T11:15:35.067 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T11:15:35.068 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T11:15:35.069 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T11:15:35.070 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T11:15:35.071 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T11:15:35.072 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T11:15:35.073 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T11:15:35.074 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T11:15:35.075 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T11:15:35.076 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T11:15:35.077 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T11:15:35.078 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T11:15:35.079 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T11:15:35.080 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T11:15:35.081 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T11:15:35.082 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T11:15:35.083 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T11:15:35.084 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T11:15:35.085 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T11:15:35.086 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T11:15:35.087 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T11:15:35.088 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T11:15:35.089 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T11:15:35.090 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T11:15:35.091 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T11:15:35.092 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T11:15:35.093 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T11:15:35.094 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T11:15:35.095 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T11:15:35.096 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T11:15:35.097 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T11:15:35.098 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T11:15:35.099 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T11:15:35.100 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T11:15:35.101 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T11:15:35.102 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T11:15:35.103 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T11:15:35.104 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T11:15:35.105 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T11:15:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T11:15:35.107 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T11:15:35.108 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T11:15:35.109 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T11:15:35.110 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T11:15:35.111 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T11:15:35.112 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T11:15:35.113 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T11:15:35.114 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T11:15:35.115 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T11:15:35.116 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T11:15:35.117 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T11:15:35.118 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T11:15:35.119 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T11:15:35.120 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T11:15:35.121 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T11:15:35.122 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T11:15:35.123 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T11:15:35.124 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T11:15:35.125 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T11:15:35.126 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T11:15:35.127 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T11:15:35.128 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T11:15:35.129 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T11:15:35.130 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T11:15:35.131 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T11:15:35.132 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T11:15:35.133 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T11:15:35.134 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T11:15:35.135 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T11:15:35.136 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T11:15:35.137 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T11:15:35.138 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T11:15:35.139 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T11:15:35.140 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T11:15:35.141 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T11:15:35.142 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T11:15:35.143 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T11:15:35.144 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T11:15:35.145 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T11:15:35.146 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T11:15:35.147 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T11:15:35.148 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T11:15:35.149 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T11:15:35.150 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T11:15:35.151 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T11:15:35.152 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T11:15:35.154 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T11:15:35.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T11:15:35.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T11:15:35.159 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T11:15:35.163 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:35.163 DEBUG:teuthology.orchestra.run.smithi169:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T11:15:35.188 INFO:teuthology.orchestra.run.smithi169.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T11:15:35.188 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T11:15:35.188 INFO:teuthology.orchestra.run.smithi169.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T11:15:35.188 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-25T11:15:35.188 INFO:teuthology.orchestra.run.smithi169.stdout:for file in vmlinuz System.map config; do 2024-04-25T11:15:35.188 INFO:teuthology.orchestra.run.smithi169.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T11:15:35.188 INFO:teuthology.orchestra.run.smithi169.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout: fi 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:done 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:35.189 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-04-25T11:15:35.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2024-04-25T11:15:35.267 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.06 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 68.el9 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Group : Unspecified 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 8264254 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T11:15:35.268 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : builder@centos.org 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:15:35.269 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T11:15:35.270 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T11:15:35.270 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:35.270 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T11:15:35.799 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-04-25T11:15:37.235 INFO:teuthology.orchestra.run.smithi169.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T11:15:37.256 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-04-25T11:15:37.258 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-25T11:15:37.275 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.7kOoBG247s 2024-04-25T11:15:37.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7kOoBG247s 2024-04-25T11:15:37.342 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.7kOoBG247s 2024-04-25T11:15:37.474 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.7kOoBG247s is 6KB 2024-04-25T11:15:37.495 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.7kOoBG247s 2024-04-25T11:15:37.509 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T11:15:37.573 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T11:15:37.573 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T11:15:37.573 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T11:15:37.573 INFO:teuthology.orchestra.run.smithi169.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T11:15:37.574 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T11:15:37.727 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2024-04-25T11:16:06.684 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T11:16:06.684 DEBUG:teuthology.orchestra.run.smithi019:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T11:16:06.770 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T11:16:06.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T11:16:06.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T11:16:06.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T11:16:06.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T11:16:06.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T11:16:06.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T11:16:06.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T11:16:06.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T11:16:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T11:16:06.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T11:16:06.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T11:16:06.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T11:16:06.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T11:16:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T11:16:06.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T11:16:06.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T11:16:06.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T11:16:06.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T11:16:06.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T11:16:06.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T11:16:06.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T11:16:06.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T11:16:06.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T11:16:06.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T11:16:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T11:16:06.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T11:16:06.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T11:16:06.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T11:16:06.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T11:16:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T11:16:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T11:16:06.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T11:16:06.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T11:16:06.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T11:16:06.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T11:16:06.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T11:16:06.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T11:16:06.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T11:16:06.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T11:16:06.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T11:16:06.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T11:16:06.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T11:16:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T11:16:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T11:16:06.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T11:16:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T11:16:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T11:16:06.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T11:16:06.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T11:16:06.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T11:16:06.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T11:16:06.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T11:16:06.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T11:16:06.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T11:16:06.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T11:16:06.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T11:16:06.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T11:16:06.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T11:16:06.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T11:16:06.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T11:16:06.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T11:16:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T11:16:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T11:16:06.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T11:16:06.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T11:16:06.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T11:16:06.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T11:16:06.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T11:16:06.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T11:16:06.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T11:16:06.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T11:16:06.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T11:16:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T11:16:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T11:16:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T11:16:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T11:16:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T11:16:06.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T11:16:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T11:16:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T11:16:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T11:16:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T11:16:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T11:16:06.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T11:16:06.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T11:16:06.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T11:16:06.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T11:16:06.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T11:16:06.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T11:16:06.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T11:16:06.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T11:16:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T11:16:06.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T11:16:06.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T11:16:06.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T11:16:06.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T11:16:06.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T11:16:06.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T11:16:06.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T11:16:06.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T11:16:06.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T11:16:06.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T11:16:06.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T11:16:06.868 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T11:16:06.869 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T11:16:06.870 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T11:16:06.870 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T11:16:06.870 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T11:16:06.870 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T11:16:06.870 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T11:16:06.870 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T11:16:06.871 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T11:16:06.872 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T11:16:06.873 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T11:16:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T11:16:06.875 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T11:16:06.876 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T11:16:06.877 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T11:16:06.878 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T11:16:06.879 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T11:16:06.880 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T11:16:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T11:16:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T11:16:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T11:16:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T11:16:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T11:16:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T11:16:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T11:16:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T11:16:06.883 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T11:16:06.884 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T11:16:06.885 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T11:16:06.886 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T11:16:06.887 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T11:16:06.888 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T11:16:06.889 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T11:16:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T11:16:06.891 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T11:16:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T11:16:06.893 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T11:16:06.894 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T11:16:06.895 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T11:16:06.896 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T11:16:06.897 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T11:16:06.898 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T11:16:06.899 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T11:16:06.900 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T11:16:06.901 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T11:16:06.902 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T11:16:06.903 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T11:16:06.904 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T11:16:06.905 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T11:16:06.906 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T11:16:06.907 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T11:16:06.908 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T11:16:06.909 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T11:16:06.910 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T11:16:06.911 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T11:16:06.912 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T11:16:06.913 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T11:16:06.914 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T11:16:06.914 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T11:16:06.914 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T11:16:06.914 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T11:16:06.914 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T11:16:06.914 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T11:16:06.915 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T11:16:06.916 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T11:16:06.917 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T11:16:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T11:16:06.919 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T11:16:06.920 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T11:16:06.920 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T11:16:06.920 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T11:16:06.920 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T11:16:06.920 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T11:16:06.920 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T11:16:06.920 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T11:16:06.924 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:06.924 DEBUG:teuthology.orchestra.run.smithi019:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T11:16:06.950 INFO:teuthology.orchestra.run.smithi019.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T11:16:06.950 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T11:16:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T11:16:06.950 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:for file in vmlinuz System.map config; do 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout: fi 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:done 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T11:16:06.951 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T11:16:06.952 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:06.952 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-04-25T11:16:06.953 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-04-25T11:16:07.038 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-04-25T11:16:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:16:07.040 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T11:16:07.040 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T11:16:07.041 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:07.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T11:16:07.730 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T11:16:07.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:16:07.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:16:07.735 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-04-25T11:16:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T11:16:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-04-25T11:16:09.166 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-04-25T11:16:09.199 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.dvpvFQUaP5 2024-04-25T11:16:09.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dvpvFQUaP5 2024-04-25T11:16:09.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.dvpvFQUaP5 2024-04-25T11:16:09.415 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.dvpvFQUaP5 is 6KB 2024-04-25T11:16:09.463 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.dvpvFQUaP5 2024-04-25T11:16:09.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T11:16:09.549 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T11:16:09.549 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T11:16:09.549 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T11:16:09.549 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T11:16:09.551 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T11:16:09.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-04-25T11:16:26.275 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T11:16:35.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:16:35.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:16:38.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T11:16:39.730 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T11:16:39.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:16:39.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:16:50.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:16:50.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:16:58.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-04-25T11:17:07.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:17:07.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:17:08.769 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T11:17:23.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:17:23.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:17:26.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-25T11:17:44.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:17:44.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:17:45.416 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T11:17:46.008 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:17:46.008 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T11:17:46.009 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-04-25T11:17:46.049 INFO:teuthology.orchestra.run.smithi169.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T11:17:46.049 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T11:17:46.049 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T11:17:46.049 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T11:17:47.053 DEBUG:teuthology.parallel:result is None 2024-04-25T11:18:07.282 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T11:18:19.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:18:19.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:18:19.618 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-25T11:18:20.271 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:18:20.272 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T11:18:20.272 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-25T11:18:20.326 INFO:teuthology.orchestra.run.smithi019.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T11:18:20.327 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T11:18:20.327 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T11:18:20.327 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T11:18:21.327 DEBUG:teuthology.parallel:result is None 2024-04-25T11:18:21.328 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T11:18:21.339 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T11:18:21.339 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:18:21.347 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:18:21.404 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T11:18:21.414 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T11:18:21.423 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T11:18:21.424 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:18:21.431 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:18:21.508 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T11:18:21.518 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T11:18:21.519 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:18:21.527 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:18:21.644 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:18:21.703 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:18:21.713 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:18:21.753 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:18:21.757 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T11:18:21.765 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T11:18:21.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:18:21.772 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:18:21.860 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T11:18:21.870 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T11:18:21.870 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:18:21.875 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:18:21.930 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:18:22.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:18:22.110 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:18:22.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:18:22.282 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:18:22.282 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:18:22.369 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:18:22.459 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:18:22.554 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:18:22.640 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:18:22.725 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:18:22.725 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:18:22.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-04-25T11:18:22.816 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-25T11:18:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:18:22.968 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:18:23.453 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T11:18:23.463 INFO:teuthology.task.internal:Starting timer... 2024-04-25T11:18:23.463 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T11:18:23.491 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T11:18:23.500 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T11:18:23.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-04-25T11:18:23.614 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-04-25T11:18:23.618 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-25T11:18:23.719 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-25T11:18:23.723 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T11:18:23.723 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-04-25T11:18:23.791 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-04-25T11:18:23.792 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-25T11:18:23.849 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-25T11:18:23.849 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-04-25T11:18:23.850 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:18:23.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:23.935 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:18:24.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:18:24.011 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T11:18:24.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:18:24.085 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:18:24.171 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T11:18:24.180 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T11:18:24.185 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T11:18:24.304 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T11:18:24.321 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T11:18:24.322 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T11:23:38.900 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-25T11:23:38.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:23:38.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:23:39.010 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-25T11:23:39.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:23:39.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:23:39.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:23:39.276 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T11:23:39.436 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:23:39.436 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T11:23:39.450 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T11:23:39.450 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:23:39.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:23:39.458 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:23:39.458 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:23:39.548 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:23:39.550 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:23:39.587 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:23:39.593 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:23:39.659 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-25T11:23:39.671 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-04-25T11:23:39.690 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-25T11:23:39.707 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-04-25T11:23:39.729 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:23:39.770 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:23:39.793 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:23:39.915 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:23:39.923 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:23:39.923 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T11:23:39.924 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.924 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.924 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.924 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:23:39.953 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:23:39.953 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T11:23:39.953 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.954 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.954 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.954 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:23:39.955 INFO:teuthology.run_tasks:Running task install... 2024-04-25T11:23:39.963 DEBUG:teuthology.task.install:project ceph 2024-04-25T11:23:39.963 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T11:23:39.963 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T11:23:39.963 INFO:teuthology.task.install:Using flavor: default 2024-04-25T11:23:39.969 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:23:39.969 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T11:23:39.969 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T11:23:39.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:23:39.972 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T11:23:39.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:23:40.118 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T11:23:40.118 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T11:23:40.200 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T11:23:40.200 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T11:23:40.301 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:23:40.301 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:23:40.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:23:40.393 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:23:40.393 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:23:40.398 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:23:40.398 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:23:40.398 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:23:40.486 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:23:40.486 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:23:40.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:23:40.615 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:23:40.641 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:23:40.723 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:23:40.727 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-04-25T11:23:40.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-04-25T11:23:40.802 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-25T11:23:40.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-25T11:23:41.072 INFO:teuthology.orchestra.run.smithi019.stdout:48 files removed 2024-04-25T11:23:41.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:23:41.124 INFO:teuthology.orchestra.run.smithi169.stdout:48 files removed 2024-04-25T11:23:41.163 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:23:42.427 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 101 kB/s | 84 kB 00:00 2024-04-25T11:23:42.552 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 84 kB/s | 84 kB 00:00 2024-04-25T11:23:43.169 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-04-25T11:23:43.252 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-04-25T11:23:43.745 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-25T11:23:43.928 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-04-25T11:23:44.735 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-04-25T11:23:47.109 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-04-25T11:23:48.469 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-04-25T11:23:51.518 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-04-25T11:23:53.211 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-04-25T11:23:55.229 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-25T11:23:56.619 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.2 MB 00:01 2024-04-25T11:23:58.598 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-25T11:23:59.921 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T11:24:06.179 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T11:24:08.706 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:24:08.708 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:24:08.730 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:24:08.731 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:24:08.732 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:24:08.734 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:24:08.736 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:24:08.740 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:24:08.741 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:24:08.818 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T11:24:08.836 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T11:24:08.837 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:24:08.838 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T11:24:08.839 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T11:24:08.840 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:24:08.841 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:24:08.842 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:24:08.843 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:24:08.844 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:24:08.845 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:24:08.846 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:24:08.847 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 4 Packages 2024-04-25T11:24:08.848 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:24:08.849 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 214 M 2024-04-25T11:24:08.849 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T11:24:09.981 INFO:teuthology.orchestra.run.smithi019.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-25T11:24:10.889 INFO:teuthology.orchestra.run.smithi019.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 951 kB/s | 863 kB 00:00 2024-04-25T11:24:11.123 INFO:teuthology.orchestra.run.smithi019.stdout:(3/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-04-25T11:24:11.474 INFO:teuthology.orchestra.run.smithi019.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 6.5 MB/s | 2.3 MB 00:00 2024-04-25T11:24:11.498 INFO:teuthology.orchestra.run.smithi019.stdout:(5/152): ceph-immutable-object-cache-19.0.0-329 242 kB/s | 147 kB 00:00 2024-04-25T11:24:11.649 INFO:teuthology.orchestra.run.smithi019.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-04-25T11:24:13.108 INFO:teuthology.orchestra.run.smithi019.stdout:(7/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x86 12 MB/s | 18 MB 00:01 2024-04-25T11:24:13.916 INFO:teuthology.orchestra.run.smithi019.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 13 MB/s | 10 MB 00:00 2024-04-25T11:24:13.983 INFO:teuthology.orchestra.run.smithi019.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 384 kB/s | 25 kB 00:00 2024-04-25T11:24:15.217 INFO:teuthology.orchestra.run.smithi019.stdout:(10/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 4.9 MB 00:03 2024-04-25T11:24:15.458 INFO:teuthology.orchestra.run.smithi019.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 932 kB/s | 224 kB 00:00 2024-04-25T11:24:15.559 INFO:teuthology.orchestra.run.smithi019.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 273 kB/s | 27 kB 00:00 2024-04-25T11:24:15.651 INFO:teuthology.orchestra.run.smithi019.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 350 kB/s | 32 kB 00:00 2024-04-25T11:24:16.066 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-04-25T11:24:16.209 INFO:teuthology.orchestra.run.smithi019.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 1.3 MB/s | 756 kB 00:00 2024-04-25T11:24:16.384 INFO:teuthology.orchestra.run.smithi019.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 908 kB/s | 158 kB 00:00 2024-04-25T11:24:16.580 INFO:teuthology.orchestra.run.smithi019.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 652 kB/s | 127 kB 00:00 2024-04-25T11:24:16.977 INFO:teuthology.orchestra.run.smithi019.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 1.2 MB/s | 497 kB 00:00 2024-04-25T11:24:17.485 INFO:teuthology.orchestra.run.smithi019.stdout:(18/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 12 MB/s | 43 MB 00:03 2024-04-25T11:24:17.527 INFO:teuthology.orchestra.run.smithi019.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 1.1 MB/s | 45 kB 00:00 2024-04-25T11:24:17.596 INFO:teuthology.orchestra.run.smithi019.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.0 MB/s | 135 kB 00:00 2024-04-25T11:24:17.661 INFO:teuthology.orchestra.run.smithi019.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.4 MB/s | 161 kB 00:00 2024-04-25T11:24:17.728 INFO:teuthology.orchestra.run.smithi019.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 4.8 MB/s | 321 kB 00:00 2024-04-25T11:24:17.797 INFO:teuthology.orchestra.run.smithi019.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 4.3 MB/s | 298 kB 00:00 2024-04-25T11:24:17.856 INFO:teuthology.orchestra.run.smithi019.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 100 kB 00:00 2024-04-25T11:24:18.115 INFO:teuthology.orchestra.run.smithi019.stdout:(25/152): ceph-common-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 20 MB 00:08 2024-04-25T11:24:18.148 INFO:teuthology.orchestra.run.smithi019.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 296 kB/s | 86 kB 00:00 2024-04-25T11:24:18.199 INFO:teuthology.orchestra.run.smithi019.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.4 MB/s | 169 kB 00:00 2024-04-25T11:24:18.249 INFO:teuthology.orchestra.run.smithi019.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 697 kB/s | 34 kB 00:00 2024-04-25T11:24:18.307 INFO:teuthology.orchestra.run.smithi019.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.4 MB/s | 142 kB 00:00 2024-04-25T11:24:18.691 INFO:teuthology.orchestra.run.smithi019.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 9.3 MB/s | 3.5 MB 00:00 2024-04-25T11:24:19.200 INFO:teuthology.orchestra.run.smithi019.stdout:(31/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 2.8 MB/s | 3.0 MB 00:01 2024-04-25T11:24:19.350 INFO:teuthology.orchestra.run.smithi019.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-25T11:24:19.383 INFO:teuthology.orchestra.run.smithi019.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.3 MB/s | 249 kB 00:00 2024-04-25T11:24:19.409 INFO:teuthology.orchestra.run.smithi019.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 860 kB/s | 49 kB 00:00 2024-04-25T11:24:19.459 INFO:teuthology.orchestra.run.smithi019.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 221 kB/s | 16 kB 00:00 2024-04-25T11:24:19.609 INFO:teuthology.orchestra.run.smithi019.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 273 kB 00:00 2024-04-25T11:24:19.718 INFO:teuthology.orchestra.run.smithi019.stdout:(37/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 1.9 MB/s | 5.2 MB 00:02 2024-04-25T11:24:19.743 INFO:teuthology.orchestra.run.smithi019.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 137 kB/s | 18 kB 00:00 2024-04-25T11:24:19.785 INFO:teuthology.orchestra.run.smithi019.stdout:(39/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.3 MB/s | 762 kB 00:00 2024-04-25T11:24:19.908 INFO:teuthology.orchestra.run.smithi019.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 426 kB/s | 80 kB 00:00 2024-04-25T11:24:19.994 INFO:teuthology.orchestra.run.smithi019.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 941 kB/s | 195 kB 00:00 2024-04-25T11:24:20.053 INFO:teuthology.orchestra.run.smithi019.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-25T11:24:20.094 INFO:teuthology.orchestra.run.smithi019.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-25T11:24:20.120 INFO:teuthology.orchestra.run.smithi019.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T11:24:20.153 INFO:teuthology.orchestra.run.smithi019.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T11:24:20.187 INFO:teuthology.orchestra.run.smithi019.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T11:24:20.229 INFO:teuthology.orchestra.run.smithi019.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T11:24:20.454 INFO:teuthology.orchestra.run.smithi019.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-04-25T11:24:20.496 INFO:teuthology.orchestra.run.smithi019.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-25T11:24:20.521 INFO:teuthology.orchestra.run.smithi019.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T11:24:20.538 INFO:teuthology.orchestra.run.smithi019.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 525 kB/s | 21 kB 00:00 2024-04-25T11:24:20.589 INFO:teuthology.orchestra.run.smithi019.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-25T11:24:20.614 INFO:teuthology.orchestra.run.smithi019.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 437 kB/s | 33 kB 00:00 2024-04-25T11:24:20.648 INFO:teuthology.orchestra.run.smithi019.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-25T11:24:20.673 INFO:teuthology.orchestra.run.smithi019.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-04-25T11:24:20.698 INFO:teuthology.orchestra.run.smithi019.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 226 kB/s | 11 kB 00:00 2024-04-25T11:24:20.732 INFO:teuthology.orchestra.run.smithi019.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T11:24:20.757 INFO:teuthology.orchestra.run.smithi019.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2024-04-25T11:24:20.891 INFO:teuthology.orchestra.run.smithi019.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-25T11:24:20.924 INFO:teuthology.orchestra.run.smithi019.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-25T11:24:20.949 INFO:teuthology.orchestra.run.smithi019.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 441 kB/s | 25 kB 00:00 2024-04-25T11:24:20.975 INFO:teuthology.orchestra.run.smithi019.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 962 kB/s | 47 kB 00:00 2024-04-25T11:24:21.050 INFO:teuthology.orchestra.run.smithi019.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.5 MB/s | 3.0 MB 00:00 2024-04-25T11:24:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-25T11:24:21.109 INFO:teuthology.orchestra.run.smithi019.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 35 kB 00:00 2024-04-25T11:24:21.134 INFO:teuthology.orchestra.run.smithi019.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-25T11:24:21.359 INFO:teuthology.orchestra.run.smithi019.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-04-25T11:24:21.468 INFO:teuthology.orchestra.run.smithi019.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T11:24:21.535 INFO:teuthology.orchestra.run.smithi019.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-04-25T11:24:21.560 INFO:teuthology.orchestra.run.smithi019.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T11:24:21.732 INFO:teuthology.orchestra.run.smithi019.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.3 MB/s | 4.7 MB 00:00 2024-04-25T11:24:21.794 INFO:teuthology.orchestra.run.smithi019.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-04-25T11:24:21.844 INFO:teuthology.orchestra.run.smithi019.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 784 kB/s | 87 kB 00:00 2024-04-25T11:24:21.911 INFO:teuthology.orchestra.run.smithi019.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T11:24:21.935 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-25T11:24:21.970 INFO:teuthology.orchestra.run.smithi019.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-25T11:24:22.020 INFO:teuthology.orchestra.run.smithi019.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 524 kB/s | 26 kB 00:00 2024-04-25T11:24:22.145 INFO:teuthology.orchestra.run.smithi019.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 9.5 MB/s | 5.8 MB 00:00 2024-04-25T11:24:22.171 INFO:teuthology.orchestra.run.smithi019.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 128 kB/s | 19 kB 00:00 2024-04-25T11:24:22.213 INFO:teuthology.orchestra.run.smithi019.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-25T11:24:22.238 INFO:teuthology.orchestra.run.smithi019.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 527 kB/s | 35 kB 00:00 2024-04-25T11:24:22.297 INFO:teuthology.orchestra.run.smithi019.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-25T11:24:22.431 INFO:teuthology.orchestra.run.smithi019.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-04-25T11:24:22.464 INFO:teuthology.orchestra.run.smithi019.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-04-25T11:24:22.491 INFO:teuthology.orchestra.run.smithi019.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T11:24:22.506 INFO:teuthology.orchestra.run.smithi019.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T11:24:22.556 INFO:teuthology.orchestra.run.smithi019.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-25T11:24:22.590 INFO:teuthology.orchestra.run.smithi019.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-25T11:24:22.624 INFO:teuthology.orchestra.run.smithi019.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T11:24:22.649 INFO:teuthology.orchestra.run.smithi019.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 884 kB/s | 51 kB 00:00 2024-04-25T11:24:22.674 INFO:teuthology.orchestra.run.smithi019.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T11:24:22.733 INFO:teuthology.orchestra.run.smithi019.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 725 kB/s | 42 kB 00:00 2024-04-25T11:24:22.851 INFO:teuthology.orchestra.run.smithi019.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.0 MB/s | 5.1 MB 00:00 2024-04-25T11:24:22.876 INFO:teuthology.orchestra.run.smithi019.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 312 kB/s | 44 kB 00:00 2024-04-25T11:24:22.909 INFO:teuthology.orchestra.run.smithi019.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T11:24:22.934 INFO:teuthology.orchestra.run.smithi019.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 623 kB/s | 36 kB 00:00 2024-04-25T11:24:22.960 INFO:teuthology.orchestra.run.smithi019.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-04-25T11:24:22.993 INFO:teuthology.orchestra.run.smithi019.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-25T11:24:23.019 INFO:teuthology.orchestra.run.smithi019.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-25T11:24:23.044 INFO:teuthology.orchestra.run.smithi019.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-25T11:24:23.069 INFO:teuthology.orchestra.run.smithi019.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-25T11:24:23.094 INFO:teuthology.orchestra.run.smithi019.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 910 kB/s | 45 kB 00:00 2024-04-25T11:24:23.220 INFO:teuthology.orchestra.run.smithi019.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-04-25T11:24:23.470 INFO:teuthology.orchestra.run.smithi019.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-04-25T11:24:23.750 INFO:teuthology.orchestra.run.smithi019.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-25T11:24:23.796 INFO:teuthology.orchestra.run.smithi019.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 955 kB/s | 548 kB 00:00 2024-04-25T11:24:23.862 INFO:teuthology.orchestra.run.smithi019.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 790 kB/s | 309 kB 00:00 2024-04-25T11:24:23.887 INFO:teuthology.orchestra.run.smithi019.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 147 kB/s | 19 kB 00:00 2024-04-25T11:24:24.377 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:24:24.379 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:24:24.398 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:24:24.399 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:24:24.401 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:24:24.402 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:24:24.404 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:24:24.408 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:24:24.409 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:24:24.479 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T11:24:24.496 INFO:teuthology.orchestra.run.smithi019.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-04-25T11:24:24.497 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T11:24:24.498 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T11:24:24.499 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T11:24:24.500 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T11:24:24.501 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:24:24.502 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:24:24.503 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:24:24.505 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:24:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:24:24.507 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:24:24.508 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-25T11:24:24.510 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T11:24:24.510 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-04-25T11:24:24.510 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 4 Packages 2024-04-25T11:24:24.510 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:24:24.510 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 214 M 2024-04-25T11:24:24.510 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-25T11:24:24.521 INFO:teuthology.orchestra.run.smithi019.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 41 kB/s | 26 kB 00:00 2024-04-25T11:24:24.547 INFO:teuthology.orchestra.run.smithi019.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 74 kB/s | 49 kB 00:00 2024-04-25T11:24:24.572 INFO:teuthology.orchestra.run.smithi019.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T11:24:24.597 INFO:teuthology.orchestra.run.smithi019.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T11:24:24.712 INFO:teuthology.orchestra.run.smithi019.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.1 MB/s | 838 kB 00:00 2024-04-25T11:24:24.765 INFO:teuthology.orchestra.run.smithi019.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-04-25T11:24:24.789 INFO:teuthology.orchestra.run.smithi019.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 231 kB/s | 43 kB 00:00 2024-04-25T11:24:24.815 INFO:teuthology.orchestra.run.smithi019.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 336 kB/s | 32 kB 00:00 2024-04-25T11:24:24.840 INFO:teuthology.orchestra.run.smithi019.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 308 kB/s | 15 kB 00:00 2024-04-25T11:24:24.865 INFO:teuthology.orchestra.run.smithi019.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-04-25T11:24:24.899 INFO:teuthology.orchestra.run.smithi019.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.4 MB/s | 358 kB 00:00 2024-04-25T11:24:24.932 INFO:teuthology.orchestra.run.smithi019.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.8 MB/s | 303 kB 00:00 2024-04-25T11:24:24.958 INFO:teuthology.orchestra.run.smithi019.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 404 kB/s | 35 kB 00:00 2024-04-25T11:24:24.991 INFO:teuthology.orchestra.run.smithi019.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.3 MB/s | 206 kB 00:00 2024-04-25T11:24:25.175 INFO:teuthology.orchestra.run.smithi019.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.5 MB/s | 2.0 MB 00:00 2024-04-25T11:24:25.200 INFO:teuthology.orchestra.run.smithi019.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 605 kB/s | 144 kB 00:00 2024-04-25T11:24:25.217 INFO:teuthology.orchestra.run.smithi019.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 48 kB/s | 11 kB 00:00 2024-04-25T11:24:25.242 INFO:teuthology.orchestra.run.smithi019.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 273 kB/s | 18 kB 00:00 2024-04-25T11:24:25.267 INFO:teuthology.orchestra.run.smithi019.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 474 kB/s | 23 kB 00:00 2024-04-25T11:24:25.294 INFO:teuthology.orchestra.run.smithi019.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 264 kB/s | 19 kB 00:00 2024-04-25T11:24:25.318 INFO:teuthology.orchestra.run.smithi019.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 265 kB/s | 20 kB 00:00 2024-04-25T11:24:25.410 INFO:teuthology.orchestra.run.smithi019.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-04-25T11:24:25.427 INFO:teuthology.orchestra.run.smithi019.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 350 kB/s | 46 kB 00:00 2024-04-25T11:24:25.452 INFO:teuthology.orchestra.run.smithi019.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 617 kB/s | 79 kB 00:00 2024-04-25T11:24:25.469 INFO:teuthology.orchestra.run.smithi019.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-04-25T11:24:25.519 INFO:teuthology.orchestra.run.smithi019.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.1 MB/s | 272 kB 00:00 2024-04-25T11:24:25.536 INFO:teuthology.orchestra.run.smithi019.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 201 kB/s | 16 kB 00:00 2024-04-25T11:24:25.561 INFO:teuthology.orchestra.run.smithi019.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 983 kB/s | 90 kB 00:00 2024-04-25T11:24:25.578 INFO:teuthology.orchestra.run.smithi019.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 708 kB/s | 28 kB 00:00 2024-04-25T11:24:25.595 INFO:teuthology.orchestra.run.smithi019.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 545 kB/s | 31 kB 00:00 2024-04-25T11:24:25.629 INFO:teuthology.orchestra.run.smithi019.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-25T11:24:25.679 INFO:teuthology.orchestra.run.smithi019.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 733 kB/s | 59 kB 00:00 2024-04-25T11:24:25.704 INFO:teuthology.orchestra.run.smithi019.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 339 kB/s | 36 kB 00:00 2024-04-25T11:24:25.729 INFO:teuthology.orchestra.run.smithi169.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-25T11:24:25.729 INFO:teuthology.orchestra.run.smithi019.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 518 kB/s | 51 kB 00:00 2024-04-25T11:24:25.763 INFO:teuthology.orchestra.run.smithi019.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-25T11:24:25.788 INFO:teuthology.orchestra.run.smithi019.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-25T11:24:25.830 INFO:teuthology.orchestra.run.smithi019.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.3 MB/s | 427 kB 00:00 2024-04-25T11:24:25.856 INFO:teuthology.orchestra.run.smithi019.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 322 kB/s | 20 kB 00:00 2024-04-25T11:24:25.881 INFO:teuthology.orchestra.run.smithi019.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T11:24:25.981 INFO:teuthology.orchestra.run.smithi019.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T11:24:26.215 INFO:teuthology.orchestra.run.smithi019.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-04-25T11:24:26.365 INFO:teuthology.orchestra.run.smithi019.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T11:24:26.507 INFO:teuthology.orchestra.run.smithi019.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-04-25T11:24:26.587 INFO:teuthology.orchestra.run.smithi169.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 863 kB 00:00 2024-04-25T11:24:26.696 INFO:teuthology.orchestra.run.smithi169.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.3 MB/s | 147 kB 00:00 2024-04-25T11:24:27.524 INFO:teuthology.orchestra.run.smithi169.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.7 MB/s | 2.3 MB 00:00 2024-04-25T11:24:27.607 INFO:teuthology.orchestra.run.smithi019.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 1.9 MB/s | 3.3 MB 00:01 2024-04-25T11:24:27.615 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:24:27.616 INFO:teuthology.orchestra.run.smithi019.stdout:Total 11 MB/s | 214 MB 00:18 2024-04-25T11:24:28.971 INFO:teuthology.orchestra.run.smithi169.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-25T11:24:30.038 INFO:teuthology.orchestra.run.smithi169.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 4.6 MB/s | 4.9 MB 00:01 2024-04-25T11:24:30.390 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T11:24:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T11:24:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T11:24:31.955 INFO:teuthology.orchestra.run.smithi169.stdout:(7/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 847 kB/s | 5.4 MB 00:06 2024-04-25T11:24:33.213 INFO:teuthology.orchestra.run.smithi169.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 2.5 MB/s | 20 MB 00:07 2024-04-25T11:24:33.304 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T11:24:33.304 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T11:24:35.263 INFO:teuthology.orchestra.run.smithi169.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 12 kB/s | 25 kB 00:02 2024-04-25T11:24:35.505 INFO:teuthology.orchestra.run.smithi169.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 3.2 MB/s | 18 MB 00:05 2024-04-25T11:24:35.622 INFO:teuthology.orchestra.run.smithi169.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.9 MB/s | 224 kB 00:00 2024-04-25T11:24:35.672 INFO:teuthology.orchestra.run.smithi169.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 550 kB/s | 27 kB 00:00 2024-04-25T11:24:35.730 INFO:teuthology.orchestra.run.smithi169.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 551 kB/s | 32 kB 00:00 2024-04-25T11:24:35.922 INFO:teuthology.orchestra.run.smithi169.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.9 MB/s | 756 kB 00:00 2024-04-25T11:24:36.014 INFO:teuthology.orchestra.run.smithi169.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T11:24:36.239 INFO:teuthology.orchestra.run.smithi169.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 565 kB/s | 127 kB 00:00 2024-04-25T11:24:36.406 INFO:teuthology.orchestra.run.smithi169.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.9 MB/s | 497 kB 00:00 2024-04-25T11:24:37.948 INFO:teuthology.orchestra.run.smithi169.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 5.2 MB 00:01 2024-04-25T11:24:38.032 INFO:teuthology.orchestra.run.smithi169.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 550 kB/s | 45 kB 00:00 2024-04-25T11:24:38.124 INFO:teuthology.orchestra.run.smithi169.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.5 MB/s | 135 kB 00:00 2024-04-25T11:24:38.224 INFO:teuthology.orchestra.run.smithi169.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.6 MB/s | 161 kB 00:00 2024-04-25T11:24:38.399 INFO:teuthology.orchestra.run.smithi169.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 1.8 MB/s | 321 kB 00:00 2024-04-25T11:24:38.716 INFO:teuthology.orchestra.run.smithi169.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 943 kB/s | 298 kB 00:00 2024-04-25T11:24:38.767 INFO:teuthology.orchestra.run.smithi169.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 2.0 MB/s | 100 kB 00:00 2024-04-25T11:24:38.925 INFO:teuthology.orchestra.run.smithi169.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 547 kB/s | 86 kB 00:00 2024-04-25T11:24:39.376 INFO:teuthology.orchestra.run.smithi169.stdout:(26/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 11 MB/s | 43 MB 00:04 2024-04-25T11:24:39.459 INFO:teuthology.orchestra.run.smithi169.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.0 MB/s | 169 kB 00:00 2024-04-25T11:24:39.535 INFO:teuthology.orchestra.run.smithi169.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 461 kB/s | 34 kB 00:00 2024-04-25T11:24:39.602 INFO:teuthology.orchestra.run.smithi169.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.1 MB/s | 142 kB 00:00 2024-04-25T11:24:39.664 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T11:24:39.764 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:24:39.887 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:24:39.918 INFO:teuthology.orchestra.run.smithi169.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.1 MB/s | 3.0 MB 00:00 2024-04-25T11:24:40.010 INFO:teuthology.orchestra.run.smithi169.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 8.7 MB/s | 3.5 MB 00:00 2024-04-25T11:24:40.094 INFO:teuthology.orchestra.run.smithi169.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.9 MB/s | 249 kB 00:00 2024-04-25T11:24:40.169 INFO:teuthology.orchestra.run.smithi169.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 665 kB/s | 49 kB 00:00 2024-04-25T11:24:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:24:40.236 INFO:teuthology.orchestra.run.smithi169.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 250 kB/s | 16 kB 00:00 2024-04-25T11:24:40.274 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:24:40.308 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:24:40.320 INFO:teuthology.orchestra.run.smithi169.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 273 kB 00:00 2024-04-25T11:24:40.428 INFO:teuthology.orchestra.run.smithi169.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 7.0 MB/s | 762 kB 00:00 2024-04-25T11:24:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:24:40.495 INFO:teuthology.orchestra.run.smithi169.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 276 kB/s | 18 kB 00:00 2024-04-25T11:24:40.621 INFO:teuthology.orchestra.run.smithi169.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 651 kB/s | 80 kB 00:00 2024-04-25T11:24:40.763 INFO:teuthology.orchestra.run.smithi169.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.6 MB/s | 803 kB 00:00 2024-04-25T11:24:40.822 INFO:teuthology.orchestra.run.smithi169.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T11:24:40.880 INFO:teuthology.orchestra.run.smithi169.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-25T11:24:40.888 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:24:41.013 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:24:41.122 INFO:teuthology.orchestra.run.smithi169.stdout:(42/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 1.1 MB/s | 10 MB 00:09 2024-04-25T11:24:41.180 INFO:teuthology.orchestra.run.smithi169.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-04-25T11:24:41.217 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:24:41.231 INFO:teuthology.orchestra.run.smithi169.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T11:24:41.248 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:24:41.281 INFO:teuthology.orchestra.run.smithi169.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T11:24:41.384 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:24:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:(46/152): ceph-mgr-diskprediction-local-19.0.0- 5.0 MB/s | 7.4 MB 00:01 2024-04-25T11:24:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:(47/152): python3-ply-3.11-14.el9.noarch.rpm 345 kB/s | 106 kB 00:00 2024-04-25T11:24:41.439 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:24:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-04-25T11:24:41.516 INFO:teuthology.orchestra.run.smithi169.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-25T11:24:41.566 INFO:teuthology.orchestra.run.smithi169.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 435 kB/s | 21 kB 00:00 2024-04-25T11:24:41.608 INFO:teuthology.orchestra.run.smithi169.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-04-25T11:24:41.642 INFO:teuthology.orchestra.run.smithi169.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-25T11:24:41.667 INFO:teuthology.orchestra.run.smithi169.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 566 kB/s | 33 kB 00:00 2024-04-25T11:24:41.700 INFO:teuthology.orchestra.run.smithi169.stdout:(54/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-25T11:24:41.726 INFO:teuthology.orchestra.run.smithi169.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 314 kB/s | 18 kB 00:00 2024-04-25T11:24:41.760 INFO:teuthology.orchestra.run.smithi169.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-25T11:24:41.785 INFO:teuthology.orchestra.run.smithi169.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T11:24:41.810 INFO:teuthology.orchestra.run.smithi169.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 914 kB/s | 45 kB 00:00 2024-04-25T11:24:41.860 INFO:teuthology.orchestra.run.smithi169.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-04-25T11:24:41.896 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:24:41.910 INFO:teuthology.orchestra.run.smithi169.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-25T11:24:41.927 INFO:teuthology.orchestra.run.smithi169.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-04-25T11:24:41.953 INFO:teuthology.orchestra.run.smithi169.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T11:24:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:24:42.020 INFO:teuthology.orchestra.run.smithi169.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.9 MB/s | 3.0 MB 00:00 2024-04-25T11:24:42.053 INFO:teuthology.orchestra.run.smithi169.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-25T11:24:42.064 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:24:42.078 INFO:teuthology.orchestra.run.smithi169.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-25T11:24:42.120 INFO:teuthology.orchestra.run.smithi169.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 354 kB/s | 35 kB 00:00 2024-04-25T11:24:42.252 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:24:42.277 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:24:42.279 INFO:teuthology.orchestra.run.smithi169.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-25T11:24:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:24:42.363 INFO:teuthology.orchestra.run.smithi169.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T11:24:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:24:42.421 INFO:teuthology.orchestra.run.smithi169.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-25T11:24:42.728 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:24:42.729 INFO:teuthology.orchestra.run.smithi169.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T11:24:42.729 INFO:teuthology.orchestra.run.smithi169.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-04-25T11:24:42.729 INFO:teuthology.orchestra.run.smithi169.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-25T11:24:42.729 INFO:teuthology.orchestra.run.smithi169.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 957 kB/s | 87 kB 00:00 2024-04-25T11:24:42.729 INFO:teuthology.orchestra.run.smithi169.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T11:24:42.761 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:24:42.799 INFO:teuthology.orchestra.run.smithi169.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-25T11:24:42.813 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:24:42.834 INFO:teuthology.orchestra.run.smithi169.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T11:24:42.857 INFO:teuthology.orchestra.run.smithi169.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 448 kB/s | 26 kB 00:00 2024-04-25T11:24:42.882 INFO:teuthology.orchestra.run.smithi169.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 387 kB/s | 19 kB 00:00 2024-04-25T11:24:42.916 INFO:teuthology.orchestra.run.smithi169.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T11:24:42.958 INFO:teuthology.orchestra.run.smithi169.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 466 kB/s | 35 kB 00:00 2024-04-25T11:24:43.015 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:24:43.083 INFO:teuthology.orchestra.run.smithi169.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-25T11:24:43.117 INFO:teuthology.orchestra.run.smithi169.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T11:24:43.149 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:24:43.150 INFO:teuthology.orchestra.run.smithi169.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-25T11:24:43.167 INFO:teuthology.orchestra.run.smithi169.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T11:24:43.196 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:24:43.223 INFO:teuthology.orchestra.run.smithi169.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 583 kB/s | 42 kB 00:00 2024-04-25T11:24:43.246 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:24:43.259 INFO:teuthology.orchestra.run.smithi169.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-25T11:24:43.293 INFO:teuthology.orchestra.run.smithi169.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 159 kB 00:00 2024-04-25T11:24:43.355 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:24:43.387 INFO:teuthology.orchestra.run.smithi169.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-25T11:24:43.409 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:24:43.410 INFO:teuthology.orchestra.run.smithi169.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-04-25T11:24:43.444 INFO:teuthology.orchestra.run.smithi169.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 943 kB/s | 54 kB 00:00 2024-04-25T11:24:43.476 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:24:43.523 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:24:43.544 INFO:teuthology.orchestra.run.smithi169.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.1 MB/s | 5.1 MB 00:00 2024-04-25T11:24:43.561 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:24:43.569 INFO:teuthology.orchestra.run.smithi169.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 335 kB/s | 42 kB 00:00 2024-04-25T11:24:43.611 INFO:teuthology.orchestra.run.smithi169.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 670 kB/s | 44 kB 00:00 2024-04-25T11:24:43.645 INFO:teuthology.orchestra.run.smithi169.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-25T11:24:43.670 INFO:teuthology.orchestra.run.smithi169.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 623 kB/s | 36 kB 00:00 2024-04-25T11:24:43.693 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:24:43.695 INFO:teuthology.orchestra.run.smithi169.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-04-25T11:24:43.746 INFO:teuthology.orchestra.run.smithi169.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T11:24:43.769 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:24:43.771 INFO:teuthology.orchestra.run.smithi169.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 776 kB/s | 58 kB 00:00 2024-04-25T11:24:43.805 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:24:43.821 INFO:teuthology.orchestra.run.smithi169.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-25T11:24:43.846 INFO:teuthology.orchestra.run.smithi169.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 852 kB/s | 64 kB 00:00 2024-04-25T11:24:43.866 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:24:43.888 INFO:teuthology.orchestra.run.smithi169.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-04-25T11:24:43.905 INFO:teuthology.orchestra.run.smithi169.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 383 kB/s | 22 kB 00:00 2024-04-25T11:24:43.984 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:24:44.031 INFO:teuthology.orchestra.run.smithi169.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-04-25T11:24:44.033 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T11:24:44.089 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T11:24:44.124 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T11:24:44.247 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:24:44.310 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:24:44.331 INFO:teuthology.orchestra.run.smithi169.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-25T11:24:44.424 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:24:44.615 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:24:44.687 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:24:44.840 INFO:teuthology.orchestra.run.smithi169.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 44 kB/s | 19 kB 00:00 2024-04-25T11:24:45.006 INFO:teuthology.orchestra.run.smithi169.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 498 kB/s | 548 kB 00:01 2024-04-25T11:24:45.040 INFO:teuthology.orchestra.run.smithi169.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 320 kB/s | 309 kB 00:00 2024-04-25T11:24:45.124 INFO:teuthology.orchestra.run.smithi169.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-04-25T11:24:45.157 INFO:teuthology.orchestra.run.smithi169.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 418 kB/s | 49 kB 00:00 2024-04-25T11:24:45.235 INFO:teuthology.orchestra.run.smithi169.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 614 kB/s | 67 kB 00:00 2024-04-25T11:24:45.293 INFO:teuthology.orchestra.run.smithi169.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-25T11:24:45.411 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:24:45.461 INFO:teuthology.orchestra.run.smithi169.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-04-25T11:24:45.511 INFO:teuthology.orchestra.run.smithi169.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-25T11:24:45.521 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:24:45.570 INFO:teuthology.orchestra.run.smithi169.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-04-25T11:24:45.571 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:24:45.625 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:24:45.661 INFO:teuthology.orchestra.run.smithi169.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-04-25T11:24:45.667 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T11:24:45.687 INFO:teuthology.orchestra.run.smithi169.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-04-25T11:24:45.712 INFO:teuthology.orchestra.run.smithi169.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 161 kB/s | 32 kB 00:00 2024-04-25T11:24:45.728 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:24:45.779 INFO:teuthology.orchestra.run.smithi169.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T11:24:45.812 INFO:teuthology.orchestra.run.smithi169.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T11:24:45.846 INFO:teuthology.orchestra.run.smithi169.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T11:24:45.888 INFO:teuthology.orchestra.run.smithi169.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-04-25T11:24:45.930 INFO:teuthology.orchestra.run.smithi169.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T11:24:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:24:46.005 INFO:teuthology.orchestra.run.smithi169.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-25T11:24:46.047 INFO:teuthology.orchestra.run.smithi169.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-25T11:24:46.056 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:24:46.114 INFO:teuthology.orchestra.run.smithi169.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-25T11:24:46.136 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:24:46.181 INFO:teuthology.orchestra.run.smithi169.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-04-25T11:24:46.206 INFO:teuthology.orchestra.run.smithi169.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-04-25T11:24:46.232 INFO:teuthology.orchestra.run.smithi169.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-04-25T11:24:46.241 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:24:46.282 INFO:teuthology.orchestra.run.smithi169.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-25T11:24:46.282 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:24:46.319 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:24:46.349 INFO:teuthology.orchestra.run.smithi169.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-25T11:24:46.366 INFO:teuthology.orchestra.run.smithi169.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-04-25T11:24:46.390 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:24:46.391 INFO:teuthology.orchestra.run.smithi169.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 727 kB/s | 79 kB 00:00 2024-04-25T11:24:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:24:46.450 INFO:teuthology.orchestra.run.smithi169.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 578 kB/s | 58 kB 00:00 2024-04-25T11:24:46.508 INFO:teuthology.orchestra.run.smithi169.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-25T11:24:46.510 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:24:46.533 INFO:teuthology.orchestra.run.smithi169.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-04-25T11:24:46.559 INFO:teuthology.orchestra.run.smithi169.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2024-04-25T11:24:46.577 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:24:46.617 INFO:teuthology.orchestra.run.smithi169.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-25T11:24:46.643 INFO:teuthology.orchestra.run.smithi169.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-04-25T11:24:46.677 INFO:teuthology.orchestra.run.smithi169.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-25T11:24:46.727 INFO:teuthology.orchestra.run.smithi169.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-25T11:24:46.752 INFO:teuthology.orchestra.run.smithi169.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 331 kB/s | 36 kB 00:00 2024-04-25T11:24:46.786 INFO:teuthology.orchestra.run.smithi169.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 471 kB/s | 51 kB 00:00 2024-04-25T11:24:46.787 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:24:46.844 INFO:teuthology.orchestra.run.smithi169.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-25T11:24:46.865 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:24:46.869 INFO:teuthology.orchestra.run.smithi169.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 769 kB/s | 90 kB 00:00 2024-04-25T11:24:46.907 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:24:46.911 INFO:teuthology.orchestra.run.smithi169.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T11:24:46.953 INFO:teuthology.orchestra.run.smithi169.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-04-25T11:24:46.966 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:24:46.987 INFO:teuthology.orchestra.run.smithi169.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T11:24:47.071 INFO:teuthology.orchestra.run.smithi169.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-25T11:24:47.113 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:24:47.155 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:24:47.213 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:24:47.254 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:24:47.263 INFO:teuthology.orchestra.run.smithi169.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.1 MB/s | 1.3 MB 00:00 2024-04-25T11:24:47.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:24:47.373 INFO:teuthology.orchestra.run.smithi169.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T11:24:47.473 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:24:47.613 INFO:teuthology.orchestra.run.smithi169.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 5.0 MB/s | 3.3 MB 00:00 2024-04-25T11:24:47.660 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:24:47.709 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:24:47.750 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:24:47.808 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:24:47.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:24:47.962 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:24:48.008 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:24:48.053 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:24:48.094 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:24:48.096 INFO:teuthology.orchestra.run.smithi169.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T11:24:48.109 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:24:48.109 INFO:teuthology.orchestra.run.smithi169.stdout:Total 9.1 MB/s | 214 MB 00:23 2024-04-25T11:24:48.168 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:24:48.216 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:24:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:24:48.413 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:24:48.464 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:24:48.535 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:24:48.676 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:24:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:24:49.773 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:24:49.827 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:24:49.860 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:24:50.014 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:24:50.081 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:24:50.193 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:24:50.236 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:24:50.486 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:24:50.516 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:24:50.631 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:24:50.981 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-25T11:24:51.221 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-25T11:24:51.221 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-25T11:24:51.528 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:24:51.676 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:24:53.351 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:24:53.447 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:24:53.485 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:24:53.540 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:24:53.978 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:24:53.982 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-25T11:24:53.983 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-25T11:24:54.026 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:24:54.150 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:24:54.183 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T11:24:54.245 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:24:54.858 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:24:54.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:24:55.013 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:24:55.047 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T11:24:58.027 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:24:58.068 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:24:58.789 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:24:58.830 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:24:59.013 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:24:59.013 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:24:59.014 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:24:59.014 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:24:59.056 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:25:00.210 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-25T11:25:00.329 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:25:00.392 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:25:00.487 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:25:00.545 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:25:00.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:25:00.740 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:25:01.203 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:25:01.319 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:25:01.497 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:25:01.536 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:25:01.664 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:25:01.736 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:25:02.085 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:25:02.154 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:25:02.182 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:25:02.335 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:25:02.366 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:25:02.453 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:25:02.484 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:25:02.551 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:25:02.854 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:25:02.910 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:25:03.108 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:25:03.236 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:25:03.292 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:25:03.342 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:25:03.448 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:25:03.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:25:03.559 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:25:03.601 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:25:03.641 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:25:03.755 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:25:03.835 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:25:03.867 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:25:03.951 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:25:04.071 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:25:04.113 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T11:25:04.168 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T11:25:04.203 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T11:25:04.322 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:25:04.380 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:25:04.489 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:25:04.673 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:25:04.742 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:25:05.430 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:25:05.548 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:25:05.583 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:25:05.629 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:25:05.670 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T11:25:05.732 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:25:05.974 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:25:06.036 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:25:06.128 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:25:06.224 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:25:06.269 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:25:06.309 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:25:06.375 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:25:06.416 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:25:06.477 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:25:06.536 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:25:06.732 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:25:06.781 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:25:06.818 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:25:06.865 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:25:06.992 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:25:07.035 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:25:07.090 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:25:07.125 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:25:07.183 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:25:07.380 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:25:07.558 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:25:07.606 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:25:07.648 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:25:07.691 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:25:07.749 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:25:07.801 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:25:07.843 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:25:07.888 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:25:07.938 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:25:08.013 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:25:08.061 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:25:08.202 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:25:08.308 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:25:08.354 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:25:08.436 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:25:08.589 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:25:08.649 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:25:09.625 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:25:09.748 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:25:09.783 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:25:09.916 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:25:10.001 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:25:10.103 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:25:10.150 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:25:10.396 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:25:10.438 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:25:10.565 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:25:11.448 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:25:11.610 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:25:12.408 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-04-25T11:25:12.409 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:12.668 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:25:12.718 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:25:12.718 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:12.718 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:25:12.719 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:13.178 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:25:13.278 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:25:13.318 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:25:13.363 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:25:13.751 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:25:13.790 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:25:13.917 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:25:13.955 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T11:25:14.012 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:25:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:25:14.568 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:25:14.601 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:25:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:25:14.618 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:14.618 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:25:14.619 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:25:14.619 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:25:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:14.702 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:25:14.728 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T11:25:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:25:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:25:14.969 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:14.969 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:25:14.969 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:25:14.969 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:25:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:25:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:25:15.584 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:15.584 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:25:15.584 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:25:15.584 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:25:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:25:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:25:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:25:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:25:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:25:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:25:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:25:16.553 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:25:16.603 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:25:16.641 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:25:16.674 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:25:17.332 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:25:17.359 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:25:17.368 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:25:17.400 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:25:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T11:25:17.537 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T11:25:17.570 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:25:18.079 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:25:18.129 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:25:18.312 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:25:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:25:18.313 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:25:18.313 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:18.355 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:25:18.957 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:25:19.005 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:25:20.188 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:25:20.213 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:25:20.366 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:25:20.593 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T11:25:20.632 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:25:20.715 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:25:20.716 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:20.716 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:25:20.716 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:25:20.716 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:25:20.716 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:20.774 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:25:20.815 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:25:20.953 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T11:25:21.030 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T11:25:22.144 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:25:22.177 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:25:22.256 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:25:22.256 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:22.256 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:25:22.256 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:25:22.256 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:25:22.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:22.311 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:25:22.403 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:25:22.403 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:22.403 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:25:22.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:22.478 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:25:22.585 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:25:22.585 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:22.585 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:25:22.585 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:25:22.586 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:25:22.586 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:23.021 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:25:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:25:23.119 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:23.119 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:25:23.119 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:25:23.119 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:25:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:28.146 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T11:25:28.205 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T11:25:28.292 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T11:25:28.340 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:25:28.387 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:25:28.746 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T11:25:28.815 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:25:28.868 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T11:25:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:25:29.865 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:25:29.931 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:25:31.386 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T11:25:31.499 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-25T11:25:31.517 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-25T11:25:31.518 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:31.582 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:25:31.744 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:25:31.765 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:25:31.794 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:25:31.794 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:31.794 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:25:31.794 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:31.804 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:25:31.804 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:25:31.930 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:25:31.970 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:25:31.971 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:25:32.111 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:25:32.257 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:25:32.571 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:25:32.571 INFO:teuthology.orchestra.run.smithi019.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T11:25:32.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:33.478 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:25:33.559 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:25:33.559 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:33.559 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:25:33.559 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:25:33.559 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:25:33.559 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:33.812 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:25:33.882 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:25:33.883 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:33.883 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:25:33.883 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:25:33.883 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:25:33.883 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:34.396 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:34.613 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:25:34.679 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:25:34.753 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:25:34.985 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:25:35.041 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:25:35.127 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:25:35.236 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:25:35.405 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:25:35.459 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:25:35.499 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:25:35.532 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:25:36.151 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:25:36.193 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:25:36.250 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T11:25:36.357 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T11:25:36.395 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:25:37.712 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:25:37.763 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:25:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T11:25:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T11:25:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T11:25:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T11:25:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T11:25:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T11:25:38.553 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T11:25:38.554 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T11:25:38.555 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T11:25:38.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T11:25:38.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T11:25:38.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T11:25:38.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T11:25:38.567 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:25:38.568 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:25:38.569 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:25:38.570 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:25:38.571 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:25:38.572 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:25:38.573 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:25:38.574 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:25:38.575 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:25:38.576 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T11:25:38.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:25:38.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:25:38.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:25:38.577 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:25:38.866 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:25:38.897 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:25:39.043 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:25:39.236 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T11:25:39.274 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:25:39.350 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:25:39.350 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:39.350 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:25:39.350 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:25:39.350 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:25:39.350 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:39.408 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:25:39.445 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:25:39.552 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T11:25:39.649 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T11:25:40.671 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:25:40.720 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:25:40.798 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:25:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:25:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:25:40.799 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:25:40.799 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:40.853 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:25:40.941 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:25:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:40.941 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:25:40.941 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:41.012 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:25:41.033 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.034 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.035 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.036 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:25:41.037 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:25:41.038 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:25:41.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:25:41.040 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:25:41.041 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.042 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:25:41.043 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:25:41.044 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T11:25:41.104 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:25:41.105 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:41.105 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:25:41.105 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:25:41.105 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:25:41.105 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:41.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:25:41.566 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:25:41.566 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:25:41.566 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:25:41.566 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:25:41.566 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:25:41.567 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:41.651 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:46.277 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T11:25:46.332 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T11:25:46.424 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T11:25:46.473 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:25:46.523 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:25:46.854 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T11:25:46.906 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:25:46.934 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T11:25:47.856 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:25:47.908 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:25:47.956 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:25:49.279 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T11:25:49.410 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:25:49.497 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:25:49.675 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:25:49.720 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:25:49.721 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:25:49.847 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:25:49.894 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:25:49.895 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:25:50.019 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:25:50.158 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:25:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:25:50.460 INFO:teuthology.orchestra.run.smithi169.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T11:25:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T11:25:56.261 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T11:25:56.262 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T11:25:56.263 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T11:25:56.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T11:25:56.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T11:25:56.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T11:25:56.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T11:25:56.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T11:25:56.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T11:25:56.275 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:25:56.276 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:25:56.277 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:25:56.278 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:25:56.279 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:25:56.280 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:25:56.281 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:25:56.282 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:25:56.283 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:25:56.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T11:25:56.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:25:56.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T11:25:56.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:25:56.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:25:56.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:25:56.284 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:25:58.681 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:25:58.681 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:58.681 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-25T11:25:58.681 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.681 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.681 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:25:58.681 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.682 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.683 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:25:58.684 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:25:58.685 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:25:58.686 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:25:58.686 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:25:58.686 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.686 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.686 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:25:58.686 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.686 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:25:58.687 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.688 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:25:58.689 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:25:58.690 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:25:58.691 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.692 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:25:58.693 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:25:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:25:58.695 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:25:58.695 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:25:58.695 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T11:25:59.277 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:59.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:25:59.535 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:25:59.608 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T11:25:59.609 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T11:25:59.609 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T11:25:59.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:25:59.870 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:25:59.929 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T11:25:59.929 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T11:25:59.929 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T11:25:59.931 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:25:59.932 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:25:59.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:26:00.030 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:00.030 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:26:00.106 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:26:00.106 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:00.106 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:26:00.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:26:00.331 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:00.331 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:26:00.398 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:26:00.504 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:26:00.504 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:00.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:26:00.598 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:26:00.734 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:00.734 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:26:00.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:26:00.910 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:26:00.910 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:00.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:26:01.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:26:01.093 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:01.093 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:26:01.159 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:26:01.261 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T11:26:01.344 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T11:26:01.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /var/log/ceph 2024-04-25T11:26:01.352 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2024-04-25T11:26:01.440 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T11:26:01.440 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:26:01.447 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:26:01.528 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T11:26:01.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:26:01.535 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:26:01.620 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T11:26:01.621 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T11:26:01.621 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T11:26:01.621 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:01.621 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:26:01.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:26:01.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T11:26:02.019 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:02.019 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:26:02.064 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:26:02.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T11:26:02.313 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T11:26:02.313 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T11:26:02.313 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/damage}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673135', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi019.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnRa7wX4JPdnmhx6mqM2K0CI7CJKm/oKVfTMnJlEeI9hGPzJCwffaZS0YEwdPeKlZ+ZxyiU5I3NCcetbZngXq0=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEf7cTDSBvuXWGk+YVTL9kClpiBtnl29eEgFtkLLFGlvZA5yYftciQnVOnYRweqQ6NQax72C3f7M/xO7E+9ntJk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T11:26:02.314 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:26:02.321 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:26:02.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:26:02.381 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:26:02.461 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:02.462 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:26:02.517 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:26:02.517 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-04-25T11:26:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:26:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:02.572 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-25T11:26:02.573 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:02.573 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:02.573 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:25:33.054496286 +0000 2024-04-25T11:26:02.573 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:23:09.241048461 +0000 2024-04-25T11:26:02.573 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:23:09.241048461 +0000 2024-04-25T11:26:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:23:09.241048461 +0000 2024-04-25T11:26:02.573 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:26:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:26:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:26:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000350607 s, 1.5 MB/s 2024-04-25T11:26:02.657 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:26:02.715 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-04-25T11:26:02.768 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:26:02.768 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:02.768 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T11:26:02.768 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:02.768 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:02.769 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:25:33.055496275 +0000 2024-04-25T11:26:02.769 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:23:09.863042114 +0000 2024-04-25T11:26:02.769 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:23:09.863042114 +0000 2024-04-25T11:26:02.769 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:23:09.863042114 +0000 2024-04-25T11:26:02.769 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:26:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:26:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:26:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000364636 s, 1.4 MB/s 2024-04-25T11:26:02.856 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:26:02.917 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-04-25T11:26:02.972 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:25:33.055496275 +0000 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:23:10.477035848 +0000 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:23:10.477035848 +0000 2024-04-25T11:26:02.973 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:23:10.477035848 +0000 2024-04-25T11:26:02.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:26:03.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:26:03.049 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:26:03.049 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.0003562 s, 1.4 MB/s 2024-04-25T11:26:03.053 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:26:03.108 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-04-25T11:26:03.159 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:26:03.159 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:03.159 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-25T11:26:03.159 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:03.159 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:03.160 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:25:33.056496265 +0000 2024-04-25T11:26:03.160 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:23:11.098029510 +0000 2024-04-25T11:26:03.160 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:23:11.098029510 +0000 2024-04-25T11:26:03.160 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:23:11.098029510 +0000 2024-04-25T11:26:03.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:26:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:26:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:26:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000336621 s, 1.5 MB/s 2024-04-25T11:26:03.244 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:26:03.302 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:03.303 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:03.303 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:26:03.354 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:26:03.355 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-25T11:26:03.397 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:26:03.397 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:03.397 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T11:26:03.397 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:03.397 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:03.397 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:25:50.891316369 +0000 2024-04-25T11:26:03.398 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:23:01.944909219 +0000 2024-04-25T11:26:03.398 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:23:01.944909219 +0000 2024-04-25T11:26:03.398 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:23:01.944909219 +0000 2024-04-25T11:26:03.398 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:26:03.508 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:26:03.508 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:26:03.508 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000424658 s, 1.2 MB/s 2024-04-25T11:26:03.511 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:26:03.558 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-25T11:26:03.635 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:25:50.892316348 +0000 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:23:02.542895788 +0000 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:23:02.542895788 +0000 2024-04-25T11:26:03.636 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:23:02.542895788 +0000 2024-04-25T11:26:03.636 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:26:03.734 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:26:03.734 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:26:03.734 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000284717 s, 1.8 MB/s 2024-04-25T11:26:03.738 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:26:03.776 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-25T11:26:03.855 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:26:03.855 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:03.855 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-25T11:26:03.855 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:03.855 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:03.855 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:25:50.892316348 +0000 2024-04-25T11:26:03.855 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:23:03.123882740 +0000 2024-04-25T11:26:03.856 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:23:03.123882740 +0000 2024-04-25T11:26:03.856 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:23:03.123882740 +0000 2024-04-25T11:26:03.856 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:26:03.955 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:26:03.955 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:26:03.955 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000273686 s, 1.9 MB/s 2024-04-25T11:26:03.958 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:26:03.998 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:25:50.893316325 +0000 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:23:03.725869220 +0000 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:23:03.725869220 +0000 2024-04-25T11:26:04.076 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:23:03.725869220 +0000 2024-04-25T11:26:04.076 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:26:04.176 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:26:04.176 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:26:04.176 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000275485 s, 1.9 MB/s 2024-04-25T11:26:04.180 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:26:04.219 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:04.219 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:26:04.219 INFO:tasks.ceph:Generating config... 2024-04-25T11:26:04.220 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T11:26:04.220 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T11:26:04.220 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T11:26:04.220 INFO:tasks.ceph:[client] pid file = 2024-04-25T11:26:04.220 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T11:26:04.220 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T11:26:04.220 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T11:26:04.220 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T11:26:04.220 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T11:26:04.220 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T11:26:04.221 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T11:26:04.221 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T11:26:04.222 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T11:26:04.223 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T11:26:04.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T11:26:04.434 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T11:26:04.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T11:26:04.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:26:04.742 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.19'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2024-04-25T11:26:04.743 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.19,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T11:26:04.743 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:04.743 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:26:04.801 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.19 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: generated fsid 23a56573-5965-4bbb-bd60-741a44babbed 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 0 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 23a56573-5965-4bbb-bd60-741a44babbed 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:last_changed 2024-04-25T11:26:05.044653+0000 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T11:26:05.044653+0000 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:min_mon_release 17 (quincy) 2024-04-25T11:26:05.036 INFO:teuthology.orchestra.run.smithi019.stdout:election_strategy: 1 2024-04-25T11:26:05.037 INFO:teuthology.orchestra.run.smithi019.stdout:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T11:26:05.037 INFO:teuthology.orchestra.run.smithi019.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2024-04-25T11:26:05.037 INFO:teuthology.orchestra.run.smithi019.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2024-04-25T11:26:05.037 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T11:26:05.046 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:26:05.099 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 23a56573-5965-4bbb-bd60-741a44babbed... 2024-04-25T11:26:05.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:26:05.104 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.218 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2024-04-25T11:26:05.219 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.220 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.19,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 23a56573-5965-4bbb-bd60-741a44babbed 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2024-04-25T11:26:05.221 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug trim objects = true 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.222 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: bdev async discard = True 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: bdev enable discard = True 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore allocator = bitmap 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2024-04-25T11:26:05.223 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 1/20 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 1/20 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 4/10 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2024-04-25T11:26:05.224 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2024-04-25T11:26:05.225 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.226 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon down mkfs grace = 300 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2024-04-25T11:26:05.227 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2024-04-25T11:26:05.228 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2024-04-25T11:26:05.229 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2024-04-25T11:26:05.229 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2024-04-25T11:26:05.229 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2024-04-25T11:26:05.229 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2024-04-25T11:26:05.229 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2024-04-25T11:26:05.229 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2024-04-25T11:26:05.229 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-04-25T11:26:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 23a56573-5965-4bbb-bd60-741a44babbed 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:26:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-04-25T11:26:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: bdev async discard = True 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: bdev enable discard = True 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore allocator = bitmap 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 1/20 2024-04-25T11:26:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 1/20 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 4/10 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-04-25T11:26:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-04-25T11:26:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:26:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: mon down mkfs grace = 300 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-04-25T11:26:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T11:26:05.254 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = True 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = True 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-04-25T11:26:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-04-25T11:26:05.281 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T11:26:05.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T11:26:05.501 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T11:26:05.501 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:05.501 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T11:26:05.555 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:05.555 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T11:26:05.608 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:26:05.608 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:05.608 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:26:05.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:26:05.738 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:05.739 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:05.833 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:26:05.834 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:05.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:26:05.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:26:05.932 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:05.932 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:06.023 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T11:26:06.023 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T11:26:06.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:26:06.246 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:26:06.263 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:26:06.497 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:26:06.512 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T11:26:06.513 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:26:06.767 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:26:06.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T11:26:06.872 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:26:07.097 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:26:07.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T11:26:07.202 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:26:07.438 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:26:07.451 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T11:26:07.526 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:26:07.724 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:26:07.737 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T11:26:07.813 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T11:26:07.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T11:26:08.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T11:26:08.260 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T11:26:08.318 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T11:26:08.498 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T11:26:08.552 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T11:26:08.730 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T11:26:08.787 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T11:26:08.787 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:26:08.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T11:26:08.881 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:08.881 INFO:tasks.ceph:role: osd.0 2024-04-25T11:26:08.881 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:26:08.881 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:08.976 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:09.397 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T11:26:09.403 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T11:26:09.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T11:26:09.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T11:26:09.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T11:26:09.760 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:09.800 INFO:tasks.ceph:role: osd.1 2024-04-25T11:26:09.800 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:26:09.800 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:26:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:09.902 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:09.902 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:09.902 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:09.902 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:09.902 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:09.903 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:09.903 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:10.274 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T11:26:10.281 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T11:26:10.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T11:26:10.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T11:26:10.496 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T11:26:10.580 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:10.580 INFO:tasks.ceph:role: osd.2 2024-04-25T11:26:10.580 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:26:10.581 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:10.677 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:10.678 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:11.100 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T11:26:11.108 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T11:26:11.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T11:26:11.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T11:26:11.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T11:26:11.449 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:11.449 INFO:tasks.ceph:role: osd.3 2024-04-25T11:26:11.449 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:26:11.449 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:26:11.544 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:11.545 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:11.955 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-04-25T11:26:11.961 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-04-25T11:26:11.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T11:26:12.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T11:26:12.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:12.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:12.488+0000 7fac9340c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T11:26:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:12.489+0000 7fac9340c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T11:26:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:12.489+0000 7fac9340c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:26:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:12.489+0000 7fac9340c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:26:12.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:12.489+0000 7fac9340c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:26:12.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:12.492+0000 7fac9340c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T11:26:15.484 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T11:26:15.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:15.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:15.799+0000 7f8a67e16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T11:26:15.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:15.799+0000 7f8a67e16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T11:26:15.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:15.799+0000 7f8a67e16640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:26:15.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:15.799+0000 7f8a67e16640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:26:15.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:15.799+0000 7f8a67e16640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:26:15.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:15.802+0000 7f8a67e16640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T11:26:19.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T11:26:19.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:19.343+0000 7fbcd5c19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T11:26:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:19.344+0000 7fbcd5c19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T11:26:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:19.344+0000 7fbcd5c19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:26:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:19.344+0000 7fbcd5c19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:26:19.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:19.344+0000 7fbcd5c19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:26:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:19.346+0000 7fbcd5c19640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T11:26:22.322 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T11:26:22.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:22.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:22.619+0000 7f4d55350640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T11:26:22.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:22.619+0000 7f4d55350640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T11:26:22.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:22.619+0000 7f4d55350640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:26:22.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:22.619+0000 7f4d55350640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:26:22.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:22.619+0000 7f4d55350640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:26:22.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:22.622+0000 7f4d55350640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T11:26:25.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T11:26:25.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T11:26:25.774 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:25.774 INFO:tasks.ceph:role: osd.4 2024-04-25T11:26:25.775 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:26:25.775 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:26:25.862 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:25.862 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:25.862 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:25.862 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:25.862 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:25.862 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:25.863 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:25.863 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:25.863 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:25.863 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:26.278 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T11:26:26.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T11:26:26.287 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T11:26:26.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T11:26:26.526 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T11:26:26.598 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:26.598 INFO:tasks.ceph:role: osd.5 2024-04-25T11:26:26.598 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:26:26.598 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:26:26.678 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:26.678 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:26.679 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:27.034 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T11:26:27.039 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T11:26:27.039 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T11:26:27.136 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T11:26:27.209 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T11:26:27.277 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:27.277 INFO:tasks.ceph:role: osd.6 2024-04-25T11:26:27.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:26:27.277 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:26:27.349 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:27.349 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:27.350 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:27.742 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T11:26:27.748 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T11:26:27.748 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T11:26:27.857 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T11:26:27.929 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T11:26:28.004 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:26:28.004 INFO:tasks.ceph:role: osd.7 2024-04-25T11:26:28.004 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:26:28.005 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:26:28.086 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:26:28.087 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:26:28.503 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2024-04-25T11:26:28.510 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2024-04-25T11:26:28.510 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T11:26:28.681 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T11:26:28.776 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:28.995 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:29.003+0000 7f8359012640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T11:26:28.996 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:29.003+0000 7f8359012640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T11:26:28.996 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:29.004+0000 7f8359012640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:26:28.996 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:29.004+0000 7f8359012640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:26:28.996 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:29.004+0000 7f8359012640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:26:28.999 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:29.007+0000 7f8359012640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T11:26:31.991 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T11:26:32.074 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:32.272 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:32.280+0000 7fc6a4b26640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T11:26:32.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:32.281+0000 7fc6a4b26640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T11:26:32.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:32.281+0000 7fc6a4b26640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:26:32.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:32.281+0000 7fc6a4b26640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:26:32.273 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:32.281+0000 7fc6a4b26640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:26:32.275 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:32.283+0000 7fc6a4b26640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T11:26:32.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:26:32.323 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:26:35.267 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T11:26:35.353 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:35.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:35.562+0000 7f07eb344640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T11:26:35.555 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:35.563+0000 7f07eb344640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T11:26:35.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:35.563+0000 7f07eb344640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:26:35.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:35.563+0000 7f07eb344640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:26:35.556 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:35.563+0000 7f07eb344640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:26:35.557 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:35.565+0000 7f07eb344640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T11:26:38.532 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T11:26:38.611 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:38.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:38.820+0000 7f9622219640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T11:26:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:38.821+0000 7f9622219640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T11:26:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:38.821+0000 7f9622219640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:26:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:38.821+0000 7f9622219640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:26:38.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:38.821+0000 7f9622219640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:26:38.815 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:26:38.823+0000 7f9622219640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T11:26:41.807 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T11:26:41.893 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T11:26:41.893 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:41.893 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T11:26:41.988 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:41.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T11:26:42.081 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:42.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T11:26:42.172 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:42.172 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T11:26:42.262 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:42.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T11:26:42.351 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:42.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T11:26:42.437 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:42.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T11:26:42.518 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:42.518 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T11:26:42.608 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:42.608 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T11:26:42.691 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:42.691 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T11:26:42.774 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:42.774 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T11:26:42.858 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:42.858 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T11:26:42.941 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:42.942 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T11:26:43.026 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:43.027 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T11:26:43.112 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:43.112 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T11:26:43.172 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:43.172 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T11:26:43.226 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:43.226 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T11:26:43.283 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:43.283 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T11:26:43.333 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T11:26:43.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:26:43.341 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.y] 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA== 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ== 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw== 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2024-04-25T11:26:43.422 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ== 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw== 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q== 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.3] 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw== 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.x] 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA== 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw== 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA== 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.4] 2024-04-25T11:26:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw== 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.5] 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw== 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.6] 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ== 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.7] 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA== 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout:[client.2] 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg== 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout:[client.1] 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q== 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout:[client.3] 2024-04-25T11:26:43.424 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q== 2024-04-25T11:26:43.446 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2024-04-25T11:26:43.446 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA== 2024-04-25T11:26:43.446 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2024-04-25T11:26:43.446 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ== 2024-04-25T11:26:43.446 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2024-04-25T11:26:43.446 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw== 2024-04-25T11:26:43.446 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ== 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw== 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q== 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw== 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA== 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw== 2024-04-25T11:26:43.447 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA== 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw== 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw== 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ== 2024-04-25T11:26:43.448 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA== 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg== 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q== 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2024-04-25T11:26:43.449 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q== 2024-04-25T11:26:43.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:26:43.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:26:43.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:43.707 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:43.920 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:43.927 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:44.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:44.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:44.369 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:44.374 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:44.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:44.595 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:44.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:44.815 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:45.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:26:45.036 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:26:45.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:45.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:45.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:45.469 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:26:45.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:45.692 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:45.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:45.909 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:46.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:46.127 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:46.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:46.342 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:26:46.553 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:46.559 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:46.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:46.779 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:46.996 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:47.003 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:47.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:47.221 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:26:47.437 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T11:26:47.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T11:26:47.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:26:47.756 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T11:26:47.825 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T11:26:47.913 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:26:48.137 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T11:26:48.221 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T11:26:48.305 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:26:48.527 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T11:26:48.596 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:48.604 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:26:48.656 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T11:26:48.657 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T11:26:48.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T11:26:48.662 INFO:tasks.ceph.mon.a:Started 2024-04-25T11:26:48.662 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T11:26:48.662 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T11:26:48.699 INFO:tasks.ceph.mon.b:Started 2024-04-25T11:26:48.699 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T11:26:48.699 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T11:26:48.706 INFO:tasks.ceph.mon.c:Started 2024-04-25T11:26:48.706 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T11:26:48.706 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T11:26:48.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T11:26:48.713 INFO:tasks.ceph.mgr.y:Started 2024-04-25T11:26:48.713 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T11:26:48.713 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T11:26:48.718 INFO:tasks.ceph.mgr.x:Started 2024-04-25T11:26:48.718 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T11:26:48.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T11:26:48.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.973+0000 7f1377c00640 1 Processor -- start 2024-04-25T11:26:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.973+0000 7f1377c00640 1 -- start start 2024-04-25T11:26:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.975+0000 7f1377c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.975+0000 7f1377c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13701609b0 con 0x7f1370163fa0 2024-04-25T11:26:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.975+0000 7f1377c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1370164cd0 con 0x7f137006c5c0 2024-04-25T11:26:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.975+0000 7f1377c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1370160c30 con 0x7f1370161710 2024-04-25T11:26:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.975+0000 7f1375c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1370163fa0 0x7f13701643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35282/0 (socket says 172.21.15.19:35282) 2024-04-25T11:26:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.976+0000 7f1375c00640 1 -- 172.21.15.19:0/2961820639 learned_addr learned my addr 172.21.15.19:0/2961820639 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:26:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.976+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.976+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.976+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.976+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:48.976+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:26:49.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.177+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:49.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.177+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:49.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.177+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:49.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.177+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:49.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.178+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:26:49.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.579+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:49.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.580+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:49.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.580+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:49.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.580+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:49.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:49.580+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:26:50.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:50.382+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:50.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:50.382+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:50.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:50.382+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:50.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:50.382+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:50.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:50.382+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:26:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f137015e080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:26:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f137015e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> v1:172.21.15.169:6789/0 conn(0x7f137006c5c0 legacy=0x7f137006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> v1:172.21.15.19:6789/0 conn(0x7f1370163fa0 legacy=0x7f13701643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:51.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1370160c30 con 0x7f1350006220 2024-04-25T11:26:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1350008dd0 con 0x7f1350003710 2024-04-25T11:26:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.976+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:51.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:26:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:51.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:51.977+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:26:52.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.178+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:52.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.178+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:26:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.179+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:26:52.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.580+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:52.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.580+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:26:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:52.581+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:26:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:26:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.383+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:53.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:53.384+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:26:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 msgr2=0x7f1350005bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003710 0x7f1350005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 msgr2=0x7f1350008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f1350006220 0x7f1350008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f1375c00640 1 -- 172.21.15.19:0/2961820639 reap_dead start 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1370160f10 con 0x7f1350006b70 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1370160c30 con 0x7f1370161710 2024-04-25T11:26:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1350008dd0 con 0x7f1350003ba0 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.977+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:54.978+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:26:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.179+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.179+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.179+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.179+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.179+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:26:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.179+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.180+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.180+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.180+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.180+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:26:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.581+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:55.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.581+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:55.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.581+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:55.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.581+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:55.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.581+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:26:55.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.581+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:55.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.582+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:55.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.582+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:55.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.582+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:55.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:55.582+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:26:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.383+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.383+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.383+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.383+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.383+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:26:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.384+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.386+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T11:26:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.386+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.386+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:56.386+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 msgr2=0x7f1350006630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1350003ba0 0x7f1350006630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 msgr2=0x7f135000b240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.169:3300/0 conn(0x7f1370161710 0x7f135000b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 >> v1:172.21.15.169:6789/0 conn(0x7f1350006b70 legacy=0x7f1350007030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1350008a60 con 0x7f1350010830 2024-04-25T11:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.978+0000 7f136e000640 1 -- 172.21.15.19:0/2961820639 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1370160c30 con 0x7f135000d8e0 2024-04-25T11:26:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.979+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.979+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 msgr2=0x7f135000dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.979+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 msgr2=0x7f135000dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.979+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:57.979+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:26:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.180+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.181+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 msgr2=0x7f135000dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.181+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 msgr2=0x7f135000dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.181+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.181+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:26:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.582+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.583+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 msgr2=0x7f135000dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T11:26:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.583+0000 7f1375200640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 msgr2=0x7f135000dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:26:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.583+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:26:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:58.583+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:26:59.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.264+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617734459 0 0) 0x7f1350008a60 con 0x7f1350010830 2024-04-25T11:26:59.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.264+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1358003520 con 0x7f1350010830 2024-04-25T11:26:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.265+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3099264060 0 0) 0x7f1358003520 con 0x7f1350010830 2024-04-25T11:26:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.265+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 msgr2=0x7f135000dd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:26:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.265+0000 7f136f400640 1 --2- 172.21.15.19:0/2961820639 >> v2:172.21.15.19:3300/0 conn(0x7f135000d8e0 0x7f135000dd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.265+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1350015380 con 0x7f1350010830 2024-04-25T11:26:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.265+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f135c007da0 con 0x7f1350010830 2024-04-25T11:26:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.265+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 >> v1:172.21.15.19:6789/0 conn(0x7f1350010830 legacy=0x7f1350012c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:59.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.266+0000 7f1375c00640 1 -- 172.21.15.19:0/2961820639 reap_dead start 2024-04-25T11:26:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.266+0000 7f136f400640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13580039b0 0x7f1358003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.266+0000 7f136f400640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358004800 0x7f1358004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.266+0000 7f136f400640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1358005330 0x7f135800b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:59.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.266+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1370160c30 con 0x7f1358005330 2024-04-25T11:26:59.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.266+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f135800c2c0 con 0x7f1358004800 2024-04-25T11:26:59.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.266+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f135800c4c0 con 0x7f13580039b0 2024-04-25T11:26:59.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.267+0000 7f1375200640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13580039b0 0x7f1358003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:59.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.267+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358004800 0x7f1358004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:59.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.267+0000 7f1375c00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1358005330 0x7f135800b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:59.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.268+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13580039b0 msgr2=0x7f1358003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:59.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.268+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13580039b0 0x7f1358003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.268+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1358005330 msgr2=0x7f135800b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:59.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.268+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1358005330 0x7f135800b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.268+0000 7f136fe00640 1 -- 172.21.15.19:0/2961820639 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1358003520 con 0x7f1358004800 2024-04-25T11:26:59.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.270+0000 7f136fe00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358004800 0x7f1358004c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1350003670 tx=0x7f1358005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c120dd35a54d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:59.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.270+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c003610 con 0x7f1358004800 2024-04-25T11:26:59.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.271+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c034410 con 0x7f1358004800 2024-04-25T11:26:59.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.271+0000 7f136f400640 1 -- 172.21.15.19:0/2961820639 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c034800 con 0x7f1358004800 2024-04-25T11:26:59.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 -- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358004800 msgr2=0x7f1358004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:59.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358004800 0x7f1358004c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1350003670 tx=0x7f1358005830 comp rx=0 tx=0).stop 2024-04-25T11:26:59.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 -- 172.21.15.19:0/2961820639 shutdown_connections 2024-04-25T11:26:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358004800 0x7f1358004c10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1358005330 0x7f135800b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 --2- 172.21.15.19:0/2961820639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13580039b0 0x7f1358003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 -- 172.21.15.19:0/2961820639 >> 172.21.15.19:0/2961820639 conn(0x7f13700d5d90 msgr2=0x7f13700d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 -- 172.21.15.19:0/2961820639 shutdown_connections 2024-04-25T11:26:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.272+0000 7f1377c00640 1 -- 172.21.15.19:0/2961820639 wait complete. 2024-04-25T11:26:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.277+0000 7f1377c00640 1 Processor -- start 2024-04-25T11:26:59.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.277+0000 7f1377c00640 1 -- start start 2024-04-25T11:26:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.279+0000 7f1377c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13580039b0 0x7f13700b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.279+0000 7f1377c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1358004800 0x7f13700c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.279+0000 7f1377c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358005330 0x7f13700cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.279+0000 7f1377c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f137015e200 con 0x7f13580039b0 2024-04-25T11:26:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.279+0000 7f1377c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f137015e080 con 0x7f1358005330 2024-04-25T11:26:59.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.279+0000 7f1377c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f137015e380 con 0x7f1358004800 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.280+0000 7f1375200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13580039b0 0x7f13700b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.280+0000 7f136fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1358004800 0x7f13700c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.280+0000 7f1375c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358005330 0x7f13700cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.280+0000 7f1375200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13580039b0 0x7f13700b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49522/0 (socket says 172.21.15.19:49522) 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.280+0000 7f1375200640 1 -- 172.21.15.19:0/366471572 learned_addr learned my addr 172.21.15.19:0/366471572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.281+0000 7f1375200640 1 -- 172.21.15.19:0/366471572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1358004800 msgr2=0x7f13700c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.281+0000 7f1375200640 1 --2- 172.21.15.19:0/366471572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1358004800 0x7f13700c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.281+0000 7f1375200640 1 -- 172.21.15.19:0/366471572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358005330 msgr2=0x7f13700cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.281+0000 7f1375200640 1 --2- 172.21.15.19:0/366471572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358005330 0x7f13700cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.281+0000 7f1375200640 1 -- 172.21.15.19:0/366471572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13701f75a0 con 0x7f13580039b0 2024-04-25T11:26:59.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.281+0000 7f1375200640 1 --2- 172.21.15.19:0/366471572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13580039b0 0x7f13700b8130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1364004650 tx=0x7f13640188f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.281+0000 7f136cc00640 1 -- 172.21.15.19:0/366471572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364025730 con 0x7f13580039b0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.282+0000 7f136cc00640 1 -- 172.21.15.19:0/366471572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13640258d0 con 0x7f13580039b0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.282+0000 7f136cc00640 1 -- 172.21.15.19:0/366471572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364025ca0 con 0x7f13580039b0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.282+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13700b7650 con 0x7f13580039b0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.282+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13700ca110 con 0x7f13580039b0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.282+0000 7f136cc00640 1 -- 172.21.15.19:0/366471572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1364037440 con 0x7f13580039b0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.282+0000 7f136cc00640 1 -- 172.21.15.19:0/366471572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f136402e830 con 0x7f13580039b0 2024-04-25T11:26:59.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.283+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13700d66d0 con 0x7f13580039b0 2024-04-25T11:26:59.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.288+0000 7f136cc00640 1 -- 172.21.15.19:0/366471572 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f136404de50 con 0x7f13580039b0 2024-04-25T11:26:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.364+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1370050740 con 0x7f13580039b0 2024-04-25T11:26:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.369+0000 7f136cc00640 1 -- 172.21.15.19:0/366471572 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f136403c020 con 0x7f13580039b0 2024-04-25T11:26:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13580039b0 msgr2=0x7f13700b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 --2- 172.21.15.19:0/366471572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13580039b0 0x7f13700b8130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1364004650 tx=0x7f13640188f0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 shutdown_connections 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 --2- 172.21.15.19:0/366471572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1358005330 0x7f13700cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 --2- 172.21.15.19:0/366471572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1358004800 0x7f13700c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 --2- 172.21.15.19:0/366471572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13580039b0 0x7f13700b8130 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 >> 172.21.15.19:0/366471572 conn(0x7f13700d5d90 msgr2=0x7f13700d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 shutdown_connections 2024-04-25T11:26:59.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:26:59.370+0000 7f1377c00640 1 -- 172.21.15.19:0/366471572 wait complete. 2024-04-25T11:26:59.387 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T11:26:59.388 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T11:26:59.388 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:59.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T11:26:59.420 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:26:59.428+0000 7fcc3b63d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:26:59.451 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:59.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T11:26:59.507 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:26:59.516+0000 7f16f4d1c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:26:59.574 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:59.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T11:26:59.604 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:26:59.612+0000 7fcc3b63d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:26:59.671 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:26:59.679+0000 7fcc3b63d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:26:59.691 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:26:59.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T11:26:59.705 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:26:59.714+0000 7f16f4d1c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:26:59.767 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:59.768 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T11:26:59.777 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:26:59.785+0000 7f16f4d1c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:26:59.847 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:59.847 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T11:26:59.911 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:26:59.911 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T11:26:59.973 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:26:59.982+0000 7fcc3b63d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:27:00.027 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:27:00.027 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T11:27:00.032 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:00.041+0000 7fcc3b63d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:27:00.080 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:00.088+0000 7f16f4d1c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:27:00.098 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new ccfa8d67-0c42-4220-8b9c-0c14439eb128 0 2024-04-25T11:27:00.137 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:00.146+0000 7f16f4d1c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:27:00.393 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.401+0000 7f41bca00640 1 Processor -- start 2024-04-25T11:27:00.393 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.401+0000 7f41bca00640 1 -- start start 2024-04-25T11:27:00.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41bca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f41b80b8ba0 0x7f41b806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41bca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b80b5270 con 0x7f41b80b91f0 2024-04-25T11:27:00.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41bca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b80d11f0 con 0x7f41b806dc10 2024-04-25T11:27:00.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41bca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f41b806e980 con 0x7f41b80b8ba0 2024-04-25T11:27:00.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41b7e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f41b806dc10 0x7f41b806e020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44960/0 (socket says 172.21.15.169:44960) 2024-04-25T11:27:00.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41b7e00640 1 -- 172.21.15.169:0/3943612855 learned_addr learned my addr 172.21.15.169:0/3943612855 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508523029 0 0) 0x7f41b80d11f0 con 0x7f41b806dc10 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.404+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f419c003520 con 0x7f41b806dc10 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 859603209 0 0) 0x7f419c003520 con 0x7f41b806dc10 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 >> v2:172.21.15.19:3300/0 conn(0x7f41b80b8ba0 msgr2=0x7f41b806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 --2- 172.21.15.169:0/3943612855 >> v2:172.21.15.19:3300/0 conn(0x7f41b80b8ba0 0x7f41b806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 >> v1:172.21.15.19:6789/0 conn(0x7f41b80b91f0 legacy=0x7f41b80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b80d0f10 con 0x7f41b806dc10 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f41b00019f0 con 0x7f41b806dc10 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 >> v1:172.21.15.169:6789/0 conn(0x7f41b806dc10 legacy=0x7f41b806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f419c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f419c004800 0x7f419c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419c0053a0 0x7f419c00ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41b806e980 con 0x7f419c0053a0 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f419c00c300 con 0x7f419c003a00 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f419c00c500 con 0x7f419c004800 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41b7400640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f419c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.405+0000 7f41afe00640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f419c004800 0x7f419c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.406+0000 7f41b7400640 1 -- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f419c004800 msgr2=0x7f419c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.406+0000 7f41b7400640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f419c004800 0x7f419c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.406+0000 7f41b7400640 1 -- 172.21.15.169:0/3943612855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419c0053a0 msgr2=0x7f419c00ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.406+0000 7f41b7400640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419c0053a0 0x7f419c00ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.406+0000 7f41b7400640 1 -- 172.21.15.169:0/3943612855 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c003520 con 0x7f419c003a00 2024-04-25T11:27:00.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.406+0000 7f41b7e00640 1 -- 172.21.15.169:0/3943612855 reap_dead start 2024-04-25T11:27:00.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.407+0000 7f41b7400640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f419c003e50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41a80016a0 tx=0x7f419c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f9513c8ca74282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:00.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.407+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8001270 con 0x7f419c003a00 2024-04-25T11:27:00.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.407+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a802e070 con 0x7f419c003a00 2024-04-25T11:27:00.400 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.407+0000 7f41b6a00640 1 -- 172.21.15.169:0/3943612855 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a80054d0 con 0x7f419c003a00 2024-04-25T11:27:00.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.408+0000 7f41bca00640 1 -- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 msgr2=0x7f419c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.408+0000 7f41bca00640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f419c003e50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f41a80016a0 tx=0x7f419c005830 comp rx=0 tx=0).stop 2024-04-25T11:27:00.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.408+0000 7f41bca00640 1 -- 172.21.15.169:0/3943612855 shutdown_connections 2024-04-25T11:27:00.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.408+0000 7f41bca00640 1 --2- 172.21.15.169:0/3943612855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f419c003e50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.408+0000 7f41bca00640 1 -- 172.21.15.169:0/3943612855 >> 172.21.15.169:0/3943612855 conn(0x7f41b80d03f0 msgr2=0x7f41b80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:00.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.408+0000 7f41bca00640 1 -- 172.21.15.169:0/3943612855 shutdown_connections 2024-04-25T11:27:00.401 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.408+0000 7f41bca00640 1 -- 172.21.15.169:0/3943612855 wait complete. 2024-04-25T11:27:00.402 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.410+0000 7f41bca00640 1 Processor -- start 2024-04-25T11:27:00.402 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.410+0000 7f41bca00640 1 -- start start 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41bca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f41b81db890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41bca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41b81f5d00 0x7f41b81eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41bca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81eb820 0x7f41b80cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41bca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41b806d6d0 con 0x7f41b81eb820 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41bca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f41b809d040 con 0x7f419c003a00 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41bca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f41b806d850 con 0x7f41b81f5d00 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41b7400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f41b81db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41afe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41b81f5d00 0x7f41b81eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41b7400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f41b81db890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57448/0 (socket says 172.21.15.169:57448) 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41b7400640 1 -- 172.21.15.169:0/3538911085 learned_addr learned my addr 172.21.15.169:0/3538911085 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41b7e00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81eb820 0x7f41b80cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41afe00640 1 -- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 msgr2=0x7f41b81db890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41afe00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f41b81db890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41afe00640 1 -- 172.21.15.169:0/3538911085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81eb820 msgr2=0x7f41b80cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41afe00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81eb820 0x7f41b80cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41afe00640 1 -- 172.21.15.169:0/3538911085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b80caaa0 con 0x7f41b81f5d00 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.412+0000 7f41b7400640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f41b81db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.413+0000 7f41afe00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41b81f5d00 0x7f41b81eb2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e870 tx=0x7f41a000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.413+0000 7f41af400640 1 -- 172.21.15.169:0/3538911085 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017070 con 0x7f41b81f5d00 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.413+0000 7f41af400640 1 -- 172.21.15.169:0/3538911085 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a0009e40 con 0x7f41b81f5d00 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.413+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b80cad40 con 0x7f41b81f5d00 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.413+0000 7f41af400640 1 -- 172.21.15.169:0/3538911085 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a000cb80 con 0x7f41b81f5d00 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.413+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b80cb300 con 0x7f41b81f5d00 2024-04-25T11:27:00.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.413+0000 7f41af400640 1 -- 172.21.15.169:0/3538911085 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f41a001a070 con 0x7f41b81f5d00 2024-04-25T11:27:00.407 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.414+0000 7f41af400640 1 -- 172.21.15.169:0/3538911085 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f41a0006830 con 0x7f41b81f5d00 2024-04-25T11:27:00.407 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.414+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b81ebdc0 con 0x7f41b81f5d00 2024-04-25T11:27:00.409 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.417+0000 7f41af400640 1 -- 172.21.15.169:0/3538911085 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f41a000cd20 con 0x7f41b81f5d00 2024-04-25T11:27:00.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.497+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ccfa8d67-0c42-4220-8b9c-0c14439eb128", "id": 0} v 0) -- 0x7f41b8076c60 con 0x7f41b81f5d00 2024-04-25T11:27:00.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.507+0000 7f41af400640 1 -- 172.21.15.169:0/3538911085 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ccfa8d67-0c42-4220-8b9c-0c14439eb128", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f41a0005a60 con 0x7f41b81f5d00 2024-04-25T11:27:00.499 INFO:teuthology.orchestra.run.smithi169.stdout:0 2024-04-25T11:27:00.500 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.508+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41b81f5d00 msgr2=0x7f41b81eb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.500 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.508+0000 7f41bca00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41b81f5d00 0x7f41b81eb2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e870 tx=0x7f41a000ed30 comp rx=0 tx=0).stop 2024-04-25T11:27:00.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.509+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 shutdown_connections 2024-04-25T11:27:00.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.509+0000 7f41bca00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41b81eb820 0x7f41b80cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.509+0000 7f41bca00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f41b81f5d00 0x7f41b81eb2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.509+0000 7f41bca00640 1 --2- 172.21.15.169:0/3538911085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f419c003a00 0x7f41b81db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.509+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 >> 172.21.15.169:0/3538911085 conn(0x7f41b80d03f0 msgr2=0x7f41b809c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:00.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.509+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 shutdown_connections 2024-04-25T11:27:00.501 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.509+0000 7f41bca00640 1 -- 172.21.15.169:0/3538911085 wait complete. 2024-04-25T11:27:00.522 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new dbc3e03d-1ba0-4402-aede-39b9bc0d5b69 1 2024-04-25T11:27:00.765 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.773+0000 7fbc2c000640 1 Processor -- start 2024-04-25T11:27:00.766 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.774+0000 7fbc2c000640 1 -- start start 2024-04-25T11:27:00.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.776+0000 7fbc2c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc2406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.776+0000 7fbc2c000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2406df20 con 0x7fbc24143660 2024-04-25T11:27:00.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.776+0000 7fbc2c000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc240709a0 con 0x7fbc240b56e0 2024-04-25T11:27:00.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.776+0000 7fbc2c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbc24070c70 con 0x7fbc2406d5d0 2024-04-25T11:27:00.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.776+0000 7fbc28c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc2406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.776+0000 7fbc28c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc2406d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41452/0 (socket says 172.21.15.169:41452) 2024-04-25T11:27:00.768 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.776+0000 7fbc28c00640 1 -- 172.21.15.169:0/2382143999 learned_addr learned my addr 172.21.15.169:0/2382143999 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:00.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.777+0000 7fbc28c00640 1 -- 172.21.15.169:0/2382143999 >> v1:172.21.15.169:6789/0 conn(0x7fbc240b56e0 legacy=0x7fbc240b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.777+0000 7fbc23e00640 1 -- 172.21.15.169:0/2382143999 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380800018 0 0) 0x7fbc240709a0 con 0x7fbc240b56e0 2024-04-25T11:27:00.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.777+0000 7fbc28c00640 1 -- 172.21.15.169:0/2382143999 >> v1:172.21.15.19:6789/0 conn(0x7fbc24143660 legacy=0x7fbc24070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:00.769 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.777+0000 7fbc28c00640 1 -- 172.21.15.169:0/2382143999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc240b4920 con 0x7fbc2406d5d0 2024-04-25T11:27:00.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.778+0000 7fbc28c00640 1 --2- 172.21.15.169:0/2382143999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc2406d9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc100098a0 tx=0x7fbc1002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0bfd819030ca8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:00.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.778+0000 7fbc23e00640 1 -- 172.21.15.169:0/2382143999 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10037070 con 0x7fbc2406d5d0 2024-04-25T11:27:00.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.778+0000 7fbc23e00640 1 -- 172.21.15.169:0/2382143999 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc10032440 con 0x7fbc2406d5d0 2024-04-25T11:27:00.770 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.778+0000 7fbc23e00640 1 -- 172.21.15.169:0/2382143999 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10032810 con 0x7fbc2406d5d0 2024-04-25T11:27:00.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.779+0000 7fbc2c000640 1 -- 172.21.15.169:0/2382143999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 msgr2=0x7fbc2406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.779+0000 7fbc2c000640 1 --2- 172.21.15.169:0/2382143999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc2406d9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc100098a0 tx=0x7fbc1002a290 comp rx=0 tx=0).stop 2024-04-25T11:27:00.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.779+0000 7fbc2c000640 1 -- 172.21.15.169:0/2382143999 shutdown_connections 2024-04-25T11:27:00.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.779+0000 7fbc2c000640 1 --2- 172.21.15.169:0/2382143999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc2406d9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.779+0000 7fbc2c000640 1 -- 172.21.15.169:0/2382143999 >> 172.21.15.169:0/2382143999 conn(0x7fbc24073120 msgr2=0x7fbc24073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:00.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.779+0000 7fbc2c000640 1 -- 172.21.15.169:0/2382143999 shutdown_connections 2024-04-25T11:27:00.771 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.779+0000 7fbc2c000640 1 -- 172.21.15.169:0/2382143999 wait complete. 2024-04-25T11:27:00.773 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.781+0000 7fbc2c000640 1 Processor -- start 2024-04-25T11:27:00.773 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.781+0000 7fbc2c000640 1 -- start start 2024-04-25T11:27:00.776 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc2c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc24140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc2c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc240b56e0 0x7fbc2413eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc2c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc24143660 0x7fbc24159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc2c000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240b51c0 con 0x7fbc240b56e0 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc2c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240b4ee0 con 0x7fbc24143660 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc2c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbc240ce0e0 con 0x7fbc2406d5d0 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc29600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc24140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc29600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc24140280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41460/0 (socket says 172.21.15.169:41460) 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.783+0000 7fbc29600640 1 -- 172.21.15.169:0/3414762412 learned_addr learned my addr 172.21.15.169:0/3414762412 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc2a000640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc24143660 0x7fbc24159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc28c00640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc240b56e0 0x7fbc2413eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc29600640 1 -- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc24143660 msgr2=0x7fbc24159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc29600640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc24143660 0x7fbc24159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.777 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc29600640 1 -- 172.21.15.169:0/3414762412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc240b56e0 msgr2=0x7fbc2413eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc29600640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc240b56e0 0x7fbc2413eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc29600640 1 -- 172.21.15.169:0/3414762412 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2413f1e0 con 0x7fbc2406d5d0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc28c00640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc240b56e0 0x7fbc2413eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc2a000640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc24143660 0x7fbc24159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc29600640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc24140280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800b3f0 tx=0x7fbc1800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.784+0000 7fbc21600640 1 -- 172.21.15.169:0/3414762412 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18015020 con 0x7fbc2406d5d0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.785+0000 7fbc21600640 1 -- 172.21.15.169:0/3414762412 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc18011830 con 0x7fbc2406d5d0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.785+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2413f4c0 con 0x7fbc2406d5d0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.785+0000 7fbc21600640 1 -- 172.21.15.169:0/3414762412 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18011c20 con 0x7fbc2406d5d0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.785+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2413fa80 con 0x7fbc2406d5d0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.785+0000 7fbc21600640 1 -- 172.21.15.169:0/3414762412 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc18018020 con 0x7fbc2406d5d0 2024-04-25T11:27:00.778 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.785+0000 7fbc21600640 1 -- 172.21.15.169:0/3414762412 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbc1801a4d0 con 0x7fbc2406d5d0 2024-04-25T11:27:00.779 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.786+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc24050740 con 0x7fbc2406d5d0 2024-04-25T11:27:00.781 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.789+0000 7fbc21600640 1 -- 172.21.15.169:0/3414762412 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbc1801f030 con 0x7fbc2406d5d0 2024-04-25T11:27:00.873 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.881+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dbc3e03d-1ba0-4402-aede-39b9bc0d5b69", "id": 1} v 0) -- 0x7fbc240cd6e0 con 0x7fbc2406d5d0 2024-04-25T11:27:00.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.888+0000 7fbc21600640 1 -- 172.21.15.169:0/3414762412 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dbc3e03d-1ba0-4402-aede-39b9bc0d5b69", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbc180035f0 con 0x7fbc2406d5d0 2024-04-25T11:27:00.880 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-25T11:27:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.889+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 msgr2=0x7fbc24140280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.889+0000 7fbc2c000640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc24140280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800b3f0 tx=0x7fbc1800b8b0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.889+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 shutdown_connections 2024-04-25T11:27:00.881 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.889+0000 7fbc2c000640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbc24143660 0x7fbc24159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.889+0000 7fbc2c000640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc240b56e0 0x7fbc2413eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.889+0000 7fbc2c000640 1 --2- 172.21.15.169:0/3414762412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbc2406d5d0 0x7fbc24140280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.889+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 >> 172.21.15.169:0/3414762412 conn(0x7fbc24073120 msgr2=0x7fbc240b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.890+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 shutdown_connections 2024-04-25T11:27:00.882 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:00.890+0000 7fbc2c000640 1 -- 172.21.15.169:0/3414762412 wait complete. 2024-04-25T11:27:00.903 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c 2 2024-04-25T11:27:01.030 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:01.039+0000 7fcc3b63d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:27:01.139 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:01.147+0000 7fcc3b63d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:27:01.141 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.150+0000 7f7cbf000640 1 Processor -- start 2024-04-25T11:27:01.141 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.150+0000 7f7cbf000640 1 -- start start 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.152+0000 7f7cbf000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7cb80b93b0 0x7f7cb81603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.152+0000 7f7cbf000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7cb80d0ef0 0x7f7cb80d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.152+0000 7f7cbf000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb8164070 con 0x7f7cb806e030 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.152+0000 7f7cbf000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb8164350 con 0x7f7cb80b93b0 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.152+0000 7f7cbf000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb8164630 con 0x7f7cb80d0ef0 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cafe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7cb806e030 0x7f7cb806e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44970/0 (socket says 172.21.15.169:44970) 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cafe00640 1 -- 172.21.15.169:0/2519505385 learned_addr learned my addr 172.21.15.169:0/2519505385 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T11:27:01.145 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cbd000640 1 --2- 172.21.15.169:0/2519505385 >> v2:172.21.15.169:3300/0 conn(0x7f7cb80d0ef0 0x7f7cb80d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cb7e00640 1 --2- 172.21.15.169:0/2519505385 >> v2:172.21.15.19:3300/0 conn(0x7f7cb80b93b0 0x7f7cb81603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cb7400640 1 -- 172.21.15.169:0/2519505385 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317348516 0 0) 0x7f7cb8164070 con 0x7f7cb806e030 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cb7400640 1 -- 172.21.15.169:0/2519505385 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c9c003520 con 0x7f7cb806e030 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cbd000640 1 -- 172.21.15.169:0/2519505385 >> v2:172.21.15.19:3300/0 conn(0x7f7cb80b93b0 msgr2=0x7f7cb81603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cbd000640 1 --2- 172.21.15.169:0/2519505385 >> v2:172.21.15.19:3300/0 conn(0x7f7cb80b93b0 0x7f7cb81603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cbd000640 1 -- 172.21.15.169:0/2519505385 >> v1:172.21.15.169:6789/0 conn(0x7f7cb806e030 legacy=0x7f7cb806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cbd000640 1 -- 172.21.15.169:0/2519505385 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb8164910 con 0x7f7cb80d0ef0 2024-04-25T11:27:01.146 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.153+0000 7f7cb7e00640 1 --2- 172.21.15.169:0/2519505385 >> v2:172.21.15.19:3300/0 conn(0x7f7cb80b93b0 0x7f7cb81603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:01.147 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.154+0000 7f7cbd000640 1 --2- 172.21.15.169:0/2519505385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb80d0ef0 0x7f7cb80d1300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80d5cb0 tx=0x7f7cb002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17e7f6889c610f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:01.147 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.154+0000 7f7cb7400640 1 -- 172.21.15.169:0/2519505385 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0007740 con 0x7f7cb80d0ef0 2024-04-25T11:27:01.147 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.154+0000 7f7cb7400640 1 -- 172.21.15.169:0/2519505385 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb00078e0 con 0x7f7cb80d0ef0 2024-04-25T11:27:01.147 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.155+0000 7f7cb7400640 1 -- 172.21.15.169:0/2519505385 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0007c90 con 0x7f7cb80d0ef0 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 -- 172.21.15.169:0/2519505385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb80d0ef0 msgr2=0x7f7cb80d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 --2- 172.21.15.169:0/2519505385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb80d0ef0 0x7f7cb80d1300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80d5cb0 tx=0x7f7cb002a5f0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 -- 172.21.15.169:0/2519505385 shutdown_connections 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 --2- 172.21.15.169:0/2519505385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb80d0ef0 0x7f7cb80d1300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 --2- 172.21.15.169:0/2519505385 >> v2:172.21.15.19:3300/0 conn(0x7f7cb80b93b0 0x7f7cb81603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 -- 172.21.15.169:0/2519505385 >> 172.21.15.169:0/2519505385 conn(0x7f7cb80d2780 msgr2=0x7f7cb80d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 -- 172.21.15.169:0/2519505385 shutdown_connections 2024-04-25T11:27:01.148 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.156+0000 7f7cbf000640 1 -- 172.21.15.169:0/2519505385 wait complete. 2024-04-25T11:27:01.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.158+0000 7f7cbf000640 1 Processor -- start 2024-04-25T11:27:01.150 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.158+0000 7f7cbf000640 1 -- start start 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cbf000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb806e030 0x7f7cb815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cbf000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cb80b93b0 0x7f7cb814a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cbf000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb80d0ef0 0x7f7cb8158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cbf000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb80d19c0 con 0x7f7cb80b93b0 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cbf000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb80d1840 con 0x7f7cb806e030 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cbf000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb80d49b0 con 0x7f7cb80d0ef0 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cb7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb806e030 0x7f7cb815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cb7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb806e030 0x7f7cb815a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57482/0 (socket says 172.21.15.169:57482) 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cb7e00640 1 -- 172.21.15.169:0/1393701899 learned_addr learned my addr 172.21.15.169:0/1393701899 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.160+0000 7f7cbd000640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb80d0ef0 0x7f7cb8158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cafe00640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cb80b93b0 0x7f7cb814a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb7e00640 1 -- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb80d0ef0 msgr2=0x7f7cb8158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb7e00640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb80d0ef0 0x7f7cb8158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb7e00640 1 -- 172.21.15.169:0/1393701899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cb80b93b0 msgr2=0x7f7cb814a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb7e00640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cb80b93b0 0x7f7cb814a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb7e00640 1 -- 172.21.15.169:0/1393701899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb80ce460 con 0x7f7cb806e030 2024-04-25T11:27:01.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cbd000640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb80d0ef0 0x7f7cb8158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:01.154 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb7e00640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb806e030 0x7f7cb815a9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800d7d0 tx=0x7f7ca800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:01.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb4c00640 1 -- 172.21.15.169:0/1393701899 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800b6c0 con 0x7f7cb806e030 2024-04-25T11:27:01.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb4c00640 1 -- 172.21.15.169:0/1393701899 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca8019070 con 0x7f7cb806e030 2024-04-25T11:27:01.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cb4c00640 1 -- 172.21.15.169:0/1393701899 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800baa0 con 0x7f7cb806e030 2024-04-25T11:27:01.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.161+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb80ce740 con 0x7f7cb806e030 2024-04-25T11:27:01.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.162+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb80ced00 con 0x7f7cb806e030 2024-04-25T11:27:01.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.162+0000 7f7cb4c00640 1 -- 172.21.15.169:0/1393701899 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ca800bc40 con 0x7f7cb806e030 2024-04-25T11:27:01.155 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.162+0000 7f7cb4c00640 1 -- 172.21.15.169:0/1393701899 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7ca80148e0 con 0x7f7cb806e030 2024-04-25T11:27:01.156 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.163+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb8050740 con 0x7f7cb806e030 2024-04-25T11:27:01.160 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.168+0000 7f7cb4c00640 1 -- 172.21.15.169:0/1393701899 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7ca8014480 con 0x7f7cb806e030 2024-04-25T11:27:01.196 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:01.204+0000 7fcc3b63d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:27:01.222 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:01.230+0000 7f16f4d1c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:27:01.235 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.243+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c", "id": 2} v 0) -- 0x7f7cb80b5900 con 0x7f7cb806e030 2024-04-25T11:27:01.242 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.251+0000 7f7cb4c00640 1 -- 172.21.15.169:0/1393701899 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7ca8014b80 con 0x7f7cb806e030 2024-04-25T11:27:01.243 INFO:teuthology.orchestra.run.smithi169.stdout:2 2024-04-25T11:27:01.243 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb806e030 msgr2=0x7f7cb815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.243 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb806e030 0x7f7cb815a9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800d7d0 tx=0x7f7ca800dc90 comp rx=0 tx=0).stop 2024-04-25T11:27:01.244 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 shutdown_connections 2024-04-25T11:27:01.244 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb80d0ef0 0x7f7cb8158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.244 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cb80b93b0 0x7f7cb814a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.244 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 --2- 172.21.15.169:0/1393701899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb806e030 0x7f7cb815a9d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.244 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 >> 172.21.15.169:0/1393701899 conn(0x7f7cb80d2780 msgr2=0x7f7cb8163620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:01.245 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 shutdown_connections 2024-04-25T11:27:01.245 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.252+0000 7f7cbf000640 1 -- 172.21.15.169:0/1393701899 wait complete. 2024-04-25T11:27:01.266 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 73047778-1fdc-4988-90c3-b714ba889607 3 2024-04-25T11:27:01.332 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:01.341+0000 7f16f4d1c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:27:01.386 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:01.394+0000 7f16f4d1c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:27:01.399 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:01.407+0000 7fcc3b63d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:27:01.466 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:01.474+0000 7fcc3b63d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:27:01.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.519+0000 7fa0fda00640 1 Processor -- start 2024-04-25T11:27:01.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.519+0000 7fa0fda00640 1 -- start start 2024-04-25T11:27:01.513 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.521+0000 7fa0fda00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0f80d3d40 0x7fa0f8160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.513 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.521+0000 7fa0fda00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f80b9110 con 0x7fa0f80b7c50 2024-04-25T11:27:01.513 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.521+0000 7fa0fda00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f80d4cf0 con 0x7fa0f80d4510 2024-04-25T11:27:01.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.522+0000 7fa0fda00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f80d5590 con 0x7fa0f80d3d40 2024-04-25T11:27:01.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.522+0000 7fa0f7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0f80d3d40 0x7fa0f8160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.522+0000 7fa0f7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0f80d3d40 0x7fa0f8160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57496/0 (socket says 172.21.15.169:57496) 2024-04-25T11:27:01.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.522+0000 7fa0f7e00640 1 -- 172.21.15.169:0/3048940978 learned_addr learned my addr 172.21.15.169:0/3048940978 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:01.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.522+0000 7fa0f7e00640 1 -- 172.21.15.169:0/3048940978 >> v1:172.21.15.169:6789/0 conn(0x7fa0f80d4510 legacy=0x7fa0f80b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.522+0000 7fa0f7e00640 1 -- 172.21.15.169:0/3048940978 >> v1:172.21.15.19:6789/0 conn(0x7fa0f80b7c50 legacy=0x7fa0f80b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.522+0000 7fa0f7e00640 1 -- 172.21.15.169:0/3048940978 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f80d4fd0 con 0x7fa0f80d3d40 2024-04-25T11:27:01.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.523+0000 7fa0f7e00640 1 --2- 172.21.15.169:0/3048940978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8160290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009870 tx=0x7fa0e802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96e75121fda20d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:01.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.524+0000 7fa0f7400640 1 -- 172.21.15.169:0/3048940978 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8037070 con 0x7fa0f80d3d40 2024-04-25T11:27:01.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.524+0000 7fa0f7400640 1 -- 172.21.15.169:0/3048940978 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e8032440 con 0x7fa0f80d3d40 2024-04-25T11:27:01.516 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.524+0000 7fa0f7400640 1 -- 172.21.15.169:0/3048940978 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8032810 con 0x7fa0f80d3d40 2024-04-25T11:27:01.516 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.525+0000 7fa0fda00640 1 -- 172.21.15.169:0/3048940978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d3d40 msgr2=0x7fa0f8160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.525+0000 7fa0fda00640 1 --2- 172.21.15.169:0/3048940978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8160290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009870 tx=0x7fa0e802a210 comp rx=0 tx=0).stop 2024-04-25T11:27:01.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.525+0000 7fa0fda00640 1 -- 172.21.15.169:0/3048940978 shutdown_connections 2024-04-25T11:27:01.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.525+0000 7fa0fda00640 1 --2- 172.21.15.169:0/3048940978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8160290 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.525+0000 7fa0fda00640 1 -- 172.21.15.169:0/3048940978 >> 172.21.15.169:0/3048940978 conn(0x7fa0f80d2540 msgr2=0x7fa0f80d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:01.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.525+0000 7fa0fda00640 1 -- 172.21.15.169:0/3048940978 shutdown_connections 2024-04-25T11:27:01.517 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.525+0000 7fa0fda00640 1 -- 172.21.15.169:0/3048940978 wait complete. 2024-04-25T11:27:01.519 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.527+0000 7fa0fda00640 1 Processor -- start 2024-04-25T11:27:01.519 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.527+0000 7fa0fda00640 1 -- start start 2024-04-25T11:27:01.522 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.522 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fda00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d4510 0x7fa0f8156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.522 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fda00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0f80cb260 0x7fa0f80cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.522 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fda00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80d4920 con 0x7fa0f80d3d40 2024-04-25T11:27:01.522 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fda00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80b93b0 con 0x7fa0f80d4510 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fda00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f80d4aa0 con 0x7fa0f80cb260 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0f80cb260 0x7fa0f80cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0efe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d4510 0x7fa0f8156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0f80cb260 0x7fa0f80cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41476/0 (socket says 172.21.15.169:41476) 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0fd000640 1 -- 172.21.15.169:0/1125537134 learned_addr learned my addr 172.21.15.169:0/1125537134 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.529+0000 7fa0f7e00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0efe00640 1 -- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0f80cb260 msgr2=0x7fa0f80cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0efe00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0f80cb260 0x7fa0f80cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0efe00640 1 -- 172.21.15.169:0/1125537134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d3d40 msgr2=0x7fa0f8152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0efe00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0efe00640 1 -- 172.21.15.169:0/1125537134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f80cbbb0 con 0x7fa0f80d4510 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0f7e00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:01.523 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0fd000640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0f80cb260 0x7fa0f80cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0efe00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d4510 0x7fa0f8156600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000c3c0 tx=0x7fa0e000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0f4c00640 1 -- 172.21.15.169:0/1125537134 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0017070 con 0x7fa0f80d4510 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0f4c00640 1 -- 172.21.15.169:0/1125537134 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e00052d0 con 0x7fa0f80d4510 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f8153160 con 0x7fa0f80d4510 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0f4c00640 1 -- 172.21.15.169:0/1125537134 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0003810 con 0x7fa0f80d4510 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f80cd0a0 con 0x7fa0f80d4510 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.530+0000 7fa0f4c00640 1 -- 172.21.15.169:0/1125537134 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa0e001a070 con 0x7fa0f80d4510 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.531+0000 7fa0f4c00640 1 -- 172.21.15.169:0/1125537134 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa0e0011cc0 con 0x7fa0f80d4510 2024-04-25T11:27:01.524 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.532+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f8076c60 con 0x7fa0f80d4510 2024-04-25T11:27:01.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.534+0000 7fa0f4c00640 1 -- 172.21.15.169:0/1125537134 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa0e0021020 con 0x7fa0f80d4510 2024-04-25T11:27:01.535 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:01.544+0000 7fcc3b63d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:27:01.587 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:01.595+0000 7f16f4d1c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:27:01.613 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.621+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "73047778-1fdc-4988-90c3-b714ba889607", "id": 3} v 0) -- 0x7fa0f80b8a30 con 0x7fa0f80d4510 2024-04-25T11:27:01.621 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.629+0000 7fa0f4c00640 1 -- 172.21.15.169:0/1125537134 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73047778-1fdc-4988-90c3-b714ba889607", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa0f80cbbb0 con 0x7fa0f80d4510 2024-04-25T11:27:01.622 INFO:teuthology.orchestra.run.smithi169.stdout:3 2024-04-25T11:27:01.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d4510 msgr2=0x7fa0f8156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d4510 0x7fa0f8156600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000c3c0 tx=0x7fa0e000c880 comp rx=0 tx=0).stop 2024-04-25T11:27:01.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 shutdown_connections 2024-04-25T11:27:01.624 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0f80d4510 0x7fa0f8156600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0f80cb260 0x7fa0f80cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 --2- 172.21.15.169:0/1125537134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d3d40 0x7fa0f8152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 >> 172.21.15.169:0/1125537134 conn(0x7fa0f80d2540 msgr2=0x7fa0f815d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:01.625 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 shutdown_connections 2024-04-25T11:27:01.626 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.631+0000 7fa0fda00640 1 -- 172.21.15.169:0/1125537134 wait complete. 2024-04-25T11:27:01.647 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new de754db2-fb58-454c-a7f6-8ede347e3b05 4 2024-04-25T11:27:01.656 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:01.664+0000 7f16f4d1c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:27:01.678 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:01.686+0000 7fcc3b63d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:27:01.721 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:01.729+0000 7f16f4d1c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:27:01.855 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:01.864+0000 7f16f4d1c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:27:01.903 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.911+0000 7f5cff200640 1 Processor -- start 2024-04-25T11:27:01.904 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.912+0000 7f5cff200640 1 -- start start 2024-04-25T11:27:01.906 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cff200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d000d24d0 0x7f5d000d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cff200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d000b7640 0x7f5d000b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cff200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d00163ce0 con 0x7f5d000b7f90 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cff200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5d001612a0 con 0x7f5d000b7640 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cff200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5d00161500 con 0x7f5d000d24d0 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cfde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d000d24d0 0x7f5d000d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cfe800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5d000b7f90 0x7f5d000d3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44990/0 (socket says 172.21.15.169:44990) 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cfde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d000d24d0 0x7f5d000d28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57520/0 (socket says 172.21.15.169:57520) 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cfe800640 1 -- 172.21.15.169:0/2923163414 learned_addr learned my addr 172.21.15.169:0/2923163414 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.914+0000 7f5cf7e00640 1 --2- 172.21.15.169:0/2923163414 >> v2:172.21.15.19:3300/0 conn(0x7f5d000b7640 0x7f5d000b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.915+0000 7f5cfde00640 1 -- 172.21.15.169:0/2923163414 >> v2:172.21.15.19:3300/0 conn(0x7f5d000b7640 msgr2=0x7f5d000b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.915+0000 7f5cfde00640 1 --2- 172.21.15.169:0/2923163414 >> v2:172.21.15.19:3300/0 conn(0x7f5d000b7640 0x7f5d000b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.915+0000 7f5cfde00640 1 -- 172.21.15.169:0/2923163414 >> v1:172.21.15.169:6789/0 conn(0x7f5d000b7f90 legacy=0x7f5d000d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.907 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.915+0000 7f5cfde00640 1 -- 172.21.15.169:0/2923163414 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d00163fa0 con 0x7f5d000d24d0 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.916+0000 7f5cfde00640 1 --2- 172.21.15.169:0/2923163414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d000d28e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d000b8710 tx=0x7f5cf002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231c48b99f676ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.916+0000 7f5cfd400640 1 -- 172.21.15.169:0/2923163414 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0037070 con 0x7f5d000d24d0 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.916+0000 7f5cfd400640 1 -- 172.21.15.169:0/2923163414 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf0032440 con 0x7f5d000d24d0 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.916+0000 7f5cfd400640 1 -- 172.21.15.169:0/2923163414 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0032810 con 0x7f5d000d24d0 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 -- 172.21.15.169:0/2923163414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 msgr2=0x7f5d000d28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 --2- 172.21.15.169:0/2923163414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d000d28e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d000b8710 tx=0x7f5cf002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 -- 172.21.15.169:0/2923163414 shutdown_connections 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 --2- 172.21.15.169:0/2923163414 >> v2:172.21.15.19:3300/0 conn(0x7f5d000b7640 0x7f5d000b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 --2- 172.21.15.169:0/2923163414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d000d28e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 -- 172.21.15.169:0/2923163414 >> 172.21.15.169:0/2923163414 conn(0x7f5d0006c5c0 msgr2=0x7f5d0006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 -- 172.21.15.169:0/2923163414 shutdown_connections 2024-04-25T11:27:01.909 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.917+0000 7f5cff200640 1 -- 172.21.15.169:0/2923163414 wait complete. 2024-04-25T11:27:01.911 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.919+0000 7f5cff200640 1 Processor -- start 2024-04-25T11:27:01.911 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.919+0000 7f5cff200640 1 -- start start 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cff200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d000b7640 0x7f5d001f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d000b7f90 0x7f5d0015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cff200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d0015ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cff200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d000d2e20 con 0x7f5d000b7640 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cff200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00164560 con 0x7f5d000d24d0 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d000b83a0 con 0x7f5d000b7f90 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cfe800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d0015ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d000b7f90 0x7f5d0015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.914 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cfe800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d0015ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57530/0 (socket says 172.21.15.169:57530) 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d000b7f90 0x7f5d0015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41492/0 (socket says 172.21.15.169:41492) 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cfe800640 1 -- 172.21.15.169:0/2045574297 learned_addr learned my addr 172.21.15.169:0/2045574297 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.921+0000 7f5cfde00640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d000b7640 0x7f5d001f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cfe800640 1 -- 172.21.15.169:0/2045574297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d000b7f90 msgr2=0x7f5d0015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cfe800640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d000b7f90 0x7f5d0015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cfe800640 1 -- 172.21.15.169:0/2045574297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d000b7640 msgr2=0x7f5d001f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cfe800640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d000b7640 0x7f5d001f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cfe800640 1 -- 172.21.15.169:0/2045574297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d00153370 con 0x7f5d000d24d0 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cfe800640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d0015ead0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800caa0 tx=0x7f5cf800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cf6000640 1 -- 172.21.15.169:0/2045574297 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8017070 con 0x7f5d000d24d0 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cf6000640 1 -- 172.21.15.169:0/2045574297 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf800ac00 con 0x7f5d000d24d0 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cf6000640 1 -- 172.21.15.169:0/2045574297 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8012400 con 0x7f5d000d24d0 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d000ce460 con 0x7f5d000d24d0 2024-04-25T11:27:01.915 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.922+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0015dd90 con 0x7f5d000d24d0 2024-04-25T11:27:01.916 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.923+0000 7f5cf6000640 1 -- 172.21.15.169:0/2045574297 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5cf801a070 con 0x7f5d000d24d0 2024-04-25T11:27:01.916 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.923+0000 7f5cf6000640 1 -- 172.21.15.169:0/2045574297 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5cf801cd00 con 0x7f5d000d24d0 2024-04-25T11:27:01.916 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.923+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d001e9e50 con 0x7f5d000d24d0 2024-04-25T11:27:01.917 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:01.926+0000 7f5cf6000640 1 -- 172.21.15.169:0/2045574297 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5cf801c7b0 con 0x7f5d000d24d0 2024-04-25T11:27:01.995 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.004+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "de754db2-fb58-454c-a7f6-8ede347e3b05", "id": 4} v 0) -- 0x7f5d00076c60 con 0x7f5d000d24d0 2024-04-25T11:27:02.003 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.011+0000 7f5cf6000640 1 -- 172.21.15.169:0/2045574297 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "de754db2-fb58-454c-a7f6-8ede347e3b05", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f5cf801c990 con 0x7f5d000d24d0 2024-04-25T11:27:02.003 INFO:teuthology.orchestra.run.smithi169.stdout:4 2024-04-25T11:27:02.005 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 msgr2=0x7f5d0015ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.005 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d0015ead0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800caa0 tx=0x7f5cf800cf60 comp rx=0 tx=0).stop 2024-04-25T11:27:02.005 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 shutdown_connections 2024-04-25T11:27:02.005 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d000d24d0 0x7f5d0015ead0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.005 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d000b7f90 0x7f5d0015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.005 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 --2- 172.21.15.169:0/2045574297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d000b7640 0x7f5d001f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.006 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 >> 172.21.15.169:0/2045574297 conn(0x7f5d0006c5c0 msgr2=0x7f5d0015d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:02.006 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 shutdown_connections 2024-04-25T11:27:02.006 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.013+0000 7f5cff200640 1 -- 172.21.15.169:0/2045574297 wait complete. 2024-04-25T11:27:02.027 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517 5 2024-04-25T11:27:02.162 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:02.170+0000 7fcc3b63d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:27:02.321 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.329+0000 7f5675c00640 1 Processor -- start 2024-04-25T11:27:02.321 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.329+0000 7f5675c00640 1 -- start start 2024-04-25T11:27:02.323 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f5675c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f567009ccc0 0x7f567006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.323 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f5675c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567006ddf0 0x7f56701646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.323 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f5675c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5670164c00 0x7f5670165010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.323 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f5675c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5670165550 con 0x7f567009ccc0 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f5675c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f56700d31d0 con 0x7f5670164c00 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f5675c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56700d34b0 con 0x7f567006ddf0 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f566f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567006ddf0 0x7f56701646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f5675200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5670164c00 0x7f5670165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f566f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567006ddf0 0x7f56701646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41498/0 (socket says 172.21.15.169:41498) 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f566f400640 1 -- 172.21.15.169:0/923512155 learned_addr learned my addr 172.21.15.169:0/923512155 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.331+0000 7f566fe00640 1 --2- 172.21.15.169:0/923512155 >> v2:172.21.15.19:3300/0 conn(0x7f567009ccc0 0x7f567006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.332+0000 7f566f400640 1 -- 172.21.15.169:0/923512155 >> v2:172.21.15.169:3300/0 conn(0x7f5670164c00 msgr2=0x7f5670165010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.332+0000 7f566f400640 1 --2- 172.21.15.169:0/923512155 >> v2:172.21.15.169:3300/0 conn(0x7f5670164c00 0x7f5670165010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.332+0000 7f566f400640 1 -- 172.21.15.169:0/923512155 >> v2:172.21.15.19:3300/0 conn(0x7f567009ccc0 msgr2=0x7f567006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.332+0000 7f566f400640 1 --2- 172.21.15.169:0/923512155 >> v2:172.21.15.19:3300/0 conn(0x7f567009ccc0 0x7f567006d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.332+0000 7f566f400640 1 -- 172.21.15.169:0/923512155 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56700d3790 con 0x7f567006ddf0 2024-04-25T11:27:02.324 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.332+0000 7f5675200640 1 --2- 172.21.15.169:0/923512155 >> v2:172.21.15.169:3300/0 conn(0x7f5670164c00 0x7f5670165010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.333+0000 7f566f400640 1 --2- 172.21.15.169:0/923512155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567006ddf0 0x7f56701646c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f567006d080 tx=0x7f565c02a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c133fc1f75adf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.333+0000 7f566ea00640 1 -- 172.21.15.169:0/923512155 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c037070 con 0x7f567006ddf0 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.333+0000 7f566ea00640 1 -- 172.21.15.169:0/923512155 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565c032440 con 0x7f567006ddf0 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.333+0000 7f566ea00640 1 -- 172.21.15.169:0/923512155 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c032810 con 0x7f567006ddf0 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 -- 172.21.15.169:0/923512155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567006ddf0 msgr2=0x7f56701646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 --2- 172.21.15.169:0/923512155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567006ddf0 0x7f56701646c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f567006d080 tx=0x7f565c02a140 comp rx=0 tx=0).stop 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 -- 172.21.15.169:0/923512155 shutdown_connections 2024-04-25T11:27:02.326 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 --2- 172.21.15.169:0/923512155 >> v2:172.21.15.169:3300/0 conn(0x7f5670164c00 0x7f5670165010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.327 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 --2- 172.21.15.169:0/923512155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567006ddf0 0x7f56701646c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.327 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 --2- 172.21.15.169:0/923512155 >> v2:172.21.15.19:3300/0 conn(0x7f567009ccc0 0x7f567006d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.327 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 -- 172.21.15.169:0/923512155 >> 172.21.15.169:0/923512155 conn(0x7f56700d0480 msgr2=0x7f56700d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:02.327 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 -- 172.21.15.169:0/923512155 shutdown_connections 2024-04-25T11:27:02.327 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.334+0000 7f5675c00640 1 -- 172.21.15.169:0/923512155 wait complete. 2024-04-25T11:27:02.328 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.336+0000 7f5675c00640 1 Processor -- start 2024-04-25T11:27:02.328 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.336+0000 7f5675c00640 1 -- start start 2024-04-25T11:27:02.331 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567006ddf0 0x7f56700cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.331 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567009ccc0 0x7f56700cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.331 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5670164c00 0x7f56700cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56700d1010 con 0x7f567009ccc0 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f56700b5220 con 0x7f567006ddf0 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f56700d1190 con 0x7f5670164c00 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5670164c00 0x7f56700cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f566fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567006ddf0 0x7f56700cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5670164c00 0x7f56700cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41504/0 (socket says 172.21.15.169:41504) 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f566fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567006ddf0 0x7f56700cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57554/0 (socket says 172.21.15.169:57554) 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f5675200640 1 -- 172.21.15.169:0/1972665224 learned_addr learned my addr 172.21.15.169:0/1972665224 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.338+0000 7f566f400640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567009ccc0 0x7f56700cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675200640 1 -- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567006ddf0 msgr2=0x7f56700cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675200640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567006ddf0 0x7f56700cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675200640 1 -- 172.21.15.169:0/1972665224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567009ccc0 msgr2=0x7f56700cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675200640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567009ccc0 0x7f56700cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675200640 1 -- 172.21.15.169:0/1972665224 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56700b41b0 con 0x7f5670164c00 2024-04-25T11:27:02.332 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f566fe00640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567006ddf0 0x7f56700cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675200640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5670164c00 0x7f56700cbaf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f566800efb0 tx=0x7f566800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5667e00640 1 -- 172.21.15.169:0/1972665224 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668018020 con 0x7f5670164c00 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5667e00640 1 -- 172.21.15.169:0/1972665224 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5668014480 con 0x7f5670164c00 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5667e00640 1 -- 172.21.15.169:0/1972665224 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668014850 con 0x7f5670164c00 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56700b4490 con 0x7f5670164c00 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.339+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56700b4a50 con 0x7f5670164c00 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.340+0000 7f5667e00640 1 -- 172.21.15.169:0/1972665224 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5668005d10 con 0x7f5670164c00 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.340+0000 7f5667e00640 1 -- 172.21.15.169:0/1972665224 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f566801de00 con 0x7f5670164c00 2024-04-25T11:27:02.333 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.340+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5670076c60 con 0x7f5670164c00 2024-04-25T11:27:02.334 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.343+0000 7f5667e00640 1 -- 172.21.15.169:0/1972665224 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5668022030 con 0x7f5670164c00 2024-04-25T11:27:02.340 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:02.349+0000 7f16f4d1c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:27:02.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.422+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517", "id": 5} v 0) -- 0x7f567009d1d0 con 0x7f5670164c00 2024-04-25T11:27:02.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.429+0000 7f5667e00640 1 -- 172.21.15.169:0/1972665224 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5668010040 con 0x7f5670164c00 2024-04-25T11:27:02.421 INFO:teuthology.orchestra.run.smithi169.stdout:5 2024-04-25T11:27:02.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5670164c00 msgr2=0x7f56700cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5670164c00 0x7f56700cbaf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f566800efb0 tx=0x7f566800c490 comp rx=0 tx=0).stop 2024-04-25T11:27:02.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 shutdown_connections 2024-04-25T11:27:02.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5670164c00 0x7f56700cbaf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567009ccc0 0x7f56700cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 --2- 172.21.15.169:0/1972665224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567006ddf0 0x7f56700cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 >> 172.21.15.169:0/1972665224 conn(0x7f56700d0480 msgr2=0x7f567009baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:02.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.430+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 shutdown_connections 2024-04-25T11:27:02.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.431+0000 7f5675c00640 1 -- 172.21.15.169:0/1972665224 wait complete. 2024-04-25T11:27:02.447 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new c44cb1d3-012a-4d38-beec-cd774e53493c 6 2024-04-25T11:27:02.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:27:02.468 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:27:02.476 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:02.484+0000 7fcc3b63d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:27:02.545 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:02.553+0000 7fcc3b63d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:27:02.654 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:02.662+0000 7f16f4d1c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:27:02.685 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:02.693+0000 7fcc3b63d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:27:02.694 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.703+0000 7f4418e00640 1 Processor -- start 2024-04-25T11:27:02.695 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.703+0000 7f4418e00640 1 -- start start 2024-04-25T11:27:02.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.705+0000 7f4418e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f441406e270 0x7f441416ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.705+0000 7f4418e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44140b53c0 con 0x7f44140d56c0 2024-04-25T11:27:02.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.705+0000 7f4418e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44140d37e0 con 0x7f4414160440 2024-04-25T11:27:02.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.705+0000 7f4418e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f44140d43f0 con 0x7f441406e270 2024-04-25T11:27:02.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4412a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4414160440 0x7f441406db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:44996/0 (socket says 172.21.15.169:44996) 2024-04-25T11:27:02.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4412a00640 1 -- 172.21.15.169:0/1728299267 learned_addr learned my addr 172.21.15.169:0/1728299267 (peer_addr_for_me v1:172.21.15.169:0/0) 2024-04-25T11:27:02.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4413e00640 1 --2- 172.21.15.169:0/1728299267 >> v2:172.21.15.169:3300/0 conn(0x7f441406e270 0x7f441416ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4412000640 1 -- 172.21.15.169:0/1728299267 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2431524596 0 0) 0x7f44140d37e0 con 0x7f4414160440 2024-04-25T11:27:02.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4412000640 1 -- 172.21.15.169:0/1728299267 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43f8003520 con 0x7f4414160440 2024-04-25T11:27:02.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4413e00640 1 -- 172.21.15.169:0/1728299267 >> v1:172.21.15.169:6789/0 conn(0x7f4414160440 legacy=0x7f441406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4413e00640 1 -- 172.21.15.169:0/1728299267 >> v1:172.21.15.19:6789/0 conn(0x7f44140d56c0 legacy=0x7f44140b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.706+0000 7f4413e00640 1 -- 172.21.15.169:0/1728299267 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44140d3ac0 con 0x7f441406e270 2024-04-25T11:27:02.699 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.707+0000 7f4413e00640 1 --2- 172.21.15.169:0/1728299267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441406e270 0x7f441416ae80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f440c009d90 tx=0x7f440c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e91108a31b1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:02.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.708+0000 7f4412000640 1 -- 172.21.15.169:0/1728299267 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c007740 con 0x7f441406e270 2024-04-25T11:27:02.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.708+0000 7f4412000640 1 -- 172.21.15.169:0/1728299267 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c0078e0 con 0x7f441406e270 2024-04-25T11:27:02.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.708+0000 7f4412000640 1 -- 172.21.15.169:0/1728299267 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c007cb0 con 0x7f441406e270 2024-04-25T11:27:02.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.708+0000 7f4418e00640 1 -- 172.21.15.169:0/1728299267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441406e270 msgr2=0x7f441416ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.708+0000 7f4418e00640 1 --2- 172.21.15.169:0/1728299267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441406e270 0x7f441416ae80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f440c009d90 tx=0x7f440c02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.709+0000 7f4418e00640 1 -- 172.21.15.169:0/1728299267 shutdown_connections 2024-04-25T11:27:02.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.709+0000 7f4418e00640 1 --2- 172.21.15.169:0/1728299267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f441406e270 0x7f441416ae80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.709+0000 7f4418e00640 1 -- 172.21.15.169:0/1728299267 >> 172.21.15.169:0/1728299267 conn(0x7f44140d2c20 msgr2=0x7f44140d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:02.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.709+0000 7f4418e00640 1 -- 172.21.15.169:0/1728299267 shutdown_connections 2024-04-25T11:27:02.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.709+0000 7f4418e00640 1 -- 172.21.15.169:0/1728299267 wait complete. 2024-04-25T11:27:02.703 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.711+0000 7f4418e00640 1 Processor -- start 2024-04-25T11:27:02.703 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.711+0000 7f4418e00640 1 -- start start 2024-04-25T11:27:02.705 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4418e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4414160440 0x7f44141f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.705 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4418e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44140cbfb0 0x7f441414ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.705 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4418e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44141559a0 0x7f441415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:02.705 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4418e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44140ba270 con 0x7f44140cbfb0 2024-04-25T11:27:02.705 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4418e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44140d4080 con 0x7f4414160440 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4418e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44140d4880 con 0x7f44141559a0 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4414160440 0x7f44141f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44141559a0 0x7f441415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4414160440 0x7f44141f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57580/0 (socket says 172.21.15.169:57580) 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413400640 1 -- 172.21.15.169:0/2532562286 learned_addr learned my addr 172.21.15.169:0/2532562286 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4412a00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44140cbfb0 0x7f441414ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413e00640 1 -- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4414160440 msgr2=0x7f44141f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413e00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4414160440 0x7f44141f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413e00640 1 -- 172.21.15.169:0/2532562286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44140cbfb0 msgr2=0x7f441414ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413e00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44140cbfb0 0x7f441414ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413e00640 1 -- 172.21.15.169:0/2532562286 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441415a090 con 0x7f44141559a0 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.713+0000 7f4413400640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4414160440 0x7f44141f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:02.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.714+0000 7f4413e00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44141559a0 0x7f441415ae50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f440c031000 tx=0x7f440c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:02.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.714+0000 7f440b400640 1 -- 172.21.15.169:0/2532562286 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c03f070 con 0x7f44141559a0 2024-04-25T11:27:02.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.714+0000 7f440b400640 1 -- 172.21.15.169:0/2532562286 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c031c00 con 0x7f44141559a0 2024-04-25T11:27:02.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.714+0000 7f440b400640 1 -- 172.21.15.169:0/2532562286 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c009450 con 0x7f44141559a0 2024-04-25T11:27:02.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.714+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441415a370 con 0x7f44141559a0 2024-04-25T11:27:02.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.714+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141f6600 con 0x7f44141559a0 2024-04-25T11:27:02.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.715+0000 7f440b400640 1 -- 172.21.15.169:0/2532562286 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f440c044460 con 0x7f44141559a0 2024-04-25T11:27:02.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.715+0000 7f440b400640 1 -- 172.21.15.169:0/2532562286 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f440c007840 con 0x7f44141559a0 2024-04-25T11:27:02.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.715+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44141f6880 con 0x7f44141559a0 2024-04-25T11:27:02.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.718+0000 7f440b400640 1 -- 172.21.15.169:0/2532562286 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f440c03b780 con 0x7f44141559a0 2024-04-25T11:27:02.718 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:02.727+0000 7f16f4d1c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:27:02.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.796+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c44cb1d3-012a-4d38-beec-cd774e53493c", "id": 6} v 0) -- 0x7f4414076c60 con 0x7f44141559a0 2024-04-25T11:27:02.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.804+0000 7f440b400640 1 -- 172.21.15.169:0/2532562286 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c44cb1d3-012a-4d38-beec-cd774e53493c", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f440c03b960 con 0x7f44141559a0 2024-04-25T11:27:02.796 INFO:teuthology.orchestra.run.smithi169.stdout:6 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44141559a0 msgr2=0x7f441415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44141559a0 0x7f441415ae50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f440c031000 tx=0x7f440c004a40 comp rx=0 tx=0).stop 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 shutdown_connections 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44141559a0 0x7f441415ae50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44140cbfb0 0x7f441414ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 --2- 172.21.15.169:0/2532562286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4414160440 0x7f44141f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 >> 172.21.15.169:0/2532562286 conn(0x7f44140d2c20 msgr2=0x7f44140b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 shutdown_connections 2024-04-25T11:27:02.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:02.806+0000 7f4418e00640 1 -- 172.21.15.169:0/2532562286 wait complete. 2024-04-25T11:27:02.822 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new e35356bf-5d6d-4f56-96a7-208752898bd9 7 2024-04-25T11:27:02.859 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:02.867+0000 7f16f4d1c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:27:03.027 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:03.035+0000 7fcc3b63d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:27:03.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.082+0000 7ff96fe00640 1 Processor -- start 2024-04-25T11:27:03.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.082+0000 7ff96fe00640 1 -- start start 2024-04-25T11:27:03.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.084+0000 7ff96fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff9700d24d0 0x7ff9700d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9700b8160 0x7ff970160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff970163ce0 con 0x7ff9700b7640 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff9700d4e70 con 0x7ff9700b8160 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff970161500 con 0x7ff9700d24d0 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff9700d24d0 0x7ff9700d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff9700d24d0 0x7ff9700d28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57590/0 (socket says 172.21.15.169:57590) 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96ea00640 1 -- 172.21.15.169:0/164882531 learned_addr learned my addr 172.21.15.169:0/164882531 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96ea00640 1 -- 172.21.15.169:0/164882531 >> v2:172.21.15.19:3300/0 conn(0x7ff9700b8160 msgr2=0x7ff970160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96f400640 1 --2- 172.21.15.169:0/164882531 >> v2:172.21.15.19:3300/0 conn(0x7ff9700b8160 0x7ff970160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96ea00640 1 --2- 172.21.15.169:0/164882531 >> v2:172.21.15.19:3300/0 conn(0x7ff9700b8160 0x7ff970160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96ea00640 1 -- 172.21.15.169:0/164882531 >> v1:172.21.15.169:6789/0 conn(0x7ff9700b7640 legacy=0x7ff9700b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.086+0000 7ff96ea00640 1 -- 172.21.15.169:0/164882531 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff970163fa0 con 0x7ff9700d24d0 2024-04-25T11:27:03.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.085+0000 7ff96e000640 1 -- 172.21.15.169:0/164882531 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3500649347 0 0) 0x7ff970163ce0 con 0x7ff9700b7640 2024-04-25T11:27:03.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.087+0000 7ff96ea00640 1 --2- 172.21.15.169:0/164882531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700d24d0 0x7ff9700d28e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9700b8710 tx=0x7ff96002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b64918d618edfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:03.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.087+0000 7ff96e000640 1 -- 172.21.15.169:0/164882531 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960037070 con 0x7ff9700d24d0 2024-04-25T11:27:03.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.087+0000 7ff96e000640 1 -- 172.21.15.169:0/164882531 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff960032440 con 0x7ff9700d24d0 2024-04-25T11:27:03.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.087+0000 7ff96e000640 1 -- 172.21.15.169:0/164882531 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960032810 con 0x7ff9700d24d0 2024-04-25T11:27:03.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 -- 172.21.15.169:0/164882531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700d24d0 msgr2=0x7ff9700d28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 --2- 172.21.15.169:0/164882531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700d24d0 0x7ff9700d28e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9700b8710 tx=0x7ff96002a270 comp rx=0 tx=0).stop 2024-04-25T11:27:03.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 -- 172.21.15.169:0/164882531 shutdown_connections 2024-04-25T11:27:03.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 --2- 172.21.15.169:0/164882531 >> v2:172.21.15.19:3300/0 conn(0x7ff9700b8160 0x7ff970160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 --2- 172.21.15.169:0/164882531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700d24d0 0x7ff9700d28e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 -- 172.21.15.169:0/164882531 >> 172.21.15.169:0/164882531 conn(0x7ff97006c5c0 msgr2=0x7ff97006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:03.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 -- 172.21.15.169:0/164882531 shutdown_connections 2024-04-25T11:27:03.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.088+0000 7ff96fe00640 1 -- 172.21.15.169:0/164882531 wait complete. 2024-04-25T11:27:03.082 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.090+0000 7ff96fe00640 1 Processor -- start 2024-04-25T11:27:03.082 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.090+0000 7ff96fe00640 1 -- start start 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9700b7640 0x7ff9700cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700b8160 0x7ff9701f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9700d24d0 0x7ff970160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700d2e20 con 0x7ff9700b7640 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff970164560 con 0x7ff9700b8160 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff970164a80 con 0x7ff9700d24d0 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff967e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700b8160 0x7ff9701f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9700d24d0 0x7ff970160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff967e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700b8160 0x7ff9701f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:57606/0 (socket says 172.21.15.169:57606) 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff967e00640 1 -- 172.21.15.169:0/4035709529 learned_addr learned my addr 172.21.15.169:0/4035709529 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:27:03.085 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.092+0000 7ff96f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9700d24d0 0x7ff970160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:41524/0 (socket says 172.21.15.169:41524) 2024-04-25T11:27:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff96ea00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9700b7640 0x7ff9700cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff967e00640 1 -- 172.21.15.169:0/4035709529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9700d24d0 msgr2=0x7ff970160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff967e00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9700d24d0 0x7ff970160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff967e00640 1 -- 172.21.15.169:0/4035709529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9700b7640 msgr2=0x7ff9700cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff967e00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9700b7640 0x7ff9700cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff967e00640 1 -- 172.21.15.169:0/4035709529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9700cdd60 con 0x7ff9700b8160 2024-04-25T11:27:03.086 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff967e00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700b8160 0x7ff9701f9000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c410 tx=0x7ff95800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:03.087 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff966a00640 1 -- 172.21.15.169:0/4035709529 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958017070 con 0x7ff9700b8160 2024-04-25T11:27:03.087 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff966a00640 1 -- 172.21.15.169:0/4035709529 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9580035f0 con 0x7ff9700b8160 2024-04-25T11:27:03.087 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff966a00640 1 -- 172.21.15.169:0/4035709529 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9580039e0 con 0x7ff9700b8160 2024-04-25T11:27:03.087 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.093+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9700ce040 con 0x7ff9700b8160 2024-04-25T11:27:03.088 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.094+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9700ce600 con 0x7ff9700b8160 2024-04-25T11:27:03.088 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.094+0000 7ff966a00640 1 -- 172.21.15.169:0/4035709529 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff95801a070 con 0x7ff9700b8160 2024-04-25T11:27:03.088 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.094+0000 7ff966a00640 1 -- 172.21.15.169:0/4035709529 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ff958012a60 con 0x7ff9700b8160 2024-04-25T11:27:03.088 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.095+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9700b6d10 con 0x7ff9700b8160 2024-04-25T11:27:03.088 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:03.095+0000 7fcc3b63d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:27:03.088 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.097+0000 7ff966a00640 1 -- 172.21.15.169:0/4035709529 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff9700b6d10 con 0x7ff9700b8160 2024-04-25T11:27:03.167 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.176+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e35356bf-5d6d-4f56-96a7-208752898bd9", "id": 7} v 0) -- 0x7ff970076c60 con 0x7ff9700b8160 2024-04-25T11:27:03.173 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.181+0000 7ff966a00640 1 -- 172.21.15.169:0/4035709529 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e35356bf-5d6d-4f56-96a7-208752898bd9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ff958021020 con 0x7ff9700b8160 2024-04-25T11:27:03.173 INFO:teuthology.orchestra.run.smithi169.stdout:7 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700b8160 msgr2=0x7ff9701f9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700b8160 0x7ff9701f9000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c410 tx=0x7ff95800c8d0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 shutdown_connections 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9700b8160 0x7ff9701f9000 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9700d24d0 0x7ff970160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 --2- 172.21.15.169:0/4035709529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9700b7640 0x7ff9700cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 >> 172.21.15.169:0/4035709529 conn(0x7ff97006c5c0 msgr2=0x7ff9700b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 shutdown_connections 2024-04-25T11:27:03.175 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:03.183+0000 7ff96fe00640 1 -- 172.21.15.169:0/4035709529 wait complete. 2024-04-25T11:27:03.198 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T11:27:03.198 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T11:27:03.206 INFO:tasks.ceph.osd.0:Started 2024-04-25T11:27:03.206 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T11:27:03.206 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T11:27:03.207 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:03.216+0000 7f16f4d1c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:27:03.213 INFO:tasks.ceph.osd.1:Started 2024-04-25T11:27:03.213 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T11:27:03.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T11:27:03.218 INFO:tasks.ceph.osd.2:Started 2024-04-25T11:27:03.218 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T11:27:03.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T11:27:03.224 INFO:tasks.ceph.osd.3:Started 2024-04-25T11:27:03.225 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T11:27:03.225 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T11:27:03.230 INFO:tasks.ceph.osd.4:Started 2024-04-25T11:27:03.230 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T11:27:03.230 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T11:27:03.236 INFO:tasks.ceph.osd.5:Started 2024-04-25T11:27:03.236 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T11:27:03.236 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T11:27:03.241 INFO:tasks.ceph.osd.6:Started 2024-04-25T11:27:03.241 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T11:27:03.241 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T11:27:03.246 INFO:tasks.ceph.osd.7:Started 2024-04-25T11:27:03.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:03.291 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:03.299+0000 7fcc3b63d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:27:03.293 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:03.301+0000 7f16f4d1c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:27:03.510 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:03.518+0000 7fcc3b63d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:27:03.528 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:03.537+0000 7f16f4d1c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:27:03.573 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:03.582+0000 7fcc3b63d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:27:03.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.595+0000 7f79f1600640 1 Processor -- start 2024-04-25T11:27:03.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.595+0000 7f79f1600640 1 -- start start 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.596+0000 7f79f1600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b8290 0x7f79ec160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.596+0000 7f79f1600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79ec15df80 0x7f79ec160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.596+0000 7f79f1600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79ec1615f0 con 0x7f79ec164b00 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.596+0000 7f79f1600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f79ec164670 con 0x7f79ec15df80 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.596+0000 7f79f1600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79ec164020 con 0x7f79ec0b8290 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79eb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79ec15df80 0x7f79ec160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79e3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b8290 0x7f79ec160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79eb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79ec15df80 0x7f79ec160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49574/0 (socket says 172.21.15.19:49574) 2024-04-25T11:27:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79eb400640 1 -- 172.21.15.19:0/1173629070 learned_addr learned my addr 172.21.15.19:0/1173629070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79eb400640 1 -- 172.21.15.19:0/1173629070 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b8290 msgr2=0x7f79ec160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79eb400640 1 --2- 172.21.15.19:0/1173629070 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b8290 0x7f79ec160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79eb400640 1 -- 172.21.15.19:0/1173629070 >> v1:172.21.15.169:6789/0 conn(0x7f79ec164b00 legacy=0x7f79ec0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.597+0000 7f79eb400640 1 -- 172.21.15.19:0/1173629070 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec164300 con 0x7f79ec15df80 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.598+0000 7f79eb400640 1 --2- 172.21.15.19:0/1173629070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec160dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0d2540 tx=0x7f79e402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5802f5c4f4a03ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.599+0000 7f79ea000640 1 -- 172.21.15.19:0/1173629070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4036070 con 0x7f79ec15df80 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.599+0000 7f79ea000640 1 -- 172.21.15.19:0/1173629070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e4031600 con 0x7f79ec15df80 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.599+0000 7f79ea000640 1 -- 172.21.15.19:0/1173629070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40319d0 con 0x7f79ec15df80 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.599+0000 7f79f1600640 1 -- 172.21.15.19:0/1173629070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 msgr2=0x7f79ec160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.599+0000 7f79f1600640 1 --2- 172.21.15.19:0/1173629070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec160dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0d2540 tx=0x7f79e402a730 comp rx=0 tx=0).stop 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.600+0000 7f79f1600640 1 -- 172.21.15.19:0/1173629070 shutdown_connections 2024-04-25T11:27:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.600+0000 7f79f1600640 1 --2- 172.21.15.19:0/1173629070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec160dd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.600+0000 7f79f1600640 1 --2- 172.21.15.19:0/1173629070 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b8290 0x7f79ec160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.600+0000 7f79f1600640 1 -- 172.21.15.19:0/1173629070 >> 172.21.15.19:0/1173629070 conn(0x7f79ec0d2bc0 msgr2=0x7f79ec0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.600+0000 7f79f1600640 1 -- 172.21.15.19:0/1173629070 shutdown_connections 2024-04-25T11:27:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.600+0000 7f79f1600640 1 -- 172.21.15.19:0/1173629070 wait complete. 2024-04-25T11:27:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.602+0000 7f79f1600640 1 Processor -- start 2024-04-25T11:27:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.602+0000 7f79f1600640 1 -- start start 2024-04-25T11:27:03.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.603+0000 7f79f1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec1f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.603+0000 7f79f1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec164b00 0x7f79ec1fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.603+0000 7f79f1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec1fa650 0x7f79ec158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.603+0000 7f79f1600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec0b4a00 con 0x7f79ec15df80 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.603+0000 7f79f1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec0b4880 con 0x7f79ec1fa650 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.603+0000 7f79f1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79ec0b4b80 con 0x7f79ec164b00 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec1f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec1f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49576/0 (socket says 172.21.15.19:49576) 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 -- 172.21.15.19:0/2664441752 learned_addr learned my addr 172.21.15.19:0/2664441752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eb400640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec1fa650 0x7f79ec158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79e3e00640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec164b00 0x7f79ec1fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:03.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 -- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec164b00 msgr2=0x7f79ec1fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec164b00 0x7f79ec1fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 -- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec1fa650 msgr2=0x7f79ec158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec1fa650 0x7f79ec158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 -- 172.21.15.19:0/2664441752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec1faa60 con 0x7f79ec15df80 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79e3e00640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec164b00 0x7f79ec1fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.604+0000 7f79eaa00640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec1f62c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c3f0 tx=0x7f79dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.605+0000 7f79e2a00640 1 -- 172.21.15.19:0/2664441752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0052d0 con 0x7f79ec15df80 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.605+0000 7f79e2a00640 1 -- 172.21.15.19:0/2664441752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc005490 con 0x7f79ec15df80 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.605+0000 7f79e2a00640 1 -- 172.21.15.19:0/2664441752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc007260 con 0x7f79ec15df80 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.605+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec148c30 con 0x7f79ec15df80 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.605+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec157690 con 0x7f79ec15df80 2024-04-25T11:27:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.605+0000 7f79e2a00640 1 -- 172.21.15.19:0/2664441752 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f79dc007400 con 0x7f79ec15df80 2024-04-25T11:27:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.605+0000 7f79e2a00640 1 -- 172.21.15.19:0/2664441752 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f79dc003a00 con 0x7f79ec15df80 2024-04-25T11:27:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.606+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ec050740 con 0x7f79ec15df80 2024-04-25T11:27:03.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.611+0000 7f79e2a00640 1 -- 172.21.15.19:0/2664441752 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f79ec1faa60 con 0x7f79ec15df80 2024-04-25T11:27:03.656 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:03.664+0000 7fcc3b63d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.695+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79ec0d4170 con 0x7f79ec15df80 2024-04-25T11:27:03.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.696+0000 7f79e2a00640 1 -- 172.21.15.19:0/2664441752 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f79dc0035f0 con 0x7f79ec15df80 2024-04-25T11:27:03.688 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:03.688 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:03.178286+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 msgr2=0x7f79ec1f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec1f62c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c3f0 tx=0x7f79dc00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 shutdown_connections 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec1fa650 0x7f79ec158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec164b00 0x7f79ec1fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 --2- 172.21.15.19:0/2664441752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79ec15df80 0x7f79ec1f62c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 >> 172.21.15.19:0/2664441752 conn(0x7f79ec0d2bc0 msgr2=0x7f79ec15c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 shutdown_connections 2024-04-25T11:27:03.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:03.697+0000 7f79f1600640 1 -- 172.21.15.19:0/2664441752 wait complete. 2024-04-25T11:27:03.710 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T11:27:03.710 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T11:27:03.710 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:03.718+0000 7fcc3b63d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:27:03.728 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:03.737+0000 7f16f4d1c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:27:03.790 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:03.798+0000 7f16f4d1c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:27:03.876 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:03.884+0000 7f16f4d1c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:27:03.932 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:03.941+0000 7f16f4d1c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:27:03.996 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T11:27:04.005+0000 7f557d217640 -1 Falling back to public interface 2024-04-25T11:27:04.001 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T11:27:04.009+0000 7fc237019640 -1 Falling back to public interface 2024-04-25T11:27:04.006 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T11:27:04.015+0000 7f4dda615640 -1 Falling back to public interface 2024-04-25T11:27:04.017 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T11:27:04.026+0000 7ff49b526640 -1 Falling back to public interface 2024-04-25T11:27:04.034 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T11:27:04.042+0000 7f1d07e16640 -1 Falling back to public interface 2024-04-25T11:27:04.038 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T11:27:04.047+0000 7f600402f640 -1 Falling back to public interface 2024-04-25T11:27:04.059 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T11:27:04.068+0000 7f62b860c640 -1 Falling back to public interface 2024-04-25T11:27:04.063 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T11:27:04.071+0000 7ff100031640 -1 Falling back to public interface 2024-04-25T11:27:04.259 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:04.268+0000 7fcc3b63d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:27:04.395 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:04.403+0000 7fcc3b63d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:27:04.515 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:04.523 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:04.530+0000 7f16f4d1c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:27:04.664 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-04-25T11:27:04.673+0000 7f16f4d1c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:27:04.753 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.762+0000 7f1f18a00640 1 Processor -- start 2024-04-25T11:27:04.754 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.762+0000 7f1f18a00640 1 -- start start 2024-04-25T11:27:04.755 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.764+0000 7f1f18a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f140b8000 0x7f1f140d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:04.756 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.764+0000 7f1f18a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f140b8410 con 0x7f1f140d28d0 2024-04-25T11:27:04.756 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.764+0000 7f1f18a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f140d49a0 con 0x7f1f140b9760 2024-04-25T11:27:04.756 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.764+0000 7f1f18a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1f140d54e0 con 0x7f1f140b8000 2024-04-25T11:27:04.758 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f0aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1f140d28d0 0x7f1f140d2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45488/0 (socket says 172.21.15.19:45488) 2024-04-25T11:27:04.758 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f0aa00640 1 -- 172.21.15.19:0/4288090727 learned_addr learned my addr 172.21.15.19:0/4288090727 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:04.758 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f13400640 1 --2- 172.21.15.19:0/4288090727 >> v2:172.21.15.169:3300/0 conn(0x7f1f140b8000 0x7f1f140d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:04.758 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213558243 0 0) 0x7f1f140b8410 con 0x7f1f140d28d0 2024-04-25T11:27:04.758 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ef8003520 con 0x7f1f140d28d0 2024-04-25T11:27:04.758 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2514063940 0 0) 0x7f1ef8003520 con 0x7f1f140d28d0 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 >> v2:172.21.15.169:3300/0 conn(0x7f1f140b8000 msgr2=0x7f1f140d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 --2- 172.21.15.19:0/4288090727 >> v2:172.21.15.169:3300/0 conn(0x7f1f140b8000 0x7f1f140d4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 >> v1:172.21.15.169:6789/0 conn(0x7f1f140b9760 legacy=0x7f1f140b9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f140d4c40 con 0x7f1f140d28d0 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1efc001670 con 0x7f1f140d28d0 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.765+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 >> v1:172.21.15.19:6789/0 conn(0x7f1f140d28d0 legacy=0x7f1f140d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f12a00640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef8003a00 0x7f1ef8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f12a00640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef8004800 0x7f1ef8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f12a00640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef80052a0 0x7f1ef800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140d54e0 con 0x7f1ef8003a00 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef800c230 con 0x7f1ef8004800 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef800c430 con 0x7f1ef80052a0 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13400640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef8003a00 0x7f1ef8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:04.759 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f0aa00640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef8004800 0x7f1ef8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:04.760 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13e00640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef80052a0 0x7f1ef800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:04.760 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13400640 1 -- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef80052a0 msgr2=0x7f1ef800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:04.760 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13400640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef80052a0 0x7f1ef800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.760 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13400640 1 -- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef8004800 msgr2=0x7f1ef8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:04.760 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13400640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef8004800 0x7f1ef8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.760 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13400640 1 -- 172.21.15.19:0/4288090727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef8003520 con 0x7f1ef8003a00 2024-04-25T11:27:04.760 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.766+0000 7f1f13e00640 1 -- 172.21.15.19:0/4288090727 reap_dead start 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.767+0000 7f1f13400640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef8003a00 0x7f1ef8003e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009f90 tx=0x7f1ef8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a89e6499cb580cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04009c10 con 0x7f1ef8003a00 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f04036070 con 0x7f1ef8003a00 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f12a00640 1 -- 172.21.15.19:0/4288090727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04031810 con 0x7f1ef8003a00 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f18a00640 1 -- 172.21.15.19:0/4288090727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef8003a00 msgr2=0x7f1ef8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f18a00640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef8003a00 0x7f1ef8003e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009f90 tx=0x7f1ef8005830 comp rx=0 tx=0).stop 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f18a00640 1 -- 172.21.15.19:0/4288090727 shutdown_connections 2024-04-25T11:27:04.761 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f18a00640 1 --2- 172.21.15.19:0/4288090727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef8003a00 0x7f1ef8003e70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.762 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.768+0000 7f1f18a00640 1 -- 172.21.15.19:0/4288090727 >> 172.21.15.19:0/4288090727 conn(0x7f1f1406e030 msgr2=0x7f1f1406e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:04.762 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.769+0000 7f1f18a00640 1 -- 172.21.15.19:0/4288090727 shutdown_connections 2024-04-25T11:27:04.762 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.769+0000 7f1f18a00640 1 -- 172.21.15.19:0/4288090727 wait complete. 2024-04-25T11:27:04.762 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.771+0000 7f1f18a00640 1 Processor -- start 2024-04-25T11:27:04.762 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.771+0000 7f1f18a00640 1 -- start start 2024-04-25T11:27:04.765 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f18a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef8003a00 0x7f1f1414fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f18a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f141f5ff0 0x7f1f1414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f18a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f1414bfb0 0x7f1f140cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f18a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141603e0 con 0x7f1f141f5ff0 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f18a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140d5200 con 0x7f1f1414bfb0 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f18a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f14160560 con 0x7f1ef8003a00 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f141f5ff0 0x7f1f1414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f141f5ff0 0x7f1f1414ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38352/0 (socket says 172.21.15.19:38352) 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 -- 172.21.15.19:0/2859431794 learned_addr learned my addr 172.21.15.19:0/2859431794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f13400640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef8003a00 0x7f1f1414fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f13e00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f1414bfb0 0x7f1f140cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 -- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef8003a00 msgr2=0x7f1f1414fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef8003a00 0x7f1f1414fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.766 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 -- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f1414bfb0 msgr2=0x7f1f140cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f1414bfb0 0x7f1f140cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.773+0000 7f1f0aa00640 1 -- 172.21.15.19:0/2859431794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1414c3c0 con 0x7f1f141f5ff0 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.774+0000 7f1f13e00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f1414bfb0 0x7f1f140cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.774+0000 7f1f0aa00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f141f5ff0 0x7f1f1414ba70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e730 tx=0x7f1efc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.774+0000 7f1f0be00640 1 -- 172.21.15.19:0/2859431794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00c6c0 con 0x7f1f141f5ff0 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.774+0000 7f1f0be00640 1 -- 172.21.15.19:0/2859431794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1efc018070 con 0x7f1f141f5ff0 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.774+0000 7f1f0be00640 1 -- 172.21.15.19:0/2859431794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00caa0 con 0x7f1f141f5ff0 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.774+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f140cc410 con 0x7f1f141f5ff0 2024-04-25T11:27:04.767 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.774+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f140cc130 con 0x7f1f141f5ff0 2024-04-25T11:27:04.768 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.775+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f14050740 con 0x7f1f141f5ff0 2024-04-25T11:27:04.768 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.776+0000 7f1f0be00640 1 -- 172.21.15.19:0/2859431794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 2) ==== 48416+0+0 (secure 0 0 0) 0x7f1efc00cc40 con 0x7f1f141f5ff0 2024-04-25T11:27:04.768 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.776+0000 7f1f0be00640 1 -- 172.21.15.19:0/2859431794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1efc04f330 con 0x7f1f141f5ff0 2024-04-25T11:27:04.770 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.779+0000 7f1f0be00640 1 -- 172.21.15.19:0/2859431794 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1efc01ad20 con 0x7f1f141f5ff0 2024-04-25T11:27:04.835 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.844+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f140d62e0 con 0x7f1f141f5ff0 2024-04-25T11:27:04.836 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.845+0000 7f1f0be00640 1 -- 172.21.15.19:0/2859431794 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1efc013420 con 0x7f1f141f5ff0 2024-04-25T11:27:04.837 INFO:teuthology.misc.health.smithi019.stdout: 2024-04-25T11:27:04.837 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:03.178286+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f141f5ff0 msgr2=0x7f1f1414ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f141f5ff0 0x7f1f1414ba70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e730 tx=0x7f1efc00ebf0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 shutdown_connections 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f1414bfb0 0x7f1f140cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f141f5ff0 0x7f1f1414ba70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 --2- 172.21.15.19:0/2859431794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef8003a00 0x7f1f1414fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 >> 172.21.15.19:0/2859431794 conn(0x7f1f1406e030 msgr2=0x7f1f141607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 shutdown_connections 2024-04-25T11:27:04.838 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:04.846+0000 7f1f18a00640 1 -- 172.21.15.19:0/2859431794 wait complete. 2024-04-25T11:27:04.856 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T11:27:05.444 INFO:tasks.ceph.mgr.x.smithi169.stderr:2024-04-25T11:27:05.452+0000 7fcbdaa00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T11:27:07.385 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T11:27:07.393+0000 7fc237019640 -1 osd.5 0 log_to_monitors true 2024-04-25T11:27:07.580 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T11:27:07.588+0000 7f557d217640 -1 osd.7 0 log_to_monitors true 2024-04-25T11:27:07.646 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T11:27:07.654+0000 7ff49b526640 -1 osd.4 0 log_to_monitors true 2024-04-25T11:27:07.711 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T11:27:07.719+0000 7ff100031640 -1 osd.6 0 log_to_monitors true 2024-04-25T11:27:07.716 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T11:27:07.725+0000 7f600402f640 -1 osd.2 0 log_to_monitors true 2024-04-25T11:27:07.718 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T11:27:07.727+0000 7f1d07e16640 -1 osd.0 0 log_to_monitors true 2024-04-25T11:27:07.739 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T11:27:07.748+0000 7f62b860c640 -1 osd.3 0 log_to_monitors true 2024-04-25T11:27:07.814 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T11:27:07.822+0000 7f4dda615640 -1 osd.1 0 log_to_monitors true 2024-04-25T11:27:09.942 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T11:27:09.950+0000 7fc232200640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T11:27:10.031 INFO:tasks.ceph.osd.5.smithi169.stderr:2024-04-25T11:27:10.039+0000 7fc227800640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:11.372 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T11:27:11.380+0000 7ff0fae00640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T11:27:11.380 INFO:tasks.ceph.osd.6.smithi169.stderr:2024-04-25T11:27:11.389+0000 7ff0f0400640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:11.413 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T11:27:11.422+0000 7ff496400640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T11:27:11.422 INFO:tasks.ceph.osd.4.smithi169.stderr:2024-04-25T11:27:11.430+0000 7ff48c400640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:11.454 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T11:27:11.463+0000 7f5578600640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T11:27:11.462 INFO:tasks.ceph.osd.7.smithi169.stderr:2024-04-25T11:27:11.470+0000 7f556dc00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:11.662 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:11.936 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T11:27:11.945+0000 7f4dd4e00640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:27:11.948 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-04-25T11:27:11.956+0000 7f4dcae00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:11.985 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.993+0000 7f4d50800640 1 Processor -- start 2024-04-25T11:27:11.985 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.993+0000 7f4d50800640 1 -- start start 2024-04-25T11:27:11.987 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d50800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d4815ec80 0x7f4d48073170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:11.987 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d50800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d480736b0 0x7f4d480b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:11.987 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d50800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d480b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:11.987 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d50800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4d48073ac0 con 0x7f4d4815ec80 2024-04-25T11:27:11.988 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d50800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4d481682e0 con 0x7f4d480736b0 2024-04-25T11:27:11.988 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d50800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4d48168580 con 0x7f4d480b6220 2024-04-25T11:27:11.988 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d4de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d4815ec80 0x7f4d48073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:11.989 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d4de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4d4815ec80 0x7f4d48073170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38398/0 (socket says 172.21.15.19:38398) 2024-04-25T11:27:11.989 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d4de00640 1 -- 172.21.15.19:0/334214467 learned_addr learned my addr 172.21.15.19:0/334214467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:11.989 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d4d400640 1 --2- 172.21.15.19:0/334214467 >> v2:172.21.15.169:3300/0 conn(0x7f4d480736b0 0x7f4d480b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:11.989 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f62b2e00640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T11:27:11.990 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.996+0000 7f4d4e800640 1 --2- 172.21.15.19:0/334214467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d480b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:11.990 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f4d4de00640 1 -- 172.21.15.19:0/334214467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 msgr2=0x7f4d480b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f4d4de00640 1 --2- 172.21.15.19:0/334214467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d480b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f4d4de00640 1 -- 172.21.15.19:0/334214467 >> v2:172.21.15.169:3300/0 conn(0x7f4d480736b0 msgr2=0x7f4d480b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f4d4de00640 1 --2- 172.21.15.19:0/334214467 >> v2:172.21.15.169:3300/0 conn(0x7f4d480736b0 0x7f4d480b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f4d4de00640 1 -- 172.21.15.19:0/334214467 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d48168860 con 0x7f4d4815ec80 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f4d4e800640 1 --2- 172.21.15.19:0/334214467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d480b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.997+0000 7f4d4d400640 1 --2- 172.21.15.19:0/334214467 >> v2:172.21.15.169:3300/0 conn(0x7f4d480736b0 0x7f4d480b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.999+0000 7f4d4de00640 1 --2- 172.21.15.19:0/334214467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4815ec80 0x7f4d48073170 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4d480742c0 tx=0x7f4d3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7658d11da8e7b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.999+0000 7f4d4ca00640 1 -- 172.21.15.19:0/334214467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c037070 con 0x7f4d4815ec80 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.999+0000 7f4d4ca00640 1 -- 172.21.15.19:0/334214467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c032440 con 0x7f4d4815ec80 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:11.999+0000 7f4d4ca00640 1 -- 172.21.15.19:0/334214467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c032810 con 0x7f4d4815ec80 2024-04-25T11:27:11.991 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.000+0000 7f4d50800640 1 -- 172.21.15.19:0/334214467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4815ec80 msgr2=0x7f4d48073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:11.992 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.000+0000 7f4d50800640 1 --2- 172.21.15.19:0/334214467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4815ec80 0x7f4d48073170 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4d480742c0 tx=0x7f4d3c02a140 comp rx=0 tx=0).stop 2024-04-25T11:27:11.992 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.001+0000 7f4d50800640 1 -- 172.21.15.19:0/334214467 shutdown_connections 2024-04-25T11:27:11.992 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.001+0000 7f4d50800640 1 --2- 172.21.15.19:0/334214467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d480b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:11.992 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.001+0000 7f4d50800640 1 --2- 172.21.15.19:0/334214467 >> v2:172.21.15.169:3300/0 conn(0x7f4d480736b0 0x7f4d480b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:11.992 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.001+0000 7f4d50800640 1 --2- 172.21.15.19:0/334214467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4815ec80 0x7f4d48073170 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:11.993 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.001+0000 7f4d50800640 1 -- 172.21.15.19:0/334214467 >> 172.21.15.19:0/334214467 conn(0x7f4d4806cfe0 msgr2=0x7f4d480b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:11.993 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.001+0000 7f4d50800640 1 -- 172.21.15.19:0/334214467 shutdown_connections 2024-04-25T11:27:11.993 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.001+0000 7f4d50800640 1 -- 172.21.15.19:0/334214467 wait complete. 2024-04-25T11:27:11.995 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-04-25T11:27:12.003+0000 7f62a8e00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:11.996 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.004+0000 7f4d50800640 1 Processor -- start 2024-04-25T11:27:11.996 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.005+0000 7f4d50800640 1 -- start start 2024-04-25T11:27:11.998 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.007+0000 7f4d50800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d480736b0 0x7f4d480d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:11.998 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.007+0000 7f4d50800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d4814c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:11.998 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.007+0000 7f4d50800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d481581b0 0x7f4d48165770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:11.998 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.007+0000 7f4d50800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d480b6630 con 0x7f4d480736b0 2024-04-25T11:27:11.999 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.007+0000 7f4d50800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48168e20 con 0x7f4d481581b0 2024-04-25T11:27:11.999 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.007+0000 7f4d50800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4815f090 con 0x7f4d480b6220 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.007+0000 7f4d4de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d480736b0 0x7f4d480d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d4814c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d480736b0 0x7f4d480d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38414/0 (socket says 172.21.15.19:38414) 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4de00640 1 -- 172.21.15.19:0/2666637158 learned_addr learned my addr 172.21.15.19:0/2666637158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4e800640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d481581b0 0x7f4d48165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4de00640 1 -- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 msgr2=0x7f4d4814c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4de00640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d4814c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4de00640 1 -- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d481581b0 msgr2=0x7f4d48165770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4de00640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d481581b0 0x7f4d48165770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.008+0000 7f4d4de00640 1 -- 172.21.15.19:0/2666637158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4814cf20 con 0x7f4d480736b0 2024-04-25T11:27:12.000 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.009+0000 7f4d4de00640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d480736b0 0x7f4d480d0450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c02a6b0 tx=0x7f4d3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:12.001 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.009+0000 7f4d42000640 1 -- 172.21.15.19:0/2666637158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c041020 con 0x7f4d480736b0 2024-04-25T11:27:12.001 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.009+0000 7f4d42000640 1 -- 172.21.15.19:0/2666637158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c030760 con 0x7f4d480736b0 2024-04-25T11:27:12.001 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.009+0000 7f4d42000640 1 -- 172.21.15.19:0/2666637158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c037040 con 0x7f4d480736b0 2024-04-25T11:27:12.001 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.009+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d480d0d30 con 0x7f4d480736b0 2024-04-25T11:27:12.002 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.009+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d48164f70 con 0x7f4d480736b0 2024-04-25T11:27:12.002 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.011+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4806a160 con 0x7f4d480736b0 2024-04-25T11:27:12.003 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.012+0000 7f4d42000640 1 -- 172.21.15.19:0/2666637158 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f4d3c044020 con 0x7f4d480736b0 2024-04-25T11:27:12.005 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.013+0000 7f4d42000640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d2c0744f0 0x7f4d2c076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:12.005 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.013+0000 7f4d42000640 1 -- 172.21.15.19:0/2666637158 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f4d3c0b4b40 con 0x7f4d480736b0 2024-04-25T11:27:12.005 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.014+0000 7f4d4d400640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d2c0744f0 0x7f4d2c076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:12.006 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.015+0000 7f4d4d400640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d2c0744f0 0x7f4d2c076a00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d480cfcf0 tx=0x7f4d38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:12.011 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.020+0000 7f4d42000640 1 -- 172.21.15.19:0/2666637158 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d3c04b030 con 0x7f4d480736b0 2024-04-25T11:27:12.013 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T11:27:12.022+0000 7f5ffe400640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T11:27:12.020 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-04-25T11:27:12.029+0000 7f5ff4400640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:12.026 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T11:27:12.034+0000 7f1d02600640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:27:12.034 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-04-25T11:27:12.043+0000 7f1cf8600640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:27:12.213 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.221+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d480d1660 con 0x7f4d480736b0 2024-04-25T11:27:12.213 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.222+0000 7f4d42000640 1 -- 172.21.15.19:0/2666637158 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7567 (secure 0 0 0) 0x7f4d3c088090 con 0x7f4d480736b0 2024-04-25T11:27:12.214 INFO:teuthology.misc.health.smithi019.stdout: 2024-04-25T11:27:12.214 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":14,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:11.037237+0000","last_up_change":"2024-04-25T11:27:11.037237+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:12.216 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d2c0744f0 msgr2=0x7f4d2c076a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:12.216 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d2c0744f0 0x7f4d2c076a00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d480cfcf0 tx=0x7f4d38008040 comp rx=0 tx=0).stop 2024-04-25T11:27:12.216 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d480736b0 msgr2=0x7f4d480d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:12.216 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d480736b0 0x7f4d480d0450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c02a6b0 tx=0x7f4d3c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 shutdown_connections 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d2c0744f0 0x7f4d2c076a00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d481581b0 0x7f4d48165770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d480b6220 0x7f4d4814c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 --2- 172.21.15.19:0/2666637158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d480736b0 0x7f4d480d0450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 >> 172.21.15.19:0/2666637158 conn(0x7f4d4806cfe0 msgr2=0x7f4d480706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 shutdown_connections 2024-04-25T11:27:12.217 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:12.225+0000 7f4d50800640 1 -- 172.21.15.19:0/2666637158 wait complete. 2024-04-25T11:27:12.236 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-25T11:27:19.042 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:19.314 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.323+0000 7f819a400640 1 Processor -- start 2024-04-25T11:27:19.315 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.323+0000 7f819a400640 1 -- start start 2024-04-25T11:27:19.317 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.325+0000 7f819a400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81940cdf40 0x7f81940ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.317 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.325+0000 7f819a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f81940cec80 0x7f8194161b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.317 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.325+0000 7f819a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8194162050 0x7f819415e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.317 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f819a400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8194162460 con 0x7f81940cdf40 2024-04-25T11:27:19.317 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f819a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f81940cf200 con 0x7f81940cec80 2024-04-25T11:27:19.317 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f819a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f81940cf4e0 con 0x7f8194162050 2024-04-25T11:27:19.318 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81940cdf40 0x7f81940ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.318 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81940cdf40 0x7f81940ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37574/0 (socket says 172.21.15.19:37574) 2024-04-25T11:27:19.318 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199000640 1 -- 172.21.15.19:0/1352696482 learned_addr learned my addr 172.21.15.19:0/1352696482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:19.318 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8193e00640 1 --2- 172.21.15.19:0/1352696482 >> v2:172.21.15.169:3300/0 conn(0x7f81940cec80 0x7f8194161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.319 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199000640 1 -- 172.21.15.19:0/1352696482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8194162050 msgr2=0x7f819415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:19.319 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199a00640 1 --2- 172.21.15.19:0/1352696482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8194162050 0x7f819415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.319 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199000640 1 --2- 172.21.15.19:0/1352696482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8194162050 0x7f819415e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.319 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199000640 1 -- 172.21.15.19:0/1352696482 >> v2:172.21.15.169:3300/0 conn(0x7f81940cec80 msgr2=0x7f8194161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:19.319 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.326+0000 7f8199000640 1 --2- 172.21.15.19:0/1352696482 >> v2:172.21.15.169:3300/0 conn(0x7f81940cec80 0x7f8194161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.319 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.327+0000 7f8199000640 1 -- 172.21.15.19:0/1352696482 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81940cf7c0 con 0x7f81940cdf40 2024-04-25T11:27:19.319 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.327+0000 7f8199a00640 1 --2- 172.21.15.19:0/1352696482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8194162050 0x7f819415e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:19.320 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.328+0000 7f8199000640 1 --2- 172.21.15.19:0/1352696482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81940cdf40 0x7f81940ce740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f81940d16d0 tx=0x7f818802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d7536ed3f2b4da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:19.320 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.328+0000 7f8193400640 1 -- 172.21.15.19:0/1352696482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188037070 con 0x7f81940cdf40 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.328+0000 7f8193400640 1 -- 172.21.15.19:0/1352696482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8188032440 con 0x7f81940cdf40 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.328+0000 7f8193400640 1 -- 172.21.15.19:0/1352696482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188032810 con 0x7f81940cdf40 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 -- 172.21.15.19:0/1352696482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81940cdf40 msgr2=0x7f81940ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 --2- 172.21.15.19:0/1352696482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81940cdf40 0x7f81940ce740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f81940d16d0 tx=0x7f818802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 -- 172.21.15.19:0/1352696482 shutdown_connections 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 --2- 172.21.15.19:0/1352696482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8194162050 0x7f819415e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 --2- 172.21.15.19:0/1352696482 >> v2:172.21.15.169:3300/0 conn(0x7f81940cec80 0x7f8194161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 --2- 172.21.15.19:0/1352696482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81940cdf40 0x7f81940ce740 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 -- 172.21.15.19:0/1352696482 >> 172.21.15.19:0/1352696482 conn(0x7f81940b3520 msgr2=0x7f81940b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 -- 172.21.15.19:0/1352696482 shutdown_connections 2024-04-25T11:27:19.321 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.329+0000 7f819a400640 1 -- 172.21.15.19:0/1352696482 wait complete. 2024-04-25T11:27:19.323 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.332+0000 7f819a400640 1 Processor -- start 2024-04-25T11:27:19.323 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.332+0000 7f819a400640 1 -- start start 2024-04-25T11:27:19.325 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.333+0000 7f819a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81940cdf40 0x7f819414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.325 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.333+0000 7f819a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81940cec80 0x7f8194150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.325 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.333+0000 7f819a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8194162050 0x7f8194151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.325 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.333+0000 7f819a400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8194162870 con 0x7f8194162050 2024-04-25T11:27:19.325 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.333+0000 7f819a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f81940cfd80 con 0x7f81940cdf40 2024-04-25T11:27:19.325 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.333+0000 7f819a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f81941eadb0 con 0x7f81940cec80 2024-04-25T11:27:19.327 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8194162050 0x7f8194151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.327 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8194162050 0x7f8194151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37582/0 (socket says 172.21.15.19:37582) 2024-04-25T11:27:19.327 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 -- 172.21.15.19:0/1293831674 learned_addr learned my addr 172.21.15.19:0/1293831674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:19.327 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199000640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81940cdf40 0x7f819414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.327 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8193e00640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81940cec80 0x7f8194150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.327 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 -- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81940cec80 msgr2=0x7f8194150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.327 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81940cec80 0x7f8194150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 -- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81940cdf40 msgr2=0x7f819414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81940cdf40 0x7f819414e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.334+0000 7f8199a00640 1 -- 172.21.15.19:0/1293831674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8194160ed0 con 0x7f8194162050 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.335+0000 7f8199a00640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8194162050 0x7f8194151490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f818c00cd80 tx=0x7f818c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.335+0000 7f8190c00640 1 -- 172.21.15.19:0/1293831674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c00f680 con 0x7f8194162050 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.335+0000 7f8190c00640 1 -- 172.21.15.19:0/1293831674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818c00f840 con 0x7f8194162050 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.335+0000 7f8190c00640 1 -- 172.21.15.19:0/1293831674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c00fc30 con 0x7f8194162050 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.335+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8194151b20 con 0x7f8194162050 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.335+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81941520e0 con 0x7f8194162050 2024-04-25T11:27:19.328 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.336+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819406d990 con 0x7f8194162050 2024-04-25T11:27:19.329 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.338+0000 7f8190c00640 1 -- 172.21.15.19:0/1293831674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f818c0223f0 con 0x7f8194162050 2024-04-25T11:27:19.330 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.339+0000 7f8190c00640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81740744b0 0x7f81740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.330 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.339+0000 7f8190c00640 1 -- 172.21.15.19:0/1293831674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f818c095b80 con 0x7f8194162050 2024-04-25T11:27:19.330 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.339+0000 7f8199000640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81740744b0 0x7f81740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.332 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.341+0000 7f8199000640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81740744b0 0x7f81740769c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8188034000 tx=0x7f818802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:19.336 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.345+0000 7f8190c00640 1 -- 172.21.15.19:0/1293831674 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f818c014030 con 0x7f8194162050 2024-04-25T11:27:19.543 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.551+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8194050740 con 0x7f8194162050 2024-04-25T11:27:19.545 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.553+0000 7f8190c00640 1 -- 172.21.15.19:0/1293831674 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12884 (secure 0 0 0) 0x7f818c060650 con 0x7f8194162050 2024-04-25T11:27:19.545 INFO:teuthology.misc.health.smithi019.stdout: 2024-04-25T11:27:19.545 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":18,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:15.407923+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81740744b0 msgr2=0x7f81740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81740744b0 0x7f81740769c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8188034000 tx=0x7f818802e000 comp rx=0 tx=0).stop 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8194162050 msgr2=0x7f8194151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8194162050 0x7f8194151490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f818c00cd80 tx=0x7f818c00a650 comp rx=0 tx=0).stop 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 shutdown_connections 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81740744b0 0x7f81740769c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8194162050 0x7f8194151490 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.548 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81940cec80 0x7f8194150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.549 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.556+0000 7f819a400640 1 --2- 172.21.15.19:0/1293831674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81940cdf40 0x7f819414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.549 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.557+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 >> 172.21.15.19:0/1293831674 conn(0x7f81940b3520 msgr2=0x7f819406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:19.549 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.557+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 shutdown_connections 2024-04-25T11:27:19.549 INFO:teuthology.misc.health.smithi019.stderr:2024-04-25T11:27:19.557+0000 7f819a400640 1 -- 172.21.15.19:0/1293831674 wait complete. 2024-04-25T11:27:19.570 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T11:27:19.571 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T11:27:19.571 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:27:19.571 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:27:19.578 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:27:19.578 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:27:19.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:27:19.585 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:27:19.585 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:27:19.585 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:27:19.593 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:27:19.593 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:27:19.593 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:27:19.600 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:27:19.600 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T11:27:19.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:19.803 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:27:19.803 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:27:19.822 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:27:19.822 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:27:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.867+0000 7f46ff400640 1 Processor -- start 2024-04-25T11:27:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.867+0000 7f46ff400640 1 -- start start 2024-04-25T11:27:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46ff400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f8144500 0x7f46f8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46ff400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46f8162310 0x7f46f80ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46ff400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f806f5f0 con 0x7f46f80cdb90 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46ff400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f46f8070200 con 0x7f46f8162310 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46ff400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f46f806f8d0 con 0x7f46f8144500 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46fd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46f8162310 0x7f46f80ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46fd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46f8162310 0x7f46f80ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37620/0 (socket says 172.21.15.19:37620) 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46efe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46f80cdb90 0x7f46f8161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37194/0 (socket says 172.21.15.19:37194) 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46fd400640 1 -- 172.21.15.19:0/2745329253 learned_addr learned my addr 172.21.15.19:0/2745329253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.869+0000 7f46fca00640 1 --2- 172.21.15.19:0/2745329253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f8144500 0x7f46f8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.870+0000 7f46fd400640 1 -- 172.21.15.19:0/2745329253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f8144500 msgr2=0x7f46f8070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.870+0000 7f46fd400640 1 --2- 172.21.15.19:0/2745329253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f8144500 0x7f46f8070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.870+0000 7f46fd400640 1 -- 172.21.15.19:0/2745329253 >> v1:172.21.15.19:6789/0 conn(0x7f46f80cdb90 legacy=0x7f46f8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.870+0000 7f46fd400640 1 -- 172.21.15.19:0/2745329253 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f806fbb0 con 0x7f46f8162310 2024-04-25T11:27:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.870+0000 7f46f7e00640 1 -- 172.21.15.19:0/2745329253 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 366775976 0 0) 0x7f46f806f5f0 con 0x7f46f80cdb90 2024-04-25T11:27:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.871+0000 7f46fd400640 1 --2- 172.21.15.19:0/2745329253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f8162310 0x7f46f80ce8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009e50 tx=0x7f46f002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfa4fe04d4b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.871+0000 7f46f7e00640 1 -- 172.21.15.19:0/2745329253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0036070 con 0x7f46f8162310 2024-04-25T11:27:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.871+0000 7f46f7e00640 1 -- 172.21.15.19:0/2745329253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f0031600 con 0x7f46f8162310 2024-04-25T11:27:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.871+0000 7f46f7e00640 1 -- 172.21.15.19:0/2745329253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f00319d0 con 0x7f46f8162310 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 -- 172.21.15.19:0/2745329253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f8162310 msgr2=0x7f46f80ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 --2- 172.21.15.19:0/2745329253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f8162310 0x7f46f80ce8c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009e50 tx=0x7f46f002a730 comp rx=0 tx=0).stop 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 -- 172.21.15.19:0/2745329253 shutdown_connections 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 --2- 172.21.15.19:0/2745329253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f8162310 0x7f46f80ce8c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 --2- 172.21.15.19:0/2745329253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f8144500 0x7f46f8070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 -- 172.21.15.19:0/2745329253 >> 172.21.15.19:0/2745329253 conn(0x7f46f8073050 msgr2=0x7f46f8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 -- 172.21.15.19:0/2745329253 shutdown_connections 2024-04-25T11:27:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.872+0000 7f46ff400640 1 -- 172.21.15.19:0/2745329253 wait complete. 2024-04-25T11:27:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.874+0000 7f46ff400640 1 Processor -- start 2024-04-25T11:27:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.874+0000 7f46ff400640 1 -- start start 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f80cdb90 0x7f46f81456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46ff400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f8144500 0x7f46f815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46ff400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f815d2e0 0x7f46f80b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46ff400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46f806f2e0 con 0x7f46f80cdb90 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46ff400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46f806f160 con 0x7f46f8144500 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46ff400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f46f80cdfa0 con 0x7f46f815d2e0 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46fca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f80cdb90 0x7f46f81456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46fca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f80cdb90 0x7f46f81456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37628/0 (socket says 172.21.15.19:37628) 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46fca00640 1 -- 172.21.15.19:0/881402576 learned_addr learned my addr 172.21.15.19:0/881402576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46efe00640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f8144500 0x7f46f815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.876+0000 7f46fd400640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f815d2e0 0x7f46f80b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46fca00640 1 -- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f815d2e0 msgr2=0x7f46f80b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46fca00640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f815d2e0 0x7f46f80b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46fca00640 1 -- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f8144500 msgr2=0x7f46f815c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46fca00640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f8144500 0x7f46f815c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46fca00640 1 -- 172.21.15.19:0/881402576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f80cc420 con 0x7f46f80cdb90 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46fca00640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f80cdb90 0x7f46f81456d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46e800e8a0 tx=0x7f46e800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46f5600640 1 -- 172.21.15.19:0/881402576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8017070 con 0x7f46f80cdb90 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46f5600640 1 -- 172.21.15.19:0/881402576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e800c940 con 0x7f46f80cdb90 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46f5600640 1 -- 172.21.15.19:0/881402576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e800cd30 con 0x7f46f80cdb90 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.877+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f80cc920 con 0x7f46f80cdb90 2024-04-25T11:27:19.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.878+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f80ccee0 con 0x7f46f80cdb90 2024-04-25T11:27:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.879+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f8050740 con 0x7f46f80cdb90 2024-04-25T11:27:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.880+0000 7f46f5600640 1 -- 172.21.15.19:0/881402576 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46e801a070 con 0x7f46f80cdb90 2024-04-25T11:27:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.881+0000 7f46f5600640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46d40744b0 0x7f46d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.881+0000 7f46f5600640 1 -- 172.21.15.19:0/881402576 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f46e80950d0 con 0x7f46f80cdb90 2024-04-25T11:27:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.882+0000 7f46efe00640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46d40744b0 0x7f46d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.882+0000 7f46efe00640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46d40744b0 0x7f46d40769c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f46e0007fd0 tx=0x7f46e0007e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:19.887+0000 7f46f5600640 1 -- 172.21.15.19:0/881402576 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46e805f550 con 0x7f46f80cdb90 2024-04-25T11:27:20.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.090+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46f80b4aa0 con 0x7f46f80cdb90 2024-04-25T11:27:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.092+0000 7f46f5600640 1 -- 172.21.15.19:0/881402576 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12884 (secure 0 0 0) 0x7f46e8063850 con 0x7f46f80cdb90 2024-04-25T11:27:20.084 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:20.084 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":18,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:15.407923+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:20.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.095+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46d40744b0 msgr2=0x7f46d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.095+0000 7f46ff400640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46d40744b0 0x7f46d40769c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f46e0007fd0 tx=0x7f46e0007e60 comp rx=0 tx=0).stop 2024-04-25T11:27:20.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.095+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f80cdb90 msgr2=0x7f46f81456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.095+0000 7f46ff400640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f80cdb90 0x7f46f81456d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46e800e8a0 tx=0x7f46e800ed60 comp rx=0 tx=0).stop 2024-04-25T11:27:20.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 shutdown_connections 2024-04-25T11:27:20.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46d40744b0 0x7f46d40769c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46f8144500 0x7f46f815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f46f815d2e0 0x7f46f80b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 --2- 172.21.15.19:0/881402576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46f80cdb90 0x7f46f81456d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 >> 172.21.15.19:0/881402576 conn(0x7f46f8073050 msgr2=0x7f46f8143430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 shutdown_connections 2024-04-25T11:27:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.096+0000 7f46ff400640 1 -- 172.21.15.19:0/881402576 wait complete. 2024-04-25T11:27:20.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:27:20.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:27:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.408+0000 7fb4dbe00640 1 Processor -- start 2024-04-25T11:27:20.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.408+0000 7fb4dbe00640 1 -- start start 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4dbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4dc0cf650 0x7fb4dc16f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4dbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4dc0d1a40 con 0x7fb4dc0c6970 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4dbe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4dc0d1d20 con 0x7fb4dc0d3530 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4dbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb4dc0ce1b0 con 0x7fb4dc0cf650 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4da000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb4dc0c6970 0x7fb4dc0c6d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37200/0 (socket says 172.21.15.19:37200) 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4da000640 1 -- 172.21.15.19:0/264268563 learned_addr learned my addr 172.21.15.19:0/264268563 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4db400640 1 --2- 172.21.15.19:0/264268563 >> v2:172.21.15.169:3300/0 conn(0x7fb4dc0cf650 0x7fb4dc16f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.411+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740709649 0 0) 0x7fb4dc0d1a40 con 0x7fb4dc0c6970 2024-04-25T11:27:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4c0003520 con 0x7fb4dc0c6970 2024-04-25T11:27:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2762807470 0 0) 0x7fb4c0003520 con 0x7fb4dc0c6970 2024-04-25T11:27:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 >> v2:172.21.15.169:3300/0 conn(0x7fb4dc0cf650 msgr2=0x7fb4dc16f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 --2- 172.21.15.19:0/264268563 >> v2:172.21.15.169:3300/0 conn(0x7fb4dc0cf650 0x7fb4dc16f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 >> v1:172.21.15.169:6789/0 conn(0x7fb4dc0d3530 legacy=0x7fb4dc0b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc06cfb0 con 0x7fb4dc0c6970 2024-04-25T11:27:20.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb4c4001670 con 0x7fb4dc0c6970 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 >> v1:172.21.15.19:6789/0 conn(0x7fb4dc0c6970 legacy=0x7fb4dc0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4c00039b0 0x7fb4c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4c0004800 0x7fb4c0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0ce1b0 con 0x7fb4c0005350 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c000c280 con 0x7fb4c00039b0 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.412+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4c000c480 con 0x7fb4c0004800 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4db400640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4daa00640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4c00039b0 0x7fb4c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4da000640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4c0004800 0x7fb4c0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4db400640 1 -- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4c0004800 msgr2=0x7fb4c0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4db400640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4c0004800 0x7fb4c0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4db400640 1 -- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4c00039b0 msgr2=0x7fb4c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:20.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4db400640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4c00039b0 0x7fb4c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4db400640 1 -- 172.21.15.19:0/264268563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c0003520 con 0x7fb4c0005350 2024-04-25T11:27:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.413+0000 7fb4da000640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4c0004800 0x7fb4c0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.414+0000 7fb4db400640 1 -- 172.21.15.19:0/264268563 reap_dead start 2024-04-25T11:27:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.414+0000 7fb4db400640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4c000b9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009ac0 tx=0x7fb4c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c91c293f8e292e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.414+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4009690 con 0x7fb4c0005350 2024-04-25T11:27:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.414+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d4004020 con 0x7fb4c0005350 2024-04-25T11:27:20.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.415+0000 7fb4d9600640 1 -- 172.21.15.19:0/264268563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4004410 con 0x7fb4c0005350 2024-04-25T11:27:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.415+0000 7fb4dbe00640 1 -- 172.21.15.19:0/264268563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 msgr2=0x7fb4c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.415+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4c000b9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009ac0 tx=0x7fb4c0005830 comp rx=0 tx=0).stop 2024-04-25T11:27:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.415+0000 7fb4dbe00640 1 -- 172.21.15.19:0/264268563 shutdown_connections 2024-04-25T11:27:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.415+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/264268563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4c000b9c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.415+0000 7fb4dbe00640 1 -- 172.21.15.19:0/264268563 >> 172.21.15.19:0/264268563 conn(0x7fb4dc0d0420 msgr2=0x7fb4dc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.415+0000 7fb4dbe00640 1 -- 172.21.15.19:0/264268563 shutdown_connections 2024-04-25T11:27:20.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.416+0000 7fb4dbe00640 1 -- 172.21.15.19:0/264268563 wait complete. 2024-04-25T11:27:20.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.418+0000 7fb4dbe00640 1 Processor -- start 2024-04-25T11:27:20.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.418+0000 7fb4dbe00640 1 -- start start 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4dc1dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4dbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc09f880 0x7fb4dc09fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4dbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b76e0 0x7fb4dc0b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4dbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc06f380 con 0x7fb4c0005350 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4dbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0d0f40 con 0x7fb4dc0b76e0 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4dbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4dc06f500 con 0x7fb4dc09f880 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4dc1dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4dc1dedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37648/0 (socket says 172.21.15.19:37648) 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.420+0000 7fb4daa00640 1 -- 172.21.15.19:0/2486789303 learned_addr learned my addr 172.21.15.19:0/2486789303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4da000640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc09f880 0x7fb4dc09fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4db400640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b76e0 0x7fb4dc0b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4daa00640 1 -- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc09f880 msgr2=0x7fb4dc09fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4daa00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc09f880 0x7fb4dc09fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4daa00640 1 -- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b76e0 msgr2=0x7fb4dc0b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4daa00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b76e0 0x7fb4dc0b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4daa00640 1 -- 172.21.15.19:0/2486789303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc0b8030 con 0x7fb4c0005350 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4db400640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b76e0 0x7fb4dc0b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4daa00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4dc1dedd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e880 tx=0x7fb4cc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4d2a00640 1 -- 172.21.15.19:0/2486789303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc003bd0 con 0x7fb4c0005350 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4d2a00640 1 -- 172.21.15.19:0/2486789303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc018070 con 0x7fb4c0005350 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4d2a00640 1 -- 172.21.15.19:0/2486789303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0050d0 con 0x7fb4c0005350 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.421+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc0a05e0 con 0x7fb4c0005350 2024-04-25T11:27:20.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.422+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc0a0ba0 con 0x7fb4c0005350 2024-04-25T11:27:20.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.423+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4dc064200 con 0x7fb4c0005350 2024-04-25T11:27:20.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.424+0000 7fb4d2a00640 1 -- 172.21.15.19:0/2486789303 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb4cc005270 con 0x7fb4c0005350 2024-04-25T11:27:20.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.425+0000 7fb4d2a00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4b80744e0 0x7fb4b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.425+0000 7fb4d2a00640 1 -- 172.21.15.19:0/2486789303 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fb4cc094ed0 con 0x7fb4c0005350 2024-04-25T11:27:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.426+0000 7fb4da000640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4b80744e0 0x7fb4b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.427+0000 7fb4da000640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4b80744e0 0x7fb4b80769f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0b8a00 tx=0x7fb4c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.430+0000 7fb4d2a00640 1 -- 172.21.15.19:0/2486789303 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4dc064200 con 0x7fb4c0005350 2024-04-25T11:27:20.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.641+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4dc0d1a40 con 0x7fb4c0005350 2024-04-25T11:27:20.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.642+0000 7fb4d2a00640 1 -- 172.21.15.19:0/2486789303 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7fb4dc0d1a40 con 0x7fb4c0005350 2024-04-25T11:27:20.633 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:27:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4b80744e0 msgr2=0x7fb4b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4b80744e0 0x7fb4b80769f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0b8a00 tx=0x7fb4c4009040 comp rx=0 tx=0).stop 2024-04-25T11:27:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 msgr2=0x7fb4dc1dedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4dc1dedd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e880 tx=0x7fb4cc00ed40 comp rx=0 tx=0).stop 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 shutdown_connections 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4b80744e0 0x7fb4b80769f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b76e0 0x7fb4dc0b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc09f880 0x7fb4dc09fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 --2- 172.21.15.19:0/2486789303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4c0005350 0x7fb4dc1dedd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 >> 172.21.15.19:0/2486789303 conn(0x7fb4dc0d0420 msgr2=0x7fb4dc1eb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 shutdown_connections 2024-04-25T11:27:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.645+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2486789303 wait complete. 2024-04-25T11:27:20.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:27:20.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.949+0000 7fbfcf200640 1 Processor -- start 2024-04-25T11:27:20.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.949+0000 7fbfcf200640 1 -- start start 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.952+0000 7fbfcf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd00d50e0 0x7fbfd00d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.952+0000 7fbfcf200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbfd0160d60 0x7fbfd0161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.952+0000 7fbfcf200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd00b7940 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.952+0000 7fbfcf200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd00d2ef0 con 0x7fbfd0160d60 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.952+0000 7fbfcf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd0161280 con 0x7fbfd00d50e0 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.953+0000 7fbfcde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd00d50e0 0x7fbfd00d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.953+0000 7fbfcde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd00d50e0 0x7fbfd00d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:43904/0 (socket says 172.21.15.19:43904) 2024-04-25T11:27:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.953+0000 7fbfcde00640 1 -- 172.21.15.19:0/1139072641 learned_addr learned my addr 172.21.15.19:0/1139072641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.953+0000 7fbfce800640 1 --2- 172.21.15.19:0/1139072641 >> v2:172.21.15.169:3300/0 conn(0x7fbfd0160d60 0x7fbfd0161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.953+0000 7fbfcd400640 1 -- 172.21.15.19:0/1139072641 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678691614 0 0) 0x7fbfb8001110 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.954+0000 7fbfcd400640 1 -- 172.21.15.19:0/1139072641 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfb4003520 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.954+0000 7fbfcde00640 1 -- 172.21.15.19:0/1139072641 >> v2:172.21.15.169:3300/0 conn(0x7fbfd0160d60 msgr2=0x7fbfd0161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.954+0000 7fbfcde00640 1 --2- 172.21.15.19:0/1139072641 >> v2:172.21.15.169:3300/0 conn(0x7fbfd0160d60 0x7fbfd0161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.954+0000 7fbfcde00640 1 -- 172.21.15.19:0/1139072641 >> v1:172.21.15.169:6789/0 conn(0x7fbfd00d2ae0 legacy=0x7fbfd0160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.954+0000 7fbfcde00640 1 -- 172.21.15.19:0/1139072641 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd00d4600 con 0x7fbfd00d50e0 2024-04-25T11:27:20.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.955+0000 7fbfcde00640 1 --2- 172.21.15.19:0/1139072641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd00d50e0 0x7fbfd00d25a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009870 tx=0x7fbfc002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7fd1343902419 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.955+0000 7fbfcd400640 1 -- 172.21.15.19:0/1139072641 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0037070 con 0x7fbfd00d50e0 2024-04-25T11:27:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.956+0000 7fbfcd400640 1 -- 172.21.15.19:0/1139072641 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0032440 con 0x7fbfd00d50e0 2024-04-25T11:27:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.956+0000 7fbfcd400640 1 -- 172.21.15.19:0/1139072641 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00327f0 con 0x7fbfd00d50e0 2024-04-25T11:27:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.956+0000 7fbfcf200640 1 -- 172.21.15.19:0/1139072641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd00d50e0 msgr2=0x7fbfd00d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.956+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1139072641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd00d50e0 0x7fbfd00d25a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009870 tx=0x7fbfc002a270 comp rx=0 tx=0).stop 2024-04-25T11:27:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.957+0000 7fbfcf200640 1 -- 172.21.15.19:0/1139072641 shutdown_connections 2024-04-25T11:27:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.957+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1139072641 >> v2:172.21.15.169:3300/0 conn(0x7fbfd0160d60 0x7fbfd0161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.957+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1139072641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd00d50e0 0x7fbfd00d25a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.957+0000 7fbfcf200640 1 -- 172.21.15.19:0/1139072641 >> 172.21.15.19:0/1139072641 conn(0x7fbfd006c5c0 msgr2=0x7fbfd006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.957+0000 7fbfcf200640 1 -- 172.21.15.19:0/1139072641 shutdown_connections 2024-04-25T11:27:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.957+0000 7fbfcf200640 1 -- 172.21.15.19:0/1139072641 wait complete. 2024-04-25T11:27:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.960+0000 7fbfcf200640 1 Processor -- start 2024-04-25T11:27:20.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.960+0000 7fbfcf200640 1 -- start start 2024-04-25T11:27:20.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.962+0000 7fbfcf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd00d2ae0 0x7fbfd01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.962+0000 7fbfcf200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd00d50e0 0x7fbfd01e9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd0160d60 0x7fbfd00c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcf200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0161560 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcf200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd00d4bc0 con 0x7fbfd00d50e0 2024-04-25T11:27:20.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd00d54f0 con 0x7fbfd0160d60 2024-04-25T11:27:20.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd00d2ae0 0x7fbfd01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd00d2ae0 0x7fbfd01559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37660/0 (socket says 172.21.15.19:37660) 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 -- 172.21.15.19:0/1373394079 learned_addr learned my addr 172.21.15.19:0/1373394079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfce800640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd0160d60 0x7fbfd00c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfc5400640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd00d50e0 0x7fbfd01e9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 -- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd0160d60 msgr2=0x7fbfd00c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd0160d60 0x7fbfd00c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 -- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd00d50e0 msgr2=0x7fbfd01e9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd00d50e0 0x7fbfd01e9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.963+0000 7fbfcde00640 1 -- 172.21.15.19:0/1373394079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd01e97a0 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.964+0000 7fbfcde00640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd00d2ae0 0x7fbfd01559f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfc002a2b0 tx=0x7fbfc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.964+0000 7fbfc6a00640 1 -- 172.21.15.19:0/1373394079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0037040 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.964+0000 7fbfc6a00640 1 -- 172.21.15.19:0/1373394079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0003bc0 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.964+0000 7fbfc6a00640 1 -- 172.21.15.19:0/1373394079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00325f0 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.964+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd0154950 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.964+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd0154f10 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.966+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd0050740 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.967+0000 7fbfc6a00640 1 -- 172.21.15.19:0/1373394079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbfc0050430 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.969+0000 7fbfc6a00640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfac0744b0 0x7fbfac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.969+0000 7fbfc6a00640 1 -- 172.21.15.19:0/1373394079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fbfc00b5fc0 con 0x7fbfd00d2ae0 2024-04-25T11:27:20.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.969+0000 7fbfc5400640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfac0744b0 0x7fbfac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.971+0000 7fbfc5400640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfac0744b0 0x7fbfac0769c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00b7760 tx=0x7fbfb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:20.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:20.972+0000 7fbfc6a00640 1 -- 172.21.15.19:0/1373394079 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfc0080a80 con 0x7fbfd00d2ae0 2024-04-25T11:27:21.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.218+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfd00b8aa0 con 0x7fbfd00d2ae0 2024-04-25T11:27:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.219+0000 7fbfc6a00640 1 -- 172.21.15.19:0/1373394079 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7fbfc0080380 con 0x7fbfd00d2ae0 2024-04-25T11:27:21.211 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:21.211 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:27:21.211 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-25T11:27:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfac0744b0 msgr2=0x7fbfac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfac0744b0 0x7fbfac0769c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00b7760 tx=0x7fbfb8009040 comp rx=0 tx=0).stop 2024-04-25T11:27:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd00d2ae0 msgr2=0x7fbfd01559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd00d2ae0 0x7fbfd01559f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfc002a2b0 tx=0x7fbfc00046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 shutdown_connections 2024-04-25T11:27:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfac0744b0 0x7fbfac0769c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfd00d50e0 0x7fbfd01e9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfd0160d60 0x7fbfd00c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 --2- 172.21.15.19:0/1373394079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd00d2ae0 0x7fbfd01559f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 >> 172.21.15.19:0/1373394079 conn(0x7fbfd006c5c0 msgr2=0x7fbfd0162110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:21.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 shutdown_connections 2024-04-25T11:27:21.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.222+0000 7fbfcf200640 1 -- 172.21.15.19:0/1373394079 wait complete. 2024-04-25T11:27:21.239 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.517+0000 7f4ea7200640 1 Processor -- start 2024-04-25T11:27:21.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.517+0000 7f4ea7200640 1 -- start start 2024-04-25T11:27:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 0x7f4ea81611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea7200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea8161720 con 0x7f4ea80d1f70 2024-04-25T11:27:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea806ddc0 con 0x7f4ea808cd60 2024-04-25T11:27:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea806e9d0 con 0x7f4ea80b4ff0 2024-04-25T11:27:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea6800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4ea80d1f70 0x7f4ea80d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37214/0 (socket says 172.21.15.19:37214) 2024-04-25T11:27:21.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea6800640 1 -- 172.21.15.19:0/388226368 learned_addr learned my addr 172.21.15.19:0/388226368 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea5e00640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 0x7f4ea81611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.520+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049669703 0 0) 0x7f4ea0001490 con 0x7f4ea80d1f70 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e8c003520 con 0x7f4ea80d1f70 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 137293864 0 0) 0x7f4e8c003520 con 0x7f4ea80d1f70 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 msgr2=0x7f4ea81611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 0x7f4ea81611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 >> v1:172.21.15.169:6789/0 conn(0x7f4ea808cd60 legacy=0x7f4ea808d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea806e0a0 con 0x7f4ea80d1f70 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4ea0001bc0 con 0x7f4ea80d1f70 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 >> v1:172.21.15.19:6789/0 conn(0x7f4ea80d1f70 legacy=0x7f4ea80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 0x7f4e8c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4e8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8c004d40 0x7f4e8c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c00bbd0 con 0x7f4e8c003f60 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c00bd80 con 0x7f4e8c004d40 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8c00bf80 con 0x7f4ea80b4ff0 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.521+0000 7f4e9fe00640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4e8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4ea5e00640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 0x7f4e8c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4ea6800640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8c004d40 0x7f4e8c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4e9fe00640 1 -- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 msgr2=0x7f4e8c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4e9fe00640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea80b4ff0 0x7f4e8c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4e9fe00640 1 -- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8c004d40 msgr2=0x7f4e8c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4e9fe00640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8c004d40 0x7f4e8c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4e9fe00640 1 -- 172.21.15.19:0/388226368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c003520 con 0x7f4e8c003f60 2024-04-25T11:27:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.522+0000 7f4ea6800640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e8c004d40 0x7f4e8c005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.523+0000 7f4e9fe00640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4e8c004800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e90001750 tx=0x7f4e8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cee5b8595c8946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.523+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90001370 con 0x7f4e8c003f60 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.523+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e900052d0 con 0x7f4e8c003f60 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.523+0000 7f4ea5400640 1 -- 172.21.15.19:0/388226368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9002abe0 con 0x7f4e8c003f60 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.524+0000 7f4ea7200640 1 -- 172.21.15.19:0/388226368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 msgr2=0x7f4e8c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.524+0000 7f4ea7200640 1 --2- 172.21.15.19:0/388226368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4e8c004800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e90001750 tx=0x7f4e8c005830 comp rx=0 tx=0).stop 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.524+0000 7f4ea6800640 1 -- 172.21.15.19:0/388226368 reap_dead start 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.524+0000 7f4ea7200640 1 -- 172.21.15.19:0/388226368 shutdown_connections 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.524+0000 7f4ea7200640 1 -- 172.21.15.19:0/388226368 >> 172.21.15.19:0/388226368 conn(0x7f4ea80b5460 msgr2=0x7f4ea80b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.524+0000 7f4ea7200640 1 -- 172.21.15.19:0/388226368 shutdown_connections 2024-04-25T11:27:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.524+0000 7f4ea7200640 1 -- 172.21.15.19:0/388226368 wait complete. 2024-04-25T11:27:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.526+0000 7f4ea7200640 1 Processor -- start 2024-04-25T11:27:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.526+0000 7f4ea7200640 1 -- start start 2024-04-25T11:27:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4ea8163e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea8156b40 0x7f4ea815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8156f50 0x7f4ea8157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea80b9180 con 0x7f4e8c003f60 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea806e660 con 0x7f4ea8156f50 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea806da20 con 0x7f4ea8156b40 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4ea8163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4ea8163e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37668/0 (socket says 172.21.15.19:37668) 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4ea5e00640 1 -- 172.21.15.19:0/2687964010 learned_addr learned my addr 172.21.15.19:0/2687964010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.528+0000 7f4e9fe00640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea8156b40 0x7f4ea815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea6800640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8156f50 0x7f4ea8157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea5e00640 1 -- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea8156b40 msgr2=0x7f4ea815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea5e00640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea8156b40 0x7f4ea815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea5e00640 1 -- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8156f50 msgr2=0x7f4ea8157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea5e00640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8156f50 0x7f4ea8157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea5e00640 1 -- 172.21.15.19:0/2687964010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea8162950 con 0x7f4e8c003f60 2024-04-25T11:27:21.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea5e00640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4ea8163e60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e98011590 tx=0x7f4e98011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4e9e000640 1 -- 172.21.15.19:0/2687964010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98019070 con 0x7f4e8c003f60 2024-04-25T11:27:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4e9e000640 1 -- 172.21.15.19:0/2687964010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98005da0 con 0x7f4e8c003f60 2024-04-25T11:27:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea8162dc0 con 0x7f4e8c003f60 2024-04-25T11:27:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4e9e000640 1 -- 172.21.15.19:0/2687964010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98006270 con 0x7f4e8c003f60 2024-04-25T11:27:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.529+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea8163380 con 0x7f4e8c003f60 2024-04-25T11:27:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.530+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea8164c30 con 0x7f4e8c003f60 2024-04-25T11:27:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.532+0000 7f4e9e000640 1 -- 172.21.15.19:0/2687964010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4e9801c070 con 0x7f4e8c003f60 2024-04-25T11:27:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.533+0000 7f4e9e000640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4e840744b0 0x7f4e840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.533+0000 7f4e9e000640 1 -- 172.21.15.19:0/2687964010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f4e980954a0 con 0x7f4e8c003f60 2024-04-25T11:27:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.533+0000 7f4e9fe00640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4e840744b0 0x7f4e840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.534+0000 7f4e9fe00640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4e840744b0 0x7f4e840769c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e90032000 tx=0x7f4e9002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.537+0000 7f4e9e000640 1 -- 172.21.15.19:0/2687964010 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ea8164c30 con 0x7f4e8c003f60 2024-04-25T11:27:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.742+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ea8050740 con 0x7f4e8c003f60 2024-04-25T11:27:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.744+0000 7f4e9e000640 1 -- 172.21.15.19:0/2687964010 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12884 (secure 0 0 0) 0x7f4e9805fee0 con 0x7f4e8c003f60 2024-04-25T11:27:21.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:21.736 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":18,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:15.407923+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.746+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4e840744b0 msgr2=0x7f4e840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.746+0000 7f4ea7200640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4e840744b0 0x7f4e840769c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4e90032000 tx=0x7f4e9002e000 comp rx=0 tx=0).stop 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 msgr2=0x7f4ea8163e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4ea8163e60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e98011590 tx=0x7f4e98011a50 comp rx=0 tx=0).stop 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 shutdown_connections 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4e840744b0 0x7f4e840769c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ea8156f50 0x7f4ea8157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ea8156b40 0x7f4ea815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 --2- 172.21.15.19:0/2687964010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e8c003f60 0x7f4ea8163e60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 >> 172.21.15.19:0/2687964010 conn(0x7f4ea80b5460 msgr2=0x7f4ea815eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 shutdown_connections 2024-04-25T11:27:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:21.747+0000 7f4ea7200640 1 -- 172.21.15.19:0/2687964010 wait complete. 2024-04-25T11:27:21.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:27:21.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:27:22.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.034+0000 7f5a76c00640 1 Processor -- start 2024-04-25T11:27:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.034+0000 7f5a76c00640 1 -- start start 2024-04-25T11:27:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.037+0000 7f5a76c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a7006dce0 0x7f5a7015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.037+0000 7f5a76c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a70070000 con 0x7f5a7015ec90 2024-04-25T11:27:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.037+0000 7f5a76c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a7006f240 con 0x7f5a70144340 2024-04-25T11:27:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.037+0000 7f5a76c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a70144750 con 0x7f5a7006dce0 2024-04-25T11:27:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.037+0000 7f5a76200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5a7015ec90 0x7f5a7015f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37220/0 (socket says 172.21.15.19:37220) 2024-04-25T11:27:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.037+0000 7f5a76200640 1 -- 172.21.15.19:0/3693199680 learned_addr learned my addr 172.21.15.19:0/3693199680 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.037+0000 7f5a74e00640 1 --2- 172.21.15.19:0/3693199680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a7006dce0 0x7f5a7015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.038+0000 7f5a6fe00640 1 -- 172.21.15.19:0/3693199680 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182695390 0 0) 0x7f5a70070000 con 0x7f5a7015ec90 2024-04-25T11:27:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.038+0000 7f5a6fe00640 1 -- 172.21.15.19:0/3693199680 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a54003520 con 0x7f5a7015ec90 2024-04-25T11:27:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.038+0000 7f5a74e00640 1 -- 172.21.15.19:0/3693199680 >> v1:172.21.15.169:6789/0 conn(0x7f5a70144340 legacy=0x7f5a7006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.038+0000 7f5a74e00640 1 -- 172.21.15.19:0/3693199680 >> v1:172.21.15.19:6789/0 conn(0x7f5a7015ec90 legacy=0x7f5a7015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.038+0000 7f5a6fe00640 1 -- 172.21.15.19:0/3693199680 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4172445113 0 0) 0x7f5a54003520 con 0x7f5a7015ec90 2024-04-25T11:27:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.038+0000 7f5a74e00640 1 -- 172.21.15.19:0/3693199680 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7006f520 con 0x7f5a7006dce0 2024-04-25T11:27:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.039+0000 7f5a74e00640 1 --2- 172.21.15.19:0/3693199680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a7006dce0 0x7f5a7015e750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0097d0 tx=0x7f5a5c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1faf922a808c61f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.040+0000 7f5a6fe00640 1 -- 172.21.15.19:0/3693199680 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c037070 con 0x7f5a7006dce0 2024-04-25T11:27:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.040+0000 7f5a6fe00640 1 -- 172.21.15.19:0/3693199680 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c02ac30 con 0x7f5a7006dce0 2024-04-25T11:27:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.040+0000 7f5a6fe00640 1 -- 172.21.15.19:0/3693199680 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68001670 con 0x7f5a7006dce0 2024-04-25T11:27:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.041+0000 7f5a76c00640 1 -- 172.21.15.19:0/3693199680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a7006dce0 msgr2=0x7f5a7015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.041+0000 7f5a76c00640 1 --2- 172.21.15.19:0/3693199680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a7006dce0 0x7f5a7015e750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0097d0 tx=0x7f5a5c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.041+0000 7f5a76c00640 1 -- 172.21.15.19:0/3693199680 shutdown_connections 2024-04-25T11:27:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.041+0000 7f5a76c00640 1 --2- 172.21.15.19:0/3693199680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a7006dce0 0x7f5a7015e750 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.041+0000 7f5a76c00640 1 -- 172.21.15.19:0/3693199680 >> 172.21.15.19:0/3693199680 conn(0x7f5a70072f40 msgr2=0x7f5a70073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.041+0000 7f5a76c00640 1 -- 172.21.15.19:0/3693199680 shutdown_connections 2024-04-25T11:27:22.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.041+0000 7f5a76c00640 1 -- 172.21.15.19:0/3693199680 wait complete. 2024-04-25T11:27:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.043+0000 7f5a76c00640 1 Processor -- start 2024-04-25T11:27:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.043+0000 7f5a76c00640 1 -- start start 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a76c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a70144340 0x7f5a701f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a76c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a7015ec90 0x7f5a700affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a76c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a700b55f0 0x7f5a701603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a76c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7015f0a0 con 0x7f5a7015ec90 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a76c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7006fae0 con 0x7f5a700b55f0 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a76c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7015f220 con 0x7f5a70144340 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a74e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a7015ec90 0x7f5a700affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a74e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a7015ec90 0x7f5a700affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37682/0 (socket says 172.21.15.19:37682) 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a74e00640 1 -- 172.21.15.19:0/246042196 learned_addr learned my addr 172.21.15.19:0/246042196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a75800640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a70144340 0x7f5a701f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a74e00640 1 -- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a70144340 msgr2=0x7f5a701f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.045+0000 7f5a76200640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a700b55f0 0x7f5a701603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a74e00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a70144340 0x7f5a701f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a74e00640 1 -- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a700b55f0 msgr2=0x7f5a701603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a74e00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a700b55f0 0x7f5a701603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a74e00640 1 -- 172.21.15.19:0/246042196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a700b09b0 con 0x7f5a7015ec90 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a75800640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a70144340 0x7f5a701f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a74e00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a7015ec90 0x7f5a700affd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098f0 tx=0x7f5a5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a6d600640 1 -- 172.21.15.19:0/246042196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c042020 con 0x7f5a7015ec90 2024-04-25T11:27:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a6d600640 1 -- 172.21.15.19:0/246042196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c003c40 con 0x7f5a7015ec90 2024-04-25T11:27:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a6d600640 1 -- 172.21.15.19:0/246042196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c037040 con 0x7f5a7015ec90 2024-04-25T11:27:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a700b0c90 con 0x7f5a7015ec90 2024-04-25T11:27:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.046+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a700b4df0 con 0x7f5a7015ec90 2024-04-25T11:27:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.047+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a700737f0 con 0x7f5a7015ec90 2024-04-25T11:27:22.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.049+0000 7f5a6d600640 1 -- 172.21.15.19:0/246042196 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a5c045020 con 0x7f5a7015ec90 2024-04-25T11:27:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.050+0000 7f5a6d600640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a500744b0 0x7f5a500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.050+0000 7f5a6d600640 1 -- 172.21.15.19:0/246042196 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f5a5c0b54d0 con 0x7f5a7015ec90 2024-04-25T11:27:22.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.051+0000 7f5a75800640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a500744b0 0x7f5a500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.052+0000 7f5a75800640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a500744b0 0x7f5a500769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a600012e0 tx=0x7f5a60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:22.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.055+0000 7f5a6d600640 1 -- 172.21.15.19:0/246042196 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a700737f0 con 0x7f5a7015ec90 2024-04-25T11:27:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.248+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5a70076c60 con 0x7f5a7015ec90 2024-04-25T11:27:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.249+0000 7f5a6d600640 1 -- 172.21.15.19:0/246042196 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7f5a5c04c020 con 0x7f5a7015ec90 2024-04-25T11:27:22.240 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:27:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.251+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a500744b0 msgr2=0x7f5a500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.251+0000 7f5a76c00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a500744b0 0x7f5a500769c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a600012e0 tx=0x7f5a60009040 comp rx=0 tx=0).stop 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a7015ec90 msgr2=0x7f5a700affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a7015ec90 0x7f5a700affd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098f0 tx=0x7f5a5c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 shutdown_connections 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a500744b0 0x7f5a500769c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a700b55f0 0x7f5a701603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a7015ec90 0x7f5a700affd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 --2- 172.21.15.19:0/246042196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a70144340 0x7f5a701f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 >> 172.21.15.19:0/246042196 conn(0x7f5a70072f40 msgr2=0x7f5a7006d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 shutdown_connections 2024-04-25T11:27:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.252+0000 7f5a76c00640 1 -- 172.21.15.19:0/246042196 wait complete. 2024-04-25T11:27:22.267 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:27:22.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.550+0000 7fc01d400640 1 Processor -- start 2024-04-25T11:27:22.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.551+0000 7fc01d400640 1 -- start start 2024-04-25T11:27:22.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.553+0000 7fc01d400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc018143170 0x7fc018160950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.553+0000 7fc01d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc0181612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.553+0000 7fc01d400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0180cdb20 con 0x7fc01806f940 2024-04-25T11:27:22.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.553+0000 7fc01d400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc018146970 con 0x7fc018143170 2024-04-25T11:27:22.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.553+0000 7fc01d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0180b4ee0 con 0x7fc018160e90 2024-04-25T11:27:22.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.554+0000 7fc017400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc0181612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.554+0000 7fc017400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc0181612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:43954/0 (socket says 172.21.15.19:43954) 2024-04-25T11:27:22.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.554+0000 7fc017400640 1 -- 172.21.15.19:0/3706822323 learned_addr learned my addr 172.21.15.19:0/3706822323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:22.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.554+0000 7fc017e00640 1 --2- 172.21.15.19:0/3706822323 >> v2:172.21.15.169:3300/0 conn(0x7fc018143170 0x7fc018160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.555+0000 7fc017400640 1 -- 172.21.15.19:0/3706822323 >> v2:172.21.15.169:3300/0 conn(0x7fc018143170 msgr2=0x7fc018160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:22.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.555+0000 7fc017400640 1 --2- 172.21.15.19:0/3706822323 >> v2:172.21.15.169:3300/0 conn(0x7fc018143170 0x7fc018160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.555+0000 7fc017400640 1 -- 172.21.15.19:0/3706822323 >> v1:172.21.15.169:6789/0 conn(0x7fc01806f940 legacy=0x7fc01806fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.555+0000 7fc017400640 1 -- 172.21.15.19:0/3706822323 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0180cde00 con 0x7fc018160e90 2024-04-25T11:27:22.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.556+0000 7fc017400640 1 --2- 172.21.15.19:0/3706822323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc0181612a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc000009870 tx=0x7fc00002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c28d712e9f6543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.556+0000 7fc016a00640 1 -- 172.21.15.19:0/3706822323 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000037070 con 0x7fc018160e90 2024-04-25T11:27:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.557+0000 7fc016a00640 1 -- 172.21.15.19:0/3706822323 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc000032440 con 0x7fc018160e90 2024-04-25T11:27:22.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.557+0000 7fc016a00640 1 -- 172.21.15.19:0/3706822323 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000032810 con 0x7fc018160e90 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.557+0000 7fc01d400640 1 -- 172.21.15.19:0/3706822323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 msgr2=0x7fc0181612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.557+0000 7fc01d400640 1 --2- 172.21.15.19:0/3706822323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc0181612a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc000009870 tx=0x7fc00002a250 comp rx=0 tx=0).stop 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.558+0000 7fc01d400640 1 -- 172.21.15.19:0/3706822323 shutdown_connections 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.558+0000 7fc01d400640 1 --2- 172.21.15.19:0/3706822323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc0181612a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.558+0000 7fc01d400640 1 --2- 172.21.15.19:0/3706822323 >> v2:172.21.15.169:3300/0 conn(0x7fc018143170 0x7fc018160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.558+0000 7fc01d400640 1 -- 172.21.15.19:0/3706822323 >> 172.21.15.19:0/3706822323 conn(0x7fc0180d1b40 msgr2=0x7fc0180d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.558+0000 7fc01d400640 1 -- 172.21.15.19:0/3706822323 shutdown_connections 2024-04-25T11:27:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.558+0000 7fc01d400640 1 -- 172.21.15.19:0/3706822323 wait complete. 2024-04-25T11:27:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.560+0000 7fc01d400640 1 Processor -- start 2024-04-25T11:27:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.560+0000 7fc01d400640 1 -- start start 2024-04-25T11:27:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc01d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01806f940 0x7fc01813c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc01d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018143170 0x7fc01815fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc01d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc018144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc01d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0181617e0 con 0x7fc018143170 2024-04-25T11:27:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc01d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc01806f440 con 0x7fc01806f940 2024-04-25T11:27:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc01d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc018143580 con 0x7fc018160e90 2024-04-25T11:27:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc017400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018143170 0x7fc01815fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc017400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018143170 0x7fc01815fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37688/0 (socket says 172.21.15.19:37688) 2024-04-25T11:27:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc017400640 1 -- 172.21.15.19:0/2531767017 learned_addr learned my addr 172.21.15.19:0/2531767017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc017e00640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01806f940 0x7fc01813c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc01ca00640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc018144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.562+0000 7fc017400640 1 -- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 msgr2=0x7fc018144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc017400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc018144d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc017400640 1 -- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01806f940 msgr2=0x7fc01813c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc017400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01806f940 0x7fc01813c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc017400640 1 -- 172.21.15.19:0/2531767017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01813cbe0 con 0x7fc018143170 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc017e00640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01806f940 0x7fc01813c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc01ca00640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc018144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc017400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018143170 0x7fc01815fd10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc00002a290 tx=0x7fc0000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc00fe00640 1 -- 172.21.15.19:0/2531767017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000041030 con 0x7fc018143170 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc00fe00640 1 -- 172.21.15.19:0/2531767017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc000030440 con 0x7fc018143170 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc00fe00640 1 -- 172.21.15.19:0/2531767017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000037040 con 0x7fc018143170 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc018160250 con 0x7fc018143170 2024-04-25T11:27:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.563+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01815da80 con 0x7fc018143170 2024-04-25T11:27:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.565+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc018050740 con 0x7fc018143170 2024-04-25T11:27:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.566+0000 7fc00fe00640 1 -- 172.21.15.19:0/2531767017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc000044070 con 0x7fc018143170 2024-04-25T11:27:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.566+0000 7fc00fe00640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbff40744b0 0x7fbff40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.567+0000 7fc00fe00640 1 -- 172.21.15.19:0/2531767017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc0000b5570 con 0x7fc018143170 2024-04-25T11:27:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.567+0000 7fc017e00640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbff40744b0 0x7fbff40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:22.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.569+0000 7fc017e00640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbff40744b0 0x7fbff40769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc008009a20 tx=0x7fc008008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:22.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.571+0000 7fc00fe00640 1 -- 172.21.15.19:0/2531767017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc000080040 con 0x7fc018143170 2024-04-25T11:27:22.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.777+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc0181429d0 con 0x7fc018143170 2024-04-25T11:27:22.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.778+0000 7fc00fe00640 1 -- 172.21.15.19:0/2531767017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fc00007f940 con 0x7fc018143170 2024-04-25T11:27:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:22.770 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:27:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:27:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T11:27:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T11:27:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:27:22.770 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbff40744b0 msgr2=0x7fbff40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbff40744b0 0x7fbff40769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc008009a20 tx=0x7fc008008040 comp rx=0 tx=0).stop 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018143170 msgr2=0x7fc01815fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018143170 0x7fc01815fd10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc00002a290 tx=0x7fc0000046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 shutdown_connections 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbff40744b0 0x7fbff40769c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc018160e90 0x7fc018144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc018143170 0x7fc01815fd10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 --2- 172.21.15.19:0/2531767017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc01806f940 0x7fc01813c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:22.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 >> 172.21.15.19:0/2531767017 conn(0x7fc0180d1b40 msgr2=0x7fc0180b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:22.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 shutdown_connections 2024-04-25T11:27:22.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:22.781+0000 7fc01d400640 1 -- 172.21.15.19:0/2531767017 wait complete. 2024-04-25T11:27:22.797 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:27:22.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:27:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.086+0000 7f14c1000640 1 Processor -- start 2024-04-25T11:27:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.086+0000 7f14c1000640 1 -- start start 2024-04-25T11:27:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14c1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc144340 0x7f14bc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14c1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14bc15ee60 0x7f14bc15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14c1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14bc06df20 con 0x7f14bc06db10 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14c1000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f14bc15f270 con 0x7f14bc15ee60 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14c1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14bc144750 con 0x7f14bc144340 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14bbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14bc15ee60 0x7f14bc15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14bbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14bc15ee60 0x7f14bc15f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37698/0 (socket says 172.21.15.19:37698) 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.089+0000 7f14bbe00640 1 -- 172.21.15.19:0/952540250 learned_addr learned my addr 172.21.15.19:0/952540250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.090+0000 7f14bb400640 1 --2- 172.21.15.19:0/952540250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc144340 0x7f14bc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.090+0000 7f14bbe00640 1 -- 172.21.15.19:0/952540250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc144340 msgr2=0x7f14bc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:23.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.090+0000 7f14bbe00640 1 --2- 172.21.15.19:0/952540250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc144340 0x7f14bc06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.090+0000 7f14bbe00640 1 -- 172.21.15.19:0/952540250 >> v1:172.21.15.169:6789/0 conn(0x7f14bc06db10 legacy=0x7f14bc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.090+0000 7f14bbe00640 1 -- 172.21.15.19:0/952540250 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc070000 con 0x7f14bc15ee60 2024-04-25T11:27:23.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.091+0000 7f14bbe00640 1 --2- 172.21.15.19:0/952540250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc15ee60 0x7f14bc15f4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009e50 tx=0x7f14b402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11e3bdfab647691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:23.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.091+0000 7f14baa00640 1 -- 172.21.15.19:0/952540250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4036070 con 0x7f14bc15ee60 2024-04-25T11:27:23.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14baa00640 1 -- 172.21.15.19:0/952540250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b4031600 con 0x7f14bc15ee60 2024-04-25T11:27:23.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14baa00640 1 -- 172.21.15.19:0/952540250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b40319d0 con 0x7f14bc15ee60 2024-04-25T11:27:23.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 -- 172.21.15.19:0/952540250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc15ee60 msgr2=0x7f14bc15f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 --2- 172.21.15.19:0/952540250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc15ee60 0x7f14bc15f4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009e50 tx=0x7f14b402a710 comp rx=0 tx=0).stop 2024-04-25T11:27:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 -- 172.21.15.19:0/952540250 shutdown_connections 2024-04-25T11:27:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 --2- 172.21.15.19:0/952540250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc15ee60 0x7f14bc15f4a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 --2- 172.21.15.19:0/952540250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc144340 0x7f14bc06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 -- 172.21.15.19:0/952540250 >> 172.21.15.19:0/952540250 conn(0x7f14bc072f40 msgr2=0x7f14bc073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 -- 172.21.15.19:0/952540250 shutdown_connections 2024-04-25T11:27:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.092+0000 7f14c1000640 1 -- 172.21.15.19:0/952540250 wait complete. 2024-04-25T11:27:23.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.095+0000 7f14c1000640 1 Processor -- start 2024-04-25T11:27:23.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.095+0000 7f14c1000640 1 -- start start 2024-04-25T11:27:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.096+0000 7f14c1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc06db10 0x7f14bc0b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.096+0000 7f14c1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc144340 0x7f14bc158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.096+0000 7f14c1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14bc15ee60 0x7f14bc159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.096+0000 7f14c1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc06f800 con 0x7f14bc144340 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.096+0000 7f14c1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc06f520 con 0x7f14bc15ee60 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.096+0000 7f14c1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14bc06fae0 con 0x7f14bc06db10 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc144340 0x7f14bc158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc144340 0x7f14bc158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37704/0 (socket says 172.21.15.19:37704) 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 -- 172.21.15.19:0/3756769625 learned_addr learned my addr 172.21.15.19:0/3756769625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14bb400640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc06db10 0x7f14bc0b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14bbe00640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14bc15ee60 0x7f14bc159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 -- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc06db10 msgr2=0x7f14bc0b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc06db10 0x7f14bc0b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 -- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14bc15ee60 msgr2=0x7f14bc159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14bc15ee60 0x7f14bc159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 -- 172.21.15.19:0/3756769625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc145c00 con 0x7f14bc144340 2024-04-25T11:27:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.097+0000 7f14b3e00640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc144340 0x7f14bc158c00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c3c0 tx=0x7f14a400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.098+0000 7f14b3400640 1 -- 172.21.15.19:0/3756769625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a40052d0 con 0x7f14bc144340 2024-04-25T11:27:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.098+0000 7f14b3400640 1 -- 172.21.15.19:0/3756769625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a4005490 con 0x7f14bc144340 2024-04-25T11:27:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.098+0000 7f14b3400640 1 -- 172.21.15.19:0/3756769625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4007260 con 0x7f14bc144340 2024-04-25T11:27:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.098+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc145ee0 con 0x7f14bc144340 2024-04-25T11:27:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.098+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc1464a0 con 0x7f14bc144340 2024-04-25T11:27:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.099+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc076c60 con 0x7f14bc144340 2024-04-25T11:27:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.100+0000 7f14b3400640 1 -- 172.21.15.19:0/3756769625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14a4007400 con 0x7f14bc144340 2024-04-25T11:27:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.101+0000 7f14b3400640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14980744e0 0x7f14980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.101+0000 7f14b3400640 1 -- 172.21.15.19:0/3756769625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f14a40943c0 con 0x7f14bc144340 2024-04-25T11:27:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.102+0000 7f14bb400640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14980744e0 0x7f14980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.103+0000 7f14bb400640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14980744e0 0x7f14980769f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009a30 tx=0x7f14ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:23.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.106+0000 7f14b3400640 1 -- 172.21.15.19:0/3756769625 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14bc06df20 con 0x7f14bc144340 2024-04-25T11:27:23.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.307+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f14bc0b7090 con 0x7f14bc144340 2024-04-25T11:27:23.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.518+0000 7f14b3400640 1 -- 172.21.15.19:0/3756769625 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f14a405ee90 con 0x7f14bc144340 2024-04-25T11:27:23.510 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T11:27:23.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14980744e0 msgr2=0x7f14980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14980744e0 0x7f14980769f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009a30 tx=0x7f14ac008040 comp rx=0 tx=0).stop 2024-04-25T11:27:23.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc144340 msgr2=0x7f14bc158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc144340 0x7f14bc158c00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c3c0 tx=0x7f14a400c7c0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 shutdown_connections 2024-04-25T11:27:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14980744e0 0x7f14980769f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14bc15ee60 0x7f14bc159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14bc144340 0x7f14bc158c00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 --2- 172.21.15.19:0/3756769625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14bc06db10 0x7f14bc0b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.523+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 >> 172.21.15.19:0/3756769625 conn(0x7f14bc072f40 msgr2=0x7f14bc1419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:23.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.524+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 shutdown_connections 2024-04-25T11:27:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.524+0000 7f14c1000640 1 -- 172.21.15.19:0/3756769625 wait complete. 2024-04-25T11:27:23.551 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:27:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.835+0000 7f2852a00640 1 Processor -- start 2024-04-25T11:27:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.835+0000 7f2852a00640 1 -- start start 2024-04-25T11:27:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.837+0000 7f2852a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 0x7f284c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.837+0000 7f2852a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f284c15adf0 0x7f284c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f2852a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f284c06df20 con 0x7f284c06db10 2024-04-25T11:27:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f2852a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f284c0713f0 con 0x7f284c15adf0 2024-04-25T11:27:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f2852a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f284c0701a0 con 0x7f284c070630 2024-04-25T11:27:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f2850a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f284c06db10 0x7f284c0b59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52568/0 (socket says 172.21.15.19:52568) 2024-04-25T11:27:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f2850a00640 1 -- 172.21.15.19:0/2165651514 learned_addr learned my addr 172.21.15.19:0/2165651514 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f284be00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 0x7f284c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f2843400640 1 --2- 172.21.15.19:0/2165651514 >> v2:172.21.15.169:3300/0 conn(0x7f284c15adf0 0x7f284c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1245707261 0 0) 0x7f2844001490 con 0x7f284c06db10 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.838+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2830003520 con 0x7f284c06db10 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1088882647 0 0) 0x7f2830003520 con 0x7f284c06db10 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 msgr2=0x7f284c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 0x7f284c0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 >> v2:172.21.15.169:3300/0 conn(0x7f284c15adf0 msgr2=0x7f284c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 --2- 172.21.15.19:0/2165651514 >> v2:172.21.15.169:3300/0 conn(0x7f284c15adf0 0x7f284c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c071110 con 0x7f284c06db10 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2844001bc0 con 0x7f284c06db10 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 >> v1:172.21.15.19:6789/0 conn(0x7f284c06db10 legacy=0x7f284c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28300039b0 0x7f2830003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 0x7f2830004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2830004d40 0x7f2830005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0713f0 con 0x7f2830004d40 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0701a0 con 0x7f28300039b0 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f283000ba40 con 0x7f284c070630 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.839+0000 7f2850a00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2830004d40 0x7f2830005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f284be00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28300039b0 0x7f2830003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f2843400640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 0x7f2830004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f2850a00640 1 -- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 msgr2=0x7f2830004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f2850a00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f284c070630 0x7f2830004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f2850a00640 1 -- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28300039b0 msgr2=0x7f2830003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f2850a00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28300039b0 0x7f2830003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f2850a00640 1 -- 172.21.15.19:0/2165651514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2830003520 con 0x7f2830004d40 2024-04-25T11:27:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.840+0000 7f284be00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28300039b0 0x7f2830003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.841+0000 7f2850a00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2830004d40 0x7f2830005150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f284c074480 tx=0x7f2830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea6d581bb7eb274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.841+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400a690 con 0x7f2830004d40 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.841+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284402f070 con 0x7f2830004d40 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.841+0000 7f284b400640 1 -- 172.21.15.19:0/2165651514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284402be40 con 0x7f2830004d40 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.842+0000 7f2852a00640 1 -- 172.21.15.19:0/2165651514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2830004d40 msgr2=0x7f2830005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.842+0000 7f2852a00640 1 --2- 172.21.15.19:0/2165651514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2830004d40 0x7f2830005150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f284c074480 tx=0x7f2830005830 comp rx=0 tx=0).stop 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.842+0000 7f2850a00640 1 -- 172.21.15.19:0/2165651514 reap_dead start 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.842+0000 7f2852a00640 1 -- 172.21.15.19:0/2165651514 shutdown_connections 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.842+0000 7f2852a00640 1 -- 172.21.15.19:0/2165651514 >> 172.21.15.19:0/2165651514 conn(0x7f284c073090 msgr2=0x7f284c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.842+0000 7f2852a00640 1 -- 172.21.15.19:0/2165651514 shutdown_connections 2024-04-25T11:27:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.842+0000 7f2852a00640 1 -- 172.21.15.19:0/2165651514 wait complete. 2024-04-25T11:27:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.844+0000 7f2852a00640 1 Processor -- start 2024-04-25T11:27:23.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.844+0000 7f2852a00640 1 -- start start 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2852a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2830004d40 0x7f284c146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2852a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c15cab0 0x7f284c149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2852a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c15cec0 0x7f284c149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2852a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0b0d00 con 0x7f284c15cec0 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2852a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f284c06fe30 con 0x7f284c15cab0 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2852a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f284c0b0e80 con 0x7f2830004d40 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2850a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c15cec0 0x7f284c149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2850a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c15cec0 0x7f284c149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46946/0 (socket says 172.21.15.19:46946) 2024-04-25T11:27:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.846+0000 7f2850a00640 1 -- 172.21.15.19:0/1570488188 learned_addr learned my addr 172.21.15.19:0/1570488188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f284be00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2830004d40 0x7f284c146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f2850a00640 1 -- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2830004d40 msgr2=0x7f284c146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f2850a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2830004d40 0x7f284c146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f2850a00640 1 -- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c15cab0 msgr2=0x7f284c149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f2850a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c15cab0 0x7f284c149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f2850a00640 1 -- 172.21.15.19:0/1570488188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c15bc10 con 0x7f284c15cec0 2024-04-25T11:27:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f284be00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2830004d40 0x7f284c146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f2850a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c15cec0 0x7f284c149850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2844000f80 tx=0x7f284c0712e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.847+0000 7f2848c00640 1 -- 172.21.15.19:0/1570488188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844039070 con 0x7f284c15cec0 2024-04-25T11:27:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.848+0000 7f2848c00640 1 -- 172.21.15.19:0/1570488188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2844001bb0 con 0x7f284c15cec0 2024-04-25T11:27:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.848+0000 7f2848c00640 1 -- 172.21.15.19:0/1570488188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400a690 con 0x7f284c15cec0 2024-04-25T11:27:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.848+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c147410 con 0x7f284c15cec0 2024-04-25T11:27:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.848+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c15c3f0 con 0x7f284c15cec0 2024-04-25T11:27:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.849+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c15be80 con 0x7f284c15cec0 2024-04-25T11:27:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.850+0000 7f2848c00640 1 -- 172.21.15.19:0/1570488188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2844044060 con 0x7f284c15cec0 2024-04-25T11:27:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.851+0000 7f2848c00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28280744b0 0x7f28280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.851+0000 7f2848c00640 1 -- 172.21.15.19:0/1570488188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f28440b4d30 con 0x7f284c15cec0 2024-04-25T11:27:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.852+0000 7f284be00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28280744b0 0x7f28280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:23.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.853+0000 7f284be00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28280744b0 0x7f28280769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f284c0cfd00 tx=0x7f283c00b3a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:23.857+0000 7f2848c00640 1 -- 172.21.15.19:0/1570488188 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f284c06df20 con 0x7f284c15cec0 2024-04-25T11:27:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.060+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f284c155b50 con 0x7f284c15cec0 2024-04-25T11:27:24.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.533+0000 7f2848c00640 1 -- 172.21.15.19:0/1570488188 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f284407f600 con 0x7f284c15cec0 2024-04-25T11:27:24.524 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T11:27:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.537+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28280744b0 msgr2=0x7f28280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.537+0000 7f2852a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28280744b0 0x7f28280769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f284c0cfd00 tx=0x7f283c00b3a0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.537+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c15cec0 msgr2=0x7f284c149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.537+0000 7f2852a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c15cec0 0x7f284c149850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2844000f80 tx=0x7f284c0712e0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.538+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 shutdown_connections 2024-04-25T11:27:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.538+0000 7f2852a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28280744b0 0x7f28280769c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.538+0000 7f2852a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c15cab0 0x7f284c149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.538+0000 7f2852a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f284c15cec0 0x7f284c149850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.538+0000 7f2852a00640 1 --2- 172.21.15.19:0/1570488188 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2830004d40 0x7f284c146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.538+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 >> 172.21.15.19:0/1570488188 conn(0x7f284c073090 msgr2=0x7f284c0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.539+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 shutdown_connections 2024-04-25T11:27:24.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.540+0000 7f2852a00640 1 -- 172.21.15.19:0/1570488188 wait complete. 2024-04-25T11:27:24.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:27:24.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.839+0000 7fde9f400640 1 Processor -- start 2024-04-25T11:27:24.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.840+0000 7fde9f400640 1 -- start start 2024-04-25T11:27:24.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.841+0000 7fde9f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde98158850 0x7fde98070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:24.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9f400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9815ef90 con 0x7fde9815e780 2024-04-25T11:27:24.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9f400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9815f270 con 0x7fde9806d5d0 2024-04-25T11:27:24.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9f400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fde98158c60 con 0x7fde98158850 2024-04-25T11:27:24.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde98158850 0x7fde98070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:24.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde98158850 0x7fde98070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46960/0 (socket says 172.21.15.19:46960) 2024-04-25T11:27:24.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9ca00640 1 -- 172.21.15.19:0/2437601159 learned_addr learned my addr 172.21.15.19:0/2437601159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:24.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9ca00640 1 -- 172.21.15.19:0/2437601159 >> v1:172.21.15.169:6789/0 conn(0x7fde9806d5d0 legacy=0x7fde9806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:24.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9ca00640 1 -- 172.21.15.19:0/2437601159 >> v1:172.21.15.19:6789/0 conn(0x7fde9815e780 legacy=0x7fde9815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:24.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.842+0000 7fde9ca00640 1 -- 172.21.15.19:0/2437601159 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9815f550 con 0x7fde98158850 2024-04-25T11:27:24.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.843+0000 7fde9ca00640 1 --2- 172.21.15.19:0/2437601159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98158850 0x7fde98070820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde88009870 tx=0x7fde8802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd84db5bc4af6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:24.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.844+0000 7fde97400640 1 -- 172.21.15.19:0/2437601159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88037070 con 0x7fde98158850 2024-04-25T11:27:24.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.844+0000 7fde97400640 1 -- 172.21.15.19:0/2437601159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde88032440 con 0x7fde98158850 2024-04-25T11:27:24.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.844+0000 7fde97400640 1 -- 172.21.15.19:0/2437601159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88032810 con 0x7fde98158850 2024-04-25T11:27:24.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.845+0000 7fde9f400640 1 -- 172.21.15.19:0/2437601159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98158850 msgr2=0x7fde98070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:24.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.845+0000 7fde9f400640 1 --2- 172.21.15.19:0/2437601159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98158850 0x7fde98070820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde88009870 tx=0x7fde8802a2a0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.845+0000 7fde9f400640 1 -- 172.21.15.19:0/2437601159 shutdown_connections 2024-04-25T11:27:24.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.845+0000 7fde9f400640 1 --2- 172.21.15.19:0/2437601159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde98158850 0x7fde98070820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.845+0000 7fde9f400640 1 -- 172.21.15.19:0/2437601159 >> 172.21.15.19:0/2437601159 conn(0x7fde980730b0 msgr2=0x7fde980734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:24.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.845+0000 7fde9f400640 1 -- 172.21.15.19:0/2437601159 shutdown_connections 2024-04-25T11:27:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.845+0000 7fde9f400640 1 -- 172.21.15.19:0/2437601159 wait complete. 2024-04-25T11:27:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.847+0000 7fde9f400640 1 Processor -- start 2024-04-25T11:27:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.847+0000 7fde9f400640 1 -- start start 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde9806d5d0 0x7fde9814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde98158850 0x7fde9814b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9815e780 0x7fde981f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9f400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde9815eb90 con 0x7fde9806d5d0 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fde98160010 con 0x7fde9815e780 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fde9815ed10 con 0x7fde98158850 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde9806d5d0 0x7fde9814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde9806d5d0 0x7fde9814b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46976/0 (socket says 172.21.15.19:46976) 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.849+0000 7fde9ca00640 1 -- 172.21.15.19:0/3951250017 learned_addr learned my addr 172.21.15.19:0/3951250017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:24.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde97e00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde98158850 0x7fde9814b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde9d400640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9815e780 0x7fde981f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde9ca00640 1 -- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde98158850 msgr2=0x7fde9814b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde9ca00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde98158850 0x7fde9814b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde9ca00640 1 -- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9815e780 msgr2=0x7fde981f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde9ca00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9815e780 0x7fde981f9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde9ca00640 1 -- 172.21.15.19:0/3951250017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde98156510 con 0x7fde9806d5d0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde97e00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde98158850 0x7fde9814b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde9ca00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde9806d5d0 0x7fde9814b030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde88000c00 tx=0x7fde880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde94c00640 1 -- 172.21.15.19:0/3951250017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88041070 con 0x7fde9806d5d0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde94c00640 1 -- 172.21.15.19:0/3951250017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8803c870 con 0x7fde9806d5d0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.850+0000 7fde94c00640 1 -- 172.21.15.19:0/3951250017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88037040 con 0x7fde9806d5d0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.851+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde981567f0 con 0x7fde9806d5d0 2024-04-25T11:27:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.851+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde98156db0 con 0x7fde9806d5d0 2024-04-25T11:27:24.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.852+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde980761c0 con 0x7fde9806d5d0 2024-04-25T11:27:24.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.853+0000 7fde94c00640 1 -- 172.21.15.19:0/3951250017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde88044070 con 0x7fde9806d5d0 2024-04-25T11:27:24.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.855+0000 7fde94c00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde780744b0 0x7fde780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:24.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.855+0000 7fde94c00640 1 -- 172.21.15.19:0/3951250017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7fde880b5ed0 con 0x7fde9806d5d0 2024-04-25T11:27:24.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.855+0000 7fde97e00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde780744b0 0x7fde780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:24.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.856+0000 7fde97e00640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde780744b0 0x7fde780769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde980cb6e0 tx=0x7fde84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:24.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:24.860+0000 7fde94c00640 1 -- 172.21.15.19:0/3951250017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde88080620 con 0x7fde9806d5d0 2024-04-25T11:27:25.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.101+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fde98050740 con 0x7fde9806d5d0 2024-04-25T11:27:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.557+0000 7fde94c00640 1 -- 172.21.15.19:0/3951250017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fde8807ff20 con 0x7fde9806d5d0 2024-04-25T11:27:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T11:27:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.569+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde780744b0 msgr2=0x7fde780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.569+0000 7fde9f400640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde780744b0 0x7fde780769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde980cb6e0 tx=0x7fde84009040 comp rx=0 tx=0).stop 2024-04-25T11:27:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.569+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde9806d5d0 msgr2=0x7fde9814b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.569+0000 7fde9f400640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde9806d5d0 0x7fde9814b030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde88000c00 tx=0x7fde880046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.572+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 shutdown_connections 2024-04-25T11:27:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.572+0000 7fde9f400640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde780744b0 0x7fde780769c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.572+0000 7fde9f400640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde9815e780 0x7fde981f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.572+0000 7fde9f400640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde98158850 0x7fde9814b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.572+0000 7fde9f400640 1 --2- 172.21.15.19:0/3951250017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde9806d5d0 0x7fde9814b030 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.572+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 >> 172.21.15.19:0/3951250017 conn(0x7fde980730b0 msgr2=0x7fde98157d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.573+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 shutdown_connections 2024-04-25T11:27:25.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.573+0000 7fde9f400640 1 -- 172.21.15.19:0/3951250017 wait complete. 2024-04-25T11:27:25.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:25.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.868+0000 7f771b200640 1 Processor -- start 2024-04-25T11:27:25.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.868+0000 7f771b200640 1 -- start start 2024-04-25T11:27:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f771b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f771c0d50e0 0x7f771c0d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f771b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c160d60 0x7f771c161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f771b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771c0b7940 con 0x7f771c0d2ae0 2024-04-25T11:27:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f771b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f771c0d2ef0 con 0x7f771c0d50e0 2024-04-25T11:27:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f771b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f771c161280 con 0x7f771c160d60 2024-04-25T11:27:25.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f7719e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f771c0d50e0 0x7f771c0d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f7719e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f771c0d50e0 0x7f771c0d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46980/0 (socket says 172.21.15.19:46980) 2024-04-25T11:27:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f7719e00640 1 -- 172.21.15.19:0/293313161 learned_addr learned my addr 172.21.15.19:0/293313161 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.870+0000 7f771a800640 1 --2- 172.21.15.19:0/293313161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c160d60 0x7f771c161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.871+0000 7f7719e00640 1 -- 172.21.15.19:0/293313161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c160d60 msgr2=0x7f771c161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.871+0000 7f7719e00640 1 --2- 172.21.15.19:0/293313161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c160d60 0x7f771c161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.871+0000 7f7719e00640 1 -- 172.21.15.19:0/293313161 >> v1:172.21.15.169:6789/0 conn(0x7f771c0d2ae0 legacy=0x7f771c160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:25.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.871+0000 7f7719e00640 1 -- 172.21.15.19:0/293313161 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c0d4600 con 0x7f771c0d50e0 2024-04-25T11:27:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.872+0000 7f7719e00640 1 --2- 172.21.15.19:0/293313161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c0d50e0 0x7f771c0d25a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f770c009870 tx=0x7f770c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f297a54ab96ef3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.872+0000 7f7718a00640 1 -- 172.21.15.19:0/293313161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c037070 con 0x7f771c0d50e0 2024-04-25T11:27:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.872+0000 7f7718a00640 1 -- 172.21.15.19:0/293313161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c032440 con 0x7f771c0d50e0 2024-04-25T11:27:25.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.872+0000 7f7718a00640 1 -- 172.21.15.19:0/293313161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c032810 con 0x7f771c0d50e0 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 -- 172.21.15.19:0/293313161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c0d50e0 msgr2=0x7f771c0d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 --2- 172.21.15.19:0/293313161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c0d50e0 0x7f771c0d25a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f770c009870 tx=0x7f770c02a200 comp rx=0 tx=0).stop 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 -- 172.21.15.19:0/293313161 shutdown_connections 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 --2- 172.21.15.19:0/293313161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c160d60 0x7f771c161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 --2- 172.21.15.19:0/293313161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c0d50e0 0x7f771c0d25a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 -- 172.21.15.19:0/293313161 >> 172.21.15.19:0/293313161 conn(0x7f771c06c5c0 msgr2=0x7f771c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 -- 172.21.15.19:0/293313161 shutdown_connections 2024-04-25T11:27:25.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.873+0000 7f771b200640 1 -- 172.21.15.19:0/293313161 wait complete. 2024-04-25T11:27:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.875+0000 7f771b200640 1 Processor -- start 2024-04-25T11:27:25.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.875+0000 7f771b200640 1 -- start start 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f771c0d2ae0 0x7f771c1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c0d50e0 0x7f771c0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c160d60 0x7f771c156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f771c161560 con 0x7f771c160d60 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f771c0d4bc0 con 0x7f771c0d2ae0 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f771c0d54f0 con 0x7f771c0d50e0 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c160d60 0x7f771c156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c160d60 0x7f771c156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46994/0 (socket says 172.21.15.19:46994) 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f771a800640 1 -- 172.21.15.19:0/1644636037 learned_addr learned my addr 172.21.15.19:0/1644636037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f7719e00640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f771c0d2ae0 0x7f771c1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.877+0000 7f7719400640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c0d50e0 0x7f771c0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771a800640 1 -- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c0d50e0 msgr2=0x7f771c0d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:25.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771a800640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c0d50e0 0x7f771c0d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771a800640 1 -- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f771c0d2ae0 msgr2=0x7f771c1f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:25.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771a800640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f771c0d2ae0 0x7f771c1f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:25.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771a800640 1 -- 172.21.15.19:0/1644636037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c159390 con 0x7f771c160d60 2024-04-25T11:27:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771a800640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c160d60 0x7f771c156040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f771400cc60 tx=0x7f771400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f7712000640 1 -- 172.21.15.19:0/1644636037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714003990 con 0x7f771c160d60 2024-04-25T11:27:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f7712000640 1 -- 172.21.15.19:0/1644636037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7714003b30 con 0x7f771c160d60 2024-04-25T11:27:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c1f73a0 con 0x7f771c160d60 2024-04-25T11:27:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f7712000640 1 -- 172.21.15.19:0/1644636037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714004020 con 0x7f771c160d60 2024-04-25T11:27:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.878+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c159650 con 0x7f771c160d60 2024-04-25T11:27:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.879+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771c1f7110 con 0x7f771c160d60 2024-04-25T11:27:25.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.881+0000 7f7712000640 1 -- 172.21.15.19:0/1644636037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f77140041c0 con 0x7f771c160d60 2024-04-25T11:27:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.882+0000 7f7712000640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76f8074500 0x7f76f8076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.882+0000 7f7712000640 1 -- 172.21.15.19:0/1644636037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f77140955d0 con 0x7f771c160d60 2024-04-25T11:27:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.882+0000 7f7719e00640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76f8074500 0x7f76f8076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:25.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.884+0000 7f7719e00640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76f8074500 0x7f76f8076a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f770c034000 tx=0x7f770c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:25.886+0000 7f7712000640 1 -- 172.21.15.19:0/1644636037 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7714014030 con 0x7f771c160d60 2024-04-25T11:27:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.086+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f771c0b8aa0 con 0x7f771c160d60 2024-04-25T11:27:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.088+0000 7f7712000640 1 -- 172.21.15.19:0/1644636037 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16622 (secure 0 0 0) 0x7f771405fe40 con 0x7f771c160d60 2024-04-25T11:27:26.080 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:26.080 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":21,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:25.103074+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76f8074500 msgr2=0x7f76f8076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76f8074500 0x7f76f8076a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f770c034000 tx=0x7f770c02e000 comp rx=0 tx=0).stop 2024-04-25T11:27:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c160d60 msgr2=0x7f771c156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c160d60 0x7f771c156040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f771400cc60 tx=0x7f771400a5c0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 shutdown_connections 2024-04-25T11:27:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76f8074500 0x7f76f8076a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f771c160d60 0x7f771c156040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f771c0d50e0 0x7f771c0d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 --2- 172.21.15.19:0/1644636037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f771c0d2ae0 0x7f771c1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 >> 172.21.15.19:0/1644636037 conn(0x7f771c06c5c0 msgr2=0x7f771c15deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:26.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 shutdown_connections 2024-04-25T11:27:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.091+0000 7f771b200640 1 -- 172.21.15.19:0/1644636037 wait complete. 2024-04-25T11:27:26.106 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.396+0000 7fa21b200640 1 Processor -- start 2024-04-25T11:27:26.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.396+0000 7fa21b200640 1 -- start start 2024-04-25T11:27:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.398+0000 7fa21b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 0x7fa21c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.398+0000 7fa21b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa21c0d1230 con 0x7fa21c144370 2024-04-25T11:27:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.398+0000 7fa21b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa21c0d06b0 con 0x7fa21c0b4ea0 2024-04-25T11:27:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.398+0000 7fa21b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa21c0d0990 con 0x7fa21c06c5c0 2024-04-25T11:27:26.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.398+0000 7fa213e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa21c144370 0x7fa21c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52592/0 (socket says 172.21.15.19:52592) 2024-04-25T11:27:26.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.398+0000 7fa213e00640 1 -- 172.21.15.19:0/3488459876 learned_addr learned my addr 172.21.15.19:0/3488459876 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:26.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.398+0000 7fa219e00640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 0x7fa21c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3889559496 0 0) 0x7fa21c0d1230 con 0x7fa21c144370 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa200003520 con 0x7fa21c144370 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 969236602 0 0) 0x7fa200003520 con 0x7fa21c144370 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 msgr2=0x7fa21c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 0x7fa21c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 >> v1:172.21.15.169:6789/0 conn(0x7fa21c0b4ea0 legacy=0x7fa21c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c0d0c70 con 0x7fa21c144370 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa204001670 con 0x7fa21c144370 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 >> v1:172.21.15.19:6789/0 conn(0x7fa21c144370 legacy=0x7fa21c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 0x7fa2000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200003f60 0x7fa200004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa200004d40 0x7fa200005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0d0990 con 0x7fa200003f60 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa20000bb50 con 0x7fa200004d40 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa20000bd50 con 0x7fa21c06c5c0 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.399+0000 7fa213e00640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200003f60 0x7fa200004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa21a800640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa200004d40 0x7fa200005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa219e00640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 0x7fa2000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa213e00640 1 -- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 msgr2=0x7fa2000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa213e00640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa21c06c5c0 0x7fa2000039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa213e00640 1 -- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa200004d40 msgr2=0x7fa200005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:26.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa213e00640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa200004d40 0x7fa200005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa213e00640 1 -- 172.21.15.19:0/3488459876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa200003520 con 0x7fa200003f60 2024-04-25T11:27:26.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.400+0000 7fa21a800640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa200004d40 0x7fa200005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:26.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.401+0000 7fa213e00640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200003f60 0x7fa200004800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0b5550 tx=0x7fa200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945891e03442701b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:26.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.401+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204007d10 con 0x7fa200003f60 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.401+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa204005ad0 con 0x7fa200003f60 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.401+0000 7fa219400640 1 -- 172.21.15.19:0/3488459876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204007070 con 0x7fa200003f60 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.402+0000 7fa21b200640 1 -- 172.21.15.19:0/3488459876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200003f60 msgr2=0x7fa200004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.402+0000 7fa21b200640 1 --2- 172.21.15.19:0/3488459876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa200003f60 0x7fa200004800 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0b5550 tx=0x7fa200005830 comp rx=0 tx=0).stop 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.402+0000 7fa21a800640 1 -- 172.21.15.19:0/3488459876 reap_dead start 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.402+0000 7fa21b200640 1 -- 172.21.15.19:0/3488459876 shutdown_connections 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.402+0000 7fa21b200640 1 -- 172.21.15.19:0/3488459876 >> 172.21.15.19:0/3488459876 conn(0x7fa21c0d1720 msgr2=0x7fa21c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.402+0000 7fa21b200640 1 -- 172.21.15.19:0/3488459876 shutdown_connections 2024-04-25T11:27:26.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.402+0000 7fa21b200640 1 -- 172.21.15.19:0/3488459876 wait complete. 2024-04-25T11:27:26.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.405+0000 7fa21b200640 1 Processor -- start 2024-04-25T11:27:26.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.405+0000 7fa21b200640 1 -- start start 2024-04-25T11:27:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.406+0000 7fa21b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa200003f60 0x7fa21c13fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.406+0000 7fa21b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa21c133d20 0x7fa21c162320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.406+0000 7fa21b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c131060 0x7fa21c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.406+0000 7fa21b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0d0450 con 0x7fa21c133d20 2024-04-25T11:27:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.406+0000 7fa21b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0d02d0 con 0x7fa21c131060 2024-04-25T11:27:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.406+0000 7fa21b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa21c147760 con 0x7fa200003f60 2024-04-25T11:27:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa21c133d20 0x7fa21c162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa21c133d20 0x7fa21c162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47010/0 (socket says 172.21.15.19:47010) 2024-04-25T11:27:26.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 -- 172.21.15.19:0/2176200752 learned_addr learned my addr 172.21.15.19:0/2176200752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa219e00640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa200003f60 0x7fa21c13fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa21a800640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c131060 0x7fa21c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 -- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa200003f60 msgr2=0x7fa21c13fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa200003f60 0x7fa21c13fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 -- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c131060 msgr2=0x7fa21c145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c131060 0x7fa21c145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 -- 172.21.15.19:0/2176200752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c144a60 con 0x7fa21c133d20 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.407+0000 7fa213e00640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa21c133d20 0x7fa21c162320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa204001970 tx=0x7fa21c0d0880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:26.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.408+0000 7fa212000640 1 -- 172.21.15.19:0/2176200752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204040020 con 0x7fa21c133d20 2024-04-25T11:27:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.408+0000 7fa212000640 1 -- 172.21.15.19:0/2176200752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa204001430 con 0x7fa21c133d20 2024-04-25T11:27:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.408+0000 7fa212000640 1 -- 172.21.15.19:0/2176200752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204037040 con 0x7fa21c133d20 2024-04-25T11:27:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.408+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c144d40 con 0x7fa21c133d20 2024-04-25T11:27:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.408+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c145300 con 0x7fa21c133d20 2024-04-25T11:27:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.409+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21c1447b0 con 0x7fa21c133d20 2024-04-25T11:27:26.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.410+0000 7fa212000640 1 -- 172.21.15.19:0/2176200752 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa204043020 con 0x7fa21c133d20 2024-04-25T11:27:26.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.412+0000 7fa212000640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1f80744b0 0x7fa1f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.412+0000 7fa212000640 1 -- 172.21.15.19:0/2176200752 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa2040b4810 con 0x7fa21c133d20 2024-04-25T11:27:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.412+0000 7fa219e00640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1f80744b0 0x7fa1f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.414+0000 7fa219e00640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1f80744b0 0x7fa1f80769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009a10 tx=0x7fa20c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:26.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.417+0000 7fa212000640 1 -- 172.21.15.19:0/2176200752 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa204005aa0 con 0x7fa21c133d20 2024-04-25T11:27:26.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.632+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa21c050740 con 0x7fa21c133d20 2024-04-25T11:27:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.633+0000 7fa212000640 1 -- 172.21.15.19:0/2176200752 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7fa20407f020 con 0x7fa21c133d20 2024-04-25T11:27:26.625 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:26.625 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1f80744b0 msgr2=0x7fa1f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1f80744b0 0x7fa1f80769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009a10 tx=0x7fa20c008040 comp rx=0 tx=0).stop 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa21c133d20 msgr2=0x7fa21c162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa21c133d20 0x7fa21c162320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa204001970 tx=0x7fa21c0d0880 comp rx=0 tx=0).stop 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 shutdown_connections 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1f80744b0 0x7fa1f80769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa21c131060 0x7fa21c145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa21c133d20 0x7fa21c162320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 --2- 172.21.15.19:0/2176200752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa200003f60 0x7fa21c13fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 >> 172.21.15.19:0/2176200752 conn(0x7fa21c0d1720 msgr2=0x7fa21c15e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.636+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 shutdown_connections 2024-04-25T11:27:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.637+0000 7fa21b200640 1 -- 172.21.15.19:0/2176200752 wait complete. 2024-04-25T11:27:26.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:27:26.835 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T11:27:26.843+0000 7f4183600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:27:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.938+0000 7f25b8600640 1 Processor -- start 2024-04-25T11:27:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.938+0000 7f25b8600640 1 -- start start 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.940+0000 7f25b8600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25b0161710 0x7f25b015e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.940+0000 7f25b8600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25b006c5c0 0x7f25b006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.940+0000 7f25b8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 0x7f25b01643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.940+0000 7f25b8600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f25b01609b0 con 0x7f25b0161710 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.940+0000 7f25b8600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f25b006cf10 con 0x7f25b006c5c0 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.940+0000 7f25b8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25b0164cd0 con 0x7f25b0163fa0 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25b0161710 0x7f25b015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25b0161710 0x7f25b015e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47022/0 (socket says 172.21.15.19:47022) 2024-04-25T11:27:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 -- 172.21.15.19:0/2337339424 learned_addr learned my addr 172.21.15.19:0/2337339424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 -- 172.21.15.19:0/2337339424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 msgr2=0x7f25b01643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 --2- 172.21.15.19:0/2337339424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 0x7f25b01643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 -- 172.21.15.19:0/2337339424 >> v2:172.21.15.169:3300/0 conn(0x7f25b006c5c0 msgr2=0x7f25b006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5200640 1 --2- 172.21.15.19:0/2337339424 >> v2:172.21.15.169:3300/0 conn(0x7f25b006c5c0 0x7f25b006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 --2- 172.21.15.19:0/2337339424 >> v2:172.21.15.169:3300/0 conn(0x7f25b006c5c0 0x7f25b006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5c00640 1 -- 172.21.15.19:0/2337339424 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b0160c30 con 0x7f25b0161710 2024-04-25T11:27:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.941+0000 7f25b5200640 1 --2- 172.21.15.19:0/2337339424 >> v2:172.21.15.169:3300/0 conn(0x7f25b006c5c0 0x7f25b006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.942+0000 7f25b5c00640 1 --2- 172.21.15.19:0/2337339424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b015e080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25b00d2770 tx=0x7f25a402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb850c19c9a99b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.943+0000 7f25afe00640 1 -- 172.21.15.19:0/2337339424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4037070 con 0x7f25b0161710 2024-04-25T11:27:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.943+0000 7f25afe00640 1 -- 172.21.15.19:0/2337339424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a4032440 con 0x7f25b0161710 2024-04-25T11:27:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.943+0000 7f25afe00640 1 -- 172.21.15.19:0/2337339424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4032810 con 0x7f25b0161710 2024-04-25T11:27:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 -- 172.21.15.19:0/2337339424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 msgr2=0x7f25b015e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 --2- 172.21.15.19:0/2337339424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b015e080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f25b00d2770 tx=0x7f25a402a160 comp rx=0 tx=0).stop 2024-04-25T11:27:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 -- 172.21.15.19:0/2337339424 shutdown_connections 2024-04-25T11:27:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 --2- 172.21.15.19:0/2337339424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 0x7f25b01643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 --2- 172.21.15.19:0/2337339424 >> v2:172.21.15.169:3300/0 conn(0x7f25b006c5c0 0x7f25b006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 --2- 172.21.15.19:0/2337339424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b015e080 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 -- 172.21.15.19:0/2337339424 >> 172.21.15.19:0/2337339424 conn(0x7f25b00d5d90 msgr2=0x7f25b00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 -- 172.21.15.19:0/2337339424 shutdown_connections 2024-04-25T11:27:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.944+0000 7f25b8600640 1 -- 172.21.15.19:0/2337339424 wait complete. 2024-04-25T11:27:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.946+0000 7f25b8600640 1 Processor -- start 2024-04-25T11:27:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.946+0000 7f25b8600640 1 -- start start 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b006c5c0 0x7f25b0155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b8600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b00cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 0x7f25b00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b8600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25b01648f0 con 0x7f25b0161710 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25b01611f0 con 0x7f25b006c5c0 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25b0164a70 con 0x7f25b0163fa0 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b00cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b00cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47030/0 (socket says 172.21.15.19:47030) 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b5200640 1 -- 172.21.15.19:0/4283414173 learned_addr learned my addr 172.21.15.19:0/4283414173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.948+0000 7f25b5c00640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b006c5c0 0x7f25b0155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b6600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 0x7f25b00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b5200640 1 -- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 msgr2=0x7f25b00c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b5200640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 0x7f25b00c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b5200640 1 -- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b006c5c0 msgr2=0x7f25b0155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b5200640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b006c5c0 0x7f25b0155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b5200640 1 -- 172.21.15.19:0/4283414173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00bace0 con 0x7f25b0161710 2024-04-25T11:27:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b5200640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b00cc0f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f259c00c7f0 tx=0x7f259c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25ad600640 1 -- 172.21.15.19:0/4283414173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c015070 con 0x7f25b0161710 2024-04-25T11:27:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25ad600640 1 -- 172.21.15.19:0/4283414173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c007820 con 0x7f25b0161710 2024-04-25T11:27:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25ad600640 1 -- 172.21.15.19:0/4283414173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c007bd0 con 0x7f25b0161710 2024-04-25T11:27:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.949+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b00bafc0 con 0x7f25b0161710 2024-04-25T11:27:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.950+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b00bb580 con 0x7f25b0161710 2024-04-25T11:27:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.951+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b0050740 con 0x7f25b0161710 2024-04-25T11:27:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.952+0000 7f25ad600640 1 -- 172.21.15.19:0/4283414173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f259c018070 con 0x7f25b0161710 2024-04-25T11:27:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.953+0000 7f25ad600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25900744b0 0x7f25900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.953+0000 7f25ad600640 1 -- 172.21.15.19:0/4283414173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f259c0954a0 con 0x7f25b0161710 2024-04-25T11:27:26.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.954+0000 7f25b5c00640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25900744b0 0x7f25900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:26.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.955+0000 7f25b5c00640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25900744b0 0x7f25900769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25a4034000 tx=0x7f25a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:26.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:26.958+0000 7f25ad600640 1 -- 172.21.15.19:0/4283414173 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f259c05fb90 con 0x7f25b0161710 2024-04-25T11:27:27.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.206+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f25b0160080 con 0x7f25b0161710 2024-04-25T11:27:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.585+0000 7f25ad600640 1 -- 172.21.15.19:0/4283414173 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f259c05f490 con 0x7f25b0161710 2024-04-25T11:27:27.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.589+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25900744b0 msgr2=0x7f25900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:27.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.589+0000 7f25b8600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25900744b0 0x7f25900769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25a4034000 tx=0x7f25a402e000 comp rx=0 tx=0).stop 2024-04-25T11:27:27.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 msgr2=0x7f25b00cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:27.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b00cc0f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f259c00c7f0 tx=0x7f259c00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 shutdown_connections 2024-04-25T11:27:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25900744b0 0x7f25900769c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25b0163fa0 0x7f25b00c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25b0161710 0x7f25b00cc0f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 --2- 172.21.15.19:0/4283414173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25b006c5c0 0x7f25b0155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 >> 172.21.15.19:0/4283414173 conn(0x7f25b00d5d90 msgr2=0x7f25b015d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.590+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 shutdown_connections 2024-04-25T11:27:27.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.591+0000 7f25b8600640 1 -- 172.21.15.19:0/4283414173 wait complete. 2024-04-25T11:27:27.617 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:27:27.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:27:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.919+0000 7f3c19800640 1 Processor -- start 2024-04-25T11:27:27.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.919+0000 7f3c19800640 1 -- start start 2024-04-25T11:27:27.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c19800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3c1406d5d0 0x7f3c1406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:27.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c19800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c140b50b0 0x7f3c140b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:27.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c19800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1406df20 con 0x7f3c1414aac0 2024-04-25T11:27:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c19800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3c1414aed0 con 0x7f3c1406d5d0 2024-04-25T11:27:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c19800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1406f460 con 0x7f3c140b50b0 2024-04-25T11:27:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c0b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3c1406d5d0 0x7f3c1406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c13e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3c1414aac0 0x7f3c141655a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52604/0 (socket says 172.21.15.19:52604) 2024-04-25T11:27:27.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.921+0000 7f3c13e00640 1 -- 172.21.15.19:0/1362009379 learned_addr learned my addr 172.21.15.19:0/1362009379 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.922+0000 7f3c18e00640 1 --2- 172.21.15.19:0/1362009379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c140b50b0 0x7f3c140b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.922+0000 7f3c0b400640 1 -- 172.21.15.19:0/1362009379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c140b50b0 msgr2=0x7f3c140b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.922+0000 7f3c0b400640 1 --2- 172.21.15.19:0/1362009379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c140b50b0 0x7f3c140b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.922+0000 7f3c0b400640 1 -- 172.21.15.19:0/1362009379 >> v1:172.21.15.19:6789/0 conn(0x7f3c1414aac0 legacy=0x7f3c141655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.922+0000 7f3c13400640 1 -- 172.21.15.19:0/1362009379 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3841702308 0 0) 0x7f3c04001110 con 0x7f3c1414aac0 2024-04-25T11:27:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.922+0000 7f3c0b400640 1 -- 172.21.15.19:0/1362009379 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1406f740 con 0x7f3c1406d5d0 2024-04-25T11:27:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.923+0000 7f3c0b400640 1 --2- 172.21.15.19:0/1362009379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1406d5d0 0x7f3c1406d9e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0097a0 tx=0x7f3bfc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8246a01dcc5bb233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.923+0000 7f3c13400640 1 -- 172.21.15.19:0/1362009379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc037070 con 0x7f3c1406d5d0 2024-04-25T11:27:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.923+0000 7f3c13400640 1 -- 172.21.15.19:0/1362009379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc02ae50 con 0x7f3c1406d5d0 2024-04-25T11:27:27.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.923+0000 7f3c13400640 1 -- 172.21.15.19:0/1362009379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc032660 con 0x7f3c1406d5d0 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 -- 172.21.15.19:0/1362009379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1406d5d0 msgr2=0x7f3c1406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 --2- 172.21.15.19:0/1362009379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1406d5d0 0x7f3c1406d9e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0097a0 tx=0x7f3bfc02a100 comp rx=0 tx=0).stop 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 -- 172.21.15.19:0/1362009379 shutdown_connections 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 --2- 172.21.15.19:0/1362009379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c140b50b0 0x7f3c140b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 --2- 172.21.15.19:0/1362009379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1406d5d0 0x7f3c1406d9e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 -- 172.21.15.19:0/1362009379 >> 172.21.15.19:0/1362009379 conn(0x7f3c14072f40 msgr2=0x7f3c14073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 -- 172.21.15.19:0/1362009379 shutdown_connections 2024-04-25T11:27:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.924+0000 7f3c19800640 1 -- 172.21.15.19:0/1362009379 wait complete. 2024-04-25T11:27:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.927+0000 7f3c19800640 1 Processor -- start 2024-04-25T11:27:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.927+0000 7f3c19800640 1 -- start start 2024-04-25T11:27:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.928+0000 7f3c19800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1406d5d0 0x7f3c1414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.928+0000 7f3c19800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c140b50b0 0x7f3c141475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.928+0000 7f3c19800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1414aac0 0x7f3c14162a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.928+0000 7f3c19800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1406ffe0 con 0x7f3c1414aac0 2024-04-25T11:27:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.928+0000 7f3c19800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1406fd00 con 0x7f3c140b50b0 2024-04-25T11:27:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.928+0000 7f3c19800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c140b4a90 con 0x7f3c1406d5d0 2024-04-25T11:27:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1414aac0 0x7f3c14162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1414aac0 0x7f3c14162a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47046/0 (socket says 172.21.15.19:47046) 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 -- 172.21.15.19:0/2974557258 learned_addr learned my addr 172.21.15.19:0/2974557258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c13e00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1406d5d0 0x7f3c1414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 -- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1406d5d0 msgr2=0x7f3c1414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1406d5d0 0x7f3c1414dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 -- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c140b50b0 msgr2=0x7f3c141475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c140b50b0 0x7f3c141475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c18e00640 1 -- 172.21.15.19:0/2974557258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c141fe4a0 con 0x7f3c1414aac0 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.929+0000 7f3c13e00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1406d5d0 0x7f3c1414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.930+0000 7f3c18e00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1414aac0 0x7f3c14162a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009fd0 tx=0x7f3c0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.930+0000 7f3c10c00640 1 -- 172.21.15.19:0/2974557258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c014420 con 0x7f3c1414aac0 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.930+0000 7f3c10c00640 1 -- 172.21.15.19:0/2974557258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c019070 con 0x7f3c1414aac0 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.930+0000 7f3c10c00640 1 -- 172.21.15.19:0/2974557258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c014800 con 0x7f3c1414aac0 2024-04-25T11:27:27.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.930+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c141fe780 con 0x7f3c1414aac0 2024-04-25T11:27:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.930+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c141fed40 con 0x7f3c1414aac0 2024-04-25T11:27:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.931+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c141ff120 con 0x7f3c1414aac0 2024-04-25T11:27:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.932+0000 7f3c10c00640 1 -- 172.21.15.19:0/2974557258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c0c0149a0 con 0x7f3c1414aac0 2024-04-25T11:27:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.933+0000 7f3c10c00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bf00744e0 0x7f3bf00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.933+0000 7f3c10c00640 1 -- 172.21.15.19:0/2974557258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3c0c095a30 con 0x7f3c1414aac0 2024-04-25T11:27:27.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.934+0000 7f3c13e00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bf00744e0 0x7f3bf00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.935+0000 7f3c13e00640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bf00744e0 0x7f3bf00769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c04000f60 tx=0x7f3c04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:27.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:27.938+0000 7f3c10c00640 1 -- 172.21.15.19:0/2974557258 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c141ff120 con 0x7f3c1414aac0 2024-04-25T11:27:28.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.177+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3c14076c60 con 0x7f3c1414aac0 2024-04-25T11:27:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.599+0000 7f3c10c00640 1 -- 172.21.15.19:0/2974557258 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f3c0c0601b0 con 0x7f3c1414aac0 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bf00744e0 msgr2=0x7f3bf00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bf00744e0 0x7f3bf00769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c04000f60 tx=0x7f3c04009040 comp rx=0 tx=0).stop 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1414aac0 msgr2=0x7f3c14162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1414aac0 0x7f3c14162a60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009fd0 tx=0x7f3c0c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 shutdown_connections 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bf00744e0 0x7f3bf00769f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c140b50b0 0x7f3c141475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c1414aac0 0x7f3c14162a60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 --2- 172.21.15.19:0/2974557258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c1406d5d0 0x7f3c1414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 >> 172.21.15.19:0/2974557258 conn(0x7f3c14072f40 msgr2=0x7f3c141641e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 shutdown_connections 2024-04-25T11:27:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.603+0000 7f3c19800640 1 -- 172.21.15.19:0/2974557258 wait complete. 2024-04-25T11:27:28.625 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:27:28.626 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:27:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.918+0000 7fa56b200640 1 Processor -- start 2024-04-25T11:27:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.918+0000 7fa56b200640 1 -- start start 2024-04-25T11:27:28.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.920+0000 7fa56b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5640b7c50 0x7fa5640b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:28.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.920+0000 7fa56b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5640d44a0 0x7fa5640d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:28.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.920+0000 7fa56b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa564161650 con 0x7fa564164860 2024-04-25T11:27:28.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.920+0000 7fa56b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa5641643d0 con 0x7fa5640b7c50 2024-04-25T11:27:28.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.920+0000 7fa56b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa5640b5270 con 0x7fa5640d44a0 2024-04-25T11:27:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa569400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5640b7c50 0x7fa5640b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa569400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5640b7c50 0x7fa5640b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47056/0 (socket says 172.21.15.19:47056) 2024-04-25T11:27:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa569400640 1 -- 172.21.15.19:0/3979912351 learned_addr learned my addr 172.21.15.19:0/3979912351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa56a800640 1 --2- 172.21.15.19:0/3979912351 >> v2:172.21.15.169:3300/0 conn(0x7fa5640d44a0 0x7fa5640d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa569400640 1 -- 172.21.15.19:0/3979912351 >> v2:172.21.15.169:3300/0 conn(0x7fa5640d44a0 msgr2=0x7fa5640d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa569400640 1 --2- 172.21.15.19:0/3979912351 >> v2:172.21.15.169:3300/0 conn(0x7fa5640d44a0 0x7fa5640d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa569400640 1 -- 172.21.15.19:0/3979912351 >> v1:172.21.15.169:6789/0 conn(0x7fa564164860 legacy=0x7fa5640b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa569400640 1 -- 172.21.15.19:0/3979912351 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564161370 con 0x7fa5640b7c50 2024-04-25T11:27:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.921+0000 7fa56a800640 1 --2- 172.21.15.19:0/3979912351 >> v2:172.21.15.169:3300/0 conn(0x7fa5640d44a0 0x7fa5640d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.922+0000 7fa569400640 1 --2- 172.21.15.19:0/3979912351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5640b7c50 0x7fa5640b4d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa564164340 tx=0x7fa55402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b941cf3c3a29f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.923+0000 7fa568a00640 1 -- 172.21.15.19:0/3979912351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554037070 con 0x7fa5640b7c50 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.923+0000 7fa568a00640 1 -- 172.21.15.19:0/3979912351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa554032440 con 0x7fa5640b7c50 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.923+0000 7fa568a00640 1 -- 172.21.15.19:0/3979912351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554032810 con 0x7fa5640b7c50 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.923+0000 7fa56b200640 1 -- 172.21.15.19:0/3979912351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5640b7c50 msgr2=0x7fa5640b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.923+0000 7fa56b200640 1 --2- 172.21.15.19:0/3979912351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5640b7c50 0x7fa5640b4d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa564164340 tx=0x7fa55402a230 comp rx=0 tx=0).stop 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.924+0000 7fa56b200640 1 -- 172.21.15.19:0/3979912351 shutdown_connections 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.924+0000 7fa56b200640 1 --2- 172.21.15.19:0/3979912351 >> v2:172.21.15.169:3300/0 conn(0x7fa5640d44a0 0x7fa5640d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.924+0000 7fa56b200640 1 --2- 172.21.15.19:0/3979912351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5640b7c50 0x7fa5640b4d30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.924+0000 7fa56b200640 1 -- 172.21.15.19:0/3979912351 >> 172.21.15.19:0/3979912351 conn(0x7fa5640d2720 msgr2=0x7fa5640d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.924+0000 7fa56b200640 1 -- 172.21.15.19:0/3979912351 shutdown_connections 2024-04-25T11:27:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.924+0000 7fa56b200640 1 -- 172.21.15.19:0/3979912351 wait complete. 2024-04-25T11:27:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.926+0000 7fa56b200640 1 Processor -- start 2024-04-25T11:27:28.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.926+0000 7fa56b200640 1 -- start start 2024-04-25T11:27:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640d44a0 0x7fa5641f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa564164860 0x7fa5641586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa564156e40 0x7fa564158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640d4df0 con 0x7fa564156e40 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa564164060 con 0x7fa564164860 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa5640d4f70 con 0x7fa5640d44a0 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa564156e40 0x7fa564158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa564156e40 0x7fa564158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47066/0 (socket says 172.21.15.19:47066) 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.928+0000 7fa56a800640 1 -- 172.21.15.19:0/2139129767 learned_addr learned my addr 172.21.15.19:0/2139129767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa569e00640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640d44a0 0x7fa5641f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa569400640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa564164860 0x7fa5641586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56a800640 1 -- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640d44a0 msgr2=0x7fa5641f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56a800640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640d44a0 0x7fa5641f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56a800640 1 -- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa564164860 msgr2=0x7fa5641586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56a800640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa564164860 0x7fa5641586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56a800640 1 -- 172.21.15.19:0/2139129767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56414a990 con 0x7fa564156e40 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa569e00640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640d44a0 0x7fa5641f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa569400640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa564164860 0x7fa5641586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56a800640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa564156e40 0x7fa564158c30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c7c0 tx=0x7fa56000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa55a000640 1 -- 172.21.15.19:0/2139129767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560015020 con 0x7fa564156e40 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa55a000640 1 -- 172.21.15.19:0/2139129767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa560004230 con 0x7fa564156e40 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa55a000640 1 -- 172.21.15.19:0/2139129767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560007620 con 0x7fa564156e40 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5641fa4c0 con 0x7fa564156e40 2024-04-25T11:27:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.929+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56414aee0 con 0x7fa564156e40 2024-04-25T11:27:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.930+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5640b7100 con 0x7fa564156e40 2024-04-25T11:27:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.932+0000 7fa55a000640 1 -- 172.21.15.19:0/2139129767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa560018020 con 0x7fa564156e40 2024-04-25T11:27:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.933+0000 7fa55a000640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa548074500 0x7fa548076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.933+0000 7fa55a000640 1 -- 172.21.15.19:0/2139129767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa5600951b0 con 0x7fa564156e40 2024-04-25T11:27:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.934+0000 7fa569e00640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa548074500 0x7fa548076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:28.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.935+0000 7fa569e00640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa548074500 0x7fa548076a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5640d3740 tx=0x7fa55c0073a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:28.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:28.938+0000 7fa55a000640 1 -- 172.21.15.19:0/2139129767 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa56001f020 con 0x7fa564156e40 2024-04-25T11:27:29.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.179+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa564050740 con 0x7fa564156e40 2024-04-25T11:27:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.181+0000 7fa55a000640 1 -- 172.21.15.19:0/2139129767 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fa5641fa4c0 con 0x7fa564156e40 2024-04-25T11:27:29.172 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:27:29.173 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:27:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.183+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa548074500 msgr2=0x7fa548076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.183+0000 7fa56b200640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa548074500 0x7fa548076a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5640d3740 tx=0x7fa55c0073a0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.183+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa564156e40 msgr2=0x7fa564158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.183+0000 7fa56b200640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa564156e40 0x7fa564158c30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c7c0 tx=0x7fa56000cc80 comp rx=0 tx=0).stop 2024-04-25T11:27:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 shutdown_connections 2024-04-25T11:27:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa548074500 0x7fa548076a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa564164860 0x7fa5641586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa564156e40 0x7fa564158c30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 --2- 172.21.15.19:0/2139129767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5640d44a0 0x7fa5641f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 >> 172.21.15.19:0/2139129767 conn(0x7fa5640d2720 msgr2=0x7fa5640bb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 shutdown_connections 2024-04-25T11:27:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.184+0000 7fa56b200640 1 -- 172.21.15.19:0/2139129767 wait complete. 2024-04-25T11:27:29.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:29.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.483+0000 7ff2a6800640 1 Processor -- start 2024-04-25T11:27:29.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.483+0000 7ff2a6800640 1 -- start start 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.485+0000 7ff2a6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff2a009b7d0 0x7ff2a0075e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.485+0000 7ff2a6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a006d590 0x7ff2a00b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.485+0000 7ff2a6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2a00b3d10 0x7ff2a00b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.485+0000 7ff2a6800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a006ddc0 con 0x7ff2a00b3d10 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.485+0000 7ff2a6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a006d100 con 0x7ff2a009b7d0 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.485+0000 7ff2a6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a00b2440 con 0x7ff2a006d590 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2a00b3d10 0x7ff2a00b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:29.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2a00b3d10 0x7ff2a00b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47072/0 (socket says 172.21.15.19:47072) 2024-04-25T11:27:29.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 -- 172.21.15.19:0/1587297073 learned_addr learned my addr 172.21.15.19:0/1587297073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a4a00640 1 --2- 172.21.15.19:0/1587297073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a006d590 0x7ff2a00b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5400640 1 --2- 172.21.15.19:0/1587297073 >> v2:172.21.15.169:3300/0 conn(0x7ff2a009b7d0 0x7ff2a0075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 -- 172.21.15.19:0/1587297073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a006d590 msgr2=0x7ff2a00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 --2- 172.21.15.19:0/1587297073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a006d590 0x7ff2a00b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 -- 172.21.15.19:0/1587297073 >> v2:172.21.15.169:3300/0 conn(0x7ff2a009b7d0 msgr2=0x7ff2a0075e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 --2- 172.21.15.19:0/1587297073 >> v2:172.21.15.169:3300/0 conn(0x7ff2a009b7d0 0x7ff2a0075e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5e00640 1 -- 172.21.15.19:0/1587297073 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a00b4660 con 0x7ff2a00b3d10 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.486+0000 7ff2a5400640 1 --2- 172.21.15.19:0/1587297073 >> v2:172.21.15.169:3300/0 conn(0x7ff2a009b7d0 0x7ff2a0075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:29.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.487+0000 7ff2a5e00640 1 --2- 172.21.15.19:0/1587297073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a00b3d10 0x7ff2a00b4120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0074580 tx=0x7ff29802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ce7b394ec237d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:29.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.487+0000 7ff29fe00640 1 -- 172.21.15.19:0/1587297073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298036070 con 0x7ff2a00b3d10 2024-04-25T11:27:29.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff29fe00640 1 -- 172.21.15.19:0/1587297073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff298031600 con 0x7ff2a00b3d10 2024-04-25T11:27:29.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff29fe00640 1 -- 172.21.15.19:0/1587297073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2980319d0 con 0x7ff2a00b3d10 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 -- 172.21.15.19:0/1587297073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a00b3d10 msgr2=0x7ff2a00b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 --2- 172.21.15.19:0/1587297073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a00b3d10 0x7ff2a00b4120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0074580 tx=0x7ff29802a610 comp rx=0 tx=0).stop 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 -- 172.21.15.19:0/1587297073 shutdown_connections 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 --2- 172.21.15.19:0/1587297073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a00b3d10 0x7ff2a00b4120 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 --2- 172.21.15.19:0/1587297073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a006d590 0x7ff2a00b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 --2- 172.21.15.19:0/1587297073 >> v2:172.21.15.169:3300/0 conn(0x7ff2a009b7d0 0x7ff2a0075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 -- 172.21.15.19:0/1587297073 >> 172.21.15.19:0/1587297073 conn(0x7ff2a0073920 msgr2=0x7ff2a00b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 -- 172.21.15.19:0/1587297073 shutdown_connections 2024-04-25T11:27:29.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.488+0000 7ff2a6800640 1 -- 172.21.15.19:0/1587297073 wait complete. 2024-04-25T11:27:29.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.491+0000 7ff2a6800640 1 Processor -- start 2024-04-25T11:27:29.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.491+0000 7ff2a6800640 1 -- start start 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.492+0000 7ff2a6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a006d590 0x7ff2a014a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.492+0000 7ff2a6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a009b7d0 0x7ff2a0159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00b3d10 0x7ff2a009d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a6800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a00af2c0 con 0x7ff2a006d590 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a00b29c0 con 0x7ff2a009b7d0 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a00af440 con 0x7ff2a00b3d10 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a006d590 0x7ff2a014a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:29.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a006d590 0x7ff2a014a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47074/0 (socket says 172.21.15.19:47074) 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 -- 172.21.15.19:0/3762794079 learned_addr learned my addr 172.21.15.19:0/3762794079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 -- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00b3d10 msgr2=0x7ff2a009d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5e00640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00b3d10 0x7ff2a009d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00b3d10 0x7ff2a009d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 -- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a009b7d0 msgr2=0x7ff2a0159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a009b7d0 0x7ff2a0159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 -- 172.21.15.19:0/3762794079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a014af10 con 0x7ff2a006d590 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.493+0000 7ff2a5400640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a006d590 0x7ff2a014a9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c820 tx=0x7ff29400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.494+0000 7ff29d600640 1 -- 172.21.15.19:0/3762794079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294015070 con 0x7ff2a006d590 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.494+0000 7ff29d600640 1 -- 172.21.15.19:0/3762794079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294007a30 con 0x7ff2a006d590 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.494+0000 7ff29d600640 1 -- 172.21.15.19:0/3762794079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294007de0 con 0x7ff2a006d590 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.494+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a01586f0 con 0x7ff2a006d590 2024-04-25T11:27:29.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.494+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a0158cb0 con 0x7ff2a006d590 2024-04-25T11:27:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.495+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a00b1940 con 0x7ff2a006d590 2024-04-25T11:27:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.496+0000 7ff29d600640 1 -- 172.21.15.19:0/3762794079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff294018070 con 0x7ff2a006d590 2024-04-25T11:27:29.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.497+0000 7ff29d600640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2800744b0 0x7ff2800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:29.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.497+0000 7ff29d600640 1 -- 172.21.15.19:0/3762794079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff294095260 con 0x7ff2a006d590 2024-04-25T11:27:29.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.498+0000 7ff2a4a00640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2800744b0 0x7ff2800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:29.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.499+0000 7ff2a4a00640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2800744b0 0x7ff2800769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006fd0 tx=0x7ff28c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:29.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.502+0000 7ff29d600640 1 -- 172.21.15.19:0/3762794079 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff29405f950 con 0x7ff2a006d590 2024-04-25T11:27:29.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.698+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2a0156fe0 con 0x7ff2a006d590 2024-04-25T11:27:29.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.701+0000 7ff29d600640 1 -- 172.21.15.19:0/3762794079 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7ff29405f250 con 0x7ff2a006d590 2024-04-25T11:27:29.693 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:29.693 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.703+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2800744b0 msgr2=0x7ff2800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.703+0000 7ff2a6800640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2800744b0 0x7ff2800769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006fd0 tx=0x7ff28c008040 comp rx=0 tx=0).stop 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.703+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a006d590 msgr2=0x7ff2a014a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.703+0000 7ff2a6800640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a006d590 0x7ff2a014a9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c820 tx=0x7ff29400cce0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 shutdown_connections 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2800744b0 0x7ff2800769c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a009b7d0 0x7ff2a0159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00b3d10 0x7ff2a009d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 --2- 172.21.15.19:0/3762794079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2a006d590 0x7ff2a014a9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 >> 172.21.15.19:0/3762794079 conn(0x7ff2a0073920 msgr2=0x7ff2a0162780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:29.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 shutdown_connections 2024-04-25T11:27:29.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:29.704+0000 7ff2a6800640 1 -- 172.21.15.19:0/3762794079 wait complete. 2024-04-25T11:27:29.720 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:27:29.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:27:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.008+0000 7f8feca00640 1 Processor -- start 2024-04-25T11:27:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.008+0000 7f8feca00640 1 -- start start 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.010+0000 7f8feca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8fe80d50e0 0x7f8fe80d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.010+0000 7f8feca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fe80d2ae0 0x7f8fe8160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.010+0000 7f8feca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe80b7940 con 0x7f8fe8160b90 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.010+0000 7f8feca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe80d2ef0 con 0x7f8fe80d2ae0 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.010+0000 7f8feca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe8161280 con 0x7f8fe80d50e0 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.011+0000 7f8fe5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fe80d2ae0 0x7f8fe8160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.011+0000 7f8fe5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fe80d2ae0 0x7f8fe8160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47078/0 (socket says 172.21.15.19:47078) 2024-04-25T11:27:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.011+0000 7f8fe5e00640 1 -- 172.21.15.19:0/1919924727 learned_addr learned my addr 172.21.15.19:0/1919924727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.011+0000 7f8fe5e00640 1 -- 172.21.15.19:0/1919924727 >> v2:172.21.15.169:3300/0 conn(0x7f8fe80d50e0 msgr2=0x7f8fe80d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.011+0000 7f8fe5e00640 1 --2- 172.21.15.19:0/1919924727 >> v2:172.21.15.169:3300/0 conn(0x7f8fe80d50e0 0x7f8fe80d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.011+0000 7f8fe5e00640 1 -- 172.21.15.19:0/1919924727 >> v1:172.21.15.169:6789/0 conn(0x7f8fe8160b90 legacy=0x7f8fe8161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.011+0000 7f8fe5e00640 1 -- 172.21.15.19:0/1919924727 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe80d4600 con 0x7f8fe80d2ae0 2024-04-25T11:27:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.012+0000 7f8fe5e00640 1 --2- 172.21.15.19:0/1919924727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d2ae0 0x7f8fe8160650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80d6cc0 tx=0x7f8fd002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190b9fbd301a809f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:30.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8fe5400640 1 -- 172.21.15.19:0/1919924727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0037070 con 0x7f8fe80d2ae0 2024-04-25T11:27:30.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8fe5400640 1 -- 172.21.15.19:0/1919924727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd0032440 con 0x7f8fe80d2ae0 2024-04-25T11:27:30.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8fe5400640 1 -- 172.21.15.19:0/1919924727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0032810 con 0x7f8fe80d2ae0 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8feca00640 1 -- 172.21.15.19:0/1919924727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d2ae0 msgr2=0x7f8fe8160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8feca00640 1 --2- 172.21.15.19:0/1919924727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d2ae0 0x7f8fe8160650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80d6cc0 tx=0x7f8fd002a230 comp rx=0 tx=0).stop 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8feca00640 1 -- 172.21.15.19:0/1919924727 shutdown_connections 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8feca00640 1 --2- 172.21.15.19:0/1919924727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d2ae0 0x7f8fe8160650 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8feca00640 1 --2- 172.21.15.19:0/1919924727 >> v2:172.21.15.169:3300/0 conn(0x7f8fe80d50e0 0x7f8fe80d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.013+0000 7f8feca00640 1 -- 172.21.15.19:0/1919924727 >> 172.21.15.19:0/1919924727 conn(0x7f8fe806c5c0 msgr2=0x7f8fe806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.014+0000 7f8feca00640 1 -- 172.21.15.19:0/1919924727 shutdown_connections 2024-04-25T11:27:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.014+0000 7f8feca00640 1 -- 172.21.15.19:0/1919924727 wait complete. 2024-04-25T11:27:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.016+0000 7f8feca00640 1 Processor -- start 2024-04-25T11:27:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.016+0000 7f8feca00640 1 -- start start 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.017+0000 7f8feca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fe80d2ae0 0x7f8fe81e7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8feca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d50e0 0x7f8fe80c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8feca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fe8160b90 0x7f8fe81e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8feca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8160fa0 con 0x7f8fe80d50e0 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8feca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe80d4bc0 con 0x7f8fe80d2ae0 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8feca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe8161560 con 0x7f8fe8160b90 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d50e0 0x7f8fe80c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d50e0 0x7f8fe80c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47092/0 (socket says 172.21.15.19:47092) 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 -- 172.21.15.19:0/2913215549 learned_addr learned my addr 172.21.15.19:0/2913215549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 -- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fe8160b90 msgr2=0x7f8fe81e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe7200640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fe8160b90 0x7f8fe81e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe6800640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fe80d2ae0 0x7f8fe81e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fe8160b90 0x7f8fe81e8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 -- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fe80d2ae0 msgr2=0x7f8fe81e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fe80d2ae0 0x7f8fe81e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe5e00640 1 -- 172.21.15.19:0/2913215549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe81e7c50 con 0x7f8fe80d50e0 2024-04-25T11:27:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.018+0000 7f8fe7200640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fe8160b90 0x7f8fe81e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.019+0000 7f8fe5e00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d50e0 0x7f8fe80c8200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0000c00 tx=0x7f8fd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.019+0000 7f8fdea00640 1 -- 172.21.15.19:0/2913215549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0041070 con 0x7f8fe80d50e0 2024-04-25T11:27:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.019+0000 7f8fdea00640 1 -- 172.21.15.19:0/2913215549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd003c870 con 0x7f8fe80d50e0 2024-04-25T11:27:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.019+0000 7f8fdea00640 1 -- 172.21.15.19:0/2913215549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0037040 con 0x7f8fe80d50e0 2024-04-25T11:27:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.019+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe81e7f30 con 0x7f8fe80d50e0 2024-04-25T11:27:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.019+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81e84f0 con 0x7f8fe80d50e0 2024-04-25T11:27:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.020+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe8050740 con 0x7f8fe80d50e0 2024-04-25T11:27:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.022+0000 7f8fdea00640 1 -- 172.21.15.19:0/2913215549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8fd0044070 con 0x7f8fe80d50e0 2024-04-25T11:27:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.023+0000 7f8fdea00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8fc40744b0 0x7f8fc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.023+0000 7f8fdea00640 1 -- 172.21.15.19:0/2913215549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8fd00b5dd0 con 0x7f8fe80d50e0 2024-04-25T11:27:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.023+0000 7f8fe6800640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8fc40744b0 0x7f8fc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.024+0000 7f8fe6800640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8fc40744b0 0x7f8fc40769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8007dc0 tx=0x7f8fd80072f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.027+0000 7f8fdea00640 1 -- 172.21.15.19:0/2913215549 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fe80b7940 con 0x7f8fe80d50e0 2024-04-25T11:27:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.228+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fe81e79e0 con 0x7f8fc40744b0 2024-04-25T11:27:30.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.252+0000 7f8fdea00640 1 -- 172.21.15.19:0/2913215549 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251916 (secure 0 0 0) 0x7f8fe81e79e0 con 0x7f8fc40744b0 2024-04-25T11:27:30.247 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:30.248 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.257+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8fc40744b0 msgr2=0x7f8fc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.257+0000 7f8feca00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8fc40744b0 0x7f8fc40769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8007dc0 tx=0x7f8fd80072f0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.257+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d50e0 msgr2=0x7f8fe80c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.257+0000 7f8feca00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d50e0 0x7f8fe80c8200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0000c00 tx=0x7f8fd00046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 shutdown_connections 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8fc40744b0 0x7f8fc40769c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fe8160b90 0x7f8fe81e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe80d50e0 0x7f8fe80c8200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 --2- 172.21.15.19:0/2913215549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fe80d2ae0 0x7f8fe81e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 >> 172.21.15.19:0/2913215549 conn(0x7f8fe806c5c0 msgr2=0x7f8fe815f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 shutdown_connections 2024-04-25T11:27:30.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.258+0000 7f8feca00640 1 -- 172.21.15.19:0/2913215549 wait complete. 2024-04-25T11:27:30.273 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T11:27:28.435054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2949120,"data_stored":1758028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.912395"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109908+0000","last_change":"2024-04-25T11:27:25.576865+0000","last_active":"2024-04-25T11:27:27.109908+0000","last_peered":"2024-04-25T11:27:27.109908+0000","last_clean":"2024-04-25T11:27:27.109908+0000","last_became_active":"2024-04-25T11:27:25.576473+0000","last_became_peered":"2024-04-25T11:27:25.576473+0000","last_unstale":"2024-04-25T11:27:27.109908+0000","last_undegraded":"2024-04-25T11:27:27.109908+0000","last_fullsized":"2024-04-25T11:27:27.109908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.596605+0000","last_change":"2024-04-25T11:27:25.591722+0000","last_active":"2024-04-25T11:27:25.596605+0000","last_peered":"2024-04-25T11:27:25.596605+0000","last_clean":"2024-04-25T11:27:25.596605+0000","last_became_active":"2024-04-25T11:27:25.591379+0000","last_became_peered":"2024-04-25T11:27:25.591379+0000","last_unstale":"2024-04-25T11:27:25.596605+0000","last_undegraded":"2024-04-25T11:27:25.596605+0000","last_fullsized":"2024-04-25T11:27:25.596605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:10.582173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.572563+0000","last_change":"2024-04-25T11:27:25.571047+0000","last_active":"2024-04-25T11:27:25.572563+0000","last_peered":"2024-04-25T11:27:25.572563+0000","last_clean":"2024-04-25T11:27:25.572563+0000","last_became_active":"2024-04-25T11:27:25.570667+0000","last_became_peered":"2024-04-25T11:27:25.570667+0000","last_unstale":"2024-04-25T11:27:25.572563+0000","last_undegraded":"2024-04-25T11:27:25.572563+0000","last_fullsized":"2024-04-25T11:27:25.572563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:47.493161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580734+0000","last_change":"2024-04-25T11:27:25.580512+0000","last_active":"2024-04-25T11:27:26.580734+0000","last_peered":"2024-04-25T11:27:26.580734+0000","last_clean":"2024-04-25T11:27:26.580734+0000","last_became_active":"2024-04-25T11:27:25.579802+0000","last_became_peered":"2024-04-25T11:27:25.579802+0000","last_unstale":"2024-04-25T11:27:26.580734+0000","last_undegraded":"2024-04-25T11:27:26.580734+0000","last_fullsized":"2024-04-25T11:27:26.580734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109585+0000","last_change":"2024-04-25T11:27:25.578185+0000","last_active":"2024-04-25T11:27:27.109585+0000","last_peered":"2024-04-25T11:27:27.109585+0000","last_clean":"2024-04-25T11:27:27.109585+0000","last_became_active":"2024-04-25T11:27:25.577850+0000","last_became_peered":"2024-04-25T11:27:25.577850+0000","last_unstale":"2024-04-25T11:27:27.109585+0000","last_undegraded":"2024-04-25T11:27:27.109585+0000","last_fullsized":"2024-04-25T11:27:27.109585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579999+0000","last_change":"2024-04-25T11:27:25.581557+0000","last_active":"2024-04-25T11:27:26.579999+0000","last_peered":"2024-04-25T11:27:26.579999+0000","last_clean":"2024-04-25T11:27:26.579999+0000","last_became_active":"2024-04-25T11:27:25.580913+0000","last_became_peered":"2024-04-25T11:27:25.580913+0000","last_unstale":"2024-04-25T11:27:26.579999+0000","last_undegraded":"2024-04-25T11:27:26.579999+0000","last_fullsized":"2024-04-25T11:27:26.579999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111438+0000","last_change":"2024-04-25T11:27:25.581148+0000","last_active":"2024-04-25T11:27:27.111438+0000","last_peered":"2024-04-25T11:27:27.111438+0000","last_clean":"2024-04-25T11:27:27.111438+0000","last_became_active":"2024-04-25T11:27:25.580733+0000","last_became_peered":"2024-04-25T11:27:25.580733+0000","last_unstale":"2024-04-25T11:27:27.111438+0000","last_undegraded":"2024-04-25T11:27:27.111438+0000","last_fullsized":"2024-04-25T11:27:27.111438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584659+0000","last_change":"2024-04-25T11:27:25.576615+0000","last_active":"2024-04-25T11:27:25.584659+0000","last_peered":"2024-04-25T11:27:25.584659+0000","last_clean":"2024-04-25T11:27:25.584659+0000","last_became_active":"2024-04-25T11:27:25.575901+0000","last_became_peered":"2024-04-25T11:27:25.575901+0000","last_unstale":"2024-04-25T11:27:25.584659+0000","last_undegraded":"2024-04-25T11:27:25.584659+0000","last_fullsized":"2024-04-25T11:27:25.584659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:51.770468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578454+0000","last_change":"2024-04-25T11:27:25.571985+0000","last_active":"2024-04-25T11:27:26.578454+0000","last_peered":"2024-04-25T11:27:26.578454+0000","last_clean":"2024-04-25T11:27:26.578454+0000","last_became_active":"2024-04-25T11:27:25.571508+0000","last_became_peered":"2024-04-25T11:27:25.571508+0000","last_unstale":"2024-04-25T11:27:26.578454+0000","last_undegraded":"2024-04-25T11:27:26.578454+0000","last_fullsized":"2024-04-25T11:27:26.578454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109007+0000","last_change":"2024-04-25T11:27:25.589844+0000","last_active":"2024-04-25T11:27:27.109007+0000","last_peered":"2024-04-25T11:27:27.109007+0000","last_clean":"2024-04-25T11:27:27.109007+0000","last_became_active":"2024-04-25T11:27:25.589511+0000","last_became_peered":"2024-04-25T11:27:25.589511+0000","last_unstale":"2024-04-25T11:27:27.109007+0000","last_undegraded":"2024-04-25T11:27:27.109007+0000","last_fullsized":"2024-04-25T11:27:27.109007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:39.805793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108504+0000","last_change":"2024-04-25T11:27:25.574670+0000","last_active":"2024-04-25T11:27:27.108504+0000","last_peered":"2024-04-25T11:27:27.108504+0000","last_clean":"2024-04-25T11:27:27.108504+0000","last_became_active":"2024-04-25T11:27:25.573712+0000","last_became_peered":"2024-04-25T11:27:25.573712+0000","last_unstale":"2024-04-25T11:27:27.108504+0000","last_undegraded":"2024-04-25T11:27:27.108504+0000","last_fullsized":"2024-04-25T11:27:27.108504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.861673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109484+0000","last_change":"2024-04-25T11:27:25.603039+0000","last_active":"2024-04-25T11:27:27.109484+0000","last_peered":"2024-04-25T11:27:27.109484+0000","last_clean":"2024-04-25T11:27:27.109484+0000","last_became_active":"2024-04-25T11:27:25.592097+0000","last_became_peered":"2024-04-25T11:27:25.592097+0000","last_unstale":"2024-04-25T11:27:27.109484+0000","last_undegraded":"2024-04-25T11:27:27.109484+0000","last_fullsized":"2024-04-25T11:27:27.109484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:47.730373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110810+0000","last_change":"2024-04-25T11:27:25.591095+0000","last_active":"2024-04-25T11:27:27.110810+0000","last_peered":"2024-04-25T11:27:27.110810+0000","last_clean":"2024-04-25T11:27:27.110810+0000","last_became_active":"2024-04-25T11:27:25.590722+0000","last_became_peered":"2024-04-25T11:27:25.590722+0000","last_unstale":"2024-04-25T11:27:27.110810+0000","last_undegraded":"2024-04-25T11:27:27.110810+0000","last_fullsized":"2024-04-25T11:27:27.110810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110340+0000","last_change":"2024-04-25T11:27:25.582044+0000","last_active":"2024-04-25T11:27:27.110340+0000","last_peered":"2024-04-25T11:27:27.110340+0000","last_clean":"2024-04-25T11:27:27.110340+0000","last_became_active":"2024-04-25T11:27:25.581715+0000","last_became_peered":"2024-04-25T11:27:25.581715+0000","last_unstale":"2024-04-25T11:27:27.110340+0000","last_undegraded":"2024-04-25T11:27:27.110340+0000","last_fullsized":"2024-04-25T11:27:27.110340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584701+0000","last_change":"2024-04-25T11:27:25.580436+0000","last_active":"2024-04-25T11:27:25.584701+0000","last_peered":"2024-04-25T11:27:25.584701+0000","last_clean":"2024-04-25T11:27:25.584701+0000","last_became_active":"2024-04-25T11:27:25.577298+0000","last_became_peered":"2024-04-25T11:27:25.577298+0000","last_unstale":"2024-04-25T11:27:25.584701+0000","last_undegraded":"2024-04-25T11:27:25.584701+0000","last_fullsized":"2024-04-25T11:27:25.584701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:39.898589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108911+0000","last_change":"2024-04-25T11:27:25.578299+0000","last_active":"2024-04-25T11:27:27.108911+0000","last_peered":"2024-04-25T11:27:27.108911+0000","last_clean":"2024-04-25T11:27:27.108911+0000","last_became_active":"2024-04-25T11:27:25.577785+0000","last_became_peered":"2024-04-25T11:27:25.577785+0000","last_unstale":"2024-04-25T11:27:27.108911+0000","last_undegraded":"2024-04-25T11:27:27.108911+0000","last_fullsized":"2024-04-25T11:27:27.108911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113353+0000","last_change":"2024-04-25T11:27:25.599893+0000","last_active":"2024-04-25T11:27:27.113353+0000","last_peered":"2024-04-25T11:27:27.113353+0000","last_clean":"2024-04-25T11:27:27.113353+0000","last_became_active":"2024-04-25T11:27:25.595315+0000","last_became_peered":"2024-04-25T11:27:25.595315+0000","last_unstale":"2024-04-25T11:27:27.113353+0000","last_undegraded":"2024-04-25T11:27:27.113353+0000","last_fullsized":"2024-04-25T11:27:27.113353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:15.050008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.907339+0000","last_change":"2024-04-25T11:27:25.594310+0000","last_active":"2024-04-25T11:27:26.907339+0000","last_peered":"2024-04-25T11:27:26.907339+0000","last_clean":"2024-04-25T11:27:26.907339+0000","last_became_active":"2024-04-25T11:27:25.593976+0000","last_became_peered":"2024-04-25T11:27:25.593976+0000","last_unstale":"2024-04-25T11:27:26.907339+0000","last_undegraded":"2024-04-25T11:27:26.907339+0000","last_fullsized":"2024-04-25T11:27:26.907339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111503+0000","last_change":"2024-04-25T11:27:25.572942+0000","last_active":"2024-04-25T11:27:27.111503+0000","last_peered":"2024-04-25T11:27:27.111503+0000","last_clean":"2024-04-25T11:27:27.111503+0000","last_became_active":"2024-04-25T11:27:25.572564+0000","last_became_peered":"2024-04-25T11:27:25.572564+0000","last_unstale":"2024-04-25T11:27:27.111503+0000","last_undegraded":"2024-04-25T11:27:27.111503+0000","last_fullsized":"2024-04-25T11:27:27.111503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:32.079241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109999+0000","last_change":"2024-04-25T11:27:25.604247+0000","last_active":"2024-04-25T11:27:27.109999+0000","last_peered":"2024-04-25T11:27:27.109999+0000","last_clean":"2024-04-25T11:27:27.109999+0000","last_became_active":"2024-04-25T11:27:25.599335+0000","last_became_peered":"2024-04-25T11:27:25.599335+0000","last_unstale":"2024-04-25T11:27:27.109999+0000","last_undegraded":"2024-04-25T11:27:27.109999+0000","last_fullsized":"2024-04-25T11:27:27.109999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:09.849779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108856+0000","last_change":"2024-04-25T11:27:25.595766+0000","last_active":"2024-04-25T11:27:27.108856+0000","last_peered":"2024-04-25T11:27:27.108856+0000","last_clean":"2024-04-25T11:27:27.108856+0000","last_became_active":"2024-04-25T11:27:25.595374+0000","last_became_peered":"2024-04-25T11:27:25.595374+0000","last_unstale":"2024-04-25T11:27:27.108856+0000","last_undegraded":"2024-04-25T11:27:27.108856+0000","last_fullsized":"2024-04-25T11:27:27.108856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577329+0000","last_change":"2024-04-25T11:27:25.591577+0000","last_active":"2024-04-25T11:27:26.577329+0000","last_peered":"2024-04-25T11:27:26.577329+0000","last_clean":"2024-04-25T11:27:26.577329+0000","last_became_active":"2024-04-25T11:27:25.591265+0000","last_became_peered":"2024-04-25T11:27:25.591265+0000","last_unstale":"2024-04-25T11:27:26.577329+0000","last_undegraded":"2024-04-25T11:27:26.577329+0000","last_fullsized":"2024-04-25T11:27:26.577329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584797+0000","last_change":"2024-04-25T11:27:25.581137+0000","last_active":"2024-04-25T11:27:25.584797+0000","last_peered":"2024-04-25T11:27:25.584797+0000","last_clean":"2024-04-25T11:27:25.584797+0000","last_became_active":"2024-04-25T11:27:25.580856+0000","last_became_peered":"2024-04-25T11:27:25.580856+0000","last_unstale":"2024-04-25T11:27:25.584797+0000","last_undegraded":"2024-04-25T11:27:25.584797+0000","last_fullsized":"2024-04-25T11:27:25.584797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:26.233769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109404+0000","last_change":"2024-04-25T11:27:25.588399+0000","last_active":"2024-04-25T11:27:27.109404+0000","last_peered":"2024-04-25T11:27:27.109404+0000","last_clean":"2024-04-25T11:27:27.109404+0000","last_became_active":"2024-04-25T11:27:25.587283+0000","last_became_peered":"2024-04-25T11:27:25.587283+0000","last_unstale":"2024-04-25T11:27:27.109404+0000","last_undegraded":"2024-04-25T11:27:27.109404+0000","last_fullsized":"2024-04-25T11:27:27.109404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.576846+0000","last_change":"2024-04-25T11:27:25.572116+0000","last_active":"2024-04-25T11:27:25.576846+0000","last_peered":"2024-04-25T11:27:25.576846+0000","last_clean":"2024-04-25T11:27:25.576846+0000","last_became_active":"2024-04-25T11:27:25.571814+0000","last_became_peered":"2024-04-25T11:27:25.571814+0000","last_unstale":"2024-04-25T11:27:25.576846+0000","last_undegraded":"2024-04-25T11:27:25.576846+0000","last_fullsized":"2024-04-25T11:27:25.576846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:42.010092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581233+0000","last_change":"2024-04-25T11:27:25.592264+0000","last_active":"2024-04-25T11:27:26.581233+0000","last_peered":"2024-04-25T11:27:26.581233+0000","last_clean":"2024-04-25T11:27:26.581233+0000","last_became_active":"2024-04-25T11:27:25.591919+0000","last_became_peered":"2024-04-25T11:27:25.591919+0000","last_unstale":"2024-04-25T11:27:26.581233+0000","last_undegraded":"2024-04-25T11:27:26.581233+0000","last_fullsized":"2024-04-25T11:27:26.581233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111469+0000","last_change":"2024-04-25T11:27:25.588173+0000","last_active":"2024-04-25T11:27:27.111469+0000","last_peered":"2024-04-25T11:27:27.111469+0000","last_clean":"2024-04-25T11:27:27.111469+0000","last_became_active":"2024-04-25T11:27:25.587836+0000","last_became_peered":"2024-04-25T11:27:25.587836+0000","last_unstale":"2024-04-25T11:27:27.111469+0000","last_undegraded":"2024-04-25T11:27:27.111469+0000","last_fullsized":"2024-04-25T11:27:27.111469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111338+0000","last_change":"2024-04-25T11:27:25.588936+0000","last_active":"2024-04-25T11:27:27.111338+0000","last_peered":"2024-04-25T11:27:27.111338+0000","last_clean":"2024-04-25T11:27:27.111338+0000","last_became_active":"2024-04-25T11:27:25.588514+0000","last_became_peered":"2024-04-25T11:27:25.588514+0000","last_unstale":"2024-04-25T11:27:27.111338+0000","last_undegraded":"2024-04-25T11:27:27.111338+0000","last_fullsized":"2024-04-25T11:27:27.111338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584468+0000","last_change":"2024-04-25T11:27:25.574933+0000","last_active":"2024-04-25T11:27:25.584468+0000","last_peered":"2024-04-25T11:27:25.584468+0000","last_clean":"2024-04-25T11:27:25.584468+0000","last_became_active":"2024-04-25T11:27:25.573752+0000","last_became_peered":"2024-04-25T11:27:25.573752+0000","last_unstale":"2024-04-25T11:27:25.584468+0000","last_undegraded":"2024-04-25T11:27:25.584468+0000","last_fullsized":"2024-04-25T11:27:25.584468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.699250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110999+0000","last_change":"2024-04-25T11:27:25.585915+0000","last_active":"2024-04-25T11:27:27.110999+0000","last_peered":"2024-04-25T11:27:27.110999+0000","last_clean":"2024-04-25T11:27:27.110999+0000","last_became_active":"2024-04-25T11:27:25.583985+0000","last_became_peered":"2024-04-25T11:27:25.583985+0000","last_unstale":"2024-04-25T11:27:27.110999+0000","last_undegraded":"2024-04-25T11:27:27.110999+0000","last_fullsized":"2024-04-25T11:27:27.110999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:57.615463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906567+0000","last_change":"2024-04-25T11:27:25.587583+0000","last_active":"2024-04-25T11:27:26.906567+0000","last_peered":"2024-04-25T11:27:26.906567+0000","last_clean":"2024-04-25T11:27:26.906567+0000","last_became_active":"2024-04-25T11:27:25.587318+0000","last_became_peered":"2024-04-25T11:27:25.587318+0000","last_unstale":"2024-04-25T11:27:26.906567+0000","last_undegraded":"2024-04-25T11:27:26.906567+0000","last_fullsized":"2024-04-25T11:27:26.906567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579991+0000","last_change":"2024-04-25T11:27:25.597364+0000","last_active":"2024-04-25T11:27:26.579991+0000","last_peered":"2024-04-25T11:27:26.579991+0000","last_clean":"2024-04-25T11:27:26.579991+0000","last_became_active":"2024-04-25T11:27:25.596928+0000","last_became_peered":"2024-04-25T11:27:25.596928+0000","last_unstale":"2024-04-25T11:27:26.579991+0000","last_undegraded":"2024-04-25T11:27:26.579991+0000","last_fullsized":"2024-04-25T11:27:26.579991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:24.646664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577102+0000","last_change":"2024-04-25T11:27:25.570795+0000","last_active":"2024-04-25T11:27:26.577102+0000","last_peered":"2024-04-25T11:27:26.577102+0000","last_clean":"2024-04-25T11:27:26.577102+0000","last_became_active":"2024-04-25T11:27:25.568542+0000","last_became_peered":"2024-04-25T11:27:25.568542+0000","last_unstale":"2024-04-25T11:27:26.577102+0000","last_undegraded":"2024-04-25T11:27:26.577102+0000","last_fullsized":"2024-04-25T11:27:26.577102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.594097+0000","last_change":"2024-04-25T11:27:24.568405+0000","last_active":"2024-04-25T11:27:25.594097+0000","last_peered":"2024-04-25T11:27:25.594097+0000","last_clean":"2024-04-25T11:27:25.594097+0000","last_became_active":"2024-04-25T11:27:24.568062+0000","last_became_peered":"2024-04-25T11:27:24.568062+0000","last_unstale":"2024-04-25T11:27:25.594097+0000","last_undegraded":"2024-04-25T11:27:25.594097+0000","last_fullsized":"2024-04-25T11:27:25.594097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:54.891145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109354+0000","last_change":"2024-04-25T11:27:25.597308+0000","last_active":"2024-04-25T11:27:27.109354+0000","last_peered":"2024-04-25T11:27:27.109354+0000","last_clean":"2024-04-25T11:27:27.109354+0000","last_became_active":"2024-04-25T11:27:25.596943+0000","last_became_peered":"2024-04-25T11:27:25.596943+0000","last_unstale":"2024-04-25T11:27:27.109354+0000","last_undegraded":"2024-04-25T11:27:27.109354+0000","last_fullsized":"2024-04-25T11:27:27.109354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111690+0000","last_change":"2024-04-25T11:27:24.572640+0000","last_active":"2024-04-25T11:27:27.111690+0000","last_peered":"2024-04-25T11:27:27.111690+0000","last_clean":"2024-04-25T11:27:27.111690+0000","last_became_active":"2024-04-25T11:27:24.572119+0000","last_became_peered":"2024-04-25T11:27:24.572119+0000","last_unstale":"2024-04-25T11:27:27.111690+0000","last_undegraded":"2024-04-25T11:27:27.111690+0000","last_fullsized":"2024-04-25T11:27:27.111690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:12.700897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110506+0000","last_change":"2024-04-25T11:27:24.576360+0000","last_active":"2024-04-25T11:27:27.110506+0000","last_peered":"2024-04-25T11:27:27.110506+0000","last_clean":"2024-04-25T11:27:27.110506+0000","last_became_active":"2024-04-25T11:27:24.572890+0000","last_became_peered":"2024-04-25T11:27:24.572890+0000","last_unstale":"2024-04-25T11:27:27.110506+0000","last_undegraded":"2024-04-25T11:27:27.110506+0000","last_fullsized":"2024-04-25T11:27:27.110506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.593312+0000","last_change":"2024-04-25T11:27:25.589395+0000","last_active":"2024-04-25T11:27:25.593312+0000","last_peered":"2024-04-25T11:27:25.593312+0000","last_clean":"2024-04-25T11:27:25.593312+0000","last_became_active":"2024-04-25T11:27:25.588593+0000","last_became_peered":"2024-04-25T11:27:25.588593+0000","last_unstale":"2024-04-25T11:27:25.593312+0000","last_undegraded":"2024-04-25T11:27:25.593312+0000","last_fullsized":"2024-04-25T11:27:25.593312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:32.080821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109801+0000","last_change":"2024-04-25T11:27:24.558134+0000","last_active":"2024-04-25T11:27:27.109801+0000","last_peered":"2024-04-25T11:27:27.109801+0000","last_clean":"2024-04-25T11:27:27.109801+0000","last_became_active":"2024-04-25T11:27:24.557406+0000","last_became_peered":"2024-04-25T11:27:24.557406+0000","last_unstale":"2024-04-25T11:27:27.109801+0000","last_undegraded":"2024-04-25T11:27:27.109801+0000","last_fullsized":"2024-04-25T11:27:27.109801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:44.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109507+0000","last_change":"2024-04-25T11:27:25.592503+0000","last_active":"2024-04-25T11:27:27.109507+0000","last_peered":"2024-04-25T11:27:27.109507+0000","last_clean":"2024-04-25T11:27:27.109507+0000","last_became_active":"2024-04-25T11:27:25.592131+0000","last_became_peered":"2024-04-25T11:27:25.592131+0000","last_unstale":"2024-04-25T11:27:27.109507+0000","last_undegraded":"2024-04-25T11:27:27.109507+0000","last_fullsized":"2024-04-25T11:27:27.109507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.022859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578279+0000","last_change":"2024-04-25T11:27:24.572792+0000","last_active":"2024-04-25T11:27:26.578279+0000","last_peered":"2024-04-25T11:27:26.578279+0000","last_clean":"2024-04-25T11:27:26.578279+0000","last_became_active":"2024-04-25T11:27:24.572419+0000","last_became_peered":"2024-04-25T11:27:24.572419+0000","last_unstale":"2024-04-25T11:27:26.578279+0000","last_undegraded":"2024-04-25T11:27:26.578279+0000","last_fullsized":"2024-04-25T11:27:26.578279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581607+0000","last_change":"2024-04-25T11:27:25.590899+0000","last_active":"2024-04-25T11:27:26.581607+0000","last_peered":"2024-04-25T11:27:26.581607+0000","last_clean":"2024-04-25T11:27:26.581607+0000","last_became_active":"2024-04-25T11:27:25.588335+0000","last_became_peered":"2024-04-25T11:27:25.588335+0000","last_unstale":"2024-04-25T11:27:26.581607+0000","last_undegraded":"2024-04-25T11:27:26.581607+0000","last_fullsized":"2024-04-25T11:27:26.581607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109319+0000","last_change":"2024-04-25T11:27:24.568514+0000","last_active":"2024-04-25T11:27:27.109319+0000","last_peered":"2024-04-25T11:27:27.109319+0000","last_clean":"2024-04-25T11:27:27.109319+0000","last_became_active":"2024-04-25T11:27:24.567092+0000","last_became_peered":"2024-04-25T11:27:24.567092+0000","last_unstale":"2024-04-25T11:27:27.109319+0000","last_undegraded":"2024-04-25T11:27:27.109319+0000","last_fullsized":"2024-04-25T11:27:27.109319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.585325+0000","last_change":"2024-04-25T11:27:25.577485+0000","last_active":"2024-04-25T11:27:25.585325+0000","last_peered":"2024-04-25T11:27:25.585325+0000","last_clean":"2024-04-25T11:27:25.585325+0000","last_became_active":"2024-04-25T11:27:25.577128+0000","last_became_peered":"2024-04-25T11:27:25.577128+0000","last_unstale":"2024-04-25T11:27:25.585325+0000","last_undegraded":"2024-04-25T11:27:25.585325+0000","last_fullsized":"2024-04-25T11:27:25.585325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.628075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906830+0000","last_change":"2024-04-25T11:27:24.581360+0000","last_active":"2024-04-25T11:27:26.906830+0000","last_peered":"2024-04-25T11:27:26.906830+0000","last_clean":"2024-04-25T11:27:26.906830+0000","last_became_active":"2024-04-25T11:27:24.580827+0000","last_became_peered":"2024-04-25T11:27:24.580827+0000","last_unstale":"2024-04-25T11:27:26.906830+0000","last_undegraded":"2024-04-25T11:27:26.906830+0000","last_fullsized":"2024-04-25T11:27:26.906830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108897+0000","last_change":"2024-04-25T11:27:25.585619+0000","last_active":"2024-04-25T11:27:27.108897+0000","last_peered":"2024-04-25T11:27:27.108897+0000","last_clean":"2024-04-25T11:27:27.108897+0000","last_became_active":"2024-04-25T11:27:25.585314+0000","last_became_peered":"2024-04-25T11:27:25.585314+0000","last_unstale":"2024-04-25T11:27:27.108897+0000","last_undegraded":"2024-04-25T11:27:27.108897+0000","last_fullsized":"2024-04-25T11:27:27.108897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581079+0000","last_change":"2024-04-25T11:27:24.573258+0000","last_active":"2024-04-25T11:27:26.581079+0000","last_peered":"2024-04-25T11:27:26.581079+0000","last_clean":"2024-04-25T11:27:26.581079+0000","last_became_active":"2024-04-25T11:27:24.572900+0000","last_became_peered":"2024-04-25T11:27:24.572900+0000","last_unstale":"2024-04-25T11:27:26.581079+0000","last_undegraded":"2024-04-25T11:27:26.581079+0000","last_fullsized":"2024-04-25T11:27:26.581079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108997+0000","last_change":"2024-04-25T11:27:25.597408+0000","last_active":"2024-04-25T11:27:27.108997+0000","last_peered":"2024-04-25T11:27:27.108997+0000","last_clean":"2024-04-25T11:27:27.108997+0000","last_became_active":"2024-04-25T11:27:25.597094+0000","last_became_peered":"2024-04-25T11:27:25.597094+0000","last_unstale":"2024-04-25T11:27:27.108997+0000","last_undegraded":"2024-04-25T11:27:27.108997+0000","last_fullsized":"2024-04-25T11:27:27.108997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:21.541582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906817+0000","last_change":"2024-04-25T11:27:24.577388+0000","last_active":"2024-04-25T11:27:26.906817+0000","last_peered":"2024-04-25T11:27:26.906817+0000","last_clean":"2024-04-25T11:27:26.906817+0000","last_became_active":"2024-04-25T11:27:24.576282+0000","last_became_peered":"2024-04-25T11:27:24.576282+0000","last_unstale":"2024-04-25T11:27:26.906817+0000","last_undegraded":"2024-04-25T11:27:26.906817+0000","last_fullsized":"2024-04-25T11:27:26.906817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110781+0000","last_change":"2024-04-25T11:27:25.580185+0000","last_active":"2024-04-25T11:27:27.110781+0000","last_peered":"2024-04-25T11:27:27.110781+0000","last_clean":"2024-04-25T11:27:27.110781+0000","last_became_active":"2024-04-25T11:27:25.579504+0000","last_became_peered":"2024-04-25T11:27:25.579504+0000","last_unstale":"2024-04-25T11:27:27.110781+0000","last_undegraded":"2024-04-25T11:27:27.110781+0000","last_fullsized":"2024-04-25T11:27:27.110781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:53.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112046+0000","last_change":"2024-04-25T11:27:24.570725+0000","last_active":"2024-04-25T11:27:27.112046+0000","last_peered":"2024-04-25T11:27:27.112046+0000","last_clean":"2024-04-25T11:27:27.112046+0000","last_became_active":"2024-04-25T11:27:24.570329+0000","last_became_peered":"2024-04-25T11:27:24.570329+0000","last_unstale":"2024-04-25T11:27:27.112046+0000","last_undegraded":"2024-04-25T11:27:27.112046+0000","last_fullsized":"2024-04-25T11:27:27.112046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108584+0000","last_change":"2024-04-25T11:27:25.593697+0000","last_active":"2024-04-25T11:27:27.108584+0000","last_peered":"2024-04-25T11:27:27.108584+0000","last_clean":"2024-04-25T11:27:27.108584+0000","last_became_active":"2024-04-25T11:27:25.593321+0000","last_became_peered":"2024-04-25T11:27:25.593321+0000","last_unstale":"2024-04-25T11:27:27.108584+0000","last_undegraded":"2024-04-25T11:27:27.108584+0000","last_fullsized":"2024-04-25T11:27:27.108584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111788+0000","last_change":"2024-04-25T11:27:24.575295+0000","last_active":"2024-04-25T11:27:27.111788+0000","last_peered":"2024-04-25T11:27:27.111788+0000","last_clean":"2024-04-25T11:27:27.111788+0000","last_became_active":"2024-04-25T11:27:24.574577+0000","last_became_peered":"2024-04-25T11:27:24.574577+0000","last_unstale":"2024-04-25T11:27:27.111788+0000","last_undegraded":"2024-04-25T11:27:27.111788+0000","last_fullsized":"2024-04-25T11:27:27.111788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577619+0000","last_change":"2024-04-25T11:27:25.576314+0000","last_active":"2024-04-25T11:27:26.577619+0000","last_peered":"2024-04-25T11:27:26.577619+0000","last_clean":"2024-04-25T11:27:26.577619+0000","last_became_active":"2024-04-25T11:27:25.575841+0000","last_became_peered":"2024-04-25T11:27:25.575841+0000","last_unstale":"2024-04-25T11:27:26.577619+0000","last_undegraded":"2024-04-25T11:27:26.577619+0000","last_fullsized":"2024-04-25T11:27:26.577619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108586+0000","last_change":"2024-04-25T11:27:24.577138+0000","last_active":"2024-04-25T11:27:27.108586+0000","last_peered":"2024-04-25T11:27:27.108586+0000","last_clean":"2024-04-25T11:27:27.108586+0000","last_became_active":"2024-04-25T11:27:24.576752+0000","last_became_peered":"2024-04-25T11:27:24.576752+0000","last_unstale":"2024-04-25T11:27:27.108586+0000","last_undegraded":"2024-04-25T11:27:27.108586+0000","last_fullsized":"2024-04-25T11:27:27.108586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110175+0000","last_change":"2024-04-25T11:27:25.590200+0000","last_active":"2024-04-25T11:27:27.110175+0000","last_peered":"2024-04-25T11:27:27.110175+0000","last_clean":"2024-04-25T11:27:27.110175+0000","last_became_active":"2024-04-25T11:27:25.588485+0000","last_became_peered":"2024-04-25T11:27:25.588485+0000","last_unstale":"2024-04-25T11:27:27.110175+0000","last_undegraded":"2024-04-25T11:27:27.110175+0000","last_fullsized":"2024-04-25T11:27:27.110175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:47.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110702+0000","last_change":"2024-04-25T11:27:24.578005+0000","last_active":"2024-04-25T11:27:27.110702+0000","last_peered":"2024-04-25T11:27:27.110702+0000","last_clean":"2024-04-25T11:27:27.110702+0000","last_became_active":"2024-04-25T11:27:24.577247+0000","last_became_peered":"2024-04-25T11:27:24.577247+0000","last_unstale":"2024-04-25T11:27:27.110702+0000","last_undegraded":"2024-04-25T11:27:27.110702+0000","last_fullsized":"2024-04-25T11:27:27.110702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581702+0000","last_change":"2024-04-25T11:27:25.604626+0000","last_active":"2024-04-25T11:27:26.581702+0000","last_peered":"2024-04-25T11:27:26.581702+0000","last_clean":"2024-04-25T11:27:26.581702+0000","last_became_active":"2024-04-25T11:27:25.604303+0000","last_became_peered":"2024-04-25T11:27:25.604303+0000","last_unstale":"2024-04-25T11:27:26.581702+0000","last_undegraded":"2024-04-25T11:27:26.581702+0000","last_fullsized":"2024-04-25T11:27:26.581702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110067+0000","last_change":"2024-04-25T11:27:24.574248+0000","last_active":"2024-04-25T11:27:27.110067+0000","last_peered":"2024-04-25T11:27:27.110067+0000","last_clean":"2024-04-25T11:27:27.110067+0000","last_became_active":"2024-04-25T11:27:24.573749+0000","last_became_peered":"2024-04-25T11:27:24.573749+0000","last_unstale":"2024-04-25T11:27:27.110067+0000","last_undegraded":"2024-04-25T11:27:27.110067+0000","last_fullsized":"2024-04-25T11:27:27.110067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110355+0000","last_change":"2024-04-25T11:27:25.578264+0000","last_active":"2024-04-25T11:27:27.110355+0000","last_peered":"2024-04-25T11:27:27.110355+0000","last_clean":"2024-04-25T11:27:27.110355+0000","last_became_active":"2024-04-25T11:27:25.577877+0000","last_became_peered":"2024-04-25T11:27:25.577877+0000","last_unstale":"2024-04-25T11:27:27.110355+0000","last_undegraded":"2024-04-25T11:27:27.110355+0000","last_fullsized":"2024-04-25T11:27:27.110355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.596985+0000","last_change":"2024-04-25T11:27:24.570309+0000","last_active":"2024-04-25T11:27:25.596985+0000","last_peered":"2024-04-25T11:27:25.596985+0000","last_clean":"2024-04-25T11:27:25.596985+0000","last_became_active":"2024-04-25T11:27:24.568762+0000","last_became_peered":"2024-04-25T11:27:24.568762+0000","last_unstale":"2024-04-25T11:27:25.596985+0000","last_undegraded":"2024-04-25T11:27:25.596985+0000","last_fullsized":"2024-04-25T11:27:25.596985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.461035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110466+0000","last_change":"2024-04-25T11:27:25.592429+0000","last_active":"2024-04-25T11:27:27.110466+0000","last_peered":"2024-04-25T11:27:27.110466+0000","last_clean":"2024-04-25T11:27:27.110466+0000","last_became_active":"2024-04-25T11:27:25.591596+0000","last_became_peered":"2024-04-25T11:27:25.591596+0000","last_unstale":"2024-04-25T11:27:27.110466+0000","last_undegraded":"2024-04-25T11:27:27.110466+0000","last_fullsized":"2024-04-25T11:27:27.110466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.577233+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.576780+0000","last_became_peered":"2024-04-25T11:27:24.576780+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111112+0000","last_change":"2024-04-25T11:27:25.587607+0000","last_active":"2024-04-25T11:27:27.111112+0000","last_peered":"2024-04-25T11:27:27.111112+0000","last_clean":"2024-04-25T11:27:27.111112+0000","last_became_active":"2024-04-25T11:27:25.587311+0000","last_became_peered":"2024-04-25T11:27:25.587311+0000","last_unstale":"2024-04-25T11:27:27.111112+0000","last_undegraded":"2024-04-25T11:27:27.111112+0000","last_fullsized":"2024-04-25T11:27:27.111112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:46.539664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.576671+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:27:26.576671+0000","last_peered":"2024-04-25T11:27:26.576671+0000","last_clean":"2024-04-25T11:27:26.576671+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:27:26.576671+0000","last_undegraded":"2024-04-25T11:27:26.576671+0000","last_fullsized":"2024-04-25T11:27:26.576671+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108857+0000","last_change":"2024-04-25T11:27:24.556284+0000","last_active":"2024-04-25T11:27:27.108857+0000","last_peered":"2024-04-25T11:27:27.108857+0000","last_clean":"2024-04-25T11:27:27.108857+0000","last_became_active":"2024-04-25T11:27:24.555672+0000","last_became_peered":"2024-04-25T11:27:24.555672+0000","last_unstale":"2024-04-25T11:27:27.108857+0000","last_undegraded":"2024-04-25T11:27:27.108857+0000","last_fullsized":"2024-04-25T11:27:27.108857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108843+0000","last_change":"2024-04-25T11:27:25.580847+0000","last_active":"2024-04-25T11:27:27.108843+0000","last_peered":"2024-04-25T11:27:27.108843+0000","last_clean":"2024-04-25T11:27:27.108843+0000","last_became_active":"2024-04-25T11:27:25.580468+0000","last_became_peered":"2024-04-25T11:27:25.580468+0000","last_unstale":"2024-04-25T11:27:27.108843+0000","last_undegraded":"2024-04-25T11:27:27.108843+0000","last_fullsized":"2024-04-25T11:27:27.108843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110149+0000","last_change":"2024-04-25T11:27:24.570617+0000","last_active":"2024-04-25T11:27:27.110149+0000","last_peered":"2024-04-25T11:27:27.110149+0000","last_clean":"2024-04-25T11:27:27.110149+0000","last_became_active":"2024-04-25T11:27:24.570392+0000","last_became_peered":"2024-04-25T11:27:24.570392+0000","last_unstale":"2024-04-25T11:27:27.110149+0000","last_undegraded":"2024-04-25T11:27:27.110149+0000","last_fullsized":"2024-04-25T11:27:27.110149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110158+0000","last_change":"2024-04-25T11:27:25.586750+0000","last_active":"2024-04-25T11:27:27.110158+0000","last_peered":"2024-04-25T11:27:27.110158+0000","last_clean":"2024-04-25T11:27:27.110158+0000","last_became_active":"2024-04-25T11:27:25.582935+0000","last_became_peered":"2024-04-25T11:27:25.582935+0000","last_unstale":"2024-04-25T11:27:27.110158+0000","last_undegraded":"2024-04-25T11:27:27.110158+0000","last_fullsized":"2024-04-25T11:27:27.110158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906813+0000","last_change":"2024-04-25T11:27:24.578213+0000","last_active":"2024-04-25T11:27:26.906813+0000","last_peered":"2024-04-25T11:27:26.906813+0000","last_clean":"2024-04-25T11:27:26.906813+0000","last_became_active":"2024-04-25T11:27:24.576970+0000","last_became_peered":"2024-04-25T11:27:24.576970+0000","last_unstale":"2024-04-25T11:27:26.906813+0000","last_undegraded":"2024-04-25T11:27:26.906813+0000","last_fullsized":"2024-04-25T11:27:26.906813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:53.598617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584833+0000","last_change":"2024-04-25T11:27:25.581442+0000","last_active":"2024-04-25T11:27:25.584833+0000","last_peered":"2024-04-25T11:27:25.584833+0000","last_clean":"2024-04-25T11:27:25.584833+0000","last_became_active":"2024-04-25T11:27:25.579355+0000","last_became_peered":"2024-04-25T11:27:25.579355+0000","last_unstale":"2024-04-25T11:27:25.584833+0000","last_undegraded":"2024-04-25T11:27:25.584833+0000","last_fullsized":"2024-04-25T11:27:25.584833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:02.918428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112309+0000","last_change":"2024-04-25T11:27:24.565287+0000","last_active":"2024-04-25T11:27:27.112309+0000","last_peered":"2024-04-25T11:27:27.112309+0000","last_clean":"2024-04-25T11:27:27.112309+0000","last_became_active":"2024-04-25T11:27:24.564368+0000","last_became_peered":"2024-04-25T11:27:24.564368+0000","last_unstale":"2024-04-25T11:27:27.112309+0000","last_undegraded":"2024-04-25T11:27:27.112309+0000","last_fullsized":"2024-04-25T11:27:27.112309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111142+0000","last_change":"2024-04-25T11:27:25.604124+0000","last_active":"2024-04-25T11:27:27.111142+0000","last_peered":"2024-04-25T11:27:27.111142+0000","last_clean":"2024-04-25T11:27:27.111142+0000","last_became_active":"2024-04-25T11:27:25.603769+0000","last_became_peered":"2024-04-25T11:27:25.603769+0000","last_unstale":"2024-04-25T11:27:27.111142+0000","last_undegraded":"2024-04-25T11:27:27.111142+0000","last_fullsized":"2024-04-25T11:27:27.111142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.550311+0000","last_change":"2024-04-25T11:27:24.572130+0000","last_active":"2024-04-25T11:27:25.550311+0000","last_peered":"2024-04-25T11:27:25.550311+0000","last_clean":"2024-04-25T11:27:25.550311+0000","last_became_active":"2024-04-25T11:27:24.569314+0000","last_became_peered":"2024-04-25T11:27:24.569314+0000","last_unstale":"2024-04-25T11:27:25.550311+0000","last_undegraded":"2024-04-25T11:27:25.550311+0000","last_fullsized":"2024-04-25T11:27:25.550311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:18.304561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580084+0000","last_change":"2024-04-25T11:27:25.597482+0000","last_active":"2024-04-25T11:27:26.580084+0000","last_peered":"2024-04-25T11:27:26.580084+0000","last_clean":"2024-04-25T11:27:26.580084+0000","last_became_active":"2024-04-25T11:27:25.595957+0000","last_became_peered":"2024-04-25T11:27:25.595957+0000","last_unstale":"2024-04-25T11:27:26.580084+0000","last_undegraded":"2024-04-25T11:27:26.580084+0000","last_fullsized":"2024-04-25T11:27:26.580084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.572383+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.572006+0000","last_became_peered":"2024-04-25T11:27:24.572006+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:09.042983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111372+0000","last_change":"2024-04-25T11:27:25.578914+0000","last_active":"2024-04-25T11:27:27.111372+0000","last_peered":"2024-04-25T11:27:27.111372+0000","last_clean":"2024-04-25T11:27:27.111372+0000","last_became_active":"2024-04-25T11:27:25.578699+0000","last_became_peered":"2024-04-25T11:27:25.578699+0000","last_unstale":"2024-04-25T11:27:27.111372+0000","last_undegraded":"2024-04-25T11:27:27.111372+0000","last_fullsized":"2024-04-25T11:27:27.111372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110225+0000","last_change":"2024-04-25T11:27:24.579698+0000","last_active":"2024-04-25T11:27:27.110225+0000","last_peered":"2024-04-25T11:27:27.110225+0000","last_clean":"2024-04-25T11:27:27.110225+0000","last_became_active":"2024-04-25T11:27:24.575825+0000","last_became_peered":"2024-04-25T11:27:24.575825+0000","last_unstale":"2024-04-25T11:27:27.110225+0000","last_undegraded":"2024-04-25T11:27:27.110225+0000","last_fullsized":"2024-04-25T11:27:27.110225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109333+0000","last_change":"2024-04-25T11:27:25.588979+0000","last_active":"2024-04-25T11:27:27.109333+0000","last_peered":"2024-04-25T11:27:27.109333+0000","last_clean":"2024-04-25T11:27:27.109333+0000","last_became_active":"2024-04-25T11:27:25.588683+0000","last_became_peered":"2024-04-25T11:27:25.588683+0000","last_unstale":"2024-04-25T11:27:27.109333+0000","last_undegraded":"2024-04-25T11:27:27.109333+0000","last_fullsized":"2024-04-25T11:27:27.109333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113417+0000","last_change":"2024-04-25T11:27:24.572521+0000","last_active":"2024-04-25T11:27:27.113417+0000","last_peered":"2024-04-25T11:27:27.113417+0000","last_clean":"2024-04-25T11:27:27.113417+0000","last_became_active":"2024-04-25T11:27:24.572257+0000","last_became_peered":"2024-04-25T11:27:24.572257+0000","last_unstale":"2024-04-25T11:27:27.113417+0000","last_undegraded":"2024-04-25T11:27:27.113417+0000","last_fullsized":"2024-04-25T11:27:27.113417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110515+0000","last_change":"2024-04-25T11:27:25.574450+0000","last_active":"2024-04-25T11:27:27.110515+0000","last_peered":"2024-04-25T11:27:27.110515+0000","last_clean":"2024-04-25T11:27:27.110515+0000","last_became_active":"2024-04-25T11:27:25.573078+0000","last_became_peered":"2024-04-25T11:27:25.573078+0000","last_unstale":"2024-04-25T11:27:27.110515+0000","last_undegraded":"2024-04-25T11:27:27.110515+0000","last_fullsized":"2024-04-25T11:27:27.110515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.823894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.107620+0000","last_change":"2024-04-25T11:27:24.566069+0000","last_active":"2024-04-25T11:27:27.107620+0000","last_peered":"2024-04-25T11:27:27.107620+0000","last_clean":"2024-04-25T11:27:27.107620+0000","last_became_active":"2024-04-25T11:27:24.565642+0000","last_became_peered":"2024-04-25T11:27:24.565642+0000","last_unstale":"2024-04-25T11:27:27.107620+0000","last_undegraded":"2024-04-25T11:27:27.107620+0000","last_fullsized":"2024-04-25T11:27:27.107620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110907+0000","last_change":"2024-04-25T11:27:25.584529+0000","last_active":"2024-04-25T11:27:27.110907+0000","last_peered":"2024-04-25T11:27:27.110907+0000","last_clean":"2024-04-25T11:27:27.110907+0000","last_became_active":"2024-04-25T11:27:25.584130+0000","last_became_peered":"2024-04-25T11:27:25.584130+0000","last_unstale":"2024-04-25T11:27:27.110907+0000","last_undegraded":"2024-04-25T11:27:27.110907+0000","last_fullsized":"2024-04-25T11:27:27.110907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:27.614041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108983+0000","last_change":"2024-04-25T11:27:24.580477+0000","last_active":"2024-04-25T11:27:27.108983+0000","last_peered":"2024-04-25T11:27:27.108983+0000","last_clean":"2024-04-25T11:27:27.108983+0000","last_became_active":"2024-04-25T11:27:24.578736+0000","last_became_peered":"2024-04-25T11:27:24.578736+0000","last_unstale":"2024-04-25T11:27:27.108983+0000","last_undegraded":"2024-04-25T11:27:27.108983+0000","last_fullsized":"2024-04-25T11:27:27.108983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:44.218131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109747+0000","last_change":"2024-04-25T11:27:25.581233+0000","last_active":"2024-04-25T11:27:27.109747+0000","last_peered":"2024-04-25T11:27:27.109747+0000","last_clean":"2024-04-25T11:27:27.109747+0000","last_became_active":"2024-04-25T11:27:25.580073+0000","last_became_peered":"2024-04-25T11:27:25.580073+0000","last_unstale":"2024-04-25T11:27:27.109747+0000","last_undegraded":"2024-04-25T11:27:27.109747+0000","last_fullsized":"2024-04-25T11:27:27.109747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:53.858736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110994+0000","last_change":"2024-04-25T11:27:24.577164+0000","last_active":"2024-04-25T11:27:27.110994+0000","last_peered":"2024-04-25T11:27:27.110994+0000","last_clean":"2024-04-25T11:27:27.110994+0000","last_became_active":"2024-04-25T11:27:24.576795+0000","last_became_peered":"2024-04-25T11:27:24.576795+0000","last_unstale":"2024-04-25T11:27:27.110994+0000","last_undegraded":"2024-04-25T11:27:27.110994+0000","last_fullsized":"2024-04-25T11:27:27.110994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906953+0000","last_change":"2024-04-25T11:27:25.594079+0000","last_active":"2024-04-25T11:27:26.906953+0000","last_peered":"2024-04-25T11:27:26.906953+0000","last_clean":"2024-04-25T11:27:26.906953+0000","last_became_active":"2024-04-25T11:27:25.592965+0000","last_became_peered":"2024-04-25T11:27:25.592965+0000","last_unstale":"2024-04-25T11:27:26.906953+0000","last_undegraded":"2024-04-25T11:27:26.906953+0000","last_fullsized":"2024-04-25T11:27:26.906953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108732+0000","last_change":"2024-04-25T11:27:24.559748+0000","last_active":"2024-04-25T11:27:27.108732+0000","last_peered":"2024-04-25T11:27:27.108732+0000","last_clean":"2024-04-25T11:27:27.108732+0000","last_became_active":"2024-04-25T11:27:24.558116+0000","last_became_peered":"2024-04-25T11:27:24.558116+0000","last_unstale":"2024-04-25T11:27:27.108732+0000","last_undegraded":"2024-04-25T11:27:27.108732+0000","last_fullsized":"2024-04-25T11:27:27.108732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111402+0000","last_change":"2024-04-25T11:27:25.578715+0000","last_active":"2024-04-25T11:27:27.111402+0000","last_peered":"2024-04-25T11:27:27.111402+0000","last_clean":"2024-04-25T11:27:27.111402+0000","last_became_active":"2024-04-25T11:27:25.578355+0000","last_became_peered":"2024-04-25T11:27:25.578355+0000","last_unstale":"2024-04-25T11:27:27.111402+0000","last_undegraded":"2024-04-25T11:27:27.111402+0000","last_fullsized":"2024-04-25T11:27:27.111402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:57.742260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111309+0000","last_change":"2024-04-25T11:27:25.587963+0000","last_active":"2024-04-25T11:27:27.111309+0000","last_peered":"2024-04-25T11:27:27.111309+0000","last_clean":"2024-04-25T11:27:27.111309+0000","last_became_active":"2024-04-25T11:27:25.587609+0000","last_became_peered":"2024-04-25T11:27:25.587609+0000","last_unstale":"2024-04-25T11:27:27.111309+0000","last_undegraded":"2024-04-25T11:27:27.111309+0000","last_fullsized":"2024-04-25T11:27:27.111309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:48.328746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110834+0000","last_change":"2024-04-25T11:27:24.575077+0000","last_active":"2024-04-25T11:27:27.110834+0000","last_peered":"2024-04-25T11:27:27.110834+0000","last_clean":"2024-04-25T11:27:27.110834+0000","last_became_active":"2024-04-25T11:27:24.574000+0000","last_became_peered":"2024-04-25T11:27:24.574000+0000","last_unstale":"2024-04-25T11:27:27.110834+0000","last_undegraded":"2024-04-25T11:27:27.110834+0000","last_fullsized":"2024-04-25T11:27:27.110834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578237+0000","last_change":"2024-04-25T11:27:25.592421+0000","last_active":"2024-04-25T11:27:26.578237+0000","last_peered":"2024-04-25T11:27:26.578237+0000","last_clean":"2024-04-25T11:27:26.578237+0000","last_became_active":"2024-04-25T11:27:25.591913+0000","last_became_peered":"2024-04-25T11:27:25.591913+0000","last_unstale":"2024-04-25T11:27:26.578237+0000","last_undegraded":"2024-04-25T11:27:26.578237+0000","last_fullsized":"2024-04-25T11:27:26.578237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108846+0000","last_change":"2024-04-25T11:27:24.570375+0000","last_active":"2024-04-25T11:27:27.108846+0000","last_peered":"2024-04-25T11:27:27.108846+0000","last_clean":"2024-04-25T11:27:27.108846+0000","last_became_active":"2024-04-25T11:27:24.570081+0000","last_became_peered":"2024-04-25T11:27:24.570081+0000","last_unstale":"2024-04-25T11:27:27.108846+0000","last_undegraded":"2024-04-25T11:27:27.108846+0000","last_fullsized":"2024-04-25T11:27:27.108846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.568828+0000","last_change":"2024-04-25T11:27:25.567438+0000","last_active":"2024-04-25T11:27:25.568828+0000","last_peered":"2024-04-25T11:27:25.568828+0000","last_clean":"2024-04-25T11:27:25.568828+0000","last_became_active":"2024-04-25T11:27:25.567114+0000","last_became_peered":"2024-04-25T11:27:25.567114+0000","last_unstale":"2024-04-25T11:27:25.568828+0000","last_undegraded":"2024-04-25T11:27:25.568828+0000","last_fullsized":"2024-04-25T11:27:25.568828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:25.294830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.594450+0000","last_change":"2024-04-25T11:27:24.572110+0000","last_active":"2024-04-25T11:27:25.594450+0000","last_peered":"2024-04-25T11:27:25.594450+0000","last_clean":"2024-04-25T11:27:25.594450+0000","last_became_active":"2024-04-25T11:27:24.571670+0000","last_became_peered":"2024-04-25T11:27:24.571670+0000","last_unstale":"2024-04-25T11:27:25.594450+0000","last_undegraded":"2024-04-25T11:27:25.594450+0000","last_fullsized":"2024-04-25T11:27:25.594450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:20.625965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.585315+0000","last_change":"2024-04-25T11:27:25.582462+0000","last_active":"2024-04-25T11:27:25.585315+0000","last_peered":"2024-04-25T11:27:25.585315+0000","last_clean":"2024-04-25T11:27:25.585315+0000","last_became_active":"2024-04-25T11:27:25.581902+0000","last_became_peered":"2024-04-25T11:27:25.581902+0000","last_unstale":"2024-04-25T11:27:25.585315+0000","last_undegraded":"2024-04-25T11:27:25.585315+0000","last_fullsized":"2024-04-25T11:27:25.585315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:47.472986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109835+0000","last_change":"2024-04-25T11:27:24.567065+0000","last_active":"2024-04-25T11:27:27.109835+0000","last_peered":"2024-04-25T11:27:27.109835+0000","last_clean":"2024-04-25T11:27:27.109835+0000","last_became_active":"2024-04-25T11:27:24.566049+0000","last_became_peered":"2024-04-25T11:27:24.566049+0000","last_unstale":"2024-04-25T11:27:27.109835+0000","last_undegraded":"2024-04-25T11:27:27.109835+0000","last_fullsized":"2024-04-25T11:27:27.109835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":69984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":72268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":71596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:27:30.278 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:27:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.620+0000 7f5444400640 1 Processor -- start 2024-04-25T11:27:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.620+0000 7f5444400640 1 -- start start 2024-04-25T11:27:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5444400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c0d0e00 0x7f543c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5444400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f543c0d4a50 0x7f543c0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5444400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f543c0d1850 con 0x7f543c06c5c0 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5444400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f543c160de0 con 0x7f543c0d4a50 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5444400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f543c1610c0 con 0x7f543c0d0e00 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5442400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f543c0d4a50 0x7f543c0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5442400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f543c0d4a50 0x7f543c0d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47098/0 (socket says 172.21.15.19:47098) 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5442400640 1 -- 172.21.15.19:0/1506716142 learned_addr learned my addr 172.21.15.19:0/1506716142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.623+0000 7f5441a00640 1 --2- 172.21.15.19:0/1506716142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c0d0e00 0x7f543c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.624+0000 7f5442400640 1 -- 172.21.15.19:0/1506716142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c0d0e00 msgr2=0x7f543c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.624+0000 7f5442400640 1 --2- 172.21.15.19:0/1506716142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c0d0e00 0x7f543c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.624+0000 7f5442400640 1 -- 172.21.15.19:0/1506716142 >> v1:172.21.15.169:6789/0 conn(0x7f543c06c5c0 legacy=0x7f543c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.624+0000 7f5442400640 1 -- 172.21.15.19:0/1506716142 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c1613a0 con 0x7f543c0d4a50 2024-04-25T11:27:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.624+0000 7f5441a00640 1 --2- 172.21.15.19:0/1506716142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c0d0e00 0x7f543c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.625+0000 7f5442400640 1 --2- 172.21.15.19:0/1506716142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d4a50 0x7f543c0d4e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5434009e50 tx=0x7f543402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41abaee1d648851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.625+0000 7f543be00640 1 -- 172.21.15.19:0/1506716142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434036070 con 0x7f543c0d4a50 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.625+0000 7f543be00640 1 -- 172.21.15.19:0/1506716142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5434031600 con 0x7f543c0d4a50 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.625+0000 7f543be00640 1 -- 172.21.15.19:0/1506716142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340319d0 con 0x7f543c0d4a50 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 -- 172.21.15.19:0/1506716142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d4a50 msgr2=0x7f543c0d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 --2- 172.21.15.19:0/1506716142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d4a50 0x7f543c0d4e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5434009e50 tx=0x7f543402a710 comp rx=0 tx=0).stop 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 -- 172.21.15.19:0/1506716142 shutdown_connections 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 --2- 172.21.15.19:0/1506716142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d4a50 0x7f543c0d4e60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 --2- 172.21.15.19:0/1506716142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c0d0e00 0x7f543c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 -- 172.21.15.19:0/1506716142 >> 172.21.15.19:0/1506716142 conn(0x7f543c0d5d90 msgr2=0x7f543c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 -- 172.21.15.19:0/1506716142 shutdown_connections 2024-04-25T11:27:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.626+0000 7f5444400640 1 -- 172.21.15.19:0/1506716142 wait complete. 2024-04-25T11:27:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.629+0000 7f5444400640 1 Processor -- start 2024-04-25T11:27:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.629+0000 7f5444400640 1 -- start start 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.630+0000 7f5444400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c06c5c0 0x7f543c14bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.630+0000 7f5444400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d0e00 0x7f543c1f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.630+0000 7f5444400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f543c0d4a50 0x7f543c1f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5444400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f543c164e00 con 0x7f543c0d0e00 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5444400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f543c0b5320 con 0x7f543c0d4a50 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5444400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f543c164f80 con 0x7f543c06c5c0 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d0e00 0x7f543c1f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d0e00 0x7f543c1f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47106/0 (socket says 172.21.15.19:47106) 2024-04-25T11:27:30.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 -- 172.21.15.19:0/2430778708 learned_addr learned my addr 172.21.15.19:0/2430778708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441a00640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c06c5c0 0x7f543c14bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5442400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f543c0d4a50 0x7f543c1f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 -- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c06c5c0 msgr2=0x7f543c14bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c06c5c0 0x7f543c14bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 -- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f543c0d4a50 msgr2=0x7f543c1f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f543c0d4a50 0x7f543c1f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441000640 1 -- 172.21.15.19:0/2430778708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c0b7f10 con 0x7f543c0d0e00 2024-04-25T11:27:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5442400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f543c0d4a50 0x7f543c1f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.631+0000 7f5441a00640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c06c5c0 0x7f543c14bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.632+0000 7f5441000640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d0e00 0x7f543c1f5f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f542800c3c0 tx=0x7f542800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.632+0000 7f5439600640 1 -- 172.21.15.19:0/2430778708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54280052d0 con 0x7f543c0d0e00 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.632+0000 7f5439600640 1 -- 172.21.15.19:0/2430778708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5428005490 con 0x7f543c0d0e00 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.632+0000 7f5439600640 1 -- 172.21.15.19:0/2430778708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428007240 con 0x7f543c0d0e00 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.632+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c0b81f0 con 0x7f543c0d0e00 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.632+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c0cb440 con 0x7f543c0d0e00 2024-04-25T11:27:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.634+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f543c0ca000 con 0x7f543c0d0e00 2024-04-25T11:27:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.635+0000 7f5439600640 1 -- 172.21.15.19:0/2430778708 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f54280073e0 con 0x7f543c0d0e00 2024-04-25T11:27:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.636+0000 7f5439600640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f541c0744b0 0x7f541c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.636+0000 7f5439600640 1 -- 172.21.15.19:0/2430778708 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f54280947d0 con 0x7f543c0d0e00 2024-04-25T11:27:30.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.637+0000 7f5441a00640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f541c0744b0 0x7f541c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:30.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.638+0000 7f5441a00640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f541c0744b0 0x7f541c0769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5430006fd0 tx=0x7f5430008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:30.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.642+0000 7f5439600640 1 -- 172.21.15.19:0/2430778708 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f542805eec0 con 0x7f543c0d0e00 2024-04-25T11:27:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.845+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f543c0b7cc0 con 0x7f541c0744b0 2024-04-25T11:27:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.865+0000 7f5439600640 1 -- 172.21.15.19:0/2430778708 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251916 (secure 0 0 0) 0x7f543c0b7cc0 con 0x7f541c0744b0 2024-04-25T11:27:30.860 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:30.861 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:27:30.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.870+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f541c0744b0 msgr2=0x7f541c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.870+0000 7f5444400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f541c0744b0 0x7f541c0769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5430006fd0 tx=0x7f5430008040 comp rx=0 tx=0).stop 2024-04-25T11:27:30.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.870+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d0e00 msgr2=0x7f543c1f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.870+0000 7f5444400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d0e00 0x7f543c1f5f00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f542800c3c0 tx=0x7f542800c880 comp rx=0 tx=0).stop 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 shutdown_connections 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f541c0744b0 0x7f541c0769c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f543c0d4a50 0x7f543c1f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f543c0d0e00 0x7f543c1f5f00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 --2- 172.21.15.19:0/2430778708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f543c06c5c0 0x7f543c14bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 >> 172.21.15.19:0/2430778708 conn(0x7f543c0d5d90 msgr2=0x7f543c0ba480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 shutdown_connections 2024-04-25T11:27:30.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:30.871+0000 7f5444400640 1 -- 172.21.15.19:0/2430778708 wait complete. 2024-04-25T11:27:30.887 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T11:27:30.436234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219080,"kb_used_data":2880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755640,"statfs":{"total":773094113280,"available":772869775360,"internally_reserved":0,"allocated":2949120,"data_stored":1758028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002570"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109908+0000","last_change":"2024-04-25T11:27:25.576865+0000","last_active":"2024-04-25T11:27:27.109908+0000","last_peered":"2024-04-25T11:27:27.109908+0000","last_clean":"2024-04-25T11:27:27.109908+0000","last_became_active":"2024-04-25T11:27:25.576473+0000","last_became_peered":"2024-04-25T11:27:25.576473+0000","last_unstale":"2024-04-25T11:27:27.109908+0000","last_undegraded":"2024-04-25T11:27:27.109908+0000","last_fullsized":"2024-04-25T11:27:27.109908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.596605+0000","last_change":"2024-04-25T11:27:25.591722+0000","last_active":"2024-04-25T11:27:25.596605+0000","last_peered":"2024-04-25T11:27:25.596605+0000","last_clean":"2024-04-25T11:27:25.596605+0000","last_became_active":"2024-04-25T11:27:25.591379+0000","last_became_peered":"2024-04-25T11:27:25.591379+0000","last_unstale":"2024-04-25T11:27:25.596605+0000","last_undegraded":"2024-04-25T11:27:25.596605+0000","last_fullsized":"2024-04-25T11:27:25.596605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:10.582173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.572563+0000","last_change":"2024-04-25T11:27:25.571047+0000","last_active":"2024-04-25T11:27:25.572563+0000","last_peered":"2024-04-25T11:27:25.572563+0000","last_clean":"2024-04-25T11:27:25.572563+0000","last_became_active":"2024-04-25T11:27:25.570667+0000","last_became_peered":"2024-04-25T11:27:25.570667+0000","last_unstale":"2024-04-25T11:27:25.572563+0000","last_undegraded":"2024-04-25T11:27:25.572563+0000","last_fullsized":"2024-04-25T11:27:25.572563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:47.493161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580734+0000","last_change":"2024-04-25T11:27:25.580512+0000","last_active":"2024-04-25T11:27:26.580734+0000","last_peered":"2024-04-25T11:27:26.580734+0000","last_clean":"2024-04-25T11:27:26.580734+0000","last_became_active":"2024-04-25T11:27:25.579802+0000","last_became_peered":"2024-04-25T11:27:25.579802+0000","last_unstale":"2024-04-25T11:27:26.580734+0000","last_undegraded":"2024-04-25T11:27:26.580734+0000","last_fullsized":"2024-04-25T11:27:26.580734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109585+0000","last_change":"2024-04-25T11:27:25.578185+0000","last_active":"2024-04-25T11:27:27.109585+0000","last_peered":"2024-04-25T11:27:27.109585+0000","last_clean":"2024-04-25T11:27:27.109585+0000","last_became_active":"2024-04-25T11:27:25.577850+0000","last_became_peered":"2024-04-25T11:27:25.577850+0000","last_unstale":"2024-04-25T11:27:27.109585+0000","last_undegraded":"2024-04-25T11:27:27.109585+0000","last_fullsized":"2024-04-25T11:27:27.109585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579999+0000","last_change":"2024-04-25T11:27:25.581557+0000","last_active":"2024-04-25T11:27:26.579999+0000","last_peered":"2024-04-25T11:27:26.579999+0000","last_clean":"2024-04-25T11:27:26.579999+0000","last_became_active":"2024-04-25T11:27:25.580913+0000","last_became_peered":"2024-04-25T11:27:25.580913+0000","last_unstale":"2024-04-25T11:27:26.579999+0000","last_undegraded":"2024-04-25T11:27:26.579999+0000","last_fullsized":"2024-04-25T11:27:26.579999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111438+0000","last_change":"2024-04-25T11:27:25.581148+0000","last_active":"2024-04-25T11:27:27.111438+0000","last_peered":"2024-04-25T11:27:27.111438+0000","last_clean":"2024-04-25T11:27:27.111438+0000","last_became_active":"2024-04-25T11:27:25.580733+0000","last_became_peered":"2024-04-25T11:27:25.580733+0000","last_unstale":"2024-04-25T11:27:27.111438+0000","last_undegraded":"2024-04-25T11:27:27.111438+0000","last_fullsized":"2024-04-25T11:27:27.111438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584659+0000","last_change":"2024-04-25T11:27:25.576615+0000","last_active":"2024-04-25T11:27:25.584659+0000","last_peered":"2024-04-25T11:27:25.584659+0000","last_clean":"2024-04-25T11:27:25.584659+0000","last_became_active":"2024-04-25T11:27:25.575901+0000","last_became_peered":"2024-04-25T11:27:25.575901+0000","last_unstale":"2024-04-25T11:27:25.584659+0000","last_undegraded":"2024-04-25T11:27:25.584659+0000","last_fullsized":"2024-04-25T11:27:25.584659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:51.770468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578454+0000","last_change":"2024-04-25T11:27:25.571985+0000","last_active":"2024-04-25T11:27:26.578454+0000","last_peered":"2024-04-25T11:27:26.578454+0000","last_clean":"2024-04-25T11:27:26.578454+0000","last_became_active":"2024-04-25T11:27:25.571508+0000","last_became_peered":"2024-04-25T11:27:25.571508+0000","last_unstale":"2024-04-25T11:27:26.578454+0000","last_undegraded":"2024-04-25T11:27:26.578454+0000","last_fullsized":"2024-04-25T11:27:26.578454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109007+0000","last_change":"2024-04-25T11:27:25.589844+0000","last_active":"2024-04-25T11:27:27.109007+0000","last_peered":"2024-04-25T11:27:27.109007+0000","last_clean":"2024-04-25T11:27:27.109007+0000","last_became_active":"2024-04-25T11:27:25.589511+0000","last_became_peered":"2024-04-25T11:27:25.589511+0000","last_unstale":"2024-04-25T11:27:27.109007+0000","last_undegraded":"2024-04-25T11:27:27.109007+0000","last_fullsized":"2024-04-25T11:27:27.109007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:39.805793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108504+0000","last_change":"2024-04-25T11:27:25.574670+0000","last_active":"2024-04-25T11:27:27.108504+0000","last_peered":"2024-04-25T11:27:27.108504+0000","last_clean":"2024-04-25T11:27:27.108504+0000","last_became_active":"2024-04-25T11:27:25.573712+0000","last_became_peered":"2024-04-25T11:27:25.573712+0000","last_unstale":"2024-04-25T11:27:27.108504+0000","last_undegraded":"2024-04-25T11:27:27.108504+0000","last_fullsized":"2024-04-25T11:27:27.108504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.861673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109484+0000","last_change":"2024-04-25T11:27:25.603039+0000","last_active":"2024-04-25T11:27:27.109484+0000","last_peered":"2024-04-25T11:27:27.109484+0000","last_clean":"2024-04-25T11:27:27.109484+0000","last_became_active":"2024-04-25T11:27:25.592097+0000","last_became_peered":"2024-04-25T11:27:25.592097+0000","last_unstale":"2024-04-25T11:27:27.109484+0000","last_undegraded":"2024-04-25T11:27:27.109484+0000","last_fullsized":"2024-04-25T11:27:27.109484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:47.730373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110810+0000","last_change":"2024-04-25T11:27:25.591095+0000","last_active":"2024-04-25T11:27:27.110810+0000","last_peered":"2024-04-25T11:27:27.110810+0000","last_clean":"2024-04-25T11:27:27.110810+0000","last_became_active":"2024-04-25T11:27:25.590722+0000","last_became_peered":"2024-04-25T11:27:25.590722+0000","last_unstale":"2024-04-25T11:27:27.110810+0000","last_undegraded":"2024-04-25T11:27:27.110810+0000","last_fullsized":"2024-04-25T11:27:27.110810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110340+0000","last_change":"2024-04-25T11:27:25.582044+0000","last_active":"2024-04-25T11:27:27.110340+0000","last_peered":"2024-04-25T11:27:27.110340+0000","last_clean":"2024-04-25T11:27:27.110340+0000","last_became_active":"2024-04-25T11:27:25.581715+0000","last_became_peered":"2024-04-25T11:27:25.581715+0000","last_unstale":"2024-04-25T11:27:27.110340+0000","last_undegraded":"2024-04-25T11:27:27.110340+0000","last_fullsized":"2024-04-25T11:27:27.110340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584701+0000","last_change":"2024-04-25T11:27:25.580436+0000","last_active":"2024-04-25T11:27:25.584701+0000","last_peered":"2024-04-25T11:27:25.584701+0000","last_clean":"2024-04-25T11:27:25.584701+0000","last_became_active":"2024-04-25T11:27:25.577298+0000","last_became_peered":"2024-04-25T11:27:25.577298+0000","last_unstale":"2024-04-25T11:27:25.584701+0000","last_undegraded":"2024-04-25T11:27:25.584701+0000","last_fullsized":"2024-04-25T11:27:25.584701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:39.898589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108911+0000","last_change":"2024-04-25T11:27:25.578299+0000","last_active":"2024-04-25T11:27:27.108911+0000","last_peered":"2024-04-25T11:27:27.108911+0000","last_clean":"2024-04-25T11:27:27.108911+0000","last_became_active":"2024-04-25T11:27:25.577785+0000","last_became_peered":"2024-04-25T11:27:25.577785+0000","last_unstale":"2024-04-25T11:27:27.108911+0000","last_undegraded":"2024-04-25T11:27:27.108911+0000","last_fullsized":"2024-04-25T11:27:27.108911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113353+0000","last_change":"2024-04-25T11:27:25.599893+0000","last_active":"2024-04-25T11:27:27.113353+0000","last_peered":"2024-04-25T11:27:27.113353+0000","last_clean":"2024-04-25T11:27:27.113353+0000","last_became_active":"2024-04-25T11:27:25.595315+0000","last_became_peered":"2024-04-25T11:27:25.595315+0000","last_unstale":"2024-04-25T11:27:27.113353+0000","last_undegraded":"2024-04-25T11:27:27.113353+0000","last_fullsized":"2024-04-25T11:27:27.113353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:15.050008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.907339+0000","last_change":"2024-04-25T11:27:25.594310+0000","last_active":"2024-04-25T11:27:26.907339+0000","last_peered":"2024-04-25T11:27:26.907339+0000","last_clean":"2024-04-25T11:27:26.907339+0000","last_became_active":"2024-04-25T11:27:25.593976+0000","last_became_peered":"2024-04-25T11:27:25.593976+0000","last_unstale":"2024-04-25T11:27:26.907339+0000","last_undegraded":"2024-04-25T11:27:26.907339+0000","last_fullsized":"2024-04-25T11:27:26.907339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111503+0000","last_change":"2024-04-25T11:27:25.572942+0000","last_active":"2024-04-25T11:27:27.111503+0000","last_peered":"2024-04-25T11:27:27.111503+0000","last_clean":"2024-04-25T11:27:27.111503+0000","last_became_active":"2024-04-25T11:27:25.572564+0000","last_became_peered":"2024-04-25T11:27:25.572564+0000","last_unstale":"2024-04-25T11:27:27.111503+0000","last_undegraded":"2024-04-25T11:27:27.111503+0000","last_fullsized":"2024-04-25T11:27:27.111503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:32.079241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109999+0000","last_change":"2024-04-25T11:27:25.604247+0000","last_active":"2024-04-25T11:27:27.109999+0000","last_peered":"2024-04-25T11:27:27.109999+0000","last_clean":"2024-04-25T11:27:27.109999+0000","last_became_active":"2024-04-25T11:27:25.599335+0000","last_became_peered":"2024-04-25T11:27:25.599335+0000","last_unstale":"2024-04-25T11:27:27.109999+0000","last_undegraded":"2024-04-25T11:27:27.109999+0000","last_fullsized":"2024-04-25T11:27:27.109999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:09.849779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108856+0000","last_change":"2024-04-25T11:27:25.595766+0000","last_active":"2024-04-25T11:27:27.108856+0000","last_peered":"2024-04-25T11:27:27.108856+0000","last_clean":"2024-04-25T11:27:27.108856+0000","last_became_active":"2024-04-25T11:27:25.595374+0000","last_became_peered":"2024-04-25T11:27:25.595374+0000","last_unstale":"2024-04-25T11:27:27.108856+0000","last_undegraded":"2024-04-25T11:27:27.108856+0000","last_fullsized":"2024-04-25T11:27:27.108856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577329+0000","last_change":"2024-04-25T11:27:25.591577+0000","last_active":"2024-04-25T11:27:26.577329+0000","last_peered":"2024-04-25T11:27:26.577329+0000","last_clean":"2024-04-25T11:27:26.577329+0000","last_became_active":"2024-04-25T11:27:25.591265+0000","last_became_peered":"2024-04-25T11:27:25.591265+0000","last_unstale":"2024-04-25T11:27:26.577329+0000","last_undegraded":"2024-04-25T11:27:26.577329+0000","last_fullsized":"2024-04-25T11:27:26.577329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584797+0000","last_change":"2024-04-25T11:27:25.581137+0000","last_active":"2024-04-25T11:27:25.584797+0000","last_peered":"2024-04-25T11:27:25.584797+0000","last_clean":"2024-04-25T11:27:25.584797+0000","last_became_active":"2024-04-25T11:27:25.580856+0000","last_became_peered":"2024-04-25T11:27:25.580856+0000","last_unstale":"2024-04-25T11:27:25.584797+0000","last_undegraded":"2024-04-25T11:27:25.584797+0000","last_fullsized":"2024-04-25T11:27:25.584797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:26.233769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109404+0000","last_change":"2024-04-25T11:27:25.588399+0000","last_active":"2024-04-25T11:27:27.109404+0000","last_peered":"2024-04-25T11:27:27.109404+0000","last_clean":"2024-04-25T11:27:27.109404+0000","last_became_active":"2024-04-25T11:27:25.587283+0000","last_became_peered":"2024-04-25T11:27:25.587283+0000","last_unstale":"2024-04-25T11:27:27.109404+0000","last_undegraded":"2024-04-25T11:27:27.109404+0000","last_fullsized":"2024-04-25T11:27:27.109404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.576846+0000","last_change":"2024-04-25T11:27:25.572116+0000","last_active":"2024-04-25T11:27:25.576846+0000","last_peered":"2024-04-25T11:27:25.576846+0000","last_clean":"2024-04-25T11:27:25.576846+0000","last_became_active":"2024-04-25T11:27:25.571814+0000","last_became_peered":"2024-04-25T11:27:25.571814+0000","last_unstale":"2024-04-25T11:27:25.576846+0000","last_undegraded":"2024-04-25T11:27:25.576846+0000","last_fullsized":"2024-04-25T11:27:25.576846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:42.010092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581233+0000","last_change":"2024-04-25T11:27:25.592264+0000","last_active":"2024-04-25T11:27:26.581233+0000","last_peered":"2024-04-25T11:27:26.581233+0000","last_clean":"2024-04-25T11:27:26.581233+0000","last_became_active":"2024-04-25T11:27:25.591919+0000","last_became_peered":"2024-04-25T11:27:25.591919+0000","last_unstale":"2024-04-25T11:27:26.581233+0000","last_undegraded":"2024-04-25T11:27:26.581233+0000","last_fullsized":"2024-04-25T11:27:26.581233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111469+0000","last_change":"2024-04-25T11:27:25.588173+0000","last_active":"2024-04-25T11:27:27.111469+0000","last_peered":"2024-04-25T11:27:27.111469+0000","last_clean":"2024-04-25T11:27:27.111469+0000","last_became_active":"2024-04-25T11:27:25.587836+0000","last_became_peered":"2024-04-25T11:27:25.587836+0000","last_unstale":"2024-04-25T11:27:27.111469+0000","last_undegraded":"2024-04-25T11:27:27.111469+0000","last_fullsized":"2024-04-25T11:27:27.111469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111338+0000","last_change":"2024-04-25T11:27:25.588936+0000","last_active":"2024-04-25T11:27:27.111338+0000","last_peered":"2024-04-25T11:27:27.111338+0000","last_clean":"2024-04-25T11:27:27.111338+0000","last_became_active":"2024-04-25T11:27:25.588514+0000","last_became_peered":"2024-04-25T11:27:25.588514+0000","last_unstale":"2024-04-25T11:27:27.111338+0000","last_undegraded":"2024-04-25T11:27:27.111338+0000","last_fullsized":"2024-04-25T11:27:27.111338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584468+0000","last_change":"2024-04-25T11:27:25.574933+0000","last_active":"2024-04-25T11:27:25.584468+0000","last_peered":"2024-04-25T11:27:25.584468+0000","last_clean":"2024-04-25T11:27:25.584468+0000","last_became_active":"2024-04-25T11:27:25.573752+0000","last_became_peered":"2024-04-25T11:27:25.573752+0000","last_unstale":"2024-04-25T11:27:25.584468+0000","last_undegraded":"2024-04-25T11:27:25.584468+0000","last_fullsized":"2024-04-25T11:27:25.584468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.699250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110999+0000","last_change":"2024-04-25T11:27:25.585915+0000","last_active":"2024-04-25T11:27:27.110999+0000","last_peered":"2024-04-25T11:27:27.110999+0000","last_clean":"2024-04-25T11:27:27.110999+0000","last_became_active":"2024-04-25T11:27:25.583985+0000","last_became_peered":"2024-04-25T11:27:25.583985+0000","last_unstale":"2024-04-25T11:27:27.110999+0000","last_undegraded":"2024-04-25T11:27:27.110999+0000","last_fullsized":"2024-04-25T11:27:27.110999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:57.615463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906567+0000","last_change":"2024-04-25T11:27:25.587583+0000","last_active":"2024-04-25T11:27:26.906567+0000","last_peered":"2024-04-25T11:27:26.906567+0000","last_clean":"2024-04-25T11:27:26.906567+0000","last_became_active":"2024-04-25T11:27:25.587318+0000","last_became_peered":"2024-04-25T11:27:25.587318+0000","last_unstale":"2024-04-25T11:27:26.906567+0000","last_undegraded":"2024-04-25T11:27:26.906567+0000","last_fullsized":"2024-04-25T11:27:26.906567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579991+0000","last_change":"2024-04-25T11:27:25.597364+0000","last_active":"2024-04-25T11:27:26.579991+0000","last_peered":"2024-04-25T11:27:26.579991+0000","last_clean":"2024-04-25T11:27:26.579991+0000","last_became_active":"2024-04-25T11:27:25.596928+0000","last_became_peered":"2024-04-25T11:27:25.596928+0000","last_unstale":"2024-04-25T11:27:26.579991+0000","last_undegraded":"2024-04-25T11:27:26.579991+0000","last_fullsized":"2024-04-25T11:27:26.579991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:24.646664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577102+0000","last_change":"2024-04-25T11:27:25.570795+0000","last_active":"2024-04-25T11:27:26.577102+0000","last_peered":"2024-04-25T11:27:26.577102+0000","last_clean":"2024-04-25T11:27:26.577102+0000","last_became_active":"2024-04-25T11:27:25.568542+0000","last_became_peered":"2024-04-25T11:27:25.568542+0000","last_unstale":"2024-04-25T11:27:26.577102+0000","last_undegraded":"2024-04-25T11:27:26.577102+0000","last_fullsized":"2024-04-25T11:27:26.577102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.594097+0000","last_change":"2024-04-25T11:27:24.568405+0000","last_active":"2024-04-25T11:27:25.594097+0000","last_peered":"2024-04-25T11:27:25.594097+0000","last_clean":"2024-04-25T11:27:25.594097+0000","last_became_active":"2024-04-25T11:27:24.568062+0000","last_became_peered":"2024-04-25T11:27:24.568062+0000","last_unstale":"2024-04-25T11:27:25.594097+0000","last_undegraded":"2024-04-25T11:27:25.594097+0000","last_fullsized":"2024-04-25T11:27:25.594097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:54.891145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109354+0000","last_change":"2024-04-25T11:27:25.597308+0000","last_active":"2024-04-25T11:27:27.109354+0000","last_peered":"2024-04-25T11:27:27.109354+0000","last_clean":"2024-04-25T11:27:27.109354+0000","last_became_active":"2024-04-25T11:27:25.596943+0000","last_became_peered":"2024-04-25T11:27:25.596943+0000","last_unstale":"2024-04-25T11:27:27.109354+0000","last_undegraded":"2024-04-25T11:27:27.109354+0000","last_fullsized":"2024-04-25T11:27:27.109354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111690+0000","last_change":"2024-04-25T11:27:24.572640+0000","last_active":"2024-04-25T11:27:27.111690+0000","last_peered":"2024-04-25T11:27:27.111690+0000","last_clean":"2024-04-25T11:27:27.111690+0000","last_became_active":"2024-04-25T11:27:24.572119+0000","last_became_peered":"2024-04-25T11:27:24.572119+0000","last_unstale":"2024-04-25T11:27:27.111690+0000","last_undegraded":"2024-04-25T11:27:27.111690+0000","last_fullsized":"2024-04-25T11:27:27.111690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:12.700897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110506+0000","last_change":"2024-04-25T11:27:24.576360+0000","last_active":"2024-04-25T11:27:27.110506+0000","last_peered":"2024-04-25T11:27:27.110506+0000","last_clean":"2024-04-25T11:27:27.110506+0000","last_became_active":"2024-04-25T11:27:24.572890+0000","last_became_peered":"2024-04-25T11:27:24.572890+0000","last_unstale":"2024-04-25T11:27:27.110506+0000","last_undegraded":"2024-04-25T11:27:27.110506+0000","last_fullsized":"2024-04-25T11:27:27.110506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.593312+0000","last_change":"2024-04-25T11:27:25.589395+0000","last_active":"2024-04-25T11:27:25.593312+0000","last_peered":"2024-04-25T11:27:25.593312+0000","last_clean":"2024-04-25T11:27:25.593312+0000","last_became_active":"2024-04-25T11:27:25.588593+0000","last_became_peered":"2024-04-25T11:27:25.588593+0000","last_unstale":"2024-04-25T11:27:25.593312+0000","last_undegraded":"2024-04-25T11:27:25.593312+0000","last_fullsized":"2024-04-25T11:27:25.593312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:32.080821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109801+0000","last_change":"2024-04-25T11:27:24.558134+0000","last_active":"2024-04-25T11:27:27.109801+0000","last_peered":"2024-04-25T11:27:27.109801+0000","last_clean":"2024-04-25T11:27:27.109801+0000","last_became_active":"2024-04-25T11:27:24.557406+0000","last_became_peered":"2024-04-25T11:27:24.557406+0000","last_unstale":"2024-04-25T11:27:27.109801+0000","last_undegraded":"2024-04-25T11:27:27.109801+0000","last_fullsized":"2024-04-25T11:27:27.109801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:44.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109507+0000","last_change":"2024-04-25T11:27:25.592503+0000","last_active":"2024-04-25T11:27:27.109507+0000","last_peered":"2024-04-25T11:27:27.109507+0000","last_clean":"2024-04-25T11:27:27.109507+0000","last_became_active":"2024-04-25T11:27:25.592131+0000","last_became_peered":"2024-04-25T11:27:25.592131+0000","last_unstale":"2024-04-25T11:27:27.109507+0000","last_undegraded":"2024-04-25T11:27:27.109507+0000","last_fullsized":"2024-04-25T11:27:27.109507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.022859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578279+0000","last_change":"2024-04-25T11:27:24.572792+0000","last_active":"2024-04-25T11:27:26.578279+0000","last_peered":"2024-04-25T11:27:26.578279+0000","last_clean":"2024-04-25T11:27:26.578279+0000","last_became_active":"2024-04-25T11:27:24.572419+0000","last_became_peered":"2024-04-25T11:27:24.572419+0000","last_unstale":"2024-04-25T11:27:26.578279+0000","last_undegraded":"2024-04-25T11:27:26.578279+0000","last_fullsized":"2024-04-25T11:27:26.578279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581607+0000","last_change":"2024-04-25T11:27:25.590899+0000","last_active":"2024-04-25T11:27:26.581607+0000","last_peered":"2024-04-25T11:27:26.581607+0000","last_clean":"2024-04-25T11:27:26.581607+0000","last_became_active":"2024-04-25T11:27:25.588335+0000","last_became_peered":"2024-04-25T11:27:25.588335+0000","last_unstale":"2024-04-25T11:27:26.581607+0000","last_undegraded":"2024-04-25T11:27:26.581607+0000","last_fullsized":"2024-04-25T11:27:26.581607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109319+0000","last_change":"2024-04-25T11:27:24.568514+0000","last_active":"2024-04-25T11:27:27.109319+0000","last_peered":"2024-04-25T11:27:27.109319+0000","last_clean":"2024-04-25T11:27:27.109319+0000","last_became_active":"2024-04-25T11:27:24.567092+0000","last_became_peered":"2024-04-25T11:27:24.567092+0000","last_unstale":"2024-04-25T11:27:27.109319+0000","last_undegraded":"2024-04-25T11:27:27.109319+0000","last_fullsized":"2024-04-25T11:27:27.109319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.585325+0000","last_change":"2024-04-25T11:27:25.577485+0000","last_active":"2024-04-25T11:27:25.585325+0000","last_peered":"2024-04-25T11:27:25.585325+0000","last_clean":"2024-04-25T11:27:25.585325+0000","last_became_active":"2024-04-25T11:27:25.577128+0000","last_became_peered":"2024-04-25T11:27:25.577128+0000","last_unstale":"2024-04-25T11:27:25.585325+0000","last_undegraded":"2024-04-25T11:27:25.585325+0000","last_fullsized":"2024-04-25T11:27:25.585325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:52.628075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906830+0000","last_change":"2024-04-25T11:27:24.581360+0000","last_active":"2024-04-25T11:27:26.906830+0000","last_peered":"2024-04-25T11:27:26.906830+0000","last_clean":"2024-04-25T11:27:26.906830+0000","last_became_active":"2024-04-25T11:27:24.580827+0000","last_became_peered":"2024-04-25T11:27:24.580827+0000","last_unstale":"2024-04-25T11:27:26.906830+0000","last_undegraded":"2024-04-25T11:27:26.906830+0000","last_fullsized":"2024-04-25T11:27:26.906830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108897+0000","last_change":"2024-04-25T11:27:25.585619+0000","last_active":"2024-04-25T11:27:27.108897+0000","last_peered":"2024-04-25T11:27:27.108897+0000","last_clean":"2024-04-25T11:27:27.108897+0000","last_became_active":"2024-04-25T11:27:25.585314+0000","last_became_peered":"2024-04-25T11:27:25.585314+0000","last_unstale":"2024-04-25T11:27:27.108897+0000","last_undegraded":"2024-04-25T11:27:27.108897+0000","last_fullsized":"2024-04-25T11:27:27.108897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581079+0000","last_change":"2024-04-25T11:27:24.573258+0000","last_active":"2024-04-25T11:27:26.581079+0000","last_peered":"2024-04-25T11:27:26.581079+0000","last_clean":"2024-04-25T11:27:26.581079+0000","last_became_active":"2024-04-25T11:27:24.572900+0000","last_became_peered":"2024-04-25T11:27:24.572900+0000","last_unstale":"2024-04-25T11:27:26.581079+0000","last_undegraded":"2024-04-25T11:27:26.581079+0000","last_fullsized":"2024-04-25T11:27:26.581079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108997+0000","last_change":"2024-04-25T11:27:25.597408+0000","last_active":"2024-04-25T11:27:27.108997+0000","last_peered":"2024-04-25T11:27:27.108997+0000","last_clean":"2024-04-25T11:27:27.108997+0000","last_became_active":"2024-04-25T11:27:25.597094+0000","last_became_peered":"2024-04-25T11:27:25.597094+0000","last_unstale":"2024-04-25T11:27:27.108997+0000","last_undegraded":"2024-04-25T11:27:27.108997+0000","last_fullsized":"2024-04-25T11:27:27.108997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:21.541582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906817+0000","last_change":"2024-04-25T11:27:24.577388+0000","last_active":"2024-04-25T11:27:26.906817+0000","last_peered":"2024-04-25T11:27:26.906817+0000","last_clean":"2024-04-25T11:27:26.906817+0000","last_became_active":"2024-04-25T11:27:24.576282+0000","last_became_peered":"2024-04-25T11:27:24.576282+0000","last_unstale":"2024-04-25T11:27:26.906817+0000","last_undegraded":"2024-04-25T11:27:26.906817+0000","last_fullsized":"2024-04-25T11:27:26.906817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110781+0000","last_change":"2024-04-25T11:27:25.580185+0000","last_active":"2024-04-25T11:27:27.110781+0000","last_peered":"2024-04-25T11:27:27.110781+0000","last_clean":"2024-04-25T11:27:27.110781+0000","last_became_active":"2024-04-25T11:27:25.579504+0000","last_became_peered":"2024-04-25T11:27:25.579504+0000","last_unstale":"2024-04-25T11:27:27.110781+0000","last_undegraded":"2024-04-25T11:27:27.110781+0000","last_fullsized":"2024-04-25T11:27:27.110781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:53.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112046+0000","last_change":"2024-04-25T11:27:24.570725+0000","last_active":"2024-04-25T11:27:27.112046+0000","last_peered":"2024-04-25T11:27:27.112046+0000","last_clean":"2024-04-25T11:27:27.112046+0000","last_became_active":"2024-04-25T11:27:24.570329+0000","last_became_peered":"2024-04-25T11:27:24.570329+0000","last_unstale":"2024-04-25T11:27:27.112046+0000","last_undegraded":"2024-04-25T11:27:27.112046+0000","last_fullsized":"2024-04-25T11:27:27.112046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108584+0000","last_change":"2024-04-25T11:27:25.593697+0000","last_active":"2024-04-25T11:27:27.108584+0000","last_peered":"2024-04-25T11:27:27.108584+0000","last_clean":"2024-04-25T11:27:27.108584+0000","last_became_active":"2024-04-25T11:27:25.593321+0000","last_became_peered":"2024-04-25T11:27:25.593321+0000","last_unstale":"2024-04-25T11:27:27.108584+0000","last_undegraded":"2024-04-25T11:27:27.108584+0000","last_fullsized":"2024-04-25T11:27:27.108584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111788+0000","last_change":"2024-04-25T11:27:24.575295+0000","last_active":"2024-04-25T11:27:27.111788+0000","last_peered":"2024-04-25T11:27:27.111788+0000","last_clean":"2024-04-25T11:27:27.111788+0000","last_became_active":"2024-04-25T11:27:24.574577+0000","last_became_peered":"2024-04-25T11:27:24.574577+0000","last_unstale":"2024-04-25T11:27:27.111788+0000","last_undegraded":"2024-04-25T11:27:27.111788+0000","last_fullsized":"2024-04-25T11:27:27.111788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577619+0000","last_change":"2024-04-25T11:27:25.576314+0000","last_active":"2024-04-25T11:27:26.577619+0000","last_peered":"2024-04-25T11:27:26.577619+0000","last_clean":"2024-04-25T11:27:26.577619+0000","last_became_active":"2024-04-25T11:27:25.575841+0000","last_became_peered":"2024-04-25T11:27:25.575841+0000","last_unstale":"2024-04-25T11:27:26.577619+0000","last_undegraded":"2024-04-25T11:27:26.577619+0000","last_fullsized":"2024-04-25T11:27:26.577619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108586+0000","last_change":"2024-04-25T11:27:24.577138+0000","last_active":"2024-04-25T11:27:27.108586+0000","last_peered":"2024-04-25T11:27:27.108586+0000","last_clean":"2024-04-25T11:27:27.108586+0000","last_became_active":"2024-04-25T11:27:24.576752+0000","last_became_peered":"2024-04-25T11:27:24.576752+0000","last_unstale":"2024-04-25T11:27:27.108586+0000","last_undegraded":"2024-04-25T11:27:27.108586+0000","last_fullsized":"2024-04-25T11:27:27.108586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110175+0000","last_change":"2024-04-25T11:27:25.590200+0000","last_active":"2024-04-25T11:27:27.110175+0000","last_peered":"2024-04-25T11:27:27.110175+0000","last_clean":"2024-04-25T11:27:27.110175+0000","last_became_active":"2024-04-25T11:27:25.588485+0000","last_became_peered":"2024-04-25T11:27:25.588485+0000","last_unstale":"2024-04-25T11:27:27.110175+0000","last_undegraded":"2024-04-25T11:27:27.110175+0000","last_fullsized":"2024-04-25T11:27:27.110175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:47.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110702+0000","last_change":"2024-04-25T11:27:24.578005+0000","last_active":"2024-04-25T11:27:27.110702+0000","last_peered":"2024-04-25T11:27:27.110702+0000","last_clean":"2024-04-25T11:27:27.110702+0000","last_became_active":"2024-04-25T11:27:24.577247+0000","last_became_peered":"2024-04-25T11:27:24.577247+0000","last_unstale":"2024-04-25T11:27:27.110702+0000","last_undegraded":"2024-04-25T11:27:27.110702+0000","last_fullsized":"2024-04-25T11:27:27.110702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581702+0000","last_change":"2024-04-25T11:27:25.604626+0000","last_active":"2024-04-25T11:27:26.581702+0000","last_peered":"2024-04-25T11:27:26.581702+0000","last_clean":"2024-04-25T11:27:26.581702+0000","last_became_active":"2024-04-25T11:27:25.604303+0000","last_became_peered":"2024-04-25T11:27:25.604303+0000","last_unstale":"2024-04-25T11:27:26.581702+0000","last_undegraded":"2024-04-25T11:27:26.581702+0000","last_fullsized":"2024-04-25T11:27:26.581702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110067+0000","last_change":"2024-04-25T11:27:24.574248+0000","last_active":"2024-04-25T11:27:27.110067+0000","last_peered":"2024-04-25T11:27:27.110067+0000","last_clean":"2024-04-25T11:27:27.110067+0000","last_became_active":"2024-04-25T11:27:24.573749+0000","last_became_peered":"2024-04-25T11:27:24.573749+0000","last_unstale":"2024-04-25T11:27:27.110067+0000","last_undegraded":"2024-04-25T11:27:27.110067+0000","last_fullsized":"2024-04-25T11:27:27.110067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110355+0000","last_change":"2024-04-25T11:27:25.578264+0000","last_active":"2024-04-25T11:27:27.110355+0000","last_peered":"2024-04-25T11:27:27.110355+0000","last_clean":"2024-04-25T11:27:27.110355+0000","last_became_active":"2024-04-25T11:27:25.577877+0000","last_became_peered":"2024-04-25T11:27:25.577877+0000","last_unstale":"2024-04-25T11:27:27.110355+0000","last_undegraded":"2024-04-25T11:27:27.110355+0000","last_fullsized":"2024-04-25T11:27:27.110355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.596985+0000","last_change":"2024-04-25T11:27:24.570309+0000","last_active":"2024-04-25T11:27:25.596985+0000","last_peered":"2024-04-25T11:27:25.596985+0000","last_clean":"2024-04-25T11:27:25.596985+0000","last_became_active":"2024-04-25T11:27:24.568762+0000","last_became_peered":"2024-04-25T11:27:24.568762+0000","last_unstale":"2024-04-25T11:27:25.596985+0000","last_undegraded":"2024-04-25T11:27:25.596985+0000","last_fullsized":"2024-04-25T11:27:25.596985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.461035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110466+0000","last_change":"2024-04-25T11:27:25.592429+0000","last_active":"2024-04-25T11:27:27.110466+0000","last_peered":"2024-04-25T11:27:27.110466+0000","last_clean":"2024-04-25T11:27:27.110466+0000","last_became_active":"2024-04-25T11:27:25.591596+0000","last_became_peered":"2024-04-25T11:27:25.591596+0000","last_unstale":"2024-04-25T11:27:27.110466+0000","last_undegraded":"2024-04-25T11:27:27.110466+0000","last_fullsized":"2024-04-25T11:27:27.110466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.577233+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.576780+0000","last_became_peered":"2024-04-25T11:27:24.576780+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111112+0000","last_change":"2024-04-25T11:27:25.587607+0000","last_active":"2024-04-25T11:27:27.111112+0000","last_peered":"2024-04-25T11:27:27.111112+0000","last_clean":"2024-04-25T11:27:27.111112+0000","last_became_active":"2024-04-25T11:27:25.587311+0000","last_became_peered":"2024-04-25T11:27:25.587311+0000","last_unstale":"2024-04-25T11:27:27.111112+0000","last_undegraded":"2024-04-25T11:27:27.111112+0000","last_fullsized":"2024-04-25T11:27:27.111112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:46.539664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.576671+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:27:26.576671+0000","last_peered":"2024-04-25T11:27:26.576671+0000","last_clean":"2024-04-25T11:27:26.576671+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:27:26.576671+0000","last_undegraded":"2024-04-25T11:27:26.576671+0000","last_fullsized":"2024-04-25T11:27:26.576671+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108857+0000","last_change":"2024-04-25T11:27:24.556284+0000","last_active":"2024-04-25T11:27:27.108857+0000","last_peered":"2024-04-25T11:27:27.108857+0000","last_clean":"2024-04-25T11:27:27.108857+0000","last_became_active":"2024-04-25T11:27:24.555672+0000","last_became_peered":"2024-04-25T11:27:24.555672+0000","last_unstale":"2024-04-25T11:27:27.108857+0000","last_undegraded":"2024-04-25T11:27:27.108857+0000","last_fullsized":"2024-04-25T11:27:27.108857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108843+0000","last_change":"2024-04-25T11:27:25.580847+0000","last_active":"2024-04-25T11:27:27.108843+0000","last_peered":"2024-04-25T11:27:27.108843+0000","last_clean":"2024-04-25T11:27:27.108843+0000","last_became_active":"2024-04-25T11:27:25.580468+0000","last_became_peered":"2024-04-25T11:27:25.580468+0000","last_unstale":"2024-04-25T11:27:27.108843+0000","last_undegraded":"2024-04-25T11:27:27.108843+0000","last_fullsized":"2024-04-25T11:27:27.108843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110149+0000","last_change":"2024-04-25T11:27:24.570617+0000","last_active":"2024-04-25T11:27:27.110149+0000","last_peered":"2024-04-25T11:27:27.110149+0000","last_clean":"2024-04-25T11:27:27.110149+0000","last_became_active":"2024-04-25T11:27:24.570392+0000","last_became_peered":"2024-04-25T11:27:24.570392+0000","last_unstale":"2024-04-25T11:27:27.110149+0000","last_undegraded":"2024-04-25T11:27:27.110149+0000","last_fullsized":"2024-04-25T11:27:27.110149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110158+0000","last_change":"2024-04-25T11:27:25.586750+0000","last_active":"2024-04-25T11:27:27.110158+0000","last_peered":"2024-04-25T11:27:27.110158+0000","last_clean":"2024-04-25T11:27:27.110158+0000","last_became_active":"2024-04-25T11:27:25.582935+0000","last_became_peered":"2024-04-25T11:27:25.582935+0000","last_unstale":"2024-04-25T11:27:27.110158+0000","last_undegraded":"2024-04-25T11:27:27.110158+0000","last_fullsized":"2024-04-25T11:27:27.110158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906813+0000","last_change":"2024-04-25T11:27:24.578213+0000","last_active":"2024-04-25T11:27:26.906813+0000","last_peered":"2024-04-25T11:27:26.906813+0000","last_clean":"2024-04-25T11:27:26.906813+0000","last_became_active":"2024-04-25T11:27:24.576970+0000","last_became_peered":"2024-04-25T11:27:24.576970+0000","last_unstale":"2024-04-25T11:27:26.906813+0000","last_undegraded":"2024-04-25T11:27:26.906813+0000","last_fullsized":"2024-04-25T11:27:26.906813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:53.598617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.584833+0000","last_change":"2024-04-25T11:27:25.581442+0000","last_active":"2024-04-25T11:27:25.584833+0000","last_peered":"2024-04-25T11:27:25.584833+0000","last_clean":"2024-04-25T11:27:25.584833+0000","last_became_active":"2024-04-25T11:27:25.579355+0000","last_became_peered":"2024-04-25T11:27:25.579355+0000","last_unstale":"2024-04-25T11:27:25.584833+0000","last_undegraded":"2024-04-25T11:27:25.584833+0000","last_fullsized":"2024-04-25T11:27:25.584833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:02.918428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112309+0000","last_change":"2024-04-25T11:27:24.565287+0000","last_active":"2024-04-25T11:27:27.112309+0000","last_peered":"2024-04-25T11:27:27.112309+0000","last_clean":"2024-04-25T11:27:27.112309+0000","last_became_active":"2024-04-25T11:27:24.564368+0000","last_became_peered":"2024-04-25T11:27:24.564368+0000","last_unstale":"2024-04-25T11:27:27.112309+0000","last_undegraded":"2024-04-25T11:27:27.112309+0000","last_fullsized":"2024-04-25T11:27:27.112309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111142+0000","last_change":"2024-04-25T11:27:25.604124+0000","last_active":"2024-04-25T11:27:27.111142+0000","last_peered":"2024-04-25T11:27:27.111142+0000","last_clean":"2024-04-25T11:27:27.111142+0000","last_became_active":"2024-04-25T11:27:25.603769+0000","last_became_peered":"2024-04-25T11:27:25.603769+0000","last_unstale":"2024-04-25T11:27:27.111142+0000","last_undegraded":"2024-04-25T11:27:27.111142+0000","last_fullsized":"2024-04-25T11:27:27.111142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.550311+0000","last_change":"2024-04-25T11:27:24.572130+0000","last_active":"2024-04-25T11:27:25.550311+0000","last_peered":"2024-04-25T11:27:25.550311+0000","last_clean":"2024-04-25T11:27:25.550311+0000","last_became_active":"2024-04-25T11:27:24.569314+0000","last_became_peered":"2024-04-25T11:27:24.569314+0000","last_unstale":"2024-04-25T11:27:25.550311+0000","last_undegraded":"2024-04-25T11:27:25.550311+0000","last_fullsized":"2024-04-25T11:27:25.550311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:18.304561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580084+0000","last_change":"2024-04-25T11:27:25.597482+0000","last_active":"2024-04-25T11:27:26.580084+0000","last_peered":"2024-04-25T11:27:26.580084+0000","last_clean":"2024-04-25T11:27:26.580084+0000","last_became_active":"2024-04-25T11:27:25.595957+0000","last_became_peered":"2024-04-25T11:27:25.595957+0000","last_unstale":"2024-04-25T11:27:26.580084+0000","last_undegraded":"2024-04-25T11:27:26.580084+0000","last_fullsized":"2024-04-25T11:27:26.580084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.572383+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.572006+0000","last_became_peered":"2024-04-25T11:27:24.572006+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:09.042983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111372+0000","last_change":"2024-04-25T11:27:25.578914+0000","last_active":"2024-04-25T11:27:27.111372+0000","last_peered":"2024-04-25T11:27:27.111372+0000","last_clean":"2024-04-25T11:27:27.111372+0000","last_became_active":"2024-04-25T11:27:25.578699+0000","last_became_peered":"2024-04-25T11:27:25.578699+0000","last_unstale":"2024-04-25T11:27:27.111372+0000","last_undegraded":"2024-04-25T11:27:27.111372+0000","last_fullsized":"2024-04-25T11:27:27.111372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110225+0000","last_change":"2024-04-25T11:27:24.579698+0000","last_active":"2024-04-25T11:27:27.110225+0000","last_peered":"2024-04-25T11:27:27.110225+0000","last_clean":"2024-04-25T11:27:27.110225+0000","last_became_active":"2024-04-25T11:27:24.575825+0000","last_became_peered":"2024-04-25T11:27:24.575825+0000","last_unstale":"2024-04-25T11:27:27.110225+0000","last_undegraded":"2024-04-25T11:27:27.110225+0000","last_fullsized":"2024-04-25T11:27:27.110225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109333+0000","last_change":"2024-04-25T11:27:25.588979+0000","last_active":"2024-04-25T11:27:27.109333+0000","last_peered":"2024-04-25T11:27:27.109333+0000","last_clean":"2024-04-25T11:27:27.109333+0000","last_became_active":"2024-04-25T11:27:25.588683+0000","last_became_peered":"2024-04-25T11:27:25.588683+0000","last_unstale":"2024-04-25T11:27:27.109333+0000","last_undegraded":"2024-04-25T11:27:27.109333+0000","last_fullsized":"2024-04-25T11:27:27.109333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113417+0000","last_change":"2024-04-25T11:27:24.572521+0000","last_active":"2024-04-25T11:27:27.113417+0000","last_peered":"2024-04-25T11:27:27.113417+0000","last_clean":"2024-04-25T11:27:27.113417+0000","last_became_active":"2024-04-25T11:27:24.572257+0000","last_became_peered":"2024-04-25T11:27:24.572257+0000","last_unstale":"2024-04-25T11:27:27.113417+0000","last_undegraded":"2024-04-25T11:27:27.113417+0000","last_fullsized":"2024-04-25T11:27:27.113417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110515+0000","last_change":"2024-04-25T11:27:25.574450+0000","last_active":"2024-04-25T11:27:27.110515+0000","last_peered":"2024-04-25T11:27:27.110515+0000","last_clean":"2024-04-25T11:27:27.110515+0000","last_became_active":"2024-04-25T11:27:25.573078+0000","last_became_peered":"2024-04-25T11:27:25.573078+0000","last_unstale":"2024-04-25T11:27:27.110515+0000","last_undegraded":"2024-04-25T11:27:27.110515+0000","last_fullsized":"2024-04-25T11:27:27.110515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.823894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.107620+0000","last_change":"2024-04-25T11:27:24.566069+0000","last_active":"2024-04-25T11:27:27.107620+0000","last_peered":"2024-04-25T11:27:27.107620+0000","last_clean":"2024-04-25T11:27:27.107620+0000","last_became_active":"2024-04-25T11:27:24.565642+0000","last_became_peered":"2024-04-25T11:27:24.565642+0000","last_unstale":"2024-04-25T11:27:27.107620+0000","last_undegraded":"2024-04-25T11:27:27.107620+0000","last_fullsized":"2024-04-25T11:27:27.107620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110907+0000","last_change":"2024-04-25T11:27:25.584529+0000","last_active":"2024-04-25T11:27:27.110907+0000","last_peered":"2024-04-25T11:27:27.110907+0000","last_clean":"2024-04-25T11:27:27.110907+0000","last_became_active":"2024-04-25T11:27:25.584130+0000","last_became_peered":"2024-04-25T11:27:25.584130+0000","last_unstale":"2024-04-25T11:27:27.110907+0000","last_undegraded":"2024-04-25T11:27:27.110907+0000","last_fullsized":"2024-04-25T11:27:27.110907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:27.614041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108983+0000","last_change":"2024-04-25T11:27:24.580477+0000","last_active":"2024-04-25T11:27:27.108983+0000","last_peered":"2024-04-25T11:27:27.108983+0000","last_clean":"2024-04-25T11:27:27.108983+0000","last_became_active":"2024-04-25T11:27:24.578736+0000","last_became_peered":"2024-04-25T11:27:24.578736+0000","last_unstale":"2024-04-25T11:27:27.108983+0000","last_undegraded":"2024-04-25T11:27:27.108983+0000","last_fullsized":"2024-04-25T11:27:27.108983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:44.218131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109747+0000","last_change":"2024-04-25T11:27:25.581233+0000","last_active":"2024-04-25T11:27:27.109747+0000","last_peered":"2024-04-25T11:27:27.109747+0000","last_clean":"2024-04-25T11:27:27.109747+0000","last_became_active":"2024-04-25T11:27:25.580073+0000","last_became_peered":"2024-04-25T11:27:25.580073+0000","last_unstale":"2024-04-25T11:27:27.109747+0000","last_undegraded":"2024-04-25T11:27:27.109747+0000","last_fullsized":"2024-04-25T11:27:27.109747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:53.858736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110994+0000","last_change":"2024-04-25T11:27:24.577164+0000","last_active":"2024-04-25T11:27:27.110994+0000","last_peered":"2024-04-25T11:27:27.110994+0000","last_clean":"2024-04-25T11:27:27.110994+0000","last_became_active":"2024-04-25T11:27:24.576795+0000","last_became_peered":"2024-04-25T11:27:24.576795+0000","last_unstale":"2024-04-25T11:27:27.110994+0000","last_undegraded":"2024-04-25T11:27:27.110994+0000","last_fullsized":"2024-04-25T11:27:27.110994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906953+0000","last_change":"2024-04-25T11:27:25.594079+0000","last_active":"2024-04-25T11:27:26.906953+0000","last_peered":"2024-04-25T11:27:26.906953+0000","last_clean":"2024-04-25T11:27:26.906953+0000","last_became_active":"2024-04-25T11:27:25.592965+0000","last_became_peered":"2024-04-25T11:27:25.592965+0000","last_unstale":"2024-04-25T11:27:26.906953+0000","last_undegraded":"2024-04-25T11:27:26.906953+0000","last_fullsized":"2024-04-25T11:27:26.906953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108732+0000","last_change":"2024-04-25T11:27:24.559748+0000","last_active":"2024-04-25T11:27:27.108732+0000","last_peered":"2024-04-25T11:27:27.108732+0000","last_clean":"2024-04-25T11:27:27.108732+0000","last_became_active":"2024-04-25T11:27:24.558116+0000","last_became_peered":"2024-04-25T11:27:24.558116+0000","last_unstale":"2024-04-25T11:27:27.108732+0000","last_undegraded":"2024-04-25T11:27:27.108732+0000","last_fullsized":"2024-04-25T11:27:27.108732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111402+0000","last_change":"2024-04-25T11:27:25.578715+0000","last_active":"2024-04-25T11:27:27.111402+0000","last_peered":"2024-04-25T11:27:27.111402+0000","last_clean":"2024-04-25T11:27:27.111402+0000","last_became_active":"2024-04-25T11:27:25.578355+0000","last_became_peered":"2024-04-25T11:27:25.578355+0000","last_unstale":"2024-04-25T11:27:27.111402+0000","last_undegraded":"2024-04-25T11:27:27.111402+0000","last_fullsized":"2024-04-25T11:27:27.111402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:57.742260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111309+0000","last_change":"2024-04-25T11:27:25.587963+0000","last_active":"2024-04-25T11:27:27.111309+0000","last_peered":"2024-04-25T11:27:27.111309+0000","last_clean":"2024-04-25T11:27:27.111309+0000","last_became_active":"2024-04-25T11:27:25.587609+0000","last_became_peered":"2024-04-25T11:27:25.587609+0000","last_unstale":"2024-04-25T11:27:27.111309+0000","last_undegraded":"2024-04-25T11:27:27.111309+0000","last_fullsized":"2024-04-25T11:27:27.111309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:48.328746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110834+0000","last_change":"2024-04-25T11:27:24.575077+0000","last_active":"2024-04-25T11:27:27.110834+0000","last_peered":"2024-04-25T11:27:27.110834+0000","last_clean":"2024-04-25T11:27:27.110834+0000","last_became_active":"2024-04-25T11:27:24.574000+0000","last_became_peered":"2024-04-25T11:27:24.574000+0000","last_unstale":"2024-04-25T11:27:27.110834+0000","last_undegraded":"2024-04-25T11:27:27.110834+0000","last_fullsized":"2024-04-25T11:27:27.110834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578237+0000","last_change":"2024-04-25T11:27:25.592421+0000","last_active":"2024-04-25T11:27:26.578237+0000","last_peered":"2024-04-25T11:27:26.578237+0000","last_clean":"2024-04-25T11:27:26.578237+0000","last_became_active":"2024-04-25T11:27:25.591913+0000","last_became_peered":"2024-04-25T11:27:25.591913+0000","last_unstale":"2024-04-25T11:27:26.578237+0000","last_undegraded":"2024-04-25T11:27:26.578237+0000","last_fullsized":"2024-04-25T11:27:26.578237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108846+0000","last_change":"2024-04-25T11:27:24.570375+0000","last_active":"2024-04-25T11:27:27.108846+0000","last_peered":"2024-04-25T11:27:27.108846+0000","last_clean":"2024-04-25T11:27:27.108846+0000","last_became_active":"2024-04-25T11:27:24.570081+0000","last_became_peered":"2024-04-25T11:27:24.570081+0000","last_unstale":"2024-04-25T11:27:27.108846+0000","last_undegraded":"2024-04-25T11:27:27.108846+0000","last_fullsized":"2024-04-25T11:27:27.108846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.568828+0000","last_change":"2024-04-25T11:27:25.567438+0000","last_active":"2024-04-25T11:27:25.568828+0000","last_peered":"2024-04-25T11:27:25.568828+0000","last_clean":"2024-04-25T11:27:25.568828+0000","last_became_active":"2024-04-25T11:27:25.567114+0000","last_became_peered":"2024-04-25T11:27:25.567114+0000","last_unstale":"2024-04-25T11:27:25.568828+0000","last_undegraded":"2024-04-25T11:27:25.568828+0000","last_fullsized":"2024-04-25T11:27:25.568828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:25.294830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.594450+0000","last_change":"2024-04-25T11:27:24.572110+0000","last_active":"2024-04-25T11:27:25.594450+0000","last_peered":"2024-04-25T11:27:25.594450+0000","last_clean":"2024-04-25T11:27:25.594450+0000","last_became_active":"2024-04-25T11:27:24.571670+0000","last_became_peered":"2024-04-25T11:27:24.571670+0000","last_unstale":"2024-04-25T11:27:25.594450+0000","last_undegraded":"2024-04-25T11:27:25.594450+0000","last_fullsized":"2024-04-25T11:27:25.594450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:20.625965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T11:27:25.585315+0000","last_change":"2024-04-25T11:27:25.582462+0000","last_active":"2024-04-25T11:27:25.585315+0000","last_peered":"2024-04-25T11:27:25.585315+0000","last_clean":"2024-04-25T11:27:25.585315+0000","last_became_active":"2024-04-25T11:27:25.581902+0000","last_became_peered":"2024-04-25T11:27:25.581902+0000","last_unstale":"2024-04-25T11:27:25.585315+0000","last_undegraded":"2024-04-25T11:27:25.585315+0000","last_fullsized":"2024-04-25T11:27:25.585315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:47.472986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109835+0000","last_change":"2024-04-25T11:27:24.567065+0000","last_active":"2024-04-25T11:27:27.109835+0000","last_peered":"2024-04-25T11:27:27.109835+0000","last_clean":"2024-04-25T11:27:27.109835+0000","last_became_active":"2024-04-25T11:27:24.566049+0000","last_became_peered":"2024-04-25T11:27:24.566049+0000","last_unstale":"2024-04-25T11:27:27.109835+0000","last_undegraded":"2024-04-25T11:27:27.109835+0000","last_fullsized":"2024-04-25T11:27:27.109835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":69984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":72268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":71596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:27:30.891 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:27:30.891 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:27:30.891 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:27:30.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T11:27:30.892 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T11:27:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.228+0000 7f5e4ac00640 1 Processor -- start 2024-04-25T11:27:31.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.229+0000 7f5e4ac00640 1 -- start start 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.231+0000 7f5e4ac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5e441443c0 0x7f5e4406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.231+0000 7f5e4ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e440cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.231+0000 7f5e4ac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e440cfb20 0x7f5e440ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.231+0000 7f5e4ac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5e440708d0 con 0x7f5e440cfb20 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.231+0000 7f5e4ac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4415f160 con 0x7f5e441443c0 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.231+0000 7f5e4ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5e440cff30 con 0x7f5e44070370 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e440cfb20 0x7f5e440ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5e440cfb20 0x7f5e440ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47108/0 (socket says 172.21.15.19:47108) 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 -- 172.21.15.19:0/1342891825 learned_addr learned my addr 172.21.15.19:0/1342891825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e43e00640 1 --2- 172.21.15.19:0/1342891825 >> v2:172.21.15.169:3300/0 conn(0x7f5e441443c0 0x7f5e4406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e43400640 1 --2- 172.21.15.19:0/1342891825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e440cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 -- 172.21.15.19:0/1342891825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 msgr2=0x7f5e440cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 --2- 172.21.15.19:0/1342891825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e440cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 -- 172.21.15.19:0/1342891825 >> v2:172.21.15.169:3300/0 conn(0x7f5e441443c0 msgr2=0x7f5e4406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 --2- 172.21.15.19:0/1342891825 >> v2:172.21.15.169:3300/0 conn(0x7f5e441443c0 0x7f5e4406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e48c00640 1 -- 172.21.15.19:0/1342891825 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4415f5f0 con 0x7f5e440cfb20 2024-04-25T11:27:31.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.232+0000 7f5e43400640 1 --2- 172.21.15.19:0/1342891825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e440cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.234+0000 7f5e48c00640 1 --2- 172.21.15.19:0/1342891825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e440cfb20 0x7f5e440ce330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e44072820 tx=0x7f5e3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6baaeec397c4e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.234+0000 7f5e42a00640 1 -- 172.21.15.19:0/1342891825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c036070 con 0x7f5e440cfb20 2024-04-25T11:27:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.234+0000 7f5e42a00640 1 -- 172.21.15.19:0/1342891825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c031600 con 0x7f5e440cfb20 2024-04-25T11:27:31.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.234+0000 7f5e42a00640 1 -- 172.21.15.19:0/1342891825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0319d0 con 0x7f5e440cfb20 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 -- 172.21.15.19:0/1342891825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e440cfb20 msgr2=0x7f5e440ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/1342891825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e440cfb20 0x7f5e440ce330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5e44072820 tx=0x7f5e3c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 -- 172.21.15.19:0/1342891825 shutdown_connections 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/1342891825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e440cfb20 0x7f5e440ce330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/1342891825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e440cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/1342891825 >> v2:172.21.15.169:3300/0 conn(0x7f5e441443c0 0x7f5e4406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 -- 172.21.15.19:0/1342891825 >> 172.21.15.19:0/1342891825 conn(0x7f5e440d1ae0 msgr2=0x7f5e440d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 -- 172.21.15.19:0/1342891825 shutdown_connections 2024-04-25T11:27:31.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.235+0000 7f5e4ac00640 1 -- 172.21.15.19:0/1342891825 wait complete. 2024-04-25T11:27:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.238+0000 7f5e4ac00640 1 Processor -- start 2024-04-25T11:27:31.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.238+0000 7f5e4ac00640 1 -- start start 2024-04-25T11:27:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.239+0000 7f5e4ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e44152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.239+0000 7f5e4ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e440cfb20 0x7f5e441e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.239+0000 7f5e4ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e441443c0 0x7f5e44132a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.239+0000 7f5e4ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4415edf0 con 0x7f5e441443c0 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.239+0000 7f5e4ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4415eb10 con 0x7f5e440cfb20 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.239+0000 7f5e4ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5e441447d0 con 0x7f5e44070370 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e441443c0 0x7f5e44132a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e441443c0 0x7f5e44132a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47120/0 (socket says 172.21.15.19:47120) 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 -- 172.21.15.19:0/2795493884 learned_addr learned my addr 172.21.15.19:0/2795493884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e43e00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e44152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e43400640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e440cfb20 0x7f5e441e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 -- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 msgr2=0x7f5e44152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e44152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 -- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e440cfb20 msgr2=0x7f5e441e8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e440cfb20 0x7f5e441e8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e48c00640 1 -- 172.21.15.19:0/2795493884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e44130bd0 con 0x7f5e441443c0 2024-04-25T11:27:31.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.240+0000 7f5e43e00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e44152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:31.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.241+0000 7f5e48c00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e441443c0 0x7f5e44132a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009f40 tx=0x7f5e3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:31.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.241+0000 7f5e3be00640 1 -- 172.21.15.19:0/2795493884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c036050 con 0x7f5e441443c0 2024-04-25T11:27:31.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.241+0000 7f5e3be00640 1 -- 172.21.15.19:0/2795493884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c042070 con 0x7f5e441443c0 2024-04-25T11:27:31.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.241+0000 7f5e3be00640 1 -- 172.21.15.19:0/2795493884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c009510 con 0x7f5e441443c0 2024-04-25T11:27:31.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.241+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e44130e70 con 0x7f5e441443c0 2024-04-25T11:27:31.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.241+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e44131430 con 0x7f5e441443c0 2024-04-25T11:27:31.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.243+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e44050740 con 0x7f5e441443c0 2024-04-25T11:27:31.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.244+0000 7f5e3be00640 1 -- 172.21.15.19:0/2795493884 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5e3c007bd0 con 0x7f5e441443c0 2024-04-25T11:27:31.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.245+0000 7f5e3be00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e200744b0 0x7f5e200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.245+0000 7f5e3be00640 1 -- 172.21.15.19:0/2795493884 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5e3c0b6c10 con 0x7f5e441443c0 2024-04-25T11:27:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.246+0000 7f5e43e00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e200744b0 0x7f5e200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.247+0000 7f5e43e00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e200744b0 0x7f5e200769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006fd0 tx=0x7f5e34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.249+0000 7f5e3be00640 1 -- 172.21.15.19:0/2795493884 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e3c081380 con 0x7f5e441443c0 2024-04-25T11:27:31.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.510+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5e4406f950 con 0x7f5e441443c0 2024-04-25T11:27:31.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.520+0000 7f5e3be00640 1 -- 172.21.15.19:0/2795493884 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186750 (secure 0 0 0) 0x7f5e3c080c80 con 0x7f5e441443c0 2024-04-25T11:27:31.513 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e200744b0 msgr2=0x7f5e200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e200744b0 0x7f5e200769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e34006fd0 tx=0x7f5e34008040 comp rx=0 tx=0).stop 2024-04-25T11:27:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e441443c0 msgr2=0x7f5e44132a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e441443c0 0x7f5e44132a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009f40 tx=0x7f5e3c004a40 comp rx=0 tx=0).stop 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 shutdown_connections 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e200744b0 0x7f5e200769c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e440cfb20 0x7f5e441e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e441443c0 0x7f5e44132a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 --2- 172.21.15.19:0/2795493884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e44070370 0x7f5e44152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 >> 172.21.15.19:0/2795493884 conn(0x7f5e440d1ae0 msgr2=0x7f5e4406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 shutdown_connections 2024-04-25T11:27:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.525+0000 7f5e4ac00640 1 -- 172.21.15.19:0/2795493884 wait complete. 2024-04-25T11:27:31.543 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":1049297059},{"type":"v1","addr":"172.21.15.169:6833","nonce":1049297059}]},"active_addr":"172.21.15.169:6833/1049297059","active_change":"2024-04-25T11:27:04.409114+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":2850992173}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":3965429846}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":4066849524}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.169:0","nonce":2289886101}]}]} 2024-04-25T11:27:31.546 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T11:27:31.546 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:27:31.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:31.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.815+0000 7f8f5d200640 1 Processor -- start 2024-04-25T11:27:31.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.816+0000 7f8f5d200640 1 -- start start 2024-04-25T11:27:31.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f5d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f58144350 0x7f8f580d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f5d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f58161af0 0x7f8f58161f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f5d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58162440 0x7f8f5806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f5d200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8f58162850 con 0x7f8f58161af0 2024-04-25T11:27:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f5d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8f5806fc70 con 0x7f8f58144350 2024-04-25T11:27:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f5d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5806ff50 con 0x7f8f58162440 2024-04-25T11:27:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f4fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f58161af0 0x7f8f58161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f4fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f58161af0 0x7f8f58161f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47130/0 (socket says 172.21.15.19:47130) 2024-04-25T11:27:31.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.818+0000 7f8f4fe00640 1 -- 172.21.15.19:0/2057487008 learned_addr learned my addr 172.21.15.19:0/2057487008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.819+0000 7f8f57e00640 1 --2- 172.21.15.19:0/2057487008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58162440 0x7f8f5806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.819+0000 7f8f57400640 1 --2- 172.21.15.19:0/2057487008 >> v2:172.21.15.169:3300/0 conn(0x7f8f58144350 0x7f8f580d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.819+0000 7f8f4fe00640 1 -- 172.21.15.19:0/2057487008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58162440 msgr2=0x7f8f5806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.819+0000 7f8f4fe00640 1 --2- 172.21.15.19:0/2057487008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58162440 0x7f8f5806c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.819+0000 7f8f4fe00640 1 -- 172.21.15.19:0/2057487008 >> v2:172.21.15.169:3300/0 conn(0x7f8f58144350 msgr2=0x7f8f580d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.819+0000 7f8f4fe00640 1 --2- 172.21.15.19:0/2057487008 >> v2:172.21.15.169:3300/0 conn(0x7f8f58144350 0x7f8f580d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.819+0000 7f8f4fe00640 1 -- 172.21.15.19:0/2057487008 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f58070230 con 0x7f8f58161af0 2024-04-25T11:27:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.820+0000 7f8f4fe00640 1 --2- 172.21.15.19:0/2057487008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f58161f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f580b6aa0 tx=0x7f8f4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce956659aa92c6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.820+0000 7f8f56a00640 1 -- 172.21.15.19:0/2057487008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40037070 con 0x7f8f58161af0 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.820+0000 7f8f56a00640 1 -- 172.21.15.19:0/2057487008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f40032440 con 0x7f8f58161af0 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.820+0000 7f8f56a00640 1 -- 172.21.15.19:0/2057487008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40032810 con 0x7f8f58161af0 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 -- 172.21.15.19:0/2057487008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 msgr2=0x7f8f58161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 --2- 172.21.15.19:0/2057487008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f58161f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f580b6aa0 tx=0x7f8f4002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 -- 172.21.15.19:0/2057487008 shutdown_connections 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 --2- 172.21.15.19:0/2057487008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f58161f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 --2- 172.21.15.19:0/2057487008 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58162440 0x7f8f5806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 --2- 172.21.15.19:0/2057487008 >> v2:172.21.15.169:3300/0 conn(0x7f8f58144350 0x7f8f580d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 -- 172.21.15.19:0/2057487008 >> 172.21.15.19:0/2057487008 conn(0x7f8f580d1720 msgr2=0x7f8f580d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 -- 172.21.15.19:0/2057487008 shutdown_connections 2024-04-25T11:27:31.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.821+0000 7f8f5d200640 1 -- 172.21.15.19:0/2057487008 wait complete. 2024-04-25T11:27:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.823+0000 7f8f5d200640 1 Processor -- start 2024-04-25T11:27:31.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.824+0000 7f8f5d200640 1 -- start start 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f5d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58144350 0x7f8f58157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f5d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f5812edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f5d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f58162440 0x7f8f5815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f5d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5806cb00 con 0x7f8f58161af0 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f5d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f58147910 con 0x7f8f58162440 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f5d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5806cc80 con 0x7f8f58144350 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f5812edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f5812edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47146/0 (socket says 172.21.15.19:47146) 2024-04-25T11:27:31.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.825+0000 7f8f4fe00640 1 -- 172.21.15.19:0/1147226680 learned_addr learned my addr 172.21.15.19:0/1147226680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f57400640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58144350 0x7f8f58157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f4fe00640 1 -- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58144350 msgr2=0x7f8f58157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f4fe00640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58144350 0x7f8f58157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f4fe00640 1 -- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f58162440 msgr2=0x7f8f5815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f57e00640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f58162440 0x7f8f5815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f4fe00640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f58162440 0x7f8f5815c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f4fe00640 1 -- 172.21.15.19:0/1147226680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f580b6630 con 0x7f8f58161af0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f57400640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58144350 0x7f8f58157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f4fe00640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f5812edf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f4002a220 tx=0x7f8f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.826+0000 7f8f4f400640 1 -- 172.21.15.19:0/1147226680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40037040 con 0x7f8f58161af0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.827+0000 7f8f4f400640 1 -- 172.21.15.19:0/1147226680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f40042070 con 0x7f8f58161af0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.827+0000 7f8f4f400640 1 -- 172.21.15.19:0/1147226680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40030bb0 con 0x7f8f58161af0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.827+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5812f960 con 0x7f8f58161af0 2024-04-25T11:27:31.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.827+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5812ff20 con 0x7f8f58161af0 2024-04-25T11:27:31.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.828+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f58050740 con 0x7f8f58161af0 2024-04-25T11:27:31.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.829+0000 7f8f4f400640 1 -- 172.21.15.19:0/1147226680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f40047460 con 0x7f8f58161af0 2024-04-25T11:27:31.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.831+0000 7f8f4f400640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f340744b0 0x7f8f340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:31.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.831+0000 7f8f4f400640 1 -- 172.21.15.19:0/1147226680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8f400b5f60 con 0x7f8f58161af0 2024-04-25T11:27:31.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.831+0000 7f8f57400640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f340744b0 0x7f8f340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:31.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.832+0000 7f8f57400640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f340744b0 0x7f8f340769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f58147d40 tx=0x7f8f48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:31.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:31.836+0000 7f8f4f400640 1 -- 172.21.15.19:0/1147226680 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f40080000 con 0x7f8f58161af0 2024-04-25T11:27:32.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.039+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f5806d910 con 0x7f8f58161af0 2024-04-25T11:27:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.043+0000 7f8f4f400640 1 -- 172.21.15.19:0/1147226680 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7f8f40084300 con 0x7f8f58161af0 2024-04-25T11:27:32.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:32.034 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.045+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f340744b0 msgr2=0x7f8f340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.045+0000 7f8f5d200640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f340744b0 0x7f8f340769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f58147d40 tx=0x7f8f48008040 comp rx=0 tx=0).stop 2024-04-25T11:27:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.045+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 msgr2=0x7f8f5812edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.045+0000 7f8f5d200640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f5812edf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f4002a220 tx=0x7f8f400046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 shutdown_connections 2024-04-25T11:27:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f340744b0 0x7f8f340769c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f58162440 0x7f8f5815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f58161af0 0x7f8f5812edf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 --2- 172.21.15.19:0/1147226680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f58144350 0x7f8f58157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 >> 172.21.15.19:0/1147226680 conn(0x7f8f580d1720 msgr2=0x7f8f580737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 shutdown_connections 2024-04-25T11:27:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.046+0000 7f8f5d200640 1 -- 172.21.15.19:0/1147226680 wait complete. 2024-04-25T11:27:32.065 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:27:32.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:32.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.359+0000 7fdb3fe00640 1 Processor -- start 2024-04-25T11:27:32.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.359+0000 7fdb3fe00640 1 -- start start 2024-04-25T11:27:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb3fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdb4006c5c0 0x7fdb400cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb3fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb40144370 0x7fdb400b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb3fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb400b50e0 con 0x7fdb400b4cd0 2024-04-25T11:27:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb3fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdb400d1230 con 0x7fdb40144370 2024-04-25T11:27:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb3fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdb400d06b0 con 0x7fdb4006c5c0 2024-04-25T11:27:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb37e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb40144370 0x7fdb400b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb37e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb40144370 0x7fdb400b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47148/0 (socket says 172.21.15.19:47148) 2024-04-25T11:27:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb37e00640 1 -- 172.21.15.19:0/3385869841 learned_addr learned my addr 172.21.15.19:0/3385869841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.362+0000 7fdb3ea00640 1 --2- 172.21.15.19:0/3385869841 >> v2:172.21.15.169:3300/0 conn(0x7fdb4006c5c0 0x7fdb400cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.363+0000 7fdb37e00640 1 -- 172.21.15.19:0/3385869841 >> v2:172.21.15.169:3300/0 conn(0x7fdb4006c5c0 msgr2=0x7fdb400cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.363+0000 7fdb37e00640 1 --2- 172.21.15.19:0/3385869841 >> v2:172.21.15.169:3300/0 conn(0x7fdb4006c5c0 0x7fdb400cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.363+0000 7fdb37e00640 1 -- 172.21.15.19:0/3385869841 >> v1:172.21.15.19:6789/0 conn(0x7fdb400b4cd0 legacy=0x7fdb40161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.363+0000 7fdb37e00640 1 -- 172.21.15.19:0/3385869841 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb400d0990 con 0x7fdb40144370 2024-04-25T11:27:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.363+0000 7fdb3ea00640 1 --2- 172.21.15.19:0/3385869841 >> v2:172.21.15.169:3300/0 conn(0x7fdb4006c5c0 0x7fdb400cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.364+0000 7fdb37e00640 1 --2- 172.21.15.19:0/3385869841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40144370 0x7fdb400b4790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb400726b0 tx=0x7fdb2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5647471669fc6f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:32.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.364+0000 7fdb3e000640 1 -- 172.21.15.19:0/3385869841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28037070 con 0x7fdb40144370 2024-04-25T11:27:32.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.364+0000 7fdb3e000640 1 -- 172.21.15.19:0/3385869841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb28032440 con 0x7fdb40144370 2024-04-25T11:27:32.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.364+0000 7fdb3e000640 1 -- 172.21.15.19:0/3385869841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28032810 con 0x7fdb40144370 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3385869841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40144370 msgr2=0x7fdb400b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/3385869841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40144370 0x7fdb400b4790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb400726b0 tx=0x7fdb2802a270 comp rx=0 tx=0).stop 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3385869841 shutdown_connections 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/3385869841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40144370 0x7fdb400b4790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/3385869841 >> v2:172.21.15.169:3300/0 conn(0x7fdb4006c5c0 0x7fdb400cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3385869841 >> 172.21.15.19:0/3385869841 conn(0x7fdb400d1720 msgr2=0x7fdb400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3385869841 shutdown_connections 2024-04-25T11:27:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.365+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3385869841 wait complete. 2024-04-25T11:27:32.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.367+0000 7fdb3fe00640 1 Processor -- start 2024-04-25T11:27:32.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.367+0000 7fdb3fe00640 1 -- start start 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006c5c0 0x7fdb400cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b4cd0 0x7fdb40153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40144370 0x7fdb40160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb400d02d0 con 0x7fdb4006c5c0 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb400d0f50 con 0x7fdb400b4cd0 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb400d0450 con 0x7fdb40144370 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006c5c0 0x7fdb400cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006c5c0 0x7fdb400cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47164/0 (socket says 172.21.15.19:47164) 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.369+0000 7fdb3ea00640 1 -- 172.21.15.19:0/159790710 learned_addr learned my addr 172.21.15.19:0/159790710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3f400640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40144370 0x7fdb40160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb37e00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b4cd0 0x7fdb40153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3ea00640 1 -- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40144370 msgr2=0x7fdb40160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3ea00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40144370 0x7fdb40160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3ea00640 1 -- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b4cd0 msgr2=0x7fdb40153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3ea00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b4cd0 0x7fdb40153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3ea00640 1 -- 172.21.15.19:0/159790710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4015b950 con 0x7fdb4006c5c0 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3f400640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40144370 0x7fdb40160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3ea00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006c5c0 0x7fdb400cd050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000d7d0 tx=0x7fdb3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb36a00640 1 -- 172.21.15.19:0/159790710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30004030 con 0x7fdb4006c5c0 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb36a00640 1 -- 172.21.15.19:0/159790710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb300041d0 con 0x7fdb4006c5c0 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb36a00640 1 -- 172.21.15.19:0/159790710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000b6c0 con 0x7fdb4006c5c0 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4015bc30 con 0x7fdb4006c5c0 2024-04-25T11:27:32.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.370+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb40160ed0 con 0x7fdb4006c5c0 2024-04-25T11:27:32.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.372+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb40050740 con 0x7fdb4006c5c0 2024-04-25T11:27:32.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.373+0000 7fdb36a00640 1 -- 172.21.15.19:0/159790710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdb3000b860 con 0x7fdb4006c5c0 2024-04-25T11:27:32.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.374+0000 7fdb36a00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:32.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.374+0000 7fdb36a00640 1 -- 172.21.15.19:0/159790710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdb300954a0 con 0x7fdb4006c5c0 2024-04-25T11:27:32.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.374+0000 7fdb37e00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:32.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.375+0000 7fdb37e00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb28034000 tx=0x7fdb2802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:32.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.379+0000 7fdb36a00640 1 -- 172.21.15.19:0/159790710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb3005fc30 con 0x7fdb4006c5c0 2024-04-25T11:27:32.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.585+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb4006e100 con 0x7fdb4006c5c0 2024-04-25T11:27:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.587+0000 7fdb36a00640 1 -- 172.21.15.19:0/159790710 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7fdb3005f530 con 0x7fdb4006c5c0 2024-04-25T11:27:32.579 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:32.579 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:32.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.590+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 msgr2=0x7fdb1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.590+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb28034000 tx=0x7fdb2802e000 comp rx=0 tx=0).stop 2024-04-25T11:27:32.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.590+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006c5c0 msgr2=0x7fdb400cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:32.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.590+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006c5c0 0x7fdb400cd050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000d7d0 tx=0x7fdb3000dc90 comp rx=0 tx=0).stop 2024-04-25T11:27:32.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 shutdown_connections 2024-04-25T11:27:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b4cd0 0x7fdb40153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40144370 0x7fdb40160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 --2- 172.21.15.19:0/159790710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4006c5c0 0x7fdb400cd050 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 >> 172.21.15.19:0/159790710 conn(0x7fdb400d1720 msgr2=0x7fdb4015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 shutdown_connections 2024-04-25T11:27:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:32.591+0000 7fdb3fe00640 1 -- 172.21.15.19:0/159790710 wait complete. 2024-04-25T11:27:32.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:27:32.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:27:32.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T11:27:32.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T11:27:32.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T11:27:32.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T11:27:32.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T11:27:32.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T11:27:32.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T11:27:32.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T11:27:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.085+0000 7f26ab200640 1 Processor -- start 2024-04-25T11:27:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.085+0000 7f26ab200640 1 -- start start 2024-04-25T11:27:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7fcb3d200640 1 Processor -- start 2024-04-25T11:27:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7f26ab200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26ac15c640 0x7f26ac0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7f26ab200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26ac1414a0 0x7f26ac1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7f26ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac141df0 0x7f26ac15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7f26ab200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f26ac142200 con 0x7f26ac1414a0 2024-04-25T11:27:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7f26ab200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f26ac15cb40 con 0x7f26ac15c640 2024-04-25T11:27:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7f26ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26ac15ce20 con 0x7f26ac141df0 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.090+0000 7f26a9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26ac1414a0 0x7f26ac1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.090+0000 7f26a9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26ac1414a0 0x7f26ac1418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47178/0 (socket says 172.21.15.19:47178) 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.090+0000 7f26a9400640 1 -- 172.21.15.19:0/970345675 learned_addr learned my addr 172.21.15.19:0/970345675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.091+0000 7f26a9400640 1 -- 172.21.15.19:0/970345675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac141df0 msgr2=0x7f26ac15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.091+0000 7f26a9e00640 1 --2- 172.21.15.19:0/970345675 >> v2:172.21.15.169:3300/0 conn(0x7f26ac15c640 0x7f26ac0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.091+0000 7f26a9400640 1 --2- 172.21.15.19:0/970345675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac141df0 0x7f26ac15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.091+0000 7f26a9400640 1 -- 172.21.15.19:0/970345675 >> v2:172.21.15.169:3300/0 conn(0x7f26ac15c640 msgr2=0x7f26ac0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.091+0000 7f26a9400640 1 --2- 172.21.15.19:0/970345675 >> v2:172.21.15.169:3300/0 conn(0x7f26ac15c640 0x7f26ac0cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.091+0000 7f26a9400640 1 -- 172.21.15.19:0/970345675 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac15d100 con 0x7f26ac1414a0 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.091+0000 7f26a9e00640 1 --2- 172.21.15.19:0/970345675 >> v2:172.21.15.169:3300/0 conn(0x7f26ac15c640 0x7f26ac0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.092+0000 7f26a9400640 1 --2- 172.21.15.19:0/970345675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1414a0 0x7f26ac1418b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f26ac071a90 tx=0x7f269c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3962a90e930807da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.089+0000 7fcb3d200640 1 -- start start 2024-04-25T11:27:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.092+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3806f130 0x7fcb381414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.092+0000 7fcb3d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb381419e0 0x7fcb38141df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.093+0000 7fcb3d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3815cb40 con 0x7fcb3815c640 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.093+0000 7fcb3d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3815ce20 con 0x7fcb381419e0 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.093+0000 7fcb3d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3815d100 con 0x7fcb3806f130 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.093+0000 7fcb35e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcb3815c640 0x7fcb380cda60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:51156/0 (socket says 172.21.15.19:51156) 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.093+0000 7fcb35e00640 1 -- 172.21.15.19:0/877243066 learned_addr learned my addr 172.21.15.19:0/877243066 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.094+0000 7fcb35400640 1 --2- 172.21.15.19:0/877243066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3806f130 0x7fcb381414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.094+0000 7f26a8a00640 1 -- 172.21.15.19:0/970345675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c037070 con 0x7f26ac1414a0 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.094+0000 7f26a8a00640 1 -- 172.21.15.19:0/970345675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269c032440 con 0x7f26ac1414a0 2024-04-25T11:27:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.094+0000 7f26a8a00640 1 -- 172.21.15.19:0/970345675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c032810 con 0x7f26ac1414a0 2024-04-25T11:27:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.094+0000 7fcb36800640 1 --2- 172.21.15.19:0/877243066 >> v2:172.21.15.169:3300/0 conn(0x7fcb381419e0 0x7fcb38141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.094+0000 7fcb34a00640 1 -- 172.21.15.19:0/877243066 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644233990 0 0) 0x7fcb3815cb40 con 0x7fcb3815c640 2024-04-25T11:27:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7fcb34a00640 1 -- 172.21.15.19:0/877243066 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb1c003520 con 0x7fcb3815c640 2024-04-25T11:27:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7f26ab200640 1 -- 172.21.15.19:0/970345675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1414a0 msgr2=0x7f26ac1418b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7f26ab200640 1 --2- 172.21.15.19:0/970345675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1414a0 0x7f26ac1418b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f26ac071a90 tx=0x7f269c02a1d0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7fcb35400640 1 -- 172.21.15.19:0/877243066 >> v2:172.21.15.169:3300/0 conn(0x7fcb381419e0 msgr2=0x7fcb38141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7fcb35400640 1 --2- 172.21.15.19:0/877243066 >> v2:172.21.15.169:3300/0 conn(0x7fcb381419e0 0x7fcb38141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7fcb35400640 1 -- 172.21.15.19:0/877243066 >> v1:172.21.15.169:6789/0 conn(0x7fcb3815c640 legacy=0x7fcb380cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7fcb35400640 1 -- 172.21.15.19:0/877243066 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3815d3e0 con 0x7fcb3806f130 2024-04-25T11:27:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.096+0000 7f26ab200640 1 -- 172.21.15.19:0/970345675 shutdown_connections 2024-04-25T11:27:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.096+0000 7f26ab200640 1 --2- 172.21.15.19:0/970345675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1414a0 0x7f26ac1418b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.096+0000 7f26ab200640 1 --2- 172.21.15.19:0/970345675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac141df0 0x7f26ac15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.096+0000 7f26ab200640 1 --2- 172.21.15.19:0/970345675 >> v2:172.21.15.169:3300/0 conn(0x7f26ac15c640 0x7f26ac0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.096+0000 7f26ab200640 1 -- 172.21.15.19:0/970345675 >> 172.21.15.19:0/970345675 conn(0x7f26ac073f00 msgr2=0x7f26ac06c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.095+0000 7f3ae8400640 1 Processor -- start 2024-04-25T11:27:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.096+0000 7f3ae8400640 1 -- start start 2024-04-25T11:27:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.097+0000 7f26ab200640 1 -- 172.21.15.19:0/970345675 shutdown_connections 2024-04-25T11:27:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.097+0000 7f26ab200640 1 -- 172.21.15.19:0/970345675 wait complete. 2024-04-25T11:27:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.098+0000 7fcb35400640 1 --2- 172.21.15.19:0/877243066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3806f130 0x7fcb381414a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097a0 tx=0x7fcb2402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8db44faebae7b0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.098+0000 7f3ae8400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ae0163ef0 0x7f3ae006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.099+0000 7f3ae8400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae0164300 con 0x7f3ae006db10 2024-04-25T11:27:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.099+0000 7f3ae8400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae006df20 con 0x7f3ae00ce060 2024-04-25T11:27:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.099+0000 7f3ae8400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae00700a0 con 0x7f3ae0163ef0 2024-04-25T11:27:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.099+0000 7fcb34a00640 1 -- 172.21.15.19:0/877243066 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24037070 con 0x7fcb3806f130 2024-04-25T11:27:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.100+0000 7f3ae5a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ae0163ef0 0x7f3ae006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.100+0000 7f3ae5a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ae0163ef0 0x7f3ae006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47192/0 (socket says 172.21.15.19:47192) 2024-04-25T11:27:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.100+0000 7f3ae5a00640 1 -- 172.21.15.19:0/2834444904 learned_addr learned my addr 172.21.15.19:0/2834444904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.099+0000 7fcb34a00640 1 -- 172.21.15.19:0/877243066 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24032440 con 0x7fcb3806f130 2024-04-25T11:27:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.099+0000 7fcb34a00640 1 -- 172.21.15.19:0/877243066 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240327f0 con 0x7fcb3806f130 2024-04-25T11:27:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.100+0000 7fcb3d200640 1 -- 172.21.15.19:0/877243066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3806f130 msgr2=0x7fcb381414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.100+0000 7fcb3d200640 1 --2- 172.21.15.19:0/877243066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3806f130 0x7fcb381414a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb240097a0 tx=0x7fcb2402a180 comp rx=0 tx=0).stop 2024-04-25T11:27:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7f3ae5a00640 1 -- 172.21.15.19:0/2834444904 >> v1:172.21.15.169:6789/0 conn(0x7f3ae00ce060 legacy=0x7f3ae016d540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:27:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7f4310a00640 1 Processor -- start 2024-04-25T11:27:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7f3ae5a00640 1 -- 172.21.15.19:0/2834444904 >> v1:172.21.15.19:6789/0 conn(0x7f3ae006db10 legacy=0x7f3ae00cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7f4310a00640 1 -- start start 2024-04-25T11:27:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7fcb3d200640 1 -- 172.21.15.19:0/877243066 shutdown_connections 2024-04-25T11:27:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7f26ab200640 1 Processor -- start 2024-04-25T11:27:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7fcb3d200640 1 --2- 172.21.15.19:0/877243066 >> v2:172.21.15.169:3300/0 conn(0x7fcb381419e0 0x7fcb38141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7fcb3d200640 1 --2- 172.21.15.19:0/877243066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3806f130 0x7fcb381414a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7fcb3d200640 1 -- 172.21.15.19:0/877243066 >> 172.21.15.19:0/877243066 conn(0x7fcb38073f00 msgr2=0x7fcb3806c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7f26ab200640 1 -- start start 2024-04-25T11:27:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.102+0000 7fcb3d200640 1 -- 172.21.15.19:0/877243066 shutdown_connections 2024-04-25T11:27:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.102+0000 7fb78ba00640 1 Processor -- start 2024-04-25T11:27:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.102+0000 7fcb3d200640 1 -- 172.21.15.19:0/877243066 wait complete. 2024-04-25T11:27:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.101+0000 7f3ae5a00640 1 -- 172.21.15.19:0/2834444904 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae016e300 con 0x7f3ae0163ef0 2024-04-25T11:27:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.102+0000 7fb78ba00640 1 -- start start 2024-04-25T11:27:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.103+0000 7f3ae5a00640 1 --2- 172.21.15.19:0/2834444904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae0163ef0 0x7f3ae006d5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009870 tx=0x7f3ad402b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd070407db1aa51c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f3adfe00640 1 -- 172.21.15.19:0/2834444904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4038070 con 0x7f3ae0163ef0 2024-04-25T11:27:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f3adfe00640 1 -- 172.21.15.19:0/2834444904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4033640 con 0x7f3ae0163ef0 2024-04-25T11:27:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f3adfe00640 1 -- 172.21.15.19:0/2834444904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4033a10 con 0x7f3ae0163ef0 2024-04-25T11:27:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f26ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac1414a0 0x7f26ac1e6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7fc1d0a00640 1 Processor -- start 2024-04-25T11:27:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f26ab200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26ac141df0 0x7f26ac1f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f26ab200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac15c640 0x7f26ac0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f26ab200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac0ce120 con 0x7f26ac15c640 2024-04-25T11:27:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f26ab200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac0cdfa0 con 0x7f26ac141df0 2024-04-25T11:27:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f26ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26ac06f130 con 0x7f26ac1414a0 2024-04-25T11:27:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7fc1d0a00640 1 -- start start 2024-04-25T11:27:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f3ae8400640 1 -- 172.21.15.19:0/2834444904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae0163ef0 msgr2=0x7f3ae006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f3ae8400640 1 --2- 172.21.15.19:0/2834444904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae0163ef0 0x7f3ae006d5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009870 tx=0x7f3ad402b3b0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f3ae8400640 1 -- 172.21.15.19:0/2834444904 shutdown_connections 2024-04-25T11:27:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f3ae8400640 1 --2- 172.21.15.19:0/2834444904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae0163ef0 0x7f3ae006d5d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f3ae8400640 1 -- 172.21.15.19:0/2834444904 >> 172.21.15.19:0/2834444904 conn(0x7f3ae0073050 msgr2=0x7f3ae0073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7fb78ba00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb77c0e1aa0 0x7fb77c0e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7fb78ba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb77c051c90 0x7fb77c0520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f26aa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac15c640 0x7f26ac0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f26a9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac1414a0 0x7f26ac1e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f26a9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac1414a0 0x7f26ac1e6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53058/0 (socket says 172.21.15.19:53058) 2024-04-25T11:27:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f26a9e00640 1 -- 172.21.15.19:0/1810068392 learned_addr learned my addr 172.21.15.19:0/1810068392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f26a9400640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26ac141df0 0x7f26ac1f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7f3ae8400640 1 -- 172.21.15.19:0/2834444904 shutdown_connections 2024-04-25T11:27:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7f3ae8400640 1 -- 172.21.15.19:0/2834444904 wait complete. 2024-04-25T11:27:33.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7fb78ba00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77c0514d0 con 0x7fb77c0389d0 2024-04-25T11:27:33.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7fb78ba00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb77c0e6420 con 0x7fb77c0e1aa0 2024-04-25T11:27:33.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7fb78ba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb77c051800 con 0x7fb77c051c90 2024-04-25T11:27:33.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7fb789000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb77c0e1aa0 0x7fb77c0e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7fb789000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb77c0e1aa0 0x7fb77c0e5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47212/0 (socket says 172.21.15.19:47212) 2024-04-25T11:27:33.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7fb789000640 1 -- 172.21.15.19:0/1388737519 learned_addr learned my addr 172.21.15.19:0/1388737519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f7a7ca00640 1 Processor -- start 2024-04-25T11:27:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f4310a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f430c06d8e0 0x7f430c0d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f4310a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f430c06dcf0 0x7f430c0d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f4310a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f430c0b3c10 con 0x7f430c0d04a0 2024-04-25T11:27:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f4310a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f430c0b9520 con 0x7f430c06dcf0 2024-04-25T11:27:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.104+0000 7f4310a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f430c0b4820 con 0x7f430c06d8e0 2024-04-25T11:27:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f430a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f430c0d04a0 0x7f430c0d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52618/0 (socket says 172.21.15.19:52618) 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f430a800640 1 -- 172.21.15.19:0/1119463764 learned_addr learned my addr 172.21.15.19:0/1119463764 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.105+0000 7f4309e00640 1 --2- 172.21.15.19:0/1119463764 >> v2:172.21.15.169:3300/0 conn(0x7f430c06d8e0 0x7f430c0d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7f4308a00640 1 -- 172.21.15.19:0/1119463764 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331995773 0 0) 0x7f430c0b3c10 con 0x7f430c0d04a0 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7f4308a00640 1 -- 172.21.15.19:0/1119463764 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42f0003520 con 0x7f430c0d04a0 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.106+0000 7f4309400640 1 --2- 172.21.15.19:0/1119463764 >> v2:172.21.15.19:3300/0 conn(0x7f430c06dcf0 0x7f430c0d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f26a9e00640 1 -- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26ac141df0 msgr2=0x7f26ac1f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f26a9e00640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26ac141df0 0x7f26ac1f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f26a9e00640 1 -- 172.21.15.19:0/1810068392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac15c640 msgr2=0x7f26ac0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f26a9e00640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac15c640 0x7f26ac0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f26a9e00640 1 -- 172.21.15.19:0/1810068392 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac14d150 con 0x7f26ac1414a0 2024-04-25T11:27:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7fb789a00640 1 --2- 172.21.15.19:0/1388737519 >> v2:172.21.15.169:3300/0 conn(0x7fb77c051c90 0x7fb77c0520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f4309400640 1 -- 172.21.15.19:0/1119463764 >> v2:172.21.15.169:3300/0 conn(0x7f430c06d8e0 msgr2=0x7f430c0d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f4309400640 1 --2- 172.21.15.19:0/1119463764 >> v2:172.21.15.169:3300/0 conn(0x7f430c06d8e0 0x7f430c0d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f4309400640 1 -- 172.21.15.19:0/1119463764 >> v1:172.21.15.19:6789/0 conn(0x7f430c0d04a0 legacy=0x7f430c0d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7fc1d0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b5260 0x7fc1cc0b9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7fc1d0a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1cc0b96b0 0x7fc1cc0b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7f26a9e00640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac1414a0 0x7f26ac1e6400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26a400cdb0 tx=0x7f26a400a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fc1d0a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1cc0d0b30 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fc1d0a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc1cc0d4420 con 0x7fc1cc0b96b0 2024-04-25T11:27:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fc1d0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc0d4700 con 0x7fc1cc0b5260 2024-04-25T11:27:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fc1caa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b5260 0x7fc1cc0b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fc1caa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b5260 0x7fc1cc0b9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53062/0 (socket says 172.21.15.19:53062) 2024-04-25T11:27:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fc1caa00640 1 -- 172.21.15.19:0/1039918308 learned_addr learned my addr 172.21.15.19:0/1039918308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7fb783400640 1 -- 172.21.15.19:0/1388737519 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2800077486 0 0) 0x7fb77c0514d0 con 0x7fb77c0389d0 2024-04-25T11:27:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7fb783400640 1 -- 172.21.15.19:0/1388737519 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb76c003520 con 0x7fb77c0389d0 2024-04-25T11:27:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a400f680 con 0x7f26ac1414a0 2024-04-25T11:27:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a400f820 con 0x7f26ac1414a0 2024-04-25T11:27:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a400fbf0 con 0x7f26ac1414a0 2024-04-25T11:27:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fb789000640 1 -- 172.21.15.19:0/1388737519 >> v2:172.21.15.169:3300/0 conn(0x7fb77c051c90 msgr2=0x7fb77c0520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fc1cbe00640 1 --2- 172.21.15.19:0/1039918308 >> v2:172.21.15.19:3300/0 conn(0x7fc1cc0b96b0 0x7fc1cc0b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f7a7ca00640 1 -- start start 2024-04-25T11:27:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fb789000640 1 --2- 172.21.15.19:0/1388737519 >> v2:172.21.15.169:3300/0 conn(0x7fb77c051c90 0x7fb77c0520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fb789000640 1 -- 172.21.15.19:0/1388737519 >> v1:172.21.15.19:6789/0 conn(0x7fb77c0389d0 legacy=0x7fb77c038de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.108+0000 7fb789000640 1 -- 172.21.15.19:0/1388737519 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c035f50 con 0x7fb77c0e1aa0 2024-04-25T11:27:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.109+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac1f2250 con 0x7f26ac1414a0 2024-04-25T11:27:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.109+0000 7fb789000640 1 --2- 172.21.15.19:0/1388737519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c0e1aa0 0x7fb77c0e5ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb77c03cfc0 tx=0x7fb77802a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a009dcc5505fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.109+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac13efe0 con 0x7f26ac1414a0 2024-04-25T11:27:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.109+0000 7fc1caa00640 1 -- 172.21.15.19:0/1039918308 >> v2:172.21.15.19:3300/0 conn(0x7fc1cc0b96b0 msgr2=0x7fc1cc0b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7fc1caa00640 1 --2- 172.21.15.19:0/1039918308 >> v2:172.21.15.19:3300/0 conn(0x7fc1cc0b96b0 0x7fc1cc0b7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7f3ae8400640 1 Processor -- start 2024-04-25T11:27:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.107+0000 7f4309400640 1 -- 172.21.15.19:0/1119463764 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c0b3ef0 con 0x7f430c06dcf0 2024-04-25T11:27:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7fc1caa00640 1 -- 172.21.15.19:0/1039918308 >> v1:172.21.15.19:6789/0 conn(0x7fc1cc0b4da0 legacy=0x7fc1cc1600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7fc1caa00640 1 -- 172.21.15.19:0/1039918308 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc0d49e0 con 0x7fc1cc0b5260 2024-04-25T11:27:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.111+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26ac050740 con 0x7f26ac1414a0 2024-04-25T11:27:33.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7fcb3d200640 1 Processor -- start 2024-04-25T11:27:33.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7f3ae8400640 1 -- start start 2024-04-25T11:27:33.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.111+0000 7fcb3d200640 1 -- start start 2024-04-25T11:27:33.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.111+0000 7fb783400640 1 -- 172.21.15.19:0/1388737519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778037070 con 0x7fb77c0e1aa0 2024-04-25T11:27:33.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.111+0000 7fb783400640 1 -- 172.21.15.19:0/1388737519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb778032440 con 0x7fb77c0e1aa0 2024-04-25T11:27:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.111+0000 7fb783400640 1 -- 172.21.15.19:0/1388737519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778032810 con 0x7fb77c0e1aa0 2024-04-25T11:27:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7f7a7ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7a70047db0 0x7f7a700e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.110+0000 7f7a7ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70048300 0x7f7a70033950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fc1caa00640 1 --2- 172.21.15.19:0/1039918308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b5260 0x7fc1cc0b9170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009870 tx=0x7fc1b802a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b5967b9a81bf87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7f7a75e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7a70047db0 0x7f7a700e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fb78ba00640 1 -- 172.21.15.19:0/1388737519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c0e1aa0 msgr2=0x7fb77c0e5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fb78ba00640 1 --2- 172.21.15.19:0/1388737519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c0e1aa0 0x7fb77c0e5ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb77c03cfc0 tx=0x7fb77802a280 comp rx=0 tx=0).stop 2024-04-25T11:27:33.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7f7a75e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7a70047db0 0x7f7a700e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:55280/0 (socket says 172.21.15.19:55280) 2024-04-25T11:27:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7f4309400640 1 --2- 172.21.15.19:0/1119463764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c06dcf0 0x7f430c0d3f60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f430c0b38b0 tx=0x7f42fc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ecf98af034f98b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fc1ca000640 1 -- 172.21.15.19:0/1039918308 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8037070 con 0x7fc1cc0b5260 2024-04-25T11:27:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fc1ca000640 1 -- 172.21.15.19:0/1039918308 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8032440 con 0x7fc1cc0b5260 2024-04-25T11:27:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fc1ca000640 1 -- 172.21.15.19:0/1039918308 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8032810 con 0x7fc1cc0b5260 2024-04-25T11:27:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fb78ba00640 1 -- 172.21.15.19:0/1388737519 shutdown_connections 2024-04-25T11:27:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fb78ba00640 1 --2- 172.21.15.19:0/1388737519 >> v2:172.21.15.169:3300/0 conn(0x7fb77c051c90 0x7fb77c0520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fb78ba00640 1 --2- 172.21.15.19:0/1388737519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c0e1aa0 0x7fb77c0e5ee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.112+0000 7fb78ba00640 1 -- 172.21.15.19:0/1388737519 >> 172.21.15.19:0/1388737519 conn(0x7fb77c056f10 msgr2=0x7fb77c057320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fb78ba00640 1 -- 172.21.15.19:0/1388737519 shutdown_connections 2024-04-25T11:27:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fb78ba00640 1 -- 172.21.15.19:0/1388737519 wait complete. 2024-04-25T11:27:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7f7a75e00640 1 -- 172.21.15.19:0/3719938502 learned_addr learned my addr 172.21.15.19:0/3719938502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb3806f130 0x7fcb380d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb381419e0 0x7fcb381fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3815c640 0x7fcb3815f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fcb3d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3806fcd0 con 0x7fcb3806f130 2024-04-25T11:27:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fcb3d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3806fb50 con 0x7fcb381419e0 2024-04-25T11:27:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7fcb3d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3806fe50 con 0x7fcb3815c640 2024-04-25T11:27:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7f3ae8400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae006db10 0x7f3ae015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/3719938502 >> v2:172.21.15.19:3300/0 conn(0x7f7a70033e90 0x7f7a700531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7004f690 con 0x7f7a70033e90 2024-04-25T11:27:33.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7a70006130 con 0x7f7a70047db0 2024-04-25T11:27:33.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.113+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7a700342a0 con 0x7f7a70048300 2024-04-25T11:27:33.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f3ae8400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00ce060 0x7f3ae0160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f4308a00640 1 -- 172.21.15.19:0/1119463764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc037070 con 0x7f430c06dcf0 2024-04-25T11:27:33.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fcb36800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3815c640 0x7fcb3815f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f3ae8400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae0163ef0 0x7f3ae0160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fcb36800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3815c640 0x7fcb3815f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53084/0 (socket says 172.21.15.19:53084) 2024-04-25T11:27:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fcb36800640 1 -- 172.21.15.19:0/3710510078 learned_addr learned my addr 172.21.15.19:0/3710510078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f26a40223f0 con 0x7f26ac1414a0 2024-04-25T11:27:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f3ae5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae006db10 0x7f3ae015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f3ae5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae006db10 0x7f3ae015c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47228/0 (socket says 172.21.15.19:47228) 2024-04-25T11:27:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f3ae5a00640 1 -- 172.21.15.19:0/3525637231 learned_addr learned my addr 172.21.15.19:0/3525637231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f4308a00640 1 -- 172.21.15.19:0/1119463764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42fc032440 con 0x7f430c06dcf0 2024-04-25T11:27:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f7a75400640 1 --2- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70048300 0x7f7a70033950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7f7a76800640 1 --2- 172.21.15.19:0/3719938502 >> v2:172.21.15.19:3300/0 conn(0x7f7a70033e90 0x7f7a700531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7f4308a00640 1 -- 172.21.15.19:0/1119463764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc037070 con 0x7f430c06dcf0 2024-04-25T11:27:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7fcb36800640 1 -- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb381419e0 msgr2=0x7fcb381fa1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:27:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7f3ae5000640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00ce060 0x7f3ae0160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7f3ae6400640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae0163ef0 0x7f3ae0160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7fcb36800640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb381419e0 0x7fcb381fa1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7fcb36800640 1 -- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb3806f130 msgr2=0x7fcb380d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7fcb35e00640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb3806f130 0x7fcb380d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7fcb36800640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb3806f130 0x7fcb380d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7fcb36800640 1 -- 172.21.15.19:0/3710510078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb381664f0 con 0x7fcb3815c640 2024-04-25T11:27:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00b4fd0 con 0x7f3ae006db10 2024-04-25T11:27:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae00b4cf0 con 0x7f3ae0163ef0 2024-04-25T11:27:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae00b52b0 con 0x7f3ae00ce060 2024-04-25T11:27:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.115+0000 7fcb35e00640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb3806f130 0x7fcb380d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 -- 172.21.15.19:0/1039918308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b5260 msgr2=0x7fc1cc0b9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 --2- 172.21.15.19:0/1039918308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b5260 0x7fc1cc0b9170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009870 tx=0x7fc1b802a1d0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 -- 172.21.15.19:0/1039918308 shutdown_connections 2024-04-25T11:27:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 --2- 172.21.15.19:0/1039918308 >> v2:172.21.15.19:3300/0 conn(0x7fc1cc0b96b0 0x7fc1cc0b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 --2- 172.21.15.19:0/1039918308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b5260 0x7fc1cc0b9170 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 -- 172.21.15.19:0/1039918308 >> 172.21.15.19:0/1039918308 conn(0x7fc1cc0d2bd0 msgr2=0x7fc1cc0d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 -- 172.21.15.19:0/1039918308 shutdown_connections 2024-04-25T11:27:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.114+0000 7fc1d0a00640 1 -- 172.21.15.19:0/1039918308 wait complete. 2024-04-25T11:27:33.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f7a75e00640 1 -- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70048300 msgr2=0x7f7a70033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f7a75e00640 1 --2- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70048300 0x7f7a70033950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f7a75e00640 1 -- 172.21.15.19:0/3719938502 >> v2:172.21.15.19:3300/0 conn(0x7f7a70033e90 msgr2=0x7f7a700531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f26a2000640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f268c0744b0 0x7f268c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f26a4096160 con 0x7f26ac1414a0 2024-04-25T11:27:33.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f26a2000640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7f268c07c700 0x7f268c07eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f268c07f220 con 0x7f268c07c700 2024-04-25T11:27:33.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f26a4096510 con 0x7f26ac1414a0 2024-04-25T11:27:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f7a75e00640 1 --2- 172.21.15.19:0/3719938502 >> v2:172.21.15.19:3300/0 conn(0x7f7a70033e90 0x7f7a700531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f7a75e00640 1 -- 172.21.15.19:0/3719938502 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7004ffa0 con 0x7f7a70047db0 2024-04-25T11:27:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f7a76800640 1 --2- 172.21.15.19:0/3719938502 >> v2:172.21.15.19:3300/0 conn(0x7f7a70033e90 0x7f7a700531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7fcb36800640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3815c640 0x7fcb3815f490 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800e850 tx=0x7fcb2800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f3ae6400640 1 -- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00ce060 msgr2=0x7f3ae0160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f3ae6400640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae00ce060 0x7f3ae0160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f3ae6400640 1 -- 172.21.15.19:0/3525637231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae006db10 msgr2=0x7f3ae015c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f3ae6400640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae006db10 0x7f3ae015c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7f3ae6400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae01614b0 con 0x7f3ae0163ef0 2024-04-25T11:27:33.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7f4310a00640 1 -- 172.21.15.19:0/1119463764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c06dcf0 msgr2=0x7f430c0d3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7f26a9400640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f268c0744b0 0x7f268c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7f4310a00640 1 --2- 172.21.15.19:0/1119463764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c06dcf0 0x7f430c0d3f60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f430c0b38b0 tx=0x7f42fc02a130 comp rx=0 tx=0).stop 2024-04-25T11:27:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.116+0000 7f703a400640 1 Processor -- start 2024-04-25T11:27:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7f703a400640 1 -- start start 2024-04-25T11:27:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7fc1d0a00640 1 Processor -- start 2024-04-25T11:27:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28017070 con 0x7fcb3815c640 2024-04-25T11:27:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7f3ae6400640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae0163ef0 0x7f3ae0160ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800bdd0 tx=0x7f3ad8009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2800ee40 con 0x7fcb3815c640 2024-04-25T11:27:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.117+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb381669f0 con 0x7fcb3815c640 2024-04-25T11:27:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb380cf880 con 0x7fcb3815c640 2024-04-25T11:27:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f4310a00640 1 -- 172.21.15.19:0/1119463764 shutdown_connections 2024-04-25T11:27:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f4310a00640 1 --2- 172.21.15.19:0/1119463764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c06dcf0 0x7f430c0d3f60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f4310a00640 1 --2- 172.21.15.19:0/1119463764 >> v2:172.21.15.169:3300/0 conn(0x7f430c06d8e0 0x7f430c0d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f4310a00640 1 -- 172.21.15.19:0/1119463764 >> 172.21.15.19:0/1119463764 conn(0x7f430c08d110 msgr2=0x7f430c08d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f7a75e00640 1 --2- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 0x7f7a700e7d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000d9b0 tx=0x7f7a6802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1346b890c30f38f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800cb20 con 0x7fcb3815c640 2024-04-25T11:27:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8019070 con 0x7f3ae0163ef0 2024-04-25T11:27:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae0167a70 con 0x7f3ae0163ef0 2024-04-25T11:27:33.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7fb78ba00640 1 Processor -- start 2024-04-25T11:27:33.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae0162110 con 0x7f3ae0163ef0 2024-04-25T11:27:33.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f7a74a00640 1 -- 172.21.15.19:0/3719938502 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68037070 con 0x7f7a70047db0 2024-04-25T11:27:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7f26aa800640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7f268c07c700 0x7f268c07eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f26a9400640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f268c0744b0 0x7f268c0769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f269c034000 tx=0x7f269c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f7a74a00640 1 -- 172.21.15.19:0/3719938502 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a68032440 con 0x7f7a70047db0 2024-04-25T11:27:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f7a74a00640 1 -- 172.21.15.19:0/3719938502 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68032810 con 0x7f7a70047db0 2024-04-25T11:27:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f4310a00640 1 -- 172.21.15.19:0/1119463764 shutdown_connections 2024-04-25T11:27:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.118+0000 7fc1d0a00640 1 -- start start 2024-04-25T11:27:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad800cdf0 con 0x7f3ae0163ef0 2024-04-25T11:27:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8014660 con 0x7f3ae0163ef0 2024-04-25T11:27:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f4310a00640 1 -- 172.21.15.19:0/1119463764 wait complete. 2024-04-25T11:27:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.119+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ae0050740 con 0x7f3ae0163ef0 2024-04-25T11:27:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb38050740 con 0x7fcb3815c640 2024-04-25T11:27:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f703a400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f703414ad60 0x7f7034070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f703a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f70340b5e40 0x7f70340b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f703a400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70340b6790 con 0x7f70341681d0 2024-04-25T11:27:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f703a400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f70340b4af0 con 0x7f703414ad60 2024-04-25T11:27:33.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f703a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f70340b4dd0 con 0x7f70340b5e40 2024-04-25T11:27:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 msgr2=0x7f7a700e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 0x7f7a700e7d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000d9b0 tx=0x7f7a6802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f7033400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f703414ad60 0x7f7034070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f7033400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f703414ad60 0x7f7034070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47248/0 (socket says 172.21.15.19:47248) 2024-04-25T11:27:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7f7033400640 1 -- 172.21.15.19:0/1460603251 learned_addr learned my addr 172.21.15.19:0/1460603251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 shutdown_connections 2024-04-25T11:27:33.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/3719938502 >> v2:172.21.15.19:3300/0 conn(0x7f7a70033e90 0x7f7a700531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70048300 0x7f7a70033950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/3719938502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 0x7f7a700e7d50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 >> 172.21.15.19:0/3719938502 conn(0x7f7a70054cf0 msgr2=0x7f7a70055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7fc1d0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0b4da0 0x7fc1cc15e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7fc1d0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1cc0b5260 0x7fc1cc162af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7032a00640 1 --2- 172.21.15.19:0/1460603251 >> v2:172.21.15.169:3300/0 conn(0x7f70340b5e40 0x7f70340b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7fc1d0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b96b0 0x7fc1cc160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7fc1d0a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0d08d0 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7fc1d0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0d0750 con 0x7fc1cc0b5260 2024-04-25T11:27:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7fc1d0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc0b8100 con 0x7fc1cc0b96b0 2024-04-25T11:27:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7033400640 1 -- 172.21.15.19:0/1460603251 >> v2:172.21.15.169:3300/0 conn(0x7f70340b5e40 msgr2=0x7f70340b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0b4da0 0x7fc1cc15e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fc1cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0b4da0 0x7fc1cc15e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47258/0 (socket says 172.21.15.19:47258) 2024-04-25T11:27:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fc1cb400640 1 -- 172.21.15.19:0/751586239 learned_addr learned my addr 172.21.15.19:0/751586239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7033400640 1 --2- 172.21.15.19:0/1460603251 >> v2:172.21.15.169:3300/0 conn(0x7f70340b5e40 0x7f70340b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7033400640 1 -- 172.21.15.19:0/1460603251 >> v1:172.21.15.169:6789/0 conn(0x7f70341681d0 legacy=0x7f70341685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.121+0000 7f7033400640 1 -- 172.21.15.19:0/1460603251 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70340b50b0 con 0x7f703414ad60 2024-04-25T11:27:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb2801a070 con 0x7fcb3815c640 2024-04-25T11:27:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.120+0000 7fb78ba00640 1 -- start start 2024-04-25T11:27:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fb78ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb77c0389d0 0x7fb77c0d9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fb78ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb77c051c90 0x7fb77c04bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fb78ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c04c170 0x7fb77c04c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fb78ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c056cb0 con 0x7fb77c04c170 2024-04-25T11:27:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fb78ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c036510 con 0x7fb77c051c90 2024-04-25T11:27:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fb78ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c055f30 con 0x7fb77c0389d0 2024-04-25T11:27:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.123+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 shutdown_connections 2024-04-25T11:27:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.123+0000 7f26aa800640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7f268c07c700 0x7f268c07eb50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fc1cb400640 1 -- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b96b0 msgr2=0x7fc1cc160ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:27:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fc1cb400640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1cc0b96b0 0x7fc1cc160ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fc1cb400640 1 -- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1cc0b5260 msgr2=0x7fc1cc162af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:27:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fc1cb400640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1cc0b5260 0x7fc1cc162af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.122+0000 7fc1cb400640 1 -- 172.21.15.19:0/751586239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc0cf990 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.123+0000 7f7a7ca00640 1 -- 172.21.15.19:0/3719938502 wait complete. 2024-04-25T11:27:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7f4310a00640 1 Processor -- start 2024-04-25T11:27:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fb789a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c04c170 0x7fb77c04c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fc1cb400640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0b4da0 0x7fc1cc15e870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c760 tx=0x7fc1c400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7f4310a00640 1 -- start start 2024-04-25T11:27:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fcb2e000640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb180744e0 0x7fcb180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcb280954f0 con 0x7fcb3815c640 2024-04-25T11:27:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fcb2e000640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fcb1807c730 0x7fcb1807eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb1807f250 con 0x7fcb1807c730 2024-04-25T11:27:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fcb2809c050 con 0x7fcb3815c640 2024-04-25T11:27:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.124+0000 7fb789a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c04c170 0x7fb77c04c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47260/0 (socket says 172.21.15.19:47260) 2024-04-25T11:27:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.125+0000 7fb789a00640 1 -- 172.21.15.19:0/2002040723 learned_addr learned my addr 172.21.15.19:0/2002040723 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.125+0000 7fcb35e00640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb180744e0 0x7fcb180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.125+0000 7fcb35400640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fcb1807c730 0x7fcb1807eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.125+0000 7fcb35400640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fcb1807c730 0x7fcb1807eb80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.125+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ad801c070 con 0x7f3ae0163ef0 2024-04-25T11:27:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.125+0000 7fb783e00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb77c051c90 0x7fb77c04bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4003b70 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4003d30 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c400aa90 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7fc1d0a00640 1 -- 172.21.15.19:0/751586239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc1634d0 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7fc1d0a00640 1 -- 172.21.15.19:0/751586239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc0ce110 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7f3add600640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3ad80957a0 con 0x7f3ae0163ef0 2024-04-25T11:27:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f3add600640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f3ac007c700 0x7f3ac007eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ad8022ec0 con 0x7f3ac007c700 2024-04-25T11:27:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3ad805bfd0 con 0x7f3ae0163ef0 2024-04-25T11:27:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7f4310a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f430c06d8e0 0x7f430c0a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7f4310a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f430c06dcf0 0x7f430c160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f3ae5a00640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7fc1d0a00640 1 -- 172.21.15.19:0/751586239 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1cc050740 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.126+0000 7fb789000640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb77c0389d0 0x7fb77c0d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7fb789a00640 1 -- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb77c0389d0 msgr2=0x7fb77c0d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb1807f250 con 0x7fcb1807c730 2024-04-25T11:27:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7fb789a00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb77c0389d0 0x7fb77c0d9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f3ae5000640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f3ac007c700 0x7f3ac007eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f268c07f220 con 0x7f268c07c700 2024-04-25T11:27:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f7033400640 1 --2- 172.21.15.19:0/1460603251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703414ad60 0x7f7034070010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7034070a70 tx=0x7f702402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aff218e5bed02a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7fb789a00640 1 -- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb77c051c90 msgr2=0x7fb77c04bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7fb789a00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb77c051c90 0x7fb77c04bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f4310a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c0d04a0 0x7f430c1f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f4310a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f430c0b38e0 con 0x7f430c0d04a0 2024-04-25T11:27:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f4310a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f430c0b44b0 con 0x7f430c06d8e0 2024-04-25T11:27:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f4310a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f430c0d0fc0 con 0x7f430c06dcf0 2024-04-25T11:27:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.127+0000 7f4309400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f430c06dcf0 0x7f430c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7f4309400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f430c06dcf0 0x7f430c160420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53106/0 (socket says 172.21.15.19:53106) 2024-04-25T11:27:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7f4309400640 1 -- 172.21.15.19:0/702826989 learned_addr learned my addr 172.21.15.19:0/702826989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7fb789a00640 1 -- 172.21.15.19:0/2002040723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c04ebb0 con 0x7fb77c04c170 2024-04-25T11:27:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7f7a7ca00640 1 Processor -- start 2024-04-25T11:27:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7fb783e00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb77c051c90 0x7fb77c04bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7f7a7ca00640 1 -- start start 2024-04-25T11:27:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7fb789000640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb77c0389d0 0x7fb77c0d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7f4309e00640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f430c06d8e0 0x7f430c0a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7fb789a00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c04c170 0x7fb77c04c580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7840519b0 tx=0x7fb78406eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.128+0000 7f4309400640 1 -- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f430c06d8e0 msgr2=0x7f430c0a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.129+0000 7f4309400640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f430c06d8e0 0x7f430c0a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.129+0000 7f4309400640 1 -- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c0d04a0 msgr2=0x7f430c1f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.129+0000 7f3ae5a00640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40090b0 tx=0x7f3ad402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.129+0000 7fcb35e00640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb180744e0 0x7fcb180769f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005c00 tx=0x7fcb30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.129+0000 7f430a800640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c0d04a0 0x7f430c1f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a7ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70033e90 0x7f7a70048710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a7ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 0x7f7a700d9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a7ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a70048300 0x7f7a7004a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a7ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70050840 con 0x7f7a70048300 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a7ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70050560 con 0x7f7a70047db0 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a7ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7a700509c0 con 0x7f7a70033e90 2024-04-25T11:27:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f3ae5000640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f3ac007c700 0x7f3ac007eb50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f4309400640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c0d04a0 0x7f430c1f9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a70048300 0x7f7a7004a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a70048300 0x7f7a7004a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47286/0 (socket says 172.21.15.19:47286) 2024-04-25T11:27:33.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 -- 172.21.15.19:0/605924097 learned_addr learned my addr 172.21.15.19:0/605924097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f4309400640 1 -- 172.21.15.19:0/702826989 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430c09f770 con 0x7f430c06dcf0 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a75e00640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70033e90 0x7f7a70048710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a75400640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 0x7f7a700d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 -- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70033e90 msgr2=0x7f7a70048710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70033e90 0x7f7a70048710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 -- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 msgr2=0x7f7a700d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 0x7f7a700d9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f7a76800640 1 -- 172.21.15.19:0/605924097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a700d9560 con 0x7f7a70048300 2024-04-25T11:27:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.130+0000 7f430a800640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f430c0d04a0 0x7f430c1f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.131+0000 7f7a75e00640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a70033e90 0x7f7a70048710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.131+0000 7f7a75400640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a70047db0 0x7f7a700d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.131+0000 7f7a76800640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a70048300 0x7f7a7004a120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a78069ed0 tx=0x7f7a7806eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.131+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== osd.2 v2:172.21.15.19:6813/2392070031 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ad8022ec0 con 0x7f3ac007c700 2024-04-25T11:27:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7809a070 con 0x7f7a70048300 2024-04-25T11:27:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a78076350 con 0x7f7a70048300 2024-04-25T11:27:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7806ee60 con 0x7f7a70048300 2024-04-25T11:27:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc1c400ac30 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a70049080 con 0x7f7a70048300 2024-04-25T11:27:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a70049640 con 0x7f7a70048300 2024-04-25T11:27:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78406ee50 con 0x7fb77c04c170 2024-04-25T11:27:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7840763d0 con 0x7fb77c04c170 2024-04-25T11:27:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c04ee90 con 0x7fb77c04c170 2024-04-25T11:27:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c04f450 con 0x7fb77c04c170 2024-04-25T11:27:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7f4309400640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f430c06dcf0 0x7f430c160420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009890 tx=0x7f42fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.133+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc037040 con 0x7f430c06dcf0 2024-04-25T11:27:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.133+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a70049d00 con 0x7f7a70048300 2024-04-25T11:27:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.133+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42fc042070 con 0x7f430c06dcf0 2024-04-25T11:27:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.132+0000 7f7032000640 1 -- 172.21.15.19:0/1460603251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024037070 con 0x7f703414ad60 2024-04-25T11:27:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.133+0000 7f7032000640 1 -- 172.21.15.19:0/1460603251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7024032440 con 0x7f703414ad60 2024-04-25T11:27:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.133+0000 7f7032000640 1 -- 172.21.15.19:0/1460603251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024032810 con 0x7f703414ad60 2024-04-25T11:27:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430c09fa50 con 0x7f430c06dcf0 2024-04-25T11:27:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7f703a400640 1 -- 172.21.15.19:0/1460603251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703414ad60 msgr2=0x7f7034070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7f703a400640 1 --2- 172.21.15.19:0/1460603251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703414ad60 0x7f7034070010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7034070a70 tx=0x7f702402a1f0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7fc1c3400640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1ac074500 0x7fc1ac076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc032830 con 0x7f430c06dcf0 2024-04-25T11:27:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc1c4095b30 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784096440 con 0x7fb77c04c170 2024-04-25T11:27:33.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7fc1c3400640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7fc1ac07c7d0 0x7fc1ac07ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1ac07f2f0 con 0x7fc1ac07c7d0 2024-04-25T11:27:33.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc1c4095ee0 con 0x7fc1cc0b4da0 2024-04-25T11:27:33.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.134+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f430c0a0010 con 0x7f430c06dcf0 2024-04-25T11:27:33.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.135+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a78095410 con 0x7f7a70048300 2024-04-25T11:27:33.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f430c050740 con 0x7f430c06dcf0 2024-04-25T11:27:33.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.135+0000 7fc1caa00640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1ac074500 0x7fc1ac076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7f703a400640 1 -- 172.21.15.19:0/1460603251 shutdown_connections 2024-04-25T11:27:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7f703a400640 1 --2- 172.21.15.19:0/1460603251 >> v2:172.21.15.169:3300/0 conn(0x7f70340b5e40 0x7f70340b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7f703a400640 1 --2- 172.21.15.19:0/1460603251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703414ad60 0x7f7034070010 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7f703a400640 1 -- 172.21.15.19:0/1460603251 >> 172.21.15.19:0/1460603251 conn(0x7f70340d1ae0 msgr2=0x7f70340d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7fc1cbe00640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7fc1ac07c7d0 0x7fc1ac07ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7f703a400640 1 -- 172.21.15.19:0/1460603251 shutdown_connections 2024-04-25T11:27:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.136+0000 7f703a400640 1 -- 172.21.15.19:0/1460603251 wait complete. 2024-04-25T11:27:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.137+0000 7f7a6e000640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a580744e0 0x7f7a580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.137+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7a78111d40 con 0x7f7a70048300 2024-04-25T11:27:33.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.137+0000 7f7a6e000640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f7a5807c7b0 0x7f7a5807ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.137+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a78071e90 con 0x7f7a5807c7b0 2024-04-25T11:27:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.137+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7a78112130 con 0x7f7a70048300 2024-04-25T11:27:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.137+0000 7fc1cbe00640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7fc1ac07c7d0 0x7fc1ac07ec20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.137+0000 7fc1caa00640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1ac074500 0x7fc1ac076a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.138+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb760001030 con 0x7fb77c04c170 2024-04-25T11:27:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.138+0000 7f7a75e00640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a580744e0 0x7f7a580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.138+0000 7f7a75400640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f7a5807c7b0 0x7f7a5807ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.139+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb78409f3f0 con 0x7fb77c04c170 2024-04-25T11:27:33.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.139+0000 7f7a75400640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f7a5807c7b0 0x7f7a5807ec00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.139+0000 7f7a75e00640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a580744e0 0x7f7a580769f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a6802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.140+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== osd.4 v2:172.21.15.169:6816/3656468664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1ac07f2f0 con 0x7fc1ac07c7d0 2024-04-25T11:27:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.140+0000 7f703a400640 1 Processor -- start 2024-04-25T11:27:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a78071e90 con 0x7f7a5807c7b0 2024-04-25T11:27:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.141+0000 7fb780c00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7640746a0 0x7fb764076bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.141+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb784111ad0 con 0x7fb77c04c170 2024-04-25T11:27:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7fb780c00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb76407c970 0x7fb76407edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb784071e50 con 0x7fb76407c970 2024-04-25T11:27:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb7841162a0 con 0x7fb77c04c170 2024-04-25T11:27:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.140+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f42fc047460 con 0x7f430c06dcf0 2024-04-25T11:27:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7f4302000640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42e80745b0 0x7f42e8076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f42fc0b6600 con 0x7f430c06dcf0 2024-04-25T11:27:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7f4302000640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f42e807c820 0x7f42e807ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42fc04fe50 con 0x7f42e807c820 2024-04-25T11:27:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7fb789000640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7640746a0 0x7fb764076bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.142+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f42fc0b69f0 con 0x7f430c06dcf0 2024-04-25T11:27:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7fb783e00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb76407c970 0x7fb76407edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7f430a800640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f42e807c820 0x7f42e807ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.141+0000 7f703a400640 1 -- start start 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7f703a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70340b5e40 0x7f7034145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7f703a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703414ad60 0x7f7034149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7f703a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70341681d0 0x7f7034162800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7f703a400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7034070550 con 0x7f70340b5e40 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7f703a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70340b5670 con 0x7f703414ad60 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.143+0000 7f703a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70340706d0 con 0x7f70341681d0 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.144+0000 7f7032a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703414ad60 0x7f7034149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.144+0000 7fb789000640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7640746a0 0x7fb764076bb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb778034000 tx=0x7fb77802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.144+0000 7f7033e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70341681d0 0x7f7034162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.144+0000 7f7033e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70341681d0 0x7f7034162800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53130/0 (socket says 172.21.15.19:53130) 2024-04-25T11:27:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.145+0000 7f4309e00640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42e80745b0 0x7f42e8076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.145+0000 7f7033e00640 1 -- 172.21.15.19:0/3205105346 learned_addr learned my addr 172.21.15.19:0/3205105346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.145+0000 7f430a800640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f42e807c820 0x7f42e807ec70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.145+0000 7f7033400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70340b5e40 0x7f7034145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.146+0000 7f7033e00640 1 -- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703414ad60 msgr2=0x7f7034149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.146+0000 7f7033e00640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703414ad60 0x7f7034149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.146+0000 7f7033e00640 1 -- 172.21.15.19:0/3205105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70340b5e40 msgr2=0x7f7034145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.146+0000 7f4309e00640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42e80745b0 0x7f42e8076ac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f430c0744c0 tx=0x7f42f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.146+0000 7f7033e00640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70340b5e40 0x7f7034145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.146+0000 7f7033e00640 1 -- 172.21.15.19:0/3205105346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703415e100 con 0x7f70341681d0 2024-04-25T11:27:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.146+0000 7f7033400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70340b5e40 0x7f7034145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.147+0000 7fb783e00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb76407c970 0x7fb76407edc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.147+0000 7f7033e00640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70341681d0 0x7f7034162800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f702c00c760 tx=0x7f702c00cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.147+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c003b90 con 0x7f70341681d0 2024-04-25T11:27:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.148+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703414a040 con 0x7f70341681d0 2024-04-25T11:27:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.148+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7034165ac0 con 0x7f70341681d0 2024-04-25T11:27:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.148+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c003d30 con 0x7f70341681d0 2024-04-25T11:27:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.148+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c00aa90 con 0x7f70341681d0 2024-04-25T11:27:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.149+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f700c001030 con 0x7f70341681d0 2024-04-25T11:27:33.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.148+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42fc04fe50 con 0x7f42e807c820 2024-04-25T11:27:33.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.150+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb784071e50 con 0x7fb76407c970 2024-04-25T11:27:33.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.152+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f702c00ac30 con 0x7f70341681d0 2024-04-25T11:27:33.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.154+0000 7f702b400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744b0 0x7f70140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.154+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f702c095440 con 0x7f70341681d0 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.154+0000 7f702b400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f701407c720 0x7f701407eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.154+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f701407f240 con 0x7f701407c720 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.154+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f702c095830 con 0x7f70341681d0 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.155+0000 7f7033400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744b0 0x7f70140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.155+0000 7f7032a00640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f701407c720 0x7f701407eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.156+0000 7f7033400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744b0 0x7f70140769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7024034000 tx=0x7f702402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.157+0000 7f7032a00640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f701407c720 0x7f701407eb70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.159+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb3806e100 con 0x7fcb1807c730 2024-04-25T11:27:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.159+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f701407f240 con 0x7f701407c720 2024-04-25T11:27:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.159+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f26ac06d000 con 0x7f268c07c700 2024-04-25T11:27:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.160+0000 7fcb2e000640 1 -- 172.21.15.19:0/3710510078 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcb3806e100 con 0x7fcb1807c730 2024-04-25T11:27:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.161+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ae0167210 con 0x7f3ac007c700 2024-04-25T11:27:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.161+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fcb1807c730 msgr2=0x7fcb1807eb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.161+0000 7fcb3d200640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fcb1807c730 0x7fcb1807eb80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.162+0000 7f26a2000640 1 -- 172.21.15.19:0/1810068392 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f26ac06d000 con 0x7f268c07c700 2024-04-25T11:27:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.162+0000 7f3add600640 1 -- 172.21.15.19:0/3525637231 <== osd.2 v2:172.21.15.19:6813/2392070031 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ae0167210 con 0x7f3ac007c700 2024-04-25T11:27:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.162+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb180744e0 msgr2=0x7fcb180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.162+0000 7fcb3d200640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb180744e0 0x7fcb180769f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005c00 tx=0x7fcb30009040 comp rx=0 tx=0).stop 2024-04-25T11:27:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.162+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3815c640 msgr2=0x7fcb3815f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.162+0000 7fcb3d200640 1 --2- 172.21.15.19:0/3710510078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb3815c640 0x7fcb3815f490 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800e850 tx=0x7fcb2800ed10 comp rx=0 tx=0).stop 2024-04-25T11:27:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.162+0000 7fcb36800640 1 -- 172.21.15.19:0/3710510078 reap_dead start 2024-04-25T11:27:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7f268c07c700 msgr2=0x7f268c07eb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7f268c07c700 0x7f268c07eb50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 shutdown_connections 2024-04-25T11:27:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 >> 172.21.15.19:0/3710510078 conn(0x7fcb38073f00 msgr2=0x7fcb38133db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f268c0744b0 msgr2=0x7f268c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f268c0744b0 0x7f268c0769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f269c034000 tx=0x7f269c02e000 comp rx=0 tx=0).stop 2024-04-25T11:27:33.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac1414a0 msgr2=0x7f26ac1e6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 --2- 172.21.15.19:0/1810068392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26ac1414a0 0x7f26ac1e6400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26a400cdb0 tx=0x7f26a400a650 comp rx=0 tx=0).stop 2024-04-25T11:27:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26aa800640 1 -- 172.21.15.19:0/1810068392 reap_dead start 2024-04-25T11:27:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f3ac007c700 msgr2=0x7f3ac007eb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f3ae8400640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f3ac007c700 0x7f3ac007eb50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 shutdown_connections 2024-04-25T11:27:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.163+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 >> 172.21.15.19:0/1810068392 conn(0x7f26ac073f00 msgr2=0x7f26ac13ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 msgr2=0x7f3ac00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7f3ae8400640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40090b0 tx=0x7f3ad402f000 comp rx=0 tx=0).stop 2024-04-25T11:27:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae0163ef0 msgr2=0x7f3ae0160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7f3ae8400640 1 --2- 172.21.15.19:0/3525637231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae0163ef0 0x7f3ae0160ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800bdd0 tx=0x7f3ad8009ed0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7f3ae6400640 1 -- 172.21.15.19:0/3525637231 reap_dead start 2024-04-25T11:27:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 shutdown_connections 2024-04-25T11:27:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 shutdown_connections 2024-04-25T11:27:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 >> 172.21.15.19:0/3525637231 conn(0x7f3ae0073050 msgr2=0x7f3ae016ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.164+0000 7fcb3d200640 1 -- 172.21.15.19:0/3710510078 wait complete. 2024-04-25T11:27:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.165+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 shutdown_connections 2024-04-25T11:27:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.165+0000 7f3ae8400640 1 -- 172.21.15.19:0/3525637231 wait complete. 2024-04-25T11:27:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.167+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 shutdown_connections 2024-04-25T11:27:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.167+0000 7f26ab200640 1 -- 172.21.15.19:0/1810068392 wait complete. 2024-04-25T11:27:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.174+0000 7fc1d0a00640 1 -- 172.21.15.19:0/751586239 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc1cc0b8a30 con 0x7fc1ac07c7d0 2024-04-25T11:27:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.176+0000 7fc1c3400640 1 -- 172.21.15.19:0/751586239 <== osd.4 v2:172.21.15.169:6816/3656468664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc1cc0b8a30 con 0x7fc1ac07c7d0 2024-04-25T11:27:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.177+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a70048e30 con 0x7f7a5807c7b0 2024-04-25T11:27:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.177+0000 7fc1c0c00640 1 -- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7fc1ac07c7d0 msgr2=0x7fc1ac07ec20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.177+0000 7fc1c0c00640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7fc1ac07c7d0 0x7fc1ac07ec20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7f7a6e000640 1 -- 172.21.15.19:0/605924097 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7a70048e30 con 0x7f7a5807c7b0 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7fc1c0c00640 1 -- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1ac074500 msgr2=0x7fc1ac076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7fc1c0c00640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1ac074500 0x7fc1ac076a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).stop 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7fc1c0c00640 1 -- 172.21.15.19:0/751586239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0b4da0 msgr2=0x7fc1cc15e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7fc1c0c00640 1 --2- 172.21.15.19:0/751586239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1cc0b4da0 0x7fc1cc15e870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c760 tx=0x7fc1c400cc20 comp rx=0 tx=0).stop 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7fc1cbe00640 1 -- 172.21.15.19:0/751586239 reap_dead start 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7fc1c0c00640 1 -- 172.21.15.19:0/751586239 shutdown_connections 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.178+0000 7fc1c0c00640 1 -- 172.21.15.19:0/751586239 >> 172.21.15.19:0/751586239 conn(0x7fc1cc0d2bd0 msgr2=0x7fc1cc0b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.179+0000 7fc1c0c00640 1 -- 172.21.15.19:0/751586239 shutdown_connections 2024-04-25T11:27:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.179+0000 7fc1c0c00640 1 -- 172.21.15.19:0/751586239 wait complete. 2024-04-25T11:27:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.179+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb760002dd0 con 0x7fb76407c970 2024-04-25T11:27:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f7a5807c7b0 msgr2=0x7f7a5807ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f7a5807c7b0 0x7f7a5807ec00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a580744e0 msgr2=0x7f7a580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a580744e0 0x7f7a580769f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a6802e000 comp rx=0 tx=0).stop 2024-04-25T11:27:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a70048300 msgr2=0x7f7a7004a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 --2- 172.21.15.19:0/605924097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a70048300 0x7f7a7004a120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a78069ed0 tx=0x7f7a7806eb20 comp rx=0 tx=0).stop 2024-04-25T11:27:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a76800640 1 -- 172.21.15.19:0/605924097 reap_dead start 2024-04-25T11:27:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7fb780c00640 1 -- 172.21.15.19:0/2002040723 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb760002dd0 con 0x7fb76407c970 2024-04-25T11:27:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 shutdown_connections 2024-04-25T11:27:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 >> 172.21.15.19:0/605924097 conn(0x7f7a70054cf0 msgr2=0x7f7a700531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.180+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 shutdown_connections 2024-04-25T11:27:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.181+0000 7f7a7ca00640 1 -- 172.21.15.19:0/605924097 wait complete. 2024-04-25T11:27:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.181+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb76407c970 msgr2=0x7fb76407edc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.181+0000 7fb78ba00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb76407c970 0x7fb76407edc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7640746a0 msgr2=0x7fb764076bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7fb78ba00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7640746a0 0x7fb764076bb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb778034000 tx=0x7fb77802e000 comp rx=0 tx=0).stop 2024-04-25T11:27:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c04c170 msgr2=0x7fb77c04c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7fb78ba00640 1 --2- 172.21.15.19:0/2002040723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77c04c170 0x7fb77c04c580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7840519b0 tx=0x7fb78406eb20 comp rx=0 tx=0).stop 2024-04-25T11:27:33.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7fb789a00640 1 -- 172.21.15.19:0/2002040723 reap_dead start 2024-04-25T11:27:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 shutdown_connections 2024-04-25T11:27:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 >> 172.21.15.19:0/2002040723 conn(0x7fb77c056f10 msgr2=0x7fb77c0e0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.181+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f430c0b8a10 con 0x7f42e807c820 2024-04-25T11:27:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 shutdown_connections 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.182+0000 7f4302000640 1 -- 172.21.15.19:0/702826989 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f430c0b8a10 con 0x7f42e807c820 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.183+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f42e807c820 msgr2=0x7f42e807ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.183+0000 7f4310a00640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f42e807c820 0x7f42e807ec70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42e80745b0 msgr2=0x7f42e8076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7f4310a00640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42e80745b0 0x7f42e8076ac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f430c0744c0 tx=0x7f42f4008040 comp rx=0 tx=0).stop 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f430c06dcf0 msgr2=0x7f430c160420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7f4310a00640 1 --2- 172.21.15.19:0/702826989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f430c06dcf0 0x7f430c160420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009890 tx=0x7f42fc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7f430a800640 1 -- 172.21.15.19:0/702826989 reap_dead start 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 shutdown_connections 2024-04-25T11:27:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.184+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 >> 172.21.15.19:0/702826989 conn(0x7f430c08d110 msgr2=0x7f430c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.185+0000 7fb78ba00640 1 -- 172.21.15.19:0/2002040723 wait complete. 2024-04-25T11:27:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.185+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 shutdown_connections 2024-04-25T11:27:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.185+0000 7f4310a00640 1 -- 172.21.15.19:0/702826989 wait complete. 2024-04-25T11:27:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.190+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f700c002dd0 con 0x7f701407c720 2024-04-25T11:27:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.191+0000 7f702b400640 1 -- 172.21.15.19:0/3205105346 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f700c002dd0 con 0x7f701407c720 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.193+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f701407c720 msgr2=0x7f701407eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.193+0000 7f703a400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f701407c720 0x7f701407eb70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.193+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744b0 msgr2=0x7f70140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.193+0000 7f703a400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744b0 0x7f70140769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7024034000 tx=0x7f702402e000 comp rx=0 tx=0).stop 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.193+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70341681d0 msgr2=0x7f7034162800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.193+0000 7f703a400640 1 --2- 172.21.15.19:0/3205105346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70341681d0 0x7f7034162800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f702c00c760 tx=0x7f702c00cc20 comp rx=0 tx=0).stop 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.193+0000 7f7033e00640 1 -- 172.21.15.19:0/3205105346 reap_dead start 2024-04-25T11:27:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.196+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 shutdown_connections 2024-04-25T11:27:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.196+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 >> 172.21.15.19:0/3205105346 conn(0x7f70340d1ae0 msgr2=0x7f7034163770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.196+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 shutdown_connections 2024-04-25T11:27:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.196+0000 7f703a400640 1 -- 172.21.15.19:0/3205105346 wait complete. 2024-04-25T11:27:33.220 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-04-25T11:27:33.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:27:33.221 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-04-25T11:27:33.222 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:27:33.222 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-04-25T11:27:33.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:27:33.223 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-04-25T11:27:33.224 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:27:33.224 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-04-25T11:27:33.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:27:33.225 INFO:teuthology.orchestra.run.smithi019.stdout:60129542150 2024-04-25T11:27:33.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:27:33.226 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-04-25T11:27:33.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:27:33.228 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-04-25T11:27:33.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:27:33.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.681+0000 7f6c03200640 1 Processor -- start 2024-04-25T11:27:33.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.681+0000 7f6c03200640 1 -- start start 2024-04-25T11:27:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.685+0000 7f6c03200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6bfc0b6a50 0x7f6bfc0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.685+0000 7f6c03200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bfc070d20 con 0x7f6bfc142650 2024-04-25T11:27:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.686+0000 7f6c03200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bfc0b4f60 con 0x7f6bfc160420 2024-04-25T11:27:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.686+0000 7f6c03200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6bfc071000 con 0x7f6bfc0b6a50 2024-04-25T11:27:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.686+0000 7f6c01200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6bfc0b6a50 0x7f6bfc0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.686+0000 7f6c01200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6bfc0b6a50 0x7f6bfc0cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:55342/0 (socket says 172.21.15.19:55342) 2024-04-25T11:27:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.686+0000 7f6c01200640 1 -- 172.21.15.19:0/3995814729 learned_addr learned my addr 172.21.15.19:0/3995814729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.688+0000 7f6c01200640 1 -- 172.21.15.19:0/3995814729 >> v1:172.21.15.169:6789/0 conn(0x7f6bfc160420 legacy=0x7f6bfc06f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:27:33.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.688+0000 7f6c01200640 1 -- 172.21.15.19:0/3995814729 >> v1:172.21.15.19:6789/0 conn(0x7f6bfc142650 legacy=0x7f6bfc15fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.688+0000 7f6c01200640 1 -- 172.21.15.19:0/3995814729 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc0b5240 con 0x7f6bfc0b6a50 2024-04-25T11:27:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.690+0000 7f6c01200640 1 --2- 172.21.15.19:0/3995814729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0cdbb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc071180 tx=0x7f6bf402bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e31de73d73fd0b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.690+0000 7f6bfaa00640 1 -- 172.21.15.19:0/3995814729 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf402d920 con 0x7f6bfc0b6a50 2024-04-25T11:27:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.690+0000 7f8607400640 1 Processor -- start 2024-04-25T11:27:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.690+0000 7f6bfaa00640 1 -- 172.21.15.19:0/3995814729 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf402dac0 con 0x7f6bfc0b6a50 2024-04-25T11:27:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.691+0000 7f6bfaa00640 1 -- 172.21.15.19:0/3995814729 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4036480 con 0x7f6bfc0b6a50 2024-04-25T11:27:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.692+0000 7f6c03200640 1 -- 172.21.15.19:0/3995814729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 msgr2=0x7f6bfc0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.692+0000 7f6c03200640 1 --2- 172.21.15.19:0/3995814729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0cdbb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc071180 tx=0x7f6bf402bc00 comp rx=0 tx=0).stop 2024-04-25T11:27:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.692+0000 7f6c03200640 1 -- 172.21.15.19:0/3995814729 shutdown_connections 2024-04-25T11:27:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.692+0000 7f6c03200640 1 --2- 172.21.15.19:0/3995814729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0cdbb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.691+0000 7f8607400640 1 -- start start 2024-04-25T11:27:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.693+0000 7f6c03200640 1 -- 172.21.15.19:0/3995814729 >> 172.21.15.19:0/3995814729 conn(0x7f6bfc073f20 msgr2=0x7f6bfc06c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.693+0000 7f8607400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f86000d0e00 0x7f86000d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.693+0000 7f8607400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f860006c5c0 0x7f860006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.693+0000 7f8607400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f860006cf10 con 0x7f86000b7810 2024-04-25T11:27:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.693+0000 7f8607400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f86000d5a60 con 0x7f86000d0e00 2024-04-25T11:27:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.693+0000 7f8607400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f86000d1850 con 0x7f860006c5c0 2024-04-25T11:27:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.694+0000 7f6c03200640 1 -- 172.21.15.19:0/3995814729 shutdown_connections 2024-04-25T11:27:33.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.694+0000 7f6c03200640 1 -- 172.21.15.19:0/3995814729 wait complete. 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.694+0000 7f8605400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f86000b7810 0x7f86000b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52652/0 (socket says 172.21.15.19:52652) 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.694+0000 7f8605400640 1 -- 172.21.15.19:0/3332317973 learned_addr learned my addr 172.21.15.19:0/3332317973 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.694+0000 7f85ffe00640 1 --2- 172.21.15.19:0/3332317973 >> v2:172.21.15.169:3300/0 conn(0x7f860006c5c0 0x7f860006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.695+0000 7f85ff400640 1 -- 172.21.15.19:0/3332317973 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428255225 0 0) 0x7f85ec001110 con 0x7f86000b7810 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.695+0000 7f85ff400640 1 -- 172.21.15.19:0/3332317973 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85e4003520 con 0x7f86000b7810 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.695+0000 7f85ffe00640 1 -- 172.21.15.19:0/3332317973 >> v2:172.21.15.19:3300/0 conn(0x7f86000d0e00 msgr2=0x7f86000d1210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.695+0000 7f85ff400640 1 -- 172.21.15.19:0/3332317973 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2953014684 0 0) 0x7f85e4003520 con 0x7f86000b7810 2024-04-25T11:27:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.697+0000 7f8604a00640 1 --2- 172.21.15.19:0/3332317973 >> v2:172.21.15.19:3300/0 conn(0x7f86000d0e00 0x7f86000d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.697+0000 7f85ffe00640 1 --2- 172.21.15.19:0/3332317973 >> v2:172.21.15.19:3300/0 conn(0x7f86000d0e00 0x7f86000d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.698+0000 7f6c03200640 1 Processor -- start 2024-04-25T11:27:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.697+0000 7f85ffe00640 1 -- 172.21.15.19:0/3332317973 >> v1:172.21.15.19:6789/0 conn(0x7f86000b7810 legacy=0x7f86000b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.697+0000 7f85ffe00640 1 -- 172.21.15.19:0/3332317973 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600160de0 con 0x7f860006c5c0 2024-04-25T11:27:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.698+0000 7f8604a00640 1 --2- 172.21.15.19:0/3332317973 >> v2:172.21.15.19:3300/0 conn(0x7f86000d0e00 0x7f86000d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.699+0000 7f85ffe00640 1 --2- 172.21.15.19:0/3332317973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f860006c5c0 0x7f860006c9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86000d5be0 tx=0x7f85f002a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b630d199a588273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.699+0000 7f85ff400640 1 -- 172.21.15.19:0/3332317973 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec001840 con 0x7f860006c5c0 2024-04-25T11:27:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f85ff400640 1 -- 172.21.15.19:0/3332317973 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f0007540 con 0x7f860006c5c0 2024-04-25T11:27:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f85ff400640 1 -- 172.21.15.19:0/3332317973 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f00078f0 con 0x7f860006c5c0 2024-04-25T11:27:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.698+0000 7f6c03200640 1 -- start start 2024-04-25T11:27:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f6c03200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f6c03200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc142650 0x7f6bfc203010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f6c03200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bfc168600 0x7f6bfc168a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f6c03200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc06f2d0 con 0x7f6bfc142650 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f6c03200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc06f150 con 0x7f6bfc0b6a50 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.700+0000 7f6c03200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6bfc06fcb0 con 0x7f6bfc168600 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.701+0000 7f8607400640 1 -- 172.21.15.19:0/3332317973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f860006c5c0 msgr2=0x7f860006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.701+0000 7f8607400640 1 --2- 172.21.15.19:0/3332317973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f860006c5c0 0x7f860006c9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86000d5be0 tx=0x7f85f002a580 comp rx=0 tx=0).stop 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.701+0000 7f8607400640 1 -- 172.21.15.19:0/3332317973 shutdown_connections 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.701+0000 7f8607400640 1 --2- 172.21.15.19:0/3332317973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f860006c5c0 0x7f860006c9d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.701+0000 7f8607400640 1 --2- 172.21.15.19:0/3332317973 >> v2:172.21.15.19:3300/0 conn(0x7f86000d0e00 0x7f86000d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.701+0000 7f8607400640 1 -- 172.21.15.19:0/3332317973 >> 172.21.15.19:0/3332317973 conn(0x7f86000d5d90 msgr2=0x7f86000d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.702+0000 7f8607400640 1 -- 172.21.15.19:0/3332317973 shutdown_connections 2024-04-25T11:27:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.702+0000 7f8607400640 1 -- 172.21.15.19:0/3332317973 wait complete. 2024-04-25T11:27:33.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.702+0000 7f6c01200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bfc168600 0x7f6bfc168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.702+0000 7f6bfbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.702+0000 7f6bfbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0d0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:55360/0 (socket says 172.21.15.19:55360) 2024-04-25T11:27:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.702+0000 7f6bfb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc142650 0x7f6bfc203010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.702+0000 7f6bfbe00640 1 -- 172.21.15.19:0/1536058744 learned_addr learned my addr 172.21.15.19:0/1536058744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.704+0000 7f6bfb400640 1 -- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bfc168600 msgr2=0x7f6bfc168a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.704+0000 7f84ed000640 1 Processor -- start 2024-04-25T11:27:33.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.705+0000 7fb59b200640 1 Processor -- start 2024-04-25T11:27:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.704+0000 7f6bfb400640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bfc168600 0x7f6bfc168a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.704+0000 7f6bfb400640 1 -- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 msgr2=0x7f6bfc0d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.704+0000 7f6bfb400640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.705+0000 7f6bfb400640 1 -- 172.21.15.19:0/1536058744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc168f50 con 0x7f6bfc142650 2024-04-25T11:27:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.705+0000 7f6bfbe00640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.705+0000 7f6c01200640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bfc168600 0x7f6bfc168a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:33.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.705+0000 7f84ed000640 1 -- start start 2024-04-25T11:27:33.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.705+0000 7f8607400640 1 Processor -- start 2024-04-25T11:27:33.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.706+0000 7f8607400640 1 -- start start 2024-04-25T11:27:33.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.706+0000 7f6bfb400640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc142650 0x7f6bfc203010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00b410 tx=0x7f6bec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.706+0000 7fb59b200640 1 -- start start 2024-04-25T11:27:33.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.706+0000 7f6bf3e00640 1 -- 172.21.15.19:0/1536058744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec015020 con 0x7f6bfc142650 2024-04-25T11:27:33.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.706+0000 7f6bf3e00640 1 -- 172.21.15.19:0/1536058744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bec011830 con 0x7f6bfc142650 2024-04-25T11:27:33.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.706+0000 7f6bf3e00640 1 -- 172.21.15.19:0/1536058744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec011c60 con 0x7f6bfc142650 2024-04-25T11:27:33.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.707+0000 7f6c03200640 1 -- 172.21.15.19:0/1536058744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc16afe0 con 0x7f6bfc142650 2024-04-25T11:27:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.709+0000 7fb59b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb5940e7e80 0x7fb594054060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.709+0000 7fb59b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5940e3de0 0x7fb5940066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.709+0000 7fb59b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5940e4de0 con 0x7fb5940545a0 2024-04-25T11:27:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.709+0000 7fb59b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb594006230 con 0x7fb5940e7e80 2024-04-25T11:27:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.709+0000 7fb59b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb5940549b0 con 0x7fb5940e3de0 2024-04-25T11:27:33.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.709+0000 7f84ed000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 0x7f84e80b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.708+0000 7f8607400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f860006c5c0 0x7f86000cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.708+0000 7f8607400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86000b7810 0x7f860015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.708+0000 7f8607400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86000d0e00 0x7f8600207080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7fb599e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb5940e7e80 0x7fb594054060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7fb599e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb5940e7e80 0x7fb594054060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47312/0 (socket says 172.21.15.19:47312) 2024-04-25T11:27:33.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7fb599e00640 1 -- 172.21.15.19:0/2721655080 learned_addr learned my addr 172.21.15.19:0/2721655080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f8607400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8600160ad0 con 0x7f86000d0e00 2024-04-25T11:27:33.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f8604a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f860006c5c0 0x7f86000cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7fb59a800640 1 --2- 172.21.15.19:0/2721655080 >> v2:172.21.15.169:3300/0 conn(0x7fb5940e3de0 0x7fb5940066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f8607400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f86001613a0 con 0x7f86000b7810 2024-04-25T11:27:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f8605400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86000d0e00 0x7f8600207080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f8604a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f860006c5c0 0x7f86000cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53162/0 (socket says 172.21.15.19:53162) 2024-04-25T11:27:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f8604a00640 1 -- 172.21.15.19:0/2911403239 learned_addr learned my addr 172.21.15.19:0/2911403239 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f85ffe00640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86000b7810 0x7f860015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.710+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f86000b5320 con 0x7f860006c5c0 2024-04-25T11:27:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.709+0000 7f6c03200640 1 -- 172.21.15.19:0/1536058744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc16b4e0 con 0x7f6bfc142650 2024-04-25T11:27:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7fb599e00640 1 -- 172.21.15.19:0/2721655080 >> v2:172.21.15.169:3300/0 conn(0x7fb5940e3de0 msgr2=0x7fb5940066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7fb599e00640 1 --2- 172.21.15.19:0/2721655080 >> v2:172.21.15.169:3300/0 conn(0x7fb5940e3de0 0x7fb5940066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc005290 con 0x7f6bfc142650 2024-04-25T11:27:33.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7f84ed000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84e80ce170 0x7f84e806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.712+0000 7f8604a00640 1 -- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86000b7810 msgr2=0x7f860015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.712+0000 7f8604a00640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86000b7810 0x7f860015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.712+0000 7f8604a00640 1 -- 172.21.15.19:0/2911403239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86000d0e00 msgr2=0x7f8600207080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 0x7f84e80b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 0x7f84e80b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53168/0 (socket says 172.21.15.19:53168) 2024-04-25T11:27:33.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.712+0000 7f84e5400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f84e80cdd60 0x7f84e80cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52666/0 (socket says 172.21.15.19:52666) 2024-04-25T11:27:33.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7fb599e00640 1 -- 172.21.15.19:0/2721655080 >> v1:172.21.15.169:6789/0 conn(0x7fb5940545a0 legacy=0x7fb5940e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7fb599e00640 1 -- 172.21.15.19:0/2721655080 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5940e4540 con 0x7fb5940e7e80 2024-04-25T11:27:33.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.711+0000 7fb59a800640 1 --2- 172.21.15.19:0/2721655080 >> v2:172.21.15.169:3300/0 conn(0x7fb5940e3de0 0x7fb5940066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.712+0000 7f84e5e00640 1 -- 172.21.15.19:0/3803884327 learned_addr learned my addr 172.21.15.19:0/3803884327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.712+0000 7f8604a00640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86000d0e00 0x7f8600207080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.712+0000 7f8604a00640 1 -- 172.21.15.19:0/2911403239 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860015e7b0 con 0x7f860006c5c0 2024-04-25T11:27:33.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.713+0000 7f8605400640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86000d0e00 0x7f8600207080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.713+0000 7f8604a00640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f860006c5c0 0x7f86000cc710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c820 tx=0x7f85f800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.713+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e8160450 con 0x7f84e80cdd60 2024-04-25T11:27:33.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.713+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f84e8160210 con 0x7f84e80ce170 2024-04-25T11:27:33.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.713+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80b6ed0 con 0x7f84e815fe00 2024-04-25T11:27:33.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.713+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 102528172 0 0) 0x7f84e8160450 con 0x7f84e80cdd60 2024-04-25T11:27:33.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84cc003520 con 0x7f84e80cdd60 2024-04-25T11:27:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e6800640 1 --2- 172.21.15.19:0/3803884327 >> v2:172.21.15.19:3300/0 conn(0x7f84e80ce170 0x7f84e806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f85fcc00640 1 -- 172.21.15.19:0/2911403239 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8015070 con 0x7f860006c5c0 2024-04-25T11:27:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f85fcc00640 1 -- 172.21.15.19:0/2911403239 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f8007a30 con 0x7f860006c5c0 2024-04-25T11:27:33.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f85fcc00640 1 -- 172.21.15.19:0/2911403239 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8007de0 con 0x7f860006c5c0 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3843264476 0 0) 0x7f84cc003520 con 0x7f84e80cdd60 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 msgr2=0x7f84e80b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 0x7f84e80b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 >> v2:172.21.15.19:3300/0 conn(0x7f84e80ce170 msgr2=0x7f84e806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 --2- 172.21.15.19:0/3803884327 >> v2:172.21.15.19:3300/0 conn(0x7f84e80ce170 0x7f84e806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e8160730 con 0x7f84e80cdd60 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f84d4001670 con 0x7f84e80cdd60 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 >> v1:172.21.15.19:6789/0 conn(0x7f84e80cdd60 legacy=0x7f84e80cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84cc0052a0 0x7f84cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc0037a0 0x7f84cc003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 0x7f84cc004130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8160210 con 0x7f84cc0037a0 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80b6ed0 con 0x7f84cc0052a0 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84cc00cc30 con 0x7f84e815fe00 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.715+0000 7f84e5e00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84cc0052a0 0x7f84cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860015ea90 con 0x7f860006c5c0 2024-04-25T11:27:33.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.714+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f860015f050 con 0x7f860006c5c0 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.716+0000 7f6bf3e00640 1 -- 172.21.15.19:0/1536058744 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6bec018020 con 0x7f6bfc142650 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.716+0000 7f84e5400640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc0037a0 0x7f84cc003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.716+0000 7f84e6800640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 0x7f84cc004130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.716+0000 7f84e5e00640 1 -- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 msgr2=0x7f84cc004130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.716+0000 7f84e5e00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84e815fe00 0x7f84cc004130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.716+0000 7f84e5e00640 1 -- 172.21.15.19:0/3803884327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc0037a0 msgr2=0x7f84cc003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f84e5e00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc0037a0 0x7f84cc003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f84e5e00640 1 -- 172.21.15.19:0/3803884327 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e8160730 con 0x7f84cc0052a0 2024-04-25T11:27:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f96b2800640 1 Processor -- start 2024-04-25T11:27:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f84e5400640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc0037a0 0x7f84cc003bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f94ae200640 1 Processor -- start 2024-04-25T11:27:33.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8005290 con 0x7f860006c5c0 2024-04-25T11:27:33.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f96b2800640 1 -- start start 2024-04-25T11:27:33.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.717+0000 7f94ae200640 1 -- start start 2024-04-25T11:27:33.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.718+0000 7fb599e00640 1 --2- 172.21.15.19:0/2721655080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e7e80 0x7fb594054060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb588009870 tx=0x7fb58802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7757c5c06ac6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.718+0000 7f6bf3e00640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6bd8074500 0x7f6bd8076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.718+0000 7f4959400640 1 Processor -- start 2024-04-25T11:27:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.718+0000 7f4959400640 1 -- start start 2024-04-25T11:27:33.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.719+0000 7f6bf3e00640 1 -- 172.21.15.19:0/1536058744 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6bec099060 con 0x7f6bfc142650 2024-04-25T11:27:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.719+0000 7f84e5e00640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84cc0052a0 0x7f84cc004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84cc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8d5d7a59b2d753f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f6bfbe00640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6bd8074500 0x7f6bd8076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f85fcc00640 1 -- 172.21.15.19:0/2911403239 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f85f8018070 con 0x7f860006c5c0 2024-04-25T11:27:33.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d80097e0 con 0x7f84cc0052a0 2024-04-25T11:27:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8038070 con 0x7f84cc0052a0 2024-04-25T11:27:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f84e4a00640 1 -- 172.21.15.19:0/3803884327 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8033610 con 0x7f84cc0052a0 2024-04-25T11:27:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f96b2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f96ac06ccd0 0x7f96ac0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f96b2800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac0d1850 con 0x7f96ac0d4a50 2024-04-25T11:27:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f96b2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac160de0 con 0x7f96ac0d0e00 2024-04-25T11:27:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.720+0000 7f96b2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f96ac1610c0 con 0x7f96ac06ccd0 2024-04-25T11:27:33.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.721+0000 7f96abe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f96ac0d4a50 0x7f96ac0d4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52680/0 (socket says 172.21.15.19:52680) 2024-04-25T11:27:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.721+0000 7f85fcc00640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85e00745b0 0x7f85e0076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f85fcc00640 1 -- 172.21.15.19:0/2911403239 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f85f8095580 con 0x7f860006c5c0 2024-04-25T11:27:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.721+0000 7f96abe00640 1 -- 172.21.15.19:0/1845199068 learned_addr learned my addr 172.21.15.19:0/1845199068 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.721+0000 7f96aaa00640 1 --2- 172.21.15.19:0/1845199068 >> v2:172.21.15.169:3300/0 conn(0x7f96ac06ccd0 0x7f96ac0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f96aa000640 1 -- 172.21.15.19:0/1845199068 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1055381979 0 0) 0x7f96ac160de0 con 0x7f96ac0d0e00 2024-04-25T11:27:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f96aa000640 1 -- 172.21.15.19:0/1845199068 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9690003520 con 0x7f96ac0d0e00 2024-04-25T11:27:33.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.721+0000 7f94ae200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80d0780 0x7f94a80707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.721+0000 7f94ae200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f94a8160510 0x7f94a815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f4959400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4954160510 0x7f49540707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f96aaa00640 1 -- 172.21.15.19:0/1845199068 >> v1:172.21.15.169:6789/0 conn(0x7f96ac0d0e00 legacy=0x7f96ac06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f96aaa00640 1 -- 172.21.15.19:0/1845199068 >> v1:172.21.15.19:6789/0 conn(0x7f96ac0d4a50 legacy=0x7f96ac0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f4959400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f4959400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49540cf9c0 con 0x7f49540d0780 2024-04-25T11:27:33.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f4959400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4954156d20 con 0x7f4954160510 2024-04-25T11:27:33.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f4959400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f49540cfca0 con 0x7f49540d1120 2024-04-25T11:27:33.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f6bfbe00640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6bd8074500 0x7f6bd8076a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6be40016e0 tx=0x7f6be4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f96aaa00640 1 -- 172.21.15.19:0/1845199068 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac1613a0 con 0x7f96ac06ccd0 2024-04-25T11:27:33.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f94ae200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a80cf9c0 con 0x7f94a80d1120 2024-04-25T11:27:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f85ffe00640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85e00745b0 0x7f85e0076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f4952800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f49540d0780 0x7f495415fe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52686/0 (socket says 172.21.15.19:52686) 2024-04-25T11:27:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f4952800640 1 -- 172.21.15.19:0/1666469153 learned_addr learned my addr 172.21.15.19:0/1666469153 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f94ae200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f94a8156d20 con 0x7f94a8160510 2024-04-25T11:27:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f94ae200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94a80cfca0 con 0x7f94a80d0780 2024-04-25T11:27:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f94a7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80d0780 0x7f94a80707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84cc0052a0 msgr2=0x7f84cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.722+0000 7f94a6a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f94a80d1120 0x7f94a815fe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:51218/0 (socket says 172.21.15.19:51218) 2024-04-25T11:27:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f84ed000640 1 --2- 172.21.15.19:0/3803884327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84cc0052a0 0x7f84cc004800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84cc005830 comp rx=0 tx=0).stop 2024-04-25T11:27:33.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f94a6a00640 1 -- 172.21.15.19:0/247498079 learned_addr learned my addr 172.21.15.19:0/247498079 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:33.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f84e6800640 1 -- 172.21.15.19:0/3803884327 reap_dead start 2024-04-25T11:27:33.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f4953200640 1 --2- 172.21.15.19:0/1666469153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f94a7e00640 1 --2- 172.21.15.19:0/247498079 >> v2:172.21.15.19:3300/0 conn(0x7f94a8160510 0x7f94a815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 shutdown_connections 2024-04-25T11:27:33.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 >> 172.21.15.19:0/3803884327 conn(0x7f84e8073050 msgr2=0x7f84e8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.723+0000 7f4951e00640 1 --2- 172.21.15.19:0/1666469153 >> v2:172.21.15.169:3300/0 conn(0x7f4954160510 0x7f49540707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751697226 0 0) 0x7f94a80cf9c0 con 0x7f94a80d1120 2024-04-25T11:27:33.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f948c003520 con 0x7f94a80d1120 2024-04-25T11:27:33.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 shutdown_connections 2024-04-25T11:27:33.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f84ed000640 1 -- 172.21.15.19:0/3803884327 wait complete. 2024-04-25T11:27:33.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f96aaa00640 1 --2- 172.21.15.19:0/1845199068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac06ccd0 0x7f96ac0d1210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96980097a0 tx=0x7f969802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7673a978ed29a9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f4953200640 1 -- 172.21.15.19:0/1666469153 >> v2:172.21.15.169:3300/0 conn(0x7f4954160510 msgr2=0x7f49540707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f4953200640 1 --2- 172.21.15.19:0/1666469153 >> v2:172.21.15.169:3300/0 conn(0x7f4954160510 0x7f49540707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f4953200640 1 -- 172.21.15.19:0/1666469153 >> v1:172.21.15.19:6789/0 conn(0x7f49540d0780 legacy=0x7f495415fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f4953200640 1 -- 172.21.15.19:0/1666469153 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49540cff80 con 0x7f49540d1120 2024-04-25T11:27:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1788337105 0 0) 0x7f948c003520 con 0x7f94a80d1120 2024-04-25T11:27:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.724+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80d0780 msgr2=0x7f94a80707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80d0780 0x7f94a80707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 >> v2:172.21.15.19:3300/0 conn(0x7f94a8160510 msgr2=0x7f94a815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 --2- 172.21.15.19:0/247498079 >> v2:172.21.15.19:3300/0 conn(0x7f94a8160510 0x7f94a815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a80cff80 con 0x7f94a80d1120 2024-04-25T11:27:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9494001670 con 0x7f94a80d1120 2024-04-25T11:27:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a7e00640 1 --2- 172.21.15.19:0/247498079 >> v2:172.21.15.19:3300/0 conn(0x7f94a8160510 0x7f94a815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 >> v1:172.21.15.169:6789/0 conn(0x7f94a80d1120 legacy=0x7f94a815fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948c0039b0 0x7f948c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948c004800 0x7f948c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f948c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8156d20 con 0x7f948c005150 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80cfca0 con 0x7f948c0039b0 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f948c00c060 con 0x7f948c004800 2024-04-25T11:27:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f96aa000640 1 -- 172.21.15.19:0/1845199068 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698037070 con 0x7f96ac06ccd0 2024-04-25T11:27:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f85ffe00640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85e00745b0 0x7f85e0076ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85f0034000 tx=0x7f85f002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f96aa000640 1 -- 172.21.15.19:0/1845199068 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9698032440 con 0x7f96ac06ccd0 2024-04-25T11:27:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.725+0000 7f94a7e00640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f948c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7f94a6a00640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948c004800 0x7f948c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7f96aa000640 1 -- 172.21.15.19:0/1845199068 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698037070 con 0x7f96ac06ccd0 2024-04-25T11:27:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7fb598a00640 1 -- 172.21.15.19:0/2721655080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588037070 con 0x7fb5940e7e80 2024-04-25T11:27:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7fb598a00640 1 -- 172.21.15.19:0/2721655080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb588032440 con 0x7fb5940e7e80 2024-04-25T11:27:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7fb598a00640 1 -- 172.21.15.19:0/2721655080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588032810 con 0x7fb5940e7e80 2024-04-25T11:27:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7f96b2800640 1 -- 172.21.15.19:0/1845199068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac06ccd0 msgr2=0x7f96ac0d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7f96b2800640 1 --2- 172.21.15.19:0/1845199068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac06ccd0 0x7f96ac0d1210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96980097a0 tx=0x7f969802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7f94a7400640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80d0780 0x7f94a80707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.726+0000 7f4953200640 1 --2- 172.21.15.19:0/1666469153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415c7b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f494c009d90 tx=0x7f494c02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ebf362e7c00389a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f94a7e00640 1 -- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948c004800 msgr2=0x7f948c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f94a7e00640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948c004800 0x7f948c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f94a7e00640 1 -- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948c0039b0 msgr2=0x7f948c003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:27:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f94a7e00640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f948c0039b0 0x7f948c003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f6bf3e00640 1 -- 172.21.15.19:0/1536058744 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6bfc168f50 con 0x7f6bfc142650 2024-04-25T11:27:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f94a7e00640 1 -- 172.21.15.19:0/247498079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948c003520 con 0x7f948c005150 2024-04-25T11:27:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f94a6a00640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948c004800 0x7f948c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f84ed000640 1 Processor -- start 2024-04-25T11:27:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7fb59b200640 1 -- 172.21.15.19:0/2721655080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e7e80 msgr2=0x7fb594054060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7fb59b200640 1 --2- 172.21.15.19:0/2721655080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e7e80 0x7fb594054060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb588009870 tx=0x7fb58802a210 comp rx=0 tx=0).stop 2024-04-25T11:27:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f96b2800640 1 -- 172.21.15.19:0/1845199068 shutdown_connections 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f96b2800640 1 --2- 172.21.15.19:0/1845199068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac06ccd0 0x7f96ac0d1210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.727+0000 7f96b2800640 1 -- 172.21.15.19:0/1845199068 >> 172.21.15.19:0/1845199068 conn(0x7f96ac0d5d90 msgr2=0x7f96ac0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7f96b2800640 1 -- 172.21.15.19:0/1845199068 shutdown_connections 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7f96b2800640 1 -- 172.21.15.19:0/1845199068 wait complete. 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7fb59b200640 1 -- 172.21.15.19:0/2721655080 shutdown_connections 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7fb59b200640 1 --2- 172.21.15.19:0/2721655080 >> v2:172.21.15.169:3300/0 conn(0x7fb5940e3de0 0x7fb5940066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7fb59b200640 1 --2- 172.21.15.19:0/2721655080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e7e80 0x7fb594054060 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7fb59b200640 1 -- 172.21.15.19:0/2721655080 >> 172.21.15.19:0/2721655080 conn(0x7fb594059120 msgr2=0x7fb594059530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7f84ed000640 1 -- start start 2024-04-25T11:27:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7fb59b200640 1 -- 172.21.15.19:0/2721655080 shutdown_connections 2024-04-25T11:27:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7fb59b200640 1 -- 172.21.15.19:0/2721655080 wait complete. 2024-04-25T11:27:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7f94a7e00640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f948c00b9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e320 tx=0x7f948c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61f47d006cce9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.728+0000 7efffb600640 1 Processor -- start 2024-04-25T11:27:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.729+0000 7efffb600640 1 -- start start 2024-04-25T11:27:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.729+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0009de0 con 0x7f948c005150 2024-04-25T11:27:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.729+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a002e610 con 0x7f948c005150 2024-04-25T11:27:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.729+0000 7f94a6000640 1 -- 172.21.15.19:0/247498079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a002e9c0 con 0x7f948c005150 2024-04-25T11:27:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84ed000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84cc0052a0 0x7f84e8154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84ed000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e8157f20 0x7f84e80c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84ed000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e80c6e20 0x7f84e815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84ed000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8160cf0 con 0x7f84e80c6e20 2024-04-25T11:27:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84ed000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84e815c9b0 con 0x7f84e8157f20 2024-04-25T11:27:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84ed000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80cfcc0 con 0x7f84cc0052a0 2024-04-25T11:27:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84e6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e80c6e20 0x7f84e815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f94ae200640 1 -- 172.21.15.19:0/247498079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 msgr2=0x7f948c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f94ae200640 1 --2- 172.21.15.19:0/247498079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f948c00b9c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e320 tx=0x7f948c005830 comp rx=0 tx=0).stop 2024-04-25T11:27:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f94a7e00640 1 -- 172.21.15.19:0/247498079 reap_dead start 2024-04-25T11:27:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84e6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e80c6e20 0x7f84e815a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47366/0 (socket says 172.21.15.19:47366) 2024-04-25T11:27:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84e5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84cc0052a0 0x7f84e8154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.730+0000 7f84e6800640 1 -- 172.21.15.19:0/2638519034 learned_addr learned my addr 172.21.15.19:0/2638519034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f84e5400640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e8157f20 0x7f84e80c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f94ae200640 1 -- 172.21.15.19:0/247498079 shutdown_connections 2024-04-25T11:27:33.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f94ae200640 1 -- 172.21.15.19:0/247498079 >> 172.21.15.19:0/247498079 conn(0x7f94a80d1ae0 msgr2=0x7f94a80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f94ae200640 1 -- 172.21.15.19:0/247498079 shutdown_connections 2024-04-25T11:27:33.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f94ae200640 1 -- 172.21.15.19:0/247498079 wait complete. 2024-04-25T11:27:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f84e6800640 1 -- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84cc0052a0 msgr2=0x7f84e8154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f84e6800640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84cc0052a0 0x7f84e8154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f84e6800640 1 -- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e8157f20 msgr2=0x7f84e80c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f84e6800640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e8157f20 0x7f84e80c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7f84e6800640 1 -- 172.21.15.19:0/2638519034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e80c7230 con 0x7f84e80c6e20 2024-04-25T11:27:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.731+0000 7fb59b200640 1 Processor -- start 2024-04-25T11:27:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7f96b2800640 1 Processor -- start 2024-04-25T11:27:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7efffb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff4072070 0x7efff415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7fb59b200640 1 -- start start 2024-04-25T11:27:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7efffb600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efff41433d0 0x7efff40d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7efffb600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efff41619f0 con 0x7efff40ccf80 2024-04-25T11:27:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7efffb600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efff40cd6b0 con 0x7efff41433d0 2024-04-25T11:27:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7efffb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efff4160c30 con 0x7efff4072070 2024-04-25T11:27:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7f96b2800640 1 -- start start 2024-04-25T11:27:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7f84e6800640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e80c6e20 0x7f84e815a4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f84cc005060 tx=0x7f84e0007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7efff8c00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7efff40ccf80 0x7efff4071960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:51228/0 (socket says 172.21.15.19:51228) 2024-04-25T11:27:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.732+0000 7efff8c00640 1 -- 172.21.15.19:0/2785553659 learned_addr learned my addr 172.21.15.19:0/2785553659 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7efff9600640 1 --2- 172.21.15.19:0/2785553659 >> v2:172.21.15.169:3300/0 conn(0x7efff41433d0 0x7efff40d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7f84de000640 1 -- 172.21.15.19:0/2638519034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4007cb0 con 0x7f84e80c6e20 2024-04-25T11:27:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e8159420 con 0x7f84e80c6e20 2024-04-25T11:27:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81599e0 con 0x7f84e80c6e20 2024-04-25T11:27:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7f84de000640 1 -- 172.21.15.19:0/2638519034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0019070 con 0x7f84e80c6e20 2024-04-25T11:27:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7f84de000640 1 -- 172.21.15.19:0/2638519034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0014650 con 0x7f84e80c6e20 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655407759 0 0) 0x7efff41619f0 con 0x7efff40ccf80 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.733+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effdc003520 con 0x7efff40ccf80 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3248781371 0 0) 0x7effdc003520 con 0x7efff40ccf80 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff4072070 msgr2=0x7efff415dfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff4072070 0x7efff415dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 >> v2:172.21.15.169:3300/0 conn(0x7efff41433d0 msgr2=0x7efff40d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 --2- 172.21.15.19:0/2785553659 >> v2:172.21.15.169:3300/0 conn(0x7efff41433d0 0x7efff40d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff4160f10 con 0x7efff40ccf80 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7effe8001670 con 0x7efff40ccf80 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 >> v1:172.21.15.169:6789/0 conn(0x7efff40ccf80 legacy=0x7efff4071960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc003a20 0x7effdc003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effdc004800 0x7effdc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc005220 0x7effdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efff40cd6b0 con 0x7effdc003a20 2024-04-25T11:27:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efff4160c30 con 0x7effdc005220 2024-04-25T11:27:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7effdc00c210 con 0x7effdc004800 2024-04-25T11:27:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7f96b2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96ac0d0e00 0x7f96ac0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96b2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96ac0d4a50 0x7f96ac0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f94ae200640 1 Processor -- start 2024-04-25T11:27:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96b2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac0ca4c0 0x7f96ac1f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96b2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac164e00 con 0x7f96ac0d0e00 2024-04-25T11:27:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96b2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0b5320 con 0x7f96ac0ca4c0 2024-04-25T11:27:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96b2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac164f80 con 0x7f96ac0d4a50 2024-04-25T11:27:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7fb59b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940545a0 0x7fb594180cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7fb59b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e3de0 0x7fb5940e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7fb59b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940e7e80 0x7fb59404ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7fb59b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5940e8290 con 0x7fb5940e3de0 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7fb59b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5940e4b00 con 0x7fb5940e7e80 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.734+0000 7fb59b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5940e41f0 con 0x7fb5940545a0 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7efff8c00640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc003a20 0x7effdc003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7efff3e00640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effdc004800 0x7effdc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7efff8c00640 1 -- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effdc004800 msgr2=0x7effdc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7efff8c00640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effdc004800 0x7effdc004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7efff8c00640 1 -- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc005220 msgr2=0x7effdc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7efff8c00640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc005220 0x7effdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7efff8c00640 1 -- 172.21.15.19:0/2785553659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff4160f10 con 0x7effdc003a20 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7fb599e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940545a0 0x7fb594180cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7fb599e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940545a0 0x7fb594180cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53222/0 (socket says 172.21.15.19:53222) 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7fb599e00640 1 -- 172.21.15.19:0/2921292816 learned_addr learned my addr 172.21.15.19:0/2921292816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96ab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96ac0d0e00 0x7f96ac0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96ab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96ac0d0e00 0x7f96ac0cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47384/0 (socket says 172.21.15.19:47384) 2024-04-25T11:27:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96ab400640 1 -- 172.21.15.19:0/2101219298 learned_addr learned my addr 172.21.15.19:0/2101219298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f96aaa00640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96ac0d4a50 0x7f96ac0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7f96abe00640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac0ca4c0 0x7f96ac1f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7fb599400640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e3de0 0x7fb5940e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c0005290 con 0x7f84e80c6e20 2024-04-25T11:27:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.735+0000 7f94ae200640 1 -- start start 2024-04-25T11:27:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7f96ab400640 1 -- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96ac0d4a50 msgr2=0x7f96ac0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7f96ab400640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96ac0d4a50 0x7f96ac0cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7f96ab400640 1 -- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac0ca4c0 msgr2=0x7f96ac1f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7f96ab400640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac0ca4c0 0x7f96ac1f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7f96ab400640 1 -- 172.21.15.19:0/2101219298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0c9700 con 0x7f96ac0d0e00 2024-04-25T11:27:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.736+0000 7fb59a800640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940e7e80 0x7fb59404ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7fb599400640 1 -- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940545a0 msgr2=0x7fb594180cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7fb599400640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940545a0 0x7fb594180cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7fb599400640 1 -- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940e7e80 msgr2=0x7fb59404ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7fb599400640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940e7e80 0x7fb59404ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7fb599400640 1 -- 172.21.15.19:0/2921292816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb594181200 con 0x7fb5940e3de0 2024-04-25T11:27:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7fb59a800640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940e7e80 0x7fb59404ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7f96ab400640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96ac0d0e00 0x7f96ac0cf910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f969c00c460 tx=0x7f969c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7efff8c00640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc003a20 0x7effdc003e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efff40760f0 tx=0x7effdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbef3a08fdf98c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7fb599400640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e3de0 0x7fb5940e5660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb594057330 tx=0x7fb58c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8007dc0 con 0x7effdc003a20 2024-04-25T11:27:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.738+0000 7fb592000640 1 -- 172.21.15.19:0/2921292816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c005220 con 0x7fb5940e3de0 2024-04-25T11:27:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.738+0000 7fb592000640 1 -- 172.21.15.19:0/2921292816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c0053c0 con 0x7fb5940e3de0 2024-04-25T11:27:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.738+0000 7fb592000640 1 -- 172.21.15.19:0/2921292816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c007270 con 0x7fb5940e3de0 2024-04-25T11:27:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe8005ad0 con 0x7effdc003a20 2024-04-25T11:27:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7efff3400640 1 -- 172.21.15.19:0/2785553659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8007070 con 0x7effdc003a20 2024-04-25T11:27:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.738+0000 7f94ae200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f94a8150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.738+0000 7f94ae200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81fb2b0 0x7f94a81fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7efffb600640 1 -- 172.21.15.19:0/2785553659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc003a20 msgr2=0x7effdc003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7efffb600640 1 --2- 172.21.15.19:0/2785553659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc003a20 0x7effdc003e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efff40760f0 tx=0x7effdc005830 comp rx=0 tx=0).stop 2024-04-25T11:27:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94ae200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81fd170 0x7f94a8152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7efff9600640 1 -- 172.21.15.19:0/2785553659 reap_dead start 2024-04-25T11:27:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f94a8150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f94a8150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47396/0 (socket says 172.21.15.19:47396) 2024-04-25T11:27:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94a7400640 1 -- 172.21.15.19:0/1986419995 learned_addr learned my addr 172.21.15.19:0/1986419995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7efffb600640 1 -- 172.21.15.19:0/2785553659 shutdown_connections 2024-04-25T11:27:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7efffb600640 1 -- 172.21.15.19:0/2785553659 >> 172.21.15.19:0/2785553659 conn(0x7efff40d2930 msgr2=0x7efff40d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7efffb600640 1 -- 172.21.15.19:0/2785553659 shutdown_connections 2024-04-25T11:27:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7f96a3400640 1 -- 172.21.15.19:0/2101219298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0052d0 con 0x7f96ac0d0e00 2024-04-25T11:27:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7f96a3400640 1 -- 172.21.15.19:0/2101219298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c005490 con 0x7f96ac0d0e00 2024-04-25T11:27:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7f96a3400640 1 -- 172.21.15.19:0/2101219298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c012870 con 0x7f96ac0d0e00 2024-04-25T11:27:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80cf620 con 0x7f948c005150 2024-04-25T11:27:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac0c99e0 con 0x7f96ac0d0e00 2024-04-25T11:27:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.737+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac1f15d0 con 0x7f96ac0d0e00 2024-04-25T11:27:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac050740 con 0x7f96ac0d0e00 2024-04-25T11:27:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80d1530 con 0x7f94a81fd170 2024-04-25T11:27:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7efffb600640 1 -- 172.21.15.19:0/2785553659 wait complete. 2024-04-25T11:27:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94a6a00640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81fb2b0 0x7f94a81fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94a80cf7a0 con 0x7f94a81fb2b0 2024-04-25T11:27:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59416d590 con 0x7fb5940e3de0 2024-04-25T11:27:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.739+0000 7f94a7e00640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81fd170 0x7f94a8152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.740+0000 7f84de000640 1 -- 172.21.15.19:0/2638519034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f84e00147f0 con 0x7f84e80c6e20 2024-04-25T11:27:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.740+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59416db50 con 0x7fb5940e3de0 2024-04-25T11:27:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.741+0000 7f94a6a00640 1 -- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81fd170 msgr2=0x7f94a8152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.741+0000 7f94a6a00640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81fd170 0x7f94a8152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.741+0000 7f94a6a00640 1 -- 172.21.15.19:0/1986419995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 msgr2=0x7f94a8150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.741+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb594005570 con 0x7fb5940e3de0 2024-04-25T11:27:33.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.741+0000 7f94a6a00640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f94a8150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.741+0000 7f94a6a00640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a8151480 con 0x7f94a81fb2b0 2024-04-25T11:27:33.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.742+0000 7f84de000640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84c80744b0 0x7f84c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.742+0000 7f84de000640 1 -- 172.21.15.19:0/2638519034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f84e00955b0 con 0x7f84e80c6e20 2024-04-25T11:27:33.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.742+0000 7f94a6a00640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81fb2b0 0x7f94a81fcc30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f94940017e0 tx=0x7f94940076d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.743+0000 7f84e5e00640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84c80744b0 0x7f84c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.743+0000 7f4951400640 1 -- 172.21.15.19:0/1666469153 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c007740 con 0x7f49540d1120 2024-04-25T11:27:33.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.743+0000 7f4951400640 1 -- 172.21.15.19:0/1666469153 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c0078e0 con 0x7f49540d1120 2024-04-25T11:27:33.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.743+0000 7f4951400640 1 -- 172.21.15.19:0/1666469153 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c007cf0 con 0x7f49540d1120 2024-04-25T11:27:33.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.743+0000 7f949f400640 1 -- 172.21.15.19:0/1986419995 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949400ea10 con 0x7f94a81fb2b0 2024-04-25T11:27:33.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.743+0000 7f85fcc00640 1 -- 172.21.15.19:0/2911403239 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85f805f6a0 con 0x7f860006c5c0 2024-04-25T11:27:33.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.744+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81570f0 con 0x7f94a81fb2b0 2024-04-25T11:27:33.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.744+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81576b0 con 0x7f94a81fb2b0 2024-04-25T11:27:33.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.744+0000 7f949f400640 1 -- 172.21.15.19:0/1986419995 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9494014070 con 0x7f94a81fb2b0 2024-04-25T11:27:33.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.744+0000 7f949f400640 1 -- 172.21.15.19:0/1986419995 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949400edd0 con 0x7f94a81fb2b0 2024-04-25T11:27:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.745+0000 7f4959400640 1 -- 172.21.15.19:0/1666469153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 msgr2=0x7f495415c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.745+0000 7f4959400640 1 --2- 172.21.15.19:0/1666469153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415c7b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f494c009d90 tx=0x7f494c02a5f0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.745+0000 7f4959400640 1 -- 172.21.15.19:0/1666469153 shutdown_connections 2024-04-25T11:27:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.745+0000 7f4959400640 1 --2- 172.21.15.19:0/1666469153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415c7b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.745+0000 7f4959400640 1 --2- 172.21.15.19:0/1666469153 >> v2:172.21.15.169:3300/0 conn(0x7f4954160510 0x7f49540707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.746+0000 7f4959400640 1 -- 172.21.15.19:0/1666469153 >> 172.21.15.19:0/1666469153 conn(0x7f49540d1ae0 msgr2=0x7f49540d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.746+0000 7f4959400640 1 -- 172.21.15.19:0/1666469153 shutdown_connections 2024-04-25T11:27:33.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.745+0000 7efffb600640 1 Processor -- start 2024-04-25T11:27:33.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.745+0000 7efffb600640 1 -- start start 2024-04-25T11:27:33.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.746+0000 7fb592000640 1 -- 172.21.15.19:0/2921292816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb58c019020 con 0x7fb5940e3de0 2024-04-25T11:27:33.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.747+0000 7f84e5e00640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84c80744b0 0x7f84c80769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d8037000 tx=0x7f84d8010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7efffb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc003a20 0x7efff406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7efffb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff40b77a0 0x7efff406e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7efffb600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40c9370 0x7efff40c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7efffb600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efff41437e0 con 0x7efff40c9370 2024-04-25T11:27:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7efffb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efff40cd390 con 0x7effdc003a20 2024-04-25T11:27:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7efffb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efff4072070 con 0x7efff40b77a0 2024-04-25T11:27:33.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9480005290 con 0x7f94a81fb2b0 2024-04-25T11:27:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7fb592000640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb57c0744b0 0x7fb57c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.748+0000 7efff8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc003a20 0x7efff406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.746+0000 7f4959400640 1 -- 172.21.15.19:0/1666469153 wait complete. 2024-04-25T11:27:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40c9370 0x7efff40c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40c9370 0x7efff40c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47408/0 (socket says 172.21.15.19:47408) 2024-04-25T11:27:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 -- 172.21.15.19:0/4199202716 learned_addr learned my addr 172.21.15.19:0/4199202716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.750+0000 7f84de000640 1 -- 172.21.15.19:0/2638519034 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84e80c7230 con 0x7f84e80c6e20 2024-04-25T11:27:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7f949f400640 1 -- 172.21.15.19:0/1986419995 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9494019470 con 0x7f94a81fb2b0 2024-04-25T11:27:33.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.750+0000 7f949f400640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.751+0000 7f949f400640 1 -- 172.21.15.19:0/1986419995 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9494094b90 con 0x7f94a81fb2b0 2024-04-25T11:27:33.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.750+0000 7f96a3400640 1 -- 172.21.15.19:0/2101219298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f969c012a10 con 0x7f96ac0d0e00 2024-04-25T11:27:33.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 -- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff40b77a0 msgr2=0x7efff406e070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:27:33.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff40b77a0 0x7efff406e070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 -- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc003a20 msgr2=0x7efff406db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc003a20 0x7efff406db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.749+0000 7efff9600640 1 -- 172.21.15.19:0/4199202716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff40b6b90 con 0x7efff40c9370 2024-04-25T11:27:33.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.750+0000 7efff9600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40c9370 0x7efff40c7830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efff406e790 tx=0x7effec00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.751+0000 7f4959400640 1 Processor -- start 2024-04-25T11:27:33.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.751+0000 7fb599e00640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb57c0744b0 0x7fb57c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.751+0000 7f4959400640 1 -- start start 2024-04-25T11:27:33.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.752+0000 7f94a7400640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.752+0000 7fb599e00640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb57c0744b0 0x7fb57c0769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb58802a250 tx=0x7fb58802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.753+0000 7f94a7400640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9498006a90 tx=0x7f9498009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.753+0000 7f96a3400640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f968c0744b0 0x7f968c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.753+0000 7f96a3400640 1 -- 172.21.15.19:0/2101219298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f969c095860 con 0x7f96ac0d0e00 2024-04-25T11:27:33.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.753+0000 7f96aaa00640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f968c0744b0 0x7f968c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.754+0000 7f4959400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49540d0780 0x7f495415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.754+0000 7f4959400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.754+0000 7f4959400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495415ab40 0x7f49541f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.754+0000 7f4952800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49540d0780 0x7f495415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.754+0000 7f96aaa00640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f968c0744b0 0x7f968c0769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9698034000 tx=0x7f969802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.755+0000 7f4959400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49540cf620 con 0x7f49540d0780 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.755+0000 7f4959400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f49540d1530 con 0x7f495415ab40 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.755+0000 7f4959400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f49540cf7a0 con 0x7f49540d1120 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.755+0000 7f4951e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.755+0000 7f4951e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:53234/0 (socket says 172.21.15.19:53234) 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.755+0000 7f4951e00640 1 -- 172.21.15.19:0/3751690853 learned_addr learned my addr 172.21.15.19:0/3751690853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.755+0000 7f4953200640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495415ab40 0x7f49541f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.756+0000 7f4953200640 1 -- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 msgr2=0x7f495415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.756+0000 7f4953200640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.756+0000 7f4953200640 1 -- 172.21.15.19:0/3751690853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49540d0780 msgr2=0x7f495415c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.756+0000 7f4953200640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49540d0780 0x7f495415c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.756+0000 7f4953200640 1 -- 172.21.15.19:0/3751690853 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495415b320 con 0x7f495415ab40 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.756+0000 7f4951e00640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.756+0000 7f4952800640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49540d0780 0x7f495415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.757+0000 7f4953200640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495415ab40 0x7f49541f69c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f494c031000 tx=0x7f494c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.758+0000 7f494aa00640 1 -- 172.21.15.19:0/3751690853 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c03f070 con 0x7f495415ab40 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.758+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495415b600 con 0x7f495415ab40 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.758+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495415bbc0 con 0x7f495415ab40 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.758+0000 7f494aa00640 1 -- 172.21.15.19:0/3751690853 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c031c80 con 0x7f495415ab40 2024-04-25T11:27:33.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.758+0000 7f494aa00640 1 -- 172.21.15.19:0/3751690853 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c009450 con 0x7f495415ab40 2024-04-25T11:27:33.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.759+0000 7fb592000640 1 -- 172.21.15.19:0/2921292816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb58c092940 con 0x7fb5940e3de0 2024-04-25T11:27:33.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.759+0000 7f96a3400640 1 -- 172.21.15.19:0/2101219298 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f969c060060 con 0x7f96ac0d0e00 2024-04-25T11:27:33.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.760+0000 7efff0c00640 1 -- 172.21.15.19:0/4199202716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec024070 con 0x7efff40c9370 2024-04-25T11:27:33.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.760+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff40b6e70 con 0x7efff40c9370 2024-04-25T11:27:33.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.760+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c005290 con 0x7f495415ab40 2024-04-25T11:27:33.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.760+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff40b7430 con 0x7efff40c9370 2024-04-25T11:27:33.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.762+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff40b6900 con 0x7efff40c9370 2024-04-25T11:27:33.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.763+0000 7f494aa00640 1 -- 172.21.15.19:0/3751690853 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f494c03b410 con 0x7f495415ab40 2024-04-25T11:27:33.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.765+0000 7f494aa00640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49340744e0 0x7f49340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.765+0000 7f494aa00640 1 -- 172.21.15.19:0/3751690853 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f494c0b6db0 con 0x7f495415ab40 2024-04-25T11:27:33.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.765+0000 7f4952800640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49340744e0 0x7f49340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.767+0000 7f4952800640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49340744e0 0x7f49340769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4944000f60 tx=0x7f4944005e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.770+0000 7efff0c00640 1 -- 172.21.15.19:0/4199202716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effec00bce0 con 0x7efff40c9370 2024-04-25T11:27:33.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.771+0000 7fb592000640 1 -- 172.21.15.19:0/2921292816 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb58c05eb20 con 0x7fb5940e3de0 2024-04-25T11:27:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.771+0000 7efff0c00640 1 -- 172.21.15.19:0/4199202716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec024070 con 0x7efff40c9370 2024-04-25T11:27:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.772+0000 7f949f400640 1 -- 172.21.15.19:0/1986419995 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f949405ecc0 con 0x7f94a81fb2b0 2024-04-25T11:27:33.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.774+0000 7efff0c00640 1 -- 172.21.15.19:0/4199202716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7effec027070 con 0x7efff40c9370 2024-04-25T11:27:33.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.776+0000 7efff0c00640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7effd4074650 0x7effd4076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:33.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.776+0000 7efff0c00640 1 -- 172.21.15.19:0/4199202716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7effec09dc90 con 0x7efff40c9370 2024-04-25T11:27:33.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.777+0000 7efff8c00640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7effd4074650 0x7effd4076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:33.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.778+0000 7efff8c00640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7effd4074650 0x7effd4076b60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effe8033000 tx=0x7effe8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:33.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.784+0000 7efff0c00640 1 -- 172.21.15.19:0/4199202716 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effec02e070 con 0x7efff40c9370 2024-04-25T11:27:33.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:33.791+0000 7f494aa00640 1 -- 172.21.15.19:0/3751690853 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f494c052020 con 0x7f495415ab40 2024-04-25T11:27:34.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.154+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6bcc005850 con 0x7f6bfc142650 2024-04-25T11:27:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.156+0000 7f6bf3e00640 1 -- 172.21.15.19:0/1536058744 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6bec05eac0 con 0x7f6bfc142650 2024-04-25T11:27:34.147 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-04-25T11:27:34.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.159+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6bd8074500 msgr2=0x7f6bd8076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.159+0000 7f6bf1600640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6bd8074500 0x7f6bd8076a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6be40016e0 tx=0x7f6be4009040 comp rx=0 tx=0).stop 2024-04-25T11:27:34.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc142650 msgr2=0x7f6bfc203010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc142650 0x7f6bfc203010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00b410 tx=0x7f6bec00b8d0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 shutdown_connections 2024-04-25T11:27:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6bd8074500 0x7f6bd8076a10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bfc168600 0x7f6bfc168a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6bfc142650 0x7f6bfc203010 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 --2- 172.21.15.19:0/1536058744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc0b6a50 0x7f6bfc0d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.160+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 >> 172.21.15.19:0/1536058744 conn(0x7f6bfc073f20 msgr2=0x7f6bfc155b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.161+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 shutdown_connections 2024-04-25T11:27:34.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.161+0000 7f6bf1600640 1 -- 172.21.15.19:0/1536058744 wait complete. 2024-04-25T11:27:34.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.177+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f84c0005580 con 0x7f84e80c6e20 2024-04-25T11:27:34.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.177+0000 7f84de000640 1 -- 172.21.15.19:0/2638519034 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f84e005f6f0 con 0x7f84e80c6e20 2024-04-25T11:27:34.169 INFO:teuthology.orchestra.run.smithi019.stdout:64424509445 2024-04-25T11:27:34.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.181+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84c80744b0 msgr2=0x7f84c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.181+0000 7f84ed000640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84c80744b0 0x7f84c80769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84d8037000 tx=0x7f84d8010040 comp rx=0 tx=0).stop 2024-04-25T11:27:34.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.181+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e80c6e20 msgr2=0x7f84e815a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.181+0000 7f84ed000640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e80c6e20 0x7f84e815a4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f84cc005060 tx=0x7f84e0007700 comp rx=0 tx=0).stop 2024-04-25T11:27:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 shutdown_connections 2024-04-25T11:27:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84c80744b0 0x7f84c80769c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e8157f20 0x7f84e80c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e80c6e20 0x7f84e815a4c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 --2- 172.21.15.19:0/2638519034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84cc0052a0 0x7f84e8154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 >> 172.21.15.19:0/2638519034 conn(0x7f84e8073050 msgr2=0x7f84e80b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 shutdown_connections 2024-04-25T11:27:34.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.185+0000 7f84ed000640 1 -- 172.21.15.19:0/2638519034 wait complete. 2024-04-25T11:27:34.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.186+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f85d8005850 con 0x7f860006c5c0 2024-04-25T11:27:34.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.187+0000 7f85fcc00640 1 -- 172.21.15.19:0/2911403239 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f85f80639a0 con 0x7f860006c5c0 2024-04-25T11:27:34.178 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-04-25T11:27:34.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.191+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85e00745b0 msgr2=0x7f85e0076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.191+0000 7f8607400640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85e00745b0 0x7f85e0076ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85f0034000 tx=0x7f85f002f000 comp rx=0 tx=0).stop 2024-04-25T11:27:34.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.191+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f860006c5c0 msgr2=0x7f86000cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.191+0000 7f8607400640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f860006c5c0 0x7f86000cc710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c820 tx=0x7f85f800cce0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.192+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 shutdown_connections 2024-04-25T11:27:34.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.192+0000 7f8607400640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85e00745b0 0x7f85e0076ac0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.192+0000 7f8607400640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86000b7810 0x7f860015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.192+0000 7f8607400640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86000d0e00 0x7f8600207080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.192+0000 7f8607400640 1 --2- 172.21.15.19:0/2911403239 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f860006c5c0 0x7f86000cc710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.192+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 >> 172.21.15.19:0/2911403239 conn(0x7f86000d5d90 msgr2=0x7f86000b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.193+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 shutdown_connections 2024-04-25T11:27:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.193+0000 7f8607400640 1 -- 172.21.15.19:0/2911403239 wait complete. 2024-04-25T11:27:34.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.1 2024-04-25T11:27:34.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.210+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f96ac073ad0 con 0x7f96ac0d0e00 2024-04-25T11:27:34.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.211+0000 7f96a3400640 1 -- 172.21.15.19:0/2101219298 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f969c05f960 con 0x7f96ac0d0e00 2024-04-25T11:27:34.202 INFO:teuthology.orchestra.run.smithi019.stdout:64424509445 2024-04-25T11:27:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f968c0744b0 msgr2=0x7f968c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f968c0744b0 0x7f968c0769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9698034000 tx=0x7f969802e000 comp rx=0 tx=0).stop 2024-04-25T11:27:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96ac0d0e00 msgr2=0x7f96ac0cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96ac0d0e00 0x7f96ac0cf910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f969c00c460 tx=0x7f969c00c920 comp rx=0 tx=0).stop 2024-04-25T11:27:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 shutdown_connections 2024-04-25T11:27:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f968c0744b0 0x7f968c0769c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96ac0ca4c0 0x7f96ac1f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96ac0d4a50 0x7f96ac0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 --2- 172.21.15.19:0/2101219298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96ac0d0e00 0x7f96ac0cf910 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f969c00c460 tx=0x7f969c00c920 comp rx=0 tx=0).stop 2024-04-25T11:27:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.214+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 >> 172.21.15.19:0/2101219298 conn(0x7f96ac0d5d90 msgr2=0x7f96ac1641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.215+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 shutdown_connections 2024-04-25T11:27:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.215+0000 7f96b2800640 1 -- 172.21.15.19:0/2101219298 wait complete. 2024-04-25T11:27:34.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.0 2024-04-25T11:27:34.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.218+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb59416d320 con 0x7fb5940e3de0 2024-04-25T11:27:34.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.218+0000 7fb592000640 1 -- 172.21.15.19:0/2921292816 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb58c05e420 con 0x7fb5940e3de0 2024-04-25T11:27:34.210 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb57c0744b0 msgr2=0x7fb57c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb57c0744b0 0x7fb57c0769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb58802a250 tx=0x7fb58802e000 comp rx=0 tx=0).stop 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e3de0 msgr2=0x7fb5940e5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e3de0 0x7fb5940e5660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb594057330 tx=0x7fb58c00c820 comp rx=0 tx=0).stop 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 shutdown_connections 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb57c0744b0 0x7fb57c0769c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5940e7e80 0x7fb59404ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5940e3de0 0x7fb5940e5660 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 --2- 172.21.15.19:0/2921292816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5940545a0 0x7fb594180cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 >> 172.21.15.19:0/2921292816 conn(0x7fb594059120 msgr2=0x7fb5940e64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 shutdown_connections 2024-04-25T11:27:34.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.222+0000 7fb59b200640 1 -- 172.21.15.19:0/2921292816 wait complete. 2024-04-25T11:27:34.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.225+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9480005580 con 0x7f94a81fb2b0 2024-04-25T11:27:34.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.226+0000 7f949f400640 1 -- 172.21.15.19:0/1986419995 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9494067300 con 0x7f94a81fb2b0 2024-04-25T11:27:34.218 INFO:teuthology.orchestra.run.smithi019.stdout:64424509445 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.229+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 msgr2=0x7f94880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.229+0000 7f94ae200640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9498006a90 tx=0x7f9498009190 comp rx=0 tx=0).stop 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.229+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81fb2b0 msgr2=0x7f94a81fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.229+0000 7f94ae200640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81fb2b0 0x7f94a81fcc30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f94940017e0 tx=0x7f94940076d0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 shutdown_connections 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81fd170 0x7f94a8152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81fb2b0 0x7f94a81fcc30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 --2- 172.21.15.19:0/1986419995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948c005150 0x7f94a8150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 >> 172.21.15.19:0/1986419995 conn(0x7f94a80d1ae0 msgr2=0x7f94a80b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 shutdown_connections 2024-04-25T11:27:34.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.230+0000 7f94ae200640 1 -- 172.21.15.19:0/1986419995 wait complete. 2024-04-25T11:27:34.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.233+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f492c005850 con 0x7f495415ab40 2024-04-25T11:27:34.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.234+0000 7f494aa00640 1 -- 172.21.15.19:0/3751690853 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f494c081110 con 0x7f495415ab40 2024-04-25T11:27:34.226 INFO:teuthology.orchestra.run.smithi019.stdout:60129542149 2024-04-25T11:27:34.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.236+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7efff40507b0 con 0x7efff40c9370 2024-04-25T11:27:34.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.237+0000 7efff0c00640 1 -- 172.21.15.19:0/4199202716 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7effec067dc0 con 0x7efff40c9370 2024-04-25T11:27:34.228 INFO:teuthology.orchestra.run.smithi019.stdout:64424509444 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.237+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49340744e0 msgr2=0x7f49340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.237+0000 7f4959400640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49340744e0 0x7f49340769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4944000f60 tx=0x7f4944005e40 comp rx=0 tx=0).stop 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.237+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495415ab40 msgr2=0x7f49541f69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.237+0000 7f4959400640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495415ab40 0x7f49541f69c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f494c031000 tx=0x7f494c004a40 comp rx=0 tx=0).stop 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 shutdown_connections 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49340744e0 0x7f49340769f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f495415ab40 0x7f49541f69c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49540d1120 0x7f495415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 --2- 172.21.15.19:0/3751690853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49540d0780 0x7f495415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 >> 172.21.15.19:0/3751690853 conn(0x7f49540d1ae0 msgr2=0x7f495406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 shutdown_connections 2024-04-25T11:27:34.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.238+0000 7f4959400640 1 -- 172.21.15.19:0/3751690853 wait complete. 2024-04-25T11:27:34.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.2 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7effd4074650 msgr2=0x7effd4076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7effd4074650 0x7effd4076b60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7effe8033000 tx=0x7effe8009030 comp rx=0 tx=0).stop 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40c9370 msgr2=0x7efff40c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40c9370 0x7efff40c7830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efff406e790 tx=0x7effec00b480 comp rx=0 tx=0).stop 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 shutdown_connections 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7effd4074650 0x7effd4076b60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efff40c9370 0x7efff40c7830 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efff40b77a0 0x7efff406e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 --2- 172.21.15.19:0/4199202716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7effdc003a20 0x7efff406db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:34.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 >> 172.21.15.19:0/4199202716 conn(0x7efff40d2930 msgr2=0x7efff415fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:34.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 shutdown_connections 2024-04-25T11:27:34.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:34.240+0000 7efffb600640 1 -- 172.21.15.19:0/4199202716 wait complete. 2024-04-25T11:27:34.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.3 2024-04-25T11:27:34.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.4 2024-04-25T11:27:34.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.6 2024-04-25T11:27:34.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.7 2024-04-25T11:27:34.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-04-25T11:27:35.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:27:35.209 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:27:35.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:27:35.243 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:27:35.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:27:35.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:27:35.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:27:35.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:27:35.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.577+0000 7fd50d400640 1 Processor -- start 2024-04-25T11:27:35.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.577+0000 7fd50d400640 1 -- start start 2024-04-25T11:27:35.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.581+0000 7fd50d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5000e3a80 0x7fd500057bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.582+0000 7fd50d400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd500058b80 0x7fd5000380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.582+0000 7fd50d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000073c0 0x7fd5000077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.583+0000 7fd50d400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd500057720 con 0x7fd5000e3a80 2024-04-25T11:27:35.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.583+0000 7fd50d400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd500038610 con 0x7fd500058b80 2024-04-25T11:27:35.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.583+0000 7fd50d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd500007d10 con 0x7fd5000073c0 2024-04-25T11:27:35.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.584+0000 7fd507200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000073c0 0x7fd5000077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.583+0000 7fd505e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd500058b80 0x7fd5000380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.584+0000 7fd505e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd500058b80 0x7fd5000380d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:37490/0 (socket says 172.21.15.19:37490) 2024-04-25T11:27:35.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.584+0000 7fd505e00640 1 -- 172.21.15.19:0/3516694166 learned_addr learned my addr 172.21.15.19:0/3516694166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.584+0000 7fd506800640 1 --2- 172.21.15.19:0/3516694166 >> v2:172.21.15.19:3300/0 conn(0x7fd5000e3a80 0x7fd500057bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.585+0000 7fd506800640 1 -- 172.21.15.19:0/3516694166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000073c0 msgr2=0x7fd5000077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.586+0000 7fd506800640 1 --2- 172.21.15.19:0/3516694166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000073c0 0x7fd5000077d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.586+0000 7fd506800640 1 -- 172.21.15.19:0/3516694166 >> v2:172.21.15.169:3300/0 conn(0x7fd500058b80 msgr2=0x7fd5000380d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.586+0000 7fd506800640 1 --2- 172.21.15.19:0/3516694166 >> v2:172.21.15.169:3300/0 conn(0x7fd500058b80 0x7fd5000380d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.586+0000 7fd506800640 1 -- 172.21.15.19:0/3516694166 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd500056df0 con 0x7fd5000e3a80 2024-04-25T11:27:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.587+0000 7fd507200640 1 --2- 172.21.15.19:0/3516694166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000073c0 0x7fd5000077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.587+0000 7fd506800640 1 --2- 172.21.15.19:0/3516694166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000e3a80 0x7fd500057bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd508000b60 tx=0x7fd50806a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774c4c468717e27e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.588+0000 7fd505400640 1 -- 172.21.15.19:0/3516694166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080b4070 con 0x7fd5000e3a80 2024-04-25T11:27:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.589+0000 7fd505400640 1 -- 172.21.15.19:0/3516694166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5080a7d40 con 0x7fd5000e3a80 2024-04-25T11:27:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.589+0000 7fd505400640 1 -- 172.21.15.19:0/3516694166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080af640 con 0x7fd5000e3a80 2024-04-25T11:27:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.590+0000 7fd50d400640 1 -- 172.21.15.19:0/3516694166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000e3a80 msgr2=0x7fd500057bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.590+0000 7fd50d400640 1 --2- 172.21.15.19:0/3516694166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000e3a80 0x7fd500057bb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd508000b60 tx=0x7fd50806a5c0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.591+0000 7fd50d400640 1 -- 172.21.15.19:0/3516694166 shutdown_connections 2024-04-25T11:27:35.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.591+0000 7fd50d400640 1 --2- 172.21.15.19:0/3516694166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000073c0 0x7fd5000077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.592+0000 7fd50d400640 1 --2- 172.21.15.19:0/3516694166 >> v2:172.21.15.169:3300/0 conn(0x7fd500058b80 0x7fd5000380d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.592+0000 7fd50d400640 1 --2- 172.21.15.19:0/3516694166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000e3a80 0x7fd500057bb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.592+0000 7fd50d400640 1 -- 172.21.15.19:0/3516694166 >> 172.21.15.19:0/3516694166 conn(0x7fd50003abe0 msgr2=0x7fd50003aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.592+0000 7fd50d400640 1 -- 172.21.15.19:0/3516694166 shutdown_connections 2024-04-25T11:27:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.592+0000 7fd50d400640 1 -- 172.21.15.19:0/3516694166 wait complete. 2024-04-25T11:27:35.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.597+0000 7fd50d400640 1 Processor -- start 2024-04-25T11:27:35.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.597+0000 7fd50d400640 1 -- start start 2024-04-25T11:27:35.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.600+0000 7fd50d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000073c0 0x7fd50017b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.601+0000 7fd506800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000073c0 0x7fd50017b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.602+0000 7fd506800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000073c0 0x7fd50017b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49014/0 (socket says 172.21.15.19:49014) 2024-04-25T11:27:35.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.602+0000 7fd506800640 1 -- 172.21.15.19:0/3829713159 learned_addr learned my addr 172.21.15.19:0/3829713159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.602+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500058b80 0x7fd500052950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.602+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e3a80 0x7fd5001817f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.602+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd500037d60 con 0x7fd5000073c0 2024-04-25T11:27:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.603+0000 7fd505e00640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500058b80 0x7fd500052950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.603+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5000573b0 con 0x7fd500058b80 2024-04-25T11:27:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.603+0000 7fd507200640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e3a80 0x7fd5001817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.603+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd500053f30 con 0x7fd5000e3a80 2024-04-25T11:27:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.605+0000 7fd505e00640 1 -- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e3a80 msgr2=0x7fd5001817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.605+0000 7fd505e00640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e3a80 0x7fd5001817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.605+0000 7fd505e00640 1 -- 172.21.15.19:0/3829713159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000073c0 msgr2=0x7fd50017b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.605+0000 7fd505e00640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000073c0 0x7fd50017b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.605+0000 7fd505e00640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5000e3e90 con 0x7fd500058b80 2024-04-25T11:27:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.606+0000 7fd505e00640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500058b80 0x7fd500052950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800e870 tx=0x7fd4f800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.607+0000 7fd4fea00640 1 -- 172.21.15.19:0/3829713159 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8017070 con 0x7fd500058b80 2024-04-25T11:27:35.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.607+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5000e4390 con 0x7fd500058b80 2024-04-25T11:27:35.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.607+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5000e4950 con 0x7fd500058b80 2024-04-25T11:27:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.608+0000 7fd4fea00640 1 -- 172.21.15.19:0/3829713159 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f800ee60 con 0x7fd500058b80 2024-04-25T11:27:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.608+0000 7fd4fea00640 1 -- 172.21.15.19:0/3829713159 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8017070 con 0x7fd500058b80 2024-04-25T11:27:35.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.611+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e0005290 con 0x7fd500058b80 2024-04-25T11:27:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.614+0000 7fd4fea00640 1 -- 172.21.15.19:0/3829713159 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4f801a070 con 0x7fd500058b80 2024-04-25T11:27:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.615+0000 7fd4fea00640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4e80746e0 0x7fd4e8076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.615+0000 7fd4fea00640 1 -- 172.21.15.19:0/3829713159 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd4f8094bf0 con 0x7fd500058b80 2024-04-25T11:27:35.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.617+0000 7fd506800640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4e80746e0 0x7fd4e8076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.618+0000 7fd506800640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4e80746e0 0x7fd4e8076bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd508051c90 tx=0x7fd5080ab000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.630+0000 7f141fe00640 1 Processor -- start 2024-04-25T11:27:35.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.631+0000 7fd4fea00640 1 -- 172.21.15.19:0/3829713159 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4f805ec90 con 0x7fd500058b80 2024-04-25T11:27:35.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.630+0000 7f141fe00640 1 -- start start 2024-04-25T11:27:35.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.633+0000 7efc03e00640 1 Processor -- start 2024-04-25T11:27:35.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.634+0000 7f141fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14200b4ff0 0x7f14200b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.634+0000 7f141fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200d21c0 0x7f14200cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.634+0000 7f141fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14200cf7f0 0x7f142008cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.635+0000 7f141ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14200b4ff0 0x7f14200b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.635+0000 7f141ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14200b4ff0 0x7f14200b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:37514/0 (socket says 172.21.15.19:37514) 2024-04-25T11:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.635+0000 7f141e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200d21c0 0x7f14200cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.635+0000 7f141e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200d21c0 0x7f14200cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48184/0 (socket says 172.21.15.19:48184) 2024-04-25T11:27:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.635+0000 7f141ea00640 1 -- 172.21.15.19:0/3462839619 learned_addr learned my addr 172.21.15.19:0/3462839619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.636+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f14200d26f0 con 0x7f14200cf7f0 2024-04-25T11:27:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.636+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f14200ce7d0 con 0x7f14200b4ff0 2024-04-25T11:27:35.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.636+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14200ceab0 con 0x7f14200d21c0 2024-04-25T11:27:35.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.637+0000 7f141f400640 1 --2- 172.21.15.19:0/3462839619 >> v2:172.21.15.19:3300/0 conn(0x7f14200cf7f0 0x7f142008cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.634+0000 7efc03e00640 1 -- start start 2024-04-25T11:27:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7f141e000640 1 -- 172.21.15.19:0/3462839619 >> v2:172.21.15.169:3300/0 conn(0x7f14200b4ff0 msgr2=0x7f14200b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7f141e000640 1 --2- 172.21.15.19:0/3462839619 >> v2:172.21.15.169:3300/0 conn(0x7f14200b4ff0 0x7f14200b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7efc03e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc047db0 0x7efbfc0e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7f141e000640 1 -- 172.21.15.19:0/3462839619 >> v2:172.21.15.19:3300/0 conn(0x7f14200cf7f0 msgr2=0x7f142008cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7f355ec00640 1 Processor -- start 2024-04-25T11:27:35.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7f141e000640 1 --2- 172.21.15.19:0/3462839619 >> v2:172.21.15.19:3300/0 conn(0x7f14200cf7f0 0x7f142008cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7f141e000640 1 -- 172.21.15.19:0/3462839619 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14200ced90 con 0x7f14200d21c0 2024-04-25T11:27:35.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7efc03e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efbfc048300 0x7efbfc033950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.639+0000 7f355ec00640 1 -- start start 2024-04-25T11:27:35.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.639+0000 7efc03e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbfc006130 con 0x7efbfc033e90 2024-04-25T11:27:35.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.639+0000 7efc02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc047db0 0x7efbfc0e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.639+0000 7efc02a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc047db0 0x7efbfc0e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48196/0 (socket says 172.21.15.19:48196) 2024-04-25T11:27:35.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.639+0000 7efc02a00640 1 -- 172.21.15.19:0/540876418 learned_addr learned my addr 172.21.15.19:0/540876418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.639+0000 7efc03e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efbfc04f690 con 0x7efbfc048300 2024-04-25T11:27:35.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.638+0000 7f141f400640 1 --2- 172.21.15.19:0/3462839619 >> v2:172.21.15.19:3300/0 conn(0x7f14200cf7f0 0x7f142008cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.640+0000 7efc03e00640 1 -- 172.21.15.19:0/540876418 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efbfc0342a0 con 0x7efbfc047db0 2024-04-25T11:27:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.640+0000 7efc02000640 1 --2- 172.21.15.19:0/540876418 >> v2:172.21.15.19:3300/0 conn(0x7efbfc048300 0x7efbfc033950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.641+0000 7efc01600640 1 -- 172.21.15.19:0/540876418 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622551406 0 0) 0x7efbfc006130 con 0x7efbfc033e90 2024-04-25T11:27:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.641+0000 7efc01600640 1 -- 172.21.15.19:0/540876418 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbe8003520 con 0x7efbfc033e90 2024-04-25T11:27:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.641+0000 7efc02a00640 1 -- 172.21.15.19:0/540876418 >> v2:172.21.15.19:3300/0 conn(0x7efbfc048300 msgr2=0x7efbfc033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7f355ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35500dfd60 0x7f355004f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7f355ec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3550054420 0x7f3550050ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7efc02a00640 1 --2- 172.21.15.19:0/540876418 >> v2:172.21.15.19:3300/0 conn(0x7efbfc048300 0x7efbfc033950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7efc02a00640 1 -- 172.21.15.19:0/540876418 >> v1:172.21.15.19:6789/0 conn(0x7efbfc033e90 legacy=0x7efbfc0531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7efc02a00640 1 -- 172.21.15.19:0/540876418 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc04ffa0 con 0x7efbfc047db0 2024-04-25T11:27:35.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7f355ec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35500e0170 con 0x7f3550050890 2024-04-25T11:27:35.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7f355ec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3550052960 con 0x7f3550054420 2024-04-25T11:27:35.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7f355ec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f35500069c0 con 0x7f35500dfd60 2024-04-25T11:27:35.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.642+0000 7efc02000640 1 --2- 172.21.15.19:0/540876418 >> v2:172.21.15.19:3300/0 conn(0x7efbfc048300 0x7efbfc033950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:35.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.643+0000 7f355ce00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3550050890 0x7f3550053d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54286/0 (socket says 172.21.15.19:54286) 2024-04-25T11:27:35.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.643+0000 7f355ce00640 1 -- 172.21.15.19:0/3336687494 learned_addr learned my addr 172.21.15.19:0/3336687494 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:35.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.643+0000 7f355d800640 1 --2- 172.21.15.19:0/3336687494 >> v2:172.21.15.169:3300/0 conn(0x7f35500dfd60 0x7f355004f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.643+0000 7f141e000640 1 --2- 172.21.15.19:0/3462839619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200d21c0 0x7f14200cf2b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f142006d5a0 tx=0x7f140c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d879bd62df2d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.643+0000 7f355e200640 1 --2- 172.21.15.19:0/3336687494 >> v2:172.21.15.19:3300/0 conn(0x7f3550054420 0x7f3550050ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f355d800640 1 -- 172.21.15.19:0/3336687494 >> v2:172.21.15.19:3300/0 conn(0x7f3550054420 msgr2=0x7f3550050ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f3557e00640 1 -- 172.21.15.19:0/3336687494 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581414960 0 0) 0x7f35500e0170 con 0x7f3550050890 2024-04-25T11:27:35.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f141d600640 1 -- 172.21.15.19:0/3462839619 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c037070 con 0x7f14200d21c0 2024-04-25T11:27:35.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f141d600640 1 -- 172.21.15.19:0/3462839619 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f140c032440 con 0x7f14200d21c0 2024-04-25T11:27:35.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f141d600640 1 -- 172.21.15.19:0/3462839619 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c02ae60 con 0x7f14200d21c0 2024-04-25T11:27:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7efc02a00640 1 --2- 172.21.15.19:0/540876418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc047db0 0x7efbfc0e7d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efc0406b8e0 tx=0x7efc040695d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6019718ecc786982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.645+0000 7efc01600640 1 -- 172.21.15.19:0/540876418 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc040b4070 con 0x7efbfc047db0 2024-04-25T11:27:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.645+0000 7efc01600640 1 -- 172.21.15.19:0/540876418 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc040a7e10 con 0x7efbfc047db0 2024-04-25T11:27:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f355d800640 1 --2- 172.21.15.19:0/3336687494 >> v2:172.21.15.19:3300/0 conn(0x7f3550054420 0x7f3550050ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f355d800640 1 -- 172.21.15.19:0/3336687494 >> v1:172.21.15.19:6789/0 conn(0x7f3550050890 legacy=0x7f3550053d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f355d800640 1 -- 172.21.15.19:0/3336687494 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3550006ca0 con 0x7f35500dfd60 2024-04-25T11:27:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.644+0000 7f355e200640 1 --2- 172.21.15.19:0/3336687494 >> v2:172.21.15.19:3300/0 conn(0x7f3550054420 0x7f3550050ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7efc01600640 1 -- 172.21.15.19:0/540876418 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc040af640 con 0x7efbfc047db0 2024-04-25T11:27:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f355d800640 1 --2- 172.21.15.19:0/3336687494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35500dfd60 0x7f355004f340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f355000d110 tx=0x7f354c02b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6310c486379c098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200d21c0 msgr2=0x7f14200cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 --2- 172.21.15.19:0/3462839619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200d21c0 0x7f14200cf2b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f142006d5a0 tx=0x7f140c02a110 comp rx=0 tx=0).stop 2024-04-25T11:27:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7efc03e00640 1 -- 172.21.15.19:0/540876418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc047db0 msgr2=0x7efbfc0e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7efc03e00640 1 --2- 172.21.15.19:0/540876418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc047db0 0x7efbfc0e7d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efc0406b8e0 tx=0x7efc040695d0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 shutdown_connections 2024-04-25T11:27:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 --2- 172.21.15.19:0/3462839619 >> v2:172.21.15.19:3300/0 conn(0x7f14200cf7f0 0x7f142008cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 --2- 172.21.15.19:0/3462839619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200d21c0 0x7f14200cf2b0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f142006d5a0 tx=0x7f140c02a110 comp rx=0 tx=0).stop 2024-04-25T11:27:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 --2- 172.21.15.19:0/3462839619 >> v2:172.21.15.169:3300/0 conn(0x7f14200b4ff0 0x7f14200b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 >> 172.21.15.19:0/3462839619 conn(0x7f14200b5460 msgr2=0x7f14200b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f3557e00640 1 -- 172.21.15.19:0/3336687494 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c00b040 con 0x7f35500dfd60 2024-04-25T11:27:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f3557e00640 1 -- 172.21.15.19:0/3336687494 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354c030710 con 0x7f35500dfd60 2024-04-25T11:27:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7efc03e00640 1 -- 172.21.15.19:0/540876418 shutdown_connections 2024-04-25T11:27:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7efc03e00640 1 --2- 172.21.15.19:0/540876418 >> v2:172.21.15.19:3300/0 conn(0x7efbfc048300 0x7efbfc033950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7efc03e00640 1 --2- 172.21.15.19:0/540876418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc047db0 0x7efbfc0e7d50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7efc03e00640 1 -- 172.21.15.19:0/540876418 >> 172.21.15.19:0/540876418 conn(0x7efbfc054cf0 msgr2=0x7efbfc055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.647+0000 7efc03e00640 1 -- 172.21.15.19:0/540876418 shutdown_connections 2024-04-25T11:27:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.648+0000 7f355ec00640 1 -- 172.21.15.19:0/3336687494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35500dfd60 msgr2=0x7f355004f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.648+0000 7f355ec00640 1 --2- 172.21.15.19:0/3336687494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35500dfd60 0x7f355004f340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f355000d110 tx=0x7f354c02b560 comp rx=0 tx=0).stop 2024-04-25T11:27:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 shutdown_connections 2024-04-25T11:27:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.646+0000 7f141fe00640 1 -- 172.21.15.19:0/3462839619 wait complete. 2024-04-25T11:27:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.649+0000 7f355ec00640 1 -- 172.21.15.19:0/3336687494 shutdown_connections 2024-04-25T11:27:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.649+0000 7f355ec00640 1 --2- 172.21.15.19:0/3336687494 >> v2:172.21.15.19:3300/0 conn(0x7f3550054420 0x7f3550050ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.649+0000 7f355ec00640 1 --2- 172.21.15.19:0/3336687494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35500dfd60 0x7f355004f340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.649+0000 7f355ec00640 1 -- 172.21.15.19:0/3336687494 >> 172.21.15.19:0/3336687494 conn(0x7f3550054cf0 msgr2=0x7f3550055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.647+0000 7efc03e00640 1 -- 172.21.15.19:0/540876418 wait complete. 2024-04-25T11:27:35.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.651+0000 7f355ec00640 1 -- 172.21.15.19:0/3336687494 shutdown_connections 2024-04-25T11:27:35.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.651+0000 7efc03e00640 1 Processor -- start 2024-04-25T11:27:35.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.652+0000 7f141fe00640 1 Processor -- start 2024-04-25T11:27:35.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.652+0000 7f141fe00640 1 -- start start 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.651+0000 7f355ec00640 1 -- 172.21.15.19:0/3336687494 wait complete. 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7f355ec00640 1 Processor -- start 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7f355ec00640 1 -- start start 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.652+0000 7efc03e00640 1 -- start start 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7efc03e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbfc033e90 0x7efbfc049120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7efc03e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc047db0 0x7efbfc04a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7efc03e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc048300 0x7efbfc04a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7efc03e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc050840 con 0x7efbfc047db0 2024-04-25T11:27:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7efc03e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc050560 con 0x7efbfc033e90 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7efc03e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efbfc0509c0 con 0x7efbfc048300 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.655+0000 7efc02000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc047db0 0x7efbfc04a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.656+0000 7efc02000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc047db0 0x7efbfc04a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49060/0 (socket says 172.21.15.19:49060) 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.656+0000 7efc02000640 1 -- 172.21.15.19:0/623065700 learned_addr learned my addr 172.21.15.19:0/623065700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.656+0000 7efc02a00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbfc033e90 0x7efbfc049120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.657+0000 7efc02000640 1 -- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc048300 msgr2=0x7efbfc04a860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.657+0000 7efc02000640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc048300 0x7efbfc04a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.657+0000 7efc02000640 1 -- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbfc033e90 msgr2=0x7efbfc049120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.657+0000 7efc02000640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbfc033e90 0x7efbfc049120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.657+0000 7efc02000640 1 -- 172.21.15.19:0/623065700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc049660 con 0x7efbfc047db0 2024-04-25T11:27:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.658+0000 7efc02000640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc047db0 0x7efbfc04a320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e8e0 tx=0x7efbf400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7f355ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3550050890 0x7f355002b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7efbfaa00640 1 -- 172.21.15.19:0/623065700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4017070 con 0x7efbfc047db0 2024-04-25T11:27:35.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7f355ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35500dfd60 0x7f3550185870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7efbfaa00640 1 -- 172.21.15.19:0/623065700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf400c940 con 0x7efbfc047db0 2024-04-25T11:27:35.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc04bf10 con 0x7efbfc047db0 2024-04-25T11:27:35.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7f355ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3550185db0 0x7f355004ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7f355ec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35500549b0 con 0x7f35500dfd60 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.660+0000 7f355d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3550050890 0x7f355002b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.660+0000 7f355d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3550050890 0x7f355002b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:37558/0 (socket says 172.21.15.19:37558) 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.660+0000 7f355d800640 1 -- 172.21.15.19:0/2198971852 learned_addr learned my addr 172.21.15.19:0/2198971852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.660+0000 7f355ec00640 1 -- 172.21.15.19:0/2198971852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3550007260 con 0x7f3550050890 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.660+0000 7f355ec00640 1 -- 172.21.15.19:0/2198971852 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f355004f880 con 0x7f3550185db0 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.660+0000 7f355e200640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3550185db0 0x7f355004ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.660+0000 7f355ce00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35500dfd60 0x7f3550185870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.659+0000 7efbfaa00640 1 -- 172.21.15.19:0/623065700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf400cd10 con 0x7efbfc047db0 2024-04-25T11:27:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.661+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc04c4d0 con 0x7efbfc047db0 2024-04-25T11:27:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.661+0000 7f355d800640 1 -- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3550185db0 msgr2=0x7f355004ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.661+0000 7f355d800640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3550185db0 0x7f355004ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.661+0000 7f355d800640 1 -- 172.21.15.19:0/2198971852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35500dfd60 msgr2=0x7f3550185870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.661+0000 7f355d800640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35500dfd60 0x7f3550185870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.661+0000 7f355d800640 1 -- 172.21.15.19:0/2198971852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355002bc00 con 0x7f3550050890 2024-04-25T11:27:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.663+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc005570 con 0x7efbfc047db0 2024-04-25T11:27:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.663+0000 7f141fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200b4ff0 0x7f14201fda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.664+0000 7f355d800640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3550050890 0x7f355002b6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f354c000c00 tx=0x7f354c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.664+0000 7f141fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14200cf7f0 0x7f14200cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.664+0000 7f141fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14200cb650 0x7f14201609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.664+0000 7f141fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14200d29d0 con 0x7f14200cb650 2024-04-25T11:27:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.665+0000 7f141fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14200cfc00 con 0x7f14200cf7f0 2024-04-25T11:27:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.665+0000 7f141ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200b4ff0 0x7f14201fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.665+0000 7f141ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200b4ff0 0x7f14201fda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48232/0 (socket says 172.21.15.19:48232) 2024-04-25T11:27:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.665+0000 7f141ea00640 1 -- 172.21.15.19:0/4203263929 learned_addr learned my addr 172.21.15.19:0/4203263929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.665+0000 7f141f400640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14200cb650 0x7f14201609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.665+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14200d2b50 con 0x7f14200b4ff0 2024-04-25T11:27:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.666+0000 7f3555600640 1 -- 172.21.15.19:0/2198971852 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c03f030 con 0x7f3550050890 2024-04-25T11:27:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.666+0000 7efbfaa00640 1 -- 172.21.15.19:0/623065700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efbf401a070 con 0x7efbfc047db0 2024-04-25T11:27:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.666+0000 7f3555600640 1 -- 172.21.15.19:0/2198971852 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354c030510 con 0x7f3550050890 2024-04-25T11:27:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.666+0000 7f355ec00640 1 -- 172.21.15.19:0/2198971852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3550186ee0 con 0x7f3550050890 2024-04-25T11:27:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.666+0000 7f3555600640 1 -- 172.21.15.19:0/2198971852 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c03b650 con 0x7f3550050890 2024-04-25T11:27:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.666+0000 7f355ec00640 1 -- 172.21.15.19:0/2198971852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3550186640 con 0x7f3550050890 2024-04-25T11:27:35.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.668+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352c005290 con 0x7f3550050890 2024-04-25T11:27:35.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.668+0000 7f141e000640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14200cf7f0 0x7f14200cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7efbfaa00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbe4074530 0x7efbe4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7f141ea00640 1 -- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14200cf7f0 msgr2=0x7f14200cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7f141ea00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14200cf7f0 0x7f14200cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7f141ea00640 1 -- 172.21.15.19:0/4203263929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14200cb650 msgr2=0x7f14201609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7f141ea00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14200cb650 0x7f14201609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7f141ea00640 1 -- 172.21.15.19:0/4203263929 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142015e470 con 0x7f14200b4ff0 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7f141e000640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14200cf7f0 0x7f14200cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.669+0000 7f141f400640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14200cb650 0x7f14201609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.670+0000 7efbfaa00640 1 -- 172.21.15.19:0/623065700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efbf405b430 con 0x7efbfc047db0 2024-04-25T11:27:35.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.670+0000 7f141ea00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200b4ff0 0x7f14201fda20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f141000e8a0 tx=0x7f141000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.670+0000 7f1416a00640 1 -- 172.21.15.19:0/4203263929 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410017070 con 0x7f14200b4ff0 2024-04-25T11:27:35.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.671+0000 7f1416a00640 1 -- 172.21.15.19:0/4203263929 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f141000c940 con 0x7f14200b4ff0 2024-04-25T11:27:35.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.671+0000 7f1416a00640 1 -- 172.21.15.19:0/4203263929 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141000ccf0 con 0x7f14200b4ff0 2024-04-25T11:27:35.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.671+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142015e750 con 0x7f14200b4ff0 2024-04-25T11:27:35.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.671+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f142015ed10 con 0x7f14200b4ff0 2024-04-25T11:27:35.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.673+0000 7efc02a00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbe4074530 0x7efbe4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.674+0000 7f3555600640 1 -- 172.21.15.19:0/2198971852 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f354c042030 con 0x7f3550050890 2024-04-25T11:27:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.674+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1420050740 con 0x7f14200b4ff0 2024-04-25T11:27:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.676+0000 7f3555600640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3538074530 0x7f3538076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.676+0000 7f3555600640 1 -- 172.21.15.19:0/2198971852 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f354c064e00 con 0x7f3550050890 2024-04-25T11:27:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.676+0000 7efc02a00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbe4074530 0x7efbe4076a40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0e82d0 tx=0x7efc040ab000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.677+0000 7f355ce00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3538074530 0x7f3538076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.678+0000 7f1416a00640 1 -- 172.21.15.19:0/4203263929 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f141001a070 con 0x7f14200b4ff0 2024-04-25T11:27:35.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.679+0000 7f1416a00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14000744e0 0x7f14000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.679+0000 7f1416a00640 1 -- 172.21.15.19:0/4203263929 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f14100958a0 con 0x7f14200b4ff0 2024-04-25T11:27:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.680+0000 7efbfaa00640 1 -- 172.21.15.19:0/623065700 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efbf405f1a0 con 0x7efbfc047db0 2024-04-25T11:27:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.680+0000 7f141e000640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14000744e0 0x7f14000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.681+0000 7f355ce00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3538074530 0x7f3538076a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3544000f60 tx=0x7f3544007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.682+0000 7f141e000640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14000744e0 0x7f14000769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f140c034000 tx=0x7f140c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.691+0000 7f3555600640 1 -- 172.21.15.19:0/2198971852 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f354c00a030 con 0x7f3550050890 2024-04-25T11:27:35.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.693+0000 7f1416a00640 1 -- 172.21.15.19:0/4203263929 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f141005f9c0 con 0x7f14200b4ff0 2024-04-25T11:27:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.705+0000 7ff2ebe00640 1 Processor -- start 2024-04-25T11:27:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.705+0000 7ff2ebe00640 1 -- start start 2024-04-25T11:27:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.708+0000 7ff2ebe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2e4144650 0x7ff2e40b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.708+0000 7ff2ebe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff2e40b6550 0x7ff2e406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.709+0000 7ff2ebe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e406df20 con 0x7ff2e406db10 2024-04-25T11:27:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.709+0000 7ff2ebe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e4145c90 con 0x7ff2e4144650 2024-04-25T11:27:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.709+0000 7ff2ebe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e40b6a60 con 0x7ff2e40b6550 2024-04-25T11:27:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.709+0000 7ff2e9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2e4144650 0x7ff2e40b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.709+0000 7ff2e9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2e4144650 0x7ff2e40b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49086/0 (socket says 172.21.15.19:49086) 2024-04-25T11:27:35.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.709+0000 7ff2e9400640 1 -- 172.21.15.19:0/1710422205 learned_addr learned my addr 172.21.15.19:0/1710422205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.710+0000 7ff2e9400640 1 -- 172.21.15.19:0/1710422205 >> v2:172.21.15.169:3300/0 conn(0x7ff2e40b6550 msgr2=0x7ff2e406d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:27:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.710+0000 7ff2e9400640 1 --2- 172.21.15.19:0/1710422205 >> v2:172.21.15.169:3300/0 conn(0x7ff2e40b6550 0x7ff2e406d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.710+0000 7ff2e9400640 1 -- 172.21.15.19:0/1710422205 >> v1:172.21.15.169:6789/0 conn(0x7ff2e406db10 legacy=0x7ff2e4161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.710+0000 7ff2e9400640 1 -- 172.21.15.19:0/1710422205 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e40b4aa0 con 0x7ff2e4144650 2024-04-25T11:27:35.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.712+0000 7ff2e9400640 1 --2- 172.21.15.19:0/1710422205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e4144650 0x7ff2e40b6010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40cf5c0 tx=0x7ff2d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efcc996703c0318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.713+0000 7ff2e3e00640 1 -- 172.21.15.19:0/1710422205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8037070 con 0x7ff2e4144650 2024-04-25T11:27:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.713+0000 7ff2e3e00640 1 -- 172.21.15.19:0/1710422205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d8032440 con 0x7ff2e4144650 2024-04-25T11:27:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.713+0000 7ff2e3e00640 1 -- 172.21.15.19:0/1710422205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8032810 con 0x7ff2e4144650 2024-04-25T11:27:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.714+0000 7ffa48e00640 1 Processor -- start 2024-04-25T11:27:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.714+0000 7ffa48e00640 1 -- start start 2024-04-25T11:27:35.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.715+0000 7ff2ebe00640 1 -- 172.21.15.19:0/1710422205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e4144650 msgr2=0x7ff2e40b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.715+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/1710422205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e4144650 0x7ff2e40b6010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40cf5c0 tx=0x7ff2d802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.715+0000 7ff2ebe00640 1 -- 172.21.15.19:0/1710422205 shutdown_connections 2024-04-25T11:27:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.715+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/1710422205 >> v2:172.21.15.169:3300/0 conn(0x7ff2e40b6550 0x7ff2e406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.715+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/1710422205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e4144650 0x7ff2e40b6010 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.715+0000 7ff2ebe00640 1 -- 172.21.15.19:0/1710422205 >> 172.21.15.19:0/1710422205 conn(0x7ff2e4073050 msgr2=0x7ff2e4073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.716+0000 7ff2ebe00640 1 -- 172.21.15.19:0/1710422205 shutdown_connections 2024-04-25T11:27:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.717+0000 7ffa48e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffa4413fe70 0x7ffa440b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.717+0000 7ffa48e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa440d0400 0x7ffa440d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.717+0000 7ffa48e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa440d0d50 con 0x7ffa441615a0 2024-04-25T11:27:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.717+0000 7ffa48e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ffa4415d510 con 0x7ffa4413fe70 2024-04-25T11:27:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.717+0000 7ffa48e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4415d7f0 con 0x7ffa440d0400 2024-04-25T11:27:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.718+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa440d0400 0x7ffa440d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.718+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa440d0400 0x7ffa440d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48234/0 (socket says 172.21.15.19:48234) 2024-04-25T11:27:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.718+0000 7ffa43e00640 1 -- 172.21.15.19:0/363548877 learned_addr learned my addr 172.21.15.19:0/363548877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.719+0000 7ffa43e00640 1 -- 172.21.15.19:0/363548877 >> v2:172.21.15.19:3300/0 conn(0x7ffa4413fe70 msgr2=0x7ffa440b6320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:27:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.719+0000 7ffa43e00640 1 --2- 172.21.15.19:0/363548877 >> v2:172.21.15.19:3300/0 conn(0x7ffa4413fe70 0x7ffa440b6320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.719+0000 7ffa43e00640 1 -- 172.21.15.19:0/363548877 >> v1:172.21.15.19:6789/0 conn(0x7ffa441615a0 legacy=0x7ffa440b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.719+0000 7ffa43e00640 1 -- 172.21.15.19:0/363548877 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4415dad0 con 0x7ffa440d0400 2024-04-25T11:27:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.716+0000 7ff2ebe00640 1 -- 172.21.15.19:0/1710422205 wait complete. 2024-04-25T11:27:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.720+0000 7ff2ebe00640 1 Processor -- start 2024-04-25T11:27:35.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.720+0000 7ff2ebe00640 1 -- start start 2024-04-25T11:27:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.721+0000 7ffa43e00640 1 --2- 172.21.15.19:0/363548877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa440d0400 0x7ffa440d0810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009870 tx=0x7ffa3c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b04595850fa30972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.721+0000 7ffa42000640 1 -- 172.21.15.19:0/363548877 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c037070 con 0x7ffa440d0400 2024-04-25T11:27:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.722+0000 7ffa42000640 1 -- 172.21.15.19:0/363548877 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3c032440 con 0x7ffa440d0400 2024-04-25T11:27:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.722+0000 7ffa42000640 1 -- 172.21.15.19:0/363548877 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c032810 con 0x7ffa440d0400 2024-04-25T11:27:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.723+0000 7ffa48e00640 1 -- 172.21.15.19:0/363548877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa440d0400 msgr2=0x7ffa440d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.723+0000 7ff2ebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e406db10 0x7ff2e4141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.723+0000 7ffa48e00640 1 --2- 172.21.15.19:0/363548877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa440d0400 0x7ffa440d0810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009870 tx=0x7ffa3c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.723+0000 7ffa48e00640 1 -- 172.21.15.19:0/363548877 shutdown_connections 2024-04-25T11:27:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ffa48e00640 1 --2- 172.21.15.19:0/363548877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa440d0400 0x7ffa440d0810 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2ebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2e40b6550 0x7ff2e41f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ffa48e00640 1 --2- 172.21.15.19:0/363548877 >> v2:172.21.15.19:3300/0 conn(0x7ffa4413fe70 0x7ffa440b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2ebe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2e406ef90 0x7ff2e415fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2e9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e406db10 0x7ff2e4141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2e9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e406db10 0x7ff2e4141390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49102/0 (socket says 172.21.15.19:49102) 2024-04-25T11:27:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2e9400640 1 -- 172.21.15.19:0/20029407 learned_addr learned my addr 172.21.15.19:0/20029407 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e40b5340 con 0x7ff2e406db10 2024-04-25T11:27:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ffa48e00640 1 -- 172.21.15.19:0/363548877 >> 172.21.15.19:0/363548877 conn(0x7ffa440d1d50 msgr2=0x7ffa44072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e40b5060 con 0x7ff2e406ef90 2024-04-25T11:27:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.724+0000 7ff2e8a00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2e40b6550 0x7ff2e41f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.725+0000 7ff2e9e00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2e406ef90 0x7ff2e415fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.725+0000 7ffa48e00640 1 -- 172.21.15.19:0/363548877 shutdown_connections 2024-04-25T11:27:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.725+0000 7ffa48e00640 1 -- 172.21.15.19:0/363548877 wait complete. 2024-04-25T11:27:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.725+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e40b5860 con 0x7ff2e40b6550 2024-04-25T11:27:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.725+0000 7f9d53000640 1 Processor -- start 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.726+0000 7ff2e9400640 1 -- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2e40b6550 msgr2=0x7ff2e41f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.726+0000 7ff2e9400640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2e40b6550 0x7ff2e41f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.726+0000 7ff2e9400640 1 -- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2e406ef90 msgr2=0x7ff2e415fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.727+0000 7ff2e9400640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2e406ef90 0x7ff2e415fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.727+0000 7ff2e9400640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e406f770 con 0x7ff2e406db10 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.728+0000 7ff2e8a00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2e40b6550 0x7ff2e41f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.728+0000 7ff2e9400640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e406db10 0x7ff2e4141390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000c00 tx=0x7ff2d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.728+0000 7ff2e1600640 1 -- 172.21.15.19:0/20029407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8041070 con 0x7ff2e406db10 2024-04-25T11:27:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.728+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e406fa50 con 0x7ff2e406db10 2024-04-25T11:27:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.728+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41600a0 con 0x7ff2e406db10 2024-04-25T11:27:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.729+0000 7ff2e1600640 1 -- 172.21.15.19:0/20029407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d8032440 con 0x7ff2e406db10 2024-04-25T11:27:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.729+0000 7f9d53000640 1 -- start start 2024-04-25T11:27:35.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.729+0000 7f350ec00640 1 Processor -- start 2024-04-25T11:27:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.730+0000 7ffa48e00640 1 Processor -- start 2024-04-25T11:27:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.730+0000 7f350ec00640 1 -- start start 2024-04-25T11:27:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.729+0000 7ff2e1600640 1 -- 172.21.15.19:0/20029407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8041070 con 0x7ff2e406db10 2024-04-25T11:27:35.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.730+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c0005290 con 0x7ff2e406db10 2024-04-25T11:27:35.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.732+0000 7f9d53000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d4c06f180 0x7f9d4c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.732+0000 7f9d53000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.732+0000 7f9d53000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d4c06d0f0 con 0x7f9d4c15de30 2024-04-25T11:27:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.732+0000 7f9d53000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9d4c160a20 con 0x7f9d4c06f180 2024-04-25T11:27:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.732+0000 7f9d53000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4c160d00 con 0x7f9d4c06cce0 2024-04-25T11:27:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.733+0000 7ff2e1600640 1 -- 172.21.15.19:0/20029407 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff2d8044070 con 0x7ff2e406db10 2024-04-25T11:27:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.731+0000 7ffa48e00640 1 -- start start 2024-04-25T11:27:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.733+0000 7f9d51200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d4c06f180 0x7f9d4c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.733+0000 7f9d52600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.733+0000 7f9d51200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d4c06f180 0x7f9d4c06c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49104/0 (socket says 172.21.15.19:49104) 2024-04-25T11:27:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.733+0000 7f9d51200640 1 -- 172.21.15.19:0/3980421296 learned_addr learned my addr 172.21.15.19:0/3980421296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.734+0000 7ff2e1600640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2c40746b0 0x7ff2c4076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.735+0000 7ff2e1600640 1 -- 172.21.15.19:0/20029407 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff2d80b5be0 con 0x7ff2e406db10 2024-04-25T11:27:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.734+0000 7f9d51200640 1 -- 172.21.15.19:0/3980421296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 msgr2=0x7f9d4c0cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.734+0000 7f9d51200640 1 --2- 172.21.15.19:0/3980421296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c0cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.734+0000 7f9d51200640 1 -- 172.21.15.19:0/3980421296 >> v1:172.21.15.169:6789/0 conn(0x7f9d4c15de30 legacy=0x7f9d4c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.734+0000 7f9d51200640 1 -- 172.21.15.19:0/3980421296 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c160fe0 con 0x7f9d4c06f180 2024-04-25T11:27:35.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.734+0000 7f9d52600640 1 --2- 172.21.15.19:0/3980421296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c0cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7ffa48e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440d0400 0x7ffa440731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7ffa48e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa4413fe70 0x7ffa44141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7ffa48e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa441615a0 0x7ffa44141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7ffa48e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4406ca20 con 0x7ffa440d0400 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7ffa48e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4406c8a0 con 0x7ffa4413fe70 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7ffa48e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4406cba0 con 0x7ffa441615a0 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7f350ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7f350ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35080b1500 0x7f35080b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7f350ec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35080b0740 con 0x7f35080d1000 2024-04-25T11:27:35.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7f350ec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f35080d2b30 con 0x7f35080b1500 2024-04-25T11:27:35.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7f350ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35080b0a20 con 0x7f35080d3790 2024-04-25T11:27:35.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7f9d51200640 1 --2- 172.21.15.19:0/3980421296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c06f180 0x7f9d4c06c7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009870 tx=0x7f9d4002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4157d10ed5b2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.735+0000 7ff2e8a00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2c40746b0 0x7ff2c4076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.736+0000 7ffa43400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440d0400 0x7ffa440731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa441615a0 0x7ffa44141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa4413fe70 0x7ffa44141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7ffa43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa441615a0 0x7ffa44141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48268/0 (socket says 172.21.15.19:48268) 2024-04-25T11:27:35.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7ffa43e00640 1 -- 172.21.15.19:0/1730003322 learned_addr learned my addr 172.21.15.19:0/1730003322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7f350cc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35080b1500 0x7f35080b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7f3507400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7f350cc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35080b1500 0x7f35080b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:37578/0 (socket says 172.21.15.19:37578) 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.737+0000 7f350cc00640 1 -- 172.21.15.19:0/1504064645 learned_addr learned my addr 172.21.15.19:0/1504064645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7ffa42a00640 1 -- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa441615a0 msgr2=0x7ffa44141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7ffa42a00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa441615a0 0x7ffa44141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7ffa42a00640 1 -- 172.21.15.19:0/1730003322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440d0400 msgr2=0x7ffa440731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7ffa42a00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440d0400 0x7ffa440731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7ffa42a00640 1 -- 172.21.15.19:0/1730003322 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4413a680 con 0x7ffa4413fe70 2024-04-25T11:27:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7ff2e8a00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2c40746b0 0x7ff2c4076bc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2e406c910 tx=0x7ff2d0009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.739+0000 7ffa42a00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa4413fe70 0x7ffa44141560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400be50 tx=0x7ffa34009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.739+0000 7ffa3b400640 1 -- 172.21.15.19:0/1730003322 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34019070 con 0x7ffa4413fe70 2024-04-25T11:27:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7f350cc00640 1 -- 172.21.15.19:0/1504064645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 msgr2=0x7f35080d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.739+0000 7ffa3b400640 1 -- 172.21.15.19:0/1730003322 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa34014440 con 0x7ffa4413fe70 2024-04-25T11:27:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.739+0000 7ffa3b400640 1 -- 172.21.15.19:0/1730003322 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34014830 con 0x7ffa4413fe70 2024-04-25T11:27:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.740+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4413a960 con 0x7ffa4413fe70 2024-04-25T11:27:35.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7f350cc00640 1 --2- 172.21.15.19:0/1504064645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7f350cc00640 1 -- 172.21.15.19:0/1504064645 >> v1:172.21.15.19:6789/0 conn(0x7f35080d1000 legacy=0x7f35080d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:35.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.738+0000 7f350cc00640 1 -- 172.21.15.19:0/1504064645 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35080b0d00 con 0x7f35080b1500 2024-04-25T11:27:35.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.740+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4413af20 con 0x7ffa4413fe70 2024-04-25T11:27:35.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.740+0000 7f350cc00640 1 --2- 172.21.15.19:0/1504064645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080b1500 0x7f35080b1910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3500009e50 tx=0x7f350002a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c54b4abe8f1929d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.742+0000 7f9d4be00640 1 -- 172.21.15.19:0/3980421296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40037070 con 0x7f9d4c06f180 2024-04-25T11:27:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.742+0000 7f9d4be00640 1 -- 172.21.15.19:0/3980421296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d40032440 con 0x7f9d4c06f180 2024-04-25T11:27:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.742+0000 7f9d4be00640 1 -- 172.21.15.19:0/3980421296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40032810 con 0x7f9d4c06f180 2024-04-25T11:27:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.742+0000 7f3506a00640 1 -- 172.21.15.19:0/1504064645 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500036070 con 0x7f35080b1500 2024-04-25T11:27:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.742+0000 7f3506a00640 1 -- 172.21.15.19:0/1504064645 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3500031600 con 0x7f35080b1500 2024-04-25T11:27:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.742+0000 7f3506a00640 1 -- 172.21.15.19:0/1504064645 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35000319d0 con 0x7f35080b1500 2024-04-25T11:27:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.742+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa44050740 con 0x7ffa4413fe70 2024-04-25T11:27:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.743+0000 7ff2e1600640 1 -- 172.21.15.19:0/20029407 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff2e406df20 con 0x7ff2e406db10 2024-04-25T11:27:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.743+0000 7f9d53000640 1 -- 172.21.15.19:0/3980421296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c06f180 msgr2=0x7f9d4c06c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.743+0000 7f9d53000640 1 --2- 172.21.15.19:0/3980421296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c06f180 0x7f9d4c06c7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009870 tx=0x7f9d4002a200 comp rx=0 tx=0).stop 2024-04-25T11:27:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.744+0000 7f9d53000640 1 -- 172.21.15.19:0/3980421296 shutdown_connections 2024-04-25T11:27:35.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.744+0000 7f9d53000640 1 --2- 172.21.15.19:0/3980421296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c0cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.744+0000 7f9d53000640 1 --2- 172.21.15.19:0/3980421296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c06f180 0x7f9d4c06c7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.744+0000 7f9d53000640 1 -- 172.21.15.19:0/3980421296 >> 172.21.15.19:0/3980421296 conn(0x7f9d4c0d1d50 msgr2=0x7f9d4c072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.745+0000 7f9d53000640 1 -- 172.21.15.19:0/3980421296 shutdown_connections 2024-04-25T11:27:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.745+0000 7ffa3b400640 1 -- 172.21.15.19:0/1730003322 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffa3401c070 con 0x7ffa4413fe70 2024-04-25T11:27:35.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.745+0000 7f9d53000640 1 -- 172.21.15.19:0/3980421296 wait complete. 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.745+0000 7f350ec00640 1 -- 172.21.15.19:0/1504064645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080b1500 msgr2=0x7f35080b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.745+0000 7f350ec00640 1 --2- 172.21.15.19:0/1504064645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080b1500 0x7f35080b1910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3500009e50 tx=0x7f350002a6d0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.746+0000 7f350ec00640 1 -- 172.21.15.19:0/1504064645 shutdown_connections 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.746+0000 7f350ec00640 1 --2- 172.21.15.19:0/1504064645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080b1500 0x7f35080b1910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.746+0000 7f350ec00640 1 --2- 172.21.15.19:0/1504064645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.746+0000 7f350ec00640 1 -- 172.21.15.19:0/1504064645 >> 172.21.15.19:0/1504064645 conn(0x7f350808d110 msgr2=0x7f350808d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.746+0000 7ffa3b400640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa20074500 0x7ffa20076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.746+0000 7ffa3b400640 1 -- 172.21.15.19:0/1730003322 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffa34095270 con 0x7ffa4413fe70 2024-04-25T11:27:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.747+0000 7ffa43400640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa20074500 0x7ffa20076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.747+0000 7f350ec00640 1 -- 172.21.15.19:0/1504064645 shutdown_connections 2024-04-25T11:27:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.747+0000 7f350ec00640 1 -- 172.21.15.19:0/1504064645 wait complete. 2024-04-25T11:27:35.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.748+0000 7ffa43400640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa20074500 0x7ffa20076a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009930 tx=0x7ffa2c009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.750+0000 7f350ec00640 1 Processor -- start 2024-04-25T11:27:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.750+0000 7f9d53000640 1 Processor -- start 2024-04-25T11:27:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.750+0000 7f350ec00640 1 -- start start 2024-04-25T11:27:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.750+0000 7f9d53000640 1 -- start start 2024-04-25T11:27:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f350ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35080b1500 0x7f35080cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f350ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080d1000 0x7f3508152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f350ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f350ec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35080b2ad0 con 0x7f35080b1500 2024-04-25T11:27:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f350ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35080d40e0 con 0x7f35080d1000 2024-04-25T11:27:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f350ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35080d2780 con 0x7f35080d3790 2024-04-25T11:27:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f3507e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35080b1500 0x7f35080cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f3507e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35080b1500 0x7f35080cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49118/0 (socket says 172.21.15.19:49118) 2024-04-25T11:27:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f3507e00640 1 -- 172.21.15.19:0/2311968787 learned_addr learned my addr 172.21.15.19:0/2311968787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.752+0000 7f9d53000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f9d53000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c06f180 0x7f9d4c1fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507400640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080d1000 0x7f3508152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f350cc00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f9d53000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c15de30 0x7f9d4c1fad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f9d53000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c1606d0 con 0x7f9d4c15de30 2024-04-25T11:27:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f9d53000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c161630 con 0x7f9d4c06f180 2024-04-25T11:27:35.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f9d53000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4c0d1010 con 0x7f9d4c06cce0 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507e00640 1 -- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 msgr2=0x7f35080ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507e00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507e00640 1 -- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080d1000 msgr2=0x7f3508152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507e00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080d1000 0x7f3508152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507e00640 1 -- 172.21.15.19:0/2311968787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3508152760 con 0x7f35080b1500 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507400640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080d1000 0x7f3508152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f350cc00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:35.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f3507e00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35080b1500 0x7f35080cb600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f34f800b420 tx=0x7f34f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.753+0000 7f9d51c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f9d51200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c06f180 0x7f9d4c1fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f34ffe00640 1 -- 172.21.15.19:0/2311968787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8011830 con 0x7f35080b1500 2024-04-25T11:27:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f34ffe00640 1 -- 172.21.15.19:0/2311968787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f8016070 con 0x7f35080b1500 2024-04-25T11:27:35.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f34ffe00640 1 -- 172.21.15.19:0/2311968787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8011c10 con 0x7f35080b1500 2024-04-25T11:27:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35080cbea0 con 0x7f35080b1500 2024-04-25T11:27:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f9d51c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48284/0 (socket says 172.21.15.19:48284) 2024-04-25T11:27:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f9d51c00640 1 -- 172.21.15.19:0/697590905 learned_addr learned my addr 172.21.15.19:0/697590905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f9d52600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c15de30 0x7f9d4c1fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.754+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350806d690 con 0x7f35080b1500 2024-04-25T11:27:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.756+0000 7f9d52600640 1 -- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 msgr2=0x7f9d4c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.756+0000 7f9d52600640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.756+0000 7f9d52600640 1 -- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c06f180 msgr2=0x7f9d4c1fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.756+0000 7f9d52600640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c06f180 0x7f9d4c1fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.756+0000 7f9d52600640 1 -- 172.21.15.19:0/697590905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c1ff250 con 0x7f9d4c15de30 2024-04-25T11:27:35.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.757+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3508050740 con 0x7f35080b1500 2024-04-25T11:27:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.757+0000 7f9d52600640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c15de30 0x7f9d4c1fad30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400efd0 tx=0x7f9d4400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.757+0000 7f9d49600640 1 -- 172.21.15.19:0/697590905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44018030 con 0x7f9d4c15de30 2024-04-25T11:27:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.757+0000 7f9d49600640 1 -- 172.21.15.19:0/697590905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4400cd00 con 0x7f9d4c15de30 2024-04-25T11:27:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.757+0000 7f9d49600640 1 -- 172.21.15.19:0/697590905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44003bb0 con 0x7f9d4c15de30 2024-04-25T11:27:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.758+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4c1fb270 con 0x7f9d4c15de30 2024-04-25T11:27:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.758+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4c1fba70 con 0x7f9d4c15de30 2024-04-25T11:27:35.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.760+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4c1fbcb0 con 0x7f9d4c15de30 2024-04-25T11:27:35.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.761+0000 7ffa3b400640 1 -- 172.21.15.19:0/1730003322 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa3405f390 con 0x7ffa4413fe70 2024-04-25T11:27:35.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.760+0000 7f34ffe00640 1 -- 172.21.15.19:0/2311968787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34f801b460 con 0x7f35080b1500 2024-04-25T11:27:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.761+0000 7f34ffe00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e80745b0 0x7f34e8076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.761+0000 7f34ffe00640 1 -- 172.21.15.19:0/2311968787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f34f8094e40 con 0x7f35080b1500 2024-04-25T11:27:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.762+0000 7f3507400640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e80745b0 0x7f34e8076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.767+0000 7f3507400640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e80745b0 0x7f34e8076ac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f350806f290 tx=0x7f34f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.767+0000 7f34ffe00640 1 -- 172.21.15.19:0/2311968787 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3508152760 con 0x7f35080b1500 2024-04-25T11:27:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.769+0000 7f9d49600640 1 -- 172.21.15.19:0/697590905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9d44005d10 con 0x7f9d4c15de30 2024-04-25T11:27:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.771+0000 7f9d49600640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9d300744b0 0x7f9d300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.771+0000 7f9d49600640 1 -- 172.21.15.19:0/697590905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9d440965e0 con 0x7f9d4c15de30 2024-04-25T11:27:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.776+0000 7f9d51c00640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9d300744b0 0x7f9d300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.776+0000 7f9d49600640 1 -- 172.21.15.19:0/697590905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d44022070 con 0x7f9d4c15de30 2024-04-25T11:27:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:35.777+0000 7f9d51c00640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9d300744b0 0x7f9d300769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c006fd0 tx=0x7f9d3c0090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.077+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd4e0005580 con 0x7fd500058b80 2024-04-25T11:27:36.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.078+0000 7fd4fea00640 1 -- 172.21.15.19:0/3829713159 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4f8062f90 con 0x7fd500058b80 2024-04-25T11:27:36.070 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-04-25T11:27:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.082+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4e80746e0 msgr2=0x7fd4e8076bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.082+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4e80746e0 0x7fd4e8076bf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd508051c90 tx=0x7fd5080ab000 comp rx=0 tx=0).stop 2024-04-25T11:27:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.082+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500058b80 msgr2=0x7fd500052950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.082+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500058b80 0x7fd500052950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800e870 tx=0x7fd4f800ed30 comp rx=0 tx=0).stop 2024-04-25T11:27:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.083+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 shutdown_connections 2024-04-25T11:27:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.083+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4e80746e0 0x7fd4e8076bf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.083+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500058b80 0x7fd500052950 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.083+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000e3a80 0x7fd5001817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.083+0000 7fd50d400640 1 --2- 172.21.15.19:0/3829713159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5000073c0 0x7fd50017b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.084+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 >> 172.21.15.19:0/3829713159 conn(0x7fd50003abe0 msgr2=0x7fd50003cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.084+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 shutdown_connections 2024-04-25T11:27:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.084+0000 7fd50d400640 1 -- 172.21.15.19:0/3829713159 wait complete. 2024-04-25T11:27:36.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-04-25T11:27:36.114 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.130+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f352c005850 con 0x7f3550050890 2024-04-25T11:27:36.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.130+0000 7f3555600640 1 -- 172.21.15.19:0/2198971852 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f354c030e90 con 0x7f3550050890 2024-04-25T11:27:36.123 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-04-25T11:27:36.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.132+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7efbfc049830 con 0x7efbfc047db0 2024-04-25T11:27:36.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.132+0000 7efbfaa00640 1 -- 172.21.15.19:0/623065700 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efbf40634a0 con 0x7efbfc047db0 2024-04-25T11:27:36.124 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-04-25T11:27:36.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3538074530 msgr2=0x7f3538076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3538074530 0x7f3538076a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3544000f60 tx=0x7f3544007e40 comp rx=0 tx=0).stop 2024-04-25T11:27:36.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3550050890 msgr2=0x7f355002b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3550050890 0x7f355002b6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f354c000c00 tx=0x7f354c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 shutdown_connections 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3538074530 0x7f3538076a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3550185db0 0x7f355004ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35500dfd60 0x7f3550185870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 --2- 172.21.15.19:0/2198971852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3550050890 0x7f355002b6c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 >> 172.21.15.19:0/2198971852 conn(0x7f3550054cf0 msgr2=0x7f3550053d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 shutdown_connections 2024-04-25T11:27:36.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.134+0000 7f354aa00640 1 -- 172.21.15.19:0/2198971852 wait complete. 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbe4074530 msgr2=0x7efbe4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbe4074530 0x7efbe4076a40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0e82d0 tx=0x7efc040ab000 comp rx=0 tx=0).stop 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc047db0 msgr2=0x7efbfc04a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc047db0 0x7efbfc04a320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e8e0 tx=0x7efbf400eda0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 shutdown_connections 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbe4074530 0x7efbe4076a40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbfc048300 0x7efbfc04a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc047db0 0x7efbfc04a320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 --2- 172.21.15.19:0/623065700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbfc033e90 0x7efbfc049120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.135+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 >> 172.21.15.19:0/623065700 conn(0x7efbfc054cf0 msgr2=0x7efbfc0371f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.136+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 shutdown_connections 2024-04-25T11:27:36.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.136+0000 7efc03e00640 1 -- 172.21.15.19:0/623065700 wait complete. 2024-04-25T11:27:36.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.141+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f14200b94a0 con 0x7f14200b4ff0 2024-04-25T11:27:36.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.142+0000 7f1416a00640 1 -- 172.21.15.19:0/4203263929 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1410068090 con 0x7f14200b4ff0 2024-04-25T11:27:36.134 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-04-25T11:27:36.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.146+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14000744e0 msgr2=0x7f14000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.146+0000 7f141fe00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14000744e0 0x7f14000769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f140c034000 tx=0x7f140c02e000 comp rx=0 tx=0).stop 2024-04-25T11:27:36.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.146+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200b4ff0 msgr2=0x7f14201fda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.146+0000 7f141fe00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200b4ff0 0x7f14201fda20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f141000e8a0 tx=0x7f141000ed60 comp rx=0 tx=0).stop 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.148+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 shutdown_connections 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.148+0000 7f141fe00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14000744e0 0x7f14000769f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.148+0000 7f141fe00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14200cf7f0 0x7f14200cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.148+0000 7f141fe00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14200cb650 0x7f14201609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.148+0000 7f141fe00640 1 --2- 172.21.15.19:0/4203263929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14200b4ff0 0x7f14201fda20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.148+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 >> 172.21.15.19:0/4203263929 conn(0x7f14200b5460 msgr2=0x7f142006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.149+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 shutdown_connections 2024-04-25T11:27:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.149+0000 7f141fe00640 1 -- 172.21.15.19:0/4203263929 wait complete. 2024-04-25T11:27:36.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-04-25T11:27:36.167 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-04-25T11:27:36.168 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.178+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9d4c1fb520 con 0x7f9d4c15de30 2024-04-25T11:27:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.178+0000 7f9d49600640 1 -- 172.21.15.19:0/697590905 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d4c1ff250 con 0x7f9d4c15de30 2024-04-25T11:27:36.169 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-04-25T11:27:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9d300744b0 msgr2=0x7f9d300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9d300744b0 0x7f9d300769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c006fd0 tx=0x7f9d3c0090e0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c15de30 msgr2=0x7f9d4c1fad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c15de30 0x7f9d4c1fad30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400efd0 tx=0x7f9d4400c550 comp rx=0 tx=0).stop 2024-04-25T11:27:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 shutdown_connections 2024-04-25T11:27:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9d300744b0 0x7f9d300769c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c06f180 0x7f9d4c1fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d4c15de30 0x7f9d4c1fad30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 --2- 172.21.15.19:0/697590905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c06cce0 0x7f9d4c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 >> 172.21.15.19:0/697590905 conn(0x7f9d4c0d1d50 msgr2=0x7f9d4c15fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 shutdown_connections 2024-04-25T11:27:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.181+0000 7f9d53000640 1 -- 172.21.15.19:0/697590905 wait complete. 2024-04-25T11:27:36.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2024-04-25T11:27:36.177 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.195+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f35080d08a0 con 0x7f35080b1500 2024-04-25T11:27:36.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.195+0000 7f34ffe00640 1 -- 172.21.15.19:0/2311968787 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f34f805f5b0 con 0x7f35080b1500 2024-04-25T11:27:36.186 INFO:teuthology.orchestra.run.smithi019.stdout:60129542150 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e80745b0 msgr2=0x7f34e8076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e80745b0 0x7f34e8076ac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f350806f290 tx=0x7f34f4008040 comp rx=0 tx=0).stop 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35080b1500 msgr2=0x7f35080cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35080b1500 0x7f35080cb600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f34f800b420 tx=0x7f34f800b8e0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 shutdown_connections 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e80745b0 0x7f34e8076ac0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f350806f290 tx=0x7f34f4008040 comp rx=0 tx=0).stop 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35080d1000 0x7f3508152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35080d3790 0x7f35080ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 --2- 172.21.15.19:0/2311968787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35080b1500 0x7f35080cb600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.198+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 >> 172.21.15.19:0/2311968787 conn(0x7f350808d110 msgr2=0x7f35080687c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.200+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 shutdown_connections 2024-04-25T11:27:36.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.200+0000 7f350ec00640 1 -- 172.21.15.19:0/2311968787 wait complete. 2024-04-25T11:27:36.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-04-25T11:27:36.199 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.212+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ffa4413b3c0 con 0x7ffa4413fe70 2024-04-25T11:27:36.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.213+0000 7ffa3b400640 1 -- 172.21.15.19:0/1730003322 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa34063690 con 0x7ffa4413fe70 2024-04-25T11:27:36.204 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-04-25T11:27:36.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.213+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff2c0005850 con 0x7ff2e406db10 2024-04-25T11:27:36.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.214+0000 7ff2e1600640 1 -- 172.21.15.19:0/20029407 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff2d807fda0 con 0x7ff2e406db10 2024-04-25T11:27:36.205 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-04-25T11:27:36.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa20074500 msgr2=0x7ffa20076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa20074500 0x7ffa20076a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009930 tx=0x7ffa2c009060 comp rx=0 tx=0).stop 2024-04-25T11:27:36.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa4413fe70 msgr2=0x7ffa44141560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa4413fe70 0x7ffa44141560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400be50 tx=0x7ffa34009f70 comp rx=0 tx=0).stop 2024-04-25T11:27:36.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 shutdown_connections 2024-04-25T11:27:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa20074500 0x7ffa20076a10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa4413fe70 0x7ffa44141560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa441615a0 0x7ffa44141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 --2- 172.21.15.19:0/1730003322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa440d0400 0x7ffa440731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 >> 172.21.15.19:0/1730003322 conn(0x7ffa440d1d50 msgr2=0x7ffa44073ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 shutdown_connections 2024-04-25T11:27:36.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.215+0000 7ffa48e00640 1 -- 172.21.15.19:0/1730003322 wait complete. 2024-04-25T11:27:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2c40746b0 msgr2=0x7ff2c4076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2c40746b0 0x7ff2c4076bc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2e406c910 tx=0x7ff2d0009060 comp rx=0 tx=0).stop 2024-04-25T11:27:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e406db10 msgr2=0x7ff2e4141390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e406db10 0x7ff2e4141390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000c00 tx=0x7ff2d80046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 shutdown_connections 2024-04-25T11:27:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2c40746b0 0x7ff2c4076bc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2e406ef90 0x7ff2e415fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2e40b6550 0x7ff2e41f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 --2- 172.21.15.19:0/20029407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2e406db10 0x7ff2e4141390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.216+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 >> 172.21.15.19:0/20029407 conn(0x7ff2e4073050 msgr2=0x7ff2e415ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.217+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 shutdown_connections 2024-04-25T11:27:36.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.217+0000 7ff2ebe00640 1 -- 172.21.15.19:0/20029407 wait complete. 2024-04-25T11:27:36.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-04-25T11:27:36.215 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-04-25T11:27:36.232 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-04-25T11:27:36.234 DEBUG:teuthology.parallel:result is None 2024-04-25T11:27:36.235 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:27:36.235 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:27:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.563+0000 7fce70000640 1 Processor -- start 2024-04-25T11:27:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.563+0000 7fce70000640 1 -- start start 2024-04-25T11:27:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.565+0000 7fce70000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fce68070680 0x7fce681647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce70000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce680b5c90 0x7fce680b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce70000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce680b65e0 con 0x7fce68160360 2024-04-25T11:27:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce70000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fce6806d100 con 0x7fce680b5c90 2024-04-25T11:27:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce70000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fce6806c7d0 con 0x7fce68070680 2024-04-25T11:27:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce6e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce680b5c90 0x7fce680b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:36.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce6e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce680b5c90 0x7fce680b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49132/0 (socket says 172.21.15.19:49132) 2024-04-25T11:27:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce6e000640 1 -- 172.21.15.19:0/4255508981 learned_addr learned my addr 172.21.15.19:0/4255508981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.566+0000 7fce6d600640 1 --2- 172.21.15.19:0/4255508981 >> v2:172.21.15.169:3300/0 conn(0x7fce68070680 0x7fce681647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.567+0000 7fce6e000640 1 -- 172.21.15.19:0/4255508981 >> v2:172.21.15.169:3300/0 conn(0x7fce68070680 msgr2=0x7fce681647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.567+0000 7fce6e000640 1 --2- 172.21.15.19:0/4255508981 >> v2:172.21.15.169:3300/0 conn(0x7fce68070680 0x7fce681647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.567+0000 7fce6e000640 1 -- 172.21.15.19:0/4255508981 >> v1:172.21.15.169:6789/0 conn(0x7fce68160360 legacy=0x7fce680af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.567+0000 7fce6e000640 1 -- 172.21.15.19:0/4255508981 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6806cab0 con 0x7fce680b5c90 2024-04-25T11:27:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.568+0000 7fce6e000640 1 --2- 172.21.15.19:0/4255508981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680b60a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fce68072b00 tx=0x7fce6002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afa638b0d529d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.568+0000 7fce67e00640 1 -- 172.21.15.19:0/4255508981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60036070 con 0x7fce680b5c90 2024-04-25T11:27:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.568+0000 7fce67e00640 1 -- 172.21.15.19:0/4255508981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce60031600 con 0x7fce680b5c90 2024-04-25T11:27:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.568+0000 7fce67e00640 1 -- 172.21.15.19:0/4255508981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce600319d0 con 0x7fce680b5c90 2024-04-25T11:27:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 -- 172.21.15.19:0/4255508981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 msgr2=0x7fce680b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 --2- 172.21.15.19:0/4255508981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680b60a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fce68072b00 tx=0x7fce6002a610 comp rx=0 tx=0).stop 2024-04-25T11:27:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 -- 172.21.15.19:0/4255508981 shutdown_connections 2024-04-25T11:27:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 --2- 172.21.15.19:0/4255508981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680b60a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 --2- 172.21.15.19:0/4255508981 >> v2:172.21.15.169:3300/0 conn(0x7fce68070680 0x7fce681647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 -- 172.21.15.19:0/4255508981 >> 172.21.15.19:0/4255508981 conn(0x7fce680d1ae0 msgr2=0x7fce680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 -- 172.21.15.19:0/4255508981 shutdown_connections 2024-04-25T11:27:36.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.569+0000 7fce70000640 1 -- 172.21.15.19:0/4255508981 wait complete. 2024-04-25T11:27:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.571+0000 7fce70000640 1 Processor -- start 2024-04-25T11:27:36.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.572+0000 7fce70000640 1 -- start start 2024-04-25T11:27:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce70000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce68070680 0x7fce6815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce70000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:36.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce70000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce68160360 0x7fce681eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:36.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce70000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce680afbb0 con 0x7fce680b5c90 2024-04-25T11:27:36.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce70000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce6806fcc0 con 0x7fce68070680 2024-04-25T11:27:36.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce70000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fce680afd30 con 0x7fce68160360 2024-04-25T11:27:36.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce6cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:36.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce6cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49138/0 (socket says 172.21.15.19:49138) 2024-04-25T11:27:36.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.573+0000 7fce6cc00640 1 -- 172.21.15.19:0/865704945 learned_addr learned my addr 172.21.15.19:0/865704945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:36.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6d600640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce68070680 0x7fce6815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:36.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6e000640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce68160360 0x7fce681eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:36.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6cc00640 1 -- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce68160360 msgr2=0x7fce681eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6cc00640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce68160360 0x7fce681eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6cc00640 1 -- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce68070680 msgr2=0x7fce6815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6cc00640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce68070680 0x7fce6815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6cc00640 1 -- 172.21.15.19:0/865704945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6815b490 con 0x7fce680b5c90 2024-04-25T11:27:36.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce6cc00640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680c6d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c3c0 tx=0x7fce5800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:36.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce65600640 1 -- 172.21.15.19:0/865704945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580052d0 con 0x7fce680b5c90 2024-04-25T11:27:36.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce65600640 1 -- 172.21.15.19:0/865704945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce58005490 con 0x7fce680b5c90 2024-04-25T11:27:36.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6815e4b0 con 0x7fce680b5c90 2024-04-25T11:27:36.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce65600640 1 -- 172.21.15.19:0/865704945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58007260 con 0x7fce680b5c90 2024-04-25T11:27:36.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.574+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6815ea70 con 0x7fce680b5c90 2024-04-25T11:27:36.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.575+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6815e200 con 0x7fce680b5c90 2024-04-25T11:27:36.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.577+0000 7fce65600640 1 -- 172.21.15.19:0/865704945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce58007400 con 0x7fce680b5c90 2024-04-25T11:27:36.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.578+0000 7fce65600640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce4c0744b0 0x7fce4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:36.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.578+0000 7fce65600640 1 -- 172.21.15.19:0/865704945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fce58094810 con 0x7fce680b5c90 2024-04-25T11:27:36.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.579+0000 7fce6d600640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce4c0744b0 0x7fce4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:36.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.580+0000 7fce6d600640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce4c0744b0 0x7fce4c0769c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce68071250 tx=0x7fce5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:36.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.582+0000 7fce65600640 1 -- 172.21.15.19:0/865704945 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce680b65e0 con 0x7fce680b5c90 2024-04-25T11:27:36.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.774+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce680d0c40 con 0x7fce4c0744b0 2024-04-25T11:27:36.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.794+0000 7fce65600640 1 -- 172.21.15.19:0/865704945 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7fce680d0c40 con 0x7fce4c0744b0 2024-04-25T11:27:36.789 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce4c0744b0 msgr2=0x7fce4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce4c0744b0 0x7fce4c0769c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce68071250 tx=0x7fce5c008040 comp rx=0 tx=0).stop 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 msgr2=0x7fce680c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680c6d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c3c0 tx=0x7fce5800c7c0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 shutdown_connections 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce4c0744b0 0x7fce4c0769c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce68160360 0x7fce681eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce680b5c90 0x7fce680c6d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 --2- 172.21.15.19:0/865704945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce68070680 0x7fce6815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.799+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 >> 172.21.15.19:0/865704945 conn(0x7fce680d1ae0 msgr2=0x7fce680b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.800+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 shutdown_connections 2024-04-25T11:27:36.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:36.800+0000 7fce70000640 1 -- 172.21.15.19:0/865704945 wait complete. 2024-04-25T11:27:36.816 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-25T11:27:36.438974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219116,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755604,"statfs":{"total":773094113280,"available":772869738496,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004589"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109908+0000","last_change":"2024-04-25T11:27:25.576865+0000","last_active":"2024-04-25T11:27:27.109908+0000","last_peered":"2024-04-25T11:27:27.109908+0000","last_clean":"2024-04-25T11:27:27.109908+0000","last_became_active":"2024-04-25T11:27:25.576473+0000","last_became_peered":"2024-04-25T11:27:25.576473+0000","last_unstale":"2024-04-25T11:27:27.109908+0000","last_undegraded":"2024-04-25T11:27:27.109908+0000","last_fullsized":"2024-04-25T11:27:27.109908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578302+0000","last_change":"2024-04-25T11:27:25.591722+0000","last_active":"2024-04-25T11:27:26.578302+0000","last_peered":"2024-04-25T11:27:26.578302+0000","last_clean":"2024-04-25T11:27:26.578302+0000","last_became_active":"2024-04-25T11:27:25.591379+0000","last_became_peered":"2024-04-25T11:27:25.591379+0000","last_unstale":"2024-04-25T11:27:26.578302+0000","last_undegraded":"2024-04-25T11:27:26.578302+0000","last_fullsized":"2024-04-25T11:27:26.578302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.430829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578349+0000","last_change":"2024-04-25T11:27:25.571047+0000","last_active":"2024-04-25T11:27:26.578349+0000","last_peered":"2024-04-25T11:27:26.578349+0000","last_clean":"2024-04-25T11:27:26.578349+0000","last_became_active":"2024-04-25T11:27:25.570667+0000","last_became_peered":"2024-04-25T11:27:25.570667+0000","last_unstale":"2024-04-25T11:27:26.578349+0000","last_undegraded":"2024-04-25T11:27:26.578349+0000","last_fullsized":"2024-04-25T11:27:26.578349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:32.385239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580734+0000","last_change":"2024-04-25T11:27:25.580512+0000","last_active":"2024-04-25T11:27:26.580734+0000","last_peered":"2024-04-25T11:27:26.580734+0000","last_clean":"2024-04-25T11:27:26.580734+0000","last_became_active":"2024-04-25T11:27:25.579802+0000","last_became_peered":"2024-04-25T11:27:25.579802+0000","last_unstale":"2024-04-25T11:27:26.580734+0000","last_undegraded":"2024-04-25T11:27:26.580734+0000","last_fullsized":"2024-04-25T11:27:26.580734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109585+0000","last_change":"2024-04-25T11:27:25.578185+0000","last_active":"2024-04-25T11:27:27.109585+0000","last_peered":"2024-04-25T11:27:27.109585+0000","last_clean":"2024-04-25T11:27:27.109585+0000","last_became_active":"2024-04-25T11:27:25.577850+0000","last_became_peered":"2024-04-25T11:27:25.577850+0000","last_unstale":"2024-04-25T11:27:27.109585+0000","last_undegraded":"2024-04-25T11:27:27.109585+0000","last_fullsized":"2024-04-25T11:27:27.109585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579999+0000","last_change":"2024-04-25T11:27:25.581557+0000","last_active":"2024-04-25T11:27:26.579999+0000","last_peered":"2024-04-25T11:27:26.579999+0000","last_clean":"2024-04-25T11:27:26.579999+0000","last_became_active":"2024-04-25T11:27:25.580913+0000","last_became_peered":"2024-04-25T11:27:25.580913+0000","last_unstale":"2024-04-25T11:27:26.579999+0000","last_undegraded":"2024-04-25T11:27:26.579999+0000","last_fullsized":"2024-04-25T11:27:26.579999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111438+0000","last_change":"2024-04-25T11:27:25.581148+0000","last_active":"2024-04-25T11:27:27.111438+0000","last_peered":"2024-04-25T11:27:27.111438+0000","last_clean":"2024-04-25T11:27:27.111438+0000","last_became_active":"2024-04-25T11:27:25.580733+0000","last_became_peered":"2024-04-25T11:27:25.580733+0000","last_unstale":"2024-04-25T11:27:27.111438+0000","last_undegraded":"2024-04-25T11:27:27.111438+0000","last_fullsized":"2024-04-25T11:27:27.111438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577698+0000","last_change":"2024-04-25T11:27:25.576615+0000","last_active":"2024-04-25T11:27:26.577698+0000","last_peered":"2024-04-25T11:27:26.577698+0000","last_clean":"2024-04-25T11:27:26.577698+0000","last_became_active":"2024-04-25T11:27:25.575901+0000","last_became_peered":"2024-04-25T11:27:25.575901+0000","last_unstale":"2024-04-25T11:27:26.577698+0000","last_undegraded":"2024-04-25T11:27:26.577698+0000","last_fullsized":"2024-04-25T11:27:26.577698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:18.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578454+0000","last_change":"2024-04-25T11:27:25.571985+0000","last_active":"2024-04-25T11:27:26.578454+0000","last_peered":"2024-04-25T11:27:26.578454+0000","last_clean":"2024-04-25T11:27:26.578454+0000","last_became_active":"2024-04-25T11:27:25.571508+0000","last_became_peered":"2024-04-25T11:27:25.571508+0000","last_unstale":"2024-04-25T11:27:26.578454+0000","last_undegraded":"2024-04-25T11:27:26.578454+0000","last_fullsized":"2024-04-25T11:27:26.578454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109007+0000","last_change":"2024-04-25T11:27:25.589844+0000","last_active":"2024-04-25T11:27:27.109007+0000","last_peered":"2024-04-25T11:27:27.109007+0000","last_clean":"2024-04-25T11:27:27.109007+0000","last_became_active":"2024-04-25T11:27:25.589511+0000","last_became_peered":"2024-04-25T11:27:25.589511+0000","last_unstale":"2024-04-25T11:27:27.109007+0000","last_undegraded":"2024-04-25T11:27:27.109007+0000","last_fullsized":"2024-04-25T11:27:27.109007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:39.805793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108504+0000","last_change":"2024-04-25T11:27:25.574670+0000","last_active":"2024-04-25T11:27:27.108504+0000","last_peered":"2024-04-25T11:27:27.108504+0000","last_clean":"2024-04-25T11:27:27.108504+0000","last_became_active":"2024-04-25T11:27:25.573712+0000","last_became_peered":"2024-04-25T11:27:25.573712+0000","last_unstale":"2024-04-25T11:27:27.108504+0000","last_undegraded":"2024-04-25T11:27:27.108504+0000","last_fullsized":"2024-04-25T11:27:27.108504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.861673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109484+0000","last_change":"2024-04-25T11:27:25.603039+0000","last_active":"2024-04-25T11:27:27.109484+0000","last_peered":"2024-04-25T11:27:27.109484+0000","last_clean":"2024-04-25T11:27:27.109484+0000","last_became_active":"2024-04-25T11:27:25.592097+0000","last_became_peered":"2024-04-25T11:27:25.592097+0000","last_unstale":"2024-04-25T11:27:27.109484+0000","last_undegraded":"2024-04-25T11:27:27.109484+0000","last_fullsized":"2024-04-25T11:27:27.109484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:47.730373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110810+0000","last_change":"2024-04-25T11:27:25.591095+0000","last_active":"2024-04-25T11:27:27.110810+0000","last_peered":"2024-04-25T11:27:27.110810+0000","last_clean":"2024-04-25T11:27:27.110810+0000","last_became_active":"2024-04-25T11:27:25.590722+0000","last_became_peered":"2024-04-25T11:27:25.590722+0000","last_unstale":"2024-04-25T11:27:27.110810+0000","last_undegraded":"2024-04-25T11:27:27.110810+0000","last_fullsized":"2024-04-25T11:27:27.110810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110340+0000","last_change":"2024-04-25T11:27:25.582044+0000","last_active":"2024-04-25T11:27:27.110340+0000","last_peered":"2024-04-25T11:27:27.110340+0000","last_clean":"2024-04-25T11:27:27.110340+0000","last_became_active":"2024-04-25T11:27:25.581715+0000","last_became_peered":"2024-04-25T11:27:25.581715+0000","last_unstale":"2024-04-25T11:27:27.110340+0000","last_undegraded":"2024-04-25T11:27:27.110340+0000","last_fullsized":"2024-04-25T11:27:27.110340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577467+0000","last_change":"2024-04-25T11:27:25.580436+0000","last_active":"2024-04-25T11:27:26.577467+0000","last_peered":"2024-04-25T11:27:26.577467+0000","last_clean":"2024-04-25T11:27:26.577467+0000","last_became_active":"2024-04-25T11:27:25.577298+0000","last_became_peered":"2024-04-25T11:27:25.577298+0000","last_unstale":"2024-04-25T11:27:26.577467+0000","last_undegraded":"2024-04-25T11:27:26.577467+0000","last_fullsized":"2024-04-25T11:27:26.577467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:44.489171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108911+0000","last_change":"2024-04-25T11:27:25.578299+0000","last_active":"2024-04-25T11:27:27.108911+0000","last_peered":"2024-04-25T11:27:27.108911+0000","last_clean":"2024-04-25T11:27:27.108911+0000","last_became_active":"2024-04-25T11:27:25.577785+0000","last_became_peered":"2024-04-25T11:27:25.577785+0000","last_unstale":"2024-04-25T11:27:27.108911+0000","last_undegraded":"2024-04-25T11:27:27.108911+0000","last_fullsized":"2024-04-25T11:27:27.108911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113353+0000","last_change":"2024-04-25T11:27:25.599893+0000","last_active":"2024-04-25T11:27:27.113353+0000","last_peered":"2024-04-25T11:27:27.113353+0000","last_clean":"2024-04-25T11:27:27.113353+0000","last_became_active":"2024-04-25T11:27:25.595315+0000","last_became_peered":"2024-04-25T11:27:25.595315+0000","last_unstale":"2024-04-25T11:27:27.113353+0000","last_undegraded":"2024-04-25T11:27:27.113353+0000","last_fullsized":"2024-04-25T11:27:27.113353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:15.050008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.907339+0000","last_change":"2024-04-25T11:27:25.594310+0000","last_active":"2024-04-25T11:27:26.907339+0000","last_peered":"2024-04-25T11:27:26.907339+0000","last_clean":"2024-04-25T11:27:26.907339+0000","last_became_active":"2024-04-25T11:27:25.593976+0000","last_became_peered":"2024-04-25T11:27:25.593976+0000","last_unstale":"2024-04-25T11:27:26.907339+0000","last_undegraded":"2024-04-25T11:27:26.907339+0000","last_fullsized":"2024-04-25T11:27:26.907339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111503+0000","last_change":"2024-04-25T11:27:25.572942+0000","last_active":"2024-04-25T11:27:27.111503+0000","last_peered":"2024-04-25T11:27:27.111503+0000","last_clean":"2024-04-25T11:27:27.111503+0000","last_became_active":"2024-04-25T11:27:25.572564+0000","last_became_peered":"2024-04-25T11:27:25.572564+0000","last_unstale":"2024-04-25T11:27:27.111503+0000","last_undegraded":"2024-04-25T11:27:27.111503+0000","last_fullsized":"2024-04-25T11:27:27.111503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:32.079241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109999+0000","last_change":"2024-04-25T11:27:25.604247+0000","last_active":"2024-04-25T11:27:27.109999+0000","last_peered":"2024-04-25T11:27:27.109999+0000","last_clean":"2024-04-25T11:27:27.109999+0000","last_became_active":"2024-04-25T11:27:25.599335+0000","last_became_peered":"2024-04-25T11:27:25.599335+0000","last_unstale":"2024-04-25T11:27:27.109999+0000","last_undegraded":"2024-04-25T11:27:27.109999+0000","last_fullsized":"2024-04-25T11:27:27.109999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:09.849779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108856+0000","last_change":"2024-04-25T11:27:25.595766+0000","last_active":"2024-04-25T11:27:27.108856+0000","last_peered":"2024-04-25T11:27:27.108856+0000","last_clean":"2024-04-25T11:27:27.108856+0000","last_became_active":"2024-04-25T11:27:25.595374+0000","last_became_peered":"2024-04-25T11:27:25.595374+0000","last_unstale":"2024-04-25T11:27:27.108856+0000","last_undegraded":"2024-04-25T11:27:27.108856+0000","last_fullsized":"2024-04-25T11:27:27.108856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577329+0000","last_change":"2024-04-25T11:27:25.591577+0000","last_active":"2024-04-25T11:27:26.577329+0000","last_peered":"2024-04-25T11:27:26.577329+0000","last_clean":"2024-04-25T11:27:26.577329+0000","last_became_active":"2024-04-25T11:27:25.591265+0000","last_became_peered":"2024-04-25T11:27:25.591265+0000","last_unstale":"2024-04-25T11:27:26.577329+0000","last_undegraded":"2024-04-25T11:27:26.577329+0000","last_fullsized":"2024-04-25T11:27:26.577329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578287+0000","last_change":"2024-04-25T11:27:25.581137+0000","last_active":"2024-04-25T11:27:26.578287+0000","last_peered":"2024-04-25T11:27:26.578287+0000","last_clean":"2024-04-25T11:27:26.578287+0000","last_became_active":"2024-04-25T11:27:25.580856+0000","last_became_peered":"2024-04-25T11:27:25.580856+0000","last_unstale":"2024-04-25T11:27:26.578287+0000","last_undegraded":"2024-04-25T11:27:26.578287+0000","last_fullsized":"2024-04-25T11:27:26.578287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:04.488802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109404+0000","last_change":"2024-04-25T11:27:25.588399+0000","last_active":"2024-04-25T11:27:27.109404+0000","last_peered":"2024-04-25T11:27:27.109404+0000","last_clean":"2024-04-25T11:27:27.109404+0000","last_became_active":"2024-04-25T11:27:25.587283+0000","last_became_peered":"2024-04-25T11:27:25.587283+0000","last_unstale":"2024-04-25T11:27:27.109404+0000","last_undegraded":"2024-04-25T11:27:27.109404+0000","last_fullsized":"2024-04-25T11:27:27.109404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578626+0000","last_change":"2024-04-25T11:27:25.572116+0000","last_active":"2024-04-25T11:27:26.578626+0000","last_peered":"2024-04-25T11:27:26.578626+0000","last_clean":"2024-04-25T11:27:26.578626+0000","last_became_active":"2024-04-25T11:27:25.571814+0000","last_became_peered":"2024-04-25T11:27:25.571814+0000","last_unstale":"2024-04-25T11:27:26.578626+0000","last_undegraded":"2024-04-25T11:27:26.578626+0000","last_fullsized":"2024-04-25T11:27:26.578626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:07.464107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581233+0000","last_change":"2024-04-25T11:27:25.592264+0000","last_active":"2024-04-25T11:27:26.581233+0000","last_peered":"2024-04-25T11:27:26.581233+0000","last_clean":"2024-04-25T11:27:26.581233+0000","last_became_active":"2024-04-25T11:27:25.591919+0000","last_became_peered":"2024-04-25T11:27:25.591919+0000","last_unstale":"2024-04-25T11:27:26.581233+0000","last_undegraded":"2024-04-25T11:27:26.581233+0000","last_fullsized":"2024-04-25T11:27:26.581233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111469+0000","last_change":"2024-04-25T11:27:25.588173+0000","last_active":"2024-04-25T11:27:27.111469+0000","last_peered":"2024-04-25T11:27:27.111469+0000","last_clean":"2024-04-25T11:27:27.111469+0000","last_became_active":"2024-04-25T11:27:25.587836+0000","last_became_peered":"2024-04-25T11:27:25.587836+0000","last_unstale":"2024-04-25T11:27:27.111469+0000","last_undegraded":"2024-04-25T11:27:27.111469+0000","last_fullsized":"2024-04-25T11:27:27.111469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111338+0000","last_change":"2024-04-25T11:27:25.588936+0000","last_active":"2024-04-25T11:27:27.111338+0000","last_peered":"2024-04-25T11:27:27.111338+0000","last_clean":"2024-04-25T11:27:27.111338+0000","last_became_active":"2024-04-25T11:27:25.588514+0000","last_became_peered":"2024-04-25T11:27:25.588514+0000","last_unstale":"2024-04-25T11:27:27.111338+0000","last_undegraded":"2024-04-25T11:27:27.111338+0000","last_fullsized":"2024-04-25T11:27:27.111338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577773+0000","last_change":"2024-04-25T11:27:25.574933+0000","last_active":"2024-04-25T11:27:26.577773+0000","last_peered":"2024-04-25T11:27:26.577773+0000","last_clean":"2024-04-25T11:27:26.577773+0000","last_became_active":"2024-04-25T11:27:25.573752+0000","last_became_peered":"2024-04-25T11:27:25.573752+0000","last_unstale":"2024-04-25T11:27:26.577773+0000","last_undegraded":"2024-04-25T11:27:26.577773+0000","last_fullsized":"2024-04-25T11:27:26.577773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:46.431504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110999+0000","last_change":"2024-04-25T11:27:25.585915+0000","last_active":"2024-04-25T11:27:27.110999+0000","last_peered":"2024-04-25T11:27:27.110999+0000","last_clean":"2024-04-25T11:27:27.110999+0000","last_became_active":"2024-04-25T11:27:25.583985+0000","last_became_peered":"2024-04-25T11:27:25.583985+0000","last_unstale":"2024-04-25T11:27:27.110999+0000","last_undegraded":"2024-04-25T11:27:27.110999+0000","last_fullsized":"2024-04-25T11:27:27.110999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:57.615463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906567+0000","last_change":"2024-04-25T11:27:25.587583+0000","last_active":"2024-04-25T11:27:26.906567+0000","last_peered":"2024-04-25T11:27:26.906567+0000","last_clean":"2024-04-25T11:27:26.906567+0000","last_became_active":"2024-04-25T11:27:25.587318+0000","last_became_peered":"2024-04-25T11:27:25.587318+0000","last_unstale":"2024-04-25T11:27:26.906567+0000","last_undegraded":"2024-04-25T11:27:26.906567+0000","last_fullsized":"2024-04-25T11:27:26.906567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579991+0000","last_change":"2024-04-25T11:27:25.597364+0000","last_active":"2024-04-25T11:27:26.579991+0000","last_peered":"2024-04-25T11:27:26.579991+0000","last_clean":"2024-04-25T11:27:26.579991+0000","last_became_active":"2024-04-25T11:27:25.596928+0000","last_became_peered":"2024-04-25T11:27:25.596928+0000","last_unstale":"2024-04-25T11:27:26.579991+0000","last_undegraded":"2024-04-25T11:27:26.579991+0000","last_fullsized":"2024-04-25T11:27:26.579991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:24.646664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577102+0000","last_change":"2024-04-25T11:27:25.570795+0000","last_active":"2024-04-25T11:27:26.577102+0000","last_peered":"2024-04-25T11:27:26.577102+0000","last_clean":"2024-04-25T11:27:26.577102+0000","last_became_active":"2024-04-25T11:27:25.568542+0000","last_became_peered":"2024-04-25T11:27:25.568542+0000","last_unstale":"2024-04-25T11:27:26.577102+0000","last_undegraded":"2024-04-25T11:27:26.577102+0000","last_fullsized":"2024-04-25T11:27:26.577102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577916+0000","last_change":"2024-04-25T11:27:24.568405+0000","last_active":"2024-04-25T11:27:26.577916+0000","last_peered":"2024-04-25T11:27:26.577916+0000","last_clean":"2024-04-25T11:27:26.577916+0000","last_became_active":"2024-04-25T11:27:24.568062+0000","last_became_peered":"2024-04-25T11:27:24.568062+0000","last_unstale":"2024-04-25T11:27:26.577916+0000","last_undegraded":"2024-04-25T11:27:26.577916+0000","last_fullsized":"2024-04-25T11:27:26.577916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:54.891145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109354+0000","last_change":"2024-04-25T11:27:25.597308+0000","last_active":"2024-04-25T11:27:27.109354+0000","last_peered":"2024-04-25T11:27:27.109354+0000","last_clean":"2024-04-25T11:27:27.109354+0000","last_became_active":"2024-04-25T11:27:25.596943+0000","last_became_peered":"2024-04-25T11:27:25.596943+0000","last_unstale":"2024-04-25T11:27:27.109354+0000","last_undegraded":"2024-04-25T11:27:27.109354+0000","last_fullsized":"2024-04-25T11:27:27.109354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111690+0000","last_change":"2024-04-25T11:27:24.572640+0000","last_active":"2024-04-25T11:27:27.111690+0000","last_peered":"2024-04-25T11:27:27.111690+0000","last_clean":"2024-04-25T11:27:27.111690+0000","last_became_active":"2024-04-25T11:27:24.572119+0000","last_became_peered":"2024-04-25T11:27:24.572119+0000","last_unstale":"2024-04-25T11:27:27.111690+0000","last_undegraded":"2024-04-25T11:27:27.111690+0000","last_fullsized":"2024-04-25T11:27:27.111690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:12.700897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110506+0000","last_change":"2024-04-25T11:27:24.576360+0000","last_active":"2024-04-25T11:27:27.110506+0000","last_peered":"2024-04-25T11:27:27.110506+0000","last_clean":"2024-04-25T11:27:27.110506+0000","last_became_active":"2024-04-25T11:27:24.572890+0000","last_became_peered":"2024-04-25T11:27:24.572890+0000","last_unstale":"2024-04-25T11:27:27.110506+0000","last_undegraded":"2024-04-25T11:27:27.110506+0000","last_fullsized":"2024-04-25T11:27:27.110506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578396+0000","last_change":"2024-04-25T11:27:25.589395+0000","last_active":"2024-04-25T11:27:26.578396+0000","last_peered":"2024-04-25T11:27:26.578396+0000","last_clean":"2024-04-25T11:27:26.578396+0000","last_became_active":"2024-04-25T11:27:25.588593+0000","last_became_peered":"2024-04-25T11:27:25.588593+0000","last_unstale":"2024-04-25T11:27:26.578396+0000","last_undegraded":"2024-04-25T11:27:26.578396+0000","last_fullsized":"2024-04-25T11:27:26.578396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:20.029930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109801+0000","last_change":"2024-04-25T11:27:24.558134+0000","last_active":"2024-04-25T11:27:27.109801+0000","last_peered":"2024-04-25T11:27:27.109801+0000","last_clean":"2024-04-25T11:27:27.109801+0000","last_became_active":"2024-04-25T11:27:24.557406+0000","last_became_peered":"2024-04-25T11:27:24.557406+0000","last_unstale":"2024-04-25T11:27:27.109801+0000","last_undegraded":"2024-04-25T11:27:27.109801+0000","last_fullsized":"2024-04-25T11:27:27.109801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:44.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109507+0000","last_change":"2024-04-25T11:27:25.592503+0000","last_active":"2024-04-25T11:27:27.109507+0000","last_peered":"2024-04-25T11:27:27.109507+0000","last_clean":"2024-04-25T11:27:27.109507+0000","last_became_active":"2024-04-25T11:27:25.592131+0000","last_became_peered":"2024-04-25T11:27:25.592131+0000","last_unstale":"2024-04-25T11:27:27.109507+0000","last_undegraded":"2024-04-25T11:27:27.109507+0000","last_fullsized":"2024-04-25T11:27:27.109507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.022859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578279+0000","last_change":"2024-04-25T11:27:24.572792+0000","last_active":"2024-04-25T11:27:26.578279+0000","last_peered":"2024-04-25T11:27:26.578279+0000","last_clean":"2024-04-25T11:27:26.578279+0000","last_became_active":"2024-04-25T11:27:24.572419+0000","last_became_peered":"2024-04-25T11:27:24.572419+0000","last_unstale":"2024-04-25T11:27:26.578279+0000","last_undegraded":"2024-04-25T11:27:26.578279+0000","last_fullsized":"2024-04-25T11:27:26.578279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581607+0000","last_change":"2024-04-25T11:27:25.590899+0000","last_active":"2024-04-25T11:27:26.581607+0000","last_peered":"2024-04-25T11:27:26.581607+0000","last_clean":"2024-04-25T11:27:26.581607+0000","last_became_active":"2024-04-25T11:27:25.588335+0000","last_became_peered":"2024-04-25T11:27:25.588335+0000","last_unstale":"2024-04-25T11:27:26.581607+0000","last_undegraded":"2024-04-25T11:27:26.581607+0000","last_fullsized":"2024-04-25T11:27:26.581607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109319+0000","last_change":"2024-04-25T11:27:24.568514+0000","last_active":"2024-04-25T11:27:27.109319+0000","last_peered":"2024-04-25T11:27:27.109319+0000","last_clean":"2024-04-25T11:27:27.109319+0000","last_became_active":"2024-04-25T11:27:24.567092+0000","last_became_peered":"2024-04-25T11:27:24.567092+0000","last_unstale":"2024-04-25T11:27:27.109319+0000","last_undegraded":"2024-04-25T11:27:27.109319+0000","last_fullsized":"2024-04-25T11:27:27.109319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579057+0000","last_change":"2024-04-25T11:27:25.577485+0000","last_active":"2024-04-25T11:27:26.579057+0000","last_peered":"2024-04-25T11:27:26.579057+0000","last_clean":"2024-04-25T11:27:26.579057+0000","last_became_active":"2024-04-25T11:27:25.577128+0000","last_became_peered":"2024-04-25T11:27:25.577128+0000","last_unstale":"2024-04-25T11:27:26.579057+0000","last_undegraded":"2024-04-25T11:27:26.579057+0000","last_fullsized":"2024-04-25T11:27:26.579057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:59.613737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906830+0000","last_change":"2024-04-25T11:27:24.581360+0000","last_active":"2024-04-25T11:27:26.906830+0000","last_peered":"2024-04-25T11:27:26.906830+0000","last_clean":"2024-04-25T11:27:26.906830+0000","last_became_active":"2024-04-25T11:27:24.580827+0000","last_became_peered":"2024-04-25T11:27:24.580827+0000","last_unstale":"2024-04-25T11:27:26.906830+0000","last_undegraded":"2024-04-25T11:27:26.906830+0000","last_fullsized":"2024-04-25T11:27:26.906830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108897+0000","last_change":"2024-04-25T11:27:25.585619+0000","last_active":"2024-04-25T11:27:27.108897+0000","last_peered":"2024-04-25T11:27:27.108897+0000","last_clean":"2024-04-25T11:27:27.108897+0000","last_became_active":"2024-04-25T11:27:25.585314+0000","last_became_peered":"2024-04-25T11:27:25.585314+0000","last_unstale":"2024-04-25T11:27:27.108897+0000","last_undegraded":"2024-04-25T11:27:27.108897+0000","last_fullsized":"2024-04-25T11:27:27.108897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581079+0000","last_change":"2024-04-25T11:27:24.573258+0000","last_active":"2024-04-25T11:27:26.581079+0000","last_peered":"2024-04-25T11:27:26.581079+0000","last_clean":"2024-04-25T11:27:26.581079+0000","last_became_active":"2024-04-25T11:27:24.572900+0000","last_became_peered":"2024-04-25T11:27:24.572900+0000","last_unstale":"2024-04-25T11:27:26.581079+0000","last_undegraded":"2024-04-25T11:27:26.581079+0000","last_fullsized":"2024-04-25T11:27:26.581079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108997+0000","last_change":"2024-04-25T11:27:25.597408+0000","last_active":"2024-04-25T11:27:27.108997+0000","last_peered":"2024-04-25T11:27:27.108997+0000","last_clean":"2024-04-25T11:27:27.108997+0000","last_became_active":"2024-04-25T11:27:25.597094+0000","last_became_peered":"2024-04-25T11:27:25.597094+0000","last_unstale":"2024-04-25T11:27:27.108997+0000","last_undegraded":"2024-04-25T11:27:27.108997+0000","last_fullsized":"2024-04-25T11:27:27.108997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:21.541582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906817+0000","last_change":"2024-04-25T11:27:24.577388+0000","last_active":"2024-04-25T11:27:26.906817+0000","last_peered":"2024-04-25T11:27:26.906817+0000","last_clean":"2024-04-25T11:27:26.906817+0000","last_became_active":"2024-04-25T11:27:24.576282+0000","last_became_peered":"2024-04-25T11:27:24.576282+0000","last_unstale":"2024-04-25T11:27:26.906817+0000","last_undegraded":"2024-04-25T11:27:26.906817+0000","last_fullsized":"2024-04-25T11:27:26.906817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110781+0000","last_change":"2024-04-25T11:27:25.580185+0000","last_active":"2024-04-25T11:27:27.110781+0000","last_peered":"2024-04-25T11:27:27.110781+0000","last_clean":"2024-04-25T11:27:27.110781+0000","last_became_active":"2024-04-25T11:27:25.579504+0000","last_became_peered":"2024-04-25T11:27:25.579504+0000","last_unstale":"2024-04-25T11:27:27.110781+0000","last_undegraded":"2024-04-25T11:27:27.110781+0000","last_fullsized":"2024-04-25T11:27:27.110781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:53.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112046+0000","last_change":"2024-04-25T11:27:24.570725+0000","last_active":"2024-04-25T11:27:27.112046+0000","last_peered":"2024-04-25T11:27:27.112046+0000","last_clean":"2024-04-25T11:27:27.112046+0000","last_became_active":"2024-04-25T11:27:24.570329+0000","last_became_peered":"2024-04-25T11:27:24.570329+0000","last_unstale":"2024-04-25T11:27:27.112046+0000","last_undegraded":"2024-04-25T11:27:27.112046+0000","last_fullsized":"2024-04-25T11:27:27.112046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108584+0000","last_change":"2024-04-25T11:27:25.593697+0000","last_active":"2024-04-25T11:27:27.108584+0000","last_peered":"2024-04-25T11:27:27.108584+0000","last_clean":"2024-04-25T11:27:27.108584+0000","last_became_active":"2024-04-25T11:27:25.593321+0000","last_became_peered":"2024-04-25T11:27:25.593321+0000","last_unstale":"2024-04-25T11:27:27.108584+0000","last_undegraded":"2024-04-25T11:27:27.108584+0000","last_fullsized":"2024-04-25T11:27:27.108584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111788+0000","last_change":"2024-04-25T11:27:24.575295+0000","last_active":"2024-04-25T11:27:27.111788+0000","last_peered":"2024-04-25T11:27:27.111788+0000","last_clean":"2024-04-25T11:27:27.111788+0000","last_became_active":"2024-04-25T11:27:24.574577+0000","last_became_peered":"2024-04-25T11:27:24.574577+0000","last_unstale":"2024-04-25T11:27:27.111788+0000","last_undegraded":"2024-04-25T11:27:27.111788+0000","last_fullsized":"2024-04-25T11:27:27.111788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577619+0000","last_change":"2024-04-25T11:27:25.576314+0000","last_active":"2024-04-25T11:27:26.577619+0000","last_peered":"2024-04-25T11:27:26.577619+0000","last_clean":"2024-04-25T11:27:26.577619+0000","last_became_active":"2024-04-25T11:27:25.575841+0000","last_became_peered":"2024-04-25T11:27:25.575841+0000","last_unstale":"2024-04-25T11:27:26.577619+0000","last_undegraded":"2024-04-25T11:27:26.577619+0000","last_fullsized":"2024-04-25T11:27:26.577619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108586+0000","last_change":"2024-04-25T11:27:24.577138+0000","last_active":"2024-04-25T11:27:27.108586+0000","last_peered":"2024-04-25T11:27:27.108586+0000","last_clean":"2024-04-25T11:27:27.108586+0000","last_became_active":"2024-04-25T11:27:24.576752+0000","last_became_peered":"2024-04-25T11:27:24.576752+0000","last_unstale":"2024-04-25T11:27:27.108586+0000","last_undegraded":"2024-04-25T11:27:27.108586+0000","last_fullsized":"2024-04-25T11:27:27.108586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110175+0000","last_change":"2024-04-25T11:27:25.590200+0000","last_active":"2024-04-25T11:27:27.110175+0000","last_peered":"2024-04-25T11:27:27.110175+0000","last_clean":"2024-04-25T11:27:27.110175+0000","last_became_active":"2024-04-25T11:27:25.588485+0000","last_became_peered":"2024-04-25T11:27:25.588485+0000","last_unstale":"2024-04-25T11:27:27.110175+0000","last_undegraded":"2024-04-25T11:27:27.110175+0000","last_fullsized":"2024-04-25T11:27:27.110175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:47.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110702+0000","last_change":"2024-04-25T11:27:24.578005+0000","last_active":"2024-04-25T11:27:27.110702+0000","last_peered":"2024-04-25T11:27:27.110702+0000","last_clean":"2024-04-25T11:27:27.110702+0000","last_became_active":"2024-04-25T11:27:24.577247+0000","last_became_peered":"2024-04-25T11:27:24.577247+0000","last_unstale":"2024-04-25T11:27:27.110702+0000","last_undegraded":"2024-04-25T11:27:27.110702+0000","last_fullsized":"2024-04-25T11:27:27.110702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581702+0000","last_change":"2024-04-25T11:27:25.604626+0000","last_active":"2024-04-25T11:27:26.581702+0000","last_peered":"2024-04-25T11:27:26.581702+0000","last_clean":"2024-04-25T11:27:26.581702+0000","last_became_active":"2024-04-25T11:27:25.604303+0000","last_became_peered":"2024-04-25T11:27:25.604303+0000","last_unstale":"2024-04-25T11:27:26.581702+0000","last_undegraded":"2024-04-25T11:27:26.581702+0000","last_fullsized":"2024-04-25T11:27:26.581702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110067+0000","last_change":"2024-04-25T11:27:24.574248+0000","last_active":"2024-04-25T11:27:27.110067+0000","last_peered":"2024-04-25T11:27:27.110067+0000","last_clean":"2024-04-25T11:27:27.110067+0000","last_became_active":"2024-04-25T11:27:24.573749+0000","last_became_peered":"2024-04-25T11:27:24.573749+0000","last_unstale":"2024-04-25T11:27:27.110067+0000","last_undegraded":"2024-04-25T11:27:27.110067+0000","last_fullsized":"2024-04-25T11:27:27.110067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110355+0000","last_change":"2024-04-25T11:27:25.578264+0000","last_active":"2024-04-25T11:27:27.110355+0000","last_peered":"2024-04-25T11:27:27.110355+0000","last_clean":"2024-04-25T11:27:27.110355+0000","last_became_active":"2024-04-25T11:27:25.577877+0000","last_became_peered":"2024-04-25T11:27:25.577877+0000","last_unstale":"2024-04-25T11:27:27.110355+0000","last_undegraded":"2024-04-25T11:27:27.110355+0000","last_fullsized":"2024-04-25T11:27:27.110355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578765+0000","last_change":"2024-04-25T11:27:24.570309+0000","last_active":"2024-04-25T11:27:26.578765+0000","last_peered":"2024-04-25T11:27:26.578765+0000","last_clean":"2024-04-25T11:27:26.578765+0000","last_became_active":"2024-04-25T11:27:24.568762+0000","last_became_peered":"2024-04-25T11:27:24.568762+0000","last_unstale":"2024-04-25T11:27:26.578765+0000","last_undegraded":"2024-04-25T11:27:26.578765+0000","last_fullsized":"2024-04-25T11:27:26.578765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.461035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110466+0000","last_change":"2024-04-25T11:27:25.592429+0000","last_active":"2024-04-25T11:27:27.110466+0000","last_peered":"2024-04-25T11:27:27.110466+0000","last_clean":"2024-04-25T11:27:27.110466+0000","last_became_active":"2024-04-25T11:27:25.591596+0000","last_became_peered":"2024-04-25T11:27:25.591596+0000","last_unstale":"2024-04-25T11:27:27.110466+0000","last_undegraded":"2024-04-25T11:27:27.110466+0000","last_fullsized":"2024-04-25T11:27:27.110466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.577233+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.576780+0000","last_became_peered":"2024-04-25T11:27:24.576780+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111112+0000","last_change":"2024-04-25T11:27:25.587607+0000","last_active":"2024-04-25T11:27:27.111112+0000","last_peered":"2024-04-25T11:27:27.111112+0000","last_clean":"2024-04-25T11:27:27.111112+0000","last_became_active":"2024-04-25T11:27:25.587311+0000","last_became_peered":"2024-04-25T11:27:25.587311+0000","last_unstale":"2024-04-25T11:27:27.111112+0000","last_undegraded":"2024-04-25T11:27:27.111112+0000","last_fullsized":"2024-04-25T11:27:27.111112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:46.539664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.576671+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:27:26.576671+0000","last_peered":"2024-04-25T11:27:26.576671+0000","last_clean":"2024-04-25T11:27:26.576671+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:27:26.576671+0000","last_undegraded":"2024-04-25T11:27:26.576671+0000","last_fullsized":"2024-04-25T11:27:26.576671+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108857+0000","last_change":"2024-04-25T11:27:24.556284+0000","last_active":"2024-04-25T11:27:27.108857+0000","last_peered":"2024-04-25T11:27:27.108857+0000","last_clean":"2024-04-25T11:27:27.108857+0000","last_became_active":"2024-04-25T11:27:24.555672+0000","last_became_peered":"2024-04-25T11:27:24.555672+0000","last_unstale":"2024-04-25T11:27:27.108857+0000","last_undegraded":"2024-04-25T11:27:27.108857+0000","last_fullsized":"2024-04-25T11:27:27.108857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108843+0000","last_change":"2024-04-25T11:27:25.580847+0000","last_active":"2024-04-25T11:27:27.108843+0000","last_peered":"2024-04-25T11:27:27.108843+0000","last_clean":"2024-04-25T11:27:27.108843+0000","last_became_active":"2024-04-25T11:27:25.580468+0000","last_became_peered":"2024-04-25T11:27:25.580468+0000","last_unstale":"2024-04-25T11:27:27.108843+0000","last_undegraded":"2024-04-25T11:27:27.108843+0000","last_fullsized":"2024-04-25T11:27:27.108843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110149+0000","last_change":"2024-04-25T11:27:24.570617+0000","last_active":"2024-04-25T11:27:27.110149+0000","last_peered":"2024-04-25T11:27:27.110149+0000","last_clean":"2024-04-25T11:27:27.110149+0000","last_became_active":"2024-04-25T11:27:24.570392+0000","last_became_peered":"2024-04-25T11:27:24.570392+0000","last_unstale":"2024-04-25T11:27:27.110149+0000","last_undegraded":"2024-04-25T11:27:27.110149+0000","last_fullsized":"2024-04-25T11:27:27.110149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110158+0000","last_change":"2024-04-25T11:27:25.586750+0000","last_active":"2024-04-25T11:27:27.110158+0000","last_peered":"2024-04-25T11:27:27.110158+0000","last_clean":"2024-04-25T11:27:27.110158+0000","last_became_active":"2024-04-25T11:27:25.582935+0000","last_became_peered":"2024-04-25T11:27:25.582935+0000","last_unstale":"2024-04-25T11:27:27.110158+0000","last_undegraded":"2024-04-25T11:27:27.110158+0000","last_fullsized":"2024-04-25T11:27:27.110158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906813+0000","last_change":"2024-04-25T11:27:24.578213+0000","last_active":"2024-04-25T11:27:26.906813+0000","last_peered":"2024-04-25T11:27:26.906813+0000","last_clean":"2024-04-25T11:27:26.906813+0000","last_became_active":"2024-04-25T11:27:24.576970+0000","last_became_peered":"2024-04-25T11:27:24.576970+0000","last_unstale":"2024-04-25T11:27:26.906813+0000","last_undegraded":"2024-04-25T11:27:26.906813+0000","last_fullsized":"2024-04-25T11:27:26.906813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:53.598617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577741+0000","last_change":"2024-04-25T11:27:25.581442+0000","last_active":"2024-04-25T11:27:26.577741+0000","last_peered":"2024-04-25T11:27:26.577741+0000","last_clean":"2024-04-25T11:27:26.577741+0000","last_became_active":"2024-04-25T11:27:25.579355+0000","last_became_peered":"2024-04-25T11:27:25.579355+0000","last_unstale":"2024-04-25T11:27:26.577741+0000","last_undegraded":"2024-04-25T11:27:26.577741+0000","last_fullsized":"2024-04-25T11:27:26.577741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.751895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112309+0000","last_change":"2024-04-25T11:27:24.565287+0000","last_active":"2024-04-25T11:27:27.112309+0000","last_peered":"2024-04-25T11:27:27.112309+0000","last_clean":"2024-04-25T11:27:27.112309+0000","last_became_active":"2024-04-25T11:27:24.564368+0000","last_became_peered":"2024-04-25T11:27:24.564368+0000","last_unstale":"2024-04-25T11:27:27.112309+0000","last_undegraded":"2024-04-25T11:27:27.112309+0000","last_fullsized":"2024-04-25T11:27:27.112309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111142+0000","last_change":"2024-04-25T11:27:25.604124+0000","last_active":"2024-04-25T11:27:27.111142+0000","last_peered":"2024-04-25T11:27:27.111142+0000","last_clean":"2024-04-25T11:27:27.111142+0000","last_became_active":"2024-04-25T11:27:25.603769+0000","last_became_peered":"2024-04-25T11:27:25.603769+0000","last_unstale":"2024-04-25T11:27:27.111142+0000","last_undegraded":"2024-04-25T11:27:27.111142+0000","last_fullsized":"2024-04-25T11:27:27.111142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577966+0000","last_change":"2024-04-25T11:27:24.572130+0000","last_active":"2024-04-25T11:27:26.577966+0000","last_peered":"2024-04-25T11:27:26.577966+0000","last_clean":"2024-04-25T11:27:26.577966+0000","last_became_active":"2024-04-25T11:27:24.569314+0000","last_became_peered":"2024-04-25T11:27:24.569314+0000","last_unstale":"2024-04-25T11:27:26.577966+0000","last_undegraded":"2024-04-25T11:27:26.577966+0000","last_fullsized":"2024-04-25T11:27:26.577966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:18.304561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580084+0000","last_change":"2024-04-25T11:27:25.597482+0000","last_active":"2024-04-25T11:27:26.580084+0000","last_peered":"2024-04-25T11:27:26.580084+0000","last_clean":"2024-04-25T11:27:26.580084+0000","last_became_active":"2024-04-25T11:27:25.595957+0000","last_became_peered":"2024-04-25T11:27:25.595957+0000","last_unstale":"2024-04-25T11:27:26.580084+0000","last_undegraded":"2024-04-25T11:27:26.580084+0000","last_fullsized":"2024-04-25T11:27:26.580084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.572383+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.572006+0000","last_became_peered":"2024-04-25T11:27:24.572006+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:09.042983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111372+0000","last_change":"2024-04-25T11:27:25.578914+0000","last_active":"2024-04-25T11:27:27.111372+0000","last_peered":"2024-04-25T11:27:27.111372+0000","last_clean":"2024-04-25T11:27:27.111372+0000","last_became_active":"2024-04-25T11:27:25.578699+0000","last_became_peered":"2024-04-25T11:27:25.578699+0000","last_unstale":"2024-04-25T11:27:27.111372+0000","last_undegraded":"2024-04-25T11:27:27.111372+0000","last_fullsized":"2024-04-25T11:27:27.111372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.486686+0000","last_change":"2024-04-25T11:27:24.579698+0000","last_active":"2024-04-25T11:27:27.486686+0000","last_peered":"2024-04-25T11:27:27.486686+0000","last_clean":"2024-04-25T11:27:27.486686+0000","last_became_active":"2024-04-25T11:27:24.575825+0000","last_became_peered":"2024-04-25T11:27:24.575825+0000","last_unstale":"2024-04-25T11:27:27.486686+0000","last_undegraded":"2024-04-25T11:27:27.486686+0000","last_fullsized":"2024-04-25T11:27:27.486686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109333+0000","last_change":"2024-04-25T11:27:25.588979+0000","last_active":"2024-04-25T11:27:27.109333+0000","last_peered":"2024-04-25T11:27:27.109333+0000","last_clean":"2024-04-25T11:27:27.109333+0000","last_became_active":"2024-04-25T11:27:25.588683+0000","last_became_peered":"2024-04-25T11:27:25.588683+0000","last_unstale":"2024-04-25T11:27:27.109333+0000","last_undegraded":"2024-04-25T11:27:27.109333+0000","last_fullsized":"2024-04-25T11:27:27.109333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113417+0000","last_change":"2024-04-25T11:27:24.572521+0000","last_active":"2024-04-25T11:27:27.113417+0000","last_peered":"2024-04-25T11:27:27.113417+0000","last_clean":"2024-04-25T11:27:27.113417+0000","last_became_active":"2024-04-25T11:27:24.572257+0000","last_became_peered":"2024-04-25T11:27:24.572257+0000","last_unstale":"2024-04-25T11:27:27.113417+0000","last_undegraded":"2024-04-25T11:27:27.113417+0000","last_fullsized":"2024-04-25T11:27:27.113417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110515+0000","last_change":"2024-04-25T11:27:25.574450+0000","last_active":"2024-04-25T11:27:27.110515+0000","last_peered":"2024-04-25T11:27:27.110515+0000","last_clean":"2024-04-25T11:27:27.110515+0000","last_became_active":"2024-04-25T11:27:25.573078+0000","last_became_peered":"2024-04-25T11:27:25.573078+0000","last_unstale":"2024-04-25T11:27:27.110515+0000","last_undegraded":"2024-04-25T11:27:27.110515+0000","last_fullsized":"2024-04-25T11:27:27.110515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.823894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.107620+0000","last_change":"2024-04-25T11:27:24.566069+0000","last_active":"2024-04-25T11:27:27.107620+0000","last_peered":"2024-04-25T11:27:27.107620+0000","last_clean":"2024-04-25T11:27:27.107620+0000","last_became_active":"2024-04-25T11:27:24.565642+0000","last_became_peered":"2024-04-25T11:27:24.565642+0000","last_unstale":"2024-04-25T11:27:27.107620+0000","last_undegraded":"2024-04-25T11:27:27.107620+0000","last_fullsized":"2024-04-25T11:27:27.107620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110907+0000","last_change":"2024-04-25T11:27:25.584529+0000","last_active":"2024-04-25T11:27:27.110907+0000","last_peered":"2024-04-25T11:27:27.110907+0000","last_clean":"2024-04-25T11:27:27.110907+0000","last_became_active":"2024-04-25T11:27:25.584130+0000","last_became_peered":"2024-04-25T11:27:25.584130+0000","last_unstale":"2024-04-25T11:27:27.110907+0000","last_undegraded":"2024-04-25T11:27:27.110907+0000","last_fullsized":"2024-04-25T11:27:27.110907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:27.614041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108983+0000","last_change":"2024-04-25T11:27:24.580477+0000","last_active":"2024-04-25T11:27:27.108983+0000","last_peered":"2024-04-25T11:27:27.108983+0000","last_clean":"2024-04-25T11:27:27.108983+0000","last_became_active":"2024-04-25T11:27:24.578736+0000","last_became_peered":"2024-04-25T11:27:24.578736+0000","last_unstale":"2024-04-25T11:27:27.108983+0000","last_undegraded":"2024-04-25T11:27:27.108983+0000","last_fullsized":"2024-04-25T11:27:27.108983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:44.218131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109747+0000","last_change":"2024-04-25T11:27:25.581233+0000","last_active":"2024-04-25T11:27:27.109747+0000","last_peered":"2024-04-25T11:27:27.109747+0000","last_clean":"2024-04-25T11:27:27.109747+0000","last_became_active":"2024-04-25T11:27:25.580073+0000","last_became_peered":"2024-04-25T11:27:25.580073+0000","last_unstale":"2024-04-25T11:27:27.109747+0000","last_undegraded":"2024-04-25T11:27:27.109747+0000","last_fullsized":"2024-04-25T11:27:27.109747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:53.858736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110994+0000","last_change":"2024-04-25T11:27:24.577164+0000","last_active":"2024-04-25T11:27:27.110994+0000","last_peered":"2024-04-25T11:27:27.110994+0000","last_clean":"2024-04-25T11:27:27.110994+0000","last_became_active":"2024-04-25T11:27:24.576795+0000","last_became_peered":"2024-04-25T11:27:24.576795+0000","last_unstale":"2024-04-25T11:27:27.110994+0000","last_undegraded":"2024-04-25T11:27:27.110994+0000","last_fullsized":"2024-04-25T11:27:27.110994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906953+0000","last_change":"2024-04-25T11:27:25.594079+0000","last_active":"2024-04-25T11:27:26.906953+0000","last_peered":"2024-04-25T11:27:26.906953+0000","last_clean":"2024-04-25T11:27:26.906953+0000","last_became_active":"2024-04-25T11:27:25.592965+0000","last_became_peered":"2024-04-25T11:27:25.592965+0000","last_unstale":"2024-04-25T11:27:26.906953+0000","last_undegraded":"2024-04-25T11:27:26.906953+0000","last_fullsized":"2024-04-25T11:27:26.906953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108732+0000","last_change":"2024-04-25T11:27:24.559748+0000","last_active":"2024-04-25T11:27:27.108732+0000","last_peered":"2024-04-25T11:27:27.108732+0000","last_clean":"2024-04-25T11:27:27.108732+0000","last_became_active":"2024-04-25T11:27:24.558116+0000","last_became_peered":"2024-04-25T11:27:24.558116+0000","last_unstale":"2024-04-25T11:27:27.108732+0000","last_undegraded":"2024-04-25T11:27:27.108732+0000","last_fullsized":"2024-04-25T11:27:27.108732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111402+0000","last_change":"2024-04-25T11:27:25.578715+0000","last_active":"2024-04-25T11:27:27.111402+0000","last_peered":"2024-04-25T11:27:27.111402+0000","last_clean":"2024-04-25T11:27:27.111402+0000","last_became_active":"2024-04-25T11:27:25.578355+0000","last_became_peered":"2024-04-25T11:27:25.578355+0000","last_unstale":"2024-04-25T11:27:27.111402+0000","last_undegraded":"2024-04-25T11:27:27.111402+0000","last_fullsized":"2024-04-25T11:27:27.111402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:57.742260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111309+0000","last_change":"2024-04-25T11:27:25.587963+0000","last_active":"2024-04-25T11:27:27.111309+0000","last_peered":"2024-04-25T11:27:27.111309+0000","last_clean":"2024-04-25T11:27:27.111309+0000","last_became_active":"2024-04-25T11:27:25.587609+0000","last_became_peered":"2024-04-25T11:27:25.587609+0000","last_unstale":"2024-04-25T11:27:27.111309+0000","last_undegraded":"2024-04-25T11:27:27.111309+0000","last_fullsized":"2024-04-25T11:27:27.111309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:48.328746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110834+0000","last_change":"2024-04-25T11:27:24.575077+0000","last_active":"2024-04-25T11:27:27.110834+0000","last_peered":"2024-04-25T11:27:27.110834+0000","last_clean":"2024-04-25T11:27:27.110834+0000","last_became_active":"2024-04-25T11:27:24.574000+0000","last_became_peered":"2024-04-25T11:27:24.574000+0000","last_unstale":"2024-04-25T11:27:27.110834+0000","last_undegraded":"2024-04-25T11:27:27.110834+0000","last_fullsized":"2024-04-25T11:27:27.110834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578237+0000","last_change":"2024-04-25T11:27:25.592421+0000","last_active":"2024-04-25T11:27:26.578237+0000","last_peered":"2024-04-25T11:27:26.578237+0000","last_clean":"2024-04-25T11:27:26.578237+0000","last_became_active":"2024-04-25T11:27:25.591913+0000","last_became_peered":"2024-04-25T11:27:25.591913+0000","last_unstale":"2024-04-25T11:27:26.578237+0000","last_undegraded":"2024-04-25T11:27:26.578237+0000","last_fullsized":"2024-04-25T11:27:26.578237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108846+0000","last_change":"2024-04-25T11:27:24.570375+0000","last_active":"2024-04-25T11:27:27.108846+0000","last_peered":"2024-04-25T11:27:27.108846+0000","last_clean":"2024-04-25T11:27:27.108846+0000","last_became_active":"2024-04-25T11:27:24.570081+0000","last_became_peered":"2024-04-25T11:27:24.570081+0000","last_unstale":"2024-04-25T11:27:27.108846+0000","last_undegraded":"2024-04-25T11:27:27.108846+0000","last_fullsized":"2024-04-25T11:27:27.108846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578618+0000","last_change":"2024-04-25T11:27:25.567438+0000","last_active":"2024-04-25T11:27:26.578618+0000","last_peered":"2024-04-25T11:27:26.578618+0000","last_clean":"2024-04-25T11:27:26.578618+0000","last_became_active":"2024-04-25T11:27:25.567114+0000","last_became_peered":"2024-04-25T11:27:25.567114+0000","last_unstale":"2024-04-25T11:27:26.578618+0000","last_undegraded":"2024-04-25T11:27:26.578618+0000","last_fullsized":"2024-04-25T11:27:26.578618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.585689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578452+0000","last_change":"2024-04-25T11:27:24.572110+0000","last_active":"2024-04-25T11:27:26.578452+0000","last_peered":"2024-04-25T11:27:26.578452+0000","last_clean":"2024-04-25T11:27:26.578452+0000","last_became_active":"2024-04-25T11:27:24.571670+0000","last_became_peered":"2024-04-25T11:27:24.571670+0000","last_unstale":"2024-04-25T11:27:26.578452+0000","last_undegraded":"2024-04-25T11:27:26.578452+0000","last_fullsized":"2024-04-25T11:27:26.578452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:20.625965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578558+0000","last_change":"2024-04-25T11:27:25.582462+0000","last_active":"2024-04-25T11:27:26.578558+0000","last_peered":"2024-04-25T11:27:26.578558+0000","last_clean":"2024-04-25T11:27:26.578558+0000","last_became_active":"2024-04-25T11:27:25.581902+0000","last_became_peered":"2024-04-25T11:27:25.581902+0000","last_unstale":"2024-04-25T11:27:26.578558+0000","last_undegraded":"2024-04-25T11:27:26.578558+0000","last_fullsized":"2024-04-25T11:27:26.578558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:42.995062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109835+0000","last_change":"2024-04-25T11:27:24.567065+0000","last_active":"2024-04-25T11:27:27.109835+0000","last_peered":"2024-04-25T11:27:27.109835+0000","last_clean":"2024-04-25T11:27:27.109835+0000","last_became_active":"2024-04-25T11:27:24.566049+0000","last_became_peered":"2024-04-25T11:27:24.566049+0000","last_unstale":"2024-04-25T11:27:27.109835+0000","last_undegraded":"2024-04-25T11:27:27.109835+0000","last_fullsized":"2024-04-25T11:27:27.109835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:27:36.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:27:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.145+0000 7fd529e00640 1 Processor -- start 2024-04-25T11:27:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.145+0000 7fd529e00640 1 -- start start 2024-04-25T11:27:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5240cf2f0 0x7fd5240b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd52415a810 0x7fd5240b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5241608d0 0x7fd524160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd52406f870 con 0x7fd5241608d0 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd524161220 con 0x7fd52415a810 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5240b4e90 con 0x7fd5240cf2f0 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5241608d0 0x7fd524160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5241608d0 0x7fd524160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49148/0 (socket says 172.21.15.19:49148) 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.147+0000 7fd529400640 1 -- 172.21.15.19:0/3770390580 learned_addr learned my addr 172.21.15.19:0/3770390580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd528a00640 1 --2- 172.21.15.19:0/3770390580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5240cf2f0 0x7fd5240b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd529400640 1 -- 172.21.15.19:0/3770390580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5240cf2f0 msgr2=0x7fd5240b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd529400640 1 --2- 172.21.15.19:0/3770390580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5240cf2f0 0x7fd5240b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd529400640 1 -- 172.21.15.19:0/3770390580 >> v2:172.21.15.169:3300/0 conn(0x7fd52415a810 msgr2=0x7fd5240b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd523e00640 1 --2- 172.21.15.19:0/3770390580 >> v2:172.21.15.169:3300/0 conn(0x7fd52415a810 0x7fd5240b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd529400640 1 --2- 172.21.15.19:0/3770390580 >> v2:172.21.15.169:3300/0 conn(0x7fd52415a810 0x7fd5240b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd529400640 1 -- 172.21.15.19:0/3770390580 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5240b74d0 con 0x7fd5241608d0 2024-04-25T11:27:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.148+0000 7fd523e00640 1 --2- 172.21.15.19:0/3770390580 >> v2:172.21.15.169:3300/0 conn(0x7fd52415a810 0x7fd5240b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.149+0000 7fd529400640 1 --2- 172.21.15.19:0/3770390580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5241608d0 0x7fd524160ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd52406e010 tx=0x7fd51c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d79bc6ed0c97ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.149+0000 7fd523400640 1 -- 172.21.15.19:0/3770390580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c037070 con 0x7fd5241608d0 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd523400640 1 -- 172.21.15.19:0/3770390580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c032440 con 0x7fd5241608d0 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd523400640 1 -- 172.21.15.19:0/3770390580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c032810 con 0x7fd5241608d0 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 -- 172.21.15.19:0/3770390580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5241608d0 msgr2=0x7fd524160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 --2- 172.21.15.19:0/3770390580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5241608d0 0x7fd524160ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd52406e010 tx=0x7fd51c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 -- 172.21.15.19:0/3770390580 shutdown_connections 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 --2- 172.21.15.19:0/3770390580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5241608d0 0x7fd524160ce0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 --2- 172.21.15.19:0/3770390580 >> v2:172.21.15.169:3300/0 conn(0x7fd52415a810 0x7fd5240b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 --2- 172.21.15.19:0/3770390580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5240cf2f0 0x7fd5240b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 -- 172.21.15.19:0/3770390580 >> 172.21.15.19:0/3770390580 conn(0x7fd5240d1720 msgr2=0x7fd5240d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 -- 172.21.15.19:0/3770390580 shutdown_connections 2024-04-25T11:27:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.150+0000 7fd529e00640 1 -- 172.21.15.19:0/3770390580 wait complete. 2024-04-25T11:27:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.152+0000 7fd529e00640 1 Processor -- start 2024-04-25T11:27:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.153+0000 7fd529e00640 1 -- start start 2024-04-25T11:27:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd529e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5240cf2f0 0x7fd524148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd529e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52415a810 0x7fd52414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd529e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5241608d0 0x7fd52414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd529e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd52406f500 con 0x7fd5240cf2f0 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd529e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd52406f220 con 0x7fd52415a810 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd529e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd52415ac20 con 0x7fd5241608d0 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd528a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5240cf2f0 0x7fd524148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd528a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5240cf2f0 0x7fd524148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49154/0 (socket says 172.21.15.19:49154) 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.154+0000 7fd528a00640 1 -- 172.21.15.19:0/1578834788 learned_addr learned my addr 172.21.15.19:0/1578834788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd523e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52415a810 0x7fd52414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd528a00640 1 -- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5241608d0 msgr2=0x7fd52414f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd528a00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5241608d0 0x7fd52414f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd528a00640 1 -- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52415a810 msgr2=0x7fd52414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd528a00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52415a810 0x7fd52414f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd528a00640 1 -- 172.21.15.19:0/1578834788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52414d8e0 con 0x7fd5240cf2f0 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd528a00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5240cf2f0 0x7fd524148c10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd51400c820 tx=0x7fd51400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd520c00640 1 -- 172.21.15.19:0/1578834788 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514015070 con 0x7fd5240cf2f0 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd520c00640 1 -- 172.21.15.19:0/1578834788 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd514004020 con 0x7fd5240cf2f0 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd520c00640 1 -- 172.21.15.19:0/1578834788 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140043f0 con 0x7fd5240cf2f0 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52414fdf0 con 0x7fd5240cf2f0 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.155+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52414dbc0 con 0x7fd5240cf2f0 2024-04-25T11:27:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.157+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd524050740 con 0x7fd5240cf2f0 2024-04-25T11:27:37.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.158+0000 7fd520c00640 1 -- 172.21.15.19:0/1578834788 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd514007830 con 0x7fd5240cf2f0 2024-04-25T11:27:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.159+0000 7fd520c00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5040744e0 0x7fd5040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.159+0000 7fd520c00640 1 -- 172.21.15.19:0/1578834788 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd514094fd0 con 0x7fd5240cf2f0 2024-04-25T11:27:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.159+0000 7fd523e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5040744e0 0x7fd5040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.160+0000 7fd523e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5040744e0 0x7fd5040769f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd510007130 tx=0x7fd510007060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.163+0000 7fd520c00640 1 -- 172.21.15.19:0/1578834788 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd51405f750 con 0x7fd5240cf2f0 2024-04-25T11:27:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.352+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd52414fc30 con 0x7fd5040744e0 2024-04-25T11:27:37.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.370+0000 7fd520c00640 1 -- 172.21.15.19:0/1578834788 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7fd51000ee60 con 0x7fd5040744e0 2024-04-25T11:27:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:37.366 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5040744e0 msgr2=0x7fd5040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5040744e0 0x7fd5040769f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd510007130 tx=0x7fd510007060 comp rx=0 tx=0).stop 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5240cf2f0 msgr2=0x7fd524148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5240cf2f0 0x7fd524148c10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd51400c820 tx=0x7fd51400cce0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 shutdown_connections 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5040744e0 0x7fd5040769f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd52415a810 0x7fd52414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5241608d0 0x7fd52414f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 --2- 172.21.15.19:0/1578834788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5240cf2f0 0x7fd524148c10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 >> 172.21.15.19:0/1578834788 conn(0x7fd5240d1720 msgr2=0x7fd524073a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 shutdown_connections 2024-04-25T11:27:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.376+0000 7fd529e00640 1 -- 172.21.15.19:0/1578834788 wait complete. 2024-04-25T11:27:37.394 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-25T11:27:36.438974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219116,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755604,"statfs":{"total":773094113280,"available":772869738496,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004589"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109908+0000","last_change":"2024-04-25T11:27:25.576865+0000","last_active":"2024-04-25T11:27:27.109908+0000","last_peered":"2024-04-25T11:27:27.109908+0000","last_clean":"2024-04-25T11:27:27.109908+0000","last_became_active":"2024-04-25T11:27:25.576473+0000","last_became_peered":"2024-04-25T11:27:25.576473+0000","last_unstale":"2024-04-25T11:27:27.109908+0000","last_undegraded":"2024-04-25T11:27:27.109908+0000","last_fullsized":"2024-04-25T11:27:27.109908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578302+0000","last_change":"2024-04-25T11:27:25.591722+0000","last_active":"2024-04-25T11:27:26.578302+0000","last_peered":"2024-04-25T11:27:26.578302+0000","last_clean":"2024-04-25T11:27:26.578302+0000","last_became_active":"2024-04-25T11:27:25.591379+0000","last_became_peered":"2024-04-25T11:27:25.591379+0000","last_unstale":"2024-04-25T11:27:26.578302+0000","last_undegraded":"2024-04-25T11:27:26.578302+0000","last_fullsized":"2024-04-25T11:27:26.578302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:06.430829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578349+0000","last_change":"2024-04-25T11:27:25.571047+0000","last_active":"2024-04-25T11:27:26.578349+0000","last_peered":"2024-04-25T11:27:26.578349+0000","last_clean":"2024-04-25T11:27:26.578349+0000","last_became_active":"2024-04-25T11:27:25.570667+0000","last_became_peered":"2024-04-25T11:27:25.570667+0000","last_unstale":"2024-04-25T11:27:26.578349+0000","last_undegraded":"2024-04-25T11:27:26.578349+0000","last_fullsized":"2024-04-25T11:27:26.578349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:32.385239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580734+0000","last_change":"2024-04-25T11:27:25.580512+0000","last_active":"2024-04-25T11:27:26.580734+0000","last_peered":"2024-04-25T11:27:26.580734+0000","last_clean":"2024-04-25T11:27:26.580734+0000","last_became_active":"2024-04-25T11:27:25.579802+0000","last_became_peered":"2024-04-25T11:27:25.579802+0000","last_unstale":"2024-04-25T11:27:26.580734+0000","last_undegraded":"2024-04-25T11:27:26.580734+0000","last_fullsized":"2024-04-25T11:27:26.580734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109585+0000","last_change":"2024-04-25T11:27:25.578185+0000","last_active":"2024-04-25T11:27:27.109585+0000","last_peered":"2024-04-25T11:27:27.109585+0000","last_clean":"2024-04-25T11:27:27.109585+0000","last_became_active":"2024-04-25T11:27:25.577850+0000","last_became_peered":"2024-04-25T11:27:25.577850+0000","last_unstale":"2024-04-25T11:27:27.109585+0000","last_undegraded":"2024-04-25T11:27:27.109585+0000","last_fullsized":"2024-04-25T11:27:27.109585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:20.885446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579999+0000","last_change":"2024-04-25T11:27:25.581557+0000","last_active":"2024-04-25T11:27:26.579999+0000","last_peered":"2024-04-25T11:27:26.579999+0000","last_clean":"2024-04-25T11:27:26.579999+0000","last_became_active":"2024-04-25T11:27:25.580913+0000","last_became_peered":"2024-04-25T11:27:25.580913+0000","last_unstale":"2024-04-25T11:27:26.579999+0000","last_undegraded":"2024-04-25T11:27:26.579999+0000","last_fullsized":"2024-04-25T11:27:26.579999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111438+0000","last_change":"2024-04-25T11:27:25.581148+0000","last_active":"2024-04-25T11:27:27.111438+0000","last_peered":"2024-04-25T11:27:27.111438+0000","last_clean":"2024-04-25T11:27:27.111438+0000","last_became_active":"2024-04-25T11:27:25.580733+0000","last_became_peered":"2024-04-25T11:27:25.580733+0000","last_unstale":"2024-04-25T11:27:27.111438+0000","last_undegraded":"2024-04-25T11:27:27.111438+0000","last_fullsized":"2024-04-25T11:27:27.111438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577698+0000","last_change":"2024-04-25T11:27:25.576615+0000","last_active":"2024-04-25T11:27:26.577698+0000","last_peered":"2024-04-25T11:27:26.577698+0000","last_clean":"2024-04-25T11:27:26.577698+0000","last_became_active":"2024-04-25T11:27:25.575901+0000","last_became_peered":"2024-04-25T11:27:25.575901+0000","last_unstale":"2024-04-25T11:27:26.577698+0000","last_undegraded":"2024-04-25T11:27:26.577698+0000","last_fullsized":"2024-04-25T11:27:26.577698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:18.148141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578454+0000","last_change":"2024-04-25T11:27:25.571985+0000","last_active":"2024-04-25T11:27:26.578454+0000","last_peered":"2024-04-25T11:27:26.578454+0000","last_clean":"2024-04-25T11:27:26.578454+0000","last_became_active":"2024-04-25T11:27:25.571508+0000","last_became_peered":"2024-04-25T11:27:25.571508+0000","last_unstale":"2024-04-25T11:27:26.578454+0000","last_undegraded":"2024-04-25T11:27:26.578454+0000","last_fullsized":"2024-04-25T11:27:26.578454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109007+0000","last_change":"2024-04-25T11:27:25.589844+0000","last_active":"2024-04-25T11:27:27.109007+0000","last_peered":"2024-04-25T11:27:27.109007+0000","last_clean":"2024-04-25T11:27:27.109007+0000","last_became_active":"2024-04-25T11:27:25.589511+0000","last_became_peered":"2024-04-25T11:27:25.589511+0000","last_unstale":"2024-04-25T11:27:27.109007+0000","last_undegraded":"2024-04-25T11:27:27.109007+0000","last_fullsized":"2024-04-25T11:27:27.109007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:39.805793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108504+0000","last_change":"2024-04-25T11:27:25.574670+0000","last_active":"2024-04-25T11:27:27.108504+0000","last_peered":"2024-04-25T11:27:27.108504+0000","last_clean":"2024-04-25T11:27:27.108504+0000","last_became_active":"2024-04-25T11:27:25.573712+0000","last_became_peered":"2024-04-25T11:27:25.573712+0000","last_unstale":"2024-04-25T11:27:27.108504+0000","last_undegraded":"2024-04-25T11:27:27.108504+0000","last_fullsized":"2024-04-25T11:27:27.108504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:58.861673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109484+0000","last_change":"2024-04-25T11:27:25.603039+0000","last_active":"2024-04-25T11:27:27.109484+0000","last_peered":"2024-04-25T11:27:27.109484+0000","last_clean":"2024-04-25T11:27:27.109484+0000","last_became_active":"2024-04-25T11:27:25.592097+0000","last_became_peered":"2024-04-25T11:27:25.592097+0000","last_unstale":"2024-04-25T11:27:27.109484+0000","last_undegraded":"2024-04-25T11:27:27.109484+0000","last_fullsized":"2024-04-25T11:27:27.109484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:47.730373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110810+0000","last_change":"2024-04-25T11:27:25.591095+0000","last_active":"2024-04-25T11:27:27.110810+0000","last_peered":"2024-04-25T11:27:27.110810+0000","last_clean":"2024-04-25T11:27:27.110810+0000","last_became_active":"2024-04-25T11:27:25.590722+0000","last_became_peered":"2024-04-25T11:27:25.590722+0000","last_unstale":"2024-04-25T11:27:27.110810+0000","last_undegraded":"2024-04-25T11:27:27.110810+0000","last_fullsized":"2024-04-25T11:27:27.110810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110340+0000","last_change":"2024-04-25T11:27:25.582044+0000","last_active":"2024-04-25T11:27:27.110340+0000","last_peered":"2024-04-25T11:27:27.110340+0000","last_clean":"2024-04-25T11:27:27.110340+0000","last_became_active":"2024-04-25T11:27:25.581715+0000","last_became_peered":"2024-04-25T11:27:25.581715+0000","last_unstale":"2024-04-25T11:27:27.110340+0000","last_undegraded":"2024-04-25T11:27:27.110340+0000","last_fullsized":"2024-04-25T11:27:27.110340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577467+0000","last_change":"2024-04-25T11:27:25.580436+0000","last_active":"2024-04-25T11:27:26.577467+0000","last_peered":"2024-04-25T11:27:26.577467+0000","last_clean":"2024-04-25T11:27:26.577467+0000","last_became_active":"2024-04-25T11:27:25.577298+0000","last_became_peered":"2024-04-25T11:27:25.577298+0000","last_unstale":"2024-04-25T11:27:26.577467+0000","last_undegraded":"2024-04-25T11:27:26.577467+0000","last_fullsized":"2024-04-25T11:27:26.577467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:44.489171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108911+0000","last_change":"2024-04-25T11:27:25.578299+0000","last_active":"2024-04-25T11:27:27.108911+0000","last_peered":"2024-04-25T11:27:27.108911+0000","last_clean":"2024-04-25T11:27:27.108911+0000","last_became_active":"2024-04-25T11:27:25.577785+0000","last_became_peered":"2024-04-25T11:27:25.577785+0000","last_unstale":"2024-04-25T11:27:27.108911+0000","last_undegraded":"2024-04-25T11:27:27.108911+0000","last_fullsized":"2024-04-25T11:27:27.108911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113353+0000","last_change":"2024-04-25T11:27:25.599893+0000","last_active":"2024-04-25T11:27:27.113353+0000","last_peered":"2024-04-25T11:27:27.113353+0000","last_clean":"2024-04-25T11:27:27.113353+0000","last_became_active":"2024-04-25T11:27:25.595315+0000","last_became_peered":"2024-04-25T11:27:25.595315+0000","last_unstale":"2024-04-25T11:27:27.113353+0000","last_undegraded":"2024-04-25T11:27:27.113353+0000","last_fullsized":"2024-04-25T11:27:27.113353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:15.050008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.907339+0000","last_change":"2024-04-25T11:27:25.594310+0000","last_active":"2024-04-25T11:27:26.907339+0000","last_peered":"2024-04-25T11:27:26.907339+0000","last_clean":"2024-04-25T11:27:26.907339+0000","last_became_active":"2024-04-25T11:27:25.593976+0000","last_became_peered":"2024-04-25T11:27:25.593976+0000","last_unstale":"2024-04-25T11:27:26.907339+0000","last_undegraded":"2024-04-25T11:27:26.907339+0000","last_fullsized":"2024-04-25T11:27:26.907339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111503+0000","last_change":"2024-04-25T11:27:25.572942+0000","last_active":"2024-04-25T11:27:27.111503+0000","last_peered":"2024-04-25T11:27:27.111503+0000","last_clean":"2024-04-25T11:27:27.111503+0000","last_became_active":"2024-04-25T11:27:25.572564+0000","last_became_peered":"2024-04-25T11:27:25.572564+0000","last_unstale":"2024-04-25T11:27:27.111503+0000","last_undegraded":"2024-04-25T11:27:27.111503+0000","last_fullsized":"2024-04-25T11:27:27.111503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:32.079241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109999+0000","last_change":"2024-04-25T11:27:25.604247+0000","last_active":"2024-04-25T11:27:27.109999+0000","last_peered":"2024-04-25T11:27:27.109999+0000","last_clean":"2024-04-25T11:27:27.109999+0000","last_became_active":"2024-04-25T11:27:25.599335+0000","last_became_peered":"2024-04-25T11:27:25.599335+0000","last_unstale":"2024-04-25T11:27:27.109999+0000","last_undegraded":"2024-04-25T11:27:27.109999+0000","last_fullsized":"2024-04-25T11:27:27.109999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:09.849779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108856+0000","last_change":"2024-04-25T11:27:25.595766+0000","last_active":"2024-04-25T11:27:27.108856+0000","last_peered":"2024-04-25T11:27:27.108856+0000","last_clean":"2024-04-25T11:27:27.108856+0000","last_became_active":"2024-04-25T11:27:25.595374+0000","last_became_peered":"2024-04-25T11:27:25.595374+0000","last_unstale":"2024-04-25T11:27:27.108856+0000","last_undegraded":"2024-04-25T11:27:27.108856+0000","last_fullsized":"2024-04-25T11:27:27.108856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577329+0000","last_change":"2024-04-25T11:27:25.591577+0000","last_active":"2024-04-25T11:27:26.577329+0000","last_peered":"2024-04-25T11:27:26.577329+0000","last_clean":"2024-04-25T11:27:26.577329+0000","last_became_active":"2024-04-25T11:27:25.591265+0000","last_became_peered":"2024-04-25T11:27:25.591265+0000","last_unstale":"2024-04-25T11:27:26.577329+0000","last_undegraded":"2024-04-25T11:27:26.577329+0000","last_fullsized":"2024-04-25T11:27:26.577329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:45.673566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578287+0000","last_change":"2024-04-25T11:27:25.581137+0000","last_active":"2024-04-25T11:27:26.578287+0000","last_peered":"2024-04-25T11:27:26.578287+0000","last_clean":"2024-04-25T11:27:26.578287+0000","last_became_active":"2024-04-25T11:27:25.580856+0000","last_became_peered":"2024-04-25T11:27:25.580856+0000","last_unstale":"2024-04-25T11:27:26.578287+0000","last_undegraded":"2024-04-25T11:27:26.578287+0000","last_fullsized":"2024-04-25T11:27:26.578287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:04.488802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109404+0000","last_change":"2024-04-25T11:27:25.588399+0000","last_active":"2024-04-25T11:27:27.109404+0000","last_peered":"2024-04-25T11:27:27.109404+0000","last_clean":"2024-04-25T11:27:27.109404+0000","last_became_active":"2024-04-25T11:27:25.587283+0000","last_became_peered":"2024-04-25T11:27:25.587283+0000","last_unstale":"2024-04-25T11:27:27.109404+0000","last_undegraded":"2024-04-25T11:27:27.109404+0000","last_fullsized":"2024-04-25T11:27:27.109404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:54.110116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578626+0000","last_change":"2024-04-25T11:27:25.572116+0000","last_active":"2024-04-25T11:27:26.578626+0000","last_peered":"2024-04-25T11:27:26.578626+0000","last_clean":"2024-04-25T11:27:26.578626+0000","last_became_active":"2024-04-25T11:27:25.571814+0000","last_became_peered":"2024-04-25T11:27:25.571814+0000","last_unstale":"2024-04-25T11:27:26.578626+0000","last_undegraded":"2024-04-25T11:27:26.578626+0000","last_fullsized":"2024-04-25T11:27:26.578626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:07.464107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581233+0000","last_change":"2024-04-25T11:27:25.592264+0000","last_active":"2024-04-25T11:27:26.581233+0000","last_peered":"2024-04-25T11:27:26.581233+0000","last_clean":"2024-04-25T11:27:26.581233+0000","last_became_active":"2024-04-25T11:27:25.591919+0000","last_became_peered":"2024-04-25T11:27:25.591919+0000","last_unstale":"2024-04-25T11:27:26.581233+0000","last_undegraded":"2024-04-25T11:27:26.581233+0000","last_fullsized":"2024-04-25T11:27:26.581233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111469+0000","last_change":"2024-04-25T11:27:25.588173+0000","last_active":"2024-04-25T11:27:27.111469+0000","last_peered":"2024-04-25T11:27:27.111469+0000","last_clean":"2024-04-25T11:27:27.111469+0000","last_became_active":"2024-04-25T11:27:25.587836+0000","last_became_peered":"2024-04-25T11:27:25.587836+0000","last_unstale":"2024-04-25T11:27:27.111469+0000","last_undegraded":"2024-04-25T11:27:27.111469+0000","last_fullsized":"2024-04-25T11:27:27.111469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111338+0000","last_change":"2024-04-25T11:27:25.588936+0000","last_active":"2024-04-25T11:27:27.111338+0000","last_peered":"2024-04-25T11:27:27.111338+0000","last_clean":"2024-04-25T11:27:27.111338+0000","last_became_active":"2024-04-25T11:27:25.588514+0000","last_became_peered":"2024-04-25T11:27:25.588514+0000","last_unstale":"2024-04-25T11:27:27.111338+0000","last_undegraded":"2024-04-25T11:27:27.111338+0000","last_fullsized":"2024-04-25T11:27:27.111338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:48.671161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577773+0000","last_change":"2024-04-25T11:27:25.574933+0000","last_active":"2024-04-25T11:27:26.577773+0000","last_peered":"2024-04-25T11:27:26.577773+0000","last_clean":"2024-04-25T11:27:26.577773+0000","last_became_active":"2024-04-25T11:27:25.573752+0000","last_became_peered":"2024-04-25T11:27:25.573752+0000","last_unstale":"2024-04-25T11:27:26.577773+0000","last_undegraded":"2024-04-25T11:27:26.577773+0000","last_fullsized":"2024-04-25T11:27:26.577773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:46.431504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110999+0000","last_change":"2024-04-25T11:27:25.585915+0000","last_active":"2024-04-25T11:27:27.110999+0000","last_peered":"2024-04-25T11:27:27.110999+0000","last_clean":"2024-04-25T11:27:27.110999+0000","last_became_active":"2024-04-25T11:27:25.583985+0000","last_became_peered":"2024-04-25T11:27:25.583985+0000","last_unstale":"2024-04-25T11:27:27.110999+0000","last_undegraded":"2024-04-25T11:27:27.110999+0000","last_fullsized":"2024-04-25T11:27:27.110999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:57.615463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906567+0000","last_change":"2024-04-25T11:27:25.587583+0000","last_active":"2024-04-25T11:27:26.906567+0000","last_peered":"2024-04-25T11:27:26.906567+0000","last_clean":"2024-04-25T11:27:26.906567+0000","last_became_active":"2024-04-25T11:27:25.587318+0000","last_became_peered":"2024-04-25T11:27:25.587318+0000","last_unstale":"2024-04-25T11:27:26.906567+0000","last_undegraded":"2024-04-25T11:27:26.906567+0000","last_fullsized":"2024-04-25T11:27:26.906567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:10.059424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579991+0000","last_change":"2024-04-25T11:27:25.597364+0000","last_active":"2024-04-25T11:27:26.579991+0000","last_peered":"2024-04-25T11:27:26.579991+0000","last_clean":"2024-04-25T11:27:26.579991+0000","last_became_active":"2024-04-25T11:27:25.596928+0000","last_became_peered":"2024-04-25T11:27:25.596928+0000","last_unstale":"2024-04-25T11:27:26.579991+0000","last_undegraded":"2024-04-25T11:27:26.579991+0000","last_fullsized":"2024-04-25T11:27:26.579991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:24.646664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577102+0000","last_change":"2024-04-25T11:27:25.570795+0000","last_active":"2024-04-25T11:27:26.577102+0000","last_peered":"2024-04-25T11:27:26.577102+0000","last_clean":"2024-04-25T11:27:26.577102+0000","last_became_active":"2024-04-25T11:27:25.568542+0000","last_became_peered":"2024-04-25T11:27:25.568542+0000","last_unstale":"2024-04-25T11:27:26.577102+0000","last_undegraded":"2024-04-25T11:27:26.577102+0000","last_fullsized":"2024-04-25T11:27:26.577102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577916+0000","last_change":"2024-04-25T11:27:24.568405+0000","last_active":"2024-04-25T11:27:26.577916+0000","last_peered":"2024-04-25T11:27:26.577916+0000","last_clean":"2024-04-25T11:27:26.577916+0000","last_became_active":"2024-04-25T11:27:24.568062+0000","last_became_peered":"2024-04-25T11:27:24.568062+0000","last_unstale":"2024-04-25T11:27:26.577916+0000","last_undegraded":"2024-04-25T11:27:26.577916+0000","last_fullsized":"2024-04-25T11:27:26.577916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:54.891145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109354+0000","last_change":"2024-04-25T11:27:25.597308+0000","last_active":"2024-04-25T11:27:27.109354+0000","last_peered":"2024-04-25T11:27:27.109354+0000","last_clean":"2024-04-25T11:27:27.109354+0000","last_became_active":"2024-04-25T11:27:25.596943+0000","last_became_peered":"2024-04-25T11:27:25.596943+0000","last_unstale":"2024-04-25T11:27:27.109354+0000","last_undegraded":"2024-04-25T11:27:27.109354+0000","last_fullsized":"2024-04-25T11:27:27.109354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:25.965995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111690+0000","last_change":"2024-04-25T11:27:24.572640+0000","last_active":"2024-04-25T11:27:27.111690+0000","last_peered":"2024-04-25T11:27:27.111690+0000","last_clean":"2024-04-25T11:27:27.111690+0000","last_became_active":"2024-04-25T11:27:24.572119+0000","last_became_peered":"2024-04-25T11:27:24.572119+0000","last_unstale":"2024-04-25T11:27:27.111690+0000","last_undegraded":"2024-04-25T11:27:27.111690+0000","last_fullsized":"2024-04-25T11:27:27.111690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:12.700897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110506+0000","last_change":"2024-04-25T11:27:24.576360+0000","last_active":"2024-04-25T11:27:27.110506+0000","last_peered":"2024-04-25T11:27:27.110506+0000","last_clean":"2024-04-25T11:27:27.110506+0000","last_became_active":"2024-04-25T11:27:24.572890+0000","last_became_peered":"2024-04-25T11:27:24.572890+0000","last_unstale":"2024-04-25T11:27:27.110506+0000","last_undegraded":"2024-04-25T11:27:27.110506+0000","last_fullsized":"2024-04-25T11:27:27.110506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578396+0000","last_change":"2024-04-25T11:27:25.589395+0000","last_active":"2024-04-25T11:27:26.578396+0000","last_peered":"2024-04-25T11:27:26.578396+0000","last_clean":"2024-04-25T11:27:26.578396+0000","last_became_active":"2024-04-25T11:27:25.588593+0000","last_became_peered":"2024-04-25T11:27:25.588593+0000","last_unstale":"2024-04-25T11:27:26.578396+0000","last_undegraded":"2024-04-25T11:27:26.578396+0000","last_fullsized":"2024-04-25T11:27:26.578396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:20.029930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109801+0000","last_change":"2024-04-25T11:27:24.558134+0000","last_active":"2024-04-25T11:27:27.109801+0000","last_peered":"2024-04-25T11:27:27.109801+0000","last_clean":"2024-04-25T11:27:27.109801+0000","last_became_active":"2024-04-25T11:27:24.557406+0000","last_became_peered":"2024-04-25T11:27:24.557406+0000","last_unstale":"2024-04-25T11:27:27.109801+0000","last_undegraded":"2024-04-25T11:27:27.109801+0000","last_fullsized":"2024-04-25T11:27:27.109801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:44.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109507+0000","last_change":"2024-04-25T11:27:25.592503+0000","last_active":"2024-04-25T11:27:27.109507+0000","last_peered":"2024-04-25T11:27:27.109507+0000","last_clean":"2024-04-25T11:27:27.109507+0000","last_became_active":"2024-04-25T11:27:25.592131+0000","last_became_peered":"2024-04-25T11:27:25.592131+0000","last_unstale":"2024-04-25T11:27:27.109507+0000","last_undegraded":"2024-04-25T11:27:27.109507+0000","last_fullsized":"2024-04-25T11:27:27.109507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.022859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578279+0000","last_change":"2024-04-25T11:27:24.572792+0000","last_active":"2024-04-25T11:27:26.578279+0000","last_peered":"2024-04-25T11:27:26.578279+0000","last_clean":"2024-04-25T11:27:26.578279+0000","last_became_active":"2024-04-25T11:27:24.572419+0000","last_became_peered":"2024-04-25T11:27:24.572419+0000","last_unstale":"2024-04-25T11:27:26.578279+0000","last_undegraded":"2024-04-25T11:27:26.578279+0000","last_fullsized":"2024-04-25T11:27:26.578279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581607+0000","last_change":"2024-04-25T11:27:25.590899+0000","last_active":"2024-04-25T11:27:26.581607+0000","last_peered":"2024-04-25T11:27:26.581607+0000","last_clean":"2024-04-25T11:27:26.581607+0000","last_became_active":"2024-04-25T11:27:25.588335+0000","last_became_peered":"2024-04-25T11:27:25.588335+0000","last_unstale":"2024-04-25T11:27:26.581607+0000","last_undegraded":"2024-04-25T11:27:26.581607+0000","last_fullsized":"2024-04-25T11:27:26.581607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109319+0000","last_change":"2024-04-25T11:27:24.568514+0000","last_active":"2024-04-25T11:27:27.109319+0000","last_peered":"2024-04-25T11:27:27.109319+0000","last_clean":"2024-04-25T11:27:27.109319+0000","last_became_active":"2024-04-25T11:27:24.567092+0000","last_became_peered":"2024-04-25T11:27:24.567092+0000","last_unstale":"2024-04-25T11:27:27.109319+0000","last_undegraded":"2024-04-25T11:27:27.109319+0000","last_fullsized":"2024-04-25T11:27:27.109319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.579057+0000","last_change":"2024-04-25T11:27:25.577485+0000","last_active":"2024-04-25T11:27:26.579057+0000","last_peered":"2024-04-25T11:27:26.579057+0000","last_clean":"2024-04-25T11:27:26.579057+0000","last_became_active":"2024-04-25T11:27:25.577128+0000","last_became_peered":"2024-04-25T11:27:25.577128+0000","last_unstale":"2024-04-25T11:27:26.579057+0000","last_undegraded":"2024-04-25T11:27:26.579057+0000","last_fullsized":"2024-04-25T11:27:26.579057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:59.613737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906830+0000","last_change":"2024-04-25T11:27:24.581360+0000","last_active":"2024-04-25T11:27:26.906830+0000","last_peered":"2024-04-25T11:27:26.906830+0000","last_clean":"2024-04-25T11:27:26.906830+0000","last_became_active":"2024-04-25T11:27:24.580827+0000","last_became_peered":"2024-04-25T11:27:24.580827+0000","last_unstale":"2024-04-25T11:27:26.906830+0000","last_undegraded":"2024-04-25T11:27:26.906830+0000","last_fullsized":"2024-04-25T11:27:26.906830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:31.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108897+0000","last_change":"2024-04-25T11:27:25.585619+0000","last_active":"2024-04-25T11:27:27.108897+0000","last_peered":"2024-04-25T11:27:27.108897+0000","last_clean":"2024-04-25T11:27:27.108897+0000","last_became_active":"2024-04-25T11:27:25.585314+0000","last_became_peered":"2024-04-25T11:27:25.585314+0000","last_unstale":"2024-04-25T11:27:27.108897+0000","last_undegraded":"2024-04-25T11:27:27.108897+0000","last_fullsized":"2024-04-25T11:27:27.108897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581079+0000","last_change":"2024-04-25T11:27:24.573258+0000","last_active":"2024-04-25T11:27:26.581079+0000","last_peered":"2024-04-25T11:27:26.581079+0000","last_clean":"2024-04-25T11:27:26.581079+0000","last_became_active":"2024-04-25T11:27:24.572900+0000","last_became_peered":"2024-04-25T11:27:24.572900+0000","last_unstale":"2024-04-25T11:27:26.581079+0000","last_undegraded":"2024-04-25T11:27:26.581079+0000","last_fullsized":"2024-04-25T11:27:26.581079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108997+0000","last_change":"2024-04-25T11:27:25.597408+0000","last_active":"2024-04-25T11:27:27.108997+0000","last_peered":"2024-04-25T11:27:27.108997+0000","last_clean":"2024-04-25T11:27:27.108997+0000","last_became_active":"2024-04-25T11:27:25.597094+0000","last_became_peered":"2024-04-25T11:27:25.597094+0000","last_unstale":"2024-04-25T11:27:27.108997+0000","last_undegraded":"2024-04-25T11:27:27.108997+0000","last_fullsized":"2024-04-25T11:27:27.108997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:21.541582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906817+0000","last_change":"2024-04-25T11:27:24.577388+0000","last_active":"2024-04-25T11:27:26.906817+0000","last_peered":"2024-04-25T11:27:26.906817+0000","last_clean":"2024-04-25T11:27:26.906817+0000","last_became_active":"2024-04-25T11:27:24.576282+0000","last_became_peered":"2024-04-25T11:27:24.576282+0000","last_unstale":"2024-04-25T11:27:26.906817+0000","last_undegraded":"2024-04-25T11:27:26.906817+0000","last_fullsized":"2024-04-25T11:27:26.906817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:45.142222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110781+0000","last_change":"2024-04-25T11:27:25.580185+0000","last_active":"2024-04-25T11:27:27.110781+0000","last_peered":"2024-04-25T11:27:27.110781+0000","last_clean":"2024-04-25T11:27:27.110781+0000","last_became_active":"2024-04-25T11:27:25.579504+0000","last_became_peered":"2024-04-25T11:27:25.579504+0000","last_unstale":"2024-04-25T11:27:27.110781+0000","last_undegraded":"2024-04-25T11:27:27.110781+0000","last_fullsized":"2024-04-25T11:27:27.110781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:53.838676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112046+0000","last_change":"2024-04-25T11:27:24.570725+0000","last_active":"2024-04-25T11:27:27.112046+0000","last_peered":"2024-04-25T11:27:27.112046+0000","last_clean":"2024-04-25T11:27:27.112046+0000","last_became_active":"2024-04-25T11:27:24.570329+0000","last_became_peered":"2024-04-25T11:27:24.570329+0000","last_unstale":"2024-04-25T11:27:27.112046+0000","last_undegraded":"2024-04-25T11:27:27.112046+0000","last_fullsized":"2024-04-25T11:27:27.112046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108584+0000","last_change":"2024-04-25T11:27:25.593697+0000","last_active":"2024-04-25T11:27:27.108584+0000","last_peered":"2024-04-25T11:27:27.108584+0000","last_clean":"2024-04-25T11:27:27.108584+0000","last_became_active":"2024-04-25T11:27:25.593321+0000","last_became_peered":"2024-04-25T11:27:25.593321+0000","last_unstale":"2024-04-25T11:27:27.108584+0000","last_undegraded":"2024-04-25T11:27:27.108584+0000","last_fullsized":"2024-04-25T11:27:27.108584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111788+0000","last_change":"2024-04-25T11:27:24.575295+0000","last_active":"2024-04-25T11:27:27.111788+0000","last_peered":"2024-04-25T11:27:27.111788+0000","last_clean":"2024-04-25T11:27:27.111788+0000","last_became_active":"2024-04-25T11:27:24.574577+0000","last_became_peered":"2024-04-25T11:27:24.574577+0000","last_unstale":"2024-04-25T11:27:27.111788+0000","last_undegraded":"2024-04-25T11:27:27.111788+0000","last_fullsized":"2024-04-25T11:27:27.111788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577619+0000","last_change":"2024-04-25T11:27:25.576314+0000","last_active":"2024-04-25T11:27:26.577619+0000","last_peered":"2024-04-25T11:27:26.577619+0000","last_clean":"2024-04-25T11:27:26.577619+0000","last_became_active":"2024-04-25T11:27:25.575841+0000","last_became_peered":"2024-04-25T11:27:25.575841+0000","last_unstale":"2024-04-25T11:27:26.577619+0000","last_undegraded":"2024-04-25T11:27:26.577619+0000","last_fullsized":"2024-04-25T11:27:26.577619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:12.438851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108586+0000","last_change":"2024-04-25T11:27:24.577138+0000","last_active":"2024-04-25T11:27:27.108586+0000","last_peered":"2024-04-25T11:27:27.108586+0000","last_clean":"2024-04-25T11:27:27.108586+0000","last_became_active":"2024-04-25T11:27:24.576752+0000","last_became_peered":"2024-04-25T11:27:24.576752+0000","last_unstale":"2024-04-25T11:27:27.108586+0000","last_undegraded":"2024-04-25T11:27:27.108586+0000","last_fullsized":"2024-04-25T11:27:27.108586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110175+0000","last_change":"2024-04-25T11:27:25.590200+0000","last_active":"2024-04-25T11:27:27.110175+0000","last_peered":"2024-04-25T11:27:27.110175+0000","last_clean":"2024-04-25T11:27:27.110175+0000","last_became_active":"2024-04-25T11:27:25.588485+0000","last_became_peered":"2024-04-25T11:27:25.588485+0000","last_unstale":"2024-04-25T11:27:27.110175+0000","last_undegraded":"2024-04-25T11:27:27.110175+0000","last_fullsized":"2024-04-25T11:27:27.110175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:47.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110702+0000","last_change":"2024-04-25T11:27:24.578005+0000","last_active":"2024-04-25T11:27:27.110702+0000","last_peered":"2024-04-25T11:27:27.110702+0000","last_clean":"2024-04-25T11:27:27.110702+0000","last_became_active":"2024-04-25T11:27:24.577247+0000","last_became_peered":"2024-04-25T11:27:24.577247+0000","last_unstale":"2024-04-25T11:27:27.110702+0000","last_undegraded":"2024-04-25T11:27:27.110702+0000","last_fullsized":"2024-04-25T11:27:27.110702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.581702+0000","last_change":"2024-04-25T11:27:25.604626+0000","last_active":"2024-04-25T11:27:26.581702+0000","last_peered":"2024-04-25T11:27:26.581702+0000","last_clean":"2024-04-25T11:27:26.581702+0000","last_became_active":"2024-04-25T11:27:25.604303+0000","last_became_peered":"2024-04-25T11:27:25.604303+0000","last_unstale":"2024-04-25T11:27:26.581702+0000","last_undegraded":"2024-04-25T11:27:26.581702+0000","last_fullsized":"2024-04-25T11:27:26.581702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:13.717337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110067+0000","last_change":"2024-04-25T11:27:24.574248+0000","last_active":"2024-04-25T11:27:27.110067+0000","last_peered":"2024-04-25T11:27:27.110067+0000","last_clean":"2024-04-25T11:27:27.110067+0000","last_became_active":"2024-04-25T11:27:24.573749+0000","last_became_peered":"2024-04-25T11:27:24.573749+0000","last_unstale":"2024-04-25T11:27:27.110067+0000","last_undegraded":"2024-04-25T11:27:27.110067+0000","last_fullsized":"2024-04-25T11:27:27.110067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110355+0000","last_change":"2024-04-25T11:27:25.578264+0000","last_active":"2024-04-25T11:27:27.110355+0000","last_peered":"2024-04-25T11:27:27.110355+0000","last_clean":"2024-04-25T11:27:27.110355+0000","last_became_active":"2024-04-25T11:27:25.577877+0000","last_became_peered":"2024-04-25T11:27:25.577877+0000","last_unstale":"2024-04-25T11:27:27.110355+0000","last_undegraded":"2024-04-25T11:27:27.110355+0000","last_fullsized":"2024-04-25T11:27:27.110355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:32.471319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578765+0000","last_change":"2024-04-25T11:27:24.570309+0000","last_active":"2024-04-25T11:27:26.578765+0000","last_peered":"2024-04-25T11:27:26.578765+0000","last_clean":"2024-04-25T11:27:26.578765+0000","last_became_active":"2024-04-25T11:27:24.568762+0000","last_became_peered":"2024-04-25T11:27:24.568762+0000","last_unstale":"2024-04-25T11:27:26.578765+0000","last_undegraded":"2024-04-25T11:27:26.578765+0000","last_fullsized":"2024-04-25T11:27:26.578765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.461035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110466+0000","last_change":"2024-04-25T11:27:25.592429+0000","last_active":"2024-04-25T11:27:27.110466+0000","last_peered":"2024-04-25T11:27:27.110466+0000","last_clean":"2024-04-25T11:27:27.110466+0000","last_became_active":"2024-04-25T11:27:25.591596+0000","last_became_peered":"2024-04-25T11:27:25.591596+0000","last_unstale":"2024-04-25T11:27:27.110466+0000","last_undegraded":"2024-04-25T11:27:27.110466+0000","last_fullsized":"2024-04-25T11:27:27.110466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:39.907225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.577233+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.576780+0000","last_became_peered":"2024-04-25T11:27:24.576780+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.008753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111112+0000","last_change":"2024-04-25T11:27:25.587607+0000","last_active":"2024-04-25T11:27:27.111112+0000","last_peered":"2024-04-25T11:27:27.111112+0000","last_clean":"2024-04-25T11:27:27.111112+0000","last_became_active":"2024-04-25T11:27:25.587311+0000","last_became_peered":"2024-04-25T11:27:25.587311+0000","last_unstale":"2024-04-25T11:27:27.111112+0000","last_undegraded":"2024-04-25T11:27:27.111112+0000","last_fullsized":"2024-04-25T11:27:27.111112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:46.539664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.576671+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:27:26.576671+0000","last_peered":"2024-04-25T11:27:26.576671+0000","last_clean":"2024-04-25T11:27:26.576671+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:27:26.576671+0000","last_undegraded":"2024-04-25T11:27:26.576671+0000","last_fullsized":"2024-04-25T11:27:26.576671+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108857+0000","last_change":"2024-04-25T11:27:24.556284+0000","last_active":"2024-04-25T11:27:27.108857+0000","last_peered":"2024-04-25T11:27:27.108857+0000","last_clean":"2024-04-25T11:27:27.108857+0000","last_became_active":"2024-04-25T11:27:24.555672+0000","last_became_peered":"2024-04-25T11:27:24.555672+0000","last_unstale":"2024-04-25T11:27:27.108857+0000","last_undegraded":"2024-04-25T11:27:27.108857+0000","last_fullsized":"2024-04-25T11:27:27.108857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108843+0000","last_change":"2024-04-25T11:27:25.580847+0000","last_active":"2024-04-25T11:27:27.108843+0000","last_peered":"2024-04-25T11:27:27.108843+0000","last_clean":"2024-04-25T11:27:27.108843+0000","last_became_active":"2024-04-25T11:27:25.580468+0000","last_became_peered":"2024-04-25T11:27:25.580468+0000","last_unstale":"2024-04-25T11:27:27.108843+0000","last_undegraded":"2024-04-25T11:27:27.108843+0000","last_fullsized":"2024-04-25T11:27:27.108843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:41.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110149+0000","last_change":"2024-04-25T11:27:24.570617+0000","last_active":"2024-04-25T11:27:27.110149+0000","last_peered":"2024-04-25T11:27:27.110149+0000","last_clean":"2024-04-25T11:27:27.110149+0000","last_became_active":"2024-04-25T11:27:24.570392+0000","last_became_peered":"2024-04-25T11:27:24.570392+0000","last_unstale":"2024-04-25T11:27:27.110149+0000","last_undegraded":"2024-04-25T11:27:27.110149+0000","last_fullsized":"2024-04-25T11:27:27.110149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:22.073290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110158+0000","last_change":"2024-04-25T11:27:25.586750+0000","last_active":"2024-04-25T11:27:27.110158+0000","last_peered":"2024-04-25T11:27:27.110158+0000","last_clean":"2024-04-25T11:27:27.110158+0000","last_became_active":"2024-04-25T11:27:25.582935+0000","last_became_peered":"2024-04-25T11:27:25.582935+0000","last_unstale":"2024-04-25T11:27:27.110158+0000","last_undegraded":"2024-04-25T11:27:27.110158+0000","last_fullsized":"2024-04-25T11:27:27.110158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906813+0000","last_change":"2024-04-25T11:27:24.578213+0000","last_active":"2024-04-25T11:27:26.906813+0000","last_peered":"2024-04-25T11:27:26.906813+0000","last_clean":"2024-04-25T11:27:26.906813+0000","last_became_active":"2024-04-25T11:27:24.576970+0000","last_became_peered":"2024-04-25T11:27:24.576970+0000","last_unstale":"2024-04-25T11:27:26.906813+0000","last_undegraded":"2024-04-25T11:27:26.906813+0000","last_fullsized":"2024-04-25T11:27:26.906813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:53.598617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577741+0000","last_change":"2024-04-25T11:27:25.581442+0000","last_active":"2024-04-25T11:27:26.577741+0000","last_peered":"2024-04-25T11:27:26.577741+0000","last_clean":"2024-04-25T11:27:26.577741+0000","last_became_active":"2024-04-25T11:27:25.579355+0000","last_became_peered":"2024-04-25T11:27:25.579355+0000","last_unstale":"2024-04-25T11:27:26.577741+0000","last_undegraded":"2024-04-25T11:27:26.577741+0000","last_fullsized":"2024-04-25T11:27:26.577741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:54.751895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.112309+0000","last_change":"2024-04-25T11:27:24.565287+0000","last_active":"2024-04-25T11:27:27.112309+0000","last_peered":"2024-04-25T11:27:27.112309+0000","last_clean":"2024-04-25T11:27:27.112309+0000","last_became_active":"2024-04-25T11:27:24.564368+0000","last_became_peered":"2024-04-25T11:27:24.564368+0000","last_unstale":"2024-04-25T11:27:27.112309+0000","last_undegraded":"2024-04-25T11:27:27.112309+0000","last_fullsized":"2024-04-25T11:27:27.112309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:40.475478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111142+0000","last_change":"2024-04-25T11:27:25.604124+0000","last_active":"2024-04-25T11:27:27.111142+0000","last_peered":"2024-04-25T11:27:27.111142+0000","last_clean":"2024-04-25T11:27:27.111142+0000","last_became_active":"2024-04-25T11:27:25.603769+0000","last_became_peered":"2024-04-25T11:27:25.603769+0000","last_unstale":"2024-04-25T11:27:27.111142+0000","last_undegraded":"2024-04-25T11:27:27.111142+0000","last_fullsized":"2024-04-25T11:27:27.111142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:39.944990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.577966+0000","last_change":"2024-04-25T11:27:24.572130+0000","last_active":"2024-04-25T11:27:26.577966+0000","last_peered":"2024-04-25T11:27:26.577966+0000","last_clean":"2024-04-25T11:27:26.577966+0000","last_became_active":"2024-04-25T11:27:24.569314+0000","last_became_peered":"2024-04-25T11:27:24.569314+0000","last_unstale":"2024-04-25T11:27:26.577966+0000","last_undegraded":"2024-04-25T11:27:26.577966+0000","last_fullsized":"2024-04-25T11:27:26.577966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:18.304561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.580084+0000","last_change":"2024-04-25T11:27:25.597482+0000","last_active":"2024-04-25T11:27:26.580084+0000","last_peered":"2024-04-25T11:27:26.580084+0000","last_clean":"2024-04-25T11:27:26.580084+0000","last_became_active":"2024-04-25T11:27:25.595957+0000","last_became_peered":"2024-04-25T11:27:25.595957+0000","last_unstale":"2024-04-25T11:27:26.580084+0000","last_undegraded":"2024-04-25T11:27:26.580084+0000","last_fullsized":"2024-04-25T11:27:26.580084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111659+0000","last_change":"2024-04-25T11:27:24.572383+0000","last_active":"2024-04-25T11:27:27.111659+0000","last_peered":"2024-04-25T11:27:27.111659+0000","last_clean":"2024-04-25T11:27:27.111659+0000","last_became_active":"2024-04-25T11:27:24.572006+0000","last_became_peered":"2024-04-25T11:27:24.572006+0000","last_unstale":"2024-04-25T11:27:27.111659+0000","last_undegraded":"2024-04-25T11:27:27.111659+0000","last_fullsized":"2024-04-25T11:27:27.111659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:09.042983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111372+0000","last_change":"2024-04-25T11:27:25.578914+0000","last_active":"2024-04-25T11:27:27.111372+0000","last_peered":"2024-04-25T11:27:27.111372+0000","last_clean":"2024-04-25T11:27:27.111372+0000","last_became_active":"2024-04-25T11:27:25.578699+0000","last_became_peered":"2024-04-25T11:27:25.578699+0000","last_unstale":"2024-04-25T11:27:27.111372+0000","last_undegraded":"2024-04-25T11:27:27.111372+0000","last_fullsized":"2024-04-25T11:27:27.111372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.486686+0000","last_change":"2024-04-25T11:27:24.579698+0000","last_active":"2024-04-25T11:27:27.486686+0000","last_peered":"2024-04-25T11:27:27.486686+0000","last_clean":"2024-04-25T11:27:27.486686+0000","last_became_active":"2024-04-25T11:27:24.575825+0000","last_became_peered":"2024-04-25T11:27:24.575825+0000","last_unstale":"2024-04-25T11:27:27.486686+0000","last_undegraded":"2024-04-25T11:27:27.486686+0000","last_fullsized":"2024-04-25T11:27:27.486686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109333+0000","last_change":"2024-04-25T11:27:25.588979+0000","last_active":"2024-04-25T11:27:27.109333+0000","last_peered":"2024-04-25T11:27:27.109333+0000","last_clean":"2024-04-25T11:27:27.109333+0000","last_became_active":"2024-04-25T11:27:25.588683+0000","last_became_peered":"2024-04-25T11:27:25.588683+0000","last_unstale":"2024-04-25T11:27:27.109333+0000","last_undegraded":"2024-04-25T11:27:27.109333+0000","last_fullsized":"2024-04-25T11:27:27.109333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:46.894788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.113417+0000","last_change":"2024-04-25T11:27:24.572521+0000","last_active":"2024-04-25T11:27:27.113417+0000","last_peered":"2024-04-25T11:27:27.113417+0000","last_clean":"2024-04-25T11:27:27.113417+0000","last_became_active":"2024-04-25T11:27:24.572257+0000","last_became_peered":"2024-04-25T11:27:24.572257+0000","last_unstale":"2024-04-25T11:27:27.113417+0000","last_undegraded":"2024-04-25T11:27:27.113417+0000","last_fullsized":"2024-04-25T11:27:27.113417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110515+0000","last_change":"2024-04-25T11:27:25.574450+0000","last_active":"2024-04-25T11:27:27.110515+0000","last_peered":"2024-04-25T11:27:27.110515+0000","last_clean":"2024-04-25T11:27:27.110515+0000","last_became_active":"2024-04-25T11:27:25.573078+0000","last_became_peered":"2024-04-25T11:27:25.573078+0000","last_unstale":"2024-04-25T11:27:27.110515+0000","last_undegraded":"2024-04-25T11:27:27.110515+0000","last_fullsized":"2024-04-25T11:27:27.110515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:10.823894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.107620+0000","last_change":"2024-04-25T11:27:24.566069+0000","last_active":"2024-04-25T11:27:27.107620+0000","last_peered":"2024-04-25T11:27:27.107620+0000","last_clean":"2024-04-25T11:27:27.107620+0000","last_became_active":"2024-04-25T11:27:24.565642+0000","last_became_peered":"2024-04-25T11:27:24.565642+0000","last_unstale":"2024-04-25T11:27:27.107620+0000","last_undegraded":"2024-04-25T11:27:27.107620+0000","last_fullsized":"2024-04-25T11:27:27.107620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:22.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110907+0000","last_change":"2024-04-25T11:27:25.584529+0000","last_active":"2024-04-25T11:27:27.110907+0000","last_peered":"2024-04-25T11:27:27.110907+0000","last_clean":"2024-04-25T11:27:27.110907+0000","last_became_active":"2024-04-25T11:27:25.584130+0000","last_became_peered":"2024-04-25T11:27:25.584130+0000","last_unstale":"2024-04-25T11:27:27.110907+0000","last_undegraded":"2024-04-25T11:27:27.110907+0000","last_fullsized":"2024-04-25T11:27:27.110907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:27.614041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108983+0000","last_change":"2024-04-25T11:27:24.580477+0000","last_active":"2024-04-25T11:27:27.108983+0000","last_peered":"2024-04-25T11:27:27.108983+0000","last_clean":"2024-04-25T11:27:27.108983+0000","last_became_active":"2024-04-25T11:27:24.578736+0000","last_became_peered":"2024-04-25T11:27:24.578736+0000","last_unstale":"2024-04-25T11:27:27.108983+0000","last_undegraded":"2024-04-25T11:27:27.108983+0000","last_fullsized":"2024-04-25T11:27:27.108983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:44.218131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109747+0000","last_change":"2024-04-25T11:27:25.581233+0000","last_active":"2024-04-25T11:27:27.109747+0000","last_peered":"2024-04-25T11:27:27.109747+0000","last_clean":"2024-04-25T11:27:27.109747+0000","last_became_active":"2024-04-25T11:27:25.580073+0000","last_became_peered":"2024-04-25T11:27:25.580073+0000","last_unstale":"2024-04-25T11:27:27.109747+0000","last_undegraded":"2024-04-25T11:27:27.109747+0000","last_fullsized":"2024-04-25T11:27:27.109747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:53.858736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110994+0000","last_change":"2024-04-25T11:27:24.577164+0000","last_active":"2024-04-25T11:27:27.110994+0000","last_peered":"2024-04-25T11:27:27.110994+0000","last_clean":"2024-04-25T11:27:27.110994+0000","last_became_active":"2024-04-25T11:27:24.576795+0000","last_became_peered":"2024-04-25T11:27:24.576795+0000","last_unstale":"2024-04-25T11:27:27.110994+0000","last_undegraded":"2024-04-25T11:27:27.110994+0000","last_fullsized":"2024-04-25T11:27:27.110994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:11.172437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.906953+0000","last_change":"2024-04-25T11:27:25.594079+0000","last_active":"2024-04-25T11:27:26.906953+0000","last_peered":"2024-04-25T11:27:26.906953+0000","last_clean":"2024-04-25T11:27:26.906953+0000","last_became_active":"2024-04-25T11:27:25.592965+0000","last_became_peered":"2024-04-25T11:27:25.592965+0000","last_unstale":"2024-04-25T11:27:26.906953+0000","last_undegraded":"2024-04-25T11:27:26.906953+0000","last_fullsized":"2024-04-25T11:27:26.906953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.241109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108732+0000","last_change":"2024-04-25T11:27:24.559748+0000","last_active":"2024-04-25T11:27:27.108732+0000","last_peered":"2024-04-25T11:27:27.108732+0000","last_clean":"2024-04-25T11:27:27.108732+0000","last_became_active":"2024-04-25T11:27:24.558116+0000","last_became_peered":"2024-04-25T11:27:24.558116+0000","last_unstale":"2024-04-25T11:27:27.108732+0000","last_undegraded":"2024-04-25T11:27:27.108732+0000","last_fullsized":"2024-04-25T11:27:27.108732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111402+0000","last_change":"2024-04-25T11:27:25.578715+0000","last_active":"2024-04-25T11:27:27.111402+0000","last_peered":"2024-04-25T11:27:27.111402+0000","last_clean":"2024-04-25T11:27:27.111402+0000","last_became_active":"2024-04-25T11:27:25.578355+0000","last_became_peered":"2024-04-25T11:27:25.578355+0000","last_unstale":"2024-04-25T11:27:27.111402+0000","last_undegraded":"2024-04-25T11:27:27.111402+0000","last_fullsized":"2024-04-25T11:27:27.111402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:57.742260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.111309+0000","last_change":"2024-04-25T11:27:25.587963+0000","last_active":"2024-04-25T11:27:27.111309+0000","last_peered":"2024-04-25T11:27:27.111309+0000","last_clean":"2024-04-25T11:27:27.111309+0000","last_became_active":"2024-04-25T11:27:25.587609+0000","last_became_peered":"2024-04-25T11:27:25.587609+0000","last_unstale":"2024-04-25T11:27:27.111309+0000","last_undegraded":"2024-04-25T11:27:27.111309+0000","last_fullsized":"2024-04-25T11:27:27.111309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:48.328746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.110834+0000","last_change":"2024-04-25T11:27:24.575077+0000","last_active":"2024-04-25T11:27:27.110834+0000","last_peered":"2024-04-25T11:27:27.110834+0000","last_clean":"2024-04-25T11:27:27.110834+0000","last_became_active":"2024-04-25T11:27:24.574000+0000","last_became_peered":"2024-04-25T11:27:24.574000+0000","last_unstale":"2024-04-25T11:27:27.110834+0000","last_undegraded":"2024-04-25T11:27:27.110834+0000","last_fullsized":"2024-04-25T11:27:27.110834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:50.224668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578237+0000","last_change":"2024-04-25T11:27:25.592421+0000","last_active":"2024-04-25T11:27:26.578237+0000","last_peered":"2024-04-25T11:27:26.578237+0000","last_clean":"2024-04-25T11:27:26.578237+0000","last_became_active":"2024-04-25T11:27:25.591913+0000","last_became_peered":"2024-04-25T11:27:25.591913+0000","last_unstale":"2024-04-25T11:27:26.578237+0000","last_undegraded":"2024-04-25T11:27:26.578237+0000","last_fullsized":"2024-04-25T11:27:26.578237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:23.089731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.108846+0000","last_change":"2024-04-25T11:27:24.570375+0000","last_active":"2024-04-25T11:27:27.108846+0000","last_peered":"2024-04-25T11:27:27.108846+0000","last_clean":"2024-04-25T11:27:27.108846+0000","last_became_active":"2024-04-25T11:27:24.570081+0000","last_became_peered":"2024-04-25T11:27:24.570081+0000","last_unstale":"2024-04-25T11:27:27.108846+0000","last_undegraded":"2024-04-25T11:27:27.108846+0000","last_fullsized":"2024-04-25T11:27:27.108846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:52.822235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578618+0000","last_change":"2024-04-25T11:27:25.567438+0000","last_active":"2024-04-25T11:27:26.578618+0000","last_peered":"2024-04-25T11:27:26.578618+0000","last_clean":"2024-04-25T11:27:26.578618+0000","last_became_active":"2024-04-25T11:27:25.567114+0000","last_became_peered":"2024-04-25T11:27:25.567114+0000","last_unstale":"2024-04-25T11:27:26.578618+0000","last_undegraded":"2024-04-25T11:27:26.578618+0000","last_fullsized":"2024-04-25T11:27:26.578618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:54.585689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578452+0000","last_change":"2024-04-25T11:27:24.572110+0000","last_active":"2024-04-25T11:27:26.578452+0000","last_peered":"2024-04-25T11:27:26.578452+0000","last_clean":"2024-04-25T11:27:26.578452+0000","last_became_active":"2024-04-25T11:27:24.571670+0000","last_became_peered":"2024-04-25T11:27:24.571670+0000","last_unstale":"2024-04-25T11:27:26.578452+0000","last_undegraded":"2024-04-25T11:27:26.578452+0000","last_fullsized":"2024-04-25T11:27:26.578452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:20.625965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:26.578558+0000","last_change":"2024-04-25T11:27:25.582462+0000","last_active":"2024-04-25T11:27:26.578558+0000","last_peered":"2024-04-25T11:27:26.578558+0000","last_clean":"2024-04-25T11:27:26.578558+0000","last_became_active":"2024-04-25T11:27:25.581902+0000","last_became_peered":"2024-04-25T11:27:25.581902+0000","last_unstale":"2024-04-25T11:27:26.578558+0000","last_undegraded":"2024-04-25T11:27:26.578558+0000","last_fullsized":"2024-04-25T11:27:26.578558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:24.519847+0000","last_clean_scrub_stamp":"2024-04-25T11:27:24.519847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:42.995062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T11:27:27.109835+0000","last_change":"2024-04-25T11:27:24.567065+0000","last_active":"2024-04-25T11:27:27.109835+0000","last_peered":"2024-04-25T11:27:27.109835+0000","last_clean":"2024-04-25T11:27:27.109835+0000","last_became_active":"2024-04-25T11:27:24.566049+0000","last_became_peered":"2024-04-25T11:27:24.566049+0000","last_unstale":"2024-04-25T11:27:27.109835+0000","last_undegraded":"2024-04-25T11:27:27.109835+0000","last_fullsized":"2024-04-25T11:27:27.109835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:23.503406+0000","last_clean_scrub_stamp":"2024-04-25T11:27:23.503406+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:23.630223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:27:37.398 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:27:37.398 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:27:37.398 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:27:37.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:27:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.727+0000 7fdb6ea00640 1 Processor -- start 2024-04-25T11:27:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.727+0000 7fdb6ea00640 1 -- start start 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb68162060 0x7fdb6806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdb6806db10 0x7fdb68168230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6ea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb681659e0 con 0x7fdb68168770 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdb68162470 con 0x7fdb68162060 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdb68168f60 con 0x7fdb6806db10 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb68162060 0x7fdb6806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdb68168770 0x7fdb6806f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54316/0 (socket says 172.21.15.19:54316) 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb68162060 0x7fdb6806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49158/0 (socket says 172.21.15.19:49158) 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.730+0000 7fdb6e000640 1 -- 172.21.15.19:0/3073753206 learned_addr learned my addr 172.21.15.19:0/3073753206 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.731+0000 7fdb6cc00640 1 --2- 172.21.15.19:0/3073753206 >> v2:172.21.15.169:3300/0 conn(0x7fdb6806db10 0x7fdb68168230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.731+0000 7fdb6d600640 1 -- 172.21.15.19:0/3073753206 >> v2:172.21.15.169:3300/0 conn(0x7fdb6806db10 msgr2=0x7fdb68168230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.731+0000 7fdb6d600640 1 --2- 172.21.15.19:0/3073753206 >> v2:172.21.15.169:3300/0 conn(0x7fdb6806db10 0x7fdb68168230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.731+0000 7fdb6d600640 1 -- 172.21.15.19:0/3073753206 >> v1:172.21.15.19:6789/0 conn(0x7fdb68168770 legacy=0x7fdb6806f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.731+0000 7fdb6d600640 1 -- 172.21.15.19:0/3073753206 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6806df20 con 0x7fdb68162060 2024-04-25T11:27:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.731+0000 7fdb6cc00640 1 --2- 172.21.15.19:0/3073753206 >> v2:172.21.15.169:3300/0 conn(0x7fdb6806db10 0x7fdb68168230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.731+0000 7fdb67e00640 1 -- 172.21.15.19:0/3073753206 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 741434220 0 0) 0x7fdb681659e0 con 0x7fdb68168770 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.732+0000 7fdb6d600640 1 --2- 172.21.15.19:0/3073753206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68162060 0x7fdb6806d5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009d90 tx=0x7fdb5c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea42e91df4a57f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.732+0000 7fdb67e00640 1 -- 172.21.15.19:0/3073753206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c031600 con 0x7fdb68162060 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.732+0000 7fdb67e00640 1 -- 172.21.15.19:0/3073753206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb5c0317a0 con 0x7fdb68162060 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.732+0000 7fdb67e00640 1 -- 172.21.15.19:0/3073753206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c031b70 con 0x7fdb68162060 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 -- 172.21.15.19:0/3073753206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68162060 msgr2=0x7fdb6806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/3073753206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68162060 0x7fdb6806d5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009d90 tx=0x7fdb5c02a610 comp rx=0 tx=0).stop 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 -- 172.21.15.19:0/3073753206 shutdown_connections 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/3073753206 >> v2:172.21.15.169:3300/0 conn(0x7fdb6806db10 0x7fdb68168230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/3073753206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68162060 0x7fdb6806d5d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 -- 172.21.15.19:0/3073753206 >> 172.21.15.19:0/3073753206 conn(0x7fdb68072ee0 msgr2=0x7fdb680732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 -- 172.21.15.19:0/3073753206 shutdown_connections 2024-04-25T11:27:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.733+0000 7fdb6ea00640 1 -- 172.21.15.19:0/3073753206 wait complete. 2024-04-25T11:27:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.735+0000 7fdb6ea00640 1 Processor -- start 2024-04-25T11:27:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.736+0000 7fdb6ea00640 1 -- start start 2024-04-25T11:27:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6806db10 0x7fdb680d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb68162060 0x7fdb6814c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68168770 0x7fdb6806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681651e0 con 0x7fdb68168770 2024-04-25T11:27:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68164f00 con 0x7fdb6806db10 2024-04-25T11:27:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb681654c0 con 0x7fdb68162060 2024-04-25T11:27:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68168770 0x7fdb6806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68168770 0x7fdb6806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49174/0 (socket says 172.21.15.19:49174) 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.737+0000 7fdb6e000640 1 -- 172.21.15.19:0/501580267 learned_addr learned my addr 172.21.15.19:0/501580267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6cc00640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb68162060 0x7fdb6814c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6d600640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6806db10 0x7fdb680d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6e000640 1 -- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb68162060 msgr2=0x7fdb6814c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6e000640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb68162060 0x7fdb6814c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6e000640 1 -- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6806db10 msgr2=0x7fdb680d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6e000640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6806db10 0x7fdb680d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6e000640 1 -- 172.21.15.19:0/501580267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6814ced0 con 0x7fdb68168770 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6d600640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6806db10 0x7fdb680d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6e000640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68168770 0x7fdb6806f130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000c440 tx=0x7fdb6000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb65600640 1 -- 172.21.15.19:0/501580267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600074e0 con 0x7fdb68168770 2024-04-25T11:27:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb65600640 1 -- 172.21.15.19:0/501580267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb600047e0 con 0x7fdb68168770 2024-04-25T11:27:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb65600640 1 -- 172.21.15.19:0/501580267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004bd0 con 0x7fdb68168770 2024-04-25T11:27:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.738+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb681fdc30 con 0x7fdb68168770 2024-04-25T11:27:37.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.739+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb681fe1f0 con 0x7fdb68168770 2024-04-25T11:27:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.740+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb68050740 con 0x7fdb68168770 2024-04-25T11:27:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.741+0000 7fdb65600640 1 -- 172.21.15.19:0/501580267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdb60011070 con 0x7fdb68168770 2024-04-25T11:27:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.742+0000 7fdb65600640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb480744e0 0x7fdb480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.743+0000 7fdb65600640 1 -- 172.21.15.19:0/501580267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdb60094e50 con 0x7fdb68168770 2024-04-25T11:27:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.743+0000 7fdb6d600640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb480744e0 0x7fdb480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.744+0000 7fdb6d600640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb480744e0 0x7fdb480769f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c034000 tx=0x7fdb5c033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:37.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.747+0000 7fdb65600640 1 -- 172.21.15.19:0/501580267 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb6005f550 con 0x7fdb68168770 2024-04-25T11:27:37.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.993+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdb68162740 con 0x7fdb68168770 2024-04-25T11:27:37.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.994+0000 7fdb65600640 1 -- 172.21.15.19:0/501580267 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdb6005ee50 con 0x7fdb68168770 2024-04-25T11:27:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:37.985 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb480744e0 msgr2=0x7fdb480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb480744e0 0x7fdb480769f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c034000 tx=0x7fdb5c033040 comp rx=0 tx=0).stop 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68168770 msgr2=0x7fdb6806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68168770 0x7fdb6806f130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000c440 tx=0x7fdb6000c900 comp rx=0 tx=0).stop 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 shutdown_connections 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb480744e0 0x7fdb480769f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb68168770 0x7fdb6806f130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb68162060 0x7fdb6814c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 --2- 172.21.15.19:0/501580267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6806db10 0x7fdb680d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:37.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.996+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 >> 172.21.15.19:0/501580267 conn(0x7fdb68072ee0 msgr2=0x7fdb680732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:37.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.997+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 shutdown_connections 2024-04-25T11:27:37.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:37.997+0000 7fdb6ea00640 1 -- 172.21.15.19:0/501580267 wait complete. 2024-04-25T11:27:38.017 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:27:38.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:38.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.311+0000 7faf13e00640 1 Processor -- start 2024-04-25T11:27:38.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.311+0000 7faf13e00640 1 -- start start 2024-04-25T11:27:38.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.313+0000 7faf13e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf140d0f10 0x7faf1406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf13e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faf14164e90 0x7faf140d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf13e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf140b2c00 con 0x7faf1406ddf0 2024-04-25T11:27:38.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf13e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faf140b9520 con 0x7faf14164e90 2024-04-25T11:27:38.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf13e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faf1409cce0 con 0x7faf140d0f10 2024-04-25T11:27:38.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf12000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7faf1406ddf0 0x7faf14164780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54328/0 (socket says 172.21.15.19:54328) 2024-04-25T11:27:38.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf12000640 1 -- 172.21.15.19:0/2668510989 learned_addr learned my addr 172.21.15.19:0/2668510989 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:38.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf13400640 1 --2- 172.21.15.19:0/2668510989 >> v2:172.21.15.19:3300/0 conn(0x7faf14164e90 0x7faf140d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf12a00640 1 --2- 172.21.15.19:0/2668510989 >> v2:172.21.15.169:3300/0 conn(0x7faf140d0f10 0x7faf1406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.314+0000 7faf11600640 1 -- 172.21.15.19:0/2668510989 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959394902 0 0) 0x7faf140b2c00 con 0x7faf1406ddf0 2024-04-25T11:27:38.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.315+0000 7faf11600640 1 -- 172.21.15.19:0/2668510989 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faef8003520 con 0x7faf1406ddf0 2024-04-25T11:27:38.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.315+0000 7faf13400640 1 -- 172.21.15.19:0/2668510989 >> v2:172.21.15.169:3300/0 conn(0x7faf140d0f10 msgr2=0x7faf1406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:38.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.315+0000 7faf13400640 1 --2- 172.21.15.19:0/2668510989 >> v2:172.21.15.169:3300/0 conn(0x7faf140d0f10 0x7faf1406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.315+0000 7faf13400640 1 -- 172.21.15.19:0/2668510989 >> v1:172.21.15.19:6789/0 conn(0x7faf1406ddf0 legacy=0x7faf14164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.315+0000 7faf13400640 1 -- 172.21.15.19:0/2668510989 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf140b9240 con 0x7faf14164e90 2024-04-25T11:27:38.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.316+0000 7faf13400640 1 --2- 172.21.15.19:0/2668510989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf14164e90 0x7faf140d1de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf14051770 tx=0x7faf0c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5506f5c7254c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:38.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.316+0000 7faf11600640 1 -- 172.21.15.19:0/2668510989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007740 con 0x7faf14164e90 2024-04-25T11:27:38.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.316+0000 7faf11600640 1 -- 172.21.15.19:0/2668510989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c0078e0 con 0x7faf14164e90 2024-04-25T11:27:38.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.316+0000 7faf11600640 1 -- 172.21.15.19:0/2668510989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007c90 con 0x7faf14164e90 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 -- 172.21.15.19:0/2668510989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf14164e90 msgr2=0x7faf140d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 --2- 172.21.15.19:0/2668510989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf14164e90 0x7faf140d1de0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf14051770 tx=0x7faf0c02a680 comp rx=0 tx=0).stop 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 -- 172.21.15.19:0/2668510989 shutdown_connections 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 --2- 172.21.15.19:0/2668510989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf14164e90 0x7faf140d1de0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 --2- 172.21.15.19:0/2668510989 >> v2:172.21.15.169:3300/0 conn(0x7faf140d0f10 0x7faf1406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 -- 172.21.15.19:0/2668510989 >> 172.21.15.19:0/2668510989 conn(0x7faf140d03f0 msgr2=0x7faf140d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 -- 172.21.15.19:0/2668510989 shutdown_connections 2024-04-25T11:27:38.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.317+0000 7faf13e00640 1 -- 172.21.15.19:0/2668510989 wait complete. 2024-04-25T11:27:38.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.320+0000 7faf13e00640 1 Processor -- start 2024-04-25T11:27:38.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.320+0000 7faf13e00640 1 -- start start 2024-04-25T11:27:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.321+0000 7faf13e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf1406ddf0 0x7faf141f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.321+0000 7faf13e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0f10 0x7faf14150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.321+0000 7faf13e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14164e90 0x7faf1415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.321+0000 7faf13e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf140b8c00 con 0x7faf1406ddf0 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.321+0000 7faf13e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faf140b42e0 con 0x7faf14164e90 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.321+0000 7faf13e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf140b8d80 con 0x7faf140d0f10 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf1406ddf0 0x7faf141f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf1406ddf0 0x7faf141f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49192/0 (socket says 172.21.15.19:49192) 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 -- 172.21.15.19:0/242925776 learned_addr learned my addr 172.21.15.19:0/242925776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 -- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0f10 msgr2=0x7faf14150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0f10 0x7faf14150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 -- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14164e90 msgr2=0x7faf1415d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14164e90 0x7faf1415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 -- 172.21.15.19:0/242925776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1415f460 con 0x7faf1406ddf0 2024-04-25T11:27:38.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.322+0000 7faf12a00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf1406ddf0 0x7faf141f6920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e840 tx=0x7faf0400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.323+0000 7faf0aa00640 1 -- 172.21.15.19:0/242925776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04017070 con 0x7faf1406ddf0 2024-04-25T11:27:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.323+0000 7faf0aa00640 1 -- 172.21.15.19:0/242925776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0400c940 con 0x7faf1406ddf0 2024-04-25T11:27:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.323+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1415c990 con 0x7faf1406ddf0 2024-04-25T11:27:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.323+0000 7faf0aa00640 1 -- 172.21.15.19:0/242925776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0400cd10 con 0x7faf1406ddf0 2024-04-25T11:27:38.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.323+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1415cc30 con 0x7faf1406ddf0 2024-04-25T11:27:38.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.324+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1406e820 con 0x7faf1406ddf0 2024-04-25T11:27:38.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.325+0000 7faf0aa00640 1 -- 172.21.15.19:0/242925776 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf0401a070 con 0x7faf1406ddf0 2024-04-25T11:27:38.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.326+0000 7faf0aa00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faef40744b0 0x7faef40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.327+0000 7faf0aa00640 1 -- 172.21.15.19:0/242925776 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faf040955d0 con 0x7faf1406ddf0 2024-04-25T11:27:38.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.327+0000 7faf12000640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faef40744b0 0x7faef40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.328+0000 7faf12000640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faef40744b0 0x7faef40769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faf00000f60 tx=0x7faf000078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:38.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.331+0000 7faf0aa00640 1 -- 172.21.15.19:0/242925776 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf0405fd50 con 0x7faf1406ddf0 2024-04-25T11:27:38.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.531+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf14163b30 con 0x7faf1406ddf0 2024-04-25T11:27:38.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.534+0000 7faf0aa00640 1 -- 172.21.15.19:0/242925776 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7faf0405f650 con 0x7faf1406ddf0 2024-04-25T11:27:38.526 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:38.526 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:38.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.536+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faef40744b0 msgr2=0x7faef40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.536+0000 7faf13e00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faef40744b0 0x7faef40769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faf00000f60 tx=0x7faf000078f0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.536+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf1406ddf0 msgr2=0x7faf141f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.536+0000 7faf13e00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf1406ddf0 0x7faf141f6920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e840 tx=0x7faf0400ed00 comp rx=0 tx=0).stop 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 shutdown_connections 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faef40744b0 0x7faef40769c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf14164e90 0x7faf1415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0f10 0x7faf14150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 --2- 172.21.15.19:0/242925776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf1406ddf0 0x7faf141f6920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 >> 172.21.15.19:0/242925776 conn(0x7faf140d03f0 msgr2=0x7faf140717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 shutdown_connections 2024-04-25T11:27:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.537+0000 7faf13e00640 1 -- 172.21.15.19:0/242925776 wait complete. 2024-04-25T11:27:38.556 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:27:38.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:27:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.848+0000 7fdfa6a00640 1 Processor -- start 2024-04-25T11:27:38.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.849+0000 7fdfa6a00640 1 -- start start 2024-04-25T11:27:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.851+0000 7fdfa6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdfa0144340 0x7fdfa006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.851+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa006db10 0x7fdfa015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.851+0000 7fdfa6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa006df20 con 0x7fdfa015ec90 2024-04-25T11:27:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.851+0000 7fdfa6a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa0144750 con 0x7fdfa0144340 2024-04-25T11:27:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.851+0000 7fdfa6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa0070000 con 0x7fdfa006db10 2024-04-25T11:27:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.852+0000 7fdfa4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa006db10 0x7fdfa015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.852+0000 7fdfa4c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa006db10 0x7fdfa015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48346/0 (socket says 172.21.15.19:48346) 2024-04-25T11:27:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.852+0000 7fdfa4c00640 1 -- 172.21.15.19:0/3220202447 learned_addr learned my addr 172.21.15.19:0/3220202447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.852+0000 7fdfa5600640 1 --2- 172.21.15.19:0/3220202447 >> v2:172.21.15.169:3300/0 conn(0x7fdfa0144340 0x7fdfa006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.853+0000 7fdfa5600640 1 -- 172.21.15.19:0/3220202447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa006db10 msgr2=0x7fdfa015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.853+0000 7fdfa5600640 1 --2- 172.21.15.19:0/3220202447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa006db10 0x7fdfa015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.853+0000 7fdfa5600640 1 -- 172.21.15.19:0/3220202447 >> v1:172.21.15.169:6789/0 conn(0x7fdfa015ec90 legacy=0x7fdfa0161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.853+0000 7fdfa5600640 1 -- 172.21.15.19:0/3220202447 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa006f240 con 0x7fdfa0144340 2024-04-25T11:27:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.854+0000 7fdfa5600640 1 --2- 172.21.15.19:0/3220202447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa0144340 0x7fdfa006d5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf940097a0 tx=0x7fdf9402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=315d32f1f60345e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.854+0000 7fdf9fe00640 1 -- 172.21.15.19:0/3220202447 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94037070 con 0x7fdfa0144340 2024-04-25T11:27:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.854+0000 7fdf9fe00640 1 -- 172.21.15.19:0/3220202447 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf94032440 con 0x7fdfa0144340 2024-04-25T11:27:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.854+0000 7fdf9fe00640 1 -- 172.21.15.19:0/3220202447 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf94032810 con 0x7fdfa0144340 2024-04-25T11:27:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.855+0000 7fdfa6a00640 1 -- 172.21.15.19:0/3220202447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa0144340 msgr2=0x7fdfa006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.855+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/3220202447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa0144340 0x7fdfa006d5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdf940097a0 tx=0x7fdf9402a180 comp rx=0 tx=0).stop 2024-04-25T11:27:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.855+0000 7fdfa6a00640 1 -- 172.21.15.19:0/3220202447 shutdown_connections 2024-04-25T11:27:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.855+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/3220202447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa006db10 0x7fdfa015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.855+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/3220202447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa0144340 0x7fdfa006d5d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.855+0000 7fdfa6a00640 1 -- 172.21.15.19:0/3220202447 >> 172.21.15.19:0/3220202447 conn(0x7fdfa0072f40 msgr2=0x7fdfa0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.856+0000 7fdfa6a00640 1 -- 172.21.15.19:0/3220202447 shutdown_connections 2024-04-25T11:27:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.856+0000 7fdfa6a00640 1 -- 172.21.15.19:0/3220202447 wait complete. 2024-04-25T11:27:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.858+0000 7fdfa6a00640 1 Processor -- start 2024-04-25T11:27:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.858+0000 7fdfa6a00640 1 -- start start 2024-04-25T11:27:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa006db10 0x7fdfa015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfa0144340 0x7fdfa0140a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa015ec90 0x7fdfa015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa6a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa006fae0 con 0x7fdfa0144340 2024-04-25T11:27:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa006f800 con 0x7fdfa006db10 2024-04-25T11:27:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa015f0a0 con 0x7fdfa015ec90 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa4c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfa0144340 0x7fdfa0140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa4c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfa0144340 0x7fdfa0140a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49206/0 (socket says 172.21.15.19:49206) 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa4c00640 1 -- 172.21.15.19:0/2024252862 learned_addr learned my addr 172.21.15.19:0/2024252862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa6000640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa015ec90 0x7fdfa015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.860+0000 7fdfa5600640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa006db10 0x7fdfa015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa4c00640 1 -- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa015ec90 msgr2=0x7fdfa015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa4c00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa015ec90 0x7fdfa015faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa4c00640 1 -- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa006db10 msgr2=0x7fdfa015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa4c00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa006db10 0x7fdfa015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa4c00640 1 -- 172.21.15.19:0/2024252862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa015b290 con 0x7fdfa0144340 2024-04-25T11:27:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa4c00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfa0144340 0x7fdfa0140a90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00e870 tx=0x7fdf8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdf9d600640 1 -- 172.21.15.19:0/2024252862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c017070 con 0x7fdfa0144340 2024-04-25T11:27:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdf9d600640 1 -- 172.21.15.19:0/2024252862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c00c940 con 0x7fdfa0144340 2024-04-25T11:27:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdf9d600640 1 -- 172.21.15.19:0/2024252862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00cd30 con 0x7fdfa0144340 2024-04-25T11:27:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa015b570 con 0x7fdfa0144340 2024-04-25T11:27:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.861+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa0144b20 con 0x7fdfa0144340 2024-04-25T11:27:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.863+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa0076c60 con 0x7fdfa0144340 2024-04-25T11:27:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.864+0000 7fdf9d600640 1 -- 172.21.15.19:0/2024252862 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf8c01a070 con 0x7fdfa0144340 2024-04-25T11:27:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.865+0000 7fdf9d600640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf800744b0 0x7fdf800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.865+0000 7fdf9d600640 1 -- 172.21.15.19:0/2024252862 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdf8c0954a0 con 0x7fdfa0144340 2024-04-25T11:27:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.866+0000 7fdfa5600640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf800744b0 0x7fdf800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.867+0000 7fdfa5600640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf800744b0 0x7fdf800769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf94034000 tx=0x7fdf9402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:38.870+0000 7fdf9d600640 1 -- 172.21.15.19:0/2024252862 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf8c05fc20 con 0x7fdfa0144340 2024-04-25T11:27:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.073+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfa015b020 con 0x7fdfa0144340 2024-04-25T11:27:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.075+0000 7fdf9d600640 1 -- 172.21.15.19:0/2024252862 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fdf8c05f520 con 0x7fdfa0144340 2024-04-25T11:27:39.066 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf800744b0 msgr2=0x7fdf800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf800744b0 0x7fdf800769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf94034000 tx=0x7fdf9402e000 comp rx=0 tx=0).stop 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfa0144340 msgr2=0x7fdfa0140a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfa0144340 0x7fdfa0140a90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00e870 tx=0x7fdf8c00ed30 comp rx=0 tx=0).stop 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 shutdown_connections 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf800744b0 0x7fdf800769c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdfa015ec90 0x7fdfa015faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfa0144340 0x7fdfa0140a90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.077+0000 7fdfa6a00640 1 --2- 172.21.15.19:0/2024252862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdfa006db10 0x7fdfa015aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.078+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 >> 172.21.15.19:0/2024252862 conn(0x7fdfa0072f40 msgr2=0x7fdfa00b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.078+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 shutdown_connections 2024-04-25T11:27:39.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.078+0000 7fdfa6a00640 1 -- 172.21.15.19:0/2024252862 wait complete. 2024-04-25T11:27:39.095 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:27:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.386+0000 7f2d21800640 1 Processor -- start 2024-04-25T11:27:39.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.386+0000 7f2d21800640 1 -- start start 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.388+0000 7f2d21800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c06dce0 0x7f2d1c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.388+0000 7f2d21800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2d1c15adf0 0x7f2d1c160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d21800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c071110 con 0x7f2d1c070630 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d21800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1c0713f0 con 0x7f2d1c15adf0 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d21800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c0701a0 con 0x7f2d1c06dce0 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d1b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2d1c15adf0 0x7f2d1c160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d1b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2d1c15adf0 0x7f2d1c160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49212/0 (socket says 172.21.15.19:49212) 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d1b400640 1 -- 172.21.15.19:0/2426653281 learned_addr learned my addr 172.21.15.19:0/2426653281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d13e00640 1 --2- 172.21.15.19:0/2426653281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c06dce0 0x7f2d1c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d1b400640 1 -- 172.21.15.19:0/2426653281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c06dce0 msgr2=0x7f2d1c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d1b400640 1 --2- 172.21.15.19:0/2426653281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c06dce0 0x7f2d1c0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.389+0000 7f2d1b400640 1 -- 172.21.15.19:0/2426653281 >> v1:172.21.15.169:6789/0 conn(0x7f2d1c070630 legacy=0x7f2d1c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.390+0000 7f2d1b400640 1 -- 172.21.15.19:0/2426653281 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c06fb50 con 0x7f2d1c15adf0 2024-04-25T11:27:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.391+0000 7f2d1b400640 1 --2- 172.21.15.19:0/2426653281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c15adf0 0x7f2d1c160d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d150 tx=0x7f2d1402c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0940cc20d5c00d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.391+0000 7f2d1a000640 1 -- 172.21.15.19:0/2426653281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400b2e0 con 0x7f2d1c15adf0 2024-04-25T11:27:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.391+0000 7f2d1a000640 1 -- 172.21.15.19:0/2426653281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d14009600 con 0x7f2d1c15adf0 2024-04-25T11:27:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.391+0000 7f2d1a000640 1 -- 172.21.15.19:0/2426653281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400b480 con 0x7f2d1c15adf0 2024-04-25T11:27:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 -- 172.21.15.19:0/2426653281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c15adf0 msgr2=0x7f2d1c160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 --2- 172.21.15.19:0/2426653281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c15adf0 0x7f2d1c160d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d150 tx=0x7f2d1402c8a0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 -- 172.21.15.19:0/2426653281 shutdown_connections 2024-04-25T11:27:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 --2- 172.21.15.19:0/2426653281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c15adf0 0x7f2d1c160d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 --2- 172.21.15.19:0/2426653281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c06dce0 0x7f2d1c0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 -- 172.21.15.19:0/2426653281 >> 172.21.15.19:0/2426653281 conn(0x7f2d1c073090 msgr2=0x7f2d1c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 -- 172.21.15.19:0/2426653281 shutdown_connections 2024-04-25T11:27:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.392+0000 7f2d21800640 1 -- 172.21.15.19:0/2426653281 wait complete. 2024-04-25T11:27:39.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.394+0000 7f2d21800640 1 Processor -- start 2024-04-25T11:27:39.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.395+0000 7f2d21800640 1 -- start start 2024-04-25T11:27:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d21800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c070630 0x7f2d1c153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d21800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d1c15adf0 0x7f2d1c14a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d21800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c151ac0 0x7f2d1c0b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d21800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0b0e80 con 0x7f2d1c151ac0 2024-04-25T11:27:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d21800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0b0d00 con 0x7f2d1c15adf0 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d21800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c070c80 con 0x7f2d1c070630 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d1b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c151ac0 0x7f2d1c0b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d1b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c151ac0 0x7f2d1c0b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49214/0 (socket says 172.21.15.19:49214) 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.396+0000 7f2d1b400640 1 -- 172.21.15.19:0/1957559494 learned_addr learned my addr 172.21.15.19:0/1957559494 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d1aa00640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c070630 0x7f2d1c153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d13e00640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d1c15adf0 0x7f2d1c14a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d1b400640 1 -- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c070630 msgr2=0x7f2d1c153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d1b400640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c070630 0x7f2d1c153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d1b400640 1 -- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d1c15adf0 msgr2=0x7f2d1c14a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d1b400640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d1c15adf0 0x7f2d1c14a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d1b400640 1 -- 172.21.15.19:0/1957559494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c152230 con 0x7f2d1c151ac0 2024-04-25T11:27:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.397+0000 7f2d1b400640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c151ac0 0x7f2d1c0b2450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d240 tx=0x7f2d14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.398+0000 7f2d12a00640 1 -- 172.21.15.19:0/1957559494 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14040030 con 0x7f2d1c151ac0 2024-04-25T11:27:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.398+0000 7f2d12a00640 1 -- 172.21.15.19:0/1957559494 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d14009370 con 0x7f2d1c151ac0 2024-04-25T11:27:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.398+0000 7f2d12a00640 1 -- 172.21.15.19:0/1957559494 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1403c8b0 con 0x7f2d1c151ac0 2024-04-25T11:27:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.398+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c152510 con 0x7f2d1c151ac0 2024-04-25T11:27:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.398+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c14c380 con 0x7f2d1c151ac0 2024-04-25T11:27:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.399+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c151fa0 con 0x7f2d1c151ac0 2024-04-25T11:27:39.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.400+0000 7f2d12a00640 1 -- 172.21.15.19:0/1957559494 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d14043070 con 0x7f2d1c151ac0 2024-04-25T11:27:39.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.401+0000 7f2d12a00640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2cf80744e0 0x7f2cf80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.401+0000 7f2d12a00640 1 -- 172.21.15.19:0/1957559494 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2d14030020 con 0x7f2d1c151ac0 2024-04-25T11:27:39.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.402+0000 7f2d1aa00640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2cf80744e0 0x7f2cf80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.403+0000 7f2d1aa00640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2cf80744e0 0x7f2cf80769f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000f60 tx=0x7f2d0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.406+0000 7f2d12a00640 1 -- 172.21.15.19:0/1957559494 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d1404a020 con 0x7f2d1c151ac0 2024-04-25T11:27:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.610+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2d1c050740 con 0x7f2d1c151ac0 2024-04-25T11:27:39.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.611+0000 7f2d12a00640 1 -- 172.21.15.19:0/1957559494 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f2d14080050 con 0x7f2d1c151ac0 2024-04-25T11:27:39.602 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:27:39.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.613+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2cf80744e0 msgr2=0x7f2cf80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.613+0000 7f2d21800640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2cf80744e0 0x7f2cf80769f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000f60 tx=0x7f2d0c009040 comp rx=0 tx=0).stop 2024-04-25T11:27:39.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c151ac0 msgr2=0x7f2d1c0b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c151ac0 0x7f2d1c0b2450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d240 tx=0x7f2d14004a40 comp rx=0 tx=0).stop 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 shutdown_connections 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2cf80744e0 0x7f2cf80769f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d1c15adf0 0x7f2d1c14a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d1c151ac0 0x7f2d1c0b2450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 --2- 172.21.15.19:0/1957559494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d1c070630 0x7f2d1c153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 >> 172.21.15.19:0/1957559494 conn(0x7f2d1c073090 msgr2=0x7f2d1c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 shutdown_connections 2024-04-25T11:27:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.614+0000 7f2d21800640 1 -- 172.21.15.19:0/1957559494 wait complete. 2024-04-25T11:27:39.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:27:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.931+0000 7fc91f200640 1 Processor -- start 2024-04-25T11:27:39.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.932+0000 7fc91f200640 1 -- start start 2024-04-25T11:27:39.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.934+0000 7fc91f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9200d1540 0x7fc92009cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.934+0000 7fc91f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc9200b71d0 0x7fc9200b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.934+0000 7fc91f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9200b9490 con 0x7fc9200d1f70 2024-04-25T11:27:39.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.934+0000 7fc91f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc9200d1b50 con 0x7fc9200d1540 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.934+0000 7fc91f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc9200b91b0 con 0x7fc9200b71d0 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc9200d1f70 0x7fc9200d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54330/0 (socket says 172.21.15.19:54330) 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91e800640 1 -- 172.21.15.19:0/376262484 learned_addr learned my addr 172.21.15.19:0/376262484 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91de00640 1 --2- 172.21.15.19:0/376262484 >> v2:172.21.15.19:3300/0 conn(0x7fc9200d1540 0x7fc92009cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc917e00640 1 --2- 172.21.15.19:0/376262484 >> v2:172.21.15.169:3300/0 conn(0x7fc9200b71d0 0x7fc9200b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91d400640 1 -- 172.21.15.19:0/376262484 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1020085660 0 0) 0x7fc9200b9490 con 0x7fc9200d1f70 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91d400640 1 -- 172.21.15.19:0/376262484 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc904003520 con 0x7fc9200d1f70 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91de00640 1 -- 172.21.15.19:0/376262484 >> v2:172.21.15.169:3300/0 conn(0x7fc9200b71d0 msgr2=0x7fc9200b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91de00640 1 --2- 172.21.15.19:0/376262484 >> v2:172.21.15.169:3300/0 conn(0x7fc9200b71d0 0x7fc9200b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91de00640 1 -- 172.21.15.19:0/376262484 >> v1:172.21.15.19:6789/0 conn(0x7fc9200d1f70 legacy=0x7fc9200d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.935+0000 7fc91de00640 1 -- 172.21.15.19:0/376262484 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9200cf910 con 0x7fc9200d1540 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.936+0000 7fc91de00640 1 --2- 172.21.15.19:0/376262484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc92009cab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc920073e40 tx=0x7fc91002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166f5b7df8024516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.937+0000 7fc91d400640 1 -- 172.21.15.19:0/376262484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc910037070 con 0x7fc9200d1540 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.937+0000 7fc91d400640 1 -- 172.21.15.19:0/376262484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc910032440 con 0x7fc9200d1540 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.937+0000 7fc91d400640 1 -- 172.21.15.19:0/376262484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9100327f0 con 0x7fc9200d1540 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.937+0000 7fc91f200640 1 -- 172.21.15.19:0/376262484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 msgr2=0x7fc92009cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.937+0000 7fc91f200640 1 --2- 172.21.15.19:0/376262484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc92009cab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc920073e40 tx=0x7fc91002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.938+0000 7fc91f200640 1 -- 172.21.15.19:0/376262484 shutdown_connections 2024-04-25T11:27:39.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.938+0000 7fc91f200640 1 --2- 172.21.15.19:0/376262484 >> v2:172.21.15.169:3300/0 conn(0x7fc9200b71d0 0x7fc9200b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.938+0000 7fc91f200640 1 --2- 172.21.15.19:0/376262484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc92009cab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.938+0000 7fc91f200640 1 -- 172.21.15.19:0/376262484 >> 172.21.15.19:0/376262484 conn(0x7fc9200d03f0 msgr2=0x7fc9200d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:39.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.938+0000 7fc91f200640 1 -- 172.21.15.19:0/376262484 shutdown_connections 2024-04-25T11:27:39.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.938+0000 7fc91f200640 1 -- 172.21.15.19:0/376262484 wait complete. 2024-04-25T11:27:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.940+0000 7fc91f200640 1 Processor -- start 2024-04-25T11:27:39.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.940+0000 7fc91f200640 1 -- start start 2024-04-25T11:27:39.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc91f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9200b71d0 0x7fc9201df270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc91f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc9201df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc91f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9200d1f70 0x7fc9200a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc91f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9200d3700 con 0x7fc9200d1540 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc91f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc9200d3580 con 0x7fc9200d1f70 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc91f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc92009cff0 con 0x7fc9200b71d0 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc917e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc9201df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc917e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc9201df7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49228/0 (socket says 172.21.15.19:49228) 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc917e00640 1 -- 172.21.15.19:0/1497049967 learned_addr learned my addr 172.21.15.19:0/1497049967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.942+0000 7fc91de00640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9200b71d0 0x7fc9201df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc91e800640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9200d1f70 0x7fc9200a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc917e00640 1 -- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9200b71d0 msgr2=0x7fc9201df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc917e00640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9200b71d0 0x7fc9201df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc917e00640 1 -- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9200d1f70 msgr2=0x7fc9200a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc917e00640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9200d1f70 0x7fc9200a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:39.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc917e00640 1 -- 172.21.15.19:0/1497049967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92009f350 con 0x7fc9200d1540 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc91e800640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9200d1f70 0x7fc9200a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc917e00640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc9201df7b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e8a0 tx=0x7fc90800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc916000640 1 -- 172.21.15.19:0/1497049967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908017070 con 0x7fc9200d1540 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc916000640 1 -- 172.21.15.19:0/1497049967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90800c940 con 0x7fc9200d1540 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc916000640 1 -- 172.21.15.19:0/1497049967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90800cd30 con 0x7fc9200d1540 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.943+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92009f630 con 0x7fc9200d1540 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.944+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92009fbf0 con 0x7fc9200d1540 2024-04-25T11:27:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.945+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92009d9e0 con 0x7fc9200d1540 2024-04-25T11:27:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.946+0000 7fc916000640 1 -- 172.21.15.19:0/1497049967 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc90801a070 con 0x7fc9200d1540 2024-04-25T11:27:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.947+0000 7fc916000640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8fc0744b0 0x7fc8fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:39.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.947+0000 7fc916000640 1 -- 172.21.15.19:0/1497049967 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc9080955a0 con 0x7fc9200d1540 2024-04-25T11:27:39.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.948+0000 7fc91de00640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8fc0744b0 0x7fc8fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.949+0000 7fc91de00640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8fc0744b0 0x7fc8fc0769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc910034000 tx=0x7fc91002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:39.952+0000 7fc916000640 1 -- 172.21.15.19:0/1497049967 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc90805fc90 con 0x7fc9200d1540 2024-04-25T11:27:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.156+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc920076c60 con 0x7fc9200d1540 2024-04-25T11:27:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.158+0000 7fc916000640 1 -- 172.21.15.19:0/1497049967 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc90805f590 con 0x7fc9200d1540 2024-04-25T11:27:40.149 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8fc0744b0 msgr2=0x7fc8fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8fc0744b0 0x7fc8fc0769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc910034000 tx=0x7fc91002e000 comp rx=0 tx=0).stop 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 msgr2=0x7fc9201df7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc9201df7b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e8a0 tx=0x7fc90800ed60 comp rx=0 tx=0).stop 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 shutdown_connections 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8fc0744b0 0x7fc8fc0769c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9200d1f70 0x7fc9200a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9200d1540 0x7fc9201df7b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 --2- 172.21.15.19:0/1497049967 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9200b71d0 0x7fc9201df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 >> 172.21.15.19:0/1497049967 conn(0x7fc9200d03f0 msgr2=0x7fc920072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:40.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.160+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 shutdown_connections 2024-04-25T11:27:40.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.161+0000 7fc91f200640 1 -- 172.21.15.19:0/1497049967 wait complete. 2024-04-25T11:27:40.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:27:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.457+0000 7f256d800640 1 Processor -- start 2024-04-25T11:27:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.458+0000 7f256d800640 1 -- start start 2024-04-25T11:27:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.460+0000 7f256d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25680b5e40 0x7f25680b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.460+0000 7f256d800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f256806c5c0 0x7f256806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.460+0000 7f256d800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25680b6790 con 0x7f256814ad60 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.460+0000 7f256d800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f25680b4af0 con 0x7f25680b5e40 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.460+0000 7f256d800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f25680b4dd0 con 0x7f256806c5c0 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f255fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25680b5e40 0x7f25680b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f255fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25680b5e40 0x7f25680b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49234/0 (socket says 172.21.15.19:49234) 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f255fe00640 1 -- 172.21.15.19:0/2377655498 learned_addr learned my addr 172.21.15.19:0/2377655498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f2567400640 1 --2- 172.21.15.19:0/2377655498 >> v2:172.21.15.169:3300/0 conn(0x7f256806c5c0 0x7f256806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f255fe00640 1 -- 172.21.15.19:0/2377655498 >> v2:172.21.15.169:3300/0 conn(0x7f256806c5c0 msgr2=0x7f256806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f255fe00640 1 --2- 172.21.15.19:0/2377655498 >> v2:172.21.15.169:3300/0 conn(0x7f256806c5c0 0x7f256806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f255fe00640 1 -- 172.21.15.19:0/2377655498 >> v1:172.21.15.169:6789/0 conn(0x7f256814ad60 legacy=0x7f2568070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f255fe00640 1 -- 172.21.15.19:0/2377655498 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25680b50b0 con 0x7f25680b5e40 2024-04-25T11:27:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.461+0000 7f2567400640 1 --2- 172.21.15.19:0/2377655498 >> v2:172.21.15.169:3300/0 conn(0x7f256806c5c0 0x7f256806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.462+0000 7f255fe00640 1 --2- 172.21.15.19:0/2377655498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25680b5e40 0x7f25680b6250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2568070a70 tx=0x7f255002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58f2964bd92ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.462+0000 7f2566000640 1 -- 172.21.15.19:0/2377655498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550037070 con 0x7f25680b5e40 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f2566000640 1 -- 172.21.15.19:0/2377655498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2550032440 con 0x7f25680b5e40 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f2566000640 1 -- 172.21.15.19:0/2377655498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550032810 con 0x7f25680b5e40 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 -- 172.21.15.19:0/2377655498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25680b5e40 msgr2=0x7f25680b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 --2- 172.21.15.19:0/2377655498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25680b5e40 0x7f25680b6250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2568070a70 tx=0x7f255002a180 comp rx=0 tx=0).stop 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 -- 172.21.15.19:0/2377655498 shutdown_connections 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 --2- 172.21.15.19:0/2377655498 >> v2:172.21.15.169:3300/0 conn(0x7f256806c5c0 0x7f256806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 --2- 172.21.15.19:0/2377655498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25680b5e40 0x7f25680b6250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 -- 172.21.15.19:0/2377655498 >> 172.21.15.19:0/2377655498 conn(0x7f25680d1ae0 msgr2=0x7f25680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 -- 172.21.15.19:0/2377655498 shutdown_connections 2024-04-25T11:27:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.463+0000 7f256d800640 1 -- 172.21.15.19:0/2377655498 wait complete. 2024-04-25T11:27:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.466+0000 7f256d800640 1 Processor -- start 2024-04-25T11:27:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.466+0000 7f256d800640 1 -- start start 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.467+0000 7f256d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f256806c5c0 0x7f256815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.467+0000 7f256d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680b5e40 0x7f256815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.467+0000 7f256d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f256814ad60 0x7f25681fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.467+0000 7f256d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f256806fb80 con 0x7f256814ad60 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.467+0000 7f256d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25680b5670 con 0x7f25680b5e40 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.467+0000 7f256d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f256806fd00 con 0x7f256806c5c0 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f256814ad60 0x7f25681fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f256814ad60 0x7f25681fa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49242/0 (socket says 172.21.15.19:49242) 2024-04-25T11:27:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 -- 172.21.15.19:0/2842030632 learned_addr learned my addr 172.21.15.19:0/2842030632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2566a00640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f256806c5c0 0x7f256815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 -- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f256806c5c0 msgr2=0x7f256815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f255fe00640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680b5e40 0x7f256815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f256806c5c0 0x7f256815ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 -- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680b5e40 msgr2=0x7f256815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680b5e40 0x7f256815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2567400640 1 -- 172.21.15.19:0/2842030632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25681fef00 con 0x7f256814ad60 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.468+0000 7f2566a00640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f256806c5c0 0x7f256815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.469+0000 7f2567400640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f256814ad60 0x7f25681fa4e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f256000cd80 tx=0x7f256000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.469+0000 7f255ea00640 1 -- 172.21.15.19:0/2842030632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256000f680 con 0x7f256814ad60 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.469+0000 7f255ea00640 1 -- 172.21.15.19:0/2842030632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256000f840 con 0x7f256814ad60 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.469+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256815d660 con 0x7f256814ad60 2024-04-25T11:27:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.469+0000 7f255ea00640 1 -- 172.21.15.19:0/2842030632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256000fc30 con 0x7f256814ad60 2024-04-25T11:27:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.469+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256815dc20 con 0x7f256814ad60 2024-04-25T11:27:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.470+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25681fec70 con 0x7f256814ad60 2024-04-25T11:27:40.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.471+0000 7f255ea00640 1 -- 172.21.15.19:0/2842030632 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f25600213f0 con 0x7f256814ad60 2024-04-25T11:27:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.472+0000 7f255ea00640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25440744e0 0x7f25440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.472+0000 7f255ea00640 1 -- 172.21.15.19:0/2842030632 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2560095ff0 con 0x7f256814ad60 2024-04-25T11:27:40.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.473+0000 7f2566a00640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25440744e0 0x7f25440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:40.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.474+0000 7f2566a00640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25440744e0 0x7f25440769f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25680b1590 tx=0x7f2558008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:40.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.477+0000 7f255ea00640 1 -- 172.21.15.19:0/2842030632 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2560014030 con 0x7f256814ad60 2024-04-25T11:27:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.724+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2568050740 con 0x7f256814ad60 2024-04-25T11:27:40.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.725+0000 7f255ea00640 1 -- 172.21.15.19:0/2842030632 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f2560060790 con 0x7f256814ad60 2024-04-25T11:27:40.717 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:40.717 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:27:40.717 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25440744e0 msgr2=0x7f25440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25440744e0 0x7f25440769f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25680b1590 tx=0x7f2558008040 comp rx=0 tx=0).stop 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f256814ad60 msgr2=0x7f25681fa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f256814ad60 0x7f25681fa4e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f256000cd80 tx=0x7f256000a650 comp rx=0 tx=0).stop 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 shutdown_connections 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25440744e0 0x7f25440769f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25680b5e40 0x7f256815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f256814ad60 0x7f25681fa4e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.728+0000 7f256d800640 1 --2- 172.21.15.19:0/2842030632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f256806c5c0 0x7f256815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.729+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 >> 172.21.15.19:0/2842030632 conn(0x7f25680d1ae0 msgr2=0x7f256806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.729+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 shutdown_connections 2024-04-25T11:27:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:40.729+0000 7f256d800640 1 -- 172.21.15.19:0/2842030632 wait complete. 2024-04-25T11:27:40.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.044+0000 7f3ae8a00640 1 Processor -- start 2024-04-25T11:27:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.044+0000 7f3ae8a00640 1 -- start start 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ae40cf2f0 0x7f3ae40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ae415a810 0x7f3ae40b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae40b4e90 con 0x7f3ae41608d0 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae8a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae406f870 con 0x7f3ae415a810 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae40b74d0 con 0x7f3ae40cf2f0 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ae415a810 0x7f3ae40b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ae415a810 0x7f3ae40b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49248/0 (socket says 172.21.15.19:49248) 2024-04-25T11:27:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae1e00640 1 -- 172.21.15.19:0/650023975 learned_addr learned my addr 172.21.15.19:0/650023975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.047+0000 7f3ae2800640 1 --2- 172.21.15.19:0/650023975 >> v2:172.21.15.169:3300/0 conn(0x7f3ae40cf2f0 0x7f3ae40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.048+0000 7f3ae1e00640 1 -- 172.21.15.19:0/650023975 >> v2:172.21.15.169:3300/0 conn(0x7f3ae40cf2f0 msgr2=0x7f3ae40b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.048+0000 7f3ae1e00640 1 --2- 172.21.15.19:0/650023975 >> v2:172.21.15.169:3300/0 conn(0x7f3ae40cf2f0 0x7f3ae40b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.048+0000 7f3ae1e00640 1 -- 172.21.15.19:0/650023975 >> v1:172.21.15.169:6789/0 conn(0x7f3ae41608d0 legacy=0x7f3ae4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.048+0000 7f3ae1e00640 1 -- 172.21.15.19:0/650023975 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae40b77b0 con 0x7f3ae415a810 2024-04-25T11:27:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.049+0000 7f3ae1e00640 1 --2- 172.21.15.19:0/650023975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae415a810 0x7f3ae40b4950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40d26d0 tx=0x7f3ad402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb0c348930aa57d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.049+0000 7f3ae1400640 1 -- 172.21.15.19:0/650023975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4037070 con 0x7f3ae415a810 2024-04-25T11:27:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.049+0000 7f3ae1400640 1 -- 172.21.15.19:0/650023975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4032440 con 0x7f3ae415a810 2024-04-25T11:27:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.049+0000 7f3ae1400640 1 -- 172.21.15.19:0/650023975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4032810 con 0x7f3ae415a810 2024-04-25T11:27:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 -- 172.21.15.19:0/650023975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae415a810 msgr2=0x7f3ae40b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/650023975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae415a810 0x7f3ae40b4950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40d26d0 tx=0x7f3ad402a230 comp rx=0 tx=0).stop 2024-04-25T11:27:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 -- 172.21.15.19:0/650023975 shutdown_connections 2024-04-25T11:27:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/650023975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae415a810 0x7f3ae40b4950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/650023975 >> v2:172.21.15.169:3300/0 conn(0x7f3ae40cf2f0 0x7f3ae40b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 -- 172.21.15.19:0/650023975 >> 172.21.15.19:0/650023975 conn(0x7f3ae40d1720 msgr2=0x7f3ae40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 -- 172.21.15.19:0/650023975 shutdown_connections 2024-04-25T11:27:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.050+0000 7f3ae8a00640 1 -- 172.21.15.19:0/650023975 wait complete. 2024-04-25T11:27:41.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.053+0000 7f3ae8a00640 1 Processor -- start 2024-04-25T11:27:41.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.053+0000 7f3ae8a00640 1 -- start start 2024-04-25T11:27:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.054+0000 7f3ae8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cf2f0 0x7f3ae41f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.054+0000 7f3ae8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae415a810 0x7f3ae40af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.054+0000 7f3ae8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae41608d0 0x7f3ae41590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.054+0000 7f3ae8a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae415ac20 con 0x7f3ae41608d0 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.054+0000 7f3ae8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae406f500 con 0x7f3ae415a810 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.054+0000 7f3ae8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae406fd00 con 0x7f3ae40cf2f0 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae41608d0 0x7f3ae41590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae41608d0 0x7f3ae41590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49260/0 (socket says 172.21.15.19:49260) 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 -- 172.21.15.19:0/3715772481 learned_addr learned my addr 172.21.15.19:0/3715772481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae2800640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cf2f0 0x7f3ae41f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae1e00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae415a810 0x7f3ae40af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 -- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cf2f0 msgr2=0x7f3ae41f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cf2f0 0x7f3ae41f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 -- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae415a810 msgr2=0x7f3ae40af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae415a810 0x7f3ae40af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 -- 172.21.15.19:0/3715772481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae4144cd0 con 0x7f3ae41608d0 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae1e00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae415a810 0x7f3ae40af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.055+0000 7f3ae3200640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae41608d0 0x7f3ae41590d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00b970 tx=0x7f3adc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.056+0000 7f3adaa00640 1 -- 172.21.15.19:0/3715772481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc019070 con 0x7f3ae41608d0 2024-04-25T11:27:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.056+0000 7f3adaa00640 1 -- 172.21.15.19:0/3715772481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3adc00cb40 con 0x7f3ae41608d0 2024-04-25T11:27:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.056+0000 7f3adaa00640 1 -- 172.21.15.19:0/3715772481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc014440 con 0x7f3ae41608d0 2024-04-25T11:27:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.056+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae4144fb0 con 0x7f3ae41608d0 2024-04-25T11:27:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.056+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae4145570 con 0x7f3ae41608d0 2024-04-25T11:27:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.057+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae4050740 con 0x7f3ae41608d0 2024-04-25T11:27:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.058+0000 7f3adaa00640 1 -- 172.21.15.19:0/3715772481 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3adc01c070 con 0x7f3ae41608d0 2024-04-25T11:27:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.059+0000 7f3adaa00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.060+0000 7f3adaa00640 1 -- 172.21.15.19:0/3715772481 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3adc094e40 con 0x7f3ae41608d0 2024-04-25T11:27:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.060+0000 7f3ae2800640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.061+0000 7f3ae2800640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3acc006fd0 tx=0x7f3acc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.064+0000 7f3adaa00640 1 -- 172.21.15.19:0/3715772481 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3adc05f660 con 0x7f3ae41608d0 2024-04-25T11:27:41.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.261+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ae406c770 con 0x7f3ae41608d0 2024-04-25T11:27:41.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.263+0000 7f3adaa00640 1 -- 172.21.15.19:0/3715772481 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7f3adc05ef60 con 0x7f3ae41608d0 2024-04-25T11:27:41.255 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:41.255 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.265+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 msgr2=0x7f3ac00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.265+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3acc006fd0 tx=0x7f3acc008040 comp rx=0 tx=0).stop 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.265+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae41608d0 msgr2=0x7f3ae41590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.265+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae41608d0 0x7f3ae41590d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00b970 tx=0x7f3adc00be30 comp rx=0 tx=0).stop 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 shutdown_connections 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ac00744b0 0x7f3ac00769c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae415a810 0x7f3ae40af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ae41608d0 0x7f3ae41590d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 --2- 172.21.15.19:0/3715772481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cf2f0 0x7f3ae41f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 >> 172.21.15.19:0/3715772481 conn(0x7f3ae40d1720 msgr2=0x7f3ae4073930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 shutdown_connections 2024-04-25T11:27:41.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.266+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3715772481 wait complete. 2024-04-25T11:27:41.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:27:41.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:27:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.564+0000 7efe39400640 1 Processor -- start 2024-04-25T11:27:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.564+0000 7efe39400640 1 -- start start 2024-04-25T11:27:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.567+0000 7efe39400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe34070630 0x7efe340cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe39400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe34071110 con 0x7efe3415adf0 2024-04-25T11:27:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe39400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3406fb50 con 0x7efe3406dce0 2024-04-25T11:27:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe39400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efe340713f0 con 0x7efe34070630 2024-04-25T11:27:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe32800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe34070630 0x7efe340cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe32800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efe34070630 0x7efe340cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49274/0 (socket says 172.21.15.19:49274) 2024-04-25T11:27:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe32800640 1 -- 172.21.15.19:0/2593024377 learned_addr learned my addr 172.21.15.19:0/2593024377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe32800640 1 -- 172.21.15.19:0/2593024377 >> v1:172.21.15.169:6789/0 conn(0x7efe3406dce0 legacy=0x7efe34160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe32800640 1 -- 172.21.15.19:0/2593024377 >> v1:172.21.15.19:6789/0 conn(0x7efe3415adf0 legacy=0x7efe3406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.568+0000 7efe32800640 1 -- 172.21.15.19:0/2593024377 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3406fe30 con 0x7efe34070630 2024-04-25T11:27:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.569+0000 7efe32800640 1 --2- 172.21.15.19:0/2593024377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34070630 0x7efe340cec50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe24009e50 tx=0x7efe2402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a42512886ef0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.570+0000 7efe31400640 1 -- 172.21.15.19:0/2593024377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24036070 con 0x7efe34070630 2024-04-25T11:27:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.570+0000 7efe31400640 1 -- 172.21.15.19:0/2593024377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe24031600 con 0x7efe34070630 2024-04-25T11:27:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.570+0000 7efe31400640 1 -- 172.21.15.19:0/2593024377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240319d0 con 0x7efe34070630 2024-04-25T11:27:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.571+0000 7efe39400640 1 -- 172.21.15.19:0/2593024377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34070630 msgr2=0x7efe340cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.571+0000 7efe39400640 1 --2- 172.21.15.19:0/2593024377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34070630 0x7efe340cec50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe24009e50 tx=0x7efe2402a6d0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.571+0000 7efe39400640 1 -- 172.21.15.19:0/2593024377 shutdown_connections 2024-04-25T11:27:41.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.571+0000 7efe39400640 1 --2- 172.21.15.19:0/2593024377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe34070630 0x7efe340cec50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.571+0000 7efe39400640 1 -- 172.21.15.19:0/2593024377 >> 172.21.15.19:0/2593024377 conn(0x7efe34073090 msgr2=0x7efe340734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.571+0000 7efe39400640 1 -- 172.21.15.19:0/2593024377 shutdown_connections 2024-04-25T11:27:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.571+0000 7efe39400640 1 -- 172.21.15.19:0/2593024377 wait complete. 2024-04-25T11:27:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.573+0000 7efe39400640 1 Processor -- start 2024-04-25T11:27:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.574+0000 7efe39400640 1 -- start start 2024-04-25T11:27:41.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.575+0000 7efe39400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe34070630 0x7efe3414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.575+0000 7efe39400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3415adf0 0x7efe3414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.575+0000 7efe39400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe34159f80 0x7efe341f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.575+0000 7efe39400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe34070c80 con 0x7efe3415adf0 2024-04-25T11:27:41.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.575+0000 7efe39400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efe340701a0 con 0x7efe34159f80 2024-04-25T11:27:41.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.575+0000 7efe39400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efe34070e00 con 0x7efe34070630 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.575+0000 7efe31e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3415adf0 0x7efe3414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe31e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3415adf0 0x7efe3414a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49286/0 (socket says 172.21.15.19:49286) 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe31e00640 1 -- 172.21.15.19:0/2811117028 learned_addr learned my addr 172.21.15.19:0/2811117028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe32800640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe34070630 0x7efe3414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe33200640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe34159f80 0x7efe341f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe31e00640 1 -- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe34070630 msgr2=0x7efe3414a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe31e00640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe34070630 0x7efe3414a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe31e00640 1 -- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe34159f80 msgr2=0x7efe341f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe31e00640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe34159f80 0x7efe341f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.576+0000 7efe31e00640 1 -- 172.21.15.19:0/2811117028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe341576f0 con 0x7efe3415adf0 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.577+0000 7efe31e00640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3415adf0 0x7efe3414a5e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c3c0 tx=0x7efe1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:41.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.577+0000 7efe2aa00640 1 -- 172.21.15.19:0/2811117028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0052d0 con 0x7efe3415adf0 2024-04-25T11:27:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.577+0000 7efe2aa00640 1 -- 172.21.15.19:0/2811117028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c005490 con 0x7efe3415adf0 2024-04-25T11:27:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.577+0000 7efe2aa00640 1 -- 172.21.15.19:0/2811117028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c007260 con 0x7efe3415adf0 2024-04-25T11:27:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.577+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe341579d0 con 0x7efe3415adf0 2024-04-25T11:27:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.577+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe34157f90 con 0x7efe3415adf0 2024-04-25T11:27:41.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.579+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe34050740 con 0x7efe3415adf0 2024-04-25T11:27:41.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.580+0000 7efe2aa00640 1 -- 172.21.15.19:0/2811117028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efe1c007400 con 0x7efe3415adf0 2024-04-25T11:27:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.581+0000 7efe2aa00640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe10074530 0x7efe10076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:41.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.581+0000 7efe2aa00640 1 -- 172.21.15.19:0/2811117028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efe1c094530 con 0x7efe3415adf0 2024-04-25T11:27:41.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.582+0000 7efe32800640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe10074530 0x7efe10076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:41.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.583+0000 7efe32800640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe10074530 0x7efe10076a40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe24033000 tx=0x7efe24009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:41.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.586+0000 7efe2aa00640 1 -- 172.21.15.19:0/2811117028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe1c05e770 con 0x7efe3415adf0 2024-04-25T11:27:41.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.794+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe34069820 con 0x7efe3415adf0 2024-04-25T11:27:41.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.796+0000 7efe2aa00640 1 -- 172.21.15.19:0/2811117028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7efe1c062a70 con 0x7efe3415adf0 2024-04-25T11:27:41.787 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe10074530 msgr2=0x7efe10076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe10074530 0x7efe10076a40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe24033000 tx=0x7efe24009000 comp rx=0 tx=0).stop 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3415adf0 msgr2=0x7efe3414a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3415adf0 0x7efe3414a5e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00c3c0 tx=0x7efe1c00c880 comp rx=0 tx=0).stop 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 shutdown_connections 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe10074530 0x7efe10076a40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe34159f80 0x7efe341f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe3415adf0 0x7efe3414a5e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 --2- 172.21.15.19:0/2811117028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe34070630 0x7efe3414a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:41.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.798+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 >> 172.21.15.19:0/2811117028 conn(0x7efe34073090 msgr2=0x7efe340cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:41.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.799+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 shutdown_connections 2024-04-25T11:27:41.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:41.799+0000 7efe39400640 1 -- 172.21.15.19:0/2811117028 wait complete. 2024-04-25T11:27:41.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:27:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.112+0000 7fe812200640 1 Processor -- start 2024-04-25T11:27:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.112+0000 7fe812200640 1 -- start start 2024-04-25T11:27:42.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.114+0000 7fe812200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe80c0af710 0x7fe80c164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.114+0000 7fe812200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c06dce0 0x7fe80c070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.114+0000 7fe812200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80c164fc0 con 0x7fe80c160390 2024-04-25T11:27:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.114+0000 7fe812200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe80c0afb20 con 0x7fe80c0af710 2024-04-25T11:27:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.114+0000 7fe812200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c06f440 con 0x7fe80c06dce0 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.115+0000 7fe80be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c06dce0 0x7fe80c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.115+0000 7fe80be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c06dce0 0x7fe80c070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48408/0 (socket says 172.21.15.19:48408) 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.115+0000 7fe80be00640 1 -- 172.21.15.19:0/281365226 learned_addr learned my addr 172.21.15.19:0/281365226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.115+0000 7fe80b400640 1 --2- 172.21.15.19:0/281365226 >> v2:172.21.15.169:3300/0 conn(0x7fe80c0af710 0x7fe80c164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.116+0000 7fe80aa00640 1 -- 172.21.15.19:0/281365226 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165518138 0 0) 0x7fe80c164fc0 con 0x7fe80c160390 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.116+0000 7fe80aa00640 1 -- 172.21.15.19:0/281365226 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7f0003520 con 0x7fe80c160390 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.116+0000 7fe80be00640 1 -- 172.21.15.19:0/281365226 >> v2:172.21.15.169:3300/0 conn(0x7fe80c0af710 msgr2=0x7fe80c164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.116+0000 7fe80be00640 1 --2- 172.21.15.19:0/281365226 >> v2:172.21.15.169:3300/0 conn(0x7fe80c0af710 0x7fe80c164a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.116+0000 7fe80be00640 1 -- 172.21.15.19:0/281365226 >> v1:172.21.15.169:6789/0 conn(0x7fe80c160390 legacy=0x7fe80c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.116+0000 7fe80be00640 1 -- 172.21.15.19:0/281365226 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c06f720 con 0x7fe80c06dce0 2024-04-25T11:27:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.117+0000 7fe80be00640 1 --2- 172.21.15.19:0/281365226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c06dce0 0x7fe80c070270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe804009e50 tx=0x7fe80402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f86ca1b96914ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.118+0000 7fe80aa00640 1 -- 172.21.15.19:0/281365226 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804036070 con 0x7fe80c06dce0 2024-04-25T11:27:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.118+0000 7fe80aa00640 1 -- 172.21.15.19:0/281365226 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe804031600 con 0x7fe80c06dce0 2024-04-25T11:27:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.118+0000 7fe80aa00640 1 -- 172.21.15.19:0/281365226 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8040319d0 con 0x7fe80c06dce0 2024-04-25T11:27:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 -- 172.21.15.19:0/281365226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c06dce0 msgr2=0x7fe80c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 --2- 172.21.15.19:0/281365226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c06dce0 0x7fe80c070270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe804009e50 tx=0x7fe80402a710 comp rx=0 tx=0).stop 2024-04-25T11:27:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 -- 172.21.15.19:0/281365226 shutdown_connections 2024-04-25T11:27:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 --2- 172.21.15.19:0/281365226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c06dce0 0x7fe80c070270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 --2- 172.21.15.19:0/281365226 >> v2:172.21.15.169:3300/0 conn(0x7fe80c0af710 0x7fe80c164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 -- 172.21.15.19:0/281365226 >> 172.21.15.19:0/281365226 conn(0x7fe80c073090 msgr2=0x7fe80c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 -- 172.21.15.19:0/281365226 shutdown_connections 2024-04-25T11:27:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.119+0000 7fe812200640 1 -- 172.21.15.19:0/281365226 wait complete. 2024-04-25T11:27:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.122+0000 7fe812200640 1 Processor -- start 2024-04-25T11:27:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.122+0000 7fe812200640 1 -- start start 2024-04-25T11:27:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.123+0000 7fe812200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe80c0af710 0x7fe80c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.123+0000 7fe812200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe80c160390 0x7fe80c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.123+0000 7fe812200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c1561b0 0x7fe80c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.123+0000 7fe812200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c06ffc0 con 0x7fe80c160390 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.123+0000 7fe812200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c06fd20 con 0x7fe80c0af710 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.123+0000 7fe812200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c06f0d0 con 0x7fe80c1561b0 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe80c160390 0x7fe80c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe80c160390 0x7fe80c15d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49288/0 (socket says 172.21.15.19:49288) 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 -- 172.21.15.19:0/1482096832 learned_addr learned my addr 172.21.15.19:0/1482096832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe80b400640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe80c0af710 0x7fe80c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe80be00640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c1561b0 0x7fe80c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 -- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c1561b0 msgr2=0x7fe80c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c1561b0 0x7fe80c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 -- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe80c0af710 msgr2=0x7fe80c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe80c0af710 0x7fe80c15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe803e00640 1 -- 172.21.15.19:0/1482096832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c0d0b70 con 0x7fe80c160390 2024-04-25T11:27:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.124+0000 7fe80b400640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe80c0af710 0x7fe80c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.125+0000 7fe803e00640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe80c160390 0x7fe80c15d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c3c0 tx=0x7fe7f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.125+0000 7fe803400640 1 -- 172.21.15.19:0/1482096832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4017070 con 0x7fe80c160390 2024-04-25T11:27:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.125+0000 7fe803400640 1 -- 172.21.15.19:0/1482096832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f40052d0 con 0x7fe80c160390 2024-04-25T11:27:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.125+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c0cfad0 con 0x7fe80c160390 2024-04-25T11:27:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.125+0000 7fe803400640 1 -- 172.21.15.19:0/1482096832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4003810 con 0x7fe80c160390 2024-04-25T11:27:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.125+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c0d0090 con 0x7fe80c160390 2024-04-25T11:27:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.126+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c0b45a0 con 0x7fe80c160390 2024-04-25T11:27:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.128+0000 7fe803400640 1 -- 172.21.15.19:0/1482096832 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe7f401a070 con 0x7fe80c160390 2024-04-25T11:27:42.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.130+0000 7fe803400640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7e80744e0 0x7fe7e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.131+0000 7fe803400640 1 -- 172.21.15.19:0/1482096832 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe7f40945f0 con 0x7fe80c160390 2024-04-25T11:27:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.131+0000 7fe80b400640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7e80744e0 0x7fe7e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.132+0000 7fe80b400640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7e80744e0 0x7fe7e80769f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009af0 tx=0x7fe7fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:42.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.139+0000 7fe803400640 1 -- 172.21.15.19:0/1482096832 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe80c0b45a0 con 0x7fe80c160390 2024-04-25T11:27:42.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.341+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe80c050740 con 0x7fe80c160390 2024-04-25T11:27:42.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.342+0000 7fe803400640 1 -- 172.21.15.19:0/1482096832 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fe7f405ece0 con 0x7fe80c160390 2024-04-25T11:27:42.333 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7e80744e0 msgr2=0x7fe7e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7e80744e0 0x7fe7e80769f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009af0 tx=0x7fe7fc008040 comp rx=0 tx=0).stop 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe80c160390 msgr2=0x7fe80c15d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe80c160390 0x7fe80c15d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c3c0 tx=0x7fe7f400c880 comp rx=0 tx=0).stop 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 shutdown_connections 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7e80744e0 0x7fe7e80769f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe80c1561b0 0x7fe80c1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe80c160390 0x7fe80c15d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 --2- 172.21.15.19:0/1482096832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe80c0af710 0x7fe80c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.344+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 >> 172.21.15.19:0/1482096832 conn(0x7fe80c073090 msgr2=0x7fe80c0b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:42.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.345+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 shutdown_connections 2024-04-25T11:27:42.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.345+0000 7fe812200640 1 -- 172.21.15.19:0/1482096832 wait complete. 2024-04-25T11:27:42.363 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:27:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.659+0000 7ff4b6a00640 1 Processor -- start 2024-04-25T11:27:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.659+0000 7ff4b6a00640 1 -- start start 2024-04-25T11:27:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.662+0000 7ff4b6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 0x7ff4b006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.662+0000 7ff4b6a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b01623f0 con 0x7ff4b006fd40 2024-04-25T11:27:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.662+0000 7ff4b6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b01626d0 con 0x7ff4b0070d20 2024-04-25T11:27:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.662+0000 7ff4b6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b006f130 con 0x7ff4b006f930 2024-04-25T11:27:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.662+0000 7ff4b6000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff4b006fd40 0x7ff4b00ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54340/0 (socket says 172.21.15.19:54340) 2024-04-25T11:27:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.662+0000 7ff4b6000640 1 -- 172.21.15.19:0/3493719318 learned_addr learned my addr 172.21.15.19:0/3493719318 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.662+0000 7ff4b4c00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 0x7ff4b006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571909309 0 0) 0x7ff4b01623f0 con 0x7ff4b006fd40 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff498003520 con 0x7ff4b006fd40 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801515401 0 0) 0x7ff4b01626d0 con 0x7ff4b0070d20 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b01623f0 con 0x7ff4b0070d20 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3117262694 0 0) 0x7ff498003520 con 0x7ff4b006fd40 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 msgr2=0x7ff4b006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 0x7ff4b006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 >> v1:172.21.15.169:6789/0 conn(0x7ff4b0070d20 legacy=0x7ff4b0161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b006f410 con 0x7ff4b006fd40 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ff4a80019f0 con 0x7ff4b006fd40 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4b4c00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 0x7ff4b006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 >> v1:172.21.15.19:6789/0 conn(0x7ff4b006fd40 legacy=0x7ff4b00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 0x7ff498003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.663+0000 7ff4afe00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff498004120 0x7ff498004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4afe00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff498005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b006f130 con 0x7ff498004d40 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff49800bc20 con 0x7ff498004120 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff49800be20 con 0x7ff4b006f930 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b6000640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff498005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b5600640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 0x7ff498003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b4c00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff498004120 0x7ff498004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b6000640 1 -- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 msgr2=0x7ff498003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:42.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b6000640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b006f930 0x7ff498003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b6000640 1 -- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff498004120 msgr2=0x7ff498004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:42.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b6000640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff498004120 0x7ff498004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b6000640 1 -- 172.21.15.19:0/3493719318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff498003520 con 0x7ff498004d40 2024-04-25T11:27:42.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.664+0000 7ff4b4c00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff498004120 0x7ff498004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.665+0000 7ff4b6000640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff498005190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00b4a20 tx=0x7ff498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca351bc920954301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.666+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800a3b0 con 0x7ff498004d40 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.666+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a802f070 con 0x7ff498004d40 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.666+0000 7ff4afe00640 1 -- 172.21.15.19:0/3493719318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a802bb10 con 0x7ff498004d40 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.666+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3493719318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 msgr2=0x7ff498005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.666+0000 7ff4b6a00640 1 --2- 172.21.15.19:0/3493719318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff498005190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00b4a20 tx=0x7ff498005830 comp rx=0 tx=0).stop 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.666+0000 7ff4b6000640 1 -- 172.21.15.19:0/3493719318 reap_dead start 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.667+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3493719318 shutdown_connections 2024-04-25T11:27:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.667+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3493719318 >> 172.21.15.19:0/3493719318 conn(0x7ff4b0073160 msgr2=0x7ff4b0073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.667+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3493719318 shutdown_connections 2024-04-25T11:27:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.667+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3493719318 wait complete. 2024-04-25T11:27:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.669+0000 7ff4b6a00640 1 Processor -- start 2024-04-25T11:27:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.670+0000 7ff4b6a00640 1 -- start start 2024-04-25T11:27:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff4b00cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4b014cf30 0x7ff4b00cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b014ef30 0x7ff4b014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b6a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b015e470 con 0x7ff498004d40 2024-04-25T11:27:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b015e2f0 con 0x7ff4b014cf30 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b00ceef0 con 0x7ff4b014ef30 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff4b00cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff4b00cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49308/0 (socket says 172.21.15.19:49308) 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.671+0000 7ff4b5600640 1 -- 172.21.15.19:0/3585408168 learned_addr learned my addr 172.21.15.19:0/3585408168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4b4c00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4b014cf30 0x7ff4b00cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4b5600640 1 -- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b014ef30 msgr2=0x7ff4b014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4b5600640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b014ef30 0x7ff4b014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4b5600640 1 -- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4b014cf30 msgr2=0x7ff4b00cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4b5600640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4b014cf30 0x7ff4b00cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4b5600640 1 -- 172.21.15.19:0/3585408168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b014de90 con 0x7ff498004d40 2024-04-25T11:27:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4b5600640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff4b00cdcb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400e7b0 tx=0x7ff4a400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:42.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.672+0000 7ff4ad600640 1 -- 172.21.15.19:0/3585408168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400ee60 con 0x7ff498004d40 2024-04-25T11:27:42.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.673+0000 7ff4ad600640 1 -- 172.21.15.19:0/3585408168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a4018070 con 0x7ff498004d40 2024-04-25T11:27:42.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.673+0000 7ff4ad600640 1 -- 172.21.15.19:0/3585408168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400c640 con 0x7ff498004d40 2024-04-25T11:27:42.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.673+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b014e170 con 0x7ff498004d40 2024-04-25T11:27:42.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.673+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b014e730 con 0x7ff498004d40 2024-04-25T11:27:42.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.674+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b0076c60 con 0x7ff498004d40 2024-04-25T11:27:42.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.675+0000 7ff4ad600640 1 -- 172.21.15.19:0/3585408168 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff4a400c7e0 con 0x7ff498004d40 2024-04-25T11:27:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.676+0000 7ff4ad600640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4900744e0 0x7ff4900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.676+0000 7ff4ad600640 1 -- 172.21.15.19:0/3585408168 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff4a4094fb0 con 0x7ff498004d40 2024-04-25T11:27:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.677+0000 7ff4b4c00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4900744e0 0x7ff4900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:42.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.678+0000 7ff4b4c00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4900744e0 0x7ff4900769f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0161660 tx=0x7ff49c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.681+0000 7ff4ad600640 1 -- 172.21.15.19:0/3585408168 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4a405f730 con 0x7ff498004d40 2024-04-25T11:27:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.883+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4b0074070 con 0x7ff498004d40 2024-04-25T11:27:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.884+0000 7ff4ad600640 1 -- 172.21.15.19:0/3585408168 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ff4a405f030 con 0x7ff498004d40 2024-04-25T11:27:42.875 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.887+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4900744e0 msgr2=0x7ff4900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.887+0000 7ff4b6a00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4900744e0 0x7ff4900769f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0161660 tx=0x7ff49c008040 comp rx=0 tx=0).stop 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 msgr2=0x7ff4b00cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff4b00cdcb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400e7b0 tx=0x7ff4a400ec70 comp rx=0 tx=0).stop 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 shutdown_connections 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4900744e0 0x7ff4900769f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4b014cf30 0x7ff4b00cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4b014ef30 0x7ff4b014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 --2- 172.21.15.19:0/3585408168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff498004d40 0x7ff4b00cdcb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 >> 172.21.15.19:0/3585408168 conn(0x7ff4b0073160 msgr2=0x7ff4b0073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 shutdown_connections 2024-04-25T11:27:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:42.888+0000 7ff4b6a00640 1 -- 172.21.15.19:0/3585408168 wait complete. 2024-04-25T11:27:42.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:27:43.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.194+0000 7f61d3000640 1 Processor -- start 2024-04-25T11:27:43.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.194+0000 7f61d3000640 1 -- start start 2024-04-25T11:27:43.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.196+0000 7f61d3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc0b4ea0 0x7f61cc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.196+0000 7f61d3000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61cc0d1230 con 0x7f61cc144370 2024-04-25T11:27:43.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.196+0000 7f61d3000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61cc0d06b0 con 0x7f61cc06c5c0 2024-04-25T11:27:43.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.197+0000 7f61d3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc0d0990 con 0x7f61cc0b4ea0 2024-04-25T11:27:43.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.197+0000 7f61d2600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f61cc144370 0x7f61cc15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54344/0 (socket says 172.21.15.19:54344) 2024-04-25T11:27:43.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.197+0000 7f61d2600640 1 -- 172.21.15.19:0/3723495408 learned_addr learned my addr 172.21.15.19:0/3723495408 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:43.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.197+0000 7f61d1200640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc0b4ea0 0x7f61cc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.197+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230025467 0 0) 0x7f61cc0d1230 con 0x7f61cc144370 2024-04-25T11:27:43.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.197+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61b4003520 con 0x7f61cc144370 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 89206385 0 0) 0x7f61b4003520 con 0x7f61cc144370 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc0b4ea0 msgr2=0x7f61cc0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc0b4ea0 0x7f61cc0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 >> v1:172.21.15.169:6789/0 conn(0x7f61cc06c5c0 legacy=0x7f61cc0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc0d0c70 con 0x7f61cc144370 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f61c40019f0 con 0x7f61cc144370 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 >> v1:172.21.15.19:6789/0 conn(0x7f61cc144370 legacy=0x7f61cc15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61b40039e0 0x7f61b4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b4004800 0x7f61b4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b4005270 0x7f61b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0d0990 con 0x7f61b40039e0 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61b400bf50 con 0x7f61b4004800 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61b400c150 con 0x7f61b4005270 2024-04-25T11:27:43.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.198+0000 7f61d1c00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61b40039e0 0x7f61b4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.199+0000 7f61d2600640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b4005270 0x7f61b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.199+0000 7f61d1c00640 1 -- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b4005270 msgr2=0x7f61b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:43.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.199+0000 7f61d1c00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b4005270 0x7f61b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.199+0000 7f61d1c00640 1 -- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b4004800 msgr2=0x7f61b4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:43.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.199+0000 7f61d1200640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b4004800 0x7f61b4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.199+0000 7f61d1c00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61b4004800 0x7f61b4004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.199+0000 7f61d1c00640 1 -- 172.21.15.19:0/3723495408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b4003520 con 0x7f61b40039e0 2024-04-25T11:27:43.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.200+0000 7f61d1c00640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61b40039e0 0x7f61b4003e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61c0001750 tx=0x7f61b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5410d4c269be648d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:43.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.200+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0001370 con 0x7f61b40039e0 2024-04-25T11:27:43.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.200+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c00052d0 con 0x7f61b40039e0 2024-04-25T11:27:43.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.200+0000 7f61cbe00640 1 -- 172.21.15.19:0/3723495408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c002abe0 con 0x7f61b40039e0 2024-04-25T11:27:43.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.201+0000 7f61d3000640 1 -- 172.21.15.19:0/3723495408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61b40039e0 msgr2=0x7f61b4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.201+0000 7f61d3000640 1 --2- 172.21.15.19:0/3723495408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61b40039e0 0x7f61b4003e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61c0001750 tx=0x7f61b4005830 comp rx=0 tx=0).stop 2024-04-25T11:27:43.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.201+0000 7f61d2600640 1 -- 172.21.15.19:0/3723495408 reap_dead start 2024-04-25T11:27:43.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.201+0000 7f61d3000640 1 -- 172.21.15.19:0/3723495408 shutdown_connections 2024-04-25T11:27:43.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.201+0000 7f61d3000640 1 -- 172.21.15.19:0/3723495408 >> 172.21.15.19:0/3723495408 conn(0x7f61cc0d1720 msgr2=0x7f61cc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:43.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.201+0000 7f61d3000640 1 -- 172.21.15.19:0/3723495408 shutdown_connections 2024-04-25T11:27:43.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.201+0000 7f61d3000640 1 -- 172.21.15.19:0/3723495408 wait complete. 2024-04-25T11:27:43.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.204+0000 7f61d3000640 1 Processor -- start 2024-04-25T11:27:43.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.204+0000 7f61d3000640 1 -- start start 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.205+0000 7f61d3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b40039e0 0x7f61cc140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.205+0000 7f61d3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc1409e0 0x7f61cc1e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.205+0000 7f61d3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc1e99f0 0x7f61cc1e7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.205+0000 7f61d3000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0d0450 con 0x7f61cc1e99f0 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.205+0000 7f61d3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0d02d0 con 0x7f61cc1409e0 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.205+0000 7f61d3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc147760 con 0x7f61b40039e0 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc1e99f0 0x7f61cc1e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc1e99f0 0x7f61cc1e7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49314/0 (socket says 172.21.15.19:49314) 2024-04-25T11:27:43.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 -- 172.21.15.19:0/781429680 learned_addr learned my addr 172.21.15.19:0/781429680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d1c00640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b40039e0 0x7f61cc140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 -- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b40039e0 msgr2=0x7f61cc140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b40039e0 0x7f61cc140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 -- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc1409e0 msgr2=0x7f61cc1e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc1409e0 0x7f61cc1e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 -- 172.21.15.19:0/781429680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc0cabb0 con 0x7f61cc1e99f0 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d1c00640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b40039e0 0x7f61cc140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.206+0000 7f61d2600640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc1e99f0 0x7f61cc1e7850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b7a50 tx=0x7f61c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.207+0000 7f61c9600640 1 -- 172.21.15.19:0/781429680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c400ec10 con 0x7f61cc1e99f0 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.207+0000 7f61c9600640 1 -- 172.21.15.19:0/781429680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c4018070 con 0x7f61cc1e99f0 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.207+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc141230 con 0x7f61cc1e99f0 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.207+0000 7f61c9600640 1 -- 172.21.15.19:0/781429680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4013420 con 0x7f61cc1e99f0 2024-04-25T11:27:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.207+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc1ea3f0 con 0x7f61cc1e99f0 2024-04-25T11:27:43.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.208+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc050740 con 0x7f61cc1e99f0 2024-04-25T11:27:43.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.209+0000 7f61c9600640 1 -- 172.21.15.19:0/781429680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61c40135c0 con 0x7f61cc1e99f0 2024-04-25T11:27:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.210+0000 7f61c9600640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.211+0000 7f61c9600640 1 -- 172.21.15.19:0/781429680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f61c4095540 con 0x7f61cc1e99f0 2024-04-25T11:27:43.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.211+0000 7f61d1c00640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.213+0000 7f61d1c00640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61c0032000 tx=0x7f61c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:43.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.215+0000 7f61c9600640 1 -- 172.21.15.19:0/781429680 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61c405fd60 con 0x7f61cc1e99f0 2024-04-25T11:27:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.460+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61cc070e80 con 0x7f61cc1e99f0 2024-04-25T11:27:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.461+0000 7f61c9600640 1 -- 172.21.15.19:0/781429680 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f61c405f660 con 0x7f61cc1e99f0 2024-04-25T11:27:43.452 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:43.452 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:27:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:27:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 msgr2=0x7f61ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61c0032000 tx=0x7f61c002e000 comp rx=0 tx=0).stop 2024-04-25T11:27:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc1e99f0 msgr2=0x7f61cc1e7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc1e99f0 0x7f61cc1e7850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b7a50 tx=0x7f61c400e530 comp rx=0 tx=0).stop 2024-04-25T11:27:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 shutdown_connections 2024-04-25T11:27:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc1409e0 0x7f61cc1e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc1e99f0 0x7f61cc1e7850 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 --2- 172.21.15.19:0/781429680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61b40039e0 0x7f61cc140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 >> 172.21.15.19:0/781429680 conn(0x7f61cc0d1720 msgr2=0x7f61cc072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 shutdown_connections 2024-04-25T11:27:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.464+0000 7f61d3000640 1 -- 172.21.15.19:0/781429680 wait complete. 2024-04-25T11:27:43.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:27:43.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.773+0000 7fb071600640 1 Processor -- start 2024-04-25T11:27:43.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.774+0000 7fb071600640 1 -- start start 2024-04-25T11:27:43.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb071600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb06c0cf9e0 0x7fb06c161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb071600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0709a0 0x7fb06c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb071600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb06c1615d0 con 0x7fb06c0b0800 2024-04-25T11:27:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb071600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb06c0cffc0 con 0x7fb06c0cf9e0 2024-04-25T11:27:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb071600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb06c0d02a0 con 0x7fb06c0709a0 2024-04-25T11:27:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb06be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb06c0cf9e0 0x7fb06c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb06be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb06c0cf9e0 0x7fb06c161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49322/0 (socket says 172.21.15.19:49322) 2024-04-25T11:27:43.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.776+0000 7fb06be00640 1 -- 172.21.15.19:0/995791058 learned_addr learned my addr 172.21.15.19:0/995791058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.777+0000 7fb070c00640 1 --2- 172.21.15.19:0/995791058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0709a0 0x7fb06c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.777+0000 7fb06be00640 1 -- 172.21.15.19:0/995791058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0709a0 msgr2=0x7fb06c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.777+0000 7fb06be00640 1 --2- 172.21.15.19:0/995791058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0709a0 0x7fb06c1569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.777+0000 7fb06be00640 1 -- 172.21.15.19:0/995791058 >> v1:172.21.15.169:6789/0 conn(0x7fb06c0b0800 legacy=0x7fb06c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.777+0000 7fb06be00640 1 -- 172.21.15.19:0/995791058 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c0d0580 con 0x7fb06c0cf9e0 2024-04-25T11:27:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.778+0000 7fb06be00640 1 --2- 172.21.15.19:0/995791058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c0cf9e0 0x7fb06c161090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009870 tx=0x7fb05c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca70abc82cb6548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.778+0000 7fb06b400640 1 -- 172.21.15.19:0/995791058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c037070 con 0x7fb06c0cf9e0 2024-04-25T11:27:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.778+0000 7fb06b400640 1 -- 172.21.15.19:0/995791058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05c032440 con 0x7fb06c0cf9e0 2024-04-25T11:27:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.778+0000 7fb06b400640 1 -- 172.21.15.19:0/995791058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c032810 con 0x7fb06c0cf9e0 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.779+0000 7fb071600640 1 -- 172.21.15.19:0/995791058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c0cf9e0 msgr2=0x7fb06c161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.779+0000 7fb071600640 1 --2- 172.21.15.19:0/995791058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c0cf9e0 0x7fb06c161090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009870 tx=0x7fb05c02a180 comp rx=0 tx=0).stop 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.779+0000 7fb071600640 1 -- 172.21.15.19:0/995791058 shutdown_connections 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.779+0000 7fb071600640 1 --2- 172.21.15.19:0/995791058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c0709a0 0x7fb06c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.779+0000 7fb071600640 1 --2- 172.21.15.19:0/995791058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c0cf9e0 0x7fb06c161090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.780+0000 7fb071600640 1 -- 172.21.15.19:0/995791058 >> 172.21.15.19:0/995791058 conn(0x7fb06c0d1720 msgr2=0x7fb06c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.780+0000 7fb071600640 1 -- 172.21.15.19:0/995791058 shutdown_connections 2024-04-25T11:27:43.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.780+0000 7fb071600640 1 -- 172.21.15.19:0/995791058 wait complete. 2024-04-25T11:27:43.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.783+0000 7fb071600640 1 Processor -- start 2024-04-25T11:27:43.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.783+0000 7fb071600640 1 -- start start 2024-04-25T11:27:43.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb071600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0b0800 0x7fb06c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb071600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c145160 0x7fb06c15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb071600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c15b2a0 0x7fb06c0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb071600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c0b0d90 con 0x7fb06c145160 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb071600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c0b0c10 con 0x7fb06c0b0800 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb071600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb06c156ef0 con 0x7fb06c15b2a0 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb063e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c145160 0x7fb06c15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb063e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c145160 0x7fb06c15ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49348/0 (socket says 172.21.15.19:49348) 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb063e00640 1 -- 172.21.15.19:0/1996699727 learned_addr learned my addr 172.21.15.19:0/1996699727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb070c00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c15b2a0 0x7fb06c0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb063e00640 1 -- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c15b2a0 msgr2=0x7fb06c0b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb063e00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c15b2a0 0x7fb06c0b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb06be00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0b0800 0x7fb06c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.785+0000 7fb063e00640 1 -- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0b0800 msgr2=0x7fb06c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb063e00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0b0800 0x7fb06c158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb063e00640 1 -- 172.21.15.19:0/1996699727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c145ad0 con 0x7fb06c145160 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb070c00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c15b2a0 0x7fb06c0b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb063e00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c145160 0x7fb06c15ad60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb05400b3e0 tx=0x7fb05400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb068c00640 1 -- 172.21.15.19:0/1996699727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054015020 con 0x7fb06c145160 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb068c00640 1 -- 172.21.15.19:0/1996699727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb054011850 con 0x7fb06c145160 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb068c00640 1 -- 172.21.15.19:0/1996699727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054011c40 con 0x7fb06c145160 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c145db0 con 0x7fb06c145160 2024-04-25T11:27:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.786+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c146370 con 0x7fb06c145160 2024-04-25T11:27:43.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.788+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06c050740 con 0x7fb06c145160 2024-04-25T11:27:43.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.789+0000 7fb068c00640 1 -- 172.21.15.19:0/1996699727 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb054018020 con 0x7fb06c145160 2024-04-25T11:27:43.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.789+0000 7fb068c00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744e0 0x7fb0480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:43.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.790+0000 7fb068c00640 1 -- 172.21.15.19:0/1996699727 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb0540951d0 con 0x7fb06c145160 2024-04-25T11:27:43.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.790+0000 7fb06be00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744e0 0x7fb0480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:43.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.791+0000 7fb06be00640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744e0 0x7fb0480769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb05c034000 tx=0x7fb05c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.795+0000 7fb068c00640 1 -- 172.21.15.19:0/1996699727 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb05401f020 con 0x7fb06c145160 2024-04-25T11:27:43.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.997+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb06c06fb60 con 0x7fb06c145160 2024-04-25T11:27:43.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:43.998+0000 7fb068c00640 1 -- 172.21.15.19:0/1996699727 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7fb05405f8c0 con 0x7fb06c145160 2024-04-25T11:27:43.990 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:43.990 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744e0 msgr2=0x7fb0480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744e0 0x7fb0480769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb05c034000 tx=0x7fb05c02e000 comp rx=0 tx=0).stop 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c145160 msgr2=0x7fb06c15ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c145160 0x7fb06c15ad60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb05400b3e0 tx=0x7fb05400b8a0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 shutdown_connections 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744e0 0x7fb0480769f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06c15b2a0 0x7fb06c0b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb06c145160 0x7fb06c15ad60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.001+0000 7fb071600640 1 --2- 172.21.15.19:0/1996699727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb06c0b0800 0x7fb06c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:43.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.002+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 >> 172.21.15.19:0/1996699727 conn(0x7fb06c0d1720 msgr2=0x7fb06c0737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:43.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.002+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 shutdown_connections 2024-04-25T11:27:43.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.002+0000 7fb071600640 1 -- 172.21.15.19:0/1996699727 wait complete. 2024-04-25T11:27:44.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:27:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.302+0000 7f09f1200640 1 Processor -- start 2024-04-25T11:27:44.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.302+0000 7f09f1200640 1 -- start start 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.304+0000 7f09f1200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f09ec0b7c50 0x7f09ec0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.304+0000 7f09f1200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09ec0d44a0 0x7f09ec0d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.304+0000 7f09f1200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09ec161650 con 0x7f09ec164860 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.304+0000 7f09f1200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec1643d0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.304+0000 7f09f1200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f09ec0b5270 con 0x7f09ec0b7c50 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09ebe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09ec0d44a0 0x7f09ec0d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09ebe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09ec0d44a0 0x7f09ec0d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49362/0 (socket says 172.21.15.19:49362) 2024-04-25T11:27:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09ebe00640 1 -- 172.21.15.19:0/948614875 learned_addr learned my addr 172.21.15.19:0/948614875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09eaa00640 1 --2- 172.21.15.19:0/948614875 >> v2:172.21.15.169:3300/0 conn(0x7f09ec0b7c50 0x7f09ec0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09ebe00640 1 -- 172.21.15.19:0/948614875 >> v2:172.21.15.169:3300/0 conn(0x7f09ec0b7c50 msgr2=0x7f09ec0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09ebe00640 1 --2- 172.21.15.19:0/948614875 >> v2:172.21.15.169:3300/0 conn(0x7f09ec0b7c50 0x7f09ec0b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09ebe00640 1 -- 172.21.15.19:0/948614875 >> v1:172.21.15.169:6789/0 conn(0x7f09ec164860 legacy=0x7f09ec0b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.305+0000 7f09ebe00640 1 -- 172.21.15.19:0/948614875 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec161370 con 0x7f09ec0d44a0 2024-04-25T11:27:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.307+0000 7f09ebe00640 1 --2- 172.21.15.19:0/948614875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec0d48b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09ec08e4c0 tx=0x7f09e402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b159bb402eccd86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.307+0000 7f09ea000640 1 -- 172.21.15.19:0/948614875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e4036070 con 0x7f09ec0d44a0 2024-04-25T11:27:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.307+0000 7f09ea000640 1 -- 172.21.15.19:0/948614875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e4031600 con 0x7f09ec0d44a0 2024-04-25T11:27:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.307+0000 7f09ea000640 1 -- 172.21.15.19:0/948614875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e40319d0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 -- 172.21.15.19:0/948614875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 msgr2=0x7f09ec0d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 --2- 172.21.15.19:0/948614875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec0d48b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09ec08e4c0 tx=0x7f09e402a730 comp rx=0 tx=0).stop 2024-04-25T11:27:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 -- 172.21.15.19:0/948614875 shutdown_connections 2024-04-25T11:27:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 --2- 172.21.15.19:0/948614875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec0d48b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 --2- 172.21.15.19:0/948614875 >> v2:172.21.15.169:3300/0 conn(0x7f09ec0b7c50 0x7f09ec0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 -- 172.21.15.19:0/948614875 >> 172.21.15.19:0/948614875 conn(0x7f09ec0d2720 msgr2=0x7f09ec0d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 -- 172.21.15.19:0/948614875 shutdown_connections 2024-04-25T11:27:44.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.308+0000 7f09f1200640 1 -- 172.21.15.19:0/948614875 wait complete. 2024-04-25T11:27:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.310+0000 7f09f1200640 1 Processor -- start 2024-04-25T11:27:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.310+0000 7f09f1200640 1 -- start start 2024-04-25T11:27:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09f1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec14ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09f1200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09ec164860 0x7f09ec15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09f1200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09ec15b0e0 0x7f09ec15f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:44.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09f1200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec0d4df0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09f1200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec164060 con 0x7f09ec164860 2024-04-25T11:27:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09f1200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec0d4f70 con 0x7f09ec15b0e0 2024-04-25T11:27:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09eb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec14ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09eb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec14ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49370/0 (socket says 172.21.15.19:49370) 2024-04-25T11:27:44.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.312+0000 7f09eb400640 1 -- 172.21.15.19:0/3237913248 learned_addr learned my addr 172.21.15.19:0/3237913248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09eb400640 1 -- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09ec15b0e0 msgr2=0x7f09ec15f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09eb400640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09ec15b0e0 0x7f09ec15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09eb400640 1 -- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09ec164860 msgr2=0x7f09ec15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09eb400640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09ec164860 0x7f09ec15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09eb400640 1 -- 172.21.15.19:0/3237913248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec14b9a0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09eb400640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec14ae20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c3f0 tx=0x7f09dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09e3400640 1 -- 172.21.15.19:0/3237913248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0052d0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09e3400640 1 -- 172.21.15.19:0/3237913248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09dc005490 con 0x7f09ec0d44a0 2024-04-25T11:27:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec14bc80 con 0x7f09ec0d44a0 2024-04-25T11:27:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.313+0000 7f09e3400640 1 -- 172.21.15.19:0/3237913248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc007260 con 0x7f09ec0d44a0 2024-04-25T11:27:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.314+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec14c240 con 0x7f09ec0d44a0 2024-04-25T11:27:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.315+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec050740 con 0x7f09ec0d44a0 2024-04-25T11:27:44.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.316+0000 7f09e3400640 1 -- 172.21.15.19:0/3237913248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09dc007400 con 0x7f09ec0d44a0 2024-04-25T11:27:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.317+0000 7f09e3400640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c80744b0 0x7f09c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.317+0000 7f09e3400640 1 -- 172.21.15.19:0/3237913248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f09dc0946f0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.318+0000 7f09eaa00640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c80744b0 0x7f09c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.319+0000 7f09eaa00640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c80744b0 0x7f09c80769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f09ec160b30 tx=0x7f09d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:44.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.322+0000 7f09e3400640 1 -- 172.21.15.19:0/3237913248 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09dc05ede0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.570+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09ec0d3f30 con 0x7f09ec0d44a0 2024-04-25T11:27:44.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.571+0000 7f09e3400640 1 -- 172.21.15.19:0/3237913248 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f09dc05e6e0 con 0x7f09ec0d44a0 2024-04-25T11:27:44.563 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:27:44.563 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:27:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:27:44.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c80744b0 msgr2=0x7f09c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c80744b0 0x7f09c80769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f09ec160b30 tx=0x7f09d4008040 comp rx=0 tx=0).stop 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 msgr2=0x7f09ec14ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec14ae20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c3f0 tx=0x7f09dc00c7f0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 shutdown_connections 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c80744b0 0x7f09c80769c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09ec164860 0x7f09ec15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09ec15b0e0 0x7f09ec15f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 --2- 172.21.15.19:0/3237913248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09ec0d44a0 0x7f09ec14ae20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.574+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 >> 172.21.15.19:0/3237913248 conn(0x7f09ec0d2720 msgr2=0x7f09ec1621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.575+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 shutdown_connections 2024-04-25T11:27:44.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:44.575+0000 7f09f1200640 1 -- 172.21.15.19:0/3237913248 wait complete. 2024-04-25T11:27:44.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:27:25.102980+0000', 'modified': '2024-04-25T11:27:28.179340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6836/2467347557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2467347557}, {'type': 'v1', 'addr': '172.21.15.169:6836', 'nonce': 2467347557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4297, 'qdb_cluster': [4297]} 2024-04-25T11:27:44.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:27:44.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:27:45.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.300+0000 7fb1fdc00640 1 Processor -- start 2024-04-25T11:27:45.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.301+0000 7fb1fdc00640 1 -- start start 2024-04-25T11:27:45.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.303+0000 7fb1fdc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1f80d1070 0x7fb1f8162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.303+0000 7fb1fdc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f806fc70 con 0x7fb1f8144e60 2024-04-25T11:27:45.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.303+0000 7fb1fdc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f806ff50 con 0x7fb1f8144350 2024-04-25T11:27:45.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.303+0000 7fb1fdc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb1f8070230 con 0x7fb1f80d1070 2024-04-25T11:27:45.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.304+0000 7fb1fd200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb1f8144e60 0x7fb1f8145270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36198/0 (socket says 172.21.15.19:36198) 2024-04-25T11:27:45.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.304+0000 7fb1fd200640 1 -- 172.21.15.19:0/3873493497 learned_addr learned my addr 172.21.15.19:0/3873493497 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.304+0000 7fb1efe00640 1 --2- 172.21.15.19:0/3873493497 >> v2:172.21.15.169:3300/0 conn(0x7fb1f80d1070 0x7fb1f8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.304+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265451509 0 0) 0x7fb1f806fc70 con 0x7fb1f8144e60 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.304+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1dc003520 con 0x7fb1f8144e60 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.304+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2394024735 0 0) 0x7fb1dc003520 con 0x7fb1f8144e60 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 >> v2:172.21.15.169:3300/0 conn(0x7fb1f80d1070 msgr2=0x7fb1f8162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 --2- 172.21.15.19:0/3873493497 >> v2:172.21.15.169:3300/0 conn(0x7fb1f80d1070 0x7fb1f8162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 >> v1:172.21.15.169:6789/0 conn(0x7fb1f8144350 legacy=0x7fb1f8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f8070510 con 0x7fb1f8144e60 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb1f00019f0 con 0x7fb1f8144e60 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 >> v1:172.21.15.19:6789/0 conn(0x7fb1f8144e60 legacy=0x7fb1f8145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1dc0039b0 0x7fb1dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1dc004800 0x7fb1dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc0051c0 0x7fb1dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8070230 con 0x7fb1dc004800 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc00c160 con 0x7fb1dc0051c0 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb1dc00c360 con 0x7fb1dc0039b0 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1efe00640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1dc004800 0x7fb1dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1dc0039b0 0x7fb1dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1fd200640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc0051c0 0x7fb1dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.305+0000 7fb1efe00640 1 -- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1dc0039b0 msgr2=0x7fb1dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.306+0000 7fb1efe00640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1dc0039b0 0x7fb1dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.306+0000 7fb1efe00640 1 -- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc0051c0 msgr2=0x7fb1dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.306+0000 7fb1efe00640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc0051c0 0x7fb1dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.306+0000 7fb1efe00640 1 -- 172.21.15.19:0/3873493497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc003520 con 0x7fb1dc004800 2024-04-25T11:27:45.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.306+0000 7fb1fd200640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc0051c0 0x7fb1dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:45.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.306+0000 7fb1fd200640 1 -- 172.21.15.19:0/3873493497 reap_dead start 2024-04-25T11:27:45.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.307+0000 7fb1efe00640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1dc004800 0x7fb1dc004c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009ac0 tx=0x7fb1dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a276568a8480975f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.307+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0036070 con 0x7fb1dc004800 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.307+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0031400 con 0x7fb1dc004800 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.307+0000 7fb1f7400640 1 -- 172.21.15.19:0/3873493497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0009690 con 0x7fb1dc004800 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.308+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3873493497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1dc004800 msgr2=0x7fb1dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.308+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1dc004800 0x7fb1dc004c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009ac0 tx=0x7fb1dc005830 comp rx=0 tx=0).stop 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.308+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3873493497 shutdown_connections 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.308+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3873493497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1dc004800 0x7fb1dc004c10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.308+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3873493497 >> 172.21.15.19:0/3873493497 conn(0x7fb1f80d1720 msgr2=0x7fb1f80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.308+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3873493497 shutdown_connections 2024-04-25T11:27:45.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.308+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3873493497 wait complete. 2024-04-25T11:27:45.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.310+0000 7fb1fdc00640 1 Processor -- start 2024-04-25T11:27:45.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.310+0000 7fb1fdc00640 1 -- start start 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fdc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc004800 0x7fb1f813d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fdc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1f813cf30 0x7fb1f81f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fdc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f81f7240 0x7fb1f80b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fdc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8161af0 con 0x7fb1f81f7240 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fdc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8147910 con 0x7fb1dc004800 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fdc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f8161c70 con 0x7fb1f813cf30 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f81f7240 0x7fb1f80b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f81f7240 0x7fb1f80b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49384/0 (socket says 172.21.15.19:49384) 2024-04-25T11:27:45.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1fd200640 1 -- 172.21.15.19:0/3030394441 learned_addr learned my addr 172.21.15.19:0/3030394441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc004800 0x7fb1f813d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.312+0000 7fb1efe00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1f813cf30 0x7fb1f81f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1fd200640 1 -- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1f813cf30 msgr2=0x7fb1f81f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1fd200640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1f813cf30 0x7fb1f81f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1fd200640 1 -- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc004800 msgr2=0x7fb1f813d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1fd200640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc004800 0x7fb1f813d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1fd200640 1 -- 172.21.15.19:0/3030394441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f815ed30 con 0x7fb1f81f7240 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc004800 0x7fb1f813d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:27:45.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1fd200640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f81f7240 0x7fb1f80b1f10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8072e90 tx=0x7fb1f000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1f4c00640 1 -- 172.21.15.19:0/3030394441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0004b60 con 0x7fb1f81f7240 2024-04-25T11:27:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1f4c00640 1 -- 172.21.15.19:0/3030394441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f0004d00 con 0x7fb1f81f7240 2024-04-25T11:27:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.313+0000 7fb1f4c00640 1 -- 172.21.15.19:0/3030394441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0007270 con 0x7fb1f81f7240 2024-04-25T11:27:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.314+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f815f010 con 0x7fb1f81f7240 2024-04-25T11:27:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.314+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f815f5d0 con 0x7fb1f81f7240 2024-04-25T11:27:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.317+0000 7fb1f4c00640 1 -- 172.21.15.19:0/3030394441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb1f001b020 con 0x7fb1f81f7240 2024-04-25T11:27:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.318+0000 7fb1f4c00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1d4074530 0x7fb1d4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.318+0000 7fb1f4c00640 1 -- 172.21.15.19:0/3030394441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb1f0095520 con 0x7fb1f81f7240 2024-04-25T11:27:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.319+0000 7fb206256740 1 Processor -- start 2024-04-25T11:27:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.319+0000 7fb206256740 1 -- start start 2024-04-25T11:27:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.319+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1d4074530 0x7fb1d4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.320+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050c1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.320+0000 7fb206256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cc73b0 0x561a050dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.320+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a050dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.320+0000 7fb206256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a04ff6600 con 0x561a04cc73b0 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.320+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561a04f93100 con 0x561a04ccf590 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.320+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561a04ff25c0 con 0x561a04cf0270 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.320+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1d4074530 0x7fb1d4076a40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8069ed0 tx=0x7fb1e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cc73b0 0x561a050dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cc73b0 0x561a050dc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49390/0 (socket says 172.21.15.19:49390) 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 -- 172.21.15.19:0/855741703 learned_addr learned my addr 172.21.15.19:0/855741703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1fd200640 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a050dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 -- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 msgr2=0x561a050c1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050c1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 -- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 msgr2=0x561a050dc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a050dc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.321+0000 7fb1efe00640 1 -- 172.21.15.19:0/855741703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a04d082d0 con 0x561a04cc73b0 2024-04-25T11:27:45.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.322+0000 7fb1efe00640 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cc73b0 0x561a050dc350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1e002abb0 tx=0x7fb1e002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3128e794248207f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.322+0000 7fb1ecc00640 1 -- 172.21.15.19:0/855741703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0036040 con 0x561a04cc73b0 2024-04-25T11:27:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.322+0000 7fb1ecc00640 1 -- 172.21.15.19:0/855741703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0041070 con 0x561a04cc73b0 2024-04-25T11:27:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.322+0000 7fb1ecc00640 1 -- 172.21.15.19:0/855741703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0003610 con 0x561a04cc73b0 2024-04-25T11:27:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 -- 172.21.15.19:0/855741703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cc73b0 msgr2=0x561a050dc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cc73b0 0x561a050dc350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1e002abb0 tx=0x7fb1e002e000 comp rx=0 tx=0).stop 2024-04-25T11:27:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 -- 172.21.15.19:0/855741703 shutdown_connections 2024-04-25T11:27:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a050dc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cc73b0 0x561a050dc350 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 --2- 172.21.15.19:0/855741703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 -- 172.21.15.19:0/855741703 >> 172.21.15.19:0/855741703 conn(0x561a04ce43c0 msgr2=0x561a05096c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 -- 172.21.15.19:0/855741703 shutdown_connections 2024-04-25T11:27:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.323+0000 7fb206256740 1 -- 172.21.15.19:0/855741703 wait complete. 2024-04-25T11:27:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.324+0000 7fb206256740 1 -- start start 2024-04-25T11:27:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.324+0000 7fb206256740 1 Processor -- start 2024-04-25T11:27:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a050b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb206256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04ccf590 0x561a050eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb206256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a04f7df70 con 0x561a04ccf590 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561a050dafc0 con 0x561a04cc73b0 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561a050d4c10 con 0x561a04cf0270 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb206256740 10 client.0 _ll_register_callbacks cb 0x561a04fec480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04ccf590 0x561a050eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04ccf590 0x561a050eef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49398/0 (socket says 172.21.15.19:49398) 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb1efe00640 1 -- 172.21.15.19:0/1502875178 learned_addr learned my addr 172.21.15.19:0/1502875178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a050b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.326+0000 7fb1fd200640 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1efe00640 1 -- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 msgr2=0x561a050ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1efe00640 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1efe00640 1 -- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 msgr2=0x561a050b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1efe00640 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a050b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1efe00640 1 -- 172.21.15.19:0/1502875178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a04dd26b0 con 0x561a04ccf590 2024-04-25T11:27:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1efe00640 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04ccf590 0x561a050eef70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009a90 tx=0x7fb1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfcd72299309bac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1e5600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:27:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1e5600640 1 -- 172.21.15.19:0/1502875178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0036040 con 0x561a04ccf590 2024-04-25T11:27:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb1e5600640 1 -- 172.21.15.19:0/1502875178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e0005b90 con 0x561a04ccf590 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.327+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e0041070 con 0x561a04ccf590 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb1e5600640 1 -- 172.21.15.19:0/1502875178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb1e0057050 con 0x561a04ccf590 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 10 client.4629 fetch_fsmap learned FSMap version 8 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x561a050efba0 con 0x561a04ccf590 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb1e5600640 1 -- 172.21.15.19:0/1502875178 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0041070 con 0x561a04ccf590 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb1e5600640 1 -- 172.21.15.19:0/1502875178 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fb1e005a020 con 0x561a04ccf590 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 10 client.4629 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 20 client.4629 populate_metadata read hostname 'smithi019' 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a04caee40 0x561a050f9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 4 client.4629 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.328+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 --> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a05004210 con 0x561a04caee40 2024-04-25T11:27:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.329+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a04caee40 0x561a050f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.330+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a04caee40 0x561a050f9590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd35ab3fc83b9d8 server_cookie=c71b88042492b9e1 in_seq=0 out_seq=0 2024-04-25T11:27:45.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.330+0000 7fb1e5600640 10 client.4629 ms_handle_connect on v2:172.21.15.169:6834/2467347557 2024-04-25T11:27:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb1e5600640 1 -- 172.21.15.19:0/1502875178 <== mds.0 v2:172.21.15.169:6834/2467347557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a05004210 con 0x561a04caee40 2024-04-25T11:27:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb1e5600640 10 client.4629 handle_command_reply: tid=0 2024-04-25T11:27:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 client.4629 shutdown 2024-04-25T11:27:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04ccf590 msgr2=0x561a050eef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04ccf590 0x561a050eef70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009a90 tx=0x7fb1e00046c0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 shutdown_connections 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a04caee40 0x561a050f9590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cf0270 0x561a050ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04ccf590 0x561a050eef70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 --2- 172.21.15.19:0/1502875178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a050b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb1e5600640 0 client.4629 ms_handle_reset on v2:172.21.15.169:6834/2467347557 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 >> 172.21.15.19:0/1502875178 conn(0x561a04ce43c0 msgr2=0x561a050be2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 shutdown_connections 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.333+0000 7fb206256740 1 -- 172.21.15.19:0/1502875178 wait complete. 2024-04-25T11:27:45.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.334+0000 7fb206256740 20 client.4629 trim_cache size 0 max 16384 2024-04-25T11:27:45.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.355+0000 7fb206256740 1 Processor -- start 2024-04-25T11:27:45.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.355+0000 7fb206256740 1 -- start start 2024-04-25T11:27:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.356+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a0514fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.356+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.356+0000 7fb206256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.356+0000 7fb206256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f80b60f0 con 0x561a04cf0270 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.356+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8160200 con 0x561a04cc73b0 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.356+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f80b5aa0 con 0x561a04ccf590 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49406/0 (socket says 172.21.15.19:49406) 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 -- 172.21.15.19:0/1198244296 learned_addr learned my addr 172.21.15.19:0/1198244296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a0514fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1efe00640 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 -- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04ccf590 msgr2=0x561a05096970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 -- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 msgr2=0x561a0514fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a0514fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.357+0000 7fb1fd200640 1 -- 172.21.15.19:0/1198244296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a050926b0 con 0x561a04cf0270 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.358+0000 7fb1fd200640 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00939d0 tx=0x7fb1f001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d683baff7de6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.358+0000 7fb1e7e00640 1 -- 172.21.15.19:0/1198244296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f005f590 con 0x561a04cf0270 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.358+0000 7fb1e7e00640 1 -- 172.21.15.19:0/1198244296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f0063940 con 0x561a04cf0270 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.358+0000 7fb1e7e00640 1 -- 172.21.15.19:0/1198244296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f008ce10 con 0x561a04cf0270 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 -- 172.21.15.19:0/1198244296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 msgr2=0x561a05096eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00939d0 tx=0x7fb1f001f040 comp rx=0 tx=0).stop 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 -- 172.21.15.19:0/1198244296 shutdown_connections 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 --2- 172.21.15.19:0/1198244296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04cc73b0 0x561a0514fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 -- 172.21.15.19:0/1198244296 >> 172.21.15.19:0/1198244296 conn(0x561a04ce43c0 msgr2=0x561a050d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 -- 172.21.15.19:0/1198244296 shutdown_connections 2024-04-25T11:27:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.359+0000 7fb206256740 1 -- 172.21.15.19:0/1198244296 wait complete. 2024-04-25T11:27:45.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.360+0000 7fb206256740 1 -- start start 2024-04-25T11:27:45.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.360+0000 7fb206256740 1 Processor -- start 2024-04-25T11:27:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cc73b0 0x561a050d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb206256740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb206256740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb206256740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a050d4700 con 0x561a04cf0270 2024-04-25T11:27:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561a04f7df70 con 0x561a04ccf590 2024-04-25T11:27:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb206256740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561a050d7e60 con 0x561a04cc73b0 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb206256740 10 client.0 _ll_register_callbacks cb 0x7fb1dc0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49410/0 (socket says 172.21.15.19:49410) 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 -- 172.21.15.19:0/275320610 learned_addr learned my addr 172.21.15.19:0/275320610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cc73b0 0x561a050d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1efe00640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 -- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cc73b0 msgr2=0x561a050d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cc73b0 0x561a050d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 -- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 msgr2=0x561a05096970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.362+0000 7fb1fd200640 1 -- 172.21.15.19:0/275320610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0514ca30 con 0x561a04cf0270 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cc73b0 0x561a050d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb1efe00640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb1fd200640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb1f005f520 tx=0x7fb1f0077040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3a4c804a01f654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb1e5600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb1e5600640 1 -- 172.21.15.19:0/275320610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f005f590 con 0x561a04cf0270 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb1e5600640 1 -- 172.21.15.19:0/275320610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb1f008cde0 con 0x561a04cf0270 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f0063940 con 0x561a04cf0270 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb1e5600640 1 -- 172.21.15.19:0/275320610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb1f0095030 con 0x561a04cf0270 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb206256740 10 client.4635 fetch_fsmap learned FSMap version 8 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.363+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x561a050c5210 con 0x561a04cf0270 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb1e5600640 1 -- 172.21.15.19:0/275320610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f0063940 con 0x561a04cf0270 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb1e5600640 1 -- 172.21.15.19:0/275320610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fb1f00951d0 con 0x561a04cf0270 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb206256740 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb206256740 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb206256740 20 client.4635 populate_metadata read hostname 'smithi019' 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb206256740 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a050f5c70 0x561a05094fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb206256740 4 client.4635 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:27:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 --> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] -- command(tid 0: {"prefix": "status"}) -- 0x561a04ffe080 con 0x561a050f5c70 2024-04-25T11:27:45.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.364+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a050f5c70 0x561a05094fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.365+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a050f5c70 0x561a05094fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e57e3d39efd6600 server_cookie=7a9db7e026b0ce6a in_seq=0 out_seq=0 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.365+0000 7fb1e5600640 10 client.4635 ms_handle_connect on v2:172.21.15.169:6834/2467347557 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.366+0000 7fb1e5600640 1 -- 172.21.15.19:0/275320610 <== mds.0 v2:172.21.15.169:6834/2467347557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x561a04ffe080 con 0x561a050f5c70 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.366+0000 7fb1e5600640 10 client.4635 handle_command_reply: tid=0 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.366+0000 7fb206256740 1 client.4635 shutdown 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.366+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 msgr2=0x561a05096eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.366+0000 7fb206256740 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb1f005f520 tx=0x7fb1f0077040 comp rx=0 tx=0).stop 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 shutdown_connections 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561a04ccf590 0x561a05096970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:6834/2467347557,v1:172.21.15.169:6836/2467347557] conn(0x561a050f5c70 0x561a05094fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a04cf0270 0x561a05096eb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 --2- 172.21.15.19:0/275320610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561a04cc73b0 0x561a050d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb1e5600640 0 client.4635 ms_handle_reset on v2:172.21.15.169:6834/2467347557 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 >> 172.21.15.19:0/275320610 conn(0x561a04ce43c0 msgr2=0x561a050b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 shutdown_connections 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 1 -- 172.21.15.19:0/275320610 wait complete. 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.367+0000 7fb206256740 20 client.4635 trim_cache size 0 max 16384 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 4297, 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:27:45.361 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:27:45.362 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 19.806199016000001, 2024-04-25T11:27:45.362 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 8, 2024-04-25T11:27:45.362 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 22, 2024-04-25T11:27:45.362 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 22, 2024-04-25T11:27:45.362 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 25.530241924999999 2024-04-25T11:27:45.362 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1d4074530 msgr2=0x7fb1d4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1d4074530 0x7fb1d4076a40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8069ed0 tx=0x7fb1e8009040 comp rx=0 tx=0).stop 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f81f7240 msgr2=0x7fb1f80b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f81f7240 0x7fb1f80b1f10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8072e90 tx=0x7fb1f000b4a0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 shutdown_connections 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1d4074530 0x7fb1d4076a40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1f81f7240 0x7fb1f80b1f10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1f813cf30 0x7fb1f81f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 --2- 172.21.15.19:0/3030394441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1dc004800 0x7fb1f813d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 >> 172.21.15.19:0/3030394441 conn(0x7fb1f80d1720 msgr2=0x7fb1f80684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:45.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 shutdown_connections 2024-04-25T11:27:45.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:45.368+0000 7fb1fdc00640 1 -- 172.21.15.19:0/3030394441 wait complete. 2024-04-25T11:27:45.387 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T11:27:45.398 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T11:27:45.398 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T11:27:45.398 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T11:27:45.398 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T11:27:45.434 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T11:27:45.434 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T11:27:45.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.434 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:27:45.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T11:27:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T11:27:45.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:27:45.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.596 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:27:45.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T11:27:45.743 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T11:27:45.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:27:45.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.749 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T11:27:45.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.806 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T11:27:45.888 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-25T11:27:45.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:27:45.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.891 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T11:27:45.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:45.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T11:27:46.027 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-25T11:27:46.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:27:46.031 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T11:27:46.031 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:27:46.031 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:27:46.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:27:46.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.334+0000 7f2b54000640 1 Processor -- start 2024-04-25T11:27:46.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.334+0000 7f2b54000640 1 -- start start 2024-04-25T11:27:46.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.336+0000 7f2b54000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 0x7f2b4c1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.336+0000 7f2b54000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2b4c160360 0x7f2b4c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.336+0000 7f2b54000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b4c06c7d0 con 0x7f2b4c0af9e0 2024-04-25T11:27:46.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.336+0000 7f2b54000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4c06d100 con 0x7f2b4c160360 2024-04-25T11:27:46.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.336+0000 7f2b54000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c06cab0 con 0x7f2b4c070680 2024-04-25T11:27:46.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.337+0000 7f2b52000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2b4c0af9e0 0x7f2b4c0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36206/0 (socket says 172.21.15.19:36206) 2024-04-25T11:27:46.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.337+0000 7f2b52000640 1 -- 172.21.15.19:0/1066686221 learned_addr learned my addr 172.21.15.19:0/1066686221 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:46.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.337+0000 7f2b51600640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 0x7f2b4c1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.337+0000 7f2b50c00640 1 --2- 172.21.15.19:0/1066686221 >> v2:172.21.15.169:3300/0 conn(0x7f2b4c160360 0x7f2b4c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.337+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984972303 0 0) 0x7f2b4c06c7d0 con 0x7f2b4c0af9e0 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.337+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b34003520 con 0x7f2b4c0af9e0 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 598311798 0 0) 0x7f2b34003520 con 0x7f2b4c0af9e0 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 msgr2=0x7f2b4c1647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 0x7f2b4c1647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 >> v2:172.21.15.169:3300/0 conn(0x7f2b4c160360 msgr2=0x7f2b4c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 --2- 172.21.15.19:0/1066686221 >> v2:172.21.15.169:3300/0 conn(0x7f2b4c160360 0x7f2b4c0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c06cd90 con 0x7f2b4c0af9e0 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2b44001670 con 0x7f2b4c0af9e0 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 >> v1:172.21.15.19:6789/0 conn(0x7f2b4c0af9e0 legacy=0x7f2b4c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:46.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 0x7f2b34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b34004d40 0x7f2b34005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c06d100 con 0x7f2b340039b0 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c06cab0 con 0x7f2b34004d40 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3400bb40 con 0x7f2b4c070680 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.338+0000 7f2b51600640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b50c00640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 0x7f2b34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b51600640 1 -- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 msgr2=0x7f2b34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b52000640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b34004d40 0x7f2b34005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b51600640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c070680 0x7f2b34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b51600640 1 -- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b34004d40 msgr2=0x7f2b34005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:46.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b51600640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b34004d40 0x7f2b34005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b51600640 1 -- 172.21.15.19:0/1066686221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b34003520 con 0x7f2b340039b0 2024-04-25T11:27:46.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.339+0000 7f2b52000640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b34004d40 0x7f2b34005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.340+0000 7f2b51600640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b34003e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009f90 tx=0x7f2b34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e511311804b9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.340+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40009c10 con 0x7f2b340039b0 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.340+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40036070 con 0x7f2b340039b0 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.340+0000 7f2b4be00640 1 -- 172.21.15.19:0/1066686221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40031810 con 0x7f2b340039b0 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.341+0000 7f2b54000640 1 -- 172.21.15.19:0/1066686221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 msgr2=0x7f2b34003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.341+0000 7f2b54000640 1 --2- 172.21.15.19:0/1066686221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b34003e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009f90 tx=0x7f2b34005830 comp rx=0 tx=0).stop 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.341+0000 7f2b52000640 1 -- 172.21.15.19:0/1066686221 reap_dead start 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.341+0000 7f2b54000640 1 -- 172.21.15.19:0/1066686221 shutdown_connections 2024-04-25T11:27:46.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.341+0000 7f2b54000640 1 -- 172.21.15.19:0/1066686221 >> 172.21.15.19:0/1066686221 conn(0x7f2b4c0d1ae0 msgr2=0x7f2b4c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:46.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.341+0000 7f2b54000640 1 -- 172.21.15.19:0/1066686221 shutdown_connections 2024-04-25T11:27:46.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.341+0000 7f2b54000640 1 -- 172.21.15.19:0/1066686221 wait complete. 2024-04-25T11:27:46.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.344+0000 7f2b54000640 1 Processor -- start 2024-04-25T11:27:46.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.344+0000 7f2b54000640 1 -- start start 2024-04-25T11:27:46.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.345+0000 7f2b54000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b4c14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.345+0000 7f2b54000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c154fd0 0x7f2b4c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.345+0000 7f2b54000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c1553e0 0x7f2b4c15f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.345+0000 7f2b54000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c0b6520 con 0x7f2b340039b0 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.345+0000 7f2b54000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c0b63a0 con 0x7f2b4c1553e0 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.345+0000 7f2b54000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c1647f0 con 0x7f2b4c154fd0 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.345+0000 7f2b51600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b4c14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b4c14b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49438/0 (socket says 172.21.15.19:49438) 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 -- 172.21.15.19:0/1979294464 learned_addr learned my addr 172.21.15.19:0/1979294464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b50c00640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c154fd0 0x7f2b4c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b52000640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c1553e0 0x7f2b4c15f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 -- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c154fd0 msgr2=0x7f2b4c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c154fd0 0x7f2b4c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 -- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c1553e0 msgr2=0x7f2b4c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c1553e0 0x7f2b4c15f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 -- 172.21.15.19:0/1979294464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c155dd0 con 0x7f2b340039b0 2024-04-25T11:27:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b50c00640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c154fd0 0x7f2b4c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.346+0000 7f2b51600640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b4c14b410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009f90 tx=0x7f2b40009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.347+0000 7f2b49600640 1 -- 172.21.15.19:0/1979294464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40009c10 con 0x7f2b340039b0 2024-04-25T11:27:46.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.347+0000 7f2b49600640 1 -- 172.21.15.19:0/1979294464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40045630 con 0x7f2b340039b0 2024-04-25T11:27:46.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.347+0000 7f2b49600640 1 -- 172.21.15.19:0/1979294464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40045a00 con 0x7f2b340039b0 2024-04-25T11:27:46.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.347+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c157e90 con 0x7f2b340039b0 2024-04-25T11:27:46.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.347+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c158450 con 0x7f2b340039b0 2024-04-25T11:27:46.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.348+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c050740 con 0x7f2b340039b0 2024-04-25T11:27:46.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.349+0000 7f2b49600640 1 -- 172.21.15.19:0/1979294464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b40005d10 con 0x7f2b340039b0 2024-04-25T11:27:46.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.350+0000 7f2b49600640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b2c0744b0 0x7f2b2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:46.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.350+0000 7f2b49600640 1 -- 172.21.15.19:0/1979294464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2b400b6c00 con 0x7f2b340039b0 2024-04-25T11:27:46.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.351+0000 7f2b50c00640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b2c0744b0 0x7f2b2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.352+0000 7f2b50c00640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b2c0744b0 0x7f2b2c0769c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b380090e0 tx=0x7f2b38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:46.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.355+0000 7f2b49600640 1 -- 172.21.15.19:0/1979294464 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b4002e050 con 0x7f2b340039b0 2024-04-25T11:27:46.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.604+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b4c06f620 con 0x7f2b340039b0 2024-04-25T11:27:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.605+0000 7f2b49600640 1 -- 172.21.15.19:0/1979294464 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f2b400812f0 con 0x7f2b340039b0 2024-04-25T11:27:46.596 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b2c0744b0 msgr2=0x7f2b2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b2c0744b0 0x7f2b2c0769c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b380090e0 tx=0x7f2b38008040 comp rx=0 tx=0).stop 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 msgr2=0x7f2b4c14b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b4c14b410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009f90 tx=0x7f2b40009750 comp rx=0 tx=0).stop 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 shutdown_connections 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b2c0744b0 0x7f2b2c0769c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4c1553e0 0x7f2b4c15f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b4c154fd0 0x7f2b4c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 --2- 172.21.15.19:0/1979294464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b340039b0 0x7f2b4c14b410 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.607+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 >> 172.21.15.19:0/1979294464 conn(0x7f2b4c0d1ae0 msgr2=0x7f2b4c0b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.608+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 shutdown_connections 2024-04-25T11:27:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46.608+0000 7f2b54000640 1 -- 172.21.15.19:0/1979294464 wait complete. 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:27:46.625 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:27:46.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:46.626 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:27:46.679 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:46.679 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:27:46.680 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:27:46.681 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42607sec preferred_lft 42607sec 2024-04-25T11:27:46.681 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:27:46.681 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:46.681 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:46.681 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:46.681 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:27:46.682 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:27:46.682 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:46.682 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:46.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:27:46.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-04-25T11:27:46.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-04-25T11:27:46.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:27:46.683 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:46.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:47.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:47.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:47.178 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:27:47.248 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:27:47.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:47.252 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:27:47.314 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:27:47.314 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:27:47.315 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:27:47.315 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:27:47.315 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:27:47.315 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:47.315 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:47.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:27:47.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:27:47.316 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:27:47.316 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:47.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:48.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:48.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:48.098 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:27:48.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:48.154 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T11:27:48.209 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:48.210 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:48.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:27:48.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T11:27:48.210 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:48.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:48.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:48.409 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T11:27:48.409 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:48.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:48.760 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:48.760 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:48.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:27:48.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:27:48.760 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:48.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:48.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:27:48.980 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:27:49.029 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:27:49.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:27:49.029 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:27:49.077 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:27:49.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:49.077 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:49.289 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:49.289 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:27:50.015 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:50.015 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:27:50.015 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:27:50.015 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:27:50.015 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:27:50.021 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:27:50.021 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:27:50.021 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:27:50.021 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:27:50.021 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:27:50.027 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:27:50.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:50.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:27:50.120 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:50.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:50.278 INFO:teuthology.orchestra.run.smithi019.stdout:client4427 2024-04-25T11:27:50.278 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:27:50.278 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:27:50.336 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:50.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:50.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:50.486 INFO:teuthology.orchestra.run.smithi019.stdout:client4427 2024-04-25T11:27:50.487 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:50.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:50.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:50.639 INFO:teuthology.orchestra.run.smithi019.stdout:client4427 2024-04-25T11:27:50.640 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23a56573-5965-4bbb-bd60-741a44babbed.client4427/status') 2024-04-25T11:27:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:50.807 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.4427 (3)172.21.15.19:0/1573143129 2024-04-25T11:27:50.807 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-25T11:27:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-25T11:27:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-25T11:27:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:70 bytes copied, 0.000137196 s, 510 kB/s 2024-04-25T11:27:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:50.821 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T11:27:50.822 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:27:50.822 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T11:27:50.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:27:51.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.103+0000 7f4dcfe00640 1 Processor -- start 2024-04-25T11:27:51.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.104+0000 7f4dcfe00640 1 -- start start 2024-04-25T11:27:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.105+0000 7f4dcfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4dd0140e00 0x7f4dd0143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.105+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dd006dce0 0x7f4dd0141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.106+0000 7f4dcfe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd0141210 con 0x7f4dd0143a90 2024-04-25T11:27:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.106+0000 7f4dcfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd0141b40 con 0x7f4dd0140e00 2024-04-25T11:27:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.106+0000 7f4dcfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd0143ea0 con 0x7f4dd006dce0 2024-04-25T11:27:51.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.106+0000 7f4dc7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4dd0143a90 0x7f4dd006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36210/0 (socket says 172.21.15.19:36210) 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.106+0000 7f4dc7e00640 1 -- 172.21.15.19:0/21249889 learned_addr learned my addr 172.21.15.19:0/21249889 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.106+0000 7f4dcf400640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dd006dce0 0x7f4dd0141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.106+0000 7f4dcea00640 1 --2- 172.21.15.19:0/21249889 >> v2:172.21.15.169:3300/0 conn(0x7f4dd0140e00 0x7f4dd0143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1680443589 0 0) 0x7f4dd0141210 con 0x7f4dd0143a90 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4db4003520 con 0x7f4dd0143a90 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 260952769 0 0) 0x7f4db4003520 con 0x7f4dd0143a90 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dd006dce0 msgr2=0x7f4dd0141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4dd006dce0 0x7f4dd0141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 >> v2:172.21.15.169:3300/0 conn(0x7f4dd0140e00 msgr2=0x7f4dd0143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 --2- 172.21.15.19:0/21249889 >> v2:172.21.15.169:3300/0 conn(0x7f4dd0140e00 0x7f4dd0143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd0141510 con 0x7f4dd0143a90 2024-04-25T11:27:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4db8001670 con 0x7f4dd0143a90 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 >> v1:172.21.15.19:6789/0 conn(0x7f4dd0143a90 legacy=0x7f4dd006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4db40039b0 0x7f4db4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4004800 0x7f4db4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4005150 0x7f4db400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0141b40 con 0x7f4db4005150 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0143ea0 con 0x7f4db40039b0 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.107+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4db400bf00 con 0x7f4db4004800 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcf400640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4005150 0x7f4db400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dc7e00640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4004800 0x7f4db4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcea00640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4db40039b0 0x7f4db4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcf400640 1 -- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4004800 msgr2=0x7f4db4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcf400640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4004800 0x7f4db4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcf400640 1 -- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4db40039b0 msgr2=0x7f4db4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcf400640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4db40039b0 0x7f4db4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcf400640 1 -- 172.21.15.19:0/21249889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db4003520 con 0x7f4db4005150 2024-04-25T11:27:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.108+0000 7f4dcea00640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4db40039b0 0x7f4db4003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.109+0000 7f4dcf400640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4005150 0x7f4db400b9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009f90 tx=0x7f4db4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d784250eebdb32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8009c10 con 0x7f4db4005150 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc8036070 con 0x7f4db4005150 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dce000640 1 -- 172.21.15.19:0/21249889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8031850 con 0x7f4db4005150 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dcfe00640 1 -- 172.21.15.19:0/21249889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4005150 msgr2=0x7f4db400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dcfe00640 1 --2- 172.21.15.19:0/21249889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4db4005150 0x7f4db400b9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009f90 tx=0x7f4db4005830 comp rx=0 tx=0).stop 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dcf400640 1 -- 172.21.15.19:0/21249889 reap_dead start 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dcfe00640 1 -- 172.21.15.19:0/21249889 shutdown_connections 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dcfe00640 1 -- 172.21.15.19:0/21249889 >> 172.21.15.19:0/21249889 conn(0x7f4dd0072ee0 msgr2=0x7f4dd00732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:51.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.110+0000 7f4dcfe00640 1 -- 172.21.15.19:0/21249889 shutdown_connections 2024-04-25T11:27:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.111+0000 7f4dcfe00640 1 -- 172.21.15.19:0/21249889 wait complete. 2024-04-25T11:27:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.113+0000 7f4dcfe00640 1 Processor -- start 2024-04-25T11:27:51.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.113+0000 7f4dcfe00640 1 -- start start 2024-04-25T11:27:51.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4005150 0x7f4dd00cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd013be40 0x7f4dd00b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd00b2ba0 0x7f4dd01555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcfe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01417d0 con 0x7f4dd013be40 2024-04-25T11:27:51.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01443d0 con 0x7f4dd00b2ba0 2024-04-25T11:27:51.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd00d1490 con 0x7f4db4005150 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd013be40 0x7f4dd00b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd013be40 0x7f4dd00b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49446/0 (socket says 172.21.15.19:49446) 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 -- 172.21.15.19:0/3534713390 learned_addr learned my addr 172.21.15.19:0/3534713390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcea00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4005150 0x7f4dd00cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcf400640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd00b2ba0 0x7f4dd01555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 -- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4005150 msgr2=0x7f4dd00cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4005150 0x7f4dd00cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 -- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd00b2ba0 msgr2=0x7f4dd01555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd00b2ba0 0x7f4dd01555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dc7e00640 1 -- 172.21.15.19:0/3534713390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd00b2fb0 con 0x7f4dd013be40 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.115+0000 7f4dcea00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4005150 0x7f4dd00cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.116+0000 7f4dc7e00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd013be40 0x7f4dd00b2660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4db800e780 tx=0x7f4db800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.116+0000 7f4dc6a00640 1 -- 172.21.15.19:0/3534713390 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db800c870 con 0x7f4dd013be40 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.116+0000 7f4dc6a00640 1 -- 172.21.15.19:0/3534713390 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db8018070 con 0x7f4dd013be40 2024-04-25T11:27:51.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.116+0000 7f4dc6a00640 1 -- 172.21.15.19:0/3534713390 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db800cc30 con 0x7f4dd013be40 2024-04-25T11:27:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.116+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd013d0c0 con 0x7f4dd013be40 2024-04-25T11:27:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.116+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd013d680 con 0x7f4dd013be40 2024-04-25T11:27:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.117+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd00cd590 con 0x7f4dd013be40 2024-04-25T11:27:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.119+0000 7f4dc6a00640 1 -- 172.21.15.19:0/3534713390 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4db801d460 con 0x7f4dd013be40 2024-04-25T11:27:51.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.120+0000 7f4dc6a00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dac0744b0 0x7f4dac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:51.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.120+0000 7f4dc6a00640 1 -- 172.21.15.19:0/3534713390 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4db80951c0 con 0x7f4dd013be40 2024-04-25T11:27:51.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.120+0000 7f4dcea00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dac0744b0 0x7f4dac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:51.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.121+0000 7f4dcea00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dac0744b0 0x7f4dac0769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009b20 tx=0x7f4dc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:51.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.125+0000 7f4dc6a00640 1 -- 172.21.15.19:0/3534713390 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dd00b2fb0 con 0x7f4dd013be40 2024-04-25T11:27:51.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.370+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4dd0050740 con 0x7f4dd013be40 2024-04-25T11:27:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.371+0000 7f4dc6a00640 1 -- 172.21.15.19:0/3534713390 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f4db805f8b0 con 0x7f4dd013be40 2024-04-25T11:27:51.362 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:27:51.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dac0744b0 msgr2=0x7f4dac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:51.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dac0744b0 0x7f4dac0769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009b20 tx=0x7f4dc0008040 comp rx=0 tx=0).stop 2024-04-25T11:27:51.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd013be40 msgr2=0x7f4dd00b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd013be40 0x7f4dd00b2660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4db800e780 tx=0x7f4db800ec40 comp rx=0 tx=0).stop 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 shutdown_connections 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dac0744b0 0x7f4dac0769c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dd00b2ba0 0x7f4dd01555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dd013be40 0x7f4dd00b2660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 --2- 172.21.15.19:0/3534713390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4db4005150 0x7f4dd00cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 >> 172.21.15.19:0/3534713390 conn(0x7f4dd0072ee0 msgr2=0x7f4dd00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 shutdown_connections 2024-04-25T11:27:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51.374+0000 7f4dcfe00640 1 -- 172.21.15.19:0/3534713390 wait complete. 2024-04-25T11:27:51.394 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:27:51.395 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T11:27:51.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:51.395 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:27:51.450 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:51.450 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:51.450 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:51.450 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:51.450 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42602sec preferred_lft 42602sec 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:27:51.451 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:27:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:51.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:51.453 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:27:51.535 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:27:51.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:51.539 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:27:51.593 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:27:51.593 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:27:51.593 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:27:51.593 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:27:51.593 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:27:51.594 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:51.594 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:51.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:27:51.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:27:51.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:27:51.595 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:51.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:51.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:51.822 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:27:51.872 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:27:51.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:51.873 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T11:27:51.926 INFO:teuthology.orchestra.run.smithi019.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T11:27:51.927 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:51.927 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:51.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:27:51.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T11:27:51.927 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:52.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:52.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:52.121 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 52:50:6f:a7:53:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:27:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5050:6fff:fea7:537f/64 scope link 2024-04-25T11:27:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:52.207 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T11:27:52.294 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.295 INFO:teuthology.orchestra.run.smithi019.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:27:52.295 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 52:50:6f:a7:53:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:27:52.295 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:27:52.295 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.295 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::5050:6fff:fea7:537f/64 scope link 2024-04-25T11:27:52.295 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:52.299 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T11:27:52.299 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:52.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:52.591 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:52.591 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:27:52.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.1 up 2024-04-25T11:27:52.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T11:27:52.591 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:27:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:52.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:27:52.808 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T11:27:52.850 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T11:27:52.851 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:27:52.851 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T11:27:52.948 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T11:27:52.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:52.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:53.165 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:53.166 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:27:53.212 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:53.212 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:27:53.212 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:27:53.212 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 2@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:27:53.213 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:27:53.215 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T11:27:53.215 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:27:53.215 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:27:53.215 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:27:53.215 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:27:53.220 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:27:53.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:53.220 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T11:27:53.299 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:53.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:53.421 INFO:teuthology.orchestra.run.smithi019.stdout:client4558 2024-04-25T11:27:53.422 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:27:53.422 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:27:53.472 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:53.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:client4558 2024-04-25T11:27:53.643 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:53.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:53.771 INFO:teuthology.orchestra.run.smithi019.stdout:client4558 2024-04-25T11:27:53.772 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23a56573-5965-4bbb-bd60-741a44babbed.client4558/status') 2024-04-25T11:27:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:53.909 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.4558 (3)172.21.15.19:0/2808103855 2024-04-25T11:27:53.909 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-25T11:27:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-25T11:27:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-25T11:27:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:70 bytes copied, 9.9307e-05 s, 705 kB/s 2024-04-25T11:27:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:53.922 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T11:27:53.922 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:27:53.922 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:27:53.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:27:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.214+0000 7f808cc00640 1 Processor -- start 2024-04-25T11:27:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.214+0000 7f808cc00640 1 -- start start 2024-04-25T11:27:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f808cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f80880d1210 0x7f80880d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f808cc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80880d1850 con 0x7f808806ccd0 2024-04-25T11:27:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f808cc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8088160de0 con 0x7f80880d0e00 2024-04-25T11:27:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f808cc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f80880d1620 con 0x7f80880d1210 2024-04-25T11:27:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f8086800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f80880d1210 0x7f80880d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f8086800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f80880d1210 0x7f80880d5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58880/0 (socket says 172.21.15.19:58880) 2024-04-25T11:27:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f8086800640 1 -- 172.21.15.19:0/2995393489 learned_addr learned my addr 172.21.15.19:0/2995393489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.217+0000 7f807d400640 1 --1- 172.21.15.19:0/2995393489 >> v1:172.21.15.19:6789/0 conn(0x7f808806ccd0 0x7f80880d4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60804/0 (socket says 172.21.15.19:60804) 2024-04-25T11:27:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.218+0000 7f8086800640 1 -- 172.21.15.19:0/2995393489 >> v1:172.21.15.169:6789/0 conn(0x7f80880d0e00 legacy=0x7f808806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.218+0000 7f8086800640 1 -- 172.21.15.19:0/2995393489 >> v1:172.21.15.19:6789/0 conn(0x7f808806ccd0 legacy=0x7f80880d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.218+0000 7f8086800640 1 -- 172.21.15.19:0/2995393489 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80881610c0 con 0x7f80880d1210 2024-04-25T11:27:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.219+0000 7f8086800640 1 --2- 172.21.15.19:0/2995393489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80880d1210 0x7f80880d5160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8080009e50 tx=0x7f808002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279003722167916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.219+0000 7f8085400640 1 -- 172.21.15.19:0/2995393489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080036070 con 0x7f80880d1210 2024-04-25T11:27:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.219+0000 7f8085400640 1 -- 172.21.15.19:0/2995393489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080031600 con 0x7f80880d1210 2024-04-25T11:27:54.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.219+0000 7f8085400640 1 -- 172.21.15.19:0/2995393489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800319d0 con 0x7f80880d1210 2024-04-25T11:27:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.220+0000 7f808cc00640 1 -- 172.21.15.19:0/2995393489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80880d1210 msgr2=0x7f80880d5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.220+0000 7f808cc00640 1 --2- 172.21.15.19:0/2995393489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80880d1210 0x7f80880d5160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8080009e50 tx=0x7f808002a730 comp rx=0 tx=0).stop 2024-04-25T11:27:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.220+0000 7f808cc00640 1 -- 172.21.15.19:0/2995393489 shutdown_connections 2024-04-25T11:27:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.220+0000 7f808cc00640 1 --2- 172.21.15.19:0/2995393489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80880d1210 0x7f80880d5160 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.220+0000 7f808cc00640 1 -- 172.21.15.19:0/2995393489 >> 172.21.15.19:0/2995393489 conn(0x7f80880d5d90 msgr2=0x7f80880d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.220+0000 7f808cc00640 1 -- 172.21.15.19:0/2995393489 shutdown_connections 2024-04-25T11:27:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.220+0000 7f808cc00640 1 -- 172.21.15.19:0/2995393489 wait complete. 2024-04-25T11:27:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.223+0000 7f808cc00640 1 Processor -- start 2024-04-25T11:27:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.223+0000 7f808cc00640 1 -- start start 2024-04-25T11:27:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.224+0000 7f808cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80880d0e00 0x7f8088156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.224+0000 7f808cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80880d1210 0x7f8088155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.224+0000 7f808cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80881fab00 0x7f80880caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.224+0000 7f808cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80880b5320 con 0x7f80881fab00 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.224+0000 7f808cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8088160ad0 con 0x7f80880d0e00 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.224+0000 7f808cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8088164e00 con 0x7f80880d1210 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80881fab00 0x7f80880caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80881fab00 0x7f80880caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58892/0 (socket says 172.21.15.19:58892) 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 -- 172.21.15.19:0/2292465564 learned_addr learned my addr 172.21.15.19:0/2292465564 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8085e00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80880d0e00 0x7f8088156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f807d400640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80880d1210 0x7f8088155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 -- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80880d1210 msgr2=0x7f8088155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80880d1210 0x7f8088155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 -- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80880d0e00 msgr2=0x7f8088156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80880d0e00 0x7f8088156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.225+0000 7f8086800640 1 -- 172.21.15.19:0/2292465564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80880c9960 con 0x7f80881fab00 2024-04-25T11:27:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.226+0000 7f8086800640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80881fab00 0x7f80880caa00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8080034040 tx=0x7f8080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.226+0000 7f807ea00640 1 -- 172.21.15.19:0/2292465564 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080036050 con 0x7f80881fab00 2024-04-25T11:27:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.226+0000 7f807ea00640 1 -- 172.21.15.19:0/2292465564 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080042070 con 0x7f80881fab00 2024-04-25T11:27:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.226+0000 7f807ea00640 1 -- 172.21.15.19:0/2292465564 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080009510 con 0x7f80881fab00 2024-04-25T11:27:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.226+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80880c9c40 con 0x7f80881fab00 2024-04-25T11:27:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.226+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80880ca200 con 0x7f80881fab00 2024-04-25T11:27:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.227+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80880d0150 con 0x7f80881fab00 2024-04-25T11:27:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.229+0000 7f807ea00640 1 -- 172.21.15.19:0/2292465564 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8080007bf0 con 0x7f80881fab00 2024-04-25T11:27:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.230+0000 7f807ea00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80640744b0 0x7f80640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.230+0000 7f807ea00640 1 -- 172.21.15.19:0/2292465564 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f80800b6120 con 0x7f80881fab00 2024-04-25T11:27:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.230+0000 7f8085e00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80640744b0 0x7f80640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.231+0000 7f8085e00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80640744b0 0x7f80640769c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f60 tx=0x7f8078009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.235+0000 7f807ea00640 1 -- 172.21.15.19:0/2292465564 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80880d0150 con 0x7f80881fab00 2024-04-25T11:27:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.482+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80880cf1a0 con 0x7f80881fab00 2024-04-25T11:27:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.483+0000 7f807ea00640 1 -- 172.21.15.19:0/2292465564 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f80800808a0 con 0x7f80881fab00 2024-04-25T11:27:54.474 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:27:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.485+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80640744b0 msgr2=0x7f80640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.485+0000 7f808cc00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80640744b0 0x7f80640769c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8078000f60 tx=0x7f8078009040 comp rx=0 tx=0).stop 2024-04-25T11:27:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80881fab00 msgr2=0x7f80880caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80881fab00 0x7f80880caa00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8080034040 tx=0x7f8080004a40 comp rx=0 tx=0).stop 2024-04-25T11:27:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 shutdown_connections 2024-04-25T11:27:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80640744b0 0x7f80640769c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80881fab00 0x7f80880caa00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80880d1210 0x7f8088155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 --2- 172.21.15.19:0/2292465564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80880d0e00 0x7f8088156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 >> 172.21.15.19:0/2292465564 conn(0x7f80880d5d90 msgr2=0x7f808815c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 shutdown_connections 2024-04-25T11:27:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:54.486+0000 7f808cc00640 1 -- 172.21.15.19:0/2292465564 wait complete. 2024-04-25T11:27:54.504 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:27:54.504 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:27:54.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-04-25T11:27:54.505 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:27:54.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:27:54.505 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:27:54.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:27:54.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:27:54.505 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:27:54.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:54.506 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-04-25T11:27:54.561 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:54.561 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:54.562 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:27:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:27:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42573sec preferred_lft 42573sec 2024-04-25T11:27:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-04-25T11:27:54.563 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:54.564 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:27:54.564 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:54.564 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:54.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:27:54.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2024-04-25T11:27:54.565 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2024-04-25T11:27:54.565 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:27:54.565 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:27:54.673 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:55.142 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:55.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:55.152 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:27:55.222 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-25T11:27:55.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:55.226 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-04-25T11:27:55.284 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-04-25T11:27:55.284 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:27:55.285 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:27:55.285 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:27:55.285 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:27:55.285 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:55.285 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:55.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:27:55.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:27:55.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:27:55.285 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:27:55.383 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:56.017 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:56.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:56.025 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T11:27:56.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:56.073 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-04-25T11:27:56.114 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:56.114 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:56.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:27:56.115 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T11:27:56.115 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:27:56.256 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:56.319 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:56.328 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T11:27:56.328 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:56.328 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:56.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T11:27:56.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:27:56.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T11:27:56.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T11:27:56.328 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T11:27:56.329 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:27:56.422 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:56.617 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:56.626 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:56.626 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:56.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2024-04-25T11:27:56.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:27:56.627 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:27:56.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:56.803 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:56.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:27:56.812 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:27:56.851 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:27:56.851 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:27:56.852 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:27:56.937 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:27:56.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:56.938 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:57.165 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:57.165 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:27:57.894 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:27:57.894 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:27:57.894 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:27:57.894 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with new device syntax: 1@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:27:57.894 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:27:57.897 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:27:57.897 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:27:57.897 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:27:57.897 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:27:57.897 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:27:57.902 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:27:57.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:57.902 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:27:57.983 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:58.091 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:58.097 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:58.106 INFO:teuthology.orchestra.run.smithi169.stdout:client4433 2024-04-25T11:27:58.107 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:27:58.107 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:27:58.159 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:58.264 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:58.269 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:58.278 INFO:teuthology.orchestra.run.smithi169.stdout:client4433 2024-04-25T11:27:58.278 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:27:58.384 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:27:58.389 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:58.398 INFO:teuthology.orchestra.run.smithi169.stdout:client4433 2024-04-25T11:27:58.399 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23a56573-5965-4bbb-bd60-741a44babbed.client4433/status') 2024-04-25T11:27:58.499 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:58.528 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4433 (3)192.168.0.1:0/3313251844 2024-04-25T11:27:58.528 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2024-04-25T11:27:58.528 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2024-04-25T11:27:58.528 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2024-04-25T11:27:58.528 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 0.000103338 s, 668 kB/s 2024-04-25T11:27:58.531 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:58.541 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T11:27:58.541 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:27:58.541 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T11:27:58.541 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:27:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.847+0000 7f5018c00640 1 Processor -- start 2024-04-25T11:27:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.847+0000 7f5018c00640 1 -- start start 2024-04-25T11:27:58.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5018c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5018c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f501406dce0 0x7f5014141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5018c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5014141210 con 0x7f5014143a90 2024-04-25T11:27:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5018c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5014141b40 con 0x7f501406dce0 2024-04-25T11:27:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5018c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5014143ea0 con 0x7f5014140e00 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5013e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f501406dce0 0x7f5014141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5013e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f501406dce0 0x7f5014141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58904/0 (socket says 172.21.15.19:58904) 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5013e00640 1 -- 172.21.15.19:0/2579698948 learned_addr learned my addr 172.21.15.19:0/2579698948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.850+0000 7f5013400640 1 --2- 172.21.15.19:0/2579698948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.851+0000 7f5013e00640 1 -- 172.21.15.19:0/2579698948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 msgr2=0x7f5014143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.851+0000 7f5013e00640 1 --2- 172.21.15.19:0/2579698948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.851+0000 7f5013e00640 1 -- 172.21.15.19:0/2579698948 >> v1:172.21.15.169:6789/0 conn(0x7f5014143a90 legacy=0x7f501406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:27:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.851+0000 7f5013e00640 1 -- 172.21.15.19:0/2579698948 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5014141510 con 0x7f501406dce0 2024-04-25T11:27:58.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.852+0000 7f5013e00640 1 --2- 172.21.15.19:0/2579698948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501406dce0 0x7f5014141fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50140b73a0 tx=0x7f500c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585045739abf8f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:58.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.852+0000 7f5012000640 1 -- 172.21.15.19:0/2579698948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c037070 con 0x7f501406dce0 2024-04-25T11:27:58.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.852+0000 7f5012000640 1 -- 172.21.15.19:0/2579698948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c032440 con 0x7f501406dce0 2024-04-25T11:27:58.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.852+0000 7f5012000640 1 -- 172.21.15.19:0/2579698948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c032810 con 0x7f501406dce0 2024-04-25T11:27:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 -- 172.21.15.19:0/2579698948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501406dce0 msgr2=0x7f5014141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 --2- 172.21.15.19:0/2579698948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501406dce0 0x7f5014141fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50140b73a0 tx=0x7f500c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:27:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 -- 172.21.15.19:0/2579698948 shutdown_connections 2024-04-25T11:27:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 --2- 172.21.15.19:0/2579698948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f501406dce0 0x7f5014141fd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 --2- 172.21.15.19:0/2579698948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 -- 172.21.15.19:0/2579698948 >> 172.21.15.19:0/2579698948 conn(0x7f5014072ee0 msgr2=0x7f50140732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 -- 172.21.15.19:0/2579698948 shutdown_connections 2024-04-25T11:27:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.853+0000 7f5018c00640 1 -- 172.21.15.19:0/2579698948 wait complete. 2024-04-25T11:27:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.855+0000 7f5018c00640 1 Processor -- start 2024-04-25T11:27:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.856+0000 7f5018c00640 1 -- start start 2024-04-25T11:27:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.857+0000 7f5018c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.857+0000 7f5018c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5014143a90 0x7f50141f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.857+0000 7f5018c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50140b6680 0x7f50140b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.857+0000 7f5018c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50141417d0 con 0x7f5014143a90 2024-04-25T11:27:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.857+0000 7f5018c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50141443d0 con 0x7f50140b6680 2024-04-25T11:27:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.857+0000 7f5018c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50140d1490 con 0x7f5014140e00 2024-04-25T11:27:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5014143a90 0x7f50141f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5014143a90 0x7f50141f5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58906/0 (socket says 172.21.15.19:58906) 2024-04-25T11:27:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 -- 172.21.15.19:0/1469814596 learned_addr learned my addr 172.21.15.19:0/1469814596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:27:58.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5013400640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:58.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5013e00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50140b6680 0x7f50140b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 -- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 msgr2=0x7f5014157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 -- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50140b6680 msgr2=0x7f50140b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50140b6680 0x7f50140b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.858+0000 7f5012a00640 1 -- 172.21.15.19:0/1469814596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50140cd280 con 0x7f5014143a90 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.859+0000 7f5012a00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5014143a90 0x7f50141f5a90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f500400ba80 tx=0x7f500400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.859+0000 7f500b400640 1 -- 172.21.15.19:0/1469814596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500400cc00 con 0x7f5014143a90 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.859+0000 7f500b400640 1 -- 172.21.15.19:0/1469814596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500400cda0 con 0x7f5014143a90 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.859+0000 7f500b400640 1 -- 172.21.15.19:0/1469814596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004003bb0 con 0x7f5014143a90 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.859+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50141570e0 con 0x7f5014143a90 2024-04-25T11:27:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.859+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50140b52f0 con 0x7f5014143a90 2024-04-25T11:27:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.860+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5014050740 con 0x7f5014143a90 2024-04-25T11:27:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.862+0000 7f500b400640 1 -- 172.21.15.19:0/1469814596 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5004014440 con 0x7f5014143a90 2024-04-25T11:27:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.862+0000 7f500b400640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ff00744b0 0x7f4ff00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:27:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.863+0000 7f500b400640 1 -- 172.21.15.19:0/1469814596 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f50040951c0 con 0x7f5014143a90 2024-04-25T11:27:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.863+0000 7f5013400640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ff00744b0 0x7f4ff00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:27:58.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.865+0000 7f5013400640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ff00744b0 0x7f4ff00769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc006fd0 tx=0x7f4ffc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:27:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:58.870+0000 7f500b400640 1 -- 172.21.15.19:0/1469814596 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f500405f940 con 0x7f5014143a90 2024-04-25T11:27:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.114+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f50140b2740 con 0x7f5014143a90 2024-04-25T11:27:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.115+0000 7f500b400640 1 -- 172.21.15.19:0/1469814596 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f500405f240 con 0x7f5014143a90 2024-04-25T11:27:59.106 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:27:59.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ff00744b0 msgr2=0x7f4ff00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ff00744b0 0x7f4ff00769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc006fd0 tx=0x7f4ffc008040 comp rx=0 tx=0).stop 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5014143a90 msgr2=0x7f50141f5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5014143a90 0x7f50141f5a90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f500400ba80 tx=0x7f500400bf40 comp rx=0 tx=0).stop 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 shutdown_connections 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ff00744b0 0x7f4ff00769c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50140b6680 0x7f50140b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5014143a90 0x7f50141f5a90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 --2- 172.21.15.19:0/1469814596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5014140e00 0x7f5014157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.117+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 >> 172.21.15.19:0/1469814596 conn(0x7f5014072ee0 msgr2=0x7f50140706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.118+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 shutdown_connections 2024-04-25T11:27:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:27:59.118+0000 7f5018c00640 1 -- 172.21.15.19:0/1469814596 wait complete. 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:27:59.134 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T11:27:59.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:59.135 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-04-25T11:27:59.193 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:59.193 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 42568sec preferred_lft 42568sec 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.194 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 06:58:32:de:08:ba brd ff:ff:ff:ff:ff:ff 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::1801:29ff:fef1:5259/64 scope link 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 06:58:32:de:08:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::458:32ff:fede:8ba/64 scope link 2024-04-25T11:27:59.195 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:59.196 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:27:59.271 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-04-25T11:27:59.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:59.275 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-04-25T11:27:59.324 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-04-25T11:27:59.325 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:27:59.325 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:27:59.325 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:27:59.325 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:27:59.325 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:59.325 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:59.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:27:59.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:27:59.325 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:27:59.326 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:27:59.428 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:59.519 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:59.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:59.528 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T11:27:59.575 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:27:59.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:59.576 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-04-25T11:27:59.627 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T11:27:59.628 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:59.628 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:59.628 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:27:59.628 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T11:27:59.628 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:27:59.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:27:59.801 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:27:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:27:59.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:59.811 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T11:27:59.883 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:59.883 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether be:35:b0:e6:d1:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::bc35:b0ff:fee6:d121/64 scope link 2024-04-25T11:27:59.884 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:27:59.887 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T11:27:59.961 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:27:59.961 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether be:35:b0:e6:d1:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::bc35:b0ff:fee6:d121/64 scope link 2024-04-25T11:27:59.962 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:27:59.965 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T11:27:59.965 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:27:59.965 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:27:59.965 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T11:27:59.965 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T11:27:59.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T11:27:59.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T11:27:59.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T11:27:59.966 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:28:00.069 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:28:00.232 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:00.241 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:28:00.242 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-04-25T11:28:00.242 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2024-04-25T11:28:00.242 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T11:28:00.242 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-04-25T11:28:00.339 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:28:00.437 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:00.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:28:00.446 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T11:28:00.488 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T11:28:00.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:28:00.489 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T11:28:00.583 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T11:28:00.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:00.583 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:28:00.785 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:28:00.785 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:28:00.828 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:28:00.828 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:28:00.828 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:28:00.828 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with new device syntax: 3@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:28:00.829 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:28:00.830 INFO:teuthology.orchestra.run.smithi169.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T11:28:00.830 INFO:teuthology.orchestra.run.smithi169.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:28:00.830 INFO:teuthology.orchestra.run.smithi169.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:28:00.830 INFO:teuthology.orchestra.run.smithi169.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:28:00.830 INFO:teuthology.orchestra.run.smithi169.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:28:00.834 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:28:00.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:00.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T11:28:00.908 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:28:01.009 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:28:01.014 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:01.023 INFO:teuthology.orchestra.run.smithi169.stdout:client4665 2024-04-25T11:28:01.024 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:28:01.024 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:01.075 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:28:01.179 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:28:01.184 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:01.194 INFO:teuthology.orchestra.run.smithi169.stdout:client4665 2024-04-25T11:28:01.194 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:28:01.296 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:28:01.301 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:01.310 INFO:teuthology.orchestra.run.smithi169.stdout:client4665 2024-04-25T11:28:01.310 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23a56573-5965-4bbb-bd60-741a44babbed.client4665/status') 2024-04-25T11:28:01.408 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:28:01.437 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4665 (3)172.21.15.169:0/2459695981 2024-04-25T11:28:01.438 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2024-04-25T11:28:01.438 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2024-04-25T11:28:01.438 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2024-04-25T11:28:01.438 INFO:teuthology.orchestra.run.smithi169.stderr:71 bytes copied, 9.6595e-05 s, 735 kB/s 2024-04-25T11:28:01.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:28:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:01.451 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T11:28:01.477 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:01.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.778+0000 7f9c7d000640 1 Processor -- start 2024-04-25T11:28:01.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.778+0000 7f9c7d000640 1 -- start start 2024-04-25T11:28:01.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.780+0000 7f9c7d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c78162930 0x7f9c780d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.780+0000 7f9c7d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c780d3e20 0x7f9c780ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.780+0000 7f9c7d000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c780d2a90 con 0x7f9c780b3680 2024-04-25T11:28:01.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.780+0000 7f9c7d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9c78161ac0 con 0x7f9c780d3e20 2024-04-25T11:28:01.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.780+0000 7f9c7d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c780d27b0 con 0x7f9c78162930 2024-04-25T11:28:01.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.781+0000 7f9c75400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9c780b3680 0x7f9c780d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60820/0 (socket says 172.21.15.19:60820) 2024-04-25T11:28:01.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.781+0000 7f9c75400640 1 -- 172.21.15.19:0/4146708512 learned_addr learned my addr 172.21.15.19:0/4146708512 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.781+0000 7f9c76800640 1 --2- 172.21.15.19:0/4146708512 >> v2:172.21.15.169:3300/0 conn(0x7f9c780d3e20 0x7f9c780ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.781+0000 7f9c75e00640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c78162930 0x7f9c780d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.781+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4292602832 0 0) 0x7f9c780d2a90 con 0x7f9c780b3680 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.781+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c5c003520 con 0x7f9c780b3680 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3046395851 0 0) 0x7f9c5c003520 con 0x7f9c780b3680 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c78162930 msgr2=0x7f9c780d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c78162930 0x7f9c780d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 >> v2:172.21.15.169:3300/0 conn(0x7f9c780d3e20 msgr2=0x7f9c780ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 --2- 172.21.15.19:0/4146708512 >> v2:172.21.15.169:3300/0 conn(0x7f9c780d3e20 0x7f9c780ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c78161e50 con 0x7f9c780b3680 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9c64001670 con 0x7f9c780b3680 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 >> v1:172.21.15.19:6789/0 conn(0x7f9c780b3680 legacy=0x7f9c780d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0039b0 0x7f9c5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c004340 0x7f9c5c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004f70 0x7f9c5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78161ac0 con 0x7f9c5c004340 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780d27b0 con 0x7f9c5c004f70 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c00bf50 con 0x7f9c5c0039b0 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.782+0000 7f9c75400640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c004340 0x7f9c5c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.783+0000 7f9c75e00640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0039b0 0x7f9c5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.783+0000 7f9c76800640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004f70 0x7f9c5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.783+0000 7f9c75400640 1 -- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0039b0 msgr2=0x7f9c5c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.783+0000 7f9c75400640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0039b0 0x7f9c5c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:01.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.783+0000 7f9c75400640 1 -- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004f70 msgr2=0x7f9c5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:01.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.783+0000 7f9c75400640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004f70 0x7f9c5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:01.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.783+0000 7f9c75400640 1 -- 172.21.15.19:0/4146708512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c003520 con 0x7f9c5c004340 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.784+0000 7f9c75400640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c004340 0x7f9c5c004860 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c240 tx=0x7f9c5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6febb94af9f6abba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.784+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64007d10 con 0x7f9c5c004340 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.784+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c64005ad0 con 0x7f9c5c004340 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.784+0000 7f9c74a00640 1 -- 172.21.15.19:0/4146708512 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64007070 con 0x7f9c5c004340 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.785+0000 7f9c7d000640 1 -- 172.21.15.19:0/4146708512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c004340 msgr2=0x7f9c5c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.785+0000 7f9c7d000640 1 --2- 172.21.15.19:0/4146708512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c5c004340 0x7f9c5c004860 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c240 tx=0x7f9c5c005830 comp rx=0 tx=0).stop 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.785+0000 7f9c76800640 1 -- 172.21.15.19:0/4146708512 reap_dead start 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.785+0000 7f9c7d000640 1 -- 172.21.15.19:0/4146708512 shutdown_connections 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.785+0000 7f9c7d000640 1 -- 172.21.15.19:0/4146708512 >> 172.21.15.19:0/4146708512 conn(0x7f9c7808d110 msgr2=0x7f9c7808d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.785+0000 7f9c7d000640 1 -- 172.21.15.19:0/4146708512 shutdown_connections 2024-04-25T11:28:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.785+0000 7f9c7d000640 1 -- 172.21.15.19:0/4146708512 wait complete. 2024-04-25T11:28:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.788+0000 7f9c7d000640 1 Processor -- start 2024-04-25T11:28:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.788+0000 7f9c7d000640 1 -- start start 2024-04-25T11:28:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.789+0000 7f9c7d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004340 0x7f9c780b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.789+0000 7f9c7d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c780b7340 0x7f9c781560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.789+0000 7f9c7d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c7815b420 0x7f9c7815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.789+0000 7f9c7d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780b3a90 con 0x7f9c780b7340 2024-04-25T11:28:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.789+0000 7f9c7d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78162410 con 0x7f9c5c004340 2024-04-25T11:28:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.789+0000 7f9c7d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c780b93f0 con 0x7f9c7815b420 2024-04-25T11:28:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c780b7340 0x7f9c781560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c780b7340 0x7f9c781560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58920/0 (socket says 172.21.15.19:58920) 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 -- 172.21.15.19:0/1489205289 learned_addr learned my addr 172.21.15.19:0/1489205289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75e00640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004340 0x7f9c780b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c76800640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c7815b420 0x7f9c7815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 -- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c7815b420 msgr2=0x7f9c7815b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c7815b420 0x7f9c7815b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 -- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004340 msgr2=0x7f9c780b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004340 0x7f9c780b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75400640 1 -- 172.21.15.19:0/1489205289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7815bd70 con 0x7f9c780b7340 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.790+0000 7f9c75e00640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004340 0x7f9c780b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.791+0000 7f9c75400640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c780b7340 0x7f9c781560d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c240 tx=0x7f9c780d34b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.791+0000 7f9c6e000640 1 -- 172.21.15.19:0/1489205289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64042020 con 0x7f9c780b7340 2024-04-25T11:28:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.791+0000 7f9c6e000640 1 -- 172.21.15.19:0/1489205289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c64004bf0 con 0x7f9c780b7340 2024-04-25T11:28:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.791+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c780b87a0 con 0x7f9c780b7340 2024-04-25T11:28:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.791+0000 7f9c6e000640 1 -- 172.21.15.19:0/1489205289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64007d10 con 0x7f9c780b7340 2024-04-25T11:28:01.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.791+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c780b6b40 con 0x7f9c780b7340 2024-04-25T11:28:01.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.793+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c780b8510 con 0x7f9c780b7340 2024-04-25T11:28:01.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.794+0000 7f9c6e000640 1 -- 172.21.15.19:0/1489205289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c64045020 con 0x7f9c780b7340 2024-04-25T11:28:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.796+0000 7f9c6e000640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c580744b0 0x7f9c580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.796+0000 7f9c6e000640 1 -- 172.21.15.19:0/1489205289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9c640b5770 con 0x7f9c780b7340 2024-04-25T11:28:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.797+0000 7f9c75e00640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c580744b0 0x7f9c580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.798+0000 7f9c75e00640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c580744b0 0x7f9c580769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009160 tx=0x7f9c68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:01.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:01.803+0000 7f9c6e000640 1 -- 172.21.15.19:0/1489205289 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c6407fe60 con 0x7f9c780b7340 2024-04-25T11:28:01.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.006+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c780cdf00 con 0x7f9c780b7340 2024-04-25T11:28:02.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.009+0000 7f9c6e000640 1 -- 172.21.15.19:0/1489205289 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7f9c780cdf00 con 0x7f9c780b7340 2024-04-25T11:28:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:02.002 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:02.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c580744b0 msgr2=0x7f9c580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c580744b0 0x7f9c580769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009160 tx=0x7f9c68008040 comp rx=0 tx=0).stop 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c780b7340 msgr2=0x7f9c781560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c780b7340 0x7f9c781560d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c240 tx=0x7f9c780d34b0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 shutdown_connections 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c580744b0 0x7f9c580769c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c7815b420 0x7f9c7815b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c780b7340 0x7f9c781560d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 --2- 172.21.15.19:0/1489205289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c004340 0x7f9c780b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 >> 172.21.15.19:0/1489205289 conn(0x7f9c7808d110 msgr2=0x7f9c78161000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.012+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 shutdown_connections 2024-04-25T11:28:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.013+0000 7f9c7d000640 1 -- 172.21.15.19:0/1489205289 wait complete. 2024-04-25T11:28:02.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:02.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.324+0000 7fe58f200640 1 Processor -- start 2024-04-25T11:28:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.324+0000 7fe58f200640 1 -- start start 2024-04-25T11:28:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.326+0000 7fe58f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5900d1f30 0x7fe5901431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.326+0000 7fe58f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5900d2820 0x7fe5900d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.326+0000 7fe58f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5900d11d0 con 0x7fe5900b5ce0 2024-04-25T11:28:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.326+0000 7fe58f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe5900d1aa0 con 0x7fe5900d2820 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.326+0000 7fe58f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe5900d3170 con 0x7fe5900d1f30 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe587e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5900d2820 0x7fe5900d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe587e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5900d2820 0x7fe5900d2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58930/0 (socket says 172.21.15.19:58930) 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe587e00640 1 -- 172.21.15.19:0/397050878 learned_addr learned my addr 172.21.15.19:0/397050878 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe58de00640 1 --2- 172.21.15.19:0/397050878 >> v2:172.21.15.169:3300/0 conn(0x7fe5900d1f30 0x7fe5901431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe587e00640 1 -- 172.21.15.19:0/397050878 >> v2:172.21.15.169:3300/0 conn(0x7fe5900d1f30 msgr2=0x7fe5901431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe587e00640 1 --2- 172.21.15.19:0/397050878 >> v2:172.21.15.169:3300/0 conn(0x7fe5900d1f30 0x7fe5901431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe587e00640 1 -- 172.21.15.19:0/397050878 >> v1:172.21.15.169:6789/0 conn(0x7fe5900b5ce0 legacy=0x7fe5900b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.327+0000 7fe587e00640 1 -- 172.21.15.19:0/397050878 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59015e1c0 con 0x7fe5900d2820 2024-04-25T11:28:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.329+0000 7fe587e00640 1 --2- 172.21.15.19:0/397050878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900d2820 0x7fe5900d2c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe590073820 tx=0x7fe57802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bb9feb5314da58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.329+0000 7fe58d400640 1 -- 172.21.15.19:0/397050878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578037070 con 0x7fe5900d2820 2024-04-25T11:28:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.329+0000 7fe58d400640 1 -- 172.21.15.19:0/397050878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe578032440 con 0x7fe5900d2820 2024-04-25T11:28:02.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.329+0000 7fe58d400640 1 -- 172.21.15.19:0/397050878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578032810 con 0x7fe5900d2820 2024-04-25T11:28:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 -- 172.21.15.19:0/397050878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900d2820 msgr2=0x7fe5900d2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 --2- 172.21.15.19:0/397050878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900d2820 0x7fe5900d2c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe590073820 tx=0x7fe57802a230 comp rx=0 tx=0).stop 2024-04-25T11:28:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 -- 172.21.15.19:0/397050878 shutdown_connections 2024-04-25T11:28:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 --2- 172.21.15.19:0/397050878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900d2820 0x7fe5900d2c30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 --2- 172.21.15.19:0/397050878 >> v2:172.21.15.169:3300/0 conn(0x7fe5900d1f30 0x7fe5901431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 -- 172.21.15.19:0/397050878 >> 172.21.15.19:0/397050878 conn(0x7fe5900b83a0 msgr2=0x7fe590070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 -- 172.21.15.19:0/397050878 shutdown_connections 2024-04-25T11:28:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.330+0000 7fe58f200640 1 -- 172.21.15.19:0/397050878 wait complete. 2024-04-25T11:28:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.333+0000 7fe58f200640 1 Processor -- start 2024-04-25T11:28:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.333+0000 7fe58f200640 1 -- start start 2024-04-25T11:28:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.334+0000 7fe58f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900b5ce0 0x7fe590141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.334+0000 7fe58f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5900d1f30 0x7fe5900b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.334+0000 7fe58f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5900d2820 0x7fe590157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.334+0000 7fe58f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe590143700 con 0x7fe5900b5ce0 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.334+0000 7fe58f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900d1730 con 0x7fe5900d1f30 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.334+0000 7fe58f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe5900d2340 con 0x7fe5900d2820 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900b5ce0 0x7fe590141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900b5ce0 0x7fe590141dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58932/0 (socket says 172.21.15.19:58932) 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 -- 172.21.15.19:0/2239734419 learned_addr learned my addr 172.21.15.19:0/2239734419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe587e00640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5900d1f30 0x7fe5900b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58e800640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5900d2820 0x7fe590157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 -- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5900d2820 msgr2=0x7fe590157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5900d2820 0x7fe590157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 -- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5900d1f30 msgr2=0x7fe5900b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5900d1f30 0x7fe5900b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58de00640 1 -- 172.21.15.19:0/2239734419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900b4580 con 0x7fe5900b5ce0 2024-04-25T11:28:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.335+0000 7fe58e800640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5900d2820 0x7fe590157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.336+0000 7fe58de00640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900b5ce0 0x7fe590141dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe58000d600 tx=0x7fe58000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.336+0000 7fe586000640 1 -- 172.21.15.19:0/2239734419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580004240 con 0x7fe5900b5ce0 2024-04-25T11:28:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.336+0000 7fe586000640 1 -- 172.21.15.19:0/2239734419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5800043e0 con 0x7fe5900b5ce0 2024-04-25T11:28:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.336+0000 7fe586000640 1 -- 172.21.15.19:0/2239734419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580003830 con 0x7fe5900b5ce0 2024-04-25T11:28:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.336+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5900b4860 con 0x7fe5900b5ce0 2024-04-25T11:28:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.336+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5900b4e20 con 0x7fe5900b5ce0 2024-04-25T11:28:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.337+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe590076c60 con 0x7fe5900b5ce0 2024-04-25T11:28:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.338+0000 7fe586000640 1 -- 172.21.15.19:0/2239734419 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe5800039d0 con 0x7fe5900b5ce0 2024-04-25T11:28:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.340+0000 7fe586000640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe56c0744e0 0x7fe56c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.340+0000 7fe586000640 1 -- 172.21.15.19:0/2239734419 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe5800955c0 con 0x7fe5900b5ce0 2024-04-25T11:28:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.341+0000 7fe587e00640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe56c0744e0 0x7fe56c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.342+0000 7fe587e00640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe56c0744e0 0x7fe56c0769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe578034000 tx=0x7fe57802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:02.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.345+0000 7fe586000640 1 -- 172.21.15.19:0/2239734419 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe58005fcb0 con 0x7fe5900b5ce0 2024-04-25T11:28:02.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.555+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5900b4040 con 0x7fe5900b5ce0 2024-04-25T11:28:02.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.556+0000 7fe586000640 1 -- 172.21.15.19:0/2239734419 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fe58005f5b0 con 0x7fe5900b5ce0 2024-04-25T11:28:02.548 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe56c0744e0 msgr2=0x7fe56c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe56c0744e0 0x7fe56c0769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe578034000 tx=0x7fe57802e000 comp rx=0 tx=0).stop 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900b5ce0 msgr2=0x7fe590141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900b5ce0 0x7fe590141dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe58000d600 tx=0x7fe58000dac0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 shutdown_connections 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe56c0744e0 0x7fe56c0769f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5900d1f30 0x7fe5900b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5900d2820 0x7fe590157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 --2- 172.21.15.19:0/2239734419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5900b5ce0 0x7fe590141dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 >> 172.21.15.19:0/2239734419 conn(0x7fe5900b83a0 msgr2=0x7fe5900b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 shutdown_connections 2024-04-25T11:28:02.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.560+0000 7fe58f200640 1 -- 172.21.15.19:0/2239734419 wait complete. 2024-04-25T11:28:02.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:02.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:28:02.715 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:28:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.875+0000 7f9467200640 1 Processor -- start 2024-04-25T11:28:02.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.876+0000 7f9467200640 1 -- start start 2024-04-25T11:28:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.878+0000 7f9467200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 0x7f94601655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.878+0000 7f9467200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f946006d5d0 0x7f946006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.878+0000 7f9467200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9460165ae0 con 0x7f9460168110 2024-04-25T11:28:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.878+0000 7f9467200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f946006df20 con 0x7f946006d5d0 2024-04-25T11:28:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.878+0000 7f9467200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f946014aed0 con 0x7f946014aac0 2024-04-25T11:28:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.879+0000 7f9466800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9460168110 0x7f9460168520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60828/0 (socket says 172.21.15.19:60828) 2024-04-25T11:28:02.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.879+0000 7f9466800640 1 -- 172.21.15.19:0/2276707555 learned_addr learned my addr 172.21.15.19:0/2276707555 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.879+0000 7f9465e00640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 0x7f94601655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.879+0000 7f9465400640 1 --2- 172.21.15.19:0/2276707555 >> v2:172.21.15.169:3300/0 conn(0x7f946006d5d0 0x7f946006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.879+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849227990 0 0) 0x7f945c001490 con 0x7f9460168110 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.879+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9448003520 con 0x7f9460168110 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.879+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266535006 0 0) 0x7f9448003520 con 0x7f9460168110 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 msgr2=0x7f94601655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 0x7f94601655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 >> v2:172.21.15.169:3300/0 conn(0x7f946006d5d0 msgr2=0x7f946006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 --2- 172.21.15.19:0/2276707555 >> v2:172.21.15.169:3300/0 conn(0x7f946006d5d0 0x7f946006d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946006f460 con 0x7f9460168110 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f945c001bc0 con 0x7f9460168110 2024-04-25T11:28:02.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 >> v1:172.21.15.19:6789/0 conn(0x7f9460168110 legacy=0x7f9460168520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94480039b0 0x7f9448003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 0x7f9448004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448004d40 0x7f9448005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f946006df20 con 0x7f9448004d40 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f946014aed0 con 0x7f94480039b0 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f944800baa0 con 0x7f946014aac0 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9466800640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448004d40 0x7f9448005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9465400640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 0x7f9448004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.880+0000 7f9465e00640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94480039b0 0x7f9448003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.881+0000 7f9466800640 1 -- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 msgr2=0x7f9448004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.881+0000 7f9466800640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946014aac0 0x7f9448004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.881+0000 7f9466800640 1 -- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94480039b0 msgr2=0x7f9448003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.881+0000 7f9466800640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94480039b0 0x7f9448003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.881+0000 7f9466800640 1 -- 172.21.15.19:0/2276707555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9448003520 con 0x7f9448004d40 2024-04-25T11:28:02.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.882+0000 7f9466800640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448004d40 0x7f9448005150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f945c00a970 tx=0x7f9448005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871779840ffa0245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.882+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c00a5c0 con 0x7f9448004d40 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.882+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945c02f070 con 0x7f9448004d40 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.882+0000 7f9464a00640 1 -- 172.21.15.19:0/2276707555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c02bde0 con 0x7f9448004d40 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.883+0000 7f9467200640 1 -- 172.21.15.19:0/2276707555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448004d40 msgr2=0x7f9448005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.883+0000 7f9467200640 1 --2- 172.21.15.19:0/2276707555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9448004d40 0x7f9448005150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f945c00a970 tx=0x7f9448005830 comp rx=0 tx=0).stop 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.883+0000 7f9466800640 1 -- 172.21.15.19:0/2276707555 reap_dead start 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.883+0000 7f9467200640 1 -- 172.21.15.19:0/2276707555 shutdown_connections 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.883+0000 7f9467200640 1 -- 172.21.15.19:0/2276707555 >> 172.21.15.19:0/2276707555 conn(0x7f9460072f40 msgr2=0x7f9460073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.883+0000 7f9467200640 1 -- 172.21.15.19:0/2276707555 shutdown_connections 2024-04-25T11:28:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.883+0000 7f9467200640 1 -- 172.21.15.19:0/2276707555 wait complete. 2024-04-25T11:28:02.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.886+0000 7f9467200640 1 Processor -- start 2024-04-25T11:28:02.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.886+0000 7f9467200640 1 -- start start 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.887+0000 7f9467200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946014dbd0 0x7f9460142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.887+0000 7f9467200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946006c5c0 0x7f946006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.887+0000 7f9467200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94601fdca0 0x7f94601fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.887+0000 7f9467200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f946006fd00 con 0x7f946014dbd0 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.887+0000 7f9467200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f946006fa20 con 0x7f94601fdca0 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.887+0000 7f9467200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f946006ffe0 con 0x7f946006c5c0 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946014dbd0 0x7f9460142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946014dbd0 0x7f9460142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58954/0 (socket says 172.21.15.19:58954) 2024-04-25T11:28:02.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 -- 172.21.15.19:0/656188331 learned_addr learned my addr 172.21.15.19:0/656188331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:02.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465400640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946006c5c0 0x7f946006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9466800640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94601fdca0 0x7f94601fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 -- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946006c5c0 msgr2=0x7f946006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946006c5c0 0x7f946006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 -- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94601fdca0 msgr2=0x7f94601fe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94601fdca0 0x7f94601fe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9465e00640 1 -- 172.21.15.19:0/656188331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946006cf10 con 0x7f946014dbd0 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.888+0000 7f9466800640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94601fdca0 0x7f94601fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.889+0000 7f9465e00640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946014dbd0 0x7f9460142460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9454011590 tx=0x7f9454009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.889+0000 7f945a000640 1 -- 172.21.15.19:0/656188331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454015930 con 0x7f946014dbd0 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.889+0000 7f945a000640 1 -- 172.21.15.19:0/656188331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9454015ad0 con 0x7f946014dbd0 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.889+0000 7f945a000640 1 -- 172.21.15.19:0/656188331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945401d400 con 0x7f946014dbd0 2024-04-25T11:28:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.889+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946014ce50 con 0x7f946014dbd0 2024-04-25T11:28:02.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.889+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946014d0f0 con 0x7f946014dbd0 2024-04-25T11:28:02.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.891+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9460076c60 con 0x7f946014dbd0 2024-04-25T11:28:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.892+0000 7f945a000640 1 -- 172.21.15.19:0/656188331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f945401d5a0 con 0x7f946014dbd0 2024-04-25T11:28:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.893+0000 7f945a000640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94440744b0 0x7f94440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.894+0000 7f945a000640 1 -- 172.21.15.19:0/656188331 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9454095760 con 0x7f946014dbd0 2024-04-25T11:28:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.894+0000 7f9465400640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94440744b0 0x7f94440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.895+0000 7f9465400640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94440744b0 0x7f94440769c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9460072db0 tx=0x7f9450008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:02.899+0000 7f945a000640 1 -- 172.21.15.19:0/656188331 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f946006cf10 con 0x7f946014dbd0 2024-04-25T11:28:03.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.103+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9460070e70 con 0x7f946014dbd0 2024-04-25T11:28:03.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.104+0000 7f945a000640 1 -- 172.21.15.19:0/656188331 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f9454019030 con 0x7f946014dbd0 2024-04-25T11:28:03.096 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:03.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94440744b0 msgr2=0x7f94440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94440744b0 0x7f94440769c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9460072db0 tx=0x7f9450008040 comp rx=0 tx=0).stop 2024-04-25T11:28:03.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946014dbd0 msgr2=0x7f9460142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946014dbd0 0x7f9460142460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9454011590 tx=0x7f9454009f90 comp rx=0 tx=0).stop 2024-04-25T11:28:03.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 shutdown_connections 2024-04-25T11:28:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94440744b0 0x7f94440769c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94601fdca0 0x7f94601fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f946006c5c0 0x7f946006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 --2- 172.21.15.19:0/656188331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946014dbd0 0x7f9460142460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 >> 172.21.15.19:0/656188331 conn(0x7f9460072f40 msgr2=0x7f94600b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 shutdown_connections 2024-04-25T11:28:03.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.107+0000 7f9467200640 1 -- 172.21.15.19:0/656188331 wait complete. 2024-04-25T11:28:03.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.423+0000 7f8fad400640 1 Processor -- start 2024-04-25T11:28:03.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.423+0000 7f8fad400640 1 -- start start 2024-04-25T11:28:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.425+0000 7f8fad400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fa8070630 0x7f8fa80cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.426+0000 7f8fad400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fa8071110 con 0x7f8fa815adf0 2024-04-25T11:28:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.426+0000 7f8fad400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fa806fb50 con 0x7f8fa806dce0 2024-04-25T11:28:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.426+0000 7f8fad400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8fa80713f0 con 0x7f8fa8070630 2024-04-25T11:28:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.426+0000 7f8fa6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fa8070630 0x7f8fa80cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.426+0000 7f8f9fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8fa815adf0 0x7f8fa806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60836/0 (socket says 172.21.15.19:60836) 2024-04-25T11:28:03.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.426+0000 7f8f9fe00640 1 -- 172.21.15.19:0/1629596819 learned_addr learned my addr 172.21.15.19:0/1629596819 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.427+0000 7f8fa6800640 1 -- 172.21.15.19:0/1629596819 >> v1:172.21.15.169:6789/0 conn(0x7f8fa806dce0 legacy=0x7f8fa8160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.427+0000 7f8fa6800640 1 -- 172.21.15.19:0/1629596819 >> v1:172.21.15.19:6789/0 conn(0x7f8fa815adf0 legacy=0x7f8fa806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.427+0000 7f8fa6800640 1 -- 172.21.15.19:0/1629596819 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa806fe30 con 0x7f8fa8070630 2024-04-25T11:28:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.427+0000 7f8fa5e00640 1 -- 172.21.15.19:0/1629596819 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4274527160 0 0) 0x7f8fa8071110 con 0x7f8fa815adf0 2024-04-25T11:28:03.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.428+0000 7f8fa6800640 1 --2- 172.21.15.19:0/1629596819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8070630 0x7f8fa80cec50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8f980097a0 tx=0x7f8f9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba686a6b50ccc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.428+0000 7f8fa5e00640 1 -- 172.21.15.19:0/1629596819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98037070 con 0x7f8fa8070630 2024-04-25T11:28:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.428+0000 7f8fa5e00640 1 -- 172.21.15.19:0/1629596819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f98032440 con 0x7f8fa8070630 2024-04-25T11:28:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.428+0000 7f8fa5e00640 1 -- 172.21.15.19:0/1629596819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98032810 con 0x7f8fa8070630 2024-04-25T11:28:03.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.429+0000 7f8fad400640 1 -- 172.21.15.19:0/1629596819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8070630 msgr2=0x7f8fa80cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.429+0000 7f8fad400640 1 --2- 172.21.15.19:0/1629596819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8070630 0x7f8fa80cec50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8f980097a0 tx=0x7f8f9802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.429+0000 7f8fad400640 1 -- 172.21.15.19:0/1629596819 shutdown_connections 2024-04-25T11:28:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.429+0000 7f8fad400640 1 --2- 172.21.15.19:0/1629596819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8070630 0x7f8fa80cec50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.429+0000 7f8fad400640 1 -- 172.21.15.19:0/1629596819 >> 172.21.15.19:0/1629596819 conn(0x7f8fa8073090 msgr2=0x7f8fa80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.429+0000 7f8fad400640 1 -- 172.21.15.19:0/1629596819 shutdown_connections 2024-04-25T11:28:03.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.429+0000 7f8fad400640 1 -- 172.21.15.19:0/1629596819 wait complete. 2024-04-25T11:28:03.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.432+0000 7f8fad400640 1 Processor -- start 2024-04-25T11:28:03.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.432+0000 7f8fad400640 1 -- start start 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.433+0000 7f8fad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fa8070630 0x7f8fa8148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.433+0000 7f8fad400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fa815adf0 0x7f8fa8157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.433+0000 7f8fad400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8158380 0x7f8fa806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.433+0000 7f8fad400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8070c80 con 0x7f8fa8158380 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.433+0000 7f8fad400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa80701a0 con 0x7f8fa815adf0 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.433+0000 7f8fad400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa8070e00 con 0x7f8fa8070630 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8158380 0x7f8fa806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8158380 0x7f8fa806c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58984/0 (socket says 172.21.15.19:58984) 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 -- 172.21.15.19:0/4106940121 learned_addr learned my addr 172.21.15.19:0/4106940121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fa815adf0 0x7f8fa8157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa6800640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fa8070630 0x7f8fa8148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 -- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fa8070630 msgr2=0x7f8fa8148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fa8070630 0x7f8fa8148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 -- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fa815adf0 msgr2=0x7f8fa8157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fa815adf0 0x7f8fa8157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa7200640 1 -- 172.21.15.19:0/4106940121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa8155380 con 0x7f8fa8158380 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fa815adf0 0x7f8fa8157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:03.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.434+0000 7f8fa6800640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fa8070630 0x7f8fa8148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.435+0000 7f8fa7200640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8158380 0x7f8fa806c5c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000c730 tx=0x7f8fa000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.435+0000 7f8f9ea00640 1 -- 172.21.15.19:0/4106940121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0003b70 con 0x7f8fa8158380 2024-04-25T11:28:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.435+0000 7f8f9ea00640 1 -- 172.21.15.19:0/4106940121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa0003d10 con 0x7f8fa8158380 2024-04-25T11:28:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.435+0000 7f8f9ea00640 1 -- 172.21.15.19:0/4106940121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0005310 con 0x7f8fa8158380 2024-04-25T11:28:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.435+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa8155660 con 0x7f8fa8158380 2024-04-25T11:28:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.435+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa81f4400 con 0x7f8fa8158380 2024-04-25T11:28:03.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.436+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa81f4660 con 0x7f8fa8158380 2024-04-25T11:28:03.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.437+0000 7f8f9ea00640 1 -- 172.21.15.19:0/4106940121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8fa00056c0 con 0x7f8fa8158380 2024-04-25T11:28:03.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.439+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f840744b0 0x7f8f840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.439+0000 7f8f9ea00640 1 -- 172.21.15.19:0/4106940121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8fa0095370 con 0x7f8fa8158380 2024-04-25T11:28:03.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.439+0000 7f8fa6800640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f840744b0 0x7f8f840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.441+0000 7f8fa6800640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f840744b0 0x7f8f840769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f98034000 tx=0x7f8f9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.443+0000 7f8f9ea00640 1 -- 172.21.15.19:0/4106940121 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fa005fa60 con 0x7f8fa8158380 2024-04-25T11:28:03.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.646+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8fa8155110 con 0x7f8fa8158380 2024-04-25T11:28:03.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.647+0000 7f8f9ea00640 1 -- 172.21.15.19:0/4106940121 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f8fa005f360 con 0x7f8fa8158380 2024-04-25T11:28:03.639 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f840744b0 msgr2=0x7f8f840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f840744b0 0x7f8f840769c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f98034000 tx=0x7f8f9802e000 comp rx=0 tx=0).stop 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8158380 msgr2=0x7f8fa806c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8158380 0x7f8fa806c5c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000c730 tx=0x7f8fa000cbf0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 shutdown_connections 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f840744b0 0x7f8f840769c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fa815adf0 0x7f8fa8157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa8158380 0x7f8fa806c5c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 --2- 172.21.15.19:0/4106940121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fa8070630 0x7f8fa8148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.649+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 >> 172.21.15.19:0/4106940121 conn(0x7f8fa8073090 msgr2=0x7f8fa8153590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.650+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 shutdown_connections 2024-04-25T11:28:03.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.650+0000 7f8fad400640 1 -- 172.21.15.19:0/4106940121 wait complete. 2024-04-25T11:28:03.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:03.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.963+0000 7f0249200640 1 Processor -- start 2024-04-25T11:28:03.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.963+0000 7f0249200640 1 -- start start 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0249200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0244164b00 0x7f02440b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0249200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f024415df80 0x7f02440b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0249200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f0244160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0249200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0244164670 con 0x7f024415df80 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0249200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f02441615f0 con 0x7f0244164b00 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0249200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0244164020 con 0x7f0244160890 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0241400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f024415df80 0x7f02440b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0241400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f024415df80 0x7f02440b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36326/0 (socket says 172.21.15.19:36326) 2024-04-25T11:28:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0241400640 1 -- 172.21.15.19:0/2734096875 learned_addr learned my addr 172.21.15.19:0/2734096875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.966+0000 7f0242800640 1 --2- 172.21.15.19:0/2734096875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f0244160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.967+0000 7f0241e00640 1 --2- 172.21.15.19:0/2734096875 >> v2:172.21.15.169:3300/0 conn(0x7f0244164b00 0x7f02440b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.967+0000 7f0241400640 1 -- 172.21.15.19:0/2734096875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 msgr2=0x7f0244160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.967+0000 7f0241400640 1 --2- 172.21.15.19:0/2734096875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f0244160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.967+0000 7f0241400640 1 -- 172.21.15.19:0/2734096875 >> v2:172.21.15.169:3300/0 conn(0x7f0244164b00 msgr2=0x7f02440b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.967+0000 7f0241400640 1 --2- 172.21.15.19:0/2734096875 >> v2:172.21.15.169:3300/0 conn(0x7f0244164b00 0x7f02440b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.967+0000 7f0241400640 1 -- 172.21.15.19:0/2734096875 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0244164300 con 0x7f024415df80 2024-04-25T11:28:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.968+0000 7f0241400640 1 --2- 172.21.15.19:0/2734096875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024415df80 0x7f02440b80c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f024408e410 tx=0x7f023002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affc3b7f715e3310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.968+0000 7f0240a00640 1 -- 172.21.15.19:0/2734096875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230037070 con 0x7f024415df80 2024-04-25T11:28:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.968+0000 7f0240a00640 1 -- 172.21.15.19:0/2734096875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0230032440 con 0x7f024415df80 2024-04-25T11:28:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.968+0000 7f0240a00640 1 -- 172.21.15.19:0/2734096875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230032810 con 0x7f024415df80 2024-04-25T11:28:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 -- 172.21.15.19:0/2734096875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024415df80 msgr2=0x7f02440b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 --2- 172.21.15.19:0/2734096875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024415df80 0x7f02440b80c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f024408e410 tx=0x7f023002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 -- 172.21.15.19:0/2734096875 shutdown_connections 2024-04-25T11:28:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 --2- 172.21.15.19:0/2734096875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f024415df80 0x7f02440b80c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 --2- 172.21.15.19:0/2734096875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f0244160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 --2- 172.21.15.19:0/2734096875 >> v2:172.21.15.169:3300/0 conn(0x7f0244164b00 0x7f02440b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 -- 172.21.15.19:0/2734096875 >> 172.21.15.19:0/2734096875 conn(0x7f02440d2bc0 msgr2=0x7f02440d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 -- 172.21.15.19:0/2734096875 shutdown_connections 2024-04-25T11:28:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.969+0000 7f0249200640 1 -- 172.21.15.19:0/2734096875 wait complete. 2024-04-25T11:28:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.971+0000 7f0249200640 1 Processor -- start 2024-04-25T11:28:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.972+0000 7f0249200640 1 -- start start 2024-04-25T11:28:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0249200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f024415df80 0x7f02440b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0249200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f024415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0249200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0244164b00 0x7f024415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0249200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02440b4a00 con 0x7f0244164b00 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0249200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02440b4880 con 0x7f024415df80 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0249200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02440b4b80 con 0x7f0244160890 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0242800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0244164b00 0x7f024415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0242800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0244164b00 0x7f024415bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36330/0 (socket says 172.21.15.19:36330) 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.973+0000 7f0242800640 1 -- 172.21.15.19:0/1745321504 learned_addr learned my addr 172.21.15.19:0/1745321504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0241400640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f024415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0241e00640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f024415df80 0x7f02440b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0242800640 1 -- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 msgr2=0x7f024415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0242800640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f024415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0242800640 1 -- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f024415df80 msgr2=0x7f02440b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0242800640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f024415df80 0x7f02440b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0242800640 1 -- 172.21.15.19:0/1745321504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02440b6fc0 con 0x7f0244164b00 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0241e00640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f024415df80 0x7f02440b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f0242800640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0244164b00 0x7f024415bd40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f023c007cf0 tx=0x7f023c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.974+0000 7f023a000640 1 -- 172.21.15.19:0/1745321504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c014420 con 0x7f0244164b00 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.975+0000 7f023a000640 1 -- 172.21.15.19:0/1745321504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f023c019070 con 0x7f0244164b00 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.975+0000 7f023a000640 1 -- 172.21.15.19:0/1745321504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c014800 con 0x7f0244164b00 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.975+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02441594a0 con 0x7f0244164b00 2024-04-25T11:28:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.975+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0244159a60 con 0x7f0244164b00 2024-04-25T11:28:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.976+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0244050740 con 0x7f0244164b00 2024-04-25T11:28:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.977+0000 7f023a000640 1 -- 172.21.15.19:0/1745321504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f023c0149a0 con 0x7f0244164b00 2024-04-25T11:28:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.979+0000 7f023a000640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02240744e0 0x7f02240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.979+0000 7f023a000640 1 -- 172.21.15.19:0/1745321504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f023c0959b0 con 0x7f0244164b00 2024-04-25T11:28:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.979+0000 7f0241e00640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02240744e0 0x7f02240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.981+0000 7f0241e00640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02240744e0 0x7f02240769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f023400b820 tx=0x7f0234008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:03.984+0000 7f023a000640 1 -- 172.21.15.19:0/1745321504 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f023c05fa50 con 0x7f0244164b00 2024-04-25T11:28:04.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.190+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02440d5190 con 0x7f0244164b00 2024-04-25T11:28:04.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.193+0000 7f023a000640 1 -- 172.21.15.19:0/1745321504 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16609 (secure 0 0 0) 0x7f023c063d50 con 0x7f0244164b00 2024-04-25T11:28:04.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:04.185 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:27:26.558126+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02240744e0 msgr2=0x7f02240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02240744e0 0x7f02240769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f023400b820 tx=0x7f0234008040 comp rx=0 tx=0).stop 2024-04-25T11:28:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0244164b00 msgr2=0x7f024415bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0244164b00 0x7f024415bd40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f023c007cf0 tx=0x7f023c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 shutdown_connections 2024-04-25T11:28:04.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02240744e0 0x7f02240769f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0244164b00 0x7f024415bd40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0244160890 0x7f024415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 --2- 172.21.15.19:0/1745321504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f024415df80 0x7f02440b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 >> 172.21.15.19:0/1745321504 conn(0x7f02440d2bc0 msgr2=0x7f024415f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 shutdown_connections 2024-04-25T11:28:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.196+0000 7f0249200640 1 -- 172.21.15.19:0/1745321504 wait complete. 2024-04-25T11:28:04.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:04.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.517+0000 7f73d6800640 1 Processor -- start 2024-04-25T11:28:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.517+0000 7f73d6800640 1 -- start start 2024-04-25T11:28:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.520+0000 7f73d6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 0x7f73d015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.520+0000 7f73d6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f73d00ce930 0x7f73d015d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.520+0000 7f73d6800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73d00cda60 con 0x7f73d006d5d0 2024-04-25T11:28:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.520+0000 7f73d6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f73d015dfe0 con 0x7f73d00ce930 2024-04-25T11:28:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.520+0000 7f73d6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73d015e2c0 con 0x7f73d00ce520 2024-04-25T11:28:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.520+0000 7f73d5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f73d006d5d0 0x7f73d006d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40540/0 (socket says 172.21.15.19:40540) 2024-04-25T11:28:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.520+0000 7f73d5e00640 1 -- 172.21.15.19:0/3072133771 learned_addr learned my addr 172.21.15.19:0/3072133771 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73d4a00640 1 --2- 172.21.15.19:0/3072133771 >> v2:172.21.15.169:3300/0 conn(0x7f73d00ce930 0x7f73d015d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73d5400640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 0x7f73d015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397871071 0 0) 0x7f73d00cda60 con 0x7f73d006d5d0 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b8005730 con 0x7f73d006d5d0 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3594032012 0 0) 0x7f73b8005730 con 0x7f73d006d5d0 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 msgr2=0x7f73d015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 0x7f73d015d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 >> v2:172.21.15.169:3300/0 conn(0x7f73d00ce930 msgr2=0x7f73d015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 --2- 172.21.15.19:0/3072133771 >> v2:172.21.15.169:3300/0 conn(0x7f73d00ce930 0x7f73d015d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.521+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d00cdd20 con 0x7f73d006d5d0 2024-04-25T11:28:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f73c80019f0 con 0x7f73d006d5d0 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 >> v1:172.21.15.19:6789/0 conn(0x7f73d006d5d0 legacy=0x7f73d006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 0x7f73b8005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006190 0x7f73b8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b8006f50 0x7f73b8007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73d015dfe0 con 0x7f73b8006f50 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73d015e2c0 con 0x7f73b8006190 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73b800dd30 con 0x7f73d00ce520 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73d5e00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b8006f50 0x7f73b8007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73d5400640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 0x7f73b8005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.522+0000 7f73d4a00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006190 0x7f73b8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.523+0000 7f73d5e00640 1 -- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 msgr2=0x7f73b8005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.523+0000 7f73d5e00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00ce520 0x7f73b8005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.523+0000 7f73d5e00640 1 -- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006190 msgr2=0x7f73b8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.523+0000 7f73d5e00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006190 0x7f73b8006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.523+0000 7f73d5e00640 1 -- 172.21.15.19:0/3072133771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b8005730 con 0x7f73b8006f50 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.524+0000 7f73d5e00640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b8006f50 0x7f73b8007360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f73c8002230 tx=0x7f73b8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebed495e9f28623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.524+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c800a410 con 0x7f73b8006f50 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.524+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c802f070 con 0x7f73b8006f50 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.524+0000 7f73cfe00640 1 -- 172.21.15.19:0/3072133771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c802bbc0 con 0x7f73b8006f50 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.525+0000 7f73d6800640 1 -- 172.21.15.19:0/3072133771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b8006f50 msgr2=0x7f73b8007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.525+0000 7f73d6800640 1 --2- 172.21.15.19:0/3072133771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b8006f50 0x7f73b8007360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f73c8002230 tx=0x7f73b8007a40 comp rx=0 tx=0).stop 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.525+0000 7f73d5e00640 1 -- 172.21.15.19:0/3072133771 reap_dead start 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.525+0000 7f73d6800640 1 -- 172.21.15.19:0/3072133771 shutdown_connections 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.525+0000 7f73d6800640 1 -- 172.21.15.19:0/3072133771 >> 172.21.15.19:0/3072133771 conn(0x7f73d0072f40 msgr2=0x7f73d0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.525+0000 7f73d6800640 1 -- 172.21.15.19:0/3072133771 shutdown_connections 2024-04-25T11:28:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.525+0000 7f73d6800640 1 -- 172.21.15.19:0/3072133771 wait complete. 2024-04-25T11:28:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.528+0000 7f73d6800640 1 Processor -- start 2024-04-25T11:28:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.528+0000 7f73d6800640 1 -- start start 2024-04-25T11:28:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.529+0000 7f73d6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006f50 0x7f73d00cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.529+0000 7f73d6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00cabf0 0x7f73d00cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.529+0000 7f73d6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d014a2a0 0x7f73d01588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.529+0000 7f73d6800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73d00b73d0 con 0x7f73d014a2a0 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.529+0000 7f73d6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0160d30 con 0x7f73b8006f50 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.529+0000 7f73d6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73d00b7550 con 0x7f73d00cabf0 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d014a2a0 0x7f73d01588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d014a2a0 0x7f73d01588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36348/0 (socket says 172.21.15.19:36348) 2024-04-25T11:28:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 -- 172.21.15.19:0/3427129340 learned_addr learned my addr 172.21.15.19:0/3427129340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5400640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006f50 0x7f73d00cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d4a00640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00cabf0 0x7f73d00cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 -- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00cabf0 msgr2=0x7f73d00cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00cabf0 0x7f73d00cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 -- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006f50 msgr2=0x7f73d00cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006f50 0x7f73d00cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5e00640 1 -- 172.21.15.19:0/3427129340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d00cc940 con 0x7f73d014a2a0 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.530+0000 7f73d5400640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006f50 0x7f73d00cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.531+0000 7f73d5e00640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d014a2a0 0x7f73d01588a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73c8000f80 tx=0x7f73c8001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.531+0000 7f73cd600640 1 -- 172.21.15.19:0/3427129340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8044070 con 0x7f73d014a2a0 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.531+0000 7f73cd600640 1 -- 172.21.15.19:0/3427129340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c800a3f0 con 0x7f73d014a2a0 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.531+0000 7f73cd600640 1 -- 172.21.15.19:0/3427129340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8001f70 con 0x7f73d014a2a0 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.531+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d00cb360 con 0x7f73d014a2a0 2024-04-25T11:28:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.531+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d00cb920 con 0x7f73d014a2a0 2024-04-25T11:28:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.532+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d00cb0d0 con 0x7f73d014a2a0 2024-04-25T11:28:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.534+0000 7f73cd600640 1 -- 172.21.15.19:0/3427129340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f73c8047070 con 0x7f73d014a2a0 2024-04-25T11:28:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.535+0000 7f73cd600640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73b00744b0 0x7f73b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.535+0000 7f73cd600640 1 -- 172.21.15.19:0/3427129340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f73c80b54d0 con 0x7f73d014a2a0 2024-04-25T11:28:04.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.536+0000 7f73d5400640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73b00744b0 0x7f73b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:04.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.537+0000 7f73d5400640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73b00744b0 0x7f73b00769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f73d00b5b40 tx=0x7f73c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.540+0000 7f73cd600640 1 -- 172.21.15.19:0/3427129340 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73c807fbc0 con 0x7f73d014a2a0 2024-04-25T11:28:04.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.740+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73d0076c60 con 0x7f73d014a2a0 2024-04-25T11:28:04.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.741+0000 7f73cd600640 1 -- 172.21.15.19:0/3427129340 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f73c807f4c0 con 0x7f73d014a2a0 2024-04-25T11:28:04.733 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.743+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73b00744b0 msgr2=0x7f73b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.743+0000 7f73d6800640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73b00744b0 0x7f73b00769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f73d00b5b40 tx=0x7f73c4008040 comp rx=0 tx=0).stop 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.743+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d014a2a0 msgr2=0x7f73d01588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.743+0000 7f73d6800640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d014a2a0 0x7f73d01588a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73c8000f80 tx=0x7f73c8001360 comp rx=0 tx=0).stop 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 shutdown_connections 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73b00744b0 0x7f73b00769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73d014a2a0 0x7f73d01588a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73d00cabf0 0x7f73d00cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 --2- 172.21.15.19:0/3427129340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b8006f50 0x7f73d00cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 >> 172.21.15.19:0/3427129340 conn(0x7f73d0072f40 msgr2=0x7f73d006f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 shutdown_connections 2024-04-25T11:28:04.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:04.744+0000 7f73d6800640 1 -- 172.21.15.19:0/3427129340 wait complete. 2024-04-25T11:28:04.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:05.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.055+0000 7f96aec00640 1 Processor -- start 2024-04-25T11:28:05.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.055+0000 7f96aec00640 1 -- start start 2024-04-25T11:28:05.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.057+0000 7f96aec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 0x7f96a81606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.057+0000 7f96aec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f96a8163c90 0x7f96a806f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.057+0000 7f96aec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a80b0090 con 0x7f96a806d5d0 2024-04-25T11:28:05.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.057+0000 7f96aec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f96a8160c80 con 0x7f96a8163c90 2024-04-25T11:28:05.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.057+0000 7f96aec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96a8160f60 con 0x7f96a815de90 2024-04-25T11:28:05.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.058+0000 7f96ae200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f96a806d5d0 0x7f96a806d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40552/0 (socket says 172.21.15.19:40552) 2024-04-25T11:28:05.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.058+0000 7f96ae200640 1 -- 172.21.15.19:0/366479002 learned_addr learned my addr 172.21.15.19:0/366479002 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:05.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.058+0000 7f96ad800640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 0x7f96a81606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.058+0000 7f96ace00640 1 --2- 172.21.15.19:0/366479002 >> v2:172.21.15.169:3300/0 conn(0x7f96a8163c90 0x7f96a806f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.058+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3179660432 0 0) 0x7f96a80b0090 con 0x7f96a806d5d0 2024-04-25T11:28:05.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.058+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9690003520 con 0x7f96a806d5d0 2024-04-25T11:28:05.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2357036166 0 0) 0x7f9690003520 con 0x7f96a806d5d0 2024-04-25T11:28:05.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 msgr2=0x7f96a81606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 0x7f96a81606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 >> v2:172.21.15.169:3300/0 conn(0x7f96a8163c90 msgr2=0x7f96a806f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 --2- 172.21.15.19:0/366479002 >> v2:172.21.15.169:3300/0 conn(0x7f96a8163c90 0x7f96a806f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a8161240 con 0x7f96a806d5d0 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f96a00019f0 con 0x7f96a806d5d0 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 >> v1:172.21.15.19:6789/0 conn(0x7f96a806d5d0 legacy=0x7f96a806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 0x7f96900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9690003f80 0x7f9690004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9690004d40 0x7f9690005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8160c80 con 0x7f9690003f80 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8160f60 con 0x7f9690004d40 2024-04-25T11:28:05.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f969000bb20 con 0x7f96a815de90 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.059+0000 7f96ace00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9690003f80 0x7f9690004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ad800640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 0x7f96900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ae200640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9690004d40 0x7f9690005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ace00640 1 -- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 msgr2=0x7f96900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ace00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96a815de90 0x7f96900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ace00640 1 -- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9690004d40 msgr2=0x7f9690005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ace00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9690004d40 0x7f9690005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ace00640 1 -- 172.21.15.19:0/366479002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9690003520 con 0x7f9690003f80 2024-04-25T11:28:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.060+0000 7f96ae200640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9690004d40 0x7f9690005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:05.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.061+0000 7f96ace00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9690003f80 0x7f9690004800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96a8074010 tx=0x7f9690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc6ebd5ba32865c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:05.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.061+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694036070 con 0x7f9690003f80 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.061+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9694031400 con 0x7f9690003f80 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.061+0000 7f96a7e00640 1 -- 172.21.15.19:0/366479002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694009690 con 0x7f9690003f80 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.062+0000 7f96aec00640 1 -- 172.21.15.19:0/366479002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9690003f80 msgr2=0x7f9690004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.062+0000 7f96aec00640 1 --2- 172.21.15.19:0/366479002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9690003f80 0x7f9690004800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f96a8074010 tx=0x7f9690005830 comp rx=0 tx=0).stop 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.062+0000 7f96ae200640 1 -- 172.21.15.19:0/366479002 reap_dead start 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.062+0000 7f96aec00640 1 -- 172.21.15.19:0/366479002 shutdown_connections 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.062+0000 7f96aec00640 1 -- 172.21.15.19:0/366479002 >> 172.21.15.19:0/366479002 conn(0x7f96a8072f40 msgr2=0x7f96a8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.062+0000 7f96aec00640 1 -- 172.21.15.19:0/366479002 shutdown_connections 2024-04-25T11:28:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.062+0000 7f96aec00640 1 -- 172.21.15.19:0/366479002 wait complete. 2024-04-25T11:28:05.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.065+0000 7f96aec00640 1 Processor -- start 2024-04-25T11:28:05.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.065+0000 7f96aec00640 1 -- start start 2024-04-25T11:28:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.068+0000 7f96aec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9690003f80 0x7f96a814b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.068+0000 7f96aec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a8156730 0x7f96a81f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.068+0000 7f96aec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a81f9e10 0x7f96a815f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.068+0000 7f96aec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96a806f780 con 0x7f96a81f9e10 2024-04-25T11:28:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.068+0000 7f96aec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f96a806f600 con 0x7f96a8156730 2024-04-25T11:28:05.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.068+0000 7f96aec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96a8070d30 con 0x7f9690003f80 2024-04-25T11:28:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.069+0000 7f96ad800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9690003f80 0x7f96a814b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.069+0000 7f96ad800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9690003f80 0x7f96a814b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:50816/0 (socket says 172.21.15.19:50816) 2024-04-25T11:28:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.069+0000 7f96ad800640 1 -- 172.21.15.19:0/295142566 learned_addr learned my addr 172.21.15.19:0/295142566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.069+0000 7f96ae200640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a81f9e10 0x7f96a815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.069+0000 7f96ace00640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a8156730 0x7f96a81f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.070+0000 7f96ae200640 1 -- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9690003f80 msgr2=0x7f96a814b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.070+0000 7f96ae200640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9690003f80 0x7f96a814b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.070+0000 7f96ae200640 1 -- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a8156730 msgr2=0x7f96a81f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.070+0000 7f96ae200640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a8156730 0x7f96a81f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.070+0000 7f96ae200640 1 -- 172.21.15.19:0/295142566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a8158eb0 con 0x7f96a81f9e10 2024-04-25T11:28:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.070+0000 7f96ae200640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a81f9e10 0x7f96a815f020 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96a8071350 tx=0x7f96a000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.071+0000 7f96a5600640 1 -- 172.21.15.19:0/295142566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0004d60 con 0x7f96a81f9e10 2024-04-25T11:28:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.071+0000 7f96a5600640 1 -- 172.21.15.19:0/295142566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a000bb80 con 0x7f96a81f9e10 2024-04-25T11:28:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.071+0000 7f96a5600640 1 -- 172.21.15.19:0/295142566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0007070 con 0x7f96a81f9e10 2024-04-25T11:28:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.071+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a81fa520 con 0x7f96a81f9e10 2024-04-25T11:28:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.071+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a8155f30 con 0x7f96a81f9e10 2024-04-25T11:28:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.072+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a81fa290 con 0x7f96a81f9e10 2024-04-25T11:28:05.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.074+0000 7f96a5600640 1 -- 172.21.15.19:0/295142566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f96a001b060 con 0x7f96a81f9e10 2024-04-25T11:28:05.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.075+0000 7f96a5600640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9688074500 0x7f9688076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.075+0000 7f96a5600640 1 -- 172.21.15.19:0/295142566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5738+0+0 (secure 0 0 0) 0x7f96a0094f60 con 0x7f96a81f9e10 2024-04-25T11:28:05.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.076+0000 7f96ad800640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9688074500 0x7f9688076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.077+0000 7f96ad800640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9688074500 0x7f9688076a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f969c009160 tx=0x7f969c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.080+0000 7f96a5600640 1 -- 172.21.15.19:0/295142566 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96a005f130 con 0x7f96a81f9e10 2024-04-25T11:28:05.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.282+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96a8076c60 con 0x7f96a81f9e10 2024-04-25T11:28:05.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.282+0000 7f96a5600640 1 -- 172.21.15.19:0/295142566 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f96a0063430 con 0x7f96a81f9e10 2024-04-25T11:28:05.274 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:05.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9688074500 msgr2=0x7f9688076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9688074500 0x7f9688076a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f969c009160 tx=0x7f969c008040 comp rx=0 tx=0).stop 2024-04-25T11:28:05.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a81f9e10 msgr2=0x7f96a815f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a81f9e10 0x7f96a815f020 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96a8071350 tx=0x7f96a000b4a0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 shutdown_connections 2024-04-25T11:28:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9688074500 0x7f9688076a10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96a8156730 0x7f96a81f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96a81f9e10 0x7f96a815f020 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 --2- 172.21.15.19:0/295142566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9690003f80 0x7f96a814b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 >> 172.21.15.19:0/295142566 conn(0x7f96a8072f40 msgr2=0x7f96a80684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 shutdown_connections 2024-04-25T11:28:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.285+0000 7f96aec00640 1 -- 172.21.15.19:0/295142566 wait complete. 2024-04-25T11:28:05.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.600+0000 7f635ee00640 1 Processor -- start 2024-04-25T11:28:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.600+0000 7f635ee00640 1 -- start start 2024-04-25T11:28:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.602+0000 7f635ee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6358143590 0x7f63580aed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.602+0000 7f635ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635806f8f0 0x7f63580b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.602+0000 7f635ee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63580700d0 con 0x7f63580b0b90 2024-04-25T11:28:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.602+0000 7f635ee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6358070970 con 0x7f6358143590 2024-04-25T11:28:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.602+0000 7f635ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f63580b4e20 con 0x7f635806f8f0 2024-04-25T11:28:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635e400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f63580b0b90 0x7f63581609e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40562/0 (socket says 172.21.15.19:40562) 2024-04-25T11:28:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635e400640 1 -- 172.21.15.19:0/2687005481 learned_addr learned my addr 172.21.15.19:0/2687005481 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635da00640 1 --2- 172.21.15.19:0/2687005481 >> v2:172.21.15.19:3300/0 conn(0x7f6358143590 0x7f63580aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635d000640 1 --2- 172.21.15.19:0/2687005481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635806f8f0 0x7f63580b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635da00640 1 -- 172.21.15.19:0/2687005481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635806f8f0 msgr2=0x7f63580b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635da00640 1 --2- 172.21.15.19:0/2687005481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635806f8f0 0x7f63580b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635da00640 1 -- 172.21.15.19:0/2687005481 >> v1:172.21.15.19:6789/0 conn(0x7f63580b0b90 legacy=0x7f63581609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f6357e00640 1 -- 172.21.15.19:0/2687005481 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538137386 0 0) 0x7f63580700d0 con 0x7f63580b0b90 2024-04-25T11:28:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.603+0000 7f635da00640 1 -- 172.21.15.19:0/2687005481 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63580703b0 con 0x7f6358143590 2024-04-25T11:28:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.604+0000 7f635da00640 1 --2- 172.21.15.19:0/2687005481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6358143590 0x7f63580aed40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f634c0097a0 tx=0x7f634c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ef1287f5c274f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:05.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.605+0000 7f6357e00640 1 -- 172.21.15.19:0/2687005481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c037070 con 0x7f6358143590 2024-04-25T11:28:05.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.605+0000 7f6357e00640 1 -- 172.21.15.19:0/2687005481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f634c02ae50 con 0x7f6358143590 2024-04-25T11:28:05.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.605+0000 7f6357e00640 1 -- 172.21.15.19:0/2687005481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c032660 con 0x7f6358143590 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.605+0000 7f635ee00640 1 -- 172.21.15.19:0/2687005481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6358143590 msgr2=0x7f63580aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.605+0000 7f635ee00640 1 --2- 172.21.15.19:0/2687005481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6358143590 0x7f63580aed40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f634c0097a0 tx=0x7f634c02a100 comp rx=0 tx=0).stop 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.606+0000 7f635ee00640 1 -- 172.21.15.19:0/2687005481 shutdown_connections 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.606+0000 7f635ee00640 1 --2- 172.21.15.19:0/2687005481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f635806f8f0 0x7f63580b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.606+0000 7f635ee00640 1 --2- 172.21.15.19:0/2687005481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6358143590 0x7f63580aed40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.606+0000 7f635ee00640 1 -- 172.21.15.19:0/2687005481 >> 172.21.15.19:0/2687005481 conn(0x7f63580d1d60 msgr2=0x7f63580d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.606+0000 7f635ee00640 1 -- 172.21.15.19:0/2687005481 shutdown_connections 2024-04-25T11:28:05.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.606+0000 7f635ee00640 1 -- 172.21.15.19:0/2687005481 wait complete. 2024-04-25T11:28:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.608+0000 7f635ee00640 1 Processor -- start 2024-04-25T11:28:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.608+0000 7f635ee00640 1 -- start start 2024-04-25T11:28:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635806f8f0 0x7f6358138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63580b0b90 0x7f63581f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6358143590 0x7f63581f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f635806fd00 con 0x7f63580b0b90 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f63580b0fa0 con 0x7f635806f8f0 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f635806fe80 con 0x7f6358143590 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63580b0b90 0x7f63581f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63580b0b90 0x7f63581f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36376/0 (socket says 172.21.15.19:36376) 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 -- 172.21.15.19:0/1502410892 learned_addr learned my addr 172.21.15.19:0/1502410892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635da00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635806f8f0 0x7f6358138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635e400640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6358143590 0x7f63581f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 -- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6358143590 msgr2=0x7f63581f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6358143590 0x7f63581f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 -- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635806f8f0 msgr2=0x7f6358138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635806f8f0 0x7f6358138e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635d000640 1 -- 172.21.15.19:0/1502410892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6358160420 con 0x7f63580b0b90 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.610+0000 7f635da00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635806f8f0 0x7f6358138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.611+0000 7f635d000640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63580b0b90 0x7f63581f77e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f634800d620 tx=0x7f634800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.611+0000 7f6355600640 1 -- 172.21.15.19:0/1502410892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634800b620 con 0x7f63580b0b90 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.611+0000 7f6355600640 1 -- 172.21.15.19:0/1502410892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6348019070 con 0x7f63580b0b90 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.611+0000 7f6355600640 1 -- 172.21.15.19:0/1502410892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634800ba00 con 0x7f63580b0b90 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.611+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6358160700 con 0x7f63580b0b90 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.611+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6358160cc0 con 0x7f63580b0b90 2024-04-25T11:28:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.612+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6358076c60 con 0x7f63580b0b90 2024-04-25T11:28:05.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.613+0000 7f6355600640 1 -- 172.21.15.19:0/1502410892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f634800bba0 con 0x7f63580b0b90 2024-04-25T11:28:05.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.614+0000 7f6355600640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f633c0744e0 0x7f633c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:05.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.614+0000 7f6355600640 1 -- 172.21.15.19:0/1502410892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5738+0+0 (secure 0 0 0) 0x7f6348095710 con 0x7f63580b0b90 2024-04-25T11:28:05.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.615+0000 7f635da00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f633c0744e0 0x7f633c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:05.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.616+0000 7f635da00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f633c0744e0 0x7f633c0769f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f634c034000 tx=0x7f634c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.620+0000 7f6355600640 1 -- 172.21.15.19:0/1502410892 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f634805fde0 con 0x7f63580b0b90 2024-04-25T11:28:05.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.820+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6358160170 con 0x7f63580b0b90 2024-04-25T11:28:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.821+0000 7f6355600640 1 -- 172.21.15.19:0/1502410892 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f634805f6e0 con 0x7f63580b0b90 2024-04-25T11:28:05.813 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f633c0744e0 msgr2=0x7f633c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f633c0744e0 0x7f633c0769f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f634c034000 tx=0x7f634c02e000 comp rx=0 tx=0).stop 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63580b0b90 msgr2=0x7f63581f77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63580b0b90 0x7f63581f77e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f634800d620 tx=0x7f634800dae0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 shutdown_connections 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f633c0744e0 0x7f633c0769f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6358143590 0x7f63581f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63580b0b90 0x7f63581f77e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 --2- 172.21.15.19:0/1502410892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f635806f8f0 0x7f6358138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.823+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 >> 172.21.15.19:0/1502410892 conn(0x7f63580d1d60 msgr2=0x7f63580b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:05.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.824+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 shutdown_connections 2024-04-25T11:28:05.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:05.824+0000 7f635ee00640 1 -- 172.21.15.19:0/1502410892 wait complete. 2024-04-25T11:28:05.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.137+0000 7f15b8e00640 1 Processor -- start 2024-04-25T11:28:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.137+0000 7f15b8e00640 1 -- start start 2024-04-25T11:28:06.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15b40b4ea0 0x7f15b415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b8e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b40d1230 con 0x7f15b4144370 2024-04-25T11:28:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b8e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f15b40d06b0 con 0x7f15b40b4ea0 2024-04-25T11:28:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15b40d0990 con 0x7f15b406c5c0 2024-04-25T11:28:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15b40b4ea0 0x7f15b415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15b40b4ea0 0x7f15b415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36390/0 (socket says 172.21.15.19:36390) 2024-04-25T11:28:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b3e00640 1 -- 172.21.15.19:0/3852313632 learned_addr learned my addr 172.21.15.19:0/3852313632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.140+0000 7f15b3400640 1 --2- 172.21.15.19:0/3852313632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.141+0000 7f15b3e00640 1 -- 172.21.15.19:0/3852313632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 msgr2=0x7f15b40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.141+0000 7f15b3e00640 1 --2- 172.21.15.19:0/3852313632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.141+0000 7f15b3e00640 1 -- 172.21.15.19:0/3852313632 >> v1:172.21.15.169:6789/0 conn(0x7f15b4144370 legacy=0x7f15b40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.141+0000 7f15b3e00640 1 -- 172.21.15.19:0/3852313632 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b40d0c70 con 0x7f15b40b4ea0 2024-04-25T11:28:06.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.142+0000 7f15b3e00640 1 --2- 172.21.15.19:0/3852313632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b40b4ea0 0x7f15b415e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009e50 tx=0x7f15ac02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7325371553f3ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:06.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.142+0000 7f15b2a00640 1 -- 172.21.15.19:0/3852313632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac036070 con 0x7f15b40b4ea0 2024-04-25T11:28:06.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.142+0000 7f15b2a00640 1 -- 172.21.15.19:0/3852313632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac031600 con 0x7f15b40b4ea0 2024-04-25T11:28:06.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.142+0000 7f15b2a00640 1 -- 172.21.15.19:0/3852313632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0319d0 con 0x7f15b40b4ea0 2024-04-25T11:28:06.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 -- 172.21.15.19:0/3852313632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b40b4ea0 msgr2=0x7f15b415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 --2- 172.21.15.19:0/3852313632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b40b4ea0 0x7f15b415e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009e50 tx=0x7f15ac02a710 comp rx=0 tx=0).stop 2024-04-25T11:28:06.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 -- 172.21.15.19:0/3852313632 shutdown_connections 2024-04-25T11:28:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 --2- 172.21.15.19:0/3852313632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b40b4ea0 0x7f15b415e770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 --2- 172.21.15.19:0/3852313632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 -- 172.21.15.19:0/3852313632 >> 172.21.15.19:0/3852313632 conn(0x7f15b40d1720 msgr2=0x7f15b40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 -- 172.21.15.19:0/3852313632 shutdown_connections 2024-04-25T11:28:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.143+0000 7f15b8e00640 1 -- 172.21.15.19:0/3852313632 wait complete. 2024-04-25T11:28:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.146+0000 7f15b8e00640 1 Processor -- start 2024-04-25T11:28:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.146+0000 7f15b8e00640 1 -- start start 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.147+0000 7f15b8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b41fb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.147+0000 7f15b8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b4144370 0x7f15b41467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.147+0000 7f15b8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b4070bb0 0x7f15b4145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.147+0000 7f15b8e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40d0450 con 0x7f15b4144370 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.147+0000 7f15b8e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40d02d0 con 0x7f15b4070bb0 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.147+0000 7f15b8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15b4147760 con 0x7f15b406c5c0 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.148+0000 7f15abe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b4144370 0x7f15b41467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.148+0000 7f15abe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b4144370 0x7f15b41467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36402/0 (socket says 172.21.15.19:36402) 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.148+0000 7f15abe00640 1 -- 172.21.15.19:0/1652700911 learned_addr learned my addr 172.21.15.19:0/1652700911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.148+0000 7f15b3400640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b41fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.148+0000 7f15b3e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b4070bb0 0x7f15b4145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.148+0000 7f15abe00640 1 -- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 msgr2=0x7f15b41fb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.148+0000 7f15abe00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b41fb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15abe00640 1 -- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b4070bb0 msgr2=0x7f15b4145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15abe00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b4070bb0 0x7f15b4145320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15abe00640 1 -- 172.21.15.19:0/1652700911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b406fb10 con 0x7f15b4144370 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15b3400640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b41fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15b3e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b4070bb0 0x7f15b4145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15abe00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b4144370 0x7f15b41467d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c3f0 tx=0x7f159c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15ab400640 1 -- 172.21.15.19:0/1652700911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c0052d0 con 0x7f15b4144370 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15ab400640 1 -- 172.21.15.19:0/1652700911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c005490 con 0x7f15b4144370 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15ab400640 1 -- 172.21.15.19:0/1652700911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c007260 con 0x7f15b4144370 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b406fdf0 con 0x7f15b4144370 2024-04-25T11:28:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.149+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b40703b0 con 0x7f15b4144370 2024-04-25T11:28:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.151+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b4146d10 con 0x7f15b4144370 2024-04-25T11:28:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.152+0000 7f15ab400640 1 -- 172.21.15.19:0/1652700911 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f159c007400 con 0x7f15b4144370 2024-04-25T11:28:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.153+0000 7f15ab400640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15900744b0 0x7f15900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.153+0000 7f15ab400640 1 -- 172.21.15.19:0/1652700911 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f159c094680 con 0x7f15b4144370 2024-04-25T11:28:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.153+0000 7f15b3400640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15900744b0 0x7f15900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.155+0000 7f15b3400640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15900744b0 0x7f15900769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b9f0 tx=0x7f15a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.158+0000 7f15ab400640 1 -- 172.21.15.19:0/1652700911 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f159c05ede0 con 0x7f15b4144370 2024-04-25T11:28:06.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.353+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15b4050740 con 0x7f15b4144370 2024-04-25T11:28:06.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.356+0000 7f15ab400640 1 -- 172.21.15.19:0/1652700911 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f159c05e6e0 con 0x7f15b4144370 2024-04-25T11:28:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:06.348 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15900744b0 msgr2=0x7f15900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15900744b0 0x7f15900769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b9f0 tx=0x7f15a4008040 comp rx=0 tx=0).stop 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b4144370 msgr2=0x7f15b41467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b4144370 0x7f15b41467d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c3f0 tx=0x7f159c00c7f0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 shutdown_connections 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15900744b0 0x7f15900769c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15b4070bb0 0x7f15b4145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15b4144370 0x7f15b41467d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 --2- 172.21.15.19:0/1652700911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15b406c5c0 0x7f15b41fb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.358+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 >> 172.21.15.19:0/1652700911 conn(0x7f15b40d1720 msgr2=0x7f15b40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.359+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 shutdown_connections 2024-04-25T11:28:06.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.359+0000 7f15b8e00640 1 -- 172.21.15.19:0/1652700911 wait complete. 2024-04-25T11:28:06.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:06.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.663+0000 7fd524200640 1 Processor -- start 2024-04-25T11:28:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.663+0000 7fd524200640 1 -- start start 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.665+0000 7fd524200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd51c070630 0x7fd51c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.665+0000 7fd524200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd51c15adf0 0x7fd51c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.665+0000 7fd524200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51c06df20 con 0x7fd51c06db10 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.665+0000 7fd524200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd51c0713f0 con 0x7fd51c070630 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.665+0000 7fd524200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd51c0701a0 con 0x7fd51c15adf0 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.665+0000 7fd521800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd51c070630 0x7fd51c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd521800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd51c070630 0x7fd51c0cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36404/0 (socket says 172.21.15.19:36404) 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd521800640 1 -- 172.21.15.19:0/963602262 learned_addr learned my addr 172.21.15.19:0/963602262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd522200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd51c06db10 0x7fd51c0b59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40566/0 (socket says 172.21.15.19:40566) 2024-04-25T11:28:06.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd520e00640 1 --2- 172.21.15.19:0/963602262 >> v2:172.21.15.169:3300/0 conn(0x7fd51c15adf0 0x7fd51c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd521800640 1 -- 172.21.15.19:0/963602262 >> v2:172.21.15.169:3300/0 conn(0x7fd51c15adf0 msgr2=0x7fd51c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd521800640 1 --2- 172.21.15.19:0/963602262 >> v2:172.21.15.169:3300/0 conn(0x7fd51c15adf0 0x7fd51c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd521800640 1 -- 172.21.15.19:0/963602262 >> v1:172.21.15.19:6789/0 conn(0x7fd51c06db10 legacy=0x7fd51c0b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd51be00640 1 -- 172.21.15.19:0/963602262 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368010060 0 0) 0x7fd514001490 con 0x7fd51c06db10 2024-04-25T11:28:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.666+0000 7fd521800640 1 -- 172.21.15.19:0/963602262 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c071110 con 0x7fd51c070630 2024-04-25T11:28:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.667+0000 7fd521800640 1 --2- 172.21.15.19:0/963602262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c0cec50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0b0550 tx=0x7fd50c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241653e9db68388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.668+0000 7fd51be00640 1 -- 172.21.15.19:0/963602262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c033640 con 0x7fd51c070630 2024-04-25T11:28:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.668+0000 7fd51be00640 1 -- 172.21.15.19:0/963602262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd50c038070 con 0x7fd51c070630 2024-04-25T11:28:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.668+0000 7fd51be00640 1 -- 172.21.15.19:0/963602262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c033a00 con 0x7fd51c070630 2024-04-25T11:28:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 -- 172.21.15.19:0/963602262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 msgr2=0x7fd51c0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 --2- 172.21.15.19:0/963602262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c0cec50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0b0550 tx=0x7fd50c02c8b0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 -- 172.21.15.19:0/963602262 shutdown_connections 2024-04-25T11:28:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 --2- 172.21.15.19:0/963602262 >> v2:172.21.15.169:3300/0 conn(0x7fd51c15adf0 0x7fd51c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 --2- 172.21.15.19:0/963602262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c0cec50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 -- 172.21.15.19:0/963602262 >> 172.21.15.19:0/963602262 conn(0x7fd51c073090 msgr2=0x7fd51c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 -- 172.21.15.19:0/963602262 shutdown_connections 2024-04-25T11:28:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.669+0000 7fd524200640 1 -- 172.21.15.19:0/963602262 wait complete. 2024-04-25T11:28:06.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.671+0000 7fd524200640 1 Processor -- start 2024-04-25T11:28:06.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.671+0000 7fd524200640 1 -- start start 2024-04-25T11:28:06.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd524200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c06db10 0x7fd51c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd524200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd524200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c15adf0 0x7fd51c15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd524200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c0b0d00 con 0x7fd51c070630 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd524200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c06fe30 con 0x7fd51c15adf0 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd524200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c0b0e80 con 0x7fd51c06db10 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd520e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd520e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c15e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36416/0 (socket says 172.21.15.19:36416) 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.673+0000 7fd520e00640 1 -- 172.21.15.19:0/718793446 learned_addr learned my addr 172.21.15.19:0/718793446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd521800640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c06db10 0x7fd51c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd522200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c15adf0 0x7fd51c15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd520e00640 1 -- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c06db10 msgr2=0x7fd51c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd520e00640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c06db10 0x7fd51c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd520e00640 1 -- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c15adf0 msgr2=0x7fd51c15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd520e00640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c15adf0 0x7fd51c15ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd520e00640 1 -- 172.21.15.19:0/718793446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c155210 con 0x7fd51c070630 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd521800640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c06db10 0x7fd51c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd522200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c15adf0 0x7fd51c15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd520e00640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c15e8c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d5f0 tx=0x7fd50800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd519600640 1 -- 172.21.15.19:0/718793446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080125f0 con 0x7fd51c070630 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd519600640 1 -- 172.21.15.19:0/718793446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd508019070 con 0x7fd51c070630 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd519600640 1 -- 172.21.15.19:0/718793446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080129d0 con 0x7fd51c070630 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.674+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c155720 con 0x7fd51c070630 2024-04-25T11:28:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.675+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c155ce0 con 0x7fd51c070630 2024-04-25T11:28:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.676+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c050740 con 0x7fd51c070630 2024-04-25T11:28:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.677+0000 7fd519600640 1 -- 172.21.15.19:0/718793446 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd508012b70 con 0x7fd51c070630 2024-04-25T11:28:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.678+0000 7fd519600640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4fc074500 0x7fd4fc076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.678+0000 7fd519600640 1 -- 172.21.15.19:0/718793446 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fd5080956c0 con 0x7fd51c070630 2024-04-25T11:28:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.679+0000 7fd521800640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4fc074500 0x7fd4fc076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.680+0000 7fd521800640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4fc074500 0x7fd4fc076a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd50c037000 tx=0x7fd50c032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:06.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.683+0000 7fd519600640 1 -- 172.21.15.19:0/718793446 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd50805fe10 con 0x7fd51c070630 2024-04-25T11:28:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.888+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd51c0b67f0 con 0x7fd51c070630 2024-04-25T11:28:06.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.889+0000 7fd519600640 1 -- 172.21.15.19:0/718793446 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fd50805f710 con 0x7fd51c070630 2024-04-25T11:28:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4fc074500 msgr2=0x7fd4fc076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4fc074500 0x7fd4fc076a10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd50c037000 tx=0x7fd50c032000 comp rx=0 tx=0).stop 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 msgr2=0x7fd51c15e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c15e8c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d5f0 tx=0x7fd50800dab0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 shutdown_connections 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4fc074500 0x7fd4fc076a10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd51c15adf0 0x7fd51c15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd51c070630 0x7fd51c15e8c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 --2- 172.21.15.19:0/718793446 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd51c06db10 0x7fd51c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:06.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 >> 172.21.15.19:0/718793446 conn(0x7fd51c073090 msgr2=0x7fd51c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:06.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 shutdown_connections 2024-04-25T11:28:06.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:06.892+0000 7fd524200640 1 -- 172.21.15.19:0/718793446 wait complete. 2024-04-25T11:28:06.913 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:07.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.214+0000 7fe93b200640 1 Processor -- start 2024-04-25T11:28:07.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.215+0000 7fe93b200640 1 -- start start 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.217+0000 7fe93b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe93c0b60c0 0x7fe93c16f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.217+0000 7fe93b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c14aed0 con 0x7fe93c06d5d0 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.217+0000 7fe93b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c06f460 con 0x7fe93c14aac0 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.217+0000 7fe93b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe93c06f740 con 0x7fe93c0b60c0 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.217+0000 7fe931400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe93c06d5d0 0x7fe93c06d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40568/0 (socket says 172.21.15.19:40568) 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.217+0000 7fe931400640 1 -- 172.21.15.19:0/4039304872 learned_addr learned my addr 172.21.15.19:0/4039304872 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.218+0000 7fe93a800640 1 --2- 172.21.15.19:0/4039304872 >> v2:172.21.15.19:3300/0 conn(0x7fe93c0b60c0 0x7fe93c16f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.218+0000 7fe939400640 1 -- 172.21.15.19:0/4039304872 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483000997 0 0) 0x7fe93c14aed0 con 0x7fe93c06d5d0 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.218+0000 7fe939400640 1 -- 172.21.15.19:0/4039304872 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe920003520 con 0x7fe93c06d5d0 2024-04-25T11:28:07.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.218+0000 7fe93a800640 1 -- 172.21.15.19:0/4039304872 >> v1:172.21.15.169:6789/0 conn(0x7fe93c14aac0 legacy=0x7fe93c1655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.218+0000 7fe93a800640 1 -- 172.21.15.19:0/4039304872 >> v1:172.21.15.19:6789/0 conn(0x7fe93c06d5d0 legacy=0x7fe93c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.218+0000 7fe93a800640 1 -- 172.21.15.19:0/4039304872 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c06fa20 con 0x7fe93c0b60c0 2024-04-25T11:28:07.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.219+0000 7fe93a800640 1 --2- 172.21.15.19:0/4039304872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c0b60c0 0x7fe93c16f4a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe93402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f31b967763ff011 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:07.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.220+0000 7fe939400640 1 -- 172.21.15.19:0/4039304872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934031600 con 0x7fe93c0b60c0 2024-04-25T11:28:07.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.220+0000 7fe939400640 1 -- 172.21.15.19:0/4039304872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9340317a0 con 0x7fe93c0b60c0 2024-04-25T11:28:07.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.220+0000 7fe939400640 1 -- 172.21.15.19:0/4039304872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934031b70 con 0x7fe93c0b60c0 2024-04-25T11:28:07.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.220+0000 7fe93b200640 1 -- 172.21.15.19:0/4039304872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c0b60c0 msgr2=0x7fe93c16f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.220+0000 7fe93b200640 1 --2- 172.21.15.19:0/4039304872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c0b60c0 0x7fe93c16f4a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe934009d90 tx=0x7fe93402a5d0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.220+0000 7fe93b200640 1 -- 172.21.15.19:0/4039304872 shutdown_connections 2024-04-25T11:28:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.220+0000 7fe93b200640 1 --2- 172.21.15.19:0/4039304872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c0b60c0 0x7fe93c16f4a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.221+0000 7fe93b200640 1 -- 172.21.15.19:0/4039304872 >> 172.21.15.19:0/4039304872 conn(0x7fe93c072f40 msgr2=0x7fe93c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.221+0000 7fe93b200640 1 -- 172.21.15.19:0/4039304872 shutdown_connections 2024-04-25T11:28:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.221+0000 7fe93b200640 1 -- 172.21.15.19:0/4039304872 wait complete. 2024-04-25T11:28:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.223+0000 7fe93b200640 1 Processor -- start 2024-04-25T11:28:07.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.223+0000 7fe93b200640 1 -- start start 2024-04-25T11:28:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe93c06d5d0 0x7fe93c147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe93c0b60c0 0x7fe93c1620f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c14aac0 0x7fe93c14c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0b4a90 con 0x7fe93c14aac0 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c06ffe0 con 0x7fe93c0b60c0 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe93c06f0c0 con 0x7fe93c06d5d0 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c14aac0 0x7fe93c14c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c14aac0 0x7fe93c14c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36420/0 (socket says 172.21.15.19:36420) 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 -- 172.21.15.19:0/2066502468 learned_addr learned my addr 172.21.15.19:0/2066502468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe931400640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe93c0b60c0 0x7fe93c1620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe939e00640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe93c06d5d0 0x7fe93c147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 -- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe93c06d5d0 msgr2=0x7fe93c147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe93c06d5d0 0x7fe93c147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 -- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe93c0b60c0 msgr2=0x7fe93c1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe93c0b60c0 0x7fe93c1620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe93a800640 1 -- 172.21.15.19:0/2066502468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c1fafa0 con 0x7fe93c14aac0 2024-04-25T11:28:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.225+0000 7fe939e00640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe93c06d5d0 0x7fe93c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.226+0000 7fe93a800640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c14aac0 0x7fe93c14c7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe934009eb0 tx=0x7fe934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.226+0000 7fe932a00640 1 -- 172.21.15.19:0/2066502468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93403f070 con 0x7fe93c14aac0 2024-04-25T11:28:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.226+0000 7fe932a00640 1 -- 172.21.15.19:0/2066502468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe934030b60 con 0x7fe93c14aac0 2024-04-25T11:28:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.226+0000 7fe932a00640 1 -- 172.21.15.19:0/2066502468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934009450 con 0x7fe93c14aac0 2024-04-25T11:28:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.226+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c167480 con 0x7fe93c14aac0 2024-04-25T11:28:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.226+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c1fb210 con 0x7fe93c14aac0 2024-04-25T11:28:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.228+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93c076c60 con 0x7fe93c14aac0 2024-04-25T11:28:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.229+0000 7fe932a00640 1 -- 172.21.15.19:0/2066502468 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe93403b410 con 0x7fe93c14aac0 2024-04-25T11:28:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.230+0000 7fe932a00640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9180744b0 0x7fe9180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.231+0000 7fe932a00640 1 -- 172.21.15.19:0/2066502468 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fe9340b6180 con 0x7fe93c14aac0 2024-04-25T11:28:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.231+0000 7fe939e00640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9180744b0 0x7fe9180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.233+0000 7fe939e00640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9180744b0 0x7fe9180769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe93c14e3a0 tx=0x7fe92c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.235+0000 7fe932a00640 1 -- 172.21.15.19:0/2066502468 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe934080200 con 0x7fe93c14aac0 2024-04-25T11:28:07.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.437+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe93c167740 con 0x7fe93c14aac0 2024-04-25T11:28:07.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.439+0000 7fe932a00640 1 -- 172.21.15.19:0/2066502468 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fe934084500 con 0x7fe93c14aac0 2024-04-25T11:28:07.431 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9180744b0 msgr2=0x7fe9180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9180744b0 0x7fe9180769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe93c14e3a0 tx=0x7fe92c009040 comp rx=0 tx=0).stop 2024-04-25T11:28:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c14aac0 msgr2=0x7fe93c14c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c14aac0 0x7fe93c14c7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe934009eb0 tx=0x7fe934004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 shutdown_connections 2024-04-25T11:28:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9180744b0 0x7fe9180769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe93c0b60c0 0x7fe93c1620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93c14aac0 0x7fe93c14c7b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 --2- 172.21.15.19:0/2066502468 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe93c06d5d0 0x7fe93c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 >> 172.21.15.19:0/2066502468 conn(0x7fe93c072f40 msgr2=0x7fe93c0b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 shutdown_connections 2024-04-25T11:28:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.442+0000 7fe93b200640 1 -- 172.21.15.19:0/2066502468 wait complete. 2024-04-25T11:28:07.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:07.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.763+0000 7fbf8e800640 1 Processor -- start 2024-04-25T11:28:07.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.763+0000 7fbf8e800640 1 -- start start 2024-04-25T11:28:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.766+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 0x7fbf880cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.766+0000 7fbf8e800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf88071110 con 0x7fbf8815adf0 2024-04-25T11:28:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.766+0000 7fbf8e800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf8806fb50 con 0x7fbf8806dce0 2024-04-25T11:28:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.766+0000 7fbf8e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbf880713f0 con 0x7fbf88070630 2024-04-25T11:28:07.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.766+0000 7fbf86a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbf8815adf0 0x7fbf8806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40576/0 (socket says 172.21.15.19:40576) 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.766+0000 7fbf86a00640 1 -- 172.21.15.19:0/496929630 learned_addr learned my addr 172.21.15.19:0/496929630 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.766+0000 7fbf87400640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 0x7fbf880cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258966204 0 0) 0x7fbf88071110 con 0x7fbf8815adf0 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf6c003520 con 0x7fbf8815adf0 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1276693403 0 0) 0x7fbf6c003520 con 0x7fbf8815adf0 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 msgr2=0x7fbf880cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 0x7fbf880cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 >> v1:172.21.15.169:6789/0 conn(0x7fbf8806dce0 legacy=0x7fbf88160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8806fe30 con 0x7fbf8815adf0 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fbf74001670 con 0x7fbf8815adf0 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 >> v1:172.21.15.19:6789/0 conn(0x7fbf8815adf0 legacy=0x7fbf8806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 0x7fbf6c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf6c004d40 0x7fbf6c0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf880713f0 con 0x7fbf6c003fb0 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c00de60 con 0x7fbf6c004d40 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.767+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c00e060 con 0x7fbf88070630 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf86a00640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf87400640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 0x7fbf6c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf87e00640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf6c004d40 0x7fbf6c0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf86a00640 1 -- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 msgr2=0x7fbf6c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf86a00640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf88070630 0x7fbf6c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf86a00640 1 -- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf6c004d40 msgr2=0x7fbf6c0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf86a00640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf6c004d40 0x7fbf6c0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.768+0000 7fbf86a00640 1 -- 172.21.15.19:0/496929630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c003520 con 0x7fbf6c003fb0 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.769+0000 7fbf86a00640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf6c004800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbf88070a40 tx=0x7fbf6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4f56c87e5a1a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:07.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.769+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74007c80 con 0x7fbf6c003fb0 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.769+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf74005ad0 con 0x7fbf6c003fb0 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.769+0000 7fbf86000640 1 -- 172.21.15.19:0/496929630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74007070 con 0x7fbf6c003fb0 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.770+0000 7fbf8e800640 1 -- 172.21.15.19:0/496929630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 msgr2=0x7fbf6c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.770+0000 7fbf8e800640 1 --2- 172.21.15.19:0/496929630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf6c004800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbf88070a40 tx=0x7fbf6c005830 comp rx=0 tx=0).stop 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.770+0000 7fbf87e00640 1 -- 172.21.15.19:0/496929630 reap_dead start 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.770+0000 7fbf8e800640 1 -- 172.21.15.19:0/496929630 shutdown_connections 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.770+0000 7fbf8e800640 1 -- 172.21.15.19:0/496929630 >> 172.21.15.19:0/496929630 conn(0x7fbf88073090 msgr2=0x7fbf880734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.770+0000 7fbf8e800640 1 -- 172.21.15.19:0/496929630 shutdown_connections 2024-04-25T11:28:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.770+0000 7fbf8e800640 1 -- 172.21.15.19:0/496929630 wait complete. 2024-04-25T11:28:07.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.773+0000 7fbf8e800640 1 Processor -- start 2024-04-25T11:28:07.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.773+0000 7fbf8e800640 1 -- start start 2024-04-25T11:28:07.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf88142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf88152e90 0x7fbf881532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf8806c5c0 0x7fbf8806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf8e800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf880701a0 con 0x7fbf6c003fb0 2024-04-25T11:28:07.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf8e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbf880b0f10 con 0x7fbf88152e90 2024-04-25T11:28:07.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf8e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbf88070c80 con 0x7fbf8806c5c0 2024-04-25T11:28:07.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf87400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf88142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf87400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf88142200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36432/0 (socket says 172.21.15.19:36432) 2024-04-25T11:28:07.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.775+0000 7fbf87400640 1 -- 172.21.15.19:0/1826067238 learned_addr learned my addr 172.21.15.19:0/1826067238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf87e00640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf8806c5c0 0x7fbf8806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf86a00640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf88152e90 0x7fbf881532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf87400640 1 -- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf8806c5c0 msgr2=0x7fbf8806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf87400640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf8806c5c0 0x7fbf8806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf87400640 1 -- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf88152e90 msgr2=0x7fbf881532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf87400640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf88152e90 0x7fbf881532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf87400640 1 -- 172.21.15.19:0/1826067238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8806cf10 con 0x7fbf6c003fb0 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf87400640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf88142200 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf78011600 tx=0x7fbf78011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.776+0000 7fbf7f400640 1 -- 172.21.15.19:0/1826067238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78015870 con 0x7fbf6c003fb0 2024-04-25T11:28:07.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.777+0000 7fbf7f400640 1 -- 172.21.15.19:0/1826067238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf78015a10 con 0x7fbf6c003fb0 2024-04-25T11:28:07.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.777+0000 7fbf7f400640 1 -- 172.21.15.19:0/1826067238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78015e00 con 0x7fbf6c003fb0 2024-04-25T11:28:07.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.777+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf88153c00 con 0x7fbf6c003fb0 2024-04-25T11:28:07.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.777+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf88151e70 con 0x7fbf6c003fb0 2024-04-25T11:28:07.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.778+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf88050740 con 0x7fbf6c003fb0 2024-04-25T11:28:07.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.779+0000 7fbf7f400640 1 -- 172.21.15.19:0/1826067238 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbf7800f450 con 0x7fbf6c003fb0 2024-04-25T11:28:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.780+0000 7fbf7f400640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf680744e0 0x7fbf680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:07.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.780+0000 7fbf7f400640 1 -- 172.21.15.19:0/1826067238 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fbf78095b50 con 0x7fbf6c003fb0 2024-04-25T11:28:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.781+0000 7fbf86a00640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf680744e0 0x7fbf680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.782+0000 7fbf86a00640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf680744e0 0x7fbf680769f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbf74032000 tx=0x7fbf7402d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:07.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.785+0000 7fbf7f400640 1 -- 172.21.15.19:0/1826067238 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf8806cf10 con 0x7fbf6c003fb0 2024-04-25T11:28:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.992+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbf880b30a0 con 0x7fbf6c003fb0 2024-04-25T11:28:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.993+0000 7fbf7f400640 1 -- 172.21.15.19:0/1826067238 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fbf78019030 con 0x7fbf6c003fb0 2024-04-25T11:28:07.985 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.995+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf680744e0 msgr2=0x7fbf680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.995+0000 7fbf8e800640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf680744e0 0x7fbf680769f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbf74032000 tx=0x7fbf7402d000 comp rx=0 tx=0).stop 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.995+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 msgr2=0x7fbf88142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.995+0000 7fbf8e800640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf88142200 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf78011600 tx=0x7fbf78011ac0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 shutdown_connections 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf680744e0 0x7fbf680769f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbf88152e90 0x7fbf881532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbf8806c5c0 0x7fbf8806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 --2- 172.21.15.19:0/1826067238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf6c003fb0 0x7fbf88142200 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 >> 172.21.15.19:0/1826067238 conn(0x7fbf88073090 msgr2=0x7fbf880d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 shutdown_connections 2024-04-25T11:28:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:07.996+0000 7fbf8e800640 1 -- 172.21.15.19:0/1826067238 wait complete. 2024-04-25T11:28:08.014 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:08.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.305+0000 7f3ec6a00640 1 Processor -- start 2024-04-25T11:28:08.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.305+0000 7f3ec6a00640 1 -- start start 2024-04-25T11:28:08.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.307+0000 7f3ec6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ec00b4ff0 0x7f3ec00b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.307+0000 7f3ec6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ec00cf9c0 0x7f3ec0161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.307+0000 7f3ec6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec00d26f0 con 0x7f3ec00d21c0 2024-04-25T11:28:08.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.307+0000 7f3ec6a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec00ce7d0 con 0x7f3ec00cf9c0 2024-04-25T11:28:08.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.307+0000 7f3ec6a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec00ceab0 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.308+0000 7f3ec6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ec00cf9c0 0x7f3ec0161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.308+0000 7f3ec5600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ec00b4ff0 0x7f3ec00b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.308+0000 7f3ec6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ec00cf9c0 0x7f3ec0161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36448/0 (socket says 172.21.15.19:36448) 2024-04-25T11:28:08.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.308+0000 7f3ec6000640 1 -- 172.21.15.19:0/706335635 learned_addr learned my addr 172.21.15.19:0/706335635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:08.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.309+0000 7f3ec6000640 1 -- 172.21.15.19:0/706335635 >> v2:172.21.15.169:3300/0 conn(0x7f3ec00b4ff0 msgr2=0x7f3ec00b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:08.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.309+0000 7f3ec6000640 1 --2- 172.21.15.19:0/706335635 >> v2:172.21.15.169:3300/0 conn(0x7f3ec00b4ff0 0x7f3ec00b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.309+0000 7f3ec6000640 1 -- 172.21.15.19:0/706335635 >> v1:172.21.15.169:6789/0 conn(0x7f3ec00d21c0 legacy=0x7f3ec00cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.309+0000 7f3ec6000640 1 -- 172.21.15.19:0/706335635 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec00ced90 con 0x7f3ec00cf9c0 2024-04-25T11:28:08.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.310+0000 7f3ec6000640 1 --2- 172.21.15.19:0/706335635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00cf9c0 0x7f3ec0161bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ec006d5a0 tx=0x7f3eb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f334535f13cd8210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:08.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ebfe00640 1 -- 172.21.15.19:0/706335635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8037070 con 0x7f3ec00cf9c0 2024-04-25T11:28:08.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ebfe00640 1 -- 172.21.15.19:0/706335635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb8032440 con 0x7f3ec00cf9c0 2024-04-25T11:28:08.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ebfe00640 1 -- 172.21.15.19:0/706335635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8032810 con 0x7f3ec00cf9c0 2024-04-25T11:28:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ec6a00640 1 -- 172.21.15.19:0/706335635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00cf9c0 msgr2=0x7f3ec0161bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/706335635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00cf9c0 0x7f3ec0161bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ec006d5a0 tx=0x7f3eb802a270 comp rx=0 tx=0).stop 2024-04-25T11:28:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ec6a00640 1 -- 172.21.15.19:0/706335635 shutdown_connections 2024-04-25T11:28:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/706335635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00cf9c0 0x7f3ec0161bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/706335635 >> v2:172.21.15.169:3300/0 conn(0x7f3ec00b4ff0 0x7f3ec00b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.311+0000 7f3ec6a00640 1 -- 172.21.15.19:0/706335635 >> 172.21.15.19:0/706335635 conn(0x7f3ec00b5460 msgr2=0x7f3ec00b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.312+0000 7f3ec6a00640 1 -- 172.21.15.19:0/706335635 shutdown_connections 2024-04-25T11:28:08.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.312+0000 7f3ec6a00640 1 -- 172.21.15.19:0/706335635 wait complete. 2024-04-25T11:28:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.314+0000 7f3ec6a00640 1 Processor -- start 2024-04-25T11:28:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.314+0000 7f3ec6a00640 1 -- start start 2024-04-25T11:28:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00b4ff0 0x7f3ec015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec00cf9c0 0x7f3ec015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec00d21c0 0x7f3ec01fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec6a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec00d2b50 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec00d29d0 con 0x7f3ec00d21c0 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec00b6d00 con 0x7f3ec00cf9c0 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00b4ff0 0x7f3ec015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00b4ff0 0x7f3ec015ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36462/0 (socket says 172.21.15.19:36462) 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.316+0000 7f3ec5600640 1 -- 172.21.15.19:0/2709180534 learned_addr learned my addr 172.21.15.19:0/2709180534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec6000640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec00d21c0 0x7f3ec01fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec4c00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec00cf9c0 0x7f3ec015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec5600640 1 -- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec00cf9c0 msgr2=0x7f3ec015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec5600640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec00cf9c0 0x7f3ec015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec5600640 1 -- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec00d21c0 msgr2=0x7f3ec01fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec5600640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec00d21c0 0x7f3ec01fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec5600640 1 -- 172.21.15.19:0/2709180534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec009d5d0 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec4c00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec00cf9c0 0x7f3ec015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec5600640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00b4ff0 0x7f3ec015ebd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e840 tx=0x7f3eb000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ebd600640 1 -- 172.21.15.19:0/2709180534 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0017070 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ebd600640 1 -- 172.21.15.19:0/2709180534 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb000c940 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec009da40 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ebd600640 1 -- 172.21.15.19:0/2709180534 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000cd10 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.317+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec00a00f0 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.319+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec00cde10 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.320+0000 7f3ebd600640 1 -- 172.21.15.19:0/2709180534 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3eb001a070 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.321+0000 7f3ebd600640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ea40744e0 0x7f3ea40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.321+0000 7f3ebd600640 1 -- 172.21.15.19:0/2709180534 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f3eb00955c0 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.322+0000 7f3ec4c00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ea40744e0 0x7f3ea40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.323+0000 7f3ec4c00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ea40744e0 0x7f3ea40769f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00c6a40 tx=0x7f3eb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.326+0000 7f3ebd600640 1 -- 172.21.15.19:0/2709180534 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ec00cde10 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.532+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ec0050740 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.534+0000 7f3ebd600640 1 -- 172.21.15.19:0/2709180534 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f3eb005fd20 con 0x7f3ec00b4ff0 2024-04-25T11:28:08.527 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:08.527 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:08.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ea40744e0 msgr2=0x7f3ea40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ea40744e0 0x7f3ea40769f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00c6a40 tx=0x7f3eb4009040 comp rx=0 tx=0).stop 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00b4ff0 msgr2=0x7f3ec015ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00b4ff0 0x7f3ec015ebd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e840 tx=0x7f3eb000ed00 comp rx=0 tx=0).stop 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 shutdown_connections 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ea40744e0 0x7f3ea40769f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ec00d21c0 0x7f3ec01fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ec00cf9c0 0x7f3ec015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 --2- 172.21.15.19:0/2709180534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec00b4ff0 0x7f3ec015ebd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.537+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 >> 172.21.15.19:0/2709180534 conn(0x7f3ec00b5460 msgr2=0x7f3ec00c94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.538+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 shutdown_connections 2024-04-25T11:28:08.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.538+0000 7f3ec6a00640 1 -- 172.21.15.19:0/2709180534 wait complete. 2024-04-25T11:28:08.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:08.555 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.847+0000 7f1a52c00640 1 Processor -- start 2024-04-25T11:28:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.848+0000 7f1a52c00640 1 -- start start 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.850+0000 7f1a52c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c0cd9a0 0x7f1a4c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.850+0000 7f1a52c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a4c0b0af0 0x7f1a4c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.850+0000 7f1a52c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c06df20 con 0x7f1a4c06db10 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.850+0000 7f1a52c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1a4c0ceab0 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.850+0000 7f1a52c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c0cdea0 con 0x7f1a4c0cd9a0 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.851+0000 7f1a4b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a4c0b0af0 0x7f1a4c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.851+0000 7f1a4b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a4c0b0af0 0x7f1a4c06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36464/0 (socket says 172.21.15.19:36464) 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.851+0000 7f1a4b400640 1 -- 172.21.15.19:0/3310234527 learned_addr learned my addr 172.21.15.19:0/3310234527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.851+0000 7f1a4be00640 1 --2- 172.21.15.19:0/3310234527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c0cd9a0 0x7f1a4c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.851+0000 7f1a4b400640 1 -- 172.21.15.19:0/3310234527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c0cd9a0 msgr2=0x7f1a4c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.851+0000 7f1a4b400640 1 --2- 172.21.15.19:0/3310234527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c0cd9a0 0x7f1a4c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.851+0000 7f1a4b400640 1 -- 172.21.15.19:0/3310234527 >> v1:172.21.15.169:6789/0 conn(0x7f1a4c06db10 legacy=0x7f1a4c161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.852+0000 7f1a4b400640 1 -- 172.21.15.19:0/3310234527 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c0ce180 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.853+0000 7f1a4b400640 1 --2- 172.21.15.19:0/3310234527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c06d5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d1b516b24a7e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.853+0000 7f1a4aa00640 1 -- 172.21.15.19:0/3310234527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38037070 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.853+0000 7f1a4aa00640 1 -- 172.21.15.19:0/3310234527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a38032440 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.853+0000 7f1a4aa00640 1 -- 172.21.15.19:0/3310234527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38032810 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 -- 172.21.15.19:0/3310234527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 msgr2=0x7f1a4c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 --2- 172.21.15.19:0/3310234527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c06d5d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a3802a270 comp rx=0 tx=0).stop 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 -- 172.21.15.19:0/3310234527 shutdown_connections 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 --2- 172.21.15.19:0/3310234527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c06d5d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 --2- 172.21.15.19:0/3310234527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c0cd9a0 0x7f1a4c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 -- 172.21.15.19:0/3310234527 >> 172.21.15.19:0/3310234527 conn(0x7f1a4c073090 msgr2=0x7f1a4c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 -- 172.21.15.19:0/3310234527 shutdown_connections 2024-04-25T11:28:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.854+0000 7f1a52c00640 1 -- 172.21.15.19:0/3310234527 wait complete. 2024-04-25T11:28:08.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.857+0000 7f1a52c00640 1 Processor -- start 2024-04-25T11:28:08.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.857+0000 7f1a52c00640 1 -- start start 2024-04-25T11:28:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a52c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c06db10 0x7f1a4c146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a52c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c1f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a52c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4c0cd9a0 0x7f1a4c0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a52c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0b0f00 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a52c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0ce740 con 0x7f1a4c0cd9a0 2024-04-25T11:28:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a52c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c0cef40 con 0x7f1a4c06db10 2024-04-25T11:28:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c1f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c1f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36474/0 (socket says 172.21.15.19:36474) 2024-04-25T11:28:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a4b400640 1 -- 172.21.15.19:0/580069353 learned_addr learned my addr 172.21.15.19:0/580069353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a4be00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c06db10 0x7f1a4c146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.859+0000 7f1a50c00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4c0cd9a0 0x7f1a4c0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a4b400640 1 -- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c06db10 msgr2=0x7f1a4c146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a4b400640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c06db10 0x7f1a4c146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a4b400640 1 -- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4c0cd9a0 msgr2=0x7f1a4c0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a4b400640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4c0cd9a0 0x7f1a4c0b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a4b400640 1 -- 172.21.15.19:0/580069353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c1f9db0 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a4b400640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c1f9450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1a38000c00 tx=0x7f1a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a43e00640 1 -- 172.21.15.19:0/580069353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38037040 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a43e00640 1 -- 172.21.15.19:0/580069353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a38042070 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a43e00640 1 -- 172.21.15.19:0/580069353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38030880 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.860+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c1fa090 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.861+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c157b90 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.862+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c050740 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.863+0000 7f1a43e00640 1 -- 172.21.15.19:0/580069353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a38030a20 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.864+0000 7f1a43e00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a2c0744e0 0x7f1a2c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.864+0000 7f1a43e00640 1 -- 172.21.15.19:0/580069353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f1a380b5ae0 con 0x7f1a4c0b0af0 2024-04-25T11:28:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.865+0000 7f1a4be00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a2c0744e0 0x7f1a2c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.866+0000 7f1a4be00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a2c0744e0 0x7f1a2c0769f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0073f0 tx=0x7f1a3c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:08.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:08.869+0000 7f1a43e00640 1 -- 172.21.15.19:0/580069353 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a38080240 con 0x7f1a4c0b0af0 2024-04-25T11:28:09.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.081+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a4c155530 con 0x7f1a4c0b0af0 2024-04-25T11:28:09.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.082+0000 7f1a43e00640 1 -- 172.21.15.19:0/580069353 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f1a38088230 con 0x7f1a4c0b0af0 2024-04-25T11:28:09.074 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:09.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a2c0744e0 msgr2=0x7f1a2c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a2c0744e0 0x7f1a2c0769f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0073f0 tx=0x7f1a3c00b040 comp rx=0 tx=0).stop 2024-04-25T11:28:09.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 msgr2=0x7f1a4c1f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c1f9450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1a38000c00 tx=0x7f1a380046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 shutdown_connections 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a2c0744e0 0x7f1a2c0769f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4c0cd9a0 0x7f1a4c0b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a4c0b0af0 0x7f1a4c1f9450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 --2- 172.21.15.19:0/580069353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4c06db10 0x7f1a4c146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 >> 172.21.15.19:0/580069353 conn(0x7f1a4c073090 msgr2=0x7f1a4c161dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 shutdown_connections 2024-04-25T11:28:09.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.085+0000 7f1a52c00640 1 -- 172.21.15.19:0/580069353 wait complete. 2024-04-25T11:28:09.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.402+0000 7f6d54400640 1 Processor -- start 2024-04-25T11:28:09.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.403+0000 7f6d54400640 1 -- start start 2024-04-25T11:28:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.405+0000 7f6d54400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d4c070630 0x7f6d4c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.406+0000 7f6d54400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d4c071110 con 0x7f6d4c06dce0 2024-04-25T11:28:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.406+0000 7f6d54400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d4c06fb50 con 0x7f6d4c15adf0 2024-04-25T11:28:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.406+0000 7f6d54400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4c0713f0 con 0x7f6d4c070630 2024-04-25T11:28:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.406+0000 7f6d52400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6d4c06dce0 0x7f6d4c160d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40586/0 (socket says 172.21.15.19:40586) 2024-04-25T11:28:09.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.406+0000 7f6d52400640 1 -- 172.21.15.19:0/2600413996 learned_addr learned my addr 172.21.15.19:0/2600413996 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.406+0000 7f6d51a00640 1 --2- 172.21.15.19:0/2600413996 >> v2:172.21.15.169:3300/0 conn(0x7f6d4c070630 0x7f6d4c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.406+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715710535 0 0) 0x7f6d4c071110 con 0x7f6d4c06dce0 2024-04-25T11:28:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d34005730 con 0x7f6d4c06dce0 2024-04-25T11:28:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1644130747 0 0) 0x7f6d34005730 con 0x7f6d4c06dce0 2024-04-25T11:28:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 >> v2:172.21.15.169:3300/0 conn(0x7f6d4c070630 msgr2=0x7f6d4c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:09.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 --2- 172.21.15.19:0/2600413996 >> v2:172.21.15.169:3300/0 conn(0x7f6d4c070630 0x7f6d4c0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 >> v1:172.21.15.169:6789/0 conn(0x7f6d4c15adf0 legacy=0x7f6d4c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c06fe30 con 0x7f6d4c06dce0 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f6d44001670 con 0x7f6d4c06dce0 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 >> v1:172.21.15.19:6789/0 conn(0x7f6d4c06dce0 legacy=0x7f6d4c160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d34005bc0 0x7f6d34006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d34006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d34007460 0x7f6d3400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c0713f0 con 0x7f6d34006a10 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3400e3f0 con 0x7f6d34005bc0 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3400e5f0 con 0x7f6d34007460 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.407+0000 7f6d51000640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d34006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d51a00640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d34005bc0 0x7f6d34006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d51000640 1 -- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d34007460 msgr2=0x7f6d3400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d51000640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d34007460 0x7f6d3400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d51000640 1 -- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d34005bc0 msgr2=0x7f6d34006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:09.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d51000640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d34005bc0 0x7f6d34006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d51000640 1 -- 172.21.15.19:0/2600413996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d34005730 con 0x7f6d34006a10 2024-04-25T11:28:09.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d51a00640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d34005bc0 0x7f6d34006030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:09.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.408+0000 7f6d52400640 1 -- 172.21.15.19:0/2600413996 reap_dead start 2024-04-25T11:28:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.409+0000 7f6d51000640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d34006e20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d38001750 tx=0x7f6d34007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcf7756d40cc193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.409+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38001370 con 0x7f6d34006a10 2024-04-25T11:28:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.409+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d380052d0 con 0x7f6d34006a10 2024-04-25T11:28:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.409+0000 7f6d4be00640 1 -- 172.21.15.19:0/2600413996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3802abc0 con 0x7f6d34006a10 2024-04-25T11:28:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.410+0000 7f6d54400640 1 -- 172.21.15.19:0/2600413996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 msgr2=0x7f6d34006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.410+0000 7f6d54400640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d34006e20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d38001750 tx=0x7f6d34007a40 comp rx=0 tx=0).stop 2024-04-25T11:28:09.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.410+0000 7f6d54400640 1 -- 172.21.15.19:0/2600413996 shutdown_connections 2024-04-25T11:28:09.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.410+0000 7f6d54400640 1 --2- 172.21.15.19:0/2600413996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d34006e20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.410+0000 7f6d54400640 1 -- 172.21.15.19:0/2600413996 >> 172.21.15.19:0/2600413996 conn(0x7f6d4c073090 msgr2=0x7f6d4c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:09.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.410+0000 7f6d54400640 1 -- 172.21.15.19:0/2600413996 shutdown_connections 2024-04-25T11:28:09.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.410+0000 7f6d54400640 1 -- 172.21.15.19:0/2600413996 wait complete. 2024-04-25T11:28:09.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.413+0000 7f6d54400640 1 Processor -- start 2024-04-25T11:28:09.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.413+0000 7f6d54400640 1 -- start start 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d54400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d4c15dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d54400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d4c1437b0 0x7f6d4c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d54400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d4c06d110 0x7f6d4c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d54400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c0701a0 con 0x7f6d34006a10 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d54400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c0b0f10 con 0x7f6d4c06d110 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d54400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4c070c80 con 0x7f6d4c1437b0 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d4c15dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d4c15dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36504/0 (socket says 172.21.15.19:36504) 2024-04-25T11:28:09.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51a00640 1 -- 172.21.15.19:0/2083385780 learned_addr learned my addr 172.21.15.19:0/2083385780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51a00640 1 -- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d4c1437b0 msgr2=0x7f6d4c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d52400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d4c06d110 0x7f6d4c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51000640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d4c1437b0 0x7f6d4c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51a00640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d4c1437b0 0x7f6d4c06cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51a00640 1 -- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d4c06d110 msgr2=0x7f6d4c0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.415+0000 7f6d51a00640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d4c06d110 0x7f6d4c0ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d51a00640 1 -- 172.21.15.19:0/2083385780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c06d520 con 0x7f6d34006a10 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d52400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d4c06d110 0x7f6d4c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d51a00640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d4c15dd60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000f380 tx=0x7f6d4000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:09.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d49600640 1 -- 172.21.15.19:0/2083385780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40016620 con 0x7f6d34006a10 2024-04-25T11:28:09.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d49600640 1 -- 172.21.15.19:0/2083385780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4001b070 con 0x7f6d34006a10 2024-04-25T11:28:09.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d49600640 1 -- 172.21.15.19:0/2083385780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d400169c0 con 0x7f6d34006a10 2024-04-25T11:28:09.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c160080 con 0x7f6d34006a10 2024-04-25T11:28:09.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.416+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c160640 con 0x7f6d34006a10 2024-04-25T11:28:09.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.418+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4c050740 con 0x7f6d34006a10 2024-04-25T11:28:09.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.419+0000 7f6d49600640 1 -- 172.21.15.19:0/2083385780 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6d40005d10 con 0x7f6d34006a10 2024-04-25T11:28:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.420+0000 7f6d49600640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6d2c0744b0 0x7f6d2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.420+0000 7f6d49600640 1 -- 172.21.15.19:0/2083385780 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f6d40095f50 con 0x7f6d34006a10 2024-04-25T11:28:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.421+0000 7f6d51000640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6d2c0744b0 0x7f6d2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.422+0000 7f6d51000640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6d2c0744b0 0x7f6d2c0769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6d38032000 tx=0x7f6d3802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.425+0000 7f6d49600640 1 -- 172.21.15.19:0/2083385780 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d40025020 con 0x7f6d34006a10 2024-04-25T11:28:09.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.628+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6d4c154800 con 0x7f6d34006a10 2024-04-25T11:28:09.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.629+0000 7f6d49600640 1 -- 172.21.15.19:0/2083385780 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f6d400606b0 con 0x7f6d34006a10 2024-04-25T11:28:09.621 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.631+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6d2c0744b0 msgr2=0x7f6d2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.631+0000 7f6d54400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6d2c0744b0 0x7f6d2c0769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6d38032000 tx=0x7f6d3802e000 comp rx=0 tx=0).stop 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.631+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 msgr2=0x7f6d4c15dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.631+0000 7f6d54400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d4c15dd60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000f380 tx=0x7f6d4000f840 comp rx=0 tx=0).stop 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 shutdown_connections 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6d2c0744b0 0x7f6d2c0769c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d4c06d110 0x7f6d4c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d4c1437b0 0x7f6d4c06cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 --2- 172.21.15.19:0/2083385780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d34006a10 0x7f6d4c15dd60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 >> 172.21.15.19:0/2083385780 conn(0x7f6d4c073090 msgr2=0x7f6d4c0ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 shutdown_connections 2024-04-25T11:28:09.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.632+0000 7f6d54400640 1 -- 172.21.15.19:0/2083385780 wait complete. 2024-04-25T11:28:09.649 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.945+0000 7fd7aac00640 1 Processor -- start 2024-04-25T11:28:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.945+0000 7fd7aac00640 1 -- start start 2024-04-25T11:28:09.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.947+0000 7fd7aac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd7a41601e0 0x7fd7a40b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.947+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.947+0000 7fd7aac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a40d3240 con 0x7fd7a40b8150 2024-04-25T11:28:09.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.947+0000 7fd7aac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a40b4e90 con 0x7fd7a41601e0 2024-04-25T11:28:09.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.947+0000 7fd7aac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a40d3e10 con 0x7fd7a40d28f0 2024-04-25T11:28:09.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.948+0000 7fd7a9800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd7a41601e0 0x7fd7a40b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.948+0000 7fd7a9800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd7a41601e0 0x7fd7a40b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36506/0 (socket says 172.21.15.19:36506) 2024-04-25T11:28:09.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.948+0000 7fd7a9800640 1 -- 172.21.15.19:0/2380172893 learned_addr learned my addr 172.21.15.19:0/2380172893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:09.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.948+0000 7fd7aa200640 1 --2- 172.21.15.19:0/2380172893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.948+0000 7fd7a9800640 1 -- 172.21.15.19:0/2380172893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 msgr2=0x7fd7a40d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:09.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.948+0000 7fd7a9800640 1 --2- 172.21.15.19:0/2380172893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.949+0000 7fd7a9800640 1 -- 172.21.15.19:0/2380172893 >> v1:172.21.15.19:6789/0 conn(0x7fd7a40b8150 legacy=0x7fd7a40b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.949+0000 7fd7a9800640 1 -- 172.21.15.19:0/2380172893 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a40d40f0 con 0x7fd7a41601e0 2024-04-25T11:28:09.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.948+0000 7fd7a3e00640 1 -- 172.21.15.19:0/2380172893 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101509714 0 0) 0x7fd790001110 con 0x7fd7a40b8150 2024-04-25T11:28:09.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.949+0000 7fd7aa200640 1 --2- 172.21.15.19:0/2380172893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.950+0000 7fd7a9800640 1 --2- 172.21.15.19:0/2380172893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a41601e0 0x7fd7a40b4660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd79802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabce72ba4ea2a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.950+0000 7fd7a3e00640 1 -- 172.21.15.19:0/2380172893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798037070 con 0x7fd7a41601e0 2024-04-25T11:28:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.950+0000 7fd7a3e00640 1 -- 172.21.15.19:0/2380172893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798032440 con 0x7fd7a41601e0 2024-04-25T11:28:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.950+0000 7fd7a3e00640 1 -- 172.21.15.19:0/2380172893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798032810 con 0x7fd7a41601e0 2024-04-25T11:28:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 -- 172.21.15.19:0/2380172893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a41601e0 msgr2=0x7fd7a40b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 --2- 172.21.15.19:0/2380172893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a41601e0 0x7fd7a40b4660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd79802a250 comp rx=0 tx=0).stop 2024-04-25T11:28:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 -- 172.21.15.19:0/2380172893 shutdown_connections 2024-04-25T11:28:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 --2- 172.21.15.19:0/2380172893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 --2- 172.21.15.19:0/2380172893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a41601e0 0x7fd7a40b4660 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 -- 172.21.15.19:0/2380172893 >> 172.21.15.19:0/2380172893 conn(0x7fd7a406dfe0 msgr2=0x7fd7a406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 -- 172.21.15.19:0/2380172893 shutdown_connections 2024-04-25T11:28:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.951+0000 7fd7aac00640 1 -- 172.21.15.19:0/2380172893 wait complete. 2024-04-25T11:28:09.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.953+0000 7fd7aac00640 1 Processor -- start 2024-04-25T11:28:09.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.954+0000 7fd7aac00640 1 -- start start 2024-04-25T11:28:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.955+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a40b8150 0x7fd7a40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.955+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.955+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a41601e0 0x7fd7a4154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.955+0000 7fd7aac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a40b4ba0 con 0x7fd7a40b8150 2024-04-25T11:28:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.955+0000 7fd7aac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a40d46b0 con 0x7fd7a41601e0 2024-04-25T11:28:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.955+0000 7fd7aac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a40b9890 con 0x7fd7a40d28f0 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.955+0000 7fd7a9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a40b8150 0x7fd7a40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a40b8150 0x7fd7a40ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36520/0 (socket says 172.21.15.19:36520) 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 -- 172.21.15.19:0/298706663 learned_addr learned my addr 172.21.15.19:0/298706663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a8e00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7aa200640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a41601e0 0x7fd7a4154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 -- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 msgr2=0x7fd7a40b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40b62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 -- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a41601e0 msgr2=0x7fd7a4154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a41601e0 0x7fd7a4154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 -- 172.21.15.19:0/298706663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a40cce30 con 0x7fd7a40b8150 2024-04-25T11:28:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.956+0000 7fd7a9800640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a40b8150 0x7fd7a40ce410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd7980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.957+0000 7fd7a1600640 1 -- 172.21.15.19:0/298706663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798037040 con 0x7fd7a40b8150 2024-04-25T11:28:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.957+0000 7fd7a1600640 1 -- 172.21.15.19:0/298706663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798004230 con 0x7fd7a40b8150 2024-04-25T11:28:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.957+0000 7fd7a1600640 1 -- 172.21.15.19:0/298706663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798003610 con 0x7fd7a40b8150 2024-04-25T11:28:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.957+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a40cd110 con 0x7fd7a40b8150 2024-04-25T11:28:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.957+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a40cdbf0 con 0x7fd7a40b8150 2024-04-25T11:28:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.958+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a4076c60 con 0x7fd7a40b8150 2024-04-25T11:28:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.960+0000 7fd7a1600640 1 -- 172.21.15.19:0/298706663 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd7980037b0 con 0x7fd7a40b8150 2024-04-25T11:28:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.960+0000 7fd7a1600640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7840744b0 0x7fd7840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.961+0000 7fd7a1600640 1 -- 172.21.15.19:0/298706663 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fd7980b6120 con 0x7fd7a40b8150 2024-04-25T11:28:09.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.961+0000 7fd7a8e00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7840744b0 0x7fd7840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:09.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.963+0000 7fd7a8e00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7840744b0 0x7fd7840769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7a415f740 tx=0x7fd790009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:09.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:09.965+0000 7fd7a1600640 1 -- 172.21.15.19:0/298706663 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd798080890 con 0x7fd7a40b8150 2024-04-25T11:28:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.166+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7a40ccb60 con 0x7fd7a40b8150 2024-04-25T11:28:10.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.167+0000 7fd7a1600640 1 -- 172.21.15.19:0/298706663 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fd798080190 con 0x7fd7a40b8150 2024-04-25T11:28:10.159 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7840744b0 msgr2=0x7fd7840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7840744b0 0x7fd7840769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7a415f740 tx=0x7fd790009040 comp rx=0 tx=0).stop 2024-04-25T11:28:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a40b8150 msgr2=0x7fd7a40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a40b8150 0x7fd7a40ce410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd798009870 tx=0x7fd7980046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 shutdown_connections 2024-04-25T11:28:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7840744b0 0x7fd7840769c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7a41601e0 0x7fd7a4154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7a40d28f0 0x7fd7a40b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 --2- 172.21.15.19:0/298706663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7a40b8150 0x7fd7a40ce410 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 >> 172.21.15.19:0/298706663 conn(0x7fd7a406dfe0 msgr2=0x7fd7a40b9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 shutdown_connections 2024-04-25T11:28:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.170+0000 7fd7aac00640 1 -- 172.21.15.19:0/298706663 wait complete. 2024-04-25T11:28:10.211 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2024-04-25T11:28:10.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:10.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.501+0000 7f9344000640 1 Processor -- start 2024-04-25T11:28:10.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.501+0000 7f9344000640 1 -- start start 2024-04-25T11:28:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.503+0000 7f9344000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f933c1434d0 0x7f933c070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.503+0000 7f9344000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c0d0ed0 0x7f933c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.503+0000 7f9344000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f933c0709c0 con 0x7f933c160be0 2024-04-25T11:28:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.503+0000 7f9344000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f933c1438e0 con 0x7f933c1434d0 2024-04-25T11:28:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.503+0000 7f9344000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f933c06f3e0 con 0x7f933c0d0ed0 2024-04-25T11:28:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9341600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f933c1434d0 0x7f933c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9342000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f933c160be0 0x7f933c160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40608/0 (socket says 172.21.15.19:40608) 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9342000640 1 -- 172.21.15.19:0/864592319 learned_addr learned my addr 172.21.15.19:0/864592319 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9340c00640 1 --2- 172.21.15.19:0/864592319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c0d0ed0 0x7f933c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9341600640 1 -- 172.21.15.19:0/864592319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c0d0ed0 msgr2=0x7f933c0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9341600640 1 --2- 172.21.15.19:0/864592319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c0d0ed0 0x7f933c0d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9341600640 1 -- 172.21.15.19:0/864592319 >> v1:172.21.15.19:6789/0 conn(0x7f933c160be0 legacy=0x7f933c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f9341600640 1 -- 172.21.15.19:0/864592319 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c06f6c0 con 0x7f933c1434d0 2024-04-25T11:28:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.504+0000 7f933be00640 1 -- 172.21.15.19:0/864592319 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1533209383 0 0) 0x7f933c0709c0 con 0x7f933c160be0 2024-04-25T11:28:10.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.505+0000 7f9341600640 1 --2- 172.21.15.19:0/864592319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c070480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93300097a0 tx=0x7f933002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74705bc3a77150b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.506+0000 7f933be00640 1 -- 172.21.15.19:0/864592319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330037070 con 0x7f933c1434d0 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.506+0000 7f933be00640 1 -- 172.21.15.19:0/864592319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330032440 con 0x7f933c1434d0 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.506+0000 7f933be00640 1 -- 172.21.15.19:0/864592319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330032810 con 0x7f933c1434d0 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.506+0000 7f9344000640 1 -- 172.21.15.19:0/864592319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 msgr2=0x7f933c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.506+0000 7f9344000640 1 --2- 172.21.15.19:0/864592319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c070480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93300097a0 tx=0x7f933002a180 comp rx=0 tx=0).stop 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.506+0000 7f9344000640 1 -- 172.21.15.19:0/864592319 shutdown_connections 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.506+0000 7f9344000640 1 --2- 172.21.15.19:0/864592319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c0d0ed0 0x7f933c0d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.507+0000 7f9344000640 1 --2- 172.21.15.19:0/864592319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c070480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.507+0000 7f9344000640 1 -- 172.21.15.19:0/864592319 >> 172.21.15.19:0/864592319 conn(0x7f933c0d1ae0 msgr2=0x7f933c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.507+0000 7f9344000640 1 -- 172.21.15.19:0/864592319 shutdown_connections 2024-04-25T11:28:10.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.507+0000 7f9344000640 1 -- 172.21.15.19:0/864592319 wait complete. 2024-04-25T11:28:10.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.509+0000 7f9344000640 1 Processor -- start 2024-04-25T11:28:10.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.509+0000 7f9344000640 1 -- start start 2024-04-25T11:28:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.510+0000 7f9344000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c0d0ed0 0x7f933c0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.510+0000 7f9344000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.510+0000 7f9344000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c160be0 0x7f933c1386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.510+0000 7f9344000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f933c06f090 con 0x7f933c1434d0 2024-04-25T11:28:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.510+0000 7f9344000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f933c06fc80 con 0x7f933c0d0ed0 2024-04-25T11:28:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.510+0000 7f9344000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f933c0b51a0 con 0x7f933c160be0 2024-04-25T11:28:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36536/0 (socket says 172.21.15.19:36536) 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 -- 172.21.15.19:0/401303365 learned_addr learned my addr 172.21.15.19:0/401303365 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 -- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c160be0 msgr2=0x7f933c1386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c160be0 0x7f933c1386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 -- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c0d0ed0 msgr2=0x7f933c0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c0d0ed0 0x7f933c0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 -- 172.21.15.19:0/401303365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c139260 con 0x7f933c1434d0 2024-04-25T11:28:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.511+0000 7f9340c00640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c144120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f932800d5c0 tx=0x7f932800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.512+0000 7f9339600640 1 -- 172.21.15.19:0/401303365 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280125f0 con 0x7f933c1434d0 2024-04-25T11:28:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.512+0000 7f9339600640 1 -- 172.21.15.19:0/401303365 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9328019070 con 0x7f933c1434d0 2024-04-25T11:28:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.512+0000 7f9339600640 1 -- 172.21.15.19:0/401303365 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280129d0 con 0x7f933c1434d0 2024-04-25T11:28:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.512+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933c139540 con 0x7f933c1434d0 2024-04-25T11:28:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.512+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933c13bfb0 con 0x7f933c1434d0 2024-04-25T11:28:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.513+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f933c050740 con 0x7f933c1434d0 2024-04-25T11:28:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.515+0000 7f9339600640 1 -- 172.21.15.19:0/401303365 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9328012b70 con 0x7f933c1434d0 2024-04-25T11:28:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.516+0000 7f9339600640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f931c0744e0 0x7f931c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.516+0000 7f9339600640 1 -- 172.21.15.19:0/401303365 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f9328095660 con 0x7f933c1434d0 2024-04-25T11:28:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.516+0000 7f9341600640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f931c0744e0 0x7f931c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:10.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.518+0000 7f9341600640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f931c0744e0 0x7f931c0769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9330034000 tx=0x7f933002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:10.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.521+0000 7f9339600640 1 -- 172.21.15.19:0/401303365 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f932805fdc0 con 0x7f933c1434d0 2024-04-25T11:28:10.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.716+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f933c0b60e0 con 0x7f933c1434d0 2024-04-25T11:28:10.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.719+0000 7f9339600640 1 -- 172.21.15.19:0/401303365 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f932805f6c0 con 0x7f933c1434d0 2024-04-25T11:28:10.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:10.712 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f931c0744e0 msgr2=0x7f931c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f931c0744e0 0x7f931c0769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9330034000 tx=0x7f933002c000 comp rx=0 tx=0).stop 2024-04-25T11:28:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 msgr2=0x7f933c144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c144120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f932800d5c0 tx=0x7f932800da80 comp rx=0 tx=0).stop 2024-04-25T11:28:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 shutdown_connections 2024-04-25T11:28:10.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f931c0744e0 0x7f931c0769f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f933c160be0 0x7f933c1386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f933c1434d0 0x7f933c144120 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 --2- 172.21.15.19:0/401303365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f933c0d0ed0 0x7f933c0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 >> 172.21.15.19:0/401303365 conn(0x7f933c0d1ae0 msgr2=0x7f933c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 shutdown_connections 2024-04-25T11:28:10.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:10.722+0000 7f9344000640 1 -- 172.21.15.19:0/401303365 wait complete. 2024-04-25T11:28:10.742 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:10.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.039+0000 7f5681a00640 1 Processor -- start 2024-04-25T11:28:11.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.039+0000 7f5681a00640 1 -- start start 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f5681a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f567c070630 0x7f567c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f5681a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c06dce0 0x7f567c160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f5681a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f567c071110 con 0x7f567c15adf0 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f5681a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f567c0713f0 con 0x7f567c070630 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f5681a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f567c0701a0 con 0x7f567c06dce0 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f567aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f567c070630 0x7f567c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f5672000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f567c15adf0 0x7f567c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40618/0 (socket says 172.21.15.19:40618) 2024-04-25T11:28:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.041+0000 7f5672000640 1 -- 172.21.15.19:0/1132037193 learned_addr learned my addr 172.21.15.19:0/1132037193 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.042+0000 7f567b400640 1 --2- 172.21.15.19:0/1132037193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c06dce0 0x7f567c160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.042+0000 7f567aa00640 1 -- 172.21.15.19:0/1132037193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c06dce0 msgr2=0x7f567c160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.042+0000 7f567aa00640 1 --2- 172.21.15.19:0/1132037193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c06dce0 0x7f567c160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.042+0000 7f567aa00640 1 -- 172.21.15.19:0/1132037193 >> v1:172.21.15.19:6789/0 conn(0x7f567c15adf0 legacy=0x7f567c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.042+0000 7f567aa00640 1 -- 172.21.15.19:0/1132037193 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c06fb50 con 0x7f567c070630 2024-04-25T11:28:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.042+0000 7f567a000640 1 -- 172.21.15.19:0/1132037193 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 540130603 0 0) 0x7f567c071110 con 0x7f567c15adf0 2024-04-25T11:28:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.043+0000 7f567aa00640 1 --2- 172.21.15.19:0/1132037193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c070630 0x7f567c0cec50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f566c0097a0 tx=0x7f566c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a740c7467ed90bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.043+0000 7f567a000640 1 -- 172.21.15.19:0/1132037193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c037070 con 0x7f567c070630 2024-04-25T11:28:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.043+0000 7f567a000640 1 -- 172.21.15.19:0/1132037193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566c02ae50 con 0x7f567c070630 2024-04-25T11:28:11.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.043+0000 7f567a000640 1 -- 172.21.15.19:0/1132037193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c032660 con 0x7f567c070630 2024-04-25T11:28:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 -- 172.21.15.19:0/1132037193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c070630 msgr2=0x7f567c0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 --2- 172.21.15.19:0/1132037193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c070630 0x7f567c0cec50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f566c0097a0 tx=0x7f566c02a100 comp rx=0 tx=0).stop 2024-04-25T11:28:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 -- 172.21.15.19:0/1132037193 shutdown_connections 2024-04-25T11:28:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 --2- 172.21.15.19:0/1132037193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c06dce0 0x7f567c160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 --2- 172.21.15.19:0/1132037193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c070630 0x7f567c0cec50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 -- 172.21.15.19:0/1132037193 >> 172.21.15.19:0/1132037193 conn(0x7f567c073090 msgr2=0x7f567c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:11.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 -- 172.21.15.19:0/1132037193 shutdown_connections 2024-04-25T11:28:11.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.044+0000 7f5681a00640 1 -- 172.21.15.19:0/1132037193 wait complete. 2024-04-25T11:28:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.047+0000 7f5681a00640 1 Processor -- start 2024-04-25T11:28:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.047+0000 7f5681a00640 1 -- start start 2024-04-25T11:28:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.048+0000 7f5681a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c070630 0x7f567c14c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.048+0000 7f5681a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c15adf0 0x7f567c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.048+0000 7f5681a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567c1531c0 0x7f567c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.048+0000 7f5681a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0b0e80 con 0x7f567c15adf0 2024-04-25T11:28:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.048+0000 7f5681a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0b0d00 con 0x7f567c1531c0 2024-04-25T11:28:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.048+0000 7f5681a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f567c070c80 con 0x7f567c070630 2024-04-25T11:28:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c15adf0 0x7f567c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c15adf0 0x7f567c154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36552/0 (socket says 172.21.15.19:36552) 2024-04-25T11:28:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 -- 172.21.15.19:0/805416981 learned_addr learned my addr 172.21.15.19:0/805416981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f567b400640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567c1531c0 0x7f567c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f567aa00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c070630 0x7f567c14c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 -- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c070630 msgr2=0x7f567c14c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c070630 0x7f567c14c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 -- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567c1531c0 msgr2=0x7f567c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567c1531c0 0x7f567c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f5672000640 1 -- 172.21.15.19:0/805416981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c15db30 con 0x7f567c15adf0 2024-04-25T11:28:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.049+0000 7f567b400640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567c1531c0 0x7f567c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.050+0000 7f5672000640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c15adf0 0x7f567c154be0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f566400c430 tx=0x7f566400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.050+0000 7f5673400640 1 -- 172.21.15.19:0/805416981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640074e0 con 0x7f567c15adf0 2024-04-25T11:28:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.050+0000 7f5673400640 1 -- 172.21.15.19:0/805416981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56640047e0 con 0x7f567c15adf0 2024-04-25T11:28:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.050+0000 7f5673400640 1 -- 172.21.15.19:0/805416981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664004bd0 con 0x7f567c15adf0 2024-04-25T11:28:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.050+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c15de10 con 0x7f567c15adf0 2024-04-25T11:28:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.050+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c15e3d0 con 0x7f567c15adf0 2024-04-25T11:28:11.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.051+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567c050740 con 0x7f567c15adf0 2024-04-25T11:28:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.052+0000 7f5673400640 1 -- 172.21.15.19:0/805416981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5664011070 con 0x7f567c15adf0 2024-04-25T11:28:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.053+0000 7f5673400640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f56580744b0 0x7f56580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.054+0000 7f5673400640 1 -- 172.21.15.19:0/805416981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f5664094f40 con 0x7f567c15adf0 2024-04-25T11:28:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.054+0000 7f567aa00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f56580744b0 0x7f56580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.056+0000 7f567aa00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f56580744b0 0x7f56580769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:11.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.059+0000 7f5673400640 1 -- 172.21.15.19:0/805416981 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f566405f610 con 0x7f567c15adf0 2024-04-25T11:28:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.263+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f567c15d850 con 0x7f567c15adf0 2024-04-25T11:28:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.264+0000 7f5673400640 1 -- 172.21.15.19:0/805416981 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f566405ef10 con 0x7f567c15adf0 2024-04-25T11:28:11.255 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f56580744b0 msgr2=0x7f56580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f56580744b0 0x7f56580769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c02e000 comp rx=0 tx=0).stop 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c15adf0 msgr2=0x7f567c154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c15adf0 0x7f567c154be0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f566400c430 tx=0x7f566400c8f0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 shutdown_connections 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f56580744b0 0x7f56580769c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f567c1531c0 0x7f567c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f567c15adf0 0x7f567c154be0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 --2- 172.21.15.19:0/805416981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f567c070630 0x7f567c14c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 >> 172.21.15.19:0/805416981 conn(0x7f567c073090 msgr2=0x7f567c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 shutdown_connections 2024-04-25T11:28:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.266+0000 7f5681a00640 1 -- 172.21.15.19:0/805416981 wait complete. 2024-04-25T11:28:11.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:11.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.583+0000 7f0f0b600640 1 Processor -- start 2024-04-25T11:28:11.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.584+0000 7f0f0b600640 1 -- start start 2024-04-25T11:28:11.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f0b600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0f040af710 0x7f0f04164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f0b600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0f04160390 0x7f0f0406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f0b600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0406df20 con 0x7f0f0406db10 2024-04-25T11:28:11.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f0b600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0f040afb20 con 0x7f0f040af710 2024-04-25T11:28:11.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f0b600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0f04164fc0 con 0x7f0f04160390 2024-04-25T11:28:11.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f08c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0f040af710 0x7f0f04164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f08c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0f040af710 0x7f0f04164a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36554/0 (socket says 172.21.15.19:36554) 2024-04-25T11:28:11.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f08c00640 1 -- 172.21.15.19:0/3741470544 learned_addr learned my addr 172.21.15.19:0/3741470544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:11.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.586+0000 7f0f09600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f0406db10 0x7f0f040b4a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40620/0 (socket says 172.21.15.19:40620) 2024-04-25T11:28:11.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.587+0000 7f0f03e00640 1 --2- 172.21.15.19:0/3741470544 >> v2:172.21.15.169:3300/0 conn(0x7f0f04160390 0x7f0f0406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.587+0000 7f0f08c00640 1 -- 172.21.15.19:0/3741470544 >> v2:172.21.15.169:3300/0 conn(0x7f0f04160390 msgr2=0x7f0f0406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:11.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.587+0000 7f0f08c00640 1 --2- 172.21.15.19:0/3741470544 >> v2:172.21.15.169:3300/0 conn(0x7f0f04160390 0x7f0f0406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.587+0000 7f0f08c00640 1 -- 172.21.15.19:0/3741470544 >> v1:172.21.15.19:6789/0 conn(0x7f0f0406db10 legacy=0x7f0f040b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.587+0000 7f0f08c00640 1 -- 172.21.15.19:0/3741470544 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0406f440 con 0x7f0f040af710 2024-04-25T11:28:11.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.587+0000 7f0f03400640 1 -- 172.21.15.19:0/3741470544 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407567065 0 0) 0x7f0efc001490 con 0x7f0f0406db10 2024-04-25T11:28:11.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.588+0000 7f0f08c00640 1 --2- 172.21.15.19:0/3741470544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04164a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f040b5d10 tx=0x7f0eec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8379864533c2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:11.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.589+0000 7f0f03400640 1 -- 172.21.15.19:0/3741470544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec037070 con 0x7f0f040af710 2024-04-25T11:28:11.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.589+0000 7f0f03400640 1 -- 172.21.15.19:0/3741470544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec032440 con 0x7f0f040af710 2024-04-25T11:28:11.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.589+0000 7f0f03400640 1 -- 172.21.15.19:0/3741470544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec032810 con 0x7f0f040af710 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.589+0000 7f0f0b600640 1 -- 172.21.15.19:0/3741470544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 msgr2=0x7f0f04164a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.589+0000 7f0f0b600640 1 --2- 172.21.15.19:0/3741470544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04164a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f040b5d10 tx=0x7f0eec02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.590+0000 7f0f0b600640 1 -- 172.21.15.19:0/3741470544 shutdown_connections 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.590+0000 7f0f0b600640 1 --2- 172.21.15.19:0/3741470544 >> v2:172.21.15.169:3300/0 conn(0x7f0f04160390 0x7f0f0406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.590+0000 7f0f0b600640 1 --2- 172.21.15.19:0/3741470544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04164a80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.590+0000 7f0f0b600640 1 -- 172.21.15.19:0/3741470544 >> 172.21.15.19:0/3741470544 conn(0x7f0f04073090 msgr2=0x7f0f040734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.590+0000 7f0f0b600640 1 -- 172.21.15.19:0/3741470544 shutdown_connections 2024-04-25T11:28:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.590+0000 7f0f0b600640 1 -- 172.21.15.19:0/3741470544 wait complete. 2024-04-25T11:28:11.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.592+0000 7f0f0b600640 1 Processor -- start 2024-04-25T11:28:11.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.592+0000 7f0f0b600640 1 -- start start 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f0b600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f0406db10 0x7f0f0414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f0b600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f0b600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f04160390 0x7f0f0415eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f0b600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0406ffc0 con 0x7f0f040af710 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f0b600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0406fce0 con 0x7f0f0406db10 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f0b600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0406f0d0 con 0x7f0f04160390 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36566/0 (socket says 172.21.15.19:36566) 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.594+0000 7f0f03e00640 1 -- 172.21.15.19:0/1177393150 learned_addr learned my addr 172.21.15.19:0/1177393150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f08c00640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f0406db10 0x7f0f0414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f09600640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f04160390 0x7f0f0415eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f03e00640 1 -- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f04160390 msgr2=0x7f0f0415eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f03e00640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f04160390 0x7f0f0415eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f03e00640 1 -- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f0406db10 msgr2=0x7f0f0414f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f03e00640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f0406db10 0x7f0f0414f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f03e00640 1 -- 172.21.15.19:0/1177393150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f04154550 con 0x7f0f040af710 2024-04-25T11:28:11.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f03e00640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04153700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400d620 tx=0x7f0ef400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:11.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f00c00640 1 -- 172.21.15.19:0/1177393150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400b620 con 0x7f0f040af710 2024-04-25T11:28:11.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.595+0000 7f0f00c00640 1 -- 172.21.15.19:0/1177393150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef4019070 con 0x7f0f040af710 2024-04-25T11:28:11.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.596+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f04154830 con 0x7f0f040af710 2024-04-25T11:28:11.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.596+0000 7f0f00c00640 1 -- 172.21.15.19:0/1177393150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400ba00 con 0x7f0f040af710 2024-04-25T11:28:11.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.596+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f04154df0 con 0x7f0f040af710 2024-04-25T11:28:11.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.597+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f04154030 con 0x7f0f040af710 2024-04-25T11:28:11.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.598+0000 7f0f00c00640 1 -- 172.21.15.19:0/1177393150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ef400bba0 con 0x7f0f040af710 2024-04-25T11:28:11.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.599+0000 7f0f00c00640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0ee00744e0 0x7f0ee00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:11.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.599+0000 7f0f00c00640 1 -- 172.21.15.19:0/1177393150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f0ef4095750 con 0x7f0f040af710 2024-04-25T11:28:11.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.600+0000 7f0f08c00640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0ee00744e0 0x7f0ee00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:11.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.601+0000 7f0f08c00640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0ee00744e0 0x7f0ee00769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0eec034000 tx=0x7f0eec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:11.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.604+0000 7f0f00c00640 1 -- 172.21.15.19:0/1177393150 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f04154030 con 0x7f0f040af710 2024-04-25T11:28:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.806+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f04050740 con 0x7f0f040af710 2024-04-25T11:28:11.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.807+0000 7f0f00c00640 1 -- 172.21.15.19:0/1177393150 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f0ef405fe20 con 0x7f0f040af710 2024-04-25T11:28:11.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.809+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0ee00744e0 msgr2=0x7f0ee00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0ee00744e0 0x7f0ee00769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0eec034000 tx=0x7f0eec02e000 comp rx=0 tx=0).stop 2024-04-25T11:28:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 msgr2=0x7f0f04153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04153700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400d620 tx=0x7f0ef400dae0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 shutdown_connections 2024-04-25T11:28:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0ee00744e0 0x7f0ee00769f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f04160390 0x7f0f0415eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f040af710 0x7f0f04153700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 --2- 172.21.15.19:0/1177393150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f0406db10 0x7f0f0414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 >> 172.21.15.19:0/1177393150 conn(0x7f0f04073090 msgr2=0x7f0f040b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 shutdown_connections 2024-04-25T11:28:11.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:11.810+0000 7f0f0b600640 1 -- 172.21.15.19:0/1177393150 wait complete. 2024-04-25T11:28:11.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:12.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.124+0000 7f8ec5200640 1 Processor -- start 2024-04-25T11:28:12.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.124+0000 7f8ec5200640 1 -- start start 2024-04-25T11:28:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.126+0000 7f8ec5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ec0168c10 0x7f8ec0070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.126+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec006dce0 0x7f8ec00af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ec5200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec0164b90 con 0x7f8ec0164620 2024-04-25T11:28:12.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ec5200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec00b6530 con 0x7f8ec0168c10 2024-04-25T11:28:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ec5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec0070ea0 con 0x7f8ec006dce0 2024-04-25T11:28:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ebf400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ec0168c10 0x7f8ec0070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ebfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec006dce0 0x7f8ec00af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ebf400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ec0168c10 0x7f8ec0070550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:58362/0 (socket says 172.21.15.19:58362) 2024-04-25T11:28:12.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ebf400640 1 -- 172.21.15.19:0/2291409022 learned_addr learned my addr 172.21.15.19:0/2291409022 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:12.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.127+0000 7f8ebea00640 1 --1- 172.21.15.19:0/2291409022 >> v1:172.21.15.169:6789/0 conn(0x7f8ec0164620 0x7f8ec006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:50188/0 (socket says 172.21.15.19:50188) 2024-04-25T11:28:12.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.128+0000 7f8ebfe00640 1 -- 172.21.15.19:0/2291409022 >> v2:172.21.15.169:3300/0 conn(0x7f8ec0168c10 msgr2=0x7f8ec0070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:12.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.128+0000 7f8ebfe00640 1 --2- 172.21.15.19:0/2291409022 >> v2:172.21.15.169:3300/0 conn(0x7f8ec0168c10 0x7f8ec0070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.128+0000 7f8ebfe00640 1 -- 172.21.15.19:0/2291409022 >> v1:172.21.15.169:6789/0 conn(0x7f8ec0164620 legacy=0x7f8ec006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.128+0000 7f8ebfe00640 1 -- 172.21.15.19:0/2291409022 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec0164e70 con 0x7f8ec006dce0 2024-04-25T11:28:12.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.128+0000 7f8ebf400640 1 --2- 172.21.15.19:0/2291409022 >> v2:172.21.15.169:3300/0 conn(0x7f8ec0168c10 0x7f8ec0070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:28:12.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.130+0000 7f8ebfe00640 1 --2- 172.21.15.19:0/2291409022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec006dce0 0x7f8ec00af620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009d90 tx=0x7f8eb802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7687b28db4505d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:12.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.130+0000 7f8ebe000640 1 -- 172.21.15.19:0/2291409022 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8031600 con 0x7f8ec006dce0 2024-04-25T11:28:12.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.130+0000 7f8ebe000640 1 -- 172.21.15.19:0/2291409022 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb80317a0 con 0x7f8ec006dce0 2024-04-25T11:28:12.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.130+0000 7f8ebe000640 1 -- 172.21.15.19:0/2291409022 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8031b70 con 0x7f8ec006dce0 2024-04-25T11:28:12.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.131+0000 7f8ec5200640 1 -- 172.21.15.19:0/2291409022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec006dce0 msgr2=0x7f8ec00af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.131+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2291409022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec006dce0 0x7f8ec00af620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009d90 tx=0x7f8eb802a5d0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.131+0000 7f8ec5200640 1 -- 172.21.15.19:0/2291409022 shutdown_connections 2024-04-25T11:28:12.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.131+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2291409022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec006dce0 0x7f8ec00af620 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.131+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2291409022 >> v2:172.21.15.169:3300/0 conn(0x7f8ec0168c10 0x7f8ec0070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.131+0000 7f8ec5200640 1 -- 172.21.15.19:0/2291409022 >> 172.21.15.19:0/2291409022 conn(0x7f8ec0073160 msgr2=0x7f8ec0073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:12.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.132+0000 7f8ec5200640 1 -- 172.21.15.19:0/2291409022 shutdown_connections 2024-04-25T11:28:12.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.132+0000 7f8ec5200640 1 -- 172.21.15.19:0/2291409022 wait complete. 2024-04-25T11:28:12.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.135+0000 7f8ec5200640 1 Processor -- start 2024-04-25T11:28:12.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.135+0000 7f8ec5200640 1 -- start start 2024-04-25T11:28:12.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec0164620 0x7f8ec015a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ec0168c10 0x7f8ec014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec01fe7b0 0x7f8ec015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ec5200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0165950 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ec5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01654c0 con 0x7f8ec0168c10 2024-04-25T11:28:12.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ec5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec0070a90 con 0x7f8ec0164620 2024-04-25T11:28:12.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ebfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec01fe7b0 0x7f8ec015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ebfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec01fe7b0 0x7f8ec015bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36576/0 (socket says 172.21.15.19:36576) 2024-04-25T11:28:12.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ebfe00640 1 -- 172.21.15.19:0/2179344041 learned_addr learned my addr 172.21.15.19:0/2179344041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:12.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.137+0000 7f8ebea00640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ec0168c10 0x7f8ec014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebf400640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec0164620 0x7f8ec015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebfe00640 1 -- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec0164620 msgr2=0x7f8ec015a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebfe00640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec0164620 0x7f8ec015a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebfe00640 1 -- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ec0168c10 msgr2=0x7f8ec014cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebfe00640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ec0168c10 0x7f8ec014cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebfe00640 1 -- 172.21.15.19:0/2179344041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec015ac60 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebea00640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ec0168c10 0x7f8ec014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.138+0000 7f8ebfe00640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec01fe7b0 0x7f8ec015bd00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8eb802abc0 tx=0x7f8eb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.139+0000 7f8eb7400640 1 -- 172.21.15.19:0/2179344041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb803f070 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.139+0000 7f8eb7400640 1 -- 172.21.15.19:0/2179344041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb8030ae0 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.139+0000 7f8eb7400640 1 -- 172.21.15.19:0/2179344041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8044460 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.139+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec015af40 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.139+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec015b500 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.140+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec0076c60 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.142+0000 7f8eb7400640 1 -- 172.21.15.19:0/2179344041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8eb803b410 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.142+0000 7f8eb7400640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ea00744e0 0x7f8ea00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.143+0000 7f8eb7400640 1 -- 172.21.15.19:0/2179344041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f8eb80b6620 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.143+0000 7f8ebf400640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ea00744e0 0x7f8ea00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.144+0000 7f8ebf400640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ea00744e0 0x7f8ea00769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006cc0 tx=0x7f8eb0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:12.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.147+0000 7f8eb7400640 1 -- 172.21.15.19:0/2179344041 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8eb8080e20 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.350+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ec015a9f0 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.351+0000 7f8eb7400640 1 -- 172.21.15.19:0/2179344041 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f8eb8080720 con 0x7f8ec01fe7b0 2024-04-25T11:28:12.343 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.354+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ea00744e0 msgr2=0x7f8ea00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.354+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ea00744e0 0x7f8ea00769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006cc0 tx=0x7f8eb0009240 comp rx=0 tx=0).stop 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.354+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec01fe7b0 msgr2=0x7f8ec015bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.354+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec01fe7b0 0x7f8ec015bd00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8eb802abc0 tx=0x7f8eb8004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 shutdown_connections 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ea00744e0 0x7f8ea00769f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ec0168c10 0x7f8ec014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec01fe7b0 0x7f8ec015bd00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 --2- 172.21.15.19:0/2179344041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ec0164620 0x7f8ec015a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 >> 172.21.15.19:0/2179344041 conn(0x7f8ec0073160 msgr2=0x7f8ec0167750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 shutdown_connections 2024-04-25T11:28:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.355+0000 7f8ec5200640 1 -- 172.21.15.19:0/2179344041 wait complete. 2024-04-25T11:28:12.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-04-25T11:28:12.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.661+0000 7f8cf3200640 1 Processor -- start 2024-04-25T11:28:12.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.661+0000 7f8cf3200640 1 -- start start 2024-04-25T11:28:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 0x7f8cf4142e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf3200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf415d5e0 con 0x7f8cf406dce0 2024-04-25T11:28:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf415d8c0 con 0x7f8cf413fe10 2024-04-25T11:28:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf415dba0 con 0x7f8cf40d1040 2024-04-25T11:28:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf1400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8cf406dce0 0x7f8cf4142730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40626/0 (socket says 172.21.15.19:40626) 2024-04-25T11:28:12.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf1400640 1 -- 172.21.15.19:0/374432314 learned_addr learned my addr 172.21.15.19:0/374432314 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:12.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf2800640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 0x7f8cf4142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.663+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954691137 0 0) 0x7f8cf415d5e0 con 0x7f8cf406dce0 2024-04-25T11:28:12.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cd8003520 con 0x7f8cf406dce0 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1394507185 0 0) 0x7f8cd8003520 con 0x7f8cf406dce0 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 msgr2=0x7f8cf4142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 0x7f8cf4142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 >> v1:172.21.15.169:6789/0 conn(0x7f8cf413fe10 legacy=0x7f8cf406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf415de80 con 0x7f8cf406dce0 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8ce0001670 con 0x7f8cf406dce0 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 >> v1:172.21.15.19:6789/0 conn(0x7f8cf406dce0 legacy=0x7f8cf4142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cd80039e0 0x7f8cd8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd8004800 0x7f8cd8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 0x7f8cd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf415dba0 con 0x7f8cd8004800 2024-04-25T11:28:12.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd80055e0 con 0x7f8cd80039e0 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd800bf00 con 0x7f8cf40d1040 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.664+0000 7f8cf1400640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd8004800 0x7f8cd8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf1e00640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cd80039e0 0x7f8cd8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf2800640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 0x7f8cd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf1400640 1 -- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 msgr2=0x7f8cd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf1400640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cf40d1040 0x7f8cd800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf1400640 1 -- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cd80039e0 msgr2=0x7f8cd8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf1400640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cd80039e0 0x7f8cd8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf1400640 1 -- 172.21.15.19:0/374432314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd8003520 con 0x7f8cd8004800 2024-04-25T11:28:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.665+0000 7f8cf1e00640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cd80039e0 0x7f8cd8003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.666+0000 7f8cf1400640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd8004800 0x7f8cd8004c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c240 tx=0x7f8cd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc17264477498f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.666+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0007d10 con 0x7f8cd8004800 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.666+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce0005ad0 con 0x7f8cd8004800 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.666+0000 7f8cf0a00640 1 -- 172.21.15.19:0/374432314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0007070 con 0x7f8cd8004800 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.667+0000 7f8cf3200640 1 -- 172.21.15.19:0/374432314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd8004800 msgr2=0x7f8cd8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.667+0000 7f8cf3200640 1 --2- 172.21.15.19:0/374432314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cd8004800 0x7f8cd8004c10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c240 tx=0x7f8cd8005830 comp rx=0 tx=0).stop 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.667+0000 7f8cf2800640 1 -- 172.21.15.19:0/374432314 reap_dead start 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.667+0000 7f8cf3200640 1 -- 172.21.15.19:0/374432314 shutdown_connections 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.667+0000 7f8cf3200640 1 -- 172.21.15.19:0/374432314 >> 172.21.15.19:0/374432314 conn(0x7f8cf4072ee0 msgr2=0x7f8cf40732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:12.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.667+0000 7f8cf3200640 1 -- 172.21.15.19:0/374432314 shutdown_connections 2024-04-25T11:28:12.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.667+0000 7f8cf3200640 1 -- 172.21.15.19:0/374432314 wait complete. 2024-04-25T11:28:12.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.670+0000 7f8cf3200640 1 Processor -- start 2024-04-25T11:28:12.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.670+0000 7f8cf3200640 1 -- start start 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cd8004800 0x7f8cf41f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cf4141cf0 0x7f8cf4142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf413c490 0x7f8cf413c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4142fc0 con 0x7f8cf413c490 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4142e40 con 0x7f8cf4141cf0 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf4143140 con 0x7f8cd8004800 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf413c490 0x7f8cf413c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf413c490 0x7f8cf413c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36600/0 (socket says 172.21.15.19:36600) 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf2800640 1 -- 172.21.15.19:0/4184464901 learned_addr learned my addr 172.21.15.19:0/4184464901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf1400640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cf4141cf0 0x7f8cf4142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf1e00640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cd8004800 0x7f8cf41f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.672+0000 7f8cf2800640 1 -- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cd8004800 msgr2=0x7f8cf41f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf2800640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cd8004800 0x7f8cf41f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf2800640 1 -- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cf4141cf0 msgr2=0x7f8cf4142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf2800640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cf4141cf0 0x7f8cf4142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf2800640 1 -- 172.21.15.19:0/4184464901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf4142640 con 0x7f8cf413c490 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf1400640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cf4141cf0 0x7f8cf4142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf2800640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf413c490 0x7f8cf413c8a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098c0 tx=0x7f8cec009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cea000640 1 -- 172.21.15.19:0/4184464901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec007d20 con 0x7f8cf413c490 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cea000640 1 -- 172.21.15.19:0/4184464901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cec019070 con 0x7f8cf413c490 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cea000640 1 -- 172.21.15.19:0/4184464901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec014650 con 0x7f8cf413c490 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf413cde0 con 0x7f8cf413c490 2024-04-25T11:28:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.673+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf4145320 con 0x7f8cf413c490 2024-04-25T11:28:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.675+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf4050740 con 0x7f8cf413c490 2024-04-25T11:28:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.676+0000 7f8cea000640 1 -- 172.21.15.19:0/4184464901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8cec0147f0 con 0x7f8cf413c490 2024-04-25T11:28:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.677+0000 7f8cea000640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8cd40744e0 0x7f8cd40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:12.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.678+0000 7f8cea000640 1 -- 172.21.15.19:0/4184464901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f8cec0957a0 con 0x7f8cf413c490 2024-04-25T11:28:12.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.678+0000 7f8cf1e00640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8cd40744e0 0x7f8cd40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:12.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.679+0000 7f8cf1e00640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8cd40744e0 0x7f8cd40769f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4154410 tx=0x7f8ce4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:12.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.683+0000 7f8cea000640 1 -- 172.21.15.19:0/4184464901 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cf413cde0 con 0x7f8cf413c490 2024-04-25T11:28:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:12.967+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f8cf40b5350 con 0x7f8cf413c490 2024-04-25T11:28:13.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.904+0000 7f8cea000640 1 -- 172.21.15.19:0/4184464901 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v68) ==== 137+0+0 (secure 0 0 0) 0x7f8cf4142640 con 0x7f8cf413c490 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.907+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8cd40744e0 msgr2=0x7f8cd40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.907+0000 7f8cf3200640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8cd40744e0 0x7f8cd40769f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4154410 tx=0x7f8ce4009040 comp rx=0 tx=0).stop 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.907+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf413c490 msgr2=0x7f8cf413c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.907+0000 7f8cf3200640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf413c490 0x7f8cf413c8a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098c0 tx=0x7f8cec009d80 comp rx=0 tx=0).stop 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 shutdown_connections 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8cd40744e0 0x7f8cd40769f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cf4141cf0 0x7f8cf4142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cf413c490 0x7f8cf413c8a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 --2- 172.21.15.19:0/4184464901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cd8004800 0x7f8cf41f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 >> 172.21.15.19:0/4184464901 conn(0x7f8cf4072ee0 msgr2=0x7f8cf413ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 shutdown_connections 2024-04-25T11:28:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:13.908+0000 7f8cf3200640 1 -- 172.21.15.19:0/4184464901 wait complete. 2024-04-25T11:28:13.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:28:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.229+0000 7fbb92400640 1 Processor -- start 2024-04-25T11:28:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.229+0000 7fbb92400640 1 -- start start 2024-04-25T11:28:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.231+0000 7fbb92400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.231+0000 7fbb92400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb8c06d5d0 0x7fbb8c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.231+0000 7fbb92400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8c06df20 con 0x7fbb8c163c90 2024-04-25T11:28:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.231+0000 7fbb92400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbb8c0b0090 con 0x7fbb8c06d5d0 2024-04-25T11:28:14.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb92400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8c160c80 con 0x7fbb8c15de90 2024-04-25T11:28:14.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb91a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb8c06d5d0 0x7fbb8c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb91a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb8c06d5d0 0x7fbb8c06d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54470/0 (socket says 172.21.15.19:54470) 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb91a00640 1 -- 172.21.15.19:0/3696814359 learned_addr learned my addr 172.21.15.19:0/3696814359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb91000640 1 --2- 172.21.15.19:0/3696814359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb91a00640 1 -- 172.21.15.19:0/3696814359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 msgr2=0x7fbb8c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb91a00640 1 --2- 172.21.15.19:0/3696814359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.232+0000 7fbb91a00640 1 -- 172.21.15.19:0/3696814359 >> v1:172.21.15.169:6789/0 conn(0x7fbb8c163c90 legacy=0x7fbb8c06f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.233+0000 7fbb91a00640 1 -- 172.21.15.19:0/3696814359 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c160f60 con 0x7fbb8c06d5d0 2024-04-25T11:28:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.234+0000 7fbb91a00640 1 --2- 172.21.15.19:0/3696814359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c06d5d0 0x7fbb8c06d9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009e50 tx=0x7fbb8402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f71944935b8e8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.234+0000 7fbb8b400640 1 -- 172.21.15.19:0/3696814359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84036070 con 0x7fbb8c06d5d0 2024-04-25T11:28:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.234+0000 7fbb8b400640 1 -- 172.21.15.19:0/3696814359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb84031600 con 0x7fbb8c06d5d0 2024-04-25T11:28:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.234+0000 7fbb8b400640 1 -- 172.21.15.19:0/3696814359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb840319d0 con 0x7fbb8c06d5d0 2024-04-25T11:28:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 -- 172.21.15.19:0/3696814359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c06d5d0 msgr2=0x7fbb8c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 --2- 172.21.15.19:0/3696814359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c06d5d0 0x7fbb8c06d9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009e50 tx=0x7fbb8402a6c0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 -- 172.21.15.19:0/3696814359 shutdown_connections 2024-04-25T11:28:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 --2- 172.21.15.19:0/3696814359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c06d5d0 0x7fbb8c06d9e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 --2- 172.21.15.19:0/3696814359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 -- 172.21.15.19:0/3696814359 >> 172.21.15.19:0/3696814359 conn(0x7fbb8c072f40 msgr2=0x7fbb8c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 -- 172.21.15.19:0/3696814359 shutdown_connections 2024-04-25T11:28:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.235+0000 7fbb92400640 1 -- 172.21.15.19:0/3696814359 wait complete. 2024-04-25T11:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.237+0000 7fbb92400640 1 Processor -- start 2024-04-25T11:28:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.238+0000 7fbb92400640 1 -- start start 2024-04-25T11:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.239+0000 7fbb92400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb8c06d5d0 0x7fbb8c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.239+0000 7fbb92400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.239+0000 7fbb92400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c163c90 0x7fbb8c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.239+0000 7fbb92400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c070d30 con 0x7fbb8c163c90 2024-04-25T11:28:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.239+0000 7fbb92400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c161520 con 0x7fbb8c06d5d0 2024-04-25T11:28:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.239+0000 7fbb92400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8c070eb0 con 0x7fbb8c15de90 2024-04-25T11:28:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c163c90 0x7fbb8c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c163c90 0x7fbb8c1577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54472/0 (socket says 172.21.15.19:54472) 2024-04-25T11:28:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 -- 172.21.15.19:0/3255073443 learned_addr learned my addr 172.21.15.19:0/3255073443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:14.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91000640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb8c06d5d0 0x7fbb8c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:14.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb8be00640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:14.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 -- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 msgr2=0x7fbb8c1510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:14.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 -- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb8c06d5d0 msgr2=0x7fbb8c156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb8c06d5d0 0x7fbb8c156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.240+0000 7fbb91a00640 1 -- 172.21.15.19:0/3255073443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c0d0110 con 0x7fbb8c163c90 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.241+0000 7fbb91a00640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c163c90 0x7fbb8c1577d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbb84034040 tx=0x7fbb84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.241+0000 7fbb88c00640 1 -- 172.21.15.19:0/3255073443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84036050 con 0x7fbb8c163c90 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.241+0000 7fbb88c00640 1 -- 172.21.15.19:0/3255073443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb84042070 con 0x7fbb8c163c90 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.241+0000 7fbb88c00640 1 -- 172.21.15.19:0/3255073443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84009510 con 0x7fbb8c163c90 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.241+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8c0d03f0 con 0x7fbb8c163c90 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.241+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8c0d09b0 con 0x7fbb8c163c90 2024-04-25T11:28:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.242+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8c076c60 con 0x7fbb8c163c90 2024-04-25T11:28:14.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.244+0000 7fbb88c00640 1 -- 172.21.15.19:0/3255073443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb84007bd0 con 0x7fbb8c163c90 2024-04-25T11:28:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.245+0000 7fbb88c00640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb6c0744b0 0x7fbb6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.245+0000 7fbb88c00640 1 -- 172.21.15.19:0/3255073443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fbb840b61f0 con 0x7fbb8c163c90 2024-04-25T11:28:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.246+0000 7fbb91000640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb6c0744b0 0x7fbb6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.247+0000 7fbb91000640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb6c0744b0 0x7fbb6c0769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb80006fd0 tx=0x7fbb80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:14.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.251+0000 7fbb88c00640 1 -- 172.21.15.19:0/3255073443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb8c06df20 con 0x7fbb8c163c90 2024-04-25T11:28:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.461+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbb8c0aeb80 con 0x7fbb8c163c90 2024-04-25T11:28:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.467+0000 7fbb88c00640 1 -- 172.21.15.19:0/3255073443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fbb840808c0 con 0x7fbb8c163c90 2024-04-25T11:28:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb6c0744b0 msgr2=0x7fbb6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb6c0744b0 0x7fbb6c0769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbb80006fd0 tx=0x7fbb80008040 comp rx=0 tx=0).stop 2024-04-25T11:28:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c163c90 msgr2=0x7fbb8c1577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c163c90 0x7fbb8c1577d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbb84034040 tx=0x7fbb84004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 shutdown_connections 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb6c0744b0 0x7fbb6c0769c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb8c163c90 0x7fbb8c1577d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb8c15de90 0x7fbb8c1510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 --2- 172.21.15.19:0/3255073443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb8c06d5d0 0x7fbb8c156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 >> 172.21.15.19:0/3255073443 conn(0x7fbb8c072f40 msgr2=0x7fbb8c0b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 shutdown_connections 2024-04-25T11:28:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:14.470+0000 7fbb92400640 1 -- 172.21.15.19:0/3255073443 wait complete. 2024-04-25T11:28:14.493 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:28:14.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:28:14.590 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:28:14.711 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:28:14.711 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:14.711 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:14.766 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:14.766 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:14.815 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:14.816 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:14.865 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.187+0000 7fe688a00640 1 Processor -- start 2024-04-25T11:28:15.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.187+0000 7fe688a00640 1 -- start start 2024-04-25T11:28:15.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.189+0000 7fe688a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe6840b4ea0 0x7fe6840cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.189+0000 7fe688a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 0x7fe68415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.189+0000 7fe688a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6840d1230 con 0x7fe68406c5c0 2024-04-25T11:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.189+0000 7fe688a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe6840d06b0 con 0x7fe6840b4ea0 2024-04-25T11:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.189+0000 7fe688a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6840d0990 con 0x7fe684144370 2024-04-25T11:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.190+0000 7fe683400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe68406c5c0 0x7fe6840b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43944/0 (socket says 172.21.15.19:43944) 2024-04-25T11:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.190+0000 7fe683400640 1 -- 172.21.15.19:0/764712382 learned_addr learned my addr 172.21.15.19:0/764712382 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.190+0000 7fe67aa00640 1 --2- 172.21.15.19:0/764712382 >> v2:172.21.15.169:3300/0 conn(0x7fe6840b4ea0 0x7fe6840cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.190+0000 7fe683e00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 0x7fe68415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.190+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2816294546 0 0) 0x7fe6840d1230 con 0x7fe68406c5c0 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.190+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe668003520 con 0x7fe68406c5c0 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3873470475 0 0) 0x7fe668003520 con 0x7fe68406c5c0 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 msgr2=0x7fe68415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 0x7fe68415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 >> v2:172.21.15.169:3300/0 conn(0x7fe6840b4ea0 msgr2=0x7fe6840cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 --2- 172.21.15.19:0/764712382 >> v2:172.21.15.169:3300/0 conn(0x7fe6840b4ea0 0x7fe6840cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6840d0c70 con 0x7fe68406c5c0 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe6740019f0 con 0x7fe68406c5c0 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 >> v1:172.21.15.19:6789/0 conn(0x7fe68406c5c0 legacy=0x7fe6840b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680039b0 0x7fe668003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 0x7fe668004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe668004d40 0x7fe668005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6840d06b0 con 0x7fe6680039b0 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6840d0990 con 0x7fe668004d40 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe66800b9f0 con 0x7fe684144370 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.191+0000 7fe683400640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680039b0 0x7fe668003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.192+0000 7fe67aa00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 0x7fe668004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.192+0000 7fe683e00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe668004d40 0x7fe668005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.192+0000 7fe683400640 1 -- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 msgr2=0x7fe668004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:15.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.192+0000 7fe683400640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684144370 0x7fe668004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.192+0000 7fe683400640 1 -- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe668004d40 msgr2=0x7fe668005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.192+0000 7fe683400640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe668004d40 0x7fe668005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.192+0000 7fe683400640 1 -- 172.21.15.19:0/764712382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe668003520 con 0x7fe6680039b0 2024-04-25T11:28:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.193+0000 7fe683400640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680039b0 0x7fe668003e20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe67400a7a0 tx=0x7fe668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11900b6ac97b6199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.193+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67400a3f0 con 0x7fe6680039b0 2024-04-25T11:28:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.193+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67402f070 con 0x7fe6680039b0 2024-04-25T11:28:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.193+0000 7fe682a00640 1 -- 172.21.15.19:0/764712382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67402bc20 con 0x7fe6680039b0 2024-04-25T11:28:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.194+0000 7fe688a00640 1 -- 172.21.15.19:0/764712382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680039b0 msgr2=0x7fe668003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.194+0000 7fe688a00640 1 --2- 172.21.15.19:0/764712382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680039b0 0x7fe668003e20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe67400a7a0 tx=0x7fe668005830 comp rx=0 tx=0).stop 2024-04-25T11:28:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.194+0000 7fe683e00640 1 -- 172.21.15.19:0/764712382 reap_dead start 2024-04-25T11:28:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.194+0000 7fe688a00640 1 -- 172.21.15.19:0/764712382 shutdown_connections 2024-04-25T11:28:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.194+0000 7fe688a00640 1 -- 172.21.15.19:0/764712382 >> 172.21.15.19:0/764712382 conn(0x7fe6840d1720 msgr2=0x7fe6840d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.194+0000 7fe688a00640 1 -- 172.21.15.19:0/764712382 shutdown_connections 2024-04-25T11:28:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.195+0000 7fe688a00640 1 -- 172.21.15.19:0/764712382 wait complete. 2024-04-25T11:28:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.198+0000 7fe688a00640 1 Processor -- start 2024-04-25T11:28:15.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.199+0000 7fe688a00640 1 -- start start 2024-04-25T11:28:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.201+0000 7fe688a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe68415b7d0 0x7fe6841457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.201+0000 7fe688a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684145d10 0x7fe684070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.201+0000 7fe688a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6840709b0 0x7fe684070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.201+0000 7fe688a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6840d0450 con 0x7fe68415b7d0 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.201+0000 7fe688a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6840d02d0 con 0x7fe6840709b0 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.201+0000 7fe688a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe684147760 con 0x7fe684145d10 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe68415b7d0 0x7fe6841457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe68415b7d0 0x7fe6841457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54492/0 (socket says 172.21.15.19:54492) 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 -- 172.21.15.19:0/593630373 learned_addr learned my addr 172.21.15.19:0/593630373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe67aa00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684145d10 0x7fe684070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683e00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6840709b0 0x7fe684070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 -- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684145d10 msgr2=0x7fe684070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684145d10 0x7fe684070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 -- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6840709b0 msgr2=0x7fe684070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6840709b0 0x7fe684070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683400640 1 -- 172.21.15.19:0/593630373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe684146120 con 0x7fe68415b7d0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.202+0000 7fe683e00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6840709b0 0x7fe684070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.203+0000 7fe683400640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe68415b7d0 0x7fe6841457d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe67402b3f0 tx=0x7fe674001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.203+0000 7fe67be00640 1 -- 172.21.15.19:0/593630373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67402bbd0 con 0x7fe68415b7d0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.203+0000 7fe67be00640 1 -- 172.21.15.19:0/593630373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67402bd70 con 0x7fe68415b7d0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.203+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68406f3d0 con 0x7fe68415b7d0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.203+0000 7fe67be00640 1 -- 172.21.15.19:0/593630373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674034670 con 0x7fe68415b7d0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.203+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68406f990 con 0x7fe68415b7d0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.204+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe684050740 con 0x7fe68415b7d0 2024-04-25T11:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.206+0000 7fe67be00640 1 -- 172.21.15.19:0/593630373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe674034810 con 0x7fe68415b7d0 2024-04-25T11:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.207+0000 7fe67be00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6600744b0 0x7fe6600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.207+0000 7fe67be00640 1 -- 172.21.15.19:0/593630373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fe6740b5690 con 0x7fe68415b7d0 2024-04-25T11:28:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.207+0000 7fe67aa00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6600744b0 0x7fe6600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.209+0000 7fe67aa00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6600744b0 0x7fe6600769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0090e0 tx=0x7fe66c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.216+0000 7fe67be00640 1 -- 172.21.15.19:0/593630373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe684146120 con 0x7fe68415b7d0 2024-04-25T11:28:15.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.424+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe68406e100 con 0x7fe68415b7d0 2024-04-25T11:28:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.426+0000 7fe67be00640 1 -- 172.21.15.19:0/593630373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7fe674086e20 con 0x7fe68415b7d0 2024-04-25T11:28:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:15.419 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:15.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6600744b0 msgr2=0x7fe6600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6600744b0 0x7fe6600769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0090e0 tx=0x7fe66c008040 comp rx=0 tx=0).stop 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe68415b7d0 msgr2=0x7fe6841457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe68415b7d0 0x7fe6841457d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe67402b3f0 tx=0x7fe674001360 comp rx=0 tx=0).stop 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 shutdown_connections 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6600744b0 0x7fe6600769c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6840709b0 0x7fe684070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe684145d10 0x7fe684070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 --2- 172.21.15.19:0/593630373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe68415b7d0 0x7fe6841457d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 >> 172.21.15.19:0/593630373 conn(0x7fe6840d1720 msgr2=0x7fe6840d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 shutdown_connections 2024-04-25T11:28:15.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.430+0000 7fe688a00640 1 -- 172.21.15.19:0/593630373 wait complete. 2024-04-25T11:28:15.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:15.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:15.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.726+0000 7fa9f5600640 1 Processor -- start 2024-04-25T11:28:15.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.726+0000 7fa9f5600640 1 -- start start 2024-04-25T11:28:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9f5600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa9f00b5740 0x7fa9f0143330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9f5600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa9f0147610 0x7fa9f0148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9f5600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9f00b5c20 con 0x7fa9f01495b0 2024-04-25T11:28:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9f5600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f006c5c0 con 0x7fa9f00b5740 2024-04-25T11:28:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9f5600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f00b5f00 con 0x7fa9f0147610 2024-04-25T11:28:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa9f00b5740 0x7fa9f0143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9e7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa9f01495b0 0x7fa9f0146f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43954/0 (socket says 172.21.15.19:43954) 2024-04-25T11:28:15.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9e7e00640 1 -- 172.21.15.19:0/3864701460 learned_addr learned my addr 172.21.15.19:0/3864701460 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:15.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.729+0000 7fa9efe00640 1 --2- 172.21.15.19:0/3864701460 >> v2:172.21.15.19:3300/0 conn(0x7fa9f00b5740 0x7fa9f0143330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54500/0 (socket says 172.21.15.19:54500) 2024-04-25T11:28:15.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.730+0000 7fa9f4c00640 1 --2- 172.21.15.19:0/3864701460 >> v2:172.21.15.169:3300/0 conn(0x7fa9f0147610 0x7fa9f0148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.730+0000 7fa9efe00640 1 -- 172.21.15.19:0/3864701460 >> v2:172.21.15.169:3300/0 conn(0x7fa9f0147610 msgr2=0x7fa9f0148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:15.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.730+0000 7fa9efe00640 1 --2- 172.21.15.19:0/3864701460 >> v2:172.21.15.169:3300/0 conn(0x7fa9f0147610 0x7fa9f0148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.730+0000 7fa9efe00640 1 -- 172.21.15.19:0/3864701460 >> v1:172.21.15.19:6789/0 conn(0x7fa9f01495b0 legacy=0x7fa9f0146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.730+0000 7fa9efe00640 1 -- 172.21.15.19:0/3864701460 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f00b61e0 con 0x7fa9f00b5740 2024-04-25T11:28:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.731+0000 7fa9efe00640 1 --2- 172.21.15.19:0/3864701460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0143330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0146ba0 tx=0x7fa9e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a99515d8521cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9ef400640 1 -- 172.21.15.19:0/3864701460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0037070 con 0x7fa9f00b5740 2024-04-25T11:28:15.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9ef400640 1 -- 172.21.15.19:0/3864701460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0032440 con 0x7fa9f00b5740 2024-04-25T11:28:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9ef400640 1 -- 172.21.15.19:0/3864701460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0032810 con 0x7fa9f00b5740 2024-04-25T11:28:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9f5600640 1 -- 172.21.15.19:0/3864701460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 msgr2=0x7fa9f0143330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9f5600640 1 --2- 172.21.15.19:0/3864701460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0143330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0146ba0 tx=0x7fa9e002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9f5600640 1 -- 172.21.15.19:0/3864701460 shutdown_connections 2024-04-25T11:28:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9f5600640 1 --2- 172.21.15.19:0/3864701460 >> v2:172.21.15.169:3300/0 conn(0x7fa9f0147610 0x7fa9f0148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.732+0000 7fa9f5600640 1 --2- 172.21.15.19:0/3864701460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0143330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.733+0000 7fa9f5600640 1 -- 172.21.15.19:0/3864701460 >> 172.21.15.19:0/3864701460 conn(0x7fa9f00b83a0 msgr2=0x7fa9f0070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.733+0000 7fa9f5600640 1 -- 172.21.15.19:0/3864701460 shutdown_connections 2024-04-25T11:28:15.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.733+0000 7fa9f5600640 1 -- 172.21.15.19:0/3864701460 wait complete. 2024-04-25T11:28:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.735+0000 7fa9f5600640 1 Processor -- start 2024-04-25T11:28:15.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.735+0000 7fa9f5600640 1 -- start start 2024-04-25T11:28:15.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9f5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9f5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9f01495b0 0x7fa9f013b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9f5600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9f00d21c0 0x7fa9f00d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9f5600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0149c50 con 0x7fa9f00b5740 2024-04-25T11:28:15.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9f5600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f00b67a0 con 0x7fa9f00d21c0 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9f5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f0149dd0 con 0x7fa9f01495b0 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54508/0 (socket says 172.21.15.19:54508) 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 -- 172.21.15.19:0/1297293483 learned_addr learned my addr 172.21.15.19:0/1297293483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9f4c00640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9f00d21c0 0x7fa9f00d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 -- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9f01495b0 msgr2=0x7fa9f013b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9f01495b0 0x7fa9f013b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 -- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9f00d21c0 msgr2=0x7fa9f00d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9f00d21c0 0x7fa9f00d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.737+0000 7fa9efe00640 1 -- 172.21.15.19:0/1297293483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f00d2b10 con 0x7fa9f00b5740 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.738+0000 7fa9efe00640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0140c50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009890 tx=0x7fa9e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.738+0000 7fa9ecc00640 1 -- 172.21.15.19:0/1297293483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0037040 con 0x7fa9f00b5740 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.738+0000 7fa9ecc00640 1 -- 172.21.15.19:0/1297293483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0042070 con 0x7fa9f00b5740 2024-04-25T11:28:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.738+0000 7fa9ecc00640 1 -- 172.21.15.19:0/1297293483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0030bd0 con 0x7fa9f00b5740 2024-04-25T11:28:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.738+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f013bb60 con 0x7fa9f00b5740 2024-04-25T11:28:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.738+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f013c120 con 0x7fa9f00b5740 2024-04-25T11:28:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.739+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f0076c60 con 0x7fa9f00b5740 2024-04-25T11:28:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.741+0000 7fa9ecc00640 1 -- 172.21.15.19:0/1297293483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa9e0047460 con 0x7fa9f00b5740 2024-04-25T11:28:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.742+0000 7fa9ecc00640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9cc0744e0 0x7fa9cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.742+0000 7fa9ecc00640 1 -- 172.21.15.19:0/1297293483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fa9e00b5dc0 con 0x7fa9f00b5740 2024-04-25T11:28:15.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.742+0000 7fa9e7e00640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9cc0744e0 0x7fa9cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.743+0000 7fa9e7e00640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9cc0744e0 0x7fa9cc0769f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8000f60 tx=0x7fa9d8007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.751+0000 7fa9ecc00640 1 -- 172.21.15.19:0/1297293483 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9f00d2b10 con 0x7fa9f00b5740 2024-04-25T11:28:15.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.945+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa9f00b7110 con 0x7fa9f00b5740 2024-04-25T11:28:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.946+0000 7fa9ecc00640 1 -- 172.21.15.19:0/1297293483 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fa9e0032e20 con 0x7fa9f00b5740 2024-04-25T11:28:15.938 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:15.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.948+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9cc0744e0 msgr2=0x7fa9cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.948+0000 7fa9f5600640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9cc0744e0 0x7fa9cc0769f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8000f60 tx=0x7fa9d8007a80 comp rx=0 tx=0).stop 2024-04-25T11:28:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.948+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 msgr2=0x7fa9f0140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.948+0000 7fa9f5600640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0140c50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009890 tx=0x7fa9e00046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 shutdown_connections 2024-04-25T11:28:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9cc0744e0 0x7fa9cc0769f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9f00d21c0 0x7fa9f00d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa9f01495b0 0x7fa9f013b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 --2- 172.21.15.19:0/1297293483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f00b5740 0x7fa9f0140c50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:15.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 >> 172.21.15.19:0/1297293483 conn(0x7fa9f00b83a0 msgr2=0x7fa9f0147790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:15.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 shutdown_connections 2024-04-25T11:28:15.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:15.949+0000 7fa9f5600640 1 -- 172.21.15.19:0/1297293483 wait complete. 2024-04-25T11:28:15.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.242+0000 7f0c7b000640 1 Processor -- start 2024-04-25T11:28:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.242+0000 7f0c7b000640 1 -- start start 2024-04-25T11:28:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.244+0000 7f0c7b000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c74070630 0x7f0c740cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.244+0000 7f0c7b000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c7415adf0 0x7f0c7406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.244+0000 7f0c7b000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c7406df20 con 0x7f0c7406db10 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.244+0000 7f0c7b000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0c740713f0 con 0x7f0c7415adf0 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.244+0000 7f0c7b000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0c740701a0 con 0x7f0c74070630 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.245+0000 7f0c73400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c7415adf0 0x7f0c7406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.245+0000 7f0c73400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c7415adf0 0x7f0c7406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54514/0 (socket says 172.21.15.19:54514) 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.245+0000 7f0c73400640 1 -- 172.21.15.19:0/2643833816 learned_addr learned my addr 172.21.15.19:0/2643833816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.245+0000 7f0c73400640 1 -- 172.21.15.19:0/2643833816 >> v2:172.21.15.169:3300/0 conn(0x7f0c74070630 msgr2=0x7f0c740cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.245+0000 7f0c73400640 1 --2- 172.21.15.19:0/2643833816 >> v2:172.21.15.169:3300/0 conn(0x7f0c74070630 0x7f0c740cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.245+0000 7f0c73400640 1 -- 172.21.15.19:0/2643833816 >> v1:172.21.15.169:6789/0 conn(0x7f0c7406db10 legacy=0x7f0c740b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.245+0000 7f0c73400640 1 -- 172.21.15.19:0/2643833816 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c74071110 con 0x7f0c7415adf0 2024-04-25T11:28:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.246+0000 7f0c73400640 1 --2- 172.21.15.19:0/2643833816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c7406d5d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0c740d2970 tx=0x7f0c5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1037e39d4c51ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.246+0000 7f0c72a00640 1 -- 172.21.15.19:0/2643833816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c037070 con 0x7f0c7415adf0 2024-04-25T11:28:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.246+0000 7f0c72a00640 1 -- 172.21.15.19:0/2643833816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c5c032440 con 0x7f0c7415adf0 2024-04-25T11:28:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.246+0000 7f0c72a00640 1 -- 172.21.15.19:0/2643833816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c032810 con 0x7f0c7415adf0 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 -- 172.21.15.19:0/2643833816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 msgr2=0x7f0c7406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 --2- 172.21.15.19:0/2643833816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c7406d5d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0c740d2970 tx=0x7f0c5c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 -- 172.21.15.19:0/2643833816 shutdown_connections 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 --2- 172.21.15.19:0/2643833816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c7406d5d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 --2- 172.21.15.19:0/2643833816 >> v2:172.21.15.169:3300/0 conn(0x7f0c74070630 0x7f0c740cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 -- 172.21.15.19:0/2643833816 >> 172.21.15.19:0/2643833816 conn(0x7f0c74073090 msgr2=0x7f0c740734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 -- 172.21.15.19:0/2643833816 shutdown_connections 2024-04-25T11:28:16.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.247+0000 7f0c7b000640 1 -- 172.21.15.19:0/2643833816 wait complete. 2024-04-25T11:28:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.250+0000 7f0c7b000640 1 Processor -- start 2024-04-25T11:28:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.250+0000 7f0c7b000640 1 -- start start 2024-04-25T11:28:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.252+0000 7f0c7b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c7406db10 0x7f0c741492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.252+0000 7f0c7b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c74070630 0x7f0c740b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.252+0000 7f0c7b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c740b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.252+0000 7f0c7b000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c740b0d00 con 0x7f0c7415adf0 2024-04-25T11:28:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.252+0000 7f0c7b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7406fe30 con 0x7f0c74070630 2024-04-25T11:28:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.252+0000 7f0c7b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0c740b0e80 con 0x7f0c7406db10 2024-04-25T11:28:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c740b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c740b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54518/0 (socket says 172.21.15.19:54518) 2024-04-25T11:28:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 -- 172.21.15.19:0/3892853070 learned_addr learned my addr 172.21.15.19:0/3892853070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 -- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c7406db10 msgr2=0x7f0c741492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c7406db10 0x7f0c741492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 -- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c74070630 msgr2=0x7f0c740b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c74070630 0x7f0c740b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 -- 172.21.15.19:0/3892853070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c740b4740 con 0x7f0c7415adf0 2024-04-25T11:28:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.253+0000 7f0c79000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c740b63c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00be90 tx=0x7f0c6c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.254+0000 7f0c6be00640 1 -- 172.21.15.19:0/3892853070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c019070 con 0x7f0c7415adf0 2024-04-25T11:28:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.254+0000 7f0c6be00640 1 -- 172.21.15.19:0/3892853070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c6c014440 con 0x7f0c7415adf0 2024-04-25T11:28:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.254+0000 7f0c6be00640 1 -- 172.21.15.19:0/3892853070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c014870 con 0x7f0c7415adf0 2024-04-25T11:28:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.254+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c740b4a20 con 0x7f0c7415adf0 2024-04-25T11:28:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.254+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c740b4fe0 con 0x7f0c7415adf0 2024-04-25T11:28:16.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.255+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c74072de0 con 0x7f0c7415adf0 2024-04-25T11:28:16.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.256+0000 7f0c6be00640 1 -- 172.21.15.19:0/3892853070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c6c01c070 con 0x7f0c7415adf0 2024-04-25T11:28:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.257+0000 7f0c6be00640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c500744b0 0x7f0c500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.257+0000 7f0c6be00640 1 -- 172.21.15.19:0/3892853070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f0c6c0952d0 con 0x7f0c7415adf0 2024-04-25T11:28:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.258+0000 7f0c73e00640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c500744b0 0x7f0c500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.259+0000 7f0c73e00640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c500744b0 0x7f0c500769c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c740cfd00 tx=0x7f0c64006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:16.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.262+0000 7f0c6be00640 1 -- 172.21.15.19:0/3892853070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c74072de0 con 0x7f0c7415adf0 2024-04-25T11:28:16.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.469+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c74050740 con 0x7f0c7415adf0 2024-04-25T11:28:16.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.470+0000 7f0c6be00640 1 -- 172.21.15.19:0/3892853070 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f0c6c05f9a0 con 0x7f0c7415adf0 2024-04-25T11:28:16.463 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c500744b0 msgr2=0x7f0c500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c500744b0 0x7f0c500769c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c740cfd00 tx=0x7f0c64006bf0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 msgr2=0x7f0c740b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c740b63c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00be90 tx=0x7f0c6c00bf80 comp rx=0 tx=0).stop 2024-04-25T11:28:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 shutdown_connections 2024-04-25T11:28:16.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c500744b0 0x7f0c500769c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c74070630 0x7f0c740b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c7415adf0 0x7f0c740b63c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 --2- 172.21.15.19:0/3892853070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c7406db10 0x7f0c741492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.473+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 >> 172.21.15.19:0/3892853070 conn(0x7f0c74073090 msgr2=0x7f0c7415f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.474+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 shutdown_connections 2024-04-25T11:28:16.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.474+0000 7f0c7b000640 1 -- 172.21.15.19:0/3892853070 wait complete. 2024-04-25T11:28:16.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.784+0000 7f66d7600640 1 Processor -- start 2024-04-25T11:28:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.784+0000 7f66d7600640 1 -- start start 2024-04-25T11:28:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.786+0000 7f66d7600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66d00cd9a0 0x7f66d015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.786+0000 7f66d7600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66d00b0af0 0x7f66d006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.786+0000 7f66d7600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d006db10 0x7f66d0161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.786+0000 7f66d7600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f66d00ceab0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.786+0000 7f66d7600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f66d006df20 con 0x7f66d00b0af0 2024-04-25T11:28:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.786+0000 7f66d7600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66d00cdea0 con 0x7f66d006db10 2024-04-25T11:28:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66d00cd9a0 0x7f66d015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66d00cd9a0 0x7f66d015f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54532/0 (socket says 172.21.15.19:54532) 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 -- 172.21.15.19:0/131968512 learned_addr learned my addr 172.21.15.19:0/131968512 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66c7e00640 1 --2- 172.21.15.19:0/131968512 >> v2:172.21.15.169:3300/0 conn(0x7f66d00b0af0 0x7f66d006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d5600640 1 --2- 172.21.15.19:0/131968512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d006db10 0x7f66d0161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 -- 172.21.15.19:0/131968512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d006db10 msgr2=0x7f66d0161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 --2- 172.21.15.19:0/131968512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d006db10 0x7f66d0161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 -- 172.21.15.19:0/131968512 >> v2:172.21.15.169:3300/0 conn(0x7f66d00b0af0 msgr2=0x7f66d006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 --2- 172.21.15.19:0/131968512 >> v2:172.21.15.169:3300/0 conn(0x7f66d00b0af0 0x7f66d006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66d4c00640 1 -- 172.21.15.19:0/131968512 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d00ce180 con 0x7f66d00cd9a0 2024-04-25T11:28:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.787+0000 7f66c7e00640 1 --2- 172.21.15.19:0/131968512 >> v2:172.21.15.169:3300/0 conn(0x7f66d00b0af0 0x7f66d006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.788+0000 7f66d4c00640 1 --2- 172.21.15.19:0/131968512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d015f360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66d0162f90 tx=0x7f66c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4a470c23d60af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.789+0000 7f66cfe00640 1 -- 172.21.15.19:0/131968512 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0036070 con 0x7f66d00cd9a0 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.789+0000 7f66cfe00640 1 -- 172.21.15.19:0/131968512 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c0031600 con 0x7f66d00cd9a0 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.789+0000 7f66cfe00640 1 -- 172.21.15.19:0/131968512 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00319d0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 -- 172.21.15.19:0/131968512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 msgr2=0x7f66d015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 --2- 172.21.15.19:0/131968512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d015f360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66d0162f90 tx=0x7f66c002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 -- 172.21.15.19:0/131968512 shutdown_connections 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 --2- 172.21.15.19:0/131968512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d006db10 0x7f66d0161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 --2- 172.21.15.19:0/131968512 >> v2:172.21.15.169:3300/0 conn(0x7f66d00b0af0 0x7f66d006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 --2- 172.21.15.19:0/131968512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d015f360 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 -- 172.21.15.19:0/131968512 >> 172.21.15.19:0/131968512 conn(0x7f66d0073090 msgr2=0x7f66d00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 -- 172.21.15.19:0/131968512 shutdown_connections 2024-04-25T11:28:16.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.790+0000 7f66d7600640 1 -- 172.21.15.19:0/131968512 wait complete. 2024-04-25T11:28:16.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.793+0000 7f66d7600640 1 Processor -- start 2024-04-25T11:28:16.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.793+0000 7f66d7600640 1 -- start start 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d7600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d006db10 0x7f66d01f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d7600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d00b0af0 0x7f66d01f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d7600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d0145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d7600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66d00b0f00 con 0x7f66d00cd9a0 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d7600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66d00ce740 con 0x7f66d006db10 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d7600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66d00cef40 con 0x7f66d00b0af0 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d0145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d0145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54546/0 (socket says 172.21.15.19:54546) 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d5600640 1 -- 172.21.15.19:0/2687216087 learned_addr learned my addr 172.21.15.19:0/2687216087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.795+0000 7f66d4c00640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d006db10 0x7f66d01f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66c7e00640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d00b0af0 0x7f66d01f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66d5600640 1 -- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d00b0af0 msgr2=0x7f66d01f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66d5600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d00b0af0 0x7f66d01f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66d5600640 1 -- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d006db10 msgr2=0x7f66d01f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66d5600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d006db10 0x7f66d01f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66d5600640 1 -- 172.21.15.19:0/2687216087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01f1080 con 0x7f66d00cd9a0 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66d4c00640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d006db10 0x7f66d01f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66d5600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d0145f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66c800d510 tx=0x7f66c800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.796+0000 7f66cd600640 1 -- 172.21.15.19:0/2687216087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c80125f0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.797+0000 7f66cd600640 1 -- 172.21.15.19:0/2687216087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c8019070 con 0x7f66d00cd9a0 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.797+0000 7f66cd600640 1 -- 172.21.15.19:0/2687216087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c80129f0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.797+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d0145460 con 0x7f66d00cd9a0 2024-04-25T11:28:16.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.797+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01446a0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.798+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d0070ea0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.799+0000 7f66cd600640 1 -- 172.21.15.19:0/2687216087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66c8012b90 con 0x7f66d00cd9a0 2024-04-25T11:28:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.800+0000 7f66cd600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66ac0744b0 0x7f66ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.800+0000 7f66cd600640 1 -- 172.21.15.19:0/2687216087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f66c80956b0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.801+0000 7f66d4c00640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66ac0744b0 0x7f66ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:16.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.802+0000 7f66d4c00640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66ac0744b0 0x7f66ac0769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66c0033000 tx=0x7f66c0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:16.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:16.805+0000 7f66cd600640 1 -- 172.21.15.19:0/2687216087 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66d0070ea0 con 0x7f66d00cd9a0 2024-04-25T11:28:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.001+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66d0050740 con 0x7f66d00cd9a0 2024-04-25T11:28:16.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.001+0000 7f66cd600640 1 -- 172.21.15.19:0/2687216087 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f66c805fd80 con 0x7f66d00cd9a0 2024-04-25T11:28:16.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66ac0744b0 msgr2=0x7f66ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66ac0744b0 0x7f66ac0769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66c0033000 tx=0x7f66c0009000 comp rx=0 tx=0).stop 2024-04-25T11:28:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 msgr2=0x7f66d0145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d0145f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66c800d510 tx=0x7f66c800d9d0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 shutdown_connections 2024-04-25T11:28:16.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66ac0744b0 0x7f66ac0769c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66d00cd9a0 0x7f66d0145f80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66d00b0af0 0x7f66d01f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 --2- 172.21.15.19:0/2687216087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66d006db10 0x7f66d01f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:16.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 >> 172.21.15.19:0/2687216087 conn(0x7f66d0073090 msgr2=0x7f66d00b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:16.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.004+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 shutdown_connections 2024-04-25T11:28:16.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.005+0000 7f66d7600640 1 -- 172.21.15.19:0/2687216087 wait complete. 2024-04-25T11:28:17.021 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:28:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.313+0000 7fa0e7e00640 1 Processor -- start 2024-04-25T11:28:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.313+0000 7fa0e7e00640 1 -- start start 2024-04-25T11:28:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.316+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e01443c0 0x7fa0e006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.316+0000 7fa0e7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0e0070370 0x7fa0e00cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.316+0000 7fa0e7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e00cff30 con 0x7fa0e00cfb20 2024-04-25T11:28:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.316+0000 7fa0e7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e00708d0 con 0x7fa0e0070370 2024-04-25T11:28:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.316+0000 7fa0e7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e015f160 con 0x7fa0e01443c0 2024-04-25T11:28:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.317+0000 7fa0e5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e01443c0 0x7fa0e006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:17.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.317+0000 7fa0e4a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0e0070370 0x7fa0e00cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:17.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.317+0000 7fa0e5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e01443c0 0x7fa0e006fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:33292/0 (socket says 172.21.15.19:33292) 2024-04-25T11:28:17.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.317+0000 7fa0e5400640 1 -- 172.21.15.19:0/2456473875 learned_addr learned my addr 172.21.15.19:0/2456473875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:17.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.318+0000 7fa0e4a00640 1 -- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e01443c0 msgr2=0x7fa0e006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:17.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.318+0000 7fa0e4a00640 1 --2- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e01443c0 0x7fa0e006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.318+0000 7fa0e4a00640 1 -- 172.21.15.19:0/2456473875 >> v1:172.21.15.169:6789/0 conn(0x7fa0e00cfb20 legacy=0x7fa0e00ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:17.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.318+0000 7fa0e4a00640 1 -- 172.21.15.19:0/2456473875 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e015f5f0 con 0x7fa0e0070370 2024-04-25T11:28:17.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.318+0000 7fa0e5400640 1 --2- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e01443c0 0x7fa0e006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:28:17.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.319+0000 7fa0e4a00640 1 --2- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e0070370 0x7fa0e00cf5e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0097a0 tx=0x7fa0cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ddd88815ee51067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:17.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.319+0000 7fa0dfe00640 1 -- 172.21.15.19:0/2456473875 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc037070 con 0x7fa0e0070370 2024-04-25T11:28:17.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.319+0000 7fa0dfe00640 1 -- 172.21.15.19:0/2456473875 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0cc032440 con 0x7fa0e0070370 2024-04-25T11:28:17.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.319+0000 7fa0dfe00640 1 -- 172.21.15.19:0/2456473875 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc032810 con 0x7fa0e0070370 2024-04-25T11:28:17.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 -- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e0070370 msgr2=0x7fa0e00cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:17.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e0070370 0x7fa0e00cf5e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0097a0 tx=0x7fa0cc02a180 comp rx=0 tx=0).stop 2024-04-25T11:28:17.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 -- 172.21.15.19:0/2456473875 shutdown_connections 2024-04-25T11:28:17.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e0070370 0x7fa0e00cf5e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/2456473875 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e01443c0 0x7fa0e006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 -- 172.21.15.19:0/2456473875 >> 172.21.15.19:0/2456473875 conn(0x7fa0e00d1ae0 msgr2=0x7fa0e00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:17.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 -- 172.21.15.19:0/2456473875 shutdown_connections 2024-04-25T11:28:17.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.320+0000 7fa0e7e00640 1 -- 172.21.15.19:0/2456473875 wait complete. 2024-04-25T11:28:17.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.323+0000 7fa0e7e00640 1 Processor -- start 2024-04-25T11:28:17.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.323+0000 7fa0e7e00640 1 -- start start 2024-04-25T11:28:17.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.324+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e0070370 0x7fa0e01f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:17.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.324+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00cfb20 0x7fa0e01f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.324+0000 7fa0e7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e01443c0 0x7fa0e00c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.324+0000 7fa0e7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e015edf0 con 0x7fa0e00cfb20 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.324+0000 7fa0e7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e015eb10 con 0x7fa0e01443c0 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.324+0000 7fa0e7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e006d5d0 con 0x7fa0e0070370 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00cfb20 0x7fa0e01f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00cfb20 0x7fa0e01f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54558/0 (socket says 172.21.15.19:54558) 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 -- 172.21.15.19:0/3159803655 learned_addr learned my addr 172.21.15.19:0/3159803655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e5e00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e01443c0 0x7fa0e00c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e5400640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e0070370 0x7fa0e01f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 -- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e0070370 msgr2=0x7fa0e01f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e0070370 0x7fa0e01f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 -- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e01443c0 msgr2=0x7fa0e00c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e01443c0 0x7fa0e00c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.325+0000 7fa0e4a00640 1 -- 172.21.15.19:0/3159803655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e00b5070 con 0x7fa0e00cfb20 2024-04-25T11:28:17.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.326+0000 7fa0e4a00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00cfb20 0x7fa0e01f6d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc02a1c0 tx=0x7fa0cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.326+0000 7fa0dd600640 1 -- 172.21.15.19:0/3159803655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc041020 con 0x7fa0e00cfb20 2024-04-25T11:28:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.326+0000 7fa0dd600640 1 -- 172.21.15.19:0/3159803655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0cc030700 con 0x7fa0e00cfb20 2024-04-25T11:28:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.326+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e00b5350 con 0x7fa0e00cfb20 2024-04-25T11:28:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.326+0000 7fa0dd600640 1 -- 172.21.15.19:0/3159803655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc037040 con 0x7fa0e00cfb20 2024-04-25T11:28:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.326+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e00b5910 con 0x7fa0e00cfb20 2024-04-25T11:28:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.327+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e0050740 con 0x7fa0e00cfb20 2024-04-25T11:28:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.329+0000 7fa0dd600640 1 -- 172.21.15.19:0/3159803655 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa0cc044020 con 0x7fa0e00cfb20 2024-04-25T11:28:17.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.330+0000 7fa0dd600640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0c00744b0 0x7fa0c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:17.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.330+0000 7fa0dd600640 1 -- 172.21.15.19:0/3159803655 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fa0cc0b5bb0 con 0x7fa0e00cfb20 2024-04-25T11:28:17.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.330+0000 7fa0e5400640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0c00744b0 0x7fa0c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:17.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.331+0000 7fa0e5400640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0c00744b0 0x7fa0c00769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00d1860 tx=0x7fa0d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:17.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.334+0000 7fa0dd600640 1 -- 172.21.15.19:0/3159803655 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0cc04b030 con 0x7fa0e00cfb20 2024-04-25T11:28:17.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.549+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa0e00b4e00 con 0x7fa0e00cfb20 2024-04-25T11:28:17.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.549+0000 7fa0dd600640 1 -- 172.21.15.19:0/3159803655 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fa0cc080280 con 0x7fa0e00cfb20 2024-04-25T11:28:17.541 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:28:17.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0c00744b0 msgr2=0x7fa0c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0c00744b0 0x7fa0c00769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa0e00d1860 tx=0x7fa0d4008040 comp rx=0 tx=0).stop 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00cfb20 msgr2=0x7fa0e01f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00cfb20 0x7fa0e01f6d40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc02a1c0 tx=0x7fa0cc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 shutdown_connections 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0c00744b0 0x7fa0c00769c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0e01443c0 0x7fa0e00c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00cfb20 0x7fa0e01f6d40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 --2- 172.21.15.19:0/3159803655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0e0070370 0x7fa0e01f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.552+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 >> 172.21.15.19:0/3159803655 conn(0x7fa0e00d1ae0 msgr2=0x7fa0e00b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.553+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 shutdown_connections 2024-04-25T11:28:17.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:17.553+0000 7fa0e7e00640 1 -- 172.21.15.19:0/3159803655 wait complete. 2024-04-25T11:28:17.569 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:28:17.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:17.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:28:17.665 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:28:17.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:17.665 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:28:17.737 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:28:17.737 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:17.797 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:28:17.797 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:17.850 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:28:17.850 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:17.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:18.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.203+0000 7f37a1000640 1 Processor -- start 2024-04-25T11:28:18.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.203+0000 7f37a1000640 1 -- start start 2024-04-25T11:28:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.205+0000 7f37a1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f379c06c5c0 0x7f379c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.205+0000 7f37a1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f379c144370 0x7f379c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.205+0000 7f37a1000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379c0b50e0 con 0x7f379c0b4cd0 2024-04-25T11:28:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.205+0000 7f37a1000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f379c0d1230 con 0x7f379c06c5c0 2024-04-25T11:28:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.205+0000 7f37a1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f379c0d06b0 con 0x7f379c144370 2024-04-25T11:28:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.205+0000 7f379b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f379c06c5c0 0x7f379c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.205+0000 7f379b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f379c06c5c0 0x7f379c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54568/0 (socket says 172.21.15.19:54568) 2024-04-25T11:28:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379b400640 1 -- 172.21.15.19:0/2280834335 learned_addr learned my addr 172.21.15.19:0/2280834335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379b400640 1 -- 172.21.15.19:0/2280834335 >> v2:172.21.15.169:3300/0 conn(0x7f379c144370 msgr2=0x7f379c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379aa00640 1 --2- 172.21.15.19:0/2280834335 >> v2:172.21.15.169:3300/0 conn(0x7f379c144370 0x7f379c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379b400640 1 --2- 172.21.15.19:0/2280834335 >> v2:172.21.15.169:3300/0 conn(0x7f379c144370 0x7f379c0b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379b400640 1 -- 172.21.15.19:0/2280834335 >> v1:172.21.15.19:6789/0 conn(0x7f379c0b4cd0 legacy=0x7f379c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379b400640 1 -- 172.21.15.19:0/2280834335 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c0d0990 con 0x7f379c06c5c0 2024-04-25T11:28:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379a000640 1 -- 172.21.15.19:0/2280834335 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918586879 0 0) 0x7f3794001490 con 0x7f379c0b4cd0 2024-04-25T11:28:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.206+0000 7f379aa00640 1 --2- 172.21.15.19:0/2280834335 >> v2:172.21.15.169:3300/0 conn(0x7f379c144370 0x7f379c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.207+0000 7f379b400640 1 --2- 172.21.15.19:0/2280834335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cfa70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f379c0b5550 tx=0x7f378c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca47c3bd31ebab9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.207+0000 7f379a000640 1 -- 172.21.15.19:0/2280834335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c037070 con 0x7f379c06c5c0 2024-04-25T11:28:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.207+0000 7f379a000640 1 -- 172.21.15.19:0/2280834335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c032440 con 0x7f379c06c5c0 2024-04-25T11:28:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.207+0000 7f379a000640 1 -- 172.21.15.19:0/2280834335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c032810 con 0x7f379c06c5c0 2024-04-25T11:28:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 -- 172.21.15.19:0/2280834335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 msgr2=0x7f379c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 --2- 172.21.15.19:0/2280834335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cfa70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f379c0b5550 tx=0x7f378c02a270 comp rx=0 tx=0).stop 2024-04-25T11:28:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 -- 172.21.15.19:0/2280834335 shutdown_connections 2024-04-25T11:28:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 --2- 172.21.15.19:0/2280834335 >> v2:172.21.15.169:3300/0 conn(0x7f379c144370 0x7f379c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 --2- 172.21.15.19:0/2280834335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cfa70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 -- 172.21.15.19:0/2280834335 >> 172.21.15.19:0/2280834335 conn(0x7f379c0d1720 msgr2=0x7f379c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 -- 172.21.15.19:0/2280834335 shutdown_connections 2024-04-25T11:28:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.208+0000 7f37a1000640 1 -- 172.21.15.19:0/2280834335 wait complete. 2024-04-25T11:28:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.211+0000 7f37a1000640 1 Processor -- start 2024-04-25T11:28:18.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.211+0000 7f37a1000640 1 -- start start 2024-04-25T11:28:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.212+0000 7f37a1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.212+0000 7f37a1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379c0b4cd0 0x7f379c1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.212+0000 7f37a1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379c144370 0x7f379c161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.212+0000 7f37a1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f379c0d02d0 con 0x7f379c06c5c0 2024-04-25T11:28:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.212+0000 7f37a1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f379c0d0f50 con 0x7f379c0b4cd0 2024-04-25T11:28:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.212+0000 7f37a1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f379c0d0450 con 0x7f379c144370 2024-04-25T11:28:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54584/0 (socket says 172.21.15.19:54584) 2024-04-25T11:28:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 -- 172.21.15.19:0/653593930 learned_addr learned my addr 172.21.15.19:0/653593930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 -- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379c144370 msgr2=0x7f379c161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379c144370 0x7f379c161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 -- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379c0b4cd0 msgr2=0x7f379c1f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379aa00640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379c0b4cd0 0x7f379c1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379c0b4cd0 0x7f379c1f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.213+0000 7f379b400640 1 -- 172.21.15.19:0/653593930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c1459e0 con 0x7f379c06c5c0 2024-04-25T11:28:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.214+0000 7f379b400640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cbb00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f378c000c00 tx=0x7f378c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.214+0000 7f3793400640 1 -- 172.21.15.19:0/653593930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c037040 con 0x7f379c06c5c0 2024-04-25T11:28:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.214+0000 7f3793400640 1 -- 172.21.15.19:0/653593930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c004230 con 0x7f379c06c5c0 2024-04-25T11:28:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.214+0000 7f3793400640 1 -- 172.21.15.19:0/653593930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c03c670 con 0x7f379c06c5c0 2024-04-25T11:28:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.214+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c145cc0 con 0x7f379c06c5c0 2024-04-25T11:28:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.214+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c146280 con 0x7f379c06c5c0 2024-04-25T11:28:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.215+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c050740 con 0x7f379c06c5c0 2024-04-25T11:28:18.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.217+0000 7f3793400640 1 -- 172.21.15.19:0/653593930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f378c03a070 con 0x7f379c06c5c0 2024-04-25T11:28:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.218+0000 7f3793400640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f377c0744e0 0x7f377c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.218+0000 7f3793400640 1 -- 172.21.15.19:0/653593930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f378c0b5a50 con 0x7f379c06c5c0 2024-04-25T11:28:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.219+0000 7f379aa00640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f377c0744e0 0x7f377c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.220+0000 7f379aa00640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f377c0744e0 0x7f377c0769f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f379c143cf0 tx=0x7f3788008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:18.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.223+0000 7f3793400640 1 -- 172.21.15.19:0/653593930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f378c041020 con 0x7f379c06c5c0 2024-04-25T11:28:18.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.433+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f379c146ea0 con 0x7f379c06c5c0 2024-04-25T11:28:18.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.435+0000 7f3793400640 1 -- 172.21.15.19:0/653593930 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f378c0801b0 con 0x7f379c06c5c0 2024-04-25T11:28:18.427 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:18.428 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f377c0744e0 msgr2=0x7f377c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f377c0744e0 0x7f377c0769f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f379c143cf0 tx=0x7f3788008040 comp rx=0 tx=0).stop 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 msgr2=0x7f379c0cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cbb00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f378c000c00 tx=0x7f378c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 shutdown_connections 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f377c0744e0 0x7f377c0769f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f379c0b4cd0 0x7f379c1f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f379c144370 0x7f379c161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 --2- 172.21.15.19:0/653593930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f379c06c5c0 0x7f379c0cbb00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 >> 172.21.15.19:0/653593930 conn(0x7f379c0d1720 msgr2=0x7f379c072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:18.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 shutdown_connections 2024-04-25T11:28:18.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.439+0000 7f37a1000640 1 -- 172.21.15.19:0/653593930 wait complete. 2024-04-25T11:28:18.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:18.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:18.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.743+0000 7f7c45a00640 1 Processor -- start 2024-04-25T11:28:18.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.744+0000 7f7c45a00640 1 -- start start 2024-04-25T11:28:18.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c45a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c4006dce0 0x7f7c4015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c45a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c4015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c45a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c40070000 con 0x7f7c40144340 2024-04-25T11:28:18.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c45a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7c40144750 con 0x7f7c4006dce0 2024-04-25T11:28:18.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c45a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4006f240 con 0x7f7c4015ec90 2024-04-25T11:28:18.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c3fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7c40144340 0x7f7c4006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43976/0 (socket says 172.21.15.19:43976) 2024-04-25T11:28:18.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c3fe00640 1 -- 172.21.15.19:0/236138658 learned_addr learned my addr 172.21.15.19:0/236138658 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:18.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.746+0000 7f7c3f400640 1 --2- 172.21.15.19:0/236138658 >> v2:172.21.15.19:3300/0 conn(0x7f7c4006dce0 0x7f7c4015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.747+0000 7f7c3f400640 1 -- 172.21.15.19:0/236138658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 msgr2=0x7f7c4015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.747+0000 7f7c45000640 1 --2- 172.21.15.19:0/236138658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c4015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.747+0000 7f7c3ea00640 1 -- 172.21.15.19:0/236138658 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158795523 0 0) 0x7f7c40070000 con 0x7f7c40144340 2024-04-25T11:28:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.747+0000 7f7c3f400640 1 --2- 172.21.15.19:0/236138658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c4015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.747+0000 7f7c3f400640 1 -- 172.21.15.19:0/236138658 >> v1:172.21.15.19:6789/0 conn(0x7f7c40144340 legacy=0x7f7c4006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.747+0000 7f7c3f400640 1 -- 172.21.15.19:0/236138658 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4006f520 con 0x7f7c4006dce0 2024-04-25T11:28:18.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.747+0000 7f7c45000640 1 --2- 172.21.15.19:0/236138658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c4015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.748+0000 7f7c3f400640 1 --2- 172.21.15.19:0/236138658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c4006dce0 0x7f7c4015e750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0097a0 tx=0x7f7c2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2bcb84c1ce5475 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c3ea00640 1 -- 172.21.15.19:0/236138658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c037070 con 0x7f7c4006dce0 2024-04-25T11:28:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c3ea00640 1 -- 172.21.15.19:0/236138658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c032440 con 0x7f7c4006dce0 2024-04-25T11:28:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c3ea00640 1 -- 172.21.15.19:0/236138658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c032810 con 0x7f7c4006dce0 2024-04-25T11:28:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c45a00640 1 -- 172.21.15.19:0/236138658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c4006dce0 msgr2=0x7f7c4015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c45a00640 1 --2- 172.21.15.19:0/236138658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c4006dce0 0x7f7c4015e750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0097a0 tx=0x7f7c2c02a180 comp rx=0 tx=0).stop 2024-04-25T11:28:18.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c45a00640 1 -- 172.21.15.19:0/236138658 shutdown_connections 2024-04-25T11:28:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c45a00640 1 --2- 172.21.15.19:0/236138658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c4015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c45a00640 1 --2- 172.21.15.19:0/236138658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c4006dce0 0x7f7c4015e750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.749+0000 7f7c45a00640 1 -- 172.21.15.19:0/236138658 >> 172.21.15.19:0/236138658 conn(0x7f7c40072f40 msgr2=0x7f7c40073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.750+0000 7f7c45a00640 1 -- 172.21.15.19:0/236138658 shutdown_connections 2024-04-25T11:28:18.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.750+0000 7f7c45a00640 1 -- 172.21.15.19:0/236138658 wait complete. 2024-04-25T11:28:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.752+0000 7f7c45a00640 1 Processor -- start 2024-04-25T11:28:18.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.752+0000 7f7c45a00640 1 -- start start 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c40144340 0x7f7c4013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c40145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c40147480 0x7f7c40146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4015f0a0 con 0x7f7c40147480 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4006fae0 con 0x7f7c40144340 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4015f220 con 0x7f7c4015ec90 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c40147480 0x7f7c40146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c40147480 0x7f7c40146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54602/0 (socket says 172.21.15.19:54602) 2024-04-25T11:28:18.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.754+0000 7f7c45000640 1 -- 172.21.15.19:0/2110474245 learned_addr learned my addr 172.21.15.19:0/2110474245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c3fe00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c40144340 0x7f7c4013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c3f400640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c40145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c45000640 1 -- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 msgr2=0x7f7c40145ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c45000640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c40145ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c45000640 1 -- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c40144340 msgr2=0x7f7c4013d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c45000640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c40144340 0x7f7c4013d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c45000640 1 -- 172.21.15.19:0/2110474245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c40158700 con 0x7f7c40147480 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c3f400640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c40145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c3fe00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c40144340 0x7f7c4013d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c45000640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c40147480 0x7f7c40146400 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800cdb0 tx=0x7f7c3800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c37e00640 1 -- 172.21.15.19:0/2110474245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800f680 con 0x7f7c40147480 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c37e00640 1 -- 172.21.15.19:0/2110474245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3800f840 con 0x7f7c40147480 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c37e00640 1 -- 172.21.15.19:0/2110474245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800fc30 con 0x7f7c40147480 2024-04-25T11:28:18.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.755+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c401600d0 con 0x7f7c40147480 2024-04-25T11:28:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.756+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4015b870 con 0x7f7c40147480 2024-04-25T11:28:18.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.757+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c400b1360 con 0x7f7c40147480 2024-04-25T11:28:18.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.758+0000 7f7c37e00640 1 -- 172.21.15.19:0/2110474245 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c3800e3e0 con 0x7f7c40147480 2024-04-25T11:28:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.759+0000 7f7c37e00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c200744b0 0x7f7c200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.759+0000 7f7c37e00640 1 -- 172.21.15.19:0/2110474245 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f7c38095fd0 con 0x7f7c40147480 2024-04-25T11:28:18.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.760+0000 7f7c3fe00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c200744b0 0x7f7c200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:18.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.761+0000 7f7c3fe00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c200744b0 0x7f7c200769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7c30000f60 tx=0x7f7c30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:18.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.765+0000 7f7c37e00640 1 -- 172.21.15.19:0/2110474245 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c38014030 con 0x7f7c40147480 2024-04-25T11:28:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.971+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c40076c60 con 0x7f7c40147480 2024-04-25T11:28:18.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.972+0000 7f7c37e00640 1 -- 172.21.15.19:0/2110474245 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f7c38060730 con 0x7f7c40147480 2024-04-25T11:28:18.963 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.974+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c200744b0 msgr2=0x7f7c200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.974+0000 7f7c45a00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c200744b0 0x7f7c200769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7c30000f60 tx=0x7f7c30009040 comp rx=0 tx=0).stop 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c40147480 msgr2=0x7f7c40146400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c40147480 0x7f7c40146400 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800cdb0 tx=0x7f7c3800a650 comp rx=0 tx=0).stop 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 shutdown_connections 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c200744b0 0x7f7c200769c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c40147480 0x7f7c40146400 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c4015ec90 0x7f7c40145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 --2- 172.21.15.19:0/2110474245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c40144340 0x7f7c4013d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:18.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 >> 172.21.15.19:0/2110474245 conn(0x7f7c40072f40 msgr2=0x7f7c4015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 shutdown_connections 2024-04-25T11:28:18.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:18.975+0000 7f7c45a00640 1 -- 172.21.15.19:0/2110474245 wait complete. 2024-04-25T11:28:18.996 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.293+0000 7f082ba00640 1 Processor -- start 2024-04-25T11:28:19.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.293+0000 7f082ba00640 1 -- start start 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f082ba00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08240cd9a0 0x7f082415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f082ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f082406dce0 0x7f082415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f082ba00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08240cdea0 con 0x7f08240b0af0 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f082ba00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f08240ceab0 con 0x7f08240cd9a0 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f082ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08240ce180 con 0x7f082406dce0 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f0823e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f08240b0af0 0x7f082406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43984/0 (socket says 172.21.15.19:43984) 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f0823e00640 1 -- 172.21.15.19:0/552667863 learned_addr learned my addr 172.21.15.19:0/552667863 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:19.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.296+0000 7f0829000640 1 --2- 172.21.15.19:0/552667863 >> v2:172.21.15.19:3300/0 conn(0x7f08240cd9a0 0x7f082415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.297+0000 7f0829a00640 1 --2- 172.21.15.19:0/552667863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f082406dce0 0x7f082415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.297+0000 7f0823400640 1 -- 172.21.15.19:0/552667863 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324236258 0 0) 0x7f08240cdea0 con 0x7f08240b0af0 2024-04-25T11:28:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.297+0000 7f0823400640 1 -- 172.21.15.19:0/552667863 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0808003520 con 0x7f08240b0af0 2024-04-25T11:28:19.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.297+0000 7f0829000640 1 -- 172.21.15.19:0/552667863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f082406dce0 msgr2=0x7f082415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.297+0000 7f0829000640 1 --2- 172.21.15.19:0/552667863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f082406dce0 0x7f082415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.297+0000 7f0829000640 1 -- 172.21.15.19:0/552667863 >> v1:172.21.15.19:6789/0 conn(0x7f08240b0af0 legacy=0x7f082406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.297+0000 7f0829000640 1 -- 172.21.15.19:0/552667863 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08240ce460 con 0x7f08240cd9a0 2024-04-25T11:28:19.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.298+0000 7f0829000640 1 --2- 172.21.15.19:0/552667863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240cd9a0 0x7f082415f360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d90 tx=0x7f081402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9433b3f8cc0cd48a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.298+0000 7f0823400640 1 -- 172.21.15.19:0/552667863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814007740 con 0x7f08240cd9a0 2024-04-25T11:28:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.298+0000 7f0823400640 1 -- 172.21.15.19:0/552667863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08140078e0 con 0x7f08240cd9a0 2024-04-25T11:28:19.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.299+0000 7f0823400640 1 -- 172.21.15.19:0/552667863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814007c90 con 0x7f08240cd9a0 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.299+0000 7f082ba00640 1 -- 172.21.15.19:0/552667863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240cd9a0 msgr2=0x7f082415f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.299+0000 7f082ba00640 1 --2- 172.21.15.19:0/552667863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240cd9a0 0x7f082415f360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0814009d90 tx=0x7f081402a660 comp rx=0 tx=0).stop 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.299+0000 7f082ba00640 1 -- 172.21.15.19:0/552667863 shutdown_connections 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.299+0000 7f082ba00640 1 --2- 172.21.15.19:0/552667863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f082406dce0 0x7f082415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.299+0000 7f082ba00640 1 --2- 172.21.15.19:0/552667863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240cd9a0 0x7f082415f360 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.299+0000 7f082ba00640 1 -- 172.21.15.19:0/552667863 >> 172.21.15.19:0/552667863 conn(0x7f0824073090 msgr2=0x7f08240734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.300+0000 7f082ba00640 1 -- 172.21.15.19:0/552667863 shutdown_connections 2024-04-25T11:28:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.300+0000 7f082ba00640 1 -- 172.21.15.19:0/552667863 wait complete. 2024-04-25T11:28:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.302+0000 7f082ba00640 1 Processor -- start 2024-04-25T11:28:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.302+0000 7f082ba00640 1 -- start start 2024-04-25T11:28:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f082ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240b0af0 0x7f0824142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f082ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08240cd9a0 0x7f0824142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f082ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0824143480 0x7f08240b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f082ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08240cef40 con 0x7f08240b0af0 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f082ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08240b0f00 con 0x7f0824143480 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f082ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08240cf0c0 con 0x7f08240cd9a0 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f0829000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240b0af0 0x7f0824142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f0829000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240b0af0 0x7f0824142660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54632/0 (socket says 172.21.15.19:54632) 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f0829000640 1 -- 172.21.15.19:0/4007389812 learned_addr learned my addr 172.21.15.19:0/4007389812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f0823e00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08240cd9a0 0x7f0824142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.304+0000 7f0829a00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0824143480 0x7f08240b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.305+0000 7f0829000640 1 -- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08240cd9a0 msgr2=0x7f0824142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.305+0000 7f0829000640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08240cd9a0 0x7f0824142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.305+0000 7f0829000640 1 -- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0824143480 msgr2=0x7f08240b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.305+0000 7f0829000640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0824143480 0x7f08240b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.305+0000 7f0829000640 1 -- 172.21.15.19:0/4007389812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0824157680 con 0x7f08240b0af0 2024-04-25T11:28:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.305+0000 7f0829000640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240b0af0 0x7f0824142660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0814009eb0 tx=0x7f0814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.305+0000 7f0820c00640 1 -- 172.21.15.19:0/4007389812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081403f070 con 0x7f08240b0af0 2024-04-25T11:28:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.306+0000 7f0820c00640 1 -- 172.21.15.19:0/4007389812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0814031c40 con 0x7f08240b0af0 2024-04-25T11:28:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.306+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0824157960 con 0x7f08240b0af0 2024-04-25T11:28:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.306+0000 7f0820c00640 1 -- 172.21.15.19:0/4007389812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814009450 con 0x7f08240b0af0 2024-04-25T11:28:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.306+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0824157f20 con 0x7f08240b0af0 2024-04-25T11:28:19.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.307+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0824158160 con 0x7f08240b0af0 2024-04-25T11:28:19.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.308+0000 7f0820c00640 1 -- 172.21.15.19:0/4007389812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f081403b410 con 0x7f08240b0af0 2024-04-25T11:28:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.309+0000 7f0820c00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08000744b0 0x7f08000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.309+0000 7f0820c00640 1 -- 172.21.15.19:0/4007389812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f08140b64f0 con 0x7f08240b0af0 2024-04-25T11:28:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.310+0000 7f0823e00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08000744b0 0x7f08000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.311+0000 7f0823e00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08000744b0 0x7f08000769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f080c007fd0 tx=0x7f080c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.315+0000 7f0820c00640 1 -- 172.21.15.19:0/4007389812 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0824158160 con 0x7f08240b0af0 2024-04-25T11:28:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.523+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0824050740 con 0x7f08240b0af0 2024-04-25T11:28:19.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.524+0000 7f0820c00640 1 -- 172.21.15.19:0/4007389812 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f081404b070 con 0x7f08240b0af0 2024-04-25T11:28:19.517 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08000744b0 msgr2=0x7f08000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08000744b0 0x7f08000769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f080c007fd0 tx=0x7f080c009040 comp rx=0 tx=0).stop 2024-04-25T11:28:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240b0af0 msgr2=0x7f0824142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240b0af0 0x7f0824142660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0814009eb0 tx=0x7f0814004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 shutdown_connections 2024-04-25T11:28:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08000744b0 0x7f08000769c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0824143480 0x7f08240b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08240cd9a0 0x7f0824142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 --2- 172.21.15.19:0/4007389812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08240b0af0 0x7f0824142660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 >> 172.21.15.19:0/4007389812 conn(0x7f0824073090 msgr2=0x7f08240b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 shutdown_connections 2024-04-25T11:28:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.527+0000 7f082ba00640 1 -- 172.21.15.19:0/4007389812 wait complete. 2024-04-25T11:28:19.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.843+0000 7f2505000640 1 Processor -- start 2024-04-25T11:28:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.843+0000 7f2505000640 1 -- start start 2024-04-25T11:28:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.845+0000 7f2505000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d3d40 0x7f2500160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.845+0000 7f2505000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25000b7c50 0x7f25000b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.845+0000 7f2505000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25000b9110 con 0x7f25000d4510 2024-04-25T11:28:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.845+0000 7f2505000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f25000d5590 con 0x7f25000b7c50 2024-04-25T11:28:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.845+0000 7f2505000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25000d4cf0 con 0x7f25000d3d40 2024-04-25T11:28:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.846+0000 7f24ffe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25000b7c50 0x7f25000b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.846+0000 7f24ff400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d3d40 0x7f2500160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.846+0000 7f24ffe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f25000b7c50 0x7f25000b98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:54052/0 (socket says 172.21.15.19:54052) 2024-04-25T11:28:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.846+0000 7f24ffe00640 1 -- 172.21.15.19:0/2671671117 learned_addr learned my addr 172.21.15.19:0/2671671117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.847+0000 7f24ff400640 1 -- 172.21.15.19:0/2671671117 >> v2:172.21.15.169:3300/0 conn(0x7f25000b7c50 msgr2=0x7f25000b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.847+0000 7f24ff400640 1 --2- 172.21.15.19:0/2671671117 >> v2:172.21.15.169:3300/0 conn(0x7f25000b7c50 0x7f25000b98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.847+0000 7f24ff400640 1 -- 172.21.15.19:0/2671671117 >> v1:172.21.15.169:6789/0 conn(0x7f25000d4510 legacy=0x7f25000b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.847+0000 7f24ff400640 1 -- 172.21.15.19:0/2671671117 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25000d4fd0 con 0x7f25000d3d40 2024-04-25T11:28:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.848+0000 7f24ff400640 1 --2- 172.21.15.19:0/2671671117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d3d40 0x7f2500160290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36988fb5d9d106c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.849+0000 7f24fe000640 1 -- 172.21.15.19:0/2671671117 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0031600 con 0x7f25000d3d40 2024-04-25T11:28:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.849+0000 7f24fe000640 1 -- 172.21.15.19:0/2671671117 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f00317a0 con 0x7f25000d3d40 2024-04-25T11:28:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.849+0000 7f24fe000640 1 -- 172.21.15.19:0/2671671117 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0031b70 con 0x7f25000d3d40 2024-04-25T11:28:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 -- 172.21.15.19:0/2671671117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d3d40 msgr2=0x7f2500160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 --2- 172.21.15.19:0/2671671117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d3d40 0x7f2500160290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f002a5d0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 -- 172.21.15.19:0/2671671117 shutdown_connections 2024-04-25T11:28:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 --2- 172.21.15.19:0/2671671117 >> v2:172.21.15.169:3300/0 conn(0x7f25000b7c50 0x7f25000b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 --2- 172.21.15.19:0/2671671117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d3d40 0x7f2500160290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 -- 172.21.15.19:0/2671671117 >> 172.21.15.19:0/2671671117 conn(0x7f25000d2540 msgr2=0x7f25000d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 -- 172.21.15.19:0/2671671117 shutdown_connections 2024-04-25T11:28:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.850+0000 7f2505000640 1 -- 172.21.15.19:0/2671671117 wait complete. 2024-04-25T11:28:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.853+0000 7f2505000640 1 Processor -- start 2024-04-25T11:28:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.854+0000 7f2505000640 1 -- start start 2024-04-25T11:28:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f2505000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25000d3d40 0x7f25000cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f2505000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d4510 0x7f25000cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f2505000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250015b840 0x7f25000ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f2505000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25000d4920 con 0x7f25000d3d40 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f2505000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25000b93b0 con 0x7f250015b840 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f2505000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25000d4aa0 con 0x7f25000d4510 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f24ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25000d3d40 0x7f25000cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f24ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25000d3d40 0x7f25000cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54636/0 (socket says 172.21.15.19:54636) 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f24ff400640 1 -- 172.21.15.19:0/3540639346 learned_addr learned my addr 172.21.15.19:0/3540639346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f24ffe00640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250015b840 0x7f25000ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.856+0000 7f24fea00640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d4510 0x7f25000cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24ff400640 1 -- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d4510 msgr2=0x7f25000cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24ff400640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d4510 0x7f25000cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24ff400640 1 -- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250015b840 msgr2=0x7f25000ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24ff400640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250015b840 0x7f25000ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24ff400640 1 -- 172.21.15.19:0/3540639346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25000cbe20 con 0x7f25000d3d40 2024-04-25T11:28:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24ffe00640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250015b840 0x7f25000ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24ff400640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25000d3d40 0x7f25000cab80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24f7400640 1 -- 172.21.15.19:0/3540639346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f003f070 con 0x7f25000d3d40 2024-04-25T11:28:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24f7400640 1 -- 172.21.15.19:0/3540639346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f0030be0 con 0x7f25000d3d40 2024-04-25T11:28:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f24f7400640 1 -- 172.21.15.19:0/3540639346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0009450 con 0x7f25000d3d40 2024-04-25T11:28:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25000cc3a0 con 0x7f25000d3d40 2024-04-25T11:28:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.857+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001608a0 con 0x7f25000d3d40 2024-04-25T11:28:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.859+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2500076c60 con 0x7f25000d3d40 2024-04-25T11:28:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.860+0000 7f24f7400640 1 -- 172.21.15.19:0/3540639346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f24f0044460 con 0x7f25000d3d40 2024-04-25T11:28:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.862+0000 7f24f7400640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24dc0744b0 0x7f24dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.862+0000 7f24f7400640 1 -- 172.21.15.19:0/3540639346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f24f00b6870 con 0x7f25000d3d40 2024-04-25T11:28:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.863+0000 7f24fea00640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24dc0744b0 0x7f24dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:19.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.864+0000 7f24fea00640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24dc0744b0 0x7f24dc0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24e8007fd0 tx=0x7f24e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:19.866+0000 7f24f7400640 1 -- 172.21.15.19:0/3540639346 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24f0080980 con 0x7f25000d3d40 2024-04-25T11:28:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.071+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f25000b8a30 con 0x7f25000d3d40 2024-04-25T11:28:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.072+0000 7f24f7400640 1 -- 172.21.15.19:0/3540639346 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f24f0084c80 con 0x7f25000d3d40 2024-04-25T11:28:20.064 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:20.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24dc0744b0 msgr2=0x7f24dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24dc0744b0 0x7f24dc0769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f24e8007fd0 tx=0x7f24e8009040 comp rx=0 tx=0).stop 2024-04-25T11:28:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25000d3d40 msgr2=0x7f25000cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25000d3d40 0x7f25000cab80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009d90 tx=0x7f24f0004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 shutdown_connections 2024-04-25T11:28:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24dc0744b0 0x7f24dc0769c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250015b840 0x7f25000ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25000d4510 0x7f25000cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 --2- 172.21.15.19:0/3540639346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25000d3d40 0x7f25000cab80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 >> 172.21.15.19:0/3540639346 conn(0x7f25000d2540 msgr2=0x7f250006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:20.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 shutdown_connections 2024-04-25T11:28:20.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.075+0000 7f2505000640 1 -- 172.21.15.19:0/3540639346 wait complete. 2024-04-25T11:28:20.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:28:20.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.383+0000 7f7c35c00640 1 Processor -- start 2024-04-25T11:28:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.383+0000 7f7c35c00640 1 -- start start 2024-04-25T11:28:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c35c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c300b4ea0 0x7f7c300cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c35c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c30144370 0x7f7c3015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c35c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c300d1230 con 0x7f7c3006c5c0 2024-04-25T11:28:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c35c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7c300d06b0 con 0x7f7c300b4ea0 2024-04-25T11:28:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c35c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c300d0990 con 0x7f7c30144370 2024-04-25T11:28:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c2fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7c3006c5c0 0x7f7c300b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44000/0 (socket says 172.21.15.19:44000) 2024-04-25T11:28:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c2fe00640 1 -- 172.21.15.19:0/934438091 learned_addr learned my addr 172.21.15.19:0/934438091 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c2f400640 1 --2- 172.21.15.19:0/934438091 >> v2:172.21.15.19:3300/0 conn(0x7f7c300b4ea0 0x7f7c300cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.386+0000 7f7c35200640 1 --2- 172.21.15.19:0/934438091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c30144370 0x7f7c3015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.387+0000 7f7c2ea00640 1 -- 172.21.15.19:0/934438091 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730075116 0 0) 0x7f7c300d1230 con 0x7f7c3006c5c0 2024-04-25T11:28:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.387+0000 7f7c2ea00640 1 -- 172.21.15.19:0/934438091 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c14003520 con 0x7f7c3006c5c0 2024-04-25T11:28:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.387+0000 7f7c2f400640 1 -- 172.21.15.19:0/934438091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c30144370 msgr2=0x7f7c3015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.387+0000 7f7c2f400640 1 --2- 172.21.15.19:0/934438091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c30144370 0x7f7c3015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.387+0000 7f7c2f400640 1 -- 172.21.15.19:0/934438091 >> v1:172.21.15.19:6789/0 conn(0x7f7c3006c5c0 legacy=0x7f7c300b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.387+0000 7f7c2f400640 1 -- 172.21.15.19:0/934438091 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c300d0c70 con 0x7f7c300b4ea0 2024-04-25T11:28:20.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.388+0000 7f7c2f400640 1 --2- 172.21.15.19:0/934438091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c300b4ea0 0x7f7c300cfa70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0097a0 tx=0x7f7c1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a98569a645ad790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:20.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.388+0000 7f7c2ea00640 1 -- 172.21.15.19:0/934438091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c037070 con 0x7f7c300b4ea0 2024-04-25T11:28:20.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.388+0000 7f7c2ea00640 1 -- 172.21.15.19:0/934438091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c032440 con 0x7f7c300b4ea0 2024-04-25T11:28:20.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.388+0000 7f7c2ea00640 1 -- 172.21.15.19:0/934438091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0327f0 con 0x7f7c300b4ea0 2024-04-25T11:28:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 -- 172.21.15.19:0/934438091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c300b4ea0 msgr2=0x7f7c300cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 --2- 172.21.15.19:0/934438091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c300b4ea0 0x7f7c300cfa70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0097a0 tx=0x7f7c1c02a180 comp rx=0 tx=0).stop 2024-04-25T11:28:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 -- 172.21.15.19:0/934438091 shutdown_connections 2024-04-25T11:28:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 --2- 172.21.15.19:0/934438091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c30144370 0x7f7c3015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 --2- 172.21.15.19:0/934438091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c300b4ea0 0x7f7c300cfa70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 -- 172.21.15.19:0/934438091 >> 172.21.15.19:0/934438091 conn(0x7f7c300d1720 msgr2=0x7f7c300d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 -- 172.21.15.19:0/934438091 shutdown_connections 2024-04-25T11:28:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.389+0000 7f7c35c00640 1 -- 172.21.15.19:0/934438091 wait complete. 2024-04-25T11:28:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.392+0000 7f7c35c00640 1 Processor -- start 2024-04-25T11:28:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.392+0000 7f7c35c00640 1 -- start start 2024-04-25T11:28:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.393+0000 7f7c35c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c3006c5c0 0x7f7c30141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.393+0000 7f7c35c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c30144370 0x7f7c300cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.393+0000 7f7c35c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c300cc710 0x7f7c301457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.393+0000 7f7c35c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c300d0450 con 0x7f7c30144370 2024-04-25T11:28:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.393+0000 7f7c35c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c300d02d0 con 0x7f7c3006c5c0 2024-04-25T11:28:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.393+0000 7f7c35c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c30147760 con 0x7f7c300cc710 2024-04-25T11:28:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c30144370 0x7f7c300cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c30144370 0x7f7c300cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54650/0 (socket says 172.21.15.19:54650) 2024-04-25T11:28:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 -- 172.21.15.19:0/1094590100 learned_addr learned my addr 172.21.15.19:0/1094590100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2fe00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c3006c5c0 0x7f7c30141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c35200640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c300cc710 0x7f7c301457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 -- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c300cc710 msgr2=0x7f7c301457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c300cc710 0x7f7c301457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 -- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c3006c5c0 msgr2=0x7f7c30141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c3006c5c0 0x7f7c30141300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 -- 172.21.15.19:0/1094590100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c300cadf0 con 0x7f7c30144370 2024-04-25T11:28:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2fe00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c3006c5c0 0x7f7c30141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c35200640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c300cc710 0x7f7c301457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.394+0000 7f7c2f400640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c30144370 0x7f7c300cc1d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c035040 tx=0x7f7c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.395+0000 7f7c27e00640 1 -- 172.21.15.19:0/1094590100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c037040 con 0x7f7c30144370 2024-04-25T11:28:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.395+0000 7f7c27e00640 1 -- 172.21.15.19:0/1094590100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c030500 con 0x7f7c30144370 2024-04-25T11:28:20.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.395+0000 7f7c27e00640 1 -- 172.21.15.19:0/1094590100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0308d0 con 0x7f7c30144370 2024-04-25T11:28:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.395+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c300cb0d0 con 0x7f7c30144370 2024-04-25T11:28:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.395+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c300cb690 con 0x7f7c30144370 2024-04-25T11:28:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.396+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c300cab60 con 0x7f7c30144370 2024-04-25T11:28:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.397+0000 7f7c27e00640 1 -- 172.21.15.19:0/1094590100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c1c044020 con 0x7f7c30144370 2024-04-25T11:28:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.398+0000 7f7c27e00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c100744b0 0x7f7c100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.398+0000 7f7c27e00640 1 -- 172.21.15.19:0/1094590100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f7c1c0b5460 con 0x7f7c30144370 2024-04-25T11:28:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.399+0000 7f7c2fe00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c100744b0 0x7f7c100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.400+0000 7f7c2fe00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c100744b0 0x7f7c100769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c20005880 tx=0x7f7c20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:20.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.403+0000 7f7c27e00640 1 -- 172.21.15.19:0/1094590100 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c1c07fcf0 con 0x7f7c30144370 2024-04-25T11:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.615+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c30050740 con 0x7f7c30144370 2024-04-25T11:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.616+0000 7f7c27e00640 1 -- 172.21.15.19:0/1094590100 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f7c1c07f5f0 con 0x7f7c30144370 2024-04-25T11:28:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.618+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c100744b0 msgr2=0x7f7c100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.618+0000 7f7c35c00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c100744b0 0x7f7c100769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c20005880 tx=0x7f7c20009040 comp rx=0 tx=0).stop 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.618+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c30144370 msgr2=0x7f7c300cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.618+0000 7f7c35c00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c30144370 0x7f7c300cc1d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c035040 tx=0x7f7c1c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 shutdown_connections 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c100744b0 0x7f7c100769c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c300cc710 0x7f7c301457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c30144370 0x7f7c300cc1d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 --2- 172.21.15.19:0/1094590100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c3006c5c0 0x7f7c30141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 >> 172.21.15.19:0/1094590100 conn(0x7f7c300d1720 msgr2=0x7f7c300d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 shutdown_connections 2024-04-25T11:28:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:20.619+0000 7f7c35c00640 1 -- 172.21.15.19:0/1094590100 wait complete. 2024-04-25T11:28:20.636 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:28:20.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:20.637 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T11:28:20.735 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:28:20.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:20.735 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:28:20.791 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:20.791 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:20.850 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:20.850 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:20.900 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:20.900 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:20.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:21.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.243+0000 7f867f000640 1 Processor -- start 2024-04-25T11:28:21.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.243+0000 7f867f000640 1 -- start start 2024-04-25T11:28:21.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.245+0000 7f867f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8678162930 0x7f86780d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.245+0000 7f867f000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86780d2a90 con 0x7f86780d3e20 2024-04-25T11:28:21.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.245+0000 7f867f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86780d27b0 con 0x7f86780b3680 2024-04-25T11:28:21.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.245+0000 7f867f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8678161ac0 con 0x7f8678162930 2024-04-25T11:28:21.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.246+0000 7f867d000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f86780d3e20 0x7f86780ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44004/0 (socket says 172.21.15.19:44004) 2024-04-25T11:28:21.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.246+0000 7f867d000640 1 -- 172.21.15.19:0/1549897220 learned_addr learned my addr 172.21.15.19:0/1549897220 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:21.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.246+0000 7f8677e00640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8678162930 0x7f86780d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785670932 0 0) 0x7f86780d2a90 con 0x7f86780d3e20 2024-04-25T11:28:21.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f865c003520 con 0x7f86780d3e20 2024-04-25T11:28:21.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 655650309 0 0) 0x7f86780d27b0 con 0x7f86780b3680 2024-04-25T11:28:21.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86780d2a90 con 0x7f86780b3680 2024-04-25T11:28:21.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2652533670 0 0) 0x7f865c003520 con 0x7f86780d3e20 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8678162930 msgr2=0x7f86780d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8678162930 0x7f86780d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 >> v1:172.21.15.169:6789/0 conn(0x7f86780b3680 legacy=0x7f86780d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8678161e50 con 0x7f86780d3e20 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f86700019f0 con 0x7f86780d3e20 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 >> v1:172.21.15.19:6789/0 conn(0x7f86780d3e20 legacy=0x7f86780ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f865c003b70 0x7f865c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c004800 0x7f865c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.247+0000 7f8676a00640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c0051a0 0x7f865c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8678161ac0 con 0x7f865c004800 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f865c00c0c0 con 0x7f865c0051a0 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f865c00c2c0 con 0x7f865c003b70 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8677400640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c004800 0x7f865c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f867d000640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c0051a0 0x7f865c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8677e00640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f865c003b70 0x7f865c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8677400640 1 -- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f865c003b70 msgr2=0x7f865c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8677400640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f865c003b70 0x7f865c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8677400640 1 -- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c0051a0 msgr2=0x7f865c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8677400640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c0051a0 0x7f865c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.248+0000 7f8677400640 1 -- 172.21.15.19:0/1549897220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c003520 con 0x7f865c004800 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.249+0000 7f8677400640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c004800 0x7f865c004c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86640018d0 tx=0x7f865c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99fc5bbaa61cc10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86640014f0 con 0x7f865c004800 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86640052d0 con 0x7f865c004800 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f8676a00640 1 -- 172.21.15.19:0/1549897220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866402aca0 con 0x7f865c004800 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f867f000640 1 -- 172.21.15.19:0/1549897220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c004800 msgr2=0x7f865c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f867f000640 1 --2- 172.21.15.19:0/1549897220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f865c004800 0x7f865c004c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86640018d0 tx=0x7f865c005830 comp rx=0 tx=0).stop 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f867d000640 1 -- 172.21.15.19:0/1549897220 reap_dead start 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f867f000640 1 -- 172.21.15.19:0/1549897220 shutdown_connections 2024-04-25T11:28:21.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.250+0000 7f867f000640 1 -- 172.21.15.19:0/1549897220 >> 172.21.15.19:0/1549897220 conn(0x7f867808d110 msgr2=0x7f867808d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:21.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.251+0000 7f867f000640 1 -- 172.21.15.19:0/1549897220 shutdown_connections 2024-04-25T11:28:21.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.251+0000 7f867f000640 1 -- 172.21.15.19:0/1549897220 wait complete. 2024-04-25T11:28:21.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.253+0000 7f867f000640 1 Processor -- start 2024-04-25T11:28:21.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.253+0000 7f867f000640 1 -- start start 2024-04-25T11:28:21.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f867f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c004800 0x7f86780b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f867f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8678157e90 0x7f86780cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f867f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86780d04a0 0x7f86780d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f867f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86780b3a90 con 0x7f8678157e90 2024-04-25T11:28:21.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f867f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8678162410 con 0x7f865c004800 2024-04-25T11:28:21.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f867f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f86780b93f0 con 0x7f86780d04a0 2024-04-25T11:28:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f8677400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8678157e90 0x7f86780cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f8677e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c004800 0x7f86780b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.255+0000 7f8677400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8678157e90 0x7f86780cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54658/0 (socket says 172.21.15.19:54658) 2024-04-25T11:28:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f8677400640 1 -- 172.21.15.19:0/3669645370 learned_addr learned my addr 172.21.15.19:0/3669645370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f867d000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86780d04a0 0x7f86780d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f8677400640 1 -- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86780d04a0 msgr2=0x7f86780d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f8677400640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86780d04a0 0x7f86780d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f8677400640 1 -- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c004800 msgr2=0x7f86780b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f8677400640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c004800 0x7f86780b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f8677400640 1 -- 172.21.15.19:0/3669645370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86780d0df0 con 0x7f8678157e90 2024-04-25T11:28:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.256+0000 7f867d000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86780d04a0 0x7f86780d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.257+0000 7f8677400640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8678157e90 0x7f86780cd850 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f866402a730 tx=0x7f86640012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.257+0000 7f866fe00640 1 -- 172.21.15.19:0/3669645370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866403f020 con 0x7f8678157e90 2024-04-25T11:28:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.257+0000 7f866fe00640 1 -- 172.21.15.19:0/3669645370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8664006e50 con 0x7f8678157e90 2024-04-25T11:28:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.257+0000 7f866fe00640 1 -- 172.21.15.19:0/3669645370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664037040 con 0x7f8678157e90 2024-04-25T11:28:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.257+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8678156360 con 0x7f8678157e90 2024-04-25T11:28:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.257+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86781570d0 con 0x7f8678157e90 2024-04-25T11:28:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.258+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8678156b60 con 0x7f8678157e90 2024-04-25T11:28:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.260+0000 7f866fe00640 1 -- 172.21.15.19:0/3669645370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8664042020 con 0x7f8678157e90 2024-04-25T11:28:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.261+0000 7f866fe00640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86580744e0 0x7f86580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.261+0000 7f866fe00640 1 -- 172.21.15.19:0/3669645370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f86640b52b0 con 0x7f8678157e90 2024-04-25T11:28:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.262+0000 7f8677e00640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86580744e0 0x7f86580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.263+0000 7f8677e00640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86580744e0 0x7f86580769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86680090e0 tx=0x7f8668008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.266+0000 7f866fe00640 1 -- 172.21.15.19:0/3669645370 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8664049020 con 0x7f8678157e90 2024-04-25T11:28:21.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.476+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8678050740 con 0x7f8678157e90 2024-04-25T11:28:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.479+0000 7f866fe00640 1 -- 172.21.15.19:0/3669645370 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f86780d0df0 con 0x7f8678157e90 2024-04-25T11:28:21.471 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:21.471 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86580744e0 msgr2=0x7f86580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86580744e0 0x7f86580769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86680090e0 tx=0x7f8668008040 comp rx=0 tx=0).stop 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8678157e90 msgr2=0x7f86780cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8678157e90 0x7f86780cd850 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f866402a730 tx=0x7f86640012d0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 shutdown_connections 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86580744e0 0x7f86580769f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86780d04a0 0x7f86780d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8678157e90 0x7f86780cd850 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 --2- 172.21.15.19:0/3669645370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f865c004800 0x7f86780b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 >> 172.21.15.19:0/3669645370 conn(0x7f867808d110 msgr2=0x7f86780714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 shutdown_connections 2024-04-25T11:28:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.482+0000 7f867f000640 1 -- 172.21.15.19:0/3669645370 wait complete. 2024-04-25T11:28:21.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:21.502 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.800+0000 7f5933a00640 1 Processor -- start 2024-04-25T11:28:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.800+0000 7f5933a00640 1 -- start start 2024-04-25T11:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.803+0000 7f5933a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f592c15adf0 0x7f592c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.803+0000 7f5933a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c071110 con 0x7f592c070630 2024-04-25T11:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.803+0000 7f5933a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c06fb50 con 0x7f592c06dce0 2024-04-25T11:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.803+0000 7f5933a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f592c0713f0 con 0x7f592c15adf0 2024-04-25T11:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.803+0000 7f5931000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f592c070630 0x7f592c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44008/0 (socket says 172.21.15.19:44008) 2024-04-25T11:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.803+0000 7f5931000640 1 -- 172.21.15.19:0/1519537218 learned_addr learned my addr 172.21.15.19:0/1519537218 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.803+0000 7f5931a00640 1 --2- 172.21.15.19:0/1519537218 >> v2:172.21.15.19:3300/0 conn(0x7f592c15adf0 0x7f592c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.804+0000 7f5931a00640 1 -- 172.21.15.19:0/1519537218 >> v1:172.21.15.169:6789/0 conn(0x7f592c06dce0 legacy=0x7f592c160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.804+0000 7f5931a00640 1 -- 172.21.15.19:0/1519537218 >> v1:172.21.15.19:6789/0 conn(0x7f592c070630 legacy=0x7f592c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.804+0000 7f5931a00640 1 -- 172.21.15.19:0/1519537218 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c06fe30 con 0x7f592c15adf0 2024-04-25T11:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.804+0000 7f592be00640 1 -- 172.21.15.19:0/1519537218 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3855115981 0 0) 0x7f592c071110 con 0x7f592c070630 2024-04-25T11:28:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.805+0000 7f5931a00640 1 --2- 172.21.15.19:0/1519537218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c15adf0 0x7f592c0cec50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5924009d90 tx=0x7f592402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b0af532ccbda9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.805+0000 7f592be00640 1 -- 172.21.15.19:0/1519537218 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924031600 con 0x7f592c15adf0 2024-04-25T11:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.805+0000 7f592be00640 1 -- 172.21.15.19:0/1519537218 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59240317a0 con 0x7f592c15adf0 2024-04-25T11:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.805+0000 7f592be00640 1 -- 172.21.15.19:0/1519537218 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924031b70 con 0x7f592c15adf0 2024-04-25T11:28:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.806+0000 7f5933a00640 1 -- 172.21.15.19:0/1519537218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c15adf0 msgr2=0x7f592c0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.806+0000 7f5933a00640 1 --2- 172.21.15.19:0/1519537218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c15adf0 0x7f592c0cec50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5924009d90 tx=0x7f592402a610 comp rx=0 tx=0).stop 2024-04-25T11:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.806+0000 7f5933a00640 1 -- 172.21.15.19:0/1519537218 shutdown_connections 2024-04-25T11:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.806+0000 7f5933a00640 1 --2- 172.21.15.19:0/1519537218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c15adf0 0x7f592c0cec50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.806+0000 7f5933a00640 1 -- 172.21.15.19:0/1519537218 >> 172.21.15.19:0/1519537218 conn(0x7f592c073090 msgr2=0x7f592c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.806+0000 7f5933a00640 1 -- 172.21.15.19:0/1519537218 shutdown_connections 2024-04-25T11:28:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.806+0000 7f5933a00640 1 -- 172.21.15.19:0/1519537218 wait complete. 2024-04-25T11:28:21.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.809+0000 7f5933a00640 1 Processor -- start 2024-04-25T11:28:21.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.809+0000 7f5933a00640 1 -- start start 2024-04-25T11:28:21.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.810+0000 7f5933a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c070630 0x7f592c1e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.810+0000 7f5933a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592c15adf0 0x7f592c14dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5933a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f592c158090 0x7f592c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5933a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f592c070c80 con 0x7f592c070630 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5933a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f592c0701a0 con 0x7f592c158090 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5933a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f592c070e00 con 0x7f592c15adf0 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5931000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c070630 0x7f592c1e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5931000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c070630 0x7f592c1e9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54678/0 (socket says 172.21.15.19:54678) 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5931000640 1 -- 172.21.15.19:0/3517967387 learned_addr learned my addr 172.21.15.19:0/3517967387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5931a00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f592c158090 0x7f592c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5923e00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592c15adf0 0x7f592c14dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5931000640 1 -- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592c15adf0 msgr2=0x7f592c14dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.811+0000 7f5931000640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592c15adf0 0x7f592c14dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5931000640 1 -- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f592c158090 msgr2=0x7f592c1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5931000640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f592c158090 0x7f592c1584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5931000640 1 -- 172.21.15.19:0/3517967387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c1ea450 con 0x7f592c070630 2024-04-25T11:28:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5923e00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592c15adf0 0x7f592c14dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5931000640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c070630 0x7f592c1e9f10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f591c00c410 tx=0x7f591c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5929600640 1 -- 172.21.15.19:0/3517967387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0074e0 con 0x7f592c070630 2024-04-25T11:28:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5929600640 1 -- 172.21.15.19:0/3517967387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f591c0047e0 con 0x7f592c070630 2024-04-25T11:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c1e8eb0 con 0x7f592c070630 2024-04-25T11:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5929600640 1 -- 172.21.15.19:0/3517967387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c004bd0 con 0x7f592c070630 2024-04-25T11:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.812+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c1e9430 con 0x7f592c070630 2024-04-25T11:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.814+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592c050740 con 0x7f592c070630 2024-04-25T11:28:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.815+0000 7f5929600640 1 -- 172.21.15.19:0/3517967387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f591c011070 con 0x7f592c070630 2024-04-25T11:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.816+0000 7f5929600640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59080744e0 0x7f59080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.816+0000 7f5929600640 1 -- 172.21.15.19:0/3517967387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f591c094ea0 con 0x7f592c070630 2024-04-25T11:28:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.817+0000 7f5923e00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59080744e0 0x7f59080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.818+0000 7f5923e00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59080744e0 0x7f59080769f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f592c159e20 tx=0x7f5914008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:21.821+0000 7f5929600640 1 -- 172.21.15.19:0/3517967387 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f591c05f570 con 0x7f592c070630 2024-04-25T11:28:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.030+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f592c06d3c0 con 0x7f592c070630 2024-04-25T11:28:22.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.031+0000 7f5929600640 1 -- 172.21.15.19:0/3517967387 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f591c05ee70 con 0x7f592c070630 2024-04-25T11:28:22.023 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59080744e0 msgr2=0x7f59080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59080744e0 0x7f59080769f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f592c159e20 tx=0x7f5914008040 comp rx=0 tx=0).stop 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c070630 msgr2=0x7f592c1e9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c070630 0x7f592c1e9f10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f591c00c410 tx=0x7f591c00c8d0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 shutdown_connections 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59080744e0 0x7f59080769f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f592c158090 0x7f592c1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f592c15adf0 0x7f592c14dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 --2- 172.21.15.19:0/3517967387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f592c070630 0x7f592c1e9f10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 >> 172.21.15.19:0/3517967387 conn(0x7f592c073090 msgr2=0x7f592c0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 shutdown_connections 2024-04-25T11:28:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.034+0000 7f5933a00640 1 -- 172.21.15.19:0/3517967387 wait complete. 2024-04-25T11:28:22.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.355+0000 7f6258c00640 1 Processor -- start 2024-04-25T11:28:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.356+0000 7f6258c00640 1 -- start start 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6258c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625406dce0 0x7f625415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6258c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f625415ec90 0x7f625415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6258c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6254070000 con 0x7f6254144340 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6258c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6254144750 con 0x7f625415ec90 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6258c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f625406f240 con 0x7f625406dce0 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6253e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f625415ec90 0x7f625415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6253e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f625415ec90 0x7f625415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54694/0 (socket says 172.21.15.19:54694) 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.358+0000 7f6253e00640 1 -- 172.21.15.19:0/2853554900 learned_addr learned my addr 172.21.15.19:0/2853554900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.359+0000 7f624be00640 1 --2- 172.21.15.19:0/2853554900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625406dce0 0x7f625415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.359+0000 7f6253e00640 1 -- 172.21.15.19:0/2853554900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625406dce0 msgr2=0x7f625415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.359+0000 7f6253e00640 1 --2- 172.21.15.19:0/2853554900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625406dce0 0x7f625415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.359+0000 7f6253e00640 1 -- 172.21.15.19:0/2853554900 >> v1:172.21.15.169:6789/0 conn(0x7f6254144340 legacy=0x7f625406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.359+0000 7f6253e00640 1 -- 172.21.15.19:0/2853554900 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625406f520 con 0x7f625415ec90 2024-04-25T11:28:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.360+0000 7f6253e00640 1 --2- 172.21.15.19:0/2853554900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f625415ec90 0x7f625415f4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f624c009e50 tx=0x7f624c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742de4a21144bc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.360+0000 7f6252a00640 1 -- 172.21.15.19:0/2853554900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c036070 con 0x7f625415ec90 2024-04-25T11:28:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6252a00640 1 -- 172.21.15.19:0/2853554900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624c031600 con 0x7f625415ec90 2024-04-25T11:28:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6252a00640 1 -- 172.21.15.19:0/2853554900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c0319d0 con 0x7f625415ec90 2024-04-25T11:28:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6258c00640 1 -- 172.21.15.19:0/2853554900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f625415ec90 msgr2=0x7f625415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6258c00640 1 --2- 172.21.15.19:0/2853554900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f625415ec90 0x7f625415f4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f624c009e50 tx=0x7f624c02a6c0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6258c00640 1 -- 172.21.15.19:0/2853554900 shutdown_connections 2024-04-25T11:28:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6258c00640 1 --2- 172.21.15.19:0/2853554900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f625415ec90 0x7f625415f4a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6258c00640 1 --2- 172.21.15.19:0/2853554900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625406dce0 0x7f625415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6258c00640 1 -- 172.21.15.19:0/2853554900 >> 172.21.15.19:0/2853554900 conn(0x7f6254072f40 msgr2=0x7f6254073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.361+0000 7f6258c00640 1 -- 172.21.15.19:0/2853554900 shutdown_connections 2024-04-25T11:28:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.362+0000 7f6258c00640 1 -- 172.21.15.19:0/2853554900 wait complete. 2024-04-25T11:28:22.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.364+0000 7f6258c00640 1 Processor -- start 2024-04-25T11:28:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.364+0000 7f6258c00640 1 -- start start 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6258c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6254144340 0x7f62541580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6258c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f625415ec90 0x7f6254159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6258c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625415a010 0x7f625413e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6258c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f625415f0a0 con 0x7f6254144340 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6258c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f625406fae0 con 0x7f625415ec90 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6258c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f625415f220 con 0x7f625415a010 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6254144340 0x7f62541580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6254144340 0x7f62541580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54702/0 (socket says 172.21.15.19:54702) 2024-04-25T11:28:22.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 -- 172.21.15.19:0/2848591030 learned_addr learned my addr 172.21.15.19:0/2848591030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:22.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f624be00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f625415ec90 0x7f6254159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 -- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625415a010 msgr2=0x7f625413e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625415a010 0x7f625413e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 -- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f625415ec90 msgr2=0x7f6254159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f625415ec90 0x7f6254159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.366+0000 7f6253400640 1 -- 172.21.15.19:0/2848591030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62541f9dd0 con 0x7f6254144340 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.367+0000 7f6253400640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6254144340 0x7f62541580e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f624400b420 tx=0x7f624400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.367+0000 7f624b400640 1 -- 172.21.15.19:0/2848591030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244011830 con 0x7f6254144340 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.367+0000 7f624b400640 1 -- 172.21.15.19:0/2848591030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244016070 con 0x7f6254144340 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.367+0000 7f624b400640 1 -- 172.21.15.19:0/2848591030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244011c10 con 0x7f6254144340 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.367+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62541fa0b0 con 0x7f6254144340 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.367+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541fa670 con 0x7f6254144340 2024-04-25T11:28:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.369+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6254076c60 con 0x7f6254144340 2024-04-25T11:28:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.370+0000 7f624b400640 1 -- 172.21.15.19:0/2848591030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f624401b460 con 0x7f6254144340 2024-04-25T11:28:22.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.371+0000 7f624b400640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.371+0000 7f624b400640 1 -- 172.21.15.19:0/2848591030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f6244094d50 con 0x7f6254144340 2024-04-25T11:28:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.372+0000 7f624be00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.373+0000 7f624be00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f623c006fd0 tx=0x7f623c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.376+0000 7f624b400640 1 -- 172.21.15.19:0/2848591030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6244098030 con 0x7f6254144340 2024-04-25T11:28:22.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.578+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62540b7920 con 0x7f6254144340 2024-04-25T11:28:22.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.579+0000 7f624b400640 1 -- 172.21.15.19:0/2848591030 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f6244020020 con 0x7f6254144340 2024-04-25T11:28:22.571 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 msgr2=0x7f62300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f623c006fd0 tx=0x7f623c008040 comp rx=0 tx=0).stop 2024-04-25T11:28:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6254144340 msgr2=0x7f62541580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6254144340 0x7f62541580e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f624400b420 tx=0x7f624400b8e0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 shutdown_connections 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f625415ec90 0x7f6254159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625415a010 0x7f625413e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 --2- 172.21.15.19:0/2848591030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6254144340 0x7f62541580e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.581+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 >> 172.21.15.19:0/2848591030 conn(0x7f6254072f40 msgr2=0x7f62540b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.582+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 shutdown_connections 2024-04-25T11:28:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.582+0000 7f6258c00640 1 -- 172.21.15.19:0/2848591030 wait complete. 2024-04-25T11:28:22.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.899+0000 7fd9bb200640 1 Processor -- start 2024-04-25T11:28:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.899+0000 7fd9bb200640 1 -- start start 2024-04-25T11:28:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.901+0000 7fd9bb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9bc06dce0 0x7fd9bc0af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.901+0000 7fd9bb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9bc15fff0 0x7fd9bc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.901+0000 7fd9bb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9bc0af9b0 con 0x7fd9bc0b0a50 2024-04-25T11:28:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.901+0000 7fd9bb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd9bc0b54c0 con 0x7fd9bc06dce0 2024-04-25T11:28:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.901+0000 7fd9bb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd9bc0afc90 con 0x7fd9bc15fff0 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9b1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9bc06dce0 0x7fd9bc0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9b1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9bc06dce0 0x7fd9bc0af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54706/0 (socket says 172.21.15.19:54706) 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9b1400640 1 -- 172.21.15.19:0/3569111658 learned_addr learned my addr 172.21.15.19:0/3569111658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9ba800640 1 --2- 172.21.15.19:0/3569111658 >> v2:172.21.15.169:3300/0 conn(0x7fd9bc15fff0 0x7fd9bc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9b1400640 1 -- 172.21.15.19:0/3569111658 >> v2:172.21.15.169:3300/0 conn(0x7fd9bc15fff0 msgr2=0x7fd9bc0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9b1400640 1 --2- 172.21.15.19:0/3569111658 >> v2:172.21.15.169:3300/0 conn(0x7fd9bc15fff0 0x7fd9bc0b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9b1400640 1 -- 172.21.15.19:0/3569111658 >> v1:172.21.15.169:6789/0 conn(0x7fd9bc0b0a50 legacy=0x7fd9bc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.902+0000 7fd9b1400640 1 -- 172.21.15.19:0/3569111658 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc0aff70 con 0x7fd9bc06dce0 2024-04-25T11:28:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.904+0000 7fd9b1400640 1 --2- 172.21.15.19:0/3569111658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc06dce0 0x7fd9bc0af340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0b1570 tx=0x7fd9a402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d66f4860ee496ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.904+0000 7fd9b9400640 1 -- 172.21.15.19:0/3569111658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4037070 con 0x7fd9bc06dce0 2024-04-25T11:28:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.904+0000 7fd9b9400640 1 -- 172.21.15.19:0/3569111658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a402ae20 con 0x7fd9bc06dce0 2024-04-25T11:28:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.904+0000 7fd9b9400640 1 -- 172.21.15.19:0/3569111658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4032660 con 0x7fd9bc06dce0 2024-04-25T11:28:22.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.905+0000 7fd9bb200640 1 -- 172.21.15.19:0/3569111658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc06dce0 msgr2=0x7fd9bc0af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.905+0000 7fd9bb200640 1 --2- 172.21.15.19:0/3569111658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc06dce0 0x7fd9bc0af340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0b1570 tx=0x7fd9a402a110 comp rx=0 tx=0).stop 2024-04-25T11:28:22.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.905+0000 7fd9bb200640 1 -- 172.21.15.19:0/3569111658 shutdown_connections 2024-04-25T11:28:22.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.905+0000 7fd9bb200640 1 --2- 172.21.15.19:0/3569111658 >> v2:172.21.15.169:3300/0 conn(0x7fd9bc15fff0 0x7fd9bc0b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.905+0000 7fd9bb200640 1 --2- 172.21.15.19:0/3569111658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc06dce0 0x7fd9bc0af340 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.905+0000 7fd9bb200640 1 -- 172.21.15.19:0/3569111658 >> 172.21.15.19:0/3569111658 conn(0x7fd9bc073160 msgr2=0x7fd9bc073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.906+0000 7fd9bb200640 1 -- 172.21.15.19:0/3569111658 shutdown_connections 2024-04-25T11:28:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.906+0000 7fd9bb200640 1 -- 172.21.15.19:0/3569111658 wait complete. 2024-04-25T11:28:22.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.909+0000 7fd9bb200640 1 Processor -- start 2024-04-25T11:28:22.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.909+0000 7fd9bb200640 1 -- start start 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9bb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9bc0b0a50 0x7fd9bc0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9bb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc15fff0 0x7fd9bc1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9bb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9bc0b2940 0x7fd9bc154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9bb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc160400 con 0x7fd9bc15fff0 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9bb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0b0530 con 0x7fd9bc0b2940 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9bb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc06f090 con 0x7fd9bc0b0a50 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc15fff0 0x7fd9bc1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc15fff0 0x7fd9bc1630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54710/0 (socket says 172.21.15.19:54710) 2024-04-25T11:28:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.911+0000 7fd9b1400640 1 -- 172.21.15.19:0/436914323 learned_addr learned my addr 172.21.15.19:0/436914323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9ba800640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9bc0b2940 0x7fd9bc154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b9e00640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9bc0b0a50 0x7fd9bc0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b1400640 1 -- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9bc0b0a50 msgr2=0x7fd9bc0b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b1400640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9bc0b0a50 0x7fd9bc0b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b1400640 1 -- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9bc0b2940 msgr2=0x7fd9bc154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b1400640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9bc0b2940 0x7fd9bc154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b1400640 1 -- 172.21.15.19:0/436914323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc1f9d20 con 0x7fd9bc15fff0 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9ba800640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9bc0b2940 0x7fd9bc154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b9e00640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9bc0b0a50 0x7fd9bc0b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.912+0000 7fd9b1400640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc15fff0 0x7fd9bc1630c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402a150 tx=0x7fd9a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.913+0000 7fd9b2a00640 1 -- 172.21.15.19:0/436914323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4041070 con 0x7fd9bc15fff0 2024-04-25T11:28:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.913+0000 7fd9b2a00640 1 -- 172.21.15.19:0/436914323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9a403c670 con 0x7fd9bc15fff0 2024-04-25T11:28:22.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.913+0000 7fd9b2a00640 1 -- 172.21.15.19:0/436914323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4037040 con 0x7fd9bc15fff0 2024-04-25T11:28:22.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.913+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc1fa000 con 0x7fd9bc15fff0 2024-04-25T11:28:22.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.913+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc1fa5c0 con 0x7fd9bc15fff0 2024-04-25T11:28:22.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.914+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9bc076c60 con 0x7fd9bc15fff0 2024-04-25T11:28:22.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.915+0000 7fd9b2a00640 1 -- 172.21.15.19:0/436914323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9a4044070 con 0x7fd9bc15fff0 2024-04-25T11:28:22.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.916+0000 7fd9b2a00640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9980744e0 0x7fd9980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:22.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.916+0000 7fd9b2a00640 1 -- 172.21.15.19:0/436914323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fd9a40b5780 con 0x7fd9bc15fff0 2024-04-25T11:28:22.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.917+0000 7fd9b9e00640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9980744e0 0x7fd9980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:22.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.918+0000 7fd9b9e00640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9980744e0 0x7fd9980769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0aedd0 tx=0x7fd9ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:22.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:22.921+0000 7fd9b2a00640 1 -- 172.21.15.19:0/436914323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9a402ae20 con 0x7fd9bc15fff0 2024-04-25T11:28:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.129+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9bc073ec0 con 0x7fd9bc15fff0 2024-04-25T11:28:23.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.130+0000 7fd9b2a00640 1 -- 172.21.15.19:0/436914323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fd9a407fe50 con 0x7fd9bc15fff0 2024-04-25T11:28:23.122 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9980744e0 msgr2=0x7fd9980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9980744e0 0x7fd9980769f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0aedd0 tx=0x7fd9ac008040 comp rx=0 tx=0).stop 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc15fff0 msgr2=0x7fd9bc1630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc15fff0 0x7fd9bc1630c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402a150 tx=0x7fd9a40046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 shutdown_connections 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9980744e0 0x7fd9980769f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9bc0b2940 0x7fd9bc154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9bc15fff0 0x7fd9bc1630c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 --2- 172.21.15.19:0/436914323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9bc0b0a50 0x7fd9bc0b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 >> 172.21.15.19:0/436914323 conn(0x7fd9bc073160 msgr2=0x7fd9bc157830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 shutdown_connections 2024-04-25T11:28:23.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.133+0000 7fd9bb200640 1 -- 172.21.15.19:0/436914323 wait complete. 2024-04-25T11:28:23.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:28:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.449+0000 7fadca400640 1 Processor -- start 2024-04-25T11:28:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.449+0000 7fadca400640 1 -- start start 2024-04-25T11:28:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.452+0000 7fadca400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadc40709a0 0x7fadc41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.452+0000 7fadca400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40cffc0 con 0x7fadc40cf9e0 2024-04-25T11:28:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.452+0000 7fadca400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40d02a0 con 0x7fadc40b0800 2024-04-25T11:28:23.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.452+0000 7fadca400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fadc40d0580 con 0x7fadc40709a0 2024-04-25T11:28:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.452+0000 7fadc9000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fadc40cf9e0 0x7fadc4161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44012/0 (socket says 172.21.15.19:44012) 2024-04-25T11:28:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.453+0000 7fadc9000640 1 -- 172.21.15.19:0/2811889720 learned_addr learned my addr 172.21.15.19:0/2811889720 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.453+0000 7fadc9a00640 1 --2- 172.21.15.19:0/2811889720 >> v2:172.21.15.169:3300/0 conn(0x7fadc40709a0 0x7fadc41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.453+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2583090923 0 0) 0x7fadc40cffc0 con 0x7fadc40cf9e0 2024-04-25T11:28:23.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.453+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fada8003520 con 0x7fadc40cf9e0 2024-04-25T11:28:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2852932044 0 0) 0x7fada8003520 con 0x7fadc40cf9e0 2024-04-25T11:28:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 >> v2:172.21.15.169:3300/0 conn(0x7fadc40709a0 msgr2=0x7fadc41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 --2- 172.21.15.19:0/2811889720 >> v2:172.21.15.169:3300/0 conn(0x7fadc40709a0 0x7fadc41569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 >> v1:172.21.15.169:6789/0 conn(0x7fadc40b0800 legacy=0x7fadc4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc40d0860 con 0x7fadc40cf9e0 2024-04-25T11:28:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fadac001670 con 0x7fadc40cf9e0 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 >> v1:172.21.15.19:6789/0 conn(0x7fadc40cf9e0 legacy=0x7fadc4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada8003a00 0x7fada8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada8004800 0x7fada8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada80052a0 0x7fada800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadc40d0580 con 0x7fada8003a00 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fada800c230 con 0x7fada8004800 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fada800c430 con 0x7fada80052a0 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.454+0000 7fadc9000640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada8003a00 0x7fada8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc3e00640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada8004800 0x7fada8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc9a00640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada80052a0 0x7fada800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc9000640 1 -- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada80052a0 msgr2=0x7fada800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc9000640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada80052a0 0x7fada800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc9000640 1 -- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada8004800 msgr2=0x7fada8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc9000640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada8004800 0x7fada8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc9000640 1 -- 172.21.15.19:0/2811889720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada8003520 con 0x7fada8003a00 2024-04-25T11:28:23.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc3e00640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada8004800 0x7fada8004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:23.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.455+0000 7fadc9a00640 1 -- 172.21.15.19:0/2811889720 reap_dead start 2024-04-25T11:28:23.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.456+0000 7fadc9000640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada8003a00 0x7fada8003e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c0e0 tx=0x7fada8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07fcb07e9d88a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.456+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac007c40 con 0x7fada8003a00 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.456+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac005af0 con 0x7fada8003a00 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.456+0000 7fadc3400640 1 -- 172.21.15.19:0/2811889720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac007070 con 0x7fada8003a00 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.457+0000 7fadca400640 1 -- 172.21.15.19:0/2811889720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada8003a00 msgr2=0x7fada8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.457+0000 7fadca400640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada8003a00 0x7fada8003e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c0e0 tx=0x7fada8005830 comp rx=0 tx=0).stop 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.457+0000 7fadca400640 1 -- 172.21.15.19:0/2811889720 shutdown_connections 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.457+0000 7fadca400640 1 --2- 172.21.15.19:0/2811889720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada8003a00 0x7fada8003e70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.457+0000 7fadca400640 1 -- 172.21.15.19:0/2811889720 >> 172.21.15.19:0/2811889720 conn(0x7fadc40d1720 msgr2=0x7fadc40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.457+0000 7fadca400640 1 -- 172.21.15.19:0/2811889720 shutdown_connections 2024-04-25T11:28:23.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.457+0000 7fadca400640 1 -- 172.21.15.19:0/2811889720 wait complete. 2024-04-25T11:28:23.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.459+0000 7fadca400640 1 Processor -- start 2024-04-25T11:28:23.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.460+0000 7fadca400640 1 -- start start 2024-04-25T11:28:23.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.461+0000 7fadca400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada8003a00 0x7fadc415f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.461+0000 7fadca400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadc4154680 0x7fadc4154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.461+0000 7fadca400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc4162460 0x7fadc4162870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.461+0000 7fadca400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4161090 con 0x7fadc4154680 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.461+0000 7fadca400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fadc40b0e20 con 0x7fadc4162460 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.461+0000 7fadca400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadc4161210 con 0x7fada8003a00 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.461+0000 7fadc3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadc4154680 0x7fadc4154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadc4154680 0x7fadc4154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54730/0 (socket says 172.21.15.19:54730) 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 -- 172.21.15.19:0/2516074355 learned_addr learned my addr 172.21.15.19:0/2516074355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc9000640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada8003a00 0x7fadc415f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc9a00640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc4162460 0x7fadc4162870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 -- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada8003a00 msgr2=0x7fadc415f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada8003a00 0x7fadc415f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 -- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc4162460 msgr2=0x7fadc4162870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc4162460 0x7fadc4162870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 -- 172.21.15.19:0/2516074355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc4154fd0 con 0x7fadc4154680 2024-04-25T11:28:23.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc9000640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada8003a00 0x7fadc415f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:23.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.462+0000 7fadc3e00640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadc4154680 0x7fadc4154a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fadb400dae0 tx=0x7fadb40020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.463+0000 7fadc0c00640 1 -- 172.21.15.19:0/2516074355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4003da0 con 0x7fadc4154680 2024-04-25T11:28:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.463+0000 7fadc0c00640 1 -- 172.21.15.19:0/2516074355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb4017070 con 0x7fadc4154680 2024-04-25T11:28:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.463+0000 7fadc0c00640 1 -- 172.21.15.19:0/2516074355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb400b950 con 0x7fadc4154680 2024-04-25T11:28:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.463+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc4162db0 con 0x7fadc4154680 2024-04-25T11:28:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.463+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41f6140 con 0x7fadc4154680 2024-04-25T11:28:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.464+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc4050740 con 0x7fadc4154680 2024-04-25T11:28:23.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.465+0000 7fadc0c00640 1 -- 172.21.15.19:0/2516074355 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fadb400baf0 con 0x7fadc4154680 2024-04-25T11:28:23.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.466+0000 7fadc0c00640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fada00744b0 0x7fada00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:23.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.466+0000 7fadc0c00640 1 -- 172.21.15.19:0/2516074355 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fadb4095450 con 0x7fadc4154680 2024-04-25T11:28:23.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.467+0000 7fadc9000640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fada00744b0 0x7fada00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:23.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.468+0000 7fadc9000640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fada00744b0 0x7fada00769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fadac032000 tx=0x7fadac02d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:23.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.471+0000 7fadc0c00640 1 -- 172.21.15.19:0/2516074355 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fadc4162db0 con 0x7fadc4154680 2024-04-25T11:28:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.689+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fadc40cdc30 con 0x7fadc4154680 2024-04-25T11:28:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.689+0000 7fadc0c00640 1 -- 172.21.15.19:0/2516074355 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fadb405fbb0 con 0x7fadc4154680 2024-04-25T11:28:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:28:23.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fada00744b0 msgr2=0x7fada00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fada00744b0 0x7fada00769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fadac032000 tx=0x7fadac02d000 comp rx=0 tx=0).stop 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadc4154680 msgr2=0x7fadc4154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadc4154680 0x7fadc4154a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fadb400dae0 tx=0x7fadb40020a0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 shutdown_connections 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fada00744b0 0x7fada00769c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadc4162460 0x7fadc4162870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadc4154680 0x7fadc4154a90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 --2- 172.21.15.19:0/2516074355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada8003a00 0x7fadc415f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.692+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 >> 172.21.15.19:0/2516074355 conn(0x7fadc40d1720 msgr2=0x7fadc4161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.693+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 shutdown_connections 2024-04-25T11:28:23.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:23.693+0000 7fadca400640 1 -- 172.21.15.19:0/2516074355 wait complete. 2024-04-25T11:28:23.710 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T11:28:23.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:23.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T11:28:23.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:28:23.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:23.806 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T11:28:23.859 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:28:23.859 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:23.919 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:28:23.919 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:23.970 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:28:23.970 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:24.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:24.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.311+0000 7f4602000640 1 Processor -- start 2024-04-25T11:28:24.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.311+0000 7f4602000640 1 -- start start 2024-04-25T11:28:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.313+0000 7f4602000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 0x7f45fc0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.313+0000 7f4602000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f45fc15a810 0x7f45fc0b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.313+0000 7f4602000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45fc0b4e90 con 0x7f45fc1608d0 2024-04-25T11:28:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.314+0000 7f4602000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f45fc06f870 con 0x7f45fc15a810 2024-04-25T11:28:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.314+0000 7f4602000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc0b74d0 con 0x7f45fc0cf2f0 2024-04-25T11:28:24.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.314+0000 7f4601600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f45fc1608d0 0x7f45fc160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53186/0 (socket says 172.21.15.19:53186) 2024-04-25T11:28:24.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.314+0000 7f4601600640 1 -- 172.21.15.19:0/766304527 learned_addr learned my addr 172.21.15.19:0/766304527 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:24.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.314+0000 7f4600c00640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 0x7f45fc0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.314+0000 7f45fbe00640 1 --2- 172.21.15.19:0/766304527 >> v2:172.21.15.169:3300/0 conn(0x7f45fc15a810 0x7f45fc0b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1878897163 0 0) 0x7f45fc0b4e90 con 0x7f45fc1608d0 2024-04-25T11:28:24.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45e4003520 con 0x7f45fc1608d0 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1341189699 0 0) 0x7f45e4003520 con 0x7f45fc1608d0 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 msgr2=0x7f45fc0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 0x7f45fc0b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 >> v2:172.21.15.169:3300/0 conn(0x7f45fc15a810 msgr2=0x7f45fc0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 --2- 172.21.15.19:0/766304527 >> v2:172.21.15.169:3300/0 conn(0x7f45fc15a810 0x7f45fc0b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc0b77b0 con 0x7f45fc1608d0 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f45f40019f0 con 0x7f45fc1608d0 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 >> v1:172.21.15.19:6789/0 conn(0x7f45fc1608d0 legacy=0x7f45fc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039b0 0x7f45e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4004800 0x7f45e4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 0x7f45e4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc06f870 con 0x7f45e4004800 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc0b74d0 con 0x7f45e40039b0 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.315+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45e400b9f0 con 0x7f45fc0cf2f0 2024-04-25T11:28:24.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f45fbe00640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4004800 0x7f45e4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f45fbe00640 1 -- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 msgr2=0x7f45e4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f4600c00640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039b0 0x7f45e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f4601600640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 0x7f45e4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f45fbe00640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 0x7f45e4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f45fbe00640 1 -- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039b0 msgr2=0x7f45e4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f45fbe00640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039b0 0x7f45e4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f45fbe00640 1 -- 172.21.15.19:0/766304527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e4003520 con 0x7f45e4004800 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f4601600640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc0cf2f0 0x7f45e4005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:24.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.316+0000 7f4600c00640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45e40039b0 0x7f45e4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:24.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.317+0000 7f45fbe00640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4004800 0x7f45e4004c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009ac0 tx=0x7f45e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137c84ddd9fc57d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:24.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.318+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8036070 con 0x7f45e4004800 2024-04-25T11:28:24.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.318+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e8031400 con 0x7f45e4004800 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.318+0000 7f45fb400640 1 -- 172.21.15.19:0/766304527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8009690 con 0x7f45e4004800 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.319+0000 7f4602000640 1 -- 172.21.15.19:0/766304527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4004800 msgr2=0x7f45e4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.319+0000 7f4602000640 1 --2- 172.21.15.19:0/766304527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45e4004800 0x7f45e4004c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009ac0 tx=0x7f45e4005830 comp rx=0 tx=0).stop 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.319+0000 7f4601600640 1 -- 172.21.15.19:0/766304527 reap_dead start 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.319+0000 7f4602000640 1 -- 172.21.15.19:0/766304527 shutdown_connections 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.319+0000 7f4602000640 1 -- 172.21.15.19:0/766304527 >> 172.21.15.19:0/766304527 conn(0x7f45fc0d1720 msgr2=0x7f45fc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.319+0000 7f4602000640 1 -- 172.21.15.19:0/766304527 shutdown_connections 2024-04-25T11:28:24.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.319+0000 7f4602000640 1 -- 172.21.15.19:0/766304527 wait complete. 2024-04-25T11:28:24.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.321+0000 7f4602000640 1 Processor -- start 2024-04-25T11:28:24.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.322+0000 7f4602000640 1 -- start start 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.323+0000 7f4602000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc1fa720 0x7f45fc1f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.323+0000 7f4602000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45fc15d250 0x7f45fc15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.323+0000 7f4602000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc147a30 0x7f45fc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.323+0000 7f4602000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc15ac20 con 0x7f45fc15d250 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.323+0000 7f4602000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc06f500 con 0x7f45fc147a30 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.323+0000 7f4602000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc0b6e10 con 0x7f45fc1fa720 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45fc15d250 0x7f45fc15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45fc15d250 0x7f45fc15d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47608/0 (socket says 172.21.15.19:47608) 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 -- 172.21.15.19:0/1576489839 learned_addr learned my addr 172.21.15.19:0/1576489839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f4600c00640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc1fa720 0x7f45fc1f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f4601600640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc147a30 0x7f45fc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 -- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc1fa720 msgr2=0x7f45fc1f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc1fa720 0x7f45fc1f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 -- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc147a30 msgr2=0x7f45fc147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc147a30 0x7f45fc147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f45fbe00640 1 -- 172.21.15.19:0/1576489839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc15dba0 con 0x7f45fc15d250 2024-04-25T11:28:24.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.324+0000 7f4601600640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc147a30 0x7f45fc147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:24.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.325+0000 7f45fbe00640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45fc15d250 0x7f45fc15d660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009ac0 tx=0x7f45e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:24.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.325+0000 7f45f8c00640 1 -- 172.21.15.19:0/1576489839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8036050 con 0x7f45fc15d250 2024-04-25T11:28:24.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.325+0000 7f45f8c00640 1 -- 172.21.15.19:0/1576489839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e8042070 con 0x7f45fc15d250 2024-04-25T11:28:24.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.325+0000 7f45f8c00640 1 -- 172.21.15.19:0/1576489839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8009690 con 0x7f45fc15d250 2024-04-25T11:28:24.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.325+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc15f9d0 con 0x7f45fc15d250 2024-04-25T11:28:24.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.325+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc15ff90 con 0x7f45fc15d250 2024-04-25T11:28:24.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.327+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc050740 con 0x7f45fc15d250 2024-04-25T11:28:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.328+0000 7f45f8c00640 1 -- 172.21.15.19:0/1576489839 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f45e8047460 con 0x7f45fc15d250 2024-04-25T11:28:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.329+0000 7f45f8c00640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45dc0744b0 0x7f45dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.329+0000 7f45f8c00640 1 -- 172.21.15.19:0/1576489839 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f45e80b5ba0 con 0x7f45fc15d250 2024-04-25T11:28:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.329+0000 7f4600c00640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45dc0744b0 0x7f45dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.331+0000 7f4600c00640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45dc0744b0 0x7f45dc0769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45f00090e0 tx=0x7f45f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.334+0000 7f45f8c00640 1 -- 172.21.15.19:0/1576489839 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45fc15dba0 con 0x7f45fc15d250 2024-04-25T11:28:24.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.539+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45fc0d0da0 con 0x7f45fc15d250 2024-04-25T11:28:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.541+0000 7f45f8c00640 1 -- 172.21.15.19:0/1576489839 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f45e8080390 con 0x7f45fc15d250 2024-04-25T11:28:24.534 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:24.534 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45dc0744b0 msgr2=0x7f45dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45dc0744b0 0x7f45dc0769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45f00090e0 tx=0x7f45f0008040 comp rx=0 tx=0).stop 2024-04-25T11:28:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45fc15d250 msgr2=0x7f45fc15d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45fc15d250 0x7f45fc15d660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009ac0 tx=0x7f45e8004730 comp rx=0 tx=0).stop 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 shutdown_connections 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45dc0744b0 0x7f45dc0769c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45fc147a30 0x7f45fc147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45fc15d250 0x7f45fc15d660 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 --2- 172.21.15.19:0/1576489839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45fc1fa720 0x7f45fc1f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.544+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 >> 172.21.15.19:0/1576489839 conn(0x7f45fc0d1720 msgr2=0x7f45fc073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.545+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 shutdown_connections 2024-04-25T11:28:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.545+0000 7f4602000640 1 -- 172.21.15.19:0/1576489839 wait complete. 2024-04-25T11:28:24.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:24.562 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:24.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.864+0000 7f35a9200640 1 Processor -- start 2024-04-25T11:28:24.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.864+0000 7f35a9200640 1 -- start start 2024-04-25T11:28:24.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a9200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35a4144340 0x7f35a406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35a415ee60 0x7f35a415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a9200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a406df20 con 0x7f35a406db10 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a9200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f35a415f270 con 0x7f35a415ee60 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a9200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f35a4144750 con 0x7f35a4144340 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35a415ee60 0x7f35a415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35a415ee60 0x7f35a415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47614/0 (socket says 172.21.15.19:47614) 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.866+0000 7f35a3e00640 1 -- 172.21.15.19:0/2601929577 learned_addr learned my addr 172.21.15.19:0/2601929577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.867+0000 7f35a3400640 1 --2- 172.21.15.19:0/2601929577 >> v2:172.21.15.169:3300/0 conn(0x7f35a4144340 0x7f35a406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.867+0000 7f35a3e00640 1 -- 172.21.15.19:0/2601929577 >> v2:172.21.15.169:3300/0 conn(0x7f35a4144340 msgr2=0x7f35a406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.867+0000 7f35a3e00640 1 --2- 172.21.15.19:0/2601929577 >> v2:172.21.15.169:3300/0 conn(0x7f35a4144340 0x7f35a406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.867+0000 7f35a3e00640 1 -- 172.21.15.19:0/2601929577 >> v1:172.21.15.19:6789/0 conn(0x7f35a406db10 legacy=0x7f35a415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.867+0000 7f35a3e00640 1 -- 172.21.15.19:0/2601929577 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a4070000 con 0x7f35a415ee60 2024-04-25T11:28:24.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.867+0000 7f35a3400640 1 --2- 172.21.15.19:0/2601929577 >> v2:172.21.15.169:3300/0 conn(0x7f35a4144340 0x7f35a406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:24.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.868+0000 7f35a3e00640 1 --2- 172.21.15.19:0/2601929577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a415ee60 0x7f35a415f4a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35a40b6e30 tx=0x7f359c02a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10838a1b15b20f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:24.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.868+0000 7f35a2a00640 1 -- 172.21.15.19:0/2601929577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c036070 con 0x7f35a415ee60 2024-04-25T11:28:24.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.868+0000 7f35a2a00640 1 -- 172.21.15.19:0/2601929577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c031600 con 0x7f35a415ee60 2024-04-25T11:28:24.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.868+0000 7f35a2a00640 1 -- 172.21.15.19:0/2601929577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0319d0 con 0x7f35a415ee60 2024-04-25T11:28:24.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 -- 172.21.15.19:0/2601929577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a415ee60 msgr2=0x7f35a415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 --2- 172.21.15.19:0/2601929577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a415ee60 0x7f35a415f4a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f35a40b6e30 tx=0x7f359c02a740 comp rx=0 tx=0).stop 2024-04-25T11:28:24.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 -- 172.21.15.19:0/2601929577 shutdown_connections 2024-04-25T11:28:24.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 --2- 172.21.15.19:0/2601929577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a415ee60 0x7f35a415f4a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 --2- 172.21.15.19:0/2601929577 >> v2:172.21.15.169:3300/0 conn(0x7f35a4144340 0x7f35a406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 -- 172.21.15.19:0/2601929577 >> 172.21.15.19:0/2601929577 conn(0x7f35a4072f40 msgr2=0x7f35a4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:24.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 -- 172.21.15.19:0/2601929577 shutdown_connections 2024-04-25T11:28:24.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.869+0000 7f35a9200640 1 -- 172.21.15.19:0/2601929577 wait complete. 2024-04-25T11:28:24.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.872+0000 7f35a9200640 1 Processor -- start 2024-04-25T11:28:24.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.872+0000 7f35a9200640 1 -- start start 2024-04-25T11:28:24.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a406db10 0x7f35a415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a4144340 0x7f35a415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a415ee60 0x7f35a41f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a9200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35a406f800 con 0x7f35a406db10 2024-04-25T11:28:24.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a9200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35a406f520 con 0x7f35a4144340 2024-04-25T11:28:24.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35a406fae0 con 0x7f35a415ee60 2024-04-25T11:28:24.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a406db10 0x7f35a415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a406db10 0x7f35a415c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47628/0 (socket says 172.21.15.19:47628) 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 -- 172.21.15.19:0/1673742329 learned_addr learned my addr 172.21.15.19:0/1673742329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 -- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a415ee60 msgr2=0x7f35a41f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a415ee60 0x7f35a41f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f359be00640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a4144340 0x7f35a415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 -- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a4144340 msgr2=0x7f35a415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a4144340 0x7f35a415cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.874+0000 7f35a3400640 1 -- 172.21.15.19:0/1673742329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a41f4690 con 0x7f35a406db10 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.875+0000 7f35a3400640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a406db10 0x7f35a415c5c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f359400c820 tx=0x7f359400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.875+0000 7f359b400640 1 -- 172.21.15.19:0/1673742329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594015070 con 0x7f35a406db10 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.875+0000 7f359b400640 1 -- 172.21.15.19:0/1673742329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3594007a30 con 0x7f35a406db10 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.875+0000 7f359b400640 1 -- 172.21.15.19:0/1673742329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594007de0 con 0x7f35a406db10 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.875+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a41f4930 con 0x7f35a406db10 2024-04-25T11:28:24.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.875+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a41582f0 con 0x7f35a406db10 2024-04-25T11:28:24.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.876+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a41585b0 con 0x7f35a406db10 2024-04-25T11:28:24.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.878+0000 7f359b400640 1 -- 172.21.15.19:0/1673742329 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3594018070 con 0x7f35a406db10 2024-04-25T11:28:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.879+0000 7f359b400640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3580074530 0x7f3580076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:24.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.879+0000 7f359b400640 1 -- 172.21.15.19:0/1673742329 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f3594095120 con 0x7f35a406db10 2024-04-25T11:28:24.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.879+0000 7f359be00640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3580074530 0x7f3580076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:24.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.880+0000 7f359be00640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3580074530 0x7f3580076a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35a40b2040 tx=0x7f358c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:24.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:24.884+0000 7f359b400640 1 -- 172.21.15.19:0/1673742329 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f359405f870 con 0x7f35a406db10 2024-04-25T11:28:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.087+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35a4076c60 con 0x7f35a406db10 2024-04-25T11:28:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.088+0000 7f359b400640 1 -- 172.21.15.19:0/1673742329 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f359405f170 con 0x7f35a406db10 2024-04-25T11:28:25.080 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3580074530 msgr2=0x7f3580076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3580074530 0x7f3580076a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35a40b2040 tx=0x7f358c009040 comp rx=0 tx=0).stop 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a406db10 msgr2=0x7f35a415c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a406db10 0x7f35a415c5c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f359400c820 tx=0x7f359400cce0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 shutdown_connections 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3580074530 0x7f3580076a40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a4144340 0x7f35a415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a415ee60 0x7f35a41f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 --2- 172.21.15.19:0/1673742329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a406db10 0x7f35a415c5c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 >> 172.21.15.19:0/1673742329 conn(0x7f35a4072f40 msgr2=0x7f35a40b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 shutdown_connections 2024-04-25T11:28:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.091+0000 7f35a9200640 1 -- 172.21.15.19:0/1673742329 wait complete. 2024-04-25T11:28:25.109 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.404+0000 7f52d2800640 1 Processor -- start 2024-04-25T11:28:25.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.404+0000 7f52d2800640 1 -- start start 2024-04-25T11:28:25.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52d2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52cc0cd9a0 0x7f52cc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52d2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52cc0b0af0 0x7f52cc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52d2800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52cc06df20 con 0x7f52cc06db10 2024-04-25T11:28:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52d2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc0ceab0 con 0x7f52cc0b0af0 2024-04-25T11:28:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52d2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc0cdea0 con 0x7f52cc0cd9a0 2024-04-25T11:28:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52cbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f52cc06db10 0x7f52cc161ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53202/0 (socket says 172.21.15.19:53202) 2024-04-25T11:28:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52cbe00640 1 -- 172.21.15.19:0/1801503377 learned_addr learned my addr 172.21.15.19:0/1801503377 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:25.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.407+0000 7f52caa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52cc0b0af0 0x7f52cc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.408+0000 7f52cb400640 1 --2- 172.21.15.19:0/1801503377 >> v2:172.21.15.169:3300/0 conn(0x7f52cc0cd9a0 0x7f52cc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.408+0000 7f52ca000640 1 -- 172.21.15.19:0/1801503377 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035445258 0 0) 0x7f52c4001110 con 0x7f52cc06db10 2024-04-25T11:28:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.408+0000 7f52ca000640 1 -- 172.21.15.19:0/1801503377 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52b0003520 con 0x7f52cc06db10 2024-04-25T11:28:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.408+0000 7f52caa00640 1 -- 172.21.15.19:0/1801503377 >> v2:172.21.15.169:3300/0 conn(0x7f52cc0cd9a0 msgr2=0x7f52cc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.408+0000 7f52caa00640 1 --2- 172.21.15.19:0/1801503377 >> v2:172.21.15.169:3300/0 conn(0x7f52cc0cd9a0 0x7f52cc15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.408+0000 7f52caa00640 1 -- 172.21.15.19:0/1801503377 >> v1:172.21.15.19:6789/0 conn(0x7f52cc06db10 legacy=0x7f52cc161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.408+0000 7f52caa00640 1 -- 172.21.15.19:0/1801503377 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0ce180 con 0x7f52cc0b0af0 2024-04-25T11:28:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.409+0000 7f52caa00640 1 --2- 172.21.15.19:0/1801503377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc0b0af0 0x7f52cc06d5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52cc072860 tx=0x7f52bc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984bb1dbc1e7655b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:25.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.410+0000 7f52ca000640 1 -- 172.21.15.19:0/1801503377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc007740 con 0x7f52cc0b0af0 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.410+0000 7f52ca000640 1 -- 172.21.15.19:0/1801503377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52bc0078e0 con 0x7f52cc0b0af0 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.410+0000 7f52ca000640 1 -- 172.21.15.19:0/1801503377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc007cb0 con 0x7f52cc0b0af0 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.410+0000 7f52d2800640 1 -- 172.21.15.19:0/1801503377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc0b0af0 msgr2=0x7f52cc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.410+0000 7f52d2800640 1 --2- 172.21.15.19:0/1801503377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc0b0af0 0x7f52cc06d5d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52cc072860 tx=0x7f52bc02a680 comp rx=0 tx=0).stop 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.411+0000 7f52d2800640 1 -- 172.21.15.19:0/1801503377 shutdown_connections 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.411+0000 7f52d2800640 1 --2- 172.21.15.19:0/1801503377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc0b0af0 0x7f52cc06d5d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.411+0000 7f52d2800640 1 --2- 172.21.15.19:0/1801503377 >> v2:172.21.15.169:3300/0 conn(0x7f52cc0cd9a0 0x7f52cc15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.411+0000 7f52d2800640 1 -- 172.21.15.19:0/1801503377 >> 172.21.15.19:0/1801503377 conn(0x7f52cc073090 msgr2=0x7f52cc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.411+0000 7f52d2800640 1 -- 172.21.15.19:0/1801503377 shutdown_connections 2024-04-25T11:28:25.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.411+0000 7f52d2800640 1 -- 172.21.15.19:0/1801503377 wait complete. 2024-04-25T11:28:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.413+0000 7f52d2800640 1 Processor -- start 2024-04-25T11:28:25.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.413+0000 7f52d2800640 1 -- start start 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52d2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc06db10 0x7f52cc1fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52d2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52cc0b0af0 0x7f52cc157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52d2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52cc0cd9a0 0x7f52cc0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52d2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc0b0f00 con 0x7f52cc06db10 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52d2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc0ce740 con 0x7f52cc0b0af0 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52d2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc0cef40 con 0x7f52cc0cd9a0 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc06db10 0x7f52cc1fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc06db10 0x7f52cc1fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47648/0 (socket says 172.21.15.19:47648) 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.415+0000 7f52cb400640 1 -- 172.21.15.19:0/600104813 learned_addr learned my addr 172.21.15.19:0/600104813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52caa00640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52cc0b0af0 0x7f52cc157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52cb400640 1 -- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52cc0cd9a0 msgr2=0x7f52cc0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52cbe00640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52cc0cd9a0 0x7f52cc0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52cb400640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52cc0cd9a0 0x7f52cc0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52cb400640 1 -- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52cc0b0af0 msgr2=0x7f52cc157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52cb400640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52cc0b0af0 0x7f52cc157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52cb400640 1 -- 172.21.15.19:0/600104813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc137d40 con 0x7f52cc06db10 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52cb400640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc06db10 0x7f52cc1fa680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d620 tx=0x7f52b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52c3400640 1 -- 172.21.15.19:0/600104813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b400b620 con 0x7f52cc06db10 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52c3400640 1 -- 172.21.15.19:0/600104813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b4019070 con 0x7f52cc06db10 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.416+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc1fabc0 con 0x7f52cc06db10 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.417+0000 7f52c3400640 1 -- 172.21.15.19:0/600104813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b400ba00 con 0x7f52cc06db10 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.417+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc1e7e90 con 0x7f52cc06db10 2024-04-25T11:28:25.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.418+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc050740 con 0x7f52cc06db10 2024-04-25T11:28:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.419+0000 7f52c3400640 1 -- 172.21.15.19:0/600104813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f52b400bba0 con 0x7f52cc06db10 2024-04-25T11:28:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.420+0000 7f52c3400640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52a80744b0 0x7f52a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.420+0000 7f52c3400640 1 -- 172.21.15.19:0/600104813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f52b4095730 con 0x7f52cc06db10 2024-04-25T11:28:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.421+0000 7f52caa00640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52a80744b0 0x7f52a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.422+0000 7f52caa00640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52a80744b0 0x7f52a80769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52bc034000 tx=0x7f52bc033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.426+0000 7f52c3400640 1 -- 172.21.15.19:0/600104813 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52cc1fabc0 con 0x7f52cc06db10 2024-04-25T11:28:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.622+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52cc1e8720 con 0x7f52cc06db10 2024-04-25T11:28:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.623+0000 7f52c3400640 1 -- 172.21.15.19:0/600104813 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f52cc137d40 con 0x7f52cc06db10 2024-04-25T11:28:25.615 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:25.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52a80744b0 msgr2=0x7f52a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52a80744b0 0x7f52a80769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52bc034000 tx=0x7f52bc033040 comp rx=0 tx=0).stop 2024-04-25T11:28:25.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc06db10 msgr2=0x7f52cc1fa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc06db10 0x7f52cc1fa680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d620 tx=0x7f52b400dae0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 shutdown_connections 2024-04-25T11:28:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52a80744b0 0x7f52a80769c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52cc0b0af0 0x7f52cc157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52cc0cd9a0 0x7f52cc0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 --2- 172.21.15.19:0/600104813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52cc06db10 0x7f52cc1fa680 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 >> 172.21.15.19:0/600104813 conn(0x7f52cc073090 msgr2=0x7f52cc161e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 shutdown_connections 2024-04-25T11:28:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.626+0000 7f52d2800640 1 -- 172.21.15.19:0/600104813 wait complete. 2024-04-25T11:28:25.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:25.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.911+0000 7eff74a00640 1 Processor -- start 2024-04-25T11:28:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.912+0000 7eff74a00640 1 -- start start 2024-04-25T11:28:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.914+0000 7eff74a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7eff70070d20 0x7eff70161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.914+0000 7eff74a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006dce0 0x7eff700ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.914+0000 7eff74a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff701623f0 con 0x7eff7006f930 2024-04-25T11:28:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.914+0000 7eff74a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7eff701626d0 con 0x7eff70070d20 2024-04-25T11:28:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.914+0000 7eff74a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff7006f130 con 0x7eff7006dce0 2024-04-25T11:28:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006dce0 0x7eff700ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff66a00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7eff7006f930 0x7eff7006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:44772/0 (socket says 172.21.15.19:44772) 2024-04-25T11:28:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff66a00640 1 -- 172.21.15.19:0/2806276133 learned_addr learned my addr 172.21.15.19:0/2806276133 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff6f400640 1 --2- 172.21.15.19:0/2806276133 >> v2:172.21.15.169:3300/0 conn(0x7eff70070d20 0x7eff70161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff6f400640 1 -- 172.21.15.19:0/2806276133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006dce0 msgr2=0x7eff700ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff6f400640 1 --2- 172.21.15.19:0/2806276133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006dce0 0x7eff700ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff6f400640 1 -- 172.21.15.19:0/2806276133 >> v1:172.21.15.169:6789/0 conn(0x7eff7006f930 legacy=0x7eff7006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.915+0000 7eff6f400640 1 -- 172.21.15.19:0/2806276133 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7006f410 con 0x7eff70070d20 2024-04-25T11:28:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.917+0000 7eff6f400640 1 --2- 172.21.15.19:0/2806276133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff70161b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff60009d90 tx=0x7eff6002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e0a85e691ab790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff6ea00640 1 -- 172.21.15.19:0/2806276133 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60031600 con 0x7eff70070d20 2024-04-25T11:28:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff6ea00640 1 -- 172.21.15.19:0/2806276133 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff600317a0 con 0x7eff70070d20 2024-04-25T11:28:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff6ea00640 1 -- 172.21.15.19:0/2806276133 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60031b70 con 0x7eff70070d20 2024-04-25T11:28:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff74a00640 1 -- 172.21.15.19:0/2806276133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 msgr2=0x7eff70161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff74a00640 1 --2- 172.21.15.19:0/2806276133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff70161b30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff60009d90 tx=0x7eff6002a5d0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff74a00640 1 -- 172.21.15.19:0/2806276133 shutdown_connections 2024-04-25T11:28:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff74a00640 1 --2- 172.21.15.19:0/2806276133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006dce0 0x7eff700ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff74a00640 1 --2- 172.21.15.19:0/2806276133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff70161b30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.918+0000 7eff74a00640 1 -- 172.21.15.19:0/2806276133 >> 172.21.15.19:0/2806276133 conn(0x7eff70073160 msgr2=0x7eff70073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.919+0000 7eff74a00640 1 -- 172.21.15.19:0/2806276133 shutdown_connections 2024-04-25T11:28:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.919+0000 7eff74a00640 1 -- 172.21.15.19:0/2806276133 wait complete. 2024-04-25T11:28:25.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.921+0000 7eff74a00640 1 Processor -- start 2024-04-25T11:28:25.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.921+0000 7eff74a00640 1 -- start start 2024-04-25T11:28:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff74a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006f930 0x7eff7015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff74a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff700cff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff74a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff700d04b0 0x7eff70156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff74a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff7015e2f0 con 0x7eff700d04b0 2024-04-25T11:28:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff74a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff70162070 con 0x7eff70070d20 2024-04-25T11:28:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff74a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff7015e470 con 0x7eff7006f930 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff700d04b0 0x7eff70156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff6fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff700d04b0 0x7eff70156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47660/0 (socket says 172.21.15.19:47660) 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff6fe00640 1 -- 172.21.15.19:0/284512112 learned_addr learned my addr 172.21.15.19:0/284512112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff66a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff700cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff6f400640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006f930 0x7eff7015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.923+0000 7eff6fe00640 1 -- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006f930 msgr2=0x7eff7015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff6fe00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006f930 0x7eff7015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff6fe00640 1 -- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 msgr2=0x7eff700cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff6fe00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff700cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff6fe00640 1 -- 172.21.15.19:0/284512112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7015f1a0 con 0x7eff700d04b0 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff66a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff700cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff6fe00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff700d04b0 0x7eff70156be0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e7f0 tx=0x7eff6800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff67e00640 1 -- 172.21.15.19:0/284512112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68017070 con 0x7eff700d04b0 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff67e00640 1 -- 172.21.15.19:0/284512112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6800c730 con 0x7eff700d04b0 2024-04-25T11:28:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7015f480 con 0x7eff700d04b0 2024-04-25T11:28:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff67e00640 1 -- 172.21.15.19:0/284512112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6800cb20 con 0x7eff700d04b0 2024-04-25T11:28:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.924+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701566c0 con 0x7eff700d04b0 2024-04-25T11:28:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.925+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7015ef10 con 0x7eff700d04b0 2024-04-25T11:28:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.927+0000 7eff67e00640 1 -- 172.21.15.19:0/284512112 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7eff6801a070 con 0x7eff700d04b0 2024-04-25T11:28:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.928+0000 7eff67e00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff4c0744b0 0x7eff4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:25.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.928+0000 7eff67e00640 1 -- 172.21.15.19:0/284512112 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7eff680954e0 con 0x7eff700d04b0 2024-04-25T11:28:25.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.929+0000 7eff6f400640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff4c0744b0 0x7eff4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:25.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.930+0000 7eff6f400640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff4c0744b0 0x7eff4c0769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff60034000 tx=0x7eff60033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:25.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:25.933+0000 7eff67e00640 1 -- 172.21.15.19:0/284512112 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff6805fbb0 con 0x7eff700d04b0 2024-04-25T11:28:26.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.127+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7eff70076c60 con 0x7eff700d04b0 2024-04-25T11:28:26.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.128+0000 7eff67e00640 1 -- 172.21.15.19:0/284512112 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7eff6805f4b0 con 0x7eff700d04b0 2024-04-25T11:28:26.120 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.130+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff4c0744b0 msgr2=0x7eff4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.130+0000 7eff74a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff4c0744b0 0x7eff4c0769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff60034000 tx=0x7eff60033040 comp rx=0 tx=0).stop 2024-04-25T11:28:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.130+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff700d04b0 msgr2=0x7eff70156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.130+0000 7eff74a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff700d04b0 0x7eff70156be0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e7f0 tx=0x7eff6800ecb0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 shutdown_connections 2024-04-25T11:28:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff4c0744b0 0x7eff4c0769c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff70070d20 0x7eff700cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff700d04b0 0x7eff70156be0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 --2- 172.21.15.19:0/284512112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff7006f930 0x7eff7015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 >> 172.21.15.19:0/284512112 conn(0x7eff70073160 msgr2=0x7eff7015d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 shutdown_connections 2024-04-25T11:28:26.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.131+0000 7eff74a00640 1 -- 172.21.15.19:0/284512112 wait complete. 2024-04-25T11:28:26.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:28:26.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.439+0000 7f4c08000640 1 Processor -- start 2024-04-25T11:28:26.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.439+0000 7f4c08000640 1 -- start start 2024-04-25T11:28:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.442+0000 7f4c08000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4c000cf5e0 0x7f4c000cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.442+0000 7f4c08000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c000cff30 con 0x7f4c001443c0 2024-04-25T11:28:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.442+0000 7f4c08000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0015f5f0 con 0x7f4c0006c5c0 2024-04-25T11:28:26.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.442+0000 7f4c08000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4c000708d0 con 0x7f4c000cf5e0 2024-04-25T11:28:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.442+0000 7f4c04c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4c000cf5e0 0x7f4c000cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.443+0000 7f4c05600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4c001443c0 0x7f4c0006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53212/0 (socket says 172.21.15.19:53212) 2024-04-25T11:28:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.443+0000 7f4c05600640 1 -- 172.21.15.19:0/733832348 learned_addr learned my addr 172.21.15.19:0/733832348 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.443+0000 7f4c04c00640 1 -- 172.21.15.19:0/733832348 >> v1:172.21.15.169:6789/0 conn(0x7f4c0006c5c0 legacy=0x7f4c0006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.443+0000 7f4c04c00640 1 -- 172.21.15.19:0/733832348 >> v1:172.21.15.19:6789/0 conn(0x7f4c001443c0 legacy=0x7f4c0006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.443+0000 7f4c04c00640 1 -- 172.21.15.19:0/733832348 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0015e830 con 0x7f4c000cf5e0 2024-04-25T11:28:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.444+0000 7f4c04c00640 1 --2- 172.21.15.19:0/733832348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c000cf9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097a0 tx=0x7f4bec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed03ef34868ec7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.444+0000 7f4bffe00640 1 -- 172.21.15.19:0/733832348 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec037070 con 0x7f4c000cf5e0 2024-04-25T11:28:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.444+0000 7f4bffe00640 1 -- 172.21.15.19:0/733832348 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bec032440 con 0x7f4c000cf5e0 2024-04-25T11:28:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4bffe00640 1 -- 172.21.15.19:0/733832348 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec032810 con 0x7f4c000cf5e0 2024-04-25T11:28:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4c08000640 1 -- 172.21.15.19:0/733832348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 msgr2=0x7f4c000cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4c08000640 1 --2- 172.21.15.19:0/733832348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c000cf9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097a0 tx=0x7f4bec02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4c08000640 1 -- 172.21.15.19:0/733832348 shutdown_connections 2024-04-25T11:28:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4c08000640 1 --2- 172.21.15.19:0/733832348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c000cf9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4c08000640 1 -- 172.21.15.19:0/733832348 >> 172.21.15.19:0/733832348 conn(0x7f4c000d1ae0 msgr2=0x7f4c000d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4c08000640 1 -- 172.21.15.19:0/733832348 shutdown_connections 2024-04-25T11:28:26.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.445+0000 7f4c08000640 1 -- 172.21.15.19:0/733832348 wait complete. 2024-04-25T11:28:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.448+0000 7f4c08000640 1 Processor -- start 2024-04-25T11:28:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.448+0000 7f4c08000640 1 -- start start 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.449+0000 7f4c08000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c0006c5c0 0x7f4c00142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.449+0000 7f4c08000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c00157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.449+0000 7f4c08000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c001443c0 0x7f4c00159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.449+0000 7f4c08000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0015edf0 con 0x7f4c000cf5e0 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.449+0000 7f4c08000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0015f160 con 0x7f4c0006c5c0 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.449+0000 7f4c08000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c001447d0 con 0x7f4c001443c0 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c00157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:26.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c00157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47666/0 (socket says 172.21.15.19:47666) 2024-04-25T11:28:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 -- 172.21.15.19:0/411510536 learned_addr learned my addr 172.21.15.19:0/411510536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c05600640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c0006c5c0 0x7f4c00142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 -- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c001443c0 msgr2=0x7f4c00159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c001443c0 0x7f4c00159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 -- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c0006c5c0 msgr2=0x7f4c00142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c0006c5c0 0x7f4c00142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.450+0000 7f4c04c00640 1 -- 172.21.15.19:0/411510536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c00157e70 con 0x7f4c000cf5e0 2024-04-25T11:28:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.451+0000 7f4c04c00640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c00157930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097a0 tx=0x7f4bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.451+0000 7f4bfd600640 1 -- 172.21.15.19:0/411510536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec041070 con 0x7f4c000cf5e0 2024-04-25T11:28:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.451+0000 7f4bfd600640 1 -- 172.21.15.19:0/411510536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bec03c870 con 0x7f4c000cf5e0 2024-04-25T11:28:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.451+0000 7f4bfd600640 1 -- 172.21.15.19:0/411510536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec037040 con 0x7f4c000cf5e0 2024-04-25T11:28:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.451+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c001f7ba0 con 0x7f4c000cf5e0 2024-04-25T11:28:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.451+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c001f8160 con 0x7f4c000cf5e0 2024-04-25T11:28:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.452+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c00050740 con 0x7f4c000cf5e0 2024-04-25T11:28:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.453+0000 7f4bfd600640 1 -- 172.21.15.19:0/411510536 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4bec044070 con 0x7f4c000cf5e0 2024-04-25T11:28:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.454+0000 7f4bfd600640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4be00744b0 0x7f4be00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.455+0000 7f4bfd600640 1 -- 172.21.15.19:0/411510536 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f4bec0b5b80 con 0x7f4c000cf5e0 2024-04-25T11:28:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.455+0000 7f4c05600640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4be00744b0 0x7f4be00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:26.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.456+0000 7f4c05600640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4be00744b0 0x7f4be00769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40012e0 tx=0x7f4bf4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:26.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.459+0000 7f4bfd600640 1 -- 172.21.15.19:0/411510536 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bec0802e0 con 0x7f4c000cf5e0 2024-04-25T11:28:26.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.682+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c0006f950 con 0x7f4c000cf5e0 2024-04-25T11:28:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.683+0000 7f4bfd600640 1 -- 172.21.15.19:0/411510536 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f4bec07fbe0 con 0x7f4c000cf5e0 2024-04-25T11:28:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:28:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4be00744b0 msgr2=0x7f4be00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4be00744b0 0x7f4be00769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40012e0 tx=0x7f4bf4009040 comp rx=0 tx=0).stop 2024-04-25T11:28:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 msgr2=0x7f4c00157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:26.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c00157930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0097a0 tx=0x7f4bec0046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 shutdown_connections 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4be00744b0 0x7f4be00769c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c001443c0 0x7f4c00159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000cf5e0 0x7f4c00157930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 --2- 172.21.15.19:0/411510536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c0006c5c0 0x7f4c00142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 >> 172.21.15.19:0/411510536 conn(0x7f4c000d1ae0 msgr2=0x7f4c0006d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 shutdown_connections 2024-04-25T11:28:26.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:26.686+0000 7f4c08000640 1 -- 172.21.15.19:0/411510536 wait complete. 2024-04-25T11:28:26.703 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T11:28:26.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:26.703 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T11:28:26.800 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:28:26.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:26.800 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T11:28:26.856 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:28:26.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:28:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.156+0000 7f9c2fe00640 1 Processor -- start 2024-04-25T11:28:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.156+0000 7f9c2fe00640 1 -- start start 2024-04-25T11:28:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.158+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3006c5c0 0x7f9c300cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.158+0000 7f9c2fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c300d1230 con 0x7f9c30144370 2024-04-25T11:28:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.158+0000 7f9c2fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c300d06b0 con 0x7f9c300b4ea0 2024-04-25T11:28:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.158+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c300d0990 con 0x7f9c3006c5c0 2024-04-25T11:28:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.159+0000 7f9c2e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9c30144370 0x7f9c300b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53228/0 (socket says 172.21.15.19:53228) 2024-04-25T11:28:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.159+0000 7f9c2e000640 1 -- 172.21.15.19:0/350291360 learned_addr learned my addr 172.21.15.19:0/350291360 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.159+0000 7f9c2ea00640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3006c5c0 0x7f9c300cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.159+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510783998 0 0) 0x7f9c300d1230 con 0x7f9c30144370 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.159+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c14003520 con 0x7f9c30144370 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3590922527 0 0) 0x7f9c14003520 con 0x7f9c30144370 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3006c5c0 msgr2=0x7f9c300cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3006c5c0 0x7f9c300cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 >> v1:172.21.15.169:6789/0 conn(0x7f9c300b4ea0 legacy=0x7f9c3015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c300d0c70 con 0x7f9c30144370 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9c200019f0 con 0x7f9c30144370 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 >> v1:172.21.15.19:6789/0 conn(0x7f9c30144370 legacy=0x7f9c300b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c140039e0 0x7f9c14003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c14004800 0x7f9c14004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3006c5c0 0x7f9c140052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c300d0990 con 0x7f9c14004800 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1400bb50 con 0x7f9c140039e0 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1400bd50 con 0x7f9c3006c5c0 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.160+0000 7f9c2e000640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c14004800 0x7f9c14004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.161+0000 7f9c2ea00640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c140039e0 0x7f9c14003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.161+0000 7f9c2e000640 1 -- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3006c5c0 msgr2=0x7f9c140052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.161+0000 7f9c2e000640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3006c5c0 0x7f9c140052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.161+0000 7f9c2e000640 1 -- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c140039e0 msgr2=0x7f9c14003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.161+0000 7f9c2e000640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c140039e0 0x7f9c14003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.161+0000 7f9c2e000640 1 -- 172.21.15.19:0/350291360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c14003520 con 0x7f9c14004800 2024-04-25T11:28:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.162+0000 7f9c2e000640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c14004800 0x7f9c14004c10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c300b5550 tx=0x7f9c14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c636dc87606d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.162+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000a3b0 con 0x7f9c14004800 2024-04-25T11:28:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.162+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2002f070 con 0x7f9c14004800 2024-04-25T11:28:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.162+0000 7f9c2d600640 1 -- 172.21.15.19:0/350291360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2002bba0 con 0x7f9c14004800 2024-04-25T11:28:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.163+0000 7f9c2fe00640 1 -- 172.21.15.19:0/350291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c14004800 msgr2=0x7f9c14004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.163+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/350291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c14004800 0x7f9c14004c10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9c300b5550 tx=0x7f9c14005830 comp rx=0 tx=0).stop 2024-04-25T11:28:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.163+0000 7f9c2f400640 1 -- 172.21.15.19:0/350291360 reap_dead start 2024-04-25T11:28:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.163+0000 7f9c2fe00640 1 -- 172.21.15.19:0/350291360 shutdown_connections 2024-04-25T11:28:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.163+0000 7f9c2fe00640 1 -- 172.21.15.19:0/350291360 >> 172.21.15.19:0/350291360 conn(0x7f9c300d1720 msgr2=0x7f9c300d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.163+0000 7f9c2fe00640 1 -- 172.21.15.19:0/350291360 shutdown_connections 2024-04-25T11:28:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.163+0000 7f9c2fe00640 1 -- 172.21.15.19:0/350291360 wait complete. 2024-04-25T11:28:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.166+0000 7f9c2fe00640 1 Processor -- start 2024-04-25T11:28:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.166+0000 7f9c2fe00640 1 -- start start 2024-04-25T11:28:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.167+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c14004800 0x7f9c3015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.167+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c301f76a0 0x7f9c30152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.167+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c30161ae0 0x7f9c30161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.167+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c300d0450 con 0x7f9c30161ae0 2024-04-25T11:28:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.167+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c300d02d0 con 0x7f9c301f76a0 2024-04-25T11:28:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.167+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c30147760 con 0x7f9c14004800 2024-04-25T11:28:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c30161ae0 0x7f9c30161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c30161ae0 0x7f9c30161ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47692/0 (socket says 172.21.15.19:47692) 2024-04-25T11:28:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 -- 172.21.15.19:0/861794734 learned_addr learned my addr 172.21.15.19:0/861794734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2ea00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c14004800 0x7f9c3015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2e000640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c301f76a0 0x7f9c30152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 -- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c14004800 msgr2=0x7f9c3015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c14004800 0x7f9c3015c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 -- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c301f76a0 msgr2=0x7f9c30152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c301f76a0 0x7f9c30152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2f400640 1 -- 172.21.15.19:0/861794734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c30162430 con 0x7f9c30161ae0 2024-04-25T11:28:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.168+0000 7f9c2e000640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c301f76a0 0x7f9c30152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.169+0000 7f9c2f400640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c30161ae0 0x7f9c30161ef0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c300b69c0 tx=0x7f9c2800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.169+0000 7f9c26a00640 1 -- 172.21.15.19:0/861794734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800cea0 con 0x7f9c30161ae0 2024-04-25T11:28:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.169+0000 7f9c26a00640 1 -- 172.21.15.19:0/861794734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c280074e0 con 0x7f9c30161ae0 2024-04-25T11:28:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.169+0000 7f9c26a00640 1 -- 172.21.15.19:0/861794734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28004260 con 0x7f9c30161ae0 2024-04-25T11:28:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.169+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c301f7eb0 con 0x7f9c30161ae0 2024-04-25T11:28:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.169+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c301f8470 con 0x7f9c30161ae0 2024-04-25T11:28:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.170+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c30050740 con 0x7f9c30161ae0 2024-04-25T11:28:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.172+0000 7f9c26a00640 1 -- 172.21.15.19:0/861794734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c280057c0 con 0x7f9c30161ae0 2024-04-25T11:28:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.173+0000 7f9c26a00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c0c0744b0 0x7f9c0c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.173+0000 7f9c26a00640 1 -- 172.21.15.19:0/861794734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f9c280944b0 con 0x7f9c30161ae0 2024-04-25T11:28:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.173+0000 7f9c2ea00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c0c0744b0 0x7f9c0c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.174+0000 7f9c2ea00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c0c0744b0 0x7f9c0c0769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009a10 tx=0x7f9c18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.177+0000 7f9c26a00640 1 -- 172.21.15.19:0/861794734 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c30162430 con 0x7f9c30161ae0 2024-04-25T11:28:27.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.424+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c30162890 con 0x7f9c30161ae0 2024-04-25T11:28:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.426+0000 7f9c26a00640 1 -- 172.21.15.19:0/861794734 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f9c2805eb80 con 0x7f9c30161ae0 2024-04-25T11:28:27.418 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:27.418 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:28:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:28:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.428+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c0c0744b0 msgr2=0x7f9c0c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.428+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c0c0744b0 0x7f9c0c0769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009a10 tx=0x7f9c18008040 comp rx=0 tx=0).stop 2024-04-25T11:28:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.428+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c30161ae0 msgr2=0x7f9c30161ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.428+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c30161ae0 0x7f9c30161ef0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9c300b69c0 tx=0x7f9c2800c820 comp rx=0 tx=0).stop 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 shutdown_connections 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c0c0744b0 0x7f9c0c0769c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c301f76a0 0x7f9c30152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c30161ae0 0x7f9c30161ef0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/861794734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c14004800 0x7f9c3015c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 >> 172.21.15.19:0/861794734 conn(0x7f9c300d1720 msgr2=0x7f9c3006cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 shutdown_connections 2024-04-25T11:28:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.429+0000 7f9c2fe00640 1 -- 172.21.15.19:0/861794734 wait complete. 2024-04-25T11:28:27.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.741+0000 7f3670a00640 1 Processor -- start 2024-04-25T11:28:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.741+0000 7f3670a00640 1 -- start start 2024-04-25T11:28:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.743+0000 7f3670a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f366c06c5c0 0x7f366c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.743+0000 7f3670a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f366c144370 0x7f366c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3670a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366c0b50e0 con 0x7f366c0b4cd0 2024-04-25T11:28:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3670a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f366c0d1230 con 0x7f366c144370 2024-04-25T11:28:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3670a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f366c0d06b0 con 0x7f366c06c5c0 2024-04-25T11:28:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3663e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f366c144370 0x7f366c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3663e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f366c144370 0x7f366c0b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47708/0 (socket says 172.21.15.19:47708) 2024-04-25T11:28:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3663e00640 1 -- 172.21.15.19:0/2335371280 learned_addr learned my addr 172.21.15.19:0/2335371280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3663e00640 1 -- 172.21.15.19:0/2335371280 >> v2:172.21.15.169:3300/0 conn(0x7f366c06c5c0 msgr2=0x7f366c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f366b400640 1 --2- 172.21.15.19:0/2335371280 >> v2:172.21.15.169:3300/0 conn(0x7f366c06c5c0 0x7f366c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3663e00640 1 --2- 172.21.15.19:0/2335371280 >> v2:172.21.15.169:3300/0 conn(0x7f366c06c5c0 0x7f366c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.744+0000 7f3663e00640 1 -- 172.21.15.19:0/2335371280 >> v1:172.21.15.169:6789/0 conn(0x7f366c0b4cd0 legacy=0x7f366c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.745+0000 7f3663e00640 1 -- 172.21.15.19:0/2335371280 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c0d0990 con 0x7f366c144370 2024-04-25T11:28:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.746+0000 7f3663e00640 1 --2- 172.21.15.19:0/2335371280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c144370 0x7f366c0b4790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f366c0726b0 tx=0x7f365402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5986d68be0fc6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:27.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.746+0000 7f366aa00640 1 -- 172.21.15.19:0/2335371280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654037070 con 0x7f366c144370 2024-04-25T11:28:27.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.746+0000 7f366aa00640 1 -- 172.21.15.19:0/2335371280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3654032440 con 0x7f366c144370 2024-04-25T11:28:27.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.746+0000 7f366aa00640 1 -- 172.21.15.19:0/2335371280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654032810 con 0x7f366c144370 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 -- 172.21.15.19:0/2335371280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c144370 msgr2=0x7f366c0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 --2- 172.21.15.19:0/2335371280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c144370 0x7f366c0b4790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f366c0726b0 tx=0x7f365402a230 comp rx=0 tx=0).stop 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 -- 172.21.15.19:0/2335371280 shutdown_connections 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 --2- 172.21.15.19:0/2335371280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c144370 0x7f366c0b4790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 --2- 172.21.15.19:0/2335371280 >> v2:172.21.15.169:3300/0 conn(0x7f366c06c5c0 0x7f366c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 -- 172.21.15.19:0/2335371280 >> 172.21.15.19:0/2335371280 conn(0x7f366c0d1720 msgr2=0x7f366c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 -- 172.21.15.19:0/2335371280 shutdown_connections 2024-04-25T11:28:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.747+0000 7f3670a00640 1 -- 172.21.15.19:0/2335371280 wait complete. 2024-04-25T11:28:27.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.749+0000 7f3670a00640 1 Processor -- start 2024-04-25T11:28:27.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.750+0000 7f3670a00640 1 -- start start 2024-04-25T11:28:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3670a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f366c06c5c0 0x7f366c1602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3670a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c0b4cd0 0x7f366c133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3670a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f366c144370 0x7f366c1f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3670a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0d02d0 con 0x7f366c0b4cd0 2024-04-25T11:28:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3670a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0d0f50 con 0x7f366c144370 2024-04-25T11:28:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3670a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f366c0d0450 con 0x7f366c06c5c0 2024-04-25T11:28:27.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3663e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c0b4cd0 0x7f366c133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3663e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c0b4cd0 0x7f366c133ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47712/0 (socket says 172.21.15.19:47712) 2024-04-25T11:28:27.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.751+0000 7f3663e00640 1 -- 172.21.15.19:0/1824607658 learned_addr learned my addr 172.21.15.19:0/1824607658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.752+0000 7f3663e00640 1 -- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f366c06c5c0 msgr2=0x7f366c1602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.752+0000 7f366b400640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f366c06c5c0 0x7f366c1602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.752+0000 7f3663e00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f366c06c5c0 0x7f366c1602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.752+0000 7f3663e00640 1 -- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f366c144370 msgr2=0x7f366c1f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.752+0000 7f3663e00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f366c144370 0x7f366c1f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.752+0000 7f3663e00640 1 -- 172.21.15.19:0/1824607658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c160b50 con 0x7f366c0b4cd0 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.752+0000 7f3663e00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c0b4cd0 0x7f366c133ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f365402a270 tx=0x7f36540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.753+0000 7f3663400640 1 -- 172.21.15.19:0/1824607658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654041020 con 0x7f366c0b4cd0 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.753+0000 7f3663400640 1 -- 172.21.15.19:0/1824607658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3654030990 con 0x7f366c0b4cd0 2024-04-25T11:28:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.753+0000 7f3663400640 1 -- 172.21.15.19:0/1824607658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654037040 con 0x7f366c0b4cd0 2024-04-25T11:28:27.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.753+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c160e30 con 0x7f366c0b4cd0 2024-04-25T11:28:27.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.753+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1613f0 con 0x7f366c0b4cd0 2024-04-25T11:28:27.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.754+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c050740 con 0x7f366c0b4cd0 2024-04-25T11:28:27.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.755+0000 7f3663400640 1 -- 172.21.15.19:0/1824607658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3654044020 con 0x7f366c0b4cd0 2024-04-25T11:28:27.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.757+0000 7f3663400640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36480744b0 0x7f36480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:27.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.757+0000 7f3663400640 1 -- 172.21.15.19:0/1824607658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f36540b5d40 con 0x7f366c0b4cd0 2024-04-25T11:28:27.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.758+0000 7f366b400640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36480744b0 0x7f36480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:27.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.759+0000 7f366b400640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36480744b0 0x7f36480769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f365c00b7c0 tx=0x7f365c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:27.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.762+0000 7f3663400640 1 -- 172.21.15.19:0/1824607658 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f365404b030 con 0x7f366c0b4cd0 2024-04-25T11:28:27.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.967+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f366c0d1cd0 con 0x7f366c0b4cd0 2024-04-25T11:28:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.969+0000 7f3663400640 1 -- 172.21.15.19:0/1824607658 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f366c0b50e0 con 0x7f366c0b4cd0 2024-04-25T11:28:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:27.962 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36480744b0 msgr2=0x7f36480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36480744b0 0x7f36480769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f365c00b7c0 tx=0x7f365c008040 comp rx=0 tx=0).stop 2024-04-25T11:28:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c0b4cd0 msgr2=0x7f366c133ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c0b4cd0 0x7f366c133ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f365402a270 tx=0x7f36540046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 shutdown_connections 2024-04-25T11:28:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36480744b0 0x7f36480769c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f366c144370 0x7f366c1f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f366c0b4cd0 0x7f366c133ee0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 --2- 172.21.15.19:0/1824607658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f366c06c5c0 0x7f366c1602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 >> 172.21.15.19:0/1824607658 conn(0x7f366c0d1720 msgr2=0x7f366c146790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 shutdown_connections 2024-04-25T11:28:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:27.972+0000 7f3670a00640 1 -- 172.21.15.19:0/1824607658 wait complete. 2024-04-25T11:28:27.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:27:23.327873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:27:24.079949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:27.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:28.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.287+0000 7f70e4200640 1 Processor -- start 2024-04-25T11:28:28.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.287+0000 7f70e4200640 1 -- start start 2024-04-25T11:28:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.289+0000 7f70e4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15b710 0x7f70dc0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e4200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f70dc15bb20 0x7f70dc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e4200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70dc06db10 0x7f70dc165100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e4200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f70dc169280 con 0x7f70dc06db10 2024-04-25T11:28:28.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e4200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f70dc06df20 con 0x7f70dc15bb20 2024-04-25T11:28:28.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc168670 con 0x7f70dc15b710 2024-04-25T11:28:28.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e2200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70dc06db10 0x7f70dc165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e2200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70dc06db10 0x7f70dc165100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47728/0 (socket says 172.21.15.19:47728) 2024-04-25T11:28:28.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e2200640 1 -- 172.21.15.19:0/2875664836 learned_addr learned my addr 172.21.15.19:0/2875664836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:28.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e0e00640 1 --2- 172.21.15.19:0/2875664836 >> v2:172.21.15.169:3300/0 conn(0x7f70dc15bb20 0x7f70dc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.290+0000 7f70e1800640 1 --2- 172.21.15.19:0/2875664836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15b710 0x7f70dc0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.291+0000 7f70e2200640 1 -- 172.21.15.19:0/2875664836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15b710 msgr2=0x7f70dc0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:28.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.291+0000 7f70e2200640 1 --2- 172.21.15.19:0/2875664836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15b710 0x7f70dc0d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.291+0000 7f70e2200640 1 -- 172.21.15.19:0/2875664836 >> v2:172.21.15.169:3300/0 conn(0x7f70dc15bb20 msgr2=0x7f70dc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:28.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.291+0000 7f70e2200640 1 --2- 172.21.15.19:0/2875664836 >> v2:172.21.15.169:3300/0 conn(0x7f70dc15bb20 0x7f70dc06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.291+0000 7f70e2200640 1 -- 172.21.15.19:0/2875664836 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc168950 con 0x7f70dc06db10 2024-04-25T11:28:28.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.291+0000 7f70e0e00640 1 --2- 172.21.15.19:0/2875664836 >> v2:172.21.15.169:3300/0 conn(0x7f70dc15bb20 0x7f70dc06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.292+0000 7f70e2200640 1 --2- 172.21.15.19:0/2875664836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc06db10 0x7f70dc165100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0b4530 tx=0x7f70d402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd56199d6a026324 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.293+0000 7f70dbe00640 1 -- 172.21.15.19:0/2875664836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4036070 con 0x7f70dc06db10 2024-04-25T11:28:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.293+0000 7f70dbe00640 1 -- 172.21.15.19:0/2875664836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d4031600 con 0x7f70dc06db10 2024-04-25T11:28:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.293+0000 7f70dbe00640 1 -- 172.21.15.19:0/2875664836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d40319d0 con 0x7f70dc06db10 2024-04-25T11:28:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.293+0000 7f70e4200640 1 -- 172.21.15.19:0/2875664836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc06db10 msgr2=0x7f70dc165100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.293+0000 7f70e4200640 1 --2- 172.21.15.19:0/2875664836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc06db10 0x7f70dc165100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0b4530 tx=0x7f70d402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.294+0000 7f70e4200640 1 -- 172.21.15.19:0/2875664836 shutdown_connections 2024-04-25T11:28:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.294+0000 7f70e4200640 1 --2- 172.21.15.19:0/2875664836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc06db10 0x7f70dc165100 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.294+0000 7f70e4200640 1 --2- 172.21.15.19:0/2875664836 >> v2:172.21.15.169:3300/0 conn(0x7f70dc15bb20 0x7f70dc06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.294+0000 7f70e4200640 1 --2- 172.21.15.19:0/2875664836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15b710 0x7f70dc0d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.294+0000 7f70e4200640 1 -- 172.21.15.19:0/2875664836 >> 172.21.15.19:0/2875664836 conn(0x7f70dc073090 msgr2=0x7f70dc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.294+0000 7f70e4200640 1 -- 172.21.15.19:0/2875664836 shutdown_connections 2024-04-25T11:28:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.294+0000 7f70e4200640 1 -- 172.21.15.19:0/2875664836 wait complete. 2024-04-25T11:28:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.296+0000 7f70e4200640 1 Processor -- start 2024-04-25T11:28:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.296+0000 7f70e4200640 1 -- start start 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70dc06db10 0x7f70dc15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e4200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc15b710 0x7f70dc1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15bb20 0x7f70dc15f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e4200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0b6180 con 0x7f70dc15b710 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc168f10 con 0x7f70dc06db10 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0d1460 con 0x7f70dc15bb20 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc15b710 0x7f70dc1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc15b710 0x7f70dc1541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47734/0 (socket says 172.21.15.19:47734) 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.298+0000 7f70e0e00640 1 -- 172.21.15.19:0/216180797 learned_addr learned my addr 172.21.15.19:0/216180797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e1800640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70dc06db10 0x7f70dc15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e2200640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15bb20 0x7f70dc15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e0e00640 1 -- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15bb20 msgr2=0x7f70dc15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e0e00640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15bb20 0x7f70dc15f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e0e00640 1 -- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70dc06db10 msgr2=0x7f70dc15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e0e00640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70dc06db10 0x7f70dc15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e0e00640 1 -- 172.21.15.19:0/216180797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc1598b0 con 0x7f70dc15b710 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70e0e00640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc15b710 0x7f70dc1541c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f70c800d800 tx=0x7f70c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.299+0000 7f70d9600640 1 -- 172.21.15.19:0/216180797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8004030 con 0x7f70dc15b710 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.300+0000 7f70d9600640 1 -- 172.21.15.19:0/216180797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c80041d0 con 0x7f70dc15b710 2024-04-25T11:28:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.300+0000 7f70d9600640 1 -- 172.21.15.19:0/216180797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c800b6c0 con 0x7f70dc15b710 2024-04-25T11:28:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.300+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc159b90 con 0x7f70dc15b710 2024-04-25T11:28:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.300+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc154700 con 0x7f70dc15b710 2024-04-25T11:28:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.301+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc15ed00 con 0x7f70dc15b710 2024-04-25T11:28:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.302+0000 7f70d9600640 1 -- 172.21.15.19:0/216180797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f70c800b860 con 0x7f70dc15b710 2024-04-25T11:28:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.303+0000 7f70d9600640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70bc0744e0 0x7f70bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.303+0000 7f70d9600640 1 -- 172.21.15.19:0/216180797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f70c80954e0 con 0x7f70dc15b710 2024-04-25T11:28:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.304+0000 7f70e1800640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70bc0744e0 0x7f70bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.305+0000 7f70e1800640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70bc0744e0 0x7f70bc0769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70d0006fd0 tx=0x7f70d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:28.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.308+0000 7f70d9600640 1 -- 172.21.15.19:0/216180797 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70c805fc40 con 0x7f70dc15b710 2024-04-25T11:28:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.516+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f70dc050740 con 0x7f70dc15b710 2024-04-25T11:28:28.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.518+0000 7f70d9600640 1 -- 172.21.15.19:0/216180797 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f70c805f540 con 0x7f70dc15b710 2024-04-25T11:28:28.510 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.520+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70bc0744e0 msgr2=0x7f70bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70bc0744e0 0x7f70bc0769f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f70d0006fd0 tx=0x7f70d0008040 comp rx=0 tx=0).stop 2024-04-25T11:28:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc15b710 msgr2=0x7f70dc1541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc15b710 0x7f70dc1541c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f70c800d800 tx=0x7f70c800dcc0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 shutdown_connections 2024-04-25T11:28:28.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70bc0744e0 0x7f70bc0769f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70dc15bb20 0x7f70dc15f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70dc15b710 0x7f70dc1541c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 --2- 172.21.15.19:0/216180797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70dc06db10 0x7f70dc15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 >> 172.21.15.19:0/216180797 conn(0x7f70dc073090 msgr2=0x7f70dc0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 shutdown_connections 2024-04-25T11:28:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.521+0000 7f70e4200640 1 -- 172.21.15.19:0/216180797 wait complete. 2024-04-25T11:28:28.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.843+0000 7f3733200640 1 Processor -- start 2024-04-25T11:28:28.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.843+0000 7f3733200640 1 -- start start 2024-04-25T11:28:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.845+0000 7f3733200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f372c0ce520 0x7f372c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.845+0000 7f3733200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372c0cda60 con 0x7f372c0ce930 2024-04-25T11:28:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.845+0000 7f3733200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372c15dfe0 con 0x7f372c06dce0 2024-04-25T11:28:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.845+0000 7f3733200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f372c15e2c0 con 0x7f372c0ce520 2024-04-25T11:28:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3731e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f372c0ce520 0x7f372c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3731400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f372c0ce930 0x7f372c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53238/0 (socket says 172.21.15.19:53238) 2024-04-25T11:28:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3731400640 1 -- 172.21.15.19:0/3235534126 learned_addr learned my addr 172.21.15.19:0/3235534126 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3731e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f372c0ce520 0x7f372c15d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47744/0 (socket says 172.21.15.19:47744) 2024-04-25T11:28:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3731e00640 1 -- 172.21.15.19:0/3235534126 >> v1:172.21.15.169:6789/0 conn(0x7f372c06dce0 legacy=0x7f372c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3731e00640 1 -- 172.21.15.19:0/3235534126 >> v1:172.21.15.19:6789/0 conn(0x7f372c0ce930 legacy=0x7f372c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3731e00640 1 -- 172.21.15.19:0/3235534126 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c0cdd20 con 0x7f372c0ce520 2024-04-25T11:28:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.846+0000 7f3730a00640 1 -- 172.21.15.19:0/3235534126 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446841869 0 0) 0x7f372c0cda60 con 0x7f372c0ce930 2024-04-25T11:28:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.847+0000 7f3731e00640 1 --2- 172.21.15.19:0/3235534126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce520 0x7f372c15d4b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f372000ba40 tx=0x7f372002c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261c8b871fa7146a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.848+0000 7f3730a00640 1 -- 172.21.15.19:0/3235534126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372003a070 con 0x7f372c0ce520 2024-04-25T11:28:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.848+0000 7f3730a00640 1 -- 172.21.15.19:0/3235534126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3720031070 con 0x7f372c0ce520 2024-04-25T11:28:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.848+0000 7f3730a00640 1 -- 172.21.15.19:0/3235534126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200328f0 con 0x7f372c0ce520 2024-04-25T11:28:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.848+0000 7f3733200640 1 -- 172.21.15.19:0/3235534126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce520 msgr2=0x7f372c15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.848+0000 7f3733200640 1 --2- 172.21.15.19:0/3235534126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce520 0x7f372c15d4b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f372000ba40 tx=0x7f372002c810 comp rx=0 tx=0).stop 2024-04-25T11:28:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.849+0000 7f3733200640 1 -- 172.21.15.19:0/3235534126 shutdown_connections 2024-04-25T11:28:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.849+0000 7f3733200640 1 --2- 172.21.15.19:0/3235534126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce520 0x7f372c15d4b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.849+0000 7f3733200640 1 -- 172.21.15.19:0/3235534126 >> 172.21.15.19:0/3235534126 conn(0x7f372c072f40 msgr2=0x7f372c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.849+0000 7f3733200640 1 -- 172.21.15.19:0/3235534126 shutdown_connections 2024-04-25T11:28:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.849+0000 7f3733200640 1 -- 172.21.15.19:0/3235534126 wait complete. 2024-04-25T11:28:28.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.851+0000 7f3733200640 1 Processor -- start 2024-04-25T11:28:28.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.852+0000 7f3733200640 1 -- start start 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.853+0000 7f3733200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f372c0ce520 0x7f372c157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.853+0000 7f3733200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce930 0x7f372c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.853+0000 7f3733200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f372c0d07c0 0x7f372c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.853+0000 7f3733200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f372c0b73d0 con 0x7f372c0ce930 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.853+0000 7f3733200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f372c160d30 con 0x7f372c0d07c0 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.853+0000 7f3733200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f372c0b7550 con 0x7f372c0ce520 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce930 0x7f372c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce930 0x7f372c0d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47758/0 (socket says 172.21.15.19:47758) 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 -- 172.21.15.19:0/463047523 learned_addr learned my addr 172.21.15.19:0/463047523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731e00640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f372c0ce520 0x7f372c157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 -- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f372c0ce520 msgr2=0x7f372c157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f372c0ce520 0x7f372c157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3732800640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f372c0d07c0 0x7f372c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 -- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f372c0d07c0 msgr2=0x7f372c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f372c0d07c0 0x7f372c15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731400640 1 -- 172.21.15.19:0/463047523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c15bd50 con 0x7f372c0ce930 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.854+0000 7f3731e00640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f372c0ce520 0x7f372c157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.855+0000 7f3731400640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce930 0x7f372c0d0280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f371c00c460 tx=0x7f371c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.855+0000 7f3726000640 1 -- 172.21.15.19:0/463047523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c018070 con 0x7f372c0ce930 2024-04-25T11:28:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.855+0000 7f3726000640 1 -- 172.21.15.19:0/463047523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c007500 con 0x7f372c0ce930 2024-04-25T11:28:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.855+0000 7f3726000640 1 -- 172.21.15.19:0/463047523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004a20 con 0x7f372c0ce930 2024-04-25T11:28:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.855+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1589c0 con 0x7f372c0ce930 2024-04-25T11:28:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.855+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c158f80 con 0x7f372c0ce930 2024-04-25T11:28:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.856+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c076c60 con 0x7f372c0ce930 2024-04-25T11:28:28.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.858+0000 7f3726000640 1 -- 172.21.15.19:0/463047523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f371c004bc0 con 0x7f372c0ce930 2024-04-25T11:28:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.859+0000 7f3726000640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37100744b0 0x7f37100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.859+0000 7f3726000640 1 -- 172.21.15.19:0/463047523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f371c094590 con 0x7f372c0ce930 2024-04-25T11:28:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.859+0000 7f3731e00640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37100744b0 0x7f37100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.861+0000 7f3731e00640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37100744b0 0x7f37100769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3720037000 tx=0x7f3720032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:28.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:28.865+0000 7f3726000640 1 -- 172.21.15.19:0/463047523 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f371c05ed00 con 0x7f372c0ce930 2024-04-25T11:28:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.069+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f372c151e40 con 0x7f372c0ce930 2024-04-25T11:28:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.070+0000 7f3726000640 1 -- 172.21.15.19:0/463047523 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f372c151e40 con 0x7f372c0ce930 2024-04-25T11:28:29.062 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37100744b0 msgr2=0x7f37100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37100744b0 0x7f37100769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3720037000 tx=0x7f3720032000 comp rx=0 tx=0).stop 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce930 msgr2=0x7f372c0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce930 0x7f372c0d0280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f371c00c460 tx=0x7f371c00c920 comp rx=0 tx=0).stop 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 shutdown_connections 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37100744b0 0x7f37100769c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f372c0d07c0 0x7f372c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f372c0ce930 0x7f372c0d0280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 --2- 172.21.15.19:0/463047523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f372c0ce520 0x7f372c157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.073+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 >> 172.21.15.19:0/463047523 conn(0x7f372c072f40 msgr2=0x7f372c0b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.074+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 shutdown_connections 2024-04-25T11:28:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.074+0000 7f3733200640 1 -- 172.21.15.19:0/463047523 wait complete. 2024-04-25T11:28:29.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:29.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.388+0000 7f53c8000640 1 Processor -- start 2024-04-25T11:28:29.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.388+0000 7f53c8000640 1 -- start start 2024-04-25T11:28:29.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.390+0000 7f53c8000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f53c00cd9a0 0x7f53c015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.390+0000 7f53c8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c006dce0 0x7f53c015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.390+0000 7f53c8000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c00cdea0 con 0x7f53c00b0af0 2024-04-25T11:28:29.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.390+0000 7f53c8000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f53c00ceab0 con 0x7f53c00cd9a0 2024-04-25T11:28:29.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.390+0000 7f53c8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f53c00ce180 con 0x7f53c006dce0 2024-04-25T11:28:29.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.391+0000 7f53c4c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f53c00b0af0 0x7f53c006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53244/0 (socket says 172.21.15.19:53244) 2024-04-25T11:28:29.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.391+0000 7f53c4c00640 1 -- 172.21.15.19:0/2121753675 learned_addr learned my addr 172.21.15.19:0/2121753675 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.391+0000 7f53c5600640 1 --2- 172.21.15.19:0/2121753675 >> v2:172.21.15.169:3300/0 conn(0x7f53c00cd9a0 0x7f53c015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.391+0000 7f53c6000640 1 --2- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c006dce0 0x7f53c015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.391+0000 7f53bfe00640 1 -- 172.21.15.19:0/2121753675 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170711172 0 0) 0x7f53c00cdea0 con 0x7f53c00b0af0 2024-04-25T11:28:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.391+0000 7f53bfe00640 1 -- 172.21.15.19:0/2121753675 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53a8003520 con 0x7f53c00b0af0 2024-04-25T11:28:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.392+0000 7f53bfe00640 1 -- 172.21.15.19:0/2121753675 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3609998013 0 0) 0x7f53a8003520 con 0x7f53c00b0af0 2024-04-25T11:28:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.392+0000 7f53c5600640 1 -- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c006dce0 msgr2=0x7f53c015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:29.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.392+0000 7f53c5600640 1 --2- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c006dce0 0x7f53c015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.392+0000 7f53c5600640 1 -- 172.21.15.19:0/2121753675 >> v1:172.21.15.19:6789/0 conn(0x7f53c00b0af0 legacy=0x7f53c006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.392+0000 7f53c5600640 1 -- 172.21.15.19:0/2121753675 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c00ce460 con 0x7f53c00cd9a0 2024-04-25T11:28:29.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.393+0000 7f53c5600640 1 --2- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c00cd9a0 0x7f53c015f360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f53b00097a0 tx=0x7f53b002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612abeb1f5f19fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:29.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53bfe00640 1 -- 172.21.15.19:0/2121753675 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0037070 con 0x7f53c00cd9a0 2024-04-25T11:28:29.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53bfe00640 1 -- 172.21.15.19:0/2121753675 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b002aba0 con 0x7f53c00cd9a0 2024-04-25T11:28:29.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53bfe00640 1 -- 172.21.15.19:0/2121753675 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4001670 con 0x7f53c00cd9a0 2024-04-25T11:28:29.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53c8000640 1 -- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c00cd9a0 msgr2=0x7f53c015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53c8000640 1 --2- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c00cd9a0 0x7f53c015f360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f53b00097a0 tx=0x7f53b002a110 comp rx=0 tx=0).stop 2024-04-25T11:28:29.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53c8000640 1 -- 172.21.15.19:0/2121753675 shutdown_connections 2024-04-25T11:28:29.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53c8000640 1 --2- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c006dce0 0x7f53c015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.394+0000 7f53c8000640 1 --2- 172.21.15.19:0/2121753675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c00cd9a0 0x7f53c015f360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.395+0000 7f53c8000640 1 -- 172.21.15.19:0/2121753675 >> 172.21.15.19:0/2121753675 conn(0x7f53c0073090 msgr2=0x7f53c00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:29.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.395+0000 7f53c8000640 1 -- 172.21.15.19:0/2121753675 shutdown_connections 2024-04-25T11:28:29.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.395+0000 7f53c8000640 1 -- 172.21.15.19:0/2121753675 wait complete. 2024-04-25T11:28:29.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.397+0000 7f53c8000640 1 Processor -- start 2024-04-25T11:28:29.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.397+0000 7f53c8000640 1 -- start start 2024-04-25T11:28:29.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c8000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53c00b0af0 0x7f53c0160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c8000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c00cd9a0 0x7f53c00af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c8000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c01f94e0 0x7f53c01f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c8000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53c00cef40 con 0x7f53c00b0af0 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c8000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53c00b0f00 con 0x7f53c01f94e0 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c8000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f53c00cf0c0 con 0x7f53c00cd9a0 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53c00b0af0 0x7f53c0160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53c00b0af0 0x7f53c0160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47768/0 (socket says 172.21.15.19:47768) 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c5600640 1 -- 172.21.15.19:0/1370945632 learned_addr learned my addr 172.21.15.19:0/1370945632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c4c00640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c00cd9a0 0x7f53c00af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.399+0000 7f53c6000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c01f94e0 0x7f53c01f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c5600640 1 -- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c00cd9a0 msgr2=0x7f53c00af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c5600640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c00cd9a0 0x7f53c00af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c5600640 1 -- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c01f94e0 msgr2=0x7f53c01f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c5600640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c01f94e0 0x7f53c01f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c5600640 1 -- 172.21.15.19:0/1370945632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c01f9e30 con 0x7f53c00b0af0 2024-04-25T11:28:29.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c6000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c01f94e0 0x7f53c01f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:29.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c5600640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53c00b0af0 0x7f53c0160490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000c00 tx=0x7f53b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:29.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53bd600640 1 -- 172.21.15.19:0/1370945632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0037040 con 0x7f53c00b0af0 2024-04-25T11:28:29.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53bd600640 1 -- 172.21.15.19:0/1370945632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b0004230 con 0x7f53c00b0af0 2024-04-25T11:28:29.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53bd600640 1 -- 172.21.15.19:0/1370945632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0032e60 con 0x7f53c00b0af0 2024-04-25T11:28:29.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c00af970 con 0x7f53c00b0af0 2024-04-25T11:28:29.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.400+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c0160de0 con 0x7f53c00b0af0 2024-04-25T11:28:29.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.402+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c0050740 con 0x7f53c00b0af0 2024-04-25T11:28:29.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.403+0000 7f53bd600640 1 -- 172.21.15.19:0/1370945632 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f53b0003610 con 0x7f53c00b0af0 2024-04-25T11:28:29.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.404+0000 7f53bd600640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53a40744b0 0x7f53a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.404+0000 7f53bd600640 1 -- 172.21.15.19:0/1370945632 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f53b00b54a0 con 0x7f53c00b0af0 2024-04-25T11:28:29.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.404+0000 7f53c4c00640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53a40744b0 0x7f53a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.406+0000 7f53c4c00640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53a40744b0 0x7f53a40769c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53b40011e0 tx=0x7f53b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:29.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.409+0000 7f53bd600640 1 -- 172.21.15.19:0/1370945632 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53c01f9e30 con 0x7f53c00b0af0 2024-04-25T11:28:29.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.620+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f53c0068a70 con 0x7f53c00b0af0 2024-04-25T11:28:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.621+0000 7f53bd600640 1 -- 172.21.15.19:0/1370945632 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f53b0042020 con 0x7f53c00b0af0 2024-04-25T11:28:29.613 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53a40744b0 msgr2=0x7f53a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53a40744b0 0x7f53a40769c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53b40011e0 tx=0x7f53b4009040 comp rx=0 tx=0).stop 2024-04-25T11:28:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53c00b0af0 msgr2=0x7f53c0160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53c00b0af0 0x7f53c0160490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000c00 tx=0x7f53b00046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 shutdown_connections 2024-04-25T11:28:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53a40744b0 0x7f53a40769c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53c01f94e0 0x7f53c01f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53c00cd9a0 0x7f53c00af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 --2- 172.21.15.19:0/1370945632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53c00b0af0 0x7f53c0160490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 >> 172.21.15.19:0/1370945632 conn(0x7f53c0073090 msgr2=0x7f53c00b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 shutdown_connections 2024-04-25T11:28:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.624+0000 7f53c8000640 1 -- 172.21.15.19:0/1370945632 wait complete. 2024-04-25T11:28:29.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:28:29.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.933+0000 7fc993200640 1 Processor -- start 2024-04-25T11:28:29.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.933+0000 7fc993200640 1 -- start start 2024-04-25T11:28:29.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.935+0000 7fc993200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9940cf2f0 0x7fc9940b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.935+0000 7fc993200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc99415a810 0x7fc9940b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.935+0000 7fc993200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9940b4e90 con 0x7fc9941608d0 2024-04-25T11:28:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.935+0000 7fc993200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc99406f870 con 0x7fc9940cf2f0 2024-04-25T11:28:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc993200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940b74d0 con 0x7fc99415a810 2024-04-25T11:28:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9940cf2f0 0x7fc9940b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9940cf2f0 0x7fc9940b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47776/0 (socket says 172.21.15.19:47776) 2024-04-25T11:28:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991e00640 1 -- 172.21.15.19:0/654135668 learned_addr learned my addr 172.21.15.19:0/654135668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991e00640 1 -- 172.21.15.19:0/654135668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc99415a810 msgr2=0x7fc9940b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991e00640 1 --2- 172.21.15.19:0/654135668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc99415a810 0x7fc9940b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991e00640 1 -- 172.21.15.19:0/654135668 >> v1:172.21.15.19:6789/0 conn(0x7fc9941608d0 legacy=0x7fc994160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991e00640 1 -- 172.21.15.19:0/654135668 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9940b77b0 con 0x7fc9940cf2f0 2024-04-25T11:28:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.936+0000 7fc991400640 1 -- 172.21.15.19:0/654135668 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284995024 0 0) 0x7fc9940b4e90 con 0x7fc9941608d0 2024-04-25T11:28:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.938+0000 7fc991e00640 1 --2- 172.21.15.19:0/654135668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9940cf2f0 0x7fc9940b6e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc984009870 tx=0x7fc98402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbd21bb96f26742 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.938+0000 7fc991400640 1 -- 172.21.15.19:0/654135668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984037070 con 0x7fc9940cf2f0 2024-04-25T11:28:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.938+0000 7fc991400640 1 -- 172.21.15.19:0/654135668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc984032440 con 0x7fc9940cf2f0 2024-04-25T11:28:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.938+0000 7fc991400640 1 -- 172.21.15.19:0/654135668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984032810 con 0x7fc9940cf2f0 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 -- 172.21.15.19:0/654135668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9940cf2f0 msgr2=0x7fc9940b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 --2- 172.21.15.19:0/654135668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9940cf2f0 0x7fc9940b6e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc984009870 tx=0x7fc98402a270 comp rx=0 tx=0).stop 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 -- 172.21.15.19:0/654135668 shutdown_connections 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 --2- 172.21.15.19:0/654135668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc99415a810 0x7fc9940b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 --2- 172.21.15.19:0/654135668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9940cf2f0 0x7fc9940b6e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 -- 172.21.15.19:0/654135668 >> 172.21.15.19:0/654135668 conn(0x7fc9940d1720 msgr2=0x7fc9940d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 -- 172.21.15.19:0/654135668 shutdown_connections 2024-04-25T11:28:29.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.939+0000 7fc993200640 1 -- 172.21.15.19:0/654135668 wait complete. 2024-04-25T11:28:29.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.941+0000 7fc993200640 1 Processor -- start 2024-04-25T11:28:29.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.941+0000 7fc993200640 1 -- start start 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc993200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9940cf2f0 0x7fc99415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc993200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc99415a810 0x7fc99415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc993200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9941608d0 0x7fc994155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc993200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415ac20 con 0x7fc9941608d0 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc993200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc99406f500 con 0x7fc99415a810 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc993200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc99406fd00 con 0x7fc9940cf2f0 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc992800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9941608d0 0x7fc994155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc992800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9941608d0 0x7fc994155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47788/0 (socket says 172.21.15.19:47788) 2024-04-25T11:28:29.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.943+0000 7fc992800640 1 -- 172.21.15.19:0/811653017 learned_addr learned my addr 172.21.15.19:0/811653017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc991e00640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9940cf2f0 0x7fc99415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc98be00640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc99415a810 0x7fc99415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc992800640 1 -- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9940cf2f0 msgr2=0x7fc99415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc992800640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9940cf2f0 0x7fc99415bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc992800640 1 -- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc99415a810 msgr2=0x7fc99415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc992800640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc99415a810 0x7fc99415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc992800640 1 -- 172.21.15.19:0/811653017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99415c750 con 0x7fc9941608d0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc98be00640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc99415a810 0x7fc99415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc992800640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9941608d0 0x7fc994155200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00c780 tx=0x7fc98c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc98a000640 1 -- 172.21.15.19:0/811653017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c003b90 con 0x7fc9941608d0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc98a000640 1 -- 172.21.15.19:0/811653017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c003d30 con 0x7fc9941608d0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc98a000640 1 -- 172.21.15.19:0/811653017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c005310 con 0x7fc9941608d0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.944+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99415ca30 con 0x7fc9941608d0 2024-04-25T11:28:29.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.945+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9940ae640 con 0x7fc9941608d0 2024-04-25T11:28:29.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.946+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc994050740 con 0x7fc9941608d0 2024-04-25T11:28:29.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.947+0000 7fc98a000640 1 -- 172.21.15.19:0/811653017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc98c0054b0 con 0x7fc9941608d0 2024-04-25T11:28:29.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.948+0000 7fc98a000640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc9700744b0 0x7fc9700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:29.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.948+0000 7fc98a000640 1 -- 172.21.15.19:0/811653017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fc98c0950a0 con 0x7fc9941608d0 2024-04-25T11:28:29.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.949+0000 7fc991e00640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc9700744b0 0x7fc9700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:29.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.950+0000 7fc991e00640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc9700744b0 0x7fc9700769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc984034000 tx=0x7fc98402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:29.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:29.953+0000 7fc98a000640 1 -- 172.21.15.19:0/811653017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc98c05f890 con 0x7fc9941608d0 2024-04-25T11:28:30.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.206+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc99406c770 con 0x7fc9941608d0 2024-04-25T11:28:30.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.207+0000 7fc98a000640 1 -- 172.21.15.19:0/811653017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fc98c05f190 con 0x7fc9941608d0 2024-04-25T11:28:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:30.199 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:28:30.200 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:28:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc9700744b0 msgr2=0x7fc9700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc9700744b0 0x7fc9700769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc984034000 tx=0x7fc98402e000 comp rx=0 tx=0).stop 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9941608d0 msgr2=0x7fc994155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9941608d0 0x7fc994155200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00c780 tx=0x7fc98c00cc40 comp rx=0 tx=0).stop 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 shutdown_connections 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc9700744b0 0x7fc9700769c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc99415a810 0x7fc99415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9941608d0 0x7fc994155200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 --2- 172.21.15.19:0/811653017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9940cf2f0 0x7fc99415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 >> 172.21.15.19:0/811653017 conn(0x7fc9940d1720 msgr2=0x7fc9940b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 shutdown_connections 2024-04-25T11:28:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.210+0000 7fc993200640 1 -- 172.21.15.19:0/811653017 wait complete. 2024-04-25T11:28:30.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:30.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.516+0000 7f9656200640 1 Processor -- start 2024-04-25T11:28:30.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.516+0000 7f9656200640 1 -- start start 2024-04-25T11:28:30.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.519+0000 7f9656200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f965006e680 0x7f96500b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.519+0000 7f9656200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96500b4e80 0x7f96500cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.519+0000 7f9656200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96500b5290 con 0x7f965008cd60 2024-04-25T11:28:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.519+0000 7f9656200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f96500d1d70 con 0x7f965006e680 2024-04-25T11:28:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.519+0000 7f9656200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96500d1fa0 con 0x7f96500b4e80 2024-04-25T11:28:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.520+0000 7f9654e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f965006e680 0x7f96500b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.520+0000 7f9655800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f965008cd60 0x7f965008d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:44812/0 (socket says 172.21.15.19:44812) 2024-04-25T11:28:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.520+0000 7f9655800640 1 -- 172.21.15.19:0/2125681741 learned_addr learned my addr 172.21.15.19:0/2125681741 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.520+0000 7f964fe00640 1 --2- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96500b4e80 0x7f96500cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.521+0000 7f9654e00640 1 -- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96500b4e80 msgr2=0x7f96500cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.521+0000 7f9654e00640 1 --2- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96500b4e80 0x7f96500cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.521+0000 7f9654e00640 1 -- 172.21.15.19:0/2125681741 >> v1:172.21.15.169:6789/0 conn(0x7f965008cd60 legacy=0x7f965008d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.521+0000 7f9654e00640 1 -- 172.21.15.19:0/2125681741 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96500d2280 con 0x7f965006e680 2024-04-25T11:28:30.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.522+0000 7f9654e00640 1 --2- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f965006e680 0x7f96500b90e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9640009de0 tx=0x7f964002a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481fbead97c1373b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:30.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.522+0000 7f964f400640 1 -- 172.21.15.19:0/2125681741 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640007740 con 0x7f965006e680 2024-04-25T11:28:30.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f964f400640 1 -- 172.21.15.19:0/2125681741 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964002ae50 con 0x7f965006e680 2024-04-25T11:28:30.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f964f400640 1 -- 172.21.15.19:0/2125681741 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640007b00 con 0x7f965006e680 2024-04-25T11:28:30.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f9656200640 1 -- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f965006e680 msgr2=0x7f96500b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f9656200640 1 --2- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f965006e680 0x7f96500b90e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9640009de0 tx=0x7f964002a630 comp rx=0 tx=0).stop 2024-04-25T11:28:30.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f9656200640 1 -- 172.21.15.19:0/2125681741 shutdown_connections 2024-04-25T11:28:30.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f9656200640 1 --2- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96500b4e80 0x7f96500cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f9656200640 1 --2- 172.21.15.19:0/2125681741 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f965006e680 0x7f96500b90e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f9656200640 1 -- 172.21.15.19:0/2125681741 >> 172.21.15.19:0/2125681741 conn(0x7f96500b5460 msgr2=0x7f96500b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:30.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.523+0000 7f9656200640 1 -- 172.21.15.19:0/2125681741 shutdown_connections 2024-04-25T11:28:30.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.524+0000 7f9656200640 1 -- 172.21.15.19:0/2125681741 wait complete. 2024-04-25T11:28:30.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.526+0000 7f9656200640 1 Processor -- start 2024-04-25T11:28:30.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.526+0000 7f9656200640 1 -- start start 2024-04-25T11:28:30.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9656200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f965006e680 0x7f9650161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:30.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9656200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965008cd60 0x7f96501eab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9656200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500b4e80 0x7f965015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9656200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f965006ec40 con 0x7f965006e680 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9656200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f96500d2840 con 0x7f96500b4e80 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9656200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f965008dab0 con 0x7f965008cd60 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9654e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f965006e680 0x7f9650161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9654e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f965006e680 0x7f9650161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47794/0 (socket says 172.21.15.19:47794) 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9654e00640 1 -- 172.21.15.19:0/1128692078 learned_addr learned my addr 172.21.15.19:0/1128692078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f9655800640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500b4e80 0x7f965015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.528+0000 7f964fe00640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965008cd60 0x7f96501eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f9654e00640 1 -- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965008cd60 msgr2=0x7f96501eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f9654e00640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965008cd60 0x7f96501eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f9654e00640 1 -- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500b4e80 msgr2=0x7f965015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f9654e00640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500b4e80 0x7f965015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f9654e00640 1 -- 172.21.15.19:0/1128692078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96501627c0 con 0x7f965006e680 2024-04-25T11:28:30.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f9654e00640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f965006e680 0x7f9650161d00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9640009de0 tx=0x7f9640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:30.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f964cc00640 1 -- 172.21.15.19:0/1128692078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964003f070 con 0x7f965006e680 2024-04-25T11:28:30.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f964cc00640 1 -- 172.21.15.19:0/1128692078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964003b820 con 0x7f965006e680 2024-04-25T11:28:30.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f964cc00640 1 -- 172.21.15.19:0/1128692078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964003bbd0 con 0x7f965006e680 2024-04-25T11:28:30.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.529+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9650162aa0 con 0x7f965006e680 2024-04-25T11:28:30.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.530+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9650163060 con 0x7f965006e680 2024-04-25T11:28:30.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.531+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9650163260 con 0x7f965006e680 2024-04-25T11:28:30.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.532+0000 7f964cc00640 1 -- 172.21.15.19:0/1128692078 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9640044460 con 0x7f965006e680 2024-04-25T11:28:30.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.533+0000 7f964cc00640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96300744b0 0x7f96300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:30.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.533+0000 7f964cc00640 1 -- 172.21.15.19:0/1128692078 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f96400b6930 con 0x7f965006e680 2024-04-25T11:28:30.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.534+0000 7f964fe00640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96300744b0 0x7f96300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.535+0000 7f964fe00640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96300744b0 0x7f96300769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96501556d0 tx=0x7f963c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.538+0000 7f964cc00640 1 -- 172.21.15.19:0/1128692078 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9650163260 con 0x7f965006e680 2024-04-25T11:28:30.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.741+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96500c73d0 con 0x7f965006e680 2024-04-25T11:28:30.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.743+0000 7f964cc00640 1 -- 172.21.15.19:0/1128692078 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7f9640081000 con 0x7f965006e680 2024-04-25T11:28:30.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:30.736 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96300744b0 msgr2=0x7f96300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96300744b0 0x7f96300769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96501556d0 tx=0x7f963c008040 comp rx=0 tx=0).stop 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f965006e680 msgr2=0x7f9650161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f965006e680 0x7f9650161d00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9640009de0 tx=0x7f9640004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 shutdown_connections 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96300744b0 0x7f96300769c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96500b4e80 0x7f965015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f965008cd60 0x7f96501eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 --2- 172.21.15.19:0/1128692078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f965006e680 0x7f9650161d00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:30.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 >> 172.21.15.19:0/1128692078 conn(0x7f96500b5460 msgr2=0x7f96500d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:30.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 shutdown_connections 2024-04-25T11:28:30.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:30.746+0000 7f9656200640 1 -- 172.21.15.19:0/1128692078 wait complete. 2024-04-25T11:28:30.766 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:28:30.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:28:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.053+0000 7fef9cc00640 1 Processor -- start 2024-04-25T11:28:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.053+0000 7fef9cc00640 1 -- start start 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef9cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fef981400a0 0x7fef980b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef9cc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef98070980 0x7fef98142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef9cc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef981431d0 con 0x7fef98142730 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef9cc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fef981434b0 con 0x7fef981400a0 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef9cc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fef980b6090 con 0x7fef98070980 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef95e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fef981400a0 0x7fef980b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef95400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fef98142730 0x7fef98070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53262/0 (socket says 172.21.15.19:53262) 2024-04-25T11:28:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.055+0000 7fef95400640 1 -- 172.21.15.19:0/1746727519 learned_addr learned my addr 172.21.15.19:0/1746727519 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.056+0000 7fef96800640 1 --2- 172.21.15.19:0/1746727519 >> v2:172.21.15.169:3300/0 conn(0x7fef98070980 0x7fef98142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.056+0000 7fef95e00640 1 -- 172.21.15.19:0/1746727519 >> v2:172.21.15.169:3300/0 conn(0x7fef98070980 msgr2=0x7fef98142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.056+0000 7fef95e00640 1 --2- 172.21.15.19:0/1746727519 >> v2:172.21.15.169:3300/0 conn(0x7fef98070980 0x7fef98142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.056+0000 7fef95e00640 1 -- 172.21.15.19:0/1746727519 >> v1:172.21.15.19:6789/0 conn(0x7fef98142730 legacy=0x7fef98070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.056+0000 7fef95e00640 1 -- 172.21.15.19:0/1746727519 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef980b6370 con 0x7fef981400a0 2024-04-25T11:28:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.057+0000 7fef95e00640 1 --2- 172.21.15.19:0/1746727519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef981400a0 0x7fef980b6890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fef980b15c0 tx=0x7fef8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f843575bb92f2563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.057+0000 7fef94a00640 1 -- 172.21.15.19:0/1746727519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88037070 con 0x7fef981400a0 2024-04-25T11:28:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.057+0000 7fef94a00640 1 -- 172.21.15.19:0/1746727519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88032440 con 0x7fef981400a0 2024-04-25T11:28:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.057+0000 7fef94a00640 1 -- 172.21.15.19:0/1746727519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88032810 con 0x7fef981400a0 2024-04-25T11:28:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 -- 172.21.15.19:0/1746727519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef981400a0 msgr2=0x7fef980b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 --2- 172.21.15.19:0/1746727519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef981400a0 0x7fef980b6890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fef980b15c0 tx=0x7fef8802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 -- 172.21.15.19:0/1746727519 shutdown_connections 2024-04-25T11:28:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 --2- 172.21.15.19:0/1746727519 >> v2:172.21.15.169:3300/0 conn(0x7fef98070980 0x7fef98142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 --2- 172.21.15.19:0/1746727519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef981400a0 0x7fef980b6890 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 -- 172.21.15.19:0/1746727519 >> 172.21.15.19:0/1746727519 conn(0x7fef98073050 msgr2=0x7fef98073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 -- 172.21.15.19:0/1746727519 shutdown_connections 2024-04-25T11:28:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.058+0000 7fef9cc00640 1 -- 172.21.15.19:0/1746727519 wait complete. 2024-04-25T11:28:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.060+0000 7fef9cc00640 1 Processor -- start 2024-04-25T11:28:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.060+0000 7fef9cc00640 1 -- start start 2024-04-25T11:28:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef9cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef981400a0 0x7fef98142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef9cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef98142730 0x7fef98139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef9cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9815c040 0x7fef9815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef9cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef980b49a0 con 0x7fef9815c040 2024-04-25T11:28:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef9cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fef980cddb0 con 0x7fef98142730 2024-04-25T11:28:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef9cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fef980b4b20 con 0x7fef981400a0 2024-04-25T11:28:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef96800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9815c040 0x7fef9815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef96800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9815c040 0x7fef9815c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47810/0 (socket says 172.21.15.19:47810) 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.062+0000 7fef96800640 1 -- 172.21.15.19:0/482910669 learned_addr learned my addr 172.21.15.19:0/482910669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef95e00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef981400a0 0x7fef98142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef96800640 1 -- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef981400a0 msgr2=0x7fef98142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef96800640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef981400a0 0x7fef98142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef96800640 1 -- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef98142730 msgr2=0x7fef98139610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef95400640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef98142730 0x7fef98139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef96800640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef98142730 0x7fef98139610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef96800640 1 -- 172.21.15.19:0/482910669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9815c990 con 0x7fef9815c040 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef96800640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9815c040 0x7fef9815c450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef90007d80 tx=0x7fef9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef8e000640 1 -- 172.21.15.19:0/482910669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90014420 con 0x7fef9815c040 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef8e000640 1 -- 172.21.15.19:0/482910669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef90019070 con 0x7fef9815c040 2024-04-25T11:28:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef8e000640 1 -- 172.21.15.19:0/482910669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90014800 con 0x7fef9815c040 2024-04-25T11:28:31.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.063+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef981fa4c0 con 0x7fef9815c040 2024-04-25T11:28:31.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.064+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef981faa80 con 0x7fef9815c040 2024-04-25T11:28:31.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.065+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef98050740 con 0x7fef9815c040 2024-04-25T11:28:31.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.066+0000 7fef8e000640 1 -- 172.21.15.19:0/482910669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fef900149a0 con 0x7fef9815c040 2024-04-25T11:28:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.067+0000 7fef8e000640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fef780744e0 0x7fef780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.067+0000 7fef8e000640 1 -- 172.21.15.19:0/482910669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fef90095a20 con 0x7fef9815c040 2024-04-25T11:28:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.067+0000 7fef95e00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fef780744e0 0x7fef780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.069+0000 7fef95e00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fef780744e0 0x7fef780769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef88034000 tx=0x7fef8802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.072+0000 7fef8e000640 1 -- 172.21.15.19:0/482910669 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef90060180 con 0x7fef9815c040 2024-04-25T11:28:31.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.324+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef981fa210 con 0x7fef9815c040 2024-04-25T11:28:31.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.325+0000 7fef8e000640 1 -- 172.21.15.19:0/482910669 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fef9005fa80 con 0x7fef9815c040 2024-04-25T11:28:31.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:31.317 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:28:31.317 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:28:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fef780744e0 msgr2=0x7fef780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fef780744e0 0x7fef780769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef88034000 tx=0x7fef8802e000 comp rx=0 tx=0).stop 2024-04-25T11:28:31.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9815c040 msgr2=0x7fef9815c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9815c040 0x7fef9815c450 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef90007d80 tx=0x7fef9000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 shutdown_connections 2024-04-25T11:28:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fef780744e0 0x7fef780769f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef98142730 0x7fef98139610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef9815c040 0x7fef9815c450 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 --2- 172.21.15.19:0/482910669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef981400a0 0x7fef98142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 >> 172.21.15.19:0/482910669 conn(0x7fef98073050 msgr2=0x7fef980cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:31.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 shutdown_connections 2024-04-25T11:28:31.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.328+0000 7fef9cc00640 1 -- 172.21.15.19:0/482910669 wait complete. 2024-04-25T11:28:31.350 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:28:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.643+0000 7fd56fe00640 1 Processor -- start 2024-04-25T11:28:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.643+0000 7fd56fe00640 1 -- start start 2024-04-25T11:28:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.645+0000 7fd56fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5700b4ff0 0x7fd5700b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.645+0000 7fd56fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5700d21c0 0x7fd5700cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.645+0000 7fd56fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5700d26f0 con 0x7fd5700cf7f0 2024-04-25T11:28:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.645+0000 7fd56fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd5700ce7d0 con 0x7fd5700b4ff0 2024-04-25T11:28:31.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.645+0000 7fd56fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd5700ceab0 con 0x7fd5700d21c0 2024-04-25T11:28:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5700b4ff0 0x7fd5700b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd5700cf7f0 0x7fd57008cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53278/0 (socket says 172.21.15.19:53278) 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56f400640 1 -- 172.21.15.19:0/2703207174 learned_addr learned my addr 172.21.15.19:0/2703207174 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56e000640 1 --2- 172.21.15.19:0/2703207174 >> v2:172.21.15.169:3300/0 conn(0x7fd5700d21c0 0x7fd5700cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56d600640 1 -- 172.21.15.19:0/2703207174 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 983860705 0 0) 0x7fd5700d26f0 con 0x7fd5700cf7f0 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56d600640 1 -- 172.21.15.19:0/2703207174 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd554003520 con 0x7fd5700cf7f0 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56ea00640 1 -- 172.21.15.19:0/2703207174 >> v2:172.21.15.169:3300/0 conn(0x7fd5700d21c0 msgr2=0x7fd5700cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56ea00640 1 --2- 172.21.15.19:0/2703207174 >> v2:172.21.15.169:3300/0 conn(0x7fd5700d21c0 0x7fd5700cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56ea00640 1 -- 172.21.15.19:0/2703207174 >> v1:172.21.15.19:6789/0 conn(0x7fd5700cf7f0 legacy=0x7fd57008cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.646+0000 7fd56ea00640 1 -- 172.21.15.19:0/2703207174 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5700ced90 con 0x7fd5700b4ff0 2024-04-25T11:28:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.648+0000 7fd56ea00640 1 --2- 172.21.15.19:0/2703207174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700b4ff0 0x7fd5700b67c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd570074030 tx=0x7fd55802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f05d97775f96484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.648+0000 7fd56d600640 1 -- 172.21.15.19:0/2703207174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558037070 con 0x7fd5700b4ff0 2024-04-25T11:28:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.648+0000 7fd56d600640 1 -- 172.21.15.19:0/2703207174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd558032440 con 0x7fd5700b4ff0 2024-04-25T11:28:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.648+0000 7fd56d600640 1 -- 172.21.15.19:0/2703207174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558032810 con 0x7fd5700b4ff0 2024-04-25T11:28:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 -- 172.21.15.19:0/2703207174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700b4ff0 msgr2=0x7fd5700b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 --2- 172.21.15.19:0/2703207174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700b4ff0 0x7fd5700b67c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd570074030 tx=0x7fd55802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 -- 172.21.15.19:0/2703207174 shutdown_connections 2024-04-25T11:28:31.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 --2- 172.21.15.19:0/2703207174 >> v2:172.21.15.169:3300/0 conn(0x7fd5700d21c0 0x7fd5700cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 --2- 172.21.15.19:0/2703207174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700b4ff0 0x7fd5700b67c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 -- 172.21.15.19:0/2703207174 >> 172.21.15.19:0/2703207174 conn(0x7fd5700b5460 msgr2=0x7fd5700b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:31.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 -- 172.21.15.19:0/2703207174 shutdown_connections 2024-04-25T11:28:31.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.649+0000 7fd56fe00640 1 -- 172.21.15.19:0/2703207174 wait complete. 2024-04-25T11:28:31.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.652+0000 7fd56fe00640 1 Processor -- start 2024-04-25T11:28:31.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.652+0000 7fd56fe00640 1 -- start start 2024-04-25T11:28:31.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.653+0000 7fd56fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5700b4ff0 0x7fd5701f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.653+0000 7fd56fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700cf7f0 0x7fd5701eb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.653+0000 7fd56fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700d21c0 0x7fd57015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.653+0000 7fd56fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5700d29d0 con 0x7fd5700d21c0 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.653+0000 7fd56fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5700cfc00 con 0x7fd5700cf7f0 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.653+0000 7fd56fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5700d2b50 con 0x7fd5700b4ff0 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700d21c0 0x7fd57015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700d21c0 0x7fd57015f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47834/0 (socket says 172.21.15.19:47834) 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 -- 172.21.15.19:0/1246635944 learned_addr learned my addr 172.21.15.19:0/1246635944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56ea00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5700b4ff0 0x7fd5701f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56e000640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700cf7f0 0x7fd5701eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 -- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5700b4ff0 msgr2=0x7fd5701f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5700b4ff0 0x7fd5701f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 -- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700cf7f0 msgr2=0x7fd5701eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700cf7f0 0x7fd5701eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.654+0000 7fd56f400640 1 -- 172.21.15.19:0/1246635944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57015e7c0 con 0x7fd5700d21c0 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.655+0000 7fd56f400640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700d21c0 0x7fd57015f860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c3c0 tx=0x7fd56800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.655+0000 7fd566a00640 1 -- 172.21.15.19:0/1246635944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568017070 con 0x7fd5700d21c0 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.655+0000 7fd566a00640 1 -- 172.21.15.19:0/1246635944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd568005ce0 con 0x7fd5700d21c0 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.655+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57015eaa0 con 0x7fd5700d21c0 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.655+0000 7fd566a00640 1 -- 172.21.15.19:0/1246635944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568007310 con 0x7fd5700d21c0 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.655+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57015f060 con 0x7fd5700d21c0 2024-04-25T11:28:31.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.656+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5701ec1b0 con 0x7fd5700d21c0 2024-04-25T11:28:31.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.658+0000 7fd566a00640 1 -- 172.21.15.19:0/1246635944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd56801a070 con 0x7fd5700d21c0 2024-04-25T11:28:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.659+0000 7fd566a00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd54c0744b0 0x7fd54c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:31.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.659+0000 7fd566a00640 1 -- 172.21.15.19:0/1246635944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fd568094590 con 0x7fd5700d21c0 2024-04-25T11:28:31.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.659+0000 7fd56ea00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd54c0744b0 0x7fd54c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:31.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.661+0000 7fd56ea00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd54c0744b0 0x7fd54c0769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd558034000 tx=0x7fd55802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:31.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.663+0000 7fd566a00640 1 -- 172.21.15.19:0/1246635944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd56805ed90 con 0x7fd5700d21c0 2024-04-25T11:28:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.913+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd570050740 con 0x7fd5700d21c0 2024-04-25T11:28:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.915+0000 7fd566a00640 1 -- 172.21.15.19:0/1246635944 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fd56805e690 con 0x7fd5700d21c0 2024-04-25T11:28:31.907 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:31.907 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/7964697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":7964697},{"type":"v1","addr":"172.21.15.169:6837","nonce":7964697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2248961566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2248961566},{"type":"v1","addr":"172.21.15.19:6834","nonce":2248961566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/4237121352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":4237121352},{"type":"v1","addr":"172.21.15.19:6835","nonce":4237121352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:27:25.102980+0000","modified":"2024-04-25T11:27:28.179340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6836/2467347557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2467347557},{"type":"v1","addr":"172.21.15.169:6836","nonce":2467347557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4297,"qdb_cluster":[4297]},"id":1}]} 2024-04-25T11:28:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T11:28:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd54c0744b0 msgr2=0x7fd54c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd54c0744b0 0x7fd54c0769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd558034000 tx=0x7fd55802e000 comp rx=0 tx=0).stop 2024-04-25T11:28:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700d21c0 msgr2=0x7fd57015f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700d21c0 0x7fd57015f860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd56800c3c0 tx=0x7fd56800c880 comp rx=0 tx=0).stop 2024-04-25T11:28:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 shutdown_connections 2024-04-25T11:28:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd54c0744b0 0x7fd54c0769c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5700cf7f0 0x7fd5701eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5700d21c0 0x7fd57015f860 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 --2- 172.21.15.19:0/1246635944 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5700b4ff0 0x7fd5701f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 >> 172.21.15.19:0/1246635944 conn(0x7fd5700b5460 msgr2=0x7fd57015cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 shutdown_connections 2024-04-25T11:28:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:31.918+0000 7fd56fe00640 1 -- 172.21.15.19:0/1246635944 wait complete. 2024-04-25T11:28:31.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.232+0000 7fdbfaa00640 1 Processor -- start 2024-04-25T11:28:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.233+0000 7fdbfaa00640 1 -- start start 2024-04-25T11:28:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.235+0000 7fdbfaa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdbf40b4ea0 0x7fdbf40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.235+0000 7fdbfaa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf40d1230 con 0x7fdbf4144370 2024-04-25T11:28:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.235+0000 7fdbfaa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf40d06b0 con 0x7fdbf406c5c0 2024-04-25T11:28:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.235+0000 7fdbfaa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf40d0990 con 0x7fdbf40b4ea0 2024-04-25T11:28:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.235+0000 7fdbf8c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdbf40b4ea0 0x7fdbf40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.235+0000 7fdbfa000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdbf4144370 0x7fdbf415e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53294/0 (socket says 172.21.15.19:53294) 2024-04-25T11:28:32.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.235+0000 7fdbfa000640 1 -- 172.21.15.19:0/1013452009 learned_addr learned my addr 172.21.15.19:0/1013452009 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:32.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.236+0000 7fdbf8c00640 1 -- 172.21.15.19:0/1013452009 >> v1:172.21.15.169:6789/0 conn(0x7fdbf406c5c0 legacy=0x7fdbf40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:32.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.236+0000 7fdbf8c00640 1 -- 172.21.15.19:0/1013452009 >> v1:172.21.15.19:6789/0 conn(0x7fdbf4144370 legacy=0x7fdbf415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.236+0000 7fdbf3e00640 1 -- 172.21.15.19:0/1013452009 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099381844 0 0) 0x7fdbf40d1230 con 0x7fdbf4144370 2024-04-25T11:28:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.236+0000 7fdbf8c00640 1 -- 172.21.15.19:0/1013452009 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf40d0c70 con 0x7fdbf40b4ea0 2024-04-25T11:28:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.237+0000 7fdbf8c00640 1 --2- 172.21.15.19:0/1013452009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf40b4ea0 0x7fdbf40cfa70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d90 tx=0x7fdbe402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1d2a8f77366d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbf3e00640 1 -- 172.21.15.19:0/1013452009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4007740 con 0x7fdbf40b4ea0 2024-04-25T11:28:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbf3e00640 1 -- 172.21.15.19:0/1013452009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe40078e0 con 0x7fdbf40b4ea0 2024-04-25T11:28:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbf3e00640 1 -- 172.21.15.19:0/1013452009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4007cb0 con 0x7fdbf40b4ea0 2024-04-25T11:28:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbfaa00640 1 -- 172.21.15.19:0/1013452009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf40b4ea0 msgr2=0x7fdbf40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/1013452009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf40b4ea0 0x7fdbf40cfa70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d90 tx=0x7fdbe402a6f0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbfaa00640 1 -- 172.21.15.19:0/1013452009 shutdown_connections 2024-04-25T11:28:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/1013452009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf40b4ea0 0x7fdbf40cfa70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.238+0000 7fdbfaa00640 1 -- 172.21.15.19:0/1013452009 >> 172.21.15.19:0/1013452009 conn(0x7fdbf40d1720 msgr2=0x7fdbf40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.239+0000 7fdbfaa00640 1 -- 172.21.15.19:0/1013452009 shutdown_connections 2024-04-25T11:28:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.239+0000 7fdbfaa00640 1 -- 172.21.15.19:0/1013452009 wait complete. 2024-04-25T11:28:32.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.241+0000 7fdbfaa00640 1 Processor -- start 2024-04-25T11:28:32.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.241+0000 7fdbfaa00640 1 -- start start 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbf406c5c0 0x7fdbf4070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbf4144370 0x7fdbf40cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf4162260 0x7fdbf4140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfaa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf40d0450 con 0x7fdbf4162260 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfaa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf40d02d0 con 0x7fdbf406c5c0 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf4147760 con 0x7fdbf4144370 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf4162260 0x7fdbf4140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf4162260 0x7fdbf4140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47860/0 (socket says 172.21.15.19:47860) 2024-04-25T11:28:32.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbfa000640 1 -- 172.21.15.19:0/73107641 learned_addr learned my addr 172.21.15.19:0/73107641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:32.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbf9600640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbf406c5c0 0x7fdbf4070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.243+0000 7fdbf8c00640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbf4144370 0x7fdbf40cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbfa000640 1 -- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbf4144370 msgr2=0x7fdbf40cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbfa000640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbf4144370 0x7fdbf40cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbfa000640 1 -- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbf406c5c0 msgr2=0x7fdbf4070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbfa000640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbf406c5c0 0x7fdbf4070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbfa000640 1 -- 172.21.15.19:0/73107641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf40cb240 con 0x7fdbf4162260 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbfa000640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf4162260 0x7fdbf4140160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c410 tx=0x7fdbec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbf1600640 1 -- 172.21.15.19:0/73107641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0074e0 con 0x7fdbf4162260 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.244+0000 7fdbf1600640 1 -- 172.21.15.19:0/73107641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec0047e0 con 0x7fdbf4162260 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.245+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf40cb520 con 0x7fdbf4162260 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.245+0000 7fdbf1600640 1 -- 172.21.15.19:0/73107641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec004bd0 con 0x7fdbf4162260 2024-04-25T11:28:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.245+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf40cbae0 con 0x7fdbf4162260 2024-04-25T11:28:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.246+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf4050740 con 0x7fdbf4162260 2024-04-25T11:28:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.247+0000 7fdbf1600640 1 -- 172.21.15.19:0/73107641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdbec011070 con 0x7fdbf4162260 2024-04-25T11:28:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.248+0000 7fdbf1600640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdbd80744b0 0x7fdbd80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.248+0000 7fdbf1600640 1 -- 172.21.15.19:0/73107641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fdbec094e90 con 0x7fdbf4162260 2024-04-25T11:28:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.249+0000 7fdbf9600640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdbd80744b0 0x7fdbd80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.250+0000 7fdbf9600640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdbd80744b0 0x7fdbd80769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000f60 tx=0x7fdbe8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.253+0000 7fdbf1600640 1 -- 172.21.15.19:0/73107641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbec05f4e0 con 0x7fdbf4162260 2024-04-25T11:28:32.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.460+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbf415f240 con 0x7fdbf4162260 2024-04-25T11:28:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.462+0000 7fdbf1600640 1 -- 172.21.15.19:0/73107641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16657 (secure 0 0 0) 0x7fdbec05ede0 con 0x7fdbf4162260 2024-04-25T11:28:32.454 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:32.455 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:05.820970+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:27:23.327873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:27:24.079949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.465+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdbd80744b0 msgr2=0x7fdbd80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.465+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdbd80744b0 0x7fdbd80769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8000f60 tx=0x7fdbe8009040 comp rx=0 tx=0).stop 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.465+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf4162260 msgr2=0x7fdbf4140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.465+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf4162260 0x7fdbf4140160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c410 tx=0x7fdbec00c8d0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 shutdown_connections 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdbd80744b0 0x7fdbd80769c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbf4162260 0x7fdbf4140160 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbf4144370 0x7fdbf40cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 --2- 172.21.15.19:0/73107641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbf406c5c0 0x7fdbf4070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 >> 172.21.15.19:0/73107641 conn(0x7fdbf40d1720 msgr2=0x7fdbf40b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 shutdown_connections 2024-04-25T11:28:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.466+0000 7fdbfaa00640 1 -- 172.21.15.19:0/73107641 wait complete. 2024-04-25T11:28:32.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:28:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.783+0000 7ff0a0c00640 1 Processor -- start 2024-04-25T11:28:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.783+0000 7ff0a0c00640 1 -- start start 2024-04-25T11:28:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.784+0000 7ff0a0c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff09c0d2cb0 0x7ff09c160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.784+0000 7ff0a0c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff09c160b90 0x7ff09c161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.785+0000 7ff0a0c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0b7940 con 0x7ff09c0d50e0 2024-04-25T11:28:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.785+0000 7ff0a0c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c161280 con 0x7ff09c160b90 2024-04-25T11:28:32.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.785+0000 7ff0a0c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c0d4600 con 0x7ff09c0d2cb0 2024-04-25T11:28:32.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.785+0000 7ff09b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff09c0d50e0 0x7ff09c0d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53310/0 (socket says 172.21.15.19:53310) 2024-04-25T11:28:32.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.785+0000 7ff09b400640 1 -- 172.21.15.19:0/1863404554 learned_addr learned my addr 172.21.15.19:0/1863404554 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.785+0000 7ff09be00640 1 --2- 172.21.15.19:0/1863404554 >> v2:172.21.15.19:3300/0 conn(0x7ff09c160b90 0x7ff09c161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.785+0000 7ff09aa00640 1 --2- 172.21.15.19:0/1863404554 >> v2:172.21.15.169:3300/0 conn(0x7ff09c0d2cb0 0x7ff09c160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.786+0000 7ff09a000640 1 -- 172.21.15.19:0/1863404554 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490654079 0 0) 0x7ff08c001110 con 0x7ff09c0d50e0 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.786+0000 7ff09a000640 1 -- 172.21.15.19:0/1863404554 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff080003520 con 0x7ff09c0d50e0 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.786+0000 7ff09be00640 1 -- 172.21.15.19:0/1863404554 >> v2:172.21.15.169:3300/0 conn(0x7ff09c0d2cb0 msgr2=0x7ff09c160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.786+0000 7ff09be00640 1 --2- 172.21.15.19:0/1863404554 >> v2:172.21.15.169:3300/0 conn(0x7ff09c0d2cb0 0x7ff09c160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.786+0000 7ff09be00640 1 -- 172.21.15.19:0/1863404554 >> v1:172.21.15.19:6789/0 conn(0x7ff09c0d50e0 legacy=0x7ff09c0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.786+0000 7ff09be00640 1 -- 172.21.15.19:0/1863404554 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0d48e0 con 0x7ff09c160b90 2024-04-25T11:28:32.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.787+0000 7ff09be00640 1 --2- 172.21.15.19:0/1863404554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c160b90 0x7ff09c161810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff09c163b80 tx=0x7ff09402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1e1cd31d7803f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.787+0000 7ff09a000640 1 -- 172.21.15.19:0/1863404554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094007740 con 0x7ff09c160b90 2024-04-25T11:28:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.787+0000 7ff09a000640 1 -- 172.21.15.19:0/1863404554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0940078e0 con 0x7ff09c160b90 2024-04-25T11:28:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.787+0000 7ff09a000640 1 -- 172.21.15.19:0/1863404554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094007cb0 con 0x7ff09c160b90 2024-04-25T11:28:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.788+0000 7ff0a0c00640 1 -- 172.21.15.19:0/1863404554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c160b90 msgr2=0x7ff09c161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.788+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/1863404554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c160b90 0x7ff09c161810 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff09c163b80 tx=0x7ff09402a680 comp rx=0 tx=0).stop 2024-04-25T11:28:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.788+0000 7ff0a0c00640 1 -- 172.21.15.19:0/1863404554 shutdown_connections 2024-04-25T11:28:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.788+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/1863404554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c160b90 0x7ff09c161810 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.788+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/1863404554 >> v2:172.21.15.169:3300/0 conn(0x7ff09c0d2cb0 0x7ff09c160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.788+0000 7ff0a0c00640 1 -- 172.21.15.19:0/1863404554 >> 172.21.15.19:0/1863404554 conn(0x7ff09c06c5c0 msgr2=0x7ff09c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.789+0000 7ff0a0c00640 1 -- 172.21.15.19:0/1863404554 shutdown_connections 2024-04-25T11:28:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.789+0000 7ff0a0c00640 1 -- 172.21.15.19:0/1863404554 wait complete. 2024-04-25T11:28:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.791+0000 7ff0a0c00640 1 Processor -- start 2024-04-25T11:28:32.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.791+0000 7ff0a0c00640 1 -- start start 2024-04-25T11:28:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff0a0c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff09c0d2cb0 0x7ff09c15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff0a0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c0d50e0 0x7ff09c1ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff0a0c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff09c160b90 0x7ff09c1ea720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff0a0c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c161560 con 0x7ff09c0d50e0 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff0a0c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c160fa0 con 0x7ff09c160b90 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff0a0c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0d54f0 con 0x7ff09c0d2cb0 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff09aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c0d50e0 0x7ff09c1ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff09aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c0d50e0 0x7ff09c1ea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47884/0 (socket says 172.21.15.19:47884) 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff09aa00640 1 -- 172.21.15.19:0/200153323 learned_addr learned my addr 172.21.15.19:0/200153323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.793+0000 7ff09b400640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff09c0d2cb0 0x7ff09c15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff09be00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff09c160b90 0x7ff09c1ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff09aa00640 1 -- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff09c0d2cb0 msgr2=0x7ff09c15c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff09aa00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff09c0d2cb0 0x7ff09c15c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff09aa00640 1 -- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff09c160b90 msgr2=0x7ff09c1ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff09aa00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff09c160b90 0x7ff09c1ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff09aa00640 1 -- 172.21.15.19:0/200153323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c1eaf80 con 0x7ff09c0d50e0 2024-04-25T11:28:32.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff09aa00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c0d50e0 0x7ff09c1ea1e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff08800d5a0 tx=0x7ff08800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff093400640 1 -- 172.21.15.19:0/200153323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088019070 con 0x7ff09c0d50e0 2024-04-25T11:28:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff093400640 1 -- 172.21.15.19:0/200153323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff088004240 con 0x7ff09c0d50e0 2024-04-25T11:28:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.794+0000 7ff093400640 1 -- 172.21.15.19:0/200153323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088003610 con 0x7ff09c0d50e0 2024-04-25T11:28:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.795+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c1eb260 con 0x7ff09c0d50e0 2024-04-25T11:28:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.795+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c1eb820 con 0x7ff09c0d50e0 2024-04-25T11:28:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.796+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c1eacf0 con 0x7ff09c0d50e0 2024-04-25T11:28:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.797+0000 7ff093400640 1 -- 172.21.15.19:0/200153323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff08801c070 con 0x7ff09c0d50e0 2024-04-25T11:28:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.798+0000 7ff093400640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff07c0744e0 0x7ff07c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.798+0000 7ff093400640 1 -- 172.21.15.19:0/200153323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7ff088094ac0 con 0x7ff09c0d50e0 2024-04-25T11:28:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.799+0000 7ff09b400640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff07c0744e0 0x7ff07c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:32.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.800+0000 7ff09b400640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff07c0744e0 0x7ff07c0769f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff08c007fd0 tx=0x7ff08c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:32.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:32.803+0000 7ff093400640 1 -- 172.21.15.19:0/200153323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff08805ebd0 con 0x7ff09c0d50e0 2024-04-25T11:28:32.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:28:32.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:28:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.049+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff09c050740 con 0x7ff09c0d50e0 2024-04-25T11:28:33.046 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:28:33.055+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:28:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.065+0000 7ff093400640 1 -- 172.21.15.19:0/200153323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7ff088062ed0 con 0x7ff09c0d50e0 2024-04-25T11:28:33.056 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff07c0744e0 msgr2=0x7ff07c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff07c0744e0 0x7ff07c0769f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff08c007fd0 tx=0x7ff08c009040 comp rx=0 tx=0).stop 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c0d50e0 msgr2=0x7ff09c1ea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c0d50e0 0x7ff09c1ea1e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff08800d5a0 tx=0x7ff08800da60 comp rx=0 tx=0).stop 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 shutdown_connections 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff07c0744e0 0x7ff07c0769f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff09c160b90 0x7ff09c1ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff09c0d50e0 0x7ff09c1ea1e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 --2- 172.21.15.19:0/200153323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff09c0d2cb0 0x7ff09c15c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.068+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 >> 172.21.15.19:0/200153323 conn(0x7ff09c06c5c0 msgr2=0x7ff09c15ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.069+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 shutdown_connections 2024-04-25T11:28:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.069+0000 7ff0a0c00640 1 -- 172.21.15.19:0/200153323 wait complete. 2024-04-25T11:28:33.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:28:33.098 INFO:tasks.ceph.mds.d.smithi169.stderr: -9005> 2024-04-25T11:27:26.843+0000 7f4183600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:28:33.260 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:28:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.370+0000 7f27e9600640 1 Processor -- start 2024-04-25T11:28:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.370+0000 7f27e9600640 1 -- start start 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.372+0000 7f27e9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f27e4144310 0x7f27e406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.373+0000 7f27e9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 0x7f27e4161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.373+0000 7f27e9600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e4144720 con 0x7f27e4161a80 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.373+0000 7f27e9600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f27e406f640 con 0x7f27e4144310 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.373+0000 7f27e9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27e41624d0 con 0x7f27e40b72b0 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.373+0000 7f27e3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f27e4161a80 0x7f27e40b6ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53314/0 (socket says 172.21.15.19:53314) 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.373+0000 7f27e3400640 1 -- 172.21.15.19:0/691390179 learned_addr learned my addr 172.21.15.19:0/691390179 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.373+0000 7f27e8c00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 0x7f27e4161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e3e00640 1 --2- 172.21.15.19:0/691390179 >> v2:172.21.15.169:3300/0 conn(0x7f27e4144310 0x7f27e406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3924743644 0 0) 0x7f27e4144720 con 0x7f27e4161a80 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27c8003520 con 0x7f27e4161a80 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4013353117 0 0) 0x7f27c8003520 con 0x7f27e4161a80 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 msgr2=0x7f27e4161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 0x7f27e4161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 >> v2:172.21.15.169:3300/0 conn(0x7f27e4144310 msgr2=0x7f27e406fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 --2- 172.21.15.19:0/691390179 >> v2:172.21.15.169:3300/0 conn(0x7f27e4144310 0x7f27e406fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e41627b0 con 0x7f27e4161a80 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f27d0001670 con 0x7f27e4161a80 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 >> v1:172.21.15.19:6789/0 conn(0x7f27e4161a80 legacy=0x7f27e40b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c80039b0 0x7f27c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 0x7f27c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27c8004d40 0x7f27c8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e406f640 con 0x7f27c80039b0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41624d0 con 0x7f27c8004d40 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.374+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27c800ba10 con 0x7f27e40b72b0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.375+0000 7f27e3e00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c80039b0 0x7f27c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.375+0000 7f27e3400640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 0x7f27c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.375+0000 7f27e3e00640 1 -- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 msgr2=0x7f27c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.375+0000 7f27e3e00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e40b72b0 0x7f27c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.375+0000 7f27e3e00640 1 -- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27c8004d40 msgr2=0x7f27c8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.375+0000 7f27e3e00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27c8004d40 0x7f27c8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.375+0000 7f27e3e00640 1 -- 172.21.15.19:0/691390179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c8003520 con 0x7f27c80039b0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.376+0000 7f27e3e00640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c80039b0 0x7f27c8003e20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b7b20 tx=0x7f27c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b8f8c9c4c8b64e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.377+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4036070 con 0x7f27c80039b0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.377+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d4031400 con 0x7f27c80039b0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.377+0000 7f27e2a00640 1 -- 172.21.15.19:0/691390179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4009690 con 0x7f27c80039b0 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.377+0000 7f27e9600640 1 -- 172.21.15.19:0/691390179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c80039b0 msgr2=0x7f27c8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.377+0000 7f27e9600640 1 --2- 172.21.15.19:0/691390179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27c80039b0 0x7f27c8003e20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b7b20 tx=0x7f27c8005830 comp rx=0 tx=0).stop 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.377+0000 7f27e8c00640 1 -- 172.21.15.19:0/691390179 reap_dead start 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.378+0000 7f27e9600640 1 -- 172.21.15.19:0/691390179 shutdown_connections 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.378+0000 7f27e9600640 1 -- 172.21.15.19:0/691390179 >> 172.21.15.19:0/691390179 conn(0x7f27e40b4260 msgr2=0x7f27e4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.378+0000 7f27e9600640 1 -- 172.21.15.19:0/691390179 shutdown_connections 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.378+0000 7f27e9600640 1 -- 172.21.15.19:0/691390179 wait complete. 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.380+0000 7f27e9600640 1 Processor -- start 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.380+0000 7f27e9600640 1 -- start start 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e413be00 0x7f27e4138d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e413a9c0 0x7f27e413add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e413b310 0x7f27e41402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e9600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41478d0 con 0x7f27e413be00 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4070010 con 0x7f27e413b310 2024-04-25T11:28:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27e4147a50 con 0x7f27e413a9c0 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e413be00 0x7f27e4138d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e413be00 0x7f27e4138d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47894/0 (socket says 172.21.15.19:47894) 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.382+0000 7f27e3e00640 1 -- 172.21.15.19:0/976173172 learned_addr learned my addr 172.21.15.19:0/976173172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3400640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e413a9c0 0x7f27e413add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3e00640 1 -- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e413a9c0 msgr2=0x7f27e413add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3e00640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e413a9c0 0x7f27e413add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3e00640 1 -- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e413b310 msgr2=0x7f27e41402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3e00640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e413b310 0x7f27e41402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3e00640 1 -- 172.21.15.19:0/976173172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e40b55f0 con 0x7f27e413be00 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3400640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e413a9c0 0x7f27e413add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27e3e00640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e413be00 0x7f27e4138d30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27d4034040 tx=0x7f27d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.383+0000 7f27dbe00640 1 -- 172.21.15.19:0/976173172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4040020 con 0x7f27e413be00 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.384+0000 7f27dbe00640 1 -- 172.21.15.19:0/976173172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d403b6a0 con 0x7f27e413be00 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.384+0000 7f27dbe00640 1 -- 172.21.15.19:0/976173172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4036040 con 0x7f27e413be00 2024-04-25T11:28:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.384+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e40b58d0 con 0x7f27e413be00 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.384+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e40b5e90 con 0x7f27e413be00 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.385+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e4076c60 con 0x7f27e413be00 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.386+0000 7f27dbe00640 1 -- 172.21.15.19:0/976173172 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f27d4043020 con 0x7f27e413be00 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.387+0000 7f27dbe00640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f27c4074500 0x7f27c4076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.387+0000 7f27dbe00640 1 -- 172.21.15.19:0/976173172 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7f27d40b5f50 con 0x7f27e413be00 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.388+0000 7f27e3400640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f27c4074500 0x7f27c4076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.389+0000 7f27e3400640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f27c4074500 0x7f27c4076a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27d0000c00 tx=0x7f27d00017a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.391+0000 7f27dbe00640 1 -- 172.21.15.19:0/976173172 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27d4080790 con 0x7f27e413be00 2024-04-25T11:28:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:33.628+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f27e4073d40 con 0x7f27e413be00 2024-04-25T11:28:34.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.068+0000 7f27dbe00640 1 -- 172.21.15.19:0/976173172 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f27d4080090 con 0x7f27e413be00 2024-04-25T11:28:34.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f27c4074500 msgr2=0x7f27c4076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f27c4074500 0x7f27c4076a10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27d0000c00 tx=0x7f27d00017a0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e413be00 msgr2=0x7f27e4138d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e413be00 0x7f27e4138d30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f27d4034040 tx=0x7f27d4004730 comp rx=0 tx=0).stop 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 shutdown_connections 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f27c4074500 0x7f27c4076a10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e413b310 0x7f27e41402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27e413a9c0 0x7f27e413add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 --2- 172.21.15.19:0/976173172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e413be00 0x7f27e4138d30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 >> 172.21.15.19:0/976173172 conn(0x7f27c80039b0 msgr2=0x7f27e4070540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 shutdown_connections 2024-04-25T11:28:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.073+0000 7f27e9600640 1 -- 172.21.15.19:0/976173172 wait complete. 2024-04-25T11:28:34.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:28:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.385+0000 7f120e800640 1 Processor -- start 2024-04-25T11:28:34.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.385+0000 7f120e800640 1 -- start start 2024-04-25T11:28:34.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120e800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12080d3d40 0x7f1208160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:34.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12080d4510 0x7f12080b7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:34.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080b7a80 0x7f120806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:34.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120e800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f12080d5590 con 0x7f12080d4510 2024-04-25T11:28:34.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120e800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f12080b7e90 con 0x7f12080d3d40 2024-04-25T11:28:34.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12080b9110 con 0x7f12080b7a80 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12080d4510 0x7f12080b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12080d4510 0x7f12080b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48388/0 (socket says 172.21.15.19:48388) 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.387+0000 7f120ca00640 1 -- 172.21.15.19:0/3186149489 learned_addr learned my addr 172.21.15.19:0/3186149489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120d400640 1 --2- 172.21.15.19:0/3186149489 >> v2:172.21.15.169:3300/0 conn(0x7f12080d3d40 0x7f1208160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120de00640 1 --2- 172.21.15.19:0/3186149489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080b7a80 0x7f120806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120ca00640 1 -- 172.21.15.19:0/3186149489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080b7a80 msgr2=0x7f120806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120ca00640 1 --2- 172.21.15.19:0/3186149489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080b7a80 0x7f120806c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120ca00640 1 -- 172.21.15.19:0/3186149489 >> v2:172.21.15.169:3300/0 conn(0x7f12080d3d40 msgr2=0x7f1208160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120ca00640 1 --2- 172.21.15.19:0/3186149489 >> v2:172.21.15.169:3300/0 conn(0x7f12080d3d40 0x7f1208160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120ca00640 1 -- 172.21.15.19:0/3186149489 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12080d4cf0 con 0x7f12080d4510 2024-04-25T11:28:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.388+0000 7f120de00640 1 --2- 172.21.15.19:0/3186149489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080b7a80 0x7f120806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.390+0000 7f120ca00640 1 --2- 172.21.15.19:0/3186149489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080d4510 0x7f12080b7540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f120806d5f0 tx=0x7f11f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a710cd3463dfe73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.390+0000 7f1207e00640 1 -- 172.21.15.19:0/3186149489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8036070 con 0x7f12080d4510 2024-04-25T11:28:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.390+0000 7f1207e00640 1 -- 172.21.15.19:0/3186149489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f8031600 con 0x7f12080d4510 2024-04-25T11:28:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.390+0000 7f1207e00640 1 -- 172.21.15.19:0/3186149489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80319d0 con 0x7f12080d4510 2024-04-25T11:28:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 -- 172.21.15.19:0/3186149489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080d4510 msgr2=0x7f12080b7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 --2- 172.21.15.19:0/3186149489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080d4510 0x7f12080b7540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f120806d5f0 tx=0x7f11f802a6a0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 -- 172.21.15.19:0/3186149489 shutdown_connections 2024-04-25T11:28:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 --2- 172.21.15.19:0/3186149489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080d4510 0x7f12080b7540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 --2- 172.21.15.19:0/3186149489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080b7a80 0x7f120806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 --2- 172.21.15.19:0/3186149489 >> v2:172.21.15.169:3300/0 conn(0x7f12080d3d40 0x7f1208160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 -- 172.21.15.19:0/3186149489 >> 172.21.15.19:0/3186149489 conn(0x7f12080d2540 msgr2=0x7f12080d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:34.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 -- 172.21.15.19:0/3186149489 shutdown_connections 2024-04-25T11:28:34.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.391+0000 7f120e800640 1 -- 172.21.15.19:0/3186149489 wait complete. 2024-04-25T11:28:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.394+0000 7f120e800640 1 Processor -- start 2024-04-25T11:28:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.394+0000 7f120e800640 1 -- start start 2024-04-25T11:28:34.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.395+0000 7f120e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080b7a80 0x7f12081532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:34.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.395+0000 7f120e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12080d3d40 0x7f12080cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:34.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.395+0000 7f120e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080d4510 0x7f12080cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.395+0000 7f120e800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12080b93b0 con 0x7f12080b7a80 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.395+0000 7f120e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12080d52b0 con 0x7f12080d3d40 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.395+0000 7f120e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12080d4920 con 0x7f12080d4510 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080b7a80 0x7f12081532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080b7a80 0x7f12081532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48398/0 (socket says 172.21.15.19:48398) 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 -- 172.21.15.19:0/2674081187 learned_addr learned my addr 172.21.15.19:0/2674081187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120de00640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080d4510 0x7f12080cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120ca00640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12080d3d40 0x7f12080cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 -- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080d4510 msgr2=0x7f12080cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080d4510 0x7f12080cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 -- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12080d3d40 msgr2=0x7f12080cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12080d3d40 0x7f12080cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.396+0000 7f120d400640 1 -- 172.21.15.19:0/2674081187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1208153800 con 0x7f12080b7a80 2024-04-25T11:28:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.397+0000 7f120d400640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080b7a80 0x7f12081532c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e870 tx=0x7f11fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.397+0000 7f1205600640 1 -- 172.21.15.19:0/2674081187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc017070 con 0x7f12080b7a80 2024-04-25T11:28:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.397+0000 7f1205600640 1 -- 172.21.15.19:0/2674081187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11fc00c940 con 0x7f12080b7a80 2024-04-25T11:28:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.397+0000 7f1205600640 1 -- 172.21.15.19:0/2674081187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00cd10 con 0x7f12080b7a80 2024-04-25T11:28:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.397+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12081f6760 con 0x7f12080b7a80 2024-04-25T11:28:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.397+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081f6d20 con 0x7f12080b7a80 2024-04-25T11:28:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.398+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1208076c60 con 0x7f12080b7a80 2024-04-25T11:28:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.401+0000 7f1205600640 1 -- 172.21.15.19:0/2674081187 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f11fc01a070 con 0x7f12080b7a80 2024-04-25T11:28:34.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.403+0000 7f1205600640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11ec0744b0 0x7f11ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.403+0000 7f1205600640 1 -- 172.21.15.19:0/2674081187 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7f11fc095170 con 0x7f12080b7a80 2024-04-25T11:28:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.403+0000 7f120ca00640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11ec0744b0 0x7f11ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.404+0000 7f120ca00640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11ec0744b0 0x7f11ec0769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11f8033000 tx=0x7f11f8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.407+0000 7f1205600640 1 -- 172.21.15.19:0/2674081187 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11fc05f8a0 con 0x7f12080b7a80 2024-04-25T11:28:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:34.611+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f12080b8a30 con 0x7f12080b7a80 2024-04-25T11:28:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.083+0000 7f1205600640 1 -- 172.21.15.19:0/2674081187 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7f11fc05f1a0 con 0x7f12080b7a80 2024-04-25T11:28:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:28:35.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.086+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11ec0744b0 msgr2=0x7f11ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.086+0000 7f120e800640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11ec0744b0 0x7f11ec0769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11f8033000 tx=0x7f11f8009000 comp rx=0 tx=0).stop 2024-04-25T11:28:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.086+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080b7a80 msgr2=0x7f12081532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.086+0000 7f120e800640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080b7a80 0x7f12081532c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e870 tx=0x7f11fc00ed30 comp rx=0 tx=0).stop 2024-04-25T11:28:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.087+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 shutdown_connections 2024-04-25T11:28:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.087+0000 7f120e800640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11ec0744b0 0x7f11ec0769c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.087+0000 7f120e800640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12080d3d40 0x7f12080cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.087+0000 7f120e800640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12080d4510 0x7f12080cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.087+0000 7f120e800640 1 --2- 172.21.15.19:0/2674081187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12080b7a80 0x7f12081532c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.087+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 >> 172.21.15.19:0/2674081187 conn(0x7f12080d2540 msgr2=0x7f120815c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.088+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 shutdown_connections 2024-04-25T11:28:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.088+0000 7f120e800640 1 -- 172.21.15.19:0/2674081187 wait complete. 2024-04-25T11:28:35.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:28:35.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.403+0000 7fc85fa00640 1 Processor -- start 2024-04-25T11:28:35.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.403+0000 7fc85fa00640 1 -- start start 2024-04-25T11:28:35.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.404+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8580b0a10 0x7fc85815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:35.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.404+0000 7fc85fa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8581615e0 0x7fc8580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:35.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.404+0000 7fc85fa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8580ce6b0 con 0x7fc85806d5d0 2024-04-25T11:28:35.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.404+0000 7fc85fa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc8580d0480 con 0x7fc8581615e0 2024-04-25T11:28:35.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.404+0000 7fc85fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc8580d0740 con 0x7fc8580b0a10 2024-04-25T11:28:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.405+0000 7fc85d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8580b0a10 0x7fc85815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.405+0000 7fc85d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8580b0a10 0x7fc85815cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:50408/0 (socket says 172.21.15.19:50408) 2024-04-25T11:28:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.405+0000 7fc85d000640 1 -- 172.21.15.19:0/1168563924 learned_addr learned my addr 172.21.15.19:0/1168563924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:35.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.405+0000 7fc857e00640 1 --2- 172.21.15.19:0/1168563924 >> v2:172.21.15.169:3300/0 conn(0x7fc8581615e0 0x7fc8580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:35.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.406+0000 7fc85d000640 1 -- 172.21.15.19:0/1168563924 >> v2:172.21.15.169:3300/0 conn(0x7fc8581615e0 msgr2=0x7fc8580b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:35.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.406+0000 7fc85d000640 1 --2- 172.21.15.19:0/1168563924 >> v2:172.21.15.169:3300/0 conn(0x7fc8581615e0 0x7fc8580b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.406+0000 7fc85d000640 1 -- 172.21.15.19:0/1168563924 >> v1:172.21.15.169:6789/0 conn(0x7fc85806d5d0 legacy=0x7fc85806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:35.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.406+0000 7fc85d000640 1 -- 172.21.15.19:0/1168563924 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8580ce990 con 0x7fc8580b0a10 2024-04-25T11:28:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.407+0000 7fc85d000640 1 --2- 172.21.15.19:0/1168563924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8580b0a10 0x7fc85815cff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc84800bb10 tx=0x7fc84802c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c32a85492e0a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.408+0000 7fc857400640 1 -- 172.21.15.19:0/1168563924 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848032790 con 0x7fc8580b0a10 2024-04-25T11:28:35.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.408+0000 7fc857400640 1 -- 172.21.15.19:0/1168563924 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848038070 con 0x7fc8580b0a10 2024-04-25T11:28:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.408+0000 7fc857400640 1 -- 172.21.15.19:0/1168563924 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848032b50 con 0x7fc8580b0a10 2024-04-25T11:28:35.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 -- 172.21.15.19:0/1168563924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8580b0a10 msgr2=0x7fc85815cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 --2- 172.21.15.19:0/1168563924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8580b0a10 0x7fc85815cff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc84800bb10 tx=0x7fc84802c890 comp rx=0 tx=0).stop 2024-04-25T11:28:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 -- 172.21.15.19:0/1168563924 shutdown_connections 2024-04-25T11:28:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 --2- 172.21.15.19:0/1168563924 >> v2:172.21.15.169:3300/0 conn(0x7fc8581615e0 0x7fc8580b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 --2- 172.21.15.19:0/1168563924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8580b0a10 0x7fc85815cff0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 -- 172.21.15.19:0/1168563924 >> 172.21.15.19:0/1168563924 conn(0x7fc8580730b0 msgr2=0x7fc8580734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 -- 172.21.15.19:0/1168563924 shutdown_connections 2024-04-25T11:28:35.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.410+0000 7fc85fa00640 1 -- 172.21.15.19:0/1168563924 wait complete. 2024-04-25T11:28:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.412+0000 7fc85fa00640 1 Processor -- start 2024-04-25T11:28:35.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.412+0000 7fc85fa00640 1 -- start start 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc85806d5d0 0x7fc85814cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580b0a10 0x7fc858157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8581615e0 0x7fc85815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85fa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8580cef50 con 0x7fc85806d5d0 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85fa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8580b5360 con 0x7fc8580b0a10 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc8581619f0 con 0x7fc8581615e0 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc85806d5d0 0x7fc85814cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc85806d5d0 0x7fc85814cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48400/0 (socket says 172.21.15.19:48400) 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 -- 172.21.15.19:0/22065549 learned_addr learned my addr 172.21.15.19:0/22065549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:35.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85da00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8581615e0 0x7fc85815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc857e00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580b0a10 0x7fc858157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 -- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8581615e0 msgr2=0x7fc85815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8581615e0 0x7fc85815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 -- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580b0a10 msgr2=0x7fc858157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580b0a10 0x7fc858157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 -- 172.21.15.19:0/22065549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8580aff30 con 0x7fc85806d5d0 2024-04-25T11:28:35.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.414+0000 7fc85d000640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc85806d5d0 0x7fc85814cee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc848000c00 tx=0x7fc848004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.415+0000 7fc854c00640 1 -- 172.21.15.19:0/22065549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848043070 con 0x7fc85806d5d0 2024-04-25T11:28:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.415+0000 7fc854c00640 1 -- 172.21.15.19:0/22065549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84803c640 con 0x7fc85806d5d0 2024-04-25T11:28:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.415+0000 7fc854c00640 1 -- 172.21.15.19:0/22065549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84803ca30 con 0x7fc85806d5d0 2024-04-25T11:28:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.415+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8580afc50 con 0x7fc85806d5d0 2024-04-25T11:28:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.415+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85814a640 con 0x7fc85806d5d0 2024-04-25T11:28:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.416+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc858050740 con 0x7fc85806d5d0 2024-04-25T11:28:35.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.417+0000 7fc854c00640 1 -- 172.21.15.19:0/22065549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc848046070 con 0x7fc85806d5d0 2024-04-25T11:28:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.419+0000 7fc854c00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8340744e0 0x7fc8340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.419+0000 7fc854c00640 1 -- 172.21.15.19:0/22065549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5386+0+0 (secure 0 0 0) 0x7fc8480b5880 con 0x7fc85806d5d0 2024-04-25T11:28:35.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.420+0000 7fc857e00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8340744e0 0x7fc8340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:35.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.421+0000 7fc857e00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8340744e0 0x7fc8340769f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc858146d10 tx=0x7fc84000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:35.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.425+0000 7fc854c00640 1 -- 172.21.15.19:0/22065549 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc848080030 con 0x7fc85806d5d0 2024-04-25T11:28:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:35.638+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc858158440 con 0x7fc85806d5d0 2024-04-25T11:28:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.086+0000 7fc854c00640 1 -- 172.21.15.19:0/22065549 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7fc84807f930 con 0x7fc85806d5d0 2024-04-25T11:28:36.078 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:28:36.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.092+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8340744e0 msgr2=0x7fc8340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.092+0000 7fc85fa00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8340744e0 0x7fc8340769f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc858146d10 tx=0x7fc84000b040 comp rx=0 tx=0).stop 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.092+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc85806d5d0 msgr2=0x7fc85814cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.092+0000 7fc85fa00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc85806d5d0 0x7fc85814cee0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc848000c00 tx=0x7fc848004730 comp rx=0 tx=0).stop 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.093+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 shutdown_connections 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.093+0000 7fc85fa00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8340744e0 0x7fc8340769f0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc858146d10 tx=0x7fc84000b040 comp rx=0 tx=0).stop 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.093+0000 7fc85fa00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8580b0a10 0x7fc858157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.093+0000 7fc85fa00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8581615e0 0x7fc85815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.093+0000 7fc85fa00640 1 --2- 172.21.15.19:0/22065549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc85806d5d0 0x7fc85814cee0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.093+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 >> 172.21.15.19:0/22065549 conn(0x7fc8580730b0 msgr2=0x7fc8580b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:36.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.093+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 shutdown_connections 2024-04-25T11:28:36.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.094+0000 7fc85fa00640 1 -- 172.21.15.19:0/22065549 wait complete. 2024-04-25T11:28:36.127 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:28:36.127 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:28:36.128 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:28:36.128 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:28:36.128 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:28:36.128 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:28:36.128 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:28:36.128 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:28:36.128 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:28:36.129 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:28:36.129 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:28:36.129 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:28:36.129 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:28:36.129 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:28:36.129 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:28:36.129 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:28:36.156 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:28:36.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:28:36.157 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:28:36.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:28:36.158 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:28:36.158 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:28:36.159 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:28:36.160 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:28:36.201 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:28:36.202 DEBUG:teuthology.parallel:result is None 2024-04-25T11:28:36.205 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:28:36.205 DEBUG:teuthology.parallel:result is None 2024-04-25T11:28:36.206 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:28:36.207 DEBUG:teuthology.parallel:result is None 2024-04-25T11:28:36.212 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:28:36.212 DEBUG:teuthology.parallel:result is None 2024-04-25T11:28:36.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:28:36.433 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:28:36.433 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:28:36.444 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:28:36.446 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:28:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.485+0000 7fb826c00640 1 Processor -- start 2024-04-25T11:28:36.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.485+0000 7fb826c00640 1 -- start start 2024-04-25T11:28:36.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb826c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb82006c5c0 0x7fb8200cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb826c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8200b4ea0 0x7fb82015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb826c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8200d1230 con 0x7fb820144370 2024-04-25T11:28:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb826c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb8200d06b0 con 0x7fb82006c5c0 2024-04-25T11:28:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb826c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb8200d0990 con 0x7fb8200b4ea0 2024-04-25T11:28:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb824e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb820144370 0x7fb8200b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42654/0 (socket says 172.21.15.19:42654) 2024-04-25T11:28:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb824e00640 1 -- 172.21.15.19:0/3798260309 learned_addr learned my addr 172.21.15.19:0/3798260309 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb825800640 1 --2- 172.21.15.19:0/3798260309 >> v2:172.21.15.169:3300/0 conn(0x7fb82006c5c0 0x7fb8200cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.487+0000 7fb826200640 1 --2- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8200b4ea0 0x7fb82015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.488+0000 7fb81fe00640 1 -- 172.21.15.19:0/3798260309 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418441383 0 0) 0x7fb8200d1230 con 0x7fb820144370 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.488+0000 7fb81fe00640 1 -- 172.21.15.19:0/3798260309 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb808003520 con 0x7fb820144370 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.488+0000 7fb825800640 1 -- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8200b4ea0 msgr2=0x7fb82015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.488+0000 7fb825800640 1 --2- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8200b4ea0 0x7fb82015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.488+0000 7fb825800640 1 -- 172.21.15.19:0/3798260309 >> v1:172.21.15.19:6789/0 conn(0x7fb820144370 legacy=0x7fb8200b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.488+0000 7fb825800640 1 -- 172.21.15.19:0/3798260309 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8200d0c70 con 0x7fb82006c5c0 2024-04-25T11:28:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.489+0000 7fb825800640 1 --2- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82006c5c0 0x7fb8200cfa70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb8140097a0 tx=0x7fb81402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b25d73696137387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb81fe00640 1 -- 172.21.15.19:0/3798260309 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814037070 con 0x7fb82006c5c0 2024-04-25T11:28:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb81fe00640 1 -- 172.21.15.19:0/3798260309 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb814032440 con 0x7fb82006c5c0 2024-04-25T11:28:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb81fe00640 1 -- 172.21.15.19:0/3798260309 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814032810 con 0x7fb82006c5c0 2024-04-25T11:28:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb826c00640 1 -- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82006c5c0 msgr2=0x7fb8200cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb826c00640 1 --2- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82006c5c0 0x7fb8200cfa70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb8140097a0 tx=0x7fb81402a180 comp rx=0 tx=0).stop 2024-04-25T11:28:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb826c00640 1 -- 172.21.15.19:0/3798260309 shutdown_connections 2024-04-25T11:28:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb826c00640 1 --2- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8200b4ea0 0x7fb82015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb826c00640 1 --2- 172.21.15.19:0/3798260309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82006c5c0 0x7fb8200cfa70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb826c00640 1 -- 172.21.15.19:0/3798260309 >> 172.21.15.19:0/3798260309 conn(0x7fb8200d1720 msgr2=0x7fb8200d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.490+0000 7fb826c00640 1 -- 172.21.15.19:0/3798260309 shutdown_connections 2024-04-25T11:28:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.491+0000 7fb826c00640 1 -- 172.21.15.19:0/3798260309 wait complete. 2024-04-25T11:28:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.493+0000 7fb826c00640 1 Processor -- start 2024-04-25T11:28:36.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.493+0000 7fb826c00640 1 -- start start 2024-04-25T11:28:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb82006c5c0 0x7fb820144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb820144370 0x7fb82013d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82013fa60 0x7fb82015ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200d0450 con 0x7fb82006c5c0 2024-04-25T11:28:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200d02d0 con 0x7fb82013fa60 2024-04-25T11:28:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb820147760 con 0x7fb820144370 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb82006c5c0 0x7fb820144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb82006c5c0 0x7fb820144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48436/0 (socket says 172.21.15.19:48436) 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 -- 172.21.15.19:0/1111771370 learned_addr learned my addr 172.21.15.19:0/1111771370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb824e00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb820144370 0x7fb82013d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826200640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82013fa60 0x7fb82015ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 -- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb820144370 msgr2=0x7fb82013d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb820144370 0x7fb82013d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 -- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82013fa60 msgr2=0x7fb82015ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82013fa60 0x7fb82015ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb825800640 1 -- 172.21.15.19:0/1111771370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82013e4c0 con 0x7fb82006c5c0 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb824e00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb820144370 0x7fb82013d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:36.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.495+0000 7fb826200640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82013fa60 0x7fb82015ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.496+0000 7fb825800640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb82006c5c0 0x7fb820144780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb814035040 tx=0x7fb8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.496+0000 7fb81d600640 1 -- 172.21.15.19:0/1111771370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814041030 con 0x7fb82006c5c0 2024-04-25T11:28:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.496+0000 7fb81d600640 1 -- 172.21.15.19:0/1111771370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81403d4e0 con 0x7fb82006c5c0 2024-04-25T11:28:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.496+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82013e9c0 con 0x7fb82006c5c0 2024-04-25T11:28:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.496+0000 7fb81d600640 1 -- 172.21.15.19:0/1111771370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814037040 con 0x7fb82006c5c0 2024-04-25T11:28:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.496+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82013ef80 con 0x7fb82006c5c0 2024-04-25T11:28:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.497+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb820050740 con 0x7fb82006c5c0 2024-04-25T11:28:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.499+0000 7fb81d600640 1 -- 172.21.15.19:0/1111771370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb814044070 con 0x7fb82006c5c0 2024-04-25T11:28:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.500+0000 7fb81d600640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8000744b0 0x7fb8000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.500+0000 7fb81d600640 1 -- 172.21.15.19:0/1111771370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7fb8140b5360 con 0x7fb82006c5c0 2024-04-25T11:28:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.500+0000 7fb824e00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8000744b0 0x7fb8000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.501+0000 7fb824e00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8000744b0 0x7fb8000769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb80c007fd0 tx=0x7fb80c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.504+0000 7fb81d600640 1 -- 172.21.15.19:0/1111771370 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb81404b070 con 0x7fb82006c5c0 2024-04-25T11:28:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:36.722+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb8200d1f40 con 0x7fb82006c5c0 2024-04-25T11:28:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.131+0000 7fb81d600640 1 -- 172.21.15.19:0/1111771370 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7fb81407fdf0 con 0x7fb82006c5c0 2024-04-25T11:28:37.122 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T11:28:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.134+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8000744b0 msgr2=0x7fb8000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8000744b0 0x7fb8000769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb80c007fd0 tx=0x7fb80c009040 comp rx=0 tx=0).stop 2024-04-25T11:28:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb82006c5c0 msgr2=0x7fb820144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb82006c5c0 0x7fb820144780 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb814035040 tx=0x7fb8140046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 shutdown_connections 2024-04-25T11:28:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8000744b0 0x7fb8000769c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb82013fa60 0x7fb82015ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb820144370 0x7fb82013d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 --2- 172.21.15.19:0/1111771370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb82006c5c0 0x7fb820144780 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb814035040 tx=0x7fb8140046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 >> 172.21.15.19:0/1111771370 conn(0x7fb8200d1720 msgr2=0x7fb8200b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 shutdown_connections 2024-04-25T11:28:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.135+0000 7fb826c00640 1 -- 172.21.15.19:0/1111771370 wait complete. 2024-04-25T11:28:37.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:28:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.454+0000 7f7c8ce00640 1 Processor -- start 2024-04-25T11:28:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.454+0000 7f7c8ce00640 1 -- start start 2024-04-25T11:28:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.456+0000 7f7c8ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.456+0000 7f7c8ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c880d0b90 0x7f7c880d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.456+0000 7f7c8ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c880d14e0 con 0x7f7c880cc7d0 2024-04-25T11:28:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.456+0000 7f7c8ce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7c881606a0 con 0x7f7c880d0b90 2024-04-25T11:28:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.456+0000 7f7c8ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c880d24a0 con 0x7f7c88160290 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c880d0b90 0x7f7c880d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c880d0b90 0x7f7c880d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48452/0 (socket says 172.21.15.19:48452) 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87e00640 1 -- 172.21.15.19:0/3311064235 learned_addr learned my addr 172.21.15.19:0/3311064235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87400640 1 --1- 172.21.15.19:0/3311064235 >> v1:172.21.15.19:6789/0 conn(0x7f7c880cc7d0 0x7f7c880b92b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42658/0 (socket says 172.21.15.19:42658) 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c7fe00640 1 --2- 172.21.15.19:0/3311064235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87e00640 1 -- 172.21.15.19:0/3311064235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 msgr2=0x7f7c880b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87e00640 1 --2- 172.21.15.19:0/3311064235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87e00640 1 -- 172.21.15.19:0/3311064235 >> v1:172.21.15.19:6789/0 conn(0x7f7c880cc7d0 legacy=0x7f7c880b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c87e00640 1 -- 172.21.15.19:0/3311064235 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c880d2780 con 0x7f7c880d0b90 2024-04-25T11:28:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.457+0000 7f7c7fe00640 1 --2- 172.21.15.19:0/3311064235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.458+0000 7f7c87e00640 1 --2- 172.21.15.19:0/3311064235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880d0b90 0x7f7c880d0fa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009e50 tx=0x7f7c8002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc80e58b92bc1f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.459+0000 7f7c86a00640 1 -- 172.21.15.19:0/3311064235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80036070 con 0x7f7c880d0b90 2024-04-25T11:28:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.459+0000 7f7c86a00640 1 -- 172.21.15.19:0/3311064235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c80031600 con 0x7f7c880d0b90 2024-04-25T11:28:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.459+0000 7f7c86a00640 1 -- 172.21.15.19:0/3311064235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800319d0 con 0x7f7c880d0b90 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 -- 172.21.15.19:0/3311064235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880d0b90 msgr2=0x7f7c880d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/3311064235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880d0b90 0x7f7c880d0fa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009e50 tx=0x7f7c8002a710 comp rx=0 tx=0).stop 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 -- 172.21.15.19:0/3311064235 shutdown_connections 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/3311064235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880d0b90 0x7f7c880d0fa0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/3311064235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 -- 172.21.15.19:0/3311064235 >> 172.21.15.19:0/3311064235 conn(0x7f7c8806c5c0 msgr2=0x7f7c8806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 -- 172.21.15.19:0/3311064235 shutdown_connections 2024-04-25T11:28:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.460+0000 7f7c8ce00640 1 -- 172.21.15.19:0/3311064235 wait complete. 2024-04-25T11:28:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.463+0000 7f7c8ce00640 1 Processor -- start 2024-04-25T11:28:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.463+0000 7f7c8ce00640 1 -- start start 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c8ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880cc7d0 0x7f7c881f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c8ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c880d0b90 0x7f7c8814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c8ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c8ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c88164160 con 0x7f7c880cc7d0 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c8ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c880d2d40 con 0x7f7c880d0b90 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c8ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c881642e0 con 0x7f7c88160290 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c87400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880cc7d0 0x7f7c881f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c87400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880cc7d0 0x7f7c881f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48456/0 (socket says 172.21.15.19:48456) 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.465+0000 7f7c87400640 1 -- 172.21.15.19:0/1206873535 learned_addr learned my addr 172.21.15.19:0/1206873535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c87400640 1 -- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 msgr2=0x7f7c880b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c87400640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c87400640 1 -- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c880d0b90 msgr2=0x7f7c8814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c87400640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c880d0b90 0x7f7c8814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c87400640 1 -- 172.21.15.19:0/1206873535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8814f5a0 con 0x7f7c880cc7d0 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c87400640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880cc7d0 0x7f7c881f7240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800c470 tx=0x7f7c7800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c7f400640 1 -- 172.21.15.19:0/1206873535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780074e0 con 0x7f7c880cc7d0 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c7f400640 1 -- 172.21.15.19:0/1206873535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c78007680 con 0x7f7c880cc7d0 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c7f400640 1 -- 172.21.15.19:0/1206873535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78004a00 con 0x7f7c880cc7d0 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.466+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c880b8380 con 0x7f7c880cc7d0 2024-04-25T11:28:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.467+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8815abc0 con 0x7f7c880cc7d0 2024-04-25T11:28:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.468+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c88050740 con 0x7f7c880cc7d0 2024-04-25T11:28:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.469+0000 7f7c7f400640 1 -- 172.21.15.19:0/1206873535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c7801a020 con 0x7f7c880cc7d0 2024-04-25T11:28:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.470+0000 7f7c7f400640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c640744b0 0x7f7c640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.470+0000 7f7c7f400640 1 -- 172.21.15.19:0/1206873535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f7c78094740 con 0x7f7c880cc7d0 2024-04-25T11:28:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.471+0000 7f7c7fe00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c640744b0 0x7f7c640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.473+0000 7f7c7fe00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c640744b0 0x7f7c640769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c880b58b0 tx=0x7f7c70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.477+0000 7f7c7f400640 1 -- 172.21.15.19:0/1206873535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c880d14e0 con 0x7f7c880cc7d0 2024-04-25T11:28:37.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.753+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7c880cc540 con 0x7f7c880cc7d0 2024-04-25T11:28:37.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.755+0000 7f7c7f400640 1 -- 172.21.15.19:0/1206873535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f7c78067030 con 0x7f7c880cc7d0 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ==", 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T11:28:37.747 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw==", 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw==", 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.748 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA==", 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:28:37.749 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ==", 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.750 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw==", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q==", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw==", 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T11:28:37.752 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw==", 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw==", 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.753 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ==", 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.754 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA==", 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T11:28:37.755 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w==", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q==", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:28:37.756 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg==", 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:28:37.757 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q==", 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:28:37.758 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNPSpmHT+MHRAAPZNLFcl/fxDgWnMeN3nAHw==", 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-mds", 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQADPipmsds3DRAAHzyyj6I28bZRrUJ3Twp6vQ==", 2024-04-25T11:28:37.759 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQADPipmO/s3DRAA9Tid7PcWTTuundlllEDPTQ==", 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-osd", 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQADPipmFBY4DRAAS98KkEDC/RZrehk+S4uZ/Q==", 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.760 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQADPipmDjY4DRAAAYtCxRfBczZrvWykHiJYLQ==", 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQADPipmVFQ4DRAArNKnu0FyMnG6aB82BZzu/w==", 2024-04-25T11:28:37.761 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQADPipmTHA4DRAALjnEUBBcZCdg77fQWJ9zUw==", 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA==", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA==", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:28:37.763 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:28:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:37.764 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:28:37.764 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:28:37.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.757+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c640744b0 msgr2=0x7f7c640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.757+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c640744b0 0x7f7c640769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c880b58b0 tx=0x7f7c70008040 comp rx=0 tx=0).stop 2024-04-25T11:28:37.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.757+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880cc7d0 msgr2=0x7f7c881f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.757+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880cc7d0 0x7f7c881f7240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800c470 tx=0x7f7c7800c930 comp rx=0 tx=0).stop 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 shutdown_connections 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c640744b0 0x7f7c640769c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c880d0b90 0x7f7c8814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c88160290 0x7f7c880b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 --2- 172.21.15.19:0/1206873535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c880cc7d0 0x7f7c881f7240 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 >> 172.21.15.19:0/1206873535 conn(0x7f7c8806c5c0 msgr2=0x7f7c880d62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 shutdown_connections 2024-04-25T11:28:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:37.758+0000 7f7c8ce00640 1 -- 172.21.15.19:0/1206873535 wait complete. 2024-04-25T11:28:37.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T11:28:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.075+0000 7fc19fa00640 1 Processor -- start 2024-04-25T11:28:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.075+0000 7fc19fa00640 1 -- start start 2024-04-25T11:28:38.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.078+0000 7fc19fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198070980 0x7fc198142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.078+0000 7fc19fa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1981431d0 con 0x7fc198142730 2024-04-25T11:28:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.078+0000 7fc19fa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1981434b0 con 0x7fc1981400a0 2024-04-25T11:28:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.078+0000 7fc19fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1980b6090 con 0x7fc198070980 2024-04-25T11:28:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.078+0000 7fc197e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc198142730 0x7fc198070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42668/0 (socket says 172.21.15.19:42668) 2024-04-25T11:28:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.078+0000 7fc197e00640 1 -- 172.21.15.19:0/3107859184 learned_addr learned my addr 172.21.15.19:0/3107859184 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.078+0000 7fc19da00640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198070980 0x7fc198142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3775810870 0 0) 0x7fc1981431d0 con 0x7fc198142730 2024-04-25T11:28:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc180003520 con 0x7fc198142730 2024-04-25T11:28:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3964121033 0 0) 0x7fc180003520 con 0x7fc198142730 2024-04-25T11:28:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198070980 msgr2=0x7fc198142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198070980 0x7fc198142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 >> v1:172.21.15.169:6789/0 conn(0x7fc1981400a0 legacy=0x7fc1980b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980b6370 con 0x7fc198142730 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fc184001670 con 0x7fc198142730 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 >> v1:172.21.15.19:6789/0 conn(0x7fc198142730 legacy=0x7fc198070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc180003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc180004800 0x7fc180004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180005270 0x7fc18000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980b6090 con 0x7fc1800039e0 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc18000bf50 con 0x7fc180004800 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.079+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc18000c150 con 0x7fc180005270 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19d000640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc180003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19da00640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180005270 0x7fc18000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19d000640 1 -- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180005270 msgr2=0x7fc18000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:38.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19d000640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180005270 0x7fc18000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc197e00640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc180004800 0x7fc180004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19d000640 1 -- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc180004800 msgr2=0x7fc180004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:38.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19d000640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc180004800 0x7fc180004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19d000640 1 -- 172.21.15.19:0/3107859184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc180003520 con 0x7fc1800039e0 2024-04-25T11:28:38.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.080+0000 7fc19da00640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc180005270 0x7fc18000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.081+0000 7fc19d000640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc180003e50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc18c001720 tx=0x7fc180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d1fe972dfb87c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.081+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c001370 con 0x7fc1800039e0 2024-04-25T11:28:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c0052d0 con 0x7fc1800039e0 2024-04-25T11:28:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc197400640 1 -- 172.21.15.19:0/3107859184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02abc0 con 0x7fc1800039e0 2024-04-25T11:28:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc19fa00640 1 -- 172.21.15.19:0/3107859184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 msgr2=0x7fc180003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc19fa00640 1 --2- 172.21.15.19:0/3107859184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc180003e50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc18c001720 tx=0x7fc180005830 comp rx=0 tx=0).stop 2024-04-25T11:28:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc19da00640 1 -- 172.21.15.19:0/3107859184 reap_dead start 2024-04-25T11:28:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc19fa00640 1 -- 172.21.15.19:0/3107859184 shutdown_connections 2024-04-25T11:28:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc19fa00640 1 -- 172.21.15.19:0/3107859184 >> 172.21.15.19:0/3107859184 conn(0x7fc198073050 msgr2=0x7fc198073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.082+0000 7fc19fa00640 1 -- 172.21.15.19:0/3107859184 shutdown_connections 2024-04-25T11:28:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.083+0000 7fc19fa00640 1 -- 172.21.15.19:0/3107859184 wait complete. 2024-04-25T11:28:38.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.085+0000 7fc19fa00640 1 Processor -- start 2024-04-25T11:28:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.085+0000 7fc19fa00640 1 -- start start 2024-04-25T11:28:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19fa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc1981f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19fa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1981404b0 0x7fc198160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19fa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198160c60 0x7fc1980cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19fa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc198070b00 con 0x7fc1800039e0 2024-04-25T11:28:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19fa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc198070980 con 0x7fc1981404b0 2024-04-25T11:28:38.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19fa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc198142b40 con 0x7fc198160c60 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc1981f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc1981f94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48470/0 (socket says 172.21.15.19:48470) 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19d000640 1 -- 172.21.15.19:0/2139640704 learned_addr learned my addr 172.21.15.19:0/2139640704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc197e00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1981404b0 0x7fc198160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.087+0000 7fc19da00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198160c60 0x7fc1980cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc19d000640 1 -- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198160c60 msgr2=0x7fc1980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc19d000640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198160c60 0x7fc1980cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc19d000640 1 -- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1981404b0 msgr2=0x7fc198160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc19d000640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1981404b0 0x7fc198160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc19d000640 1 -- 172.21.15.19:0/2139640704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc198161070 con 0x7fc1800039e0 2024-04-25T11:28:38.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc19d000640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc1981f94f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc18c008fd0 tx=0x7fc18c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:38.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc194c00640 1 -- 172.21.15.19:0/2139640704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c037040 con 0x7fc1800039e0 2024-04-25T11:28:38.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc194c00640 1 -- 172.21.15.19:0/2139640704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c005480 con 0x7fc1800039e0 2024-04-25T11:28:38.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc194c00640 1 -- 172.21.15.19:0/2139640704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c007a30 con 0x7fc1800039e0 2024-04-25T11:28:38.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.088+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc198140f70 con 0x7fc1800039e0 2024-04-25T11:28:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.089+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1980caf70 con 0x7fc1800039e0 2024-04-25T11:28:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.090+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc198050740 con 0x7fc1800039e0 2024-04-25T11:28:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.091+0000 7fc194c00640 1 -- 172.21.15.19:0/2139640704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc18c044020 con 0x7fc1800039e0 2024-04-25T11:28:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.092+0000 7fc194c00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1780744b0 0x7fc1780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.092+0000 7fc194c00640 1 -- 172.21.15.19:0/2139640704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fc18c007640 con 0x7fc1800039e0 2024-04-25T11:28:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.092+0000 7fc197e00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1780744b0 0x7fc1780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.094+0000 7fc197e00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1780744b0 0x7fc1780769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1981f7620 tx=0x7fc184009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:38.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.096+0000 7fc194c00640 1 -- 172.21.15.19:0/2139640704 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc198161070 con 0x7fc1800039e0 2024-04-25T11:28:38.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.380+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc1980b5c60 con 0x7fc1800039e0 2024-04-25T11:28:38.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.386+0000 7fc194c00640 1 -- 172.21.15.19:0/2139640704 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fc18c07f9b0 con 0x7fc1800039e0 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1780744b0 msgr2=0x7fc1780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1780744b0 0x7fc1780769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1981f7620 tx=0x7fc184009040 comp rx=0 tx=0).stop 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 msgr2=0x7fc1981f94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc1981f94f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc18c008fd0 tx=0x7fc18c001120 comp rx=0 tx=0).stop 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 shutdown_connections 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1780744b0 0x7fc1780769c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1981404b0 0x7fc198160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc198160c60 0x7fc1980cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 --2- 172.21.15.19:0/2139640704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1800039e0 0x7fc1981f94f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 >> 172.21.15.19:0/2139640704 conn(0x7fc198073050 msgr2=0x7fc1980b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:38.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 shutdown_connections 2024-04-25T11:28:38.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.390+0000 7fc19fa00640 1 -- 172.21.15.19:0/2139640704 wait complete. 2024-04-25T11:28:38.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T11:28:38.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.694+0000 7f5b3cc00640 1 Processor -- start 2024-04-25T11:28:38.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.695+0000 7f5b3cc00640 1 -- start start 2024-04-25T11:28:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.697+0000 7f5b3cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b38140dd0 0x7f5b38144010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.697+0000 7f5b3cc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b380b5980 con 0x7f5b380b5570 2024-04-25T11:28:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.697+0000 7f5b3cc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b380b5c10 con 0x7f5b38162160 2024-04-25T11:28:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.697+0000 7f5b3cc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5b38144550 con 0x7f5b38140dd0 2024-04-25T11:28:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.697+0000 7f5b37400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5b380b5570 0x7f5b38161a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42680/0 (socket says 172.21.15.19:42680) 2024-04-25T11:28:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.697+0000 7f5b37400640 1 -- 172.21.15.19:0/2677054996 learned_addr learned my addr 172.21.15.19:0/2677054996 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:38.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.697+0000 7f5b37e00640 1 --2- 172.21.15.19:0/2677054996 >> v2:172.21.15.19:3300/0 conn(0x7f5b38140dd0 0x7f5b38144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.698+0000 7f5b36a00640 1 -- 172.21.15.19:0/2677054996 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 555242126 0 0) 0x7f5b380b5980 con 0x7f5b380b5570 2024-04-25T11:28:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.698+0000 7f5b36a00640 1 -- 172.21.15.19:0/2677054996 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b1c003520 con 0x7f5b380b5570 2024-04-25T11:28:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.698+0000 7f5b37e00640 1 -- 172.21.15.19:0/2677054996 >> v1:172.21.15.169:6789/0 conn(0x7f5b38162160 legacy=0x7f5b3806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.698+0000 7f5b37e00640 1 -- 172.21.15.19:0/2677054996 >> v1:172.21.15.19:6789/0 conn(0x7f5b380b5570 legacy=0x7f5b38161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.698+0000 7f5b37e00640 1 -- 172.21.15.19:0/2677054996 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b380b5ef0 con 0x7f5b38140dd0 2024-04-25T11:28:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.699+0000 7f5b37e00640 1 --2- 172.21.15.19:0/2677054996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b38144010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009d90 tx=0x7f5b3002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d404a50baa1167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b36a00640 1 -- 172.21.15.19:0/2677054996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30007740 con 0x7f5b38140dd0 2024-04-25T11:28:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b36a00640 1 -- 172.21.15.19:0/2677054996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b300078e0 con 0x7f5b38140dd0 2024-04-25T11:28:38.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b36a00640 1 -- 172.21.15.19:0/2677054996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30007cb0 con 0x7f5b38140dd0 2024-04-25T11:28:38.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b3cc00640 1 -- 172.21.15.19:0/2677054996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 msgr2=0x7f5b38144010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/2677054996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b38144010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009d90 tx=0x7f5b3002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b3cc00640 1 -- 172.21.15.19:0/2677054996 shutdown_connections 2024-04-25T11:28:38.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/2677054996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b38144010 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.700+0000 7f5b3cc00640 1 -- 172.21.15.19:0/2677054996 >> 172.21.15.19:0/2677054996 conn(0x7f5b380d1b40 msgr2=0x7f5b380d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:38.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.701+0000 7f5b3cc00640 1 -- 172.21.15.19:0/2677054996 shutdown_connections 2024-04-25T11:28:38.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.701+0000 7f5b3cc00640 1 -- 172.21.15.19:0/2677054996 wait complete. 2024-04-25T11:28:38.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.703+0000 7f5b3cc00640 1 Processor -- start 2024-04-25T11:28:38.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.703+0000 7f5b3cc00640 1 -- start start 2024-04-25T11:28:38.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b3cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b380b5570 0x7f5b3815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b3cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b380b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b3cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b380ce9d0 0x7f5b380cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b3cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381411e0 con 0x7f5b38140dd0 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b3cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b380b64b0 con 0x7f5b380ce9d0 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b3cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b380b6790 con 0x7f5b380b5570 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b380b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b380b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48488/0 (socket says 172.21.15.19:48488) 2024-04-25T11:28:38.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 -- 172.21.15.19:0/1067656152 learned_addr learned my addr 172.21.15.19:0/1067656152 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:38.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b37400640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b380b5570 0x7f5b3815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b37e00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b380ce9d0 0x7f5b380cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 -- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b380b5570 msgr2=0x7f5b3815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b380b5570 0x7f5b3815b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 -- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b380ce9d0 msgr2=0x7f5b380cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b380ce9d0 0x7f5b380cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.705+0000 7f5b2fe00640 1 -- 172.21.15.19:0/1067656152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b380cf320 con 0x7f5b38140dd0 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.706+0000 7f5b2fe00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b380b2bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000c420 tx=0x7f5b2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.706+0000 7f5b2f400640 1 -- 172.21.15.19:0/1067656152 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200052d0 con 0x7f5b38140dd0 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.706+0000 7f5b2f400640 1 -- 172.21.15.19:0/1067656152 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b20005490 con 0x7f5b38140dd0 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.706+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b380b17c0 con 0x7f5b38140dd0 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.706+0000 7f5b2f400640 1 -- 172.21.15.19:0/1067656152 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20007260 con 0x7f5b38140dd0 2024-04-25T11:28:38.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.706+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b380b1d80 con 0x7f5b38140dd0 2024-04-25T11:28:38.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.707+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b38050740 con 0x7f5b38140dd0 2024-04-25T11:28:38.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.709+0000 7f5b2f400640 1 -- 172.21.15.19:0/1067656152 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b20007400 con 0x7f5b38140dd0 2024-04-25T11:28:38.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.710+0000 7f5b2f400640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b140744e0 0x7f5b140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:38.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.710+0000 7f5b2f400640 1 -- 172.21.15.19:0/1067656152 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f5b200943a0 con 0x7f5b38140dd0 2024-04-25T11:28:38.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.710+0000 7f5b37400640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b140744e0 0x7f5b140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:38.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.712+0000 7f5b37400640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b140744e0 0x7f5b140769f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5b280011e0 tx=0x7f5b28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:38.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.714+0000 7f5b2f400640 1 -- 172.21.15.19:0/1067656152 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b380cf320 con 0x7f5b38140dd0 2024-04-25T11:28:38.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.986+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f5b3806da60 con 0x7f5b38140dd0 2024-04-25T11:28:38.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.990+0000 7f5b2f400640 1 -- 172.21.15.19:0/1067656152 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f5b2005ede0 con 0x7f5b38140dd0 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b140744e0 msgr2=0x7f5b140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b140744e0 0x7f5b140769f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5b280011e0 tx=0x7f5b28009040 comp rx=0 tx=0).stop 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 msgr2=0x7f5b380b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b380b2bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000c420 tx=0x7f5b2000c8e0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 shutdown_connections 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b140744e0 0x7f5b140769f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b380ce9d0 0x7f5b380cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b38140dd0 0x7f5b380b2bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 --2- 172.21.15.19:0/1067656152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b380b5570 0x7f5b3815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:38.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 >> 172.21.15.19:0/1067656152 conn(0x7f5b380d1b40 msgr2=0x7f5b38160c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 shutdown_connections 2024-04-25T11:28:38.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:38.993+0000 7f5b3cc00640 1 -- 172.21.15.19:0/1067656152 wait complete. 2024-04-25T11:28:39.008 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T11:28:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.295+0000 7f9949200640 1 Processor -- start 2024-04-25T11:28:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.296+0000 7f9949200640 1 -- start start 2024-04-25T11:28:39.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.298+0000 7f9949200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 0x7f9944143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.298+0000 7f9949200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9944141210 con 0x7f9944143a90 2024-04-25T11:28:39.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.298+0000 7f9949200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9944141510 con 0x7f994406dce0 2024-04-25T11:28:39.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.298+0000 7f9949200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9944141b40 con 0x7f9944140e00 2024-04-25T11:28:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f993aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9944143a90 0x7f994406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42694/0 (socket says 172.21.15.19:42694) 2024-04-25T11:28:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f993aa00640 1 -- 172.21.15.19:0/4101803339 learned_addr learned my addr 172.21.15.19:0/4101803339 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9943400640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 0x7f9944143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3457870133 0 0) 0x7f9944141210 con 0x7f9944143a90 2024-04-25T11:28:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9928003520 con 0x7f9944143a90 2024-04-25T11:28:39.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1731184072 0 0) 0x7f9928003520 con 0x7f9944143a90 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 msgr2=0x7f9944143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9942a00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 0x7f9944143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 >> v1:172.21.15.169:6789/0 conn(0x7f994406dce0 legacy=0x7f9944141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.299+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99441440f0 con 0x7f9944143a90 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f992c001690 con 0x7f9944143a90 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 >> v1:172.21.15.19:6789/0 conn(0x7f9944143a90 legacy=0x7f994406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 0x7f99280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9928003f60 0x7f9928004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928004d40 0x7f9928005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9944141b40 con 0x7f9928003f60 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f992800bb50 con 0x7f9928004d40 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f992800bd50 con 0x7f9944140e00 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f993aa00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9928003f60 0x7f9928004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f9943400640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 0x7f99280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f993aa00640 1 -- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 msgr2=0x7f99280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f993aa00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9944140e00 0x7f99280039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f993aa00640 1 -- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928004d40 msgr2=0x7f9928005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f993aa00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9928004d40 0x7f9928005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.300+0000 7f993aa00640 1 -- 172.21.15.19:0/4101803339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9928003520 con 0x7f9928003f60 2024-04-25T11:28:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.301+0000 7f993aa00640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9928003f60 0x7f9928004800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c210 tx=0x7f9928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d2c742ece64e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.302+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c007d30 con 0x7f9928003f60 2024-04-25T11:28:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.302+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c0058d0 con 0x7f9928003f60 2024-04-25T11:28:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.302+0000 7f9942a00640 1 -- 172.21.15.19:0/4101803339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c005ca0 con 0x7f9928003f60 2024-04-25T11:28:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.302+0000 7f9949200640 1 -- 172.21.15.19:0/4101803339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9928003f60 msgr2=0x7f9928004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.302+0000 7f9949200640 1 --2- 172.21.15.19:0/4101803339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9928003f60 0x7f9928004800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f992c00c210 tx=0x7f9928005830 comp rx=0 tx=0).stop 2024-04-25T11:28:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.302+0000 7f9943e00640 1 -- 172.21.15.19:0/4101803339 reap_dead start 2024-04-25T11:28:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.303+0000 7f9949200640 1 -- 172.21.15.19:0/4101803339 shutdown_connections 2024-04-25T11:28:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.303+0000 7f9949200640 1 -- 172.21.15.19:0/4101803339 >> 172.21.15.19:0/4101803339 conn(0x7f9944072ee0 msgr2=0x7f99440732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.303+0000 7f9949200640 1 -- 172.21.15.19:0/4101803339 shutdown_connections 2024-04-25T11:28:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.303+0000 7f9949200640 1 -- 172.21.15.19:0/4101803339 wait complete. 2024-04-25T11:28:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.305+0000 7f9949200640 1 Processor -- start 2024-04-25T11:28:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.305+0000 7f9949200640 1 -- start start 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9949200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928003f60 0x7f99441f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9949200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99441389c0 0x7f994414d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9949200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99441f7c90 0x7f99441f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9949200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99441417d0 con 0x7f99441f7c90 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9949200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9944143ea0 con 0x7f99441389c0 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9949200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f99440d1490 con 0x7f9928003f60 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99441f7c90 0x7f99441f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99441f7c90 0x7f99441f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48510/0 (socket says 172.21.15.19:48510) 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 -- 172.21.15.19:0/3175042205 learned_addr learned my addr 172.21.15.19:0/3175042205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943400640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928003f60 0x7f99441f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f993aa00640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99441389c0 0x7f994414d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 -- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928003f60 msgr2=0x7f99441f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928003f60 0x7f99441f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 -- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99441389c0 msgr2=0x7f994414d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99441389c0 0x7f994414d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.307+0000 7f9943e00640 1 -- 172.21.15.19:0/3175042205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9944137920 con 0x7f99441f7c90 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.308+0000 7f9943400640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928003f60 0x7f99441f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.308+0000 7f9943e00640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99441f7c90 0x7f99441f80a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f994415e500 tx=0x7f993c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.308+0000 7f993be00640 1 -- 172.21.15.19:0/3175042205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c017070 con 0x7f99441f7c90 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.308+0000 7f993be00640 1 -- 172.21.15.19:0/3175042205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c005d40 con 0x7f99441f7c90 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.308+0000 7f993be00640 1 -- 172.21.15.19:0/3175042205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c00a880 con 0x7f99441f7c90 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.308+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9944137c00 con 0x7f99441f7c90 2024-04-25T11:28:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.308+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99441381c0 con 0x7f99441f7c90 2024-04-25T11:28:39.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.309+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944137670 con 0x7f99441f7c90 2024-04-25T11:28:39.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.311+0000 7f993be00640 1 -- 172.21.15.19:0/3175042205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f993c00aa20 con 0x7f99441f7c90 2024-04-25T11:28:39.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.312+0000 7f993be00640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f99200744e0 0x7f99200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.312+0000 7f993be00640 1 -- 172.21.15.19:0/3175042205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f993c093fe0 con 0x7f99441f7c90 2024-04-25T11:28:39.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.313+0000 7f9943400640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f99200744e0 0x7f99200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.314+0000 7f9943400640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f99200744e0 0x7f99200769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99340099e0 tx=0x7f9934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:39.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.317+0000 7f993be00640 1 -- 172.21.15.19:0/3175042205 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f993c05ea20 con 0x7f99441f7c90 2024-04-25T11:28:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.584+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9944050740 con 0x7f99441f7c90 2024-04-25T11:28:39.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.591+0000 7f993be00640 1 -- 172.21.15.19:0/3175042205 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f993c05e320 con 0x7f99441f7c90 2024-04-25T11:28:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.593+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f99200744e0 msgr2=0x7f99200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.593+0000 7f9949200640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f99200744e0 0x7f99200769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99340099e0 tx=0x7f9934008040 comp rx=0 tx=0).stop 2024-04-25T11:28:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.593+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99441f7c90 msgr2=0x7f99441f80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.593+0000 7f9949200640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99441f7c90 0x7f99441f80a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f994415e500 tx=0x7f993c00cb90 comp rx=0 tx=0).stop 2024-04-25T11:28:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 shutdown_connections 2024-04-25T11:28:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f99200744e0 0x7f99200769f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99441389c0 0x7f994414d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99441f7c90 0x7f99441f80a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 --2- 172.21.15.19:0/3175042205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9928003f60 0x7f99441f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 >> 172.21.15.19:0/3175042205 conn(0x7f9944072ee0 msgr2=0x7f99440cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 shutdown_connections 2024-04-25T11:28:39.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.594+0000 7f9949200640 1 -- 172.21.15.19:0/3175042205 wait complete. 2024-04-25T11:28:39.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T11:28:39.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.899+0000 7f4f1e600640 1 Processor -- start 2024-04-25T11:28:39.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.899+0000 7f4f1e600640 1 -- start start 2024-04-25T11:28:39.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.902+0000 7f4f1e600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f180b5570 0x7f4f18140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.902+0000 7f4f1e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f18161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.902+0000 7f4f1e600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f181623a0 con 0x7f4f18161f90 2024-04-25T11:28:39.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.902+0000 7f4f1e600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4f180b5980 con 0x7f4f180b5570 2024-04-25T11:28:39.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.902+0000 7f4f1e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f180b5c10 con 0x7f4f18144010 2024-04-25T11:28:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.903+0000 7f4f17e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f18161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.903+0000 7f4f1dc00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4f18161f90 0x7f4f1815e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:49472/0 (socket says 172.21.15.19:49472) 2024-04-25T11:28:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.903+0000 7f4f1dc00640 1 -- 172.21.15.19:0/863293604 learned_addr learned my addr 172.21.15.19:0/863293604 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:39.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.903+0000 7f4f1d200640 1 --2- 172.21.15.19:0/863293604 >> v2:172.21.15.169:3300/0 conn(0x7f4f180b5570 0x7f4f18140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.904+0000 7f4f17400640 1 -- 172.21.15.19:0/863293604 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595687241 0 0) 0x7f4f10001490 con 0x7f4f18161f90 2024-04-25T11:28:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.904+0000 7f4f17400640 1 -- 172.21.15.19:0/863293604 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4efc003520 con 0x7f4f18161f90 2024-04-25T11:28:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.904+0000 7f4f17e00640 1 -- 172.21.15.19:0/863293604 >> v2:172.21.15.169:3300/0 conn(0x7f4f180b5570 msgr2=0x7f4f18140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.904+0000 7f4f17e00640 1 --2- 172.21.15.19:0/863293604 >> v2:172.21.15.169:3300/0 conn(0x7f4f180b5570 0x7f4f18140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.904+0000 7f4f17e00640 1 -- 172.21.15.19:0/863293604 >> v1:172.21.15.169:6789/0 conn(0x7f4f18161f90 legacy=0x7f4f1815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.904+0000 7f4f17e00640 1 -- 172.21.15.19:0/863293604 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f180b5ef0 con 0x7f4f18144010 2024-04-25T11:28:39.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.904+0000 7f4f1d200640 1 --2- 172.21.15.19:0/863293604 >> v2:172.21.15.169:3300/0 conn(0x7f4f180b5570 0x7f4f18140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:28:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.905+0000 7f4f17e00640 1 --2- 172.21.15.19:0/863293604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f18161a50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4f000097a0 tx=0x7f4f0002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=836fad73a40c9040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.906+0000 7f4f17400640 1 -- 172.21.15.19:0/863293604 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00037070 con 0x7f4f18144010 2024-04-25T11:28:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.906+0000 7f4f17400640 1 -- 172.21.15.19:0/863293604 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f00032440 con 0x7f4f18144010 2024-04-25T11:28:39.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.906+0000 7f4f17400640 1 -- 172.21.15.19:0/863293604 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f000327f0 con 0x7f4f18144010 2024-04-25T11:28:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.906+0000 7f4f1e600640 1 -- 172.21.15.19:0/863293604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 msgr2=0x7f4f18161a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.906+0000 7f4f1e600640 1 --2- 172.21.15.19:0/863293604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f18161a50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4f000097a0 tx=0x7f4f0002a180 comp rx=0 tx=0).stop 2024-04-25T11:28:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.906+0000 7f4f1e600640 1 -- 172.21.15.19:0/863293604 shutdown_connections 2024-04-25T11:28:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.907+0000 7f4f1e600640 1 --2- 172.21.15.19:0/863293604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f18161a50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.907+0000 7f4f1e600640 1 --2- 172.21.15.19:0/863293604 >> v2:172.21.15.169:3300/0 conn(0x7f4f180b5570 0x7f4f18140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.907+0000 7f4f1e600640 1 -- 172.21.15.19:0/863293604 >> 172.21.15.19:0/863293604 conn(0x7f4f180d1b40 msgr2=0x7f4f180d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.907+0000 7f4f1e600640 1 -- 172.21.15.19:0/863293604 shutdown_connections 2024-04-25T11:28:39.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.907+0000 7f4f1e600640 1 -- 172.21.15.19:0/863293604 wait complete. 2024-04-25T11:28:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.909+0000 7f4f1e600640 1 Processor -- start 2024-04-25T11:28:39.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.909+0000 7f4f1e600640 1 -- start start 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1e600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f180b5570 0x7f4f18160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1e600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f181f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1e600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f18161f90 0x7f4f1815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1e600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180b6790 con 0x7f4f18161f90 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1e600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180b64b0 con 0x7f4f180b5570 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1e600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f18144420 con 0x7f4f18144010 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f18161f90 0x7f4f1815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f18161f90 0x7f4f1815a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48520/0 (socket says 172.21.15.19:48520) 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 -- 172.21.15.19:0/42807678 learned_addr learned my addr 172.21.15.19:0/42807678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1d200640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f180b5570 0x7f4f18160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f17e00640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f181f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 -- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 msgr2=0x7f4f181f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f181f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 -- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f180b5570 msgr2=0x7f4f18160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f180b5570 0x7f4f18160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.911+0000 7f4f1dc00640 1 -- 172.21.15.19:0/42807678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1815e3d0 con 0x7f4f18161f90 2024-04-25T11:28:39.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.912+0000 7f4f1d200640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f180b5570 0x7f4f18160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.912+0000 7f4f1dc00640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f18161f90 0x7f4f1815a3a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000cad0 tx=0x7f4f1000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.912+0000 7f4f14c00640 1 -- 172.21.15.19:0/42807678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10017070 con 0x7f4f18161f90 2024-04-25T11:28:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.912+0000 7f4f14c00640 1 -- 172.21.15.19:0/42807678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1000abd0 con 0x7f4f18161f90 2024-04-25T11:28:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.912+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181611e0 con 0x7f4f18161f90 2024-04-25T11:28:39.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.912+0000 7f4f14c00640 1 -- 172.21.15.19:0/42807678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10012400 con 0x7f4f18161f90 2024-04-25T11:28:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.912+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1815e670 con 0x7f4f18161f90 2024-04-25T11:28:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.913+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f180b2240 con 0x7f4f18161f90 2024-04-25T11:28:39.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.915+0000 7f4f14c00640 1 -- 172.21.15.19:0/42807678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f1001a070 con 0x7f4f18161f90 2024-04-25T11:28:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.916+0000 7f4f14c00640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ef40744e0 0x7f4ef40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.916+0000 7f4f14c00640 1 -- 172.21.15.19:0/42807678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f4f10094c80 con 0x7f4f18161f90 2024-04-25T11:28:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.916+0000 7f4f1d200640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ef40744e0 0x7f4ef40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:39.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.918+0000 7f4f1d200640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ef40744e0 0x7f4ef40769f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009a20 tx=0x7f4f08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:39.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:39.922+0000 7f4f14c00640 1 -- 172.21.15.19:0/42807678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f181623a0 con 0x7f4f18161f90 2024-04-25T11:28:40.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.201+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f4f1806e100 con 0x7f4f18161f90 2024-04-25T11:28:40.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.207+0000 7f4f14c00640 1 -- 172.21.15.19:0/42807678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f4f1005f750 con 0x7f4f18161f90 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ef40744e0 msgr2=0x7f4ef40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ef40744e0 0x7f4ef40769f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009a20 tx=0x7f4f08008040 comp rx=0 tx=0).stop 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f18161f90 msgr2=0x7f4f1815a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f18161f90 0x7f4f1815a3a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000cad0 tx=0x7f4f1000cf90 comp rx=0 tx=0).stop 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 shutdown_connections 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ef40744e0 0x7f4ef40769f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f18161f90 0x7f4f1815a3a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f18144010 0x7f4f181f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 --2- 172.21.15.19:0/42807678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f180b5570 0x7f4f18160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 >> 172.21.15.19:0/42807678 conn(0x7f4f180d1b40 msgr2=0x7f4f1813ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 shutdown_connections 2024-04-25T11:28:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.211+0000 7f4f1e600640 1 -- 172.21.15.19:0/42807678 wait complete. 2024-04-25T11:28:40.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T11:28:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.515+0000 7ff5f2a00640 1 Processor -- start 2024-04-25T11:28:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.515+0000 7ff5f2a00640 1 -- start start 2024-04-25T11:28:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.517+0000 7ff5f2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 0x7ff5ec160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.517+0000 7ff5f2a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ec0d24a0 con 0x7ff5ec0b8050 2024-04-25T11:28:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.517+0000 7ff5f2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ec0d2780 con 0x7ff5ec0d0b90 2024-04-25T11:28:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.517+0000 7ff5f2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec0d2a60 con 0x7ff5ec0cc7d0 2024-04-25T11:28:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.518+0000 7ff5f0c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff5ec0b8050 0x7ff5ec0b92b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42708/0 (socket says 172.21.15.19:42708) 2024-04-25T11:28:40.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.518+0000 7ff5f0c00640 1 -- 172.21.15.19:0/2424701112 learned_addr learned my addr 172.21.15.19:0/2424701112 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:40.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.518+0000 7ff5f1600640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 0x7ff5ec160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.518+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979215504 0 0) 0x7ff5ec0d24a0 con 0x7ff5ec0b8050 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5d4003520 con 0x7ff5ec0b8050 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1619640294 0 0) 0x7ff5d4003520 con 0x7ff5ec0b8050 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 msgr2=0x7ff5ec160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 0x7ff5ec160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 >> v1:172.21.15.169:6789/0 conn(0x7ff5ec0d0b90 legacy=0x7ff5ec0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec0d2d40 con 0x7ff5ec0b8050 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ff5d80017f0 con 0x7ff5ec0b8050 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 >> v1:172.21.15.19:6789/0 conn(0x7ff5ec0b8050 legacy=0x7ff5ec0b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 0x7ff5d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4003fb0 0x7ff5d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d4004d40 0x7ff5d40051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec0d2a60 con 0x7ff5d4004d40 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d400bc50 con 0x7ff5d4003fb0 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d400be50 con 0x7ff5ec0cc7d0 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.519+0000 7ff5f2000640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d4004d40 0x7ff5d40051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.520+0000 7ff5f1600640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 0x7ff5d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.520+0000 7ff5f2000640 1 -- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 msgr2=0x7ff5d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.520+0000 7ff5f2000640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec0cc7d0 0x7ff5d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.520+0000 7ff5f0c00640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4003fb0 0x7ff5d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.520+0000 7ff5f2000640 1 -- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4003fb0 msgr2=0x7ff5d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.520+0000 7ff5f2000640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4003fb0 0x7ff5d4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.520+0000 7ff5f2000640 1 -- 172.21.15.19:0/2424701112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d4003520 con 0x7ff5d4004d40 2024-04-25T11:28:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.521+0000 7ff5f2000640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d4004d40 0x7ff5d40051b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0d3740 tx=0x7ff5d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c053fb4f91e3a940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.521+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4001a70 con 0x7ff5d4004d40 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.521+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e402ae10 con 0x7ff5d4004d40 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.521+0000 7ff5ebe00640 1 -- 172.21.15.19:0/2424701112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4003bb0 con 0x7ff5d4004d40 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.522+0000 7ff5f2a00640 1 -- 172.21.15.19:0/2424701112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d4004d40 msgr2=0x7ff5d40051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.522+0000 7ff5f2a00640 1 --2- 172.21.15.19:0/2424701112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5d4004d40 0x7ff5d40051b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0d3740 tx=0x7ff5d4005830 comp rx=0 tx=0).stop 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.522+0000 7ff5f2000640 1 -- 172.21.15.19:0/2424701112 reap_dead start 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.522+0000 7ff5f2a00640 1 -- 172.21.15.19:0/2424701112 shutdown_connections 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.522+0000 7ff5f2a00640 1 -- 172.21.15.19:0/2424701112 >> 172.21.15.19:0/2424701112 conn(0x7ff5ec06c5c0 msgr2=0x7ff5ec06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.522+0000 7ff5f2a00640 1 -- 172.21.15.19:0/2424701112 shutdown_connections 2024-04-25T11:28:40.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.522+0000 7ff5f2a00640 1 -- 172.21.15.19:0/2424701112 wait complete. 2024-04-25T11:28:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.524+0000 7ff5f2a00640 1 Processor -- start 2024-04-25T11:28:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.525+0000 7ff5f2a00640 1 -- start start 2024-04-25T11:28:40.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.526+0000 7ff5f2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4004d40 0x7ff5ec154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.526+0000 7ff5f2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec158a80 0x7ff5ec0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.526+0000 7ff5f2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ec158e90 0x7ff5ec0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.526+0000 7ff5f2a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec160410 con 0x7ff5ec158e90 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.526+0000 7ff5f2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec160290 con 0x7ff5d4004d40 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.526+0000 7ff5f2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec160590 con 0x7ff5ec158a80 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ec158e90 0x7ff5ec0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ec158e90 0x7ff5ec0cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48544/0 (socket says 172.21.15.19:48544) 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 -- 172.21.15.19:0/4063364377 learned_addr learned my addr 172.21.15.19:0/4063364377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 -- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec158a80 msgr2=0x7ff5ec0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec158a80 0x7ff5ec0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 -- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4004d40 msgr2=0x7ff5ec154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4004d40 0x7ff5ec154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 -- 172.21.15.19:0/4063364377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec0ccd80 con 0x7ff5ec158e90 2024-04-25T11:28:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.527+0000 7ff5f2000640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ec158e90 0x7ff5ec0cefc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4008fd0 tx=0x7ff5e40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.528+0000 7ff5e9600640 1 -- 172.21.15.19:0/4063364377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4037040 con 0x7ff5ec158e90 2024-04-25T11:28:40.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.528+0000 7ff5e9600640 1 -- 172.21.15.19:0/4063364377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e402ac10 con 0x7ff5ec158e90 2024-04-25T11:28:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.528+0000 7ff5e9600640 1 -- 172.21.15.19:0/4063364377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4001a70 con 0x7ff5ec158e90 2024-04-25T11:28:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.528+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec0cd060 con 0x7ff5ec158e90 2024-04-25T11:28:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.528+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec0cd620 con 0x7ff5ec158e90 2024-04-25T11:28:40.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.529+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec162940 con 0x7ff5ec158e90 2024-04-25T11:28:40.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.530+0000 7ff5e9600640 1 -- 172.21.15.19:0/4063364377 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff5e4042020 con 0x7ff5ec158e90 2024-04-25T11:28:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.531+0000 7ff5e9600640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff5cc0744b0 0x7ff5cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.531+0000 7ff5e9600640 1 -- 172.21.15.19:0/4063364377 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7ff5e40b4570 con 0x7ff5ec158e90 2024-04-25T11:28:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.532+0000 7ff5f1600640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff5cc0744b0 0x7ff5cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:40.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.533+0000 7ff5f1600640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff5cc0744b0 0x7ff5cc0769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0d6ca0 tx=0x7ff5e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.536+0000 7ff5e9600640 1 -- 172.21.15.19:0/4063364377 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5ec162940 con 0x7ff5ec158e90 2024-04-25T11:28:40.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.805+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7ff5ec050740 con 0x7ff5ec158e90 2024-04-25T11:28:40.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.810+0000 7ff5e9600640 1 -- 172.21.15.19:0/4063364377 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7ff5e407f0c0 con 0x7ff5ec158e90 2024-04-25T11:28:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff5cc0744b0 msgr2=0x7ff5cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff5cc0744b0 0x7ff5cc0769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0d6ca0 tx=0x7ff5e0008040 comp rx=0 tx=0).stop 2024-04-25T11:28:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ec158e90 msgr2=0x7ff5ec0cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ec158e90 0x7ff5ec0cefc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4008fd0 tx=0x7ff5e40014a0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 shutdown_connections 2024-04-25T11:28:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff5cc0744b0 0x7ff5cc0769c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ec158e90 0x7ff5ec0cefc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5ec158a80 0x7ff5ec0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 --2- 172.21.15.19:0/4063364377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5d4004d40 0x7ff5ec154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:40.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 >> 172.21.15.19:0/4063364377 conn(0x7ff5ec06c5c0 msgr2=0x7ff5ec15f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:40.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 shutdown_connections 2024-04-25T11:28:40.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:40.813+0000 7ff5f2a00640 1 -- 172.21.15.19:0/4063364377 wait complete. 2024-04-25T11:28:40.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T11:28:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.116+0000 7fc2d3200640 1 Processor -- start 2024-04-25T11:28:41.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.116+0000 7fc2d3200640 1 -- start start 2024-04-25T11:28:41.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2d4140e00 0x7fc2d4143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d406dce0 0x7fc2d4141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d3200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2d4141210 con 0x7fc2d4143a90 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d3200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d4141b40 con 0x7fc2d4140e00 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d4143ea0 con 0x7fc2d406dce0 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2d4140e00 0x7fc2d4143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2d4140e00 0x7fc2d4143550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48558/0 (socket says 172.21.15.19:48558) 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.119+0000 7fc2d1e00640 1 -- 172.21.15.19:0/3823876377 learned_addr learned my addr 172.21.15.19:0/3823876377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.120+0000 7fc2d2800640 1 --2- 172.21.15.19:0/3823876377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d406dce0 0x7fc2d4141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.120+0000 7fc2d1e00640 1 -- 172.21.15.19:0/3823876377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d406dce0 msgr2=0x7fc2d4141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.120+0000 7fc2d1e00640 1 --2- 172.21.15.19:0/3823876377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d406dce0 0x7fc2d4141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.120+0000 7fc2d1e00640 1 -- 172.21.15.19:0/3823876377 >> v1:172.21.15.19:6789/0 conn(0x7fc2d4143a90 legacy=0x7fc2d406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.120+0000 7fc2d1e00640 1 -- 172.21.15.19:0/3823876377 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d4141510 con 0x7fc2d4140e00 2024-04-25T11:28:41.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.120+0000 7fc2d0a00640 1 -- 172.21.15.19:0/3823876377 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133043238 0 0) 0x7fc2d4141210 con 0x7fc2d4143a90 2024-04-25T11:28:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.121+0000 7fc2d1e00640 1 --2- 172.21.15.19:0/3823876377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d4140e00 0x7fc2d4143550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4141cc0 tx=0x7fc2c402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9dc4fead6b34bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.121+0000 7fc2d0a00640 1 -- 172.21.15.19:0/3823876377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4037070 con 0x7fc2d4140e00 2024-04-25T11:28:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.121+0000 7fc2d0a00640 1 -- 172.21.15.19:0/3823876377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c4032440 con 0x7fc2d4140e00 2024-04-25T11:28:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.121+0000 7fc2d0a00640 1 -- 172.21.15.19:0/3823876377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4032810 con 0x7fc2d4140e00 2024-04-25T11:28:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.122+0000 7fc2d3200640 1 -- 172.21.15.19:0/3823876377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d4140e00 msgr2=0x7fc2d4143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.122+0000 7fc2d3200640 1 --2- 172.21.15.19:0/3823876377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d4140e00 0x7fc2d4143550 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4141cc0 tx=0x7fc2c402a250 comp rx=0 tx=0).stop 2024-04-25T11:28:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.122+0000 7fc2d3200640 1 -- 172.21.15.19:0/3823876377 shutdown_connections 2024-04-25T11:28:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.122+0000 7fc2d3200640 1 --2- 172.21.15.19:0/3823876377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d406dce0 0x7fc2d4141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.122+0000 7fc2d3200640 1 --2- 172.21.15.19:0/3823876377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d4140e00 0x7fc2d4143550 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.122+0000 7fc2d3200640 1 -- 172.21.15.19:0/3823876377 >> 172.21.15.19:0/3823876377 conn(0x7fc2d4072ee0 msgr2=0x7fc2d40732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.123+0000 7fc2d3200640 1 -- 172.21.15.19:0/3823876377 shutdown_connections 2024-04-25T11:28:41.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.123+0000 7fc2d3200640 1 -- 172.21.15.19:0/3823876377 wait complete. 2024-04-25T11:28:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.126+0000 7fc2d3200640 1 Processor -- start 2024-04-25T11:28:41.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.126+0000 7fc2d3200640 1 -- start start 2024-04-25T11:28:41.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d4140e00 0x7fc2d413f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d4143a90 0x7fc2d4153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d413cce0 0x7fc2d413d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41417d0 con 0x7fc2d413cce0 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41443d0 con 0x7fc2d4140e00 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d40d1490 con 0x7fc2d4143a90 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d413cce0 0x7fc2d413d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d413cce0 0x7fc2d413d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48562/0 (socket says 172.21.15.19:48562) 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.128+0000 7fc2d2800640 1 -- 172.21.15.19:0/1214193277 learned_addr learned my addr 172.21.15.19:0/1214193277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d1e00640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d4140e00 0x7fc2d413f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d1400640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d4143a90 0x7fc2d4153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d2800640 1 -- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d4143a90 msgr2=0x7fc2d4153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d2800640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d4143a90 0x7fc2d4153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d2800640 1 -- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d4140e00 msgr2=0x7fc2d413f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d2800640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d4140e00 0x7fc2d413f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d2800640 1 -- 172.21.15.19:0/1214193277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d4157740 con 0x7fc2d413cce0 2024-04-25T11:28:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d1e00640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d4140e00 0x7fc2d413f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.129+0000 7fc2d2800640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d413cce0 0x7fc2d413d0f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc007d80 tx=0x7fc2cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:41.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.130+0000 7fc2ca000640 1 -- 172.21.15.19:0/1214193277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc014420 con 0x7fc2d413cce0 2024-04-25T11:28:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.130+0000 7fc2ca000640 1 -- 172.21.15.19:0/1214193277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2cc019070 con 0x7fc2d413cce0 2024-04-25T11:28:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.130+0000 7fc2ca000640 1 -- 172.21.15.19:0/1214193277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc014800 con 0x7fc2d413cce0 2024-04-25T11:28:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.130+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d4157a20 con 0x7fc2d413cce0 2024-04-25T11:28:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.130+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d4157fe0 con 0x7fc2d413cce0 2024-04-25T11:28:41.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.131+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d41574b0 con 0x7fc2d413cce0 2024-04-25T11:28:41.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.132+0000 7fc2ca000640 1 -- 172.21.15.19:0/1214193277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc2cc0149a0 con 0x7fc2d413cce0 2024-04-25T11:28:41.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.133+0000 7fc2ca000640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2b00744b0 0x7fc2b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.133+0000 7fc2ca000640 1 -- 172.21.15.19:0/1214193277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fc2cc0956b0 con 0x7fc2d413cce0 2024-04-25T11:28:41.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.134+0000 7fc2d1e00640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2b00744b0 0x7fc2b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.135+0000 7fc2d1e00640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2b00744b0 0x7fc2b00769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4034000 tx=0x7fc2c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:41.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.138+0000 7fc2ca000640 1 -- 172.21.15.19:0/1214193277 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2cc060220 con 0x7fc2d413cce0 2024-04-25T11:28:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.408+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fc2d41581e0 con 0x7fc2d413cce0 2024-04-25T11:28:41.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.415+0000 7fc2ca000640 1 -- 172.21.15.19:0/1214193277 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fc2d41581e0 con 0x7fc2d413cce0 2024-04-25T11:28:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2b00744b0 msgr2=0x7fc2b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2b00744b0 0x7fc2b00769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4034000 tx=0x7fc2c402e000 comp rx=0 tx=0).stop 2024-04-25T11:28:41.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d413cce0 msgr2=0x7fc2d413d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d413cce0 0x7fc2d413d0f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc007d80 tx=0x7fc2cc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 shutdown_connections 2024-04-25T11:28:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2b00744b0 0x7fc2b00769c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d413cce0 0x7fc2d413d0f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2d4143a90 0x7fc2d4153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 --2- 172.21.15.19:0/1214193277 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2d4140e00 0x7fc2d413f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 >> 172.21.15.19:0/1214193277 conn(0x7fc2d4072ee0 msgr2=0x7fc2d406fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 shutdown_connections 2024-04-25T11:28:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.418+0000 7fc2d3200640 1 -- 172.21.15.19:0/1214193277 wait complete. 2024-04-25T11:28:41.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:41.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.717+0000 7f06bbe00640 1 Processor -- start 2024-04-25T11:28:41.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.717+0000 7f06bbe00640 1 -- start start 2024-04-25T11:28:41.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.719+0000 7f06bbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06bc0b4ea0 0x7f06bc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06bbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06bc0d1230 con 0x7f06bc06c5c0 2024-04-25T11:28:41.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06bbe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06bc0d06b0 con 0x7f06bc144370 2024-04-25T11:28:41.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06bbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f06bc0d0990 con 0x7f06bc0b4ea0 2024-04-25T11:28:41.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06baa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f06bc06c5c0 0x7f06bc0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42726/0 (socket says 172.21.15.19:42726) 2024-04-25T11:28:41.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06baa00640 1 -- 172.21.15.19:0/3150004812 learned_addr learned my addr 172.21.15.19:0/3150004812 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642876015 0 0) 0x7f06bc0d1230 con 0x7f06bc06c5c0 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06a0003520 con 0x7f06bc06c5c0 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.720+0000 7f06ba000640 1 --2- 172.21.15.19:0/3150004812 >> v2:172.21.15.169:3300/0 conn(0x7f06bc0b4ea0 0x7f06bc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1582330205 0 0) 0x7f06a0003520 con 0x7f06bc06c5c0 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 >> v2:172.21.15.169:3300/0 conn(0x7f06bc0b4ea0 msgr2=0x7f06bc0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 --2- 172.21.15.19:0/3150004812 >> v2:172.21.15.169:3300/0 conn(0x7f06bc0b4ea0 0x7f06bc0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 >> v1:172.21.15.169:6789/0 conn(0x7f06bc144370 legacy=0x7f06bc15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc0d0c70 con 0x7f06bc06c5c0 2024-04-25T11:28:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f06ac0019f0 con 0x7f06bc06c5c0 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 >> v1:172.21.15.19:6789/0 conn(0x7f06bc06c5c0 legacy=0x7f06bc0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a0003a00 0x7f06a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0004800 0x7f06a0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a0005350 0x7f06a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc0d0990 con 0x7f06a0003a00 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f06a000c250 con 0x7f06a0005350 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06a000c450 con 0x7f06a0004800 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.721+0000 7f06baa00640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a0003a00 0x7f06a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.722+0000 7f06bb400640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a0005350 0x7f06a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.722+0000 7f06baa00640 1 -- 172.21.15.19:0/3150004812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0004800 msgr2=0x7f06a0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.722+0000 7f06baa00640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0004800 0x7f06a0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.722+0000 7f06baa00640 1 -- 172.21.15.19:0/3150004812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a0005350 msgr2=0x7f06a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.722+0000 7f06baa00640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06a0005350 0x7f06a000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.722+0000 7f06baa00640 1 -- 172.21.15.19:0/3150004812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a0003520 con 0x7f06a0003a00 2024-04-25T11:28:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.722+0000 7f06bb400640 1 -- 172.21.15.19:0/3150004812 reap_dead start 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.723+0000 7f06baa00640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a0003a00 0x7f06a0003e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00a790 tx=0x7f06a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d00a5e8d71b3de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.723+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00a3b0 con 0x7f06a0003a00 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.723+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac02f070 con 0x7f06a0003a00 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.723+0000 7f06b9600640 1 -- 172.21.15.19:0/3150004812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac02bc40 con 0x7f06a0003a00 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.724+0000 7f06bbe00640 1 -- 172.21.15.19:0/3150004812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a0003a00 msgr2=0x7f06a0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.724+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a0003a00 0x7f06a0003e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00a790 tx=0x7f06a0005830 comp rx=0 tx=0).stop 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.724+0000 7f06bbe00640 1 -- 172.21.15.19:0/3150004812 shutdown_connections 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.724+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3150004812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06a0003a00 0x7f06a0003e70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.724+0000 7f06bbe00640 1 -- 172.21.15.19:0/3150004812 >> 172.21.15.19:0/3150004812 conn(0x7f06bc0d1720 msgr2=0x7f06bc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.724+0000 7f06bbe00640 1 -- 172.21.15.19:0/3150004812 shutdown_connections 2024-04-25T11:28:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.724+0000 7f06bbe00640 1 -- 172.21.15.19:0/3150004812 wait complete. 2024-04-25T11:28:41.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.727+0000 7f06bbe00640 1 Processor -- start 2024-04-25T11:28:41.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.727+0000 7f06bbe00640 1 -- start start 2024-04-25T11:28:41.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.728+0000 7f06bbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0003a00 0x7f06bc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.728+0000 7f06bbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc070660 0x7f06bc070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06bbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06bc161ae0 0x7f06bc161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06bbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc0d0450 con 0x7f06bc070660 2024-04-25T11:28:41.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06bbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc0d02d0 con 0x7f06bc161ae0 2024-04-25T11:28:41.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06bbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06bc147760 con 0x7f06a0003a00 2024-04-25T11:28:41.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06ba000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc070660 0x7f06bc070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06ba000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc070660 0x7f06bc070a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48580/0 (socket says 172.21.15.19:48580) 2024-04-25T11:28:41.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06ba000640 1 -- 172.21.15.19:0/3908208081 learned_addr learned my addr 172.21.15.19:0/3908208081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06bb400640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06bc161ae0 0x7f06bc161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06baa00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0003a00 0x7f06bc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06ba000640 1 -- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0003a00 msgr2=0x7f06bc15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06ba000640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0003a00 0x7f06bc15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06ba000640 1 -- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06bc161ae0 msgr2=0x7f06bc161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.729+0000 7f06ba000640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06bc161ae0 0x7f06bc161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.730+0000 7f06ba000640 1 -- 172.21.15.19:0/3908208081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc162430 con 0x7f06bc070660 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.730+0000 7f06ba000640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc070660 0x7f06bc070a70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f06a800d560 tx=0x7f06a800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.730+0000 7f06b2a00640 1 -- 172.21.15.19:0/3908208081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a80125f0 con 0x7f06bc070660 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.730+0000 7f06b2a00640 1 -- 172.21.15.19:0/3908208081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a8019070 con 0x7f06bc070660 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.730+0000 7f06b2a00640 1 -- 172.21.15.19:0/3908208081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a80129d0 con 0x7f06bc070660 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.730+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06bc06f220 con 0x7f06bc070660 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.730+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06bc06f7e0 con 0x7f06bc070660 2024-04-25T11:28:41.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.732+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06bc073c70 con 0x7f06bc070660 2024-04-25T11:28:41.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.733+0000 7f06b2a00640 1 -- 172.21.15.19:0/3908208081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f06a8012b70 con 0x7f06bc070660 2024-04-25T11:28:41.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.734+0000 7f06b2a00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f069c0744b0 0x7f069c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:41.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.734+0000 7f06b2a00640 1 -- 172.21.15.19:0/3908208081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f06a8095350 con 0x7f06bc070660 2024-04-25T11:28:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.735+0000 7f06baa00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f069c0744b0 0x7f069c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:41.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.736+0000 7f06baa00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f069c0744b0 0x7f069c0769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f06ac033000 tx=0x7f06ac009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.739+0000 7f06b2a00640 1 -- 172.21.15.19:0/3908208081 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06bc073c70 con 0x7f06bc070660 2024-04-25T11:28:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.950+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06bc0cdce0 con 0x7f06bc070660 2024-04-25T11:28:41.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.952+0000 7f06b2a00640 1 -- 172.21.15.19:0/3908208081 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+12891 (secure 0 0 0) 0x7f06bc162430 con 0x7f06bc070660 2024-04-25T11:28:41.943 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:41.943 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:37.149574+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.954+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f069c0744b0 msgr2=0x7f069c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.954+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f069c0744b0 0x7f069c0769c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f06ac033000 tx=0x7f06ac009030 comp rx=0 tx=0).stop 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.954+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc070660 msgr2=0x7f06bc070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.954+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc070660 0x7f06bc070a70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f06a800d560 tx=0x7f06a800da20 comp rx=0 tx=0).stop 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 shutdown_connections 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f069c0744b0 0x7f069c0769c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06bc161ae0 0x7f06bc161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc070660 0x7f06bc070a70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 --2- 172.21.15.19:0/3908208081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06a0003a00 0x7f06bc15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 >> 172.21.15.19:0/3908208081 conn(0x7f06bc0d1720 msgr2=0x7f06bc0b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 shutdown_connections 2024-04-25T11:28:41.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:41.955+0000 7f06bbe00640 1 -- 172.21.15.19:0/3908208081 wait complete. 2024-04-25T11:28:41.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:41.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.273+0000 7fd88a800640 1 Processor -- start 2024-04-25T11:28:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.273+0000 7fd88a800640 1 -- start start 2024-04-25T11:28:42.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.275+0000 7fd88a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88415de90 0x7fd8841606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.275+0000 7fd88a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd884163c90 0x7fd88406f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.275+0000 7fd88a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd88406d5d0 0x7fd88406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.275+0000 7fd88a800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd88406df20 con 0x7fd884163c90 2024-04-25T11:28:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.275+0000 7fd88a800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd8840b0090 con 0x7fd88406d5d0 2024-04-25T11:28:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.275+0000 7fd88a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd884160c80 con 0x7fd88415de90 2024-04-25T11:28:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.276+0000 7fd888a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd884163c90 0x7fd88406f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.276+0000 7fd888a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd884163c90 0x7fd88406f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48590/0 (socket says 172.21.15.19:48590) 2024-04-25T11:28:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.276+0000 7fd888a00640 1 -- 172.21.15.19:0/731569418 learned_addr learned my addr 172.21.15.19:0/731569418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:42.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.276+0000 7fd889e00640 1 --2- 172.21.15.19:0/731569418 >> v2:172.21.15.169:3300/0 conn(0x7fd88406d5d0 0x7fd88406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.276+0000 7fd889400640 1 --2- 172.21.15.19:0/731569418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88415de90 0x7fd8841606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.277+0000 7fd888a00640 1 -- 172.21.15.19:0/731569418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88415de90 msgr2=0x7fd8841606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:42.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.277+0000 7fd888a00640 1 --2- 172.21.15.19:0/731569418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88415de90 0x7fd8841606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.277+0000 7fd888a00640 1 -- 172.21.15.19:0/731569418 >> v2:172.21.15.169:3300/0 conn(0x7fd88406d5d0 msgr2=0x7fd88406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:42.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.277+0000 7fd888a00640 1 --2- 172.21.15.19:0/731569418 >> v2:172.21.15.169:3300/0 conn(0x7fd88406d5d0 0x7fd88406d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.277+0000 7fd888a00640 1 -- 172.21.15.19:0/731569418 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd884160f60 con 0x7fd884163c90 2024-04-25T11:28:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.278+0000 7fd888a00640 1 --2- 172.21.15.19:0/731569418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd884163c90 0x7fd88406f0c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd884164d90 tx=0x7fd87002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07114249e01e659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.278+0000 7fd883e00640 1 -- 172.21.15.19:0/731569418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870037070 con 0x7fd884163c90 2024-04-25T11:28:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.278+0000 7fd883e00640 1 -- 172.21.15.19:0/731569418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd870032440 con 0x7fd884163c90 2024-04-25T11:28:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.278+0000 7fd883e00640 1 -- 172.21.15.19:0/731569418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870032810 con 0x7fd884163c90 2024-04-25T11:28:42.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 -- 172.21.15.19:0/731569418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd884163c90 msgr2=0x7fd88406f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 --2- 172.21.15.19:0/731569418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd884163c90 0x7fd88406f0c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd884164d90 tx=0x7fd87002a140 comp rx=0 tx=0).stop 2024-04-25T11:28:42.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 -- 172.21.15.19:0/731569418 shutdown_connections 2024-04-25T11:28:42.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 --2- 172.21.15.19:0/731569418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd884163c90 0x7fd88406f0c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 --2- 172.21.15.19:0/731569418 >> v2:172.21.15.169:3300/0 conn(0x7fd88406d5d0 0x7fd88406d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 --2- 172.21.15.19:0/731569418 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88415de90 0x7fd8841606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 -- 172.21.15.19:0/731569418 >> 172.21.15.19:0/731569418 conn(0x7fd884072f40 msgr2=0x7fd884073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 -- 172.21.15.19:0/731569418 shutdown_connections 2024-04-25T11:28:42.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.279+0000 7fd88a800640 1 -- 172.21.15.19:0/731569418 wait complete. 2024-04-25T11:28:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.282+0000 7fd88a800640 1 Processor -- start 2024-04-25T11:28:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.282+0000 7fd88a800640 1 -- start start 2024-04-25T11:28:42.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.283+0000 7fd88a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88406d5d0 0x7fd88414be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.283+0000 7fd88a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88415de90 0x7fd88414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.283+0000 7fd88a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd884163c90 0x7fd88415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.283+0000 7fd88a800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd88406f600 con 0x7fd88415de90 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.283+0000 7fd88a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd884161520 con 0x7fd884163c90 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.283+0000 7fd88a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd88406f780 con 0x7fd88406d5d0 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88415de90 0x7fd88414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88415de90 0x7fd88414c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48606/0 (socket says 172.21.15.19:48606) 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 -- 172.21.15.19:0/563265655 learned_addr learned my addr 172.21.15.19:0/563265655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 -- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88406d5d0 msgr2=0x7fd88414be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88406d5d0 0x7fd88414be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 -- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd884163c90 msgr2=0x7fd88415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd884163c90 0x7fd88415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.284+0000 7fd888a00640 1 -- 172.21.15.19:0/563265655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88414d1e0 con 0x7fd88415de90 2024-04-25T11:28:42.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.285+0000 7fd888a00640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88415de90 0x7fd88414c520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd87002a180 tx=0x7fd8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.285+0000 7fd881600640 1 -- 172.21.15.19:0/563265655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870041020 con 0x7fd88415de90 2024-04-25T11:28:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.285+0000 7fd881600640 1 -- 172.21.15.19:0/563265655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8700306c0 con 0x7fd88415de90 2024-04-25T11:28:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.285+0000 7fd881600640 1 -- 172.21.15.19:0/563265655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870037040 con 0x7fd88415de90 2024-04-25T11:28:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.285+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88414d4c0 con 0x7fd88415de90 2024-04-25T11:28:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.285+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88414da80 con 0x7fd88415de90 2024-04-25T11:28:42.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.286+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88414cf50 con 0x7fd88415de90 2024-04-25T11:28:42.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.288+0000 7fd881600640 1 -- 172.21.15.19:0/563265655 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd870044020 con 0x7fd88415de90 2024-04-25T11:28:42.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.289+0000 7fd881600640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8640744b0 0x7fd8640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.289+0000 7fd881600640 1 -- 172.21.15.19:0/563265655 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fd8700b5790 con 0x7fd88415de90 2024-04-25T11:28:42.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.289+0000 7fd889400640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8640744b0 0x7fd8640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.291+0000 7fd889400640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8640744b0 0x7fd8640769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd878006fd0 tx=0x7fd878008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:42.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.294+0000 7fd881600640 1 -- 172.21.15.19:0/563265655 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd87004b030 con 0x7fd88415de90 2024-04-25T11:28:42.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.496+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd88415fa40 con 0x7fd88415de90 2024-04-25T11:28:42.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.497+0000 7fd881600640 1 -- 172.21.15.19:0/563265655 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7fd88415fa40 con 0x7fd88415de90 2024-04-25T11:28:42.488 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:42.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.499+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8640744b0 msgr2=0x7fd8640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.499+0000 7fd88a800640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8640744b0 0x7fd8640769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd878006fd0 tx=0x7fd878008040 comp rx=0 tx=0).stop 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88415de90 msgr2=0x7fd88414c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88415de90 0x7fd88414c520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd87002a180 tx=0x7fd8700046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 shutdown_connections 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8640744b0 0x7fd8640769c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd884163c90 0x7fd88415e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd88415de90 0x7fd88414c520 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 --2- 172.21.15.19:0/563265655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd88406d5d0 0x7fd88414be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 >> 172.21.15.19:0/563265655 conn(0x7fd884072f40 msgr2=0x7fd88415d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 shutdown_connections 2024-04-25T11:28:42.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.500+0000 7fd88a800640 1 -- 172.21.15.19:0/563265655 wait complete. 2024-04-25T11:28:42.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:28:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.806+0000 7f184e000640 1 Processor -- start 2024-04-25T11:28:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.806+0000 7f184e000640 1 -- start start 2024-04-25T11:28:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f184813fe10 0x7f184806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1848142e40 0x7f18480d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184e000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184806df20 con 0x7f184806db10 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184e000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1848143250 con 0x7f184813fe10 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f184815d5e0 con 0x7f1848142e40 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f184813fe10 0x7f184806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f184813fe10 0x7f184806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48620/0 (socket says 172.21.15.19:48620) 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f184cc00640 1 -- 172.21.15.19:0/2533041041 learned_addr learned my addr 172.21.15.19:0/2533041041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.808+0000 7f183fe00640 1 --1- 172.21.15.19:0/2533041041 >> v1:172.21.15.19:6789/0 conn(0x7f184806db10 0x7f1848142730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42728/0 (socket says 172.21.15.19:42728) 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.809+0000 7f184d600640 1 --2- 172.21.15.19:0/2533041041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1848142e40 0x7f18480d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.809+0000 7f184cc00640 1 -- 172.21.15.19:0/2533041041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1848142e40 msgr2=0x7f18480d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.809+0000 7f184cc00640 1 --2- 172.21.15.19:0/2533041041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1848142e40 0x7f18480d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.809+0000 7f184cc00640 1 -- 172.21.15.19:0/2533041041 >> v1:172.21.15.19:6789/0 conn(0x7f184806db10 legacy=0x7f1848142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.809+0000 7f184cc00640 1 -- 172.21.15.19:0/2533041041 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184815d8c0 con 0x7f184813fe10 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.809+0000 7f1847e00640 1 -- 172.21.15.19:0/2533041041 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164220381 0 0) 0x7f1830001110 con 0x7f184806db10 2024-04-25T11:28:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.809+0000 7f184d600640 1 --2- 172.21.15.19:0/2533041041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1848142e40 0x7f18480d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.810+0000 7f184cc00640 1 --2- 172.21.15.19:0/2533041041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184813fe10 0x7f184806d5d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1838009870 tx=0x7f183802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9150872036a361d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.810+0000 7f1847e00640 1 -- 172.21.15.19:0/2533041041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838037070 con 0x7f184813fe10 2024-04-25T11:28:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.810+0000 7f1847e00640 1 -- 172.21.15.19:0/2533041041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1838032440 con 0x7f184813fe10 2024-04-25T11:28:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.810+0000 7f1847e00640 1 -- 172.21.15.19:0/2533041041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838032810 con 0x7f184813fe10 2024-04-25T11:28:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 -- 172.21.15.19:0/2533041041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184813fe10 msgr2=0x7f184806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 --2- 172.21.15.19:0/2533041041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184813fe10 0x7f184806d5d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1838009870 tx=0x7f183802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 -- 172.21.15.19:0/2533041041 shutdown_connections 2024-04-25T11:28:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 --2- 172.21.15.19:0/2533041041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1848142e40 0x7f18480d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 --2- 172.21.15.19:0/2533041041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184813fe10 0x7f184806d5d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 -- 172.21.15.19:0/2533041041 >> 172.21.15.19:0/2533041041 conn(0x7f1848072ee0 msgr2=0x7f18480732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 -- 172.21.15.19:0/2533041041 shutdown_connections 2024-04-25T11:28:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.811+0000 7f184e000640 1 -- 172.21.15.19:0/2533041041 wait complete. 2024-04-25T11:28:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.813+0000 7f184e000640 1 Processor -- start 2024-04-25T11:28:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.813+0000 7f184e000640 1 -- start start 2024-04-25T11:28:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184806db10 0x7f18480ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f184813fe10 0x7f1848159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1848142e40 0x7f184815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184e000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f184815d2d0 con 0x7f184806db10 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f184815de80 con 0x7f1848142e40 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f18480d0570 con 0x7f184813fe10 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184806db10 0x7f18480ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184806db10 0x7f18480ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48624/0 (socket says 172.21.15.19:48624) 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.815+0000 7f184cc00640 1 -- 172.21.15.19:0/2973009525 learned_addr learned my addr 172.21.15.19:0/2973009525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f184cc00640 1 -- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f184813fe10 msgr2=0x7f1848159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f184cc00640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f184813fe10 0x7f1848159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f184cc00640 1 -- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1848142e40 msgr2=0x7f184815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f184cc00640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1848142e40 0x7f184815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:42.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f184cc00640 1 -- 172.21.15.19:0/2973009525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1848158720 con 0x7f184806db10 2024-04-25T11:28:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f184cc00640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184806db10 0x7f18480ce0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18380090b0 tx=0x7f18380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f1845600640 1 -- 172.21.15.19:0/2973009525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838037040 con 0x7f184806db10 2024-04-25T11:28:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f1845600640 1 -- 172.21.15.19:0/2973009525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1838004230 con 0x7f184806db10 2024-04-25T11:28:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.816+0000 7f1845600640 1 -- 172.21.15.19:0/2973009525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838003610 con 0x7f184806db10 2024-04-25T11:28:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.817+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481444d0 con 0x7f184806db10 2024-04-25T11:28:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.817+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1848146030 con 0x7f184806db10 2024-04-25T11:28:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.818+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1848050740 con 0x7f184806db10 2024-04-25T11:28:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.819+0000 7f1845600640 1 -- 172.21.15.19:0/2973009525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f183802ad10 con 0x7f184806db10 2024-04-25T11:28:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.820+0000 7f1845600640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f18240744b0 0x7f18240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:42.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.820+0000 7f1845600640 1 -- 172.21.15.19:0/2973009525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f18380b5c80 con 0x7f184806db10 2024-04-25T11:28:42.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.821+0000 7f183fe00640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f18240744b0 0x7f18240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:42.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.822+0000 7f183fe00640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f18240744b0 0x7f18240769c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1830007fd0 tx=0x7f1830007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:42.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:42.825+0000 7f1845600640 1 -- 172.21.15.19:0/2973009525 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18380806c0 con 0x7f184806db10 2024-04-25T11:28:43.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.032+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f18480b5330 con 0x7f184806db10 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.033+0000 7f1845600640 1 -- 172.21.15.19:0/2973009525 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7f183807ffc0 con 0x7f184806db10 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:28:43.024 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:28:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.035+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f18240744b0 msgr2=0x7f18240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f18240744b0 0x7f18240769c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1830007fd0 tx=0x7f1830007960 comp rx=0 tx=0).stop 2024-04-25T11:28:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184806db10 msgr2=0x7f18480ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184806db10 0x7f18480ce0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18380090b0 tx=0x7f18380046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 shutdown_connections 2024-04-25T11:28:43.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f18240744b0 0x7f18240769c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1848142e40 0x7f184815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f184813fe10 0x7f1848159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 --2- 172.21.15.19:0/2973009525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f184806db10 0x7f18480ce0e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 >> 172.21.15.19:0/2973009525 conn(0x7f1848072ee0 msgr2=0x7f18480684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 shutdown_connections 2024-04-25T11:28:43.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.036+0000 7f184e000640 1 -- 172.21.15.19:0/2973009525 wait complete. 2024-04-25T11:28:43.053 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:28:43.053 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:28:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.337+0000 7ff41d000640 1 Processor -- start 2024-04-25T11:28:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.337+0000 7ff41d000640 1 -- start start 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff41d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff418160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff41d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff418160b90 0x7ff418161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff41d000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4180b7940 con 0x7ff4180d50e0 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff41d000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff418161280 con 0x7ff418160b90 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff41d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4180d4600 con 0x7ff4180d2cb0 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff417e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff418160b90 0x7ff418161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff417e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff418160b90 0x7ff418161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48632/0 (socket says 172.21.15.19:48632) 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff417e00640 1 -- 172.21.15.19:0/2670581700 learned_addr learned my addr 172.21.15.19:0/2670581700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.340+0000 7ff416a00640 1 --2- 172.21.15.19:0/2670581700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff418160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.341+0000 7ff417e00640 1 -- 172.21.15.19:0/2670581700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 msgr2=0x7ff418160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.341+0000 7ff417e00640 1 --2- 172.21.15.19:0/2670581700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff418160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.341+0000 7ff417e00640 1 -- 172.21.15.19:0/2670581700 >> v1:172.21.15.169:6789/0 conn(0x7ff4180d50e0 legacy=0x7ff4180d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.341+0000 7ff417e00640 1 -- 172.21.15.19:0/2670581700 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4180d48e0 con 0x7ff418160b90 2024-04-25T11:28:43.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.342+0000 7ff417e00640 1 --2- 172.21.15.19:0/2670581700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff418161810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff410009870 tx=0x7ff41002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dddd2e5a50d15f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:43.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.342+0000 7ff416000640 1 -- 172.21.15.19:0/2670581700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410037070 con 0x7ff418160b90 2024-04-25T11:28:43.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.342+0000 7ff416000640 1 -- 172.21.15.19:0/2670581700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410032440 con 0x7ff418160b90 2024-04-25T11:28:43.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.342+0000 7ff416000640 1 -- 172.21.15.19:0/2670581700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410032810 con 0x7ff418160b90 2024-04-25T11:28:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 -- 172.21.15.19:0/2670581700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 msgr2=0x7ff418161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 --2- 172.21.15.19:0/2670581700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff418161810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff410009870 tx=0x7ff41002a250 comp rx=0 tx=0).stop 2024-04-25T11:28:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 -- 172.21.15.19:0/2670581700 shutdown_connections 2024-04-25T11:28:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 --2- 172.21.15.19:0/2670581700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff418161810 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 --2- 172.21.15.19:0/2670581700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff418160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 -- 172.21.15.19:0/2670581700 >> 172.21.15.19:0/2670581700 conn(0x7ff41806c5c0 msgr2=0x7ff41806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 -- 172.21.15.19:0/2670581700 shutdown_connections 2024-04-25T11:28:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.343+0000 7ff41d000640 1 -- 172.21.15.19:0/2670581700 wait complete. 2024-04-25T11:28:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.346+0000 7ff41d000640 1 Processor -- start 2024-04-25T11:28:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.346+0000 7ff41d000640 1 -- start start 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.347+0000 7ff41d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff41815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.347+0000 7ff41d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4180d50e0 0x7ff4180ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.347+0000 7ff41d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff4180cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff41d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff418161560 con 0x7ff418160b90 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff41d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff418160fa0 con 0x7ff4180d50e0 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff41d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4180d54f0 con 0x7ff4180d2cb0 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff417e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff4180cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff417e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff4180cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48634/0 (socket says 172.21.15.19:48634) 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff417e00640 1 -- 172.21.15.19:0/2884129889 learned_addr learned my addr 172.21.15.19:0/2884129889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff416a00640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4180d50e0 0x7ff4180ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff417400640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff41815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff417e00640 1 -- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 msgr2=0x7ff41815c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.348+0000 7ff417e00640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff41815c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff417e00640 1 -- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4180d50e0 msgr2=0x7ff4180ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff417e00640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4180d50e0 0x7ff4180ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff417e00640 1 -- 172.21.15.19:0/2884129889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4180b51e0 con 0x7ff418160b90 2024-04-25T11:28:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff416a00640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4180d50e0 0x7ff4180ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:28:43.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff417e00640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff4180cd710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff41002a290 tx=0x7ff4100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:43.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff40f400640 1 -- 172.21.15.19:0/2884129889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410037040 con 0x7ff418160b90 2024-04-25T11:28:43.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff40f400640 1 -- 172.21.15.19:0/2884129889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410042070 con 0x7ff418160b90 2024-04-25T11:28:43.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff40f400640 1 -- 172.21.15.19:0/2884129889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410030920 con 0x7ff418160b90 2024-04-25T11:28:43.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.349+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41815c890 con 0x7ff418160b90 2024-04-25T11:28:43.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.350+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4180cb370 con 0x7ff418160b90 2024-04-25T11:28:43.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.351+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4180cae00 con 0x7ff418160b90 2024-04-25T11:28:43.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.352+0000 7ff40f400640 1 -- 172.21.15.19:0/2884129889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff410030ac0 con 0x7ff418160b90 2024-04-25T11:28:43.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.353+0000 7ff40f400640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3f40744b0 0x7ff3f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:43.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.353+0000 7ff40f400640 1 -- 172.21.15.19:0/2884129889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7ff4100b5900 con 0x7ff418160b90 2024-04-25T11:28:43.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.354+0000 7ff417400640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3f40744b0 0x7ff3f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:43.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.355+0000 7ff417400640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3f40744b0 0x7ff3f40769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff408000f60 tx=0x7ff408009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:43.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.359+0000 7ff40f400640 1 -- 172.21.15.19:0/2884129889 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4180b51e0 con 0x7ff418160b90 2024-04-25T11:28:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:43.562+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff418076c60 con 0x7ff418160b90 2024-04-25T11:28:44.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.233+0000 7ff40f400640 1 -- 172.21.15.19:0/2884129889 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7ff410080340 con 0x7ff418160b90 2024-04-25T11:28:44.225 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T11:28:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.237+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3f40744b0 msgr2=0x7ff3f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.237+0000 7ff41d000640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3f40744b0 0x7ff3f40769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff408000f60 tx=0x7ff408009040 comp rx=0 tx=0).stop 2024-04-25T11:28:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.237+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 msgr2=0x7ff4180cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.237+0000 7ff41d000640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff4180cd710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff41002a290 tx=0x7ff4100046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 shutdown_connections 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3f40744b0 0x7ff3f40769c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4180d50e0 0x7ff4180ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff418160b90 0x7ff4180cd710 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 --2- 172.21.15.19:0/2884129889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4180d2cb0 0x7ff41815c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 >> 172.21.15.19:0/2884129889 conn(0x7ff41806c5c0 msgr2=0x7ff41815ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 shutdown_connections 2024-04-25T11:28:44.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.238+0000 7ff41d000640 1 -- 172.21.15.19:0/2884129889 wait complete. 2024-04-25T11:28:44.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:28:44.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.555+0000 7f00e3e00640 1 Processor -- start 2024-04-25T11:28:44.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.555+0000 7f00e3e00640 1 -- start start 2024-04-25T11:28:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.557+0000 7f00e3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f00e406dce0 0x7f00e41606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.557+0000 7f00e3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e40b0090 con 0x7f00e4163c90 2024-04-25T11:28:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.557+0000 7f00e3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e4160c80 con 0x7f00e415de90 2024-04-25T11:28:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.557+0000 7f00e3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f00e4160f60 con 0x7f00e406dce0 2024-04-25T11:28:44.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.558+0000 7f00e3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f00e406dce0 0x7f00e41606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:44.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.558+0000 7f00e3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f00e406dce0 0x7f00e41606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43018/0 (socket says 172.21.15.19:43018) 2024-04-25T11:28:44.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.558+0000 7f00e3400640 1 -- 172.21.15.19:0/545572181 learned_addr learned my addr 172.21.15.19:0/545572181 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:44.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.558+0000 7f00e3400640 1 -- 172.21.15.19:0/545572181 >> v1:172.21.15.169:6789/0 conn(0x7f00e415de90 legacy=0x7f00e406f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.558+0000 7f00e3400640 1 -- 172.21.15.19:0/545572181 >> v1:172.21.15.19:6789/0 conn(0x7f00e4163c90 legacy=0x7f00e406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.558+0000 7f00e3400640 1 -- 172.21.15.19:0/545572181 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e4161240 con 0x7f00e406dce0 2024-04-25T11:28:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.558+0000 7f00e1600640 1 -- 172.21.15.19:0/545572181 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497477535 0 0) 0x7f00e40b0090 con 0x7f00e4163c90 2024-04-25T11:28:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.560+0000 7f00e3400640 1 --2- 172.21.15.19:0/545572181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e406dce0 0x7f00e41606a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009e50 tx=0x7f00dc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba2c38387198efb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.560+0000 7f00e1600640 1 -- 172.21.15.19:0/545572181 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc036070 con 0x7f00e406dce0 2024-04-25T11:28:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.560+0000 7f00e1600640 1 -- 172.21.15.19:0/545572181 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc031600 con 0x7f00e406dce0 2024-04-25T11:28:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.560+0000 7f00e1600640 1 -- 172.21.15.19:0/545572181 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0319d0 con 0x7f00e406dce0 2024-04-25T11:28:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.561+0000 7f00e3e00640 1 -- 172.21.15.19:0/545572181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e406dce0 msgr2=0x7f00e41606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.561+0000 7f00e3e00640 1 --2- 172.21.15.19:0/545572181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e406dce0 0x7f00e41606a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009e50 tx=0x7f00dc02a730 comp rx=0 tx=0).stop 2024-04-25T11:28:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.561+0000 7f00e3e00640 1 -- 172.21.15.19:0/545572181 shutdown_connections 2024-04-25T11:28:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.561+0000 7f00e3e00640 1 --2- 172.21.15.19:0/545572181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e406dce0 0x7f00e41606a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.561+0000 7f00e3e00640 1 -- 172.21.15.19:0/545572181 >> 172.21.15.19:0/545572181 conn(0x7f00e4072f40 msgr2=0x7f00e4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.561+0000 7f00e3e00640 1 -- 172.21.15.19:0/545572181 shutdown_connections 2024-04-25T11:28:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.561+0000 7f00e3e00640 1 -- 172.21.15.19:0/545572181 wait complete. 2024-04-25T11:28:44.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.563+0000 7f00e3e00640 1 Processor -- start 2024-04-25T11:28:44.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.564+0000 7f00e3e00640 1 -- start start 2024-04-25T11:28:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e415de90 0x7f00e4147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4163c90 0x7f00e4149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:44.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e415c070 0x7f00e415c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4070eb0 con 0x7f00e415de90 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4070d30 con 0x7f00e415c070 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f00e40b0570 con 0x7f00e4163c90 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e415de90 0x7f00e4147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e415de90 0x7f00e4147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43022/0 (socket says 172.21.15.19:43022) 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.565+0000 7f00e2a00640 1 -- 172.21.15.19:0/3875572518 learned_addr learned my addr 172.21.15.19:0/3875572518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.566+0000 7f00e2a00640 1 -- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4163c90 msgr2=0x7f00e4149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.566+0000 7f00e2a00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4163c90 0x7f00e4149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.566+0000 7f00e2a00640 1 -- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e415c070 msgr2=0x7f00e415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.566+0000 7f00e2a00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e415c070 0x7f00e415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.566+0000 7f00e2a00640 1 -- 172.21.15.19:0/3875572518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e415c9c0 con 0x7f00e415de90 2024-04-25T11:28:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.566+0000 7f00e2a00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e415de90 0x7f00e4147730 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f00d400c3c0 tx=0x7f00d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.566+0000 7f00daa00640 1 -- 172.21.15.19:0/3875572518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4017070 con 0x7f00e415de90 2024-04-25T11:28:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.567+0000 7f00daa00640 1 -- 172.21.15.19:0/3875572518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d40052f0 con 0x7f00e415de90 2024-04-25T11:28:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.567+0000 7f00daa00640 1 -- 172.21.15.19:0/3875572518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4003830 con 0x7f00e415de90 2024-04-25T11:28:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.567+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e414a420 con 0x7f00e415de90 2024-04-25T11:28:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.567+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e414ac10 con 0x7f00e415de90 2024-04-25T11:28:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.568+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e414ae10 con 0x7f00e415de90 2024-04-25T11:28:44.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.569+0000 7f00daa00640 1 -- 172.21.15.19:0/3875572518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00d401a070 con 0x7f00e415de90 2024-04-25T11:28:44.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.570+0000 7f00daa00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00c40744b0 0x7f00c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:44.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.571+0000 7f00daa00640 1 -- 172.21.15.19:0/3875572518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7f00d40943a0 con 0x7f00e415de90 2024-04-25T11:28:44.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.571+0000 7f00e2000640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00c40744b0 0x7f00c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:44.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.572+0000 7f00e2000640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00c40744b0 0x7f00c40769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f00d0000f60 tx=0x7f00d00079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:44.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.576+0000 7f00daa00640 1 -- 172.21.15.19:0/3875572518 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00e415c9c0 con 0x7f00e415de90 2024-04-25T11:28:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:44.789+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f00e4076c60 con 0x7f00e415de90 2024-04-25T11:28:45.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.240+0000 7f00daa00640 1 -- 172.21.15.19:0/3875572518 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7f00d405ed00 con 0x7f00e415de90 2024-04-25T11:28:45.231 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T11:28:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00c40744b0 msgr2=0x7f00c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00c40744b0 0x7f00c40769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f00d0000f60 tx=0x7f00d00079d0 comp rx=0 tx=0).stop 2024-04-25T11:28:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e415de90 msgr2=0x7f00e4147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:45.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e415de90 0x7f00e4147730 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f00d400c3c0 tx=0x7f00d400c880 comp rx=0 tx=0).stop 2024-04-25T11:28:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 shutdown_connections 2024-04-25T11:28:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00c40744b0 0x7f00c40769c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f00e415c070 0x7f00e415c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f00e4163c90 0x7f00e4149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 --2- 172.21.15.19:0/3875572518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e415de90 0x7f00e4147730 secure :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f00d400c3c0 tx=0x7f00d400c880 comp rx=0 tx=0).stop 2024-04-25T11:28:45.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.244+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 >> 172.21.15.19:0/3875572518 conn(0x7f00e4072f40 msgr2=0x7f00e415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.246+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 shutdown_connections 2024-04-25T11:28:45.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.247+0000 7f00e3e00640 1 -- 172.21.15.19:0/3875572518 wait complete. 2024-04-25T11:28:45.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:28:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.550+0000 7f61c6e00640 1 Processor -- start 2024-04-25T11:28:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.550+0000 7f61c6e00640 1 -- start start 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61c6e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61c00d1210 0x7f61c00d5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61c6e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c00d1850 con 0x7f61c006ccd0 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61c6e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c0160de0 con 0x7f61c00d0e00 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61c6e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f61c00d1620 con 0x7f61c00d1210 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61c4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61c00d1210 0x7f61c00d5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61c4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61c00d1210 0x7f61c00d5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43028/0 (socket says 172.21.15.19:43028) 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61c4e00640 1 -- 172.21.15.19:0/2946960692 learned_addr learned my addr 172.21.15.19:0/2946960692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.552+0000 7f61bf400640 1 --1- 172.21.15.19:0/2946960692 >> v1:172.21.15.19:6789/0 conn(0x7f61c006ccd0 0x7f61c00d4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39868/0 (socket says 172.21.15.19:39868) 2024-04-25T11:28:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.553+0000 7f61c4e00640 1 -- 172.21.15.19:0/2946960692 >> v1:172.21.15.169:6789/0 conn(0x7f61c00d0e00 legacy=0x7f61c006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.553+0000 7f61c4e00640 1 -- 172.21.15.19:0/2946960692 >> v1:172.21.15.19:6789/0 conn(0x7f61c006ccd0 legacy=0x7f61c00d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.553+0000 7f61c4e00640 1 -- 172.21.15.19:0/2946960692 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01610c0 con 0x7f61c00d1210 2024-04-25T11:28:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.554+0000 7f61c4e00640 1 --2- 172.21.15.19:0/2946960692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c00d1210 0x7f61c00d5160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009e50 tx=0x7f61b802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ff3ef941056122 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.554+0000 7f61bea00640 1 -- 172.21.15.19:0/2946960692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8036070 con 0x7f61c00d1210 2024-04-25T11:28:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.554+0000 7f61bea00640 1 -- 172.21.15.19:0/2946960692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8031600 con 0x7f61c00d1210 2024-04-25T11:28:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.554+0000 7f61bea00640 1 -- 172.21.15.19:0/2946960692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80319d0 con 0x7f61c00d1210 2024-04-25T11:28:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.555+0000 7f61c6e00640 1 -- 172.21.15.19:0/2946960692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c00d1210 msgr2=0x7f61c00d5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.555+0000 7f61c6e00640 1 --2- 172.21.15.19:0/2946960692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c00d1210 0x7f61c00d5160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009e50 tx=0x7f61b802a730 comp rx=0 tx=0).stop 2024-04-25T11:28:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.555+0000 7f61c6e00640 1 -- 172.21.15.19:0/2946960692 shutdown_connections 2024-04-25T11:28:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.555+0000 7f61c6e00640 1 --2- 172.21.15.19:0/2946960692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c00d1210 0x7f61c00d5160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.555+0000 7f61c6e00640 1 -- 172.21.15.19:0/2946960692 >> 172.21.15.19:0/2946960692 conn(0x7f61c00d5d90 msgr2=0x7f61c00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.555+0000 7f61c6e00640 1 -- 172.21.15.19:0/2946960692 shutdown_connections 2024-04-25T11:28:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.555+0000 7f61c6e00640 1 -- 172.21.15.19:0/2946960692 wait complete. 2024-04-25T11:28:45.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.558+0000 7f61c6e00640 1 Processor -- start 2024-04-25T11:28:45.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.558+0000 7f61c6e00640 1 -- start start 2024-04-25T11:28:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.559+0000 7f61c6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c00d0e00 0x7f61c0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.559+0000 7f61c6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c00d1210 0x7f61c00cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.559+0000 7f61c6e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c0163340 0x7f61c0163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.559+0000 7f61c6e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61c00b5320 con 0x7f61c0163340 2024-04-25T11:28:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.559+0000 7f61c6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0160ad0 con 0x7f61c00d0e00 2024-04-25T11:28:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.559+0000 7f61c6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61c0164e00 con 0x7f61c00d1210 2024-04-25T11:28:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c0163340 0x7f61c0163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c0163340 0x7f61c0163750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43034/0 (socket says 172.21.15.19:43034) 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 -- 172.21.15.19:0/3952826442 learned_addr learned my addr 172.21.15.19:0/3952826442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61bfe00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c00d0e00 0x7f61c0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61bf400640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c00d1210 0x7f61c00cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 -- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c00d1210 msgr2=0x7f61c00cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c00d1210 0x7f61c00cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 -- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c00d0e00 msgr2=0x7f61c0154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c00d0e00 0x7f61c0154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.560+0000 7f61c4e00640 1 -- 172.21.15.19:0/3952826442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c00ca820 con 0x7f61c0163340 2024-04-25T11:28:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.561+0000 7f61c4e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c0163340 0x7f61c0163750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009f70 tx=0x7f61b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.561+0000 7f61b7e00640 1 -- 172.21.15.19:0/3952826442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b803f020 con 0x7f61c0163340 2024-04-25T11:28:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.561+0000 7f61b7e00640 1 -- 172.21.15.19:0/3952826442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8037040 con 0x7f61c0163340 2024-04-25T11:28:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.561+0000 7f61b7e00640 1 -- 172.21.15.19:0/3952826442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8036070 con 0x7f61c0163340 2024-04-25T11:28:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.561+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c00cab00 con 0x7f61c0163340 2024-04-25T11:28:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.561+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c00cb0c0 con 0x7f61c0163340 2024-04-25T11:28:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.562+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c00ca570 con 0x7f61c0163340 2024-04-25T11:28:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.564+0000 7f61b7e00640 1 -- 172.21.15.19:0/3952826442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61b8042020 con 0x7f61c0163340 2024-04-25T11:28:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.565+0000 7f61b7e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61a00744e0 0x7f61a00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.565+0000 7f61b7e00640 1 -- 172.21.15.19:0/3952826442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7f61b80b5770 con 0x7f61c0163340 2024-04-25T11:28:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.566+0000 7f61bfe00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61a00744e0 0x7f61a00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.567+0000 7f61bfe00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61a00744e0 0x7f61a00769f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f61b0007fd0 tx=0x7f61b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.571+0000 7f61b7e00640 1 -- 172.21.15.19:0/3952826442 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61b8049030 con 0x7f61c0163340 2024-04-25T11:28:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:45.818+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f61c0050740 con 0x7f61c0163340 2024-04-25T11:28:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.292+0000 7f61b7e00640 1 -- 172.21.15.19:0/3952826442 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f61b807f870 con 0x7f61c0163340 2024-04-25T11:28:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T11:28:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.296+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61a00744e0 msgr2=0x7f61a00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.296+0000 7f61c6e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61a00744e0 0x7f61a00769f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f61b0007fd0 tx=0x7f61b0009040 comp rx=0 tx=0).stop 2024-04-25T11:28:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.296+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c0163340 msgr2=0x7f61c0163750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.296+0000 7f61c6e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c0163340 0x7f61c0163750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009f70 tx=0x7f61b8004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.299+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 shutdown_connections 2024-04-25T11:28:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.299+0000 7f61c6e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61a00744e0 0x7f61a00769f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.299+0000 7f61c6e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61c0163340 0x7f61c0163750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.299+0000 7f61c6e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61c00d1210 0x7f61c00cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.299+0000 7f61c6e00640 1 --2- 172.21.15.19:0/3952826442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61c00d0e00 0x7f61c0154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.299+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 >> 172.21.15.19:0/3952826442 conn(0x7f61c00d5d90 msgr2=0x7f61c00b63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.299+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 shutdown_connections 2024-04-25T11:28:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.305+0000 7f61c6e00640 1 -- 172.21.15.19:0/3952826442 wait complete. 2024-04-25T11:28:46.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:46.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.576+0000 7f9ee1800640 1 Processor -- start 2024-04-25T11:28:46.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.576+0000 7f9ee1800640 1 -- start start 2024-04-25T11:28:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.578+0000 7f9ee1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc070e20 0x7f9edc15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.578+0000 7f9ee1800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9edc06f180 0x7f9edc06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.578+0000 7f9ee1800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9edc06d7a0 con 0x7f9edc06d390 2024-04-25T11:28:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.578+0000 7f9ee1800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9edc071230 con 0x7f9edc06f180 2024-04-25T11:28:46.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.578+0000 7f9ee1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc06f8f0 con 0x7f9edc070e20 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.579+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc070e20 0x7f9edc15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.579+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc070e20 0x7f9edc15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:47934/0 (socket says 172.21.15.19:47934) 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.579+0000 7f9edbe00640 1 -- 172.21.15.19:0/3042496726 learned_addr learned my addr 172.21.15.19:0/3042496726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.579+0000 7f9edb400640 1 --2- 172.21.15.19:0/3042496726 >> v2:172.21.15.169:3300/0 conn(0x7f9edc06f180 0x7f9edc06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.580+0000 7f9edaa00640 1 -- 172.21.15.19:0/3042496726 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528973766 0 0) 0x7f9ed4001490 con 0x7f9edc06d390 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.580+0000 7f9edbe00640 1 -- 172.21.15.19:0/3042496726 >> v2:172.21.15.169:3300/0 conn(0x7f9edc06f180 msgr2=0x7f9edc06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.580+0000 7f9edbe00640 1 --2- 172.21.15.19:0/3042496726 >> v2:172.21.15.169:3300/0 conn(0x7f9edc06f180 0x7f9edc06ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.580+0000 7f9edbe00640 1 -- 172.21.15.19:0/3042496726 >> v1:172.21.15.169:6789/0 conn(0x7f9edc06d390 legacy=0x7f9edc0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.580+0000 7f9edbe00640 1 -- 172.21.15.19:0/3042496726 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc06fbd0 con 0x7f9edc070e20 2024-04-25T11:28:46.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.581+0000 7f9edbe00640 1 --2- 172.21.15.19:0/3042496726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc070e20 0x7f9edc15c800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00bb00 tx=0x7f9ecc02c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8aa649b72c0683e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:46.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.582+0000 7f9edaa00640 1 -- 172.21.15.19:0/3042496726 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc03a070 con 0x7f9edc070e20 2024-04-25T11:28:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.582+0000 7f9edaa00640 1 -- 172.21.15.19:0/3042496726 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ecc032480 con 0x7f9edc070e20 2024-04-25T11:28:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.582+0000 7f9edaa00640 1 -- 172.21.15.19:0/3042496726 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc032850 con 0x7f9edc070e20 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 -- 172.21.15.19:0/3042496726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc070e20 msgr2=0x7f9edc15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 --2- 172.21.15.19:0/3042496726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc070e20 0x7f9edc15c800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00bb00 tx=0x7f9ecc02c700 comp rx=0 tx=0).stop 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 -- 172.21.15.19:0/3042496726 shutdown_connections 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 --2- 172.21.15.19:0/3042496726 >> v2:172.21.15.169:3300/0 conn(0x7f9edc06f180 0x7f9edc06ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 --2- 172.21.15.19:0/3042496726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc070e20 0x7f9edc15c800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 -- 172.21.15.19:0/3042496726 >> 172.21.15.19:0/3042496726 conn(0x7f9edc0d1720 msgr2=0x7f9edc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 -- 172.21.15.19:0/3042496726 shutdown_connections 2024-04-25T11:28:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.583+0000 7f9ee1800640 1 -- 172.21.15.19:0/3042496726 wait complete. 2024-04-25T11:28:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.585+0000 7f9ee1800640 1 Processor -- start 2024-04-25T11:28:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.585+0000 7f9ee1800640 1 -- start start 2024-04-25T11:28:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee1800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9edc06d390 0x7f9edc1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee1800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc06f180 0x7f9edc13eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9edc070e20 0x7f9edc073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:46.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee1800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc0b6e30 con 0x7f9edc070e20 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee1800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc070190 con 0x7f9edc06d390 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee1800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc06f590 con 0x7f9edc06f180 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9edc070e20 0x7f9edc073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9edc070e20 0x7f9edc073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43040/0 (socket says 172.21.15.19:43040) 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.587+0000 7f9ee0e00640 1 -- 172.21.15.19:0/2943148906 learned_addr learned my addr 172.21.15.19:0/2943148906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9edb400640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc06f180 0x7f9edc13eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9edbe00640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9edc06d390 0x7f9edc1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ee0e00640 1 -- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc06f180 msgr2=0x7f9edc13eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ee0e00640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc06f180 0x7f9edc13eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ee0e00640 1 -- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9edc06d390 msgr2=0x7f9edc1f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ee0e00640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9edc06d390 0x7f9edc1f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ee0e00640 1 -- 172.21.15.19:0/2943148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc13fa80 con 0x7f9edc070e20 2024-04-25T11:28:46.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ee0e00640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9edc070e20 0x7f9edc073120 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c780 tx=0x7f9ed400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ed3e00640 1 -- 172.21.15.19:0/2943148906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4003b70 con 0x7f9edc070e20 2024-04-25T11:28:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ed3e00640 1 -- 172.21.15.19:0/2943148906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed4003d10 con 0x7f9edc070e20 2024-04-25T11:28:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.588+0000 7f9ed3e00640 1 -- 172.21.15.19:0/2943148906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4005270 con 0x7f9edc070e20 2024-04-25T11:28:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.589+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc13fd60 con 0x7f9edc070e20 2024-04-25T11:28:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.589+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc140320 con 0x7f9edc070e20 2024-04-25T11:28:46.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.590+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc13f7f0 con 0x7f9edc070e20 2024-04-25T11:28:46.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.591+0000 7f9ed3e00640 1 -- 172.21.15.19:0/2943148906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ed4005410 con 0x7f9edc070e20 2024-04-25T11:28:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.592+0000 7f9ed3e00640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9eb80744e0 0x7f9eb80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.592+0000 7f9ed3e00640 1 -- 172.21.15.19:0/2943148906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9ed4095060 con 0x7f9edc070e20 2024-04-25T11:28:46.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.593+0000 7f9edbe00640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9eb80744e0 0x7f9eb80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:46.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.594+0000 7f9edbe00640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9eb80744e0 0x7f9eb80769f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc037000 tx=0x7f9ecc030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.602+0000 7f9ed3e00640 1 -- 172.21.15.19:0/2943148906 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9edc06d7a0 con 0x7f9edc070e20 2024-04-25T11:28:46.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.804+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9edc0708d0 con 0x7f9edc070e20 2024-04-25T11:28:46.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.807+0000 7f9ed3e00640 1 -- 172.21.15.19:0/2943148906 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16622 (secure 0 0 0) 0x7f9edc0708d0 con 0x7f9edc070e20 2024-04-25T11:28:46.798 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:46.798 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:45.819863+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9eb80744e0 msgr2=0x7f9eb80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9eb80744e0 0x7f9eb80769f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc037000 tx=0x7f9ecc030000 comp rx=0 tx=0).stop 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9edc070e20 msgr2=0x7f9edc073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9edc070e20 0x7f9edc073120 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400c780 tx=0x7f9ed400cb80 comp rx=0 tx=0).stop 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 shutdown_connections 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9eb80744e0 0x7f9eb80769f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9edc070e20 0x7f9edc073120 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9edc06f180 0x7f9edc13eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 --2- 172.21.15.19:0/2943148906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9edc06d390 0x7f9edc1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 >> 172.21.15.19:0/2943148906 conn(0x7f9edc0d1720 msgr2=0x7f9edc0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 shutdown_connections 2024-04-25T11:28:46.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:46.810+0000 7f9ee1800640 1 -- 172.21.15.19:0/2943148906 wait complete. 2024-04-25T11:28:46.825 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.090+0000 7f1ab6400640 1 Processor -- start 2024-04-25T11:28:47.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.090+0000 7f1ab6400640 1 -- start start 2024-04-25T11:28:47.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1ab6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab006c5c0 0x7f1ab00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1ab6400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ab0144370 0x7f1ab00b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1ab6400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab00b50e0 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1ab6400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab00d1230 con 0x7f1ab0144370 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1ab6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab00d06b0 con 0x7f1ab006c5c0 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1aafe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ab0144370 0x7f1ab00b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1aafe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ab0144370 0x7f1ab00b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43056/0 (socket says 172.21.15.19:43056) 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1aafe00640 1 -- 172.21.15.19:0/2743131162 learned_addr learned my addr 172.21.15.19:0/2743131162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1ab5a00640 1 --1- 172.21.15.19:0/2743131162 >> v1:172.21.15.19:6789/0 conn(0x7f1ab00b4cd0 0x7f1ab0161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39876/0 (socket says 172.21.15.19:39876) 2024-04-25T11:28:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.093+0000 7f1ab5000640 1 --2- 172.21.15.19:0/2743131162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab006c5c0 0x7f1ab00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.094+0000 7f1aafe00640 1 -- 172.21.15.19:0/2743131162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab006c5c0 msgr2=0x7f1ab00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.094+0000 7f1aafe00640 1 --2- 172.21.15.19:0/2743131162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab006c5c0 0x7f1ab00cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.094+0000 7f1aafe00640 1 -- 172.21.15.19:0/2743131162 >> v1:172.21.15.19:6789/0 conn(0x7f1ab00b4cd0 legacy=0x7f1ab0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.094+0000 7f1aafe00640 1 -- 172.21.15.19:0/2743131162 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab00d0990 con 0x7f1ab0144370 2024-04-25T11:28:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.094+0000 7f1ab5000640 1 --2- 172.21.15.19:0/2743131162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab006c5c0 0x7f1ab00cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:47.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.095+0000 7f1aafe00640 1 --2- 172.21.15.19:0/2743131162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab0144370 0x7f1ab00b4790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bf940a1225b71a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.095+0000 7f1aaf400640 1 -- 172.21.15.19:0/2743131162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c037070 con 0x7f1ab0144370 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.095+0000 7f1aaf400640 1 -- 172.21.15.19:0/2743131162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9c032440 con 0x7f1ab0144370 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.095+0000 7f1aaf400640 1 -- 172.21.15.19:0/2743131162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c032810 con 0x7f1ab0144370 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 -- 172.21.15.19:0/2743131162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab0144370 msgr2=0x7f1ab00b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 --2- 172.21.15.19:0/2743131162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab0144370 0x7f1ab00b4790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c02a250 comp rx=0 tx=0).stop 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 -- 172.21.15.19:0/2743131162 shutdown_connections 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 --2- 172.21.15.19:0/2743131162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab0144370 0x7f1ab00b4790 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 --2- 172.21.15.19:0/2743131162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab006c5c0 0x7f1ab00cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 -- 172.21.15.19:0/2743131162 >> 172.21.15.19:0/2743131162 conn(0x7f1ab00d1720 msgr2=0x7f1ab00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 -- 172.21.15.19:0/2743131162 shutdown_connections 2024-04-25T11:28:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.096+0000 7f1ab6400640 1 -- 172.21.15.19:0/2743131162 wait complete. 2024-04-25T11:28:47.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.098+0000 7f1ab6400640 1 Processor -- start 2024-04-25T11:28:47.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.099+0000 7f1ab6400640 1 -- start start 2024-04-25T11:28:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1ab6400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab006c5c0 0x7f1ab01423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1ab6400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab00b4cd0 0x7f1ab00b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1ab6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab0144370 0x7f1ab00b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1ab6400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00d02d0 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1ab6400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab00d0f50 con 0x7f1ab006c5c0 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1ab6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab00d0450 con 0x7f1ab0144370 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1aafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab00b4cd0 0x7f1ab00b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1aafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab00b4cd0 0x7f1ab00b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43068/0 (socket says 172.21.15.19:43068) 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.100+0000 7f1aafe00640 1 -- 172.21.15.19:0/4145057195 learned_addr learned my addr 172.21.15.19:0/4145057195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1ab5a00640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab0144370 0x7f1ab00b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1ab5000640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab006c5c0 0x7f1ab01423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1aafe00640 1 -- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab0144370 msgr2=0x7f1ab00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1aafe00640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab0144370 0x7f1ab00b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1aafe00640 1 -- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab006c5c0 msgr2=0x7f1ab01423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1aafe00640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab006c5c0 0x7f1ab01423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1aafe00640 1 -- 172.21.15.19:0/4145057195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab0070b60 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1aafe00640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab00b4cd0 0x7f1ab00b28c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02a290 tx=0x7f1a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.101+0000 7f1aacc00640 1 -- 172.21.15.19:0/4145057195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c037040 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.102+0000 7f1aacc00640 1 -- 172.21.15.19:0/4145057195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9c042070 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.102+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab0161190 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.102+0000 7f1aacc00640 1 -- 172.21.15.19:0/4145057195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c030bb0 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.102+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab0161750 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.103+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab0050740 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.104+0000 7f1aacc00640 1 -- 172.21.15.19:0/4145057195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a9c047460 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.105+0000 7f1aacc00640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a900744b0 0x7f1a900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.105+0000 7f1aacc00640 1 -- 172.21.15.19:0/4145057195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1a9c0b5f60 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.106+0000 7f1ab5000640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a900744b0 0x7f1a900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.107+0000 7f1ab5000640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a900744b0 0x7f1a900769c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400b840 tx=0x7f1aa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:47.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.110+0000 7f1aacc00640 1 -- 172.21.15.19:0/4145057195 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a9c080650 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.311+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ab00cfd70 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.313+0000 7f1aacc00640 1 -- 172.21.15.19:0/4145057195 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16609 (secure 0 0 0) 0x7f1a9c07ff50 con 0x7f1ab00b4cd0 2024-04-25T11:28:47.304 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:47.304 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:47.266661+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.315+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a900744b0 msgr2=0x7f1a900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.315+0000 7f1ab6400640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a900744b0 0x7f1a900769c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400b840 tx=0x7f1aa4008040 comp rx=0 tx=0).stop 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab00b4cd0 msgr2=0x7f1ab00b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab00b4cd0 0x7f1ab00b28c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c02a290 tx=0x7f1a9c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 shutdown_connections 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a900744b0 0x7f1a900769c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ab0144370 0x7f1ab00b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab00b4cd0 0x7f1ab00b28c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 --2- 172.21.15.19:0/4145057195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ab006c5c0 0x7f1ab01423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 >> 172.21.15.19:0/4145057195 conn(0x7f1ab00d1720 msgr2=0x7f1ab0146b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 shutdown_connections 2024-04-25T11:28:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.316+0000 7f1ab6400640 1 -- 172.21.15.19:0/4145057195 wait complete. 2024-04-25T11:28:47.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:28:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.617+0000 7f83c9200640 1 Processor -- start 2024-04-25T11:28:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.617+0000 7f83c9200640 1 -- start start 2024-04-25T11:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.619+0000 7f83c9200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83c4163ca0 0x7f83c40b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.619+0000 7f83c9200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c40b97d0 con 0x7f83c4160340 2024-04-25T11:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.619+0000 7f83c9200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c41641f0 con 0x7f83c40b4ce0 2024-04-25T11:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.619+0000 7f83c9200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f83c41644d0 con 0x7f83c4163ca0 2024-04-25T11:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.619+0000 7f83c3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f83c4160340 0x7f83c40b90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39880/0 (socket says 172.21.15.19:39880) 2024-04-25T11:28:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.619+0000 7f83c3400640 1 -- 172.21.15.19:0/3222277838 learned_addr learned my addr 172.21.15.19:0/3222277838 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2a00640 1 --2- 172.21.15.19:0/3222277838 >> v2:172.21.15.169:3300/0 conn(0x7f83c4163ca0 0x7f83c40b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86057055 0 0) 0x7f83b4001490 con 0x7f83c4160340 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83a8003520 con 0x7f83c4160340 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 920069763 0 0) 0x7f83a8003520 con 0x7f83c4160340 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 >> v2:172.21.15.169:3300/0 conn(0x7f83c4163ca0 msgr2=0x7f83c40b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 --2- 172.21.15.19:0/3222277838 >> v2:172.21.15.169:3300/0 conn(0x7f83c4163ca0 0x7f83c40b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 >> v1:172.21.15.169:6789/0 conn(0x7f83c40b4ce0 legacy=0x7f83c40d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c41647b0 con 0x7f83c4160340 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f83b4001bc0 con 0x7f83c4160340 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.620+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 >> v1:172.21.15.19:6789/0 conn(0x7f83c4160340 legacy=0x7f83c40b90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c2000640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c2000640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a8004800 0x7f83a8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c2000640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a80051c0 0x7f83a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41644d0 con 0x7f83a80039b0 2024-04-25T11:28:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f83a800c150 con 0x7f83a80051c0 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83a800c350 con 0x7f83a8004800 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3400640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3400640 1 -- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a8004800 msgr2=0x7f83a8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3400640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83a8004800 0x7f83a8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3e00640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a80051c0 0x7f83a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3400640 1 -- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a80051c0 msgr2=0x7f83a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3400640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a80051c0 0x7f83a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3400640 1 -- 172.21.15.19:0/3222277838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a8003520 con 0x7f83a80039b0 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3e00640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83a80051c0 0x7f83a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.621+0000 7f83c3e00640 1 -- 172.21.15.19:0/3222277838 reap_dead start 2024-04-25T11:28:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.622+0000 7f83c3400640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83a8003e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83b400aa50 tx=0x7f83a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5eddb4aadfb3a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.623+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b400a670 con 0x7f83a80039b0 2024-04-25T11:28:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.623+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b402f070 con 0x7f83a80039b0 2024-04-25T11:28:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.623+0000 7f83c2000640 1 -- 172.21.15.19:0/3222277838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b402be40 con 0x7f83a80039b0 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.623+0000 7f83c9200640 1 -- 172.21.15.19:0/3222277838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 msgr2=0x7f83a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.623+0000 7f83c9200640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83a8003e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83b400aa50 tx=0x7f83a8005830 comp rx=0 tx=0).stop 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.624+0000 7f83c9200640 1 -- 172.21.15.19:0/3222277838 shutdown_connections 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.624+0000 7f83c9200640 1 --2- 172.21.15.19:0/3222277838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83a8003e20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.624+0000 7f83c9200640 1 -- 172.21.15.19:0/3222277838 >> 172.21.15.19:0/3222277838 conn(0x7f83c40d29a0 msgr2=0x7f83c40d2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.624+0000 7f83c9200640 1 -- 172.21.15.19:0/3222277838 shutdown_connections 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.624+0000 7f83c9200640 1 -- 172.21.15.19:0/3222277838 wait complete. 2024-04-25T11:28:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.626+0000 7f83c9200640 1 Processor -- start 2024-04-25T11:28:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.626+0000 7f83c9200640 1 -- start start 2024-04-25T11:28:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83c414e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83c40c9630 0x7f83c41622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83c4162800 0x7f83c41e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c9200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40d1690 con 0x7f83a80039b0 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c9200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40d1510 con 0x7f83c4162800 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83c40b90c0 con 0x7f83c40c9630 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83c414e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83c414e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43082/0 (socket says 172.21.15.19:43082) 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c3400640 1 -- 172.21.15.19:0/1425026034 learned_addr learned my addr 172.21.15.19:0/1425026034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c2a00640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83c40c9630 0x7f83c41622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.628+0000 7f83c3e00640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83c4162800 0x7f83c41e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c3400640 1 -- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83c40c9630 msgr2=0x7f83c41622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c3400640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83c40c9630 0x7f83c41622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c3400640 1 -- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83c4162800 msgr2=0x7f83c41e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c3400640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83c4162800 0x7f83c41e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c3400640 1 -- 172.21.15.19:0/1425026034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c414b9e0 con 0x7f83a80039b0 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c2a00640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83c40c9630 0x7f83c41622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c3400640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83c414e3d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f83b400a0b0 tx=0x7f83b4001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83bb400640 1 -- 172.21.15.19:0/1425026034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4044070 con 0x7f83a80039b0 2024-04-25T11:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83bb400640 1 -- 172.21.15.19:0/1425026034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b402bdb0 con 0x7f83a80039b0 2024-04-25T11:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c4162c10 con 0x7f83a80039b0 2024-04-25T11:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.629+0000 7f83bb400640 1 -- 172.21.15.19:0/1425026034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b400a670 con 0x7f83a80039b0 2024-04-25T11:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.630+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c414bf30 con 0x7f83a80039b0 2024-04-25T11:28:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.631+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c4076c60 con 0x7f83a80039b0 2024-04-25T11:28:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.632+0000 7f83bb400640 1 -- 172.21.15.19:0/1425026034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83b4047070 con 0x7f83a80039b0 2024-04-25T11:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.633+0000 7f83bb400640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83a00744b0 0x7f83a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.633+0000 7f83bb400640 1 -- 172.21.15.19:0/1425026034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f83b40b5740 con 0x7f83a80039b0 2024-04-25T11:28:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.633+0000 7f83c2a00640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83a00744b0 0x7f83a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.635+0000 7f83c2a00640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83a00744b0 0x7f83a00769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0090e0 tx=0x7f83ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.637+0000 7f83bb400640 1 -- 172.21.15.19:0/1425026034 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83c4162c10 con 0x7f83a80039b0 2024-04-25T11:28:47.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:47.874+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f83c40d3f20 con 0x7f83a80039b0 2024-04-25T11:28:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.306+0000 7f83bb400640 1 -- 172.21.15.19:0/1425026034 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f83c40b97d0 con 0x7f83a80039b0 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83a00744b0 msgr2=0x7f83a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83a00744b0 0x7f83a00769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0090e0 tx=0x7f83ac008040 comp rx=0 tx=0).stop 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 msgr2=0x7f83c414e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83c414e3d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f83b400a0b0 tx=0x7f83b4001f90 comp rx=0 tx=0).stop 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 shutdown_connections 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83a00744b0 0x7f83a00769c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83c4162800 0x7f83c41e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83c40c9630 0x7f83c41622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 --2- 172.21.15.19:0/1425026034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83a80039b0 0x7f83c414e3d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.311+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 >> 172.21.15.19:0/1425026034 conn(0x7f83c40d29a0 msgr2=0x7f83c40b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.312+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 shutdown_connections 2024-04-25T11:28:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.312+0000 7f83c9200640 1 -- 172.21.15.19:0/1425026034 wait complete. 2024-04-25T11:28:48.334 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:28:48.334 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:28:48.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.634+0000 7fc0de400640 1 Processor -- start 2024-04-25T11:28:48.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.635+0000 7fc0de400640 1 -- start start 2024-04-25T11:28:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.636+0000 7fc0de400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 0x7fc0d8161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.637+0000 7fc0de400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d8162450 con 0x7fc0d806cd40 2024-04-25T11:28:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.637+0000 7fc0de400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d806d3a0 con 0x7fc0d8162040 2024-04-25T11:28:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.637+0000 7fc0de400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d806d150 con 0x7fc0d81441d0 2024-04-25T11:28:48.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.637+0000 7fc0dda00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc0d806cd40 0x7fc0d815fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39892/0 (socket says 172.21.15.19:39892) 2024-04-25T11:28:48.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.637+0000 7fc0dda00640 1 -- 172.21.15.19:0/106686287 learned_addr learned my addr 172.21.15.19:0/106686287 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:48.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.637+0000 7fc0dd000640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 0x7fc0d8161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.637+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251961034 0 0) 0x7fc0d0001110 con 0x7fc0d806cd40 2024-04-25T11:28:48.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0bc003520 con 0x7fc0d806cd40 2024-04-25T11:28:48.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3826442380 0 0) 0x7fc0bc003520 con 0x7fc0d806cd40 2024-04-25T11:28:48.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 msgr2=0x7fc0d8161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:48.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 0x7fc0d8161b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 >> v1:172.21.15.169:6789/0 conn(0x7fc0d8162040 legacy=0x7fc0d806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d806d680 con 0x7fc0d806cd40 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fc0d0001840 con 0x7fc0d806cd40 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 >> v1:172.21.15.19:6789/0 conn(0x7fc0d806cd40 legacy=0x7fc0d815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0bc0039e0 0x7fc0bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 0x7fc0bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc004d40 0x7fc0bc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d806d150 con 0x7fc0bc0039e0 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc00bb50 con 0x7fc0bc004d40 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0bc00bd00 con 0x7fc0d81441d0 2024-04-25T11:28:48.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.638+0000 7fc0dd000640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0bc0039e0 0x7fc0bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0d7e00640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 0x7fc0bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0dd000640 1 -- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 msgr2=0x7fc0bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0dd000640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0d81441d0 0x7fc0bc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0dda00640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc004d40 0x7fc0bc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0dd000640 1 -- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc004d40 msgr2=0x7fc0bc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0dd000640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc004d40 0x7fc0bc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0dd000640 1 -- 172.21.15.19:0/106686287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc003520 con 0x7fc0bc0039e0 2024-04-25T11:28:48.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.639+0000 7fc0dda00640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc004d40 0x7fc0bc005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:48.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.640+0000 7fc0dd000640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0bc0039e0 0x7fc0bc003e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009ac0 tx=0x7fc0bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625fd5c37a1988d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:48.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.640+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8036070 con 0x7fc0bc0039e0 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c8031400 con 0x7fc0bc0039e0 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0d7400640 1 -- 172.21.15.19:0/106686287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8009690 con 0x7fc0bc0039e0 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0de400640 1 -- 172.21.15.19:0/106686287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0bc0039e0 msgr2=0x7fc0bc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0de400640 1 --2- 172.21.15.19:0/106686287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0bc0039e0 0x7fc0bc003e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009ac0 tx=0x7fc0bc005830 comp rx=0 tx=0).stop 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0dda00640 1 -- 172.21.15.19:0/106686287 reap_dead start 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0de400640 1 -- 172.21.15.19:0/106686287 shutdown_connections 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0de400640 1 -- 172.21.15.19:0/106686287 >> 172.21.15.19:0/106686287 conn(0x7fc0d8072f40 msgr2=0x7fc0d8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0de400640 1 -- 172.21.15.19:0/106686287 shutdown_connections 2024-04-25T11:28:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.641+0000 7fc0de400640 1 -- 172.21.15.19:0/106686287 wait complete. 2024-04-25T11:28:48.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.644+0000 7fc0de400640 1 Processor -- start 2024-04-25T11:28:48.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.644+0000 7fc0de400640 1 -- start start 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0de400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc0039e0 0x7fc0d80cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0de400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d80af230 0x7fc0d80af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0de400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d80b5790 0x7fc0d80b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0de400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d81445e0 con 0x7fc0d80af230 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0de400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d806dc40 con 0x7fc0d80b5790 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0de400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d8161b00 con 0x7fc0bc0039e0 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d80af230 0x7fc0d80af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d80af230 0x7fc0d80af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43104/0 (socket says 172.21.15.19:43104) 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0d7e00640 1 -- 172.21.15.19:0/3139575758 learned_addr learned my addr 172.21.15.19:0/3139575758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0dd000640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc0039e0 0x7fc0d80cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0dda00640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d80b5790 0x7fc0d80b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0d7e00640 1 -- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc0039e0 msgr2=0x7fc0d80cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0d7e00640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc0039e0 0x7fc0d80cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0d7e00640 1 -- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d80b5790 msgr2=0x7fc0d80b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.646+0000 7fc0d7e00640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d80b5790 0x7fc0d80b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.647+0000 7fc0d7e00640 1 -- 172.21.15.19:0/3139575758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d80b60e0 con 0x7fc0d80af230 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.647+0000 7fc0d7e00640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d80af230 0x7fc0d80af640 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400db90 tx=0x7fc0c400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.647+0000 7fc0d4c00640 1 -- 172.21.15.19:0/3139575758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400bdb0 con 0x7fc0d80af230 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.647+0000 7fc0d4c00640 1 -- 172.21.15.19:0/3139575758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c4017070 con 0x7fc0d80af230 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.647+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d80afeb0 con 0x7fc0d80af230 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.647+0000 7fc0d4c00640 1 -- 172.21.15.19:0/3139575758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4003bd0 con 0x7fc0d80af230 2024-04-25T11:28:48.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.647+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d80b0470 con 0x7fc0d80af230 2024-04-25T11:28:48.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.649+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d80afc20 con 0x7fc0d80af230 2024-04-25T11:28:48.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.650+0000 7fc0d4c00640 1 -- 172.21.15.19:0/3139575758 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc0c4005070 con 0x7fc0d80af230 2024-04-25T11:28:48.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.651+0000 7fc0d4c00640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0b80744b0 0x7fc0b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:48.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.651+0000 7fc0d4c00640 1 -- 172.21.15.19:0/3139575758 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc0c4095540 con 0x7fc0d80af230 2024-04-25T11:28:48.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.652+0000 7fc0dd000640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0b80744b0 0x7fc0b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:48.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.653+0000 7fc0dd000640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0b80744b0 0x7fc0b80769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8033000 tx=0x7fc0c8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:48.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.656+0000 7fc0d4c00640 1 -- 172.21.15.19:0/3139575758 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0d80b60e0 con 0x7fc0d80af230 2024-04-25T11:28:48.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:48.903+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc0d8076c60 con 0x7fc0d80af230 2024-04-25T11:28:49.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.320+0000 7fc0d4c00640 1 -- 172.21.15.19:0/3139575758 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fc0c4003d70 con 0x7fc0d80af230 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0b80744b0 msgr2=0x7fc0b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0b80744b0 0x7fc0b80769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8033000 tx=0x7fc0c8008030 comp rx=0 tx=0).stop 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d80af230 msgr2=0x7fc0d80af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d80af230 0x7fc0d80af640 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400db90 tx=0x7fc0c400b4a0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 shutdown_connections 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0b80744b0 0x7fc0b80769c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0d80b5790 0x7fc0d80b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d80af230 0x7fc0d80af640 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 --2- 172.21.15.19:0/3139575758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc0039e0 0x7fc0d80cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 >> 172.21.15.19:0/3139575758 conn(0x7fc0d8072f40 msgr2=0x7fc0d80b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 shutdown_connections 2024-04-25T11:28:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.324+0000 7fc0de400640 1 -- 172.21.15.19:0/3139575758 wait complete. 2024-04-25T11:28:49.347 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:28:49.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:28:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.640+0000 7f3304600640 1 Processor -- start 2024-04-25T11:28:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.640+0000 7f3304600640 1 -- start start 2024-04-25T11:28:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.643+0000 7f3304600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32fc160360 0x7f32fc1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.643+0000 7f3304600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32fc06c7d0 con 0x7f32fc0b5c90 2024-04-25T11:28:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.643+0000 7f3304600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32fc06cab0 con 0x7f32fc070680 2024-04-25T11:28:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.643+0000 7f3304600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f32fc06d100 con 0x7f32fc160360 2024-04-25T11:28:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.643+0000 7f3302600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f32fc0b5c90 0x7f32fc0b60a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39898/0 (socket says 172.21.15.19:39898) 2024-04-25T11:28:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.643+0000 7f3302600640 1 -- 172.21.15.19:0/3290433874 learned_addr learned my addr 172.21.15.19:0/3290433874 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.643+0000 7f3301200640 1 --2- 172.21.15.19:0/3290433874 >> v2:172.21.15.19:3300/0 conn(0x7f32fc160360 0x7f32fc1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.644+0000 7f3301200640 1 -- 172.21.15.19:0/3290433874 >> v1:172.21.15.169:6789/0 conn(0x7f32fc070680 legacy=0x7f32fc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.644+0000 7f3301200640 1 -- 172.21.15.19:0/3290433874 >> v1:172.21.15.19:6789/0 conn(0x7f32fc0b5c90 legacy=0x7f32fc0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.644+0000 7f3301200640 1 -- 172.21.15.19:0/3290433874 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc06cd90 con 0x7f32fc160360 2024-04-25T11:28:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.644+0000 7f32fbe00640 1 -- 172.21.15.19:0/3290433874 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571705271 0 0) 0x7f32fc06c7d0 con 0x7f32fc0b5c90 2024-04-25T11:28:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.645+0000 7f3301200640 1 --2- 172.21.15.19:0/3290433874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc160360 0x7f32fc1647f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f32fc06d280 tx=0x7f32e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1cf5aa3e1860f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.645+0000 7f32fbe00640 1 -- 172.21.15.19:0/3290433874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8037070 con 0x7f32fc160360 2024-04-25T11:28:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.645+0000 7f32fbe00640 1 -- 172.21.15.19:0/3290433874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e8032440 con 0x7f32fc160360 2024-04-25T11:28:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.645+0000 7f32fbe00640 1 -- 172.21.15.19:0/3290433874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8032810 con 0x7f32fc160360 2024-04-25T11:28:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.646+0000 7f3304600640 1 -- 172.21.15.19:0/3290433874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc160360 msgr2=0x7f32fc1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.646+0000 7f3304600640 1 --2- 172.21.15.19:0/3290433874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc160360 0x7f32fc1647f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f32fc06d280 tx=0x7f32e802a180 comp rx=0 tx=0).stop 2024-04-25T11:28:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.646+0000 7f3304600640 1 -- 172.21.15.19:0/3290433874 shutdown_connections 2024-04-25T11:28:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.646+0000 7f3304600640 1 --2- 172.21.15.19:0/3290433874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc160360 0x7f32fc1647f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.646+0000 7f3304600640 1 -- 172.21.15.19:0/3290433874 >> 172.21.15.19:0/3290433874 conn(0x7f32fc0d1ae0 msgr2=0x7f32fc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.647+0000 7f3304600640 1 -- 172.21.15.19:0/3290433874 shutdown_connections 2024-04-25T11:28:49.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.647+0000 7f3304600640 1 -- 172.21.15.19:0/3290433874 wait complete. 2024-04-25T11:28:49.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.649+0000 7f3304600640 1 Processor -- start 2024-04-25T11:28:49.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.649+0000 7f3304600640 1 -- start start 2024-04-25T11:28:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.650+0000 7f3304600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32fc070680 0x7f32fc148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3304600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc0b5c90 0x7f32fc155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3304600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32fc160360 0x7f32fc161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3304600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc0afd30 con 0x7f32fc0b5c90 2024-04-25T11:28:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3304600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc0afbb0 con 0x7f32fc070680 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3304600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32fc06d590 con 0x7f32fc160360 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc0b5c90 0x7f32fc155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc0b5c90 0x7f32fc155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43118/0 (socket says 172.21.15.19:43118) 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301200640 1 -- 172.21.15.19:0/458622680 learned_addr learned my addr 172.21.15.19:0/458622680 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3302600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32fc160360 0x7f32fc161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301c00640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32fc070680 0x7f32fc148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301200640 1 -- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32fc160360 msgr2=0x7f32fc161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301200640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32fc160360 0x7f32fc161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301200640 1 -- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32fc070680 msgr2=0x7f32fc148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.651+0000 7f3301200640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32fc070680 0x7f32fc148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.652+0000 7f3301200640 1 -- 172.21.15.19:0/458622680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc156d40 con 0x7f32fc0b5c90 2024-04-25T11:28:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.652+0000 7f3301c00640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32fc070680 0x7f32fc148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.652+0000 7f3301200640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc0b5c90 0x7f32fc155fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.652+0000 7f32f9600640 1 -- 172.21.15.19:0/458622680 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8041070 con 0x7f32fc0b5c90 2024-04-25T11:28:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.652+0000 7f32f9600640 1 -- 172.21.15.19:0/458622680 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e803c890 con 0x7f32fc0b5c90 2024-04-25T11:28:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.652+0000 7f32f9600640 1 -- 172.21.15.19:0/458622680 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8037040 con 0x7f32fc0b5c90 2024-04-25T11:28:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.652+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc157020 con 0x7f32fc0b5c90 2024-04-25T11:28:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.653+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc146070 con 0x7f32fc0b5c90 2024-04-25T11:28:49.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.654+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32fc156ab0 con 0x7f32fc0b5c90 2024-04-25T11:28:49.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.655+0000 7f32f9600640 1 -- 172.21.15.19:0/458622680 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f32e8044070 con 0x7f32fc0b5c90 2024-04-25T11:28:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.656+0000 7f32f9600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32dc0744b0 0x7f32dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.656+0000 7f32f9600640 1 -- 172.21.15.19:0/458622680 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f32e80b5f00 con 0x7f32fc0b5c90 2024-04-25T11:28:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.657+0000 7f3301c00640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32dc0744b0 0x7f32dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.658+0000 7f3301c00640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32dc0744b0 0x7f32dc0769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32f0000f60 tx=0x7f32f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:49.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.661+0000 7f32f9600640 1 -- 172.21.15.19:0/458622680 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32e8080680 con 0x7f32fc0b5c90 2024-04-25T11:28:49.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.902+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32fc050740 con 0x7f32fc0b5c90 2024-04-25T11:28:49.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.904+0000 7f32f9600640 1 -- 172.21.15.19:0/458622680 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f32e807ff80 con 0x7f32fc0b5c90 2024-04-25T11:28:49.895 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:49.895 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:28:48.905254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3831883468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3831883468},{"type":"v1","addr":"172.21.15.19:6835","nonce":3831883468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5064,"qdb_cluster":[5064]},"id":2}]} 2024-04-25T11:28:49.895 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32dc0744b0 msgr2=0x7f32dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32dc0744b0 0x7f32dc0769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32f0000f60 tx=0x7f32f0009040 comp rx=0 tx=0).stop 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc0b5c90 msgr2=0x7f32fc155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc0b5c90 0x7f32fc155fd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e80046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 shutdown_connections 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32dc0744b0 0x7f32dc0769c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32fc160360 0x7f32fc161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32fc0b5c90 0x7f32fc155fd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 --2- 172.21.15.19:0/458622680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32fc070680 0x7f32fc148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:49.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 >> 172.21.15.19:0/458622680 conn(0x7f32fc0d1ae0 msgr2=0x7f32fc0b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:49.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 shutdown_connections 2024-04-25T11:28:49.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:49.907+0000 7f3304600640 1 -- 172.21.15.19:0/458622680 wait complete. 2024-04-25T11:28:49.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.195+0000 7f2966e00640 1 Processor -- start 2024-04-25T11:28:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.196+0000 7f2966e00640 1 -- start start 2024-04-25T11:28:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.198+0000 7f2966e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f29600b4ea0 0x7f29600cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.198+0000 7f2966e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29600d1230 con 0x7f2960144370 2024-04-25T11:28:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.198+0000 7f2966e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29600d06b0 con 0x7f296006c5c0 2024-04-25T11:28:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.198+0000 7f2966e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f29600d0990 con 0x7f29600b4ea0 2024-04-25T11:28:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.198+0000 7f2965000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f29600b4ea0 0x7f29600cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.198+0000 7f2966400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2960144370 0x7f296015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39902/0 (socket says 172.21.15.19:39902) 2024-04-25T11:28:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.198+0000 7f2966400640 1 -- 172.21.15.19:0/3338098755 learned_addr learned my addr 172.21.15.19:0/3338098755 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.199+0000 7f2965000640 1 -- 172.21.15.19:0/3338098755 >> v1:172.21.15.169:6789/0 conn(0x7f296006c5c0 legacy=0x7f29600b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.199+0000 7f2965000640 1 -- 172.21.15.19:0/3338098755 >> v1:172.21.15.19:6789/0 conn(0x7f2960144370 legacy=0x7f296015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.199+0000 7f2965000640 1 -- 172.21.15.19:0/3338098755 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29600d0c70 con 0x7f29600b4ea0 2024-04-25T11:28:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.199+0000 7f295fe00640 1 -- 172.21.15.19:0/3338098755 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 281675123 0 0) 0x7f29600d1230 con 0x7f2960144370 2024-04-25T11:28:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.200+0000 7f2965000640 1 --2- 172.21.15.19:0/3338098755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600b4ea0 0x7f29600cfa70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2950009d90 tx=0x7f295002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404efc8d8212b1f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.200+0000 7f295fe00640 1 -- 172.21.15.19:0/3338098755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950007740 con 0x7f29600b4ea0 2024-04-25T11:28:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.200+0000 7f295fe00640 1 -- 172.21.15.19:0/3338098755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29500078e0 con 0x7f29600b4ea0 2024-04-25T11:28:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.200+0000 7f295fe00640 1 -- 172.21.15.19:0/3338098755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950007cb0 con 0x7f29600b4ea0 2024-04-25T11:28:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.201+0000 7f2966e00640 1 -- 172.21.15.19:0/3338098755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600b4ea0 msgr2=0x7f29600cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.201+0000 7f2966e00640 1 --2- 172.21.15.19:0/3338098755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600b4ea0 0x7f29600cfa70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2950009d90 tx=0x7f295002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.201+0000 7f2966e00640 1 -- 172.21.15.19:0/3338098755 shutdown_connections 2024-04-25T11:28:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.201+0000 7f2966e00640 1 --2- 172.21.15.19:0/3338098755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600b4ea0 0x7f29600cfa70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.201+0000 7f2966e00640 1 -- 172.21.15.19:0/3338098755 >> 172.21.15.19:0/3338098755 conn(0x7f29600d1720 msgr2=0x7f29600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.201+0000 7f2966e00640 1 -- 172.21.15.19:0/3338098755 shutdown_connections 2024-04-25T11:28:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.201+0000 7f2966e00640 1 -- 172.21.15.19:0/3338098755 wait complete. 2024-04-25T11:28:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.203+0000 7f2966e00640 1 Processor -- start 2024-04-25T11:28:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.203+0000 7f2966e00640 1 -- start start 2024-04-25T11:28:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.205+0000 7f2966e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f296006c5c0 0x7f29600c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.205+0000 7f2966e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2960144370 0x7f29600cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.205+0000 7f2966e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600ccb90 0x7f29600ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.205+0000 7f2966e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29600d0450 con 0x7f29600ccb90 2024-04-25T11:28:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.205+0000 7f2966e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f29600d02d0 con 0x7f2960144370 2024-04-25T11:28:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.205+0000 7f2966e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2960147760 con 0x7f296006c5c0 2024-04-25T11:28:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.205+0000 7f2966400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600ccb90 0x7f29600ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600ccb90 0x7f29600ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43136/0 (socket says 172.21.15.19:43136) 2024-04-25T11:28:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 -- 172.21.15.19:0/2054142103 learned_addr learned my addr 172.21.15.19:0/2054142103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2965000640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2960144370 0x7f29600cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2965a00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f296006c5c0 0x7f29600c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 -- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f296006c5c0 msgr2=0x7f29600c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f296006c5c0 0x7f29600c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 -- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2960144370 msgr2=0x7f29600cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2960144370 0x7f29600cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 -- 172.21.15.19:0/2054142103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29600cd4e0 con 0x7f29600ccb90 2024-04-25T11:28:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.206+0000 7f2966400640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600ccb90 0x7f29600ccfa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f295800c410 tx=0x7f295800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.207+0000 7f295d600640 1 -- 172.21.15.19:0/2054142103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580074e0 con 0x7f29600ccb90 2024-04-25T11:28:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.207+0000 7f295d600640 1 -- 172.21.15.19:0/2054142103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29580047e0 con 0x7f29600ccb90 2024-04-25T11:28:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.207+0000 7f295d600640 1 -- 172.21.15.19:0/2054142103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958004bd0 con 0x7f29600ccb90 2024-04-25T11:28:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.207+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29600c71d0 con 0x7f29600ccb90 2024-04-25T11:28:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.207+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29600cbb70 con 0x7f29600ccb90 2024-04-25T11:28:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.209+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29600c73a0 con 0x7f29600ccb90 2024-04-25T11:28:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.210+0000 7f295d600640 1 -- 172.21.15.19:0/2054142103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2958011070 con 0x7f29600ccb90 2024-04-25T11:28:50.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.211+0000 7f295d600640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29440744b0 0x7f29440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.211+0000 7f295d600640 1 -- 172.21.15.19:0/2054142103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2958094ae0 con 0x7f29600ccb90 2024-04-25T11:28:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.212+0000 7f2965a00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29440744b0 0x7f29440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.213+0000 7f2965a00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29440744b0 0x7f29440769c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f29540017b0 tx=0x7f2954009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:50.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.215+0000 7f295d600640 1 -- 172.21.15.19:0/2054142103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f295805ed40 con 0x7f29600ccb90 2024-04-25T11:28:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.419+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2960050740 con 0x7f29600ccb90 2024-04-25T11:28:50.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.423+0000 7f295d600640 1 -- 172.21.15.19:0/2054142103 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16609 (secure 0 0 0) 0x7f2958063040 con 0x7f29600ccb90 2024-04-25T11:28:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:50.414 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:47.266661+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.425+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29440744b0 msgr2=0x7f29440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.425+0000 7f2966e00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29440744b0 0x7f29440769c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f29540017b0 tx=0x7f2954009040 comp rx=0 tx=0).stop 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.425+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600ccb90 msgr2=0x7f29600ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.425+0000 7f2966e00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600ccb90 0x7f29600ccfa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f295800c410 tx=0x7f295800c8d0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.426+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 shutdown_connections 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.426+0000 7f2966e00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29440744b0 0x7f29440769c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.426+0000 7f2966e00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2960144370 0x7f29600cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.426+0000 7f2966e00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29600ccb90 0x7f29600ccfa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.426+0000 7f2966e00640 1 --2- 172.21.15.19:0/2054142103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f296006c5c0 0x7f29600c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.426+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 >> 172.21.15.19:0/2054142103 conn(0x7f29600d1720 msgr2=0x7f296015eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.426+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 shutdown_connections 2024-04-25T11:28:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.427+0000 7f2966e00640 1 -- 172.21.15.19:0/2054142103 wait complete. 2024-04-25T11:28:50.445 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:28:50.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:28:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.730+0000 7f2014e00640 1 Processor -- start 2024-04-25T11:28:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.730+0000 7f2014e00640 1 -- start start 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.732+0000 7f2014e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20100b6a00 0x7f201015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.732+0000 7f2014e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f201006d560 0x7f20100b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.732+0000 7f2014e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20101611e0 con 0x7f201015a2f0 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.732+0000 7f2014e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2010160d50 con 0x7f20100b6a00 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f2014e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f201015a700 con 0x7f201006d560 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20100b6a00 0x7f201015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20100b6a00 0x7f201015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43146/0 (socket says 172.21.15.19:43146) 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200de00640 1 -- 172.21.15.19:0/1642097090 learned_addr learned my addr 172.21.15.19:0/1642097090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200d400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f201015a2f0 0x7f201006ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39908/0 (socket says 172.21.15.19:39908) 2024-04-25T11:28:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200e800640 1 --2- 172.21.15.19:0/1642097090 >> v2:172.21.15.169:3300/0 conn(0x7f201006d560 0x7f20100b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200de00640 1 -- 172.21.15.19:0/1642097090 >> v2:172.21.15.169:3300/0 conn(0x7f201006d560 msgr2=0x7f20100b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200de00640 1 --2- 172.21.15.19:0/1642097090 >> v2:172.21.15.169:3300/0 conn(0x7f201006d560 0x7f20100b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200de00640 1 -- 172.21.15.19:0/1642097090 >> v1:172.21.15.19:6789/0 conn(0x7f201015a2f0 legacy=0x7f201006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200de00640 1 -- 172.21.15.19:0/1642097090 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2010160420 con 0x7f20100b6a00 2024-04-25T11:28:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.733+0000 7f200e800640 1 --2- 172.21.15.19:0/1642097090 >> v2:172.21.15.169:3300/0 conn(0x7f201006d560 0x7f20100b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.734+0000 7f200de00640 1 --2- 172.21.15.19:0/1642097090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f201015d6c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2010160cc0 tx=0x7f200002c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8312460a3d5954a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.735+0000 7f200ca00640 1 -- 172.21.15.19:0/1642097090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200003a070 con 0x7f20100b6a00 2024-04-25T11:28:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.735+0000 7f200ca00640 1 -- 172.21.15.19:0/1642097090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2000032480 con 0x7f20100b6a00 2024-04-25T11:28:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.735+0000 7f200ca00640 1 -- 172.21.15.19:0/1642097090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000032850 con 0x7f20100b6a00 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 -- 172.21.15.19:0/1642097090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 msgr2=0x7f201015d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 --2- 172.21.15.19:0/1642097090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f201015d6c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2010160cc0 tx=0x7f200002c720 comp rx=0 tx=0).stop 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 -- 172.21.15.19:0/1642097090 shutdown_connections 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 --2- 172.21.15.19:0/1642097090 >> v2:172.21.15.169:3300/0 conn(0x7f201006d560 0x7f20100b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 --2- 172.21.15.19:0/1642097090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f201015d6c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 -- 172.21.15.19:0/1642097090 >> 172.21.15.19:0/1642097090 conn(0x7f20100d1ae0 msgr2=0x7f20100d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 -- 172.21.15.19:0/1642097090 shutdown_connections 2024-04-25T11:28:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.736+0000 7f2014e00640 1 -- 172.21.15.19:0/1642097090 wait complete. 2024-04-25T11:28:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.739+0000 7f2014e00640 1 Processor -- start 2024-04-25T11:28:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.739+0000 7f2014e00640 1 -- start start 2024-04-25T11:28:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.741+0000 7f2014e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f201006d560 0x7f2010156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.741+0000 7f2014e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f20101fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.741+0000 7f2014e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201015a2f0 0x7f2010155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.741+0000 7f2014e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20100ce580 con 0x7f20100b6a00 2024-04-25T11:28:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.741+0000 7f2014e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20101609e0 con 0x7f201006d560 2024-04-25T11:28:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.741+0000 7f2014e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20100ce700 con 0x7f201015a2f0 2024-04-25T11:28:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f20101fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f20101fa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43160/0 (socket says 172.21.15.19:43160) 2024-04-25T11:28:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 -- 172.21.15.19:0/4024792030 learned_addr learned my addr 172.21.15.19:0/4024792030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200de00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f201006d560 0x7f2010156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200e800640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201015a2f0 0x7f2010155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 -- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201015a2f0 msgr2=0x7f2010155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201015a2f0 0x7f2010155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 -- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f201006d560 msgr2=0x7f2010156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f201006d560 0x7f2010156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200d400640 1 -- 172.21.15.19:0/4024792030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101576e0 con 0x7f20100b6a00 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.742+0000 7f200e800640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201015a2f0 0x7f2010155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.743+0000 7f200d400640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f20101fa7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c440 tx=0x7f1ffc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.743+0000 7f2006000640 1 -- 172.21.15.19:0/4024792030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0074e0 con 0x7f20100b6a00 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.743+0000 7f2006000640 1 -- 172.21.15.19:0/4024792030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ffc007680 con 0x7f20100b6a00 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.743+0000 7f2006000640 1 -- 172.21.15.19:0/4024792030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc004a20 con 0x7f20100b6a00 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.743+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20101579c0 con 0x7f20100b6a00 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.743+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2010154890 con 0x7f20100b6a00 2024-04-25T11:28:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.745+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2010157450 con 0x7f20100b6a00 2024-04-25T11:28:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.746+0000 7f2006000640 1 -- 172.21.15.19:0/4024792030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ffc01a020 con 0x7f20100b6a00 2024-04-25T11:28:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.747+0000 7f2006000640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ff00744e0 0x7f1ff00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.748+0000 7f2006000640 1 -- 172.21.15.19:0/4024792030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1ffc094a70 con 0x7f20100b6a00 2024-04-25T11:28:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.748+0000 7f200de00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ff00744e0 0x7f1ff00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:50.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.749+0000 7f200de00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ff00744e0 0x7f1ff00769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2000037000 tx=0x7f2000030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:50.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.753+0000 7f2006000640 1 -- 172.21.15.19:0/4024792030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ffc05f160 con 0x7f20100b6a00 2024-04-25T11:28:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.952+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20100d1ef0 con 0x7f1ff00744e0 2024-04-25T11:28:50.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.975+0000 7f2006000640 1 -- 172.21.15.19:0/4024792030 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251900 (secure 0 0 0) 0x7f20100d1ef0 con 0x7f1ff00744e0 2024-04-25T11:28:50.970 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:28:50.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ff00744e0 msgr2=0x7f1ff00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ff00744e0 0x7f1ff00769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2000037000 tx=0x7f2000030000 comp rx=0 tx=0).stop 2024-04-25T11:28:50.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 msgr2=0x7f20101fa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:50.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f20101fa7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00c440 tx=0x7f1ffc00c900 comp rx=0 tx=0).stop 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 shutdown_connections 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ff00744e0 0x7f1ff00769f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f201015a2f0 0x7f2010155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20100b6a00 0x7f20101fa7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 --2- 172.21.15.19:0/4024792030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f201006d560 0x7f2010156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 >> 172.21.15.19:0/4024792030 conn(0x7f20100d1ae0 msgr2=0x7f201006fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 shutdown_connections 2024-04-25T11:28:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:50.982+0000 7f2014e00640 1 -- 172.21.15.19:0/4024792030 wait complete. 2024-04-25T11:28:50.999 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-25T11:28:50.469023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220392,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754328,"statfs":{"total":773094113280,"available":772868431872,"internally_reserved":0,"allocated":4268032,"data_stored":2346932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002808"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280569+0000","last_change":"2024-04-25T11:28:46.302606+0000","last_active":"2024-04-25T11:28:47.280569+0000","last_peered":"2024-04-25T11:28:47.280569+0000","last_clean":"2024-04-25T11:28:47.280569+0000","last_became_active":"2024-04-25T11:28:46.302289+0000","last_became_peered":"2024-04-25T11:28:46.302289+0000","last_unstale":"2024-04-25T11:28:47.280569+0000","last_undegraded":"2024-04-25T11:28:47.280569+0000","last_fullsized":"2024-04-25T11:28:47.280569+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:20.202317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.327717+0000","last_change":"2024-04-25T11:28:46.322282+0000","last_active":"2024-04-25T11:28:46.327717+0000","last_peered":"2024-04-25T11:28:46.327717+0000","last_clean":"2024-04-25T11:28:46.327717+0000","last_became_active":"2024-04-25T11:28:46.321867+0000","last_became_peered":"2024-04-25T11:28:46.321867+0000","last_unstale":"2024-04-25T11:28:46.327717+0000","last_undegraded":"2024-04-25T11:28:46.327717+0000","last_fullsized":"2024-04-25T11:28:46.327717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:53.011440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.308946+0000","last_change":"2024-04-25T11:28:46.298428+0000","last_active":"2024-04-25T11:28:46.308946+0000","last_peered":"2024-04-25T11:28:46.308946+0000","last_clean":"2024-04-25T11:28:46.308946+0000","last_became_active":"2024-04-25T11:28:46.298042+0000","last_became_peered":"2024-04-25T11:28:46.298042+0000","last_unstale":"2024-04-25T11:28:46.308946+0000","last_undegraded":"2024-04-25T11:28:46.308946+0000","last_fullsized":"2024-04-25T11:28:46.308946+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:52.958663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332683+0000","last_change":"2024-04-25T11:28:46.330442+0000","last_active":"2024-04-25T11:28:46.332683+0000","last_peered":"2024-04-25T11:28:46.332683+0000","last_clean":"2024-04-25T11:28:46.332683+0000","last_became_active":"2024-04-25T11:28:46.330076+0000","last_became_peered":"2024-04-25T11:28:46.330076+0000","last_unstale":"2024-04-25T11:28:46.332683+0000","last_undegraded":"2024-04-25T11:28:46.332683+0000","last_fullsized":"2024-04-25T11:28:46.332683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:29.672156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.310728+0000","last_change":"2024-04-25T11:28:46.306819+0000","last_active":"2024-04-25T11:28:46.310728+0000","last_peered":"2024-04-25T11:28:46.310728+0000","last_clean":"2024-04-25T11:28:46.310728+0000","last_became_active":"2024-04-25T11:28:46.306425+0000","last_became_peered":"2024-04-25T11:28:46.306425+0000","last_unstale":"2024-04-25T11:28:46.310728+0000","last_undegraded":"2024-04-25T11:28:46.310728+0000","last_fullsized":"2024-04-25T11:28:46.310728+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:15.284448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281550+0000","last_change":"2024-04-25T11:28:46.308036+0000","last_active":"2024-04-25T11:28:47.281550+0000","last_peered":"2024-04-25T11:28:47.281550+0000","last_clean":"2024-04-25T11:28:47.281550+0000","last_became_active":"2024-04-25T11:28:46.307674+0000","last_became_peered":"2024-04-25T11:28:46.307674+0000","last_unstale":"2024-04-25T11:28:47.281550+0000","last_undegraded":"2024-04-25T11:28:47.281550+0000","last_fullsized":"2024-04-25T11:28:47.281550+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:36.077794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.279965+0000","last_change":"2024-04-25T11:28:46.311355+0000","last_active":"2024-04-25T11:28:47.279965+0000","last_peered":"2024-04-25T11:28:47.279965+0000","last_clean":"2024-04-25T11:28:47.279965+0000","last_became_active":"2024-04-25T11:28:46.310414+0000","last_became_peered":"2024-04-25T11:28:46.310414+0000","last_unstale":"2024-04-25T11:28:47.279965+0000","last_undegraded":"2024-04-25T11:28:47.279965+0000","last_fullsized":"2024-04-25T11:28:47.279965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:24.033667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280234+0000","last_change":"2024-04-25T11:28:46.305771+0000","last_active":"2024-04-25T11:28:47.280234+0000","last_peered":"2024-04-25T11:28:47.280234+0000","last_clean":"2024-04-25T11:28:47.280234+0000","last_became_active":"2024-04-25T11:28:46.303025+0000","last_became_peered":"2024-04-25T11:28:46.303025+0000","last_unstale":"2024-04-25T11:28:47.280234+0000","last_undegraded":"2024-04-25T11:28:47.280234+0000","last_fullsized":"2024-04-25T11:28:47.280234+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:29.672156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.316804+0000","last_change":"2024-04-25T11:28:46.310794+0000","last_active":"2024-04-25T11:28:46.316804+0000","last_peered":"2024-04-25T11:28:46.316804+0000","last_clean":"2024-04-25T11:28:46.316804+0000","last_became_active":"2024-04-25T11:28:46.310475+0000","last_became_peered":"2024-04-25T11:28:46.310475+0000","last_unstale":"2024-04-25T11:28:46.316804+0000","last_undegraded":"2024-04-25T11:28:46.316804+0000","last_fullsized":"2024-04-25T11:28:46.316804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:32.405896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.327872+0000","last_change":"2024-04-25T11:28:46.316155+0000","last_active":"2024-04-25T11:28:46.327872+0000","last_peered":"2024-04-25T11:28:46.327872+0000","last_clean":"2024-04-25T11:28:46.327872+0000","last_became_active":"2024-04-25T11:28:46.315761+0000","last_became_peered":"2024-04-25T11:28:46.315761+0000","last_unstale":"2024-04-25T11:28:46.327872+0000","last_undegraded":"2024-04-25T11:28:46.327872+0000","last_fullsized":"2024-04-25T11:28:46.327872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:43.367845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277575+0000","last_change":"2024-04-25T11:28:46.320651+0000","last_active":"2024-04-25T11:28:47.277575+0000","last_peered":"2024-04-25T11:28:47.277575+0000","last_clean":"2024-04-25T11:28:47.277575+0000","last_became_active":"2024-04-25T11:28:46.320336+0000","last_became_peered":"2024-04-25T11:28:46.320336+0000","last_unstale":"2024-04-25T11:28:47.277575+0000","last_undegraded":"2024-04-25T11:28:47.277575+0000","last_fullsized":"2024-04-25T11:28:47.277575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.693119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.290007+0000","last_change":"2024-04-25T11:28:46.287019+0000","last_active":"2024-04-25T11:28:46.290007+0000","last_peered":"2024-04-25T11:28:46.290007+0000","last_clean":"2024-04-25T11:28:46.290007+0000","last_became_active":"2024-04-25T11:28:46.286700+0000","last_became_peered":"2024-04-25T11:28:46.286700+0000","last_unstale":"2024-04-25T11:28:46.290007+0000","last_undegraded":"2024-04-25T11:28:46.290007+0000","last_fullsized":"2024-04-25T11:28:46.290007+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:34.333832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.331512+0000","last_change":"2024-04-25T11:28:46.329848+0000","last_active":"2024-04-25T11:28:46.331512+0000","last_peered":"2024-04-25T11:28:46.331512+0000","last_clean":"2024-04-25T11:28:46.331512+0000","last_became_active":"2024-04-25T11:28:46.327527+0000","last_became_peered":"2024-04-25T11:28:46.327527+0000","last_unstale":"2024-04-25T11:28:46.331512+0000","last_undegraded":"2024-04-25T11:28:46.331512+0000","last_fullsized":"2024-04-25T11:28:46.331512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:21.347782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329275+0000","last_change":"2024-04-25T11:28:46.313015+0000","last_active":"2024-04-25T11:28:46.329275+0000","last_peered":"2024-04-25T11:28:46.329275+0000","last_clean":"2024-04-25T11:28:46.329275+0000","last_became_active":"2024-04-25T11:28:46.312555+0000","last_became_peered":"2024-04-25T11:28:46.312555+0000","last_unstale":"2024-04-25T11:28:46.329275+0000","last_undegraded":"2024-04-25T11:28:46.329275+0000","last_fullsized":"2024-04-25T11:28:46.329275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.396310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.307957+0000","last_change":"2024-04-25T11:28:46.299941+0000","last_active":"2024-04-25T11:28:46.307957+0000","last_peered":"2024-04-25T11:28:46.307957+0000","last_clean":"2024-04-25T11:28:46.307957+0000","last_became_active":"2024-04-25T11:28:46.299576+0000","last_became_peered":"2024-04-25T11:28:46.299576+0000","last_unstale":"2024-04-25T11:28:46.307957+0000","last_undegraded":"2024-04-25T11:28:46.307957+0000","last_fullsized":"2024-04-25T11:28:46.307957+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:58.755204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.318559+0000","last_change":"2024-04-25T11:28:46.311053+0000","last_active":"2024-04-25T11:28:46.318559+0000","last_peered":"2024-04-25T11:28:46.318559+0000","last_clean":"2024-04-25T11:28:46.318559+0000","last_became_active":"2024-04-25T11:28:46.310356+0000","last_became_peered":"2024-04-25T11:28:46.310356+0000","last_unstale":"2024-04-25T11:28:46.318559+0000","last_undegraded":"2024-04-25T11:28:46.318559+0000","last_fullsized":"2024-04-25T11:28:46.318559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:50.938051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278481+0000","last_change":"2024-04-25T11:28:46.306967+0000","last_active":"2024-04-25T11:28:47.278481+0000","last_peered":"2024-04-25T11:28:47.278481+0000","last_clean":"2024-04-25T11:28:47.278481+0000","last_became_active":"2024-04-25T11:28:46.306653+0000","last_became_peered":"2024-04-25T11:28:46.306653+0000","last_unstale":"2024-04-25T11:28:47.278481+0000","last_undegraded":"2024-04-25T11:28:47.278481+0000","last_fullsized":"2024-04-25T11:28:47.278481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:19.214556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281186+0000","last_change":"2024-04-25T11:28:46.305331+0000","last_active":"2024-04-25T11:28:47.281186+0000","last_peered":"2024-04-25T11:28:47.281186+0000","last_clean":"2024-04-25T11:28:47.281186+0000","last_became_active":"2024-04-25T11:28:46.304948+0000","last_became_peered":"2024-04-25T11:28:46.304948+0000","last_unstale":"2024-04-25T11:28:47.281186+0000","last_undegraded":"2024-04-25T11:28:47.281186+0000","last_fullsized":"2024-04-25T11:28:47.281186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:38.803502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281185+0000","last_change":"2024-04-25T11:28:46.311840+0000","last_active":"2024-04-25T11:28:47.281185+0000","last_peered":"2024-04-25T11:28:47.281185+0000","last_clean":"2024-04-25T11:28:47.281185+0000","last_became_active":"2024-04-25T11:28:46.311146+0000","last_became_peered":"2024-04-25T11:28:46.311146+0000","last_unstale":"2024-04-25T11:28:47.281185+0000","last_undegraded":"2024-04-25T11:28:47.281185+0000","last_fullsized":"2024-04-25T11:28:47.281185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:17.030731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.301170+0000","last_change":"2024-04-25T11:28:46.293701+0000","last_active":"2024-04-25T11:28:46.301170+0000","last_peered":"2024-04-25T11:28:46.301170+0000","last_clean":"2024-04-25T11:28:46.301170+0000","last_became_active":"2024-04-25T11:28:46.292177+0000","last_became_peered":"2024-04-25T11:28:46.292177+0000","last_unstale":"2024-04-25T11:28:46.301170+0000","last_undegraded":"2024-04-25T11:28:46.301170+0000","last_fullsized":"2024-04-25T11:28:46.301170+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:36.077794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.290920+0000","last_change":"2024-04-25T11:28:46.288002+0000","last_active":"2024-04-25T11:28:46.290920+0000","last_peered":"2024-04-25T11:28:46.290920+0000","last_clean":"2024-04-25T11:28:46.290920+0000","last_became_active":"2024-04-25T11:28:46.287672+0000","last_became_peered":"2024-04-25T11:28:46.287672+0000","last_unstale":"2024-04-25T11:28:46.290920+0000","last_undegraded":"2024-04-25T11:28:46.290920+0000","last_fullsized":"2024-04-25T11:28:46.290920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:21.347782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280465+0000","last_change":"2024-04-25T11:28:46.316356+0000","last_active":"2024-04-25T11:28:47.280465+0000","last_peered":"2024-04-25T11:28:47.280465+0000","last_clean":"2024-04-25T11:28:47.280465+0000","last_became_active":"2024-04-25T11:28:46.316005+0000","last_became_peered":"2024-04-25T11:28:46.316005+0000","last_unstale":"2024-04-25T11:28:47.280465+0000","last_undegraded":"2024-04-25T11:28:47.280465+0000","last_fullsized":"2024-04-25T11:28:47.280465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:51.371881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.297577+0000","last_change":"2024-04-25T11:28:46.289410+0000","last_active":"2024-04-25T11:28:46.297577+0000","last_peered":"2024-04-25T11:28:46.297577+0000","last_clean":"2024-04-25T11:28:46.297577+0000","last_became_active":"2024-04-25T11:28:46.286458+0000","last_became_peered":"2024-04-25T11:28:46.286458+0000","last_unstale":"2024-04-25T11:28:46.297577+0000","last_undegraded":"2024-04-25T11:28:46.297577+0000","last_fullsized":"2024-04-25T11:28:46.297577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:17.030731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280344+0000","last_change":"2024-04-25T11:28:46.321486+0000","last_active":"2024-04-25T11:28:47.280344+0000","last_peered":"2024-04-25T11:28:47.280344+0000","last_clean":"2024-04-25T11:28:47.280344+0000","last_became_active":"2024-04-25T11:28:46.321120+0000","last_became_peered":"2024-04-25T11:28:46.321120+0000","last_unstale":"2024-04-25T11:28:47.280344+0000","last_undegraded":"2024-04-25T11:28:47.280344+0000","last_fullsized":"2024-04-25T11:28:47.280344+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:57.256080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329406+0000","last_change":"2024-04-25T11:28:46.322755+0000","last_active":"2024-04-25T11:28:46.329406+0000","last_peered":"2024-04-25T11:28:46.329406+0000","last_clean":"2024-04-25T11:28:46.329406+0000","last_became_active":"2024-04-25T11:28:46.322383+0000","last_became_peered":"2024-04-25T11:28:46.322383+0000","last_unstale":"2024-04-25T11:28:46.329406+0000","last_undegraded":"2024-04-25T11:28:46.329406+0000","last_fullsized":"2024-04-25T11:28:46.329406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:02.769083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329195+0000","last_change":"2024-04-25T11:28:46.316540+0000","last_active":"2024-04-25T11:28:46.329195+0000","last_peered":"2024-04-25T11:28:46.329195+0000","last_clean":"2024-04-25T11:28:46.329195+0000","last_became_active":"2024-04-25T11:28:46.312668+0000","last_became_peered":"2024-04-25T11:28:46.312668+0000","last_unstale":"2024-04-25T11:28:46.329195+0000","last_undegraded":"2024-04-25T11:28:46.329195+0000","last_fullsized":"2024-04-25T11:28:46.329195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:21.347782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.323342+0000","last_change":"2024-04-25T11:28:46.318598+0000","last_active":"2024-04-25T11:28:46.323342+0000","last_peered":"2024-04-25T11:28:46.323342+0000","last_clean":"2024-04-25T11:28:46.323342+0000","last_became_active":"2024-04-25T11:28:46.318271+0000","last_became_peered":"2024-04-25T11:28:46.318271+0000","last_unstale":"2024-04-25T11:28:46.323342+0000","last_undegraded":"2024-04-25T11:28:46.323342+0000","last_fullsized":"2024-04-25T11:28:46.323342+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:53.246076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281393+0000","last_change":"2024-04-25T11:28:46.315551+0000","last_active":"2024-04-25T11:28:47.281393+0000","last_peered":"2024-04-25T11:28:47.281393+0000","last_clean":"2024-04-25T11:28:47.281393+0000","last_became_active":"2024-04-25T11:28:46.315254+0000","last_became_peered":"2024-04-25T11:28:46.315254+0000","last_unstale":"2024-04-25T11:28:47.281393+0000","last_undegraded":"2024-04-25T11:28:47.281393+0000","last_fullsized":"2024-04-25T11:28:47.281393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:35.205793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.326108+0000","last_change":"2024-04-25T11:28:46.320950+0000","last_active":"2024-04-25T11:28:46.326108+0000","last_peered":"2024-04-25T11:28:46.326108+0000","last_clean":"2024-04-25T11:28:46.326108+0000","last_became_active":"2024-04-25T11:28:46.320661+0000","last_became_peered":"2024-04-25T11:28:46.320661+0000","last_unstale":"2024-04-25T11:28:46.326108+0000","last_undegraded":"2024-04-25T11:28:46.326108+0000","last_fullsized":"2024-04-25T11:28:46.326108+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:24.033667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.317243+0000","last_change":"2024-04-25T11:28:46.305125+0000","last_active":"2024-04-25T11:28:46.317243+0000","last_peered":"2024-04-25T11:28:46.317243+0000","last_clean":"2024-04-25T11:28:46.317243+0000","last_became_active":"2024-04-25T11:28:46.304829+0000","last_became_peered":"2024-04-25T11:28:46.304829+0000","last_unstale":"2024-04-25T11:28:46.317243+0000","last_undegraded":"2024-04-25T11:28:46.317243+0000","last_fullsized":"2024-04-25T11:28:46.317243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:02.769083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280144+0000","last_change":"2024-04-25T11:28:46.314283+0000","last_active":"2024-04-25T11:28:47.280144+0000","last_peered":"2024-04-25T11:28:47.280144+0000","last_clean":"2024-04-25T11:28:47.280144+0000","last_became_active":"2024-04-25T11:28:46.311050+0000","last_became_peered":"2024-04-25T11:28:46.311050+0000","last_unstale":"2024-04-25T11:28:47.280144+0000","last_undegraded":"2024-04-25T11:28:47.280144+0000","last_fullsized":"2024-04-25T11:28:47.280144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:43.812507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.275794+0000","last_change":"2024-04-25T11:28:46.309859+0000","last_active":"2024-04-25T11:28:47.275794+0000","last_peered":"2024-04-25T11:28:47.275794+0000","last_clean":"2024-04-25T11:28:47.275794+0000","last_became_active":"2024-04-25T11:28:46.309008+0000","last_became_peered":"2024-04-25T11:28:46.309008+0000","last_unstale":"2024-04-25T11:28:47.275794+0000","last_undegraded":"2024-04-25T11:28:47.275794+0000","last_fullsized":"2024-04-25T11:28:47.275794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:47.127556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.301277+0000","last_change":"2024-04-25T11:28:46.295983+0000","last_active":"2024-04-25T11:28:46.301277+0000","last_peered":"2024-04-25T11:28:46.301277+0000","last_clean":"2024-04-25T11:28:46.301277+0000","last_became_active":"2024-04-25T11:28:46.295535+0000","last_became_peered":"2024-04-25T11:28:46.295535+0000","last_unstale":"2024-04-25T11:28:46.301277+0000","last_undegraded":"2024-04-25T11:28:46.301277+0000","last_fullsized":"2024-04-25T11:28:46.301277+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:02.014028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278845+0000","last_change":"2024-04-25T11:28:45.266930+0000","last_active":"2024-04-25T11:28:47.278845+0000","last_peered":"2024-04-25T11:28:47.278845+0000","last_clean":"2024-04-25T11:28:47.278845+0000","last_became_active":"2024-04-25T11:28:45.266520+0000","last_became_peered":"2024-04-25T11:28:45.266520+0000","last_unstale":"2024-04-25T11:28:47.278845+0000","last_undegraded":"2024-04-25T11:28:47.278845+0000","last_fullsized":"2024-04-25T11:28:47.278845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:08.296718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280770+0000","last_change":"2024-04-25T11:28:46.314511+0000","last_active":"2024-04-25T11:28:47.280770+0000","last_peered":"2024-04-25T11:28:47.280770+0000","last_clean":"2024-04-25T11:28:47.280770+0000","last_became_active":"2024-04-25T11:28:46.313909+0000","last_became_peered":"2024-04-25T11:28:46.313909+0000","last_unstale":"2024-04-25T11:28:47.280770+0000","last_undegraded":"2024-04-25T11:28:47.280770+0000","last_fullsized":"2024-04-25T11:28:47.280770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:46.850279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329228+0000","last_change":"2024-04-25T11:28:45.290759+0000","last_active":"2024-04-25T11:28:46.329228+0000","last_peered":"2024-04-25T11:28:46.329228+0000","last_clean":"2024-04-25T11:28:46.329228+0000","last_became_active":"2024-04-25T11:28:45.290411+0000","last_became_peered":"2024-04-25T11:28:45.290411+0000","last_unstale":"2024-04-25T11:28:46.329228+0000","last_undegraded":"2024-04-25T11:28:46.329228+0000","last_fullsized":"2024-04-25T11:28:46.329228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:37.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281560+0000","last_change":"2024-04-25T11:28:46.321741+0000","last_active":"2024-04-25T11:28:47.281560+0000","last_peered":"2024-04-25T11:28:47.281560+0000","last_clean":"2024-04-25T11:28:47.281560+0000","last_became_active":"2024-04-25T11:28:46.321440+0000","last_became_peered":"2024-04-25T11:28:46.321440+0000","last_unstale":"2024-04-25T11:28:47.281560+0000","last_undegraded":"2024-04-25T11:28:47.281560+0000","last_fullsized":"2024-04-25T11:28:47.281560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:11.647467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.282035+0000","last_change":"2024-04-25T11:28:45.283578+0000","last_active":"2024-04-25T11:28:47.282035+0000","last_peered":"2024-04-25T11:28:47.282035+0000","last_clean":"2024-04-25T11:28:47.282035+0000","last_became_active":"2024-04-25T11:28:45.283288+0000","last_became_peered":"2024-04-25T11:28:45.283288+0000","last_unstale":"2024-04-25T11:28:47.282035+0000","last_undegraded":"2024-04-25T11:28:47.282035+0000","last_fullsized":"2024-04-25T11:28:47.282035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:50.300240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332322+0000","last_change":"2024-04-25T11:28:46.327008+0000","last_active":"2024-04-25T11:28:46.332322+0000","last_peered":"2024-04-25T11:28:46.332322+0000","last_clean":"2024-04-25T11:28:46.332322+0000","last_became_active":"2024-04-25T11:28:46.326656+0000","last_became_peered":"2024-04-25T11:28:46.326656+0000","last_unstale":"2024-04-25T11:28:46.332322+0000","last_undegraded":"2024-04-25T11:28:46.332322+0000","last_fullsized":"2024-04-25T11:28:46.332322+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:50.938051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.331925+0000","last_change":"2024-04-25T11:28:45.289229+0000","last_active":"2024-04-25T11:28:46.331925+0000","last_peered":"2024-04-25T11:28:46.331925+0000","last_clean":"2024-04-25T11:28:46.331925+0000","last_became_active":"2024-04-25T11:28:45.288677+0000","last_became_peered":"2024-04-25T11:28:45.288677+0000","last_unstale":"2024-04-25T11:28:46.331925+0000","last_undegraded":"2024-04-25T11:28:46.331925+0000","last_fullsized":"2024-04-25T11:28:46.331925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:16.025087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.318382+0000","last_change":"2024-04-25T11:28:46.313967+0000","last_active":"2024-04-25T11:28:46.318382+0000","last_peered":"2024-04-25T11:28:46.318382+0000","last_clean":"2024-04-25T11:28:46.318382+0000","last_became_active":"2024-04-25T11:28:46.313682+0000","last_became_peered":"2024-04-25T11:28:46.313682+0000","last_unstale":"2024-04-25T11:28:46.318382+0000","last_undegraded":"2024-04-25T11:28:46.318382+0000","last_fullsized":"2024-04-25T11:28:46.318382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.752011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280968+0000","last_change":"2024-04-25T11:28:45.272785+0000","last_active":"2024-04-25T11:28:47.280968+0000","last_peered":"2024-04-25T11:28:47.280968+0000","last_clean":"2024-04-25T11:28:47.280968+0000","last_became_active":"2024-04-25T11:28:45.272502+0000","last_became_peered":"2024-04-25T11:28:45.272502+0000","last_unstale":"2024-04-25T11:28:47.280968+0000","last_undegraded":"2024-04-25T11:28:47.280968+0000","last_fullsized":"2024-04-25T11:28:47.280968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280849+0000","last_change":"2024-04-25T11:28:46.311260+0000","last_active":"2024-04-25T11:28:47.280849+0000","last_peered":"2024-04-25T11:28:47.280849+0000","last_clean":"2024-04-25T11:28:47.280849+0000","last_became_active":"2024-04-25T11:28:46.310887+0000","last_became_peered":"2024-04-25T11:28:46.310887+0000","last_unstale":"2024-04-25T11:28:47.280849+0000","last_undegraded":"2024-04-25T11:28:47.280849+0000","last_fullsized":"2024-04-25T11:28:47.280849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:09.302361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280500+0000","last_change":"2024-04-25T11:28:45.281862+0000","last_active":"2024-04-25T11:28:47.280500+0000","last_peered":"2024-04-25T11:28:47.280500+0000","last_clean":"2024-04-25T11:28:47.280500+0000","last_became_active":"2024-04-25T11:28:45.281547+0000","last_became_peered":"2024-04-25T11:28:45.281547+0000","last_unstale":"2024-04-25T11:28:47.280500+0000","last_undegraded":"2024-04-25T11:28:47.280500+0000","last_fullsized":"2024-04-25T11:28:47.280500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:31.400252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.302661+0000","last_change":"2024-04-25T11:28:46.292599+0000","last_active":"2024-04-25T11:28:46.302661+0000","last_peered":"2024-04-25T11:28:46.302661+0000","last_clean":"2024-04-25T11:28:46.302661+0000","last_became_active":"2024-04-25T11:28:46.292282+0000","last_became_peered":"2024-04-25T11:28:46.292282+0000","last_unstale":"2024-04-25T11:28:46.302661+0000","last_undegraded":"2024-04-25T11:28:46.302661+0000","last_fullsized":"2024-04-25T11:28:46.302661+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:10.030975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280768+0000","last_change":"2024-04-25T11:28:45.285376+0000","last_active":"2024-04-25T11:28:47.280768+0000","last_peered":"2024-04-25T11:28:47.280768+0000","last_clean":"2024-04-25T11:28:47.280768+0000","last_became_active":"2024-04-25T11:28:45.285057+0000","last_became_peered":"2024-04-25T11:28:45.285057+0000","last_unstale":"2024-04-25T11:28:47.280768+0000","last_undegraded":"2024-04-25T11:28:47.280768+0000","last_fullsized":"2024-04-25T11:28:47.280768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:09.025331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329367+0000","last_change":"2024-04-25T11:28:46.320762+0000","last_active":"2024-04-25T11:28:46.329367+0000","last_peered":"2024-04-25T11:28:46.329367+0000","last_clean":"2024-04-25T11:28:46.329367+0000","last_became_active":"2024-04-25T11:28:46.320433+0000","last_became_peered":"2024-04-25T11:28:46.320433+0000","last_unstale":"2024-04-25T11:28:46.329367+0000","last_undegraded":"2024-04-25T11:28:46.329367+0000","last_fullsized":"2024-04-25T11:28:46.329367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:58.755204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280686+0000","last_change":"2024-04-25T11:28:45.275667+0000","last_active":"2024-04-25T11:28:47.280686+0000","last_peered":"2024-04-25T11:28:47.280686+0000","last_clean":"2024-04-25T11:28:47.280686+0000","last_became_active":"2024-04-25T11:28:45.275315+0000","last_became_peered":"2024-04-25T11:28:45.275315+0000","last_unstale":"2024-04-25T11:28:47.280686+0000","last_undegraded":"2024-04-25T11:28:47.280686+0000","last_fullsized":"2024-04-25T11:28:47.280686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:03.456052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277826+0000","last_change":"2024-04-25T11:28:46.302084+0000","last_active":"2024-04-25T11:28:47.277826+0000","last_peered":"2024-04-25T11:28:47.277826+0000","last_clean":"2024-04-25T11:28:47.277826+0000","last_became_active":"2024-04-25T11:28:46.301771+0000","last_became_peered":"2024-04-25T11:28:46.301771+0000","last_unstale":"2024-04-25T11:28:47.277826+0000","last_undegraded":"2024-04-25T11:28:47.277826+0000","last_fullsized":"2024-04-25T11:28:47.277826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:14.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.294477+0000","last_change":"2024-04-25T11:28:45.272745+0000","last_active":"2024-04-25T11:28:46.294477+0000","last_peered":"2024-04-25T11:28:46.294477+0000","last_clean":"2024-04-25T11:28:46.294477+0000","last_became_active":"2024-04-25T11:28:45.272444+0000","last_became_peered":"2024-04-25T11:28:45.272444+0000","last_unstale":"2024-04-25T11:28:46.294477+0000","last_undegraded":"2024-04-25T11:28:46.294477+0000","last_fullsized":"2024-04-25T11:28:46.294477+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:31.850523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.328347+0000","last_change":"2024-04-25T11:28:46.316314+0000","last_active":"2024-04-25T11:28:46.328347+0000","last_peered":"2024-04-25T11:28:46.328347+0000","last_clean":"2024-04-25T11:28:46.328347+0000","last_became_active":"2024-04-25T11:28:46.312794+0000","last_became_peered":"2024-04-25T11:28:46.312794+0000","last_unstale":"2024-04-25T11:28:46.328347+0000","last_undegraded":"2024-04-25T11:28:46.328347+0000","last_fullsized":"2024-04-25T11:28:46.328347+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.343940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.289287+0000","last_change":"2024-04-25T11:28:45.279928+0000","last_active":"2024-04-25T11:28:46.289287+0000","last_peered":"2024-04-25T11:28:46.289287+0000","last_clean":"2024-04-25T11:28:46.289287+0000","last_became_active":"2024-04-25T11:28:45.279427+0000","last_became_peered":"2024-04-25T11:28:45.279427+0000","last_unstale":"2024-04-25T11:28:46.289287+0000","last_undegraded":"2024-04-25T11:28:46.289287+0000","last_fullsized":"2024-04-25T11:28:46.289287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:49.608625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332659+0000","last_change":"2024-04-25T11:28:46.329960+0000","last_active":"2024-04-25T11:28:46.332659+0000","last_peered":"2024-04-25T11:28:46.332659+0000","last_clean":"2024-04-25T11:28:46.332659+0000","last_became_active":"2024-04-25T11:28:46.329553+0000","last_became_peered":"2024-04-25T11:28:46.329553+0000","last_unstale":"2024-04-25T11:28:46.332659+0000","last_undegraded":"2024-04-25T11:28:46.332659+0000","last_fullsized":"2024-04-25T11:28:46.332659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:43.812507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277173+0000","last_change":"2024-04-25T11:28:45.269784+0000","last_active":"2024-04-25T11:28:47.277173+0000","last_peered":"2024-04-25T11:28:47.277173+0000","last_clean":"2024-04-25T11:28:47.277173+0000","last_became_active":"2024-04-25T11:28:45.269034+0000","last_became_peered":"2024-04-25T11:28:45.269034+0000","last_unstale":"2024-04-25T11:28:47.277173+0000","last_undegraded":"2024-04-25T11:28:47.277173+0000","last_fullsized":"2024-04-25T11:28:47.277173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:50.626568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277036+0000","last_change":"2024-04-25T11:28:46.314974+0000","last_active":"2024-04-25T11:28:47.277036+0000","last_peered":"2024-04-25T11:28:47.277036+0000","last_clean":"2024-04-25T11:28:47.277036+0000","last_became_active":"2024-04-25T11:28:46.314686+0000","last_became_peered":"2024-04-25T11:28:46.314686+0000","last_unstale":"2024-04-25T11:28:47.277036+0000","last_undegraded":"2024-04-25T11:28:47.277036+0000","last_fullsized":"2024-04-25T11:28:47.277036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:15.290099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.336074+0000","last_change":"2024-04-25T11:28:45.291183+0000","last_active":"2024-04-25T11:28:46.336074+0000","last_peered":"2024-04-25T11:28:46.336074+0000","last_clean":"2024-04-25T11:28:46.336074+0000","last_became_active":"2024-04-25T11:28:45.290779+0000","last_became_peered":"2024-04-25T11:28:45.290779+0000","last_unstale":"2024-04-25T11:28:46.336074+0000","last_undegraded":"2024-04-25T11:28:46.336074+0000","last_fullsized":"2024-04-25T11:28:46.336074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:03.456052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.304435+0000","last_change":"2024-04-25T11:28:46.300972+0000","last_active":"2024-04-25T11:28:46.304435+0000","last_peered":"2024-04-25T11:28:46.304435+0000","last_clean":"2024-04-25T11:28:46.304435+0000","last_became_active":"2024-04-25T11:28:46.300495+0000","last_became_peered":"2024-04-25T11:28:46.300495+0000","last_unstale":"2024-04-25T11:28:46.304435+0000","last_undegraded":"2024-04-25T11:28:46.304435+0000","last_fullsized":"2024-04-25T11:28:46.304435+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:43.367845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281090+0000","last_change":"2024-04-25T11:28:45.272277+0000","last_active":"2024-04-25T11:28:47.281090+0000","last_peered":"2024-04-25T11:28:47.281090+0000","last_clean":"2024-04-25T11:28:47.281090+0000","last_became_active":"2024-04-25T11:28:45.272059+0000","last_became_peered":"2024-04-25T11:28:45.272059+0000","last_unstale":"2024-04-25T11:28:47.281090+0000","last_undegraded":"2024-04-25T11:28:47.281090+0000","last_fullsized":"2024-04-25T11:28:47.281090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.050436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332397+0000","last_change":"2024-04-25T11:28:46.327318+0000","last_active":"2024-04-25T11:28:46.332397+0000","last_peered":"2024-04-25T11:28:46.332397+0000","last_clean":"2024-04-25T11:28:46.332397+0000","last_became_active":"2024-04-25T11:28:46.327004+0000","last_became_peered":"2024-04-25T11:28:46.327004+0000","last_unstale":"2024-04-25T11:28:46.332397+0000","last_undegraded":"2024-04-25T11:28:46.332397+0000","last_fullsized":"2024-04-25T11:28:46.332397+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:05.863257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.302492+0000","last_change":"2024-04-25T11:28:45.281317+0000","last_active":"2024-04-25T11:28:46.302492+0000","last_peered":"2024-04-25T11:28:46.302492+0000","last_clean":"2024-04-25T11:28:46.302492+0000","last_became_active":"2024-04-25T11:28:45.279342+0000","last_became_peered":"2024-04-25T11:28:45.279342+0000","last_unstale":"2024-04-25T11:28:46.302492+0000","last_undegraded":"2024-04-25T11:28:46.302492+0000","last_fullsized":"2024-04-25T11:28:46.302492+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:35.072151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280759+0000","last_change":"2024-04-25T11:28:46.314842+0000","last_active":"2024-04-25T11:28:47.280759+0000","last_peered":"2024-04-25T11:28:47.280759+0000","last_clean":"2024-04-25T11:28:47.280759+0000","last_became_active":"2024-04-25T11:28:46.310198+0000","last_became_peered":"2024-04-25T11:28:46.310198+0000","last_unstale":"2024-04-25T11:28:47.280759+0000","last_undegraded":"2024-04-25T11:28:47.280759+0000","last_fullsized":"2024-04-25T11:28:47.280759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:51.632211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.297508+0000","last_change":"2024-04-25T11:28:45.282193+0000","last_active":"2024-04-25T11:28:46.297508+0000","last_peered":"2024-04-25T11:28:46.297508+0000","last_clean":"2024-04-25T11:28:46.297508+0000","last_became_active":"2024-04-25T11:28:45.281826+0000","last_became_peered":"2024-04-25T11:28:45.281826+0000","last_unstale":"2024-04-25T11:28:46.297508+0000","last_undegraded":"2024-04-25T11:28:46.297508+0000","last_fullsized":"2024-04-25T11:28:46.297508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.050436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.276315+0000","last_change":"2024-04-25T11:28:46.308172+0000","last_active":"2024-04-25T11:28:47.276315+0000","last_peered":"2024-04-25T11:28:47.276315+0000","last_clean":"2024-04-25T11:28:47.276315+0000","last_became_active":"2024-04-25T11:28:46.307878+0000","last_became_peered":"2024-04-25T11:28:46.307878+0000","last_unstale":"2024-04-25T11:28:47.276315+0000","last_undegraded":"2024-04-25T11:28:47.276315+0000","last_fullsized":"2024-04-25T11:28:47.276315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:07.559061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280300+0000","last_change":"2024-04-25T11:28:45.286494+0000","last_active":"2024-04-25T11:28:47.280300+0000","last_peered":"2024-04-25T11:28:47.280300+0000","last_clean":"2024-04-25T11:28:47.280300+0000","last_became_active":"2024-04-25T11:28:45.286166+0000","last_became_peered":"2024-04-25T11:28:45.286166+0000","last_unstale":"2024-04-25T11:28:47.280300+0000","last_undegraded":"2024-04-25T11:28:47.280300+0000","last_fullsized":"2024-04-25T11:28:47.280300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:33.328188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278362+0000","last_change":"2024-04-25T11:28:46.319185+0000","last_active":"2024-04-25T11:28:47.278362+0000","last_peered":"2024-04-25T11:28:47.278362+0000","last_clean":"2024-04-25T11:28:47.278362+0000","last_became_active":"2024-04-25T11:28:46.318242+0000","last_became_peered":"2024-04-25T11:28:46.318242+0000","last_unstale":"2024-04-25T11:28:47.278362+0000","last_undegraded":"2024-04-25T11:28:47.278362+0000","last_fullsized":"2024-04-25T11:28:47.278362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:51.404114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280860+0000","last_change":"2024-04-25T11:28:45.287663+0000","last_active":"2024-04-25T11:28:47.280860+0000","last_peered":"2024-04-25T11:28:47.280860+0000","last_clean":"2024-04-25T11:28:47.280860+0000","last_became_active":"2024-04-25T11:28:45.287322+0000","last_became_peered":"2024-04-25T11:28:45.287322+0000","last_unstale":"2024-04-25T11:28:47.280860+0000","last_undegraded":"2024-04-25T11:28:47.280860+0000","last_fullsized":"2024-04-25T11:28:47.280860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:27.801726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.298984+0000","last_change":"2024-04-25T11:28:46.291827+0000","last_active":"2024-04-25T11:28:46.298984+0000","last_peered":"2024-04-25T11:28:46.298984+0000","last_clean":"2024-04-25T11:28:46.298984+0000","last_became_active":"2024-04-25T11:28:46.291420+0000","last_became_peered":"2024-04-25T11:28:46.291420+0000","last_unstale":"2024-04-25T11:28:46.298984+0000","last_undegraded":"2024-04-25T11:28:46.298984+0000","last_fullsized":"2024-04-25T11:28:46.298984+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:28.807369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281048+0000","last_change":"2024-04-25T11:28:45.272075+0000","last_active":"2024-04-25T11:28:47.281048+0000","last_peered":"2024-04-25T11:28:47.281048+0000","last_clean":"2024-04-25T11:28:47.281048+0000","last_became_active":"2024-04-25T11:28:45.271248+0000","last_became_peered":"2024-04-25T11:28:45.271248+0000","last_unstale":"2024-04-25T11:28:47.281048+0000","last_undegraded":"2024-04-25T11:28:47.281048+0000","last_fullsized":"2024-04-25T11:28:47.281048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:16.025087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.314173+0000","last_change":"2024-04-25T11:28:46.311005+0000","last_active":"2024-04-25T11:28:46.314173+0000","last_peered":"2024-04-25T11:28:46.314173+0000","last_clean":"2024-04-25T11:28:46.314173+0000","last_became_active":"2024-04-25T11:28:46.309575+0000","last_became_peered":"2024-04-25T11:28:46.309575+0000","last_unstale":"2024-04-25T11:28:46.314173+0000","last_undegraded":"2024-04-25T11:28:46.314173+0000","last_fullsized":"2024-04-25T11:28:46.314173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:45.954999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277291+0000","last_change":"2024-04-25T11:28:45.278364+0000","last_active":"2024-04-25T11:28:47.277291+0000","last_peered":"2024-04-25T11:28:47.277291+0000","last_clean":"2024-04-25T11:28:47.277291+0000","last_became_active":"2024-04-25T11:28:45.276953+0000","last_became_peered":"2024-04-25T11:28:45.276953+0000","last_unstale":"2024-04-25T11:28:47.277291+0000","last_undegraded":"2024-04-25T11:28:47.277291+0000","last_fullsized":"2024-04-25T11:28:47.277291+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.844636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281961+0000","last_change":"2024-04-25T11:28:46.325237+0000","last_active":"2024-04-25T11:28:47.281961+0000","last_peered":"2024-04-25T11:28:47.281961+0000","last_clean":"2024-04-25T11:28:47.281961+0000","last_became_active":"2024-04-25T11:28:46.324875+0000","last_became_peered":"2024-04-25T11:28:46.324875+0000","last_unstale":"2024-04-25T11:28:47.281961+0000","last_undegraded":"2024-04-25T11:28:47.281961+0000","last_fullsized":"2024-04-25T11:28:47.281961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:43.367845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.284008+0000","last_change":"2024-04-25T11:28:45.286891+0000","last_active":"2024-04-25T11:28:46.284008+0000","last_peered":"2024-04-25T11:28:46.284008+0000","last_clean":"2024-04-25T11:28:46.284008+0000","last_became_active":"2024-04-25T11:28:45.286523+0000","last_became_peered":"2024-04-25T11:28:45.286523+0000","last_unstale":"2024-04-25T11:28:46.284008+0000","last_undegraded":"2024-04-25T11:28:46.284008+0000","last_fullsized":"2024-04-25T11:28:46.284008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:46.338296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.303327+0000","last_change":"2024-04-25T11:28:46.298137+0000","last_active":"2024-04-25T11:28:46.303327+0000","last_peered":"2024-04-25T11:28:46.303327+0000","last_clean":"2024-04-25T11:28:46.303327+0000","last_became_active":"2024-04-25T11:28:46.296872+0000","last_became_peered":"2024-04-25T11:28:46.296872+0000","last_unstale":"2024-04-25T11:28:46.303327+0000","last_undegraded":"2024-04-25T11:28:46.303327+0000","last_fullsized":"2024-04-25T11:28:46.303327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.396310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329067+0000","last_change":"2024-04-25T11:28:45.281197+0000","last_active":"2024-04-25T11:28:46.329067+0000","last_peered":"2024-04-25T11:28:46.329067+0000","last_clean":"2024-04-25T11:28:46.329067+0000","last_became_active":"2024-04-25T11:28:45.280442+0000","last_became_peered":"2024-04-25T11:28:45.280442+0000","last_unstale":"2024-04-25T11:28:46.329067+0000","last_undegraded":"2024-04-25T11:28:46.329067+0000","last_fullsized":"2024-04-25T11:28:46.329067+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:29.553007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.315551+0000","last_change":"2024-04-25T11:28:46.312929+0000","last_active":"2024-04-25T11:28:46.315551+0000","last_peered":"2024-04-25T11:28:46.315551+0000","last_clean":"2024-04-25T11:28:46.315551+0000","last_became_active":"2024-04-25T11:28:46.311167+0000","last_became_peered":"2024-04-25T11:28:46.311167+0000","last_unstale":"2024-04-25T11:28:46.315551+0000","last_undegraded":"2024-04-25T11:28:46.315551+0000","last_fullsized":"2024-04-25T11:28:46.315551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:41.890750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280799+0000","last_change":"2024-04-25T11:28:45.271924+0000","last_active":"2024-04-25T11:28:47.280799+0000","last_peered":"2024-04-25T11:28:47.280799+0000","last_clean":"2024-04-25T11:28:47.280799+0000","last_became_active":"2024-04-25T11:28:45.271265+0000","last_became_peered":"2024-04-25T11:28:45.271265+0000","last_unstale":"2024-04-25T11:28:47.280799+0000","last_undegraded":"2024-04-25T11:28:47.280799+0000","last_fullsized":"2024-04-25T11:28:47.280799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.746368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.324804+0000","last_change":"2024-04-25T11:28:46.318984+0000","last_active":"2024-04-25T11:28:46.324804+0000","last_peered":"2024-04-25T11:28:46.324804+0000","last_clean":"2024-04-25T11:28:46.324804+0000","last_became_active":"2024-04-25T11:28:46.318641+0000","last_became_peered":"2024-04-25T11:28:46.318641+0000","last_unstale":"2024-04-25T11:28:46.324804+0000","last_undegraded":"2024-04-25T11:28:46.324804+0000","last_fullsized":"2024-04-25T11:28:46.324804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.343940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.273224+0000","last_change":"2024-04-25T11:28:45.281461+0000","last_active":"2024-04-25T11:28:46.273224+0000","last_peered":"2024-04-25T11:28:46.273224+0000","last_clean":"2024-04-25T11:28:46.273224+0000","last_became_active":"2024-04-25T11:28:45.281198+0000","last_became_peered":"2024-04-25T11:28:45.281198+0000","last_unstale":"2024-04-25T11:28:46.273224+0000","last_undegraded":"2024-04-25T11:28:46.273224+0000","last_fullsized":"2024-04-25T11:28:46.273224+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:39.706502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.310048+0000","last_change":"2024-04-25T11:28:46.295213+0000","last_active":"2024-04-25T11:28:46.310048+0000","last_peered":"2024-04-25T11:28:46.310048+0000","last_clean":"2024-04-25T11:28:46.310048+0000","last_became_active":"2024-04-25T11:28:46.294914+0000","last_became_peered":"2024-04-25T11:28:46.294914+0000","last_unstale":"2024-04-25T11:28:46.310048+0000","last_undegraded":"2024-04-25T11:28:46.310048+0000","last_fullsized":"2024-04-25T11:28:46.310048+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:51.305884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.275936+0000","last_change":"2024-04-25T11:28:45.276137+0000","last_active":"2024-04-25T11:28:47.275936+0000","last_peered":"2024-04-25T11:28:47.275936+0000","last_clean":"2024-04-25T11:28:47.275936+0000","last_became_active":"2024-04-25T11:28:45.275801+0000","last_became_peered":"2024-04-25T11:28:45.275801+0000","last_unstale":"2024-04-25T11:28:47.275936+0000","last_undegraded":"2024-04-25T11:28:47.275936+0000","last_fullsized":"2024-04-25T11:28:47.275936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:19.196673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281781+0000","last_change":"2024-04-25T11:28:46.318339+0000","last_active":"2024-04-25T11:28:47.281781+0000","last_peered":"2024-04-25T11:28:47.281781+0000","last_clean":"2024-04-25T11:28:47.281781+0000","last_became_active":"2024-04-25T11:28:46.316841+0000","last_became_peered":"2024-04-25T11:28:46.316841+0000","last_unstale":"2024-04-25T11:28:47.281781+0000","last_undegraded":"2024-04-25T11:28:47.281781+0000","last_fullsized":"2024-04-25T11:28:47.281781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:02.014028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.327081+0000","last_change":"2024-04-25T11:28:45.286372+0000","last_active":"2024-04-25T11:28:46.327081+0000","last_peered":"2024-04-25T11:28:46.327081+0000","last_clean":"2024-04-25T11:28:46.327081+0000","last_became_active":"2024-04-25T11:28:45.286102+0000","last_became_peered":"2024-04-25T11:28:45.286102+0000","last_unstale":"2024-04-25T11:28:46.327081+0000","last_undegraded":"2024-04-25T11:28:46.327081+0000","last_fullsized":"2024-04-25T11:28:46.327081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:37.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278810+0000","last_change":"2024-04-25T11:28:46.304246+0000","last_active":"2024-04-25T11:28:47.278810+0000","last_peered":"2024-04-25T11:28:47.278810+0000","last_clean":"2024-04-25T11:28:47.278810+0000","last_became_active":"2024-04-25T11:28:46.303958+0000","last_became_peered":"2024-04-25T11:28:46.303958+0000","last_unstale":"2024-04-25T11:28:47.278810+0000","last_undegraded":"2024-04-25T11:28:47.278810+0000","last_fullsized":"2024-04-25T11:28:47.278810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.154062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.289749+0000","last_change":"2024-04-25T11:28:45.280023+0000","last_active":"2024-04-25T11:28:46.289749+0000","last_peered":"2024-04-25T11:28:46.289749+0000","last_clean":"2024-04-25T11:28:46.289749+0000","last_became_active":"2024-04-25T11:28:45.277502+0000","last_became_peered":"2024-04-25T11:28:45.277502+0000","last_unstale":"2024-04-25T11:28:46.289749+0000","last_undegraded":"2024-04-25T11:28:46.289749+0000","last_fullsized":"2024-04-25T11:28:46.289749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:31.118311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281815+0000","last_change":"2024-04-25T11:28:46.320847+0000","last_active":"2024-04-25T11:28:47.281815+0000","last_peered":"2024-04-25T11:28:47.281815+0000","last_clean":"2024-04-25T11:28:47.281815+0000","last_became_active":"2024-04-25T11:28:46.320509+0000","last_became_peered":"2024-04-25T11:28:46.320509+0000","last_unstale":"2024-04-25T11:28:47.281815+0000","last_undegraded":"2024-04-25T11:28:47.281815+0000","last_fullsized":"2024-04-25T11:28:47.281815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.396310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.279740+0000","last_change":"2024-04-25T11:28:45.278672+0000","last_active":"2024-04-25T11:28:46.279740+0000","last_peered":"2024-04-25T11:28:46.279740+0000","last_clean":"2024-04-25T11:28:46.279740+0000","last_became_active":"2024-04-25T11:28:45.278310+0000","last_became_peered":"2024-04-25T11:28:45.278310+0000","last_unstale":"2024-04-25T11:28:46.279740+0000","last_undegraded":"2024-04-25T11:28:46.279740+0000","last_fullsized":"2024-04-25T11:28:46.279740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.050436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.328802+0000","last_change":"2024-04-25T11:28:46.317098+0000","last_active":"2024-04-25T11:28:46.328802+0000","last_peered":"2024-04-25T11:28:46.328802+0000","last_clean":"2024-04-25T11:28:46.328802+0000","last_became_active":"2024-04-25T11:28:46.312251+0000","last_became_peered":"2024-04-25T11:28:46.312251+0000","last_unstale":"2024-04-25T11:28:46.328802+0000","last_undegraded":"2024-04-25T11:28:46.328802+0000","last_fullsized":"2024-04-25T11:28:46.328802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:05.863257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.279728+0000","last_change":"2024-04-25T11:28:45.286410+0000","last_active":"2024-04-25T11:28:46.279728+0000","last_peered":"2024-04-25T11:28:46.279728+0000","last_clean":"2024-04-25T11:28:46.279728+0000","last_became_active":"2024-04-25T11:28:45.283400+0000","last_became_peered":"2024-04-25T11:28:45.283400+0000","last_unstale":"2024-04-25T11:28:46.279728+0000","last_undegraded":"2024-04-25T11:28:46.279728+0000","last_fullsized":"2024-04-25T11:28:46.279728+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332351+0000","last_change":"2024-04-25T11:28:46.327082+0000","last_active":"2024-04-25T11:28:46.332351+0000","last_peered":"2024-04-25T11:28:46.332351+0000","last_clean":"2024-04-25T11:28:46.332351+0000","last_became_active":"2024-04-25T11:28:46.326448+0000","last_became_peered":"2024-04-25T11:28:46.326448+0000","last_unstale":"2024-04-25T11:28:46.332351+0000","last_undegraded":"2024-04-25T11:28:46.332351+0000","last_fullsized":"2024-04-25T11:28:46.332351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:21.792690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.279283+0000","last_change":"2024-04-25T11:28:45.284540+0000","last_active":"2024-04-25T11:28:46.279283+0000","last_peered":"2024-04-25T11:28:46.279283+0000","last_clean":"2024-04-25T11:28:46.279283+0000","last_became_active":"2024-04-25T11:28:45.284234+0000","last_became_peered":"2024-04-25T11:28:45.284234+0000","last_unstale":"2024-04-25T11:28:46.279283+0000","last_undegraded":"2024-04-25T11:28:46.279283+0000","last_fullsized":"2024-04-25T11:28:46.279283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.746368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332536+0000","last_change":"2024-04-25T11:28:46.328826+0000","last_active":"2024-04-25T11:28:46.332536+0000","last_peered":"2024-04-25T11:28:46.332536+0000","last_clean":"2024-04-25T11:28:46.332536+0000","last_became_active":"2024-04-25T11:28:46.328552+0000","last_became_peered":"2024-04-25T11:28:46.328552+0000","last_unstale":"2024-04-25T11:28:46.332536+0000","last_undegraded":"2024-04-25T11:28:46.332536+0000","last_fullsized":"2024-04-25T11:28:46.332536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:24.033667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280528+0000","last_change":"2024-04-25T11:28:45.280244+0000","last_active":"2024-04-25T11:28:47.280528+0000","last_peered":"2024-04-25T11:28:47.280528+0000","last_clean":"2024-04-25T11:28:47.280528+0000","last_became_active":"2024-04-25T11:28:45.279399+0000","last_became_peered":"2024-04-25T11:28:45.279399+0000","last_unstale":"2024-04-25T11:28:47.280528+0000","last_undegraded":"2024-04-25T11:28:47.280528+0000","last_fullsized":"2024-04-25T11:28:47.280528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:37.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.276490+0000","last_change":"2024-04-25T11:28:46.304874+0000","last_active":"2024-04-25T11:28:47.276490+0000","last_peered":"2024-04-25T11:28:47.276490+0000","last_clean":"2024-04-25T11:28:47.276490+0000","last_became_active":"2024-04-25T11:28:46.304127+0000","last_became_peered":"2024-04-25T11:28:46.304127+0000","last_unstale":"2024-04-25T11:28:47.276490+0000","last_undegraded":"2024-04-25T11:28:47.276490+0000","last_fullsized":"2024-04-25T11:28:47.276490+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.508594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.311967+0000","last_change":"2024-04-25T11:28:45.284257+0000","last_active":"2024-04-25T11:28:46.311967+0000","last_peered":"2024-04-25T11:28:46.311967+0000","last_clean":"2024-04-25T11:28:46.311967+0000","last_became_active":"2024-04-25T11:28:45.283952+0000","last_became_peered":"2024-04-25T11:28:45.283952+0000","last_unstale":"2024-04-25T11:28:46.311967+0000","last_undegraded":"2024-04-25T11:28:46.311967+0000","last_fullsized":"2024-04-25T11:28:46.311967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:01.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.319157+0000","last_change":"2024-04-25T11:28:46.311260+0000","last_active":"2024-04-25T11:28:46.319157+0000","last_peered":"2024-04-25T11:28:46.319157+0000","last_clean":"2024-04-25T11:28:46.319157+0000","last_became_active":"2024-04-25T11:28:46.310951+0000","last_became_peered":"2024-04-25T11:28:46.310951+0000","last_unstale":"2024-04-25T11:28:46.319157+0000","last_undegraded":"2024-04-25T11:28:46.319157+0000","last_fullsized":"2024-04-25T11:28:46.319157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:21.792690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277406+0000","last_change":"2024-04-25T11:28:45.274548+0000","last_active":"2024-04-25T11:28:47.277406+0000","last_peered":"2024-04-25T11:28:47.277406+0000","last_clean":"2024-04-25T11:28:47.277406+0000","last_became_active":"2024-04-25T11:28:45.274222+0000","last_became_peered":"2024-04-25T11:28:45.274222+0000","last_unstale":"2024-04-25T11:28:47.277406+0000","last_undegraded":"2024-04-25T11:28:47.277406+0000","last_fullsized":"2024-04-25T11:28:47.277406+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:34.200150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":132,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.273171+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:28:46.273171+0000","last_peered":"2024-04-25T11:28:46.273171+0000","last_clean":"2024-04-25T11:28:46.273171+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:28:46.273171+0000","last_undegraded":"2024-04-25T11:28:46.273171+0000","last_fullsized":"2024-04-25T11:28:46.273171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":146806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":145948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":735610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":736122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:28:51.003 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:28:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.340+0000 7f0caae00640 1 Processor -- start 2024-04-25T11:28:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.340+0000 7f0caae00640 1 -- start start 2024-04-25T11:28:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.342+0000 7f0caae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0ca40cf2f0 0x7f0ca40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0caae00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca40b74d0 con 0x7f0ca415a810 2024-04-25T11:28:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0caae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca40b77b0 con 0x7f0ca41608d0 2024-04-25T11:28:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0caae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca406f870 con 0x7f0ca40cf2f0 2024-04-25T11:28:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0ca9000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0ca415a810 0x7f0ca40b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39918/0 (socket says 172.21.15.19:39918) 2024-04-25T11:28:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0ca9000640 1 -- 172.21.15.19:0/1635346464 learned_addr learned my addr 172.21.15.19:0/1635346464 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994747044 0 0) 0x7f0ca40b74d0 con 0x7f0ca415a810 2024-04-25T11:28:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0ca9a00640 1 --2- 172.21.15.19:0/1635346464 >> v2:172.21.15.169:3300/0 conn(0x7f0ca40cf2f0 0x7f0ca40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.343+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c8c003520 con 0x7f0ca415a810 2024-04-25T11:28:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4140495291 0 0) 0x7f0c8c003520 con 0x7f0ca415a810 2024-04-25T11:28:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 >> v2:172.21.15.169:3300/0 conn(0x7f0ca40cf2f0 msgr2=0x7f0ca40b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 --2- 172.21.15.19:0/1635346464 >> v2:172.21.15.169:3300/0 conn(0x7f0ca40cf2f0 0x7f0ca40b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 >> v1:172.21.15.169:6789/0 conn(0x7f0ca41608d0 legacy=0x7f0ca4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca406f220 con 0x7f0ca415a810 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f0c90001670 con 0x7f0ca415a810 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 >> v1:172.21.15.19:6789/0 conn(0x7f0ca415a810 legacy=0x7f0ca40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c003a00 0x7f0c8c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c8c004800 0x7f0c8c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c005220 0x7f0c8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca406f870 con 0x7f0c8c005220 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c00c1b0 con 0x7f0c8c003a00 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.344+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c00c3b0 con 0x7f0c8c004800 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0caa400640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c005220 0x7f0c8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0ca9a00640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c003a00 0x7f0c8c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0ca9000640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c8c004800 0x7f0c8c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0caa400640 1 -- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c8c004800 msgr2=0x7f0c8c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0caa400640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c8c004800 0x7f0c8c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0caa400640 1 -- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c003a00 msgr2=0x7f0c8c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0caa400640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c003a00 0x7f0c8c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.345+0000 7f0caa400640 1 -- 172.21.15.19:0/1635346464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c003520 con 0x7f0c8c005220 2024-04-25T11:28:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.346+0000 7f0caa400640 1 -- 172.21.15.19:0/1635346464 reap_dead start 2024-04-25T11:28:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.347+0000 7f0caa400640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c005220 0x7f0c8c00b9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c001e20 tx=0x7f0c8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66c91bca9f9547b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.347+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c001a70 con 0x7f0c8c005220 2024-04-25T11:28:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.347+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9c02ae40 con 0x7f0c8c005220 2024-04-25T11:28:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.347+0000 7f0ca3e00640 1 -- 172.21.15.19:0/1635346464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c003b90 con 0x7f0c8c005220 2024-04-25T11:28:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.348+0000 7f0caae00640 1 -- 172.21.15.19:0/1635346464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c005220 msgr2=0x7f0c8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.348+0000 7f0caae00640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c005220 0x7f0c8c00b9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c001e20 tx=0x7f0c8c005830 comp rx=0 tx=0).stop 2024-04-25T11:28:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.348+0000 7f0caae00640 1 -- 172.21.15.19:0/1635346464 shutdown_connections 2024-04-25T11:28:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.348+0000 7f0caae00640 1 --2- 172.21.15.19:0/1635346464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c005220 0x7f0c8c00b9c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.348+0000 7f0caae00640 1 -- 172.21.15.19:0/1635346464 >> 172.21.15.19:0/1635346464 conn(0x7f0ca40d1720 msgr2=0x7f0ca40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.348+0000 7f0caae00640 1 -- 172.21.15.19:0/1635346464 shutdown_connections 2024-04-25T11:28:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.348+0000 7f0caae00640 1 -- 172.21.15.19:0/1635346464 wait complete. 2024-04-25T11:28:51.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.351+0000 7f0caae00640 1 Processor -- start 2024-04-25T11:28:51.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.351+0000 7f0caae00640 1 -- start start 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.353+0000 7f0caae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c005220 0x7f0ca40ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.353+0000 7f0caae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca40aa410 0x7f0ca40aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.353+0000 7f0caae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ca4156340 0x7f0ca4156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0caae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40b6e10 con 0x7f0ca40aa410 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0caae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca415ac20 con 0x7f0c8c005220 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0caae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca40b6f90 con 0x7f0ca4156340 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca40aa410 0x7f0ca40aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca40aa410 0x7f0ca40aa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43180/0 (socket says 172.21.15.19:43180) 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 -- 172.21.15.19:0/1115767574 learned_addr learned my addr 172.21.15.19:0/1115767574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9a00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c005220 0x7f0ca40ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0caa400640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ca4156340 0x7f0ca4156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 -- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ca4156340 msgr2=0x7f0ca4156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ca4156340 0x7f0ca4156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 -- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c005220 msgr2=0x7f0ca40ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c005220 0x7f0ca40ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.354+0000 7f0ca9000640 1 -- 172.21.15.19:0/1115767574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca415fcd0 con 0x7f0ca40aa410 2024-04-25T11:28:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.355+0000 7f0ca9000640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca40aa410 0x7f0ca40aa820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e7b0 tx=0x7f0c9000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.355+0000 7f0ca1600640 1 -- 172.21.15.19:0/1115767574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000ee60 con 0x7f0ca40aa410 2024-04-25T11:28:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.355+0000 7f0ca1600640 1 -- 172.21.15.19:0/1115767574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c90018070 con 0x7f0ca40aa410 2024-04-25T11:28:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.355+0000 7f0ca1600640 1 -- 172.21.15.19:0/1115767574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000cb10 con 0x7f0ca40aa410 2024-04-25T11:28:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.355+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca415ffb0 con 0x7f0ca40aa410 2024-04-25T11:28:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.355+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca4160570 con 0x7f0ca40aa410 2024-04-25T11:28:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.357+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca415fa40 con 0x7f0ca40aa410 2024-04-25T11:28:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.358+0000 7f0ca1600640 1 -- 172.21.15.19:0/1115767574 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c9000ccb0 con 0x7f0ca40aa410 2024-04-25T11:28:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.359+0000 7f0ca1600640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c840744b0 0x7f0c840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.359+0000 7f0ca1600640 1 -- 172.21.15.19:0/1115767574 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0c90095130 con 0x7f0ca40aa410 2024-04-25T11:28:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.359+0000 7f0ca9a00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c840744b0 0x7f0c840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.361+0000 7f0ca9a00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c840744b0 0x7f0c840769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009990 tx=0x7f0c98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.364+0000 7f0ca1600640 1 -- 172.21.15.19:0/1115767574 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c9005f8b0 con 0x7f0ca40aa410 2024-04-25T11:28:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.560+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ca4050740 con 0x7f0c840744b0 2024-04-25T11:28:51.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.580+0000 7f0ca1600640 1 -- 172.21.15.19:0/1115767574 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251900 (secure 0 0 0) 0x7f0ca4050740 con 0x7f0c840744b0 2024-04-25T11:28:51.575 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:51.576 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:28:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c840744b0 msgr2=0x7f0c840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c840744b0 0x7f0c840769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009990 tx=0x7f0c98008040 comp rx=0 tx=0).stop 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca40aa410 msgr2=0x7f0ca40aa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca40aa410 0x7f0ca40aa820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e7b0 tx=0x7f0c9000ec70 comp rx=0 tx=0).stop 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 shutdown_connections 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c840744b0 0x7f0c840769c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ca4156340 0x7f0ca4156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ca40aa410 0x7f0ca40aa820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 --2- 172.21.15.19:0/1115767574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8c005220 0x7f0ca40ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.585+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 >> 172.21.15.19:0/1115767574 conn(0x7f0ca40d1720 msgr2=0x7f0ca4159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.586+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 shutdown_connections 2024-04-25T11:28:51.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.586+0000 7f0caae00640 1 -- 172.21.15.19:0/1115767574 wait complete. 2024-04-25T11:28:51.600 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-25T11:28:50.469023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220392,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754328,"statfs":{"total":773094113280,"available":772868431872,"internally_reserved":0,"allocated":4268032,"data_stored":2346932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002808"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280569+0000","last_change":"2024-04-25T11:28:46.302606+0000","last_active":"2024-04-25T11:28:47.280569+0000","last_peered":"2024-04-25T11:28:47.280569+0000","last_clean":"2024-04-25T11:28:47.280569+0000","last_became_active":"2024-04-25T11:28:46.302289+0000","last_became_peered":"2024-04-25T11:28:46.302289+0000","last_unstale":"2024-04-25T11:28:47.280569+0000","last_undegraded":"2024-04-25T11:28:47.280569+0000","last_fullsized":"2024-04-25T11:28:47.280569+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:20.202317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.327717+0000","last_change":"2024-04-25T11:28:46.322282+0000","last_active":"2024-04-25T11:28:46.327717+0000","last_peered":"2024-04-25T11:28:46.327717+0000","last_clean":"2024-04-25T11:28:46.327717+0000","last_became_active":"2024-04-25T11:28:46.321867+0000","last_became_peered":"2024-04-25T11:28:46.321867+0000","last_unstale":"2024-04-25T11:28:46.327717+0000","last_undegraded":"2024-04-25T11:28:46.327717+0000","last_fullsized":"2024-04-25T11:28:46.327717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:53.011440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.308946+0000","last_change":"2024-04-25T11:28:46.298428+0000","last_active":"2024-04-25T11:28:46.308946+0000","last_peered":"2024-04-25T11:28:46.308946+0000","last_clean":"2024-04-25T11:28:46.308946+0000","last_became_active":"2024-04-25T11:28:46.298042+0000","last_became_peered":"2024-04-25T11:28:46.298042+0000","last_unstale":"2024-04-25T11:28:46.308946+0000","last_undegraded":"2024-04-25T11:28:46.308946+0000","last_fullsized":"2024-04-25T11:28:46.308946+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:52.958663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332683+0000","last_change":"2024-04-25T11:28:46.330442+0000","last_active":"2024-04-25T11:28:46.332683+0000","last_peered":"2024-04-25T11:28:46.332683+0000","last_clean":"2024-04-25T11:28:46.332683+0000","last_became_active":"2024-04-25T11:28:46.330076+0000","last_became_peered":"2024-04-25T11:28:46.330076+0000","last_unstale":"2024-04-25T11:28:46.332683+0000","last_undegraded":"2024-04-25T11:28:46.332683+0000","last_fullsized":"2024-04-25T11:28:46.332683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:29.672156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.310728+0000","last_change":"2024-04-25T11:28:46.306819+0000","last_active":"2024-04-25T11:28:46.310728+0000","last_peered":"2024-04-25T11:28:46.310728+0000","last_clean":"2024-04-25T11:28:46.310728+0000","last_became_active":"2024-04-25T11:28:46.306425+0000","last_became_peered":"2024-04-25T11:28:46.306425+0000","last_unstale":"2024-04-25T11:28:46.310728+0000","last_undegraded":"2024-04-25T11:28:46.310728+0000","last_fullsized":"2024-04-25T11:28:46.310728+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:15.284448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281550+0000","last_change":"2024-04-25T11:28:46.308036+0000","last_active":"2024-04-25T11:28:47.281550+0000","last_peered":"2024-04-25T11:28:47.281550+0000","last_clean":"2024-04-25T11:28:47.281550+0000","last_became_active":"2024-04-25T11:28:46.307674+0000","last_became_peered":"2024-04-25T11:28:46.307674+0000","last_unstale":"2024-04-25T11:28:47.281550+0000","last_undegraded":"2024-04-25T11:28:47.281550+0000","last_fullsized":"2024-04-25T11:28:47.281550+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:36.077794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.279965+0000","last_change":"2024-04-25T11:28:46.311355+0000","last_active":"2024-04-25T11:28:47.279965+0000","last_peered":"2024-04-25T11:28:47.279965+0000","last_clean":"2024-04-25T11:28:47.279965+0000","last_became_active":"2024-04-25T11:28:46.310414+0000","last_became_peered":"2024-04-25T11:28:46.310414+0000","last_unstale":"2024-04-25T11:28:47.279965+0000","last_undegraded":"2024-04-25T11:28:47.279965+0000","last_fullsized":"2024-04-25T11:28:47.279965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:24.033667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280234+0000","last_change":"2024-04-25T11:28:46.305771+0000","last_active":"2024-04-25T11:28:47.280234+0000","last_peered":"2024-04-25T11:28:47.280234+0000","last_clean":"2024-04-25T11:28:47.280234+0000","last_became_active":"2024-04-25T11:28:46.303025+0000","last_became_peered":"2024-04-25T11:28:46.303025+0000","last_unstale":"2024-04-25T11:28:47.280234+0000","last_undegraded":"2024-04-25T11:28:47.280234+0000","last_fullsized":"2024-04-25T11:28:47.280234+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:29.672156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.316804+0000","last_change":"2024-04-25T11:28:46.310794+0000","last_active":"2024-04-25T11:28:46.316804+0000","last_peered":"2024-04-25T11:28:46.316804+0000","last_clean":"2024-04-25T11:28:46.316804+0000","last_became_active":"2024-04-25T11:28:46.310475+0000","last_became_peered":"2024-04-25T11:28:46.310475+0000","last_unstale":"2024-04-25T11:28:46.316804+0000","last_undegraded":"2024-04-25T11:28:46.316804+0000","last_fullsized":"2024-04-25T11:28:46.316804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:32.405896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.327872+0000","last_change":"2024-04-25T11:28:46.316155+0000","last_active":"2024-04-25T11:28:46.327872+0000","last_peered":"2024-04-25T11:28:46.327872+0000","last_clean":"2024-04-25T11:28:46.327872+0000","last_became_active":"2024-04-25T11:28:46.315761+0000","last_became_peered":"2024-04-25T11:28:46.315761+0000","last_unstale":"2024-04-25T11:28:46.327872+0000","last_undegraded":"2024-04-25T11:28:46.327872+0000","last_fullsized":"2024-04-25T11:28:46.327872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:43.367845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277575+0000","last_change":"2024-04-25T11:28:46.320651+0000","last_active":"2024-04-25T11:28:47.277575+0000","last_peered":"2024-04-25T11:28:47.277575+0000","last_clean":"2024-04-25T11:28:47.277575+0000","last_became_active":"2024-04-25T11:28:46.320336+0000","last_became_peered":"2024-04-25T11:28:46.320336+0000","last_unstale":"2024-04-25T11:28:47.277575+0000","last_undegraded":"2024-04-25T11:28:47.277575+0000","last_fullsized":"2024-04-25T11:28:47.277575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:05.693119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.290007+0000","last_change":"2024-04-25T11:28:46.287019+0000","last_active":"2024-04-25T11:28:46.290007+0000","last_peered":"2024-04-25T11:28:46.290007+0000","last_clean":"2024-04-25T11:28:46.290007+0000","last_became_active":"2024-04-25T11:28:46.286700+0000","last_became_peered":"2024-04-25T11:28:46.286700+0000","last_unstale":"2024-04-25T11:28:46.290007+0000","last_undegraded":"2024-04-25T11:28:46.290007+0000","last_fullsized":"2024-04-25T11:28:46.290007+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:34.333832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.331512+0000","last_change":"2024-04-25T11:28:46.329848+0000","last_active":"2024-04-25T11:28:46.331512+0000","last_peered":"2024-04-25T11:28:46.331512+0000","last_clean":"2024-04-25T11:28:46.331512+0000","last_became_active":"2024-04-25T11:28:46.327527+0000","last_became_peered":"2024-04-25T11:28:46.327527+0000","last_unstale":"2024-04-25T11:28:46.331512+0000","last_undegraded":"2024-04-25T11:28:46.331512+0000","last_fullsized":"2024-04-25T11:28:46.331512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:21.347782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329275+0000","last_change":"2024-04-25T11:28:46.313015+0000","last_active":"2024-04-25T11:28:46.329275+0000","last_peered":"2024-04-25T11:28:46.329275+0000","last_clean":"2024-04-25T11:28:46.329275+0000","last_became_active":"2024-04-25T11:28:46.312555+0000","last_became_peered":"2024-04-25T11:28:46.312555+0000","last_unstale":"2024-04-25T11:28:46.329275+0000","last_undegraded":"2024-04-25T11:28:46.329275+0000","last_fullsized":"2024-04-25T11:28:46.329275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.396310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.307957+0000","last_change":"2024-04-25T11:28:46.299941+0000","last_active":"2024-04-25T11:28:46.307957+0000","last_peered":"2024-04-25T11:28:46.307957+0000","last_clean":"2024-04-25T11:28:46.307957+0000","last_became_active":"2024-04-25T11:28:46.299576+0000","last_became_peered":"2024-04-25T11:28:46.299576+0000","last_unstale":"2024-04-25T11:28:46.307957+0000","last_undegraded":"2024-04-25T11:28:46.307957+0000","last_fullsized":"2024-04-25T11:28:46.307957+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:58.755204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.318559+0000","last_change":"2024-04-25T11:28:46.311053+0000","last_active":"2024-04-25T11:28:46.318559+0000","last_peered":"2024-04-25T11:28:46.318559+0000","last_clean":"2024-04-25T11:28:46.318559+0000","last_became_active":"2024-04-25T11:28:46.310356+0000","last_became_peered":"2024-04-25T11:28:46.310356+0000","last_unstale":"2024-04-25T11:28:46.318559+0000","last_undegraded":"2024-04-25T11:28:46.318559+0000","last_fullsized":"2024-04-25T11:28:46.318559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:50.938051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278481+0000","last_change":"2024-04-25T11:28:46.306967+0000","last_active":"2024-04-25T11:28:47.278481+0000","last_peered":"2024-04-25T11:28:47.278481+0000","last_clean":"2024-04-25T11:28:47.278481+0000","last_became_active":"2024-04-25T11:28:46.306653+0000","last_became_peered":"2024-04-25T11:28:46.306653+0000","last_unstale":"2024-04-25T11:28:47.278481+0000","last_undegraded":"2024-04-25T11:28:47.278481+0000","last_fullsized":"2024-04-25T11:28:47.278481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:19.214556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281186+0000","last_change":"2024-04-25T11:28:46.305331+0000","last_active":"2024-04-25T11:28:47.281186+0000","last_peered":"2024-04-25T11:28:47.281186+0000","last_clean":"2024-04-25T11:28:47.281186+0000","last_became_active":"2024-04-25T11:28:46.304948+0000","last_became_peered":"2024-04-25T11:28:46.304948+0000","last_unstale":"2024-04-25T11:28:47.281186+0000","last_undegraded":"2024-04-25T11:28:47.281186+0000","last_fullsized":"2024-04-25T11:28:47.281186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:38.803502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281185+0000","last_change":"2024-04-25T11:28:46.311840+0000","last_active":"2024-04-25T11:28:47.281185+0000","last_peered":"2024-04-25T11:28:47.281185+0000","last_clean":"2024-04-25T11:28:47.281185+0000","last_became_active":"2024-04-25T11:28:46.311146+0000","last_became_peered":"2024-04-25T11:28:46.311146+0000","last_unstale":"2024-04-25T11:28:47.281185+0000","last_undegraded":"2024-04-25T11:28:47.281185+0000","last_fullsized":"2024-04-25T11:28:47.281185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:17.030731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.301170+0000","last_change":"2024-04-25T11:28:46.293701+0000","last_active":"2024-04-25T11:28:46.301170+0000","last_peered":"2024-04-25T11:28:46.301170+0000","last_clean":"2024-04-25T11:28:46.301170+0000","last_became_active":"2024-04-25T11:28:46.292177+0000","last_became_peered":"2024-04-25T11:28:46.292177+0000","last_unstale":"2024-04-25T11:28:46.301170+0000","last_undegraded":"2024-04-25T11:28:46.301170+0000","last_fullsized":"2024-04-25T11:28:46.301170+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:36.077794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.290920+0000","last_change":"2024-04-25T11:28:46.288002+0000","last_active":"2024-04-25T11:28:46.290920+0000","last_peered":"2024-04-25T11:28:46.290920+0000","last_clean":"2024-04-25T11:28:46.290920+0000","last_became_active":"2024-04-25T11:28:46.287672+0000","last_became_peered":"2024-04-25T11:28:46.287672+0000","last_unstale":"2024-04-25T11:28:46.290920+0000","last_undegraded":"2024-04-25T11:28:46.290920+0000","last_fullsized":"2024-04-25T11:28:46.290920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:21.347782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280465+0000","last_change":"2024-04-25T11:28:46.316356+0000","last_active":"2024-04-25T11:28:47.280465+0000","last_peered":"2024-04-25T11:28:47.280465+0000","last_clean":"2024-04-25T11:28:47.280465+0000","last_became_active":"2024-04-25T11:28:46.316005+0000","last_became_peered":"2024-04-25T11:28:46.316005+0000","last_unstale":"2024-04-25T11:28:47.280465+0000","last_undegraded":"2024-04-25T11:28:47.280465+0000","last_fullsized":"2024-04-25T11:28:47.280465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:51.371881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.297577+0000","last_change":"2024-04-25T11:28:46.289410+0000","last_active":"2024-04-25T11:28:46.297577+0000","last_peered":"2024-04-25T11:28:46.297577+0000","last_clean":"2024-04-25T11:28:46.297577+0000","last_became_active":"2024-04-25T11:28:46.286458+0000","last_became_peered":"2024-04-25T11:28:46.286458+0000","last_unstale":"2024-04-25T11:28:46.297577+0000","last_undegraded":"2024-04-25T11:28:46.297577+0000","last_fullsized":"2024-04-25T11:28:46.297577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:17.030731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280344+0000","last_change":"2024-04-25T11:28:46.321486+0000","last_active":"2024-04-25T11:28:47.280344+0000","last_peered":"2024-04-25T11:28:47.280344+0000","last_clean":"2024-04-25T11:28:47.280344+0000","last_became_active":"2024-04-25T11:28:46.321120+0000","last_became_peered":"2024-04-25T11:28:46.321120+0000","last_unstale":"2024-04-25T11:28:47.280344+0000","last_undegraded":"2024-04-25T11:28:47.280344+0000","last_fullsized":"2024-04-25T11:28:47.280344+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:57.256080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329406+0000","last_change":"2024-04-25T11:28:46.322755+0000","last_active":"2024-04-25T11:28:46.329406+0000","last_peered":"2024-04-25T11:28:46.329406+0000","last_clean":"2024-04-25T11:28:46.329406+0000","last_became_active":"2024-04-25T11:28:46.322383+0000","last_became_peered":"2024-04-25T11:28:46.322383+0000","last_unstale":"2024-04-25T11:28:46.329406+0000","last_undegraded":"2024-04-25T11:28:46.329406+0000","last_fullsized":"2024-04-25T11:28:46.329406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:02.769083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329195+0000","last_change":"2024-04-25T11:28:46.316540+0000","last_active":"2024-04-25T11:28:46.329195+0000","last_peered":"2024-04-25T11:28:46.329195+0000","last_clean":"2024-04-25T11:28:46.329195+0000","last_became_active":"2024-04-25T11:28:46.312668+0000","last_became_peered":"2024-04-25T11:28:46.312668+0000","last_unstale":"2024-04-25T11:28:46.329195+0000","last_undegraded":"2024-04-25T11:28:46.329195+0000","last_fullsized":"2024-04-25T11:28:46.329195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:21.347782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.323342+0000","last_change":"2024-04-25T11:28:46.318598+0000","last_active":"2024-04-25T11:28:46.323342+0000","last_peered":"2024-04-25T11:28:46.323342+0000","last_clean":"2024-04-25T11:28:46.323342+0000","last_became_active":"2024-04-25T11:28:46.318271+0000","last_became_peered":"2024-04-25T11:28:46.318271+0000","last_unstale":"2024-04-25T11:28:46.323342+0000","last_undegraded":"2024-04-25T11:28:46.323342+0000","last_fullsized":"2024-04-25T11:28:46.323342+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:53.246076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281393+0000","last_change":"2024-04-25T11:28:46.315551+0000","last_active":"2024-04-25T11:28:47.281393+0000","last_peered":"2024-04-25T11:28:47.281393+0000","last_clean":"2024-04-25T11:28:47.281393+0000","last_became_active":"2024-04-25T11:28:46.315254+0000","last_became_peered":"2024-04-25T11:28:46.315254+0000","last_unstale":"2024-04-25T11:28:47.281393+0000","last_undegraded":"2024-04-25T11:28:47.281393+0000","last_fullsized":"2024-04-25T11:28:47.281393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:35.205793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.326108+0000","last_change":"2024-04-25T11:28:46.320950+0000","last_active":"2024-04-25T11:28:46.326108+0000","last_peered":"2024-04-25T11:28:46.326108+0000","last_clean":"2024-04-25T11:28:46.326108+0000","last_became_active":"2024-04-25T11:28:46.320661+0000","last_became_peered":"2024-04-25T11:28:46.320661+0000","last_unstale":"2024-04-25T11:28:46.326108+0000","last_undegraded":"2024-04-25T11:28:46.326108+0000","last_fullsized":"2024-04-25T11:28:46.326108+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:24.033667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.317243+0000","last_change":"2024-04-25T11:28:46.305125+0000","last_active":"2024-04-25T11:28:46.317243+0000","last_peered":"2024-04-25T11:28:46.317243+0000","last_clean":"2024-04-25T11:28:46.317243+0000","last_became_active":"2024-04-25T11:28:46.304829+0000","last_became_peered":"2024-04-25T11:28:46.304829+0000","last_unstale":"2024-04-25T11:28:46.317243+0000","last_undegraded":"2024-04-25T11:28:46.317243+0000","last_fullsized":"2024-04-25T11:28:46.317243+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:02.769083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280144+0000","last_change":"2024-04-25T11:28:46.314283+0000","last_active":"2024-04-25T11:28:47.280144+0000","last_peered":"2024-04-25T11:28:47.280144+0000","last_clean":"2024-04-25T11:28:47.280144+0000","last_became_active":"2024-04-25T11:28:46.311050+0000","last_became_peered":"2024-04-25T11:28:46.311050+0000","last_unstale":"2024-04-25T11:28:47.280144+0000","last_undegraded":"2024-04-25T11:28:47.280144+0000","last_fullsized":"2024-04-25T11:28:47.280144+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:43.812507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.275794+0000","last_change":"2024-04-25T11:28:46.309859+0000","last_active":"2024-04-25T11:28:47.275794+0000","last_peered":"2024-04-25T11:28:47.275794+0000","last_clean":"2024-04-25T11:28:47.275794+0000","last_became_active":"2024-04-25T11:28:46.309008+0000","last_became_peered":"2024-04-25T11:28:46.309008+0000","last_unstale":"2024-04-25T11:28:47.275794+0000","last_undegraded":"2024-04-25T11:28:47.275794+0000","last_fullsized":"2024-04-25T11:28:47.275794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:47.127556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.301277+0000","last_change":"2024-04-25T11:28:46.295983+0000","last_active":"2024-04-25T11:28:46.301277+0000","last_peered":"2024-04-25T11:28:46.301277+0000","last_clean":"2024-04-25T11:28:46.301277+0000","last_became_active":"2024-04-25T11:28:46.295535+0000","last_became_peered":"2024-04-25T11:28:46.295535+0000","last_unstale":"2024-04-25T11:28:46.301277+0000","last_undegraded":"2024-04-25T11:28:46.301277+0000","last_fullsized":"2024-04-25T11:28:46.301277+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:02.014028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278845+0000","last_change":"2024-04-25T11:28:45.266930+0000","last_active":"2024-04-25T11:28:47.278845+0000","last_peered":"2024-04-25T11:28:47.278845+0000","last_clean":"2024-04-25T11:28:47.278845+0000","last_became_active":"2024-04-25T11:28:45.266520+0000","last_became_peered":"2024-04-25T11:28:45.266520+0000","last_unstale":"2024-04-25T11:28:47.278845+0000","last_undegraded":"2024-04-25T11:28:47.278845+0000","last_fullsized":"2024-04-25T11:28:47.278845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:08.296718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280770+0000","last_change":"2024-04-25T11:28:46.314511+0000","last_active":"2024-04-25T11:28:47.280770+0000","last_peered":"2024-04-25T11:28:47.280770+0000","last_clean":"2024-04-25T11:28:47.280770+0000","last_became_active":"2024-04-25T11:28:46.313909+0000","last_became_peered":"2024-04-25T11:28:46.313909+0000","last_unstale":"2024-04-25T11:28:47.280770+0000","last_undegraded":"2024-04-25T11:28:47.280770+0000","last_fullsized":"2024-04-25T11:28:47.280770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:46.850279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329228+0000","last_change":"2024-04-25T11:28:45.290759+0000","last_active":"2024-04-25T11:28:46.329228+0000","last_peered":"2024-04-25T11:28:46.329228+0000","last_clean":"2024-04-25T11:28:46.329228+0000","last_became_active":"2024-04-25T11:28:45.290411+0000","last_became_peered":"2024-04-25T11:28:45.290411+0000","last_unstale":"2024-04-25T11:28:46.329228+0000","last_undegraded":"2024-04-25T11:28:46.329228+0000","last_fullsized":"2024-04-25T11:28:46.329228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:37.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281560+0000","last_change":"2024-04-25T11:28:46.321741+0000","last_active":"2024-04-25T11:28:47.281560+0000","last_peered":"2024-04-25T11:28:47.281560+0000","last_clean":"2024-04-25T11:28:47.281560+0000","last_became_active":"2024-04-25T11:28:46.321440+0000","last_became_peered":"2024-04-25T11:28:46.321440+0000","last_unstale":"2024-04-25T11:28:47.281560+0000","last_undegraded":"2024-04-25T11:28:47.281560+0000","last_fullsized":"2024-04-25T11:28:47.281560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:11.647467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.282035+0000","last_change":"2024-04-25T11:28:45.283578+0000","last_active":"2024-04-25T11:28:47.282035+0000","last_peered":"2024-04-25T11:28:47.282035+0000","last_clean":"2024-04-25T11:28:47.282035+0000","last_became_active":"2024-04-25T11:28:45.283288+0000","last_became_peered":"2024-04-25T11:28:45.283288+0000","last_unstale":"2024-04-25T11:28:47.282035+0000","last_undegraded":"2024-04-25T11:28:47.282035+0000","last_fullsized":"2024-04-25T11:28:47.282035+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:50.300240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332322+0000","last_change":"2024-04-25T11:28:46.327008+0000","last_active":"2024-04-25T11:28:46.332322+0000","last_peered":"2024-04-25T11:28:46.332322+0000","last_clean":"2024-04-25T11:28:46.332322+0000","last_became_active":"2024-04-25T11:28:46.326656+0000","last_became_peered":"2024-04-25T11:28:46.326656+0000","last_unstale":"2024-04-25T11:28:46.332322+0000","last_undegraded":"2024-04-25T11:28:46.332322+0000","last_fullsized":"2024-04-25T11:28:46.332322+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:50.938051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.331925+0000","last_change":"2024-04-25T11:28:45.289229+0000","last_active":"2024-04-25T11:28:46.331925+0000","last_peered":"2024-04-25T11:28:46.331925+0000","last_clean":"2024-04-25T11:28:46.331925+0000","last_became_active":"2024-04-25T11:28:45.288677+0000","last_became_peered":"2024-04-25T11:28:45.288677+0000","last_unstale":"2024-04-25T11:28:46.331925+0000","last_undegraded":"2024-04-25T11:28:46.331925+0000","last_fullsized":"2024-04-25T11:28:46.331925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:16.025087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.318382+0000","last_change":"2024-04-25T11:28:46.313967+0000","last_active":"2024-04-25T11:28:46.318382+0000","last_peered":"2024-04-25T11:28:46.318382+0000","last_clean":"2024-04-25T11:28:46.318382+0000","last_became_active":"2024-04-25T11:28:46.313682+0000","last_became_peered":"2024-04-25T11:28:46.313682+0000","last_unstale":"2024-04-25T11:28:46.318382+0000","last_undegraded":"2024-04-25T11:28:46.318382+0000","last_fullsized":"2024-04-25T11:28:46.318382+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.752011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280968+0000","last_change":"2024-04-25T11:28:45.272785+0000","last_active":"2024-04-25T11:28:47.280968+0000","last_peered":"2024-04-25T11:28:47.280968+0000","last_clean":"2024-04-25T11:28:47.280968+0000","last_became_active":"2024-04-25T11:28:45.272502+0000","last_became_peered":"2024-04-25T11:28:45.272502+0000","last_unstale":"2024-04-25T11:28:47.280968+0000","last_undegraded":"2024-04-25T11:28:47.280968+0000","last_fullsized":"2024-04-25T11:28:47.280968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280849+0000","last_change":"2024-04-25T11:28:46.311260+0000","last_active":"2024-04-25T11:28:47.280849+0000","last_peered":"2024-04-25T11:28:47.280849+0000","last_clean":"2024-04-25T11:28:47.280849+0000","last_became_active":"2024-04-25T11:28:46.310887+0000","last_became_peered":"2024-04-25T11:28:46.310887+0000","last_unstale":"2024-04-25T11:28:47.280849+0000","last_undegraded":"2024-04-25T11:28:47.280849+0000","last_fullsized":"2024-04-25T11:28:47.280849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:09.302361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280500+0000","last_change":"2024-04-25T11:28:45.281862+0000","last_active":"2024-04-25T11:28:47.280500+0000","last_peered":"2024-04-25T11:28:47.280500+0000","last_clean":"2024-04-25T11:28:47.280500+0000","last_became_active":"2024-04-25T11:28:45.281547+0000","last_became_peered":"2024-04-25T11:28:45.281547+0000","last_unstale":"2024-04-25T11:28:47.280500+0000","last_undegraded":"2024-04-25T11:28:47.280500+0000","last_fullsized":"2024-04-25T11:28:47.280500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:31.400252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.302661+0000","last_change":"2024-04-25T11:28:46.292599+0000","last_active":"2024-04-25T11:28:46.302661+0000","last_peered":"2024-04-25T11:28:46.302661+0000","last_clean":"2024-04-25T11:28:46.302661+0000","last_became_active":"2024-04-25T11:28:46.292282+0000","last_became_peered":"2024-04-25T11:28:46.292282+0000","last_unstale":"2024-04-25T11:28:46.302661+0000","last_undegraded":"2024-04-25T11:28:46.302661+0000","last_fullsized":"2024-04-25T11:28:46.302661+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:10.030975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280768+0000","last_change":"2024-04-25T11:28:45.285376+0000","last_active":"2024-04-25T11:28:47.280768+0000","last_peered":"2024-04-25T11:28:47.280768+0000","last_clean":"2024-04-25T11:28:47.280768+0000","last_became_active":"2024-04-25T11:28:45.285057+0000","last_became_peered":"2024-04-25T11:28:45.285057+0000","last_unstale":"2024-04-25T11:28:47.280768+0000","last_undegraded":"2024-04-25T11:28:47.280768+0000","last_fullsized":"2024-04-25T11:28:47.280768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:09.025331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329367+0000","last_change":"2024-04-25T11:28:46.320762+0000","last_active":"2024-04-25T11:28:46.329367+0000","last_peered":"2024-04-25T11:28:46.329367+0000","last_clean":"2024-04-25T11:28:46.329367+0000","last_became_active":"2024-04-25T11:28:46.320433+0000","last_became_peered":"2024-04-25T11:28:46.320433+0000","last_unstale":"2024-04-25T11:28:46.329367+0000","last_undegraded":"2024-04-25T11:28:46.329367+0000","last_fullsized":"2024-04-25T11:28:46.329367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:58.755204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280686+0000","last_change":"2024-04-25T11:28:45.275667+0000","last_active":"2024-04-25T11:28:47.280686+0000","last_peered":"2024-04-25T11:28:47.280686+0000","last_clean":"2024-04-25T11:28:47.280686+0000","last_became_active":"2024-04-25T11:28:45.275315+0000","last_became_peered":"2024-04-25T11:28:45.275315+0000","last_unstale":"2024-04-25T11:28:47.280686+0000","last_undegraded":"2024-04-25T11:28:47.280686+0000","last_fullsized":"2024-04-25T11:28:47.280686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:03.456052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277826+0000","last_change":"2024-04-25T11:28:46.302084+0000","last_active":"2024-04-25T11:28:47.277826+0000","last_peered":"2024-04-25T11:28:47.277826+0000","last_clean":"2024-04-25T11:28:47.277826+0000","last_became_active":"2024-04-25T11:28:46.301771+0000","last_became_peered":"2024-04-25T11:28:46.301771+0000","last_unstale":"2024-04-25T11:28:47.277826+0000","last_undegraded":"2024-04-25T11:28:47.277826+0000","last_fullsized":"2024-04-25T11:28:47.277826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:14.618746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.294477+0000","last_change":"2024-04-25T11:28:45.272745+0000","last_active":"2024-04-25T11:28:46.294477+0000","last_peered":"2024-04-25T11:28:46.294477+0000","last_clean":"2024-04-25T11:28:46.294477+0000","last_became_active":"2024-04-25T11:28:45.272444+0000","last_became_peered":"2024-04-25T11:28:45.272444+0000","last_unstale":"2024-04-25T11:28:46.294477+0000","last_undegraded":"2024-04-25T11:28:46.294477+0000","last_fullsized":"2024-04-25T11:28:46.294477+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:31.850523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.328347+0000","last_change":"2024-04-25T11:28:46.316314+0000","last_active":"2024-04-25T11:28:46.328347+0000","last_peered":"2024-04-25T11:28:46.328347+0000","last_clean":"2024-04-25T11:28:46.328347+0000","last_became_active":"2024-04-25T11:28:46.312794+0000","last_became_peered":"2024-04-25T11:28:46.312794+0000","last_unstale":"2024-04-25T11:28:46.328347+0000","last_undegraded":"2024-04-25T11:28:46.328347+0000","last_fullsized":"2024-04-25T11:28:46.328347+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.343940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.289287+0000","last_change":"2024-04-25T11:28:45.279928+0000","last_active":"2024-04-25T11:28:46.289287+0000","last_peered":"2024-04-25T11:28:46.289287+0000","last_clean":"2024-04-25T11:28:46.289287+0000","last_became_active":"2024-04-25T11:28:45.279427+0000","last_became_peered":"2024-04-25T11:28:45.279427+0000","last_unstale":"2024-04-25T11:28:46.289287+0000","last_undegraded":"2024-04-25T11:28:46.289287+0000","last_fullsized":"2024-04-25T11:28:46.289287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:49.608625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332659+0000","last_change":"2024-04-25T11:28:46.329960+0000","last_active":"2024-04-25T11:28:46.332659+0000","last_peered":"2024-04-25T11:28:46.332659+0000","last_clean":"2024-04-25T11:28:46.332659+0000","last_became_active":"2024-04-25T11:28:46.329553+0000","last_became_peered":"2024-04-25T11:28:46.329553+0000","last_unstale":"2024-04-25T11:28:46.332659+0000","last_undegraded":"2024-04-25T11:28:46.332659+0000","last_fullsized":"2024-04-25T11:28:46.332659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:43.812507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277173+0000","last_change":"2024-04-25T11:28:45.269784+0000","last_active":"2024-04-25T11:28:47.277173+0000","last_peered":"2024-04-25T11:28:47.277173+0000","last_clean":"2024-04-25T11:28:47.277173+0000","last_became_active":"2024-04-25T11:28:45.269034+0000","last_became_peered":"2024-04-25T11:28:45.269034+0000","last_unstale":"2024-04-25T11:28:47.277173+0000","last_undegraded":"2024-04-25T11:28:47.277173+0000","last_fullsized":"2024-04-25T11:28:47.277173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:50.626568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277036+0000","last_change":"2024-04-25T11:28:46.314974+0000","last_active":"2024-04-25T11:28:47.277036+0000","last_peered":"2024-04-25T11:28:47.277036+0000","last_clean":"2024-04-25T11:28:47.277036+0000","last_became_active":"2024-04-25T11:28:46.314686+0000","last_became_peered":"2024-04-25T11:28:46.314686+0000","last_unstale":"2024-04-25T11:28:47.277036+0000","last_undegraded":"2024-04-25T11:28:47.277036+0000","last_fullsized":"2024-04-25T11:28:47.277036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:15.290099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.336074+0000","last_change":"2024-04-25T11:28:45.291183+0000","last_active":"2024-04-25T11:28:46.336074+0000","last_peered":"2024-04-25T11:28:46.336074+0000","last_clean":"2024-04-25T11:28:46.336074+0000","last_became_active":"2024-04-25T11:28:45.290779+0000","last_became_peered":"2024-04-25T11:28:45.290779+0000","last_unstale":"2024-04-25T11:28:46.336074+0000","last_undegraded":"2024-04-25T11:28:46.336074+0000","last_fullsized":"2024-04-25T11:28:46.336074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:03.456052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.304435+0000","last_change":"2024-04-25T11:28:46.300972+0000","last_active":"2024-04-25T11:28:46.304435+0000","last_peered":"2024-04-25T11:28:46.304435+0000","last_clean":"2024-04-25T11:28:46.304435+0000","last_became_active":"2024-04-25T11:28:46.300495+0000","last_became_peered":"2024-04-25T11:28:46.300495+0000","last_unstale":"2024-04-25T11:28:46.304435+0000","last_undegraded":"2024-04-25T11:28:46.304435+0000","last_fullsized":"2024-04-25T11:28:46.304435+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:43.367845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281090+0000","last_change":"2024-04-25T11:28:45.272277+0000","last_active":"2024-04-25T11:28:47.281090+0000","last_peered":"2024-04-25T11:28:47.281090+0000","last_clean":"2024-04-25T11:28:47.281090+0000","last_became_active":"2024-04-25T11:28:45.272059+0000","last_became_peered":"2024-04-25T11:28:45.272059+0000","last_unstale":"2024-04-25T11:28:47.281090+0000","last_undegraded":"2024-04-25T11:28:47.281090+0000","last_fullsized":"2024-04-25T11:28:47.281090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.050436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332397+0000","last_change":"2024-04-25T11:28:46.327318+0000","last_active":"2024-04-25T11:28:46.332397+0000","last_peered":"2024-04-25T11:28:46.332397+0000","last_clean":"2024-04-25T11:28:46.332397+0000","last_became_active":"2024-04-25T11:28:46.327004+0000","last_became_peered":"2024-04-25T11:28:46.327004+0000","last_unstale":"2024-04-25T11:28:46.332397+0000","last_undegraded":"2024-04-25T11:28:46.332397+0000","last_fullsized":"2024-04-25T11:28:46.332397+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:05.863257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.302492+0000","last_change":"2024-04-25T11:28:45.281317+0000","last_active":"2024-04-25T11:28:46.302492+0000","last_peered":"2024-04-25T11:28:46.302492+0000","last_clean":"2024-04-25T11:28:46.302492+0000","last_became_active":"2024-04-25T11:28:45.279342+0000","last_became_peered":"2024-04-25T11:28:45.279342+0000","last_unstale":"2024-04-25T11:28:46.302492+0000","last_undegraded":"2024-04-25T11:28:46.302492+0000","last_fullsized":"2024-04-25T11:28:46.302492+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:35.072151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280759+0000","last_change":"2024-04-25T11:28:46.314842+0000","last_active":"2024-04-25T11:28:47.280759+0000","last_peered":"2024-04-25T11:28:47.280759+0000","last_clean":"2024-04-25T11:28:47.280759+0000","last_became_active":"2024-04-25T11:28:46.310198+0000","last_became_peered":"2024-04-25T11:28:46.310198+0000","last_unstale":"2024-04-25T11:28:47.280759+0000","last_undegraded":"2024-04-25T11:28:47.280759+0000","last_fullsized":"2024-04-25T11:28:47.280759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:51.632211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.297508+0000","last_change":"2024-04-25T11:28:45.282193+0000","last_active":"2024-04-25T11:28:46.297508+0000","last_peered":"2024-04-25T11:28:46.297508+0000","last_clean":"2024-04-25T11:28:46.297508+0000","last_became_active":"2024-04-25T11:28:45.281826+0000","last_became_peered":"2024-04-25T11:28:45.281826+0000","last_unstale":"2024-04-25T11:28:46.297508+0000","last_undegraded":"2024-04-25T11:28:46.297508+0000","last_fullsized":"2024-04-25T11:28:46.297508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.050436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.276315+0000","last_change":"2024-04-25T11:28:46.308172+0000","last_active":"2024-04-25T11:28:47.276315+0000","last_peered":"2024-04-25T11:28:47.276315+0000","last_clean":"2024-04-25T11:28:47.276315+0000","last_became_active":"2024-04-25T11:28:46.307878+0000","last_became_peered":"2024-04-25T11:28:46.307878+0000","last_unstale":"2024-04-25T11:28:47.276315+0000","last_undegraded":"2024-04-25T11:28:47.276315+0000","last_fullsized":"2024-04-25T11:28:47.276315+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:07.559061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280300+0000","last_change":"2024-04-25T11:28:45.286494+0000","last_active":"2024-04-25T11:28:47.280300+0000","last_peered":"2024-04-25T11:28:47.280300+0000","last_clean":"2024-04-25T11:28:47.280300+0000","last_became_active":"2024-04-25T11:28:45.286166+0000","last_became_peered":"2024-04-25T11:28:45.286166+0000","last_unstale":"2024-04-25T11:28:47.280300+0000","last_undegraded":"2024-04-25T11:28:47.280300+0000","last_fullsized":"2024-04-25T11:28:47.280300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:33.328188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278362+0000","last_change":"2024-04-25T11:28:46.319185+0000","last_active":"2024-04-25T11:28:47.278362+0000","last_peered":"2024-04-25T11:28:47.278362+0000","last_clean":"2024-04-25T11:28:47.278362+0000","last_became_active":"2024-04-25T11:28:46.318242+0000","last_became_peered":"2024-04-25T11:28:46.318242+0000","last_unstale":"2024-04-25T11:28:47.278362+0000","last_undegraded":"2024-04-25T11:28:47.278362+0000","last_fullsized":"2024-04-25T11:28:47.278362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:51.404114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280860+0000","last_change":"2024-04-25T11:28:45.287663+0000","last_active":"2024-04-25T11:28:47.280860+0000","last_peered":"2024-04-25T11:28:47.280860+0000","last_clean":"2024-04-25T11:28:47.280860+0000","last_became_active":"2024-04-25T11:28:45.287322+0000","last_became_peered":"2024-04-25T11:28:45.287322+0000","last_unstale":"2024-04-25T11:28:47.280860+0000","last_undegraded":"2024-04-25T11:28:47.280860+0000","last_fullsized":"2024-04-25T11:28:47.280860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:27.801726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.298984+0000","last_change":"2024-04-25T11:28:46.291827+0000","last_active":"2024-04-25T11:28:46.298984+0000","last_peered":"2024-04-25T11:28:46.298984+0000","last_clean":"2024-04-25T11:28:46.298984+0000","last_became_active":"2024-04-25T11:28:46.291420+0000","last_became_peered":"2024-04-25T11:28:46.291420+0000","last_unstale":"2024-04-25T11:28:46.298984+0000","last_undegraded":"2024-04-25T11:28:46.298984+0000","last_fullsized":"2024-04-25T11:28:46.298984+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:28.807369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281048+0000","last_change":"2024-04-25T11:28:45.272075+0000","last_active":"2024-04-25T11:28:47.281048+0000","last_peered":"2024-04-25T11:28:47.281048+0000","last_clean":"2024-04-25T11:28:47.281048+0000","last_became_active":"2024-04-25T11:28:45.271248+0000","last_became_peered":"2024-04-25T11:28:45.271248+0000","last_unstale":"2024-04-25T11:28:47.281048+0000","last_undegraded":"2024-04-25T11:28:47.281048+0000","last_fullsized":"2024-04-25T11:28:47.281048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:16.025087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.314173+0000","last_change":"2024-04-25T11:28:46.311005+0000","last_active":"2024-04-25T11:28:46.314173+0000","last_peered":"2024-04-25T11:28:46.314173+0000","last_clean":"2024-04-25T11:28:46.314173+0000","last_became_active":"2024-04-25T11:28:46.309575+0000","last_became_peered":"2024-04-25T11:28:46.309575+0000","last_unstale":"2024-04-25T11:28:46.314173+0000","last_undegraded":"2024-04-25T11:28:46.314173+0000","last_fullsized":"2024-04-25T11:28:46.314173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:45.954999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277291+0000","last_change":"2024-04-25T11:28:45.278364+0000","last_active":"2024-04-25T11:28:47.277291+0000","last_peered":"2024-04-25T11:28:47.277291+0000","last_clean":"2024-04-25T11:28:47.277291+0000","last_became_active":"2024-04-25T11:28:45.276953+0000","last_became_peered":"2024-04-25T11:28:45.276953+0000","last_unstale":"2024-04-25T11:28:47.277291+0000","last_undegraded":"2024-04-25T11:28:47.277291+0000","last_fullsized":"2024-04-25T11:28:47.277291+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.844636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281961+0000","last_change":"2024-04-25T11:28:46.325237+0000","last_active":"2024-04-25T11:28:47.281961+0000","last_peered":"2024-04-25T11:28:47.281961+0000","last_clean":"2024-04-25T11:28:47.281961+0000","last_became_active":"2024-04-25T11:28:46.324875+0000","last_became_peered":"2024-04-25T11:28:46.324875+0000","last_unstale":"2024-04-25T11:28:47.281961+0000","last_undegraded":"2024-04-25T11:28:47.281961+0000","last_fullsized":"2024-04-25T11:28:47.281961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:43.367845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.284008+0000","last_change":"2024-04-25T11:28:45.286891+0000","last_active":"2024-04-25T11:28:46.284008+0000","last_peered":"2024-04-25T11:28:46.284008+0000","last_clean":"2024-04-25T11:28:46.284008+0000","last_became_active":"2024-04-25T11:28:45.286523+0000","last_became_peered":"2024-04-25T11:28:45.286523+0000","last_unstale":"2024-04-25T11:28:46.284008+0000","last_undegraded":"2024-04-25T11:28:46.284008+0000","last_fullsized":"2024-04-25T11:28:46.284008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:46.338296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.303327+0000","last_change":"2024-04-25T11:28:46.298137+0000","last_active":"2024-04-25T11:28:46.303327+0000","last_peered":"2024-04-25T11:28:46.303327+0000","last_clean":"2024-04-25T11:28:46.303327+0000","last_became_active":"2024-04-25T11:28:46.296872+0000","last_became_peered":"2024-04-25T11:28:46.296872+0000","last_unstale":"2024-04-25T11:28:46.303327+0000","last_undegraded":"2024-04-25T11:28:46.303327+0000","last_fullsized":"2024-04-25T11:28:46.303327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.396310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.329067+0000","last_change":"2024-04-25T11:28:45.281197+0000","last_active":"2024-04-25T11:28:46.329067+0000","last_peered":"2024-04-25T11:28:46.329067+0000","last_clean":"2024-04-25T11:28:46.329067+0000","last_became_active":"2024-04-25T11:28:45.280442+0000","last_became_peered":"2024-04-25T11:28:45.280442+0000","last_unstale":"2024-04-25T11:28:46.329067+0000","last_undegraded":"2024-04-25T11:28:46.329067+0000","last_fullsized":"2024-04-25T11:28:46.329067+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:29.553007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.315551+0000","last_change":"2024-04-25T11:28:46.312929+0000","last_active":"2024-04-25T11:28:46.315551+0000","last_peered":"2024-04-25T11:28:46.315551+0000","last_clean":"2024-04-25T11:28:46.315551+0000","last_became_active":"2024-04-25T11:28:46.311167+0000","last_became_peered":"2024-04-25T11:28:46.311167+0000","last_unstale":"2024-04-25T11:28:46.315551+0000","last_undegraded":"2024-04-25T11:28:46.315551+0000","last_fullsized":"2024-04-25T11:28:46.315551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:41.890750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280799+0000","last_change":"2024-04-25T11:28:45.271924+0000","last_active":"2024-04-25T11:28:47.280799+0000","last_peered":"2024-04-25T11:28:47.280799+0000","last_clean":"2024-04-25T11:28:47.280799+0000","last_became_active":"2024-04-25T11:28:45.271265+0000","last_became_peered":"2024-04-25T11:28:45.271265+0000","last_unstale":"2024-04-25T11:28:47.280799+0000","last_undegraded":"2024-04-25T11:28:47.280799+0000","last_fullsized":"2024-04-25T11:28:47.280799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.746368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.324804+0000","last_change":"2024-04-25T11:28:46.318984+0000","last_active":"2024-04-25T11:28:46.324804+0000","last_peered":"2024-04-25T11:28:46.324804+0000","last_clean":"2024-04-25T11:28:46.324804+0000","last_became_active":"2024-04-25T11:28:46.318641+0000","last_became_peered":"2024-04-25T11:28:46.318641+0000","last_unstale":"2024-04-25T11:28:46.324804+0000","last_undegraded":"2024-04-25T11:28:46.324804+0000","last_fullsized":"2024-04-25T11:28:46.324804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.343940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.273224+0000","last_change":"2024-04-25T11:28:45.281461+0000","last_active":"2024-04-25T11:28:46.273224+0000","last_peered":"2024-04-25T11:28:46.273224+0000","last_clean":"2024-04-25T11:28:46.273224+0000","last_became_active":"2024-04-25T11:28:45.281198+0000","last_became_peered":"2024-04-25T11:28:45.281198+0000","last_unstale":"2024-04-25T11:28:46.273224+0000","last_undegraded":"2024-04-25T11:28:46.273224+0000","last_fullsized":"2024-04-25T11:28:46.273224+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:39.706502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.310048+0000","last_change":"2024-04-25T11:28:46.295213+0000","last_active":"2024-04-25T11:28:46.310048+0000","last_peered":"2024-04-25T11:28:46.310048+0000","last_clean":"2024-04-25T11:28:46.310048+0000","last_became_active":"2024-04-25T11:28:46.294914+0000","last_became_peered":"2024-04-25T11:28:46.294914+0000","last_unstale":"2024-04-25T11:28:46.310048+0000","last_undegraded":"2024-04-25T11:28:46.310048+0000","last_fullsized":"2024-04-25T11:28:46.310048+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:51.305884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.275936+0000","last_change":"2024-04-25T11:28:45.276137+0000","last_active":"2024-04-25T11:28:47.275936+0000","last_peered":"2024-04-25T11:28:47.275936+0000","last_clean":"2024-04-25T11:28:47.275936+0000","last_became_active":"2024-04-25T11:28:45.275801+0000","last_became_peered":"2024-04-25T11:28:45.275801+0000","last_unstale":"2024-04-25T11:28:47.275936+0000","last_undegraded":"2024-04-25T11:28:47.275936+0000","last_fullsized":"2024-04-25T11:28:47.275936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:19.196673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281781+0000","last_change":"2024-04-25T11:28:46.318339+0000","last_active":"2024-04-25T11:28:47.281781+0000","last_peered":"2024-04-25T11:28:47.281781+0000","last_clean":"2024-04-25T11:28:47.281781+0000","last_became_active":"2024-04-25T11:28:46.316841+0000","last_became_peered":"2024-04-25T11:28:46.316841+0000","last_unstale":"2024-04-25T11:28:47.281781+0000","last_undegraded":"2024-04-25T11:28:47.281781+0000","last_fullsized":"2024-04-25T11:28:47.281781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:02.014028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.327081+0000","last_change":"2024-04-25T11:28:45.286372+0000","last_active":"2024-04-25T11:28:46.327081+0000","last_peered":"2024-04-25T11:28:46.327081+0000","last_clean":"2024-04-25T11:28:46.327081+0000","last_became_active":"2024-04-25T11:28:45.286102+0000","last_became_peered":"2024-04-25T11:28:45.286102+0000","last_unstale":"2024-04-25T11:28:46.327081+0000","last_undegraded":"2024-04-25T11:28:46.327081+0000","last_fullsized":"2024-04-25T11:28:46.327081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:37.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.278810+0000","last_change":"2024-04-25T11:28:46.304246+0000","last_active":"2024-04-25T11:28:47.278810+0000","last_peered":"2024-04-25T11:28:47.278810+0000","last_clean":"2024-04-25T11:28:47.278810+0000","last_became_active":"2024-04-25T11:28:46.303958+0000","last_became_peered":"2024-04-25T11:28:46.303958+0000","last_unstale":"2024-04-25T11:28:47.278810+0000","last_undegraded":"2024-04-25T11:28:47.278810+0000","last_fullsized":"2024-04-25T11:28:47.278810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.154062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.289749+0000","last_change":"2024-04-25T11:28:45.280023+0000","last_active":"2024-04-25T11:28:46.289749+0000","last_peered":"2024-04-25T11:28:46.289749+0000","last_clean":"2024-04-25T11:28:46.289749+0000","last_became_active":"2024-04-25T11:28:45.277502+0000","last_became_peered":"2024-04-25T11:28:45.277502+0000","last_unstale":"2024-04-25T11:28:46.289749+0000","last_undegraded":"2024-04-25T11:28:46.289749+0000","last_fullsized":"2024-04-25T11:28:46.289749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:31.118311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.281815+0000","last_change":"2024-04-25T11:28:46.320847+0000","last_active":"2024-04-25T11:28:47.281815+0000","last_peered":"2024-04-25T11:28:47.281815+0000","last_clean":"2024-04-25T11:28:47.281815+0000","last_became_active":"2024-04-25T11:28:46.320509+0000","last_became_peered":"2024-04-25T11:28:46.320509+0000","last_unstale":"2024-04-25T11:28:47.281815+0000","last_undegraded":"2024-04-25T11:28:47.281815+0000","last_fullsized":"2024-04-25T11:28:47.281815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:13.396310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.279740+0000","last_change":"2024-04-25T11:28:45.278672+0000","last_active":"2024-04-25T11:28:46.279740+0000","last_peered":"2024-04-25T11:28:46.279740+0000","last_clean":"2024-04-25T11:28:46.279740+0000","last_became_active":"2024-04-25T11:28:45.278310+0000","last_became_peered":"2024-04-25T11:28:45.278310+0000","last_unstale":"2024-04-25T11:28:46.279740+0000","last_undegraded":"2024-04-25T11:28:46.279740+0000","last_fullsized":"2024-04-25T11:28:46.279740+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.050436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.328802+0000","last_change":"2024-04-25T11:28:46.317098+0000","last_active":"2024-04-25T11:28:46.328802+0000","last_peered":"2024-04-25T11:28:46.328802+0000","last_clean":"2024-04-25T11:28:46.328802+0000","last_became_active":"2024-04-25T11:28:46.312251+0000","last_became_peered":"2024-04-25T11:28:46.312251+0000","last_unstale":"2024-04-25T11:28:46.328802+0000","last_undegraded":"2024-04-25T11:28:46.328802+0000","last_fullsized":"2024-04-25T11:28:46.328802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:05.863257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.279728+0000","last_change":"2024-04-25T11:28:45.286410+0000","last_active":"2024-04-25T11:28:46.279728+0000","last_peered":"2024-04-25T11:28:46.279728+0000","last_clean":"2024-04-25T11:28:46.279728+0000","last_became_active":"2024-04-25T11:28:45.283400+0000","last_became_peered":"2024-04-25T11:28:45.283400+0000","last_unstale":"2024-04-25T11:28:46.279728+0000","last_undegraded":"2024-04-25T11:28:46.279728+0000","last_fullsized":"2024-04-25T11:28:46.279728+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332351+0000","last_change":"2024-04-25T11:28:46.327082+0000","last_active":"2024-04-25T11:28:46.332351+0000","last_peered":"2024-04-25T11:28:46.332351+0000","last_clean":"2024-04-25T11:28:46.332351+0000","last_became_active":"2024-04-25T11:28:46.326448+0000","last_became_peered":"2024-04-25T11:28:46.326448+0000","last_unstale":"2024-04-25T11:28:46.332351+0000","last_undegraded":"2024-04-25T11:28:46.332351+0000","last_fullsized":"2024-04-25T11:28:46.332351+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:21.792690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.279283+0000","last_change":"2024-04-25T11:28:45.284540+0000","last_active":"2024-04-25T11:28:46.279283+0000","last_peered":"2024-04-25T11:28:46.279283+0000","last_clean":"2024-04-25T11:28:46.279283+0000","last_became_active":"2024-04-25T11:28:45.284234+0000","last_became_peered":"2024-04-25T11:28:45.284234+0000","last_unstale":"2024-04-25T11:28:46.279283+0000","last_undegraded":"2024-04-25T11:28:46.279283+0000","last_fullsized":"2024-04-25T11:28:46.279283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.746368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.332536+0000","last_change":"2024-04-25T11:28:46.328826+0000","last_active":"2024-04-25T11:28:46.332536+0000","last_peered":"2024-04-25T11:28:46.332536+0000","last_clean":"2024-04-25T11:28:46.332536+0000","last_became_active":"2024-04-25T11:28:46.328552+0000","last_became_peered":"2024-04-25T11:28:46.328552+0000","last_unstale":"2024-04-25T11:28:46.332536+0000","last_undegraded":"2024-04-25T11:28:46.332536+0000","last_fullsized":"2024-04-25T11:28:46.332536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:24.033667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.280528+0000","last_change":"2024-04-25T11:28:45.280244+0000","last_active":"2024-04-25T11:28:47.280528+0000","last_peered":"2024-04-25T11:28:47.280528+0000","last_clean":"2024-04-25T11:28:47.280528+0000","last_became_active":"2024-04-25T11:28:45.279399+0000","last_became_peered":"2024-04-25T11:28:45.279399+0000","last_unstale":"2024-04-25T11:28:47.280528+0000","last_undegraded":"2024-04-25T11:28:47.280528+0000","last_fullsized":"2024-04-25T11:28:47.280528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:37.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.276490+0000","last_change":"2024-04-25T11:28:46.304874+0000","last_active":"2024-04-25T11:28:47.276490+0000","last_peered":"2024-04-25T11:28:47.276490+0000","last_clean":"2024-04-25T11:28:47.276490+0000","last_became_active":"2024-04-25T11:28:46.304127+0000","last_became_peered":"2024-04-25T11:28:46.304127+0000","last_unstale":"2024-04-25T11:28:47.276490+0000","last_undegraded":"2024-04-25T11:28:47.276490+0000","last_fullsized":"2024-04-25T11:28:47.276490+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.508594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.311967+0000","last_change":"2024-04-25T11:28:45.284257+0000","last_active":"2024-04-25T11:28:46.311967+0000","last_peered":"2024-04-25T11:28:46.311967+0000","last_clean":"2024-04-25T11:28:46.311967+0000","last_became_active":"2024-04-25T11:28:45.283952+0000","last_became_peered":"2024-04-25T11:28:45.283952+0000","last_unstale":"2024-04-25T11:28:46.311967+0000","last_undegraded":"2024-04-25T11:28:46.311967+0000","last_fullsized":"2024-04-25T11:28:46.311967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:01.008385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.319157+0000","last_change":"2024-04-25T11:28:46.311260+0000","last_active":"2024-04-25T11:28:46.319157+0000","last_peered":"2024-04-25T11:28:46.319157+0000","last_clean":"2024-04-25T11:28:46.319157+0000","last_became_active":"2024-04-25T11:28:46.310951+0000","last_became_peered":"2024-04-25T11:28:46.310951+0000","last_unstale":"2024-04-25T11:28:46.319157+0000","last_undegraded":"2024-04-25T11:28:46.319157+0000","last_fullsized":"2024-04-25T11:28:46.319157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:45.233767+0000","last_clean_scrub_stamp":"2024-04-25T11:28:45.233767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:21.792690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T11:28:47.277406+0000","last_change":"2024-04-25T11:28:45.274548+0000","last_active":"2024-04-25T11:28:47.277406+0000","last_peered":"2024-04-25T11:28:47.277406+0000","last_clean":"2024-04-25T11:28:47.277406+0000","last_became_active":"2024-04-25T11:28:45.274222+0000","last_became_peered":"2024-04-25T11:28:45.274222+0000","last_unstale":"2024-04-25T11:28:47.277406+0000","last_undegraded":"2024-04-25T11:28:47.277406+0000","last_fullsized":"2024-04-25T11:28:47.277406+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:28:44.228124+0000","last_clean_scrub_stamp":"2024-04-25T11:28:44.228124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:34.200150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":132,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T11:28:46.273171+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:28:46.273171+0000","last_peered":"2024-04-25T11:28:46.273171+0000","last_clean":"2024-04-25T11:28:46.273171+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:28:46.273171+0000","last_undegraded":"2024-04-25T11:28:46.273171+0000","last_fullsized":"2024-04-25T11:28:46.273171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":146806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":145948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":735610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":736122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:28:51.604 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:28:51.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:28:51.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.944+0000 7fee9a000640 1 Processor -- start 2024-04-25T11:28:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.944+0000 7fee9a000640 1 -- start start 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.946+0000 7fee9a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9415e180 0x7fee94161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.946+0000 7fee9a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fee940d1470 0x7fee940d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee9a000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee94162150 con 0x7fee940d1880 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee9a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fee940b84b0 con 0x7fee940d1470 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee9a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee940d4420 con 0x7fee9415e180 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee93e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fee940d1470 0x7fee940d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee93e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fee940d1470 0x7fee940d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:39776/0 (socket says 172.21.15.19:39776) 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee93e00640 1 -- 172.21.15.19:0/593948240 learned_addr learned my addr 172.21.15.19:0/593948240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee99600640 1 --1- 172.21.15.19:0/593948240 >> v1:172.21.15.169:6789/0 conn(0x7fee940d1880 0x7fee94160e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:43318/0 (socket says 172.21.15.19:43318) 2024-04-25T11:28:51.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.947+0000 7fee98c00640 1 --2- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9415e180 0x7fee94161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.948+0000 7fee93e00640 1 -- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9415e180 msgr2=0x7fee94161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.948+0000 7fee93e00640 1 --2- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9415e180 0x7fee94161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.948+0000 7fee93e00640 1 -- 172.21.15.19:0/593948240 >> v1:172.21.15.169:6789/0 conn(0x7fee940d1880 legacy=0x7fee94160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.948+0000 7fee93e00640 1 -- 172.21.15.19:0/593948240 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee940b7930 con 0x7fee940d1470 2024-04-25T11:28:51.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.949+0000 7fee93e00640 1 --2- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1470 0x7fee940d22b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009870 tx=0x7fee7c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e268389ec71d33b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:51.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.950+0000 7fee93400640 1 -- 172.21.15.19:0/593948240 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c037070 con 0x7fee940d1470 2024-04-25T11:28:51.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.950+0000 7fee93400640 1 -- 172.21.15.19:0/593948240 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7c032440 con 0x7fee940d1470 2024-04-25T11:28:51.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.950+0000 7fee93400640 1 -- 172.21.15.19:0/593948240 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c032810 con 0x7fee940d1470 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 -- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1470 msgr2=0x7fee940d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 --2- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1470 0x7fee940d22b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009870 tx=0x7fee7c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 -- 172.21.15.19:0/593948240 shutdown_connections 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 --2- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1470 0x7fee940d22b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 --2- 172.21.15.19:0/593948240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee9415e180 0x7fee94161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 -- 172.21.15.19:0/593948240 >> 172.21.15.19:0/593948240 conn(0x7fee9406dc30 msgr2=0x7fee9406e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 -- 172.21.15.19:0/593948240 shutdown_connections 2024-04-25T11:28:51.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.951+0000 7fee9a000640 1 -- 172.21.15.19:0/593948240 wait complete. 2024-04-25T11:28:51.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.954+0000 7fee9a000640 1 Processor -- start 2024-04-25T11:28:51.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.954+0000 7fee9a000640 1 -- start start 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.955+0000 7fee9a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee940d1470 0x7fee941fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.955+0000 7fee9a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1880 0x7fee940ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.955+0000 7fee9a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9415e180 0x7fee941f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.955+0000 7fee9a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee940b81d0 con 0x7fee9415e180 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.955+0000 7fee9a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee940b7ef0 con 0x7fee940d1880 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.955+0000 7fee9a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fee940b75e0 con 0x7fee940d1470 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9415e180 0x7fee941f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9415e180 0x7fee941f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43182/0 (socket says 172.21.15.19:43182) 2024-04-25T11:28:51.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 -- 172.21.15.19:0/3130715114 learned_addr learned my addr 172.21.15.19:0/3130715114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee98c00640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee940d1470 0x7fee941fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee93e00640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1880 0x7fee940ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 -- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee940d1470 msgr2=0x7fee941fa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee940d1470 0x7fee941fa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 -- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1880 msgr2=0x7fee940ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1880 0x7fee940ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.956+0000 7fee99600640 1 -- 172.21.15.19:0/3130715114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee941ead60 con 0x7fee9415e180 2024-04-25T11:28:51.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.957+0000 7fee99600640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9415e180 0x7fee941f60d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00c3f0 tx=0x7fee8c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.957+0000 7fee90c00640 1 -- 172.21.15.19:0/3130715114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0074e0 con 0x7fee9415e180 2024-04-25T11:28:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.957+0000 7fee90c00640 1 -- 172.21.15.19:0/3130715114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c007680 con 0x7fee9415e180 2024-04-25T11:28:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.957+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee941eb040 con 0x7fee9415e180 2024-04-25T11:28:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.957+0000 7fee90c00640 1 -- 172.21.15.19:0/3130715114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0049e0 con 0x7fee9415e180 2024-04-25T11:28:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.957+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee941eb600 con 0x7fee9415e180 2024-04-25T11:28:51.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.959+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee94068c30 con 0x7fee9415e180 2024-04-25T11:28:51.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.960+0000 7fee90c00640 1 -- 172.21.15.19:0/3130715114 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee8c01a020 con 0x7fee9415e180 2024-04-25T11:28:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.961+0000 7fee90c00640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee700744e0 0x7fee700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:51.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.961+0000 7fee90c00640 1 -- 172.21.15.19:0/3130715114 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fee8c094990 con 0x7fee9415e180 2024-04-25T11:28:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.962+0000 7fee98c00640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee700744e0 0x7fee700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:51.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.963+0000 7fee98c00640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee700744e0 0x7fee700769f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee940b8dc0 tx=0x7fee8400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:51.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:51.966+0000 7fee90c00640 1 -- 172.21.15.19:0/3130715114 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee94068c30 con 0x7fee9415e180 2024-04-25T11:28:52.133 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:28:52.142+0000 7f0536000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:28:52.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.269+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fee941fae70 con 0x7fee9415e180 2024-04-25T11:28:52.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.275+0000 7fee90c00640 1 -- 172.21.15.19:0/3130715114 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fee8c05f080 con 0x7fee9415e180 2024-04-25T11:28:52.266 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T11:28:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:28:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T11:28:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T11:28:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:28:52.267 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T11:28:52.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee700744e0 msgr2=0x7fee700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee700744e0 0x7fee700769f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fee940b8dc0 tx=0x7fee8400b040 comp rx=0 tx=0).stop 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9415e180 msgr2=0x7fee941f60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9415e180 0x7fee941f60d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00c3f0 tx=0x7fee8c00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 shutdown_connections 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee700744e0 0x7fee700769f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee940d1880 0x7fee940ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee9415e180 0x7fee941f60d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 --2- 172.21.15.19:0/3130715114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fee940d1470 0x7fee941fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 >> 172.21.15.19:0/3130715114 conn(0x7fee9406dc30 msgr2=0x7fee940b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 shutdown_connections 2024-04-25T11:28:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.278+0000 7fee9a000640 1 -- 172.21.15.19:0/3130715114 wait complete. 2024-04-25T11:28:52.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:28:52.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.586+0000 7f69d2e00640 1 Processor -- start 2024-04-25T11:28:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.586+0000 7f69d2e00640 1 -- start start 2024-04-25T11:28:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d2e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f69cc0cf2f0 0x7f69cc0cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d2e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69cc06c5c0 0x7f69cc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d2e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69cc071180 con 0x7f69cc168010 2024-04-25T11:28:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d2e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f69cc070cf0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d2e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f69cc0700e0 con 0x7f69cc0cf2f0 2024-04-25T11:28:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d1000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f69cc168010 0x7f69cc168420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39922/0 (socket says 172.21.15.19:39922) 2024-04-25T11:28:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d1000640 1 -- 172.21.15.19:0/2437256427 learned_addr learned my addr 172.21.15.19:0/2437256427 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d2400640 1 --2- 172.21.15.19:0/2437256427 >> v2:172.21.15.19:3300/0 conn(0x7f69cc06c5c0 0x7f69cc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.589+0000 7f69d1a00640 1 --2- 172.21.15.19:0/2437256427 >> v2:172.21.15.169:3300/0 conn(0x7f69cc0cf2f0 0x7f69cc0cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.590+0000 7f69d2400640 1 -- 172.21.15.19:0/2437256427 >> v2:172.21.15.169:3300/0 conn(0x7f69cc0cf2f0 msgr2=0x7f69cc0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.590+0000 7f69d2400640 1 --2- 172.21.15.19:0/2437256427 >> v2:172.21.15.169:3300/0 conn(0x7f69cc0cf2f0 0x7f69cc0cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.590+0000 7f69d2400640 1 -- 172.21.15.19:0/2437256427 >> v1:172.21.15.19:6789/0 conn(0x7f69cc168010 legacy=0x7f69cc168420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.590+0000 7f69d2400640 1 -- 172.21.15.19:0/2437256427 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc0703c0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.590+0000 7f69cbe00640 1 -- 172.21.15.19:0/2437256427 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913464899 0 0) 0x7f69cc071180 con 0x7f69cc168010 2024-04-25T11:28:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.591+0000 7f69d2400640 1 --2- 172.21.15.19:0/2437256427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc06c9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0726b0 tx=0x7f69c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8892cfab1ab6e010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:52.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.591+0000 7f69cbe00640 1 -- 172.21.15.19:0/2437256427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4007740 con 0x7f69cc06c5c0 2024-04-25T11:28:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.591+0000 7f69cbe00640 1 -- 172.21.15.19:0/2437256427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c40078e0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.591+0000 7f69cbe00640 1 -- 172.21.15.19:0/2437256427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4007cb0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 -- 172.21.15.19:0/2437256427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 msgr2=0x7f69cc06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 --2- 172.21.15.19:0/2437256427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc06c9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0726b0 tx=0x7f69c402a680 comp rx=0 tx=0).stop 2024-04-25T11:28:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 -- 172.21.15.19:0/2437256427 shutdown_connections 2024-04-25T11:28:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 --2- 172.21.15.19:0/2437256427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc06c9d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 --2- 172.21.15.19:0/2437256427 >> v2:172.21.15.169:3300/0 conn(0x7f69cc0cf2f0 0x7f69cc0cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 -- 172.21.15.19:0/2437256427 >> 172.21.15.19:0/2437256427 conn(0x7f69cc0d1720 msgr2=0x7f69cc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 -- 172.21.15.19:0/2437256427 shutdown_connections 2024-04-25T11:28:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.592+0000 7f69d2e00640 1 -- 172.21.15.19:0/2437256427 wait complete. 2024-04-25T11:28:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.595+0000 7f69d2e00640 1 Processor -- start 2024-04-25T11:28:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.595+0000 7f69d2e00640 1 -- start start 2024-04-25T11:28:52.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:52.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69cc0cf2f0 0x7f69cc13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:52.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69cc168010 0x7f69cc0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:52.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d2e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc168d00 con 0x7f69cc06c5c0 2024-04-25T11:28:52.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc070980 con 0x7f69cc168010 2024-04-25T11:28:52.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f69cc168e80 con 0x7f69cc0cf2f0 2024-04-25T11:28:52.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:52.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc15dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43190/0 (socket says 172.21.15.19:43190) 2024-04-25T11:28:52.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 -- 172.21.15.19:0/3234138777 learned_addr learned my addr 172.21.15.19:0/3234138777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 -- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69cc0cf2f0 msgr2=0x7f69cc13d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69cc0cf2f0 0x7f69cc13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d2400640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69cc168010 0x7f69cc0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 -- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69cc168010 msgr2=0x7f69cc0b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69cc168010 0x7f69cc0b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.597+0000 7f69d1a00640 1 -- 172.21.15.19:0/3234138777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc13e300 con 0x7f69cc06c5c0 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.598+0000 7f69d1a00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc15dd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f69c000d5d0 tx=0x7f69c000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.598+0000 7f69c9600640 1 -- 172.21.15.19:0/3234138777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0004240 con 0x7f69cc06c5c0 2024-04-25T11:28:52.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.598+0000 7f69c9600640 1 -- 172.21.15.19:0/3234138777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c00043e0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.598+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc13b720 con 0x7f69cc06c5c0 2024-04-25T11:28:52.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.598+0000 7f69c9600640 1 -- 172.21.15.19:0/3234138777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0003830 con 0x7f69cc06c5c0 2024-04-25T11:28:52.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.598+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc13bce0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.599+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69cc15e8a0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.601+0000 7f69c9600640 1 -- 172.21.15.19:0/3234138777 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f69c00039d0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.602+0000 7f69c9600640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69ac0744e0 0x7f69ac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:52.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.602+0000 7f69c9600640 1 -- 172.21.15.19:0/3234138777 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f69c0095f90 con 0x7f69cc06c5c0 2024-04-25T11:28:52.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.603+0000 7f69d1000640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69ac0744e0 0x7f69ac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:52.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.604+0000 7f69d1000640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69ac0744e0 0x7f69ac0769f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69b8007fd0 tx=0x7f69b8007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:52.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.606+0000 7f69c9600640 1 -- 172.21.15.19:0/3234138777 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69cc15e8a0 con 0x7f69cc06c5c0 2024-04-25T11:28:52.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.846+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69cc06f820 con 0x7f69cc06c5c0 2024-04-25T11:28:52.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.848+0000 7f69c9600640 1 -- 172.21.15.19:0/3234138777 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f69cc06f820 con 0x7f69cc06c5c0 2024-04-25T11:28:52.838 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:52.839 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:28:48.905254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3831883468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3831883468},{"type":"v1","addr":"172.21.15.19:6835","nonce":3831883468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5064,"qdb_cluster":[5064]},"id":2}]} 2024-04-25T11:28:52.839 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69ac0744e0 msgr2=0x7f69ac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69ac0744e0 0x7f69ac0769f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f69b8007fd0 tx=0x7f69b8007960 comp rx=0 tx=0).stop 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 msgr2=0x7f69cc15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc15dd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f69c000d5d0 tx=0x7f69c000da90 comp rx=0 tx=0).stop 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 shutdown_connections 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69ac0744e0 0x7f69ac0769f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69cc168010 0x7f69cc0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69cc0cf2f0 0x7f69cc13d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 --2- 172.21.15.19:0/3234138777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69cc06c5c0 0x7f69cc15dd50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 >> 172.21.15.19:0/3234138777 conn(0x7f69cc0d1720 msgr2=0x7f69cc0b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 shutdown_connections 2024-04-25T11:28:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:52.850+0000 7f69d2e00640 1 -- 172.21.15.19:0/3234138777 wait complete. 2024-04-25T11:28:53.271 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:28:45.819822+0000', 'modified': '2024-04-25T11:28:48.905254+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5064}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5064': {'gid': 5064, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/3831883468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3831883468}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3831883468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5064, 'qdb_cluster': [5064]} 2024-04-25T11:28:53.271 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:28:53.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:28:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.573+0000 7fc36be00640 1 Processor -- start 2024-04-25T11:28:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.573+0000 7fc36be00640 1 -- start start 2024-04-25T11:28:53.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.575+0000 7fc36be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc36c0cf2f0 0x7fc36c0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.576+0000 7fc36be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c1608d0 0x7fc36c160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.576+0000 7fc36be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc36c161220 con 0x7fc36c15a810 2024-04-25T11:28:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.576+0000 7fc36be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc36c06f870 con 0x7fc36c0cf2f0 2024-04-25T11:28:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.576+0000 7fc36be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc36c0b74d0 con 0x7fc36c1608d0 2024-04-25T11:28:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.576+0000 7fc36b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c1608d0 0x7fc36c160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.576+0000 7fc36b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c1608d0 0x7fc36c160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:48078/0 (socket says 172.21.15.19:48078) 2024-04-25T11:28:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc36b400640 1 -- 172.21.15.19:0/835205524 learned_addr learned my addr 172.21.15.19:0/835205524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc36a000640 1 --1- 172.21.15.19:0/835205524 >> v1:172.21.15.169:6789/0 conn(0x7fc36c15a810 0x7fc36c0b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:43324/0 (socket says 172.21.15.19:43324) 2024-04-25T11:28:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc36aa00640 1 --2- 172.21.15.19:0/835205524 >> v2:172.21.15.169:3300/0 conn(0x7fc36c0cf2f0 0x7fc36c0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc369600640 1 -- 172.21.15.19:0/835205524 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121180178 0 0) 0x7fc35c001110 con 0x7fc36c15a810 2024-04-25T11:28:53.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc369600640 1 -- 172.21.15.19:0/835205524 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc350003520 con 0x7fc36c15a810 2024-04-25T11:28:53.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc36b400640 1 -- 172.21.15.19:0/835205524 >> v2:172.21.15.169:3300/0 conn(0x7fc36c0cf2f0 msgr2=0x7fc36c0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc36b400640 1 --2- 172.21.15.19:0/835205524 >> v2:172.21.15.169:3300/0 conn(0x7fc36c0cf2f0 0x7fc36c0b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc36b400640 1 -- 172.21.15.19:0/835205524 >> v1:172.21.15.169:6789/0 conn(0x7fc36c15a810 legacy=0x7fc36c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.577+0000 7fc36b400640 1 -- 172.21.15.19:0/835205524 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c0b77b0 con 0x7fc36c1608d0 2024-04-25T11:28:53.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.578+0000 7fc36aa00640 1 --2- 172.21.15.19:0/835205524 >> v2:172.21.15.169:3300/0 conn(0x7fc36c0cf2f0 0x7fc36c0b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:28:53.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.579+0000 7fc36b400640 1 --2- 172.21.15.19:0/835205524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c1608d0 0x7fc36c160ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc364009e50 tx=0x7fc36402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0bdf9d24e00fad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.580+0000 7fc369600640 1 -- 172.21.15.19:0/835205524 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364036070 con 0x7fc36c1608d0 2024-04-25T11:28:53.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.580+0000 7fc369600640 1 -- 172.21.15.19:0/835205524 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc364031600 con 0x7fc36c1608d0 2024-04-25T11:28:53.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.580+0000 7fc369600640 1 -- 172.21.15.19:0/835205524 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3640319d0 con 0x7fc36c1608d0 2024-04-25T11:28:53.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.580+0000 7fc36be00640 1 -- 172.21.15.19:0/835205524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c1608d0 msgr2=0x7fc36c160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.580+0000 7fc36be00640 1 --2- 172.21.15.19:0/835205524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c1608d0 0x7fc36c160ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc364009e50 tx=0x7fc36402a710 comp rx=0 tx=0).stop 2024-04-25T11:28:53.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.581+0000 7fc36be00640 1 -- 172.21.15.19:0/835205524 shutdown_connections 2024-04-25T11:28:53.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.581+0000 7fc36be00640 1 --2- 172.21.15.19:0/835205524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c1608d0 0x7fc36c160ce0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.581+0000 7fc36be00640 1 --2- 172.21.15.19:0/835205524 >> v2:172.21.15.169:3300/0 conn(0x7fc36c0cf2f0 0x7fc36c0b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.581+0000 7fc36be00640 1 -- 172.21.15.19:0/835205524 >> 172.21.15.19:0/835205524 conn(0x7fc36c0d1720 msgr2=0x7fc36c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:53.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.581+0000 7fc36be00640 1 -- 172.21.15.19:0/835205524 shutdown_connections 2024-04-25T11:28:53.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.581+0000 7fc36be00640 1 -- 172.21.15.19:0/835205524 wait complete. 2024-04-25T11:28:53.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.584+0000 7fc36be00640 1 Processor -- start 2024-04-25T11:28:53.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.584+0000 7fc36be00640 1 -- start start 2024-04-25T11:28:53.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.585+0000 7fc36be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c0cf2f0 0x7fc36c15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.585+0000 7fc36be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc36c15a810 0x7fc36c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.585+0000 7fc36be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c1608d0 0x7fc36c0d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.585+0000 7fc36be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c15ac20 con 0x7fc36c1608d0 2024-04-25T11:28:53.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.585+0000 7fc36be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c06f500 con 0x7fc36c15a810 2024-04-25T11:28:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.585+0000 7fc36be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc36c06fd00 con 0x7fc36c0cf2f0 2024-04-25T11:28:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c1608d0 0x7fc36c0d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c1608d0 0x7fc36c0d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43202/0 (socket says 172.21.15.19:43202) 2024-04-25T11:28:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 -- 172.21.15.19:0/1036412221 learned_addr learned my addr 172.21.15.19:0/1036412221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36aa00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c0cf2f0 0x7fc36c15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36a000640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc36c15a810 0x7fc36c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 -- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c0cf2f0 msgr2=0x7fc36c15f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c0cf2f0 0x7fc36c15f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 -- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc36c15a810 msgr2=0x7fc36c0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc36c15a810 0x7fc36c0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.586+0000 7fc36b400640 1 -- 172.21.15.19:0/1036412221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c15f9a0 con 0x7fc36c1608d0 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.587+0000 7fc36b400640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c1608d0 0x7fc36c0d0220 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc364009f70 tx=0x7fc364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.587+0000 7fc363400640 1 -- 172.21.15.19:0/1036412221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36403f020 con 0x7fc36c1608d0 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.587+0000 7fc363400640 1 -- 172.21.15.19:0/1036412221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc364037040 con 0x7fc36c1608d0 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.587+0000 7fc363400640 1 -- 172.21.15.19:0/1036412221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364036070 con 0x7fc36c1608d0 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.587+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c15feb0 con 0x7fc36c1608d0 2024-04-25T11:28:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.587+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c1f9e50 con 0x7fc36c1608d0 2024-04-25T11:28:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.590+0000 7fc363400640 1 -- 172.21.15.19:0/1036412221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc364042020 con 0x7fc36c1608d0 2024-04-25T11:28:53.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.592+0000 7fc363400640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3480744e0 0x7fc3480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.592+0000 7fc363400640 1 -- 172.21.15.19:0/1036412221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc3640b54a0 con 0x7fc36c1608d0 2024-04-25T11:28:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.593+0000 7fc36aa00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3480744e0 0x7fc3480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.593+0000 7fc378dee740 1 Processor -- start 2024-04-25T11:28:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.593+0000 7fc378dee740 1 -- start start 2024-04-25T11:28:53.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.594+0000 7fc36aa00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3480744e0 0x7fc3480769f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc354009a30 tx=0x7fc354008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc378dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f04f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7f05480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc378dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eae7e23600 con 0x55eae7b1d270 2024-04-25T11:28:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eae7dc0100 con 0x55eae7afc590 2024-04-25T11:28:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eae7e1f5c0 con 0x55eae7af43b0 2024-04-25T11:28:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc36aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc36aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43218/0 (socket says 172.21.15.19:43218) 2024-04-25T11:28:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc36aa00640 1 -- 172.21.15.19:0/523842058 learned_addr learned my addr 172.21.15.19:0/523842058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc36b400640 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7f05480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.595+0000 7fc36a000640 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f04f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36aa00640 1 -- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 msgr2=0x55eae7f04f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36aa00640 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f04f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36aa00640 1 -- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 msgr2=0x55eae7f05480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36aa00640 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7f05480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36aa00640 1 -- 172.21.15.19:0/523842058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae7b352d0 con 0x55eae7b1d270 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36a000640 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f04f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36b400640 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7f05480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc36aa00640 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eea210 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc354011d20 tx=0x7fc354014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307283671a5865c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc35b400640 1 -- 172.21.15.19:0/523842058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354011d50 con 0x55eae7b1d270 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc35b400640 1 -- 172.21.15.19:0/523842058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354026070 con 0x55eae7b1d270 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.596+0000 7fc35b400640 1 -- 172.21.15.19:0/523842058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354021640 con 0x55eae7b1d270 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 -- 172.21.15.19:0/523842058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 msgr2=0x55eae7eea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eea210 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc354011d20 tx=0x7fc354014000 comp rx=0 tx=0).stop 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 -- 172.21.15.19:0/523842058 shutdown_connections 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7f05480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f04f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 --2- 172.21.15.19:0/523842058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eea210 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 -- 172.21.15.19:0/523842058 >> 172.21.15.19:0/523842058 conn(0x55eae7b113c0 msgr2=0x55eae7ebf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 -- 172.21.15.19:0/523842058 shutdown_connections 2024-04-25T11:28:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.597+0000 7fc378dee740 1 -- 172.21.15.19:0/523842058 wait complete. 2024-04-25T11:28:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.598+0000 7fc378dee740 1 -- start start 2024-04-25T11:28:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.599+0000 7fc378dee740 1 Processor -- start 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.600+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7ee68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.600+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7afc590 0x55eae7efdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.600+0000 7fc378dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.600+0000 7fc378dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eae7daaf70 con 0x55eae7b1d270 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.600+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eae7f15b30 con 0x55eae7af43b0 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.600+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eae7efc400 con 0x55eae7afc590 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.600+0000 7fc378dee740 10 client.0 _ll_register_callbacks cb 0x55eae7e19480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:28:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43232/0 (socket says 172.21.15.19:43232) 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 -- 172.21.15.19:0/555755808 learned_addr learned my addr 172.21.15.19:0/555755808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36a000640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7afc590 0x55eae7efdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36aa00640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7ee68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 -- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7afc590 msgr2=0x55eae7efdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7afc590 0x55eae7efdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 -- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 msgr2=0x55eae7ee68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7ee68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 -- 172.21.15.19:0/555755808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae7bff6b0 con 0x55eae7b1d270 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36a000640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7afc590 0x55eae7efdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc36b400640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eed090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3640094a0 tx=0x7fc36407eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71e6db2e0f0c5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.601+0000 7fc34fe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc34fe00640 1 -- 172.21.15.19:0/555755808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3640b52c0 con 0x55eae7b1d270 2024-04-25T11:28:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc34fe00640 1 -- 172.21.15.19:0/555755808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc364087de0 con 0x55eae7b1d270 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc354026070 con 0x55eae7b1d270 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc34fe00640 1 -- 172.21.15.19:0/555755808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc36407fb10 con 0x55eae7b1d270 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc378dee740 10 client.5253 fetch_fsmap learned FSMap version 17 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55eae7f15870 con 0x55eae7b1d270 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc34fe00640 1 -- 172.21.15.19:0/555755808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354026070 con 0x55eae7b1d270 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc34fe00640 1 -- 172.21.15.19:0/555755808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fc36407e670 con 0x55eae7b1d270 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc378dee740 10 client.5253 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc378dee740 10 client.5253 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc378dee740 20 client.5253 populate_metadata read hostname 'smithi019' 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.602+0000 7fc378dee740 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7adbe40 0x55eae7eff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.603+0000 7fc378dee740 4 client.5253 mds_command: new command op to 5064 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.603+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 --> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eae7e31210 con 0x55eae7adbe40 2024-04-25T11:28:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.603+0000 7fc36aa00640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7adbe40 0x55eae7eff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.603+0000 7fc36aa00640 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7adbe40 0x55eae7eff5f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54eaad36a2b06ed server_cookie=2d66795a0fb8d3e2 in_seq=0 out_seq=0 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.603+0000 7fc34fe00640 10 client.5253 ms_handle_connect on v2:172.21.15.19:6833/3831883468 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.604+0000 7fc34fe00640 1 -- 172.21.15.19:0/555755808 <== mds.0 v2:172.21.15.19:6833/3831883468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eae7e31210 con 0x55eae7adbe40 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.604+0000 7fc34fe00640 10 client.5253 handle_command_reply: tid=0 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.604+0000 7fc378dee740 1 client.5253 shutdown 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 msgr2=0x55eae7eed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eed090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3640094a0 tx=0x7fc36407eb80 comp rx=0 tx=0).stop 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 shutdown_connections 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7adbe40 0x55eae7eff5f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7eed090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7afc590 0x55eae7efdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 --2- 172.21.15.19:0/555755808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7ee68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc34fe00640 0 client.5253 ms_handle_reset on v2:172.21.15.19:6833/3831883468 2024-04-25T11:28:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 >> 172.21.15.19:0/555755808 conn(0x55eae7b113c0 msgr2=0x55eae7ebf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 shutdown_connections 2024-04-25T11:28:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 1 -- 172.21.15.19:0/555755808 wait complete. 2024-04-25T11:28:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.605+0000 7fc378dee740 20 client.5253 trim_cache size 0 max 16384 2024-04-25T11:28:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.626+0000 7fc378dee740 1 Processor -- start 2024-04-25T11:28:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.626+0000 7fc378dee740 1 -- start start 2024-04-25T11:28:53.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.627+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7f13b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc378dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7afc590 0x55eae7efd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7b1d270 0x55eae7efde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc378dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c201d40 con 0x55eae7afc590 2024-04-25T11:28:53.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c147c50 con 0x55eae7af43b0 2024-04-25T11:28:53.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc36c1482a0 con 0x55eae7b1d270 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc36a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7afc590 0x55eae7efd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc36a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7afc590 0x55eae7efd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43246/0 (socket says 172.21.15.19:43246) 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc36a000640 1 -- 172.21.15.19:0/1213657104 learned_addr learned my addr 172.21.15.19:0/1213657104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc36aa00640 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7f13b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc36b400640 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7b1d270 0x55eae7efde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc36a000640 1 -- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7b1d270 msgr2=0x55eae7efde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.628+0000 7fc36a000640 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7b1d270 0x55eae7efde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.629+0000 7fc36a000640 1 -- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 msgr2=0x55eae7f13b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.629+0000 7fc36a000640 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7f13b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.629+0000 7fc36a000640 1 -- 172.21.15.19:0/1213657104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae7ebd8c0 con 0x55eae7afc590 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.629+0000 7fc36a000640 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7afc590 0x55eae7efd8c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc36c074630 tx=0x7fc35c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfa78a30c24f654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.629+0000 7fc35aa00640 1 -- 172.21.15.19:0/1213657104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0076f0 con 0x55eae7afc590 2024-04-25T11:28:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.629+0000 7fc35aa00640 1 -- 172.21.15.19:0/1213657104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35c019070 con 0x55eae7afc590 2024-04-25T11:28:53.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.629+0000 7fc35aa00640 1 -- 172.21.15.19:0/1213657104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c001600 con 0x55eae7afc590 2024-04-25T11:28:53.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 -- 172.21.15.19:0/1213657104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7afc590 msgr2=0x55eae7efd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7afc590 0x55eae7efd8c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc36c074630 tx=0x7fc35c000ed0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 -- 172.21.15.19:0/1213657104 shutdown_connections 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7b1d270 0x55eae7efde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7afc590 0x55eae7efd8c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 --2- 172.21.15.19:0/1213657104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7af43b0 0x55eae7f13b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 -- 172.21.15.19:0/1213657104 >> 172.21.15.19:0/1213657104 conn(0x55eae7b113c0 msgr2=0x55eae7eb5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 -- 172.21.15.19:0/1213657104 shutdown_connections 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.630+0000 7fc378dee740 1 -- 172.21.15.19:0/1213657104 wait complete. 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.631+0000 7fc378dee740 1 -- start start 2024-04-25T11:28:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.631+0000 7fc378dee740 1 Processor -- start 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f81bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc378dee740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7ee4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc378dee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7f04f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc378dee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eae7efe6b0 con 0x55eae7b1d270 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eae7daaf70 con 0x55eae7afc590 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc378dee740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eae7edc3d0 con 0x55eae7af43b0 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc378dee740 10 client.0 _ll_register_callbacks cb 0x7fc364031e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc36b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7f04f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc36b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7f04f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43256/0 (socket says 172.21.15.19:43256) 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.633+0000 7fc36b400640 1 -- 172.21.15.19:0/3320348438 learned_addr learned my addr 172.21.15.19:0/3320348438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36aa00640 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f81bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36a000640 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7ee4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36b400640 1 -- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 msgr2=0x55eae7f81bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36b400640 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f81bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36b400640 1 -- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 msgr2=0x55eae7ee4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36b400640 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7ee4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36b400640 1 -- 172.21.15.19:0/3320348438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae7f1f410 con 0x55eae7b1d270 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc36b400640 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7f04f60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc36407f110 tx=0x7fc3640b3800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e611ba9f13a30d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc34fe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc34fe00640 1 -- 172.21.15.19:0/3320348438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36407f1e0 con 0x55eae7b1d270 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc34fe00640 1 -- 172.21.15.19:0/3320348438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc364087fe0 con 0x55eae7b1d270 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.634+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35c019070 con 0x55eae7b1d270 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc34fe00640 1 -- 172.21.15.19:0/3320348438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc364031600 con 0x55eae7b1d270 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 10 client.5259 fetch_fsmap learned FSMap version 17 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55eae7f1e6c0 con 0x55eae7b1d270 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc34fe00640 1 -- 172.21.15.19:0/3320348438 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35c019070 con 0x55eae7b1d270 2024-04-25T11:28:53.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc34fe00640 1 -- 172.21.15.19:0/3320348438 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fc36407ad80 con 0x55eae7b1d270 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 10 client.5259 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 10 client.5259 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 20 client.5259 populate_metadata read hostname 'smithi019' 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7efdd20 0x55eae7ebdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 4 client.5259 mds_command: new command op to 5064 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 --> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] -- command(tid 0: {"prefix": "status"}) -- 0x55eae7e2b080 con 0x55eae7efdd20 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.635+0000 7fc36aa00640 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7efdd20 0x55eae7ebdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.636+0000 7fc36aa00640 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7efdd20 0x55eae7ebdd20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c68de28e000983d server_cookie=97c102f2a24f9a73 in_seq=0 out_seq=0 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.636+0000 7fc34fe00640 10 client.5259 ms_handle_connect on v2:172.21.15.19:6833/3831883468 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.636+0000 7fc34fe00640 1 -- 172.21.15.19:0/3320348438 <== mds.0 v2:172.21.15.19:6833/3831883468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55eae7e2b080 con 0x55eae7efdd20 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.636+0000 7fc34fe00640 10 client.5259 handle_command_reply: tid=0 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.636+0000 7fc378dee740 1 client.5259 shutdown 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 msgr2=0x55eae7f04f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7f04f60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc36407f110 tx=0x7fc3640b3800 comp rx=0 tx=0).stop 2024-04-25T11:28:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 shutdown_connections 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:6833/3831883468,v1:172.21.15.19:6835/3831883468] conn(0x55eae7efdd20 0x55eae7ebdd20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eae7afc590 0x55eae7ee4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eae7b1d270 0x55eae7f04f60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 --2- 172.21.15.19:0/3320348438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eae7af43b0 0x55eae7f81bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc34fe00640 0 client.5259 ms_handle_reset on v2:172.21.15.19:6833/3831883468 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 >> 172.21.15.19:0/3320348438 conn(0x55eae7b113c0 msgr2=0x55eae7eedec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 shutdown_connections 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 1 -- 172.21.15.19:0/3320348438 wait complete. 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.637+0000 7fc378dee740 20 client.5259 trim_cache size 0 max 16384 2024-04-25T11:28:53.630 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5064, 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3456434379999997, 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 17, 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 35, 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 35, 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 17.189192798000001 2024-04-25T11:28:53.631 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:28:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3480744e0 msgr2=0x7fc3480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3480744e0 0x7fc3480769f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc354009a30 tx=0x7fc354008040 comp rx=0 tx=0).stop 2024-04-25T11:28:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c1608d0 msgr2=0x7fc36c0d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c1608d0 0x7fc36c0d0220 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc364009f70 tx=0x7fc364004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 shutdown_connections 2024-04-25T11:28:53.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3480744e0 0x7fc3480769f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc36c15a810 0x7fc36c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc36c1608d0 0x7fc36c0d0220 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 --2- 172.21.15.19:0/1036412221 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36c0cf2f0 0x7fc36c15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 >> 172.21.15.19:0/1036412221 conn(0x7fc36c0d1720 msgr2=0x7fc36c159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 shutdown_connections 2024-04-25T11:28:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.638+0000 7fc36be00640 1 -- 172.21.15.19:0/1036412221 wait complete. 2024-04-25T11:28:53.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:28:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.951+0000 7f4811200640 1 Processor -- start 2024-04-25T11:28:53.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.952+0000 7f4811200640 1 -- start start 2024-04-25T11:28:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.954+0000 7f4811200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f480c142fe0 0x7f480c168e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.954+0000 7f4811200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f480c141210 con 0x7f480c140e00 2024-04-25T11:28:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.954+0000 7f4811200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f480c141510 con 0x7f480c06d5d0 2024-04-25T11:28:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.955+0000 7f4811200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f480c141b40 con 0x7f480c142fe0 2024-04-25T11:28:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.955+0000 7f480b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f480c140e00 0x7f480c143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51590/0 (socket says 172.21.15.19:51590) 2024-04-25T11:28:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.955+0000 7f480b400640 1 -- 172.21.15.19:0/2705269164 learned_addr learned my addr 172.21.15.19:0/2705269164 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.955+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487349264 0 0) 0x7f480c141210 con 0x7f480c140e00 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.955+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47f0003520 con 0x7f480c140e00 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.955+0000 7f480be00640 1 --2- 172.21.15.19:0/2705269164 >> v2:172.21.15.169:3300/0 conn(0x7f480c142fe0 0x7f480c168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 710333501 0 0) 0x7f47f0003520 con 0x7f480c140e00 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 >> v2:172.21.15.169:3300/0 conn(0x7f480c142fe0 msgr2=0x7f480c168e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 --2- 172.21.15.19:0/2705269164 >> v2:172.21.15.169:3300/0 conn(0x7f480c142fe0 0x7f480c168e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 >> v1:172.21.15.169:6789/0 conn(0x7f480c06d5d0 legacy=0x7f480c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1440f0 con 0x7f480c140e00 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f47fc001670 con 0x7f480c140e00 2024-04-25T11:28:53.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 >> v1:172.21.15.19:6789/0 conn(0x7f480c140e00 legacy=0x7f480c143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f00039b0 0x7f47f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f47f0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47f00051c0 0x7f47f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f480c141b40 con 0x7f47f0004800 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47f000c160 con 0x7f47f00039b0 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f47f000c360 con 0x7f47f00051c0 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.956+0000 7f4803e00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f47f0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f480b400640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f00039b0 0x7f47f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f480be00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47f00051c0 0x7f47f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f4803e00640 1 -- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47f00051c0 msgr2=0x7f47f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f4803e00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47f00051c0 0x7f47f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f4803e00640 1 -- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f00039b0 msgr2=0x7f47f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f4803e00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47f00039b0 0x7f47f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f4803e00640 1 -- 172.21.15.19:0/2705269164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f0003520 con 0x7f47f0004800 2024-04-25T11:28:53.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.957+0000 7f480be00640 1 -- 172.21.15.19:0/2705269164 reap_dead start 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.958+0000 7f4803e00640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f47f0004c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001750 tx=0x7f47f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf7129d63f55bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.958+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f4001370 con 0x7f47f0004800 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.958+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f40052d0 con 0x7f47f0004800 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.958+0000 7f480aa00640 1 -- 172.21.15.19:0/2705269164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f402abc0 con 0x7f47f0004800 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.959+0000 7f4811200640 1 -- 172.21.15.19:0/2705269164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 msgr2=0x7f47f0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.959+0000 7f4811200640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f47f0004c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001750 tx=0x7f47f0005830 comp rx=0 tx=0).stop 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.959+0000 7f4811200640 1 -- 172.21.15.19:0/2705269164 shutdown_connections 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.959+0000 7f4811200640 1 --2- 172.21.15.19:0/2705269164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f47f0004c10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.959+0000 7f4811200640 1 -- 172.21.15.19:0/2705269164 >> 172.21.15.19:0/2705269164 conn(0x7f480c072ee0 msgr2=0x7f480c0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.959+0000 7f4811200640 1 -- 172.21.15.19:0/2705269164 shutdown_connections 2024-04-25T11:28:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.959+0000 7f4811200640 1 -- 172.21.15.19:0/2705269164 wait complete. 2024-04-25T11:28:53.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.962+0000 7f4811200640 1 Processor -- start 2024-04-25T11:28:53.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.962+0000 7f4811200640 1 -- start start 2024-04-25T11:28:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.963+0000 7f4811200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f480c1fb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.963+0000 7f4811200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f480c146df0 0x7f480c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.963+0000 7f4811200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f480c0b61a0 0x7f480c133c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.963+0000 7f4811200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0d1490 con 0x7f47f0004800 2024-04-25T11:28:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.963+0000 7f4811200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1417d0 con 0x7f480c0b61a0 2024-04-25T11:28:53.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.963+0000 7f4811200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f480c06d9e0 con 0x7f480c146df0 2024-04-25T11:28:53.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f480c1fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f480c1fb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37292/0 (socket says 172.21.15.19:37292) 2024-04-25T11:28:53.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 -- 172.21.15.19:0/1077751665 learned_addr learned my addr 172.21.15.19:0/1077751665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480be00640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f480c0b61a0 0x7f480c133c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 -- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f480c146df0 msgr2=0x7f480c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f480c146df0 0x7f480c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 -- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f480c0b61a0 msgr2=0x7f480c133c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f480c0b61a0 0x7f480c133c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480b400640 1 -- 172.21.15.19:0/1077751665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1448e0 con 0x7f47f0004800 2024-04-25T11:28:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.964+0000 7f480be00640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f480c0b61a0 0x7f480c133c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.965+0000 7f480b400640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f480c1fb100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e730 tx=0x7f47fc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.965+0000 7f4803400640 1 -- 172.21.15.19:0/1077751665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00c6c0 con 0x7f47f0004800 2024-04-25T11:28:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.965+0000 7f4803400640 1 -- 172.21.15.19:0/1077751665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47fc018070 con 0x7f47f0004800 2024-04-25T11:28:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.965+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c144bc0 con 0x7f47f0004800 2024-04-25T11:28:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.965+0000 7f4803400640 1 -- 172.21.15.19:0/1077751665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00caa0 con 0x7f47f0004800 2024-04-25T11:28:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.965+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c145180 con 0x7f47f0004800 2024-04-25T11:28:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.966+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c050740 con 0x7f47f0004800 2024-04-25T11:28:53.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.967+0000 7f4803400640 1 -- 172.21.15.19:0/1077751665 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f47fc01d460 con 0x7f47f0004800 2024-04-25T11:28:53.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.968+0000 7f4803400640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47e80744b0 0x7f47e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:53.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.969+0000 7f4803400640 1 -- 172.21.15.19:0/1077751665 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f47fc095270 con 0x7f47f0004800 2024-04-25T11:28:53.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.969+0000 7f4803e00640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47e80744b0 0x7f47e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.970+0000 7f4803e00640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47e80744b0 0x7f47e80769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f4032000 tx=0x7f47f402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:53.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:53.974+0000 7f4803400640 1 -- 172.21.15.19:0/1077751665 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47fc05f9f0 con 0x7f47f0004800 2024-04-25T11:28:54.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.220+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f480c0b6f30 con 0x7f47f0004800 2024-04-25T11:28:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.221+0000 7f4803400640 1 -- 172.21.15.19:0/1077751665 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f47fc05f2f0 con 0x7f47f0004800 2024-04-25T11:28:54.212 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47e80744b0 msgr2=0x7f47e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47e80744b0 0x7f47e80769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f4032000 tx=0x7f47f402c000 comp rx=0 tx=0).stop 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 msgr2=0x7f480c1fb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f480c1fb100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e730 tx=0x7f47fc00ebf0 comp rx=0 tx=0).stop 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 shutdown_connections 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47e80744b0 0x7f47e80769c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f480c0b61a0 0x7f480c133c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f480c146df0 0x7f480c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 --2- 172.21.15.19:0/1077751665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47f0004800 0x7f480c1fb100 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 >> 172.21.15.19:0/1077751665 conn(0x7f480c072ee0 msgr2=0x7f480c06ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.224+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 shutdown_connections 2024-04-25T11:28:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54.225+0000 7f4811200640 1 -- 172.21.15.19:0/1077751665 wait complete. 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:28:54.241 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:28:54.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:54.241 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:28:54.292 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:28:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:28:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42539sec preferred_lft 42539sec 2024-04-25T11:28:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:28:54.294 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:28:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:28:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:28:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:28:54.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:54.296 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:28:54.377 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:28:54.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:54.382 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:28:54.437 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:28:54.437 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:28:54.438 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:28:54.438 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:28:54.438 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:28:54.438 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:28:54.438 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:28:54.438 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:28:54.438 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:28:54.438 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:28:54.438 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:28:54.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:28:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:54.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:54.651 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:28:54.699 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:28:54.699 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:28:54.700 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:28:54.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:54.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:28:54.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:28:54.786 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:28:54.841 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:28:54.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:28:54.842 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:28:54.896 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:28:54.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:54.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:28:55.109 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:28:55.109 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:28:55.156 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:28:55.156 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:28:55.156 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:28:55.156 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:28:55.156 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:28:55.159 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:28:55.159 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:28:55.159 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:28:55.159 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:28:55.159 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:28:55.165 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:28:55.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:55.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:28:55.244 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:28:55.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:28:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:55.389 INFO:teuthology.orchestra.run.smithi019.stdout:client5268 2024-04-25T11:28:55.389 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:55.389 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:55.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:55.445 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T11:28:55.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:55.513 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T11:28:55.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:55.575 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T11:28:55.633 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:28:55.634 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:28:55.634 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:28:55.634 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:28:55.634 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T11:28:55.634 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:28:55.779 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-04-25T11:28:55.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:28:55.793 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:55.794 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:55.893 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:28:55.893 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:28:55.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:28:56.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.245+0000 7f3aeb200640 1 Processor -- start 2024-04-25T11:28:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.245+0000 7f3aeb200640 1 -- start start 2024-04-25T11:28:56.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.247+0000 7f3aeb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.247+0000 7f3aeb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3aec06d560 0x7f3aec0b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aeb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aec071230 con 0x7f3aec06f180 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aeb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3aec06f8f0 con 0x7f3aec06d560 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aeb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec06fbd0 con 0x7f3aec070e20 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aea800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3aec06d560 0x7f3aec0b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aea800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3aec06d560 0x7f3aec0b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37302/0 (socket says 172.21.15.19:37302) 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aea800640 1 -- 172.21.15.19:0/3273220787 learned_addr learned my addr 172.21.15.19:0/3273220787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3ae9e00640 1 --2- 172.21.15.19:0/3273220787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aea800640 1 -- 172.21.15.19:0/3273220787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 msgr2=0x7f3aec15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aea800640 1 --2- 172.21.15.19:0/3273220787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.248+0000 7f3aea800640 1 -- 172.21.15.19:0/3273220787 >> v1:172.21.15.169:6789/0 conn(0x7f3aec06f180 legacy=0x7f3aec06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.249+0000 7f3aea800640 1 -- 172.21.15.19:0/3273220787 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec06feb0 con 0x7f3aec06d560 2024-04-25T11:28:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.250+0000 7f3aea800640 1 --2- 172.21.15.19:0/3273220787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec06d560 0x7f3aec0b5e20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d150 tx=0x7f3ae402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc43415cf5e6dde5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.250+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3273220787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400b2e0 con 0x7f3aec06d560 2024-04-25T11:28:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.250+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3273220787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae4009600 con 0x7f3aec06d560 2024-04-25T11:28:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.250+0000 7f3ae8a00640 1 -- 172.21.15.19:0/3273220787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400b480 con 0x7f3aec06d560 2024-04-25T11:28:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 -- 172.21.15.19:0/3273220787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec06d560 msgr2=0x7f3aec0b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 --2- 172.21.15.19:0/3273220787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec06d560 0x7f3aec0b5e20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d150 tx=0x7f3ae402c8e0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 -- 172.21.15.19:0/3273220787 shutdown_connections 2024-04-25T11:28:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 --2- 172.21.15.19:0/3273220787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec06d560 0x7f3aec0b5e20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 --2- 172.21.15.19:0/3273220787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 -- 172.21.15.19:0/3273220787 >> 172.21.15.19:0/3273220787 conn(0x7f3aec0d1720 msgr2=0x7f3aec0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 -- 172.21.15.19:0/3273220787 shutdown_connections 2024-04-25T11:28:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.251+0000 7f3aeb200640 1 -- 172.21.15.19:0/3273220787 wait complete. 2024-04-25T11:28:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.253+0000 7f3aeb200640 1 Processor -- start 2024-04-25T11:28:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.254+0000 7f3aeb200640 1 -- start start 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.255+0000 7f3aeb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aec06f180 0x7f3aec154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.255+0000 7f3aeb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.255+0000 7f3aeb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec14ee70 0x7f3aec1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.255+0000 7f3aeb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec06c5c0 con 0x7f3aec14ee70 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.255+0000 7f3aeb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec06f590 con 0x7f3aec06f180 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.255+0000 7f3aeb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3aec06c740 con 0x7f3aec070e20 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec14ee70 0x7f3aec1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec14ee70 0x7f3aec1f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37312/0 (socket says 172.21.15.19:37312) 2024-04-25T11:28:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 -- 172.21.15.19:0/2286977073 learned_addr learned my addr 172.21.15.19:0/2286977073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3ae9e00640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aec06f180 0x7f3aec154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3ae9400640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec14e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 -- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 msgr2=0x7f3aec14e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec14e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 -- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aec06f180 msgr2=0x7f3aec154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aec06f180 0x7f3aec154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3aea800640 1 -- 172.21.15.19:0/2286977073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec152dc0 con 0x7f3aec14ee70 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.256+0000 7f3ae9e00640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aec06f180 0x7f3aec154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.257+0000 7f3aea800640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec14ee70 0x7f3aec1f5510 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d270 tx=0x7f3ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.257+0000 7f3ae2000640 1 -- 172.21.15.19:0/2286977073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4041030 con 0x7f3aec14ee70 2024-04-25T11:28:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.257+0000 7f3ae2000640 1 -- 172.21.15.19:0/2286977073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae403d650 con 0x7f3aec14ee70 2024-04-25T11:28:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.257+0000 7f3ae2000640 1 -- 172.21.15.19:0/2286977073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae403da00 con 0x7f3aec14ee70 2024-04-25T11:28:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.257+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec1530a0 con 0x7f3aec14ee70 2024-04-25T11:28:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.257+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec153660 con 0x7f3aec14ee70 2024-04-25T11:28:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.259+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aec152b30 con 0x7f3aec14ee70 2024-04-25T11:28:56.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.260+0000 7f3ae2000640 1 -- 172.21.15.19:0/2286977073 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ae4044030 con 0x7f3aec14ee70 2024-04-25T11:28:56.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.261+0000 7f3ae2000640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3acc0744b0 0x7f3acc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.261+0000 7f3ae2000640 1 -- 172.21.15.19:0/2286977073 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae4030020 con 0x7f3aec14ee70 2024-04-25T11:28:56.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.262+0000 7f3ae9e00640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3acc0744b0 0x7f3acc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.263+0000 7f3ae9e00640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3acc0744b0 0x7f3acc0769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00b820 tx=0x7f3adc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:56.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.266+0000 7f3ae2000640 1 -- 172.21.15.19:0/2286977073 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ae404b030 con 0x7f3aec14ee70 2024-04-25T11:28:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.475+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3aec0d1c50 con 0x7f3aec14ee70 2024-04-25T11:28:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.478+0000 7f3ae2000640 1 -- 172.21.15.19:0/2286977073 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16609 (secure 0 0 0) 0x7f3ae4082600 con 0x7f3aec14ee70 2024-04-25T11:28:56.470 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:56.470 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:28:47.266661+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:28:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3acc0744b0 msgr2=0x7f3acc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3acc0744b0 0x7f3acc0769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00b820 tx=0x7f3adc008040 comp rx=0 tx=0).stop 2024-04-25T11:28:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec14ee70 msgr2=0x7f3aec1f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec14ee70 0x7f3aec1f5510 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400d270 tx=0x7f3ae4004a40 comp rx=0 tx=0).stop 2024-04-25T11:28:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 shutdown_connections 2024-04-25T11:28:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3acc0744b0 0x7f3acc0769c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3aec14ee70 0x7f3aec1f5510 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3aec070e20 0x7f3aec14e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 --2- 172.21.15.19:0/2286977073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3aec06f180 0x7f3aec154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 >> 172.21.15.19:0/2286977073 conn(0x7f3aec0d1720 msgr2=0x7f3aec158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 shutdown_connections 2024-04-25T11:28:56.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.482+0000 7f3aeb200640 1 -- 172.21.15.19:0/2286977073 wait complete. 2024-04-25T11:28:56.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:28:43.580410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:28:44.810835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:28:56.501 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:28:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.798+0000 7fa3e7e00640 1 Processor -- start 2024-04-25T11:28:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.798+0000 7fa3e7e00640 1 -- start start 2024-04-25T11:28:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.800+0000 7fa3e7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa3e80b60c0 0x7fa3e816f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.800+0000 7fa3e7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e814aed0 con 0x7fa3e814aac0 2024-04-25T11:28:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.800+0000 7fa3e7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e806f460 con 0x7fa3e806d5d0 2024-04-25T11:28:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.800+0000 7fa3e7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e806f740 con 0x7fa3e80b60c0 2024-04-25T11:28:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.800+0000 7fa3e6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa3e814aac0 0x7fa3e81655a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51600/0 (socket says 172.21.15.19:51600) 2024-04-25T11:28:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.800+0000 7fa3e6a00640 1 -- 172.21.15.19:0/3603213339 learned_addr learned my addr 172.21.15.19:0/3603213339 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.800+0000 7fa3e7400640 1 --2- 172.21.15.19:0/3603213339 >> v2:172.21.15.19:3300/0 conn(0x7fa3e80b60c0 0x7fa3e816f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.801+0000 7fa3e6000640 1 -- 172.21.15.19:0/3603213339 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036179179 0 0) 0x7fa3e814aed0 con 0x7fa3e814aac0 2024-04-25T11:28:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.801+0000 7fa3e6000640 1 -- 172.21.15.19:0/3603213339 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3cc003520 con 0x7fa3e814aac0 2024-04-25T11:28:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.801+0000 7fa3e7400640 1 -- 172.21.15.19:0/3603213339 >> v1:172.21.15.169:6789/0 conn(0x7fa3e806d5d0 legacy=0x7fa3e806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.801+0000 7fa3e7400640 1 -- 172.21.15.19:0/3603213339 >> v1:172.21.15.19:6789/0 conn(0x7fa3e814aac0 legacy=0x7fa3e81655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.801+0000 7fa3e7400640 1 -- 172.21.15.19:0/3603213339 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e806fa20 con 0x7fa3e80b60c0 2024-04-25T11:28:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.802+0000 7fa3e7400640 1 --2- 172.21.15.19:0/3603213339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e816f4a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009d90 tx=0x7fa3e002a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b740db4f2b8466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.802+0000 7fa3e6000640 1 -- 172.21.15.19:0/3603213339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0007740 con 0x7fa3e80b60c0 2024-04-25T11:28:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.802+0000 7fa3e6000640 1 -- 172.21.15.19:0/3603213339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e00078e0 con 0x7fa3e80b60c0 2024-04-25T11:28:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.802+0000 7fa3e6000640 1 -- 172.21.15.19:0/3603213339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0007cb0 con 0x7fa3e80b60c0 2024-04-25T11:28:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.803+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3603213339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 msgr2=0x7fa3e816f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.803+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3603213339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e816f4a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009d90 tx=0x7fa3e002a580 comp rx=0 tx=0).stop 2024-04-25T11:28:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.803+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3603213339 shutdown_connections 2024-04-25T11:28:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.803+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3603213339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e816f4a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.803+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3603213339 >> 172.21.15.19:0/3603213339 conn(0x7fa3e8072f40 msgr2=0x7fa3e8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.804+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3603213339 shutdown_connections 2024-04-25T11:28:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.804+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3603213339 wait complete. 2024-04-25T11:28:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.806+0000 7fa3e7e00640 1 Processor -- start 2024-04-25T11:28:56.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.806+0000 7fa3e7e00640 1 -- start start 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e806d5d0 0x7fa3e8161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e814b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e814aac0 0x7fa3e8168110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3e7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e80b4a90 con 0x7fa3e80b60c0 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3e7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e806ffe0 con 0x7fa3e806d5d0 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3e7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e806f0c0 con 0x7fa3e814aac0 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e814b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e814b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37328/0 (socket says 172.21.15.19:37328) 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3dfe00640 1 -- 172.21.15.19:0/3620057255 learned_addr learned my addr 172.21.15.19:0/3620057255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.808+0000 7fa3e6a00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e806d5d0 0x7fa3e8161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3e7400640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e814aac0 0x7fa3e8168110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3dfe00640 1 -- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e814aac0 msgr2=0x7fa3e8168110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3dfe00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e814aac0 0x7fa3e8168110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3dfe00640 1 -- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e806d5d0 msgr2=0x7fa3e8161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3dfe00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e806d5d0 0x7fa3e8161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3dfe00640 1 -- 172.21.15.19:0/3620057255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e814bea0 con 0x7fa3e80b60c0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3e7400640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e814aac0 0x7fa3e8168110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3dfe00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e814b960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000b3e0 tx=0x7fa3d000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3df400640 1 -- 172.21.15.19:0/3620057255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0015020 con 0x7fa3e80b60c0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3df400640 1 -- 172.21.15.19:0/3620057255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d0011830 con 0x7fa3e80b60c0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3df400640 1 -- 172.21.15.19:0/3620057255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0011c20 con 0x7fa3e80b60c0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.809+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e814d940 con 0x7fa3e80b60c0 2024-04-25T11:28:56.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.810+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e814df00 con 0x7fa3e80b60c0 2024-04-25T11:28:56.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.811+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e8076c60 con 0x7fa3e80b60c0 2024-04-25T11:28:56.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.812+0000 7fa3df400640 1 -- 172.21.15.19:0/3620057255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa3d0018020 con 0x7fa3e80b60c0 2024-04-25T11:28:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.814+0000 7fa3df400640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3c4074530 0x7fa3c4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:56.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.814+0000 7fa3df400640 1 -- 172.21.15.19:0/3620057255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa3d0094ff0 con 0x7fa3e80b60c0 2024-04-25T11:28:56.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.815+0000 7fa3e6a00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3c4074530 0x7fa3c4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:56.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.816+0000 7fa3e6a00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3c4074530 0x7fa3c4076a40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005880 tx=0x7fa3d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:56.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:56.819+0000 7fa3df400640 1 -- 172.21.15.19:0/3620057255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3d001f020 con 0x7fa3e80b60c0 2024-04-25T11:28:57.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.051+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3e8070e70 con 0x7fa3e80b60c0 2024-04-25T11:28:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.051+0000 7fa3df400640 1 -- 172.21.15.19:0/3620057255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fa3d005f8e0 con 0x7fa3e80b60c0 2024-04-25T11:28:57.042 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3c4074530 msgr2=0x7fa3c4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3c4074530 0x7fa3c4076a40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005880 tx=0x7fa3d8009040 comp rx=0 tx=0).stop 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 msgr2=0x7fa3e814b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e814b960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000b3e0 tx=0x7fa3d000b8a0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 shutdown_connections 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3c4074530 0x7fa3c4076a40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3e814aac0 0x7fa3e8168110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3e80b60c0 0x7fa3e814b960 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 --2- 172.21.15.19:0/3620057255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3e806d5d0 0x7fa3e8161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 >> 172.21.15.19:0/3620057255 conn(0x7fa3e8072f40 msgr2=0x7fa3e8163fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:57.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 shutdown_connections 2024-04-25T11:28:57.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.054+0000 7fa3e7e00640 1 -- 172.21.15.19:0/3620057255 wait complete. 2024-04-25T11:28:57.072 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:28:57.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.369+0000 7f148a800640 1 Processor -- start 2024-04-25T11:28:57.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.370+0000 7f148a800640 1 -- start start 2024-04-25T11:28:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f148a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1484144340 0x7f148406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f148a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f148415ee60 0x7f148415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f148a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f148406df20 con 0x7f148406db10 2024-04-25T11:28:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f148a800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f148415f270 con 0x7f148415ee60 2024-04-25T11:28:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f148a800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1484144750 con 0x7f1484144340 2024-04-25T11:28:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f1489e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f148415ee60 0x7f148415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f1489e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f148415ee60 0x7f148415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37334/0 (socket says 172.21.15.19:37334) 2024-04-25T11:28:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.372+0000 7f1489e00640 1 -- 172.21.15.19:0/2195894971 learned_addr learned my addr 172.21.15.19:0/2195894971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.373+0000 7f1489400640 1 --2- 172.21.15.19:0/2195894971 >> v2:172.21.15.169:3300/0 conn(0x7f1484144340 0x7f148406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.373+0000 7f1489e00640 1 -- 172.21.15.19:0/2195894971 >> v2:172.21.15.169:3300/0 conn(0x7f1484144340 msgr2=0x7f148406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.373+0000 7f1489e00640 1 --2- 172.21.15.19:0/2195894971 >> v2:172.21.15.169:3300/0 conn(0x7f1484144340 0x7f148406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.373+0000 7f1489e00640 1 -- 172.21.15.19:0/2195894971 >> v1:172.21.15.169:6789/0 conn(0x7f148406db10 legacy=0x7f148415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.373+0000 7f1489e00640 1 -- 172.21.15.19:0/2195894971 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1484070000 con 0x7f148415ee60 2024-04-25T11:28:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.374+0000 7f1489e00640 1 --2- 172.21.15.19:0/2195894971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f148415f4a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f14840b6e30 tx=0x7f147c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a239a1e2628224eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.374+0000 7f1483e00640 1 -- 172.21.15.19:0/2195894971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c037070 con 0x7f148415ee60 2024-04-25T11:28:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.374+0000 7f1483e00640 1 -- 172.21.15.19:0/2195894971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c032440 con 0x7f148415ee60 2024-04-25T11:28:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.374+0000 7f1483e00640 1 -- 172.21.15.19:0/2195894971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c032810 con 0x7f148415ee60 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.375+0000 7f148a800640 1 -- 172.21.15.19:0/2195894971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 msgr2=0x7f148415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.375+0000 7f148a800640 1 --2- 172.21.15.19:0/2195894971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f148415f4a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f14840b6e30 tx=0x7f147c02a230 comp rx=0 tx=0).stop 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.376+0000 7f148a800640 1 -- 172.21.15.19:0/2195894971 shutdown_connections 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.376+0000 7f148a800640 1 --2- 172.21.15.19:0/2195894971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f148415f4a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.376+0000 7f148a800640 1 --2- 172.21.15.19:0/2195894971 >> v2:172.21.15.169:3300/0 conn(0x7f1484144340 0x7f148406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.376+0000 7f148a800640 1 -- 172.21.15.19:0/2195894971 >> 172.21.15.19:0/2195894971 conn(0x7f1484072f40 msgr2=0x7f1484073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.376+0000 7f148a800640 1 -- 172.21.15.19:0/2195894971 shutdown_connections 2024-04-25T11:28:57.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.376+0000 7f148a800640 1 -- 172.21.15.19:0/2195894971 wait complete. 2024-04-25T11:28:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.378+0000 7f148a800640 1 Processor -- start 2024-04-25T11:28:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.378+0000 7f148a800640 1 -- start start 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f148a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148406db10 0x7f14841f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f148a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1484144340 0x7f1484133950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f148a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f1484133e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f148a800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f148406f800 con 0x7f148415ee60 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f148a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f148406f520 con 0x7f1484144340 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f148a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f148406fae0 con 0x7f148406db10 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f1489e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f1484133e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f1489e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f1484133e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37346/0 (socket says 172.21.15.19:37346) 2024-04-25T11:28:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f1489e00640 1 -- 172.21.15.19:0/3248831210 learned_addr learned my addr 172.21.15.19:0/3248831210 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f1489400640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148406db10 0x7f14841f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f1488a00640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1484144340 0x7f1484133950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.380+0000 7f1489e00640 1 -- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148406db10 msgr2=0x7f14841f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1489e00640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148406db10 0x7f14841f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1489e00640 1 -- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1484144340 msgr2=0x7f1484133950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1489e00640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1484144340 0x7f1484133950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1489e00640 1 -- 172.21.15.19:0/3248831210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14840af5a0 con 0x7f148415ee60 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1489400640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148406db10 0x7f14841f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1489e00640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f1484133e90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f147c02a270 tx=0x7f147c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1481600640 1 -- 172.21.15.19:0/3248831210 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c041020 con 0x7f148415ee60 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1481600640 1 -- 172.21.15.19:0/3248831210 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c0306c0 con 0x7f148415ee60 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f1481600640 1 -- 172.21.15.19:0/3248831210 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c037040 con 0x7f148415ee60 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14840af880 con 0x7f148415ee60 2024-04-25T11:28:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.381+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14840afe40 con 0x7f148415ee60 2024-04-25T11:28:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.383+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1484076c60 con 0x7f148415ee60 2024-04-25T11:28:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.384+0000 7f1481600640 1 -- 172.21.15.19:0/3248831210 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f147c044020 con 0x7f148415ee60 2024-04-25T11:28:57.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.385+0000 7f1481600640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1464074500 0x7f1464076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.385+0000 7f1481600640 1 -- 172.21.15.19:0/3248831210 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f147c0b5aa0 con 0x7f148415ee60 2024-04-25T11:28:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.386+0000 7f1489400640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1464074500 0x7f1464076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.387+0000 7f1489400640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1464074500 0x7f1464076a10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1478009f50 tx=0x7f147800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.390+0000 7f1481600640 1 -- 172.21.15.19:0/3248831210 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f147c04b030 con 0x7f148415ee60 2024-04-25T11:28:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.597+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14840b7920 con 0x7f148415ee60 2024-04-25T11:28:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.598+0000 7f1481600640 1 -- 172.21.15.19:0/3248831210 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f148406df20 con 0x7f148415ee60 2024-04-25T11:28:57.589 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:28:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1464074500 msgr2=0x7f1464076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1464074500 0x7f1464076a10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1478009f50 tx=0x7f147800b040 comp rx=0 tx=0).stop 2024-04-25T11:28:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 msgr2=0x7f1484133e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f1484133e90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f147c02a270 tx=0x7f147c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 shutdown_connections 2024-04-25T11:28:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1464074500 0x7f1464076a10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1484144340 0x7f1484133950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f148415ee60 0x7f1484133e90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 --2- 172.21.15.19:0/3248831210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f148406db10 0x7f14841f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 >> 172.21.15.19:0/3248831210 conn(0x7f1484072f40 msgr2=0x7f148413fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 shutdown_connections 2024-04-25T11:28:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.601+0000 7f148a800640 1 -- 172.21.15.19:0/3248831210 wait complete. 2024-04-25T11:28:57.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:28:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.899+0000 7f43cf600640 1 Processor -- start 2024-04-25T11:28:57.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.899+0000 7f43cf600640 1 -- start start 2024-04-25T11:28:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.901+0000 7f43cf600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80cd9a0 0x7f43c815f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.901+0000 7f43cf600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f43c806dce0 0x7f43c815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.901+0000 7f43cf600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c80cdea0 con 0x7f43c80b0af0 2024-04-25T11:28:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.901+0000 7f43cf600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f43c80ceab0 con 0x7f43c806dce0 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.901+0000 7f43cf600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43c80ce180 con 0x7f43c80cd9a0 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.901+0000 7f43c7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f43c80b0af0 0x7f43c806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51610/0 (socket says 172.21.15.19:51610) 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.901+0000 7f43c7e00640 1 -- 172.21.15.19:0/3071249523 learned_addr learned my addr 172.21.15.19:0/3071249523 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.902+0000 7f43cd600640 1 --2- 172.21.15.19:0/3071249523 >> v2:172.21.15.19:3300/0 conn(0x7f43c806dce0 0x7f43c815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.902+0000 7f43c7400640 1 -- 172.21.15.19:0/3071249523 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 161438666 0 0) 0x7f43c80cdea0 con 0x7f43c80b0af0 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.902+0000 7f43c7400640 1 -- 172.21.15.19:0/3071249523 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43ac003520 con 0x7f43c80b0af0 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.902+0000 7f43cd600640 1 -- 172.21.15.19:0/3071249523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80cd9a0 msgr2=0x7f43c815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.902+0000 7f43cd600640 1 --2- 172.21.15.19:0/3071249523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80cd9a0 0x7f43c815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.902+0000 7f43cd600640 1 -- 172.21.15.19:0/3071249523 >> v1:172.21.15.19:6789/0 conn(0x7f43c80b0af0 legacy=0x7f43c806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.902+0000 7f43cd600640 1 -- 172.21.15.19:0/3071249523 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c80ce460 con 0x7f43c806dce0 2024-04-25T11:28:57.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.903+0000 7f43cd600640 1 --2- 172.21.15.19:0/3071249523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c806dce0 0x7f43c815faa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f43c00097a0 tx=0x7f43c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc18362b62ec47d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.903+0000 7f43c7400640 1 -- 172.21.15.19:0/3071249523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0037070 con 0x7f43c806dce0 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.903+0000 7f43c7400640 1 -- 172.21.15.19:0/3071249523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c0032440 con 0x7f43c806dce0 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.903+0000 7f43c7400640 1 -- 172.21.15.19:0/3071249523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00327f0 con 0x7f43c806dce0 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 -- 172.21.15.19:0/3071249523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c806dce0 msgr2=0x7f43c815faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 --2- 172.21.15.19:0/3071249523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c806dce0 0x7f43c815faa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f43c00097a0 tx=0x7f43c002a180 comp rx=0 tx=0).stop 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 -- 172.21.15.19:0/3071249523 shutdown_connections 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 --2- 172.21.15.19:0/3071249523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c806dce0 0x7f43c815faa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 --2- 172.21.15.19:0/3071249523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c80cd9a0 0x7f43c815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 -- 172.21.15.19:0/3071249523 >> 172.21.15.19:0/3071249523 conn(0x7f43c8073090 msgr2=0x7f43c80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 -- 172.21.15.19:0/3071249523 shutdown_connections 2024-04-25T11:28:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.904+0000 7f43cf600640 1 -- 172.21.15.19:0/3071249523 wait complete. 2024-04-25T11:28:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.907+0000 7f43cf600640 1 Processor -- start 2024-04-25T11:28:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.907+0000 7f43cf600640 1 -- start start 2024-04-25T11:28:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.908+0000 7f43cf600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80b0af0 0x7f43c813f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.908+0000 7f43cf600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c80cd9a0 0x7f43c81f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.908+0000 7f43cf600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c813c1e0 0x7f43c813d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.908+0000 7f43cf600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80cef40 con 0x7f43c80cd9a0 2024-04-25T11:28:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.908+0000 7f43cf600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43c80b0f00 con 0x7f43c80b0af0 2024-04-25T11:28:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.908+0000 7f43cf600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43c80cf0c0 con 0x7f43c813c1e0 2024-04-25T11:28:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c80cd9a0 0x7f43c81f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c80cd9a0 0x7f43c81f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37362/0 (socket says 172.21.15.19:37362) 2024-04-25T11:28:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 -- 172.21.15.19:0/3890604484 learned_addr learned my addr 172.21.15.19:0/3890604484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:57.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43ccc00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80b0af0 0x7f43c813f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43cd600640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c813c1e0 0x7f43c813d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 -- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c813c1e0 msgr2=0x7f43c813d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c813c1e0 0x7f43c813d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 -- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80b0af0 msgr2=0x7f43c813f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80b0af0 0x7f43c813f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43c7e00640 1 -- 172.21.15.19:0/3890604484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c813fdb0 con 0x7f43c80cd9a0 2024-04-25T11:28:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.909+0000 7f43ccc00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80b0af0 0x7f43c813f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.910+0000 7f43c7e00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c80cd9a0 0x7f43c81f4c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c3c0 tx=0x7f43b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.910+0000 7f43c4c00640 1 -- 172.21.15.19:0/3890604484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0017070 con 0x7f43c80cd9a0 2024-04-25T11:28:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.910+0000 7f43c4c00640 1 -- 172.21.15.19:0/3890604484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b0005ce0 con 0x7f43c80cd9a0 2024-04-25T11:28:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.910+0000 7f43c4c00640 1 -- 172.21.15.19:0/3890604484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0007310 con 0x7f43c80cd9a0 2024-04-25T11:28:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.910+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c80b1380 con 0x7f43c80cd9a0 2024-04-25T11:28:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.910+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c80b1940 con 0x7f43c80cd9a0 2024-04-25T11:28:57.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.911+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c8050740 con 0x7f43c80cd9a0 2024-04-25T11:28:57.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.912+0000 7f43c4c00640 1 -- 172.21.15.19:0/3890604484 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f43b001a070 con 0x7f43c80cd9a0 2024-04-25T11:28:57.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.913+0000 7f43c4c00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43a40744b0 0x7f43a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:57.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.913+0000 7f43c4c00640 1 -- 172.21.15.19:0/3890604484 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f43b0094630 con 0x7f43c80cd9a0 2024-04-25T11:28:57.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.914+0000 7f43ccc00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43a40744b0 0x7f43a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:57.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.915+0000 7f43ccc00640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43a40744b0 0x7f43a40769c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43b8007eb0 tx=0x7f43b80073c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:57.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:57.919+0000 7f43c4c00640 1 -- 172.21.15.19:0/3890604484 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43b005ed20 con 0x7f43c80cd9a0 2024-04-25T11:28:58.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.123+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43c806c900 con 0x7f43c80cd9a0 2024-04-25T11:28:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.124+0000 7f43c4c00640 1 -- 172.21.15.19:0/3890604484 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f43b005e620 con 0x7f43c80cd9a0 2024-04-25T11:28:58.115 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:28:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43a40744b0 msgr2=0x7f43a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43a40744b0 0x7f43a40769c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43b8007eb0 tx=0x7f43b80073c0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c80cd9a0 msgr2=0x7f43c81f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c80cd9a0 0x7f43c81f4c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c3c0 tx=0x7f43b000c880 comp rx=0 tx=0).stop 2024-04-25T11:28:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 shutdown_connections 2024-04-25T11:28:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43a40744b0 0x7f43a40769c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43c813c1e0 0x7f43c813d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43c80cd9a0 0x7f43c81f4c00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 --2- 172.21.15.19:0/3890604484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43c80b0af0 0x7f43c813f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.127+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 >> 172.21.15.19:0/3890604484 conn(0x7f43c8073090 msgr2=0x7f43c8160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.128+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 shutdown_connections 2024-04-25T11:28:58.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.128+0000 7f43cf600640 1 -- 172.21.15.19:0/3890604484 wait complete. 2024-04-25T11:28:58.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:28:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.440+0000 7fdf1fe00640 1 Processor -- start 2024-04-25T11:28:58.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.440+0000 7fdf1fe00640 1 -- start start 2024-04-25T11:28:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.442+0000 7fdf1fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf2006c5c0 0x7fdf200cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.442+0000 7fdf1fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf200b4ea0 0x7fdf2015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.442+0000 7fdf1fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf200d1230 con 0x7fdf20144370 2024-04-25T11:28:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.443+0000 7fdf1fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdf200d06b0 con 0x7fdf2006c5c0 2024-04-25T11:28:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.443+0000 7fdf1fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdf200d0990 con 0x7fdf200b4ea0 2024-04-25T11:28:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.443+0000 7fdf1ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf2006c5c0 0x7fdf200cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.443+0000 7fdf17e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdf20144370 0x7fdf200b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51618/0 (socket says 172.21.15.19:51618) 2024-04-25T11:28:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.443+0000 7fdf17e00640 1 -- 172.21.15.19:0/2742835493 learned_addr learned my addr 172.21.15.19:0/2742835493 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.443+0000 7fdf1f400640 1 --2- 172.21.15.19:0/2742835493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf200b4ea0 0x7fdf2015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.444+0000 7fdf1ea00640 1 -- 172.21.15.19:0/2742835493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf200b4ea0 msgr2=0x7fdf2015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.444+0000 7fdf1ea00640 1 --2- 172.21.15.19:0/2742835493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf200b4ea0 0x7fdf2015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.444+0000 7fdf1ea00640 1 -- 172.21.15.19:0/2742835493 >> v1:172.21.15.19:6789/0 conn(0x7fdf20144370 legacy=0x7fdf200b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.444+0000 7fdf1ea00640 1 -- 172.21.15.19:0/2742835493 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf200d0c70 con 0x7fdf2006c5c0 2024-04-25T11:28:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.444+0000 7fdf1e000640 1 -- 172.21.15.19:0/2742835493 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314757815 0 0) 0x7fdf200d1230 con 0x7fdf20144370 2024-04-25T11:28:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.445+0000 7fdf1ea00640 1 --2- 172.21.15.19:0/2742835493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf200cfa70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf100097a0 tx=0x7fdf1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784ef413adba8d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.445+0000 7fdf1e000640 1 -- 172.21.15.19:0/2742835493 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10037070 con 0x7fdf2006c5c0 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.445+0000 7fdf1e000640 1 -- 172.21.15.19:0/2742835493 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf10032440 con 0x7fdf2006c5c0 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.445+0000 7fdf1e000640 1 -- 172.21.15.19:0/2742835493 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10032810 con 0x7fdf2006c5c0 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 -- 172.21.15.19:0/2742835493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 msgr2=0x7fdf200cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/2742835493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf200cfa70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf100097a0 tx=0x7fdf1002a180 comp rx=0 tx=0).stop 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 -- 172.21.15.19:0/2742835493 shutdown_connections 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/2742835493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf200b4ea0 0x7fdf2015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/2742835493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf200cfa70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 -- 172.21.15.19:0/2742835493 >> 172.21.15.19:0/2742835493 conn(0x7fdf200d1720 msgr2=0x7fdf200d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 -- 172.21.15.19:0/2742835493 shutdown_connections 2024-04-25T11:28:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.446+0000 7fdf1fe00640 1 -- 172.21.15.19:0/2742835493 wait complete. 2024-04-25T11:28:58.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.449+0000 7fdf1fe00640 1 Processor -- start 2024-04-25T11:28:58.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.449+0000 7fdf1fe00640 1 -- start start 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.450+0000 7fdf1fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf2015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.450+0000 7fdf1fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf20144370 0x7fdf20146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf20137080 0x7fdf20142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf200d0450 con 0x7fdf2006c5c0 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf200d02d0 con 0x7fdf20144370 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdf20147760 con 0x7fdf20137080 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf2015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf2015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37376/0 (socket says 172.21.15.19:37376) 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 -- 172.21.15.19:0/176432905 learned_addr learned my addr 172.21.15.19:0/176432905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf17e00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf20144370 0x7fdf20146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 -- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf20137080 msgr2=0x7fdf20142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1f400640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf20137080 0x7fdf20142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf20137080 0x7fdf20142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 -- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf20144370 msgr2=0x7fdf20146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf20144370 0x7fdf20146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1ea00640 1 -- 172.21.15.19:0/176432905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201600d0 con 0x7fdf2006c5c0 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf17e00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf20144370 0x7fdf20146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.451+0000 7fdf1f400640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf20137080 0x7fdf20142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.452+0000 7fdf1ea00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf2015f910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdf100097a0 tx=0x7fdf100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.452+0000 7fdf17400640 1 -- 172.21.15.19:0/176432905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10037050 con 0x7fdf2006c5c0 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.452+0000 7fdf17400640 1 -- 172.21.15.19:0/176432905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf10038040 con 0x7fdf2006c5c0 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.452+0000 7fdf17400640 1 -- 172.21.15.19:0/176432905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10004470 con 0x7fdf2006c5c0 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.452+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201603b0 con 0x7fdf2006c5c0 2024-04-25T11:28:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.452+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf20160970 con 0x7fdf2006c5c0 2024-04-25T11:28:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.453+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf20050740 con 0x7fdf2006c5c0 2024-04-25T11:28:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.455+0000 7fdf17400640 1 -- 172.21.15.19:0/176432905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf10030900 con 0x7fdf2006c5c0 2024-04-25T11:28:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.456+0000 7fdf17400640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdefc0744b0 0x7fdefc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:58.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.456+0000 7fdf17400640 1 -- 172.21.15.19:0/176432905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdf100b5c30 con 0x7fdf2006c5c0 2024-04-25T11:28:58.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.456+0000 7fdf17e00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdefc0744b0 0x7fdefc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:58.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.458+0000 7fdf17e00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdefc0744b0 0x7fdefc0769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdf20069a70 tx=0x7fdf08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.461+0000 7fdf17400640 1 -- 172.21.15.19:0/176432905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf100803b0 con 0x7fdf2006c5c0 2024-04-25T11:28:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.682+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf2015e660 con 0x7fdf2006c5c0 2024-04-25T11:28:58.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.683+0000 7fdf17400640 1 -- 172.21.15.19:0/176432905 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7fdf100883a0 con 0x7fdf2006c5c0 2024-04-25T11:28:58.673 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:28:58.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdefc0744b0 msgr2=0x7fdefc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdefc0744b0 0x7fdefc0769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdf20069a70 tx=0x7fdf08009040 comp rx=0 tx=0).stop 2024-04-25T11:28:58.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 msgr2=0x7fdf2015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:58.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf2015f910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdf100097a0 tx=0x7fdf100046c0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 shutdown_connections 2024-04-25T11:28:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdefc0744b0 0x7fdefc0769c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf20144370 0x7fdf20146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf20137080 0x7fdf20142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 --2- 172.21.15.19:0/176432905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf2006c5c0 0x7fdf2015f910 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.685+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 >> 172.21.15.19:0/176432905 conn(0x7fdf200d1720 msgr2=0x7fdf2015c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.686+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 shutdown_connections 2024-04-25T11:28:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:58.686+0000 7fdf1fe00640 1 -- 172.21.15.19:0/176432905 wait complete. 2024-04-25T11:28:58.703 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:28:58.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:58.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:28:58.846 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:28:58.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:28:58.846 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:28:58.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:28:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.157+0000 7f82e8a00640 1 Processor -- start 2024-04-25T11:28:59.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.157+0000 7f82e8a00640 1 -- start start 2024-04-25T11:28:59.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e4070e20 0x7f82e415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f82e406d560 0x7f82e40b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e8a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e4071230 con 0x7f82e406f180 2024-04-25T11:28:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e8a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f82e406f8f0 con 0x7f82e406d560 2024-04-25T11:28:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82e406fbd0 con 0x7f82e4070e20 2024-04-25T11:28:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e1e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f82e406f180 0x7f82e406ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51626/0 (socket says 172.21.15.19:51626) 2024-04-25T11:28:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e1e00640 1 -- 172.21.15.19:0/608990169 learned_addr learned my addr 172.21.15.19:0/608990169 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:28:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.159+0000 7f82e3200640 1 --2- 172.21.15.19:0/608990169 >> v2:172.21.15.19:3300/0 conn(0x7f82e406d560 0x7f82e40b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.160+0000 7f82e2800640 1 --2- 172.21.15.19:0/608990169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e4070e20 0x7f82e415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.160+0000 7f82e3200640 1 -- 172.21.15.19:0/608990169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e4070e20 msgr2=0x7f82e415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:28:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.160+0000 7f82e3200640 1 --2- 172.21.15.19:0/608990169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e4070e20 0x7f82e415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.160+0000 7f82e3200640 1 -- 172.21.15.19:0/608990169 >> v1:172.21.15.19:6789/0 conn(0x7f82e406f180 legacy=0x7f82e406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.160+0000 7f82e3200640 1 -- 172.21.15.19:0/608990169 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e406feb0 con 0x7f82e406d560 2024-04-25T11:28:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.160+0000 7f82e1400640 1 -- 172.21.15.19:0/608990169 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802634378 0 0) 0x7f82e4071230 con 0x7f82e406f180 2024-04-25T11:28:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.160+0000 7f82e2800640 1 --2- 172.21.15.19:0/608990169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e4070e20 0x7f82e415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.161+0000 7f82e3200640 1 --2- 172.21.15.19:0/608990169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e406d560 0x7f82e40b5e20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0097a0 tx=0x7f82dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4b15cc23de806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.161+0000 7f82e1400640 1 -- 172.21.15.19:0/608990169 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc037070 con 0x7f82e406d560 2024-04-25T11:28:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.161+0000 7f82e1400640 1 -- 172.21.15.19:0/608990169 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82dc032440 con 0x7f82e406d560 2024-04-25T11:28:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.161+0000 7f82e1400640 1 -- 172.21.15.19:0/608990169 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc032810 con 0x7f82e406d560 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 -- 172.21.15.19:0/608990169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e406d560 msgr2=0x7f82e40b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 --2- 172.21.15.19:0/608990169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e406d560 0x7f82e40b5e20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0097a0 tx=0x7f82dc02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 -- 172.21.15.19:0/608990169 shutdown_connections 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 --2- 172.21.15.19:0/608990169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e406d560 0x7f82e40b5e20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 --2- 172.21.15.19:0/608990169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e4070e20 0x7f82e415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 -- 172.21.15.19:0/608990169 >> 172.21.15.19:0/608990169 conn(0x7f82e40d1720 msgr2=0x7f82e40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 -- 172.21.15.19:0/608990169 shutdown_connections 2024-04-25T11:28:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.162+0000 7f82e8a00640 1 -- 172.21.15.19:0/608990169 wait complete. 2024-04-25T11:28:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.165+0000 7f82e8a00640 1 Processor -- start 2024-04-25T11:28:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.165+0000 7f82e8a00640 1 -- start start 2024-04-25T11:28:59.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.166+0000 7f82e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e406f180 0x7f82e4140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.166+0000 7f82e8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e4070e20 0x7f82e415f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e4141eb0 0x7f82e41428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e8a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82e406c5c0 con 0x7f82e4070e20 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82e406f590 con 0x7f82e4141eb0 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82e406c740 con 0x7f82e406f180 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e4070e20 0x7f82e415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e4070e20 0x7f82e415f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37396/0 (socket says 172.21.15.19:37396) 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 -- 172.21.15.19:0/2802035622 learned_addr learned my addr 172.21.15.19:0/2802035622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e2800640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e406f180 0x7f82e4140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 -- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e406f180 msgr2=0x7f82e4140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e3200640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e4141eb0 0x7f82e41428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e406f180 0x7f82e4140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 -- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e4141eb0 msgr2=0x7f82e41428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e4141eb0 0x7f82e41428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e1e00640 1 -- 172.21.15.19:0/2802035622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e4140e10 con 0x7f82e4070e20 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.167+0000 7f82e2800640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e406f180 0x7f82e4140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.168+0000 7f82e1e00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e4070e20 0x7f82e415f990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f82e40720f0 tx=0x7f82cc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.168+0000 7f82daa00640 1 -- 172.21.15.19:0/2802035622 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0074e0 con 0x7f82e4070e20 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.168+0000 7f82daa00640 1 -- 172.21.15.19:0/2802035622 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc0047e0 con 0x7f82e4070e20 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.168+0000 7f82daa00640 1 -- 172.21.15.19:0/2802035622 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc004bf0 con 0x7f82e4070e20 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.168+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e41410f0 con 0x7f82e4070e20 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.168+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e41416b0 con 0x7f82e4070e20 2024-04-25T11:28:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.169+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e4050740 con 0x7f82e4070e20 2024-04-25T11:28:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.171+0000 7f82daa00640 1 -- 172.21.15.19:0/2802035622 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82cc011070 con 0x7f82e4070e20 2024-04-25T11:28:59.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.172+0000 7f82daa00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82c00744b0 0x7f82c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:28:59.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.172+0000 7f82daa00640 1 -- 172.21.15.19:0/2802035622 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f82cc094e60 con 0x7f82e4070e20 2024-04-25T11:28:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.172+0000 7f82e2800640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82c00744b0 0x7f82c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:28:59.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.174+0000 7f82e2800640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82c00744b0 0x7f82c00769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007f40 tx=0x7f82d4007490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:28:59.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.177+0000 7f82daa00640 1 -- 172.21.15.19:0/2802035622 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82cc05f550 con 0x7f82e4070e20 2024-04-25T11:28:59.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.419+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82e4140ba0 con 0x7f82e4070e20 2024-04-25T11:28:59.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.420+0000 7f82daa00640 1 -- 172.21.15.19:0/2802035622 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f82cc05ee50 con 0x7f82e4070e20 2024-04-25T11:28:59.415 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:28:59.415 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:28:48.905254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5064":{"gid":5064,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3831883468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3831883468},{"type":"v1","addr":"172.21.15.19:6835","nonce":3831883468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5064,"qdb_cluster":[5064]},"id":2}]} 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82c00744b0 msgr2=0x7f82c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82c00744b0 0x7f82c00769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007f40 tx=0x7f82d4007490 comp rx=0 tx=0).stop 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e4070e20 msgr2=0x7f82e415f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e4070e20 0x7f82e415f990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f82e40720f0 tx=0x7f82cc00c870 comp rx=0 tx=0).stop 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 shutdown_connections 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82c00744b0 0x7f82c00769c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82e4141eb0 0x7f82e41428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82e4070e20 0x7f82e415f990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 --2- 172.21.15.19:0/2802035622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82e406f180 0x7f82e4140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 >> 172.21.15.19:0/2802035622 conn(0x7f82e40d1720 msgr2=0x7f82e40b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 shutdown_connections 2024-04-25T11:28:59.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:28:59.423+0000 7f82e8a00640 1 -- 172.21.15.19:0/2802035622 wait complete. 2024-04-25T11:28:59.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T11:28:59.757 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-04-25T11:28:59.757 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T11:28:59.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:29:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.048+0000 7f377b200640 1 Processor -- start 2024-04-25T11:29:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.049+0000 7f377b200640 1 -- start start 2024-04-25T11:29:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.051+0000 7f377b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f377c144370 0x7f377c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.051+0000 7f377b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377c0d1230 con 0x7f377c0b4ea0 2024-04-25T11:29:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.051+0000 7f377b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377c0d06b0 con 0x7f377c06c5c0 2024-04-25T11:29:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.051+0000 7f377b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f377c0d0990 con 0x7f377c144370 2024-04-25T11:29:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.052+0000 7f377a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f377c144370 0x7f377c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.052+0000 7f377a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f377c144370 0x7f377c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37398/0 (socket says 172.21.15.19:37398) 2024-04-25T11:29:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.052+0000 7f377a800640 1 -- 172.21.15.19:0/1708061768 learned_addr learned my addr 172.21.15.19:0/1708061768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.052+0000 7f377a800640 1 -- 172.21.15.19:0/1708061768 >> v1:172.21.15.169:6789/0 conn(0x7f377c06c5c0 legacy=0x7f377c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.052+0000 7f377a800640 1 -- 172.21.15.19:0/1708061768 >> v1:172.21.15.19:6789/0 conn(0x7f377c0b4ea0 legacy=0x7f377c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.052+0000 7f3779400640 1 -- 172.21.15.19:0/1708061768 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942689206 0 0) 0x7f377c0d1230 con 0x7f377c0b4ea0 2024-04-25T11:29:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.052+0000 7f377a800640 1 -- 172.21.15.19:0/1708061768 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c0d0c70 con 0x7f377c144370 2024-04-25T11:29:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.053+0000 7f377a800640 1 --2- 172.21.15.19:0/1708061768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c144370 0x7f377c0cfa70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3774009e50 tx=0x7f377402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea9d98ce76fd0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.053+0000 7f3779400640 1 -- 172.21.15.19:0/1708061768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774036070 con 0x7f377c144370 2024-04-25T11:29:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f3779400640 1 -- 172.21.15.19:0/1708061768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3774031600 con 0x7f377c144370 2024-04-25T11:29:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f3779400640 1 -- 172.21.15.19:0/1708061768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37740319d0 con 0x7f377c144370 2024-04-25T11:29:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f377b200640 1 -- 172.21.15.19:0/1708061768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c144370 msgr2=0x7f377c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f377b200640 1 --2- 172.21.15.19:0/1708061768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c144370 0x7f377c0cfa70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3774009e50 tx=0x7f377402a760 comp rx=0 tx=0).stop 2024-04-25T11:29:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f377b200640 1 -- 172.21.15.19:0/1708061768 shutdown_connections 2024-04-25T11:29:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f377b200640 1 --2- 172.21.15.19:0/1708061768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c144370 0x7f377c0cfa70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f377b200640 1 -- 172.21.15.19:0/1708061768 >> 172.21.15.19:0/1708061768 conn(0x7f377c0d1720 msgr2=0x7f377c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f377b200640 1 -- 172.21.15.19:0/1708061768 shutdown_connections 2024-04-25T11:29:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.054+0000 7f377b200640 1 -- 172.21.15.19:0/1708061768 wait complete. 2024-04-25T11:29:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.057+0000 7f377b200640 1 Processor -- start 2024-04-25T11:29:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.057+0000 7f377b200640 1 -- start start 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.058+0000 7f377b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f377c06c5c0 0x7f377c13f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.058+0000 7f377b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f377c144370 0x7f377c1610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.058+0000 7f377b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c1f3c30 0x7f377c1f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0d0450 con 0x7f377c1f3c30 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0d02d0 con 0x7f377c144370 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f377c147760 con 0x7f377c06c5c0 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c1f3c30 0x7f377c1f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c1f3c30 0x7f377c1f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37406/0 (socket says 172.21.15.19:37406) 2024-04-25T11:29:00.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 -- 172.21.15.19:0/583776528 learned_addr learned my addr 172.21.15.19:0/583776528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:00.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 -- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f377c06c5c0 msgr2=0x7f377c13f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f377c06c5c0 0x7f377c13f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f3773e00640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f377c144370 0x7f377c1610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 -- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f377c144370 msgr2=0x7f377c1610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f377c144370 0x7f377c1610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.059+0000 7f377a800640 1 -- 172.21.15.19:0/583776528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c1615f0 con 0x7f377c1f3c30 2024-04-25T11:29:00.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.060+0000 7f377a800640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c1f3c30 0x7f377c1f4040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3774034040 tx=0x7f3774004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.060+0000 7f3772000640 1 -- 172.21.15.19:0/583776528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774041020 con 0x7f377c1f3c30 2024-04-25T11:29:00.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.060+0000 7f3772000640 1 -- 172.21.15.19:0/583776528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3774007780 con 0x7f377c1f3c30 2024-04-25T11:29:00.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.060+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c160010 con 0x7f377c1f3c30 2024-04-25T11:29:00.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.060+0000 7f3772000640 1 -- 172.21.15.19:0/583776528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774036040 con 0x7f377c1f3c30 2024-04-25T11:29:00.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.060+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1605d0 con 0x7f377c1f3c30 2024-04-25T11:29:00.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.061+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c15fd80 con 0x7f377c1f3c30 2024-04-25T11:29:00.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.063+0000 7f3772000640 1 -- 172.21.15.19:0/583776528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3774044020 con 0x7f377c1f3c30 2024-04-25T11:29:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.064+0000 7f3772000640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37580744b0 0x7f37580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.064+0000 7f3772000640 1 -- 172.21.15.19:0/583776528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f37740b5f70 con 0x7f377c1f3c30 2024-04-25T11:29:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.064+0000 7f3779e00640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37580744b0 0x7f37580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.066+0000 7f3779e00640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37580744b0 0x7f37580769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f376c006fd0 tx=0x7f376c006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.069+0000 7f3772000640 1 -- 172.21.15.19:0/583776528 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f377404b030 con 0x7f377c1f3c30 2024-04-25T11:29:00.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.302+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f377c050740 con 0x7f377c1f3c30 2024-04-25T11:29:00.295 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:29:00.304+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:29:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.308+0000 7f3772000640 1 -- 172.21.15.19:0/583776528 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f3774080160 con 0x7f377c1f3c30 2024-04-25T11:29:00.299 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37580744b0 msgr2=0x7f37580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37580744b0 0x7f37580769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f376c006fd0 tx=0x7f376c006bc0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c1f3c30 msgr2=0x7f377c1f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c1f3c30 0x7f377c1f4040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3774034040 tx=0x7f3774004a40 comp rx=0 tx=0).stop 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 shutdown_connections 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37580744b0 0x7f37580769c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f377c144370 0x7f377c1610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c1f3c30 0x7f377c1f4040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 --2- 172.21.15.19:0/583776528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f377c06c5c0 0x7f377c13f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 >> 172.21.15.19:0/583776528 conn(0x7f377c0d1720 msgr2=0x7f377c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 shutdown_connections 2024-04-25T11:29:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.313+0000 7f377b200640 1 -- 172.21.15.19:0/583776528 wait complete. 2024-04-25T11:29:00.315 INFO:tasks.ceph.mds.a.smithi019.stderr: -4697> 2024-04-25T11:28:52.142+0000 7f0536000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:29:00.332 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:29:00.454 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:29:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.521+0000 7fd67855ff40 1 Processor -- start 2024-04-25T11:29:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.522+0000 7fd67855ff40 1 -- start start 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.523+0000 7fd67855ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6580022d0 0x559b5a9cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.523+0000 7fd67855ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559b5a9d4d10 0x559b5a9dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.523+0000 7fd67855ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 0x559b5a92f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.523+0000 7fd67855ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559b5a8aad20 con 0x7fd6580022d0 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.523+0000 7fd67855ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559b5a8aed60 con 0x559b5a9ddc50 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.523+0000 7fd67855ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559b5a94cd10 con 0x559b5a9d4d10 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.523+0000 7fd671600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6580022d0 0x559b5a9cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6580022d0 0x559b5a9cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37434/0 (socket says 172.21.15.19:37434) 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 -- 172.21.15.19:0/3923870932 learned_addr learned my addr 172.21.15.19:0/3923870932 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd672000640 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 0x559b5a92f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd670c00640 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559b5a9d4d10 0x559b5a9dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 -- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559b5a9d4d10 msgr2=0x559b5a9dd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559b5a9d4d10 0x559b5a9dd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 -- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 msgr2=0x559b5a92f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 0x559b5a92f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 -- 172.21.15.19:0/3923870932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5a9dd2f0 con 0x7fd6580022d0 2024-04-25T11:29:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.524+0000 7fd671600640 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6580022d0 0x559b5a9cfb90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd664046b20 tx=0x7fd664050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b15c7de79a42dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.525+0000 7fd66b400640 1 -- 172.21.15.19:0/3923870932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664001630 con 0x7fd6580022d0 2024-04-25T11:29:00.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.525+0000 7fd66b400640 1 -- 172.21.15.19:0/3923870932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd664066070 con 0x7fd6580022d0 2024-04-25T11:29:00.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.525+0000 7fd66b400640 1 -- 172.21.15.19:0/3923870932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664061650 con 0x7fd6580022d0 2024-04-25T11:29:00.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.525+0000 7fd67855ff40 1 -- 172.21.15.19:0/3923870932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6580022d0 msgr2=0x559b5a9cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.525+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6580022d0 0x559b5a9cfb90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd664046b20 tx=0x7fd664050000 comp rx=0 tx=0).stop 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.526+0000 7fd67855ff40 1 -- 172.21.15.19:0/3923870932 shutdown_connections 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.526+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 0x559b5a92f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.526+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559b5a9d4d10 0x559b5a9dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.526+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3923870932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6580022d0 0x559b5a9cfb90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.526+0000 7fd67855ff40 1 -- 172.21.15.19:0/3923870932 >> 172.21.15.19:0/3923870932 conn(0x559b5a826ac0 msgr2=0x559b5a9da760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.526+0000 7fd67855ff40 1 -- 172.21.15.19:0/3923870932 shutdown_connections 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.526+0000 7fd67855ff40 1 -- 172.21.15.19:0/3923870932 wait complete. 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.527+0000 7fd67855ff40 1 Processor -- start 2024-04-25T11:29:00.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.527+0000 7fd67855ff40 1 -- start start 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.528+0000 7fd67855ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b5a9d4d10 0x559b5aa6bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.528+0000 7fd67855ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 0x559b5a92f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.528+0000 7fd67855ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6580022d0 0x559b5a92fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.528+0000 7fd67855ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559b5a9d7410 con 0x559b5a9d4d10 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.528+0000 7fd67855ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559b5a94a120 con 0x559b5a9ddc50 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.528+0000 7fd67855ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559b5aa748b0 con 0x7fd6580022d0 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b5a9d4d10 0x559b5aa6bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b5a9d4d10 0x559b5aa6bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37448/0 (socket says 172.21.15.19:37448) 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 -- 172.21.15.19:0/3876749142 learned_addr learned my addr 172.21.15.19:0/3876749142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd672000640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6580022d0 0x559b5a92fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd670c00640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 0x559b5a92f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 -- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6580022d0 msgr2=0x559b5a92fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6580022d0 0x559b5a92fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 -- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 msgr2=0x559b5a92f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559b5a9ddc50 0x559b5a92f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.529+0000 7fd671600640 1 -- 172.21.15.19:0/3876749142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b5aa72db0 con 0x559b5a9d4d10 2024-04-25T11:29:00.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.530+0000 7fd671600640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b5a9d4d10 0x559b5aa6bd20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd664001ed0 tx=0x7fd664001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.530+0000 7fd66a000640 1 -- 172.21.15.19:0/3876749142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664075070 con 0x559b5a9d4d10 2024-04-25T11:29:00.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.530+0000 7fd66a000640 1 -- 172.21.15.19:0/3876749142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd664061650 con 0x559b5a9d4d10 2024-04-25T11:29:00.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.530+0000 7fd66a000640 1 -- 172.21.15.19:0/3876749142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664061a00 con 0x559b5a9d4d10 2024-04-25T11:29:00.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.530+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559b5aa72790 con 0x559b5a9d4d10 2024-04-25T11:29:00.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.530+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b5aa72a70 con 0x559b5a9d4d10 2024-04-25T11:29:00.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.532+0000 7fd66a000640 1 -- 172.21.15.19:0/3876749142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd664078070 con 0x559b5a9d4d10 2024-04-25T11:29:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.533+0000 7fd66a000640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd650072e90 0x7fd6500753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.533+0000 7fd66a000640 1 -- 172.21.15.19:0/3876749142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fd6640d9840 con 0x559b5a9d4d10 2024-04-25T11:29:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.534+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x559b5aa73570 0x559b5aa73980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.534+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36) -- 0x559b5aa73ec0 con 0x559b5aa73570 2024-04-25T11:29:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.534+0000 7fd670c00640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd650072e90 0x7fd6500753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.534+0000 7fd672000640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x559b5aa73570 0x559b5aa73980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.535+0000 7fd670c00640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd650072e90 0x7fd6500753a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x559b5a9d42b0 tx=0x7fd65c005840 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.535+0000 7fd672000640 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x559b5aa73570 0x559b5aa73980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd672000640 1 -- 172.21.15.19:0/3876749142 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v36'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd66c007d90 con 0x559b5aa73570 2024-04-25T11:29:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x559b5aa73570 msgr2=0x559b5aa73980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x559b5aa73570 0x559b5aa73980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd650072e90 msgr2=0x7fd6500753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd650072e90 0x7fd6500753a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x559b5a9d42b0 tx=0x7fd65c005840 comp rx=0 tx=0).stop 2024-04-25T11:29:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b5a9d4d10 msgr2=0x559b5aa6bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 --2- 172.21.15.19:0/3876749142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559b5a9d4d10 0x559b5aa6bd20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd664001ed0 tx=0x7fd664001a40 comp rx=0 tx=0).stop 2024-04-25T11:29:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd672000640 1 -- 172.21.15.19:0/3876749142 reap_dead start 2024-04-25T11:29:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 shutdown_connections 2024-04-25T11:29:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.544+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 >> 172.21.15.19:0/3876749142 conn(0x559b5a826ac0 msgr2=0x559b5a9da760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.545+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 shutdown_connections 2024-04-25T11:29:00.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.545+0000 7fd67855ff40 1 -- 172.21.15.19:0/3876749142 wait complete. 2024-04-25T11:29:00.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:29:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.825+0000 7f1a46400640 1 Processor -- start 2024-04-25T11:29:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.825+0000 7f1a46400640 1 -- start start 2024-04-25T11:29:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.827+0000 7f1a46400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 0x7f1a40168e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.827+0000 7f1a46400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a40070000 con 0x7f1a4006dce0 2024-04-25T11:29:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.827+0000 7f1a46400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4006f240 con 0x7f1a40144340 2024-04-25T11:29:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.827+0000 7f1a46400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4015f270 con 0x7f1a4015ee60 2024-04-25T11:29:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.828+0000 7f1a3fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1a4006dce0 0x7f1a4015e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51674/0 (socket says 172.21.15.19:51674) 2024-04-25T11:29:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.828+0000 7f1a3fe00640 1 -- 172.21.15.19:0/3643685035 learned_addr learned my addr 172.21.15.19:0/3643685035 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.828+0000 7f1a45a00640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 0x7f1a40168e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.828+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750377443 0 0) 0x7f1a40070000 con 0x7f1a4006dce0 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.828+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a24003520 con 0x7f1a4006dce0 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3049422012 0 0) 0x7f1a24003520 con 0x7f1a4006dce0 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 msgr2=0x7f1a40168e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 0x7f1a40168e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 >> v1:172.21.15.169:6789/0 conn(0x7f1a40144340 legacy=0x7f1a4006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4006f520 con 0x7f1a4006dce0 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1a28001670 con 0x7f1a4006dce0 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 >> v1:172.21.15.19:6789/0 conn(0x7f1a4006dce0 legacy=0x7f1a4015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a240039e0 0x7f1a24003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 0x7f1a24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d40 0x7f1a24005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2400bac0 con 0x7f1a240039e0 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2400bc70 con 0x7f1a24004d40 2024-04-25T11:29:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a2400be70 con 0x7f1a4015ee60 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.829+0000 7f1a45000640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a240039e0 0x7f1a24003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.830+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 0x7f1a24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.830+0000 7f1a45a00640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d40 0x7f1a24005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.830+0000 7f1a45000640 1 -- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 msgr2=0x7f1a24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.830+0000 7f1a45000640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a4015ee60 0x7f1a24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.830+0000 7f1a45000640 1 -- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d40 msgr2=0x7f1a24005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.830+0000 7f1a45000640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a24004d40 0x7f1a24005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.830+0000 7f1a45000640 1 -- 172.21.15.19:0/3643685035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a24003520 con 0x7f1a240039e0 2024-04-25T11:29:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.831+0000 7f1a45000640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a240039e0 0x7f1a24003e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a30001750 tx=0x7f1a24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272a171e5bffb424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.831+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30001370 con 0x7f1a240039e0 2024-04-25T11:29:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.831+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a300052d0 con 0x7f1a240039e0 2024-04-25T11:29:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.831+0000 7f1a3f400640 1 -- 172.21.15.19:0/3643685035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3002ab20 con 0x7f1a240039e0 2024-04-25T11:29:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.832+0000 7f1a46400640 1 -- 172.21.15.19:0/3643685035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a240039e0 msgr2=0x7f1a24003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.832+0000 7f1a46400640 1 --2- 172.21.15.19:0/3643685035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a240039e0 0x7f1a24003e50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a30001750 tx=0x7f1a24005830 comp rx=0 tx=0).stop 2024-04-25T11:29:00.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.832+0000 7f1a45a00640 1 -- 172.21.15.19:0/3643685035 reap_dead start 2024-04-25T11:29:00.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.832+0000 7f1a46400640 1 -- 172.21.15.19:0/3643685035 shutdown_connections 2024-04-25T11:29:00.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.832+0000 7f1a46400640 1 -- 172.21.15.19:0/3643685035 >> 172.21.15.19:0/3643685035 conn(0x7f1a40072f40 msgr2=0x7f1a40073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:00.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.832+0000 7f1a46400640 1 -- 172.21.15.19:0/3643685035 shutdown_connections 2024-04-25T11:29:00.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.832+0000 7f1a46400640 1 -- 172.21.15.19:0/3643685035 wait complete. 2024-04-25T11:29:00.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.835+0000 7f1a46400640 1 Processor -- start 2024-04-25T11:29:00.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.835+0000 7f1a46400640 1 -- start start 2024-04-25T11:29:00.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a46400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039e0 0x7f1a401583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a46400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4015d5f0 0x7f1a40159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a46400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40159850 0x7f1a40159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a46400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4006fae0 con 0x7f1a40159850 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a46400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a40144750 con 0x7f1a4015d5f0 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a46400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a401604b0 con 0x7f1a240039e0 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a45a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40159850 0x7f1a40159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a45a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40159850 0x7f1a40159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37464/0 (socket says 172.21.15.19:37464) 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a45a00640 1 -- 172.21.15.19:0/3300118391 learned_addr learned my addr 172.21.15.19:0/3300118391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4015d5f0 0x7f1a40159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.837+0000 7f1a45000640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039e0 0x7f1a401583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a45a00640 1 -- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039e0 msgr2=0x7f1a401583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a45a00640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039e0 0x7f1a401583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a45a00640 1 -- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4015d5f0 msgr2=0x7f1a40159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a45a00640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4015d5f0 0x7f1a40159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a45a00640 1 -- 172.21.15.19:0/3300118391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4015a1a0 con 0x7f1a40159850 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a45a00640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40159850 0x7f1a40159c60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a400d1790 tx=0x7f1a380115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a3cc00640 1 -- 172.21.15.19:0/3300118391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38011e40 con 0x7f1a40159850 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a3cc00640 1 -- 172.21.15.19:0/3300118391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3801b070 con 0x7f1a40159850 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a3cc00640 1 -- 172.21.15.19:0/3300118391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38003990 con 0x7f1a40159850 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a40158930 con 0x7f1a40159850 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.838+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a400af8f0 con 0x7f1a40159850 2024-04-25T11:29:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.840+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a40158ff0 con 0x7f1a40159850 2024-04-25T11:29:00.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.841+0000 7f1a3cc00640 1 -- 172.21.15.19:0/3300118391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a38005d10 con 0x7f1a40159850 2024-04-25T11:29:00.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.842+0000 7f1a3cc00640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744e0 0x7f1a1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:00.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.842+0000 7f1a3cc00640 1 -- 172.21.15.19:0/3300118391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f1a380956f0 con 0x7f1a40159850 2024-04-25T11:29:00.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.843+0000 7f1a45000640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744e0 0x7f1a1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:00.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.844+0000 7f1a45000640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744e0 0x7f1a1c0769f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a30032000 tx=0x7f1a3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:00.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:00.847+0000 7f1a3cc00640 1 -- 172.21.15.19:0/3300118391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a40158ff0 con 0x7f1a40159850 2024-04-25T11:29:01.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.096+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1a400b7920 con 0x7f1a40159850 2024-04-25T11:29:01.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.322+0000 7f1a3cc00640 1 -- 172.21.15.19:0/3300118391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v19) ==== 162+0+0 (secure 0 0 0) 0x7f1a40158930 con 0x7f1a40159850 2024-04-25T11:29:01.313 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:29:01.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744e0 msgr2=0x7f1a1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744e0 0x7f1a1c0769f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a30032000 tx=0x7f1a3002e000 comp rx=0 tx=0).stop 2024-04-25T11:29:01.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40159850 msgr2=0x7f1a40159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40159850 0x7f1a40159c60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1a400d1790 tx=0x7f1a380115a0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 shutdown_connections 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744e0 0x7f1a1c0769f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4015d5f0 0x7f1a40159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40159850 0x7f1a40159c60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 --2- 172.21.15.19:0/3300118391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a240039e0 0x7f1a401583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.326+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 >> 172.21.15.19:0/3300118391 conn(0x7f1a40072f40 msgr2=0x7f1a4006d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.327+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 shutdown_connections 2024-04-25T11:29:01.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.327+0000 7f1a46400640 1 -- 172.21.15.19:0/3300118391 wait complete. 2024-04-25T11:29:01.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:01.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.632+0000 7fc478a00640 1 Processor -- start 2024-04-25T11:29:01.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.632+0000 7fc478a00640 1 -- start start 2024-04-25T11:29:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.634+0000 7fc478a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4740b7c50 0x7fc4740b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.634+0000 7fc478a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc4740d44a0 0x7fc4740d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.634+0000 7fc478a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc474161650 con 0x7fc474164860 2024-04-25T11:29:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.634+0000 7fc478a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc4741643d0 con 0x7fc4740d44a0 2024-04-25T11:29:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.634+0000 7fc478a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc4740b5270 con 0x7fc4740b7c50 2024-04-25T11:29:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.635+0000 7fc473400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc474164860 0x7fc4740b7540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51688/0 (socket says 172.21.15.19:51688) 2024-04-25T11:29:01.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.635+0000 7fc473400640 1 -- 172.21.15.19:0/1030222793 learned_addr learned my addr 172.21.15.19:0/1030222793 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:01.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.635+0000 7fc473e00640 1 --2- 172.21.15.19:0/1030222793 >> v2:172.21.15.19:3300/0 conn(0x7fc4740d44a0 0x7fc4740d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:01.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.635+0000 7fc472a00640 1 --2- 172.21.15.19:0/1030222793 >> v2:172.21.15.169:3300/0 conn(0x7fc4740b7c50 0x7fc4740b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:01.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.635+0000 7fc472000640 1 -- 172.21.15.19:0/1030222793 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1880733204 0 0) 0x7fc474161650 con 0x7fc474164860 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.636+0000 7fc472000640 1 -- 172.21.15.19:0/1030222793 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc458003520 con 0x7fc474164860 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.636+0000 7fc473e00640 1 -- 172.21.15.19:0/1030222793 >> v2:172.21.15.169:3300/0 conn(0x7fc4740b7c50 msgr2=0x7fc4740b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.636+0000 7fc473e00640 1 --2- 172.21.15.19:0/1030222793 >> v2:172.21.15.169:3300/0 conn(0x7fc4740b7c50 0x7fc4740b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.636+0000 7fc473e00640 1 -- 172.21.15.19:0/1030222793 >> v1:172.21.15.19:6789/0 conn(0x7fc474164860 legacy=0x7fc4740b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.636+0000 7fc473e00640 1 -- 172.21.15.19:0/1030222793 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc474161370 con 0x7fc4740d44a0 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.636+0000 7fc472a00640 1 --2- 172.21.15.19:0/1030222793 >> v2:172.21.15.169:3300/0 conn(0x7fc4740b7c50 0x7fc4740b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.637+0000 7fc473e00640 1 --2- 172.21.15.19:0/1030222793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4740d44a0 0x7fc4740d48b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc47408e4c0 tx=0x7fc46c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dfa42483a5e134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.637+0000 7fc472000640 1 -- 172.21.15.19:0/1030222793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c007740 con 0x7fc4740d44a0 2024-04-25T11:29:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.637+0000 7fc472000640 1 -- 172.21.15.19:0/1030222793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46c0078e0 con 0x7fc4740d44a0 2024-04-25T11:29:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.637+0000 7fc472000640 1 -- 172.21.15.19:0/1030222793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c007cb0 con 0x7fc4740d44a0 2024-04-25T11:29:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 -- 172.21.15.19:0/1030222793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4740d44a0 msgr2=0x7fc4740d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 --2- 172.21.15.19:0/1030222793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4740d44a0 0x7fc4740d48b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc47408e4c0 tx=0x7fc46c02a680 comp rx=0 tx=0).stop 2024-04-25T11:29:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 -- 172.21.15.19:0/1030222793 shutdown_connections 2024-04-25T11:29:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 --2- 172.21.15.19:0/1030222793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4740d44a0 0x7fc4740d48b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 --2- 172.21.15.19:0/1030222793 >> v2:172.21.15.169:3300/0 conn(0x7fc4740b7c50 0x7fc4740b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 -- 172.21.15.19:0/1030222793 >> 172.21.15.19:0/1030222793 conn(0x7fc4740d2720 msgr2=0x7fc4740d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 -- 172.21.15.19:0/1030222793 shutdown_connections 2024-04-25T11:29:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.638+0000 7fc478a00640 1 -- 172.21.15.19:0/1030222793 wait complete. 2024-04-25T11:29:01.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.640+0000 7fc478a00640 1 Processor -- start 2024-04-25T11:29:01.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.641+0000 7fc478a00640 1 -- start start 2024-04-25T11:29:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.642+0000 7fc478a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4740d44a0 0x7fc4740cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.642+0000 7fc478a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474164860 0x7fc474152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.642+0000 7fc478a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4740c95b0 0x7fc4741f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.642+0000 7fc478a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4740d4df0 con 0x7fc474164860 2024-04-25T11:29:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.642+0000 7fc478a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc474164060 con 0x7fc4740d44a0 2024-04-25T11:29:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.642+0000 7fc478a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc4740d4f70 con 0x7fc4740c95b0 2024-04-25T11:29:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474164860 0x7fc474152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:01.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474164860 0x7fc474152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37476/0 (socket says 172.21.15.19:37476) 2024-04-25T11:29:01.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 -- 172.21.15.19:0/3500040333 learned_addr learned my addr 172.21.15.19:0/3500040333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:01.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc473400640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4740d44a0 0x7fc4740cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:01.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc473e00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4740c95b0 0x7fc4741f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 -- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4740c95b0 msgr2=0x7fc4741f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4740c95b0 0x7fc4741f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 -- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4740d44a0 msgr2=0x7fc4740cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4740d44a0 0x7fc4740cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.643+0000 7fc472a00640 1 -- 172.21.15.19:0/3500040333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc474153450 con 0x7fc474164860 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.644+0000 7fc472a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474164860 0x7fc474152f10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc46000d800 tx=0x7fc46000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.644+0000 7fc46b400640 1 -- 172.21.15.19:0/3500040333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460004030 con 0x7fc474164860 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.644+0000 7fc46b400640 1 -- 172.21.15.19:0/3500040333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4600041d0 con 0x7fc474164860 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.644+0000 7fc46b400640 1 -- 172.21.15.19:0/3500040333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460009640 con 0x7fc474164860 2024-04-25T11:29:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.644+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4740bb5e0 con 0x7fc474164860 2024-04-25T11:29:01.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.644+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741f7f10 con 0x7fc474164860 2024-04-25T11:29:01.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.645+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474050740 con 0x7fc474164860 2024-04-25T11:29:01.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.647+0000 7fc46b400640 1 -- 172.21.15.19:0/3500040333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc4600097e0 con 0x7fc474164860 2024-04-25T11:29:01.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.648+0000 7fc46b400640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4540744b0 0x7fc4540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:01.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.648+0000 7fc46b400640 1 -- 172.21.15.19:0/3500040333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fc460095450 con 0x7fc474164860 2024-04-25T11:29:01.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.649+0000 7fc473400640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4540744b0 0x7fc4540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.650+0000 7fc473400640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4540744b0 0x7fc4540769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc464005880 tx=0x7fc464009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.653+0000 7fc46b400640 1 -- 172.21.15.19:0/3500040333 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc46005fb70 con 0x7fc474164860 2024-04-25T11:29:01.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.896+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4740d3f30 con 0x7fc474164860 2024-04-25T11:29:01.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.897+0000 7fc46b400640 1 -- 172.21.15.19:0/3500040333 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5172 (secure 0 0 0) 0x7fc46005f470 con 0x7fc474164860 2024-04-25T11:29:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:01.889 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:01.329106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-25T11:29:01.889 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 20 2024-04-25T11:29:01.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4540744b0 msgr2=0x7fc4540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4540744b0 0x7fc4540769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc464005880 tx=0x7fc464009040 comp rx=0 tx=0).stop 2024-04-25T11:29:01.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474164860 msgr2=0x7fc474152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474164860 0x7fc474152f10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc46000d800 tx=0x7fc46000dcc0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 shutdown_connections 2024-04-25T11:29:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4540744b0 0x7fc4540769c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4740c95b0 0x7fc4741f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc474164860 0x7fc474152f10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 --2- 172.21.15.19:0/3500040333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4740d44a0 0x7fc4740cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 >> 172.21.15.19:0/3500040333 conn(0x7fc4740d2720 msgr2=0x7fc474161540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 shutdown_connections 2024-04-25T11:29:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:01.900+0000 7fc478a00640 1 -- 172.21.15.19:0/3500040333 wait complete. 2024-04-25T11:29:02.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:28:45.819822+0000', 'modified': '2024-04-25T11:29:01.329106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5331}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5331': {'gid': 5331, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/3972745609', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3972745609}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3972745609}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:29:02.318 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5331:up:replay 2024-04-25T11:29:02.971 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:29:02.979 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:29:03.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.615+0000 7fdbbd200640 1 Processor -- start 2024-04-25T11:29:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.616+0000 7fdbbd200640 1 -- start start 2024-04-25T11:29:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.618+0000 7fdbbd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 0x7fdbb80b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.618+0000 7fdbbd200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdbb80d2030 0x7fdbb80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.618+0000 7fdbbd200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb80b3ef0 con 0x7fdbb80cfb90 2024-04-25T11:29:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.618+0000 7fdbbd200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb8164610 con 0x7fdbb80d2030 2024-04-25T11:29:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.618+0000 7fdbbd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb8163fc0 con 0x7fdbb8164aa0 2024-04-25T11:29:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.618+0000 7fdbb6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdbb80cfb90 0x7fdbb80d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51700/0 (socket says 172.21.15.19:51700) 2024-04-25T11:29:03.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.618+0000 7fdbb6a00640 1 -- 172.21.15.19:0/2244204195 learned_addr learned my addr 172.21.15.19:0/2244204195 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb7e00640 1 --2- 172.21.15.19:0/2244204195 >> v2:172.21.15.169:3300/0 conn(0x7fdbb80d2030 0x7fdbb80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb7400640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 0x7fdbb80b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379657174 0 0) 0x7fdbb80b3ef0 con 0x7fdbb80cfb90 2024-04-25T11:29:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb98003520 con 0x7fdbb80cfb90 2024-04-25T11:29:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2838640497 0 0) 0x7fdb98003520 con 0x7fdbb80cfb90 2024-04-25T11:29:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 msgr2=0x7fdbb80b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:03.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 0x7fdbb80b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 >> v2:172.21.15.169:3300/0 conn(0x7fdbb80d2030 msgr2=0x7fdbb80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 --2- 172.21.15.19:0/2244204195 >> v2:172.21.15.169:3300/0 conn(0x7fdbb80d2030 0x7fdbb80b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb81642a0 con 0x7fdbb80cfb90 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fdba00019f0 con 0x7fdbb80cfb90 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 >> v1:172.21.15.19:6789/0 conn(0x7fdbb80cfb90 legacy=0x7fdbb80d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 0x7fdb980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdb98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb98004d40 0x7fdb98005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8164610 con 0x7fdb98003f80 2024-04-25T11:29:03.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8163fc0 con 0x7fdb98004d40 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.619+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9800ba70 con 0x7fdbb8164aa0 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb6a00640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdb98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb7400640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 0x7fdb980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb7e00640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb98004d40 0x7fdb98005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb6a00640 1 -- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 msgr2=0x7fdb980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb6a00640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb8164aa0 0x7fdb980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb6a00640 1 -- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb98004d40 msgr2=0x7fdb98005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb6a00640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb98004d40 0x7fdb98005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.620+0000 7fdbb6a00640 1 -- 172.21.15.19:0/2244204195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb98003520 con 0x7fdb98003f80 2024-04-25T11:29:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.621+0000 7fdbb6a00640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdb98004800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80d3890 tx=0x7fdb98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3324ef10b15884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.622+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba000a410 con 0x7fdb98003f80 2024-04-25T11:29:03.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.622+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba002f070 con 0x7fdb98003f80 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.622+0000 7fdbb6000640 1 -- 172.21.15.19:0/2244204195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba002bbe0 con 0x7fdb98003f80 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.622+0000 7fdbbd200640 1 -- 172.21.15.19:0/2244204195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 msgr2=0x7fdb98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.622+0000 7fdbbd200640 1 --2- 172.21.15.19:0/2244204195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdb98004800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80d3890 tx=0x7fdb98005830 comp rx=0 tx=0).stop 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.623+0000 7fdbb7e00640 1 -- 172.21.15.19:0/2244204195 reap_dead start 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.623+0000 7fdbbd200640 1 -- 172.21.15.19:0/2244204195 shutdown_connections 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.623+0000 7fdbbd200640 1 -- 172.21.15.19:0/2244204195 >> 172.21.15.19:0/2244204195 conn(0x7fdbb809b7d0 msgr2=0x7fdbb809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.623+0000 7fdbbd200640 1 -- 172.21.15.19:0/2244204195 shutdown_connections 2024-04-25T11:29:03.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.623+0000 7fdbbd200640 1 -- 172.21.15.19:0/2244204195 wait complete. 2024-04-25T11:29:03.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.626+0000 7fdbbd200640 1 Processor -- start 2024-04-25T11:29:03.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.626+0000 7fdbbd200640 1 -- start start 2024-04-25T11:29:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.628+0000 7fdbbd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdbb81547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.628+0000 7fdbbd200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb80b20e0 0x7fdbb806d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.628+0000 7fdbbd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb806db50 0x7fdbb8155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.628+0000 7fdbbd200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb80b3a90 con 0x7fdb98003f80 2024-04-25T11:29:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.628+0000 7fdbbd200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb80b3910 con 0x7fdbb80b20e0 2024-04-25T11:29:03.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.628+0000 7fdbbd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb80b3c10 con 0x7fdbb806db50 2024-04-25T11:29:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdbb81547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdbb81547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37498/0 (socket says 172.21.15.19:37498) 2024-04-25T11:29:03.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 -- 172.21.15.19:0/2793421483 learned_addr learned my addr 172.21.15.19:0/2793421483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb6a00640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb80b20e0 0x7fdbb806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 -- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb806db50 msgr2=0x7fdbb8155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb806db50 0x7fdbb8155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 -- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb80b20e0 msgr2=0x7fdbb806d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb80b20e0 0x7fdbb806d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.629+0000 7fdbb7400640 1 -- 172.21.15.19:0/2793421483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb806df60 con 0x7fdb98003f80 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.630+0000 7fdbb7400640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdbb81547a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdba8011600 tx=0x7fdba8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.630+0000 7fdbaf400640 1 -- 172.21.15.19:0/2793421483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8015870 con 0x7fdb98003f80 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.630+0000 7fdbaf400640 1 -- 172.21.15.19:0/2793421483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba8015a10 con 0x7fdb98003f80 2024-04-25T11:29:03.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.630+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb815fc20 con 0x7fdb98003f80 2024-04-25T11:29:03.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.630+0000 7fdbaf400640 1 -- 172.21.15.19:0/2793421483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8015e00 con 0x7fdb98003f80 2024-04-25T11:29:03.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.630+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb80b03a0 con 0x7fdb98003f80 2024-04-25T11:29:03.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.631+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb815fdf0 con 0x7fdb98003f80 2024-04-25T11:29:03.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.633+0000 7fdbaf400640 1 -- 172.21.15.19:0/2793421483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdba8013430 con 0x7fdb98003f80 2024-04-25T11:29:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.634+0000 7fdbaf400640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb940744b0 0x7fdb940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.635+0000 7fdbaf400640 1 -- 172.21.15.19:0/2793421483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fdba8095c10 con 0x7fdb98003f80 2024-04-25T11:29:03.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.635+0000 7fdbb6a00640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb940744b0 0x7fdb940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:03.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.636+0000 7fdbb6a00640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb940744b0 0x7fdb940769c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdba0033000 tx=0x7fdba0007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:03.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.640+0000 7fdbaf400640 1 -- 172.21.15.19:0/2793421483 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbb806df60 con 0x7fdb98003f80 2024-04-25T11:29:03.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.884+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb8050740 con 0x7fdb98003f80 2024-04-25T11:29:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.885+0000 7fdbaf400640 1 -- 172.21.15.19:0/2793421483 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5172 (secure 0 0 0) 0x7fdba8019030 con 0x7fdb98003f80 2024-04-25T11:29:03.875 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:03.875 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:02.344275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-25T11:29:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 22 2024-04-25T11:29:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.887+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb940744b0 msgr2=0x7fdb940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb940744b0 0x7fdb940769c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdba0033000 tx=0x7fdba0007000 comp rx=0 tx=0).stop 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 msgr2=0x7fdbb81547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdbb81547a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdba8011600 tx=0x7fdba8011ac0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 shutdown_connections 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb940744b0 0x7fdb940769c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdbb80b20e0 0x7fdbb806d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdbb806db50 0x7fdbb8155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 --2- 172.21.15.19:0/2793421483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb98003f80 0x7fdbb81547a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 >> 172.21.15.19:0/2793421483 conn(0x7fdbb809b7d0 msgr2=0x7fdbb815ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 shutdown_connections 2024-04-25T11:29:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:03.888+0000 7fdbbd200640 1 -- 172.21.15.19:0/2793421483 wait complete. 2024-04-25T11:29:04.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:28:45.819822+0000', 'modified': '2024-04-25T11:29:02.344275+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5331}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5331': {'gid': 5331, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.19:6835/3972745609', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3972745609}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3972745609}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:29:04.307 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5331:up:rejoin 2024-04-25T11:29:05.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.615+0000 7f21f3e00640 1 Processor -- start 2024-04-25T11:29:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.615+0000 7f21f3e00640 1 -- start start 2024-04-25T11:29:05.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.617+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 0x7f21f4160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.617+0000 7f21f3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f40b7940 con 0x7f21f40d50e0 2024-04-25T11:29:05.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.617+0000 7f21f3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f4161280 con 0x7f21f4160b90 2024-04-25T11:29:05.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.617+0000 7f21f3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21f40d4600 con 0x7f21f40d2cb0 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.617+0000 7f21f2a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f21f40d50e0 0x7f21f40d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42974/0 (socket says 172.21.15.19:42974) 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.617+0000 7f21f2a00640 1 -- 172.21.15.19:0/1088848712 learned_addr learned my addr 172.21.15.19:0/1088848712 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f2000640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 0x7f21f4160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2957557620 0 0) 0x7f21e4001110 con 0x7f21f40d50e0 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21d8003520 con 0x7f21f40d50e0 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2543575412 0 0) 0x7f21d8003520 con 0x7f21f40d50e0 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 msgr2=0x7f21f4160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 0x7f21f4160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 >> v1:172.21.15.169:6789/0 conn(0x7f21f4160b90 legacy=0x7f21f4161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40d48e0 con 0x7f21f40d50e0 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f21e4001860 con 0x7f21f40d50e0 2024-04-25T11:29:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f2000640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 0x7f21f4160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 >> v1:172.21.15.19:6789/0 conn(0x7f21f40d50e0 legacy=0x7f21f40d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 0x7f21d80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d8004040 0x7f21d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004d90 0x7f21d8005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21f40d4600 con 0x7f21d8004040 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21d800bc20 con 0x7f21d8004d90 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.618+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21d800be20 con 0x7f21f40d2cb0 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f2000640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d8004040 0x7f21d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f2a00640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 0x7f21d80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f3400640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004d90 0x7f21d8005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f2000640 1 -- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 msgr2=0x7f21d80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f2000640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f40d2cb0 0x7f21d80039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f2000640 1 -- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004d90 msgr2=0x7f21d8005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f2000640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004d90 0x7f21d8005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f2000640 1 -- 172.21.15.19:0/1088848712 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d8003520 con 0x7f21d8004040 2024-04-25T11:29:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.619+0000 7f21f3400640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004d90 0x7f21d8005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.620+0000 7f21f2000640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d8004040 0x7f21d8004800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f21e00098a0 tx=0x7f21d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c75d891280d8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000f040 con 0x7f21d8004040 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e002ad00 con 0x7f21d8004040 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f1600640 1 -- 172.21.15.19:0/1088848712 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0004250 con 0x7f21d8004040 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f3e00640 1 -- 172.21.15.19:0/1088848712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d8004040 msgr2=0x7f21d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f3e00640 1 --2- 172.21.15.19:0/1088848712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d8004040 0x7f21d8004800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f21e00098a0 tx=0x7f21d8005830 comp rx=0 tx=0).stop 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f3400640 1 -- 172.21.15.19:0/1088848712 reap_dead start 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f3e00640 1 -- 172.21.15.19:0/1088848712 shutdown_connections 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.621+0000 7f21f3e00640 1 -- 172.21.15.19:0/1088848712 >> 172.21.15.19:0/1088848712 conn(0x7f21f406c5c0 msgr2=0x7f21f406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.622+0000 7f21f3e00640 1 -- 172.21.15.19:0/1088848712 shutdown_connections 2024-04-25T11:29:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.622+0000 7f21f3e00640 1 -- 172.21.15.19:0/1088848712 wait complete. 2024-04-25T11:29:05.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.624+0000 7f21f3e00640 1 Processor -- start 2024-04-25T11:29:05.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.624+0000 7f21f3e00640 1 -- start start 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004040 0x7f21f414e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f41eba00 0x7f21f41f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f41f6ec0 0x7f21f41e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4161560 con 0x7f21f41f6ec0 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4160fa0 con 0x7f21d8004040 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21f40d54f0 con 0x7f21f41eba00 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f41f6ec0 0x7f21f41e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f41f6ec0 0x7f21f41e8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37886/0 (socket says 172.21.15.19:37886) 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f3400640 1 -- 172.21.15.19:0/2880197127 learned_addr learned my addr 172.21.15.19:0/2880197127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f2a00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004040 0x7f21f414e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.626+0000 7f21f2000640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f41eba00 0x7f21f41f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21f3400640 1 -- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f41eba00 msgr2=0x7f21f41f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21f3400640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f41eba00 0x7f21f41f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21f3400640 1 -- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004040 msgr2=0x7f21f414e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21f3400640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004040 0x7f21f414e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21f3400640 1 -- 172.21.15.19:0/2880197127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40ca100 con 0x7f21f41f6ec0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21f2a00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004040 0x7f21f414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21f3400640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f41f6ec0 0x7f21f41e8d60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f21f40d3710 tx=0x7f21ec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21eaa00640 1 -- 172.21.15.19:0/2880197127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec006d50 con 0x7f21f41f6ec0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21eaa00640 1 -- 172.21.15.19:0/2880197127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec003970 con 0x7f21f41f6ec0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.627+0000 7f21eaa00640 1 -- 172.21.15.19:0/2880197127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec003da0 con 0x7f21f41f6ec0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.628+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f40ca570 con 0x7f21f41f6ec0 2024-04-25T11:29:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.628+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f40cab30 con 0x7f21f41f6ec0 2024-04-25T11:29:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.629+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f40ca2d0 con 0x7f21f41f6ec0 2024-04-25T11:29:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.630+0000 7f21eaa00640 1 -- 172.21.15.19:0/2880197127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f21ec01b020 con 0x7f21f41f6ec0 2024-04-25T11:29:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.631+0000 7f21eaa00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d40744e0 0x7f21d40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.631+0000 7f21eaa00640 1 -- 172.21.15.19:0/2880197127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5911+0+0 (secure 0 0 0) 0x7f21ec095060 con 0x7f21f41f6ec0 2024-04-25T11:29:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.632+0000 7f21f2a00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d40744e0 0x7f21d40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.634+0000 7f21f2a00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d40744e0 0x7f21d40769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f21e4001010 tx=0x7f21e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.636+0000 7f21eaa00640 1 -- 172.21.15.19:0/2880197127 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21f40ca2d0 con 0x7f21f41f6ec0 2024-04-25T11:29:05.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.886+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21f4050740 con 0x7f21f41f6ec0 2024-04-25T11:29:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.887+0000 7f21eaa00640 1 -- 172.21.15.19:0/2880197127 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5179 (secure 0 0 0) 0x7f21ec05f710 con 0x7f21f41f6ec0 2024-04-25T11:29:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:05.878 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:04.349778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5331,"qdb_cluster":[5331]},"id":2}]} 2024-04-25T11:29:05.878 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 23 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.890+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d40744e0 msgr2=0x7f21d40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.890+0000 7f21f3e00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d40744e0 0x7f21d40769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f21e4001010 tx=0x7f21e4009040 comp rx=0 tx=0).stop 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.890+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f41f6ec0 msgr2=0x7f21f41e8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.890+0000 7f21f3e00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f41f6ec0 0x7f21f41e8d60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f21f40d3710 tx=0x7f21ec00dc00 comp rx=0 tx=0).stop 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.891+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 shutdown_connections 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.891+0000 7f21f3e00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d40744e0 0x7f21d40769f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.891+0000 7f21f3e00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f41f6ec0 0x7f21f41e8d60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.891+0000 7f21f3e00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f41eba00 0x7f21f41f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.891+0000 7f21f3e00640 1 --2- 172.21.15.19:0/2880197127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21d8004040 0x7f21f414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.891+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 >> 172.21.15.19:0/2880197127 conn(0x7f21f406c5c0 msgr2=0x7f21f40b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.891+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 shutdown_connections 2024-04-25T11:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:05.892+0000 7f21f3e00640 1 -- 172.21.15.19:0/2880197127 wait complete. 2024-04-25T11:29:06.316 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:28:45.819822+0000', 'modified': '2024-04-25T11:29:04.349778+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 36, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5331}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5331': {'gid': 5331, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/3972745609', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3972745609}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3972745609}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5331, 'qdb_cluster': [5331]} 2024-04-25T11:29:06.316 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:29:06.316 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:29:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.631+0000 7f3246000640 1 Processor -- start 2024-04-25T11:29:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.631+0000 7f3246000640 1 -- start start 2024-04-25T11:29:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.634+0000 7f3246000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32400b4d30 0x7f32400d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.634+0000 7f3246000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3240161650 con 0x7f3240164860 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.634+0000 7f3246000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3240161370 con 0x7f32400b7c50 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.634+0000 7f3246000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f32401643d0 con 0x7f32400b4d30 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.634+0000 7f3245600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32400b4d30 0x7f32400d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.634+0000 7f3244c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3240164860 0x7f32400b7540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42978/0 (socket says 172.21.15.19:42978) 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.634+0000 7f3244c00640 1 -- 172.21.15.19:0/950302560 learned_addr learned my addr 172.21.15.19:0/950302560 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.635+0000 7f3245600640 1 -- 172.21.15.19:0/950302560 >> v1:172.21.15.169:6789/0 conn(0x7f32400b7c50 legacy=0x7f32400d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.635+0000 7f3245600640 1 -- 172.21.15.19:0/950302560 >> v1:172.21.15.19:6789/0 conn(0x7f3240164860 legacy=0x7f32400b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.635+0000 7f3245600640 1 -- 172.21.15.19:0/950302560 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3240163d80 con 0x7f32400b4d30 2024-04-25T11:29:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.635+0000 7f323f400640 1 -- 172.21.15.19:0/950302560 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992438857 0 0) 0x7f3240161650 con 0x7f3240164860 2024-04-25T11:29:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.636+0000 7f3245600640 1 --2- 172.21.15.19:0/950302560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4d30 0x7f32400d4bb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3238009d90 tx=0x7f323802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8840d2290e4a1228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.636+0000 7f323f400640 1 -- 172.21.15.19:0/950302560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238007740 con 0x7f32400b4d30 2024-04-25T11:29:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.636+0000 7f323f400640 1 -- 172.21.15.19:0/950302560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32380078e0 con 0x7f32400b4d30 2024-04-25T11:29:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.636+0000 7f323f400640 1 -- 172.21.15.19:0/950302560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238007cb0 con 0x7f32400b4d30 2024-04-25T11:29:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.637+0000 7f3246000640 1 -- 172.21.15.19:0/950302560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4d30 msgr2=0x7f32400d4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.637+0000 7f3246000640 1 --2- 172.21.15.19:0/950302560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4d30 0x7f32400d4bb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3238009d90 tx=0x7f323802a6f0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.637+0000 7f3246000640 1 -- 172.21.15.19:0/950302560 shutdown_connections 2024-04-25T11:29:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.637+0000 7f3246000640 1 --2- 172.21.15.19:0/950302560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4d30 0x7f32400d4bb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.637+0000 7f3246000640 1 -- 172.21.15.19:0/950302560 >> 172.21.15.19:0/950302560 conn(0x7f32400d2720 msgr2=0x7f32400d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.637+0000 7f3246000640 1 -- 172.21.15.19:0/950302560 shutdown_connections 2024-04-25T11:29:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.637+0000 7f3246000640 1 -- 172.21.15.19:0/950302560 wait complete. 2024-04-25T11:29:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.640+0000 7f3246000640 1 Processor -- start 2024-04-25T11:29:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.640+0000 7f3246000640 1 -- start start 2024-04-25T11:29:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.641+0000 7f3246000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32400b4d30 0x7f3240155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.641+0000 7f3246000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3240164860 0x7f32400ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.641+0000 7f3246000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400ac9d0 0x7f32400acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.641+0000 7f3246000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32400d50f0 con 0x7f3240164860 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.641+0000 7f3246000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32400b5140 con 0x7f32400b4d30 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.641+0000 7f3246000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32400d5270 con 0x7f32400ac9d0 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3240164860 0x7f32400ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3240164860 0x7f32400ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37906/0 (socket says 172.21.15.19:37906) 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 -- 172.21.15.19:0/3693470767 learned_addr learned my addr 172.21.15.19:0/3693470767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f3244c00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32400b4d30 0x7f3240155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 -- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400ac9d0 msgr2=0x7f32400acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f3245600640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400ac9d0 0x7f32400acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400ac9d0 0x7f32400acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 -- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32400b4d30 msgr2=0x7f3240155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32400b4d30 0x7f3240155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f323fe00640 1 -- 172.21.15.19:0/3693470767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32401f61b0 con 0x7f3240164860 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.642+0000 7f3244c00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32400b4d30 0x7f3240155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.643+0000 7f323fe00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3240164860 0x7f32400ac490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c3f0 tx=0x7f322c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.643+0000 7f323cc00640 1 -- 172.21.15.19:0/3693470767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c017070 con 0x7f3240164860 2024-04-25T11:29:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.643+0000 7f323cc00640 1 -- 172.21.15.19:0/3693470767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c0052f0 con 0x7f3240164860 2024-04-25T11:29:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.643+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32400cec00 con 0x7f3240164860 2024-04-25T11:29:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.643+0000 7f323cc00640 1 -- 172.21.15.19:0/3693470767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c003830 con 0x7f3240164860 2024-04-25T11:29:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.643+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32400ce920 con 0x7f3240164860 2024-04-25T11:29:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.646+0000 7f323cc00640 1 -- 172.21.15.19:0/3693470767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f322c01a070 con 0x7f3240164860 2024-04-25T11:29:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.647+0000 7f323cc00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32200744b0 0x7f32200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.647+0000 7f323cc00640 1 -- 172.21.15.19:0/3693470767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f322c094740 con 0x7f3240164860 2024-04-25T11:29:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.647+0000 7f324e64f740 1 Processor -- start 2024-04-25T11:29:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.648+0000 7f324e64f740 1 -- start start 2024-04-25T11:29:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.648+0000 7f3244c00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32200744b0 0x7f32200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.649+0000 7f3244c00640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32200744b0 0x7f32200769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3234000f60 tx=0x7f3234009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.649+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.649+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be5d3b0 0x56379c26df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.649+0000 7f324e64f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c26e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.649+0000 7f324e64f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56379c18c600 con 0x56379be65590 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.649+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56379c129100 con 0x56379be5d3b0 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.649+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56379c1885c0 con 0x56379be86270 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c26e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c26e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37922/0 (socket says 172.21.15.19:37922) 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 -- 172.21.15.19:0/2552327690 learned_addr learned my addr 172.21.15.19:0/2552327690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3244c00640 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f323fe00640 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be5d3b0 0x56379c26df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 -- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be86270 msgr2=0x56379c253210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 -- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be5d3b0 msgr2=0x56379c26df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be5d3b0 0x56379c26df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f3245600640 1 -- 172.21.15.19:0/2552327690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56379be9e2d0 con 0x56379be65590 2024-04-25T11:29:06.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.650+0000 7f323fe00640 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be5d3b0 0x56379c26df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.651+0000 7f3245600640 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c26e480 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3238009e80 tx=0x7f323802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4901f4761aaa5d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.651+0000 7f3231200640 1 -- 172.21.15.19:0/2552327690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32380079b0 con 0x56379be65590 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.651+0000 7f3231200640 1 -- 172.21.15.19:0/2552327690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3238007b50 con 0x56379be65590 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.651+0000 7f3231200640 1 -- 172.21.15.19:0/2552327690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238045480 con 0x56379be65590 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 -- 172.21.15.19:0/2552327690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 msgr2=0x56379c26e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c26e480 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3238009e80 tx=0x7f323802e000 comp rx=0 tx=0).stop 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 -- 172.21.15.19:0/2552327690 shutdown_connections 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be5d3b0 0x56379c26df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c26e480 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 --2- 172.21.15.19:0/2552327690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 -- 172.21.15.19:0/2552327690 >> 172.21.15.19:0/2552327690 conn(0x56379be7a3c0 msgr2=0x56379c228870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:06.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 -- 172.21.15.19:0/2552327690 shutdown_connections 2024-04-25T11:29:06.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.652+0000 7f324e64f740 1 -- 172.21.15.19:0/2552327690 wait complete. 2024-04-25T11:29:06.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.653+0000 7f324e64f740 1 -- start start 2024-04-25T11:29:06.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.653+0000 7f324e64f740 1 Processor -- start 2024-04-25T11:29:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c27c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be65590 0x56379c26e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f324e64f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be86270 0x56379c2693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f324e64f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56379c113f70 con 0x56379be86270 2024-04-25T11:29:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56379c269e90 con 0x56379be65590 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56379c255550 con 0x56379be5d3b0 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f324e64f740 10 client.0 _ll_register_callbacks cb 0x56379c182480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f3245600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be86270 0x56379c2693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.655+0000 7f3245600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be86270 0x56379c2693a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37936/0 (socket says 172.21.15.19:37936) 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3245600640 1 -- 172.21.15.19:0/82773134 learned_addr learned my addr 172.21.15.19:0/82773134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3244c00640 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c27c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3245600640 1 -- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 msgr2=0x56379c27c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f323fe00640 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be65590 0x56379c26e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3245600640 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c27c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3245600640 1 -- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be65590 msgr2=0x56379c26e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3245600640 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be65590 0x56379c26e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3245600640 1 -- 172.21.15.19:0/82773134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56379bf686b0 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3245600640 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be86270 0x56379c2693a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3238009d90 tx=0x7f3238004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e81841a180a201b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.656+0000 7f3225600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f3225600640 1 -- 172.21.15.19:0/82773134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238003b90 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3238007b50 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f3225600640 1 -- 172.21.15.19:0/82773134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f3238003d30 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f3225600640 1 -- 172.21.15.19:0/82773134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3238057050 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 10 client.5382 fetch_fsmap learned FSMap version 23 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56379c269bd0 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f3225600640 1 -- 172.21.15.19:0/82773134 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3238007b50 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f3225600640 1 -- 172.21.15.19:0/82773134 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f323805a020 con 0x56379be86270 2024-04-25T11:29:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 10 client.5382 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 10 client.5382 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 20 client.5382 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379be44e40 0x56379c280610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 4 client.5382 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.657+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56379c19a210 con 0x56379be44e40 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.658+0000 7f3244c00640 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379be44e40 0x56379c280610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.658+0000 7f3244c00640 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379be44e40 0x56379c280610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b173e3a79a8c75 server_cookie=652a1f89cc522a42 in_seq=0 out_seq=0 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.658+0000 7f3225600640 10 client.5382 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.659+0000 7f3225600640 1 -- 172.21.15.19:0/82773134 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56379c19a210 con 0x56379be44e40 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.659+0000 7f3225600640 10 client.5382 handle_command_reply: tid=0 2024-04-25T11:29:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.659+0000 7f324e64f740 1 client.5382 shutdown 2024-04-25T11:29:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be86270 msgr2=0x56379c2693a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be86270 0x56379c2693a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3238009d90 tx=0x7f3238004ab0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 shutdown_connections 2024-04-25T11:29:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379be44e40 0x56379c280610 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be65590 0x56379c26e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be86270 0x56379c2693a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 --2- 172.21.15.19:0/82773134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c27c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f3225600640 0 client.5382 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:06.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 >> 172.21.15.19:0/82773134 conn(0x56379be7a3c0 msgr2=0x56379c253210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:06.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 shutdown_connections 2024-04-25T11:29:06.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 1 -- 172.21.15.19:0/82773134 wait complete. 2024-04-25T11:29:06.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.660+0000 7f324e64f740 20 client.5382 trim_cache size 0 max 16384 2024-04-25T11:29:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.681+0000 7f324e64f740 1 Processor -- start 2024-04-25T11:29:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.681+0000 7f324e64f740 1 -- start start 2024-04-25T11:29:06.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c26f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f324e64f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c224cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f324e64f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3240201e50 con 0x56379be65590 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32401fa050 con 0x56379be86270 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32401fa6a0 con 0x56379be5d3b0 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f323fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c224cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f323fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c224cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37946/0 (socket says 172.21.15.19:37946) 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f323fe00640 1 -- 172.21.15.19:0/1733311479 learned_addr learned my addr 172.21.15.19:0/1733311479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f323fe00640 1 -- 172.21.15.19:0/1733311479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 msgr2=0x56379c26f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f323fe00640 1 --2- 172.21.15.19:0/1733311479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c26f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.683+0000 7f323fe00640 1 -- 172.21.15.19:0/1733311479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 msgr2=0x56379c253210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.684+0000 7f323fe00640 1 --2- 172.21.15.19:0/1733311479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.684+0000 7f323fe00640 1 -- 172.21.15.19:0/1733311479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56379c26f660 con 0x56379be65590 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.684+0000 7f323fe00640 1 --2- 172.21.15.19:0/1733311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c224cf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f322c00cdb0 tx=0x7f322c05e2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d421ed7aaf14be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.684+0000 7f3227e00640 1 -- 172.21.15.19:0/1733311479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c062a10 con 0x56379be65590 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.684+0000 7f3227e00640 1 -- 172.21.15.19:0/1733311479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c0674c0 con 0x56379be65590 2024-04-25T11:29:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.684+0000 7f3227e00640 1 -- 172.21.15.19:0/1733311479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c05dac0 con 0x56379be65590 2024-04-25T11:29:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 -- 172.21.15.19:0/1733311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 msgr2=0x56379c224cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 --2- 172.21.15.19:0/1733311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c224cf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f322c00cdb0 tx=0x7f322c05e2b0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 -- 172.21.15.19:0/1733311479 shutdown_connections 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 --2- 172.21.15.19:0/1733311479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 --2- 172.21.15.19:0/1733311479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c224cf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 --2- 172.21.15.19:0/1733311479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c26f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 -- 172.21.15.19:0/1733311479 >> 172.21.15.19:0/1733311479 conn(0x56379be7a3c0 msgr2=0x56379c26d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 -- 172.21.15.19:0/1733311479 shutdown_connections 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.685+0000 7f324e64f740 1 -- 172.21.15.19:0/1733311479 wait complete. 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.686+0000 7f324e64f740 1 -- start start 2024-04-25T11:29:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.686+0000 7f324e64f740 1 Processor -- start 2024-04-25T11:29:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c2290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f324e64f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c2eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f324e64f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f324e64f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56379c256d10 con 0x56379be65590 2024-04-25T11:29:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56379c269940 con 0x56379be86270 2024-04-25T11:29:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f324e64f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56379c26bd50 con 0x56379be5d3b0 2024-04-25T11:29:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f324e64f740 10 client.0 _ll_register_callbacks cb 0x7f3238007540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c2eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c2eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37958/0 (socket says 172.21.15.19:37958) 2024-04-25T11:29:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 -- 172.21.15.19:0/2095235506 learned_addr learned my addr 172.21.15.19:0/2095235506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 -- 172.21.15.19:0/2095235506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 msgr2=0x56379c2290e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c2290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 -- 172.21.15.19:0/2095235506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 msgr2=0x56379c253210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.688+0000 7f323fe00640 1 -- 172.21.15.19:0/2095235506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56379c2299b0 con 0x56379be65590 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f323fe00640 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c2eed80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f322c05a6e0 tx=0x7f322c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b797c3de3b1c4354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f3225600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f3225600640 1 -- 172.21.15.19:0/2095235506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c059fd0 con 0x56379be65590 2024-04-25T11:29:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f3225600640 1 -- 172.21.15.19:0/2095235506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f322c05a170 con 0x56379be65590 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f322c0674c0 con 0x56379be65590 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f3225600640 1 -- 172.21.15.19:0/2095235506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f322c073ab0 con 0x56379be65590 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f324e64f740 10 client.5388 fetch_fsmap learned FSMap version 23 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56379c2ef740 con 0x56379be65590 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.689+0000 7f3225600640 1 -- 172.21.15.19:0/2095235506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c0674c0 con 0x56379be65590 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f3225600640 1 -- 172.21.15.19:0/2095235506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f322c073c50 con 0x56379be65590 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f324e64f740 10 client.5388 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f324e64f740 10 client.5388 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f324e64f740 20 client.5388 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f324e64f740 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379c2eb350 0x56379c253750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f324e64f740 4 client.5388 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "status"}) -- 0x56379c194080 con 0x56379c2eb350 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.690+0000 7f3244c00640 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379c2eb350 0x56379c253750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f3244c00640 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379c2eb350 0x56379c253750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af6b2fea8e9af07 server_cookie=2ba2c50267ddea48 in_seq=0 out_seq=0 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f3225600640 10 client.5388 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f3225600640 1 -- 172.21.15.19:0/2095235506 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56379c194080 con 0x56379c2eb350 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f3225600640 10 client.5388 handle_command_reply: tid=0 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 client.5388 shutdown 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 msgr2=0x56379c2eed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c2eed80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f322c05a6e0 tx=0x7f322c000ed0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 shutdown_connections 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56379c2eb350 0x56379c253750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56379be86270 0x56379c253210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56379be65590 0x56379c2eed80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 --2- 172.21.15.19:0/2095235506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56379be5d3b0 0x56379c2290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f3225600640 0 client.5388 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 >> 172.21.15.19:0/2095235506 conn(0x56379be7a3c0 msgr2=0x56379c26d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 shutdown_connections 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.691+0000 7f324e64f740 1 -- 172.21.15.19:0/2095235506 wait complete. 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f324e64f740 20 client.5388 trim_cache size 0 max 16384 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:29:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5331, 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.3527713639999996, 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 23, 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 38, 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 36, 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 6.2216226150000002 2024-04-25T11:29:06.686 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32200744b0 msgr2=0x7f32200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32200744b0 0x7f32200769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3234000f60 tx=0x7f3234009040 comp rx=0 tx=0).stop 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3240164860 msgr2=0x7f32400ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3240164860 0x7f32400ac490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c3f0 tx=0x7f322c00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 shutdown_connections 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32200744b0 0x7f32200769c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400ac9d0 0x7f32400acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3240164860 0x7f32400ac490 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 --2- 172.21.15.19:0/3693470767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32400b4d30 0x7f3240155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 >> 172.21.15.19:0/3693470767 conn(0x7f32400d2720 msgr2=0x7f3240160840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:06.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 shutdown_connections 2024-04-25T11:29:06.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.692+0000 7f3246000640 1 -- 172.21.15.19:0/3693470767 wait complete. 2024-04-25T11:29:06.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:29:06.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.999+0000 7f72d5e00640 1 Processor -- start 2024-04-25T11:29:06.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:06.999+0000 7f72d5e00640 1 -- start start 2024-04-25T11:29:06.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.001+0000 7f72d5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72d00cf5e0 0x7f72d00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.001+0000 7f72d5e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d00cff30 con 0x7f72d006c5c0 2024-04-25T11:29:06.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.001+0000 7f72d5e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72d015f5f0 con 0x7f72d01443c0 2024-04-25T11:29:06.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.001+0000 7f72d5e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f72d00708d0 con 0x7f72d00cf5e0 2024-04-25T11:29:06.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.002+0000 7f72cf400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f72d006c5c0 0x7f72d006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42986/0 (socket says 172.21.15.19:42986) 2024-04-25T11:29:06.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.002+0000 7f72cf400640 1 -- 172.21.15.19:0/2347462693 learned_addr learned my addr 172.21.15.19:0/2347462693 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:06.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.002+0000 7f72ce000640 1 --2- 172.21.15.19:0/2347462693 >> v2:172.21.15.169:3300/0 conn(0x7f72d00cf5e0 0x7f72d00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.002+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578642097 0 0) 0x7f72c8001110 con 0x7f72d006c5c0 2024-04-25T11:29:06.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.002+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72b4003520 con 0x7f72d006c5c0 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3395916862 0 0) 0x7f72b4003520 con 0x7f72d006c5c0 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 >> v2:172.21.15.169:3300/0 conn(0x7f72d00cf5e0 msgr2=0x7f72d00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 --2- 172.21.15.19:0/2347462693 >> v2:172.21.15.169:3300/0 conn(0x7f72d00cf5e0 0x7f72d00cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 >> v1:172.21.15.169:6789/0 conn(0x7f72d01443c0 legacy=0x7f72d006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d015e830 con 0x7f72d006c5c0 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f72c8001840 con 0x7f72d006c5c0 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 >> v1:172.21.15.19:6789/0 conn(0x7f72d006c5c0 legacy=0x7f72d006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4004800 0x7f72b4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b4005270 0x7f72b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72d00708d0 con 0x7f72b4003a00 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72b400c190 con 0x7f72b4005270 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72b400c390 con 0x7f72b4004800 2024-04-25T11:29:06.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.003+0000 7f72cea00640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72ce000640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4004800 0x7f72b4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cf400640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b4005270 0x7f72b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cea00640 1 -- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4004800 msgr2=0x7f72b4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cea00640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4004800 0x7f72b4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cea00640 1 -- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b4005270 msgr2=0x7f72b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cea00640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b4005270 0x7f72b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cea00640 1 -- 172.21.15.19:0/2347462693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b4003520 con 0x7f72b4003a00 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cf400640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b4005270 0x7f72b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:06.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.004+0000 7f72cf400640 1 -- 172.21.15.19:0/2347462693 reap_dead start 2024-04-25T11:29:06.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.005+0000 7f72cea00640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72b4003e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f72c0001e20 tx=0x7f72b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb1554e87ea8869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:06.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.005+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0001a70 con 0x7f72b4003a00 2024-04-25T11:29:06.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.005+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c002ae10 con 0x7f72b4003a00 2024-04-25T11:29:06.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.005+0000 7f72cd600640 1 -- 172.21.15.19:0/2347462693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0003bb0 con 0x7f72b4003a00 2024-04-25T11:29:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.006+0000 7f72d5e00640 1 -- 172.21.15.19:0/2347462693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 msgr2=0x7f72b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.006+0000 7f72d5e00640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72b4003e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f72c0001e20 tx=0x7f72b4005830 comp rx=0 tx=0).stop 2024-04-25T11:29:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.006+0000 7f72d5e00640 1 -- 172.21.15.19:0/2347462693 shutdown_connections 2024-04-25T11:29:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.006+0000 7f72d5e00640 1 --2- 172.21.15.19:0/2347462693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72b4003e70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.006+0000 7f72d5e00640 1 -- 172.21.15.19:0/2347462693 >> 172.21.15.19:0/2347462693 conn(0x7f72d00d1ae0 msgr2=0x7f72d00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.006+0000 7f72d5e00640 1 -- 172.21.15.19:0/2347462693 shutdown_connections 2024-04-25T11:29:06.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.006+0000 7f72d5e00640 1 -- 172.21.15.19:0/2347462693 wait complete. 2024-04-25T11:29:06.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.009+0000 7f72d5e00640 1 Processor -- start 2024-04-25T11:29:06.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.009+0000 7f72d5e00640 1 -- start start 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72d01f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d01f3a70 0x7f72d015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d015dc50 0x7f72d00aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72d5e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72d015edf0 con 0x7f72b4003a00 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72d5e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72d015f160 con 0x7f72d01f3a70 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72d5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72d01447d0 con 0x7f72d015dc50 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72cea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72d01f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72cea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72d01f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37974/0 (socket says 172.21.15.19:37974) 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72cea00640 1 -- 172.21.15.19:0/4238863699 learned_addr learned my addr 172.21.15.19:0/4238863699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72cf400640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d015dc50 0x7f72d00aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.011+0000 7f72ce000640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d01f3a70 0x7f72d015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72cea00640 1 -- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d015dc50 msgr2=0x7f72d00aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72cea00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d015dc50 0x7f72d00aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72cea00640 1 -- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d01f3a70 msgr2=0x7f72d015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72cea00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d01f3a70 0x7f72d015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72cea00640 1 -- 172.21.15.19:0/4238863699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d01f33a0 con 0x7f72b4003a00 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72ce000640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d01f3a70 0x7f72d015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72cea00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72d01f28b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005040 tx=0x7f72c00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72c7400640 1 -- 172.21.15.19:0/4238863699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0037040 con 0x7f72b4003a00 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72c7400640 1 -- 172.21.15.19:0/4238863699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c002ac10 con 0x7f72b4003a00 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d01f3680 con 0x7f72b4003a00 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.012+0000 7f72c7400640 1 -- 172.21.15.19:0/4238863699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0007db0 con 0x7f72b4003a00 2024-04-25T11:29:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.013+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d01f3f20 con 0x7f72b4003a00 2024-04-25T11:29:07.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.014+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d0162b00 con 0x7f72b4003a00 2024-04-25T11:29:07.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.015+0000 7f72c7400640 1 -- 172.21.15.19:0/4238863699 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f72c0044020 con 0x7f72b4003a00 2024-04-25T11:29:07.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.016+0000 7f72c7400640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72ac0744b0 0x7f72ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:07.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.016+0000 7f72c7400640 1 -- 172.21.15.19:0/4238863699 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f72c00b51b0 con 0x7f72b4003a00 2024-04-25T11:29:07.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.017+0000 7f72ce000640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72ac0744b0 0x7f72ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:07.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.018+0000 7f72ce000640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72ac0744b0 0x7f72ac0769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f72b80090e0 tx=0x7f72b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:07.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.022+0000 7f72c7400640 1 -- 172.21.15.19:0/4238863699 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72d0162b00 con 0x7f72b4003a00 2024-04-25T11:29:07.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.267+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f72d0068f70 con 0x7f72b4003a00 2024-04-25T11:29:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.268+0000 7f72c7400640 1 -- 172.21.15.19:0/4238863699 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f72d0068f70 con 0x7f72b4003a00 2024-04-25T11:29:07.258 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72ac0744b0 msgr2=0x7f72ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72ac0744b0 0x7f72ac0769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f72b80090e0 tx=0x7f72b8008040 comp rx=0 tx=0).stop 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 msgr2=0x7f72d01f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72d01f28b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005040 tx=0x7f72c00014a0 comp rx=0 tx=0).stop 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 shutdown_connections 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72ac0744b0 0x7f72ac0769c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72d01f3a70 0x7f72d015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72d015dc50 0x7f72d00aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 --2- 172.21.15.19:0/4238863699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4003a00 0x7f72d01f28b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 >> 172.21.15.19:0/4238863699 conn(0x7f72d00d1ae0 msgr2=0x7f72d015c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 shutdown_connections 2024-04-25T11:29:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07.272+0000 7f72d5e00640 1 -- 172.21.15.19:0/4238863699 wait complete. 2024-04-25T11:29:07.289 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:29:07.289 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:29:07.290 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:29:07.290 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:29:07.290 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:07.290 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:29:07.290 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:29:07.290 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:29:07.290 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:29:07.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:07.290 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:29:07.331 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:29:07.341+0000 7fa144000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:29:07.343 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42526sec preferred_lft 42526sec 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:29:07.344 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:29:07.345 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:07.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:07.347 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:29:07.473 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:29:07.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:07.478 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:29:07.536 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:29:07.536 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:29:07.537 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:29:07.537 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:29:07.537 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:29:07.537 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:29:07.537 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:29:07.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:29:07.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:29:07.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:29:07.537 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:29:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:29:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:29:07.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:07.761 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:29:07.810 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:29:07.810 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:29:07.810 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:29:07.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:07.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:29:07.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:29:07.893 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:07.950 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:29:07.951 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:29:07.951 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:08.005 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:29:08.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:08.006 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:29:08.218 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:29:08.218 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:29:08.263 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:29:08.263 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:29:08.264 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:29:08.264 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:29:08.264 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:29:08.265 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:29:08.266 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:29:08.266 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:29:08.266 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:29:08.266 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:29:08.271 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:29:08.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:08.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:08.344 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:29:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:29:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:29:08.464 INFO:teuthology.orchestra.run.smithi019.stdout:client4760 2024-04-25T11:29:08.464 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:29:08.464 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:08.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:08.521 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T11:29:08.583 INFO:teuthology.orchestra.run.smithi019.stdout:file_undamaged 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T11:29:08.585 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T11:29:08.586 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:29:08.586 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T11:29:08.586 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T11:29:08.586 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T11:29:08.586 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T11:29:08.586 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:29:08.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:08 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T11:29:08.749 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T11:29:08.750 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:09.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.085+0000 7f5f49600640 1 Processor -- start 2024-04-25T11:29:09.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.086+0000 7f5f49600640 1 -- start start 2024-04-25T11:29:09.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.088+0000 7f5f49600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5f440cf9e0 0x7f5f44161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.088+0000 7f5f49600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f440709a0 0x7f5f441569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.088+0000 7f5f49600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f441615d0 con 0x7f5f440b0800 2024-04-25T11:29:09.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.088+0000 7f5f49600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5f440cffc0 con 0x7f5f440cf9e0 2024-04-25T11:29:09.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.088+0000 7f5f49600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5f440d02a0 con 0x7f5f440709a0 2024-04-25T11:29:09.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f43e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5f440cf9e0 0x7f5f44161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f43e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5f440cf9e0 0x7f5f44161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37978/0 (socket says 172.21.15.19:37978) 2024-04-25T11:29:09.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f43e00640 1 -- 172.21.15.19:0/2266480353 learned_addr learned my addr 172.21.15.19:0/2266480353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:09.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f48c00640 1 --2- 172.21.15.19:0/2266480353 >> v2:172.21.15.169:3300/0 conn(0x7f5f440709a0 0x7f5f441569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f43e00640 1 -- 172.21.15.19:0/2266480353 >> v2:172.21.15.169:3300/0 conn(0x7f5f440709a0 msgr2=0x7f5f441569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f43e00640 1 --2- 172.21.15.19:0/2266480353 >> v2:172.21.15.169:3300/0 conn(0x7f5f440709a0 0x7f5f441569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f43e00640 1 -- 172.21.15.19:0/2266480353 >> v1:172.21.15.169:6789/0 conn(0x7f5f440b0800 legacy=0x7f5f44070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.089+0000 7f5f43e00640 1 -- 172.21.15.19:0/2266480353 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f440d0580 con 0x7f5f440cf9e0 2024-04-25T11:29:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.090+0000 7f5f48c00640 1 --2- 172.21.15.19:0/2266480353 >> v2:172.21.15.169:3300/0 conn(0x7f5f440709a0 0x7f5f441569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.091+0000 7f5f43e00640 1 --2- 172.21.15.19:0/2266480353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f44161090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5f44074610 tx=0x7f5f3402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba52d87640bcd44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.091+0000 7f5f42a00640 1 -- 172.21.15.19:0/2266480353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34036070 con 0x7f5f440cf9e0 2024-04-25T11:29:09.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f42a00640 1 -- 172.21.15.19:0/2266480353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f34031600 con 0x7f5f440cf9e0 2024-04-25T11:29:09.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f42a00640 1 -- 172.21.15.19:0/2266480353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f340319d0 con 0x7f5f440cf9e0 2024-04-25T11:29:09.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f49600640 1 -- 172.21.15.19:0/2266480353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 msgr2=0x7f5f44161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f49600640 1 --2- 172.21.15.19:0/2266480353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f44161090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5f44074610 tx=0x7f5f3402a660 comp rx=0 tx=0).stop 2024-04-25T11:29:09.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f49600640 1 -- 172.21.15.19:0/2266480353 shutdown_connections 2024-04-25T11:29:09.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f49600640 1 --2- 172.21.15.19:0/2266480353 >> v2:172.21.15.169:3300/0 conn(0x7f5f440709a0 0x7f5f441569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f49600640 1 --2- 172.21.15.19:0/2266480353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f44161090 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f49600640 1 -- 172.21.15.19:0/2266480353 >> 172.21.15.19:0/2266480353 conn(0x7f5f440d1720 msgr2=0x7f5f440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.092+0000 7f5f49600640 1 -- 172.21.15.19:0/2266480353 shutdown_connections 2024-04-25T11:29:09.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.093+0000 7f5f49600640 1 -- 172.21.15.19:0/2266480353 wait complete. 2024-04-25T11:29:09.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.095+0000 7f5f49600640 1 Processor -- start 2024-04-25T11:29:09.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.095+0000 7f5f49600640 1 -- start start 2024-04-25T11:29:09.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f49600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f440b0800 0x7f5f440ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f49600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f441fa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f49600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f441fa670 0x7f5f440b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f49600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f440b0d90 con 0x7f5f440cf9e0 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f49600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f440b0c10 con 0x7f5f441fa670 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f49600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f44156ef0 con 0x7f5f440b0800 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f43400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f441fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f43400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f441fa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37984/0 (socket says 172.21.15.19:37984) 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f43400640 1 -- 172.21.15.19:0/1819063099 learned_addr learned my addr 172.21.15.19:0/1819063099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.097+0000 7f5f43e00640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f440b0800 0x7f5f440ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f48c00640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f441fa670 0x7f5f440b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f43400640 1 -- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f440b0800 msgr2=0x7f5f440ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f43400640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f440b0800 0x7f5f440ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f43400640 1 -- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f441fa670 msgr2=0x7f5f440b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f43400640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f441fa670 0x7f5f440b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f43400640 1 -- 172.21.15.19:0/1819063099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f440c67a0 con 0x7f5f440cf9e0 2024-04-25T11:29:09.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f43400640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f441fa130 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b420 tx=0x7f5f3000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f3be00640 1 -- 172.21.15.19:0/1819063099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011830 con 0x7f5f440cf9e0 2024-04-25T11:29:09.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f3be00640 1 -- 172.21.15.19:0/1819063099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f30016070 con 0x7f5f440cf9e0 2024-04-25T11:29:09.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.098+0000 7f5f3be00640 1 -- 172.21.15.19:0/1819063099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011c10 con 0x7f5f440cf9e0 2024-04-25T11:29:09.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.099+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f440c6a80 con 0x7f5f440cf9e0 2024-04-25T11:29:09.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.099+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f440c7040 con 0x7f5f440cf9e0 2024-04-25T11:29:09.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.100+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f44050740 con 0x7f5f440cf9e0 2024-04-25T11:29:09.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.101+0000 7f5f3be00640 1 -- 172.21.15.19:0/1819063099 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f3001b460 con 0x7f5f440cf9e0 2024-04-25T11:29:09.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.103+0000 7f5f3be00640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f240744e0 0x7f5f240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.103+0000 7f5f3be00640 1 -- 172.21.15.19:0/1819063099 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f5f30094eb0 con 0x7f5f440cf9e0 2024-04-25T11:29:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.103+0000 7f5f43e00640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f240744e0 0x7f5f240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.105+0000 7f5f43e00640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f240744e0 0x7f5f240769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f34033000 tx=0x7f5f34009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.108+0000 7f5f3be00640 1 -- 172.21.15.19:0/1819063099 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f30020020 con 0x7f5f440cf9e0 2024-04-25T11:29:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.362+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f4406fb60 con 0x7f5f440cf9e0 2024-04-25T11:29:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.363+0000 7f5f3be00640 1 -- 172.21.15.19:0/1819063099 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5179 (secure 0 0 0) 0x7f5f3005f4d0 con 0x7f5f440cf9e0 2024-04-25T11:29:09.354 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:09.354 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:04.349778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5331,"qdb_cluster":[5331]},"id":2}]} 2024-04-25T11:29:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 23 2024-04-25T11:29:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f240744e0 msgr2=0x7f5f240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f240744e0 0x7f5f240769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f34033000 tx=0x7f5f34009000 comp rx=0 tx=0).stop 2024-04-25T11:29:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 msgr2=0x7f5f441fa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f441fa130 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b420 tx=0x7f5f3000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 shutdown_connections 2024-04-25T11:29:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f240744e0 0x7f5f240769f0 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f34033000 tx=0x7f5f34009000 comp rx=0 tx=0).stop 2024-04-25T11:29:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f441fa670 0x7f5f440b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f440cf9e0 0x7f5f441fa130 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 --2- 172.21.15.19:0/1819063099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f440b0800 0x7f5f440ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.366+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 >> 172.21.15.19:0/1819063099 conn(0x7f5f440d1720 msgr2=0x7f5f440af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.367+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 shutdown_connections 2024-04-25T11:29:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.367+0000 7f5f49600640 1 -- 172.21.15.19:0/1819063099 wait complete. 2024-04-25T11:29:09.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T11:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.683+0000 7f5df6800640 1 Processor -- start 2024-04-25T11:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.683+0000 7f5df6800640 1 -- start start 2024-04-25T11:29:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.685+0000 7f5df6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5df006e6f0 0x7f5df00b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.685+0000 7f5df6800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df00b5e90 con 0x7f5df0146520 2024-04-25T11:29:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.685+0000 7f5df6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df00b6170 con 0x7f5df00cff70 2024-04-25T11:29:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.685+0000 7f5df6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5df006c7f0 con 0x7f5df006e6f0 2024-04-25T11:29:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.685+0000 7f5df5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5df0146520 0x7f5df006cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42990/0 (socket says 172.21.15.19:42990) 2024-04-25T11:29:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.685+0000 7f5df5e00640 1 -- 172.21.15.19:0/1205451974 learned_addr learned my addr 172.21.15.19:0/1205451974 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5df4a00640 1 --2- 172.21.15.19:0/1205451974 >> v2:172.21.15.169:3300/0 conn(0x7f5df006e6f0 0x7f5df00b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767788337 0 0) 0x7f5df00b5e90 con 0x7f5df0146520 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dd4003520 con 0x7f5df0146520 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 732129263 0 0) 0x7f5dd4003520 con 0x7f5df0146520 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 >> v2:172.21.15.169:3300/0 conn(0x7f5df006e6f0 msgr2=0x7f5df00b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 --2- 172.21.15.19:0/1205451974 >> v2:172.21.15.169:3300/0 conn(0x7f5df006e6f0 0x7f5df00b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 >> v1:172.21.15.169:6789/0 conn(0x7f5df00cff70 legacy=0x7f5df006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df00b6450 con 0x7f5df0146520 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f5de0001670 con 0x7f5df0146520 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 >> v1:172.21.15.19:6789/0 conn(0x7f5df0146520 legacy=0x7f5df006cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd40039b0 0x7f5dd4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd4004800 0x7f5dd4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.686+0000 7f5defe00640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd4005270 0x7f5dd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5df006c7f0 con 0x7f5dd40039b0 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd400c200 con 0x7f5dd4005270 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd400c400 con 0x7f5dd4004800 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5400640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd40039b0 0x7f5dd4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5e00640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd4005270 0x7f5dd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df4a00640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd4004800 0x7f5dd4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5400640 1 -- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd4004800 msgr2=0x7f5dd4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5400640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd4004800 0x7f5dd4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5400640 1 -- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd4005270 msgr2=0x7f5dd400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5400640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dd4005270 0x7f5dd400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5400640 1 -- 172.21.15.19:0/1205451974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd4003520 con 0x7f5dd40039b0 2024-04-25T11:29:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.687+0000 7f5df5e00640 1 -- 172.21.15.19:0/1205451974 reap_dead start 2024-04-25T11:29:09.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.688+0000 7f5df5400640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd40039b0 0x7f5dd4003e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5de8001e50 tx=0x7f5dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fccbc3b95a42ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.689+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8001a70 con 0x7f5dd40039b0 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.689+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de802ae40 con 0x7f5dd40039b0 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.689+0000 7f5defe00640 1 -- 172.21.15.19:0/1205451974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8003bb0 con 0x7f5dd40039b0 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.689+0000 7f5df6800640 1 -- 172.21.15.19:0/1205451974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd40039b0 msgr2=0x7f5dd4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.689+0000 7f5df6800640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd40039b0 0x7f5dd4003e20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5de8001e50 tx=0x7f5dd4005830 comp rx=0 tx=0).stop 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.689+0000 7f5df6800640 1 -- 172.21.15.19:0/1205451974 shutdown_connections 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.690+0000 7f5df6800640 1 --2- 172.21.15.19:0/1205451974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dd40039b0 0x7f5dd4003e20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.690+0000 7f5df6800640 1 -- 172.21.15.19:0/1205451974 >> 172.21.15.19:0/1205451974 conn(0x7f5df00b4440 msgr2=0x7f5df00b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.690+0000 7f5df6800640 1 -- 172.21.15.19:0/1205451974 shutdown_connections 2024-04-25T11:29:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.690+0000 7f5df6800640 1 -- 172.21.15.19:0/1205451974 wait complete. 2024-04-25T11:29:09.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.692+0000 7f5df6800640 1 Processor -- start 2024-04-25T11:29:09.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.692+0000 7f5df6800640 1 -- start start 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40039b0 0x7f5df0153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df0140d60 0x7f5df0143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df01420d0 0x7f5df00cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df6800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5df00b7640 con 0x7f5df0140d60 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5df00b74c0 con 0x7f5df01420d0 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5df00b5740 con 0x7f5dd40039b0 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df0140d60 0x7f5df0143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df0140d60 0x7f5df0143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37996/0 (socket says 172.21.15.19:37996) 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df4a00640 1 -- 172.21.15.19:0/1966389715 learned_addr learned my addr 172.21.15.19:0/1966389715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df5400640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40039b0 0x7f5df0153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.694+0000 7f5df5e00640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df01420d0 0x7f5df00cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df4a00640 1 -- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40039b0 msgr2=0x7f5df0153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df4a00640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40039b0 0x7f5df0153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df4a00640 1 -- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df01420d0 msgr2=0x7f5df00cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df4a00640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df01420d0 0x7f5df00cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df4a00640 1 -- 172.21.15.19:0/1966389715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df015ffe0 con 0x7f5df0140d60 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df5e00640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df01420d0 0x7f5df00cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df4a00640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df0140d60 0x7f5df0143d50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00f630 tx=0x7f5ddc00faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5ded600640 1 -- 172.21.15.19:0/1966389715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc019020 con 0x7f5df0140d60 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5ded600640 1 -- 172.21.15.19:0/1966389715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ddc014870 con 0x7f5df0140d60 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5ded600640 1 -- 172.21.15.19:0/1966389715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc014c20 con 0x7f5df0140d60 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.695+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df0142cb0 con 0x7f5df0140d60 2024-04-25T11:29:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.696+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df0143270 con 0x7f5df0140d60 2024-04-25T11:29:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.698+0000 7f5ded600640 1 -- 172.21.15.19:0/1966389715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5ddc01c020 con 0x7f5df0140d60 2024-04-25T11:29:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.699+0000 7f5ded600640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5dd00744b0 0x7f5dd00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.699+0000 7f5ded600640 1 -- 172.21.15.19:0/1966389715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f5ddc095bd0 con 0x7f5df0140d60 2024-04-25T11:29:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.700+0000 7f5df5400640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5dd00744b0 0x7f5dd00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.700+0000 7f5dfee36740 1 Processor -- start 2024-04-25T11:29:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.700+0000 7f5dfee36740 1 -- start start 2024-04-25T11:29:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5df5400640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5dd00744b0 0x7f5dd00769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5de8032000 tx=0x7f5de802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ec10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7e9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5dfee36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5569c7dfdbc0 con 0x5569c7ad6670 2024-04-25T11:29:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5569c7d99ed0 con 0x5569c7ace490 2024-04-25T11:29:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5569c7df9b80 con 0x5569c7af7350 2024-04-25T11:29:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5df5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7e9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5df5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7e9a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38000/0 (socket says 172.21.15.19:38000) 2024-04-25T11:29:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5df5e00640 1 -- 172.21.15.19:0/4264291438 learned_addr learned my addr 172.21.15.19:0/4264291438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.702+0000 7f5df5400640 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5df4a00640 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ec10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5df5e00640 1 -- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 msgr2=0x5569c7ec4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5df5e00640 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5df5e00640 1 -- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 msgr2=0x5569c7ec10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5df5e00640 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ec10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5df5e00640 1 -- 172.21.15.19:0/4264291438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c7bdb7c0 con 0x5569c7ad6670 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5df5e00640 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7e9a3c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5df00d3330 tx=0x7f5de0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4f98860a106014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5de5c00640 1 -- 172.21.15.19:0/4264291438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0019070 con 0x5569c7ad6670 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5de5c00640 1 -- 172.21.15.19:0/4264291438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de000f8a0 con 0x5569c7ad6670 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.703+0000 7f5de5c00640 1 -- 172.21.15.19:0/4264291438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0010040 con 0x5569c7ad6670 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 -- 172.21.15.19:0/4264291438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 msgr2=0x5569c7e9a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7e9a3c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5df00d3330 tx=0x7f5de0005d30 comp rx=0 tx=0).stop 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 -- 172.21.15.19:0/4264291438 shutdown_connections 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ec10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7e9a3c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 --2- 172.21.15.19:0/4264291438 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 -- 172.21.15.19:0/4264291438 >> 172.21.15.19:0/4264291438 conn(0x5569c7aeb4a0 msgr2=0x5569c7e99b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 -- 172.21.15.19:0/4264291438 shutdown_connections 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.704+0000 7f5dfee36740 1 -- 172.21.15.19:0/4264291438 wait complete. 2024-04-25T11:29:09.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.705+0000 7f5dfee36740 1 -- start start 2024-04-25T11:29:09.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.706+0000 7f5dfee36740 1 Processor -- start 2024-04-25T11:29:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.707+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ace490 0x5569c7ed62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.707+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7ec69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.707+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7af7350 0x5569c7ec6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.707+0000 7f5dfee36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5569c7d849f0 con 0x5569c7ad6670 2024-04-25T11:29:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.707+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5569c7ebfdc0 con 0x5569c7af7350 2024-04-25T11:29:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.707+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5569c7eb4070 con 0x5569c7ace490 2024-04-25T11:29:09.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.707+0000 7f5dfee36740 10 client.0 _ll_register_callbacks cb 0x5569c7d82f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:09.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7ec69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7ec69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38008/0 (socket says 172.21.15.19:38008) 2024-04-25T11:29:09.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 -- 172.21.15.19:0/3552527025 learned_addr learned my addr 172.21.15.19:0/3552527025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 -- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ace490 msgr2=0x5569c7ed62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df5400640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ace490 0x5569c7ed62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df5e00640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7af7350 0x5569c7ec6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ace490 0x5569c7ed62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 -- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7af7350 msgr2=0x5569c7ec6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7af7350 0x5569c7ec6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df4a00640 1 -- 172.21.15.19:0/3552527025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c7c3bf50 con 0x5569c7ad6670 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df5400640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ace490 0x5569c7ed62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.708+0000 7f5df5e00640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7af7350 0x5569c7ec6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5df4a00640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7ec69c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00f600 tx=0x7f5ddc05f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb83b5931a0b908 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dda000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dda000640 1 -- 172.21.15.19:0/3552527025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc05fb80 con 0x5569c7ad6670 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dda000640 1 -- 172.21.15.19:0/3552527025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f5ddc066130 con 0x5569c7ad6670 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de000f8a0 con 0x5569c7ad6670 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dda000640 1 -- 172.21.15.19:0/3552527025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc063f30 con 0x5569c7ad6670 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dfee36740 10 client.5412 fetch_fsmap learned FSMap version 23 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5569c7acf0b0 con 0x5569c7ad6670 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dda000640 1 -- 172.21.15.19:0/3552527025 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de000f8a0 con 0x5569c7ad6670 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.709+0000 7f5dda000640 1 -- 172.21.15.19:0/3552527025 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f5ddc05b4f0 con 0x5569c7ad6670 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.710+0000 7f5dfee36740 10 client.5412 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.710+0000 7f5dfee36740 10 client.5412 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.710+0000 7f5dfee36740 20 client.5412 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.710+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7ab6130 0x5569c7eb5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.710+0000 7f5dfee36740 4 client.5412 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:29:09.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.710+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569c7e0b7d0 con 0x5569c7ab6130 2024-04-25T11:29:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.710+0000 7f5df5400640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7ab6130 0x5569c7eb5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.711+0000 7f5df5400640 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7ab6130 0x5569c7eb5890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6cfecbbbe72e94 server_cookie=cd5d54351f14d976 in_seq=0 out_seq=0 2024-04-25T11:29:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.711+0000 7f5dda000640 10 client.5412 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dda000640 1 -- 172.21.15.19:0/3552527025 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569c7e0b7d0 con 0x5569c7ab6130 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dda000640 10 client.5412 handle_command_reply: tid=0 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 client.5412 shutdown 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 msgr2=0x5569c7ec69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7ec69c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00f600 tx=0x7f5ddc05f3e0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 shutdown_connections 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7ab6130 0x5569c7eb5890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7af7350 0x5569c7ec6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ad6670 0x5569c7ec69c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dda000640 0 client.5412 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3552527025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ace490 0x5569c7ed62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 >> 172.21.15.19:0/3552527025 conn(0x5569c7aeb4a0 msgr2=0x5569c7e93f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 shutdown_connections 2024-04-25T11:29:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 1 -- 172.21.15.19:0/3552527025 wait complete. 2024-04-25T11:29:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.712+0000 7f5dfee36740 20 client.5412 trim_cache size 0 max 16384 2024-04-25T11:29:09.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.728+0000 7f5dfee36740 1 Processor -- start 2024-04-25T11:29:09.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.728+0000 7f5dfee36740 1 -- start start 2024-04-25T11:29:09.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.729+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ed7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.729+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ad6670 0x5569c7e93f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.729+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5dfee36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0202250 con 0x5569c7af7350 2024-04-25T11:29:09.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5df00cd990 con 0x5569c7ace490 2024-04-25T11:29:09.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5df0201c30 con 0x5569c7ad6670 2024-04-25T11:29:09.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38014/0 (socket says 172.21.15.19:38014) 2024-04-25T11:29:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 -- 172.21.15.19:0/2142346932 learned_addr learned my addr 172.21.15.19:0/2142346932 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df4a00640 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ad6670 0x5569c7e93f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5400640 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ed7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 -- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ad6670 msgr2=0x5569c7e93f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ad6670 0x5569c7e93f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 -- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 msgr2=0x5569c7ed7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ed7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df5e00640 1 -- 172.21.15.19:0/2142346932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c7ed6640 con 0x5569c7af7350 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.730+0000 7f5df4a00640 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ad6670 0x5569c7e93f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.731+0000 7f5df5e00640 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7af7350 0x5569c7ec4760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005d70 tx=0x7f5de0007050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efd9dda7d183b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.731+0000 7f5de5200640 1 -- 172.21.15.19:0/2142346932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0019070 con 0x5569c7af7350 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.731+0000 7f5de5200640 1 -- 172.21.15.19:0/2142346932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de001a040 con 0x5569c7af7350 2024-04-25T11:29:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.731+0000 7f5de5200640 1 -- 172.21.15.19:0/2142346932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de000fc80 con 0x5569c7af7350 2024-04-25T11:29:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 -- 172.21.15.19:0/2142346932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7af7350 msgr2=0x5569c7ec4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7af7350 0x5569c7ec4760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005d70 tx=0x7f5de0007050 comp rx=0 tx=0).stop 2024-04-25T11:29:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 -- 172.21.15.19:0/2142346932 shutdown_connections 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7af7350 0x5569c7ec4760 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7ad6670 0x5569c7e93f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 --2- 172.21.15.19:0/2142346932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ace490 0x5569c7ed7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 -- 172.21.15.19:0/2142346932 >> 172.21.15.19:0/2142346932 conn(0x5569c7aeb4a0 msgr2=0x5569c7e8fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 -- 172.21.15.19:0/2142346932 shutdown_connections 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.732+0000 7f5dfee36740 1 -- 172.21.15.19:0/2142346932 wait complete. 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.733+0000 7f5dfee36740 1 -- start start 2024-04-25T11:29:09.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.733+0000 7f5dfee36740 1 Processor -- start 2024-04-25T11:29:09.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ace490 0x5569c7f59040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ad6670 0x5569c7ed4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5dfee36740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ed5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5dfee36740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5569c7ed6810 con 0x5569c7ace490 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5569c7eb1050 con 0x5569c7ad6670 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5dfee36740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5569c7eb36e0 con 0x5569c7af7350 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5dfee36740 10 client.0 _ll_register_callbacks cb 0x7f5ddc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5df5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ace490 0x5569c7f59040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5df5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ace490 0x5569c7f59040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38030/0 (socket says 172.21.15.19:38030) 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5df5400640 1 -- 172.21.15.19:0/3260231693 learned_addr learned my addr 172.21.15.19:0/3260231693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.735+0000 7f5df4a00640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ad6670 0x5569c7ed4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df5400640 1 -- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 msgr2=0x5569c7ed5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df5e00640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ed5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df5400640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ed5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df5400640 1 -- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ad6670 msgr2=0x5569c7ed4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df5400640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ad6670 0x5569c7ed4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df5400640 1 -- 172.21.15.19:0/3260231693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c7ec1980 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df4a00640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ad6670 0x5569c7ed4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5df5400640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ace490 0x5569c7f59040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5de802bfd0 tx=0x7f5de8001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52729344f67c3c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5dda000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5dda000640 1 -- 172.21.15.19:0/3260231693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8053070 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5dda000640 1 -- 172.21.15.19:0/3260231693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f5de8003e00 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.736+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de001a040 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dda000640 1 -- 172.21.15.19:0/3260231693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5de805d050 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 10 client.5418 fetch_fsmap learned FSMap version 23 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5569c7e8ffb0 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dda000640 1 -- 172.21.15.19:0/3260231693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de001a040 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dda000640 1 -- 172.21.15.19:0/3260231693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f5de8060020 con 0x5569c7ace490 2024-04-25T11:29:09.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 10 client.5418 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T11:29:09.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 10 client.5418 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:29:09.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 20 client.5418 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:09.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7f65170 0x5569c7f67590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:09.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 4 client.5418 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T11:29:09.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5569c7e05640 con 0x5569c7f65170 2024-04-25T11:29:09.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.737+0000 7f5df4a00640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7f65170 0x5569c7f67590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:09.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.738+0000 7f5df4a00640 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7f65170 0x5569c7f67590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e239aa74e745f6 server_cookie=99560d11db926608 in_seq=0 out_seq=0 2024-04-25T11:29:09.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.738+0000 7f5dda000640 10 client.5418 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:09.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.738+0000 7f5dda000640 1 -- 172.21.15.19:0/3260231693 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x5569c7e05640 con 0x5569c7f65170 2024-04-25T11:29:09.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.738+0000 7f5dda000640 10 client.5418 handle_command_reply: tid=0 2024-04-25T11:29:09.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.738+0000 7f5dfee36740 1 client.5418 shutdown 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ace490 msgr2=0x5569c7f59040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ace490 0x5569c7f59040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5de802bfd0 tx=0x7f5de8001510 comp rx=0 tx=0).stop 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 shutdown_connections 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5569c7f65170 0x5569c7f67590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5569c7ad6670 0x5569c7ed4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5569c7af7350 0x5569c7ed5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 --2- 172.21.15.19:0/3260231693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5569c7ace490 0x5569c7f59040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dda000640 0 client.5418 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 >> 172.21.15.19:0/3260231693 conn(0x5569c7aeb4a0 msgr2=0x5569c7e96060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 shutdown_connections 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 1 -- 172.21.15.19:0/3260231693 wait complete. 2024-04-25T11:29:09.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.739+0000 7f5dfee36740 20 client.5418 trim_cache size 0 max 16384 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "dentry", 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3452871827, 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627776, 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: "frag": "*", 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "file_to_be_damaged", 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_id": "head", 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:29:09.733 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:29:09.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5dd00744b0 msgr2=0x7f5dd00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5dd00744b0 0x7f5dd00769c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5de8032000 tx=0x7f5de802e000 comp rx=0 tx=0).stop 2024-04-25T11:29:09.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df0140d60 msgr2=0x7f5df0143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:09.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df0140d60 0x7f5df0143d50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00f630 tx=0x7f5ddc00faf0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 shutdown_connections 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5dd00744b0 0x7f5dd00769c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5df01420d0 0x7f5df00cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5df0140d60 0x7f5df0143d50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 --2- 172.21.15.19:0/1966389715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5dd40039b0 0x7f5df0153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 >> 172.21.15.19:0/1966389715 conn(0x7f5df00b4440 msgr2=0x7f5df006d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 shutdown_connections 2024-04-25T11:29:09.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09.740+0000 7f5df6800640 1 -- 172.21.15.19:0/1966389715 wait complete. 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> import errno 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> f.close() 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> except IOError as e: 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(errno.EIO) 2024-04-25T11:29:09.757 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:29:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:09 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-25T11:29:09.913 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-25T11:29:09.913 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T11:29:09.914 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T11:29:09.915 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:29:10.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-25T11:29:10.127 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-25T11:29:10.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:10.128 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T11:29:10.231 INFO:teuthology.orchestra.run.smithi019.stdout:2 2024-04-25T11:29:10.231 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:29:10.231 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:10.288 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:29:10.288 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:10.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:29:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.643+0000 7f0bdb200640 1 Processor -- start 2024-04-25T11:29:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.643+0000 7f0bdb200640 1 -- start start 2024-04-25T11:29:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bdb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0bd41443c0 0x7f0bd406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bdb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bd40cfcf0 0x7f0bd406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bdb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd415f5f0 con 0x7f0bd4070370 2024-04-25T11:29:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bdb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd40708d0 con 0x7f0bd40cfcf0 2024-04-25T11:29:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bdb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd415f160 con 0x7f0bd41443c0 2024-04-25T11:29:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bd9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bd40cfcf0 0x7f0bd406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bd9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bd40cfcf0 0x7f0bd406c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38040/0 (socket says 172.21.15.19:38040) 2024-04-25T11:29:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.645+0000 7f0bd9200640 1 -- 172.21.15.19:0/19226906 learned_addr learned my addr 172.21.15.19:0/19226906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.646+0000 7f0bd9200640 1 -- 172.21.15.19:0/19226906 >> v2:172.21.15.169:3300/0 conn(0x7f0bd41443c0 msgr2=0x7f0bd406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.646+0000 7f0bd9200640 1 --2- 172.21.15.19:0/19226906 >> v2:172.21.15.169:3300/0 conn(0x7f0bd41443c0 0x7f0bd406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.646+0000 7f0bd9200640 1 -- 172.21.15.19:0/19226906 >> v1:172.21.15.169:6789/0 conn(0x7f0bd4070370 legacy=0x7f0bd40cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.646+0000 7f0bd9200640 1 -- 172.21.15.19:0/19226906 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd415e830 con 0x7f0bd40cfcf0 2024-04-25T11:29:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.647+0000 7f0bd9200640 1 --2- 172.21.15.19:0/19226906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd40cfcf0 0x7f0bd406c5c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4072820 tx=0x7f0bcc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45c049cb8f9792c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.647+0000 7f0bd2a00640 1 -- 172.21.15.19:0/19226906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc037070 con 0x7f0bd40cfcf0 2024-04-25T11:29:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.647+0000 7f0bd2a00640 1 -- 172.21.15.19:0/19226906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc032440 con 0x7f0bd40cfcf0 2024-04-25T11:29:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.647+0000 7f0bd2a00640 1 -- 172.21.15.19:0/19226906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc032810 con 0x7f0bd40cfcf0 2024-04-25T11:29:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 -- 172.21.15.19:0/19226906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd40cfcf0 msgr2=0x7f0bd406c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 --2- 172.21.15.19:0/19226906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd40cfcf0 0x7f0bd406c5c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4072820 tx=0x7f0bcc02a230 comp rx=0 tx=0).stop 2024-04-25T11:29:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 -- 172.21.15.19:0/19226906 shutdown_connections 2024-04-25T11:29:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 --2- 172.21.15.19:0/19226906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd40cfcf0 0x7f0bd406c5c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 --2- 172.21.15.19:0/19226906 >> v2:172.21.15.169:3300/0 conn(0x7f0bd41443c0 0x7f0bd406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 -- 172.21.15.19:0/19226906 >> 172.21.15.19:0/19226906 conn(0x7f0bd40d1ae0 msgr2=0x7f0bd40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 -- 172.21.15.19:0/19226906 shutdown_connections 2024-04-25T11:29:10.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.648+0000 7f0bdb200640 1 -- 172.21.15.19:0/19226906 wait complete. 2024-04-25T11:29:10.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.651+0000 7f0bdb200640 1 Processor -- start 2024-04-25T11:29:10.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.651+0000 7f0bdb200640 1 -- start start 2024-04-25T11:29:10.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.652+0000 7f0bdb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd4070370 0x7f0bd41ea910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.652+0000 7f0bdb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd41443c0 0x7f0bd415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.652+0000 7f0bdb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd41424b0 0x7f0bd415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.652+0000 7f0bdb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41447d0 con 0x7f0bd4070370 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.652+0000 7f0bdb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd415edf0 con 0x7f0bd41443c0 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.652+0000 7f0bdb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd4162b00 con 0x7f0bd41424b0 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd4070370 0x7f0bd41ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd4070370 0x7f0bd41ea910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38056/0 (socket says 172.21.15.19:38056) 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 -- 172.21.15.19:0/4229064646 learned_addr learned my addr 172.21.15.19:0/4229064646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3400640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd41443c0 0x7f0bd415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd9200640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd41424b0 0x7f0bd415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 -- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd41424b0 msgr2=0x7f0bd415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd41424b0 0x7f0bd415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 -- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd41443c0 msgr2=0x7f0bd415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:10.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd41443c0 0x7f0bd415b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.653+0000 7f0bd3e00640 1 -- 172.21.15.19:0/4229064646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd412f350 con 0x7f0bd4070370 2024-04-25T11:29:10.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.654+0000 7f0bd3e00640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd4070370 0x7f0bd41ea910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400b9c0 tx=0x7f0bc400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:10.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.654+0000 7f0bcbe00640 1 -- 172.21.15.19:0/4229064646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4019070 con 0x7f0bd4070370 2024-04-25T11:29:10.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.654+0000 7f0bcbe00640 1 -- 172.21.15.19:0/4229064646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc400cba0 con 0x7f0bd4070370 2024-04-25T11:29:10.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.654+0000 7f0bcbe00640 1 -- 172.21.15.19:0/4229064646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4014440 con 0x7f0bd4070370 2024-04-25T11:29:10.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.654+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd412f630 con 0x7f0bd4070370 2024-04-25T11:29:10.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.654+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd412fbf0 con 0x7f0bd4070370 2024-04-25T11:29:10.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.655+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd4050740 con 0x7f0bd4070370 2024-04-25T11:29:10.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.657+0000 7f0bcbe00640 1 -- 172.21.15.19:0/4229064646 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0bc401c070 con 0x7f0bd4070370 2024-04-25T11:29:10.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.658+0000 7f0bcbe00640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bb00744b0 0x7f0bb00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:10.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.658+0000 7f0bcbe00640 1 -- 172.21.15.19:0/4229064646 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f0bc4095080 con 0x7f0bd4070370 2024-04-25T11:29:10.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.658+0000 7f0bd3400640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bb00744b0 0x7f0bb00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:10.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.660+0000 7f0bd3400640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bb00744b0 0x7f0bb00769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006fd0 tx=0x7f0bbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:10.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.662+0000 7f0bcbe00640 1 -- 172.21.15.19:0/4229064646 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0bc405f730 con 0x7f0bd4070370 2024-04-25T11:29:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.874+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bd406f950 con 0x7f0bd4070370 2024-04-25T11:29:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.876+0000 7f0bcbe00640 1 -- 172.21.15.19:0/4229064646 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16987 (secure 0 0 0) 0x7f0bc405f030 con 0x7f0bd4070370 2024-04-25T11:29:10.867 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:10.867 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:29:06.584671+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3831883468":"2024-04-26T11:29:00.304477+0000","172.21.15.19:6833/3831883468":"2024-04-26T11:29:00.304477+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:29:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bb00744b0 msgr2=0x7f0bb00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bb00744b0 0x7f0bb00769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006fd0 tx=0x7f0bbc008040 comp rx=0 tx=0).stop 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd4070370 msgr2=0x7f0bd41ea910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd4070370 0x7f0bd41ea910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400b9c0 tx=0x7f0bc400be80 comp rx=0 tx=0).stop 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 shutdown_connections 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bb00744b0 0x7f0bb00769c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bd41443c0 0x7f0bd415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bd41424b0 0x7f0bd415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 --2- 172.21.15.19:0/4229064646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bd4070370 0x7f0bd41ea910 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 >> 172.21.15.19:0/4229064646 conn(0x7f0bd40d1ae0 msgr2=0x7f0bd40b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 shutdown_connections 2024-04-25T11:29:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:10.879+0000 7f0bdb200640 1 -- 172.21.15.19:0/4229064646 wait complete. 2024-04-25T11:29:10.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:28:43.580410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:28:44.810835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:29:10.897 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:29:11.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.194+0000 7fabd6000640 1 Processor -- start 2024-04-25T11:29:11.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.194+0000 7fabd6000640 1 -- start start 2024-04-25T11:29:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.197+0000 7fabd6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 0x7fabd00b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.197+0000 7fabd6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fabd00d0c10 0x7fabd00b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.197+0000 7fabd6000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabd00b3710 con 0x7fabd00b3110 2024-04-25T11:29:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.197+0000 7fabd6000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fabd00b39f0 con 0x7fabd00d0c10 2024-04-25T11:29:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.197+0000 7fabd6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fabd00b3cd0 con 0x7fabd00cfc70 2024-04-25T11:29:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.197+0000 7fabcfe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fabd00b3110 0x7fabd00d0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42994/0 (socket says 172.21.15.19:42994) 2024-04-25T11:29:11.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.197+0000 7fabcfe00640 1 -- 172.21.15.19:0/2583240705 learned_addr learned my addr 172.21.15.19:0/2583240705 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:11.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabd4c00640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 0x7fabd00b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabd5600640 1 --2- 172.21.15.19:0/2583240705 >> v2:172.21.15.169:3300/0 conn(0x7fabd00d0c10 0x7fabd00b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 279033640 0 0) 0x7fabd00b3710 con 0x7fabd00b3110 2024-04-25T11:29:11.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabb8003520 con 0x7fabd00b3110 2024-04-25T11:29:11.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 32569662 0 0) 0x7fabb8003520 con 0x7fabd00b3110 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 msgr2=0x7fabd00b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 0x7fabd00b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 >> v2:172.21.15.169:3300/0 conn(0x7fabd00d0c10 msgr2=0x7fabd00b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 --2- 172.21.15.19:0/2583240705 >> v2:172.21.15.169:3300/0 conn(0x7fabd00d0c10 0x7fabd00b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd00b4040 con 0x7fabd00b3110 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.198+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fabbc001670 con 0x7fabd00b3110 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 >> v1:172.21.15.19:6789/0 conn(0x7fabd00b3110 legacy=0x7fabd00d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcf400640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb80039b0 0x7fabb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcf400640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb8004800 0x7fabb8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcf400640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 0x7fabb8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabd00b39f0 con 0x7fabb80039b0 2024-04-25T11:29:11.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fabd00b3cd0 con 0x7fabb8004800 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fabb800ba80 con 0x7fabd00cfc70 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd4c00640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb80039b0 0x7fabb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd5600640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 0x7fabb8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabcfe00640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb8004800 0x7fabb8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd4c00640 1 -- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 msgr2=0x7fabb8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd4c00640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 0x7fabb8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd4c00640 1 -- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb8004800 msgr2=0x7fabb8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd4c00640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb8004800 0x7fabb8004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd4c00640 1 -- 172.21.15.19:0/2583240705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb8003520 con 0x7fabb80039b0 2024-04-25T11:29:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.199+0000 7fabd5600640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd00cfc70 0x7fabb8005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.200+0000 7fabd4c00640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb80039b0 0x7fabb8003e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009ac0 tx=0x7fabb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea576147740d77ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.201+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4036070 con 0x7fabb80039b0 2024-04-25T11:29:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.201+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc4031400 con 0x7fabb80039b0 2024-04-25T11:29:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.201+0000 7fabcf400640 1 -- 172.21.15.19:0/2583240705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4009690 con 0x7fabb80039b0 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.201+0000 7fabd6000640 1 -- 172.21.15.19:0/2583240705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb80039b0 msgr2=0x7fabb8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.201+0000 7fabd6000640 1 --2- 172.21.15.19:0/2583240705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabb80039b0 0x7fabb8003e20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009ac0 tx=0x7fabb8005830 comp rx=0 tx=0).stop 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.201+0000 7fabd5600640 1 -- 172.21.15.19:0/2583240705 reap_dead start 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.202+0000 7fabd6000640 1 -- 172.21.15.19:0/2583240705 shutdown_connections 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.202+0000 7fabd6000640 1 -- 172.21.15.19:0/2583240705 >> 172.21.15.19:0/2583240705 conn(0x7fabd008d180 msgr2=0x7fabd008d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.202+0000 7fabd6000640 1 -- 172.21.15.19:0/2583240705 shutdown_connections 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.202+0000 7fabd6000640 1 -- 172.21.15.19:0/2583240705 wait complete. 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.204+0000 7fabd6000640 1 Processor -- start 2024-04-25T11:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.204+0000 7fabd6000640 1 -- start start 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb80039b0 0x7fabd0153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd00cab30 0x7fabd015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd01600b0 0x7fabd00b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd6000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fabd00b8f20 con 0x7fabd00cab30 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fabd00b95b0 con 0x7fabb80039b0 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fabd00b90a0 con 0x7fabd01600b0 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd00cab30 0x7fabd015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd00cab30 0x7fabd015fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38082/0 (socket says 172.21.15.19:38082) 2024-04-25T11:29:11.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabcfe00640 1 -- 172.21.15.19:0/1401869945 learned_addr learned my addr 172.21.15.19:0/1401869945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:11.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd4c00640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb80039b0 0x7fabd0153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.206+0000 7fabd5600640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd01600b0 0x7fabd00b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabcfe00640 1 -- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd01600b0 msgr2=0x7fabd00b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabcfe00640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd01600b0 0x7fabd00b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabcfe00640 1 -- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb80039b0 msgr2=0x7fabd0153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabcfe00640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb80039b0 0x7fabd0153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabcfe00640 1 -- 172.21.15.19:0/1401869945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd015ead0 con 0x7fabd00cab30 2024-04-25T11:29:11.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabcfe00640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd00cab30 0x7fabd015fb70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabd0074df0 tx=0x7fabbc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:11.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabccc00640 1 -- 172.21.15.19:0/1401869945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc019070 con 0x7fabd00cab30 2024-04-25T11:29:11.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd015edb0 con 0x7fabd00cab30 2024-04-25T11:29:11.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabccc00640 1 -- 172.21.15.19:0/1401869945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc0072d0 con 0x7fabd00cab30 2024-04-25T11:29:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabccc00640 1 -- 172.21.15.19:0/1401869945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0047e0 con 0x7fabd00cab30 2024-04-25T11:29:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.207+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd015f370 con 0x7fabd00cab30 2024-04-25T11:29:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.209+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd015e840 con 0x7fabd00cab30 2024-04-25T11:29:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.210+0000 7fabccc00640 1 -- 172.21.15.19:0/1401869945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fabbc004980 con 0x7fabd00cab30 2024-04-25T11:29:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.211+0000 7fabccc00640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fabb00744e0 0x7fabb00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.211+0000 7fabccc00640 1 -- 172.21.15.19:0/1401869945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fabbc094c30 con 0x7fabd00cab30 2024-04-25T11:29:11.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.212+0000 7fabd4c00640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fabb00744e0 0x7fabb00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.213+0000 7fabd4c00640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fabb00744e0 0x7fabb00769f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fabc4033000 tx=0x7fabc4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:11.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.216+0000 7fabccc00640 1 -- 172.21.15.19:0/1401869945 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fabbc05f1d0 con 0x7fabd00cab30 2024-04-25T11:29:11.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.424+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabd0076c60 con 0x7fabd00cab30 2024-04-25T11:29:11.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.424+0000 7fabccc00640 1 -- 172.21.15.19:0/1401869945 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7fabbc067250 con 0x7fabd00cab30 2024-04-25T11:29:11.415 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:29:11.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fabb00744e0 msgr2=0x7fabb00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fabb00744e0 0x7fabb00769f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fabc4033000 tx=0x7fabc4008030 comp rx=0 tx=0).stop 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd00cab30 msgr2=0x7fabd015fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd00cab30 0x7fabd015fb70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fabd0074df0 tx=0x7fabbc00dc00 comp rx=0 tx=0).stop 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 shutdown_connections 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fabb00744e0 0x7fabb00769f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fabd01600b0 0x7fabd00b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fabd00cab30 0x7fabd015fb70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 --2- 172.21.15.19:0/1401869945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabb80039b0 0x7fabd0153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 >> 172.21.15.19:0/1401869945 conn(0x7fabd008d180 msgr2=0x7fabd00b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.427+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 shutdown_connections 2024-04-25T11:29:11.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.428+0000 7fabd6000640 1 -- 172.21.15.19:0/1401869945 wait complete. 2024-04-25T11:29:11.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:29:11.598 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:29:11.608+0000 7fb164400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T11:29:11.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.742+0000 7fb8a6800640 1 Processor -- start 2024-04-25T11:29:11.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.742+0000 7fb8a6800640 1 -- start start 2024-04-25T11:29:11.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.744+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b5260 0x7fb8a00b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.744+0000 7fb8a6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb8a00b80a0 0x7fb8a00d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.744+0000 7fb8a6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a00b84b0 con 0x7fb8a00b4d80 2024-04-25T11:29:11.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.744+0000 7fb8a6800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a00d4420 con 0x7fb8a00b80a0 2024-04-25T11:29:11.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.744+0000 7fb8a6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a00d0b10 con 0x7fb8a00b5260 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb8a00b80a0 0x7fb8a00d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb8a00b80a0 0x7fb8a00d51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38092/0 (socket says 172.21.15.19:38092) 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb8a00b4d80 0x7fb8a00b9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:39742/0 (socket says 172.21.15.19:39742) 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b5260 0x7fb8a00b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5e00640 1 -- 172.21.15.19:0/176505783 learned_addr learned my addr 172.21.15.19:0/176505783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5e00640 1 -- 172.21.15.19:0/176505783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b5260 msgr2=0x7fb8a00b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5e00640 1 --2- 172.21.15.19:0/176505783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b5260 0x7fb8a00b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5e00640 1 -- 172.21.15.19:0/176505783 >> v1:172.21.15.169:6789/0 conn(0x7fb8a00b4d80 legacy=0x7fb8a00b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.745+0000 7fb8a5e00640 1 -- 172.21.15.19:0/176505783 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00d46c0 con 0x7fb8a00b80a0 2024-04-25T11:29:11.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.747+0000 7fb8a5e00640 1 --2- 172.21.15.19:0/176505783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b80a0 0x7fb8a00d51a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb898009870 tx=0x7fb89802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dded3ba28c1b1a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:11.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.747+0000 7fb89fe00640 1 -- 172.21.15.19:0/176505783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898037070 con 0x7fb8a00b80a0 2024-04-25T11:29:11.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.747+0000 7fb89fe00640 1 -- 172.21.15.19:0/176505783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb898032440 con 0x7fb8a00b80a0 2024-04-25T11:29:11.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.747+0000 7fb89fe00640 1 -- 172.21.15.19:0/176505783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898032810 con 0x7fb8a00b80a0 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 -- 172.21.15.19:0/176505783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b80a0 msgr2=0x7fb8a00d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 --2- 172.21.15.19:0/176505783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b80a0 0x7fb8a00d51a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb898009870 tx=0x7fb89802a270 comp rx=0 tx=0).stop 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 -- 172.21.15.19:0/176505783 shutdown_connections 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 --2- 172.21.15.19:0/176505783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b80a0 0x7fb8a00d51a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 --2- 172.21.15.19:0/176505783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b5260 0x7fb8a00b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 -- 172.21.15.19:0/176505783 >> 172.21.15.19:0/176505783 conn(0x7fb8a00d2bd0 msgr2=0x7fb8a00d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 -- 172.21.15.19:0/176505783 shutdown_connections 2024-04-25T11:29:11.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.748+0000 7fb8a6800640 1 -- 172.21.15.19:0/176505783 wait complete. 2024-04-25T11:29:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.751+0000 7fb8a6800640 1 Processor -- start 2024-04-25T11:29:11.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.751+0000 7fb8a6800640 1 -- start start 2024-04-25T11:29:11.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.752+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b4d80 0x7fb8a00cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.752+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b5260 0x7fb8a00cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.752+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00b80a0 0x7fb8a00cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.752+0000 7fb8a6800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a00d0770 con 0x7fb8a00b5260 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.752+0000 7fb8a6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a00d4c80 con 0x7fb8a00b80a0 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.752+0000 7fb8a6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a00d08f0 con 0x7fb8a00b4d80 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b5260 0x7fb8a00cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b5260 0x7fb8a00cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38102/0 (socket says 172.21.15.19:38102) 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 -- 172.21.15.19:0/547554672 learned_addr learned my addr 172.21.15.19:0/547554672 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a5400640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b4d80 0x7fb8a00cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a5e00640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00b80a0 0x7fb8a00cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 -- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b4d80 msgr2=0x7fb8a00cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b4d80 0x7fb8a00cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 -- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00b80a0 msgr2=0x7fb8a00cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00b80a0 0x7fb8a00cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a4a00640 1 -- 172.21.15.19:0/547554672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00cf9a0 con 0x7fb8a00b5260 2024-04-25T11:29:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.753+0000 7fb8a5400640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b4d80 0x7fb8a00cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.754+0000 7fb8a4a00640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b5260 0x7fb8a00cef20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e8a0 tx=0x7fb88c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.754+0000 7fb89d600640 1 -- 172.21.15.19:0/547554672 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c017070 con 0x7fb8a00b5260 2024-04-25T11:29:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.754+0000 7fb89d600640 1 -- 172.21.15.19:0/547554672 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88c00c940 con 0x7fb8a00b5260 2024-04-25T11:29:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.754+0000 7fb89d600640 1 -- 172.21.15.19:0/547554672 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00cd30 con 0x7fb8a00b5260 2024-04-25T11:29:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.754+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a00cbd20 con 0x7fb8a00b5260 2024-04-25T11:29:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.754+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a00ce160 con 0x7fb8a00b5260 2024-04-25T11:29:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.755+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a0076c60 con 0x7fb8a00b5260 2024-04-25T11:29:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.757+0000 7fb89d600640 1 -- 172.21.15.19:0/547554672 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb88c01a070 con 0x7fb8a00b5260 2024-04-25T11:29:11.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.758+0000 7fb89d600640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8800744b0 0x7fb8800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:11.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.758+0000 7fb89d600640 1 -- 172.21.15.19:0/547554672 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fb88c0952b0 con 0x7fb8a00b5260 2024-04-25T11:29:11.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.759+0000 7fb8a5400640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8800744b0 0x7fb8800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:11.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.760+0000 7fb8a5400640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8800744b0 0x7fb8800769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb894007fd0 tx=0x7fb894009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.763+0000 7fb89d600640 1 -- 172.21.15.19:0/547554672 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb88c05f960 con 0x7fb8a00b5260 2024-04-25T11:29:11.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.970+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8a00b70b0 con 0x7fb8a00b5260 2024-04-25T11:29:11.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.971+0000 7fb89d600640 1 -- 172.21.15.19:0/547554672 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7fb88c05f260 con 0x7fb8a00b5260 2024-04-25T11:29:11.961 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:29:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8800744b0 msgr2=0x7fb8800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8800744b0 0x7fb8800769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb894007fd0 tx=0x7fb894009040 comp rx=0 tx=0).stop 2024-04-25T11:29:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b5260 msgr2=0x7fb8a00cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:11.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b5260 0x7fb8a00cef20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e8a0 tx=0x7fb88c00ed60 comp rx=0 tx=0).stop 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 shutdown_connections 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8800744b0 0x7fb8800769c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8a00b80a0 0x7fb8a00cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8a00b5260 0x7fb8a00cef20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.973+0000 7fb8a6800640 1 --2- 172.21.15.19:0/547554672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8a00b4d80 0x7fb8a00cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.974+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 >> 172.21.15.19:0/547554672 conn(0x7fb8a00d2bd0 msgr2=0x7fb8a00b90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.974+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 shutdown_connections 2024-04-25T11:29:11.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:11.974+0000 7fb8a6800640 1 -- 172.21.15.19:0/547554672 wait complete. 2024-04-25T11:29:11.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:29:12.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.296+0000 7f241cc00640 1 Processor -- start 2024-04-25T11:29:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.296+0000 7f241cc00640 1 -- start start 2024-04-25T11:29:12.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.298+0000 7f241cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f24180b0a10 0x7f241815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.298+0000 7f241cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241806d5d0 0x7f241806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.298+0000 7f241cc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241806df20 con 0x7f24181615e0 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.298+0000 7f241cc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f24180d0480 con 0x7f24180b0a10 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.298+0000 7f241cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f24180d0740 con 0x7f241806d5d0 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2415e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f24180b0a10 0x7f241815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2415e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f24180b0a10 0x7f241815cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38118/0 (socket says 172.21.15.19:38118) 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2415e00640 1 -- 172.21.15.19:0/899983152 learned_addr learned my addr 172.21.15.19:0/899983152 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2416800640 1 --2- 172.21.15.19:0/899983152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241806d5d0 0x7f241806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2415e00640 1 -- 172.21.15.19:0/899983152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241806d5d0 msgr2=0x7f241806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2415e00640 1 --2- 172.21.15.19:0/899983152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241806d5d0 0x7f241806d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2415e00640 1 -- 172.21.15.19:0/899983152 >> v1:172.21.15.169:6789/0 conn(0x7f24181615e0 legacy=0x7f24180b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.299+0000 7f2415e00640 1 -- 172.21.15.19:0/899983152 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24180ce6b0 con 0x7f24180b0a10 2024-04-25T11:29:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.300+0000 7f2415e00640 1 --2- 172.21.15.19:0/899983152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24180b0a10 0x7f241815cff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f24180725a0 tx=0x7f240802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f952ee3a81e49e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.301+0000 7f2414a00640 1 -- 172.21.15.19:0/899983152 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408037070 con 0x7f24180b0a10 2024-04-25T11:29:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.301+0000 7f2414a00640 1 -- 172.21.15.19:0/899983152 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240802ae50 con 0x7f24180b0a10 2024-04-25T11:29:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.301+0000 7f2414a00640 1 -- 172.21.15.19:0/899983152 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408032660 con 0x7f24180b0a10 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 -- 172.21.15.19:0/899983152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24180b0a10 msgr2=0x7f241815cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 --2- 172.21.15.19:0/899983152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24180b0a10 0x7f241815cff0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f24180725a0 tx=0x7f240802a140 comp rx=0 tx=0).stop 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 -- 172.21.15.19:0/899983152 shutdown_connections 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 --2- 172.21.15.19:0/899983152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f241806d5d0 0x7f241806d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 --2- 172.21.15.19:0/899983152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24180b0a10 0x7f241815cff0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 -- 172.21.15.19:0/899983152 >> 172.21.15.19:0/899983152 conn(0x7f24180730b0 msgr2=0x7f24180734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 -- 172.21.15.19:0/899983152 shutdown_connections 2024-04-25T11:29:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.302+0000 7f241cc00640 1 -- 172.21.15.19:0/899983152 wait complete. 2024-04-25T11:29:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.304+0000 7f241cc00640 1 Processor -- start 2024-04-25T11:29:12.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.304+0000 7f241cc00640 1 -- start start 2024-04-25T11:29:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f241cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241806d5d0 0x7f24181551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f241cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24180b0a10 0x7f24180aa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f241cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24181615e0 0x7f24180ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f241cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24180cef50 con 0x7f24181615e0 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f241cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f24180cec70 con 0x7f241806d5d0 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f241cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f24181619f0 con 0x7f24180b0a10 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f2416800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24181615e0 0x7f24180ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f2416800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24181615e0 0x7f24180ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38132/0 (socket says 172.21.15.19:38132) 2024-04-25T11:29:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.306+0000 7f2416800640 1 -- 172.21.15.19:0/7365079 learned_addr learned my addr 172.21.15.19:0/7365079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:12.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2415400640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24180b0a10 0x7f24180aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2415e00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241806d5d0 0x7f24181551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2416800640 1 -- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24180b0a10 msgr2=0x7f24180aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2416800640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24180b0a10 0x7f24180aa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2416800640 1 -- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241806d5d0 msgr2=0x7f24181551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2416800640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241806d5d0 0x7f24181551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2416800640 1 -- 172.21.15.19:0/7365079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24180aaf60 con 0x7f24181615e0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f2416800640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24181615e0 0x7f24180ad710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2410009fd0 tx=0x7f241000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f240e000640 1 -- 172.21.15.19:0/7365079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410014420 con 0x7f24181615e0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f240e000640 1 -- 172.21.15.19:0/7365079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2410019070 con 0x7f24181615e0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f240e000640 1 -- 172.21.15.19:0/7365079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410014800 con 0x7f24181615e0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.307+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24180ab240 con 0x7f24181615e0 2024-04-25T11:29:12.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.308+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24180ab800 con 0x7f24181615e0 2024-04-25T11:29:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.309+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2418050740 con 0x7f24181615e0 2024-04-25T11:29:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.310+0000 7f240e000640 1 -- 172.21.15.19:0/7365079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f24100149a0 con 0x7f24181615e0 2024-04-25T11:29:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.311+0000 7f240e000640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23f80744b0 0x7f23f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.311+0000 7f240e000640 1 -- 172.21.15.19:0/7365079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f2410095bc0 con 0x7f24181615e0 2024-04-25T11:29:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.312+0000 7f2415e00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23f80744b0 0x7f23f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.314+0000 7f2415e00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23f80744b0 0x7f23f80769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2408034000 tx=0x7f240802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:12.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.317+0000 7f240e000640 1 -- 172.21.15.19:0/7365079 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2410060270 con 0x7f24181615e0 2024-04-25T11:29:12.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.520+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24180aacf0 con 0x7f24181615e0 2024-04-25T11:29:12.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.521+0000 7f240e000640 1 -- 172.21.15.19:0/7365079 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f241005fb70 con 0x7f24181615e0 2024-04-25T11:29:12.511 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:29:12.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23f80744b0 msgr2=0x7f23f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23f80744b0 0x7f23f80769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2408034000 tx=0x7f240802e000 comp rx=0 tx=0).stop 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24181615e0 msgr2=0x7f24180ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24181615e0 0x7f24180ad710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2410009fd0 tx=0x7f241000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 shutdown_connections 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23f80744b0 0x7f23f80769c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24181615e0 0x7f24180ad710 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24180b0a10 0x7f24180aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 --2- 172.21.15.19:0/7365079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f241806d5d0 0x7f24181551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 >> 172.21.15.19:0/7365079 conn(0x7f24180730b0 msgr2=0x7f2418158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 shutdown_connections 2024-04-25T11:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.524+0000 7f241cc00640 1 -- 172.21.15.19:0/7365079 wait complete. 2024-04-25T11:29:12.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:29:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.833+0000 7fb33f200640 1 Processor -- start 2024-04-25T11:29:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.834+0000 7fb33f200640 1 -- start start 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.836+0000 7fb33f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3400b4ea0 0x7fb3400cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.836+0000 7fb33f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb340144370 0x7fb34015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.836+0000 7fb33f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3400d1230 con 0x7fb34006c5c0 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.836+0000 7fb33f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb3400d06b0 con 0x7fb340144370 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.836+0000 7fb33f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb3400d0990 con 0x7fb3400b4ea0 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.836+0000 7fb33e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb340144370 0x7fb34015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.836+0000 7fb33de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb34006c5c0 0x7fb3400b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43004/0 (socket says 172.21.15.19:43004) 2024-04-25T11:29:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.837+0000 7fb33de00640 1 -- 172.21.15.19:0/2576529683 learned_addr learned my addr 172.21.15.19:0/2576529683 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.837+0000 7fb33d400640 1 --2- 172.21.15.19:0/2576529683 >> v2:172.21.15.169:3300/0 conn(0x7fb3400b4ea0 0x7fb3400cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.837+0000 7fb33e800640 1 -- 172.21.15.19:0/2576529683 >> v2:172.21.15.169:3300/0 conn(0x7fb3400b4ea0 msgr2=0x7fb3400cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.837+0000 7fb33e800640 1 --2- 172.21.15.19:0/2576529683 >> v2:172.21.15.169:3300/0 conn(0x7fb3400b4ea0 0x7fb3400cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.837+0000 7fb33e800640 1 -- 172.21.15.19:0/2576529683 >> v1:172.21.15.19:6789/0 conn(0x7fb34006c5c0 legacy=0x7fb3400b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.837+0000 7fb33e800640 1 -- 172.21.15.19:0/2576529683 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3400d0c70 con 0x7fb340144370 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.838+0000 7fb33e800640 1 --2- 172.21.15.19:0/2576529683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb34015e770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb3400b5550 tx=0x7fb33802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e69be6a0701fb8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.839+0000 7fb33ca00640 1 -- 172.21.15.19:0/2576529683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338007740 con 0x7fb340144370 2024-04-25T11:29:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.839+0000 7fb33ca00640 1 -- 172.21.15.19:0/2576529683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3380078e0 con 0x7fb340144370 2024-04-25T11:29:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.839+0000 7fb33ca00640 1 -- 172.21.15.19:0/2576529683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338007cb0 con 0x7fb340144370 2024-04-25T11:29:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.839+0000 7fb33f200640 1 -- 172.21.15.19:0/2576529683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 msgr2=0x7fb34015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.839+0000 7fb33f200640 1 --2- 172.21.15.19:0/2576529683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb34015e770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb3400b5550 tx=0x7fb33802a680 comp rx=0 tx=0).stop 2024-04-25T11:29:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.840+0000 7fb33f200640 1 -- 172.21.15.19:0/2576529683 shutdown_connections 2024-04-25T11:29:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.840+0000 7fb33f200640 1 --2- 172.21.15.19:0/2576529683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb34015e770 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.840+0000 7fb33f200640 1 --2- 172.21.15.19:0/2576529683 >> v2:172.21.15.169:3300/0 conn(0x7fb3400b4ea0 0x7fb3400cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.840+0000 7fb33f200640 1 -- 172.21.15.19:0/2576529683 >> 172.21.15.19:0/2576529683 conn(0x7fb3400d1720 msgr2=0x7fb3400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.840+0000 7fb33f200640 1 -- 172.21.15.19:0/2576529683 shutdown_connections 2024-04-25T11:29:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.840+0000 7fb33f200640 1 -- 172.21.15.19:0/2576529683 wait complete. 2024-04-25T11:29:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.842+0000 7fb33f200640 1 Processor -- start 2024-04-25T11:29:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.842+0000 7fb33f200640 1 -- start start 2024-04-25T11:29:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb34006c5c0 0x7fb3401f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb3400b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3401f4bc0 0x7fb3400b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3400d0450 con 0x7fb340144370 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3400d02d0 con 0x7fb34006c5c0 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb340147760 con 0x7fb3401f4bc0 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb3400b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb3400b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38154/0 (socket says 172.21.15.19:38154) 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33d400640 1 -- 172.21.15.19:0/4153667163 learned_addr learned my addr 172.21.15.19:0/4153667163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33de00640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb34006c5c0 0x7fb3401f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33e800640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3401f4bc0 0x7fb3400b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33d400640 1 -- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3401f4bc0 msgr2=0x7fb3400b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33d400640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3401f4bc0 0x7fb3400b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33d400640 1 -- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb34006c5c0 msgr2=0x7fb3401f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.844+0000 7fb33d400640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb34006c5c0 0x7fb3401f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb33d400640 1 -- 172.21.15.19:0/4153667163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3400b1120 con 0x7fb340144370 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb33de00640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb34006c5c0 0x7fb3401f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb33d400640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb3400b0be0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00d620 tx=0x7fb32c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb336000640 1 -- 172.21.15.19:0/4153667163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00b620 con 0x7fb340144370 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb336000640 1 -- 172.21.15.19:0/4153667163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32c019070 con 0x7fb340144370 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3401f6330 con 0x7fb340144370 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb336000640 1 -- 172.21.15.19:0/4153667163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00ba00 con 0x7fb340144370 2024-04-25T11:29:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.845+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3401f68f0 con 0x7fb340144370 2024-04-25T11:29:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.846+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34015f910 con 0x7fb340144370 2024-04-25T11:29:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.848+0000 7fb336000640 1 -- 172.21.15.19:0/4153667163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb32c00bba0 con 0x7fb340144370 2024-04-25T11:29:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.849+0000 7fb336000640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3200744e0 0x7fb3200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.849+0000 7fb336000640 1 -- 172.21.15.19:0/4153667163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fb32c095660 con 0x7fb340144370 2024-04-25T11:29:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.850+0000 7fb33de00640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3200744e0 0x7fb3200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:12.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.852+0000 7fb33de00640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3200744e0 0x7fb3200769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb330000f60 tx=0x7fb330009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:12.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:12.854+0000 7fb336000640 1 -- 172.21.15.19:0/4153667163 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb32c05fda0 con 0x7fb340144370 2024-04-25T11:29:13.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.071+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb34015cc30 con 0x7fb340144370 2024-04-25T11:29:13.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.072+0000 7fb336000640 1 -- 172.21.15.19:0/4153667163 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v38) ==== 80+0+122 (secure 0 0 0) 0x7fb32c05f6a0 con 0x7fb340144370 2024-04-25T11:29:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3831883468 2024-04-26T11:29:00.304477+0000 2024-04-25T11:29:13.063 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6833/3831883468 2024-04-26T11:29:00.304477+0000 2024-04-25T11:29:13.063 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-04-25T11:29:13.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3200744e0 msgr2=0x7fb3200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3200744e0 0x7fb3200769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb330000f60 tx=0x7fb330009040 comp rx=0 tx=0).stop 2024-04-25T11:29:13.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 msgr2=0x7fb3400b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb3400b0be0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00d620 tx=0x7fb32c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 shutdown_connections 2024-04-25T11:29:13.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3200744e0 0x7fb3200769f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3401f4bc0 0x7fb3400b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb340144370 0x7fb3400b0be0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 --2- 172.21.15.19:0/4153667163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb34006c5c0 0x7fb3401f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 >> 172.21.15.19:0/4153667163 conn(0x7fb3400d1720 msgr2=0x7fb3400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.075+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 shutdown_connections 2024-04-25T11:29:13.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.076+0000 7fb33f200640 1 -- 172.21.15.19:0/4153667163 wait complete. 2024-04-25T11:29:13.091 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:29:13.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:13.091 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:13.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:29:13.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:13.177 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:29:13.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:13.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2024-04-25T11:29:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.521+0000 7fb578800640 1 Processor -- start 2024-04-25T11:29:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.521+0000 7fb578800640 1 -- start start 2024-04-25T11:29:13.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.523+0000 7fb578800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5700cd9a0 0x7fb57015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.523+0000 7fb578800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5700b0af0 0x7fb57006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.523+0000 7fb578800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb57006db10 0x7fb570161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.523+0000 7fb578800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb5700ceab0 con 0x7fb57006db10 2024-04-25T11:29:13.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.523+0000 7fb578800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb57006df20 con 0x7fb5700b0af0 2024-04-25T11:29:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.523+0000 7fb578800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5700cdea0 con 0x7fb5700cd9a0 2024-04-25T11:29:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.524+0000 7fb576800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb57006db10 0x7fb570161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.524+0000 7fb576800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb57006db10 0x7fb570161ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38162/0 (socket says 172.21.15.19:38162) 2024-04-25T11:29:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.524+0000 7fb576800640 1 -- 172.21.15.19:0/1792656082 learned_addr learned my addr 172.21.15.19:0/1792656082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.524+0000 7fb575e00640 1 --2- 172.21.15.19:0/1792656082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5700cd9a0 0x7fb57015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.524+0000 7fb575400640 1 --2- 172.21.15.19:0/1792656082 >> v2:172.21.15.169:3300/0 conn(0x7fb5700b0af0 0x7fb57006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.525+0000 7fb576800640 1 -- 172.21.15.19:0/1792656082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5700cd9a0 msgr2=0x7fb57015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.525+0000 7fb576800640 1 --2- 172.21.15.19:0/1792656082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5700cd9a0 0x7fb57015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.525+0000 7fb576800640 1 -- 172.21.15.19:0/1792656082 >> v2:172.21.15.169:3300/0 conn(0x7fb5700b0af0 msgr2=0x7fb57006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.525+0000 7fb576800640 1 --2- 172.21.15.19:0/1792656082 >> v2:172.21.15.169:3300/0 conn(0x7fb5700b0af0 0x7fb57006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.525+0000 7fb576800640 1 -- 172.21.15.19:0/1792656082 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5700ce180 con 0x7fb57006db10 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.525+0000 7fb575400640 1 --2- 172.21.15.19:0/1792656082 >> v2:172.21.15.169:3300/0 conn(0x7fb5700b0af0 0x7fb57006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:13.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.526+0000 7fb576800640 1 --2- 172.21.15.19:0/1792656082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57006db10 0x7fb570161ed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5700b4530 tx=0x7fb56c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85a73bd93633e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.526+0000 7fb574a00640 1 -- 172.21.15.19:0/1792656082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c036070 con 0x7fb57006db10 2024-04-25T11:29:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.526+0000 7fb574a00640 1 -- 172.21.15.19:0/1792656082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c031600 con 0x7fb57006db10 2024-04-25T11:29:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.526+0000 7fb574a00640 1 -- 172.21.15.19:0/1792656082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0319d0 con 0x7fb57006db10 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 -- 172.21.15.19:0/1792656082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57006db10 msgr2=0x7fb570161ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 --2- 172.21.15.19:0/1792656082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57006db10 0x7fb570161ed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5700b4530 tx=0x7fb56c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 -- 172.21.15.19:0/1792656082 shutdown_connections 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 --2- 172.21.15.19:0/1792656082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57006db10 0x7fb570161ed0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 --2- 172.21.15.19:0/1792656082 >> v2:172.21.15.169:3300/0 conn(0x7fb5700b0af0 0x7fb57006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 --2- 172.21.15.19:0/1792656082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5700cd9a0 0x7fb57015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 -- 172.21.15.19:0/1792656082 >> 172.21.15.19:0/1792656082 conn(0x7fb570073090 msgr2=0x7fb5700734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 -- 172.21.15.19:0/1792656082 shutdown_connections 2024-04-25T11:29:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.527+0000 7fb578800640 1 -- 172.21.15.19:0/1792656082 wait complete. 2024-04-25T11:29:13.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.530+0000 7fb578800640 1 Processor -- start 2024-04-25T11:29:13.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.530+0000 7fb578800640 1 -- start start 2024-04-25T11:29:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.531+0000 7fb578800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb57006db10 0x7fb570146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.531+0000 7fb578800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5700b0af0 0x7fb5700aa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.531+0000 7fb578800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5700cd9a0 0x7fb5700aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb578800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5700b0f00 con 0x7fb5700b0af0 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb578800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5700ce740 con 0x7fb5700cd9a0 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb578800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5700cef40 con 0x7fb57006db10 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5700b0af0 0x7fb5700aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5700b0af0 0x7fb5700aa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38164/0 (socket says 172.21.15.19:38164) 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575400640 1 -- 172.21.15.19:0/1816817115 learned_addr learned my addr 172.21.15.19:0/1816817115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575e00640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb57006db10 0x7fb570146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb576800640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5700cd9a0 0x7fb5700aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575400640 1 -- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb57006db10 msgr2=0x7fb570146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575400640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb57006db10 0x7fb570146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575400640 1 -- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5700cd9a0 msgr2=0x7fb5700aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.532+0000 7fb575400640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5700cd9a0 0x7fb5700aa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.533+0000 7fb575400640 1 -- 172.21.15.19:0/1816817115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5700ab810 con 0x7fb5700b0af0 2024-04-25T11:29:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.533+0000 7fb575400640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5700b0af0 0x7fb5700aa230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c820 tx=0x7fb56000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.533+0000 7fb56a000640 1 -- 172.21.15.19:0/1816817115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560015070 con 0x7fb5700b0af0 2024-04-25T11:29:13.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.533+0000 7fb56a000640 1 -- 172.21.15.19:0/1816817115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb560007a30 con 0x7fb5700b0af0 2024-04-25T11:29:13.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.533+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5700abd10 con 0x7fb5700b0af0 2024-04-25T11:29:13.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.533+0000 7fb56a000640 1 -- 172.21.15.19:0/1816817115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560007e00 con 0x7fb5700b0af0 2024-04-25T11:29:13.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.533+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701f6cc0 con 0x7fb5700b0af0 2024-04-25T11:29:13.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.536+0000 7fb56a000640 1 -- 172.21.15.19:0/1816817115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb560018070 con 0x7fb5700b0af0 2024-04-25T11:29:13.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.538+0000 7fb56a000640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5540744b0 0x7fb5540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.538+0000 7fb56a000640 1 -- 172.21.15.19:0/1816817115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fb560095180 con 0x7fb5700b0af0 2024-04-25T11:29:13.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.538+0000 7fb57f974740 1 Processor -- start 2024-04-25T11:29:13.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.538+0000 7fb57f974740 1 -- start start 2024-04-25T11:29:13.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.538+0000 7fb575e00640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5540744b0 0x7fb5540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.539+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189569c5b0 0x561895a68b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.539+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a83850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.539+0000 7fb57f974740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189567b8d0 0x561895a83d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb57f974740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5618959a2db0 con 0x56189567b8d0 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56189593f000 con 0x56189569c5b0 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56189599ed70 con 0x5618956736f0 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb575e00640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5540744b0 0x7fb5540769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb564006fd0 tx=0x7fb564008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189567b8d0 0x561895a83d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189567b8d0 0x561895a83d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38170/0 (socket says 172.21.15.19:38170) 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 -- 172.21.15.19:0/2943363529 learned_addr learned my addr 172.21.15.19:0/2943363529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb575400640 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a83850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb575e00640 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189569c5b0 0x561895a68b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 -- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 msgr2=0x561895a83850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a83850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 -- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189569c5b0 msgr2=0x561895a68b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189569c5b0 0x561895a68b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.540+0000 7fb576800640 1 -- 172.21.15.19:0/2943363529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561895780a20 con 0x56189567b8d0 2024-04-25T11:29:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.541+0000 7fb576800640 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189567b8d0 0x561895a83d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb56c02ac90 tx=0x7fb56c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd8e1761011e69c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.541+0000 7fb55e600640 1 -- 172.21.15.19:0/2943363529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c036040 con 0x56189567b8d0 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.541+0000 7fb55e600640 1 -- 172.21.15.19:0/2943363529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c03c640 con 0x56189567b8d0 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.541+0000 7fb55e600640 1 -- 172.21.15.19:0/2943363529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c03ca30 con 0x56189567b8d0 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 -- 172.21.15.19:0/2943363529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189567b8d0 msgr2=0x561895a83d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189567b8d0 0x561895a83d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb56c02ac90 tx=0x7fb56c02e000 comp rx=0 tx=0).stop 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 -- 172.21.15.19:0/2943363529 shutdown_connections 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189567b8d0 0x561895a83d90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a83850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 --2- 172.21.15.19:0/2943363529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189569c5b0 0x561895a68b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 -- 172.21.15.19:0/2943363529 >> 172.21.15.19:0/2943363529 conn(0x561895690700 msgr2=0x561895a3e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 -- 172.21.15.19:0/2943363529 shutdown_connections 2024-04-25T11:29:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.542+0000 7fb57f974740 1 -- 172.21.15.19:0/2943363529 wait complete. 2024-04-25T11:29:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.543+0000 7fb57f974740 1 -- start start 2024-04-25T11:29:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.543+0000 7fb57f974740 1 Processor -- start 2024-04-25T11:29:13.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a82bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189567b8d0 0x561895a7e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb57f974740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a7e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb57f974740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561895929c50 con 0x56189569c5b0 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561895a7f7e0 con 0x56189567b8d0 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561895a834f0 con 0x5618956736f0 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb57f974740 10 client.0 _ll_register_callbacks cb 0x561895999fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a7e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a7e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38182/0 (socket says 172.21.15.19:38182) 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.545+0000 7fb576800640 1 -- 172.21.15.19:0/3532038811 learned_addr learned my addr 172.21.15.19:0/3532038811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb575e00640 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a82bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb576800640 1 -- 172.21.15.19:0/3532038811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 msgr2=0x561895a82bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb576800640 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a82bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb576800640 1 -- 172.21.15.19:0/3532038811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189567b8d0 msgr2=0x561895a7e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb576800640 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189567b8d0 0x561895a7e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb576800640 1 -- 172.21.15.19:0/3532038811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561895674310 con 0x56189569c5b0 2024-04-25T11:29:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb576800640 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a7e9e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009f10 tx=0x7fb56c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc0db69f60a7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb552a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb552a00640 1 -- 172.21.15.19:0/3532038811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c036070 con 0x56189569c5b0 2024-04-25T11:29:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb552a00640 1 -- 172.21.15.19:0/3532038811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fb56c03ca00 con 0x56189569c5b0 2024-04-25T11:29:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.546+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb56c03c640 con 0x56189569c5b0 2024-04-25T11:29:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb552a00640 1 -- 172.21.15.19:0/3532038811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb56c052050 con 0x56189569c5b0 2024-04-25T11:29:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 10 client.5463 fetch_fsmap learned FSMap version 24 2024-04-25T11:29:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5618957e11b0 con 0x56189569c5b0 2024-04-25T11:29:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb552a00640 1 -- 172.21.15.19:0/3532038811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c03c640 con 0x56189569c5b0 2024-04-25T11:29:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb552a00640 1 -- 172.21.15.19:0/3532038811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7fb56c055020 con 0x56189569c5b0 2024-04-25T11:29:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 10 client.5463 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:29:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 10 client.5463 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 20 client.5463 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56189565b180 0x561895a86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 4 client.5463 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.547+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618959b09c0 con 0x56189565b180 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.548+0000 7fb575e00640 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56189565b180 0x561895a86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.548+0000 7fb575e00640 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56189565b180 0x561895a86800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453bfd59b5ffa4e server_cookie=b2d89f388c1c801 in_seq=0 out_seq=0 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.548+0000 7fb552a00640 10 client.5463 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.549+0000 7fb552a00640 1 -- 172.21.15.19:0/3532038811 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5618959b09c0 con 0x56189565b180 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.549+0000 7fb552a00640 10 client.5463 handle_command_reply: tid=0 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.549+0000 7fb57f974740 1 client.5463 shutdown 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.549+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 msgr2=0x561895a7e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.549+0000 7fb57f974740 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a7e9e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009f10 tx=0x7fb56c02c000 comp rx=0 tx=0).stop 2024-04-25T11:29:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 shutdown_connections 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x56189565b180 0x561895a86800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56189567b8d0 0x561895a7e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a7e9e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 --2- 172.21.15.19:0/3532038811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5618956736f0 0x561895a82bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb552a00640 0 client.5463 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 >> 172.21.15.19:0/3532038811 conn(0x561895690700 msgr2=0x561895a68c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 shutdown_connections 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 1 -- 172.21.15.19:0/3532038811 wait complete. 2024-04-25T11:29:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.550+0000 7fb57f974740 20 client.5463 trim_cache size 0 max 16384 2024-04-25T11:29:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.570+0000 7fb57f974740 1 Processor -- start 2024-04-25T11:29:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.571+0000 7fb57f974740 1 -- start start 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 0x561895a64b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 0x561895a3e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb57f974740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb57f974740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb570203160 con 0x56189569c5b0 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701f9dc0 con 0x5618956736f0 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5700af170 con 0x56189567b8d0 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a86800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38184/0 (socket says 172.21.15.19:38184) 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.572+0000 7fb576800640 1 -- 172.21.15.19:0/3225047595 learned_addr learned my addr 172.21.15.19:0/3225047595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb576800640 1 -- 172.21.15.19:0/3225047595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 msgr2=0x561895a3e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb576800640 1 --2- 172.21.15.19:0/3225047595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 0x561895a3e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb576800640 1 -- 172.21.15.19:0/3225047595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 msgr2=0x561895a64b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb576800640 1 --2- 172.21.15.19:0/3225047595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 0x561895a64b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb576800640 1 -- 172.21.15.19:0/3225047595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56189564d490 con 0x56189569c5b0 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb576800640 1 --2- 172.21.15.19:0/3225047595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a86800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb56c002230 tx=0x7fb56c03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77450d1ecfd8ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb55dc00640 1 -- 172.21.15.19:0/3225047595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c036070 con 0x56189569c5b0 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb55dc00640 1 -- 172.21.15.19:0/3225047595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c04a070 con 0x56189569c5b0 2024-04-25T11:29:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.573+0000 7fb55dc00640 1 -- 172.21.15.19:0/3225047595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c004000 con 0x56189569c5b0 2024-04-25T11:29:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 -- 172.21.15.19:0/3225047595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 msgr2=0x561895a86800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 --2- 172.21.15.19:0/3225047595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a86800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb56c002230 tx=0x7fb56c03b000 comp rx=0 tx=0).stop 2024-04-25T11:29:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 -- 172.21.15.19:0/3225047595 shutdown_connections 2024-04-25T11:29:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 --2- 172.21.15.19:0/3225047595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a86800 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 --2- 172.21.15.19:0/3225047595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 0x561895a3e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 --2- 172.21.15.19:0/3225047595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 0x561895a64b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 -- 172.21.15.19:0/3225047595 >> 172.21.15.19:0/3225047595 conn(0x561895690700 msgr2=0x561895a68c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 -- 172.21.15.19:0/3225047595 shutdown_connections 2024-04-25T11:29:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.574+0000 7fb57f974740 1 -- 172.21.15.19:0/3225047595 wait complete. 2024-04-25T11:29:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.575+0000 7fb57f974740 1 -- start start 2024-04-25T11:29:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.576+0000 7fb57f974740 1 Processor -- start 2024-04-25T11:29:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 0x561895a3de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb57f974740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 0x561895a3e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb57f974740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a82010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb57f974740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5700af7c0 con 0x56189569c5b0 2024-04-25T11:29:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561895ae71e0 con 0x5618956736f0 2024-04-25T11:29:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb57f974740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561895a3f6c0 con 0x56189567b8d0 2024-04-25T11:29:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb57f974740 10 client.0 _ll_register_callbacks cb 0x7fb56c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a82010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb576800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a82010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38194/0 (socket says 172.21.15.19:38194) 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.577+0000 7fb576800640 1 -- 172.21.15.19:0/1385934184 learned_addr learned my addr 172.21.15.19:0/1385934184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb575e00640 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 0x561895a3de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb575400640 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 0x561895a3e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb576800640 1 -- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 msgr2=0x561895a3e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb576800640 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 0x561895a3e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb576800640 1 -- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 msgr2=0x561895a3de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb576800640 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 0x561895a3de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb576800640 1 -- 172.21.15.19:0/1385934184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561895a80920 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb576800640 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a82010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb56c03e010 tx=0x7fb56c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5163cc8818841165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb552a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.578+0000 7fb552a00640 1 -- 172.21.15.19:0/1385934184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c036040 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb552a00640 1 -- 172.21.15.19:0/1385934184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fb56c05fb80 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb56c04a070 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb552a00640 1 -- 172.21.15.19:0/1385934184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb56c05fd80 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 10 client.5469 fetch_fsmap learned FSMap version 24 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x561895ae6fe0 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb552a00640 1 -- 172.21.15.19:0/1385934184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb56c04a070 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb552a00640 1 -- 172.21.15.19:0/1385934184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7fb56c055040 con 0x56189569c5b0 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 10 client.5469 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 10 client.5469 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 20 client.5469 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x561895ae6900 0x561895a82550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 4 client.5469 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2024-04-25T11:29:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.579+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) -- 0x5618959aa830 con 0x561895ae6900 2024-04-25T11:29:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.580+0000 7fb575e00640 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x561895ae6900 0x561895a82550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.580+0000 7fb575e00640 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x561895ae6900 0x561895a82550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152069d9b1897e57 server_cookie=f8281176bc88c215 in_seq=0 out_seq=0 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.580+0000 7fb552a00640 10 client.5469 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb552a00640 1 -- 172.21.15.19:0/1385934184 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618959aa830 con 0x561895ae6900 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb552a00640 10 client.5469 handle_command_reply: tid=0 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 client.5469 shutdown 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 msgr2=0x561895a82010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a82010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb56c03e010 tx=0x7fb56c004ab0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 shutdown_connections 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x561895ae6900 0x561895a82550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56189569c5b0 0x561895a82010 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56189567b8d0 0x561895a3e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 --2- 172.21.15.19:0/1385934184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5618956736f0 0x561895a3de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb552a00640 0 client.5469 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.581+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 >> 172.21.15.19:0/1385934184 conn(0x561895690700 msgr2=0x561895a64c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.582+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 shutdown_connections 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.582+0000 7fb57f974740 1 -- 172.21.15.19:0/1385934184 wait complete. 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.582+0000 7fb57f974740 20 client.5469 trim_cache size 0 max 16384 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "fd45f08b-70d3-4795-9bf4-8fd6ff16f574", 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-04-25T11:29:13.575 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:29:13.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.582+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5540744b0 msgr2=0x7fb5540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.582+0000 7fb578800640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5540744b0 0x7fb5540769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb564006fd0 tx=0x7fb564008040 comp rx=0 tx=0).stop 2024-04-25T11:29:13.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.582+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5700b0af0 msgr2=0x7fb5700aa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.582+0000 7fb578800640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5700b0af0 0x7fb5700aa230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c820 tx=0x7fb56000cce0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 shutdown_connections 2024-04-25T11:29:13.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5540744b0 0x7fb5540769c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5700cd9a0 0x7fb5700aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5700b0af0 0x7fb5700aa230 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 --2- 172.21.15.19:0/1816817115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb57006db10 0x7fb570146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 >> 172.21.15.19:0/1816817115 conn(0x7fb570073090 msgr2=0x7fb57006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 shutdown_connections 2024-04-25T11:29:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.583+0000 7fb578800640 1 -- 172.21.15.19:0/1816817115 wait complete. 2024-04-25T11:29:13.599 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "fd45f08b-70d3-4795-9bf4-8fd6ff16f574", "mode": "asynchronous" } 2024-04-25T11:29:13.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:13.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T11:29:13.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.897+0000 7ff313e00640 1 Processor -- start 2024-04-25T11:29:13.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.897+0000 7ff313e00640 1 -- start start 2024-04-25T11:29:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.899+0000 7ff313e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff314070970 0x7ff314167dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.899+0000 7ff313e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31415e340 con 0x7ff31413ff60 2024-04-25T11:29:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.899+0000 7ff313e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3140d02a0 con 0x7ff3140d1340 2024-04-25T11:29:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.899+0000 7ff313e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff3140d0580 con 0x7ff314070970 2024-04-25T11:29:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.899+0000 7ff312a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff31413ff60 0x7ff31415dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33732/0 (socket says 172.21.15.19:33732) 2024-04-25T11:29:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.899+0000 7ff312a00640 1 -- 172.21.15.19:0/728191490 learned_addr learned my addr 172.21.15.19:0/728191490 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.899+0000 7ff313400640 1 --2- 172.21.15.19:0/728191490 >> v2:172.21.15.169:3300/0 conn(0x7ff314070970 0x7ff314167dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.900+0000 7ff311600640 1 -- 172.21.15.19:0/728191490 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2405019364 0 0) 0x7ff31415e340 con 0x7ff31413ff60 2024-04-25T11:29:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.900+0000 7ff311600640 1 -- 172.21.15.19:0/728191490 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2f8003520 con 0x7ff31413ff60 2024-04-25T11:29:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.900+0000 7ff313400640 1 -- 172.21.15.19:0/728191490 >> v1:172.21.15.169:6789/0 conn(0x7ff3140d1340 legacy=0x7ff314070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.900+0000 7ff313400640 1 -- 172.21.15.19:0/728191490 >> v1:172.21.15.19:6789/0 conn(0x7ff31413ff60 legacy=0x7ff31415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.900+0000 7ff313400640 1 -- 172.21.15.19:0/728191490 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3140d0860 con 0x7ff314070970 2024-04-25T11:29:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.901+0000 7ff313400640 1 --2- 172.21.15.19:0/728191490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff314070970 0x7ff314167dc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d81a6ad1fc6bfa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.902+0000 7ff311600640 1 -- 172.21.15.19:0/728191490 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c007740 con 0x7ff314070970 2024-04-25T11:29:13.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.902+0000 7ff311600640 1 -- 172.21.15.19:0/728191490 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c0078e0 con 0x7ff314070970 2024-04-25T11:29:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.902+0000 7ff311600640 1 -- 172.21.15.19:0/728191490 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c007cb0 con 0x7ff314070970 2024-04-25T11:29:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.903+0000 7ff313e00640 1 -- 172.21.15.19:0/728191490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff314070970 msgr2=0x7ff314167dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.903+0000 7ff313e00640 1 --2- 172.21.15.19:0/728191490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff314070970 0x7ff314167dc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c02a660 comp rx=0 tx=0).stop 2024-04-25T11:29:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.903+0000 7ff313e00640 1 -- 172.21.15.19:0/728191490 shutdown_connections 2024-04-25T11:29:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.903+0000 7ff313e00640 1 --2- 172.21.15.19:0/728191490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff314070970 0x7ff314167dc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.903+0000 7ff313e00640 1 -- 172.21.15.19:0/728191490 >> 172.21.15.19:0/728191490 conn(0x7ff3140b4260 msgr2=0x7ff314072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.903+0000 7ff313e00640 1 -- 172.21.15.19:0/728191490 shutdown_connections 2024-04-25T11:29:13.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.903+0000 7ff313e00640 1 -- 172.21.15.19:0/728191490 wait complete. 2024-04-25T11:29:13.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.906+0000 7ff313e00640 1 Processor -- start 2024-04-25T11:29:13.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.906+0000 7ff313e00640 1 -- start start 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.907+0000 7ff313e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31413ff60 0x7ff3141fa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.907+0000 7ff313e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3141f5b20 0x7ff31406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.907+0000 7ff313e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3141f5f30 0x7ff314070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.907+0000 7ff313e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff31415d600 con 0x7ff31413ff60 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.907+0000 7ff313e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3140d0e20 con 0x7ff3141f5b20 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.907+0000 7ff313e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff31415d780 con 0x7ff3141f5f30 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31413ff60 0x7ff3141fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31413ff60 0x7ff3141fa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58646/0 (socket says 172.21.15.19:58646) 2024-04-25T11:29:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 -- 172.21.15.19:0/229091148 learned_addr learned my addr 172.21.15.19:0/229091148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312000640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3141f5b20 0x7ff31406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 -- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3141f5f30 msgr2=0x7ff314070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3141f5f30 0x7ff314070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 -- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3141f5b20 msgr2=0x7ff31406fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3141f5b20 0x7ff31406fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.908+0000 7ff312a00640 1 -- 172.21.15.19:0/229091148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff314157120 con 0x7ff31413ff60 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.909+0000 7ff312a00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31413ff60 0x7ff3141fa260 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff304001eb0 tx=0x7ff30400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.909+0000 7ff30aa00640 1 -- 172.21.15.19:0/229091148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304017070 con 0x7ff31413ff60 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.909+0000 7ff30aa00640 1 -- 172.21.15.19:0/229091148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304005d00 con 0x7ff31413ff60 2024-04-25T11:29:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.909+0000 7ff30aa00640 1 -- 172.21.15.19:0/229091148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304007310 con 0x7ff31413ff60 2024-04-25T11:29:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.909+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3140b5b20 con 0x7ff31413ff60 2024-04-25T11:29:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.909+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31414b750 con 0x7ff31413ff60 2024-04-25T11:29:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.912+0000 7ff30aa00640 1 -- 172.21.15.19:0/229091148 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff30401a070 con 0x7ff31413ff60 2024-04-25T11:29:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.913+0000 7ff30aa00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2f40744e0 0x7ff2f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.914+0000 7ff30aa00640 1 -- 172.21.15.19:0/229091148 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7ff304094630 con 0x7ff31413ff60 2024-04-25T11:29:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.914+0000 7ff320a64740 1 Processor -- start 2024-04-25T11:29:13.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.914+0000 7ff320a64740 1 -- start start 2024-04-25T11:29:13.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.914+0000 7ff312000640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2f40744e0 0x7ff2f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bad05390 0x5614bb0d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff320a64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bacdc4d0 0x5614bb0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2f40744e0 0x7ff2f40769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff300009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff320a64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5614bafb8030 con 0x5614bacdc4d0 2024-04-25T11:29:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5614bafd37f0 con 0x5614bace46b0 2024-04-25T11:29:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5614baf8e7e0 con 0x5614bad05390 2024-04-25T11:29:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bacdc4d0 0x5614bb0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bacdc4d0 0x5614bb0cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58654/0 (socket says 172.21.15.19:58654) 2024-04-25T11:29:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 -- 172.21.15.19:0/4200115652 learned_addr learned my addr 172.21.15.19:0/4200115652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff313400640 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312a00640 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bad05390 0x5614bb0d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 -- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bad05390 msgr2=0x5614bb0d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bad05390 0x5614bb0d27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 -- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 msgr2=0x5614bb0a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.916+0000 7ff312000640 1 -- 172.21.15.19:0/4200115652 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614baf9a050 con 0x5614bacdc4d0 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.917+0000 7ff313400640 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.917+0000 7ff312000640 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bacdc4d0 0x5614bb0cf170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3000052d0 tx=0x7ff300011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ee11acbddcfd35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.917+0000 7ff2ff400640 1 -- 172.21.15.19:0/4200115652 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300021450 con 0x5614bacdc4d0 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.917+0000 7ff2ff400640 1 -- 172.21.15.19:0/4200115652 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3000215f0 con 0x5614bacdc4d0 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.917+0000 7ff2ff400640 1 -- 172.21.15.19:0/4200115652 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000219a0 con 0x5614bacdc4d0 2024-04-25T11:29:13.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 -- 172.21.15.19:0/4200115652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bacdc4d0 msgr2=0x5614bb0cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bacdc4d0 0x5614bb0cf170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3000052d0 tx=0x7ff300011000 comp rx=0 tx=0).stop 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 -- 172.21.15.19:0/4200115652 shutdown_connections 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bacdc4d0 0x5614bb0cf170 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 --2- 172.21.15.19:0/4200115652 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bad05390 0x5614bb0d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 -- 172.21.15.19:0/4200115652 >> 172.21.15.19:0/4200115652 conn(0x5614bacf94e0 msgr2=0x5614bb0a7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 -- 172.21.15.19:0/4200115652 shutdown_connections 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.918+0000 7ff320a64740 1 -- 172.21.15.19:0/4200115652 wait complete. 2024-04-25T11:29:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.919+0000 7ff320a64740 1 -- start start 2024-04-25T11:29:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.920+0000 7ff320a64740 1 Processor -- start 2024-04-25T11:29:13.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.921+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.921+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.921+0000 7ff320a64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.921+0000 7ff320a64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5614bb010200 con 0x5614bad05390 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.921+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5614bb00aee0 con 0x5614bace46b0 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.921+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5614bb017e70 con 0x5614bacdc4d0 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.921+0000 7ff320a64740 10 client.0 _ll_register_callbacks cb 0x5614bb001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0d1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58658/0 (socket says 172.21.15.19:58658) 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 -- 172.21.15.19:0/374099194 learned_addr learned my addr 172.21.15.19:0/374099194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff312000640 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff312a00640 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 -- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 msgr2=0x5614bb0d5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0d5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 -- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 msgr2=0x5614bb0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.922+0000 7ff313400640 1 -- 172.21.15.19:0/374099194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614bade77d0 con 0x5614bad05390 2024-04-25T11:29:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff313400640 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0d1f40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009eb0 tx=0x7ff30c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d72e8c0b06dcaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff2f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff2f3e00640 1 -- 172.21.15.19:0/374099194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c040070 con 0x5614bad05390 2024-04-25T11:29:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff2f3e00640 1 -- 172.21.15.19:0/374099194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7ff30c045660 con 0x5614bad05390 2024-04-25T11:29:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3000215f0 con 0x5614bad05390 2024-04-25T11:29:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff2f3e00640 1 -- 172.21.15.19:0/374099194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff30c04d050 con 0x5614bad05390 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff320a64740 10 client.5481 fetch_fsmap learned FSMap version 24 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.923+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5614bad1d3f0 con 0x5614bad05390 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff2f3e00640 1 -- 172.21.15.19:0/374099194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3000215f0 con 0x5614bad05390 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff2f3e00640 1 -- 172.21.15.19:0/374099194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7ff30c050020 con 0x5614bad05390 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff320a64740 10 client.5481 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff320a64740 10 client.5481 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff320a64740 20 client.5481 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff320a64740 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bacc3f60 0x5614bb0fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff320a64740 4 client.5481 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614bb0197e0 con 0x5614bacc3f60 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.924+0000 7ff312a00640 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bacc3f60 0x5614bb0fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.925+0000 7ff312a00640 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bacc3f60 0x5614bb0fcdc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe58a396e183326 server_cookie=97e30d1b975943ca in_seq=0 out_seq=0 2024-04-25T11:29:13.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.925+0000 7ff2f3e00640 10 client.5481 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.925+0000 7ff2f3e00640 1 -- 172.21.15.19:0/374099194 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614bb0197e0 con 0x5614bacc3f60 2024-04-25T11:29:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.925+0000 7ff2f3e00640 10 client.5481 handle_command_reply: tid=0 2024-04-25T11:29:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 client.5481 shutdown 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 msgr2=0x5614bb0d1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0d1f40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009eb0 tx=0x7ff30c004ab0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 shutdown_connections 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bacc3f60 0x5614bb0fcdc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0d1f40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 --2- 172.21.15.19:0/374099194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0d5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff2f3e00640 0 client.5481 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 >> 172.21.15.19:0/374099194 conn(0x5614bacf94e0 msgr2=0x5614bb0f0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 shutdown_connections 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 1 -- 172.21.15.19:0/374099194 wait complete. 2024-04-25T11:29:13.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.926+0000 7ff320a64740 20 client.5481 trim_cache size 0 max 16384 2024-04-25T11:29:13.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.946+0000 7ff320a64740 1 Processor -- start 2024-04-25T11:29:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.946+0000 7ff320a64740 1 -- start start 2024-04-25T11:29:13.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff320a64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff320a64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff314202e40 con 0x5614bad05390 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff314201f10 con 0x5614bace46b0 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff3140b6f10 con 0x5614bacdc4d0 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff313400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff313400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58664/0 (socket says 172.21.15.19:58664) 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff313400640 1 -- 172.21.15.19:0/509481116 learned_addr learned my addr 172.21.15.19:0/509481116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff312000640 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.948+0000 7ff312a00640 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff313400640 1 -- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 msgr2=0x5614bb0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff313400640 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff313400640 1 -- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 msgr2=0x5614bb0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff313400640 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff313400640 1 -- 172.21.15.19:0/509481116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614bacdd0f0 con 0x5614bad05390 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff313400640 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0cf620 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff30c02ac50 tx=0x7ff30c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca48190d5dbfc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff2fea00640 1 -- 172.21.15.19:0/509481116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c040070 con 0x5614bad05390 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff2fea00640 1 -- 172.21.15.19:0/509481116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c045c00 con 0x5614bad05390 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.949+0000 7ff2fea00640 1 -- 172.21.15.19:0/509481116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003990 con 0x5614bad05390 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 -- 172.21.15.19:0/509481116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 msgr2=0x5614bb0cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0cf620 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff30c02ac50 tx=0x7ff30c03d000 comp rx=0 tx=0).stop 2024-04-25T11:29:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 -- 172.21.15.19:0/509481116 shutdown_connections 2024-04-25T11:29:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bace46b0 0x5614bb0cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0cf620 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 --2- 172.21.15.19:0/509481116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bacdc4d0 0x5614bb0ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 -- 172.21.15.19:0/509481116 >> 172.21.15.19:0/509481116 conn(0x5614bacf94e0 msgr2=0x5614bb0fb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 -- 172.21.15.19:0/509481116 shutdown_connections 2024-04-25T11:29:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.950+0000 7ff320a64740 1 -- 172.21.15.19:0/509481116 wait complete. 2024-04-25T11:29:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.951+0000 7ff320a64740 1 -- start start 2024-04-25T11:29:13.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.952+0000 7ff320a64740 1 Processor -- start 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.953+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bacdc4d0 0x5614bb168df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.953+0000 7ff320a64740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bace46b0 0x5614bb0e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.953+0000 7ff320a64740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.953+0000 7ff320a64740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5614bb018fd0 con 0x5614bad05390 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.953+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3140b7560 con 0x5614bacdc4d0 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.953+0000 7ff320a64740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5614bb0d52a0 con 0x5614bace46b0 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.953+0000 7ff320a64740 10 client.0 _ll_register_callbacks cb 0x7ff2f80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0e3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58666/0 (socket says 172.21.15.19:58666) 2024-04-25T11:29:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 -- 172.21.15.19:0/2210702324 learned_addr learned my addr 172.21.15.19:0/2210702324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff312a00640 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bacdc4d0 0x5614bb168df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 -- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bace46b0 msgr2=0x5614bb0e2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bace46b0 0x5614bb0e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 -- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bacdc4d0 msgr2=0x5614bb168df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bacdc4d0 0x5614bb168df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 -- 172.21.15.19:0/2210702324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614bafd3a30 con 0x5614bad05390 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff312a00640 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bacdc4d0 0x5614bb168df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.954+0000 7ff313400640 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0e3340 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d10 tx=0x7ff30c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24d662de5ac7b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff2f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:13.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff2f3e00640 1 -- 172.21.15.19:0/2210702324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c040070 con 0x5614bad05390 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff2f3e00640 1 -- 172.21.15.19:0/2210702324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7ff30c003990 con 0x5614bad05390 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30c045c00 con 0x5614bad05390 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff2f3e00640 1 -- 172.21.15.19:0/2210702324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff30c003b90 con 0x5614bad05390 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff320a64740 10 client.5487 fetch_fsmap learned FSMap version 24 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x5614bb10f5b0 con 0x5614bad05390 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.955+0000 7ff2f3e00640 1 -- 172.21.15.19:0/2210702324 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c045c00 con 0x5614bad05390 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff2f3e00640 1 -- 172.21.15.19:0/2210702324 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7ff30c050040 con 0x5614bad05390 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff320a64740 10 client.5487 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff320a64740 10 client.5487 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff320a64740 20 client.5487 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff320a64740 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bb0a8910 0x5614bb0d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff320a64740 4 client.5487 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:29:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5614bb0eb140 con 0x5614bb0a8910 2024-04-25T11:29:13.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.956+0000 7ff312a00640 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bb0a8910 0x5614bb0d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:13.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff2f3e00640 10 client.5487 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff312a00640 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bb0a8910 0x5614bb0d4080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56985172f9eeab1 server_cookie=c03430d04413e1c4 in_seq=0 out_seq=0 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff2f3e00640 1 -- 172.21.15.19:0/2210702324 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x5614bb0eb140 con 0x5614bb0a8910 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff2f3e00640 10 client.5487 handle_command_reply: tid=0 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 client.5487 shutdown 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 msgr2=0x5614bb0e3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0e3340 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d10 tx=0x7ff30c02f000 comp rx=0 tx=0).stop 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 shutdown_connections 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5614bb0a8910 0x5614bb0d4080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5614bad05390 0x5614bb0e3340 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5614bace46b0 0x5614bb0e2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 --2- 172.21.15.19:0/2210702324 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5614bacdc4d0 0x5614bb168df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff2f3e00640 0 client.5487 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 >> 172.21.15.19:0/2210702324 conn(0x5614bacf94e0 msgr2=0x5614bb0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 shutdown_connections 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.957+0000 7ff320a64740 1 -- 172.21.15.19:0/2210702324 wait complete. 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff320a64740 20 client.5487 trim_cache size 0 max 16384 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-04-25T11:29:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "fd45f08b-70d3-4795-9bf4-8fd6ff16f574": { 2024-04-25T11:29:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/subdir", 2024-04-25T11:29:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "fd45f08b-70d3-4795-9bf4-8fd6ff16f574", 2024-04-25T11:29:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "repair" 2024-04-25T11:29:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:29:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:29:13.952 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2f40744e0 msgr2=0x7ff2f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2f40744e0 0x7ff2f40769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff300009040 comp rx=0 tx=0).stop 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31413ff60 msgr2=0x7ff3141fa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31413ff60 0x7ff3141fa260 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff304001eb0 tx=0x7ff30400c820 comp rx=0 tx=0).stop 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 shutdown_connections 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2f40744e0 0x7ff2f40769f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3141f5b20 0x7ff31406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3141f5f30 0x7ff314070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 --2- 172.21.15.19:0/229091148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff31413ff60 0x7ff3141fa260 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 >> 172.21.15.19:0/229091148 conn(0x7ff3140d1340 msgr2=0x7ff31413d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 shutdown_connections 2024-04-25T11:29:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:13.958+0000 7ff313e00640 1 -- 172.21.15.19:0/229091148 wait complete. 2024-04-25T11:29:13.974 INFO:tasks.cephfs.filesystem:scrub status for tag:fd45f08b-70d3-4795-9bf4-8fd6ff16f574 - {'path': '/subdir', 'tag': 'fd45f08b-70d3-4795-9bf4-8fd6ff16f574', 'options': 'repair'} 2024-04-25T11:29:33.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:29:33.118 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:29:43.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:43.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T11:29:44.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.278+0000 7fdf28c00640 1 Processor -- start 2024-04-25T11:29:44.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.278+0000 7fdf28c00640 1 -- start start 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.280+0000 7fdf28c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdf240cfc70 0x7fdf240b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.280+0000 7fdf28c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240b3110 0x7fdf240d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.280+0000 7fdf28c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf240d0a40 0x7fdf240ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.280+0000 7fdf28c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdf240d0e50 con 0x7fdf240d0a40 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.280+0000 7fdf28c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdf240b3710 con 0x7fdf240cfc70 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.280+0000 7fdf28c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdf240b39f0 con 0x7fdf240b3110 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf240d0a40 0x7fdf240ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf240d0a40 0x7fdf240ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38884/0 (socket says 172.21.15.19:38884) 2024-04-25T11:29:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 -- 172.21.15.19:0/2994849935 learned_addr learned my addr 172.21.15.19:0/2994849935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf22a00640 1 --2- 172.21.15.19:0/2994849935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240b3110 0x7fdf240d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23400640 1 --2- 172.21.15.19:0/2994849935 >> v2:172.21.15.169:3300/0 conn(0x7fdf240cfc70 0x7fdf240b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 -- 172.21.15.19:0/2994849935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240b3110 msgr2=0x7fdf240d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 --2- 172.21.15.19:0/2994849935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240b3110 0x7fdf240d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 -- 172.21.15.19:0/2994849935 >> v2:172.21.15.169:3300/0 conn(0x7fdf240cfc70 msgr2=0x7fdf240b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 --2- 172.21.15.19:0/2994849935 >> v2:172.21.15.169:3300/0 conn(0x7fdf240cfc70 0x7fdf240b8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.281+0000 7fdf23e00640 1 -- 172.21.15.19:0/2994849935 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf240b3cd0 con 0x7fdf240d0a40 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.282+0000 7fdf23e00640 1 --2- 172.21.15.19:0/2994849935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240d0a40 0x7fdf240ce490 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf24074df0 tx=0x7fdf1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c392cde2a7d94e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.283+0000 7fdf22000640 1 -- 172.21.15.19:0/2994849935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c037070 con 0x7fdf240d0a40 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.283+0000 7fdf22000640 1 -- 172.21.15.19:0/2994849935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c032440 con 0x7fdf240d0a40 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.283+0000 7fdf22000640 1 -- 172.21.15.19:0/2994849935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c032810 con 0x7fdf240d0a40 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.283+0000 7fdf28c00640 1 -- 172.21.15.19:0/2994849935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240d0a40 msgr2=0x7fdf240ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.283+0000 7fdf28c00640 1 --2- 172.21.15.19:0/2994849935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240d0a40 0x7fdf240ce490 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf24074df0 tx=0x7fdf1c02a270 comp rx=0 tx=0).stop 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.284+0000 7fdf28c00640 1 -- 172.21.15.19:0/2994849935 shutdown_connections 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.284+0000 7fdf28c00640 1 --2- 172.21.15.19:0/2994849935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240d0a40 0x7fdf240ce490 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.284+0000 7fdf28c00640 1 --2- 172.21.15.19:0/2994849935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240b3110 0x7fdf240d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.284+0000 7fdf28c00640 1 --2- 172.21.15.19:0/2994849935 >> v2:172.21.15.169:3300/0 conn(0x7fdf240cfc70 0x7fdf240b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.284+0000 7fdf28c00640 1 -- 172.21.15.19:0/2994849935 >> 172.21.15.19:0/2994849935 conn(0x7fdf2408d180 msgr2=0x7fdf2408d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.284+0000 7fdf28c00640 1 -- 172.21.15.19:0/2994849935 shutdown_connections 2024-04-25T11:29:44.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.284+0000 7fdf28c00640 1 -- 172.21.15.19:0/2994849935 wait complete. 2024-04-25T11:29:44.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.286+0000 7fdf28c00640 1 Processor -- start 2024-04-25T11:29:44.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.286+0000 7fdf28c00640 1 -- start start 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf28c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240b3110 0x7fdf240c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf28c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240cfc70 0x7fdf24154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf28c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf240d0a40 0x7fdf240b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf28c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf240b95b0 con 0x7fdf240b3110 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf28c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf240cf7e0 con 0x7fdf240d0a40 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf28c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdf240ce9d0 con 0x7fdf240cfc70 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240b3110 0x7fdf240c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240b3110 0x7fdf240c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38892/0 (socket says 172.21.15.19:38892) 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf23400640 1 -- 172.21.15.19:0/1706462133 learned_addr learned my addr 172.21.15.19:0/1706462133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.288+0000 7fdf22a00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240cfc70 0x7fdf24154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf23e00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf240d0a40 0x7fdf240b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf23400640 1 -- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240cfc70 msgr2=0x7fdf24154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf23400640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240cfc70 0x7fdf24154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf23400640 1 -- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf240d0a40 msgr2=0x7fdf240b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf23400640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf240d0a40 0x7fdf240b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf23400640 1 -- 172.21.15.19:0/1706462133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf240c9780 con 0x7fdf240b3110 2024-04-25T11:29:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf23400640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240b3110 0x7fdf240c8e80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400cd00 tx=0x7fdf1400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf1b400640 1 -- 172.21.15.19:0/1706462133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14003990 con 0x7fdf240b3110 2024-04-25T11:29:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf1b400640 1 -- 172.21.15.19:0/1706462133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf14003b30 con 0x7fdf240b3110 2024-04-25T11:29:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf1b400640 1 -- 172.21.15.19:0/1706462133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004020 con 0x7fdf240b3110 2024-04-25T11:29:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.289+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf240c9a60 con 0x7fdf240b3110 2024-04-25T11:29:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.290+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2415ea80 con 0x7fdf240b3110 2024-04-25T11:29:44.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.292+0000 7fdf1b400640 1 -- 172.21.15.19:0/1706462133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf140041c0 con 0x7fdf240b3110 2024-04-25T11:29:44.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.293+0000 7fdf1b400640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf040744b0 0x7fdf040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.293+0000 7fdf1b400640 1 -- 172.21.15.19:0/1706462133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fdf140957d0 con 0x7fdf240b3110 2024-04-25T11:29:44.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.294+0000 7fdf22a00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf040744b0 0x7fdf040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.294+0000 7fdf3123e740 1 Processor -- start 2024-04-25T11:29:44.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.294+0000 7fdf3123e740 1 -- start start 2024-04-25T11:29:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.295+0000 7fdf22a00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf040744b0 0x7fdf040769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d921e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d924b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9224430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf3123e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590d9134030 con 0x5590d8e606b0 2024-04-25T11:29:44.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5590d914f7f0 con 0x5590d8e584d0 2024-04-25T11:29:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5590d910a7e0 con 0x5590d8e81390 2024-04-25T11:29:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9224430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9224430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38900/0 (socket says 172.21.15.19:38900) 2024-04-25T11:29:44.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.296+0000 7fdf23e00640 1 -- 172.21.15.19:0/4213896314 learned_addr learned my addr 172.21.15.19:0/4213896314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf23400640 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d921e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf22a00640 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d924b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf23e00640 1 -- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 msgr2=0x5590d921e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf23e00640 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d921e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf23e00640 1 -- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 msgr2=0x5590d924b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf23e00640 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d924b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf23e00640 1 -- 172.21.15.19:0/4213896314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d9116050 con 0x5590d8e606b0 2024-04-25T11:29:44.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf22a00640 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d924b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.297+0000 7fdf23e00640 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9224430 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02a860 tx=0x7fdf1c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e1d0a2970ce16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.298+0000 7fdf0f400640 1 -- 172.21.15.19:0/4213896314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c041070 con 0x5590d8e606b0 2024-04-25T11:29:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.298+0000 7fdf0f400640 1 -- 172.21.15.19:0/4213896314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c032800 con 0x5590d8e606b0 2024-04-25T11:29:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.298+0000 7fdf0f400640 1 -- 172.21.15.19:0/4213896314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c037040 con 0x5590d8e606b0 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 -- 172.21.15.19:0/4213896314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 msgr2=0x5590d9224430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9224430 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02a860 tx=0x7fdf1c02c000 comp rx=0 tx=0).stop 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 -- 172.21.15.19:0/4213896314 shutdown_connections 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d924b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9224430 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 --2- 172.21.15.19:0/4213896314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d921e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 -- 172.21.15.19:0/4213896314 >> 172.21.15.19:0/4213896314 conn(0x5590d8e754e0 msgr2=0x5590d9223bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 -- 172.21.15.19:0/4213896314 shutdown_connections 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.299+0000 7fdf3123e740 1 -- 172.21.15.19:0/4213896314 wait complete. 2024-04-25T11:29:44.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.300+0000 7fdf3123e740 1 -- start start 2024-04-25T11:29:44.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.300+0000 7fdf3123e740 1 Processor -- start 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d92684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9240d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d924ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf3123e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590d918c200 con 0x5590d8e606b0 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5590d9186ee0 con 0x5590d8e584d0 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5590d9193e70 con 0x5590d8e81390 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf3123e740 10 client.0 _ll_register_callbacks cb 0x5590d917d270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf22a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9240d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf22a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9240d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38908/0 (socket says 172.21.15.19:38908) 2024-04-25T11:29:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf22a00640 1 -- 172.21.15.19:0/3800223440 learned_addr learned my addr 172.21.15.19:0/3800223440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf23e00640 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d924ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.302+0000 7fdf23400640 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d92684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf22a00640 1 -- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 msgr2=0x5590d924ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf22a00640 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d924ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf22a00640 1 -- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 msgr2=0x5590d92684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf22a00640 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d92684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf22a00640 1 -- 172.21.15.19:0/3800223440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d8f637d0 con 0x5590d8e606b0 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf22a00640 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9240d70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000bfd0 tx=0x7fdf10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb554544c91fc00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf03e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf03e00640 1 -- 172.21.15.19:0/3800223440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10015820 con 0x5590d8e606b0 2024-04-25T11:29:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf03e00640 1 -- 172.21.15.19:0/3800223440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fdf100159c0 con 0x5590d8e606b0 2024-04-25T11:29:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.303+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c032800 con 0x5590d8e606b0 2024-04-25T11:29:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf03e00640 1 -- 172.21.15.19:0/3800223440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf1002e050 con 0x5590d8e606b0 2024-04-25T11:29:44.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 10 client.5499 fetch_fsmap learned FSMap version 25 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x5590d8e993f0 con 0x5590d8e606b0 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf03e00640 1 -- 172.21.15.19:0/3800223440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c032800 con 0x5590d8e606b0 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf03e00640 1 -- 172.21.15.19:0/3800223440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fdf10031020 con 0x5590d8e606b0 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 10 client.5499 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 10 client.5499 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 20 client.5499 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d8e3ff60 0x5590d9267350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 4 client.5499 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590d91957e0 con 0x5590d8e3ff60 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.304+0000 7fdf23400640 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d8e3ff60 0x5590d9267350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.305+0000 7fdf03e00640 10 client.5499 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:44.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.305+0000 7fdf23400640 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d8e3ff60 0x5590d9267350 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e983c307ed8b9b server_cookie=bc94709ef88cb996 in_seq=0 out_seq=0 2024-04-25T11:29:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf03e00640 1 -- 172.21.15.19:0/3800223440 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590d91957e0 con 0x5590d8e3ff60 2024-04-25T11:29:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf03e00640 10 client.5499 handle_command_reply: tid=0 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 client.5499 shutdown 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 msgr2=0x5590d9240d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9240d70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000bfd0 tx=0x7fdf10004730 comp rx=0 tx=0).stop 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 shutdown_connections 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d8e3ff60 0x5590d9267350 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d924ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e606b0 0x5590d9240d70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3800223440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d92684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf03e00640 0 client.5499 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 >> 172.21.15.19:0/3800223440 conn(0x5590d8e754e0 msgr2=0x5590d924b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 shutdown_connections 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.306+0000 7fdf3123e740 1 -- 172.21.15.19:0/3800223440 wait complete. 2024-04-25T11:29:44.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.307+0000 7fdf3123e740 20 client.5499 trim_cache size 0 max 16384 2024-04-25T11:29:44.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.327+0000 7fdf3123e740 1 Processor -- start 2024-04-25T11:29:44.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.327+0000 7fdf3123e740 1 -- start start 2024-04-25T11:29:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.328+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d9290b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e606b0 0x5590d9265080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e81390 0x5590d92655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf3123e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241feb20 con 0x5590d8e81390 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2406ed60 con 0x5590d8e584d0 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2415fc30 con 0x5590d8e606b0 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e81390 0x5590d92655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e81390 0x5590d92655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38920/0 (socket says 172.21.15.19:38920) 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 -- 172.21.15.19:0/2318141397 learned_addr learned my addr 172.21.15.19:0/2318141397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf22a00640 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e606b0 0x5590d9265080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23400640 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d9290b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 -- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e606b0 msgr2=0x5590d9265080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e606b0 0x5590d9265080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 -- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 msgr2=0x5590d9290b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d9290b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23e00640 1 -- 172.21.15.19:0/2318141397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590d8e590f0 con 0x5590d8e81390 2024-04-25T11:29:44.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.329+0000 7fdf23400640 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d9290b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:44.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.330+0000 7fdf23e00640 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e81390 0x5590d92655c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009930 tx=0x7fdf1c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00a37187d28ece1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.330+0000 7fdf0ea00640 1 -- 172.21.15.19:0/2318141397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c037040 con 0x5590d8e81390 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.330+0000 7fdf0ea00640 1 -- 172.21.15.19:0/2318141397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c003610 con 0x5590d8e81390 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.330+0000 7fdf0ea00640 1 -- 172.21.15.19:0/2318141397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c041040 con 0x5590d8e81390 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 -- 172.21.15.19:0/2318141397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e81390 msgr2=0x5590d92655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e81390 0x5590d92655c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009930 tx=0x7fdf1c008030 comp rx=0 tx=0).stop 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 -- 172.21.15.19:0/2318141397 shutdown_connections 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e81390 0x5590d92655c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e606b0 0x5590d9265080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 --2- 172.21.15.19:0/2318141397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e584d0 0x5590d9290b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 -- 172.21.15.19:0/2318141397 >> 172.21.15.19:0/2318141397 conn(0x5590d8e754e0 msgr2=0x5590d9221ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 -- 172.21.15.19:0/2318141397 shutdown_connections 2024-04-25T11:29:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.331+0000 7fdf3123e740 1 -- 172.21.15.19:0/2318141397 wait complete. 2024-04-25T11:29:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.332+0000 7fdf3123e740 1 -- start start 2024-04-25T11:29:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.332+0000 7fdf3123e740 1 Processor -- start 2024-04-25T11:29:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e584d0 0x5590d9244670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e606b0 0x5590d9255fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf3123e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d9256500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf3123e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590d9194fd0 con 0x5590d8e584d0 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24160280 con 0x5590d8e606b0 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf3123e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5590d92676a0 con 0x5590d8e81390 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf3123e740 10 client.0 _ll_register_callbacks cb 0x7fdf10006e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e584d0 0x5590d9244670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e584d0 0x5590d9244670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38932/0 (socket says 172.21.15.19:38932) 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf23400640 1 -- 172.21.15.19:0/3332326184 learned_addr learned my addr 172.21.15.19:0/3332326184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.334+0000 7fdf23400640 1 -- 172.21.15.19:0/3332326184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 msgr2=0x5590d9256500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf23400640 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d9256500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf23400640 1 -- 172.21.15.19:0/3332326184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e606b0 msgr2=0x5590d9255fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf23400640 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e606b0 0x5590d9255fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf23400640 1 -- 172.21.15.19:0/3332326184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590d914fa30 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf23400640 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e584d0 0x5590d9244670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007130 tx=0x7fdf14004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f9111a61c0e1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf03e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf03e00640 1 -- 172.21.15.19:0/3332326184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1401d040 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf03e00640 1 -- 172.21.15.19:0/3332326184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fdf14074d40 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c003610 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf03e00640 1 -- 172.21.15.19:0/3332326184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdf1405fe80 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf3123e740 10 client.5505 fetch_fsmap learned FSMap version 25 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.335+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x5590d9254fe0 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf03e00640 1 -- 172.21.15.19:0/3332326184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c003610 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf03e00640 1 -- 172.21.15.19:0/3332326184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fdf1405e9e0 con 0x5590d8e584d0 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf3123e740 10 client.5505 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf3123e740 10 client.5505 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T11:29:44.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf3123e740 20 client.5505 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d92ed720 0x5590d92edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf3123e740 4 client.5505 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590d918f650 con 0x5590d92ed720 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.336+0000 7fdf22a00640 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d92ed720 0x5590d92edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf22a00640 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d92ed720 0x5590d92edb30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abd01a8727e1e79 server_cookie=8883ed81f8ae8527 in_seq=0 out_seq=0 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf03e00640 10 client.5505 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf03e00640 1 -- 172.21.15.19:0/3332326184 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5590d918f650 con 0x5590d92ed720 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf03e00640 10 client.5505 handle_command_reply: tid=0 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 client.5505 shutdown 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e584d0 msgr2=0x5590d9244670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e584d0 0x5590d9244670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf14007130 tx=0x7fdf14004aa0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 shutdown_connections 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x5590d92ed720 0x5590d92edb30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590d8e606b0 0x5590d9255fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590d8e81390 0x5590d9256500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 --2- 172.21.15.19:0/3332326184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590d8e584d0 0x5590d9244670 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf03e00640 0 client.5505 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.337+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 >> 172.21.15.19:0/3332326184 conn(0x5590d8e754e0 msgr2=0x5590d9265080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 shutdown_connections 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf3123e740 1 -- 172.21.15.19:0/3332326184 wait complete. 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf3123e740 20 client.5505 trim_cache size 0 max 16384 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-04-25T11:29:44.331 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-04-25T11:29:44.332 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:29:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf040744b0 msgr2=0x7fdf040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf040744b0 0x7fdf040769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).stop 2024-04-25T11:29:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240b3110 msgr2=0x7fdf240c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240b3110 0x7fdf240c8e80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400cd00 tx=0x7fdf1400a5c0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 shutdown_connections 2024-04-25T11:29:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf040744b0 0x7fdf040769c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf240d0a40 0x7fdf240b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf240cfc70 0x7fdf24154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 --2- 172.21.15.19:0/1706462133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf240b3110 0x7fdf240c8e80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.338+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 >> 172.21.15.19:0/1706462133 conn(0x7fdf2408d180 msgr2=0x7fdf2415df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.339+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 shutdown_connections 2024-04-25T11:29:44.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.339+0000 7fdf28c00640 1 -- 172.21.15.19:0/1706462133 wait complete. 2024-04-25T11:29:44.357 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T11:29:44.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:29:44.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.649+0000 7f2b7b200640 1 Processor -- start 2024-04-25T11:29:44.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.649+0000 7f2b7b200640 1 -- start start 2024-04-25T11:29:44.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c140e00 0x7f2b7c143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b7c06dce0 0x7f2b7c141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c141210 con 0x7f2b7c143a90 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c141b40 con 0x7f2b7c06dce0 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c143ea0 con 0x7f2b7c140e00 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b7c06dce0 0x7f2b7c141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b7c06dce0 0x7f2b7c141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38940/0 (socket says 172.21.15.19:38940) 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.651+0000 7f2b7a800640 1 -- 172.21.15.19:0/1942537725 learned_addr learned my addr 172.21.15.19:0/1942537725 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.652+0000 7f2b79e00640 1 --2- 172.21.15.19:0/1942537725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c140e00 0x7f2b7c143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.652+0000 7f2b7a800640 1 -- 172.21.15.19:0/1942537725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c140e00 msgr2=0x7f2b7c143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.652+0000 7f2b7a800640 1 --2- 172.21.15.19:0/1942537725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c140e00 0x7f2b7c143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.652+0000 7f2b7a800640 1 -- 172.21.15.19:0/1942537725 >> v1:172.21.15.169:6789/0 conn(0x7f2b7c143a90 legacy=0x7f2b7c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:44.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.652+0000 7f2b7a800640 1 -- 172.21.15.19:0/1942537725 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c141510 con 0x7f2b7c06dce0 2024-04-25T11:29:44.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.653+0000 7f2b7a800640 1 --2- 172.21.15.19:0/1942537725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c06dce0 0x7f2b7c141fd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0b73a0 tx=0x7f2b7402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac833cd7061e17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.653+0000 7f2b78a00640 1 -- 172.21.15.19:0/1942537725 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74036070 con 0x7f2b7c06dce0 2024-04-25T11:29:44.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b78a00640 1 -- 172.21.15.19:0/1942537725 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74031600 con 0x7f2b7c06dce0 2024-04-25T11:29:44.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b78a00640 1 -- 172.21.15.19:0/1942537725 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740319d0 con 0x7f2b7c06dce0 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b7b200640 1 -- 172.21.15.19:0/1942537725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c06dce0 msgr2=0x7f2b7c141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b7b200640 1 --2- 172.21.15.19:0/1942537725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c06dce0 0x7f2b7c141fd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0b73a0 tx=0x7f2b7402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b7b200640 1 -- 172.21.15.19:0/1942537725 shutdown_connections 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b7b200640 1 --2- 172.21.15.19:0/1942537725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c06dce0 0x7f2b7c141fd0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b7b200640 1 --2- 172.21.15.19:0/1942537725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c140e00 0x7f2b7c143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b7b200640 1 -- 172.21.15.19:0/1942537725 >> 172.21.15.19:0/1942537725 conn(0x7f2b7c072ee0 msgr2=0x7f2b7c0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.654+0000 7f2b7b200640 1 -- 172.21.15.19:0/1942537725 shutdown_connections 2024-04-25T11:29:44.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.655+0000 7f2b7b200640 1 -- 172.21.15.19:0/1942537725 wait complete. 2024-04-25T11:29:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.657+0000 7f2b7b200640 1 Processor -- start 2024-04-25T11:29:44.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.657+0000 7f2b7b200640 1 -- start start 2024-04-25T11:29:44.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.658+0000 7f2b7b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c140e00 0x7f2b7c162490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b7b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c143a90 0x7f2b7c13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b7b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b7c1f6250 0x7f2b7c0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b7b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1417d0 con 0x7f2b7c140e00 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b7b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1443d0 con 0x7f2b7c1f6250 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b7b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c0d1490 con 0x7f2b7c143a90 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c140e00 0x7f2b7c162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c140e00 0x7f2b7c162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38956/0 (socket says 172.21.15.19:38956) 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 -- 172.21.15.19:0/516581532 learned_addr learned my addr 172.21.15.19:0/516581532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 -- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c143a90 msgr2=0x7f2b7c13bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79400640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c143a90 0x7f2b7c13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b7a800640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b7c1f6250 0x7f2b7c0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c143a90 0x7f2b7c13bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 -- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b7c1f6250 msgr2=0x7f2b7c0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b7c1f6250 0x7f2b7c0cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.659+0000 7f2b79e00640 1 -- 172.21.15.19:0/516581532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c157a70 con 0x7f2b7c140e00 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.660+0000 7f2b79e00640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c140e00 0x7f2b7c162490 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d620 tx=0x7f2b6c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.660+0000 7f2b72000640 1 -- 172.21.15.19:0/516581532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00b620 con 0x7f2b7c140e00 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.660+0000 7f2b72000640 1 -- 172.21.15.19:0/516581532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c019070 con 0x7f2b7c140e00 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.660+0000 7f2b72000640 1 -- 172.21.15.19:0/516581532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00ba00 con 0x7f2b7c140e00 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.660+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c157d50 con 0x7f2b7c140e00 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.660+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c144c00 con 0x7f2b7c140e00 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.661+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c157f20 con 0x7f2b7c140e00 2024-04-25T11:29:44.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.663+0000 7f2b72000640 1 -- 172.21.15.19:0/516581532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b6c00bba0 con 0x7f2b7c140e00 2024-04-25T11:29:44.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.664+0000 7f2b72000640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:44.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.664+0000 7f2b72000640 1 -- 172.21.15.19:0/516581532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f2b6c095750 con 0x7f2b7c140e00 2024-04-25T11:29:44.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.665+0000 7f2b79400640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:44.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.666+0000 7f2b79400640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b68006fd0 tx=0x7f2b68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:44.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.670+0000 7f2b72000640 1 -- 172.21.15.19:0/516581532 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b6c05fe00 con 0x7f2b7c140e00 2024-04-25T11:29:44.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.913+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b7c050740 con 0x7f2b7c140e00 2024-04-25T11:29:44.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.914+0000 7f2b72000640 1 -- 172.21.15.19:0/516581532 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f2b6c05f700 con 0x7f2b7c140e00 2024-04-25T11:29:44.905 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:29:44.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b5c0744b0 msgr2=0x7f2b5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b68006fd0 tx=0x7f2b68008040 comp rx=0 tx=0).stop 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c140e00 msgr2=0x7f2b7c162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c140e00 0x7f2b7c162490 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d620 tx=0x7f2b6c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 shutdown_connections 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b7c1f6250 0x7f2b7c0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b7c143a90 0x7f2b7c13bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 --2- 172.21.15.19:0/516581532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b7c140e00 0x7f2b7c162490 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 >> 172.21.15.19:0/516581532 conn(0x7f2b7c072ee0 msgr2=0x7f2b7c070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.916+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 shutdown_connections 2024-04-25T11:29:44.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:44.917+0000 7f2b7b200640 1 -- 172.21.15.19:0/516581532 wait complete. 2024-04-25T11:29:44.935 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:29:44.935 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:29:44.936 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:29:44.936 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:29:44.936 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:44.936 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:29:44.936 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:29:44.936 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:29:44.936 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:29:44.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:44.936 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:29:44.989 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:29:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:29:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:29:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:44.989 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42489sec preferred_lft 42489sec 2024-04-25T11:29:44.990 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:29:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:44.992 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:29:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:29:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:29:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:29:44.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:44.993 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:29:45.072 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:29:45.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:45.076 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:29:45.131 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:29:45.131 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:29:45.131 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:29:45.131 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:29:45.131 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:29:45.132 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:29:45.132 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:29:45.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:29:45.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:29:45.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:29:45.133 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:29:45.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:29:45.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:29:45.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:45.357 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:29:45.410 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:29:45.410 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:29:45.411 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:29:45.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:45.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:29:45.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:29:45.505 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:45.561 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:29:45.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:29:45.562 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:45.616 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:29:45.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:45.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:29:45.831 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:29:45.831 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:29:45.876 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:29:45.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:29:45.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:29:45.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:29:45.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:29:45.878 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:29:45.879 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:29:45.879 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:29:45.879 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:29:45.879 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:29:45.884 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:29:45.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:45.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:45.967 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:29:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:29:46.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:29:46.093 INFO:teuthology.orchestra.run.smithi019.stdout:client4820 2024-04-25T11:29:46.093 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:29:46.093 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:46.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:46.147 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T11:29:46.206 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:29:46.206 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:29:46.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.407+0000 7f73bc477f40 1 Processor -- start 2024-04-25T11:29:46.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.408+0000 7f73bc477f40 1 -- start start 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73bc477f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e01023e210 0x55e01030a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73bc477f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 0x55e01023cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73bc477f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e010317fd0 0x55e01023d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73bc477f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e010286ea0 con 0x55e01023e210 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73bc477f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e0101e4d20 con 0x55e01023e620 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73bc477f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e010287420 con 0x55e010317fd0 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73b5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e01023e210 0x55e01030a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73b5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e01023e210 0x55e01030a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38968/0 (socket says 172.21.15.19:38968) 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.409+0000 7f73b5600640 1 -- 172.21.15.19:0/645114558 learned_addr learned my addr 172.21.15.19:0/645114558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73b5600640 1 -- 172.21.15.19:0/645114558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e010317fd0 msgr2=0x55e01023d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73b5600640 1 --2- 172.21.15.19:0/645114558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e010317fd0 0x55e01023d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73b5600640 1 -- 172.21.15.19:0/645114558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 msgr2=0x55e01023cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73b5600640 1 --2- 172.21.15.19:0/645114558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 0x55e01023cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73b5600640 1 -- 172.21.15.19:0/645114558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01028bd30 con 0x55e01023e210 2024-04-25T11:29:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73b5600640 1 --2- 172.21.15.19:0/645114558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e01023e210 0x55e01030a270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f73a8045f30 tx=0x7f73a8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6120e5bbec04f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73af400640 1 -- 172.21.15.19:0/645114558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8053860 con 0x55e01023e210 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73af400640 1 -- 172.21.15.19:0/645114558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a8053a00 con 0x55e01023e210 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.410+0000 7f73af400640 1 -- 172.21.15.19:0/645114558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8053df0 con 0x55e01023e210 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 -- 172.21.15.19:0/645114558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e01023e210 msgr2=0x55e01030a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 --2- 172.21.15.19:0/645114558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e01023e210 0x55e01030a270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f73a8045f30 tx=0x7f73a8051000 comp rx=0 tx=0).stop 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 -- 172.21.15.19:0/645114558 shutdown_connections 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 --2- 172.21.15.19:0/645114558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 0x55e01023cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 --2- 172.21.15.19:0/645114558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e010317fd0 0x55e01023d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 --2- 172.21.15.19:0/645114558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e01023e210 0x55e01030a270 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 -- 172.21.15.19:0/645114558 >> 172.21.15.19:0/645114558 conn(0x55e010160ac0 msgr2=0x55e010314740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 -- 172.21.15.19:0/645114558 shutdown_connections 2024-04-25T11:29:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.411+0000 7f73bc477f40 1 -- 172.21.15.19:0/645114558 wait complete. 2024-04-25T11:29:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.412+0000 7f73bc477f40 1 Processor -- start 2024-04-25T11:29:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.412+0000 7f73bc477f40 1 -- start start 2024-04-25T11:29:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73bc477f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e01023e210 0x55e010306910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73bc477f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 0x55e010311ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73bc477f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e010317fd0 0x55e010310a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73bc477f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e01026ebc0 con 0x55e010317fd0 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73bc477f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e01026f0c0 con 0x55e01023e620 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73bc477f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e01030c710 con 0x55e01023e210 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73b6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e010317fd0 0x55e010310a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73b6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e010317fd0 0x55e010310a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38974/0 (socket says 172.21.15.19:38974) 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.414+0000 7f73b6000640 1 -- 172.21.15.19:0/1203159443 learned_addr learned my addr 172.21.15.19:0/1203159443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b5600640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e01023e210 0x55e010306910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b4c00640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 0x55e010311ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b6000640 1 -- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e01023e210 msgr2=0x55e010306910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b6000640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e01023e210 0x55e010306910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b6000640 1 -- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 msgr2=0x55e010311ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b6000640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 0x55e010311ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b6000640 1 -- 172.21.15.19:0/1203159443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01023d430 con 0x55e010317fd0 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b5600640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e01023e210 0x55e010306910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b4c00640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e01023e620 0x55e010311ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.415+0000 7f73b6000640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e010317fd0 0x55e010310a30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009da0 tx=0x7f73b0006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.416+0000 7f73ae000640 1 -- 172.21.15.19:0/1203159443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0017070 con 0x55e010317fd0 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.416+0000 7f73ae000640 1 -- 172.21.15.19:0/1203159443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b0005e10 con 0x55e010317fd0 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.416+0000 7f73ae000640 1 -- 172.21.15.19:0/1203159443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0006e60 con 0x55e010317fd0 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.416+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e01023d710 con 0x55e010317fd0 2024-04-25T11:29:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.416+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e010310230 con 0x55e010317fd0 2024-04-25T11:29:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.418+0000 7f73ae000640 1 -- 172.21.15.19:0/1203159443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f73b001a070 con 0x55e010317fd0 2024-04-25T11:29:46.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.419+0000 7f73ae000640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7398072e90 0x7f73980753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.419+0000 7f73ae000640 1 -- 172.21.15.19:0/1203159443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f73b0092f30 con 0x55e010317fd0 2024-04-25T11:29:46.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.420+0000 7f73bc477f40 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e0103aec30 0x55e0103af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.420+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38) -- 0x55e010306e50 con 0x55e0103aec30 2024-04-25T11:29:46.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.420+0000 7f73b5600640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7398072e90 0x7f73980753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.421+0000 7f73b4c00640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e0103aec30 0x55e0103af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.422+0000 7f73b5600640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7398072e90 0x7f73980753a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73a8025d00 tx=0x7f73a8053000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.422+0000 7f73b4c00640 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e0103aec30 0x55e0103af040 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:46.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.425+0000 7f73b4c00640 1 -- 172.21.15.19:0/1203159443 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v38'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73a400c600 con 0x55e0103aec30 2024-04-25T11:29:46.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e0103aec30 msgr2=0x55e0103af040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e0103aec30 0x55e0103af040 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7398072e90 msgr2=0x7f73980753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7398072e90 0x7f73980753a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73a8025d00 tx=0x7f73a8053000 comp rx=0 tx=0).stop 2024-04-25T11:29:46.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e010317fd0 msgr2=0x55e010310a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 --2- 172.21.15.19:0/1203159443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e010317fd0 0x55e010310a30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009da0 tx=0x7f73b0006750 comp rx=0 tx=0).stop 2024-04-25T11:29:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73b6000640 1 -- 172.21.15.19:0/1203159443 reap_dead start 2024-04-25T11:29:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 shutdown_connections 2024-04-25T11:29:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 >> 172.21.15.19:0/1203159443 conn(0x55e010160ac0 msgr2=0x55e01026c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 shutdown_connections 2024-04-25T11:29:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.426+0000 7f73bc477f40 1 -- 172.21.15.19:0/1203159443 wait complete. 2024-04-25T11:29:46.431 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:46.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.724+0000 7fece7e00640 1 Processor -- start 2024-04-25T11:29:46.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.724+0000 7fece7e00640 1 -- start start 2024-04-25T11:29:46.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.726+0000 7fece7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fece8144370 0x7fece80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.726+0000 7fece7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece80d1230 con 0x7fece806c5c0 2024-04-25T11:29:46.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.726+0000 7fece7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece80d06b0 con 0x7fece80b4ea0 2024-04-25T11:29:46.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.726+0000 7fece7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fece80d0990 con 0x7fece8144370 2024-04-25T11:29:46.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.727+0000 7fece6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fece806c5c0 0x7fece80b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35838/0 (socket says 172.21.15.19:35838) 2024-04-25T11:29:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.727+0000 7fece6a00640 1 -- 172.21.15.19:0/2684480218 learned_addr learned my addr 172.21.15.19:0/2684480218 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.727+0000 7fece7400640 1 --2- 172.21.15.19:0/2684480218 >> v2:172.21.15.169:3300/0 conn(0x7fece8144370 0x7fece80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.727+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923651566 0 0) 0x7fece80d1230 con 0x7fece806c5c0 2024-04-25T11:29:46.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feccc003520 con 0x7fece806c5c0 2024-04-25T11:29:46.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2397081702 0 0) 0x7feccc003520 con 0x7fece806c5c0 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 >> v2:172.21.15.169:3300/0 conn(0x7fece8144370 msgr2=0x7fece80cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 --2- 172.21.15.19:0/2684480218 >> v2:172.21.15.169:3300/0 conn(0x7fece8144370 0x7fece80cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 >> v1:172.21.15.169:6789/0 conn(0x7fece80b4ea0 legacy=0x7fece815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece80d0c70 con 0x7fece806c5c0 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fecd0001670 con 0x7fece806c5c0 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece7400640 1 --2- 172.21.15.19:0/2684480218 >> v2:172.21.15.169:3300/0 conn(0x7fece8144370 0x7fece80cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 >> v1:172.21.15.19:6789/0 conn(0x7fece806c5c0 legacy=0x7fece80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc003a00 0x7feccc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc004800 0x7feccc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feccc005350 0x7feccc00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fece80d0990 con 0x7feccc003a00 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feccc010660 con 0x7feccc005350 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feccc010860 con 0x7feccc004800 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.728+0000 7fece6a00640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc003a00 0x7feccc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece6000640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc004800 0x7feccc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece7400640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feccc005350 0x7feccc00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece6a00640 1 -- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc004800 msgr2=0x7feccc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece6a00640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc004800 0x7feccc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece6a00640 1 -- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feccc005350 msgr2=0x7feccc00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece6a00640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feccc005350 0x7feccc00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece6a00640 1 -- 172.21.15.19:0/2684480218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc003520 con 0x7feccc003a00 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece7400640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feccc005350 0x7feccc00fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece7400640 1 -- 172.21.15.19:0/2684480218 reap_dead start 2024-04-25T11:29:46.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.729+0000 7fece6000640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc004800 0x7feccc004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.730+0000 7fece6a00640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc003a00 0x7feccc003e70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c110 tx=0x7feccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bedc6fbff90c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.730+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0007c40 con 0x7feccc003a00 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.730+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd0005af0 con 0x7feccc003a00 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.730+0000 7fece5600640 1 -- 172.21.15.19:0/2684480218 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0007070 con 0x7feccc003a00 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.731+0000 7fece7e00640 1 -- 172.21.15.19:0/2684480218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc003a00 msgr2=0x7feccc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.731+0000 7fece7e00640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc003a00 0x7feccc003e70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fecd000c110 tx=0x7feccc005830 comp rx=0 tx=0).stop 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.731+0000 7fece7e00640 1 -- 172.21.15.19:0/2684480218 shutdown_connections 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.731+0000 7fece7e00640 1 --2- 172.21.15.19:0/2684480218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc003a00 0x7feccc003e70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.731+0000 7fece7e00640 1 -- 172.21.15.19:0/2684480218 >> 172.21.15.19:0/2684480218 conn(0x7fece80d1720 msgr2=0x7fece80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.731+0000 7fece7e00640 1 -- 172.21.15.19:0/2684480218 shutdown_connections 2024-04-25T11:29:46.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.731+0000 7fece7e00640 1 -- 172.21.15.19:0/2684480218 wait complete. 2024-04-25T11:29:46.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.734+0000 7fece7e00640 1 Processor -- start 2024-04-25T11:29:46.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.734+0000 7fece7e00640 1 -- start start 2024-04-25T11:29:46.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.735+0000 7fece7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc003a00 0x7fece8159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.735+0000 7fece7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece813e560 0x7fece8143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.735+0000 7fece7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fece813e970 0x7fece81439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.735+0000 7fece7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fece80d0450 con 0x7fece813e970 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.735+0000 7fece7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fece80d02d0 con 0x7fece813e560 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.735+0000 7fece7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fece8147760 con 0x7feccc003a00 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fece813e970 0x7fece81439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fece813e970 0x7fece81439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38982/0 (socket says 172.21.15.19:38982) 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 -- 172.21.15.19:0/1374323453 learned_addr learned my addr 172.21.15.19:0/1374323453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece6a00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc003a00 0x7fece8159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 -- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc003a00 msgr2=0x7fece8159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc003a00 0x7fece8159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece6000640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece813e560 0x7fece8143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 -- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece813e560 msgr2=0x7fece8143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece813e560 0x7fece8143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.736+0000 7fece7400640 1 -- 172.21.15.19:0/1374323453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece8159bc0 con 0x7fece813e970 2024-04-25T11:29:46.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.737+0000 7fece7400640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fece813e970 0x7fece81439a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fece80b7a50 tx=0x7fece0004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:46.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.737+0000 7fecdea00640 1 -- 172.21.15.19:0/1374323453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0019070 con 0x7fece813e970 2024-04-25T11:29:46.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.737+0000 7fecdea00640 1 -- 172.21.15.19:0/1374323453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece0007cc0 con 0x7fece813e970 2024-04-25T11:29:46.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.737+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece813d890 con 0x7fece813e970 2024-04-25T11:29:46.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.737+0000 7fecdea00640 1 -- 172.21.15.19:0/1374323453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece0014640 con 0x7fece813e970 2024-04-25T11:29:46.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.737+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece813f9e0 con 0x7fece813e970 2024-04-25T11:29:46.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.738+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece8050740 con 0x7fece813e970 2024-04-25T11:29:46.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.740+0000 7fecdea00640 1 -- 172.21.15.19:0/1374323453 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fece001c070 con 0x7fece813e970 2024-04-25T11:29:46.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.741+0000 7fecdea00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc40744b0 0x7fecc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:46.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.741+0000 7fecdea00640 1 -- 172.21.15.19:0/1374323453 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fece0094f20 con 0x7fece813e970 2024-04-25T11:29:46.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.742+0000 7fece6a00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc40744b0 0x7fecc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:46.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.743+0000 7fece6a00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc40744b0 0x7fecc40769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fecd0032000 tx=0x7fecd0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:46.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.745+0000 7fecdea00640 1 -- 172.21.15.19:0/1374323453 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fece8159bc0 con 0x7fece813e970 2024-04-25T11:29:46.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.993+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fece806ca70 con 0x7fece813e970 2024-04-25T11:29:46.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.995+0000 7fecdea00640 1 -- 172.21.15.19:0/1374323453 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5179 (secure 0 0 0) 0x7fece005f650 con 0x7fece813e970 2024-04-25T11:29:46.986 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:46.986 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:19.352476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5331,"qdb_cluster":[5331]},"id":2}]} 2024-04-25T11:29:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T11:29:46.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc40744b0 msgr2=0x7fecc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc40744b0 0x7fecc40769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fecd0032000 tx=0x7fecd0009030 comp rx=0 tx=0).stop 2024-04-25T11:29:46.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fece813e970 msgr2=0x7fece81439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:46.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fece813e970 0x7fece81439a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fece80b7a50 tx=0x7fece0004780 comp rx=0 tx=0).stop 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 shutdown_connections 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc40744b0 0x7fecc40769c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fece813e560 0x7fece8143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fece813e970 0x7fece81439a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 --2- 172.21.15.19:0/1374323453 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feccc003a00 0x7fece8159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 >> 172.21.15.19:0/1374323453 conn(0x7fece80d1720 msgr2=0x7fece806d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 shutdown_connections 2024-04-25T11:29:46.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:46.998+0000 7fece7e00640 1 -- 172.21.15.19:0/1374323453 wait complete. 2024-04-25T11:29:47.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3452871827 2024-04-25T11:29:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.306+0000 7fd7c8a00640 1 Processor -- start 2024-04-25T11:29:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.306+0000 7fd7c8a00640 1 -- start start 2024-04-25T11:29:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.309+0000 7fd7c8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c40b1c70 0x7fd7c4163660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.309+0000 7fd7c8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7c406dc80 0x7fd7c40af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.309+0000 7fd7c8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c40b22d0 con 0x7fd7c406d870 2024-04-25T11:29:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.309+0000 7fd7c8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c409bb00 con 0x7fd7c406dc80 2024-04-25T11:29:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.309+0000 7fd7c8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c40b2830 con 0x7fd7c40b1c70 2024-04-25T11:29:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c40b1c70 0x7fd7c4163660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd7c406dc80 0x7fd7c40af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c1e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fd7c406d870 0x7fd7c40af130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:49488/0 (socket says 172.21.15.19:49488) 2024-04-25T11:29:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c1e00640 1 -- 172.21.15.19:0/948351760 learned_addr learned my addr 172.21.15.19:0/948351760 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c3200640 1 -- 172.21.15.19:0/948351760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c40b1c70 msgr2=0x7fd7c4163660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c3200640 1 --2- 172.21.15.19:0/948351760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c40b1c70 0x7fd7c4163660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c3200640 1 -- 172.21.15.19:0/948351760 >> v1:172.21.15.169:6789/0 conn(0x7fd7c406d870 legacy=0x7fd7c40af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.310+0000 7fd7c3200640 1 -- 172.21.15.19:0/948351760 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c40b2570 con 0x7fd7c406dc80 2024-04-25T11:29:47.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.312+0000 7fd7c3200640 1 --2- 172.21.15.19:0/948351760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 0x7fd7c40af840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40b29b0 tx=0x7fd7bc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8f1e7e1ff991c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:47.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c1400640 1 -- 172.21.15.19:0/948351760 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007740 con 0x7fd7c406dc80 2024-04-25T11:29:47.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c1400640 1 -- 172.21.15.19:0/948351760 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc0078e0 con 0x7fd7c406dc80 2024-04-25T11:29:47.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c1400640 1 -- 172.21.15.19:0/948351760 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007cb0 con 0x7fd7c406dc80 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c8a00640 1 -- 172.21.15.19:0/948351760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 msgr2=0x7fd7c40af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/948351760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 0x7fd7c40af840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40b29b0 tx=0x7fd7bc02a660 comp rx=0 tx=0).stop 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c8a00640 1 -- 172.21.15.19:0/948351760 shutdown_connections 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/948351760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 0x7fd7c40af840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/948351760 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c40b1c70 0x7fd7c4163660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.313+0000 7fd7c8a00640 1 -- 172.21.15.19:0/948351760 >> 172.21.15.19:0/948351760 conn(0x7fd7c4073a30 msgr2=0x7fd7c40b7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.314+0000 7fd7c8a00640 1 -- 172.21.15.19:0/948351760 shutdown_connections 2024-04-25T11:29:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.314+0000 7fd7c8a00640 1 -- 172.21.15.19:0/948351760 wait complete. 2024-04-25T11:29:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.316+0000 7fd7c8a00640 1 Processor -- start 2024-04-25T11:29:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.316+0000 7fd7c8a00640 1 -- start start 2024-04-25T11:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c406d870 0x7fd7c409c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 0x7fd7c40d1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c40b1c70 0x7fd7c40cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c8a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4163c80 con 0x7fd7c40b1c70 2024-04-25T11:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c406d350 con 0x7fd7c406dc80 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c4163e00 con 0x7fd7c406d870 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c40b1c70 0x7fd7c40cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c40b1c70 0x7fd7c40cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38986/0 (socket says 172.21.15.19:38986) 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c3200640 1 -- 172.21.15.19:0/887815918 learned_addr learned my addr 172.21.15.19:0/887815918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 0x7fd7c40d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.319+0000 7fd7c2800640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c406d870 0x7fd7c409c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.320+0000 7fd7c3200640 1 -- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c406d870 msgr2=0x7fd7c409c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.320+0000 7fd7c3200640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c406d870 0x7fd7c409c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.320+0000 7fd7c3200640 1 -- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 msgr2=0x7fd7c40d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.320+0000 7fd7c3200640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 0x7fd7c40d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.320+0000 7fd7c3200640 1 -- 172.21.15.19:0/887815918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c409da70 con 0x7fd7c40b1c70 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.320+0000 7fd7c2800640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c406d870 0x7fd7c409c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.320+0000 7fd7c3200640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c40b1c70 0x7fd7c40cbda0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009e50 tx=0x7fd7bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.321+0000 7fd7baa00640 1 -- 172.21.15.19:0/887815918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc03f070 con 0x7fd7c40b1c70 2024-04-25T11:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.321+0000 7fd7baa00640 1 -- 172.21.15.19:0/887815918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc031b70 con 0x7fd7c40b1c70 2024-04-25T11:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.321+0000 7fd7baa00640 1 -- 172.21.15.19:0/887815918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc044460 con 0x7fd7c40b1c70 2024-04-25T11:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.321+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c409c9d0 con 0x7fd7c40b1c70 2024-04-25T11:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.321+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c409cf90 con 0x7fd7c40b1c70 2024-04-25T11:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.325+0000 7fd7baa00640 1 -- 172.21.15.19:0/887815918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd7bc03b410 con 0x7fd7c40b1c70 2024-04-25T11:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.326+0000 7fd7baa00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7a40744b0 0x7fd7a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.326+0000 7fd7baa00640 1 -- 172.21.15.19:0/887815918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fd7bc0b6a20 con 0x7fd7c40b1c70 2024-04-25T11:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.326+0000 7fd7d0414740 1 Processor -- start 2024-04-25T11:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.327+0000 7fd7d0414740 1 -- start start 2024-04-25T11:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.327+0000 7fd7c2800640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7a40744b0 0x7fd7a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.328+0000 7fd7c2800640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7a40744b0 0x7fd7a40769c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009af0 tx=0x7fd7b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.328+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c6802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.328+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.328+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c654910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.328+0000 7fd7d0414740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c5ba900 con 0x55ea0c28b3e0 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.328+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c5571f0 con 0x55ea0c2935c0 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.328+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ea0c5b68c0 con 0x55ea0c2b42a0 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38988/0 (socket says 172.21.15.19:38988) 2024-04-25T11:29:47.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 -- 172.21.15.19:0/580294897 learned_addr learned my addr 172.21.15.19:0/580294897 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c2800640 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c6802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c3200640 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c654910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 -- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 msgr2=0x55ea0c6802f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c6802f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 -- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 msgr2=0x55ea0c654910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c654910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 -- 172.21.15.19:0/580294897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0c398710 con 0x55ea0c28b3e0 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.329+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6543d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40b37f0 tx=0x7fd7b0007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173dc5d55417c9b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.330+0000 7fd7af000640 1 -- 172.21.15.19:0/580294897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0005ac0 con 0x55ea0c28b3e0 2024-04-25T11:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.330+0000 7fd7af000640 1 -- 172.21.15.19:0/580294897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b0018070 con 0x55ea0c28b3e0 2024-04-25T11:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.330+0000 7fd7af000640 1 -- 172.21.15.19:0/580294897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0005e60 con 0x55ea0c28b3e0 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.330+0000 7fd7d0414740 1 -- 172.21.15.19:0/580294897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 msgr2=0x55ea0c6543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.330+0000 7fd7d0414740 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6543d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40b37f0 tx=0x7fd7b0007e80 comp rx=0 tx=0).stop 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.331+0000 7fd7d0414740 1 -- 172.21.15.19:0/580294897 shutdown_connections 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.331+0000 7fd7d0414740 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c654910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.331+0000 7fd7d0414740 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6543d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.331+0000 7fd7d0414740 1 --2- 172.21.15.19:0/580294897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c6802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.331+0000 7fd7d0414740 1 -- 172.21.15.19:0/580294897 >> 172.21.15.19:0/580294897 conn(0x55ea0c2a83f0 msgr2=0x55ea0c69b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.331+0000 7fd7d0414740 1 -- 172.21.15.19:0/580294897 shutdown_connections 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.331+0000 7fd7d0414740 1 -- 172.21.15.19:0/580294897 wait complete. 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.332+0000 7fd7d0414740 1 -- start start 2024-04-25T11:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.332+0000 7fd7d0414740 1 Processor -- start 2024-04-25T11:29:47.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c28b3e0 0x55ea0c679870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c663420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c2b42a0 0x55ea0c6835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7d0414740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c542040 con 0x55ea0c2b42a0 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c6625b0 con 0x55ea0c2935c0 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ea0c6776f0 con 0x55ea0c28b3e0 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7d0414740 10 client.0 _ll_register_callbacks cb 0x55ea0c572220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7c3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c2b42a0 0x55ea0c6835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7c3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c2b42a0 0x55ea0c6835f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38994/0 (socket says 172.21.15.19:38994) 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7c3200640 1 -- 172.21.15.19:0/2323978220 learned_addr learned my addr 172.21.15.19:0/2323978220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7c2800640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c28b3e0 0x55ea0c679870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c663420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7c3200640 1 -- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c28b3e0 msgr2=0x55ea0c679870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.334+0000 7fd7c3200640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c28b3e0 0x55ea0c679870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7c3200640 1 -- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 msgr2=0x55ea0c663420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7c3200640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c663420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7c3200640 1 -- 172.21.15.19:0/2323978220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0c3f8ea0 con 0x55ea0c2b42a0 2024-04-25T11:29:47.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c663420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:47.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7c2800640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c28b3e0 0x55ea0c679870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7c3200640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c2b42a0 0x55ea0c6835f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0b4de0 tx=0x7fd7bc080200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be86cd980e2f4835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7a3400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7a3400640 1 -- 172.21.15.19:0/2323978220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc049040 con 0x55ea0c2b42a0 2024-04-25T11:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7a3400640 1 -- 172.21.15.19:0/2323978220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fd7bc0893d0 con 0x55ea0c2b42a0 2024-04-25T11:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b0018070 con 0x55ea0c2b42a0 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7a3400640 1 -- 172.21.15.19:0/2323978220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7bc081190 con 0x55ea0c2b42a0 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7d0414740 10 client.5535 fetch_fsmap learned FSMap version 25 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.335+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55ea0c28c000 con 0x55ea0c2b42a0 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7a3400640 1 -- 172.21.15.19:0/2323978220 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b0018070 con 0x55ea0c2b42a0 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7a3400640 1 -- 172.21.15.19:0/2323978220 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fd7bc087070 con 0x55ea0c2b42a0 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7d0414740 10 client.5535 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7d0414740 10 client.5535 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7d0414740 20 client.5535 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c273080 0x55ea0c6618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7d0414740 4 client.5535 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0c5c8510 con 0x55ea0c273080 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.336+0000 7fd7c2800640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c273080 0x55ea0c6618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.337+0000 7fd7c2800640 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c273080 0x55ea0c6618f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f6b0426382b6d7 server_cookie=69dbc9219175fd03 in_seq=0 out_seq=0 2024-04-25T11:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.337+0000 7fd7a3400640 10 client.5535 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7a3400640 1 -- 172.21.15.19:0/2323978220 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea0c5c8510 con 0x55ea0c273080 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7a3400640 10 client.5535 handle_command_reply: tid=0 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 client.5535 shutdown 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c2b42a0 msgr2=0x55ea0c6835f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c2b42a0 0x55ea0c6835f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0b4de0 tx=0x7fd7bc080200 comp rx=0 tx=0).stop 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 shutdown_connections 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c273080 0x55ea0c6618f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c663420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c2b42a0 0x55ea0c6835f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2323978220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c28b3e0 0x55ea0c679870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 >> 172.21.15.19:0/2323978220 conn(0x55ea0c2a83f0 msgr2=0x55ea0c6572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7a3400640 0 client.5535 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.338+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 shutdown_connections 2024-04-25T11:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.339+0000 7fd7d0414740 1 -- 172.21.15.19:0/2323978220 wait complete. 2024-04-25T11:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.339+0000 7fd7d0414740 20 client.5535 trim_cache size 0 max 16384 2024-04-25T11:29:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.355+0000 7fd7d0414740 1 Processor -- start 2024-04-25T11:29:47.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.355+0000 7fd7d0414740 1 -- start start 2024-04-25T11:29:47.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.356+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.356+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.356+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c680030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.356+0000 7fd7d0414740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c4202090 con 0x55ea0c28b3e0 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.356+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c42013f0 con 0x55ea0c2935c0 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.356+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c4201a40 con 0x55ea0c2b42a0 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39008/0 (socket says 172.21.15.19:39008) 2024-04-25T11:29:47.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 -- 172.21.15.19:0/2499043655 learned_addr learned my addr 172.21.15.19:0/2499043655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 -- 172.21.15.19:0/2499043655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 msgr2=0x55ea0c680030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 --2- 172.21.15.19:0/2499043655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c680030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 -- 172.21.15.19:0/2499043655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 msgr2=0x55ea0c6765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 --2- 172.21.15.19:0/2499043655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 -- 172.21.15.19:0/2499043655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0c6789b0 con 0x55ea0c28b3e0 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.357+0000 7fd7c2800640 1 --2- 172.21.15.19:0/2499043655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6840f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40097d0 tx=0x7fd7b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbe3077abcb411a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7ae600640 1 -- 172.21.15.19:0/2499043655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4030070 con 0x55ea0c28b3e0 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7ae600640 1 -- 172.21.15.19:0/2499043655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b401a040 con 0x55ea0c28b3e0 2024-04-25T11:29:47.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7ae600640 1 -- 172.21.15.19:0/2499043655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b402a2d0 con 0x55ea0c28b3e0 2024-04-25T11:29:47.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7d0414740 1 -- 172.21.15.19:0/2499043655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 msgr2=0x55ea0c6840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2499043655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6840f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40097d0 tx=0x7fd7b4014000 comp rx=0 tx=0).stop 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7d0414740 1 -- 172.21.15.19:0/2499043655 shutdown_connections 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2499043655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2499043655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c680030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7d0414740 1 --2- 172.21.15.19:0/2499043655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c6840f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.358+0000 7fd7d0414740 1 -- 172.21.15.19:0/2499043655 >> 172.21.15.19:0/2499043655 conn(0x55ea0c2a83f0 msgr2=0x55ea0c67d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.359+0000 7fd7d0414740 1 -- 172.21.15.19:0/2499043655 shutdown_connections 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.359+0000 7fd7d0414740 1 -- 172.21.15.19:0/2499043655 wait complete. 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.360+0000 7fd7d0414740 1 -- start start 2024-04-25T11:29:47.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.360+0000 7fd7d0414740 1 Processor -- start 2024-04-25T11:29:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.361+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c7185d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.361+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.361+0000 7fd7d0414740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c655730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.361+0000 7fd7d0414740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c675570 con 0x55ea0c28b3e0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.361+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c676340 con 0x55ea0c2935c0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.361+0000 7fd7d0414740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ea0c675b00 con 0x55ea0c2b42a0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.361+0000 7fd7d0414740 10 client.0 _ll_register_callbacks cb 0x7fd7bc009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c7185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c7185d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39016/0 (socket says 172.21.15.19:39016) 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 -- 172.21.15.19:0/1678577656 learned_addr learned my addr 172.21.15.19:0/1678577656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c3200640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c655730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 -- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 msgr2=0x55ea0c655730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c655730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 -- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 msgr2=0x55ea0c6bd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6bd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c2800640 1 -- 172.21.15.19:0/1678577656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0c71c650 con 0x55ea0c28b3e0 2024-04-25T11:29:47.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.362+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6bd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7c2800640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c7185d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7b402e040 tx=0x7fd7b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df088103d27dcc6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7a3400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7a3400640 1 -- 172.21.15.19:0/1678577656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4030030 con 0x55ea0c28b3e0 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7a3400640 1 -- 172.21.15.19:0/1678577656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fd7b4029040 con 0x55ea0c28b3e0 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7b401a040 con 0x55ea0c28b3e0 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7a3400640 1 -- 172.21.15.19:0/1678577656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7b4040050 con 0x55ea0c28b3e0 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7d0414740 10 client.5541 fetch_fsmap learned FSMap version 25 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.363+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55ea0c69b7c0 con 0x55ea0c28b3e0 2024-04-25T11:29:47.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7a3400640 1 -- 172.21.15.19:0/1678577656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b401a040 con 0x55ea0c28b3e0 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7a3400640 1 -- 172.21.15.19:0/1678577656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fd7b4043020 con 0x55ea0c28b3e0 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7d0414740 10 client.5541 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7d0414740 10 client.5541 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7d0414740 20 client.5541 populate_metadata read hostname 'smithi019' 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7d0414740 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c67ccd0 0x55ea0c6acd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7d0414740 4 client.5541 mds_command: new command op to 5331 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3452871827}] 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 --> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3452871827}) -- 0x55ea0c5c2380 con 0x55ea0c67ccd0 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.364+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c67ccd0 0x55ea0c6acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.365+0000 7fd7c1e00640 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c67ccd0 0x55ea0c6acd80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d22874717612c4 server_cookie=bf2a6d8678524da8 in_seq=0 out_seq=0 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.365+0000 7fd7a3400640 10 client.5541 ms_handle_connect on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:47.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.365+0000 7fd7a3400640 1 -- 172.21.15.19:0/1678577656 <== mds.0 v2:172.21.15.19:6833/3972745609 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x55ea0c5c2380 con 0x55ea0c67ccd0 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.365+0000 7fd7a3400640 10 client.5541 handle_command_reply: tid=0 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.365+0000 7fd7d0414740 1 client.5541 shutdown 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.365+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 msgr2=0x55ea0c7185d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.365+0000 7fd7d0414740 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c7185d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7b402e040 tx=0x7fd7b40046c0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 shutdown_connections 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:6833/3972745609,v1:172.21.15.19:6835/3972745609] conn(0x55ea0c67ccd0 0x55ea0c6acd80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea0c2935c0 0x55ea0c6bd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea0c2b42a0 0x55ea0c655730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7a3400640 0 client.5541 ms_handle_reset on v2:172.21.15.19:6833/3972745609 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 --2- 172.21.15.19:0/1678577656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea0c28b3e0 0x55ea0c7185d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 >> 172.21.15.19:0/1678577656 conn(0x55ea0c2a83f0 msgr2=0x55ea0c654470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 shutdown_connections 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 1 -- 172.21.15.19:0/1678577656 wait complete. 2024-04-25T11:29:47.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.366+0000 7fd7d0414740 20 client.5541 trim_cache size 0 max 16384 2024-04-25T11:29:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7a40744b0 msgr2=0x7fd7a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7a40744b0 0x7fd7a40769c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009af0 tx=0x7fd7b4008040 comp rx=0 tx=0).stop 2024-04-25T11:29:47.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c40b1c70 msgr2=0x7fd7c40cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c40b1c70 0x7fd7c40cbda0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009e50 tx=0x7fd7bc004a40 comp rx=0 tx=0).stop 2024-04-25T11:29:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 shutdown_connections 2024-04-25T11:29:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7a40744b0 0x7fd7a40769c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd7c406dc80 0x7fd7c40d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7c40b1c70 0x7fd7c40cbda0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 --2- 172.21.15.19:0/887815918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7c406d870 0x7fd7c409c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 >> 172.21.15.19:0/887815918 conn(0x7fd7c4073a30 msgr2=0x7fd7c40b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 shutdown_connections 2024-04-25T11:29:47.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47.367+0000 7fd7c8a00640 1 -- 172.21.15.19:0/887815918 wait complete. 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> import errno 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> f.close() 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> except IOError as e: 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(errno.EIO) 2024-04-25T11:29:47.383 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:29:47.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:29:47.554 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:29:47.555 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:29:47.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:29:47.833 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:29:47.833 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:29:47.834 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:29:47.834 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:47.895 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:29:47.896 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:47.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:29:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.256+0000 7f7de5800640 1 Processor -- start 2024-04-25T11:29:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.256+0000 7f7de5800640 1 -- start start 2024-04-25T11:29:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.258+0000 7f7de5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de00b4ea0 0x7f7de00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.258+0000 7f7de5800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de00d1230 con 0x7f7de0144370 2024-04-25T11:29:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.258+0000 7f7de5800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de00d06b0 con 0x7f7de006c5c0 2024-04-25T11:29:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.258+0000 7f7de5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7de00d0990 con 0x7f7de00b4ea0 2024-04-25T11:29:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.258+0000 7f7de4e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7de0144370 0x7f7de015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35840/0 (socket says 172.21.15.19:35840) 2024-04-25T11:29:48.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.258+0000 7f7de4e00640 1 -- 172.21.15.19:0/3959604830 learned_addr learned my addr 172.21.15.19:0/3959604830 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:48.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.258+0000 7f7dd7e00640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de00b4ea0 0x7f7de00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782274649 0 0) 0x7f7de00d1230 con 0x7f7de0144370 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dc4003520 con 0x7f7de0144370 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1154782654 0 0) 0x7f7dc4003520 con 0x7f7de0144370 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de00b4ea0 msgr2=0x7f7de00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de00b4ea0 0x7f7de00cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 >> v1:172.21.15.169:6789/0 conn(0x7f7de006c5c0 legacy=0x7f7de00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de00d0c70 con 0x7f7de0144370 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7dd80019f0 con 0x7f7de0144370 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 >> v1:172.21.15.19:6789/0 conn(0x7f7de0144370 legacy=0x7f7de015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dc40039e0 0x7f7dc4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc4004800 0x7f7dc4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dc40051a0 0x7f7dc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7de00d0990 con 0x7f7dc40039e0 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc400bf00 con 0x7f7dc4004800 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.259+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc400c100 con 0x7f7dc40051a0 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dc40039e0 0x7f7dc4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7de4e00640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dc40051a0 0x7f7dc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7dd7e00640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc4004800 0x7f7dc4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7ddfe00640 1 -- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dc40051a0 msgr2=0x7f7dc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7dc40051a0 0x7f7dc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7ddfe00640 1 -- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc4004800 msgr2=0x7f7dc4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc4004800 0x7f7dc4004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.260+0000 7f7ddfe00640 1 -- 172.21.15.19:0/3959604830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc4003520 con 0x7f7dc40039e0 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.261+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dc40039e0 0x7f7dc4003e50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0001720 tx=0x7f7dc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c4f24a46eb9a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.261+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0001370 con 0x7f7dc40039e0 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.262+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd00052d0 con 0x7f7dc40039e0 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.262+0000 7f7ddf400640 1 -- 172.21.15.19:0/3959604830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002abc0 con 0x7f7dc40039e0 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.262+0000 7f7de5800640 1 -- 172.21.15.19:0/3959604830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dc40039e0 msgr2=0x7f7dc4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.262+0000 7f7de5800640 1 --2- 172.21.15.19:0/3959604830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7dc40039e0 0x7f7dc4003e50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0001720 tx=0x7f7dc4005830 comp rx=0 tx=0).stop 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.262+0000 7f7de4e00640 1 -- 172.21.15.19:0/3959604830 reap_dead start 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.262+0000 7f7de5800640 1 -- 172.21.15.19:0/3959604830 shutdown_connections 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.262+0000 7f7de5800640 1 -- 172.21.15.19:0/3959604830 >> 172.21.15.19:0/3959604830 conn(0x7f7de00d1720 msgr2=0x7f7de00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.263+0000 7f7de5800640 1 -- 172.21.15.19:0/3959604830 shutdown_connections 2024-04-25T11:29:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.263+0000 7f7de5800640 1 -- 172.21.15.19:0/3959604830 wait complete. 2024-04-25T11:29:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.265+0000 7f7de5800640 1 Processor -- start 2024-04-25T11:29:48.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.265+0000 7f7de5800640 1 -- start start 2024-04-25T11:29:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7de5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc40039e0 0x7f7de00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7de5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7de0162320 0x7f7de013fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7de5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de0140180 0x7f7de006f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7de5800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7de00d0450 con 0x7f7de0162320 2024-04-25T11:29:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7de5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7de00d02d0 con 0x7f7dc40039e0 2024-04-25T11:29:48.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7de5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7de0147760 con 0x7f7de0140180 2024-04-25T11:29:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7dd7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7de0162320 0x7f7de013fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7dd7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7de0162320 0x7f7de013fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39036/0 (socket says 172.21.15.19:39036) 2024-04-25T11:29:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7dd7e00640 1 -- 172.21.15.19:0/2775230049 learned_addr learned my addr 172.21.15.19:0/2775230049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7de4e00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de0140180 0x7f7de006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.267+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc40039e0 0x7f7de00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7dd7e00640 1 -- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de0140180 msgr2=0x7f7de006f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7dd7e00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de0140180 0x7f7de006f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7dd7e00640 1 -- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc40039e0 msgr2=0x7f7de00cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7dd7e00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc40039e0 0x7f7de00cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7dd7e00640 1 -- 172.21.15.19:0/2775230049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de0140590 con 0x7f7de0162320 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7de4e00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de0140180 0x7f7de006f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc40039e0 0x7f7de00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7dd7e00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7de0162320 0x7f7de013fc40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8010890 tx=0x7f7dc8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7ddcc00640 1 -- 172.21.15.19:0/2775230049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800f040 con 0x7f7de0162320 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7ddcc00640 1 -- 172.21.15.19:0/2775230049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc800eb60 con 0x7f7de0162320 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de00cb050 con 0x7f7de0162320 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7ddcc00640 1 -- 172.21.15.19:0/2775230049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc801c400 con 0x7f7de0162320 2024-04-25T11:29:48.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.268+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de00cb7b0 con 0x7f7de0162320 2024-04-25T11:29:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.270+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de00cb220 con 0x7f7de0162320 2024-04-25T11:29:48.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.271+0000 7f7ddcc00640 1 -- 172.21.15.19:0/2775230049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7dc801c5a0 con 0x7f7de0162320 2024-04-25T11:29:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.272+0000 7f7ddcc00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dbc0744e0 0x7f7dbc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.272+0000 7f7ddcc00640 1 -- 172.21.15.19:0/2775230049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f7dc80958b0 con 0x7f7de0162320 2024-04-25T11:29:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.273+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dbc0744e0 0x7f7dbc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.274+0000 7f7ddfe00640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dbc0744e0 0x7f7dbc0769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0032000 tx=0x7f7dd002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.277+0000 7f7ddcc00640 1 -- 172.21.15.19:0/2775230049 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7de0140590 con 0x7f7de0162320 2024-04-25T11:29:48.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.481+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7de00cc1b0 con 0x7f7de0162320 2024-04-25T11:29:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.483+0000 7f7ddcc00640 1 -- 172.21.15.19:0/2775230049 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16987 (secure 0 0 0) 0x7f7dc8018030 con 0x7f7de0162320 2024-04-25T11:29:48.475 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:48.475 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:29:06.584671+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3831883468":"2024-04-26T11:29:00.304477+0000","172.21.15.19:6833/3831883468":"2024-04-26T11:29:00.304477+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:29:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.485+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dbc0744e0 msgr2=0x7f7dbc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.485+0000 7f7de5800640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dbc0744e0 0x7f7dbc0769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0032000 tx=0x7f7dd002e000 comp rx=0 tx=0).stop 2024-04-25T11:29:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.485+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7de0162320 msgr2=0x7f7de013fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.485+0000 7f7de5800640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7de0162320 0x7f7de013fc40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8010890 tx=0x7f7dc8010d50 comp rx=0 tx=0).stop 2024-04-25T11:29:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 shutdown_connections 2024-04-25T11:29:48.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dbc0744e0 0x7f7dbc0769f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7de0140180 0x7f7de006f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7de0162320 0x7f7de013fc40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 --2- 172.21.15.19:0/2775230049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7dc40039e0 0x7f7de00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 >> 172.21.15.19:0/2775230049 conn(0x7f7de00d1720 msgr2=0x7f7de015f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 shutdown_connections 2024-04-25T11:29:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.486+0000 7f7de5800640 1 -- 172.21.15.19:0/2775230049 wait complete. 2024-04-25T11:29:48.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:28:43.580410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:28:44.810835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:29:48.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:29:48.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.804+0000 7fec8f200640 1 Processor -- start 2024-04-25T11:29:48.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.804+0000 7fec8f200640 1 -- start start 2024-04-25T11:29:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.806+0000 7fec8f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec900cf870 0x7fec900d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.806+0000 7fec8f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 0x7fec90142130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.806+0000 7fec8f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec90142670 con 0x7fec9015d620 2024-04-25T11:29:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.806+0000 7fec8f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fec900ce470 con 0x7fec900cf870 2024-04-25T11:29:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.806+0000 7fec8f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec900cfc80 con 0x7fec90141d20 2024-04-25T11:29:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.807+0000 7fec8d400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fec9015d620 0x7fec900cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35848/0 (socket says 172.21.15.19:35848) 2024-04-25T11:29:48.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.807+0000 7fec8d400640 1 -- 172.21.15.19:0/2009343178 learned_addr learned my addr 172.21.15.19:0/2009343178 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.807+0000 7fec8e800640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 0x7fec90142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.807+0000 7fec8de00640 1 --2- 172.21.15.19:0/2009343178 >> v2:172.21.15.169:3300/0 conn(0x7fec900cf870 0x7fec900d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.807+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752751774 0 0) 0x7fec78001110 con 0x7fec9015d620 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.807+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec74003520 con 0x7fec9015d620 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4102982605 0 0) 0x7fec74003520 con 0x7fec9015d620 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 msgr2=0x7fec90142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 0x7fec90142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 >> v2:172.21.15.169:3300/0 conn(0x7fec900cf870 msgr2=0x7fec900d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 --2- 172.21.15.19:0/2009343178 >> v2:172.21.15.169:3300/0 conn(0x7fec900cf870 0x7fec900d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec900cff10 con 0x7fec9015d620 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fec78001840 con 0x7fec9015d620 2024-04-25T11:29:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 >> v1:172.21.15.19:6789/0 conn(0x7fec9015d620 legacy=0x7fec900cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec740039e0 0x7fec74003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 0x7fec74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec74004d40 0x7fec74005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec900ce470 con 0x7fec74004d40 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec900cfc80 con 0x7fec740039e0 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec7400ba70 con 0x7fec90141d20 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8e800640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec74004d40 0x7fec74005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.808+0000 7fec8d400640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 0x7fec74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.809+0000 7fec8de00640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec740039e0 0x7fec74003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.809+0000 7fec8e800640 1 -- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 msgr2=0x7fec74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.809+0000 7fec8e800640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec90141d20 0x7fec74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.809+0000 7fec8e800640 1 -- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec740039e0 msgr2=0x7fec74003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.809+0000 7fec8e800640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec740039e0 0x7fec74003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.809+0000 7fec8e800640 1 -- 172.21.15.19:0/2009343178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec74003520 con 0x7fec74004d40 2024-04-25T11:29:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.809+0000 7fec8de00640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec740039e0 0x7fec74003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.810+0000 7fec8e800640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec74004d40 0x7fec74005150 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fec88009f90 tx=0x7fec74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3c8bf4791f5550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.810+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88009c10 con 0x7fec74004d40 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.810+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88036070 con 0x7fec74004d40 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.810+0000 7fec8ca00640 1 -- 172.21.15.19:0/2009343178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88031850 con 0x7fec74004d40 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.811+0000 7fec8f200640 1 -- 172.21.15.19:0/2009343178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec74004d40 msgr2=0x7fec74005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.811+0000 7fec8f200640 1 --2- 172.21.15.19:0/2009343178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec74004d40 0x7fec74005150 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fec88009f90 tx=0x7fec74005830 comp rx=0 tx=0).stop 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.811+0000 7fec8e800640 1 -- 172.21.15.19:0/2009343178 reap_dead start 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.811+0000 7fec8f200640 1 -- 172.21.15.19:0/2009343178 shutdown_connections 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.811+0000 7fec8f200640 1 -- 172.21.15.19:0/2009343178 >> 172.21.15.19:0/2009343178 conn(0x7fec90072f40 msgr2=0x7fec90073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.811+0000 7fec8f200640 1 -- 172.21.15.19:0/2009343178 shutdown_connections 2024-04-25T11:29:48.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.811+0000 7fec8f200640 1 -- 172.21.15.19:0/2009343178 wait complete. 2024-04-25T11:29:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.813+0000 7fec8f200640 1 Processor -- start 2024-04-25T11:29:48.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.814+0000 7fec8f200640 1 -- start start 2024-04-25T11:29:48.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74004d40 0x7fec9013ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec90152310 0x7fec90151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9013f170 0x7fec90151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec900d07b0 con 0x7fec90152310 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec900d04d0 con 0x7fec9013f170 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec900d0a90 con 0x7fec74004d40 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec90152310 0x7fec90151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.815+0000 7fec8d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec90152310 0x7fec90151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39056/0 (socket says 172.21.15.19:39056) 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8d400640 1 -- 172.21.15.19:0/3939789599 learned_addr learned my addr 172.21.15.19:0/3939789599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8de00640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74004d40 0x7fec9013ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8d400640 1 -- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74004d40 msgr2=0x7fec9013ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8d400640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74004d40 0x7fec9013ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8d400640 1 -- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9013f170 msgr2=0x7fec90151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8e800640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9013f170 0x7fec90151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8d400640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9013f170 0x7fec90151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8d400640 1 -- 172.21.15.19:0/3939789599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec900cc020 con 0x7fec90152310 2024-04-25T11:29:48.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.816+0000 7fec8d400640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec90152310 0x7fec90151770 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fec7800e780 tx=0x7fec7800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.817+0000 7fec86000640 1 -- 172.21.15.19:0/3939789599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7800c660 con 0x7fec90152310 2024-04-25T11:29:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.817+0000 7fec86000640 1 -- 172.21.15.19:0/3939789599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec78018070 con 0x7fec90152310 2024-04-25T11:29:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.817+0000 7fec86000640 1 -- 172.21.15.19:0/3939789599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7800ca60 con 0x7fec90152310 2024-04-25T11:29:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.817+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec900cc300 con 0x7fec90152310 2024-04-25T11:29:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.817+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec90153e10 con 0x7fec90152310 2024-04-25T11:29:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.818+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec90076c60 con 0x7fec90152310 2024-04-25T11:29:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.819+0000 7fec86000640 1 -- 172.21.15.19:0/3939789599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec7800cc00 con 0x7fec90152310 2024-04-25T11:29:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.820+0000 7fec86000640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec6c0744e0 0x7fec6c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.820+0000 7fec86000640 1 -- 172.21.15.19:0/3939789599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fec78094ee0 con 0x7fec90152310 2024-04-25T11:29:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.821+0000 7fec8de00640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec6c0744e0 0x7fec6c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.822+0000 7fec8de00640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec6c0744e0 0x7fec6c0769f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fec90073db0 tx=0x7fec80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:48.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:48.825+0000 7fec86000640 1 -- 172.21.15.19:0/3939789599 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec90142670 con 0x7fec90152310 2024-04-25T11:29:49.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.035+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec900b4c80 con 0x7fec90152310 2024-04-25T11:29:49.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.036+0000 7fec86000640 1 -- 172.21.15.19:0/3939789599 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7fec7805f500 con 0x7fec90152310 2024-04-25T11:29:49.027 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:29:49.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec6c0744e0 msgr2=0x7fec6c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec6c0744e0 0x7fec6c0769f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fec90073db0 tx=0x7fec80008040 comp rx=0 tx=0).stop 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec90152310 msgr2=0x7fec90151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec90152310 0x7fec90151770 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fec7800e780 tx=0x7fec7800ec40 comp rx=0 tx=0).stop 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 shutdown_connections 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec6c0744e0 0x7fec6c0769f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec9013f170 0x7fec90151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec90152310 0x7fec90151770 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 --2- 172.21.15.19:0/3939789599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec74004d40 0x7fec9013ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 >> 172.21.15.19:0/3939789599 conn(0x7fec90072f40 msgr2=0x7fec9015bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 shutdown_connections 2024-04-25T11:29:49.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.039+0000 7fec8f200640 1 -- 172.21.15.19:0/3939789599 wait complete. 2024-04-25T11:29:49.059 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:29:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.354+0000 7f5073e00640 1 Processor -- start 2024-04-25T11:29:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.355+0000 7f5073e00640 1 -- start start 2024-04-25T11:29:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.357+0000 7f5073e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 0x7f507406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.357+0000 7f5073e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f507406df20 con 0x7f50740ce520 2024-04-25T11:29:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.357+0000 7f5073e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50740cda60 con 0x7f507406db10 2024-04-25T11:29:49.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.357+0000 7f5073e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f507415dfe0 con 0x7f50740ce930 2024-04-25T11:29:49.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.358+0000 7f5072a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f50740ce520 0x7f507415d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35862/0 (socket says 172.21.15.19:35862) 2024-04-25T11:29:49.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.358+0000 7f5072a00640 1 -- 172.21.15.19:0/693912926 learned_addr learned my addr 172.21.15.19:0/693912926 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.358+0000 7f506be00640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 0x7f507406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.358+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464859478 0 0) 0x7f5064001110 con 0x7f50740ce520 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.358+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5058005730 con 0x7f50740ce520 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4034602475 0 0) 0x7f5058005730 con 0x7f50740ce520 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 msgr2=0x7f507406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 0x7f507406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 >> v1:172.21.15.169:6789/0 conn(0x7f507406db10 legacy=0x7f50740d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507415e2c0 con 0x7f50740ce520 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f5064001840 con 0x7f50740ce520 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 >> v1:172.21.15.19:6789/0 conn(0x7f50740ce520 legacy=0x7f507415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5058005bf0 0x7f5058006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 0x7f5058006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f5058007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f507415dfe0 con 0x7f5058006f50 2024-04-25T11:29:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f505800dd90 con 0x7f5058005bf0 2024-04-25T11:29:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f505800df90 con 0x7f50740ce930 2024-04-25T11:29:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f5073400640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f5058007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.359+0000 7f506be00640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 0x7f5058006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.360+0000 7f5072a00640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5058005bf0 0x7f5058006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.360+0000 7f5073400640 1 -- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 msgr2=0x7f5058006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.360+0000 7f5073400640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50740ce930 0x7f5058006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.360+0000 7f5073400640 1 -- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5058005bf0 msgr2=0x7f5058006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.360+0000 7f5073400640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5058005bf0 0x7f5058006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.360+0000 7f5073400640 1 -- 172.21.15.19:0/693912926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5058005730 con 0x7f5058006f50 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.360+0000 7f5072a00640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5058005bf0 0x7f5058006060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.361+0000 7f5073400640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f5058007360 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f506c001e20 tx=0x7f5058007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d262af245029b658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.362+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c001a70 con 0x7f5058006f50 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.362+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c02ae40 con 0x7f5058006f50 2024-04-25T11:29:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.362+0000 7f5072000640 1 -- 172.21.15.19:0/693912926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c003bb0 con 0x7f5058006f50 2024-04-25T11:29:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.363+0000 7f5073e00640 1 -- 172.21.15.19:0/693912926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 msgr2=0x7f5058007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.363+0000 7f5073e00640 1 --2- 172.21.15.19:0/693912926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f5058007360 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f506c001e20 tx=0x7f5058007a40 comp rx=0 tx=0).stop 2024-04-25T11:29:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.363+0000 7f5073400640 1 -- 172.21.15.19:0/693912926 reap_dead start 2024-04-25T11:29:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.363+0000 7f5073e00640 1 -- 172.21.15.19:0/693912926 shutdown_connections 2024-04-25T11:29:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.363+0000 7f5073e00640 1 -- 172.21.15.19:0/693912926 >> 172.21.15.19:0/693912926 conn(0x7f5074072f40 msgr2=0x7f5074073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.363+0000 7f5073e00640 1 -- 172.21.15.19:0/693912926 shutdown_connections 2024-04-25T11:29:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.363+0000 7f5073e00640 1 -- 172.21.15.19:0/693912926 wait complete. 2024-04-25T11:29:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.365+0000 7f5073e00640 1 Processor -- start 2024-04-25T11:29:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.365+0000 7f5073e00640 1 -- start start 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5073e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f50741453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5073e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406f750 0x7f50741e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5073e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50741e78c0 0x7f507406fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5073e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5074160d30 con 0x7f5058006f50 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5073e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50740ce000 con 0x7f507406f750 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5073e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f507415dbc0 con 0x7f50741e78c0 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5072a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f50741453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5072a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f50741453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39066/0 (socket says 172.21.15.19:39066) 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.367+0000 7f5072a00640 1 -- 172.21.15.19:0/3251972296 learned_addr learned my addr 172.21.15.19:0/3251972296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f506be00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406f750 0x7f50741e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f5073400640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50741e78c0 0x7f507406fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f5072a00640 1 -- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50741e78c0 msgr2=0x7f507406fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f5072a00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50741e78c0 0x7f507406fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f5072a00640 1 -- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406f750 msgr2=0x7f50741e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f5072a00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406f750 0x7f50741e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f5072a00640 1 -- 172.21.15.19:0/3251972296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5074160410 con 0x7f5058006f50 2024-04-25T11:29:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.368+0000 7f5072a00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f50741453e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5064007300 tx=0x7f50640077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.369+0000 7f506aa00640 1 -- 172.21.15.19:0/3251972296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064005e60 con 0x7f5058006f50 2024-04-25T11:29:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.369+0000 7f506aa00640 1 -- 172.21.15.19:0/3251972296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5064018070 con 0x7f5058006f50 2024-04-25T11:29:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.369+0000 7f506aa00640 1 -- 172.21.15.19:0/3251972296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506400ea70 con 0x7f5058006f50 2024-04-25T11:29:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.369+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5074152690 con 0x7f5058006f50 2024-04-25T11:29:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.369+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507415fb70 con 0x7f5058006f50 2024-04-25T11:29:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.370+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5074076c60 con 0x7f5058006f50 2024-04-25T11:29:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.371+0000 7f506aa00640 1 -- 172.21.15.19:0/3251972296 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f506400ec10 con 0x7f5058006f50 2024-04-25T11:29:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.372+0000 7f506aa00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50500744e0 0x7f50500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.372+0000 7f506aa00640 1 -- 172.21.15.19:0/3251972296 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f5064095380 con 0x7f5058006f50 2024-04-25T11:29:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.373+0000 7f506be00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50500744e0 0x7f50500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.374+0000 7f506be00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50500744e0 0x7f50500769f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f507406f2d0 tx=0x7f505c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:49.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.378+0000 7f506aa00640 1 -- 172.21.15.19:0/3251972296 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f507406df20 con 0x7f5058006f50 2024-04-25T11:29:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.585+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50740b5480 con 0x7f5058006f50 2024-04-25T11:29:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.586+0000 7f506aa00640 1 -- 172.21.15.19:0/3251972296 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f506405f9a0 con 0x7f5058006f50 2024-04-25T11:29:49.580 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:29:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50500744e0 msgr2=0x7f50500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50500744e0 0x7f50500769f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f507406f2d0 tx=0x7f505c008040 comp rx=0 tx=0).stop 2024-04-25T11:29:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 msgr2=0x7f50741453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f50741453e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5064007300 tx=0x7f50640077c0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 shutdown_connections 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50500744e0 0x7f50500769f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f507406f750 0x7f50741e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50741e78c0 0x7f507406fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 --2- 172.21.15.19:0/3251972296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5058006f50 0x7f50741453e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 >> 172.21.15.19:0/3251972296 conn(0x7f5074072f40 msgr2=0x7f507406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 shutdown_connections 2024-04-25T11:29:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.589+0000 7f5073e00640 1 -- 172.21.15.19:0/3251972296 wait complete. 2024-04-25T11:29:49.605 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:29:49.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.906+0000 7f8f19600640 1 Processor -- start 2024-04-25T11:29:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.906+0000 7f8f19600640 1 -- start start 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.908+0000 7f8f19600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f1406f930 0x7f8f1406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.908+0000 7f8f19600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f1406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.908+0000 7f8f19600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f141623f0 con 0x7f8f14070d20 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.908+0000 7f8f19600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8f141626d0 con 0x7f8f1406f930 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.908+0000 7f8f19600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1406f130 con 0x7f8f1406fd40 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f13400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f1406f930 0x7f8f1406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f13400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f1406f930 0x7f8f1406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39074/0 (socket says 172.21.15.19:39074) 2024-04-25T11:29:49.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f13400640 1 -- 172.21.15.19:0/1050853133 learned_addr learned my addr 172.21.15.19:0/1050853133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:49.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f13400640 1 -- 172.21.15.19:0/1050853133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 msgr2=0x7f8f1406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:49.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f18c00640 1 --2- 172.21.15.19:0/1050853133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f1406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f13400640 1 --2- 172.21.15.19:0/1050853133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f1406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f13400640 1 -- 172.21.15.19:0/1050853133 >> v1:172.21.15.169:6789/0 conn(0x7f8f14070d20 legacy=0x7f8f14161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:49.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f13400640 1 -- 172.21.15.19:0/1050853133 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1406f410 con 0x7f8f1406f930 2024-04-25T11:29:49.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.909+0000 7f8f18c00640 1 --2- 172.21.15.19:0/1050853133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f1406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:49.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.910+0000 7f8f13400640 1 --2- 172.21.15.19:0/1050853133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1406f930 0x7f8f1406d5d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009870 tx=0x7f8f0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aab67002b42cdb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.911+0000 7f8f12a00640 1 -- 172.21.15.19:0/1050853133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00037070 con 0x7f8f1406f930 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.911+0000 7f8f12a00640 1 -- 172.21.15.19:0/1050853133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f00032440 con 0x7f8f1406f930 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.911+0000 7f8f12a00640 1 -- 172.21.15.19:0/1050853133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00032810 con 0x7f8f1406f930 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.911+0000 7f8f19600640 1 -- 172.21.15.19:0/1050853133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1406f930 msgr2=0x7f8f1406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.911+0000 7f8f19600640 1 --2- 172.21.15.19:0/1050853133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1406f930 0x7f8f1406d5d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009870 tx=0x7f8f0002a180 comp rx=0 tx=0).stop 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.912+0000 7f8f19600640 1 -- 172.21.15.19:0/1050853133 shutdown_connections 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.912+0000 7f8f19600640 1 --2- 172.21.15.19:0/1050853133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f1406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.912+0000 7f8f19600640 1 --2- 172.21.15.19:0/1050853133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f1406f930 0x7f8f1406d5d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.912+0000 7f8f19600640 1 -- 172.21.15.19:0/1050853133 >> 172.21.15.19:0/1050853133 conn(0x7f8f14073160 msgr2=0x7f8f14073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.912+0000 7f8f19600640 1 -- 172.21.15.19:0/1050853133 shutdown_connections 2024-04-25T11:29:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.912+0000 7f8f19600640 1 -- 172.21.15.19:0/1050853133 wait complete. 2024-04-25T11:29:49.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.914+0000 7f8f19600640 1 Processor -- start 2024-04-25T11:29:49.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.914+0000 7f8f19600640 1 -- start start 2024-04-25T11:29:49.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f19600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1406f930 0x7f8f14160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f19600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f141ea260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f19600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f14070d20 0x7f8f141f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f19600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1415e470 con 0x7f8f14070d20 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f19600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1415e2f0 con 0x7f8f1406f930 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f19600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f140ce7e0 con 0x7f8f1406fd40 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f18c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f14070d20 0x7f8f141f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f18c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f14070d20 0x7f8f141f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39088/0 (socket says 172.21.15.19:39088) 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.916+0000 7f8f18c00640 1 -- 172.21.15.19:0/4022900746 learned_addr learned my addr 172.21.15.19:0/4022900746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f13400640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f141ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f13e00640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1406f930 0x7f8f14160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f18c00640 1 -- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 msgr2=0x7f8f141ea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f18c00640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f141ea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f18c00640 1 -- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1406f930 msgr2=0x7f8f14160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f18c00640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1406f930 0x7f8f14160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f18c00640 1 -- 172.21.15.19:0/4022900746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f141e8720 con 0x7f8f14070d20 2024-04-25T11:29:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f18c00640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f14070d20 0x7f8f141f9c00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c5a0 tx=0x7f8f0c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.917+0000 7f8f0be00640 1 -- 172.21.15.19:0/4022900746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c015070 con 0x7f8f14070d20 2024-04-25T11:29:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.918+0000 7f8f0be00640 1 -- 172.21.15.19:0/4022900746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c011820 con 0x7f8f14070d20 2024-04-25T11:29:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.918+0000 7f8f0be00640 1 -- 172.21.15.19:0/4022900746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c011c10 con 0x7f8f14070d20 2024-04-25T11:29:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.918+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f141e8a00 con 0x7f8f14070d20 2024-04-25T11:29:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.918+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f141e8fc0 con 0x7f8f14070d20 2024-04-25T11:29:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.919+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f14068c30 con 0x7f8f14070d20 2024-04-25T11:29:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.920+0000 7f8f0be00640 1 -- 172.21.15.19:0/4022900746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f0c018070 con 0x7f8f14070d20 2024-04-25T11:29:49.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.921+0000 7f8f0be00640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ef40744b0 0x7f8ef40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:49.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.921+0000 7f8f0be00640 1 -- 172.21.15.19:0/4022900746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f8f0c095190 con 0x7f8f14070d20 2024-04-25T11:29:49.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.922+0000 7f8f13e00640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ef40744b0 0x7f8ef40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:49.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.923+0000 7f8f13e00640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ef40744b0 0x7f8ef40769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8f140b1350 tx=0x7f8f040094d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:49.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:49.926+0000 7f8f0be00640 1 -- 172.21.15.19:0/4022900746 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f14068c30 con 0x7f8f14070d20 2024-04-25T11:29:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.132+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f14076c60 con 0x7f8f14070d20 2024-04-25T11:29:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.133+0000 7f8f0be00640 1 -- 172.21.15.19:0/4022900746 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f8f0c05f840 con 0x7f8f14070d20 2024-04-25T11:29:50.124 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.135+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ef40744b0 msgr2=0x7f8ef40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.135+0000 7f8f19600640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ef40744b0 0x7f8ef40769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8f140b1350 tx=0x7f8f040094d0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.135+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f14070d20 msgr2=0x7f8f141f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f14070d20 0x7f8f141f9c00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c5a0 tx=0x7f8f0c00ca60 comp rx=0 tx=0).stop 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 shutdown_connections 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ef40744b0 0x7f8ef40769c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f14070d20 0x7f8f141f9c00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f1406fd40 0x7f8f141ea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 --2- 172.21.15.19:0/4022900746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f1406f930 0x7f8f14160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 >> 172.21.15.19:0/4022900746 conn(0x7f8f14073160 msgr2=0x7f8f1415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:50.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 shutdown_connections 2024-04-25T11:29:50.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.136+0000 7f8f19600640 1 -- 172.21.15.19:0/4022900746 wait complete. 2024-04-25T11:29:50.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:29:50.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.454+0000 7f89e7e00640 1 Processor -- start 2024-04-25T11:29:50.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.454+0000 7f89e7e00640 1 -- start start 2024-04-25T11:29:50.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89e80d2cb0 0x7f89e8160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:50.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f89e8160b90 0x7f89e8161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89e80b7940 con 0x7f89e80d50e0 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f89e8161280 con 0x7f89e80d2cb0 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f89e80d4600 con 0x7f89e8160b90 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89e80d2cb0 0x7f89e8160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89e80d2cb0 0x7f89e8160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39100/0 (socket says 172.21.15.19:39100) 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e6000640 1 -- 172.21.15.19:0/3398784042 learned_addr learned my addr 172.21.15.19:0/3398784042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.457+0000 7f89e7400640 1 --2- 172.21.15.19:0/3398784042 >> v2:172.21.15.169:3300/0 conn(0x7f89e8160b90 0x7f89e8161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.458+0000 7f89e6000640 1 -- 172.21.15.19:0/3398784042 >> v2:172.21.15.169:3300/0 conn(0x7f89e8160b90 msgr2=0x7f89e8161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.458+0000 7f89e6000640 1 --2- 172.21.15.19:0/3398784042 >> v2:172.21.15.169:3300/0 conn(0x7f89e8160b90 0x7f89e8161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.458+0000 7f89e6000640 1 -- 172.21.15.19:0/3398784042 >> v1:172.21.15.169:6789/0 conn(0x7f89e80d50e0 legacy=0x7f89e80d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:50.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.458+0000 7f89e6000640 1 -- 172.21.15.19:0/3398784042 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e80d48e0 con 0x7f89e80d2cb0 2024-04-25T11:29:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.459+0000 7f89e6000640 1 --2- 172.21.15.19:0/3398784042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e80d2cb0 0x7f89e8160650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f89e80d3710 tx=0x7f89d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386505be7f24e1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.459+0000 7f89e5600640 1 -- 172.21.15.19:0/3398784042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0036070 con 0x7f89e80d2cb0 2024-04-25T11:29:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.459+0000 7f89e5600640 1 -- 172.21.15.19:0/3398784042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d0031600 con 0x7f89e80d2cb0 2024-04-25T11:29:50.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.459+0000 7f89e5600640 1 -- 172.21.15.19:0/3398784042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d00319d0 con 0x7f89e80d2cb0 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 -- 172.21.15.19:0/3398784042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e80d2cb0 msgr2=0x7f89e8160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3398784042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e80d2cb0 0x7f89e8160650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f89e80d3710 tx=0x7f89d002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 -- 172.21.15.19:0/3398784042 shutdown_connections 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3398784042 >> v2:172.21.15.169:3300/0 conn(0x7f89e8160b90 0x7f89e8161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3398784042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e80d2cb0 0x7f89e8160650 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 -- 172.21.15.19:0/3398784042 >> 172.21.15.19:0/3398784042 conn(0x7f89e806c5c0 msgr2=0x7f89e806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 -- 172.21.15.19:0/3398784042 shutdown_connections 2024-04-25T11:29:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.460+0000 7f89e7e00640 1 -- 172.21.15.19:0/3398784042 wait complete. 2024-04-25T11:29:50.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.463+0000 7f89e7e00640 1 Processor -- start 2024-04-25T11:29:50.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.463+0000 7f89e7e00640 1 -- start start 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89e80d2cb0 0x7f89e814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89e80d50e0 0x7f89e814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e8160b90 0x7f89e814f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8161560 con 0x7f89e8160b90 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8160fa0 con 0x7f89e80d2cb0 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f89e80d54f0 con 0x7f89e80d50e0 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e8160b90 0x7f89e814f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e8160b90 0x7f89e814f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39108/0 (socket says 172.21.15.19:39108) 2024-04-25T11:29:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 -- 172.21.15.19:0/3331545257 learned_addr learned my addr 172.21.15.19:0/3331545257 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e6000640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89e80d50e0 0x7f89e814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e6a00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89e80d2cb0 0x7f89e814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 -- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89e80d50e0 msgr2=0x7f89e814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89e80d50e0 0x7f89e814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 -- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89e80d2cb0 msgr2=0x7f89e814ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89e80d2cb0 0x7f89e814ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.465+0000 7f89e7400640 1 -- 172.21.15.19:0/3331545257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e80cdd40 con 0x7f89e8160b90 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.466+0000 7f89e7400640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e8160b90 0x7f89e814f4b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f89e000d600 tx=0x7f89e000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.466+0000 7f89dea00640 1 -- 172.21.15.19:0/3331545257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e00125f0 con 0x7f89e8160b90 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.466+0000 7f89dea00640 1 -- 172.21.15.19:0/3331545257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e0019070 con 0x7f89e8160b90 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.466+0000 7f89dea00640 1 -- 172.21.15.19:0/3331545257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e00129f0 con 0x7f89e8160b90 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.466+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e80ce020 con 0x7f89e8160b90 2024-04-25T11:29:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.466+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e814e510 con 0x7f89e8160b90 2024-04-25T11:29:50.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.467+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e80cdab0 con 0x7f89e8160b90 2024-04-25T11:29:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.469+0000 7f89dea00640 1 -- 172.21.15.19:0/3331545257 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f89e0012b90 con 0x7f89e8160b90 2024-04-25T11:29:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.470+0000 7f89dea00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f89c40744e0 0x7f89c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.470+0000 7f89dea00640 1 -- 172.21.15.19:0/3331545257 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f89e0095770 con 0x7f89e8160b90 2024-04-25T11:29:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.471+0000 7f89e6a00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f89c40744e0 0x7f89c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.472+0000 7f89e6a00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f89c40744e0 0x7f89c40769f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89d800a750 tx=0x7f89d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:50.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.475+0000 7f89dea00640 1 -- 172.21.15.19:0/3331545257 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89e005fda0 con 0x7f89e8160b90 2024-04-25T11:29:50.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.689+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f89e814e020 con 0x7f89e8160b90 2024-04-25T11:29:50.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.690+0000 7f89dea00640 1 -- 172.21.15.19:0/3331545257 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v38) ==== 80+0+122 (secure 0 0 0) 0x7f89e005f6a0 con 0x7f89e8160b90 2024-04-25T11:29:50.681 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3831883468 2024-04-26T11:29:00.304477+0000 2024-04-25T11:29:50.681 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6833/3831883468 2024-04-26T11:29:00.304477+0000 2024-04-25T11:29:50.681 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.692+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f89c40744e0 msgr2=0x7f89c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.692+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f89c40744e0 0x7f89c40769f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f89d800a750 tx=0x7f89d8008040 comp rx=0 tx=0).stop 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.692+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e8160b90 msgr2=0x7f89e814f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.692+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e8160b90 0x7f89e814f4b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f89e000d600 tx=0x7f89e000dac0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 shutdown_connections 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f89c40744e0 0x7f89c40769f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89e8160b90 0x7f89e814f4b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89e80d50e0 0x7f89e814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 --2- 172.21.15.19:0/3331545257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89e80d2cb0 0x7f89e814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 >> 172.21.15.19:0/3331545257 conn(0x7f89e806c5c0 msgr2=0x7f89e80b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:50.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 shutdown_connections 2024-04-25T11:29:50.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:50.693+0000 7f89e7e00640 1 -- 172.21.15.19:0/3331545257 wait complete. 2024-04-25T11:29:50.710 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:29:50.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:50.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:29:50.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:29:50.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:29:50.808 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:29:50.860 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:29:50.860 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:29:50.860 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:50.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:29:50.919 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:29:50.919 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:29:50.919 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:50.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:29:50.978 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:29:50.978 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:29:50.978 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:29:51.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:29:51.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.333+0000 7f88fd600640 1 Processor -- start 2024-04-25T11:29:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.333+0000 7f88fd600640 1 -- start start 2024-04-25T11:29:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.335+0000 7f88fd600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.335+0000 7f88fd600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88f80cdb90 0x7f88f8161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.335+0000 7f88fd600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f806f5f0 con 0x7f88f80cdfa0 2024-04-25T11:29:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.335+0000 7f88fd600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f88f8070200 con 0x7f88f80cdb90 2024-04-25T11:29:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.335+0000 7f88fd600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88f806f8d0 con 0x7f88f8144500 2024-04-25T11:29:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.336+0000 7f88f6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.336+0000 7f88f6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88f80cdb90 0x7f88f8161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.336+0000 7f88f7400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f88f80cdfa0 0x7f88f80ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:49552/0 (socket says 172.21.15.19:49552) 2024-04-25T11:29:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.336+0000 7f88f7400640 1 -- 172.21.15.19:0/438896262 learned_addr learned my addr 172.21.15.19:0/438896262 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.337+0000 7f88f6a00640 1 -- 172.21.15.19:0/438896262 >> v2:172.21.15.169:3300/0 conn(0x7f88f80cdb90 msgr2=0x7f88f8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.337+0000 7f88f6a00640 1 --2- 172.21.15.19:0/438896262 >> v2:172.21.15.169:3300/0 conn(0x7f88f80cdb90 0x7f88f8161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.337+0000 7f88f6a00640 1 -- 172.21.15.19:0/438896262 >> v1:172.21.15.169:6789/0 conn(0x7f88f80cdfa0 legacy=0x7f88f80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.337+0000 7f88f6a00640 1 -- 172.21.15.19:0/438896262 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f806fbb0 con 0x7f88f8144500 2024-04-25T11:29:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.337+0000 7f88f6000640 1 --2- 172.21.15.19:0/438896262 >> v2:172.21.15.169:3300/0 conn(0x7f88f80cdb90 0x7f88f8161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:29:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.338+0000 7f88f6a00640 1 --2- 172.21.15.19:0/438896262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f8070690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88e80097a0 tx=0x7f88e802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f5e7c338b1d878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.339+0000 7f88f5600640 1 -- 172.21.15.19:0/438896262 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8037070 con 0x7f88f8144500 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.339+0000 7f88f5600640 1 -- 172.21.15.19:0/438896262 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e8032440 con 0x7f88f8144500 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.339+0000 7f88f5600640 1 -- 172.21.15.19:0/438896262 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8032810 con 0x7f88f8144500 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.339+0000 7f88fd600640 1 -- 172.21.15.19:0/438896262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 msgr2=0x7f88f8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.339+0000 7f88fd600640 1 --2- 172.21.15.19:0/438896262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f8070690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88e80097a0 tx=0x7f88e802a180 comp rx=0 tx=0).stop 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.339+0000 7f88fd600640 1 -- 172.21.15.19:0/438896262 shutdown_connections 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.340+0000 7f88fd600640 1 --2- 172.21.15.19:0/438896262 >> v2:172.21.15.169:3300/0 conn(0x7f88f80cdb90 0x7f88f8161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.340+0000 7f88fd600640 1 --2- 172.21.15.19:0/438896262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f8070690 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.340+0000 7f88fd600640 1 -- 172.21.15.19:0/438896262 >> 172.21.15.19:0/438896262 conn(0x7f88f8073050 msgr2=0x7f88f8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:51.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.340+0000 7f88fd600640 1 -- 172.21.15.19:0/438896262 shutdown_connections 2024-04-25T11:29:51.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.340+0000 7f88fd600640 1 -- 172.21.15.19:0/438896262 wait complete. 2024-04-25T11:29:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.342+0000 7f88fd600640 1 Processor -- start 2024-04-25T11:29:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.342+0000 7f88fd600640 1 -- start start 2024-04-25T11:29:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88fd600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f80cdb90 0x7f88f81fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88fd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88f80cdfa0 0x7f88f813c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88fd600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f80b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88fd600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88f806f2e0 con 0x7f88f80cdfa0 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88fd600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88f806f160 con 0x7f88f80cdb90 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88fd600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88f8162140 con 0x7f88f8144500 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88f6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88f80cdfa0 0x7f88f813c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88f6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88f80cdfa0 0x7f88f813c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39120/0 (socket says 172.21.15.19:39120) 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88f6000640 1 -- 172.21.15.19:0/1938431490 learned_addr learned my addr 172.21.15.19:0/1938431490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88f6a00640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f80cdb90 0x7f88f81fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88f7400640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f80b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88f6000640 1 -- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 msgr2=0x7f88f80b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.344+0000 7f88f6000640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f80b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88f6000640 1 -- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f80cdb90 msgr2=0x7f88f81fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88f6000640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f80cdb90 0x7f88f81fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88f6000640 1 -- 172.21.15.19:0/1938431490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f813c730 con 0x7f88f80cdfa0 2024-04-25T11:29:51.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88f6000640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88f80cdfa0 0x7f88f813c1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f88e400e840 tx=0x7f88e400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88eea00640 1 -- 172.21.15.19:0/1938431490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4017070 con 0x7f88f80cdfa0 2024-04-25T11:29:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88eea00640 1 -- 172.21.15.19:0/1938431490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e400c940 con 0x7f88f80cdfa0 2024-04-25T11:29:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88eea00640 1 -- 172.21.15.19:0/1938431490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e400cd30 con 0x7f88f80cdfa0 2024-04-25T11:29:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f80b6290 con 0x7f88f80cdfa0 2024-04-25T11:29:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.345+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f80b67a0 con 0x7f88f80cdfa0 2024-04-25T11:29:51.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.346+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f8050740 con 0x7f88f80cdfa0 2024-04-25T11:29:51.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.348+0000 7f88eea00640 1 -- 172.21.15.19:0/1938431490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88e401a070 con 0x7f88f80cdfa0 2024-04-25T11:29:51.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.349+0000 7f88eea00640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88d80744e0 0x7f88d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.349+0000 7f88eea00640 1 -- 172.21.15.19:0/1938431490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f88e403ce00 con 0x7f88f80cdfa0 2024-04-25T11:29:51.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.350+0000 7f88f6a00640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88d80744e0 0x7f88d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.351+0000 7f88f6a00640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88d80744e0 0x7f88d80769f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88e8034000 tx=0x7f88e802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:51.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.353+0000 7f88eea00640 1 -- 172.21.15.19:0/1938431490 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88f813c730 con 0x7f88f80cdfa0 2024-04-25T11:29:51.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.596+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88f8143a30 con 0x7f88f80cdfa0 2024-04-25T11:29:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.598+0000 7f88eea00640 1 -- 172.21.15.19:0/1938431490 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5179 (secure 0 0 0) 0x7f88e405fdf0 con 0x7f88f80cdfa0 2024-04-25T11:29:51.589 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:51.589 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:19.352476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5331,"qdb_cluster":[5331]},"id":2}]} 2024-04-25T11:29:51.589 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T11:29:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.600+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88d80744e0 msgr2=0x7f88d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.600+0000 7f88fd600640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88d80744e0 0x7f88d80769f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88e8034000 tx=0x7f88e802e000 comp rx=0 tx=0).stop 2024-04-25T11:29:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.600+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88f80cdfa0 msgr2=0x7f88f813c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.600+0000 7f88fd600640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88f80cdfa0 0x7f88f813c1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f88e400e840 tx=0x7f88e400ed00 comp rx=0 tx=0).stop 2024-04-25T11:29:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 shutdown_connections 2024-04-25T11:29:51.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88d80744e0 0x7f88d80769f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f8144500 0x7f88f80b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88f80cdfa0 0x7f88f813c1f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 --2- 172.21.15.19:0/1938431490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f80cdb90 0x7f88f81fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 >> 172.21.15.19:0/1938431490 conn(0x7f88f8073050 msgr2=0x7f88f8161070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 shutdown_connections 2024-04-25T11:29:51.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.601+0000 7f88fd600640 1 -- 172.21.15.19:0/1938431490 wait complete. 2024-04-25T11:29:51.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:29:51.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.922+0000 7f3cb2e00640 1 Processor -- start 2024-04-25T11:29:51.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.923+0000 7f3cb2e00640 1 -- start start 2024-04-25T11:29:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.925+0000 7f3cb2e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3cac168720 0x7f3cac0cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.925+0000 7f3cb2e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cac071180 con 0x7f3cac06c5c0 2024-04-25T11:29:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.925+0000 7f3cb2e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cac0700e0 con 0x7f3cac0cf2f0 2024-04-25T11:29:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.925+0000 7f3cb2e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3cac070cf0 con 0x7f3cac168720 2024-04-25T11:29:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.925+0000 7f3cb2400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3cac06c5c0 0x7f3cac06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35868/0 (socket says 172.21.15.19:35868) 2024-04-25T11:29:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.925+0000 7f3cb2400640 1 -- 172.21.15.19:0/1778247007 learned_addr learned my addr 172.21.15.19:0/1778247007 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:51.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081408188 0 0) 0x7f3cac071180 con 0x7f3cac06c5c0 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cb1000640 1 --2- 172.21.15.19:0/1778247007 >> v2:172.21.15.169:3300/0 conn(0x7f3cac168720 0x7f3cac0cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c94003520 con 0x7f3cac06c5c0 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1895954507 0 0) 0x7f3c94003520 con 0x7f3cac06c5c0 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 >> v2:172.21.15.169:3300/0 conn(0x7f3cac168720 msgr2=0x7f3cac0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 --2- 172.21.15.19:0/1778247007 >> v2:172.21.15.169:3300/0 conn(0x7f3cac168720 0x7f3cac0cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 >> v1:172.21.15.169:6789/0 conn(0x7f3cac0cf2f0 legacy=0x7f3cac168010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac0703c0 con 0x7f3cac06c5c0 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f3ca40019f0 con 0x7f3cac06c5c0 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.926+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 >> v1:172.21.15.19:6789/0 conn(0x7f3cac06c5c0 legacy=0x7f3cac06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cabe00640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c940039b0 0x7f3c94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cabe00640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c94004800 0x7f3c94004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cabe00640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c94005270 0x7f3c9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac070cf0 con 0x7f3c94004800 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9400c200 con 0x7f3c940039b0 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9400c400 con 0x7f3c94005270 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb1000640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c94004800 0x7f3c94004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb1a00640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c940039b0 0x7f3c94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb1000640 1 -- 172.21.15.19:0/1778247007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c94005270 msgr2=0x7f3c9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb1000640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c94005270 0x7f3c9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb1000640 1 -- 172.21.15.19:0/1778247007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c940039b0 msgr2=0x7f3c94003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb1000640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c940039b0 0x7f3c94003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb1000640 1 -- 172.21.15.19:0/1778247007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c94003520 con 0x7f3c94004800 2024-04-25T11:29:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.927+0000 7f3cb2400640 1 -- 172.21.15.19:0/1778247007 reap_dead start 2024-04-25T11:29:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.928+0000 7f3cb1000640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c94004800 0x7f3c94004c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800ba30 tx=0x7f3c94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ef4b7736cd60ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.929+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98030070 con 0x7f3c94004800 2024-04-25T11:29:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.929+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9802ae00 con 0x7f3c94004800 2024-04-25T11:29:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.929+0000 7f3cabe00640 1 -- 172.21.15.19:0/1778247007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800b600 con 0x7f3c94004800 2024-04-25T11:29:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.929+0000 7f3cb2e00640 1 -- 172.21.15.19:0/1778247007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c94004800 msgr2=0x7f3c94004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.929+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c94004800 0x7f3c94004c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800ba30 tx=0x7f3c94005830 comp rx=0 tx=0).stop 2024-04-25T11:29:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.930+0000 7f3cb2e00640 1 -- 172.21.15.19:0/1778247007 shutdown_connections 2024-04-25T11:29:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.930+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/1778247007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c94004800 0x7f3c94004c10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.930+0000 7f3cb2e00640 1 -- 172.21.15.19:0/1778247007 >> 172.21.15.19:0/1778247007 conn(0x7f3cac0d1720 msgr2=0x7f3cac0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.930+0000 7f3cb2e00640 1 -- 172.21.15.19:0/1778247007 shutdown_connections 2024-04-25T11:29:51.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.930+0000 7f3cb2e00640 1 -- 172.21.15.19:0/1778247007 wait complete. 2024-04-25T11:29:51.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.932+0000 7f3cb2e00640 1 Processor -- start 2024-04-25T11:29:51.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.932+0000 7f3cb2e00640 1 -- start start 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c94004800 0x7f3cac14a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cac163d80 0x7f3cac1fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cac160e60 0x7f3cac161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb2e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac168d00 con 0x7f3cac163d80 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac070980 con 0x7f3c94004800 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3cac168e80 con 0x7f3cac160e60 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cac163d80 0x7f3cac1fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cac163d80 0x7f3cac1fa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39138/0 (socket says 172.21.15.19:39138) 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb1000640 1 -- 172.21.15.19:0/3211365797 learned_addr learned my addr 172.21.15.19:0/3211365797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb2400640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cac160e60 0x7f3cac161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.934+0000 7f3cb1a00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c94004800 0x7f3cac14a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3cb1000640 1 -- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cac160e60 msgr2=0x7f3cac161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3cb1000640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cac160e60 0x7f3cac161270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3cb1000640 1 -- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c94004800 msgr2=0x7f3cac14a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3cb1000640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c94004800 0x7f3cac14a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3cb1000640 1 -- 172.21.15.19:0/3211365797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac1600a0 con 0x7f3cac163d80 2024-04-25T11:29:51.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3cb2400640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cac160e60 0x7f3cac161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:51.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3cb1000640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cac163d80 0x7f3cac1fa4e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800bb50 tx=0x7f3c98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:51.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.935+0000 7f3ca9600640 1 -- 172.21.15.19:0/3211365797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98042030 con 0x7f3cac163d80 2024-04-25T11:29:51.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.936+0000 7f3ca9600640 1 -- 172.21.15.19:0/3211365797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9802ac00 con 0x7f3cac163d80 2024-04-25T11:29:51.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.936+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac160380 con 0x7f3cac163d80 2024-04-25T11:29:51.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.936+0000 7f3ca9600640 1 -- 172.21.15.19:0/3211365797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800b600 con 0x7f3cac163d80 2024-04-25T11:29:51.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.936+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1fad80 con 0x7f3cac163d80 2024-04-25T11:29:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.937+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cac050740 con 0x7f3cac163d80 2024-04-25T11:29:51.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.938+0000 7f3ca9600640 1 -- 172.21.15.19:0/3211365797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c98045070 con 0x7f3cac163d80 2024-04-25T11:29:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.939+0000 7f3ca9600640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3c8c0744b0 0x7f3c8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.939+0000 7f3ca9600640 1 -- 172.21.15.19:0/3211365797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f3c980b5800 con 0x7f3cac163d80 2024-04-25T11:29:51.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.940+0000 7f3cb1a00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3c8c0744b0 0x7f3c8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:51.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.941+0000 7f3cb1a00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3c8c0744b0 0x7f3c8c0769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0007fd0 tx=0x7f3ca0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:51.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:51.944+0000 7f3ca9600640 1 -- 172.21.15.19:0/3211365797 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c9804d020 con 0x7f3cac163d80 2024-04-25T11:29:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.152+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cac165390 con 0x7f3cac163d80 2024-04-25T11:29:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.154+0000 7f3ca9600640 1 -- 172.21.15.19:0/3211365797 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16987 (secure 0 0 0) 0x7f3c9807fe20 con 0x7f3cac163d80 2024-04-25T11:29:52.146 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:52.146 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:29:06.584671+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3831883468":"2024-04-26T11:29:00.304477+0000","172.21.15.19:6833/3831883468":"2024-04-26T11:29:00.304477+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3c8c0744b0 msgr2=0x7f3c8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3c8c0744b0 0x7f3c8c0769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0007fd0 tx=0x7f3ca0009040 comp rx=0 tx=0).stop 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cac163d80 msgr2=0x7f3cac1fa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cac163d80 0x7f3cac1fa4e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800bb50 tx=0x7f3c98004730 comp rx=0 tx=0).stop 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 shutdown_connections 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3c8c0744b0 0x7f3c8c0769c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3cac160e60 0x7f3cac161270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cac163d80 0x7f3cac1fa4e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 --2- 172.21.15.19:0/3211365797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c94004800 0x7f3cac14a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.157+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 >> 172.21.15.19:0/3211365797 conn(0x7f3cac0d1720 msgr2=0x7f3cac0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.158+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 shutdown_connections 2024-04-25T11:29:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.158+0000 7f3cb2e00640 1 -- 172.21.15.19:0/3211365797 wait complete. 2024-04-25T11:29:52.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:28:43.580410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:28:44.810835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:29:52.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:29:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.469+0000 7f0fbbe00640 1 Processor -- start 2024-04-25T11:29:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.469+0000 7f0fbbe00640 1 -- start start 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.471+0000 7f0fbbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fb4161710 0x7f0fb415e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.471+0000 7f0fbbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0fb406c5c0 0x7f0fb406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.471+0000 7f0fbbe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fb41609b0 con 0x7f0fb4163fa0 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.471+0000 7f0fbbe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb406cf10 con 0x7f0fb4161710 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.471+0000 7f0fbbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb4164cd0 con 0x7f0fb406c5c0 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fb4161710 0x7f0fb415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fb4161710 0x7f0fb415e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39154/0 (socket says 172.21.15.19:39154) 2024-04-25T11:29:52.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb9400640 1 -- 172.21.15.19:0/4036504353 learned_addr learned my addr 172.21.15.19:0/4036504353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:52.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb9400640 1 -- 172.21.15.19:0/4036504353 >> v2:172.21.15.169:3300/0 conn(0x7f0fb406c5c0 msgr2=0x7f0fb406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:52.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb8a00640 1 --2- 172.21.15.19:0/4036504353 >> v2:172.21.15.169:3300/0 conn(0x7f0fb406c5c0 0x7f0fb406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:52.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb9400640 1 --2- 172.21.15.19:0/4036504353 >> v2:172.21.15.169:3300/0 conn(0x7f0fb406c5c0 0x7f0fb406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb9400640 1 -- 172.21.15.19:0/4036504353 >> v1:172.21.15.169:6789/0 conn(0x7f0fb4163fa0 legacy=0x7f0fb41643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:52.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.472+0000 7f0fb9400640 1 -- 172.21.15.19:0/4036504353 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb4160c30 con 0x7f0fb4161710 2024-04-25T11:29:52.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.473+0000 7f0fb9400640 1 --2- 172.21.15.19:0/4036504353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb4161710 0x7f0fb415e080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40d2770 tx=0x7f0fa802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446c04a10d990652 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:52.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fb3e00640 1 -- 172.21.15.19:0/4036504353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8037070 con 0x7f0fb4161710 2024-04-25T11:29:52.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fb3e00640 1 -- 172.21.15.19:0/4036504353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa8032440 con 0x7f0fb4161710 2024-04-25T11:29:52.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fb3e00640 1 -- 172.21.15.19:0/4036504353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8032810 con 0x7f0fb4161710 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fbbe00640 1 -- 172.21.15.19:0/4036504353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb4161710 msgr2=0x7f0fb415e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/4036504353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb4161710 0x7f0fb415e080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40d2770 tx=0x7f0fa802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fbbe00640 1 -- 172.21.15.19:0/4036504353 shutdown_connections 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/4036504353 >> v2:172.21.15.169:3300/0 conn(0x7f0fb406c5c0 0x7f0fb406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/4036504353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb4161710 0x7f0fb415e080 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.474+0000 7f0fbbe00640 1 -- 172.21.15.19:0/4036504353 >> 172.21.15.19:0/4036504353 conn(0x7f0fb40d5d90 msgr2=0x7f0fb40d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.475+0000 7f0fbbe00640 1 -- 172.21.15.19:0/4036504353 shutdown_connections 2024-04-25T11:29:52.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.475+0000 7f0fbbe00640 1 -- 172.21.15.19:0/4036504353 wait complete. 2024-04-25T11:29:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.477+0000 7f0fbbe00640 1 Processor -- start 2024-04-25T11:29:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.477+0000 7f0fbbe00640 1 -- start start 2024-04-25T11:29:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fbbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb406c5c0 0x7f0fb40ba430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fbbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4161710 0x7f0fb40ba970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fbbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb4163fa0 0x7f0fb40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fbbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb40d49f0 con 0x7f0fb406c5c0 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fbbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41611f0 con 0x7f0fb4163fa0 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fbbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb40d4b70 con 0x7f0fb4161710 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb406c5c0 0x7f0fb40ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb406c5c0 0x7f0fb40ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39170/0 (socket says 172.21.15.19:39170) 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 -- 172.21.15.19:0/360343139 learned_addr learned my addr 172.21.15.19:0/360343139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb8a00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4161710 0x7f0fb40ba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 -- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4161710 msgr2=0x7f0fb40ba970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4161710 0x7f0fb40ba970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 -- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb4163fa0 msgr2=0x7f0fb40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9e00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb4163fa0 0x7f0fb40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb4163fa0 0x7f0fb40cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.479+0000 7f0fb9400640 1 -- 172.21.15.19:0/360343139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb40c99d0 con 0x7f0fb406c5c0 2024-04-25T11:29:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.480+0000 7f0fb9e00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb4163fa0 0x7f0fb40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.480+0000 7f0fb9400640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb406c5c0 0x7f0fb40ba430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000c00 tx=0x7f0fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.480+0000 7f0fb1600640 1 -- 172.21.15.19:0/360343139 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8041020 con 0x7f0fb406c5c0 2024-04-25T11:29:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.480+0000 7f0fb1600640 1 -- 172.21.15.19:0/360343139 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa8030970 con 0x7f0fb406c5c0 2024-04-25T11:29:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.480+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb40c9cb0 con 0x7f0fb406c5c0 2024-04-25T11:29:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.480+0000 7f0fb1600640 1 -- 172.21.15.19:0/360343139 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8037040 con 0x7f0fb406c5c0 2024-04-25T11:29:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.480+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb40ca270 con 0x7f0fb406c5c0 2024-04-25T11:29:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.482+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb4050740 con 0x7f0fb406c5c0 2024-04-25T11:29:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.483+0000 7f0fb1600640 1 -- 172.21.15.19:0/360343139 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0fa8044020 con 0x7f0fb406c5c0 2024-04-25T11:29:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.485+0000 7f0fb1600640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f980744b0 0x7f0f980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.485+0000 7f0fb1600640 1 -- 172.21.15.19:0/360343139 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f0fa80b5df0 con 0x7f0fb406c5c0 2024-04-25T11:29:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.485+0000 7f0fb8a00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f980744b0 0x7f0f980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.487+0000 7f0fb8a00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f980744b0 0x7f0f980769c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40cc190 tx=0x7f0fa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.490+0000 7f0fb1600640 1 -- 172.21.15.19:0/360343139 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fa804b030 con 0x7f0fb406c5c0 2024-04-25T11:29:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.698+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0fb40d0db0 con 0x7f0fb406c5c0 2024-04-25T11:29:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.699+0000 7f0fb1600640 1 -- 172.21.15.19:0/360343139 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f0fb41609b0 con 0x7f0fb406c5c0 2024-04-25T11:29:52.690 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f980744b0 msgr2=0x7f0f980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f980744b0 0x7f0f980769c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40cc190 tx=0x7f0fa4008040 comp rx=0 tx=0).stop 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb406c5c0 msgr2=0x7f0fb40ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb406c5c0 0x7f0fb40ba430 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000c00 tx=0x7f0fa80046c0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 shutdown_connections 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f980744b0 0x7f0f980769c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb4163fa0 0x7f0fb40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4161710 0x7f0fb40ba970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 --2- 172.21.15.19:0/360343139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb406c5c0 0x7f0fb40ba430 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.702+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 >> 172.21.15.19:0/360343139 conn(0x7f0fb40d5d90 msgr2=0x7f0fb40d4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.703+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 shutdown_connections 2024-04-25T11:29:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:52.703+0000 7f0fbbe00640 1 -- 172.21.15.19:0/360343139 wait complete. 2024-04-25T11:29:52.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:29:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.020+0000 7f7fddc00640 1 Processor -- start 2024-04-25T11:29:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.020+0000 7f7fddc00640 1 -- start start 2024-04-25T11:29:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.023+0000 7f7fddc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 0x7f7fd806cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.023+0000 7f7fddc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd806d3a0 con 0x7f7fd8162210 2024-04-25T11:29:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.023+0000 7f7fddc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd806d680 con 0x7f7fd81441d0 2024-04-25T11:29:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.023+0000 7f7fddc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd806d960 con 0x7f7fd8160a50 2024-04-25T11:29:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.023+0000 7f7fd7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7fd8162210 0x7f7fd806c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35872/0 (socket says 172.21.15.19:35872) 2024-04-25T11:29:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.023+0000 7f7fd7400640 1 -- 172.21.15.19:0/3705480833 learned_addr learned my addr 172.21.15.19:0/3705480833 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.023+0000 7f7fdd200640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 0x7f7fd806cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736766766 0 0) 0x7f7fd806d3a0 con 0x7f7fd8162210 2024-04-25T11:29:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fbc003520 con 0x7f7fd8162210 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1943971268 0 0) 0x7f7fbc003520 con 0x7f7fd8162210 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 msgr2=0x7f7fd806cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 0x7f7fd806cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 >> v1:172.21.15.169:6789/0 conn(0x7f7fd81441d0 legacy=0x7f7fd8161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd806dc40 con 0x7f7fd8162210 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7fc8001670 con 0x7f7fd8162210 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 >> v1:172.21.15.19:6789/0 conn(0x7f7fd8162210 legacy=0x7f7fd806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc0039e0 0x7f7fbc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 0x7f7fbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc004d40 0x7f7fbc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd806d960 con 0x7f7fbc0039e0 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc00ff60 con 0x7f7fbc004d40 2024-04-25T11:29:53.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.024+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc010110 con 0x7f7fd8160a50 2024-04-25T11:29:53.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7e00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc0039e0 0x7f7fbc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fdd200640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc004d40 0x7f7fbc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7400640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 0x7f7fbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7e00640 1 -- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 msgr2=0x7f7fbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7e00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 0x7f7fbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7e00640 1 -- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc004d40 msgr2=0x7f7fbc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7e00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc004d40 0x7f7fbc005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7e00640 1 -- 172.21.15.19:0/3705480833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc003520 con 0x7f7fbc0039e0 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fdd200640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc004d40 0x7f7fbc005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.025+0000 7f7fd7400640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd8160a50 0x7f7fbc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.026+0000 7f7fd7e00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc0039e0 0x7f7fbc003e10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0001910 tx=0x7f7fbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aefe199a9ca78ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.027+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0037070 con 0x7f7fbc0039e0 2024-04-25T11:29:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.027+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc002ac50 con 0x7f7fbc0039e0 2024-04-25T11:29:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.027+0000 7f7fd6a00640 1 -- 172.21.15.19:0/3705480833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00014e0 con 0x7f7fbc0039e0 2024-04-25T11:29:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.027+0000 7f7fddc00640 1 -- 172.21.15.19:0/3705480833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc0039e0 msgr2=0x7f7fbc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.027+0000 7f7fddc00640 1 --2- 172.21.15.19:0/3705480833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc0039e0 0x7f7fbc003e10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0001910 tx=0x7f7fbc005830 comp rx=0 tx=0).stop 2024-04-25T11:29:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.027+0000 7f7fdd200640 1 -- 172.21.15.19:0/3705480833 reap_dead start 2024-04-25T11:29:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.028+0000 7f7fddc00640 1 -- 172.21.15.19:0/3705480833 shutdown_connections 2024-04-25T11:29:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.028+0000 7f7fddc00640 1 -- 172.21.15.19:0/3705480833 >> 172.21.15.19:0/3705480833 conn(0x7f7fd8072f40 msgr2=0x7f7fd8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.028+0000 7f7fddc00640 1 -- 172.21.15.19:0/3705480833 shutdown_connections 2024-04-25T11:29:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.028+0000 7f7fddc00640 1 -- 172.21.15.19:0/3705480833 wait complete. 2024-04-25T11:29:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.030+0000 7f7fddc00640 1 Processor -- start 2024-04-25T11:29:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.030+0000 7f7fddc00640 1 -- start start 2024-04-25T11:29:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fddc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc0039e0 0x7f7fd80ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fddc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd80b54f0 0x7f7fd815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fddc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd80d0420 0x7f7fd80d0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fddc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd806cec0 con 0x7f7fd80b54f0 2024-04-25T11:29:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fddc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd806cd40 con 0x7f7fbc0039e0 2024-04-25T11:29:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fddc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd806d040 con 0x7f7fd80d0420 2024-04-25T11:29:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd80b54f0 0x7f7fd815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fd7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd80b54f0 0x7f7fd815b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39176/0 (socket says 172.21.15.19:39176) 2024-04-25T11:29:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.032+0000 7f7fd7400640 1 -- 172.21.15.19:0/2947217906 learned_addr learned my addr 172.21.15.19:0/2947217906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fd7e00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc0039e0 0x7f7fd80ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fdd200640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd80d0420 0x7f7fd80d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fd7400640 1 -- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd80d0420 msgr2=0x7f7fd80d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fd7400640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd80d0420 0x7f7fd80d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fd7400640 1 -- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc0039e0 msgr2=0x7f7fd80ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fd7400640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc0039e0 0x7f7fd80ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fd7400640 1 -- 172.21.15.19:0/2947217906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd80d0d70 con 0x7f7fd80b54f0 2024-04-25T11:29:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fd7400640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd80b54f0 0x7f7fd815b240 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800d770 tx=0x7f7fc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fcfe00640 1 -- 172.21.15.19:0/2947217906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8019070 con 0x7f7fd80b54f0 2024-04-25T11:29:53.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.033+0000 7f7fcfe00640 1 -- 172.21.15.19:0/2947217906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc80074e0 con 0x7f7fd80b54f0 2024-04-25T11:29:53.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.034+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd80ceea0 con 0x7f7fd80b54f0 2024-04-25T11:29:53.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.034+0000 7f7fcfe00640 1 -- 172.21.15.19:0/2947217906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8004a20 con 0x7f7fd80b54f0 2024-04-25T11:29:53.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.034+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd80b5f10 con 0x7f7fd80b54f0 2024-04-25T11:29:53.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.035+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd8076c60 con 0x7f7fd80b54f0 2024-04-25T11:29:53.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.036+0000 7f7fcfe00640 1 -- 172.21.15.19:0/2947217906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7fc801c070 con 0x7f7fd80b54f0 2024-04-25T11:29:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.037+0000 7f7fcfe00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7fb40744e0 0x7f7fb40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.037+0000 7f7fcfe00640 1 -- 172.21.15.19:0/2947217906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f7fc8094d10 con 0x7f7fd80b54f0 2024-04-25T11:29:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.038+0000 7f7fd7e00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7fb40744e0 0x7f7fb40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.039+0000 7f7fd7e00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7fb40744e0 0x7f7fb40769f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0034000 tx=0x7f7fc002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.042+0000 7f7fcfe00640 1 -- 172.21.15.19:0/2947217906 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fc809a020 con 0x7f7fd80b54f0 2024-04-25T11:29:53.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.246+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7fd80707e0 con 0x7f7fd80b54f0 2024-04-25T11:29:53.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.248+0000 7f7fcfe00640 1 -- 172.21.15.19:0/2947217906 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f7fd80d0d70 con 0x7f7fd80b54f0 2024-04-25T11:29:53.239 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:29:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7fb40744e0 msgr2=0x7f7fb40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7fb40744e0 0x7f7fb40769f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0034000 tx=0x7f7fc002e000 comp rx=0 tx=0).stop 2024-04-25T11:29:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd80b54f0 msgr2=0x7f7fd815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd80b54f0 0x7f7fd815b240 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800d770 tx=0x7f7fc800dc30 comp rx=0 tx=0).stop 2024-04-25T11:29:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 shutdown_connections 2024-04-25T11:29:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7fb40744e0 0x7f7fb40769f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7fd80d0420 0x7f7fd80d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd80b54f0 0x7f7fd815b240 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 --2- 172.21.15.19:0/2947217906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fbc0039e0 0x7f7fd80ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 >> 172.21.15.19:0/2947217906 conn(0x7f7fd8072f40 msgr2=0x7f7fd8143010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.250+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 shutdown_connections 2024-04-25T11:29:53.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.251+0000 7f7fddc00640 1 -- 172.21.15.19:0/2947217906 wait complete. 2024-04-25T11:29:53.269 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:29:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.586+0000 7f597d400640 1 Processor -- start 2024-04-25T11:29:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.586+0000 7f597d400640 1 -- start start 2024-04-25T11:29:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.588+0000 7f597d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f59780aed40 0x7f59780b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.589+0000 7f597d400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59780700d0 con 0x7f59780b0b90 2024-04-25T11:29:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.589+0000 7f597d400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59780703b0 con 0x7f5978143590 2024-04-25T11:29:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.589+0000 7f597d400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5978070970 con 0x7f59780aed40 2024-04-25T11:29:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.589+0000 7f5977400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f59780aed40 0x7f59780b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.589+0000 7f597ca00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f59780b0b90 0x7f59780b52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35888/0 (socket says 172.21.15.19:35888) 2024-04-25T11:29:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.589+0000 7f597ca00640 1 -- 172.21.15.19:0/1283740820 learned_addr learned my addr 172.21.15.19:0/1283740820 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.590+0000 7f5977400640 1 -- 172.21.15.19:0/1283740820 >> v1:172.21.15.169:6789/0 conn(0x7f5978143590 legacy=0x7f597806f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.590+0000 7f5977400640 1 -- 172.21.15.19:0/1283740820 >> v1:172.21.15.19:6789/0 conn(0x7f59780b0b90 legacy=0x7f59780b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.590+0000 7f5977400640 1 -- 172.21.15.19:0/1283740820 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5978070690 con 0x7f59780aed40 2024-04-25T11:29:53.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.591+0000 7f5977400640 1 --2- 172.21.15.19:0/1283740820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59780aed40 0x7f59780b0650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5978070af0 tx=0x7f596002a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94eac4d62e990dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:53.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.591+0000 7f5976a00640 1 -- 172.21.15.19:0/1283740820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960037070 con 0x7f59780aed40 2024-04-25T11:29:53.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.591+0000 7f5976a00640 1 -- 172.21.15.19:0/1283740820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596002ae30 con 0x7f59780aed40 2024-04-25T11:29:53.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.591+0000 7f5976a00640 1 -- 172.21.15.19:0/1283740820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960032640 con 0x7f59780aed40 2024-04-25T11:29:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.592+0000 7f597d400640 1 -- 172.21.15.19:0/1283740820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59780aed40 msgr2=0x7f59780b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.592+0000 7f597d400640 1 --2- 172.21.15.19:0/1283740820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59780aed40 0x7f59780b0650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5978070af0 tx=0x7f596002a120 comp rx=0 tx=0).stop 2024-04-25T11:29:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.592+0000 7f597d400640 1 -- 172.21.15.19:0/1283740820 shutdown_connections 2024-04-25T11:29:53.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.592+0000 7f597d400640 1 --2- 172.21.15.19:0/1283740820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59780aed40 0x7f59780b0650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.592+0000 7f597d400640 1 -- 172.21.15.19:0/1283740820 >> 172.21.15.19:0/1283740820 conn(0x7f59780d1d60 msgr2=0x7f59780d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:53.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.592+0000 7f597d400640 1 -- 172.21.15.19:0/1283740820 shutdown_connections 2024-04-25T11:29:53.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.592+0000 7f597d400640 1 -- 172.21.15.19:0/1283740820 wait complete. 2024-04-25T11:29:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.595+0000 7f597d400640 1 Processor -- start 2024-04-25T11:29:53.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.595+0000 7f597d400640 1 -- start start 2024-04-25T11:29:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.596+0000 7f597d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59780aed40 0x7f59781426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59780b0b90 0x7f5978143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5978143590 0x7f59781e6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59780b0fa0 con 0x7f5978143590 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59780b4e20 con 0x7f59780b0b90 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f59780b59c0 con 0x7f59780aed40 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5978143590 0x7f59781e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5978143590 0x7f59781e6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39196/0 (socket says 172.21.15.19:39196) 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 -- 172.21.15.19:0/2433076526 learned_addr learned my addr 172.21.15.19:0/2433076526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f5977e00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59780aed40 0x7f59781426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 -- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59780aed40 msgr2=0x7f59781426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59780aed40 0x7f59781426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 -- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59780b0b90 msgr2=0x7f5978143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59780b0b90 0x7f5978143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f597ca00640 1 -- 172.21.15.19:0/2433076526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5978144620 con 0x7f5978143590 2024-04-25T11:29:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.597+0000 7f5977e00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59780aed40 0x7f59781426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.598+0000 7f597ca00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5978143590 0x7f59781e6800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f597000c340 tx=0x7f597000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.598+0000 7f596fe00640 1 -- 172.21.15.19:0/2433076526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970017070 con 0x7f5978143590 2024-04-25T11:29:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.598+0000 7f596fe00640 1 -- 172.21.15.19:0/2433076526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5970005ce0 con 0x7f5978143590 2024-04-25T11:29:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.598+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5978144900 con 0x7f5978143590 2024-04-25T11:29:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.598+0000 7f596fe00640 1 -- 172.21.15.19:0/2433076526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59700072d0 con 0x7f5978143590 2024-04-25T11:29:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.598+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5978144ec0 con 0x7f5978143590 2024-04-25T11:29:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.599+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5978076c60 con 0x7f5978143590 2024-04-25T11:29:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.601+0000 7f596fe00640 1 -- 172.21.15.19:0/2433076526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f597001a070 con 0x7f5978143590 2024-04-25T11:29:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.602+0000 7f596fe00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.602+0000 7f596fe00640 1 -- 172.21.15.19:0/2433076526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f59700945a0 con 0x7f5978143590 2024-04-25T11:29:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.603+0000 7f5977e00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.604+0000 7f5977e00640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5968001cd0 tx=0x7f5968009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:53.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.606+0000 7f596fe00640 1 -- 172.21.15.19:0/2433076526 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f597005e570 con 0x7f5978143590 2024-04-25T11:29:53.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.806+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5978142e40 con 0x7f5978143590 2024-04-25T11:29:53.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.808+0000 7f596fe00640 1 -- 172.21.15.19:0/2433076526 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f5970062870 con 0x7f5978143590 2024-04-25T11:29:53.799 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:29:53.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 msgr2=0x7f59540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5968001cd0 tx=0x7f5968009040 comp rx=0 tx=0).stop 2024-04-25T11:29:53.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5978143590 msgr2=0x7f59781e6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:53.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5978143590 0x7f59781e6800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f597000c340 tx=0x7f597000c800 comp rx=0 tx=0).stop 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 shutdown_connections 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59780b0b90 0x7f5978143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5978143590 0x7f59781e6800 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 --2- 172.21.15.19:0/2433076526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59780aed40 0x7f59781426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 >> 172.21.15.19:0/2433076526 conn(0x7f59780d1d60 msgr2=0x7f59780b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 shutdown_connections 2024-04-25T11:29:53.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:53.810+0000 7f597d400640 1 -- 172.21.15.19:0/2433076526 wait complete. 2024-04-25T11:29:53.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.087+0000 7f5b5fe00640 1 Processor -- start 2024-04-25T11:29:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.087+0000 7f5b5fe00640 1 -- start start 2024-04-25T11:29:54.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.089+0000 7f5b5fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b6006c5c0 0x7f5b600cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.089+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b60144370 0x7f5b600b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.089+0000 7f5b5fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b600b4cd0 0x7f5b60161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.089+0000 7f5b5fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5b600b50e0 con 0x7f5b600b4cd0 2024-04-25T11:29:54.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.089+0000 7f5b5fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5b600d1230 con 0x7f5b6006c5c0 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.089+0000 7f5b5fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b600d06b0 con 0x7f5b60144370 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b600b4cd0 0x7f5b60161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b600b4cd0 0x7f5b60161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33896/0 (socket says 172.21.15.19:33896) 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 -- 172.21.15.19:0/1951746195 learned_addr learned my addr 172.21.15.19:0/1951746195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 -- 172.21.15.19:0/1951746195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b60144370 msgr2=0x7f5b600b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5e000640 1 --2- 172.21.15.19:0/1951746195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b60144370 0x7f5b600b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 --2- 172.21.15.19:0/1951746195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b60144370 0x7f5b600b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 -- 172.21.15.19:0/1951746195 >> v2:172.21.15.169:3300/0 conn(0x7f5b6006c5c0 msgr2=0x7f5b600cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5ea00640 1 --2- 172.21.15.19:0/1951746195 >> v2:172.21.15.169:3300/0 conn(0x7f5b6006c5c0 0x7f5b600cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 --2- 172.21.15.19:0/1951746195 >> v2:172.21.15.169:3300/0 conn(0x7f5b6006c5c0 0x7f5b600cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5f400640 1 -- 172.21.15.19:0/1951746195 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b600d0990 con 0x7f5b600b4cd0 2024-04-25T11:29:54.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5e000640 1 --2- 172.21.15.19:0/1951746195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b60144370 0x7f5b600b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:54.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.090+0000 7f5b5ea00640 1 --2- 172.21.15.19:0/1951746195 >> v2:172.21.15.169:3300/0 conn(0x7f5b6006c5c0 0x7f5b600cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:54.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.091+0000 7f5b5f400640 1 --2- 172.21.15.19:0/1951746195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b600b4cd0 0x7f5b60161a10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5b600726b0 tx=0x7f5b5802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6554dd39b9fb76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:54.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.092+0000 7f5b5d600640 1 -- 172.21.15.19:0/1951746195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58036070 con 0x7f5b600b4cd0 2024-04-25T11:29:54.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.092+0000 7f5b5d600640 1 -- 172.21.15.19:0/1951746195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b58031600 con 0x7f5b600b4cd0 2024-04-25T11:29:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.092+0000 7f5b5d600640 1 -- 172.21.15.19:0/1951746195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580319d0 con 0x7f5b600b4cd0 2024-04-25T11:29:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.092+0000 7f5b5fe00640 1 -- 172.21.15.19:0/1951746195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b600b4cd0 msgr2=0x7f5b60161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.092+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/1951746195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b600b4cd0 0x7f5b60161a10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5b600726b0 tx=0x7f5b5802a690 comp rx=0 tx=0).stop 2024-04-25T11:29:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.093+0000 7f5b5fe00640 1 -- 172.21.15.19:0/1951746195 shutdown_connections 2024-04-25T11:29:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.093+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/1951746195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b600b4cd0 0x7f5b60161a10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.093+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/1951746195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b60144370 0x7f5b600b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.093+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/1951746195 >> v2:172.21.15.169:3300/0 conn(0x7f5b6006c5c0 0x7f5b600cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.093+0000 7f5b5fe00640 1 -- 172.21.15.19:0/1951746195 >> 172.21.15.19:0/1951746195 conn(0x7f5b600d1720 msgr2=0x7f5b600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.093+0000 7f5b5fe00640 1 -- 172.21.15.19:0/1951746195 shutdown_connections 2024-04-25T11:29:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.093+0000 7f5b5fe00640 1 -- 172.21.15.19:0/1951746195 wait complete. 2024-04-25T11:29:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.095+0000 7f5b5fe00640 1 Processor -- start 2024-04-25T11:29:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.095+0000 7f5b5fe00640 1 -- start start 2024-04-25T11:29:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b6006c5c0 0x7f5b601622b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b600b4cd0 0x7f5b601f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b60144370 0x7f5b60160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b600d02d0 con 0x7f5b6006c5c0 2024-04-25T11:29:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b600d0f50 con 0x7f5b60144370 2024-04-25T11:29:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b600d0450 con 0x7f5b600b4cd0 2024-04-25T11:29:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b6006c5c0 0x7f5b601622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b6006c5c0 0x7f5b601622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33908/0 (socket says 172.21.15.19:33908) 2024-04-25T11:29:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 -- 172.21.15.19:0/2537950543 learned_addr learned my addr 172.21.15.19:0/2537950543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 -- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b600b4cd0 msgr2=0x7f5b601f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b600b4cd0 0x7f5b601f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 -- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b60144370 msgr2=0x7f5b60160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b60144370 0x7f5b60160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.097+0000 7f5b5ea00640 1 -- 172.21.15.19:0/2537950543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b601f6460 con 0x7f5b6006c5c0 2024-04-25T11:29:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.098+0000 7f5b5ea00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b6006c5c0 0x7f5b601622b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c7f0 tx=0x7f5b4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.098+0000 7f5b56a00640 1 -- 172.21.15.19:0/2537950543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48015070 con 0x7f5b6006c5c0 2024-04-25T11:29:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.098+0000 7f5b56a00640 1 -- 172.21.15.19:0/2537950543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b480077a0 con 0x7f5b6006c5c0 2024-04-25T11:29:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.098+0000 7f5b56a00640 1 -- 172.21.15.19:0/2537950543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48007b50 con 0x7f5b6006c5c0 2024-04-25T11:29:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.098+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b601f6740 con 0x7f5b6006c5c0 2024-04-25T11:29:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.098+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b601f6d00 con 0x7f5b6006c5c0 2024-04-25T11:29:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.099+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b60147010 con 0x7f5b6006c5c0 2024-04-25T11:29:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.100+0000 7f5b56a00640 1 -- 172.21.15.19:0/2537950543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b48018070 con 0x7f5b6006c5c0 2024-04-25T11:29:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.101+0000 7f5b56a00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b3c0744e0 0x7f5b3c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.102+0000 7f5b56a00640 1 -- 172.21.15.19:0/2537950543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f5b48095490 con 0x7f5b6006c5c0 2024-04-25T11:29:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.102+0000 7f5b5e000640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b3c0744e0 0x7f5b3c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.103+0000 7f5b5e000640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b3c0744e0 0x7f5b3c0769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b50006b10 tx=0x7f5b50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.107+0000 7f5b56a00640 1 -- 172.21.15.19:0/2537950543 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b4805fab0 con 0x7f5b6006c5c0 2024-04-25T11:29:54.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.339+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b6006e100 con 0x7f5b6006c5c0 2024-04-25T11:29:54.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.340+0000 7f5b56a00640 1 -- 172.21.15.19:0/2537950543 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5179 (secure 0 0 0) 0x7f5b4805f3b0 con 0x7f5b6006c5c0 2024-04-25T11:29:54.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:54.332 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:19.352476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5331,"qdb_cluster":[5331]},"id":2}]} 2024-04-25T11:29:54.332 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T11:29:54.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b3c0744e0 msgr2=0x7f5b3c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b3c0744e0 0x7f5b3c0769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b50006b10 tx=0x7f5b50008040 comp rx=0 tx=0).stop 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b6006c5c0 msgr2=0x7f5b601622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b6006c5c0 0x7f5b601622b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c7f0 tx=0x7f5b4800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 shutdown_connections 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b3c0744e0 0x7f5b3c0769f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b60144370 0x7f5b60160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b600b4cd0 0x7f5b601f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 --2- 172.21.15.19:0/2537950543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b6006c5c0 0x7f5b601622b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 >> 172.21.15.19:0/2537950543 conn(0x7f5b600d1720 msgr2=0x7f5b600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.343+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 shutdown_connections 2024-04-25T11:29:54.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.344+0000 7f5b5fe00640 1 -- 172.21.15.19:0/2537950543 wait complete. 2024-04-25T11:29:54.364 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:29:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.651+0000 7f11a6e00640 1 Processor -- start 2024-04-25T11:29:54.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.651+0000 7f11a6e00640 1 -- start start 2024-04-25T11:29:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.653+0000 7f11a6e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11a015bbe0 0x7f11a0072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f11a6e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a00b1bc0 con 0x7f11a00d06c0 2024-04-25T11:29:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f11a6e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a00b1ea0 con 0x7f11a00d0ad0 2024-04-25T11:29:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f11a6e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f11a006cdc0 con 0x7f11a015bbe0 2024-04-25T11:29:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f11a5a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11a015bbe0 0x7f11a0072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f11a5000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f11a00d06c0 0x7f11a006cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43256/0 (socket says 172.21.15.19:43256) 2024-04-25T11:29:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f11a5000640 1 -- 172.21.15.19:0/1014692231 learned_addr learned my addr 172.21.15.19:0/1014692231 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f119fe00640 1 -- 172.21.15.19:0/1014692231 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358329186 0 0) 0x7f11a00b1bc0 con 0x7f11a00d06c0 2024-04-25T11:29:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f119fe00640 1 -- 172.21.15.19:0/1014692231 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1184003520 con 0x7f11a00d06c0 2024-04-25T11:29:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.654+0000 7f11a5a00640 1 -- 172.21.15.19:0/1014692231 >> v1:172.21.15.169:6789/0 conn(0x7f11a00d0ad0 legacy=0x7f11a015e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.655+0000 7f11a5a00640 1 -- 172.21.15.19:0/1014692231 >> v1:172.21.15.19:6789/0 conn(0x7f11a00d06c0 legacy=0x7f11a006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.655+0000 7f11a5a00640 1 -- 172.21.15.19:0/1014692231 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a006c5c0 con 0x7f11a015bbe0 2024-04-25T11:29:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.656+0000 7f11a5a00640 1 --2- 172.21.15.19:0/1014692231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a015bbe0 0x7f11a0072190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f11900097a0 tx=0x7f119002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd54527ee0b3cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.656+0000 7f119fe00640 1 -- 172.21.15.19:0/1014692231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190037070 con 0x7f11a015bbe0 2024-04-25T11:29:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.656+0000 7f119fe00640 1 -- 172.21.15.19:0/1014692231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119002ae50 con 0x7f11a015bbe0 2024-04-25T11:29:54.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.656+0000 7f119fe00640 1 -- 172.21.15.19:0/1014692231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190032640 con 0x7f11a015bbe0 2024-04-25T11:29:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.657+0000 7f11a6e00640 1 -- 172.21.15.19:0/1014692231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a015bbe0 msgr2=0x7f11a0072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.657+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1014692231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a015bbe0 0x7f11a0072190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f11900097a0 tx=0x7f119002a140 comp rx=0 tx=0).stop 2024-04-25T11:29:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.657+0000 7f11a6e00640 1 -- 172.21.15.19:0/1014692231 shutdown_connections 2024-04-25T11:29:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.657+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1014692231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a015bbe0 0x7f11a0072190 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.657+0000 7f11a6e00640 1 -- 172.21.15.19:0/1014692231 >> 172.21.15.19:0/1014692231 conn(0x7f11a0070270 msgr2=0x7f11a0070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.657+0000 7f11a6e00640 1 -- 172.21.15.19:0/1014692231 shutdown_connections 2024-04-25T11:29:54.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.657+0000 7f11a6e00640 1 -- 172.21.15.19:0/1014692231 wait complete. 2024-04-25T11:29:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.659+0000 7f11a6e00640 1 Processor -- start 2024-04-25T11:29:54.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.660+0000 7f11a6e00640 1 -- start start 2024-04-25T11:29:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a00d06c0 0x7f11a0145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a6e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a00d0ad0 0x7f11a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a015bbe0 0x7f11a0156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a6e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11a006c930 con 0x7f11a00d0ad0 2024-04-25T11:29:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11a00b1970 con 0x7f11a00d06c0 2024-04-25T11:29:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11a006d6e0 con 0x7f11a015bbe0 2024-04-25T11:29:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a00d0ad0 0x7f11a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a00d0ad0 0x7f11a015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33930/0 (socket says 172.21.15.19:33930) 2024-04-25T11:29:54.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.661+0000 7f11a5000640 1 -- 172.21.15.19:0/1861301645 learned_addr learned my addr 172.21.15.19:0/1861301645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a6400640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a015bbe0 0x7f11a0156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a5a00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a00d06c0 0x7f11a0145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a5000640 1 -- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a015bbe0 msgr2=0x7f11a0156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a5000640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a015bbe0 0x7f11a0156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a5000640 1 -- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a00d06c0 msgr2=0x7f11a0145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a5000640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a00d06c0 0x7f11a0145210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a5000640 1 -- 172.21.15.19:0/1861301645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a0145750 con 0x7f11a00d0ad0 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f11a5000640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a00d0ad0 0x7f11a015d4c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c3c0 tx=0x7f118c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.662+0000 7f119d600640 1 -- 172.21.15.19:0/1861301645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c017070 con 0x7f11a00d0ad0 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.663+0000 7f119d600640 1 -- 172.21.15.19:0/1861301645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c005ce0 con 0x7f11a00d0ad0 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.663+0000 7f119d600640 1 -- 172.21.15.19:0/1861301645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0072f0 con 0x7f11a00d0ad0 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.663+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a01f6670 con 0x7f11a00d0ad0 2024-04-25T11:29:54.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.663+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a01f6c30 con 0x7f11a00d0ad0 2024-04-25T11:29:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.664+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a015dcf0 con 0x7f11a00d0ad0 2024-04-25T11:29:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.665+0000 7f119d600640 1 -- 172.21.15.19:0/1861301645 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f118c01a070 con 0x7f11a00d0ad0 2024-04-25T11:29:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.667+0000 7f119d600640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11800744b0 0x7f11800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:54.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.667+0000 7f119d600640 1 -- 172.21.15.19:0/1861301645 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f118c0946a0 con 0x7f11a00d0ad0 2024-04-25T11:29:54.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.667+0000 7f11a5a00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11800744b0 0x7f11800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:54.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.669+0000 7f11a5a00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11800744b0 0x7f11800769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1190034000 tx=0x7f119002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:54.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.672+0000 7f119d600640 1 -- 172.21.15.19:0/1861301645 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11a0145750 con 0x7f11a00d0ad0 2024-04-25T11:29:54.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.870+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11a0050740 con 0x7f11a00d0ad0 2024-04-25T11:29:54.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.873+0000 7f119d600640 1 -- 172.21.15.19:0/1861301645 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16987 (secure 0 0 0) 0x7f118c05ed50 con 0x7f11a00d0ad0 2024-04-25T11:29:54.864 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:54.864 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:29:06.584671+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3831883468":"2024-04-26T11:29:00.304477+0000","172.21.15.19:6833/3831883468":"2024-04-26T11:29:00.304477+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11800744b0 msgr2=0x7f11800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11800744b0 0x7f11800769c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1190034000 tx=0x7f119002e000 comp rx=0 tx=0).stop 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a00d0ad0 msgr2=0x7f11a015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a00d0ad0 0x7f11a015d4c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c3c0 tx=0x7f118c00c880 comp rx=0 tx=0).stop 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 shutdown_connections 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11800744b0 0x7f11800769c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a015bbe0 0x7f11a0156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a00d0ad0 0x7f11a015d4c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 --2- 172.21.15.19:0/1861301645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a00d06c0 0x7f11a0145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.875+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 >> 172.21.15.19:0/1861301645 conn(0x7f11a0070270 msgr2=0x7f11a00687c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.876+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 shutdown_connections 2024-04-25T11:29:54.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:54.876+0000 7f11a6e00640 1 -- 172.21.15.19:0/1861301645 wait complete. 2024-04-25T11:29:54.895 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:29:54.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:55.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.176+0000 7fb06e800640 1 Processor -- start 2024-04-25T11:29:55.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.176+0000 7fb06e800640 1 -- start start 2024-04-25T11:29:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb06e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb0681434d0 0x7fb068070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb06e800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb0681612f0 0x7fb0680d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb06e800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0680709c0 con 0x7fb0680d0ed0 2024-04-25T11:29:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb06e800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb0681438e0 con 0x7fb0681434d0 2024-04-25T11:29:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb06e800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb06806f3e0 con 0x7fb0681612f0 2024-04-25T11:29:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb067400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb0681434d0 0x7fb068070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb067400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb0681434d0 0x7fb068070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33934/0 (socket says 172.21.15.19:33934) 2024-04-25T11:29:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.179+0000 7fb067400640 1 -- 172.21.15.19:0/4219183571 learned_addr learned my addr 172.21.15.19:0/4219183571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.180+0000 7fb067400640 1 -- 172.21.15.19:0/4219183571 >> v2:172.21.15.169:3300/0 conn(0x7fb0681612f0 msgr2=0x7fb0680d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.180+0000 7fb067400640 1 --2- 172.21.15.19:0/4219183571 >> v2:172.21.15.169:3300/0 conn(0x7fb0681612f0 0x7fb0680d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.180+0000 7fb067400640 1 -- 172.21.15.19:0/4219183571 >> v1:172.21.15.169:6789/0 conn(0x7fb0680d0ed0 legacy=0x7fb068160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.180+0000 7fb067400640 1 -- 172.21.15.19:0/4219183571 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06806f6c0 con 0x7fb0681434d0 2024-04-25T11:29:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.181+0000 7fb067400640 1 --2- 172.21.15.19:0/4219183571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb068070480 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb05800bb10 tx=0x7fb05802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179b2805b854fd9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.181+0000 7fb066000640 1 -- 172.21.15.19:0/4219183571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05802f070 con 0x7fb0681434d0 2024-04-25T11:29:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.181+0000 7fb066000640 1 -- 172.21.15.19:0/4219183571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb058038070 con 0x7fb0681434d0 2024-04-25T11:29:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.181+0000 7fb066000640 1 -- 172.21.15.19:0/4219183571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058032a00 con 0x7fb0681434d0 2024-04-25T11:29:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 -- 172.21.15.19:0/4219183571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 msgr2=0x7fb068070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 --2- 172.21.15.19:0/4219183571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb068070480 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb05800bb10 tx=0x7fb05802c920 comp rx=0 tx=0).stop 2024-04-25T11:29:55.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 -- 172.21.15.19:0/4219183571 shutdown_connections 2024-04-25T11:29:55.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 --2- 172.21.15.19:0/4219183571 >> v2:172.21.15.169:3300/0 conn(0x7fb0681612f0 0x7fb0680d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 --2- 172.21.15.19:0/4219183571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb068070480 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 -- 172.21.15.19:0/4219183571 >> 172.21.15.19:0/4219183571 conn(0x7fb0680d1ae0 msgr2=0x7fb0680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:55.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 -- 172.21.15.19:0/4219183571 shutdown_connections 2024-04-25T11:29:55.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.182+0000 7fb06e800640 1 -- 172.21.15.19:0/4219183571 wait complete. 2024-04-25T11:29:55.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.185+0000 7fb06e800640 1 Processor -- start 2024-04-25T11:29:55.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.185+0000 7fb06e800640 1 -- start start 2024-04-25T11:29:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.186+0000 7fb06e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0680d0ed0 0x7fb06813d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.186+0000 7fb06e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb06813fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.186+0000 7fb06e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06815a550 0x7fb06813ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.186+0000 7fb06e800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb06806f090 con 0x7fb0681434d0 2024-04-25T11:29:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.186+0000 7fb06e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb06806fc80 con 0x7fb0680d0ed0 2024-04-25T11:29:55.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.186+0000 7fb06e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb0680b51a0 con 0x7fb06815a550 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.186+0000 7fb066a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb06813fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb06813fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33938/0 (socket says 172.21.15.19:33938) 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 -- 172.21.15.19:0/3217321325 learned_addr learned my addr 172.21.15.19:0/3217321325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb067400640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0680d0ed0 0x7fb06813d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 -- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06815a550 msgr2=0x7fb06813ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb067e00640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06815a550 0x7fb06813ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06815a550 0x7fb06813ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 -- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0680d0ed0 msgr2=0x7fb06813d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0680d0ed0 0x7fb06813d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 -- 172.21.15.19:0/3217321325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06815ada0 con 0x7fb0681434d0 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb067400640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0680d0ed0 0x7fb06813d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:55.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb067e00640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06815a550 0x7fb06813ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.187+0000 7fb066a00640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb06813fa90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb05400b3c0 tx=0x7fb05400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.188+0000 7fb05f400640 1 -- 172.21.15.19:0/3217321325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054011830 con 0x7fb0681434d0 2024-04-25T11:29:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.188+0000 7fb05f400640 1 -- 172.21.15.19:0/3217321325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb054016070 con 0x7fb0681434d0 2024-04-25T11:29:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.188+0000 7fb05f400640 1 -- 172.21.15.19:0/3217321325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054011c30 con 0x7fb0681434d0 2024-04-25T11:29:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.188+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb068146640 con 0x7fb0681434d0 2024-04-25T11:29:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.188+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0680b1a90 con 0x7fb0681434d0 2024-04-25T11:29:55.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.189+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0681463a0 con 0x7fb0681434d0 2024-04-25T11:29:55.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.190+0000 7fb05f400640 1 -- 172.21.15.19:0/3217321325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb05401b460 con 0x7fb0681434d0 2024-04-25T11:29:55.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.191+0000 7fb05f400640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744b0 0x7fb0480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.191+0000 7fb05f400640 1 -- 172.21.15.19:0/3217321325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fb054095070 con 0x7fb0681434d0 2024-04-25T11:29:55.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.192+0000 7fb067400640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744b0 0x7fb0480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.193+0000 7fb067400640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744b0 0x7fb0480769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb058037000 tx=0x7fb058032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:55.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.196+0000 7fb05f400640 1 -- 172.21.15.19:0/3217321325 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0681463a0 con 0x7fb0681434d0 2024-04-25T11:29:55.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.429+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb068050740 con 0x7fb0681434d0 2024-04-25T11:29:55.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.430+0000 7fb05f400640 1 -- 172.21.15.19:0/3217321325 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5179 (secure 0 0 0) 0x7fb05405f6a0 con 0x7fb0681434d0 2024-04-25T11:29:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:55.422 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:19.352476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5331,"qdb_cluster":[5331]},"id":2}]} 2024-04-25T11:29:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T11:29:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.432+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744b0 msgr2=0x7fb0480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.432+0000 7fb06e800640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744b0 0x7fb0480769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb058037000 tx=0x7fb058032000 comp rx=0 tx=0).stop 2024-04-25T11:29:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.432+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 msgr2=0x7fb06813fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.432+0000 7fb06e800640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb06813fa90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb05400b3c0 tx=0x7fb05400b880 comp rx=0 tx=0).stop 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 shutdown_connections 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0480744b0 0x7fb0480769c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb06815a550 0x7fb06813ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0681434d0 0x7fb06813fa90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 --2- 172.21.15.19:0/3217321325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0680d0ed0 0x7fb06813d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 >> 172.21.15.19:0/3217321325 conn(0x7fb0680d1ae0 msgr2=0x7fb06806cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 shutdown_connections 2024-04-25T11:29:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.433+0000 7fb06e800640 1 -- 172.21.15.19:0/3217321325 wait complete. 2024-04-25T11:29:55.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:29:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.722+0000 7faa97200640 1 Processor -- start 2024-04-25T11:29:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.722+0000 7faa97200640 1 -- start start 2024-04-25T11:29:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.724+0000 7faa97200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faa980d3350 0x7faa9806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.724+0000 7faa97200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980d3760 con 0x7faa98163e10 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.724+0000 7faa97200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980d22b0 con 0x7faa980d1dd0 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.724+0000 7faa97200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faa980d2ec0 con 0x7faa980d3350 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.725+0000 7faa96800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faa980d3350 0x7faa9806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.725+0000 7faa96800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faa980d3350 0x7faa9806dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33944/0 (socket says 172.21.15.19:33944) 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.725+0000 7faa96800640 1 -- 172.21.15.19:0/1879909448 learned_addr learned my addr 172.21.15.19:0/1879909448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.725+0000 7faa96800640 1 -- 172.21.15.19:0/1879909448 >> v1:172.21.15.169:6789/0 conn(0x7faa980d1dd0 legacy=0x7faa980cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.725+0000 7faa96800640 1 -- 172.21.15.19:0/1879909448 >> v1:172.21.15.19:6789/0 conn(0x7faa98163e10 legacy=0x7faa980ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.725+0000 7faa96800640 1 -- 172.21.15.19:0/1879909448 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980d2590 con 0x7faa980d3350 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.726+0000 7faa96800640 1 --2- 172.21.15.19:0/1879909448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa9806dda0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faa90009e50 tx=0x7faa9002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae4f77f70d72d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa94a00640 1 -- 172.21.15.19:0/1879909448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90036070 con 0x7faa980d3350 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa94a00640 1 -- 172.21.15.19:0/1879909448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa90031600 con 0x7faa980d3350 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa94a00640 1 -- 172.21.15.19:0/1879909448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900319d0 con 0x7faa980d3350 2024-04-25T11:29:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa97200640 1 -- 172.21.15.19:0/1879909448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 msgr2=0x7faa9806dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa97200640 1 --2- 172.21.15.19:0/1879909448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa9806dda0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7faa90009e50 tx=0x7faa9002a6d0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa97200640 1 -- 172.21.15.19:0/1879909448 shutdown_connections 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa97200640 1 --2- 172.21.15.19:0/1879909448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa9806dda0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa97200640 1 -- 172.21.15.19:0/1879909448 >> 172.21.15.19:0/1879909448 conn(0x7faa980b5490 msgr2=0x7faa980b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.727+0000 7faa97200640 1 -- 172.21.15.19:0/1879909448 shutdown_connections 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.728+0000 7faa97200640 1 -- 172.21.15.19:0/1879909448 wait complete. 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.730+0000 7faa97200640 1 Processor -- start 2024-04-25T11:29:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.730+0000 7faa97200640 1 -- start start 2024-04-25T11:29:55.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.731+0000 7faa97200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa980d1dd0 0x7faa98162a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.731+0000 7faa97200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa98155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.731+0000 7faa97200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa98163e10 0x7faa980c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.731+0000 7faa97200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b9330 con 0x7faa980d3350 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.731+0000 7faa97200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa980d2b50 con 0x7faa98163e10 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.731+0000 7faa97200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faa9806e2e0 con 0x7faa980d1dd0 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa98155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa98155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33950/0 (socket says 172.21.15.19:33950) 2024-04-25T11:29:55.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 -- 172.21.15.19:0/3526847662 learned_addr learned my addr 172.21.15.19:0/3526847662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:55.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa96800640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa98163e10 0x7faa980c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95e00640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa980d1dd0 0x7faa98162a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 -- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa980d1dd0 msgr2=0x7faa98162a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa980d1dd0 0x7faa98162a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 -- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa98163e10 msgr2=0x7faa980c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa98163e10 0x7faa980c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 -- 172.21.15.19:0/3526847662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980c86e0 con 0x7faa980d3350 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.732+0000 7faa95400640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa98155ff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c3f0 tx=0x7faa8400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.733+0000 7faa8e000640 1 -- 172.21.15.19:0/3526847662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840052d0 con 0x7faa980d3350 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.733+0000 7faa8e000640 1 -- 172.21.15.19:0/3526847662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa84005490 con 0x7faa980d3350 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.733+0000 7faa8e000640 1 -- 172.21.15.19:0/3526847662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84007260 con 0x7faa980d3350 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.733+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa98161a10 con 0x7faa980d3350 2024-04-25T11:29:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.733+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa98161fd0 con 0x7faa980d3350 2024-04-25T11:29:55.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.734+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa98050740 con 0x7faa980d3350 2024-04-25T11:29:55.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.735+0000 7faa8e000640 1 -- 172.21.15.19:0/3526847662 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faa84007400 con 0x7faa980d3350 2024-04-25T11:29:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.736+0000 7faa8e000640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa780744b0 0x7faa780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.736+0000 7faa8e000640 1 -- 172.21.15.19:0/3526847662 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7faa840947d0 con 0x7faa980d3350 2024-04-25T11:29:55.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.737+0000 7faa95e00640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa780744b0 0x7faa780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:55.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.738+0000 7faa95e00640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa780744b0 0x7faa780769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa980b9a10 tx=0x7faa88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.741+0000 7faa8e000640 1 -- 172.21.15.19:0/3526847662 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa8405ee80 con 0x7faa980d3350 2024-04-25T11:29:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.983+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa980b3050 con 0x7faa980d3350 2024-04-25T11:29:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.985+0000 7faa8e000640 1 -- 172.21.15.19:0/3526847662 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5179 (secure 0 0 0) 0x7faa84065e20 con 0x7faa980d3350 2024-04-25T11:29:55.976 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:55.976 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:28:45.819822+0000","modified":"2024-04-25T11:29:19.352476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5331":{"gid":5331,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.19:6835/3972745609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3972745609},{"type":"v1","addr":"172.21.15.19:6835","nonce":3972745609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5331,"qdb_cluster":[5331]},"id":2}]} 2024-04-25T11:29:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 25 2024-04-25T11:29:55.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa780744b0 msgr2=0x7faa780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa780744b0 0x7faa780769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa980b9a10 tx=0x7faa88008040 comp rx=0 tx=0).stop 2024-04-25T11:29:55.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 msgr2=0x7faa98155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa98155ff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c3f0 tx=0x7faa8400c8b0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 shutdown_connections 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa780744b0 0x7faa780769c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa98163e10 0x7faa980c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa980d3350 0x7faa98155ff0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 --2- 172.21.15.19:0/3526847662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa980d1dd0 0x7faa98162a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 >> 172.21.15.19:0/3526847662 conn(0x7faa980b5490 msgr2=0x7faa980b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 shutdown_connections 2024-04-25T11:29:55.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:55.987+0000 7faa97200640 1 -- 172.21.15.19:0/3526847662 wait complete. 2024-04-25T11:29:56.002 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:29:56.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.290+0000 7fda43c00640 1 Processor -- start 2024-04-25T11:29:56.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.290+0000 7fda43c00640 1 -- start start 2024-04-25T11:29:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda43c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda3c143440 0x7fda3c160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda43c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c1610a0 0x7fda3c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda43c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3c1614b0 con 0x7fda3c0d0830 2024-04-25T11:29:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda43c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c06f4a0 con 0x7fda3c143440 2024-04-25T11:29:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda43c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c070940 con 0x7fda3c1610a0 2024-04-25T11:29:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda41c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fda3c0d0830 0x7fda3c15d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43274/0 (socket says 172.21.15.19:43274) 2024-04-25T11:29:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda41c00640 1 -- 172.21.15.19:0/3144162447 learned_addr learned my addr 172.21.15.19:0/3144162447 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.293+0000 7fda41200640 1 --2- 172.21.15.19:0/3144162447 >> v2:172.21.15.19:3300/0 conn(0x7fda3c143440 0x7fda3c160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.294+0000 7fda3be00640 1 --2- 172.21.15.19:0/3144162447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c1610a0 0x7fda3c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.294+0000 7fda41200640 1 -- 172.21.15.19:0/3144162447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c1610a0 msgr2=0x7fda3c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.294+0000 7fda41200640 1 --2- 172.21.15.19:0/3144162447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c1610a0 0x7fda3c0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.294+0000 7fda41200640 1 -- 172.21.15.19:0/3144162447 >> v1:172.21.15.19:6789/0 conn(0x7fda3c0d0830 legacy=0x7fda3c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.294+0000 7fda41200640 1 -- 172.21.15.19:0/3144162447 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c143850 con 0x7fda3c143440 2024-04-25T11:29:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.294+0000 7fda3b400640 1 -- 172.21.15.19:0/3144162447 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923522696 0 0) 0x7fda34001110 con 0x7fda3c0d0830 2024-04-25T11:29:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.295+0000 7fda41200640 1 --2- 172.21.15.19:0/3144162447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c160b60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009d90 tx=0x7fda2c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea89a5663b7a413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.296+0000 7fda3b400640 1 -- 172.21.15.19:0/3144162447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c031600 con 0x7fda3c143440 2024-04-25T11:29:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.296+0000 7fda3b400640 1 -- 172.21.15.19:0/3144162447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c0317a0 con 0x7fda3c143440 2024-04-25T11:29:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.296+0000 7fda3b400640 1 -- 172.21.15.19:0/3144162447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c031b70 con 0x7fda3c143440 2024-04-25T11:29:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.296+0000 7fda43c00640 1 -- 172.21.15.19:0/3144162447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 msgr2=0x7fda3c160b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.296+0000 7fda43c00640 1 --2- 172.21.15.19:0/3144162447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c160b60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009d90 tx=0x7fda2c02a5d0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.297+0000 7fda43c00640 1 -- 172.21.15.19:0/3144162447 shutdown_connections 2024-04-25T11:29:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.297+0000 7fda43c00640 1 --2- 172.21.15.19:0/3144162447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c1610a0 0x7fda3c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.297+0000 7fda43c00640 1 --2- 172.21.15.19:0/3144162447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c160b60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.297+0000 7fda43c00640 1 -- 172.21.15.19:0/3144162447 >> 172.21.15.19:0/3144162447 conn(0x7fda3c0d1ae0 msgr2=0x7fda3c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.297+0000 7fda43c00640 1 -- 172.21.15.19:0/3144162447 shutdown_connections 2024-04-25T11:29:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.297+0000 7fda43c00640 1 -- 172.21.15.19:0/3144162447 wait complete. 2024-04-25T11:29:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.299+0000 7fda43c00640 1 Processor -- start 2024-04-25T11:29:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.299+0000 7fda43c00640 1 -- start start 2024-04-25T11:29:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda43c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0d0830 0x7fda3c140f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda43c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda43c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c1610a0 0x7fda3c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda43c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c0d1400 con 0x7fda3c143440 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda43c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c0d1120 con 0x7fda3c1610a0 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda43c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c06f130 con 0x7fda3c0d0830 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda3be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda3be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c0ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33968/0 (socket says 172.21.15.19:33968) 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.301+0000 7fda3be00640 1 -- 172.21.15.19:0/3290252293 learned_addr learned my addr 172.21.15.19:0/3290252293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda41200640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0d0830 0x7fda3c140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda41c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c1610a0 0x7fda3c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda3be00640 1 -- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0d0830 msgr2=0x7fda3c140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda3be00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0d0830 0x7fda3c140f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda3be00640 1 -- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c1610a0 msgr2=0x7fda3c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda3be00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c1610a0 0x7fda3c0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda3be00640 1 -- 172.21.15.19:0/3290252293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c1ea770 con 0x7fda3c143440 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda41200640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0d0830 0x7fda3c140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda41c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c1610a0 0x7fda3c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda3be00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c0ccf00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d5f0 tx=0x7fda2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda38c00640 1 -- 172.21.15.19:0/3290252293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240125f0 con 0x7fda3c143440 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda38c00640 1 -- 172.21.15.19:0/3290252293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda24019070 con 0x7fda3c143440 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda38c00640 1 -- 172.21.15.19:0/3290252293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240129d0 con 0x7fda3c143440 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.302+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c1eaa50 con 0x7fda3c143440 2024-04-25T11:29:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.303+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c13ecd0 con 0x7fda3c143440 2024-04-25T11:29:56.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.304+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c050740 con 0x7fda3c143440 2024-04-25T11:29:56.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.305+0000 7fda38c00640 1 -- 172.21.15.19:0/3290252293 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda24012b70 con 0x7fda3c143440 2024-04-25T11:29:56.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.307+0000 7fda38c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda180744b0 0x7fda180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.307+0000 7fda38c00640 1 -- 172.21.15.19:0/3290252293 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fda24095740 con 0x7fda3c143440 2024-04-25T11:29:56.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.307+0000 7fda41200640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda180744b0 0x7fda180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.309+0000 7fda41200640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda180744b0 0x7fda180769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda2c034000 tx=0x7fda2c033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:56.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.312+0000 7fda38c00640 1 -- 172.21.15.19:0/3290252293 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda2405fdf0 con 0x7fda3c143440 2024-04-25T11:29:56.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.518+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda3c069a60 con 0x7fda3c143440 2024-04-25T11:29:56.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.522+0000 7fda38c00640 1 -- 172.21.15.19:0/3290252293 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16987 (secure 0 0 0) 0x7fda2405f6f0 con 0x7fda3c143440 2024-04-25T11:29:56.513 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:29:56.513 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:29:06.584671+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:28:43.580410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T11:28:44.810835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3831883468":"2024-04-26T11:29:00.304477+0000","172.21.15.19:6833/3831883468":"2024-04-26T11:29:00.304477+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:29:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda180744b0 msgr2=0x7fda180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda180744b0 0x7fda180769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda2c034000 tx=0x7fda2c033040 comp rx=0 tx=0).stop 2024-04-25T11:29:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 msgr2=0x7fda3c0ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c0ccf00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda2400d5f0 tx=0x7fda2400dab0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 shutdown_connections 2024-04-25T11:29:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda180744b0 0x7fda180769c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3c1610a0 0x7fda3c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda3c143440 0x7fda3c0ccf00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 --2- 172.21.15.19:0/3290252293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda3c0d0830 0x7fda3c140f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 >> 172.21.15.19:0/3290252293 conn(0x7fda3c0d1ae0 msgr2=0x7fda3c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 shutdown_connections 2024-04-25T11:29:56.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.525+0000 7fda43c00640 1 -- 172.21.15.19:0/3290252293 wait complete. 2024-04-25T11:29:56.541 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:29:56.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.822+0000 7fe1ca200640 1 Processor -- start 2024-04-25T11:29:56.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.823+0000 7fe1ca200640 1 -- start start 2024-04-25T11:29:56.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.825+0000 7fe1ca200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe1c4070e20 0x7fe1c415c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.825+0000 7fe1ca200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 0x7fe1c406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.825+0000 7fe1ca200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c406d7a0 con 0x7fe1c406d390 2024-04-25T11:29:56.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.825+0000 7fe1ca200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c4071230 con 0x7fe1c4070e20 2024-04-25T11:29:56.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.825+0000 7fe1ca200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c406f8f0 con 0x7fe1c406f180 2024-04-25T11:29:56.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.826+0000 7fe1c9800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe1c406d390 0x7fe1c40cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43280/0 (socket says 172.21.15.19:43280) 2024-04-25T11:29:56.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.826+0000 7fe1c9800640 1 -- 172.21.15.19:0/1413737679 learned_addr learned my addr 172.21.15.19:0/1413737679 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:56.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.826+0000 7fe1c8e00640 1 --2- 172.21.15.19:0/1413737679 >> v2:172.21.15.169:3300/0 conn(0x7fe1c4070e20 0x7fe1c415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.826+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 0x7fe1c406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.826+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731632486 0 0) 0x7fe1bc001110 con 0x7fe1c406d390 2024-04-25T11:29:56.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.826+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1a8005730 con 0x7fe1c406d390 2024-04-25T11:29:56.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3287665611 0 0) 0x7fe1a8005730 con 0x7fe1c406d390 2024-04-25T11:29:56.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 msgr2=0x7fe1c406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:56.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 0x7fe1c406ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 >> v2:172.21.15.169:3300/0 conn(0x7fe1c4070e20 msgr2=0x7fe1c415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:56.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 --2- 172.21.15.19:0/1413737679 >> v2:172.21.15.169:3300/0 conn(0x7fe1c4070e20 0x7fe1c415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c406fbd0 con 0x7fe1c406d390 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe1bc001840 con 0x7fe1c406d390 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 >> v1:172.21.15.19:6789/0 conn(0x7fe1c406d390 legacy=0x7fe1c40cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8005bc0 0x7fe1a8006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a8006a10 0x7fe1a8006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 0x7fe1a8007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4071230 con 0x7fe1a8006a10 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c406f8f0 con 0x7fe1a8005bc0 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a8005560 con 0x7fe1c406f180 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a8006a10 0x7fe1a8006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.827+0000 7fe1c8e00640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8005bc0 0x7fe1a8006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.828+0000 7fe1c9800640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 0x7fe1a8007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.828+0000 7fe1c3e00640 1 -- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 msgr2=0x7fe1a8007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:56.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.828+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c406f180 0x7fe1a8007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.828+0000 7fe1c3e00640 1 -- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8005bc0 msgr2=0x7fe1a8006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:56.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.828+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8005bc0 0x7fe1a8006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.828+0000 7fe1c3e00640 1 -- 172.21.15.19:0/1413737679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a8005730 con 0x7fe1a8006a10 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.829+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a8006a10 0x7fe1a8006e20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009fc0 tx=0x7fe1a8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a860ae7a253d972b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.829+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0034020 con 0x7fe1a8006a10 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.829+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b00305f0 con 0x7fe1a8006a10 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.829+0000 7fe1c3400640 1 -- 172.21.15.19:0/1413737679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0009c10 con 0x7fe1a8006a10 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.830+0000 7fe1ca200640 1 -- 172.21.15.19:0/1413737679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a8006a10 msgr2=0x7fe1a8006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.830+0000 7fe1ca200640 1 --2- 172.21.15.19:0/1413737679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a8006a10 0x7fe1a8006e20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009fc0 tx=0x7fe1a8007a40 comp rx=0 tx=0).stop 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.830+0000 7fe1c9800640 1 -- 172.21.15.19:0/1413737679 reap_dead start 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.830+0000 7fe1ca200640 1 -- 172.21.15.19:0/1413737679 shutdown_connections 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.830+0000 7fe1ca200640 1 -- 172.21.15.19:0/1413737679 >> 172.21.15.19:0/1413737679 conn(0x7fe1c40d1720 msgr2=0x7fe1c40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.830+0000 7fe1ca200640 1 -- 172.21.15.19:0/1413737679 shutdown_connections 2024-04-25T11:29:56.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.830+0000 7fe1ca200640 1 -- 172.21.15.19:0/1413737679 wait complete. 2024-04-25T11:29:56.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.833+0000 7fe1ca200640 1 Processor -- start 2024-04-25T11:29:56.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.833+0000 7fe1ca200640 1 -- start start 2024-04-25T11:29:56.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.834+0000 7fe1ca200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8006a10 0x7fe1c413cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1ca200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c41432e0 0x7fe1c413d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1ca200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c415dfc0 0x7fe1c41f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1ca200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c40b6e30 con 0x7fe1c41432e0 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1ca200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4070190 con 0x7fe1a8006a10 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1ca200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c406f590 con 0x7fe1c415dfc0 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c41432e0 0x7fe1c413d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c41432e0 0x7fe1c413d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33980/0 (socket says 172.21.15.19:33980) 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 -- 172.21.15.19:0/2519508345 learned_addr learned my addr 172.21.15.19:0/2519508345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c8e00640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8006a10 0x7fe1c413cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c9800640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c415dfc0 0x7fe1c41f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 -- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c415dfc0 msgr2=0x7fe1c41f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c415dfc0 0x7fe1c41f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 -- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8006a10 msgr2=0x7fe1c413cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8006a10 0x7fe1c413cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c3e00640 1 -- 172.21.15.19:0/2519508345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c415cf20 con 0x7fe1c41432e0 2024-04-25T11:29:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.835+0000 7fe1c9800640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c415dfc0 0x7fe1c41f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:56.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.836+0000 7fe1c3e00640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c41432e0 0x7fe1c413d720 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000e230 tx=0x7fe1b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.836+0000 7fe1c0c00640 1 -- 172.21.15.19:0/2519508345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0040070 con 0x7fe1c41432e0 2024-04-25T11:29:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.836+0000 7fe1c0c00640 1 -- 172.21.15.19:0/2519508345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b0003990 con 0x7fe1c41432e0 2024-04-25T11:29:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.836+0000 7fe1c0c00640 1 -- 172.21.15.19:0/2519508345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0034040 con 0x7fe1c41432e0 2024-04-25T11:29:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.836+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c415d200 con 0x7fe1c41432e0 2024-04-25T11:29:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.836+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c415d7c0 con 0x7fe1c41432e0 2024-04-25T11:29:56.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.838+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c4050740 con 0x7fe1c41432e0 2024-04-25T11:29:56.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.839+0000 7fe1c0c00640 1 -- 172.21.15.19:0/2519508345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1b0037020 con 0x7fe1c41432e0 2024-04-25T11:29:56.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.840+0000 7fe1c0c00640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1a40744b0 0x7fe1a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:56.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.840+0000 7fe1c0c00640 1 -- 172.21.15.19:0/2519508345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fe1b00b5fd0 con 0x7fe1c41432e0 2024-04-25T11:29:56.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.841+0000 7fe1c8e00640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1a40744b0 0x7fe1a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:56.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.842+0000 7fe1c8e00640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1a40744b0 0x7fe1a40769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009a90 tx=0x7fe1b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:56.848+0000 7fe1c0c00640 1 -- 172.21.15.19:0/2519508345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1b00805f0 con 0x7fe1c41432e0 2024-04-25T11:29:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.098+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe1c415cc40 con 0x7fe1c41432e0 2024-04-25T11:29:57.090 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:29:57.099+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:29:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.104+0000 7fe1c0c00640 1 -- 172.21.15.19:0/2519508345 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fe1b007fef0 con 0x7fe1c41432e0 2024-04-25T11:29:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:29:57.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1a40744b0 msgr2=0x7fe1a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:57.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1a40744b0 0x7fe1a40769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009a90 tx=0x7fe1b4008040 comp rx=0 tx=0).stop 2024-04-25T11:29:57.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c41432e0 msgr2=0x7fe1c413d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:57.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c41432e0 0x7fe1c413d720 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000e230 tx=0x7fe1b0009750 comp rx=0 tx=0).stop 2024-04-25T11:29:57.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 shutdown_connections 2024-04-25T11:29:57.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1a40744b0 0x7fe1a40769c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c415dfc0 0x7fe1c41f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c41432e0 0x7fe1c413d720 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 --2- 172.21.15.19:0/2519508345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a8006a10 0x7fe1c413cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 >> 172.21.15.19:0/2519508345 conn(0x7fe1c40d1720 msgr2=0x7fe1c415b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:57.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 shutdown_connections 2024-04-25T11:29:57.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.107+0000 7fe1ca200640 1 -- 172.21.15.19:0/2519508345 wait complete. 2024-04-25T11:29:57.113 INFO:tasks.ceph.mds.a.smithi019.stderr: -3866> 2024-04-25T11:29:07.341+0000 7fa144000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:29:57.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:29:57.263 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:29:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.408+0000 7f418fe00640 1 Processor -- start 2024-04-25T11:29:57.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.408+0000 7f418fe00640 1 -- start start 2024-04-25T11:29:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.410+0000 7f418fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.410+0000 7f418fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f419006e0a0 0x7f41900b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.410+0000 7f418fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f419006e4b0 con 0x7f41900d56c0 2024-04-25T11:29:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.410+0000 7f418fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f41900d43f0 con 0x7f419006e0a0 2024-04-25T11:29:57.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.410+0000 7f418fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f41900b53c0 con 0x7f4190160440 2024-04-25T11:29:57.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.411+0000 7f418e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:57.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.411+0000 7f418e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419006db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:40202/0 (socket says 172.21.15.19:40202) 2024-04-25T11:29:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.411+0000 7f418e000640 1 -- 172.21.15.19:0/2948010295 learned_addr learned my addr 172.21.15.19:0/2948010295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.411+0000 7f418ea00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f41900d56c0 0x7f41900b4cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:34482/0 (socket says 172.21.15.19:34482) 2024-04-25T11:29:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.411+0000 7f418f400640 1 --2- 172.21.15.19:0/2948010295 >> v2:172.21.15.169:3300/0 conn(0x7f419006e0a0 0x7f41900b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.412+0000 7f418e000640 1 -- 172.21.15.19:0/2948010295 >> v2:172.21.15.169:3300/0 conn(0x7f419006e0a0 msgr2=0x7f41900b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.412+0000 7f418e000640 1 --2- 172.21.15.19:0/2948010295 >> v2:172.21.15.169:3300/0 conn(0x7f419006e0a0 0x7f41900b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.412+0000 7f418e000640 1 -- 172.21.15.19:0/2948010295 >> v1:172.21.15.169:6789/0 conn(0x7f41900d56c0 legacy=0x7f41900b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.412+0000 7f418e000640 1 -- 172.21.15.19:0/2948010295 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41900d37e0 con 0x7f4190160440 2024-04-25T11:29:57.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.413+0000 7f418e000640 1 --2- 172.21.15.19:0/2948010295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419006db60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4178009870 tx=0x7f417802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708a5e23562d1c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418d600640 1 -- 172.21.15.19:0/2948010295 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178037070 con 0x7f4190160440 2024-04-25T11:29:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418d600640 1 -- 172.21.15.19:0/2948010295 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4178032440 con 0x7f4190160440 2024-04-25T11:29:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418d600640 1 -- 172.21.15.19:0/2948010295 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178032810 con 0x7f4190160440 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418fe00640 1 -- 172.21.15.19:0/2948010295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 msgr2=0x7f419006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418fe00640 1 --2- 172.21.15.19:0/2948010295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419006db60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4178009870 tx=0x7f417802a270 comp rx=0 tx=0).stop 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418fe00640 1 -- 172.21.15.19:0/2948010295 shutdown_connections 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418fe00640 1 --2- 172.21.15.19:0/2948010295 >> v2:172.21.15.169:3300/0 conn(0x7f419006e0a0 0x7f41900b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418fe00640 1 --2- 172.21.15.19:0/2948010295 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419006db60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.414+0000 7f418fe00640 1 -- 172.21.15.19:0/2948010295 >> 172.21.15.19:0/2948010295 conn(0x7f41900d2c20 msgr2=0x7f41900d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.415+0000 7f418fe00640 1 -- 172.21.15.19:0/2948010295 shutdown_connections 2024-04-25T11:29:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.415+0000 7f418fe00640 1 -- 172.21.15.19:0/2948010295 wait complete. 2024-04-25T11:29:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.417+0000 7f418fe00640 1 Processor -- start 2024-04-25T11:29:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.417+0000 7f418fe00640 1 -- start start 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419006e0a0 0x7f41901f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41901e5560 0x7f41900b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41900d4080 con 0x7f419006e0a0 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f41900d3da0 con 0x7f41901e5560 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f41900d4880 con 0x7f4190160440 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419006e0a0 0x7f41901f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419006e0a0 0x7f41901f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33998/0 (socket says 172.21.15.19:33998) 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418ea00640 1 -- 172.21.15.19:0/3169882174 learned_addr learned my addr 172.21.15.19:0/3169882174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.419+0000 7f418e000640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418f400640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41901e5560 0x7f41900b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418ea00640 1 -- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 msgr2=0x7f419014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418ea00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419014f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418ea00640 1 -- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41901e5560 msgr2=0x7f41900b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418ea00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41901e5560 0x7f41900b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418ea00640 1 -- 172.21.15.19:0/3169882174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419014f550 con 0x7f419006e0a0 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418f400640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41901e5560 0x7f41900b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418ea00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419006e0a0 0x7f41901f6970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f418000c440 tx=0x7f418000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f4186a00640 1 -- 172.21.15.19:0/3169882174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800074e0 con 0x7f419006e0a0 2024-04-25T11:29:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f4186a00640 1 -- 172.21.15.19:0/3169882174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4180007680 con 0x7f419006e0a0 2024-04-25T11:29:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f4186a00640 1 -- 172.21.15.19:0/3169882174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180004a20 con 0x7f419006e0a0 2024-04-25T11:29:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.420+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41901f71e0 con 0x7f419006e0a0 2024-04-25T11:29:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.421+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41901f77a0 con 0x7f419006e0a0 2024-04-25T11:29:57.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.422+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4190076c60 con 0x7f419006e0a0 2024-04-25T11:29:57.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.423+0000 7f4186a00640 1 -- 172.21.15.19:0/3169882174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f418001a020 con 0x7f419006e0a0 2024-04-25T11:29:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.424+0000 7f4186a00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f416c0744e0 0x7f416c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.424+0000 7f4186a00640 1 -- 172.21.15.19:0/3169882174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5997+0+0 (secure 0 0 0) 0x7f4180094ad0 con 0x7f419006e0a0 2024-04-25T11:29:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.425+0000 7f418e000640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f416c0744e0 0x7f416c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.426+0000 7f418e000640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f416c0744e0 0x7f416c0769f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4178034000 tx=0x7f417802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:57.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.429+0000 7f4186a00640 1 -- 172.21.15.19:0/3169882174 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f419006e4b0 con 0x7f419006e0a0 2024-04-25T11:29:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:57.670+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f419015f2c0 con 0x7f419006e0a0 2024-04-25T11:29:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.107+0000 7f4186a00640 1 -- 172.21.15.19:0/3169882174 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f418005f140 con 0x7f419006e0a0 2024-04-25T11:29:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f416c0744e0 msgr2=0x7f416c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f416c0744e0 0x7f416c0769f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4178034000 tx=0x7f417802e000 comp rx=0 tx=0).stop 2024-04-25T11:29:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419006e0a0 msgr2=0x7f41901f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419006e0a0 0x7f41901f6970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f418000c440 tx=0x7f418000c900 comp rx=0 tx=0).stop 2024-04-25T11:29:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 shutdown_connections 2024-04-25T11:29:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f416c0744e0 0x7f416c0769f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41901e5560 0x7f41900b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4190160440 0x7f419014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 --2- 172.21.15.19:0/3169882174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419006e0a0 0x7f41901f6970 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 >> 172.21.15.19:0/3169882174 conn(0x7f41900d2c20 msgr2=0x7f41900d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 shutdown_connections 2024-04-25T11:29:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.111+0000 7f418fe00640 1 -- 172.21.15.19:0/3169882174 wait complete. 2024-04-25T11:29:58.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:29:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.430+0000 7f7af4a00640 1 Processor -- start 2024-04-25T11:29:58.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.431+0000 7f7af4a00640 1 -- start start 2024-04-25T11:29:58.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.433+0000 7f7af4a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7af015e160 0x7f7af00ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:58.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.433+0000 7f7af4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af0167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:58.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.433+0000 7f7af4a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af00cd440 con 0x7f7af006f370 2024-04-25T11:29:58.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.433+0000 7f7af4a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7af006cab0 con 0x7f7af015e160 2024-04-25T11:29:58.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.433+0000 7f7af4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7af006df50 con 0x7f7af0165b20 2024-04-25T11:29:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.434+0000 7f7aefe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af0167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.434+0000 7f7aefe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af0167f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:40216/0 (socket says 172.21.15.19:40216) 2024-04-25T11:29:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.434+0000 7f7aefe00640 1 -- 172.21.15.19:0/3212402663 learned_addr learned my addr 172.21.15.19:0/3212402663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.434+0000 7f7ae7e00640 1 --1- 172.21.15.19:0/3212402663 >> v1:172.21.15.169:6789/0 conn(0x7f7af006f370 0x7f7af006f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:34488/0 (socket says 172.21.15.19:34488) 2024-04-25T11:29:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.434+0000 7f7aef400640 1 --2- 172.21.15.19:0/3212402663 >> v2:172.21.15.169:3300/0 conn(0x7f7af015e160 0x7f7af00ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.435+0000 7f7aefe00640 1 -- 172.21.15.19:0/3212402663 >> v2:172.21.15.169:3300/0 conn(0x7f7af015e160 msgr2=0x7f7af00ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.435+0000 7f7aefe00640 1 --2- 172.21.15.19:0/3212402663 >> v2:172.21.15.169:3300/0 conn(0x7f7af015e160 0x7f7af00ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.435+0000 7f7aefe00640 1 -- 172.21.15.19:0/3212402663 >> v1:172.21.15.169:6789/0 conn(0x7f7af006f370 legacy=0x7f7af006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.435+0000 7f7aefe00640 1 -- 172.21.15.19:0/3212402663 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af00cdbe0 con 0x7f7af0165b20 2024-04-25T11:29:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.435+0000 7f7aef400640 1 --2- 172.21.15.19:0/3212402663 >> v2:172.21.15.169:3300/0 conn(0x7f7af015e160 0x7f7af00ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:29:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.436+0000 7f7aefe00640 1 --2- 172.21.15.19:0/3212402663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af0167f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009e50 tx=0x7f7ae802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae130c2690bbf660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.437+0000 7f7aeea00640 1 -- 172.21.15.19:0/3212402663 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8036070 con 0x7f7af0165b20 2024-04-25T11:29:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.437+0000 7f7aeea00640 1 -- 172.21.15.19:0/3212402663 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae8031600 con 0x7f7af0165b20 2024-04-25T11:29:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.437+0000 7f7aeea00640 1 -- 172.21.15.19:0/3212402663 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80319d0 con 0x7f7af0165b20 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.437+0000 7f7af4a00640 1 -- 172.21.15.19:0/3212402663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 msgr2=0x7f7af0167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.437+0000 7f7af4a00640 1 --2- 172.21.15.19:0/3212402663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af0167f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009e50 tx=0x7f7ae802a680 comp rx=0 tx=0).stop 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.438+0000 7f7af4a00640 1 -- 172.21.15.19:0/3212402663 shutdown_connections 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.438+0000 7f7af4a00640 1 --2- 172.21.15.19:0/3212402663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af0167f40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.438+0000 7f7af4a00640 1 --2- 172.21.15.19:0/3212402663 >> v2:172.21.15.169:3300/0 conn(0x7f7af015e160 0x7f7af00ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.438+0000 7f7af4a00640 1 -- 172.21.15.19:0/3212402663 >> 172.21.15.19:0/3212402663 conn(0x7f7af00ce3b0 msgr2=0x7f7af00ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.438+0000 7f7af4a00640 1 -- 172.21.15.19:0/3212402663 shutdown_connections 2024-04-25T11:29:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.438+0000 7f7af4a00640 1 -- 172.21.15.19:0/3212402663 wait complete. 2024-04-25T11:29:58.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.441+0000 7f7af4a00640 1 Processor -- start 2024-04-25T11:29:58.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.441+0000 7f7af4a00640 1 -- start start 2024-04-25T11:29:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7af4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af006f370 0x7f7af015f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7af4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af00cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7af4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7af00cb310 0x7f7af00c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7af4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7af006c740 con 0x7f7af00cb310 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7af4a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7af00714a0 con 0x7f7af006f370 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7af4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7af00b4820 con 0x7f7af0165b20 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7aefe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7af00cb310 0x7f7af00c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7aefe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7af00cb310 0x7f7af00c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34004/0 (socket says 172.21.15.19:34004) 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7aefe00640 1 -- 172.21.15.19:0/1899748223 learned_addr learned my addr 172.21.15.19:0/1899748223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.443+0000 7f7aef400640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af006f370 0x7f7af015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.444+0000 7f7ae7e00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af00cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.444+0000 7f7aefe00640 1 -- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 msgr2=0x7f7af00cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.444+0000 7f7aefe00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af00cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.444+0000 7f7aefe00640 1 -- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af006f370 msgr2=0x7f7af015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.444+0000 7f7aefe00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af006f370 0x7f7af015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.444+0000 7f7aefe00640 1 -- 172.21.15.19:0/1899748223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af015fd70 con 0x7f7af00cb310 2024-04-25T11:29:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.445+0000 7f7aefe00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7af00cb310 0x7f7af00c7a80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009f70 tx=0x7f7ae8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.445+0000 7f7ae7400640 1 -- 172.21.15.19:0/1899748223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae803f020 con 0x7f7af00cb310 2024-04-25T11:29:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.445+0000 7f7ae7400640 1 -- 172.21.15.19:0/1899748223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae8037040 con 0x7f7af00cb310 2024-04-25T11:29:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.445+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af00a08d0 con 0x7f7af00cb310 2024-04-25T11:29:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.445+0000 7f7ae7400640 1 -- 172.21.15.19:0/1899748223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8036070 con 0x7f7af00cb310 2024-04-25T11:29:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.445+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af00a0e90 con 0x7f7af00cb310 2024-04-25T11:29:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.447+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af006dd70 con 0x7f7af00cb310 2024-04-25T11:29:58.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.448+0000 7f7ae7400640 1 -- 172.21.15.19:0/1899748223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ae8042020 con 0x7f7af00cb310 2024-04-25T11:29:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.450+0000 7f7ae7400640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7acc0744b0 0x7f7acc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.450+0000 7f7ae7400640 1 -- 172.21.15.19:0/1899748223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5997+0+0 (secure 0 0 0) 0x7f7ae80b5660 con 0x7f7af00cb310 2024-04-25T11:29:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.450+0000 7f7aef400640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7acc0744b0 0x7f7acc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.452+0000 7f7aef400640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7acc0744b0 0x7f7acc0769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009560 tx=0x7f7ae0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.455+0000 7f7ae7400640 1 -- 172.21.15.19:0/1899748223 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7af006dd70 con 0x7f7af00cb310 2024-04-25T11:29:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:58.665+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7af0076c60 con 0x7f7af00cb310 2024-04-25T11:29:59.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.119+0000 7f7ae7400640 1 -- 172.21.15.19:0/1899748223 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) ==== 181+0+0 (secure 0 0 0) 0x7f7ae8049030 con 0x7f7af00cb310 2024-04-25T11:29:59.111 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:29:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7acc0744b0 msgr2=0x7f7acc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7acc0744b0 0x7f7acc0769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009560 tx=0x7f7ae0009190 comp rx=0 tx=0).stop 2024-04-25T11:29:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7af00cb310 msgr2=0x7f7af00c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7af00cb310 0x7f7af00c7a80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009f70 tx=0x7f7ae8004a40 comp rx=0 tx=0).stop 2024-04-25T11:29:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 shutdown_connections 2024-04-25T11:29:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7acc0744b0 0x7f7acc0769c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7af00cb310 0x7f7af00c7a80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7af0165b20 0x7f7af00cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 --2- 172.21.15.19:0/1899748223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7af006f370 0x7f7af015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.123+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 >> 172.21.15.19:0/1899748223 conn(0x7f7af00ce3b0 msgr2=0x7f7af00b6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.124+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 shutdown_connections 2024-04-25T11:29:59.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.124+0000 7f7af4a00640 1 -- 172.21.15.19:0/1899748223 wait complete. 2024-04-25T11:29:59.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:29:59.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.435+0000 7f1344800640 1 Processor -- start 2024-04-25T11:29:59.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.435+0000 7f1344800640 1 -- start start 2024-04-25T11:29:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.438+0000 7f1344800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f133c06dce0 0x7f133c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.438+0000 7f1344800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c0cdea0 con 0x7f133c0b0af0 2024-04-25T11:29:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.438+0000 7f1344800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c0ce180 con 0x7f133c0cd9a0 2024-04-25T11:29:59.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.438+0000 7f1344800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f133c0ceab0 con 0x7f133c06dce0 2024-04-25T11:29:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.438+0000 7f1342800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f133c0b0af0 0x7f133c15faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43290/0 (socket says 172.21.15.19:43290) 2024-04-25T11:29:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.438+0000 7f1342800640 1 -- 172.21.15.19:0/4101537115 learned_addr learned my addr 172.21.15.19:0/4101537115 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:29:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.438+0000 7f1341400640 1 --2- 172.21.15.19:0/4101537115 >> v2:172.21.15.169:3300/0 conn(0x7f133c06dce0 0x7f133c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3757782709 0 0) 0x7f133c0cdea0 con 0x7f133c0b0af0 2024-04-25T11:29:59.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1328003520 con 0x7f133c0b0af0 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 333822346 0 0) 0x7f1328003520 con 0x7f133c0b0af0 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 >> v2:172.21.15.169:3300/0 conn(0x7f133c06dce0 msgr2=0x7f133c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 --2- 172.21.15.19:0/4101537115 >> v2:172.21.15.169:3300/0 conn(0x7f133c06dce0 0x7f133c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 >> v1:172.21.15.169:6789/0 conn(0x7f133c0cd9a0 legacy=0x7f133c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c0ce460 con 0x7f133c0b0af0 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1338001670 con 0x7f133c0b0af0 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 >> v1:172.21.15.19:6789/0 conn(0x7f133c0b0af0 legacy=0x7f133c15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280039b0 0x7f1328003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1328004800 0x7f1328004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13280051c0 0x7f132800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0ceab0 con 0x7f13280051c0 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f132800c160 con 0x7f1328004800 2024-04-25T11:29:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.439+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f132800c360 con 0x7f13280039b0 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1342800640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13280051c0 0x7f132800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1341e00640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280039b0 0x7f1328003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1341400640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1328004800 0x7f1328004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1342800640 1 -- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280039b0 msgr2=0x7f1328003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1342800640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280039b0 0x7f1328003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1342800640 1 -- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1328004800 msgr2=0x7f1328004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1342800640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1328004800 0x7f1328004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.440+0000 7f1342800640 1 -- 172.21.15.19:0/4101537115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1328003520 con 0x7f13280051c0 2024-04-25T11:29:59.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.441+0000 7f1342800640 1 -- 172.21.15.19:0/4101537115 reap_dead start 2024-04-25T11:29:59.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.441+0000 7f1342800640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13280051c0 0x7f132800b9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f133800c240 tx=0x7f1328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329de501edd4efcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:59.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.442+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338007d10 con 0x7f13280051c0 2024-04-25T11:29:59.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.442+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338005ad0 con 0x7f13280051c0 2024-04-25T11:29:59.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.442+0000 7f1340a00640 1 -- 172.21.15.19:0/4101537115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338007070 con 0x7f13280051c0 2024-04-25T11:29:59.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.442+0000 7f1344800640 1 -- 172.21.15.19:0/4101537115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13280051c0 msgr2=0x7f132800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.442+0000 7f1344800640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13280051c0 0x7f132800b9c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f133800c240 tx=0x7f1328005830 comp rx=0 tx=0).stop 2024-04-25T11:29:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.443+0000 7f1344800640 1 -- 172.21.15.19:0/4101537115 shutdown_connections 2024-04-25T11:29:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.443+0000 7f1344800640 1 --2- 172.21.15.19:0/4101537115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13280051c0 0x7f132800b9c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.443+0000 7f1344800640 1 -- 172.21.15.19:0/4101537115 >> 172.21.15.19:0/4101537115 conn(0x7f133c073090 msgr2=0x7f133c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:29:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.443+0000 7f1344800640 1 -- 172.21.15.19:0/4101537115 shutdown_connections 2024-04-25T11:29:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.443+0000 7f1344800640 1 -- 172.21.15.19:0/4101537115 wait complete. 2024-04-25T11:29:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.445+0000 7f1344800640 1 Processor -- start 2024-04-25T11:29:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.445+0000 7f1344800640 1 -- start start 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1344800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280051c0 0x7f133c157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1344800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f133c13f210 0x7f133c146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1344800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f133c146e90 0x7f133c144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1344800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0cef40 con 0x7f133c13f210 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1344800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0b0f00 con 0x7f133c146e90 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1344800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f133c0cf0c0 con 0x7f13280051c0 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1341400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f133c13f210 0x7f133c146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1341400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f133c13f210 0x7f133c146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34028/0 (socket says 172.21.15.19:34028) 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1341400640 1 -- 172.21.15.19:0/2067604795 learned_addr learned my addr 172.21.15.19:0/2067604795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1342800640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f133c146e90 0x7f133c144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.447+0000 7f1341e00640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280051c0 0x7f133c157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1341400640 1 -- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280051c0 msgr2=0x7f133c157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1341400640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280051c0 0x7f133c157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1341400640 1 -- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f133c146e90 msgr2=0x7f133c144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:29:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1341400640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f133c146e90 0x7f133c144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:29:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1341400640 1 -- 172.21.15.19:0/2067604795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c143770 con 0x7f133c13f210 2024-04-25T11:29:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1341e00640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280051c0 0x7f133c157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:29:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1341400640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f133c13f210 0x7f133c146950 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f132c011570 tx=0x7f132c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1332000640 1 -- 172.21.15.19:0/2067604795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c015930 con 0x7f133c13f210 2024-04-25T11:29:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1332000640 1 -- 172.21.15.19:0/2067604795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132c015ad0 con 0x7f133c13f210 2024-04-25T11:29:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c143a50 con 0x7f133c13f210 2024-04-25T11:29:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1332000640 1 -- 172.21.15.19:0/2067604795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c00f460 con 0x7f133c13f210 2024-04-25T11:29:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.448+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c144010 con 0x7f133c13f210 2024-04-25T11:29:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.449+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c050740 con 0x7f133c13f210 2024-04-25T11:29:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.451+0000 7f1332000640 1 -- 172.21.15.19:0/2067604795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f132c00f600 con 0x7f133c13f210 2024-04-25T11:29:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.452+0000 7f1332000640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13200744b0 0x7f13200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:29:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.452+0000 7f1332000640 1 -- 172.21.15.19:0/2067604795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5559+0+0 (secure 0 0 0) 0x7f132c095bd0 con 0x7f133c13f210 2024-04-25T11:29:59.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.453+0000 7f1341e00640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13200744b0 0x7f13200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:29:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.454+0000 7f1341e00640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13200744b0 0x7f13200769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1334000f80 tx=0x7f1334009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:29:59.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.457+0000 7f1332000640 1 -- 172.21.15.19:0/2067604795 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f132c019030 con 0x7f133c13f210 2024-04-25T11:29:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:29:59.659+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f133c143500 con 0x7f133c13f210 2024-04-25T11:30:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.134+0000 7f1332000640 1 -- 172.21.15.19:0/2067604795 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) ==== 169+0+0 (secure 0 0 0) 0x7f132c060350 con 0x7f133c13f210 2024-04-25T11:30:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:30:00.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13200744b0 msgr2=0x7f13200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13200744b0 0x7f13200769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1334000f80 tx=0x7f1334009040 comp rx=0 tx=0).stop 2024-04-25T11:30:00.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f133c13f210 msgr2=0x7f133c146950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f133c13f210 0x7f133c146950 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f132c011570 tx=0x7f132c011a30 comp rx=0 tx=0).stop 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 shutdown_connections 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13200744b0 0x7f13200769c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f133c146e90 0x7f133c144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f133c13f210 0x7f133c146950 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 --2- 172.21.15.19:0/2067604795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13280051c0 0x7f133c157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 >> 172.21.15.19:0/2067604795 conn(0x7f133c073090 msgr2=0x7f133c1614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.139+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 shutdown_connections 2024-04-25T11:30:00.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.140+0000 7f1344800640 1 -- 172.21.15.19:0/2067604795 wait complete. 2024-04-25T11:30:00.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:30:00.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.463+0000 7faf4dc00640 1 Processor -- start 2024-04-25T11:30:00.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.463+0000 7faf4dc00640 1 -- start start 2024-04-25T11:30:00.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.465+0000 7faf4dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf4806f930 0x7faf4806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:00.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.465+0000 7faf4dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf4806fd40 0x7faf4806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:00.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.465+0000 7faf4dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf481623f0 con 0x7faf48070d20 2024-04-25T11:30:00.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.465+0000 7faf4dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faf481626d0 con 0x7faf4806fd40 2024-04-25T11:30:00.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.465+0000 7faf4dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf4806f130 con 0x7faf4806f930 2024-04-25T11:30:00.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.466+0000 7faf47400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf4806f930 0x7faf4806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:00.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.466+0000 7faf47400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf4806f930 0x7faf4806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:40250/0 (socket says 172.21.15.19:40250) 2024-04-25T11:30:00.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.466+0000 7faf47400640 1 -- 172.21.15.19:0/1000342370 learned_addr learned my addr 172.21.15.19:0/1000342370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:00.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.466+0000 7faf47e00640 1 --1- 172.21.15.19:0/1000342370 >> v1:172.21.15.169:6789/0 conn(0x7faf48070d20 0x7faf48161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:34514/0 (socket says 172.21.15.19:34514) 2024-04-25T11:30:00.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.466+0000 7faf4d200640 1 --2- 172.21.15.19:0/1000342370 >> v2:172.21.15.169:3300/0 conn(0x7faf4806fd40 0x7faf4806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:00.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.467+0000 7faf47400640 1 -- 172.21.15.19:0/1000342370 >> v2:172.21.15.169:3300/0 conn(0x7faf4806fd40 msgr2=0x7faf4806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:00.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.467+0000 7faf47400640 1 --2- 172.21.15.19:0/1000342370 >> v2:172.21.15.169:3300/0 conn(0x7faf4806fd40 0x7faf4806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.467+0000 7faf47400640 1 -- 172.21.15.19:0/1000342370 >> v1:172.21.15.169:6789/0 conn(0x7faf48070d20 legacy=0x7faf48161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.467+0000 7faf47400640 1 -- 172.21.15.19:0/1000342370 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4806f410 con 0x7faf4806f930 2024-04-25T11:30:00.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.468+0000 7faf47400640 1 --2- 172.21.15.19:0/1000342370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf4806f930 0x7faf4806d5d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf30009870 tx=0x7faf3002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6328e8b29bdcc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:00.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.469+0000 7faf46a00640 1 -- 172.21.15.19:0/1000342370 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30037070 con 0x7faf4806f930 2024-04-25T11:30:00.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.469+0000 7faf46a00640 1 -- 172.21.15.19:0/1000342370 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf30032440 con 0x7faf4806f930 2024-04-25T11:30:00.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.469+0000 7faf46a00640 1 -- 172.21.15.19:0/1000342370 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30032810 con 0x7faf4806f930 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 -- 172.21.15.19:0/1000342370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf4806f930 msgr2=0x7faf4806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 --2- 172.21.15.19:0/1000342370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf4806f930 0x7faf4806d5d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf30009870 tx=0x7faf3002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 -- 172.21.15.19:0/1000342370 shutdown_connections 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 --2- 172.21.15.19:0/1000342370 >> v2:172.21.15.169:3300/0 conn(0x7faf4806fd40 0x7faf4806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 --2- 172.21.15.19:0/1000342370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf4806f930 0x7faf4806d5d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 -- 172.21.15.19:0/1000342370 >> 172.21.15.19:0/1000342370 conn(0x7faf48073160 msgr2=0x7faf48073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 -- 172.21.15.19:0/1000342370 shutdown_connections 2024-04-25T11:30:00.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.470+0000 7faf4dc00640 1 -- 172.21.15.19:0/1000342370 wait complete. 2024-04-25T11:30:00.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.473+0000 7faf4dc00640 1 Processor -- start 2024-04-25T11:30:00.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.473+0000 7faf4dc00640 1 -- start start 2024-04-25T11:30:00.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.474+0000 7faf4dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4806f930 0x7faf48155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.474+0000 7faf4dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf4806fd40 0x7faf480cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.474+0000 7faf4dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf48070d20 0x7faf4814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.474+0000 7faf4dc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf4815e470 con 0x7faf4806f930 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.474+0000 7faf4dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faf4815e2f0 con 0x7faf4806fd40 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.474+0000 7faf4dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf480ce7e0 con 0x7faf48070d20 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4806f930 0x7faf48155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4806f930 0x7faf48155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34044/0 (socket says 172.21.15.19:34044) 2024-04-25T11:30:00.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 -- 172.21.15.19:0/2721733314 learned_addr learned my addr 172.21.15.19:0/2721733314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf4d200640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf48070d20 0x7faf4814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47400640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf4806fd40 0x7faf480cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 -- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf48070d20 msgr2=0x7faf4814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf48070d20 0x7faf4814d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 -- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf4806fd40 msgr2=0x7faf480cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf4806fd40 0x7faf480cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47e00640 1 -- 172.21.15.19:0/2721733314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf48159630 con 0x7faf4806f930 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.475+0000 7faf47400640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf4806fd40 0x7faf480cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.476+0000 7faf47e00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4806f930 0x7faf48155b30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c470 tx=0x7faf3800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.476+0000 7faf3fe00640 1 -- 172.21.15.19:0/2721733314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380074e0 con 0x7faf4806f930 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.476+0000 7faf3fe00640 1 -- 172.21.15.19:0/2721733314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf38007680 con 0x7faf4806f930 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.476+0000 7faf3fe00640 1 -- 172.21.15.19:0/2721733314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38004a00 con 0x7faf4806f930 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.476+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf48160050 con 0x7faf4806f930 2024-04-25T11:30:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.476+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4814ddc0 con 0x7faf4806f930 2024-04-25T11:30:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.477+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf48076c60 con 0x7faf4806f930 2024-04-25T11:30:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.479+0000 7faf3fe00640 1 -- 172.21.15.19:0/2721733314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf3801a020 con 0x7faf4806f930 2024-04-25T11:30:00.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.480+0000 7faf3fe00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf240744b0 0x7faf240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:00.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.480+0000 7faf3fe00640 1 -- 172.21.15.19:0/2721733314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5029+0+0 (secure 0 0 0) 0x7faf380946f0 con 0x7faf4806f930 2024-04-25T11:30:00.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.481+0000 7faf47400640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf240744b0 0x7faf240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:00.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.482+0000 7faf47400640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf240744b0 0x7faf240769c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faf30034000 tx=0x7faf3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.486+0000 7faf3fe00640 1 -- 172.21.15.19:0/2721733314 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf3805f1a0 con 0x7faf4806f930 2024-04-25T11:30:00.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.713+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7faf48074070 con 0x7faf4806f930 2024-04-25T11:30:00.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.719+0000 7faf3fe00640 1 -- 172.21.15.19:0/2721733314 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7faf3805eaa0 con 0x7faf4806f930 2024-04-25T11:30:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf240744b0 msgr2=0x7faf240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf240744b0 0x7faf240769c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faf30034000 tx=0x7faf3002e000 comp rx=0 tx=0).stop 2024-04-25T11:30:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4806f930 msgr2=0x7faf48155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4806f930 0x7faf48155b30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c470 tx=0x7faf3800c930 comp rx=0 tx=0).stop 2024-04-25T11:30:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 shutdown_connections 2024-04-25T11:30:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf240744b0 0x7faf240769c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf4806fd40 0x7faf480cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf48070d20 0x7faf4814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 --2- 172.21.15.19:0/2721733314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf4806f930 0x7faf48155b30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:00.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 >> 172.21.15.19:0/2721733314 conn(0x7faf48073160 msgr2=0x7faf480cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:00.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 shutdown_connections 2024-04-25T11:30:00.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:00.722+0000 7faf4dc00640 1 -- 172.21.15.19:0/2721733314 wait complete. 2024-04-25T11:30:00.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-04-25T11:30:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.035+0000 7f909cc00640 1 Processor -- start 2024-04-25T11:30:01.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.035+0000 7f909cc00640 1 -- start start 2024-04-25T11:30:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.038+0000 7f909cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 0x7f9098140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.038+0000 7f909cc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90980b5980 con 0x7f9098162160 2024-04-25T11:30:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.038+0000 7f909cc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90980b5c10 con 0x7f9098144010 2024-04-25T11:30:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.038+0000 7f909cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f90980b5ef0 con 0x7f90980b5570 2024-04-25T11:30:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.038+0000 7f9097e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9098162160 0x7f909806c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43306/0 (socket says 172.21.15.19:43306) 2024-04-25T11:30:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.038+0000 7f9097e00640 1 -- 172.21.15.19:0/2946301946 learned_addr learned my addr 172.21.15.19:0/2946301946 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.038+0000 7f9097400640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 0x7f9098140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1988492888 0 0) 0x7f90980b5980 con 0x7f9098162160 2024-04-25T11:30:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f907c003520 con 0x7f9098162160 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725541701 0 0) 0x7f90980b5c10 con 0x7f9098144010 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90980b5980 con 0x7f9098144010 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 654981005 0 0) 0x7f907c003520 con 0x7f9098162160 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 msgr2=0x7f9098140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 0x7f9098140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 >> v1:172.21.15.169:6789/0 conn(0x7f9098144010 legacy=0x7f9098161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90980b61d0 con 0x7f9098162160 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9090001970 con 0x7f9098162160 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 >> v1:172.21.15.19:6789/0 conn(0x7f9098162160 legacy=0x7f909806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 0x7f907c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c004120 0x7f907c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f907c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90980b5ef0 con 0x7f907c004d40 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f907c00bb80 con 0x7f907c004120 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.039+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f907c00bd80 con 0x7f90980b5570 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f9097e00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f907c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f9097400640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 0x7f907c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f908fe00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c004120 0x7f907c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f9097e00640 1 -- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 msgr2=0x7f907c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f9097e00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90980b5570 0x7f907c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f9097e00640 1 -- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c004120 msgr2=0x7f907c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f9097e00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c004120 0x7f907c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f9097e00640 1 -- 172.21.15.19:0/2946301946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c003520 con 0x7f907c004d40 2024-04-25T11:30:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.040+0000 7f908fe00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f907c004120 0x7f907c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:01.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.041+0000 7f9097e00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f907c005190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f909000a6a0 tx=0x7f907c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcbd03d7c7c19bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909000a2f0 con 0x7f907c004d40 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909002f070 con 0x7f907c004d40 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f9096a00640 1 -- 172.21.15.19:0/2946301946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909002bc40 con 0x7f907c004d40 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f909cc00640 1 -- 172.21.15.19:0/2946301946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 msgr2=0x7f907c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f909cc00640 1 --2- 172.21.15.19:0/2946301946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f907c005190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f909000a6a0 tx=0x7f907c005830 comp rx=0 tx=0).stop 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f9097e00640 1 -- 172.21.15.19:0/2946301946 reap_dead start 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f909cc00640 1 -- 172.21.15.19:0/2946301946 shutdown_connections 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.042+0000 7f909cc00640 1 -- 172.21.15.19:0/2946301946 >> 172.21.15.19:0/2946301946 conn(0x7f90980d1b40 msgr2=0x7f90980d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.043+0000 7f909cc00640 1 -- 172.21.15.19:0/2946301946 shutdown_connections 2024-04-25T11:30:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.043+0000 7f909cc00640 1 -- 172.21.15.19:0/2946301946 wait complete. 2024-04-25T11:30:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.045+0000 7f909cc00640 1 Processor -- start 2024-04-25T11:30:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.045+0000 7f909cc00640 1 -- start start 2024-04-25T11:30:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f909cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f909815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f909cc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9098156f70 0x7f90981567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f909cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90980cdd60 0x7f9098070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f909cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9098144420 con 0x7f907c004d40 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f909cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f90980b6790 con 0x7f90980cdd60 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f909cc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9098140dd0 con 0x7f9098156f70 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f9097400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f909815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f9097400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f909815f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34064/0 (socket says 172.21.15.19:34064) 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f9097400640 1 -- 172.21.15.19:0/194090177 learned_addr learned my addr 172.21.15.19:0/194090177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f9097e00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90980cdd60 0x7f9098070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.047+0000 7f908fe00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9098156f70 0x7f90981567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f9097400640 1 -- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9098156f70 msgr2=0x7f90981567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f9097400640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9098156f70 0x7f90981567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f9097400640 1 -- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90980cdd60 msgr2=0x7f9098070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:01.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f9097400640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90980cdd60 0x7f9098070850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:01.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f9097400640 1 -- 172.21.15.19:0/194090177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9098141fb0 con 0x7f907c004d40 2024-04-25T11:30:01.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f9097400640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f909815f4e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9088011600 tx=0x7f9088011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:01.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f908f400640 1 -- 172.21.15.19:0/194090177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908800f8c0 con 0x7f907c004d40 2024-04-25T11:30:01.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f908f400640 1 -- 172.21.15.19:0/194090177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908800fa60 con 0x7f907c004d40 2024-04-25T11:30:01.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9098156d20 con 0x7f907c004d40 2024-04-25T11:30:01.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.048+0000 7f908f400640 1 -- 172.21.15.19:0/194090177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908800fe10 con 0x7f907c004d40 2024-04-25T11:30:01.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.049+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9098142250 con 0x7f907c004d40 2024-04-25T11:30:01.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.050+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9098050740 con 0x7f907c004d40 2024-04-25T11:30:01.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.051+0000 7f908f400640 1 -- 172.21.15.19:0/194090177 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9088015460 con 0x7f907c004d40 2024-04-25T11:30:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.052+0000 7f908f400640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90740744e0 0x7f90740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.052+0000 7f908f400640 1 -- 172.21.15.19:0/194090177 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5029+0+0 (secure 0 0 0) 0x7f90880958f0 con 0x7f907c004d40 2024-04-25T11:30:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.053+0000 7f908fe00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90740744e0 0x7f90740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.054+0000 7f908fe00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90740744e0 0x7f90740769f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9098158950 tx=0x7f9080009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.057+0000 7f908f400640 1 -- 172.21.15.19:0/194090177 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9088019030 con 0x7f907c004d40 2024-04-25T11:30:01.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:01.341+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f9098162660 con 0x7f907c004d40 2024-04-25T11:30:02.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.148+0000 7f908f400640 1 -- 172.21.15.19:0/194090177 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v158) ==== 134+0+0 (secure 0 0 0) 0x7f9088060310 con 0x7f907c004d40 2024-04-25T11:30:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90740744e0 msgr2=0x7f90740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90740744e0 0x7f90740769f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9098158950 tx=0x7f9080009040 comp rx=0 tx=0).stop 2024-04-25T11:30:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 msgr2=0x7f909815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:02.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f909815f4e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9088011600 tx=0x7f9088011ac0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 shutdown_connections 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90740744e0 0x7f90740769f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90980cdd60 0x7f9098070850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9098156f70 0x7f90981567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 --2- 172.21.15.19:0/194090177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f907c004d40 0x7f909815f4e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 >> 172.21.15.19:0/194090177 conn(0x7f90980d1b40 msgr2=0x7f9098157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 shutdown_connections 2024-04-25T11:30:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.152+0000 7f909cc00640 1 -- 172.21.15.19:0/194090177 wait complete. 2024-04-25T11:30:02.177 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2024-04-25T11:30:02.178 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2024-04-25T11:30:02.178 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2024-04-25T11:30:02.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:02.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.472+0000 7f881e200640 1 Processor -- start 2024-04-25T11:30:02.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.472+0000 7f881e200640 1 -- start start 2024-04-25T11:30:02.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.474+0000 7f881e200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88180d0ac0 0x7f88180d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.474+0000 7f881e200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f88180b9e30 0x7f88180ba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.474+0000 7f881e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88180d1410 con 0x7f88180b4e40 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.474+0000 7f881e200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f88180d2f90 con 0x7f88180d0ac0 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.474+0000 7f881e200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f88180b90b0 con 0x7f88180b9e30 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f8817e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88180d0ac0 0x7f88180d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f8817e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88180d0ac0 0x7f88180d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34080/0 (socket says 172.21.15.19:34080) 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f8817e00640 1 -- 172.21.15.19:0/507485023 learned_addr learned my addr 172.21.15.19:0/507485023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:02.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f881d800640 1 --2- 172.21.15.19:0/507485023 >> v2:172.21.15.169:3300/0 conn(0x7f88180b9e30 0x7f88180ba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:02.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f8817e00640 1 -- 172.21.15.19:0/507485023 >> v2:172.21.15.169:3300/0 conn(0x7f88180b9e30 msgr2=0x7f88180ba240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:02.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f8817e00640 1 --2- 172.21.15.19:0/507485023 >> v2:172.21.15.169:3300/0 conn(0x7f88180b9e30 0x7f88180ba240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f8817e00640 1 -- 172.21.15.19:0/507485023 >> v1:172.21.15.169:6789/0 conn(0x7f88180b4e40 legacy=0x7f88180d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:02.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f8817e00640 1 -- 172.21.15.19:0/507485023 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88180d31f0 con 0x7f88180d0ac0 2024-04-25T11:30:02.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.475+0000 7f881d800640 1 --2- 172.21.15.19:0/507485023 >> v2:172.21.15.169:3300/0 conn(0x7f88180b9e30 0x7f88180ba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.476+0000 7f8817e00640 1 --2- 172.21.15.19:0/507485023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180d0ac0 0x7f88180d0ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f88180d3d20 tx=0x7f880402a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e01a4cb63f6583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.477+0000 7f8817400640 1 -- 172.21.15.19:0/507485023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880402adf0 con 0x7f88180d0ac0 2024-04-25T11:30:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.477+0000 7f8817400640 1 -- 172.21.15.19:0/507485023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8804004030 con 0x7f88180d0ac0 2024-04-25T11:30:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.477+0000 7f8817400640 1 -- 172.21.15.19:0/507485023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004460 con 0x7f88180d0ac0 2024-04-25T11:30:02.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 -- 172.21.15.19:0/507485023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180d0ac0 msgr2=0x7f88180d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 --2- 172.21.15.19:0/507485023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180d0ac0 0x7f88180d0ed0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f88180d3d20 tx=0x7f880402a0e0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 -- 172.21.15.19:0/507485023 shutdown_connections 2024-04-25T11:30:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 --2- 172.21.15.19:0/507485023 >> v2:172.21.15.169:3300/0 conn(0x7f88180b9e30 0x7f88180ba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 --2- 172.21.15.19:0/507485023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180d0ac0 0x7f88180d0ed0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 -- 172.21.15.19:0/507485023 >> 172.21.15.19:0/507485023 conn(0x7f881806dfe0 msgr2=0x7f881806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 -- 172.21.15.19:0/507485023 shutdown_connections 2024-04-25T11:30:02.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.478+0000 7f881e200640 1 -- 172.21.15.19:0/507485023 wait complete. 2024-04-25T11:30:02.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.480+0000 7f881e200640 1 Processor -- start 2024-04-25T11:30:02.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.480+0000 7f881e200640 1 -- start start 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180b4e40 0x7f881814bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88180b9e30 0x7f88181eaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88180d0ac0 0x7f88181eb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881e200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88180b81f0 con 0x7f88180b4e40 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88180b9630 con 0x7f88180b9e30 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88180b8370 con 0x7f88180d0ac0 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180b4e40 0x7f881814bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:02.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180b4e40 0x7f881814bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34092/0 (socket says 172.21.15.19:34092) 2024-04-25T11:30:02.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881ce00640 1 -- 172.21.15.19:0/3344627458 learned_addr learned my addr 172.21.15.19:0/3344627458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:02.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f8817e00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88180b9e30 0x7f88181eaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:02.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881d800640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88180d0ac0 0x7f88181eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.482+0000 7f881ce00640 1 -- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88180d0ac0 msgr2=0x7f88181eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881ce00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88180d0ac0 0x7f88181eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881ce00640 1 -- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88180b9e30 msgr2=0x7f88181eaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881ce00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88180b9e30 0x7f88181eaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881ce00640 1 -- 172.21.15.19:0/3344627458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881814c6f0 con 0x7f88180b4e40 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f8817e00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88180b9e30 0x7f88181eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881d800640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88180d0ac0 0x7f88181eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881ce00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180b4e40 0x7f881814bb10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f880800ba80 tx=0x7f880800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f8814c00640 1 -- 172.21.15.19:0/3344627458 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880800cc00 con 0x7f88180b4e40 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f8814c00640 1 -- 172.21.15.19:0/3344627458 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880800cda0 con 0x7f88180b4e40 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f8814c00640 1 -- 172.21.15.19:0/3344627458 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808003bb0 con 0x7f88180b4e40 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881814c9d0 con 0x7f88180b4e40 2024-04-25T11:30:02.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.483+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881814cf90 con 0x7f88180b4e40 2024-04-25T11:30:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.484+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88180bb9f0 con 0x7f88180b4e40 2024-04-25T11:30:02.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.486+0000 7f8814c00640 1 -- 172.21.15.19:0/3344627458 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8808014440 con 0x7f88180b4e40 2024-04-25T11:30:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.487+0000 7f8814c00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f87f80744e0 0x7f87f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.487+0000 7f8814c00640 1 -- 172.21.15.19:0/3344627458 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5029+0+0 (secure 0 0 0) 0x7f8808094f60 con 0x7f88180b4e40 2024-04-25T11:30:02.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.487+0000 7f8817e00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f87f80744e0 0x7f87f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.489+0000 7f8817e00640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f87f80744e0 0x7f87f80769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8804033000 tx=0x7f880402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:02.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.492+0000 7f8814c00640 1 -- 172.21.15.19:0/3344627458 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f880805f8f0 con 0x7f88180b4e40 2024-04-25T11:30:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.695+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8818076080 con 0x7f88180b4e40 2024-04-25T11:30:02.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.697+0000 7f8814c00640 1 -- 172.21.15.19:0/3344627458 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+13150 (secure 0 0 0) 0x7f880805f1f0 con 0x7f88180b4e40 2024-04-25T11:30:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:02.689 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:00.121589+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3972745609":"2024-04-26T11:29:57.100684+0000","172.21.15.19:6833/3972745609":"2024-04-26T11:29:57.100684+0000","172.21.15.19:6835/3831883468":"2024-04-26T11:29:00.304477+0000","172.21.15.19:6833/3831883468":"2024-04-26T11:29:00.304477+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f87f80744e0 msgr2=0x7f87f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f87f80744e0 0x7f87f80769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8804033000 tx=0x7f880402e000 comp rx=0 tx=0).stop 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180b4e40 msgr2=0x7f881814bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180b4e40 0x7f881814bb10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f880800ba80 tx=0x7f880800bf40 comp rx=0 tx=0).stop 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 shutdown_connections 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f87f80744e0 0x7f87f80769f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88180b9e30 0x7f88181eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88180d0ac0 0x7f88181eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 --2- 172.21.15.19:0/3344627458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88180b4e40 0x7f881814bb10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 >> 172.21.15.19:0/3344627458 conn(0x7f881806dfe0 msgr2=0x7f8818160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.700+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 shutdown_connections 2024-04-25T11:30:02.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:02.701+0000 7f881e200640 1 -- 172.21.15.19:0/3344627458 wait complete. 2024-04-25T11:30:02.718 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:30:02.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:30:02.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.006+0000 7efe7e400640 1 Processor -- start 2024-04-25T11:30:02.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.007+0000 7efe7e400640 1 -- start start 2024-04-25T11:30:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.009+0000 7efe7e400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efe7815cff0 0x7efe780b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.009+0000 7efe7e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe780ce6b0 con 0x7efe7806d5d0 2024-04-25T11:30:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.009+0000 7efe7e400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe780ce990 con 0x7efe780b0a10 2024-04-25T11:30:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.009+0000 7efe7e400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efe780d0480 con 0x7efe7815cff0 2024-04-25T11:30:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.010+0000 7efe77e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efe7806d5d0 0x7efe7806d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43320/0 (socket says 172.21.15.19:43320) 2024-04-25T11:30:03.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.010+0000 7efe77e00640 1 -- 172.21.15.19:0/524384535 learned_addr learned my addr 172.21.15.19:0/524384535 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.010+0000 7efe76a00640 1 --2- 172.21.15.19:0/524384535 >> v2:172.21.15.169:3300/0 conn(0x7efe7815cff0 0x7efe780b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.010+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 318769535 0 0) 0x7efe780ce6b0 con 0x7efe7806d5d0 2024-04-25T11:30:03.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe5c005730 con 0x7efe7806d5d0 2024-04-25T11:30:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3841900614 0 0) 0x7efe5c005730 con 0x7efe7806d5d0 2024-04-25T11:30:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 >> v2:172.21.15.169:3300/0 conn(0x7efe7815cff0 msgr2=0x7efe780b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:03.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 --2- 172.21.15.19:0/524384535 >> v2:172.21.15.169:3300/0 conn(0x7efe7815cff0 0x7efe780b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 >> v1:172.21.15.169:6789/0 conn(0x7efe780b0a10 legacy=0x7efe781615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe780cec70 con 0x7efe7806d5d0 2024-04-25T11:30:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efe700019f0 con 0x7efe7806d5d0 2024-04-25T11:30:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 >> v1:172.21.15.19:6789/0 conn(0x7efe7806d5d0 legacy=0x7efe7806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe5c005c10 0x7efe5c006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe5c006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe5c0074b0 0x7efe5c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c00e410 con 0x7efe5c006a10 2024-04-25T11:30:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c00e5c0 con 0x7efe5c005c10 2024-04-25T11:30:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efe5c00e7c0 con 0x7efe5c0074b0 2024-04-25T11:30:03.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.011+0000 7efe76a00640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe5c006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.012+0000 7efe77400640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe5c005c10 0x7efe5c006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.012+0000 7efe76a00640 1 -- 172.21.15.19:0/524384535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe5c0074b0 msgr2=0x7efe5c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.012+0000 7efe76a00640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe5c0074b0 0x7efe5c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.012+0000 7efe76a00640 1 -- 172.21.15.19:0/524384535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe5c005c10 msgr2=0x7efe5c006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.012+0000 7efe76a00640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe5c005c10 0x7efe5c006080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.012+0000 7efe76a00640 1 -- 172.21.15.19:0/524384535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c005730 con 0x7efe5c006a10 2024-04-25T11:30:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.012+0000 7efe77e00640 1 -- 172.21.15.19:0/524384535 reap_dead start 2024-04-25T11:30:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.013+0000 7efe76a00640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe5c006e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efe64009ac0 tx=0x7efe5c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed21cf7cd8f6b682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.013+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64036070 con 0x7efe5c006a10 2024-04-25T11:30:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.013+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe64031400 con 0x7efe5c006a10 2024-04-25T11:30:03.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe76000640 1 -- 172.21.15.19:0/524384535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64009690 con 0x7efe5c006a10 2024-04-25T11:30:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe7e400640 1 -- 172.21.15.19:0/524384535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 msgr2=0x7efe5c006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe7e400640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe5c006e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efe64009ac0 tx=0x7efe5c007a40 comp rx=0 tx=0).stop 2024-04-25T11:30:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe7e400640 1 -- 172.21.15.19:0/524384535 shutdown_connections 2024-04-25T11:30:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe7e400640 1 --2- 172.21.15.19:0/524384535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe5c006e20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe7e400640 1 -- 172.21.15.19:0/524384535 >> 172.21.15.19:0/524384535 conn(0x7efe780730b0 msgr2=0x7efe780734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe7e400640 1 -- 172.21.15.19:0/524384535 shutdown_connections 2024-04-25T11:30:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.014+0000 7efe7e400640 1 -- 172.21.15.19:0/524384535 wait complete. 2024-04-25T11:30:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.017+0000 7efe7e400640 1 Processor -- start 2024-04-25T11:30:03.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.017+0000 7efe7e400640 1 -- start start 2024-04-25T11:30:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.018+0000 7efe7e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe7814a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.018+0000 7efe7e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe7815f7e0 0x7efe780c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe7e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe7815fbf0 0x7efe780c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe7e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe780cef50 con 0x7efe5c006a10 2024-04-25T11:30:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe7e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efe780d0740 con 0x7efe7815fbf0 2024-04-25T11:30:03.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe7e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efe7806d9e0 con 0x7efe7815f7e0 2024-04-25T11:30:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe7814a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe7814a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34102/0 (socket says 172.21.15.19:34102) 2024-04-25T11:30:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe77400640 1 -- 172.21.15.19:0/2172302202 learned_addr learned my addr 172.21.15.19:0/2172302202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe77e00640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe7815fbf0 0x7efe780c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.019+0000 7efe76a00640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe7815f7e0 0x7efe780c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe77400640 1 -- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe7815f7e0 msgr2=0x7efe780c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe77400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe7815f7e0 0x7efe780c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe77400640 1 -- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe7815fbf0 msgr2=0x7efe780c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe77400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe7815fbf0 0x7efe780c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe77400640 1 -- 172.21.15.19:0/2172302202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7814a970 con 0x7efe5c006a10 2024-04-25T11:30:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe77400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe7814a430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efe6800e490 tx=0x7efe6800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe6f400640 1 -- 172.21.15.19:0/2172302202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe680052e0 con 0x7efe5c006a10 2024-04-25T11:30:03.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe6f400640 1 -- 172.21.15.19:0/2172302202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe68018070 con 0x7efe5c006a10 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe6f400640 1 -- 172.21.15.19:0/2172302202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68003830 con 0x7efe5c006a10 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.020+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7815eb60 con 0x7efe5c006a10 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.021+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7814b0d0 con 0x7efe5c006a10 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.022+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe780cf740 con 0x7efe5c006a10 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.024+0000 7efe6f400640 1 -- 172.21.15.19:0/2172302202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efe680039d0 con 0x7efe5c006a10 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.025+0000 7efe6f400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe580744e0 0x7efe580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.025+0000 7efe6f400640 1 -- 172.21.15.19:0/2172302202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5029+0+0 (secure 0 0 0) 0x7efe68094ab0 con 0x7efe5c006a10 2024-04-25T11:30:03.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.025+0000 7efe76a00640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe580744e0 0x7efe580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.027+0000 7efe76a00640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe580744e0 0x7efe580769f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe64033000 tx=0x7efe64008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:03.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.030+0000 7efe6f400640 1 -- 172.21.15.19:0/2172302202 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe780cf740 con 0x7efe5c006a10 2024-04-25T11:30:03.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.247+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe780cfd40 con 0x7efe5c006a10 2024-04-25T11:30:03.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.248+0000 7efe6f400640 1 -- 172.21.15.19:0/2172302202 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7efe780cfd40 con 0x7efe5c006a10 2024-04-25T11:30:03.240 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:30:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe580744e0 msgr2=0x7efe580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe580744e0 0x7efe580769f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe64033000 tx=0x7efe64008030 comp rx=0 tx=0).stop 2024-04-25T11:30:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 msgr2=0x7efe7814a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe7814a430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efe6800e490 tx=0x7efe6800e950 comp rx=0 tx=0).stop 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 shutdown_connections 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe580744e0 0x7efe580769f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe7815fbf0 0x7efe780c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe7815f7e0 0x7efe780c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 --2- 172.21.15.19:0/2172302202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe5c006a10 0x7efe7814a430 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 >> 172.21.15.19:0/2172302202 conn(0x7efe780730b0 msgr2=0x7efe780b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.251+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 shutdown_connections 2024-04-25T11:30:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.252+0000 7efe7e400640 1 -- 172.21.15.19:0/2172302202 wait complete. 2024-04-25T11:30:03.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:30:03.255 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:30:03.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-04-25T11:30:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.596+0000 7efdf2a00640 1 Processor -- start 2024-04-25T11:30:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.596+0000 7efdf2a00640 1 -- start start 2024-04-25T11:30:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.599+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec06d5d0 0x7efdec06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.599+0000 7efdf2a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdec06df20 con 0x7efdec140e00 2024-04-25T11:30:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.599+0000 7efdf2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdec141210 con 0x7efdec141fd0 2024-04-25T11:30:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.599+0000 7efdf2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdec141b40 con 0x7efdec06d5d0 2024-04-25T11:30:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.599+0000 7efdf1600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efdec140e00 0x7efdec143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43336/0 (socket says 172.21.15.19:43336) 2024-04-25T11:30:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.599+0000 7efdf1600640 1 -- 172.21.15.19:0/1302006302 learned_addr learned my addr 172.21.15.19:0/1302006302 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.599+0000 7efdf0c00640 1 --2- 172.21.15.19:0/1302006302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec06d5d0 0x7efdec06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.600+0000 7efdebe00640 1 -- 172.21.15.19:0/1302006302 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4150589851 0 0) 0x7efde0001490 con 0x7efdec140e00 2024-04-25T11:30:03.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.600+0000 7efdebe00640 1 -- 172.21.15.19:0/1302006302 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdd4003520 con 0x7efdec140e00 2024-04-25T11:30:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.600+0000 7efdf0c00640 1 -- 172.21.15.19:0/1302006302 >> v1:172.21.15.169:6789/0 conn(0x7efdec141fd0 legacy=0x7efdec1423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.600+0000 7efdf0c00640 1 -- 172.21.15.19:0/1302006302 >> v1:172.21.15.19:6789/0 conn(0x7efdec140e00 legacy=0x7efdec143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.600+0000 7efdf0c00640 1 -- 172.21.15.19:0/1302006302 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec141510 con 0x7efdec06d5d0 2024-04-25T11:30:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.601+0000 7efdf0c00640 1 --2- 172.21.15.19:0/1302006302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec06d5d0 0x7efdec06d9e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdec1413e0 tx=0x7efdd802a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4fd48f30d8ec9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdebe00640 1 -- 172.21.15.19:0/1302006302 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8037070 con 0x7efdec06d5d0 2024-04-25T11:30:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdebe00640 1 -- 172.21.15.19:0/1302006302 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd8032440 con 0x7efdec06d5d0 2024-04-25T11:30:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdebe00640 1 -- 172.21.15.19:0/1302006302 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd80327f0 con 0x7efdec06d5d0 2024-04-25T11:30:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdf2a00640 1 -- 172.21.15.19:0/1302006302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec06d5d0 msgr2=0x7efdec06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdf2a00640 1 --2- 172.21.15.19:0/1302006302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec06d5d0 0x7efdec06d9e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efdec1413e0 tx=0x7efdd802a200 comp rx=0 tx=0).stop 2024-04-25T11:30:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdf2a00640 1 -- 172.21.15.19:0/1302006302 shutdown_connections 2024-04-25T11:30:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdf2a00640 1 --2- 172.21.15.19:0/1302006302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec06d5d0 0x7efdec06d9e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.602+0000 7efdf2a00640 1 -- 172.21.15.19:0/1302006302 >> 172.21.15.19:0/1302006302 conn(0x7efdec072ee0 msgr2=0x7efdec0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.603+0000 7efdf2a00640 1 -- 172.21.15.19:0/1302006302 shutdown_connections 2024-04-25T11:30:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.603+0000 7efdf2a00640 1 -- 172.21.15.19:0/1302006302 wait complete. 2024-04-25T11:30:03.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.605+0000 7efdf2a00640 1 Processor -- start 2024-04-25T11:30:03.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.605+0000 7efdf2a00640 1 -- start start 2024-04-25T11:30:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdec06d5d0 0x7efdec13ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec140e00 0x7efdec12aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdec141fd0 0x7efdec13d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf2a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1417d0 con 0x7efdec06d5d0 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdec1443d0 con 0x7efdec141fd0 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdec0d1490 con 0x7efdec140e00 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdec06d5d0 0x7efdec13ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdec06d5d0 0x7efdec13ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34104/0 (socket says 172.21.15.19:34104) 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf1600640 1 -- 172.21.15.19:0/71804294 learned_addr learned my addr 172.21.15.19:0/71804294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf0c00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec140e00 0x7efdec12aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.607+0000 7efdf2000640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdec141fd0 0x7efdec13d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf1600640 1 -- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec140e00 msgr2=0x7efdec12aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf1600640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec140e00 0x7efdec12aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf1600640 1 -- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdec141fd0 msgr2=0x7efdec13d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf1600640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdec141fd0 0x7efdec13d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf1600640 1 -- 172.21.15.19:0/71804294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec13eed0 con 0x7efdec06d5d0 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf2000640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdec141fd0 0x7efdec13d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf1600640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdec06d5d0 0x7efdec13ff70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efde000cbf0 tx=0x7efde000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:03.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efde9600640 1 -- 172.21.15.19:0/71804294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0017070 con 0x7efdec06d5d0 2024-04-25T11:30:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efde9600640 1 -- 172.21.15.19:0/71804294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde000ad40 con 0x7efdec06d5d0 2024-04-25T11:30:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efde9600640 1 -- 172.21.15.19:0/71804294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0012620 con 0x7efdec06d5d0 2024-04-25T11:30:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.608+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec13f1b0 con 0x7efdec06d5d0 2024-04-25T11:30:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.609+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec13f770 con 0x7efdec06d5d0 2024-04-25T11:30:03.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.610+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec050740 con 0x7efdec06d5d0 2024-04-25T11:30:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.612+0000 7efde9600640 1 -- 172.21.15.19:0/71804294 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efde001a070 con 0x7efdec06d5d0 2024-04-25T11:30:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.613+0000 7efde9600640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdcc0744b0 0x7efdcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.613+0000 7efde9600640 1 -- 172.21.15.19:0/71804294 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5029+0+0 (secure 0 0 0) 0x7efde0094db0 con 0x7efdec06d5d0 2024-04-25T11:30:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.614+0000 7efdf0c00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdcc0744b0 0x7efdcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:03.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.615+0000 7efdf0c00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdcc0744b0 0x7efdcc0769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efdd8034000 tx=0x7efdd802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:03.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.619+0000 7efde9600640 1 -- 172.21.15.19:0/71804294 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efde0099070 con 0x7efdec06d5d0 2024-04-25T11:30:03.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:03.900+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7efdec0b7710 con 0x7efdec06d5d0 2024-04-25T11:30:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.163+0000 7efde9600640 1 -- 172.21.15.19:0/71804294 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v160) ==== 144+0+0 (secure 0 0 0) 0x7efdec06df20 con 0x7efdec06d5d0 2024-04-25T11:30:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdcc0744b0 msgr2=0x7efdcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdcc0744b0 0x7efdcc0769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efdd8034000 tx=0x7efdd802e000 comp rx=0 tx=0).stop 2024-04-25T11:30:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdec06d5d0 msgr2=0x7efdec13ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdec06d5d0 0x7efdec13ff70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efde000cbf0 tx=0x7efde000a430 comp rx=0 tx=0).stop 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 shutdown_connections 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdcc0744b0 0x7efdcc0769c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdec141fd0 0x7efdec13d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdec140e00 0x7efdec12aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 --2- 172.21.15.19:0/71804294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdec06d5d0 0x7efdec13ff70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.167+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 >> 172.21.15.19:0/71804294 conn(0x7efdec072ee0 msgr2=0x7efdec0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.168+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 shutdown_connections 2024-04-25T11:30:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.168+0000 7efdf2a00640 1 -- 172.21.15.19:0/71804294 wait complete. 2024-04-25T11:30:04.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:30:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.490+0000 7f6876e00640 1 Processor -- start 2024-04-25T11:30:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.490+0000 7f6876e00640 1 -- start start 2024-04-25T11:30:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.492+0000 7f6876e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f687014aac0 0x7f68701655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.492+0000 7f6876e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687006d5d0 0x7f687006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.492+0000 7f6876e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6870165ae0 con 0x7f6870168110 2024-04-25T11:30:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.492+0000 7f6876e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f687006df20 con 0x7f687014aac0 2024-04-25T11:30:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.492+0000 7f6876e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f687014aed0 con 0x7f687006d5d0 2024-04-25T11:30:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f687014aac0 0x7f68701655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f687014aac0 0x7f68701655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38222/0 (socket says 172.21.15.19:38222) 2024-04-25T11:30:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875a00640 1 -- 172.21.15.19:0/2637287470 learned_addr learned my addr 172.21.15.19:0/2637287470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875a00640 1 -- 172.21.15.19:0/2637287470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687006d5d0 msgr2=0x7f687006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875000640 1 --2- 172.21.15.19:0/2637287470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687006d5d0 0x7f687006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875a00640 1 --2- 172.21.15.19:0/2637287470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687006d5d0 0x7f687006d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875a00640 1 -- 172.21.15.19:0/2637287470 >> v1:172.21.15.169:6789/0 conn(0x7f6870168110 legacy=0x7f6870168520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.493+0000 7f6875a00640 1 -- 172.21.15.19:0/2637287470 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687006f460 con 0x7f687014aac0 2024-04-25T11:30:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.494+0000 7f6875a00640 1 --2- 172.21.15.19:0/2637287470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687014aac0 0x7f68701655a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6864009e80 tx=0x7f686402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd97c357b6d7f33a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f686fe00640 1 -- 172.21.15.19:0/2637287470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864036070 con 0x7f687014aac0 2024-04-25T11:30:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f686fe00640 1 -- 172.21.15.19:0/2637287470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6864031600 con 0x7f687014aac0 2024-04-25T11:30:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f686fe00640 1 -- 172.21.15.19:0/2637287470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640319d0 con 0x7f687014aac0 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f6876e00640 1 -- 172.21.15.19:0/2637287470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687014aac0 msgr2=0x7f68701655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f6876e00640 1 --2- 172.21.15.19:0/2637287470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687014aac0 0x7f68701655a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6864009e80 tx=0x7f686402a660 comp rx=0 tx=0).stop 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f6876e00640 1 -- 172.21.15.19:0/2637287470 shutdown_connections 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f6876e00640 1 --2- 172.21.15.19:0/2637287470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f687006d5d0 0x7f687006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f6876e00640 1 --2- 172.21.15.19:0/2637287470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687014aac0 0x7f68701655a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.495+0000 7f6876e00640 1 -- 172.21.15.19:0/2637287470 >> 172.21.15.19:0/2637287470 conn(0x7f6870072f40 msgr2=0x7f6870073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.496+0000 7f6876e00640 1 -- 172.21.15.19:0/2637287470 shutdown_connections 2024-04-25T11:30:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.496+0000 7f6876e00640 1 -- 172.21.15.19:0/2637287470 wait complete. 2024-04-25T11:30:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.498+0000 7f6876e00640 1 Processor -- start 2024-04-25T11:30:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.498+0000 7f6876e00640 1 -- start start 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6876e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687006d5d0 0x7f68700aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6876e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687014aac0 0x7f68701618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6876e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6870168110 0x7f68700b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6876e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f687006fd00 con 0x7f687006d5d0 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6876e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f687006fa20 con 0x7f687014aac0 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6876e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f687006ffe0 con 0x7f6870168110 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687006d5d0 0x7f68700aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687006d5d0 0x7f68700aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38226/0 (socket says 172.21.15.19:38226) 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875a00640 1 -- 172.21.15.19:0/2266298345 learned_addr learned my addr 172.21.15.19:0/2266298345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875000640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687014aac0 0x7f68701618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875a00640 1 -- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6870168110 msgr2=0x7f68700b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6876400640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6870168110 0x7f68700b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875a00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6870168110 0x7f68700b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875a00640 1 -- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687014aac0 msgr2=0x7f68701618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.500+0000 7f6875a00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687014aac0 0x7f68701618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f6875a00640 1 -- 172.21.15.19:0/2266298345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6870161e20 con 0x7f687006d5d0 2024-04-25T11:30:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f6875000640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687014aac0 0x7f68701618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f6876400640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6870168110 0x7f68700b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f6875a00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687006d5d0 0x7f68700aaa70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6864034040 tx=0x7f6864004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:04.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f686d600640 1 -- 172.21.15.19:0/2266298345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864036040 con 0x7f687006d5d0 2024-04-25T11:30:04.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f686d600640 1 -- 172.21.15.19:0/2266298345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6864003990 con 0x7f687006d5d0 2024-04-25T11:30:04.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68700b0c90 con 0x7f687006d5d0 2024-04-25T11:30:04.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f686d600640 1 -- 172.21.15.19:0/2266298345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864009510 con 0x7f687006d5d0 2024-04-25T11:30:04.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.501+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68701fb9f0 con 0x7f687006d5d0 2024-04-25T11:30:04.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.503+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6870076c60 con 0x7f687006d5d0 2024-04-25T11:30:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.504+0000 7f686d600640 1 -- 172.21.15.19:0/2266298345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6864007820 con 0x7f687006d5d0 2024-04-25T11:30:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.505+0000 7f686d600640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68500744b0 0x7f68500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.505+0000 7f686d600640 1 -- 172.21.15.19:0/2266298345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5029+0+0 (secure 0 0 0) 0x7f68640b6340 con 0x7f687006d5d0 2024-04-25T11:30:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.505+0000 7f6875000640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68500744b0 0x7f68500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:04.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.507+0000 7f6875000640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68500744b0 0x7f68500769c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6870165f30 tx=0x7f685c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:04.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.510+0000 7f686d600640 1 -- 172.21.15.19:0/2266298345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6870165ae0 con 0x7f687006d5d0 2024-04-25T11:30:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.714+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6870070e70 con 0x7f687006d5d0 2024-04-25T11:30:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.723+0000 7f686d600640 1 -- 172.21.15.19:0/2266298345 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f6864080cd0 con 0x7f687006d5d0 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.725+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68500744b0 msgr2=0x7f68500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.725+0000 7f6876e00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68500744b0 0x7f68500769c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6870165f30 tx=0x7f685c008040 comp rx=0 tx=0).stop 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.725+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687006d5d0 msgr2=0x7f68700aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.725+0000 7f6876e00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687006d5d0 0x7f68700aaa70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6864034040 tx=0x7f6864004a40 comp rx=0 tx=0).stop 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 shutdown_connections 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68500744b0 0x7f68500769c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f687014aac0 0x7f68701618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6870168110 0x7f68700b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 --2- 172.21.15.19:0/2266298345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f687006d5d0 0x7f68700aaa70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:04.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 >> 172.21.15.19:0/2266298345 conn(0x7f6870072f40 msgr2=0x7f6870162c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:04.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 shutdown_connections 2024-04-25T11:30:04.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:04.726+0000 7f6876e00640 1 -- 172.21.15.19:0/2266298345 wait complete. 2024-04-25T11:30:04.744 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:30:04.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:30:04.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:30:04.973 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:30:04.974 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:30:04.974 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:05.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:30:05.035 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:30:05.035 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:05.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:30:05.093 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:30:05.093 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:05.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:30:05.154 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:30:05.154 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:05.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:30:05.211 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:30:05.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:05.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.513+0000 7f8531800640 1 Processor -- start 2024-04-25T11:30:05.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.513+0000 7f8531800640 1 -- start start 2024-04-25T11:30:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.515+0000 7f8531800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f852c06c5c0 0x7f852c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.515+0000 7f8531800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c0b4ea0 0x7f852c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.516+0000 7f8531800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f852c0d1230 con 0x7f852c144370 2024-04-25T11:30:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.516+0000 7f8531800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f852c0d06b0 con 0x7f852c06c5c0 2024-04-25T11:30:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.516+0000 7f8531800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f852c0d0990 con 0x7f852c0b4ea0 2024-04-25T11:30:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.516+0000 7f852b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f852c144370 0x7f852c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37668/0 (socket says 172.21.15.19:37668) 2024-04-25T11:30:05.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.516+0000 7f852b400640 1 -- 172.21.15.19:0/3885385555 learned_addr learned my addr 172.21.15.19:0/3885385555 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.516+0000 7f8530e00640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c0b4ea0 0x7f852c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852be00640 1 --2- 172.21.15.19:0/3885385555 >> v2:172.21.15.169:3300/0 conn(0x7f852c06c5c0 0x7f852c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588746111 0 0) 0x7f852c0d1230 con 0x7f852c144370 2024-04-25T11:30:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8510003520 con 0x7f852c144370 2024-04-25T11:30:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3832770833 0 0) 0x7f8510003520 con 0x7f852c144370 2024-04-25T11:30:05.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c0b4ea0 msgr2=0x7f852c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c0b4ea0 0x7f852c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 >> v2:172.21.15.169:3300/0 conn(0x7f852c06c5c0 msgr2=0x7f852c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 --2- 172.21.15.19:0/3885385555 >> v2:172.21.15.169:3300/0 conn(0x7f852c06c5c0 0x7f852c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c0d0c70 con 0x7f852c144370 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8514001670 con 0x7f852c144370 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 >> v1:172.21.15.19:6789/0 conn(0x7f852c144370 legacy=0x7f852c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85100039b0 0x7f8510003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f8510004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8510004ed0 0x7f851000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f852c0d06b0 con 0x7f8510004340 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f852c0d0990 con 0x7f8510004ed0 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.517+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f851000bf00 con 0x7f85100039b0 2024-04-25T11:30:05.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f852b400640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f8510004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f852be00640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85100039b0 0x7f8510003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f852b400640 1 -- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85100039b0 msgr2=0x7f8510003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f852b400640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85100039b0 0x7f8510003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f852b400640 1 -- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8510004ed0 msgr2=0x7f851000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f8530e00640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8510004ed0 0x7f851000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f852b400640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8510004ed0 0x7f851000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.518+0000 7f852b400640 1 -- 172.21.15.19:0/3885385555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8510003520 con 0x7f8510004340 2024-04-25T11:30:05.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.519+0000 7f852b400640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f8510004860 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f851400c290 tx=0x7f8510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a904f087e285d9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:05.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.520+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514007d50 con 0x7f8510004340 2024-04-25T11:30:05.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.520+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8514005ad0 con 0x7f8510004340 2024-04-25T11:30:05.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.520+0000 7f852aa00640 1 -- 172.21.15.19:0/3885385555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514007070 con 0x7f8510004340 2024-04-25T11:30:05.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.520+0000 7f8531800640 1 -- 172.21.15.19:0/3885385555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 msgr2=0x7f8510004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:05.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.520+0000 7f8531800640 1 --2- 172.21.15.19:0/3885385555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f8510004860 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f851400c290 tx=0x7f8510005830 comp rx=0 tx=0).stop 2024-04-25T11:30:05.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.520+0000 7f8530e00640 1 -- 172.21.15.19:0/3885385555 reap_dead start 2024-04-25T11:30:05.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.521+0000 7f8531800640 1 -- 172.21.15.19:0/3885385555 shutdown_connections 2024-04-25T11:30:05.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.521+0000 7f8531800640 1 -- 172.21.15.19:0/3885385555 >> 172.21.15.19:0/3885385555 conn(0x7f852c0d1720 msgr2=0x7f852c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:05.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.521+0000 7f8531800640 1 -- 172.21.15.19:0/3885385555 shutdown_connections 2024-04-25T11:30:05.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.521+0000 7f8531800640 1 -- 172.21.15.19:0/3885385555 wait complete. 2024-04-25T11:30:05.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.523+0000 7f8531800640 1 Processor -- start 2024-04-25T11:30:05.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.523+0000 7f8531800640 1 -- start start 2024-04-25T11:30:05.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f8531800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f852c1e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f8531800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f852c133f20 0x7f852c134330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f8531800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c134870 0x7f852c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f8531800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f852c0d0450 con 0x7f8510004340 2024-04-25T11:30:05.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f8531800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f852c0d02d0 con 0x7f852c133f20 2024-04-25T11:30:05.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f8531800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f852c147760 con 0x7f852c134870 2024-04-25T11:30:05.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f852be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f852c1e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f852be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f852c1e8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38256/0 (socket says 172.21.15.19:38256) 2024-04-25T11:30:05.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.525+0000 7f852be00640 1 -- 172.21.15.19:0/916952562 learned_addr learned my addr 172.21.15.19:0/916952562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:05.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f8530e00640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c134870 0x7f852c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852b400640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f852c133f20 0x7f852c134330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852be00640 1 -- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c134870 msgr2=0x7f852c0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:05.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852be00640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c134870 0x7f852c0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852be00640 1 -- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f852c133f20 msgr2=0x7f852c134330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:05.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852be00640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f852c133f20 0x7f852c134330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852be00640 1 -- 172.21.15.19:0/916952562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c161dd0 con 0x7f8510004340 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852b400640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f852c133f20 0x7f852c134330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.526+0000 7f852be00640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f852c1e8030 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f851c011600 tx=0x7f851c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.527+0000 7f8523e00640 1 -- 172.21.15.19:0/916952562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c015870 con 0x7f8510004340 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.527+0000 7f8523e00640 1 -- 172.21.15.19:0/916952562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f851c015a10 con 0x7f8510004340 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.527+0000 7f8523e00640 1 -- 172.21.15.19:0/916952562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c015dc0 con 0x7f8510004340 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.527+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1620b0 con 0x7f8510004340 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.527+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c162670 con 0x7f8510004340 2024-04-25T11:30:05.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.528+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c146ea0 con 0x7f8510004340 2024-04-25T11:30:05.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.529+0000 7f8523e00640 1 -- 172.21.15.19:0/916952562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f851c00f490 con 0x7f8510004340 2024-04-25T11:30:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.530+0000 7f8523e00640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85080744b0 0x7f85080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.530+0000 7f8523e00640 1 -- 172.21.15.19:0/916952562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5029+0+0 (secure 0 0 0) 0x7f851c095970 con 0x7f8510004340 2024-04-25T11:30:05.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.531+0000 7f852b400640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85080744b0 0x7f85080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:05.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.532+0000 7f852b400640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85080744b0 0x7f85080769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8514033000 tx=0x7f8514009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:05.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.535+0000 7f8523e00640 1 -- 172.21.15.19:0/916952562 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f852c146ea0 con 0x7f8510004340 2024-04-25T11:30:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.780+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f852c068f10 con 0x7f8510004340 2024-04-25T11:30:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.780+0000 7f8523e00640 1 -- 172.21.15.19:0/916952562 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3866 (secure 0 0 0) 0x7f852c068f10 con 0x7f8510004340 2024-04-25T11:30:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:05.772 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/200743296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":200743296},{"type":"v1","addr":"172.21.15.169:6836","nonce":200743296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4792,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/23196265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":23196265},{"type":"v1","addr":"172.21.15.19:6834","nonce":23196265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/2281308777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":2281308777},{"type":"v1","addr":"172.21.15.169:6837","nonce":2281308777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1206974895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1206974895},{"type":"v1","addr":"172.21.15.19:6835","nonce":1206974895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-04-25T11:30:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85080744b0 msgr2=0x7f85080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85080744b0 0x7f85080769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8514033000 tx=0x7f8514009030 comp rx=0 tx=0).stop 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 msgr2=0x7f852c1e8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f852c1e8030 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f851c011600 tx=0x7f851c011ac0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 shutdown_connections 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85080744b0 0x7f85080769c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f852c133f20 0x7f852c134330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f852c134870 0x7f852c0cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 --2- 172.21.15.19:0/916952562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8510004340 0x7f852c1e8030 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 >> 172.21.15.19:0/916952562 conn(0x7f852c0d1720 msgr2=0x7f852c146790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 shutdown_connections 2024-04-25T11:30:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:05.784+0000 7f8531800640 1 -- 172.21.15.19:0/916952562 wait complete. 2024-04-25T11:30:05.803 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:30:05.803 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:30:05.804 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:30:05.804 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:30:05.804 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:30:05.804 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:30:05.804 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:30:05.804 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:30:05.805 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:30:05.805 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:30:05.805 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:30:05.805 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:30:05.805 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:30:05.805 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:30:05.805 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:30:05.805 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:30:05.836 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:30:05.836 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:30:05.837 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:30:05.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:30:05.842 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:30:05.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:30:05.848 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:30:05.848 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:30:05.889 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:30:05.889 DEBUG:teuthology.parallel:result is None 2024-04-25T11:30:05.896 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:30:05.896 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:30:05.897 DEBUG:teuthology.parallel:result is None 2024-04-25T11:30:05.897 DEBUG:teuthology.parallel:result is None 2024-04-25T11:30:05.902 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:30:05.903 DEBUG:teuthology.parallel:result is None 2024-04-25T11:30:05.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:30:06.116 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:30:06.118 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:30:06.122 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:30:06.126 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:30:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.170+0000 7f5312800640 1 Processor -- start 2024-04-25T11:30:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.170+0000 7f5312800640 1 -- start start 2024-04-25T11:30:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.172+0000 7f5312800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f530c070680 0x7f530c1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.172+0000 7f5312800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c0b5c90 0x7f530c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.172+0000 7f5312800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f530c0b65e0 con 0x7f530c160360 2024-04-25T11:30:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.172+0000 7f5312800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f530c06d100 con 0x7f530c070680 2024-04-25T11:30:06.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.172+0000 7f5312800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f530c06c7d0 con 0x7f530c0b5c90 2024-04-25T11:30:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f530c070680 0x7f530c1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f530c070680 0x7f530c1647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38284/0 (socket says 172.21.15.19:38284) 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530b400640 1 -- 172.21.15.19:0/4171882696 learned_addr learned my addr 172.21.15.19:0/4171882696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530be00640 1 --2- 172.21.15.19:0/4171882696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c0b5c90 0x7f530c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530b400640 1 -- 172.21.15.19:0/4171882696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c0b5c90 msgr2=0x7f530c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530b400640 1 --2- 172.21.15.19:0/4171882696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c0b5c90 0x7f530c0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530b400640 1 -- 172.21.15.19:0/4171882696 >> v1:172.21.15.169:6789/0 conn(0x7f530c160360 legacy=0x7f530c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530a000640 1 -- 172.21.15.19:0/4171882696 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66689947 0 0) 0x7f52f8001110 con 0x7f530c160360 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530b400640 1 -- 172.21.15.19:0/4171882696 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c06cab0 con 0x7f530c070680 2024-04-25T11:30:06.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.173+0000 7f530be00640 1 --2- 172.21.15.19:0/4171882696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c0b5c90 0x7f530c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.175+0000 7f530b400640 1 --2- 172.21.15.19:0/4171882696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c070680 0x7f530c1647f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009870 tx=0x7f52fc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f38f79e8d2f072d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.175+0000 7f530a000640 1 -- 172.21.15.19:0/4171882696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc037070 con 0x7f530c070680 2024-04-25T11:30:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.175+0000 7f530a000640 1 -- 172.21.15.19:0/4171882696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52fc032440 con 0x7f530c070680 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f530a000640 1 -- 172.21.15.19:0/4171882696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc032810 con 0x7f530c070680 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 -- 172.21.15.19:0/4171882696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c070680 msgr2=0x7f530c1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 --2- 172.21.15.19:0/4171882696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c070680 0x7f530c1647f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009870 tx=0x7f52fc02a130 comp rx=0 tx=0).stop 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 -- 172.21.15.19:0/4171882696 shutdown_connections 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 --2- 172.21.15.19:0/4171882696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c0b5c90 0x7f530c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 --2- 172.21.15.19:0/4171882696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c070680 0x7f530c1647f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 -- 172.21.15.19:0/4171882696 >> 172.21.15.19:0/4171882696 conn(0x7f530c0d1ae0 msgr2=0x7f530c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 -- 172.21.15.19:0/4171882696 shutdown_connections 2024-04-25T11:30:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.176+0000 7f5312800640 1 -- 172.21.15.19:0/4171882696 wait complete. 2024-04-25T11:30:06.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.179+0000 7f5312800640 1 Processor -- start 2024-04-25T11:30:06.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.179+0000 7f5312800640 1 -- start start 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f5312800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c070680 0x7f530c14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f5312800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f530c0b5c90 0x7f530c14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f5312800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c160360 0x7f530c15e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f5312800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f530c0afbb0 con 0x7f530c160360 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f5312800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f530c06fcc0 con 0x7f530c0b5c90 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f5312800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f530c0afd30 con 0x7f530c070680 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f530be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c160360 0x7f530c15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f530be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c160360 0x7f530c15e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38290/0 (socket says 172.21.15.19:38290) 2024-04-25T11:30:06.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f530be00640 1 -- 172.21.15.19:0/3443919517 learned_addr learned my addr 172.21.15.19:0/3443919517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:06.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f530aa00640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f530c0b5c90 0x7f530c14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:06.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.181+0000 7f530b400640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c070680 0x7f530c14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:06.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.182+0000 7f530be00640 1 -- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c070680 msgr2=0x7f530c14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:06.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.182+0000 7f530be00640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c070680 0x7f530c14e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:06.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.182+0000 7f530be00640 1 -- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f530c0b5c90 msgr2=0x7f530c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:06.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.182+0000 7f530be00640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f530c0b5c90 0x7f530c14b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:06.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.182+0000 7f530be00640 1 -- 172.21.15.19:0/3443919517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c14bca0 con 0x7f530c160360 2024-04-25T11:30:06.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.182+0000 7f530aa00640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f530c0b5c90 0x7f530c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:30:06.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.182+0000 7f530be00640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c160360 0x7f530c15e400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5304007df0 tx=0x7f530400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:06.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.186+0000 7f5303400640 1 -- 172.21.15.19:0/3443919517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530401a070 con 0x7f530c160360 2024-04-25T11:30:06.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.186+0000 7f5303400640 1 -- 172.21.15.19:0/3443919517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5304003990 con 0x7f530c160360 2024-04-25T11:30:06.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.186+0000 7f5303400640 1 -- 172.21.15.19:0/3443919517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5304003d80 con 0x7f530c160360 2024-04-25T11:30:06.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.186+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530c14b8e0 con 0x7f530c160360 2024-04-25T11:30:06.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.186+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530c1faa10 con 0x7f530c160360 2024-04-25T11:30:06.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.187+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530c1fa4a0 con 0x7f530c160360 2024-04-25T11:30:06.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.189+0000 7f5303400640 1 -- 172.21.15.19:0/3443919517 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f530401d070 con 0x7f530c160360 2024-04-25T11:30:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.190+0000 7f5303400640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52ec0744b0 0x7f52ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.190+0000 7f5303400640 1 -- 172.21.15.19:0/3443919517 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5029+0+0 (secure 0 0 0) 0x7f53040951a0 con 0x7f530c160360 2024-04-25T11:30:06.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.191+0000 7f530b400640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52ec0744b0 0x7f52ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:06.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.192+0000 7f530b400640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52ec0744b0 0x7f52ec0769c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f52fc034000 tx=0x7f52fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:06.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.200+0000 7f5303400640 1 -- 172.21.15.19:0/3443919517 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5304010070 con 0x7f530c160360 2024-04-25T11:30:06.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:06.423+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f530c050740 con 0x7f530c160360 2024-04-25T11:30:07.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.222+0000 7f5303400640 1 -- 172.21.15.19:0/3443919517 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) ==== 97+0+0 (secure 0 0 0) 0x7f530405f570 con 0x7f530c160360 2024-04-25T11:30:07.213 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52ec0744b0 msgr2=0x7f52ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52ec0744b0 0x7f52ec0769c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f52fc034000 tx=0x7f52fc02e000 comp rx=0 tx=0).stop 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c160360 msgr2=0x7f530c15e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c160360 0x7f530c15e400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5304007df0 tx=0x7f530400d4a0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 shutdown_connections 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52ec0744b0 0x7f52ec0769c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f530c0b5c90 0x7f530c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f530c160360 0x7f530c15e400 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 --2- 172.21.15.19:0/3443919517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f530c070680 0x7f530c14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 >> 172.21.15.19:0/3443919517 conn(0x7f530c0d1ae0 msgr2=0x7f530c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 shutdown_connections 2024-04-25T11:30:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.226+0000 7f5312800640 1 -- 172.21.15.19:0/3443919517 wait complete. 2024-04-25T11:30:07.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:30:07.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.526+0000 7f4cfec00640 1 Processor -- start 2024-04-25T11:30:07.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.526+0000 7f4cfec00640 1 -- start start 2024-04-25T11:30:07.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.529+0000 7f4cfec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:07.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.529+0000 7f4cfec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4cf8144010 0x7f4cf8161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:07.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.529+0000 7f4cfec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf81623a0 con 0x7f4cf8161f90 2024-04-25T11:30:07.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.529+0000 7f4cfec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf80b5980 con 0x7f4cf8144010 2024-04-25T11:30:07.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.529+0000 7f4cfec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf80b5c10 con 0x7f4cf80b5570 2024-04-25T11:30:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.529+0000 7f4cfce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4cf8144010 0x7f4cf8161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.530+0000 7f4cfd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.530+0000 7f4cfd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:59840/0 (socket says 172.21.15.19:59840) 2024-04-25T11:30:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.530+0000 7f4cfd800640 1 -- 172.21.15.19:0/4146000553 learned_addr learned my addr 172.21.15.19:0/4146000553 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.530+0000 7f4cfce00640 1 --2- 172.21.15.19:0/4146000553 >> v2:172.21.15.169:3300/0 conn(0x7f4cf8144010 0x7f4cf8161a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:43274/0 (socket says 172.21.15.19:43274) 2024-04-25T11:30:07.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.530+0000 7f4cfe200640 1 --1- 172.21.15.19:0/4146000553 >> v1:172.21.15.169:6789/0 conn(0x7f4cf8161f90 0x7f4cf815e460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:58044/0 (socket says 172.21.15.19:58044) 2024-04-25T11:30:07.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.530+0000 7f4cfce00640 1 -- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 msgr2=0x7f4cf8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:07.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.530+0000 7f4cfce00640 1 --2- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.531+0000 7f4cfce00640 1 -- 172.21.15.19:0/4146000553 >> v1:172.21.15.169:6789/0 conn(0x7f4cf8161f90 legacy=0x7f4cf815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.531+0000 7f4cfce00640 1 -- 172.21.15.19:0/4146000553 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf80b5ef0 con 0x7f4cf8144010 2024-04-25T11:30:07.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.531+0000 7f4cfd800640 1 --2- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:30:07.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.532+0000 7f4cfce00640 1 --2- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8144010 0x7f4cf8161a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40097a0 tx=0x7f4ce402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5d4240b7e00b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:07.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.532+0000 7f4cf7e00640 1 -- 172.21.15.19:0/4146000553 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4037070 con 0x7f4cf8144010 2024-04-25T11:30:07.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.532+0000 7f4cf7e00640 1 -- 172.21.15.19:0/4146000553 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce4032440 con 0x7f4cf8144010 2024-04-25T11:30:07.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.532+0000 7f4cf7e00640 1 -- 172.21.15.19:0/4146000553 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4032810 con 0x7f4cf8144010 2024-04-25T11:30:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 -- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8144010 msgr2=0x7f4cf8161a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 --2- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8144010 0x7f4cf8161a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ce40097a0 tx=0x7f4ce402a180 comp rx=0 tx=0).stop 2024-04-25T11:30:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 -- 172.21.15.19:0/4146000553 shutdown_connections 2024-04-25T11:30:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 --2- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8144010 0x7f4cf8161a50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 --2- 172.21.15.19:0/4146000553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 -- 172.21.15.19:0/4146000553 >> 172.21.15.19:0/4146000553 conn(0x7f4cf80d1b40 msgr2=0x7f4cf80d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:07.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 -- 172.21.15.19:0/4146000553 shutdown_connections 2024-04-25T11:30:07.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.533+0000 7f4cfec00640 1 -- 172.21.15.19:0/4146000553 wait complete. 2024-04-25T11:30:07.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.536+0000 7f4cfec00640 1 Processor -- start 2024-04-25T11:30:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.536+0000 7f4cfec00640 1 -- start start 2024-04-25T11:30:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.537+0000 7f4cfec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.537+0000 7f4cfec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8144010 0x7f4cf8153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.537+0000 7f4cfec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8161f90 0x7f4cf80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.537+0000 7f4cfec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf80b6790 con 0x7f4cf8144010 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.537+0000 7f4cfec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf80b64b0 con 0x7f4cf8161f90 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.537+0000 7f4cfec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf8144420 con 0x7f4cf80b5570 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8144010 0x7f4cf8153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8144010 0x7f4cf8153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38302/0 (socket says 172.21.15.19:38302) 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 -- 172.21.15.19:0/1343268968 learned_addr learned my addr 172.21.15.19:0/1343268968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfe200640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8161f90 0x7f4cf80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfd800640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 -- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 msgr2=0x7f4cf8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 -- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8161f90 msgr2=0x7f4cf80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8161f90 0x7f4cf80ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfce00640 1 -- 172.21.15.19:0/1343268968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf8141310 con 0x7f4cf8144010 2024-04-25T11:30:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfd800640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.538+0000 7f4cfe200640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8161f90 0x7f4cf80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.539+0000 7f4cfce00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8144010 0x7f4cf8153eb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4000c00 tx=0x7f4ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.539+0000 7f4cf5600640 1 -- 172.21.15.19:0/1343268968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4037040 con 0x7f4cf8144010 2024-04-25T11:30:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.539+0000 7f4cf5600640 1 -- 172.21.15.19:0/1343268968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce4042070 con 0x7f4cf8144010 2024-04-25T11:30:07.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.539+0000 7f4cf5600640 1 -- 172.21.15.19:0/1343268968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4030920 con 0x7f4cf8144010 2024-04-25T11:30:07.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.539+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf806f6d0 con 0x7f4cf8144010 2024-04-25T11:30:07.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.539+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf806fc90 con 0x7f4cf8144010 2024-04-25T11:30:07.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.540+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf806f440 con 0x7f4cf8144010 2024-04-25T11:30:07.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.542+0000 7f4cf5600640 1 -- 172.21.15.19:0/1343268968 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ce4030ac0 con 0x7f4cf8144010 2024-04-25T11:30:07.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.542+0000 7f4cf5600640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cd80744e0 0x7f4cd80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:07.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.543+0000 7f4cf5600640 1 -- 172.21.15.19:0/1343268968 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f4ce40b5940 con 0x7f4cf8144010 2024-04-25T11:30:07.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.543+0000 7f4cfd800640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cd80744e0 0x7f4cd80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:07.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.545+0000 7f4cfd800640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cd80744e0 0x7f4cd80769f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009a20 tx=0x7f4cec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:07.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.547+0000 7f4cf5600640 1 -- 172.21.15.19:0/1343268968 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ce407fcb0 con 0x7f4cf8144010 2024-04-25T11:30:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.820+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4cf8050740 con 0x7f4cf8144010 2024-04-25T11:30:07.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.821+0000 7f4cf5600640 1 -- 172.21.15.19:0/1343268968 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f4ce4083fb0 con 0x7f4cf8144010 2024-04-25T11:30:07.812 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ==", 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:30:07.813 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw==", 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw==", 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA==", 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ==", 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.816 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw==", 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.817 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q==", 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw==", 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw==", 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.819 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw==", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ==", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.820 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA==", 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T11:30:07.821 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w==", 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q==", 2024-04-25T11:30:07.822 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:30:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:30:07.823 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg==", 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:30:07.824 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q==", 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T11:30:07.825 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNPSpmHT+MHRAAPZNLFcl/fxDgWnMeN3nAHw==", 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA==", 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:30:07.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA==", 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:30:07.827 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:30:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cd80744e0 msgr2=0x7f4cd80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cd80744e0 0x7f4cd80769f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009a20 tx=0x7f4cec008040 comp rx=0 tx=0).stop 2024-04-25T11:30:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8144010 msgr2=0x7f4cf8153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8144010 0x7f4cf8153eb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4000c00 tx=0x7f4ce40046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 shutdown_connections 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cd80744e0 0x7f4cd80769f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf8161f90 0x7f4cf80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cf8144010 0x7f4cf8153eb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 --2- 172.21.15.19:0/1343268968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cf80b5570 0x7f4cf8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 >> 172.21.15.19:0/1343268968 conn(0x7f4cf80d1b40 msgr2=0x7f4cf8160400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 shutdown_connections 2024-04-25T11:30:07.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:07.824+0000 7f4cfec00640 1 -- 172.21.15.19:0/1343268968 wait complete. 2024-04-25T11:30:07.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.108+0000 7f44ae600640 1 Processor -- start 2024-04-25T11:30:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.108+0000 7f44ae600640 1 -- start start 2024-04-25T11:30:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.110+0000 7f44ae600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44a81443c0 0x7f44a806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.110+0000 7f44ae600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 0x7f44a80cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.110+0000 7f44ae600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a80cff30 con 0x7f44a80cfb20 2024-04-25T11:30:08.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.110+0000 7f44ae600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f44a80708d0 con 0x7f44a81443c0 2024-04-25T11:30:08.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.110+0000 7f44ae600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44a815f160 con 0x7f44a8070370 2024-04-25T11:30:08.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.111+0000 7f44a7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f44a80cfb20 0x7f44a80ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37686/0 (socket says 172.21.15.19:37686) 2024-04-25T11:30:08.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.111+0000 7f44a7e00640 1 -- 172.21.15.19:0/2970657420 learned_addr learned my addr 172.21.15.19:0/2970657420 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.111+0000 7f44a7400640 1 --2- 172.21.15.19:0/2970657420 >> v2:172.21.15.169:3300/0 conn(0x7f44a81443c0 0x7f44a806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.111+0000 7f44a6a00640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 0x7f44a80cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.111+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650562143 0 0) 0x7f44a0001110 con 0x7f44a80cfb20 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.111+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f448c003520 con 0x7f44a80cfb20 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 428906438 0 0) 0x7f448c003520 con 0x7f44a80cfb20 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 msgr2=0x7f44a80cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 0x7f44a80cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 >> v2:172.21.15.169:3300/0 conn(0x7f44a81443c0 msgr2=0x7f44a806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 --2- 172.21.15.19:0/2970657420 >> v2:172.21.15.169:3300/0 conn(0x7f44a81443c0 0x7f44a806fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a815f5f0 con 0x7f44a80cfb20 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f44a0001840 con 0x7f44a80cfb20 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 >> v1:172.21.15.19:6789/0 conn(0x7f44a80cfb20 legacy=0x7f44a80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c0039b0 0x7f448c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 0x7f448c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c004d40 0x7f448c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80708d0 con 0x7f448c004d40 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44a815f160 con 0x7f448c0039b0 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f448c00b9c0 con 0x7f44a8070370 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a7e00640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c004d40 0x7f448c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.112+0000 7f44a7400640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c0039b0 0x7f448c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.113+0000 7f44a6a00640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 0x7f448c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.113+0000 7f44a7e00640 1 -- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 msgr2=0x7f448c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.113+0000 7f44a7e00640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a8070370 0x7f448c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.113+0000 7f44a7e00640 1 -- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c0039b0 msgr2=0x7f448c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.113+0000 7f44a7e00640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c0039b0 0x7f448c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.113+0000 7f44a7e00640 1 -- 172.21.15.19:0/2970657420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c003520 con 0x7f448c004d40 2024-04-25T11:30:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.114+0000 7f44a7e00640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c004d40 0x7f448c005150 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c3e0 tx=0x7f448c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5e601bb47679d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.114+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a000c0e0 con 0x7f448c004d40 2024-04-25T11:30:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.114+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0034070 con 0x7f448c004d40 2024-04-25T11:30:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.114+0000 7f44a6000640 1 -- 172.21.15.19:0/2970657420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0005d30 con 0x7f448c004d40 2024-04-25T11:30:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.115+0000 7f44ae600640 1 -- 172.21.15.19:0/2970657420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c004d40 msgr2=0x7f448c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.115+0000 7f44ae600640 1 --2- 172.21.15.19:0/2970657420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f448c004d40 0x7f448c005150 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c3e0 tx=0x7f448c005830 comp rx=0 tx=0).stop 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.115+0000 7f44a7e00640 1 -- 172.21.15.19:0/2970657420 reap_dead start 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.115+0000 7f44ae600640 1 -- 172.21.15.19:0/2970657420 shutdown_connections 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.115+0000 7f44ae600640 1 -- 172.21.15.19:0/2970657420 >> 172.21.15.19:0/2970657420 conn(0x7f44a80d1ae0 msgr2=0x7f44a80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.115+0000 7f44ae600640 1 -- 172.21.15.19:0/2970657420 shutdown_connections 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.115+0000 7f44ae600640 1 -- 172.21.15.19:0/2970657420 wait complete. 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.118+0000 7f44ae600640 1 Processor -- start 2024-04-25T11:30:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.118+0000 7f44ae600640 1 -- start start 2024-04-25T11:30:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.119+0000 7f44ae600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c004d40 0x7f44a813e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.119+0000 7f44ae600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a80b1100 0x7f44a80b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.119+0000 7f44ae600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b1a50 0x7f44a80af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.119+0000 7f44ae600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44a815edf0 con 0x7f44a80b1a50 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.119+0000 7f44ae600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44a815eb10 con 0x7f448c004d40 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.119+0000 7f44ae600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44a806d5d0 con 0x7f44a80b1100 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b1a50 0x7f44a80af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b1a50 0x7f44a80af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38316/0 (socket says 172.21.15.19:38316) 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 -- 172.21.15.19:0/3657916653 learned_addr learned my addr 172.21.15.19:0/3657916653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a6a00640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a80b1100 0x7f44a80b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7400640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c004d40 0x7f44a813e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 -- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a80b1100 msgr2=0x7f44a80b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a80b1100 0x7f44a80b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 -- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c004d40 msgr2=0x7f44a813e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c004d40 0x7f44a813e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.120+0000 7f44a7e00640 1 -- 172.21.15.19:0/3657916653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a81e9f10 con 0x7f44a80b1a50 2024-04-25T11:30:08.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.121+0000 7f44a7e00640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b1a50 0x7f44a80af7e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f44a0001b40 tx=0x7f44a815f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.121+0000 7f449f400640 1 -- 172.21.15.19:0/3657916653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0007070 con 0x7f44a80b1a50 2024-04-25T11:30:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.121+0000 7f449f400640 1 -- 172.21.15.19:0/3657916653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a003f030 con 0x7f44a80b1a50 2024-04-25T11:30:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.121+0000 7f449f400640 1 -- 172.21.15.19:0/3657916653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0007410 con 0x7f44a80b1a50 2024-04-25T11:30:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.121+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a80aea60 con 0x7f44a80b1a50 2024-04-25T11:30:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.121+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a80aed00 con 0x7f44a80b1a50 2024-04-25T11:30:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.123+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a80d2340 con 0x7f44a80b1a50 2024-04-25T11:30:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.124+0000 7f449f400640 1 -- 172.21.15.19:0/3657916653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f44a00075b0 con 0x7f44a80b1a50 2024-04-25T11:30:08.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.124+0000 7f449f400640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4488074530 0x7f4488076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.125+0000 7f449f400640 1 -- 172.21.15.19:0/3657916653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f44a00b58a0 con 0x7f44a80b1a50 2024-04-25T11:30:08.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.125+0000 7f44a7400640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4488074530 0x7f4488076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.126+0000 7f44a7400640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4488074530 0x7f4488076a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f44a80d1860 tx=0x7f449800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:08.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.129+0000 7f449f400640 1 -- 172.21.15.19:0/3657916653 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44a80d2340 con 0x7f44a80b1a50 2024-04-25T11:30:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.334+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44a8050740 con 0x7f44a80b1a50 2024-04-25T11:30:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.335+0000 7f449f400640 1 -- 172.21.15.19:0/3657916653 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+12891 (secure 0 0 0) 0x7f44a80cff30 con 0x7f44a80b1a50 2024-04-25T11:30:08.326 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:08.326 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":46,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:07.213496+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4488074530 msgr2=0x7f4488076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4488074530 0x7f4488076a40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f44a80d1860 tx=0x7f449800b040 comp rx=0 tx=0).stop 2024-04-25T11:30:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b1a50 msgr2=0x7f44a80af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b1a50 0x7f44a80af7e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f44a0001b40 tx=0x7f44a815f7c0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 shutdown_connections 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4488074530 0x7f4488076a40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b1a50 0x7f44a80af7e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44a80b1100 0x7f44a80b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 --2- 172.21.15.19:0/3657916653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448c004d40 0x7f44a813e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.337+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 >> 172.21.15.19:0/3657916653 conn(0x7f44a80d1ae0 msgr2=0x7f44a815c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.338+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 shutdown_connections 2024-04-25T11:30:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.338+0000 7f44ae600640 1 -- 172.21.15.19:0/3657916653 wait complete. 2024-04-25T11:30:08.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:30:08.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:30:08.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.649+0000 7fd867200640 1 Processor -- start 2024-04-25T11:30:08.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.650+0000 7fd867200640 1 -- start start 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd867200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd868144340 0x7fd86806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd867200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86806db10 0x7fd86815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd867200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86806df20 con 0x7fd86815ec90 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd867200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd868144750 con 0x7fd868144340 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd867200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd868070000 con 0x7fd86806db10 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd865e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd868144340 0x7fd86806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd866800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd86815ec90 0x7fd868161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37692/0 (socket says 172.21.15.19:37692) 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd866800640 1 -- 172.21.15.19:0/348702623 learned_addr learned my addr 172.21.15.19:0/348702623 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.652+0000 7fd85d400640 1 --2- 172.21.15.19:0/348702623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86806db10 0x7fd86815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.653+0000 7fd865e00640 1 -- 172.21.15.19:0/348702623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86806db10 msgr2=0x7fd86815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.653+0000 7fd865e00640 1 --2- 172.21.15.19:0/348702623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86806db10 0x7fd86815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.653+0000 7fd865e00640 1 -- 172.21.15.19:0/348702623 >> v1:172.21.15.19:6789/0 conn(0x7fd86815ec90 legacy=0x7fd868161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.653+0000 7fd865e00640 1 -- 172.21.15.19:0/348702623 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86806f240 con 0x7fd868144340 2024-04-25T11:30:08.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.654+0000 7fd865e00640 1 --2- 172.21.15.19:0/348702623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868144340 0x7fd86806d5d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd8580097a0 tx=0x7fd85802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0e6993ba346403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:08.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.655+0000 7fd865400640 1 -- 172.21.15.19:0/348702623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858037070 con 0x7fd868144340 2024-04-25T11:30:08.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.655+0000 7fd865400640 1 -- 172.21.15.19:0/348702623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85802ad10 con 0x7fd868144340 2024-04-25T11:30:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.655+0000 7fd865400640 1 -- 172.21.15.19:0/348702623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858032660 con 0x7fd868144340 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.655+0000 7fd867200640 1 -- 172.21.15.19:0/348702623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868144340 msgr2=0x7fd86806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.655+0000 7fd867200640 1 --2- 172.21.15.19:0/348702623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868144340 0x7fd86806d5d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd8580097a0 tx=0x7fd85802a080 comp rx=0 tx=0).stop 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.656+0000 7fd867200640 1 -- 172.21.15.19:0/348702623 shutdown_connections 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.656+0000 7fd867200640 1 --2- 172.21.15.19:0/348702623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86806db10 0x7fd86815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.656+0000 7fd867200640 1 --2- 172.21.15.19:0/348702623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd868144340 0x7fd86806d5d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.656+0000 7fd867200640 1 -- 172.21.15.19:0/348702623 >> 172.21.15.19:0/348702623 conn(0x7fd868072f40 msgr2=0x7fd868073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.656+0000 7fd867200640 1 -- 172.21.15.19:0/348702623 shutdown_connections 2024-04-25T11:30:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.656+0000 7fd867200640 1 -- 172.21.15.19:0/348702623 wait complete. 2024-04-25T11:30:08.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.658+0000 7fd867200640 1 Processor -- start 2024-04-25T11:30:08.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.658+0000 7fd867200640 1 -- start start 2024-04-25T11:30:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd867200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd86806db10 0x7fd86815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd867200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd868144340 0x7fd8681e9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd867200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86815ec90 0x7fd86815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd867200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd86806fae0 con 0x7fd86806db10 2024-04-25T11:30:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd867200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd86806f800 con 0x7fd868144340 2024-04-25T11:30:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd867200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd86815f0a0 con 0x7fd86815ec90 2024-04-25T11:30:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd865e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd86806db10 0x7fd86815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd865e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd86806db10 0x7fd86815d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38334/0 (socket says 172.21.15.19:38334) 2024-04-25T11:30:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.660+0000 7fd865e00640 1 -- 172.21.15.19:0/624826603 learned_addr learned my addr 172.21.15.19:0/624826603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd866800640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86815ec90 0x7fd86815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd865e00640 1 -- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86815ec90 msgr2=0x7fd86815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd85d400640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd868144340 0x7fd8681e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd865e00640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86815ec90 0x7fd86815b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd865e00640 1 -- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd868144340 msgr2=0x7fd8681e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd865e00640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd868144340 0x7fd8681e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd865e00640 1 -- 172.21.15.19:0/624826603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8681e7d50 con 0x7fd86806db10 2024-04-25T11:30:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd865e00640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd86806db10 0x7fd86815d3a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd858000c00 tx=0x7fd8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd85ea00640 1 -- 172.21.15.19:0/624826603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858037040 con 0x7fd86806db10 2024-04-25T11:30:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd85ea00640 1 -- 172.21.15.19:0/624826603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd858004250 con 0x7fd86806db10 2024-04-25T11:30:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd85ea00640 1 -- 172.21.15.19:0/624826603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858003610 con 0x7fd86806db10 2024-04-25T11:30:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.661+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8681e8030 con 0x7fd86806db10 2024-04-25T11:30:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.662+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681e85f0 con 0x7fd86806db10 2024-04-25T11:30:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.663+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd868076c60 con 0x7fd86806db10 2024-04-25T11:30:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.664+0000 7fd85ea00640 1 -- 172.21.15.19:0/624826603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd85802abd0 con 0x7fd86806db10 2024-04-25T11:30:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.665+0000 7fd85ea00640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8440744b0 0x7fd8440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.665+0000 7fd85ea00640 1 -- 172.21.15.19:0/624826603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7fd8580b5850 con 0x7fd86806db10 2024-04-25T11:30:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.666+0000 7fd85d400640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8440744b0 0x7fd8440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.667+0000 7fd85d400640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8440744b0 0x7fd8440769c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd850006fd0 tx=0x7fd850008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.670+0000 7fd85ea00640 1 -- 172.21.15.19:0/624826603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd85804b070 con 0x7fd86806db10 2024-04-25T11:30:08.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.876+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd868162870 con 0x7fd86806db10 2024-04-25T11:30:08.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.877+0000 7fd85ea00640 1 -- 172.21.15.19:0/624826603 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fd858080290 con 0x7fd86806db10 2024-04-25T11:30:08.868 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.880+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8440744b0 msgr2=0x7fd8440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.880+0000 7fd867200640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8440744b0 0x7fd8440769c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd850006fd0 tx=0x7fd850008040 comp rx=0 tx=0).stop 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.880+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd86806db10 msgr2=0x7fd86815d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.880+0000 7fd867200640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd86806db10 0x7fd86815d3a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd858000c00 tx=0x7fd8580046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 shutdown_connections 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8440744b0 0x7fd8440769c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd868144340 0x7fd8681e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd86815ec90 0x7fd86815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 --2- 172.21.15.19:0/624826603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd86806db10 0x7fd86815d3a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 >> 172.21.15.19:0/624826603 conn(0x7fd868072f40 msgr2=0x7fd868142e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 shutdown_connections 2024-04-25T11:30:08.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:08.881+0000 7fd867200640 1 -- 172.21.15.19:0/624826603 wait complete. 2024-04-25T11:30:08.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:30:09.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.197+0000 7f3222800640 1 Processor -- start 2024-04-25T11:30:09.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.198+0000 7f3222800640 1 -- start start 2024-04-25T11:30:09.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.200+0000 7f3222800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f321c0d0ed0 0x7f321c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.200+0000 7f3222800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c06f3e0 con 0x7f321c1434d0 2024-04-25T11:30:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.200+0000 7f3222800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c06f6c0 con 0x7f321c160be0 2024-04-25T11:30:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.200+0000 7f3222800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f321c1438e0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f321b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f321c1434d0 0x7f321c070480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37708/0 (socket says 172.21.15.19:37708) 2024-04-25T11:30:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f321b400640 1 -- 172.21.15.19:0/951800453 learned_addr learned my addr 172.21.15.19:0/951800453 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:09.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f3212a00640 1 --2- 172.21.15.19:0/951800453 >> v2:172.21.15.19:3300/0 conn(0x7f321c0d0ed0 0x7f321c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f321aa00640 1 -- 172.21.15.19:0/951800453 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 365882509 0 0) 0x7f321c06f3e0 con 0x7f321c1434d0 2024-04-25T11:30:09.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f321aa00640 1 -- 172.21.15.19:0/951800453 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3200003520 con 0x7f321c1434d0 2024-04-25T11:30:09.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f3212a00640 1 -- 172.21.15.19:0/951800453 >> v1:172.21.15.169:6789/0 conn(0x7f321c160be0 legacy=0x7f321c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:09.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f3212a00640 1 -- 172.21.15.19:0/951800453 >> v1:172.21.15.19:6789/0 conn(0x7f321c1434d0 legacy=0x7f321c070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.201+0000 7f3212a00640 1 -- 172.21.15.19:0/951800453 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c06f9a0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.203+0000 7f3212a00640 1 --2- 172.21.15.19:0/951800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c0d12e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f32040097d0 tx=0x7f320402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3215fe65bbc77a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:09.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.205+0000 7f321aa00640 1 -- 172.21.15.19:0/951800453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204037070 con 0x7f321c0d0ed0 2024-04-25T11:30:09.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.205+0000 7f321aa00640 1 -- 172.21.15.19:0/951800453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3204032440 con 0x7f321c0d0ed0 2024-04-25T11:30:09.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.205+0000 7f321aa00640 1 -- 172.21.15.19:0/951800453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204032810 con 0x7f321c0d0ed0 2024-04-25T11:30:09.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.206+0000 7f3222800640 1 -- 172.21.15.19:0/951800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 msgr2=0x7f321c0d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.206+0000 7f3222800640 1 --2- 172.21.15.19:0/951800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c0d12e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f32040097d0 tx=0x7f320402a210 comp rx=0 tx=0).stop 2024-04-25T11:30:09.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.206+0000 7f3222800640 1 -- 172.21.15.19:0/951800453 shutdown_connections 2024-04-25T11:30:09.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.206+0000 7f3222800640 1 --2- 172.21.15.19:0/951800453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c0d12e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.206+0000 7f3222800640 1 -- 172.21.15.19:0/951800453 >> 172.21.15.19:0/951800453 conn(0x7f321c0d1ae0 msgr2=0x7f321c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:09.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.206+0000 7f3222800640 1 -- 172.21.15.19:0/951800453 shutdown_connections 2024-04-25T11:30:09.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.206+0000 7f3222800640 1 -- 172.21.15.19:0/951800453 wait complete. 2024-04-25T11:30:09.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.209+0000 7f3222800640 1 Processor -- start 2024-04-25T11:30:09.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.209+0000 7f3222800640 1 -- start start 2024-04-25T11:30:09.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f3222800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c13eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f3222800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f321c1434d0 0x7f321c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f3222800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f321c160be0 0x7f321c15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f3222800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0b51a0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f3222800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f321c06f090 con 0x7f321c1434d0 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f3222800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0b5320 con 0x7f321c160be0 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f321b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c13eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f321b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c13eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38344/0 (socket says 172.21.15.19:38344) 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f321b400640 1 -- 172.21.15.19:0/2478186862 learned_addr learned my addr 172.21.15.19:0/2478186862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f3212a00640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f321c1434d0 0x7f321c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.211+0000 7f321be00640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f321c160be0 0x7f321c15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.212+0000 7f321b400640 1 -- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f321c160be0 msgr2=0x7f321c15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.212+0000 7f321b400640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f321c160be0 0x7f321c15e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.212+0000 7f321b400640 1 -- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f321c1434d0 msgr2=0x7f321c0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.212+0000 7f321b400640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f321c1434d0 0x7f321c0ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.212+0000 7f321b400640 1 -- 172.21.15.19:0/2478186862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c154bf0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.212+0000 7f3212a00640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f321c1434d0 0x7f321c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:09.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.212+0000 7f321b400640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c13eeb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f320c00c390 tx=0x7f320c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:09.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.216+0000 7f3213e00640 1 -- 172.21.15.19:0/2478186862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c017070 con 0x7f321c0d0ed0 2024-04-25T11:30:09.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.216+0000 7f3213e00640 1 -- 172.21.15.19:0/2478186862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c005ce0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.216+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c0af220 con 0x7f321c0d0ed0 2024-04-25T11:30:09.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.216+0000 7f3213e00640 1 -- 172.21.15.19:0/2478186862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0072f0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.216+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c0af7e0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.217+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c160450 con 0x7f321c0d0ed0 2024-04-25T11:30:09.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.223+0000 7f3213e00640 1 -- 172.21.15.19:0/2478186862 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f320c01a070 con 0x7f321c0d0ed0 2024-04-25T11:30:09.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.224+0000 7f3213e00640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31f80744b0 0x7f31f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.224+0000 7f3213e00640 1 -- 172.21.15.19:0/2478186862 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f320c0946b0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.225+0000 7f3212a00640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31f80744b0 0x7f31f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.226+0000 7f3212a00640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31f80744b0 0x7f31f80769c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3204034000 tx=0x7f320402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:09.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.231+0000 7f3213e00640 1 -- 172.21.15.19:0/2478186862 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f320c021020 con 0x7f321c0d0ed0 2024-04-25T11:30:09.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.439+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f321c160170 con 0x7f321c0d0ed0 2024-04-25T11:30:09.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.440+0000 7f3213e00640 1 -- 172.21.15.19:0/2478186862 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) ==== 84+0+65 (secure 0 0 0) 0x7f320c05f1f0 con 0x7f321c0d0ed0 2024-04-25T11:30:09.431 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:09.431 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:30:09.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:30:09.431 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T11:30:09.431 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T11:30:09.432 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:30:09.432 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31f80744b0 msgr2=0x7f31f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31f80744b0 0x7f31f80769c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3204034000 tx=0x7f320402e000 comp rx=0 tx=0).stop 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 msgr2=0x7f321c13eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c13eeb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f320c00c390 tx=0x7f320c00c850 comp rx=0 tx=0).stop 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 shutdown_connections 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31f80744b0 0x7f31f80769c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f321c1434d0 0x7f321c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f321c160be0 0x7f321c15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 --2- 172.21.15.19:0/2478186862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f321c0d0ed0 0x7f321c13eeb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 >> 172.21.15.19:0/2478186862 conn(0x7f321c0d1ae0 msgr2=0x7f321c0d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:09.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 shutdown_connections 2024-04-25T11:30:09.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.443+0000 7f3222800640 1 -- 172.21.15.19:0/2478186862 wait complete. 2024-04-25T11:30:09.462 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:30:09.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:30:09.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.759+0000 7f2651200640 1 Processor -- start 2024-04-25T11:30:09.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.759+0000 7f2651200640 1 -- start start 2024-04-25T11:30:09.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.761+0000 7f2651200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f264c0b6910 0x7f264c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.761+0000 7f2651200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f264c0b7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.761+0000 7f2651200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0b7bb0 con 0x7f264c0b8980 2024-04-25T11:30:09.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.761+0000 7f2651200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f264c0d2990 con 0x7f264c0b6910 2024-04-25T11:30:09.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.761+0000 7f2651200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0b94a0 con 0x7f264c0b7260 2024-04-25T11:30:09.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.762+0000 7f264b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f264c0b8980 0x7f264c0ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37720/0 (socket says 172.21.15.19:37720) 2024-04-25T11:30:09.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.762+0000 7f264b400640 1 -- 172.21.15.19:0/4081746937 learned_addr learned my addr 172.21.15.19:0/4081746937 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:09.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.762+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 376883225 0 0) 0x7f264c0b7bb0 con 0x7f264c0b8980 2024-04-25T11:30:09.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.762+0000 7f264aa00640 1 --2- 172.21.15.19:0/4081746937 >> v2:172.21.15.169:3300/0 conn(0x7f264c0b6910 0x7f264c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.762+0000 7f264be00640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f264c0b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.762+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2630003520 con 0x7f264c0b8980 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.762+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3076716405 0 0) 0x7f2630003520 con 0x7f264c0b8980 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 msgr2=0x7f264c0b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f264c0b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 >> v2:172.21.15.169:3300/0 conn(0x7f264c0b6910 msgr2=0x7f264c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 --2- 172.21.15.19:0/4081746937 >> v2:172.21.15.169:3300/0 conn(0x7f264c0b6910 0x7f264c0b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0b91c0 con 0x7f264c0b8980 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f263c001670 con 0x7f264c0b8980 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264aa00640 1 --2- 172.21.15.19:0/4081746937 >> v2:172.21.15.169:3300/0 conn(0x7f264c0b6910 0x7f264c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264be00640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f264c0b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 >> v1:172.21.15.19:6789/0 conn(0x7f264c0b8980 legacy=0x7f264c0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f2630003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2630004800 0x7f2630004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f2630005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0d2990 con 0x7f26300039b0 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0b94a0 con 0x7f2630004800 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f263000bc00 con 0x7f264c0b7260 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264b400640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f2630003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264aa00640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2630004800 0x7f2630004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.763+0000 7f264be00640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f2630005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.764+0000 7f264b400640 1 -- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 msgr2=0x7f2630005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.764+0000 7f264b400640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f2630005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.764+0000 7f264b400640 1 -- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2630004800 msgr2=0x7f2630004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.764+0000 7f264b400640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2630004800 0x7f2630004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.764+0000 7f264b400640 1 -- 172.21.15.19:0/4081746937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2630003520 con 0x7f26300039b0 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.764+0000 7f264be00640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b7260 0x7f2630005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:09.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.765+0000 7f264b400640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f2630003e20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f263c007fd0 tx=0x7f2630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4790139669acfd02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:09.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.765+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c007d50 con 0x7f26300039b0 2024-04-25T11:30:09.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.765+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c005af0 con 0x7f26300039b0 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.765+0000 7f264a000640 1 -- 172.21.15.19:0/4081746937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c007070 con 0x7f26300039b0 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.766+0000 7f2651200640 1 -- 172.21.15.19:0/4081746937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 msgr2=0x7f2630003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.766+0000 7f2651200640 1 --2- 172.21.15.19:0/4081746937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f2630003e20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f263c007fd0 tx=0x7f2630005830 comp rx=0 tx=0).stop 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.766+0000 7f264be00640 1 -- 172.21.15.19:0/4081746937 reap_dead start 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.766+0000 7f2651200640 1 -- 172.21.15.19:0/4081746937 shutdown_connections 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.766+0000 7f2651200640 1 -- 172.21.15.19:0/4081746937 >> 172.21.15.19:0/4081746937 conn(0x7f264c0d0390 msgr2=0x7f264c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.766+0000 7f2651200640 1 -- 172.21.15.19:0/4081746937 shutdown_connections 2024-04-25T11:30:09.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.766+0000 7f2651200640 1 -- 172.21.15.19:0/4081746937 wait complete. 2024-04-25T11:30:09.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.769+0000 7f2651200640 1 Processor -- start 2024-04-25T11:30:09.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.769+0000 7f2651200640 1 -- start start 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.770+0000 7f2651200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f264c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.770+0000 7f2651200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b2b50 0x7f264c0a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f2651200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f264c0a0770 0x7f264c1f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f2651200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f264c09cab0 con 0x7f26300039b0 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f2651200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cf0f0 con 0x7f264c0a0770 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f2651200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f264c09cc30 con 0x7f264c0b2b50 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f264c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f264c0cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38368/0 (socket says 172.21.15.19:38368) 2024-04-25T11:30:09.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 -- 172.21.15.19:0/249956703 learned_addr learned my addr 172.21.15.19:0/249956703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264aa00640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b2b50 0x7f264c0a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264be00640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f264c0a0770 0x7f264c1f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 -- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b2b50 msgr2=0x7f264c0a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b2b50 0x7f264c0a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 -- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f264c0a0770 msgr2=0x7f264c1f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f264c0a0770 0x7f264c1f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.771+0000 7f264b400640 1 -- 172.21.15.19:0/249956703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0a0b80 con 0x7f26300039b0 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.772+0000 7f264b400640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f264c0cc6c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f263c001970 tx=0x7f263c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.772+0000 7f2643400640 1 -- 172.21.15.19:0/249956703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c042020 con 0x7f26300039b0 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.772+0000 7f2643400640 1 -- 172.21.15.19:0/249956703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c004bf0 con 0x7f26300039b0 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.772+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c0ccc00 con 0x7f26300039b0 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.772+0000 7f2643400640 1 -- 172.21.15.19:0/249956703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c001c90 con 0x7f26300039b0 2024-04-25T11:30:09.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.772+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c0b35a0 con 0x7f26300039b0 2024-04-25T11:30:09.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.774+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c076c60 con 0x7f26300039b0 2024-04-25T11:30:09.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.775+0000 7f2643400640 1 -- 172.21.15.19:0/249956703 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f263c045020 con 0x7f26300039b0 2024-04-25T11:30:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.776+0000 7f2643400640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f262c0744b0 0x7f262c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.776+0000 7f2643400640 1 -- 172.21.15.19:0/249956703 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f263c0b58b0 con 0x7f26300039b0 2024-04-25T11:30:09.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.777+0000 7f264aa00640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f262c0744b0 0x7f262c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:09.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.778+0000 7f264aa00640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f262c0744b0 0x7f262c0769c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f264c0b8e40 tx=0x7f2638008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:09.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.781+0000 7f2643400640 1 -- 172.21.15.19:0/249956703 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f263c04c030 con 0x7f26300039b0 2024-04-25T11:30:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:09.990+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f264c0d3600 con 0x7f26300039b0 2024-04-25T11:30:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.223+0000 7f2643400640 1 -- 172.21.15.19:0/249956703 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) ==== 138+0+0 (secure 0 0 0) 0x7f264c0a0b80 con 0x7f26300039b0 2024-04-25T11:30:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T11:30:10.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f262c0744b0 msgr2=0x7f262c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:10.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f262c0744b0 0x7f262c0769c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f264c0b8e40 tx=0x7f2638008040 comp rx=0 tx=0).stop 2024-04-25T11:30:10.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 msgr2=0x7f264c0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:10.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f264c0cc6c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f263c001970 tx=0x7f263c001a40 comp rx=0 tx=0).stop 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 shutdown_connections 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f262c0744b0 0x7f262c0769c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f264c0a0770 0x7f264c1f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f264c0b2b50 0x7f264c0a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 --2- 172.21.15.19:0/249956703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300039b0 0x7f264c0cc6c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.227+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 >> 172.21.15.19:0/249956703 conn(0x7f264c0d0390 msgr2=0x7f264c0b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.228+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 shutdown_connections 2024-04-25T11:30:10.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.228+0000 7f2651200640 1 -- 172.21.15.19:0/249956703 wait complete. 2024-04-25T11:30:10.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:30:10.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.545+0000 7fd303000640 1 Processor -- start 2024-04-25T11:30:10.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.545+0000 7fd303000640 1 -- start start 2024-04-25T11:30:10.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd303000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2fc06dce0 0x7fd2fc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:10.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd303000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd2fc0b0af0 0x7fd2fc15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:10.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd303000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc0cdea0 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd303000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd2fc0ceab0 con 0x7fd2fc06dce0 2024-04-25T11:30:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd303000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd2fc0ce180 con 0x7fd2fc0b0af0 2024-04-25T11:30:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd2fb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2fc06dce0 0x7fd2fc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd2fb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2fc06dce0 0x7fd2fc15f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38374/0 (socket says 172.21.15.19:38374) 2024-04-25T11:30:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.548+0000 7fd2fb400640 1 -- 172.21.15.19:0/22355977 learned_addr learned my addr 172.21.15.19:0/22355977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.549+0000 7fd301000640 1 --2- 172.21.15.19:0/22355977 >> v2:172.21.15.169:3300/0 conn(0x7fd2fc0b0af0 0x7fd2fc15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.549+0000 7fd2fb400640 1 -- 172.21.15.19:0/22355977 >> v2:172.21.15.169:3300/0 conn(0x7fd2fc0b0af0 msgr2=0x7fd2fc15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.549+0000 7fd2fb400640 1 --2- 172.21.15.19:0/22355977 >> v2:172.21.15.169:3300/0 conn(0x7fd2fc0b0af0 0x7fd2fc15faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.549+0000 7fd2fb400640 1 -- 172.21.15.19:0/22355977 >> v1:172.21.15.169:6789/0 conn(0x7fd2fc0cd9a0 legacy=0x7fd2fc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.549+0000 7fd2fb400640 1 -- 172.21.15.19:0/22355977 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0ce460 con 0x7fd2fc06dce0 2024-04-25T11:30:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.549+0000 7fd301000640 1 --2- 172.21.15.19:0/22355977 >> v2:172.21.15.169:3300/0 conn(0x7fd2fc0b0af0 0x7fd2fc15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:10.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.550+0000 7fd2fb400640 1 --2- 172.21.15.19:0/22355977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc06dce0 0x7fd2fc15f360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc148020 tx=0x7fd2e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dfbec5394f0171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.551+0000 7fd2faa00640 1 -- 172.21.15.19:0/22355977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8037070 con 0x7fd2fc06dce0 2024-04-25T11:30:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.551+0000 7fd2faa00640 1 -- 172.21.15.19:0/22355977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8032440 con 0x7fd2fc06dce0 2024-04-25T11:30:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.551+0000 7fd2faa00640 1 -- 172.21.15.19:0/22355977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8032810 con 0x7fd2fc06dce0 2024-04-25T11:30:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 -- 172.21.15.19:0/22355977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc06dce0 msgr2=0x7fd2fc15f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 --2- 172.21.15.19:0/22355977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc06dce0 0x7fd2fc15f360 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc148020 tx=0x7fd2e802a230 comp rx=0 tx=0).stop 2024-04-25T11:30:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 -- 172.21.15.19:0/22355977 shutdown_connections 2024-04-25T11:30:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 --2- 172.21.15.19:0/22355977 >> v2:172.21.15.169:3300/0 conn(0x7fd2fc0b0af0 0x7fd2fc15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 --2- 172.21.15.19:0/22355977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc06dce0 0x7fd2fc15f360 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 -- 172.21.15.19:0/22355977 >> 172.21.15.19:0/22355977 conn(0x7fd2fc073090 msgr2=0x7fd2fc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 -- 172.21.15.19:0/22355977 shutdown_connections 2024-04-25T11:30:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.552+0000 7fd303000640 1 -- 172.21.15.19:0/22355977 wait complete. 2024-04-25T11:30:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.555+0000 7fd303000640 1 Processor -- start 2024-04-25T11:30:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.555+0000 7fd303000640 1 -- start start 2024-04-25T11:30:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.557+0000 7fd303000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2fc0b0af0 0x7fd2fc1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.557+0000 7fd303000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc0cd9a0 0x7fd2fc143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.557+0000 7fd303000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2fc1441d0 0x7fd2fc1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.557+0000 7fd303000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0cef40 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.557+0000 7fd303000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0b0f00 con 0x7fd2fc0b0af0 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.557+0000 7fd303000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc0cf0c0 con 0x7fd2fc1441d0 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc0cd9a0 0x7fd2fc143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc0cd9a0 0x7fd2fc143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38382/0 (socket says 172.21.15.19:38382) 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 -- 172.21.15.19:0/370004615 learned_addr learned my addr 172.21.15.19:0/370004615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fbe00640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2fc0b0af0 0x7fd2fc1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd301000640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2fc1441d0 0x7fd2fc1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 -- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2fc1441d0 msgr2=0x7fd2fc1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2fc1441d0 0x7fd2fc1445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 -- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2fc0b0af0 msgr2=0x7fd2fc1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2fc0b0af0 0x7fd2fc1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fb400640 1 -- 172.21.15.19:0/370004615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0aad50 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.558+0000 7fd2fbe00640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2fc0b0af0 0x7fd2fc1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.559+0000 7fd2fb400640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc0cd9a0 0x7fd2fc143c90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd2e802a270 tx=0x7fd2e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.559+0000 7fd2f3e00640 1 -- 172.21.15.19:0/370004615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8037040 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.559+0000 7fd2f3e00640 1 -- 172.21.15.19:0/370004615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8042070 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.559+0000 7fd2f3e00640 1 -- 172.21.15.19:0/370004615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8030bb0 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.559+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc0ab030 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.559+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc143760 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.561+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc050740 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.562+0000 7fd2f3e00640 1 -- 172.21.15.19:0/370004615 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd2e8047460 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.563+0000 7fd2f3e00640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2dc0744e0 0x7fd2dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.563+0000 7fd2f3e00640 1 -- 172.21.15.19:0/370004615 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5227+0+0 (secure 0 0 0) 0x7fd2e80b5cd0 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.564+0000 7fd2fbe00640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2dc0744e0 0x7fd2dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.566+0000 7fd2fbe00640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2dc0744e0 0x7fd2dc0769f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00a830 tx=0x7fd2ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:10.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.569+0000 7fd2f3e00640 1 -- 172.21.15.19:0/370004615 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2e80805a0 con 0x7fd2fc0cd9a0 2024-04-25T11:30:10.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:10.776+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd2fc0b0390 con 0x7fd2fc0cd9a0 2024-04-25T11:30:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.230+0000 7fd2f3e00640 1 -- 172.21.15.19:0/370004615 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) ==== 170+0+0 (secure 0 0 0) 0x7fd2e807fea0 con 0x7fd2fc0cd9a0 2024-04-25T11:30:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T11:30:11.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.234+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2dc0744e0 msgr2=0x7fd2dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:11.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.234+0000 7fd303000640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2dc0744e0 0x7fd2dc0769f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00a830 tx=0x7fd2ec008040 comp rx=0 tx=0).stop 2024-04-25T11:30:11.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.234+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc0cd9a0 msgr2=0x7fd2fc143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:11.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.234+0000 7fd303000640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc0cd9a0 0x7fd2fc143c90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd2e802a270 tx=0x7fd2e80046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.235+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 shutdown_connections 2024-04-25T11:30:11.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.235+0000 7fd303000640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2dc0744e0 0x7fd2dc0769f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.235+0000 7fd303000640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2fc1441d0 0x7fd2fc1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.235+0000 7fd303000640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2fc0cd9a0 0x7fd2fc143c90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.235+0000 7fd303000640 1 --2- 172.21.15.19:0/370004615 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2fc0b0af0 0x7fd2fc1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.235+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 >> 172.21.15.19:0/370004615 conn(0x7fd2fc073090 msgr2=0x7fd2fc147760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:11.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.235+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 shutdown_connections 2024-04-25T11:30:11.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.236+0000 7fd303000640 1 -- 172.21.15.19:0/370004615 wait complete. 2024-04-25T11:30:11.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:30:11.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.536+0000 7fa928a00640 1 Processor -- start 2024-04-25T11:30:11.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.536+0000 7fa928a00640 1 -- start start 2024-04-25T11:30:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.538+0000 7fa928a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa92415bb20 0x7fa92406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.539+0000 7fa928a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa92406df20 con 0x7fa92406db10 2024-04-25T11:30:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.539+0000 7fa928a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9240d1630 con 0x7fa92415b710 2024-04-25T11:30:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.539+0000 7fa928a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa924169280 con 0x7fa92415bb20 2024-04-25T11:30:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.539+0000 7fa921400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa92415bb20 0x7fa92406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.539+0000 7fa921400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa92415bb20 0x7fa92406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38388/0 (socket says 172.21.15.19:38388) 2024-04-25T11:30:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.539+0000 7fa921400640 1 -- 172.21.15.19:0/885672326 learned_addr learned my addr 172.21.15.19:0/885672326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.540+0000 7fa921400640 1 -- 172.21.15.19:0/885672326 >> v1:172.21.15.169:6789/0 conn(0x7fa92415b710 legacy=0x7fa9240d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.540+0000 7fa921400640 1 -- 172.21.15.19:0/885672326 >> v1:172.21.15.19:6789/0 conn(0x7fa92406db10 legacy=0x7fa924165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.540+0000 7fa921400640 1 -- 172.21.15.19:0/885672326 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa924168670 con 0x7fa92415bb20 2024-04-25T11:30:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.541+0000 7fa921400640 1 --2- 172.21.15.19:0/885672326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415bb20 0x7fa92406d5d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009870 tx=0x7fa90c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a9a5e5a0f0c4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.541+0000 7fa920a00640 1 -- 172.21.15.19:0/885672326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c037070 con 0x7fa92415bb20 2024-04-25T11:30:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.541+0000 7fa920a00640 1 -- 172.21.15.19:0/885672326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90c02ae50 con 0x7fa92415bb20 2024-04-25T11:30:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.541+0000 7fa920a00640 1 -- 172.21.15.19:0/885672326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c032860 con 0x7fa92415bb20 2024-04-25T11:30:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.542+0000 7fa928a00640 1 -- 172.21.15.19:0/885672326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415bb20 msgr2=0x7fa92406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.542+0000 7fa928a00640 1 --2- 172.21.15.19:0/885672326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415bb20 0x7fa92406d5d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009870 tx=0x7fa90c02a300 comp rx=0 tx=0).stop 2024-04-25T11:30:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.542+0000 7fa928a00640 1 -- 172.21.15.19:0/885672326 shutdown_connections 2024-04-25T11:30:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.542+0000 7fa928a00640 1 --2- 172.21.15.19:0/885672326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415bb20 0x7fa92406d5d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.542+0000 7fa928a00640 1 -- 172.21.15.19:0/885672326 >> 172.21.15.19:0/885672326 conn(0x7fa924073090 msgr2=0x7fa9240734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.542+0000 7fa928a00640 1 -- 172.21.15.19:0/885672326 shutdown_connections 2024-04-25T11:30:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.542+0000 7fa928a00640 1 -- 172.21.15.19:0/885672326 wait complete. 2024-04-25T11:30:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.545+0000 7fa928a00640 1 Processor -- start 2024-04-25T11:30:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.545+0000 7fa928a00640 1 -- start start 2024-04-25T11:30:11.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.546+0000 7fa928a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa92406db10 0x7fa924154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:11.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.546+0000 7fa928a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415b710 0x7fa9241f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:11.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.546+0000 7fa928a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa92415bb20 0x7fa92415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:11.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.546+0000 7fa928a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa924168f10 con 0x7fa92415b710 2024-04-25T11:30:11.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.546+0000 7fa928a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa924168c30 con 0x7fa92406db10 2024-04-25T11:30:11.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.546+0000 7fa928a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9240b6180 con 0x7fa92415bb20 2024-04-25T11:30:11.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415b710 0x7fa9241f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:11.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415b710 0x7fa9241f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38400/0 (socket says 172.21.15.19:38400) 2024-04-25T11:30:11.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 -- 172.21.15.19:0/3726407823 learned_addr learned my addr 172.21.15.19:0/3726407823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:11.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921e00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa92406db10 0x7fa924154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:11.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa922800640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa92415bb20 0x7fa92415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:11.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 -- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa92415bb20 msgr2=0x7fa92415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa92415bb20 0x7fa92415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 -- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa92406db10 msgr2=0x7fa924154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa92406db10 0x7fa924154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa921400640 1 -- 172.21.15.19:0/3726407823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92415c600 con 0x7fa92415b710 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.547+0000 7fa922800640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa92415bb20 0x7fa92415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.548+0000 7fa921400640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415b710 0x7fa9241f02a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009870 tx=0x7fa90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.548+0000 7fa91a000640 1 -- 172.21.15.19:0/3726407823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c041070 con 0x7fa92415b710 2024-04-25T11:30:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.548+0000 7fa91a000640 1 -- 172.21.15.19:0/3726407823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90c03c870 con 0x7fa92415b710 2024-04-25T11:30:11.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.548+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92415c8e0 con 0x7fa92415b710 2024-04-25T11:30:11.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.548+0000 7fa91a000640 1 -- 172.21.15.19:0/3726407823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c037040 con 0x7fa92415b710 2024-04-25T11:30:11.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.548+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92415cea0 con 0x7fa92415b710 2024-04-25T11:30:11.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.550+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa924070230 con 0x7fa92415b710 2024-04-25T11:30:11.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.551+0000 7fa91a000640 1 -- 172.21.15.19:0/3726407823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa90c044070 con 0x7fa92415b710 2024-04-25T11:30:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.553+0000 7fa91a000640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9000744b0 0x7fa9000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.553+0000 7fa91a000640 1 -- 172.21.15.19:0/3726407823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5609+0+0 (secure 0 0 0) 0x7fa90c0b5cd0 con 0x7fa92415b710 2024-04-25T11:30:11.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.553+0000 7fa921e00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9000744b0 0x7fa9000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:11.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.555+0000 7fa921e00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9000744b0 0x7fa9000769c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa914007fd0 tx=0x7fa914009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:11.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.560+0000 7fa91a000640 1 -- 172.21.15.19:0/3726407823 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa924070230 con 0x7fa92415b710 2024-04-25T11:30:11.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:11.807+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa924050740 con 0x7fa92415b710 2024-04-25T11:30:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.288+0000 7fa91a000640 1 -- 172.21.15.19:0/3726407823 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v30) ==== 172+0+0 (secure 0 0 0) 0x7fa90c080420 con 0x7fa92415b710 2024-04-25T11:30:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T11:30:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.294+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9000744b0 msgr2=0x7fa9000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.294+0000 7fa928a00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9000744b0 0x7fa9000769c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa914007fd0 tx=0x7fa914009040 comp rx=0 tx=0).stop 2024-04-25T11:30:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.294+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415b710 msgr2=0x7fa9241f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.294+0000 7fa928a00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415b710 0x7fa9241f02a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009870 tx=0x7fa90c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.295+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 shutdown_connections 2024-04-25T11:30:12.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.295+0000 7fa928a00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9000744b0 0x7fa9000769c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.295+0000 7fa928a00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa92415bb20 0x7fa92415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.295+0000 7fa928a00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa92415b710 0x7fa9241f02a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.295+0000 7fa928a00640 1 --2- 172.21.15.19:0/3726407823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa92406db10 0x7fa924154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.295+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 >> 172.21.15.19:0/3726407823 conn(0x7fa924073090 msgr2=0x7fa9240b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:12.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.298+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 shutdown_connections 2024-04-25T11:30:12.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.299+0000 7fa928a00640 1 -- 172.21.15.19:0/3726407823 wait complete. 2024-04-25T11:30:12.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:12.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.595+0000 7f50d3e00640 1 Processor -- start 2024-04-25T11:30:12.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.595+0000 7f50d3e00640 1 -- start start 2024-04-25T11:30:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50d40cf2f0 0x7f50d40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 0x7f50d40b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d40b4e90 con 0x7f50d41608d0 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f50d406f870 con 0x7f50d40cf2f0 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50d40b74d0 con 0x7f50d415a810 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50d40cf2f0 0x7f50d40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50d40cf2f0 0x7f50d40b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38416/0 (socket says 172.21.15.19:38416) 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.597+0000 7f50d2a00640 1 -- 172.21.15.19:0/3958339690 learned_addr learned my addr 172.21.15.19:0/3958339690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.598+0000 7f50d2000640 1 --2- 172.21.15.19:0/3958339690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 0x7f50d40b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.598+0000 7f50d2a00640 1 -- 172.21.15.19:0/3958339690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 msgr2=0x7f50d40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.598+0000 7f50d2a00640 1 --2- 172.21.15.19:0/3958339690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 0x7f50d40b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.598+0000 7f50d2a00640 1 -- 172.21.15.19:0/3958339690 >> v1:172.21.15.169:6789/0 conn(0x7f50d41608d0 legacy=0x7f50d4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.598+0000 7f50d2a00640 1 -- 172.21.15.19:0/3958339690 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d40b77b0 con 0x7f50d40cf2f0 2024-04-25T11:30:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.599+0000 7f50d2a00640 1 --2- 172.21.15.19:0/3958339690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d40b6e10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f50d40d26d0 tx=0x7f50c402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595a3be180bb15ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.599+0000 7f50d1600640 1 -- 172.21.15.19:0/3958339690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4037070 con 0x7f50d40cf2f0 2024-04-25T11:30:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.599+0000 7f50d1600640 1 -- 172.21.15.19:0/3958339690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c4032440 con 0x7f50d40cf2f0 2024-04-25T11:30:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.599+0000 7f50d1600640 1 -- 172.21.15.19:0/3958339690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4032810 con 0x7f50d40cf2f0 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 -- 172.21.15.19:0/3958339690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 msgr2=0x7f50d40b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 --2- 172.21.15.19:0/3958339690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d40b6e10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f50d40d26d0 tx=0x7f50c402a200 comp rx=0 tx=0).stop 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 -- 172.21.15.19:0/3958339690 shutdown_connections 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 --2- 172.21.15.19:0/3958339690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 0x7f50d40b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 --2- 172.21.15.19:0/3958339690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d40b6e10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 -- 172.21.15.19:0/3958339690 >> 172.21.15.19:0/3958339690 conn(0x7f50d40d1720 msgr2=0x7f50d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 -- 172.21.15.19:0/3958339690 shutdown_connections 2024-04-25T11:30:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.600+0000 7f50d3e00640 1 -- 172.21.15.19:0/3958339690 wait complete. 2024-04-25T11:30:12.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.603+0000 7f50d3e00640 1 Processor -- start 2024-04-25T11:30:12.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.603+0000 7f50d3e00640 1 -- start start 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d4153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 0x7f50d41547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d41608d0 0x7f50d4151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50d415ac20 con 0x7f50d40cf2f0 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50d406f500 con 0x7f50d41608d0 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50d406fd00 con 0x7f50d415a810 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d4153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:12.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d4153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38432/0 (socket says 172.21.15.19:38432) 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 -- 172.21.15.19:0/1712439241 learned_addr learned my addr 172.21.15.19:0/1712439241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 -- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 msgr2=0x7f50d41547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 0x7f50d41547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 -- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d41608d0 msgr2=0x7f50d4151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d3400640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d41608d0 0x7f50d4151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d41608d0 0x7f50d4151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 -- 172.21.15.19:0/1712439241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d4154d20 con 0x7f50d40cf2f0 2024-04-25T11:30:12.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.605+0000 7f50d2a00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d4153ed0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:12.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.606+0000 7f50caa00640 1 -- 172.21.15.19:0/1712439241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4041070 con 0x7f50d40cf2f0 2024-04-25T11:30:12.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.606+0000 7f50caa00640 1 -- 172.21.15.19:0/1712439241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c403c870 con 0x7f50d40cf2f0 2024-04-25T11:30:12.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.606+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d41407c0 con 0x7f50d40cf2f0 2024-04-25T11:30:12.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.606+0000 7f50caa00640 1 -- 172.21.15.19:0/1712439241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4037040 con 0x7f50d40cf2f0 2024-04-25T11:30:12.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.606+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d4140d80 con 0x7f50d40cf2f0 2024-04-25T11:30:12.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.607+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d4050740 con 0x7f50d40cf2f0 2024-04-25T11:30:12.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.608+0000 7f50caa00640 1 -- 172.21.15.19:0/1712439241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f50c4044070 con 0x7f50d40cf2f0 2024-04-25T11:30:12.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.609+0000 7f50caa00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50b40744e0 0x7f50b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:12.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.609+0000 7f50caa00640 1 -- 172.21.15.19:0/1712439241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f50c40b5e30 con 0x7f50d40cf2f0 2024-04-25T11:30:12.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.610+0000 7f50d2000640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50b40744e0 0x7f50b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:12.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.611+0000 7f50d2000640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50b40744e0 0x7f50b40769f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f50d4074040 tx=0x7f50c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:12.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.615+0000 7f50caa00640 1 -- 172.21.15.19:0/1712439241 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50c40805b0 con 0x7f50d40cf2f0 2024-04-25T11:30:12.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.821+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50d415a050 con 0x7f50d40cf2f0 2024-04-25T11:30:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.824+0000 7f50caa00640 1 -- 172.21.15.19:0/1712439241 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16622 (secure 0 0 0) 0x7f50c407feb0 con 0x7f50d40cf2f0 2024-04-25T11:30:12.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":49,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:11.809963+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:30:10.008986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T11:30:10.799803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:12.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50b40744e0 msgr2=0x7f50b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:12.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50b40744e0 0x7f50b40769f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f50d4074040 tx=0x7f50c0008040 comp rx=0 tx=0).stop 2024-04-25T11:30:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 msgr2=0x7f50d4153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d4153ed0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000c00 tx=0x7f50c40046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 shutdown_connections 2024-04-25T11:30:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50b40744e0 0x7f50b40769f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50d41608d0 0x7f50d4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50d415a810 0x7f50d41547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 --2- 172.21.15.19:0/1712439241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50d40cf2f0 0x7f50d4153ed0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 >> 172.21.15.19:0/1712439241 conn(0x7f50d40d1720 msgr2=0x7f50d40cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.827+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 shutdown_connections 2024-04-25T11:30:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:12.828+0000 7f50d3e00640 1 -- 172.21.15.19:0/1712439241 wait complete. 2024-04-25T11:30:12.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:13.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.136+0000 7f0475a00640 1 Processor -- start 2024-04-25T11:30:13.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.136+0000 7f0475a00640 1 -- start start 2024-04-25T11:30:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.138+0000 7f0475a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04700b5760 0x7f047006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.138+0000 7f0475a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0470071b90 0x7f04701681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04700720d0 con 0x7f0470071780 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f04700723b0 con 0x7f0470071b90 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04700d07f0 con 0x7f04700b5760 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0470071b90 0x7f04701681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0470071b90 0x7f04701681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38442/0 (socket says 172.21.15.19:38442) 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475000640 1 -- 172.21.15.19:0/1395540487 learned_addr learned my addr 172.21.15.19:0/1395540487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f046fe00640 1 --2- 172.21.15.19:0/1395540487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04700b5760 0x7f047006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475000640 1 -- 172.21.15.19:0/1395540487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04700b5760 msgr2=0x7f047006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.139+0000 7f0475000640 1 --2- 172.21.15.19:0/1395540487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04700b5760 0x7f047006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.140+0000 7f0475000640 1 -- 172.21.15.19:0/1395540487 >> v1:172.21.15.169:6789/0 conn(0x7f0470071780 legacy=0x7f04700b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:13.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.140+0000 7f0475000640 1 -- 172.21.15.19:0/1395540487 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04700d0ad0 con 0x7f0470071b90 2024-04-25T11:30:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.141+0000 7f0475000640 1 --2- 172.21.15.19:0/1395540487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f04701681b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0468009870 tx=0x7f046802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fb584cc0d9259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.141+0000 7f046ea00640 1 -- 172.21.15.19:0/1395540487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468037070 con 0x7f0470071b90 2024-04-25T11:30:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.141+0000 7f046ea00640 1 -- 172.21.15.19:0/1395540487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046802ae10 con 0x7f0470071b90 2024-04-25T11:30:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.141+0000 7f046ea00640 1 -- 172.21.15.19:0/1395540487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468032660 con 0x7f0470071b90 2024-04-25T11:30:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 -- 172.21.15.19:0/1395540487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 msgr2=0x7f04701681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 --2- 172.21.15.19:0/1395540487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f04701681b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0468009870 tx=0x7f046802a100 comp rx=0 tx=0).stop 2024-04-25T11:30:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 -- 172.21.15.19:0/1395540487 shutdown_connections 2024-04-25T11:30:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 --2- 172.21.15.19:0/1395540487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f04701681b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 --2- 172.21.15.19:0/1395540487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04700b5760 0x7f047006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 -- 172.21.15.19:0/1395540487 >> 172.21.15.19:0/1395540487 conn(0x7f0470070270 msgr2=0x7f0470070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 -- 172.21.15.19:0/1395540487 shutdown_connections 2024-04-25T11:30:13.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.142+0000 7f0475a00640 1 -- 172.21.15.19:0/1395540487 wait complete. 2024-04-25T11:30:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.144+0000 7f0475a00640 1 Processor -- start 2024-04-25T11:30:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.145+0000 7f0475a00640 1 -- start start 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.146+0000 7f0475a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0470071780 0x7f047015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.146+0000 7f0475a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f0470165510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.146+0000 7f0475a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700b5760 0x7f0470167120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.146+0000 7f0475a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0470164b30 con 0x7f0470071b90 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.146+0000 7f0475a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04701649b0 con 0x7f04700b5760 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.146+0000 7f0475a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04700b6960 con 0x7f0470071780 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.146+0000 7f046f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f0470165510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f0470165510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38454/0 (socket says 172.21.15.19:38454) 2024-04-25T11:30:13.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 -- 172.21.15.19:0/1631648109 learned_addr learned my addr 172.21.15.19:0/1631648109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 -- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0470071780 msgr2=0x7f047015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0470071780 0x7f047015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 -- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700b5760 msgr2=0x7f0470167120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700b5760 0x7f0470167120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 -- 172.21.15.19:0/1631648109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0470166280 con 0x7f0470071b90 2024-04-25T11:30:13.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f046f400640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f0470165510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f045c00ba50 tx=0x7f045c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.147+0000 7f0467e00640 1 -- 172.21.15.19:0/1631648109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c019070 con 0x7f0470071b90 2024-04-25T11:30:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.148+0000 7f0467e00640 1 -- 172.21.15.19:0/1631648109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c00cc20 con 0x7f0470071b90 2024-04-25T11:30:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.148+0000 7f0467e00640 1 -- 172.21.15.19:0/1631648109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c014680 con 0x7f0470071b90 2024-04-25T11:30:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.148+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0470166560 con 0x7f0470071b90 2024-04-25T11:30:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.148+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0470166b20 con 0x7f0470071b90 2024-04-25T11:30:13.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.149+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0470165ff0 con 0x7f0470071b90 2024-04-25T11:30:13.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.150+0000 7f0467e00640 1 -- 172.21.15.19:0/1631648109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f045c01c070 con 0x7f0470071b90 2024-04-25T11:30:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.151+0000 7f0467e00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04500744e0 0x7f04500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.151+0000 7f0467e00640 1 -- 172.21.15.19:0/1631648109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f045c094ec0 con 0x7f0470071b90 2024-04-25T11:30:13.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.152+0000 7f046fe00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04500744e0 0x7f04500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.153+0000 7f046fe00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04500744e0 0x7f04500769f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0460006fd0 tx=0x7f0460008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:13.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.157+0000 7f0467e00640 1 -- 172.21.15.19:0/1631648109 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f045c05f000 con 0x7f0470071b90 2024-04-25T11:30:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.370+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0470069a30 con 0x7f0470071b90 2024-04-25T11:30:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.373+0000 7f0467e00640 1 -- 172.21.15.19:0/1631648109 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16609 (secure 0 0 0) 0x7f0470069a30 con 0x7f0470071b90 2024-04-25T11:30:13.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:13.363 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:13.289711+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:30:10.008986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T11:30:10.799803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04500744e0 msgr2=0x7f04500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04500744e0 0x7f04500769f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0460006fd0 tx=0x7f0460008040 comp rx=0 tx=0).stop 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 msgr2=0x7f0470165510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f0470165510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f045c00ba50 tx=0x7f045c00bf10 comp rx=0 tx=0).stop 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 shutdown_connections 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04500744e0 0x7f04500769f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04700b5760 0x7f0470167120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0470071b90 0x7f0470165510 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 --2- 172.21.15.19:0/1631648109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0470071780 0x7f047015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 >> 172.21.15.19:0/1631648109 conn(0x7f0470070270 msgr2=0x7f0470070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:13.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 shutdown_connections 2024-04-25T11:30:13.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.376+0000 7f0475a00640 1 -- 172.21.15.19:0/1631648109 wait complete. 2024-04-25T11:30:13.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:30:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.690+0000 7fc198600640 1 Processor -- start 2024-04-25T11:30:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.690+0000 7fc198600640 1 -- start start 2024-04-25T11:30:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.692+0000 7fc198600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900cd9a0 0x7fc19015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.692+0000 7fc198600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1900b0af0 0x7fc19006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.692+0000 7fc198600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc19006db10 0x7fc190161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.692+0000 7fc198600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc1900ceab0 con 0x7fc1900b0af0 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.692+0000 7fc198600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc19006df20 con 0x7fc19006db10 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.693+0000 7fc198600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900cdea0 con 0x7fc1900cd9a0 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.693+0000 7fc195200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1900b0af0 0x7fc19006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.693+0000 7fc195200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1900b0af0 0x7fc19006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38458/0 (socket says 172.21.15.19:38458) 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.693+0000 7fc195200640 1 -- 172.21.15.19:0/1777711725 learned_addr learned my addr 172.21.15.19:0/1777711725 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.693+0000 7fc196600640 1 --2- 172.21.15.19:0/1777711725 >> v2:172.21.15.169:3300/0 conn(0x7fc19006db10 0x7fc190161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.693+0000 7fc195c00640 1 --2- 172.21.15.19:0/1777711725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900cd9a0 0x7fc19015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.694+0000 7fc195200640 1 -- 172.21.15.19:0/1777711725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900cd9a0 msgr2=0x7fc19015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.694+0000 7fc195200640 1 --2- 172.21.15.19:0/1777711725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900cd9a0 0x7fc19015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.694+0000 7fc195200640 1 -- 172.21.15.19:0/1777711725 >> v2:172.21.15.169:3300/0 conn(0x7fc19006db10 msgr2=0x7fc190161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:13.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.694+0000 7fc195200640 1 --2- 172.21.15.19:0/1777711725 >> v2:172.21.15.169:3300/0 conn(0x7fc19006db10 0x7fc190161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.694+0000 7fc195200640 1 -- 172.21.15.19:0/1777711725 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1900ce180 con 0x7fc1900b0af0 2024-04-25T11:30:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.694+0000 7fc196600640 1 --2- 172.21.15.19:0/1777711725 >> v2:172.21.15.169:3300/0 conn(0x7fc19006db10 0x7fc190161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.695+0000 7fc195200640 1 --2- 172.21.15.19:0/1777711725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1900b0af0 0x7fc19006d5d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc190073020 tx=0x7fc17c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00b855c2896588b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.695+0000 7fc18fe00640 1 -- 172.21.15.19:0/1777711725 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c037070 con 0x7fc1900b0af0 2024-04-25T11:30:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.695+0000 7fc18fe00640 1 -- 172.21.15.19:0/1777711725 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc17c032440 con 0x7fc1900b0af0 2024-04-25T11:30:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.695+0000 7fc18fe00640 1 -- 172.21.15.19:0/1777711725 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c032810 con 0x7fc1900b0af0 2024-04-25T11:30:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.696+0000 7fc198600640 1 -- 172.21.15.19:0/1777711725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1900b0af0 msgr2=0x7fc19006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.696+0000 7fc198600640 1 --2- 172.21.15.19:0/1777711725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1900b0af0 0x7fc19006d5d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc190073020 tx=0x7fc17c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.697+0000 7fc198600640 1 -- 172.21.15.19:0/1777711725 shutdown_connections 2024-04-25T11:30:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.697+0000 7fc198600640 1 --2- 172.21.15.19:0/1777711725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1900b0af0 0x7fc19006d5d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.697+0000 7fc198600640 1 --2- 172.21.15.19:0/1777711725 >> v2:172.21.15.169:3300/0 conn(0x7fc19006db10 0x7fc190161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.697+0000 7fc198600640 1 --2- 172.21.15.19:0/1777711725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900cd9a0 0x7fc19015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.697+0000 7fc198600640 1 -- 172.21.15.19:0/1777711725 >> 172.21.15.19:0/1777711725 conn(0x7fc190073090 msgr2=0x7fc1900734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.697+0000 7fc198600640 1 -- 172.21.15.19:0/1777711725 shutdown_connections 2024-04-25T11:30:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.697+0000 7fc198600640 1 -- 172.21.15.19:0/1777711725 wait complete. 2024-04-25T11:30:13.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.700+0000 7fc198600640 1 Processor -- start 2024-04-25T11:30:13.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.700+0000 7fc198600640 1 -- start start 2024-04-25T11:30:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc198600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19006db10 0x7fc1901421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc198600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900b0af0 0x7fc19006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc198600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1900cd9a0 0x7fc190156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc198600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900b0f00 con 0x7fc19006db10 2024-04-25T11:30:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc198600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc1900ce740 con 0x7fc1900cd9a0 2024-04-25T11:30:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc198600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900cef40 con 0x7fc1900b0af0 2024-04-25T11:30:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc195c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19006db10 0x7fc1901421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc195c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19006db10 0x7fc1901421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38464/0 (socket says 172.21.15.19:38464) 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.702+0000 7fc195c00640 1 -- 172.21.15.19:0/2539313693 learned_addr learned my addr 172.21.15.19:0/2539313693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc196600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1900cd9a0 0x7fc190156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc195200640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900b0af0 0x7fc19006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc195c00640 1 -- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900b0af0 msgr2=0x7fc19006f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc195c00640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900b0af0 0x7fc19006f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc195c00640 1 -- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1900cd9a0 msgr2=0x7fc190156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc195c00640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1900cd9a0 0x7fc190156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc195c00640 1 -- 172.21.15.19:0/2539313693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901471a0 con 0x7fc19006db10 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.703+0000 7fc195c00640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19006db10 0x7fc1901421d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e8a0 tx=0x7fc18400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.704+0000 7fc18d600640 1 -- 172.21.15.19:0/2539313693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184017070 con 0x7fc19006db10 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.704+0000 7fc18d600640 1 -- 172.21.15.19:0/2539313693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18400c940 con 0x7fc19006db10 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.704+0000 7fc18d600640 1 -- 172.21.15.19:0/2539313693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400cd30 con 0x7fc19006db10 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.704+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc190147480 con 0x7fc19006db10 2024-04-25T11:30:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.704+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc190147a40 con 0x7fc19006db10 2024-04-25T11:30:13.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.705+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc190146f00 con 0x7fc19006db10 2024-04-25T11:30:13.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.706+0000 7fc18d600640 1 -- 172.21.15.19:0/2539313693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc18401a070 con 0x7fc19006db10 2024-04-25T11:30:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.708+0000 7fc18d600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1700744b0 0x7fc1700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.708+0000 7fc18d600640 1 -- 172.21.15.19:0/2539313693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fc184095530 con 0x7fc19006db10 2024-04-25T11:30:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.709+0000 7fc195200640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1700744b0 0x7fc1700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.710+0000 7fc195200640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1700744b0 0x7fc1700769c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc17c034000 tx=0x7fc17c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.712+0000 7fc18d600640 1 -- 172.21.15.19:0/2539313693 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc190146f00 con 0x7fc19006db10 2024-04-25T11:30:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:13.960+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc190050740 con 0x7fc19006db10 2024-04-25T11:30:14.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.317+0000 7fc18d600640 1 -- 172.21.15.19:0/2539313693 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7fc18405fc20 con 0x7fc19006db10 2024-04-25T11:30:14.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1700744b0 msgr2=0x7fc1700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:14.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1700744b0 0x7fc1700769c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc17c034000 tx=0x7fc17c02e000 comp rx=0 tx=0).stop 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19006db10 msgr2=0x7fc1901421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19006db10 0x7fc1901421d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e8a0 tx=0x7fc18400eca0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 shutdown_connections 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1700744b0 0x7fc1700769c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1900cd9a0 0x7fc190156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1900b0af0 0x7fc19006f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 --2- 172.21.15.19:0/2539313693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc19006db10 0x7fc1901421d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.322+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 >> 172.21.15.19:0/2539313693 conn(0x7fc190073090 msgr2=0x7fc1900cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.323+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 shutdown_connections 2024-04-25T11:30:14.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.323+0000 7fc198600640 1 -- 172.21.15.19:0/2539313693 wait complete. 2024-04-25T11:30:14.342 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:30:14.343 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:30:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.643+0000 7f08d2a00640 1 Processor -- start 2024-04-25T11:30:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.643+0000 7f08d2a00640 1 -- start start 2024-04-25T11:30:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.645+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 0x7f08cc16afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.646+0000 7f08d2a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc1612a0 con 0x7f08cc1647c0 2024-04-25T11:30:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.646+0000 7f08d2a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc161580 con 0x7f08cc0b8060 2024-04-25T11:30:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.646+0000 7f08d2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc164330 con 0x7f08cc0d4bb0 2024-04-25T11:30:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.646+0000 7f08d1600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f08cc1647c0 0x7f08cc0b7950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38838/0 (socket says 172.21.15.19:38838) 2024-04-25T11:30:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.646+0000 7f08d1600640 1 -- 172.21.15.19:0/1347916921 learned_addr learned my addr 172.21.15.19:0/1347916921 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.646+0000 7f08d2000640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 0x7f08cc16afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.646+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826396371 0 0) 0x7f08cc1612a0 con 0x7f08cc1647c0 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08b4003520 con 0x7f08cc1647c0 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1675573837 0 0) 0x7f08b4003520 con 0x7f08cc1647c0 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 msgr2=0x7f08cc16afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 0x7f08cc16afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 >> v1:172.21.15.169:6789/0 conn(0x7f08cc0b8060 legacy=0x7f08cc0d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc163ce0 con 0x7f08cc1647c0 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f08c0001670 con 0x7f08cc1647c0 2024-04-25T11:30:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 >> v1:172.21.15.19:6789/0 conn(0x7f08cc1647c0 legacy=0x7f08cc0b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 0x7f08b4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b4003f80 0x7f08b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08b4005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc164330 con 0x7f08b4004d40 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08b400bb80 con 0x7f08b4003f80 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08b400bd80 con 0x7f08cc0d4bb0 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.647+0000 7f08d2000640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08b4005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.648+0000 7f08d1600640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 0x7f08b4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.648+0000 7f08d0c00640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b4003f80 0x7f08b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.648+0000 7f08d2000640 1 -- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 msgr2=0x7f08b4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.648+0000 7f08d2000640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0d4bb0 0x7f08b4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.648+0000 7f08d2000640 1 -- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b4003f80 msgr2=0x7f08b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.648+0000 7f08d2000640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08b4003f80 0x7f08b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.648+0000 7f08d2000640 1 -- 172.21.15.19:0/1347916921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b4003520 con 0x7f08b4004d40 2024-04-25T11:30:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.649+0000 7f08d2000640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08b4005170 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009fc0 tx=0x7f08b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919f046e773e345b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.649+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4034020 con 0x7f08b4004d40 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.649+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c40305f0 con 0x7f08b4004d40 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.649+0000 7f08cbe00640 1 -- 172.21.15.19:0/1347916921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4009c10 con 0x7f08b4004d40 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.650+0000 7f08d2a00640 1 -- 172.21.15.19:0/1347916921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 msgr2=0x7f08b4005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.650+0000 7f08d2a00640 1 --2- 172.21.15.19:0/1347916921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08b4005170 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009fc0 tx=0x7f08b4005830 comp rx=0 tx=0).stop 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.650+0000 7f08d2000640 1 -- 172.21.15.19:0/1347916921 reap_dead start 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.650+0000 7f08d2a00640 1 -- 172.21.15.19:0/1347916921 shutdown_connections 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.650+0000 7f08d2a00640 1 -- 172.21.15.19:0/1347916921 >> 172.21.15.19:0/1347916921 conn(0x7f08cc0d2a10 msgr2=0x7f08cc0d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.650+0000 7f08d2a00640 1 -- 172.21.15.19:0/1347916921 shutdown_connections 2024-04-25T11:30:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.650+0000 7f08d2a00640 1 -- 172.21.15.19:0/1347916921 wait complete. 2024-04-25T11:30:14.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.653+0000 7f08d2a00640 1 Processor -- start 2024-04-25T11:30:14.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.653+0000 7f08d2a00640 1 -- start start 2024-04-25T11:30:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08cc0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0cf140 0x7f08cc15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08cc1f6360 0x7f08cc1f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d2a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc164bd0 con 0x7f08b4004d40 2024-04-25T11:30:14.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d2a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc0d0b40 con 0x7f08cc1f6360 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d2a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0d4fc0 con 0x7f08cc0cf140 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08cc0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08cc0adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34156/0 (socket says 172.21.15.19:34156) 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 -- 172.21.15.19:0/156086854 learned_addr learned my addr 172.21.15.19:0/156086854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d0c00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0cf140 0x7f08cc15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 -- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0cf140 msgr2=0x7f08cc15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0cf140 0x7f08cc15e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d2000640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08cc1f6360 0x7f08cc1f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 -- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08cc1f6360 msgr2=0x7f08cc1f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08cc1f6360 0x7f08cc1f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d1600640 1 -- 172.21.15.19:0/156086854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc0aca60 con 0x7f08b4004d40 2024-04-25T11:30:14.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.655+0000 7f08d0c00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0cf140 0x7f08cc15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.656+0000 7f08d1600640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08cc0adb00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e840 tx=0x7f08c000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.656+0000 7f08c9600640 1 -- 172.21.15.19:0/156086854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c000c870 con 0x7f08b4004d40 2024-04-25T11:30:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.656+0000 7f08c9600640 1 -- 172.21.15.19:0/156086854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c0018070 con 0x7f08b4004d40 2024-04-25T11:30:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.656+0000 7f08c9600640 1 -- 172.21.15.19:0/156086854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c000cc50 con 0x7f08b4004d40 2024-04-25T11:30:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.656+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc0acd40 con 0x7f08b4004d40 2024-04-25T11:30:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.656+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc0ad300 con 0x7f08b4004d40 2024-04-25T11:30:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.658+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc0b8380 con 0x7f08b4004d40 2024-04-25T11:30:14.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.659+0000 7f08c9600640 1 -- 172.21.15.19:0/156086854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f08c0013420 con 0x7f08b4004d40 2024-04-25T11:30:14.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.660+0000 7f08c9600640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08ac0744e0 0x7f08ac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:14.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.660+0000 7f08c9600640 1 -- 172.21.15.19:0/156086854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f08c0094e10 con 0x7f08b4004d40 2024-04-25T11:30:14.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.661+0000 7f08d0c00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08ac0744e0 0x7f08ac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:14.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.662+0000 7f08d0c00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08ac0744e0 0x7f08ac0769f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f08cc06e4a0 tx=0x7f08b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:14.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.666+0000 7f08c9600640 1 -- 172.21.15.19:0/156086854 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08c0098030 con 0x7f08b4004d40 2024-04-25T11:30:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:14.913+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f08cc076c60 con 0x7f08b4004d40 2024-04-25T11:30:15.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.321+0000 7f08c9600640 1 -- 172.21.15.19:0/156086854 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f08c005f590 con 0x7f08b4004d40 2024-04-25T11:30:15.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08ac0744e0 msgr2=0x7f08ac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:15.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08ac0744e0 0x7f08ac0769f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f08cc06e4a0 tx=0x7f08b8009040 comp rx=0 tx=0).stop 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 msgr2=0x7f08cc0adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08cc0adb00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e840 tx=0x7f08c000ed00 comp rx=0 tx=0).stop 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 shutdown_connections 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08ac0744e0 0x7f08ac0769f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08cc1f6360 0x7f08cc1f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08cc0cf140 0x7f08cc15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 --2- 172.21.15.19:0/156086854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08b4004d40 0x7f08cc0adb00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 >> 172.21.15.19:0/156086854 conn(0x7f08cc0d2a10 msgr2=0x7f08cc0d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.324+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 shutdown_connections 2024-04-25T11:30:15.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.325+0000 7f08d2a00640 1 -- 172.21.15.19:0/156086854 wait complete. 2024-04-25T11:30:15.344 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:30:15.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.639+0000 7f60f4e00640 1 Processor -- start 2024-04-25T11:30:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.639+0000 7f60f4e00640 1 -- start start 2024-04-25T11:30:15.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.641+0000 7f60f4e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f60f00cf2f0 0x7f60f00b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:15.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.641+0000 7f60f4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60f015a810 0x7f60f00b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:15.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.641+0000 7f60f4e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f00b4e90 con 0x7f60f01608d0 2024-04-25T11:30:15.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.641+0000 7f60f4e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f60f006f870 con 0x7f60f015a810 2024-04-25T11:30:15.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.641+0000 7f60f4e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f60f00b74d0 con 0x7f60f00cf2f0 2024-04-25T11:30:15.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.642+0000 7f60eea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60f015a810 0x7f60f00b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:15.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.642+0000 7f60eea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60f015a810 0x7f60f00b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34164/0 (socket says 172.21.15.19:34164) 2024-04-25T11:30:15.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.642+0000 7f60eea00640 1 -- 172.21.15.19:0/2693161503 learned_addr learned my addr 172.21.15.19:0/2693161503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:15.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.642+0000 7f60ef400640 1 --2- 172.21.15.19:0/2693161503 >> v2:172.21.15.169:3300/0 conn(0x7f60f00cf2f0 0x7f60f00b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:15.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.643+0000 7f60eea00640 1 -- 172.21.15.19:0/2693161503 >> v2:172.21.15.169:3300/0 conn(0x7f60f00cf2f0 msgr2=0x7f60f00b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:15.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.643+0000 7f60eea00640 1 --2- 172.21.15.19:0/2693161503 >> v2:172.21.15.169:3300/0 conn(0x7f60f00cf2f0 0x7f60f00b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.643+0000 7f60eea00640 1 -- 172.21.15.19:0/2693161503 >> v1:172.21.15.169:6789/0 conn(0x7f60f01608d0 legacy=0x7f60f0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:15.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.643+0000 7f60eea00640 1 -- 172.21.15.19:0/2693161503 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f00b77b0 con 0x7f60f015a810 2024-04-25T11:30:15.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.644+0000 7f60eea00640 1 --2- 172.21.15.19:0/2693161503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f015a810 0x7f60f00b4950 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f60f00d26d0 tx=0x7f60d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda0f0a46bad30e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:15.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.644+0000 7f60ee000640 1 -- 172.21.15.19:0/2693161503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8037070 con 0x7f60f015a810 2024-04-25T11:30:15.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.644+0000 7f60ee000640 1 -- 172.21.15.19:0/2693161503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d8032440 con 0x7f60f015a810 2024-04-25T11:30:15.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.644+0000 7f60ee000640 1 -- 172.21.15.19:0/2693161503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8032810 con 0x7f60f015a810 2024-04-25T11:30:15.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 -- 172.21.15.19:0/2693161503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f015a810 msgr2=0x7f60f00b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:15.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 --2- 172.21.15.19:0/2693161503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f015a810 0x7f60f00b4950 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f60f00d26d0 tx=0x7f60d802a230 comp rx=0 tx=0).stop 2024-04-25T11:30:15.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 -- 172.21.15.19:0/2693161503 shutdown_connections 2024-04-25T11:30:15.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 --2- 172.21.15.19:0/2693161503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f015a810 0x7f60f00b4950 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 --2- 172.21.15.19:0/2693161503 >> v2:172.21.15.169:3300/0 conn(0x7f60f00cf2f0 0x7f60f00b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 -- 172.21.15.19:0/2693161503 >> 172.21.15.19:0/2693161503 conn(0x7f60f00d1720 msgr2=0x7f60f00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:15.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 -- 172.21.15.19:0/2693161503 shutdown_connections 2024-04-25T11:30:15.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.645+0000 7f60f4e00640 1 -- 172.21.15.19:0/2693161503 wait complete. 2024-04-25T11:30:15.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.648+0000 7f60f4e00640 1 Processor -- start 2024-04-25T11:30:15.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.648+0000 7f60f4e00640 1 -- start start 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.649+0000 7f60f4e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f00cf2f0 0x7f60f00af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.649+0000 7f60f4e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f015a810 0x7f60f0159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60f4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f01608d0 0x7f60f01f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60f4e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60f015ac20 con 0x7f60f01608d0 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60f4e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60f006f500 con 0x7f60f015a810 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60f4e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60f006fd00 con 0x7f60f00cf2f0 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f01608d0 0x7f60f01f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f01608d0 0x7f60f01f9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34172/0 (socket says 172.21.15.19:34172) 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60efe00640 1 -- 172.21.15.19:0/1854341636 learned_addr learned my addr 172.21.15.19:0/1854341636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:15.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60ef400640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f00cf2f0 0x7f60f00af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:15.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60eea00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f015a810 0x7f60f0159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60efe00640 1 -- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f00cf2f0 msgr2=0x7f60f00af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.650+0000 7f60efe00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f00cf2f0 0x7f60f00af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60efe00640 1 -- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f015a810 msgr2=0x7f60f0159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60efe00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f015a810 0x7f60f0159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60efe00640 1 -- 172.21.15.19:0/1854341636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f00afaf0 con 0x7f60f01608d0 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60efe00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f01608d0 0x7f60f01f9e40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f60e800b410 tx=0x7f60e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60e7400640 1 -- 172.21.15.19:0/1854341636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8015030 con 0x7f60f01608d0 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60e7400640 1 -- 172.21.15.19:0/1854341636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e8004020 con 0x7f60f01608d0 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60e7400640 1 -- 172.21.15.19:0/1854341636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8004410 con 0x7f60f01608d0 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f00afdd0 con 0x7f60f01608d0 2024-04-25T11:30:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.651+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f015b2d0 con 0x7f60f01608d0 2024-04-25T11:30:15.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.652+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f0050740 con 0x7f60f01608d0 2024-04-25T11:30:15.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.654+0000 7f60e7400640 1 -- 172.21.15.19:0/1854341636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f60e8018070 con 0x7f60f01608d0 2024-04-25T11:30:15.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.655+0000 7f60e7400640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60cc0744e0 0x7f60cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.655+0000 7f60e7400640 1 -- 172.21.15.19:0/1854341636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f60e8094d20 con 0x7f60f01608d0 2024-04-25T11:30:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.655+0000 7f60ef400640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60cc0744e0 0x7f60cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:15.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.657+0000 7f60ef400640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60cc0744e0 0x7f60cc0769f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f60e00073f0 tx=0x7f60e000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:15.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.660+0000 7f60e7400640 1 -- 172.21.15.19:0/1854341636 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60e805f4a0 con 0x7f60f01608d0 2024-04-25T11:30:15.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.907+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60f00b6f80 con 0x7f60f01608d0 2024-04-25T11:30:15.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.909+0000 7f60e7400640 1 -- 172.21.15.19:0/1854341636 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5184 (secure 0 0 0) 0x7f60e805eda0 con 0x7f60f01608d0 2024-04-25T11:30:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:15.899 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:14.915169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3038680263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3038680263},{"type":"v1","addr":"172.21.15.19:6835","nonce":3038680263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":3}]} 2024-04-25T11:30:15.899 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-04-25T11:30:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60cc0744e0 msgr2=0x7f60cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60cc0744e0 0x7f60cc0769f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f60e00073f0 tx=0x7f60e000b040 comp rx=0 tx=0).stop 2024-04-25T11:30:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f01608d0 msgr2=0x7f60f01f9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:15.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f01608d0 0x7f60f01f9e40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f60e800b410 tx=0x7f60e800b8d0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 shutdown_connections 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60cc0744e0 0x7f60cc0769f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f015a810 0x7f60f0159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f01608d0 0x7f60f01f9e40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 --2- 172.21.15.19:0/1854341636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f00cf2f0 0x7f60f00af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 >> 172.21.15.19:0/1854341636 conn(0x7f60f00d1720 msgr2=0x7f60f00d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 shutdown_connections 2024-04-25T11:30:15.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:15.912+0000 7f60f4e00640 1 -- 172.21.15.19:0/1854341636 wait complete. 2024-04-25T11:30:15.931 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:16.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.224+0000 7fb610a00640 1 Processor -- start 2024-04-25T11:30:16.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.224+0000 7fb610a00640 1 -- start start 2024-04-25T11:30:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb610a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb60c0b75f0 0x7fb60c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb610a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb60c1605c0 0x7fb60c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb610a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c169000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb610a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb60c0cf680 con 0x7fb60c1605c0 2024-04-25T11:30:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb610a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb60c06c6f0 con 0x7fb60c0b75f0 2024-04-25T11:30:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb610a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb60c0cfb40 con 0x7fb60c0cf270 2024-04-25T11:30:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb609e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb60c1605c0 0x7fb60c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb609e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb60c1605c0 0x7fb60c0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34188/0 (socket says 172.21.15.19:34188) 2024-04-25T11:30:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.227+0000 7fb609e00640 1 -- 172.21.15.19:0/652266004 learned_addr learned my addr 172.21.15.19:0/652266004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.228+0000 7fb60a800640 1 --2- 172.21.15.19:0/652266004 >> v2:172.21.15.169:3300/0 conn(0x7fb60c0b75f0 0x7fb60c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.228+0000 7fb609e00640 1 -- 172.21.15.19:0/652266004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 msgr2=0x7fb60c169000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.228+0000 7fb60b200640 1 --2- 172.21.15.19:0/652266004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.228+0000 7fb609e00640 1 --2- 172.21.15.19:0/652266004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c169000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.228+0000 7fb609e00640 1 -- 172.21.15.19:0/652266004 >> v2:172.21.15.169:3300/0 conn(0x7fb60c0b75f0 msgr2=0x7fb60c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.228+0000 7fb609e00640 1 --2- 172.21.15.19:0/652266004 >> v2:172.21.15.169:3300/0 conn(0x7fb60c0b75f0 0x7fb60c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.228+0000 7fb609e00640 1 -- 172.21.15.19:0/652266004 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c0764e0 con 0x7fb60c1605c0 2024-04-25T11:30:16.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.229+0000 7fb609e00640 1 --2- 172.21.15.19:0/652266004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c0ced30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cfa90 tx=0x7fb5f802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc28f041f707a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:16.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.229+0000 7fb609400640 1 -- 172.21.15.19:0/652266004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8037070 con 0x7fb60c1605c0 2024-04-25T11:30:16.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.229+0000 7fb609400640 1 -- 172.21.15.19:0/652266004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f8032440 con 0x7fb60c1605c0 2024-04-25T11:30:16.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.229+0000 7fb609400640 1 -- 172.21.15.19:0/652266004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8032810 con 0x7fb60c1605c0 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 -- 172.21.15.19:0/652266004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 msgr2=0x7fb60c0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 --2- 172.21.15.19:0/652266004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c0ced30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cfa90 tx=0x7fb5f802a1d0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 -- 172.21.15.19:0/652266004 shutdown_connections 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 --2- 172.21.15.19:0/652266004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c0ced30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 --2- 172.21.15.19:0/652266004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c169000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 --2- 172.21.15.19:0/652266004 >> v2:172.21.15.169:3300/0 conn(0x7fb60c0b75f0 0x7fb60c0ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 -- 172.21.15.19:0/652266004 >> 172.21.15.19:0/652266004 conn(0x7fb60c09b7d0 msgr2=0x7fb60c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 -- 172.21.15.19:0/652266004 shutdown_connections 2024-04-25T11:30:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.230+0000 7fb610a00640 1 -- 172.21.15.19:0/652266004 wait complete. 2024-04-25T11:30:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.233+0000 7fb610a00640 1 Processor -- start 2024-04-25T11:30:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.233+0000 7fb610a00640 1 -- start start 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.234+0000 7fb610a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60c0b75f0 0x7fb60c151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.234+0000 7fb610a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.234+0000 7fb610a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.234+0000 7fb610a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c076200 con 0x7fb60c1605c0 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.234+0000 7fb610a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c075f20 con 0x7fb60c0b75f0 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.234+0000 7fb610a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb60c06cb80 con 0x7fb60c0cf270 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c15bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34204/0 (socket says 172.21.15.19:34204) 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 -- 172.21.15.19:0/1022605264 learned_addr learned my addr 172.21.15.19:0/1022605264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb609e00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 -- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 msgr2=0x7fb60c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60a800640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60c0b75f0 0x7fb60c151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 -- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60c0b75f0 msgr2=0x7fb60c151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60c0b75f0 0x7fb60c151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.235+0000 7fb60b200640 1 -- 172.21.15.19:0/1022605264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c151fc0 con 0x7fb60c1605c0 2024-04-25T11:30:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.236+0000 7fb60b200640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c15bce0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb60400cc30 tx=0x7fb60400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.236+0000 7fb602a00640 1 -- 172.21.15.19:0/1022605264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60400ce40 con 0x7fb60c1605c0 2024-04-25T11:30:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.236+0000 7fb602a00640 1 -- 172.21.15.19:0/1022605264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb604003990 con 0x7fb60c1605c0 2024-04-25T11:30:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.236+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c0d06d0 con 0x7fb60c1605c0 2024-04-25T11:30:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.236+0000 7fb602a00640 1 -- 172.21.15.19:0/1022605264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604003d40 con 0x7fb60c1605c0 2024-04-25T11:30:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.236+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c0d0ee0 con 0x7fb60c1605c0 2024-04-25T11:30:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.237+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c0d0950 con 0x7fb60c1605c0 2024-04-25T11:30:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.238+0000 7fb602a00640 1 -- 172.21.15.19:0/1022605264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb604003ee0 con 0x7fb60c1605c0 2024-04-25T11:30:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.239+0000 7fb602a00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5ec0744e0 0x7fb5ec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.240+0000 7fb602a00640 1 -- 172.21.15.19:0/1022605264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fb604095670 con 0x7fb60c1605c0 2024-04-25T11:30:16.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.240+0000 7fb60a800640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5ec0744e0 0x7fb5ec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.242+0000 7fb60a800640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5ec0744e0 0x7fb5ec0769f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb60c072680 tx=0x7fb5fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.245+0000 7fb602a00640 1 -- 172.21.15.19:0/1022605264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb604014030 con 0x7fb60c1605c0 2024-04-25T11:30:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.456+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb60c050740 con 0x7fb60c1605c0 2024-04-25T11:30:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.458+0000 7fb602a00640 1 -- 172.21.15.19:0/1022605264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16609 (secure 0 0 0) 0x7fb60405fdf0 con 0x7fb60c1605c0 2024-04-25T11:30:16.449 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:16.449 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:13.289711+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:30:10.008986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T11:30:10.799803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:16.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5ec0744e0 msgr2=0x7fb5ec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5ec0744e0 0x7fb5ec0769f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb60c072680 tx=0x7fb5fc008040 comp rx=0 tx=0).stop 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 msgr2=0x7fb60c15bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c15bce0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb60400cc30 tx=0x7fb60400a5c0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 shutdown_connections 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5ec0744e0 0x7fb5ec0769f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb60c1605c0 0x7fb60c15bce0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb60c0cf270 0x7fb60c0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 --2- 172.21.15.19:0/1022605264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb60c0b75f0 0x7fb60c151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.461+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 >> 172.21.15.19:0/1022605264 conn(0x7fb60c09b7d0 msgr2=0x7fb60c0afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.462+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 shutdown_connections 2024-04-25T11:30:16.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.462+0000 7fb610a00640 1 -- 172.21.15.19:0/1022605264 wait complete. 2024-04-25T11:30:16.479 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:30:16.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:30:16.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.778+0000 7f79ca400640 1 Processor -- start 2024-04-25T11:30:16.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.778+0000 7f79ca400640 1 -- start start 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79ca400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79c40d50e0 0x7f79c40d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79ca400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d2ae0 0x7f79c4160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79ca400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c40b7940 con 0x7f79c4160b90 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79ca400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f79c40d2ef0 con 0x7f79c40d50e0 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79ca400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79c4161280 con 0x7f79c40d2ae0 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79c9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79c40d50e0 0x7f79c40d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79c9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79c40d50e0 0x7f79c40d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34214/0 (socket says 172.21.15.19:34214) 2024-04-25T11:30:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.780+0000 7f79c9000640 1 -- 172.21.15.19:0/4139947476 learned_addr learned my addr 172.21.15.19:0/4139947476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:16.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.781+0000 7f79c3e00640 1 --2- 172.21.15.19:0/4139947476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d2ae0 0x7f79c4160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.781+0000 7f79c9000640 1 -- 172.21.15.19:0/4139947476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d2ae0 msgr2=0x7f79c4160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:16.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.781+0000 7f79c9000640 1 --2- 172.21.15.19:0/4139947476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d2ae0 0x7f79c4160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.781+0000 7f79c9000640 1 -- 172.21.15.19:0/4139947476 >> v1:172.21.15.169:6789/0 conn(0x7f79c4160b90 legacy=0x7f79c4161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:16.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.781+0000 7f79c9000640 1 -- 172.21.15.19:0/4139947476 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c40d4600 con 0x7f79c40d50e0 2024-04-25T11:30:16.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.782+0000 7f79c9000640 1 --2- 172.21.15.19:0/4139947476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c40d50e0 0x7f79c40d25a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009870 tx=0x7f79b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f60b0dc2c865af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.783+0000 7f79c3400640 1 -- 172.21.15.19:0/4139947476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8037070 con 0x7f79c40d50e0 2024-04-25T11:30:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.783+0000 7f79c3400640 1 -- 172.21.15.19:0/4139947476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b8032440 con 0x7f79c40d50e0 2024-04-25T11:30:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.783+0000 7f79c3400640 1 -- 172.21.15.19:0/4139947476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8032810 con 0x7f79c40d50e0 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 -- 172.21.15.19:0/4139947476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c40d50e0 msgr2=0x7f79c40d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 --2- 172.21.15.19:0/4139947476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c40d50e0 0x7f79c40d25a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009870 tx=0x7f79b802a210 comp rx=0 tx=0).stop 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 -- 172.21.15.19:0/4139947476 shutdown_connections 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 --2- 172.21.15.19:0/4139947476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d2ae0 0x7f79c4160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 --2- 172.21.15.19:0/4139947476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c40d50e0 0x7f79c40d25a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 -- 172.21.15.19:0/4139947476 >> 172.21.15.19:0/4139947476 conn(0x7f79c406c5c0 msgr2=0x7f79c406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 -- 172.21.15.19:0/4139947476 shutdown_connections 2024-04-25T11:30:16.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.784+0000 7f79ca400640 1 -- 172.21.15.19:0/4139947476 wait complete. 2024-04-25T11:30:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.786+0000 7f79ca400640 1 Processor -- start 2024-04-25T11:30:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.786+0000 7f79ca400640 1 -- start start 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79ca400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d2ae0 0x7f79c4150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79ca400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d50e0 0x7f79c41e9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79ca400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4160b90 0x7f79c4156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79ca400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4160fa0 con 0x7f79c4160b90 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79ca400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79c40d4bc0 con 0x7f79c40d2ae0 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79ca400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79c4161560 con 0x7f79c40d50e0 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79c9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4160b90 0x7f79c4156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79c9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4160b90 0x7f79c4156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34222/0 (socket says 172.21.15.19:34222) 2024-04-25T11:30:16.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.788+0000 7f79c9a00640 1 -- 172.21.15.19:0/33107041 learned_addr learned my addr 172.21.15.19:0/33107041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c3e00640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d50e0 0x7f79c41e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c9000640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d2ae0 0x7f79c4150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c9a00640 1 -- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d50e0 msgr2=0x7f79c41e9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c9a00640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d50e0 0x7f79c41e9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c9a00640 1 -- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d2ae0 msgr2=0x7f79c4150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c9a00640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d2ae0 0x7f79c4150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c9a00640 1 -- 172.21.15.19:0/33107041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c4157940 con 0x7f79c4160b90 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c3e00640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d50e0 0x7f79c41e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c9a00640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4160b90 0x7f79c4156020 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00b970 tx=0x7f79bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:16.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c0c00640 1 -- 172.21.15.19:0/33107041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc019070 con 0x7f79c4160b90 2024-04-25T11:30:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c0c00640 1 -- 172.21.15.19:0/33107041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79bc00cb40 con 0x7f79c4160b90 2024-04-25T11:30:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79c0c00640 1 -- 172.21.15.19:0/33107041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc014440 con 0x7f79c4160b90 2024-04-25T11:30:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.789+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c4157c20 con 0x7f79c4160b90 2024-04-25T11:30:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.790+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c41581e0 con 0x7f79c4160b90 2024-04-25T11:30:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.791+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c4050740 con 0x7f79c4160b90 2024-04-25T11:30:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.792+0000 7f79c0c00640 1 -- 172.21.15.19:0/33107041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79bc01c070 con 0x7f79c4160b90 2024-04-25T11:30:16.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.793+0000 7f79c0c00640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79a80744b0 0x7f79a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:16.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.793+0000 7f79c0c00640 1 -- 172.21.15.19:0/33107041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f79bc094ee0 con 0x7f79c4160b90 2024-04-25T11:30:16.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.794+0000 7f79c9000640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79a80744b0 0x7f79a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:16.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.795+0000 7f79c9000640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79a80744b0 0x7f79a80769c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79b8034000 tx=0x7f79b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:16.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.798+0000 7f79c0c00640 1 -- 172.21.15.19:0/33107041 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79bc05f660 con 0x7f79c4160b90 2024-04-25T11:30:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:16.998+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79c40b8aa0 con 0x7f79a80744b0 2024-04-25T11:30:17.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.021+0000 7f79c0c00640 1 -- 172.21.15.19:0/33107041 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250748 (secure 0 0 0) 0x7f79c40b8aa0 con 0x7f79a80744b0 2024-04-25T11:30:17.014 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:17.016 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:30:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79a80744b0 msgr2=0x7f79a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79a80744b0 0x7f79a80769c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79b8034000 tx=0x7f79b802e000 comp rx=0 tx=0).stop 2024-04-25T11:30:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4160b90 msgr2=0x7f79c4156020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4160b90 0x7f79c4156020 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00b970 tx=0x7f79bc00be30 comp rx=0 tx=0).stop 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 shutdown_connections 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79a80744b0 0x7f79a80769c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4160b90 0x7f79c4156020 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79c40d50e0 0x7f79c41e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 --2- 172.21.15.19:0/33107041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79c40d2ae0 0x7f79c4150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 >> 172.21.15.19:0/33107041 conn(0x7f79c406c5c0 msgr2=0x7f79c415f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 shutdown_connections 2024-04-25T11:30:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.026+0000 7f79ca400640 1 -- 172.21.15.19:0/33107041 wait complete. 2024-04-25T11:30:17.045 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-04-25T11:30:16.505179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230032,"kb_used_data":5528,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744688,"statfs":{"total":773094113280,"available":772858560512,"internally_reserved":0,"allocated":5660672,"data_stored":2998197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":229756466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002160"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.333693+0000","last_change":"2024-04-25T11:30:12.328001+0000","last_active":"2024-04-25T11:30:12.333693+0000","last_peered":"2024-04-25T11:30:12.333693+0000","last_clean":"2024-04-25T11:30:12.333693+0000","last_became_active":"2024-04-25T11:30:12.327555+0000","last_became_peered":"2024-04-25T11:30:12.327555+0000","last_unstale":"2024-04-25T11:30:12.333693+0000","last_undegraded":"2024-04-25T11:30:12.333693+0000","last_fullsized":"2024-04-25T11:30:12.333693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:03.855581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325206+0000","last_change":"2024-04-25T11:30:12.316817+0000","last_active":"2024-04-25T11:30:12.325206+0000","last_peered":"2024-04-25T11:30:12.325206+0000","last_clean":"2024-04-25T11:30:12.325206+0000","last_became_active":"2024-04-25T11:30:12.311768+0000","last_became_peered":"2024-04-25T11:30:12.311768+0000","last_unstale":"2024-04-25T11:30:12.325206+0000","last_undegraded":"2024-04-25T11:30:12.325206+0000","last_fullsized":"2024-04-25T11:30:12.325206+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.319448+0000","last_change":"2024-04-25T11:30:12.315596+0000","last_active":"2024-04-25T11:30:12.319448+0000","last_peered":"2024-04-25T11:30:12.319448+0000","last_clean":"2024-04-25T11:30:12.319448+0000","last_became_active":"2024-04-25T11:30:12.315259+0000","last_became_peered":"2024-04-25T11:30:12.315259+0000","last_unstale":"2024-04-25T11:30:12.319448+0000","last_undegraded":"2024-04-25T11:30:12.319448+0000","last_fullsized":"2024-04-25T11:30:12.319448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:18.773842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.256254+0000","last_change":"2024-04-25T11:30:11.256254+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.256254+0000","last_undegraded":"2024-04-25T11:30:11.256254+0000","last_fullsized":"2024-04-25T11:30:11.256254+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327312+0000","last_change":"2024-04-25T11:30:12.320809+0000","last_active":"2024-04-25T11:30:12.327312+0000","last_peered":"2024-04-25T11:30:12.327312+0000","last_clean":"2024-04-25T11:30:12.327312+0000","last_became_active":"2024-04-25T11:30:12.318568+0000","last_became_peered":"2024-04-25T11:30:12.318568+0000","last_unstale":"2024-04-25T11:30:12.327312+0000","last_undegraded":"2024-04-25T11:30:12.327312+0000","last_fullsized":"2024-04-25T11:30:12.327312+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324645+0000","last_change":"2024-04-25T11:30:12.316853+0000","last_active":"2024-04-25T11:30:12.324645+0000","last_peered":"2024-04-25T11:30:12.324645+0000","last_clean":"2024-04-25T11:30:12.324645+0000","last_became_active":"2024-04-25T11:30:12.316509+0000","last_became_peered":"2024-04-25T11:30:12.316509+0000","last_unstale":"2024-04-25T11:30:12.324645+0000","last_undegraded":"2024-04-25T11:30:12.324645+0000","last_fullsized":"2024-04-25T11:30:12.324645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.328160+0000","last_change":"2024-04-25T11:30:12.323818+0000","last_active":"2024-04-25T11:30:12.328160+0000","last_peered":"2024-04-25T11:30:12.328160+0000","last_clean":"2024-04-25T11:30:12.328160+0000","last_became_active":"2024-04-25T11:30:12.322940+0000","last_became_peered":"2024-04-25T11:30:12.322940+0000","last_unstale":"2024-04-25T11:30:12.328160+0000","last_undegraded":"2024-04-25T11:30:12.328160+0000","last_fullsized":"2024-04-25T11:30:12.328160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:40.175748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254522+0000","last_change":"2024-04-25T11:30:11.254522+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254522+0000","last_undegraded":"2024-04-25T11:30:11.254522+0000","last_fullsized":"2024-04-25T11:30:11.254522+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.323941+0000","last_change":"2024-04-25T11:30:12.316458+0000","last_active":"2024-04-25T11:30:12.323941+0000","last_peered":"2024-04-25T11:30:12.323941+0000","last_clean":"2024-04-25T11:30:12.323941+0000","last_became_active":"2024-04-25T11:30:12.315188+0000","last_became_peered":"2024-04-25T11:30:12.315188+0000","last_unstale":"2024-04-25T11:30:12.323941+0000","last_undegraded":"2024-04-25T11:30:12.323941+0000","last_fullsized":"2024-04-25T11:30:12.323941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:07.988273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.315531+0000","last_change":"2024-04-25T11:30:12.311133+0000","last_active":"2024-04-25T11:30:12.315531+0000","last_peered":"2024-04-25T11:30:12.315531+0000","last_clean":"2024-04-25T11:30:12.315531+0000","last_became_active":"2024-04-25T11:30:12.310670+0000","last_became_peered":"2024-04-25T11:30:12.310670+0000","last_unstale":"2024-04-25T11:30:12.315531+0000","last_undegraded":"2024-04-25T11:30:12.315531+0000","last_fullsized":"2024-04-25T11:30:12.315531+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.265764+0000","last_change":"2024-04-25T11:30:11.265764+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.265764+0000","last_undegraded":"2024-04-25T11:30:11.265764+0000","last_fullsized":"2024-04-25T11:30:11.265764+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.318340+0000","last_change":"2024-04-25T11:30:12.306558+0000","last_active":"2024-04-25T11:30:12.318340+0000","last_peered":"2024-04-25T11:30:12.318340+0000","last_clean":"2024-04-25T11:30:12.318340+0000","last_became_active":"2024-04-25T11:30:12.306159+0000","last_became_peered":"2024-04-25T11:30:12.306159+0000","last_unstale":"2024-04-25T11:30:12.318340+0000","last_undegraded":"2024-04-25T11:30:12.318340+0000","last_fullsized":"2024-04-25T11:30:12.318340+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.323321+0000","last_change":"2024-04-25T11:30:12.315317+0000","last_active":"2024-04-25T11:30:12.323321+0000","last_peered":"2024-04-25T11:30:12.323321+0000","last_clean":"2024-04-25T11:30:12.323321+0000","last_became_active":"2024-04-25T11:30:12.314919+0000","last_became_peered":"2024-04-25T11:30:12.314919+0000","last_unstale":"2024-04-25T11:30:12.323321+0000","last_undegraded":"2024-04-25T11:30:12.323321+0000","last_fullsized":"2024-04-25T11:30:12.323321+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:32.390065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.253292+0000","last_change":"2024-04-25T11:30:11.253292+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.253292+0000","last_undegraded":"2024-04-25T11:30:11.253292+0000","last_fullsized":"2024-04-25T11:30:11.253292+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.252332+0000","last_change":"2024-04-25T11:30:11.252332+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.252332+0000","last_undegraded":"2024-04-25T11:30:11.252332+0000","last_fullsized":"2024-04-25T11:30:11.252332+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327221+0000","last_change":"2024-04-25T11:30:12.320656+0000","last_active":"2024-04-25T11:30:12.327221+0000","last_peered":"2024-04-25T11:30:12.327221+0000","last_clean":"2024-04-25T11:30:12.327221+0000","last_became_active":"2024-04-25T11:30:12.319501+0000","last_became_peered":"2024-04-25T11:30:12.319501+0000","last_unstale":"2024-04-25T11:30:12.327221+0000","last_undegraded":"2024-04-25T11:30:12.327221+0000","last_fullsized":"2024-04-25T11:30:12.327221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324268+0000","last_change":"2024-04-25T11:30:12.312204+0000","last_active":"2024-04-25T11:30:12.324268+0000","last_peered":"2024-04-25T11:30:12.324268+0000","last_clean":"2024-04-25T11:30:12.324268+0000","last_became_active":"2024-04-25T11:30:12.311798+0000","last_became_peered":"2024-04-25T11:30:12.311798+0000","last_unstale":"2024-04-25T11:30:12.324268+0000","last_undegraded":"2024-04-25T11:30:12.324268+0000","last_fullsized":"2024-04-25T11:30:12.324268+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.296840+0000","last_change":"2024-04-25T11:30:12.292655+0000","last_active":"2024-04-25T11:30:12.296840+0000","last_peered":"2024-04-25T11:30:12.296840+0000","last_clean":"2024-04-25T11:30:12.296840+0000","last_became_active":"2024-04-25T11:30:12.292173+0000","last_became_peered":"2024-04-25T11:30:12.292173+0000","last_unstale":"2024-04-25T11:30:12.296840+0000","last_undegraded":"2024-04-25T11:30:12.296840+0000","last_fullsized":"2024-04-25T11:30:12.296840+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:40.609036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327149+0000","last_change":"2024-04-25T11:30:12.318490+0000","last_active":"2024-04-25T11:30:12.327149+0000","last_peered":"2024-04-25T11:30:12.327149+0000","last_clean":"2024-04-25T11:30:12.327149+0000","last_became_active":"2024-04-25T11:30:12.317442+0000","last_became_peered":"2024-04-25T11:30:12.317442+0000","last_unstale":"2024-04-25T11:30:12.327149+0000","last_undegraded":"2024-04-25T11:30:12.327149+0000","last_fullsized":"2024-04-25T11:30:12.327149+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.315731+0000","last_change":"2024-04-25T11:30:12.311311+0000","last_active":"2024-04-25T11:30:12.315731+0000","last_peered":"2024-04-25T11:30:12.315731+0000","last_clean":"2024-04-25T11:30:12.315731+0000","last_became_active":"2024-04-25T11:30:12.310846+0000","last_became_peered":"2024-04-25T11:30:12.310846+0000","last_unstale":"2024-04-25T11:30:12.315731+0000","last_undegraded":"2024-04-25T11:30:12.315731+0000","last_fullsized":"2024-04-25T11:30:12.315731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.314933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254710+0000","last_change":"2024-04-25T11:30:11.254710+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254710+0000","last_undegraded":"2024-04-25T11:30:11.254710+0000","last_fullsized":"2024-04-25T11:30:11.254710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324559+0000","last_change":"2024-04-25T11:30:12.318465+0000","last_active":"2024-04-25T11:30:12.324559+0000","last_peered":"2024-04-25T11:30:12.324559+0000","last_clean":"2024-04-25T11:30:12.324559+0000","last_became_active":"2024-04-25T11:30:12.318076+0000","last_became_peered":"2024-04-25T11:30:12.318076+0000","last_unstale":"2024-04-25T11:30:12.324559+0000","last_undegraded":"2024-04-25T11:30:12.324559+0000","last_fullsized":"2024-04-25T11:30:12.324559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:21.763769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.313691+0000","last_change":"2024-04-25T11:30:12.308548+0000","last_active":"2024-04-25T11:30:12.313691+0000","last_peered":"2024-04-25T11:30:12.313691+0000","last_clean":"2024-04-25T11:30:12.313691+0000","last_became_active":"2024-04-25T11:30:12.308148+0000","last_became_peered":"2024-04-25T11:30:12.308148+0000","last_unstale":"2024-04-25T11:30:12.313691+0000","last_undegraded":"2024-04-25T11:30:12.313691+0000","last_fullsized":"2024-04-25T11:30:12.313691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:57.673678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324424+0000","last_change":"2024-04-25T11:30:12.312781+0000","last_active":"2024-04-25T11:30:12.324424+0000","last_peered":"2024-04-25T11:30:12.324424+0000","last_clean":"2024-04-25T11:30:12.324424+0000","last_became_active":"2024-04-25T11:30:12.312453+0000","last_became_peered":"2024-04-25T11:30:12.312453+0000","last_unstale":"2024-04-25T11:30:12.324424+0000","last_undegraded":"2024-04-25T11:30:12.324424+0000","last_fullsized":"2024-04-25T11:30:12.324424+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.328444+0000","last_change":"2024-04-25T11:30:12.325014+0000","last_active":"2024-04-25T11:30:12.328444+0000","last_peered":"2024-04-25T11:30:12.328444+0000","last_clean":"2024-04-25T11:30:12.328444+0000","last_became_active":"2024-04-25T11:30:12.324660+0000","last_became_peered":"2024-04-25T11:30:12.324660+0000","last_unstale":"2024-04-25T11:30:12.328444+0000","last_undegraded":"2024-04-25T11:30:12.328444+0000","last_fullsized":"2024-04-25T11:30:12.328444+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.265187+0000","last_change":"2024-04-25T11:30:11.265187+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.265187+0000","last_undegraded":"2024-04-25T11:30:11.265187+0000","last_fullsized":"2024-04-25T11:30:11.265187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304485+0000","last_change":"2024-04-25T11:30:12.300829+0000","last_active":"2024-04-25T11:30:12.304485+0000","last_peered":"2024-04-25T11:30:12.304485+0000","last_clean":"2024-04-25T11:30:12.304485+0000","last_became_active":"2024-04-25T11:30:12.300457+0000","last_became_peered":"2024-04-25T11:30:12.300457+0000","last_unstale":"2024-04-25T11:30:12.304485+0000","last_undegraded":"2024-04-25T11:30:12.304485+0000","last_fullsized":"2024-04-25T11:30:12.304485+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:29.722464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.309487+0000","last_change":"2024-04-25T11:30:12.306296+0000","last_active":"2024-04-25T11:30:12.309487+0000","last_peered":"2024-04-25T11:30:12.309487+0000","last_clean":"2024-04-25T11:30:12.309487+0000","last_became_active":"2024-04-25T11:30:12.305922+0000","last_became_peered":"2024-04-25T11:30:12.305922+0000","last_unstale":"2024-04-25T11:30:12.309487+0000","last_undegraded":"2024-04-25T11:30:12.309487+0000","last_fullsized":"2024-04-25T11:30:12.309487+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:33.370286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.320671+0000","last_change":"2024-04-25T11:30:12.314344+0000","last_active":"2024-04-25T11:30:12.320671+0000","last_peered":"2024-04-25T11:30:12.320671+0000","last_clean":"2024-04-25T11:30:12.320671+0000","last_became_active":"2024-04-25T11:30:12.314033+0000","last_became_peered":"2024-04-25T11:30:12.314033+0000","last_unstale":"2024-04-25T11:30:12.320671+0000","last_undegraded":"2024-04-25T11:30:12.320671+0000","last_fullsized":"2024-04-25T11:30:12.320671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:10.049040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.316322+0000","last_change":"2024-04-25T11:30:12.306312+0000","last_active":"2024-04-25T11:30:12.316322+0000","last_peered":"2024-04-25T11:30:12.316322+0000","last_clean":"2024-04-25T11:30:12.316322+0000","last_became_active":"2024-04-25T11:30:12.305928+0000","last_became_peered":"2024-04-25T11:30:12.305928+0000","last_unstale":"2024-04-25T11:30:12.316322+0000","last_undegraded":"2024-04-25T11:30:12.316322+0000","last_fullsized":"2024-04-25T11:30:12.316322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:07.988273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.322258+0000","last_change":"2024-04-25T11:30:12.309122+0000","last_active":"2024-04-25T11:30:12.322258+0000","last_peered":"2024-04-25T11:30:12.322258+0000","last_clean":"2024-04-25T11:30:12.322258+0000","last_became_active":"2024-04-25T11:30:12.308689+0000","last_became_peered":"2024-04-25T11:30:12.308689+0000","last_unstale":"2024-04-25T11:30:12.322258+0000","last_undegraded":"2024-04-25T11:30:12.322258+0000","last_fullsized":"2024-04-25T11:30:12.322258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:10.291351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.252849+0000","last_change":"2024-04-25T11:30:11.252849+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.252849+0000","last_undegraded":"2024-04-25T11:30:11.252849+0000","last_fullsized":"2024-04-25T11:30:11.252849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325542+0000","last_change":"2024-04-25T11:30:12.324170+0000","last_active":"2024-04-25T11:30:12.325542+0000","last_peered":"2024-04-25T11:30:12.325542+0000","last_clean":"2024-04-25T11:30:12.325542+0000","last_became_active":"2024-04-25T11:30:12.323509+0000","last_became_peered":"2024-04-25T11:30:12.323509+0000","last_unstale":"2024-04-25T11:30:12.325542+0000","last_undegraded":"2024-04-25T11:30:12.325542+0000","last_fullsized":"2024-04-25T11:30:12.325542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:01.898568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338824+0000","last_change":"2024-04-25T11:30:11.269060+0000","last_active":"2024-04-25T11:30:12.338824+0000","last_peered":"2024-04-25T11:30:12.338824+0000","last_clean":"2024-04-25T11:30:12.338824+0000","last_became_active":"2024-04-25T11:30:11.266286+0000","last_became_peered":"2024-04-25T11:30:11.266286+0000","last_unstale":"2024-04-25T11:30:12.338824+0000","last_undegraded":"2024-04-25T11:30:12.338824+0000","last_fullsized":"2024-04-25T11:30:12.338824+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:25.474525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327322+0000","last_change":"2024-04-25T11:30:12.319935+0000","last_active":"2024-04-25T11:30:12.327322+0000","last_peered":"2024-04-25T11:30:12.327322+0000","last_clean":"2024-04-25T11:30:12.327322+0000","last_became_active":"2024-04-25T11:30:12.319486+0000","last_became_peered":"2024-04-25T11:30:12.319486+0000","last_unstale":"2024-04-25T11:30:12.327322+0000","last_undegraded":"2024-04-25T11:30:12.327322+0000","last_fullsized":"2024-04-25T11:30:12.327322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.339577+0000","last_change":"2024-04-25T11:30:11.257450+0000","last_active":"2024-04-25T11:30:12.339577+0000","last_peered":"2024-04-25T11:30:12.339577+0000","last_clean":"2024-04-25T11:30:12.339577+0000","last_became_active":"2024-04-25T11:30:11.257097+0000","last_became_peered":"2024-04-25T11:30:11.257097+0000","last_unstale":"2024-04-25T11:30:12.339577+0000","last_undegraded":"2024-04-25T11:30:12.339577+0000","last_fullsized":"2024-04-25T11:30:12.339577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.501573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325736+0000","last_change":"2024-04-25T11:30:12.318193+0000","last_active":"2024-04-25T11:30:12.325736+0000","last_peered":"2024-04-25T11:30:12.325736+0000","last_clean":"2024-04-25T11:30:12.325736+0000","last_became_active":"2024-04-25T11:30:12.317611+0000","last_became_peered":"2024-04-25T11:30:12.317611+0000","last_unstale":"2024-04-25T11:30:12.325736+0000","last_undegraded":"2024-04-25T11:30:12.325736+0000","last_fullsized":"2024-04-25T11:30:12.325736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.334754+0000","last_change":"2024-04-25T11:30:11.271679+0000","last_active":"2024-04-25T11:30:12.334754+0000","last_peered":"2024-04-25T11:30:12.334754+0000","last_clean":"2024-04-25T11:30:12.334754+0000","last_became_active":"2024-04-25T11:30:11.270812+0000","last_became_peered":"2024-04-25T11:30:11.270812+0000","last_unstale":"2024-04-25T11:30:12.334754+0000","last_undegraded":"2024-04-25T11:30:12.334754+0000","last_fullsized":"2024-04-25T11:30:12.334754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:01.243237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.261566+0000","last_change":"2024-04-25T11:30:11.261566+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.261566+0000","last_undegraded":"2024-04-25T11:30:11.261566+0000","last_fullsized":"2024-04-25T11:30:11.261566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.339891+0000","last_change":"2024-04-25T11:30:11.259259+0000","last_active":"2024-04-25T11:30:12.339891+0000","last_peered":"2024-04-25T11:30:12.339891+0000","last_clean":"2024-04-25T11:30:12.339891+0000","last_became_active":"2024-04-25T11:30:11.258919+0000","last_became_peered":"2024-04-25T11:30:11.258919+0000","last_unstale":"2024-04-25T11:30:12.339891+0000","last_undegraded":"2024-04-25T11:30:12.339891+0000","last_fullsized":"2024-04-25T11:30:12.339891+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:49.077411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.320288+0000","last_change":"2024-04-25T11:30:12.310915+0000","last_active":"2024-04-25T11:30:12.320288+0000","last_peered":"2024-04-25T11:30:12.320288+0000","last_clean":"2024-04-25T11:30:12.320288+0000","last_became_active":"2024-04-25T11:30:12.310580+0000","last_became_peered":"2024-04-25T11:30:12.310580+0000","last_unstale":"2024-04-25T11:30:12.320288+0000","last_undegraded":"2024-04-25T11:30:12.320288+0000","last_fullsized":"2024-04-25T11:30:12.320288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:26.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338450+0000","last_change":"2024-04-25T11:30:11.278985+0000","last_active":"2024-04-25T11:30:12.338450+0000","last_peered":"2024-04-25T11:30:12.338450+0000","last_clean":"2024-04-25T11:30:12.338450+0000","last_became_active":"2024-04-25T11:30:11.273156+0000","last_became_peered":"2024-04-25T11:30:11.273156+0000","last_unstale":"2024-04-25T11:30:12.338450+0000","last_undegraded":"2024-04-25T11:30:12.338450+0000","last_fullsized":"2024-04-25T11:30:12.338450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:37.174642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325053+0000","last_change":"2024-04-25T11:30:12.318436+0000","last_active":"2024-04-25T11:30:12.325053+0000","last_peered":"2024-04-25T11:30:12.325053+0000","last_clean":"2024-04-25T11:30:12.325053+0000","last_became_active":"2024-04-25T11:30:12.318108+0000","last_became_peered":"2024-04-25T11:30:12.318108+0000","last_unstale":"2024-04-25T11:30:12.325053+0000","last_undegraded":"2024-04-25T11:30:12.325053+0000","last_fullsized":"2024-04-25T11:30:12.325053+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338843+0000","last_change":"2024-04-25T11:30:11.263150+0000","last_active":"2024-04-25T11:30:12.338843+0000","last_peered":"2024-04-25T11:30:12.338843+0000","last_clean":"2024-04-25T11:30:12.338843+0000","last_became_active":"2024-04-25T11:30:11.262389+0000","last_became_peered":"2024-04-25T11:30:11.262389+0000","last_unstale":"2024-04-25T11:30:12.338843+0000","last_undegraded":"2024-04-25T11:30:12.338843+0000","last_fullsized":"2024-04-25T11:30:12.338843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:44.476276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321591+0000","last_change":"2024-04-25T11:30:12.313164+0000","last_active":"2024-04-25T11:30:12.321591+0000","last_peered":"2024-04-25T11:30:12.321591+0000","last_clean":"2024-04-25T11:30:12.321591+0000","last_became_active":"2024-04-25T11:30:12.311004+0000","last_became_peered":"2024-04-25T11:30:12.311004+0000","last_unstale":"2024-04-25T11:30:12.321591+0000","last_undegraded":"2024-04-25T11:30:12.321591+0000","last_fullsized":"2024-04-25T11:30:12.321591+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:32.067456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.303499+0000","last_change":"2024-04-25T11:30:11.272497+0000","last_active":"2024-04-25T11:30:12.303499+0000","last_peered":"2024-04-25T11:30:12.303499+0000","last_clean":"2024-04-25T11:30:12.303499+0000","last_became_active":"2024-04-25T11:30:11.272150+0000","last_became_peered":"2024-04-25T11:30:11.272150+0000","last_unstale":"2024-04-25T11:30:12.303499+0000","last_undegraded":"2024-04-25T11:30:12.303499+0000","last_fullsized":"2024-04-25T11:30:12.303499+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:09.285233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.329154+0000","last_change":"2024-04-25T11:30:12.326351+0000","last_active":"2024-04-25T11:30:12.329154+0000","last_peered":"2024-04-25T11:30:12.329154+0000","last_clean":"2024-04-25T11:30:12.329154+0000","last_became_active":"2024-04-25T11:30:12.326055+0000","last_became_peered":"2024-04-25T11:30:12.326055+0000","last_unstale":"2024-04-25T11:30:12.329154+0000","last_undegraded":"2024-04-25T11:30:12.329154+0000","last_fullsized":"2024-04-25T11:30:12.329154+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:26.804870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304084+0000","last_change":"2024-04-25T11:30:11.280682+0000","last_active":"2024-04-25T11:30:12.304084+0000","last_peered":"2024-04-25T11:30:12.304084+0000","last_clean":"2024-04-25T11:30:12.304084+0000","last_became_active":"2024-04-25T11:30:11.279049+0000","last_became_peered":"2024-04-25T11:30:11.279049+0000","last_unstale":"2024-04-25T11:30:12.304084+0000","last_undegraded":"2024-04-25T11:30:12.304084+0000","last_fullsized":"2024-04-25T11:30:12.304084+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:40.524169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.316369+0000","last_change":"2024-04-25T11:30:12.307371+0000","last_active":"2024-04-25T11:30:12.316369+0000","last_peered":"2024-04-25T11:30:12.316369+0000","last_clean":"2024-04-25T11:30:12.316369+0000","last_became_active":"2024-04-25T11:30:12.307030+0000","last_became_peered":"2024-04-25T11:30:12.307030+0000","last_unstale":"2024-04-25T11:30:12.316369+0000","last_undegraded":"2024-04-25T11:30:12.316369+0000","last_fullsized":"2024-04-25T11:30:12.316369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.144352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.281829+0000","last_change":"2024-04-25T11:30:11.267439+0000","last_active":"2024-04-25T11:30:12.281829+0000","last_peered":"2024-04-25T11:30:12.281829+0000","last_clean":"2024-04-25T11:30:12.281829+0000","last_became_active":"2024-04-25T11:30:11.267110+0000","last_became_peered":"2024-04-25T11:30:11.267110+0000","last_unstale":"2024-04-25T11:30:12.281829+0000","last_undegraded":"2024-04-25T11:30:12.281829+0000","last_fullsized":"2024-04-25T11:30:12.281829+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:30.677292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.318253+0000","last_change":"2024-04-25T11:30:12.311247+0000","last_active":"2024-04-25T11:30:12.318253+0000","last_peered":"2024-04-25T11:30:12.318253+0000","last_clean":"2024-04-25T11:30:12.318253+0000","last_became_active":"2024-04-25T11:30:12.310904+0000","last_became_peered":"2024-04-25T11:30:12.310904+0000","last_unstale":"2024-04-25T11:30:12.318253+0000","last_undegraded":"2024-04-25T11:30:12.318253+0000","last_fullsized":"2024-04-25T11:30:12.318253+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:23.098668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304732+0000","last_change":"2024-04-25T11:30:11.272789+0000","last_active":"2024-04-25T11:30:12.304732+0000","last_peered":"2024-04-25T11:30:12.304732+0000","last_clean":"2024-04-25T11:30:12.304732+0000","last_became_active":"2024-04-25T11:30:11.272445+0000","last_became_peered":"2024-04-25T11:30:11.272445+0000","last_unstale":"2024-04-25T11:30:12.304732+0000","last_undegraded":"2024-04-25T11:30:12.304732+0000","last_fullsized":"2024-04-25T11:30:12.304732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.849463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254783+0000","last_change":"2024-04-25T11:30:11.254783+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254783+0000","last_undegraded":"2024-04-25T11:30:11.254783+0000","last_fullsized":"2024-04-25T11:30:11.254783+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338671+0000","last_change":"2024-04-25T11:30:11.264837+0000","last_active":"2024-04-25T11:30:12.338671+0000","last_peered":"2024-04-25T11:30:12.338671+0000","last_clean":"2024-04-25T11:30:12.338671+0000","last_became_active":"2024-04-25T11:30:11.264166+0000","last_became_peered":"2024-04-25T11:30:11.264166+0000","last_unstale":"2024-04-25T11:30:12.338671+0000","last_undegraded":"2024-04-25T11:30:12.338671+0000","last_fullsized":"2024-04-25T11:30:12.338671+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327792+0000","last_change":"2024-04-25T11:30:12.315841+0000","last_active":"2024-04-25T11:30:12.327792+0000","last_peered":"2024-04-25T11:30:12.327792+0000","last_clean":"2024-04-25T11:30:12.327792+0000","last_became_active":"2024-04-25T11:30:12.315087+0000","last_became_peered":"2024-04-25T11:30:12.315087+0000","last_unstale":"2024-04-25T11:30:12.327792+0000","last_undegraded":"2024-04-25T11:30:12.327792+0000","last_fullsized":"2024-04-25T11:30:12.327792+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:30.242892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.341752+0000","last_change":"2024-04-25T11:30:11.273982+0000","last_active":"2024-04-25T11:30:12.341752+0000","last_peered":"2024-04-25T11:30:12.341752+0000","last_clean":"2024-04-25T11:30:12.341752+0000","last_became_active":"2024-04-25T11:30:11.272233+0000","last_became_peered":"2024-04-25T11:30:11.272233+0000","last_unstale":"2024-04-25T11:30:12.341752+0000","last_undegraded":"2024-04-25T11:30:12.341752+0000","last_fullsized":"2024-04-25T11:30:12.341752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:34.401573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.296892+0000","last_change":"2024-04-25T11:30:12.294967+0000","last_active":"2024-04-25T11:30:12.296892+0000","last_peered":"2024-04-25T11:30:12.296892+0000","last_clean":"2024-04-25T11:30:12.296892+0000","last_became_active":"2024-04-25T11:30:12.294545+0000","last_became_peered":"2024-04-25T11:30:12.294545+0000","last_unstale":"2024-04-25T11:30:12.296892+0000","last_undegraded":"2024-04-25T11:30:12.296892+0000","last_fullsized":"2024-04-25T11:30:12.296892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:17.394404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.334053+0000","last_change":"2024-04-25T11:30:11.278510+0000","last_active":"2024-04-25T11:30:12.334053+0000","last_peered":"2024-04-25T11:30:12.334053+0000","last_clean":"2024-04-25T11:30:12.334053+0000","last_became_active":"2024-04-25T11:30:11.278136+0000","last_became_peered":"2024-04-25T11:30:11.278136+0000","last_unstale":"2024-04-25T11:30:12.334053+0000","last_undegraded":"2024-04-25T11:30:12.334053+0000","last_fullsized":"2024-04-25T11:30:12.334053+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:44.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327613+0000","last_change":"2024-04-25T11:30:12.326783+0000","last_active":"2024-04-25T11:30:12.327613+0000","last_peered":"2024-04-25T11:30:12.327613+0000","last_clean":"2024-04-25T11:30:12.327613+0000","last_became_active":"2024-04-25T11:30:12.325755+0000","last_became_peered":"2024-04-25T11:30:12.325755+0000","last_unstale":"2024-04-25T11:30:12.327613+0000","last_undegraded":"2024-04-25T11:30:12.327613+0000","last_fullsized":"2024-04-25T11:30:12.327613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:20.342509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304553+0000","last_change":"2024-04-25T11:30:11.269357+0000","last_active":"2024-04-25T11:30:12.304553+0000","last_peered":"2024-04-25T11:30:12.304553+0000","last_clean":"2024-04-25T11:30:12.304553+0000","last_became_active":"2024-04-25T11:30:11.268999+0000","last_became_peered":"2024-04-25T11:30:11.268999+0000","last_unstale":"2024-04-25T11:30:12.304553+0000","last_undegraded":"2024-04-25T11:30:12.304553+0000","last_fullsized":"2024-04-25T11:30:12.304553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321284+0000","last_change":"2024-04-25T11:30:12.316130+0000","last_active":"2024-04-25T11:30:12.321284+0000","last_peered":"2024-04-25T11:30:12.321284+0000","last_clean":"2024-04-25T11:30:12.321284+0000","last_became_active":"2024-04-25T11:30:12.315793+0000","last_became_peered":"2024-04-25T11:30:12.315793+0000","last_unstale":"2024-04-25T11:30:12.321284+0000","last_undegraded":"2024-04-25T11:30:12.321284+0000","last_fullsized":"2024-04-25T11:30:12.321284+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:23.098668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.300732+0000","last_change":"2024-04-25T11:30:11.279239+0000","last_active":"2024-04-25T11:30:12.300732+0000","last_peered":"2024-04-25T11:30:12.300732+0000","last_clean":"2024-04-25T11:30:12.300732+0000","last_became_active":"2024-04-25T11:30:11.278889+0000","last_became_peered":"2024-04-25T11:30:11.278889+0000","last_unstale":"2024-04-25T11:30:12.300732+0000","last_undegraded":"2024-04-25T11:30:12.300732+0000","last_fullsized":"2024-04-25T11:30:12.300732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:12.914763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324849+0000","last_change":"2024-04-25T11:30:12.306734+0000","last_active":"2024-04-25T11:30:12.324849+0000","last_peered":"2024-04-25T11:30:12.324849+0000","last_clean":"2024-04-25T11:30:12.324849+0000","last_became_active":"2024-04-25T11:30:12.306242+0000","last_became_peered":"2024-04-25T11:30:12.306242+0000","last_unstale":"2024-04-25T11:30:12.324849+0000","last_undegraded":"2024-04-25T11:30:12.324849+0000","last_fullsized":"2024-04-25T11:30:12.324849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:10.049040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.340720+0000","last_change":"2024-04-25T11:30:11.274461+0000","last_active":"2024-04-25T11:30:12.340720+0000","last_peered":"2024-04-25T11:30:12.340720+0000","last_clean":"2024-04-25T11:30:12.340720+0000","last_became_active":"2024-04-25T11:30:11.272811+0000","last_became_peered":"2024-04-25T11:30:11.272811+0000","last_unstale":"2024-04-25T11:30:12.340720+0000","last_undegraded":"2024-04-25T11:30:12.340720+0000","last_fullsized":"2024-04-25T11:30:12.340720+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:01.676380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325409+0000","last_change":"2024-04-25T11:30:12.321405+0000","last_active":"2024-04-25T11:30:12.325409+0000","last_peered":"2024-04-25T11:30:12.325409+0000","last_clean":"2024-04-25T11:30:12.325409+0000","last_became_active":"2024-04-25T11:30:12.319648+0000","last_became_peered":"2024-04-25T11:30:12.319648+0000","last_unstale":"2024-04-25T11:30:12.325409+0000","last_undegraded":"2024-04-25T11:30:12.325409+0000","last_fullsized":"2024-04-25T11:30:12.325409+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:18.773842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T11:30:11.283123+0000","last_change":"2024-04-25T11:30:11.281057+0000","last_active":"2024-04-25T11:30:11.283123+0000","last_peered":"2024-04-25T11:30:11.283123+0000","last_clean":"2024-04-25T11:30:11.283123+0000","last_became_active":"2024-04-25T11:30:11.279339+0000","last_became_peered":"2024-04-25T11:30:11.279339+0000","last_unstale":"2024-04-25T11:30:11.283123+0000","last_undegraded":"2024-04-25T11:30:11.283123+0000","last_fullsized":"2024-04-25T11:30:11.283123+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:18.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321209+0000","last_change":"2024-04-25T11:30:12.315868+0000","last_active":"2024-04-25T11:30:12.321209+0000","last_peered":"2024-04-25T11:30:12.321209+0000","last_clean":"2024-04-25T11:30:12.321209+0000","last_became_active":"2024-04-25T11:30:12.315511+0000","last_became_peered":"2024-04-25T11:30:12.315511+0000","last_unstale":"2024-04-25T11:30:12.321209+0000","last_undegraded":"2024-04-25T11:30:12.321209+0000","last_fullsized":"2024-04-25T11:30:12.321209+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.144352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327577+0000","last_change":"2024-04-25T11:30:11.271207+0000","last_active":"2024-04-25T11:30:12.327577+0000","last_peered":"2024-04-25T11:30:12.327577+0000","last_clean":"2024-04-25T11:30:12.327577+0000","last_became_active":"2024-04-25T11:30:11.270083+0000","last_became_peered":"2024-04-25T11:30:11.270083+0000","last_unstale":"2024-04-25T11:30:12.327577+0000","last_undegraded":"2024-04-25T11:30:12.327577+0000","last_fullsized":"2024-04-25T11:30:12.327577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:40.274271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325710+0000","last_change":"2024-04-25T11:30:12.317980+0000","last_active":"2024-04-25T11:30:12.325710+0000","last_peered":"2024-04-25T11:30:12.325710+0000","last_clean":"2024-04-25T11:30:12.325710+0000","last_became_active":"2024-04-25T11:30:12.317574+0000","last_became_peered":"2024-04-25T11:30:12.317574+0000","last_unstale":"2024-04-25T11:30:12.325710+0000","last_undegraded":"2024-04-25T11:30:12.325710+0000","last_fullsized":"2024-04-25T11:30:12.325710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327952+0000","last_change":"2024-04-25T11:30:11.276120+0000","last_active":"2024-04-25T11:30:12.327952+0000","last_peered":"2024-04-25T11:30:12.327952+0000","last_clean":"2024-04-25T11:30:12.327952+0000","last_became_active":"2024-04-25T11:30:11.275804+0000","last_became_peered":"2024-04-25T11:30:11.275804+0000","last_unstale":"2024-04-25T11:30:12.327952+0000","last_undegraded":"2024-04-25T11:30:12.327952+0000","last_fullsized":"2024-04-25T11:30:12.327952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:37.492766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.319419+0000","last_change":"2024-04-25T11:30:12.315454+0000","last_active":"2024-04-25T11:30:12.319419+0000","last_peered":"2024-04-25T11:30:12.319419+0000","last_clean":"2024-04-25T11:30:12.319419+0000","last_became_active":"2024-04-25T11:30:12.315114+0000","last_became_peered":"2024-04-25T11:30:12.315114+0000","last_unstale":"2024-04-25T11:30:12.319419+0000","last_undegraded":"2024-04-25T11:30:12.319419+0000","last_fullsized":"2024-04-25T11:30:12.319419+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:21.763769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304284+0000","last_change":"2024-04-25T11:30:11.279419+0000","last_active":"2024-04-25T11:30:12.304284+0000","last_peered":"2024-04-25T11:30:12.304284+0000","last_clean":"2024-04-25T11:30:12.304284+0000","last_became_active":"2024-04-25T11:30:11.279074+0000","last_became_peered":"2024-04-25T11:30:11.279074+0000","last_unstale":"2024-04-25T11:30:12.304284+0000","last_undegraded":"2024-04-25T11:30:12.304284+0000","last_fullsized":"2024-04-25T11:30:12.304284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:27.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.326110+0000","last_change":"2024-04-25T11:30:12.320762+0000","last_active":"2024-04-25T11:30:12.326110+0000","last_peered":"2024-04-25T11:30:12.326110+0000","last_clean":"2024-04-25T11:30:12.326110+0000","last_became_active":"2024-04-25T11:30:12.320207+0000","last_became_peered":"2024-04-25T11:30:12.320207+0000","last_unstale":"2024-04-25T11:30:12.326110+0000","last_undegraded":"2024-04-25T11:30:12.326110+0000","last_fullsized":"2024-04-25T11:30:12.326110+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:18.987772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.335531+0000","last_change":"2024-04-25T11:30:11.273562+0000","last_active":"2024-04-25T11:30:12.335531+0000","last_peered":"2024-04-25T11:30:12.335531+0000","last_clean":"2024-04-25T11:30:12.335531+0000","last_became_active":"2024-04-25T11:30:11.273187+0000","last_became_peered":"2024-04-25T11:30:11.273187+0000","last_unstale":"2024-04-25T11:30:12.335531+0000","last_undegraded":"2024-04-25T11:30:12.335531+0000","last_fullsized":"2024-04-25T11:30:12.335531+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:37.187391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.322121+0000","last_change":"2024-04-25T11:30:12.310068+0000","last_active":"2024-04-25T11:30:12.322121+0000","last_peered":"2024-04-25T11:30:12.322121+0000","last_clean":"2024-04-25T11:30:12.322121+0000","last_became_active":"2024-04-25T11:30:12.309236+0000","last_became_peered":"2024-04-25T11:30:12.309236+0000","last_unstale":"2024-04-25T11:30:12.322121+0000","last_undegraded":"2024-04-25T11:30:12.322121+0000","last_fullsized":"2024-04-25T11:30:12.322121+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.510835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305149+0000","last_change":"2024-04-25T11:30:11.260102+0000","last_active":"2024-04-25T11:30:12.305149+0000","last_peered":"2024-04-25T11:30:12.305149+0000","last_clean":"2024-04-25T11:30:12.305149+0000","last_became_active":"2024-04-25T11:30:11.259712+0000","last_became_peered":"2024-04-25T11:30:11.259712+0000","last_unstale":"2024-04-25T11:30:12.305149+0000","last_undegraded":"2024-04-25T11:30:12.305149+0000","last_fullsized":"2024-04-25T11:30:12.305149+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:31.383947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.251703+0000","last_change":"2024-04-25T11:30:11.251703+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.251703+0000","last_undegraded":"2024-04-25T11:30:11.251703+0000","last_fullsized":"2024-04-25T11:30:11.251703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304576+0000","last_change":"2024-04-25T11:30:11.274700+0000","last_active":"2024-04-25T11:30:12.304576+0000","last_peered":"2024-04-25T11:30:12.304576+0000","last_clean":"2024-04-25T11:30:12.304576+0000","last_became_active":"2024-04-25T11:30:11.274283+0000","last_became_peered":"2024-04-25T11:30:11.274283+0000","last_unstale":"2024-04-25T11:30:12.304576+0000","last_undegraded":"2024-04-25T11:30:12.304576+0000","last_fullsized":"2024-04-25T11:30:12.304576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.081180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.250705+0000","last_change":"2024-04-25T11:30:11.250705+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.250705+0000","last_undegraded":"2024-04-25T11:30:11.250705+0000","last_fullsized":"2024-04-25T11:30:11.250705+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.297476+0000","last_change":"2024-04-25T11:30:11.266163+0000","last_active":"2024-04-25T11:30:12.297476+0000","last_peered":"2024-04-25T11:30:12.297476+0000","last_clean":"2024-04-25T11:30:12.297476+0000","last_became_active":"2024-04-25T11:30:11.265810+0000","last_became_peered":"2024-04-25T11:30:11.265810+0000","last_unstale":"2024-04-25T11:30:12.297476+0000","last_undegraded":"2024-04-25T11:30:12.297476+0000","last_fullsized":"2024-04-25T11:30:12.297476+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:20.553784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.329132+0000","last_change":"2024-04-25T11:30:12.325830+0000","last_active":"2024-04-25T11:30:12.329132+0000","last_peered":"2024-04-25T11:30:12.329132+0000","last_clean":"2024-04-25T11:30:12.329132+0000","last_became_active":"2024-04-25T11:30:12.325485+0000","last_became_peered":"2024-04-25T11:30:12.325485+0000","last_unstale":"2024-04-25T11:30:12.329132+0000","last_undegraded":"2024-04-25T11:30:12.329132+0000","last_fullsized":"2024-04-25T11:30:12.329132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T11:30:11.276720+0000","last_change":"2024-04-25T11:30:11.274375+0000","last_active":"2024-04-25T11:30:11.276720+0000","last_peered":"2024-04-25T11:30:11.276720+0000","last_clean":"2024-04-25T11:30:11.276720+0000","last_became_active":"2024-04-25T11:30:11.273757+0000","last_became_peered":"2024-04-25T11:30:11.273757+0000","last_unstale":"2024-04-25T11:30:11.276720+0000","last_undegraded":"2024-04-25T11:30:11.276720+0000","last_fullsized":"2024-04-25T11:30:11.276720+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:43.406052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321025+0000","last_change":"2024-04-25T11:30:12.314257+0000","last_active":"2024-04-25T11:30:12.321025+0000","last_peered":"2024-04-25T11:30:12.321025+0000","last_clean":"2024-04-25T11:30:12.321025+0000","last_became_active":"2024-04-25T11:30:12.313948+0000","last_became_peered":"2024-04-25T11:30:12.313948+0000","last_unstale":"2024-04-25T11:30:12.321025+0000","last_undegraded":"2024-04-25T11:30:12.321025+0000","last_fullsized":"2024-04-25T11:30:12.321025+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305887+0000","last_change":"2024-04-25T11:30:11.263543+0000","last_active":"2024-04-25T11:30:12.305887+0000","last_peered":"2024-04-25T11:30:12.305887+0000","last_clean":"2024-04-25T11:30:12.305887+0000","last_became_active":"2024-04-25T11:30:11.263181+0000","last_became_peered":"2024-04-25T11:30:11.263181+0000","last_unstale":"2024-04-25T11:30:12.305887+0000","last_undegraded":"2024-04-25T11:30:12.305887+0000","last_fullsized":"2024-04-25T11:30:12.305887+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.090545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327820+0000","last_change":"2024-04-25T11:30:12.316106+0000","last_active":"2024-04-25T11:30:12.327820+0000","last_peered":"2024-04-25T11:30:12.327820+0000","last_clean":"2024-04-25T11:30:12.327820+0000","last_became_active":"2024-04-25T11:30:12.315513+0000","last_became_peered":"2024-04-25T11:30:12.315513+0000","last_unstale":"2024-04-25T11:30:12.327820+0000","last_undegraded":"2024-04-25T11:30:12.327820+0000","last_fullsized":"2024-04-25T11:30:12.327820+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:56.498661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.306699+0000","last_change":"2024-04-25T11:30:11.264133+0000","last_active":"2024-04-25T11:30:12.306699+0000","last_peered":"2024-04-25T11:30:12.306699+0000","last_clean":"2024-04-25T11:30:12.306699+0000","last_became_active":"2024-04-25T11:30:11.263693+0000","last_became_peered":"2024-04-25T11:30:11.263693+0000","last_unstale":"2024-04-25T11:30:12.306699+0000","last_undegraded":"2024-04-25T11:30:12.306699+0000","last_fullsized":"2024-04-25T11:30:12.306699+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.388286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.326385+0000","last_change":"2024-04-25T11:30:12.321099+0000","last_active":"2024-04-25T11:30:12.326385+0000","last_peered":"2024-04-25T11:30:12.326385+0000","last_clean":"2024-04-25T11:30:12.326385+0000","last_became_active":"2024-04-25T11:30:12.319720+0000","last_became_peered":"2024-04-25T11:30:12.319720+0000","last_unstale":"2024-04-25T11:30:12.326385+0000","last_undegraded":"2024-04-25T11:30:12.326385+0000","last_fullsized":"2024-04-25T11:30:12.326385+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:28.188588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305559+0000","last_change":"2024-04-25T11:30:11.278565+0000","last_active":"2024-04-25T11:30:12.305559+0000","last_peered":"2024-04-25T11:30:12.305559+0000","last_clean":"2024-04-25T11:30:12.305559+0000","last_became_active":"2024-04-25T11:30:11.278040+0000","last_became_peered":"2024-04-25T11:30:11.278040+0000","last_unstale":"2024-04-25T11:30:12.305559+0000","last_undegraded":"2024-04-25T11:30:12.305559+0000","last_fullsized":"2024-04-25T11:30:12.305559+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.982156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.255841+0000","last_change":"2024-04-25T11:30:11.255841+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.255841+0000","last_undegraded":"2024-04-25T11:30:11.255841+0000","last_fullsized":"2024-04-25T11:30:11.255841+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.298802+0000","last_change":"2024-04-25T11:30:11.272593+0000","last_active":"2024-04-25T11:30:12.298802+0000","last_peered":"2024-04-25T11:30:12.298802+0000","last_clean":"2024-04-25T11:30:12.298802+0000","last_became_active":"2024-04-25T11:30:11.272280+0000","last_became_peered":"2024-04-25T11:30:11.272280+0000","last_unstale":"2024-04-25T11:30:12.298802+0000","last_undegraded":"2024-04-25T11:30:12.298802+0000","last_fullsized":"2024-04-25T11:30:12.298802+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:29.649052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.302366+0000","last_change":"2024-04-25T11:30:12.296593+0000","last_active":"2024-04-25T11:30:12.302366+0000","last_peered":"2024-04-25T11:30:12.302366+0000","last_clean":"2024-04-25T11:30:12.302366+0000","last_became_active":"2024-04-25T11:30:12.296265+0000","last_became_peered":"2024-04-25T11:30:12.296265+0000","last_unstale":"2024-04-25T11:30:12.302366+0000","last_undegraded":"2024-04-25T11:30:12.302366+0000","last_fullsized":"2024-04-25T11:30:12.302366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:45.204846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304061+0000","last_change":"2024-04-25T11:30:11.271824+0000","last_active":"2024-04-25T11:30:12.304061+0000","last_peered":"2024-04-25T11:30:12.304061+0000","last_clean":"2024-04-25T11:30:12.304061+0000","last_became_active":"2024-04-25T11:30:11.270927+0000","last_became_peered":"2024-04-25T11:30:11.270927+0000","last_unstale":"2024-04-25T11:30:12.304061+0000","last_undegraded":"2024-04-25T11:30:12.304061+0000","last_fullsized":"2024-04-25T11:30:12.304061+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:19.313207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.303104+0000","last_change":"2024-04-25T11:30:12.298869+0000","last_active":"2024-04-25T11:30:12.303104+0000","last_peered":"2024-04-25T11:30:12.303104+0000","last_clean":"2024-04-25T11:30:12.303104+0000","last_became_active":"2024-04-25T11:30:12.298520+0000","last_became_peered":"2024-04-25T11:30:12.298520+0000","last_unstale":"2024-04-25T11:30:12.303104+0000","last_undegraded":"2024-04-25T11:30:12.303104+0000","last_fullsized":"2024-04-25T11:30:12.303104+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:32.390065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304469+0000","last_change":"2024-04-25T11:30:11.275287+0000","last_active":"2024-04-25T11:30:12.304469+0000","last_peered":"2024-04-25T11:30:12.304469+0000","last_clean":"2024-04-25T11:30:12.304469+0000","last_became_active":"2024-04-25T11:30:11.274574+0000","last_became_peered":"2024-04-25T11:30:11.274574+0000","last_unstale":"2024-04-25T11:30:12.304469+0000","last_undegraded":"2024-04-25T11:30:12.304469+0000","last_fullsized":"2024-04-25T11:30:12.304469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:39.602919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.264576+0000","last_change":"2024-04-25T11:30:11.264576+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.264576+0000","last_undegraded":"2024-04-25T11:30:11.264576+0000","last_fullsized":"2024-04-25T11:30:11.264576+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.335329+0000","last_change":"2024-04-25T11:30:11.272554+0000","last_active":"2024-04-25T11:30:12.335329+0000","last_peered":"2024-04-25T11:30:12.335329+0000","last_clean":"2024-04-25T11:30:12.335329+0000","last_became_active":"2024-04-25T11:30:11.272199+0000","last_became_peered":"2024-04-25T11:30:11.272199+0000","last_unstale":"2024-04-25T11:30:12.335329+0000","last_undegraded":"2024-04-25T11:30:12.335329+0000","last_fullsized":"2024-04-25T11:30:12.335329+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:32.204333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":161,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.276461+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:30:12.276461+0000","last_peered":"2024-04-25T11:30:12.276461+0000","last_clean":"2024-04-25T11:30:12.276461+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:30:12.276461+0000","last_undegraded":"2024-04-25T11:30:12.276461+0000","last_fullsized":"2024-04-25T11:30:12.276461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1167360,"data_stored":823540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1150976,"data_stored":816695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:30:17.050 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:30:17.203 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:30:17.213+0000 7fadcd800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:30:17.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.390+0000 7fb152000640 1 Processor -- start 2024-04-25T11:30:17.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.390+0000 7fb152000640 1 -- start start 2024-04-25T11:30:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb152000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c070e20 0x7fb14c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb152000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb14c06f180 0x7fb14c06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb152000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb14c06d390 0x7fb14c0cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb152000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb14c06d7a0 con 0x7fb14c06d390 2024-04-25T11:30:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb152000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb14c071230 con 0x7fb14c06f180 2024-04-25T11:30:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb152000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c06f8f0 con 0x7fb14c070e20 2024-04-25T11:30:17.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb151600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb14c06d390 0x7fb14c0cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb151600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb14c06d390 0x7fb14c0cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34226/0 (socket says 172.21.15.19:34226) 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb151600640 1 -- 172.21.15.19:0/2433805301 learned_addr learned my addr 172.21.15.19:0/2433805301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb150c00640 1 --2- 172.21.15.19:0/2433805301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c070e20 0x7fb14c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.392+0000 7fb14be00640 1 --2- 172.21.15.19:0/2433805301 >> v2:172.21.15.169:3300/0 conn(0x7fb14c06f180 0x7fb14c06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.393+0000 7fb151600640 1 -- 172.21.15.19:0/2433805301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c070e20 msgr2=0x7fb14c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.393+0000 7fb151600640 1 --2- 172.21.15.19:0/2433805301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c070e20 0x7fb14c15c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.393+0000 7fb151600640 1 -- 172.21.15.19:0/2433805301 >> v2:172.21.15.169:3300/0 conn(0x7fb14c06f180 msgr2=0x7fb14c06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.393+0000 7fb151600640 1 --2- 172.21.15.19:0/2433805301 >> v2:172.21.15.169:3300/0 conn(0x7fb14c06f180 0x7fb14c06ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.393+0000 7fb151600640 1 -- 172.21.15.19:0/2433805301 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c06fbd0 con 0x7fb14c06d390 2024-04-25T11:30:17.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.394+0000 7fb151600640 1 --2- 172.21.15.19:0/2433805301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c0cf2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0d27c0 tx=0x7fb14402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae84c923e552509c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:17.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.395+0000 7fb14b400640 1 -- 172.21.15.19:0/2433805301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144036070 con 0x7fb14c06d390 2024-04-25T11:30:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.395+0000 7fb14b400640 1 -- 172.21.15.19:0/2433805301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb144031600 con 0x7fb14c06d390 2024-04-25T11:30:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.395+0000 7fb14b400640 1 -- 172.21.15.19:0/2433805301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440319d0 con 0x7fb14c06d390 2024-04-25T11:30:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 -- 172.21.15.19:0/2433805301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 msgr2=0x7fb14c0cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 --2- 172.21.15.19:0/2433805301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c0cf2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0d27c0 tx=0x7fb14402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 -- 172.21.15.19:0/2433805301 shutdown_connections 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 --2- 172.21.15.19:0/2433805301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c0cf2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 --2- 172.21.15.19:0/2433805301 >> v2:172.21.15.169:3300/0 conn(0x7fb14c06f180 0x7fb14c06ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 --2- 172.21.15.19:0/2433805301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c070e20 0x7fb14c15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 -- 172.21.15.19:0/2433805301 >> 172.21.15.19:0/2433805301 conn(0x7fb14c0d1720 msgr2=0x7fb14c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 -- 172.21.15.19:0/2433805301 shutdown_connections 2024-04-25T11:30:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.396+0000 7fb152000640 1 -- 172.21.15.19:0/2433805301 wait complete. 2024-04-25T11:30:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.398+0000 7fb152000640 1 Processor -- start 2024-04-25T11:30:17.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.398+0000 7fb152000640 1 -- start start 2024-04-25T11:30:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb152000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c168e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb152000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c06f180 0x7fb14c14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb152000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14c070e20 0x7fb14c149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb152000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c06f590 con 0x7fb14c06d390 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb152000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c070190 con 0x7fb14c070e20 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb152000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c06c5c0 con 0x7fb14c06f180 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb150c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb150c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c168e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34236/0 (socket says 172.21.15.19:34236) 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.400+0000 7fb150c00640 1 -- 172.21.15.19:0/1143319584 learned_addr learned my addr 172.21.15.19:0/1143319584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb14be00640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c06f180 0x7fb14c14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb151600640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14c070e20 0x7fb14c149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb150c00640 1 -- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c06f180 msgr2=0x7fb14c14d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb150c00640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c06f180 0x7fb14c14d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb150c00640 1 -- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14c070e20 msgr2=0x7fb14c149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb150c00640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14c070e20 0x7fb14c149810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb150c00640 1 -- 172.21.15.19:0/1143319584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c14c4c0 con 0x7fb14c06d390 2024-04-25T11:30:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb150c00640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c168e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb14000d620 tx=0x7fb14000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb148c00640 1 -- 172.21.15.19:0/1143319584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14000b620 con 0x7fb14c06d390 2024-04-25T11:30:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb148c00640 1 -- 172.21.15.19:0/1143319584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb140019070 con 0x7fb14c06d390 2024-04-25T11:30:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb148c00640 1 -- 172.21.15.19:0/1143319584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14000ba00 con 0x7fb14c06d390 2024-04-25T11:30:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.401+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c14c7a0 con 0x7fb14c06d390 2024-04-25T11:30:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.402+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c14cd60 con 0x7fb14c06d390 2024-04-25T11:30:17.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.403+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c050740 con 0x7fb14c06d390 2024-04-25T11:30:17.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.404+0000 7fb148c00640 1 -- 172.21.15.19:0/1143319584 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb14000bba0 con 0x7fb14c06d390 2024-04-25T11:30:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.405+0000 7fb148c00640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1300744e0 0x7fb1300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.405+0000 7fb148c00640 1 -- 172.21.15.19:0/1143319584 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fb1400955f0 con 0x7fb14c06d390 2024-04-25T11:30:17.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.406+0000 7fb14be00640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1300744e0 0x7fb1300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.407+0000 7fb14be00640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1300744e0 0x7fb1300769f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb14c153470 tx=0x7fb13c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:17.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.410+0000 7fb148c00640 1 -- 172.21.15.19:0/1143319584 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb14005fdf0 con 0x7fb14c06d390 2024-04-25T11:30:17.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.611+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb14c15fb60 con 0x7fb1300744e0 2024-04-25T11:30:17.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.632+0000 7fb148c00640 1 -- 172.21.15.19:0/1143319584 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250748 (secure 0 0 0) 0x7fb14c15fb60 con 0x7fb1300744e0 2024-04-25T11:30:17.625 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:17.626 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1300744e0 msgr2=0x7fb1300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1300744e0 0x7fb1300769f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb14c153470 tx=0x7fb13c008040 comp rx=0 tx=0).stop 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 msgr2=0x7fb14c168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c168e10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb14000d620 tx=0x7fb14000dae0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 shutdown_connections 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1300744e0 0x7fb1300769f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb14c070e20 0x7fb14c149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb14c06f180 0x7fb14c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 --2- 172.21.15.19:0/1143319584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb14c06d390 0x7fb14c168e10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 >> 172.21.15.19:0/1143319584 conn(0x7fb14c0d1720 msgr2=0x7fb14c0b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 shutdown_connections 2024-04-25T11:30:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.637+0000 7fb152000640 1 -- 172.21.15.19:0/1143319584 wait complete. 2024-04-25T11:30:17.655 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-04-25T11:30:16.505179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230032,"kb_used_data":5528,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744688,"statfs":{"total":773094113280,"available":772858560512,"internally_reserved":0,"allocated":5660672,"data_stored":2998197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":229756466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002160"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.333693+0000","last_change":"2024-04-25T11:30:12.328001+0000","last_active":"2024-04-25T11:30:12.333693+0000","last_peered":"2024-04-25T11:30:12.333693+0000","last_clean":"2024-04-25T11:30:12.333693+0000","last_became_active":"2024-04-25T11:30:12.327555+0000","last_became_peered":"2024-04-25T11:30:12.327555+0000","last_unstale":"2024-04-25T11:30:12.333693+0000","last_undegraded":"2024-04-25T11:30:12.333693+0000","last_fullsized":"2024-04-25T11:30:12.333693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:03.855581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325206+0000","last_change":"2024-04-25T11:30:12.316817+0000","last_active":"2024-04-25T11:30:12.325206+0000","last_peered":"2024-04-25T11:30:12.325206+0000","last_clean":"2024-04-25T11:30:12.325206+0000","last_became_active":"2024-04-25T11:30:12.311768+0000","last_became_peered":"2024-04-25T11:30:12.311768+0000","last_unstale":"2024-04-25T11:30:12.325206+0000","last_undegraded":"2024-04-25T11:30:12.325206+0000","last_fullsized":"2024-04-25T11:30:12.325206+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.319448+0000","last_change":"2024-04-25T11:30:12.315596+0000","last_active":"2024-04-25T11:30:12.319448+0000","last_peered":"2024-04-25T11:30:12.319448+0000","last_clean":"2024-04-25T11:30:12.319448+0000","last_became_active":"2024-04-25T11:30:12.315259+0000","last_became_peered":"2024-04-25T11:30:12.315259+0000","last_unstale":"2024-04-25T11:30:12.319448+0000","last_undegraded":"2024-04-25T11:30:12.319448+0000","last_fullsized":"2024-04-25T11:30:12.319448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:18.773842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.256254+0000","last_change":"2024-04-25T11:30:11.256254+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.256254+0000","last_undegraded":"2024-04-25T11:30:11.256254+0000","last_fullsized":"2024-04-25T11:30:11.256254+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327312+0000","last_change":"2024-04-25T11:30:12.320809+0000","last_active":"2024-04-25T11:30:12.327312+0000","last_peered":"2024-04-25T11:30:12.327312+0000","last_clean":"2024-04-25T11:30:12.327312+0000","last_became_active":"2024-04-25T11:30:12.318568+0000","last_became_peered":"2024-04-25T11:30:12.318568+0000","last_unstale":"2024-04-25T11:30:12.327312+0000","last_undegraded":"2024-04-25T11:30:12.327312+0000","last_fullsized":"2024-04-25T11:30:12.327312+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324645+0000","last_change":"2024-04-25T11:30:12.316853+0000","last_active":"2024-04-25T11:30:12.324645+0000","last_peered":"2024-04-25T11:30:12.324645+0000","last_clean":"2024-04-25T11:30:12.324645+0000","last_became_active":"2024-04-25T11:30:12.316509+0000","last_became_peered":"2024-04-25T11:30:12.316509+0000","last_unstale":"2024-04-25T11:30:12.324645+0000","last_undegraded":"2024-04-25T11:30:12.324645+0000","last_fullsized":"2024-04-25T11:30:12.324645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.328160+0000","last_change":"2024-04-25T11:30:12.323818+0000","last_active":"2024-04-25T11:30:12.328160+0000","last_peered":"2024-04-25T11:30:12.328160+0000","last_clean":"2024-04-25T11:30:12.328160+0000","last_became_active":"2024-04-25T11:30:12.322940+0000","last_became_peered":"2024-04-25T11:30:12.322940+0000","last_unstale":"2024-04-25T11:30:12.328160+0000","last_undegraded":"2024-04-25T11:30:12.328160+0000","last_fullsized":"2024-04-25T11:30:12.328160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:40.175748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254522+0000","last_change":"2024-04-25T11:30:11.254522+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254522+0000","last_undegraded":"2024-04-25T11:30:11.254522+0000","last_fullsized":"2024-04-25T11:30:11.254522+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.323941+0000","last_change":"2024-04-25T11:30:12.316458+0000","last_active":"2024-04-25T11:30:12.323941+0000","last_peered":"2024-04-25T11:30:12.323941+0000","last_clean":"2024-04-25T11:30:12.323941+0000","last_became_active":"2024-04-25T11:30:12.315188+0000","last_became_peered":"2024-04-25T11:30:12.315188+0000","last_unstale":"2024-04-25T11:30:12.323941+0000","last_undegraded":"2024-04-25T11:30:12.323941+0000","last_fullsized":"2024-04-25T11:30:12.323941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:07.988273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.315531+0000","last_change":"2024-04-25T11:30:12.311133+0000","last_active":"2024-04-25T11:30:12.315531+0000","last_peered":"2024-04-25T11:30:12.315531+0000","last_clean":"2024-04-25T11:30:12.315531+0000","last_became_active":"2024-04-25T11:30:12.310670+0000","last_became_peered":"2024-04-25T11:30:12.310670+0000","last_unstale":"2024-04-25T11:30:12.315531+0000","last_undegraded":"2024-04-25T11:30:12.315531+0000","last_fullsized":"2024-04-25T11:30:12.315531+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.265764+0000","last_change":"2024-04-25T11:30:11.265764+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.265764+0000","last_undegraded":"2024-04-25T11:30:11.265764+0000","last_fullsized":"2024-04-25T11:30:11.265764+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.318340+0000","last_change":"2024-04-25T11:30:12.306558+0000","last_active":"2024-04-25T11:30:12.318340+0000","last_peered":"2024-04-25T11:30:12.318340+0000","last_clean":"2024-04-25T11:30:12.318340+0000","last_became_active":"2024-04-25T11:30:12.306159+0000","last_became_peered":"2024-04-25T11:30:12.306159+0000","last_unstale":"2024-04-25T11:30:12.318340+0000","last_undegraded":"2024-04-25T11:30:12.318340+0000","last_fullsized":"2024-04-25T11:30:12.318340+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.323321+0000","last_change":"2024-04-25T11:30:12.315317+0000","last_active":"2024-04-25T11:30:12.323321+0000","last_peered":"2024-04-25T11:30:12.323321+0000","last_clean":"2024-04-25T11:30:12.323321+0000","last_became_active":"2024-04-25T11:30:12.314919+0000","last_became_peered":"2024-04-25T11:30:12.314919+0000","last_unstale":"2024-04-25T11:30:12.323321+0000","last_undegraded":"2024-04-25T11:30:12.323321+0000","last_fullsized":"2024-04-25T11:30:12.323321+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:32.390065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.253292+0000","last_change":"2024-04-25T11:30:11.253292+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.253292+0000","last_undegraded":"2024-04-25T11:30:11.253292+0000","last_fullsized":"2024-04-25T11:30:11.253292+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.252332+0000","last_change":"2024-04-25T11:30:11.252332+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.252332+0000","last_undegraded":"2024-04-25T11:30:11.252332+0000","last_fullsized":"2024-04-25T11:30:11.252332+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327221+0000","last_change":"2024-04-25T11:30:12.320656+0000","last_active":"2024-04-25T11:30:12.327221+0000","last_peered":"2024-04-25T11:30:12.327221+0000","last_clean":"2024-04-25T11:30:12.327221+0000","last_became_active":"2024-04-25T11:30:12.319501+0000","last_became_peered":"2024-04-25T11:30:12.319501+0000","last_unstale":"2024-04-25T11:30:12.327221+0000","last_undegraded":"2024-04-25T11:30:12.327221+0000","last_fullsized":"2024-04-25T11:30:12.327221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324268+0000","last_change":"2024-04-25T11:30:12.312204+0000","last_active":"2024-04-25T11:30:12.324268+0000","last_peered":"2024-04-25T11:30:12.324268+0000","last_clean":"2024-04-25T11:30:12.324268+0000","last_became_active":"2024-04-25T11:30:12.311798+0000","last_became_peered":"2024-04-25T11:30:12.311798+0000","last_unstale":"2024-04-25T11:30:12.324268+0000","last_undegraded":"2024-04-25T11:30:12.324268+0000","last_fullsized":"2024-04-25T11:30:12.324268+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.296840+0000","last_change":"2024-04-25T11:30:12.292655+0000","last_active":"2024-04-25T11:30:12.296840+0000","last_peered":"2024-04-25T11:30:12.296840+0000","last_clean":"2024-04-25T11:30:12.296840+0000","last_became_active":"2024-04-25T11:30:12.292173+0000","last_became_peered":"2024-04-25T11:30:12.292173+0000","last_unstale":"2024-04-25T11:30:12.296840+0000","last_undegraded":"2024-04-25T11:30:12.296840+0000","last_fullsized":"2024-04-25T11:30:12.296840+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:40.609036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327149+0000","last_change":"2024-04-25T11:30:12.318490+0000","last_active":"2024-04-25T11:30:12.327149+0000","last_peered":"2024-04-25T11:30:12.327149+0000","last_clean":"2024-04-25T11:30:12.327149+0000","last_became_active":"2024-04-25T11:30:12.317442+0000","last_became_peered":"2024-04-25T11:30:12.317442+0000","last_unstale":"2024-04-25T11:30:12.327149+0000","last_undegraded":"2024-04-25T11:30:12.327149+0000","last_fullsized":"2024-04-25T11:30:12.327149+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.315731+0000","last_change":"2024-04-25T11:30:12.311311+0000","last_active":"2024-04-25T11:30:12.315731+0000","last_peered":"2024-04-25T11:30:12.315731+0000","last_clean":"2024-04-25T11:30:12.315731+0000","last_became_active":"2024-04-25T11:30:12.310846+0000","last_became_peered":"2024-04-25T11:30:12.310846+0000","last_unstale":"2024-04-25T11:30:12.315731+0000","last_undegraded":"2024-04-25T11:30:12.315731+0000","last_fullsized":"2024-04-25T11:30:12.315731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.314933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254710+0000","last_change":"2024-04-25T11:30:11.254710+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254710+0000","last_undegraded":"2024-04-25T11:30:11.254710+0000","last_fullsized":"2024-04-25T11:30:11.254710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324559+0000","last_change":"2024-04-25T11:30:12.318465+0000","last_active":"2024-04-25T11:30:12.324559+0000","last_peered":"2024-04-25T11:30:12.324559+0000","last_clean":"2024-04-25T11:30:12.324559+0000","last_became_active":"2024-04-25T11:30:12.318076+0000","last_became_peered":"2024-04-25T11:30:12.318076+0000","last_unstale":"2024-04-25T11:30:12.324559+0000","last_undegraded":"2024-04-25T11:30:12.324559+0000","last_fullsized":"2024-04-25T11:30:12.324559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:21.763769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.313691+0000","last_change":"2024-04-25T11:30:12.308548+0000","last_active":"2024-04-25T11:30:12.313691+0000","last_peered":"2024-04-25T11:30:12.313691+0000","last_clean":"2024-04-25T11:30:12.313691+0000","last_became_active":"2024-04-25T11:30:12.308148+0000","last_became_peered":"2024-04-25T11:30:12.308148+0000","last_unstale":"2024-04-25T11:30:12.313691+0000","last_undegraded":"2024-04-25T11:30:12.313691+0000","last_fullsized":"2024-04-25T11:30:12.313691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:57.673678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324424+0000","last_change":"2024-04-25T11:30:12.312781+0000","last_active":"2024-04-25T11:30:12.324424+0000","last_peered":"2024-04-25T11:30:12.324424+0000","last_clean":"2024-04-25T11:30:12.324424+0000","last_became_active":"2024-04-25T11:30:12.312453+0000","last_became_peered":"2024-04-25T11:30:12.312453+0000","last_unstale":"2024-04-25T11:30:12.324424+0000","last_undegraded":"2024-04-25T11:30:12.324424+0000","last_fullsized":"2024-04-25T11:30:12.324424+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.328444+0000","last_change":"2024-04-25T11:30:12.325014+0000","last_active":"2024-04-25T11:30:12.328444+0000","last_peered":"2024-04-25T11:30:12.328444+0000","last_clean":"2024-04-25T11:30:12.328444+0000","last_became_active":"2024-04-25T11:30:12.324660+0000","last_became_peered":"2024-04-25T11:30:12.324660+0000","last_unstale":"2024-04-25T11:30:12.328444+0000","last_undegraded":"2024-04-25T11:30:12.328444+0000","last_fullsized":"2024-04-25T11:30:12.328444+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.265187+0000","last_change":"2024-04-25T11:30:11.265187+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.265187+0000","last_undegraded":"2024-04-25T11:30:11.265187+0000","last_fullsized":"2024-04-25T11:30:11.265187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304485+0000","last_change":"2024-04-25T11:30:12.300829+0000","last_active":"2024-04-25T11:30:12.304485+0000","last_peered":"2024-04-25T11:30:12.304485+0000","last_clean":"2024-04-25T11:30:12.304485+0000","last_became_active":"2024-04-25T11:30:12.300457+0000","last_became_peered":"2024-04-25T11:30:12.300457+0000","last_unstale":"2024-04-25T11:30:12.304485+0000","last_undegraded":"2024-04-25T11:30:12.304485+0000","last_fullsized":"2024-04-25T11:30:12.304485+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:29.722464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.309487+0000","last_change":"2024-04-25T11:30:12.306296+0000","last_active":"2024-04-25T11:30:12.309487+0000","last_peered":"2024-04-25T11:30:12.309487+0000","last_clean":"2024-04-25T11:30:12.309487+0000","last_became_active":"2024-04-25T11:30:12.305922+0000","last_became_peered":"2024-04-25T11:30:12.305922+0000","last_unstale":"2024-04-25T11:30:12.309487+0000","last_undegraded":"2024-04-25T11:30:12.309487+0000","last_fullsized":"2024-04-25T11:30:12.309487+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:33.370286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.320671+0000","last_change":"2024-04-25T11:30:12.314344+0000","last_active":"2024-04-25T11:30:12.320671+0000","last_peered":"2024-04-25T11:30:12.320671+0000","last_clean":"2024-04-25T11:30:12.320671+0000","last_became_active":"2024-04-25T11:30:12.314033+0000","last_became_peered":"2024-04-25T11:30:12.314033+0000","last_unstale":"2024-04-25T11:30:12.320671+0000","last_undegraded":"2024-04-25T11:30:12.320671+0000","last_fullsized":"2024-04-25T11:30:12.320671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:10.049040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.316322+0000","last_change":"2024-04-25T11:30:12.306312+0000","last_active":"2024-04-25T11:30:12.316322+0000","last_peered":"2024-04-25T11:30:12.316322+0000","last_clean":"2024-04-25T11:30:12.316322+0000","last_became_active":"2024-04-25T11:30:12.305928+0000","last_became_peered":"2024-04-25T11:30:12.305928+0000","last_unstale":"2024-04-25T11:30:12.316322+0000","last_undegraded":"2024-04-25T11:30:12.316322+0000","last_fullsized":"2024-04-25T11:30:12.316322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:07.988273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.322258+0000","last_change":"2024-04-25T11:30:12.309122+0000","last_active":"2024-04-25T11:30:12.322258+0000","last_peered":"2024-04-25T11:30:12.322258+0000","last_clean":"2024-04-25T11:30:12.322258+0000","last_became_active":"2024-04-25T11:30:12.308689+0000","last_became_peered":"2024-04-25T11:30:12.308689+0000","last_unstale":"2024-04-25T11:30:12.322258+0000","last_undegraded":"2024-04-25T11:30:12.322258+0000","last_fullsized":"2024-04-25T11:30:12.322258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:10.291351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.252849+0000","last_change":"2024-04-25T11:30:11.252849+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.252849+0000","last_undegraded":"2024-04-25T11:30:11.252849+0000","last_fullsized":"2024-04-25T11:30:11.252849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325542+0000","last_change":"2024-04-25T11:30:12.324170+0000","last_active":"2024-04-25T11:30:12.325542+0000","last_peered":"2024-04-25T11:30:12.325542+0000","last_clean":"2024-04-25T11:30:12.325542+0000","last_became_active":"2024-04-25T11:30:12.323509+0000","last_became_peered":"2024-04-25T11:30:12.323509+0000","last_unstale":"2024-04-25T11:30:12.325542+0000","last_undegraded":"2024-04-25T11:30:12.325542+0000","last_fullsized":"2024-04-25T11:30:12.325542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:01.898568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338824+0000","last_change":"2024-04-25T11:30:11.269060+0000","last_active":"2024-04-25T11:30:12.338824+0000","last_peered":"2024-04-25T11:30:12.338824+0000","last_clean":"2024-04-25T11:30:12.338824+0000","last_became_active":"2024-04-25T11:30:11.266286+0000","last_became_peered":"2024-04-25T11:30:11.266286+0000","last_unstale":"2024-04-25T11:30:12.338824+0000","last_undegraded":"2024-04-25T11:30:12.338824+0000","last_fullsized":"2024-04-25T11:30:12.338824+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:25.474525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327322+0000","last_change":"2024-04-25T11:30:12.319935+0000","last_active":"2024-04-25T11:30:12.327322+0000","last_peered":"2024-04-25T11:30:12.327322+0000","last_clean":"2024-04-25T11:30:12.327322+0000","last_became_active":"2024-04-25T11:30:12.319486+0000","last_became_peered":"2024-04-25T11:30:12.319486+0000","last_unstale":"2024-04-25T11:30:12.327322+0000","last_undegraded":"2024-04-25T11:30:12.327322+0000","last_fullsized":"2024-04-25T11:30:12.327322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.339577+0000","last_change":"2024-04-25T11:30:11.257450+0000","last_active":"2024-04-25T11:30:12.339577+0000","last_peered":"2024-04-25T11:30:12.339577+0000","last_clean":"2024-04-25T11:30:12.339577+0000","last_became_active":"2024-04-25T11:30:11.257097+0000","last_became_peered":"2024-04-25T11:30:11.257097+0000","last_unstale":"2024-04-25T11:30:12.339577+0000","last_undegraded":"2024-04-25T11:30:12.339577+0000","last_fullsized":"2024-04-25T11:30:12.339577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.501573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325736+0000","last_change":"2024-04-25T11:30:12.318193+0000","last_active":"2024-04-25T11:30:12.325736+0000","last_peered":"2024-04-25T11:30:12.325736+0000","last_clean":"2024-04-25T11:30:12.325736+0000","last_became_active":"2024-04-25T11:30:12.317611+0000","last_became_peered":"2024-04-25T11:30:12.317611+0000","last_unstale":"2024-04-25T11:30:12.325736+0000","last_undegraded":"2024-04-25T11:30:12.325736+0000","last_fullsized":"2024-04-25T11:30:12.325736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.334754+0000","last_change":"2024-04-25T11:30:11.271679+0000","last_active":"2024-04-25T11:30:12.334754+0000","last_peered":"2024-04-25T11:30:12.334754+0000","last_clean":"2024-04-25T11:30:12.334754+0000","last_became_active":"2024-04-25T11:30:11.270812+0000","last_became_peered":"2024-04-25T11:30:11.270812+0000","last_unstale":"2024-04-25T11:30:12.334754+0000","last_undegraded":"2024-04-25T11:30:12.334754+0000","last_fullsized":"2024-04-25T11:30:12.334754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:01.243237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.261566+0000","last_change":"2024-04-25T11:30:11.261566+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.261566+0000","last_undegraded":"2024-04-25T11:30:11.261566+0000","last_fullsized":"2024-04-25T11:30:11.261566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.339891+0000","last_change":"2024-04-25T11:30:11.259259+0000","last_active":"2024-04-25T11:30:12.339891+0000","last_peered":"2024-04-25T11:30:12.339891+0000","last_clean":"2024-04-25T11:30:12.339891+0000","last_became_active":"2024-04-25T11:30:11.258919+0000","last_became_peered":"2024-04-25T11:30:11.258919+0000","last_unstale":"2024-04-25T11:30:12.339891+0000","last_undegraded":"2024-04-25T11:30:12.339891+0000","last_fullsized":"2024-04-25T11:30:12.339891+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:49.077411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.320288+0000","last_change":"2024-04-25T11:30:12.310915+0000","last_active":"2024-04-25T11:30:12.320288+0000","last_peered":"2024-04-25T11:30:12.320288+0000","last_clean":"2024-04-25T11:30:12.320288+0000","last_became_active":"2024-04-25T11:30:12.310580+0000","last_became_peered":"2024-04-25T11:30:12.310580+0000","last_unstale":"2024-04-25T11:30:12.320288+0000","last_undegraded":"2024-04-25T11:30:12.320288+0000","last_fullsized":"2024-04-25T11:30:12.320288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:26.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338450+0000","last_change":"2024-04-25T11:30:11.278985+0000","last_active":"2024-04-25T11:30:12.338450+0000","last_peered":"2024-04-25T11:30:12.338450+0000","last_clean":"2024-04-25T11:30:12.338450+0000","last_became_active":"2024-04-25T11:30:11.273156+0000","last_became_peered":"2024-04-25T11:30:11.273156+0000","last_unstale":"2024-04-25T11:30:12.338450+0000","last_undegraded":"2024-04-25T11:30:12.338450+0000","last_fullsized":"2024-04-25T11:30:12.338450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:37.174642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325053+0000","last_change":"2024-04-25T11:30:12.318436+0000","last_active":"2024-04-25T11:30:12.325053+0000","last_peered":"2024-04-25T11:30:12.325053+0000","last_clean":"2024-04-25T11:30:12.325053+0000","last_became_active":"2024-04-25T11:30:12.318108+0000","last_became_peered":"2024-04-25T11:30:12.318108+0000","last_unstale":"2024-04-25T11:30:12.325053+0000","last_undegraded":"2024-04-25T11:30:12.325053+0000","last_fullsized":"2024-04-25T11:30:12.325053+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338843+0000","last_change":"2024-04-25T11:30:11.263150+0000","last_active":"2024-04-25T11:30:12.338843+0000","last_peered":"2024-04-25T11:30:12.338843+0000","last_clean":"2024-04-25T11:30:12.338843+0000","last_became_active":"2024-04-25T11:30:11.262389+0000","last_became_peered":"2024-04-25T11:30:11.262389+0000","last_unstale":"2024-04-25T11:30:12.338843+0000","last_undegraded":"2024-04-25T11:30:12.338843+0000","last_fullsized":"2024-04-25T11:30:12.338843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:44.476276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321591+0000","last_change":"2024-04-25T11:30:12.313164+0000","last_active":"2024-04-25T11:30:12.321591+0000","last_peered":"2024-04-25T11:30:12.321591+0000","last_clean":"2024-04-25T11:30:12.321591+0000","last_became_active":"2024-04-25T11:30:12.311004+0000","last_became_peered":"2024-04-25T11:30:12.311004+0000","last_unstale":"2024-04-25T11:30:12.321591+0000","last_undegraded":"2024-04-25T11:30:12.321591+0000","last_fullsized":"2024-04-25T11:30:12.321591+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:32.067456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.303499+0000","last_change":"2024-04-25T11:30:11.272497+0000","last_active":"2024-04-25T11:30:12.303499+0000","last_peered":"2024-04-25T11:30:12.303499+0000","last_clean":"2024-04-25T11:30:12.303499+0000","last_became_active":"2024-04-25T11:30:11.272150+0000","last_became_peered":"2024-04-25T11:30:11.272150+0000","last_unstale":"2024-04-25T11:30:12.303499+0000","last_undegraded":"2024-04-25T11:30:12.303499+0000","last_fullsized":"2024-04-25T11:30:12.303499+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:09.285233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.329154+0000","last_change":"2024-04-25T11:30:12.326351+0000","last_active":"2024-04-25T11:30:12.329154+0000","last_peered":"2024-04-25T11:30:12.329154+0000","last_clean":"2024-04-25T11:30:12.329154+0000","last_became_active":"2024-04-25T11:30:12.326055+0000","last_became_peered":"2024-04-25T11:30:12.326055+0000","last_unstale":"2024-04-25T11:30:12.329154+0000","last_undegraded":"2024-04-25T11:30:12.329154+0000","last_fullsized":"2024-04-25T11:30:12.329154+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:26.804870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304084+0000","last_change":"2024-04-25T11:30:11.280682+0000","last_active":"2024-04-25T11:30:12.304084+0000","last_peered":"2024-04-25T11:30:12.304084+0000","last_clean":"2024-04-25T11:30:12.304084+0000","last_became_active":"2024-04-25T11:30:11.279049+0000","last_became_peered":"2024-04-25T11:30:11.279049+0000","last_unstale":"2024-04-25T11:30:12.304084+0000","last_undegraded":"2024-04-25T11:30:12.304084+0000","last_fullsized":"2024-04-25T11:30:12.304084+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:40.524169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.316369+0000","last_change":"2024-04-25T11:30:12.307371+0000","last_active":"2024-04-25T11:30:12.316369+0000","last_peered":"2024-04-25T11:30:12.316369+0000","last_clean":"2024-04-25T11:30:12.316369+0000","last_became_active":"2024-04-25T11:30:12.307030+0000","last_became_peered":"2024-04-25T11:30:12.307030+0000","last_unstale":"2024-04-25T11:30:12.316369+0000","last_undegraded":"2024-04-25T11:30:12.316369+0000","last_fullsized":"2024-04-25T11:30:12.316369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.144352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.281829+0000","last_change":"2024-04-25T11:30:11.267439+0000","last_active":"2024-04-25T11:30:12.281829+0000","last_peered":"2024-04-25T11:30:12.281829+0000","last_clean":"2024-04-25T11:30:12.281829+0000","last_became_active":"2024-04-25T11:30:11.267110+0000","last_became_peered":"2024-04-25T11:30:11.267110+0000","last_unstale":"2024-04-25T11:30:12.281829+0000","last_undegraded":"2024-04-25T11:30:12.281829+0000","last_fullsized":"2024-04-25T11:30:12.281829+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:30.677292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.318253+0000","last_change":"2024-04-25T11:30:12.311247+0000","last_active":"2024-04-25T11:30:12.318253+0000","last_peered":"2024-04-25T11:30:12.318253+0000","last_clean":"2024-04-25T11:30:12.318253+0000","last_became_active":"2024-04-25T11:30:12.310904+0000","last_became_peered":"2024-04-25T11:30:12.310904+0000","last_unstale":"2024-04-25T11:30:12.318253+0000","last_undegraded":"2024-04-25T11:30:12.318253+0000","last_fullsized":"2024-04-25T11:30:12.318253+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:23.098668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304732+0000","last_change":"2024-04-25T11:30:11.272789+0000","last_active":"2024-04-25T11:30:12.304732+0000","last_peered":"2024-04-25T11:30:12.304732+0000","last_clean":"2024-04-25T11:30:12.304732+0000","last_became_active":"2024-04-25T11:30:11.272445+0000","last_became_peered":"2024-04-25T11:30:11.272445+0000","last_unstale":"2024-04-25T11:30:12.304732+0000","last_undegraded":"2024-04-25T11:30:12.304732+0000","last_fullsized":"2024-04-25T11:30:12.304732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.849463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254783+0000","last_change":"2024-04-25T11:30:11.254783+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254783+0000","last_undegraded":"2024-04-25T11:30:11.254783+0000","last_fullsized":"2024-04-25T11:30:11.254783+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338671+0000","last_change":"2024-04-25T11:30:11.264837+0000","last_active":"2024-04-25T11:30:12.338671+0000","last_peered":"2024-04-25T11:30:12.338671+0000","last_clean":"2024-04-25T11:30:12.338671+0000","last_became_active":"2024-04-25T11:30:11.264166+0000","last_became_peered":"2024-04-25T11:30:11.264166+0000","last_unstale":"2024-04-25T11:30:12.338671+0000","last_undegraded":"2024-04-25T11:30:12.338671+0000","last_fullsized":"2024-04-25T11:30:12.338671+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327792+0000","last_change":"2024-04-25T11:30:12.315841+0000","last_active":"2024-04-25T11:30:12.327792+0000","last_peered":"2024-04-25T11:30:12.327792+0000","last_clean":"2024-04-25T11:30:12.327792+0000","last_became_active":"2024-04-25T11:30:12.315087+0000","last_became_peered":"2024-04-25T11:30:12.315087+0000","last_unstale":"2024-04-25T11:30:12.327792+0000","last_undegraded":"2024-04-25T11:30:12.327792+0000","last_fullsized":"2024-04-25T11:30:12.327792+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:30.242892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.341752+0000","last_change":"2024-04-25T11:30:11.273982+0000","last_active":"2024-04-25T11:30:12.341752+0000","last_peered":"2024-04-25T11:30:12.341752+0000","last_clean":"2024-04-25T11:30:12.341752+0000","last_became_active":"2024-04-25T11:30:11.272233+0000","last_became_peered":"2024-04-25T11:30:11.272233+0000","last_unstale":"2024-04-25T11:30:12.341752+0000","last_undegraded":"2024-04-25T11:30:12.341752+0000","last_fullsized":"2024-04-25T11:30:12.341752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:34.401573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.296892+0000","last_change":"2024-04-25T11:30:12.294967+0000","last_active":"2024-04-25T11:30:12.296892+0000","last_peered":"2024-04-25T11:30:12.296892+0000","last_clean":"2024-04-25T11:30:12.296892+0000","last_became_active":"2024-04-25T11:30:12.294545+0000","last_became_peered":"2024-04-25T11:30:12.294545+0000","last_unstale":"2024-04-25T11:30:12.296892+0000","last_undegraded":"2024-04-25T11:30:12.296892+0000","last_fullsized":"2024-04-25T11:30:12.296892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:17.394404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.334053+0000","last_change":"2024-04-25T11:30:11.278510+0000","last_active":"2024-04-25T11:30:12.334053+0000","last_peered":"2024-04-25T11:30:12.334053+0000","last_clean":"2024-04-25T11:30:12.334053+0000","last_became_active":"2024-04-25T11:30:11.278136+0000","last_became_peered":"2024-04-25T11:30:11.278136+0000","last_unstale":"2024-04-25T11:30:12.334053+0000","last_undegraded":"2024-04-25T11:30:12.334053+0000","last_fullsized":"2024-04-25T11:30:12.334053+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:44.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327613+0000","last_change":"2024-04-25T11:30:12.326783+0000","last_active":"2024-04-25T11:30:12.327613+0000","last_peered":"2024-04-25T11:30:12.327613+0000","last_clean":"2024-04-25T11:30:12.327613+0000","last_became_active":"2024-04-25T11:30:12.325755+0000","last_became_peered":"2024-04-25T11:30:12.325755+0000","last_unstale":"2024-04-25T11:30:12.327613+0000","last_undegraded":"2024-04-25T11:30:12.327613+0000","last_fullsized":"2024-04-25T11:30:12.327613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:20.342509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304553+0000","last_change":"2024-04-25T11:30:11.269357+0000","last_active":"2024-04-25T11:30:12.304553+0000","last_peered":"2024-04-25T11:30:12.304553+0000","last_clean":"2024-04-25T11:30:12.304553+0000","last_became_active":"2024-04-25T11:30:11.268999+0000","last_became_peered":"2024-04-25T11:30:11.268999+0000","last_unstale":"2024-04-25T11:30:12.304553+0000","last_undegraded":"2024-04-25T11:30:12.304553+0000","last_fullsized":"2024-04-25T11:30:12.304553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321284+0000","last_change":"2024-04-25T11:30:12.316130+0000","last_active":"2024-04-25T11:30:12.321284+0000","last_peered":"2024-04-25T11:30:12.321284+0000","last_clean":"2024-04-25T11:30:12.321284+0000","last_became_active":"2024-04-25T11:30:12.315793+0000","last_became_peered":"2024-04-25T11:30:12.315793+0000","last_unstale":"2024-04-25T11:30:12.321284+0000","last_undegraded":"2024-04-25T11:30:12.321284+0000","last_fullsized":"2024-04-25T11:30:12.321284+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:23.098668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.300732+0000","last_change":"2024-04-25T11:30:11.279239+0000","last_active":"2024-04-25T11:30:12.300732+0000","last_peered":"2024-04-25T11:30:12.300732+0000","last_clean":"2024-04-25T11:30:12.300732+0000","last_became_active":"2024-04-25T11:30:11.278889+0000","last_became_peered":"2024-04-25T11:30:11.278889+0000","last_unstale":"2024-04-25T11:30:12.300732+0000","last_undegraded":"2024-04-25T11:30:12.300732+0000","last_fullsized":"2024-04-25T11:30:12.300732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:12.914763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324849+0000","last_change":"2024-04-25T11:30:12.306734+0000","last_active":"2024-04-25T11:30:12.324849+0000","last_peered":"2024-04-25T11:30:12.324849+0000","last_clean":"2024-04-25T11:30:12.324849+0000","last_became_active":"2024-04-25T11:30:12.306242+0000","last_became_peered":"2024-04-25T11:30:12.306242+0000","last_unstale":"2024-04-25T11:30:12.324849+0000","last_undegraded":"2024-04-25T11:30:12.324849+0000","last_fullsized":"2024-04-25T11:30:12.324849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:10.049040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.340720+0000","last_change":"2024-04-25T11:30:11.274461+0000","last_active":"2024-04-25T11:30:12.340720+0000","last_peered":"2024-04-25T11:30:12.340720+0000","last_clean":"2024-04-25T11:30:12.340720+0000","last_became_active":"2024-04-25T11:30:11.272811+0000","last_became_peered":"2024-04-25T11:30:11.272811+0000","last_unstale":"2024-04-25T11:30:12.340720+0000","last_undegraded":"2024-04-25T11:30:12.340720+0000","last_fullsized":"2024-04-25T11:30:12.340720+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:01.676380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325409+0000","last_change":"2024-04-25T11:30:12.321405+0000","last_active":"2024-04-25T11:30:12.325409+0000","last_peered":"2024-04-25T11:30:12.325409+0000","last_clean":"2024-04-25T11:30:12.325409+0000","last_became_active":"2024-04-25T11:30:12.319648+0000","last_became_peered":"2024-04-25T11:30:12.319648+0000","last_unstale":"2024-04-25T11:30:12.325409+0000","last_undegraded":"2024-04-25T11:30:12.325409+0000","last_fullsized":"2024-04-25T11:30:12.325409+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:18.773842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T11:30:11.283123+0000","last_change":"2024-04-25T11:30:11.281057+0000","last_active":"2024-04-25T11:30:11.283123+0000","last_peered":"2024-04-25T11:30:11.283123+0000","last_clean":"2024-04-25T11:30:11.283123+0000","last_became_active":"2024-04-25T11:30:11.279339+0000","last_became_peered":"2024-04-25T11:30:11.279339+0000","last_unstale":"2024-04-25T11:30:11.283123+0000","last_undegraded":"2024-04-25T11:30:11.283123+0000","last_fullsized":"2024-04-25T11:30:11.283123+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:18.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321209+0000","last_change":"2024-04-25T11:30:12.315868+0000","last_active":"2024-04-25T11:30:12.321209+0000","last_peered":"2024-04-25T11:30:12.321209+0000","last_clean":"2024-04-25T11:30:12.321209+0000","last_became_active":"2024-04-25T11:30:12.315511+0000","last_became_peered":"2024-04-25T11:30:12.315511+0000","last_unstale":"2024-04-25T11:30:12.321209+0000","last_undegraded":"2024-04-25T11:30:12.321209+0000","last_fullsized":"2024-04-25T11:30:12.321209+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.144352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327577+0000","last_change":"2024-04-25T11:30:11.271207+0000","last_active":"2024-04-25T11:30:12.327577+0000","last_peered":"2024-04-25T11:30:12.327577+0000","last_clean":"2024-04-25T11:30:12.327577+0000","last_became_active":"2024-04-25T11:30:11.270083+0000","last_became_peered":"2024-04-25T11:30:11.270083+0000","last_unstale":"2024-04-25T11:30:12.327577+0000","last_undegraded":"2024-04-25T11:30:12.327577+0000","last_fullsized":"2024-04-25T11:30:12.327577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:40.274271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325710+0000","last_change":"2024-04-25T11:30:12.317980+0000","last_active":"2024-04-25T11:30:12.325710+0000","last_peered":"2024-04-25T11:30:12.325710+0000","last_clean":"2024-04-25T11:30:12.325710+0000","last_became_active":"2024-04-25T11:30:12.317574+0000","last_became_peered":"2024-04-25T11:30:12.317574+0000","last_unstale":"2024-04-25T11:30:12.325710+0000","last_undegraded":"2024-04-25T11:30:12.325710+0000","last_fullsized":"2024-04-25T11:30:12.325710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327952+0000","last_change":"2024-04-25T11:30:11.276120+0000","last_active":"2024-04-25T11:30:12.327952+0000","last_peered":"2024-04-25T11:30:12.327952+0000","last_clean":"2024-04-25T11:30:12.327952+0000","last_became_active":"2024-04-25T11:30:11.275804+0000","last_became_peered":"2024-04-25T11:30:11.275804+0000","last_unstale":"2024-04-25T11:30:12.327952+0000","last_undegraded":"2024-04-25T11:30:12.327952+0000","last_fullsized":"2024-04-25T11:30:12.327952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:37.492766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.319419+0000","last_change":"2024-04-25T11:30:12.315454+0000","last_active":"2024-04-25T11:30:12.319419+0000","last_peered":"2024-04-25T11:30:12.319419+0000","last_clean":"2024-04-25T11:30:12.319419+0000","last_became_active":"2024-04-25T11:30:12.315114+0000","last_became_peered":"2024-04-25T11:30:12.315114+0000","last_unstale":"2024-04-25T11:30:12.319419+0000","last_undegraded":"2024-04-25T11:30:12.319419+0000","last_fullsized":"2024-04-25T11:30:12.319419+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:21.763769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304284+0000","last_change":"2024-04-25T11:30:11.279419+0000","last_active":"2024-04-25T11:30:12.304284+0000","last_peered":"2024-04-25T11:30:12.304284+0000","last_clean":"2024-04-25T11:30:12.304284+0000","last_became_active":"2024-04-25T11:30:11.279074+0000","last_became_peered":"2024-04-25T11:30:11.279074+0000","last_unstale":"2024-04-25T11:30:12.304284+0000","last_undegraded":"2024-04-25T11:30:12.304284+0000","last_fullsized":"2024-04-25T11:30:12.304284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:27.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.326110+0000","last_change":"2024-04-25T11:30:12.320762+0000","last_active":"2024-04-25T11:30:12.326110+0000","last_peered":"2024-04-25T11:30:12.326110+0000","last_clean":"2024-04-25T11:30:12.326110+0000","last_became_active":"2024-04-25T11:30:12.320207+0000","last_became_peered":"2024-04-25T11:30:12.320207+0000","last_unstale":"2024-04-25T11:30:12.326110+0000","last_undegraded":"2024-04-25T11:30:12.326110+0000","last_fullsized":"2024-04-25T11:30:12.326110+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:18.987772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.335531+0000","last_change":"2024-04-25T11:30:11.273562+0000","last_active":"2024-04-25T11:30:12.335531+0000","last_peered":"2024-04-25T11:30:12.335531+0000","last_clean":"2024-04-25T11:30:12.335531+0000","last_became_active":"2024-04-25T11:30:11.273187+0000","last_became_peered":"2024-04-25T11:30:11.273187+0000","last_unstale":"2024-04-25T11:30:12.335531+0000","last_undegraded":"2024-04-25T11:30:12.335531+0000","last_fullsized":"2024-04-25T11:30:12.335531+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:37.187391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.322121+0000","last_change":"2024-04-25T11:30:12.310068+0000","last_active":"2024-04-25T11:30:12.322121+0000","last_peered":"2024-04-25T11:30:12.322121+0000","last_clean":"2024-04-25T11:30:12.322121+0000","last_became_active":"2024-04-25T11:30:12.309236+0000","last_became_peered":"2024-04-25T11:30:12.309236+0000","last_unstale":"2024-04-25T11:30:12.322121+0000","last_undegraded":"2024-04-25T11:30:12.322121+0000","last_fullsized":"2024-04-25T11:30:12.322121+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.510835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305149+0000","last_change":"2024-04-25T11:30:11.260102+0000","last_active":"2024-04-25T11:30:12.305149+0000","last_peered":"2024-04-25T11:30:12.305149+0000","last_clean":"2024-04-25T11:30:12.305149+0000","last_became_active":"2024-04-25T11:30:11.259712+0000","last_became_peered":"2024-04-25T11:30:11.259712+0000","last_unstale":"2024-04-25T11:30:12.305149+0000","last_undegraded":"2024-04-25T11:30:12.305149+0000","last_fullsized":"2024-04-25T11:30:12.305149+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:31.383947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.251703+0000","last_change":"2024-04-25T11:30:11.251703+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.251703+0000","last_undegraded":"2024-04-25T11:30:11.251703+0000","last_fullsized":"2024-04-25T11:30:11.251703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304576+0000","last_change":"2024-04-25T11:30:11.274700+0000","last_active":"2024-04-25T11:30:12.304576+0000","last_peered":"2024-04-25T11:30:12.304576+0000","last_clean":"2024-04-25T11:30:12.304576+0000","last_became_active":"2024-04-25T11:30:11.274283+0000","last_became_peered":"2024-04-25T11:30:11.274283+0000","last_unstale":"2024-04-25T11:30:12.304576+0000","last_undegraded":"2024-04-25T11:30:12.304576+0000","last_fullsized":"2024-04-25T11:30:12.304576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.081180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.250705+0000","last_change":"2024-04-25T11:30:11.250705+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.250705+0000","last_undegraded":"2024-04-25T11:30:11.250705+0000","last_fullsized":"2024-04-25T11:30:11.250705+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.297476+0000","last_change":"2024-04-25T11:30:11.266163+0000","last_active":"2024-04-25T11:30:12.297476+0000","last_peered":"2024-04-25T11:30:12.297476+0000","last_clean":"2024-04-25T11:30:12.297476+0000","last_became_active":"2024-04-25T11:30:11.265810+0000","last_became_peered":"2024-04-25T11:30:11.265810+0000","last_unstale":"2024-04-25T11:30:12.297476+0000","last_undegraded":"2024-04-25T11:30:12.297476+0000","last_fullsized":"2024-04-25T11:30:12.297476+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:20.553784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.329132+0000","last_change":"2024-04-25T11:30:12.325830+0000","last_active":"2024-04-25T11:30:12.329132+0000","last_peered":"2024-04-25T11:30:12.329132+0000","last_clean":"2024-04-25T11:30:12.329132+0000","last_became_active":"2024-04-25T11:30:12.325485+0000","last_became_peered":"2024-04-25T11:30:12.325485+0000","last_unstale":"2024-04-25T11:30:12.329132+0000","last_undegraded":"2024-04-25T11:30:12.329132+0000","last_fullsized":"2024-04-25T11:30:12.329132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T11:30:11.276720+0000","last_change":"2024-04-25T11:30:11.274375+0000","last_active":"2024-04-25T11:30:11.276720+0000","last_peered":"2024-04-25T11:30:11.276720+0000","last_clean":"2024-04-25T11:30:11.276720+0000","last_became_active":"2024-04-25T11:30:11.273757+0000","last_became_peered":"2024-04-25T11:30:11.273757+0000","last_unstale":"2024-04-25T11:30:11.276720+0000","last_undegraded":"2024-04-25T11:30:11.276720+0000","last_fullsized":"2024-04-25T11:30:11.276720+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:43.406052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321025+0000","last_change":"2024-04-25T11:30:12.314257+0000","last_active":"2024-04-25T11:30:12.321025+0000","last_peered":"2024-04-25T11:30:12.321025+0000","last_clean":"2024-04-25T11:30:12.321025+0000","last_became_active":"2024-04-25T11:30:12.313948+0000","last_became_peered":"2024-04-25T11:30:12.313948+0000","last_unstale":"2024-04-25T11:30:12.321025+0000","last_undegraded":"2024-04-25T11:30:12.321025+0000","last_fullsized":"2024-04-25T11:30:12.321025+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305887+0000","last_change":"2024-04-25T11:30:11.263543+0000","last_active":"2024-04-25T11:30:12.305887+0000","last_peered":"2024-04-25T11:30:12.305887+0000","last_clean":"2024-04-25T11:30:12.305887+0000","last_became_active":"2024-04-25T11:30:11.263181+0000","last_became_peered":"2024-04-25T11:30:11.263181+0000","last_unstale":"2024-04-25T11:30:12.305887+0000","last_undegraded":"2024-04-25T11:30:12.305887+0000","last_fullsized":"2024-04-25T11:30:12.305887+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.090545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327820+0000","last_change":"2024-04-25T11:30:12.316106+0000","last_active":"2024-04-25T11:30:12.327820+0000","last_peered":"2024-04-25T11:30:12.327820+0000","last_clean":"2024-04-25T11:30:12.327820+0000","last_became_active":"2024-04-25T11:30:12.315513+0000","last_became_peered":"2024-04-25T11:30:12.315513+0000","last_unstale":"2024-04-25T11:30:12.327820+0000","last_undegraded":"2024-04-25T11:30:12.327820+0000","last_fullsized":"2024-04-25T11:30:12.327820+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:56.498661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.306699+0000","last_change":"2024-04-25T11:30:11.264133+0000","last_active":"2024-04-25T11:30:12.306699+0000","last_peered":"2024-04-25T11:30:12.306699+0000","last_clean":"2024-04-25T11:30:12.306699+0000","last_became_active":"2024-04-25T11:30:11.263693+0000","last_became_peered":"2024-04-25T11:30:11.263693+0000","last_unstale":"2024-04-25T11:30:12.306699+0000","last_undegraded":"2024-04-25T11:30:12.306699+0000","last_fullsized":"2024-04-25T11:30:12.306699+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.388286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.326385+0000","last_change":"2024-04-25T11:30:12.321099+0000","last_active":"2024-04-25T11:30:12.326385+0000","last_peered":"2024-04-25T11:30:12.326385+0000","last_clean":"2024-04-25T11:30:12.326385+0000","last_became_active":"2024-04-25T11:30:12.319720+0000","last_became_peered":"2024-04-25T11:30:12.319720+0000","last_unstale":"2024-04-25T11:30:12.326385+0000","last_undegraded":"2024-04-25T11:30:12.326385+0000","last_fullsized":"2024-04-25T11:30:12.326385+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:28.188588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305559+0000","last_change":"2024-04-25T11:30:11.278565+0000","last_active":"2024-04-25T11:30:12.305559+0000","last_peered":"2024-04-25T11:30:12.305559+0000","last_clean":"2024-04-25T11:30:12.305559+0000","last_became_active":"2024-04-25T11:30:11.278040+0000","last_became_peered":"2024-04-25T11:30:11.278040+0000","last_unstale":"2024-04-25T11:30:12.305559+0000","last_undegraded":"2024-04-25T11:30:12.305559+0000","last_fullsized":"2024-04-25T11:30:12.305559+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.982156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.255841+0000","last_change":"2024-04-25T11:30:11.255841+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.255841+0000","last_undegraded":"2024-04-25T11:30:11.255841+0000","last_fullsized":"2024-04-25T11:30:11.255841+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.298802+0000","last_change":"2024-04-25T11:30:11.272593+0000","last_active":"2024-04-25T11:30:12.298802+0000","last_peered":"2024-04-25T11:30:12.298802+0000","last_clean":"2024-04-25T11:30:12.298802+0000","last_became_active":"2024-04-25T11:30:11.272280+0000","last_became_peered":"2024-04-25T11:30:11.272280+0000","last_unstale":"2024-04-25T11:30:12.298802+0000","last_undegraded":"2024-04-25T11:30:12.298802+0000","last_fullsized":"2024-04-25T11:30:12.298802+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:29.649052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.302366+0000","last_change":"2024-04-25T11:30:12.296593+0000","last_active":"2024-04-25T11:30:12.302366+0000","last_peered":"2024-04-25T11:30:12.302366+0000","last_clean":"2024-04-25T11:30:12.302366+0000","last_became_active":"2024-04-25T11:30:12.296265+0000","last_became_peered":"2024-04-25T11:30:12.296265+0000","last_unstale":"2024-04-25T11:30:12.302366+0000","last_undegraded":"2024-04-25T11:30:12.302366+0000","last_fullsized":"2024-04-25T11:30:12.302366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:45.204846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304061+0000","last_change":"2024-04-25T11:30:11.271824+0000","last_active":"2024-04-25T11:30:12.304061+0000","last_peered":"2024-04-25T11:30:12.304061+0000","last_clean":"2024-04-25T11:30:12.304061+0000","last_became_active":"2024-04-25T11:30:11.270927+0000","last_became_peered":"2024-04-25T11:30:11.270927+0000","last_unstale":"2024-04-25T11:30:12.304061+0000","last_undegraded":"2024-04-25T11:30:12.304061+0000","last_fullsized":"2024-04-25T11:30:12.304061+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:19.313207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.303104+0000","last_change":"2024-04-25T11:30:12.298869+0000","last_active":"2024-04-25T11:30:12.303104+0000","last_peered":"2024-04-25T11:30:12.303104+0000","last_clean":"2024-04-25T11:30:12.303104+0000","last_became_active":"2024-04-25T11:30:12.298520+0000","last_became_peered":"2024-04-25T11:30:12.298520+0000","last_unstale":"2024-04-25T11:30:12.303104+0000","last_undegraded":"2024-04-25T11:30:12.303104+0000","last_fullsized":"2024-04-25T11:30:12.303104+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:32.390065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304469+0000","last_change":"2024-04-25T11:30:11.275287+0000","last_active":"2024-04-25T11:30:12.304469+0000","last_peered":"2024-04-25T11:30:12.304469+0000","last_clean":"2024-04-25T11:30:12.304469+0000","last_became_active":"2024-04-25T11:30:11.274574+0000","last_became_peered":"2024-04-25T11:30:11.274574+0000","last_unstale":"2024-04-25T11:30:12.304469+0000","last_undegraded":"2024-04-25T11:30:12.304469+0000","last_fullsized":"2024-04-25T11:30:12.304469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:39.602919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.264576+0000","last_change":"2024-04-25T11:30:11.264576+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.264576+0000","last_undegraded":"2024-04-25T11:30:11.264576+0000","last_fullsized":"2024-04-25T11:30:11.264576+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.335329+0000","last_change":"2024-04-25T11:30:11.272554+0000","last_active":"2024-04-25T11:30:12.335329+0000","last_peered":"2024-04-25T11:30:12.335329+0000","last_clean":"2024-04-25T11:30:12.335329+0000","last_became_active":"2024-04-25T11:30:11.272199+0000","last_became_peered":"2024-04-25T11:30:11.272199+0000","last_unstale":"2024-04-25T11:30:12.335329+0000","last_undegraded":"2024-04-25T11:30:12.335329+0000","last_fullsized":"2024-04-25T11:30:12.335329+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:32.204333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":161,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.276461+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:30:12.276461+0000","last_peered":"2024-04-25T11:30:12.276461+0000","last_clean":"2024-04-25T11:30:12.276461+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:30:12.276461+0000","last_undegraded":"2024-04-25T11:30:12.276461+0000","last_fullsized":"2024-04-25T11:30:12.276461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1167360,"data_stored":823540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1150976,"data_stored":816695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:30:17.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:30:17.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.997+0000 7fd2db000640 1 Processor -- start 2024-04-25T11:30:17.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.997+0000 7fd2db000640 1 -- start start 2024-04-25T11:30:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.999+0000 7fd2db000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2d40d0ed0 0x7fd2d40d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.999+0000 7fd2db000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd2d4160be0 0x7fd2d4160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.999+0000 7fd2db000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d4161530 con 0x7fd2d41434d0 2024-04-25T11:30:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.999+0000 7fd2db000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d41438e0 con 0x7fd2d40d0ed0 2024-04-25T11:30:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:17.999+0000 7fd2db000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d406f3e0 con 0x7fd2d4160be0 2024-04-25T11:30:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2d40d0ed0 0x7fd2d40d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2d40d0ed0 0x7fd2d40d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34238/0 (socket says 172.21.15.19:34238) 2024-04-25T11:30:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d3400640 1 -- 172.21.15.19:0/2539834233 learned_addr learned my addr 172.21.15.19:0/2539834233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d9000640 1 --2- 172.21.15.19:0/2539834233 >> v2:172.21.15.169:3300/0 conn(0x7fd2d4160be0 0x7fd2d4160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d3400640 1 -- 172.21.15.19:0/2539834233 >> v2:172.21.15.169:3300/0 conn(0x7fd2d4160be0 msgr2=0x7fd2d4160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d3400640 1 --2- 172.21.15.19:0/2539834233 >> v2:172.21.15.169:3300/0 conn(0x7fd2d4160be0 0x7fd2d4160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d3400640 1 -- 172.21.15.19:0/2539834233 >> v1:172.21.15.169:6789/0 conn(0x7fd2d41434d0 legacy=0x7fd2d4070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.000+0000 7fd2d3400640 1 -- 172.21.15.19:0/2539834233 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d406f6c0 con 0x7fd2d40d0ed0 2024-04-25T11:30:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.001+0000 7fd2d3400640 1 --2- 172.21.15.19:0/2539834233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d40d12e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009870 tx=0x7fd2c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b9d935cddfc17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.002+0000 7fd2d2a00640 1 -- 172.21.15.19:0/2539834233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0037070 con 0x7fd2d40d0ed0 2024-04-25T11:30:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.002+0000 7fd2d2a00640 1 -- 172.21.15.19:0/2539834233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0032440 con 0x7fd2d40d0ed0 2024-04-25T11:30:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.002+0000 7fd2d2a00640 1 -- 172.21.15.19:0/2539834233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0032810 con 0x7fd2d40d0ed0 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.002+0000 7fd2db000640 1 -- 172.21.15.19:0/2539834233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d40d0ed0 msgr2=0x7fd2d40d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.002+0000 7fd2db000640 1 --2- 172.21.15.19:0/2539834233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d40d12e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009870 tx=0x7fd2c002a270 comp rx=0 tx=0).stop 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.003+0000 7fd2db000640 1 -- 172.21.15.19:0/2539834233 shutdown_connections 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.003+0000 7fd2db000640 1 --2- 172.21.15.19:0/2539834233 >> v2:172.21.15.169:3300/0 conn(0x7fd2d4160be0 0x7fd2d4160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.003+0000 7fd2db000640 1 --2- 172.21.15.19:0/2539834233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d40d12e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.003+0000 7fd2db000640 1 -- 172.21.15.19:0/2539834233 >> 172.21.15.19:0/2539834233 conn(0x7fd2d40d1ae0 msgr2=0x7fd2d40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.003+0000 7fd2db000640 1 -- 172.21.15.19:0/2539834233 shutdown_connections 2024-04-25T11:30:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.003+0000 7fd2db000640 1 -- 172.21.15.19:0/2539834233 wait complete. 2024-04-25T11:30:17.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.005+0000 7fd2db000640 1 Processor -- start 2024-04-25T11:30:17.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.005+0000 7fd2db000640 1 -- start start 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2db000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d413e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2db000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2d41434d0 0x7fd2d41f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2db000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4160be0 0x7fd2d40af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2db000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d406f090 con 0x7fd2d4160be0 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2db000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d406fc80 con 0x7fd2d40d0ed0 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2db000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d40b51a0 con 0x7fd2d41434d0 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2d9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4160be0 0x7fd2d40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2d9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4160be0 0x7fd2d40af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34254/0 (socket says 172.21.15.19:34254) 2024-04-25T11:30:17.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.007+0000 7fd2d9000640 1 -- 172.21.15.19:0/2989054301 learned_addr learned my addr 172.21.15.19:0/2989054301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d3e00640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d413e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d9000640 1 -- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2d41434d0 msgr2=0x7fd2d41f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d9000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2d41434d0 0x7fd2d41f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d9000640 1 -- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d40d0ed0 msgr2=0x7fd2d413e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d9000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d413e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d9000640 1 -- 172.21.15.19:0/2989054301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d413ed60 con 0x7fd2d4160be0 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d3e00640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d413e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2d9000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4160be0 0x7fd2d40af690 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc007e60 tx=0x7fd2cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2cbe00640 1 -- 172.21.15.19:0/2989054301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc014420 con 0x7fd2d4160be0 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2cbe00640 1 -- 172.21.15.19:0/2989054301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2cc019070 con 0x7fd2d4160be0 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2cbe00640 1 -- 172.21.15.19:0/2989054301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc014820 con 0x7fd2d4160be0 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d413e810 con 0x7fd2d4160be0 2024-04-25T11:30:17.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.008+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d4151910 con 0x7fd2d4160be0 2024-04-25T11:30:18.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.010+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d4068f70 con 0x7fd2d4160be0 2024-04-25T11:30:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.011+0000 7fd2cbe00640 1 -- 172.21.15.19:0/2989054301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd2cc0149c0 con 0x7fd2d4160be0 2024-04-25T11:30:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.013+0000 7fd2cbe00640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2b40744b0 0x7fd2b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.013+0000 7fd2cbe00640 1 -- 172.21.15.19:0/2989054301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fd2cc095ac0 con 0x7fd2d4160be0 2024-04-25T11:30:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.013+0000 7fd2d3e00640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2b40744b0 0x7fd2b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.014+0000 7fd2d3e00640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2b40744b0 0x7fd2b40769c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400a830 tx=0x7fd2c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.018+0000 7fd2cbe00640 1 -- 172.21.15.19:0/2989054301 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2d4068f70 con 0x7fd2d4160be0 2024-04-25T11:30:18.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.218+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2d41516a0 con 0x7fd2b40744b0 2024-04-25T11:30:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.238+0000 7fd2cbe00640 1 -- 172.21.15.19:0/2989054301 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250748 (secure 0 0 0) 0x7fd2d41516a0 con 0x7fd2b40744b0 2024-04-25T11:30:18.233 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:30:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2b40744b0 msgr2=0x7fd2b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2b40744b0 0x7fd2b40769c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400a830 tx=0x7fd2c4008040 comp rx=0 tx=0).stop 2024-04-25T11:30:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4160be0 msgr2=0x7fd2d40af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4160be0 0x7fd2d40af690 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc007e60 tx=0x7fd2cc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 shutdown_connections 2024-04-25T11:30:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2b40744b0 0x7fd2b40769c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4160be0 0x7fd2d40af690 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2d41434d0 0x7fd2d41f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 --2- 172.21.15.19:0/2989054301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2d40d0ed0 0x7fd2d413e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 >> 172.21.15.19:0/2989054301 conn(0x7fd2d40d1ae0 msgr2=0x7fd2d415ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:18.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 shutdown_connections 2024-04-25T11:30:18.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.244+0000 7fd2db000640 1 -- 172.21.15.19:0/2989054301 wait complete. 2024-04-25T11:30:18.264 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-04-25T11:30:16.505179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230032,"kb_used_data":5528,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744688,"statfs":{"total":773094113280,"available":772858560512,"internally_reserved":0,"allocated":5660672,"data_stored":2998197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":229756466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002160"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.333693+0000","last_change":"2024-04-25T11:30:12.328001+0000","last_active":"2024-04-25T11:30:12.333693+0000","last_peered":"2024-04-25T11:30:12.333693+0000","last_clean":"2024-04-25T11:30:12.333693+0000","last_became_active":"2024-04-25T11:30:12.327555+0000","last_became_peered":"2024-04-25T11:30:12.327555+0000","last_unstale":"2024-04-25T11:30:12.333693+0000","last_undegraded":"2024-04-25T11:30:12.333693+0000","last_fullsized":"2024-04-25T11:30:12.333693+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:03.855581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325206+0000","last_change":"2024-04-25T11:30:12.316817+0000","last_active":"2024-04-25T11:30:12.325206+0000","last_peered":"2024-04-25T11:30:12.325206+0000","last_clean":"2024-04-25T11:30:12.325206+0000","last_became_active":"2024-04-25T11:30:12.311768+0000","last_became_peered":"2024-04-25T11:30:12.311768+0000","last_unstale":"2024-04-25T11:30:12.325206+0000","last_undegraded":"2024-04-25T11:30:12.325206+0000","last_fullsized":"2024-04-25T11:30:12.325206+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.319448+0000","last_change":"2024-04-25T11:30:12.315596+0000","last_active":"2024-04-25T11:30:12.319448+0000","last_peered":"2024-04-25T11:30:12.319448+0000","last_clean":"2024-04-25T11:30:12.319448+0000","last_became_active":"2024-04-25T11:30:12.315259+0000","last_became_peered":"2024-04-25T11:30:12.315259+0000","last_unstale":"2024-04-25T11:30:12.319448+0000","last_undegraded":"2024-04-25T11:30:12.319448+0000","last_fullsized":"2024-04-25T11:30:12.319448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:18.773842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.256254+0000","last_change":"2024-04-25T11:30:11.256254+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.256254+0000","last_undegraded":"2024-04-25T11:30:11.256254+0000","last_fullsized":"2024-04-25T11:30:11.256254+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327312+0000","last_change":"2024-04-25T11:30:12.320809+0000","last_active":"2024-04-25T11:30:12.327312+0000","last_peered":"2024-04-25T11:30:12.327312+0000","last_clean":"2024-04-25T11:30:12.327312+0000","last_became_active":"2024-04-25T11:30:12.318568+0000","last_became_peered":"2024-04-25T11:30:12.318568+0000","last_unstale":"2024-04-25T11:30:12.327312+0000","last_undegraded":"2024-04-25T11:30:12.327312+0000","last_fullsized":"2024-04-25T11:30:12.327312+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324645+0000","last_change":"2024-04-25T11:30:12.316853+0000","last_active":"2024-04-25T11:30:12.324645+0000","last_peered":"2024-04-25T11:30:12.324645+0000","last_clean":"2024-04-25T11:30:12.324645+0000","last_became_active":"2024-04-25T11:30:12.316509+0000","last_became_peered":"2024-04-25T11:30:12.316509+0000","last_unstale":"2024-04-25T11:30:12.324645+0000","last_undegraded":"2024-04-25T11:30:12.324645+0000","last_fullsized":"2024-04-25T11:30:12.324645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.328160+0000","last_change":"2024-04-25T11:30:12.323818+0000","last_active":"2024-04-25T11:30:12.328160+0000","last_peered":"2024-04-25T11:30:12.328160+0000","last_clean":"2024-04-25T11:30:12.328160+0000","last_became_active":"2024-04-25T11:30:12.322940+0000","last_became_peered":"2024-04-25T11:30:12.322940+0000","last_unstale":"2024-04-25T11:30:12.328160+0000","last_undegraded":"2024-04-25T11:30:12.328160+0000","last_fullsized":"2024-04-25T11:30:12.328160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:40.175748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254522+0000","last_change":"2024-04-25T11:30:11.254522+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254522+0000","last_undegraded":"2024-04-25T11:30:11.254522+0000","last_fullsized":"2024-04-25T11:30:11.254522+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.323941+0000","last_change":"2024-04-25T11:30:12.316458+0000","last_active":"2024-04-25T11:30:12.323941+0000","last_peered":"2024-04-25T11:30:12.323941+0000","last_clean":"2024-04-25T11:30:12.323941+0000","last_became_active":"2024-04-25T11:30:12.315188+0000","last_became_peered":"2024-04-25T11:30:12.315188+0000","last_unstale":"2024-04-25T11:30:12.323941+0000","last_undegraded":"2024-04-25T11:30:12.323941+0000","last_fullsized":"2024-04-25T11:30:12.323941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:07.988273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.315531+0000","last_change":"2024-04-25T11:30:12.311133+0000","last_active":"2024-04-25T11:30:12.315531+0000","last_peered":"2024-04-25T11:30:12.315531+0000","last_clean":"2024-04-25T11:30:12.315531+0000","last_became_active":"2024-04-25T11:30:12.310670+0000","last_became_peered":"2024-04-25T11:30:12.310670+0000","last_unstale":"2024-04-25T11:30:12.315531+0000","last_undegraded":"2024-04-25T11:30:12.315531+0000","last_fullsized":"2024-04-25T11:30:12.315531+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.265764+0000","last_change":"2024-04-25T11:30:11.265764+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.265764+0000","last_undegraded":"2024-04-25T11:30:11.265764+0000","last_fullsized":"2024-04-25T11:30:11.265764+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.318340+0000","last_change":"2024-04-25T11:30:12.306558+0000","last_active":"2024-04-25T11:30:12.318340+0000","last_peered":"2024-04-25T11:30:12.318340+0000","last_clean":"2024-04-25T11:30:12.318340+0000","last_became_active":"2024-04-25T11:30:12.306159+0000","last_became_peered":"2024-04-25T11:30:12.306159+0000","last_unstale":"2024-04-25T11:30:12.318340+0000","last_undegraded":"2024-04-25T11:30:12.318340+0000","last_fullsized":"2024-04-25T11:30:12.318340+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:08.096663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.323321+0000","last_change":"2024-04-25T11:30:12.315317+0000","last_active":"2024-04-25T11:30:12.323321+0000","last_peered":"2024-04-25T11:30:12.323321+0000","last_clean":"2024-04-25T11:30:12.323321+0000","last_became_active":"2024-04-25T11:30:12.314919+0000","last_became_peered":"2024-04-25T11:30:12.314919+0000","last_unstale":"2024-04-25T11:30:12.323321+0000","last_undegraded":"2024-04-25T11:30:12.323321+0000","last_fullsized":"2024-04-25T11:30:12.323321+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:32.390065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.253292+0000","last_change":"2024-04-25T11:30:11.253292+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.253292+0000","last_undegraded":"2024-04-25T11:30:11.253292+0000","last_fullsized":"2024-04-25T11:30:11.253292+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.252332+0000","last_change":"2024-04-25T11:30:11.252332+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.252332+0000","last_undegraded":"2024-04-25T11:30:11.252332+0000","last_fullsized":"2024-04-25T11:30:11.252332+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327221+0000","last_change":"2024-04-25T11:30:12.320656+0000","last_active":"2024-04-25T11:30:12.327221+0000","last_peered":"2024-04-25T11:30:12.327221+0000","last_clean":"2024-04-25T11:30:12.327221+0000","last_became_active":"2024-04-25T11:30:12.319501+0000","last_became_peered":"2024-04-25T11:30:12.319501+0000","last_unstale":"2024-04-25T11:30:12.327221+0000","last_undegraded":"2024-04-25T11:30:12.327221+0000","last_fullsized":"2024-04-25T11:30:12.327221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324268+0000","last_change":"2024-04-25T11:30:12.312204+0000","last_active":"2024-04-25T11:30:12.324268+0000","last_peered":"2024-04-25T11:30:12.324268+0000","last_clean":"2024-04-25T11:30:12.324268+0000","last_became_active":"2024-04-25T11:30:12.311798+0000","last_became_peered":"2024-04-25T11:30:12.311798+0000","last_unstale":"2024-04-25T11:30:12.324268+0000","last_undegraded":"2024-04-25T11:30:12.324268+0000","last_fullsized":"2024-04-25T11:30:12.324268+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.296840+0000","last_change":"2024-04-25T11:30:12.292655+0000","last_active":"2024-04-25T11:30:12.296840+0000","last_peered":"2024-04-25T11:30:12.296840+0000","last_clean":"2024-04-25T11:30:12.296840+0000","last_became_active":"2024-04-25T11:30:12.292173+0000","last_became_peered":"2024-04-25T11:30:12.292173+0000","last_unstale":"2024-04-25T11:30:12.296840+0000","last_undegraded":"2024-04-25T11:30:12.296840+0000","last_fullsized":"2024-04-25T11:30:12.296840+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:40.609036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327149+0000","last_change":"2024-04-25T11:30:12.318490+0000","last_active":"2024-04-25T11:30:12.327149+0000","last_peered":"2024-04-25T11:30:12.327149+0000","last_clean":"2024-04-25T11:30:12.327149+0000","last_became_active":"2024-04-25T11:30:12.317442+0000","last_became_peered":"2024-04-25T11:30:12.317442+0000","last_unstale":"2024-04-25T11:30:12.327149+0000","last_undegraded":"2024-04-25T11:30:12.327149+0000","last_fullsized":"2024-04-25T11:30:12.327149+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.315731+0000","last_change":"2024-04-25T11:30:12.311311+0000","last_active":"2024-04-25T11:30:12.315731+0000","last_peered":"2024-04-25T11:30:12.315731+0000","last_clean":"2024-04-25T11:30:12.315731+0000","last_became_active":"2024-04-25T11:30:12.310846+0000","last_became_peered":"2024-04-25T11:30:12.310846+0000","last_unstale":"2024-04-25T11:30:12.315731+0000","last_undegraded":"2024-04-25T11:30:12.315731+0000","last_fullsized":"2024-04-25T11:30:12.315731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.314933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254710+0000","last_change":"2024-04-25T11:30:11.254710+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254710+0000","last_undegraded":"2024-04-25T11:30:11.254710+0000","last_fullsized":"2024-04-25T11:30:11.254710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324559+0000","last_change":"2024-04-25T11:30:12.318465+0000","last_active":"2024-04-25T11:30:12.324559+0000","last_peered":"2024-04-25T11:30:12.324559+0000","last_clean":"2024-04-25T11:30:12.324559+0000","last_became_active":"2024-04-25T11:30:12.318076+0000","last_became_peered":"2024-04-25T11:30:12.318076+0000","last_unstale":"2024-04-25T11:30:12.324559+0000","last_undegraded":"2024-04-25T11:30:12.324559+0000","last_fullsized":"2024-04-25T11:30:12.324559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:21.763769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.313691+0000","last_change":"2024-04-25T11:30:12.308548+0000","last_active":"2024-04-25T11:30:12.313691+0000","last_peered":"2024-04-25T11:30:12.313691+0000","last_clean":"2024-04-25T11:30:12.313691+0000","last_became_active":"2024-04-25T11:30:12.308148+0000","last_became_peered":"2024-04-25T11:30:12.308148+0000","last_unstale":"2024-04-25T11:30:12.313691+0000","last_undegraded":"2024-04-25T11:30:12.313691+0000","last_fullsized":"2024-04-25T11:30:12.313691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:57.673678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324424+0000","last_change":"2024-04-25T11:30:12.312781+0000","last_active":"2024-04-25T11:30:12.324424+0000","last_peered":"2024-04-25T11:30:12.324424+0000","last_clean":"2024-04-25T11:30:12.324424+0000","last_became_active":"2024-04-25T11:30:12.312453+0000","last_became_peered":"2024-04-25T11:30:12.312453+0000","last_unstale":"2024-04-25T11:30:12.324424+0000","last_undegraded":"2024-04-25T11:30:12.324424+0000","last_fullsized":"2024-04-25T11:30:12.324424+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.328444+0000","last_change":"2024-04-25T11:30:12.325014+0000","last_active":"2024-04-25T11:30:12.328444+0000","last_peered":"2024-04-25T11:30:12.328444+0000","last_clean":"2024-04-25T11:30:12.328444+0000","last_became_active":"2024-04-25T11:30:12.324660+0000","last_became_peered":"2024-04-25T11:30:12.324660+0000","last_unstale":"2024-04-25T11:30:12.328444+0000","last_undegraded":"2024-04-25T11:30:12.328444+0000","last_fullsized":"2024-04-25T11:30:12.328444+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.265187+0000","last_change":"2024-04-25T11:30:11.265187+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.265187+0000","last_undegraded":"2024-04-25T11:30:11.265187+0000","last_fullsized":"2024-04-25T11:30:11.265187+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304485+0000","last_change":"2024-04-25T11:30:12.300829+0000","last_active":"2024-04-25T11:30:12.304485+0000","last_peered":"2024-04-25T11:30:12.304485+0000","last_clean":"2024-04-25T11:30:12.304485+0000","last_became_active":"2024-04-25T11:30:12.300457+0000","last_became_peered":"2024-04-25T11:30:12.300457+0000","last_unstale":"2024-04-25T11:30:12.304485+0000","last_undegraded":"2024-04-25T11:30:12.304485+0000","last_fullsized":"2024-04-25T11:30:12.304485+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:29.722464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.309487+0000","last_change":"2024-04-25T11:30:12.306296+0000","last_active":"2024-04-25T11:30:12.309487+0000","last_peered":"2024-04-25T11:30:12.309487+0000","last_clean":"2024-04-25T11:30:12.309487+0000","last_became_active":"2024-04-25T11:30:12.305922+0000","last_became_peered":"2024-04-25T11:30:12.305922+0000","last_unstale":"2024-04-25T11:30:12.309487+0000","last_undegraded":"2024-04-25T11:30:12.309487+0000","last_fullsized":"2024-04-25T11:30:12.309487+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:33.370286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.320671+0000","last_change":"2024-04-25T11:30:12.314344+0000","last_active":"2024-04-25T11:30:12.320671+0000","last_peered":"2024-04-25T11:30:12.320671+0000","last_clean":"2024-04-25T11:30:12.320671+0000","last_became_active":"2024-04-25T11:30:12.314033+0000","last_became_peered":"2024-04-25T11:30:12.314033+0000","last_unstale":"2024-04-25T11:30:12.320671+0000","last_undegraded":"2024-04-25T11:30:12.320671+0000","last_fullsized":"2024-04-25T11:30:12.320671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:10.049040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.316322+0000","last_change":"2024-04-25T11:30:12.306312+0000","last_active":"2024-04-25T11:30:12.316322+0000","last_peered":"2024-04-25T11:30:12.316322+0000","last_clean":"2024-04-25T11:30:12.316322+0000","last_became_active":"2024-04-25T11:30:12.305928+0000","last_became_peered":"2024-04-25T11:30:12.305928+0000","last_unstale":"2024-04-25T11:30:12.316322+0000","last_undegraded":"2024-04-25T11:30:12.316322+0000","last_fullsized":"2024-04-25T11:30:12.316322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:07.988273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.322258+0000","last_change":"2024-04-25T11:30:12.309122+0000","last_active":"2024-04-25T11:30:12.322258+0000","last_peered":"2024-04-25T11:30:12.322258+0000","last_clean":"2024-04-25T11:30:12.322258+0000","last_became_active":"2024-04-25T11:30:12.308689+0000","last_became_peered":"2024-04-25T11:30:12.308689+0000","last_unstale":"2024-04-25T11:30:12.322258+0000","last_undegraded":"2024-04-25T11:30:12.322258+0000","last_fullsized":"2024-04-25T11:30:12.322258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:10.291351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.252849+0000","last_change":"2024-04-25T11:30:11.252849+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.252849+0000","last_undegraded":"2024-04-25T11:30:11.252849+0000","last_fullsized":"2024-04-25T11:30:11.252849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325542+0000","last_change":"2024-04-25T11:30:12.324170+0000","last_active":"2024-04-25T11:30:12.325542+0000","last_peered":"2024-04-25T11:30:12.325542+0000","last_clean":"2024-04-25T11:30:12.325542+0000","last_became_active":"2024-04-25T11:30:12.323509+0000","last_became_peered":"2024-04-25T11:30:12.323509+0000","last_unstale":"2024-04-25T11:30:12.325542+0000","last_undegraded":"2024-04-25T11:30:12.325542+0000","last_fullsized":"2024-04-25T11:30:12.325542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:01.898568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338824+0000","last_change":"2024-04-25T11:30:11.269060+0000","last_active":"2024-04-25T11:30:12.338824+0000","last_peered":"2024-04-25T11:30:12.338824+0000","last_clean":"2024-04-25T11:30:12.338824+0000","last_became_active":"2024-04-25T11:30:11.266286+0000","last_became_peered":"2024-04-25T11:30:11.266286+0000","last_unstale":"2024-04-25T11:30:12.338824+0000","last_undegraded":"2024-04-25T11:30:12.338824+0000","last_fullsized":"2024-04-25T11:30:12.338824+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:25.474525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327322+0000","last_change":"2024-04-25T11:30:12.319935+0000","last_active":"2024-04-25T11:30:12.327322+0000","last_peered":"2024-04-25T11:30:12.327322+0000","last_clean":"2024-04-25T11:30:12.327322+0000","last_became_active":"2024-04-25T11:30:12.319486+0000","last_became_peered":"2024-04-25T11:30:12.319486+0000","last_unstale":"2024-04-25T11:30:12.327322+0000","last_undegraded":"2024-04-25T11:30:12.327322+0000","last_fullsized":"2024-04-25T11:30:12.327322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.339577+0000","last_change":"2024-04-25T11:30:11.257450+0000","last_active":"2024-04-25T11:30:12.339577+0000","last_peered":"2024-04-25T11:30:12.339577+0000","last_clean":"2024-04-25T11:30:12.339577+0000","last_became_active":"2024-04-25T11:30:11.257097+0000","last_became_peered":"2024-04-25T11:30:11.257097+0000","last_unstale":"2024-04-25T11:30:12.339577+0000","last_undegraded":"2024-04-25T11:30:12.339577+0000","last_fullsized":"2024-04-25T11:30:12.339577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.501573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325736+0000","last_change":"2024-04-25T11:30:12.318193+0000","last_active":"2024-04-25T11:30:12.325736+0000","last_peered":"2024-04-25T11:30:12.325736+0000","last_clean":"2024-04-25T11:30:12.325736+0000","last_became_active":"2024-04-25T11:30:12.317611+0000","last_became_peered":"2024-04-25T11:30:12.317611+0000","last_unstale":"2024-04-25T11:30:12.325736+0000","last_undegraded":"2024-04-25T11:30:12.325736+0000","last_fullsized":"2024-04-25T11:30:12.325736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.334754+0000","last_change":"2024-04-25T11:30:11.271679+0000","last_active":"2024-04-25T11:30:12.334754+0000","last_peered":"2024-04-25T11:30:12.334754+0000","last_clean":"2024-04-25T11:30:12.334754+0000","last_became_active":"2024-04-25T11:30:11.270812+0000","last_became_peered":"2024-04-25T11:30:11.270812+0000","last_unstale":"2024-04-25T11:30:12.334754+0000","last_undegraded":"2024-04-25T11:30:12.334754+0000","last_fullsized":"2024-04-25T11:30:12.334754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:01.243237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.261566+0000","last_change":"2024-04-25T11:30:11.261566+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.261566+0000","last_undegraded":"2024-04-25T11:30:11.261566+0000","last_fullsized":"2024-04-25T11:30:11.261566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.339891+0000","last_change":"2024-04-25T11:30:11.259259+0000","last_active":"2024-04-25T11:30:12.339891+0000","last_peered":"2024-04-25T11:30:12.339891+0000","last_clean":"2024-04-25T11:30:12.339891+0000","last_became_active":"2024-04-25T11:30:11.258919+0000","last_became_peered":"2024-04-25T11:30:11.258919+0000","last_unstale":"2024-04-25T11:30:12.339891+0000","last_undegraded":"2024-04-25T11:30:12.339891+0000","last_fullsized":"2024-04-25T11:30:12.339891+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:49.077411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.320288+0000","last_change":"2024-04-25T11:30:12.310915+0000","last_active":"2024-04-25T11:30:12.320288+0000","last_peered":"2024-04-25T11:30:12.320288+0000","last_clean":"2024-04-25T11:30:12.320288+0000","last_became_active":"2024-04-25T11:30:12.310580+0000","last_became_peered":"2024-04-25T11:30:12.310580+0000","last_unstale":"2024-04-25T11:30:12.320288+0000","last_undegraded":"2024-04-25T11:30:12.320288+0000","last_fullsized":"2024-04-25T11:30:12.320288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:26.486680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338450+0000","last_change":"2024-04-25T11:30:11.278985+0000","last_active":"2024-04-25T11:30:12.338450+0000","last_peered":"2024-04-25T11:30:12.338450+0000","last_clean":"2024-04-25T11:30:12.338450+0000","last_became_active":"2024-04-25T11:30:11.273156+0000","last_became_peered":"2024-04-25T11:30:11.273156+0000","last_unstale":"2024-04-25T11:30:12.338450+0000","last_undegraded":"2024-04-25T11:30:12.338450+0000","last_fullsized":"2024-04-25T11:30:12.338450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:37.174642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325053+0000","last_change":"2024-04-25T11:30:12.318436+0000","last_active":"2024-04-25T11:30:12.325053+0000","last_peered":"2024-04-25T11:30:12.325053+0000","last_clean":"2024-04-25T11:30:12.325053+0000","last_became_active":"2024-04-25T11:30:12.318108+0000","last_became_peered":"2024-04-25T11:30:12.318108+0000","last_unstale":"2024-04-25T11:30:12.325053+0000","last_undegraded":"2024-04-25T11:30:12.325053+0000","last_fullsized":"2024-04-25T11:30:12.325053+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338843+0000","last_change":"2024-04-25T11:30:11.263150+0000","last_active":"2024-04-25T11:30:12.338843+0000","last_peered":"2024-04-25T11:30:12.338843+0000","last_clean":"2024-04-25T11:30:12.338843+0000","last_became_active":"2024-04-25T11:30:11.262389+0000","last_became_peered":"2024-04-25T11:30:11.262389+0000","last_unstale":"2024-04-25T11:30:12.338843+0000","last_undegraded":"2024-04-25T11:30:12.338843+0000","last_fullsized":"2024-04-25T11:30:12.338843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:44.476276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321591+0000","last_change":"2024-04-25T11:30:12.313164+0000","last_active":"2024-04-25T11:30:12.321591+0000","last_peered":"2024-04-25T11:30:12.321591+0000","last_clean":"2024-04-25T11:30:12.321591+0000","last_became_active":"2024-04-25T11:30:12.311004+0000","last_became_peered":"2024-04-25T11:30:12.311004+0000","last_unstale":"2024-04-25T11:30:12.321591+0000","last_undegraded":"2024-04-25T11:30:12.321591+0000","last_fullsized":"2024-04-25T11:30:12.321591+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:32.067456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.303499+0000","last_change":"2024-04-25T11:30:11.272497+0000","last_active":"2024-04-25T11:30:12.303499+0000","last_peered":"2024-04-25T11:30:12.303499+0000","last_clean":"2024-04-25T11:30:12.303499+0000","last_became_active":"2024-04-25T11:30:11.272150+0000","last_became_peered":"2024-04-25T11:30:11.272150+0000","last_unstale":"2024-04-25T11:30:12.303499+0000","last_undegraded":"2024-04-25T11:30:12.303499+0000","last_fullsized":"2024-04-25T11:30:12.303499+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:09.285233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.329154+0000","last_change":"2024-04-25T11:30:12.326351+0000","last_active":"2024-04-25T11:30:12.329154+0000","last_peered":"2024-04-25T11:30:12.329154+0000","last_clean":"2024-04-25T11:30:12.329154+0000","last_became_active":"2024-04-25T11:30:12.326055+0000","last_became_peered":"2024-04-25T11:30:12.326055+0000","last_unstale":"2024-04-25T11:30:12.329154+0000","last_undegraded":"2024-04-25T11:30:12.329154+0000","last_fullsized":"2024-04-25T11:30:12.329154+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:26.804870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304084+0000","last_change":"2024-04-25T11:30:11.280682+0000","last_active":"2024-04-25T11:30:12.304084+0000","last_peered":"2024-04-25T11:30:12.304084+0000","last_clean":"2024-04-25T11:30:12.304084+0000","last_became_active":"2024-04-25T11:30:11.279049+0000","last_became_peered":"2024-04-25T11:30:11.279049+0000","last_unstale":"2024-04-25T11:30:12.304084+0000","last_undegraded":"2024-04-25T11:30:12.304084+0000","last_fullsized":"2024-04-25T11:30:12.304084+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:40.524169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.316369+0000","last_change":"2024-04-25T11:30:12.307371+0000","last_active":"2024-04-25T11:30:12.316369+0000","last_peered":"2024-04-25T11:30:12.316369+0000","last_clean":"2024-04-25T11:30:12.316369+0000","last_became_active":"2024-04-25T11:30:12.307030+0000","last_became_peered":"2024-04-25T11:30:12.307030+0000","last_unstale":"2024-04-25T11:30:12.316369+0000","last_undegraded":"2024-04-25T11:30:12.316369+0000","last_fullsized":"2024-04-25T11:30:12.316369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.144352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.281829+0000","last_change":"2024-04-25T11:30:11.267439+0000","last_active":"2024-04-25T11:30:12.281829+0000","last_peered":"2024-04-25T11:30:12.281829+0000","last_clean":"2024-04-25T11:30:12.281829+0000","last_became_active":"2024-04-25T11:30:11.267110+0000","last_became_peered":"2024-04-25T11:30:11.267110+0000","last_unstale":"2024-04-25T11:30:12.281829+0000","last_undegraded":"2024-04-25T11:30:12.281829+0000","last_fullsized":"2024-04-25T11:30:12.281829+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:30.677292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.318253+0000","last_change":"2024-04-25T11:30:12.311247+0000","last_active":"2024-04-25T11:30:12.318253+0000","last_peered":"2024-04-25T11:30:12.318253+0000","last_clean":"2024-04-25T11:30:12.318253+0000","last_became_active":"2024-04-25T11:30:12.310904+0000","last_became_peered":"2024-04-25T11:30:12.310904+0000","last_unstale":"2024-04-25T11:30:12.318253+0000","last_undegraded":"2024-04-25T11:30:12.318253+0000","last_fullsized":"2024-04-25T11:30:12.318253+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:23.098668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304732+0000","last_change":"2024-04-25T11:30:11.272789+0000","last_active":"2024-04-25T11:30:12.304732+0000","last_peered":"2024-04-25T11:30:12.304732+0000","last_clean":"2024-04-25T11:30:12.304732+0000","last_became_active":"2024-04-25T11:30:11.272445+0000","last_became_peered":"2024-04-25T11:30:11.272445+0000","last_unstale":"2024-04-25T11:30:12.304732+0000","last_undegraded":"2024-04-25T11:30:12.304732+0000","last_fullsized":"2024-04-25T11:30:12.304732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.849463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.254783+0000","last_change":"2024-04-25T11:30:11.254783+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.254783+0000","last_undegraded":"2024-04-25T11:30:11.254783+0000","last_fullsized":"2024-04-25T11:30:11.254783+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.338671+0000","last_change":"2024-04-25T11:30:11.264837+0000","last_active":"2024-04-25T11:30:12.338671+0000","last_peered":"2024-04-25T11:30:12.338671+0000","last_clean":"2024-04-25T11:30:12.338671+0000","last_became_active":"2024-04-25T11:30:11.264166+0000","last_became_peered":"2024-04-25T11:30:11.264166+0000","last_unstale":"2024-04-25T11:30:12.338671+0000","last_undegraded":"2024-04-25T11:30:12.338671+0000","last_fullsized":"2024-04-25T11:30:12.338671+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327792+0000","last_change":"2024-04-25T11:30:12.315841+0000","last_active":"2024-04-25T11:30:12.327792+0000","last_peered":"2024-04-25T11:30:12.327792+0000","last_clean":"2024-04-25T11:30:12.327792+0000","last_became_active":"2024-04-25T11:30:12.315087+0000","last_became_peered":"2024-04-25T11:30:12.315087+0000","last_unstale":"2024-04-25T11:30:12.327792+0000","last_undegraded":"2024-04-25T11:30:12.327792+0000","last_fullsized":"2024-04-25T11:30:12.327792+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:30.242892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.341752+0000","last_change":"2024-04-25T11:30:11.273982+0000","last_active":"2024-04-25T11:30:12.341752+0000","last_peered":"2024-04-25T11:30:12.341752+0000","last_clean":"2024-04-25T11:30:12.341752+0000","last_became_active":"2024-04-25T11:30:11.272233+0000","last_became_peered":"2024-04-25T11:30:11.272233+0000","last_unstale":"2024-04-25T11:30:12.341752+0000","last_undegraded":"2024-04-25T11:30:12.341752+0000","last_fullsized":"2024-04-25T11:30:12.341752+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:34.401573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.296892+0000","last_change":"2024-04-25T11:30:12.294967+0000","last_active":"2024-04-25T11:30:12.296892+0000","last_peered":"2024-04-25T11:30:12.296892+0000","last_clean":"2024-04-25T11:30:12.296892+0000","last_became_active":"2024-04-25T11:30:12.294545+0000","last_became_peered":"2024-04-25T11:30:12.294545+0000","last_unstale":"2024-04-25T11:30:12.296892+0000","last_undegraded":"2024-04-25T11:30:12.296892+0000","last_fullsized":"2024-04-25T11:30:12.296892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:17.394404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.334053+0000","last_change":"2024-04-25T11:30:11.278510+0000","last_active":"2024-04-25T11:30:12.334053+0000","last_peered":"2024-04-25T11:30:12.334053+0000","last_clean":"2024-04-25T11:30:12.334053+0000","last_became_active":"2024-04-25T11:30:11.278136+0000","last_became_peered":"2024-04-25T11:30:11.278136+0000","last_unstale":"2024-04-25T11:30:12.334053+0000","last_undegraded":"2024-04-25T11:30:12.334053+0000","last_fullsized":"2024-04-25T11:30:12.334053+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:44.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327613+0000","last_change":"2024-04-25T11:30:12.326783+0000","last_active":"2024-04-25T11:30:12.327613+0000","last_peered":"2024-04-25T11:30:12.327613+0000","last_clean":"2024-04-25T11:30:12.327613+0000","last_became_active":"2024-04-25T11:30:12.325755+0000","last_became_peered":"2024-04-25T11:30:12.325755+0000","last_unstale":"2024-04-25T11:30:12.327613+0000","last_undegraded":"2024-04-25T11:30:12.327613+0000","last_fullsized":"2024-04-25T11:30:12.327613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:20.342509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304553+0000","last_change":"2024-04-25T11:30:11.269357+0000","last_active":"2024-04-25T11:30:12.304553+0000","last_peered":"2024-04-25T11:30:12.304553+0000","last_clean":"2024-04-25T11:30:12.304553+0000","last_became_active":"2024-04-25T11:30:11.268999+0000","last_became_peered":"2024-04-25T11:30:11.268999+0000","last_unstale":"2024-04-25T11:30:12.304553+0000","last_undegraded":"2024-04-25T11:30:12.304553+0000","last_fullsized":"2024-04-25T11:30:12.304553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321284+0000","last_change":"2024-04-25T11:30:12.316130+0000","last_active":"2024-04-25T11:30:12.321284+0000","last_peered":"2024-04-25T11:30:12.321284+0000","last_clean":"2024-04-25T11:30:12.321284+0000","last_became_active":"2024-04-25T11:30:12.315793+0000","last_became_peered":"2024-04-25T11:30:12.315793+0000","last_unstale":"2024-04-25T11:30:12.321284+0000","last_undegraded":"2024-04-25T11:30:12.321284+0000","last_fullsized":"2024-04-25T11:30:12.321284+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:23.098668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.300732+0000","last_change":"2024-04-25T11:30:11.279239+0000","last_active":"2024-04-25T11:30:12.300732+0000","last_peered":"2024-04-25T11:30:12.300732+0000","last_clean":"2024-04-25T11:30:12.300732+0000","last_became_active":"2024-04-25T11:30:11.278889+0000","last_became_peered":"2024-04-25T11:30:11.278889+0000","last_unstale":"2024-04-25T11:30:12.300732+0000","last_undegraded":"2024-04-25T11:30:12.300732+0000","last_fullsized":"2024-04-25T11:30:12.300732+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:12.914763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.324849+0000","last_change":"2024-04-25T11:30:12.306734+0000","last_active":"2024-04-25T11:30:12.324849+0000","last_peered":"2024-04-25T11:30:12.324849+0000","last_clean":"2024-04-25T11:30:12.324849+0000","last_became_active":"2024-04-25T11:30:12.306242+0000","last_became_peered":"2024-04-25T11:30:12.306242+0000","last_unstale":"2024-04-25T11:30:12.324849+0000","last_undegraded":"2024-04-25T11:30:12.324849+0000","last_fullsized":"2024-04-25T11:30:12.324849+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:10.049040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.340720+0000","last_change":"2024-04-25T11:30:11.274461+0000","last_active":"2024-04-25T11:30:12.340720+0000","last_peered":"2024-04-25T11:30:12.340720+0000","last_clean":"2024-04-25T11:30:12.340720+0000","last_became_active":"2024-04-25T11:30:11.272811+0000","last_became_peered":"2024-04-25T11:30:11.272811+0000","last_unstale":"2024-04-25T11:30:12.340720+0000","last_undegraded":"2024-04-25T11:30:12.340720+0000","last_fullsized":"2024-04-25T11:30:12.340720+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:01.676380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325409+0000","last_change":"2024-04-25T11:30:12.321405+0000","last_active":"2024-04-25T11:30:12.325409+0000","last_peered":"2024-04-25T11:30:12.325409+0000","last_clean":"2024-04-25T11:30:12.325409+0000","last_became_active":"2024-04-25T11:30:12.319648+0000","last_became_peered":"2024-04-25T11:30:12.319648+0000","last_unstale":"2024-04-25T11:30:12.325409+0000","last_undegraded":"2024-04-25T11:30:12.325409+0000","last_fullsized":"2024-04-25T11:30:12.325409+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:18.773842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T11:30:11.283123+0000","last_change":"2024-04-25T11:30:11.281057+0000","last_active":"2024-04-25T11:30:11.283123+0000","last_peered":"2024-04-25T11:30:11.283123+0000","last_clean":"2024-04-25T11:30:11.283123+0000","last_became_active":"2024-04-25T11:30:11.279339+0000","last_became_peered":"2024-04-25T11:30:11.279339+0000","last_unstale":"2024-04-25T11:30:11.283123+0000","last_undegraded":"2024-04-25T11:30:11.283123+0000","last_fullsized":"2024-04-25T11:30:11.283123+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:18.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321209+0000","last_change":"2024-04-25T11:30:12.315868+0000","last_active":"2024-04-25T11:30:12.321209+0000","last_peered":"2024-04-25T11:30:12.321209+0000","last_clean":"2024-04-25T11:30:12.321209+0000","last_became_active":"2024-04-25T11:30:12.315511+0000","last_became_peered":"2024-04-25T11:30:12.315511+0000","last_unstale":"2024-04-25T11:30:12.321209+0000","last_undegraded":"2024-04-25T11:30:12.321209+0000","last_fullsized":"2024-04-25T11:30:12.321209+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.144352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327577+0000","last_change":"2024-04-25T11:30:11.271207+0000","last_active":"2024-04-25T11:30:12.327577+0000","last_peered":"2024-04-25T11:30:12.327577+0000","last_clean":"2024-04-25T11:30:12.327577+0000","last_became_active":"2024-04-25T11:30:11.270083+0000","last_became_peered":"2024-04-25T11:30:11.270083+0000","last_unstale":"2024-04-25T11:30:12.327577+0000","last_undegraded":"2024-04-25T11:30:12.327577+0000","last_fullsized":"2024-04-25T11:30:12.327577+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:40.274271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.325710+0000","last_change":"2024-04-25T11:30:12.317980+0000","last_active":"2024-04-25T11:30:12.325710+0000","last_peered":"2024-04-25T11:30:12.325710+0000","last_clean":"2024-04-25T11:30:12.325710+0000","last_became_active":"2024-04-25T11:30:12.317574+0000","last_became_peered":"2024-04-25T11:30:12.317574+0000","last_unstale":"2024-04-25T11:30:12.325710+0000","last_undegraded":"2024-04-25T11:30:12.325710+0000","last_fullsized":"2024-04-25T11:30:12.325710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327952+0000","last_change":"2024-04-25T11:30:11.276120+0000","last_active":"2024-04-25T11:30:12.327952+0000","last_peered":"2024-04-25T11:30:12.327952+0000","last_clean":"2024-04-25T11:30:12.327952+0000","last_became_active":"2024-04-25T11:30:11.275804+0000","last_became_peered":"2024-04-25T11:30:11.275804+0000","last_unstale":"2024-04-25T11:30:12.327952+0000","last_undegraded":"2024-04-25T11:30:12.327952+0000","last_fullsized":"2024-04-25T11:30:12.327952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:37.492766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.319419+0000","last_change":"2024-04-25T11:30:12.315454+0000","last_active":"2024-04-25T11:30:12.319419+0000","last_peered":"2024-04-25T11:30:12.319419+0000","last_clean":"2024-04-25T11:30:12.319419+0000","last_became_active":"2024-04-25T11:30:12.315114+0000","last_became_peered":"2024-04-25T11:30:12.315114+0000","last_unstale":"2024-04-25T11:30:12.319419+0000","last_undegraded":"2024-04-25T11:30:12.319419+0000","last_fullsized":"2024-04-25T11:30:12.319419+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:21.763769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304284+0000","last_change":"2024-04-25T11:30:11.279419+0000","last_active":"2024-04-25T11:30:12.304284+0000","last_peered":"2024-04-25T11:30:12.304284+0000","last_clean":"2024-04-25T11:30:12.304284+0000","last_became_active":"2024-04-25T11:30:11.279074+0000","last_became_peered":"2024-04-25T11:30:11.279074+0000","last_unstale":"2024-04-25T11:30:12.304284+0000","last_undegraded":"2024-04-25T11:30:12.304284+0000","last_fullsized":"2024-04-25T11:30:12.304284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:27.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.326110+0000","last_change":"2024-04-25T11:30:12.320762+0000","last_active":"2024-04-25T11:30:12.326110+0000","last_peered":"2024-04-25T11:30:12.326110+0000","last_clean":"2024-04-25T11:30:12.326110+0000","last_became_active":"2024-04-25T11:30:12.320207+0000","last_became_peered":"2024-04-25T11:30:12.320207+0000","last_unstale":"2024-04-25T11:30:12.326110+0000","last_undegraded":"2024-04-25T11:30:12.326110+0000","last_fullsized":"2024-04-25T11:30:12.326110+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:18.987772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.335531+0000","last_change":"2024-04-25T11:30:11.273562+0000","last_active":"2024-04-25T11:30:12.335531+0000","last_peered":"2024-04-25T11:30:12.335531+0000","last_clean":"2024-04-25T11:30:12.335531+0000","last_became_active":"2024-04-25T11:30:11.273187+0000","last_became_peered":"2024-04-25T11:30:11.273187+0000","last_unstale":"2024-04-25T11:30:12.335531+0000","last_undegraded":"2024-04-25T11:30:12.335531+0000","last_fullsized":"2024-04-25T11:30:12.335531+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:37.187391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.322121+0000","last_change":"2024-04-25T11:30:12.310068+0000","last_active":"2024-04-25T11:30:12.322121+0000","last_peered":"2024-04-25T11:30:12.322121+0000","last_clean":"2024-04-25T11:30:12.322121+0000","last_became_active":"2024-04-25T11:30:12.309236+0000","last_became_peered":"2024-04-25T11:30:12.309236+0000","last_unstale":"2024-04-25T11:30:12.322121+0000","last_undegraded":"2024-04-25T11:30:12.322121+0000","last_fullsized":"2024-04-25T11:30:12.322121+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.510835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305149+0000","last_change":"2024-04-25T11:30:11.260102+0000","last_active":"2024-04-25T11:30:12.305149+0000","last_peered":"2024-04-25T11:30:12.305149+0000","last_clean":"2024-04-25T11:30:12.305149+0000","last_became_active":"2024-04-25T11:30:11.259712+0000","last_became_peered":"2024-04-25T11:30:11.259712+0000","last_unstale":"2024-04-25T11:30:12.305149+0000","last_undegraded":"2024-04-25T11:30:12.305149+0000","last_fullsized":"2024-04-25T11:30:12.305149+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:31.383947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.251703+0000","last_change":"2024-04-25T11:30:11.251703+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.251703+0000","last_undegraded":"2024-04-25T11:30:11.251703+0000","last_fullsized":"2024-04-25T11:30:11.251703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304576+0000","last_change":"2024-04-25T11:30:11.274700+0000","last_active":"2024-04-25T11:30:12.304576+0000","last_peered":"2024-04-25T11:30:12.304576+0000","last_clean":"2024-04-25T11:30:12.304576+0000","last_became_active":"2024-04-25T11:30:11.274283+0000","last_became_peered":"2024-04-25T11:30:11.274283+0000","last_unstale":"2024-04-25T11:30:12.304576+0000","last_undegraded":"2024-04-25T11:30:12.304576+0000","last_fullsized":"2024-04-25T11:30:12.304576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.081180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.250705+0000","last_change":"2024-04-25T11:30:11.250705+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.250705+0000","last_undegraded":"2024-04-25T11:30:11.250705+0000","last_fullsized":"2024-04-25T11:30:11.250705+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.297476+0000","last_change":"2024-04-25T11:30:11.266163+0000","last_active":"2024-04-25T11:30:12.297476+0000","last_peered":"2024-04-25T11:30:12.297476+0000","last_clean":"2024-04-25T11:30:12.297476+0000","last_became_active":"2024-04-25T11:30:11.265810+0000","last_became_peered":"2024-04-25T11:30:11.265810+0000","last_unstale":"2024-04-25T11:30:12.297476+0000","last_undegraded":"2024-04-25T11:30:12.297476+0000","last_fullsized":"2024-04-25T11:30:12.297476+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:20.553784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.329132+0000","last_change":"2024-04-25T11:30:12.325830+0000","last_active":"2024-04-25T11:30:12.329132+0000","last_peered":"2024-04-25T11:30:12.329132+0000","last_clean":"2024-04-25T11:30:12.329132+0000","last_became_active":"2024-04-25T11:30:12.325485+0000","last_became_peered":"2024-04-25T11:30:12.325485+0000","last_unstale":"2024-04-25T11:30:12.329132+0000","last_undegraded":"2024-04-25T11:30:12.329132+0000","last_fullsized":"2024-04-25T11:30:12.329132+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T11:30:11.276720+0000","last_change":"2024-04-25T11:30:11.274375+0000","last_active":"2024-04-25T11:30:11.276720+0000","last_peered":"2024-04-25T11:30:11.276720+0000","last_clean":"2024-04-25T11:30:11.276720+0000","last_became_active":"2024-04-25T11:30:11.273757+0000","last_became_peered":"2024-04-25T11:30:11.273757+0000","last_unstale":"2024-04-25T11:30:11.276720+0000","last_undegraded":"2024-04-25T11:30:11.276720+0000","last_fullsized":"2024-04-25T11:30:11.276720+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:43.406052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.321025+0000","last_change":"2024-04-25T11:30:12.314257+0000","last_active":"2024-04-25T11:30:12.321025+0000","last_peered":"2024-04-25T11:30:12.321025+0000","last_clean":"2024-04-25T11:30:12.321025+0000","last_became_active":"2024-04-25T11:30:12.313948+0000","last_became_peered":"2024-04-25T11:30:12.313948+0000","last_unstale":"2024-04-25T11:30:12.321025+0000","last_undegraded":"2024-04-25T11:30:12.321025+0000","last_fullsized":"2024-04-25T11:30:12.321025+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305887+0000","last_change":"2024-04-25T11:30:11.263543+0000","last_active":"2024-04-25T11:30:12.305887+0000","last_peered":"2024-04-25T11:30:12.305887+0000","last_clean":"2024-04-25T11:30:12.305887+0000","last_became_active":"2024-04-25T11:30:11.263181+0000","last_became_peered":"2024-04-25T11:30:11.263181+0000","last_unstale":"2024-04-25T11:30:12.305887+0000","last_undegraded":"2024-04-25T11:30:12.305887+0000","last_fullsized":"2024-04-25T11:30:12.305887+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.090545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.327820+0000","last_change":"2024-04-25T11:30:12.316106+0000","last_active":"2024-04-25T11:30:12.327820+0000","last_peered":"2024-04-25T11:30:12.327820+0000","last_clean":"2024-04-25T11:30:12.327820+0000","last_became_active":"2024-04-25T11:30:12.315513+0000","last_became_peered":"2024-04-25T11:30:12.315513+0000","last_unstale":"2024-04-25T11:30:12.327820+0000","last_undegraded":"2024-04-25T11:30:12.327820+0000","last_fullsized":"2024-04-25T11:30:12.327820+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:56.498661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.306699+0000","last_change":"2024-04-25T11:30:11.264133+0000","last_active":"2024-04-25T11:30:12.306699+0000","last_peered":"2024-04-25T11:30:12.306699+0000","last_clean":"2024-04-25T11:30:12.306699+0000","last_became_active":"2024-04-25T11:30:11.263693+0000","last_became_peered":"2024-04-25T11:30:11.263693+0000","last_unstale":"2024-04-25T11:30:12.306699+0000","last_undegraded":"2024-04-25T11:30:12.306699+0000","last_fullsized":"2024-04-25T11:30:12.306699+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.388286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.326385+0000","last_change":"2024-04-25T11:30:12.321099+0000","last_active":"2024-04-25T11:30:12.326385+0000","last_peered":"2024-04-25T11:30:12.326385+0000","last_clean":"2024-04-25T11:30:12.326385+0000","last_became_active":"2024-04-25T11:30:12.319720+0000","last_became_peered":"2024-04-25T11:30:12.319720+0000","last_unstale":"2024-04-25T11:30:12.326385+0000","last_undegraded":"2024-04-25T11:30:12.326385+0000","last_fullsized":"2024-04-25T11:30:12.326385+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:28.188588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.305559+0000","last_change":"2024-04-25T11:30:11.278565+0000","last_active":"2024-04-25T11:30:12.305559+0000","last_peered":"2024-04-25T11:30:12.305559+0000","last_clean":"2024-04-25T11:30:12.305559+0000","last_became_active":"2024-04-25T11:30:11.278040+0000","last_became_peered":"2024-04-25T11:30:11.278040+0000","last_unstale":"2024-04-25T11:30:12.305559+0000","last_undegraded":"2024-04-25T11:30:12.305559+0000","last_fullsized":"2024-04-25T11:30:12.305559+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.982156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.255841+0000","last_change":"2024-04-25T11:30:11.255841+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.255841+0000","last_undegraded":"2024-04-25T11:30:11.255841+0000","last_fullsized":"2024-04-25T11:30:11.255841+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.298802+0000","last_change":"2024-04-25T11:30:11.272593+0000","last_active":"2024-04-25T11:30:12.298802+0000","last_peered":"2024-04-25T11:30:12.298802+0000","last_clean":"2024-04-25T11:30:12.298802+0000","last_became_active":"2024-04-25T11:30:11.272280+0000","last_became_peered":"2024-04-25T11:30:11.272280+0000","last_unstale":"2024-04-25T11:30:12.298802+0000","last_undegraded":"2024-04-25T11:30:12.298802+0000","last_fullsized":"2024-04-25T11:30:12.298802+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:29.649052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.302366+0000","last_change":"2024-04-25T11:30:12.296593+0000","last_active":"2024-04-25T11:30:12.302366+0000","last_peered":"2024-04-25T11:30:12.302366+0000","last_clean":"2024-04-25T11:30:12.302366+0000","last_became_active":"2024-04-25T11:30:12.296265+0000","last_became_peered":"2024-04-25T11:30:12.296265+0000","last_unstale":"2024-04-25T11:30:12.302366+0000","last_undegraded":"2024-04-25T11:30:12.302366+0000","last_fullsized":"2024-04-25T11:30:12.302366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:45.204846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304061+0000","last_change":"2024-04-25T11:30:11.271824+0000","last_active":"2024-04-25T11:30:12.304061+0000","last_peered":"2024-04-25T11:30:12.304061+0000","last_clean":"2024-04-25T11:30:12.304061+0000","last_became_active":"2024-04-25T11:30:11.270927+0000","last_became_peered":"2024-04-25T11:30:11.270927+0000","last_unstale":"2024-04-25T11:30:12.304061+0000","last_undegraded":"2024-04-25T11:30:12.304061+0000","last_fullsized":"2024-04-25T11:30:12.304061+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:19.313207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.303104+0000","last_change":"2024-04-25T11:30:12.298869+0000","last_active":"2024-04-25T11:30:12.303104+0000","last_peered":"2024-04-25T11:30:12.303104+0000","last_clean":"2024-04-25T11:30:12.303104+0000","last_became_active":"2024-04-25T11:30:12.298520+0000","last_became_peered":"2024-04-25T11:30:12.298520+0000","last_unstale":"2024-04-25T11:30:12.303104+0000","last_undegraded":"2024-04-25T11:30:12.303104+0000","last_fullsized":"2024-04-25T11:30:12.303104+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:32.390065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.304469+0000","last_change":"2024-04-25T11:30:11.275287+0000","last_active":"2024-04-25T11:30:12.304469+0000","last_peered":"2024-04-25T11:30:12.304469+0000","last_clean":"2024-04-25T11:30:12.304469+0000","last_became_active":"2024-04-25T11:30:11.274574+0000","last_became_peered":"2024-04-25T11:30:11.274574+0000","last_unstale":"2024-04-25T11:30:12.304469+0000","last_undegraded":"2024-04-25T11:30:12.304469+0000","last_fullsized":"2024-04-25T11:30:12.304469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:39.602919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-25T11:30:11.264576+0000","last_change":"2024-04-25T11:30:11.264576+0000","last_active":"2024-04-25T11:30:11.224057+0000","last_peered":"2024-04-25T11:30:11.224057+0000","last_clean":"2024-04-25T11:30:11.224057+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:30:11.264576+0000","last_undegraded":"2024-04-25T11:30:11.264576+0000","last_fullsized":"2024-04-25T11:30:11.264576+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.335329+0000","last_change":"2024-04-25T11:30:11.272554+0000","last_active":"2024-04-25T11:30:12.335329+0000","last_peered":"2024-04-25T11:30:12.335329+0000","last_clean":"2024-04-25T11:30:12.335329+0000","last_became_active":"2024-04-25T11:30:11.272199+0000","last_became_peered":"2024-04-25T11:30:11.272199+0000","last_unstale":"2024-04-25T11:30:12.335329+0000","last_undegraded":"2024-04-25T11:30:12.335329+0000","last_fullsized":"2024-04-25T11:30:12.335329+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:32.204333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":161,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T11:30:12.276461+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:30:12.276461+0000","last_peered":"2024-04-25T11:30:12.276461+0000","last_clean":"2024-04-25T11:30:12.276461+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:30:12.276461+0000","last_undegraded":"2024-04-25T11:30:12.276461+0000","last_fullsized":"2024-04-25T11:30:12.276461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1167360,"data_stored":823540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":226327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1150976,"data_stored":816695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:30:18.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-25T11:30:18.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.256254+0000', 'last_change': '2024-04-25T11:30:11.256254+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.256254+0000', 'last_undegraded': '2024-04-25T11:30:11.256254+0000', 'last_fullsized': '2024-04-25T11:30:11.256254+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.254522+0000', 'last_change': '2024-04-25T11:30:11.254522+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.254522+0000', 'last_undegraded': '2024-04-25T11:30:11.254522+0000', 'last_fullsized': '2024-04-25T11:30:11.254522+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.265764+0000', 'last_change': '2024-04-25T11:30:11.265764+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.265764+0000', 'last_undegraded': '2024-04-25T11:30:11.265764+0000', 'last_fullsized': '2024-04-25T11:30:11.265764+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.253292+0000', 'last_change': '2024-04-25T11:30:11.253292+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.253292+0000', 'last_undegraded': '2024-04-25T11:30:11.253292+0000', 'last_fullsized': '2024-04-25T11:30:11.253292+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.252332+0000', 'last_change': '2024-04-25T11:30:11.252332+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.252332+0000', 'last_undegraded': '2024-04-25T11:30:11.252332+0000', 'last_fullsized': '2024-04-25T11:30:11.252332+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.254710+0000', 'last_change': '2024-04-25T11:30:11.254710+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.254710+0000', 'last_undegraded': '2024-04-25T11:30:11.254710+0000', 'last_fullsized': '2024-04-25T11:30:11.254710+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.265187+0000', 'last_change': '2024-04-25T11:30:11.265187+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.265187+0000', 'last_undegraded': '2024-04-25T11:30:11.265187+0000', 'last_fullsized': '2024-04-25T11:30:11.265187+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.252849+0000', 'last_change': '2024-04-25T11:30:11.252849+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.252849+0000', 'last_undegraded': '2024-04-25T11:30:11.252849+0000', 'last_fullsized': '2024-04-25T11:30:11.252849+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.261566+0000', 'last_change': '2024-04-25T11:30:11.261566+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.261566+0000', 'last_undegraded': '2024-04-25T11:30:11.261566+0000', 'last_fullsized': '2024-04-25T11:30:11.261566+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.254783+0000', 'last_change': '2024-04-25T11:30:11.254783+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.254783+0000', 'last_undegraded': '2024-04-25T11:30:11.254783+0000', 'last_fullsized': '2024-04-25T11:30:11.254783+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.251703+0000', 'last_change': '2024-04-25T11:30:11.251703+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.251703+0000', 'last_undegraded': '2024-04-25T11:30:11.251703+0000', 'last_fullsized': '2024-04-25T11:30:11.251703+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T11:30:18.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.250705+0000', 'last_change': '2024-04-25T11:30:11.250705+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.250705+0000', 'last_undegraded': '2024-04-25T11:30:11.250705+0000', 'last_fullsized': '2024-04-25T11:30:11.250705+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T11:30:18.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.255841+0000', 'last_change': '2024-04-25T11:30:11.255841+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.255841+0000', 'last_undegraded': '2024-04-25T11:30:11.255841+0000', 'last_fullsized': '2024-04-25T11:30:11.255841+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T11:30:18.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:30:11.264576+0000', 'last_change': '2024-04-25T11:30:11.264576+0000', 'last_active': '2024-04-25T11:30:11.224057+0000', 'last_peered': '2024-04-25T11:30:11.224057+0000', 'last_clean': '2024-04-25T11:30:11.224057+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:30:11.264576+0000', 'last_undegraded': '2024-04-25T11:30:11.264576+0000', 'last_fullsized': '2024-04-25T11:30:11.264576+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'last_clean_scrub_stamp': '2024-04-25T11:30:11.224057+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:30:18.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T11:30:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.593+0000 7fcdb3c00640 1 Processor -- start 2024-04-25T11:30:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.593+0000 7fcdb3c00640 1 -- start start 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdb3c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcdac1443c0 0x7fcdac06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdb3c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdac070370 0x7fcdac0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdb3c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdb3c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcdac0708d0 con 0x7fcdac070370 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdb3c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcdac15f160 con 0x7fcdac1443c0 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdb3c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcdac0cff30 con 0x7fcdac0cfb20 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdabe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdac070370 0x7fcdac0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdabe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdac070370 0x7fcdac0cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34260/0 (socket says 172.21.15.19:34260) 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.596+0000 7fcdabe00640 1 -- 172.21.15.19:0/4115139539 learned_addr learned my addr 172.21.15.19:0/4115139539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdb1200640 1 --2- 172.21.15.19:0/4115139539 >> v2:172.21.15.169:3300/0 conn(0x7fcdac1443c0 0x7fcdac06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdabe00640 1 -- 172.21.15.19:0/4115139539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 msgr2=0x7fcdac0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdb1c00640 1 --2- 172.21.15.19:0/4115139539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdabe00640 1 --2- 172.21.15.19:0/4115139539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac0ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdabe00640 1 -- 172.21.15.19:0/4115139539 >> v2:172.21.15.169:3300/0 conn(0x7fcdac1443c0 msgr2=0x7fcdac06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdabe00640 1 --2- 172.21.15.19:0/4115139539 >> v2:172.21.15.169:3300/0 conn(0x7fcdac1443c0 0x7fcdac06fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdabe00640 1 -- 172.21.15.19:0/4115139539 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac15f5f0 con 0x7fcdac070370 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.597+0000 7fcdb1200640 1 --2- 172.21.15.19:0/4115139539 >> v2:172.21.15.169:3300/0 conn(0x7fcdac1443c0 0x7fcdac06fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.598+0000 7fcdabe00640 1 --2- 172.21.15.19:0/4115139539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac070370 0x7fcdac0cf5e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcdac072820 tx=0x7fcd9402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75c16159fef3294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.598+0000 7fcdab400640 1 -- 172.21.15.19:0/4115139539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94036070 con 0x7fcdac070370 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.598+0000 7fcdab400640 1 -- 172.21.15.19:0/4115139539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd94031600 con 0x7fcdac070370 2024-04-25T11:30:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.598+0000 7fcdab400640 1 -- 172.21.15.19:0/4115139539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd940319d0 con 0x7fcdac070370 2024-04-25T11:30:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 -- 172.21.15.19:0/4115139539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac070370 msgr2=0x7fcdac0cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/4115139539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac070370 0x7fcdac0cf5e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcdac072820 tx=0x7fcd9402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 -- 172.21.15.19:0/4115139539 shutdown_connections 2024-04-25T11:30:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/4115139539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac070370 0x7fcdac0cf5e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/4115139539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/4115139539 >> v2:172.21.15.169:3300/0 conn(0x7fcdac1443c0 0x7fcdac06fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 -- 172.21.15.19:0/4115139539 >> 172.21.15.19:0/4115139539 conn(0x7fcdac0d1ae0 msgr2=0x7fcdac0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 -- 172.21.15.19:0/4115139539 shutdown_connections 2024-04-25T11:30:18.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.599+0000 7fcdb3c00640 1 -- 172.21.15.19:0/4115139539 wait complete. 2024-04-25T11:30:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.602+0000 7fcdb3c00640 1 Processor -- start 2024-04-25T11:30:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.602+0000 7fcdb3c00640 1 -- start start 2024-04-25T11:30:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.603+0000 7fcdb3c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac070370 0x7fcdac143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.603+0000 7fcdb3c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac1427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.603+0000 7fcdb3c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac1443c0 0x7fcdac142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.603+0000 7fcdb3c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac15edf0 con 0x7fcdac1443c0 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.603+0000 7fcdb3c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac15eb10 con 0x7fcdac070370 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.603+0000 7fcdb3c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcdac1447d0 con 0x7fcdac0cfb20 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac1443c0 0x7fcdac142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac1443c0 0x7fcdac142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34262/0 (socket says 172.21.15.19:34262) 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 -- 172.21.15.19:0/2526195910 learned_addr learned my addr 172.21.15.19:0/2526195910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1200640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac070370 0x7fcdac143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdabe00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac1427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 -- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 msgr2=0x7fcdac1427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac1427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 -- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac070370 msgr2=0x7fcdac143430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac070370 0x7fcdac143430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.604+0000 7fcdb1c00640 1 -- 172.21.15.19:0/2526195910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac160540 con 0x7fcdac1443c0 2024-04-25T11:30:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.605+0000 7fcdb1c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac1443c0 0x7fcdac142d30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c4c0 tx=0x7fcda400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:18.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.605+0000 7fcda8c00640 1 -- 172.21.15.19:0/2526195910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4015070 con 0x7fcdac1443c0 2024-04-25T11:30:18.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.605+0000 7fcda8c00640 1 -- 172.21.15.19:0/2526195910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda4004220 con 0x7fcdac1443c0 2024-04-25T11:30:18.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.605+0000 7fcda8c00640 1 -- 172.21.15.19:0/2526195910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4003610 con 0x7fcdac1443c0 2024-04-25T11:30:18.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.605+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdac160820 con 0x7fcdac1443c0 2024-04-25T11:30:18.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.605+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdac160de0 con 0x7fcdac1443c0 2024-04-25T11:30:18.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.606+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdac1602b0 con 0x7fcdac1443c0 2024-04-25T11:30:18.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.608+0000 7fcda8c00640 1 -- 172.21.15.19:0/2526195910 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcda40037b0 con 0x7fcdac1443c0 2024-04-25T11:30:18.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.609+0000 7fcda8c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd880744b0 0x7fcd880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:18.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.609+0000 7fcda8c00640 1 -- 172.21.15.19:0/2526195910 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fcda4094f00 con 0x7fcdac1443c0 2024-04-25T11:30:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.609+0000 7fcdb1200640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd880744b0 0x7fcd880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.611+0000 7fcdb1200640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd880744b0 0x7fcd880769c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0d1860 tx=0x7fcd9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:18.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.614+0000 7fcda8c00640 1 -- 172.21.15.19:0/2526195910 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcda405f680 con 0x7fcdac1443c0 2024-04-25T11:30:18.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.889+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fcdac050740 con 0x7fcdac1443c0 2024-04-25T11:30:18.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.890+0000 7fcda8c00640 1 -- 172.21.15.19:0/2526195910 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1262 (secure 0 0 0) 0x7fcda405ef80 con 0x7fcdac1443c0 2024-04-25T11:30:18.881 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:18.881 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1714044432,"num_in_osds":8,"osd_in_since":1714044423,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"creating+peering","count":14}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592756,"bytes_used":235552768,"bytes_avail":772858560512,"bytes_total":773094113280,"inactive_pgs_ratio":0.14432989060878754,"write_bytes_sec":2217,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":34,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5721}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-25T11:29:38.490570+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T11:30:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd880744b0 msgr2=0x7fcd880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd880744b0 0x7fcd880769c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0d1860 tx=0x7fcd9c008040 comp rx=0 tx=0).stop 2024-04-25T11:30:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac1443c0 msgr2=0x7fcdac142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac1443c0 0x7fcdac142d30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcda400c4c0 tx=0x7fcda400c980 comp rx=0 tx=0).stop 2024-04-25T11:30:18.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 shutdown_connections 2024-04-25T11:30:18.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd880744b0 0x7fcd880769c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdac1443c0 0x7fcdac142d30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cfb20 0x7fcdac1427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 --2- 172.21.15.19:0/2526195910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac070370 0x7fcdac143430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:18.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 >> 172.21.15.19:0/2526195910 conn(0x7fcdac0d1ae0 msgr2=0x7fcdac0b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:18.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 shutdown_connections 2024-04-25T11:30:18.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:18.893+0000 7fcdb3c00640 1 -- 172.21.15.19:0/2526195910 wait complete. 2024-04-25T11:30:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T11:30:18.908 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:30:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.188+0000 7fa2a7e00640 1 Processor -- start 2024-04-25T11:30:19.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.188+0000 7fa2a7e00640 1 -- start start 2024-04-25T11:30:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2a806d560 0x7fa2a815c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2a806f180 0x7fa2a80b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a8071230 con 0x7fa2a8070e20 2024-04-25T11:30:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a806f8f0 con 0x7fa2a806d560 2024-04-25T11:30:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a806fbd0 con 0x7fa2a806f180 2024-04-25T11:30:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2a806d560 0x7fa2a815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:19.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2a806d560 0x7fa2a815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34264/0 (socket says 172.21.15.19:34264) 2024-04-25T11:30:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.190+0000 7fa2a6000640 1 -- 172.21.15.19:0/1555617073 learned_addr learned my addr 172.21.15.19:0/1555617073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:19.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.191+0000 7fa2a6000640 1 -- 172.21.15.19:0/1555617073 >> v2:172.21.15.169:3300/0 conn(0x7fa2a806f180 msgr2=0x7fa2a80b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.191+0000 7fa2a7400640 1 --2- 172.21.15.19:0/1555617073 >> v2:172.21.15.169:3300/0 conn(0x7fa2a806f180 0x7fa2a80b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.191+0000 7fa2a6000640 1 --2- 172.21.15.19:0/1555617073 >> v2:172.21.15.169:3300/0 conn(0x7fa2a806f180 0x7fa2a80b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.191+0000 7fa2a6000640 1 -- 172.21.15.19:0/1555617073 >> v1:172.21.15.169:6789/0 conn(0x7fa2a8070e20 legacy=0x7fa2a806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.191+0000 7fa2a6000640 1 -- 172.21.15.19:0/1555617073 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a806feb0 con 0x7fa2a806d560 2024-04-25T11:30:19.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.192+0000 7fa2a6000640 1 --2- 172.21.15.19:0/1555617073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a806d560 0x7fa2a815c800 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8072cb0 tx=0x7fa29002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cd00030a4432ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.192+0000 7fa2a5600640 1 -- 172.21.15.19:0/1555617073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290036070 con 0x7fa2a806d560 2024-04-25T11:30:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.192+0000 7fa2a5600640 1 -- 172.21.15.19:0/1555617073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa290031600 con 0x7fa2a806d560 2024-04-25T11:30:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.192+0000 7fa2a5600640 1 -- 172.21.15.19:0/1555617073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900319d0 con 0x7fa2a806d560 2024-04-25T11:30:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 -- 172.21.15.19:0/1555617073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a806d560 msgr2=0x7fa2a815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/1555617073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a806d560 0x7fa2a815c800 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8072cb0 tx=0x7fa29002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 -- 172.21.15.19:0/1555617073 shutdown_connections 2024-04-25T11:30:19.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/1555617073 >> v2:172.21.15.169:3300/0 conn(0x7fa2a806f180 0x7fa2a80b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/1555617073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a806d560 0x7fa2a815c800 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 -- 172.21.15.19:0/1555617073 >> 172.21.15.19:0/1555617073 conn(0x7fa2a80d1720 msgr2=0x7fa2a80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 -- 172.21.15.19:0/1555617073 shutdown_connections 2024-04-25T11:30:19.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.193+0000 7fa2a7e00640 1 -- 172.21.15.19:0/1555617073 wait complete. 2024-04-25T11:30:19.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.195+0000 7fa2a7e00640 1 Processor -- start 2024-04-25T11:30:19.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.196+0000 7fa2a7e00640 1 -- start start 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a806f180 0x7fa2a814c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2a8070e20 0x7fa2a8147360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a8133250 0x7fa2a8134a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a806c5c0 con 0x7fa2a8133250 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a806f590 con 0x7fa2a806f180 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a806c740 con 0x7fa2a8070e20 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a8133250 0x7fa2a8134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a8133250 0x7fa2a8134a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34266/0 (socket says 172.21.15.19:34266) 2024-04-25T11:30:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.197+0000 7fa2a7400640 1 -- 172.21.15.19:0/2935657969 learned_addr learned my addr 172.21.15.19:0/2935657969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a6a00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a806f180 0x7fa2a814c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a7400640 1 -- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2a8070e20 msgr2=0x7fa2a8147360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a6000640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2a8070e20 0x7fa2a8147360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a7400640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2a8070e20 0x7fa2a8147360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a7400640 1 -- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a806f180 msgr2=0x7fa2a814c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a7400640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a806f180 0x7fa2a814c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a7400640 1 -- 172.21.15.19:0/2935657969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8133980 con 0x7fa2a8133250 2024-04-25T11:30:19.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.198+0000 7fa2a7400640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a8133250 0x7fa2a8134a20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000c550 tx=0x7fa2a000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:19.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.199+0000 7fa29ea00640 1 -- 172.21.15.19:0/2935657969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0015070 con 0x7fa2a8133250 2024-04-25T11:30:19.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.199+0000 7fa29ea00640 1 -- 172.21.15.19:0/2935657969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a0004220 con 0x7fa2a8133250 2024-04-25T11:30:19.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.199+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a8133c60 con 0x7fa2a8133250 2024-04-25T11:30:19.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.199+0000 7fa29ea00640 1 -- 172.21.15.19:0/2935657969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0003610 con 0x7fa2a8133250 2024-04-25T11:30:19.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.199+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a8134220 con 0x7fa2a8133250 2024-04-25T11:30:19.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.200+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a8050740 con 0x7fa2a8133250 2024-04-25T11:30:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.201+0000 7fa29ea00640 1 -- 172.21.15.19:0/2935657969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa2a00037b0 con 0x7fa2a8133250 2024-04-25T11:30:19.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.203+0000 7fa29ea00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa284074500 0x7fa284076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:19.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.203+0000 7fa29ea00640 1 -- 172.21.15.19:0/2935657969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fa2a0094f80 con 0x7fa2a8133250 2024-04-25T11:30:19.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.203+0000 7fa2a6a00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa284074500 0x7fa284076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:19.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.205+0000 7fa2a6a00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa284074500 0x7fa284076a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa298006fd0 tx=0x7fa298008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:19.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.208+0000 7fa29ea00640 1 -- 172.21.15.19:0/2935657969 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2a005f6f0 con 0x7fa2a8133250 2024-04-25T11:30:19.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.414+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2a815e8b0 con 0x7fa284074500 2024-04-25T11:30:19.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.433+0000 7fa29ea00640 1 -- 172.21.15.19:0/2935657969 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fa2a815e8b0 con 0x7fa284074500 2024-04-25T11:30:19.427 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:19.429 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:30:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.438+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa284074500 msgr2=0x7fa284076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.438+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa284074500 0x7fa284076a10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa298006fd0 tx=0x7fa298008040 comp rx=0 tx=0).stop 2024-04-25T11:30:19.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.438+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a8133250 msgr2=0x7fa2a8134a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.438+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a8133250 0x7fa2a8134a20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000c550 tx=0x7fa2a000ca10 comp rx=0 tx=0).stop 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 shutdown_connections 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa284074500 0x7fa284076a10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2a8133250 0x7fa2a8134a20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2a8070e20 0x7fa2a8147360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 --2- 172.21.15.19:0/2935657969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2a806f180 0x7fa2a814c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 >> 172.21.15.19:0/2935657969 conn(0x7fa2a80d1720 msgr2=0x7fa2a8150ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 shutdown_connections 2024-04-25T11:30:19.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:19.439+0000 7fa2a7e00640 1 -- 172.21.15.19:0/2935657969 wait complete. 2024-04-25T11:30:19.454 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-04-25T11:30:18.505934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234436,"kb_used_data":5768,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740284,"statfs":{"total":773094113280,"available":772854050816,"internally_reserved":0,"allocated":5906432,"data_stored":3100772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.270658"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.300247+0000","last_change":"2024-04-25T11:30:12.328001+0000","last_active":"2024-04-25T11:30:13.300247+0000","last_peered":"2024-04-25T11:30:13.300247+0000","last_clean":"2024-04-25T11:30:13.300247+0000","last_became_active":"2024-04-25T11:30:12.327555+0000","last_became_peered":"2024-04-25T11:30:12.327555+0000","last_unstale":"2024-04-25T11:30:13.300247+0000","last_undegraded":"2024-04-25T11:30:13.300247+0000","last_fullsized":"2024-04-25T11:30:13.300247+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302159+0000","last_change":"2024-04-25T11:30:12.316817+0000","last_active":"2024-04-25T11:30:13.302159+0000","last_peered":"2024-04-25T11:30:13.302159+0000","last_clean":"2024-04-25T11:30:13.302159+0000","last_became_active":"2024-04-25T11:30:12.311768+0000","last_became_peered":"2024-04-25T11:30:12.311768+0000","last_unstale":"2024-04-25T11:30:13.302159+0000","last_undegraded":"2024-04-25T11:30:13.302159+0000","last_fullsized":"2024-04-25T11:30:13.302159+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:02.682497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304083+0000","last_change":"2024-04-25T11:30:12.315596+0000","last_active":"2024-04-25T11:30:13.304083+0000","last_peered":"2024-04-25T11:30:13.304083+0000","last_clean":"2024-04-25T11:30:13.304083+0000","last_became_active":"2024-04-25T11:30:12.315259+0000","last_became_peered":"2024-04-25T11:30:12.315259+0000","last_unstale":"2024-04-25T11:30:13.304083+0000","last_undegraded":"2024-04-25T11:30:13.304083+0000","last_fullsized":"2024-04-25T11:30:13.304083+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:03.212211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326300+0000","last_change":"2024-04-25T11:30:12.306582+0000","last_active":"2024-04-25T11:30:13.326300+0000","last_peered":"2024-04-25T11:30:13.326300+0000","last_clean":"2024-04-25T11:30:13.326300+0000","last_became_active":"2024-04-25T11:30:12.306118+0000","last_became_peered":"2024-04-25T11:30:12.306118+0000","last_unstale":"2024-04-25T11:30:13.326300+0000","last_undegraded":"2024-04-25T11:30:13.326300+0000","last_fullsized":"2024-04-25T11:30:13.326300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:55.746271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299288+0000","last_change":"2024-04-25T11:30:12.320809+0000","last_active":"2024-04-25T11:30:13.299288+0000","last_peered":"2024-04-25T11:30:13.299288+0000","last_clean":"2024-04-25T11:30:13.299288+0000","last_became_active":"2024-04-25T11:30:12.318568+0000","last_became_peered":"2024-04-25T11:30:12.318568+0000","last_unstale":"2024-04-25T11:30:13.299288+0000","last_undegraded":"2024-04-25T11:30:13.299288+0000","last_fullsized":"2024-04-25T11:30:13.299288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382958+0000","last_change":"2024-04-25T11:30:12.316853+0000","last_active":"2024-04-25T11:30:13.382958+0000","last_peered":"2024-04-25T11:30:13.382958+0000","last_clean":"2024-04-25T11:30:13.382958+0000","last_became_active":"2024-04-25T11:30:12.316509+0000","last_became_peered":"2024-04-25T11:30:12.316509+0000","last_unstale":"2024-04-25T11:30:13.382958+0000","last_undegraded":"2024-04-25T11:30:13.382958+0000","last_fullsized":"2024-04-25T11:30:13.382958+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331346+0000","last_change":"2024-04-25T11:30:12.323818+0000","last_active":"2024-04-25T11:30:13.331346+0000","last_peered":"2024-04-25T11:30:13.331346+0000","last_clean":"2024-04-25T11:30:13.331346+0000","last_became_active":"2024-04-25T11:30:12.322940+0000","last_became_peered":"2024-04-25T11:30:12.322940+0000","last_unstale":"2024-04-25T11:30:13.331346+0000","last_undegraded":"2024-04-25T11:30:13.331346+0000","last_fullsized":"2024-04-25T11:30:13.331346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:26.075584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325316+0000","last_change":"2024-04-25T11:30:12.311821+0000","last_active":"2024-04-25T11:30:13.325316+0000","last_peered":"2024-04-25T11:30:13.325316+0000","last_clean":"2024-04-25T11:30:13.325316+0000","last_became_active":"2024-04-25T11:30:12.311464+0000","last_became_peered":"2024-04-25T11:30:12.311464+0000","last_unstale":"2024-04-25T11:30:13.325316+0000","last_undegraded":"2024-04-25T11:30:13.325316+0000","last_fullsized":"2024-04-25T11:30:13.325316+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:57.890018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383837+0000","last_change":"2024-04-25T11:30:12.316458+0000","last_active":"2024-04-25T11:30:13.383837+0000","last_peered":"2024-04-25T11:30:13.383837+0000","last_clean":"2024-04-25T11:30:13.383837+0000","last_became_active":"2024-04-25T11:30:12.315188+0000","last_became_peered":"2024-04-25T11:30:12.315188+0000","last_unstale":"2024-04-25T11:30:13.383837+0000","last_undegraded":"2024-04-25T11:30:13.383837+0000","last_fullsized":"2024-04-25T11:30:13.383837+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385057+0000","last_change":"2024-04-25T11:30:12.311133+0000","last_active":"2024-04-25T11:30:14.385057+0000","last_peered":"2024-04-25T11:30:14.385057+0000","last_clean":"2024-04-25T11:30:14.385057+0000","last_became_active":"2024-04-25T11:30:12.310670+0000","last_became_peered":"2024-04-25T11:30:12.310670+0000","last_unstale":"2024-04-25T11:30:14.385057+0000","last_undegraded":"2024-04-25T11:30:14.385057+0000","last_fullsized":"2024-04-25T11:30:14.385057+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:18.525690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326805+0000","last_change":"2024-04-25T11:30:12.319414+0000","last_active":"2024-04-25T11:30:13.326805+0000","last_peered":"2024-04-25T11:30:13.326805+0000","last_clean":"2024-04-25T11:30:13.326805+0000","last_became_active":"2024-04-25T11:30:12.317760+0000","last_became_peered":"2024-04-25T11:30:12.317760+0000","last_unstale":"2024-04-25T11:30:13.326805+0000","last_undegraded":"2024-04-25T11:30:13.326805+0000","last_fullsized":"2024-04-25T11:30:13.326805+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:25.408994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.303853+0000","last_change":"2024-04-25T11:30:12.306558+0000","last_active":"2024-04-25T11:30:13.303853+0000","last_peered":"2024-04-25T11:30:13.303853+0000","last_clean":"2024-04-25T11:30:13.303853+0000","last_became_active":"2024-04-25T11:30:12.306159+0000","last_became_peered":"2024-04-25T11:30:12.306159+0000","last_unstale":"2024-04-25T11:30:13.303853+0000","last_undegraded":"2024-04-25T11:30:13.303853+0000","last_fullsized":"2024-04-25T11:30:13.303853+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382778+0000","last_change":"2024-04-25T11:30:12.315317+0000","last_active":"2024-04-25T11:30:13.382778+0000","last_peered":"2024-04-25T11:30:13.382778+0000","last_clean":"2024-04-25T11:30:13.382778+0000","last_became_active":"2024-04-25T11:30:12.314919+0000","last_became_peered":"2024-04-25T11:30:12.314919+0000","last_unstale":"2024-04-25T11:30:13.382778+0000","last_undegraded":"2024-04-25T11:30:13.382778+0000","last_fullsized":"2024-04-25T11:30:13.382778+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325520+0000","last_change":"2024-04-25T11:30:12.315951+0000","last_active":"2024-04-25T11:30:13.325520+0000","last_peered":"2024-04-25T11:30:13.325520+0000","last_clean":"2024-04-25T11:30:13.325520+0000","last_became_active":"2024-04-25T11:30:12.315574+0000","last_became_peered":"2024-04-25T11:30:12.315574+0000","last_unstale":"2024-04-25T11:30:13.325520+0000","last_undegraded":"2024-04-25T11:30:13.325520+0000","last_fullsized":"2024-04-25T11:30:13.325520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:03.301131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326215+0000","last_change":"2024-04-25T11:30:12.304594+0000","last_active":"2024-04-25T11:30:13.326215+0000","last_peered":"2024-04-25T11:30:13.326215+0000","last_clean":"2024-04-25T11:30:13.326215+0000","last_became_active":"2024-04-25T11:30:12.304268+0000","last_became_peered":"2024-04-25T11:30:12.304268+0000","last_unstale":"2024-04-25T11:30:13.326215+0000","last_undegraded":"2024-04-25T11:30:13.326215+0000","last_fullsized":"2024-04-25T11:30:13.326215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:01.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.298741+0000","last_change":"2024-04-25T11:30:12.320656+0000","last_active":"2024-04-25T11:30:13.298741+0000","last_peered":"2024-04-25T11:30:13.298741+0000","last_clean":"2024-04-25T11:30:13.298741+0000","last_became_active":"2024-04-25T11:30:12.319501+0000","last_became_peered":"2024-04-25T11:30:12.319501+0000","last_unstale":"2024-04-25T11:30:13.298741+0000","last_undegraded":"2024-04-25T11:30:13.298741+0000","last_fullsized":"2024-04-25T11:30:13.298741+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:02.682497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305002+0000","last_change":"2024-04-25T11:30:12.312204+0000","last_active":"2024-04-25T11:30:13.305002+0000","last_peered":"2024-04-25T11:30:13.305002+0000","last_clean":"2024-04-25T11:30:13.305002+0000","last_became_active":"2024-04-25T11:30:12.311798+0000","last_became_peered":"2024-04-25T11:30:12.311798+0000","last_unstale":"2024-04-25T11:30:13.305002+0000","last_undegraded":"2024-04-25T11:30:13.305002+0000","last_fullsized":"2024-04-25T11:30:13.305002+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:28.188588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305555+0000","last_change":"2024-04-25T11:30:12.292655+0000","last_active":"2024-04-25T11:30:13.305555+0000","last_peered":"2024-04-25T11:30:13.305555+0000","last_clean":"2024-04-25T11:30:13.305555+0000","last_became_active":"2024-04-25T11:30:12.292173+0000","last_became_peered":"2024-04-25T11:30:12.292173+0000","last_unstale":"2024-04-25T11:30:13.305555+0000","last_undegraded":"2024-04-25T11:30:13.305555+0000","last_fullsized":"2024-04-25T11:30:13.305555+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:38.180760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299459+0000","last_change":"2024-04-25T11:30:12.318490+0000","last_active":"2024-04-25T11:30:13.299459+0000","last_peered":"2024-04-25T11:30:13.299459+0000","last_clean":"2024-04-25T11:30:13.299459+0000","last_became_active":"2024-04-25T11:30:12.317442+0000","last_became_peered":"2024-04-25T11:30:12.317442+0000","last_unstale":"2024-04-25T11:30:13.299459+0000","last_undegraded":"2024-04-25T11:30:13.299459+0000","last_fullsized":"2024-04-25T11:30:13.299459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.314933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385166+0000","last_change":"2024-04-25T11:30:12.311311+0000","last_active":"2024-04-25T11:30:14.385166+0000","last_peered":"2024-04-25T11:30:14.385166+0000","last_clean":"2024-04-25T11:30:14.385166+0000","last_became_active":"2024-04-25T11:30:12.310846+0000","last_became_peered":"2024-04-25T11:30:12.310846+0000","last_unstale":"2024-04-25T11:30:14.385166+0000","last_undegraded":"2024-04-25T11:30:14.385166+0000","last_fullsized":"2024-04-25T11:30:14.385166+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:21.559902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326029+0000","last_change":"2024-04-25T11:30:12.299502+0000","last_active":"2024-04-25T11:30:13.326029+0000","last_peered":"2024-04-25T11:30:13.326029+0000","last_clean":"2024-04-25T11:30:13.326029+0000","last_became_active":"2024-04-25T11:30:12.297154+0000","last_became_peered":"2024-04-25T11:30:12.297154+0000","last_unstale":"2024-04-25T11:30:13.326029+0000","last_undegraded":"2024-04-25T11:30:13.326029+0000","last_fullsized":"2024-04-25T11:30:13.326029+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:31.195209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.382679+0000","last_change":"2024-04-25T11:30:12.318465+0000","last_active":"2024-04-25T11:30:14.382679+0000","last_peered":"2024-04-25T11:30:14.382679+0000","last_clean":"2024-04-25T11:30:14.382679+0000","last_became_active":"2024-04-25T11:30:12.318076+0000","last_became_peered":"2024-04-25T11:30:12.318076+0000","last_unstale":"2024-04-25T11:30:14.382679+0000","last_undegraded":"2024-04-25T11:30:14.382679+0000","last_fullsized":"2024-04-25T11:30:14.382679+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:13.920881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331477+0000","last_change":"2024-04-25T11:30:12.308548+0000","last_active":"2024-04-25T11:30:13.331477+0000","last_peered":"2024-04-25T11:30:13.331477+0000","last_clean":"2024-04-25T11:30:13.331477+0000","last_became_active":"2024-04-25T11:30:12.308148+0000","last_became_peered":"2024-04-25T11:30:12.308148+0000","last_unstale":"2024-04-25T11:30:13.331477+0000","last_undegraded":"2024-04-25T11:30:13.331477+0000","last_fullsized":"2024-04-25T11:30:13.331477+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:19.436394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302162+0000","last_change":"2024-04-25T11:30:12.312781+0000","last_active":"2024-04-25T11:30:13.302162+0000","last_peered":"2024-04-25T11:30:13.302162+0000","last_clean":"2024-04-25T11:30:13.302162+0000","last_became_active":"2024-04-25T11:30:12.312453+0000","last_became_peered":"2024-04-25T11:30:12.312453+0000","last_unstale":"2024-04-25T11:30:13.302162+0000","last_undegraded":"2024-04-25T11:30:13.302162+0000","last_fullsized":"2024-04-25T11:30:13.302162+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.300794+0000","last_change":"2024-04-25T11:30:12.325014+0000","last_active":"2024-04-25T11:30:13.300794+0000","last_peered":"2024-04-25T11:30:13.300794+0000","last_clean":"2024-04-25T11:30:13.300794+0000","last_became_active":"2024-04-25T11:30:12.324660+0000","last_became_peered":"2024-04-25T11:30:12.324660+0000","last_unstale":"2024-04-25T11:30:13.300794+0000","last_undegraded":"2024-04-25T11:30:13.300794+0000","last_fullsized":"2024-04-25T11:30:13.300794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326645+0000","last_change":"2024-04-25T11:30:12.312808+0000","last_active":"2024-04-25T11:30:13.326645+0000","last_peered":"2024-04-25T11:30:13.326645+0000","last_clean":"2024-04-25T11:30:13.326645+0000","last_became_active":"2024-04-25T11:30:12.312389+0000","last_became_peered":"2024-04-25T11:30:12.312389+0000","last_unstale":"2024-04-25T11:30:13.326645+0000","last_undegraded":"2024-04-25T11:30:13.326645+0000","last_fullsized":"2024-04-25T11:30:13.326645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:32.464656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331879+0000","last_change":"2024-04-25T11:30:12.300829+0000","last_active":"2024-04-25T11:30:13.331879+0000","last_peered":"2024-04-25T11:30:13.331879+0000","last_clean":"2024-04-25T11:30:13.331879+0000","last_became_active":"2024-04-25T11:30:12.300457+0000","last_became_peered":"2024-04-25T11:30:12.300457+0000","last_unstale":"2024-04-25T11:30:13.331879+0000","last_undegraded":"2024-04-25T11:30:13.331879+0000","last_fullsized":"2024-04-25T11:30:13.331879+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:24.995116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331922+0000","last_change":"2024-04-25T11:30:12.306296+0000","last_active":"2024-04-25T11:30:13.331922+0000","last_peered":"2024-04-25T11:30:13.331922+0000","last_clean":"2024-04-25T11:30:13.331922+0000","last_became_active":"2024-04-25T11:30:12.305922+0000","last_became_peered":"2024-04-25T11:30:12.305922+0000","last_unstale":"2024-04-25T11:30:13.331922+0000","last_undegraded":"2024-04-25T11:30:13.331922+0000","last_fullsized":"2024-04-25T11:30:13.331922+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:32.527270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299595+0000","last_change":"2024-04-25T11:30:12.314344+0000","last_active":"2024-04-25T11:30:13.299595+0000","last_peered":"2024-04-25T11:30:13.299595+0000","last_clean":"2024-04-25T11:30:13.299595+0000","last_became_active":"2024-04-25T11:30:12.314033+0000","last_became_peered":"2024-04-25T11:30:12.314033+0000","last_unstale":"2024-04-25T11:30:13.299595+0000","last_undegraded":"2024-04-25T11:30:13.299595+0000","last_fullsized":"2024-04-25T11:30:13.299595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:32.067456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.301820+0000","last_change":"2024-04-25T11:30:12.306312+0000","last_active":"2024-04-25T11:30:13.301820+0000","last_peered":"2024-04-25T11:30:13.301820+0000","last_clean":"2024-04-25T11:30:13.301820+0000","last_became_active":"2024-04-25T11:30:12.305928+0000","last_became_peered":"2024-04-25T11:30:12.305928+0000","last_unstale":"2024-04-25T11:30:13.301820+0000","last_undegraded":"2024-04-25T11:30:13.301820+0000","last_fullsized":"2024-04-25T11:30:13.301820+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:03.855581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383300+0000","last_change":"2024-04-25T11:30:12.309122+0000","last_active":"2024-04-25T11:30:13.383300+0000","last_peered":"2024-04-25T11:30:13.383300+0000","last_clean":"2024-04-25T11:30:13.383300+0000","last_became_active":"2024-04-25T11:30:12.308689+0000","last_became_peered":"2024-04-25T11:30:12.308689+0000","last_unstale":"2024-04-25T11:30:13.383300+0000","last_undegraded":"2024-04-25T11:30:13.383300+0000","last_fullsized":"2024-04-25T11:30:13.383300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325495+0000","last_change":"2024-04-25T11:30:12.307262+0000","last_active":"2024-04-25T11:30:13.325495+0000","last_peered":"2024-04-25T11:30:13.325495+0000","last_clean":"2024-04-25T11:30:13.325495+0000","last_became_active":"2024-04-25T11:30:12.305770+0000","last_became_peered":"2024-04-25T11:30:12.305770+0000","last_unstale":"2024-04-25T11:30:13.325495+0000","last_undegraded":"2024-04-25T11:30:13.325495+0000","last_fullsized":"2024-04-25T11:30:13.325495+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:15.978494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383539+0000","last_change":"2024-04-25T11:30:12.324170+0000","last_active":"2024-04-25T11:30:14.383539+0000","last_peered":"2024-04-25T11:30:14.383539+0000","last_clean":"2024-04-25T11:30:14.383539+0000","last_became_active":"2024-04-25T11:30:12.323509+0000","last_became_peered":"2024-04-25T11:30:12.323509+0000","last_unstale":"2024-04-25T11:30:14.383539+0000","last_undegraded":"2024-04-25T11:30:14.383539+0000","last_fullsized":"2024-04-25T11:30:14.383539+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:38.180760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.384030+0000","last_change":"2024-04-25T11:30:11.269060+0000","last_active":"2024-04-25T11:30:13.384030+0000","last_peered":"2024-04-25T11:30:13.384030+0000","last_clean":"2024-04-25T11:30:13.384030+0000","last_became_active":"2024-04-25T11:30:11.266286+0000","last_became_peered":"2024-04-25T11:30:11.266286+0000","last_unstale":"2024-04-25T11:30:13.384030+0000","last_undegraded":"2024-04-25T11:30:13.384030+0000","last_fullsized":"2024-04-25T11:30:13.384030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:25.474525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.301123+0000","last_change":"2024-04-25T11:30:12.319935+0000","last_active":"2024-04-25T11:30:13.301123+0000","last_peered":"2024-04-25T11:30:13.301123+0000","last_clean":"2024-04-25T11:30:13.301123+0000","last_became_active":"2024-04-25T11:30:12.319486+0000","last_became_peered":"2024-04-25T11:30:12.319486+0000","last_unstale":"2024-04-25T11:30:13.301123+0000","last_undegraded":"2024-04-25T11:30:13.301123+0000","last_fullsized":"2024-04-25T11:30:13.301123+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:18.987772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331959+0000","last_change":"2024-04-25T11:30:11.257450+0000","last_active":"2024-04-25T11:30:13.331959+0000","last_peered":"2024-04-25T11:30:13.331959+0000","last_clean":"2024-04-25T11:30:13.331959+0000","last_became_active":"2024-04-25T11:30:11.257097+0000","last_became_peered":"2024-04-25T11:30:11.257097+0000","last_unstale":"2024-04-25T11:30:13.331959+0000","last_undegraded":"2024-04-25T11:30:13.331959+0000","last_fullsized":"2024-04-25T11:30:13.331959+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.501573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304090+0000","last_change":"2024-04-25T11:30:12.318193+0000","last_active":"2024-04-25T11:30:13.304090+0000","last_peered":"2024-04-25T11:30:13.304090+0000","last_clean":"2024-04-25T11:30:13.304090+0000","last_became_active":"2024-04-25T11:30:12.317611+0000","last_became_peered":"2024-04-25T11:30:12.317611+0000","last_unstale":"2024-04-25T11:30:13.304090+0000","last_undegraded":"2024-04-25T11:30:13.304090+0000","last_fullsized":"2024-04-25T11:30:13.304090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383612+0000","last_change":"2024-04-25T11:30:11.271679+0000","last_active":"2024-04-25T11:30:13.383612+0000","last_peered":"2024-04-25T11:30:13.383612+0000","last_clean":"2024-04-25T11:30:13.383612+0000","last_became_active":"2024-04-25T11:30:11.270812+0000","last_became_peered":"2024-04-25T11:30:11.270812+0000","last_unstale":"2024-04-25T11:30:13.383612+0000","last_undegraded":"2024-04-25T11:30:13.383612+0000","last_fullsized":"2024-04-25T11:30:13.383612+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:01.243237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326908+0000","last_change":"2024-04-25T11:30:12.306099+0000","last_active":"2024-04-25T11:30:13.326908+0000","last_peered":"2024-04-25T11:30:13.326908+0000","last_clean":"2024-04-25T11:30:13.326908+0000","last_became_active":"2024-04-25T11:30:12.305699+0000","last_became_peered":"2024-04-25T11:30:12.305699+0000","last_unstale":"2024-04-25T11:30:13.326908+0000","last_undegraded":"2024-04-25T11:30:13.326908+0000","last_fullsized":"2024-04-25T11:30:13.326908+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:04.320407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331887+0000","last_change":"2024-04-25T11:30:11.259259+0000","last_active":"2024-04-25T11:30:13.331887+0000","last_peered":"2024-04-25T11:30:13.331887+0000","last_clean":"2024-04-25T11:30:13.331887+0000","last_became_active":"2024-04-25T11:30:11.258919+0000","last_became_peered":"2024-04-25T11:30:11.258919+0000","last_unstale":"2024-04-25T11:30:13.331887+0000","last_undegraded":"2024-04-25T11:30:13.331887+0000","last_fullsized":"2024-04-25T11:30:13.331887+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:49.077411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331346+0000","last_change":"2024-04-25T11:30:12.310915+0000","last_active":"2024-04-25T11:30:13.331346+0000","last_peered":"2024-04-25T11:30:13.331346+0000","last_clean":"2024-04-25T11:30:13.331346+0000","last_became_active":"2024-04-25T11:30:12.310580+0000","last_became_peered":"2024-04-25T11:30:12.310580+0000","last_unstale":"2024-04-25T11:30:13.331346+0000","last_undegraded":"2024-04-25T11:30:13.331346+0000","last_fullsized":"2024-04-25T11:30:13.331346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:04.359361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383505+0000","last_change":"2024-04-25T11:30:11.278985+0000","last_active":"2024-04-25T11:30:13.383505+0000","last_peered":"2024-04-25T11:30:13.383505+0000","last_clean":"2024-04-25T11:30:13.383505+0000","last_became_active":"2024-04-25T11:30:11.273156+0000","last_became_peered":"2024-04-25T11:30:11.273156+0000","last_unstale":"2024-04-25T11:30:13.383505+0000","last_undegraded":"2024-04-25T11:30:13.383505+0000","last_fullsized":"2024-04-25T11:30:13.383505+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:37.174642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384624+0000","last_change":"2024-04-25T11:30:12.318436+0000","last_active":"2024-04-25T11:30:14.384624+0000","last_peered":"2024-04-25T11:30:14.384624+0000","last_clean":"2024-04-25T11:30:14.384624+0000","last_became_active":"2024-04-25T11:30:12.318108+0000","last_became_peered":"2024-04-25T11:30:12.318108+0000","last_unstale":"2024-04-25T11:30:14.384624+0000","last_undegraded":"2024-04-25T11:30:14.384624+0000","last_fullsized":"2024-04-25T11:30:14.384624+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:26.537536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332130+0000","last_change":"2024-04-25T11:30:11.263150+0000","last_active":"2024-04-25T11:30:13.332130+0000","last_peered":"2024-04-25T11:30:13.332130+0000","last_clean":"2024-04-25T11:30:13.332130+0000","last_became_active":"2024-04-25T11:30:11.262389+0000","last_became_peered":"2024-04-25T11:30:11.262389+0000","last_unstale":"2024-04-25T11:30:13.332130+0000","last_undegraded":"2024-04-25T11:30:13.332130+0000","last_fullsized":"2024-04-25T11:30:13.332130+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:44.476276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383515+0000","last_change":"2024-04-25T11:30:12.313164+0000","last_active":"2024-04-25T11:30:14.383515+0000","last_peered":"2024-04-25T11:30:14.383515+0000","last_clean":"2024-04-25T11:30:14.383515+0000","last_became_active":"2024-04-25T11:30:12.311004+0000","last_became_peered":"2024-04-25T11:30:12.311004+0000","last_unstale":"2024-04-25T11:30:14.383515+0000","last_undegraded":"2024-04-25T11:30:14.383515+0000","last_fullsized":"2024-04-25T11:30:14.383515+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:08.835414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.300245+0000","last_change":"2024-04-25T11:30:11.272497+0000","last_active":"2024-04-25T11:30:13.300245+0000","last_peered":"2024-04-25T11:30:13.300245+0000","last_clean":"2024-04-25T11:30:13.300245+0000","last_became_active":"2024-04-25T11:30:11.272150+0000","last_became_peered":"2024-04-25T11:30:11.272150+0000","last_unstale":"2024-04-25T11:30:13.300245+0000","last_undegraded":"2024-04-25T11:30:13.300245+0000","last_fullsized":"2024-04-25T11:30:13.300245+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:09.285233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331581+0000","last_change":"2024-04-25T11:30:12.326351+0000","last_active":"2024-04-25T11:30:13.331581+0000","last_peered":"2024-04-25T11:30:13.331581+0000","last_clean":"2024-04-25T11:30:13.331581+0000","last_became_active":"2024-04-25T11:30:12.326055+0000","last_became_peered":"2024-04-25T11:30:12.326055+0000","last_unstale":"2024-04-25T11:30:13.331581+0000","last_undegraded":"2024-04-25T11:30:13.331581+0000","last_fullsized":"2024-04-25T11:30:13.331581+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:55.817801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383980+0000","last_change":"2024-04-25T11:30:11.280682+0000","last_active":"2024-04-25T11:30:14.383980+0000","last_peered":"2024-04-25T11:30:14.383980+0000","last_clean":"2024-04-25T11:30:14.383980+0000","last_became_active":"2024-04-25T11:30:11.279049+0000","last_became_peered":"2024-04-25T11:30:11.279049+0000","last_unstale":"2024-04-25T11:30:14.383980+0000","last_undegraded":"2024-04-25T11:30:14.383980+0000","last_fullsized":"2024-04-25T11:30:14.383980+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:40.524169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305810+0000","last_change":"2024-04-25T11:30:12.307371+0000","last_active":"2024-04-25T11:30:13.305810+0000","last_peered":"2024-04-25T11:30:13.305810+0000","last_clean":"2024-04-25T11:30:13.305810+0000","last_became_active":"2024-04-25T11:30:12.307030+0000","last_became_peered":"2024-04-25T11:30:12.307030+0000","last_unstale":"2024-04-25T11:30:13.305810+0000","last_undegraded":"2024-04-25T11:30:13.305810+0000","last_fullsized":"2024-04-25T11:30:13.305810+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:20.319325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305016+0000","last_change":"2024-04-25T11:30:11.267439+0000","last_active":"2024-04-25T11:30:13.305016+0000","last_peered":"2024-04-25T11:30:13.305016+0000","last_clean":"2024-04-25T11:30:13.305016+0000","last_became_active":"2024-04-25T11:30:11.267110+0000","last_became_peered":"2024-04-25T11:30:11.267110+0000","last_unstale":"2024-04-25T11:30:13.305016+0000","last_undegraded":"2024-04-25T11:30:13.305016+0000","last_fullsized":"2024-04-25T11:30:13.305016+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:30.677292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305244+0000","last_change":"2024-04-25T11:30:12.311247+0000","last_active":"2024-04-25T11:30:13.305244+0000","last_peered":"2024-04-25T11:30:13.305244+0000","last_clean":"2024-04-25T11:30:13.305244+0000","last_became_active":"2024-04-25T11:30:12.310904+0000","last_became_peered":"2024-04-25T11:30:12.310904+0000","last_unstale":"2024-04-25T11:30:13.305244+0000","last_undegraded":"2024-04-25T11:30:13.305244+0000","last_fullsized":"2024-04-25T11:30:13.305244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:26.480643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385339+0000","last_change":"2024-04-25T11:30:11.272789+0000","last_active":"2024-04-25T11:30:14.385339+0000","last_peered":"2024-04-25T11:30:14.385339+0000","last_clean":"2024-04-25T11:30:14.385339+0000","last_became_active":"2024-04-25T11:30:11.272445+0000","last_became_peered":"2024-04-25T11:30:11.272445+0000","last_unstale":"2024-04-25T11:30:14.385339+0000","last_undegraded":"2024-04-25T11:30:14.385339+0000","last_fullsized":"2024-04-25T11:30:14.385339+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.849463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326363+0000","last_change":"2024-04-25T11:30:12.317391+0000","last_active":"2024-04-25T11:30:13.326363+0000","last_peered":"2024-04-25T11:30:13.326363+0000","last_clean":"2024-04-25T11:30:13.326363+0000","last_became_active":"2024-04-25T11:30:12.316343+0000","last_became_peered":"2024-04-25T11:30:12.316343+0000","last_unstale":"2024-04-25T11:30:13.326363+0000","last_undegraded":"2024-04-25T11:30:13.326363+0000","last_fullsized":"2024-04-25T11:30:13.326363+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:31.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304619+0000","last_change":"2024-04-25T11:30:11.264837+0000","last_active":"2024-04-25T11:30:13.304619+0000","last_peered":"2024-04-25T11:30:13.304619+0000","last_clean":"2024-04-25T11:30:13.304619+0000","last_became_active":"2024-04-25T11:30:11.264166+0000","last_became_peered":"2024-04-25T11:30:11.264166+0000","last_unstale":"2024-04-25T11:30:13.304619+0000","last_undegraded":"2024-04-25T11:30:13.304619+0000","last_fullsized":"2024-04-25T11:30:13.304619+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332105+0000","last_change":"2024-04-25T11:30:12.315841+0000","last_active":"2024-04-25T11:30:13.332105+0000","last_peered":"2024-04-25T11:30:13.332105+0000","last_clean":"2024-04-25T11:30:13.332105+0000","last_became_active":"2024-04-25T11:30:12.315087+0000","last_became_peered":"2024-04-25T11:30:12.315087+0000","last_unstale":"2024-04-25T11:30:13.332105+0000","last_undegraded":"2024-04-25T11:30:13.332105+0000","last_fullsized":"2024-04-25T11:30:13.332105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:16.661201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385485+0000","last_change":"2024-04-25T11:30:11.273982+0000","last_active":"2024-04-25T11:30:14.385485+0000","last_peered":"2024-04-25T11:30:14.385485+0000","last_clean":"2024-04-25T11:30:14.385485+0000","last_became_active":"2024-04-25T11:30:11.272233+0000","last_became_peered":"2024-04-25T11:30:11.272233+0000","last_unstale":"2024-04-25T11:30:14.385485+0000","last_undegraded":"2024-04-25T11:30:14.385485+0000","last_fullsized":"2024-04-25T11:30:14.385485+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:34.401573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302691+0000","last_change":"2024-04-25T11:30:12.294967+0000","last_active":"2024-04-25T11:30:13.302691+0000","last_peered":"2024-04-25T11:30:13.302691+0000","last_clean":"2024-04-25T11:30:13.302691+0000","last_became_active":"2024-04-25T11:30:12.294545+0000","last_became_peered":"2024-04-25T11:30:12.294545+0000","last_unstale":"2024-04-25T11:30:13.302691+0000","last_undegraded":"2024-04-25T11:30:13.302691+0000","last_fullsized":"2024-04-25T11:30:13.302691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:20.342509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383236+0000","last_change":"2024-04-25T11:30:11.278510+0000","last_active":"2024-04-25T11:30:14.383236+0000","last_peered":"2024-04-25T11:30:14.383236+0000","last_clean":"2024-04-25T11:30:14.383236+0000","last_became_active":"2024-04-25T11:30:11.278136+0000","last_became_peered":"2024-04-25T11:30:11.278136+0000","last_unstale":"2024-04-25T11:30:14.383236+0000","last_undegraded":"2024-04-25T11:30:14.383236+0000","last_fullsized":"2024-04-25T11:30:14.383236+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:44.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383439+0000","last_change":"2024-04-25T11:30:12.326783+0000","last_active":"2024-04-25T11:30:14.383439+0000","last_peered":"2024-04-25T11:30:14.383439+0000","last_clean":"2024-04-25T11:30:14.383439+0000","last_became_active":"2024-04-25T11:30:12.325755+0000","last_became_peered":"2024-04-25T11:30:12.325755+0000","last_unstale":"2024-04-25T11:30:14.383439+0000","last_undegraded":"2024-04-25T11:30:14.383439+0000","last_fullsized":"2024-04-25T11:30:14.383439+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:26.480643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.298822+0000","last_change":"2024-04-25T11:30:11.269357+0000","last_active":"2024-04-25T11:30:13.298822+0000","last_peered":"2024-04-25T11:30:13.298822+0000","last_clean":"2024-04-25T11:30:13.298822+0000","last_became_active":"2024-04-25T11:30:11.268999+0000","last_became_peered":"2024-04-25T11:30:11.268999+0000","last_unstale":"2024-04-25T11:30:13.298822+0000","last_undegraded":"2024-04-25T11:30:13.298822+0000","last_fullsized":"2024-04-25T11:30:13.298822+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.382544+0000","last_change":"2024-04-25T11:30:12.316130+0000","last_active":"2024-04-25T11:30:14.382544+0000","last_peered":"2024-04-25T11:30:14.382544+0000","last_clean":"2024-04-25T11:30:14.382544+0000","last_became_active":"2024-04-25T11:30:12.315793+0000","last_became_peered":"2024-04-25T11:30:12.315793+0000","last_unstale":"2024-04-25T11:30:14.382544+0000","last_undegraded":"2024-04-25T11:30:14.382544+0000","last_fullsized":"2024-04-25T11:30:14.382544+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:03.212211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382778+0000","last_change":"2024-04-25T11:30:11.279239+0000","last_active":"2024-04-25T11:30:13.382778+0000","last_peered":"2024-04-25T11:30:13.382778+0000","last_clean":"2024-04-25T11:30:13.382778+0000","last_became_active":"2024-04-25T11:30:11.278889+0000","last_became_peered":"2024-04-25T11:30:11.278889+0000","last_unstale":"2024-04-25T11:30:13.382778+0000","last_undegraded":"2024-04-25T11:30:13.382778+0000","last_fullsized":"2024-04-25T11:30:13.382778+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:12.914763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302765+0000","last_change":"2024-04-25T11:30:12.306734+0000","last_active":"2024-04-25T11:30:13.302765+0000","last_peered":"2024-04-25T11:30:13.302765+0000","last_clean":"2024-04-25T11:30:13.302765+0000","last_became_active":"2024-04-25T11:30:12.306242+0000","last_became_peered":"2024-04-25T11:30:12.306242+0000","last_unstale":"2024-04-25T11:30:13.302765+0000","last_undegraded":"2024-04-25T11:30:13.302765+0000","last_fullsized":"2024-04-25T11:30:13.302765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.314933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383946+0000","last_change":"2024-04-25T11:30:11.274461+0000","last_active":"2024-04-25T11:30:14.383946+0000","last_peered":"2024-04-25T11:30:14.383946+0000","last_clean":"2024-04-25T11:30:14.383946+0000","last_became_active":"2024-04-25T11:30:11.272811+0000","last_became_peered":"2024-04-25T11:30:11.272811+0000","last_unstale":"2024-04-25T11:30:14.383946+0000","last_undegraded":"2024-04-25T11:30:14.383946+0000","last_fullsized":"2024-04-25T11:30:14.383946+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:01.676380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383627+0000","last_change":"2024-04-25T11:30:12.321405+0000","last_active":"2024-04-25T11:30:14.383627+0000","last_peered":"2024-04-25T11:30:14.383627+0000","last_clean":"2024-04-25T11:30:14.383627+0000","last_became_active":"2024-04-25T11:30:12.319648+0000","last_became_peered":"2024-04-25T11:30:12.319648+0000","last_unstale":"2024-04-25T11:30:14.383627+0000","last_undegraded":"2024-04-25T11:30:14.383627+0000","last_fullsized":"2024-04-25T11:30:14.383627+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:20.319325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326873+0000","last_change":"2024-04-25T11:30:11.281057+0000","last_active":"2024-04-25T11:30:13.326873+0000","last_peered":"2024-04-25T11:30:13.326873+0000","last_clean":"2024-04-25T11:30:13.326873+0000","last_became_active":"2024-04-25T11:30:11.279339+0000","last_became_peered":"2024-04-25T11:30:11.279339+0000","last_unstale":"2024-04-25T11:30:13.326873+0000","last_undegraded":"2024-04-25T11:30:13.326873+0000","last_fullsized":"2024-04-25T11:30:13.326873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:54.034098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384062+0000","last_change":"2024-04-25T11:30:12.315868+0000","last_active":"2024-04-25T11:30:14.384062+0000","last_peered":"2024-04-25T11:30:14.384062+0000","last_clean":"2024-04-25T11:30:14.384062+0000","last_became_active":"2024-04-25T11:30:12.315511+0000","last_became_peered":"2024-04-25T11:30:12.315511+0000","last_unstale":"2024-04-25T11:30:14.384062+0000","last_undegraded":"2024-04-25T11:30:14.384062+0000","last_fullsized":"2024-04-25T11:30:14.384062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:28.097135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305821+0000","last_change":"2024-04-25T11:30:11.271207+0000","last_active":"2024-04-25T11:30:13.305821+0000","last_peered":"2024-04-25T11:30:13.305821+0000","last_clean":"2024-04-25T11:30:13.305821+0000","last_became_active":"2024-04-25T11:30:11.270083+0000","last_became_peered":"2024-04-25T11:30:11.270083+0000","last_unstale":"2024-04-25T11:30:13.305821+0000","last_undegraded":"2024-04-25T11:30:13.305821+0000","last_fullsized":"2024-04-25T11:30:13.305821+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:40.274271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304313+0000","last_change":"2024-04-25T11:30:12.317980+0000","last_active":"2024-04-25T11:30:13.304313+0000","last_peered":"2024-04-25T11:30:13.304313+0000","last_clean":"2024-04-25T11:30:13.304313+0000","last_became_active":"2024-04-25T11:30:12.317574+0000","last_became_peered":"2024-04-25T11:30:12.317574+0000","last_unstale":"2024-04-25T11:30:13.304313+0000","last_undegraded":"2024-04-25T11:30:13.304313+0000","last_fullsized":"2024-04-25T11:30:13.304313+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:18.987772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304679+0000","last_change":"2024-04-25T11:30:11.276120+0000","last_active":"2024-04-25T11:30:13.304679+0000","last_peered":"2024-04-25T11:30:13.304679+0000","last_clean":"2024-04-25T11:30:13.304679+0000","last_became_active":"2024-04-25T11:30:11.275804+0000","last_became_peered":"2024-04-25T11:30:11.275804+0000","last_unstale":"2024-04-25T11:30:13.304679+0000","last_undegraded":"2024-04-25T11:30:13.304679+0000","last_fullsized":"2024-04-25T11:30:13.304679+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:37.492766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304165+0000","last_change":"2024-04-25T11:30:12.315454+0000","last_active":"2024-04-25T11:30:13.304165+0000","last_peered":"2024-04-25T11:30:13.304165+0000","last_clean":"2024-04-25T11:30:13.304165+0000","last_became_active":"2024-04-25T11:30:12.315114+0000","last_became_peered":"2024-04-25T11:30:12.315114+0000","last_unstale":"2024-04-25T11:30:13.304165+0000","last_undegraded":"2024-04-25T11:30:13.304165+0000","last_fullsized":"2024-04-25T11:30:13.304165+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:13.920881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382972+0000","last_change":"2024-04-25T11:30:11.279419+0000","last_active":"2024-04-25T11:30:13.382972+0000","last_peered":"2024-04-25T11:30:13.382972+0000","last_clean":"2024-04-25T11:30:13.382972+0000","last_became_active":"2024-04-25T11:30:11.279074+0000","last_became_peered":"2024-04-25T11:30:11.279074+0000","last_unstale":"2024-04-25T11:30:13.382972+0000","last_undegraded":"2024-04-25T11:30:13.382972+0000","last_fullsized":"2024-04-25T11:30:13.382972+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:27.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384953+0000","last_change":"2024-04-25T11:30:12.320762+0000","last_active":"2024-04-25T11:30:14.384953+0000","last_peered":"2024-04-25T11:30:14.384953+0000","last_clean":"2024-04-25T11:30:14.384953+0000","last_became_active":"2024-04-25T11:30:12.320207+0000","last_became_peered":"2024-04-25T11:30:12.320207+0000","last_unstale":"2024-04-25T11:30:14.384953+0000","last_undegraded":"2024-04-25T11:30:14.384953+0000","last_fullsized":"2024-04-25T11:30:14.384953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.953885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384575+0000","last_change":"2024-04-25T11:30:11.273562+0000","last_active":"2024-04-25T11:30:14.384575+0000","last_peered":"2024-04-25T11:30:14.384575+0000","last_clean":"2024-04-25T11:30:14.384575+0000","last_became_active":"2024-04-25T11:30:11.273187+0000","last_became_peered":"2024-04-25T11:30:11.273187+0000","last_unstale":"2024-04-25T11:30:14.384575+0000","last_undegraded":"2024-04-25T11:30:14.384575+0000","last_fullsized":"2024-04-25T11:30:14.384575+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:37.187391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382848+0000","last_change":"2024-04-25T11:30:12.310068+0000","last_active":"2024-04-25T11:30:13.382848+0000","last_peered":"2024-04-25T11:30:13.382848+0000","last_clean":"2024-04-25T11:30:13.382848+0000","last_became_active":"2024-04-25T11:30:12.309236+0000","last_became_peered":"2024-04-25T11:30:12.309236+0000","last_unstale":"2024-04-25T11:30:13.382848+0000","last_undegraded":"2024-04-25T11:30:13.382848+0000","last_fullsized":"2024-04-25T11:30:13.382848+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.298956+0000","last_change":"2024-04-25T11:30:11.260102+0000","last_active":"2024-04-25T11:30:13.298956+0000","last_peered":"2024-04-25T11:30:13.298956+0000","last_clean":"2024-04-25T11:30:13.298956+0000","last_became_active":"2024-04-25T11:30:11.259712+0000","last_became_peered":"2024-04-25T11:30:11.259712+0000","last_unstale":"2024-04-25T11:30:13.298956+0000","last_undegraded":"2024-04-25T11:30:13.298956+0000","last_fullsized":"2024-04-25T11:30:13.298956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:31.383947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325790+0000","last_change":"2024-04-25T11:30:12.303739+0000","last_active":"2024-04-25T11:30:13.325790+0000","last_peered":"2024-04-25T11:30:13.325790+0000","last_clean":"2024-04-25T11:30:13.325790+0000","last_became_active":"2024-04-25T11:30:12.303424+0000","last_became_peered":"2024-04-25T11:30:12.303424+0000","last_unstale":"2024-04-25T11:30:13.325790+0000","last_undegraded":"2024-04-25T11:30:13.325790+0000","last_fullsized":"2024-04-25T11:30:13.325790+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:38.400496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384442+0000","last_change":"2024-04-25T11:30:11.274700+0000","last_active":"2024-04-25T11:30:14.384442+0000","last_peered":"2024-04-25T11:30:14.384442+0000","last_clean":"2024-04-25T11:30:14.384442+0000","last_became_active":"2024-04-25T11:30:11.274283+0000","last_became_peered":"2024-04-25T11:30:11.274283+0000","last_unstale":"2024-04-25T11:30:14.384442+0000","last_undegraded":"2024-04-25T11:30:14.384442+0000","last_fullsized":"2024-04-25T11:30:14.384442+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.081180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326156+0000","last_change":"2024-04-25T11:30:12.314120+0000","last_active":"2024-04-25T11:30:13.326156+0000","last_peered":"2024-04-25T11:30:13.326156+0000","last_clean":"2024-04-25T11:30:13.326156+0000","last_became_active":"2024-04-25T11:30:12.311763+0000","last_became_peered":"2024-04-25T11:30:12.311763+0000","last_unstale":"2024-04-25T11:30:13.326156+0000","last_undegraded":"2024-04-25T11:30:13.326156+0000","last_fullsized":"2024-04-25T11:30:13.326156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:44.693103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331207+0000","last_change":"2024-04-25T11:30:11.266163+0000","last_active":"2024-04-25T11:30:13.331207+0000","last_peered":"2024-04-25T11:30:13.331207+0000","last_clean":"2024-04-25T11:30:13.331207+0000","last_became_active":"2024-04-25T11:30:11.265810+0000","last_became_peered":"2024-04-25T11:30:11.265810+0000","last_unstale":"2024-04-25T11:30:13.331207+0000","last_undegraded":"2024-04-25T11:30:13.331207+0000","last_fullsized":"2024-04-25T11:30:13.331207+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:20.553784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332339+0000","last_change":"2024-04-25T11:30:12.325830+0000","last_active":"2024-04-25T11:30:13.332339+0000","last_peered":"2024-04-25T11:30:13.332339+0000","last_clean":"2024-04-25T11:30:13.332339+0000","last_became_active":"2024-04-25T11:30:12.325485+0000","last_became_peered":"2024-04-25T11:30:12.325485+0000","last_unstale":"2024-04-25T11:30:13.332339+0000","last_undegraded":"2024-04-25T11:30:13.332339+0000","last_fullsized":"2024-04-25T11:30:13.332339+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:19.598915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.327135+0000","last_change":"2024-04-25T11:30:11.274375+0000","last_active":"2024-04-25T11:30:13.327135+0000","last_peered":"2024-04-25T11:30:13.327135+0000","last_clean":"2024-04-25T11:30:13.327135+0000","last_became_active":"2024-04-25T11:30:11.273757+0000","last_became_peered":"2024-04-25T11:30:11.273757+0000","last_unstale":"2024-04-25T11:30:13.327135+0000","last_undegraded":"2024-04-25T11:30:13.327135+0000","last_fullsized":"2024-04-25T11:30:13.327135+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:29.071007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385239+0000","last_change":"2024-04-25T11:30:12.314257+0000","last_active":"2024-04-25T11:30:14.385239+0000","last_peered":"2024-04-25T11:30:14.385239+0000","last_clean":"2024-04-25T11:30:14.385239+0000","last_became_active":"2024-04-25T11:30:12.313948+0000","last_became_peered":"2024-04-25T11:30:12.313948+0000","last_unstale":"2024-04-25T11:30:14.385239+0000","last_undegraded":"2024-04-25T11:30:14.385239+0000","last_fullsized":"2024-04-25T11:30:14.385239+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:33.210451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299751+0000","last_change":"2024-04-25T11:30:11.263543+0000","last_active":"2024-04-25T11:30:13.299751+0000","last_peered":"2024-04-25T11:30:13.299751+0000","last_clean":"2024-04-25T11:30:13.299751+0000","last_became_active":"2024-04-25T11:30:11.263181+0000","last_became_peered":"2024-04-25T11:30:11.263181+0000","last_unstale":"2024-04-25T11:30:13.299751+0000","last_undegraded":"2024-04-25T11:30:13.299751+0000","last_fullsized":"2024-04-25T11:30:13.299751+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.090545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332211+0000","last_change":"2024-04-25T11:30:12.316106+0000","last_active":"2024-04-25T11:30:13.332211+0000","last_peered":"2024-04-25T11:30:13.332211+0000","last_clean":"2024-04-25T11:30:13.332211+0000","last_became_active":"2024-04-25T11:30:12.315513+0000","last_became_peered":"2024-04-25T11:30:12.315513+0000","last_unstale":"2024-04-25T11:30:13.332211+0000","last_undegraded":"2024-04-25T11:30:13.332211+0000","last_fullsized":"2024-04-25T11:30:13.332211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:30.085036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383644+0000","last_change":"2024-04-25T11:30:11.264133+0000","last_active":"2024-04-25T11:30:14.383644+0000","last_peered":"2024-04-25T11:30:14.383644+0000","last_clean":"2024-04-25T11:30:14.383644+0000","last_became_active":"2024-04-25T11:30:11.263693+0000","last_became_peered":"2024-04-25T11:30:11.263693+0000","last_unstale":"2024-04-25T11:30:14.383644+0000","last_undegraded":"2024-04-25T11:30:14.383644+0000","last_fullsized":"2024-04-25T11:30:14.383644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.388286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384676+0000","last_change":"2024-04-25T11:30:12.321099+0000","last_active":"2024-04-25T11:30:14.384676+0000","last_peered":"2024-04-25T11:30:14.384676+0000","last_clean":"2024-04-25T11:30:14.384676+0000","last_became_active":"2024-04-25T11:30:12.319720+0000","last_became_peered":"2024-04-25T11:30:12.319720+0000","last_unstale":"2024-04-25T11:30:14.384676+0000","last_undegraded":"2024-04-25T11:30:14.384676+0000","last_fullsized":"2024-04-25T11:30:14.384676+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:18.863099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299587+0000","last_change":"2024-04-25T11:30:11.278565+0000","last_active":"2024-04-25T11:30:13.299587+0000","last_peered":"2024-04-25T11:30:13.299587+0000","last_clean":"2024-04-25T11:30:13.299587+0000","last_became_active":"2024-04-25T11:30:11.278040+0000","last_became_peered":"2024-04-25T11:30:11.278040+0000","last_unstale":"2024-04-25T11:30:13.299587+0000","last_undegraded":"2024-04-25T11:30:13.299587+0000","last_fullsized":"2024-04-25T11:30:13.299587+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.982156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325709+0000","last_change":"2024-04-25T11:30:12.301766+0000","last_active":"2024-04-25T11:30:13.325709+0000","last_peered":"2024-04-25T11:30:13.325709+0000","last_clean":"2024-04-25T11:30:13.325709+0000","last_became_active":"2024-04-25T11:30:12.300217+0000","last_became_peered":"2024-04-25T11:30:12.300217+0000","last_unstale":"2024-04-25T11:30:13.325709+0000","last_undegraded":"2024-04-25T11:30:13.325709+0000","last_fullsized":"2024-04-25T11:30:13.325709+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:06.266221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331567+0000","last_change":"2024-04-25T11:30:11.272593+0000","last_active":"2024-04-25T11:30:13.331567+0000","last_peered":"2024-04-25T11:30:13.331567+0000","last_clean":"2024-04-25T11:30:13.331567+0000","last_became_active":"2024-04-25T11:30:11.272280+0000","last_became_peered":"2024-04-25T11:30:11.272280+0000","last_unstale":"2024-04-25T11:30:13.331567+0000","last_undegraded":"2024-04-25T11:30:13.331567+0000","last_fullsized":"2024-04-25T11:30:13.331567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:29.649052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302087+0000","last_change":"2024-04-25T11:30:12.296593+0000","last_active":"2024-04-25T11:30:13.302087+0000","last_peered":"2024-04-25T11:30:13.302087+0000","last_clean":"2024-04-25T11:30:13.302087+0000","last_became_active":"2024-04-25T11:30:12.296265+0000","last_became_peered":"2024-04-25T11:30:12.296265+0000","last_unstale":"2024-04-25T11:30:13.302087+0000","last_undegraded":"2024-04-25T11:30:13.302087+0000","last_fullsized":"2024-04-25T11:30:13.302087+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:01.898568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383156+0000","last_change":"2024-04-25T11:30:11.271824+0000","last_active":"2024-04-25T11:30:13.383156+0000","last_peered":"2024-04-25T11:30:13.383156+0000","last_clean":"2024-04-25T11:30:13.383156+0000","last_became_active":"2024-04-25T11:30:11.270927+0000","last_became_peered":"2024-04-25T11:30:11.270927+0000","last_unstale":"2024-04-25T11:30:13.383156+0000","last_undegraded":"2024-04-25T11:30:13.383156+0000","last_fullsized":"2024-04-25T11:30:13.383156+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:19.313207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.303079+0000","last_change":"2024-04-25T11:30:12.298869+0000","last_active":"2024-04-25T11:30:13.303079+0000","last_peered":"2024-04-25T11:30:13.303079+0000","last_clean":"2024-04-25T11:30:13.303079+0000","last_became_active":"2024-04-25T11:30:12.298520+0000","last_became_peered":"2024-04-25T11:30:12.298520+0000","last_unstale":"2024-04-25T11:30:13.303079+0000","last_undegraded":"2024-04-25T11:30:13.303079+0000","last_fullsized":"2024-04-25T11:30:13.303079+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:32.067456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.382576+0000","last_change":"2024-04-25T11:30:11.275287+0000","last_active":"2024-04-25T11:30:14.382576+0000","last_peered":"2024-04-25T11:30:14.382576+0000","last_clean":"2024-04-25T11:30:14.382576+0000","last_became_active":"2024-04-25T11:30:11.274574+0000","last_became_peered":"2024-04-25T11:30:11.274574+0000","last_unstale":"2024-04-25T11:30:14.382576+0000","last_undegraded":"2024-04-25T11:30:14.382576+0000","last_fullsized":"2024-04-25T11:30:14.382576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:39.602919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.327025+0000","last_change":"2024-04-25T11:30:12.310311+0000","last_active":"2024-04-25T11:30:13.327025+0000","last_peered":"2024-04-25T11:30:13.327025+0000","last_clean":"2024-04-25T11:30:13.327025+0000","last_became_active":"2024-04-25T11:30:12.309966+0000","last_became_peered":"2024-04-25T11:30:12.309966+0000","last_unstale":"2024-04-25T11:30:13.327025+0000","last_undegraded":"2024-04-25T11:30:13.327025+0000","last_fullsized":"2024-04-25T11:30:13.327025+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:41.192970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332242+0000","last_change":"2024-04-25T11:30:11.272554+0000","last_active":"2024-04-25T11:30:13.332242+0000","last_peered":"2024-04-25T11:30:13.332242+0000","last_clean":"2024-04-25T11:30:13.332242+0000","last_became_active":"2024-04-25T11:30:11.272199+0000","last_became_peered":"2024-04-25T11:30:11.272199+0000","last_unstale":"2024-04-25T11:30:13.332242+0000","last_undegraded":"2024-04-25T11:30:13.332242+0000","last_fullsized":"2024-04-25T11:30:13.332242+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:32.204333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302393+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:30:13.302393+0000","last_peered":"2024-04-25T11:30:13.302393+0000","last_clean":"2024-04-25T11:30:13.302393+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:30:13.302393+0000","last_undegraded":"2024-04-25T11:30:13.302393+0000","last_fullsized":"2024-04-25T11:30:13.302393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":239436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":589824,"data_stored":240018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":241106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":241014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":585728,"data_stored":239492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":829840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":589824,"data_stored":240074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":829792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:30:22.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:30:22.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.758+0000 7f908d000640 1 Processor -- start 2024-04-25T11:30:22.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.758+0000 7f908d000640 1 -- start start 2024-04-25T11:30:22.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.760+0000 7f908d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f90880b4ea0 0x7f90880cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.760+0000 7f908d000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90880d1230 con 0x7f9088144370 2024-04-25T11:30:22.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.760+0000 7f908d000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90880d06b0 con 0x7f908806c5c0 2024-04-25T11:30:22.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.760+0000 7f908d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f90880d0990 con 0x7f90880b4ea0 2024-04-25T11:30:22.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.761+0000 7f9087e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9088144370 0x7f908815e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38840/0 (socket says 172.21.15.19:38840) 2024-04-25T11:30:22.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.761+0000 7f9087e00640 1 -- 172.21.15.19:0/403850440 learned_addr learned my addr 172.21.15.19:0/403850440 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.761+0000 7f907fe00640 1 --2- 172.21.15.19:0/403850440 >> v2:172.21.15.169:3300/0 conn(0x7f90880b4ea0 0x7f90880cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.761+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 480562465 0 0) 0x7f90880d1230 con 0x7f9088144370 2024-04-25T11:30:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.761+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f906c003520 con 0x7f9088144370 2024-04-25T11:30:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 669579559 0 0) 0x7f906c003520 con 0x7f9088144370 2024-04-25T11:30:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 >> v2:172.21.15.169:3300/0 conn(0x7f90880b4ea0 msgr2=0x7f90880cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 --2- 172.21.15.19:0/403850440 >> v2:172.21.15.169:3300/0 conn(0x7f90880b4ea0 0x7f90880cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 >> v1:172.21.15.169:6789/0 conn(0x7f908806c5c0 legacy=0x7f90880b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90880d0c70 con 0x7f9088144370 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f90800019f0 con 0x7f9088144370 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 >> v1:172.21.15.19:6789/0 conn(0x7f9088144370 legacy=0x7f908815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c003a00 0x7f906c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f906c004800 0x7f906c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906c005270 0x7f906c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90880d0990 con 0x7f906c004800 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f906c00c190 con 0x7f906c005270 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f906c00c390 con 0x7f906c003a00 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.762+0000 7f907fe00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f906c004800 0x7f906c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f9087e00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906c005270 0x7f906c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f9087400640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c003a00 0x7f906c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f907fe00640 1 -- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c003a00 msgr2=0x7f906c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:22.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f907fe00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c003a00 0x7f906c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f907fe00640 1 -- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906c005270 msgr2=0x7f906c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:22.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f907fe00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f906c005270 0x7f906c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f907fe00640 1 -- 172.21.15.19:0/403850440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c003520 con 0x7f906c004800 2024-04-25T11:30:22.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.763+0000 7f9087e00640 1 -- 172.21.15.19:0/403850440 reap_dead start 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.764+0000 7f907fe00640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f906c004800 0x7f906c004c10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9070009ac0 tx=0x7f906c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e66e3a4b03c0001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.764+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070036070 con 0x7f906c004800 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.764+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9070031400 con 0x7f906c004800 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.764+0000 7f9086a00640 1 -- 172.21.15.19:0/403850440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070009690 con 0x7f906c004800 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.765+0000 7f908d000640 1 -- 172.21.15.19:0/403850440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f906c004800 msgr2=0x7f906c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.765+0000 7f908d000640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f906c004800 0x7f906c004c10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9070009ac0 tx=0x7f906c005830 comp rx=0 tx=0).stop 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.765+0000 7f908d000640 1 -- 172.21.15.19:0/403850440 shutdown_connections 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.765+0000 7f908d000640 1 --2- 172.21.15.19:0/403850440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f906c004800 0x7f906c004c10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.765+0000 7f908d000640 1 -- 172.21.15.19:0/403850440 >> 172.21.15.19:0/403850440 conn(0x7f90880d1720 msgr2=0x7f90880d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.765+0000 7f908d000640 1 -- 172.21.15.19:0/403850440 shutdown_connections 2024-04-25T11:30:22.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.765+0000 7f908d000640 1 -- 172.21.15.19:0/403850440 wait complete. 2024-04-25T11:30:22.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.768+0000 7f908d000640 1 Processor -- start 2024-04-25T11:30:22.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.768+0000 7f908d000640 1 -- start start 2024-04-25T11:30:22.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f908d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c004800 0x7f90881613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f908d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9088162440 0x7f908815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f908d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9088144780 0x7f90881f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f908d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90880d0450 con 0x7f9088162440 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f908d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f90880d02d0 con 0x7f9088144780 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f908d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9088147760 con 0x7f906c004800 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f907fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9088162440 0x7f908815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f907fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9088162440 0x7f908815d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34286/0 (socket says 172.21.15.19:34286) 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f907fe00640 1 -- 172.21.15.19:0/2317565681 learned_addr learned my addr 172.21.15.19:0/2317565681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.770+0000 7f9087400640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c004800 0x7f90881613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f9087e00640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9088144780 0x7f90881f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907fe00640 1 -- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c004800 msgr2=0x7f90881613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907fe00640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c004800 0x7f90881613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907fe00640 1 -- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9088144780 msgr2=0x7f90881f98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907fe00640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9088144780 0x7f90881f98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907fe00640 1 -- 172.21.15.19:0/2317565681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9088160310 con 0x7f9088162440 2024-04-25T11:30:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907fe00640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9088162440 0x7f908815d530 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9070009be0 tx=0x7f9070004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907f400640 1 -- 172.21.15.19:0/2317565681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907003f020 con 0x7f9088162440 2024-04-25T11:30:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907f400640 1 -- 172.21.15.19:0/2317565681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9070037040 con 0x7f9088162440 2024-04-25T11:30:22.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f907f400640 1 -- 172.21.15.19:0/2317565681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070036070 con 0x7f9088162440 2024-04-25T11:30:22.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.771+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90881605f0 con 0x7f9088162440 2024-04-25T11:30:22.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.772+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9088160bb0 con 0x7f9088162440 2024-04-25T11:30:22.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.773+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9088050740 con 0x7f9088162440 2024-04-25T11:30:22.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.774+0000 7f907f400640 1 -- 172.21.15.19:0/2317565681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9070042020 con 0x7f9088162440 2024-04-25T11:30:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.775+0000 7f907f400640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90640744b0 0x7f90640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:22.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.776+0000 7f907f400640 1 -- 172.21.15.19:0/2317565681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f90700b54e0 con 0x7f9088162440 2024-04-25T11:30:22.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.776+0000 7f9087400640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90640744b0 0x7f90640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:22.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.777+0000 7f9087400640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90640744b0 0x7f90640769c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f908806bc20 tx=0x7f9078009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:22.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.780+0000 7f907f400640 1 -- 172.21.15.19:0/2317565681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9070049030 con 0x7f9088162440 2024-04-25T11:30:22.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.979+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90881416b0 con 0x7f90640744b0 2024-04-25T11:30:22.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:22.998+0000 7f907f400640 1 -- 172.21.15.19:0/2317565681 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7f90881416b0 con 0x7f90640744b0 2024-04-25T11:30:22.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:22.993 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:30:22.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90640744b0 msgr2=0x7f90640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:22.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90640744b0 0x7f90640769c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f908806bc20 tx=0x7f9078009040 comp rx=0 tx=0).stop 2024-04-25T11:30:22.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9088162440 msgr2=0x7f908815d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:22.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9088162440 0x7f908815d530 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9070009be0 tx=0x7f9070004730 comp rx=0 tx=0).stop 2024-04-25T11:30:22.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 shutdown_connections 2024-04-25T11:30:22.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f90640744b0 0x7f90640769c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9088144780 0x7f90881f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9088162440 0x7f908815d530 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 --2- 172.21.15.19:0/2317565681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f906c004800 0x7f90881613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:22.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.003+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 >> 172.21.15.19:0/2317565681 conn(0x7f90880d1720 msgr2=0x7f9088072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:22.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.004+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 shutdown_connections 2024-04-25T11:30:22.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.004+0000 7f908d000640 1 -- 172.21.15.19:0/2317565681 wait complete. 2024-04-25T11:30:23.020 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-04-25T11:30:22.508021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234436,"kb_used_data":5768,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754740284,"statfs":{"total":773094113280,"available":772854050816,"internally_reserved":0,"allocated":5906432,"data_stored":3100772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004355"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.300247+0000","last_change":"2024-04-25T11:30:12.328001+0000","last_active":"2024-04-25T11:30:13.300247+0000","last_peered":"2024-04-25T11:30:13.300247+0000","last_clean":"2024-04-25T11:30:13.300247+0000","last_became_active":"2024-04-25T11:30:12.327555+0000","last_became_peered":"2024-04-25T11:30:12.327555+0000","last_unstale":"2024-04-25T11:30:13.300247+0000","last_undegraded":"2024-04-25T11:30:13.300247+0000","last_fullsized":"2024-04-25T11:30:13.300247+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302159+0000","last_change":"2024-04-25T11:30:12.316817+0000","last_active":"2024-04-25T11:30:13.302159+0000","last_peered":"2024-04-25T11:30:13.302159+0000","last_clean":"2024-04-25T11:30:13.302159+0000","last_became_active":"2024-04-25T11:30:12.311768+0000","last_became_peered":"2024-04-25T11:30:12.311768+0000","last_unstale":"2024-04-25T11:30:13.302159+0000","last_undegraded":"2024-04-25T11:30:13.302159+0000","last_fullsized":"2024-04-25T11:30:13.302159+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:02.682497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304083+0000","last_change":"2024-04-25T11:30:12.315596+0000","last_active":"2024-04-25T11:30:13.304083+0000","last_peered":"2024-04-25T11:30:13.304083+0000","last_clean":"2024-04-25T11:30:13.304083+0000","last_became_active":"2024-04-25T11:30:12.315259+0000","last_became_peered":"2024-04-25T11:30:12.315259+0000","last_unstale":"2024-04-25T11:30:13.304083+0000","last_undegraded":"2024-04-25T11:30:13.304083+0000","last_fullsized":"2024-04-25T11:30:13.304083+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:03.212211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326300+0000","last_change":"2024-04-25T11:30:12.306582+0000","last_active":"2024-04-25T11:30:13.326300+0000","last_peered":"2024-04-25T11:30:13.326300+0000","last_clean":"2024-04-25T11:30:13.326300+0000","last_became_active":"2024-04-25T11:30:12.306118+0000","last_became_peered":"2024-04-25T11:30:12.306118+0000","last_unstale":"2024-04-25T11:30:13.326300+0000","last_undegraded":"2024-04-25T11:30:13.326300+0000","last_fullsized":"2024-04-25T11:30:13.326300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:55.746271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299288+0000","last_change":"2024-04-25T11:30:12.320809+0000","last_active":"2024-04-25T11:30:13.299288+0000","last_peered":"2024-04-25T11:30:13.299288+0000","last_clean":"2024-04-25T11:30:13.299288+0000","last_became_active":"2024-04-25T11:30:12.318568+0000","last_became_peered":"2024-04-25T11:30:12.318568+0000","last_unstale":"2024-04-25T11:30:13.299288+0000","last_undegraded":"2024-04-25T11:30:13.299288+0000","last_fullsized":"2024-04-25T11:30:13.299288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382958+0000","last_change":"2024-04-25T11:30:12.316853+0000","last_active":"2024-04-25T11:30:13.382958+0000","last_peered":"2024-04-25T11:30:13.382958+0000","last_clean":"2024-04-25T11:30:13.382958+0000","last_became_active":"2024-04-25T11:30:12.316509+0000","last_became_peered":"2024-04-25T11:30:12.316509+0000","last_unstale":"2024-04-25T11:30:13.382958+0000","last_undegraded":"2024-04-25T11:30:13.382958+0000","last_fullsized":"2024-04-25T11:30:13.382958+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:41.530287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331346+0000","last_change":"2024-04-25T11:30:12.323818+0000","last_active":"2024-04-25T11:30:13.331346+0000","last_peered":"2024-04-25T11:30:13.331346+0000","last_clean":"2024-04-25T11:30:13.331346+0000","last_became_active":"2024-04-25T11:30:12.322940+0000","last_became_peered":"2024-04-25T11:30:12.322940+0000","last_unstale":"2024-04-25T11:30:13.331346+0000","last_undegraded":"2024-04-25T11:30:13.331346+0000","last_fullsized":"2024-04-25T11:30:13.331346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:26.075584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325316+0000","last_change":"2024-04-25T11:30:12.311821+0000","last_active":"2024-04-25T11:30:13.325316+0000","last_peered":"2024-04-25T11:30:13.325316+0000","last_clean":"2024-04-25T11:30:13.325316+0000","last_became_active":"2024-04-25T11:30:12.311464+0000","last_became_peered":"2024-04-25T11:30:12.311464+0000","last_unstale":"2024-04-25T11:30:13.325316+0000","last_undegraded":"2024-04-25T11:30:13.325316+0000","last_fullsized":"2024-04-25T11:30:13.325316+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:57.890018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383837+0000","last_change":"2024-04-25T11:30:12.316458+0000","last_active":"2024-04-25T11:30:13.383837+0000","last_peered":"2024-04-25T11:30:13.383837+0000","last_clean":"2024-04-25T11:30:13.383837+0000","last_became_active":"2024-04-25T11:30:12.315188+0000","last_became_peered":"2024-04-25T11:30:12.315188+0000","last_unstale":"2024-04-25T11:30:13.383837+0000","last_undegraded":"2024-04-25T11:30:13.383837+0000","last_fullsized":"2024-04-25T11:30:13.383837+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:40.087298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385057+0000","last_change":"2024-04-25T11:30:12.311133+0000","last_active":"2024-04-25T11:30:14.385057+0000","last_peered":"2024-04-25T11:30:14.385057+0000","last_clean":"2024-04-25T11:30:14.385057+0000","last_became_active":"2024-04-25T11:30:12.310670+0000","last_became_peered":"2024-04-25T11:30:12.310670+0000","last_unstale":"2024-04-25T11:30:14.385057+0000","last_undegraded":"2024-04-25T11:30:14.385057+0000","last_fullsized":"2024-04-25T11:30:14.385057+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:18.525690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326805+0000","last_change":"2024-04-25T11:30:12.319414+0000","last_active":"2024-04-25T11:30:13.326805+0000","last_peered":"2024-04-25T11:30:13.326805+0000","last_clean":"2024-04-25T11:30:13.326805+0000","last_became_active":"2024-04-25T11:30:12.317760+0000","last_became_peered":"2024-04-25T11:30:12.317760+0000","last_unstale":"2024-04-25T11:30:13.326805+0000","last_undegraded":"2024-04-25T11:30:13.326805+0000","last_fullsized":"2024-04-25T11:30:13.326805+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:25.408994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.303853+0000","last_change":"2024-04-25T11:30:12.306558+0000","last_active":"2024-04-25T11:30:13.303853+0000","last_peered":"2024-04-25T11:30:13.303853+0000","last_clean":"2024-04-25T11:30:13.303853+0000","last_became_active":"2024-04-25T11:30:12.306159+0000","last_became_peered":"2024-04-25T11:30:12.306159+0000","last_unstale":"2024-04-25T11:30:13.303853+0000","last_undegraded":"2024-04-25T11:30:13.303853+0000","last_fullsized":"2024-04-25T11:30:13.303853+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:16.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382778+0000","last_change":"2024-04-25T11:30:12.315317+0000","last_active":"2024-04-25T11:30:13.382778+0000","last_peered":"2024-04-25T11:30:13.382778+0000","last_clean":"2024-04-25T11:30:13.382778+0000","last_became_active":"2024-04-25T11:30:12.314919+0000","last_became_peered":"2024-04-25T11:30:12.314919+0000","last_unstale":"2024-04-25T11:30:13.382778+0000","last_undegraded":"2024-04-25T11:30:13.382778+0000","last_fullsized":"2024-04-25T11:30:13.382778+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:24.913125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325520+0000","last_change":"2024-04-25T11:30:12.315951+0000","last_active":"2024-04-25T11:30:13.325520+0000","last_peered":"2024-04-25T11:30:13.325520+0000","last_clean":"2024-04-25T11:30:13.325520+0000","last_became_active":"2024-04-25T11:30:12.315574+0000","last_became_peered":"2024-04-25T11:30:12.315574+0000","last_unstale":"2024-04-25T11:30:13.325520+0000","last_undegraded":"2024-04-25T11:30:13.325520+0000","last_fullsized":"2024-04-25T11:30:13.325520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:03.301131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326215+0000","last_change":"2024-04-25T11:30:12.304594+0000","last_active":"2024-04-25T11:30:13.326215+0000","last_peered":"2024-04-25T11:30:13.326215+0000","last_clean":"2024-04-25T11:30:13.326215+0000","last_became_active":"2024-04-25T11:30:12.304268+0000","last_became_peered":"2024-04-25T11:30:12.304268+0000","last_unstale":"2024-04-25T11:30:13.326215+0000","last_undegraded":"2024-04-25T11:30:13.326215+0000","last_fullsized":"2024-04-25T11:30:13.326215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:01.543749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.298741+0000","last_change":"2024-04-25T11:30:12.320656+0000","last_active":"2024-04-25T11:30:13.298741+0000","last_peered":"2024-04-25T11:30:13.298741+0000","last_clean":"2024-04-25T11:30:13.298741+0000","last_became_active":"2024-04-25T11:30:12.319501+0000","last_became_peered":"2024-04-25T11:30:12.319501+0000","last_unstale":"2024-04-25T11:30:13.298741+0000","last_undegraded":"2024-04-25T11:30:13.298741+0000","last_fullsized":"2024-04-25T11:30:13.298741+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:02.682497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305002+0000","last_change":"2024-04-25T11:30:12.312204+0000","last_active":"2024-04-25T11:30:13.305002+0000","last_peered":"2024-04-25T11:30:13.305002+0000","last_clean":"2024-04-25T11:30:13.305002+0000","last_became_active":"2024-04-25T11:30:12.311798+0000","last_became_peered":"2024-04-25T11:30:12.311798+0000","last_unstale":"2024-04-25T11:30:13.305002+0000","last_undegraded":"2024-04-25T11:30:13.305002+0000","last_fullsized":"2024-04-25T11:30:13.305002+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:28.188588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305555+0000","last_change":"2024-04-25T11:30:12.292655+0000","last_active":"2024-04-25T11:30:13.305555+0000","last_peered":"2024-04-25T11:30:13.305555+0000","last_clean":"2024-04-25T11:30:13.305555+0000","last_became_active":"2024-04-25T11:30:12.292173+0000","last_became_peered":"2024-04-25T11:30:12.292173+0000","last_unstale":"2024-04-25T11:30:13.305555+0000","last_undegraded":"2024-04-25T11:30:13.305555+0000","last_fullsized":"2024-04-25T11:30:13.305555+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:38.180760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299459+0000","last_change":"2024-04-25T11:30:12.318490+0000","last_active":"2024-04-25T11:30:13.299459+0000","last_peered":"2024-04-25T11:30:13.299459+0000","last_clean":"2024-04-25T11:30:13.299459+0000","last_became_active":"2024-04-25T11:30:12.317442+0000","last_became_peered":"2024-04-25T11:30:12.317442+0000","last_unstale":"2024-04-25T11:30:13.299459+0000","last_undegraded":"2024-04-25T11:30:13.299459+0000","last_fullsized":"2024-04-25T11:30:13.299459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.314933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385166+0000","last_change":"2024-04-25T11:30:12.311311+0000","last_active":"2024-04-25T11:30:14.385166+0000","last_peered":"2024-04-25T11:30:14.385166+0000","last_clean":"2024-04-25T11:30:14.385166+0000","last_became_active":"2024-04-25T11:30:12.310846+0000","last_became_peered":"2024-04-25T11:30:12.310846+0000","last_unstale":"2024-04-25T11:30:14.385166+0000","last_undegraded":"2024-04-25T11:30:14.385166+0000","last_fullsized":"2024-04-25T11:30:14.385166+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:21.559902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326029+0000","last_change":"2024-04-25T11:30:12.299502+0000","last_active":"2024-04-25T11:30:13.326029+0000","last_peered":"2024-04-25T11:30:13.326029+0000","last_clean":"2024-04-25T11:30:13.326029+0000","last_became_active":"2024-04-25T11:30:12.297154+0000","last_became_peered":"2024-04-25T11:30:12.297154+0000","last_unstale":"2024-04-25T11:30:13.326029+0000","last_undegraded":"2024-04-25T11:30:13.326029+0000","last_fullsized":"2024-04-25T11:30:13.326029+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:31.195209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.382679+0000","last_change":"2024-04-25T11:30:12.318465+0000","last_active":"2024-04-25T11:30:14.382679+0000","last_peered":"2024-04-25T11:30:14.382679+0000","last_clean":"2024-04-25T11:30:14.382679+0000","last_became_active":"2024-04-25T11:30:12.318076+0000","last_became_peered":"2024-04-25T11:30:12.318076+0000","last_unstale":"2024-04-25T11:30:14.382679+0000","last_undegraded":"2024-04-25T11:30:14.382679+0000","last_fullsized":"2024-04-25T11:30:14.382679+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:13.920881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331477+0000","last_change":"2024-04-25T11:30:12.308548+0000","last_active":"2024-04-25T11:30:13.331477+0000","last_peered":"2024-04-25T11:30:13.331477+0000","last_clean":"2024-04-25T11:30:13.331477+0000","last_became_active":"2024-04-25T11:30:12.308148+0000","last_became_peered":"2024-04-25T11:30:12.308148+0000","last_unstale":"2024-04-25T11:30:13.331477+0000","last_undegraded":"2024-04-25T11:30:13.331477+0000","last_fullsized":"2024-04-25T11:30:13.331477+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:19.436394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302162+0000","last_change":"2024-04-25T11:30:12.312781+0000","last_active":"2024-04-25T11:30:13.302162+0000","last_peered":"2024-04-25T11:30:13.302162+0000","last_clean":"2024-04-25T11:30:13.302162+0000","last_became_active":"2024-04-25T11:30:12.312453+0000","last_became_peered":"2024-04-25T11:30:12.312453+0000","last_unstale":"2024-04-25T11:30:13.302162+0000","last_undegraded":"2024-04-25T11:30:13.302162+0000","last_fullsized":"2024-04-25T11:30:13.302162+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:10.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.300794+0000","last_change":"2024-04-25T11:30:12.325014+0000","last_active":"2024-04-25T11:30:13.300794+0000","last_peered":"2024-04-25T11:30:13.300794+0000","last_clean":"2024-04-25T11:30:13.300794+0000","last_became_active":"2024-04-25T11:30:12.324660+0000","last_became_peered":"2024-04-25T11:30:12.324660+0000","last_unstale":"2024-04-25T11:30:13.300794+0000","last_undegraded":"2024-04-25T11:30:13.300794+0000","last_fullsized":"2024-04-25T11:30:13.300794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326645+0000","last_change":"2024-04-25T11:30:12.312808+0000","last_active":"2024-04-25T11:30:13.326645+0000","last_peered":"2024-04-25T11:30:13.326645+0000","last_clean":"2024-04-25T11:30:13.326645+0000","last_became_active":"2024-04-25T11:30:12.312389+0000","last_became_peered":"2024-04-25T11:30:12.312389+0000","last_unstale":"2024-04-25T11:30:13.326645+0000","last_undegraded":"2024-04-25T11:30:13.326645+0000","last_fullsized":"2024-04-25T11:30:13.326645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:32.464656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331879+0000","last_change":"2024-04-25T11:30:12.300829+0000","last_active":"2024-04-25T11:30:13.331879+0000","last_peered":"2024-04-25T11:30:13.331879+0000","last_clean":"2024-04-25T11:30:13.331879+0000","last_became_active":"2024-04-25T11:30:12.300457+0000","last_became_peered":"2024-04-25T11:30:12.300457+0000","last_unstale":"2024-04-25T11:30:13.331879+0000","last_undegraded":"2024-04-25T11:30:13.331879+0000","last_fullsized":"2024-04-25T11:30:13.331879+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:24.995116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331922+0000","last_change":"2024-04-25T11:30:12.306296+0000","last_active":"2024-04-25T11:30:13.331922+0000","last_peered":"2024-04-25T11:30:13.331922+0000","last_clean":"2024-04-25T11:30:13.331922+0000","last_became_active":"2024-04-25T11:30:12.305922+0000","last_became_peered":"2024-04-25T11:30:12.305922+0000","last_unstale":"2024-04-25T11:30:13.331922+0000","last_undegraded":"2024-04-25T11:30:13.331922+0000","last_fullsized":"2024-04-25T11:30:13.331922+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:32.527270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299595+0000","last_change":"2024-04-25T11:30:12.314344+0000","last_active":"2024-04-25T11:30:13.299595+0000","last_peered":"2024-04-25T11:30:13.299595+0000","last_clean":"2024-04-25T11:30:13.299595+0000","last_became_active":"2024-04-25T11:30:12.314033+0000","last_became_peered":"2024-04-25T11:30:12.314033+0000","last_unstale":"2024-04-25T11:30:13.299595+0000","last_undegraded":"2024-04-25T11:30:13.299595+0000","last_fullsized":"2024-04-25T11:30:13.299595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:32.067456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.301820+0000","last_change":"2024-04-25T11:30:12.306312+0000","last_active":"2024-04-25T11:30:13.301820+0000","last_peered":"2024-04-25T11:30:13.301820+0000","last_clean":"2024-04-25T11:30:13.301820+0000","last_became_active":"2024-04-25T11:30:12.305928+0000","last_became_peered":"2024-04-25T11:30:12.305928+0000","last_unstale":"2024-04-25T11:30:13.301820+0000","last_undegraded":"2024-04-25T11:30:13.301820+0000","last_fullsized":"2024-04-25T11:30:13.301820+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:03.855581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383300+0000","last_change":"2024-04-25T11:30:12.309122+0000","last_active":"2024-04-25T11:30:13.383300+0000","last_peered":"2024-04-25T11:30:13.383300+0000","last_clean":"2024-04-25T11:30:13.383300+0000","last_became_active":"2024-04-25T11:30:12.308689+0000","last_became_peered":"2024-04-25T11:30:12.308689+0000","last_unstale":"2024-04-25T11:30:13.383300+0000","last_undegraded":"2024-04-25T11:30:13.383300+0000","last_fullsized":"2024-04-25T11:30:13.383300+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:35.407691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325495+0000","last_change":"2024-04-25T11:30:12.307262+0000","last_active":"2024-04-25T11:30:13.325495+0000","last_peered":"2024-04-25T11:30:13.325495+0000","last_clean":"2024-04-25T11:30:13.325495+0000","last_became_active":"2024-04-25T11:30:12.305770+0000","last_became_peered":"2024-04-25T11:30:12.305770+0000","last_unstale":"2024-04-25T11:30:13.325495+0000","last_undegraded":"2024-04-25T11:30:13.325495+0000","last_fullsized":"2024-04-25T11:30:13.325495+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:15.978494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383539+0000","last_change":"2024-04-25T11:30:12.324170+0000","last_active":"2024-04-25T11:30:14.383539+0000","last_peered":"2024-04-25T11:30:14.383539+0000","last_clean":"2024-04-25T11:30:14.383539+0000","last_became_active":"2024-04-25T11:30:12.323509+0000","last_became_peered":"2024-04-25T11:30:12.323509+0000","last_unstale":"2024-04-25T11:30:14.383539+0000","last_undegraded":"2024-04-25T11:30:14.383539+0000","last_fullsized":"2024-04-25T11:30:14.383539+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:38.180760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.384030+0000","last_change":"2024-04-25T11:30:11.269060+0000","last_active":"2024-04-25T11:30:13.384030+0000","last_peered":"2024-04-25T11:30:13.384030+0000","last_clean":"2024-04-25T11:30:13.384030+0000","last_became_active":"2024-04-25T11:30:11.266286+0000","last_became_peered":"2024-04-25T11:30:11.266286+0000","last_unstale":"2024-04-25T11:30:13.384030+0000","last_undegraded":"2024-04-25T11:30:13.384030+0000","last_fullsized":"2024-04-25T11:30:13.384030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:25.474525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.301123+0000","last_change":"2024-04-25T11:30:12.319935+0000","last_active":"2024-04-25T11:30:13.301123+0000","last_peered":"2024-04-25T11:30:13.301123+0000","last_clean":"2024-04-25T11:30:13.301123+0000","last_became_active":"2024-04-25T11:30:12.319486+0000","last_became_peered":"2024-04-25T11:30:12.319486+0000","last_unstale":"2024-04-25T11:30:13.301123+0000","last_undegraded":"2024-04-25T11:30:13.301123+0000","last_fullsized":"2024-04-25T11:30:13.301123+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:18.987772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331959+0000","last_change":"2024-04-25T11:30:11.257450+0000","last_active":"2024-04-25T11:30:13.331959+0000","last_peered":"2024-04-25T11:30:13.331959+0000","last_clean":"2024-04-25T11:30:13.331959+0000","last_became_active":"2024-04-25T11:30:11.257097+0000","last_became_peered":"2024-04-25T11:30:11.257097+0000","last_unstale":"2024-04-25T11:30:13.331959+0000","last_undegraded":"2024-04-25T11:30:13.331959+0000","last_fullsized":"2024-04-25T11:30:13.331959+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.501573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304090+0000","last_change":"2024-04-25T11:30:12.318193+0000","last_active":"2024-04-25T11:30:13.304090+0000","last_peered":"2024-04-25T11:30:13.304090+0000","last_clean":"2024-04-25T11:30:13.304090+0000","last_became_active":"2024-04-25T11:30:12.317611+0000","last_became_peered":"2024-04-25T11:30:12.317611+0000","last_unstale":"2024-04-25T11:30:13.304090+0000","last_undegraded":"2024-04-25T11:30:13.304090+0000","last_fullsized":"2024-04-25T11:30:13.304090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383612+0000","last_change":"2024-04-25T11:30:11.271679+0000","last_active":"2024-04-25T11:30:13.383612+0000","last_peered":"2024-04-25T11:30:13.383612+0000","last_clean":"2024-04-25T11:30:13.383612+0000","last_became_active":"2024-04-25T11:30:11.270812+0000","last_became_peered":"2024-04-25T11:30:11.270812+0000","last_unstale":"2024-04-25T11:30:13.383612+0000","last_undegraded":"2024-04-25T11:30:13.383612+0000","last_fullsized":"2024-04-25T11:30:13.383612+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:01.243237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326908+0000","last_change":"2024-04-25T11:30:12.306099+0000","last_active":"2024-04-25T11:30:13.326908+0000","last_peered":"2024-04-25T11:30:13.326908+0000","last_clean":"2024-04-25T11:30:13.326908+0000","last_became_active":"2024-04-25T11:30:12.305699+0000","last_became_peered":"2024-04-25T11:30:12.305699+0000","last_unstale":"2024-04-25T11:30:13.326908+0000","last_undegraded":"2024-04-25T11:30:13.326908+0000","last_fullsized":"2024-04-25T11:30:13.326908+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:04.320407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331887+0000","last_change":"2024-04-25T11:30:11.259259+0000","last_active":"2024-04-25T11:30:13.331887+0000","last_peered":"2024-04-25T11:30:13.331887+0000","last_clean":"2024-04-25T11:30:13.331887+0000","last_became_active":"2024-04-25T11:30:11.258919+0000","last_became_peered":"2024-04-25T11:30:11.258919+0000","last_unstale":"2024-04-25T11:30:13.331887+0000","last_undegraded":"2024-04-25T11:30:13.331887+0000","last_fullsized":"2024-04-25T11:30:13.331887+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:49.077411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331346+0000","last_change":"2024-04-25T11:30:12.310915+0000","last_active":"2024-04-25T11:30:13.331346+0000","last_peered":"2024-04-25T11:30:13.331346+0000","last_clean":"2024-04-25T11:30:13.331346+0000","last_became_active":"2024-04-25T11:30:12.310580+0000","last_became_peered":"2024-04-25T11:30:12.310580+0000","last_unstale":"2024-04-25T11:30:13.331346+0000","last_undegraded":"2024-04-25T11:30:13.331346+0000","last_fullsized":"2024-04-25T11:30:13.331346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:04.359361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383505+0000","last_change":"2024-04-25T11:30:11.278985+0000","last_active":"2024-04-25T11:30:13.383505+0000","last_peered":"2024-04-25T11:30:13.383505+0000","last_clean":"2024-04-25T11:30:13.383505+0000","last_became_active":"2024-04-25T11:30:11.273156+0000","last_became_peered":"2024-04-25T11:30:11.273156+0000","last_unstale":"2024-04-25T11:30:13.383505+0000","last_undegraded":"2024-04-25T11:30:13.383505+0000","last_fullsized":"2024-04-25T11:30:13.383505+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:37.174642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384624+0000","last_change":"2024-04-25T11:30:12.318436+0000","last_active":"2024-04-25T11:30:14.384624+0000","last_peered":"2024-04-25T11:30:14.384624+0000","last_clean":"2024-04-25T11:30:14.384624+0000","last_became_active":"2024-04-25T11:30:12.318108+0000","last_became_peered":"2024-04-25T11:30:12.318108+0000","last_unstale":"2024-04-25T11:30:14.384624+0000","last_undegraded":"2024-04-25T11:30:14.384624+0000","last_fullsized":"2024-04-25T11:30:14.384624+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:26.537536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332130+0000","last_change":"2024-04-25T11:30:11.263150+0000","last_active":"2024-04-25T11:30:13.332130+0000","last_peered":"2024-04-25T11:30:13.332130+0000","last_clean":"2024-04-25T11:30:13.332130+0000","last_became_active":"2024-04-25T11:30:11.262389+0000","last_became_peered":"2024-04-25T11:30:11.262389+0000","last_unstale":"2024-04-25T11:30:13.332130+0000","last_undegraded":"2024-04-25T11:30:13.332130+0000","last_fullsized":"2024-04-25T11:30:13.332130+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:44.476276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383515+0000","last_change":"2024-04-25T11:30:12.313164+0000","last_active":"2024-04-25T11:30:14.383515+0000","last_peered":"2024-04-25T11:30:14.383515+0000","last_clean":"2024-04-25T11:30:14.383515+0000","last_became_active":"2024-04-25T11:30:12.311004+0000","last_became_peered":"2024-04-25T11:30:12.311004+0000","last_unstale":"2024-04-25T11:30:14.383515+0000","last_undegraded":"2024-04-25T11:30:14.383515+0000","last_fullsized":"2024-04-25T11:30:14.383515+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:08.835414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.300245+0000","last_change":"2024-04-25T11:30:11.272497+0000","last_active":"2024-04-25T11:30:13.300245+0000","last_peered":"2024-04-25T11:30:13.300245+0000","last_clean":"2024-04-25T11:30:13.300245+0000","last_became_active":"2024-04-25T11:30:11.272150+0000","last_became_peered":"2024-04-25T11:30:11.272150+0000","last_unstale":"2024-04-25T11:30:13.300245+0000","last_undegraded":"2024-04-25T11:30:13.300245+0000","last_fullsized":"2024-04-25T11:30:13.300245+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:09.285233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331581+0000","last_change":"2024-04-25T11:30:12.326351+0000","last_active":"2024-04-25T11:30:13.331581+0000","last_peered":"2024-04-25T11:30:13.331581+0000","last_clean":"2024-04-25T11:30:13.331581+0000","last_became_active":"2024-04-25T11:30:12.326055+0000","last_became_peered":"2024-04-25T11:30:12.326055+0000","last_unstale":"2024-04-25T11:30:13.331581+0000","last_undegraded":"2024-04-25T11:30:13.331581+0000","last_fullsized":"2024-04-25T11:30:13.331581+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:55.817801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383980+0000","last_change":"2024-04-25T11:30:11.280682+0000","last_active":"2024-04-25T11:30:14.383980+0000","last_peered":"2024-04-25T11:30:14.383980+0000","last_clean":"2024-04-25T11:30:14.383980+0000","last_became_active":"2024-04-25T11:30:11.279049+0000","last_became_peered":"2024-04-25T11:30:11.279049+0000","last_unstale":"2024-04-25T11:30:14.383980+0000","last_undegraded":"2024-04-25T11:30:14.383980+0000","last_fullsized":"2024-04-25T11:30:14.383980+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:40.524169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305810+0000","last_change":"2024-04-25T11:30:12.307371+0000","last_active":"2024-04-25T11:30:13.305810+0000","last_peered":"2024-04-25T11:30:13.305810+0000","last_clean":"2024-04-25T11:30:13.305810+0000","last_became_active":"2024-04-25T11:30:12.307030+0000","last_became_peered":"2024-04-25T11:30:12.307030+0000","last_unstale":"2024-04-25T11:30:13.305810+0000","last_undegraded":"2024-04-25T11:30:13.305810+0000","last_fullsized":"2024-04-25T11:30:13.305810+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:20.319325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305016+0000","last_change":"2024-04-25T11:30:11.267439+0000","last_active":"2024-04-25T11:30:13.305016+0000","last_peered":"2024-04-25T11:30:13.305016+0000","last_clean":"2024-04-25T11:30:13.305016+0000","last_became_active":"2024-04-25T11:30:11.267110+0000","last_became_peered":"2024-04-25T11:30:11.267110+0000","last_unstale":"2024-04-25T11:30:13.305016+0000","last_undegraded":"2024-04-25T11:30:13.305016+0000","last_fullsized":"2024-04-25T11:30:13.305016+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:30.677292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305244+0000","last_change":"2024-04-25T11:30:12.311247+0000","last_active":"2024-04-25T11:30:13.305244+0000","last_peered":"2024-04-25T11:30:13.305244+0000","last_clean":"2024-04-25T11:30:13.305244+0000","last_became_active":"2024-04-25T11:30:12.310904+0000","last_became_peered":"2024-04-25T11:30:12.310904+0000","last_unstale":"2024-04-25T11:30:13.305244+0000","last_undegraded":"2024-04-25T11:30:13.305244+0000","last_fullsized":"2024-04-25T11:30:13.305244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:26.480643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385339+0000","last_change":"2024-04-25T11:30:11.272789+0000","last_active":"2024-04-25T11:30:14.385339+0000","last_peered":"2024-04-25T11:30:14.385339+0000","last_clean":"2024-04-25T11:30:14.385339+0000","last_became_active":"2024-04-25T11:30:11.272445+0000","last_became_peered":"2024-04-25T11:30:11.272445+0000","last_unstale":"2024-04-25T11:30:14.385339+0000","last_undegraded":"2024-04-25T11:30:14.385339+0000","last_fullsized":"2024-04-25T11:30:14.385339+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:02.849463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326363+0000","last_change":"2024-04-25T11:30:12.317391+0000","last_active":"2024-04-25T11:30:13.326363+0000","last_peered":"2024-04-25T11:30:13.326363+0000","last_clean":"2024-04-25T11:30:13.326363+0000","last_became_active":"2024-04-25T11:30:12.316343+0000","last_became_peered":"2024-04-25T11:30:12.316343+0000","last_unstale":"2024-04-25T11:30:13.326363+0000","last_undegraded":"2024-04-25T11:30:13.326363+0000","last_fullsized":"2024-04-25T11:30:13.326363+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:31.132295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304619+0000","last_change":"2024-04-25T11:30:11.264837+0000","last_active":"2024-04-25T11:30:13.304619+0000","last_peered":"2024-04-25T11:30:13.304619+0000","last_clean":"2024-04-25T11:30:13.304619+0000","last_became_active":"2024-04-25T11:30:11.264166+0000","last_became_peered":"2024-04-25T11:30:11.264166+0000","last_unstale":"2024-04-25T11:30:13.304619+0000","last_undegraded":"2024-04-25T11:30:13.304619+0000","last_fullsized":"2024-04-25T11:30:13.304619+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332105+0000","last_change":"2024-04-25T11:30:12.315841+0000","last_active":"2024-04-25T11:30:13.332105+0000","last_peered":"2024-04-25T11:30:13.332105+0000","last_clean":"2024-04-25T11:30:13.332105+0000","last_became_active":"2024-04-25T11:30:12.315087+0000","last_became_peered":"2024-04-25T11:30:12.315087+0000","last_unstale":"2024-04-25T11:30:13.332105+0000","last_undegraded":"2024-04-25T11:30:13.332105+0000","last_fullsized":"2024-04-25T11:30:13.332105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:16.661201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385485+0000","last_change":"2024-04-25T11:30:11.273982+0000","last_active":"2024-04-25T11:30:14.385485+0000","last_peered":"2024-04-25T11:30:14.385485+0000","last_clean":"2024-04-25T11:30:14.385485+0000","last_became_active":"2024-04-25T11:30:11.272233+0000","last_became_peered":"2024-04-25T11:30:11.272233+0000","last_unstale":"2024-04-25T11:30:14.385485+0000","last_undegraded":"2024-04-25T11:30:14.385485+0000","last_fullsized":"2024-04-25T11:30:14.385485+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:34.401573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302691+0000","last_change":"2024-04-25T11:30:12.294967+0000","last_active":"2024-04-25T11:30:13.302691+0000","last_peered":"2024-04-25T11:30:13.302691+0000","last_clean":"2024-04-25T11:30:13.302691+0000","last_became_active":"2024-04-25T11:30:12.294545+0000","last_became_peered":"2024-04-25T11:30:12.294545+0000","last_unstale":"2024-04-25T11:30:13.302691+0000","last_undegraded":"2024-04-25T11:30:13.302691+0000","last_fullsized":"2024-04-25T11:30:13.302691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:20.342509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383236+0000","last_change":"2024-04-25T11:30:11.278510+0000","last_active":"2024-04-25T11:30:14.383236+0000","last_peered":"2024-04-25T11:30:14.383236+0000","last_clean":"2024-04-25T11:30:14.383236+0000","last_became_active":"2024-04-25T11:30:11.278136+0000","last_became_peered":"2024-04-25T11:30:11.278136+0000","last_unstale":"2024-04-25T11:30:14.383236+0000","last_undegraded":"2024-04-25T11:30:14.383236+0000","last_fullsized":"2024-04-25T11:30:14.383236+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:44.198728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383439+0000","last_change":"2024-04-25T11:30:12.326783+0000","last_active":"2024-04-25T11:30:14.383439+0000","last_peered":"2024-04-25T11:30:14.383439+0000","last_clean":"2024-04-25T11:30:14.383439+0000","last_became_active":"2024-04-25T11:30:12.325755+0000","last_became_peered":"2024-04-25T11:30:12.325755+0000","last_unstale":"2024-04-25T11:30:14.383439+0000","last_undegraded":"2024-04-25T11:30:14.383439+0000","last_fullsized":"2024-04-25T11:30:14.383439+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:26.480643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.298822+0000","last_change":"2024-04-25T11:30:11.269357+0000","last_active":"2024-04-25T11:30:13.298822+0000","last_peered":"2024-04-25T11:30:13.298822+0000","last_clean":"2024-04-25T11:30:13.298822+0000","last_became_active":"2024-04-25T11:30:11.268999+0000","last_became_peered":"2024-04-25T11:30:11.268999+0000","last_unstale":"2024-04-25T11:30:13.298822+0000","last_undegraded":"2024-04-25T11:30:13.298822+0000","last_fullsized":"2024-04-25T11:30:13.298822+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:53.504717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.382544+0000","last_change":"2024-04-25T11:30:12.316130+0000","last_active":"2024-04-25T11:30:14.382544+0000","last_peered":"2024-04-25T11:30:14.382544+0000","last_clean":"2024-04-25T11:30:14.382544+0000","last_became_active":"2024-04-25T11:30:12.315793+0000","last_became_peered":"2024-04-25T11:30:12.315793+0000","last_unstale":"2024-04-25T11:30:14.382544+0000","last_undegraded":"2024-04-25T11:30:14.382544+0000","last_fullsized":"2024-04-25T11:30:14.382544+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:03.212211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382778+0000","last_change":"2024-04-25T11:30:11.279239+0000","last_active":"2024-04-25T11:30:13.382778+0000","last_peered":"2024-04-25T11:30:13.382778+0000","last_clean":"2024-04-25T11:30:13.382778+0000","last_became_active":"2024-04-25T11:30:11.278889+0000","last_became_peered":"2024-04-25T11:30:11.278889+0000","last_unstale":"2024-04-25T11:30:13.382778+0000","last_undegraded":"2024-04-25T11:30:13.382778+0000","last_fullsized":"2024-04-25T11:30:13.382778+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:12.914763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302765+0000","last_change":"2024-04-25T11:30:12.306734+0000","last_active":"2024-04-25T11:30:13.302765+0000","last_peered":"2024-04-25T11:30:13.302765+0000","last_clean":"2024-04-25T11:30:13.302765+0000","last_became_active":"2024-04-25T11:30:12.306242+0000","last_became_peered":"2024-04-25T11:30:12.306242+0000","last_unstale":"2024-04-25T11:30:13.302765+0000","last_undegraded":"2024-04-25T11:30:13.302765+0000","last_fullsized":"2024-04-25T11:30:13.302765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.314933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383946+0000","last_change":"2024-04-25T11:30:11.274461+0000","last_active":"2024-04-25T11:30:14.383946+0000","last_peered":"2024-04-25T11:30:14.383946+0000","last_clean":"2024-04-25T11:30:14.383946+0000","last_became_active":"2024-04-25T11:30:11.272811+0000","last_became_peered":"2024-04-25T11:30:11.272811+0000","last_unstale":"2024-04-25T11:30:14.383946+0000","last_undegraded":"2024-04-25T11:30:14.383946+0000","last_fullsized":"2024-04-25T11:30:14.383946+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:01.676380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383627+0000","last_change":"2024-04-25T11:30:12.321405+0000","last_active":"2024-04-25T11:30:14.383627+0000","last_peered":"2024-04-25T11:30:14.383627+0000","last_clean":"2024-04-25T11:30:14.383627+0000","last_became_active":"2024-04-25T11:30:12.319648+0000","last_became_peered":"2024-04-25T11:30:12.319648+0000","last_unstale":"2024-04-25T11:30:14.383627+0000","last_undegraded":"2024-04-25T11:30:14.383627+0000","last_fullsized":"2024-04-25T11:30:14.383627+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:20.319325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326873+0000","last_change":"2024-04-25T11:30:11.281057+0000","last_active":"2024-04-25T11:30:13.326873+0000","last_peered":"2024-04-25T11:30:13.326873+0000","last_clean":"2024-04-25T11:30:13.326873+0000","last_became_active":"2024-04-25T11:30:11.279339+0000","last_became_peered":"2024-04-25T11:30:11.279339+0000","last_unstale":"2024-04-25T11:30:13.326873+0000","last_undegraded":"2024-04-25T11:30:13.326873+0000","last_fullsized":"2024-04-25T11:30:13.326873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:54.034098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384062+0000","last_change":"2024-04-25T11:30:12.315868+0000","last_active":"2024-04-25T11:30:14.384062+0000","last_peered":"2024-04-25T11:30:14.384062+0000","last_clean":"2024-04-25T11:30:14.384062+0000","last_became_active":"2024-04-25T11:30:12.315511+0000","last_became_peered":"2024-04-25T11:30:12.315511+0000","last_unstale":"2024-04-25T11:30:14.384062+0000","last_undegraded":"2024-04-25T11:30:14.384062+0000","last_fullsized":"2024-04-25T11:30:14.384062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:28.097135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.305821+0000","last_change":"2024-04-25T11:30:11.271207+0000","last_active":"2024-04-25T11:30:13.305821+0000","last_peered":"2024-04-25T11:30:13.305821+0000","last_clean":"2024-04-25T11:30:13.305821+0000","last_became_active":"2024-04-25T11:30:11.270083+0000","last_became_peered":"2024-04-25T11:30:11.270083+0000","last_unstale":"2024-04-25T11:30:13.305821+0000","last_undegraded":"2024-04-25T11:30:13.305821+0000","last_fullsized":"2024-04-25T11:30:13.305821+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:40.274271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304313+0000","last_change":"2024-04-25T11:30:12.317980+0000","last_active":"2024-04-25T11:30:13.304313+0000","last_peered":"2024-04-25T11:30:13.304313+0000","last_clean":"2024-04-25T11:30:13.304313+0000","last_became_active":"2024-04-25T11:30:12.317574+0000","last_became_peered":"2024-04-25T11:30:12.317574+0000","last_unstale":"2024-04-25T11:30:13.304313+0000","last_undegraded":"2024-04-25T11:30:13.304313+0000","last_fullsized":"2024-04-25T11:30:13.304313+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:18.987772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304679+0000","last_change":"2024-04-25T11:30:11.276120+0000","last_active":"2024-04-25T11:30:13.304679+0000","last_peered":"2024-04-25T11:30:13.304679+0000","last_clean":"2024-04-25T11:30:13.304679+0000","last_became_active":"2024-04-25T11:30:11.275804+0000","last_became_peered":"2024-04-25T11:30:11.275804+0000","last_unstale":"2024-04-25T11:30:13.304679+0000","last_undegraded":"2024-04-25T11:30:13.304679+0000","last_fullsized":"2024-04-25T11:30:13.304679+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:37.492766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.304165+0000","last_change":"2024-04-25T11:30:12.315454+0000","last_active":"2024-04-25T11:30:13.304165+0000","last_peered":"2024-04-25T11:30:13.304165+0000","last_clean":"2024-04-25T11:30:13.304165+0000","last_became_active":"2024-04-25T11:30:12.315114+0000","last_became_peered":"2024-04-25T11:30:12.315114+0000","last_unstale":"2024-04-25T11:30:13.304165+0000","last_undegraded":"2024-04-25T11:30:13.304165+0000","last_fullsized":"2024-04-25T11:30:13.304165+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:13.920881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382972+0000","last_change":"2024-04-25T11:30:11.279419+0000","last_active":"2024-04-25T11:30:13.382972+0000","last_peered":"2024-04-25T11:30:13.382972+0000","last_clean":"2024-04-25T11:30:13.382972+0000","last_became_active":"2024-04-25T11:30:11.279074+0000","last_became_peered":"2024-04-25T11:30:11.279074+0000","last_unstale":"2024-04-25T11:30:13.382972+0000","last_undegraded":"2024-04-25T11:30:13.382972+0000","last_fullsized":"2024-04-25T11:30:13.382972+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:27.091017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384953+0000","last_change":"2024-04-25T11:30:12.320762+0000","last_active":"2024-04-25T11:30:14.384953+0000","last_peered":"2024-04-25T11:30:14.384953+0000","last_clean":"2024-04-25T11:30:14.384953+0000","last_became_active":"2024-04-25T11:30:12.320207+0000","last_became_peered":"2024-04-25T11:30:12.320207+0000","last_unstale":"2024-04-25T11:30:14.384953+0000","last_undegraded":"2024-04-25T11:30:14.384953+0000","last_fullsized":"2024-04-25T11:30:14.384953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.953885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384575+0000","last_change":"2024-04-25T11:30:11.273562+0000","last_active":"2024-04-25T11:30:14.384575+0000","last_peered":"2024-04-25T11:30:14.384575+0000","last_clean":"2024-04-25T11:30:14.384575+0000","last_became_active":"2024-04-25T11:30:11.273187+0000","last_became_peered":"2024-04-25T11:30:11.273187+0000","last_unstale":"2024-04-25T11:30:14.384575+0000","last_undegraded":"2024-04-25T11:30:14.384575+0000","last_fullsized":"2024-04-25T11:30:14.384575+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:37.187391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.382848+0000","last_change":"2024-04-25T11:30:12.310068+0000","last_active":"2024-04-25T11:30:13.382848+0000","last_peered":"2024-04-25T11:30:13.382848+0000","last_clean":"2024-04-25T11:30:13.382848+0000","last_became_active":"2024-04-25T11:30:12.309236+0000","last_became_peered":"2024-04-25T11:30:12.309236+0000","last_unstale":"2024-04-25T11:30:13.382848+0000","last_undegraded":"2024-04-25T11:30:13.382848+0000","last_fullsized":"2024-04-25T11:30:13.382848+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:38.193508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.298956+0000","last_change":"2024-04-25T11:30:11.260102+0000","last_active":"2024-04-25T11:30:13.298956+0000","last_peered":"2024-04-25T11:30:13.298956+0000","last_clean":"2024-04-25T11:30:13.298956+0000","last_became_active":"2024-04-25T11:30:11.259712+0000","last_became_peered":"2024-04-25T11:30:11.259712+0000","last_unstale":"2024-04-25T11:30:13.298956+0000","last_undegraded":"2024-04-25T11:30:13.298956+0000","last_fullsized":"2024-04-25T11:30:13.298956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:31.383947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325790+0000","last_change":"2024-04-25T11:30:12.303739+0000","last_active":"2024-04-25T11:30:13.325790+0000","last_peered":"2024-04-25T11:30:13.325790+0000","last_clean":"2024-04-25T11:30:13.325790+0000","last_became_active":"2024-04-25T11:30:12.303424+0000","last_became_peered":"2024-04-25T11:30:12.303424+0000","last_unstale":"2024-04-25T11:30:13.325790+0000","last_undegraded":"2024-04-25T11:30:13.325790+0000","last_fullsized":"2024-04-25T11:30:13.325790+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:38.400496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384442+0000","last_change":"2024-04-25T11:30:11.274700+0000","last_active":"2024-04-25T11:30:14.384442+0000","last_peered":"2024-04-25T11:30:14.384442+0000","last_clean":"2024-04-25T11:30:14.384442+0000","last_became_active":"2024-04-25T11:30:11.274283+0000","last_became_peered":"2024-04-25T11:30:11.274283+0000","last_unstale":"2024-04-25T11:30:14.384442+0000","last_undegraded":"2024-04-25T11:30:14.384442+0000","last_fullsized":"2024-04-25T11:30:14.384442+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:39.081180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.326156+0000","last_change":"2024-04-25T11:30:12.314120+0000","last_active":"2024-04-25T11:30:13.326156+0000","last_peered":"2024-04-25T11:30:13.326156+0000","last_clean":"2024-04-25T11:30:13.326156+0000","last_became_active":"2024-04-25T11:30:12.311763+0000","last_became_peered":"2024-04-25T11:30:12.311763+0000","last_unstale":"2024-04-25T11:30:13.326156+0000","last_undegraded":"2024-04-25T11:30:13.326156+0000","last_fullsized":"2024-04-25T11:30:13.326156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:44.693103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331207+0000","last_change":"2024-04-25T11:30:11.266163+0000","last_active":"2024-04-25T11:30:13.331207+0000","last_peered":"2024-04-25T11:30:13.331207+0000","last_clean":"2024-04-25T11:30:13.331207+0000","last_became_active":"2024-04-25T11:30:11.265810+0000","last_became_peered":"2024-04-25T11:30:11.265810+0000","last_unstale":"2024-04-25T11:30:13.331207+0000","last_undegraded":"2024-04-25T11:30:13.331207+0000","last_fullsized":"2024-04-25T11:30:13.331207+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:20.553784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332339+0000","last_change":"2024-04-25T11:30:12.325830+0000","last_active":"2024-04-25T11:30:13.332339+0000","last_peered":"2024-04-25T11:30:13.332339+0000","last_clean":"2024-04-25T11:30:13.332339+0000","last_became_active":"2024-04-25T11:30:12.325485+0000","last_became_peered":"2024-04-25T11:30:12.325485+0000","last_unstale":"2024-04-25T11:30:13.332339+0000","last_undegraded":"2024-04-25T11:30:13.332339+0000","last_fullsized":"2024-04-25T11:30:13.332339+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:19.598915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.327135+0000","last_change":"2024-04-25T11:30:11.274375+0000","last_active":"2024-04-25T11:30:13.327135+0000","last_peered":"2024-04-25T11:30:13.327135+0000","last_clean":"2024-04-25T11:30:13.327135+0000","last_became_active":"2024-04-25T11:30:11.273757+0000","last_became_peered":"2024-04-25T11:30:11.273757+0000","last_unstale":"2024-04-25T11:30:13.327135+0000","last_undegraded":"2024-04-25T11:30:13.327135+0000","last_fullsized":"2024-04-25T11:30:13.327135+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:29.071007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.385239+0000","last_change":"2024-04-25T11:30:12.314257+0000","last_active":"2024-04-25T11:30:14.385239+0000","last_peered":"2024-04-25T11:30:14.385239+0000","last_clean":"2024-04-25T11:30:14.385239+0000","last_became_active":"2024-04-25T11:30:12.313948+0000","last_became_peered":"2024-04-25T11:30:12.313948+0000","last_unstale":"2024-04-25T11:30:14.385239+0000","last_undegraded":"2024-04-25T11:30:14.385239+0000","last_fullsized":"2024-04-25T11:30:14.385239+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:33.210451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299751+0000","last_change":"2024-04-25T11:30:11.263543+0000","last_active":"2024-04-25T11:30:13.299751+0000","last_peered":"2024-04-25T11:30:13.299751+0000","last_clean":"2024-04-25T11:30:13.299751+0000","last_became_active":"2024-04-25T11:30:11.263181+0000","last_became_peered":"2024-04-25T11:30:11.263181+0000","last_unstale":"2024-04-25T11:30:13.299751+0000","last_undegraded":"2024-04-25T11:30:13.299751+0000","last_fullsized":"2024-04-25T11:30:13.299751+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.090545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332211+0000","last_change":"2024-04-25T11:30:12.316106+0000","last_active":"2024-04-25T11:30:13.332211+0000","last_peered":"2024-04-25T11:30:13.332211+0000","last_clean":"2024-04-25T11:30:13.332211+0000","last_became_active":"2024-04-25T11:30:12.315513+0000","last_became_peered":"2024-04-25T11:30:12.315513+0000","last_unstale":"2024-04-25T11:30:13.332211+0000","last_undegraded":"2024-04-25T11:30:13.332211+0000","last_fullsized":"2024-04-25T11:30:13.332211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:30.085036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.383644+0000","last_change":"2024-04-25T11:30:11.264133+0000","last_active":"2024-04-25T11:30:14.383644+0000","last_peered":"2024-04-25T11:30:14.383644+0000","last_clean":"2024-04-25T11:30:14.383644+0000","last_became_active":"2024-04-25T11:30:11.263693+0000","last_became_peered":"2024-04-25T11:30:11.263693+0000","last_unstale":"2024-04-25T11:30:14.383644+0000","last_undegraded":"2024-04-25T11:30:14.383644+0000","last_fullsized":"2024-04-25T11:30:14.383644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.388286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.384676+0000","last_change":"2024-04-25T11:30:12.321099+0000","last_active":"2024-04-25T11:30:14.384676+0000","last_peered":"2024-04-25T11:30:14.384676+0000","last_clean":"2024-04-25T11:30:14.384676+0000","last_became_active":"2024-04-25T11:30:12.319720+0000","last_became_peered":"2024-04-25T11:30:12.319720+0000","last_unstale":"2024-04-25T11:30:14.384676+0000","last_undegraded":"2024-04-25T11:30:14.384676+0000","last_fullsized":"2024-04-25T11:30:14.384676+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:18.863099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.299587+0000","last_change":"2024-04-25T11:30:11.278565+0000","last_active":"2024-04-25T11:30:13.299587+0000","last_peered":"2024-04-25T11:30:13.299587+0000","last_clean":"2024-04-25T11:30:13.299587+0000","last_became_active":"2024-04-25T11:30:11.278040+0000","last_became_peered":"2024-04-25T11:30:11.278040+0000","last_unstale":"2024-04-25T11:30:13.299587+0000","last_undegraded":"2024-04-25T11:30:13.299587+0000","last_fullsized":"2024-04-25T11:30:13.299587+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.982156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.325709+0000","last_change":"2024-04-25T11:30:12.301766+0000","last_active":"2024-04-25T11:30:13.325709+0000","last_peered":"2024-04-25T11:30:13.325709+0000","last_clean":"2024-04-25T11:30:13.325709+0000","last_became_active":"2024-04-25T11:30:12.300217+0000","last_became_peered":"2024-04-25T11:30:12.300217+0000","last_unstale":"2024-04-25T11:30:13.325709+0000","last_undegraded":"2024-04-25T11:30:13.325709+0000","last_fullsized":"2024-04-25T11:30:13.325709+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:06.266221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.331567+0000","last_change":"2024-04-25T11:30:11.272593+0000","last_active":"2024-04-25T11:30:13.331567+0000","last_peered":"2024-04-25T11:30:13.331567+0000","last_clean":"2024-04-25T11:30:13.331567+0000","last_became_active":"2024-04-25T11:30:11.272280+0000","last_became_peered":"2024-04-25T11:30:11.272280+0000","last_unstale":"2024-04-25T11:30:13.331567+0000","last_undegraded":"2024-04-25T11:30:13.331567+0000","last_fullsized":"2024-04-25T11:30:13.331567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:29.649052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302087+0000","last_change":"2024-04-25T11:30:12.296593+0000","last_active":"2024-04-25T11:30:13.302087+0000","last_peered":"2024-04-25T11:30:13.302087+0000","last_clean":"2024-04-25T11:30:13.302087+0000","last_became_active":"2024-04-25T11:30:12.296265+0000","last_became_peered":"2024-04-25T11:30:12.296265+0000","last_unstale":"2024-04-25T11:30:13.302087+0000","last_undegraded":"2024-04-25T11:30:13.302087+0000","last_fullsized":"2024-04-25T11:30:13.302087+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:01.898568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.383156+0000","last_change":"2024-04-25T11:30:11.271824+0000","last_active":"2024-04-25T11:30:13.383156+0000","last_peered":"2024-04-25T11:30:13.383156+0000","last_clean":"2024-04-25T11:30:13.383156+0000","last_became_active":"2024-04-25T11:30:11.270927+0000","last_became_peered":"2024-04-25T11:30:11.270927+0000","last_unstale":"2024-04-25T11:30:13.383156+0000","last_undegraded":"2024-04-25T11:30:13.383156+0000","last_fullsized":"2024-04-25T11:30:13.383156+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:19.313207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.303079+0000","last_change":"2024-04-25T11:30:12.298869+0000","last_active":"2024-04-25T11:30:13.303079+0000","last_peered":"2024-04-25T11:30:13.303079+0000","last_clean":"2024-04-25T11:30:13.303079+0000","last_became_active":"2024-04-25T11:30:12.298520+0000","last_became_peered":"2024-04-25T11:30:12.298520+0000","last_unstale":"2024-04-25T11:30:13.303079+0000","last_undegraded":"2024-04-25T11:30:13.303079+0000","last_fullsized":"2024-04-25T11:30:13.303079+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:32.067456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:14.382576+0000","last_change":"2024-04-25T11:30:11.275287+0000","last_active":"2024-04-25T11:30:14.382576+0000","last_peered":"2024-04-25T11:30:14.382576+0000","last_clean":"2024-04-25T11:30:14.382576+0000","last_became_active":"2024-04-25T11:30:11.274574+0000","last_became_peered":"2024-04-25T11:30:11.274574+0000","last_unstale":"2024-04-25T11:30:14.382576+0000","last_undegraded":"2024-04-25T11:30:14.382576+0000","last_fullsized":"2024-04-25T11:30:14.382576+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:39.602919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.327025+0000","last_change":"2024-04-25T11:30:12.310311+0000","last_active":"2024-04-25T11:30:13.327025+0000","last_peered":"2024-04-25T11:30:13.327025+0000","last_clean":"2024-04-25T11:30:13.327025+0000","last_became_active":"2024-04-25T11:30:12.309966+0000","last_became_peered":"2024-04-25T11:30:12.309966+0000","last_unstale":"2024-04-25T11:30:13.327025+0000","last_undegraded":"2024-04-25T11:30:13.327025+0000","last_fullsized":"2024-04-25T11:30:13.327025+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:11.224057+0000","last_clean_scrub_stamp":"2024-04-25T11:30:11.224057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:41.192970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.332242+0000","last_change":"2024-04-25T11:30:11.272554+0000","last_active":"2024-04-25T11:30:13.332242+0000","last_peered":"2024-04-25T11:30:13.332242+0000","last_clean":"2024-04-25T11:30:13.332242+0000","last_became_active":"2024-04-25T11:30:11.272199+0000","last_became_peered":"2024-04-25T11:30:11.272199+0000","last_unstale":"2024-04-25T11:30:13.332242+0000","last_undegraded":"2024-04-25T11:30:13.332242+0000","last_fullsized":"2024-04-25T11:30:13.332242+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:30:10.217939+0000","last_clean_scrub_stamp":"2024-04-25T11:30:10.217939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:32.204333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T11:30:13.302393+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:30:13.302393+0000","last_peered":"2024-04-25T11:30:13.302393+0000","last_clean":"2024-04-25T11:30:13.302393+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:30:13.302393+0000","last_undegraded":"2024-04-25T11:30:13.302393+0000","last_fullsized":"2024-04-25T11:30:13.302393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":239436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":589824,"data_stored":240018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":241106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":241014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":585728,"data_stored":239492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":829840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":589824,"data_stored":240074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":829792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:30:23.024 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:30:23.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:30:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.372+0000 7f0bf7800640 1 Processor -- start 2024-04-25T11:30:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.372+0000 7f0bf7800640 1 -- start start 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf7800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bf0070630 0x7f0bf00cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf015adf0 0x7f0bf006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf7800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf006df20 con 0x7f0bf006db10 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf7800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf00713f0 con 0x7f0bf0070630 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf00701a0 con 0x7f0bf015adf0 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bf0070630 0x7f0bf00cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bf0070630 0x7f0bf00cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34294/0 (socket says 172.21.15.19:34294) 2024-04-25T11:30:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1887126283 learned_addr learned my addr 172.21.15.19:0/1887126283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:23.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.375+0000 7f0befe00640 1 --2- 172.21.15.19:0/1887126283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf015adf0 0x7f0bf006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.376+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1887126283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf015adf0 msgr2=0x7f0bf006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:23.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.376+0000 7f0bf4e00640 1 --2- 172.21.15.19:0/1887126283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf015adf0 0x7f0bf006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.376+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1887126283 >> v1:172.21.15.169:6789/0 conn(0x7f0bf006db10 legacy=0x7f0bf00b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:23.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.376+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1887126283 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf0071110 con 0x7f0bf0070630 2024-04-25T11:30:23.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.377+0000 7f0bf4e00640 1 --2- 172.21.15.19:0/1887126283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf0070630 0x7f0bf00cec50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e50 tx=0x7f0be002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11371c7aad813e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:23.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.377+0000 7f0bef400640 1 -- 172.21.15.19:0/1887126283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0036070 con 0x7f0bf0070630 2024-04-25T11:30:23.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.377+0000 7f0bef400640 1 -- 172.21.15.19:0/1887126283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be0031600 con 0x7f0bf0070630 2024-04-25T11:30:23.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.377+0000 7f0bef400640 1 -- 172.21.15.19:0/1887126283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be00319d0 con 0x7f0bf0070630 2024-04-25T11:30:23.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 -- 172.21.15.19:0/1887126283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf0070630 msgr2=0x7f0bf00cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1887126283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf0070630 0x7f0bf00cec50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e50 tx=0x7f0be002a630 comp rx=0 tx=0).stop 2024-04-25T11:30:23.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 -- 172.21.15.19:0/1887126283 shutdown_connections 2024-04-25T11:30:23.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1887126283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf015adf0 0x7f0bf006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1887126283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf0070630 0x7f0bf00cec50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 -- 172.21.15.19:0/1887126283 >> 172.21.15.19:0/1887126283 conn(0x7f0bf0073090 msgr2=0x7f0bf00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 -- 172.21.15.19:0/1887126283 shutdown_connections 2024-04-25T11:30:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.378+0000 7f0bf7800640 1 -- 172.21.15.19:0/1887126283 wait complete. 2024-04-25T11:30:23.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.381+0000 7f0bf7800640 1 Processor -- start 2024-04-25T11:30:23.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.381+0000 7f0bf7800640 1 -- start start 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf7800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf006db10 0x7f0bf0140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf0070630 0x7f0bf01534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf7800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bf015adf0 0x7f0bf013de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf7800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf00b0d00 con 0x7f0bf006db10 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf7800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf006fe30 con 0x7f0bf015adf0 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf00b0e80 con 0x7f0bf0070630 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf006db10 0x7f0bf0140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf006db10 0x7f0bf0140920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34310/0 (socket says 172.21.15.19:34310) 2024-04-25T11:30:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1144345155 learned_addr learned my addr 172.21.15.19:0/1144345155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0befe00640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf0070630 0x7f0bf01534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf0070630 msgr2=0x7f0bf01534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf0070630 0x7f0bf01534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bf015adf0 msgr2=0x7f0bf013de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bf015adf0 0x7f0bf013de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.383+0000 7f0bf4e00640 1 -- 172.21.15.19:0/1144345155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf0140e60 con 0x7f0bf006db10 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.384+0000 7f0bf4e00640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf006db10 0x7f0bf0140920 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e50 tx=0x7f0be0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.384+0000 7f0becc00640 1 -- 172.21.15.19:0/1144345155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0036050 con 0x7f0bf006db10 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.384+0000 7f0becc00640 1 -- 172.21.15.19:0/1144345155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be0042070 con 0x7f0bf006db10 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.384+0000 7f0becc00640 1 -- 172.21.15.19:0/1144345155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0009510 con 0x7f0bf006db10 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.384+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf0153a30 con 0x7f0bf006db10 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.384+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf0152a10 con 0x7f0bf006db10 2024-04-25T11:30:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.386+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf0050740 con 0x7f0bf006db10 2024-04-25T11:30:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.387+0000 7f0becc00640 1 -- 172.21.15.19:0/1144345155 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0be0007c50 con 0x7f0bf006db10 2024-04-25T11:30:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.388+0000 7f0becc00640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bcc0744b0 0x7f0bcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.388+0000 7f0becc00640 1 -- 172.21.15.19:0/1144345155 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f0be00b61b0 con 0x7f0bf006db10 2024-04-25T11:30:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.389+0000 7f0befe00640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bcc0744b0 0x7f0bcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.390+0000 7f0befe00640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bcc0744b0 0x7f0bcc0769c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8006fd0 tx=0x7f0bd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.392+0000 7f0becc00640 1 -- 172.21.15.19:0/1144345155 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0bf0140e60 con 0x7f0bf006db10 2024-04-25T11:30:23.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.680+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0bf00b29d0 con 0x7f0bf006db10 2024-04-25T11:30:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.687+0000 7f0becc00640 1 -- 172.21.15.19:0/1144345155 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f0bf006df20 con 0x7f0bf006db10 2024-04-25T11:30:23.677 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T11:30:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:30:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T11:30:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T11:30:23.677 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:30:23.677 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bcc0744b0 msgr2=0x7f0bcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bcc0744b0 0x7f0bcc0769c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8006fd0 tx=0x7f0bd8008040 comp rx=0 tx=0).stop 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf006db10 msgr2=0x7f0bf0140920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf006db10 0x7f0bf0140920 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009e50 tx=0x7f0be0004a40 comp rx=0 tx=0).stop 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 shutdown_connections 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0bcc0744b0 0x7f0bcc0769c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0bf015adf0 0x7f0bf013de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0bf0070630 0x7f0bf01534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.689+0000 7f0bf7800640 1 --2- 172.21.15.19:0/1144345155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bf006db10 0x7f0bf0140920 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.690+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 >> 172.21.15.19:0/1144345155 conn(0x7f0bf0073090 msgr2=0x7f0bf015fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.690+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 shutdown_connections 2024-04-25T11:30:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.690+0000 7f0bf7800640 1 -- 172.21.15.19:0/1144345155 wait complete. 2024-04-25T11:30:23.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.991+0000 7f7222a00640 1 Processor -- start 2024-04-25T11:30:23.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.991+0000 7f7222a00640 1 -- start start 2024-04-25T11:30:23.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.993+0000 7f7222a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 0x7f721c0d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.993+0000 7f7222a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721c0b7940 con 0x7f721c160d60 2024-04-25T11:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.993+0000 7f7222a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721c0d2ef0 con 0x7f721c0d2ae0 2024-04-25T11:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.993+0000 7f7222a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f721c161280 con 0x7f721c0d50e0 2024-04-25T11:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.994+0000 7f7222000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f721c160d60 0x7f721c161810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58594/0 (socket says 172.21.15.19:58594) 2024-04-25T11:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.994+0000 7f7222000640 1 -- 172.21.15.19:0/2582519186 learned_addr learned my addr 172.21.15.19:0/2582519186 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.994+0000 7f7221600640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 0x7f721c0d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.994+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005268236 0 0) 0x7f7214001490 con 0x7f721c160d60 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.994+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7204003520 con 0x7f721c160d60 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1867539985 0 0) 0x7f7204003520 con 0x7f721c160d60 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 msgr2=0x7f721c0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 0x7f721c0d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 >> v1:172.21.15.169:6789/0 conn(0x7f721c0d2ae0 legacy=0x7f721c160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c0d4600 con 0x7f721c160d60 2024-04-25T11:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7214001bc0 con 0x7f721c160d60 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 >> v1:172.21.15.19:6789/0 conn(0x7f721c160d60 legacy=0x7f721c161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72040039e0 0x7f7204003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7204004800 0x7f7204004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 0x7f72040052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f721c161280 con 0x7f72040039e0 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f720400bb50 con 0x7f7204004800 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f720400bd50 con 0x7f721c0d50e0 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.995+0000 7f7221600640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72040039e0 0x7f7204003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7220c00640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7204004800 0x7f7204004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7222000640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 0x7f72040052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7221600640 1 -- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 msgr2=0x7f72040052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7221600640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c0d50e0 0x7f72040052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7221600640 1 -- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7204004800 msgr2=0x7f7204004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7221600640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7204004800 0x7f7204004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7221600640 1 -- 172.21.15.19:0/2582519186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7204003520 con 0x7f72040039e0 2024-04-25T11:30:23.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.996+0000 7f7220c00640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7204004800 0x7f7204004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.998+0000 7f7221600640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72040039e0 0x7f7204003e50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7210009ac0 tx=0x7f7204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388acc3b5ea238ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.998+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210036070 con 0x7f72040039e0 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.998+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7210031400 con 0x7f72040039e0 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.998+0000 7f721be00640 1 -- 172.21.15.19:0/2582519186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210009690 con 0x7f72040039e0 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.999+0000 7f7222a00640 1 -- 172.21.15.19:0/2582519186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72040039e0 msgr2=0x7f7204003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.999+0000 7f7222a00640 1 --2- 172.21.15.19:0/2582519186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72040039e0 0x7f7204003e50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7210009ac0 tx=0x7f7204005830 comp rx=0 tx=0).stop 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.999+0000 7f7222000640 1 -- 172.21.15.19:0/2582519186 reap_dead start 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.999+0000 7f7222a00640 1 -- 172.21.15.19:0/2582519186 shutdown_connections 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.999+0000 7f7222a00640 1 -- 172.21.15.19:0/2582519186 >> 172.21.15.19:0/2582519186 conn(0x7f721c06c5c0 msgr2=0x7f721c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.999+0000 7f7222a00640 1 -- 172.21.15.19:0/2582519186 shutdown_connections 2024-04-25T11:30:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:23.999+0000 7f7222a00640 1 -- 172.21.15.19:0/2582519186 wait complete. 2024-04-25T11:30:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.001+0000 7f7222a00640 1 Processor -- start 2024-04-25T11:30:23.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.002+0000 7f7222a00640 1 -- start start 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.003+0000 7f7222a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040039e0 0x7f721c1fa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.003+0000 7f7222a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c154190 0x7f721c154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.003+0000 7f7222a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c155340 0x7f721c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.003+0000 7f7222a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f721c161560 con 0x7f721c155340 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.003+0000 7f7222a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f721c0d4bc0 con 0x7f72040039e0 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.003+0000 7f7222a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f721c0d54f0 con 0x7f721c154190 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.003+0000 7f7222000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c155340 0x7f721c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c155340 0x7f721c152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58702/0 (socket says 172.21.15.19:58702) 2024-04-25T11:30:23.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 -- 172.21.15.19:0/169120803 learned_addr learned my addr 172.21.15.19:0/169120803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7221600640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040039e0 0x7f721c1fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7220c00640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c154190 0x7f721c154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 -- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c154190 msgr2=0x7f721c154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c154190 0x7f721c154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 -- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040039e0 msgr2=0x7f721c1fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040039e0 0x7f721c1fa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 -- 172.21.15.19:0/169120803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c1574f0 con 0x7f721c155340 2024-04-25T11:30:23.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.004+0000 7f7222000640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c155340 0x7f721c152980 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f721c161170 tx=0x7f721400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:23.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.005+0000 7f7219600640 1 -- 172.21.15.19:0/169120803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721400eb90 con 0x7f721c155340 2024-04-25T11:30:23.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.005+0000 7f7219600640 1 -- 172.21.15.19:0/169120803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f721400ed50 con 0x7f721c155340 2024-04-25T11:30:23.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.005+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c1577d0 con 0x7f721c155340 2024-04-25T11:30:23.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.005+0000 7f7219600640 1 -- 172.21.15.19:0/169120803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214014670 con 0x7f721c155340 2024-04-25T11:30:23.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.005+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c157d90 con 0x7f721c155340 2024-04-25T11:30:23.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.006+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721c157260 con 0x7f721c155340 2024-04-25T11:30:23.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.008+0000 7f7219600640 1 -- 172.21.15.19:0/169120803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7214014810 con 0x7f721c155340 2024-04-25T11:30:23.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.009+0000 7f7219600640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71fc0744b0 0x7f71fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:23.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.009+0000 7f7219600640 1 -- 172.21.15.19:0/169120803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f7214095090 con 0x7f721c155340 2024-04-25T11:30:23.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.010+0000 7f7221600640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71fc0744b0 0x7f71fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:24.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.011+0000 7f7221600640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71fc0744b0 0x7f71fc0769c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7210033000 tx=0x7f7210008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.014+0000 7f7219600640 1 -- 172.21.15.19:0/169120803 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f721c0b7940 con 0x7f721c155340 2024-04-25T11:30:24.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.261+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f721c050740 con 0x7f721c155340 2024-04-25T11:30:24.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.262+0000 7f7219600640 1 -- 172.21.15.19:0/169120803 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5184 (secure 0 0 0) 0x7f721405f780 con 0x7f721c155340 2024-04-25T11:30:24.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:24.252 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:14.915169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3038680263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3038680263},{"type":"v1","addr":"172.21.15.19:6835","nonce":3038680263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":3}]} 2024-04-25T11:30:24.253 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-04-25T11:30:24.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71fc0744b0 msgr2=0x7f71fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:24.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71fc0744b0 0x7f71fc0769c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7210033000 tx=0x7f7210008030 comp rx=0 tx=0).stop 2024-04-25T11:30:24.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c155340 msgr2=0x7f721c152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:24.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c155340 0x7f721c152980 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f721c161170 tx=0x7f721400e530 comp rx=0 tx=0).stop 2024-04-25T11:30:24.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 shutdown_connections 2024-04-25T11:30:24.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71fc0744b0 0x7f71fc0769c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f721c155340 0x7f721c152980 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f721c154190 0x7f721c154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 --2- 172.21.15.19:0/169120803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72040039e0 0x7f721c1fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 >> 172.21.15.19:0/169120803 conn(0x7f721c06c5c0 msgr2=0x7f721c159360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:24.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.265+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 shutdown_connections 2024-04-25T11:30:24.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.266+0000 7f7222a00640 1 -- 172.21.15.19:0/169120803 wait complete. 2024-04-25T11:30:24.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:30:11.809876+0000', 'modified': '2024-04-25T11:30:14.915169+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5721}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5721': {'gid': 5721, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/3038680263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 3038680263}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3038680263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5721, 'qdb_cluster': [5721]} 2024-04-25T11:30:24.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:30:24.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:30:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.991+0000 7f0e31000640 1 Processor -- start 2024-04-25T11:30:24.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.991+0000 7f0e31000640 1 -- start start 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.993+0000 7f0e31000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e2c0d50e0 0x7f0e2c0d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.993+0000 7f0e31000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e2c160d60 0x7f0e2c161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.993+0000 7f0e31000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2c0b7940 con 0x7f0e2c0d2ae0 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.993+0000 7f0e31000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2c0d2ef0 con 0x7f0e2c160d60 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.993+0000 7f0e31000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2c161280 con 0x7f0e2c0d50e0 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.994+0000 7f0e2be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e2c160d60 0x7f0e2c161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.994+0000 7f0e2aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0e2c0d2ae0 0x7f0e2c160650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58598/0 (socket says 172.21.15.19:58598) 2024-04-25T11:30:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.994+0000 7f0e2aa00640 1 -- 172.21.15.19:0/3092194410 learned_addr learned my addr 172.21.15.19:0/3092194410 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.994+0000 7f0e2b400640 1 --2- 172.21.15.19:0/3092194410 >> v2:172.21.15.169:3300/0 conn(0x7f0e2c0d50e0 0x7f0e2c0d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.994+0000 7f0e2be00640 1 -- 172.21.15.19:0/3092194410 >> v2:172.21.15.169:3300/0 conn(0x7f0e2c0d50e0 msgr2=0x7f0e2c0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.994+0000 7f0e2be00640 1 --2- 172.21.15.19:0/3092194410 >> v2:172.21.15.169:3300/0 conn(0x7f0e2c0d50e0 0x7f0e2c0d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.994+0000 7f0e2be00640 1 -- 172.21.15.19:0/3092194410 >> v1:172.21.15.19:6789/0 conn(0x7f0e2c0d2ae0 legacy=0x7f0e2c160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.995+0000 7f0e2be00640 1 -- 172.21.15.19:0/3092194410 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c0d4600 con 0x7f0e2c160d60 2024-04-25T11:30:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.995+0000 7f0e2b400640 1 --2- 172.21.15.19:0/3092194410 >> v2:172.21.15.169:3300/0 conn(0x7f0e2c0d50e0 0x7f0e2c0d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.996+0000 7f0e2be00640 1 --2- 172.21.15.19:0/3092194410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c161810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0d6cc0 tx=0x7f0e2402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a421307ded6af252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.996+0000 7f0e2a000640 1 -- 172.21.15.19:0/3092194410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007740 con 0x7f0e2c160d60 2024-04-25T11:30:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.996+0000 7f0e2a000640 1 -- 172.21.15.19:0/3092194410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e240078e0 con 0x7f0e2c160d60 2024-04-25T11:30:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.996+0000 7f0e2a000640 1 -- 172.21.15.19:0/3092194410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007cb0 con 0x7f0e2c160d60 2024-04-25T11:30:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 -- 172.21.15.19:0/3092194410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 msgr2=0x7f0e2c161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 --2- 172.21.15.19:0/3092194410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c161810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0d6cc0 tx=0x7f0e2402a680 comp rx=0 tx=0).stop 2024-04-25T11:30:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 -- 172.21.15.19:0/3092194410 shutdown_connections 2024-04-25T11:30:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 --2- 172.21.15.19:0/3092194410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c161810 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 --2- 172.21.15.19:0/3092194410 >> v2:172.21.15.169:3300/0 conn(0x7f0e2c0d50e0 0x7f0e2c0d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 -- 172.21.15.19:0/3092194410 >> 172.21.15.19:0/3092194410 conn(0x7f0e2c06c5c0 msgr2=0x7f0e2c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 -- 172.21.15.19:0/3092194410 shutdown_connections 2024-04-25T11:30:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:24.997+0000 7f0e31000640 1 -- 172.21.15.19:0/3092194410 wait complete. 2024-04-25T11:30:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.000+0000 7f0e31000640 1 Processor -- start 2024-04-25T11:30:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.000+0000 7f0e31000640 1 -- start start 2024-04-25T11:30:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.001+0000 7f0e31000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2c0d2ae0 0x7f0e2c154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.001+0000 7f0e31000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2c0d50e0 0x7f0e2c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.001+0000 7f0e31000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.001+0000 7f0e31000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c161560 con 0x7f0e2c160d60 2024-04-25T11:30:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.001+0000 7f0e31000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c0d4bc0 con 0x7f0e2c0d50e0 2024-04-25T11:30:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.001+0000 7f0e31000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2c0d54f0 con 0x7f0e2c0d2ae0 2024-04-25T11:30:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58724/0 (socket says 172.21.15.19:58724) 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 -- 172.21.15.19:0/2509320513 learned_addr learned my addr 172.21.15.19:0/2509320513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2aa00640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2c0d50e0 0x7f0e2c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2b400640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2c0d2ae0 0x7f0e2c154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 -- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2c0d2ae0 msgr2=0x7f0e2c154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2c0d2ae0 0x7f0e2c154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 -- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2c0d50e0 msgr2=0x7f0e2c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2c0d50e0 0x7f0e2c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.002+0000 7f0e2be00640 1 -- 172.21.15.19:0/2509320513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2c157df0 con 0x7f0e2c160d60 2024-04-25T11:30:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.003+0000 7f0e2be00640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c162730 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402ac70 tx=0x7f0e24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.003+0000 7f0e23400640 1 -- 172.21.15.19:0/2509320513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2403f070 con 0x7f0e2c160d60 2024-04-25T11:30:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.003+0000 7f0e23400640 1 -- 172.21.15.19:0/2509320513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24031c90 con 0x7f0e2c160d60 2024-04-25T11:30:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.003+0000 7f0e23400640 1 -- 172.21.15.19:0/2509320513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24044680 con 0x7f0e2c160d60 2024-04-25T11:30:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.003+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c1580d0 con 0x7f0e2c160d60 2024-04-25T11:30:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.003+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c158690 con 0x7f0e2c160d60 2024-04-25T11:30:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.006+0000 7f0e23400640 1 -- 172.21.15.19:0/2509320513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e2403b410 con 0x7f0e2c160d60 2024-04-25T11:30:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.008+0000 7f0e23400640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e08074500 0x7f0e08076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.008+0000 7f0e23400640 1 -- 172.21.15.19:0/2509320513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f0e240b6bf0 con 0x7f0e2c160d60 2024-04-25T11:30:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.008+0000 7f0e3959c740 1 Processor -- start 2024-04-25T11:30:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.008+0000 7f0e2b400640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e08074500 0x7f0e08076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.008+0000 7f0e3959c740 1 -- start start 2024-04-25T11:30:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e2b400640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e08074500 0x7f0e08076a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400a950 tx=0x7f0e14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b86210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562330798590 0x562330ba1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e3959c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562330abf600 con 0x562330798590 2024-04-25T11:30:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562330a5c100 con 0x5623307903b0 2024-04-25T11:30:25.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562330abb5c0 con 0x5623307b9270 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.010+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562330798590 0x562330ba1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562330798590 0x562330ba1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58726/0 (socket says 172.21.15.19:58726) 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 -- 172.21.15.19:0/1664502635 learned_addr learned my addr 172.21.15.19:0/1664502635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2aa00640 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2b400640 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b86210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 -- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 msgr2=0x562330b86210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b86210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 -- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 msgr2=0x562330ba0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 -- 172.21.15.19:0/1664502635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623307d12d0 con 0x562330798590 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2b400640 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b86210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.011+0000 7f0e2be00640 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562330798590 0x562330ba1480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e2407d450 tx=0x7f0e24009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48021895e3312dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.012+0000 7f0e1be00640 1 -- 172.21.15.19:0/1664502635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24080970 con 0x562330798590 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.012+0000 7f0e1be00640 1 -- 172.21.15.19:0/1664502635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24084070 con 0x562330798590 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.012+0000 7f0e1be00640 1 -- 172.21.15.19:0/1664502635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e240812e0 con 0x562330798590 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.012+0000 7f0e3959c740 1 -- 172.21.15.19:0/1664502635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562330798590 msgr2=0x562330ba1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.012+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562330798590 0x562330ba1480 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e2407d450 tx=0x7f0e24009b90 comp rx=0 tx=0).stop 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.013+0000 7f0e3959c740 1 -- 172.21.15.19:0/1664502635 shutdown_connections 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.013+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.013+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562330798590 0x562330ba1480 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.013+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1664502635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b86210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.013+0000 7f0e3959c740 1 -- 172.21.15.19:0/1664502635 >> 172.21.15.19:0/1664502635 conn(0x5623307ad3c0 msgr2=0x562330b5b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.013+0000 7f0e3959c740 1 -- 172.21.15.19:0/1664502635 shutdown_connections 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.013+0000 7f0e3959c740 1 -- 172.21.15.19:0/1664502635 wait complete. 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.014+0000 7f0e3959c740 1 -- start start 2024-04-25T11:30:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.014+0000 7f0e3959c740 1 Processor -- start 2024-04-25T11:30:25.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307903b0 0x562330b82270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330b99e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b9b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e3959c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562330abf600 con 0x5623307903b0 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562330a46f70 con 0x562330798590 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562330b9be40 con 0x5623307b9270 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e3959c740 10 client.0 _ll_register_callbacks cb 0x562330ab5480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307903b0 0x562330b82270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307903b0 0x562330b82270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58728/0 (socket says 172.21.15.19:58728) 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e2b400640 1 -- 172.21.15.19:0/2175849527 learned_addr learned my addr 172.21.15.19:0/2175849527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e2aa00640 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330b99e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.016+0000 7f0e2be00640 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b9b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e2b400640 1 -- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 msgr2=0x562330b9b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e2b400640 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b9b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e2b400640 1 -- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 msgr2=0x562330b99e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e2b400640 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330b99e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e2b400640 1 -- 172.21.15.19:0/2175849527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233089b6b0 con 0x5623307903b0 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e2b400640 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307903b0 0x562330b82270 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e14013730 tx=0x7f0e14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f426a5665b5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e18c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e18c00640 1 -- 172.21.15.19:0/2175849527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14014880 con 0x5623307903b0 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e18c00640 1 -- 172.21.15.19:0/2175849527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f0e14014a20 con 0x5623307903b0 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.017+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e24084070 con 0x5623307903b0 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e18c00640 1 -- 172.21.15.19:0/2175849527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0e14014c20 con 0x5623307903b0 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 10 client.5880 fetch_fsmap learned FSMap version 34 2024-04-25T11:30:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x562330b9bb80 con 0x5623307903b0 2024-04-25T11:30:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e18c00640 1 -- 172.21.15.19:0/2175849527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24084070 con 0x5623307903b0 2024-04-25T11:30:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e18c00640 1 -- 172.21.15.19:0/2175849527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f0e14031080 con 0x5623307903b0 2024-04-25T11:30:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 10 client.5880 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T11:30:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 10 client.5880 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 20 client.5880 populate_metadata read hostname 'smithi019' 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330777e40 0x562330b982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 4 client.5880 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.018+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 --> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562330acd210 con 0x562330777e40 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.019+0000 7f0e2aa00640 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330777e40 0x562330b982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.019+0000 7f0e2aa00640 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330777e40 0x562330b982b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9c28e4ce12b8d9 server_cookie=c45a38686cf4dff5 in_seq=0 out_seq=0 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.019+0000 7f0e18c00640 10 client.5880 ms_handle_connect on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.020+0000 7f0e18c00640 1 -- 172.21.15.19:0/2175849527 <== mds.0 v2:172.21.15.19:6834/3038680263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562330acd210 con 0x562330777e40 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.020+0000 7f0e18c00640 10 client.5880 handle_command_reply: tid=0 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.020+0000 7f0e3959c740 1 client.5880 shutdown 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307903b0 msgr2=0x562330b82270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307903b0 0x562330b82270 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e14013730 tx=0x7f0e14004730 comp rx=0 tx=0).stop 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 shutdown_connections 2024-04-25T11:30:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330777e40 0x562330b982b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330b99e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307b9270 0x562330b9b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 --2- 172.21.15.19:0/2175849527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307903b0 0x562330b82270 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e18c00640 0 client.5880 ms_handle_reset on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 >> 172.21.15.19:0/2175849527 conn(0x5623307ad3c0 msgr2=0x562330b82e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 shutdown_connections 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 1 -- 172.21.15.19:0/2175849527 wait complete. 2024-04-25T11:30:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.021+0000 7f0e3959c740 20 client.5880 trim_cache size 0 max 16384 2024-04-25T11:30:25.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.042+0000 7f0e3959c740 1 Processor -- start 2024-04-25T11:30:25.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.042+0000 7f0e3959c740 1 -- start start 2024-04-25T11:30:25.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.043+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562330798590 0x562330b88130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b88670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e3959c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c201cf0 con 0x5623307b9270 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c201050 con 0x5623307903b0 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2c2016a0 con 0x562330798590 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b88670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b88670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58734/0 (socket says 172.21.15.19:58734) 2024-04-25T11:30:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e2be00640 1 -- 172.21.15.19:0/3500247341 learned_addr learned my addr 172.21.15.19:0/3500247341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e2b400640 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.044+0000 7f0e2aa00640 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562330798590 0x562330b88130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e2be00640 1 -- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562330798590 msgr2=0x562330b88130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e2be00640 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562330798590 0x562330b88130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e2be00640 1 -- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 msgr2=0x562330ba0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e2be00640 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e2be00640 1 -- 172.21.15.19:0/3500247341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562330b598c0 con 0x5623307b9270 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e2be00640 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b88670 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0e24080b60 tx=0x7f0e24087000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8f74e3be4b677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e1b400640 1 -- 172.21.15.19:0/3500247341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24080e60 con 0x5623307b9270 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e1b400640 1 -- 172.21.15.19:0/3500247341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24084020 con 0x5623307b9270 2024-04-25T11:30:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.045+0000 7f0e1b400640 1 -- 172.21.15.19:0/3500247341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24081300 con 0x5623307b9270 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 -- 172.21.15.19:0/3500247341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 msgr2=0x562330b88670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b88670 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0e24080b60 tx=0x7f0e24087000 comp rx=0 tx=0).stop 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 -- 172.21.15.19:0/3500247341 shutdown_connections 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b88670 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562330798590 0x562330b88130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 --2- 172.21.15.19:0/3500247341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5623307903b0 0x562330ba0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 -- 172.21.15.19:0/3500247341 >> 172.21.15.19:0/3500247341 conn(0x5623307ad3c0 msgr2=0x562330b9db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 -- 172.21.15.19:0/3500247341 shutdown_connections 2024-04-25T11:30:25.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.046+0000 7f0e3959c740 1 -- 172.21.15.19:0/3500247341 wait complete. 2024-04-25T11:30:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.047+0000 7f0e3959c740 1 -- start start 2024-04-25T11:30:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.048+0000 7f0e3959c740 1 Processor -- start 2024-04-25T11:30:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.049+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307903b0 0x562330bae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.049+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330baee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.049+0000 7f0e3959c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b55830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.049+0000 7f0e3959c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562330bb0a00 con 0x5623307b9270 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.049+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562330bb05b0 con 0x562330798590 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.049+0000 7f0e3959c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562330b88f50 con 0x5623307903b0 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.049+0000 7f0e3959c740 10 client.0 _ll_register_callbacks cb 0x7f0e1400a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b55830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b55830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58736/0 (socket says 172.21.15.19:58736) 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 -- 172.21.15.19:0/1544107999 learned_addr learned my addr 172.21.15.19:0/1544107999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2b400640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307903b0 0x562330bae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2aa00640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330baee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 -- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307903b0 msgr2=0x562330bae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307903b0 0x562330bae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 -- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 msgr2=0x562330baee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330baee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 -- 172.21.15.19:0/1544107999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562330c1d3a0 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2b400640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307903b0 0x562330bae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e2be00640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b55830 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0e24047000 tx=0x7f0e24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657cc31c2de96a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.050+0000 7f0e18c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e18c00640 1 -- 172.21.15.19:0/1544107999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24085160 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e18c00640 1 -- 172.21.15.19:0/1544107999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f0e240a0040 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e24084020 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e18c00640 1 -- 172.21.15.19:0/1544107999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0e24089b80 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 10 client.5886 fetch_fsmap learned FSMap version 34 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x562330b875d0 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e18c00640 1 -- 172.21.15.19:0/1544107999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e24084020 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e18c00640 1 -- 172.21.15.19:0/1544107999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f0e24087070 con 0x5623307b9270 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 10 client.5886 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 10 client.5886 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:30:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 20 client.5886 populate_metadata read hostname 'smithi019' 2024-04-25T11:30:25.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330b55d70 0x562330b82800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 4 client.5886 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:30:25.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.051+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 --> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] -- command(tid 0: {"prefix": "status"}) -- 0x562330ac7080 con 0x562330b55d70 2024-04-25T11:30:25.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.052+0000 7f0e2b400640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330b55d70 0x562330b82800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.052+0000 7f0e2b400640 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330b55d70 0x562330b82800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b981d8ca117ad3dd server_cookie=98ab7cd5e2760001 in_seq=0 out_seq=0 2024-04-25T11:30:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.052+0000 7f0e18c00640 10 client.5886 ms_handle_connect on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e18c00640 1 -- 172.21.15.19:0/1544107999 <== mds.0 v2:172.21.15.19:6834/3038680263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562330ac7080 con 0x562330b55d70 2024-04-25T11:30:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e18c00640 10 client.5886 handle_command_reply: tid=0 2024-04-25T11:30:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 client.5886 shutdown 2024-04-25T11:30:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 msgr2=0x562330b55830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b55830 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0e24047000 tx=0x7f0e24009750 comp rx=0 tx=0).stop 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 shutdown_connections 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x562330b55d70 0x562330b82800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562330798590 0x562330baee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623307b9270 0x562330b55830 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 --2- 172.21.15.19:0/1544107999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5623307903b0 0x562330bae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e18c00640 0 client.5886 ms_handle_reset on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 >> 172.21.15.19:0/1544107999 conn(0x5623307ad3c0 msgr2=0x562330b9db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 shutdown_connections 2024-04-25T11:30:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 1 -- 172.21.15.19:0/1544107999 wait complete. 2024-04-25T11:30:25.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.053+0000 7f0e3959c740 20 client.5886 trim_cache size 0 max 16384 2024-04-25T11:30:25.047 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:30:25.047 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:30:25.047 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:30:25.047 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5721, 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 12.75702705, 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 34, 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 50, 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 50, 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 18.919012859999999 2024-04-25T11:30:25.048 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:30:25.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e08074500 msgr2=0x7f0e08076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e08074500 0x7f0e08076a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400a950 tx=0x7f0e14008040 comp rx=0 tx=0).stop 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 msgr2=0x7f0e2c162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c162730 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e2402ac70 tx=0x7f0e24004a40 comp rx=0 tx=0).stop 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 shutdown_connections 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e08074500 0x7f0e08076a10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e2c0d50e0 0x7f0e2c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e2c160d60 0x7f0e2c162730 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 --2- 172.21.15.19:0/2509320513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e2c0d2ae0 0x7f0e2c154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 >> 172.21.15.19:0/2509320513 conn(0x7f0e2c06c5c0 msgr2=0x7f0e2c15fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:25.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 shutdown_connections 2024-04-25T11:30:25.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.054+0000 7f0e31000640 1 -- 172.21.15.19:0/2509320513 wait complete. 2024-04-25T11:30:25.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:30:25.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.363+0000 7f531f000640 1 Processor -- start 2024-04-25T11:30:25.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.363+0000 7f531f000640 1 -- start start 2024-04-25T11:30:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.366+0000 7f531f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f531806f160 0x7f5318147430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.366+0000 7f531f000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5318149df0 0x7f531814a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.366+0000 7f531f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531814a740 con 0x7f53180d0360 2024-04-25T11:30:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.366+0000 7f531f000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f53180cdb90 con 0x7f5318149df0 2024-04-25T11:30:25.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.366+0000 7f531f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f53180cde70 con 0x7f531806f160 2024-04-25T11:30:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.366+0000 7f531d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5318149df0 0x7f531814a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.366+0000 7f531d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5318149df0 0x7f531814a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58746/0 (socket says 172.21.15.19:58746) 2024-04-25T11:30:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.367+0000 7f531d000640 1 -- 172.21.15.19:0/3437285351 learned_addr learned my addr 172.21.15.19:0/3437285351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.367+0000 7f5317e00640 1 --2- 172.21.15.19:0/3437285351 >> v2:172.21.15.169:3300/0 conn(0x7f531806f160 0x7f5318147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.367+0000 7f531d000640 1 -- 172.21.15.19:0/3437285351 >> v2:172.21.15.169:3300/0 conn(0x7f531806f160 msgr2=0x7f5318147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.367+0000 7f531d000640 1 --2- 172.21.15.19:0/3437285351 >> v2:172.21.15.169:3300/0 conn(0x7f531806f160 0x7f5318147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.367+0000 7f531d000640 1 -- 172.21.15.19:0/3437285351 >> v1:172.21.15.169:6789/0 conn(0x7f53180d0360 legacy=0x7f53180d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.367+0000 7f531d000640 1 -- 172.21.15.19:0/3437285351 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53180d0f20 con 0x7f5318149df0 2024-04-25T11:30:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.368+0000 7f531d000640 1 --2- 172.21.15.19:0/3437285351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814a200 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f53100098a0 tx=0x7f531002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f908a02ce6792597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.369+0000 7f5316a00640 1 -- 172.21.15.19:0/3437285351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310037070 con 0x7f5318149df0 2024-04-25T11:30:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.369+0000 7f5316a00640 1 -- 172.21.15.19:0/3437285351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5310032440 con 0x7f5318149df0 2024-04-25T11:30:25.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.369+0000 7f5316a00640 1 -- 172.21.15.19:0/3437285351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310032810 con 0x7f5318149df0 2024-04-25T11:30:25.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.369+0000 7f531f000640 1 -- 172.21.15.19:0/3437285351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 msgr2=0x7f531814a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.369+0000 7f531f000640 1 --2- 172.21.15.19:0/3437285351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814a200 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f53100098a0 tx=0x7f531002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.370+0000 7f531f000640 1 -- 172.21.15.19:0/3437285351 shutdown_connections 2024-04-25T11:30:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.370+0000 7f531f000640 1 --2- 172.21.15.19:0/3437285351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814a200 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.370+0000 7f531f000640 1 --2- 172.21.15.19:0/3437285351 >> v2:172.21.15.169:3300/0 conn(0x7f531806f160 0x7f5318147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.370+0000 7f531f000640 1 -- 172.21.15.19:0/3437285351 >> 172.21.15.19:0/3437285351 conn(0x7f5318073050 msgr2=0x7f5318073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.370+0000 7f531f000640 1 -- 172.21.15.19:0/3437285351 shutdown_connections 2024-04-25T11:30:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.370+0000 7f531f000640 1 -- 172.21.15.19:0/3437285351 wait complete. 2024-04-25T11:30:25.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.372+0000 7f531f000640 1 Processor -- start 2024-04-25T11:30:25.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.372+0000 7f531f000640 1 -- start start 2024-04-25T11:30:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531806f160 0x7f53181fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53180d0360 0x7f531815e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f531814ab90 con 0x7f5318149df0 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53180d14e0 con 0x7f531806f160 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5318147970 con 0x7f53180d0360 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58756/0 (socket says 172.21.15.19:58756) 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.374+0000 7f531d000640 1 -- 172.21.15.19:0/550575214 learned_addr learned my addr 172.21.15.19:0/550575214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:25.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f5317e00640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531806f160 0x7f53181fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f5317400640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53180d0360 0x7f531815e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f531d000640 1 -- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53180d0360 msgr2=0x7f531815e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f531d000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53180d0360 0x7f531815e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f531d000640 1 -- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531806f160 msgr2=0x7f53181fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f531d000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531806f160 0x7f53181fe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f531d000640 1 -- 172.21.15.19:0/550575214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53181fb9d0 con 0x7f5318149df0 2024-04-25T11:30:25.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f5317e00640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531806f160 0x7f53181fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:25.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.375+0000 7f531d000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814d0b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f53100098a0 tx=0x7f53100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.376+0000 7f530fe00640 1 -- 172.21.15.19:0/550575214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310041020 con 0x7f5318149df0 2024-04-25T11:30:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.376+0000 7f530fe00640 1 -- 172.21.15.19:0/550575214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531003d700 con 0x7f5318149df0 2024-04-25T11:30:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.376+0000 7f530fe00640 1 -- 172.21.15.19:0/550575214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310037040 con 0x7f5318149df0 2024-04-25T11:30:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.376+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53181fbcb0 con 0x7f5318149df0 2024-04-25T11:30:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.376+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53181fc270 con 0x7f5318149df0 2024-04-25T11:30:25.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.377+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5318050740 con 0x7f5318149df0 2024-04-25T11:30:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.378+0000 7f530fe00640 1 -- 172.21.15.19:0/550575214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5310044020 con 0x7f5318149df0 2024-04-25T11:30:25.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.379+0000 7f530fe00640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52f80744e0 0x7f52f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.380+0000 7f530fe00640 1 -- 172.21.15.19:0/550575214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f53100b59b0 con 0x7f5318149df0 2024-04-25T11:30:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.380+0000 7f5317e00640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52f80744e0 0x7f52f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:25.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.381+0000 7f5317e00640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52f80744e0 0x7f52f80769f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5308009fd0 tx=0x7f530800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:25.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.385+0000 7f530fe00640 1 -- 172.21.15.19:0/550575214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f531004b030 con 0x7f5318149df0 2024-04-25T11:30:25.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.630+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f53180b4b70 con 0x7f5318149df0 2024-04-25T11:30:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.631+0000 7f530fe00640 1 -- 172.21.15.19:0/550575214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f53100800a0 con 0x7f5318149df0 2024-04-25T11:30:25.621 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:30:25.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52f80744e0 msgr2=0x7f52f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52f80744e0 0x7f52f80769f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5308009fd0 tx=0x7f530800b040 comp rx=0 tx=0).stop 2024-04-25T11:30:25.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 msgr2=0x7f531814d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:25.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814d0b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f53100098a0 tx=0x7f53100046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 shutdown_connections 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f52f80744e0 0x7f52f80769f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5318149df0 0x7f531814d0b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53180d0360 0x7f531815e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 --2- 172.21.15.19:0/550575214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f531806f160 0x7f53181fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 >> 172.21.15.19:0/550575214 conn(0x7f5318073050 msgr2=0x7f5318144de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.634+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 shutdown_connections 2024-04-25T11:30:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25.635+0000 7f531f000640 1 -- 172.21.15.19:0/550575214 wait complete. 2024-04-25T11:30:25.651 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:30:25.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:30:25.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:30:25.652 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:30:25.652 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:30:25.652 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:30:25.652 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:30:25.652 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:30:25.652 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:30:25.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:25.652 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:30:25.705 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:30:25.706 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:30:25.706 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:30:25.706 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:30:25.706 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:30:25.706 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:30:25.706 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:30:25.707 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:30:25.707 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:30:25.707 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:30:25.707 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:30:25.707 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:30:25.708 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:30:25.708 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:30:25.708 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:30:25.708 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:30:25.708 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42448sec preferred_lft 42448sec 2024-04-25T11:30:25.708 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:30:25.709 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:30:25.709 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:30:25.709 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:30:25.709 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:30:25.709 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:30:25.709 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:30:25.710 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:30:25.710 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:30:25.710 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:30:25.710 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:30:25.710 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:30:25.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:25.712 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:30:25.785 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:30:25.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:25.789 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:30:25.844 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:30:25.844 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:30:25.844 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:30:25.844 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:30:25.844 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:30:25.845 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:30:25.845 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:30:25.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:30:25.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:30:25.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:30:25.845 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:30:25.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:30:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:30:26.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:26.069 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:30:26.117 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:30:26.117 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:30:26.118 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:30:26.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:26.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:30:26.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:30:26.194 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:30:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:30:26.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:30:26.247 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:30:26.302 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:30:26.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:26.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:30:26.526 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:30:26.526 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:30:26.574 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:30:26.575 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:30:26.575 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:30:26.575 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:30:26.575 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:30:26.577 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:30:26.577 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:30:26.577 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:30:26.577 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:30:26.577 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:30:26.581 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:30:26.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:26.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:30:26.654 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:30:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:30:26.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:30:26.789 INFO:teuthology.orchestra.run.smithi019.stdout:client5263 2024-04-25T11:30:26.790 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:30:26.790 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:26.844 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/b') 2024-04-25T11:30:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:30:26.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-04-25T11:30:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.291+0000 7f2284e00640 1 Processor -- start 2024-04-25T11:30:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.291+0000 7f2284e00640 1 -- start start 2024-04-25T11:30:27.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.293+0000 7f2284e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22800d56c0 0x7f22800b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.293+0000 7f2284e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2280160440 0x7f228006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.293+0000 7f2284e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228006e4b0 con 0x7f228006e0a0 2024-04-25T11:30:27.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.293+0000 7f2284e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f22800d43f0 con 0x7f22800d56c0 2024-04-25T11:30:27.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.293+0000 7f2284e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22800d37e0 con 0x7f2280160440 2024-04-25T11:30:27.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.294+0000 7f227f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22800d56c0 0x7f22800b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.294+0000 7f227f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22800d56c0 0x7f22800b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58772/0 (socket says 172.21.15.19:58772) 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.294+0000 7f227f400640 1 -- 172.21.15.19:0/4242065970 learned_addr learned my addr 172.21.15.19:0/4242065970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.294+0000 7f227ea00640 1 --2- 172.21.15.19:0/4242065970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2280160440 0x7f228006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.295+0000 7f227f400640 1 -- 172.21.15.19:0/4242065970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2280160440 msgr2=0x7f228006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.295+0000 7f227f400640 1 --2- 172.21.15.19:0/4242065970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2280160440 0x7f228006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.295+0000 7f227f400640 1 -- 172.21.15.19:0/4242065970 >> v1:172.21.15.169:6789/0 conn(0x7f228006e0a0 legacy=0x7f22800d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.295+0000 7f227f400640 1 -- 172.21.15.19:0/4242065970 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22800d3ac0 con 0x7f22800d56c0 2024-04-25T11:30:27.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.296+0000 7f227f400640 1 --2- 172.21.15.19:0/4242065970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22800d56c0 0x7f22800b4cb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2270009870 tx=0x7f227002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f44de05479cf78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.296+0000 7f227e000640 1 -- 172.21.15.19:0/4242065970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270037070 con 0x7f22800d56c0 2024-04-25T11:30:27.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.296+0000 7f227e000640 1 -- 172.21.15.19:0/4242065970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270032440 con 0x7f22800d56c0 2024-04-25T11:30:27.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.296+0000 7f227e000640 1 -- 172.21.15.19:0/4242065970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270032810 con 0x7f22800d56c0 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 -- 172.21.15.19:0/4242065970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22800d56c0 msgr2=0x7f22800b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 --2- 172.21.15.19:0/4242065970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22800d56c0 0x7f22800b4cb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2270009870 tx=0x7f227002a270 comp rx=0 tx=0).stop 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 -- 172.21.15.19:0/4242065970 shutdown_connections 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 --2- 172.21.15.19:0/4242065970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2280160440 0x7f228006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 --2- 172.21.15.19:0/4242065970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22800d56c0 0x7f22800b4cb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 -- 172.21.15.19:0/4242065970 >> 172.21.15.19:0/4242065970 conn(0x7f22800d2c20 msgr2=0x7f22800d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 -- 172.21.15.19:0/4242065970 shutdown_connections 2024-04-25T11:30:27.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.297+0000 7f2284e00640 1 -- 172.21.15.19:0/4242065970 wait complete. 2024-04-25T11:30:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.300+0000 7f2284e00640 1 Processor -- start 2024-04-25T11:30:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.300+0000 7f2284e00640 1 -- start start 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.301+0000 7f2284e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f228006e0a0 0x7f228014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f2284e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280160440 0x7f22800ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f2284e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800cd160 0x7f228015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f2284e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22800ba270 con 0x7f228006e0a0 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f2284e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22800d4080 con 0x7f2280160440 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f2284e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22800d4880 con 0x7f22800cd160 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f227f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f228006e0a0 0x7f228014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f227f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f228006e0a0 0x7f228014f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58776/0 (socket says 172.21.15.19:58776) 2024-04-25T11:30:27.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f227f400640 1 -- 172.21.15.19:0/4176320976 learned_addr learned my addr 172.21.15.19:0/4176320976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:27.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f227ea00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280160440 0x7f22800ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.302+0000 7f227fe00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800cd160 0x7f228015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.303+0000 7f227f400640 1 -- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800cd160 msgr2=0x7f228015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.303+0000 7f227f400640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800cd160 0x7f228015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.303+0000 7f227f400640 1 -- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280160440 msgr2=0x7f22800ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.303+0000 7f227f400640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280160440 0x7f22800ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.303+0000 7f227f400640 1 -- 172.21.15.19:0/4176320976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22800cfc20 con 0x7f228006e0a0 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.303+0000 7f227f400640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f228006e0a0 0x7f228014f0a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f227002a2b0 tx=0x7f22700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.304+0000 7f2277400640 1 -- 172.21.15.19:0/4176320976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270041020 con 0x7f228006e0a0 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.304+0000 7f2277400640 1 -- 172.21.15.19:0/4176320976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22700306c0 con 0x7f228006e0a0 2024-04-25T11:30:27.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.304+0000 7f2277400640 1 -- 172.21.15.19:0/4176320976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270037040 con 0x7f228006e0a0 2024-04-25T11:30:27.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.304+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22800cff00 con 0x7f228006e0a0 2024-04-25T11:30:27.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.304+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22800d04c0 con 0x7f228006e0a0 2024-04-25T11:30:27.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.307+0000 7f2277400640 1 -- 172.21.15.19:0/4176320976 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2270044020 con 0x7f228006e0a0 2024-04-25T11:30:27.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.308+0000 7f2277400640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22600744e0 0x7f22600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.308+0000 7f2277400640 1 -- 172.21.15.19:0/4176320976 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f22700b5af0 con 0x7f228006e0a0 2024-04-25T11:30:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.308+0000 7f227ea00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22600744e0 0x7f22600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.309+0000 7f228d370740 1 Processor -- start 2024-04-25T11:30:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.309+0000 7f228d370740 1 -- start start 2024-04-25T11:30:27.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.309+0000 7f227ea00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22600744e0 0x7f22600769f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22800d0cc0 tx=0x7f226c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.310+0000 7f228d370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x56311651d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.310+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631161274d0 0x56311651a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.310+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56311612f6b0 0x5631164f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.310+0000 7f228d370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563116403030 con 0x563116150390 2024-04-25T11:30:27.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.310+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56311641e7f0 con 0x56311612f6b0 2024-04-25T11:30:27.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.310+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5631163d97e0 con 0x5631161274d0 2024-04-25T11:30:27.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x56311651d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x56311651d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58792/0 (socket says 172.21.15.19:58792) 2024-04-25T11:30:27.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 -- 172.21.15.19:0/517187731 learned_addr learned my addr 172.21.15.19:0/517187731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 -- 172.21.15.19:0/517187731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631161274d0 msgr2=0x56311651a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 --2- 172.21.15.19:0/517187731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631161274d0 0x56311651a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 -- 172.21.15.19:0/517187731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56311612f6b0 msgr2=0x5631164f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 --2- 172.21.15.19:0/517187731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56311612f6b0 0x5631164f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 -- 172.21.15.19:0/517187731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631163e5050 con 0x563116150390 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.311+0000 7f227f400640 1 --2- 172.21.15.19:0/517187731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x56311651d7d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2270033fd0 tx=0x7f227007f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6b612909c5f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.312+0000 7f226be00640 1 -- 172.21.15.19:0/517187731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227007f570 con 0x563116150390 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.312+0000 7f226be00640 1 -- 172.21.15.19:0/517187731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270080160 con 0x563116150390 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.312+0000 7f226be00640 1 -- 172.21.15.19:0/517187731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22700a1270 con 0x563116150390 2024-04-25T11:30:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.312+0000 7f228d370740 1 -- 172.21.15.19:0/517187731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 msgr2=0x56311651d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.312+0000 7f228d370740 1 --2- 172.21.15.19:0/517187731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x56311651d7d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2270033fd0 tx=0x7f227007f3b0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.313+0000 7f228d370740 1 -- 172.21.15.19:0/517187731 shutdown_connections 2024-04-25T11:30:27.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.313+0000 7f228d370740 1 --2- 172.21.15.19:0/517187731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56311612f6b0 0x5631164f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.313+0000 7f228d370740 1 --2- 172.21.15.19:0/517187731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631161274d0 0x56311651a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.313+0000 7f228d370740 1 --2- 172.21.15.19:0/517187731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x56311651d7d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.313+0000 7f228d370740 1 -- 172.21.15.19:0/517187731 >> 172.21.15.19:0/517187731 conn(0x5631161444e0 msgr2=0x5631164f2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.313+0000 7f228d370740 1 -- 172.21.15.19:0/517187731 shutdown_connections 2024-04-25T11:30:27.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.313+0000 7f228d370740 1 -- 172.21.15.19:0/517187731 wait complete. 2024-04-25T11:30:27.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.314+0000 7f228d370740 1 -- start start 2024-04-25T11:30:27.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.314+0000 7f228d370740 1 Processor -- start 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161274d0 0x563116514210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165162a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f228d370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x5631165167e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f228d370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56311645b200 con 0x563116150390 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563116455ee0 con 0x5631161274d0 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563116462e70 con 0x56311612f6b0 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f228d370740 10 client.0 _ll_register_callbacks cb 0x56311644c270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:30:27.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f227fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x5631165167e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f227fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x5631165167e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58798/0 (socket says 172.21.15.19:58798) 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.316+0000 7f227fe00640 1 -- 172.21.15.19:0/2438619895 learned_addr learned my addr 172.21.15.19:0/2438619895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227f400640 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161274d0 0x563116514210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227ea00640 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227fe00640 1 -- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 msgr2=0x5631165162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227fe00640 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227fe00640 1 -- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161274d0 msgr2=0x563116514210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227fe00640 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161274d0 0x563116514210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227fe00640 1 -- 172.21.15.19:0/2438619895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631162327d0 con 0x563116150390 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f227fe00640 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x5631165167e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f227800e260 tx=0x7f227800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99643d18223f9b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f2268c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f2268c00640 1 -- 172.21.15.19:0/2438619895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278019060 con 0x563116150390 2024-04-25T11:30:27.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f2268c00640 1 -- 172.21.15.19:0/2438619895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f2278003b80 con 0x563116150390 2024-04-25T11:30:27.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.317+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2270080160 con 0x563116150390 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f2268c00640 1 -- 172.21.15.19:0/2438619895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2278003d80 con 0x563116150390 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 10 client.5904 fetch_fsmap learned FSMap version 34 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5631161683f0 con 0x563116150390 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f2268c00640 1 -- 172.21.15.19:0/2438619895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270080160 con 0x563116150390 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f2268c00640 1 -- 172.21.15.19:0/2438619895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f2278006b50 con 0x563116150390 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 10 client.5904 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 10 client.5904 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 20 client.5904 populate_metadata read hostname 'smithi019' 2024-04-25T11:30:27.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x56311610ef60 0x563116537ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 4 client.5904 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.318+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 --> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631164647e0 con 0x56311610ef60 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.319+0000 7f227f400640 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x56311610ef60 0x563116537ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.319+0000 7f227f400640 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x56311610ef60 0x563116537ac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f5fd2de1ad295c server_cookie=dd62aa5efbb75009 in_seq=0 out_seq=0 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.319+0000 7f2268c00640 10 client.5904 ms_handle_connect on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.320+0000 7f2268c00640 1 -- 172.21.15.19:0/2438619895 <== mds.0 v2:172.21.15.19:6834/3038680263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5631164647e0 con 0x56311610ef60 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.320+0000 7f2268c00640 10 client.5904 handle_command_reply: tid=0 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.320+0000 7f228d370740 1 client.5904 shutdown 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.320+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 msgr2=0x5631165167e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.320+0000 7f228d370740 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x5631165167e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f227800e260 tx=0x7f227800e720 comp rx=0 tx=0).stop 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 shutdown_connections 2024-04-25T11:30:27.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x56311610ef60 0x563116537ac0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563116150390 0x5631165167e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 --2- 172.21.15.19:0/2438619895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161274d0 0x563116514210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f2268c00640 0 client.5904 ms_handle_reset on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 >> 172.21.15.19:0/2438619895 conn(0x5631161444e0 msgr2=0x5631164f2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 shutdown_connections 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 1 -- 172.21.15.19:0/2438619895 wait complete. 2024-04-25T11:30:27.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.321+0000 7f228d370740 20 client.5904 trim_cache size 0 max 16384 2024-04-25T11:30:27.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.339+0000 7f228d370740 1 Processor -- start 2024-04-25T11:30:27.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.339+0000 7f228d370740 1 -- start start 2024-04-25T11:30:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.340+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.340+0000 7f228d370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x563116513d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.340+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165142b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.340+0000 7f228d370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2280202e60 con 0x5631161274d0 2024-04-25T11:30:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.340+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2280201e80 con 0x5631161444e0 2024-04-25T11:30:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.340+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f22801630b0 con 0x56311612f6b0 2024-04-25T11:30:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x563116513d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x563116513d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58814/0 (socket says 172.21.15.19:58814) 2024-04-25T11:30:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 -- 172.21.15.19:0/2359575260 learned_addr learned my addr 172.21.15.19:0/2359575260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:27.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227f400640 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227fe00640 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 -- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 msgr2=0x5631165142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 -- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 msgr2=0x5631165355a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165355a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227ea00640 1 -- 172.21.15.19:0/2359575260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631161280f0 con 0x5631161274d0 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227f400640 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165355a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.341+0000 7f227fe00640 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.342+0000 7f227ea00640 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x563116513d70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f226c010a20 tx=0x7f226c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a530e64fc6231a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.342+0000 7f226b400640 1 -- 172.21.15.19:0/2359575260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c016b90 con 0x5631161274d0 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.342+0000 7f226b400640 1 -- 172.21.15.19:0/2359575260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c016d30 con 0x5631161274d0 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.342+0000 7f226b400640 1 -- 172.21.15.19:0/2359575260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c004020 con 0x5631161274d0 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 -- 172.21.15.19:0/2359575260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 msgr2=0x563116513d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x563116513d70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f226c010a20 tx=0x7f226c013000 comp rx=0 tx=0).stop 2024-04-25T11:30:27.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 -- 172.21.15.19:0/2359575260 shutdown_connections 2024-04-25T11:30:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x5631165142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x563116513d70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 --2- 172.21.15.19:0/2359575260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165355a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 -- 172.21.15.19:0/2359575260 >> 172.21.15.19:0/2359575260 conn(0x56311651cee0 msgr2=0x5631165162a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 -- 172.21.15.19:0/2359575260 shutdown_connections 2024-04-25T11:30:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.343+0000 7f228d370740 1 -- 172.21.15.19:0/2359575260 wait complete. 2024-04-25T11:30:27.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.344+0000 7f228d370740 1 -- start start 2024-04-25T11:30:27.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.344+0000 7f228d370740 1 Processor -- start 2024-04-25T11:30:27.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.346+0000 7f228d370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x5631165b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.346+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x563116533c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.346+0000 7f228d370740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165397c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.346+0000 7f228d370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563116463fd0 con 0x5631161274d0 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.346+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2280163700 con 0x5631161444e0 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.346+0000 7f228d370740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5631165348a0 con 0x56311612f6b0 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.346+0000 7f228d370740 10 client.0 _ll_register_callbacks cb 0x7f2278001500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x5631165b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x5631165b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58818/0 (socket says 172.21.15.19:58818) 2024-04-25T11:30:27.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 -- 172.21.15.19:0/948773503 learned_addr learned my addr 172.21.15.19:0/948773503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:27.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 -- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 msgr2=0x563116533c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227ea00640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x563116533c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227fe00640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x563116533c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 -- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 msgr2=0x5631165397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165397c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227f400640 1 -- 172.21.15.19:0/948773503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311641ea30 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.347+0000 7f227fe00640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f227f400640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x5631165b7b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f227007c220 tx=0x7f2270008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8e4cd31272ce95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f2268c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f2268c00640 1 -- 172.21.15.19:0/948773503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22700af060 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f2268c00640 1 -- 172.21.15.19:0/948773503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f2270088430 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226c016d30 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f2268c00640 1 -- 172.21.15.19:0/948773503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2270080160 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f228d370740 10 client.5910 fetch_fsmap learned FSMap version 34 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5631165b8040 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.348+0000 7f2268c00640 1 -- 172.21.15.19:0/948773503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c016d30 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f2268c00640 1 -- 172.21.15.19:0/948773503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f227009b070 con 0x5631161274d0 2024-04-25T11:30:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f228d370740 10 client.5910 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f228d370740 10 client.5910 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f228d370740 20 client.5910 populate_metadata read hostname 'smithi019' 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f228d370740 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x5631160e43a0 0x563116539d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f228d370740 4 client.5910 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 --> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56311645e650 con 0x5631160e43a0 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.349+0000 7f227ea00640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x5631160e43a0 0x563116539d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.350+0000 7f227ea00640 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x5631160e43a0 0x563116539d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34db4e5180b44e6 server_cookie=67e14c90d14fc611 in_seq=0 out_seq=0 2024-04-25T11:30:27.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.350+0000 7f2268c00640 10 client.5910 ms_handle_connect on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f2268c00640 1 -- 172.21.15.19:0/948773503 <== mds.0 v2:172.21.15.19:6834/3038680263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56311645e650 con 0x5631160e43a0 2024-04-25T11:30:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f2268c00640 10 client.5910 handle_command_reply: tid=0 2024-04-25T11:30:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f228d370740 1 client.5910 shutdown 2024-04-25T11:30:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 msgr2=0x5631165b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f228d370740 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x5631165b7b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f227007c220 tx=0x7f2270008070 comp rx=0 tx=0).stop 2024-04-25T11:30:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 shutdown_connections 2024-04-25T11:30:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f228d370740 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:6834/3038680263,v1:172.21.15.19:6835/3038680263] conn(0x5631160e43a0 0x563116539d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f228d370740 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631161444e0 0x5631165397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.370+0000 7f228d370740 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56311612f6b0 0x563116533c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.371+0000 7f228d370740 1 --2- 172.21.15.19:0/948773503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631161274d0 0x5631165b7b00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.371+0000 7f2268c00640 0 client.5910 ms_handle_reset on v2:172.21.15.19:6834/3038680263 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.371+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 >> 172.21.15.19:0/948773503 conn(0x56311651cee0 msgr2=0x563116537880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.371+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 shutdown_connections 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.371+0000 7f228d370740 1 -- 172.21.15.19:0/948773503 wait complete. 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.371+0000 7f228d370740 20 client.5910 trim_cache size 0 max 16384 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:30:27.362 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-04-25T11:30:27.363 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T11:30:27.363 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:30:27.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22600744e0 msgr2=0x7f22600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22600744e0 0x7f22600769f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22800d0cc0 tx=0x7f226c008040 comp rx=0 tx=0).stop 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f228006e0a0 msgr2=0x7f228014f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f228006e0a0 0x7f228014f0a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f227002a2b0 tx=0x7f22700046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 shutdown_connections 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22600744e0 0x7f22600769f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2280160440 0x7f22800ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22800cd160 0x7f228015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 --2- 172.21.15.19:0/4176320976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f228006e0a0 0x7f228014f0a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 >> 172.21.15.19:0/4176320976 conn(0x7f22800d2c20 msgr2=0x7f228015ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 shutdown_connections 2024-04-25T11:30:27.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.372+0000 7f2284e00640 1 -- 172.21.15.19:0/4176320976 wait complete. 2024-04-25T11:30:27.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-04-25T11:30:27.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.677+0000 7efedb000640 1 Processor -- start 2024-04-25T11:30:27.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.677+0000 7efedb000640 1 -- start start 2024-04-25T11:30:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.679+0000 7efedb000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efed40cf900 0x7efed40d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.679+0000 7efedb000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed40d31d0 con 0x7efed409ccc0 2024-04-25T11:30:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.679+0000 7efedb000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed40d34b0 con 0x7efed40d1f20 2024-04-25T11:30:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.679+0000 7efedb000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efed40d2f80 con 0x7efed40cf900 2024-04-25T11:30:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.679+0000 7efed9c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efed409ccc0 0x7efed406d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58612/0 (socket says 172.21.15.19:58612) 2024-04-25T11:30:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.679+0000 7efed9c00640 1 -- 172.21.15.19:0/3674671163 learned_addr learned my addr 172.21.15.19:0/3674671163 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:27.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efeda600640 1 --2- 172.21.15.19:0/3674671163 >> v2:172.21.15.169:3300/0 conn(0x7efed40cf900 0x7efed40d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1670336693 0 0) 0x7efed40d31d0 con 0x7efed409ccc0 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeb8003520 con 0x7efed409ccc0 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 297496329 0 0) 0x7efeb8003520 con 0x7efed409ccc0 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 >> v2:172.21.15.169:3300/0 conn(0x7efed40cf900 msgr2=0x7efed40d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 --2- 172.21.15.19:0/3674671163 >> v2:172.21.15.169:3300/0 conn(0x7efed40cf900 0x7efed40d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 >> v1:172.21.15.169:6789/0 conn(0x7efed40d1f20 legacy=0x7efed40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed40d3790 con 0x7efed409ccc0 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.680+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efec4001b70 con 0x7efed409ccc0 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 >> v1:172.21.15.19:6789/0 conn(0x7efed409ccc0 legacy=0x7efed406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed3e00640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb80039b0 0x7efeb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed3e00640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb8004800 0x7efeb8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed3e00640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8005250 0x7efeb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efed40d2f80 con 0x7efeb80039b0 2024-04-25T11:30:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efeb800c1e0 con 0x7efeb8004800 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efeb800c3e0 con 0x7efeb8005250 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9c00640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb80039b0 0x7efeb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9200640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb8004800 0x7efeb8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efeda600640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8005250 0x7efeb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9c00640 1 -- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8005250 msgr2=0x7efeb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9c00640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeb8005250 0x7efeb800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9c00640 1 -- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb8004800 msgr2=0x7efeb8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9c00640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb8004800 0x7efeb8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9c00640 1 -- 172.21.15.19:0/3674671163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb8003520 con 0x7efeb80039b0 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.681+0000 7efed9200640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb8004800 0x7efeb8004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.682+0000 7efeda600640 1 -- 172.21.15.19:0/3674671163 reap_dead start 2024-04-25T11:30:27.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.683+0000 7efed9c00640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb80039b0 0x7efeb8003e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efec400c530 tx=0x7efeb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37f021a80cd842e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.683+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec400c180 con 0x7efeb80039b0 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.683+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec4032020 con 0x7efeb80039b0 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.683+0000 7efed3e00640 1 -- 172.21.15.19:0/3674671163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4005e00 con 0x7efeb80039b0 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.684+0000 7efedb000640 1 -- 172.21.15.19:0/3674671163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb80039b0 msgr2=0x7efeb8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.684+0000 7efedb000640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb80039b0 0x7efeb8003e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efec400c530 tx=0x7efeb8005830 comp rx=0 tx=0).stop 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.684+0000 7efedb000640 1 -- 172.21.15.19:0/3674671163 shutdown_connections 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.684+0000 7efedb000640 1 --2- 172.21.15.19:0/3674671163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb80039b0 0x7efeb8003e20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.684+0000 7efedb000640 1 -- 172.21.15.19:0/3674671163 >> 172.21.15.19:0/3674671163 conn(0x7efed40d0480 msgr2=0x7efed40d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.684+0000 7efedb000640 1 -- 172.21.15.19:0/3674671163 shutdown_connections 2024-04-25T11:30:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.684+0000 7efedb000640 1 -- 172.21.15.19:0/3674671163 wait complete. 2024-04-25T11:30:27.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.687+0000 7efedb000640 1 Processor -- start 2024-04-25T11:30:27.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.687+0000 7efedb000640 1 -- start start 2024-04-25T11:30:27.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.688+0000 7efedb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb80039b0 0x7efed40cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.688+0000 7efedb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efed41ea3b0 0x7efed41e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.688+0000 7efedb000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed40caab0 0x7efed40caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.688+0000 7efedb000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efed40d1010 con 0x7efed40caab0 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.688+0000 7efedb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efed40b5220 con 0x7efeb80039b0 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.688+0000 7efedb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efed40d1190 con 0x7efed41ea3b0 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed40caab0 0x7efed40caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed40caab0 0x7efed40caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58838/0 (socket says 172.21.15.19:58838) 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 -- 172.21.15.19:0/968368368 learned_addr learned my addr 172.21.15.19:0/968368368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efed9c00640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb80039b0 0x7efed40cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efed9200640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efed41ea3b0 0x7efed41e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 -- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efed41ea3b0 msgr2=0x7efed41e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efed41ea3b0 0x7efed41e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 -- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb80039b0 msgr2=0x7efed40cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb80039b0 0x7efed40cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.689+0000 7efeda600640 1 -- 172.21.15.19:0/968368368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed40cb400 con 0x7efed40caab0 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.690+0000 7efeda600640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed40caab0 0x7efed40caec0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efed40b2db0 tx=0x7efecc010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.690+0000 7efed1600640 1 -- 172.21.15.19:0/968368368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00f040 con 0x7efed40caab0 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.690+0000 7efed1600640 1 -- 172.21.15.19:0/968368368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efecc01c070 con 0x7efed40caab0 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.690+0000 7efed1600640 1 -- 172.21.15.19:0/968368368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00eb00 con 0x7efed40caab0 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.690+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed40cc750 con 0x7efed40caab0 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.690+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed40ccd10 con 0x7efed40caab0 2024-04-25T11:30:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.691+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed4076c60 con 0x7efed40caab0 2024-04-25T11:30:27.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.693+0000 7efed1600640 1 -- 172.21.15.19:0/968368368 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efecc00eca0 con 0x7efed40caab0 2024-04-25T11:30:27.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.694+0000 7efed1600640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efeb40744b0 0x7efeb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:27.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.694+0000 7efed1600640 1 -- 172.21.15.19:0/968368368 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7efecc094dd0 con 0x7efed40caab0 2024-04-25T11:30:27.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.694+0000 7efed9c00640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efeb40744b0 0x7efeb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:27.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.696+0000 7efed9c00640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efeb40744b0 0x7efeb40769c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efec4033000 tx=0x7efec4009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:27.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.699+0000 7efed1600640 1 -- 172.21.15.19:0/968368368 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efecc026020 con 0x7efed40caab0 2024-04-25T11:30:27.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.906+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7efed41eab20 con 0x7efed40caab0 2024-04-25T11:30:27.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.915+0000 7efed1600640 1 -- 172.21.15.19:0/968368368 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v13)=0 v13) ==== 135+0+0 (secure 0 0 0) 0x7efed40cb400 con 0x7efed40caab0 2024-04-25T11:30:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.917+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efeb40744b0 msgr2=0x7efeb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.917+0000 7efedb000640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efeb40744b0 0x7efeb40769c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efec4033000 tx=0x7efec4009000 comp rx=0 tx=0).stop 2024-04-25T11:30:27.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed40caab0 msgr2=0x7efed40caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed40caab0 0x7efed40caec0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efed40b2db0 tx=0x7efecc010c20 comp rx=0 tx=0).stop 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 shutdown_connections 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efeb40744b0 0x7efeb40769c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efed40caab0 0x7efed40caec0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efed41ea3b0 0x7efed41e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 --2- 172.21.15.19:0/968368368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeb80039b0 0x7efed40cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 >> 172.21.15.19:0/968368368 conn(0x7efed40d0480 msgr2=0x7efed40b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:27.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 shutdown_connections 2024-04-25T11:30:27.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:27.918+0000 7efedb000640 1 -- 172.21.15.19:0/968368368 wait complete. 2024-04-25T11:30:27.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-04-25T11:30:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.236+0000 7f17aae00640 1 Processor -- start 2024-04-25T11:30:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.236+0000 7f17aae00640 1 -- start start 2024-04-25T11:30:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17aae00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f17a406dce0 0x7f17a41606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17aae00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a40b0090 con 0x7f17a4163c90 2024-04-25T11:30:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17aae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a4160c80 con 0x7f17a415de90 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17aae00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f17a4160f60 con 0x7f17a406dce0 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17a9000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f17a4163c90 0x7f17a406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58614/0 (socket says 172.21.15.19:58614) 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17a9000640 1 -- 172.21.15.19:0/3000816230 learned_addr learned my addr 172.21.15.19:0/3000816230 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17aa400640 1 --2- 172.21.15.19:0/3000816230 >> v2:172.21.15.19:3300/0 conn(0x7f17a406dce0 0x7f17a41606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.239+0000 7f17a3e00640 1 -- 172.21.15.19:0/3000816230 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1623303927 0 0) 0x7f17a40b0090 con 0x7f17a4163c90 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.240+0000 7f17a3e00640 1 -- 172.21.15.19:0/3000816230 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f178c003520 con 0x7f17a4163c90 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.240+0000 7f17aa400640 1 -- 172.21.15.19:0/3000816230 >> v1:172.21.15.169:6789/0 conn(0x7f17a415de90 legacy=0x7f17a406f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.240+0000 7f17aa400640 1 -- 172.21.15.19:0/3000816230 >> v1:172.21.15.19:6789/0 conn(0x7f17a4163c90 legacy=0x7f17a406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.240+0000 7f17aa400640 1 -- 172.21.15.19:0/3000816230 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a4161240 con 0x7f17a406dce0 2024-04-25T11:30:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.241+0000 7f17aa400640 1 --2- 172.21.15.19:0/3000816230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a406dce0 0x7f17a41606a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f179c009d90 tx=0x7f179c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee4ffbf1d114d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.241+0000 7f17a3e00640 1 -- 172.21.15.19:0/3000816230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c007740 con 0x7f17a406dce0 2024-04-25T11:30:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.241+0000 7f17a3e00640 1 -- 172.21.15.19:0/3000816230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f179c0078e0 con 0x7f17a406dce0 2024-04-25T11:30:28.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.241+0000 7f17a3e00640 1 -- 172.21.15.19:0/3000816230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c007cb0 con 0x7f17a406dce0 2024-04-25T11:30:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.242+0000 7f17aae00640 1 -- 172.21.15.19:0/3000816230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a406dce0 msgr2=0x7f17a41606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.242+0000 7f17aae00640 1 --2- 172.21.15.19:0/3000816230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a406dce0 0x7f17a41606a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f179c009d90 tx=0x7f179c02a660 comp rx=0 tx=0).stop 2024-04-25T11:30:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.242+0000 7f17aae00640 1 -- 172.21.15.19:0/3000816230 shutdown_connections 2024-04-25T11:30:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.242+0000 7f17aae00640 1 --2- 172.21.15.19:0/3000816230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a406dce0 0x7f17a41606a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.242+0000 7f17aae00640 1 -- 172.21.15.19:0/3000816230 >> 172.21.15.19:0/3000816230 conn(0x7f17a4072f40 msgr2=0x7f17a4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.242+0000 7f17aae00640 1 -- 172.21.15.19:0/3000816230 shutdown_connections 2024-04-25T11:30:28.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.242+0000 7f17aae00640 1 -- 172.21.15.19:0/3000816230 wait complete. 2024-04-25T11:30:28.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.245+0000 7f17aae00640 1 Processor -- start 2024-04-25T11:30:28.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.245+0000 7f17aae00640 1 -- start start 2024-04-25T11:30:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.246+0000 7f17aae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17a415de90 0x7f17a414b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.246+0000 7f17aae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17a4163c90 0x7f17a414e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a414e720 0x7f17a4162af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4070eb0 con 0x7f17a414e720 2024-04-25T11:30:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4070d30 con 0x7f17a4163c90 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f17a40b0570 con 0x7f17a415de90 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a414e720 0x7f17a4162af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a414e720 0x7f17a4162af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58858/0 (socket says 172.21.15.19:58858) 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 -- 172.21.15.19:0/1819331056 learned_addr learned my addr 172.21.15.19:0/1819331056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17a9000640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17a4163c90 0x7f17a414e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17a9a00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17a415de90 0x7f17a414b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 -- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17a415de90 msgr2=0x7f17a414b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17a415de90 0x7f17a414b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 -- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17a4163c90 msgr2=0x7f17a414e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17a4163c90 0x7f17a414e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:28.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17aa400640 1 -- 172.21.15.19:0/1819331056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a414c050 con 0x7f17a414e720 2024-04-25T11:30:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17a9000640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17a4163c90 0x7f17a414e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.247+0000 7f17a9a00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17a415de90 0x7f17a414b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.248+0000 7f17aa400640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a414e720 0x7f17a4162af0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f179c009eb0 tx=0x7f179c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.248+0000 7f17a1600640 1 -- 172.21.15.19:0/1819331056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c03f070 con 0x7f17a414e720 2024-04-25T11:30:28.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.248+0000 7f17a1600640 1 -- 172.21.15.19:0/1819331056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f179c031bc0 con 0x7f17a414e720 2024-04-25T11:30:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.248+0000 7f17a1600640 1 -- 172.21.15.19:0/1819331056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c009450 con 0x7f17a414e720 2024-04-25T11:30:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.248+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a414c330 con 0x7f17a414e720 2024-04-25T11:30:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.248+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a40c86d0 con 0x7f17a414e720 2024-04-25T11:30:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.250+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a40c8930 con 0x7f17a414e720 2024-04-25T11:30:28.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.251+0000 7f17a1600640 1 -- 172.21.15.19:0/1819331056 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f179c03b410 con 0x7f17a414e720 2024-04-25T11:30:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.252+0000 7f17a1600640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17840744b0 0x7f17840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.252+0000 7f17a1600640 1 -- 172.21.15.19:0/1819331056 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f179c0b6bd0 con 0x7f17a414e720 2024-04-25T11:30:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.253+0000 7f17a9a00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17840744b0 0x7f17840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.255+0000 7f17a9a00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17840744b0 0x7f17840769c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17a40b7880 tx=0x7f1798009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.258+0000 7f17a1600640 1 -- 172.21.15.19:0/1819331056 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f179c080d00 con 0x7f17a414e720 2024-04-25T11:30:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.471+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f17a4076c60 con 0x7f17a414e720 2024-04-25T11:30:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.479+0000 7f17a1600640 1 -- 172.21.15.19:0/1819331056 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v14)=0 v14) ==== 138+0+0 (secure 0 0 0) 0x7f179c085000 con 0x7f17a414e720 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.482+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17840744b0 msgr2=0x7f17840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.482+0000 7f17aae00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17840744b0 0x7f17840769c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f17a40b7880 tx=0x7f1798009040 comp rx=0 tx=0).stop 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.482+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a414e720 msgr2=0x7f17a4162af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.482+0000 7f17aae00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a414e720 0x7f17a4162af0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f179c009eb0 tx=0x7f179c004a40 comp rx=0 tx=0).stop 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 shutdown_connections 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17840744b0 0x7f17840769c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17a4163c90 0x7f17a414e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17a414e720 0x7f17a4162af0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 --2- 172.21.15.19:0/1819331056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17a415de90 0x7f17a414b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:28.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 >> 172.21.15.19:0/1819331056 conn(0x7f17a4072f40 msgr2=0x7f17a40b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:28.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 shutdown_connections 2024-04-25T11:30:28.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:28.483+0000 7f17aae00640 1 -- 172.21.15.19:0/1819331056 wait complete. 2024-04-25T11:30:33.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:30:33.386 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:30:33.504 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mv a/b a/c; sync .') 2024-04-25T11:30:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:30:33.631 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:30:33.641+0000 7fadd0a00640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5263:44 nref=7 cr=0x56422cb08a80)) (dversion lock w=1 last_client=5263) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x56422ca2c280] 2024-04-25T11:30:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:30:33.645 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:30:33.645 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:33.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.994+0000 7f07bb400640 1 Processor -- start 2024-04-25T11:30:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.994+0000 7f07bb400640 1 -- start start 2024-04-25T11:30:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.997+0000 7f07bb400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f07b40cf5e0 0x7f07b40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.997+0000 7f07bb400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b40cff30 con 0x7f07b41443c0 2024-04-25T11:30:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.997+0000 7f07bb400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b415f5f0 con 0x7f07b406c5c0 2024-04-25T11:30:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.997+0000 7f07bb400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f07b40708d0 con 0x7f07b40cf5e0 2024-04-25T11:30:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.998+0000 7f07b8a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f07b41443c0 0x7f07b406fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44572/0 (socket says 172.21.15.19:44572) 2024-04-25T11:30:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.998+0000 7f07b8a00640 1 -- 172.21.15.19:0/1398927659 learned_addr learned my addr 172.21.15.19:0/1398927659 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.998+0000 7f07b3e00640 1 --2- 172.21.15.19:0/1398927659 >> v2:172.21.15.169:3300/0 conn(0x7f07b40cf5e0 0x7f07b40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.998+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459765654 0 0) 0x7f07a4001490 con 0x7f07b41443c0 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.998+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0798003520 con 0x7f07b41443c0 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 913345934 0 0) 0x7f0798003520 con 0x7f07b41443c0 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 >> v2:172.21.15.169:3300/0 conn(0x7f07b40cf5e0 msgr2=0x7f07b40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 --2- 172.21.15.19:0/1398927659 >> v2:172.21.15.169:3300/0 conn(0x7f07b40cf5e0 0x7f07b40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 >> v1:172.21.15.169:6789/0 conn(0x7f07b406c5c0 legacy=0x7f07b406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b415e830 con 0x7f07b41443c0 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f07a4001bc0 con 0x7f07b41443c0 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 >> v1:172.21.15.19:6789/0 conn(0x7f07b41443c0 legacy=0x7f07b406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0798003a00 0x7f0798003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0798004800 0x7f0798004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798005270 0x7f079800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07b40708d0 con 0x7f0798004800 2024-04-25T11:30:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f079800c190 con 0x7f0798003a00 2024-04-25T11:30:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f079800c390 con 0x7f0798005270 2024-04-25T11:30:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3e00640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0798004800 0x7f0798004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b8a00640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0798003a00 0x7f0798003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:33.999+0000 7f07b3e00640 1 -- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798005270 msgr2=0x7f079800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.000+0000 7f07b3e00640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798005270 0x7f079800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.000+0000 7f07b3e00640 1 -- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0798003a00 msgr2=0x7f0798003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.000+0000 7f07b3e00640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0798003a00 0x7f0798003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.000+0000 7f07b3e00640 1 -- 172.21.15.19:0/1398927659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0798003520 con 0x7f0798004800 2024-04-25T11:30:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.000+0000 7f07b8a00640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0798003a00 0x7f0798003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.000+0000 7f07b9400640 1 -- 172.21.15.19:0/1398927659 reap_dead start 2024-04-25T11:30:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.001+0000 7f07b3e00640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0798004800 0x7f0798004c10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f079c009a90 tx=0x7f0798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866d620c54ccd0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.001+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c036070 con 0x7f0798004800 2024-04-25T11:30:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.001+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c031400 con 0x7f0798004800 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.001+0000 7f07b3400640 1 -- 172.21.15.19:0/1398927659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c009690 con 0x7f0798004800 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.002+0000 7f07bb400640 1 -- 172.21.15.19:0/1398927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0798004800 msgr2=0x7f0798004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.002+0000 7f07bb400640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0798004800 0x7f0798004c10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f079c009a90 tx=0x7f0798005830 comp rx=0 tx=0).stop 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.002+0000 7f07bb400640 1 -- 172.21.15.19:0/1398927659 shutdown_connections 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.002+0000 7f07bb400640 1 --2- 172.21.15.19:0/1398927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0798004800 0x7f0798004c10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.002+0000 7f07bb400640 1 -- 172.21.15.19:0/1398927659 >> 172.21.15.19:0/1398927659 conn(0x7f07b40d1ae0 msgr2=0x7f07b40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.002+0000 7f07bb400640 1 -- 172.21.15.19:0/1398927659 shutdown_connections 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.002+0000 7f07bb400640 1 -- 172.21.15.19:0/1398927659 wait complete. 2024-04-25T11:30:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.004+0000 7f07bb400640 1 Processor -- start 2024-04-25T11:30:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.005+0000 7f07bb400640 1 -- start start 2024-04-25T11:30:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.006+0000 7f07bb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798004800 0x7f07b40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.006+0000 7f07bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b413b720 0x7f07b41f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.006+0000 7f07bb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b415fad0 0x7f07b415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.006+0000 7f07bb400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07b415edf0 con 0x7f07b413b720 2024-04-25T11:30:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.006+0000 7f07bb400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07b415f160 con 0x7f07b415fad0 2024-04-25T11:30:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.006+0000 7f07bb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07b41447d0 con 0x7f0798004800 2024-04-25T11:30:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.006+0000 7f07b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b413b720 0x7f07b41f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b413b720 0x7f07b41f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55702/0 (socket says 172.21.15.19:55702) 2024-04-25T11:30:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 -- 172.21.15.19:0/466006797 learned_addr learned my addr 172.21.15.19:0/466006797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b8a00640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798004800 0x7f07b40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 -- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798004800 msgr2=0x7f07b40c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798004800 0x7f07b40c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b9400640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b415fad0 0x7f07b415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 -- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b415fad0 msgr2=0x7f07b415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b415fad0 0x7f07b415fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 -- 172.21.15.19:0/466006797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b4160420 con 0x7f07b413b720 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.007+0000 7f07b3e00640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b413b720 0x7f07b41f6130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f079c034040 tx=0x7f079c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.008+0000 7f07b0c00640 1 -- 172.21.15.19:0/466006797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c03f020 con 0x7f07b413b720 2024-04-25T11:30:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.008+0000 7f07b0c00640 1 -- 172.21.15.19:0/466006797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c037040 con 0x7f07b413b720 2024-04-25T11:30:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.008+0000 7f07b0c00640 1 -- 172.21.15.19:0/466006797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c036070 con 0x7f07b413b720 2024-04-25T11:30:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.008+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b4135a80 con 0x7f07b413b720 2024-04-25T11:30:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.008+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41fa240 con 0x7f07b413b720 2024-04-25T11:30:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.009+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4050740 con 0x7f07b413b720 2024-04-25T11:30:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.010+0000 7f07b0c00640 1 -- 172.21.15.19:0/466006797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f079c042020 con 0x7f07b413b720 2024-04-25T11:30:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.012+0000 7f07b0c00640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07900744b0 0x7f07900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.012+0000 7f07b0c00640 1 -- 172.21.15.19:0/466006797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f079c0b54a0 con 0x7f07b413b720 2024-04-25T11:30:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.013+0000 7f07b8a00640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07900744b0 0x7f07900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.014+0000 7f07b8a00640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07900744b0 0x7f07900769c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f07a4001630 tx=0x7f07a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.018+0000 7f07b0c00640 1 -- 172.21.15.19:0/466006797 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f079c049030 con 0x7f07b413b720 2024-04-25T11:30:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.223+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07b406f950 con 0x7f07b413b720 2024-04-25T11:30:34.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.226+0000 7f07b0c00640 1 -- 172.21.15.19:0/466006797 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16609 (secure 0 0 0) 0x7f07b4160420 con 0x7f07b413b720 2024-04-25T11:30:34.216 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:34.216 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:13.289711+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:30:10.008986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T11:30:10.799803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.228+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07900744b0 msgr2=0x7f07900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.228+0000 7f07bb400640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07900744b0 0x7f07900769c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f07a4001630 tx=0x7f07a4009040 comp rx=0 tx=0).stop 2024-04-25T11:30:34.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.228+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b413b720 msgr2=0x7f07b41f6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.228+0000 7f07bb400640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b413b720 0x7f07b41f6130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f079c034040 tx=0x7f079c004730 comp rx=0 tx=0).stop 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.228+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 shutdown_connections 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.229+0000 7f07bb400640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07900744b0 0x7f07900769c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.229+0000 7f07bb400640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07b415fad0 0x7f07b415fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.229+0000 7f07bb400640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b413b720 0x7f07b41f6130 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.229+0000 7f07bb400640 1 --2- 172.21.15.19:0/466006797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0798004800 0x7f07b40c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.229+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 >> 172.21.15.19:0/466006797 conn(0x7f07b40d1ae0 msgr2=0x7f07b415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.229+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 shutdown_connections 2024-04-25T11:30:34.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.229+0000 7f07bb400640 1 -- 172.21.15.19:0/466006797 wait complete. 2024-04-25T11:30:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:30:10.008986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:30:10.799803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:30:34.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:30:34.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.537+0000 7ff7b8600640 1 Processor -- start 2024-04-25T11:30:34.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.537+0000 7ff7b8600640 1 -- start start 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b8600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff7b00b0af0 0x7ff7b006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b8600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7b006db10 0x7ff7b0161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b8600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b00ceab0 con 0x7ff7b006db10 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b8600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b006df20 con 0x7ff7b00b0af0 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b00cdea0 con 0x7ff7b00cd9a0 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b6600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7b006db10 0x7ff7b0161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b6600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7b006db10 0x7ff7b0161ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55708/0 (socket says 172.21.15.19:55708) 2024-04-25T11:30:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.540+0000 7ff7b6600640 1 -- 172.21.15.19:0/940049545 learned_addr learned my addr 172.21.15.19:0/940049545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:34.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b5200640 1 --2- 172.21.15.19:0/940049545 >> v2:172.21.15.169:3300/0 conn(0x7ff7b00b0af0 0x7ff7b006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b5c00640 1 --2- 172.21.15.19:0/940049545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b6600640 1 -- 172.21.15.19:0/940049545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 msgr2=0x7ff7b015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b6600640 1 --2- 172.21.15.19:0/940049545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b6600640 1 -- 172.21.15.19:0/940049545 >> v2:172.21.15.169:3300/0 conn(0x7ff7b00b0af0 msgr2=0x7ff7b006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b6600640 1 --2- 172.21.15.19:0/940049545 >> v2:172.21.15.169:3300/0 conn(0x7ff7b00b0af0 0x7ff7b006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b6600640 1 -- 172.21.15.19:0/940049545 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b00ce180 con 0x7ff7b006db10 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b5200640 1 --2- 172.21.15.19:0/940049545 >> v2:172.21.15.169:3300/0 conn(0x7ff7b00b0af0 0x7ff7b006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.541+0000 7ff7b5c00640 1 --2- 172.21.15.19:0/940049545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.542+0000 7ff7b6600640 1 --2- 172.21.15.19:0/940049545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b006db10 0x7ff7b0161ed0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b4530 tx=0x7ff7a802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eab3d6a7defae95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.542+0000 7ff7afe00640 1 -- 172.21.15.19:0/940049545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8036070 con 0x7ff7b006db10 2024-04-25T11:30:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.542+0000 7ff7afe00640 1 -- 172.21.15.19:0/940049545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a8031600 con 0x7ff7b006db10 2024-04-25T11:30:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.542+0000 7ff7afe00640 1 -- 172.21.15.19:0/940049545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80319d0 con 0x7ff7b006db10 2024-04-25T11:30:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 -- 172.21.15.19:0/940049545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b006db10 msgr2=0x7ff7b0161ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 --2- 172.21.15.19:0/940049545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b006db10 0x7ff7b0161ed0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b4530 tx=0x7ff7a802a690 comp rx=0 tx=0).stop 2024-04-25T11:30:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 -- 172.21.15.19:0/940049545 shutdown_connections 2024-04-25T11:30:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 --2- 172.21.15.19:0/940049545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b006db10 0x7ff7b0161ed0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 --2- 172.21.15.19:0/940049545 >> v2:172.21.15.169:3300/0 conn(0x7ff7b00b0af0 0x7ff7b006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 --2- 172.21.15.19:0/940049545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 -- 172.21.15.19:0/940049545 >> 172.21.15.19:0/940049545 conn(0x7ff7b0073090 msgr2=0x7ff7b00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 -- 172.21.15.19:0/940049545 shutdown_connections 2024-04-25T11:30:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.543+0000 7ff7b8600640 1 -- 172.21.15.19:0/940049545 wait complete. 2024-04-25T11:30:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.546+0000 7ff7b8600640 1 Processor -- start 2024-04-25T11:30:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.546+0000 7ff7b8600640 1 -- start start 2024-04-25T11:30:34.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.547+0000 7ff7b8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b006db10 0x7ff7b01582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.547+0000 7ff7b8600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b00b0af0 0x7ff7b013a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b01f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b8600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b00b0f00 con 0x7ff7b00b0af0 2024-04-25T11:30:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b00ce740 con 0x7ff7b006db10 2024-04-25T11:30:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b00cef40 con 0x7ff7b00cd9a0 2024-04-25T11:30:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b00b0af0 0x7ff7b013a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b00b0af0 0x7ff7b013a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55710/0 (socket says 172.21.15.19:55710) 2024-04-25T11:30:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 -- 172.21.15.19:0/2232915137 learned_addr learned my addr 172.21.15.19:0/2232915137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:34.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5c00640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b006db10 0x7ff7b01582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b6600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b01f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 -- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 msgr2=0x7ff7b01f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:34.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b01f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 -- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b006db10 msgr2=0x7ff7b01582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b006db10 0x7ff7b01582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.548+0000 7ff7b5200640 1 -- 172.21.15.19:0/2232915137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b0158830 con 0x7ff7b00b0af0 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.549+0000 7ff7b6600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b01f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.549+0000 7ff7b5200640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b00b0af0 0x7ff7b013a090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00c7f0 tx=0x7ff79c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.549+0000 7ff7ad600640 1 -- 172.21.15.19:0/2232915137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c015070 con 0x7ff7b00b0af0 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.549+0000 7ff7ad600640 1 -- 172.21.15.19:0/2232915137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c007820 con 0x7ff7b00b0af0 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.549+0000 7ff7ad600640 1 -- 172.21.15.19:0/2232915137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c007c30 con 0x7ff7b00b0af0 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.549+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b0143fe0 con 0x7ff7b00b0af0 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.549+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b0157810 con 0x7ff7b00b0af0 2024-04-25T11:30:34.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.551+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b0050740 con 0x7ff7b00b0af0 2024-04-25T11:30:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.552+0000 7ff7ad600640 1 -- 172.21.15.19:0/2232915137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff79c018070 con 0x7ff7b00b0af0 2024-04-25T11:30:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.553+0000 7ff7ad600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7900744b0 0x7ff7900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.553+0000 7ff7ad600640 1 -- 172.21.15.19:0/2232915137 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7ff79c095290 con 0x7ff7b00b0af0 2024-04-25T11:30:34.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.554+0000 7ff7b5c00640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7900744b0 0x7ff7900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:34.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.555+0000 7ff7b5c00640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7900744b0 0x7ff7900769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400a8d0 tx=0x7ff7a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:34.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.558+0000 7ff7ad600640 1 -- 172.21.15.19:0/2232915137 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff79c05fb40 con 0x7ff7b00b0af0 2024-04-25T11:30:34.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.762+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7b01629a0 con 0x7ff7b00b0af0 2024-04-25T11:30:34.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.764+0000 7ff7ad600640 1 -- 172.21.15.19:0/2232915137 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7ff79c05f440 con 0x7ff7b00b0af0 2024-04-25T11:30:34.753 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.766+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7900744b0 msgr2=0x7ff7900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.766+0000 7ff7b8600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7900744b0 0x7ff7900769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400a8d0 tx=0x7ff7a4008040 comp rx=0 tx=0).stop 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.766+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b00b0af0 msgr2=0x7ff7b013a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.766+0000 7ff7b8600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b00b0af0 0x7ff7b013a090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00c7f0 tx=0x7ff79c00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 shutdown_connections 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7900744b0 0x7ff7900769c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7b00cd9a0 0x7ff7b01f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b00b0af0 0x7ff7b013a090 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 --2- 172.21.15.19:0/2232915137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7b006db10 0x7ff7b01582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 >> 172.21.15.19:0/2232915137 conn(0x7ff7b0073090 msgr2=0x7ff7b0147130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 shutdown_connections 2024-04-25T11:30:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:34.767+0000 7ff7b8600640 1 -- 172.21.15.19:0/2232915137 wait complete. 2024-04-25T11:30:34.784 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:30:35.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.083+0000 7f414be00640 1 Processor -- start 2024-04-25T11:30:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.084+0000 7f414be00640 1 -- start start 2024-04-25T11:30:35.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.086+0000 7f414be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f414c144340 0x7f414c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.086+0000 7f414be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.086+0000 7f414be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f414c06df20 con 0x7f414c15ec90 2024-04-25T11:30:35.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.086+0000 7f414be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f414c144750 con 0x7f414c144340 2024-04-25T11:30:35.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.086+0000 7f414be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f414c070000 con 0x7f414c06db10 2024-04-25T11:30:35.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.087+0000 7f414b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f414c15ec90 0x7f414c161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44588/0 (socket says 172.21.15.19:44588) 2024-04-25T11:30:35.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.087+0000 7f414b400640 1 -- 172.21.15.19:0/2396798693 learned_addr learned my addr 172.21.15.19:0/2396798693 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.087+0000 7f4143e00640 1 --2- 172.21.15.19:0/2396798693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.087+0000 7f414aa00640 1 --2- 172.21.15.19:0/2396798693 >> v2:172.21.15.169:3300/0 conn(0x7f414c144340 0x7f414c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.087+0000 7f414a000640 1 -- 172.21.15.19:0/2396798693 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542219635 0 0) 0x7f4144001490 con 0x7f414c15ec90 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.088+0000 7f414a000640 1 -- 172.21.15.19:0/2396798693 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4130003520 con 0x7f414c15ec90 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.088+0000 7f4143e00640 1 -- 172.21.15.19:0/2396798693 >> v2:172.21.15.169:3300/0 conn(0x7f414c144340 msgr2=0x7f414c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.088+0000 7f4143e00640 1 --2- 172.21.15.19:0/2396798693 >> v2:172.21.15.169:3300/0 conn(0x7f414c144340 0x7f414c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.088+0000 7f414a000640 1 -- 172.21.15.19:0/2396798693 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1125863113 0 0) 0x7f4130003520 con 0x7f414c15ec90 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.088+0000 7f4143e00640 1 -- 172.21.15.19:0/2396798693 >> v1:172.21.15.19:6789/0 conn(0x7f414c15ec90 legacy=0x7f414c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.088+0000 7f4143e00640 1 -- 172.21.15.19:0/2396798693 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c06f240 con 0x7f414c06db10 2024-04-25T11:30:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.089+0000 7f4143e00640 1 --2- 172.21.15.19:0/2396798693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c15e750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41340097a0 tx=0x7f413402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58defdf5cc6bacde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.090+0000 7f414a000640 1 -- 172.21.15.19:0/2396798693 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134037070 con 0x7f414c06db10 2024-04-25T11:30:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.090+0000 7f414a000640 1 -- 172.21.15.19:0/2396798693 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413402aba0 con 0x7f414c06db10 2024-04-25T11:30:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.090+0000 7f414a000640 1 -- 172.21.15.19:0/2396798693 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144001bc0 con 0x7f414c06db10 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 -- 172.21.15.19:0/2396798693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 msgr2=0x7f414c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 --2- 172.21.15.19:0/2396798693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c15e750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41340097a0 tx=0x7f413402a110 comp rx=0 tx=0).stop 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 -- 172.21.15.19:0/2396798693 shutdown_connections 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 --2- 172.21.15.19:0/2396798693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c15e750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 --2- 172.21.15.19:0/2396798693 >> v2:172.21.15.169:3300/0 conn(0x7f414c144340 0x7f414c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 -- 172.21.15.19:0/2396798693 >> 172.21.15.19:0/2396798693 conn(0x7f414c072f40 msgr2=0x7f414c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 -- 172.21.15.19:0/2396798693 shutdown_connections 2024-04-25T11:30:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.091+0000 7f414be00640 1 -- 172.21.15.19:0/2396798693 wait complete. 2024-04-25T11:30:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.093+0000 7f414be00640 1 Processor -- start 2024-04-25T11:30:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.094+0000 7f414be00640 1 -- start start 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.095+0000 7f414be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.095+0000 7f414be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414c144340 0x7f414c161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.095+0000 7f414be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c15ec90 0x7f414c161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.095+0000 7f414be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f414c06fae0 con 0x7f414c144340 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.095+0000 7f414be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f414c06f800 con 0x7f414c15ec90 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.095+0000 7f414be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f414c15f0a0 con 0x7f414c06db10 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414c144340 0x7f414c161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414c144340 0x7f414c161670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55716/0 (socket says 172.21.15.19:55716) 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 -- 172.21.15.19:0/321454448 learned_addr learned my addr 172.21.15.19:0/321454448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f414aa00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f414b400640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c15ec90 0x7f414c161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 -- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 msgr2=0x7f414c144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 -- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c15ec90 msgr2=0x7f414c161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c15ec90 0x7f414c161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.096+0000 7f4143e00640 1 -- 172.21.15.19:0/321454448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c1f65e0 con 0x7f414c144340 2024-04-25T11:30:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.097+0000 7f4143e00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414c144340 0x7f414c161670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4134035040 tx=0x7f41340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.097+0000 7f4142a00640 1 -- 172.21.15.19:0/321454448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134041030 con 0x7f414c144340 2024-04-25T11:30:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.097+0000 7f4142a00640 1 -- 172.21.15.19:0/321454448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413403d420 con 0x7f414c144340 2024-04-25T11:30:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.097+0000 7f4142a00640 1 -- 172.21.15.19:0/321454448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134037040 con 0x7f414c144340 2024-04-25T11:30:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.097+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c1f68c0 con 0x7f414c144340 2024-04-25T11:30:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.097+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c1f8570 con 0x7f414c144340 2024-04-25T11:30:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.098+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414c076c60 con 0x7f414c144340 2024-04-25T11:30:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.100+0000 7f4142a00640 1 -- 172.21.15.19:0/321454448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4134044070 con 0x7f414c144340 2024-04-25T11:30:35.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.101+0000 7f4142a00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f41280744b0 0x7f41280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.101+0000 7f4142a00640 1 -- 172.21.15.19:0/321454448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f41340b5540 con 0x7f414c144340 2024-04-25T11:30:35.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.101+0000 7f414aa00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f41280744b0 0x7f41280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.103+0000 7f414aa00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f41280744b0 0x7f41280769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f413c009a30 tx=0x7f413c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.106+0000 7f4142a00640 1 -- 172.21.15.19:0/321454448 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f413407fc30 con 0x7f414c144340 2024-04-25T11:30:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.316+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f414c0b7920 con 0x7f414c144340 2024-04-25T11:30:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.317+0000 7f4142a00640 1 -- 172.21.15.19:0/321454448 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) ==== 104+0+11 (secure 0 0 0) 0x7f413407f530 con 0x7f414c144340 2024-04-25T11:30:35.307 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.319+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f41280744b0 msgr2=0x7f41280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.319+0000 7f414be00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f41280744b0 0x7f41280769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f413c009a30 tx=0x7f413c008040 comp rx=0 tx=0).stop 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.319+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414c144340 msgr2=0x7f414c161670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.319+0000 7f414be00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414c144340 0x7f414c161670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4134035040 tx=0x7f41340046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 shutdown_connections 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f41280744b0 0x7f41280769c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c15ec90 0x7f414c161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f414c144340 0x7f414c161670 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 --2- 172.21.15.19:0/321454448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f414c06db10 0x7f414c144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 >> 172.21.15.19:0/321454448 conn(0x7f414c072f40 msgr2=0x7f414c0b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 shutdown_connections 2024-04-25T11:30:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.320+0000 7f414be00640 1 -- 172.21.15.19:0/321454448 wait complete. 2024-04-25T11:30:35.338 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:30:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.640+0000 7f1627e00640 1 Processor -- start 2024-04-25T11:30:35.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.640+0000 7f1627e00640 1 -- start start 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1627e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f162806f930 0x7f162806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1627e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f162806fd40 0x7f162806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1627e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16281623f0 con 0x7f1628070d20 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1627e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f16281626d0 con 0x7f162806fd40 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1627e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f162806f130 con 0x7f162806f930 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1626a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1628070d20 0x7f1628161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44600/0 (socket says 172.21.15.19:44600) 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1626a00640 1 -- 172.21.15.19:0/490996693 learned_addr learned my addr 172.21.15.19:0/490996693 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1627400640 1 --2- 172.21.15.19:0/490996693 >> v2:172.21.15.19:3300/0 conn(0x7f162806fd40 0x7f162806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.643+0000 7f1626000640 1 --2- 172.21.15.19:0/490996693 >> v2:172.21.15.169:3300/0 conn(0x7f162806f930 0x7f162806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.644+0000 7f1625600640 1 -- 172.21.15.19:0/490996693 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1687159997 0 0) 0x7f16281623f0 con 0x7f1628070d20 2024-04-25T11:30:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.644+0000 7f1625600640 1 -- 172.21.15.19:0/490996693 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f160c003520 con 0x7f1628070d20 2024-04-25T11:30:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.644+0000 7f1627400640 1 -- 172.21.15.19:0/490996693 >> v2:172.21.15.169:3300/0 conn(0x7f162806f930 msgr2=0x7f162806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.644+0000 7f1627400640 1 --2- 172.21.15.19:0/490996693 >> v2:172.21.15.169:3300/0 conn(0x7f162806f930 0x7f162806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.644+0000 7f1627400640 1 -- 172.21.15.19:0/490996693 >> v1:172.21.15.19:6789/0 conn(0x7f1628070d20 legacy=0x7f1628161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.644+0000 7f1627400640 1 -- 172.21.15.19:0/490996693 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162806f410 con 0x7f162806fd40 2024-04-25T11:30:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.645+0000 7f1627400640 1 --2- 172.21.15.19:0/490996693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162806db10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1628161a90 tx=0x7f162002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a05ecd90de9de12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:35.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.645+0000 7f1625600640 1 -- 172.21.15.19:0/490996693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620037070 con 0x7f162806fd40 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.645+0000 7f1625600640 1 -- 172.21.15.19:0/490996693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1620032440 con 0x7f162806fd40 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.645+0000 7f1625600640 1 -- 172.21.15.19:0/490996693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162002ae60 con 0x7f162806fd40 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 -- 172.21.15.19:0/490996693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 msgr2=0x7f162806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 --2- 172.21.15.19:0/490996693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162806db10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1628161a90 tx=0x7f162002a110 comp rx=0 tx=0).stop 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 -- 172.21.15.19:0/490996693 shutdown_connections 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 --2- 172.21.15.19:0/490996693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162806db10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 --2- 172.21.15.19:0/490996693 >> v2:172.21.15.169:3300/0 conn(0x7f162806f930 0x7f162806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 -- 172.21.15.19:0/490996693 >> 172.21.15.19:0/490996693 conn(0x7f1628073160 msgr2=0x7f1628073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 -- 172.21.15.19:0/490996693 shutdown_connections 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.646+0000 7f1627e00640 1 -- 172.21.15.19:0/490996693 wait complete. 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.649+0000 7f1627e00640 1 Processor -- start 2024-04-25T11:30:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.649+0000 7f1627e00640 1 -- start start 2024-04-25T11:30:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.650+0000 7f1627e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162806f930 0x7f1628150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.650+0000 7f1627e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162813c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.650+0000 7f1627e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1628070d20 0x7f162813c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.650+0000 7f1627e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f162815e470 con 0x7f162806fd40 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.650+0000 7f1627e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f162815e2f0 con 0x7f1628070d20 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.650+0000 7f1627e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f16280ce7e0 con 0x7f162806f930 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162813c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162813c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55726/0 (socket says 172.21.15.19:55726) 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 -- 172.21.15.19:0/1262917159 learned_addr learned my addr 172.21.15.19:0/1262917159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:35.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626a00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162806f930 0x7f1628150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 -- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162806f930 msgr2=0x7f1628150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162806f930 0x7f1628150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 -- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1628070d20 msgr2=0x7f162813c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1628070d20 0x7f162813c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.651+0000 7f1626000640 1 -- 172.21.15.19:0/1262917159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16281511e0 con 0x7f162806fd40 2024-04-25T11:30:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.652+0000 7f1626000640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162813c120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f161400d620 tx=0x7f161400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.652+0000 7f161ea00640 1 -- 172.21.15.19:0/1262917159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140125f0 con 0x7f162806fd40 2024-04-25T11:30:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.652+0000 7f161ea00640 1 -- 172.21.15.19:0/1262917159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1614019070 con 0x7f162806fd40 2024-04-25T11:30:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.652+0000 7f161ea00640 1 -- 172.21.15.19:0/1262917159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140129d0 con 0x7f162806fd40 2024-04-25T11:30:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.652+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16281514c0 con 0x7f162806fd40 2024-04-25T11:30:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.652+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162813cc60 con 0x7f162806fd40 2024-04-25T11:30:35.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.653+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1628076c60 con 0x7f162806fd40 2024-04-25T11:30:35.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.655+0000 7f161ea00640 1 -- 172.21.15.19:0/1262917159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1614012b70 con 0x7f162806fd40 2024-04-25T11:30:35.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.656+0000 7f161ea00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f16080744e0 0x7f16080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:35.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.656+0000 7f161ea00640 1 -- 172.21.15.19:0/1262917159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f16140955d0 con 0x7f162806fd40 2024-04-25T11:30:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.656+0000 7f1626a00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f16080744e0 0x7f16080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:35.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.658+0000 7f1626a00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f16080744e0 0x7f16080769f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16180012e0 tx=0x7f1618009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.660+0000 7f161ea00640 1 -- 172.21.15.19:0/1262917159 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f161405fcc0 con 0x7f162806fd40 2024-04-25T11:30:35.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.866+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f16280ce360 con 0x7f162806fd40 2024-04-25T11:30:35.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.868+0000 7f161ea00640 1 -- 172.21.15.19:0/1262917159 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) ==== 100+0+11 (secure 0 0 0) 0x7f161405f5c0 con 0x7f162806fd40 2024-04-25T11:30:35.857 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:30:35.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f16080744e0 msgr2=0x7f16080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f16080744e0 0x7f16080769f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16180012e0 tx=0x7f1618009040 comp rx=0 tx=0).stop 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 msgr2=0x7f162813c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162813c120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f161400d620 tx=0x7f161400dae0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 shutdown_connections 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f16080744e0 0x7f16080769f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1628070d20 0x7f162813c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162806fd40 0x7f162813c120 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.870+0000 7f1627e00640 1 --2- 172.21.15.19:0/1262917159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f162806f930 0x7f1628150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.871+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 >> 172.21.15.19:0/1262917159 conn(0x7f1628073160 msgr2=0x7f16280684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.871+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 shutdown_connections 2024-04-25T11:30:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:35.871+0000 7f1627e00640 1 -- 172.21.15.19:0/1262917159 wait complete. 2024-04-25T11:30:35.886 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:30:36.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.173+0000 7f8361000640 1 Processor -- start 2024-04-25T11:30:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.173+0000 7f8361000640 1 -- start start 2024-04-25T11:30:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.175+0000 7f8361000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f835c06c5c0 0x7f835c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.175+0000 7f8361000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f835c144370 0x7f835c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.175+0000 7f8361000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c0b50e0 con 0x7f835c0b4cd0 2024-04-25T11:30:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.175+0000 7f8361000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0d1230 con 0x7f835c06c5c0 2024-04-25T11:30:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.175+0000 7f8361000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0d06b0 con 0x7f835c144370 2024-04-25T11:30:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f835c06c5c0 0x7f835c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f835c0b4cd0 0x7f835c161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44608/0 (socket says 172.21.15.19:44608) 2024-04-25T11:30:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835be00640 1 -- 172.21.15.19:0/3866944103 learned_addr learned my addr 172.21.15.19:0/3866944103 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f8353e00640 1 --2- 172.21.15.19:0/3866944103 >> v2:172.21.15.169:3300/0 conn(0x7f835c144370 0x7f835c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835aa00640 1 -- 172.21.15.19:0/3866944103 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760338385 0 0) 0x7f8354001490 con 0x7f835c0b4cd0 2024-04-25T11:30:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835b400640 1 -- 172.21.15.19:0/3866944103 >> v2:172.21.15.169:3300/0 conn(0x7f835c144370 msgr2=0x7f835c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835b400640 1 --2- 172.21.15.19:0/3866944103 >> v2:172.21.15.169:3300/0 conn(0x7f835c144370 0x7f835c0b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835b400640 1 -- 172.21.15.19:0/3866944103 >> v1:172.21.15.19:6789/0 conn(0x7f835c0b4cd0 legacy=0x7f835c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.176+0000 7f835b400640 1 -- 172.21.15.19:0/3866944103 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0d0990 con 0x7f835c06c5c0 2024-04-25T11:30:36.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.177+0000 7f835b400640 1 --2- 172.21.15.19:0/3866944103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c0cfa70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f835c0b5550 tx=0x7f834c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe93e8b146d7ef14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:36.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.178+0000 7f835aa00640 1 -- 172.21.15.19:0/3866944103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c037070 con 0x7f835c06c5c0 2024-04-25T11:30:36.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.178+0000 7f835aa00640 1 -- 172.21.15.19:0/3866944103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834c032440 con 0x7f835c06c5c0 2024-04-25T11:30:36.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.178+0000 7f835aa00640 1 -- 172.21.15.19:0/3866944103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c032810 con 0x7f835c06c5c0 2024-04-25T11:30:36.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.178+0000 7f8361000640 1 -- 172.21.15.19:0/3866944103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 msgr2=0x7f835c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.178+0000 7f8361000640 1 --2- 172.21.15.19:0/3866944103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c0cfa70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f835c0b5550 tx=0x7f834c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.179+0000 7f8361000640 1 -- 172.21.15.19:0/3866944103 shutdown_connections 2024-04-25T11:30:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.179+0000 7f8361000640 1 --2- 172.21.15.19:0/3866944103 >> v2:172.21.15.169:3300/0 conn(0x7f835c144370 0x7f835c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.179+0000 7f8361000640 1 --2- 172.21.15.19:0/3866944103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c0cfa70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.179+0000 7f8361000640 1 -- 172.21.15.19:0/3866944103 >> 172.21.15.19:0/3866944103 conn(0x7f835c0d1720 msgr2=0x7f835c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.179+0000 7f8361000640 1 -- 172.21.15.19:0/3866944103 shutdown_connections 2024-04-25T11:30:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.179+0000 7f8361000640 1 -- 172.21.15.19:0/3866944103 wait complete. 2024-04-25T11:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.181+0000 7f8361000640 1 Processor -- start 2024-04-25T11:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.181+0000 7f8361000640 1 -- start start 2024-04-25T11:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.183+0000 7f8361000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.183+0000 7f8361000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f835c0b4cd0 0x7f835c145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.183+0000 7f8361000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f835c144370 0x7f835c0af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.183+0000 7f8361000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0d02d0 con 0x7f835c06c5c0 2024-04-25T11:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.183+0000 7f8361000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0d0f50 con 0x7f835c0b4cd0 2024-04-25T11:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.183+0000 7f8361000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f835c0d0450 con 0x7f835c144370 2024-04-25T11:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55732/0 (socket says 172.21.15.19:55732) 2024-04-25T11:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 -- 172.21.15.19:0/1793315211 learned_addr learned my addr 172.21.15.19:0/1793315211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f8353e00640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f835c0b4cd0 0x7f835c145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835be00640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f835c144370 0x7f835c0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 -- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f835c144370 msgr2=0x7f835c0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f835c144370 0x7f835c0af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 -- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f835c0b4cd0 msgr2=0x7f835c145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f835c0b4cd0 0x7f835c145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835b400640 1 -- 172.21.15.19:0/1793315211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c1f7030 con 0x7f835c06c5c0 2024-04-25T11:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f8353e00640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f835c0b4cd0 0x7f835c145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.184+0000 7f835be00640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f835c144370 0x7f835c0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.185+0000 7f835b400640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c15d2c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f834c000c00 tx=0x7f834c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.185+0000 7f8353400640 1 -- 172.21.15.19:0/1793315211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c037040 con 0x7f835c06c5c0 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.185+0000 7f8353400640 1 -- 172.21.15.19:0/1793315211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834c042070 con 0x7f835c06c5c0 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.185+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c145e10 con 0x7f835c06c5c0 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.185+0000 7f8353400640 1 -- 172.21.15.19:0/1793315211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c030bd0 con 0x7f835c06c5c0 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.185+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c15fbf0 con 0x7f835c06c5c0 2024-04-25T11:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.186+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c1f7200 con 0x7f835c06c5c0 2024-04-25T11:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.188+0000 7f8353400640 1 -- 172.21.15.19:0/1793315211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f834c047460 con 0x7f835c06c5c0 2024-04-25T11:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.189+0000 7f8353400640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83380744b0 0x7f83380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.189+0000 7f8353400640 1 -- 172.21.15.19:0/1793315211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f834c0b5ed0 con 0x7f835c06c5c0 2024-04-25T11:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.189+0000 7f8353e00640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83380744b0 0x7f83380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.191+0000 7f8353e00640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83380744b0 0x7f83380769c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8344006fd0 tx=0x7f8344008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:36.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.194+0000 7f8353400640 1 -- 172.21.15.19:0/1793315211 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f834c032e20 con 0x7f835c06c5c0 2024-04-25T11:30:36.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.422+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f835c050740 con 0x7f835c06c5c0 2024-04-25T11:30:36.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.423+0000 7f8353400640 1 -- 172.21.15.19:0/1793315211 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7f834c0805c0 con 0x7f835c06c5c0 2024-04-25T11:30:36.413 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:30:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83380744b0 msgr2=0x7f83380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83380744b0 0x7f83380769c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8344006fd0 tx=0x7f8344008040 comp rx=0 tx=0).stop 2024-04-25T11:30:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 msgr2=0x7f835c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c15d2c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f834c000c00 tx=0x7f834c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 shutdown_connections 2024-04-25T11:30:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83380744b0 0x7f83380769c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f835c0b4cd0 0x7f835c145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f835c144370 0x7f835c0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 --2- 172.21.15.19:0/1793315211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f835c06c5c0 0x7f835c15d2c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 >> 172.21.15.19:0/1793315211 conn(0x7f835c0d1720 msgr2=0x7f835c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 shutdown_connections 2024-04-25T11:30:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.426+0000 7f8361000640 1 -- 172.21.15.19:0/1793315211 wait complete. 2024-04-25T11:30:36.440 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:30:36.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:36.440 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:30:36.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:30:36.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:30:36.536 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:30:36.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.882+0000 7f5b85000640 1 Processor -- start 2024-04-25T11:30:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.882+0000 7f5b85000640 1 -- start start 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.884+0000 7f5b85000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b801443c0 0x7f5b8006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b85000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b80070370 0x7f5b800cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b85000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b800cff30 con 0x7f5b800cfb20 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b85000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5b800708d0 con 0x7f5b80070370 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b85000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8015f160 con 0x7f5b801443c0 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b7e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5b800cfb20 0x7f5b800ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44618/0 (socket says 172.21.15.19:44618) 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b7e800640 1 -- 172.21.15.19:0/1719515098 learned_addr learned my addr 172.21.15.19:0/1719515098 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b77e00640 1 --2- 172.21.15.19:0/1719515098 >> v2:172.21.15.19:3300/0 conn(0x7f5b80070370 0x7f5b800cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.885+0000 7f5b7de00640 1 --2- 172.21.15.19:0/1719515098 >> v2:172.21.15.169:3300/0 conn(0x7f5b801443c0 0x7f5b8006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.886+0000 7f5b7d400640 1 -- 172.21.15.19:0/1719515098 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299110321 0 0) 0x7f5b78001490 con 0x7f5b800cfb20 2024-04-25T11:30:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.886+0000 7f5b7d400640 1 -- 172.21.15.19:0/1719515098 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b64003520 con 0x7f5b800cfb20 2024-04-25T11:30:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.886+0000 7f5b77e00640 1 -- 172.21.15.19:0/1719515098 >> v2:172.21.15.169:3300/0 conn(0x7f5b801443c0 msgr2=0x7f5b8006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.886+0000 7f5b77e00640 1 --2- 172.21.15.19:0/1719515098 >> v2:172.21.15.169:3300/0 conn(0x7f5b801443c0 0x7f5b8006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.886+0000 7f5b77e00640 1 -- 172.21.15.19:0/1719515098 >> v1:172.21.15.19:6789/0 conn(0x7f5b800cfb20 legacy=0x7f5b800ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.886+0000 7f5b77e00640 1 -- 172.21.15.19:0/1719515098 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8015f5f0 con 0x7f5b80070370 2024-04-25T11:30:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.887+0000 7f5b77e00640 1 --2- 172.21.15.19:0/1719515098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b800cf5e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5b80072820 tx=0x7f5b6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1b528b6f260f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.887+0000 7f5b7d400640 1 -- 172.21.15.19:0/1719515098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68037070 con 0x7f5b80070370 2024-04-25T11:30:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.887+0000 7f5b7d400640 1 -- 172.21.15.19:0/1719515098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68032440 con 0x7f5b80070370 2024-04-25T11:30:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.887+0000 7f5b7d400640 1 -- 172.21.15.19:0/1719515098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68032810 con 0x7f5b80070370 2024-04-25T11:30:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 -- 172.21.15.19:0/1719515098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 msgr2=0x7f5b800cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 --2- 172.21.15.19:0/1719515098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b800cf5e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5b80072820 tx=0x7f5b6802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 -- 172.21.15.19:0/1719515098 shutdown_connections 2024-04-25T11:30:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 --2- 172.21.15.19:0/1719515098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b800cf5e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 --2- 172.21.15.19:0/1719515098 >> v2:172.21.15.169:3300/0 conn(0x7f5b801443c0 0x7f5b8006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 -- 172.21.15.19:0/1719515098 >> 172.21.15.19:0/1719515098 conn(0x7f5b800d1ae0 msgr2=0x7f5b800d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 -- 172.21.15.19:0/1719515098 shutdown_connections 2024-04-25T11:30:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.888+0000 7f5b85000640 1 -- 172.21.15.19:0/1719515098 wait complete. 2024-04-25T11:30:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.890+0000 7f5b85000640 1 Processor -- start 2024-04-25T11:30:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.891+0000 7f5b85000640 1 -- start start 2024-04-25T11:30:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.892+0000 7f5b85000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b80131a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.892+0000 7f5b85000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b800cfb20 0x7f5b80152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.892+0000 7f5b85000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b801443c0 0x7f5b801f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.892+0000 7f5b85000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8015edf0 con 0x7f5b80070370 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.892+0000 7f5b85000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8015eb10 con 0x7f5b800cfb20 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.892+0000 7f5b85000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8006d5d0 con 0x7f5b801443c0 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b80131a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b80131a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55740/0 (socket says 172.21.15.19:55740) 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 -- 172.21.15.19:0/284119588 learned_addr learned my addr 172.21.15.19:0/284119588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b77e00640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b800cfb20 0x7f5b80152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7e800640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b801443c0 0x7f5b801f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 -- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b801443c0 msgr2=0x7f5b801f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b801443c0 0x7f5b801f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 -- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b800cfb20 msgr2=0x7f5b80152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b800cfb20 0x7f5b80152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.893+0000 7f5b7de00640 1 -- 172.21.15.19:0/284119588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8013e650 con 0x7f5b80070370 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.894+0000 7f5b7de00640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b80131a80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000d5d0 tx=0x7f5b7000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.894+0000 7f5b76000640 1 -- 172.21.15.19:0/284119588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70004240 con 0x7f5b80070370 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.894+0000 7f5b76000640 1 -- 172.21.15.19:0/284119588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b700043e0 con 0x7f5b80070370 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.894+0000 7f5b76000640 1 -- 172.21.15.19:0/284119588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70003830 con 0x7f5b80070370 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.894+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8013e930 con 0x7f5b80070370 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.894+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8013eef0 con 0x7f5b80070370 2024-04-25T11:30:36.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.895+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b80050740 con 0x7f5b80070370 2024-04-25T11:30:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.896+0000 7f5b76000640 1 -- 172.21.15.19:0/284119588 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b700039d0 con 0x7f5b80070370 2024-04-25T11:30:36.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.897+0000 7f5b76000640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b5c0744b0 0x7f5b5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:36.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.897+0000 7f5b76000640 1 -- 172.21.15.19:0/284119588 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f5b700955f0 con 0x7f5b80070370 2024-04-25T11:30:36.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.898+0000 7f5b77e00640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b5c0744b0 0x7f5b5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:36.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.899+0000 7f5b77e00640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b5c0744b0 0x7f5b5c0769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b68034000 tx=0x7f5b6802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:36.902+0000 7f5b76000640 1 -- 172.21.15.19:0/284119588 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b7005fce0 con 0x7f5b80070370 2024-04-25T11:30:36.939 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:30:36.949+0000 7fb164400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T11:30:37.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.147+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8006f950 con 0x7f5b80070370 2024-04-25T11:30:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.148+0000 7f5b76000640 1 -- 172.21.15.19:0/284119588 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5184 (secure 0 0 0) 0x7f5b7005f5e0 con 0x7f5b80070370 2024-04-25T11:30:37.138 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:37.139 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:36.342557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":9,"addr":"172.21.15.19:6835/3038680263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3038680263},{"type":"v1","addr":"172.21.15.19:6835","nonce":3038680263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":3}]} 2024-04-25T11:30:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-04-25T11:30:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b5c0744b0 msgr2=0x7f5b5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b5c0744b0 0x7f5b5c0769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5b68034000 tx=0x7f5b6802e000 comp rx=0 tx=0).stop 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 msgr2=0x7f5b80131a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b80131a80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000d5d0 tx=0x7f5b7000da90 comp rx=0 tx=0).stop 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 shutdown_connections 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5b5c0744b0 0x7f5b5c0769c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b800cfb20 0x7f5b80152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b801443c0 0x7f5b801f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 --2- 172.21.15.19:0/284119588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b80070370 0x7f5b80131a80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.151+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 >> 172.21.15.19:0/284119588 conn(0x7f5b800d1ae0 msgr2=0x7f5b8015d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.152+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 shutdown_connections 2024-04-25T11:30:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.152+0000 7f5b85000640 1 -- 172.21.15.19:0/284119588 wait complete. 2024-04-25T11:30:37.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.468+0000 7f84da200640 1 Processor -- start 2024-04-25T11:30:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.468+0000 7f84da200640 1 -- start start 2024-04-25T11:30:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84da200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d40b6e10 0x7f84d415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84da200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84d41608d0 0x7f84d4160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84da200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d4161220 con 0x7f84d40cf2f0 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84da200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f84d406f870 con 0x7f84d41608d0 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84da200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84d40b7220 con 0x7f84d40b6e10 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84d9800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84d41608d0 0x7f84d4160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84d9800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84d41608d0 0x7f84d4160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55748/0 (socket says 172.21.15.19:55748) 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.470+0000 7f84d9800640 1 -- 172.21.15.19:0/1814962237 learned_addr learned my addr 172.21.15.19:0/1814962237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.471+0000 7f84d3e00640 1 --2- 172.21.15.19:0/1814962237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d40b6e10 0x7f84d415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.471+0000 7f84d9800640 1 -- 172.21.15.19:0/1814962237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d40b6e10 msgr2=0x7f84d415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.471+0000 7f84d9800640 1 --2- 172.21.15.19:0/1814962237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d40b6e10 0x7f84d415a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.471+0000 7f84d9800640 1 -- 172.21.15.19:0/1814962237 >> v1:172.21.15.169:6789/0 conn(0x7f84d40cf2f0 legacy=0x7f84d40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.471+0000 7f84d9800640 1 -- 172.21.15.19:0/1814962237 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d40b74d0 con 0x7f84d41608d0 2024-04-25T11:30:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.471+0000 7f84d3e00640 1 --2- 172.21.15.19:0/1814962237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d40b6e10 0x7f84d415a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.472+0000 7f84d9800640 1 --2- 172.21.15.19:0/1814962237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d41608d0 0x7f84d4160ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009e50 tx=0x7f84cc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a910a5ee6980a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.472+0000 7f84d3400640 1 -- 172.21.15.19:0/1814962237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc036070 con 0x7f84d41608d0 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.472+0000 7f84d3400640 1 -- 172.21.15.19:0/1814962237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84cc031600 con 0x7f84d41608d0 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.472+0000 7f84d3400640 1 -- 172.21.15.19:0/1814962237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0319d0 con 0x7f84d41608d0 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 -- 172.21.15.19:0/1814962237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d41608d0 msgr2=0x7f84d4160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 --2- 172.21.15.19:0/1814962237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d41608d0 0x7f84d4160ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009e50 tx=0x7f84cc02a6c0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 -- 172.21.15.19:0/1814962237 shutdown_connections 2024-04-25T11:30:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 --2- 172.21.15.19:0/1814962237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d41608d0 0x7f84d4160ce0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 --2- 172.21.15.19:0/1814962237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d40b6e10 0x7f84d415a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 -- 172.21.15.19:0/1814962237 >> 172.21.15.19:0/1814962237 conn(0x7f84d40d1720 msgr2=0x7f84d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 -- 172.21.15.19:0/1814962237 shutdown_connections 2024-04-25T11:30:37.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.473+0000 7f84da200640 1 -- 172.21.15.19:0/1814962237 wait complete. 2024-04-25T11:30:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.475+0000 7f84da200640 1 Processor -- start 2024-04-25T11:30:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.476+0000 7f84da200640 1 -- start start 2024-04-25T11:30:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84da200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84d40b6e10 0x7f84d415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84da200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d40cf2f0 0x7f84d41492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84da200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d41608d0 0x7f84d415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84da200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84d406f500 con 0x7f84d40cf2f0 2024-04-25T11:30:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84da200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f84d406f220 con 0x7f84d40b6e10 2024-04-25T11:30:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84da200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84d406fd00 con 0x7f84d41608d0 2024-04-25T11:30:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d40cf2f0 0x7f84d41492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d40cf2f0 0x7f84d41492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55764/0 (socket says 172.21.15.19:55764) 2024-04-25T11:30:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.477+0000 7f84d3e00640 1 -- 172.21.15.19:0/1177990800 learned_addr learned my addr 172.21.15.19:0/1177990800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d3e00640 1 -- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d41608d0 msgr2=0x7f84d415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d3e00640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d41608d0 0x7f84d415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d3e00640 1 -- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84d40b6e10 msgr2=0x7f84d415caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d3e00640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84d40b6e10 0x7f84d415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d3e00640 1 -- 172.21.15.19:0/1177990800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d415b5e0 con 0x7f84d40cf2f0 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d3e00640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d40cf2f0 0x7f84d41492c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c7f0 tx=0x7f84c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d0c00640 1 -- 172.21.15.19:0/1177990800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0015070 con 0x7f84d40cf2f0 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.479+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d415b8c0 con 0x7f84d40cf2f0 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.478+0000 7f84d0c00640 1 -- 172.21.15.19:0/1177990800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c00077c0 con 0x7f84d40cf2f0 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.479+0000 7f84d0c00640 1 -- 172.21.15.19:0/1177990800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0007b70 con 0x7f84d40cf2f0 2024-04-25T11:30:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.479+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d415be80 con 0x7f84d40cf2f0 2024-04-25T11:30:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.480+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d4050740 con 0x7f84d40cf2f0 2024-04-25T11:30:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.481+0000 7f84d0c00640 1 -- 172.21.15.19:0/1177990800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f84c0018070 con 0x7f84d40cf2f0 2024-04-25T11:30:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.482+0000 7f84d0c00640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84b40744b0 0x7f84b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.482+0000 7f84d0c00640 1 -- 172.21.15.19:0/1177990800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f84c0095310 con 0x7f84d40cf2f0 2024-04-25T11:30:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.483+0000 7f84d8e00640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84b40744b0 0x7f84b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.484+0000 7f84d8e00640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84b40744b0 0x7f84b40769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f84c8006fd0 tx=0x7f84c8006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.487+0000 7f84d0c00640 1 -- 172.21.15.19:0/1177990800 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84c005fa90 con 0x7f84d40cf2f0 2024-04-25T11:30:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.731+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84d406c770 con 0x7f84d40cf2f0 2024-04-25T11:30:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.732+0000 7f84d0c00640 1 -- 172.21.15.19:0/1177990800 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5184 (secure 0 0 0) 0x7f84c005f390 con 0x7f84d40cf2f0 2024-04-25T11:30:37.722 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:37.722 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:36.342557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":9,"addr":"172.21.15.19:6835/3038680263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3038680263},{"type":"v1","addr":"172.21.15.19:6835","nonce":3038680263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5721,"qdb_cluster":[5721]},"id":3}]} 2024-04-25T11:30:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84b40744b0 msgr2=0x7f84b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84b40744b0 0x7f84b40769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f84c8006fd0 tx=0x7f84c8006bc0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d40cf2f0 msgr2=0x7f84d41492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d40cf2f0 0x7f84d41492c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c7f0 tx=0x7f84c000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 shutdown_connections 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84b40744b0 0x7f84b40769c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84d41608d0 0x7f84d415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84d40cf2f0 0x7f84d41492c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 --2- 172.21.15.19:0/1177990800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84d40b6e10 0x7f84d415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 >> 172.21.15.19:0/1177990800 conn(0x7f84d40d1720 msgr2=0x7f84d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 shutdown_connections 2024-04-25T11:30:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:37.735+0000 7f84da200640 1 -- 172.21.15.19:0/1177990800 wait complete. 2024-04-25T11:30:37.754 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /a 0 2024-04-25T11:30:38.047 INFO:teuthology.orchestra.run.smithi019.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-04-25T11:30:33.640420+0000","btime":"2024-04-25T11:30:26.979488+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-25T11:30:33.640420+0000","atime":"2024-04-25T11:30:26.979488+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-25T11:30:33.640420+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-25T11:30:33.640420+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-25T11:30:33.640420+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-25T11:30:38.048 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-25T11:30:33.640420+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-04-25T11:30:26.979488+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-25T11:30:26.979488+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-04-25T11:30:33.640420+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-25T11:30:33.640420+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-25T11:30:33.640420+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-25T11:30:33.640420+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-04-25T11:30:38.048 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-04-25T11:30:33.640420+0000', 'btime': '2024-04-25T11:30:26.979488+0000', 'mode': 16877, 'uid': 1000, 'gid': 1316, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-04-25T11:30:33.640420+0000', 'atime': '2024-04-25T11:30:26.979488+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-04-25T11:30:33.640420+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-25T11:30:33.640420+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-25T11:30:33.640420+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-04-25T11:30:38.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:30:38.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.349+0000 7f4c5ce00640 1 Processor -- start 2024-04-25T11:30:38.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.349+0000 7f4c5ce00640 1 -- start start 2024-04-25T11:30:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.351+0000 7f4c5ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c58144500 0x7f4c58070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.351+0000 7f4c5ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c580cdb90 0x7f4c58161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.352+0000 7f4c5ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5806f5f0 con 0x7f4c580cdfa0 2024-04-25T11:30:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.352+0000 7f4c5ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4c58070200 con 0x7f4c580cdb90 2024-04-25T11:30:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.352+0000 7f4c5ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5806f8d0 con 0x7f4c58144500 2024-04-25T11:30:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.352+0000 7f4c56800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4c580cdfa0 0x7f4c580ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44626/0 (socket says 172.21.15.19:44626) 2024-04-25T11:30:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.352+0000 7f4c56800640 1 -- 172.21.15.19:0/1324139210 learned_addr learned my addr 172.21.15.19:0/1324139210 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.352+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c58144500 0x7f4c58070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c55400640 1 --2- 172.21.15.19:0/1324139210 >> v2:172.21.15.169:3300/0 conn(0x7f4c580cdb90 0x7f4c58161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515760329 0 0) 0x7f4c5806f5f0 con 0x7f4c580cdfa0 2024-04-25T11:30:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c3c003520 con 0x7f4c580cdfa0 2024-04-25T11:30:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2030677033 0 0) 0x7f4c3c003520 con 0x7f4c580cdfa0 2024-04-25T11:30:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c58144500 msgr2=0x7f4c58070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c58144500 0x7f4c58070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 >> v2:172.21.15.169:3300/0 conn(0x7f4c580cdb90 msgr2=0x7f4c58161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 --2- 172.21.15.19:0/1324139210 >> v2:172.21.15.169:3300/0 conn(0x7f4c580cdb90 0x7f4c58161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5806fbb0 con 0x7f4c580cdfa0 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4c500019f0 con 0x7f4c580cdfa0 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 >> v1:172.21.15.19:6789/0 conn(0x7f4c580cdfa0 legacy=0x7f4c580ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c004800 0x7f4c3c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c58144500 0x7f4c3c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c58070200 con 0x7f4c3c0039b0 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.353+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5806f8d0 con 0x7f4c3c004800 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.354+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c00b9c0 con 0x7f4c58144500 2024-04-25T11:30:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.354+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.354+0000 7f4c55e00640 1 -- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c58144500 msgr2=0x7f4c3c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.354+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c58144500 0x7f4c3c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.354+0000 7f4c55e00640 1 -- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c004800 msgr2=0x7f4c3c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.354+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c3c004800 0x7f4c3c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.354+0000 7f4c55e00640 1 -- 172.21.15.19:0/1324139210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c003520 con 0x7f4c3c0039b0 2024-04-25T11:30:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.355+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c3c003e20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009ac0 tx=0x7f4c3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b96fd291d327cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.356+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48036070 con 0x7f4c3c0039b0 2024-04-25T11:30:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.356+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c48031400 con 0x7f4c3c0039b0 2024-04-25T11:30:38.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.356+0000 7f4c54a00640 1 -- 172.21.15.19:0/1324139210 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48009690 con 0x7f4c3c0039b0 2024-04-25T11:30:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.356+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1324139210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 msgr2=0x7f4c3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.356+0000 7f4c5ce00640 1 --2- 172.21.15.19:0/1324139210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c3c003e20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009ac0 tx=0x7f4c3c005830 comp rx=0 tx=0).stop 2024-04-25T11:30:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.356+0000 7f4c56800640 1 -- 172.21.15.19:0/1324139210 reap_dead start 2024-04-25T11:30:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.357+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1324139210 shutdown_connections 2024-04-25T11:30:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.357+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1324139210 >> 172.21.15.19:0/1324139210 conn(0x7f4c58073050 msgr2=0x7f4c58073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.357+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1324139210 shutdown_connections 2024-04-25T11:30:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.357+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1324139210 wait complete. 2024-04-25T11:30:38.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.359+0000 7f4c5ce00640 1 Processor -- start 2024-04-25T11:30:38.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.359+0000 7f4c5ce00640 1 -- start start 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c5ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c5815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c5ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c580cad20 0x7f4c58146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c5ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c580cb250 0x7f4c58146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c5ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5806f2e0 con 0x7f4c3c0039b0 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c5ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5806f160 con 0x7f4c580cb250 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c5ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c58070690 con 0x7f4c580cad20 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c55e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c5815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c55e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c5815d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55792/0 (socket says 172.21.15.19:55792) 2024-04-25T11:30:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.361+0000 7f4c55e00640 1 -- 172.21.15.19:0/1144952010 learned_addr learned my addr 172.21.15.19:0/1144952010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:38.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c55e00640 1 -- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c580cad20 msgr2=0x7f4c58146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c55400640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c580cad20 0x7f4c58146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c580cad20 0x7f4c58146910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c55e00640 1 -- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c580cb250 msgr2=0x7f4c58146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c56800640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c580cb250 0x7f4c58146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c580cb250 0x7f4c58146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c55e00640 1 -- 172.21.15.19:0/1144952010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c580cc8b0 con 0x7f4c3c0039b0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.362+0000 7f4c55e00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c5815d8b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4c48034040 tx=0x7f4c48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.363+0000 7f4c4e000640 1 -- 172.21.15.19:0/1144952010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48040020 con 0x7f4c3c0039b0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.363+0000 7f4c4e000640 1 -- 172.21.15.19:0/1144952010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c48003ac0 con 0x7f4c3c0039b0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.363+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c580ccb90 con 0x7f4c3c0039b0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.363+0000 7f4c4e000640 1 -- 172.21.15.19:0/1144952010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48036040 con 0x7f4c3c0039b0 2024-04-25T11:30:38.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.363+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c580cd150 con 0x7f4c3c0039b0 2024-04-25T11:30:38.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.364+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c58050740 con 0x7f4c3c0039b0 2024-04-25T11:30:38.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.365+0000 7f4c4e000640 1 -- 172.21.15.19:0/1144952010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4c48043020 con 0x7f4c3c0039b0 2024-04-25T11:30:38.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.366+0000 7f4c4e000640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.366+0000 7f4c4e000640 1 -- 172.21.15.19:0/1144952010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f4c480b55e0 con 0x7f4c3c0039b0 2024-04-25T11:30:38.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.367+0000 7f4c55400640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.368+0000 7f4c55400640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c580cc390 tx=0x7f4c44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:38.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.371+0000 7f4c4e000640 1 -- 172.21.15.19:0/1144952010 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c4804a070 con 0x7f4c3c0039b0 2024-04-25T11:30:38.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.618+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4c58147cb0 con 0x7f4c3c0039b0 2024-04-25T11:30:38.609 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:30:38.619+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:30:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.623+0000 7f4c4e000640 1 -- 172.21.15.19:0/1144952010 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f4c480044e0 con 0x7f4c3c0039b0 2024-04-25T11:30:38.612 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:30:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.625+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 msgr2=0x7f4c380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.625+0000 7f4c5ce00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c580cc390 tx=0x7f4c44008040 comp rx=0 tx=0).stop 2024-04-25T11:30:38.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.625+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 msgr2=0x7f4c5815d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.625+0000 7f4c5ce00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c5815d8b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4c48034040 tx=0x7f4c48004730 comp rx=0 tx=0).stop 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 shutdown_connections 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c580cb250 0x7f4c58146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c580cad20 0x7f4c58146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 --2- 172.21.15.19:0/1144952010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c3c0039b0 0x7f4c5815d8b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 >> 172.21.15.19:0/1144952010 conn(0x7f4c58073050 msgr2=0x7f4c580ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:38.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 shutdown_connections 2024-04-25T11:30:38.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.626+0000 7f4c5ce00640 1 -- 172.21.15.19:0/1144952010 wait complete. 2024-04-25T11:30:38.631 INFO:tasks.ceph.mds.a.smithi019.stderr: -4482> 2024-04-25T11:30:17.213+0000 7fadcd800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:30:38.646 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-25T11:30:38.646 INFO:tasks.ceph.mds.a.smithi019.stderr: -492> 2024-04-25T11:30:33.641+0000 7fadd0a00640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5263:44 nref=7 cr=0x56422cb08a80)) (dversion lock w=1 last_client=5263) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x56422ca2c280] 2024-04-25T11:30:38.778 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:30:38.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.923+0000 7f64bc200640 1 Processor -- start 2024-04-25T11:30:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.923+0000 7f64bc200640 1 -- start start 2024-04-25T11:30:38.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.925+0000 7f64bc200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f64b40b6510 0x7f64b40ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.925+0000 7f64bc200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64b406d5d0 0x7f64b406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.925+0000 7f64bc200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64b406df20 con 0x7f64b4160d30 2024-04-25T11:30:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.925+0000 7f64bc200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f64b4074350 con 0x7f64b40b6510 2024-04-25T11:30:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.925+0000 7f64bc200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f64b415d690 con 0x7f64b406d5d0 2024-04-25T11:30:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b8e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f64b4160d30 0x7f64b4073c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44642/0 (socket says 172.21.15.19:44642) 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b8e00640 1 -- 172.21.15.19:0/2271725152 learned_addr learned my addr 172.21.15.19:0/2271725152 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b9800640 1 --2- 172.21.15.19:0/2271725152 >> v2:172.21.15.19:3300/0 conn(0x7f64b40b6510 0x7f64b40ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64ba200640 1 --2- 172.21.15.19:0/2271725152 >> v2:172.21.15.169:3300/0 conn(0x7f64b406d5d0 0x7f64b406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b3e00640 1 -- 172.21.15.19:0/2271725152 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2970578850 0 0) 0x7f64a0001110 con 0x7f64b4160d30 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b3e00640 1 -- 172.21.15.19:0/2271725152 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f649c003520 con 0x7f64b4160d30 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b9800640 1 -- 172.21.15.19:0/2271725152 >> v2:172.21.15.169:3300/0 conn(0x7f64b406d5d0 msgr2=0x7f64b406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b9800640 1 --2- 172.21.15.19:0/2271725152 >> v2:172.21.15.169:3300/0 conn(0x7f64b406d5d0 0x7f64b406d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b9800640 1 -- 172.21.15.19:0/2271725152 >> v1:172.21.15.19:6789/0 conn(0x7f64b4160d30 legacy=0x7f64b4073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.926+0000 7f64b9800640 1 -- 172.21.15.19:0/2271725152 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b415cd80 con 0x7f64b40b6510 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.927+0000 7f64b9800640 1 --2- 172.21.15.19:0/2271725152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b40ce260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097a0 tx=0x7f64a802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815866f6203ff554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.928+0000 7f64b3e00640 1 -- 172.21.15.19:0/2271725152 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8037070 con 0x7f64b40b6510 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.928+0000 7f64b3e00640 1 -- 172.21.15.19:0/2271725152 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a802adc0 con 0x7f64b40b6510 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.928+0000 7f64b3e00640 1 -- 172.21.15.19:0/2271725152 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8032640 con 0x7f64b40b6510 2024-04-25T11:30:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 -- 172.21.15.19:0/2271725152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 msgr2=0x7f64b40ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 --2- 172.21.15.19:0/2271725152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b40ce260 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f64a80097a0 tx=0x7f64a802a0b0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 -- 172.21.15.19:0/2271725152 shutdown_connections 2024-04-25T11:30:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 --2- 172.21.15.19:0/2271725152 >> v2:172.21.15.169:3300/0 conn(0x7f64b406d5d0 0x7f64b406d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 --2- 172.21.15.19:0/2271725152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b40ce260 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 -- 172.21.15.19:0/2271725152 >> 172.21.15.19:0/2271725152 conn(0x7f64b4073120 msgr2=0x7f64b4073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 -- 172.21.15.19:0/2271725152 shutdown_connections 2024-04-25T11:30:38.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.929+0000 7f64bc200640 1 -- 172.21.15.19:0/2271725152 wait complete. 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.931+0000 7f64bc200640 1 Processor -- start 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.931+0000 7f64bc200640 1 -- start start 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64bc200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b406d5d0 0x7f64b4154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64bc200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b41f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64bc200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b4160d30 0x7f64b41f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64bc200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f64b415d320 con 0x7f64b40b6510 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64bc200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64b415d040 con 0x7f64b4160d30 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64bc200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f64b4161140 con 0x7f64b406d5d0 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b41f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b41f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55828/0 (socket says 172.21.15.19:55828) 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 -- 172.21.15.19:0/2943509143 learned_addr learned my addr 172.21.15.19:0/2943509143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 -- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b406d5d0 msgr2=0x7f64b4154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b406d5d0 0x7f64b4154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 -- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b4160d30 msgr2=0x7f64b41f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b4160d30 0x7f64b41f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.933+0000 7f64b8e00640 1 -- 172.21.15.19:0/2943509143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b4155470 con 0x7f64b40b6510 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.934+0000 7f64b8e00640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b41f7ff0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c3f0 tx=0x7f64a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.934+0000 7f64b1600640 1 -- 172.21.15.19:0/2943509143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0017070 con 0x7f64b40b6510 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.934+0000 7f64b1600640 1 -- 172.21.15.19:0/2943509143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a0005ce0 con 0x7f64b40b6510 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.934+0000 7f64b1600640 1 -- 172.21.15.19:0/2943509143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a00072f0 con 0x7f64b40b6510 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.934+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b4155750 con 0x7f64b40b6510 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.934+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b4155d10 con 0x7f64b40b6510 2024-04-25T11:30:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.935+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b40761c0 con 0x7f64b40b6510 2024-04-25T11:30:38.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.936+0000 7f64b1600640 1 -- 172.21.15.19:0/2943509143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f64a001a070 con 0x7f64b40b6510 2024-04-25T11:30:38.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.937+0000 7f64b1600640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64940744b0 0x7f64940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:38.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.937+0000 7f64b1600640 1 -- 172.21.15.19:0/2943509143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f64a0094690 con 0x7f64b40b6510 2024-04-25T11:30:38.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.938+0000 7f64b9800640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64940744b0 0x7f64940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:38.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.939+0000 7f64b9800640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64940744b0 0x7f64940769c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f64a8034000 tx=0x7f64a802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:38.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:38.942+0000 7f64b1600640 1 -- 172.21.15.19:0/2943509143 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64b406df20 con 0x7f64b40b6510 2024-04-25T11:30:39.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.139+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f64b4050740 con 0x7f64b40b6510 2024-04-25T11:30:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.147+0000 7f64b1600640 1 -- 172.21.15.19:0/2943509143 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f64a005edb0 con 0x7f64b40b6510 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64940744b0 msgr2=0x7f64940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64940744b0 0x7f64940769c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f64a8034000 tx=0x7f64a802c000 comp rx=0 tx=0).stop 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 msgr2=0x7f64b41f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b41f7ff0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c3f0 tx=0x7f64a000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 shutdown_connections 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64940744b0 0x7f64940769c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64b4160d30 0x7f64b41f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64b40b6510 0x7f64b41f7ff0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 --2- 172.21.15.19:0/2943509143 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64b406d5d0 0x7f64b4154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 >> 172.21.15.19:0/2943509143 conn(0x7f64b4073120 msgr2=0x7f64b40b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 shutdown_connections 2024-04-25T11:30:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.151+0000 7f64bc200640 1 -- 172.21.15.19:0/2943509143 wait complete. 2024-04-25T11:30:39.166 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-04-25T11:30:39.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.452+0000 7f475ea00640 1 Processor -- start 2024-04-25T11:30:39.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.452+0000 7f475ea00640 1 -- start start 2024-04-25T11:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.454+0000 7f475ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f475806d5d0 0x7f475806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f475ea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4758070c30 con 0x7f4758158850 2024-04-25T11:30:39.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f475ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475806df20 con 0x7f4758070820 2024-04-25T11:30:39.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f475ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4758158c60 con 0x7f475806d5d0 2024-04-25T11:30:39.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f4757e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4758158850 0x7f475815e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44648/0 (socket says 172.21.15.19:44648) 2024-04-25T11:30:39.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f4757e00640 1 -- 172.21.15.19:0/1502314422 learned_addr learned my addr 172.21.15.19:0/1502314422 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:39.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f475ca00640 1 --2- 172.21.15.19:0/1502314422 >> v2:172.21.15.169:3300/0 conn(0x7f475806d5d0 0x7f475806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1690590988 0 0) 0x7f4758070c30 con 0x7f4758158850 2024-04-25T11:30:39.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.455+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f473c003520 con 0x7f4758158850 2024-04-25T11:30:39.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1129919159 0 0) 0x7f473c003520 con 0x7f4758158850 2024-04-25T11:30:39.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 >> v2:172.21.15.169:3300/0 conn(0x7f475806d5d0 msgr2=0x7f475806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:39.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 --2- 172.21.15.19:0/1502314422 >> v2:172.21.15.169:3300/0 conn(0x7f475806d5d0 0x7f475806d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 >> v1:172.21.15.169:6789/0 conn(0x7f4758070820 legacy=0x7f475815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475815ef90 con 0x7f4758158850 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4748001690 con 0x7f4758158850 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 >> v1:172.21.15.19:6789/0 conn(0x7f4758158850 legacy=0x7f475815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f473c003a00 0x7f473c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c004800 0x7f473c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473c005270 0x7f473c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4758158c60 con 0x7f473c005270 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f473c00c190 con 0x7f473c004800 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f473c00c390 con 0x7f473c003a00 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.456+0000 7f475ca00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473c005270 0x7f473c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f4757e00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f473c003a00 0x7f473c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f4757400640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c004800 0x7f473c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f475ca00640 1 -- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f473c003a00 msgr2=0x7f473c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f475ca00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f473c003a00 0x7f473c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f475ca00640 1 -- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c004800 msgr2=0x7f473c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f475ca00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c004800 0x7f473c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f475ca00640 1 -- 172.21.15.19:0/1502314422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c003520 con 0x7f473c005270 2024-04-25T11:30:39.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.457+0000 7f4757e00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f473c003a00 0x7f473c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.458+0000 7f475ca00640 1 -- 172.21.15.19:0/1502314422 reap_dead start 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.458+0000 7f475ca00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473c005270 0x7f473c00b9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4750009f90 tx=0x7f473c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cbd4e3eee00638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.458+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750009c10 con 0x7f473c005270 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4750031440 con 0x7f473c005270 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f4756a00640 1 -- 172.21.15.19:0/1502314422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750031810 con 0x7f473c005270 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f475ea00640 1 -- 172.21.15.19:0/1502314422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473c005270 msgr2=0x7f473c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f475ea00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473c005270 0x7f473c00b9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4750009f90 tx=0x7f473c005830 comp rx=0 tx=0).stop 2024-04-25T11:30:39.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f475ea00640 1 -- 172.21.15.19:0/1502314422 shutdown_connections 2024-04-25T11:30:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f475ea00640 1 --2- 172.21.15.19:0/1502314422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f473c005270 0x7f473c00b9c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f475ea00640 1 -- 172.21.15.19:0/1502314422 >> 172.21.15.19:0/1502314422 conn(0x7f47580730b0 msgr2=0x7f47580734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f475ea00640 1 -- 172.21.15.19:0/1502314422 shutdown_connections 2024-04-25T11:30:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.459+0000 7f475ea00640 1 -- 172.21.15.19:0/1502314422 wait complete. 2024-04-25T11:30:39.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.462+0000 7f475ea00640 1 Processor -- start 2024-04-25T11:30:39.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.462+0000 7f475ea00640 1 -- start start 2024-04-25T11:30:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c005270 0x7f4758135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47580cd880 0x7f475815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4758146880 0x7f47581592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f475806f040 con 0x7f4758146880 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f475815f550 con 0x7f473c005270 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f475806d9e0 con 0x7f47580cd880 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4758146880 0x7f47581592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4758146880 0x7f47581592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55838/0 (socket says 172.21.15.19:55838) 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f475ca00640 1 -- 172.21.15.19:0/1578611091 learned_addr learned my addr 172.21.15.19:0/1578611091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f4757e00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c005270 0x7f4758135390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.464+0000 7f4757400640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47580cd880 0x7f475815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f475ca00640 1 -- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47580cd880 msgr2=0x7f475815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f475ca00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47580cd880 0x7f475815a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f475ca00640 1 -- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c005270 msgr2=0x7f4758135390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f475ca00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c005270 0x7f4758135390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f475ca00640 1 -- 172.21.15.19:0/1578611091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47581358d0 con 0x7f4758146880 2024-04-25T11:30:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f475ca00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4758146880 0x7f47581592e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f475002aee0 tx=0x7f4750009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f474fe00640 1 -- 172.21.15.19:0/1578611091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750009c10 con 0x7f4758146880 2024-04-25T11:30:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f474fe00640 1 -- 172.21.15.19:0/1578611091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4750043070 con 0x7f4758146880 2024-04-25T11:30:39.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47581457e0 con 0x7f4758146880 2024-04-25T11:30:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.465+0000 7f474fe00640 1 -- 172.21.15.19:0/1578611091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475003c8b0 con 0x7f4758146880 2024-04-25T11:30:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.466+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4758145da0 con 0x7f4758146880 2024-04-25T11:30:39.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.467+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4758050740 con 0x7f4758146880 2024-04-25T11:30:39.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.468+0000 7f474fe00640 1 -- 172.21.15.19:0/1578611091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4750005d10 con 0x7f4758146880 2024-04-25T11:30:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.469+0000 7f474fe00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47380744b0 0x7f47380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.469+0000 7f474fe00640 1 -- 172.21.15.19:0/1578611091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f47500b5e70 con 0x7f4758146880 2024-04-25T11:30:39.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.470+0000 7f4757e00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47380744b0 0x7f47380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.471+0000 7f4757e00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47380744b0 0x7f47380769c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4748000c00 tx=0x7f4748001570 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:39.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.474+0000 7f474fe00640 1 -- 172.21.15.19:0/1578611091 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4750080590 con 0x7f4758146880 2024-04-25T11:30:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.682+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7f475806e100 con 0x7f4758146880 2024-04-25T11:30:39.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.683+0000 7f474fe00640 1 -- 172.21.15.19:0/1578611091 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v15)=0 v15) ==== 135+0+0 (secure 0 0 0) 0x7f475007fe90 con 0x7f4758146880 2024-04-25T11:30:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47380744b0 msgr2=0x7f47380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47380744b0 0x7f47380769c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4748000c00 tx=0x7f4748001570 comp rx=0 tx=0).stop 2024-04-25T11:30:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4758146880 msgr2=0x7f47581592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4758146880 0x7f47581592e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f475002aee0 tx=0x7f4750009750 comp rx=0 tx=0).stop 2024-04-25T11:30:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 shutdown_connections 2024-04-25T11:30:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47380744b0 0x7f47380769c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4758146880 0x7f47581592e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47580cd880 0x7f475815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 --2- 172.21.15.19:0/1578611091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f473c005270 0x7f4758135390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.685+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 >> 172.21.15.19:0/1578611091 conn(0x7f47580730b0 msgr2=0x7f475815e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.686+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 shutdown_connections 2024-04-25T11:30:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.686+0000 7f475ea00640 1 -- 172.21.15.19:0/1578611091 wait complete. 2024-04-25T11:30:39.701 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:30:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.989+0000 7f0a6ea00640 1 Processor -- start 2024-04-25T11:30:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.990+0000 7f0a6ea00640 1 -- start start 2024-04-25T11:30:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0a680ce520 0x7f0a6815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a6806dce0 0x7f0a6815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6ea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a680cda60 con 0x7f0a680ce930 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6815dfe0 con 0x7f0a6806dce0 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6815e2c0 con 0x7f0a680ce520 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6cc00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0a680ce930 0x7f0a6806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44658/0 (socket says 172.21.15.19:44658) 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1955118703 learned_addr learned my addr 172.21.15.19:0/1955118703 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6e000640 1 --2- 172.21.15.19:0/1955118703 >> v2:172.21.15.19:3300/0 conn(0x7f0a6806dce0 0x7f0a6815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a6d600640 1 --2- 172.21.15.19:0/1955118703 >> v2:172.21.15.169:3300/0 conn(0x7f0a680ce520 0x7f0a6815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.992+0000 7f0a67e00640 1 -- 172.21.15.19:0/1955118703 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2132136010 0 0) 0x7f0a680cda60 con 0x7f0a680ce930 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.993+0000 7f0a67e00640 1 -- 172.21.15.19:0/1955118703 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a50005730 con 0x7f0a680ce930 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.993+0000 7f0a6e000640 1 -- 172.21.15.19:0/1955118703 >> v2:172.21.15.169:3300/0 conn(0x7f0a680ce520 msgr2=0x7f0a6815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.993+0000 7f0a6e000640 1 --2- 172.21.15.19:0/1955118703 >> v2:172.21.15.169:3300/0 conn(0x7f0a680ce520 0x7f0a6815d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.993+0000 7f0a6e000640 1 -- 172.21.15.19:0/1955118703 >> v1:172.21.15.19:6789/0 conn(0x7f0a680ce930 legacy=0x7f0a6806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.993+0000 7f0a6e000640 1 -- 172.21.15.19:0/1955118703 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a680cdd20 con 0x7f0a6806dce0 2024-04-25T11:30:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.993+0000 7f0a6d600640 1 --2- 172.21.15.19:0/1955118703 >> v2:172.21.15.169:3300/0 conn(0x7f0a680ce520 0x7f0a6815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.994+0000 7f0a6e000640 1 --2- 172.21.15.19:0/1955118703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6806dce0 0x7f0a6815d9f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0a680713e0 tx=0x7f0a6002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4b14ea0f6a96f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.994+0000 7f0a67e00640 1 -- 172.21.15.19:0/1955118703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60007740 con 0x7f0a6806dce0 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.994+0000 7f0a67e00640 1 -- 172.21.15.19:0/1955118703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a600078e0 con 0x7f0a6806dce0 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.994+0000 7f0a67e00640 1 -- 172.21.15.19:0/1955118703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60007cb0 con 0x7f0a6806dce0 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 -- 172.21.15.19:0/1955118703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6806dce0 msgr2=0x7f0a6815d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/1955118703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6806dce0 0x7f0a6815d9f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0a680713e0 tx=0x7f0a6002a5f0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 -- 172.21.15.19:0/1955118703 shutdown_connections 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/1955118703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6806dce0 0x7f0a6815d9f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/1955118703 >> v2:172.21.15.169:3300/0 conn(0x7f0a680ce520 0x7f0a6815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 -- 172.21.15.19:0/1955118703 >> 172.21.15.19:0/1955118703 conn(0x7f0a68072f40 msgr2=0x7f0a68073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 -- 172.21.15.19:0/1955118703 shutdown_connections 2024-04-25T11:30:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.995+0000 7f0a6ea00640 1 -- 172.21.15.19:0/1955118703 wait complete. 2024-04-25T11:30:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.997+0000 7f0a6ea00640 1 Processor -- start 2024-04-25T11:30:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.998+0000 7f0a6ea00640 1 -- start start 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.999+0000 7f0a6ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a680ce520 0x7f0a68143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.999+0000 7f0a6ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a680ce930 0x7f0a6814e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.999+0000 7f0a6ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6814ef20 0x7f0a6815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.999+0000 7f0a6ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a680b73d0 con 0x7f0a6814ef20 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.999+0000 7f0a6ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68160d30 con 0x7f0a680ce520 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:39.999+0000 7f0a6ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0a680b7550 con 0x7f0a680ce930 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6814ef20 0x7f0a6815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6814ef20 0x7f0a6815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55864/0 (socket says 172.21.15.19:55864) 2024-04-25T11:30:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 -- 172.21.15.19:0/2176624883 learned_addr learned my addr 172.21.15.19:0/2176624883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6d600640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a680ce520 0x7f0a68143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6cc00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a680ce930 0x7f0a6814e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 -- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a680ce930 msgr2=0x7f0a6814e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a680ce930 0x7f0a6814e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 -- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a680ce520 msgr2=0x7f0a68143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a680ce520 0x7f0a68143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 -- 172.21.15.19:0/2176624883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6814ff70 con 0x7f0a6814ef20 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6cc00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a680ce930 0x7f0a6814e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.000+0000 7f0a6e000640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6814ef20 0x7f0a6815b850 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e50 tx=0x7f0a60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.001+0000 7f0a65600640 1 -- 172.21.15.19:0/2176624883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6003f070 con 0x7f0a6814ef20 2024-04-25T11:30:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.001+0000 7f0a65600640 1 -- 172.21.15.19:0/2176624883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a60031bf0 con 0x7f0a6814ef20 2024-04-25T11:30:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.001+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6814f6d0 con 0x7f0a6814ef20 2024-04-25T11:30:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.001+0000 7f0a65600640 1 -- 172.21.15.19:0/2176624883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60009450 con 0x7f0a6814ef20 2024-04-25T11:30:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.001+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6814fc90 con 0x7f0a6814ef20 2024-04-25T11:30:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.002+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a68076c60 con 0x7f0a6814ef20 2024-04-25T11:30:39.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.004+0000 7f0a65600640 1 -- 172.21.15.19:0/2176624883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0a6003b410 con 0x7f0a6814ef20 2024-04-25T11:30:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.005+0000 7f0a65600640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0a480744b0 0x7f0a480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.005+0000 7f0a65600640 1 -- 172.21.15.19:0/2176624883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f0a600b6d30 con 0x7f0a6814ef20 2024-04-25T11:30:39.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.005+0000 7f0a6d600640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0a480744b0 0x7f0a480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:39.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.007+0000 7f0a6d600640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0a480744b0 0x7f0a480769c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c006fd0 tx=0x7f0a5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:39.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.009+0000 7f0a65600640 1 -- 172.21.15.19:0/2176624883 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a60080d70 con 0x7f0a6814ef20 2024-04-25T11:30:40.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.246+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f0a68156230 con 0x7f0a6814ef20 2024-04-25T11:30:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.993+0000 7f0a65600640 1 -- 172.21.15.19:0/2176624883 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v38) ==== 162+0+0 (secure 0 0 0) 0x7f0a60085070 con 0x7f0a6814ef20 2024-04-25T11:30:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.996+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0a480744b0 msgr2=0x7f0a480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.996+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0a480744b0 0x7f0a480769c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c006fd0 tx=0x7f0a5c008040 comp rx=0 tx=0).stop 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.996+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6814ef20 msgr2=0x7f0a6815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.996+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6814ef20 0x7f0a6815b850 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e50 tx=0x7f0a60004a40 comp rx=0 tx=0).stop 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 shutdown_connections 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0a480744b0 0x7f0a480769c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a6814ef20 0x7f0a6815b850 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a680ce930 0x7f0a6814e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 --2- 172.21.15.19:0/2176624883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a680ce520 0x7f0a68143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 >> 172.21.15.19:0/2176624883 conn(0x7f0a68072f40 msgr2=0x7f0a681567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 shutdown_connections 2024-04-25T11:30:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:40.997+0000 7f0a6ea00640 1 -- 172.21.15.19:0/2176624883 wait complete. 2024-04-25T11:30:41.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:41.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.296+0000 7f09e3e00640 1 Processor -- start 2024-04-25T11:30:41.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.296+0000 7f09e3e00640 1 -- start start 2024-04-25T11:30:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09e3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09e40b4ea0 0x7f09e40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09e3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f09e4144370 0x7f09e415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09e3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e40d1230 con 0x7f09e406c5c0 2024-04-25T11:30:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09e3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f09e40d06b0 con 0x7f09e40b4ea0 2024-04-25T11:30:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09e3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f09e40d0990 con 0x7f09e4144370 2024-04-25T11:30:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09dbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09e40b4ea0 0x7f09e40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09dbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09e40b4ea0 0x7f09e40cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55878/0 (socket says 172.21.15.19:55878) 2024-04-25T11:30:41.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.299+0000 7f09dbe00640 1 -- 172.21.15.19:0/3563671843 learned_addr learned my addr 172.21.15.19:0/3563671843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:41.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.300+0000 7f09e3400640 1 --2- 172.21.15.19:0/3563671843 >> v2:172.21.15.169:3300/0 conn(0x7f09e4144370 0x7f09e415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.300+0000 7f09dbe00640 1 -- 172.21.15.19:0/3563671843 >> v2:172.21.15.169:3300/0 conn(0x7f09e4144370 msgr2=0x7f09e415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.300+0000 7f09dbe00640 1 --2- 172.21.15.19:0/3563671843 >> v2:172.21.15.169:3300/0 conn(0x7f09e4144370 0x7f09e415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.300+0000 7f09dbe00640 1 -- 172.21.15.19:0/3563671843 >> v1:172.21.15.169:6789/0 conn(0x7f09e406c5c0 legacy=0x7f09e40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.300+0000 7f09dbe00640 1 -- 172.21.15.19:0/3563671843 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e40d0c70 con 0x7f09e40b4ea0 2024-04-25T11:30:41.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.301+0000 7f09dbe00640 1 --2- 172.21.15.19:0/3563671843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e40b4ea0 0x7f09e40cfa70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f09e40d26c0 tx=0x7f09cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d196502280942ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.301+0000 7f09e2000640 1 -- 172.21.15.19:0/3563671843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc037070 con 0x7f09e40b4ea0 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.301+0000 7f09e2000640 1 -- 172.21.15.19:0/3563671843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc032440 con 0x7f09e40b4ea0 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.301+0000 7f09e2000640 1 -- 172.21.15.19:0/3563671843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc032810 con 0x7f09e40b4ea0 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 -- 172.21.15.19:0/3563671843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e40b4ea0 msgr2=0x7f09e40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3563671843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e40b4ea0 0x7f09e40cfa70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f09e40d26c0 tx=0x7f09cc02a230 comp rx=0 tx=0).stop 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 -- 172.21.15.19:0/3563671843 shutdown_connections 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3563671843 >> v2:172.21.15.169:3300/0 conn(0x7f09e4144370 0x7f09e415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3563671843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e40b4ea0 0x7f09e40cfa70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 -- 172.21.15.19:0/3563671843 >> 172.21.15.19:0/3563671843 conn(0x7f09e40d1720 msgr2=0x7f09e40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 -- 172.21.15.19:0/3563671843 shutdown_connections 2024-04-25T11:30:41.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.302+0000 7f09e3e00640 1 -- 172.21.15.19:0/3563671843 wait complete. 2024-04-25T11:30:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.305+0000 7f09e3e00640 1 Processor -- start 2024-04-25T11:30:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.305+0000 7f09e3e00640 1 -- start start 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.306+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e406c5c0 0x7f09e4162520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.306+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e4144370 0x7f09e40cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09e40cba60 0x7f09e4145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09e40d0450 con 0x7f09e406c5c0 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f09e40d02d0 con 0x7f09e4144370 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f09e4147760 con 0x7f09e40cba60 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e406c5c0 0x7f09e4162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e406c5c0 0x7f09e4162520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55880/0 (socket says 172.21.15.19:55880) 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e2a00640 1 -- 172.21.15.19:0/3947387647 learned_addr learned my addr 172.21.15.19:0/3947387647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e3400640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09e40cba60 0x7f09e4145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09dbe00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e4144370 0x7f09e40cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e2a00640 1 -- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09e40cba60 msgr2=0x7f09e4145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.307+0000 7f09e2a00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09e40cba60 0x7f09e4145d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09e2a00640 1 -- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e4144370 msgr2=0x7f09e40cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09e2a00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e4144370 0x7f09e40cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09e2a00640 1 -- 172.21.15.19:0/3947387647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e40cc210 con 0x7f09e406c5c0 2024-04-25T11:30:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09e2a00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e406c5c0 0x7f09e4162520 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b3f0 tx=0x7f09d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09daa00640 1 -- 172.21.15.19:0/3947387647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4015020 con 0x7f09e406c5c0 2024-04-25T11:30:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09daa00640 1 -- 172.21.15.19:0/3947387647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d4011830 con 0x7f09e406c5c0 2024-04-25T11:30:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09daa00640 1 -- 172.21.15.19:0/3947387647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4011c60 con 0x7f09e406c5c0 2024-04-25T11:30:41.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e40cc4f0 con 0x7f09e406c5c0 2024-04-25T11:30:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.308+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e4130580 con 0x7f09e406c5c0 2024-04-25T11:30:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.310+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e415dfd0 con 0x7f09e406c5c0 2024-04-25T11:30:41.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.311+0000 7f09daa00640 1 -- 172.21.15.19:0/3947387647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09d4018020 con 0x7f09e406c5c0 2024-04-25T11:30:41.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.312+0000 7f09daa00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c00744b0 0x7f09c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.312+0000 7f09daa00640 1 -- 172.21.15.19:0/3947387647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f09d4094e70 con 0x7f09e406c5c0 2024-04-25T11:30:41.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.313+0000 7f09dbe00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c00744b0 0x7f09c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.314+0000 7f09dbe00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c00744b0 0x7f09c00769c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09cc034000 tx=0x7f09cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.317+0000 7f09daa00640 1 -- 172.21.15.19:0/3947387647 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09e415dfd0 con 0x7f09e406c5c0 2024-04-25T11:30:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.549+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09e4050740 con 0x7f09e406c5c0 2024-04-25T11:30:41.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.551+0000 7f09daa00640 1 -- 172.21.15.19:0/3947387647 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5178 (secure 0 0 0) 0x7f09d401f030 con 0x7f09e406c5c0 2024-04-25T11:30:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:41.541 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":39,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:40.999929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T11:30:41.541 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 39 2024-04-25T11:30:41.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.553+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c00744b0 msgr2=0x7f09c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.553+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c00744b0 0x7f09c00769c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f09cc034000 tx=0x7f09cc02e000 comp rx=0 tx=0).stop 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.553+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e406c5c0 msgr2=0x7f09e4162520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.553+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e406c5c0 0x7f09e4162520 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b3f0 tx=0x7f09d400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 shutdown_connections 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09c00744b0 0x7f09c00769c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09e4144370 0x7f09e40cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09e40cba60 0x7f09e4145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 --2- 172.21.15.19:0/3947387647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09e406c5c0 0x7f09e4162520 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 >> 172.21.15.19:0/3947387647 conn(0x7f09e40d1720 msgr2=0x7f09e40b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 shutdown_connections 2024-04-25T11:30:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.554+0000 7f09e3e00640 1 -- 172.21.15.19:0/3947387647 wait complete. 2024-04-25T11:30:41.570 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-04-25T11:30:41.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.856+0000 7ff311200640 1 Processor -- start 2024-04-25T11:30:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.856+0000 7ff311200640 1 -- start start 2024-04-25T11:30:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.858+0000 7ff311200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff30c070970 0x7ff30c167dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.858+0000 7ff311200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30c15e340 con 0x7ff30c13ff60 2024-04-25T11:30:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.858+0000 7ff311200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30c0d02a0 con 0x7ff30c0d1340 2024-04-25T11:30:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.858+0000 7ff311200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff30c0d0580 con 0x7ff30c070970 2024-04-25T11:30:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.859+0000 7ff30b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff30c13ff60 0x7ff30c15dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44666/0 (socket says 172.21.15.19:44666) 2024-04-25T11:30:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.859+0000 7ff30b400640 1 -- 172.21.15.19:0/1482413431 learned_addr learned my addr 172.21.15.19:0/1482413431 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.859+0000 7ff30be00640 1 --2- 172.21.15.19:0/1482413431 >> v2:172.21.15.169:3300/0 conn(0x7ff30c070970 0x7ff30c167dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.859+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761606186 0 0) 0x7ff30c15e340 con 0x7ff30c13ff60 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.859+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2f0003520 con 0x7ff30c13ff60 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3268786231 0 0) 0x7ff2f0003520 con 0x7ff30c13ff60 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 >> v2:172.21.15.169:3300/0 conn(0x7ff30c070970 msgr2=0x7ff30c167dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 --2- 172.21.15.19:0/1482413431 >> v2:172.21.15.169:3300/0 conn(0x7ff30c070970 0x7ff30c167dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 >> v1:172.21.15.169:6789/0 conn(0x7ff30c0d1340 legacy=0x7ff30c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c0d0860 con 0x7ff30c13ff60 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ff2fc001670 con 0x7ff30c13ff60 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 >> v1:172.21.15.19:6789/0 conn(0x7ff30c13ff60 legacy=0x7ff30c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2f00039b0 0x7ff2f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2f0004800 0x7ff2f0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2f0005250 0x7ff2f000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0d0580 con 0x7ff2f00039b0 2024-04-25T11:30:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f000e3f0 con 0x7ff2f0004800 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f000e5f0 con 0x7ff2f0005250 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff30b400640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2f00039b0 0x7ff2f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.860+0000 7ff303e00640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2f0004800 0x7ff2f0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.861+0000 7ff30be00640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2f0005250 0x7ff2f000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.861+0000 7ff30b400640 1 -- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2f0005250 msgr2=0x7ff2f000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.861+0000 7ff30b400640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2f0005250 0x7ff2f000dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.861+0000 7ff30b400640 1 -- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2f0004800 msgr2=0x7ff2f0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.861+0000 7ff30b400640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2f0004800 0x7ff2f0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.861+0000 7ff30be00640 1 -- 172.21.15.19:0/1482413431 reap_dead start 2024-04-25T11:30:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.861+0000 7ff30b400640 1 -- 172.21.15.19:0/1482413431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f0003520 con 0x7ff2f00039b0 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.862+0000 7ff30b400640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2f00039b0 0x7ff2f0003e20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0724c0 tx=0x7ff2f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586e7a0898630143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.862+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc007d30 con 0x7ff2f00039b0 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.862+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc005af0 con 0x7ff2f00039b0 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.862+0000 7ff30aa00640 1 -- 172.21.15.19:0/1482413431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc007070 con 0x7ff2f00039b0 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.863+0000 7ff311200640 1 -- 172.21.15.19:0/1482413431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2f00039b0 msgr2=0x7ff2f0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.863+0000 7ff311200640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2f00039b0 0x7ff2f0003e20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0724c0 tx=0x7ff2f0005830 comp rx=0 tx=0).stop 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.863+0000 7ff311200640 1 -- 172.21.15.19:0/1482413431 shutdown_connections 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.863+0000 7ff311200640 1 --2- 172.21.15.19:0/1482413431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2f00039b0 0x7ff2f0003e20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.863+0000 7ff311200640 1 -- 172.21.15.19:0/1482413431 >> 172.21.15.19:0/1482413431 conn(0x7ff30c0b4260 msgr2=0x7ff30c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.863+0000 7ff311200640 1 -- 172.21.15.19:0/1482413431 shutdown_connections 2024-04-25T11:30:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.863+0000 7ff311200640 1 -- 172.21.15.19:0/1482413431 wait complete. 2024-04-25T11:30:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.865+0000 7ff311200640 1 Processor -- start 2024-04-25T11:30:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.865+0000 7ff311200640 1 -- start start 2024-04-25T11:30:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff311200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff30c1553e0 0x7ff30c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff311200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff30c15fa10 0x7ff30c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff311200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30c0b5e30 0x7ff30c13b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff311200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c15d600 con 0x7ff30c1553e0 2024-04-25T11:30:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff311200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0d0e20 con 0x7ff30c0b5e30 2024-04-25T11:30:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff311200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c15d780 con 0x7ff30c15fa10 2024-04-25T11:30:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff30b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff30c1553e0 0x7ff30c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff30b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff30c1553e0 0x7ff30c15f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55896/0 (socket says 172.21.15.19:55896) 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff30b400640 1 -- 172.21.15.19:0/3962839868 learned_addr learned my addr 172.21.15.19:0/3962839868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff303e00640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff30c15fa10 0x7ff30c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.867+0000 7ff30be00640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30c0b5e30 0x7ff30c13b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff30b400640 1 -- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff30c15fa10 msgr2=0x7ff30c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff30b400640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff30c15fa10 0x7ff30c0b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff30b400640 1 -- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30c0b5e30 msgr2=0x7ff30c13b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff30b400640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30c0b5e30 0x7ff30c13b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff30b400640 1 -- 172.21.15.19:0/3962839868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c13a060 con 0x7ff30c1553e0 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff30be00640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30c0b5e30 0x7ff30c13b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff30b400640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff30c1553e0 0x7ff30c15f4d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc02acf0 tx=0x7ff2fc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff303400640 1 -- 172.21.15.19:0/3962839868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc037040 con 0x7ff30c1553e0 2024-04-25T11:30:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff303400640 1 -- 172.21.15.19:0/3962839868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc041070 con 0x7ff30c1553e0 2024-04-25T11:30:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff303400640 1 -- 172.21.15.19:0/3962839868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc001670 con 0x7ff30c1553e0 2024-04-25T11:30:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c13a300 con 0x7ff30c1553e0 2024-04-25T11:30:41.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.868+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c13a8c0 con 0x7ff30c1553e0 2024-04-25T11:30:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.872+0000 7ff303400640 1 -- 172.21.15.19:0/3962839868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff2fc0049e0 con 0x7ff30c1553e0 2024-04-25T11:30:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.872+0000 7ff31986d740 1 Processor -- start 2024-04-25T11:30:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.873+0000 7ff31986d740 1 -- start start 2024-04-25T11:30:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.873+0000 7ff303400640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2e80744e0 0x7ff2e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.873+0000 7ff303400640 1 -- 172.21.15.19:0/3962839868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7ff2fc0b4ff0 con 0x7ff30c1553e0 2024-04-25T11:30:41.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff30be00640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2e80744e0 0x7ff2e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 0x5568ca06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff31986d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c714d0 0x5568ca068580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c796b0 0x5568ca041840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff31986d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568c9f4d030 con 0x5568c9c714d0 2024-04-25T11:30:41.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5568c9f687f0 con 0x5568c9c796b0 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5568c9f237e0 con 0x5568c9c9a390 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.874+0000 7ff303e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c714d0 0x5568ca068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c714d0 0x5568ca068580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55908/0 (socket says 172.21.15.19:55908) 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 -- 172.21.15.19:0/292635012 learned_addr learned my addr 172.21.15.19:0/292635012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff30be00640 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c796b0 0x5568ca041840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff30b400640 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 0x5568ca06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 -- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 msgr2=0x5568ca06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 0x5568ca06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 -- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c796b0 msgr2=0x5568ca041840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c796b0 0x5568ca041840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 -- 172.21.15.19:0/292635012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c9f2f050 con 0x5568c9c714d0 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff30be00640 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c796b0 0x5568ca041840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:41.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff30be00640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2e80744e0 0x7ff2e80769f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff30c074320 tx=0x7ff304009750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.875+0000 7ff303e00640 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c714d0 0x5568ca068580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4001610 tx=0x7ff2f4001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeade9057c85ec75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff2fbe00640 1 -- 172.21.15.19:0/292635012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4001d20 con 0x5568c9c714d0 2024-04-25T11:30:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff2fbe00640 1 -- 172.21.15.19:0/292635012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4018070 con 0x5568c9c714d0 2024-04-25T11:30:41.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff2fbe00640 1 -- 172.21.15.19:0/292635012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400f750 con 0x5568c9c714d0 2024-04-25T11:30:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff31986d740 1 -- 172.21.15.19:0/292635012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c714d0 msgr2=0x5568ca068580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff31986d740 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c714d0 0x5568ca068580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4001610 tx=0x7ff2f4001ad0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff31986d740 1 -- 172.21.15.19:0/292635012 shutdown_connections 2024-04-25T11:30:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff31986d740 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c796b0 0x5568ca041840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff31986d740 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c714d0 0x5568ca068580 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff31986d740 1 --2- 172.21.15.19:0/292635012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 0x5568ca06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.876+0000 7ff31986d740 1 -- 172.21.15.19:0/292635012 >> 172.21.15.19:0/292635012 conn(0x5568c9c8e4e0 msgr2=0x5568ca040fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.877+0000 7ff31986d740 1 -- 172.21.15.19:0/292635012 shutdown_connections 2024-04-25T11:30:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.877+0000 7ff31986d740 1 -- 172.21.15.19:0/292635012 wait complete. 2024-04-25T11:30:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.878+0000 7ff31986d740 1 -- start start 2024-04-25T11:30:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.878+0000 7ff31986d740 1 Processor -- start 2024-04-25T11:30:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.879+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c714d0 0x5568ca096030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.879+0000 7ff31986d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca06e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.879+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c9a390 0x5568ca06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.879+0000 7ff31986d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568c9fa5200 con 0x5568c9c796b0 2024-04-25T11:30:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.879+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5568c9f9fee0 con 0x5568c9c9a390 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.879+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5568c9face70 con 0x5568c9c714d0 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.879+0000 7ff31986d740 10 client.0 _ll_register_callbacks cb 0x5568c9f96270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca06e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55914/0 (socket says 172.21.15.19:55914) 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 -- 172.21.15.19:0/4186160542 learned_addr learned my addr 172.21.15.19:0/4186160542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff30b400640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c714d0 0x5568ca096030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff30be00640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c9a390 0x5568ca06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 -- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c714d0 msgr2=0x5568ca096030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c714d0 0x5568ca096030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 -- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c9a390 msgr2=0x5568ca06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c9a390 0x5568ca06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff303e00640 1 -- 172.21.15.19:0/4186160542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c9d7c7d0 con 0x5568c9c796b0 2024-04-25T11:30:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff30b400640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c714d0 0x5568ca096030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.880+0000 7ff30be00640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c9a390 0x5568ca06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff303e00640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca06e130 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4014000 tx=0x7ff2f4001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c88c1cf5ac12e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff2f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff2f8c00640 1 -- 172.21.15.19:0/4186160542 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4001d20 con 0x5568c9c796b0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff2f8c00640 1 -- 172.21.15.19:0/4186160542 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7ff2f40067d0 con 0x5568c9c796b0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f4018070 con 0x5568c9c796b0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff2f8c00640 1 -- 172.21.15.19:0/4186160542 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff2f402d050 con 0x5568c9c796b0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff31986d740 10 client.6033 fetch_fsmap learned FSMap version 39 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.881+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5568c9cb23f0 con 0x5568c9c796b0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff2f8c00640 1 -- 172.21.15.19:0/4186160542 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4018070 con 0x5568c9c796b0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff2f8c00640 1 -- 172.21.15.19:0/4186160542 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 3486+0+0 (secure 0 0 0) 0x7ff2f4030020 con 0x5568c9c796b0 2024-04-25T11:30:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff31986d740 10 client.6033 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff31986d740 10 client.6033 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff31986d740 20 client.6033 populate_metadata read hostname 'smithi019' 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff31986d740 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568c9c58f60 0x5568ca067df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff31986d740 4 client.6033 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 --> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568c9fae7e0 con 0x5568c9c58f60 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.882+0000 7ff30b400640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568c9c58f60 0x5568ca067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.883+0000 7ff30b400640 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568c9c58f60 0x5568ca067df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e546b7757658765 server_cookie=8be94300cebb07f in_seq=0 out_seq=0 2024-04-25T11:30:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.883+0000 7ff2f8c00640 10 client.6033 ms_handle_connect on v2:172.21.15.19:6834/3964766888 2024-04-25T11:30:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.883+0000 7ff2f8c00640 1 -- 172.21.15.19:0/4186160542 <== mds.0 v2:172.21.15.19:6834/3964766888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568c9fae7e0 con 0x5568c9c58f60 2024-04-25T11:30:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.883+0000 7ff2f8c00640 10 client.6033 handle_command_reply: tid=0 2024-04-25T11:30:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 client.6033 shutdown 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 msgr2=0x5568ca06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca06e130 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4014000 tx=0x7ff2f4001190 comp rx=0 tx=0).stop 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 shutdown_connections 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568c9c58f60 0x5568ca067df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c9a390 0x5568ca06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca06e130 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff2f8c00640 0 client.6033 ms_handle_reset on v2:172.21.15.19:6834/3964766888 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 --2- 172.21.15.19:0/4186160542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c714d0 0x5568ca096030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 >> 172.21.15.19:0/4186160542 conn(0x5568c9c8e4e0 msgr2=0x5568ca040370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 shutdown_connections 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 1 -- 172.21.15.19:0/4186160542 wait complete. 2024-04-25T11:30:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.884+0000 7ff31986d740 20 client.6033 trim_cache size 0 max 16384 2024-04-25T11:30:41.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.901+0000 7ff31986d740 1 Processor -- start 2024-04-25T11:30:41.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.901+0000 7ff31986d740 1 -- start start 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 0x5568ca095030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c796b0 0x5568ca0365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff31986d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c9a390 0x5568ca036af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff31986d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1feb90 con 0x5568c9c9a390 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1fdbd0 con 0x5568c9c714d0 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c1fcf30 con 0x5568c9c796b0 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff30be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c9a390 0x5568ca036af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff30be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c9a390 0x5568ca036af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55924/0 (socket says 172.21.15.19:55924) 2024-04-25T11:30:41.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.902+0000 7ff30be00640 1 -- 172.21.15.19:0/114251179 learned_addr learned my addr 172.21.15.19:0/114251179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff30b400640 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 0x5568ca095030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff303e00640 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c796b0 0x5568ca0365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff30be00640 1 -- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c796b0 msgr2=0x5568ca0365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff30be00640 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c796b0 0x5568ca0365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff30be00640 1 -- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 msgr2=0x5568ca095030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff30be00640 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 0x5568ca095030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff30be00640 1 -- 172.21.15.19:0/114251179 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c9c720f0 con 0x5568c9c9a390 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.903+0000 7ff30be00640 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c9a390 0x5568ca036af0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff30c157a00 tx=0x7ff304019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3882ee7955e1d8e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff2fb400640 1 -- 172.21.15.19:0/114251179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304029070 con 0x5568c9c9a390 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff2fb400640 1 -- 172.21.15.19:0/114251179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304020040 con 0x5568c9c9a390 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff2fb400640 1 -- 172.21.15.19:0/114251179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304019e60 con 0x5568c9c9a390 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 -- 172.21.15.19:0/114251179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c9a390 msgr2=0x5568ca036af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c9a390 0x5568ca036af0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff30c157a00 tx=0x7ff304019000 comp rx=0 tx=0).stop 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 -- 172.21.15.19:0/114251179 shutdown_connections 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c9a390 0x5568ca036af0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c796b0 0x5568ca0365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 --2- 172.21.15.19:0/114251179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 0x5568ca095030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 -- 172.21.15.19:0/114251179 >> 172.21.15.19:0/114251179 conn(0x5568c9c8e4e0 msgr2=0x5568ca03f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.904+0000 7ff31986d740 1 -- 172.21.15.19:0/114251179 shutdown_connections 2024-04-25T11:30:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.905+0000 7ff31986d740 1 -- 172.21.15.19:0/114251179 wait complete. 2024-04-25T11:30:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.905+0000 7ff31986d740 1 -- start start 2024-04-25T11:30:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.906+0000 7ff31986d740 1 Processor -- start 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 0x5568ca040450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff31986d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca07f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff31986d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 0x5568ca089d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff31986d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568c9fadfd0 con 0x5568c9c796b0 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1fd580 con 0x5568c9c714d0 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff31986d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5568ca041210 con 0x5568c9c9a390 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff31986d740 10 client.0 _ll_register_callbacks cb 0x7ff2f40014b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:30:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff303e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca07f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff303e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca07f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55930/0 (socket says 172.21.15.19:55930) 2024-04-25T11:30:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.907+0000 7ff303e00640 1 -- 172.21.15.19:0/1458658670 learned_addr learned my addr 172.21.15.19:0/1458658670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff303e00640 1 -- 172.21.15.19:0/1458658670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 msgr2=0x5568ca089d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff303e00640 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 0x5568ca089d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff303e00640 1 -- 172.21.15.19:0/1458658670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 msgr2=0x5568ca040450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff303e00640 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 0x5568ca040450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff303e00640 1 -- 172.21.15.19:0/1458658670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c9f68a30 con 0x5568c9c796b0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff303e00640 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca07f750 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40011d0 tx=0x7ff2f4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b1d11d116a0539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff2f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff2f8c00640 1 -- 172.21.15.19:0/1458658670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4007250 con 0x5568c9c796b0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff2f8c00640 1 -- 172.21.15.19:0/1458658670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7ff2f400f650 con 0x5568c9c796b0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.908+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff304020040 con 0x5568c9c796b0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff2f8c00640 1 -- 172.21.15.19:0/1458658670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff2f4040050 con 0x5568c9c796b0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 10 client.6039 fetch_fsmap learned FSMap version 39 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5568ca095620 con 0x5568c9c796b0 2024-04-25T11:30:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff2f8c00640 1 -- 172.21.15.19:0/1458658670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304020040 con 0x5568c9c796b0 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff2f8c00640 1 -- 172.21.15.19:0/1458658670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 39) ==== 3486+0+0 (secure 0 0 0) 0x7ff2f4043020 con 0x5568c9c796b0 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 10 client.6039 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 10 client.6039 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 20 client.6039 populate_metadata read hostname 'smithi019' 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568ca06def0 0x5568ca06e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 4 client.6039 mds_command: new command op to 5991 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 --> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568c9fa8650 con 0x5568ca06def0 2024-04-25T11:30:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.909+0000 7ff30b400640 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568ca06def0 0x5568ca06e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff30b400640 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568ca06def0 0x5568ca06e300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194bc4e395a7afdc server_cookie=5a632e84e17ae4c2 in_seq=0 out_seq=0 2024-04-25T11:30:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff2f8c00640 10 client.6039 ms_handle_connect on v2:172.21.15.19:6834/3964766888 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff2f8c00640 1 -- 172.21.15.19:0/1458658670 <== mds.0 v2:172.21.15.19:6834/3964766888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568c9fa8650 con 0x5568ca06def0 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff2f8c00640 10 client.6039 handle_command_reply: tid=0 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff31986d740 1 client.6039 shutdown 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 msgr2=0x5568ca07f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff31986d740 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca07f750 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40011d0 tx=0x7ff2f4009040 comp rx=0 tx=0).stop 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 shutdown_connections 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.910+0000 7ff31986d740 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:6834/3964766888,v1:172.21.15.19:6835/3964766888] conn(0x5568ca06def0 0x5568ca06e300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff31986d740 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5568c9c9a390 0x5568ca089d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff31986d740 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568c9c796b0 0x5568ca07f750 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff31986d740 1 --2- 172.21.15.19:0/1458658670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5568c9c714d0 0x5568ca040450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 >> 172.21.15.19:0/1458658670 conn(0x5568c9c8e4e0 msgr2=0x5568ca042d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff2f8c00640 0 client.6039 ms_handle_reset on v2:172.21.15.19:6834/3964766888 2024-04-25T11:30:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 shutdown_connections 2024-04-25T11:30:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff31986d740 1 -- 172.21.15.19:0/1458658670 wait complete. 2024-04-25T11:30:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff31986d740 20 client.6039 trim_cache size 0 max 16384 2024-04-25T11:30:41.905 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:30:41.905 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-04-25T11:30:41.905 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T11:30:41.905 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:30:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2e80744e0 msgr2=0x7ff2e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff311200640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2e80744e0 0x7ff2e80769f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff30c074320 tx=0x7ff304009750 comp rx=0 tx=0).stop 2024-04-25T11:30:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff30c1553e0 msgr2=0x7ff30c15f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.911+0000 7ff311200640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff30c1553e0 0x7ff30c15f4d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc02acf0 tx=0x7ff2fc001a40 comp rx=0 tx=0).stop 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 shutdown_connections 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff2e80744e0 0x7ff2e80769f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff30c0b5e30 0x7ff30c13b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff30c15fa10 0x7ff30c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 --2- 172.21.15.19:0/3962839868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff30c1553e0 0x7ff30c15f4d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 >> 172.21.15.19:0/3962839868 conn(0x7ff2f00039b0 msgr2=0x7ff30c1571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 shutdown_connections 2024-04-25T11:30:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:41.912+0000 7ff311200640 1 -- 172.21.15.19:0/3962839868 wait complete. 2024-04-25T11:30:41.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.210+0000 7f8249c00640 1 Processor -- start 2024-04-25T11:30:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.210+0000 7f8249c00640 1 -- start start 2024-04-25T11:30:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.213+0000 7f8249c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8244144370 0x7f82440cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.213+0000 7f8249c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82440d1230 con 0x7f824406c5c0 2024-04-25T11:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.213+0000 7f8249c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82440d06b0 con 0x7f82440b4ea0 2024-04-25T11:30:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.213+0000 7f8249c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f82440d0990 con 0x7f8244144370 2024-04-25T11:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.213+0000 7f8249200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8244144370 0x7f82440cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.213+0000 7f8243e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f824406c5c0 0x7f82440b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44680/0 (socket says 172.21.15.19:44680) 2024-04-25T11:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.213+0000 7f8243e00640 1 -- 172.21.15.19:0/2756910196 learned_addr learned my addr 172.21.15.19:0/2756910196 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.214+0000 7f8249200640 1 -- 172.21.15.19:0/2756910196 >> v1:172.21.15.169:6789/0 conn(0x7f82440b4ea0 legacy=0x7f824415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.214+0000 7f8249200640 1 -- 172.21.15.19:0/2756910196 >> v1:172.21.15.19:6789/0 conn(0x7f824406c5c0 legacy=0x7f82440b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.214+0000 7f8249200640 1 -- 172.21.15.19:0/2756910196 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82440d0c70 con 0x7f8244144370 2024-04-25T11:30:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.215+0000 7f8249200640 1 --2- 172.21.15.19:0/2756910196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8244144370 0x7f82440cfa70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f823c009d90 tx=0x7f823c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7798bb9aba284eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.215+0000 7f8242a00640 1 -- 172.21.15.19:0/2756910196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c007740 con 0x7f8244144370 2024-04-25T11:30:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.215+0000 7f8242a00640 1 -- 172.21.15.19:0/2756910196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823c0078e0 con 0x7f8244144370 2024-04-25T11:30:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.215+0000 7f8242a00640 1 -- 172.21.15.19:0/2756910196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c007cb0 con 0x7f8244144370 2024-04-25T11:30:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.216+0000 7f8249c00640 1 -- 172.21.15.19:0/2756910196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8244144370 msgr2=0x7f82440cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.216+0000 7f8249c00640 1 --2- 172.21.15.19:0/2756910196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8244144370 0x7f82440cfa70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f823c009d90 tx=0x7f823c02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.216+0000 7f8249c00640 1 -- 172.21.15.19:0/2756910196 shutdown_connections 2024-04-25T11:30:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.216+0000 7f8249c00640 1 --2- 172.21.15.19:0/2756910196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8244144370 0x7f82440cfa70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.216+0000 7f8249c00640 1 -- 172.21.15.19:0/2756910196 >> 172.21.15.19:0/2756910196 conn(0x7f82440d1720 msgr2=0x7f82440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.216+0000 7f8249c00640 1 -- 172.21.15.19:0/2756910196 shutdown_connections 2024-04-25T11:30:42.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.216+0000 7f8249c00640 1 -- 172.21.15.19:0/2756910196 wait complete. 2024-04-25T11:30:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.219+0000 7f8249c00640 1 Processor -- start 2024-04-25T11:30:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.219+0000 7f8249c00640 1 -- start start 2024-04-25T11:30:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.220+0000 7f8249c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f824406c5c0 0x7f824415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.220+0000 7f8249c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8244144370 0x7f82441358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.220+0000 7f8249c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824415f910 0x7f8244146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.220+0000 7f8249c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82440d0450 con 0x7f824406c5c0 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.220+0000 7f8249c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82440d02d0 con 0x7f824415f910 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.220+0000 7f8249c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8244147760 con 0x7f8244144370 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.220+0000 7f8243e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f824406c5c0 0x7f824415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f824406c5c0 0x7f824415d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55948/0 (socket says 172.21.15.19:55948) 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 -- 172.21.15.19:0/2989482322 learned_addr learned my addr 172.21.15.19:0/2989482322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8249200640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824415f910 0x7f8244146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243400640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8244144370 0x7f82441358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 -- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8244144370 msgr2=0x7f82441358a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8244144370 0x7f82441358a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 -- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824415f910 msgr2=0x7f8244146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824415f910 0x7f8244146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 -- 172.21.15.19:0/2989482322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8244145410 con 0x7f824406c5c0 2024-04-25T11:30:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8249200640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824415f910 0x7f8244146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.221+0000 7f8243e00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f824406c5c0 0x7f824415d770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f823400c4b0 tx=0x7f823400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.222+0000 7f823be00640 1 -- 172.21.15.19:0/2989482322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234017070 con 0x7f824406c5c0 2024-04-25T11:30:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.222+0000 7f823be00640 1 -- 172.21.15.19:0/2989482322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8234005ce0 con 0x7f824406c5c0 2024-04-25T11:30:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.222+0000 7f823be00640 1 -- 172.21.15.19:0/2989482322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234007270 con 0x7f824406c5c0 2024-04-25T11:30:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.222+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82441456f0 con 0x7f824406c5c0 2024-04-25T11:30:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.222+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8244145cb0 con 0x7f824406c5c0 2024-04-25T11:30:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.223+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8244050740 con 0x7f824406c5c0 2024-04-25T11:30:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.224+0000 7f823be00640 1 -- 172.21.15.19:0/2989482322 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f823401a070 con 0x7f824406c5c0 2024-04-25T11:30:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.225+0000 7f823be00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82240744b0 0x7f82240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.226+0000 7f823be00640 1 -- 172.21.15.19:0/2989482322 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f8234094450 con 0x7f824406c5c0 2024-04-25T11:30:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.226+0000 7f8243400640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82240744b0 0x7f82240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.228+0000 7f8243400640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82240744b0 0x7f82240769c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8230001500 tx=0x7f8230009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.230+0000 7f823be00640 1 -- 172.21.15.19:0/2989482322 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f823405e650 con 0x7f824406c5c0 2024-04-25T11:30:42.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.479+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f824415f340 con 0x7f824406c5c0 2024-04-25T11:30:42.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.481+0000 7f823be00640 1 -- 172.21.15.19:0/2989482322 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5181 (secure 0 0 0) 0x7f8234062950 con 0x7f824406c5c0 2024-04-25T11:30:42.472 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:42.472 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:41.024756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T11:30:42.472 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 40 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82240744b0 msgr2=0x7f82240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82240744b0 0x7f82240769c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8230001500 tx=0x7f8230009040 comp rx=0 tx=0).stop 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f824406c5c0 msgr2=0x7f824415d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f824406c5c0 0x7f824415d770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f823400c4b0 tx=0x7f823400c970 comp rx=0 tx=0).stop 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 shutdown_connections 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82240744b0 0x7f82240769c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824415f910 0x7f8244146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8244144370 0x7f82441358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 --2- 172.21.15.19:0/2989482322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f824406c5c0 0x7f824415d770 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 >> 172.21.15.19:0/2989482322 conn(0x7f82440d1720 msgr2=0x7f82440b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:42.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 shutdown_connections 2024-04-25T11:30:42.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.484+0000 7f8249c00640 1 -- 172.21.15.19:0/2989482322 wait complete. 2024-04-25T11:30:42.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:42.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.796+0000 7f520ec00640 1 Processor -- start 2024-04-25T11:30:42.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.796+0000 7f520ec00640 1 -- start start 2024-04-25T11:30:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.798+0000 7f520ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520815cd00 0x7f52080c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.798+0000 7f520ec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52080c70a0 0x7f5208073980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.798+0000 7f520ec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52080c9090 con 0x7f52080c74b0 2024-04-25T11:30:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.798+0000 7f520ec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f520815d450 con 0x7f52080c70a0 2024-04-25T11:30:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.798+0000 7f520ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f520815d730 con 0x7f520815cd00 2024-04-25T11:30:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52080c70a0 0x7f5208073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52080c70a0 0x7f5208073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55962/0 (socket says 172.21.15.19:55962) 2024-04-25T11:30:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207400640 1 -- 172.21.15.19:0/1211695412 learned_addr learned my addr 172.21.15.19:0/1211695412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:42.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f520cc00640 1 --1- 172.21.15.19:0/1211695412 >> v1:172.21.15.19:6789/0 conn(0x7f52080c74b0 0x7f520806cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44694/0 (socket says 172.21.15.19:44694) 2024-04-25T11:30:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207e00640 1 --2- 172.21.15.19:0/1211695412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520815cd00 0x7f52080c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207400640 1 -- 172.21.15.19:0/1211695412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520815cd00 msgr2=0x7f52080c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207400640 1 --2- 172.21.15.19:0/1211695412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520815cd00 0x7f52080c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207400640 1 -- 172.21.15.19:0/1211695412 >> v1:172.21.15.19:6789/0 conn(0x7f52080c74b0 legacy=0x7f520806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.799+0000 7f5207400640 1 -- 172.21.15.19:0/1211695412 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520815da10 con 0x7f52080c70a0 2024-04-25T11:30:42.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.800+0000 7f5207400640 1 --2- 172.21.15.19:0/1211695412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080c70a0 0x7f5208073980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009870 tx=0x7f51f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac85c55652e342e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:42.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.801+0000 7f5206a00640 1 -- 172.21.15.19:0/1211695412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4037070 con 0x7f52080c70a0 2024-04-25T11:30:42.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.801+0000 7f5206a00640 1 -- 172.21.15.19:0/1211695412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f4032440 con 0x7f52080c70a0 2024-04-25T11:30:42.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.801+0000 7f5206a00640 1 -- 172.21.15.19:0/1211695412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4032810 con 0x7f52080c70a0 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 -- 172.21.15.19:0/1211695412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080c70a0 msgr2=0x7f5208073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 --2- 172.21.15.19:0/1211695412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080c70a0 0x7f5208073980 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009870 tx=0x7f51f402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 -- 172.21.15.19:0/1211695412 shutdown_connections 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 --2- 172.21.15.19:0/1211695412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52080c70a0 0x7f5208073980 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 --2- 172.21.15.19:0/1211695412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f520815cd00 0x7f52080c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 -- 172.21.15.19:0/1211695412 >> 172.21.15.19:0/1211695412 conn(0x7f52080d21c0 msgr2=0x7f52080d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 -- 172.21.15.19:0/1211695412 shutdown_connections 2024-04-25T11:30:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.802+0000 7f520ec00640 1 -- 172.21.15.19:0/1211695412 wait complete. 2024-04-25T11:30:42.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.804+0000 7f520ec00640 1 Processor -- start 2024-04-25T11:30:42.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.804+0000 7f520ec00640 1 -- start start 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52080c70a0 0x7f520814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52080c74b0 0x7f520814ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815cd00 0x7f520815fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520ec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52081539d0 con 0x7f520815cd00 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f520815d110 con 0x7f52080c70a0 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5208153b50 con 0x7f52080c74b0 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815cd00 0x7f520815fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815cd00 0x7f520815fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55964/0 (socket says 172.21.15.19:55964) 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.806+0000 7f520cc00640 1 -- 172.21.15.19:0/4074348308 learned_addr learned my addr 172.21.15.19:0/4074348308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f5207e00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52080c70a0 0x7f520814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f5207400640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52080c74b0 0x7f520814ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f520cc00640 1 -- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52080c74b0 msgr2=0x7f520814ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f520cc00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52080c74b0 0x7f520814ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f520cc00640 1 -- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52080c70a0 msgr2=0x7f520814a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f520cc00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52080c70a0 0x7f520814a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f520cc00640 1 -- 172.21.15.19:0/4074348308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520814f790 con 0x7f520815cd00 2024-04-25T11:30:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f5207400640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52080c74b0 0x7f520814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:42.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f520cc00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815cd00 0x7f520815fd10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f520000ca70 tx=0x7f520000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.807+0000 7f51ffe00640 1 -- 172.21.15.19:0/4074348308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520000ab70 con 0x7f520815cd00 2024-04-25T11:30:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.808+0000 7f51ffe00640 1 -- 172.21.15.19:0/4074348308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520000ad10 con 0x7f520815cd00 2024-04-25T11:30:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.808+0000 7f51ffe00640 1 -- 172.21.15.19:0/4074348308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52000072d0 con 0x7f520815cd00 2024-04-25T11:30:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.808+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520814fa70 con 0x7f520815cd00 2024-04-25T11:30:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.808+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520814f210 con 0x7f520815cd00 2024-04-25T11:30:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.809+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520814f4e0 con 0x7f520815cd00 2024-04-25T11:30:42.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.810+0000 7f51ffe00640 1 -- 172.21.15.19:0/4074348308 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5200007470 con 0x7f520815cd00 2024-04-25T11:30:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.812+0000 7f51ffe00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f51e80744b0 0x7f51e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.812+0000 7f51ffe00640 1 -- 172.21.15.19:0/4074348308 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f5200095870 con 0x7f520815cd00 2024-04-25T11:30:42.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.813+0000 7f5207e00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f51e80744b0 0x7f51e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.814+0000 7f5207e00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f51e80744b0 0x7f51e80769c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51f8006fd0 tx=0x7f51f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:42.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:42.817+0000 7f51ffe00640 1 -- 172.21.15.19:0/4074348308 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52080c9090 con 0x7f520815cd00 2024-04-25T11:30:43.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.065+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52080745d0 con 0x7f520815cd00 2024-04-25T11:30:43.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.067+0000 7f51ffe00640 1 -- 172.21.15.19:0/4074348308 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5178 (secure 0 0 0) 0x7f520005ff00 con 0x7f520815cd00 2024-04-25T11:30:43.057 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:43.057 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:42.010061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T11:30:43.057 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 41 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.069+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f51e80744b0 msgr2=0x7f51e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f51e80744b0 0x7f51e80769c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51f8006fd0 tx=0x7f51f8008040 comp rx=0 tx=0).stop 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815cd00 msgr2=0x7f520815fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815cd00 0x7f520815fd10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f520000ca70 tx=0x7f520000cf30 comp rx=0 tx=0).stop 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 shutdown_connections 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f51e80744b0 0x7f51e80769c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f520815cd00 0x7f520815fd10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52080c74b0 0x7f520814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 --2- 172.21.15.19:0/4074348308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52080c70a0 0x7f520814a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 >> 172.21.15.19:0/4074348308 conn(0x7f52080d21c0 msgr2=0x7f52080cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 shutdown_connections 2024-04-25T11:30:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.070+0000 7f520ec00640 1 -- 172.21.15.19:0/4074348308 wait complete. 2024-04-25T11:30:43.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.379+0000 7fbd79000640 1 Processor -- start 2024-04-25T11:30:43.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.379+0000 7fbd79000640 1 -- start start 2024-04-25T11:30:43.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.381+0000 7fbd79000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd740b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd79000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd741608d0 0x7fbd74160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd79000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd74161220 con 0x7fbd7415a810 2024-04-25T11:30:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd79000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbd7406f870 con 0x7fbd741608d0 2024-04-25T11:30:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd79000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd740b74d0 con 0x7fbd740cf2f0 2024-04-25T11:30:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd73e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd741608d0 0x7fbd74160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd73e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd741608d0 0x7fbd74160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55970/0 (socket says 172.21.15.19:55970) 2024-04-25T11:30:43.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd73e00640 1 -- 172.21.15.19:0/1249651470 learned_addr learned my addr 172.21.15.19:0/1249651470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.382+0000 7fbd73400640 1 --2- 172.21.15.19:0/1249651470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd740b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.383+0000 7fbd73e00640 1 -- 172.21.15.19:0/1249651470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 msgr2=0x7fbd740b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.383+0000 7fbd73e00640 1 --2- 172.21.15.19:0/1249651470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd740b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.383+0000 7fbd73e00640 1 -- 172.21.15.19:0/1249651470 >> v1:172.21.15.169:6789/0 conn(0x7fbd7415a810 legacy=0x7fbd740b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:43.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.383+0000 7fbd73e00640 1 -- 172.21.15.19:0/1249651470 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd740b77b0 con 0x7fbd741608d0 2024-04-25T11:30:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.384+0000 7fbd73e00640 1 --2- 172.21.15.19:0/1249651470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd741608d0 0x7fbd74160ce0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbd7406f9f0 tx=0x7fbd6c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3a05bfafba6b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.384+0000 7fbd72000640 1 -- 172.21.15.19:0/1249651470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c032770 con 0x7fbd741608d0 2024-04-25T11:30:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.384+0000 7fbd72000640 1 -- 172.21.15.19:0/1249651470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c038070 con 0x7fbd741608d0 2024-04-25T11:30:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.384+0000 7fbd72000640 1 -- 172.21.15.19:0/1249651470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c032b30 con 0x7fbd741608d0 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 -- 172.21.15.19:0/1249651470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd741608d0 msgr2=0x7fbd74160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 --2- 172.21.15.19:0/1249651470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd741608d0 0x7fbd74160ce0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbd7406f9f0 tx=0x7fbd6c02c8b0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 -- 172.21.15.19:0/1249651470 shutdown_connections 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 --2- 172.21.15.19:0/1249651470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd741608d0 0x7fbd74160ce0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 --2- 172.21.15.19:0/1249651470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd740b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 -- 172.21.15.19:0/1249651470 >> 172.21.15.19:0/1249651470 conn(0x7fbd740d1720 msgr2=0x7fbd740d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 -- 172.21.15.19:0/1249651470 shutdown_connections 2024-04-25T11:30:43.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.385+0000 7fbd79000640 1 -- 172.21.15.19:0/1249651470 wait complete. 2024-04-25T11:30:43.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.388+0000 7fbd79000640 1 Processor -- start 2024-04-25T11:30:43.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.388+0000 7fbd79000640 1 -- start start 2024-04-25T11:30:43.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.389+0000 7fbd79000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd741fa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:43.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.389+0000 7fbd79000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd7415a810 0x7fbd74158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:43.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.389+0000 7fbd79000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd741608d0 0x7fbd741f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:43.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd79000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7415ac20 con 0x7fbd7415a810 2024-04-25T11:30:43.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd79000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7406f500 con 0x7fbd741608d0 2024-04-25T11:30:43.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd79000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7406fd00 con 0x7fbd740cf2f0 2024-04-25T11:30:43.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd7415a810 0x7fbd74158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:43.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd7415a810 0x7fbd74158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55978/0 (socket says 172.21.15.19:55978) 2024-04-25T11:30:43.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 -- 172.21.15.19:0/2865648978 learned_addr learned my addr 172.21.15.19:0/2865648978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd73e00640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd741608d0 0x7fbd741f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 -- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 msgr2=0x7fbd741fa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd73400640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd741fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd741fa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 -- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd741608d0 msgr2=0x7fbd741f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd741608d0 0x7fbd741f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd72a00640 1 -- 172.21.15.19:0/2865648978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd741fa860 con 0x7fbd7415a810 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.390+0000 7fbd73e00640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd741608d0 0x7fbd741f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.391+0000 7fbd72a00640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd7415a810 0x7fbd74158520 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00ba80 tx=0x7fbd5c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.391+0000 7fbd6b400640 1 -- 172.21.15.19:0/2865648978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00cc00 con 0x7fbd7415a810 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.391+0000 7fbd6b400640 1 -- 172.21.15.19:0/2865648978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd5c00cda0 con 0x7fbd7415a810 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.391+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd74157480 con 0x7fbd7415a810 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.391+0000 7fbd6b400640 1 -- 172.21.15.19:0/2865648978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c003bb0 con 0x7fbd7415a810 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.391+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd74157a40 con 0x7fbd7415a810 2024-04-25T11:30:43.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.392+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd74050740 con 0x7fbd7415a810 2024-04-25T11:30:43.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.394+0000 7fbd6b400640 1 -- 172.21.15.19:0/2865648978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd5c014440 con 0x7fbd7415a810 2024-04-25T11:30:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.395+0000 7fbd6b400640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd500744b0 0x7fbd500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.395+0000 7fbd6b400640 1 -- 172.21.15.19:0/2865648978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fbd5c0952b0 con 0x7fbd7415a810 2024-04-25T11:30:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.395+0000 7fbd73400640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd500744b0 0x7fbd500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:43.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.397+0000 7fbd73400640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd500744b0 0x7fbd500769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006fd0 tx=0x7fbd64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.399+0000 7fbd6b400640 1 -- 172.21.15.19:0/2865648978 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd741fa860 con 0x7fbd7415a810 2024-04-25T11:30:43.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.647+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd7406c770 con 0x7fbd7415a810 2024-04-25T11:30:43.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.648+0000 7fbd6b400640 1 -- 172.21.15.19:0/2865648978 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5178 (secure 0 0 0) 0x7fbd5c05f940 con 0x7fbd7415a810 2024-04-25T11:30:43.638 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:43.638 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:42.010061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T11:30:43.639 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 41 2024-04-25T11:30:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd500744b0 msgr2=0x7fbd500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd500744b0 0x7fbd500769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006fd0 tx=0x7fbd64008040 comp rx=0 tx=0).stop 2024-04-25T11:30:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd7415a810 msgr2=0x7fbd74158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd7415a810 0x7fbd74158520 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00ba80 tx=0x7fbd5c00bf40 comp rx=0 tx=0).stop 2024-04-25T11:30:43.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 shutdown_connections 2024-04-25T11:30:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd500744b0 0x7fbd500769c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd741608d0 0x7fbd741f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd7415a810 0x7fbd74158520 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 --2- 172.21.15.19:0/2865648978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd740cf2f0 0x7fbd741fa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 >> 172.21.15.19:0/2865648978 conn(0x7fbd740d1720 msgr2=0x7fbd740cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 shutdown_connections 2024-04-25T11:30:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:43.651+0000 7fbd79000640 1 -- 172.21.15.19:0/2865648978 wait complete. 2024-04-25T11:30:43.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /a 0 2024-04-25T11:30:43.975 INFO:teuthology.orchestra.run.smithi019.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-04-25T11:30:33.640420+0000","btime":"2024-04-25T11:30:26.979488+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-25T11:30:33.640420+0000","atime":"2024-04-25T11:30:26.979488+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-25T11:30:33.640420+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-25T11:30:33.640420+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-25T11:30:33.640420+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirtyscattered":1,"dirfrag":1,"dirty":1},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-25T11:30:43.976 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-25T11:30:33.640420+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-04-25T11:30:26.979488+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-25T11:30:26.979488+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-04-25T11:30:33.640420+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-25T11:30:33.640420+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-25T11:30:33.640420+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1, "dirtyscattered": 1 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-25T11:30:33.640420+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-04-25T11:30:43.976 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-04-25T11:30:33.640420+0000', 'btime': '2024-04-25T11:30:26.979488+0000', 'mode': 16877, 'uid': 1000, 'gid': 1316, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-04-25T11:30:33.640420+0000', 'atime': '2024-04-25T11:30:26.979488+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-04-25T11:30:33.640420+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-25T11:30:33.640420+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-04-25T11:30:33.640420+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirtyscattered': 1, 'dirfrag': 1, 'dirty': 1}, 'nref': 3, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-04-25T11:30:43.976 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:30:43.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:30:44.080 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:30:44.187 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:30:44.187 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:30:44.187 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:30:44.187 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:44.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:30:44.250 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:30:44.251 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:30:44.251 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:44.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:30:44.315 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:30:44.315 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:30:44.315 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:44.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:30:44.373 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:30:44.373 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:30:44.373 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:44.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:30:44.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:44.634 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:30:44.644+0000 7f03c5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:30:44.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.735+0000 7f628b200640 1 Processor -- start 2024-04-25T11:30:44.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.735+0000 7f628b200640 1 -- start start 2024-04-25T11:30:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.737+0000 7f628b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f628c070e20 0x7f628c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.737+0000 7f628b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f628c071230 con 0x7f628c06f180 2024-04-25T11:30:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.737+0000 7f628b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f628c06f8f0 con 0x7f628c06d560 2024-04-25T11:30:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.737+0000 7f628b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f628c06fbd0 con 0x7f628c070e20 2024-04-25T11:30:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.738+0000 7f6289400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f628c06f180 0x7f628c06ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41572/0 (socket says 172.21.15.19:41572) 2024-04-25T11:30:44.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.738+0000 7f6289400640 1 -- 172.21.15.19:0/2951549282 learned_addr learned my addr 172.21.15.19:0/2951549282 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.738+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198629807 0 0) 0x7f628c071230 con 0x7f628c06f180 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.738+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6270004630 con 0x7f628c06f180 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.738+0000 7f6289e00640 1 --2- 172.21.15.19:0/2951549282 >> v2:172.21.15.169:3300/0 conn(0x7f628c070e20 0x7f628c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3120959520 0 0) 0x7f6270004630 con 0x7f628c06f180 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 >> v2:172.21.15.169:3300/0 conn(0x7f628c070e20 msgr2=0x7f628c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 --2- 172.21.15.19:0/2951549282 >> v2:172.21.15.169:3300/0 conn(0x7f628c070e20 0x7f628c15c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 >> v1:172.21.15.169:6789/0 conn(0x7f628c06d560 legacy=0x7f628c0b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c06feb0 con 0x7f628c06f180 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f627c0019f0 con 0x7f628c06f180 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 >> v1:172.21.15.19:6789/0 conn(0x7f628c06f180 legacy=0x7f628c06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:44.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6270004ac0 0x7f6270004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6270005910 0x7f6270005d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62700062d0 0x7f6270003440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f628c06fbd0 con 0x7f62700062d0 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6270003be0 con 0x7f6270004ac0 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6270003de0 con 0x7f6270005910 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.739+0000 7f628a800640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62700062d0 0x7f6270003440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f6289e00640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6270004ac0 0x7f6270004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f6289400640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6270005910 0x7f6270005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f628a800640 1 -- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6270005910 msgr2=0x7f6270005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f628a800640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6270005910 0x7f6270005d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f628a800640 1 -- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6270004ac0 msgr2=0x7f6270004f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f628a800640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6270004ac0 0x7f6270004f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:44.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f628a800640 1 -- 172.21.15.19:0/2951549282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6270004630 con 0x7f62700062d0 2024-04-25T11:30:44.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.740+0000 7f6289e00640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6270004ac0 0x7f6270004f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.741+0000 7f628a800640 1 -- 172.21.15.19:0/2951549282 reap_dead start 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.741+0000 7f628a800640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62700062d0 0x7f6270003440 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6284001750 tx=0x7f6270006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8001f4a200c1ebad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.741+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284001370 con 0x7f62700062d0 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.741+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6284005310 con 0x7f62700062d0 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.741+0000 7f6288a00640 1 -- 172.21.15.19:0/2951549282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628402ab20 con 0x7f62700062d0 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.742+0000 7f628b200640 1 -- 172.21.15.19:0/2951549282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62700062d0 msgr2=0x7f6270003440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.742+0000 7f628b200640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62700062d0 0x7f6270003440 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6284001750 tx=0x7f6270006940 comp rx=0 tx=0).stop 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.742+0000 7f628b200640 1 -- 172.21.15.19:0/2951549282 shutdown_connections 2024-04-25T11:30:44.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.742+0000 7f628b200640 1 --2- 172.21.15.19:0/2951549282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62700062d0 0x7f6270003440 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:44.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.742+0000 7f628b200640 1 -- 172.21.15.19:0/2951549282 >> 172.21.15.19:0/2951549282 conn(0x7f628c0d1720 msgr2=0x7f628c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:44.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.742+0000 7f628b200640 1 -- 172.21.15.19:0/2951549282 shutdown_connections 2024-04-25T11:30:44.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.742+0000 7f628b200640 1 -- 172.21.15.19:0/2951549282 wait complete. 2024-04-25T11:30:44.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.745+0000 7f628b200640 1 Processor -- start 2024-04-25T11:30:44.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.745+0000 7f628b200640 1 -- start start 2024-04-25T11:30:44.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.746+0000 7f628b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62700062d0 0x7f628c14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.746+0000 7f628b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c0ccd60 0x7f628c14e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.746+0000 7f628b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c14ea90 0x7f628c1e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.746+0000 7f628b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f628c15c800 con 0x7f628c0ccd60 2024-04-25T11:30:44.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.746+0000 7f628b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f628c06f590 con 0x7f628c14ea90 2024-04-25T11:30:44.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.746+0000 7f628b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f628c15c980 con 0x7f62700062d0 2024-04-25T11:30:44.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c0ccd60 0x7f628c14e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c0ccd60 0x7f628c14e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47764/0 (socket says 172.21.15.19:47764) 2024-04-25T11:30:44.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 -- 172.21.15.19:0/3735302824 learned_addr learned my addr 172.21.15.19:0/3735302824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:44.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289e00640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62700062d0 0x7f628c14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 -- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62700062d0 msgr2=0x7f628c14df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:44.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f628a800640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c14ea90 0x7f628c1e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62700062d0 0x7f628c14df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 -- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c14ea90 msgr2=0x7f628c1e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c14ea90 0x7f628c1e61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.747+0000 7f6289400640 1 -- 172.21.15.19:0/3735302824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c0cd170 con 0x7f628c0ccd60 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.748+0000 7f6289400640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c0ccd60 0x7f628c14e550 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f627c002150 tx=0x7f627c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.748+0000 7f6282000640 1 -- 172.21.15.19:0/3735302824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c019070 con 0x7f628c0ccd60 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.748+0000 7f6282000640 1 -- 172.21.15.19:0/3735302824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627c004d60 con 0x7f628c0ccd60 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.748+0000 7f6282000640 1 -- 172.21.15.19:0/3735302824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c0072f0 con 0x7f628c0ccd60 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.748+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c0cbcc0 con 0x7f628c0ccd60 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.748+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c0cc280 con 0x7f628c0ccd60 2024-04-25T11:30:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.749+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628c050740 con 0x7f628c0ccd60 2024-04-25T11:30:44.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.750+0000 7f6282000640 1 -- 172.21.15.19:0/3735302824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f627c01c070 con 0x7f628c0ccd60 2024-04-25T11:30:44.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.751+0000 7f6282000640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62680744b0 0x7f62680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:44.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.752+0000 7f6282000640 1 -- 172.21.15.19:0/3735302824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f627c095550 con 0x7f628c0ccd60 2024-04-25T11:30:44.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.752+0000 7f6289e00640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62680744b0 0x7f62680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:44.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.753+0000 7f6289e00640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62680744b0 0x7f62680769c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f62740098b0 tx=0x7f6274008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:44.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:44.757+0000 7f6282000640 1 -- 172.21.15.19:0/3735302824 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f627c05fbe0 con 0x7f628c0ccd60 2024-04-25T11:30:44.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.006+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f628c06caf0 con 0x7f628c0ccd60 2024-04-25T11:30:44.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.007+0000 7f6282000640 1 -- 172.21.15.19:0/3735302824 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5185 (secure 0 0 0) 0x7f627c05f4e0 con 0x7f628c0ccd60 2024-04-25T11:30:44.997 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:44.998 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:44.021828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5991,"qdb_cluster":[5991]},"id":3}]} 2024-04-25T11:30:44.998 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-04-25T11:30:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62680744b0 msgr2=0x7f62680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62680744b0 0x7f62680769c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f62740098b0 tx=0x7f6274008040 comp rx=0 tx=0).stop 2024-04-25T11:30:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c0ccd60 msgr2=0x7f628c14e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c0ccd60 0x7f628c14e550 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f627c002150 tx=0x7f627c00b4a0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 shutdown_connections 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62680744b0 0x7f62680769c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f628c14ea90 0x7f628c1e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f628c0ccd60 0x7f628c14e550 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 --2- 172.21.15.19:0/3735302824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62700062d0 0x7f628c14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.010+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 >> 172.21.15.19:0/3735302824 conn(0x7f628c0d1720 msgr2=0x7f628c0d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.011+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 shutdown_connections 2024-04-25T11:30:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.011+0000 7f628b200640 1 -- 172.21.15.19:0/3735302824 wait complete. 2024-04-25T11:30:45.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:45.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.313+0000 7fb0f9e00640 1 Processor -- start 2024-04-25T11:30:45.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.313+0000 7fb0f9e00640 1 -- start start 2024-04-25T11:30:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.316+0000 7fb0f9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 0x7fb0f40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.316+0000 7fb0f9e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f4161650 con 0x7fb0f40b7540 2024-04-25T11:30:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.316+0000 7fb0f9e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f4161370 con 0x7fb0f40d44a0 2024-04-25T11:30:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.316+0000 7fb0f9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f41643d0 con 0x7fb0f4164860 2024-04-25T11:30:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.316+0000 7fb0f3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb0f40b7540 0x7fb0f40b7950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41574/0 (socket says 172.21.15.19:41574) 2024-04-25T11:30:45.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.316+0000 7fb0f3e00640 1 -- 172.21.15.19:0/113394243 learned_addr learned my addr 172.21.15.19:0/113394243 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f8a00640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 0x7fb0f40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483166061 0 0) 0x7fb0f4161650 con 0x7fb0f40b7540 2024-04-25T11:30:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0dc003520 con 0x7fb0f40b7540 2024-04-25T11:30:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 974822522 0 0) 0x7fb0dc003520 con 0x7fb0f40b7540 2024-04-25T11:30:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 msgr2=0x7fb0f40b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 0x7fb0f40b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 >> v1:172.21.15.169:6789/0 conn(0x7fb0f40d44a0 legacy=0x7fb0f40d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f4163d80 con 0x7fb0f40b7540 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb0e0001670 con 0x7fb0f40b7540 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 >> v1:172.21.15.19:6789/0 conn(0x7fb0f40b7540 legacy=0x7fb0f40b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 0x7fb0dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0dc003fb0 0x7fb0dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0dc004d40 0x7fb0dc0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f41643d0 con 0x7fb0dc003fb0 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc00bbb0 con 0x7fb0dc004d40 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.317+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc00bdb0 con 0x7fb0f4164860 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f3e00640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0dc003fb0 0x7fb0dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f8a00640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 0x7fb0dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f9400640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0dc004d40 0x7fb0dc0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f3e00640 1 -- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 msgr2=0x7fb0dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:45.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f3e00640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0f4164860 0x7fb0dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f3e00640 1 -- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0dc004d40 msgr2=0x7fb0dc0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f3e00640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0dc004d40 0x7fb0dc0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.318+0000 7fb0f3e00640 1 -- 172.21.15.19:0/113394243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc003520 con 0x7fb0dc003fb0 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.319+0000 7fb0f3e00640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0dc003fb0 0x7fb0dc004800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c0e0 tx=0x7fb0dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a4eefcf3fadeb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0007c40 con 0x7fb0dc003fb0 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e0005af0 con 0x7fb0dc003fb0 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f3400640 1 -- 172.21.15.19:0/113394243 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0007070 con 0x7fb0dc003fb0 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f9e00640 1 -- 172.21.15.19:0/113394243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0dc003fb0 msgr2=0x7fb0dc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f9e00640 1 --2- 172.21.15.19:0/113394243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0dc003fb0 0x7fb0dc004800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c0e0 tx=0x7fb0dc005830 comp rx=0 tx=0).stop 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f9400640 1 -- 172.21.15.19:0/113394243 reap_dead start 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f9e00640 1 -- 172.21.15.19:0/113394243 shutdown_connections 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.320+0000 7fb0f9e00640 1 -- 172.21.15.19:0/113394243 >> 172.21.15.19:0/113394243 conn(0x7fb0f40d2720 msgr2=0x7fb0f40d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.321+0000 7fb0f9e00640 1 -- 172.21.15.19:0/113394243 shutdown_connections 2024-04-25T11:30:45.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.321+0000 7fb0f9e00640 1 -- 172.21.15.19:0/113394243 wait complete. 2024-04-25T11:30:45.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.323+0000 7fb0f9e00640 1 Processor -- start 2024-04-25T11:30:45.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.323+0000 7fb0f9e00640 1 -- start start 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0dc003fb0 0x7fb0f414bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f40cd180 0x7fb0f40cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f41eb760 0x7fb0f41ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40d5140 con 0x7fb0f41eb760 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40d4fc0 con 0x7fb0f40cd180 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f40b4d30 con 0x7fb0dc003fb0 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f41eb760 0x7fb0f41ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f41eb760 0x7fb0f41ebb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47786/0 (socket says 172.21.15.19:47786) 2024-04-25T11:30:45.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f9400640 1 -- 172.21.15.19:0/1635247748 learned_addr learned my addr 172.21.15.19:0/1635247748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:45.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f3e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f40cd180 0x7fb0f40cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.325+0000 7fb0f8a00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0dc003fb0 0x7fb0f414bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9400640 1 -- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0dc003fb0 msgr2=0x7fb0f414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9400640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0dc003fb0 0x7fb0f414bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9400640 1 -- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f40cd180 msgr2=0x7fb0f40cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9400640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f40cd180 0x7fb0f40cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9400640 1 -- 172.21.15.19:0/1635247748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f40cf0f0 con 0x7fb0f41eb760 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f8a00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0dc003fb0 0x7fb0f414bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f3e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f40cd180 0x7fb0f40cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9400640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f41eb760 0x7fb0f41ebb70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b8140 tx=0x7fb0ec00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f0c00640 1 -- 172.21.15.19:0/1635247748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003970 con 0x7fb0f41eb760 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f0c00640 1 -- 172.21.15.19:0/1635247748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec018070 con 0x7fb0f41eb760 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f41ec0b0 con 0x7fb0f41eb760 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f0c00640 1 -- 172.21.15.19:0/1635247748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003d50 con 0x7fb0f41eb760 2024-04-25T11:30:45.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.326+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f40cf640 con 0x7fb0f41eb760 2024-04-25T11:30:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.327+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f4050740 con 0x7fb0f41eb760 2024-04-25T11:30:45.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.329+0000 7fb0f0c00640 1 -- 172.21.15.19:0/1635247748 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb0ec0050d0 con 0x7fb0f41eb760 2024-04-25T11:30:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.330+0000 7fb0f0c00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0d40744b0 0x7fb0d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.330+0000 7fb0f0c00640 1 -- 172.21.15.19:0/1635247748 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fb0ec094fc0 con 0x7fb0f41eb760 2024-04-25T11:30:45.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.330+0000 7fb0f8a00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0d40744b0 0x7fb0d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.332+0000 7fb0f8a00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0d40744b0 0x7fb0d40769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40d3740 tx=0x7fb0e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:45.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.335+0000 7fb0f0c00640 1 -- 172.21.15.19:0/1635247748 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0f41ec0b0 con 0x7fb0f41eb760 2024-04-25T11:30:45.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.544+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0f40d3f30 con 0x7fb0f41eb760 2024-04-25T11:30:45.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.546+0000 7fb0f0c00640 1 -- 172.21.15.19:0/1635247748 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16738 (secure 0 0 0) 0x7fb0ec05f6f0 con 0x7fb0f41eb760 2024-04-25T11:30:45.536 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:45.537 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:38.619771+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:30:10.008986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T11:30:10.799803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3038680263":"2024-04-26T11:30:38.619698+0000","172.21.15.19:6834/3038680263":"2024-04-26T11:30:38.619698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0d40744b0 msgr2=0x7fb0d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0d40744b0 0x7fb0d40769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40d3740 tx=0x7fb0e8008040 comp rx=0 tx=0).stop 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f41eb760 msgr2=0x7fb0f41ebb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f41eb760 0x7fb0f41ebb70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b8140 tx=0x7fb0ec00ec10 comp rx=0 tx=0).stop 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 shutdown_connections 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb0d40744b0 0x7fb0d40769c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0f40cd180 0x7fb0f40cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0f41eb760 0x7fb0f41ebb70 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 --2- 172.21.15.19:0/1635247748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb0dc003fb0 0x7fb0f414bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 >> 172.21.15.19:0/1635247748 conn(0x7fb0f40d2720 msgr2=0x7fb0f40d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 shutdown_connections 2024-04-25T11:30:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.549+0000 7fb0f9e00640 1 -- 172.21.15.19:0/1635247748 wait complete. 2024-04-25T11:30:45.568 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:30:10.008986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:30:10.799803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:30:45.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:30:45.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.865+0000 7f5e3b400640 1 Processor -- start 2024-04-25T11:30:45.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.865+0000 7f5e3b400640 1 -- start start 2024-04-25T11:30:45.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.867+0000 7f5e3b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5e34161710 0x7f5e3415e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.867+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e3406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.867+0000 7f5e3b400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e341609b0 con 0x7f5e34163fa0 2024-04-25T11:30:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.867+0000 7f5e3b400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3406cf10 con 0x7f5e34161710 2024-04-25T11:30:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.867+0000 7f5e3b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5e34164cd0 con 0x7f5e3406c5c0 2024-04-25T11:30:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.867+0000 7f5e39400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5e34163fa0 0x7f5e341643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41586/0 (socket says 172.21.15.19:41586) 2024-04-25T11:30:45.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.867+0000 7f5e39400640 1 -- 172.21.15.19:0/749344359 learned_addr learned my addr 172.21.15.19:0/749344359 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e33e00640 1 --2- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e3406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e38a00640 1 --2- 172.21.15.19:0/749344359 >> v2:172.21.15.169:3300/0 conn(0x7f5e34161710 0x7f5e3415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e33400640 1 -- 172.21.15.19:0/749344359 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785423312 0 0) 0x7f5e2c001490 con 0x7f5e34163fa0 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e33400640 1 -- 172.21.15.19:0/749344359 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e1c003520 con 0x7f5e34163fa0 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e38a00640 1 -- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 msgr2=0x7f5e3406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e38a00640 1 --2- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e3406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e38a00640 1 -- 172.21.15.19:0/749344359 >> v1:172.21.15.19:6789/0 conn(0x7f5e34163fa0 legacy=0x7f5e341643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e33400640 1 -- 172.21.15.19:0/749344359 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 778804430 0 0) 0x7f5e1c003520 con 0x7f5e34163fa0 2024-04-25T11:30:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.868+0000 7f5e38a00640 1 -- 172.21.15.19:0/749344359 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e34160c30 con 0x7f5e34161710 2024-04-25T11:30:45.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.870+0000 7f5e38a00640 1 --2- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34161710 0x7f5e3415e080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5e280097a0 tx=0x7f5e2802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a88bdf84506d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:45.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.870+0000 7f5e33400640 1 -- 172.21.15.19:0/749344359 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28037070 con 0x7f5e34161710 2024-04-25T11:30:45.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.870+0000 7f5e33400640 1 -- 172.21.15.19:0/749344359 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2802ab10 con 0x7f5e34161710 2024-04-25T11:30:45.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.870+0000 7f5e33400640 1 -- 172.21.15.19:0/749344359 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c001bc0 con 0x7f5e34161710 2024-04-25T11:30:45.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 -- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34161710 msgr2=0x7f5e3415e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 --2- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34161710 0x7f5e3415e080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5e280097a0 tx=0x7f5e2802a080 comp rx=0 tx=0).stop 2024-04-25T11:30:45.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 -- 172.21.15.19:0/749344359 shutdown_connections 2024-04-25T11:30:45.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 --2- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e3406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 --2- 172.21.15.19:0/749344359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34161710 0x7f5e3415e080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 -- 172.21.15.19:0/749344359 >> 172.21.15.19:0/749344359 conn(0x7f5e340d5d90 msgr2=0x7f5e340d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 -- 172.21.15.19:0/749344359 shutdown_connections 2024-04-25T11:30:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.871+0000 7f5e3b400640 1 -- 172.21.15.19:0/749344359 wait complete. 2024-04-25T11:30:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.874+0000 7f5e3b400640 1 Processor -- start 2024-04-25T11:30:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.874+0000 7f5e3b400640 1 -- start start 2024-04-25T11:30:45.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.875+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e340cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.875+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e34161710 0x7f5e34158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.875+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34163fa0 0x7f5e3415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.875+0000 7f5e3b400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e340d49f0 con 0x7f5e34161710 2024-04-25T11:30:45.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.875+0000 7f5e3b400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5e341611f0 con 0x7f5e34163fa0 2024-04-25T11:30:45.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.875+0000 7f5e3b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5e340d4b70 con 0x7f5e3406c5c0 2024-04-25T11:30:45.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e34161710 0x7f5e34158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e34161710 0x7f5e34158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47796/0 (socket says 172.21.15.19:47796) 2024-04-25T11:30:45.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 -- 172.21.15.19:0/2183513595 learned_addr learned my addr 172.21.15.19:0/2183513595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:45.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e39400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34163fa0 0x7f5e3415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e38a00640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e340cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 -- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 msgr2=0x7f5e340cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e340cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 -- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34163fa0 msgr2=0x7f5e3415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34163fa0 0x7f5e3415a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e33e00640 1 -- 172.21.15.19:0/2183513595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e341fa870 con 0x7f5e34161710 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.876+0000 7f5e39400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34163fa0 0x7f5e3415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.877+0000 7f5e33e00640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e34161710 0x7f5e34158850 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e870 tx=0x7f5e2000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.877+0000 7f5e30c00640 1 -- 172.21.15.19:0/2183513595 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20017070 con 0x7f5e34161710 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.877+0000 7f5e30c00640 1 -- 172.21.15.19:0/2183513595 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2000c940 con 0x7f5e34161710 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.877+0000 7f5e30c00640 1 -- 172.21.15.19:0/2183513595 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000cd30 con 0x7f5e34161710 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.877+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e341fab50 con 0x7f5e34161710 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.877+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e34155c70 con 0x7f5e34161710 2024-04-25T11:30:45.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.879+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e341fb070 con 0x7f5e34161710 2024-04-25T11:30:45.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.880+0000 7f5e30c00640 1 -- 172.21.15.19:0/2183513595 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5e2001a070 con 0x7f5e34161710 2024-04-25T11:30:45.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.881+0000 7f5e30c00640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e140744b0 0x7f5e140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.881+0000 7f5e30c00640 1 -- 172.21.15.19:0/2183513595 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f5e20095200 con 0x7f5e34161710 2024-04-25T11:30:45.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.882+0000 7f5e38a00640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e140744b0 0x7f5e140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:45.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.883+0000 7f5e38a00640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e140744b0 0x7f5e140769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5e28034000 tx=0x7f5e2802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:45.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:45.886+0000 7f5e30c00640 1 -- 172.21.15.19:0/2183513595 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e341fb070 con 0x7f5e34161710 2024-04-25T11:30:46.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.092+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e34050740 con 0x7f5e34161710 2024-04-25T11:30:46.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.093+0000 7f5e30c00640 1 -- 172.21.15.19:0/2183513595 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f5e2005f890 con 0x7f5e34161710 2024-04-25T11:30:46.083 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e140744b0 msgr2=0x7f5e140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e140744b0 0x7f5e140769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5e28034000 tx=0x7f5e2802e000 comp rx=0 tx=0).stop 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e34161710 msgr2=0x7f5e34158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e34161710 0x7f5e34158850 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e870 tx=0x7f5e2000ed30 comp rx=0 tx=0).stop 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 shutdown_connections 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5e140744b0 0x7f5e140769c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5e34163fa0 0x7f5e3415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e34161710 0x7f5e34158850 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 --2- 172.21.15.19:0/2183513595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5e3406c5c0 0x7f5e340cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 >> 172.21.15.19:0/2183513595 conn(0x7f5e340d5d90 msgr2=0x7f5e340d4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 shutdown_connections 2024-04-25T11:30:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.096+0000 7f5e3b400640 1 -- 172.21.15.19:0/2183513595 wait complete. 2024-04-25T11:30:46.114 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:30:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.413+0000 7fa132800640 1 Processor -- start 2024-04-25T11:30:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.413+0000 7fa132800640 1 -- start start 2024-04-25T11:30:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.415+0000 7fa132800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa12c0ce520 0x7fa12c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.415+0000 7fa132800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0ce930 0x7fa12c15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.415+0000 7fa132800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa12c06d5d0 0x7fa12c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.415+0000 7fa132800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa12c06df20 con 0x7fa12c0ce520 2024-04-25T11:30:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.415+0000 7fa132800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa12c0cda60 con 0x7fa12c06d5d0 2024-04-25T11:30:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.415+0000 7fa132800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa12c15dfe0 con 0x7fa12c0ce930 2024-04-25T11:30:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa12c0ce520 0x7fa12c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa12c0ce520 0x7fa12c15d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47812/0 (socket says 172.21.15.19:47812) 2024-04-25T11:30:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 -- 172.21.15.19:0/4197568511 learned_addr learned my addr 172.21.15.19:0/4197568511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131e00640 1 --2- 172.21.15.19:0/4197568511 >> v2:172.21.15.169:3300/0 conn(0x7fa12c06d5d0 0x7fa12c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa130a00640 1 --2- 172.21.15.19:0/4197568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0ce930 0x7fa12c15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 -- 172.21.15.19:0/4197568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0ce930 msgr2=0x7fa12c15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 --2- 172.21.15.19:0/4197568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0ce930 0x7fa12c15d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 -- 172.21.15.19:0/4197568511 >> v2:172.21.15.169:3300/0 conn(0x7fa12c06d5d0 msgr2=0x7fa12c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 --2- 172.21.15.19:0/4197568511 >> v2:172.21.15.169:3300/0 conn(0x7fa12c06d5d0 0x7fa12c06d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131400640 1 -- 172.21.15.19:0/4197568511 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c15e2c0 con 0x7fa12c0ce520 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa130a00640 1 --2- 172.21.15.19:0/4197568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0ce930 0x7fa12c15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.416+0000 7fa131e00640 1 --2- 172.21.15.19:0/4197568511 >> v2:172.21.15.169:3300/0 conn(0x7fa12c06d5d0 0x7fa12c06d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.417+0000 7fa131400640 1 --2- 172.21.15.19:0/4197568511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce520 0x7fa12c15d4b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa12c072ed0 tx=0x7fa11c02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfbd9b1f7346867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.418+0000 7fa12be00640 1 -- 172.21.15.19:0/4197568511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c032700 con 0x7fa12c0ce520 2024-04-25T11:30:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.418+0000 7fa12be00640 1 -- 172.21.15.19:0/4197568511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c0328a0 con 0x7fa12c0ce520 2024-04-25T11:30:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.418+0000 7fa12be00640 1 -- 172.21.15.19:0/4197568511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c032c70 con 0x7fa12c0ce520 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 -- 172.21.15.19:0/4197568511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce520 msgr2=0x7fa12c15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 --2- 172.21.15.19:0/4197568511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce520 0x7fa12c15d4b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa12c072ed0 tx=0x7fa11c02c800 comp rx=0 tx=0).stop 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 -- 172.21.15.19:0/4197568511 shutdown_connections 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 --2- 172.21.15.19:0/4197568511 >> v2:172.21.15.169:3300/0 conn(0x7fa12c06d5d0 0x7fa12c06d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 --2- 172.21.15.19:0/4197568511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c0ce930 0x7fa12c15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 --2- 172.21.15.19:0/4197568511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce520 0x7fa12c15d4b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 -- 172.21.15.19:0/4197568511 >> 172.21.15.19:0/4197568511 conn(0x7fa12c072f40 msgr2=0x7fa12c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 -- 172.21.15.19:0/4197568511 shutdown_connections 2024-04-25T11:30:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.419+0000 7fa132800640 1 -- 172.21.15.19:0/4197568511 wait complete. 2024-04-25T11:30:46.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.421+0000 7fa132800640 1 Processor -- start 2024-04-25T11:30:46.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.421+0000 7fa132800640 1 -- start start 2024-04-25T11:30:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa132800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c06d5d0 0x7fa12c149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa132800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0ce520 0x7fa12c0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa132800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce930 0x7fa12c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa132800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c160d30 con 0x7fa12c0ce930 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa132800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c0ce000 con 0x7fa12c0ce520 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa132800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa12c0b73d0 con 0x7fa12c06d5d0 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa131e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce930 0x7fa12c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa131e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce930 0x7fa12c15f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47814/0 (socket says 172.21.15.19:47814) 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.423+0000 7fa131e00640 1 -- 172.21.15.19:0/1885968109 learned_addr learned my addr 172.21.15.19:0/1885968109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa130a00640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0ce520 0x7fa12c0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131400640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c06d5d0 0x7fa12c149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131e00640 1 -- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c06d5d0 msgr2=0x7fa12c149100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131e00640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c06d5d0 0x7fa12c149100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131e00640 1 -- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0ce520 msgr2=0x7fa12c0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131e00640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0ce520 0x7fa12c0cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131e00640 1 -- 172.21.15.19:0/1885968109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c0cb8b0 con 0x7fa12c0ce930 2024-04-25T11:30:46.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131400640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c06d5d0 0x7fa12c149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:46.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa131e00640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce930 0x7fa12c15f9f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa12400ccf0 tx=0x7fa12400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.424+0000 7fa129600640 1 -- 172.21.15.19:0/1885968109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12400f680 con 0x7fa12c0ce930 2024-04-25T11:30:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.425+0000 7fa129600640 1 -- 172.21.15.19:0/1885968109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12400f820 con 0x7fa12c0ce930 2024-04-25T11:30:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.425+0000 7fa129600640 1 -- 172.21.15.19:0/1885968109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12400fc10 con 0x7fa12c0ce930 2024-04-25T11:30:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.425+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c0cbb90 con 0x7fa12c0ce930 2024-04-25T11:30:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.425+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c0cc150 con 0x7fa12c0ce930 2024-04-25T11:30:46.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.426+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12c0b4b70 con 0x7fa12c0ce930 2024-04-25T11:30:46.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.427+0000 7fa129600640 1 -- 172.21.15.19:0/1885968109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa1240213f0 con 0x7fa12c0ce930 2024-04-25T11:30:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.428+0000 7fa129600640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1100744e0 0x7fa1100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.429+0000 7fa129600640 1 -- 172.21.15.19:0/1885968109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fa124095ff0 con 0x7fa12c0ce930 2024-04-25T11:30:46.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.429+0000 7fa131400640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1100744e0 0x7fa1100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.431+0000 7fa131400640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1100744e0 0x7fa1100769f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa11c037000 tx=0x7fa11c032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:46.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.434+0000 7fa129600640 1 -- 172.21.15.19:0/1885968109 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa12c0b4b70 con 0x7fa12c0ce930 2024-04-25T11:30:46.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.636+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa12c076c60 con 0x7fa12c0ce930 2024-04-25T11:30:46.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.638+0000 7fa129600640 1 -- 172.21.15.19:0/1885968109 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) ==== 104+0+11 (secure 0 0 0) 0x7fa124014030 con 0x7fa12c0ce930 2024-04-25T11:30:46.627 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:30:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1100744e0 msgr2=0x7fa1100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1100744e0 0x7fa1100769f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa11c037000 tx=0x7fa11c032000 comp rx=0 tx=0).stop 2024-04-25T11:30:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce930 msgr2=0x7fa12c15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce930 0x7fa12c15f9f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa12400ccf0 tx=0x7fa12400a650 comp rx=0 tx=0).stop 2024-04-25T11:30:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 shutdown_connections 2024-04-25T11:30:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1100744e0 0x7fa1100769f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa12c0ce520 0x7fa12c0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa12c0ce930 0x7fa12c15f9f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 --2- 172.21.15.19:0/1885968109 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa12c06d5d0 0x7fa12c149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.640+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 >> 172.21.15.19:0/1885968109 conn(0x7fa12c072f40 msgr2=0x7fa12c15bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:46.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.641+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 shutdown_connections 2024-04-25T11:30:46.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.641+0000 7fa132800640 1 -- 172.21.15.19:0/1885968109 wait complete. 2024-04-25T11:30:46.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:30:46.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.952+0000 7f4ad7200640 1 Processor -- start 2024-04-25T11:30:46.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.952+0000 7f4ad7200640 1 -- start start 2024-04-25T11:30:46.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.954+0000 7f4ad7200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4ad80b6980 0x7f4ad81608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.954+0000 7f4ad7200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad8160110 con 0x7f4ad8160e30 2024-04-25T11:30:46.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.954+0000 7f4ad7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad80b51a0 con 0x7f4ad8070880 2024-04-25T11:30:46.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.954+0000 7f4ad7200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad80b5480 con 0x7f4ad80b6980 2024-04-25T11:30:46.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.955+0000 7f4ad5400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4ad80b6980 0x7f4ad81608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.955+0000 7f4ad6800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4ad8160e30 0x7f4ad806ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41594/0 (socket says 172.21.15.19:41594) 2024-04-25T11:30:46.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.955+0000 7f4ad6800640 1 -- 172.21.15.19:0/276048645 learned_addr learned my addr 172.21.15.19:0/276048645 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:46.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.955+0000 7f4ad5400640 1 -- 172.21.15.19:0/276048645 >> v1:172.21.15.169:6789/0 conn(0x7f4ad8070880 legacy=0x7f4ad80b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:46.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.955+0000 7f4ad5400640 1 -- 172.21.15.19:0/276048645 >> v1:172.21.15.19:6789/0 conn(0x7f4ad8160e30 legacy=0x7f4ad806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.955+0000 7f4ad5400640 1 -- 172.21.15.19:0/276048645 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad80b5760 con 0x7f4ad80b6980 2024-04-25T11:30:46.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.955+0000 7f4ad4a00640 1 -- 172.21.15.19:0/276048645 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243846958 0 0) 0x7f4ad8160110 con 0x7f4ad8160e30 2024-04-25T11:30:46.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.956+0000 7f4ad5400640 1 --2- 172.21.15.19:0/276048645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad80b6980 0x7f4ad81608f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009d90 tx=0x7f4ac002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b9c4ce6f6c5947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:46.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.957+0000 7f4ad4a00640 1 -- 172.21.15.19:0/276048645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0007740 con 0x7f4ad80b6980 2024-04-25T11:30:46.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.957+0000 7f4ad4a00640 1 -- 172.21.15.19:0/276048645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac00078e0 con 0x7f4ad80b6980 2024-04-25T11:30:46.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.957+0000 7f4ad4a00640 1 -- 172.21.15.19:0/276048645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0007cb0 con 0x7f4ad80b6980 2024-04-25T11:30:46.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.957+0000 7f4ad7200640 1 -- 172.21.15.19:0/276048645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad80b6980 msgr2=0x7f4ad81608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.957+0000 7f4ad7200640 1 --2- 172.21.15.19:0/276048645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad80b6980 0x7f4ad81608f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009d90 tx=0x7f4ac002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.958+0000 7f4ad7200640 1 -- 172.21.15.19:0/276048645 shutdown_connections 2024-04-25T11:30:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.958+0000 7f4ad7200640 1 --2- 172.21.15.19:0/276048645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad80b6980 0x7f4ad81608f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.958+0000 7f4ad7200640 1 -- 172.21.15.19:0/276048645 >> 172.21.15.19:0/276048645 conn(0x7f4ad80d1d60 msgr2=0x7f4ad80d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.958+0000 7f4ad7200640 1 -- 172.21.15.19:0/276048645 shutdown_connections 2024-04-25T11:30:46.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.958+0000 7f4ad7200640 1 -- 172.21.15.19:0/276048645 wait complete. 2024-04-25T11:30:46.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.960+0000 7f4ad7200640 1 Processor -- start 2024-04-25T11:30:46.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.961+0000 7f4ad7200640 1 -- start start 2024-04-25T11:30:46.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.962+0000 7f4ad7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ad8070880 0x7f4ad814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.962+0000 7f4ad7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad80b6980 0x7f4ad8149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.962+0000 7f4ad7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad8160e30 0x7f4ad814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.962+0000 7f4ad7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad806d6e0 con 0x7f4ad8160e30 2024-04-25T11:30:46.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.962+0000 7f4ad7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad806d560 con 0x7f4ad80b6980 2024-04-25T11:30:46.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.962+0000 7f4ad7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad806c5c0 con 0x7f4ad8070880 2024-04-25T11:30:46.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad8160e30 0x7f4ad814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad8160e30 0x7f4ad814a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47826/0 (socket says 172.21.15.19:47826) 2024-04-25T11:30:46.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 -- 172.21.15.19:0/1269424893 learned_addr learned my addr 172.21.15.19:0/1269424893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad5e00640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ad8070880 0x7f4ad814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad5400640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad80b6980 0x7f4ad8149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 -- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ad8070880 msgr2=0x7f4ad814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ad8070880 0x7f4ad814edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 -- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad80b6980 msgr2=0x7f4ad8149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad80b6980 0x7f4ad8149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:46.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad6800640 1 -- 172.21.15.19:0/1269424893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad8147ab0 con 0x7f4ad8160e30 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad5e00640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ad8070880 0x7f4ad814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.963+0000 7f4ad5400640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad80b6980 0x7f4ad8149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.964+0000 7f4ad6800640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad8160e30 0x7f4ad814a330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c410 tx=0x7f4ad000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.964+0000 7f4ace000640 1 -- 172.21.15.19:0/1269424893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00074e0 con 0x7f4ad8160e30 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.964+0000 7f4ace000640 1 -- 172.21.15.19:0/1269424893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad00047e0 con 0x7f4ad8160e30 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.964+0000 7f4ace000640 1 -- 172.21.15.19:0/1269424893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0004bd0 con 0x7f4ad8160e30 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.964+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad80cf100 con 0x7f4ad8160e30 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.964+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad8154910 con 0x7f4ad8160e30 2024-04-25T11:30:46.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.965+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad8076c60 con 0x7f4ad8160e30 2024-04-25T11:30:46.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.967+0000 7f4ace000640 1 -- 172.21.15.19:0/1269424893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ad0011070 con 0x7f4ad8160e30 2024-04-25T11:30:46.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.968+0000 7f4ace000640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ab40744b0 0x7f4ab40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:46.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.968+0000 7f4ace000640 1 -- 172.21.15.19:0/1269424893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f4ad0094d70 con 0x7f4ad8160e30 2024-04-25T11:30:46.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.969+0000 7f4ad5e00640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ab40744b0 0x7f4ab40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:46.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.970+0000 7f4ad5e00640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ab40744b0 0x7f4ab40769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800a830 tx=0x7f4ac8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:46.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:46.973+0000 7f4ace000640 1 -- 172.21.15.19:0/1269424893 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ad005f490 con 0x7f4ad8160e30 2024-04-25T11:30:47.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.172+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ad80cdce0 con 0x7f4ad8160e30 2024-04-25T11:30:47.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.173+0000 7f4ace000640 1 -- 172.21.15.19:0/1269424893 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) ==== 100+0+11 (secure 0 0 0) 0x7f4ad005ed90 con 0x7f4ad8160e30 2024-04-25T11:30:47.163 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:30:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ab40744b0 msgr2=0x7f4ab40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ab40744b0 0x7f4ab40769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800a830 tx=0x7f4ac8008040 comp rx=0 tx=0).stop 2024-04-25T11:30:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad8160e30 msgr2=0x7f4ad814a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad8160e30 0x7f4ad814a330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c410 tx=0x7f4ad000c8d0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 shutdown_connections 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ab40744b0 0x7f4ab40769c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ad80b6980 0x7f4ad8149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ad8160e30 0x7f4ad814a330 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 --2- 172.21.15.19:0/1269424893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ad8070880 0x7f4ad814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 >> 172.21.15.19:0/1269424893 conn(0x7f4ad80d1d60 msgr2=0x7f4ad806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.176+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 shutdown_connections 2024-04-25T11:30:47.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.177+0000 7f4ad7200640 1 -- 172.21.15.19:0/1269424893 wait complete. 2024-04-25T11:30:47.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:47.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.482+0000 7fdd30a00640 1 Processor -- start 2024-04-25T11:30:47.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.482+0000 7fdd30a00640 1 -- start start 2024-04-25T11:30:47.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.485+0000 7fdd30a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdd2c144370 0x7fdd2c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.485+0000 7fdd30a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2c0d1230 con 0x7fdd2c06c5c0 2024-04-25T11:30:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.485+0000 7fdd30a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2c0d06b0 con 0x7fdd2c0b4ea0 2024-04-25T11:30:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.485+0000 7fdd30a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdd2c0d0990 con 0x7fdd2c144370 2024-04-25T11:30:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.485+0000 7fdd2b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdd2c06c5c0 0x7fdd2c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41610/0 (socket says 172.21.15.19:41610) 2024-04-25T11:30:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.485+0000 7fdd2b400640 1 -- 172.21.15.19:0/1259060071 learned_addr learned my addr 172.21.15.19:0/1259060071 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593372489 0 0) 0x7fdd2c0d1230 con 0x7fdd2c06c5c0 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd10003520 con 0x7fdd2c06c5c0 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2be00640 1 --2- 172.21.15.19:0/1259060071 >> v2:172.21.15.169:3300/0 conn(0x7fdd2c144370 0x7fdd2c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3603887630 0 0) 0x7fdd10003520 con 0x7fdd2c06c5c0 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 >> v2:172.21.15.169:3300/0 conn(0x7fdd2c144370 msgr2=0x7fdd2c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 --2- 172.21.15.19:0/1259060071 >> v2:172.21.15.169:3300/0 conn(0x7fdd2c144370 0x7fdd2c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 >> v1:172.21.15.169:6789/0 conn(0x7fdd2c0b4ea0 legacy=0x7fdd2c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c0d0c70 con 0x7fdd2c06c5c0 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fdd1c001670 con 0x7fdd2c06c5c0 2024-04-25T11:30:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2be00640 1 --2- 172.21.15.19:0/1259060071 >> v2:172.21.15.169:3300/0 conn(0x7fdd2c144370 0x7fdd2c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 >> v1:172.21.15.19:6789/0 conn(0x7fdd2c06c5c0 legacy=0x7fdd2c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd10004800 0x7fdd10004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd10005350 0x7fdd1000ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0d0990 con 0x7fdd100039b0 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1000c2e0 con 0x7fdd10005350 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.486+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1000c4e0 con 0x7fdd10004800 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2b400640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2aa00640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd10004800 0x7fdd10004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2be00640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd10005350 0x7fdd1000ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2b400640 1 -- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd10004800 msgr2=0x7fdd10004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2b400640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd10004800 0x7fdd10004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2b400640 1 -- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd10005350 msgr2=0x7fdd1000ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2b400640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd10005350 0x7fdd1000ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2be00640 1 -- 172.21.15.19:0/1259060071 reap_dead start 2024-04-25T11:30:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.487+0000 7fdd2b400640 1 -- 172.21.15.19:0/1259060071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c0d0c70 con 0x7fdd100039b0 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.488+0000 7fdd2b400640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd10003e20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c001050 tx=0x7fdd10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5748cdeafc6c270f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c007d10 con 0x7fdd100039b0 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c005af0 con 0x7fdd100039b0 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd2a000640 1 -- 172.21.15.19:0/1259060071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c007070 con 0x7fdd100039b0 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd30a00640 1 -- 172.21.15.19:0/1259060071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 msgr2=0x7fdd10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd30a00640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd10003e20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c001050 tx=0x7fdd10005830 comp rx=0 tx=0).stop 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd30a00640 1 -- 172.21.15.19:0/1259060071 shutdown_connections 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd30a00640 1 --2- 172.21.15.19:0/1259060071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd10003e20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd30a00640 1 -- 172.21.15.19:0/1259060071 >> 172.21.15.19:0/1259060071 conn(0x7fdd2c0d1720 msgr2=0x7fdd2c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd30a00640 1 -- 172.21.15.19:0/1259060071 shutdown_connections 2024-04-25T11:30:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.489+0000 7fdd30a00640 1 -- 172.21.15.19:0/1259060071 wait complete. 2024-04-25T11:30:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.492+0000 7fdd30a00640 1 Processor -- start 2024-04-25T11:30:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.492+0000 7fdd30a00640 1 -- start start 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.493+0000 7fdd30a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd2c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.493+0000 7fdd30a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2c142f70 0x7fdd2c1f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.493+0000 7fdd30a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2c15bd40 0x7fdd2c15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.493+0000 7fdd30a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0d0450 con 0x7fdd100039b0 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.493+0000 7fdd30a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2c0d02d0 con 0x7fdd2c142f70 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.493+0000 7fdd30a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2c147760 con 0x7fdd2c15bd40 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd2c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd2c158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47844/0 (socket says 172.21.15.19:47844) 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 -- 172.21.15.19:0/3887822684 learned_addr learned my addr 172.21.15.19:0/3887822684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2aa00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2c142f70 0x7fdd2c1f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2be00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2c15bd40 0x7fdd2c15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 -- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2c15bd40 msgr2=0x7fdd2c15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2c15bd40 0x7fdd2c15c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 -- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2c142f70 msgr2=0x7fdd2c1f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2c142f70 0x7fdd2c1f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.494+0000 7fdd2b400640 1 -- 172.21.15.19:0/3887822684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2c141ed0 con 0x7fdd100039b0 2024-04-25T11:30:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.495+0000 7fdd2b400640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd2c158de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c001050 tx=0x7fdd1c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.495+0000 7fdd23400640 1 -- 172.21.15.19:0/3887822684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c041020 con 0x7fdd100039b0 2024-04-25T11:30:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.495+0000 7fdd23400640 1 -- 172.21.15.19:0/3887822684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd1c007cf0 con 0x7fdd100039b0 2024-04-25T11:30:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.495+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2c1421b0 con 0x7fdd100039b0 2024-04-25T11:30:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.495+0000 7fdd23400640 1 -- 172.21.15.19:0/3887822684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c007070 con 0x7fdd100039b0 2024-04-25T11:30:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.495+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd2c142770 con 0x7fdd100039b0 2024-04-25T11:30:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.496+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2c050740 con 0x7fdd100039b0 2024-04-25T11:30:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.497+0000 7fdd23400640 1 -- 172.21.15.19:0/3887822684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd1c044020 con 0x7fdd100039b0 2024-04-25T11:30:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.498+0000 7fdd23400640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd0c074480 0x7fdd0c076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.499+0000 7fdd23400640 1 -- 172.21.15.19:0/3887822684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fdd1c0b59e0 con 0x7fdd100039b0 2024-04-25T11:30:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.499+0000 7fdd2aa00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd0c074480 0x7fdd0c076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.500+0000 7fdd2aa00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd0c074480 0x7fdd0c076990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0713f0 tx=0x7fdd18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.503+0000 7fdd23400640 1 -- 172.21.15.19:0/3887822684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd1c04b030 con 0x7fdd100039b0 2024-04-25T11:30:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.743+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd2c15df90 con 0x7fdd100039b0 2024-04-25T11:30:47.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.745+0000 7fdd23400640 1 -- 172.21.15.19:0/3887822684 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5185 (secure 0 0 0) 0x7fdd1c080100 con 0x7fdd100039b0 2024-04-25T11:30:47.734 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:47.735 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:44.021828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5991,"qdb_cluster":[5991]},"id":3}]} 2024-04-25T11:30:47.735 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd0c074480 msgr2=0x7fdd0c076990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd0c074480 0x7fdd0c076990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0713f0 tx=0x7fdd18009040 comp rx=0 tx=0).stop 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 msgr2=0x7fdd2c158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd2c158de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c001050 tx=0x7fdd1c000fe0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 shutdown_connections 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd0c074480 0x7fdd0c076990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd2c142f70 0x7fdd2c1f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd2c15bd40 0x7fdd2c15c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 --2- 172.21.15.19:0/3887822684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd100039b0 0x7fdd2c158de0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 >> 172.21.15.19:0/3887822684 conn(0x7fdd2c0d1720 msgr2=0x7fdd2c06d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:47.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 shutdown_connections 2024-04-25T11:30:47.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:47.747+0000 7fdd30a00640 1 -- 172.21.15.19:0/3887822684 wait complete. 2024-04-25T11:30:47.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:48.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.041+0000 7f7c31600640 1 Processor -- start 2024-04-25T11:30:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.041+0000 7f7c31600640 1 -- start start 2024-04-25T11:30:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.043+0000 7f7c31600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c2c070680 0x7f7c2c1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.043+0000 7f7c31600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7c2c0b5c90 0x7f7c2c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.043+0000 7f7c31600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c2c0b65e0 con 0x7f7c2c160360 2024-04-25T11:30:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.043+0000 7f7c31600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2c06d100 con 0x7f7c2c070680 2024-04-25T11:30:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.043+0000 7f7c31600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2c06c7d0 con 0x7f7c2c0b5c90 2024-04-25T11:30:48.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c2c070680 0x7f7c2c1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7c2c070680 0x7f7c2c1647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47858/0 (socket says 172.21.15.19:47858) 2024-04-25T11:30:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2aa00640 1 -- 172.21.15.19:0/618619296 learned_addr learned my addr 172.21.15.19:0/618619296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2b400640 1 --2- 172.21.15.19:0/618619296 >> v2:172.21.15.169:3300/0 conn(0x7f7c2c0b5c90 0x7f7c2c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2aa00640 1 -- 172.21.15.19:0/618619296 >> v2:172.21.15.169:3300/0 conn(0x7f7c2c0b5c90 msgr2=0x7f7c2c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2aa00640 1 --2- 172.21.15.19:0/618619296 >> v2:172.21.15.169:3300/0 conn(0x7f7c2c0b5c90 0x7f7c2c0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2aa00640 1 -- 172.21.15.19:0/618619296 >> v1:172.21.15.169:6789/0 conn(0x7f7c2c160360 legacy=0x7f7c2c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.044+0000 7f7c2aa00640 1 -- 172.21.15.19:0/618619296 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c06cab0 con 0x7f7c2c070680 2024-04-25T11:30:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.045+0000 7f7c2aa00640 1 --2- 172.21.15.19:0/618619296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c070680 0x7f7c2c1647f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0b11b0 tx=0x7f7c1c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375dac2f7030d052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.046+0000 7f7c29600640 1 -- 172.21.15.19:0/618619296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c037070 con 0x7f7c2c070680 2024-04-25T11:30:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.046+0000 7f7c29600640 1 -- 172.21.15.19:0/618619296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c02ae20 con 0x7f7c2c070680 2024-04-25T11:30:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.046+0000 7f7c29600640 1 -- 172.21.15.19:0/618619296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c032660 con 0x7f7c2c070680 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 -- 172.21.15.19:0/618619296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c070680 msgr2=0x7f7c2c1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 --2- 172.21.15.19:0/618619296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c070680 0x7f7c2c1647f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0b11b0 tx=0x7f7c1c02a110 comp rx=0 tx=0).stop 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 -- 172.21.15.19:0/618619296 shutdown_connections 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 --2- 172.21.15.19:0/618619296 >> v2:172.21.15.169:3300/0 conn(0x7f7c2c0b5c90 0x7f7c2c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 --2- 172.21.15.19:0/618619296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c070680 0x7f7c2c1647f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 -- 172.21.15.19:0/618619296 >> 172.21.15.19:0/618619296 conn(0x7f7c2c0d1ae0 msgr2=0x7f7c2c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 -- 172.21.15.19:0/618619296 shutdown_connections 2024-04-25T11:30:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.047+0000 7f7c31600640 1 -- 172.21.15.19:0/618619296 wait complete. 2024-04-25T11:30:48.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.050+0000 7f7c31600640 1 Processor -- start 2024-04-25T11:30:48.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.050+0000 7f7c31600640 1 -- start start 2024-04-25T11:30:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.052+0000 7f7c31600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c2c070680 0x7f7c2c147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.052+0000 7f7c31600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c2c0b5c90 0x7f7c2c163d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.052+0000 7f7c31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c160360 0x7f7c2c1642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.052+0000 7f7c31600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c0afbb0 con 0x7f7c2c160360 2024-04-25T11:30:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.052+0000 7f7c31600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c06fcc0 con 0x7f7c2c0b5c90 2024-04-25T11:30:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.052+0000 7f7c31600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2c0afd30 con 0x7f7c2c070680 2024-04-25T11:30:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c160360 0x7f7c2c1642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c160360 0x7f7c2c1642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47860/0 (socket says 172.21.15.19:47860) 2024-04-25T11:30:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 -- 172.21.15.19:0/1962996364 learned_addr learned my addr 172.21.15.19:0/1962996364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:48.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2aa00640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c2c070680 0x7f7c2c147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 -- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c2c070680 msgr2=0x7f7c2c147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c2c070680 0x7f7c2c147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 -- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c2c0b5c90 msgr2=0x7f7c2c163d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c2c0b5c90 0x7f7c2c163d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.053+0000 7f7c2b400640 1 -- 172.21.15.19:0/1962996364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c14df90 con 0x7f7c2c160360 2024-04-25T11:30:48.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.054+0000 7f7c2b400640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c160360 0x7f7c2c1642b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007db0 tx=0x7f7c2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.057+0000 7f7c22a00640 1 -- 172.21.15.19:0/1962996364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24014420 con 0x7f7c2c160360 2024-04-25T11:30:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.057+0000 7f7c22a00640 1 -- 172.21.15.19:0/1962996364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c24019070 con 0x7f7c2c160360 2024-04-25T11:30:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.057+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c1478c0 con 0x7f7c2c160360 2024-04-25T11:30:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.057+0000 7f7c22a00640 1 -- 172.21.15.19:0/1962996364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24014800 con 0x7f7c2c160360 2024-04-25T11:30:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.057+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c147e40 con 0x7f7c2c160360 2024-04-25T11:30:48.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.058+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2c0af080 con 0x7f7c2c160360 2024-04-25T11:30:48.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.059+0000 7f7c22a00640 1 -- 172.21.15.19:0/1962996364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c240149a0 con 0x7f7c2c160360 2024-04-25T11:30:48.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.060+0000 7f7c22a00640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c080744e0 0x7f7c080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.060+0000 7f7c22a00640 1 -- 172.21.15.19:0/1962996364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7f7c24095af0 con 0x7f7c2c160360 2024-04-25T11:30:48.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.061+0000 7f7c2aa00640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c080744e0 0x7f7c080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.062+0000 7f7c2aa00640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c080744e0 0x7f7c080769f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c034000 tx=0x7f7c1c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:48.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.065+0000 7f7c22a00640 1 -- 172.21.15.19:0/1962996364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c2c0af080 con 0x7f7c2c160360 2024-04-25T11:30:48.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.275+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c2c0d23b0 con 0x7f7c2c160360 2024-04-25T11:30:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.277+0000 7f7c22a00640 1 -- 172.21.15.19:0/1962996364 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16738 (secure 0 0 0) 0x7f7c2c0d23b0 con 0x7f7c2c160360 2024-04-25T11:30:48.267 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:48.267 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:38.619771+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:30:10.008986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T11:30:10.799803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3038680263":"2024-04-26T11:30:38.619698+0000","172.21.15.19:6834/3038680263":"2024-04-26T11:30:38.619698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:48.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.280+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c080744e0 msgr2=0x7f7c080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.280+0000 7f7c31600640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c080744e0 0x7f7c080769f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c034000 tx=0x7f7c1c02e000 comp rx=0 tx=0).stop 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.280+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c160360 msgr2=0x7f7c2c1642b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.280+0000 7f7c31600640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c160360 0x7f7c2c1642b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007db0 tx=0x7f7c2400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 shutdown_connections 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7c080744e0 0x7f7c080769f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c2c0b5c90 0x7f7c2c163d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7c2c160360 0x7f7c2c1642b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 --2- 172.21.15.19:0/1962996364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7c2c070680 0x7f7c2c147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 >> 172.21.15.19:0/1962996364 conn(0x7f7c2c0d1ae0 msgr2=0x7f7c2c0b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 shutdown_connections 2024-04-25T11:30:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.281+0000 7f7c31600640 1 -- 172.21.15.19:0/1962996364 wait complete. 2024-04-25T11:30:48.299 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:30:48.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.596+0000 7fed17000640 1 Processor -- start 2024-04-25T11:30:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.596+0000 7fed17000640 1 -- start start 2024-04-25T11:30:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.599+0000 7fed17000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed10160360 0x7fed101647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.599+0000 7fed17000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1006c7d0 con 0x7fed10070680 2024-04-25T11:30:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.599+0000 7fed17000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1006cab0 con 0x7fed100b5c90 2024-04-25T11:30:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.599+0000 7fed17000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fed1006d100 con 0x7fed10160360 2024-04-25T11:30:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.599+0000 7fed0f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed10160360 0x7fed101647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.599+0000 7fed0fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fed10070680 0x7fed100af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41620/0 (socket says 172.21.15.19:41620) 2024-04-25T11:30:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.599+0000 7fed0fe00640 1 -- 172.21.15.19:0/1440248761 learned_addr learned my addr 172.21.15.19:0/1440248761 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.600+0000 7fed0f400640 1 -- 172.21.15.19:0/1440248761 >> v1:172.21.15.169:6789/0 conn(0x7fed100b5c90 legacy=0x7fed100b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.600+0000 7fed0f400640 1 -- 172.21.15.19:0/1440248761 >> v1:172.21.15.19:6789/0 conn(0x7fed10070680 legacy=0x7fed100af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.600+0000 7fed0ea00640 1 -- 172.21.15.19:0/1440248761 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431464235 0 0) 0x7fed1006c7d0 con 0x7fed10070680 2024-04-25T11:30:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.600+0000 7fed0f400640 1 -- 172.21.15.19:0/1440248761 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1006cd90 con 0x7fed10160360 2024-04-25T11:30:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.601+0000 7fed0f400640 1 --2- 172.21.15.19:0/1440248761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed10160360 0x7fed101647f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fed1006d280 tx=0x7fecf802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc68db375e80371e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:48.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.601+0000 7fed0ea00640 1 -- 172.21.15.19:0/1440248761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8037070 con 0x7fed10160360 2024-04-25T11:30:48.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.601+0000 7fed0ea00640 1 -- 172.21.15.19:0/1440248761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf8032440 con 0x7fed10160360 2024-04-25T11:30:48.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.602+0000 7fed0ea00640 1 -- 172.21.15.19:0/1440248761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8032810 con 0x7fed10160360 2024-04-25T11:30:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.602+0000 7fed17000640 1 -- 172.21.15.19:0/1440248761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed10160360 msgr2=0x7fed101647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.602+0000 7fed17000640 1 --2- 172.21.15.19:0/1440248761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed10160360 0x7fed101647f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fed1006d280 tx=0x7fecf802a180 comp rx=0 tx=0).stop 2024-04-25T11:30:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.602+0000 7fed17000640 1 -- 172.21.15.19:0/1440248761 shutdown_connections 2024-04-25T11:30:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.602+0000 7fed17000640 1 --2- 172.21.15.19:0/1440248761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed10160360 0x7fed101647f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.602+0000 7fed17000640 1 -- 172.21.15.19:0/1440248761 >> 172.21.15.19:0/1440248761 conn(0x7fed100d1ae0 msgr2=0x7fed100d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.602+0000 7fed17000640 1 -- 172.21.15.19:0/1440248761 shutdown_connections 2024-04-25T11:30:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.603+0000 7fed17000640 1 -- 172.21.15.19:0/1440248761 wait complete. 2024-04-25T11:30:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.605+0000 7fed17000640 1 Processor -- start 2024-04-25T11:30:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.605+0000 7fed17000640 1 -- start start 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.606+0000 7fed17000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed10070680 0x7fed10151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.606+0000 7fed17000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed100b5c90 0x7fed100c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.606+0000 7fed17000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed100c8280 0x7fed100ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed17000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed100afd30 con 0x7fed100c8280 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed17000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed100afbb0 con 0x7fed100b5c90 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed17000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed1006d590 con 0x7fed10070680 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed15000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed100c8280 0x7fed100ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed15000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed100c8280 0x7fed100ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47874/0 (socket says 172.21.15.19:47874) 2024-04-25T11:30:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed15000640 1 -- 172.21.15.19:0/2698521924 learned_addr learned my addr 172.21.15.19:0/2698521924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed0fe00640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed10070680 0x7fed10151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.607+0000 7fed0f400640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed100b5c90 0x7fed100c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed15000640 1 -- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed10070680 msgr2=0x7fed10151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed15000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed10070680 0x7fed10151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed15000640 1 -- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed100b5c90 msgr2=0x7fed100c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed15000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed100b5c90 0x7fed100c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed15000640 1 -- 172.21.15.19:0/2698521924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed100c8b10 con 0x7fed100c8280 2024-04-25T11:30:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed15000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed100c8280 0x7fed100ca440 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c680 tx=0x7fed0800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed07e00640 1 -- 172.21.15.19:0/2698521924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08003b70 con 0x7fed100c8280 2024-04-25T11:30:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed07e00640 1 -- 172.21.15.19:0/2698521924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed08003d10 con 0x7fed100c8280 2024-04-25T11:30:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed100c8df0 con 0x7fed100c8280 2024-04-25T11:30:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed07e00640 1 -- 172.21.15.19:0/2698521924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08005310 con 0x7fed100c8280 2024-04-25T11:30:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.608+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed100c93b0 con 0x7fed100c8280 2024-04-25T11:30:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.609+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed100c8880 con 0x7fed100c8280 2024-04-25T11:30:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.611+0000 7fed07e00640 1 -- 172.21.15.19:0/2698521924 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed080054b0 con 0x7fed100c8280 2024-04-25T11:30:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.612+0000 7fed07e00640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecec0744b0 0x7fecec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.612+0000 7fed07e00640 1 -- 172.21.15.19:0/2698521924 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fed08095170 con 0x7fed100c8280 2024-04-25T11:30:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.613+0000 7fed0fe00640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecec0744b0 0x7fecec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.614+0000 7fed0fe00640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecec0744b0 0x7fecec0769c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fed00000f60 tx=0x7fed00007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.617+0000 7fed07e00640 1 -- 172.21.15.19:0/2698521924 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed0805f800 con 0x7fed100c8280 2024-04-25T11:30:48.786 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:30:48.797+0000 7f03c7200640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth (dversion lock) pv=8 v=4 ino=0x10000000001 state=1610612736 | inodepin=1 dirty=1 0x562e4df93400] 2024-04-25T11:30:48.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.869+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed10050740 con 0x7fed100c8280 2024-04-25T11:30:48.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.871+0000 7fed07e00640 1 -- 172.21.15.19:0/2698521924 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5185 (secure 0 0 0) 0x7fed0805f100 con 0x7fed100c8280 2024-04-25T11:30:48.861 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:48.861 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:44.021828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5991,"qdb_cluster":[5991]},"id":3}]} 2024-04-25T11:30:48.861 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-04-25T11:30:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecec0744b0 msgr2=0x7fecec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecec0744b0 0x7fecec0769c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fed00000f60 tx=0x7fed00007d00 comp rx=0 tx=0).stop 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed100c8280 msgr2=0x7fed100ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed100c8280 0x7fed100ca440 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c680 tx=0x7fed0800cb40 comp rx=0 tx=0).stop 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 shutdown_connections 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecec0744b0 0x7fecec0769c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed100b5c90 0x7fed100c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed100c8280 0x7fed100ca440 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 --2- 172.21.15.19:0/2698521924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed10070680 0x7fed10151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 >> 172.21.15.19:0/2698521924 conn(0x7fed100d1ae0 msgr2=0x7fed100b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 shutdown_connections 2024-04-25T11:30:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:48.874+0000 7fed17000640 1 -- 172.21.15.19:0/2698521924 wait complete. 2024-04-25T11:30:48.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:30:49.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.186+0000 7fdeb0e00640 1 Processor -- start 2024-04-25T11:30:49.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.186+0000 7fdeb0e00640 1 -- start start 2024-04-25T11:30:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.189+0000 7fdeb0e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdeac06c5c0 0x7fdeac0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.189+0000 7fdeb0e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeac0d1230 con 0x7fdeac144370 2024-04-25T11:30:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.189+0000 7fdeb0e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeac0d06b0 con 0x7fdeac0b4ea0 2024-04-25T11:30:49.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.189+0000 7fdeb0e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdeac0d0990 con 0x7fdeac06c5c0 2024-04-25T11:30:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.189+0000 7fdeaaa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdeac144370 0x7fdeac0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41634/0 (socket says 172.21.15.19:41634) 2024-04-25T11:30:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.189+0000 7fdeaaa00640 1 -- 172.21.15.19:0/799002541 learned_addr learned my addr 172.21.15.19:0/799002541 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:49.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.189+0000 7fdeab400640 1 --2- 172.21.15.19:0/799002541 >> v2:172.21.15.169:3300/0 conn(0x7fdeac06c5c0 0x7fdeac0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1944659739 0 0) 0x7fdeac0d1230 con 0x7fdeac144370 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde90003520 con 0x7fdeac144370 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3650818699 0 0) 0x7fde90003520 con 0x7fdeac144370 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 >> v2:172.21.15.169:3300/0 conn(0x7fdeac06c5c0 msgr2=0x7fdeac0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 --2- 172.21.15.19:0/799002541 >> v2:172.21.15.169:3300/0 conn(0x7fdeac06c5c0 0x7fdeac0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 >> v1:172.21.15.169:6789/0 conn(0x7fdeac0b4ea0 legacy=0x7fdeac15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac0d0c70 con 0x7fdeac144370 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fde940019f0 con 0x7fdeac144370 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.190+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 >> v1:172.21.15.19:6789/0 conn(0x7fdeac144370 legacy=0x7fdeac0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeaa000640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90003a00 0x7fde90003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeaa000640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90004800 0x7fde90004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeaa000640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde90005270 0x7fde9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac0d0990 con 0x7fde90003a00 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000c190 con 0x7fde90004800 2024-04-25T11:30:49.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fde9000c390 con 0x7fde90005270 2024-04-25T11:30:49.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeab400640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90003a00 0x7fde90003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeaaa00640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90004800 0x7fde90004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeab400640 1 -- 172.21.15.19:0/799002541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde90005270 msgr2=0x7fde9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeab400640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde90005270 0x7fde9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeab400640 1 -- 172.21.15.19:0/799002541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90004800 msgr2=0x7fde90004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeab400640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde90004800 0x7fde90004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeab400640 1 -- 172.21.15.19:0/799002541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde90003520 con 0x7fde90003a00 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.191+0000 7fdeabe00640 1 -- 172.21.15.19:0/799002541 reap_dead start 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.192+0000 7fdeab400640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90003a00 0x7fde90003e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009ac0 tx=0x7fde90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2a2630d5bde18c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.193+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c036070 con 0x7fde90003a00 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.193+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9c031400 con 0x7fde90003a00 2024-04-25T11:30:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.193+0000 7fdeaa000640 1 -- 172.21.15.19:0/799002541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c009690 con 0x7fde90003a00 2024-04-25T11:30:49.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.193+0000 7fdeb0e00640 1 -- 172.21.15.19:0/799002541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90003a00 msgr2=0x7fde90003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.194+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90003a00 0x7fde90003e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009ac0 tx=0x7fde90005830 comp rx=0 tx=0).stop 2024-04-25T11:30:49.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.194+0000 7fdeb0e00640 1 -- 172.21.15.19:0/799002541 shutdown_connections 2024-04-25T11:30:49.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.194+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/799002541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde90003a00 0x7fde90003e70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.194+0000 7fdeb0e00640 1 -- 172.21.15.19:0/799002541 >> 172.21.15.19:0/799002541 conn(0x7fdeac0d1720 msgr2=0x7fdeac0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:49.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.194+0000 7fdeb0e00640 1 -- 172.21.15.19:0/799002541 shutdown_connections 2024-04-25T11:30:49.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.194+0000 7fdeb0e00640 1 -- 172.21.15.19:0/799002541 wait complete. 2024-04-25T11:30:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.196+0000 7fdeb0e00640 1 Processor -- start 2024-04-25T11:30:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.197+0000 7fdeb0e00640 1 -- start start 2024-04-25T11:30:49.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeb0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeac142fc0 0x7fdeac140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeb0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeac1f77b0 0x7fdeac1f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeb0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeac1f8100 0x7fdeac158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeb0e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac0d0450 con 0x7fdeac142fc0 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeb0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac0d02d0 con 0x7fdeac1f77b0 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeb0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdeac147760 con 0x7fdeac1f8100 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeac142fc0 0x7fdeac140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeac142fc0 0x7fdeac140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47888/0 (socket says 172.21.15.19:47888) 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.198+0000 7fdeab400640 1 -- 172.21.15.19:0/1822098513 learned_addr learned my addr 172.21.15.19:0/1822098513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.199+0000 7fdeab400640 1 -- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeac1f8100 msgr2=0x7fdeac158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.199+0000 7fdeab400640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeac1f8100 0x7fdeac158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.199+0000 7fdeab400640 1 -- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeac1f77b0 msgr2=0x7fdeac1f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.199+0000 7fdeab400640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeac1f77b0 0x7fdeac1f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.199+0000 7fdeab400640 1 -- 172.21.15.19:0/1822098513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac156ab0 con 0x7fdeac142fc0 2024-04-25T11:30:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.199+0000 7fdeab400640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeac142fc0 0x7fdeac140230 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fde9c02ab30 tx=0x7fde9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.200+0000 7fdea3400640 1 -- 172.21.15.19:0/1822098513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c03f020 con 0x7fdeac142fc0 2024-04-25T11:30:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.200+0000 7fdea3400640 1 -- 172.21.15.19:0/1822098513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9c037040 con 0x7fdeac142fc0 2024-04-25T11:30:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.200+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeac156d90 con 0x7fdeac142fc0 2024-04-25T11:30:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.200+0000 7fdea3400640 1 -- 172.21.15.19:0/1822098513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c036070 con 0x7fdeac142fc0 2024-04-25T11:30:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.200+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeac157350 con 0x7fdeac142fc0 2024-04-25T11:30:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.201+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeac0d1450 con 0x7fdeac142fc0 2024-04-25T11:30:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.202+0000 7fdea3400640 1 -- 172.21.15.19:0/1822098513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde9c0317d0 con 0x7fdeac142fc0 2024-04-25T11:30:49.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.203+0000 7fdea3400640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde880744b0 0x7fde880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.204+0000 7fdea3400640 1 -- 172.21.15.19:0/1822098513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fde9c0b4840 con 0x7fdeac142fc0 2024-04-25T11:30:49.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.204+0000 7fdeaaa00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde880744b0 0x7fde880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.206+0000 7fdeaaa00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde880744b0 0x7fde880769c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde9400d210 tx=0x7fde94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:49.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.211+0000 7fdea3400640 1 -- 172.21.15.19:0/1822098513 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdeac0d1450 con 0x7fdeac142fc0 2024-04-25T11:30:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.459+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdeac050740 con 0x7fdeac142fc0 2024-04-25T11:30:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.461+0000 7fdea3400640 1 -- 172.21.15.19:0/1822098513 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5185 (secure 0 0 0) 0x7fde9c0044e0 con 0x7fdeac142fc0 2024-04-25T11:30:49.451 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:49.451 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":42,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:30:11.809876+0000","modified":"2024-04-25T11:30:44.021828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5991},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5991":{"gid":5991,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/3964766888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3964766888},{"type":"v1","addr":"172.21.15.19:6835","nonce":3964766888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5991,"qdb_cluster":[5991]},"id":3}]} 2024-04-25T11:30:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 42 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde880744b0 msgr2=0x7fde880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde880744b0 0x7fde880769c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fde9400d210 tx=0x7fde94009040 comp rx=0 tx=0).stop 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeac142fc0 msgr2=0x7fdeac140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeac142fc0 0x7fdeac140230 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fde9c02ab30 tx=0x7fde9c004730 comp rx=0 tx=0).stop 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 shutdown_connections 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde880744b0 0x7fde880769c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeac1f77b0 0x7fdeac1f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeac1f8100 0x7fdeac158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 --2- 172.21.15.19:0/1822098513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeac142fc0 0x7fdeac140230 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 >> 172.21.15.19:0/1822098513 conn(0x7fdeac0d1720 msgr2=0x7fdeac145020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 shutdown_connections 2024-04-25T11:30:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.464+0000 7fdeb0e00640 1 -- 172.21.15.19:0/1822098513 wait complete. 2024-04-25T11:30:49.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.771+0000 7fc13a000640 1 Processor -- start 2024-04-25T11:30:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.771+0000 7fc13a000640 1 -- start start 2024-04-25T11:30:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.773+0000 7fc13a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc134164860 0x7fc1340b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc13a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1340d44a0 0x7fc1340d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc13a000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc134161650 con 0x7fc1340b7540 2024-04-25T11:30:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc13a000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc1341643d0 con 0x7fc134164860 2024-04-25T11:30:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc13a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1340b5270 con 0x7fc1340d44a0 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc138c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc134164860 0x7fc1340b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc138c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc134164860 0x7fc1340b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47904/0 (socket says 172.21.15.19:47904) 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc138c00640 1 -- 172.21.15.19:0/4171058827 learned_addr learned my addr 172.21.15.19:0/4171058827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.774+0000 7fc139600640 1 --2- 172.21.15.19:0/4171058827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1340d44a0 0x7fc1340d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.775+0000 7fc138c00640 1 -- 172.21.15.19:0/4171058827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1340d44a0 msgr2=0x7fc1340d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.775+0000 7fc138c00640 1 --2- 172.21.15.19:0/4171058827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1340d44a0 0x7fc1340d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.775+0000 7fc138c00640 1 -- 172.21.15.19:0/4171058827 >> v1:172.21.15.169:6789/0 conn(0x7fc1340b7540 legacy=0x7fc1340b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.775+0000 7fc138c00640 1 -- 172.21.15.19:0/4171058827 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc134161370 con 0x7fc134164860 2024-04-25T11:30:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.776+0000 7fc138c00640 1 --2- 172.21.15.19:0/4171058827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc134164860 0x7fc1340b4d30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc124009870 tx=0x7fc12402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921b76ac49acf4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.777+0000 7fc133400640 1 -- 172.21.15.19:0/4171058827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124037070 con 0x7fc134164860 2024-04-25T11:30:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.777+0000 7fc133400640 1 -- 172.21.15.19:0/4171058827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc124032440 con 0x7fc134164860 2024-04-25T11:30:49.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.777+0000 7fc133400640 1 -- 172.21.15.19:0/4171058827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124032810 con 0x7fc134164860 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 -- 172.21.15.19:0/4171058827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc134164860 msgr2=0x7fc1340b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 --2- 172.21.15.19:0/4171058827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc134164860 0x7fc1340b4d30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc124009870 tx=0x7fc12402a210 comp rx=0 tx=0).stop 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 -- 172.21.15.19:0/4171058827 shutdown_connections 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 --2- 172.21.15.19:0/4171058827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1340d44a0 0x7fc1340d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 --2- 172.21.15.19:0/4171058827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc134164860 0x7fc1340b4d30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 -- 172.21.15.19:0/4171058827 >> 172.21.15.19:0/4171058827 conn(0x7fc1340d2720 msgr2=0x7fc1340d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 -- 172.21.15.19:0/4171058827 shutdown_connections 2024-04-25T11:30:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.778+0000 7fc13a000640 1 -- 172.21.15.19:0/4171058827 wait complete. 2024-04-25T11:30:49.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.781+0000 7fc13a000640 1 Processor -- start 2024-04-25T11:30:49.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.781+0000 7fc13a000640 1 -- start start 2024-04-25T11:30:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.782+0000 7fc13a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340b7540 0x7fc1341ec020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.782+0000 7fc13a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1340d44a0 0x7fc1340cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc13a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc134164860 0x7fc13414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc13a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1340d4df0 con 0x7fc1340b7540 2024-04-25T11:30:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc13a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc134164060 con 0x7fc1340d44a0 2024-04-25T11:30:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc13a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1340d4f70 con 0x7fc134164860 2024-04-25T11:30:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340b7540 0x7fc1341ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340b7540 0x7fc1341ec020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47906/0 (socket says 172.21.15.19:47906) 2024-04-25T11:30:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 -- 172.21.15.19:0/1252253191 learned_addr learned my addr 172.21.15.19:0/1252253191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc139600640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc134164860 0x7fc13414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc133e00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1340d44a0 0x7fc1340cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 -- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc134164860 msgr2=0x7fc13414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc134164860 0x7fc13414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 -- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1340d44a0 msgr2=0x7fc1340cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1340d44a0 0x7fc1340cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc138c00640 1 -- 172.21.15.19:0/1252253191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1340c9650 con 0x7fc1340b7540 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc139600640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc134164860 0x7fc13414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.783+0000 7fc133e00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1340d44a0 0x7fc1340cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.784+0000 7fc138c00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340b7540 0x7fc1341ec020 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc124000c00 tx=0x7fc1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.784+0000 7fc130c00640 1 -- 172.21.15.19:0/1252253191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124041070 con 0x7fc1340b7540 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.784+0000 7fc130c00640 1 -- 172.21.15.19:0/1252253191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12403c870 con 0x7fc1340b7540 2024-04-25T11:30:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.784+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1340c9930 con 0x7fc1340b7540 2024-04-25T11:30:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.784+0000 7fc130c00640 1 -- 172.21.15.19:0/1252253191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124037040 con 0x7fc1340b7540 2024-04-25T11:30:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.784+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1340c9ef0 con 0x7fc1340b7540 2024-04-25T11:30:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.786+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc134050740 con 0x7fc1340b7540 2024-04-25T11:30:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.787+0000 7fc130c00640 1 -- 172.21.15.19:0/1252253191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc124044070 con 0x7fc1340b7540 2024-04-25T11:30:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.789+0000 7fc130c00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1100744e0 0x7fc1100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:49.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.789+0000 7fc130c00640 1 -- 172.21.15.19:0/1252253191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fc1240b5ce0 con 0x7fc1340b7540 2024-04-25T11:30:49.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.789+0000 7fc133e00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1100744e0 0x7fc1100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:49.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.791+0000 7fc133e00640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1100744e0 0x7fc1100769f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc1340b73a0 tx=0x7fc11c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.794+0000 7fc130c00640 1 -- 172.21.15.19:0/1252253191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc124080400 con 0x7fc1340b7540 2024-04-25T11:30:49.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:49.998+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1340b5050 con 0x7fc1340b7540 2024-04-25T11:30:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.001+0000 7fc130c00640 1 -- 172.21.15.19:0/1252253191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16738 (secure 0 0 0) 0x7fc12407fd00 con 0x7fc1340b7540 2024-04-25T11:30:49.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:49.991 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:38.619771+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:30:10.008986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T11:30:10.799803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3038680263":"2024-04-26T11:30:38.619698+0000","172.21.15.19:6834/3038680263":"2024-04-26T11:30:38.619698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.003+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1100744e0 msgr2=0x7fc1100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.003+0000 7fc13a000640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1100744e0 0x7fc1100769f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc1340b73a0 tx=0x7fc11c008040 comp rx=0 tx=0).stop 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.003+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340b7540 msgr2=0x7fc1341ec020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340b7540 0x7fc1341ec020 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc124000c00 tx=0x7fc1240046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 shutdown_connections 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1100744e0 0x7fc1100769f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1340d44a0 0x7fc1340cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc134164860 0x7fc13414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 --2- 172.21.15.19:0/1252253191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1340b7540 0x7fc1341ec020 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 >> 172.21.15.19:0/1252253191 conn(0x7fc1340d2720 msgr2=0x7fc13415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 shutdown_connections 2024-04-25T11:30:49.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.004+0000 7fc13a000640 1 -- 172.21.15.19:0/1252253191 wait complete. 2024-04-25T11:30:50.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:30:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.298+0000 7fb89f200640 1 Processor -- start 2024-04-25T11:30:50.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.298+0000 7fb89f200640 1 -- start start 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb898143170 0x7fb898160950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb898160e90 0x7fb8981612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8980cdb20 con 0x7fb89806f940 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb898146970 con 0x7fb898160e90 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb8980b4ee0 con 0x7fb898143170 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb898160e90 0x7fb8981612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb89806f940 0x7fb89806fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41650/0 (socket says 172.21.15.19:41650) 2024-04-25T11:30:50.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.301+0000 7fb89e800640 1 -- 172.21.15.19:0/3110461475 learned_addr learned my addr 172.21.15.19:0/3110461475 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.302+0000 7fb89de00640 1 --2- 172.21.15.19:0/3110461475 >> v2:172.21.15.169:3300/0 conn(0x7fb898143170 0x7fb898160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.302+0000 7fb89d400640 1 -- 172.21.15.19:0/3110461475 >> v2:172.21.15.169:3300/0 conn(0x7fb898143170 msgr2=0x7fb898160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.302+0000 7fb89d400640 1 --2- 172.21.15.19:0/3110461475 >> v2:172.21.15.169:3300/0 conn(0x7fb898143170 0x7fb898160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.302+0000 7fb89d400640 1 -- 172.21.15.19:0/3110461475 >> v1:172.21.15.19:6789/0 conn(0x7fb89806f940 legacy=0x7fb89806fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.302+0000 7fb89d400640 1 -- 172.21.15.19:0/3110461475 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8980cde00 con 0x7fb898160e90 2024-04-25T11:30:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.302+0000 7fb89ca00640 1 -- 172.21.15.19:0/3110461475 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726952720 0 0) 0x7fb8980cdb20 con 0x7fb89806f940 2024-04-25T11:30:50.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.303+0000 7fb89d400640 1 --2- 172.21.15.19:0/3110461475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898160e90 0x7fb8981612a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb8980d26e0 tx=0x7fb88c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e4d1b05f7403cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.303+0000 7fb89ca00640 1 -- 172.21.15.19:0/3110461475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c037070 con 0x7fb898160e90 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.303+0000 7fb89ca00640 1 -- 172.21.15.19:0/3110461475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88c032440 con 0x7fb898160e90 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.303+0000 7fb89ca00640 1 -- 172.21.15.19:0/3110461475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c02ae60 con 0x7fb898160e90 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 -- 172.21.15.19:0/3110461475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898160e90 msgr2=0x7fb8981612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 --2- 172.21.15.19:0/3110461475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898160e90 0x7fb8981612a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb8980d26e0 tx=0x7fb88c02a110 comp rx=0 tx=0).stop 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 -- 172.21.15.19:0/3110461475 shutdown_connections 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 --2- 172.21.15.19:0/3110461475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb898160e90 0x7fb8981612a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 --2- 172.21.15.19:0/3110461475 >> v2:172.21.15.169:3300/0 conn(0x7fb898143170 0x7fb898160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 -- 172.21.15.19:0/3110461475 >> 172.21.15.19:0/3110461475 conn(0x7fb8980d1b40 msgr2=0x7fb8980d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 -- 172.21.15.19:0/3110461475 shutdown_connections 2024-04-25T11:30:50.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.304+0000 7fb89f200640 1 -- 172.21.15.19:0/3110461475 wait complete. 2024-04-25T11:30:50.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.307+0000 7fb89f200640 1 Processor -- start 2024-04-25T11:30:50.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.307+0000 7fb89f200640 1 -- start start 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89806f940 0x7fb898157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb898143170 0x7fb8981eab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb898160e90 0x7fb8981eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981617e0 con 0x7fb89806f940 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb89806f440 con 0x7fb898160e90 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb898143580 con 0x7fb898143170 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89806f940 0x7fb898157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89806f940 0x7fb898157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47924/0 (socket says 172.21.15.19:47924) 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89de00640 1 -- 172.21.15.19:0/3706121541 learned_addr learned my addr 172.21.15.19:0/3706121541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.309+0000 7fb89d400640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb898143170 0x7fb8981eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89e800640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb898160e90 0x7fb8981eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89de00640 1 -- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb898143170 msgr2=0x7fb8981eab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89de00640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb898143170 0x7fb8981eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89de00640 1 -- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb898160e90 msgr2=0x7fb8981eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89de00640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb898160e90 0x7fb8981eb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89de00640 1 -- 172.21.15.19:0/3706121541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb898157c70 con 0x7fb89806f940 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89de00640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89806f940 0x7fb898157730 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb88400d620 tx=0x7fb88400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb892000640 1 -- 172.21.15.19:0/3706121541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88400b620 con 0x7fb89806f940 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb892000640 1 -- 172.21.15.19:0/3706121541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb884019070 con 0x7fb89806f940 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8981391a0 con 0x7fb89806f940 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb892000640 1 -- 172.21.15.19:0/3706121541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88400ba00 con 0x7fb89806f940 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.310+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8981442d0 con 0x7fb89806f940 2024-04-25T11:30:50.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.312+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb898050740 con 0x7fb89806f940 2024-04-25T11:30:50.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.313+0000 7fb892000640 1 -- 172.21.15.19:0/3706121541 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb88400bba0 con 0x7fb89806f940 2024-04-25T11:30:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.314+0000 7fb892000640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8780744e0 0x7fb8780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.314+0000 7fb892000640 1 -- 172.21.15.19:0/3706121541 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5795+0+0 (secure 0 0 0) 0x7fb8840957b0 con 0x7fb89806f940 2024-04-25T11:30:50.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.315+0000 7fb89d400640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8780744e0 0x7fb8780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.316+0000 7fb89d400640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8780744e0 0x7fb8780769f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb88c034000 tx=0x7fb88c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:50.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.318+0000 7fb892000640 1 -- 172.21.15.19:0/3706121541 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb88405fe40 con 0x7fb89806f940 2024-04-25T11:30:50.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.573+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb8981615c0 con 0x7fb89806f940 2024-04-25T11:30:50.564 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:30:50.574+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:30:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.578+0000 7fb892000640 1 -- 172.21.15.19:0/3706121541 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7fb88405f740 con 0x7fb89806f940 2024-04-25T11:30:50.568 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:30:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.581+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8780744e0 msgr2=0x7fb8780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.581+0000 7fb89f200640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8780744e0 0x7fb8780769f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb88c034000 tx=0x7fb88c02e000 comp rx=0 tx=0).stop 2024-04-25T11:30:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.581+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89806f940 msgr2=0x7fb898157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.581+0000 7fb89f200640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89806f940 0x7fb898157730 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb88400d620 tx=0x7fb88400dae0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 shutdown_connections 2024-04-25T11:30:50.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8780744e0 0x7fb8780769f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb898160e90 0x7fb8981eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb898143170 0x7fb8981eab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 --2- 172.21.15.19:0/3706121541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb89806f940 0x7fb898157730 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 >> 172.21.15.19:0/3706121541 conn(0x7fb8980d1b40 msgr2=0x7fb8980d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 shutdown_connections 2024-04-25T11:30:50.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.582+0000 7fb89f200640 1 -- 172.21.15.19:0/3706121541 wait complete. 2024-04-25T11:30:50.582 INFO:tasks.ceph.mds.a.smithi019.stderr: -299> 2024-04-25T11:30:44.644+0000 7f03c5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:30:50.582 INFO:tasks.ceph.mds.a.smithi019.stderr: -110> 2024-04-25T11:30:48.797+0000 7f03c7200640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth (dversion lock) pv=8 v=4 ino=0x10000000001 state=1610612736 | inodepin=1 dirty=1 0x562e4df93400] 2024-04-25T11:30:50.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:30:50.722 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:30:50.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.875+0000 7f7e2ce00640 1 Processor -- start 2024-04-25T11:30:50.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.876+0000 7f7e2ce00640 1 -- start start 2024-04-25T11:30:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.877+0000 7f7e2ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e280b1a70 0x7f7e281644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.877+0000 7f7e2ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e28164b40 con 0x7f7e2809b9e0 2024-04-25T11:30:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.877+0000 7f7e2ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e280b2790 con 0x7f7e2806d3b0 2024-04-25T11:30:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.877+0000 7f7e2ce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7e280b2a70 con 0x7f7e280b1a70 2024-04-25T11:30:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.878+0000 7f7e26a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7e2809b9e0 0x7f7e280af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41658/0 (socket says 172.21.15.19:41658) 2024-04-25T11:30:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.878+0000 7f7e26a00640 1 -- 172.21.15.19:0/1351784160 learned_addr learned my addr 172.21.15.19:0/1351784160 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.878+0000 7f7e27e00640 1 --2- 172.21.15.19:0/1351784160 >> v2:172.21.15.19:3300/0 conn(0x7f7e280b1a70 0x7f7e281644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.878+0000 7f7e26000640 1 -- 172.21.15.19:0/1351784160 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3021299410 0 0) 0x7f7e28164b40 con 0x7f7e2809b9e0 2024-04-25T11:30:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.878+0000 7f7e26000640 1 -- 172.21.15.19:0/1351784160 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e0c003520 con 0x7f7e2809b9e0 2024-04-25T11:30:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.879+0000 7f7e26000640 1 -- 172.21.15.19:0/1351784160 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3494658383 0 0) 0x7f7e0c003520 con 0x7f7e2809b9e0 2024-04-25T11:30:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.879+0000 7f7e27e00640 1 -- 172.21.15.19:0/1351784160 >> v1:172.21.15.169:6789/0 conn(0x7f7e2806d3b0 legacy=0x7f7e28163dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.879+0000 7f7e27e00640 1 -- 172.21.15.19:0/1351784160 >> v1:172.21.15.19:6789/0 conn(0x7f7e2809b9e0 legacy=0x7f7e280af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.879+0000 7f7e27e00640 1 -- 172.21.15.19:0/1351784160 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e28160590 con 0x7f7e280b1a70 2024-04-25T11:30:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.880+0000 7f7e27e00640 1 --2- 172.21.15.19:0/1351784160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e280b1a70 0x7f7e281644d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7e200097a0 tx=0x7f7e2002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bd5536795b5b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.880+0000 7f7e26000640 1 -- 172.21.15.19:0/1351784160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20037070 con 0x7f7e280b1a70 2024-04-25T11:30:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.880+0000 7f7e26000640 1 -- 172.21.15.19:0/1351784160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2002abc0 con 0x7f7e280b1a70 2024-04-25T11:30:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.880+0000 7f7e26000640 1 -- 172.21.15.19:0/1351784160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10001670 con 0x7f7e280b1a70 2024-04-25T11:30:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.881+0000 7f7e2ce00640 1 -- 172.21.15.19:0/1351784160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e280b1a70 msgr2=0x7f7e281644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.881+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/1351784160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e280b1a70 0x7f7e281644d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7e200097a0 tx=0x7f7e2002a130 comp rx=0 tx=0).stop 2024-04-25T11:30:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.881+0000 7f7e2ce00640 1 -- 172.21.15.19:0/1351784160 shutdown_connections 2024-04-25T11:30:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.881+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/1351784160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e280b1a70 0x7f7e281644d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.881+0000 7f7e2ce00640 1 -- 172.21.15.19:0/1351784160 >> 172.21.15.19:0/1351784160 conn(0x7f7e280739c0 msgr2=0x7f7e280b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.881+0000 7f7e2ce00640 1 -- 172.21.15.19:0/1351784160 shutdown_connections 2024-04-25T11:30:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.881+0000 7f7e2ce00640 1 -- 172.21.15.19:0/1351784160 wait complete. 2024-04-25T11:30:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.884+0000 7f7e2ce00640 1 Processor -- start 2024-04-25T11:30:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.884+0000 7f7e2ce00640 1 -- start start 2024-04-25T11:30:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.885+0000 7f7e2ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2809b9e0 0x7f7e2815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.885+0000 7f7e2ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e280b1a70 0x7f7e28156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.885+0000 7f7e2ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28151690 0x7f7e28151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.885+0000 7f7e2ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e280afb00 con 0x7f7e2809b9e0 2024-04-25T11:30:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.885+0000 7f7e2ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e280af980 con 0x7f7e28151690 2024-04-25T11:30:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.885+0000 7f7e2ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e280afc80 con 0x7f7e280b1a70 2024-04-25T11:30:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2809b9e0 0x7f7e2815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2809b9e0 0x7f7e2815e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47954/0 (socket says 172.21.15.19:47954) 2024-04-25T11:30:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 -- 172.21.15.19:0/2622744145 learned_addr learned my addr 172.21.15.19:0/2622744145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 -- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e280b1a70 msgr2=0x7f7e28156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e280b1a70 0x7f7e28156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 -- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28151690 msgr2=0x7f7e28151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28151690 0x7f7e28151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.886+0000 7f7e27400640 1 -- 172.21.15.19:0/2622744145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e28151fe0 con 0x7f7e2809b9e0 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.887+0000 7f7e27400640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2809b9e0 0x7f7e2815e620 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c6b0 tx=0x7f7e1800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.887+0000 7f7e1f400640 1 -- 172.21.15.19:0/2622744145 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18016020 con 0x7f7e2809b9e0 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.887+0000 7f7e1f400640 1 -- 172.21.15.19:0/2622744145 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e18006d50 con 0x7f7e2809b9e0 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.887+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e28161f50 con 0x7f7e2809b9e0 2024-04-25T11:30:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.887+0000 7f7e1f400640 1 -- 172.21.15.19:0/2622744145 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18003bb0 con 0x7f7e2809b9e0 2024-04-25T11:30:50.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.887+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e28162510 con 0x7f7e2809b9e0 2024-04-25T11:30:50.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.889+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e28076c60 con 0x7f7e2809b9e0 2024-04-25T11:30:50.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.890+0000 7f7e1f400640 1 -- 172.21.15.19:0/2622744145 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e180050d0 con 0x7f7e2809b9e0 2024-04-25T11:30:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.891+0000 7f7e1f400640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e040744b0 0x7f7e040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.891+0000 7f7e1f400640 1 -- 172.21.15.19:0/2622744145 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5881+0+0 (secure 0 0 0) 0x7f7e18094ea0 con 0x7f7e2809b9e0 2024-04-25T11:30:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.892+0000 7f7e26a00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e040744b0 0x7f7e040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:50.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.893+0000 7f7e26a00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e040744b0 0x7f7e040769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e28163be0 tx=0x7f7e100010a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:50.896+0000 7f7e1f400640 1 -- 172.21.15.19:0/2622744145 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e18020020 con 0x7f7e2809b9e0 2024-04-25T11:30:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.144+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7e280b1850 con 0x7f7e2809b9e0 2024-04-25T11:30:51.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.582+0000 7f7e1f400640 1 -- 172.21.15.19:0/2622744145 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7f7e1805f600 con 0x7f7e2809b9e0 2024-04-25T11:30:51.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.586+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e040744b0 msgr2=0x7f7e040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:51.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.586+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e040744b0 0x7f7e040769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e28163be0 tx=0x7f7e100010a0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.586+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2809b9e0 msgr2=0x7f7e2815e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:51.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.586+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2809b9e0 0x7f7e2815e620 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c6b0 tx=0x7f7e1800cb70 comp rx=0 tx=0).stop 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 shutdown_connections 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e040744b0 0x7f7e040769c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28151690 0x7f7e28151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e280b1a70 0x7f7e28156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 --2- 172.21.15.19:0/2622744145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e2809b9e0 0x7f7e2815e620 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 >> 172.21.15.19:0/2622744145 conn(0x7f7e280739c0 msgr2=0x7f7e28158650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 shutdown_connections 2024-04-25T11:30:51.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.587+0000 7f7e2ce00640 1 -- 172.21.15.19:0/2622744145 wait complete. 2024-04-25T11:30:51.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:30:51.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.907+0000 7fb94ce00640 1 Processor -- start 2024-04-25T11:30:51.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.907+0000 7fb94ce00640 1 -- start start 2024-04-25T11:30:51.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.909+0000 7fb94ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815e160 0x7fb9480ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.909+0000 7fb94ce00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb94806f370 0x7fb94806f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.909+0000 7fb94ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb94806fcc0 con 0x7fb94806cf40 2024-04-25T11:30:51.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.909+0000 7fb94ce00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb94806cab0 con 0x7fb94806f370 2024-04-25T11:30:51.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.909+0000 7fb94ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9480714a0 con 0x7fb94815e160 2024-04-25T11:30:51.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.910+0000 7fb947e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb94806cf40 0x7fb94806d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41664/0 (socket says 172.21.15.19:41664) 2024-04-25T11:30:51.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.910+0000 7fb947e00640 1 -- 172.21.15.19:0/2241910269 learned_addr learned my addr 172.21.15.19:0/2241910269 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:51.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.910+0000 7fb947400640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815e160 0x7fb9480ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.910+0000 7fb946a00640 1 --2- 172.21.15.19:0/2241910269 >> v2:172.21.15.169:3300/0 conn(0x7fb94806f370 0x7fb94806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3823395891 0 0) 0x7fb940001490 con 0x7fb94806cf40 2024-04-25T11:30:51.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb92c003520 con 0x7fb94806cf40 2024-04-25T11:30:51.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 769403630 0 0) 0x7fb92c003520 con 0x7fb94806cf40 2024-04-25T11:30:51.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815e160 msgr2=0x7fb9480ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:51.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815e160 0x7fb9480ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 >> v2:172.21.15.169:3300/0 conn(0x7fb94806f370 msgr2=0x7fb94806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:51.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 --2- 172.21.15.19:0/2241910269 >> v2:172.21.15.169:3300/0 conn(0x7fb94806f370 0x7fb94806f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9480cd440 con 0x7fb94806cf40 2024-04-25T11:30:51.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb940001bc0 con 0x7fb94806cf40 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 >> v1:172.21.15.19:6789/0 conn(0x7fb94806cf40 legacy=0x7fb94806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c0039b0 0x7fb92c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c004800 0x7fb92c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb92c005150 0x7fb92c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb94806cab0 con 0x7fb92c005150 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9480714a0 con 0x7fb92c0039b0 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.911+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c00bf00 con 0x7fb92c004800 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb947e00640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb92c005150 0x7fb92c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb947400640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c0039b0 0x7fb92c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb946a00640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c004800 0x7fb92c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb947e00640 1 -- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c004800 msgr2=0x7fb92c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:51.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb947e00640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c004800 0x7fb92c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb947e00640 1 -- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c0039b0 msgr2=0x7fb92c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:51.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb947e00640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c0039b0 0x7fb92c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb947e00640 1 -- 172.21.15.19:0/2241910269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c003520 con 0x7fb92c005150 2024-04-25T11:30:51.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.912+0000 7fb946a00640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c004800 0x7fb92c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.913+0000 7fb947e00640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb92c005150 0x7fb92c00b9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb9480cc4a0 tx=0x7fb92c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5a87c30fa81f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.914+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94000a690 con 0x7fb92c005150 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.914+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94002f070 con 0x7fb92c005150 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.914+0000 7fb946000640 1 -- 172.21.15.19:0/2241910269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940034470 con 0x7fb92c005150 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.914+0000 7fb94ce00640 1 -- 172.21.15.19:0/2241910269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb92c005150 msgr2=0x7fb92c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.914+0000 7fb94ce00640 1 --2- 172.21.15.19:0/2241910269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb92c005150 0x7fb92c00b9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb9480cc4a0 tx=0x7fb92c005830 comp rx=0 tx=0).stop 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.915+0000 7fb947e00640 1 -- 172.21.15.19:0/2241910269 reap_dead start 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.915+0000 7fb94ce00640 1 -- 172.21.15.19:0/2241910269 shutdown_connections 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.915+0000 7fb94ce00640 1 -- 172.21.15.19:0/2241910269 >> 172.21.15.19:0/2241910269 conn(0x7fb9480ce3b0 msgr2=0x7fb9480ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:51.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.915+0000 7fb94ce00640 1 -- 172.21.15.19:0/2241910269 shutdown_connections 2024-04-25T11:30:51.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.915+0000 7fb94ce00640 1 -- 172.21.15.19:0/2241910269 wait complete. 2024-04-25T11:30:51.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.917+0000 7fb94ce00640 1 Processor -- start 2024-04-25T11:30:51.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.918+0000 7fb94ce00640 1 -- start start 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.919+0000 7fb94ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948156e90 0x7fb94815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.919+0000 7fb94ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9481f5a70 0x7fb94815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.919+0000 7fb94ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815c610 0x7fb94814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.919+0000 7fb94ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb94806c740 con 0x7fb948156e90 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.919+0000 7fb94ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9480cdec0 con 0x7fb9481f5a70 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.919+0000 7fb94ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9480b4820 con 0x7fb94815c610 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948156e90 0x7fb94815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948156e90 0x7fb94815a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47974/0 (socket says 172.21.15.19:47974) 2024-04-25T11:30:51.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 -- 172.21.15.19:0/1350526639 learned_addr learned my addr 172.21.15.19:0/1350526639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb946a00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9481f5a70 0x7fb94815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947e00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815c610 0x7fb94814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 -- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815c610 msgr2=0x7fb94814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815c610 0x7fb94814ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 -- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9481f5a70 msgr2=0x7fb94815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9481f5a70 0x7fb94815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:51.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.920+0000 7fb947400640 1 -- 172.21.15.19:0/1350526639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94815ca20 con 0x7fb948156e90 2024-04-25T11:30:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.921+0000 7fb947400640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948156e90 0x7fb94815a310 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb938011560 tx=0x7fb938009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.921+0000 7fb93f400640 1 -- 172.21.15.19:0/1350526639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938015930 con 0x7fb948156e90 2024-04-25T11:30:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.921+0000 7fb93f400640 1 -- 172.21.15.19:0/1350526639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb938015ad0 con 0x7fb948156e90 2024-04-25T11:30:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.921+0000 7fb93f400640 1 -- 172.21.15.19:0/1350526639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93801d400 con 0x7fb948156e90 2024-04-25T11:30:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.921+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94814f160 con 0x7fb948156e90 2024-04-25T11:30:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.921+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb948157440 con 0x7fb948156e90 2024-04-25T11:30:51.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.922+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948076c60 con 0x7fb948156e90 2024-04-25T11:30:51.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.924+0000 7fb93f400640 1 -- 172.21.15.19:0/1350526639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb93801d5a0 con 0x7fb948156e90 2024-04-25T11:30:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.925+0000 7fb93f400640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb928074500 0x7fb928076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.926+0000 7fb93f400640 1 -- 172.21.15.19:0/1350526639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5881+0+0 (secure 0 0 0) 0x7fb938095b80 con 0x7fb948156e90 2024-04-25T11:30:51.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.926+0000 7fb946a00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb928074500 0x7fb928076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:51.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.928+0000 7fb946a00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb928074500 0x7fb928076a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9480a2da0 tx=0x7fb934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:51.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:51.931+0000 7fb93f400640 1 -- 172.21.15.19:0/1350526639 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb94815ca20 con 0x7fb948156e90 2024-04-25T11:30:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:52.135+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb9480d1260 con 0x7fb948156e90 2024-04-25T11:30:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.101+0000 7fb93f400640 1 -- 172.21.15.19:0/1350526639 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7fb938019030 con 0x7fb948156e90 2024-04-25T11:30:53.091 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:30:53.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb928074500 msgr2=0x7fb928076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:53.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb928074500 0x7fb928076a10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9480a2da0 tx=0x7fb934008040 comp rx=0 tx=0).stop 2024-04-25T11:30:53.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948156e90 msgr2=0x7fb94815a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948156e90 0x7fb94815a310 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb938011560 tx=0x7fb938009f90 comp rx=0 tx=0).stop 2024-04-25T11:30:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 shutdown_connections 2024-04-25T11:30:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb928074500 0x7fb928076a10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9481f5a70 0x7fb94815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb94815c610 0x7fb94814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 --2- 172.21.15.19:0/1350526639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb948156e90 0x7fb94815a310 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 >> 172.21.15.19:0/1350526639 conn(0x7fb9480ce3b0 msgr2=0x7fb9480b6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.109+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 shutdown_connections 2024-04-25T11:30:53.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.110+0000 7fb94ce00640 1 -- 172.21.15.19:0/1350526639 wait complete. 2024-04-25T11:30:53.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:30:53.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.428+0000 7f9757e00640 1 Processor -- start 2024-04-25T11:30:53.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.428+0000 7f9757e00640 1 -- start start 2024-04-25T11:30:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.430+0000 7f9757e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975815e160 0x7f97580ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.430+0000 7f9757e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f975806f370 0x7f975806f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.430+0000 7f9757e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975806fcc0 con 0x7f975806cf40 2024-04-25T11:30:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.430+0000 7f9757e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f975806cab0 con 0x7f975806f370 2024-04-25T11:30:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.430+0000 7f9757e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f97580714a0 con 0x7f975815e160 2024-04-25T11:30:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f975806f370 0x7f975806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f975806f370 0x7f975806f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47988/0 (socket says 172.21.15.19:47988) 2024-04-25T11:30:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756000640 1 -- 172.21.15.19:0/4252772795 learned_addr learned my addr 172.21.15.19:0/4252772795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756a00640 1 --2- 172.21.15.19:0/4252772795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975815e160 0x7f97580ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756000640 1 -- 172.21.15.19:0/4252772795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975815e160 msgr2=0x7f97580ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756000640 1 --2- 172.21.15.19:0/4252772795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975815e160 0x7f97580ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756000640 1 -- 172.21.15.19:0/4252772795 >> v1:172.21.15.169:6789/0 conn(0x7f975806cf40 legacy=0x7f975806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:53.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.431+0000 7f9756000640 1 -- 172.21.15.19:0/4252772795 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97580cd440 con 0x7f975806f370 2024-04-25T11:30:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.433+0000 7f9756000640 1 --2- 172.21.15.19:0/4252772795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806f370 0x7f975806f780 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9744009870 tx=0x7f974402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194da7920d35540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.433+0000 7f9755600640 1 -- 172.21.15.19:0/4252772795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744037070 con 0x7f975806f370 2024-04-25T11:30:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.433+0000 7f9755600640 1 -- 172.21.15.19:0/4252772795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9744032440 con 0x7f975806f370 2024-04-25T11:30:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.433+0000 7f9755600640 1 -- 172.21.15.19:0/4252772795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744032810 con 0x7f975806f370 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 -- 172.21.15.19:0/4252772795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806f370 msgr2=0x7f975806f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 --2- 172.21.15.19:0/4252772795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806f370 0x7f975806f780 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9744009870 tx=0x7f974402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 -- 172.21.15.19:0/4252772795 shutdown_connections 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 --2- 172.21.15.19:0/4252772795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806f370 0x7f975806f780 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 --2- 172.21.15.19:0/4252772795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975815e160 0x7f97580ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 -- 172.21.15.19:0/4252772795 >> 172.21.15.19:0/4252772795 conn(0x7f97580ce3b0 msgr2=0x7f97580ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 -- 172.21.15.19:0/4252772795 shutdown_connections 2024-04-25T11:30:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.434+0000 7f9757e00640 1 -- 172.21.15.19:0/4252772795 wait complete. 2024-04-25T11:30:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.436+0000 7f9757e00640 1 Processor -- start 2024-04-25T11:30:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.437+0000 7f9757e00640 1 -- start start 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9757e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806cf40 0x7f97581d8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9757e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975806f370 0x7f97581f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9757e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97581dd6c0 0x7f97581ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9757e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f975806c740 con 0x7f975806cf40 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9757e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f97580cdec0 con 0x7f975806f370 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9757e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f97580b4820 con 0x7f97581dd6c0 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9756a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806cf40 0x7f97581d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9756a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806cf40 0x7f97581d8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48004/0 (socket says 172.21.15.19:48004) 2024-04-25T11:30:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.438+0000 7f9756a00640 1 -- 172.21.15.19:0/3648595639 learned_addr learned my addr 172.21.15.19:0/3648595639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.439+0000 7f9756a00640 1 -- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97581dd6c0 msgr2=0x7f97581ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.439+0000 7f9756a00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97581dd6c0 0x7f97581ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.439+0000 7f9756a00640 1 -- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975806f370 msgr2=0x7f97581f90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.439+0000 7f9756a00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975806f370 0x7f97581f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.439+0000 7f9756a00640 1 -- 172.21.15.19:0/3648595639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97581dc120 con 0x7f975806cf40 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.439+0000 7f9756a00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806cf40 0x7f97581d8ca0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f974800d5d0 tx=0x7f974800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.439+0000 7f974ea00640 1 -- 172.21.15.19:0/3648595639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97480125f0 con 0x7f975806cf40 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.440+0000 7f974ea00640 1 -- 172.21.15.19:0/3648595639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748019070 con 0x7f975806cf40 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.440+0000 7f974ea00640 1 -- 172.21.15.19:0/3648595639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97480129d0 con 0x7f975806cf40 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.440+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97581dc620 con 0x7f975806cf40 2024-04-25T11:30:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.440+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97581dcbe0 con 0x7f975806cf40 2024-04-25T11:30:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.441+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9758076c60 con 0x7f975806cf40 2024-04-25T11:30:53.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.442+0000 7f974ea00640 1 -- 172.21.15.19:0/3648595639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9748012b70 con 0x7f975806cf40 2024-04-25T11:30:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.443+0000 7f974ea00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97380744b0 0x7f97380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.444+0000 7f974ea00640 1 -- 172.21.15.19:0/3648595639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5443+0+0 (secure 0 0 0) 0x7f9748095640 con 0x7f975806cf40 2024-04-25T11:30:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.444+0000 7f9756000640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97380744b0 0x7f97380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.446+0000 7f9756000640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97380744b0 0x7f97380769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9744034000 tx=0x7f974402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.448+0000 7f974ea00640 1 -- 172.21.15.19:0/3648595639 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f974805fec0 con 0x7f975806cf40 2024-04-25T11:30:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:53.656+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f97580d1170 con 0x7f975806cf40 2024-04-25T11:30:54.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.118+0000 7f974ea00640 1 -- 172.21.15.19:0/3648595639 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7f974805f7c0 con 0x7f975806cf40 2024-04-25T11:30:54.108 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:30:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97380744b0 msgr2=0x7f97380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97380744b0 0x7f97380769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9744034000 tx=0x7f974402c000 comp rx=0 tx=0).stop 2024-04-25T11:30:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806cf40 msgr2=0x7f97581d8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806cf40 0x7f97581d8ca0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f974800d5d0 tx=0x7f974800da90 comp rx=0 tx=0).stop 2024-04-25T11:30:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 shutdown_connections 2024-04-25T11:30:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97380744b0 0x7f97380769c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975806f370 0x7f97581f90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97581dd6c0 0x7f97581ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 --2- 172.21.15.19:0/3648595639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975806cf40 0x7f97581d8ca0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.123+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 >> 172.21.15.19:0/3648595639 conn(0x7f97580ce3b0 msgr2=0x7f97580b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.125+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 shutdown_connections 2024-04-25T11:30:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.126+0000 7f9757e00640 1 -- 172.21.15.19:0/3648595639 wait complete. 2024-04-25T11:30:54.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_abort_on_newly_corrupt_dentry 2024-04-25T11:30:54.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.435+0000 7fe73e800640 1 Processor -- start 2024-04-25T11:30:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.435+0000 7fe73e800640 1 -- start start 2024-04-25T11:30:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.437+0000 7fe73e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 0x7fe73815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.437+0000 7fe73e800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7380ce650 0x7fe7380cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.437+0000 7fe73e800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe73806c810 con 0x7fe73806d5d0 2024-04-25T11:30:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.437+0000 7fe73e800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe73815e330 con 0x7fe7380ce650 2024-04-25T11:30:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.437+0000 7fe73e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe73806c5c0 con 0x7fe7380aedb0 2024-04-25T11:30:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.438+0000 7fe73de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe73806d5d0 0x7fe738161a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50776/0 (socket says 172.21.15.19:50776) 2024-04-25T11:30:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.438+0000 7fe73de00640 1 -- 172.21.15.19:0/2134410794 learned_addr learned my addr 172.21.15.19:0/2134410794 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.438+0000 7fe73d400640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 0x7fe73815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.438+0000 7fe73ca00640 1 --2- 172.21.15.19:0/2134410794 >> v2:172.21.15.169:3300/0 conn(0x7fe7380ce650 0x7fe7380cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.438+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860442213 0 0) 0x7fe73806c810 con 0x7fe73806d5d0 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.438+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe720003520 con 0x7fe73806d5d0 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3788779419 0 0) 0x7fe720003520 con 0x7fe73806d5d0 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 msgr2=0x7fe73815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 0x7fe73815ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 >> v2:172.21.15.169:3300/0 conn(0x7fe7380ce650 msgr2=0x7fe7380cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 --2- 172.21.15.19:0/2134410794 >> v2:172.21.15.169:3300/0 conn(0x7fe7380ce650 0x7fe7380cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73806caf0 con 0x7fe73806d5d0 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe730001670 con 0x7fe73806d5d0 2024-04-25T11:30:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 >> v1:172.21.15.19:6789/0 conn(0x7fe73806d5d0 legacy=0x7fe738161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 0x7fe7200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe720003f30 0x7fe720004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe720004d40 0x7fe720005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe73815e330 con 0x7fe720004d40 2024-04-25T11:30:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe73806c5c0 con 0x7fe720003f30 2024-04-25T11:30:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe72000bac0 con 0x7fe7380aedb0 2024-04-25T11:30:54.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.439+0000 7fe73de00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe720004d40 0x7fe720005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73ca00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe720003f30 0x7fe720004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73d400640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 0x7fe7200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73de00640 1 -- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 msgr2=0x7fe7200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73de00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 0x7fe7200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73de00640 1 -- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe720003f30 msgr2=0x7fe720004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73de00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe720003f30 0x7fe720004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73de00640 1 -- 172.21.15.19:0/2134410794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe720003520 con 0x7fe720004d40 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73ca00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe720003f30 0x7fe720004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.440+0000 7fe73d400640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7380aedb0 0x7fe7200039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.441+0000 7fe73de00640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe720004d40 0x7fe720005150 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe730007fd0 tx=0x7fe720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb524485deeebbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.441+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730007d50 con 0x7fe720004d40 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.441+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe730005ad0 con 0x7fe720004d40 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.441+0000 7fe737e00640 1 -- 172.21.15.19:0/2134410794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730007070 con 0x7fe720004d40 2024-04-25T11:30:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.442+0000 7fe73e800640 1 -- 172.21.15.19:0/2134410794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe720004d40 msgr2=0x7fe720005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.442+0000 7fe73e800640 1 --2- 172.21.15.19:0/2134410794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe720004d40 0x7fe720005150 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe730007fd0 tx=0x7fe720005830 comp rx=0 tx=0).stop 2024-04-25T11:30:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.442+0000 7fe73de00640 1 -- 172.21.15.19:0/2134410794 reap_dead start 2024-04-25T11:30:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.442+0000 7fe73e800640 1 -- 172.21.15.19:0/2134410794 shutdown_connections 2024-04-25T11:30:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.442+0000 7fe73e800640 1 -- 172.21.15.19:0/2134410794 >> 172.21.15.19:0/2134410794 conn(0x7fe7380d1d60 msgr2=0x7fe7380d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.442+0000 7fe73e800640 1 -- 172.21.15.19:0/2134410794 shutdown_connections 2024-04-25T11:30:54.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.442+0000 7fe73e800640 1 -- 172.21.15.19:0/2134410794 wait complete. 2024-04-25T11:30:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.444+0000 7fe73e800640 1 Processor -- start 2024-04-25T11:30:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.445+0000 7fe73e800640 1 -- start start 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.446+0000 7fe73e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe720004d40 0x7fe73806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.446+0000 7fe73e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73814fe50 0x7fe7381f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.446+0000 7fe73e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7381fa2f0 0x7fe73815a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.446+0000 7fe73e800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7380b5e60 con 0x7fe7381fa2f0 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.446+0000 7fe73e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe73806d0b0 con 0x7fe73814fe50 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.446+0000 7fe73e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7380b5fe0 con 0x7fe720004d40 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.446+0000 7fe73de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7381fa2f0 0x7fe73815a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7381fa2f0 0x7fe73815a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38082/0 (socket says 172.21.15.19:38082) 2024-04-25T11:30:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73de00640 1 -- 172.21.15.19:0/2401857786 learned_addr learned my addr 172.21.15.19:0/2401857786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73d400640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe720004d40 0x7fe73806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73ca00640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73814fe50 0x7fe7381f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73de00640 1 -- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe720004d40 msgr2=0x7fe73806f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73de00640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe720004d40 0x7fe73806f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73de00640 1 -- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73814fe50 msgr2=0x7fe7381f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73de00640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73814fe50 0x7fe7381f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.447+0000 7fe73de00640 1 -- 172.21.15.19:0/2401857786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73806fe00 con 0x7fe7381fa2f0 2024-04-25T11:30:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.448+0000 7fe73de00640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7381fa2f0 0x7fe73815a8e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe730000c00 tx=0x7fe73806ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.448+0000 7fe735600640 1 -- 172.21.15.19:0/2401857786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730041070 con 0x7fe7381fa2f0 2024-04-25T11:30:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.448+0000 7fe735600640 1 -- 172.21.15.19:0/2401857786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe730004d20 con 0x7fe7381fa2f0 2024-04-25T11:30:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.448+0000 7fe735600640 1 -- 172.21.15.19:0/2401857786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730007d50 con 0x7fe7381fa2f0 2024-04-25T11:30:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.448+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe738070800 con 0x7fe7381fa2f0 2024-04-25T11:30:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.448+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe738070520 con 0x7fe7381fa2f0 2024-04-25T11:30:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.449+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe738076c60 con 0x7fe7381fa2f0 2024-04-25T11:30:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.450+0000 7fe735600640 1 -- 172.21.15.19:0/2401857786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe730044070 con 0x7fe7381fa2f0 2024-04-25T11:30:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.451+0000 7fe735600640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7180744b0 0x7fe7180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.452+0000 7fe735600640 1 -- 172.21.15.19:0/2401857786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7fe7300b5160 con 0x7fe7381fa2f0 2024-04-25T11:30:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.452+0000 7fe73d400640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7180744b0 0x7fe7180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.453+0000 7fe73d400640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7180744b0 0x7fe7180769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe72c003be0 tx=0x7fe72c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:54.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.457+0000 7fe735600640 1 -- 172.21.15.19:0/2401857786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe73007fc10 con 0x7fe7381fa2f0 2024-04-25T11:30:54.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.667+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"} v 0) -- 0x7fe73806ffd0 con 0x7fe7381fa2f0 2024-04-25T11:30:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.676+0000 7fe735600640 1 -- 172.21.15.19:0/2401857786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"}]=0 v16) ==== 116+0+0 (secure 0 0 0) 0x7fe73007f510 con 0x7fe7381fa2f0 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.679+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7180744b0 msgr2=0x7fe7180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.679+0000 7fe73e800640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7180744b0 0x7fe7180769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe72c003be0 tx=0x7fe72c00b040 comp rx=0 tx=0).stop 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7381fa2f0 msgr2=0x7fe73815a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7381fa2f0 0x7fe73815a8e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe730000c00 tx=0x7fe73806ccc0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 shutdown_connections 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7180744b0 0x7fe7180769c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73814fe50 0x7fe7381f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7381fa2f0 0x7fe73815a8e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 --2- 172.21.15.19:0/2401857786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe720004d40 0x7fe73806f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 >> 172.21.15.19:0/2401857786 conn(0x7fe7380d1d60 msgr2=0x7fe7381604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:54.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 shutdown_connections 2024-04-25T11:30:54.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.680+0000 7fe73e800640 1 -- 172.21.15.19:0/2401857786 wait complete. 2024-04-25T11:30:54.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-25T11:30:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.991+0000 7effc0600640 1 Processor -- start 2024-04-25T11:30:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.991+0000 7effc0600640 1 -- start start 2024-04-25T11:30:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effc0600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7effb806ccd0 0x7effb80d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effc0600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb80d1830 con 0x7effb80d0de0 2024-04-25T11:30:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effc0600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb8160dc0 con 0x7effb80d4a30 2024-04-25T11:30:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effc0600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7effb81610a0 con 0x7effb806ccd0 2024-04-25T11:30:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effbdc00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7effb80d0de0 0x7effb806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50778/0 (socket says 172.21.15.19:50778) 2024-04-25T11:30:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effbdc00640 1 -- 172.21.15.19:0/3864610192 learned_addr learned my addr 172.21.15.19:0/3864610192 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effbd200640 1 --2- 172.21.15.19:0/3864610192 >> v2:172.21.15.169:3300/0 conn(0x7effb806ccd0 0x7effb80d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.993+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 266416144 0 0) 0x7effb80d1830 con 0x7effb80d0de0 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff9c003520 con 0x7effb80d0de0 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4105681980 0 0) 0x7eff9c003520 con 0x7effb80d0de0 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 >> v2:172.21.15.169:3300/0 conn(0x7effb806ccd0 msgr2=0x7effb80d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 --2- 172.21.15.19:0/3864610192 >> v2:172.21.15.169:3300/0 conn(0x7effb806ccd0 0x7effb80d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 >> v1:172.21.15.169:6789/0 conn(0x7effb80d4a30 legacy=0x7effb80d4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb8161380 con 0x7effb80d0de0 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7effa8001670 con 0x7effb80d0de0 2024-04-25T11:30:54.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 >> v1:172.21.15.19:6789/0 conn(0x7effb80d0de0 legacy=0x7effb806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c0039b0 0x7eff9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c004800 0x7eff9c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff9c005300 0x7eff9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effb81610a0 con 0x7eff9c004800 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c00c2a0 con 0x7eff9c0039b0 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff9c00c4a0 con 0x7eff9c005300 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.994+0000 7effbd200640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c004800 0x7eff9c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbdc00640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c0039b0 0x7eff9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbd200640 1 -- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff9c005300 msgr2=0x7eff9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbe600640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff9c005300 0x7eff9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbd200640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff9c005300 0x7eff9c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbd200640 1 -- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c0039b0 msgr2=0x7eff9c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbd200640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c0039b0 0x7eff9c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbd200640 1 -- 172.21.15.19:0/3864610192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9c003520 con 0x7eff9c004800 2024-04-25T11:30:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbe600640 1 -- 172.21.15.19:0/3864610192 reap_dead start 2024-04-25T11:30:54.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.995+0000 7effbdc00640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c0039b0 0x7eff9c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:30:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.996+0000 7effbd200640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c004800 0x7eff9c004c10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7effa400ba30 tx=0x7eff9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330c084d69bac690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.996+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4030070 con 0x7eff9c004800 2024-04-25T11:30:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.996+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa402ade0 con 0x7eff9c004800 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.996+0000 7effb7e00640 1 -- 172.21.15.19:0/3864610192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa400b600 con 0x7eff9c004800 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.997+0000 7effc0600640 1 -- 172.21.15.19:0/3864610192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c004800 msgr2=0x7eff9c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.997+0000 7effc0600640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c004800 0x7eff9c004c10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7effa400ba30 tx=0x7eff9c005830 comp rx=0 tx=0).stop 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.997+0000 7effc0600640 1 -- 172.21.15.19:0/3864610192 shutdown_connections 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.997+0000 7effc0600640 1 --2- 172.21.15.19:0/3864610192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c004800 0x7eff9c004c10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.997+0000 7effc0600640 1 -- 172.21.15.19:0/3864610192 >> 172.21.15.19:0/3864610192 conn(0x7effb80d5d70 msgr2=0x7effb80d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.998+0000 7effc0600640 1 -- 172.21.15.19:0/3864610192 shutdown_connections 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:54.998+0000 7effc0600640 1 -- 172.21.15.19:0/3864610192 wait complete. 2024-04-25T11:30:54.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.000+0000 7effc0600640 1 Processor -- start 2024-04-25T11:30:54.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.000+0000 7effc0600640 1 -- start start 2024-04-25T11:30:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effc0600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c004800 0x7effb81f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effc0600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effb814a9e0 0x7effb814adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effc0600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effb80cf2a0 0x7effb80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effc0600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effb8164de0 con 0x7effb80cf2a0 2024-04-25T11:30:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effc0600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7effb80b5300 con 0x7eff9c004800 2024-04-25T11:30:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effc0600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7effb8164f60 con 0x7effb814a9e0 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effbe600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effb80cf2a0 0x7effb80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effbe600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effb80cf2a0 0x7effb80cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38098/0 (socket says 172.21.15.19:38098) 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effbe600640 1 -- 172.21.15.19:0/2710012551 learned_addr learned my addr 172.21.15.19:0/2710012551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effbdc00640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c004800 0x7effb81f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.002+0000 7effbd200640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effb814a9e0 0x7effb814adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effbe600640 1 -- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effb814a9e0 msgr2=0x7effb814adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effbe600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effb814a9e0 0x7effb814adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effbe600640 1 -- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c004800 msgr2=0x7effb81f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effbe600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c004800 0x7effb81f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effbe600640 1 -- 172.21.15.19:0/2710012551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb814b330 con 0x7effb80cf2a0 2024-04-25T11:30:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effbd200640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effb814a9e0 0x7effb814adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effbe600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effb80cf2a0 0x7effb80cf6b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7effb806d3f0 tx=0x7effb000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effb5600640 1 -- 172.21.15.19:0/2710012551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0018070 con 0x7effb80cf2a0 2024-04-25T11:30:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effb5600640 1 -- 172.21.15.19:0/2710012551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb0003b70 con 0x7effb80cf2a0 2024-04-25T11:30:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb80ce200 con 0x7effb80cf2a0 2024-04-25T11:30:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effb5600640 1 -- 172.21.15.19:0/2710012551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0005070 con 0x7effb80cf2a0 2024-04-25T11:30:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.003+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb80ce7c0 con 0x7effb80cf2a0 2024-04-25T11:30:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.005+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb80cdf60 con 0x7effb80cf2a0 2024-04-25T11:30:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.006+0000 7effb5600640 1 -- 172.21.15.19:0/2710012551 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7effb001b070 con 0x7effb80cf2a0 2024-04-25T11:30:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.007+0000 7effb5600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff980744b0 0x7eff980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.007+0000 7effb5600640 1 -- 172.21.15.19:0/2710012551 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7effb0094750 con 0x7effb80cf2a0 2024-04-25T11:30:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.007+0000 7effbdc00640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff980744b0 0x7eff980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.009+0000 7effbdc00640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff980744b0 0x7eff980769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effa8001eb0 tx=0x7effa8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.012+0000 7effb5600640 1 -- 172.21.15.19:0/2710012551 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effb80cdf60 con 0x7effb80cf2a0 2024-04-25T11:30:55.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.209+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7effb8050740 con 0x7effb80cf2a0 2024-04-25T11:30:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.209+0000 7effb5600640 1 -- 172.21.15.19:0/2710012551 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7effb814b330 con 0x7effb80cf2a0 2024-04-25T11:30:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff980744b0 msgr2=0x7eff980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff980744b0 0x7eff980769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effa8001eb0 tx=0x7effa8009040 comp rx=0 tx=0).stop 2024-04-25T11:30:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effb80cf2a0 msgr2=0x7effb80cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effb80cf2a0 0x7effb80cf6b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7effb806d3f0 tx=0x7effb000dc00 comp rx=0 tx=0).stop 2024-04-25T11:30:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 shutdown_connections 2024-04-25T11:30:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7eff980744b0 0x7eff980769c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effb80cf2a0 0x7effb80cf6b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7effb814a9e0 0x7effb814adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 --2- 172.21.15.19:0/2710012551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff9c004800 0x7effb81f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 >> 172.21.15.19:0/2710012551 conn(0x7effb80d5d70 msgr2=0x7effb8161d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 shutdown_connections 2024-04-25T11:30:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.212+0000 7effc0600640 1 -- 172.21.15.19:0/2710012551 wait complete. 2024-04-25T11:30:55.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:30:55.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.510+0000 7f8041e00640 1 Processor -- start 2024-04-25T11:30:55.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.510+0000 7f8041e00640 1 -- start start 2024-04-25T11:30:55.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.513+0000 7f8041e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f803c070d20 0x7f803c161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:55.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.513+0000 7f8041e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c1623f0 con 0x7f803c06dce0 2024-04-25T11:30:55.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.513+0000 7f8041e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c1626d0 con 0x7f803c06f930 2024-04-25T11:30:55.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.513+0000 7f8041e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f803c06f130 con 0x7f803c070d20 2024-04-25T11:30:55.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.513+0000 7f8041400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f803c06dce0 0x7f803c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50786/0 (socket says 172.21.15.19:50786) 2024-04-25T11:30:55.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.513+0000 7f8041400640 1 -- 172.21.15.19:0/914199619 learned_addr learned my addr 172.21.15.19:0/914199619 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:55.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.513+0000 7f8040a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f803c070d20 0x7f803c161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.514+0000 7f803b400640 1 -- 172.21.15.19:0/914199619 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615619522 0 0) 0x7f803c1623f0 con 0x7f803c06dce0 2024-04-25T11:30:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.514+0000 7f803b400640 1 -- 172.21.15.19:0/914199619 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8024003520 con 0x7f803c06dce0 2024-04-25T11:30:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.514+0000 7f8040a00640 1 -- 172.21.15.19:0/914199619 >> v1:172.21.15.169:6789/0 conn(0x7f803c06f930 legacy=0x7f803c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.514+0000 7f8040a00640 1 -- 172.21.15.19:0/914199619 >> v1:172.21.15.19:6789/0 conn(0x7f803c06dce0 legacy=0x7f803c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.514+0000 7f8040a00640 1 -- 172.21.15.19:0/914199619 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c06f410 con 0x7f803c070d20 2024-04-25T11:30:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.515+0000 7f8040a00640 1 --2- 172.21.15.19:0/914199619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c070d20 0x7f803c161b30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2249080b49de24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.516+0000 7f803b400640 1 -- 172.21.15.19:0/914199619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030037070 con 0x7f803c070d20 2024-04-25T11:30:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.516+0000 7f803b400640 1 -- 172.21.15.19:0/914199619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8030032440 con 0x7f803c070d20 2024-04-25T11:30:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.516+0000 7f803b400640 1 -- 172.21.15.19:0/914199619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030032810 con 0x7f803c070d20 2024-04-25T11:30:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.516+0000 7f8041e00640 1 -- 172.21.15.19:0/914199619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c070d20 msgr2=0x7f803c161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.516+0000 7f8041e00640 1 --2- 172.21.15.19:0/914199619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c070d20 0x7f803c161b30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.517+0000 7f8041e00640 1 -- 172.21.15.19:0/914199619 shutdown_connections 2024-04-25T11:30:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.517+0000 7f8041e00640 1 --2- 172.21.15.19:0/914199619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c070d20 0x7f803c161b30 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.517+0000 7f8041e00640 1 -- 172.21.15.19:0/914199619 >> 172.21.15.19:0/914199619 conn(0x7f803c073160 msgr2=0x7f803c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.517+0000 7f8041e00640 1 -- 172.21.15.19:0/914199619 shutdown_connections 2024-04-25T11:30:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.517+0000 7f8041e00640 1 -- 172.21.15.19:0/914199619 wait complete. 2024-04-25T11:30:55.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.519+0000 7f8041e00640 1 Processor -- start 2024-04-25T11:30:55.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.519+0000 7f8041e00640 1 -- start start 2024-04-25T11:30:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8041e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c06f930 0x7f803c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8041e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f803c070d20 0x7f803c0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8041e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f803c15f000 0x7f803c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8041e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f803c15e2f0 con 0x7f803c06f930 2024-04-25T11:30:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8041e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f803c162070 con 0x7f803c15f000 2024-04-25T11:30:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8041e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f803c15e470 con 0x7f803c070d20 2024-04-25T11:30:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8040a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c06f930 0x7f803c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8040a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c06f930 0x7f803c156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38116/0 (socket says 172.21.15.19:38116) 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.521+0000 7f8040a00640 1 -- 172.21.15.19:0/2429748227 learned_addr learned my addr 172.21.15.19:0/2429748227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f803be00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f803c070d20 0x7f803c0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8041400640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f803c15f000 0x7f803c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8040a00640 1 -- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f803c070d20 msgr2=0x7f803c0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8040a00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f803c070d20 0x7f803c0cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8040a00640 1 -- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f803c15f000 msgr2=0x7f803c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8040a00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f803c15f000 0x7f803c14f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8040a00640 1 -- 172.21.15.19:0/2429748227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c157350 con 0x7f803c06f930 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f803be00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f803c070d20 0x7f803c0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8041400640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f803c15f000 0x7f803c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.522+0000 7f8040a00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c06f930 0x7f803c156920 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8030000c00 tx=0x7f80300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.523+0000 7f8038c00640 1 -- 172.21.15.19:0/2429748227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030037040 con 0x7f803c06f930 2024-04-25T11:30:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.523+0000 7f8038c00640 1 -- 172.21.15.19:0/2429748227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8030042070 con 0x7f803c06f930 2024-04-25T11:30:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.523+0000 7f8038c00640 1 -- 172.21.15.19:0/2429748227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030030940 con 0x7f803c06f930 2024-04-25T11:30:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.523+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c0cb210 con 0x7f803c06f930 2024-04-25T11:30:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.523+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1575c0 con 0x7f803c06f930 2024-04-25T11:30:55.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.524+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803c076c60 con 0x7f803c06f930 2024-04-25T11:30:55.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.525+0000 7f8038c00640 1 -- 172.21.15.19:0/2429748227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8030030ae0 con 0x7f803c06f930 2024-04-25T11:30:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.526+0000 7f8038c00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f801c0744b0 0x7f801c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.526+0000 7f8038c00640 1 -- 172.21.15.19:0/2429748227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7f80300b57a0 con 0x7f803c06f930 2024-04-25T11:30:55.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.527+0000 7f803be00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f801c0744b0 0x7f801c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:55.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.528+0000 7f803be00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f801c0744b0 0x7f801c0769c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f803c0b5af0 tx=0x7f8028009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.531+0000 7f8038c00640 1 -- 172.21.15.19:0/2429748227 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80300801c0 con 0x7f803c06f930 2024-04-25T11:30:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.735+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f803c074070 con 0x7f803c06f930 2024-04-25T11:30:55.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.742+0000 7f8038c00640 1 -- 172.21.15.19:0/2429748227 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f803007fac0 con 0x7f803c06f930 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.744+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f801c0744b0 msgr2=0x7f801c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.744+0000 7f8041e00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f801c0744b0 0x7f801c0769c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f803c0b5af0 tx=0x7f8028009040 comp rx=0 tx=0).stop 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c06f930 msgr2=0x7f803c156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c06f930 0x7f803c156920 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8030000c00 tx=0x7f80300046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 shutdown_connections 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f801c0744b0 0x7f801c0769c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f803c15f000 0x7f803c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f803c070d20 0x7f803c0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 --2- 172.21.15.19:0/2429748227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c06f930 0x7f803c156920 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 >> 172.21.15.19:0/2429748227 conn(0x7f803c073160 msgr2=0x7f803c0b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 shutdown_connections 2024-04-25T11:30:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:55.745+0000 7f8041e00640 1 -- 172.21.15.19:0/2429748227 wait complete. 2024-04-25T11:30:55.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-04-25T11:30:56.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.041+0000 7f378ee00640 1 Processor -- start 2024-04-25T11:30:56.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.041+0000 7f378ee00640 1 -- start start 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f378ee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3788164b00 0x7f37880b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f378ee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f378815df80 0x7f37880b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f378ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788160890 0x7f3788160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f378ee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3788164670 con 0x7f378815df80 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f378ee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f37881615f0 con 0x7f3788164b00 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f378ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3788164020 con 0x7f3788160890 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f3787400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f378815df80 0x7f37880b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f3787400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f378815df80 0x7f37880b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38128/0 (socket says 172.21.15.19:38128) 2024-04-25T11:30:56.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.044+0000 7f3787400640 1 -- 172.21.15.19:0/3674714811 learned_addr learned my addr 172.21.15.19:0/3674714811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f3787e00640 1 --2- 172.21.15.19:0/3674714811 >> v2:172.21.15.169:3300/0 conn(0x7f3788164b00 0x7f37880b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f378ce00640 1 --2- 172.21.15.19:0/3674714811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788160890 0x7f3788160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f3787400640 1 -- 172.21.15.19:0/3674714811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788160890 msgr2=0x7f3788160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f3787400640 1 --2- 172.21.15.19:0/3674714811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788160890 0x7f3788160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f3787400640 1 -- 172.21.15.19:0/3674714811 >> v2:172.21.15.169:3300/0 conn(0x7f3788164b00 msgr2=0x7f37880b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:56.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f3787400640 1 --2- 172.21.15.19:0/3674714811 >> v2:172.21.15.169:3300/0 conn(0x7f3788164b00 0x7f37880b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f3787400640 1 -- 172.21.15.19:0/3674714811 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3788164300 con 0x7f378815df80 2024-04-25T11:30:56.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.045+0000 7f378ce00640 1 --2- 172.21.15.19:0/3674714811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788160890 0x7f3788160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:56.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.046+0000 7f3787400640 1 --2- 172.21.15.19:0/3674714811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f37880b80c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f378808e410 tx=0x7f377002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a7e04d2a863ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.046+0000 7f3786a00640 1 -- 172.21.15.19:0/3674714811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770037070 con 0x7f378815df80 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.046+0000 7f3786a00640 1 -- 172.21.15.19:0/3674714811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3770032440 con 0x7f378815df80 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.046+0000 7f3786a00640 1 -- 172.21.15.19:0/3674714811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770032810 con 0x7f378815df80 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 -- 172.21.15.19:0/3674714811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 msgr2=0x7f37880b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 --2- 172.21.15.19:0/3674714811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f37880b80c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f378808e410 tx=0x7f377002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 -- 172.21.15.19:0/3674714811 shutdown_connections 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 --2- 172.21.15.19:0/3674714811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f37880b80c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 --2- 172.21.15.19:0/3674714811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788160890 0x7f3788160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 --2- 172.21.15.19:0/3674714811 >> v2:172.21.15.169:3300/0 conn(0x7f3788164b00 0x7f37880b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 -- 172.21.15.19:0/3674714811 >> 172.21.15.19:0/3674714811 conn(0x7f37880d2bc0 msgr2=0x7f37880d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 -- 172.21.15.19:0/3674714811 shutdown_connections 2024-04-25T11:30:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.047+0000 7f378ee00640 1 -- 172.21.15.19:0/3674714811 wait complete. 2024-04-25T11:30:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.049+0000 7f378ee00640 1 Processor -- start 2024-04-25T11:30:56.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.049+0000 7f378ee00640 1 -- start start 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f378ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f378815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f378ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3788160890 0x7f37881f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f378ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788164b00 0x7f37881fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f378ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37880b4a00 con 0x7f378815df80 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f378ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37880b4880 con 0x7f3788160890 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f378ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f37880b4b80 con 0x7f3788164b00 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f3787e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f378815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f3787e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f378815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38130/0 (socket says 172.21.15.19:38130) 2024-04-25T11:30:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.051+0000 7f3787e00640 1 -- 172.21.15.19:0/1150514018 learned_addr learned my addr 172.21.15.19:0/1150514018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f3787400640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3788160890 0x7f37881f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f378ce00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788164b00 0x7f37881fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f3787e00640 1 -- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788164b00 msgr2=0x7f37881fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f3787e00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788164b00 0x7f37881fa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f3787e00640 1 -- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3788160890 msgr2=0x7f37881f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f3787e00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3788160890 0x7f37881f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f3787e00640 1 -- 172.21.15.19:0/1150514018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378815a580 con 0x7f378815df80 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f3787e00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f378815a040 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3778007e70 tx=0x7f377800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.052+0000 7f377fe00640 1 -- 172.21.15.19:0/1150514018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778014620 con 0x7f378815df80 2024-04-25T11:30:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.053+0000 7f377fe00640 1 -- 172.21.15.19:0/1150514018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3778019070 con 0x7f378815df80 2024-04-25T11:30:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.053+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378815aa90 con 0x7f378815df80 2024-04-25T11:30:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.053+0000 7f377fe00640 1 -- 172.21.15.19:0/1150514018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778014a00 con 0x7f378815df80 2024-04-25T11:30:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.053+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3788157650 con 0x7f378815df80 2024-04-25T11:30:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.054+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37881f7870 con 0x7f378815df80 2024-04-25T11:30:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.055+0000 7f377fe00640 1 -- 172.21.15.19:0/1150514018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3778014ba0 con 0x7f378815df80 2024-04-25T11:30:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.056+0000 7f377fe00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37640744b0 0x7f37640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.056+0000 7f377fe00640 1 -- 172.21.15.19:0/1150514018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7f37780958f0 con 0x7f378815df80 2024-04-25T11:30:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.057+0000 7f3787400640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37640744b0 0x7f37640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.058+0000 7f3787400640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37640744b0 0x7f37640769c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3770034000 tx=0x7f377002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.061+0000 7f377fe00640 1 -- 172.21.15.19:0/1150514018 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37881f7870 con 0x7f378815df80 2024-04-25T11:30:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:56.332+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7f3788050740 con 0x7f378815df80 2024-04-25T11:30:57.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.136+0000 7f377fe00640 1 -- 172.21.15.19:0/1150514018 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v208) ==== 141+0+0 (secure 0 0 0) 0x7f3778060280 con 0x7f378815df80 2024-04-25T11:30:57.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.139+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37640744b0 msgr2=0x7f37640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.139+0000 7f378ee00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37640744b0 0x7f37640769c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3770034000 tx=0x7f377002e000 comp rx=0 tx=0).stop 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.139+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 msgr2=0x7f378815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f378815a040 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3778007e70 tx=0x7f377800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 shutdown_connections 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37640744b0 0x7f37640769c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3788160890 0x7f37881f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3788164b00 0x7f37881fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 --2- 172.21.15.19:0/1150514018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f378815df80 0x7f378815a040 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 >> 172.21.15.19:0/1150514018 conn(0x7f37880d2bc0 msgr2=0x7f378815bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 shutdown_connections 2024-04-25T11:30:57.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.140+0000 7f378ee00640 1 -- 172.21.15.19:0/1150514018 wait complete. 2024-04-25T11:30:57.159 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2024-04-25T11:30:57.160 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-04-25T11:30:57.160 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2024-04-25T11:30:57.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:30:57.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.438+0000 7f217a600640 1 Processor -- start 2024-04-25T11:30:57.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.438+0000 7f217a600640 1 -- start start 2024-04-25T11:30:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.440+0000 7f217a600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21740b4ea0 0x7f21740cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.440+0000 7f217a600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21740d1230 con 0x7f2174144370 2024-04-25T11:30:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.440+0000 7f217a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21740d06b0 con 0x7f217406c5c0 2024-04-25T11:30:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.440+0000 7f217a600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f21740d0990 con 0x7f21740b4ea0 2024-04-25T11:30:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2173e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21740b4ea0 0x7f21740cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2179c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2174144370 0x7f217415e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50790/0 (socket says 172.21.15.19:50790) 2024-04-25T11:30:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2179c00640 1 -- 172.21.15.19:0/1061225187 learned_addr learned my addr 172.21.15.19:0/1061225187 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2173400640 1 -- 172.21.15.19:0/1061225187 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717879892 0 0) 0x7f21740d1230 con 0x7f2174144370 2024-04-25T11:30:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2173400640 1 -- 172.21.15.19:0/1061225187 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2158003520 con 0x7f2174144370 2024-04-25T11:30:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2173e00640 1 -- 172.21.15.19:0/1061225187 >> v1:172.21.15.169:6789/0 conn(0x7f217406c5c0 legacy=0x7f21740b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2173e00640 1 -- 172.21.15.19:0/1061225187 >> v1:172.21.15.19:6789/0 conn(0x7f2174144370 legacy=0x7f217415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.441+0000 7f2173e00640 1 -- 172.21.15.19:0/1061225187 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21740d0c70 con 0x7f21740b4ea0 2024-04-25T11:30:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.442+0000 7f2173e00640 1 --2- 172.21.15.19:0/1061225187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21740b4ea0 0x7f21740cfa70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f215c0097a0 tx=0x7f215c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ab39cf0a171299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.443+0000 7f2173400640 1 -- 172.21.15.19:0/1061225187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c037070 con 0x7f21740b4ea0 2024-04-25T11:30:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.443+0000 7f2173400640 1 -- 172.21.15.19:0/1061225187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c032440 con 0x7f21740b4ea0 2024-04-25T11:30:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.443+0000 7f2173400640 1 -- 172.21.15.19:0/1061225187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c032810 con 0x7f21740b4ea0 2024-04-25T11:30:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.444+0000 7f217a600640 1 -- 172.21.15.19:0/1061225187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21740b4ea0 msgr2=0x7f21740cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.444+0000 7f217a600640 1 --2- 172.21.15.19:0/1061225187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21740b4ea0 0x7f21740cfa70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f215c0097a0 tx=0x7f215c02a230 comp rx=0 tx=0).stop 2024-04-25T11:30:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.444+0000 7f217a600640 1 -- 172.21.15.19:0/1061225187 shutdown_connections 2024-04-25T11:30:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.444+0000 7f217a600640 1 --2- 172.21.15.19:0/1061225187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21740b4ea0 0x7f21740cfa70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.444+0000 7f217a600640 1 -- 172.21.15.19:0/1061225187 >> 172.21.15.19:0/1061225187 conn(0x7f21740d1720 msgr2=0x7f21740d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.444+0000 7f217a600640 1 -- 172.21.15.19:0/1061225187 shutdown_connections 2024-04-25T11:30:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.444+0000 7f217a600640 1 -- 172.21.15.19:0/1061225187 wait complete. 2024-04-25T11:30:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.448+0000 7f217a600640 1 Processor -- start 2024-04-25T11:30:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.448+0000 7f217a600640 1 -- start start 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f217a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f217406c5c0 0x7f2174153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f217a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174144370 0x7f2174140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f217a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f217415ada0 0x7f217415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f217a600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21740d0450 con 0x7f217415ada0 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f217a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21740d02d0 con 0x7f2174144370 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f217a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2174147760 con 0x7f217406c5c0 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f2179c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f217415ada0 0x7f217415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f2179c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f217415ada0 0x7f217415b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38142/0 (socket says 172.21.15.19:38142) 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f2179c00640 1 -- 172.21.15.19:0/1120450028 learned_addr learned my addr 172.21.15.19:0/1120450028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f2173e00640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174144370 0x7f2174140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.450+0000 7f2179200640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f217406c5c0 0x7f2174153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2179c00640 1 -- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f217406c5c0 msgr2=0x7f2174153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2179c00640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f217406c5c0 0x7f2174153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2179c00640 1 -- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174144370 msgr2=0x7f2174140850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2179c00640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174144370 0x7f2174140850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2179c00640 1 -- 172.21.15.19:0/1120450028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217415b6f0 con 0x7f217415ada0 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2179c00640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f217415ada0 0x7f217415b1b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c3c0 tx=0x7f216c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2170c00640 1 -- 172.21.15.19:0/1120450028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c017070 con 0x7f217415ada0 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2170c00640 1 -- 172.21.15.19:0/1120450028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216c005ce0 con 0x7f217415ada0 2024-04-25T11:30:57.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217415fab0 con 0x7f217415ada0 2024-04-25T11:30:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.451+0000 7f2170c00640 1 -- 172.21.15.19:0/1120450028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c007310 con 0x7f217415ada0 2024-04-25T11:30:57.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.452+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217415fd50 con 0x7f217415ada0 2024-04-25T11:30:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.453+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2174050740 con 0x7f217415ada0 2024-04-25T11:30:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.454+0000 7f2170c00640 1 -- 172.21.15.19:0/1120450028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f216c01a070 con 0x7f217415ada0 2024-04-25T11:30:57.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.455+0000 7f2170c00640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21500744b0 0x7f21500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.455+0000 7f2170c00640 1 -- 172.21.15.19:0/1120450028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7f216c0943c0 con 0x7f217415ada0 2024-04-25T11:30:57.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.456+0000 7f2179200640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21500744b0 0x7f21500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.457+0000 7f2179200640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21500744b0 0x7f21500769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2164001ca0 tx=0x7f2164009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:57.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.460+0000 7f2170c00640 1 -- 172.21.15.19:0/1120450028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f217415b6f0 con 0x7f217415ada0 2024-04-25T11:30:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.655+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f217406d750 con 0x7f217415ada0 2024-04-25T11:30:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.658+0000 7f2170c00640 1 -- 172.21.15.19:0/1120450028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+13150 (secure 0 0 0) 0x7f216c05ed50 con 0x7f217415ada0 2024-04-25T11:30:57.648 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:30:57.648 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":54,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:30:54.102474+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3964766888":"2024-04-26T11:30:50.574739+0000","172.21.15.19:6835/3038680263":"2024-04-26T11:30:38.619698+0000","172.21.15.19:6834/3964766888":"2024-04-26T11:30:50.574739+0000","172.21.15.19:6834/3038680263":"2024-04-26T11:30:38.619698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:30:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21500744b0 msgr2=0x7f21500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21500744b0 0x7f21500769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2164001ca0 tx=0x7f2164009040 comp rx=0 tx=0).stop 2024-04-25T11:30:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f217415ada0 msgr2=0x7f217415b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f217415ada0 0x7f217415b1b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c3c0 tx=0x7f216c00c880 comp rx=0 tx=0).stop 2024-04-25T11:30:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 shutdown_connections 2024-04-25T11:30:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21500744b0 0x7f21500769c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2174144370 0x7f2174140850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f217415ada0 0x7f217415b1b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 --2- 172.21.15.19:0/1120450028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f217406c5c0 0x7f2174153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 >> 172.21.15.19:0/1120450028 conn(0x7f21740d1720 msgr2=0x7f217406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 shutdown_connections 2024-04-25T11:30:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.660+0000 7f217a600640 1 -- 172.21.15.19:0/1120450028 wait complete. 2024-04-25T11:30:57.677 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:30:57.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:30:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.956+0000 7fafcaa00640 1 Processor -- start 2024-04-25T11:30:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.956+0000 7fafcaa00640 1 -- start start 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.958+0000 7fafcaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc40ce520 0x7fafc415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.958+0000 7fafcaa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fafc40ce930 0x7fafc415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.958+0000 7fafcaa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fafc406d5d0 0x7fafc406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.958+0000 7fafcaa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fafc406df20 con 0x7fafc406d5d0 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.958+0000 7fafcaa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fafc40cda60 con 0x7fafc40ce930 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.958+0000 7fafcaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fafc415dfe0 con 0x7fafc40ce520 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.958+0000 7fafca000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fafc406d5d0 0x7fafc406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafca000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fafc406d5d0 0x7fafc406d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38150/0 (socket says 172.21.15.19:38150) 2024-04-25T11:30:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafca000640 1 -- 172.21.15.19:0/1248572988 learned_addr learned my addr 172.21.15.19:0/1248572988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafc9600640 1 --2- 172.21.15.19:0/1248572988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc40ce520 0x7fafc415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafc8c00640 1 --2- 172.21.15.19:0/1248572988 >> v2:172.21.15.169:3300/0 conn(0x7fafc40ce930 0x7fafc415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafca000640 1 -- 172.21.15.19:0/1248572988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc40ce520 msgr2=0x7fafc415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafca000640 1 --2- 172.21.15.19:0/1248572988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc40ce520 0x7fafc415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafca000640 1 -- 172.21.15.19:0/1248572988 >> v2:172.21.15.169:3300/0 conn(0x7fafc40ce930 msgr2=0x7fafc415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafca000640 1 --2- 172.21.15.19:0/1248572988 >> v2:172.21.15.169:3300/0 conn(0x7fafc40ce930 0x7fafc415d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.959+0000 7fafca000640 1 -- 172.21.15.19:0/1248572988 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc415e2c0 con 0x7fafc406d5d0 2024-04-25T11:30:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.960+0000 7fafca000640 1 --2- 172.21.15.19:0/1248572988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc406d5d0 0x7fafc406d9e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fafc40712f0 tx=0x7fafbc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208c57cd397fecf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafc3e00640 1 -- 172.21.15.19:0/1248572988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc036070 con 0x7fafc406d5d0 2024-04-25T11:30:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafc3e00640 1 -- 172.21.15.19:0/1248572988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc031600 con 0x7fafc406d5d0 2024-04-25T11:30:57.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafc3e00640 1 -- 172.21.15.19:0/1248572988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0319d0 con 0x7fafc406d5d0 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafcaa00640 1 -- 172.21.15.19:0/1248572988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc406d5d0 msgr2=0x7fafc406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafcaa00640 1 --2- 172.21.15.19:0/1248572988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc406d5d0 0x7fafc406d9e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fafc40712f0 tx=0x7fafbc02a610 comp rx=0 tx=0).stop 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafcaa00640 1 -- 172.21.15.19:0/1248572988 shutdown_connections 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafcaa00640 1 --2- 172.21.15.19:0/1248572988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc406d5d0 0x7fafc406d9e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafcaa00640 1 --2- 172.21.15.19:0/1248572988 >> v2:172.21.15.169:3300/0 conn(0x7fafc40ce930 0x7fafc415d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafcaa00640 1 --2- 172.21.15.19:0/1248572988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc40ce520 0x7fafc415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.961+0000 7fafcaa00640 1 -- 172.21.15.19:0/1248572988 >> 172.21.15.19:0/1248572988 conn(0x7fafc4072f40 msgr2=0x7fafc4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.962+0000 7fafcaa00640 1 -- 172.21.15.19:0/1248572988 shutdown_connections 2024-04-25T11:30:57.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.962+0000 7fafcaa00640 1 -- 172.21.15.19:0/1248572988 wait complete. 2024-04-25T11:30:57.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.964+0000 7fafcaa00640 1 Processor -- start 2024-04-25T11:30:57.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.964+0000 7fafcaa00640 1 -- start start 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafcaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc406d5d0 0x7fafc414b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafcaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40ce520 0x7fafc4159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafcaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafc40ce930 0x7fafc40d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafcaa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafc4160d30 con 0x7fafc40ce520 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafcaa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fafc40ce000 con 0x7fafc40ce930 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafcaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fafc40b73d0 con 0x7fafc406d5d0 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40ce520 0x7fafc4159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40ce520 0x7fafc4159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38166/0 (socket says 172.21.15.19:38166) 2024-04-25T11:30:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 -- 172.21.15.19:0/2846923654 learned_addr learned my addr 172.21.15.19:0/2846923654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:57.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc9600640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc406d5d0 0x7fafc414b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafca000640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafc40ce930 0x7fafc40d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 -- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc406d5d0 msgr2=0x7fafc414b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc406d5d0 0x7fafc414b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 -- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafc40ce930 msgr2=0x7fafc40d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafc40ce930 0x7fafc40d00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.966+0000 7fafc8c00640 1 -- 172.21.15.19:0/2846923654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc4150130 con 0x7fafc40ce520 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.967+0000 7fafc8c00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40ce520 0x7fafc4159fc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fafb000d5c0 tx=0x7fafb000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.967+0000 7fafc1600640 1 -- 172.21.15.19:0/2846923654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00125f0 con 0x7fafc40ce520 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.967+0000 7fafc1600640 1 -- 172.21.15.19:0/2846923654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb0019070 con 0x7fafc40ce520 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.967+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc4150410 con 0x7fafc40ce520 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.967+0000 7fafc1600640 1 -- 172.21.15.19:0/2846923654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00129d0 con 0x7fafc40ce520 2024-04-25T11:30:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.967+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc41509d0 con 0x7fafc40ce520 2024-04-25T11:30:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.968+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc4076c60 con 0x7fafc40ce520 2024-04-25T11:30:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.970+0000 7fafc1600640 1 -- 172.21.15.19:0/2846923654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fafb0012b70 con 0x7fafc40ce520 2024-04-25T11:30:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.971+0000 7fafc1600640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafa40744b0 0x7fafa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.971+0000 7fafc1600640 1 -- 172.21.15.19:0/2846923654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7fafb0095ef0 con 0x7fafc40ce520 2024-04-25T11:30:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.972+0000 7fafc9600640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafa40744b0 0x7fafa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.973+0000 7fafc9600640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafa40744b0 0x7fafa40769c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006fd0 tx=0x7fafb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:57.975+0000 7fafc1600640 1 -- 172.21.15.19:0/2846923654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafb0060900 con 0x7fafc40ce520 2024-04-25T11:30:58.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.178+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafc4068d20 con 0x7fafc40ce520 2024-04-25T11:30:58.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.179+0000 7fafc1600640 1 -- 172.21.15.19:0/2846923654 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7fafb0060200 con 0x7fafc40ce520 2024-04-25T11:30:58.169 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafa40744b0 msgr2=0x7fafa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafa40744b0 0x7fafa40769c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006fd0 tx=0x7fafb4008040 comp rx=0 tx=0).stop 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40ce520 msgr2=0x7fafc4159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40ce520 0x7fafc4159fc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fafb000d5c0 tx=0x7fafb000da80 comp rx=0 tx=0).stop 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 shutdown_connections 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafa40744b0 0x7fafa40769c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafc40ce930 0x7fafc40d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc40ce520 0x7fafc4159fc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 --2- 172.21.15.19:0/2846923654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafc406d5d0 0x7fafc414b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 >> 172.21.15.19:0/2846923654 conn(0x7fafc4072f40 msgr2=0x7fafc415b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 shutdown_connections 2024-04-25T11:30:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.182+0000 7fafcaa00640 1 -- 172.21.15.19:0/2846923654 wait complete. 2024-04-25T11:30:58.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-04-25T11:30:58.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.494+0000 7fdb43e00640 1 Processor -- start 2024-04-25T11:30:58.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.494+0000 7fdb43e00640 1 -- start start 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb43e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb3c1443c0 0x7fdb3c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb43e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdb3c070370 0x7fdb3c0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb43e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb3c0cff30 con 0x7fdb3c0cfb20 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb43e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdb3c0708d0 con 0x7fdb3c1443c0 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb43e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdb3c15f160 con 0x7fdb3c070370 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb41400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb3c1443c0 0x7fdb3c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb41400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb3c1443c0 0x7fdb3c06fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38182/0 (socket says 172.21.15.19:38182) 2024-04-25T11:30:58.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.497+0000 7fdb41400640 1 -- 172.21.15.19:0/2309702302 learned_addr learned my addr 172.21.15.19:0/2309702302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:58.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.498+0000 7fdb40a00640 1 --2- 172.21.15.19:0/2309702302 >> v2:172.21.15.169:3300/0 conn(0x7fdb3c070370 0x7fdb3c0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:58.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.498+0000 7fdb41400640 1 -- 172.21.15.19:0/2309702302 >> v2:172.21.15.169:3300/0 conn(0x7fdb3c070370 msgr2=0x7fdb3c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:58.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.498+0000 7fdb41400640 1 --2- 172.21.15.19:0/2309702302 >> v2:172.21.15.169:3300/0 conn(0x7fdb3c070370 0x7fdb3c0cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.498+0000 7fdb41400640 1 -- 172.21.15.19:0/2309702302 >> v1:172.21.15.169:6789/0 conn(0x7fdb3c0cfb20 legacy=0x7fdb3c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:58.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.498+0000 7fdb41400640 1 -- 172.21.15.19:0/2309702302 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c15f5f0 con 0x7fdb3c1443c0 2024-04-25T11:30:58.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.499+0000 7fdb41400640 1 --2- 172.21.15.19:0/2309702302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c1443c0 0x7fdb3c06fe30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c06f920 tx=0x7fdb3002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a383a3cba0e1303d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:58.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.499+0000 7fdb3be00640 1 -- 172.21.15.19:0/2309702302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30037070 con 0x7fdb3c1443c0 2024-04-25T11:30:58.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.499+0000 7fdb3be00640 1 -- 172.21.15.19:0/2309702302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb30032440 con 0x7fdb3c1443c0 2024-04-25T11:30:58.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.499+0000 7fdb3be00640 1 -- 172.21.15.19:0/2309702302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30032810 con 0x7fdb3c1443c0 2024-04-25T11:30:58.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.500+0000 7fdb43e00640 1 -- 172.21.15.19:0/2309702302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c1443c0 msgr2=0x7fdb3c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:58.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.500+0000 7fdb43e00640 1 --2- 172.21.15.19:0/2309702302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c1443c0 0x7fdb3c06fe30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c06f920 tx=0x7fdb3002a230 comp rx=0 tx=0).stop 2024-04-25T11:30:58.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.501+0000 7fdb43e00640 1 -- 172.21.15.19:0/2309702302 shutdown_connections 2024-04-25T11:30:58.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.501+0000 7fdb43e00640 1 --2- 172.21.15.19:0/2309702302 >> v2:172.21.15.169:3300/0 conn(0x7fdb3c070370 0x7fdb3c0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.501+0000 7fdb43e00640 1 --2- 172.21.15.19:0/2309702302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c1443c0 0x7fdb3c06fe30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.501+0000 7fdb43e00640 1 -- 172.21.15.19:0/2309702302 >> 172.21.15.19:0/2309702302 conn(0x7fdb3c0d1ae0 msgr2=0x7fdb3c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:58.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.501+0000 7fdb43e00640 1 -- 172.21.15.19:0/2309702302 shutdown_connections 2024-04-25T11:30:58.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.501+0000 7fdb43e00640 1 -- 172.21.15.19:0/2309702302 wait complete. 2024-04-25T11:30:58.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.503+0000 7fdb43e00640 1 Processor -- start 2024-04-25T11:30:58.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.503+0000 7fdb43e00640 1 -- start start 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c070370 0x7fdb3c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c0cfb20 0x7fdb3c1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb43e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c1443c0 0x7fdb3c1421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb43e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c15edf0 con 0x7fdb3c070370 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb43e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c15eb10 con 0x7fdb3c0cfb20 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb43e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3c06d5d0 con 0x7fdb3c1443c0 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb41400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c070370 0x7fdb3c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:58.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb41400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c070370 0x7fdb3c155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38196/0 (socket says 172.21.15.19:38196) 2024-04-25T11:30:58.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.505+0000 7fdb41400640 1 -- 172.21.15.19:0/1740544035 learned_addr learned my addr 172.21.15.19:0/1740544035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41400640 1 -- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c1443c0 msgr2=0x7fdb3c1421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb40a00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c0cfb20 0x7fdb3c1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c1443c0 0x7fdb3c1421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41400640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c1443c0 0x7fdb3c1421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41400640 1 -- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c0cfb20 msgr2=0x7fdb3c1590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41400640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c0cfb20 0x7fdb3c1590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41400640 1 -- 172.21.15.19:0/1740544035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c140950 con 0x7fdb3c070370 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c1443c0 0x7fdb3c1421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:58.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb40a00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c0cfb20 0x7fdb3c1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:30:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.506+0000 7fdb41400640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c070370 0x7fdb3c155260 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdb3002a270 tx=0x7fdb300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.507+0000 7fdb39600640 1 -- 172.21.15.19:0/1740544035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30041070 con 0x7fdb3c070370 2024-04-25T11:30:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.507+0000 7fdb39600640 1 -- 172.21.15.19:0/1740544035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3003c870 con 0x7fdb3c070370 2024-04-25T11:30:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.507+0000 7fdb39600640 1 -- 172.21.15.19:0/1740544035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30037040 con 0x7fdb3c070370 2024-04-25T11:30:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.507+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c158010 con 0x7fdb3c070370 2024-04-25T11:30:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.507+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c1585d0 con 0x7fdb3c070370 2024-04-25T11:30:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.508+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3c050740 con 0x7fdb3c070370 2024-04-25T11:30:58.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.509+0000 7fdb39600640 1 -- 172.21.15.19:0/1740544035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdb30044070 con 0x7fdb3c070370 2024-04-25T11:30:58.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.510+0000 7fdb39600640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:58.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.510+0000 7fdb39600640 1 -- 172.21.15.19:0/1740544035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7fdb300b5bc0 con 0x7fdb3c070370 2024-04-25T11:30:58.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.511+0000 7fdb40a00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:58.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.512+0000 7fdb40a00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c160af0 tx=0x7fdb28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:58.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.515+0000 7fdb39600640 1 -- 172.21.15.19:0/1740544035 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb3c0cff30 con 0x7fdb3c070370 2024-04-25T11:30:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:58.791+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7fdb3c06f950 con 0x7fdb3c070370 2024-04-25T11:30:59.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.143+0000 7fdb39600640 1 -- 172.21.15.19:0/1740544035 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v210) ==== 145+0+0 (secure 0 0 0) 0x7fdb300805e0 con 0x7fdb3c070370 2024-04-25T11:30:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 msgr2=0x7fdb1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c160af0 tx=0x7fdb28008040 comp rx=0 tx=0).stop 2024-04-25T11:30:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c070370 msgr2=0x7fdb3c155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c070370 0x7fdb3c155260 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdb3002a270 tx=0x7fdb300046c0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 shutdown_connections 2024-04-25T11:30:59.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb1c0744e0 0x7fdb1c0769f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb3c0cfb20 0x7fdb3c1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb3c1443c0 0x7fdb3c1421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 --2- 172.21.15.19:0/1740544035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb3c070370 0x7fdb3c155260 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.146+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 >> 172.21.15.19:0/1740544035 conn(0x7fdb3c0d1ae0 msgr2=0x7fdb3c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:59.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.147+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 shutdown_connections 2024-04-25T11:30:59.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.147+0000 7fdb43e00640 1 -- 172.21.15.19:0/1740544035 wait complete. 2024-04-25T11:30:59.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:30:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.455+0000 7f8252200640 1 Processor -- start 2024-04-25T11:30:59.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.455+0000 7f8252200640 1 -- start start 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.457+0000 7f8252200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f824c06dce0 0x7f824c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.457+0000 7f8252200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 0x7f824c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.457+0000 7f8252200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c070000 con 0x7f824c144340 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.457+0000 7f8252200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f824c144750 con 0x7f824c06dce0 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.457+0000 7f8252200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f824c06f240 con 0x7f824c15ec90 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.458+0000 7f8250e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f824c144340 0x7f824c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50792/0 (socket says 172.21.15.19:50792) 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.458+0000 7f8250e00640 1 -- 172.21.15.19:0/4265639971 learned_addr learned my addr 172.21.15.19:0/4265639971 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.458+0000 7f8251800640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 0x7f824c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.458+0000 7f824be00640 1 --2- 172.21.15.19:0/4265639971 >> v2:172.21.15.169:3300/0 conn(0x7f824c06dce0 0x7f824c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.458+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396176773 0 0) 0x7f824c070000 con 0x7f824c144340 2024-04-25T11:30:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8234003520 con 0x7f824c144340 2024-04-25T11:30:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3310411361 0 0) 0x7f8234003520 con 0x7f824c144340 2024-04-25T11:30:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 msgr2=0x7f824c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 0x7f824c15f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 >> v2:172.21.15.169:3300/0 conn(0x7f824c06dce0 msgr2=0x7f824c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 --2- 172.21.15.19:0/4265639971 >> v2:172.21.15.169:3300/0 conn(0x7f824c06dce0 0x7f824c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c06f520 con 0x7f824c144340 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8240001670 con 0x7f824c144340 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 >> v1:172.21.15.19:6789/0 conn(0x7f824c144340 legacy=0x7f824c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 0x7f82340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f8234004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8234004d40 0x7f8234005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f824c144750 con 0x7f8234003f80 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f824c06f240 con 0x7f8234004d40 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.459+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f823400ba70 con 0x7f824c15ec90 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f824be00640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f8234004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f8251800640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8234004d40 0x7f8234005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f8250e00640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 0x7f82340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f824be00640 1 -- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 msgr2=0x7f82340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f824be00640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c15ec90 0x7f82340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f824be00640 1 -- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8234004d40 msgr2=0x7f8234005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f824be00640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8234004d40 0x7f8234005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.460+0000 7f824be00640 1 -- 172.21.15.19:0/4265639971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8234003520 con 0x7f8234003f80 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.461+0000 7f824be00640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f8234004800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8238009a90 tx=0x7f8234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32fdd892a36dfc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.462+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238036070 con 0x7f8234003f80 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.462+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8238031400 con 0x7f8234003f80 2024-04-25T11:30:59.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.462+0000 7f824b400640 1 -- 172.21.15.19:0/4265639971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238009690 con 0x7f8234003f80 2024-04-25T11:30:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.462+0000 7f8252200640 1 -- 172.21.15.19:0/4265639971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 msgr2=0x7f8234004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.462+0000 7f8252200640 1 --2- 172.21.15.19:0/4265639971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f8234004800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8238009a90 tx=0x7f8234005830 comp rx=0 tx=0).stop 2024-04-25T11:30:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.462+0000 7f8251800640 1 -- 172.21.15.19:0/4265639971 reap_dead start 2024-04-25T11:30:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.463+0000 7f8252200640 1 -- 172.21.15.19:0/4265639971 shutdown_connections 2024-04-25T11:30:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.463+0000 7f8252200640 1 -- 172.21.15.19:0/4265639971 >> 172.21.15.19:0/4265639971 conn(0x7f824c072f40 msgr2=0x7f824c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.463+0000 7f8252200640 1 -- 172.21.15.19:0/4265639971 shutdown_connections 2024-04-25T11:30:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.463+0000 7f8252200640 1 -- 172.21.15.19:0/4265639971 wait complete. 2024-04-25T11:30:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.465+0000 7f8252200640 1 Processor -- start 2024-04-25T11:30:59.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.466+0000 7f8252200640 1 -- start start 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8252200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f824c15dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8252200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c140050 0x7f824c13c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8252200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c140460 0x7f824c13c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8252200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f824c15f0a0 con 0x7f8234003f80 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8252200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f824c06fae0 con 0x7f824c140460 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8252200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f824c15f220 con 0x7f824c140050 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8250e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f824c15dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8250e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f824c15dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38210/0 (socket says 172.21.15.19:38210) 2024-04-25T11:30:59.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.467+0000 7f8250e00640 1 -- 172.21.15.19:0/690174102 learned_addr learned my addr 172.21.15.19:0/690174102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.468+0000 7f8251800640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c140460 0x7f824c13c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.468+0000 7f8250e00640 1 -- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c140050 msgr2=0x7f824c13c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.468+0000 7f8250e00640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c140050 0x7f824c13c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.468+0000 7f8250e00640 1 -- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c140460 msgr2=0x7f824c13c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.468+0000 7f8250e00640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c140460 0x7f824c13c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.468+0000 7f8250e00640 1 -- 172.21.15.19:0/690174102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c0b4890 con 0x7f8234003f80 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.468+0000 7f8250e00640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f824c15dce0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f824000e780 tx=0x7f824000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.469+0000 7f8248c00640 1 -- 172.21.15.19:0/690174102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000ee90 con 0x7f8234003f80 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.469+0000 7f8248c00640 1 -- 172.21.15.19:0/690174102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240018070 con 0x7f8234003f80 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.469+0000 7f8248c00640 1 -- 172.21.15.19:0/690174102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000caf0 con 0x7f8234003f80 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.469+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c0b5ee0 con 0x7f8234003f80 2024-04-25T11:30:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.469+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c0b64a0 con 0x7f8234003f80 2024-04-25T11:30:59.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.470+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824c076c60 con 0x7f8234003f80 2024-04-25T11:30:59.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.471+0000 7f8248c00640 1 -- 172.21.15.19:0/690174102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f824000cc90 con 0x7f8234003f80 2024-04-25T11:30:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.472+0000 7f8248c00640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f822c0745f0 0x7f822c076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:30:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.472+0000 7f8248c00640 1 -- 172.21.15.19:0/690174102 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7f8240094e20 con 0x7f8234003f80 2024-04-25T11:30:59.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.473+0000 7f824be00640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f822c0745f0 0x7f822c076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:30:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.474+0000 7f824be00640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f822c0745f0 0x7f822c076b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8238033000 tx=0x7f823802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:30:59.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.477+0000 7f8248c00640 1 -- 172.21.15.19:0/690174102 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f824005f8d0 con 0x7f8234003f80 2024-04-25T11:30:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.684+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f824c0b6c40 con 0x7f8234003f80 2024-04-25T11:30:59.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.693+0000 7f8248c00640 1 -- 172.21.15.19:0/690174102 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f824005f1d0 con 0x7f8234003f80 2024-04-25T11:30:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f822c0745f0 msgr2=0x7f822c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f822c0745f0 0x7f822c076b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8238033000 tx=0x7f823802e000 comp rx=0 tx=0).stop 2024-04-25T11:30:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 msgr2=0x7f824c15dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:30:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f824c15dce0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f824000e780 tx=0x7f824000ec40 comp rx=0 tx=0).stop 2024-04-25T11:30:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 shutdown_connections 2024-04-25T11:30:59.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f822c0745f0 0x7f822c076b00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f824c140460 0x7f824c13c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f824c140050 0x7f824c13c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 --2- 172.21.15.19:0/690174102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8234003f80 0x7f824c15dce0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:30:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 >> 172.21.15.19:0/690174102 conn(0x7f824c072f40 msgr2=0x7f824c1421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:30:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.696+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 shutdown_connections 2024-04-25T11:30:59.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:30:59.697+0000 7f8252200640 1 -- 172.21.15.19:0/690174102 wait complete. 2024-04-25T11:30:59.714 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:30:59.714 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:30:59.820 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:30:59.938 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:30:59.938 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:30:59.938 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:30:59.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:30:59.999 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:30:59.999 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:31:00.062 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:31:00.062 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:31:00.062 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:31:00.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:31:00.121 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:31:00.121 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:31:00.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:31:00.179 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:31:00.179 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:00.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.484+0000 7fbb81200640 1 Processor -- start 2024-04-25T11:31:00.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.485+0000 7fbb81200640 1 -- start start 2024-04-25T11:31:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.487+0000 7fbb81200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb7c0cf2f0 0x7fbb7c0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.487+0000 7fbb81200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb7c0b74d0 con 0x7fbb7c1608d0 2024-04-25T11:31:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.487+0000 7fbb81200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb7c0b77b0 con 0x7fbb7c15a810 2024-04-25T11:31:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.487+0000 7fbb81200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbb7c06f870 con 0x7fbb7c0cf2f0 2024-04-25T11:31:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.488+0000 7fbb7be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbb7c1608d0 0x7fbb7c160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50800/0 (socket says 172.21.15.19:50800) 2024-04-25T11:31:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.488+0000 7fbb7be00640 1 -- 172.21.15.19:0/327458255 learned_addr learned my addr 172.21.15.19:0/327458255 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.488+0000 7fbb7b400640 1 --2- 172.21.15.19:0/327458255 >> v2:172.21.15.169:3300/0 conn(0x7fbb7c0cf2f0 0x7fbb7c0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.488+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836376762 0 0) 0x7fbb7c0b74d0 con 0x7fbb7c1608d0 2024-04-25T11:31:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.488+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb60005730 con 0x7fbb7c1608d0 2024-04-25T11:31:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.488+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 136434977 0 0) 0x7fbb60005730 con 0x7fbb7c1608d0 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 >> v2:172.21.15.169:3300/0 conn(0x7fbb7c0cf2f0 msgr2=0x7fbb7c0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 --2- 172.21.15.19:0/327458255 >> v2:172.21.15.169:3300/0 conn(0x7fbb7c0cf2f0 0x7fbb7c0b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 >> v1:172.21.15.169:6789/0 conn(0x7fbb7c15a810 legacy=0x7fbb7c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c06f220 con 0x7fbb7c1608d0 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fbb740019f0 con 0x7fbb7c1608d0 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 >> v1:172.21.15.19:6789/0 conn(0x7fbb7c1608d0 legacy=0x7fbb7c160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb60005c10 0x7fbb600060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60006a10 0x7fbb60006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb60007430 0x7fbb6000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c06f870 con 0x7fbb60007430 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6000e3c0 con 0x7fbb60006a10 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6000e5c0 con 0x7fbb60005c10 2024-04-25T11:31:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7be00640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb60007430 0x7fbb6000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7b400640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb60005c10 0x7fbb600060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.489+0000 7fbb7aa00640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60006a10 0x7fbb60006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.490+0000 7fbb7be00640 1 -- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb60005c10 msgr2=0x7fbb600060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.490+0000 7fbb7be00640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb60005c10 0x7fbb600060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.490+0000 7fbb7be00640 1 -- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60006a10 msgr2=0x7fbb60006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.490+0000 7fbb7be00640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60006a10 0x7fbb60006e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.490+0000 7fbb7be00640 1 -- 172.21.15.19:0/327458255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb60005730 con 0x7fbb60007430 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.490+0000 7fbb7be00640 1 -- 172.21.15.19:0/327458255 reap_dead start 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.491+0000 7fbb7be00640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb60007430 0x7fbb6000dbd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400a740 tx=0x7fbb60007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adab0488b6c3e849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.491+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400a390 con 0x7fbb60007430 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.491+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb7402f070 con 0x7fbb60007430 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.491+0000 7fbb7a000640 1 -- 172.21.15.19:0/327458255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7402bbc0 con 0x7fbb60007430 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.492+0000 7fbb81200640 1 -- 172.21.15.19:0/327458255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb60007430 msgr2=0x7fbb6000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.492+0000 7fbb81200640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb60007430 0x7fbb6000dbd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400a740 tx=0x7fbb60007a40 comp rx=0 tx=0).stop 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.492+0000 7fbb81200640 1 -- 172.21.15.19:0/327458255 shutdown_connections 2024-04-25T11:31:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.492+0000 7fbb81200640 1 --2- 172.21.15.19:0/327458255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb60007430 0x7fbb6000dbd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.492+0000 7fbb81200640 1 -- 172.21.15.19:0/327458255 >> 172.21.15.19:0/327458255 conn(0x7fbb7c0d1720 msgr2=0x7fbb7c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.492+0000 7fbb81200640 1 -- 172.21.15.19:0/327458255 shutdown_connections 2024-04-25T11:31:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.492+0000 7fbb81200640 1 -- 172.21.15.19:0/327458255 wait complete. 2024-04-25T11:31:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.494+0000 7fbb81200640 1 Processor -- start 2024-04-25T11:31:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.494+0000 7fbb81200640 1 -- start start 2024-04-25T11:31:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb81200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60007430 0x7fbb7c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb81200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1f92d0 0x7fbb7c146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb81200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c1f96e0 0x7fbb7c1472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb81200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c0b6e10 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb81200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c15ac20 con 0x7fbb60007430 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb81200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c0b6f90 con 0x7fbb7c1f96e0 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb7aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1f92d0 0x7fbb7c146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb7aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1f92d0 0x7fbb7c146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38224/0 (socket says 172.21.15.19:38224) 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.496+0000 7fbb7aa00640 1 -- 172.21.15.19:0/1018445097 learned_addr learned my addr 172.21.15.19:0/1018445097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7b400640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60007430 0x7fbb7c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7be00640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c1f96e0 0x7fbb7c1472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7aa00640 1 -- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c1f96e0 msgr2=0x7fbb7c1472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7aa00640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c1f96e0 0x7fbb7c1472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7aa00640 1 -- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60007430 msgr2=0x7fbb7c155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7aa00640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60007430 0x7fbb7c155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7aa00640 1 -- 172.21.15.19:0/1018445097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c0af0f0 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.497+0000 7fbb7aa00640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1f92d0 0x7fbb7c146d90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800e4d0 tx=0x7fbb6800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.498+0000 7fbb73400640 1 -- 172.21.15.19:0/1018445097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb680052e0 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.498+0000 7fbb73400640 1 -- 172.21.15.19:0/1018445097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb68018070 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.498+0000 7fbb73400640 1 -- 172.21.15.19:0/1018445097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68003810 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.498+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c0af3d0 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.498+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c15e6d0 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.499+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c050740 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.500+0000 7fbb73400640 1 -- 172.21.15.19:0/1018445097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb680039b0 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.501+0000 7fbb73400640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb5c0744b0 0x7fbb5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.502+0000 7fbb73400640 1 -- 172.21.15.19:0/1018445097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5029+0+0 (secure 0 0 0) 0x7fbb68094c10 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.502+0000 7fbb7b400640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb5c0744b0 0x7fbb5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.503+0000 7fbb7b400640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb5c0744b0 0x7fbb5c0769c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009990 tx=0x7fbb6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.506+0000 7fbb73400640 1 -- 172.21.15.19:0/1018445097 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb6805f630 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.749+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb7c06c770 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.751+0000 7fbb73400640 1 -- 172.21.15.19:0/1018445097 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+3872 (secure 0 0 0) 0x7fbb6805ef30 con 0x7fbb7c1f92d0 2024-04-25T11:31:00.740 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:00.740 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3562625879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":3562625879},{"type":"v1","addr":"172.21.15.169:6837","nonce":3562625879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5164,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2834560541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2834560541},{"type":"v1","addr":"172.21.15.19:6833","nonce":2834560541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/129229089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":129229089},{"type":"v1","addr":"172.21.15.169:6835","nonce":129229089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3450731986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3450731986},{"type":"v1","addr":"172.21.15.19:6835","nonce":3450731986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[]} 2024-04-25T11:31:00.741 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 44 2024-04-25T11:31:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb5c0744b0 msgr2=0x7fbb5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb5c0744b0 0x7fbb5c0769c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009990 tx=0x7fbb6c008040 comp rx=0 tx=0).stop 2024-04-25T11:31:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1f92d0 msgr2=0x7fbb7c146d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1f92d0 0x7fbb7c146d90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800e4d0 tx=0x7fbb6800e990 comp rx=0 tx=0).stop 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 shutdown_connections 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbb5c0744b0 0x7fbb5c0769c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb7c1f96e0 0x7fbb7c1472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.753+0000 7fbb81200640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb7c1f92d0 0x7fbb7c146d90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.754+0000 7fbb81200640 1 --2- 172.21.15.19:0/1018445097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb60007430 0x7fbb7c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.754+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 >> 172.21.15.19:0/1018445097 conn(0x7fbb7c0d1720 msgr2=0x7fbb7c0cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.754+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 shutdown_connections 2024-04-25T11:31:00.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:00.754+0000 7fbb81200640 1 -- 172.21.15.19:0/1018445097 wait complete. 2024-04-25T11:31:00.771 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:31:00.771 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:31:00.772 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:31:00.772 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:31:00.772 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:31:00.772 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:31:00.772 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:31:00.772 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:31:00.772 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:31:00.772 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:31:00.773 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:31:00.773 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:31:00.773 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:31:00.773 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:31:00.773 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:31:00.773 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:31:00.797 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:31:00.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:31:00.802 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:31:00.802 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:31:00.803 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:31:00.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:31:00.817 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:31:00.817 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:31:00.850 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:31:00.851 DEBUG:teuthology.parallel:result is None 2024-04-25T11:31:00.857 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:31:00.857 DEBUG:teuthology.parallel:result is None 2024-04-25T11:31:00.864 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:31:00.865 DEBUG:teuthology.parallel:result is None 2024-04-25T11:31:00.871 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:31:00.871 DEBUG:teuthology.parallel:result is None 2024-04-25T11:31:00.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:31:01.078 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:31:01.089 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:31:01.096 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:31:01.101 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:31:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.140+0000 7f7d57000640 1 Processor -- start 2024-04-25T11:31:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.140+0000 7f7d57000640 1 -- start start 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d57000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7d500d0e00 0x7f7d500d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d57000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7d5006c5c0 0x7f7d5006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d57000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d5006cf10 con 0x7f7d500b7810 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d57000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7d500d5a60 con 0x7f7d500d0e00 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d57000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7d500d1850 con 0x7f7d5006c5c0 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d4fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7d500d0e00 0x7f7d500d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d55000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7d500b7810 0x7f7d500b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50812/0 (socket says 172.21.15.19:50812) 2024-04-25T11:31:01.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.142+0000 7f7d55000640 1 -- 172.21.15.19:0/3045409290 learned_addr learned my addr 172.21.15.19:0/3045409290 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.143+0000 7f7d47400640 1 --2- 172.21.15.19:0/3045409290 >> v2:172.21.15.169:3300/0 conn(0x7f7d5006c5c0 0x7f7d5006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.143+0000 7f7d4fe00640 1 -- 172.21.15.19:0/3045409290 >> v2:172.21.15.169:3300/0 conn(0x7f7d5006c5c0 msgr2=0x7f7d5006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.143+0000 7f7d4fe00640 1 --2- 172.21.15.19:0/3045409290 >> v2:172.21.15.169:3300/0 conn(0x7f7d5006c5c0 0x7f7d5006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.143+0000 7f7d4fe00640 1 -- 172.21.15.19:0/3045409290 >> v1:172.21.15.19:6789/0 conn(0x7f7d500b7810 legacy=0x7f7d500b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.143+0000 7f7d4fe00640 1 -- 172.21.15.19:0/3045409290 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d50160de0 con 0x7f7d500d0e00 2024-04-25T11:31:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.143+0000 7f7d4f400640 1 -- 172.21.15.19:0/3045409290 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235795367 0 0) 0x7f7d48001110 con 0x7f7d500b7810 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.144+0000 7f7d4fe00640 1 --2- 172.21.15.19:0/3045409290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500d0e00 0x7f7d500d1210 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7d500d5be0 tx=0x7f7d4002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8470e98c1028bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.144+0000 7f7d4f400640 1 -- 172.21.15.19:0/3045409290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40031600 con 0x7f7d500d0e00 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.144+0000 7f7d4f400640 1 -- 172.21.15.19:0/3045409290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d400317a0 con 0x7f7d500d0e00 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.144+0000 7f7d4f400640 1 -- 172.21.15.19:0/3045409290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40031b70 con 0x7f7d500d0e00 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 -- 172.21.15.19:0/3045409290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500d0e00 msgr2=0x7f7d500d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 --2- 172.21.15.19:0/3045409290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500d0e00 0x7f7d500d1210 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7d500d5be0 tx=0x7f7d4002a5d0 comp rx=0 tx=0).stop 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 -- 172.21.15.19:0/3045409290 shutdown_connections 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 --2- 172.21.15.19:0/3045409290 >> v2:172.21.15.169:3300/0 conn(0x7f7d5006c5c0 0x7f7d5006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 --2- 172.21.15.19:0/3045409290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500d0e00 0x7f7d500d1210 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 -- 172.21.15.19:0/3045409290 >> 172.21.15.19:0/3045409290 conn(0x7f7d500d5d90 msgr2=0x7f7d500d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 -- 172.21.15.19:0/3045409290 shutdown_connections 2024-04-25T11:31:01.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.145+0000 7f7d57000640 1 -- 172.21.15.19:0/3045409290 wait complete. 2024-04-25T11:31:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.148+0000 7f7d57000640 1 Processor -- start 2024-04-25T11:31:01.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.148+0000 7f7d57000640 1 -- start start 2024-04-25T11:31:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.150+0000 7f7d57000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d5006c5c0 0x7f7d500c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.150+0000 7f7d57000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500b7810 0x7f7d5014cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.150+0000 7f7d57000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500d0e00 0x7f7d5014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.150+0000 7f7d57000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50160ad0 con 0x7f7d500b7810 2024-04-25T11:31:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.150+0000 7f7d57000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501613a0 con 0x7f7d500d0e00 2024-04-25T11:31:01.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.150+0000 7f7d57000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d500b5320 con 0x7f7d5006c5c0 2024-04-25T11:31:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500d0e00 0x7f7d5014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d4fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d5006c5c0 0x7f7d500c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500d0e00 0x7f7d5014e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:47934/0 (socket says 172.21.15.19:47934) 2024-04-25T11:31:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 -- 172.21.15.19:0/1059321570 learned_addr learned my addr 172.21.15.19:0/1059321570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d4fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d5006c5c0 0x7f7d500c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:59994/0 (socket says 172.21.15.19:59994) 2024-04-25T11:31:01.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d47400640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500b7810 0x7f7d5014cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 -- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d5006c5c0 msgr2=0x7f7d500c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d5006c5c0 0x7f7d500c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 -- 172.21.15.19:0/1059321570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500b7810 msgr2=0x7f7d5014cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500b7810 0x7f7d5014cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.151+0000 7f7d55000640 1 -- 172.21.15.19:0/1059321570 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5014d370 con 0x7f7d500d0e00 2024-04-25T11:31:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.152+0000 7f7d47400640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500b7810 0x7f7d5014cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:31:01.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.152+0000 7f7d55000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500d0e00 0x7f7d5014e8b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c410 tx=0x7f7d4800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.155+0000 7f7d4cc00640 1 -- 172.21.15.19:0/1059321570 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480074e0 con 0x7f7d500d0e00 2024-04-25T11:31:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.155+0000 7f7d4cc00640 1 -- 172.21.15.19:0/1059321570 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d480047e0 con 0x7f7d500d0e00 2024-04-25T11:31:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.155+0000 7f7d4cc00640 1 -- 172.21.15.19:0/1059321570 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48004bd0 con 0x7f7d500d0e00 2024-04-25T11:31:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.155+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5014d810 con 0x7f7d500d0e00 2024-04-25T11:31:01.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.155+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5014ddd0 con 0x7f7d500d0e00 2024-04-25T11:31:01.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.157+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d50050740 con 0x7f7d500d0e00 2024-04-25T11:31:01.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.167+0000 7f7d4cc00640 1 -- 172.21.15.19:0/1059321570 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d48011070 con 0x7f7d500d0e00 2024-04-25T11:31:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.168+0000 7f7d4cc00640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d2c0744e0 0x7f7d2c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.169+0000 7f7d4cc00640 1 -- 172.21.15.19:0/1059321570 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5029+0+0 (secure 0 0 0) 0x7f7d48094fa0 con 0x7f7d500d0e00 2024-04-25T11:31:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.169+0000 7f7d4fe00640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d2c0744e0 0x7f7d2c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.170+0000 7f7d4fe00640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d2c0744e0 0x7f7d2c0769f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7d40034000 tx=0x7f7d4002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:01.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.174+0000 7f7d4cc00640 1 -- 172.21.15.19:0/1059321570 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d5006cf10 con 0x7f7d500d0e00 2024-04-25T11:31:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:01.390+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7d5014d540 con 0x7f7d500d0e00 2024-04-25T11:31:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.197+0000 7f7d4cc00640 1 -- 172.21.15.19:0/1059321570 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) ==== 97+0+0 (secure 0 0 0) 0x7f7d4805f3f0 con 0x7f7d500d0e00 2024-04-25T11:31:02.187 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d2c0744e0 msgr2=0x7f7d2c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d2c0744e0 0x7f7d2c0769f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7d40034000 tx=0x7f7d4002f000 comp rx=0 tx=0).stop 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500d0e00 msgr2=0x7f7d5014e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500d0e00 0x7f7d5014e8b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c410 tx=0x7f7d4800c8d0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 shutdown_connections 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d2c0744e0 0x7f7d2c0769f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d500d0e00 0x7f7d5014e8b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d500b7810 0x7f7d5014cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 --2- 172.21.15.19:0/1059321570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d5006c5c0 0x7f7d500c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 >> 172.21.15.19:0/1059321570 conn(0x7f7d500d5d90 msgr2=0x7f7d50164470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 shutdown_connections 2024-04-25T11:31:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.201+0000 7f7d57000640 1 -- 172.21.15.19:0/1059321570 wait complete. 2024-04-25T11:31:02.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:31:02.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.501+0000 7fe87da00640 1 Processor -- start 2024-04-25T11:31:02.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.501+0000 7fe87da00640 1 -- start start 2024-04-25T11:31:02.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.504+0000 7fe87da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe878162160 0x7fe878140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.504+0000 7fe87da00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8780b5980 con 0x7fe878144010 2024-04-25T11:31:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.504+0000 7fe87da00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8780b5c10 con 0x7fe8780b5570 2024-04-25T11:31:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.504+0000 7fe87da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe8780b5ef0 con 0x7fe878162160 2024-04-25T11:31:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.504+0000 7fe87d000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe878144010 0x7fe87806c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50826/0 (socket says 172.21.15.19:50826) 2024-04-25T11:31:02.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.504+0000 7fe87d000640 1 -- 172.21.15.19:0/3089813260 learned_addr learned my addr 172.21.15.19:0/3089813260 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe877400640 1 --2- 172.21.15.19:0/3089813260 >> v2:172.21.15.169:3300/0 conn(0x7fe878162160 0x7fe878140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646739355 0 0) 0x7fe8780b5980 con 0x7fe878144010 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe85c003520 con 0x7fe878144010 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2566723583 0 0) 0x7fe85c003520 con 0x7fe878144010 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 >> v2:172.21.15.169:3300/0 conn(0x7fe878162160 msgr2=0x7fe878140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 --2- 172.21.15.19:0/3089813260 >> v2:172.21.15.169:3300/0 conn(0x7fe878162160 0x7fe878140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 >> v1:172.21.15.169:6789/0 conn(0x7fe8780b5570 legacy=0x7fe878161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8780b61d0 con 0x7fe878144010 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe8700019f0 con 0x7fe878144010 2024-04-25T11:31:02.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 >> v1:172.21.15.19:6789/0 conn(0x7fe878144010 legacy=0x7fe87806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c0039b0 0x7fe85c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.505+0000 7fe876a00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c004800 0x7fe85c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe876a00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe85c0051c0 0x7fe85c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780b5ef0 con 0x7fe85c0039b0 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c00c160 con 0x7fe85c0051c0 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe85c00c360 con 0x7fe85c004800 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe877e00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c0039b0 0x7fe85c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe877e00640 1 -- 172.21.15.19:0/3089813260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c004800 msgr2=0x7fe85c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe877e00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c004800 0x7fe85c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe877e00640 1 -- 172.21.15.19:0/3089813260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe85c0051c0 msgr2=0x7fe85c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe877e00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe85c0051c0 0x7fe85c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe877e00640 1 -- 172.21.15.19:0/3089813260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c003520 con 0x7fe85c0039b0 2024-04-25T11:31:02.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.506+0000 7fe87d000640 1 -- 172.21.15.19:0/3089813260 reap_dead start 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.507+0000 7fe877e00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c0039b0 0x7fe85c003e20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe868001750 tx=0x7fe85c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fb87e87f449926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868001370 con 0x7fe85c0039b0 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8680052d0 con 0x7fe85c0039b0 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe876a00640 1 -- 172.21.15.19:0/3089813260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86802ab00 con 0x7fe85c0039b0 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe87da00640 1 -- 172.21.15.19:0/3089813260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c0039b0 msgr2=0x7fe85c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe87da00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c0039b0 0x7fe85c003e20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe868001750 tx=0x7fe85c005830 comp rx=0 tx=0).stop 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe87da00640 1 -- 172.21.15.19:0/3089813260 shutdown_connections 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe87da00640 1 --2- 172.21.15.19:0/3089813260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c0039b0 0x7fe85c003e20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe87da00640 1 -- 172.21.15.19:0/3089813260 >> 172.21.15.19:0/3089813260 conn(0x7fe8780d1b40 msgr2=0x7fe8780d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:02.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe87da00640 1 -- 172.21.15.19:0/3089813260 shutdown_connections 2024-04-25T11:31:02.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.508+0000 7fe87da00640 1 -- 172.21.15.19:0/3089813260 wait complete. 2024-04-25T11:31:02.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.511+0000 7fe87da00640 1 Processor -- start 2024-04-25T11:31:02.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.511+0000 7fe87da00640 1 -- start start 2024-04-25T11:31:02.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.512+0000 7fe87da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c0039b0 0x7fe878161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8780cc2e0 0x7fe8780cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8780cd980 0x7fe8780cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe878144420 con 0x7fe8780cd980 2024-04-25T11:31:02.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780b6790 con 0x7fe8780cc2e0 2024-04-25T11:31:02.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe878140dd0 con 0x7fe85c0039b0 2024-04-25T11:31:02.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8780cd980 0x7fe8780cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8780cd980 0x7fe8780cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38288/0 (socket says 172.21.15.19:38288) 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87d000640 1 -- 172.21.15.19:0/1413430750 learned_addr learned my addr 172.21.15.19:0/1413430750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe877400640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8780cc2e0 0x7fe8780cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe877e00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c0039b0 0x7fe878161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87d000640 1 -- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c0039b0 msgr2=0x7fe878161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87d000640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c0039b0 0x7fe878161380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87d000640 1 -- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8780cc2e0 msgr2=0x7fe8780cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.513+0000 7fe87d000640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8780cc2e0 0x7fe8780cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.514+0000 7fe87d000640 1 -- 172.21.15.19:0/1413430750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe878140020 con 0x7fe8780cd980 2024-04-25T11:31:02.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.514+0000 7fe87d000640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8780cd980 0x7fe8780cdd90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe878155550 tx=0x7fe87000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:02.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.514+0000 7fe86fe00640 1 -- 172.21.15.19:0/1413430750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87000ce70 con 0x7fe8780cd980 2024-04-25T11:31:02.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.514+0000 7fe86fe00640 1 -- 172.21.15.19:0/1413430750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87000a5b0 con 0x7fe8780cd980 2024-04-25T11:31:02.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.514+0000 7fe86fe00640 1 -- 172.21.15.19:0/1413430750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87000a9a0 con 0x7fe8780cd980 2024-04-25T11:31:02.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.514+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87813fdb0 con 0x7fe8780cd980 2024-04-25T11:31:02.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.514+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe878141880 con 0x7fe8780cd980 2024-04-25T11:31:02.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.515+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe878050740 con 0x7fe8780cd980 2024-04-25T11:31:02.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.517+0000 7fe86fe00640 1 -- 172.21.15.19:0/1413430750 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe870011070 con 0x7fe8780cd980 2024-04-25T11:31:02.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.518+0000 7fe86fe00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8580744b0 0x7fe8580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:02.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.518+0000 7fe86fe00640 1 -- 172.21.15.19:0/1413430750 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fe870094980 con 0x7fe8780cd980 2024-04-25T11:31:02.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.518+0000 7fe877e00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8580744b0 0x7fe8580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:02.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.520+0000 7fe877e00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8580744b0 0x7fe8580769c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe868032000 tx=0x7fe86802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:02.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.523+0000 7fe86fe00640 1 -- 172.21.15.19:0/1413430750 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe87001a060 con 0x7fe8780cd980 2024-04-25T11:31:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.802+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe87806e100 con 0x7fe8780cd980 2024-04-25T11:31:02.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.804+0000 7fe86fe00640 1 -- 172.21.15.19:0/1413430750 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fe87005f3c0 con 0x7fe8780cd980 2024-04-25T11:31:02.793 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ==", 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.794 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw==", 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T11:31:02.795 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw==", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA==", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:31:02.796 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ==", 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.797 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw==", 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q==", 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.798 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw==", 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.799 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw==", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw==", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.800 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ==", 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA==", 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w==", 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q==", 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg==", 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q==", 2024-04-25T11:31:02.804 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNPSpmHT+MHRAAPZNLFcl/fxDgWnMeN3nAHw==", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:31:02.805 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA==", 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T11:31:02.806 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA==", 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:31:02.807 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:31:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8580744b0 msgr2=0x7fe8580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8580744b0 0x7fe8580769c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe868032000 tx=0x7fe86802e000 comp rx=0 tx=0).stop 2024-04-25T11:31:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8780cd980 msgr2=0x7fe8780cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8780cd980 0x7fe8780cdd90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe878155550 tx=0x7fe87000cc20 comp rx=0 tx=0).stop 2024-04-25T11:31:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 shutdown_connections 2024-04-25T11:31:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8580744b0 0x7fe8580769c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8780cc2e0 0x7fe8780cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8780cd980 0x7fe8780cdd90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 --2- 172.21.15.19:0/1413430750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe85c0039b0 0x7fe878161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 >> 172.21.15.19:0/1413430750 conn(0x7fe8780d1b40 msgr2=0x7fe87815c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 shutdown_connections 2024-04-25T11:31:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:02.807+0000 7fe87da00640 1 -- 172.21.15.19:0/1413430750 wait complete. 2024-04-25T11:31:02.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:31:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.121+0000 7fa05f000640 1 Processor -- start 2024-04-25T11:31:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.122+0000 7fa05f000640 1 -- start start 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.124+0000 7fa05f000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa058144500 0x7fa058070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.124+0000 7fa05f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0580cdb90 0x7fa058161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.124+0000 7fa05f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.124+0000 7fa05f000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa058070200 con 0x7fa058144500 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.124+0000 7fa05f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa05806f5f0 con 0x7fa0580cdb90 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.124+0000 7fa05f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa05806f8d0 con 0x7fa0580cdfa0 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.124+0000 7fa057e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa058144500 0x7fa058070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa057e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa058144500 0x7fa058070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38298/0 (socket says 172.21.15.19:38298) 2024-04-25T11:31:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa057e00640 1 -- 172.21.15.19:0/3330490576 learned_addr learned my addr 172.21.15.19:0/3330490576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa05d000640 1 --2- 172.21.15.19:0/3330490576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa04fe00640 1 --2- 172.21.15.19:0/3330490576 >> v2:172.21.15.169:3300/0 conn(0x7fa0580cdb90 0x7fa058161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa057e00640 1 -- 172.21.15.19:0/3330490576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 msgr2=0x7fa058162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa057e00640 1 --2- 172.21.15.19:0/3330490576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa057e00640 1 -- 172.21.15.19:0/3330490576 >> v2:172.21.15.169:3300/0 conn(0x7fa0580cdb90 msgr2=0x7fa058161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa057e00640 1 --2- 172.21.15.19:0/3330490576 >> v2:172.21.15.169:3300/0 conn(0x7fa0580cdb90 0x7fa058161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa057e00640 1 -- 172.21.15.19:0/3330490576 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05806fbb0 con 0x7fa058144500 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.125+0000 7fa04fe00640 1 --2- 172.21.15.19:0/3330490576 >> v2:172.21.15.169:3300/0 conn(0x7fa0580cdb90 0x7fa058161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.126+0000 7fa057e00640 1 --2- 172.21.15.19:0/3330490576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058070690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa0580b6df0 tx=0x7fa04802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d631e99df4fccddf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:03.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.127+0000 7fa057400640 1 -- 172.21.15.19:0/3330490576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048037070 con 0x7fa058144500 2024-04-25T11:31:03.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.127+0000 7fa057400640 1 -- 172.21.15.19:0/3330490576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa048032440 con 0x7fa058144500 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.127+0000 7fa057400640 1 -- 172.21.15.19:0/3330490576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048032810 con 0x7fa058144500 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 -- 172.21.15.19:0/3330490576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 msgr2=0x7fa058070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 --2- 172.21.15.19:0/3330490576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058070690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa0580b6df0 tx=0x7fa04802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 -- 172.21.15.19:0/3330490576 shutdown_connections 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 --2- 172.21.15.19:0/3330490576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 --2- 172.21.15.19:0/3330490576 >> v2:172.21.15.169:3300/0 conn(0x7fa0580cdb90 0x7fa058161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 --2- 172.21.15.19:0/3330490576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058070690 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 -- 172.21.15.19:0/3330490576 >> 172.21.15.19:0/3330490576 conn(0x7fa058073050 msgr2=0x7fa058073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 -- 172.21.15.19:0/3330490576 shutdown_connections 2024-04-25T11:31:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.128+0000 7fa05f000640 1 -- 172.21.15.19:0/3330490576 wait complete. 2024-04-25T11:31:03.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.131+0000 7fa05f000640 1 Processor -- start 2024-04-25T11:31:03.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.131+0000 7fa05f000640 1 -- start start 2024-04-25T11:31:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.133+0000 7fa05f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0580cdb90 0x7fa05814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.133+0000 7fa05f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.133+0000 7fa05f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.133+0000 7fa05f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa05806f2e0 con 0x7fa058144500 2024-04-25T11:31:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.133+0000 7fa05f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa05806f160 con 0x7fa0580cdb90 2024-04-25T11:31:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.133+0000 7fa05f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa0580ce8c0 con 0x7fa0580cdfa0 2024-04-25T11:31:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058142f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38300/0 (socket says 172.21.15.19:38300) 2024-04-25T11:31:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 -- 172.21.15.19:0/3080228683 learned_addr learned my addr 172.21.15.19:0/3080228683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa04fe00640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa057e00640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0580cdb90 0x7fa05814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 -- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 msgr2=0x7fa058150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 -- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0580cdb90 msgr2=0x7fa05814e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0580cdb90 0x7fa05814e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.134+0000 7fa05d000640 1 -- 172.21.15.19:0/3080228683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa058151db0 con 0x7fa058144500 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.135+0000 7fa05d000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058142f10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa050007e60 tx=0x7fa05000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.135+0000 7fa054c00640 1 -- 172.21.15.19:0/3080228683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050014420 con 0x7fa058144500 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.135+0000 7fa054c00640 1 -- 172.21.15.19:0/3080228683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050019070 con 0x7fa058144500 2024-04-25T11:31:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.135+0000 7fa054c00640 1 -- 172.21.15.19:0/3080228683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050014820 con 0x7fa058144500 2024-04-25T11:31:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.135+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa058152090 con 0x7fa058144500 2024-04-25T11:31:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.135+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa058152650 con 0x7fa058144500 2024-04-25T11:31:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.136+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa058050740 con 0x7fa058144500 2024-04-25T11:31:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.138+0000 7fa054c00640 1 -- 172.21.15.19:0/3080228683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa0500149c0 con 0x7fa058144500 2024-04-25T11:31:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.139+0000 7fa054c00640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0340744e0 0x7fa0340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.139+0000 7fa054c00640 1 -- 172.21.15.19:0/3080228683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fa050095710 con 0x7fa058144500 2024-04-25T11:31:03.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.139+0000 7fa057e00640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0340744e0 0x7fa0340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.141+0000 7fa057e00640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0340744e0 0x7fa0340769f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa048034000 tx=0x7fa04802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.143+0000 7fa054c00640 1 -- 172.21.15.19:0/3080228683 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa050060150 con 0x7fa058144500 2024-04-25T11:31:03.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.352+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0580b2f70 con 0x7fa058144500 2024-04-25T11:31:03.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.354+0000 7fa054c00640 1 -- 172.21.15.19:0/3080228683 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+12891 (secure 0 0 0) 0x7fa05005fa50 con 0x7fa058144500 2024-04-25T11:31:03.343 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:03.344 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":59,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:31:02.189221+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:31:03.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.356+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0340744e0 msgr2=0x7fa0340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.356+0000 7fa05f000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0340744e0 0x7fa0340769f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa048034000 tx=0x7fa04802e000 comp rx=0 tx=0).stop 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.356+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 msgr2=0x7fa058142f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.356+0000 7fa05f000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058142f10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa050007e60 tx=0x7fa05000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.356+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 shutdown_connections 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.357+0000 7fa05f000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0340744e0 0x7fa0340769f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.357+0000 7fa05f000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa058144500 0x7fa058142f10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.357+0000 7fa05f000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0580cdfa0 0x7fa058150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.357+0000 7fa05f000640 1 --2- 172.21.15.19:0/3080228683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0580cdb90 0x7fa05814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.357+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 >> 172.21.15.19:0/3080228683 conn(0x7fa058073050 msgr2=0x7fa058160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.357+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 shutdown_connections 2024-04-25T11:31:03.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.357+0000 7fa05f000640 1 -- 172.21.15.19:0/3080228683 wait complete. 2024-04-25T11:31:03.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:31:03.375 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:31:03.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:31:03.575 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:31:03.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.663+0000 7f2f7f400640 1 Processor -- start 2024-04-25T11:31:03.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.663+0000 7f2f7f400640 1 -- start start 2024-04-25T11:31:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.665+0000 7f2f7f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f7815de40 0x7f2f7806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f7f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f78164720 0x7f2f780d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f7f400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f7806cf10 con 0x7f2f7806cb00 2024-04-25T11:31:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f7f400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2f78164d40 con 0x7f2f78164720 2024-04-25T11:31:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f7f400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2f78160cc0 con 0x7f2f7815de40 2024-04-25T11:31:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f77e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2f7806cb00 0x7f2f78164010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50842/0 (socket says 172.21.15.19:50842) 2024-04-25T11:31:03.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f77e00640 1 -- 172.21.15.19:0/2230079517 learned_addr learned my addr 172.21.15.19:0/2230079517 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:03.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f7d400640 1 --2- 172.21.15.19:0/2230079517 >> v2:172.21.15.19:3300/0 conn(0x7f2f78164720 0x7f2f780d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f7ca00640 1 --2- 172.21.15.19:0/2230079517 >> v2:172.21.15.169:3300/0 conn(0x7f2f7815de40 0x7f2f7806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.666+0000 7f2f77400640 1 -- 172.21.15.19:0/2230079517 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2974516582 0 0) 0x7f2f60001110 con 0x7f2f7806cb00 2024-04-25T11:31:03.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.667+0000 7f2f77400640 1 -- 172.21.15.19:0/2230079517 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f5c003520 con 0x7f2f7806cb00 2024-04-25T11:31:03.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.667+0000 7f2f7d400640 1 -- 172.21.15.19:0/2230079517 >> v2:172.21.15.169:3300/0 conn(0x7f2f7815de40 msgr2=0x7f2f7806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:03.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.667+0000 7f2f7d400640 1 --2- 172.21.15.19:0/2230079517 >> v2:172.21.15.169:3300/0 conn(0x7f2f7815de40 0x7f2f7806c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.667+0000 7f2f7d400640 1 -- 172.21.15.19:0/2230079517 >> v1:172.21.15.19:6789/0 conn(0x7f2f7806cb00 legacy=0x7f2f78164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.667+0000 7f2f7d400640 1 -- 172.21.15.19:0/2230079517 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f78160fa0 con 0x7f2f78164720 2024-04-25T11:31:03.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.668+0000 7f2f7d400640 1 --2- 172.21.15.19:0/2230079517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f780d0cb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2f7806d1b0 tx=0x7f2f7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13096e815471a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:03.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.668+0000 7f2f77400640 1 -- 172.21.15.19:0/2230079517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70037070 con 0x7f2f78164720 2024-04-25T11:31:03.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.668+0000 7f2f77400640 1 -- 172.21.15.19:0/2230079517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70032440 con 0x7f2f78164720 2024-04-25T11:31:03.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.668+0000 7f2f77400640 1 -- 172.21.15.19:0/2230079517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f700327f0 con 0x7f2f78164720 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 -- 172.21.15.19:0/2230079517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 msgr2=0x7f2f780d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 --2- 172.21.15.19:0/2230079517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f780d0cb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2f7806d1b0 tx=0x7f2f7002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 -- 172.21.15.19:0/2230079517 shutdown_connections 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 --2- 172.21.15.19:0/2230079517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f780d0cb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 --2- 172.21.15.19:0/2230079517 >> v2:172.21.15.169:3300/0 conn(0x7f2f7815de40 0x7f2f7806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 -- 172.21.15.19:0/2230079517 >> 172.21.15.19:0/2230079517 conn(0x7f2f780d5d70 msgr2=0x7f2f780d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 -- 172.21.15.19:0/2230079517 shutdown_connections 2024-04-25T11:31:03.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.669+0000 7f2f7f400640 1 -- 172.21.15.19:0/2230079517 wait complete. 2024-04-25T11:31:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.672+0000 7f2f7f400640 1 Processor -- start 2024-04-25T11:31:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.672+0000 7f2f7f400640 1 -- start start 2024-04-25T11:31:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.673+0000 7f2f7f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f7806cb00 0x7f2f780cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.673+0000 7f2f7f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f7815de40 0x7f2f78150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.673+0000 7f2f7f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f781509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7f400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7815e250 con 0x7f2f78164720 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f78160960 con 0x7f2f7806cb00 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7815e3d0 con 0x7f2f7815de40 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f781509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f781509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38324/0 (socket says 172.21.15.19:38324) 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 -- 172.21.15.19:0/3070251244 learned_addr learned my addr 172.21.15.19:0/3070251244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7ca00640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f7806cb00 0x7f2f780cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f77e00640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f7815de40 0x7f2f78150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 -- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f7815de40 msgr2=0x7f2f78150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f7815de40 0x7f2f78150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 -- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f7806cb00 msgr2=0x7f2f780cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f7806cb00 0x7f2f780cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f7d400640 1 -- 172.21.15.19:0/3070251244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f780ce7e0 con 0x7f2f78164720 2024-04-25T11:31:03.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.674+0000 7f2f77e00640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f7815de40 0x7f2f78150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:31:03.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.675+0000 7f2f7d400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f781509a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009890 tx=0x7f2f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:03.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.675+0000 7f2f74c00640 1 -- 172.21.15.19:0/3070251244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70041020 con 0x7f2f78164720 2024-04-25T11:31:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.675+0000 7f2f74c00640 1 -- 172.21.15.19:0/3070251244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7003d620 con 0x7f2f78164720 2024-04-25T11:31:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.675+0000 7f2f74c00640 1 -- 172.21.15.19:0/3070251244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70037040 con 0x7f2f78164720 2024-04-25T11:31:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.675+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f780b6bd0 con 0x7f2f78164720 2024-04-25T11:31:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.675+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f780b7190 con 0x7f2f78164720 2024-04-25T11:31:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.676+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f780b6a00 con 0x7f2f78164720 2024-04-25T11:31:03.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.678+0000 7f2f74c00640 1 -- 172.21.15.19:0/3070251244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2f70044020 con 0x7f2f78164720 2024-04-25T11:31:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.679+0000 7f2f74c00640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f540744e0 0x7f2f540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.679+0000 7f2f74c00640 1 -- 172.21.15.19:0/3070251244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f2f700b5650 con 0x7f2f78164720 2024-04-25T11:31:03.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.679+0000 7f2f7ca00640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f540744e0 0x7f2f540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.680+0000 7f2f7ca00640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f540744e0 0x7f2f540769f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2f680073f0 tx=0x7f2f6800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.683+0000 7f2f74c00640 1 -- 172.21.15.19:0/3070251244 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f7004b030 con 0x7f2f78164720 2024-04-25T11:31:03.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.885+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2f780ce570 con 0x7f2f78164720 2024-04-25T11:31:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.885+0000 7f2f74c00640 1 -- 172.21.15.19:0/3070251244 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f2f70080130 con 0x7f2f78164720 2024-04-25T11:31:03.875 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f540744e0 msgr2=0x7f2f540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f540744e0 0x7f2f540769f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2f680073f0 tx=0x7f2f6800b040 comp rx=0 tx=0).stop 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 msgr2=0x7f2f781509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f781509a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009890 tx=0x7f2f700046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 shutdown_connections 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f540744e0 0x7f2f540769f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f78164720 0x7f2f781509a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f7815de40 0x7f2f78150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 --2- 172.21.15.19:0/3070251244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f7806cb00 0x7f2f780cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 >> 172.21.15.19:0/3070251244 conn(0x7f2f780d5d70 msgr2=0x7f2f7815f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 shutdown_connections 2024-04-25T11:31:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:03.888+0000 7f2f7f400640 1 -- 172.21.15.19:0/3070251244 wait complete. 2024-04-25T11:31:03.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:31:04.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.245+0000 7f8264a00640 1 Processor -- start 2024-04-25T11:31:04.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.245+0000 7f8264a00640 1 -- start start 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.247+0000 7f8264a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f826009ce20 0x7f82600d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.247+0000 7f8264a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f82600d24a0 0x7f82600d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.247+0000 7f8264a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600b71a0 0x7f82600b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.247+0000 7f8264a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f82600d0ee0 con 0x7f826009ce20 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.247+0000 7f8264a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f826009d230 con 0x7f82600d24a0 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.247+0000 7f8264a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82600d1130 con 0x7f82600b71a0 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f826009ce20 0x7f82600d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f826009ce20 0x7f82600d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42274/0 (socket says 172.21.15.19:42274) 2024-04-25T11:31:04.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 -- 172.21.15.19:0/4238143892 learned_addr learned my addr 172.21.15.19:0/4238143892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:04.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825f200640 1 --2- 172.21.15.19:0/4238143892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600b71a0 0x7f82600b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825de00640 1 --2- 172.21.15.19:0/4238143892 >> v2:172.21.15.169:3300/0 conn(0x7f82600d24a0 0x7f82600d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 -- 172.21.15.19:0/4238143892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600b71a0 msgr2=0x7f82600b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 --2- 172.21.15.19:0/4238143892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600b71a0 0x7f82600b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 -- 172.21.15.19:0/4238143892 >> v2:172.21.15.169:3300/0 conn(0x7f82600d24a0 msgr2=0x7f82600d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 --2- 172.21.15.19:0/4238143892 >> v2:172.21.15.169:3300/0 conn(0x7f82600d24a0 0x7f82600d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.248+0000 7f825e800640 1 -- 172.21.15.19:0/4238143892 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600cfae0 con 0x7f826009ce20 2024-04-25T11:31:04.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.249+0000 7f825e800640 1 --2- 172.21.15.19:0/4238143892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f826009ce20 0x7f82600d1f60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f82600d3590 tx=0x7f825002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd522bb24620cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:04.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.250+0000 7f825d400640 1 -- 172.21.15.19:0/4238143892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250036070 con 0x7f826009ce20 2024-04-25T11:31:04.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.250+0000 7f825d400640 1 -- 172.21.15.19:0/4238143892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8250031600 con 0x7f826009ce20 2024-04-25T11:31:04.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.250+0000 7f825d400640 1 -- 172.21.15.19:0/4238143892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82500319d0 con 0x7f826009ce20 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 -- 172.21.15.19:0/4238143892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f826009ce20 msgr2=0x7f82600d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 --2- 172.21.15.19:0/4238143892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f826009ce20 0x7f82600d1f60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f82600d3590 tx=0x7f825002a610 comp rx=0 tx=0).stop 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 -- 172.21.15.19:0/4238143892 shutdown_connections 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 --2- 172.21.15.19:0/4238143892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600b71a0 0x7f82600b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 --2- 172.21.15.19:0/4238143892 >> v2:172.21.15.169:3300/0 conn(0x7f82600d24a0 0x7f82600d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 --2- 172.21.15.19:0/4238143892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f826009ce20 0x7f82600d1f60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 -- 172.21.15.19:0/4238143892 >> 172.21.15.19:0/4238143892 conn(0x7f82600d03c0 msgr2=0x7f82600d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 -- 172.21.15.19:0/4238143892 shutdown_connections 2024-04-25T11:31:04.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.251+0000 7f8264a00640 1 -- 172.21.15.19:0/4238143892 wait complete. 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.254+0000 7f8264a00640 1 Processor -- start 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.255+0000 7f8264a00640 1 -- start start 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f8264a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826009ce20 0x7f82600c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f8264a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82600b71a0 0x7f82600c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f8264a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600d24a0 0x7f826006e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f8264a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82600b8e70 con 0x7f82600b71a0 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f8264a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f82600d1a40 con 0x7f826009ce20 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f8264a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82600b8ff0 con 0x7f82600d24a0 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f825de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82600b71a0 0x7f82600c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f825de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82600b71a0 0x7f82600c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42280/0 (socket says 172.21.15.19:42280) 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.257+0000 7f825de00640 1 -- 172.21.15.19:0/3724455733 learned_addr learned my addr 172.21.15.19:0/3724455733 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825e800640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826009ce20 0x7f82600c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825f200640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600d24a0 0x7f826006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825de00640 1 -- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600d24a0 msgr2=0x7f826006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825de00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600d24a0 0x7f826006e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825de00640 1 -- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826009ce20 msgr2=0x7f82600c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825de00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826009ce20 0x7f82600c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825de00640 1 -- 172.21.15.19:0/3724455733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600cb4a0 con 0x7f82600b71a0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825e800640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826009ce20 0x7f82600c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f825de00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82600b71a0 0x7f82600c7990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f824800c7f0 tx=0x7f824800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f8256a00640 1 -- 172.21.15.19:0/3724455733 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248015070 con 0x7f82600b71a0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f8256a00640 1 -- 172.21.15.19:0/3724455733 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8248004220 con 0x7f82600b71a0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f8256a00640 1 -- 172.21.15.19:0/3724455733 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248007620 con 0x7f82600b71a0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.258+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82600cb780 con 0x7f82600b71a0 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.259+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82600cbd40 con 0x7f82600b71a0 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.260+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8260050740 con 0x7f82600b71a0 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.261+0000 7f8256a00640 1 -- 172.21.15.19:0/3724455733 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82480079d0 con 0x7f82600b71a0 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.262+0000 7f8256a00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f823c0744e0 0x7f823c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.262+0000 7f8256a00640 1 -- 172.21.15.19:0/3724455733 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f8248094ef0 con 0x7f82600b71a0 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.263+0000 7f825e800640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f823c0744e0 0x7f823c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.264+0000 7f825e800640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f823c0744e0 0x7f823c0769f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8250033000 tx=0x7f8250009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:04.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.267+0000 7f8256a00640 1 -- 172.21.15.19:0/3724455733 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f824805f9d0 con 0x7f82600b71a0 2024-04-25T11:31:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.471+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f82600d1f60 con 0x7f82600b71a0 2024-04-25T11:31:04.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.472+0000 7f8256a00640 1 -- 172.21.15.19:0/3724455733 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) ==== 84+0+65 (secure 0 0 0) 0x7f824805f2d0 con 0x7f82600b71a0 2024-04-25T11:31:04.461 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:04.462 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:31:04.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:31:04.462 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T11:31:04.462 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T11:31:04.462 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:31:04.462 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:31:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.474+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f823c0744e0 msgr2=0x7f823c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.474+0000 7f8264a00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f823c0744e0 0x7f823c0769f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8250033000 tx=0x7f8250009000 comp rx=0 tx=0).stop 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.474+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82600b71a0 msgr2=0x7f82600c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.474+0000 7f8264a00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82600b71a0 0x7f82600c7990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f824800c7f0 tx=0x7f824800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 shutdown_connections 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f823c0744e0 0x7f823c0769f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82600d24a0 0x7f826006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82600b71a0 0x7f82600c7990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 --2- 172.21.15.19:0/3724455733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f826009ce20 0x7f82600c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 >> 172.21.15.19:0/3724455733 conn(0x7f82600d03c0 msgr2=0x7f82601e8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 shutdown_connections 2024-04-25T11:31:04.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.475+0000 7f8264a00640 1 -- 172.21.15.19:0/3724455733 wait complete. 2024-04-25T11:31:04.491 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:31:04.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:31:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.792+0000 7f5fe3200640 1 Processor -- start 2024-04-25T11:31:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.792+0000 7f5fe3200640 1 -- start start 2024-04-25T11:31:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.794+0000 7f5fe3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe41606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fe4163c90 0x7f5fe406f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe40b0090 con 0x7f5fe406d5d0 2024-04-25T11:31:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe3200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe4160c80 con 0x7f5fe4163c90 2024-04-25T11:31:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe4160f60 con 0x7f5fe415de90 2024-04-25T11:31:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fe4163c90 0x7f5fe406f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fe4163c90 0x7f5fe406f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42284/0 (socket says 172.21.15.19:42284) 2024-04-25T11:31:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe1400640 1 -- 172.21.15.19:0/648170005 learned_addr learned my addr 172.21.15.19:0/648170005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.795+0000 7f5fe1e00640 1 --2- 172.21.15.19:0/648170005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe41606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.796+0000 7f5fe1400640 1 -- 172.21.15.19:0/648170005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 msgr2=0x7f5fe41606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:04.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.796+0000 7f5fe1400640 1 --2- 172.21.15.19:0/648170005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe41606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.796+0000 7f5fe1400640 1 -- 172.21.15.19:0/648170005 >> v1:172.21.15.169:6789/0 conn(0x7f5fe406d5d0 legacy=0x7f5fe406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.796+0000 7f5fe1400640 1 -- 172.21.15.19:0/648170005 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe4161240 con 0x7f5fe4163c90 2024-04-25T11:31:04.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.797+0000 7f5fe1400640 1 --2- 172.21.15.19:0/648170005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe406f0c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009870 tx=0x7f5fd002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5b495bc8ee5bde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:04.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.798+0000 7f5fe0a00640 1 -- 172.21.15.19:0/648170005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0037070 con 0x7f5fe4163c90 2024-04-25T11:31:04.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.798+0000 7f5fe0a00640 1 -- 172.21.15.19:0/648170005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd0032440 con 0x7f5fe4163c90 2024-04-25T11:31:04.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.798+0000 7f5fe0a00640 1 -- 172.21.15.19:0/648170005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0032810 con 0x7f5fe4163c90 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 -- 172.21.15.19:0/648170005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 msgr2=0x7f5fe406f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 --2- 172.21.15.19:0/648170005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe406f0c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009870 tx=0x7f5fd002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 -- 172.21.15.19:0/648170005 shutdown_connections 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 --2- 172.21.15.19:0/648170005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe406f0c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 --2- 172.21.15.19:0/648170005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe41606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 -- 172.21.15.19:0/648170005 >> 172.21.15.19:0/648170005 conn(0x7f5fe4072f40 msgr2=0x7f5fe4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 -- 172.21.15.19:0/648170005 shutdown_connections 2024-04-25T11:31:04.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.799+0000 7f5fe3200640 1 -- 172.21.15.19:0/648170005 wait complete. 2024-04-25T11:31:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.802+0000 7f5fe3200640 1 Processor -- start 2024-04-25T11:31:04.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.802+0000 7f5fe3200640 1 -- start start 2024-04-25T11:31:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.803+0000 7f5fe3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe406d5d0 0x7f5fe4156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.803+0000 7f5fe3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.803+0000 7f5fe3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe40b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.803+0000 7f5fe3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe406f780 con 0x7f5fe4163c90 2024-04-25T11:31:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.803+0000 7f5fe3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe406f600 con 0x7f5fe406d5d0 2024-04-25T11:31:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.803+0000 7f5fe3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe4070d30 con 0x7f5fe415de90 2024-04-25T11:31:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe40b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe40b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42300/0 (socket says 172.21.15.19:42300) 2024-04-25T11:31:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 -- 172.21.15.19:0/2000487041 learned_addr learned my addr 172.21.15.19:0/2000487041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe1e00640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe406d5d0 0x7f5fe4156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe1400640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 -- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 msgr2=0x7f5fe415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe415eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 -- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe406d5d0 msgr2=0x7f5fe4156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe406d5d0 0x7f5fe4156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.804+0000 7f5fe2800640 1 -- 172.21.15.19:0/2000487041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe4156cd0 con 0x7f5fe4163c90 2024-04-25T11:31:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.805+0000 7f5fe2800640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe40b5b40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c680 tx=0x7f5fdc00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.805+0000 7f5fda000640 1 -- 172.21.15.19:0/2000487041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc003b70 con 0x7f5fe4163c90 2024-04-25T11:31:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.805+0000 7f5fda000640 1 -- 172.21.15.19:0/2000487041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc003d10 con 0x7f5fe4163c90 2024-04-25T11:31:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.805+0000 7f5fda000640 1 -- 172.21.15.19:0/2000487041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc005310 con 0x7f5fe4163c90 2024-04-25T11:31:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.805+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe415f980 con 0x7f5fe4163c90 2024-04-25T11:31:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.805+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe415ff40 con 0x7f5fe4163c90 2024-04-25T11:31:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.807+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe4076c60 con 0x7f5fe4163c90 2024-04-25T11:31:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.808+0000 7f5fda000640 1 -- 172.21.15.19:0/2000487041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5fdc0054b0 con 0x7f5fe4163c90 2024-04-25T11:31:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.810+0000 7f5fda000640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5fc40744b0 0x7f5fc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.810+0000 7f5fda000640 1 -- 172.21.15.19:0/2000487041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f5fdc094d60 con 0x7f5fe4163c90 2024-04-25T11:31:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.810+0000 7f5fe1e00640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5fc40744b0 0x7f5fc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:04.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.812+0000 7f5fe1e00640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5fc40744b0 0x7f5fc40769c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006fd0 tx=0x7f5fd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:04.815+0000 7f5fda000640 1 -- 172.21.15.19:0/2000487041 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fdc05f7a0 con 0x7f5fe4163c90 2024-04-25T11:31:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.020+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5fe40d0a20 con 0x7f5fe4163c90 2024-04-25T11:31:05.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.217+0000 7f5fda000640 1 -- 172.21.15.19:0/2000487041 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) ==== 138+0+0 (secure 0 0 0) 0x7f5fdc05f0a0 con 0x7f5fe4163c90 2024-04-25T11:31:05.207 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T11:31:05.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5fc40744b0 msgr2=0x7f5fc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:05.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5fc40744b0 0x7f5fc40769c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4006fd0 tx=0x7f5fd4008040 comp rx=0 tx=0).stop 2024-04-25T11:31:05.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 msgr2=0x7f5fe40b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:05.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe40b5b40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c680 tx=0x7f5fdc00cb40 comp rx=0 tx=0).stop 2024-04-25T11:31:05.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 shutdown_connections 2024-04-25T11:31:05.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5fc40744b0 0x7f5fc40769c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe4163c90 0x7f5fe40b5b40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fe415de90 0x7f5fe415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 --2- 172.21.15.19:0/2000487041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fe406d5d0 0x7f5fe4156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.221+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 >> 172.21.15.19:0/2000487041 conn(0x7f5fe4072f40 msgr2=0x7f5fe415d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:05.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.222+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 shutdown_connections 2024-04-25T11:31:05.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.222+0000 7f5fe3200640 1 -- 172.21.15.19:0/2000487041 wait complete. 2024-04-25T11:31:05.247 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:31:05.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.538+0000 7fe262200640 1 Processor -- start 2024-04-25T11:31:05.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.538+0000 7fe262200640 1 -- start start 2024-04-25T11:31:05.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe262200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0d5240 0x7fe25c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:05.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe262200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe25c0bae50 0x7fe25c06e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:05.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe262200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe25c0d1420 con 0x7fe25c06dd60 2024-04-25T11:31:05.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe262200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe25c0b9130 con 0x7fe25c0bae50 2024-04-25T11:31:05.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe262200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe25c0b99b0 con 0x7fe25c0d5240 2024-04-25T11:31:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe261800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe25c0bae50 0x7fe25c06e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe261800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe25c0bae50 0x7fe25c06e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42302/0 (socket says 172.21.15.19:42302) 2024-04-25T11:31:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.541+0000 7fe261800640 1 -- 172.21.15.19:0/2525501574 learned_addr learned my addr 172.21.15.19:0/2525501574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.542+0000 7fe260e00640 1 --2- 172.21.15.19:0/2525501574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0d5240 0x7fe25c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.542+0000 7fe261800640 1 -- 172.21.15.19:0/2525501574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0d5240 msgr2=0x7fe25c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.542+0000 7fe261800640 1 --2- 172.21.15.19:0/2525501574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0d5240 0x7fe25c0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.542+0000 7fe261800640 1 -- 172.21.15.19:0/2525501574 >> v1:172.21.15.169:6789/0 conn(0x7fe25c06dd60 legacy=0x7fe25c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:05.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.542+0000 7fe261800640 1 -- 172.21.15.19:0/2525501574 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c0b9360 con 0x7fe25c0bae50 2024-04-25T11:31:05.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.543+0000 7fe261800640 1 --2- 172.21.15.19:0/2525501574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c0bae50 0x7fe25c06e170 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0b53b0 tx=0x7fe25402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89c5a69df7a31ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:05.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.543+0000 7fe25be00640 1 -- 172.21.15.19:0/2525501574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254036070 con 0x7fe25c0bae50 2024-04-25T11:31:05.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.543+0000 7fe25be00640 1 -- 172.21.15.19:0/2525501574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe254031600 con 0x7fe25c0bae50 2024-04-25T11:31:05.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.543+0000 7fe25be00640 1 -- 172.21.15.19:0/2525501574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2540319d0 con 0x7fe25c0bae50 2024-04-25T11:31:05.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 -- 172.21.15.19:0/2525501574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c0bae50 msgr2=0x7fe25c06e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:05.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 --2- 172.21.15.19:0/2525501574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c0bae50 0x7fe25c06e170 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0b53b0 tx=0x7fe25402a6c0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 -- 172.21.15.19:0/2525501574 shutdown_connections 2024-04-25T11:31:05.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 --2- 172.21.15.19:0/2525501574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c0bae50 0x7fe25c06e170 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 --2- 172.21.15.19:0/2525501574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0d5240 0x7fe25c0d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 -- 172.21.15.19:0/2525501574 >> 172.21.15.19:0/2525501574 conn(0x7fe25c0d5d50 msgr2=0x7fe25c0d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:05.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 -- 172.21.15.19:0/2525501574 shutdown_connections 2024-04-25T11:31:05.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.544+0000 7fe262200640 1 -- 172.21.15.19:0/2525501574 wait complete. 2024-04-25T11:31:05.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.547+0000 7fe262200640 1 Processor -- start 2024-04-25T11:31:05.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.547+0000 7fe262200640 1 -- start start 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.548+0000 7fe262200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c06dd60 0x7fe25c15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.548+0000 7fe262200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0bae50 0x7fe25c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.548+0000 7fe262200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c0d5240 0x7fe25c1f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe262200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c0d0ff0 con 0x7fe25c06dd60 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe262200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c0d1700 con 0x7fe25c0d5240 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe262200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe25c0d1170 con 0x7fe25c0bae50 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c06dd60 0x7fe25c15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c06dd60 0x7fe25c15dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42318/0 (socket says 172.21.15.19:42318) 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 -- 172.21.15.19:0/2035511794 learned_addr learned my addr 172.21.15.19:0/2035511794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe261800640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c0d5240 0x7fe25c1f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe253e00640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0bae50 0x7fe25c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:05.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 -- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0bae50 msgr2=0x7fe25c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0bae50 0x7fe25c15e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 -- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c0d5240 msgr2=0x7fe25c1f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c0d5240 0x7fe25c1f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe260e00640 1 -- 172.21.15.19:0/2035511794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c149800 con 0x7fe25c06dd60 2024-04-25T11:31:05.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.549+0000 7fe261800640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c0d5240 0x7fe25c1f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:05.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.550+0000 7fe260e00640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c06dd60 0x7fe25c15dda0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00d620 tx=0x7fe24c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.550+0000 7fe259600640 1 -- 172.21.15.19:0/2035511794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00b620 con 0x7fe25c06dd60 2024-04-25T11:31:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.550+0000 7fe259600640 1 -- 172.21.15.19:0/2035511794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24c019070 con 0x7fe25c06dd60 2024-04-25T11:31:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.550+0000 7fe259600640 1 -- 172.21.15.19:0/2035511794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00ba00 con 0x7fe25c06dd60 2024-04-25T11:31:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.550+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c14bdd0 con 0x7fe25c06dd60 2024-04-25T11:31:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.550+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c14c390 con 0x7fe25c06dd60 2024-04-25T11:31:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.551+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25c076c60 con 0x7fe25c06dd60 2024-04-25T11:31:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.553+0000 7fe259600640 1 -- 172.21.15.19:0/2035511794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe24c00bba0 con 0x7fe25c06dd60 2024-04-25T11:31:05.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.554+0000 7fe259600640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2380744e0 0x7fe2380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:05.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.554+0000 7fe259600640 1 -- 172.21.15.19:0/2035511794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5227+0+0 (secure 0 0 0) 0x7fe24c095f40 con 0x7fe25c06dd60 2024-04-25T11:31:05.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.555+0000 7fe253e00640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2380744e0 0x7fe2380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.556+0000 7fe253e00640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2380744e0 0x7fe2380769f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0d5890 tx=0x7fe244008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:05.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.559+0000 7fe259600640 1 -- 172.21.15.19:0/2035511794 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe24c060890 con 0x7fe25c06dd60 2024-04-25T11:31:05.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:05.765+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe25c0bbaf0 con 0x7fe25c06dd60 2024-04-25T11:31:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.226+0000 7fe259600640 1 -- 172.21.15.19:0/2035511794 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) ==== 170+0+0 (secure 0 0 0) 0x7fe24c060190 con 0x7fe25c06dd60 2024-04-25T11:31:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T11:31:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.230+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2380744e0 msgr2=0x7fe2380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.230+0000 7fe262200640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2380744e0 0x7fe2380769f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0d5890 tx=0x7fe244008040 comp rx=0 tx=0).stop 2024-04-25T11:31:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.230+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c06dd60 msgr2=0x7fe25c15dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.230+0000 7fe262200640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c06dd60 0x7fe25c15dda0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00d620 tx=0x7fe24c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.231+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 shutdown_connections 2024-04-25T11:31:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.231+0000 7fe262200640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2380744e0 0x7fe2380769f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.231+0000 7fe262200640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe25c0d5240 0x7fe25c1f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.231+0000 7fe262200640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe25c0bae50 0x7fe25c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.231+0000 7fe262200640 1 --2- 172.21.15.19:0/2035511794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe25c06dd60 0x7fe25c15dda0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.231+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 >> 172.21.15.19:0/2035511794 conn(0x7fe25c0d5d50 msgr2=0x7fe25c06cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:06.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.232+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 shutdown_connections 2024-04-25T11:31:06.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.234+0000 7fe262200640 1 -- 172.21.15.19:0/2035511794 wait complete. 2024-04-25T11:31:06.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:31:06.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.528+0000 7efd87000640 1 Processor -- start 2024-04-25T11:31:06.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.528+0000 7efd87000640 1 -- start start 2024-04-25T11:31:06.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.531+0000 7efd87000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd8015cfa0 0x7efd801458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.531+0000 7efd87000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8015d5c0 con 0x7efd800ce250 2024-04-25T11:31:06.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.531+0000 7efd87000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8015d8a0 con 0x7efd8006d5d0 2024-04-25T11:31:06.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.531+0000 7efd87000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efd800b4ec0 con 0x7efd8015cfa0 2024-04-25T11:31:06.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.531+0000 7efd7fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd800ce250 0x7efd800b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58568/0 (socket says 172.21.15.19:58568) 2024-04-25T11:31:06.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.531+0000 7efd7fe00640 1 -- 172.21.15.19:0/1485411417 learned_addr learned my addr 172.21.15.19:0/1485411417 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:06.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184412128 0 0) 0x7efd8015d5c0 con 0x7efd800ce250 2024-04-25T11:31:06.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7f400640 1 --2- 172.21.15.19:0/1485411417 >> v2:172.21.15.169:3300/0 conn(0x7efd8015cfa0 0x7efd801458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd64003520 con 0x7efd800ce250 2024-04-25T11:31:06.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4094867135 0 0) 0x7efd64003520 con 0x7efd800ce250 2024-04-25T11:31:06.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 >> v2:172.21.15.169:3300/0 conn(0x7efd8015cfa0 msgr2=0x7efd801458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 --2- 172.21.15.19:0/1485411417 >> v2:172.21.15.169:3300/0 conn(0x7efd8015cfa0 0x7efd801458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 >> v1:172.21.15.169:6789/0 conn(0x7efd8006d5d0 legacy=0x7efd8006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8015db80 con 0x7efd800ce250 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efd68001670 con 0x7efd800ce250 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.532+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 >> v1:172.21.15.19:6789/0 conn(0x7efd800ce250 legacy=0x7efd800b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7ea00640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd64003c00 0x7efd64008770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7ea00640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd64008cb0 0x7efd6400b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7ea00640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6400b660 0x7efd6400db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd800b4ec0 con 0x7efd6400b660 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd6400e450 con 0x7efd64008cb0 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd6400e650 con 0x7efd64003c00 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd85000640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6400b660 0x7efd6400db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7fe00640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd64003c00 0x7efd64008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7f400640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd64008cb0 0x7efd6400b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd85000640 1 -- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd64003c00 msgr2=0x7efd64008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:06.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd85000640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd64003c00 0x7efd64008770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd85000640 1 -- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd64008cb0 msgr2=0x7efd6400b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:06.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd85000640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd64008cb0 0x7efd6400b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd85000640 1 -- 172.21.15.19:0/1485411417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd64003520 con 0x7efd6400b660 2024-04-25T11:31:06.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.533+0000 7efd7f400640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd64008cb0 0x7efd6400b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.534+0000 7efd85000640 1 -- 172.21.15.19:0/1485411417 reap_dead start 2024-04-25T11:31:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.535+0000 7efd85000640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6400b660 0x7efd6400db70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efd780044b0 tx=0x7efd64003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357c7db92a52fe49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.535+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78004080 con 0x7efd6400b660 2024-04-25T11:31:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.535+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7802ada0 con 0x7efd6400b660 2024-04-25T11:31:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.535+0000 7efd7ea00640 1 -- 172.21.15.19:0/1485411417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78006d90 con 0x7efd6400b660 2024-04-25T11:31:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.536+0000 7efd87000640 1 -- 172.21.15.19:0/1485411417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6400b660 msgr2=0x7efd6400db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.536+0000 7efd87000640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6400b660 0x7efd6400db70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efd780044b0 tx=0x7efd64003810 comp rx=0 tx=0).stop 2024-04-25T11:31:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.536+0000 7efd87000640 1 -- 172.21.15.19:0/1485411417 shutdown_connections 2024-04-25T11:31:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.536+0000 7efd87000640 1 --2- 172.21.15.19:0/1485411417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6400b660 0x7efd6400db70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.536+0000 7efd87000640 1 -- 172.21.15.19:0/1485411417 >> 172.21.15.19:0/1485411417 conn(0x7efd80073090 msgr2=0x7efd800734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.536+0000 7efd87000640 1 -- 172.21.15.19:0/1485411417 shutdown_connections 2024-04-25T11:31:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.536+0000 7efd87000640 1 -- 172.21.15.19:0/1485411417 wait complete. 2024-04-25T11:31:06.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.538+0000 7efd87000640 1 Processor -- start 2024-04-25T11:31:06.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.538+0000 7efd87000640 1 -- start start 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd87000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6400b660 0x7efd8015b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd87000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8015c240 0x7efd800b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd87000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8015c650 0x7efd800b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd87000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd80071310 con 0x7efd8015c240 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd87000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd8015de60 con 0x7efd6400b660 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd87000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd801458c0 con 0x7efd8015c650 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd7f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8015c240 0x7efd800b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd7f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8015c240 0x7efd800b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42340/0 (socket says 172.21.15.19:42340) 2024-04-25T11:31:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.540+0000 7efd7f400640 1 -- 172.21.15.19:0/3770962715 learned_addr learned my addr 172.21.15.19:0/3770962715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd85000640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8015c650 0x7efd800b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd7fe00640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6400b660 0x7efd8015b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd7f400640 1 -- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8015c650 msgr2=0x7efd800b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd7f400640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8015c650 0x7efd800b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd7f400640 1 -- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6400b660 msgr2=0x7efd8015b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd7f400640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6400b660 0x7efd8015b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd7f400640 1 -- 172.21.15.19:0/3770962715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8015be30 con 0x7efd8015c240 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd7f400640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8015c240 0x7efd800b2220 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efd70010880 tx=0x7efd70010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.541+0000 7efd77e00640 1 -- 172.21.15.19:0/3770962715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7000f040 con 0x7efd8015c240 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.542+0000 7efd77e00640 1 -- 172.21.15.19:0/3770962715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7000eb60 con 0x7efd8015c240 2024-04-25T11:31:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.542+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd801f9a50 con 0x7efd8015c240 2024-04-25T11:31:06.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.542+0000 7efd77e00640 1 -- 172.21.15.19:0/3770962715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7001c400 con 0x7efd8015c240 2024-04-25T11:31:06.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.542+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd801fa010 con 0x7efd8015c240 2024-04-25T11:31:06.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.543+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd801f97a0 con 0x7efd8015c240 2024-04-25T11:31:06.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.544+0000 7efd77e00640 1 -- 172.21.15.19:0/3770962715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd7001c5a0 con 0x7efd8015c240 2024-04-25T11:31:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.545+0000 7efd77e00640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd5c0744b0 0x7efd5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.546+0000 7efd77e00640 1 -- 172.21.15.19:0/3770962715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5609+0+0 (secure 0 0 0) 0x7efd70095c20 con 0x7efd8015c240 2024-04-25T11:31:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.546+0000 7efd7fe00640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd5c0744b0 0x7efd5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.548+0000 7efd7fe00640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd5c0744b0 0x7efd5c0769c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd68001050 tx=0x7efd68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:06.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.550+0000 7efd77e00640 1 -- 172.21.15.19:0/3770962715 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd70018030 con 0x7efd8015c240 2024-04-25T11:31:06.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:06.787+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efd80050740 con 0x7efd8015c240 2024-04-25T11:31:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.280+0000 7efd77e00640 1 -- 172.21.15.19:0/3770962715 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v47) ==== 172+0+0 (secure 0 0 0) 0x7efd70060480 con 0x7efd8015c240 2024-04-25T11:31:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T11:31:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.291+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd5c0744b0 msgr2=0x7efd5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.291+0000 7efd87000640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd5c0744b0 0x7efd5c0769c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd68001050 tx=0x7efd68009040 comp rx=0 tx=0).stop 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.291+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8015c240 msgr2=0x7efd800b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.291+0000 7efd87000640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8015c240 0x7efd800b2220 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efd70010880 tx=0x7efd70010d40 comp rx=0 tx=0).stop 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.292+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 shutdown_connections 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.292+0000 7efd87000640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd5c0744b0 0x7efd5c0769c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.292+0000 7efd87000640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8015c650 0x7efd800b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.292+0000 7efd87000640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8015c240 0x7efd800b2220 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.292+0000 7efd87000640 1 --2- 172.21.15.19:0/3770962715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6400b660 0x7efd8015b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.292+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 >> 172.21.15.19:0/3770962715 conn(0x7efd80073090 msgr2=0x7efd800d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.293+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 shutdown_connections 2024-04-25T11:31:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.293+0000 7efd87000640 1 -- 172.21.15.19:0/3770962715 wait complete. 2024-04-25T11:31:07.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:31:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.583+0000 7fcc28800640 1 Processor -- start 2024-04-25T11:31:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.583+0000 7fcc28800640 1 -- start start 2024-04-25T11:31:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.585+0000 7fcc28800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc200cf5e0 0x7fcc200cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.585+0000 7fcc28800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc2006c5c0 0x7fcc2006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.585+0000 7fcc28800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc200cff30 con 0x7fcc201443c0 2024-04-25T11:31:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.585+0000 7fcc28800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcc200708d0 con 0x7fcc2006c5c0 2024-04-25T11:31:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.585+0000 7fcc28800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2015f160 con 0x7fcc200cf5e0 2024-04-25T11:31:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.586+0000 7fcc26800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcc2006c5c0 0x7fcc2006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.586+0000 7fcc25400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc200cf5e0 0x7fcc200cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.586+0000 7fcc25e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fcc201443c0 0x7fcc2006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:43892/0 (socket says 172.21.15.19:43892) 2024-04-25T11:31:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.586+0000 7fcc25e00640 1 -- 172.21.15.19:0/3421028297 learned_addr learned my addr 172.21.15.19:0/3421028297 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.587+0000 7fcc24a00640 1 -- 172.21.15.19:0/3421028297 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023839419 0 0) 0x7fcc14001110 con 0x7fcc201443c0 2024-04-25T11:31:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.587+0000 7fcc24a00640 1 -- 172.21.15.19:0/3421028297 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc08003520 con 0x7fcc201443c0 2024-04-25T11:31:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.587+0000 7fcc25400640 1 -- 172.21.15.19:0/3421028297 >> v2:172.21.15.169:3300/0 conn(0x7fcc2006c5c0 msgr2=0x7fcc2006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.587+0000 7fcc25400640 1 --2- 172.21.15.19:0/3421028297 >> v2:172.21.15.169:3300/0 conn(0x7fcc2006c5c0 0x7fcc2006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.587+0000 7fcc25400640 1 -- 172.21.15.19:0/3421028297 >> v1:172.21.15.169:6789/0 conn(0x7fcc201443c0 legacy=0x7fcc2006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.587+0000 7fcc25400640 1 -- 172.21.15.19:0/3421028297 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2015f5f0 con 0x7fcc200cf5e0 2024-04-25T11:31:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.587+0000 7fcc26800640 1 --2- 172.21.15.19:0/3421028297 >> v2:172.21.15.169:3300/0 conn(0x7fcc2006c5c0 0x7fcc2006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:31:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.589+0000 7fcc25400640 1 --2- 172.21.15.19:0/3421028297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc200cf5e0 0x7fcc200cf9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc100097a0 tx=0x7fcc1002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4474779366330f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.590+0000 7fcc24a00640 1 -- 172.21.15.19:0/3421028297 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10037070 con 0x7fcc200cf5e0 2024-04-25T11:31:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.590+0000 7fcc24a00640 1 -- 172.21.15.19:0/3421028297 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc10032440 con 0x7fcc200cf5e0 2024-04-25T11:31:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.590+0000 7fcc24a00640 1 -- 172.21.15.19:0/3421028297 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10032810 con 0x7fcc200cf5e0 2024-04-25T11:31:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.590+0000 7fcc28800640 1 -- 172.21.15.19:0/3421028297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc200cf5e0 msgr2=0x7fcc200cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.590+0000 7fcc28800640 1 --2- 172.21.15.19:0/3421028297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc200cf5e0 0x7fcc200cf9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc100097a0 tx=0x7fcc1002a180 comp rx=0 tx=0).stop 2024-04-25T11:31:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.591+0000 7fcc28800640 1 -- 172.21.15.19:0/3421028297 shutdown_connections 2024-04-25T11:31:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.591+0000 7fcc28800640 1 --2- 172.21.15.19:0/3421028297 >> v2:172.21.15.169:3300/0 conn(0x7fcc2006c5c0 0x7fcc2006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.591+0000 7fcc28800640 1 --2- 172.21.15.19:0/3421028297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc200cf5e0 0x7fcc200cf9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.591+0000 7fcc28800640 1 -- 172.21.15.19:0/3421028297 >> 172.21.15.19:0/3421028297 conn(0x7fcc200d1ae0 msgr2=0x7fcc200d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.591+0000 7fcc28800640 1 -- 172.21.15.19:0/3421028297 shutdown_connections 2024-04-25T11:31:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.591+0000 7fcc28800640 1 -- 172.21.15.19:0/3421028297 wait complete. 2024-04-25T11:31:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.594+0000 7fcc28800640 1 Processor -- start 2024-04-25T11:31:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.594+0000 7fcc28800640 1 -- start start 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.596+0000 7fcc28800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc2006c5c0 0x7fcc2015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.596+0000 7fcc28800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc200cf5e0 0x7fcc200b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.596+0000 7fcc28800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc201443c0 0x7fcc2006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.596+0000 7fcc28800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2015edf0 con 0x7fcc2006c5c0 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.596+0000 7fcc28800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2015eb10 con 0x7fcc200cf5e0 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.596+0000 7fcc28800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcc201447d0 con 0x7fcc201443c0 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc2006c5c0 0x7fcc2015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc2006c5c0 0x7fcc2015b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42348/0 (socket says 172.21.15.19:42348) 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 -- 172.21.15.19:0/1129033003 learned_addr learned my addr 172.21.15.19:0/1129033003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25400640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc200cf5e0 0x7fcc200b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc26800640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc201443c0 0x7fcc2006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 -- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc201443c0 msgr2=0x7fcc2006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc201443c0 0x7fcc2006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 -- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc200cf5e0 msgr2=0x7fcc200b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc200cf5e0 0x7fcc200b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.597+0000 7fcc25e00640 1 -- 172.21.15.19:0/1129033003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc200b1030 con 0x7fcc2006c5c0 2024-04-25T11:31:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.598+0000 7fcc25e00640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc2006c5c0 0x7fcc2015b2d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c3c0 tx=0x7fcc1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.598+0000 7fcc1a000640 1 -- 172.21.15.19:0/1129033003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14017070 con 0x7fcc2006c5c0 2024-04-25T11:31:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.598+0000 7fcc1a000640 1 -- 172.21.15.19:0/1129033003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc14005ce0 con 0x7fcc2006c5c0 2024-04-25T11:31:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.598+0000 7fcc1a000640 1 -- 172.21.15.19:0/1129033003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc140072f0 con 0x7fcc2006c5c0 2024-04-25T11:31:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.598+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc201e8a70 con 0x7fcc2006c5c0 2024-04-25T11:31:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.598+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201e9030 con 0x7fcc2006c5c0 2024-04-25T11:31:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.599+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc20050740 con 0x7fcc2006c5c0 2024-04-25T11:31:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.601+0000 7fcc1a000640 1 -- 172.21.15.19:0/1129033003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcc1401a070 con 0x7fcc2006c5c0 2024-04-25T11:31:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.602+0000 7fcc1a000640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc040744b0 0x7fcc040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.602+0000 7fcc1a000640 1 -- 172.21.15.19:0/1129033003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fcc14094570 con 0x7fcc2006c5c0 2024-04-25T11:31:07.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.602+0000 7fcc25400640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc040744b0 0x7fcc040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.604+0000 7fcc25400640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc040744b0 0x7fcc040769c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc10034000 tx=0x7fcc1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.606+0000 7fcc1a000640 1 -- 172.21.15.19:0/1129033003 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc200cff30 con 0x7fcc2006c5c0 2024-04-25T11:31:07.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.813+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc2006f950 con 0x7fcc2006c5c0 2024-04-25T11:31:07.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.816+0000 7fcc1a000640 1 -- 172.21.15.19:0/1129033003 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16678 (secure 0 0 0) 0x7fcc1405ed90 con 0x7fcc2006c5c0 2024-04-25T11:31:07.807 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:07.807 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":62,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:31:06.789500+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:31:07.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc040744b0 msgr2=0x7fcc040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc040744b0 0x7fcc040769c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc10034000 tx=0x7fcc1002e000 comp rx=0 tx=0).stop 2024-04-25T11:31:07.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc2006c5c0 msgr2=0x7fcc2015b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc2006c5c0 0x7fcc2015b2d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c3c0 tx=0x7fcc1400c880 comp rx=0 tx=0).stop 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 shutdown_connections 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc040744b0 0x7fcc040769c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcc200cf5e0 0x7fcc200b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcc201443c0 0x7fcc2006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 --2- 172.21.15.19:0/1129033003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcc2006c5c0 0x7fcc2015b2d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.820+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 >> 172.21.15.19:0/1129033003 conn(0x7fcc200d1ae0 msgr2=0x7fcc2015f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.821+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 shutdown_connections 2024-04-25T11:31:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:07.821+0000 7fcc28800640 1 -- 172.21.15.19:0/1129033003 wait complete. 2024-04-25T11:31:07.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:31:08.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.135+0000 7fa77a800640 1 Processor -- start 2024-04-25T11:31:08.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.136+0000 7fa77a800640 1 -- start start 2024-04-25T11:31:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.138+0000 7fa77a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7740cf9e0 0x7fa774161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.138+0000 7fa77a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740709a0 0x7fa7741569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.138+0000 7fa77a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7741615d0 con 0x7fa7740b0800 2024-04-25T11:31:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.138+0000 7fa77a800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa7740cffc0 con 0x7fa7740cf9e0 2024-04-25T11:31:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.138+0000 7fa77a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa7740d02a0 con 0x7fa7740709a0 2024-04-25T11:31:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.139+0000 7fa779e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740709a0 0x7fa7741569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.139+0000 7fa779400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa7740cf9e0 0x7fa774161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.139+0000 7fa779e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740709a0 0x7fa7741569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:56992/0 (socket says 172.21.15.19:56992) 2024-04-25T11:31:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.139+0000 7fa779e00640 1 -- 172.21.15.19:0/2299830368 learned_addr learned my addr 172.21.15.19:0/2299830368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.139+0000 7fa778a00640 1 --1- 172.21.15.19:0/2299830368 >> v1:172.21.15.169:6789/0 conn(0x7fa7740b0800 0x7fa774070290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:43906/0 (socket says 172.21.15.19:43906) 2024-04-25T11:31:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.140+0000 7fa773e00640 1 -- 172.21.15.19:0/2299830368 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 517669294 0 0) 0x7fa760001110 con 0x7fa7740b0800 2024-04-25T11:31:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.140+0000 7fa773e00640 1 -- 172.21.15.19:0/2299830368 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa75c003520 con 0x7fa7740b0800 2024-04-25T11:31:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.140+0000 7fa779400640 1 -- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740709a0 msgr2=0x7fa7741569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.140+0000 7fa779400640 1 --2- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740709a0 0x7fa7741569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.140+0000 7fa779400640 1 -- 172.21.15.19:0/2299830368 >> v1:172.21.15.169:6789/0 conn(0x7fa7740b0800 legacy=0x7fa774070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.140+0000 7fa779400640 1 -- 172.21.15.19:0/2299830368 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7740d0580 con 0x7fa7740cf9e0 2024-04-25T11:31:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.140+0000 7fa779e00640 1 --2- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740709a0 0x7fa7741569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:31:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.142+0000 7fa779400640 1 --2- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 0x7fa774161090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097a0 tx=0x7fa76802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bc676531603262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.143+0000 7fa773e00640 1 -- 172.21.15.19:0/2299830368 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768037070 con 0x7fa7740cf9e0 2024-04-25T11:31:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.143+0000 7fa773e00640 1 -- 172.21.15.19:0/2299830368 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76802ae50 con 0x7fa7740cf9e0 2024-04-25T11:31:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.143+0000 7fa773e00640 1 -- 172.21.15.19:0/2299830368 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768032660 con 0x7fa7740cf9e0 2024-04-25T11:31:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 -- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 msgr2=0x7fa774161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 --2- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 0x7fa774161090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097a0 tx=0x7fa76802a100 comp rx=0 tx=0).stop 2024-04-25T11:31:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 -- 172.21.15.19:0/2299830368 shutdown_connections 2024-04-25T11:31:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 --2- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740709a0 0x7fa7741569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 --2- 172.21.15.19:0/2299830368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 0x7fa774161090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 -- 172.21.15.19:0/2299830368 >> 172.21.15.19:0/2299830368 conn(0x7fa7740d1720 msgr2=0x7fa7740d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 -- 172.21.15.19:0/2299830368 shutdown_connections 2024-04-25T11:31:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.144+0000 7fa77a800640 1 -- 172.21.15.19:0/2299830368 wait complete. 2024-04-25T11:31:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.146+0000 7fa77a800640 1 Processor -- start 2024-04-25T11:31:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.147+0000 7fa77a800640 1 -- start start 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.148+0000 7fa77a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740b0800 0x7fa774146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.148+0000 7fa77a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 0x7fa774150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.148+0000 7fa77a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa774151140 0x7fa7741592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.148+0000 7fa77a800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7740b0d90 con 0x7fa774151140 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.148+0000 7fa77a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa7740b0c10 con 0x7fa7740cf9e0 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.148+0000 7fa77a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa774156ef0 con 0x7fa7740b0800 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa774151140 0x7fa7741592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa774151140 0x7fa7741592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42364/0 (socket says 172.21.15.19:42364) 2024-04-25T11:31:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 -- 172.21.15.19:0/3544440263 learned_addr learned my addr 172.21.15.19:0/3544440263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa778a00640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 0x7fa774150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779400640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740b0800 0x7fa774146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 -- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740b0800 msgr2=0x7fa774146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740b0800 0x7fa774146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 -- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 msgr2=0x7fa774150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 0x7fa774150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779e00640 1 -- 172.21.15.19:0/3544440263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7741466b0 con 0x7fa774151140 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.149+0000 7fa779400640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740b0800 0x7fa774146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.150+0000 7fa779e00640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa774151140 0x7fa7741592e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00ef90 tx=0x7fa76c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.150+0000 7fa771600640 1 -- 172.21.15.19:0/3544440263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00cbe0 con 0x7fa774151140 2024-04-25T11:31:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.150+0000 7fa771600640 1 -- 172.21.15.19:0/3544440263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c00cd80 con 0x7fa774151140 2024-04-25T11:31:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.150+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7741518c0 con 0x7fa774151140 2024-04-25T11:31:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.150+0000 7fa771600640 1 -- 172.21.15.19:0/3544440263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c003bd0 con 0x7fa774151140 2024-04-25T11:31:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.150+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa774151e80 con 0x7fa774151140 2024-04-25T11:31:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.151+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa774151630 con 0x7fa774151140 2024-04-25T11:31:08.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.153+0000 7fa771600640 1 -- 172.21.15.19:0/3544440263 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa76c01b030 con 0x7fa774151140 2024-04-25T11:31:08.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.153+0000 7fa771600640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7540744b0 0x7fa7540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.154+0000 7fa771600640 1 -- 172.21.15.19:0/3544440263 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa76c095600 con 0x7fa774151140 2024-04-25T11:31:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.154+0000 7fa779400640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7540744b0 0x7fa7540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.155+0000 7fa779400640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7540744b0 0x7fa7540769c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa768034000 tx=0x7fa76802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.158+0000 7fa771600640 1 -- 172.21.15.19:0/3544440263 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa76c022020 con 0x7fa774151140 2024-04-25T11:31:08.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.367+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa774050740 con 0x7fa774151140 2024-04-25T11:31:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.370+0000 7fa771600640 1 -- 172.21.15.19:0/3544440263 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16665 (secure 0 0 0) 0x7fa76c010040 con 0x7fa774151140 2024-04-25T11:31:08.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:08.360 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:31:08.278038+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:31:08.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7540744b0 msgr2=0x7fa7540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7540744b0 0x7fa7540769c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa768034000 tx=0x7fa76802e000 comp rx=0 tx=0).stop 2024-04-25T11:31:08.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa774151140 msgr2=0x7fa7741592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa774151140 0x7fa7741592e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00ef90 tx=0x7fa76c00c490 comp rx=0 tx=0).stop 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 shutdown_connections 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7540744b0 0x7fa7540769c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7740cf9e0 0x7fa774150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa774151140 0x7fa7741592e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 --2- 172.21.15.19:0/3544440263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7740b0800 0x7fa774146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.372+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 >> 172.21.15.19:0/3544440263 conn(0x7fa7740d1720 msgr2=0x7fa7741617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.373+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 shutdown_connections 2024-04-25T11:31:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.373+0000 7fa77a800640 1 -- 172.21.15.19:0/3544440263 wait complete. 2024-04-25T11:31:08.391 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:31:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.682+0000 7f2f57e00640 1 Processor -- start 2024-04-25T11:31:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.682+0000 7f2f57e00640 1 -- start start 2024-04-25T11:31:08.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.684+0000 7f2f57e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f5814d960 0x7f2f5814a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.684+0000 7f2f57e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2f580ce2e0 0x7f2f580ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.684+0000 7f2f57e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f580cec30 con 0x7f2f580d0cf0 2024-04-25T11:31:08.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.684+0000 7f2f57e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2f580b4f10 con 0x7f2f5814d960 2024-04-25T11:31:08.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.684+0000 7f2f57e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2f580b57b0 con 0x7f2f580ce2e0 2024-04-25T11:31:08.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.685+0000 7f2f4fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2f580d0cf0 0x7f2f5806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58584/0 (socket says 172.21.15.19:58584) 2024-04-25T11:31:08.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.685+0000 7f2f56a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f5814d960 0x7f2f5814a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.685+0000 7f2f4fe00640 1 -- 172.21.15.19:0/1553565936 learned_addr learned my addr 172.21.15.19:0/1553565936 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:08.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.685+0000 7f2f57400640 1 --2- 172.21.15.19:0/1553565936 >> v2:172.21.15.169:3300/0 conn(0x7f2f580ce2e0 0x7f2f580ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.686+0000 7f2f56a00640 1 -- 172.21.15.19:0/1553565936 >> v2:172.21.15.169:3300/0 conn(0x7f2f580ce2e0 msgr2=0x7f2f580ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:08.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.686+0000 7f2f56a00640 1 --2- 172.21.15.19:0/1553565936 >> v2:172.21.15.169:3300/0 conn(0x7f2f580ce2e0 0x7f2f580ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.686+0000 7f2f56a00640 1 -- 172.21.15.19:0/1553565936 >> v1:172.21.15.19:6789/0 conn(0x7f2f580d0cf0 legacy=0x7f2f5806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.686+0000 7f2f56a00640 1 -- 172.21.15.19:0/1553565936 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f580d1460 con 0x7f2f5814d960 2024-04-25T11:31:08.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.686+0000 7f2f57400640 1 --2- 172.21.15.19:0/1553565936 >> v2:172.21.15.169:3300/0 conn(0x7f2f580ce2e0 0x7f2f580ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:31:08.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.687+0000 7f2f56a00640 1 --2- 172.21.15.19:0/1553565936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5814a9c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2f580b4e30 tx=0x7f2f4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9060ee5956289d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:08.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.687+0000 7f2f56000640 1 -- 172.21.15.19:0/1553565936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48037070 con 0x7f2f5814d960 2024-04-25T11:31:08.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.687+0000 7f2f56000640 1 -- 172.21.15.19:0/1553565936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f48032440 con 0x7f2f5814d960 2024-04-25T11:31:08.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.687+0000 7f2f56000640 1 -- 172.21.15.19:0/1553565936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48032810 con 0x7f2f5814d960 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 -- 172.21.15.19:0/1553565936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 msgr2=0x7f2f5814a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 --2- 172.21.15.19:0/1553565936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5814a9c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2f580b4e30 tx=0x7f2f4802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 -- 172.21.15.19:0/1553565936 shutdown_connections 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 --2- 172.21.15.19:0/1553565936 >> v2:172.21.15.169:3300/0 conn(0x7f2f580ce2e0 0x7f2f580ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 --2- 172.21.15.19:0/1553565936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5814a9c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 -- 172.21.15.19:0/1553565936 >> 172.21.15.19:0/1553565936 conn(0x7f2f58072f40 msgr2=0x7f2f58073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 -- 172.21.15.19:0/1553565936 shutdown_connections 2024-04-25T11:31:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.688+0000 7f2f57e00640 1 -- 172.21.15.19:0/1553565936 wait complete. 2024-04-25T11:31:08.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.691+0000 7f2f57e00640 1 Processor -- start 2024-04-25T11:31:08.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.691+0000 7f2f57e00640 1 -- start start 2024-04-25T11:31:08.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.692+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f580ce2e0 0x7f2f581423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.692+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f580d0cf0 0x7f2f58165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.692+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5806ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.692+0000 7f2f57e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580d1100 con 0x7f2f5814d960 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.692+0000 7f2f57e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580b5440 con 0x7f2f580d0cf0 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.692+0000 7f2f57e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2f580b5c40 con 0x7f2f580ce2e0 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5806ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5806ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42390/0 (socket says 172.21.15.19:42390) 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 -- 172.21.15.19:0/715694270 learned_addr learned my addr 172.21.15.19:0/715694270 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f4fe00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f580d0cf0 0x7f2f58165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f56a00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f580ce2e0 0x7f2f581423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 -- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f580ce2e0 msgr2=0x7f2f581423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f580ce2e0 0x7f2f581423f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 -- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f580d0cf0 msgr2=0x7f2f58165bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f580d0cf0 0x7f2f58165bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.693+0000 7f2f57400640 1 -- 172.21.15.19:0/715694270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f581fabd0 con 0x7f2f5814d960 2024-04-25T11:31:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.694+0000 7f2f57400640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5806ce30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2f50007e20 tx=0x7f2f5000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.694+0000 7f2f4ea00640 1 -- 172.21.15.19:0/715694270 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5001a070 con 0x7f2f5814d960 2024-04-25T11:31:08.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.694+0000 7f2f4ea00640 1 -- 172.21.15.19:0/715694270 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f50003b90 con 0x7f2f5814d960 2024-04-25T11:31:08.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.694+0000 7f2f4ea00640 1 -- 172.21.15.19:0/715694270 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5000de80 con 0x7f2f5814d960 2024-04-25T11:31:08.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.694+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f581fae90 con 0x7f2f5814d960 2024-04-25T11:31:08.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.694+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f581fb450 con 0x7f2f5814d960 2024-04-25T11:31:08.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.696+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f581665b0 con 0x7f2f5814d960 2024-04-25T11:31:08.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.697+0000 7f2f4ea00640 1 -- 172.21.15.19:0/715694270 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2f5001d070 con 0x7f2f5814d960 2024-04-25T11:31:08.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.699+0000 7f2f4ea00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f340744b0 0x7f2f340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:08.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.699+0000 7f2f4ea00640 1 -- 172.21.15.19:0/715694270 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f2f50095520 con 0x7f2f5814d960 2024-04-25T11:31:08.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.699+0000 7f2f56a00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f340744b0 0x7f2f340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.701+0000 7f2f56a00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f340744b0 0x7f2f340769c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2f48034000 tx=0x7f2f4802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.704+0000 7f2f4ea00640 1 -- 172.21.15.19:0/715694270 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f50010070 con 0x7f2f5814d960 2024-04-25T11:31:08.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:08.955+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2f58076c60 con 0x7f2f5814d960 2024-04-25T11:31:09.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.303+0000 7f2f4ea00640 1 -- 172.21.15.19:0/715694270 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) ==== 118+0+0 (secure 0 0 0) 0x7f2f5005fc10 con 0x7f2f5814d960 2024-04-25T11:31:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f340744b0 msgr2=0x7f2f340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f340744b0 0x7f2f340769c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2f48034000 tx=0x7f2f4802e000 comp rx=0 tx=0).stop 2024-04-25T11:31:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 msgr2=0x7f2f5806ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5806ce30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2f50007e20 tx=0x7f2f5000d4a0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 shutdown_connections 2024-04-25T11:31:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2f340744b0 0x7f2f340769c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f580d0cf0 0x7f2f58165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f5814d960 0x7f2f5806ce30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.307+0000 7f2f57e00640 1 --2- 172.21.15.19:0/715694270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f580ce2e0 0x7f2f581423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.308+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 >> 172.21.15.19:0/715694270 conn(0x7f2f58072f40 msgr2=0x7f2f5806c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.308+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 shutdown_connections 2024-04-25T11:31:09.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.308+0000 7f2f57e00640 1 -- 172.21.15.19:0/715694270 wait complete. 2024-04-25T11:31:09.331 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:31:09.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:31:09.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.636+0000 7f07a1a00640 1 Processor -- start 2024-04-25T11:31:09.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.636+0000 7f07a1a00640 1 -- start start 2024-04-25T11:31:09.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.638+0000 7f07a1a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f079c0d4250 0x7f079c0bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:09.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.638+0000 7f07a1a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f079c0d4900 con 0x7f079c0b4db0 2024-04-25T11:31:09.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.638+0000 7f07a1a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f079c0d4be0 con 0x7f079c0b7c90 2024-04-25T11:31:09.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.638+0000 7f07a1a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f079c0d4ec0 con 0x7f079c0d4250 2024-04-25T11:31:09.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.638+0000 7f079b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f079c0b4db0 0x7f079c0b7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58592/0 (socket says 172.21.15.19:58592) 2024-04-25T11:31:09.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.638+0000 7f079b400640 1 -- 172.21.15.19:0/4255025910 learned_addr learned my addr 172.21.15.19:0/4255025910 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:09.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.638+0000 7f079be00640 1 --2- 172.21.15.19:0/4255025910 >> v2:172.21.15.19:3300/0 conn(0x7f079c0d4250 0x7f079c0bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:09.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.639+0000 7f079aa00640 1 -- 172.21.15.19:0/4255025910 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708953695 0 0) 0x7f079c0d4900 con 0x7f079c0b4db0 2024-04-25T11:31:09.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.639+0000 7f079aa00640 1 -- 172.21.15.19:0/4255025910 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0780003520 con 0x7f079c0b4db0 2024-04-25T11:31:09.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.639+0000 7f079be00640 1 -- 172.21.15.19:0/4255025910 >> v1:172.21.15.169:6789/0 conn(0x7f079c0b7c90 legacy=0x7f079c163ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:09.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.639+0000 7f079be00640 1 -- 172.21.15.19:0/4255025910 >> v1:172.21.15.19:6789/0 conn(0x7f079c0b4db0 legacy=0x7f079c0b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:09.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.639+0000 7f079be00640 1 -- 172.21.15.19:0/4255025910 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c0d51a0 con 0x7f079c0d4250 2024-04-25T11:31:09.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.640+0000 7f079be00640 1 --2- 172.21.15.19:0/4255025910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0d4250 0x7f079c0bb2e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f079402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d21b3f3eb2529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:09.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.640+0000 7f079aa00640 1 -- 172.21.15.19:0/4255025910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794007740 con 0x7f079c0d4250 2024-04-25T11:31:09.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.640+0000 7f079aa00640 1 -- 172.21.15.19:0/4255025910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07940078e0 con 0x7f079c0d4250 2024-04-25T11:31:09.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.640+0000 7f079aa00640 1 -- 172.21.15.19:0/4255025910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794007cb0 con 0x7f079c0d4250 2024-04-25T11:31:09.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.641+0000 7f07a1a00640 1 -- 172.21.15.19:0/4255025910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0d4250 msgr2=0x7f079c0bb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:09.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.641+0000 7f07a1a00640 1 --2- 172.21.15.19:0/4255025910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0d4250 0x7f079c0bb2e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f079402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.641+0000 7f07a1a00640 1 -- 172.21.15.19:0/4255025910 shutdown_connections 2024-04-25T11:31:09.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.641+0000 7f07a1a00640 1 --2- 172.21.15.19:0/4255025910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0d4250 0x7f079c0bb2e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.641+0000 7f07a1a00640 1 -- 172.21.15.19:0/4255025910 >> 172.21.15.19:0/4255025910 conn(0x7f079c0d2760 msgr2=0x7f079c0d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:09.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.641+0000 7f07a1a00640 1 -- 172.21.15.19:0/4255025910 shutdown_connections 2024-04-25T11:31:09.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.641+0000 7f07a1a00640 1 -- 172.21.15.19:0/4255025910 wait complete. 2024-04-25T11:31:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.644+0000 7f07a1a00640 1 Processor -- start 2024-04-25T11:31:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.644+0000 7f07a1a00640 1 -- start start 2024-04-25T11:31:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.645+0000 7f07a1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0b4db0 0x7f079c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.645+0000 7f07a1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f079c0d4250 0x7f079c0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.645+0000 7f07a1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f079c0cedb0 0x7f079c0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.645+0000 7f07a1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f079c0b91f0 con 0x7f079c0b4db0 2024-04-25T11:31:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.645+0000 7f07a1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f079c0d1700 con 0x7f079c0cedb0 2024-04-25T11:31:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.645+0000 7f07a1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f079c0b9370 con 0x7f079c0d4250 2024-04-25T11:31:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0b4db0 0x7f079c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0b4db0 0x7f079c156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42422/0 (socket says 172.21.15.19:42422) 2024-04-25T11:31:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 -- 172.21.15.19:0/729467184 learned_addr learned my addr 172.21.15.19:0/729467184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f07a1000640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f079c0cedb0 0x7f079c0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079b400640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f079c0d4250 0x7f079c0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 -- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f079c0d4250 msgr2=0x7f079c0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f079c0d4250 0x7f079c0aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 -- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f079c0cedb0 msgr2=0x7f079c0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f079c0cedb0 0x7f079c0cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.646+0000 7f079be00640 1 -- 172.21.15.19:0/729467184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c0cf700 con 0x7f079c0b4db0 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.647+0000 7f079be00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0b4db0 0x7f079c156d80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f0794004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.647+0000 7f0793e00640 1 -- 172.21.15.19:0/729467184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794040070 con 0x7f079c0b4db0 2024-04-25T11:31:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.647+0000 7f0793e00640 1 -- 172.21.15.19:0/729467184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0794031c40 con 0x7f079c0b4db0 2024-04-25T11:31:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.647+0000 7f0793e00640 1 -- 172.21.15.19:0/729467184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794009450 con 0x7f079c0b4db0 2024-04-25T11:31:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.647+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079c0afde0 con 0x7f079c0b4db0 2024-04-25T11:31:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.647+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079c0ae690 con 0x7f079c0b4db0 2024-04-25T11:31:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.648+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f079c0affb0 con 0x7f079c0b4db0 2024-04-25T11:31:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.650+0000 7f0793e00640 1 -- 172.21.15.19:0/729467184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0794005d10 con 0x7f079c0b4db0 2024-04-25T11:31:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.651+0000 7f0793e00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f077c0744b0 0x7f077c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.651+0000 7f0793e00640 1 -- 172.21.15.19:0/729467184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f07940b6800 con 0x7f079c0b4db0 2024-04-25T11:31:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.652+0000 7f079b400640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f077c0744b0 0x7f077c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.653+0000 7f079b400640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f077c0744b0 0x7f077c0769c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f079c0d0ba0 tx=0x7f0788009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.657+0000 7f0793e00640 1 -- 172.21.15.19:0/729467184 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f079404c070 con 0x7f079c0b4db0 2024-04-25T11:31:09.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:09.897+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f079c076c60 con 0x7f079c0b4db0 2024-04-25T11:31:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.306+0000 7f0793e00640 1 -- 172.21.15.19:0/729467184 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f079c0cf700 con 0x7f079c0b4db0 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f077c0744b0 msgr2=0x7f077c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f077c0744b0 0x7f077c0769c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f079c0d0ba0 tx=0x7f0788009040 comp rx=0 tx=0).stop 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0b4db0 msgr2=0x7f079c156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0b4db0 0x7f079c156d80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f0794004a40 comp rx=0 tx=0).stop 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 shutdown_connections 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f077c0744b0 0x7f077c0769c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f079c0cedb0 0x7f079c0cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f079c0d4250 0x7f079c0aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 --2- 172.21.15.19:0/729467184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f079c0b4db0 0x7f079c156d80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 >> 172.21.15.19:0/729467184 conn(0x7f079c0d2760 msgr2=0x7f079c0d5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 shutdown_connections 2024-04-25T11:31:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.310+0000 7f07a1a00640 1 -- 172.21.15.19:0/729467184 wait complete. 2024-04-25T11:31:10.330 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:31:10.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:10.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.632+0000 7fe58be00640 1 Processor -- start 2024-04-25T11:31:10.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.632+0000 7fe58be00640 1 -- start start 2024-04-25T11:31:10.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.634+0000 7fe58be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe58c0b7c50 0x7fe58c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.634+0000 7fe58be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe58c0d44a0 0x7fe58c0d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.634+0000 7fe58be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c161650 con 0x7fe58c164860 2024-04-25T11:31:10.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.634+0000 7fe58be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c1643d0 con 0x7fe58c0b7c50 2024-04-25T11:31:10.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.634+0000 7fe58be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c0b5270 con 0x7fe58c0d44a0 2024-04-25T11:31:10.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.635+0000 7fe58aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe58c164860 0x7fe58c0b7540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58604/0 (socket says 172.21.15.19:58604) 2024-04-25T11:31:10.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.635+0000 7fe58aa00640 1 -- 172.21.15.19:0/11427350 learned_addr learned my addr 172.21.15.19:0/11427350 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.635+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525755354 0 0) 0x7fe58c161650 con 0x7fe58c164860 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.635+0000 7fe58a000640 1 --2- 172.21.15.19:0/11427350 >> v2:172.21.15.169:3300/0 conn(0x7fe58c0b7c50 0x7fe58c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.635+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe570003520 con 0x7fe58c164860 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.635+0000 7fe58b400640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe58c0d44a0 0x7fe58c0d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1263638611 0 0) 0x7fe570003520 con 0x7fe58c164860 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe58c0d44a0 msgr2=0x7fe58c0d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe58c0d44a0 0x7fe58c0d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 >> v2:172.21.15.169:3300/0 conn(0x7fe58c0b7c50 msgr2=0x7fe58c0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:10.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 --2- 172.21.15.19:0/11427350 >> v2:172.21.15.169:3300/0 conn(0x7fe58c0b7c50 0x7fe58c0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c161370 con 0x7fe58c164860 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe5840019f0 con 0x7fe58c164860 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 >> v1:172.21.15.19:6789/0 conn(0x7fe58c164860 legacy=0x7fe58c0b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5700039b0 0x7fe570003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe570004800 0x7fe570004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe58c0d44a0 0x7fe570005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1643d0 con 0x7fe570004800 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0b5270 con 0x7fe5700039b0 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe57000ba40 con 0x7fe58c0d44a0 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.636+0000 7fe58a000640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe570004800 0x7fe570004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.637+0000 7fe58aa00640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5700039b0 0x7fe570003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.637+0000 7fe58a000640 1 -- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe58c0d44a0 msgr2=0x7fe570005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.637+0000 7fe58a000640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe58c0d44a0 0x7fe570005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.637+0000 7fe58a000640 1 -- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5700039b0 msgr2=0x7fe570003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.637+0000 7fe58a000640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5700039b0 0x7fe570003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.637+0000 7fe58a000640 1 -- 172.21.15.19:0/11427350 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570003520 con 0x7fe570004800 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.638+0000 7fe58a000640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe570004800 0x7fe570004c10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe574009a90 tx=0x7fe570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13091605a6e58344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.638+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574036070 con 0x7fe570004800 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.638+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe574031400 con 0x7fe570004800 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.638+0000 7fe589600640 1 -- 172.21.15.19:0/11427350 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574009690 con 0x7fe570004800 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.639+0000 7fe58be00640 1 -- 172.21.15.19:0/11427350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe570004800 msgr2=0x7fe570004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.639+0000 7fe58be00640 1 --2- 172.21.15.19:0/11427350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe570004800 0x7fe570004c10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe574009a90 tx=0x7fe570005830 comp rx=0 tx=0).stop 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.639+0000 7fe58b400640 1 -- 172.21.15.19:0/11427350 reap_dead start 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.639+0000 7fe58be00640 1 -- 172.21.15.19:0/11427350 shutdown_connections 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.639+0000 7fe58be00640 1 -- 172.21.15.19:0/11427350 >> 172.21.15.19:0/11427350 conn(0x7fe58c0d2720 msgr2=0x7fe58c0d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.640+0000 7fe58be00640 1 -- 172.21.15.19:0/11427350 shutdown_connections 2024-04-25T11:31:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.640+0000 7fe58be00640 1 -- 172.21.15.19:0/11427350 wait complete. 2024-04-25T11:31:10.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.642+0000 7fe58be00640 1 Processor -- start 2024-04-25T11:31:10.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.643+0000 7fe58be00640 1 -- start start 2024-04-25T11:31:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.644+0000 7fe58be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe570004800 0x7fe58c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.644+0000 7fe58be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe58c0ac490 0x7fe58c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.644+0000 7fe58be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe58c0cbc10 0x7fe58c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.644+0000 7fe58be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0d48b0 con 0x7fe58c0ac490 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.644+0000 7fe58be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c164060 con 0x7fe58c0cbc10 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.644+0000 7fe58be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c0d4a30 con 0x7fe570004800 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe58c0ac490 0x7fe58c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe58c0ac490 0x7fe58c0ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42444/0 (socket says 172.21.15.19:42444) 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 -- 172.21.15.19:0/2077421511 learned_addr learned my addr 172.21.15.19:0/2077421511 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 -- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe570004800 msgr2=0x7fe58c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58aa00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe570004800 0x7fe58c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58b400640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe58c0cbc10 0x7fe58c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe570004800 0x7fe58c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 -- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe58c0cbc10 msgr2=0x7fe58c0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe58c0cbc10 0x7fe58c0cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58a000640 1 -- 172.21.15.19:0/2077421511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c0acde0 con 0x7fe58c0ac490 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.645+0000 7fe58aa00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe570004800 0x7fe58c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:10.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.646+0000 7fe58a000640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe58c0ac490 0x7fe58c0ac8a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe574034040 tx=0x7fe574004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.646+0000 7fe582a00640 1 -- 172.21.15.19:0/2077421511 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574041020 con 0x7fe58c0ac490 2024-04-25T11:31:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.646+0000 7fe582a00640 1 -- 172.21.15.19:0/2077421511 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57403c470 con 0x7fe58c0ac490 2024-04-25T11:31:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.646+0000 7fe582a00640 1 -- 172.21.15.19:0/2077421511 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574036040 con 0x7fe58c0ac490 2024-04-25T11:31:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.646+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c0cf3d0 con 0x7fe58c0ac490 2024-04-25T11:31:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.646+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c0cee10 con 0x7fe58c0ac490 2024-04-25T11:31:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.647+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c050740 con 0x7fe58c0ac490 2024-04-25T11:31:10.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.649+0000 7fe582a00640 1 -- 172.21.15.19:0/2077421511 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe574044020 con 0x7fe58c0ac490 2024-04-25T11:31:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.650+0000 7fe582a00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe5680744e0 0x7fe5680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:10.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.650+0000 7fe582a00640 1 -- 172.21.15.19:0/2077421511 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fe5740b5b20 con 0x7fe58c0ac490 2024-04-25T11:31:10.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.650+0000 7fe58aa00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe5680744e0 0x7fe5680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:10.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.652+0000 7fe58aa00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe5680744e0 0x7fe5680769f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0d3740 tx=0x7fe584001310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:10.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.655+0000 7fe582a00640 1 -- 172.21.15.19:0/2077421511 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe57404b030 con 0x7fe58c0ac490 2024-04-25T11:31:10.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.900+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe58c0d3f30 con 0x7fe58c0ac490 2024-04-25T11:31:10.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.902+0000 7fe582a00640 1 -- 172.21.15.19:0/2077421511 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5184 (secure 0 0 0) 0x7fe58c0acde0 con 0x7fe58c0ac490 2024-04-25T11:31:10.892 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:10.892 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:09.898969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/751039234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":751039234},{"type":"v1","addr":"172.21.15.19:6833","nonce":751039234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6231,"qdb_cluster":[6231]},"id":4}]} 2024-04-25T11:31:10.892 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 51 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe5680744e0 msgr2=0x7fe5680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe5680744e0 0x7fe5680769f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0d3740 tx=0x7fe584001310 comp rx=0 tx=0).stop 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe58c0ac490 msgr2=0x7fe58c0ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe58c0ac490 0x7fe58c0ac8a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe574034040 tx=0x7fe574004730 comp rx=0 tx=0).stop 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 shutdown_connections 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe5680744e0 0x7fe5680769f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe58c0cbc10 0x7fe58c0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe58c0ac490 0x7fe58c0ac8a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 --2- 172.21.15.19:0/2077421511 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe570004800 0x7fe58c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:10.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 >> 172.21.15.19:0/2077421511 conn(0x7fe58c0d2720 msgr2=0x7fe58c161860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:10.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 shutdown_connections 2024-04-25T11:31:10.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:10.905+0000 7fe58be00640 1 -- 172.21.15.19:0/2077421511 wait complete. 2024-04-25T11:31:10.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:31:11.147 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:31:11.158+0000 7f061f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:31:11.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.216+0000 7ffac5800640 1 Processor -- start 2024-04-25T11:31:11.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.216+0000 7ffac5800640 1 -- start start 2024-04-25T11:31:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.218+0000 7ffac5800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffac00cf2f0 0x7ffac00b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.218+0000 7ffac5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.218+0000 7ffac5800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac0161220 con 0x7ffac015a810 2024-04-25T11:31:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.218+0000 7ffac5800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ffac006f870 con 0x7ffac00cf2f0 2024-04-25T11:31:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.218+0000 7ffac5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffac00b74d0 con 0x7ffac01608d0 2024-04-25T11:31:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffab7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffac015a810 0x7ffac00b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58614/0 (socket says 172.21.15.19:58614) 2024-04-25T11:31:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffab7400640 1 -- 172.21.15.19:0/2062782774 learned_addr learned my addr 172.21.15.19:0/2062782774 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffabfe00640 1 --2- 172.21.15.19:0/2062782774 >> v2:172.21.15.19:3300/0 conn(0x7ffac00cf2f0 0x7ffac00b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffac4e00640 1 --2- 172.21.15.19:0/2062782774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffabf400640 1 -- 172.21.15.19:0/2062782774 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766452925 0 0) 0x7ffaa8001110 con 0x7ffac015a810 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffabf400640 1 -- 172.21.15.19:0/2062782774 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaa4003520 con 0x7ffac015a810 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffabfe00640 1 -- 172.21.15.19:0/2062782774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 msgr2=0x7ffac0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.219+0000 7ffabfe00640 1 --2- 172.21.15.19:0/2062782774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.220+0000 7ffabfe00640 1 -- 172.21.15.19:0/2062782774 >> v1:172.21.15.19:6789/0 conn(0x7ffac015a810 legacy=0x7ffac00b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.220+0000 7ffabfe00640 1 -- 172.21.15.19:0/2062782774 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac00b77b0 con 0x7ffac00cf2f0 2024-04-25T11:31:11.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.220+0000 7ffac4e00640 1 --2- 172.21.15.19:0/2062782774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.221+0000 7ffabfe00640 1 --2- 172.21.15.19:0/2062782774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac00cf2f0 0x7ffac00b6e10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009d90 tx=0x7ffab002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af68915536bb927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.221+0000 7ffabf400640 1 -- 172.21.15.19:0/2062782774 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0007740 con 0x7ffac00cf2f0 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.221+0000 7ffabf400640 1 -- 172.21.15.19:0/2062782774 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab00078e0 con 0x7ffac00cf2f0 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.221+0000 7ffabf400640 1 -- 172.21.15.19:0/2062782774 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0007cb0 con 0x7ffac00cf2f0 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 -- 172.21.15.19:0/2062782774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac00cf2f0 msgr2=0x7ffac00b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 --2- 172.21.15.19:0/2062782774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac00cf2f0 0x7ffac00b6e10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009d90 tx=0x7ffab002a660 comp rx=0 tx=0).stop 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 -- 172.21.15.19:0/2062782774 shutdown_connections 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 --2- 172.21.15.19:0/2062782774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 --2- 172.21.15.19:0/2062782774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac00cf2f0 0x7ffac00b6e10 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 -- 172.21.15.19:0/2062782774 >> 172.21.15.19:0/2062782774 conn(0x7ffac00d1720 msgr2=0x7ffac00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:11.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 -- 172.21.15.19:0/2062782774 shutdown_connections 2024-04-25T11:31:11.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.222+0000 7ffac5800640 1 -- 172.21.15.19:0/2062782774 wait complete. 2024-04-25T11:31:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.225+0000 7ffac5800640 1 Processor -- start 2024-04-25T11:31:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.225+0000 7ffac5800640 1 -- start start 2024-04-25T11:31:11.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.226+0000 7ffac5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cf2f0 0x7ffac0157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.226+0000 7ffac5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac015a810 0x7ffac015ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.226+0000 7ffac5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.226+0000 7ffac5800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffac015ac20 con 0x7ffac015a810 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.226+0000 7ffac5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffac006f500 con 0x7ffac00cf2f0 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.226+0000 7ffac5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffac006fd00 con 0x7ffac01608d0 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.226+0000 7ffab7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac015a810 0x7ffac015ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac015a810 0x7ffac015ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42462/0 (socket says 172.21.15.19:42462) 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 -- 172.21.15.19:0/595705866 learned_addr learned my addr 172.21.15.19:0/595705866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffabfe00640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cf2f0 0x7ffac0157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 -- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 msgr2=0x7ffac0152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 -- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cf2f0 msgr2=0x7ffac0157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cf2f0 0x7ffac0157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 -- 172.21.15.19:0/595705866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac0152ca0 con 0x7ffac015a810 2024-04-25T11:31:11.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.227+0000 7ffab7400640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac015a810 0x7ffac015ffd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800c700 tx=0x7ffaa800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.228+0000 7ffabcc00640 1 -- 172.21.15.19:0/595705866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8017070 con 0x7ffac015a810 2024-04-25T11:31:11.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.228+0000 7ffabcc00640 1 -- 172.21.15.19:0/595705866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa8005a70 con 0x7ffac015a810 2024-04-25T11:31:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.228+0000 7ffabcc00640 1 -- 172.21.15.19:0/595705866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8005e60 con 0x7ffac015a810 2024-04-25T11:31:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.228+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac01543b0 con 0x7ffac015a810 2024-04-25T11:31:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.228+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac0152f80 con 0x7ffac015a810 2024-04-25T11:31:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.229+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac014eb10 con 0x7ffac015a810 2024-04-25T11:31:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.230+0000 7ffabcc00640 1 -- 172.21.15.19:0/595705866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffaa801a070 con 0x7ffac015a810 2024-04-25T11:31:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.231+0000 7ffabcc00640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa9c0744e0 0x7ffa9c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.231+0000 7ffabcc00640 1 -- 172.21.15.19:0/595705866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7ffaa8094520 con 0x7ffac015a810 2024-04-25T11:31:11.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.232+0000 7ffabfe00640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa9c0744e0 0x7ffa9c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.233+0000 7ffabfe00640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa9c0744e0 0x7ffa9c0769f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffab0034000 tx=0x7ffab0033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:11.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.236+0000 7ffabcc00640 1 -- 172.21.15.19:0/595705866 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffac014eb10 con 0x7ffac015a810 2024-04-25T11:31:11.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.442+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffac0050740 con 0x7ffac015a810 2024-04-25T11:31:11.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.445+0000 7ffabcc00640 1 -- 172.21.15.19:0/595705866 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16665 (secure 0 0 0) 0x7ffac0161220 con 0x7ffac015a810 2024-04-25T11:31:11.435 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:11.436 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:31:08.278038+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:31:11.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa9c0744e0 msgr2=0x7ffa9c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa9c0744e0 0x7ffa9c0769f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffab0034000 tx=0x7ffab0033040 comp rx=0 tx=0).stop 2024-04-25T11:31:11.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac015a810 msgr2=0x7ffac015ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac015a810 0x7ffac015ffd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800c700 tx=0x7ffaa800cbc0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 shutdown_connections 2024-04-25T11:31:11.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa9c0744e0 0x7ffa9c0769f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac01608d0 0x7ffac0152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffac015a810 0x7ffac015ffd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 --2- 172.21.15.19:0/595705866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cf2f0 0x7ffac0157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 >> 172.21.15.19:0/595705866 conn(0x7ffac00d1720 msgr2=0x7ffac00b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 shutdown_connections 2024-04-25T11:31:11.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.448+0000 7ffac5800640 1 -- 172.21.15.19:0/595705866 wait complete. 2024-04-25T11:31:11.462 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:31:11.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:31:11.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.749+0000 7f7a01800640 1 Processor -- start 2024-04-25T11:31:11.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.749+0000 7f7a01800640 1 -- start start 2024-04-25T11:31:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.752+0000 7f7a01800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79fc071e90 0x7f79fc0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.752+0000 7f7a01800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 0x7f79fc0d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.752+0000 7f7a01800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc1607c0 con 0x7f79fc0b6270 2024-04-25T11:31:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.752+0000 7f7a01800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc0b6ca0 con 0x7f79fc071e90 2024-04-25T11:31:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.752+0000 7f7a01800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc1613b0 con 0x7f79fc0d06c0 2024-04-25T11:31:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.752+0000 7f79fbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f79fc0b6270 0x7f79fc071780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58628/0 (socket says 172.21.15.19:58628) 2024-04-25T11:31:11.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.752+0000 7f79fbe00640 1 -- 172.21.15.19:0/1968227519 learned_addr learned my addr 172.21.15.19:0/1968227519 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79fb400640 1 --2- 172.21.15.19:0/1968227519 >> v2:172.21.15.169:3300/0 conn(0x7f79fc071e90 0x7f79fc0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f7a00e00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 0x7f79fc0d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1840800176 0 0) 0x7f79fc1607c0 con 0x7f79fc0b6270 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79e0003520 con 0x7f79fc0b6270 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 987127511 0 0) 0x7f79e0003520 con 0x7f79fc0b6270 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 msgr2=0x7f79fc0d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 0x7f79fc0d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 >> v2:172.21.15.169:3300/0 conn(0x7f79fc071e90 msgr2=0x7f79fc0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 --2- 172.21.15.19:0/1968227519 >> v2:172.21.15.169:3300/0 conn(0x7f79fc071e90 0x7f79fc0b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc160a80 con 0x7f79fc0b6270 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f79ec001670 con 0x7f79fc0b6270 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 >> v1:172.21.15.19:6789/0 conn(0x7f79fc0b6270 legacy=0x7f79fc071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79e00039b0 0x7f79e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e0004800 0x7f79e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 0x7f79e0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0b6ca0 con 0x7f79e00039b0 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc1613b0 con 0x7f79e0004800 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.753+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79e000b9f0 con 0x7f79fc0d06c0 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f79fbe00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79e00039b0 0x7f79e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f79fb400640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e0004800 0x7f79e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f7a00e00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 0x7f79e0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f79fbe00640 1 -- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 msgr2=0x7f79e0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:11.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f79fbe00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0d06c0 0x7f79e0005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f79fbe00640 1 -- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e0004800 msgr2=0x7f79e0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f79fbe00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e0004800 0x7f79e0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.754+0000 7f79fbe00640 1 -- 172.21.15.19:0/1968227519 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e0003520 con 0x7f79e00039b0 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.755+0000 7f79fbe00640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79e00039b0 0x7f79e0003e20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c160 tx=0x7f79e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b84a9b557314cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec007c80 con 0x7f79e00039b0 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec005ad0 con 0x7f79e00039b0 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f79faa00640 1 -- 172.21.15.19:0/1968227519 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec007070 con 0x7f79e00039b0 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f7a01800640 1 -- 172.21.15.19:0/1968227519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79e00039b0 msgr2=0x7f79e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f7a01800640 1 --2- 172.21.15.19:0/1968227519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79e00039b0 0x7f79e0003e20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c160 tx=0x7f79e0005830 comp rx=0 tx=0).stop 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f7a00e00640 1 -- 172.21.15.19:0/1968227519 reap_dead start 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f7a01800640 1 -- 172.21.15.19:0/1968227519 shutdown_connections 2024-04-25T11:31:11.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.756+0000 7f7a01800640 1 -- 172.21.15.19:0/1968227519 >> 172.21.15.19:0/1968227519 conn(0x7f79fc070270 msgr2=0x7f79fc070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:11.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.757+0000 7f7a01800640 1 -- 172.21.15.19:0/1968227519 shutdown_connections 2024-04-25T11:31:11.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.757+0000 7f7a01800640 1 -- 172.21.15.19:0/1968227519 wait complete. 2024-04-25T11:31:11.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.759+0000 7f7a01800640 1 Processor -- start 2024-04-25T11:31:11.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.759+0000 7f7a01800640 1 -- start start 2024-04-25T11:31:11.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f7a01800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e00039b0 0x7f79fc14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f7a01800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79fc0ccaa0 0x7f79fc0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f7a01800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0cceb0 0x7f79fc0cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f7a01800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0b1c20 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f7a01800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc161040 con 0x7f79e00039b0 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f7a01800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0b1da0 con 0x7f79fc0cceb0 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f79fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79fc0ccaa0 0x7f79fc0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f79fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79fc0ccaa0 0x7f79fc0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42492/0 (socket says 172.21.15.19:42492) 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.761+0000 7f79fb400640 1 -- 172.21.15.19:0/3649937043 learned_addr learned my addr 172.21.15.19:0/3649937043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fbe00640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e00039b0 0x7f79fc14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f7a00e00640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0cceb0 0x7f79fc0cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fb400640 1 -- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0cceb0 msgr2=0x7f79fc0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fb400640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0cceb0 0x7f79fc0cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fb400640 1 -- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e00039b0 msgr2=0x7f79fc14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fb400640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e00039b0 0x7f79fc14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fb400640 1 -- 172.21.15.19:0/3649937043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc14c7a0 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fbe00640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e00039b0 0x7f79fc14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.762+0000 7f79fb400640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79fc0ccaa0 0x7f79fc0cba50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011540 tx=0x7f79e8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:11.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.763+0000 7f79f3e00640 1 -- 172.21.15.19:0/3649937043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8015930 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.763+0000 7f79f3e00640 1 -- 172.21.15.19:0/3649937043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e8015ad0 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.763+0000 7f79f3e00640 1 -- 172.21.15.19:0/3649937043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e801d400 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.763+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc15ac40 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.763+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc15b200 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.764+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc050740 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.766+0000 7f79f3e00640 1 -- 172.21.15.19:0/3649937043 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79e801d5a0 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.767+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79dc0744e0 0x7f79dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.768+0000 7f79f3e00640 1 -- 172.21.15.19:0/3649937043 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f79e8095be0 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.768+0000 7f79fbe00640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79dc0744e0 0x7f79dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:11.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.770+0000 7f79fbe00640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79dc0744e0 0x7f79dc0769f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f79ec032000 tx=0x7f79ec007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:11.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.773+0000 7f79f3e00640 1 -- 172.21.15.19:0/3649937043 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79e8019030 con 0x7f79fc0ccaa0 2024-04-25T11:31:11.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.972+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79fc0cefd0 con 0x7f79dc0744e0 2024-04-25T11:31:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.992+0000 7f79f3e00640 1 -- 172.21.15.19:0/3649937043 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250981 (secure 0 0 0) 0x7f79fc0cefd0 con 0x7f79dc0744e0 2024-04-25T11:31:11.985 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:31:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.996+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79dc0744e0 msgr2=0x7f79dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.996+0000 7f7a01800640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79dc0744e0 0x7f79dc0769f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f79ec032000 tx=0x7f79ec007000 comp rx=0 tx=0).stop 2024-04-25T11:31:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79fc0ccaa0 msgr2=0x7f79fc0cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79fc0ccaa0 0x7f79fc0cba50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011540 tx=0x7f79e8011a00 comp rx=0 tx=0).stop 2024-04-25T11:31:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 shutdown_connections 2024-04-25T11:31:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79dc0744e0 0x7f79dc0769f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79fc0cceb0 0x7f79fc0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79fc0ccaa0 0x7f79fc0cba50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 --2- 172.21.15.19:0/3649937043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79e00039b0 0x7f79fc14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 >> 172.21.15.19:0/3649937043 conn(0x7f79fc070270 msgr2=0x7f79fc0b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 shutdown_connections 2024-04-25T11:31:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:11.997+0000 7f7a01800640 1 -- 172.21.15.19:0/3649937043 wait complete. 2024-04-25T11:31:12.013 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-04-25T11:31:10.529169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":139,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248000,"kb_used_data":6868,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726720,"statfs":{"total":773094113280,"available":772840161280,"internally_reserved":0,"allocated":7032832,"data_stored":3608818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.308648"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.309850+0000","last_change":"2024-04-25T11:31:07.304701+0000","last_active":"2024-04-25T11:31:07.309850+0000","last_peered":"2024-04-25T11:31:07.309850+0000","last_clean":"2024-04-25T11:31:07.309850+0000","last_became_active":"2024-04-25T11:31:07.304192+0000","last_became_peered":"2024-04-25T11:31:07.304192+0000","last_unstale":"2024-04-25T11:31:07.309850+0000","last_undegraded":"2024-04-25T11:31:07.309850+0000","last_fullsized":"2024-04-25T11:31:07.309850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.352760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.319005+0000","last_change":"2024-04-25T11:31:07.316843+0000","last_active":"2024-04-25T11:31:07.319005+0000","last_peered":"2024-04-25T11:31:07.319005+0000","last_clean":"2024-04-25T11:31:07.319005+0000","last_became_active":"2024-04-25T11:31:07.316438+0000","last_became_peered":"2024-04-25T11:31:07.316438+0000","last_unstale":"2024-04-25T11:31:07.319005+0000","last_undegraded":"2024-04-25T11:31:07.319005+0000","last_fullsized":"2024-04-25T11:31:07.319005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:24.715863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.307131+0000","last_change":"2024-04-25T11:31:07.298387+0000","last_active":"2024-04-25T11:31:07.307131+0000","last_peered":"2024-04-25T11:31:07.307131+0000","last_clean":"2024-04-25T11:31:07.307131+0000","last_became_active":"2024-04-25T11:31:07.297861+0000","last_became_peered":"2024-04-25T11:31:07.297861+0000","last_unstale":"2024-04-25T11:31:07.307131+0000","last_undegraded":"2024-04-25T11:31:07.307131+0000","last_fullsized":"2024-04-25T11:31:07.307131+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:12.203670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.310809+0000","last_change":"2024-04-25T11:31:07.305886+0000","last_active":"2024-04-25T11:31:07.310809+0000","last_peered":"2024-04-25T11:31:07.310809+0000","last_clean":"2024-04-25T11:31:07.310809+0000","last_became_active":"2024-04-25T11:31:07.304514+0000","last_became_peered":"2024-04-25T11:31:07.304514+0000","last_unstale":"2024-04-25T11:31:07.310809+0000","last_undegraded":"2024-04-25T11:31:07.310809+0000","last_fullsized":"2024-04-25T11:31:07.310809+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.528615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.298659+0000","last_change":"2024-04-25T11:31:07.290012+0000","last_active":"2024-04-25T11:31:07.298659+0000","last_peered":"2024-04-25T11:31:07.298659+0000","last_clean":"2024-04-25T11:31:07.298659+0000","last_became_active":"2024-04-25T11:31:07.289658+0000","last_became_peered":"2024-04-25T11:31:07.289658+0000","last_unstale":"2024-04-25T11:31:07.298659+0000","last_undegraded":"2024-04-25T11:31:07.298659+0000","last_fullsized":"2024-04-25T11:31:07.298659+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:24.715863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.299035+0000","last_change":"2024-04-25T11:31:07.288917+0000","last_active":"2024-04-25T11:31:07.299035+0000","last_peered":"2024-04-25T11:31:07.299035+0000","last_clean":"2024-04-25T11:31:07.299035+0000","last_became_active":"2024-04-25T11:31:07.288510+0000","last_became_peered":"2024-04-25T11:31:07.288510+0000","last_unstale":"2024-04-25T11:31:07.299035+0000","last_undegraded":"2024-04-25T11:31:07.299035+0000","last_fullsized":"2024-04-25T11:31:07.299035+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.501090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.297193+0000","last_change":"2024-04-25T11:31:07.291281+0000","last_active":"2024-04-25T11:31:07.297193+0000","last_peered":"2024-04-25T11:31:07.297193+0000","last_clean":"2024-04-25T11:31:07.297193+0000","last_became_active":"2024-04-25T11:31:07.290866+0000","last_became_peered":"2024-04-25T11:31:07.290866+0000","last_unstale":"2024-04-25T11:31:07.297193+0000","last_undegraded":"2024-04-25T11:31:07.297193+0000","last_fullsized":"2024-04-25T11:31:07.297193+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:05.327797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.240482+0000","last_change":"2024-04-25T11:31:06.240482+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.240482+0000","last_undegraded":"2024-04-25T11:31:06.240482+0000","last_fullsized":"2024-04-25T11:31:06.240482+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.312160+0000","last_change":"2024-04-25T11:31:07.306210+0000","last_active":"2024-04-25T11:31:07.312160+0000","last_peered":"2024-04-25T11:31:07.312160+0000","last_clean":"2024-04-25T11:31:07.312160+0000","last_became_active":"2024-04-25T11:31:07.305864+0000","last_became_peered":"2024-04-25T11:31:07.305864+0000","last_unstale":"2024-04-25T11:31:07.312160+0000","last_undegraded":"2024-04-25T11:31:07.312160+0000","last_fullsized":"2024-04-25T11:31:07.312160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:45.076928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318064+0000","last_change":"2024-04-25T11:31:07.313076+0000","last_active":"2024-04-25T11:31:07.318064+0000","last_peered":"2024-04-25T11:31:07.318064+0000","last_clean":"2024-04-25T11:31:07.318064+0000","last_became_active":"2024-04-25T11:31:07.310245+0000","last_became_peered":"2024-04-25T11:31:07.310245+0000","last_unstale":"2024-04-25T11:31:07.318064+0000","last_undegraded":"2024-04-25T11:31:07.318064+0000","last_fullsized":"2024-04-25T11:31:07.318064+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:50.811200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.297745+0000","last_change":"2024-04-25T11:31:07.292601+0000","last_active":"2024-04-25T11:31:07.297745+0000","last_peered":"2024-04-25T11:31:07.297745+0000","last_clean":"2024-04-25T11:31:07.297745+0000","last_became_active":"2024-04-25T11:31:07.292250+0000","last_became_peered":"2024-04-25T11:31:07.292250+0000","last_unstale":"2024-04-25T11:31:07.297745+0000","last_undegraded":"2024-04-25T11:31:07.297745+0000","last_fullsized":"2024-04-25T11:31:07.297745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:12.203670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.307769+0000","last_change":"2024-04-25T11:31:07.300826+0000","last_active":"2024-04-25T11:31:07.307769+0000","last_peered":"2024-04-25T11:31:07.307769+0000","last_clean":"2024-04-25T11:31:07.307769+0000","last_became_active":"2024-04-25T11:31:07.300489+0000","last_became_peered":"2024-04-25T11:31:07.300489+0000","last_unstale":"2024-04-25T11:31:07.307769+0000","last_undegraded":"2024-04-25T11:31:07.307769+0000","last_fullsized":"2024-04-25T11:31:07.307769+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:25.648569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.315114+0000","last_change":"2024-04-25T11:31:07.305473+0000","last_active":"2024-04-25T11:31:07.315114+0000","last_peered":"2024-04-25T11:31:07.315114+0000","last_clean":"2024-04-25T11:31:07.315114+0000","last_became_active":"2024-04-25T11:31:07.304950+0000","last_became_peered":"2024-04-25T11:31:07.304950+0000","last_unstale":"2024-04-25T11:31:07.315114+0000","last_undegraded":"2024-04-25T11:31:07.315114+0000","last_fullsized":"2024-04-25T11:31:07.315114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:50.811200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.307427+0000","last_change":"2024-04-25T11:31:07.290883+0000","last_active":"2024-04-25T11:31:07.307427+0000","last_peered":"2024-04-25T11:31:07.307427+0000","last_clean":"2024-04-25T11:31:07.307427+0000","last_became_active":"2024-04-25T11:31:07.288696+0000","last_became_peered":"2024-04-25T11:31:07.288696+0000","last_unstale":"2024-04-25T11:31:07.307427+0000","last_undegraded":"2024-04-25T11:31:07.307427+0000","last_fullsized":"2024-04-25T11:31:07.307427+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:25.078435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.315283+0000","last_change":"2024-04-25T11:31:07.308421+0000","last_active":"2024-04-25T11:31:07.315283+0000","last_peered":"2024-04-25T11:31:07.315283+0000","last_clean":"2024-04-25T11:31:07.315283+0000","last_became_active":"2024-04-25T11:31:07.305522+0000","last_became_peered":"2024-04-25T11:31:07.305522+0000","last_unstale":"2024-04-25T11:31:07.315283+0000","last_undegraded":"2024-04-25T11:31:07.315283+0000","last_fullsized":"2024-04-25T11:31:07.315283+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.988515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.306209+0000","last_change":"2024-04-25T11:31:07.299464+0000","last_active":"2024-04-25T11:31:07.306209+0000","last_peered":"2024-04-25T11:31:07.306209+0000","last_clean":"2024-04-25T11:31:07.306209+0000","last_became_active":"2024-04-25T11:31:07.298017+0000","last_became_peered":"2024-04-25T11:31:07.298017+0000","last_unstale":"2024-04-25T11:31:07.306209+0000","last_undegraded":"2024-04-25T11:31:07.306209+0000","last_fullsized":"2024-04-25T11:31:07.306209+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:52.205743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.314853+0000","last_change":"2024-04-25T11:31:07.305536+0000","last_active":"2024-04-25T11:31:07.314853+0000","last_peered":"2024-04-25T11:31:07.314853+0000","last_clean":"2024-04-25T11:31:07.314853+0000","last_became_active":"2024-04-25T11:31:07.304679+0000","last_became_peered":"2024-04-25T11:31:07.304679+0000","last_unstale":"2024-04-25T11:31:07.314853+0000","last_undegraded":"2024-04-25T11:31:07.314853+0000","last_fullsized":"2024-04-25T11:31:07.314853+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:21.068983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.319656+0000","last_change":"2024-04-25T11:31:07.311466+0000","last_active":"2024-04-25T11:31:07.319656+0000","last_peered":"2024-04-25T11:31:07.319656+0000","last_clean":"2024-04-25T11:31:07.319656+0000","last_became_active":"2024-04-25T11:31:07.304996+0000","last_became_peered":"2024-04-25T11:31:07.304996+0000","last_unstale":"2024-04-25T11:31:07.319656+0000","last_undegraded":"2024-04-25T11:31:07.319656+0000","last_fullsized":"2024-04-25T11:31:07.319656+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.933478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318423+0000","last_change":"2024-04-25T11:31:07.311429+0000","last_active":"2024-04-25T11:31:07.318423+0000","last_peered":"2024-04-25T11:31:07.318423+0000","last_clean":"2024-04-25T11:31:07.318423+0000","last_became_active":"2024-04-25T11:31:07.310949+0000","last_became_peered":"2024-04-25T11:31:07.310949+0000","last_unstale":"2024-04-25T11:31:07.318423+0000","last_undegraded":"2024-04-25T11:31:07.318423+0000","last_fullsized":"2024-04-25T11:31:07.318423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:25.648569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.313412+0000","last_change":"2024-04-25T11:31:07.309181+0000","last_active":"2024-04-25T11:31:07.313412+0000","last_peered":"2024-04-25T11:31:07.313412+0000","last_clean":"2024-04-25T11:31:07.313412+0000","last_became_active":"2024-04-25T11:31:07.306167+0000","last_became_peered":"2024-04-25T11:31:07.306167+0000","last_unstale":"2024-04-25T11:31:07.313412+0000","last_undegraded":"2024-04-25T11:31:07.313412+0000","last_fullsized":"2024-04-25T11:31:07.313412+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.933478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.317037+0000","last_change":"2024-04-25T11:31:07.311217+0000","last_active":"2024-04-25T11:31:07.317037+0000","last_peered":"2024-04-25T11:31:07.317037+0000","last_clean":"2024-04-25T11:31:07.317037+0000","last_became_active":"2024-04-25T11:31:07.310916+0000","last_became_peered":"2024-04-25T11:31:07.310916+0000","last_unstale":"2024-04-25T11:31:07.317037+0000","last_undegraded":"2024-04-25T11:31:07.317037+0000","last_fullsized":"2024-04-25T11:31:07.317037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:27.520669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.290686+0000","last_change":"2024-04-25T11:31:07.287353+0000","last_active":"2024-04-25T11:31:07.290686+0000","last_peered":"2024-04-25T11:31:07.290686+0000","last_clean":"2024-04-25T11:31:07.290686+0000","last_became_active":"2024-04-25T11:31:07.286852+0000","last_became_peered":"2024-04-25T11:31:07.286852+0000","last_unstale":"2024-04-25T11:31:07.290686+0000","last_undegraded":"2024-04-25T11:31:07.290686+0000","last_fullsized":"2024-04-25T11:31:07.290686+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:42.042946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.317935+0000","last_change":"2024-04-25T11:31:07.308241+0000","last_active":"2024-04-25T11:31:07.317935+0000","last_peered":"2024-04-25T11:31:07.317935+0000","last_clean":"2024-04-25T11:31:07.317935+0000","last_became_active":"2024-04-25T11:31:07.307926+0000","last_became_peered":"2024-04-25T11:31:07.307926+0000","last_unstale":"2024-04-25T11:31:07.317935+0000","last_undegraded":"2024-04-25T11:31:07.317935+0000","last_fullsized":"2024-04-25T11:31:07.317935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:12.756996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.300165+0000","last_change":"2024-04-25T11:31:07.290140+0000","last_active":"2024-04-25T11:31:07.300165+0000","last_peered":"2024-04-25T11:31:07.300165+0000","last_clean":"2024-04-25T11:31:07.300165+0000","last_became_active":"2024-04-25T11:31:07.289783+0000","last_became_peered":"2024-04-25T11:31:07.289783+0000","last_unstale":"2024-04-25T11:31:07.300165+0000","last_undegraded":"2024-04-25T11:31:07.300165+0000","last_fullsized":"2024-04-25T11:31:07.300165+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:13.519089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318895+0000","last_change":"2024-04-25T11:31:07.312296+0000","last_active":"2024-04-25T11:31:07.318895+0000","last_peered":"2024-04-25T11:31:07.318895+0000","last_clean":"2024-04-25T11:31:07.318895+0000","last_became_active":"2024-04-25T11:31:07.311984+0000","last_became_peered":"2024-04-25T11:31:07.311984+0000","last_unstale":"2024-04-25T11:31:07.318895+0000","last_undegraded":"2024-04-25T11:31:07.318895+0000","last_fullsized":"2024-04-25T11:31:07.318895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:45.076928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321765+0000","last_change":"2024-04-25T11:31:07.316101+0000","last_active":"2024-04-25T11:31:07.321765+0000","last_peered":"2024-04-25T11:31:07.321765+0000","last_clean":"2024-04-25T11:31:07.321765+0000","last_became_active":"2024-04-25T11:31:07.315666+0000","last_became_peered":"2024-04-25T11:31:07.315666+0000","last_unstale":"2024-04-25T11:31:07.321765+0000","last_undegraded":"2024-04-25T11:31:07.321765+0000","last_fullsized":"2024-04-25T11:31:07.321765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.654600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318447+0000","last_change":"2024-04-25T11:31:07.312406+0000","last_active":"2024-04-25T11:31:07.318447+0000","last_peered":"2024-04-25T11:31:07.318447+0000","last_clean":"2024-04-25T11:31:07.318447+0000","last_became_active":"2024-04-25T11:31:07.311728+0000","last_became_peered":"2024-04-25T11:31:07.311728+0000","last_unstale":"2024-04-25T11:31:07.318447+0000","last_undegraded":"2024-04-25T11:31:07.318447+0000","last_fullsized":"2024-04-25T11:31:07.318447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.501090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.245004+0000","last_change":"2024-04-25T11:31:06.245004+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.245004+0000","last_undegraded":"2024-04-25T11:31:06.245004+0000","last_fullsized":"2024-04-25T11:31:06.245004+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.240280+0000","last_change":"2024-04-25T11:31:06.240280+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.240280+0000","last_undegraded":"2024-04-25T11:31:06.240280+0000","last_fullsized":"2024-04-25T11:31:06.240280+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.306974+0000","last_change":"2024-04-25T11:31:07.293372+0000","last_active":"2024-04-25T11:31:07.306974+0000","last_peered":"2024-04-25T11:31:07.306974+0000","last_clean":"2024-04-25T11:31:07.306974+0000","last_became_active":"2024-04-25T11:31:07.293003+0000","last_became_peered":"2024-04-25T11:31:07.293003+0000","last_unstale":"2024-04-25T11:31:07.306974+0000","last_undegraded":"2024-04-25T11:31:07.306974+0000","last_fullsized":"2024-04-25T11:31:07.306974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:34.416219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318232+0000","last_change":"2024-04-25T11:31:07.309813+0000","last_active":"2024-04-25T11:31:07.318232+0000","last_peered":"2024-04-25T11:31:07.318232+0000","last_clean":"2024-04-25T11:31:07.318232+0000","last_became_active":"2024-04-25T11:31:07.309163+0000","last_became_peered":"2024-04-25T11:31:07.309163+0000","last_unstale":"2024-04-25T11:31:07.318232+0000","last_undegraded":"2024-04-25T11:31:07.318232+0000","last_fullsized":"2024-04-25T11:31:07.318232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.203850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.306616+0000","last_change":"2024-04-25T11:31:07.302815+0000","last_active":"2024-04-25T11:31:07.306616+0000","last_peered":"2024-04-25T11:31:07.306616+0000","last_clean":"2024-04-25T11:31:07.306616+0000","last_became_active":"2024-04-25T11:31:07.302468+0000","last_became_peered":"2024-04-25T11:31:07.302468+0000","last_unstale":"2024-04-25T11:31:07.306616+0000","last_undegraded":"2024-04-25T11:31:07.306616+0000","last_fullsized":"2024-04-25T11:31:07.306616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:34.731773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.306514+0000","last_change":"2024-04-25T11:31:07.300690+0000","last_active":"2024-04-25T11:31:07.306514+0000","last_peered":"2024-04-25T11:31:07.306514+0000","last_clean":"2024-04-25T11:31:07.306514+0000","last_became_active":"2024-04-25T11:31:07.300376+0000","last_became_peered":"2024-04-25T11:31:07.300376+0000","last_unstale":"2024-04-25T11:31:07.306514+0000","last_undegraded":"2024-04-25T11:31:07.306514+0000","last_fullsized":"2024-04-25T11:31:07.306514+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:21.798269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321733+0000","last_change":"2024-04-25T11:31:06.264715+0000","last_active":"2024-04-25T11:31:07.321733+0000","last_peered":"2024-04-25T11:31:07.321733+0000","last_clean":"2024-04-25T11:31:07.321733+0000","last_became_active":"2024-04-25T11:31:06.263282+0000","last_became_peered":"2024-04-25T11:31:06.263282+0000","last_unstale":"2024-04-25T11:31:07.321733+0000","last_undegraded":"2024-04-25T11:31:07.321733+0000","last_fullsized":"2024-04-25T11:31:07.321733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:18.401152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.314944+0000","last_change":"2024-04-25T11:31:07.305693+0000","last_active":"2024-04-25T11:31:07.314944+0000","last_peered":"2024-04-25T11:31:07.314944+0000","last_clean":"2024-04-25T11:31:07.314944+0000","last_became_active":"2024-04-25T11:31:07.305177+0000","last_became_peered":"2024-04-25T11:31:07.305177+0000","last_unstale":"2024-04-25T11:31:07.314944+0000","last_undegraded":"2024-04-25T11:31:07.314944+0000","last_fullsized":"2024-04-25T11:31:07.314944+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:14.429793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.322443+0000","last_change":"2024-04-25T11:31:06.265130+0000","last_active":"2024-04-25T11:31:07.322443+0000","last_peered":"2024-04-25T11:31:07.322443+0000","last_clean":"2024-04-25T11:31:07.322443+0000","last_became_active":"2024-04-25T11:31:06.263941+0000","last_became_peered":"2024-04-25T11:31:06.263941+0000","last_unstale":"2024-04-25T11:31:07.322443+0000","last_undegraded":"2024-04-25T11:31:07.322443+0000","last_fullsized":"2024-04-25T11:31:07.322443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.501947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.243647+0000","last_change":"2024-04-25T11:31:06.243647+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.243647+0000","last_undegraded":"2024-04-25T11:31:06.243647+0000","last_fullsized":"2024-04-25T11:31:06.243647+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.319496+0000","last_change":"2024-04-25T11:31:06.269010+0000","last_active":"2024-04-25T11:31:07.319496+0000","last_peered":"2024-04-25T11:31:07.319496+0000","last_clean":"2024-04-25T11:31:07.319496+0000","last_became_active":"2024-04-25T11:31:06.268645+0000","last_became_peered":"2024-04-25T11:31:06.268645+0000","last_unstale":"2024-04-25T11:31:07.319496+0000","last_undegraded":"2024-04-25T11:31:07.319496+0000","last_fullsized":"2024-04-25T11:31:07.319496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:24.219149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.299461+0000","last_change":"2024-04-25T11:31:07.290517+0000","last_active":"2024-04-25T11:31:07.299461+0000","last_peered":"2024-04-25T11:31:07.299461+0000","last_clean":"2024-04-25T11:31:07.299461+0000","last_became_active":"2024-04-25T11:31:07.290177+0000","last_became_peered":"2024-04-25T11:31:07.290177+0000","last_unstale":"2024-04-25T11:31:07.299461+0000","last_undegraded":"2024-04-25T11:31:07.299461+0000","last_fullsized":"2024-04-25T11:31:07.299461+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:28.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321476+0000","last_change":"2024-04-25T11:31:06.248267+0000","last_active":"2024-04-25T11:31:07.321476+0000","last_peered":"2024-04-25T11:31:07.321476+0000","last_clean":"2024-04-25T11:31:07.321476+0000","last_became_active":"2024-04-25T11:31:06.247895+0000","last_became_peered":"2024-04-25T11:31:06.247895+0000","last_unstale":"2024-04-25T11:31:07.321476+0000","last_undegraded":"2024-04-25T11:31:07.321476+0000","last_fullsized":"2024-04-25T11:31:07.321476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:36.075589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.237849+0000","last_change":"2024-04-25T11:31:06.237849+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.237849+0000","last_undegraded":"2024-04-25T11:31:06.237849+0000","last_fullsized":"2024-04-25T11:31:06.237849+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.313991+0000","last_change":"2024-04-25T11:31:06.265746+0000","last_active":"2024-04-25T11:31:07.313991+0000","last_peered":"2024-04-25T11:31:07.313991+0000","last_clean":"2024-04-25T11:31:07.313991+0000","last_became_active":"2024-04-25T11:31:06.265259+0000","last_became_peered":"2024-04-25T11:31:06.265259+0000","last_unstale":"2024-04-25T11:31:07.313991+0000","last_undegraded":"2024-04-25T11:31:07.313991+0000","last_fullsized":"2024-04-25T11:31:07.313991+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:13.412651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.311934+0000","last_change":"2024-04-25T11:31:07.303002+0000","last_active":"2024-04-25T11:31:07.311934+0000","last_peered":"2024-04-25T11:31:07.311934+0000","last_clean":"2024-04-25T11:31:07.311934+0000","last_became_active":"2024-04-25T11:31:07.302386+0000","last_became_peered":"2024-04-25T11:31:07.302386+0000","last_unstale":"2024-04-25T11:31:07.311934+0000","last_undegraded":"2024-04-25T11:31:07.311934+0000","last_fullsized":"2024-04-25T11:31:07.311934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:49.681967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.320355+0000","last_change":"2024-04-25T11:31:06.268400+0000","last_active":"2024-04-25T11:31:07.320355+0000","last_peered":"2024-04-25T11:31:07.320355+0000","last_clean":"2024-04-25T11:31:07.320355+0000","last_became_active":"2024-04-25T11:31:06.268051+0000","last_became_peered":"2024-04-25T11:31:06.268051+0000","last_unstale":"2024-04-25T11:31:07.320355+0000","last_undegraded":"2024-04-25T11:31:07.320355+0000","last_fullsized":"2024-04-25T11:31:07.320355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.649935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.307376+0000","last_change":"2024-04-25T11:31:07.294940+0000","last_active":"2024-04-25T11:31:07.307376+0000","last_peered":"2024-04-25T11:31:07.307376+0000","last_clean":"2024-04-25T11:31:07.307376+0000","last_became_active":"2024-04-25T11:31:07.294387+0000","last_became_peered":"2024-04-25T11:31:07.294387+0000","last_unstale":"2024-04-25T11:31:07.307376+0000","last_undegraded":"2024-04-25T11:31:07.307376+0000","last_fullsized":"2024-04-25T11:31:07.307376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:19.418294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.270338+0000","last_change":"2024-04-25T11:31:06.256010+0000","last_active":"2024-04-25T11:31:07.270338+0000","last_peered":"2024-04-25T11:31:07.270338+0000","last_clean":"2024-04-25T11:31:07.270338+0000","last_became_active":"2024-04-25T11:31:06.255555+0000","last_became_peered":"2024-04-25T11:31:06.255555+0000","last_unstale":"2024-04-25T11:31:07.270338+0000","last_undegraded":"2024-04-25T11:31:07.270338+0000","last_fullsized":"2024-04-25T11:31:07.270338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.304675+0000","last_change":"2024-04-25T11:31:07.295236+0000","last_active":"2024-04-25T11:31:07.304675+0000","last_peered":"2024-04-25T11:31:07.304675+0000","last_clean":"2024-04-25T11:31:07.304675+0000","last_became_active":"2024-04-25T11:31:07.294855+0000","last_became_peered":"2024-04-25T11:31:07.294855+0000","last_unstale":"2024-04-25T11:31:07.304675+0000","last_undegraded":"2024-04-25T11:31:07.304675+0000","last_fullsized":"2024-04-25T11:31:07.304675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:35.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.285755+0000","last_change":"2024-04-25T11:31:06.261137+0000","last_active":"2024-04-25T11:31:07.285755+0000","last_peered":"2024-04-25T11:31:07.285755+0000","last_clean":"2024-04-25T11:31:07.285755+0000","last_became_active":"2024-04-25T11:31:06.260698+0000","last_became_peered":"2024-04-25T11:31:06.260698+0000","last_unstale":"2024-04-25T11:31:07.285755+0000","last_undegraded":"2024-04-25T11:31:07.285755+0000","last_fullsized":"2024-04-25T11:31:07.285755+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:51.188601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.314149+0000","last_change":"2024-04-25T11:31:07.299480+0000","last_active":"2024-04-25T11:31:07.314149+0000","last_peered":"2024-04-25T11:31:07.314149+0000","last_clean":"2024-04-25T11:31:07.314149+0000","last_became_active":"2024-04-25T11:31:07.299171+0000","last_became_peered":"2024-04-25T11:31:07.299171+0000","last_unstale":"2024-04-25T11:31:07.314149+0000","last_undegraded":"2024-04-25T11:31:07.314149+0000","last_fullsized":"2024-04-25T11:31:07.314149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:14.429793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.273530+0000","last_change":"2024-04-25T11:31:06.262349+0000","last_active":"2024-04-25T11:31:07.273530+0000","last_peered":"2024-04-25T11:31:07.273530+0000","last_clean":"2024-04-25T11:31:07.273530+0000","last_became_active":"2024-04-25T11:31:06.261724+0000","last_became_peered":"2024-04-25T11:31:06.261724+0000","last_unstale":"2024-04-25T11:31:07.273530+0000","last_undegraded":"2024-04-25T11:31:07.273530+0000","last_fullsized":"2024-04-25T11:31:07.273530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:12.839356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.307989+0000","last_change":"2024-04-25T11:31:07.298405+0000","last_active":"2024-04-25T11:31:07.307989+0000","last_peered":"2024-04-25T11:31:07.307989+0000","last_clean":"2024-04-25T11:31:07.307989+0000","last_became_active":"2024-04-25T11:31:07.298094+0000","last_became_peered":"2024-04-25T11:31:07.298094+0000","last_unstale":"2024-04-25T11:31:07.307989+0000","last_undegraded":"2024-04-25T11:31:07.307989+0000","last_fullsized":"2024-04-25T11:31:07.307989+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:42.042946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T11:31:06.273153+0000","last_change":"2024-04-25T11:31:06.270417+0000","last_active":"2024-04-25T11:31:06.273153+0000","last_peered":"2024-04-25T11:31:06.273153+0000","last_clean":"2024-04-25T11:31:06.273153+0000","last_became_active":"2024-04-25T11:31:06.268102+0000","last_became_peered":"2024-04-25T11:31:06.268102+0000","last_unstale":"2024-04-25T11:31:06.273153+0000","last_undegraded":"2024-04-25T11:31:06.273153+0000","last_fullsized":"2024-04-25T11:31:06.273153+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:49.947131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.238070+0000","last_change":"2024-04-25T11:31:06.238070+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.238070+0000","last_undegraded":"2024-04-25T11:31:06.238070+0000","last_fullsized":"2024-04-25T11:31:06.238070+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"62'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.324028+0000","last_change":"2024-04-25T11:31:06.248828+0000","last_active":"2024-04-25T11:31:07.324028+0000","last_peered":"2024-04-25T11:31:07.324028+0000","last_clean":"2024-04-25T11:31:07.324028+0000","last_became_active":"2024-04-25T11:31:06.247850+0000","last_became_peered":"2024-04-25T11:31:06.247850+0000","last_unstale":"2024-04-25T11:31:07.324028+0000","last_undegraded":"2024-04-25T11:31:07.324028+0000","last_fullsized":"2024-04-25T11:31:07.324028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:58.873512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.286804+0000","last_change":"2024-04-25T11:31:07.279903+0000","last_active":"2024-04-25T11:31:07.286804+0000","last_peered":"2024-04-25T11:31:07.286804+0000","last_clean":"2024-04-25T11:31:07.286804+0000","last_became_active":"2024-04-25T11:31:07.278929+0000","last_became_peered":"2024-04-25T11:31:07.278929+0000","last_unstale":"2024-04-25T11:31:07.286804+0000","last_undegraded":"2024-04-25T11:31:07.286804+0000","last_fullsized":"2024-04-25T11:31:07.286804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:58.395944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.326782+0000","last_change":"2024-04-25T11:31:06.259780+0000","last_active":"2024-04-25T11:31:07.326782+0000","last_peered":"2024-04-25T11:31:07.326782+0000","last_clean":"2024-04-25T11:31:07.326782+0000","last_became_active":"2024-04-25T11:31:06.259210+0000","last_became_peered":"2024-04-25T11:31:06.259210+0000","last_unstale":"2024-04-25T11:31:07.326782+0000","last_undegraded":"2024-04-25T11:31:07.326782+0000","last_fullsized":"2024-04-25T11:31:07.326782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:58.335618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.295742+0000","last_change":"2024-04-25T11:31:07.286062+0000","last_active":"2024-04-25T11:31:07.295742+0000","last_peered":"2024-04-25T11:31:07.295742+0000","last_clean":"2024-04-25T11:31:07.295742+0000","last_became_active":"2024-04-25T11:31:07.285670+0000","last_became_peered":"2024-04-25T11:31:07.285670+0000","last_unstale":"2024-04-25T11:31:07.295742+0000","last_undegraded":"2024-04-25T11:31:07.295742+0000","last_fullsized":"2024-04-25T11:31:07.295742+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:12.947284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T11:31:06.273146+0000","last_change":"2024-04-25T11:31:06.270339+0000","last_active":"2024-04-25T11:31:06.273146+0000","last_peered":"2024-04-25T11:31:06.273146+0000","last_clean":"2024-04-25T11:31:06.273146+0000","last_became_active":"2024-04-25T11:31:06.266784+0000","last_became_peered":"2024-04-25T11:31:06.266784+0000","last_unstale":"2024-04-25T11:31:06.273146+0000","last_undegraded":"2024-04-25T11:31:06.273146+0000","last_fullsized":"2024-04-25T11:31:06.273146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:36.332003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.308644+0000","last_change":"2024-04-25T11:31:07.300848+0000","last_active":"2024-04-25T11:31:07.308644+0000","last_peered":"2024-04-25T11:31:07.308644+0000","last_clean":"2024-04-25T11:31:07.308644+0000","last_became_active":"2024-04-25T11:31:07.300080+0000","last_became_peered":"2024-04-25T11:31:07.300080+0000","last_unstale":"2024-04-25T11:31:07.308644+0000","last_undegraded":"2024-04-25T11:31:07.308644+0000","last_fullsized":"2024-04-25T11:31:07.308644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:34.731773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.262121+0000","last_change":"2024-04-25T11:31:06.254543+0000","last_active":"2024-04-25T11:31:07.262121+0000","last_peered":"2024-04-25T11:31:07.262121+0000","last_clean":"2024-04-25T11:31:07.262121+0000","last_became_active":"2024-04-25T11:31:06.254238+0000","last_became_peered":"2024-04-25T11:31:06.254238+0000","last_unstale":"2024-04-25T11:31:07.262121+0000","last_undegraded":"2024-04-25T11:31:07.262121+0000","last_fullsized":"2024-04-25T11:31:07.262121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:00.767019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.304486+0000","last_change":"2024-04-25T11:31:07.295447+0000","last_active":"2024-04-25T11:31:07.304486+0000","last_peered":"2024-04-25T11:31:07.304486+0000","last_clean":"2024-04-25T11:31:07.304486+0000","last_became_active":"2024-04-25T11:31:07.295127+0000","last_became_peered":"2024-04-25T11:31:07.295127+0000","last_unstale":"2024-04-25T11:31:07.304486+0000","last_undegraded":"2024-04-25T11:31:07.304486+0000","last_fullsized":"2024-04-25T11:31:07.304486+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:40.475792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.264548+0000","last_change":"2024-04-25T11:31:06.250388+0000","last_active":"2024-04-25T11:31:07.264548+0000","last_peered":"2024-04-25T11:31:07.264548+0000","last_clean":"2024-04-25T11:31:07.264548+0000","last_became_active":"2024-04-25T11:31:06.250060+0000","last_became_peered":"2024-04-25T11:31:06.250060+0000","last_unstale":"2024-04-25T11:31:07.264548+0000","last_undegraded":"2024-04-25T11:31:07.264548+0000","last_fullsized":"2024-04-25T11:31:07.264548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:14.538099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.305277+0000","last_change":"2024-04-25T11:31:07.300418+0000","last_active":"2024-04-25T11:31:07.305277+0000","last_peered":"2024-04-25T11:31:07.305277+0000","last_clean":"2024-04-25T11:31:07.305277+0000","last_became_active":"2024-04-25T11:31:07.300081+0000","last_became_peered":"2024-04-25T11:31:07.300081+0000","last_unstale":"2024-04-25T11:31:07.305277+0000","last_undegraded":"2024-04-25T11:31:07.305277+0000","last_fullsized":"2024-04-25T11:31:07.305277+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:06.770782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"62'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.324848+0000","last_change":"2024-04-25T11:31:06.254133+0000","last_active":"2024-04-25T11:31:07.324848+0000","last_peered":"2024-04-25T11:31:07.324848+0000","last_clean":"2024-04-25T11:31:07.324848+0000","last_became_active":"2024-04-25T11:31:06.253811+0000","last_became_peered":"2024-04-25T11:31:06.253811+0000","last_unstale":"2024-04-25T11:31:07.324848+0000","last_undegraded":"2024-04-25T11:31:07.324848+0000","last_fullsized":"2024-04-25T11:31:07.324848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:14.279767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.311709+0000","last_change":"2024-04-25T11:31:07.292883+0000","last_active":"2024-04-25T11:31:07.311709+0000","last_peered":"2024-04-25T11:31:07.311709+0000","last_clean":"2024-04-25T11:31:07.311709+0000","last_became_active":"2024-04-25T11:31:07.287159+0000","last_became_peered":"2024-04-25T11:31:07.287159+0000","last_unstale":"2024-04-25T11:31:07.311709+0000","last_undegraded":"2024-04-25T11:31:07.311709+0000","last_fullsized":"2024-04-25T11:31:07.311709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.592314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321476+0000","last_change":"2024-04-25T11:31:06.274421+0000","last_active":"2024-04-25T11:31:07.321476+0000","last_peered":"2024-04-25T11:31:07.321476+0000","last_clean":"2024-04-25T11:31:07.321476+0000","last_became_active":"2024-04-25T11:31:06.274101+0000","last_became_peered":"2024-04-25T11:31:06.274101+0000","last_unstale":"2024-04-25T11:31:07.321476+0000","last_undegraded":"2024-04-25T11:31:07.321476+0000","last_fullsized":"2024-04-25T11:31:07.321476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:12.839356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.307944+0000","last_change":"2024-04-25T11:31:07.288525+0000","last_active":"2024-04-25T11:31:07.307944+0000","last_peered":"2024-04-25T11:31:07.307944+0000","last_clean":"2024-04-25T11:31:07.307944+0000","last_became_active":"2024-04-25T11:31:07.288093+0000","last_became_peered":"2024-04-25T11:31:07.288093+0000","last_unstale":"2024-04-25T11:31:07.307944+0000","last_undegraded":"2024-04-25T11:31:07.307944+0000","last_fullsized":"2024-04-25T11:31:07.307944+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:27.520669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T11:31:06.272094+0000","last_change":"2024-04-25T11:31:06.268805+0000","last_active":"2024-04-25T11:31:06.272094+0000","last_peered":"2024-04-25T11:31:06.272094+0000","last_clean":"2024-04-25T11:31:06.272094+0000","last_became_active":"2024-04-25T11:31:06.268469+0000","last_became_peered":"2024-04-25T11:31:06.268469+0000","last_unstale":"2024-04-25T11:31:06.272094+0000","last_undegraded":"2024-04-25T11:31:06.272094+0000","last_fullsized":"2024-04-25T11:31:06.272094+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:54.904968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.319040+0000","last_change":"2024-04-25T11:31:07.317102+0000","last_active":"2024-04-25T11:31:07.319040+0000","last_peered":"2024-04-25T11:31:07.319040+0000","last_clean":"2024-04-25T11:31:07.319040+0000","last_became_active":"2024-04-25T11:31:07.316799+0000","last_became_peered":"2024-04-25T11:31:07.316799+0000","last_unstale":"2024-04-25T11:31:07.319040+0000","last_undegraded":"2024-04-25T11:31:07.319040+0000","last_fullsized":"2024-04-25T11:31:07.319040+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:28.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.319359+0000","last_change":"2024-04-25T11:31:06.261186+0000","last_active":"2024-04-25T11:31:07.319359+0000","last_peered":"2024-04-25T11:31:07.319359+0000","last_clean":"2024-04-25T11:31:07.319359+0000","last_became_active":"2024-04-25T11:31:06.260862+0000","last_became_peered":"2024-04-25T11:31:06.260862+0000","last_unstale":"2024-04-25T11:31:07.319359+0000","last_undegraded":"2024-04-25T11:31:07.319359+0000","last_fullsized":"2024-04-25T11:31:07.319359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:18.971373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.311086+0000","last_change":"2024-04-25T11:31:07.306288+0000","last_active":"2024-04-25T11:31:07.311086+0000","last_peered":"2024-04-25T11:31:07.311086+0000","last_clean":"2024-04-25T11:31:07.311086+0000","last_became_active":"2024-04-25T11:31:07.305940+0000","last_became_peered":"2024-04-25T11:31:07.305940+0000","last_unstale":"2024-04-25T11:31:07.311086+0000","last_undegraded":"2024-04-25T11:31:07.311086+0000","last_fullsized":"2024-04-25T11:31:07.311086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:10.241688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T11:31:06.273053+0000","last_change":"2024-04-25T11:31:06.269600+0000","last_active":"2024-04-25T11:31:06.273053+0000","last_peered":"2024-04-25T11:31:06.273053+0000","last_clean":"2024-04-25T11:31:06.273053+0000","last_became_active":"2024-04-25T11:31:06.268545+0000","last_became_peered":"2024-04-25T11:31:06.268545+0000","last_unstale":"2024-04-25T11:31:06.273053+0000","last_undegraded":"2024-04-25T11:31:06.273053+0000","last_fullsized":"2024-04-25T11:31:06.273053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:31.182729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.297579+0000","last_change":"2024-04-25T11:31:07.292122+0000","last_active":"2024-04-25T11:31:07.297579+0000","last_peered":"2024-04-25T11:31:07.297579+0000","last_clean":"2024-04-25T11:31:07.297579+0000","last_became_active":"2024-04-25T11:31:07.290352+0000","last_became_peered":"2024-04-25T11:31:07.290352+0000","last_unstale":"2024-04-25T11:31:07.297579+0000","last_undegraded":"2024-04-25T11:31:07.297579+0000","last_fullsized":"2024-04-25T11:31:07.297579+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.203850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.317656+0000","last_change":"2024-04-25T11:31:06.262444+0000","last_active":"2024-04-25T11:31:07.317656+0000","last_peered":"2024-04-25T11:31:07.317656+0000","last_clean":"2024-04-25T11:31:07.317656+0000","last_became_active":"2024-04-25T11:31:06.260269+0000","last_became_peered":"2024-04-25T11:31:06.260269+0000","last_unstale":"2024-04-25T11:31:07.317656+0000","last_undegraded":"2024-04-25T11:31:07.317656+0000","last_fullsized":"2024-04-25T11:31:07.317656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:13.575172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.238877+0000","last_change":"2024-04-25T11:31:06.238877+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.238877+0000","last_undegraded":"2024-04-25T11:31:06.238877+0000","last_fullsized":"2024-04-25T11:31:06.238877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T11:31:06.266965+0000","last_change":"2024-04-25T11:31:06.262430+0000","last_active":"2024-04-25T11:31:06.266965+0000","last_peered":"2024-04-25T11:31:06.266965+0000","last_clean":"2024-04-25T11:31:06.266965+0000","last_became_active":"2024-04-25T11:31:06.262088+0000","last_became_peered":"2024-04-25T11:31:06.262088+0000","last_unstale":"2024-04-25T11:31:06.266965+0000","last_undegraded":"2024-04-25T11:31:06.266965+0000","last_fullsized":"2024-04-25T11:31:06.266965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:44.254909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.237249+0000","last_change":"2024-04-25T11:31:06.237249+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.237249+0000","last_undegraded":"2024-04-25T11:31:06.237249+0000","last_fullsized":"2024-04-25T11:31:06.237249+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.271026+0000","last_change":"2024-04-25T11:31:06.268722+0000","last_active":"2024-04-25T11:31:07.271026+0000","last_peered":"2024-04-25T11:31:07.271026+0000","last_clean":"2024-04-25T11:31:07.271026+0000","last_became_active":"2024-04-25T11:31:06.268257+0000","last_became_peered":"2024-04-25T11:31:06.268257+0000","last_unstale":"2024-04-25T11:31:07.271026+0000","last_undegraded":"2024-04-25T11:31:07.271026+0000","last_fullsized":"2024-04-25T11:31:07.271026+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:02.811671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.311739+0000","last_change":"2024-04-25T11:31:07.302694+0000","last_active":"2024-04-25T11:31:07.311739+0000","last_peered":"2024-04-25T11:31:07.311739+0000","last_clean":"2024-04-25T11:31:07.311739+0000","last_became_active":"2024-04-25T11:31:07.302350+0000","last_became_peered":"2024-04-25T11:31:07.302350+0000","last_unstale":"2024-04-25T11:31:07.311739+0000","last_undegraded":"2024-04-25T11:31:07.311739+0000","last_fullsized":"2024-04-25T11:31:07.311739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:49.681967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321570+0000","last_change":"2024-04-25T11:31:06.271800+0000","last_active":"2024-04-25T11:31:07.321570+0000","last_peered":"2024-04-25T11:31:07.321570+0000","last_clean":"2024-04-25T11:31:07.321570+0000","last_became_active":"2024-04-25T11:31:06.271483+0000","last_became_peered":"2024-04-25T11:31:06.271483+0000","last_unstale":"2024-04-25T11:31:07.321570+0000","last_undegraded":"2024-04-25T11:31:07.321570+0000","last_fullsized":"2024-04-25T11:31:07.321570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:11.930142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.307968+0000","last_change":"2024-04-25T11:31:07.289917+0000","last_active":"2024-04-25T11:31:07.307968+0000","last_peered":"2024-04-25T11:31:07.307968+0000","last_clean":"2024-04-25T11:31:07.307968+0000","last_became_active":"2024-04-25T11:31:07.289583+0000","last_became_peered":"2024-04-25T11:31:07.289583+0000","last_unstale":"2024-04-25T11:31:07.307968+0000","last_undegraded":"2024-04-25T11:31:07.307968+0000","last_fullsized":"2024-04-25T11:31:07.307968+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.654600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.293506+0000","last_change":"2024-04-25T11:31:06.272906+0000","last_active":"2024-04-25T11:31:07.293506+0000","last_peered":"2024-04-25T11:31:07.293506+0000","last_clean":"2024-04-25T11:31:07.293506+0000","last_became_active":"2024-04-25T11:31:06.272242+0000","last_became_peered":"2024-04-25T11:31:06.272242+0000","last_unstale":"2024-04-25T11:31:07.293506+0000","last_undegraded":"2024-04-25T11:31:07.293506+0000","last_fullsized":"2024-04-25T11:31:07.293506+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:50.474918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318214+0000","last_change":"2024-04-25T11:31:07.310583+0000","last_active":"2024-04-25T11:31:07.318214+0000","last_peered":"2024-04-25T11:31:07.318214+0000","last_clean":"2024-04-25T11:31:07.318214+0000","last_became_active":"2024-04-25T11:31:07.309600+0000","last_became_peered":"2024-04-25T11:31:07.309600+0000","last_unstale":"2024-04-25T11:31:07.318214+0000","last_undegraded":"2024-04-25T11:31:07.318214+0000","last_fullsized":"2024-04-25T11:31:07.318214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:40.475792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.271755+0000","last_change":"2024-04-25T11:31:06.254976+0000","last_active":"2024-04-25T11:31:07.271755+0000","last_peered":"2024-04-25T11:31:07.271755+0000","last_clean":"2024-04-25T11:31:07.271755+0000","last_became_active":"2024-04-25T11:31:06.254500+0000","last_became_peered":"2024-04-25T11:31:06.254500+0000","last_unstale":"2024-04-25T11:31:07.271755+0000","last_undegraded":"2024-04-25T11:31:07.271755+0000","last_fullsized":"2024-04-25T11:31:07.271755+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:20.513794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.237270+0000","last_change":"2024-04-25T11:31:06.237270+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.237270+0000","last_undegraded":"2024-04-25T11:31:06.237270+0000","last_fullsized":"2024-04-25T11:31:06.237270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.284500+0000","last_change":"2024-04-25T11:31:06.241045+0000","last_active":"2024-04-25T11:31:07.284500+0000","last_peered":"2024-04-25T11:31:07.284500+0000","last_clean":"2024-04-25T11:31:07.284500+0000","last_became_active":"2024-04-25T11:31:06.238913+0000","last_became_peered":"2024-04-25T11:31:06.238913+0000","last_unstale":"2024-04-25T11:31:07.284500+0000","last_undegraded":"2024-04-25T11:31:07.284500+0000","last_fullsized":"2024-04-25T11:31:07.284500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:49.794058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.299008+0000","last_change":"2024-04-25T11:31:07.288902+0000","last_active":"2024-04-25T11:31:07.299008+0000","last_peered":"2024-04-25T11:31:07.299008+0000","last_clean":"2024-04-25T11:31:07.299008+0000","last_became_active":"2024-04-25T11:31:07.288281+0000","last_became_peered":"2024-04-25T11:31:07.288281+0000","last_unstale":"2024-04-25T11:31:07.299008+0000","last_undegraded":"2024-04-25T11:31:07.299008+0000","last_fullsized":"2024-04-25T11:31:07.299008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:45.076928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.295942+0000","last_change":"2024-04-25T11:31:06.274502+0000","last_active":"2024-04-25T11:31:07.295942+0000","last_peered":"2024-04-25T11:31:07.295942+0000","last_clean":"2024-04-25T11:31:07.295942+0000","last_became_active":"2024-04-25T11:31:06.274181+0000","last_became_peered":"2024-04-25T11:31:06.274181+0000","last_unstale":"2024-04-25T11:31:07.295942+0000","last_undegraded":"2024-04-25T11:31:07.295942+0000","last_fullsized":"2024-04-25T11:31:07.295942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:20.513794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.315089+0000","last_change":"2024-04-25T11:31:07.301414+0000","last_active":"2024-04-25T11:31:07.315089+0000","last_peered":"2024-04-25T11:31:07.315089+0000","last_clean":"2024-04-25T11:31:07.315089+0000","last_became_active":"2024-04-25T11:31:07.301012+0000","last_became_peered":"2024-04-25T11:31:07.301012+0000","last_unstale":"2024-04-25T11:31:07.315089+0000","last_undegraded":"2024-04-25T11:31:07.315089+0000","last_fullsized":"2024-04-25T11:31:07.315089+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:21.068983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.283220+0000","last_change":"2024-04-25T11:31:06.258161+0000","last_active":"2024-04-25T11:31:07.283220+0000","last_peered":"2024-04-25T11:31:07.283220+0000","last_clean":"2024-04-25T11:31:07.283220+0000","last_became_active":"2024-04-25T11:31:06.257780+0000","last_became_peered":"2024-04-25T11:31:06.257780+0000","last_unstale":"2024-04-25T11:31:07.283220+0000","last_undegraded":"2024-04-25T11:31:07.283220+0000","last_fullsized":"2024-04-25T11:31:07.283220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:20.051841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.239257+0000","last_change":"2024-04-25T11:31:06.239257+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.239257+0000","last_undegraded":"2024-04-25T11:31:06.239257+0000","last_fullsized":"2024-04-25T11:31:06.239257+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.291861+0000","last_change":"2024-04-25T11:31:06.269909+0000","last_active":"2024-04-25T11:31:07.291861+0000","last_peered":"2024-04-25T11:31:07.291861+0000","last_clean":"2024-04-25T11:31:07.291861+0000","last_became_active":"2024-04-25T11:31:06.269503+0000","last_became_peered":"2024-04-25T11:31:06.269503+0000","last_unstale":"2024-04-25T11:31:07.291861+0000","last_undegraded":"2024-04-25T11:31:07.291861+0000","last_fullsized":"2024-04-25T11:31:07.291861+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:20.462937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.316159+0000","last_change":"2024-04-25T11:31:07.306668+0000","last_active":"2024-04-25T11:31:07.316159+0000","last_peered":"2024-04-25T11:31:07.316159+0000","last_clean":"2024-04-25T11:31:07.316159+0000","last_became_active":"2024-04-25T11:31:07.306128+0000","last_became_peered":"2024-04-25T11:31:07.306128+0000","last_unstale":"2024-04-25T11:31:07.316159+0000","last_undegraded":"2024-04-25T11:31:07.316159+0000","last_fullsized":"2024-04-25T11:31:07.316159+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:16.553301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T11:31:06.252765+0000","last_change":"2024-04-25T11:31:06.246690+0000","last_active":"2024-04-25T11:31:06.252765+0000","last_peered":"2024-04-25T11:31:06.252765+0000","last_clean":"2024-04-25T11:31:06.252765+0000","last_became_active":"2024-04-25T11:31:06.246068+0000","last_became_peered":"2024-04-25T11:31:06.246068+0000","last_unstale":"2024-04-25T11:31:06.252765+0000","last_undegraded":"2024-04-25T11:31:06.252765+0000","last_fullsized":"2024-04-25T11:31:06.252765+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:26.428313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T11:31:06.236590+0000","last_change":"2024-04-25T11:31:06.236590+0000","last_active":"2024-04-25T11:31:06.217456+0000","last_peered":"2024-04-25T11:31:06.217456+0000","last_clean":"2024-04-25T11:31:06.217456+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:31:06.236590+0000","last_undegraded":"2024-04-25T11:31:06.236590+0000","last_fullsized":"2024-04-25T11:31:06.236590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.286892+0000","last_change":"2024-04-25T11:31:06.259986+0000","last_active":"2024-04-25T11:31:07.286892+0000","last_peered":"2024-04-25T11:31:07.286892+0000","last_clean":"2024-04-25T11:31:07.286892+0000","last_became_active":"2024-04-25T11:31:06.259472+0000","last_became_peered":"2024-04-25T11:31:06.259472+0000","last_unstale":"2024-04-25T11:31:07.286892+0000","last_undegraded":"2024-04-25T11:31:07.286892+0000","last_fullsized":"2024-04-25T11:31:07.286892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:20.781128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":186,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.270928+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:31:07.270928+0000","last_peered":"2024-04-25T11:31:07.270928+0000","last_clean":"2024-04-25T11:31:07.270928+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:31:07.270928+0000","last_undegraded":"2024-04-25T11:31:07.270928+0000","last_fullsized":"2024-04-25T11:31:07.270928+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":737280,"data_stored":308370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":301595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":757760,"data_stored":310142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1318912,"data_stored":891963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":301595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542193,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":891963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:31:12.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:31:12.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.353+0000 7efdf3e00640 1 Processor -- start 2024-04-25T11:31:12.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.353+0000 7efdf3e00640 1 -- start start 2024-04-25T11:31:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.355+0000 7efdf3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efdf409b7d0 0x7efdf4075e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.355+0000 7efdf3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf40b2440 con 0x7efdf40b3ee0 2024-04-25T11:31:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.355+0000 7efdf3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf40b26e0 con 0x7efdf406d590 2024-04-25T11:31:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.355+0000 7efdf3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efdf406ddc0 con 0x7efdf409b7d0 2024-04-25T11:31:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.355+0000 7efdf3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efdf40b3ee0 0x7efdf4169db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58638/0 (socket says 172.21.15.19:58638) 2024-04-25T11:31:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.355+0000 7efdf3400640 1 -- 172.21.15.19:0/779321545 learned_addr learned my addr 172.21.15.19:0/779321545 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf2a00640 1 --2- 172.21.15.19:0/779321545 >> v2:172.21.15.169:3300/0 conn(0x7efdf409b7d0 0x7efdf4075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860504860 0 0) 0x7efdf40b2440 con 0x7efdf40b3ee0 2024-04-25T11:31:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdd8003520 con 0x7efdf40b3ee0 2024-04-25T11:31:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 673781940 0 0) 0x7efdd8003520 con 0x7efdf40b3ee0 2024-04-25T11:31:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 >> v2:172.21.15.169:3300/0 conn(0x7efdf409b7d0 msgr2=0x7efdf4075e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 --2- 172.21.15.19:0/779321545 >> v2:172.21.15.169:3300/0 conn(0x7efdf409b7d0 0x7efdf4075e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 >> v1:172.21.15.169:6789/0 conn(0x7efdf406d590 legacy=0x7efdf40b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf40b29c0 con 0x7efdf40b3ee0 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efdec0019f0 con 0x7efdf40b3ee0 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 >> v1:172.21.15.19:6789/0 conn(0x7efdf40b3ee0 legacy=0x7efdf4169db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd80039b0 0x7efdd8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd8004800 0x7efdd8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdd8005250 0x7efdd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdf406ddc0 con 0x7efdd8004800 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdd800c1e0 con 0x7efdd80039b0 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.356+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdd800c3e0 con 0x7efdd8005250 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf2000640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd8004800 0x7efdd8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf2a00640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd80039b0 0x7efdd8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf3400640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdd8005250 0x7efdd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf2000640 1 -- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdd8005250 msgr2=0x7efdd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf2000640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdd8005250 0x7efdd800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf2000640 1 -- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd80039b0 msgr2=0x7efdd8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf2000640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd80039b0 0x7efdd8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf2000640 1 -- 172.21.15.19:0/779321545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd8003520 con 0x7efdd8004800 2024-04-25T11:31:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.357+0000 7efdf3400640 1 -- 172.21.15.19:0/779321545 reap_dead start 2024-04-25T11:31:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.358+0000 7efdf2000640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd8004800 0x7efdd8004c10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efddc001720 tx=0x7efdd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbeceec690afa87b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.359+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc001370 con 0x7efdd8004800 2024-04-25T11:31:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.359+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efddc0052d0 con 0x7efdd8004800 2024-04-25T11:31:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.359+0000 7efdf1600640 1 -- 172.21.15.19:0/779321545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc02ab90 con 0x7efdd8004800 2024-04-25T11:31:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.359+0000 7efdf3e00640 1 -- 172.21.15.19:0/779321545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd8004800 msgr2=0x7efdd8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.359+0000 7efdf3e00640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd8004800 0x7efdd8004c10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efddc001720 tx=0x7efdd8005830 comp rx=0 tx=0).stop 2024-04-25T11:31:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.360+0000 7efdf3e00640 1 -- 172.21.15.19:0/779321545 shutdown_connections 2024-04-25T11:31:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.360+0000 7efdf3e00640 1 --2- 172.21.15.19:0/779321545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdd8004800 0x7efdd8004c10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.360+0000 7efdf3e00640 1 -- 172.21.15.19:0/779321545 >> 172.21.15.19:0/779321545 conn(0x7efdf4073920 msgr2=0x7efdf40b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.360+0000 7efdf3e00640 1 -- 172.21.15.19:0/779321545 shutdown_connections 2024-04-25T11:31:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.360+0000 7efdf3e00640 1 -- 172.21.15.19:0/779321545 wait complete. 2024-04-25T11:31:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.362+0000 7efdf3e00640 1 Processor -- start 2024-04-25T11:31:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.362+0000 7efdf3e00640 1 -- start start 2024-04-25T11:31:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd8004800 0x7efdf4149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf41e8a20 0x7efdf409f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdf409f6d0 0x7efdf409bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdf40b4680 con 0x7efdf409f6d0 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efdf40b4500 con 0x7efdd8004800 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efdf406d100 con 0x7efdf41e8a20 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdf409f6d0 0x7efdf409bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdf409f6d0 0x7efdf409bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42512/0 (socket says 172.21.15.19:42512) 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.364+0000 7efdf3400640 1 -- 172.21.15.19:0/1126959861 learned_addr learned my addr 172.21.15.19:0/1126959861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf2000640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf41e8a20 0x7efdf409f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf2a00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd8004800 0x7efdf4149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf3400640 1 -- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf41e8a20 msgr2=0x7efdf409f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf3400640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf41e8a20 0x7efdf409f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf3400640 1 -- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd8004800 msgr2=0x7efdf4149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf3400640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd8004800 0x7efdf4149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf3400640 1 -- 172.21.15.19:0/1126959861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf409fae0 con 0x7efdf409f6d0 2024-04-25T11:31:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf3400640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdf409f6d0 0x7efdf409bf50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efdf406f140 tx=0x7efdec00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdeaa00640 1 -- 172.21.15.19:0/1126959861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec00ec10 con 0x7efdf409f6d0 2024-04-25T11:31:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdeaa00640 1 -- 172.21.15.19:0/1126959861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec018070 con 0x7efdf409f6d0 2024-04-25T11:31:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdeaa00640 1 -- 172.21.15.19:0/1126959861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec013420 con 0x7efdf409f6d0 2024-04-25T11:31:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.365+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf41e94a0 con 0x7efdf409f6d0 2024-04-25T11:31:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.366+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf41e9a60 con 0x7efdf409f6d0 2024-04-25T11:31:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.367+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf4050740 con 0x7efdf409f6d0 2024-04-25T11:31:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.368+0000 7efdeaa00640 1 -- 172.21.15.19:0/1126959861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efdec0135c0 con 0x7efdf409f6d0 2024-04-25T11:31:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.369+0000 7efdeaa00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdd00744e0 0x7efdd00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.369+0000 7efdeaa00640 1 -- 172.21.15.19:0/1126959861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7efdec095600 con 0x7efdf409f6d0 2024-04-25T11:31:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.370+0000 7efdf2a00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdd00744e0 0x7efdd00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.371+0000 7efdf2a00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdd00744e0 0x7efdd00769f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efdf406bc20 tx=0x7efde4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.374+0000 7efdeaa00640 1 -- 172.21.15.19:0/1126959861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdf409fae0 con 0x7efdf409f6d0 2024-04-25T11:31:12.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.574+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efdf40b1940 con 0x7efdd00744e0 2024-04-25T11:31:12.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.594+0000 7efdeaa00640 1 -- 172.21.15.19:0/1126959861 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7efdf40b1940 con 0x7efdd00744e0 2024-04-25T11:31:12.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdd00744e0 msgr2=0x7efdd00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdd00744e0 0x7efdd00769f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efdf406bc20 tx=0x7efde4008040 comp rx=0 tx=0).stop 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdf409f6d0 msgr2=0x7efdf409bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdf409f6d0 0x7efdf409bf50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efdf406f140 tx=0x7efdec00e530 comp rx=0 tx=0).stop 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 shutdown_connections 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efdd00744e0 0x7efdd00769f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdf409f6d0 0x7efdf409bf50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efdf41e8a20 0x7efdf409f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 --2- 172.21.15.19:0/1126959861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efdd8004800 0x7efdf4149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 >> 172.21.15.19:0/1126959861 conn(0x7efdf4073920 msgr2=0x7efdf4160770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 shutdown_connections 2024-04-25T11:31:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.600+0000 7efdf3e00640 1 -- 172.21.15.19:0/1126959861 wait complete. 2024-04-25T11:31:12.618 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-04-25T11:31:12.529870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248140,"kb_used_data":6992,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726580,"statfs":{"total":773094113280,"available":772840017920,"internally_reserved":0,"allocated":7159808,"data_stored":3668146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.300374"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.304075+0000","last_change":"2024-04-25T11:31:07.304701+0000","last_active":"2024-04-25T11:31:08.304075+0000","last_peered":"2024-04-25T11:31:08.304075+0000","last_clean":"2024-04-25T11:31:08.304075+0000","last_became_active":"2024-04-25T11:31:07.304192+0000","last_became_peered":"2024-04-25T11:31:07.304192+0000","last_unstale":"2024-04-25T11:31:08.304075+0000","last_undegraded":"2024-04-25T11:31:08.304075+0000","last_fullsized":"2024-04-25T11:31:08.304075+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:34.731773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.319005+0000","last_change":"2024-04-25T11:31:07.316843+0000","last_active":"2024-04-25T11:31:07.319005+0000","last_peered":"2024-04-25T11:31:07.319005+0000","last_clean":"2024-04-25T11:31:07.319005+0000","last_became_active":"2024-04-25T11:31:07.316438+0000","last_became_peered":"2024-04-25T11:31:07.316438+0000","last_unstale":"2024-04-25T11:31:07.319005+0000","last_undegraded":"2024-04-25T11:31:07.319005+0000","last_fullsized":"2024-04-25T11:31:07.319005+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:24.715863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309812+0000","last_change":"2024-04-25T11:31:07.298387+0000","last_active":"2024-04-25T11:31:08.309812+0000","last_peered":"2024-04-25T11:31:08.309812+0000","last_clean":"2024-04-25T11:31:08.309812+0000","last_became_active":"2024-04-25T11:31:07.297861+0000","last_became_peered":"2024-04-25T11:31:07.297861+0000","last_unstale":"2024-04-25T11:31:08.309812+0000","last_undegraded":"2024-04-25T11:31:08.309812+0000","last_fullsized":"2024-04-25T11:31:08.309812+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:16.291743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.499616+0000","last_change":"2024-04-25T11:31:07.305886+0000","last_active":"2024-04-25T11:31:09.499616+0000","last_peered":"2024-04-25T11:31:09.499616+0000","last_clean":"2024-04-25T11:31:09.499616+0000","last_became_active":"2024-04-25T11:31:07.304514+0000","last_became_peered":"2024-04-25T11:31:07.304514+0000","last_unstale":"2024-04-25T11:31:09.499616+0000","last_undegraded":"2024-04-25T11:31:09.499616+0000","last_fullsized":"2024-04-25T11:31:09.499616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:22.858882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.302580+0000","last_change":"2024-04-25T11:31:07.290012+0000","last_active":"2024-04-25T11:31:08.302580+0000","last_peered":"2024-04-25T11:31:08.302580+0000","last_clean":"2024-04-25T11:31:08.302580+0000","last_became_active":"2024-04-25T11:31:07.289658+0000","last_became_peered":"2024-04-25T11:31:07.289658+0000","last_unstale":"2024-04-25T11:31:08.302580+0000","last_undegraded":"2024-04-25T11:31:08.302580+0000","last_fullsized":"2024-04-25T11:31:08.302580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:25.078435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.301428+0000","last_change":"2024-04-25T11:31:07.288917+0000","last_active":"2024-04-25T11:31:08.301428+0000","last_peered":"2024-04-25T11:31:08.301428+0000","last_clean":"2024-04-25T11:31:08.301428+0000","last_became_active":"2024-04-25T11:31:07.288510+0000","last_became_peered":"2024-04-25T11:31:07.288510+0000","last_unstale":"2024-04-25T11:31:08.301428+0000","last_undegraded":"2024-04-25T11:31:08.301428+0000","last_fullsized":"2024-04-25T11:31:08.301428+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:27.520669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.498618+0000","last_change":"2024-04-25T11:31:07.291281+0000","last_active":"2024-04-25T11:31:09.498618+0000","last_peered":"2024-04-25T11:31:09.498618+0000","last_clean":"2024-04-25T11:31:09.498618+0000","last_became_active":"2024-04-25T11:31:07.290866+0000","last_became_peered":"2024-04-25T11:31:07.290866+0000","last_unstale":"2024-04-25T11:31:09.498618+0000","last_undegraded":"2024-04-25T11:31:09.498618+0000","last_fullsized":"2024-04-25T11:31:09.498618+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:47.042932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.299378+0000","last_change":"2024-04-25T11:31:07.285533+0000","last_active":"2024-04-25T11:31:08.299378+0000","last_peered":"2024-04-25T11:31:08.299378+0000","last_clean":"2024-04-25T11:31:08.299378+0000","last_became_active":"2024-04-25T11:31:07.285176+0000","last_became_peered":"2024-04-25T11:31:07.285176+0000","last_unstale":"2024-04-25T11:31:08.299378+0000","last_undegraded":"2024-04-25T11:31:08.299378+0000","last_fullsized":"2024-04-25T11:31:08.299378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.296607+0000","last_change":"2024-04-25T11:31:07.306210+0000","last_active":"2024-04-25T11:31:08.296607+0000","last_peered":"2024-04-25T11:31:08.296607+0000","last_clean":"2024-04-25T11:31:08.296607+0000","last_became_active":"2024-04-25T11:31:07.305864+0000","last_became_peered":"2024-04-25T11:31:07.305864+0000","last_unstale":"2024-04-25T11:31:08.296607+0000","last_undegraded":"2024-04-25T11:31:08.296607+0000","last_fullsized":"2024-04-25T11:31:08.296607+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:35.169147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.303458+0000","last_change":"2024-04-25T11:31:07.313076+0000","last_active":"2024-04-25T11:31:08.303458+0000","last_peered":"2024-04-25T11:31:08.303458+0000","last_clean":"2024-04-25T11:31:08.303458+0000","last_became_active":"2024-04-25T11:31:07.310245+0000","last_became_peered":"2024-04-25T11:31:07.310245+0000","last_unstale":"2024-04-25T11:31:08.303458+0000","last_undegraded":"2024-04-25T11:31:08.303458+0000","last_fullsized":"2024-04-25T11:31:08.303458+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:19.418294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.297745+0000","last_change":"2024-04-25T11:31:07.292601+0000","last_active":"2024-04-25T11:31:07.297745+0000","last_peered":"2024-04-25T11:31:07.297745+0000","last_clean":"2024-04-25T11:31:07.297745+0000","last_became_active":"2024-04-25T11:31:07.292250+0000","last_became_peered":"2024-04-25T11:31:07.292250+0000","last_unstale":"2024-04-25T11:31:07.297745+0000","last_undegraded":"2024-04-25T11:31:07.297745+0000","last_fullsized":"2024-04-25T11:31:07.297745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:12.203670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.296138+0000","last_change":"2024-04-25T11:31:07.300826+0000","last_active":"2024-04-25T11:31:08.296138+0000","last_peered":"2024-04-25T11:31:08.296138+0000","last_clean":"2024-04-25T11:31:08.296138+0000","last_became_active":"2024-04-25T11:31:07.300489+0000","last_became_peered":"2024-04-25T11:31:07.300489+0000","last_unstale":"2024-04-25T11:31:08.296138+0000","last_undegraded":"2024-04-25T11:31:08.296138+0000","last_fullsized":"2024-04-25T11:31:08.296138+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:51.492060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.315114+0000","last_change":"2024-04-25T11:31:07.305473+0000","last_active":"2024-04-25T11:31:07.315114+0000","last_peered":"2024-04-25T11:31:07.315114+0000","last_clean":"2024-04-25T11:31:07.315114+0000","last_became_active":"2024-04-25T11:31:07.304950+0000","last_became_peered":"2024-04-25T11:31:07.304950+0000","last_unstale":"2024-04-25T11:31:07.315114+0000","last_undegraded":"2024-04-25T11:31:07.315114+0000","last_fullsized":"2024-04-25T11:31:07.315114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:50.811200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309432+0000","last_change":"2024-04-25T11:31:07.290883+0000","last_active":"2024-04-25T11:31:08.309432+0000","last_peered":"2024-04-25T11:31:08.309432+0000","last_clean":"2024-04-25T11:31:08.309432+0000","last_became_active":"2024-04-25T11:31:07.288696+0000","last_became_peered":"2024-04-25T11:31:07.288696+0000","last_unstale":"2024-04-25T11:31:08.309432+0000","last_undegraded":"2024-04-25T11:31:08.309432+0000","last_fullsized":"2024-04-25T11:31:08.309432+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:40.919872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.315283+0000","last_change":"2024-04-25T11:31:07.308421+0000","last_active":"2024-04-25T11:31:07.315283+0000","last_peered":"2024-04-25T11:31:07.315283+0000","last_clean":"2024-04-25T11:31:07.315283+0000","last_became_active":"2024-04-25T11:31:07.305522+0000","last_became_peered":"2024-04-25T11:31:07.305522+0000","last_unstale":"2024-04-25T11:31:07.315283+0000","last_undegraded":"2024-04-25T11:31:07.315283+0000","last_fullsized":"2024-04-25T11:31:07.315283+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.988515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.299386+0000","last_change":"2024-04-25T11:31:07.299464+0000","last_active":"2024-04-25T11:31:08.299386+0000","last_peered":"2024-04-25T11:31:08.299386+0000","last_clean":"2024-04-25T11:31:08.299386+0000","last_became_active":"2024-04-25T11:31:07.298017+0000","last_became_peered":"2024-04-25T11:31:07.298017+0000","last_unstale":"2024-04-25T11:31:08.299386+0000","last_undegraded":"2024-04-25T11:31:08.299386+0000","last_fullsized":"2024-04-25T11:31:08.299386+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:19.988515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.304332+0000","last_change":"2024-04-25T11:31:07.305536+0000","last_active":"2024-04-25T11:31:08.304332+0000","last_peered":"2024-04-25T11:31:08.304332+0000","last_clean":"2024-04-25T11:31:08.304332+0000","last_became_active":"2024-04-25T11:31:07.304679+0000","last_became_peered":"2024-04-25T11:31:07.304679+0000","last_unstale":"2024-04-25T11:31:08.304332+0000","last_undegraded":"2024-04-25T11:31:08.304332+0000","last_fullsized":"2024-04-25T11:31:08.304332+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:49.681967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309919+0000","last_change":"2024-04-25T11:31:07.311466+0000","last_active":"2024-04-25T11:31:08.309919+0000","last_peered":"2024-04-25T11:31:08.309919+0000","last_clean":"2024-04-25T11:31:08.309919+0000","last_became_active":"2024-04-25T11:31:07.304996+0000","last_became_peered":"2024-04-25T11:31:07.304996+0000","last_unstale":"2024-04-25T11:31:08.309919+0000","last_undegraded":"2024-04-25T11:31:08.309919+0000","last_fullsized":"2024-04-25T11:31:08.309919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:19.448925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318423+0000","last_change":"2024-04-25T11:31:07.311429+0000","last_active":"2024-04-25T11:31:07.318423+0000","last_peered":"2024-04-25T11:31:07.318423+0000","last_clean":"2024-04-25T11:31:07.318423+0000","last_became_active":"2024-04-25T11:31:07.310949+0000","last_became_peered":"2024-04-25T11:31:07.310949+0000","last_unstale":"2024-04-25T11:31:07.318423+0000","last_undegraded":"2024-04-25T11:31:07.318423+0000","last_fullsized":"2024-04-25T11:31:07.318423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:25.648569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.313412+0000","last_change":"2024-04-25T11:31:07.309181+0000","last_active":"2024-04-25T11:31:07.313412+0000","last_peered":"2024-04-25T11:31:07.313412+0000","last_clean":"2024-04-25T11:31:07.313412+0000","last_became_active":"2024-04-25T11:31:07.306167+0000","last_became_peered":"2024-04-25T11:31:07.306167+0000","last_unstale":"2024-04-25T11:31:07.313412+0000","last_undegraded":"2024-04-25T11:31:07.313412+0000","last_fullsized":"2024-04-25T11:31:07.313412+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.933478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.317037+0000","last_change":"2024-04-25T11:31:07.311217+0000","last_active":"2024-04-25T11:31:07.317037+0000","last_peered":"2024-04-25T11:31:07.317037+0000","last_clean":"2024-04-25T11:31:07.317037+0000","last_became_active":"2024-04-25T11:31:07.310916+0000","last_became_peered":"2024-04-25T11:31:07.310916+0000","last_unstale":"2024-04-25T11:31:07.317037+0000","last_undegraded":"2024-04-25T11:31:07.317037+0000","last_fullsized":"2024-04-25T11:31:07.317037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:27.520669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.290686+0000","last_change":"2024-04-25T11:31:07.287353+0000","last_active":"2024-04-25T11:31:07.290686+0000","last_peered":"2024-04-25T11:31:07.290686+0000","last_clean":"2024-04-25T11:31:07.290686+0000","last_became_active":"2024-04-25T11:31:07.286852+0000","last_became_peered":"2024-04-25T11:31:07.286852+0000","last_unstale":"2024-04-25T11:31:07.290686+0000","last_undegraded":"2024-04-25T11:31:07.290686+0000","last_fullsized":"2024-04-25T11:31:07.290686+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:42.042946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.498827+0000","last_change":"2024-04-25T11:31:07.308241+0000","last_active":"2024-04-25T11:31:09.498827+0000","last_peered":"2024-04-25T11:31:09.498827+0000","last_clean":"2024-04-25T11:31:09.498827+0000","last_became_active":"2024-04-25T11:31:07.307926+0000","last_became_peered":"2024-04-25T11:31:07.307926+0000","last_unstale":"2024-04-25T11:31:09.498827+0000","last_undegraded":"2024-04-25T11:31:09.498827+0000","last_fullsized":"2024-04-25T11:31:09.498827+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:53.706199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.295509+0000","last_change":"2024-04-25T11:31:07.290140+0000","last_active":"2024-04-25T11:31:08.295509+0000","last_peered":"2024-04-25T11:31:08.295509+0000","last_clean":"2024-04-25T11:31:08.295509+0000","last_became_active":"2024-04-25T11:31:07.289783+0000","last_became_peered":"2024-04-25T11:31:07.289783+0000","last_unstale":"2024-04-25T11:31:08.295509+0000","last_undegraded":"2024-04-25T11:31:08.295509+0000","last_fullsized":"2024-04-25T11:31:08.295509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:52.667077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318895+0000","last_change":"2024-04-25T11:31:07.312296+0000","last_active":"2024-04-25T11:31:07.318895+0000","last_peered":"2024-04-25T11:31:07.318895+0000","last_clean":"2024-04-25T11:31:07.318895+0000","last_became_active":"2024-04-25T11:31:07.311984+0000","last_became_peered":"2024-04-25T11:31:07.311984+0000","last_unstale":"2024-04-25T11:31:07.318895+0000","last_undegraded":"2024-04-25T11:31:07.318895+0000","last_fullsized":"2024-04-25T11:31:07.318895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:45.076928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321765+0000","last_change":"2024-04-25T11:31:07.316101+0000","last_active":"2024-04-25T11:31:07.321765+0000","last_peered":"2024-04-25T11:31:07.321765+0000","last_clean":"2024-04-25T11:31:07.321765+0000","last_became_active":"2024-04-25T11:31:07.315666+0000","last_became_peered":"2024-04-25T11:31:07.315666+0000","last_unstale":"2024-04-25T11:31:07.321765+0000","last_undegraded":"2024-04-25T11:31:07.321765+0000","last_fullsized":"2024-04-25T11:31:07.321765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.654600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318447+0000","last_change":"2024-04-25T11:31:07.312406+0000","last_active":"2024-04-25T11:31:07.318447+0000","last_peered":"2024-04-25T11:31:07.318447+0000","last_clean":"2024-04-25T11:31:07.318447+0000","last_became_active":"2024-04-25T11:31:07.311728+0000","last_became_peered":"2024-04-25T11:31:07.311728+0000","last_unstale":"2024-04-25T11:31:07.318447+0000","last_undegraded":"2024-04-25T11:31:07.318447+0000","last_fullsized":"2024-04-25T11:31:07.318447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.501090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.300124+0000","last_change":"2024-04-25T11:31:07.301230+0000","last_active":"2024-04-25T11:31:08.300124+0000","last_peered":"2024-04-25T11:31:08.300124+0000","last_clean":"2024-04-25T11:31:08.300124+0000","last_became_active":"2024-04-25T11:31:07.300774+0000","last_became_peered":"2024-04-25T11:31:07.300774+0000","last_unstale":"2024-04-25T11:31:08.300124+0000","last_undegraded":"2024-04-25T11:31:08.300124+0000","last_fullsized":"2024-04-25T11:31:08.300124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:44.159435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.300581+0000","last_change":"2024-04-25T11:31:07.305330+0000","last_active":"2024-04-25T11:31:08.300581+0000","last_peered":"2024-04-25T11:31:08.300581+0000","last_clean":"2024-04-25T11:31:08.300581+0000","last_became_active":"2024-04-25T11:31:07.301454+0000","last_became_peered":"2024-04-25T11:31:07.301454+0000","last_unstale":"2024-04-25T11:31:08.300581+0000","last_undegraded":"2024-04-25T11:31:08.300581+0000","last_fullsized":"2024-04-25T11:31:08.300581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:13.706152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.499679+0000","last_change":"2024-04-25T11:31:07.293372+0000","last_active":"2024-04-25T11:31:09.499679+0000","last_peered":"2024-04-25T11:31:09.499679+0000","last_clean":"2024-04-25T11:31:09.499679+0000","last_became_active":"2024-04-25T11:31:07.293003+0000","last_became_peered":"2024-04-25T11:31:07.293003+0000","last_unstale":"2024-04-25T11:31:09.499679+0000","last_undegraded":"2024-04-25T11:31:09.499679+0000","last_fullsized":"2024-04-25T11:31:09.499679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:47.506555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318232+0000","last_change":"2024-04-25T11:31:07.309813+0000","last_active":"2024-04-25T11:31:07.318232+0000","last_peered":"2024-04-25T11:31:07.318232+0000","last_clean":"2024-04-25T11:31:07.318232+0000","last_became_active":"2024-04-25T11:31:07.309163+0000","last_became_peered":"2024-04-25T11:31:07.309163+0000","last_unstale":"2024-04-25T11:31:07.318232+0000","last_undegraded":"2024-04-25T11:31:07.318232+0000","last_fullsized":"2024-04-25T11:31:07.318232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:28.203850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.306616+0000","last_change":"2024-04-25T11:31:07.302815+0000","last_active":"2024-04-25T11:31:07.306616+0000","last_peered":"2024-04-25T11:31:07.306616+0000","last_clean":"2024-04-25T11:31:07.306616+0000","last_became_active":"2024-04-25T11:31:07.302468+0000","last_became_peered":"2024-04-25T11:31:07.302468+0000","last_unstale":"2024-04-25T11:31:07.306616+0000","last_undegraded":"2024-04-25T11:31:07.306616+0000","last_fullsized":"2024-04-25T11:31:07.306616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:34.731773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.303965+0000","last_change":"2024-04-25T11:31:07.300690+0000","last_active":"2024-04-25T11:31:08.303965+0000","last_peered":"2024-04-25T11:31:08.303965+0000","last_clean":"2024-04-25T11:31:08.303965+0000","last_became_active":"2024-04-25T11:31:07.300376+0000","last_became_peered":"2024-04-25T11:31:07.300376+0000","last_unstale":"2024-04-25T11:31:08.303965+0000","last_undegraded":"2024-04-25T11:31:08.303965+0000","last_fullsized":"2024-04-25T11:31:08.303965+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:12.203670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321733+0000","last_change":"2024-04-25T11:31:06.264715+0000","last_active":"2024-04-25T11:31:07.321733+0000","last_peered":"2024-04-25T11:31:07.321733+0000","last_clean":"2024-04-25T11:31:07.321733+0000","last_became_active":"2024-04-25T11:31:06.263282+0000","last_became_peered":"2024-04-25T11:31:06.263282+0000","last_unstale":"2024-04-25T11:31:07.321733+0000","last_undegraded":"2024-04-25T11:31:07.321733+0000","last_fullsized":"2024-04-25T11:31:07.321733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:18.401152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.304399+0000","last_change":"2024-04-25T11:31:07.305693+0000","last_active":"2024-04-25T11:31:08.304399+0000","last_peered":"2024-04-25T11:31:08.304399+0000","last_clean":"2024-04-25T11:31:08.304399+0000","last_became_active":"2024-04-25T11:31:07.305177+0000","last_became_peered":"2024-04-25T11:31:07.305177+0000","last_unstale":"2024-04-25T11:31:08.304399+0000","last_undegraded":"2024-04-25T11:31:08.304399+0000","last_fullsized":"2024-04-25T11:31:08.304399+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.933478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.322443+0000","last_change":"2024-04-25T11:31:06.265130+0000","last_active":"2024-04-25T11:31:07.322443+0000","last_peered":"2024-04-25T11:31:07.322443+0000","last_clean":"2024-04-25T11:31:07.322443+0000","last_became_active":"2024-04-25T11:31:06.263941+0000","last_became_peered":"2024-04-25T11:31:06.263941+0000","last_unstale":"2024-04-25T11:31:07.322443+0000","last_undegraded":"2024-04-25T11:31:07.322443+0000","last_fullsized":"2024-04-25T11:31:07.322443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.501947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.303201+0000","last_change":"2024-04-25T11:31:07.300782+0000","last_active":"2024-04-25T11:31:08.303201+0000","last_peered":"2024-04-25T11:31:08.303201+0000","last_clean":"2024-04-25T11:31:08.303201+0000","last_became_active":"2024-04-25T11:31:07.299778+0000","last_became_peered":"2024-04-25T11:31:07.299778+0000","last_unstale":"2024-04-25T11:31:08.303201+0000","last_undegraded":"2024-04-25T11:31:08.303201+0000","last_fullsized":"2024-04-25T11:31:08.303201+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:52.760747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.303659+0000","last_change":"2024-04-25T11:31:06.269010+0000","last_active":"2024-04-25T11:31:08.303659+0000","last_peered":"2024-04-25T11:31:08.303659+0000","last_clean":"2024-04-25T11:31:08.303659+0000","last_became_active":"2024-04-25T11:31:06.268645+0000","last_became_peered":"2024-04-25T11:31:06.268645+0000","last_unstale":"2024-04-25T11:31:08.303659+0000","last_undegraded":"2024-04-25T11:31:08.303659+0000","last_fullsized":"2024-04-25T11:31:08.303659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:24.219149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.303485+0000","last_change":"2024-04-25T11:31:07.290517+0000","last_active":"2024-04-25T11:31:08.303485+0000","last_peered":"2024-04-25T11:31:08.303485+0000","last_clean":"2024-04-25T11:31:08.303485+0000","last_became_active":"2024-04-25T11:31:07.290177+0000","last_became_peered":"2024-04-25T11:31:07.290177+0000","last_unstale":"2024-04-25T11:31:08.303485+0000","last_undegraded":"2024-04-25T11:31:08.303485+0000","last_fullsized":"2024-04-25T11:31:08.303485+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:42.042946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.499907+0000","last_change":"2024-04-25T11:31:06.248267+0000","last_active":"2024-04-25T11:31:09.499907+0000","last_peered":"2024-04-25T11:31:09.499907+0000","last_clean":"2024-04-25T11:31:09.499907+0000","last_became_active":"2024-04-25T11:31:06.247895+0000","last_became_peered":"2024-04-25T11:31:06.247895+0000","last_unstale":"2024-04-25T11:31:09.499907+0000","last_undegraded":"2024-04-25T11:31:09.499907+0000","last_fullsized":"2024-04-25T11:31:09.499907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:36.075589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.300454+0000","last_change":"2024-04-25T11:31:07.278698+0000","last_active":"2024-04-25T11:31:08.300454+0000","last_peered":"2024-04-25T11:31:08.300454+0000","last_clean":"2024-04-25T11:31:08.300454+0000","last_became_active":"2024-04-25T11:31:07.278015+0000","last_became_peered":"2024-04-25T11:31:07.278015+0000","last_unstale":"2024-04-25T11:31:08.300454+0000","last_undegraded":"2024-04-25T11:31:08.300454+0000","last_fullsized":"2024-04-25T11:31:08.300454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:30.651018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309497+0000","last_change":"2024-04-25T11:31:06.265746+0000","last_active":"2024-04-25T11:31:08.309497+0000","last_peered":"2024-04-25T11:31:08.309497+0000","last_clean":"2024-04-25T11:31:08.309497+0000","last_became_active":"2024-04-25T11:31:06.265259+0000","last_became_peered":"2024-04-25T11:31:06.265259+0000","last_unstale":"2024-04-25T11:31:08.309497+0000","last_undegraded":"2024-04-25T11:31:08.309497+0000","last_fullsized":"2024-04-25T11:31:08.309497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:13.412651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.311934+0000","last_change":"2024-04-25T11:31:07.303002+0000","last_active":"2024-04-25T11:31:07.311934+0000","last_peered":"2024-04-25T11:31:07.311934+0000","last_clean":"2024-04-25T11:31:07.311934+0000","last_became_active":"2024-04-25T11:31:07.302386+0000","last_became_peered":"2024-04-25T11:31:07.302386+0000","last_unstale":"2024-04-25T11:31:07.311934+0000","last_undegraded":"2024-04-25T11:31:07.311934+0000","last_fullsized":"2024-04-25T11:31:07.311934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:49.681967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.302286+0000","last_change":"2024-04-25T11:31:06.268400+0000","last_active":"2024-04-25T11:31:08.302286+0000","last_peered":"2024-04-25T11:31:08.302286+0000","last_clean":"2024-04-25T11:31:08.302286+0000","last_became_active":"2024-04-25T11:31:06.268051+0000","last_became_peered":"2024-04-25T11:31:06.268051+0000","last_unstale":"2024-04-25T11:31:08.302286+0000","last_undegraded":"2024-04-25T11:31:08.302286+0000","last_fullsized":"2024-04-25T11:31:08.302286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.649935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309264+0000","last_change":"2024-04-25T11:31:07.294940+0000","last_active":"2024-04-25T11:31:08.309264+0000","last_peered":"2024-04-25T11:31:08.309264+0000","last_clean":"2024-04-25T11:31:08.309264+0000","last_became_active":"2024-04-25T11:31:07.294387+0000","last_became_peered":"2024-04-25T11:31:07.294387+0000","last_unstale":"2024-04-25T11:31:08.309264+0000","last_undegraded":"2024-04-25T11:31:08.309264+0000","last_fullsized":"2024-04-25T11:31:08.309264+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:42.033407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.295531+0000","last_change":"2024-04-25T11:31:06.256010+0000","last_active":"2024-04-25T11:31:08.295531+0000","last_peered":"2024-04-25T11:31:08.295531+0000","last_clean":"2024-04-25T11:31:08.295531+0000","last_became_active":"2024-04-25T11:31:06.255555+0000","last_became_peered":"2024-04-25T11:31:06.255555+0000","last_unstale":"2024-04-25T11:31:08.295531+0000","last_undegraded":"2024-04-25T11:31:08.295531+0000","last_fullsized":"2024-04-25T11:31:08.295531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:22.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.301754+0000","last_change":"2024-04-25T11:31:07.295236+0000","last_active":"2024-04-25T11:31:08.301754+0000","last_peered":"2024-04-25T11:31:08.301754+0000","last_clean":"2024-04-25T11:31:08.301754+0000","last_became_active":"2024-04-25T11:31:07.294855+0000","last_became_peered":"2024-04-25T11:31:07.294855+0000","last_unstale":"2024-04-25T11:31:08.301754+0000","last_undegraded":"2024-04-25T11:31:08.301754+0000","last_fullsized":"2024-04-25T11:31:08.301754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:11.654600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.285755+0000","last_change":"2024-04-25T11:31:06.261137+0000","last_active":"2024-04-25T11:31:07.285755+0000","last_peered":"2024-04-25T11:31:07.285755+0000","last_clean":"2024-04-25T11:31:07.285755+0000","last_became_active":"2024-04-25T11:31:06.260698+0000","last_became_peered":"2024-04-25T11:31:06.260698+0000","last_unstale":"2024-04-25T11:31:07.285755+0000","last_undegraded":"2024-04-25T11:31:07.285755+0000","last_fullsized":"2024-04-25T11:31:07.285755+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:51.188601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.314149+0000","last_change":"2024-04-25T11:31:07.299480+0000","last_active":"2024-04-25T11:31:07.314149+0000","last_peered":"2024-04-25T11:31:07.314149+0000","last_clean":"2024-04-25T11:31:07.314149+0000","last_became_active":"2024-04-25T11:31:07.299171+0000","last_became_peered":"2024-04-25T11:31:07.299171+0000","last_unstale":"2024-04-25T11:31:07.314149+0000","last_undegraded":"2024-04-25T11:31:07.314149+0000","last_fullsized":"2024-04-25T11:31:07.314149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:14.429793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.296469+0000","last_change":"2024-04-25T11:31:06.262349+0000","last_active":"2024-04-25T11:31:08.296469+0000","last_peered":"2024-04-25T11:31:08.296469+0000","last_clean":"2024-04-25T11:31:08.296469+0000","last_became_active":"2024-04-25T11:31:06.261724+0000","last_became_peered":"2024-04-25T11:31:06.261724+0000","last_unstale":"2024-04-25T11:31:08.296469+0000","last_undegraded":"2024-04-25T11:31:08.296469+0000","last_fullsized":"2024-04-25T11:31:08.296469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:12.839356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.310143+0000","last_change":"2024-04-25T11:31:07.298405+0000","last_active":"2024-04-25T11:31:08.310143+0000","last_peered":"2024-04-25T11:31:08.310143+0000","last_clean":"2024-04-25T11:31:08.310143+0000","last_became_active":"2024-04-25T11:31:07.298094+0000","last_became_peered":"2024-04-25T11:31:07.298094+0000","last_unstale":"2024-04-25T11:31:08.310143+0000","last_undegraded":"2024-04-25T11:31:08.310143+0000","last_fullsized":"2024-04-25T11:31:08.310143+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:41.925553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.301917+0000","last_change":"2024-04-25T11:31:06.270417+0000","last_active":"2024-04-25T11:31:08.301917+0000","last_peered":"2024-04-25T11:31:08.301917+0000","last_clean":"2024-04-25T11:31:08.301917+0000","last_became_active":"2024-04-25T11:31:06.268102+0000","last_became_peered":"2024-04-25T11:31:06.268102+0000","last_unstale":"2024-04-25T11:31:08.301917+0000","last_undegraded":"2024-04-25T11:31:08.301917+0000","last_fullsized":"2024-04-25T11:31:08.301917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:58.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.302108+0000","last_change":"2024-04-25T11:31:07.301198+0000","last_active":"2024-04-25T11:31:08.302108+0000","last_peered":"2024-04-25T11:31:08.302108+0000","last_clean":"2024-04-25T11:31:08.302108+0000","last_became_active":"2024-04-25T11:31:07.300303+0000","last_became_peered":"2024-04-25T11:31:07.300303+0000","last_unstale":"2024-04-25T11:31:08.302108+0000","last_undegraded":"2024-04-25T11:31:08.302108+0000","last_fullsized":"2024-04-25T11:31:08.302108+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:51.879017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"62'3","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.500776+0000","last_change":"2024-04-25T11:31:06.248828+0000","last_active":"2024-04-25T11:31:09.500776+0000","last_peered":"2024-04-25T11:31:09.500776+0000","last_clean":"2024-04-25T11:31:09.500776+0000","last_became_active":"2024-04-25T11:31:06.247850+0000","last_became_peered":"2024-04-25T11:31:06.247850+0000","last_unstale":"2024-04-25T11:31:09.500776+0000","last_undegraded":"2024-04-25T11:31:09.500776+0000","last_fullsized":"2024-04-25T11:31:09.500776+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:58.873512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.500205+0000","last_change":"2024-04-25T11:31:07.279903+0000","last_active":"2024-04-25T11:31:09.500205+0000","last_peered":"2024-04-25T11:31:09.500205+0000","last_clean":"2024-04-25T11:31:09.500205+0000","last_became_active":"2024-04-25T11:31:07.278929+0000","last_became_peered":"2024-04-25T11:31:07.278929+0000","last_unstale":"2024-04-25T11:31:09.500205+0000","last_undegraded":"2024-04-25T11:31:09.500205+0000","last_fullsized":"2024-04-25T11:31:09.500205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:00.175434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.326782+0000","last_change":"2024-04-25T11:31:06.259780+0000","last_active":"2024-04-25T11:31:07.326782+0000","last_peered":"2024-04-25T11:31:07.326782+0000","last_clean":"2024-04-25T11:31:07.326782+0000","last_became_active":"2024-04-25T11:31:06.259210+0000","last_became_peered":"2024-04-25T11:31:06.259210+0000","last_unstale":"2024-04-25T11:31:07.326782+0000","last_undegraded":"2024-04-25T11:31:07.326782+0000","last_fullsized":"2024-04-25T11:31:07.326782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:58.335618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.295719+0000","last_change":"2024-04-25T11:31:07.286062+0000","last_active":"2024-04-25T11:31:08.295719+0000","last_peered":"2024-04-25T11:31:08.295719+0000","last_clean":"2024-04-25T11:31:08.295719+0000","last_became_active":"2024-04-25T11:31:07.285670+0000","last_became_peered":"2024-04-25T11:31:07.285670+0000","last_unstale":"2024-04-25T11:31:08.295719+0000","last_undegraded":"2024-04-25T11:31:08.295719+0000","last_fullsized":"2024-04-25T11:31:08.295719+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:21.480079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.300918+0000","last_change":"2024-04-25T11:31:06.270339+0000","last_active":"2024-04-25T11:31:08.300918+0000","last_peered":"2024-04-25T11:31:08.300918+0000","last_clean":"2024-04-25T11:31:08.300918+0000","last_became_active":"2024-04-25T11:31:06.266784+0000","last_became_peered":"2024-04-25T11:31:06.266784+0000","last_unstale":"2024-04-25T11:31:08.300918+0000","last_undegraded":"2024-04-25T11:31:08.300918+0000","last_fullsized":"2024-04-25T11:31:08.300918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:48.678840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309412+0000","last_change":"2024-04-25T11:31:07.300848+0000","last_active":"2024-04-25T11:31:08.309412+0000","last_peered":"2024-04-25T11:31:08.309412+0000","last_clean":"2024-04-25T11:31:08.309412+0000","last_became_active":"2024-04-25T11:31:07.300080+0000","last_became_peered":"2024-04-25T11:31:07.300080+0000","last_unstale":"2024-04-25T11:31:08.309412+0000","last_undegraded":"2024-04-25T11:31:08.309412+0000","last_fullsized":"2024-04-25T11:31:08.309412+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:00.017832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.499929+0000","last_change":"2024-04-25T11:31:06.254543+0000","last_active":"2024-04-25T11:31:09.499929+0000","last_peered":"2024-04-25T11:31:09.499929+0000","last_clean":"2024-04-25T11:31:09.499929+0000","last_became_active":"2024-04-25T11:31:06.254238+0000","last_became_peered":"2024-04-25T11:31:06.254238+0000","last_unstale":"2024-04-25T11:31:09.499929+0000","last_undegraded":"2024-04-25T11:31:09.499929+0000","last_fullsized":"2024-04-25T11:31:09.499929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:00.767019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.295355+0000","last_change":"2024-04-25T11:31:07.295447+0000","last_active":"2024-04-25T11:31:08.295355+0000","last_peered":"2024-04-25T11:31:08.295355+0000","last_clean":"2024-04-25T11:31:08.295355+0000","last_became_active":"2024-04-25T11:31:07.295127+0000","last_became_peered":"2024-04-25T11:31:07.295127+0000","last_unstale":"2024-04-25T11:31:08.295355+0000","last_undegraded":"2024-04-25T11:31:08.295355+0000","last_fullsized":"2024-04-25T11:31:08.295355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:28.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.499122+0000","last_change":"2024-04-25T11:31:06.250388+0000","last_active":"2024-04-25T11:31:09.499122+0000","last_peered":"2024-04-25T11:31:09.499122+0000","last_clean":"2024-04-25T11:31:09.499122+0000","last_became_active":"2024-04-25T11:31:06.250060+0000","last_became_peered":"2024-04-25T11:31:06.250060+0000","last_unstale":"2024-04-25T11:31:09.499122+0000","last_undegraded":"2024-04-25T11:31:09.499122+0000","last_fullsized":"2024-04-25T11:31:09.499122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:14.538099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.499007+0000","last_change":"2024-04-25T11:31:07.300418+0000","last_active":"2024-04-25T11:31:09.499007+0000","last_peered":"2024-04-25T11:31:09.499007+0000","last_clean":"2024-04-25T11:31:09.499007+0000","last_became_active":"2024-04-25T11:31:07.300081+0000","last_became_peered":"2024-04-25T11:31:07.300081+0000","last_unstale":"2024-04-25T11:31:09.499007+0000","last_undegraded":"2024-04-25T11:31:09.499007+0000","last_fullsized":"2024-04-25T11:31:09.499007+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:25.473038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"62'8","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.499114+0000","last_change":"2024-04-25T11:31:06.254133+0000","last_active":"2024-04-25T11:31:09.499114+0000","last_peered":"2024-04-25T11:31:09.499114+0000","last_clean":"2024-04-25T11:31:09.499114+0000","last_became_active":"2024-04-25T11:31:06.253811+0000","last_became_peered":"2024-04-25T11:31:06.253811+0000","last_unstale":"2024-04-25T11:31:09.499114+0000","last_undegraded":"2024-04-25T11:31:09.499114+0000","last_fullsized":"2024-04-25T11:31:09.499114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:14.279767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.310181+0000","last_change":"2024-04-25T11:31:07.292883+0000","last_active":"2024-04-25T11:31:08.310181+0000","last_peered":"2024-04-25T11:31:08.310181+0000","last_clean":"2024-04-25T11:31:08.310181+0000","last_became_active":"2024-04-25T11:31:07.287159+0000","last_became_peered":"2024-04-25T11:31:07.287159+0000","last_unstale":"2024-04-25T11:31:08.310181+0000","last_undegraded":"2024-04-25T11:31:08.310181+0000","last_fullsized":"2024-04-25T11:31:08.310181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:37.947332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321476+0000","last_change":"2024-04-25T11:31:06.274421+0000","last_active":"2024-04-25T11:31:07.321476+0000","last_peered":"2024-04-25T11:31:07.321476+0000","last_clean":"2024-04-25T11:31:07.321476+0000","last_became_active":"2024-04-25T11:31:06.274101+0000","last_became_peered":"2024-04-25T11:31:06.274101+0000","last_unstale":"2024-04-25T11:31:07.321476+0000","last_undegraded":"2024-04-25T11:31:07.321476+0000","last_fullsized":"2024-04-25T11:31:07.321476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:12.839356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.310009+0000","last_change":"2024-04-25T11:31:07.288525+0000","last_active":"2024-04-25T11:31:08.310009+0000","last_peered":"2024-04-25T11:31:08.310009+0000","last_clean":"2024-04-25T11:31:08.310009+0000","last_became_active":"2024-04-25T11:31:07.288093+0000","last_became_peered":"2024-04-25T11:31:07.288093+0000","last_unstale":"2024-04-25T11:31:08.310009+0000","last_undegraded":"2024-04-25T11:31:08.310009+0000","last_fullsized":"2024-04-25T11:31:08.310009+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.302229+0000","last_change":"2024-04-25T11:31:06.268805+0000","last_active":"2024-04-25T11:31:08.302229+0000","last_peered":"2024-04-25T11:31:08.302229+0000","last_clean":"2024-04-25T11:31:08.302229+0000","last_became_active":"2024-04-25T11:31:06.268469+0000","last_became_peered":"2024-04-25T11:31:06.268469+0000","last_unstale":"2024-04-25T11:31:08.302229+0000","last_undegraded":"2024-04-25T11:31:08.302229+0000","last_fullsized":"2024-04-25T11:31:08.302229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:56.303135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.319040+0000","last_change":"2024-04-25T11:31:07.317102+0000","last_active":"2024-04-25T11:31:07.319040+0000","last_peered":"2024-04-25T11:31:07.319040+0000","last_clean":"2024-04-25T11:31:07.319040+0000","last_became_active":"2024-04-25T11:31:07.316799+0000","last_became_peered":"2024-04-25T11:31:07.316799+0000","last_unstale":"2024-04-25T11:31:07.319040+0000","last_undegraded":"2024-04-25T11:31:07.319040+0000","last_fullsized":"2024-04-25T11:31:07.319040+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:28.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.310004+0000","last_change":"2024-04-25T11:31:06.261186+0000","last_active":"2024-04-25T11:31:08.310004+0000","last_peered":"2024-04-25T11:31:08.310004+0000","last_clean":"2024-04-25T11:31:08.310004+0000","last_became_active":"2024-04-25T11:31:06.260862+0000","last_became_peered":"2024-04-25T11:31:06.260862+0000","last_unstale":"2024-04-25T11:31:08.310004+0000","last_undegraded":"2024-04-25T11:31:08.310004+0000","last_fullsized":"2024-04-25T11:31:08.310004+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:18.971373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:09.500685+0000","last_change":"2024-04-25T11:31:07.306288+0000","last_active":"2024-04-25T11:31:09.500685+0000","last_peered":"2024-04-25T11:31:09.500685+0000","last_clean":"2024-04-25T11:31:09.500685+0000","last_became_active":"2024-04-25T11:31:07.305940+0000","last_became_peered":"2024-04-25T11:31:07.305940+0000","last_unstale":"2024-04-25T11:31:09.500685+0000","last_undegraded":"2024-04-25T11:31:09.500685+0000","last_fullsized":"2024-04-25T11:31:09.500685+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:32.933169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.302428+0000","last_change":"2024-04-25T11:31:06.269600+0000","last_active":"2024-04-25T11:31:08.302428+0000","last_peered":"2024-04-25T11:31:08.302428+0000","last_clean":"2024-04-25T11:31:08.302428+0000","last_became_active":"2024-04-25T11:31:06.268545+0000","last_became_peered":"2024-04-25T11:31:06.268545+0000","last_unstale":"2024-04-25T11:31:08.302428+0000","last_undegraded":"2024-04-25T11:31:08.302428+0000","last_fullsized":"2024-04-25T11:31:08.302428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.324416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.295807+0000","last_change":"2024-04-25T11:31:07.292122+0000","last_active":"2024-04-25T11:31:08.295807+0000","last_peered":"2024-04-25T11:31:08.295807+0000","last_clean":"2024-04-25T11:31:08.295807+0000","last_became_active":"2024-04-25T11:31:07.290352+0000","last_became_peered":"2024-04-25T11:31:07.290352+0000","last_unstale":"2024-04-25T11:31:08.295807+0000","last_undegraded":"2024-04-25T11:31:08.295807+0000","last_fullsized":"2024-04-25T11:31:08.295807+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:25.236291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.317656+0000","last_change":"2024-04-25T11:31:06.262444+0000","last_active":"2024-04-25T11:31:07.317656+0000","last_peered":"2024-04-25T11:31:07.317656+0000","last_clean":"2024-04-25T11:31:07.317656+0000","last_became_active":"2024-04-25T11:31:06.260269+0000","last_became_peered":"2024-04-25T11:31:06.260269+0000","last_unstale":"2024-04-25T11:31:07.317656+0000","last_undegraded":"2024-04-25T11:31:07.317656+0000","last_fullsized":"2024-04-25T11:31:07.317656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:13.575172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.300946+0000","last_change":"2024-04-25T11:31:07.294111+0000","last_active":"2024-04-25T11:31:08.300946+0000","last_peered":"2024-04-25T11:31:08.300946+0000","last_clean":"2024-04-25T11:31:08.300946+0000","last_became_active":"2024-04-25T11:31:07.291384+0000","last_became_peered":"2024-04-25T11:31:07.291384+0000","last_unstale":"2024-04-25T11:31:08.300946+0000","last_undegraded":"2024-04-25T11:31:08.300946+0000","last_fullsized":"2024-04-25T11:31:08.300946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:37.490222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.300605+0000","last_change":"2024-04-25T11:31:06.262430+0000","last_active":"2024-04-25T11:31:08.300605+0000","last_peered":"2024-04-25T11:31:08.300605+0000","last_clean":"2024-04-25T11:31:08.300605+0000","last_became_active":"2024-04-25T11:31:06.262088+0000","last_became_peered":"2024-04-25T11:31:06.262088+0000","last_unstale":"2024-04-25T11:31:08.300605+0000","last_undegraded":"2024-04-25T11:31:08.300605+0000","last_fullsized":"2024-04-25T11:31:08.300605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:23.416178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.299703+0000","last_change":"2024-04-25T11:31:07.298188+0000","last_active":"2024-04-25T11:31:08.299703+0000","last_peered":"2024-04-25T11:31:08.299703+0000","last_clean":"2024-04-25T11:31:08.299703+0000","last_became_active":"2024-04-25T11:31:07.297220+0000","last_became_peered":"2024-04-25T11:31:07.297220+0000","last_unstale":"2024-04-25T11:31:08.299703+0000","last_undegraded":"2024-04-25T11:31:08.299703+0000","last_fullsized":"2024-04-25T11:31:08.299703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:56.592783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.295500+0000","last_change":"2024-04-25T11:31:06.268722+0000","last_active":"2024-04-25T11:31:08.295500+0000","last_peered":"2024-04-25T11:31:08.295500+0000","last_clean":"2024-04-25T11:31:08.295500+0000","last_became_active":"2024-04-25T11:31:06.268257+0000","last_became_peered":"2024-04-25T11:31:06.268257+0000","last_unstale":"2024-04-25T11:31:08.295500+0000","last_undegraded":"2024-04-25T11:31:08.295500+0000","last_fullsized":"2024-04-25T11:31:08.295500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:02.811671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.310157+0000","last_change":"2024-04-25T11:31:07.302694+0000","last_active":"2024-04-25T11:31:08.310157+0000","last_peered":"2024-04-25T11:31:08.310157+0000","last_clean":"2024-04-25T11:31:08.310157+0000","last_became_active":"2024-04-25T11:31:07.302350+0000","last_became_peered":"2024-04-25T11:31:07.302350+0000","last_unstale":"2024-04-25T11:31:08.310157+0000","last_undegraded":"2024-04-25T11:31:08.310157+0000","last_fullsized":"2024-04-25T11:31:08.310157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:41.575356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.321570+0000","last_change":"2024-04-25T11:31:06.271800+0000","last_active":"2024-04-25T11:31:07.321570+0000","last_peered":"2024-04-25T11:31:07.321570+0000","last_clean":"2024-04-25T11:31:07.321570+0000","last_became_active":"2024-04-25T11:31:06.271483+0000","last_became_peered":"2024-04-25T11:31:06.271483+0000","last_unstale":"2024-04-25T11:31:07.321570+0000","last_undegraded":"2024-04-25T11:31:07.321570+0000","last_fullsized":"2024-04-25T11:31:07.321570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:11.930142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.310225+0000","last_change":"2024-04-25T11:31:07.289917+0000","last_active":"2024-04-25T11:31:08.310225+0000","last_peered":"2024-04-25T11:31:08.310225+0000","last_clean":"2024-04-25T11:31:08.310225+0000","last_became_active":"2024-04-25T11:31:07.289583+0000","last_became_peered":"2024-04-25T11:31:07.289583+0000","last_unstale":"2024-04-25T11:31:08.310225+0000","last_undegraded":"2024-04-25T11:31:08.310225+0000","last_fullsized":"2024-04-25T11:31:08.310225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:28.024997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"63'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.303843+0000","last_change":"2024-04-25T11:31:06.272906+0000","last_active":"2024-04-25T11:31:08.303843+0000","last_peered":"2024-04-25T11:31:08.303843+0000","last_clean":"2024-04-25T11:31:08.303843+0000","last_became_active":"2024-04-25T11:31:06.272242+0000","last_became_peered":"2024-04-25T11:31:06.272242+0000","last_unstale":"2024-04-25T11:31:08.303843+0000","last_undegraded":"2024-04-25T11:31:08.303843+0000","last_fullsized":"2024-04-25T11:31:08.303843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:50.474918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.318214+0000","last_change":"2024-04-25T11:31:07.310583+0000","last_active":"2024-04-25T11:31:07.318214+0000","last_peered":"2024-04-25T11:31:07.318214+0000","last_clean":"2024-04-25T11:31:07.318214+0000","last_became_active":"2024-04-25T11:31:07.309600+0000","last_became_peered":"2024-04-25T11:31:07.309600+0000","last_unstale":"2024-04-25T11:31:07.318214+0000","last_undegraded":"2024-04-25T11:31:07.318214+0000","last_fullsized":"2024-04-25T11:31:07.318214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:40.475792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.296255+0000","last_change":"2024-04-25T11:31:06.254976+0000","last_active":"2024-04-25T11:31:08.296255+0000","last_peered":"2024-04-25T11:31:08.296255+0000","last_clean":"2024-04-25T11:31:08.296255+0000","last_became_active":"2024-04-25T11:31:06.254500+0000","last_became_peered":"2024-04-25T11:31:06.254500+0000","last_unstale":"2024-04-25T11:31:08.296255+0000","last_undegraded":"2024-04-25T11:31:08.296255+0000","last_fullsized":"2024-04-25T11:31:08.296255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:20.513794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.303384+0000","last_change":"2024-04-25T11:31:07.300714+0000","last_active":"2024-04-25T11:31:08.303384+0000","last_peered":"2024-04-25T11:31:08.303384+0000","last_clean":"2024-04-25T11:31:08.303384+0000","last_became_active":"2024-04-25T11:31:07.300249+0000","last_became_peered":"2024-04-25T11:31:07.300249+0000","last_unstale":"2024-04-25T11:31:08.303384+0000","last_undegraded":"2024-04-25T11:31:08.303384+0000","last_fullsized":"2024-04-25T11:31:08.303384+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:27.194875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309598+0000","last_change":"2024-04-25T11:31:06.241045+0000","last_active":"2024-04-25T11:31:08.309598+0000","last_peered":"2024-04-25T11:31:08.309598+0000","last_clean":"2024-04-25T11:31:08.309598+0000","last_became_active":"2024-04-25T11:31:06.238913+0000","last_became_peered":"2024-04-25T11:31:06.238913+0000","last_unstale":"2024-04-25T11:31:08.309598+0000","last_undegraded":"2024-04-25T11:31:08.309598+0000","last_fullsized":"2024-04-25T11:31:08.309598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:49.794058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.302560+0000","last_change":"2024-04-25T11:31:07.288902+0000","last_active":"2024-04-25T11:31:08.302560+0000","last_peered":"2024-04-25T11:31:08.302560+0000","last_clean":"2024-04-25T11:31:08.302560+0000","last_became_active":"2024-04-25T11:31:07.288281+0000","last_became_peered":"2024-04-25T11:31:07.288281+0000","last_unstale":"2024-04-25T11:31:08.302560+0000","last_undegraded":"2024-04-25T11:31:08.302560+0000","last_fullsized":"2024-04-25T11:31:08.302560+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.592314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.295942+0000","last_change":"2024-04-25T11:31:06.274502+0000","last_active":"2024-04-25T11:31:07.295942+0000","last_peered":"2024-04-25T11:31:07.295942+0000","last_clean":"2024-04-25T11:31:07.295942+0000","last_became_active":"2024-04-25T11:31:06.274181+0000","last_became_peered":"2024-04-25T11:31:06.274181+0000","last_unstale":"2024-04-25T11:31:07.295942+0000","last_undegraded":"2024-04-25T11:31:07.295942+0000","last_fullsized":"2024-04-25T11:31:07.295942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:20.513794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.315089+0000","last_change":"2024-04-25T11:31:07.301414+0000","last_active":"2024-04-25T11:31:07.315089+0000","last_peered":"2024-04-25T11:31:07.315089+0000","last_clean":"2024-04-25T11:31:07.315089+0000","last_became_active":"2024-04-25T11:31:07.301012+0000","last_became_peered":"2024-04-25T11:31:07.301012+0000","last_unstale":"2024-04-25T11:31:07.315089+0000","last_undegraded":"2024-04-25T11:31:07.315089+0000","last_fullsized":"2024-04-25T11:31:07.315089+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:21.068983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.309203+0000","last_change":"2024-04-25T11:31:06.258161+0000","last_active":"2024-04-25T11:31:08.309203+0000","last_peered":"2024-04-25T11:31:08.309203+0000","last_clean":"2024-04-25T11:31:08.309203+0000","last_became_active":"2024-04-25T11:31:06.257780+0000","last_became_peered":"2024-04-25T11:31:06.257780+0000","last_unstale":"2024-04-25T11:31:08.309203+0000","last_undegraded":"2024-04-25T11:31:08.309203+0000","last_fullsized":"2024-04-25T11:31:08.309203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:20.051841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.299705+0000","last_change":"2024-04-25T11:31:07.289094+0000","last_active":"2024-04-25T11:31:08.299705+0000","last_peered":"2024-04-25T11:31:08.299705+0000","last_clean":"2024-04-25T11:31:08.299705+0000","last_became_active":"2024-04-25T11:31:07.288667+0000","last_became_peered":"2024-04-25T11:31:07.288667+0000","last_unstale":"2024-04-25T11:31:08.299705+0000","last_undegraded":"2024-04-25T11:31:08.299705+0000","last_fullsized":"2024-04-25T11:31:08.299705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:39.377853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.302016+0000","last_change":"2024-04-25T11:31:06.269909+0000","last_active":"2024-04-25T11:31:08.302016+0000","last_peered":"2024-04-25T11:31:08.302016+0000","last_clean":"2024-04-25T11:31:08.302016+0000","last_became_active":"2024-04-25T11:31:06.269503+0000","last_became_peered":"2024-04-25T11:31:06.269503+0000","last_unstale":"2024-04-25T11:31:08.302016+0000","last_undegraded":"2024-04-25T11:31:08.302016+0000","last_fullsized":"2024-04-25T11:31:08.302016+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:20.462937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.316159+0000","last_change":"2024-04-25T11:31:07.306668+0000","last_active":"2024-04-25T11:31:07.316159+0000","last_peered":"2024-04-25T11:31:07.316159+0000","last_clean":"2024-04-25T11:31:07.316159+0000","last_became_active":"2024-04-25T11:31:07.306128+0000","last_became_peered":"2024-04-25T11:31:07.306128+0000","last_unstale":"2024-04-25T11:31:07.316159+0000","last_undegraded":"2024-04-25T11:31:07.316159+0000","last_fullsized":"2024-04-25T11:31:07.316159+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:16.553301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.299682+0000","last_change":"2024-04-25T11:31:06.246690+0000","last_active":"2024-04-25T11:31:08.299682+0000","last_peered":"2024-04-25T11:31:08.299682+0000","last_clean":"2024-04-25T11:31:08.299682+0000","last_became_active":"2024-04-25T11:31:06.246068+0000","last_became_peered":"2024-04-25T11:31:06.246068+0000","last_unstale":"2024-04-25T11:31:08.299682+0000","last_undegraded":"2024-04-25T11:31:08.299682+0000","last_fullsized":"2024-04-25T11:31:08.299682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:40.825349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.300195+0000","last_change":"2024-04-25T11:31:07.303860+0000","last_active":"2024-04-25T11:31:08.300195+0000","last_peered":"2024-04-25T11:31:08.300195+0000","last_clean":"2024-04-25T11:31:08.300195+0000","last_became_active":"2024-04-25T11:31:07.302756+0000","last_became_peered":"2024-04-25T11:31:07.302756+0000","last_unstale":"2024-04-25T11:31:08.300195+0000","last_undegraded":"2024-04-25T11:31:08.300195+0000","last_fullsized":"2024-04-25T11:31:08.300195+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:06.217456+0000","last_clean_scrub_stamp":"2024-04-25T11:31:06.217456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:34.451549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T11:31:07.286892+0000","last_change":"2024-04-25T11:31:06.259986+0000","last_active":"2024-04-25T11:31:07.286892+0000","last_peered":"2024-04-25T11:31:07.286892+0000","last_clean":"2024-04-25T11:31:07.286892+0000","last_became_active":"2024-04-25T11:31:06.259472+0000","last_became_peered":"2024-04-25T11:31:06.259472+0000","last_unstale":"2024-04-25T11:31:07.286892+0000","last_undegraded":"2024-04-25T11:31:07.286892+0000","last_fullsized":"2024-04-25T11:31:07.286892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:31:05.200314+0000","last_clean_scrub_stamp":"2024-04-25T11:31:05.200314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:20.781128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":188,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T11:31:08.295710+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:31:08.295710+0000","last_peered":"2024-04-25T11:31:08.295710+0000","last_clean":"2024-04-25T11:31:08.295710+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:31:08.295710+0000","last_undegraded":"2024-04-25T11:31:08.295710+0000","last_fullsized":"2024-04-25T11:31:08.295710+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":301595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":753664,"data_stored":314692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":737280,"data_stored":308370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":314760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":316442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":905620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":301595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542194,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1347584,"data_stored":905072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:31:12.621 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:31:12.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:31:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.968+0000 7f9812800640 1 Processor -- start 2024-04-25T11:31:12.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.968+0000 7f9812800640 1 -- start start 2024-04-25T11:31:12.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.970+0000 7f9812800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 0x7f980c1458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.970+0000 7f9812800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f980c15d5c0 con 0x7f980c0ce250 2024-04-25T11:31:12.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.970+0000 7f9812800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f980c15d8a0 con 0x7f980c06d5d0 2024-04-25T11:31:12.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.970+0000 7f9812800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f980c0b4ec0 con 0x7f980c15cfa0 2024-04-25T11:31:12.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.970+0000 7f980b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f980c0ce250 0x7f980c0b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58642/0 (socket says 172.21.15.19:58642) 2024-04-25T11:31:12.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.970+0000 7f980b400640 1 -- 172.21.15.19:0/1656523424 learned_addr learned my addr 172.21.15.19:0/1656523424 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980aa00640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 0x7f980c1458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2893757370 0 0) 0x7f980c15d5c0 con 0x7f980c0ce250 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97ec003520 con 0x7f980c0ce250 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 457316405 0 0) 0x7f97ec003520 con 0x7f980c0ce250 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 msgr2=0x7f980c1458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980a000640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 0x7f980c1458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 >> v1:172.21.15.169:6789/0 conn(0x7f980c06d5d0 legacy=0x7f980c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.971+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c15db80 con 0x7f980c0ce250 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f97fc001670 con 0x7f980c0ce250 2024-04-25T11:31:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 >> v1:172.21.15.19:6789/0 conn(0x7f980c0ce250 legacy=0x7f980c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 0x7f97ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec003f60 0x7f97ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97ec004d40 0x7f97ec005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f980c0b4ec0 con 0x7f97ec004d40 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec00cd10 con 0x7f97ec003f60 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f97ec00cf10 con 0x7f980c15cfa0 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980be00640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97ec004d40 0x7f97ec005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980aa00640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec003f60 0x7f97ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.972+0000 7f980b400640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 0x7f97ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.973+0000 7f980be00640 1 -- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 msgr2=0x7f97ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.973+0000 7f980be00640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c15cfa0 0x7f97ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.973+0000 7f980be00640 1 -- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec003f60 msgr2=0x7f97ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.973+0000 7f980be00640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec003f60 0x7f97ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.973+0000 7f980be00640 1 -- 172.21.15.19:0/1656523424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec003520 con 0x7f97ec004d40 2024-04-25T11:31:12.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.974+0000 7f980be00640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97ec004d40 0x7f97ec005150 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9804001a30 tx=0x7f97ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1bb2723655b04d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.974+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804001a70 con 0x7f97ec004d40 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.974+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9804003b70 con 0x7f97ec004d40 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.974+0000 7f980a000640 1 -- 172.21.15.19:0/1656523424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040050d0 con 0x7f97ec004d40 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.975+0000 7f9812800640 1 -- 172.21.15.19:0/1656523424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97ec004d40 msgr2=0x7f97ec005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.975+0000 7f9812800640 1 --2- 172.21.15.19:0/1656523424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97ec004d40 0x7f97ec005150 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9804001a30 tx=0x7f97ec005830 comp rx=0 tx=0).stop 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.975+0000 7f980be00640 1 -- 172.21.15.19:0/1656523424 reap_dead start 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.975+0000 7f9812800640 1 -- 172.21.15.19:0/1656523424 shutdown_connections 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.975+0000 7f9812800640 1 -- 172.21.15.19:0/1656523424 >> 172.21.15.19:0/1656523424 conn(0x7f980c073090 msgr2=0x7f980c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.975+0000 7f9812800640 1 -- 172.21.15.19:0/1656523424 shutdown_connections 2024-04-25T11:31:12.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.975+0000 7f9812800640 1 -- 172.21.15.19:0/1656523424 wait complete. 2024-04-25T11:31:12.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.977+0000 7f9812800640 1 Processor -- start 2024-04-25T11:31:12.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.978+0000 7f9812800640 1 -- start start 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.979+0000 7f9812800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec004d40 0x7f980c1f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.979+0000 7f9812800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c135f60 0x7f980c0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.979+0000 7f9812800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c0affa0 0x7f980c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.979+0000 7f9812800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f980c071310 con 0x7f980c135f60 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.979+0000 7f9812800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f980c15de60 con 0x7f97ec004d40 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.979+0000 7f9812800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f980c1458c0 con 0x7f980c0affa0 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c135f60 0x7f980c0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c135f60 0x7f980c0afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42524/0 (socket says 172.21.15.19:42524) 2024-04-25T11:31:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 -- 172.21.15.19:0/3982049155 learned_addr learned my addr 172.21.15.19:0/3982049155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980b400640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec004d40 0x7f980c1f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980be00640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c0affa0 0x7f980c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 -- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c0affa0 msgr2=0x7f980c0b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c0affa0 0x7f980c0b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 -- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec004d40 msgr2=0x7f980c1f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec004d40 0x7f980c1f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 -- 172.21.15.19:0/3982049155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c0b08f0 con 0x7f980c135f60 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980b400640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec004d40 0x7f980c1f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.980+0000 7f980aa00640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c135f60 0x7f980c0afa60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f97f4011630 tx=0x7f97f4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.981+0000 7f9803400640 1 -- 172.21.15.19:0/3982049155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f400f920 con 0x7f980c135f60 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.981+0000 7f9803400640 1 -- 172.21.15.19:0/3982049155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f400fac0 con 0x7f980c135f60 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.981+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c0b2820 con 0x7f980c135f60 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.981+0000 7f9803400640 1 -- 172.21.15.19:0/3982049155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f400fe70 con 0x7f980c135f60 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.981+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c0b2de0 con 0x7f980c135f60 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.982+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980c0b3020 con 0x7f980c135f60 2024-04-25T11:31:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.983+0000 7f9803400640 1 -- 172.21.15.19:0/3982049155 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f97f4015460 con 0x7f980c135f60 2024-04-25T11:31:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.984+0000 7f9803400640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97e80744e0 0x7f97e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.985+0000 7f9803400640 1 -- 172.21.15.19:0/3982049155 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f97f4095bc0 con 0x7f980c135f60 2024-04-25T11:31:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.985+0000 7f980b400640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97e80744e0 0x7f97e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.986+0000 7f980b400640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97e80744e0 0x7f97e80769f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f97fc001050 tx=0x7f97fc001d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:12.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:12.989+0000 7f9803400640 1 -- 172.21.15.19:0/3982049155 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97f4019030 con 0x7f980c135f60 2024-04-25T11:31:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.257+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f980c050740 con 0x7f980c135f60 2024-04-25T11:31:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.264+0000 7f9803400640 1 -- 172.21.15.19:0/3982049155 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f97f4060340 con 0x7f980c135f60 2024-04-25T11:31:13.254 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T11:31:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:31:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T11:31:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T11:31:13.254 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:31:13.254 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T11:31:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97e80744e0 msgr2=0x7f97e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97e80744e0 0x7f97e80769f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f97fc001050 tx=0x7f97fc001d50 comp rx=0 tx=0).stop 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c135f60 msgr2=0x7f980c0afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c135f60 0x7f980c0afa60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f97f4011630 tx=0x7f97f4011af0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 shutdown_connections 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97e80744e0 0x7f97e80769f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f980c0affa0 0x7f980c0b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c135f60 0x7f980c0afa60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 --2- 172.21.15.19:0/3982049155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97ec004d40 0x7f980c1f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 >> 172.21.15.19:0/3982049155 conn(0x7f980c073090 msgr2=0x7f980c0b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 shutdown_connections 2024-04-25T11:31:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.267+0000 7f9812800640 1 -- 172.21.15.19:0/3982049155 wait complete. 2024-04-25T11:31:13.282 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.564+0000 7f8dccc00640 1 Processor -- start 2024-04-25T11:31:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.564+0000 7f8dccc00640 1 -- start start 2024-04-25T11:31:13.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.566+0000 7f8dccc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8dc80b4ea0 0x7f8dc80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.566+0000 7f8dccc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc80d1230 con 0x7f8dc806c5c0 2024-04-25T11:31:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.566+0000 7f8dccc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc80d06b0 con 0x7f8dc8144370 2024-04-25T11:31:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.566+0000 7f8dccc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8dc80d0990 con 0x7f8dc80b4ea0 2024-04-25T11:31:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.566+0000 7f8dc7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8dc806c5c0 0x7f8dc80b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58652/0 (socket says 172.21.15.19:58652) 2024-04-25T11:31:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.566+0000 7f8dc7400640 1 -- 172.21.15.19:0/639927659 learned_addr learned my addr 172.21.15.19:0/639927659 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/639927659 >> v2:172.21.15.169:3300/0 conn(0x7f8dc80b4ea0 0x7f8dc80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426541974 0 0) 0x7f8dc80d1230 con 0x7f8dc806c5c0 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dac003520 con 0x7f8dc806c5c0 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2556227433 0 0) 0x7f8dac003520 con 0x7f8dc806c5c0 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 >> v2:172.21.15.169:3300/0 conn(0x7f8dc80b4ea0 msgr2=0x7f8dc80cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 --2- 172.21.15.19:0/639927659 >> v2:172.21.15.169:3300/0 conn(0x7f8dc80b4ea0 0x7f8dc80cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 >> v1:172.21.15.169:6789/0 conn(0x7f8dc8144370 legacy=0x7f8dc815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc80d0c70 con 0x7f8dc806c5c0 2024-04-25T11:31:13.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8db80019f0 con 0x7f8dc806c5c0 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 >> v1:172.21.15.19:6789/0 conn(0x7f8dc806c5c0 legacy=0x7f8dc80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.567+0000 7f8dc6000640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dac0039b0 0x7f8dac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6000640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac004800 0x7f8dac004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6000640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0051c0 0x7f8dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80d0990 con 0x7f8dac004800 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac00c160 con 0x7f8dac0051c0 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac00c360 con 0x7f8dac0039b0 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac004800 0x7f8dac004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc7400640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dac0039b0 0x7f8dac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6a00640 1 -- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dac0039b0 msgr2=0x7f8dac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc7e00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0051c0 0x7f8dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dac0039b0 0x7f8dac003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6a00640 1 -- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0051c0 msgr2=0x7f8dac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0051c0 0x7f8dac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc6a00640 1 -- 172.21.15.19:0/639927659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac003520 con 0x7f8dac004800 2024-04-25T11:31:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc7e00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac0051c0 0x7f8dac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.568+0000 7f8dc7e00640 1 -- 172.21.15.19:0/639927659 reap_dead start 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.569+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac004800 0x7f8dac004c10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009ac0 tx=0x7f8dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8e172436eb1f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.570+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0036070 con 0x7f8dac004800 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.570+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0031400 con 0x7f8dac004800 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.570+0000 7f8dc6000640 1 -- 172.21.15.19:0/639927659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0009690 con 0x7f8dac004800 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.570+0000 7f8dccc00640 1 -- 172.21.15.19:0/639927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac004800 msgr2=0x7f8dac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.570+0000 7f8dccc00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac004800 0x7f8dac004c10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009ac0 tx=0x7f8dac005830 comp rx=0 tx=0).stop 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.571+0000 7f8dccc00640 1 -- 172.21.15.19:0/639927659 shutdown_connections 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.571+0000 7f8dccc00640 1 --2- 172.21.15.19:0/639927659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dac004800 0x7f8dac004c10 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.571+0000 7f8dccc00640 1 -- 172.21.15.19:0/639927659 >> 172.21.15.19:0/639927659 conn(0x7f8dc80d1720 msgr2=0x7f8dc80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.571+0000 7f8dccc00640 1 -- 172.21.15.19:0/639927659 shutdown_connections 2024-04-25T11:31:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.571+0000 7f8dccc00640 1 -- 172.21.15.19:0/639927659 wait complete. 2024-04-25T11:31:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.573+0000 7f8dccc00640 1 Processor -- start 2024-04-25T11:31:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.573+0000 7f8dccc00640 1 -- start start 2024-04-25T11:31:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dccc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac004800 0x7f8dc81f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dccc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc813ff70 0x7f8dc80cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dccc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dc80cb050 0x7f8dc806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dccc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80d0450 con 0x7f8dc813ff70 2024-04-25T11:31:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dccc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80d02d0 con 0x7f8dac004800 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dccc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc8147760 con 0x7f8dc80cb050 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dc6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc813ff70 0x7f8dc80cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dc6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc813ff70 0x7f8dc80cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42534/0 (socket says 172.21.15.19:42534) 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.575+0000 7f8dc6a00640 1 -- 172.21.15.19:0/221389633 learned_addr learned my addr 172.21.15.19:0/221389633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc7400640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac004800 0x7f8dc81f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc7e00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dc80cb050 0x7f8dc806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc6a00640 1 -- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dc80cb050 msgr2=0x7f8dc806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dc80cb050 0x7f8dc806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc6a00640 1 -- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac004800 msgr2=0x7f8dc81f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac004800 0x7f8dc81f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc6a00640 1 -- 172.21.15.19:0/221389633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc806f180 con 0x7f8dc813ff70 2024-04-25T11:31:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc7400640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac004800 0x7f8dc81f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.576+0000 7f8dc6a00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc813ff70 0x7f8dc80cab10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8db0034040 tx=0x7f8db0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.577+0000 7f8dbf400640 1 -- 172.21.15.19:0/221389633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db003f020 con 0x7f8dc813ff70 2024-04-25T11:31:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.577+0000 7f8dbf400640 1 -- 172.21.15.19:0/221389633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0037040 con 0x7f8dc813ff70 2024-04-25T11:31:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.577+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc806f5f0 con 0x7f8dc813ff70 2024-04-25T11:31:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.577+0000 7f8dbf400640 1 -- 172.21.15.19:0/221389633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0036070 con 0x7f8dc813ff70 2024-04-25T11:31:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.577+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc806fbb0 con 0x7f8dc813ff70 2024-04-25T11:31:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.578+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc806f350 con 0x7f8dc813ff70 2024-04-25T11:31:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.579+0000 7f8dbf400640 1 -- 172.21.15.19:0/221389633 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8db0042020 con 0x7f8dc813ff70 2024-04-25T11:31:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.580+0000 7f8dbf400640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8da40744e0 0x7f8da40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.581+0000 7f8dbf400640 1 -- 172.21.15.19:0/221389633 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f8db00b5390 con 0x7f8dc813ff70 2024-04-25T11:31:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.581+0000 7f8dc7400640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8da40744e0 0x7f8da40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.582+0000 7f8dc7400640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8da40744e0 0x7f8da40769f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8db8001580 tx=0x7f8db8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.585+0000 7f8dbf400640 1 -- 172.21.15.19:0/221389633 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8dc806f350 con 0x7f8dc813ff70 2024-04-25T11:31:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.833+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dc8050740 con 0x7f8dc813ff70 2024-04-25T11:31:13.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.834+0000 7f8dbf400640 1 -- 172.21.15.19:0/221389633 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5184 (secure 0 0 0) 0x7f8db0049030 con 0x7f8dc813ff70 2024-04-25T11:31:13.823 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:13.823 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:09.898969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6833/751039234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":751039234},{"type":"v1","addr":"172.21.15.19:6833","nonce":751039234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6231,"qdb_cluster":[6231]},"id":4}]} 2024-04-25T11:31:13.824 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 51 2024-04-25T11:31:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8da40744e0 msgr2=0x7f8da40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8da40744e0 0x7f8da40769f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8db8001580 tx=0x7f8db8009040 comp rx=0 tx=0).stop 2024-04-25T11:31:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc813ff70 msgr2=0x7f8dc80cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc813ff70 0x7f8dc80cab10 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8db0034040 tx=0x7f8db0004730 comp rx=0 tx=0).stop 2024-04-25T11:31:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 shutdown_connections 2024-04-25T11:31:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8da40744e0 0x7f8da40769f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dc80cb050 0x7f8dc806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dc813ff70 0x7f8dc80cab10 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 --2- 172.21.15.19:0/221389633 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dac004800 0x7f8dc81f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.836+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 >> 172.21.15.19:0/221389633 conn(0x7f8dc80d1720 msgr2=0x7f8dc80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.837+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 shutdown_connections 2024-04-25T11:31:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:13.837+0000 7f8dccc00640 1 -- 172.21.15.19:0/221389633 wait complete. 2024-04-25T11:31:14.256 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:31:06.789413+0000', 'modified': '2024-04-25T11:31:09.898969+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6231}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6231': {'gid': 6231, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6833/751039234', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 751039234}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 751039234}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6231, 'qdb_cluster': [6231]} 2024-04-25T11:31:14.256 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:31:14.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:31:14.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.557+0000 7fe1fc400640 1 Processor -- start 2024-04-25T11:31:14.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.557+0000 7fe1fc400640 1 -- start start 2024-04-25T11:31:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.559+0000 7fe1fc400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe1f41443c0 0x7fe1f406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.559+0000 7fe1fc400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f4070370 0x7fe1f40cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.559+0000 7fe1fc400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1f40cfb20 0x7fe1f40ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.559+0000 7fe1fc400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f40708d0 con 0x7fe1f40cfb20 2024-04-25T11:31:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.559+0000 7fe1fc400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f415f160 con 0x7fe1f41443c0 2024-04-25T11:31:14.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.559+0000 7fe1fc400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f40cff30 con 0x7fe1f4070370 2024-04-25T11:31:14.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.560+0000 7fe1fa400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1f40cfb20 0x7fe1f40ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.560+0000 7fe1fa400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1f40cfb20 0x7fe1f40ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40934/0 (socket says 172.21.15.19:40934) 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.560+0000 7fe1fa400640 1 -- 172.21.15.19:0/210306377 learned_addr learned my addr 172.21.15.19:0/210306377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.560+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/210306377 >> v2:172.21.15.169:3300/0 conn(0x7fe1f41443c0 0x7fe1f406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.560+0000 7fe1f9000640 1 --2- 172.21.15.19:0/210306377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f4070370 0x7fe1f40cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.561+0000 7fe1fa400640 1 -- 172.21.15.19:0/210306377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f4070370 msgr2=0x7fe1f40cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.561+0000 7fe1fa400640 1 --2- 172.21.15.19:0/210306377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f4070370 0x7fe1f40cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.561+0000 7fe1fa400640 1 -- 172.21.15.19:0/210306377 >> v2:172.21.15.169:3300/0 conn(0x7fe1f41443c0 msgr2=0x7fe1f406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.561+0000 7fe1fa400640 1 --2- 172.21.15.19:0/210306377 >> v2:172.21.15.169:3300/0 conn(0x7fe1f41443c0 0x7fe1f406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.561+0000 7fe1fa400640 1 -- 172.21.15.19:0/210306377 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f415f5f0 con 0x7fe1f40cfb20 2024-04-25T11:31:14.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.561+0000 7fe1f9000640 1 --2- 172.21.15.19:0/210306377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f4070370 0x7fe1f40cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:14.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.562+0000 7fe1fa400640 1 --2- 172.21.15.19:0/210306377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f40cfb20 0x7fe1f40ce330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4072820 tx=0x7fe1ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70a133010bde4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.563+0000 7fe1f3e00640 1 -- 172.21.15.19:0/210306377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec036070 con 0x7fe1f40cfb20 2024-04-25T11:31:14.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.563+0000 7fe1f3e00640 1 -- 172.21.15.19:0/210306377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec031600 con 0x7fe1f40cfb20 2024-04-25T11:31:14.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.563+0000 7fe1f3e00640 1 -- 172.21.15.19:0/210306377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0319d0 con 0x7fe1f40cfb20 2024-04-25T11:31:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.563+0000 7fe1fc400640 1 -- 172.21.15.19:0/210306377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f40cfb20 msgr2=0x7fe1f40ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.563+0000 7fe1fc400640 1 --2- 172.21.15.19:0/210306377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f40cfb20 0x7fe1f40ce330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4072820 tx=0x7fe1ec02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.564+0000 7fe1fc400640 1 -- 172.21.15.19:0/210306377 shutdown_connections 2024-04-25T11:31:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.564+0000 7fe1fc400640 1 --2- 172.21.15.19:0/210306377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f40cfb20 0x7fe1f40ce330 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.564+0000 7fe1fc400640 1 --2- 172.21.15.19:0/210306377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f4070370 0x7fe1f40cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.564+0000 7fe1fc400640 1 --2- 172.21.15.19:0/210306377 >> v2:172.21.15.169:3300/0 conn(0x7fe1f41443c0 0x7fe1f406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.564+0000 7fe1fc400640 1 -- 172.21.15.19:0/210306377 >> 172.21.15.19:0/210306377 conn(0x7fe1f40d1ae0 msgr2=0x7fe1f40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:14.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.564+0000 7fe1fc400640 1 -- 172.21.15.19:0/210306377 shutdown_connections 2024-04-25T11:31:14.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.564+0000 7fe1fc400640 1 -- 172.21.15.19:0/210306377 wait complete. 2024-04-25T11:31:14.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.566+0000 7fe1fc400640 1 Processor -- start 2024-04-25T11:31:14.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.567+0000 7fe1fc400640 1 -- start start 2024-04-25T11:31:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.568+0000 7fe1fc400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f4070370 0x7fe1f41f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.568+0000 7fe1fc400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f40cfb20 0x7fe1f415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.568+0000 7fe1fc400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1f41443c0 0x7fe1f415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.568+0000 7fe1fc400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f415edf0 con 0x7fe1f4070370 2024-04-25T11:31:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.568+0000 7fe1fc400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f415eb10 con 0x7fe1f41443c0 2024-04-25T11:31:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.568+0000 7fe1fc400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f41447d0 con 0x7fe1f40cfb20 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.569+0000 7fe1f9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f4070370 0x7fe1f41f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.569+0000 7fe1f9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f4070370 0x7fe1f41f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40950/0 (socket says 172.21.15.19:40950) 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.569+0000 7fe1f9a00640 1 -- 172.21.15.19:0/1480790969 learned_addr learned my addr 172.21.15.19:0/1480790969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.569+0000 7fe1fa400640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1f41443c0 0x7fe1f415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.569+0000 7fe1f9000640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f40cfb20 0x7fe1f415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.569+0000 7fe1f9a00640 1 -- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f40cfb20 msgr2=0x7fe1f415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.569+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f40cfb20 0x7fe1f415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1f9a00640 1 -- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1f41443c0 msgr2=0x7fe1f415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1f41443c0 0x7fe1f415af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1f9a00640 1 -- 172.21.15.19:0/1480790969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f40b5c20 con 0x7fe1f4070370 2024-04-25T11:31:14.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f4070370 0x7fe1f41f5b00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d600 tx=0x7fe1e800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1f1600640 1 -- 172.21.15.19:0/1480790969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8004240 con 0x7fe1f4070370 2024-04-25T11:31:14.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1f1600640 1 -- 172.21.15.19:0/1480790969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e80043e0 con 0x7fe1f4070370 2024-04-25T11:31:14.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1f1600640 1 -- 172.21.15.19:0/1480790969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8003830 con 0x7fe1f4070370 2024-04-25T11:31:14.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f4154e00 con 0x7fe1f4070370 2024-04-25T11:31:14.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.570+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f4154b20 con 0x7fe1f4070370 2024-04-25T11:31:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.574+0000 7fe1f1600640 1 -- 172.21.15.19:0/1480790969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1e80039d0 con 0x7fe1f4070370 2024-04-25T11:31:14.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.575+0000 7fe1f1600640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1d80744b0 0x7fe1d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.576+0000 7fe203464740 1 Processor -- start 2024-04-25T11:31:14.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.576+0000 7fe1f1600640 1 -- 172.21.15.19:0/1480790969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fe1e80955f0 con 0x7fe1f4070370 2024-04-25T11:31:14.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.576+0000 7fe203464740 1 -- start start 2024-04-25T11:31:14.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.576+0000 7fe1f9000640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1d80744b0 0x7fe1d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.577+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fce270 0x55d51539b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.577+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fa53b0 0x55d5153b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.577+0000 7fe203464740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fad590 0x55d5153b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.577+0000 7fe203464740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d5152d4600 con 0x55d514fad590 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.577+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d515271100 con 0x55d514fce270 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.577+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d5152d05c0 con 0x55d514fa53b0 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fad590 0x55d5153b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fad590 0x55d5153b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40964/0 (socket says 172.21.15.19:40964) 2024-04-25T11:31:14.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 -- 172.21.15.19:0/3555432779 learned_addr learned my addr 172.21.15.19:0/3555432779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1f9000640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1d80744b0 0x7fe1d80769c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe1f406dfd0 tx=0x7fe1e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1f9000640 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fa53b0 0x55d5153b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fce270 0x55d51539b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 -- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fa53b0 msgr2=0x55d5153b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fa53b0 0x55d5153b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 -- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fce270 msgr2=0x55d51539b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fce270 0x55d51539b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1fa400640 1 -- 172.21.15.19:0/3555432779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d514fe62d0 con 0x55d514fad590 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.578+0000 7fe1f9000640 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fa53b0 0x55d5153b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:14.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.579+0000 7fe1fa400640 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fad590 0x55d5153b62b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec02ac90 tx=0x7fe1ec02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8722cc97def7d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.579+0000 7fe1e2000640 1 -- 172.21.15.19:0/3555432779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec036040 con 0x55d514fad590 2024-04-25T11:31:14.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.579+0000 7fe1e2000640 1 -- 172.21.15.19:0/3555432779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec03c640 con 0x55d514fad590 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.579+0000 7fe1e2000640 1 -- 172.21.15.19:0/3555432779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec03c9f0 con 0x55d514fad590 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 -- 172.21.15.19:0/3555432779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fad590 msgr2=0x55d5153b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fad590 0x55d5153b62b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec02ac90 tx=0x7fe1ec02e000 comp rx=0 tx=0).stop 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 -- 172.21.15.19:0/3555432779 shutdown_connections 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fad590 0x55d5153b62b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fa53b0 0x55d5153b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 --2- 172.21.15.19:0/3555432779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fce270 0x55d51539b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 -- 172.21.15.19:0/3555432779 >> 172.21.15.19:0/3555432779 conn(0x55d514fc23c0 msgr2=0x55d5153706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 -- 172.21.15.19:0/3555432779 shutdown_connections 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.580+0000 7fe203464740 1 -- 172.21.15.19:0/3555432779 wait complete. 2024-04-25T11:31:14.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.581+0000 7fe203464740 1 -- start start 2024-04-25T11:31:14.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.582+0000 7fe203464740 1 Processor -- start 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.583+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.583+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d5153b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.583+0000 7fe203464740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.583+0000 7fe203464740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d51525bf70 con 0x55d514fce270 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.583+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d5153b1100 con 0x55d514fa53b0 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.583+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d5153b6790 con 0x55d514fad590 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.583+0000 7fe203464740 10 client.0 _ll_register_callbacks cb 0x55d5152ca480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40972/0 (socket says 172.21.15.19:40972) 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 -- 172.21.15.19:0/1066560812 learned_addr learned my addr 172.21.15.19:0/1066560812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1f9000640 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d5153b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 -- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 msgr2=0x55d5153b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d5153b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 -- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 msgr2=0x55d5153b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 -- 172.21.15.19:0/1066560812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5150b06b0 con 0x55d514fce270 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1fa400640 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153b0840 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec02a6e0 tx=0x7fe1ec02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5966b69567584fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.584+0000 7fe1d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe1d6a00640 1 -- 172.21.15.19:0/1066560812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec036070 con 0x55d514fce270 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ec03c640 con 0x55d514fce270 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe1d6a00640 1 -- 172.21.15.19:0/1066560812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fe1ec03c440 con 0x55d514fce270 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe1d6a00640 1 -- 172.21.15.19:0/1066560812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe1ec051050 con 0x55d514fce270 2024-04-25T11:31:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe203464740 10 client.6378 fetch_fsmap learned FSMap version 51 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55d5153b0e40 con 0x55d514fce270 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe1d6a00640 1 -- 172.21.15.19:0/1066560812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec03c640 con 0x55d514fce270 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.585+0000 7fe1d6a00640 1 -- 172.21.15.19:0/1066560812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fe1ec054020 con 0x55d514fce270 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.586+0000 7fe203464740 10 client.6378 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.586+0000 7fe203464740 10 client.6378 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.586+0000 7fe203464740 20 client.6378 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.586+0000 7fe203464740 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d514f8ce40 0x55d5153c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.586+0000 7fe203464740 4 client.6378 mds_command: new command op to 6231 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:31:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.586+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 --> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5152e2210 con 0x55d514f8ce40 2024-04-25T11:31:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.586+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d514f8ce40 0x55d5153c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.587+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d514f8ce40 0x55d5153c8d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7140b08cda66b7 server_cookie=23e37aeac7a82f65 in_seq=0 out_seq=0 2024-04-25T11:31:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.587+0000 7fe1d6a00640 10 client.6378 ms_handle_connect on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.587+0000 7fe1d6a00640 1 -- 172.21.15.19:0/1066560812 <== mds.0 v2:172.21.15.19:6832/751039234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d5152e2210 con 0x55d514f8ce40 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.587+0000 7fe1d6a00640 10 client.6378 handle_command_reply: tid=0 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 client.6378 shutdown 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 msgr2=0x55d5153b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153b0840 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec02a6e0 tx=0x7fe1ec02c000 comp rx=0 tx=0).stop 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 shutdown_connections 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d514f8ce40 0x55d5153c8d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153b0840 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d5153b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 --2- 172.21.15.19:0/1066560812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe1d6a00640 0 client.6378 ms_handle_reset on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 >> 172.21.15.19:0/1066560812 conn(0x55d514fc23c0 msgr2=0x55d51539b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 shutdown_connections 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 1 -- 172.21.15.19:0/1066560812 wait complete. 2024-04-25T11:31:14.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.588+0000 7fe203464740 20 client.6378 trim_cache size 0 max 16384 2024-04-25T11:31:14.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.609+0000 7fe203464740 1 Processor -- start 2024-04-25T11:31:14.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.609+0000 7fe203464740 1 -- start start 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.611+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.611+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d51539ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.611+0000 7fe203464740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.611+0000 7fe203464740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4201e40 con 0x55d514fce270 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.611+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41fa7b0 con 0x55d514fa53b0 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.611+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f40b5780 con 0x55d514fad590 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40978/0 (socket says 172.21.15.19:40978) 2024-04-25T11:31:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 -- 172.21.15.19:0/3131512901 learned_addr learned my addr 172.21.15.19:0/3131512901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1f9000640 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d51539ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 -- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 msgr2=0x55d51539ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d51539ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 -- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 msgr2=0x55d5153cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 -- 172.21.15.19:0/3131512901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5152d0780 con 0x55d514fce270 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.612+0000 7fe1fa400640 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153c7230 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec03cb90 tx=0x7fe1ec03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc3a4c3b7e0187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.613+0000 7fe1e1600640 1 -- 172.21.15.19:0/3131512901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec036030 con 0x55d514fce270 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.613+0000 7fe1e1600640 1 -- 172.21.15.19:0/3131512901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec03c540 con 0x55d514fce270 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.613+0000 7fe1e1600640 1 -- 172.21.15.19:0/3131512901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec045650 con 0x55d514fce270 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 -- 172.21.15.19:0/3131512901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 msgr2=0x55d5153c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153c7230 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec03cb90 tx=0x7fe1ec03b000 comp rx=0 tx=0).stop 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 -- 172.21.15.19:0/3131512901 shutdown_connections 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fce270 0x55d5153c7230 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fad590 0x55d51539ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 --2- 172.21.15.19:0/3131512901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fa53b0 0x55d5153cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 -- 172.21.15.19:0/3131512901 >> 172.21.15.19:0/3131512901 conn(0x55d514fc23c0 msgr2=0x55d5153b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 -- 172.21.15.19:0/3131512901 shutdown_connections 2024-04-25T11:31:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.614+0000 7fe203464740 1 -- 172.21.15.19:0/3131512901 wait complete. 2024-04-25T11:31:14.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.615+0000 7fe203464740 1 -- start start 2024-04-25T11:31:14.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.615+0000 7fe203464740 1 Processor -- start 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe203464740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fa53b0 0x55d515395790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fad590 0x55d5154329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe203464740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fce270 0x55d51539dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe203464740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d515396320 con 0x55d514fa53b0 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d5153b62f0 con 0x55d514fad590 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe203464740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d5153b9eb0 con 0x55d514fce270 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe203464740 10 client.0 _ll_register_callbacks cb 0x7fe1ec009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1f9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fa53b0 0x55d515395790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1f9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fa53b0 0x55d515395790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40992/0 (socket says 172.21.15.19:40992) 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1f9a00640 1 -- 172.21.15.19:0/2686914852 learned_addr learned my addr 172.21.15.19:0/2686914852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1fa400640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fce270 0x55d51539dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1f9000640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fad590 0x55d5154329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1f9a00640 1 -- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fce270 msgr2=0x55d51539dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fce270 0x55d51539dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.617+0000 7fe1f9a00640 1 -- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fad590 msgr2=0x55d5154329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fad590 0x55d5154329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1f9a00640 1 -- 172.21.15.19:0/2686914852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5153ba840 con 0x55d514fa53b0 2024-04-25T11:31:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1f9000640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fad590 0x55d5154329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1f9a00640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fa53b0 0x55d515395790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe1e805f570 tx=0x7fe1e8093950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3499d5b1ecd294b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1d6a00640 1 -- 172.21.15.19:0/2686914852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8063990 con 0x55d514fa53b0 2024-04-25T11:31:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1d6a00640 1 -- 172.21.15.19:0/2686914852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fe1e805e840 con 0x55d514fa53b0 2024-04-25T11:31:14.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ec03c540 con 0x55d514fa53b0 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe1d6a00640 1 -- 172.21.15.19:0/2686914852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe1e8068440 con 0x55d514fa53b0 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe203464740 10 client.6384 fetch_fsmap learned FSMap version 51 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.618+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55d5154342e0 con 0x55d514fa53b0 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe1d6a00640 1 -- 172.21.15.19:0/2686914852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec03c540 con 0x55d514fa53b0 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe1d6a00640 1 -- 172.21.15.19:0/2686914852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7fe1e8013070 con 0x55d514fa53b0 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe203464740 10 client.6384 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe203464740 10 client.6384 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe203464740 20 client.6384 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe203464740 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d5153d8630 0x55d515433260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe203464740 4 client.6384 mds_command: new command op to 6231 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 --> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] -- command(tid 0: {"prefix": "status"}) -- 0x55d5152dc080 con 0x55d5153d8630 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.619+0000 7fe1f9000640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d5153d8630 0x55d515433260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.620+0000 7fe1d6a00640 10 client.6384 ms_handle_connect on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.620+0000 7fe1f9000640 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d5153d8630 0x55d515433260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb6392769ed24b1 server_cookie=4be0077e1298f8d9 in_seq=0 out_seq=0 2024-04-25T11:31:14.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe1d6a00640 1 -- 172.21.15.19:0/2686914852 <== mds.0 v2:172.21.15.19:6832/751039234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d5152dc080 con 0x55d5153d8630 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe1d6a00640 10 client.6384 handle_command_reply: tid=0 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 client.6384 shutdown 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fa53b0 msgr2=0x55d515395790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fa53b0 0x55d515395790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe1e805f570 tx=0x7fe1e8093950 comp rx=0 tx=0).stop 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 shutdown_connections 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x55d5153d8630 0x55d515433260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d514fad590 0x55d5154329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d514fce270 0x55d51539dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 --2- 172.21.15.19:0/2686914852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d514fa53b0 0x55d515395790 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe1d6a00640 0 client.6384 ms_handle_reset on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 >> 172.21.15.19:0/2686914852 conn(0x55d514fc23c0 msgr2=0x55d5153b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 shutdown_connections 2024-04-25T11:31:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 1 -- 172.21.15.19:0/2686914852 wait complete. 2024-04-25T11:31:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.621+0000 7fe203464740 20 client.6384 trim_cache size 0 max 16384 2024-04-25T11:31:14.613 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:31:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:31:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6231, 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3397632189999999, 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 51, 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 63, 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 63, 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.528264783999999 2024-04-25T11:31:14.614 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1d80744b0 msgr2=0x7fe1d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1d80744b0 0x7fe1d80769c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe1f406dfd0 tx=0x7fe1e4008040 comp rx=0 tx=0).stop 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f4070370 msgr2=0x7fe1f41f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f4070370 0x7fe1f41f5b00 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d600 tx=0x7fe1e800dac0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 shutdown_connections 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1d80744b0 0x7fe1d80769c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1f41443c0 0x7fe1f415af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1f40cfb20 0x7fe1f415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 --2- 172.21.15.19:0/1480790969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1f4070370 0x7fe1f41f5b00 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 >> 172.21.15.19:0/1480790969 conn(0x7fe1f40d1ae0 msgr2=0x7fe1f4152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 shutdown_connections 2024-04-25T11:31:14.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.622+0000 7fe1fc400640 1 -- 172.21.15.19:0/1480790969 wait complete. 2024-04-25T11:31:14.639 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:31:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.921+0000 7f7f01000640 1 Processor -- start 2024-04-25T11:31:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.922+0000 7f7f01000640 1 -- start start 2024-04-25T11:31:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.924+0000 7f7f01000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7efc06d5d0 0x7f7efc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.924+0000 7f7f01000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc06df20 con 0x7f7efc141fd0 2024-04-25T11:31:14.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.924+0000 7f7f01000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7efc141210 con 0x7f7efc140e00 2024-04-25T11:31:14.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.924+0000 7f7f01000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7efc141b40 con 0x7f7efc06d5d0 2024-04-25T11:31:14.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.925+0000 7f7efbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7efc141fd0 0x7f7efc1423e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60028/0 (socket says 172.21.15.19:60028) 2024-04-25T11:31:14.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.925+0000 7f7efbe00640 1 -- 172.21.15.19:0/470578713 learned_addr learned my addr 172.21.15.19:0/470578713 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:14.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.925+0000 7f7ef3e00640 1 --2- 172.21.15.19:0/470578713 >> v2:172.21.15.169:3300/0 conn(0x7f7efc06d5d0 0x7f7efc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.925+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652511806 0 0) 0x7f7ef4001490 con 0x7f7efc141fd0 2024-04-25T11:31:14.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.925+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ee0003520 con 0x7f7efc141fd0 2024-04-25T11:31:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2089026685 0 0) 0x7f7ee0003520 con 0x7f7efc141fd0 2024-04-25T11:31:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 >> v2:172.21.15.169:3300/0 conn(0x7f7efc06d5d0 msgr2=0x7f7efc06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 --2- 172.21.15.19:0/470578713 >> v2:172.21.15.169:3300/0 conn(0x7f7efc06d5d0 0x7f7efc06d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 >> v1:172.21.15.169:6789/0 conn(0x7f7efc140e00 legacy=0x7f7efc143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc141510 con 0x7f7efc141fd0 2024-04-25T11:31:14.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7ef4001bc0 con 0x7f7efc141fd0 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 >> v1:172.21.15.19:6789/0 conn(0x7f7efc141fd0 legacy=0x7f7efc1423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ee0003a00 0x7f7ee0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0004800 0x7f7ee0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ee0005270 0x7f7ee000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc141b40 con 0x7f7ee0003a00 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee000c190 con 0x7f7ee0005270 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee000c390 con 0x7f7ee0004800 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.926+0000 7f7efb400640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ee0003a00 0x7f7ee0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7efbe00640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ee0005270 0x7f7ee000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7ef3e00640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0004800 0x7f7ee0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7efb400640 1 -- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0004800 msgr2=0x7f7ee0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7efb400640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0004800 0x7f7ee0004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7efb400640 1 -- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ee0005270 msgr2=0x7f7ee000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7efb400640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ee0005270 0x7f7ee000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7efb400640 1 -- 172.21.15.19:0/470578713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee0003520 con 0x7f7ee0003a00 2024-04-25T11:31:14.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.927+0000 7f7efbe00640 1 -- 172.21.15.19:0/470578713 reap_dead start 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.928+0000 7f7efb400640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ee0003a00 0x7f7ee0003e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7eec001750 tx=0x7f7ee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4397bf0830cab13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.928+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec001370 con 0x7f7ee0003a00 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.928+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec0052d0 con 0x7f7ee0003a00 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.928+0000 7f7efaa00640 1 -- 172.21.15.19:0/470578713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec02abc0 con 0x7f7ee0003a00 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.929+0000 7f7f01000640 1 -- 172.21.15.19:0/470578713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ee0003a00 msgr2=0x7f7ee0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.929+0000 7f7f01000640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ee0003a00 0x7f7ee0003e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7eec001750 tx=0x7f7ee0005830 comp rx=0 tx=0).stop 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.929+0000 7f7f01000640 1 -- 172.21.15.19:0/470578713 shutdown_connections 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.929+0000 7f7f01000640 1 --2- 172.21.15.19:0/470578713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ee0003a00 0x7f7ee0003e70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.929+0000 7f7f01000640 1 -- 172.21.15.19:0/470578713 >> 172.21.15.19:0/470578713 conn(0x7f7efc072ee0 msgr2=0x7f7efc0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.929+0000 7f7f01000640 1 -- 172.21.15.19:0/470578713 shutdown_connections 2024-04-25T11:31:14.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.929+0000 7f7f01000640 1 -- 172.21.15.19:0/470578713 wait complete. 2024-04-25T11:31:14.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.932+0000 7f7f01000640 1 Processor -- start 2024-04-25T11:31:14.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.932+0000 7f7f01000640 1 -- start start 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7f01000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0003a00 0x7f7efc147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7f01000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1fa0a0 0x7f7efc1fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7f01000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7efc0b5a80 0x7f7efc0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7f01000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1417d0 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7f01000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1443d0 con 0x7f7efc0b5a80 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7f01000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc0d1490 con 0x7f7ee0003a00 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7ef3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1fa0a0 0x7f7efc1fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7ef3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1fa0a0 0x7f7efc1fa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41002/0 (socket says 172.21.15.19:41002) 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7ef3e00640 1 -- 172.21.15.19:0/3169717139 learned_addr learned my addr 172.21.15.19:0/3169717139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.934+0000 7f7efbe00640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7efc0b5a80 0x7f7efc0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7efb400640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0003a00 0x7f7efc147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7ef3e00640 1 -- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0003a00 msgr2=0x7f7efc147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7ef3e00640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0003a00 0x7f7efc147160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7ef3e00640 1 -- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7efc0b5a80 msgr2=0x7f7efc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7ef3e00640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7efc0b5a80 0x7f7efc0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:14.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7ef3e00640 1 -- 172.21.15.19:0/3169717139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc0b63d0 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7ef3e00640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1fa0a0 0x7f7efc1fa4b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4010790 tx=0x7f7ee4010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.935+0000 7f7ef3400640 1 -- 172.21.15.19:0/3169717139 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400f040 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.936+0000 7f7ef3400640 1 -- 172.21.15.19:0/3169717139 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee400eb60 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.936+0000 7f7ef3400640 1 -- 172.21.15.19:0/3169717139 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4010ea0 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.936+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc130bb0 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.936+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc1300d0 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.937+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7efc050740 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.938+0000 7f7ef3400640 1 -- 172.21.15.19:0/3169717139 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ee401c400 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.940+0000 7f7ef3400640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7ed80744e0 0x7f7ed80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.940+0000 7f7ef3400640 1 -- 172.21.15.19:0/3169717139 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f7ee4095710 con 0x7f7efc1fa0a0 2024-04-25T11:31:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.941+0000 7f7efb400640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7ed80744e0 0x7f7ed80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:14.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.942+0000 7f7efb400640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7ed80744e0 0x7f7ed80769f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7eec032000 tx=0x7f7eec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:14.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:14.945+0000 7f7ef3400640 1 -- 172.21.15.19:0/3169717139 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7efc0b63d0 con 0x7f7efc1fa0a0 2024-04-25T11:31:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.192+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7efc140390 con 0x7f7efc1fa0a0 2024-04-25T11:31:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.193+0000 7f7ef3400640 1 -- 172.21.15.19:0/3169717139 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f7ee4018030 con 0x7f7efc1fa0a0 2024-04-25T11:31:15.183 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:31:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.196+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7ed80744e0 msgr2=0x7f7ed80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.196+0000 7f7f01000640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7ed80744e0 0x7f7ed80769f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7eec032000 tx=0x7f7eec02e000 comp rx=0 tx=0).stop 2024-04-25T11:31:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.196+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1fa0a0 msgr2=0x7f7efc1fa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.196+0000 7f7f01000640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1fa0a0 0x7f7efc1fa4b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4010790 tx=0x7f7ee4010c50 comp rx=0 tx=0).stop 2024-04-25T11:31:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.196+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 shutdown_connections 2024-04-25T11:31:15.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.196+0000 7f7f01000640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7ed80744e0 0x7f7ed80769f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.197+0000 7f7f01000640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7efc0b5a80 0x7f7efc0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.197+0000 7f7f01000640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7efc1fa0a0 0x7f7efc1fa4b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.197+0000 7f7f01000640 1 --2- 172.21.15.19:0/3169717139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ee0003a00 0x7f7efc147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.197+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 >> 172.21.15.19:0/3169717139 conn(0x7f7efc072ee0 msgr2=0x7f7efc15dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.197+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 shutdown_connections 2024-04-25T11:31:15.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15.197+0000 7f7f01000640 1 -- 172.21.15.19:0/3169717139 wait complete. 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:31:15.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:31:15.218 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:31:15.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:15.218 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:31:15.275 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:31:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:31:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:31:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:31:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:15.276 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:31:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:31:15.277 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:31:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:31:15.278 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:31:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:31:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:31:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42398sec preferred_lft 42398sec 2024-04-25T11:31:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:31:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:15.279 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:31:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:31:15.279 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:15.280 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:31:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:31:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:31:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:15.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:15.282 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:31:15.366 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:31:15.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:15.370 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:31:15.423 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:31:15.423 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:31:15.423 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:31:15.423 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:31:15.423 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:31:15.424 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:31:15.424 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:31:15.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:31:15.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:31:15.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:31:15.424 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:31:15.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:31:15.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:31:15.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:15.649 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:31:15.698 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:31:15.698 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:31:15.699 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:31:15.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:15.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:31:15.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:31:15.782 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:15.840 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:31:15.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:31:15.840 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:15.889 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:31:15.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:15.889 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:31:16.098 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:31:16.098 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:31:16.144 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:31:16.144 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:31:16.144 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:31:16.144 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:31:16.144 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:31:16.146 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:31:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:31:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:31:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:31:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:31:16.150 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:31:16.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:16.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:16.228 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:31:16.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:31:16.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:31:16.366 INFO:teuthology.orchestra.run.smithi019.stdout:client5177 2024-04-25T11:31:16.366 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:31:16.367 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:31:16.419 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d; sync .') 2024-04-25T11:31:16.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:31:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:31:16.577 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:31:16.577 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:31:16.633 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:31:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.923+0000 7f4d6b200640 1 Processor -- start 2024-04-25T11:31:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.923+0000 7f4d6b200640 1 -- start start 2024-04-25T11:31:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.926+0000 7f4d6b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d640b4ea0 0x7f4d640cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.926+0000 7f4d6b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d6415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.926+0000 7f4d6b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d640d1230 con 0x7f4d6406c5c0 2024-04-25T11:31:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.926+0000 7f4d6b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4d640d06b0 con 0x7f4d640b4ea0 2024-04-25T11:31:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.926+0000 7f4d6b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4d640d0990 con 0x7f4d64144370 2024-04-25T11:31:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.926+0000 7f4d69400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d640b4ea0 0x7f4d640cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.927+0000 7f4d6a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d6415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.927+0000 7f4d69400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d640b4ea0 0x7f4d640cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:33912/0 (socket says 172.21.15.19:33912) 2024-04-25T11:31:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.927+0000 7f4d69400640 1 -- 172.21.15.19:0/3952372164 learned_addr learned my addr 172.21.15.19:0/3952372164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.927+0000 7f4d69e00640 1 --1- 172.21.15.19:0/3952372164 >> v1:172.21.15.169:6789/0 conn(0x7f4d6406c5c0 0x7f4d640b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:36604/0 (socket says 172.21.15.19:36604) 2024-04-25T11:31:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.928+0000 7f4d6a800640 1 -- 172.21.15.19:0/3952372164 >> v2:172.21.15.169:3300/0 conn(0x7f4d640b4ea0 msgr2=0x7f4d640cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.928+0000 7f4d6a800640 1 --2- 172.21.15.19:0/3952372164 >> v2:172.21.15.169:3300/0 conn(0x7f4d640b4ea0 0x7f4d640cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.928+0000 7f4d6a800640 1 -- 172.21.15.19:0/3952372164 >> v1:172.21.15.169:6789/0 conn(0x7f4d6406c5c0 legacy=0x7f4d640b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.928+0000 7f4d6a800640 1 -- 172.21.15.19:0/3952372164 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d640d0c70 con 0x7f4d64144370 2024-04-25T11:31:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.928+0000 7f4d69400640 1 --2- 172.21.15.19:0/3952372164 >> v2:172.21.15.169:3300/0 conn(0x7f4d640b4ea0 0x7f4d640cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:31:16.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.929+0000 7f4d6a800640 1 --2- 172.21.15.19:0/3952372164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d6415e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009d90 tx=0x7f4d6002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c5173731da82e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.929+0000 7f4d68a00640 1 -- 172.21.15.19:0/3952372164 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60007740 con 0x7f4d64144370 2024-04-25T11:31:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d68a00640 1 -- 172.21.15.19:0/3952372164 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d600078e0 con 0x7f4d64144370 2024-04-25T11:31:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d68a00640 1 -- 172.21.15.19:0/3952372164 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60007cb0 con 0x7f4d64144370 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 -- 172.21.15.19:0/3952372164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 msgr2=0x7f4d6415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 --2- 172.21.15.19:0/3952372164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d6415e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009d90 tx=0x7f4d6002a660 comp rx=0 tx=0).stop 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 -- 172.21.15.19:0/3952372164 shutdown_connections 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 --2- 172.21.15.19:0/3952372164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d6415e770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 --2- 172.21.15.19:0/3952372164 >> v2:172.21.15.169:3300/0 conn(0x7f4d640b4ea0 0x7f4d640cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 -- 172.21.15.19:0/3952372164 >> 172.21.15.19:0/3952372164 conn(0x7f4d640d1720 msgr2=0x7f4d640d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 -- 172.21.15.19:0/3952372164 shutdown_connections 2024-04-25T11:31:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.930+0000 7f4d6b200640 1 -- 172.21.15.19:0/3952372164 wait complete. 2024-04-25T11:31:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.933+0000 7f4d6b200640 1 Processor -- start 2024-04-25T11:31:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.933+0000 7f4d6b200640 1 -- start start 2024-04-25T11:31:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.934+0000 7f4d6b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d6406c5c0 0x7f4d64162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.934+0000 7f4d6b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d641609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.934+0000 7f4d6b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d64141110 0x7f4d640b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.934+0000 7f4d6b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640d0450 con 0x7f4d64141110 2024-04-25T11:31:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.934+0000 7f4d6b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640d02d0 con 0x7f4d6406c5c0 2024-04-25T11:31:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.934+0000 7f4d6b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4d64147760 con 0x7f4d64144370 2024-04-25T11:31:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.935+0000 7f4d6a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d64141110 0x7f4d640b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.935+0000 7f4d6a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d64141110 0x7f4d640b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41016/0 (socket says 172.21.15.19:41016) 2024-04-25T11:31:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.935+0000 7f4d6a800640 1 -- 172.21.15.19:0/2318897673 learned_addr learned my addr 172.21.15.19:0/2318897673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.935+0000 7f4d69e00640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d6406c5c0 0x7f4d64162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.935+0000 7f4d69400640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d641609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.935+0000 7f4d6a800640 1 -- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 msgr2=0x7f4d641609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d6a800640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d641609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d6a800640 1 -- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d6406c5c0 msgr2=0x7f4d64162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d6a800640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d6406c5c0 0x7f4d64162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d6a800640 1 -- 172.21.15.19:0/2318897673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6415f910 con 0x7f4d64141110 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d6a800640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d64141110 0x7f4d640b2900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009eb0 tx=0x7f4d60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d5e000640 1 -- 172.21.15.19:0/2318897673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6003f070 con 0x7f4d64141110 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d5e000640 1 -- 172.21.15.19:0/2318897673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d60031bc0 con 0x7f4d64141110 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d5e000640 1 -- 172.21.15.19:0/2318897673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60009450 con 0x7f4d64141110 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.936+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6415fbf0 con 0x7f4d64141110 2024-04-25T11:31:16.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.937+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d641601b0 con 0x7f4d64141110 2024-04-25T11:31:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.938+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d641626b0 con 0x7f4d64141110 2024-04-25T11:31:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.939+0000 7f4d5e000640 1 -- 172.21.15.19:0/2318897673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4d6003b410 con 0x7f4d64141110 2024-04-25T11:31:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.940+0000 7f4d5e000640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d480744b0 0x7f4d480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.940+0000 7f4d5e000640 1 -- 172.21.15.19:0/2318897673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f4d600b64a0 con 0x7f4d64141110 2024-04-25T11:31:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.941+0000 7f4d69e00640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d480744b0 0x7f4d480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.942+0000 7f4d69e00640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d480744b0 0x7f4d480769c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d58000f60 tx=0x7f4d58007c80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:16.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:16.945+0000 7f4d5e000640 1 -- 172.21.15.19:0/2318897673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d6004b070 con 0x7f4d64141110 2024-04-25T11:31:17.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.156+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d64050740 con 0x7f4d64141110 2024-04-25T11:31:17.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.158+0000 7f4d5e000640 1 -- 172.21.15.19:0/2318897673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16665 (secure 0 0 0) 0x7f4d60080b90 con 0x7f4d64141110 2024-04-25T11:31:17.149 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:17.149 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:31:08.278038+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:31:17.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d480744b0 msgr2=0x7f4d480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:17.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d480744b0 0x7f4d480769c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d58000f60 tx=0x7f4d58007c80 comp rx=0 tx=0).stop 2024-04-25T11:31:17.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d64141110 msgr2=0x7f4d640b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:17.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d64141110 0x7f4d640b2900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009eb0 tx=0x7f4d60004a40 comp rx=0 tx=0).stop 2024-04-25T11:31:17.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 shutdown_connections 2024-04-25T11:31:17.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4d480744b0 0x7f4d480769c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d64141110 0x7f4d640b2900 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d64144370 0x7f4d641609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.162+0000 7f4d6b200640 1 --2- 172.21.15.19:0/2318897673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d6406c5c0 0x7f4d64162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.163+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 >> 172.21.15.19:0/2318897673 conn(0x7f4d640d1720 msgr2=0x7f4d6415d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:17.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.163+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 shutdown_connections 2024-04-25T11:31:17.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.163+0000 7f4d6b200640 1 -- 172.21.15.19:0/2318897673 wait complete. 2024-04-25T11:31:17.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:31:05.039941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:31:05.784246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:31:17.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:31:17.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.487+0000 7fdae4800640 1 Processor -- start 2024-04-25T11:31:17.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.488+0000 7fdae4800640 1 -- start start 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.490+0000 7fdae4800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdadc070630 0x7fdadc0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.490+0000 7fdae4800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdadc06dce0 0x7fdadc160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.490+0000 7fdae4800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdadc071110 con 0x7fdadc15adf0 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.490+0000 7fdae4800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdadc0713f0 con 0x7fdadc070630 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.490+0000 7fdae4800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdadc0701a0 con 0x7fdadc06dce0 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.490+0000 7fdae1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdadc070630 0x7fdadc0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.491+0000 7fdae1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdadc070630 0x7fdadc0cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41020/0 (socket says 172.21.15.19:41020) 2024-04-25T11:31:17.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.491+0000 7fdae1e00640 1 -- 172.21.15.19:0/2582073985 learned_addr learned my addr 172.21.15.19:0/2582073985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.491+0000 7fdae1e00640 1 -- 172.21.15.19:0/2582073985 >> v2:172.21.15.169:3300/0 conn(0x7fdadc06dce0 msgr2=0x7fdadc160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.491+0000 7fdae1e00640 1 --2- 172.21.15.19:0/2582073985 >> v2:172.21.15.169:3300/0 conn(0x7fdadc06dce0 0x7fdadc160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.491+0000 7fdae1e00640 1 -- 172.21.15.19:0/2582073985 >> v1:172.21.15.169:6789/0 conn(0x7fdadc15adf0 legacy=0x7fdadc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.491+0000 7fdae1e00640 1 -- 172.21.15.19:0/2582073985 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc06fb50 con 0x7fdadc070630 2024-04-25T11:31:17.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.492+0000 7fdae1e00640 1 --2- 172.21.15.19:0/2582073985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc0cec50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0d0090 tx=0x7fdad002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6962ebb41536f260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.493+0000 7fdae0a00640 1 -- 172.21.15.19:0/2582073985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0037070 con 0x7fdadc070630 2024-04-25T11:31:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.493+0000 7fdae0a00640 1 -- 172.21.15.19:0/2582073985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad0032440 con 0x7fdadc070630 2024-04-25T11:31:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.493+0000 7fdae0a00640 1 -- 172.21.15.19:0/2582073985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0032810 con 0x7fdadc070630 2024-04-25T11:31:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.493+0000 7fdae4800640 1 -- 172.21.15.19:0/2582073985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 msgr2=0x7fdadc0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.493+0000 7fdae4800640 1 --2- 172.21.15.19:0/2582073985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc0cec50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0d0090 tx=0x7fdad002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.494+0000 7fdae4800640 1 -- 172.21.15.19:0/2582073985 shutdown_connections 2024-04-25T11:31:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.494+0000 7fdae4800640 1 --2- 172.21.15.19:0/2582073985 >> v2:172.21.15.169:3300/0 conn(0x7fdadc06dce0 0x7fdadc160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.494+0000 7fdae4800640 1 --2- 172.21.15.19:0/2582073985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc0cec50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.494+0000 7fdae4800640 1 -- 172.21.15.19:0/2582073985 >> 172.21.15.19:0/2582073985 conn(0x7fdadc073090 msgr2=0x7fdadc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.494+0000 7fdae4800640 1 -- 172.21.15.19:0/2582073985 shutdown_connections 2024-04-25T11:31:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.494+0000 7fdae4800640 1 -- 172.21.15.19:0/2582073985 wait complete. 2024-04-25T11:31:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.496+0000 7fdae4800640 1 Processor -- start 2024-04-25T11:31:17.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.496+0000 7fdae4800640 1 -- start start 2024-04-25T11:31:17.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae4800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc14c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:17.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae4800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc15adf0 0x7fdadc15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:17.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae4800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15e4c0 0x7fdadc1f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:17.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae4800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc0b0e80 con 0x7fdadc070630 2024-04-25T11:31:17.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae4800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc0b0d00 con 0x7fdadc15adf0 2024-04-25T11:31:17.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae4800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdadc070c80 con 0x7fdadc15e4c0 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc14c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc14c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41032/0 (socket says 172.21.15.19:41032) 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae1e00640 1 -- 172.21.15.19:0/2267055216 learned_addr learned my addr 172.21.15.19:0/2267055216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.498+0000 7fdae1400640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc15adf0 0x7fdadc15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae1e00640 1 -- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15e4c0 msgr2=0x7fdadc1f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae2800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15e4c0 0x7fdadc1f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae1e00640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15e4c0 0x7fdadc1f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae1e00640 1 -- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc15adf0 msgr2=0x7fdadc15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae1e00640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc15adf0 0x7fdadc15fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae1e00640 1 -- 172.21.15.19:0/2267055216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc06ce90 con 0x7fdadc070630 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae1400640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc15adf0 0x7fdadc15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:17.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae2800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15e4c0 0x7fdadc1f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdae1e00640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc14c670 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdad0000c00 tx=0x7fdad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:17.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.499+0000 7fdad6000640 1 -- 172.21.15.19:0/2267055216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0041070 con 0x7fdadc070630 2024-04-25T11:31:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.500+0000 7fdad6000640 1 -- 172.21.15.19:0/2267055216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad003c870 con 0x7fdadc070630 2024-04-25T11:31:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.500+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc06d170 con 0x7fdadc070630 2024-04-25T11:31:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.500+0000 7fdad6000640 1 -- 172.21.15.19:0/2267055216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0037040 con 0x7fdadc070630 2024-04-25T11:31:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.500+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc1f25b0 con 0x7fdadc070630 2024-04-25T11:31:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.501+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdadc156430 con 0x7fdadc070630 2024-04-25T11:31:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.502+0000 7fdad6000640 1 -- 172.21.15.19:0/2267055216 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdad0044070 con 0x7fdadc070630 2024-04-25T11:31:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.504+0000 7fdad6000640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdac00744b0 0x7fdac00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.504+0000 7fdad6000640 1 -- 172.21.15.19:0/2267055216 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fdad00b5eb0 con 0x7fdadc070630 2024-04-25T11:31:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.504+0000 7fdae1400640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdac00744b0 0x7fdac00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:17.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.506+0000 7fdae1400640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdac00744b0 0x7fdac00769c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdadc06d780 tx=0x7fdacc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:17.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.508+0000 7fdad6000640 1 -- 172.21.15.19:0/2267055216 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdadc156430 con 0x7fdadc070630 2024-04-25T11:31:17.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.715+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdadc050740 con 0x7fdadc070630 2024-04-25T11:31:17.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.717+0000 7fdad6000640 1 -- 172.21.15.19:0/2267055216 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7fdad00805a0 con 0x7fdadc070630 2024-04-25T11:31:17.707 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdac00744b0 msgr2=0x7fdac00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdac00744b0 0x7fdac00769c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdadc06d780 tx=0x7fdacc008040 comp rx=0 tx=0).stop 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 msgr2=0x7fdadc14c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc14c670 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdad0000c00 tx=0x7fdad00046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 shutdown_connections 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdac00744b0 0x7fdac00769c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc15adf0 0x7fdadc15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15e4c0 0x7fdadc1f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 --2- 172.21.15.19:0/2267055216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdadc070630 0x7fdadc14c670 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 >> 172.21.15.19:0/2267055216 conn(0x7fdadc073090 msgr2=0x7fdadc0b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:17.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 shutdown_connections 2024-04-25T11:31:17.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:17.720+0000 7fdae4800640 1 -- 172.21.15.19:0/2267055216 wait complete. 2024-04-25T11:31:17.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:31:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.039+0000 7f7e31e00640 1 Processor -- start 2024-04-25T11:31:18.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.040+0000 7f7e31e00640 1 -- start start 2024-04-25T11:31:18.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.042+0000 7f7e31e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c06dce0 0x7f7e2c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.042+0000 7f7e31e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c070000 con 0x7f7e2c144340 2024-04-25T11:31:18.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.042+0000 7f7e31e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c06f240 con 0x7f7e2c15ec90 2024-04-25T11:31:18.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.042+0000 7f7e31e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c144750 con 0x7f7e2c06dce0 2024-04-25T11:31:18.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.042+0000 7f7e30a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7e2c144340 0x7f7e2c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60030/0 (socket says 172.21.15.19:60030) 2024-04-25T11:31:18.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.043+0000 7f7e30a00640 1 -- 172.21.15.19:0/2815677586 learned_addr learned my addr 172.21.15.19:0/2815677586 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.043+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c06dce0 0x7f7e2c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.043+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295668901 0 0) 0x7f7e2c070000 con 0x7f7e2c144340 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.043+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e14003520 con 0x7f7e2c144340 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 429432689 0 0) 0x7f7e14003520 con 0x7f7e2c144340 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c06dce0 msgr2=0x7f7e2c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c06dce0 0x7f7e2c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 >> v1:172.21.15.169:6789/0 conn(0x7f7e2c15ec90 legacy=0x7f7e2c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c06f520 con 0x7f7e2c144340 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7e20001670 con 0x7f7e2c144340 2024-04-25T11:31:18.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 >> v1:172.21.15.19:6789/0 conn(0x7f7e2c144340 legacy=0x7f7e2c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e140039e0 0x7f7e14003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e140048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e140050c0 0x7f7e1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c144750 con 0x7f7e14004370 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1400bf60 con 0x7f7e140050c0 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.044+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1400c160 con 0x7f7e140039e0 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e140048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e30a00640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e140039e0 0x7f7e14003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e31400640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e140050c0 0x7f7e1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e2be00640 1 -- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e140039e0 msgr2=0x7f7e14003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e140039e0 0x7f7e14003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e2be00640 1 -- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e140050c0 msgr2=0x7f7e1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e140050c0 0x7f7e1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.045+0000 7f7e2be00640 1 -- 172.21.15.19:0/2815677586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e14003520 con 0x7f7e14004370 2024-04-25T11:31:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.046+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e140048c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0b1490 tx=0x7f7e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9900b813186956fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18036070 con 0x7f7e14004370 2024-04-25T11:31:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e18031400 con 0x7f7e14004370 2024-04-25T11:31:18.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e2b400640 1 -- 172.21.15.19:0/2815677586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18009690 con 0x7f7e14004370 2024-04-25T11:31:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e31e00640 1 -- 172.21.15.19:0/2815677586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 msgr2=0x7f7e140048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e31e00640 1 --2- 172.21.15.19:0/2815677586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e140048c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0b1490 tx=0x7f7e14005830 comp rx=0 tx=0).stop 2024-04-25T11:31:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e31400640 1 -- 172.21.15.19:0/2815677586 reap_dead start 2024-04-25T11:31:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e31e00640 1 -- 172.21.15.19:0/2815677586 shutdown_connections 2024-04-25T11:31:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.047+0000 7f7e31e00640 1 -- 172.21.15.19:0/2815677586 >> 172.21.15.19:0/2815677586 conn(0x7f7e2c072f40 msgr2=0x7f7e2c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.048+0000 7f7e31e00640 1 -- 172.21.15.19:0/2815677586 shutdown_connections 2024-04-25T11:31:18.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.048+0000 7f7e31e00640 1 -- 172.21.15.19:0/2815677586 wait complete. 2024-04-25T11:31:18.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.050+0000 7f7e31e00640 1 Processor -- start 2024-04-25T11:31:18.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.050+0000 7f7e31e00640 1 -- start start 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e31e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e2c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e31e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c134df0 0x7f7e2c130a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e31e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c0b2740 0x7f7e2c0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e31e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c15f0a0 con 0x7f7e14004370 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e31e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c06fae0 con 0x7f7e2c0b2740 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e31e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c15f220 con 0x7f7e2c134df0 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e30a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e2c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e30a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e2c1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41062/0 (socket says 172.21.15.19:41062) 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.052+0000 7f7e30a00640 1 -- 172.21.15.19:0/2647382515 learned_addr learned my addr 172.21.15.19:0/2647382515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c134df0 0x7f7e2c130a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e31400640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c0b2740 0x7f7e2c0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e30a00640 1 -- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c134df0 msgr2=0x7f7e2c130a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e30a00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c134df0 0x7f7e2c130a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e30a00640 1 -- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c0b2740 msgr2=0x7f7e2c0b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e30a00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c0b2740 0x7f7e2c0b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e30a00640 1 -- 172.21.15.19:0/2647382515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c159c30 con 0x7f7e14004370 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e31400640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c0b2740 0x7f7e2c0b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e30a00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e2c1596f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7e20001b80 tx=0x7f7e20007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e28c00640 1 -- 172.21.15.19:0/2647382515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000f960 con 0x7f7e14004370 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e28c00640 1 -- 172.21.15.19:0/2647382515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e20018070 con 0x7f7e14004370 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e28c00640 1 -- 172.21.15.19:0/2647382515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000fd60 con 0x7f7e14004370 2024-04-25T11:31:18.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.053+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c15a3b0 con 0x7f7e14004370 2024-04-25T11:31:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.054+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c158ef0 con 0x7f7e14004370 2024-04-25T11:31:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.055+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c076c60 con 0x7f7e14004370 2024-04-25T11:31:18.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.056+0000 7f7e28c00640 1 -- 172.21.15.19:0/2647382515 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e2001d460 con 0x7f7e14004370 2024-04-25T11:31:18.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.057+0000 7f7e28c00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e0c0744b0 0x7f7e0c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.057+0000 7f7e28c00640 1 -- 172.21.15.19:0/2647382515 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f7e20095440 con 0x7f7e14004370 2024-04-25T11:31:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.058+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e0c0744b0 0x7f7e0c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.059+0000 7f7e2be00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e0c0744b0 0x7f7e0c0769c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7e18033000 tx=0x7f7e18008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:18.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.064+0000 7f7e28c00640 1 -- 172.21.15.19:0/2647382515 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e2005fb30 con 0x7f7e14004370 2024-04-25T11:31:18.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.271+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e2c0b7920 con 0x7f7e14004370 2024-04-25T11:31:18.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.273+0000 7f7e28c00640 1 -- 172.21.15.19:0/2647382515 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) ==== 104+0+11 (secure 0 0 0) 0x7f7e2005f430 con 0x7f7e14004370 2024-04-25T11:31:18.263 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e0c0744b0 msgr2=0x7f7e0c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e0c0744b0 0x7f7e0c0769c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7e18033000 tx=0x7f7e18008030 comp rx=0 tx=0).stop 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 msgr2=0x7f7e2c1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e2c1596f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7e20001b80 tx=0x7f7e20007720 comp rx=0 tx=0).stop 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 shutdown_connections 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e0c0744b0 0x7f7e0c0769c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c0b2740 0x7f7e2c0b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e2c134df0 0x7f7e2c130a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 --2- 172.21.15.19:0/2647382515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e14004370 0x7f7e2c1596f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 >> 172.21.15.19:0/2647382515 conn(0x7f7e2c072f40 msgr2=0x7f7e2c1432a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 shutdown_connections 2024-04-25T11:31:18.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.275+0000 7f7e31e00640 1 -- 172.21.15.19:0/2647382515 wait complete. 2024-04-25T11:31:18.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:31:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.588+0000 7f2185600640 1 Processor -- start 2024-04-25T11:31:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.588+0000 7f2185600640 1 -- start start 2024-04-25T11:31:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.590+0000 7f2185600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f218015cfa0 0x7f21801458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.590+0000 7f2185600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218015d5c0 con 0x7f21800ce250 2024-04-25T11:31:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.590+0000 7f2185600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218015d8a0 con 0x7f218006d5d0 2024-04-25T11:31:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.590+0000 7f2185600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f21800b4ec0 con 0x7f218015cfa0 2024-04-25T11:31:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.591+0000 7f217ea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f21800ce250 0x7f21800b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60042/0 (socket says 172.21.15.19:60042) 2024-04-25T11:31:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.591+0000 7f217ea00640 1 -- 172.21.15.19:0/1673308846 learned_addr learned my addr 172.21.15.19:0/1673308846 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.591+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3689055231 0 0) 0x7f218015d5c0 con 0x7f21800ce250 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.591+0000 7f217e000640 1 --2- 172.21.15.19:0/1673308846 >> v2:172.21.15.169:3300/0 conn(0x7f218015cfa0 0x7f21801458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.591+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2164003440 con 0x7f21800ce250 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3834223361 0 0) 0x7f2164003440 con 0x7f21800ce250 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 >> v2:172.21.15.169:3300/0 conn(0x7f218015cfa0 msgr2=0x7f21801458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 --2- 172.21.15.19:0/1673308846 >> v2:172.21.15.169:3300/0 conn(0x7f218015cfa0 0x7f21801458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 >> v1:172.21.15.169:6789/0 conn(0x7f218006d5d0 legacy=0x7f218006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218015db80 con 0x7f21800ce250 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f21700019f0 con 0x7f21800ce250 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 >> v1:172.21.15.19:6789/0 conn(0x7f21800ce250 legacy=0x7f21800b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164003ad0 0x7f2164003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164008950 0x7f216400ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216400b6d0 0x7f216400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21800b4ec0 con 0x7f216400b6d0 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f216400e4c0 con 0x7f2164008950 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f216400e6c0 con 0x7f2164003ad0 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.592+0000 7f217f400640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216400b6d0 0x7f216400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217e000640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164008950 0x7f216400ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217ea00640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164003ad0 0x7f2164003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217f400640 1 -- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164003ad0 msgr2=0x7f2164003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:18.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217f400640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164003ad0 0x7f2164003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217f400640 1 -- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164008950 msgr2=0x7f216400ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217f400640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2164008950 0x7f216400ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217f400640 1 -- 172.21.15.19:0/1673308846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2164003440 con 0x7f216400b6d0 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.593+0000 7f217ea00640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2164003ad0 0x7f2164003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.594+0000 7f217f400640 1 -- 172.21.15.19:0/1673308846 reap_dead start 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.595+0000 7f217f400640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216400b6d0 0x7f216400dbe0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2178003d80 tx=0x7f2164003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb739502d4d949d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.595+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178003980 con 0x7f216400b6d0 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.595+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2178007240 con 0x7f216400b6d0 2024-04-25T11:31:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.595+0000 7f217d600640 1 -- 172.21.15.19:0/1673308846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178007630 con 0x7f216400b6d0 2024-04-25T11:31:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.596+0000 7f2185600640 1 -- 172.21.15.19:0/1673308846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216400b6d0 msgr2=0x7f216400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.596+0000 7f2185600640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216400b6d0 0x7f216400dbe0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2178003d80 tx=0x7f2164003730 comp rx=0 tx=0).stop 2024-04-25T11:31:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.596+0000 7f2185600640 1 -- 172.21.15.19:0/1673308846 shutdown_connections 2024-04-25T11:31:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.596+0000 7f2185600640 1 --2- 172.21.15.19:0/1673308846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f216400b6d0 0x7f216400dbe0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.596+0000 7f2185600640 1 -- 172.21.15.19:0/1673308846 >> 172.21.15.19:0/1673308846 conn(0x7f2180073090 msgr2=0x7f21800734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.596+0000 7f2185600640 1 -- 172.21.15.19:0/1673308846 shutdown_connections 2024-04-25T11:31:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.596+0000 7f2185600640 1 -- 172.21.15.19:0/1673308846 wait complete. 2024-04-25T11:31:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.599+0000 7f2185600640 1 Processor -- start 2024-04-25T11:31:18.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.599+0000 7f2185600640 1 -- start start 2024-04-25T11:31:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.600+0000 7f2185600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216400b6d0 0x7f218015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.600+0000 7f2185600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218015f810 0x7f2180142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.600+0000 7f2185600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2180142560 0x7f2180142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.600+0000 7f2185600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2180071310 con 0x7f218015f810 2024-04-25T11:31:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.600+0000 7f2185600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f218015de60 con 0x7f2180142560 2024-04-25T11:31:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.600+0000 7f2185600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21801458c0 con 0x7f216400b6d0 2024-04-25T11:31:18.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218015f810 0x7f2180142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218015f810 0x7f2180142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41082/0 (socket says 172.21.15.19:41082) 2024-04-25T11:31:18.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 -- 172.21.15.19:0/959524204 learned_addr learned my addr 172.21.15.19:0/959524204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217ea00640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216400b6d0 0x7f218015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217f400640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2180142560 0x7f2180142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 -- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216400b6d0 msgr2=0x7f218015f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216400b6d0 0x7f218015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 -- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2180142560 msgr2=0x7f2180142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2180142560 0x7f2180142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217e000640 1 -- 172.21.15.19:0/959524204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218015fc20 con 0x7f218015f810 2024-04-25T11:31:18.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.601+0000 7f217f400640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2180142560 0x7f2180142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.602+0000 7f217e000640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218015f810 0x7f2180142020 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f216800b590 tx=0x7f216800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.602+0000 7f2176a00640 1 -- 172.21.15.19:0/959524204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168018070 con 0x7f218015f810 2024-04-25T11:31:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.602+0000 7f2176a00640 1 -- 172.21.15.19:0/959524204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21680106e0 con 0x7f218015f810 2024-04-25T11:31:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.602+0000 7f2176a00640 1 -- 172.21.15.19:0/959524204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168010ab0 con 0x7f218015f810 2024-04-25T11:31:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.602+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21801f6ca0 con 0x7f218015f810 2024-04-25T11:31:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.602+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21801f7260 con 0x7f218015f810 2024-04-25T11:31:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.603+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21801f6a00 con 0x7f218015f810 2024-04-25T11:31:18.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.605+0000 7f2176a00640 1 -- 172.21.15.19:0/959524204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f216801b070 con 0x7f218015f810 2024-04-25T11:31:18.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.607+0000 7f2176a00640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f215c0744b0 0x7f215c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:18.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.607+0000 7f2176a00640 1 -- 172.21.15.19:0/959524204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f2168095590 con 0x7f218015f810 2024-04-25T11:31:18.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.607+0000 7f217ea00640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f215c0744b0 0x7f215c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:18.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.609+0000 7f217ea00640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f215c0744b0 0x7f215c0769c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f21800b6c80 tx=0x7f2170001250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:18.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.613+0000 7f2176a00640 1 -- 172.21.15.19:0/959524204 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21801f6a00 con 0x7f218015f810 2024-04-25T11:31:18.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.815+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2180050740 con 0x7f218015f810 2024-04-25T11:31:18.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.816+0000 7f2176a00640 1 -- 172.21.15.19:0/959524204 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) ==== 100+0+11 (secure 0 0 0) 0x7f218015fc20 con 0x7f218015f810 2024-04-25T11:31:18.806 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:31:18.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.818+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f215c0744b0 msgr2=0x7f215c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.818+0000 7f2185600640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f215c0744b0 0x7f215c0769c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f21800b6c80 tx=0x7f2170001250 comp rx=0 tx=0).stop 2024-04-25T11:31:18.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.818+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218015f810 msgr2=0x7f2180142020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:18.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.818+0000 7f2185600640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218015f810 0x7f2180142020 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f216800b590 tx=0x7f216800ba50 comp rx=0 tx=0).stop 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 shutdown_connections 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f215c0744b0 0x7f215c0769c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2180142560 0x7f2180142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f218015f810 0x7f2180142020 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 --2- 172.21.15.19:0/959524204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f216400b6d0 0x7f218015f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 >> 172.21.15.19:0/959524204 conn(0x7f2180073090 msgr2=0x7f2180141230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 shutdown_connections 2024-04-25T11:31:18.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:18.819+0000 7f2185600640 1 -- 172.21.15.19:0/959524204 wait complete. 2024-04-25T11:31:18.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:31:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.130+0000 7f094c400640 1 Processor -- start 2024-04-25T11:31:19.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.130+0000 7f094c400640 1 -- start start 2024-04-25T11:31:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.132+0000 7f094c400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f09440cf5e0 0x7f09440cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.132+0000 7f094c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 0x7f094406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.132+0000 7f094c400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09440cff30 con 0x7f09441443c0 2024-04-25T11:31:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.132+0000 7f094c400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f09440708d0 con 0x7f09440cf5e0 2024-04-25T11:31:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.132+0000 7f094c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f094415f160 con 0x7f094406c5c0 2024-04-25T11:31:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.133+0000 7f0949a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f09441443c0 0x7f094406fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60054/0 (socket says 172.21.15.19:60054) 2024-04-25T11:31:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.133+0000 7f0949a00640 1 -- 172.21.15.19:0/4036794758 learned_addr learned my addr 172.21.15.19:0/4036794758 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.133+0000 7f094a400640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 0x7f094406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.133+0000 7f0949000640 1 --2- 172.21.15.19:0/4036794758 >> v2:172.21.15.169:3300/0 conn(0x7f09440cf5e0 0x7f09440cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.133+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917489013 0 0) 0x7f0938001110 con 0x7f09441443c0 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.133+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f092c003520 con 0x7f09441443c0 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2824312561 0 0) 0x7f092c003520 con 0x7f09441443c0 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 msgr2=0x7f094406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 0x7f094406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 >> v2:172.21.15.169:3300/0 conn(0x7f09440cf5e0 msgr2=0x7f09440cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 --2- 172.21.15.19:0/4036794758 >> v2:172.21.15.169:3300/0 conn(0x7f09440cf5e0 0x7f09440cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094415f5f0 con 0x7f09441443c0 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f0938001840 con 0x7f09441443c0 2024-04-25T11:31:19.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 >> v1:172.21.15.19:6789/0 conn(0x7f09441443c0 legacy=0x7f094406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f092c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 0x7f092c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092c004d40 0x7f092c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09440708d0 con 0x7f092c0039b0 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f094415f160 con 0x7f092c004d40 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f092c00b9c0 con 0x7f094406c5c0 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.134+0000 7f0949a00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f092c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f0949000640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 0x7f092c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f094a400640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092c004d40 0x7f092c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f0949a00640 1 -- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 msgr2=0x7f092c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f0949a00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094406c5c0 0x7f092c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f0949a00640 1 -- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092c004d40 msgr2=0x7f092c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f0949a00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092c004d40 0x7f092c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f0949a00640 1 -- 172.21.15.19:0/4036794758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c003520 con 0x7f092c0039b0 2024-04-25T11:31:19.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.135+0000 7f094a400640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f092c004d40 0x7f092c005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.136+0000 7f0949a00640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f092c003e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f093800c3b0 tx=0x7f092c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e35192ce7429b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.136+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093800c0e0 con 0x7f092c0039b0 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.136+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0938034070 con 0x7f092c0039b0 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.136+0000 7f0943e00640 1 -- 172.21.15.19:0/4036794758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938005d30 con 0x7f092c0039b0 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.137+0000 7f094c400640 1 -- 172.21.15.19:0/4036794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 msgr2=0x7f092c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.137+0000 7f094c400640 1 --2- 172.21.15.19:0/4036794758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f092c003e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f093800c3b0 tx=0x7f092c005830 comp rx=0 tx=0).stop 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.137+0000 7f094a400640 1 -- 172.21.15.19:0/4036794758 reap_dead start 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.137+0000 7f094c400640 1 -- 172.21.15.19:0/4036794758 shutdown_connections 2024-04-25T11:31:19.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.137+0000 7f094c400640 1 -- 172.21.15.19:0/4036794758 >> 172.21.15.19:0/4036794758 conn(0x7f09440d1ae0 msgr2=0x7f09440d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.137+0000 7f094c400640 1 -- 172.21.15.19:0/4036794758 shutdown_connections 2024-04-25T11:31:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.137+0000 7f094c400640 1 -- 172.21.15.19:0/4036794758 wait complete. 2024-04-25T11:31:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.140+0000 7f094c400640 1 Processor -- start 2024-04-25T11:31:19.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.140+0000 7f094c400640 1 -- start start 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.141+0000 7f094c400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f094415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.141+0000 7f094c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094415b860 0x7f09440b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.141+0000 7f094c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094415b310 0x7f09440b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.141+0000 7f094c400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f094415edf0 con 0x7f092c0039b0 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.141+0000 7f094c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f094415eb10 con 0x7f094415b860 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.141+0000 7f094c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f09441447d0 con 0x7f094415b310 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f094415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f094415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41106/0 (socket says 172.21.15.19:41106) 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 -- 172.21.15.19:0/2637276639 learned_addr learned my addr 172.21.15.19:0/2637276639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949000640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094415b860 0x7f09440b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f094a400640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094415b310 0x7f09440b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 -- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094415b310 msgr2=0x7f09440b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094415b310 0x7f09440b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 -- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094415b860 msgr2=0x7f09440b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094415b860 0x7f09440b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949a00640 1 -- 172.21.15.19:0/2637276639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09440aab20 con 0x7f092c0039b0 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.142+0000 7f0949000640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094415b860 0x7f09440b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.143+0000 7f0949a00640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f094415ad80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f093800c4d0 tx=0x7f094415f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.143+0000 7f0941600640 1 -- 172.21.15.19:0/2637276639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938007070 con 0x7f092c0039b0 2024-04-25T11:31:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.143+0000 7f0941600640 1 -- 172.21.15.19:0/2637276639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0938007210 con 0x7f092c0039b0 2024-04-25T11:31:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.143+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09441f2c80 con 0x7f092c0039b0 2024-04-25T11:31:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.143+0000 7f0941600640 1 -- 172.21.15.19:0/2637276639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380075c0 con 0x7f092c0039b0 2024-04-25T11:31:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.143+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09440ab0a0 con 0x7f092c0039b0 2024-04-25T11:31:19.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.144+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09440b6aa0 con 0x7f092c0039b0 2024-04-25T11:31:19.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.146+0000 7f0941600640 1 -- 172.21.15.19:0/2637276639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f093802f440 con 0x7f092c0039b0 2024-04-25T11:31:19.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.147+0000 7f0941600640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09240744b0 0x7f09240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.147+0000 7f0941600640 1 -- 172.21.15.19:0/2637276639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f09380b58a0 con 0x7f092c0039b0 2024-04-25T11:31:19.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.147+0000 7f0949000640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09240744b0 0x7f09240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.149+0000 7f0949000640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09240744b0 0x7f09240769c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f09300090e0 tx=0x7f0930008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.152+0000 7f0941600640 1 -- 172.21.15.19:0/2637276639 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0938049030 con 0x7f092c0039b0 2024-04-25T11:31:19.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.374+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0944050740 con 0x7f092c0039b0 2024-04-25T11:31:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.375+0000 7f0941600640 1 -- 172.21.15.19:0/2637276639 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7f0938040070 con 0x7f092c0039b0 2024-04-25T11:31:19.366 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09240744b0 msgr2=0x7f09240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09240744b0 0x7f09240769c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f09300090e0 tx=0x7f0930008040 comp rx=0 tx=0).stop 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 msgr2=0x7f094415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f094415ad80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f093800c4d0 tx=0x7f094415f7c0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 shutdown_connections 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f09240744b0 0x7f09240769c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f094415b860 0x7f09440b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f094415b310 0x7f09440b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 --2- 172.21.15.19:0/2637276639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f092c0039b0 0x7f094415ad80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 >> 172.21.15.19:0/2637276639 conn(0x7f09440d1ae0 msgr2=0x7f094406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.378+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 shutdown_connections 2024-04-25T11:31:19.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.379+0000 7f094c400640 1 -- 172.21.15.19:0/2637276639 wait complete. 2024-04-25T11:31:19.398 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:31:19.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:19.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:19.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:31:19.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:19.489 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:31:19.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-04-25T11:31:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.823+0000 7f126dc00640 1 Processor -- start 2024-04-25T11:31:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.823+0000 7f126dc00640 1 -- start start 2024-04-25T11:31:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.825+0000 7f126dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1268155350 0x7f126815f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.825+0000 7f126dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126806dce0 0x7f126815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.825+0000 7f126dc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1268162650 con 0x7f1268161ce0 2024-04-25T11:31:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.825+0000 7f126dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1268162c10 con 0x7f1268155350 2024-04-25T11:31:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.825+0000 7f126dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1268162930 con 0x7f126806dce0 2024-04-25T11:31:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.826+0000 7f1266000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1268161ce0 0x7f126806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60062/0 (socket says 172.21.15.19:60062) 2024-04-25T11:31:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.826+0000 7f1266000640 1 -- 172.21.15.19:0/2669413964 learned_addr learned my addr 172.21.15.19:0/2669413964 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.826+0000 7f1267400640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126806dce0 0x7f126815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.826+0000 7f1266a00640 1 --2- 172.21.15.19:0/2669413964 >> v2:172.21.15.169:3300/0 conn(0x7f1268155350 0x7f126815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.826+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2582785309 0 0) 0x7f1268162650 con 0x7f1268161ce0 2024-04-25T11:31:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.826+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f124c003520 con 0x7f1268161ce0 2024-04-25T11:31:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1395251685 0 0) 0x7f124c003520 con 0x7f1268161ce0 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126806dce0 msgr2=0x7f126815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f126806dce0 0x7f126815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 >> v2:172.21.15.169:3300/0 conn(0x7f1268155350 msgr2=0x7f126815f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 --2- 172.21.15.19:0/2669413964 >> v2:172.21.15.169:3300/0 conn(0x7f1268155350 0x7f126815f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126815e890 con 0x7f1268161ce0 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1254001670 con 0x7f1268161ce0 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 >> v1:172.21.15.19:6789/0 conn(0x7f1268161ce0 legacy=0x7f126806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c0039b0 0x7f124c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f124c004800 0x7f124c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f124c005150 0x7f124c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1268162c10 con 0x7f124c004800 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1268162930 con 0x7f124c0039b0 2024-04-25T11:31:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f124c00bf00 con 0x7f124c005150 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.827+0000 7f1266000640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f124c004800 0x7f124c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1266a00640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c0039b0 0x7f124c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1267400640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f124c005150 0x7f124c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1266000640 1 -- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f124c005150 msgr2=0x7f124c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1266000640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f124c005150 0x7f124c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1266000640 1 -- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c0039b0 msgr2=0x7f124c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1266000640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c0039b0 0x7f124c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1266000640 1 -- 172.21.15.19:0/2669413964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c003520 con 0x7f124c004800 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1267400640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f124c005150 0x7f124c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.828+0000 7f1266a00640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c0039b0 0x7f124c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.829+0000 7f1266000640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f124c004800 0x7f124c004c10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f125400c160 tx=0x7f124c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fee8b6b8c4d5619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.829+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254007c80 con 0x7f124c004800 2024-04-25T11:31:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.829+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1254005ad0 con 0x7f124c004800 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.829+0000 7f1265600640 1 -- 172.21.15.19:0/2669413964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254007070 con 0x7f124c004800 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.830+0000 7f126dc00640 1 -- 172.21.15.19:0/2669413964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f124c004800 msgr2=0x7f124c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.830+0000 7f126dc00640 1 --2- 172.21.15.19:0/2669413964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f124c004800 0x7f124c004c10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f125400c160 tx=0x7f124c005830 comp rx=0 tx=0).stop 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.830+0000 7f1267400640 1 -- 172.21.15.19:0/2669413964 reap_dead start 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.830+0000 7f126dc00640 1 -- 172.21.15.19:0/2669413964 shutdown_connections 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.830+0000 7f126dc00640 1 -- 172.21.15.19:0/2669413964 >> 172.21.15.19:0/2669413964 conn(0x7f1268073120 msgr2=0x7f1268073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.830+0000 7f126dc00640 1 -- 172.21.15.19:0/2669413964 shutdown_connections 2024-04-25T11:31:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.830+0000 7f126dc00640 1 -- 172.21.15.19:0/2669413964 wait complete. 2024-04-25T11:31:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.833+0000 7f126dc00640 1 Processor -- start 2024-04-25T11:31:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.833+0000 7f126dc00640 1 -- start start 2024-04-25T11:31:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.834+0000 7f126dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c004800 0x7f12681f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.834+0000 7f126dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12681f6950 0x7f12681f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.834+0000 7f126dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12681603b0 0x7f12681607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.834+0000 7f126dc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12680707b0 con 0x7f12681603b0 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.834+0000 7f126dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f126815f5d0 con 0x7f124c004800 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.834+0000 7f126dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1268070930 con 0x7f12681f6950 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12681603b0 0x7f12681607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12681603b0 0x7f12681607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41124/0 (socket says 172.21.15.19:41124) 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 -- 172.21.15.19:0/2733482467 learned_addr learned my addr 172.21.15.19:0/2733482467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1266a00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c004800 0x7f12681f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1266000640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12681f6950 0x7f12681f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 -- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12681f6950 msgr2=0x7f12681f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12681f6950 0x7f12681f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 -- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c004800 msgr2=0x7f12681f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c004800 0x7f12681f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1267400640 1 -- 172.21.15.19:0/2733482467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681f72a0 con 0x7f12681603b0 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.835+0000 7f1266a00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c004800 0x7f12681f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.836+0000 7f1267400640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12681603b0 0x7f12681607c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1268158d80 tx=0x7f126000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.836+0000 7f125ea00640 1 -- 172.21.15.19:0/2733482467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126001c070 con 0x7f12681603b0 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.836+0000 7f125ea00640 1 -- 172.21.15.19:0/2733482467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1260017400 con 0x7f12681603b0 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.836+0000 7f125ea00640 1 -- 172.21.15.19:0/2733482467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12600177f0 con 0x7f12681603b0 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.836+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1268160d00 con 0x7f12681603b0 2024-04-25T11:31:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.836+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1268156e10 con 0x7f12681603b0 2024-04-25T11:31:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.839+0000 7f125ea00640 1 -- 172.21.15.19:0/2733482467 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1260005d10 con 0x7f12681603b0 2024-04-25T11:31:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.840+0000 7f125ea00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12480744b0 0x7f12480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.840+0000 7f125ea00640 1 -- 172.21.15.19:0/2733482467 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f12600953b0 con 0x7f12681603b0 2024-04-25T11:31:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.841+0000 7f1274c46740 1 Processor -- start 2024-04-25T11:31:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.841+0000 7f1266a00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12480744b0 0x7f12480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.841+0000 7f1274c46740 1 -- start start 2024-04-25T11:31:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.842+0000 7f1266a00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12480744b0 0x7f12480769c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1258006fd0 tx=0x7f1258008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1274c46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b20e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b20bb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1274c46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639b1fcb030 con 0x5639b1d18390 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639b1fe67f0 con 0x5639b1cef4d0 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5639b1fa17e0 con 0x5639b1cf76b0 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b20e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b20e55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41126/0 (socket says 172.21.15.19:41126) 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 -- 172.21.15.19:0/758903229 learned_addr learned my addr 172.21.15.19:0/758903229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1267400640 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b20bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266000640 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 -- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 msgr2=0x5639b20bb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b20bb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 -- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 msgr2=0x5639b20e1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20e1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.843+0000 7f1266a00640 1 -- 172.21.15.19:0/758903229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b1fad050 con 0x5639b1d18390 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.844+0000 7f1266000640 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20e1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.844+0000 7f1266a00640 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b20e55f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1258010a60 tx=0x7f1258013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5d4c4f3b0623d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.844+0000 7f1253400640 1 -- 172.21.15.19:0/758903229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125801c050 con 0x5639b1d18390 2024-04-25T11:31:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.844+0000 7f1253400640 1 -- 172.21.15.19:0/758903229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1258026070 con 0x5639b1d18390 2024-04-25T11:31:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.844+0000 7f1253400640 1 -- 172.21.15.19:0/758903229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125800f040 con 0x5639b1d18390 2024-04-25T11:31:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 -- 172.21.15.19:0/758903229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 msgr2=0x5639b20e55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b20e55f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1258010a60 tx=0x7f1258013000 comp rx=0 tx=0).stop 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 -- 172.21.15.19:0/758903229 shutdown_connections 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20e1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b20bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 --2- 172.21.15.19:0/758903229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b20e55f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 -- 172.21.15.19:0/758903229 >> 172.21.15.19:0/758903229 conn(0x5639b1d0c4e0 msgr2=0x5639b20ba9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 -- 172.21.15.19:0/758903229 shutdown_connections 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.845+0000 7f1274c46740 1 -- 172.21.15.19:0/758903229 wait complete. 2024-04-25T11:31:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.846+0000 7f1274c46740 1 -- start start 2024-04-25T11:31:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.847+0000 7f1274c46740 1 Processor -- start 2024-04-25T11:31:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.848+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.848+0000 7f1274c46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cf76b0 0x5639b20fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.848+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 0x5639b20e7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.848+0000 7f1274c46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639b2023200 con 0x5639b1cf76b0 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.848+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5639b201dee0 con 0x5639b1cef4d0 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.848+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5639b202ae70 con 0x5639b1d18390 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.848+0000 7f1274c46740 10 client.0 _ll_register_callbacks cb 0x5639b2014270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cf76b0 0x5639b20fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cf76b0 0x5639b20fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41134/0 (socket says 172.21.15.19:41134) 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 -- 172.21.15.19:0/3113491496 learned_addr learned my addr 172.21.15.19:0/3113491496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266a00640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 -- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 msgr2=0x5639b20e7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1267400640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 0x5639b20e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 0x5639b20e7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 -- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 msgr2=0x5639b20de4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20de4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266000640 1 -- 172.21.15.19:0/3113491496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b1dfa7d0 con 0x5639b1cf76b0 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.849+0000 7f1266a00640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1266000640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cf76b0 0x5639b20fe680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f125400c160 tx=0x7f1254001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a55a1bcae8b5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1247e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1247e00640 1 -- 172.21.15.19:0/3113491496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254043020 con 0x5639b1cf76b0 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1247e00640 1 -- 172.21.15.19:0/3113491496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f1254001640 con 0x5639b1cf76b0 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258026070 con 0x5639b1cf76b0 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1247e00640 1 -- 172.21.15.19:0/3113491496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f12540047e0 con 0x5639b1cf76b0 2024-04-25T11:31:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1274c46740 10 client.6429 fetch_fsmap learned FSMap version 51 2024-04-25T11:31:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5639b1d303f0 con 0x5639b1cf76b0 2024-04-25T11:31:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.850+0000 7f1247e00640 1 -- 172.21.15.19:0/3113491496 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1258026070 con 0x5639b1cf76b0 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1247e00640 1 -- 172.21.15.19:0/3113491496 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f1254004980 con 0x5639b1cf76b0 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1274c46740 10 client.6429 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1274c46740 10 client.6429 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1274c46740 20 client.6429 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1274c46740 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b1cd6f60 0x5639b2112390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1274c46740 4 client.6429 mds_command: new command op to 6231 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 --> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639b202c7e0 con 0x5639b1cd6f60 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.851+0000 7f1266a00640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b1cd6f60 0x5639b2112390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.852+0000 7f1266a00640 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b1cd6f60 0x5639b2112390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4657ff9f5d7ea6 server_cookie=d18d3abd57e55ab7 in_seq=0 out_seq=0 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.852+0000 7f1247e00640 10 client.6429 ms_handle_connect on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.852+0000 7f1247e00640 1 -- 172.21.15.19:0/3113491496 <== mds.0 v2:172.21.15.19:6832/751039234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639b202c7e0 con 0x5639b1cd6f60 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.852+0000 7f1247e00640 10 client.6429 handle_command_reply: tid=0 2024-04-25T11:31:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 client.6429 shutdown 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cf76b0 msgr2=0x5639b20fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cf76b0 0x5639b20fe680 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f125400c160 tx=0x7f1254001970 comp rx=0 tx=0).stop 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 shutdown_connections 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b1cd6f60 0x5639b2112390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 0x5639b20e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cf76b0 0x5639b20fe680 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 --2- 172.21.15.19:0/3113491496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b20de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1247e00640 0 client.6429 ms_handle_reset on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 >> 172.21.15.19:0/3113491496 conn(0x5639b1d0c4e0 msgr2=0x5639b20b9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 shutdown_connections 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 1 -- 172.21.15.19:0/3113491496 wait complete. 2024-04-25T11:31:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.853+0000 7f1274c46740 20 client.6429 trim_cache size 0 max 16384 2024-04-25T11:31:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.870+0000 7f1274c46740 1 Processor -- start 2024-04-25T11:31:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.871+0000 7f1274c46740 1 -- start start 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1274c46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cef4d0 0x5639b2122ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cf76b0 0x5639b20e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 0x5639b21126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1274c46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12682030a0 con 0x5639b1cef4d0 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1268202080 con 0x5639b1cf76b0 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12681586b0 con 0x5639b1d18390 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1266a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cef4d0 0x5639b2122ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1266a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cef4d0 0x5639b2122ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41148/0 (socket says 172.21.15.19:41148) 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.872+0000 7f1266a00640 1 -- 172.21.15.19:0/1926058450 learned_addr learned my addr 172.21.15.19:0/1926058450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1266a00640 1 -- 172.21.15.19:0/1926058450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 msgr2=0x5639b21126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1266a00640 1 --2- 172.21.15.19:0/1926058450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 0x5639b21126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1266a00640 1 -- 172.21.15.19:0/1926058450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cf76b0 msgr2=0x5639b20e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1266a00640 1 --2- 172.21.15.19:0/1926058450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cf76b0 0x5639b20e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1266a00640 1 -- 172.21.15.19:0/1926058450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b1cf00f0 con 0x5639b1cef4d0 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1266a00640 1 --2- 172.21.15.19:0/1926058450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cef4d0 0x5639b2122ce0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x5639b20f7170 tx=0x7f1258015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e104e6eb0d98b9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1252a00640 1 -- 172.21.15.19:0/1926058450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258039070 con 0x5639b1cef4d0 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1252a00640 1 -- 172.21.15.19:0/1926058450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1258026040 con 0x5639b1cef4d0 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.873+0000 7f1252a00640 1 -- 172.21.15.19:0/1926058450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125801c040 con 0x5639b1cef4d0 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 -- 172.21.15.19:0/1926058450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cef4d0 msgr2=0x5639b2122ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 --2- 172.21.15.19:0/1926058450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cef4d0 0x5639b2122ce0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x5639b20f7170 tx=0x7f1258015000 comp rx=0 tx=0).stop 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 -- 172.21.15.19:0/1926058450 shutdown_connections 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 --2- 172.21.15.19:0/1926058450 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cf76b0 0x5639b20e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 --2- 172.21.15.19:0/1926058450 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1d18390 0x5639b21126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 --2- 172.21.15.19:0/1926058450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1cef4d0 0x5639b2122ce0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 -- 172.21.15.19:0/1926058450 >> 172.21.15.19:0/1926058450 conn(0x5639b1d0c4e0 msgr2=0x5639b20e8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.874+0000 7f1274c46740 1 -- 172.21.15.19:0/1926058450 shutdown_connections 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.875+0000 7f1274c46740 1 -- 172.21.15.19:0/1926058450 wait complete. 2024-04-25T11:31:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.875+0000 7f1274c46740 1 -- start start 2024-04-25T11:31:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.876+0000 7f1274c46740 1 Processor -- start 2024-04-25T11:31:19.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.877+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b2160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.877+0000 7f1274c46740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b21126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.877+0000 7f1274c46740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b2112c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.877+0000 7f1274c46740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639b202bfd0 con 0x5639b1d18390 2024-04-25T11:31:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.877+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12681563c0 con 0x5639b1cef4d0 2024-04-25T11:31:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.877+0000 7f1274c46740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5639b217fb80 con 0x5639b1cf76b0 2024-04-25T11:31:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.877+0000 7f1274c46740 10 client.0 _ll_register_callbacks cb 0x7f124c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b2112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b2112c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41152/0 (socket says 172.21.15.19:41152) 2024-04-25T11:31:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 -- 172.21.15.19:0/2362751255 learned_addr learned my addr 172.21.15.19:0/2362751255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1266a00640 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b2160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1266000640 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b21126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 -- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 msgr2=0x5639b21126f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b21126f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 -- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 msgr2=0x5639b2160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b2160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 -- 172.21.15.19:0/2362751255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b1fe6a30 con 0x5639b1d18390 2024-04-25T11:31:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1267400640 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b2112c30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f12600937e0 tx=0x7f126005fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c0a63056be5393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.878+0000 7f1247e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1247e00640 1 -- 172.21.15.19:0/2362751255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260067f70 con 0x5639b1d18390 2024-04-25T11:31:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1247e00640 1 -- 172.21.15.19:0/2362751255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f126005e690 con 0x5639b1d18390 2024-04-25T11:31:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258026040 con 0x5639b1d18390 2024-04-25T11:31:19.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1247e00640 1 -- 172.21.15.19:0/2362751255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f126005fb30 con 0x5639b1d18390 2024-04-25T11:31:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 10 client.6435 fetch_fsmap learned FSMap version 51 2024-04-25T11:31:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5639b2164e40 con 0x5639b1d18390 2024-04-25T11:31:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1247e00640 1 -- 172.21.15.19:0/2362751255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1258026040 con 0x5639b1d18390 2024-04-25T11:31:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1247e00640 1 -- 172.21.15.19:0/2362751255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f1260078b90 con 0x5639b1d18390 2024-04-25T11:31:19.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 10 client.6435 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T11:31:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 10 client.6435 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T11:31:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 20 client.6435 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b217f720 0x5639b21616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 4 client.6435 mds_command: new command op to 6231 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T11:31:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.879+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 --> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639b21045e0 con 0x5639b217f720 2024-04-25T11:31:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.880+0000 7f1266a00640 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b217f720 0x5639b21616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.880+0000 7f1266a00640 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b217f720 0x5639b21616e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84369eac2b2a921 server_cookie=d3919bce2ad4dd0d in_seq=0 out_seq=0 2024-04-25T11:31:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.880+0000 7f1247e00640 10 client.6435 ms_handle_connect on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.900+0000 7f1247e00640 1 -- 172.21.15.19:0/2362751255 <== mds.0 v2:172.21.15.19:6832/751039234 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639b21045e0 con 0x5639b217f720 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.900+0000 7f1247e00640 10 client.6435 handle_command_reply: tid=0 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.900+0000 7f1274c46740 1 client.6435 shutdown 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 msgr2=0x5639b2112c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b2112c30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f12600937e0 tx=0x7f126005fa50 comp rx=0 tx=0).stop 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 shutdown_connections 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:6832/751039234,v1:172.21.15.19:6833/751039234] conn(0x5639b217f720 0x5639b21616e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b1d18390 0x5639b2112c30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5639b1cf76b0 0x5639b21126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 --2- 172.21.15.19:0/2362751255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5639b1cef4d0 0x5639b2160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1247e00640 0 client.6435 ms_handle_reset on v2:172.21.15.19:6832/751039234 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 >> 172.21.15.19:0/2362751255 conn(0x5639b1d0c4e0 msgr2=0x5639b20e4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 shutdown_connections 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 1 -- 172.21.15.19:0/2362751255 wait complete. 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.901+0000 7f1274c46740 20 client.6435 trim_cache size 0 max 16384 2024-04-25T11:31:19.893 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:31:19.894 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-04-25T11:31:19.894 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T11:31:19.894 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12480744b0 msgr2=0x7f12480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12480744b0 0x7f12480769c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1258006fd0 tx=0x7f1258008040 comp rx=0 tx=0).stop 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12681603b0 msgr2=0x7f12681607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12681603b0 0x7f12681607c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1268158d80 tx=0x7f126000f4a0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 shutdown_connections 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12480744b0 0x7f12480769c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12681603b0 0x7f12681607c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12681f6950 0x7f12681f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 --2- 172.21.15.19:0/2733482467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f124c004800 0x7f12681f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 >> 172.21.15.19:0/2733482467 conn(0x7f1268073120 msgr2=0x7f126815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 shutdown_connections 2024-04-25T11:31:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:19.902+0000 7f126dc00640 1 -- 172.21.15.19:0/2733482467 wait complete. 2024-04-25T11:31:19.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:31:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.196+0000 7fc2b1e00640 1 Processor -- start 2024-04-25T11:31:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.196+0000 7fc2b1e00640 1 -- start start 2024-04-25T11:31:20.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.198+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac0b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:20.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.198+0000 7fc2b1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2ac163cd0 0x7fc2ac06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.199+0000 7fc2b1e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac164640 con 0x7fc2ac1640e0 2024-04-25T11:31:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.199+0000 7fc2b1e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac160c60 con 0x7fc2ac163cd0 2024-04-25T11:31:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.199+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac160690 con 0x7fc2ac1610f0 2024-04-25T11:31:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.199+0000 7fc2abe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2ac163cd0 0x7fc2ac06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.199+0000 7fc2abe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc2ac163cd0 0x7fc2ac06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41168/0 (socket says 172.21.15.19:41168) 2024-04-25T11:31:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.199+0000 7fc2abe00640 1 -- 172.21.15.19:0/2188440846 learned_addr learned my addr 172.21.15.19:0/2188440846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.199+0000 7fc2b0a00640 1 --2- 172.21.15.19:0/2188440846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac0b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.200+0000 7fc2abe00640 1 -- 172.21.15.19:0/2188440846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 msgr2=0x7fc2ac0b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.200+0000 7fc2abe00640 1 --2- 172.21.15.19:0/2188440846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac0b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.200+0000 7fc2abe00640 1 -- 172.21.15.19:0/2188440846 >> v1:172.21.15.169:6789/0 conn(0x7fc2ac1640e0 legacy=0x7fc2ac1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.200+0000 7fc2abe00640 1 -- 172.21.15.19:0/2188440846 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac164920 con 0x7fc2ac163cd0 2024-04-25T11:31:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.201+0000 7fc2abe00640 1 --2- 172.21.15.19:0/2188440846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac163cd0 0x7fc2ac06d5d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009e50 tx=0x7fc29c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4241487b0ef72f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.201+0000 7fc2ab400640 1 -- 172.21.15.19:0/2188440846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c036070 con 0x7fc2ac163cd0 2024-04-25T11:31:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.201+0000 7fc2ab400640 1 -- 172.21.15.19:0/2188440846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29c031600 con 0x7fc2ac163cd0 2024-04-25T11:31:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.201+0000 7fc2ab400640 1 -- 172.21.15.19:0/2188440846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0319d0 con 0x7fc2ac163cd0 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2188440846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac163cd0 msgr2=0x7fc2ac06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2188440846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac163cd0 0x7fc2ac06d5d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009e50 tx=0x7fc29c02a730 comp rx=0 tx=0).stop 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2188440846 shutdown_connections 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2188440846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac163cd0 0x7fc2ac06d5d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2188440846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac0b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2188440846 >> 172.21.15.19:0/2188440846 conn(0x7fc2ac072ee0 msgr2=0x7fc2ac0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2188440846 shutdown_connections 2024-04-25T11:31:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.202+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2188440846 wait complete. 2024-04-25T11:31:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.205+0000 7fc2b1e00640 1 Processor -- start 2024-04-25T11:31:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.205+0000 7fc2b1e00640 1 -- start start 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.206+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.206+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ac163cd0 0x7fc2ac14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac1640e0 0x7fc2ac1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac161500 con 0x7fc2ac1640e0 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0b0d70 con 0x7fc2ac163cd0 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac153d40 con 0x7fc2ac1610f0 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac1640e0 0x7fc2ac1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac1640e0 0x7fc2ac1550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41182/0 (socket says 172.21.15.19:41182) 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 -- 172.21.15.19:0/2079036018 learned_addr learned my addr 172.21.15.19:0/2079036018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b0a00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2abe00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ac163cd0 0x7fc2ac14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 -- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 msgr2=0x7fc2ac1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 -- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ac163cd0 msgr2=0x7fc2ac14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ac163cd0 0x7fc2ac14ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.207+0000 7fc2b1400640 1 -- 172.21.15.19:0/2079036018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac156a20 con 0x7fc2ac1640e0 2024-04-25T11:31:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.208+0000 7fc2b1400640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac1640e0 0x7fc2ac1550f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400c390 tx=0x7fc2a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.208+0000 7fc2a8c00640 1 -- 172.21.15.19:0/2079036018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4017070 con 0x7fc2ac1640e0 2024-04-25T11:31:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.208+0000 7fc2a8c00640 1 -- 172.21.15.19:0/2079036018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a40052d0 con 0x7fc2ac1640e0 2024-04-25T11:31:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.208+0000 7fc2a8c00640 1 -- 172.21.15.19:0/2079036018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4003810 con 0x7fc2ac1640e0 2024-04-25T11:31:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.208+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac156d00 con 0x7fc2ac1640e0 2024-04-25T11:31:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.208+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1572c0 con 0x7fc2ac1640e0 2024-04-25T11:31:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.209+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac156790 con 0x7fc2ac1640e0 2024-04-25T11:31:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.211+0000 7fc2a8c00640 1 -- 172.21.15.19:0/2079036018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc2a401a070 con 0x7fc2ac1640e0 2024-04-25T11:31:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.213+0000 7fc2a8c00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2880744e0 0x7fc2880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.213+0000 7fc2a8c00640 1 -- 172.21.15.19:0/2079036018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fc2a40945d0 con 0x7fc2ac1640e0 2024-04-25T11:31:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.213+0000 7fc2b0a00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2880744e0 0x7fc2880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:20.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.215+0000 7fc2b0a00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2880744e0 0x7fc2880769f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc294006fd0 tx=0x7fc294008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.218+0000 7fc2a8c00640 1 -- 172.21.15.19:0/2079036018 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2a405ecc0 con 0x7fc2ac1640e0 2024-04-25T11:31:20.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.464+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2ac050740 con 0x7fc2ac1640e0 2024-04-25T11:31:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.465+0000 7fc2a8c00640 1 -- 172.21.15.19:0/2079036018 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7fc2a405e5c0 con 0x7fc2ac1640e0 2024-04-25T11:31:20.456 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:31:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.467+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2880744e0 msgr2=0x7fc2880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.467+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2880744e0 0x7fc2880769f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc294006fd0 tx=0x7fc294008040 comp rx=0 tx=0).stop 2024-04-25T11:31:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.467+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac1640e0 msgr2=0x7fc2ac1550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.467+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac1640e0 0x7fc2ac1550f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400c390 tx=0x7fc2a400c850 comp rx=0 tx=0).stop 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 shutdown_connections 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2880744e0 0x7fc2880769f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2ac163cd0 0x7fc2ac14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2ac1640e0 0x7fc2ac1550f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 --2- 172.21.15.19:0/2079036018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2ac1610f0 0x7fc2ac1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 >> 172.21.15.19:0/2079036018 conn(0x7fc2ac072ee0 msgr2=0x7fc2ac0af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 shutdown_connections 2024-04-25T11:31:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20.468+0000 7fc2b1e00640 1 -- 172.21.15.19:0/2079036018 wait complete. 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:31:20.485 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:31:20.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:20.486 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:31:20.540 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:31:20.540 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:31:20.540 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:31:20.541 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42393sec preferred_lft 42393sec 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:31:20.542 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:31:20.543 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:20.543 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:31:20.543 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:31:20.543 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:31:20.543 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:31:20.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:20.544 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:31:20.624 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:31:20.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:20.628 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:31:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:31:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:31:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:31:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:31:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:31:20.684 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:31:20.684 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:31:20.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:31:20.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:31:20.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:31:20.684 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:31:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:31:20.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:31:20.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:20.907 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:31:20.955 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:31:20.955 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:31:20.956 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:31:20.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:20.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:31:21.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:31:21.037 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:21.091 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:31:21.091 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:31:21.091 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:21.143 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:31:21.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:21.143 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:31:21.352 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:31:21.353 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:31:21.394 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:31:21.394 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:31:21.394 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:31:21.394 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:31:21.394 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:31:21.396 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:31:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:31:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:31:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:31:21.396 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:31:21.401 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:31:21.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:21.401 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:21.471 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:31:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:31:21.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:31:21.599 INFO:teuthology.orchestra.run.smithi019.stdout:client5581 2024-04-25T11:31:21.599 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:31:21.599 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:31:21.652 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 711 a/b/d; sync .') 2024-04-25T11:31:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:31:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:31:21.783 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:31:21.783 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:31:21.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:31:22.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.122+0000 7f6932000640 1 Processor -- start 2024-04-25T11:31:22.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.122+0000 7f6932000640 1 -- start start 2024-04-25T11:31:22.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.124+0000 7f6932000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f692c06c5c0 0x7f692c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.124+0000 7f6932000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f692c0d1230 con 0x7f692c144370 2024-04-25T11:31:22.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.124+0000 7f6932000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f692c0d06b0 con 0x7f692c0b4ea0 2024-04-25T11:31:22.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.124+0000 7f6932000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f692c0d0990 con 0x7f692c06c5c0 2024-04-25T11:31:22.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.125+0000 7f692be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f692c144370 0x7f692c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60070/0 (socket says 172.21.15.19:60070) 2024-04-25T11:31:22.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.125+0000 7f692be00640 1 -- 172.21.15.19:0/2545896134 learned_addr learned my addr 172.21.15.19:0/2545896134 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.125+0000 7f6930c00640 1 --2- 172.21.15.19:0/2545896134 >> v2:172.21.15.169:3300/0 conn(0x7f692c06c5c0 0x7f692c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.125+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660328154 0 0) 0x7f692c0d1230 con 0x7f692c144370 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.125+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6910003520 con 0x7f692c144370 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3029907803 0 0) 0x7f6910003520 con 0x7f692c144370 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 >> v2:172.21.15.169:3300/0 conn(0x7f692c06c5c0 msgr2=0x7f692c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 --2- 172.21.15.19:0/2545896134 >> v2:172.21.15.169:3300/0 conn(0x7f692c06c5c0 0x7f692c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 >> v1:172.21.15.169:6789/0 conn(0x7f692c0b4ea0 legacy=0x7f692c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c0d0c70 con 0x7f692c144370 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f6914001670 con 0x7f692c144370 2024-04-25T11:31:22.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 >> v1:172.21.15.19:6789/0 conn(0x7f692c144370 legacy=0x7f692c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69100039b0 0x7f6910003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6910004800 0x7f6910004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69100051c0 0x7f691000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0d0990 con 0x7f6910004800 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f691000c160 con 0x7f69100051c0 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f691000c360 con 0x7f69100039b0 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.126+0000 7f692be00640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6910004800 0x7f6910004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f6930c00640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69100039b0 0x7f6910003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f6931600640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69100051c0 0x7f691000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f692be00640 1 -- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69100039b0 msgr2=0x7f6910003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:22.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f692be00640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69100039b0 0x7f6910003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f692be00640 1 -- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69100051c0 msgr2=0x7f691000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:22.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f692be00640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69100051c0 0x7f691000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f692be00640 1 -- 172.21.15.19:0/2545896134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6910003520 con 0x7f6910004800 2024-04-25T11:31:22.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.127+0000 7f6931600640 1 -- 172.21.15.19:0/2545896134 reap_dead start 2024-04-25T11:31:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.128+0000 7f692be00640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6910004800 0x7f6910004c10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f691400c240 tx=0x7f6910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f59e3db6f978e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.128+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914007d10 con 0x7f6910004800 2024-04-25T11:31:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.128+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914005ad0 con 0x7f6910004800 2024-04-25T11:31:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.128+0000 7f692b400640 1 -- 172.21.15.19:0/2545896134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914007070 con 0x7f6910004800 2024-04-25T11:31:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.129+0000 7f6932000640 1 -- 172.21.15.19:0/2545896134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6910004800 msgr2=0x7f6910004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.129+0000 7f6932000640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6910004800 0x7f6910004c10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f691400c240 tx=0x7f6910005830 comp rx=0 tx=0).stop 2024-04-25T11:31:22.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.129+0000 7f6932000640 1 -- 172.21.15.19:0/2545896134 shutdown_connections 2024-04-25T11:31:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.129+0000 7f6932000640 1 --2- 172.21.15.19:0/2545896134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6910004800 0x7f6910004c10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.129+0000 7f6932000640 1 -- 172.21.15.19:0/2545896134 >> 172.21.15.19:0/2545896134 conn(0x7f692c0d1720 msgr2=0x7f692c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.129+0000 7f6932000640 1 -- 172.21.15.19:0/2545896134 shutdown_connections 2024-04-25T11:31:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.129+0000 7f6932000640 1 -- 172.21.15.19:0/2545896134 wait complete. 2024-04-25T11:31:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.132+0000 7f6932000640 1 Processor -- start 2024-04-25T11:31:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.132+0000 7f6932000640 1 -- start start 2024-04-25T11:31:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.133+0000 7f6932000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6910004800 0x7f692c1f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.133+0000 7f6932000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f692c0cab10 0x7f692c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.133+0000 7f6932000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692c141d60 0x7f692c142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.133+0000 7f6932000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0d0450 con 0x7f692c0cab10 2024-04-25T11:31:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.133+0000 7f6932000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0d02d0 con 0x7f692c141d60 2024-04-25T11:31:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.133+0000 7f6932000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f692c147760 con 0x7f6910004800 2024-04-25T11:31:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.133+0000 7f692be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f692c0cab10 0x7f692c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f692c0cab10 0x7f692c0caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41202/0 (socket says 172.21.15.19:41202) 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 -- 172.21.15.19:0/2206936696 learned_addr learned my addr 172.21.15.19:0/2206936696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f6930c00640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6910004800 0x7f692c1f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f6931600640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692c141d60 0x7f692c142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 -- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6910004800 msgr2=0x7f692c1f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6910004800 0x7f692c1f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 -- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692c141d60 msgr2=0x7f692c142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692c141d60 0x7f692c142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 -- 172.21.15.19:0/2206936696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1426b0 con 0x7f692c0cab10 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.134+0000 7f692be00640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f692c0cab10 0x7f692c0caf20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f691400c240 tx=0x7f692c0d0880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.135+0000 7f6928c00640 1 -- 172.21.15.19:0/2206936696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914041070 con 0x7f692c0cab10 2024-04-25T11:31:22.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.135+0000 7f6928c00640 1 -- 172.21.15.19:0/2206936696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914007070 con 0x7f692c0cab10 2024-04-25T11:31:22.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.135+0000 7f6928c00640 1 -- 172.21.15.19:0/2206936696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914007d10 con 0x7f692c0cab10 2024-04-25T11:31:22.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.135+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c1f5f60 con 0x7f692c0cab10 2024-04-25T11:31:22.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.135+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c0cba90 con 0x7f692c0cab10 2024-04-25T11:31:22.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.136+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692c0d2060 con 0x7f692c0cab10 2024-04-25T11:31:22.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.137+0000 7f6928c00640 1 -- 172.21.15.19:0/2206936696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6914044070 con 0x7f692c0cab10 2024-04-25T11:31:22.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.139+0000 7f6928c00640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69080744b0 0x7f69080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.139+0000 7f6928c00640 1 -- 172.21.15.19:0/2206936696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f69140b5630 con 0x7f692c0cab10 2024-04-25T11:31:22.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.140+0000 7f6930c00640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69080744b0 0x7f69080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.141+0000 7f6930c00640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69080744b0 0x7f69080769c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f691c007380 tx=0x7f691c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:22.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.144+0000 7f6928c00640 1 -- 172.21.15.19:0/2206936696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f692c0d2060 con 0x7f692c0cab10 2024-04-25T11:31:22.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.352+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f692c050740 con 0x7f692c0cab10 2024-04-25T11:31:22.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.354+0000 7f6928c00640 1 -- 172.21.15.19:0/2206936696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16665 (secure 0 0 0) 0x7f692c1426b0 con 0x7f692c0cab10 2024-04-25T11:31:22.345 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:22.345 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:31:08.278038+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:31:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69080744b0 msgr2=0x7f69080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69080744b0 0x7f69080769c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f691c007380 tx=0x7f691c00b040 comp rx=0 tx=0).stop 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f692c0cab10 msgr2=0x7f692c0caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f692c0cab10 0x7f692c0caf20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f691400c240 tx=0x7f692c0d0880 comp rx=0 tx=0).stop 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 shutdown_connections 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f69080744b0 0x7f69080769c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f692c141d60 0x7f692c142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f692c0cab10 0x7f692c0caf20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 --2- 172.21.15.19:0/2206936696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6910004800 0x7f692c1f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 >> 172.21.15.19:0/2206936696 conn(0x7f692c0d1720 msgr2=0x7f692c072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 shutdown_connections 2024-04-25T11:31:22.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.357+0000 7f6932000640 1 -- 172.21.15.19:0/2206936696 wait complete. 2024-04-25T11:31:22.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:31:05.039941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:31:05.784246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:31:22.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:31:22.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.674+0000 7fa106c00640 1 Processor -- start 2024-04-25T11:31:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.674+0000 7fa106c00640 1 -- start start 2024-04-25T11:31:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.676+0000 7fa106c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa100144340 0x7fa10006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.676+0000 7fa106c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.676+0000 7fa106c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa10015ec90 0x7fa100161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.676+0000 7fa106c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa100144750 con 0x7fa100144340 2024-04-25T11:31:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.676+0000 7fa106c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa10006df20 con 0x7fa10015ec90 2024-04-25T11:31:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.676+0000 7fa106c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa100070000 con 0x7fa10006db10 2024-04-25T11:31:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa100144340 0x7fa10006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa100144340 0x7fa10006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41208/0 (socket says 172.21.15.19:41208) 2024-04-25T11:31:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 -- 172.21.15.19:0/2095513028 learned_addr learned my addr 172.21.15.19:0/2095513028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa104e00640 1 --2- 172.21.15.19:0/2095513028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa106200640 1 --2- 172.21.15.19:0/2095513028 >> v2:172.21.15.169:3300/0 conn(0x7fa10015ec90 0x7fa100161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 -- 172.21.15.19:0/2095513028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 msgr2=0x7fa10015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 --2- 172.21.15.19:0/2095513028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 -- 172.21.15.19:0/2095513028 >> v2:172.21.15.169:3300/0 conn(0x7fa10015ec90 msgr2=0x7fa100161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 --2- 172.21.15.19:0/2095513028 >> v2:172.21.15.169:3300/0 conn(0x7fa10015ec90 0x7fa100161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.677+0000 7fa105800640 1 -- 172.21.15.19:0/2095513028 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10006f240 con 0x7fa100144340 2024-04-25T11:31:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.678+0000 7fa106200640 1 --2- 172.21.15.19:0/2095513028 >> v2:172.21.15.169:3300/0 conn(0x7fa10015ec90 0x7fa100161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.679+0000 7fa105800640 1 --2- 172.21.15.19:0/2095513028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa10006d5d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa100147c30 tx=0x7fa0f402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8038eeb4329d6b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.679+0000 7fa0ffe00640 1 -- 172.21.15.19:0/2095513028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4036070 con 0x7fa100144340 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.679+0000 7fa0ffe00640 1 -- 172.21.15.19:0/2095513028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f4031600 con 0x7fa100144340 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.679+0000 7fa0ffe00640 1 -- 172.21.15.19:0/2095513028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40319d0 con 0x7fa100144340 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 -- 172.21.15.19:0/2095513028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 msgr2=0x7fa10006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 --2- 172.21.15.19:0/2095513028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa10006d5d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa100147c30 tx=0x7fa0f402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 -- 172.21.15.19:0/2095513028 shutdown_connections 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 --2- 172.21.15.19:0/2095513028 >> v2:172.21.15.169:3300/0 conn(0x7fa10015ec90 0x7fa100161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 --2- 172.21.15.19:0/2095513028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 --2- 172.21.15.19:0/2095513028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa10006d5d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 -- 172.21.15.19:0/2095513028 >> 172.21.15.19:0/2095513028 conn(0x7fa100072f40 msgr2=0x7fa100073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 -- 172.21.15.19:0/2095513028 shutdown_connections 2024-04-25T11:31:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.680+0000 7fa106c00640 1 -- 172.21.15.19:0/2095513028 wait complete. 2024-04-25T11:31:22.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.683+0000 7fa106c00640 1 Processor -- start 2024-04-25T11:31:22.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.683+0000 7fa106c00640 1 -- start start 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.684+0000 7fa106c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10013f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.684+0000 7fa106c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa1001e9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.684+0000 7fa106c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa10015ec90 0x7fa1001612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.684+0000 7fa106c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa10006fae0 con 0x7fa100144340 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.684+0000 7fa106c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa10006f800 con 0x7fa10015ec90 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.684+0000 7fa106c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa10015f0a0 con 0x7fa10006db10 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.684+0000 7fa104e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa1001e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa104e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa1001e9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41218/0 (socket says 172.21.15.19:41218) 2024-04-25T11:31:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa104e00640 1 -- 172.21.15.19:0/2999159414 learned_addr learned my addr 172.21.15.19:0/2999159414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa105800640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10013f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa106200640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa10015ec90 0x7fa1001612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa104e00640 1 -- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 msgr2=0x7fa10013f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa104e00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10013f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa104e00640 1 -- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa10015ec90 msgr2=0x7fa1001612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa104e00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa10015ec90 0x7fa1001612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa104e00640 1 -- 172.21.15.19:0/2999159414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1001f7bc0 con 0x7fa100144340 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.685+0000 7fa105800640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10013f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.686+0000 7fa104e00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa1001e9200 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e840 tx=0x7fa0ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.686+0000 7fa0fd600640 1 -- 172.21.15.19:0/2999159414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec017070 con 0x7fa100144340 2024-04-25T11:31:22.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.686+0000 7fa0fd600640 1 -- 172.21.15.19:0/2999159414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ec00c730 con 0x7fa100144340 2024-04-25T11:31:22.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.686+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1001f80f0 con 0x7fa100144340 2024-04-25T11:31:22.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.686+0000 7fa0fd600640 1 -- 172.21.15.19:0/2999159414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00cb20 con 0x7fa100144340 2024-04-25T11:31:22.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.686+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1001f86b0 con 0x7fa100144340 2024-04-25T11:31:22.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.687+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1001f7e60 con 0x7fa100144340 2024-04-25T11:31:22.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.688+0000 7fa0fd600640 1 -- 172.21.15.19:0/2999159414 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa0ec01a070 con 0x7fa100144340 2024-04-25T11:31:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.689+0000 7fa0fd600640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0e00744b0 0x7fa0e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:22.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.690+0000 7fa0fd600640 1 -- 172.21.15.19:0/2999159414 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fa0ec0950d0 con 0x7fa100144340 2024-04-25T11:31:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.690+0000 7fa105800640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0e00744b0 0x7fa0e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:22.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.691+0000 7fa105800640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0e00744b0 0x7fa0e00769c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4033000 tx=0x7fa0f4009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:22.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.695+0000 7fa0fd600640 1 -- 172.21.15.19:0/2999159414 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0ec05f7c0 con 0x7fa100144340 2024-04-25T11:31:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.900+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa10006a1f0 con 0x7fa100144340 2024-04-25T11:31:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.901+0000 7fa0fd600640 1 -- 172.21.15.19:0/2999159414 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7fa10006a1f0 con 0x7fa100144340 2024-04-25T11:31:22.892 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:31:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0e00744b0 msgr2=0x7fa0e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0e00744b0 0x7fa0e00769c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4033000 tx=0x7fa0f4009000 comp rx=0 tx=0).stop 2024-04-25T11:31:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 msgr2=0x7fa1001e9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa1001e9200 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e840 tx=0x7fa0ec00ed00 comp rx=0 tx=0).stop 2024-04-25T11:31:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 shutdown_connections 2024-04-25T11:31:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0e00744b0 0x7fa0e00769c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa10015ec90 0x7fa1001612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa100144340 0x7fa1001e9200 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 --2- 172.21.15.19:0/2999159414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa10006db10 0x7fa10013f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.904+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 >> 172.21.15.19:0/2999159414 conn(0x7fa100072f40 msgr2=0x7fa100073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.905+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 shutdown_connections 2024-04-25T11:31:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:22.905+0000 7fa106c00640 1 -- 172.21.15.19:0/2999159414 wait complete. 2024-04-25T11:31:22.921 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:31:23.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.222+0000 7f643f000640 1 Processor -- start 2024-04-25T11:31:23.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.222+0000 7f643f000640 1 -- start start 2024-04-25T11:31:23.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f643f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64381434c0 0x7f6438160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f643f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f64380d0e40 0x7f64380d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f643f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64380d03a0 con 0x7f64380b4e00 2024-04-25T11:31:23.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f643f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f64381438d0 con 0x7f64380d0e40 2024-04-25T11:31:23.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f643f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6438161600 con 0x7f64381434c0 2024-04-25T11:31:23.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f6437e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64381434c0 0x7f6438160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f6437e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64381434c0 0x7f6438160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:43534/0 (socket says 172.21.15.19:43534) 2024-04-25T11:31:23.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f6437e00640 1 -- 172.21.15.19:0/3230619803 learned_addr learned my addr 172.21.15.19:0/3230619803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:23.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.225+0000 7f6437400640 1 --2- 172.21.15.19:0/3230619803 >> v2:172.21.15.169:3300/0 conn(0x7f64380d0e40 0x7f64380d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.226+0000 7f6437e00640 1 -- 172.21.15.19:0/3230619803 >> v2:172.21.15.169:3300/0 conn(0x7f64380d0e40 msgr2=0x7f64380d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:23.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.226+0000 7f6437e00640 1 --2- 172.21.15.19:0/3230619803 >> v2:172.21.15.169:3300/0 conn(0x7f64380d0e40 0x7f64380d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.226+0000 7f6437e00640 1 -- 172.21.15.19:0/3230619803 >> v1:172.21.15.169:6789/0 conn(0x7f64380b4e00 legacy=0x7f643806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:23.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.226+0000 7f6437e00640 1 -- 172.21.15.19:0/3230619803 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64381618e0 con 0x7f64381434c0 2024-04-25T11:31:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.228+0000 7f6437e00640 1 --2- 172.21.15.19:0/3230619803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64381434c0 0x7f6438160c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6428009870 tx=0x7f642802a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333eb52ee6d82473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.228+0000 7f6436a00640 1 -- 172.21.15.19:0/3230619803 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428037070 con 0x7f64381434c0 2024-04-25T11:31:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.228+0000 7f6436a00640 1 -- 172.21.15.19:0/3230619803 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6428032440 con 0x7f64381434c0 2024-04-25T11:31:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.228+0000 7f6436a00640 1 -- 172.21.15.19:0/3230619803 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428032810 con 0x7f64381434c0 2024-04-25T11:31:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 -- 172.21.15.19:0/3230619803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64381434c0 msgr2=0x7f6438160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 --2- 172.21.15.19:0/3230619803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64381434c0 0x7f6438160c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6428009870 tx=0x7f642802a200 comp rx=0 tx=0).stop 2024-04-25T11:31:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 -- 172.21.15.19:0/3230619803 shutdown_connections 2024-04-25T11:31:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 --2- 172.21.15.19:0/3230619803 >> v2:172.21.15.169:3300/0 conn(0x7f64380d0e40 0x7f64380d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 --2- 172.21.15.19:0/3230619803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64381434c0 0x7f6438160c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 -- 172.21.15.19:0/3230619803 >> 172.21.15.19:0/3230619803 conn(0x7f64380730b0 msgr2=0x7f64380734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 -- 172.21.15.19:0/3230619803 shutdown_connections 2024-04-25T11:31:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.229+0000 7f643f000640 1 -- 172.21.15.19:0/3230619803 wait complete. 2024-04-25T11:31:23.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.233+0000 7f643f000640 1 Processor -- start 2024-04-25T11:31:23.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.233+0000 7f643f000640 1 -- start start 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.235+0000 7f643f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380b4e00 0x7f64381559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.235+0000 7f643f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380d0e40 0x7f6438142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.235+0000 7f643f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64381434c0 0x7f6438144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.235+0000 7f643f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f64381469b0 con 0x7f64381434c0 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.235+0000 7f643f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f64380d0920 con 0x7f64380b4e00 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.235+0000 7f643f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6438146b30 con 0x7f64380d0e40 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64381434c0 0x7f6438144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64381434c0 0x7f6438144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41228/0 (socket says 172.21.15.19:41228) 2024-04-25T11:31:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 -- 172.21.15.19:0/1445180349 learned_addr learned my addr 172.21.15.19:0/1445180349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f6437e00640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380b4e00 0x7f64381559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f6437400640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380d0e40 0x7f6438142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 -- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380d0e40 msgr2=0x7f6438142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380d0e40 0x7f6438142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 -- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380b4e00 msgr2=0x7f64381559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380b4e00 0x7f64381559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.236+0000 7f643d000640 1 -- 172.21.15.19:0/1445180349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6438155f10 con 0x7f64381434c0 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.237+0000 7f643d000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64381434c0 0x7f6438144ac0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f643000be90 tx=0x7f643000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.237+0000 7f642fe00640 1 -- 172.21.15.19:0/1445180349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430019070 con 0x7f64381434c0 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.237+0000 7f642fe00640 1 -- 172.21.15.19:0/1445180349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6430014440 con 0x7f64381434c0 2024-04-25T11:31:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.237+0000 7f642fe00640 1 -- 172.21.15.19:0/1445180349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430014870 con 0x7f64381434c0 2024-04-25T11:31:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.237+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6438140f20 con 0x7f64381434c0 2024-04-25T11:31:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.237+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64381414e0 con 0x7f64381434c0 2024-04-25T11:31:23.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.238+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6438050740 con 0x7f64381434c0 2024-04-25T11:31:23.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.240+0000 7f642fe00640 1 -- 172.21.15.19:0/1445180349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f643001c070 con 0x7f64381434c0 2024-04-25T11:31:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.241+0000 7f642fe00640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64180744b0 0x7f64180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.241+0000 7f642fe00640 1 -- 172.21.15.19:0/1445180349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f6430095150 con 0x7f64381434c0 2024-04-25T11:31:23.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.241+0000 7f6437e00640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64180744b0 0x7f64180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.243+0000 7f6437e00640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64180744b0 0x7f64180769c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6428034000 tx=0x7f642802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:23.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.247+0000 7f642fe00640 1 -- 172.21.15.19:0/1445180349 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f643005f280 con 0x7f64381434c0 2024-04-25T11:31:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.466+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f643806d070 con 0x7f64381434c0 2024-04-25T11:31:23.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.467+0000 7f642fe00640 1 -- 172.21.15.19:0/1445180349 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) ==== 104+0+11 (secure 0 0 0) 0x7f6430063580 con 0x7f64381434c0 2024-04-25T11:31:23.458 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:31:23.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64180744b0 msgr2=0x7f64180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64180744b0 0x7f64180769c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6428034000 tx=0x7f642802e000 comp rx=0 tx=0).stop 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64381434c0 msgr2=0x7f6438144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64381434c0 0x7f6438144ac0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f643000be90 tx=0x7f643000bf80 comp rx=0 tx=0).stop 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 shutdown_connections 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f64180744b0 0x7f64180769c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64381434c0 0x7f6438144ac0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f64380d0e40 0x7f6438142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 --2- 172.21.15.19:0/1445180349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f64380b4e00 0x7f64381559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 >> 172.21.15.19:0/1445180349 conn(0x7f64380730b0 msgr2=0x7f643806d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 shutdown_connections 2024-04-25T11:31:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.470+0000 7f643f000640 1 -- 172.21.15.19:0/1445180349 wait complete. 2024-04-25T11:31:23.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:31:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.787+0000 7f5054c00640 1 Processor -- start 2024-04-25T11:31:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.787+0000 7f5054c00640 1 -- start start 2024-04-25T11:31:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.790+0000 7f5054c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50500af340 0x7f505015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.790+0000 7f5054c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50500af9b0 con 0x7f50500b0a50 2024-04-25T11:31:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.790+0000 7f5054c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50500afc90 con 0x7f505006dce0 2024-04-25T11:31:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.790+0000 7f5054c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f50500af750 con 0x7f50500af340 2024-04-25T11:31:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.790+0000 7f504fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50500af340 0x7f505015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.790+0000 7f504f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f50500b0a50 0x7f505006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46072/0 (socket says 172.21.15.19:46072) 2024-04-25T11:31:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.790+0000 7f504f400640 1 -- 172.21.15.19:0/2721642492 learned_addr learned my addr 172.21.15.19:0/2721642492 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.791+0000 7f504fe00640 1 -- 172.21.15.19:0/2721642492 >> v1:172.21.15.169:6789/0 conn(0x7f505006dce0 legacy=0x7f505006f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.791+0000 7f504e000640 1 -- 172.21.15.19:0/2721642492 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078018991 0 0) 0x7f50500af9b0 con 0x7f50500b0a50 2024-04-25T11:31:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.791+0000 7f504fe00640 1 -- 172.21.15.19:0/2721642492 >> v1:172.21.15.19:6789/0 conn(0x7f50500b0a50 legacy=0x7f505006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.791+0000 7f504fe00640 1 -- 172.21.15.19:0/2721642492 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50500aff70 con 0x7f50500af340 2024-04-25T11:31:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.793+0000 7f504fe00640 1 --2- 172.21.15.19:0/2721642492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f505015fff0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f50400097a0 tx=0x7f504002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b31c7db55ab82af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.793+0000 7f504e000640 1 -- 172.21.15.19:0/2721642492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040037070 con 0x7f50500af340 2024-04-25T11:31:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.793+0000 7f504e000640 1 -- 172.21.15.19:0/2721642492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5040032440 con 0x7f50500af340 2024-04-25T11:31:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.793+0000 7f504e000640 1 -- 172.21.15.19:0/2721642492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040032810 con 0x7f50500af340 2024-04-25T11:31:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.794+0000 7f5054c00640 1 -- 172.21.15.19:0/2721642492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 msgr2=0x7f505015fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.794+0000 7f5054c00640 1 --2- 172.21.15.19:0/2721642492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f505015fff0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f50400097a0 tx=0x7f504002a230 comp rx=0 tx=0).stop 2024-04-25T11:31:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.794+0000 7f5054c00640 1 -- 172.21.15.19:0/2721642492 shutdown_connections 2024-04-25T11:31:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.794+0000 7f5054c00640 1 --2- 172.21.15.19:0/2721642492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f505015fff0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.794+0000 7f5054c00640 1 -- 172.21.15.19:0/2721642492 >> 172.21.15.19:0/2721642492 conn(0x7f5050073160 msgr2=0x7f5050073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.794+0000 7f5054c00640 1 -- 172.21.15.19:0/2721642492 shutdown_connections 2024-04-25T11:31:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.794+0000 7f5054c00640 1 -- 172.21.15.19:0/2721642492 wait complete. 2024-04-25T11:31:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.797+0000 7f5054c00640 1 Processor -- start 2024-04-25T11:31:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.797+0000 7f5054c00640 1 -- start start 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.798+0000 7f5054c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f50500d1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.798+0000 7f5054c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50500b0a50 0x7f505015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.798+0000 7f5054c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5050157570 0x7f50500cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.798+0000 7f5054c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50500b4f80 con 0x7f50500af340 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.798+0000 7f5054c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50500b0530 con 0x7f50500b0a50 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.798+0000 7f5054c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50500b5100 con 0x7f5050157570 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f50500d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f50500d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58640/0 (socket says 172.21.15.19:58640) 2024-04-25T11:31:23.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 -- 172.21.15.19:0/1146157574 learned_addr learned my addr 172.21.15.19:0/1146157574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504ea00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50500b0a50 0x7f505015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 -- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5050157570 msgr2=0x7f50500cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5050157570 0x7f50500cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 -- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50500b0a50 msgr2=0x7f505015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50500b0a50 0x7f505015c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.799+0000 7f504f400640 1 -- 172.21.15.19:0/1146157574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5050160ac0 con 0x7f50500af340 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.800+0000 7f504f400640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f50500d1020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f504800c440 tx=0x7f504800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.800+0000 7f5047400640 1 -- 172.21.15.19:0/1146157574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50480074e0 con 0x7f50500af340 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.800+0000 7f5047400640 1 -- 172.21.15.19:0/1146157574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50480047e0 con 0x7f50500af340 2024-04-25T11:31:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.800+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505015b240 con 0x7f50500af340 2024-04-25T11:31:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.800+0000 7f5047400640 1 -- 172.21.15.19:0/1146157574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048004bd0 con 0x7f50500af340 2024-04-25T11:31:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.800+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505015cea0 con 0x7f50500af340 2024-04-25T11:31:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.801+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5050076c60 con 0x7f50500af340 2024-04-25T11:31:23.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.802+0000 7f5047400640 1 -- 172.21.15.19:0/1146157574 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5048011070 con 0x7f50500af340 2024-04-25T11:31:23.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.803+0000 7f5047400640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f502c0744e0 0x7f502c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.803+0000 7f5047400640 1 -- 172.21.15.19:0/1146157574 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f5048094e30 con 0x7f50500af340 2024-04-25T11:31:23.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.804+0000 7f504ea00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f502c0744e0 0x7f502c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:23.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.805+0000 7f504ea00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f502c0744e0 0x7f502c0769f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5038007fd0 tx=0x7f5038009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:23.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:23.809+0000 7f5047400640 1 -- 172.21.15.19:0/1146157574 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f504805f560 con 0x7f50500af340 2024-04-25T11:31:24.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.011+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5050073ec0 con 0x7f50500af340 2024-04-25T11:31:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.012+0000 7f5047400640 1 -- 172.21.15.19:0/1146157574 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) ==== 100+0+11 (secure 0 0 0) 0x7f504805ee60 con 0x7f50500af340 2024-04-25T11:31:24.002 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:31:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.014+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f502c0744e0 msgr2=0x7f502c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.014+0000 7f5054c00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f502c0744e0 0x7f502c0769f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5038007fd0 tx=0x7f5038009040 comp rx=0 tx=0).stop 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.014+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 msgr2=0x7f50500d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.014+0000 7f5054c00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f50500d1020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f504800c440 tx=0x7f504800c900 comp rx=0 tx=0).stop 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 shutdown_connections 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f502c0744e0 0x7f502c0769f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50500b0a50 0x7f505015c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5050157570 0x7f50500cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 --2- 172.21.15.19:0/1146157574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50500af340 0x7f50500d1020 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 >> 172.21.15.19:0/1146157574 conn(0x7f5050073160 msgr2=0x7f505006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 shutdown_connections 2024-04-25T11:31:24.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.015+0000 7f5054c00640 1 -- 172.21.15.19:0/1146157574 wait complete. 2024-04-25T11:31:24.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:31:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.330+0000 7ff726c00640 1 Processor -- start 2024-04-25T11:31:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.330+0000 7ff726c00640 1 -- start start 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.332+0000 7ff726c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff7200cf2f0 0x7ff7200cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.332+0000 7ff726c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff720168010 0x7ff720168420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.332+0000 7ff726c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7201649d0 0x7ff720164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff726c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff720070cf0 con 0x7ff720168010 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff726c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff720168960 con 0x7ff7200cf2f0 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff726c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff720071180 con 0x7ff7201649d0 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff720168010 0x7ff720168420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff720168010 0x7ff720168420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58656/0 (socket says 172.21.15.19:58656) 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 -- 172.21.15.19:0/1762459572 learned_addr learned my addr 172.21.15.19:0/1762459572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff725800640 1 --2- 172.21.15.19:0/1762459572 >> v2:172.21.15.169:3300/0 conn(0x7ff7200cf2f0 0x7ff7200cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff726200640 1 --2- 172.21.15.19:0/1762459572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7201649d0 0x7ff720164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 -- 172.21.15.19:0/1762459572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7201649d0 msgr2=0x7ff720164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 --2- 172.21.15.19:0/1762459572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7201649d0 0x7ff720164de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 -- 172.21.15.19:0/1762459572 >> v2:172.21.15.169:3300/0 conn(0x7ff7200cf2f0 msgr2=0x7ff7200cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 --2- 172.21.15.19:0/1762459572 >> v2:172.21.15.169:3300/0 conn(0x7ff7200cf2f0 0x7ff7200cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff724e00640 1 -- 172.21.15.19:0/1762459572 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7200700e0 con 0x7ff720168010 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.333+0000 7ff725800640 1 --2- 172.21.15.19:0/1762459572 >> v2:172.21.15.169:3300/0 conn(0x7ff7200cf2f0 0x7ff7200cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.335+0000 7ff724e00640 1 --2- 172.21.15.19:0/1762459572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff720168010 0x7ff720168420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7200726b0 tx=0x7ff70c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5522e77778c62d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.335+0000 7ff71fe00640 1 -- 172.21.15.19:0/1762459572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c037070 con 0x7ff720168010 2024-04-25T11:31:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.335+0000 7ff71fe00640 1 -- 172.21.15.19:0/1762459572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70c032440 con 0x7ff720168010 2024-04-25T11:31:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.335+0000 7ff71fe00640 1 -- 172.21.15.19:0/1762459572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c032810 con 0x7ff720168010 2024-04-25T11:31:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 -- 172.21.15.19:0/1762459572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff720168010 msgr2=0x7ff720168420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 --2- 172.21.15.19:0/1762459572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff720168010 0x7ff720168420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7200726b0 tx=0x7ff70c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 -- 172.21.15.19:0/1762459572 shutdown_connections 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 --2- 172.21.15.19:0/1762459572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff720168010 0x7ff720168420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 --2- 172.21.15.19:0/1762459572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7201649d0 0x7ff720164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 --2- 172.21.15.19:0/1762459572 >> v2:172.21.15.169:3300/0 conn(0x7ff7200cf2f0 0x7ff7200cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 -- 172.21.15.19:0/1762459572 >> 172.21.15.19:0/1762459572 conn(0x7ff7200d1720 msgr2=0x7ff7200d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 -- 172.21.15.19:0/1762459572 shutdown_connections 2024-04-25T11:31:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.336+0000 7ff726c00640 1 -- 172.21.15.19:0/1762459572 wait complete. 2024-04-25T11:31:24.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.338+0000 7ff726c00640 1 Processor -- start 2024-04-25T11:31:24.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.339+0000 7ff726c00640 1 -- start start 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff726c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7200cf2f0 0x7ff720161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff726c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7201649d0 0x7ff7201eee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff726c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720168010 0x7ff72014b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff726c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff720070980 con 0x7ff7201649d0 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff726c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7200706a0 con 0x7ff720168010 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff726c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff720168d00 con 0x7ff7200cf2f0 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff724e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7201649d0 0x7ff7201eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff724e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7201649d0 0x7ff7201eee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58670/0 (socket says 172.21.15.19:58670) 2024-04-25T11:31:24.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.340+0000 7ff724e00640 1 -- 172.21.15.19:0/342608857 learned_addr learned my addr 172.21.15.19:0/342608857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff726200640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720168010 0x7ff72014b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff725800640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7200cf2f0 0x7ff720161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff724e00640 1 -- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7200cf2f0 msgr2=0x7ff720161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff724e00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7200cf2f0 0x7ff720161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff724e00640 1 -- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720168010 msgr2=0x7ff72014b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff724e00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720168010 0x7ff72014b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff724e00640 1 -- 172.21.15.19:0/342608857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72014bb60 con 0x7ff7201649d0 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff726200640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720168010 0x7ff72014b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff725800640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7200cf2f0 0x7ff720161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.341+0000 7ff724e00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7201649d0 0x7ff7201eee30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff70c02a220 tx=0x7ff70c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:24.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.342+0000 7ff71d600640 1 -- 172.21.15.19:0/342608857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c037040 con 0x7ff7201649d0 2024-04-25T11:31:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.342+0000 7ff71d600640 1 -- 172.21.15.19:0/342608857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70c042070 con 0x7ff7201649d0 2024-04-25T11:31:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.342+0000 7ff71d600640 1 -- 172.21.15.19:0/342608857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c030bb0 con 0x7ff7201649d0 2024-04-25T11:31:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.342+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201607a0 con 0x7ff7201649d0 2024-04-25T11:31:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.342+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff720160d60 con 0x7ff7201649d0 2024-04-25T11:31:24.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.343+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff720160510 con 0x7ff7201649d0 2024-04-25T11:31:24.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.344+0000 7ff71d600640 1 -- 172.21.15.19:0/342608857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff70c047460 con 0x7ff7201649d0 2024-04-25T11:31:24.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.346+0000 7ff71d600640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7000744b0 0x7ff7000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:24.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.346+0000 7ff71d600640 1 -- 172.21.15.19:0/342608857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7ff70c0b5ef0 con 0x7ff7201649d0 2024-04-25T11:31:24.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.347+0000 7ff725800640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7000744b0 0x7ff7000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:24.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.348+0000 7ff725800640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7000744b0 0x7ff7000769c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff7200d11e0 tx=0x7ff7100074b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:24.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.351+0000 7ff71d600640 1 -- 172.21.15.19:0/342608857 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff70c080020 con 0x7ff7201649d0 2024-04-25T11:31:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.568+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff720050740 con 0x7ff7201649d0 2024-04-25T11:31:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.569+0000 7ff71d600640 1 -- 172.21.15.19:0/342608857 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7ff70c084320 con 0x7ff7201649d0 2024-04-25T11:31:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:31:24.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7000744b0 msgr2=0x7ff7000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:24.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7000744b0 0x7ff7000769c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff7200d11e0 tx=0x7ff7100074b0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7201649d0 msgr2=0x7ff7201eee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7201649d0 0x7ff7201eee30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff70c02a220 tx=0x7ff70c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 shutdown_connections 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7000744b0 0x7ff7000769c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff720168010 0x7ff72014b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7201649d0 0x7ff7201eee30 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 --2- 172.21.15.19:0/342608857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7200cf2f0 0x7ff720161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 >> 172.21.15.19:0/342608857 conn(0x7ff7200d1720 msgr2=0x7ff7200b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 shutdown_connections 2024-04-25T11:31:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:24.572+0000 7ff726c00640 1 -- 172.21.15.19:0/342608857 wait complete. 2024-04-25T11:31:24.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:31:24.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:24.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:31:24.694 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:31:24.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:31:24.694 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:31:24.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:31:25.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.035+0000 7f8b5dc00640 1 Processor -- start 2024-04-25T11:31:25.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.035+0000 7f8b5dc00640 1 -- start start 2024-04-25T11:31:25.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.038+0000 7f8b5dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 0x7f8b58168e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.038+0000 7f8b5dc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b5806fc70 con 0x7f8b580d1070 2024-04-25T11:31:25.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.038+0000 7f8b5dc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b5806ff50 con 0x7f8b58144350 2024-04-25T11:31:25.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.038+0000 7f8b5dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8b58070230 con 0x7f8b58145e70 2024-04-25T11:31:25.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.038+0000 7f8b4fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8b580d1070 0x7f8b58162200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46074/0 (socket says 172.21.15.19:46074) 2024-04-25T11:31:25.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.038+0000 7f8b4fe00640 1 -- 172.21.15.19:0/2647337663 learned_addr learned my addr 172.21.15.19:0/2647337663 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:25.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b5d200640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 0x7f8b58168e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968187196 0 0) 0x7f8b5806fc70 con 0x7f8b580d1070 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b3c003520 con 0x7f8b580d1070 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4107830189 0 0) 0x7f8b3c003520 con 0x7f8b580d1070 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 msgr2=0x7f8b58168e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 0x7f8b58168e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 >> v1:172.21.15.169:6789/0 conn(0x7f8b58144350 legacy=0x7f8b58161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b58070510 con 0x7f8b580d1070 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8b40001690 con 0x7f8b580d1070 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 >> v1:172.21.15.19:6789/0 conn(0x7f8b580d1070 legacy=0x7f8b58162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 0x7f8b3c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b58070230 con 0x7f8b3c0039e0 2024-04-25T11:31:25.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c00baf0 con 0x7f8b3c004800 2024-04-25T11:31:25.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.039+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3c00bcf0 con 0x7f8b58145e70 2024-04-25T11:31:25.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.040+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.040+0000 7f8b4fe00640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.040+0000 7f8b5d200640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 0x7f8b3c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.041+0000 7f8b57e00640 1 -- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 msgr2=0x7f8b3c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:25.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.041+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58145e70 0x7f8b3c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.041+0000 7f8b57e00640 1 -- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c004800 msgr2=0x7f8b3c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:25.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.041+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c004800 0x7f8b3c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.041+0000 7f8b57e00640 1 -- 172.21.15.19:0/2647337663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3c003520 con 0x7f8b3c0039e0 2024-04-25T11:31:25.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.042+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8b58147d40 tx=0x7f8b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe0216a47e480d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48001370 con 0x7f8b3c0039e0 2024-04-25T11:31:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b480052d0 con 0x7f8b3c0039e0 2024-04-25T11:31:25.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b57400640 1 -- 172.21.15.19:0/2647337663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4802ab90 con 0x7f8b3c0039e0 2024-04-25T11:31:25.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2647337663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0039e0 msgr2=0x7f8b3c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b5dc00640 1 --2- 172.21.15.19:0/2647337663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3c0039e0 0x7f8b3c003e50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8b58147d40 tx=0x7f8b3c005830 comp rx=0 tx=0).stop 2024-04-25T11:31:25.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b5d200640 1 -- 172.21.15.19:0/2647337663 reap_dead start 2024-04-25T11:31:25.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2647337663 shutdown_connections 2024-04-25T11:31:25.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2647337663 >> 172.21.15.19:0/2647337663 conn(0x7f8b580d1720 msgr2=0x7f8b580d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:25.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.043+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2647337663 shutdown_connections 2024-04-25T11:31:25.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.044+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2647337663 wait complete. 2024-04-25T11:31:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.046+0000 7f8b5dc00640 1 Processor -- start 2024-04-25T11:31:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.046+0000 7f8b5dc00640 1 -- start start 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.047+0000 7f8b5dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c0039e0 0x7f8b5815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b5dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b5815ea50 0x7f8b58160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b5dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58160d60 0x7f8b581f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b5dc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b580b4890 con 0x7f8b5815ea50 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b5dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8b58147910 con 0x7f8b3c0039e0 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b5dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8b580b4a10 con 0x7f8b58160d60 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b5815ea50 0x7f8b58160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b5815ea50 0x7f8b58160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58698/0 (socket says 172.21.15.19:58698) 2024-04-25T11:31:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 -- 172.21.15.19:0/2693176591 learned_addr learned my addr 172.21.15.19:0/2693176591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c0039e0 0x7f8b5815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b5d200640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58160d60 0x7f8b581f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 -- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58160d60 msgr2=0x7f8b581f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58160d60 0x7f8b581f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 -- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c0039e0 msgr2=0x7f8b5815b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c0039e0 0x7f8b5815b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b4fe00640 1 -- 172.21.15.19:0/2693176591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b58161170 con 0x7f8b5815ea50 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.048+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c0039e0 0x7f8b5815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.049+0000 7f8b4fe00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b5815ea50 0x7f8b58160820 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8b40004f70 tx=0x7f8b400074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.049+0000 7f8b54c00640 1 -- 172.21.15.19:0/2693176591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40007740 con 0x7f8b5815ea50 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.049+0000 7f8b54c00640 1 -- 172.21.15.19:0/2693176591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b40018070 con 0x7f8b5815ea50 2024-04-25T11:31:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.049+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5815b9d0 con 0x7f8b5815ea50 2024-04-25T11:31:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.049+0000 7f8b54c00640 1 -- 172.21.15.19:0/2693176591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000fba0 con 0x7f8b5815ea50 2024-04-25T11:31:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.049+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5815cc80 con 0x7f8b5815ea50 2024-04-25T11:31:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.050+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b58143010 con 0x7f8b5815ea50 2024-04-25T11:31:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.052+0000 7f8b54c00640 1 -- 172.21.15.19:0/2693176591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8b4000fd40 con 0x7f8b5815ea50 2024-04-25T11:31:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.053+0000 7f8b54c00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b340745f0 0x7f8b34076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.053+0000 7f8b54c00640 1 -- 172.21.15.19:0/2693176591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f8b40095260 con 0x7f8b5815ea50 2024-04-25T11:31:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.053+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b340745f0 0x7f8b34076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.055+0000 7f8b57e00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b340745f0 0x7f8b34076b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8b48032000 tx=0x7f8b4802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:25.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.058+0000 7f8b54c00640 1 -- 172.21.15.19:0/2693176591 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b58161170 con 0x7f8b5815ea50 2024-04-25T11:31:25.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.307+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8b5815f230 con 0x7f8b5815ea50 2024-04-25T11:31:25.299 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:31:25.309+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:31:25.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.313+0000 7f8b54c00640 1 -- 172.21.15.19:0/2693176591 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) ==== 161+0+0 (secure 0 0 0) 0x7f8b4005f9e0 con 0x7f8b5815ea50 2024-04-25T11:31:25.304 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:31:25.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b340745f0 msgr2=0x7f8b34076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b340745f0 0x7f8b34076b00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8b48032000 tx=0x7f8b4802e000 comp rx=0 tx=0).stop 2024-04-25T11:31:25.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b5815ea50 msgr2=0x7f8b58160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b5815ea50 0x7f8b58160820 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8b40004f70 tx=0x7f8b400074f0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 shutdown_connections 2024-04-25T11:31:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b340745f0 0x7f8b34076b00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b58160d60 0x7f8b581f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b5815ea50 0x7f8b58160820 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 --2- 172.21.15.19:0/2693176591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b3c0039e0 0x7f8b5815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 >> 172.21.15.19:0/2693176591 conn(0x7f8b580d1720 msgr2=0x7f8b581617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:25.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.316+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 shutdown_connections 2024-04-25T11:31:25.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.317+0000 7f8b5dc00640 1 -- 172.21.15.19:0/2693176591 wait complete. 2024-04-25T11:31:25.337 INFO:tasks.ceph.mds.c.smithi019.stderr: -5533> 2024-04-25T11:31:11.158+0000 7f061f400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:31:25.340 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2024-04-25T11:31:25.508 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:31:25.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.617+0000 7f0367200640 1 Processor -- start 2024-04-25T11:31:25.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.617+0000 7f0367200640 1 -- start start 2024-04-25T11:31:25.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.619+0000 7f0367200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0368144340 0x7f036806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.619+0000 7f0367200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f036806db10 0x7f036815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.619+0000 7f0367200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036806df20 con 0x7f036815ec90 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.619+0000 7f0367200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0368144750 con 0x7f036806db10 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.619+0000 7f0367200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0368070000 con 0x7f0368144340 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f036806db10 0x7f036815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f036806db10 0x7f036815e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58730/0 (socket says 172.21.15.19:58730) 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365400640 1 -- 172.21.15.19:0/1099218016 learned_addr learned my addr 172.21.15.19:0/1099218016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365e00640 1 --2- 172.21.15.19:0/1099218016 >> v2:172.21.15.169:3300/0 conn(0x7f0368144340 0x7f036806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365400640 1 -- 172.21.15.19:0/1099218016 >> v2:172.21.15.169:3300/0 conn(0x7f0368144340 msgr2=0x7f036806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:25.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365400640 1 --2- 172.21.15.19:0/1099218016 >> v2:172.21.15.169:3300/0 conn(0x7f0368144340 0x7f036806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365400640 1 -- 172.21.15.19:0/1099218016 >> v1:172.21.15.169:6789/0 conn(0x7f036815ec90 legacy=0x7f0368161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:25.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.620+0000 7f0365400640 1 -- 172.21.15.19:0/1099218016 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036806f240 con 0x7f036806db10 2024-04-25T11:31:25.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.621+0000 7f0365400640 1 --2- 172.21.15.19:0/1099218016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036806db10 0x7f036815e750 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f03680b6e30 tx=0x7f035402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e361ee4cd91eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:25.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.622+0000 7f0364a00640 1 -- 172.21.15.19:0/1099218016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354037070 con 0x7f036806db10 2024-04-25T11:31:25.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.622+0000 7f0364a00640 1 -- 172.21.15.19:0/1099218016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0354032440 con 0x7f036806db10 2024-04-25T11:31:25.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.622+0000 7f0364a00640 1 -- 172.21.15.19:0/1099218016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354032810 con 0x7f036806db10 2024-04-25T11:31:25.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 -- 172.21.15.19:0/1099218016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036806db10 msgr2=0x7f036815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 --2- 172.21.15.19:0/1099218016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036806db10 0x7f036815e750 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f03680b6e30 tx=0x7f035402a230 comp rx=0 tx=0).stop 2024-04-25T11:31:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 -- 172.21.15.19:0/1099218016 shutdown_connections 2024-04-25T11:31:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 --2- 172.21.15.19:0/1099218016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036806db10 0x7f036815e750 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 --2- 172.21.15.19:0/1099218016 >> v2:172.21.15.169:3300/0 conn(0x7f0368144340 0x7f036806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 -- 172.21.15.19:0/1099218016 >> 172.21.15.19:0/1099218016 conn(0x7f0368072f40 msgr2=0x7f0368073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 -- 172.21.15.19:0/1099218016 shutdown_connections 2024-04-25T11:31:25.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.623+0000 7f0367200640 1 -- 172.21.15.19:0/1099218016 wait complete. 2024-04-25T11:31:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.625+0000 7f0367200640 1 Processor -- start 2024-04-25T11:31:25.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.625+0000 7f0367200640 1 -- start start 2024-04-25T11:31:25.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0367200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f036806db10 0x7f0368158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0367200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0368144340 0x7f03680b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0367200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036815ec90 0x7f03680b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0367200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f036806fae0 con 0x7f0368144340 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0367200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f036806f800 con 0x7f036806db10 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0367200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f036815f0a0 con 0x7f036815ec90 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0365400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0368144340 0x7f03680b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0365400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0368144340 0x7f03680b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58740/0 (socket says 172.21.15.19:58740) 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0365400640 1 -- 172.21.15.19:0/1307029977 learned_addr learned my addr 172.21.15.19:0/1307029977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.627+0000 7f0365e00640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f036806db10 0x7f0368158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0366800640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036815ec90 0x7f03680b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0365400640 1 -- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036815ec90 msgr2=0x7f03680b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0365400640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036815ec90 0x7f03680b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0365400640 1 -- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f036806db10 msgr2=0x7f0368158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0365400640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f036806db10 0x7f0368158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0365400640 1 -- 172.21.15.19:0/1307029977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03680af4e0 con 0x7f0368144340 2024-04-25T11:31:25.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0366800640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036815ec90 0x7f03680b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:25.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f0365400640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0368144340 0x7f03680b2560 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f035402a270 tx=0x7f03540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.628+0000 7f035e000640 1 -- 172.21.15.19:0/1307029977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354041020 con 0x7f0368144340 2024-04-25T11:31:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.629+0000 7f035e000640 1 -- 172.21.15.19:0/1307029977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03540306c0 con 0x7f0368144340 2024-04-25T11:31:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.629+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036815f9e0 con 0x7f0368144340 2024-04-25T11:31:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.629+0000 7f035e000640 1 -- 172.21.15.19:0/1307029977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354037040 con 0x7f0368144340 2024-04-25T11:31:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.629+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03680b10b0 con 0x7f0368144340 2024-04-25T11:31:25.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.630+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0368076c60 con 0x7f0368144340 2024-04-25T11:31:25.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.631+0000 7f035e000640 1 -- 172.21.15.19:0/1307029977 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0354044020 con 0x7f0368144340 2024-04-25T11:31:25.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.633+0000 7f035e000640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03480744b0 0x7f03480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:25.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.633+0000 7f035e000640 1 -- 172.21.15.19:0/1307029977 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f03540b5c10 con 0x7f0368144340 2024-04-25T11:31:25.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.634+0000 7f0365e00640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03480744b0 0x7f03480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:25.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.635+0000 7f0365e00640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03480744b0 0x7f03480769c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0358006fd0 tx=0x7f0358008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:25.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.637+0000 7f035e000640 1 -- 172.21.15.19:0/1307029977 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f035404b030 con 0x7f0368144340 2024-04-25T11:31:25.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.844+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) -- 0x7f03680cff60 con 0x7f0368144340 2024-04-25T11:31:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.853+0000 7f035e000640 1 -- 172.21.15.19:0/1307029977 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v19)=0 v19) ==== 139+0+0 (secure 0 0 0) 0x7f036806df20 con 0x7f0368144340 2024-04-25T11:31:25.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03480744b0 msgr2=0x7f03480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03480744b0 0x7f03480769c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0358006fd0 tx=0x7f0358008040 comp rx=0 tx=0).stop 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0368144340 msgr2=0x7f03680b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0368144340 0x7f03680b2560 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f035402a270 tx=0x7f03540046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 shutdown_connections 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03480744b0 0x7f03480769c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036815ec90 0x7f03680b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0368144340 0x7f03680b2560 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 --2- 172.21.15.19:0/1307029977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f036806db10 0x7f0368158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 >> 172.21.15.19:0/1307029977 conn(0x7f0368072f40 msgr2=0x7f03680b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:25.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.856+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 shutdown_connections 2024-04-25T11:31:25.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:25.857+0000 7f0367200640 1 -- 172.21.15.19:0/1307029977 wait complete. 2024-04-25T11:31:30.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:31:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.176+0000 7f20f4200640 1 Processor -- start 2024-04-25T11:31:31.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.176+0000 7f20f4200640 1 -- start start 2024-04-25T11:31:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.179+0000 7f20f4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec0b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.179+0000 7f20f4200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20ec15df80 0x7f20ec0b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.179+0000 7f20f4200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ec1615f0 con 0x7f20ec160890 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.179+0000 7f20f4200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f20ec1613b0 con 0x7f20ec15df80 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.179+0000 7f20f4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec164670 con 0x7f20ec164b00 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.179+0000 7f20f0e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20ec15df80 0x7f20ec0b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.180+0000 7f20f0e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20ec15df80 0x7f20ec0b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58748/0 (socket says 172.21.15.19:58748) 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.180+0000 7f20f0e00640 1 -- 172.21.15.19:0/804624866 learned_addr learned my addr 172.21.15.19:0/804624866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.180+0000 7f20f1800640 1 --2- 172.21.15.19:0/804624866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec0b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.180+0000 7f20f0e00640 1 -- 172.21.15.19:0/804624866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 msgr2=0x7f20ec0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.180+0000 7f20f0e00640 1 --2- 172.21.15.19:0/804624866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec0b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.180+0000 7f20f0e00640 1 -- 172.21.15.19:0/804624866 >> v1:172.21.15.169:6789/0 conn(0x7f20ec160890 legacy=0x7f20ec160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:31.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.180+0000 7f20f0e00640 1 -- 172.21.15.19:0/804624866 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec164020 con 0x7f20ec15df80 2024-04-25T11:31:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.181+0000 7f20f0e00640 1 --2- 172.21.15.19:0/804624866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec0b80c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009870 tx=0x7f20d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525f7d232ca47035 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.181+0000 7f20ebe00640 1 -- 172.21.15.19:0/804624866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8037070 con 0x7f20ec15df80 2024-04-25T11:31:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20ebe00640 1 -- 172.21.15.19:0/804624866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d8032440 con 0x7f20ec15df80 2024-04-25T11:31:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20ebe00640 1 -- 172.21.15.19:0/804624866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8032810 con 0x7f20ec15df80 2024-04-25T11:31:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20f4200640 1 -- 172.21.15.19:0/804624866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 msgr2=0x7f20ec0b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20f4200640 1 --2- 172.21.15.19:0/804624866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec0b80c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009870 tx=0x7f20d802a210 comp rx=0 tx=0).stop 2024-04-25T11:31:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20f4200640 1 -- 172.21.15.19:0/804624866 shutdown_connections 2024-04-25T11:31:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20f4200640 1 --2- 172.21.15.19:0/804624866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec0b80c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20f4200640 1 --2- 172.21.15.19:0/804624866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec0b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.182+0000 7f20f4200640 1 -- 172.21.15.19:0/804624866 >> 172.21.15.19:0/804624866 conn(0x7f20ec0d2bc0 msgr2=0x7f20ec0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.183+0000 7f20f4200640 1 -- 172.21.15.19:0/804624866 shutdown_connections 2024-04-25T11:31:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.183+0000 7f20f4200640 1 -- 172.21.15.19:0/804624866 wait complete. 2024-04-25T11:31:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.185+0000 7f20f4200640 1 Processor -- start 2024-04-25T11:31:31.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.185+0000 7f20f4200640 1 -- start start 2024-04-25T11:31:31.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f4200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:31.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec160890 0x7f20ec156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:31.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f4200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec1ea550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:31.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f4200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0b4a00 con 0x7f20ec15df80 2024-04-25T11:31:31.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0b4880 con 0x7f20ec160890 2024-04-25T11:31:31.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f4200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec0b4b80 con 0x7f20ec164b00 2024-04-25T11:31:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58758/0 (socket says 172.21.15.19:58758) 2024-04-25T11:31:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f1800640 1 -- 172.21.15.19:0/638224087 learned_addr learned my addr 172.21.15.19:0/638224087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.187+0000 7f20f0e00640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec160890 0x7f20ec156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20f2200640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec1ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20f1800640 1 -- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 msgr2=0x7f20ec1ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:31.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20f1800640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec1ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20f1800640 1 -- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec160890 msgr2=0x7f20ec156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20f1800640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec160890 0x7f20ec156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20f1800640 1 -- 172.21.15.19:0/638224087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec157e60 con 0x7f20ec15df80 2024-04-25T11:31:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20f1800640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec158f00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d5f0 tx=0x7f20e000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20e9600640 1 -- 172.21.15.19:0/638224087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00125f0 con 0x7f20ec15df80 2024-04-25T11:31:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.188+0000 7f20e9600640 1 -- 172.21.15.19:0/638224087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e0019070 con 0x7f20ec15df80 2024-04-25T11:31:31.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.189+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec158140 con 0x7f20ec15df80 2024-04-25T11:31:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.189+0000 7f20e9600640 1 -- 172.21.15.19:0/638224087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00129d0 con 0x7f20ec15df80 2024-04-25T11:31:31.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.189+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec158700 con 0x7f20ec15df80 2024-04-25T11:31:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.190+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec050740 con 0x7f20ec15df80 2024-04-25T11:31:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.191+0000 7f20e9600640 1 -- 172.21.15.19:0/638224087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f20e0012b70 con 0x7f20ec15df80 2024-04-25T11:31:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.193+0000 7f20e9600640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20cc0744e0 0x7f20cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.193+0000 7f20e9600640 1 -- 172.21.15.19:0/638224087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f20e0095750 con 0x7f20ec15df80 2024-04-25T11:31:31.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.194+0000 7f20f0e00640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20cc0744e0 0x7f20cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.195+0000 7f20f0e00640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20cc0744e0 0x7f20cc0769f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f20d8034000 tx=0x7f20d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:31.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.198+0000 7f20e9600640 1 -- 172.21.15.19:0/638224087 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20e005fde0 con 0x7f20ec15df80 2024-04-25T11:31:31.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.448+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f20ec0d53a0 con 0x7f20ec15df80 2024-04-25T11:31:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.870+0000 7f20e9600640 1 -- 172.21.15.19:0/638224087 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v54) ==== 162+0+0 (secure 0 0 0) 0x7f20e005f6e0 con 0x7f20ec15df80 2024-04-25T11:31:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20cc0744e0 msgr2=0x7f20cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20cc0744e0 0x7f20cc0769f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f20d8034000 tx=0x7f20d802e000 comp rx=0 tx=0).stop 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 msgr2=0x7f20ec158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec158f00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d5f0 tx=0x7f20e000dab0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 shutdown_connections 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20cc0744e0 0x7f20cc0769f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20ec160890 0x7f20ec156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20ec164b00 0x7f20ec1ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 --2- 172.21.15.19:0/638224087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20ec15df80 0x7f20ec158f00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 >> 172.21.15.19:0/638224087 conn(0x7f20ec0d2bc0 msgr2=0x7f20ec0d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.874+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 shutdown_connections 2024-04-25T11:31:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:31.875+0000 7f20f4200640 1 -- 172.21.15.19:0/638224087 wait complete. 2024-04-25T11:31:31.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:32.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.183+0000 7fdc9b600640 1 Processor -- start 2024-04-25T11:31:32.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.184+0000 7fdc9b600640 1 -- start start 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.186+0000 7fdc9b600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc940cf5e0 0x7fdc940cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.186+0000 7fdc9b600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdc9406c5c0 0x7fdc9406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.186+0000 7fdc9b600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc940cff30 con 0x7fdc941443c0 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.186+0000 7fdc9b600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdc940708d0 con 0x7fdc940cf5e0 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.186+0000 7fdc9b600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdc9415f160 con 0x7fdc9406c5c0 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc93e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc940cf5e0 0x7fdc940cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc93e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc940cf5e0 0x7fdc940cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58762/0 (socket says 172.21.15.19:58762) 2024-04-25T11:31:32.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc93e00640 1 -- 172.21.15.19:0/502529593 learned_addr learned my addr 172.21.15.19:0/502529593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc99600640 1 --2- 172.21.15.19:0/502529593 >> v2:172.21.15.169:3300/0 conn(0x7fdc9406c5c0 0x7fdc9406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc93e00640 1 -- 172.21.15.19:0/502529593 >> v2:172.21.15.169:3300/0 conn(0x7fdc9406c5c0 msgr2=0x7fdc9406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc93e00640 1 --2- 172.21.15.19:0/502529593 >> v2:172.21.15.169:3300/0 conn(0x7fdc9406c5c0 0x7fdc9406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc93e00640 1 -- 172.21.15.19:0/502529593 >> v1:172.21.15.169:6789/0 conn(0x7fdc941443c0 legacy=0x7fdc9406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.187+0000 7fdc93e00640 1 -- 172.21.15.19:0/502529593 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9415f5f0 con 0x7fdc940cf5e0 2024-04-25T11:31:32.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.188+0000 7fdc93e00640 1 --2- 172.21.15.19:0/502529593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc940cf5e0 0x7fdc940cf9f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdc9415c250 tx=0x7fdc8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8e93c280d4a449 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:32.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.189+0000 7fdc93400640 1 -- 172.21.15.19:0/502529593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80037070 con 0x7fdc940cf5e0 2024-04-25T11:31:32.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.189+0000 7fdc93400640 1 -- 172.21.15.19:0/502529593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc80032440 con 0x7fdc940cf5e0 2024-04-25T11:31:32.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.189+0000 7fdc93400640 1 -- 172.21.15.19:0/502529593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80032810 con 0x7fdc940cf5e0 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.189+0000 7fdc9b600640 1 -- 172.21.15.19:0/502529593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc940cf5e0 msgr2=0x7fdc940cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.189+0000 7fdc9b600640 1 --2- 172.21.15.19:0/502529593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc940cf5e0 0x7fdc940cf9f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdc9415c250 tx=0x7fdc8002a230 comp rx=0 tx=0).stop 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.190+0000 7fdc9b600640 1 -- 172.21.15.19:0/502529593 shutdown_connections 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.190+0000 7fdc9b600640 1 --2- 172.21.15.19:0/502529593 >> v2:172.21.15.169:3300/0 conn(0x7fdc9406c5c0 0x7fdc9406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.190+0000 7fdc9b600640 1 --2- 172.21.15.19:0/502529593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc940cf5e0 0x7fdc940cf9f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.190+0000 7fdc9b600640 1 -- 172.21.15.19:0/502529593 >> 172.21.15.19:0/502529593 conn(0x7fdc940d1ae0 msgr2=0x7fdc940d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.190+0000 7fdc9b600640 1 -- 172.21.15.19:0/502529593 shutdown_connections 2024-04-25T11:31:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.190+0000 7fdc9b600640 1 -- 172.21.15.19:0/502529593 wait complete. 2024-04-25T11:31:32.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.192+0000 7fdc9b600640 1 Processor -- start 2024-04-25T11:31:32.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.192+0000 7fdc9b600640 1 -- start start 2024-04-25T11:31:32.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc9b600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc9406c5c0 0x7fdc94142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:32.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc9b600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc940cf5e0 0x7fdc941fa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:32.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc9b600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc941443c0 0x7fdc940b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:32.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc9b600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9415edf0 con 0x7fdc941443c0 2024-04-25T11:31:32.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc9b600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9415eb10 con 0x7fdc9406c5c0 2024-04-25T11:31:32.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc9b600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc941447d0 con 0x7fdc940cf5e0 2024-04-25T11:31:32.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc99600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc941443c0 0x7fdc940b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:32.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc99600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc941443c0 0x7fdc940b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58778/0 (socket says 172.21.15.19:58778) 2024-04-25T11:31:32.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc99600640 1 -- 172.21.15.19:0/3216226091 learned_addr learned my addr 172.21.15.19:0/3216226091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc93e00640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc940cf5e0 0x7fdc941fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.194+0000 7fdc98c00640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc9406c5c0 0x7fdc94142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc99600640 1 -- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc940cf5e0 msgr2=0x7fdc941fa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc99600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc940cf5e0 0x7fdc941fa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc99600640 1 -- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc9406c5c0 msgr2=0x7fdc94142e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc99600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc9406c5c0 0x7fdc94142e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc99600640 1 -- 172.21.15.19:0/3216226091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc941faa60 con 0x7fdc941443c0 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc99600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc941443c0 0x7fdc940b58c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c007d80 tx=0x7fdc8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc90c00640 1 -- 172.21.15.19:0/3216226091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c014420 con 0x7fdc941443c0 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc90c00640 1 -- 172.21.15.19:0/3216226091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c019070 con 0x7fdc941443c0 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc90c00640 1 -- 172.21.15.19:0/3216226091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c014800 con 0x7fdc941443c0 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.195+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc941ea160 con 0x7fdc941443c0 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.196+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc941ea950 con 0x7fdc941443c0 2024-04-25T11:31:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.197+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc94050740 con 0x7fdc941443c0 2024-04-25T11:31:32.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.198+0000 7fdc90c00640 1 -- 172.21.15.19:0/3216226091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc8c0149a0 con 0x7fdc941443c0 2024-04-25T11:31:32.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.199+0000 7fdc90c00640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc740744b0 0x7fdc740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:32.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.199+0000 7fdc90c00640 1 -- 172.21.15.19:0/3216226091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fdc8c095b90 con 0x7fdc941443c0 2024-04-25T11:31:32.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.200+0000 7fdc98c00640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc740744b0 0x7fdc740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.201+0000 7fdc98c00640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc740744b0 0x7fdc740769c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006fd0 tx=0x7fdc88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:32.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.204+0000 7fdc90c00640 1 -- 172.21.15.19:0/3216226091 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc941faa60 con 0x7fdc941443c0 2024-04-25T11:31:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.448+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc9406f950 con 0x7fdc941443c0 2024-04-25T11:31:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.449+0000 7fdc90c00640 1 -- 172.21.15.19:0/3216226091 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5181 (secure 0 0 0) 0x7fdc8c060220 con 0x7fdc941443c0 2024-04-25T11:31:32.440 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:32.440 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:31.877225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-25T11:31:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 55 2024-04-25T11:31:32.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc740744b0 msgr2=0x7fdc740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:32.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc740744b0 0x7fdc740769c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006fd0 tx=0x7fdc88008040 comp rx=0 tx=0).stop 2024-04-25T11:31:32.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc941443c0 msgr2=0x7fdc940b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:32.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc941443c0 0x7fdc940b58c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c007d80 tx=0x7fdc8c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 shutdown_connections 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc740744b0 0x7fdc740769c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc941443c0 0x7fdc940b58c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc940cf5e0 0x7fdc941fa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 --2- 172.21.15.19:0/3216226091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc9406c5c0 0x7fdc94142e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 >> 172.21.15.19:0/3216226091 conn(0x7fdc940d1ae0 msgr2=0x7fdc940b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 shutdown_connections 2024-04-25T11:31:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:32.452+0000 7fdc9b600640 1 -- 172.21.15.19:0/3216226091 wait complete. 2024-04-25T11:31:32.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:31:06.789413+0000', 'modified': '2024-04-25T11:31:31.877225+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 64, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6833/1883095429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 1883095429}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 1883095429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:31:32.873 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6486:up:replay 2024-04-25T11:31:33.705 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:31:33.714 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:31:33.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:34.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.171+0000 7fbdb6400640 1 Processor -- start 2024-04-25T11:31:34.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.171+0000 7fbdb6400640 1 -- start start 2024-04-25T11:31:34.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.173+0000 7fbdb6400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbdb0160360 0x7fbdb01647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:34.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.173+0000 7fbdb6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb00b5c90 0x7fbdb00b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:34.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.173+0000 7fbdb6400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb00b65e0 con 0x7fbdb0070680 2024-04-25T11:31:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.174+0000 7fbdb6400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb006d100 con 0x7fbdb0160360 2024-04-25T11:31:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.174+0000 7fbdb6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb006c7d0 con 0x7fbdb00b5c90 2024-04-25T11:31:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.174+0000 7fbdaf400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbdb0070680 0x7fbdb00af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59812/0 (socket says 172.21.15.19:59812) 2024-04-25T11:31:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.174+0000 7fbdaf400640 1 -- 172.21.15.19:0/2761486770 learned_addr learned my addr 172.21.15.19:0/2761486770 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.174+0000 7fbdaea00640 1 --2- 172.21.15.19:0/2761486770 >> v2:172.21.15.19:3300/0 conn(0x7fbdb0160360 0x7fbdb01647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.175+0000 7fbdaea00640 1 -- 172.21.15.19:0/2761486770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb00b5c90 msgr2=0x7fbdb00b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.175+0000 7fbdaea00640 1 --2- 172.21.15.19:0/2761486770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb00b5c90 0x7fbdb00b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.175+0000 7fbdaea00640 1 -- 172.21.15.19:0/2761486770 >> v1:172.21.15.19:6789/0 conn(0x7fbdb0070680 legacy=0x7fbdb00af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.175+0000 7fbdaea00640 1 -- 172.21.15.19:0/2761486770 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb006cab0 con 0x7fbdb0160360 2024-04-25T11:31:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.175+0000 7fbdae000640 1 -- 172.21.15.19:0/2761486770 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879594815 0 0) 0x7fbda0001490 con 0x7fbdb0070680 2024-04-25T11:31:34.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.176+0000 7fbdaea00640 1 --2- 172.21.15.19:0/2761486770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb0160360 0x7fbdb01647f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0097a0 tx=0x7fbd9c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7fb6ef6d881fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.176+0000 7fbdae000640 1 -- 172.21.15.19:0/2761486770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c037070 con 0x7fbdb0160360 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.176+0000 7fbdae000640 1 -- 172.21.15.19:0/2761486770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c02adb0 con 0x7fbdb0160360 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.176+0000 7fbdae000640 1 -- 172.21.15.19:0/2761486770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c032660 con 0x7fbdb0160360 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 -- 172.21.15.19:0/2761486770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb0160360 msgr2=0x7fbdb01647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2761486770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb0160360 0x7fbdb01647f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0097a0 tx=0x7fbd9c02a060 comp rx=0 tx=0).stop 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 -- 172.21.15.19:0/2761486770 shutdown_connections 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2761486770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb00b5c90 0x7fbdb00b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2761486770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb0160360 0x7fbdb01647f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 -- 172.21.15.19:0/2761486770 >> 172.21.15.19:0/2761486770 conn(0x7fbdb00d1ae0 msgr2=0x7fbdb00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 -- 172.21.15.19:0/2761486770 shutdown_connections 2024-04-25T11:31:34.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.177+0000 7fbdb6400640 1 -- 172.21.15.19:0/2761486770 wait complete. 2024-04-25T11:31:34.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.180+0000 7fbdb6400640 1 Processor -- start 2024-04-25T11:31:34.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.180+0000 7fbdb6400640 1 -- start start 2024-04-25T11:31:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.182+0000 7fbdb6400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdb0070680 0x7fbdb015f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.182+0000 7fbdb6400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb00b5c90 0x7fbdb01550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.182+0000 7fbdb6400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb0160360 0x7fbdb015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.182+0000 7fbdb6400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb00afbb0 con 0x7fbdb00b5c90 2024-04-25T11:31:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.182+0000 7fbdb6400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb006fcc0 con 0x7fbdb0070680 2024-04-25T11:31:34.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.182+0000 7fbdb6400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb00afd30 con 0x7fbdb0160360 2024-04-25T11:31:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb00b5c90 0x7fbdb01550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdaea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb00b5c90 0x7fbdb01550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36716/0 (socket says 172.21.15.19:36716) 2024-04-25T11:31:34.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdaea00640 1 -- 172.21.15.19:0/2302018661 learned_addr learned my addr 172.21.15.19:0/2302018661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:34.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdafe00640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb0160360 0x7fbdb015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:34.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdaf400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdb0070680 0x7fbdb015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:34.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdaea00640 1 -- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb0160360 msgr2=0x7fbdb015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:34.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdaea00640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb0160360 0x7fbdb015c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.183+0000 7fbdaea00640 1 -- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdb0070680 msgr2=0x7fbdb015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:34.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.184+0000 7fbdaea00640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdb0070680 0x7fbdb015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.184+0000 7fbdaea00640 1 -- 172.21.15.19:0/2302018661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb0155c30 con 0x7fbdb00b5c90 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.184+0000 7fbdaea00640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb00b5c90 0x7fbdb01550c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c035040 tx=0x7fbd9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.184+0000 7fbda7400640 1 -- 172.21.15.19:0/2302018661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c037040 con 0x7fbdb00b5c90 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.184+0000 7fbda7400640 1 -- 172.21.15.19:0/2302018661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c004030 con 0x7fbdb00b5c90 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.184+0000 7fbda7400640 1 -- 172.21.15.19:0/2302018661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c006e10 con 0x7fbdb00b5c90 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.184+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb0155f10 con 0x7fbdb00b5c90 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.185+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb0156ed0 con 0x7fbdb00b5c90 2024-04-25T11:31:34.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.186+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb0050740 con 0x7fbdb00b5c90 2024-04-25T11:31:34.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.187+0000 7fbda7400640 1 -- 172.21.15.19:0/2302018661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd9c03d650 con 0x7fbdb00b5c90 2024-04-25T11:31:34.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.188+0000 7fbda7400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd900744b0 0x7fbd900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:34.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.188+0000 7fbda7400640 1 -- 172.21.15.19:0/2302018661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fbd9c0b5830 con 0x7fbdb00b5c90 2024-04-25T11:31:34.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.189+0000 7fbdaf400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd900744b0 0x7fbd900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:34.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.190+0000 7fbdaf400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd900744b0 0x7fbd900769c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbda0007ed0 tx=0x7fbda0007e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:34.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.194+0000 7fbda7400640 1 -- 172.21.15.19:0/2302018661 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd9c042020 con 0x7fbdb00b5c90 2024-04-25T11:31:34.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.436+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdb01559c0 con 0x7fbdb00b5c90 2024-04-25T11:31:34.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.437+0000 7fbda7400640 1 -- 172.21.15.19:0/2302018661 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5181 (secure 0 0 0) 0x7fbd9c084e20 con 0x7fbdb00b5c90 2024-04-25T11:31:34.428 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":57,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:32.899061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-25T11:31:34.428 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 57 2024-04-25T11:31:34.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.439+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd900744b0 msgr2=0x7fbd900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:34.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.439+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd900744b0 0x7fbd900769c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbda0007ed0 tx=0x7fbda0007e60 comp rx=0 tx=0).stop 2024-04-25T11:31:34.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.439+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb00b5c90 msgr2=0x7fbdb01550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:34.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.439+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb00b5c90 0x7fbdb01550c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c035040 tx=0x7fbd9c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 shutdown_connections 2024-04-25T11:31:34.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd900744b0 0x7fbd900769c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdb0160360 0x7fbdb015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbdb00b5c90 0x7fbdb01550c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 --2- 172.21.15.19:0/2302018661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdb0070680 0x7fbdb015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:34.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 >> 172.21.15.19:0/2302018661 conn(0x7fbdb00d1ae0 msgr2=0x7fbdb00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:34.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 shutdown_connections 2024-04-25T11:31:34.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:34.440+0000 7fbdb6400640 1 -- 172.21.15.19:0/2302018661 wait complete. 2024-04-25T11:31:34.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:31:06.789413+0000', 'modified': '2024-04-25T11:31:32.899061+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 64, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.19:6833/1883095429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 1883095429}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 1883095429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:31:34.857 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6486:up:rejoin 2024-04-25T11:31:35.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:36.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.151+0000 7f42a2600640 1 Processor -- start 2024-04-25T11:31:36.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.151+0000 7f42a2600640 1 -- start start 2024-04-25T11:31:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.153+0000 7f42a2600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f429c0b4ea0 0x7f429c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.153+0000 7f42a2600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429c0d1230 con 0x7f429c144370 2024-04-25T11:31:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.153+0000 7f42a2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429c0d06b0 con 0x7f429c06c5c0 2024-04-25T11:31:36.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.153+0000 7f42a2600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f429c0d0990 con 0x7f429c0b4ea0 2024-04-25T11:31:36.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.154+0000 7f429be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f429c0b4ea0 0x7f429c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:36.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.154+0000 7f42a1c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f429c144370 0x7f429c15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59828/0 (socket says 172.21.15.19:59828) 2024-04-25T11:31:36.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.154+0000 7f42a1c00640 1 -- 172.21.15.19:0/639801868 learned_addr learned my addr 172.21.15.19:0/639801868 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:36.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.154+0000 7f429be00640 1 -- 172.21.15.19:0/639801868 >> v1:172.21.15.169:6789/0 conn(0x7f429c06c5c0 legacy=0x7f429c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:36.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.154+0000 7f429be00640 1 -- 172.21.15.19:0/639801868 >> v1:172.21.15.19:6789/0 conn(0x7f429c144370 legacy=0x7f429c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:36.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.154+0000 7f429be00640 1 -- 172.21.15.19:0/639801868 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c0d0c70 con 0x7f429c0b4ea0 2024-04-25T11:31:36.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.154+0000 7f429b400640 1 -- 172.21.15.19:0/639801868 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335630204 0 0) 0x7f429c0d1230 con 0x7f429c144370 2024-04-25T11:31:36.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.155+0000 7f429be00640 1 --2- 172.21.15.19:0/639801868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c0b4ea0 0x7f429c0cfa70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f42840097a0 tx=0x7f428402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4db945cc46792a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:36.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.156+0000 7f429b400640 1 -- 172.21.15.19:0/639801868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284037070 con 0x7f429c0b4ea0 2024-04-25T11:31:36.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.156+0000 7f429b400640 1 -- 172.21.15.19:0/639801868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4284032440 con 0x7f429c0b4ea0 2024-04-25T11:31:36.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.156+0000 7f429b400640 1 -- 172.21.15.19:0/639801868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284032810 con 0x7f429c0b4ea0 2024-04-25T11:31:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.157+0000 7f42a2600640 1 -- 172.21.15.19:0/639801868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c0b4ea0 msgr2=0x7f429c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.157+0000 7f42a2600640 1 --2- 172.21.15.19:0/639801868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c0b4ea0 0x7f429c0cfa70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f42840097a0 tx=0x7f428402a230 comp rx=0 tx=0).stop 2024-04-25T11:31:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.157+0000 7f42a2600640 1 -- 172.21.15.19:0/639801868 shutdown_connections 2024-04-25T11:31:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.157+0000 7f42a2600640 1 --2- 172.21.15.19:0/639801868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c0b4ea0 0x7f429c0cfa70 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.157+0000 7f42a2600640 1 -- 172.21.15.19:0/639801868 >> 172.21.15.19:0/639801868 conn(0x7f429c0d1720 msgr2=0x7f429c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.157+0000 7f42a2600640 1 -- 172.21.15.19:0/639801868 shutdown_connections 2024-04-25T11:31:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.157+0000 7f42a2600640 1 -- 172.21.15.19:0/639801868 wait complete. 2024-04-25T11:31:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.159+0000 7f42a2600640 1 Processor -- start 2024-04-25T11:31:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.159+0000 7f42a2600640 1 -- start start 2024-04-25T11:31:36.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c06c5c0 0x7f429c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:36.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c144370 0x7f429c15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:36.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c135ad0 0x7f429c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:36.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0d0450 con 0x7f429c06c5c0 2024-04-25T11:31:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0d02d0 con 0x7f429c144370 2024-04-25T11:31:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f429c147760 con 0x7f429c135ad0 2024-04-25T11:31:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c06c5c0 0x7f429c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c06c5c0 0x7f429c0cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36730/0 (socket says 172.21.15.19:36730) 2024-04-25T11:31:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.161+0000 7f42a1200640 1 -- 172.21.15.19:0/594948084 learned_addr learned my addr 172.21.15.19:0/594948084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a1200640 1 -- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c135ad0 msgr2=0x7f429c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a1200640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c135ad0 0x7f429c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f429be00640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c144370 0x7f429c15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a1200640 1 -- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c144370 msgr2=0x7f429c15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a1200640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c144370 0x7f429c15b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a1200640 1 -- 172.21.15.19:0/594948084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c1627c0 con 0x7f429c06c5c0 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a1200640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c06c5c0 0x7f429c0cc7b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c400 tx=0x7f428c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f4298c00640 1 -- 172.21.15.19:0/594948084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c017070 con 0x7f429c06c5c0 2024-04-25T11:31:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f4298c00640 1 -- 172.21.15.19:0/594948084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f428c0052d0 con 0x7f429c06c5c0 2024-04-25T11:31:36.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f4298c00640 1 -- 172.21.15.19:0/594948084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c003830 con 0x7f429c06c5c0 2024-04-25T11:31:36.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429c0cccf0 con 0x7f429c06c5c0 2024-04-25T11:31:36.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.162+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429c1458c0 con 0x7f429c06c5c0 2024-04-25T11:31:36.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.164+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429c050740 con 0x7f429c06c5c0 2024-04-25T11:31:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.165+0000 7f4298c00640 1 -- 172.21.15.19:0/594948084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f428c01a070 con 0x7f429c06c5c0 2024-04-25T11:31:36.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.166+0000 7f4298c00640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42780744b0 0x7f42780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:36.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.166+0000 7f4298c00640 1 -- 172.21.15.19:0/594948084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f428c094620 con 0x7f429c06c5c0 2024-04-25T11:31:36.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.167+0000 7f429be00640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42780744b0 0x7f42780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:36.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.168+0000 7f429be00640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42780744b0 0x7f42780769c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4284034000 tx=0x7f428402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:36.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.170+0000 7f4298c00640 1 -- 172.21.15.19:0/594948084 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f428c05ed50 con 0x7f429c06c5c0 2024-04-25T11:31:36.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.421+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f429c1419f0 con 0x7f429c06c5c0 2024-04-25T11:31:36.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.422+0000 7f4298c00640 1 -- 172.21.15.19:0/594948084 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5188 (secure 0 0 0) 0x7f428c05e650 con 0x7f429c06c5c0 2024-04-25T11:31:36.413 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:36.413 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:34.909423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":4}]} 2024-04-25T11:31:36.413 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T11:31:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.425+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42780744b0 msgr2=0x7f42780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.425+0000 7f42a2600640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42780744b0 0x7f42780769c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4284034000 tx=0x7f428402e000 comp rx=0 tx=0).stop 2024-04-25T11:31:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.425+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c06c5c0 msgr2=0x7f429c0cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:36.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.425+0000 7f42a2600640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c06c5c0 0x7f429c0cc7b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c400 tx=0x7f428c00c8c0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 shutdown_connections 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42780744b0 0x7f42780769c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c144370 0x7f429c15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c135ad0 0x7f429c0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 --2- 172.21.15.19:0/594948084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c06c5c0 0x7f429c0cc7b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 >> 172.21.15.19:0/594948084 conn(0x7f429c0d1720 msgr2=0x7f429c0b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 shutdown_connections 2024-04-25T11:31:36.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:36.426+0000 7f42a2600640 1 -- 172.21.15.19:0/594948084 wait complete. 2024-04-25T11:31:36.438 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:31:36.447+0000 7f89bee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:31:36.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:31:06.789413+0000', 'modified': '2024-04-25T11:31:34.909423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 64, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6486}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6486': {'gid': 6486, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6833/1883095429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 1883095429}, {'type': 'v1', 'addr': '172.21.15.19:6833', 'nonce': 1883095429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6486, 'qdb_cluster': [6486]} 2024-04-25T11:31:36.846 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:31:36.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:31:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.153+0000 7f6b67800640 1 Processor -- start 2024-04-25T11:31:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.153+0000 7f6b67800640 1 -- start start 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.155+0000 7f6b67800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b6014ad60 0x7f6b60070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.155+0000 7f6b67800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b600b5e40 0x7f6b600b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.155+0000 7f6b67800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b600b6790 con 0x7f6b601681d0 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.155+0000 7f6b67800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6b600b4af0 con 0x7f6b600b5e40 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.155+0000 7f6b67800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b600b4dd0 con 0x7f6b6014ad60 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.155+0000 7f6b5fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b600b5e40 0x7f6b600b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.156+0000 7f6b5fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b600b5e40 0x7f6b600b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36740/0 (socket says 172.21.15.19:36740) 2024-04-25T11:31:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.156+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2279127781 learned_addr learned my addr 172.21.15.19:0/2279127781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.156+0000 7f6b64e00640 1 --2- 172.21.15.19:0/2279127781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b6014ad60 0x7f6b60070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.156+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2279127781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b6014ad60 msgr2=0x7f6b60070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.156+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/2279127781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b6014ad60 0x7f6b60070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.156+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2279127781 >> v1:172.21.15.169:6789/0 conn(0x7f6b601681d0 legacy=0x7f6b601685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.156+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2279127781 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b600b50b0 con 0x7f6b600b5e40 2024-04-25T11:31:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.157+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/2279127781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b600b5e40 0x7f6b600b6250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6b60071410 tx=0x7f6b4c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22d50b6d16a0feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.158+0000 7f6b5f400640 1 -- 172.21.15.19:0/2279127781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c037070 con 0x7f6b600b5e40 2024-04-25T11:31:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.158+0000 7f6b5f400640 1 -- 172.21.15.19:0/2279127781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4c032440 con 0x7f6b600b5e40 2024-04-25T11:31:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.158+0000 7f6b5f400640 1 -- 172.21.15.19:0/2279127781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c032810 con 0x7f6b600b5e40 2024-04-25T11:31:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.158+0000 7f6b67800640 1 -- 172.21.15.19:0/2279127781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b600b5e40 msgr2=0x7f6b600b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.158+0000 7f6b67800640 1 --2- 172.21.15.19:0/2279127781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b600b5e40 0x7f6b600b6250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6b60071410 tx=0x7f6b4c02a160 comp rx=0 tx=0).stop 2024-04-25T11:31:37.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.159+0000 7f6b67800640 1 -- 172.21.15.19:0/2279127781 shutdown_connections 2024-04-25T11:31:37.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.159+0000 7f6b67800640 1 --2- 172.21.15.19:0/2279127781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b600b5e40 0x7f6b600b6250 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.159+0000 7f6b67800640 1 --2- 172.21.15.19:0/2279127781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b6014ad60 0x7f6b60070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.159+0000 7f6b67800640 1 -- 172.21.15.19:0/2279127781 >> 172.21.15.19:0/2279127781 conn(0x7f6b600d1ae0 msgr2=0x7f6b600d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.159+0000 7f6b67800640 1 -- 172.21.15.19:0/2279127781 shutdown_connections 2024-04-25T11:31:37.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.159+0000 7f6b67800640 1 -- 172.21.15.19:0/2279127781 wait complete. 2024-04-25T11:31:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.161+0000 7f6b67800640 1 Processor -- start 2024-04-25T11:31:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.161+0000 7f6b67800640 1 -- start start 2024-04-25T11:31:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b67800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b600b5e40 0x7f6b60165a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b67800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6014ad60 0x7f6b6013b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b67800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601681d0 0x7f6b6013bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b67800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60070550 con 0x7f6b6014ad60 2024-04-25T11:31:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b67800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b600b5670 con 0x7f6b601681d0 2024-04-25T11:31:37.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b67800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b600706d0 con 0x7f6b600b5e40 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6014ad60 0x7f6b6013b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6014ad60 0x7f6b6013b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36742/0 (socket says 172.21.15.19:36742) 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2140866979 learned_addr learned my addr 172.21.15.19:0/2140866979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b64e00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b600b5e40 0x7f6b60165a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.163+0000 7f6b65800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601681d0 0x7f6b6013bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b600b5e40 msgr2=0x7f6b60165a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b600b5e40 0x7f6b60165a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601681d0 msgr2=0x7f6b6013bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601681d0 0x7f6b6013bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5fe00640 1 -- 172.21.15.19:0/2140866979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b60165f60 con 0x7f6b6014ad60 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b64e00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b600b5e40 0x7f6b60165a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b65800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601681d0 0x7f6b6013bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:37.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6014ad60 0x7f6b6013b750 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c000c00 tx=0x7f6b4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5cc00640 1 -- 172.21.15.19:0/2140866979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c041020 con 0x7f6b6014ad60 2024-04-25T11:31:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5cc00640 1 -- 172.21.15.19:0/2140866979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4c030970 con 0x7f6b6014ad60 2024-04-25T11:31:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b60166470 con 0x7f6b6014ad60 2024-04-25T11:31:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.164+0000 7f6b5cc00640 1 -- 172.21.15.19:0/2140866979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c037040 con 0x7f6b6014ad60 2024-04-25T11:31:37.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.165+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b60166a30 con 0x7f6b6014ad60 2024-04-25T11:31:37.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.167+0000 7f6b5cc00640 1 -- 172.21.15.19:0/2140866979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b4c044020 con 0x7f6b6014ad60 2024-04-25T11:31:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.168+0000 7f6b5cc00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b400744e0 0x7f6b400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.168+0000 7f6b5cc00640 1 -- 172.21.15.19:0/2140866979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f6b4c0b5c90 con 0x7f6b6014ad60 2024-04-25T11:31:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.169+0000 7f6b6e96b740 1 Processor -- start 2024-04-25T11:31:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.169+0000 7f6b6e96b740 1 -- start start 2024-04-25T11:31:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.169+0000 7f6b64e00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b400744e0 0x7f6b400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.170+0000 7f6b64e00640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b400744e0 0x7f6b400769f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6b600b1590 tx=0x7f6b54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.170+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.170+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cfd4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.170+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.170+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56321cef3600 con 0x56321cbcc590 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56321ce90100 con 0x56321cbed270 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56321ceef5c0 con 0x56321cbc43b0 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36746/0 (socket says 172.21.15.19:36746) 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 -- 172.21.15.19:0/2355112272 learned_addr learned my addr 172.21.15.19:0/2355112272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cfd4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b64e00640 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 -- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 msgr2=0x56321cfd4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cfd4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 -- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 msgr2=0x56321cfba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b65800640 1 -- 172.21.15.19:0/2355112272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321cc052d0 con 0x56321cbcc590 2024-04-25T11:31:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cfd4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.171+0000 7f6b64e00640 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.172+0000 7f6b65800640 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd52b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007f60 tx=0x7f6b58007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120832576821a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.172+0000 7f6b51600640 1 -- 172.21.15.19:0/2355112272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58014070 con 0x56321cbcc590 2024-04-25T11:31:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.172+0000 7f6b51600640 1 -- 172.21.15.19:0/2355112272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5800f850 con 0x56321cbcc590 2024-04-25T11:31:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.172+0000 7f6b51600640 1 -- 172.21.15.19:0/2355112272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800fc40 con 0x56321cbcc590 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 -- 172.21.15.19:0/2355112272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 msgr2=0x56321cfd52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd52b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007f60 tx=0x7f6b58007f90 comp rx=0 tx=0).stop 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 -- 172.21.15.19:0/2355112272 shutdown_connections 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd52b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cfd4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/2355112272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 -- 172.21.15.19:0/2355112272 >> 172.21.15.19:0/2355112272 conn(0x56321cbe13c0 msgr2=0x56321cf8f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 -- 172.21.15.19:0/2355112272 shutdown_connections 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.173+0000 7f6b6e96b740 1 -- 172.21.15.19:0/2355112272 wait complete. 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.174+0000 7f6b6e96b740 1 -- start start 2024-04-25T11:31:37.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.174+0000 7f6b6e96b740 1 Processor -- start 2024-04-25T11:31:37.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cf8e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfd0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56321ce7af70 con 0x56321cbcc590 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56321cfd2ff0 con 0x56321cbed270 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56321cfbd070 con 0x56321cbc43b0 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b6e96b740 10 client.0 _ll_register_callbacks cb 0x56321cee9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36756/0 (socket says 172.21.15.19:36756) 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.176+0000 7f6b5fe00640 1 -- 172.21.15.19:0/3342353866 learned_addr learned my addr 172.21.15.19:0/3342353866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b64e00640 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cf8e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b5fe00640 1 -- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 msgr2=0x56321cf8e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b65800640 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfd0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cf8e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b5fe00640 1 -- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 msgr2=0x56321cfd0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfd0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b5fe00640 1 -- 172.21.15.19:0/3342353866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321cccf6b0 con 0x56321cbcc590 2024-04-25T11:31:37.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd0a20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c07f9b0 tx=0x7f6b4c030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ebe37c199b9ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b46000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b46000640 1 -- 172.21.15.19:0/3342353866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c099210 con 0x56321cbcc590 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b46000640 1 -- 172.21.15.19:0/3342353866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f6b4c07ef10 con 0x56321cbcc590 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b5800f850 con 0x56321cbcc590 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b46000640 1 -- 172.21.15.19:0/3342353866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6b4c0803b0 con 0x56321cbcc590 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b6e96b740 10 client.6534 fetch_fsmap learned FSMap version 58 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.177+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x56321cfbdb60 con 0x56321cbcc590 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b46000640 1 -- 172.21.15.19:0/3342353866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5800f850 con 0x56321cbcc590 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b46000640 1 -- 172.21.15.19:0/3342353866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f6b4c0af020 con 0x56321cbcc590 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b6e96b740 10 client.6534 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b6e96b740 10 client.6534 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b6e96b740 20 client.6534 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321cbabe40 0x56321cfbb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b6e96b740 4 client.6534 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:31:37.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 --> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321cf01210 con 0x56321cbabe40 2024-04-25T11:31:37.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.178+0000 7f6b64e00640 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321cbabe40 0x56321cfbb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.179+0000 7f6b64e00640 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321cbabe40 0x56321cfbb310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d684f52411c6 server_cookie=aaac448dfce3374e in_seq=0 out_seq=0 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.179+0000 7f6b46000640 10 client.6534 ms_handle_connect on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b46000640 1 -- 172.21.15.19:0/3342353866 <== mds.0 v2:172.21.15.19:6832/1883095429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56321cf01210 con 0x56321cbabe40 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b46000640 10 client.6534 handle_command_reply: tid=0 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 client.6534 shutdown 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 msgr2=0x56321cfd0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd0a20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c07f9b0 tx=0x7f6b4c030f90 comp rx=0 tx=0).stop 2024-04-25T11:31:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 shutdown_connections 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbed270 0x56321cfd0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321cbabe40 0x56321cfbb310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfd0a20 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/3342353866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbc43b0 0x56321cf8e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b46000640 0 client.6534 ms_handle_reset on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.180+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 >> 172.21.15.19:0/3342353866 conn(0x56321cbe13c0 msgr2=0x56321cf8f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.181+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 shutdown_connections 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.181+0000 7f6b6e96b740 1 -- 172.21.15.19:0/3342353866 wait complete. 2024-04-25T11:31:37.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.181+0000 7f6b6e96b740 20 client.6534 trim_cache size 0 max 16384 2024-04-25T11:31:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.201+0000 7f6b6e96b740 1 Processor -- start 2024-04-25T11:31:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.201+0000 7f6b6e96b740 1 -- start start 2024-04-25T11:31:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbc43b0 0x56321cfd0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbcc590 0x56321cfb52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321cfb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b602064d0 con 0x56321cbc43b0 2024-04-25T11:31:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60149600 con 0x56321cbcc590 2024-04-25T11:31:37.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b60149c50 con 0x56321cbed270 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b64e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbc43b0 0x56321cfd0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b64e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbc43b0 0x56321cfd0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36772/0 (socket says 172.21.15.19:36772) 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.203+0000 7f6b64e00640 1 -- 172.21.15.19:0/1019335503 learned_addr learned my addr 172.21.15.19:0/1019335503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b65800640 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321cfb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbcc590 0x56321cfb52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b64e00640 1 -- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 msgr2=0x56321cfb5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b64e00640 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321cfb5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b64e00640 1 -- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbcc590 msgr2=0x56321cfb52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b64e00640 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbcc590 0x56321cfb52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b64e00640 1 -- 172.21.15.19:0/1019335503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321cf8d6f0 con 0x56321cbc43b0 2024-04-25T11:31:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbcc590 0x56321cfb52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b65800640 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321cfb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b64e00640 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbc43b0 0x56321cfd0f60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x56321cfcff40 tx=0x7f6b54013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faef65b60a435c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.204+0000 7f6b50c00640 1 -- 172.21.15.19:0/1019335503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54012b40 con 0x56321cbc43b0 2024-04-25T11:31:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.205+0000 7f6b50c00640 1 -- 172.21.15.19:0/1019335503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b54012ce0 con 0x56321cbc43b0 2024-04-25T11:31:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.205+0000 7f6b50c00640 1 -- 172.21.15.19:0/1019335503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b540253e0 con 0x56321cbc43b0 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.205+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1019335503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbc43b0 msgr2=0x56321cfd0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.205+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbc43b0 0x56321cfd0f60 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x56321cfcff40 tx=0x7f6b54013000 comp rx=0 tx=0).stop 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.206+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1019335503 shutdown_connections 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.206+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbcc590 0x56321cfb52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.206+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321cfb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.206+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1019335503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbc43b0 0x56321cfd0f60 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.206+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1019335503 >> 172.21.15.19:0/1019335503 conn(0x56321cbe13c0 msgr2=0x56321cf8f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.206+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1019335503 shutdown_connections 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.206+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1019335503 wait complete. 2024-04-25T11:31:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.207+0000 7f6b6e96b740 1 -- start start 2024-04-25T11:31:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.207+0000 7f6b6e96b740 1 Processor -- start 2024-04-25T11:31:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.208+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbc43b0 0x56321cfbafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.208+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfbb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.208+0000 7f6b6e96b740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321d051710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.208+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56321cfb3810 con 0x56321cbcc590 2024-04-25T11:31:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.208+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56321cfd2a00 con 0x56321cbc43b0 2024-04-25T11:31:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.208+0000 7f6b6e96b740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56321cfcfd90 con 0x56321cbed270 2024-04-25T11:31:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.208+0000 7f6b6e96b740 10 client.0 _ll_register_callbacks cb 0x7f6b4c0091c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfbb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfbb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36776/0 (socket says 172.21.15.19:36776) 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 -- 172.21.15.19:0/1364767372 learned_addr learned my addr 172.21.15.19:0/1364767372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 -- 172.21.15.19:0/1364767372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 msgr2=0x56321d051710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321d051710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 -- 172.21.15.19:0/1364767372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbc43b0 msgr2=0x56321cfbafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbc43b0 0x56321cfbafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 -- 172.21.15.19:0/1364767372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321cfb57c0 con 0x56321cbcc590 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b5fe00640 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfbb4e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c07cf20 tx=0x7f6b4c0b3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4496283aacee7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b46000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.209+0000 7f6b46000640 1 -- 172.21.15.19:0/1364767372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c099210 con 0x56321cbcc590 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b46000640 1 -- 172.21.15.19:0/1364767372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f6b4c07b620 con 0x56321cbcc590 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b54012ce0 con 0x56321cbcc590 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b46000640 1 -- 172.21.15.19:0/1364767372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6b4c0a6050 con 0x56321cbcc590 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 10 client.6540 fetch_fsmap learned FSMap version 58 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x56321cfb57c0 con 0x56321cbcc590 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b46000640 1 -- 172.21.15.19:0/1364767372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b54012ce0 con 0x56321cbcc590 2024-04-25T11:31:37.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b46000640 1 -- 172.21.15.19:0/1364767372 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 3494+0+0 (secure 0 0 0) 0x7f6b4c0a9080 con 0x56321cbcc590 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 10 client.6540 fetch_fsmap finished waiting for FSMap version 58 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 10 client.6540 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 20 client.6540 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321d0520f0 0x56321cf89270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 4 client.6540 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.210+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 --> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] -- command(tid 0: {"prefix": "status"}) -- 0x56321cefb080 con 0x56321d0520f0 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.211+0000 7f6b64e00640 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321d0520f0 0x56321cf89270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.211+0000 7f6b64e00640 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321d0520f0 0x56321cf89270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93eb5757afbb6910 server_cookie=d35eff2d360d74f5 in_seq=0 out_seq=0 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.211+0000 7f6b46000640 10 client.6540 ms_handle_connect on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.211+0000 7f6b46000640 1 -- 172.21.15.19:0/1364767372 <== mds.0 v2:172.21.15.19:6832/1883095429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56321cefb080 con 0x56321d0520f0 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.211+0000 7f6b46000640 10 client.6540 handle_command_reply: tid=0 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.211+0000 7f6b6e96b740 1 client.6540 shutdown 2024-04-25T11:31:37.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 msgr2=0x56321cfbb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfbb4e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c07cf20 tx=0x7f6b4c0b3f90 comp rx=0 tx=0).stop 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 shutdown_connections 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x56321d0520f0 0x56321cf89270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56321cbed270 0x56321d051710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56321cbcc590 0x56321cfbb4e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 --2- 172.21.15.19:0/1364767372 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56321cbc43b0 0x56321cfbafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b46000640 0 client.6540 ms_handle_reset on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 >> 172.21.15.19:0/1364767372 conn(0x56321cbe13c0 msgr2=0x56321cfd05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 shutdown_connections 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 1 -- 172.21.15.19:0/1364767372 wait complete. 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.212+0000 7f6b6e96b740 20 client.6540 trim_cache size 0 max 16384 2024-04-25T11:31:37.205 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6486, 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.3308154209999996, 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 58, 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 64, 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 64, 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 11.689247761000001 2024-04-25T11:31:37.206 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:31:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b400744e0 msgr2=0x7f6b400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b400744e0 0x7f6b400769f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6b600b1590 tx=0x7f6b54008040 comp rx=0 tx=0).stop 2024-04-25T11:31:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6014ad60 msgr2=0x7f6b6013b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6014ad60 0x7f6b6013b750 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c000c00 tx=0x7f6b4c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 shutdown_connections 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b400744e0 0x7f6b400769f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b601681d0 0x7f6b6013bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b6014ad60 0x7f6b6013b750 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 --2- 172.21.15.19:0/2140866979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b600b5e40 0x7f6b60165a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 >> 172.21.15.19:0/2140866979 conn(0x7f6b600d1ae0 msgr2=0x7f6b60161080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 shutdown_connections 2024-04-25T11:31:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.213+0000 7f6b67800640 1 -- 172.21.15.19:0/2140866979 wait complete. 2024-04-25T11:31:37.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.519+0000 7f351ee00640 1 Processor -- start 2024-04-25T11:31:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.519+0000 7f351ee00640 1 -- start start 2024-04-25T11:31:37.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.521+0000 7f351ee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35181400a0 0x7f35180b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f351ee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3518142730 0x7f3518142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f351ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35181431d0 con 0x7f3518070270 2024-04-25T11:31:37.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f351ee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f35181434b0 con 0x7f3518142730 2024-04-25T11:31:37.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f351ee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f35180b6090 con 0x7f35181400a0 2024-04-25T11:31:37.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f3517400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3518142730 0x7f3518142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f3517400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3518142730 0x7f3518142b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36790/0 (socket says 172.21.15.19:36790) 2024-04-25T11:31:37.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f3517400640 1 -- 172.21.15.19:0/509791004 learned_addr learned my addr 172.21.15.19:0/509791004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.522+0000 7f3517e00640 1 --2- 172.21.15.19:0/509791004 >> v2:172.21.15.169:3300/0 conn(0x7f35181400a0 0x7f35180b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.523+0000 7f3517400640 1 -- 172.21.15.19:0/509791004 >> v2:172.21.15.169:3300/0 conn(0x7f35181400a0 msgr2=0x7f35180b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.523+0000 7f3517400640 1 --2- 172.21.15.19:0/509791004 >> v2:172.21.15.169:3300/0 conn(0x7f35181400a0 0x7f35180b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.523+0000 7f3517400640 1 -- 172.21.15.19:0/509791004 >> v1:172.21.15.169:6789/0 conn(0x7f3518070270 legacy=0x7f3518070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:37.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.523+0000 7f3517400640 1 -- 172.21.15.19:0/509791004 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35180b6370 con 0x7f3518142730 2024-04-25T11:31:37.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.524+0000 7f3517400640 1 --2- 172.21.15.19:0/509791004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f3518142b40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3518071510 tx=0x7f350402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e27ecb4e5e3742 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.524+0000 7f3516a00640 1 -- 172.21.15.19:0/509791004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504036070 con 0x7f3518142730 2024-04-25T11:31:37.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.524+0000 7f3516a00640 1 -- 172.21.15.19:0/509791004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3504031600 con 0x7f3518142730 2024-04-25T11:31:37.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.524+0000 7f3516a00640 1 -- 172.21.15.19:0/509791004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35040319d0 con 0x7f3518142730 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.525+0000 7f351ee00640 1 -- 172.21.15.19:0/509791004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 msgr2=0x7f3518142b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.525+0000 7f351ee00640 1 --2- 172.21.15.19:0/509791004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f3518142b40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3518071510 tx=0x7f350402a6f0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.525+0000 7f351ee00640 1 -- 172.21.15.19:0/509791004 shutdown_connections 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.525+0000 7f351ee00640 1 --2- 172.21.15.19:0/509791004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f3518142b40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.525+0000 7f351ee00640 1 --2- 172.21.15.19:0/509791004 >> v2:172.21.15.169:3300/0 conn(0x7f35181400a0 0x7f35180b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.525+0000 7f351ee00640 1 -- 172.21.15.19:0/509791004 >> 172.21.15.19:0/509791004 conn(0x7f3518073050 msgr2=0x7f3518073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.525+0000 7f351ee00640 1 -- 172.21.15.19:0/509791004 shutdown_connections 2024-04-25T11:31:37.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.526+0000 7f351ee00640 1 -- 172.21.15.19:0/509791004 wait complete. 2024-04-25T11:31:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.528+0000 7f351ee00640 1 Processor -- start 2024-04-25T11:31:37.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.529+0000 7f351ee00640 1 -- start start 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3518070270 0x7f35181533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35181400a0 0x7f3518155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f35181324c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35180b49a0 con 0x7f3518142730 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35180cddb0 con 0x7f3518070270 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35180b4b20 con 0x7f35181400a0 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f35181324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f35181324c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36794/0 (socket says 172.21.15.19:36794) 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f351ce00640 1 -- 172.21.15.19:0/110644517 learned_addr learned my addr 172.21.15.19:0/110644517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f3517400640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35181400a0 0x7f3518155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.531+0000 7f3517e00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3518070270 0x7f35181533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f351ce00640 1 -- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35181400a0 msgr2=0x7f3518155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f351ce00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35181400a0 0x7f3518155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f351ce00640 1 -- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3518070270 msgr2=0x7f35181533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f351ce00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3518070270 0x7f35181533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f351ce00640 1 -- 172.21.15.19:0/110644517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3518154270 con 0x7f3518142730 2024-04-25T11:31:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f351ce00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f35181324c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f351000b440 tx=0x7f351000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f350fe00640 1 -- 172.21.15.19:0/110644517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510011830 con 0x7f3518142730 2024-04-25T11:31:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f350fe00640 1 -- 172.21.15.19:0/110644517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510016070 con 0x7f3518142730 2024-04-25T11:31:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f350fe00640 1 -- 172.21.15.19:0/110644517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510011c10 con 0x7f3518142730 2024-04-25T11:31:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.532+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35181547a0 con 0x7f3518142730 2024-04-25T11:31:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.533+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3518154d60 con 0x7f3518142730 2024-04-25T11:31:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.534+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3518050740 con 0x7f3518142730 2024-04-25T11:31:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.535+0000 7f350fe00640 1 -- 172.21.15.19:0/110644517 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f351001b460 con 0x7f3518142730 2024-04-25T11:31:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.536+0000 7f350fe00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34f80744e0 0x7f34f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.536+0000 7f350fe00640 1 -- 172.21.15.19:0/110644517 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f3510094d10 con 0x7f3518142730 2024-04-25T11:31:37.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.537+0000 7f3517e00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34f80744e0 0x7f34f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:37.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.538+0000 7f3517e00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34f80744e0 0x7f34f80769f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3518073000 tx=0x7f3508008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:37.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.541+0000 7f350fe00640 1 -- 172.21.15.19:0/110644517 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f351009a020 con 0x7f3518142730 2024-04-25T11:31:37.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.787+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35181544c0 con 0x7f3518142730 2024-04-25T11:31:37.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.789+0000 7f350fe00640 1 -- 172.21.15.19:0/110644517 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5188 (secure 0 0 0) 0x7f3510020020 con 0x7f3518142730 2024-04-25T11:31:37.779 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:34.909423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":4}]} 2024-04-25T11:31:37.780 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-04-25T11:31:37.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34f80744e0 msgr2=0x7f34f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34f80744e0 0x7f34f80769f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3518073000 tx=0x7f3508008040 comp rx=0 tx=0).stop 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 msgr2=0x7f35181324c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f35181324c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f351000b440 tx=0x7f351000b900 comp rx=0 tx=0).stop 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 shutdown_connections 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34f80744e0 0x7f34f80769f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3518142730 0x7f35181324c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35181400a0 0x7f3518155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 --2- 172.21.15.19:0/110644517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3518070270 0x7f35181533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 >> 172.21.15.19:0/110644517 conn(0x7f3518073050 msgr2=0x7f35180b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 shutdown_connections 2024-04-25T11:31:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:37.792+0000 7f351ee00640 1 -- 172.21.15.19:0/110644517 wait complete. 2024-04-25T11:31:37.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2024-04-25T11:31:38.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.109+0000 7f80af200640 1 Processor -- start 2024-04-25T11:31:38.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.109+0000 7f80af200640 1 -- start start 2024-04-25T11:31:38.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.111+0000 7f80af200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f80b0144310 0x7f80b006fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:38.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.111+0000 7f80af200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0161a80 0x7f80b0161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:38.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.111+0000 7f80af200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b0144720 con 0x7f80b00b6ba0 2024-04-25T11:31:38.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.111+0000 7f80af200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f80b006f640 con 0x7f80b0144310 2024-04-25T11:31:38.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.111+0000 7f80af200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f80b01624d0 con 0x7f80b0161a80 2024-04-25T11:31:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0161a80 0x7f80b0161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ade00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f80b0144310 0x7f80b006fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0161a80 0x7f80b0161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:38160/0 (socket says 172.21.15.19:38160) 2024-04-25T11:31:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ad400640 1 -- 172.21.15.19:0/840379770 learned_addr learned my addr 172.21.15.19:0/840379770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ade00640 1 -- 172.21.15.19:0/840379770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0161a80 msgr2=0x7f80b0161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ade00640 1 --2- 172.21.15.19:0/840379770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0161a80 0x7f80b0161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ade00640 1 -- 172.21.15.19:0/840379770 >> v1:172.21.15.169:6789/0 conn(0x7f80b00b6ba0 legacy=0x7f80b00b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.112+0000 7f80ade00640 1 -- 172.21.15.19:0/840379770 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b01627b0 con 0x7f80b0144310 2024-04-25T11:31:38.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.114+0000 7f80ade00640 1 --2- 172.21.15.19:0/840379770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b0144310 0x7f80b006fad0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f80a00097a0 tx=0x7f80a002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb87219afb456910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.115+0000 7f80aca00640 1 -- 172.21.15.19:0/840379770 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0037070 con 0x7f80b0144310 2024-04-25T11:31:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.115+0000 7f80aca00640 1 -- 172.21.15.19:0/840379770 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a002ae40 con 0x7f80b0144310 2024-04-25T11:31:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.115+0000 7f80aca00640 1 -- 172.21.15.19:0/840379770 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0032660 con 0x7f80b0144310 2024-04-25T11:31:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.115+0000 7f80af200640 1 -- 172.21.15.19:0/840379770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b0144310 msgr2=0x7f80b006fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.115+0000 7f80af200640 1 --2- 172.21.15.19:0/840379770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b0144310 0x7f80b006fad0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f80a00097a0 tx=0x7f80a002a130 comp rx=0 tx=0).stop 2024-04-25T11:31:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.116+0000 7f80af200640 1 -- 172.21.15.19:0/840379770 shutdown_connections 2024-04-25T11:31:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.116+0000 7f80af200640 1 --2- 172.21.15.19:0/840379770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0161a80 0x7f80b0161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.116+0000 7f80af200640 1 --2- 172.21.15.19:0/840379770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b0144310 0x7f80b006fad0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.116+0000 7f80af200640 1 -- 172.21.15.19:0/840379770 >> 172.21.15.19:0/840379770 conn(0x7f80b00b4260 msgr2=0x7f80b0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.116+0000 7f80af200640 1 -- 172.21.15.19:0/840379770 shutdown_connections 2024-04-25T11:31:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.116+0000 7f80af200640 1 -- 172.21.15.19:0/840379770 wait complete. 2024-04-25T11:31:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.118+0000 7f80af200640 1 Processor -- start 2024-04-25T11:31:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.119+0000 7f80af200640 1 -- start start 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.120+0000 7f80af200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0144310 0x7f80b01fa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.120+0000 7f80af200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b0161a80 0x7f80b0141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.120+0000 7f80af200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b013d110 0x7f80b0142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.120+0000 7f80af200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80b01478d0 con 0x7f80b0161a80 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.120+0000 7f80af200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0070010 con 0x7f80b013d110 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.120+0000 7f80af200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f80b0147a50 con 0x7f80b0144310 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b0161a80 0x7f80b0141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b0161a80 0x7f80b0141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36810/0 (socket says 172.21.15.19:36810) 2024-04-25T11:31:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 -- 172.21.15.19:0/2836793048 learned_addr learned my addr 172.21.15.19:0/2836793048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ade00640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0144310 0x7f80b01fa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ae800640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b013d110 0x7f80b0142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 -- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0144310 msgr2=0x7f80b01fa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0144310 0x7f80b01fa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 -- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b013d110 msgr2=0x7f80b0142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b013d110 0x7f80b0142350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ad400640 1 -- 172.21.15.19:0/2836793048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b0144fa0 con 0x7f80b0161a80 2024-04-25T11:31:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.121+0000 7f80ae800640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b013d110 0x7f80b0142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.122+0000 7f80ad400640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b0161a80 0x7f80b0141e10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f809800efc0 tx=0x7f809800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.122+0000 7f80a6000640 1 -- 172.21.15.19:0/2836793048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809800cda0 con 0x7f80b0161a80 2024-04-25T11:31:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.122+0000 7f80a6000640 1 -- 172.21.15.19:0/2836793048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8098019070 con 0x7f80b0161a80 2024-04-25T11:31:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.122+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b0145280 con 0x7f80b0161a80 2024-04-25T11:31:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.122+0000 7f80a6000640 1 -- 172.21.15.19:0/2836793048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80980146a0 con 0x7f80b0161a80 2024-04-25T11:31:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.122+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b0145840 con 0x7f80b0161a80 2024-04-25T11:31:38.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.123+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b0144d10 con 0x7f80b0161a80 2024-04-25T11:31:38.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.125+0000 7f80a6000640 1 -- 172.21.15.19:0/2836793048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8098005d10 con 0x7f80b0161a80 2024-04-25T11:31:38.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.126+0000 7f80a6000640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f808c0744e0 0x7f808c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:38.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.126+0000 7f80a6000640 1 -- 172.21.15.19:0/2836793048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f8098095730 con 0x7f80b0161a80 2024-04-25T11:31:38.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.126+0000 7f80ade00640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f808c0744e0 0x7f808c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:38.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.127+0000 7f80ade00640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f808c0744e0 0x7f808c0769f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f80a0034000 tx=0x7f80a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:38.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.130+0000 7f80a6000640 1 -- 172.21.15.19:0/2836793048 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8098010040 con 0x7f80b0161a80 2024-04-25T11:31:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.418+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) -- 0x7f80b0076c60 con 0x7f80b0161a80 2024-04-25T11:31:38.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.940+0000 7f80a6000640 1 -- 172.21.15.19:0/2836793048 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.6486 v59) ==== 112+0+0 (secure 0 0 0) 0x7f809805fe50 con 0x7f80b0161a80 2024-04-25T11:31:38.931 INFO:teuthology.orchestra.run.smithi019.stderr:freezing mds.6486 2024-04-25T11:31:38.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f808c0744e0 msgr2=0x7f808c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:38.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f808c0744e0 0x7f808c0769f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f80a0034000 tx=0x7f80a002e000 comp rx=0 tx=0).stop 2024-04-25T11:31:38.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b0161a80 msgr2=0x7f80b0141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b0161a80 0x7f80b0141e10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f809800efc0 tx=0x7f809800c490 comp rx=0 tx=0).stop 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 shutdown_connections 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f808c0744e0 0x7f808c0769f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80b013d110 0x7f80b0142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80b0161a80 0x7f80b0141e10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 --2- 172.21.15.19:0/2836793048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80b0144310 0x7f80b01fa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 >> 172.21.15.19:0/2836793048 conn(0x7f80b00b6ba0 msgr2=0x7f80b015f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 shutdown_connections 2024-04-25T11:31:38.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:38.944+0000 7f80af200640 1 -- 172.21.15.19:0/2836793048 wait complete. 2024-04-25T11:31:38.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T11:31:38.977 DEBUG:teuthology.orchestra.run.smithi019:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2024-04-25T11:31:38.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T11:31:39.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.257+0000 7f1cad400640 1 Processor -- start 2024-04-25T11:31:39.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.257+0000 7f1cad400640 1 -- start start 2024-04-25T11:31:39.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.259+0000 7f1cad400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1ca8164ad0 0x7f1ca80b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.259+0000 7f1cad400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ca8163f30 0x7f1ca8164340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.259+0000 7f1cad400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca8164880 con 0x7f1ca80d1f30 2024-04-25T11:31:39.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.259+0000 7f1cad400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca8164ee0 con 0x7f1ca8163f30 2024-04-25T11:31:39.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.259+0000 7f1cad400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca80b31e0 con 0x7f1ca8164ad0 2024-04-25T11:31:39.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ca8163f30 0x7f1ca8164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca7200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ca80d1f30 0x7f1ca80d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59842/0 (socket says 172.21.15.19:59842) 2024-04-25T11:31:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca7200640 1 -- 172.21.15.19:0/2183243752 learned_addr learned my addr 172.21.15.19:0/2183243752 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:39.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca6800640 1 --2- 172.21.15.19:0/2183243752 >> v2:172.21.15.169:3300/0 conn(0x7f1ca8164ad0 0x7f1ca80b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca5e00640 1 -- 172.21.15.19:0/2183243752 >> v2:172.21.15.169:3300/0 conn(0x7f1ca8164ad0 msgr2=0x7f1ca80b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/2183243752 >> v2:172.21.15.169:3300/0 conn(0x7f1ca8164ad0 0x7f1ca80b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca5e00640 1 -- 172.21.15.19:0/2183243752 >> v1:172.21.15.19:6789/0 conn(0x7f1ca80d1f30 legacy=0x7f1ca80d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.260+0000 7f1ca5e00640 1 -- 172.21.15.19:0/2183243752 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca80d3170 con 0x7f1ca8163f30 2024-04-25T11:31:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.261+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/2183243752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8163f30 0x7f1ca8164340 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8074530 tx=0x7f1c9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e49b8ca5ae8541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.262+0000 7f1ca5400640 1 -- 172.21.15.19:0/2183243752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94037070 con 0x7f1ca8163f30 2024-04-25T11:31:39.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.262+0000 7f1ca5400640 1 -- 172.21.15.19:0/2183243752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c94032440 con 0x7f1ca8163f30 2024-04-25T11:31:39.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.262+0000 7f1ca5400640 1 -- 172.21.15.19:0/2183243752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94032810 con 0x7f1ca8163f30 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 -- 172.21.15.19:0/2183243752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8163f30 msgr2=0x7f1ca8164340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 --2- 172.21.15.19:0/2183243752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8163f30 0x7f1ca8164340 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8074530 tx=0x7f1c9402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 -- 172.21.15.19:0/2183243752 shutdown_connections 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 --2- 172.21.15.19:0/2183243752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8163f30 0x7f1ca8164340 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 --2- 172.21.15.19:0/2183243752 >> v2:172.21.15.169:3300/0 conn(0x7f1ca8164ad0 0x7f1ca80b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 -- 172.21.15.19:0/2183243752 >> 172.21.15.19:0/2183243752 conn(0x7f1ca80d3aa0 msgr2=0x7f1ca80d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 -- 172.21.15.19:0/2183243752 shutdown_connections 2024-04-25T11:31:39.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.263+0000 7f1cad400640 1 -- 172.21.15.19:0/2183243752 wait complete. 2024-04-25T11:31:39.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.266+0000 7f1cad400640 1 Processor -- start 2024-04-25T11:31:39.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.266+0000 7f1cad400640 1 -- start start 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.267+0000 7f1cad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ca80d1f30 0x7f1ca80b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.267+0000 7f1cad400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ca8163f30 0x7f1ca8151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.267+0000 7f1cad400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8164ad0 0x7f1ca80b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.267+0000 7f1cad400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca806d9b0 con 0x7f1ca8164ad0 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.267+0000 7f1cad400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca806d6d0 con 0x7f1ca8163f30 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.267+0000 7f1cad400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca80b9430 con 0x7f1ca80d1f30 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8164ad0 0x7f1ca80b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8164ad0 0x7f1ca80b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36828/0 (socket says 172.21.15.19:36828) 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 -- 172.21.15.19:0/320066501 learned_addr learned my addr 172.21.15.19:0/320066501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ca8163f30 0x7f1ca8151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca6800640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ca80d1f30 0x7f1ca80b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 -- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ca80d1f30 msgr2=0x7f1ca80b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ca80d1f30 0x7f1ca80b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 -- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ca8163f30 msgr2=0x7f1ca8151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ca8163f30 0x7f1ca8151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.268+0000 7f1ca7200640 1 -- 172.21.15.19:0/320066501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca81610a0 con 0x7f1ca8164ad0 2024-04-25T11:31:39.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.269+0000 7f1ca7200640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ca8164ad0 0x7f1ca80b3400 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000cab0 tx=0x7f1ca000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.269+0000 7f1c9ea00640 1 -- 172.21.15.19:0/320066501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0017070 con 0x7f1ca8164ad0 2024-04-25T11:31:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.269+0000 7f1c9ea00640 1 -- 172.21.15.19:0/320066501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca000ac30 con 0x7f1ca8164ad0 2024-04-25T11:31:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.269+0000 7f1cad400640 1 -- 172.21.15.19:0/320066501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca8161380 con 0x7f1ca8164ad0 2024-04-25T11:31:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.269+0000 7f1c9ea00640 1 -- 172.21.15.19:0/320066501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0012400 con 0x7f1ca8164ad0 2024-04-25T11:31:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.269+0000 7f1cad400640 1 -- 172.21.15.19:0/320066501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca8161940 con 0x7f1ca8164ad0 2024-04-25T11:31:39.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.272+0000 7f1c9ea00640 1 -- 172.21.15.19:0/320066501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ca001a070 con 0x7f1ca8164ad0 2024-04-25T11:31:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.273+0000 7f1c9ea00640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c880744e0 0x7f1c880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.273+0000 7f1c9ea00640 1 -- 172.21.15.19:0/320066501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f1ca0094f40 con 0x7f1ca8164ad0 2024-04-25T11:31:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.274+0000 7f1ca6800640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c880744e0 0x7f1c880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.275+0000 7f1ca6800640 1 --2- 172.21.15.19:0/320066501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c880744e0 0x7f1c880769f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c98006fd0 tx=0x7f1c98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.276+0000 7f1cb4427740 1 Processor -- start 2024-04-25T11:31:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.276+0000 7f1cb4427740 1 -- start start 2024-04-25T11:31:39.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.278+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb745c0 0x5556eef39bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.278+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef17400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.278+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb538e0 0x5556eef17940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.278+0000 7f1cb4427740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556eee336d0 con 0x5556eeb538e0 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.278+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5556eee76e10 con 0x5556eeb745c0 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.278+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5556eee26fb0 con 0x5556eeb4b700 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb538e0 0x5556eef17940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb538e0 0x5556eef17940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36844/0 (socket says 172.21.15.19:36844) 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 -- 172.21.15.19:0/2814824244 learned_addr learned my addr 172.21.15.19:0/2814824244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef17400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca6800640 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb745c0 0x5556eef39bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 -- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 msgr2=0x5556eef17400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef17400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 -- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb745c0 msgr2=0x5556eef39bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb745c0 0x5556eef39bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.279+0000 7f1ca7200640 1 -- 172.21.15.19:0/2814824244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556eee05560 con 0x5556eeb538e0 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.280+0000 7f1ca7200640 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb538e0 0x5556eef17940 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1ca005b000 tx=0x7f1ca005e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a458b0f2e4ec62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.280+0000 7f1c93000640 1 -- 172.21.15.19:0/2814824244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0067d30 con 0x5556eeb538e0 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.280+0000 7f1c93000640 1 -- 172.21.15.19:0/2814824244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca00678a0 con 0x5556eeb538e0 2024-04-25T11:31:39.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.280+0000 7f1c93000640 1 -- 172.21.15.19:0/2814824244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004b60 con 0x5556eeb538e0 2024-04-25T11:31:39.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.281+0000 7f1cb4427740 1 -- 172.21.15.19:0/2814824244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb538e0 msgr2=0x5556eef17940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.281+0000 7f1cb4427740 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb538e0 0x5556eef17940 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1ca005b000 tx=0x7f1ca005e820 comp rx=0 tx=0).stop 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.282+0000 7f1cb4427740 1 -- 172.21.15.19:0/2814824244 shutdown_connections 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.282+0000 7f1cb4427740 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb538e0 0x5556eef17940 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.282+0000 7f1cb4427740 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef17400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.282+0000 7f1cb4427740 1 --2- 172.21.15.19:0/2814824244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb745c0 0x5556eef39bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.282+0000 7f1cb4427740 1 -- 172.21.15.19:0/2814824244 >> 172.21.15.19:0/2814824244 conn(0x5556eeb68710 msgr2=0x5556eef14bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.282+0000 7f1cb4427740 1 -- 172.21.15.19:0/2814824244 shutdown_connections 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.282+0000 7f1cb4427740 1 -- 172.21.15.19:0/2814824244 wait complete. 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.283+0000 7f1cb4427740 1 -- start start 2024-04-25T11:31:39.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.283+0000 7f1cb4427740 1 Processor -- start 2024-04-25T11:31:39.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.283+0000 7fb36ce00640 1 Processor -- start 2024-04-25T11:31:39.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.284+0000 7fb36ce00640 1 -- start start 2024-04-25T11:31:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.285+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb4b700 0x5556eef3a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.285+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb538e0 0x5556eef5b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.285+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eef5b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.285+0000 7f1cb4427740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556eee7a160 con 0x5556eeb745c0 2024-04-25T11:31:39.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.285+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5556eee40ef0 con 0x5556eeb4b700 2024-04-25T11:31:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.285+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5556eee7f480 con 0x5556eeb538e0 2024-04-25T11:31:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.285+0000 7f1cb4427740 10 client.0 _ll_register_callbacks cb 0x5556eee46990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eef5b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eef5b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36860/0 (socket says 172.21.15.19:36860) 2024-04-25T11:31:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 -- 172.21.15.19:0/1070203377 learned_addr learned my addr 172.21.15.19:0/1070203377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb538e0 0x5556eef5b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb4b700 0x5556eef3a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7fb36ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 0x7fb3680b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 -- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb538e0 msgr2=0x5556eef5b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb538e0 0x5556eef5b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 -- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb4b700 msgr2=0x5556eef3a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb4b700 0x5556eef3a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7f1ca7200640 1 -- 172.21.15.19:0/1070203377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556eec58a30 con 0x5556eeb745c0 2024-04-25T11:31:39.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7fb36ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb36806c5c0 0x7fb36806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7fb36ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3680b5ec0 con 0x7fb3681443b0 2024-04-25T11:31:39.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7fb36ce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb3680b51e0 con 0x7fb36806c5c0 2024-04-25T11:31:39.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.286+0000 7fb36ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb3681447c0 con 0x7fb3680b4890 2024-04-25T11:31:39.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7f1ca7200640 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eef5b8a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1ca005ef10 tx=0x7f1ca0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f77b021f7cf047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb366800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb36806c5c0 0x7fb36806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb366800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb36806c5c0 0x7fb36806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36868/0 (socket says 172.21.15.19:36868) 2024-04-25T11:31:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb366800640 1 -- 172.21.15.19:0/3350490686 learned_addr learned my addr 172.21.15.19:0/3350490686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7f1c87400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7f1c87400640 1 -- 172.21.15.19:0/1070203377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca005e1e0 con 0x5556eeb745c0 2024-04-25T11:31:39.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb35d400640 1 --2- 172.21.15.19:0/3350490686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 0x7fb3680b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb366800640 1 -- 172.21.15.19:0/3350490686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 msgr2=0x7fb3680b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb366800640 1 --2- 172.21.15.19:0/3350490686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 0x7fb3680b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb366800640 1 -- 172.21.15.19:0/3350490686 >> v1:172.21.15.169:6789/0 conn(0x7fb3681443b0 legacy=0x7fb3680b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7fb366800640 1 -- 172.21.15.19:0/3350490686 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3680d02f0 con 0x7fb36806c5c0 2024-04-25T11:31:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7f1c87400640 1 -- 172.21.15.19:0/1070203377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f1ca0079dd0 con 0x5556eeb745c0 2024-04-25T11:31:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.287+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca00678a0 con 0x5556eeb745c0 2024-04-25T11:31:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.288+0000 7f1c87400640 1 -- 172.21.15.19:0/1070203377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1ca005e380 con 0x5556eeb745c0 2024-04-25T11:31:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.288+0000 7f1cb4427740 10 client.6564 fetch_fsmap learned FSMap version 59 2024-04-25T11:31:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.288+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5556eeb8c620 con 0x5556eeb745c0 2024-04-25T11:31:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1c87400640 1 -- 172.21.15.19:0/1070203377 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca00678a0 con 0x5556eeb745c0 2024-04-25T11:31:39.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1c87400640 1 -- 172.21.15.19:0/1070203377 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7f1ca0080370 con 0x5556eeb745c0 2024-04-25T11:31:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1cb4427740 10 client.6564 fetch_fsmap finished waiting for FSMap version 59 2024-04-25T11:31:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1cb4427740 10 client.6564 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T11:31:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1cb4427740 20 client.6564 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1cb4427740 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eeb33190 0x5556eef58a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1cb4427740 4 client.6564 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:31:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.289+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 --> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556eee88a60 con 0x5556eeb33190 2024-04-25T11:31:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.290+0000 7fb366800640 1 --2- 172.21.15.19:0/3350490686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb36806c5c0 0x7fb36806c9d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb360009870 tx=0x7fb36002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6444a9841ecb3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.290+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eeb33190 0x5556eef58a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.290+0000 7fb365400640 1 -- 172.21.15.19:0/3350490686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360037070 con 0x7fb36806c5c0 2024-04-25T11:31:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.290+0000 7fb365400640 1 -- 172.21.15.19:0/3350490686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb360032440 con 0x7fb36806c5c0 2024-04-25T11:31:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.290+0000 7fb365400640 1 -- 172.21.15.19:0/3350490686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360032810 con 0x7fb36806c5c0 2024-04-25T11:31:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.291+0000 7fb36ce00640 1 -- 172.21.15.19:0/3350490686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb36806c5c0 msgr2=0x7fb36806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.291+0000 7fb36ce00640 1 --2- 172.21.15.19:0/3350490686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb36806c5c0 0x7fb36806c9d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb360009870 tx=0x7fb36002a180 comp rx=0 tx=0).stop 2024-04-25T11:31:39.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.290+0000 7f1c87400640 10 client.6564 ms_handle_connect on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:39.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.290+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eeb33190 0x5556eef58a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2aa473c67afa382 server_cookie=bbcc4e6ed53446e9 in_seq=0 out_seq=0 2024-04-25T11:31:39.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.291+0000 7fb36ce00640 1 -- 172.21.15.19:0/3350490686 shutdown_connections 2024-04-25T11:31:39.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7fb36ce00640 1 --2- 172.21.15.19:0/3350490686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb36806c5c0 0x7fb36806c9d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.291+0000 7f1c87400640 1 -- 172.21.15.19:0/1070203377 <== mds.0 v2:172.21.15.19:6832/1883095429 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556eee88a60 con 0x5556eeb33190 2024-04-25T11:31:39.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1c87400640 10 client.6564 handle_command_reply: tid=0 2024-04-25T11:31:39.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7fb36ce00640 1 --2- 172.21.15.19:0/3350490686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 0x7fb3680b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7fb36ce00640 1 -- 172.21.15.19:0/3350490686 >> 172.21.15.19:0/3350490686 conn(0x7fb3680ce2e0 msgr2=0x7fb3680ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:39.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 client.6564 shutdown 2024-04-25T11:31:39.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7fb36ce00640 1 -- 172.21.15.19:0/3350490686 shutdown_connections 2024-04-25T11:31:39.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7fb36ce00640 1 -- 172.21.15.19:0/3350490686 wait complete. 2024-04-25T11:31:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 msgr2=0x5556eef5b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eef5b8a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1ca005ef10 tx=0x7f1ca0001250 comp rx=0 tx=0).stop 2024-04-25T11:31:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 shutdown_connections 2024-04-25T11:31:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eeb33190 0x5556eef58a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eef5b8a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb538e0 0x5556eef5b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 --2- 172.21.15.19:0/1070203377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb4b700 0x5556eef3a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 >> 172.21.15.19:0/1070203377 conn(0x5556eeb68710 msgr2=0x5556eef38170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:39.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.292+0000 7f1c87400640 0 client.6564 ms_handle_reset on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:39.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.293+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 shutdown_connections 2024-04-25T11:31:39.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.293+0000 7f1cb4427740 1 -- 172.21.15.19:0/1070203377 wait complete. 2024-04-25T11:31:39.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.293+0000 7f1cb4427740 20 client.6564 trim_cache size 0 max 16384 2024-04-25T11:31:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.294+0000 7fb36ce00640 1 Processor -- start 2024-04-25T11:31:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.295+0000 7fb36ce00640 1 -- start start 2024-04-25T11:31:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.296+0000 7fb36ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb36806c5c0 0x7fb368147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.296+0000 7fb36ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 0x7fb368143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.296+0000 7fb36ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3681443b0 0x7fb3681449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.296+0000 7fb36ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3680d08b0 con 0x7fb3681443b0 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.296+0000 7fb36ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3680d05d0 con 0x7fb36806c5c0 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.296+0000 7fb36ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb3680d0b90 con 0x7fb3680b4890 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.297+0000 7fb366800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3681443b0 0x7fb3681449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.297+0000 7fb366800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3681443b0 0x7fb3681449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36884/0 (socket says 172.21.15.19:36884) 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.297+0000 7fb366800640 1 -- 172.21.15.19:0/85194840 learned_addr learned my addr 172.21.15.19:0/85194840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.297+0000 7fb35d400640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 0x7fb368143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.297+0000 7fb365e00640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb36806c5c0 0x7fb368147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.298+0000 7fb366800640 1 -- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 msgr2=0x7fb368143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.298+0000 7fb366800640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3680b4890 0x7fb368143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.298+0000 7fb366800640 1 -- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb36806c5c0 msgr2=0x7fb368147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.298+0000 7fb366800640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb36806c5c0 0x7fb368147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.298+0000 7fb366800640 1 -- 172.21.15.19:0/85194840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36815a1f0 con 0x7fb3681443b0 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.299+0000 7fb366800640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3681443b0 0x7fb3681449d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb360009870 tx=0x7fb3600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.299+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360041020 con 0x7fb3681443b0 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.299+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3600306c0 con 0x7fb3681443b0 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.299+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360037040 con 0x7fb3681443b0 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.299+0000 7fb36ce00640 1 -- 172.21.15.19:0/85194840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36815a6f0 con 0x7fb3681443b0 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.299+0000 7fb36ce00640 1 -- 172.21.15.19:0/85194840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36815acb0 con 0x7fb3681443b0 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.301+0000 7fb36ce00640 1 -- 172.21.15.19:0/85194840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb3680cfef0 con 0x7fb3681443b0 2024-04-25T11:31:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.303+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb360044020 con 0x7fb3681443b0 2024-04-25T11:31:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.304+0000 7fb35ea00640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb344074530 0x7fb344076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.305+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fb3600b5e50 con 0x7fb3681443b0 2024-04-25T11:31:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.305+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+513 (secure 0 0 0) 0x7fb36004b030 con 0x7fb3681443b0 2024-04-25T11:31:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.305+0000 7fb36ce00640 1 -- 172.21.15.19:0/85194840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fb36815a410 con 0x7fb3681443b0 2024-04-25T11:31:39.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.305+0000 7fb365e00640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb344074530 0x7fb344076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.306+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 7 ==== log(1 entries from seq 686 at 2024-04-25T11:31:38.420812+0000) ==== 309+0+0 (secure 0 0 0) 0x7fb36007c210 con 0x7fb3681443b0 2024-04-25T11:31:39.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.307+0000 7fb365e00640 1 --2- 172.21.15.19:0/85194840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb344074530 0x7fb344076a40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb358009f20 tx=0x7fb3580094d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.302 INFO:teuthology.orchestra.run.smithi019.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T11:31:39.302 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T11:31:39.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:39.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.316+0000 7f1cb4427740 1 Processor -- start 2024-04-25T11:31:39.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.316+0000 7f1cb4427740 1 -- start start 2024-04-25T11:31:39.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.317+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.317+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.317+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.317+0000 7f1cb4427740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8203040 con 0x5556eeb745c0 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.317+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8202060 con 0x5556eeb538e0 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.317+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca82013c0 con 0x5556eeb4b700 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36900/0 (socket says 172.21.15.19:36900) 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 -- 172.21.15.19:0/3681368946 learned_addr learned my addr 172.21.15.19:0/3681368946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca6800640 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 -- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 msgr2=0x5556eef132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 -- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 msgr2=0x5556eef11760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef11760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca7200640 1 -- 172.21.15.19:0/3681368946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556eeb4c320 con 0x5556eeb745c0 2024-04-25T11:31:39.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.318+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.319+0000 7f1ca7200640 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd1b90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000ca80 tx=0x7f1ca0007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745c176e429e110b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.319+0000 7f1c92600640 1 -- 172.21.15.19:0/3681368946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00678a0 con 0x5556eeb745c0 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.319+0000 7f1c92600640 1 -- 172.21.15.19:0/3681368946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca0067d30 con 0x5556eeb745c0 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.319+0000 7f1c92600640 1 -- 172.21.15.19:0/3681368946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca005a8f0 con 0x5556eeb745c0 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 -- 172.21.15.19:0/3681368946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 msgr2=0x5556eefd1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd1b90 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000ca80 tx=0x7f1ca0007040 comp rx=0 tx=0).stop 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 -- 172.21.15.19:0/3681368946 shutdown_connections 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd1b90 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 --2- 172.21.15.19:0/3681368946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eef132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 -- 172.21.15.19:0/3681368946 >> 172.21.15.19:0/3681368946 conn(0x5556eeb68710 msgr2=0x5556eef594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:39.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 -- 172.21.15.19:0/3681368946 shutdown_connections 2024-04-25T11:31:39.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.320+0000 7f1cb4427740 1 -- 172.21.15.19:0/3681368946 wait complete. 2024-04-25T11:31:39.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.321+0000 7f1cb4427740 1 -- start start 2024-04-25T11:31:39.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.321+0000 7f1cb4427740 1 Processor -- start 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eefc0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef10dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1cb4427740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1cb4427740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556eee870f0 con 0x5556eeb745c0 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8201a10 con 0x5556eeb538e0 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1cb4427740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5556eefd8c90 con 0x5556eeb4b700 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1cb4427740 10 client.0 _ll_register_callbacks cb 0x7f1ca005e6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.323+0000 7f1ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36914/0 (socket says 172.21.15.19:36914) 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca7200640 1 -- 172.21.15.19:0/1156846417 learned_addr learned my addr 172.21.15.19:0/1156846417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eefc0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca5e00640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef10dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca7200640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 msgr2=0x5556eefc0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca7200640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eefc0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca7200640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 msgr2=0x5556eef10dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca7200640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556eeb538e0 0x5556eef10dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca7200640 1 -- 172.21.15.19:0/1156846417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556eef11310 con 0x5556eeb745c0 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556eeb4b700 0x5556eefc0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1ca7200640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556eeb745c0 0x5556eefd8020 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1ca005f140 tx=0x7f1ca005b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d608ea95eea1fd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1c87400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.324+0000 7f1c87400640 1 -- 172.21.15.19:0/1156846417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca00678a0 con 0x5556eeb745c0 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1c87400640 1 -- 172.21.15.19:0/1156846417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f1ca005de60 con 0x5556eeb745c0 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 1 -- 172.21.15.19:0/1156846417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca0067d30 con 0x5556eeb745c0 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1c87400640 1 -- 172.21.15.19:0/1156846417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1ca006e050 con 0x5556eeb745c0 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 10 client.6576 fetch_fsmap learned FSMap version 59 2024-04-25T11:31:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 1 -- 172.21.15.19:0/1156846417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5556eef11310 con 0x5556eeb745c0 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1c87400640 1 -- 172.21.15.19:0/1156846417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca0067d30 con 0x5556eeb745c0 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1c87400640 1 -- 172.21.15.19:0/1156846417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7f1ca0071020 con 0x5556eeb745c0 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 10 client.6576 fetch_fsmap finished waiting for FSMap version 59 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 10 client.6576 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 20 client.6576 populate_metadata read hostname 'smithi019' 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 4 client.6576 mds_command: new command op to 6486 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.325+0000 7f1cb4427740 1 -- 172.21.15.19:0/1156846417 --> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556eee828d0 con 0x5556eef7f240 2024-04-25T11:31:39.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.326+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.326+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc41dd90bf80111 server_cookie=f6ff79fc21490656 in_seq=0 out_seq=0 2024-04-25T11:31:39.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.326+0000 7f1c87400640 10 client.6576 ms_handle_connect on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:39.320 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:31:39.329+0000 7f89b9e00640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x5578b6d1f180] 2024-04-25T11:31:39.320 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:31:39.329+0000 7f89b9e00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x5578b6d1f180] 2024-04-25T11:31:39.321 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f89b9e00640 time 2024-04-25T11:31:39.331567+0000 2024-04-25T11:31:39.321 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:31:39.321 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.322 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f89c904bfcd] 2024-04-25T11:31:39.322 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.322 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.322 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.322 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.322 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.323 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.323 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.323 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.323 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.323 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.323 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.324 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.324 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.324 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.324 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.324 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.324 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.325 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.325 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.325 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.325 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.325 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.325 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.326 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.326 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.326 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:31:39.331+0000 7f89b9e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f89b9e00640 time 2024-04-25T11:31:39.331567+0000 2024-04-25T11:31:39.326 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:31:39.326 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.327 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.327 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f89c904bfcd] 2024-04-25T11:31:39.327 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.327 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.327 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.327 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.328 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.329 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.329 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.329 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.329 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.329 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.329 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.330 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.330 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.330 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.330 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.330 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.331 INFO:tasks.ceph.mds.c.smithi019.stderr:*** Caught signal (Aborted) ** 2024-04-25T11:31:39.331 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f89b9e00640 thread_name:MR_Finisher 2024-04-25T11:31:39.331 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.331 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f89c863e6f0] 2024-04-25T11:31:39.331 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f89c868b94c] 2024-04-25T11:31:39.331 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f89c904c13f] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.332 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 29: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr: 30: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.333 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:31:39.333+0000 7f89b9e00640 -1 *** Caught signal (Aborted) ** 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f89b9e00640 thread_name:MR_Finisher 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f89c863e6f0] 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f89c868b94c] 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:31:39.334 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f89c904c13f] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.335 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.336 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 29: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 30: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T11:31:39.337 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.338 INFO:tasks.ceph.mds.c.smithi019.stderr: -259> 2024-04-25T11:31:36.447+0000 7f89bee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:31:39.339 INFO:tasks.ceph.mds.c.smithi019.stderr: -11> 2024-04-25T11:31:39.329+0000 7f89b9e00640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x5578b6d1f180] 2024-04-25T11:31:39.339 INFO:tasks.ceph.mds.c.smithi019.stderr: -9> 2024-04-25T11:31:39.329+0000 7f89b9e00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x5578b6d1f180] 2024-04-25T11:31:39.339 INFO:tasks.ceph.mds.c.smithi019.stderr: -3> 2024-04-25T11:31:39.331+0000 7f89b9e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f89b9e00640 time 2024-04-25T11:31:39.331567+0000 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f89c904bfcd] 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.340 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.341 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.342 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.343 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.343 INFO:tasks.ceph.mds.c.smithi019.stderr: 0> 2024-04-25T11:31:39.333+0000 7f89b9e00640 -1 *** Caught signal (Aborted) ** 2024-04-25T11:31:39.343 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f89b9e00640 thread_name:MR_Finisher 2024-04-25T11:31:39.343 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.343 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.343 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f89c863e6f0] 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f89c868b94c] 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f89c904c13f] 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.344 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.345 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 29: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.346 INFO:tasks.ceph.mds.c.smithi019.stderr: 30: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: -259> 2024-04-25T11:31:36.447+0000 7f89bee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: -11> 2024-04-25T11:31:39.329+0000 7f89b9e00640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x5578b6d1f180] 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: -9> 2024-04-25T11:31:39.329+0000 7f89b9e00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x5578b6d1f180] 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: -3> 2024-04-25T11:31:39.331+0000 7f89b9e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f89b9e00640 time 2024-04-25T11:31:39.331567+0000 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f89c904bfcd] 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.347 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.348 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 0> 2024-04-25T11:31:39.333+0000 7f89b9e00640 -1 *** Caught signal (Aborted) ** 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f89b9e00640 thread_name:MR_Finisher 2024-04-25T11:31:39.349 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.350 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:31:39.350 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f89c863e6f0] 2024-04-25T11:31:39.350 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f89c868b94c] 2024-04-25T11:31:39.351 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:31:39.351 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:31:39.351 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f89c904c13f] 2024-04-25T11:31:39.351 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x55788caff937] 2024-04-25T11:31:39.351 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x55788cdc4df6] 2024-04-25T11:31:39.351 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x3f00a1) [0x55788cdd60a1] 2024-04-25T11:31:39.351 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (CDir::_omap_commit(int)+0x7ef) [0x55788cdd6e49] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (CDir::_commit(unsigned long, int)+0x5ba) [0x55788cdd7a5a] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x11f) [0x55788cdd7d1d] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0x10bf) [0x55788cb69b29] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDLog::try_expire(LogSegment*, int)+0x6f) [0x55788ce8c833] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDLog::trim_all()+0x85e) [0x55788ce8d582] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: ceph-mds(+0x11ec11) [0x55788cb04c11] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: ceph-mds(+0x11ed6f) [0x55788cb04d6f] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: ceph-mds(+0x160b39) [0x55788cb46b39] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: ceph-mds(+0xf9569) [0x55788cadf569] 2024-04-25T11:31:39.352 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (MDSInternalContextWrapper::finish(int)+0x12) [0x55788ce7a928] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: ceph-mds(+0x49490c) [0x55788ce7a90c] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (MDSContext::complete(int)+0x7c) [0x55788ce814be] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (MDSIOContextBase::complete(int)+0x501) [0x55788ce88373] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: (C_IO_Wrapper::complete(int)+0x3c) [0x55788ce88622] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: (Finisher::finisher_thread_entry()+0x495) [0x7f89c8fd212d] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 26: ceph-mds(+0x11120d) [0x55788caf720d] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 27: (Thread::entry_wrapper()+0x43) [0x7f89c902487f] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 28: (Thread::_entry_func(void*)+0xd) [0x7f89c902489b] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 29: /lib64/libc.so.6(+0x89c02) [0x7f89c8689c02] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 30: /lib64/libc.so.6(+0x10ec40) [0x7f89c870ec40] 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T11:31:39.353 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:31:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.568+0000 7fe1c5400640 1 Processor -- start 2024-04-25T11:31:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.568+0000 7fe1c5400640 1 -- start start 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.571+0000 7fe1c5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe1c0071140 0x7fe1c00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.571+0000 7fe1c5400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1c00d0780 0x7fe1c00d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.571+0000 7fe1c5400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c00cf9c0 con 0x7fe1c00b4970 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.571+0000 7fe1c5400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c0156d20 con 0x7fe1c00d0780 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.571+0000 7fe1c5400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c00d1660 con 0x7fe1c0071140 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1b7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1c00d0780 0x7fe1c00d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1b7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe1c00d0780 0x7fe1c00d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36928/0 (socket says 172.21.15.19:36928) 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1b7e00640 1 -- 172.21.15.19:0/2716364920 learned_addr learned my addr 172.21.15.19:0/2716364920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1be800640 1 --2- 172.21.15.19:0/2716364920 >> v2:172.21.15.169:3300/0 conn(0x7fe1c0071140 0x7fe1c00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1b7e00640 1 -- 172.21.15.19:0/2716364920 >> v2:172.21.15.169:3300/0 conn(0x7fe1c0071140 msgr2=0x7fe1c00707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1b7e00640 1 --2- 172.21.15.19:0/2716364920 >> v2:172.21.15.169:3300/0 conn(0x7fe1c0071140 0x7fe1c00707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1b7e00640 1 -- 172.21.15.19:0/2716364920 >> v1:172.21.15.169:6789/0 conn(0x7fe1c00b4970 legacy=0x7fe1c00b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1b7e00640 1 -- 172.21.15.19:0/2716364920 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c00cfca0 con 0x7fe1c00d0780 2024-04-25T11:31:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.572+0000 7fe1be800640 1 --2- 172.21.15.19:0/2716364920 >> v2:172.21.15.169:3300/0 conn(0x7fe1c0071140 0x7fe1c00707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:31:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.573+0000 7fe1b7e00640 1 --2- 172.21.15.19:0/2716364920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c00d1120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0072540 tx=0x7fe1a802c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ffdcbc7debf6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.574+0000 7fe1bde00640 1 -- 172.21.15.19:0/2716364920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a803a070 con 0x7fe1c00d0780 2024-04-25T11:31:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.574+0000 7fe1bde00640 1 -- 172.21.15.19:0/2716364920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a8032480 con 0x7fe1c00d0780 2024-04-25T11:31:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.574+0000 7fe1bde00640 1 -- 172.21.15.19:0/2716364920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8032650 con 0x7fe1c00d0780 2024-04-25T11:31:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 -- 172.21.15.19:0/2716364920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 msgr2=0x7fe1c00d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2716364920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c00d1120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0072540 tx=0x7fe1a802c620 comp rx=0 tx=0).stop 2024-04-25T11:31:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 -- 172.21.15.19:0/2716364920 shutdown_connections 2024-04-25T11:31:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2716364920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c00d1120 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2716364920 >> v2:172.21.15.169:3300/0 conn(0x7fe1c0071140 0x7fe1c00707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 -- 172.21.15.19:0/2716364920 >> 172.21.15.19:0/2716364920 conn(0x7fe1c00d1ae0 msgr2=0x7fe1c00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 -- 172.21.15.19:0/2716364920 shutdown_connections 2024-04-25T11:31:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.575+0000 7fe1c5400640 1 -- 172.21.15.19:0/2716364920 wait complete. 2024-04-25T11:31:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.578+0000 7fe1c5400640 1 Processor -- start 2024-04-25T11:31:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.578+0000 7fe1c5400640 1 -- start start 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.580+0000 7fe1c5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c0071140 0x7fe1c0158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.580+0000 7fe1c5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c00b4970 0x7fe1c014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.580+0000 7fe1c5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c0156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.580+0000 7fe1c5400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00cf620 con 0x7fe1c00d0780 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1c5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00d0260 con 0x7fe1c00b4970 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1c5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c00cf7a0 con 0x7fe1c0071140 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c0156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c0156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36940/0 (socket says 172.21.15.19:36940) 2024-04-25T11:31:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 -- 172.21.15.19:0/2763056404 learned_addr learned my addr 172.21.15.19:0/2763056404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1b7e00640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c00b4970 0x7fe1c014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 -- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c0071140 msgr2=0x7fe1c0158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c0071140 0x7fe1c0158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 -- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c00b4970 msgr2=0x7fe1c014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c00b4970 0x7fe1c014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.581+0000 7fe1bf200640 1 -- 172.21.15.19:0/2763056404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c0158890 con 0x7fe1c00d0780 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.582+0000 7fe1bf200640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c0156ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800ba10 tx=0x7fe1b800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.582+0000 7fe1b6a00640 1 -- 172.21.15.19:0/2763056404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8019070 con 0x7fe1c00d0780 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.582+0000 7fe1b6a00640 1 -- 172.21.15.19:0/2763056404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b800cb40 con 0x7fe1c00d0780 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.582+0000 7fe1b6a00640 1 -- 172.21.15.19:0/2763056404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8014440 con 0x7fe1c00d0780 2024-04-25T11:31:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.582+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c015bf30 con 0x7fe1c00d0780 2024-04-25T11:31:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.582+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c015de50 con 0x7fe1c00d0780 2024-04-25T11:31:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.584+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c006cd60 con 0x7fe1c00d0780 2024-04-25T11:31:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.585+0000 7fe1b6a00640 1 -- 172.21.15.19:0/2763056404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1b801c070 con 0x7fe1c00d0780 2024-04-25T11:31:39.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.586+0000 7fe1b6a00640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe19c0744b0 0x7fe19c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:39.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.586+0000 7fe1b6a00640 1 -- 172.21.15.19:0/2763056404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fe1b8094ec0 con 0x7fe1c00d0780 2024-04-25T11:31:39.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.587+0000 7fe1be800640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe19c0744b0 0x7fe19c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:39.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.588+0000 7fe1be800640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe19c0744b0 0x7fe19c0769c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006fd0 tx=0x7fe1b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:31:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:31:39.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:31:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-25T11:31:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk reading from fd=17 : -104 Unknown error -104 2024-04-25T11:31:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:31:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-04-25T11:31:39.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.590+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:31:39.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.594+0000 7fe1b6a00640 1 -- 172.21.15.19:0/2763056404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1c006cd60 con 0x7fe1c00d0780 2024-04-25T11:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.791+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.791+0000 7f1c87400640 1 client.6576 ms_handle_refused on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.791+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:31:39.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.842+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1c006fa70 con 0x7fe1c00d0780 2024-04-25T11:31:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.842+0000 7fe1b6a00640 1 -- 172.21.15.19:0/2763056404 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5188 (secure 0 0 0) 0x7fe1c006fa70 con 0x7fe1c00d0780 2024-04-25T11:31:39.833 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:39.833 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:38.420979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":4}]} 2024-04-25T11:31:39.833 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 59 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe19c0744b0 msgr2=0x7fe19c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe19c0744b0 0x7fe19c0769c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006fd0 tx=0x7fe1b0008040 comp rx=0 tx=0).stop 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 msgr2=0x7fe1c0156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c0156ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800ba10 tx=0x7fe1b800bed0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 shutdown_connections 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe19c0744b0 0x7fe19c0769c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1c00b4970 0x7fe1c014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1c00d0780 0x7fe1c0156ea0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 --2- 172.21.15.19:0/2763056404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1c0071140 0x7fe1c0158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 >> 172.21.15.19:0/2763056404 conn(0x7fe1c00d1ae0 msgr2=0x7fe1c00b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 shutdown_connections 2024-04-25T11:31:39.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.845+0000 7fe1c5400640 1 -- 172.21.15.19:0/2763056404 wait complete. 2024-04-25T11:31:39.864 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T11:31:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:39.945+0000 7fb35ea00640 1 -- 172.21.15.19:0/85194840 <== mon.0 v2:172.21.15.19:3300/0 8 ==== log(2 entries from seq 687 at 2024-04-25T11:31:38.941539+0000) ==== 635+0+0 (secure 0 0 0) 0x7fb3600801e0 con 0x7fb3681443b0 2024-04-25T11:31:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: cluster: 2024-04-25T11:31:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: id: 23a56573-5965-4bbb-bd60-741a44babbed 2024-04-25T11:31:39.936 INFO:teuthology.orchestra.run.smithi019.stdout: health: HEALTH_OK 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: services: 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: mon: 3 daemons, quorum a,b,c (age 4m) 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: mgr: x(active, since 4m), standbys: y 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: osd: 8 osds: 8 up (since 4m), 8 in (since 4m) 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: data: 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: volumes: 1/1 healthy 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: pools: 3 pools, 97 pgs 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: objects: 29 objects, 602 KiB 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: usage: 243 MiB used, 720 GiB / 720 GiB avail 2024-04-25T11:31:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: pgs: 97 active+clean 2024-04-25T11:31:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: io: 2024-04-25T11:31:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: client: 7.7 KiB/s rd, 597 B/s wr, 6 op/s rd, 0 op/s wr 2024-04-25T11:31:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:39.938 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:39.938 INFO:teuthology.orchestra.run.smithi019.stdout:2024-04-25T11:31:39.331522+0000 mds.c [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x5578b6d1f180] 2024-04-25T11:31:40.023 INFO:tasks.ceph.mds.c.smithi019.stderr:daemon-helper: command crashed with signal 6 2024-04-25T11:31:40.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:40.192+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:40.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:40.192+0000 7f1c87400640 1 client.6576 ms_handle_refused on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:40.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:40.192+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:31:40.883 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T11:31:40.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:40.993+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:40.993+0000 7f1c87400640 1 client.6576 ms_handle_refused on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:40.993+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:31:42.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:42.596+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:42.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:42.596+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=15 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T11:31:42.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:42.596+0000 7f1c87400640 1 client.6576 ms_handle_refused on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:44.865 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.169+0000 7f67ffe00640 1 Processor -- start 2024-04-25T11:31:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.169+0000 7f67ffe00640 1 -- start start 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.171+0000 7f67ffe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68000b6e10 0x7f680015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.171+0000 7f67ffe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68001608d0 0x7f6800160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67ffe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6800161220 con 0x7f68000cf2f0 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67ffe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f680006f870 con 0x7f68000b6e10 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67ffe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68000b7220 con 0x7f68001608d0 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67fe000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68000b6e10 0x7f680015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67fe000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68000b6e10 0x7f680015a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50926/0 (socket says 172.21.15.19:50926) 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67fe000640 1 -- 172.21.15.19:0/209242366 learned_addr learned my addr 172.21.15.19:0/209242366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67ff400640 1 --2- 172.21.15.19:0/209242366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68001608d0 0x7f6800160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67fe000640 1 -- 172.21.15.19:0/209242366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68001608d0 msgr2=0x7f6800160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67fe000640 1 --2- 172.21.15.19:0/209242366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68001608d0 0x7f6800160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.172+0000 7f67fe000640 1 -- 172.21.15.19:0/209242366 >> v1:172.21.15.19:6789/0 conn(0x7f68000cf2f0 legacy=0x7f68000b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.173+0000 7f67fe000640 1 -- 172.21.15.19:0/209242366 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68000b74d0 con 0x7f68000b6e10 2024-04-25T11:31:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.173+0000 7f67ff400640 1 --2- 172.21.15.19:0/209242366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68001608d0 0x7f6800160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.174+0000 7f67fe000640 1 --2- 172.21.15.19:0/209242366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000b6e10 0x7f680015a810 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009870 tx=0x7f67ec02c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb4c25589b552c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.174+0000 7f67fd600640 1 -- 172.21.15.19:0/209242366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec038070 con 0x7f68000b6e10 2024-04-25T11:31:45.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.174+0000 7f67fd600640 1 -- 172.21.15.19:0/209242366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67ec031710 con 0x7f68000b6e10 2024-04-25T11:31:45.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.174+0000 7f67fd600640 1 -- 172.21.15.19:0/209242366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec031ae0 con 0x7f68000b6e10 2024-04-25T11:31:45.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 -- 172.21.15.19:0/209242366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000b6e10 msgr2=0x7f680015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 --2- 172.21.15.19:0/209242366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000b6e10 0x7f680015a810 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009870 tx=0x7f67ec02c460 comp rx=0 tx=0).stop 2024-04-25T11:31:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 -- 172.21.15.19:0/209242366 shutdown_connections 2024-04-25T11:31:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 --2- 172.21.15.19:0/209242366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68001608d0 0x7f6800160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 --2- 172.21.15.19:0/209242366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000b6e10 0x7f680015a810 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 -- 172.21.15.19:0/209242366 >> 172.21.15.19:0/209242366 conn(0x7f68000d1720 msgr2=0x7f68000d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 -- 172.21.15.19:0/209242366 shutdown_connections 2024-04-25T11:31:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.175+0000 7f67ffe00640 1 -- 172.21.15.19:0/209242366 wait complete. 2024-04-25T11:31:45.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.177+0000 7f67ffe00640 1 Processor -- start 2024-04-25T11:31:45.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.178+0000 7f67ffe00640 1 -- start start 2024-04-25T11:31:45.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.179+0000 7f67ffe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68000b6e10 0x7f680015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:45.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.179+0000 7f67ffe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000cf2f0 0x7f680015f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:45.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.179+0000 7f67ffe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68001608d0 0x7f6800146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.179+0000 7f67ffe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f680006f500 con 0x7f68000cf2f0 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.179+0000 7f67ffe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f680006f220 con 0x7f68001608d0 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.179+0000 7f67ffe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f680006fd00 con 0x7f68000b6e10 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000cf2f0 0x7f680015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000cf2f0 0x7f680015f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50940/0 (socket says 172.21.15.19:50940) 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 -- 172.21.15.19:0/1240887278 learned_addr learned my addr 172.21.15.19:0/1240887278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fea00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68000b6e10 0x7f680015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67ff400640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68001608d0 0x7f6800146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 -- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68000b6e10 msgr2=0x7f680015b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68000b6e10 0x7f680015b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 -- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68001608d0 msgr2=0x7f6800146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68001608d0 0x7f6800146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.180+0000 7f67fe000640 1 -- 172.21.15.19:0/1240887278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680015c140 con 0x7f68000cf2f0 2024-04-25T11:31:45.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.181+0000 7f67fe000640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000cf2f0 0x7f680015f770 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f67ec000c00 tx=0x7f67ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.181+0000 7f67f7400640 1 -- 172.21.15.19:0/1240887278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec038040 con 0x7f68000cf2f0 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.181+0000 7f67f7400640 1 -- 172.21.15.19:0/1240887278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67ec033830 con 0x7f68000cf2f0 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.181+0000 7f67f7400640 1 -- 172.21.15.19:0/1240887278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec033c20 con 0x7f68000cf2f0 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.181+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680015c420 con 0x7f68000cf2f0 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.181+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680015c9e0 con 0x7f68000cf2f0 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.182+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6800050740 con 0x7f68000cf2f0 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.184+0000 7f67f7400640 1 -- 172.21.15.19:0/1240887278 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f67ec03e430 con 0x7f68000cf2f0 2024-04-25T11:31:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.185+0000 7f67f7400640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f67e00744b0 0x7f67e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.185+0000 7f67f7400640 1 -- 172.21.15.19:0/1240887278 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f67ec0b6060 con 0x7f68000cf2f0 2024-04-25T11:31:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.185+0000 7f67fea00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f67e00744b0 0x7f67e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:45.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.187+0000 7f67fea00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f67e00744b0 0x7f67e00769c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f67f0000f60 tx=0x7f67f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.190+0000 7f67f7400640 1 -- 172.21.15.19:0/1240887278 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67ec0806f0 con 0x7f68000cf2f0 2024-04-25T11:31:45.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.442+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f680006c770 con 0x7f68000cf2f0 2024-04-25T11:31:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.443+0000 7f67f7400640 1 -- 172.21.15.19:0/1240887278 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5188 (secure 0 0 0) 0x7f67ec07fff0 con 0x7f68000cf2f0 2024-04-25T11:31:45.434 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:45.434 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:38.420979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":4}]} 2024-04-25T11:31:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 59 2024-04-25T11:31:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f67e00744b0 msgr2=0x7f67e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f67e00744b0 0x7f67e00769c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f67f0000f60 tx=0x7f67f0009040 comp rx=0 tx=0).stop 2024-04-25T11:31:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000cf2f0 msgr2=0x7f680015f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000cf2f0 0x7f680015f770 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f67ec000c00 tx=0x7f67ec0046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 shutdown_connections 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f67e00744b0 0x7f67e00769c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68001608d0 0x7f6800146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68000cf2f0 0x7f680015f770 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 --2- 172.21.15.19:0/1240887278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68000b6e10 0x7f680015b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 >> 172.21.15.19:0/1240887278 conn(0x7f68000d1720 msgr2=0x7f68000b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 shutdown_connections 2024-04-25T11:31:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.447+0000 7f67ffe00640 1 -- 172.21.15.19:0/1240887278 wait complete. 2024-04-25T11:31:45.464 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T11:31:45.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.799+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:45.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.799+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=15 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-25T11:31:45.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:45.799+0000 7f1c87400640 1 client.6576 ms_handle_refused on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:47.495 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-04-25T11:31:50.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.772+0000 7f9b73e00640 1 Processor -- start 2024-04-25T11:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.772+0000 7f9b73e00640 1 -- start start 2024-04-25T11:31:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.774+0000 7f9b73e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9b740cf2f0 0x7f9b740b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.774+0000 7f9b73e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b741608d0 0x7f9b74160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.774+0000 7f9b73e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b74161220 con 0x7f9b7415a810 2024-04-25T11:31:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.774+0000 7f9b73e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7406f870 con 0x7f9b741608d0 2024-04-25T11:31:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.774+0000 7f9b73e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9b740b74d0 con 0x7f9b740cf2f0 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b73400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b741608d0 0x7f9b74160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b73400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b741608d0 0x7f9b74160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50950/0 (socket says 172.21.15.19:50950) 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b73400640 1 -- 172.21.15.19:0/1566353035 learned_addr learned my addr 172.21.15.19:0/1566353035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b72a00640 1 --2- 172.21.15.19:0/1566353035 >> v2:172.21.15.169:3300/0 conn(0x7f9b740cf2f0 0x7f9b740b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b73400640 1 -- 172.21.15.19:0/1566353035 >> v2:172.21.15.169:3300/0 conn(0x7f9b740cf2f0 msgr2=0x7f9b740b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b73400640 1 --2- 172.21.15.19:0/1566353035 >> v2:172.21.15.169:3300/0 conn(0x7f9b740cf2f0 0x7f9b740b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b73400640 1 -- 172.21.15.19:0/1566353035 >> v1:172.21.15.169:6789/0 conn(0x7f9b7415a810 legacy=0x7f9b740b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.775+0000 7f9b73400640 1 -- 172.21.15.19:0/1566353035 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b740b77b0 con 0x7f9b741608d0 2024-04-25T11:31:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.777+0000 7f9b73400640 1 --2- 172.21.15.19:0/1566353035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b741608d0 0x7f9b74160ce0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9b740d26d0 tx=0x7f9b6c02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e951fb21e538bf4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:50.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.777+0000 7f9b72000640 1 -- 172.21.15.19:0/1566353035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c009600 con 0x7f9b741608d0 2024-04-25T11:31:50.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.777+0000 7f9b72000640 1 -- 172.21.15.19:0/1566353035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0097a0 con 0x7f9b741608d0 2024-04-25T11:31:50.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.777+0000 7f9b72000640 1 -- 172.21.15.19:0/1566353035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c009b70 con 0x7f9b741608d0 2024-04-25T11:31:50.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 -- 172.21.15.19:0/1566353035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b741608d0 msgr2=0x7f9b74160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:50.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 --2- 172.21.15.19:0/1566353035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b741608d0 0x7f9b74160ce0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9b740d26d0 tx=0x7f9b6c02c900 comp rx=0 tx=0).stop 2024-04-25T11:31:50.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 -- 172.21.15.19:0/1566353035 shutdown_connections 2024-04-25T11:31:50.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 --2- 172.21.15.19:0/1566353035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b741608d0 0x7f9b74160ce0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:50.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 --2- 172.21.15.19:0/1566353035 >> v2:172.21.15.169:3300/0 conn(0x7f9b740cf2f0 0x7f9b740b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 -- 172.21.15.19:0/1566353035 >> 172.21.15.19:0/1566353035 conn(0x7f9b740d1720 msgr2=0x7f9b740d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 -- 172.21.15.19:0/1566353035 shutdown_connections 2024-04-25T11:31:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.778+0000 7f9b73e00640 1 -- 172.21.15.19:0/1566353035 wait complete. 2024-04-25T11:31:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.781+0000 7f9b73e00640 1 Processor -- start 2024-04-25T11:31:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.781+0000 7f9b73e00640 1 -- start start 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b73e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b740cf2f0 0x7f9b741505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b73e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7415a810 0x7f9b74153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b73e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b741608d0 0x7f9b740b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b73e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7415ac20 con 0x7f9b7415a810 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b73e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7406f500 con 0x7f9b741608d0 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b73e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7406fd00 con 0x7f9b740cf2f0 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7415a810 0x7f9b74153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7415a810 0x7f9b74153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50962/0 (socket says 172.21.15.19:50962) 2024-04-25T11:31:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 -- 172.21.15.19:0/3025111243 learned_addr learned my addr 172.21.15.19:0/3025111243 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b72a00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b740cf2f0 0x7f9b741505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b73400640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b741608d0 0x7f9b740b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 -- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b740cf2f0 msgr2=0x7f9b741505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b740cf2f0 0x7f9b741505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 -- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b741608d0 msgr2=0x7f9b740b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b741608d0 0x7f9b740b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.783+0000 7f9b6a000640 1 -- 172.21.15.19:0/3025111243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b74150b20 con 0x7f9b7415a810 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.784+0000 7f9b6a000640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7415a810 0x7f9b74153e60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b420 tx=0x7f9b5c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.784+0000 7f9b6b400640 1 -- 172.21.15.19:0/3025111243 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c011830 con 0x7f9b7415a810 2024-04-25T11:31:50.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.784+0000 7f9b6b400640 1 -- 172.21.15.19:0/3025111243 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b5c016070 con 0x7f9b7415a810 2024-04-25T11:31:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.784+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7415b310 con 0x7f9b7415a810 2024-04-25T11:31:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.784+0000 7f9b6b400640 1 -- 172.21.15.19:0/3025111243 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c011c10 con 0x7f9b7415a810 2024-04-25T11:31:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.784+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7415b8d0 con 0x7f9b7415a810 2024-04-25T11:31:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.787+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b74050740 con 0x7f9b7415a810 2024-04-25T11:31:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.787+0000 7f9b6b400640 1 -- 172.21.15.19:0/3025111243 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9b5c01b460 con 0x7f9b7415a810 2024-04-25T11:31:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.789+0000 7f9b6b400640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b500744b0 0x7f9b500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.789+0000 7f9b6b400640 1 -- 172.21.15.19:0/3025111243 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f9b5c094e80 con 0x7f9b7415a810 2024-04-25T11:31:50.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.789+0000 7f9b72a00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b500744b0 0x7f9b500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:50.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.790+0000 7f9b72a00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b500744b0 0x7f9b500769c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006fd0 tx=0x7f9b64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:50.795+0000 7f9b6b400640 1 -- 172.21.15.19:0/3025111243 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b5c020020 con 0x7f9b7415a810 2024-04-25T11:31:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.044+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b7406c770 con 0x7f9b7415a810 2024-04-25T11:31:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.045+0000 7f9b6b400640 1 -- 172.21.15.19:0/3025111243 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5188 (secure 0 0 0) 0x7f9b5c05f510 con 0x7f9b7415a810 2024-04-25T11:31:51.035 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:51.035 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":59,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:38.420979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":4}]} 2024-04-25T11:31:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 59 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b500744b0 msgr2=0x7f9b500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b500744b0 0x7f9b500769c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b64006fd0 tx=0x7f9b64008040 comp rx=0 tx=0).stop 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7415a810 msgr2=0x7f9b74153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7415a810 0x7f9b74153e60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b420 tx=0x7f9b5c00b8e0 comp rx=0 tx=0).stop 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 shutdown_connections 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b500744b0 0x7f9b500769c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b741608d0 0x7f9b740b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:51.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b7415a810 0x7f9b74153e60 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:51.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.048+0000 7f9b73e00640 1 --2- 172.21.15.19:0/3025111243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b740cf2f0 0x7f9b741505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:51.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.049+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 >> 172.21.15.19:0/3025111243 conn(0x7f9b740d1720 msgr2=0x7f9b740cf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:51.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.049+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 shutdown_connections 2024-04-25T11:31:51.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:51.049+0000 7f9b73e00640 1 -- 172.21.15.19:0/3025111243 wait complete. 2024-04-25T11:31:51.064 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T11:31:52.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:52.203+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:52.203+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-04-25T11:31:52.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:52.203+0000 7f1c87400640 1 client.6576 ms_handle_refused on v2:172.21.15.19:6832/1883095429 2024-04-25T11:31:54.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-04-25T11:31:56.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.373+0000 7efe44c00640 1 Processor -- start 2024-04-25T11:31:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.373+0000 7efe44c00640 1 -- start start 2024-04-25T11:31:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe44c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400b4ff0 0x7efe401611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe44c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efe400d1f70 0x7efe400d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe44c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe40161720 con 0x7efe4008cd60 2024-04-25T11:31:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe44c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efe4006e9d0 con 0x7efe400d1f70 2024-04-25T11:31:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe44c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efe4006ddc0 con 0x7efe400b4ff0 2024-04-25T11:31:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe3f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400b4ff0 0x7efe401611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe3f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400b4ff0 0x7efe401611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52060/0 (socket says 172.21.15.19:52060) 2024-04-25T11:31:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe3f400640 1 -- 172.21.15.19:0/2150038170 learned_addr learned my addr 172.21.15.19:0/2150038170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.376+0000 7efe3fe00640 1 --2- 172.21.15.19:0/2150038170 >> v2:172.21.15.169:3300/0 conn(0x7efe400d1f70 0x7efe400d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.377+0000 7efe3f400640 1 -- 172.21.15.19:0/2150038170 >> v2:172.21.15.169:3300/0 conn(0x7efe400d1f70 msgr2=0x7efe400d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.377+0000 7efe3f400640 1 --2- 172.21.15.19:0/2150038170 >> v2:172.21.15.169:3300/0 conn(0x7efe400d1f70 0x7efe400d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.377+0000 7efe3f400640 1 -- 172.21.15.19:0/2150038170 >> v1:172.21.15.169:6789/0 conn(0x7efe4008cd60 legacy=0x7efe4008d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.377+0000 7efe3f400640 1 -- 172.21.15.19:0/2150038170 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4006e0a0 con 0x7efe400b4ff0 2024-04-25T11:31:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.377+0000 7efe3fe00640 1 --2- 172.21.15.19:0/2150038170 >> v2:172.21.15.169:3300/0 conn(0x7efe400d1f70 0x7efe400d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:31:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.379+0000 7efe3f400640 1 --2- 172.21.15.19:0/2150038170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400b4ff0 0x7efe401611e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efe30009850 tx=0x7efe3002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20105669a0f46d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.379+0000 7efe3ea00640 1 -- 172.21.15.19:0/2150038170 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30004030 con 0x7efe400b4ff0 2024-04-25T11:31:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.379+0000 7efe3ea00640 1 -- 172.21.15.19:0/2150038170 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe300041d0 con 0x7efe400b4ff0 2024-04-25T11:31:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.379+0000 7efe3ea00640 1 -- 172.21.15.19:0/2150038170 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30003610 con 0x7efe400b4ff0 2024-04-25T11:31:56.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 -- 172.21.15.19:0/2150038170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400b4ff0 msgr2=0x7efe401611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:56.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 --2- 172.21.15.19:0/2150038170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400b4ff0 0x7efe401611e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efe30009850 tx=0x7efe3002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 -- 172.21.15.19:0/2150038170 shutdown_connections 2024-04-25T11:31:56.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 --2- 172.21.15.19:0/2150038170 >> v2:172.21.15.169:3300/0 conn(0x7efe400d1f70 0x7efe400d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 --2- 172.21.15.19:0/2150038170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400b4ff0 0x7efe401611e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 -- 172.21.15.19:0/2150038170 >> 172.21.15.19:0/2150038170 conn(0x7efe400b5460 msgr2=0x7efe400b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:56.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 -- 172.21.15.19:0/2150038170 shutdown_connections 2024-04-25T11:31:56.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.380+0000 7efe44c00640 1 -- 172.21.15.19:0/2150038170 wait complete. 2024-04-25T11:31:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.383+0000 7efe44c00640 1 Processor -- start 2024-04-25T11:31:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.383+0000 7efe44c00640 1 -- start start 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.384+0000 7efe44c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe4008cd60 0x7efe40152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.384+0000 7efe44c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe400b4ff0 0x7efe40158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe44c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400d1f70 0x7efe40155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe44c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe400b9180 con 0x7efe4008cd60 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe44c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efe4006e660 con 0x7efe400b4ff0 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe44c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efe4006da20 con 0x7efe400d1f70 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe4008cd60 0x7efe40152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe4008cd60 0x7efe40152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51132/0 (socket says 172.21.15.19:51132) 2024-04-25T11:31:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 -- 172.21.15.19:0/3571821464 learned_addr learned my addr 172.21.15.19:0/3571821464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe36a00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe400b4ff0 0x7efe40158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3fe00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400d1f70 0x7efe40155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 -- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400d1f70 msgr2=0x7efe40155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400d1f70 0x7efe40155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 -- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe400b4ff0 msgr2=0x7efe40158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe400b4ff0 0x7efe40158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.385+0000 7efe3f400640 1 -- 172.21.15.19:0/3571821464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe40162fc0 con 0x7efe4008cd60 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.386+0000 7efe3f400640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe4008cd60 0x7efe40152920 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.386+0000 7efe37e00640 1 -- 172.21.15.19:0/3571821464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30041020 con 0x7efe4008cd60 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.386+0000 7efe37e00640 1 -- 172.21.15.19:0/3571821464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe30003c50 con 0x7efe4008cd60 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.386+0000 7efe37e00640 1 -- 172.21.15.19:0/3571821464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30004030 con 0x7efe4008cd60 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.386+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe401632a0 con 0x7efe4008cd60 2024-04-25T11:31:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.386+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe40163860 con 0x7efe4008cd60 2024-04-25T11:31:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.388+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe40050740 con 0x7efe4008cd60 2024-04-25T11:31:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.389+0000 7efe37e00640 1 -- 172.21.15.19:0/3571821464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efe30044020 con 0x7efe4008cd60 2024-04-25T11:31:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.391+0000 7efe37e00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe1c0744b0 0x7efe1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.391+0000 7efe37e00640 1 -- 172.21.15.19:0/3571821464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7efe300b5e00 con 0x7efe4008cd60 2024-04-25T11:31:56.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.391+0000 7efe36a00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe1c0744b0 0x7efe1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.393+0000 7efe36a00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe1c0744b0 0x7efe1c0769c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:56.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.396+0000 7efe37e00640 1 -- 172.21.15.19:0/3571821464 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe3004b030 con 0x7efe4008cd60 2024-04-25T11:31:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.642+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe400b6370 con 0x7efe4008cd60 2024-04-25T11:31:56.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.643+0000 7efe37e00640 1 -- 172.21.15.19:0/3571821464 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5236 (secure 0 0 0) 0x7efe30080520 con 0x7efe4008cd60 2024-04-25T11:31:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:56.633 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":60,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:54.012023+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6486":{"gid":6486,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6833/1883095429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1883095429},{"type":"v1","addr":"172.21.15.19:6833","nonce":1883095429}]},"join_fscid":-1,"laggy_since":"2024-04-25T11:31:54.012023+0000","export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6486,"qdb_cluster":[6486]},"id":4}]} 2024-04-25T11:31:56.633 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 60 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe1c0744b0 msgr2=0x7efe1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe1c0744b0 0x7efe1c0769c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).stop 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe4008cd60 msgr2=0x7efe40152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe4008cd60 0x7efe40152920 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe300046c0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 shutdown_connections 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efe1c0744b0 0x7efe1c0769c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe400b4ff0 0x7efe40158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe400d1f70 0x7efe40155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 --2- 172.21.15.19:0/3571821464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe4008cd60 0x7efe40152920 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.646+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 >> 172.21.15.19:0/3571821464 conn(0x7efe400b5460 msgr2=0x7efe400b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.647+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 shutdown_connections 2024-04-25T11:31:56.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:56.647+0000 7efe44c00640 1 -- 172.21.15.19:0/3571821464 wait complete. 2024-04-25T11:31:56.666 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-04-25T11:31:56.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-04-25T11:31:57.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.060+0000 7fa19fe00640 1 Processor -- start 2024-04-25T11:31:57.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.060+0000 7fa19fe00640 1 -- start start 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.062+0000 7fa19fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1a013ff60 0x7fa1a015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.062+0000 7fa19fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa1a0070970 0x7fa1a0140490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a00d02a0 con 0x7fa1a00d1340 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a00d0580 con 0x7fa1a013ff60 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a00d0860 con 0x7fa1a0070970 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1a013ff60 0x7fa1a015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa1a00d1340 0x7fa1a0070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45004/0 (socket says 172.21.15.19:45004) 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1a013ff60 0x7fa1a015dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51140/0 (socket says 172.21.15.19:51140) 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19e000640 1 -- 172.21.15.19:0/3434210969 learned_addr learned my addr 172.21.15.19:0/3434210969 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19f400640 1 --2- 172.21.15.19:0/3434210969 >> v2:172.21.15.169:3300/0 conn(0x7fa1a0070970 0x7fa1a0140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19ea00640 1 -- 172.21.15.19:0/3434210969 >> v2:172.21.15.169:3300/0 conn(0x7fa1a0070970 msgr2=0x7fa1a0140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19ea00640 1 --2- 172.21.15.19:0/3434210969 >> v2:172.21.15.169:3300/0 conn(0x7fa1a0070970 0x7fa1a0140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.063+0000 7fa19ea00640 1 -- 172.21.15.19:0/3434210969 >> v1:172.21.15.19:6789/0 conn(0x7fa1a00d1340 legacy=0x7fa1a0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.064+0000 7fa19ea00640 1 -- 172.21.15.19:0/3434210969 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a00d0b40 con 0x7fa1a013ff60 2024-04-25T11:31:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.064+0000 7fa19d600640 1 -- 172.21.15.19:0/3434210969 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809446118 0 0) 0x7fa1a00d02a0 con 0x7fa1a00d1340 2024-04-25T11:31:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.065+0000 7fa19ea00640 1 --2- 172.21.15.19:0/3434210969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a013ff60 0x7fa1a015dc30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00714e0 tx=0x7fa19002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcef24f0fd6077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.065+0000 7fa19d600640 1 -- 172.21.15.19:0/3434210969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190007740 con 0x7fa1a013ff60 2024-04-25T11:31:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.065+0000 7fa19d600640 1 -- 172.21.15.19:0/3434210969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1900078e0 con 0x7fa1a013ff60 2024-04-25T11:31:57.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.065+0000 7fa19d600640 1 -- 172.21.15.19:0/3434210969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190007cb0 con 0x7fa1a013ff60 2024-04-25T11:31:57.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 -- 172.21.15.19:0/3434210969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a013ff60 msgr2=0x7fa1a015dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 --2- 172.21.15.19:0/3434210969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a013ff60 0x7fa1a015dc30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00714e0 tx=0x7fa19002a5f0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 -- 172.21.15.19:0/3434210969 shutdown_connections 2024-04-25T11:31:57.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 --2- 172.21.15.19:0/3434210969 >> v2:172.21.15.169:3300/0 conn(0x7fa1a0070970 0x7fa1a0140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 --2- 172.21.15.19:0/3434210969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a013ff60 0x7fa1a015dc30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 -- 172.21.15.19:0/3434210969 >> 172.21.15.19:0/3434210969 conn(0x7fa1a00b4260 msgr2=0x7fa1a0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:57.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 -- 172.21.15.19:0/3434210969 shutdown_connections 2024-04-25T11:31:57.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.066+0000 7fa19fe00640 1 -- 172.21.15.19:0/3434210969 wait complete. 2024-04-25T11:31:57.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.068+0000 7fa19fe00640 1 Processor -- start 2024-04-25T11:31:57.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.068+0000 7fa19fe00640 1 -- start start 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1a013ff60 0x7fa1a0153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1a01397b0 0x7fa1a01f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a01f9aa0 0x7fa1a013d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a015d780 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a015d600 con 0x7fa1a013ff60 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a015e170 con 0x7fa1a01397b0 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a01f9aa0 0x7fa1a013d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a01f9aa0 0x7fa1a013d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51142/0 (socket says 172.21.15.19:51142) 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.070+0000 7fa19f400640 1 -- 172.21.15.19:0/399340462 learned_addr learned my addr 172.21.15.19:0/399340462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19ea00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1a013ff60 0x7fa1a0153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19e000640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1a01397b0 0x7fa1a01f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19f400640 1 -- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1a01397b0 msgr2=0x7fa1a01f9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19f400640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1a01397b0 0x7fa1a01f9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19f400640 1 -- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1a013ff60 msgr2=0x7fa1a0153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19f400640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1a013ff60 0x7fa1a0153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19f400640 1 -- 172.21.15.19:0/399340462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a01f9eb0 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19ea00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1a013ff60 0x7fa1a0153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19e000640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1a01397b0 0x7fa1a01f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19f400640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a01f9aa0 0x7fa1a013d5e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa19800d5d0 tx=0x7fa19800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa196a00640 1 -- 172.21.15.19:0/399340462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980125f0 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa196a00640 1 -- 172.21.15.19:0/399340462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa198019070 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a0157680 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.071+0000 7fa196a00640 1 -- 172.21.15.19:0/399340462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980129b0 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.072+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a0160410 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.073+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0076c60 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.074+0000 7fa196a00640 1 -- 172.21.15.19:0/399340462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa198012b50 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.075+0000 7fa196a00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa17c0744e0 0x7fa17c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.076+0000 7fa196a00640 1 -- 172.21.15.19:0/399340462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fa198095640 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.076+0000 7fa19ea00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa17c0744e0 0x7fa17c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.078+0000 7fa19ea00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa17c0744e0 0x7fa17c0769f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa190034000 tx=0x7fa190033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:57.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.080+0000 7fa196a00640 1 -- 172.21.15.19:0/399340462 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1a01f9eb0 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.285+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7fa1a013b620 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.292+0000 7fa196a00640 1 -- 172.21.15.19:0/399340462 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fa19805fce0 con 0x7fa1a01f9aa0 2024-04-25T11:31:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa17c0744e0 msgr2=0x7fa17c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa17c0744e0 0x7fa17c0769f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa190034000 tx=0x7fa190033040 comp rx=0 tx=0).stop 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a01f9aa0 msgr2=0x7fa1a013d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a01f9aa0 0x7fa1a013d5e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa19800d5d0 tx=0x7fa19800da90 comp rx=0 tx=0).stop 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 shutdown_connections 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa17c0744e0 0x7fa17c0769f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a01f9aa0 0x7fa1a013d5e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1a01397b0 0x7fa1a01f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 --2- 172.21.15.19:0/399340462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1a013ff60 0x7fa1a0153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 >> 172.21.15.19:0/399340462 conn(0x7fa1a00d1340 msgr2=0x7fa1a014fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 shutdown_connections 2024-04-25T11:31:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.295+0000 7fa19fe00640 1 -- 172.21.15.19:0/399340462 wait complete. 2024-04-25T11:31:57.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2024-04-25T11:31:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.595+0000 7f4f76400640 1 Processor -- start 2024-04-25T11:31:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.595+0000 7f4f76400640 1 -- start start 2024-04-25T11:31:57.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.597+0000 7f4f76400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f7014ddc0 0x7f4f700b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.597+0000 7f4f76400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70168030 0x7f4f70168440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.597+0000 7f4f76400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f70168980 con 0x7f4f70164a40 2024-04-25T11:31:57.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.597+0000 7f4f76400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4f700b5ba0 con 0x7f4f7014ddc0 2024-04-25T11:31:57.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.597+0000 7f4f76400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f700b5e80 con 0x7f4f70168030 2024-04-25T11:31:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.598+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70168030 0x7f4f70168440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.598+0000 7f4f75000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f7014ddc0 0x7f4f700b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.598+0000 7f4f75000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f7014ddc0 0x7f4f700b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:49244/0 (socket says 172.21.15.19:49244) 2024-04-25T11:31:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.598+0000 7f4f75000640 1 -- 172.21.15.19:0/2926533704 learned_addr learned my addr 172.21.15.19:0/2926533704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.598+0000 7f4f75a00640 1 --1- 172.21.15.19:0/2926533704 >> v1:172.21.15.169:6789/0 conn(0x7f4f70164a40 0x7f4f70164e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:43604/0 (socket says 172.21.15.19:43604) 2024-04-25T11:31:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.599+0000 7f4f6fe00640 1 -- 172.21.15.19:0/2926533704 >> v2:172.21.15.169:3300/0 conn(0x7f4f7014ddc0 msgr2=0x7f4f700b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.599+0000 7f4f6fe00640 1 --2- 172.21.15.19:0/2926533704 >> v2:172.21.15.169:3300/0 conn(0x7f4f7014ddc0 0x7f4f700b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.599+0000 7f4f6fe00640 1 -- 172.21.15.19:0/2926533704 >> v1:172.21.15.169:6789/0 conn(0x7f4f70164a40 legacy=0x7f4f70164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.599+0000 7f4f6fe00640 1 -- 172.21.15.19:0/2926533704 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700b6160 con 0x7f4f70168030 2024-04-25T11:31:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.599+0000 7f4f75000640 1 --2- 172.21.15.19:0/2926533704 >> v2:172.21.15.169:3300/0 conn(0x7f4f7014ddc0 0x7f4f700b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:31:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.601+0000 7f4f6fe00640 1 --2- 172.21.15.19:0/2926533704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70168030 0x7f4f70168440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0097a0 tx=0x7f4f5c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ecfcaf63d08e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.601+0000 7f4f6f400640 1 -- 172.21.15.19:0/2926533704 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c037070 con 0x7f4f70168030 2024-04-25T11:31:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.601+0000 7f4f6f400640 1 -- 172.21.15.19:0/2926533704 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5c032440 con 0x7f4f70168030 2024-04-25T11:31:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.601+0000 7f4f6f400640 1 -- 172.21.15.19:0/2926533704 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c032810 con 0x7f4f70168030 2024-04-25T11:31:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 -- 172.21.15.19:0/2926533704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70168030 msgr2=0x7f4f70168440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 --2- 172.21.15.19:0/2926533704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70168030 0x7f4f70168440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0097a0 tx=0x7f4f5c02a180 comp rx=0 tx=0).stop 2024-04-25T11:31:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 -- 172.21.15.19:0/2926533704 shutdown_connections 2024-04-25T11:31:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 --2- 172.21.15.19:0/2926533704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70168030 0x7f4f70168440 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 --2- 172.21.15.19:0/2926533704 >> v2:172.21.15.169:3300/0 conn(0x7f4f7014ddc0 0x7f4f700b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 -- 172.21.15.19:0/2926533704 >> 172.21.15.19:0/2926533704 conn(0x7f4f700b4260 msgr2=0x7f4f70072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 -- 172.21.15.19:0/2926533704 shutdown_connections 2024-04-25T11:31:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.602+0000 7f4f76400640 1 -- 172.21.15.19:0/2926533704 wait complete. 2024-04-25T11:31:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.604+0000 7f4f76400640 1 Processor -- start 2024-04-25T11:31:57.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.604+0000 7f4f76400640 1 -- start start 2024-04-25T11:31:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f76400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70164a40 0x7f4f70148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f76400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f70168030 0x7f4f701fa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f76400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7015ed70 0x7f4f70162f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f76400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f700b57e0 con 0x7f4f7015ed70 2024-04-25T11:31:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f76400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7006f430 con 0x7f4f70168030 2024-04-25T11:31:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f76400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4f700b5960 con 0x7f4f70164a40 2024-04-25T11:31:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f75a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7015ed70 0x7f4f70162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f75a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7015ed70 0x7f4f70162f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51144/0 (socket says 172.21.15.19:51144) 2024-04-25T11:31:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.606+0000 7f4f75a00640 1 -- 172.21.15.19:0/126117494 learned_addr learned my addr 172.21.15.19:0/126117494 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f6fe00640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f70168030 0x7f4f701fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f75000640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70164a40 0x7f4f70148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f75a00640 1 -- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70164a40 msgr2=0x7f4f70148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f75a00640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70164a40 0x7f4f70148b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f75a00640 1 -- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f70168030 msgr2=0x7f4f701fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f75a00640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f70168030 0x7f4f701fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f75a00640 1 -- 172.21.15.19:0/126117494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f701490b0 con 0x7f4f7015ed70 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f6fe00640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f70168030 0x7f4f701fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:31:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f75a00640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7015ed70 0x7f4f70162f10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c390 tx=0x7f4f6800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f6cc00640 1 -- 172.21.15.19:0/126117494 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680074e0 con 0x7f4f7015ed70 2024-04-25T11:31:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f6cc00640 1 -- 172.21.15.19:0/126117494 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f68007680 con 0x7f4f7015ed70 2024-04-25T11:31:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f70161e70 con 0x7f4f7015ed70 2024-04-25T11:31:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.607+0000 7f4f6cc00640 1 -- 172.21.15.19:0/126117494 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f680049e0 con 0x7f4f7015ed70 2024-04-25T11:31:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.608+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f70162430 con 0x7f4f7015ed70 2024-04-25T11:31:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.609+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f70076c60 con 0x7f4f7015ed70 2024-04-25T11:31:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.610+0000 7f4f6cc00640 1 -- 172.21.15.19:0/126117494 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f6801a020 con 0x7f4f7015ed70 2024-04-25T11:31:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.612+0000 7f4f6cc00640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f500744b0 0x7f4f500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.612+0000 7f4f6cc00640 1 -- 172.21.15.19:0/126117494 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f4f68094aa0 con 0x7f4f7015ed70 2024-04-25T11:31:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.612+0000 7f4f75000640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f500744b0 0x7f4f500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.613+0000 7f4f75000640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f500744b0 0x7f4f500769c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4f700b66b0 tx=0x7f4f6000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.617+0000 7f4f6cc00640 1 -- 172.21.15.19:0/126117494 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f70168980 con 0x7f4f7015ed70 2024-04-25T11:31:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:57.886+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) -- 0x7f4f70161c00 con 0x7f4f7015ed70 2024-04-25T11:31:58.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.293+0000 7f4f6cc00640 1 -- 172.21.15.19:0/126117494 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.6486 v61) ==== 115+0+0 (secure 0 0 0) 0x7f4f6805f130 con 0x7f4f7015ed70 2024-04-25T11:31:58.283 INFO:teuthology.orchestra.run.smithi019.stderr:unfreezing mds.6486 2024-04-25T11:31:58.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.296+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f500744b0 msgr2=0x7f4f500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:58.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.296+0000 7f4f76400640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f500744b0 0x7f4f500769c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4f700b66b0 tx=0x7f4f6000b040 comp rx=0 tx=0).stop 2024-04-25T11:31:58.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7015ed70 msgr2=0x7f4f70162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7015ed70 0x7f4f70162f10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c390 tx=0x7f4f6800c850 comp rx=0 tx=0).stop 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 shutdown_connections 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f500744b0 0x7f4f500769c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f70168030 0x7f4f701fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7015ed70 0x7f4f70162f10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 --2- 172.21.15.19:0/126117494 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f70164a40 0x7f4f70148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 >> 172.21.15.19:0/126117494 conn(0x7f4f7014ddc0 msgr2=0x7f4f7014ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 shutdown_connections 2024-04-25T11:31:58.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.297+0000 7f4f76400640 1 -- 172.21.15.19:0/126117494 wait complete. 2024-04-25T11:31:58.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -n kernel.core_pattern 2024-04-25T11:31:58.402 INFO:teuthology.orchestra.run.smithi019.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:31:58.405 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:31:58.405 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-04-25T11:31:58.497 INFO:teuthology.orchestra.run.smithi019.stdout:1714044699.52787.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-04-25T11:31:58.502 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1714044699.52787.core'] 2024-04-25T11:31:58.502 INFO:tasks.cephfs.cephfs_test_case:Found core file 1714044699.52787.core, deleting it 2024-04-25T11:31:58.502 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1714044699.52787.core 2024-04-25T11:31:58.621 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:31:58.622 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:31:58.622 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:31:58.622 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:31:58.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:31:58.622 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi019 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-04-25T11:31:58.623 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:31:58.623 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:31:58.630 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:31:58.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:31:58.859 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:31:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.901+0000 7ffa5a800640 1 Processor -- start 2024-04-25T11:31:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.901+0000 7ffa5a800640 1 -- start start 2024-04-25T11:31:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.903+0000 7ffa5a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa541612f0 0x7ffa540d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.903+0000 7ffa5a800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5406f3e0 con 0x7ffa540d0ed0 2024-04-25T11:31:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.903+0000 7ffa5a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5406f6c0 con 0x7ffa541434d0 2024-04-25T11:31:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.903+0000 7ffa5a800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ffa541438e0 con 0x7ffa541612f0 2024-04-25T11:31:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.904+0000 7ffa4be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffa540d0ed0 0x7ffa54160be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45022/0 (socket says 172.21.15.19:45022) 2024-04-25T11:31:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.904+0000 7ffa4be00640 1 -- 172.21.15.19:0/3906123053 learned_addr learned my addr 172.21.15.19:0/3906123053 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:31:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.904+0000 7ffa53e00640 1 --2- 172.21.15.19:0/3906123053 >> v2:172.21.15.169:3300/0 conn(0x7ffa541612f0 0x7ffa540d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.904+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3460890918 0 0) 0x7ffa5406f3e0 con 0x7ffa540d0ed0 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.904+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa38003520 con 0x7ffa540d0ed0 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 204053368 0 0) 0x7ffa38003520 con 0x7ffa540d0ed0 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 >> v2:172.21.15.169:3300/0 conn(0x7ffa541612f0 msgr2=0x7ffa540d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 --2- 172.21.15.19:0/3906123053 >> v2:172.21.15.169:3300/0 conn(0x7ffa541612f0 0x7ffa540d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 >> v1:172.21.15.169:6789/0 conn(0x7ffa541434d0 legacy=0x7ffa54070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5406f9a0 con 0x7ffa540d0ed0 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ffa3c001670 con 0x7ffa540d0ed0 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 >> v1:172.21.15.19:6789/0 conn(0x7ffa540d0ed0 legacy=0x7ffa54160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa380039e0 0x7ffa38003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa38004800 0x7ffa38004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa38005270 0x7ffa3800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541438e0 con 0x7ffa380039e0 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3800e3a0 con 0x7ffa38005270 2024-04-25T11:31:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3800e5a0 con 0x7ffa38004800 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.905+0000 7ffa53400640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa380039e0 0x7ffa38003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa4be00640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa38004800 0x7ffa38004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa53e00640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa38005270 0x7ffa3800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa53400640 1 -- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa38004800 msgr2=0x7ffa38004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa53400640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa38004800 0x7ffa38004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa53400640 1 -- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa38005270 msgr2=0x7ffa3800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa53400640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa38005270 0x7ffa3800dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa53e00640 1 -- 172.21.15.19:0/3906123053 reap_dead start 2024-04-25T11:31:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.906+0000 7ffa53400640 1 -- 172.21.15.19:0/3906123053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa38003520 con 0x7ffa380039e0 2024-04-25T11:31:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.907+0000 7ffa53400640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa380039e0 0x7ffa38003e50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ffa44001750 tx=0x7ffa38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef192c5101821d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.907+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44001370 con 0x7ffa380039e0 2024-04-25T11:31:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.907+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa440052d0 con 0x7ffa380039e0 2024-04-25T11:31:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.907+0000 7ffa52a00640 1 -- 172.21.15.19:0/3906123053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4402abc0 con 0x7ffa380039e0 2024-04-25T11:31:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.908+0000 7ffa5a800640 1 -- 172.21.15.19:0/3906123053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa380039e0 msgr2=0x7ffa38003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.908+0000 7ffa5a800640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa380039e0 0x7ffa38003e50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ffa44001750 tx=0x7ffa38005830 comp rx=0 tx=0).stop 2024-04-25T11:31:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.908+0000 7ffa5a800640 1 -- 172.21.15.19:0/3906123053 shutdown_connections 2024-04-25T11:31:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.908+0000 7ffa5a800640 1 --2- 172.21.15.19:0/3906123053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa380039e0 0x7ffa38003e50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.908+0000 7ffa5a800640 1 -- 172.21.15.19:0/3906123053 >> 172.21.15.19:0/3906123053 conn(0x7ffa540d1ae0 msgr2=0x7ffa540d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.908+0000 7ffa5a800640 1 -- 172.21.15.19:0/3906123053 shutdown_connections 2024-04-25T11:31:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.908+0000 7ffa5a800640 1 -- 172.21.15.19:0/3906123053 wait complete. 2024-04-25T11:31:58.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.911+0000 7ffa5a800640 1 Processor -- start 2024-04-25T11:31:58.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.911+0000 7ffa5a800640 1 -- start start 2024-04-25T11:31:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.912+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa380039e0 0x7ffa540b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.912+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa5415ab60 0x7ffa540af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.912+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa541f5ca0 0x7ffa541f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.912+0000 7ffa5a800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa540d12e0 con 0x7ffa541f5ca0 2024-04-25T11:31:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.912+0000 7ffa5a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5406f090 con 0x7ffa380039e0 2024-04-25T11:31:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.912+0000 7ffa5a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa540d1460 con 0x7ffa5415ab60 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa541f5ca0 0x7ffa541f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa541f5ca0 0x7ffa541f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51164/0 (socket says 172.21.15.19:51164) 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 -- 172.21.15.19:0/1993397369 learned_addr learned my addr 172.21.15.19:0/1993397369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53400640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa380039e0 0x7ffa540b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa4be00640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa5415ab60 0x7ffa540af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 -- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa5415ab60 msgr2=0x7ffa540af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa5415ab60 0x7ffa540af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 -- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa380039e0 msgr2=0x7ffa540b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa380039e0 0x7ffa540b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.913+0000 7ffa53e00640 1 -- 172.21.15.19:0/1993397369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa540afb40 con 0x7ffa541f5ca0 2024-04-25T11:31:58.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.914+0000 7ffa53e00640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa541f5ca0 0x7ffa541f60b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffa540cfa70 tx=0x7ffa4c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:58.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.914+0000 7ffa4b400640 1 -- 172.21.15.19:0/1993397369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c016620 con 0x7ffa541f5ca0 2024-04-25T11:31:58.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.914+0000 7ffa4b400640 1 -- 172.21.15.19:0/1993397369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c01b070 con 0x7ffa541f5ca0 2024-04-25T11:31:58.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.914+0000 7ffa4b400640 1 -- 172.21.15.19:0/1993397369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c016a20 con 0x7ffa541f5ca0 2024-04-25T11:31:58.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.914+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa540afe20 con 0x7ffa541f5ca0 2024-04-25T11:31:58.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.914+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa540b03e0 con 0x7ffa541f5ca0 2024-04-25T11:31:58.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.915+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa54050740 con 0x7ffa541f5ca0 2024-04-25T11:31:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.917+0000 7ffa4b400640 1 -- 172.21.15.19:0/1993397369 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffa4c005d10 con 0x7ffa541f5ca0 2024-04-25T11:31:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.918+0000 7ffa4b400640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa300744b0 0x7ffa300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:31:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.918+0000 7ffa4b400640 1 -- 172.21.15.19:0/1993397369 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7ffa4c095fc0 con 0x7ffa541f5ca0 2024-04-25T11:31:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.918+0000 7ffa53400640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa300744b0 0x7ffa300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:31:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.920+0000 7ffa53400640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa300744b0 0x7ffa300769c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffa44032000 tx=0x7ffa4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:31:58.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:58.923+0000 7ffa4b400640 1 -- 172.21.15.19:0/1993397369 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa4c025020 con 0x7ffa541f5ca0 2024-04-25T11:31:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.172+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa5406e100 con 0x7ffa541f5ca0 2024-04-25T11:31:59.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.173+0000 7ffa4b400640 1 -- 172.21.15.19:0/1993397369 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+4356 (secure 0 0 0) 0x7ffa4c060600 con 0x7ffa541f5ca0 2024-04-25T11:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:58.299053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5452":{"gid":5452,"name":"d","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-25T11:31:59.163 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 62 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa300744b0 msgr2=0x7ffa300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa300744b0 0x7ffa300769c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffa44032000 tx=0x7ffa4402e000 comp rx=0 tx=0).stop 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa541f5ca0 msgr2=0x7ffa541f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa541f5ca0 0x7ffa541f60b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ffa540cfa70 tx=0x7ffa4c00f520 comp rx=0 tx=0).stop 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 shutdown_connections 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa300744b0 0x7ffa300769c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa541f5ca0 0x7ffa541f60b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa5415ab60 0x7ffa540af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 --2- 172.21.15.19:0/1993397369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa380039e0 0x7ffa540b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 >> 172.21.15.19:0/1993397369 conn(0x7ffa540d1ae0 msgr2=0x7ffa540d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 shutdown_connections 2024-04-25T11:31:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:31:59.176+0000 7ffa5a800640 1 -- 172.21.15.19:0/1993397369 wait complete. 2024-04-25T11:31:59.597 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:31:06.789413+0000', 'modified': '2024-04-25T11:31:58.299053+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5452': {'gid': 5452, 'name': 'd', 'rank': 0, 'incarnation': 62, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.169:6837/1885933205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1885933205}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1885933205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:31:59.597 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5452:up:replay 2024-04-25T11:32:00.598 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:32:00.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T11:32:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.902+0000 7fc426400640 1 Processor -- start 2024-04-25T11:32:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.902+0000 7fc426400640 1 -- start start 2024-04-25T11:32:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.904+0000 7fc426400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4200d0f10 0x7fc42006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.904+0000 7fc426400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4200b2c00 con 0x7fc420164e90 2024-04-25T11:32:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.904+0000 7fc426400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4200b9240 con 0x7fc42006ddf0 2024-04-25T11:32:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.904+0000 7fc426400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc4200b9520 con 0x7fc4200d0f10 2024-04-25T11:32:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.904+0000 7fc425a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc420164e90 0x7fc4200d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45024/0 (socket says 172.21.15.19:45024) 2024-04-25T11:32:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.904+0000 7fc425a00640 1 -- 172.21.15.19:0/1773921557 learned_addr learned my addr 172.21.15.19:0/1773921557 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc425000640 1 --2- 172.21.15.19:0/1773921557 >> v2:172.21.15.169:3300/0 conn(0x7fc4200d0f10 0x7fc42006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1339844168 0 0) 0x7fc4200b2c00 con 0x7fc420164e90 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc404003520 con 0x7fc420164e90 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2289672617 0 0) 0x7fc404003520 con 0x7fc420164e90 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 >> v2:172.21.15.169:3300/0 conn(0x7fc4200d0f10 msgr2=0x7fc42006d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 --2- 172.21.15.19:0/1773921557 >> v2:172.21.15.169:3300/0 conn(0x7fc4200d0f10 0x7fc42006d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 >> v1:172.21.15.169:6789/0 conn(0x7fc42006ddf0 legacy=0x7fc420164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42009d170 con 0x7fc420164e90 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fc418001670 con 0x7fc420164e90 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 >> v1:172.21.15.19:6789/0 conn(0x7fc420164e90 legacy=0x7fc4200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc404003a00 0x7fc404003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.905+0000 7fc41f400640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc404004800 0x7fc404004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc41f400640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc404005270 0x7fc40400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4200b9520 con 0x7fc404003a00 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc40400c190 con 0x7fc404004800 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc40400c390 con 0x7fc404005270 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc425000640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc404003a00 0x7fc404003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc41fe00640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc404004800 0x7fc404004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc425000640 1 -- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc404005270 msgr2=0x7fc40400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc425000640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc404005270 0x7fc40400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc425000640 1 -- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc404004800 msgr2=0x7fc404004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc425000640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc404004800 0x7fc404004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc425000640 1 -- 172.21.15.19:0/1773921557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc404003520 con 0x7fc404003a00 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc41fe00640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc404004800 0x7fc404004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.906+0000 7fc425a00640 1 -- 172.21.15.19:0/1773921557 reap_dead start 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.908+0000 7fc425000640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc404003a00 0x7fc404003e70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc41000e140 tx=0x7fc404005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2c56bfb53873c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.908+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410009c60 con 0x7fc404003a00 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.908+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc410036070 con 0x7fc404003a00 2024-04-25T11:32:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.908+0000 7fc41f400640 1 -- 172.21.15.19:0/1773921557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410031840 con 0x7fc404003a00 2024-04-25T11:32:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.908+0000 7fc426400640 1 -- 172.21.15.19:0/1773921557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc404003a00 msgr2=0x7fc404003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.908+0000 7fc426400640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc404003a00 0x7fc404003e70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc41000e140 tx=0x7fc404005830 comp rx=0 tx=0).stop 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.909+0000 7fc426400640 1 -- 172.21.15.19:0/1773921557 shutdown_connections 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.909+0000 7fc426400640 1 --2- 172.21.15.19:0/1773921557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc404003a00 0x7fc404003e70 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.909+0000 7fc426400640 1 -- 172.21.15.19:0/1773921557 >> 172.21.15.19:0/1773921557 conn(0x7fc4200d03f0 msgr2=0x7fc4200d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.909+0000 7fc426400640 1 -- 172.21.15.19:0/1773921557 shutdown_connections 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.909+0000 7fc426400640 1 -- 172.21.15.19:0/1773921557 wait complete. 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.911+0000 7fc426400640 1 Processor -- start 2024-04-25T11:32:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.912+0000 7fc426400640 1 -- start start 2024-04-25T11:32:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.913+0000 7fc426400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc404003a00 0x7fc42009ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.913+0000 7fc426400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc420150900 0x7fc42009e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.913+0000 7fc426400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc420152730 0x7fc420152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.913+0000 7fc426400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4201652a0 con 0x7fc420150900 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.913+0000 7fc426400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc42009cce0 con 0x7fc420152730 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.913+0000 7fc426400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc420165420 con 0x7fc404003a00 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc420150900 0x7fc42009e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc420150900 0x7fc42009e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51184/0 (socket says 172.21.15.19:51184) 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 -- 172.21.15.19:0/2773733189 learned_addr learned my addr 172.21.15.19:0/2773733189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc425a00640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc420152730 0x7fc420152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:00.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 -- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc404003a00 msgr2=0x7fc42009ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc404003a00 0x7fc42009ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 -- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc420152730 msgr2=0x7fc420152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc420152730 0x7fc420152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.914+0000 7fc41fe00640 1 -- 172.21.15.19:0/2773733189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc420151690 con 0x7fc420150900 2024-04-25T11:32:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.915+0000 7fc41fe00640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc420150900 0x7fc42009e0e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc40c000f60 tx=0x7fc40c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.915+0000 7fc41cc00640 1 -- 172.21.15.19:0/2773733189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c005480 con 0x7fc420150900 2024-04-25T11:32:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.915+0000 7fc41cc00640 1 -- 172.21.15.19:0/2773733189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c017070 con 0x7fc420150900 2024-04-25T11:32:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.915+0000 7fc41cc00640 1 -- 172.21.15.19:0/2773733189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c003810 con 0x7fc420150900 2024-04-25T11:32:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.915+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc420151970 con 0x7fc420150900 2024-04-25T11:32:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.915+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc420151f30 con 0x7fc420150900 2024-04-25T11:32:00.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.916+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42009e620 con 0x7fc420150900 2024-04-25T11:32:00.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.917+0000 7fc41cc00640 1 -- 172.21.15.19:0/2773733189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc40c0039b0 con 0x7fc420150900 2024-04-25T11:32:00.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.918+0000 7fc41cc00640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4000744e0 0x7fc4000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:00.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.919+0000 7fc41cc00640 1 -- 172.21.15.19:0/2773733189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7fc40c095240 con 0x7fc420150900 2024-04-25T11:32:00.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.919+0000 7fc425000640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4000744e0 0x7fc4000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:00.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.921+0000 7fc425000640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4000744e0 0x7fc4000769f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc410035000 tx=0x7fc41002d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:00.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:00.924+0000 7fc41cc00640 1 -- 172.21.15.19:0/2773733189 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc40c05f880 con 0x7fc420150900 2024-04-25T11:32:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.167+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc42006e3f0 con 0x7fc420150900 2024-04-25T11:32:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.168+0000 7fc41cc00640 1 -- 172.21.15.19:0/2773733189 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5182 (secure 0 0 0) 0x7fc40c05f180 con 0x7fc420150900 2024-04-25T11:32:01.158 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:32:01.158 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3770220424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3770220424},{"type":"v1","addr":"172.21.15.19:6833","nonce":3770220424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:31:59.324262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5452":{"gid":5452,"name":"d","rank":0,"incarnation":62,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-04-25T11:32:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 64 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4000744e0 msgr2=0x7fc4000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4000744e0 0x7fc4000769f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc410035000 tx=0x7fc41002d000 comp rx=0 tx=0).stop 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc420150900 msgr2=0x7fc42009e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc420150900 0x7fc42009e0e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc40c000f60 tx=0x7fc40c00d9f0 comp rx=0 tx=0).stop 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 shutdown_connections 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4000744e0 0x7fc4000769f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc420152730 0x7fc420152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc420150900 0x7fc42009e0e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 --2- 172.21.15.19:0/2773733189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc404003a00 0x7fc42009ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.171+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 >> 172.21.15.19:0/2773733189 conn(0x7fc4200d03f0 msgr2=0x7fc420072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.172+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 shutdown_connections 2024-04-25T11:32:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:01.172+0000 7fc426400640 1 -- 172.21.15.19:0/2773733189 wait complete. 2024-04-25T11:32:01.589 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:31:06.789413+0000', 'modified': '2024-04-25T11:31:59.324262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5452': {'gid': 5452, 'name': 'd', 'rank': 0, 'incarnation': 62, 'state': 'up:rejoin', 'state_seq': 17, 'addr': '172.21.15.169:6837/1885933205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1885933205}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1885933205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:32:01.590 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5452:up:rejoin 2024-04-25T11:32:02.178 INFO:tasks.ceph.mds.d.smithi169.stderr:2024-04-25T11:32:02.188+0000 7ff844400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:32:02.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:32:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.895+0000 7f2bdd000640 1 Processor -- start 2024-04-25T11:32:02.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.895+0000 7f2bdd000640 1 -- start start 2024-04-25T11:32:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.897+0000 7f2bdd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 0x7f2bd80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.897+0000 7f2bdd000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd80d2330 con 0x7f2bd80d33d0 2024-04-25T11:32:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.897+0000 7f2bdd000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd80d2610 con 0x7f2bd80d0be0 2024-04-25T11:32:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.897+0000 7f2bdd000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd80d28f0 con 0x7f2bd80d45e0 2024-04-25T11:32:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.898+0000 7f2bd5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2bd80d33d0 0x7f2bd80d04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45036/0 (socket says 172.21.15.19:45036) 2024-04-25T11:32:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.898+0000 7f2bd5e00640 1 -- 172.21.15.19:0/1534890599 learned_addr learned my addr 172.21.15.19:0/1534890599 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:32:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.898+0000 7f2bd6800640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 0x7f2bd80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.898+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1772179168 0 0) 0x7f2bd80d2330 con 0x7f2bd80d33d0 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.898+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bbc003520 con 0x7f2bd80d33d0 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 840245139 0 0) 0x7f2bbc003520 con 0x7f2bd80d33d0 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 msgr2=0x7f2bd80b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 0x7f2bd80b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 >> v1:172.21.15.169:6789/0 conn(0x7f2bd80d0be0 legacy=0x7f2bd80ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80d2bd0 con 0x7f2bd80d33d0 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2bc8001810 con 0x7f2bd80d33d0 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 >> v1:172.21.15.19:6789/0 conn(0x7f2bd80d33d0 legacy=0x7f2bd80d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 0x7f2bbc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc003fb0 0x7f2bbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bbc004d40 0x7f2bbc0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80d28f0 con 0x7f2bbc004d40 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc00bbb0 con 0x7f2bbc003fb0 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc00bdb0 con 0x7f2bd80d45e0 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.899+0000 7f2bd6800640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bbc004d40 0x7f2bbc0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.900+0000 7f2bcfe00640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc003fb0 0x7f2bbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.900+0000 7f2bd5e00640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 0x7f2bbc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.900+0000 7f2bd6800640 1 -- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 msgr2=0x7f2bbc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.900+0000 7f2bd6800640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd80d45e0 0x7f2bbc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.900+0000 7f2bd6800640 1 -- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc003fb0 msgr2=0x7f2bbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.900+0000 7f2bd6800640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bbc003fb0 0x7f2bbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.900+0000 7f2bd6800640 1 -- 172.21.15.19:0/1534890599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc003520 con 0x7f2bbc004d40 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.901+0000 7f2bd6800640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bbc004d40 0x7f2bbc0051b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2bd81651d0 tx=0x7f2bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3dee0110b3ba12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.901+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0009c10 con 0x7f2bbc004d40 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.901+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd0036070 con 0x7f2bbc004d40 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.901+0000 7f2bd5400640 1 -- 172.21.15.19:0/1534890599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0031830 con 0x7f2bbc004d40 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.902+0000 7f2bdd000640 1 -- 172.21.15.19:0/1534890599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bbc004d40 msgr2=0x7f2bbc0051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.902+0000 7f2bdd000640 1 --2- 172.21.15.19:0/1534890599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bbc004d40 0x7f2bbc0051b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2bd81651d0 tx=0x7f2bbc005830 comp rx=0 tx=0).stop 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.902+0000 7f2bd6800640 1 -- 172.21.15.19:0/1534890599 reap_dead start 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.902+0000 7f2bdd000640 1 -- 172.21.15.19:0/1534890599 shutdown_connections 2024-04-25T11:32:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.902+0000 7f2bdd000640 1 -- 172.21.15.19:0/1534890599 >> 172.21.15.19:0/1534890599 conn(0x7f2bd80d3ac0 msgr2=0x7f2bd80d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.902+0000 7f2bdd000640 1 -- 172.21.15.19:0/1534890599 shutdown_connections 2024-04-25T11:32:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.902+0000 7f2bdd000640 1 -- 172.21.15.19:0/1534890599 wait complete. 2024-04-25T11:32:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.905+0000 7f2bdd000640 1 Processor -- start 2024-04-25T11:32:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.905+0000 7f2bdd000640 1 -- start start 2024-04-25T11:32:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.906+0000 7f2bdd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bd814eef0 0x7f2bd81e9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.906+0000 7f2bdd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd81e9b60 0x7f2bd81e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.906+0000 7f2bdd000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bd81e55e0 0x7f2bd81e59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.906+0000 7f2bdd000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80d2040 con 0x7f2bd814eef0 2024-04-25T11:32:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.906+0000 7f2bdd000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80d1ec0 con 0x7f2bd81e55e0 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.906+0000 7f2bdd000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd806dc70 con 0x7f2bd81e9b60 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bd814eef0 0x7f2bd81e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bd814eef0 0x7f2bd81e9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51214/0 (socket says 172.21.15.19:51214) 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 -- 172.21.15.19:0/2175504093 learned_addr learned my addr 172.21.15.19:0/2175504093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bcfe00640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd81e9b60 0x7f2bd81e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd6800640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bd81e55e0 0x7f2bd81e59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 -- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd81e9b60 msgr2=0x7f2bd81e9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd81e9b60 0x7f2bd81e9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 -- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bd81e55e0 msgr2=0x7f2bd81e59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bd81e55e0 0x7f2bd81e59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd5e00640 1 -- 172.21.15.19:0/2175504093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd81e6b20 con 0x7f2bd814eef0 2024-04-25T11:32:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.907+0000 7f2bd6800640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bd81e55e0 0x7f2bd81e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:32:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.908+0000 7f2bd5e00640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bd814eef0 0x7f2bd81e9620 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001980 tx=0x7f2bc800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.908+0000 7f2bce000640 1 -- 172.21.15.19:0/2175504093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ee40 con 0x7f2bd814eef0 2024-04-25T11:32:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.908+0000 7f2bce000640 1 -- 172.21.15.19:0/2175504093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc8018070 con 0x7f2bd814eef0 2024-04-25T11:32:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.908+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd81ea6d0 con 0x7f2bd814eef0 2024-04-25T11:32:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.908+0000 7f2bce000640 1 -- 172.21.15.19:0/2175504093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8007520 con 0x7f2bd814eef0 2024-04-25T11:32:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.908+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81eac90 con 0x7f2bd814eef0 2024-04-25T11:32:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.909+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd8050740 con 0x7f2bd814eef0 2024-04-25T11:32:02.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.911+0000 7f2bce000640 1 -- 172.21.15.19:0/2175504093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2bc80057a0 con 0x7f2bd814eef0 2024-04-25T11:32:02.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.912+0000 7f2bce000640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bb40744b0 0x7f2bb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:02.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.912+0000 7f2bce000640 1 -- 172.21.15.19:0/2175504093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7f2bc8095200 con 0x7f2bd814eef0 2024-04-25T11:32:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.913+0000 7f2bcfe00640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bb40744b0 0x7f2bb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.914+0000 7f2bcfe00640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bb40744b0 0x7f2bb40769c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0007fd0 tx=0x7f2bc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:02.917+0000 7f2bce000640 1 -- 172.21.15.19:0/2175504093 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bc805fa00 con 0x7f2bd814eef0 2024-04-25T11:32:03.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.159+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bd80d1140 con 0x7f2bd814eef0 2024-04-25T11:32:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.161+0000 7f2bce000640 1 -- 172.21.15.19:0/2175504093 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7f2bc805f300 con 0x7f2bd814eef0 2024-04-25T11:32:03.150 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:32:03.150 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3770220424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3770220424},{"type":"v1","addr":"172.21.15.19:6833","nonce":3770220424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:32:01.321084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5452":{"gid":5452,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":18,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5452,"qdb_cluster":[5452]},"id":4}]} 2024-04-25T11:32:03.150 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.163+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bb40744b0 msgr2=0x7f2bb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.163+0000 7f2bdd000640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bb40744b0 0x7f2bb40769c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0007fd0 tx=0x7f2bc0009040 comp rx=0 tx=0).stop 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.163+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bd814eef0 msgr2=0x7f2bd81e9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.163+0000 7f2bdd000640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bd814eef0 0x7f2bd81e9620 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001980 tx=0x7f2bc800ebf0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 shutdown_connections 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bb40744b0 0x7f2bb40769c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bd81e55e0 0x7f2bd81e59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bd81e9b60 0x7f2bd81e9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 --2- 172.21.15.19:0/2175504093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bd814eef0 0x7f2bd81e9620 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 >> 172.21.15.19:0/2175504093 conn(0x7f2bd80d3ac0 msgr2=0x7f2bd80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:03.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 shutdown_connections 2024-04-25T11:32:03.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.164+0000 7f2bdd000640 1 -- 172.21.15.19:0/2175504093 wait complete. 2024-04-25T11:32:03.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:31:06.789413+0000', 'modified': '2024-04-25T11:32:01.321084+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5452': {'gid': 5452, 'name': 'd', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.169:6837/1885933205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1885933205}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1885933205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5452, 'qdb_cluster': [5452]} 2024-04-25T11:32:03.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:32:03.583 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:32:03.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:32:03.852 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:32:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.887+0000 7f725f200640 1 Processor -- start 2024-04-25T11:32:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.887+0000 7f725f200640 1 -- start start 2024-04-25T11:32:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.889+0000 7f725f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7260144370 0x7f72600cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.889+0000 7f725f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72600d1230 con 0x7f72600b4ea0 2024-04-25T11:32:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.889+0000 7f725f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72600d06b0 con 0x7f726006c5c0 2024-04-25T11:32:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.889+0000 7f725f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f72600d0990 con 0x7f7260144370 2024-04-25T11:32:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.890+0000 7f725e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7260144370 0x7f72600cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.890+0000 7f7255400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f72600b4ea0 0x7f726015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50228/0 (socket says 172.21.15.19:50228) 2024-04-25T11:32:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.890+0000 7f7255400640 1 -- 172.21.15.19:0/36266639 learned_addr learned my addr 172.21.15.19:0/36266639 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:32:03.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.890+0000 7f725e800640 1 -- 172.21.15.19:0/36266639 >> v1:172.21.15.169:6789/0 conn(0x7f726006c5c0 legacy=0x7f72600b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.890+0000 7f725e800640 1 -- 172.21.15.19:0/36266639 >> v1:172.21.15.19:6789/0 conn(0x7f72600b4ea0 legacy=0x7f726015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.890+0000 7f725e800640 1 -- 172.21.15.19:0/36266639 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72600d0c70 con 0x7f7260144370 2024-04-25T11:32:03.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.890+0000 7f725d400640 1 -- 172.21.15.19:0/36266639 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1541909287 0 0) 0x7f72600d1230 con 0x7f72600b4ea0 2024-04-25T11:32:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.892+0000 7f725e800640 1 --2- 172.21.15.19:0/36266639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f72600cfa70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f72580097a0 tx=0x7f725802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e684d9c59ddac22e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.892+0000 7f725d400640 1 -- 172.21.15.19:0/36266639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258037070 con 0x7f7260144370 2024-04-25T11:32:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.892+0000 7f725d400640 1 -- 172.21.15.19:0/36266639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7258032440 con 0x7f7260144370 2024-04-25T11:32:03.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.892+0000 7f725d400640 1 -- 172.21.15.19:0/36266639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258032810 con 0x7f7260144370 2024-04-25T11:32:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.893+0000 7f725f200640 1 -- 172.21.15.19:0/36266639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 msgr2=0x7f72600cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.893+0000 7f725f200640 1 --2- 172.21.15.19:0/36266639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f72600cfa70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f72580097a0 tx=0x7f725802a230 comp rx=0 tx=0).stop 2024-04-25T11:32:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.893+0000 7f725f200640 1 -- 172.21.15.19:0/36266639 shutdown_connections 2024-04-25T11:32:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.893+0000 7f725f200640 1 --2- 172.21.15.19:0/36266639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f72600cfa70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.893+0000 7f725f200640 1 -- 172.21.15.19:0/36266639 >> 172.21.15.19:0/36266639 conn(0x7f72600d1720 msgr2=0x7f72600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:03.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.893+0000 7f725f200640 1 -- 172.21.15.19:0/36266639 shutdown_connections 2024-04-25T11:32:03.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.893+0000 7f725f200640 1 -- 172.21.15.19:0/36266639 wait complete. 2024-04-25T11:32:03.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.896+0000 7f725f200640 1 Processor -- start 2024-04-25T11:32:03.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.896+0000 7f725f200640 1 -- start start 2024-04-25T11:32:03.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.897+0000 7f725f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726006c5c0 0x7f726006f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.897+0000 7f725f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f7260145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.897+0000 7f725f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72600cc150 0x7f72600cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.897+0000 7f725f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72600d0450 con 0x7f7260144370 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.897+0000 7f725f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72600d02d0 con 0x7f726006c5c0 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.897+0000 7f725f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7260147760 con 0x7f72600cc150 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f7260145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f7260145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44792/0 (socket says 172.21.15.19:44792) 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 -- 172.21.15.19:0/1313214412 learned_addr learned my addr 172.21.15.19:0/1313214412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f725de00640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726006c5c0 0x7f726006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f725e800640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72600cc150 0x7f72600cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 -- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72600cc150 msgr2=0x7f72600cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72600cc150 0x7f72600cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 -- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726006c5c0 msgr2=0x7f726006f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726006c5c0 0x7f726006f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.898+0000 7f7255400640 1 -- 172.21.15.19:0/1313214412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7260143bf0 con 0x7f7260144370 2024-04-25T11:32:03.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.899+0000 7f7255400640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f7260145750 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f724800c420 tx=0x7f724800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:03.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.899+0000 7f7256a00640 1 -- 172.21.15.19:0/1313214412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72480074e0 con 0x7f7260144370 2024-04-25T11:32:03.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.899+0000 7f7256a00640 1 -- 172.21.15.19:0/1313214412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72480047e0 con 0x7f7260144370 2024-04-25T11:32:03.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.899+0000 7f7256a00640 1 -- 172.21.15.19:0/1313214412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248004c10 con 0x7f7260144370 2024-04-25T11:32:03.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.899+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7260146560 con 0x7f7260144370 2024-04-25T11:32:03.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.899+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7260145230 con 0x7f7260144370 2024-04-25T11:32:03.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.902+0000 7f7256a00640 1 -- 172.21.15.19:0/1313214412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7248011070 con 0x7f7260144370 2024-04-25T11:32:03.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.903+0000 7f7256a00640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f723c0744b0 0x7f723c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.903+0000 7f7256a00640 1 -- 172.21.15.19:0/1313214412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7f7248094fa0 con 0x7f7260144370 2024-04-25T11:32:03.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.903+0000 7f725de00640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f723c0744b0 0x7f723c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.904+0000 7f726be45740 1 Processor -- start 2024-04-25T11:32:03.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.905+0000 7f725de00640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f723c0744b0 0x7f723c0769c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f72500012e0 tx=0x7f7250009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:03.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.905+0000 7f726be45740 1 -- start start 2024-04-25T11:32:03.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f726be45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8854950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac889c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac8492590 0x556ac8853d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f726be45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556ac87b9600 con 0x556ac84b3270 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556ac8756100 con 0x556ac8492590 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556ac87b55c0 con 0x556ac848a3b0 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f725de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8854950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f725de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8854950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44806/0 (socket says 172.21.15.19:44806) 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.906+0000 7f725de00640 1 -- 172.21.15.19:0/2804286915 learned_addr learned my addr 172.21.15.19:0/2804286915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f725e800640 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac8492590 0x556ac8853d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f7255400640 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac889c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f725de00640 1 -- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 msgr2=0x556ac889c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f725de00640 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac889c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f725de00640 1 -- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac8492590 msgr2=0x556ac8853d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f725de00640 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac8492590 0x556ac8853d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f725de00640 1 -- 172.21.15.19:0/2804286915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac84cb2d0 con 0x556ac84b3270 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.907+0000 7f725de00640 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8854950 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7250003ad0 tx=0x7f7250011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276f8e55fc15d29c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.908+0000 7f724ea00640 1 -- 172.21.15.19:0/2804286915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72500052e0 con 0x556ac84b3270 2024-04-25T11:32:03.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.908+0000 7f724ea00640 1 -- 172.21.15.19:0/2804286915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7250005480 con 0x556ac84b3270 2024-04-25T11:32:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.908+0000 7f724ea00640 1 -- 172.21.15.19:0/2804286915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250021450 con 0x556ac84b3270 2024-04-25T11:32:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.908+0000 7f726be45740 1 -- 172.21.15.19:0/2804286915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 msgr2=0x556ac8854950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.908+0000 7f726be45740 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8854950 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7250003ad0 tx=0x7f7250011000 comp rx=0 tx=0).stop 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.909+0000 7f726be45740 1 -- 172.21.15.19:0/2804286915 shutdown_connections 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.909+0000 7f726be45740 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac8492590 0x556ac8853d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.909+0000 7f726be45740 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac889c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.909+0000 7f726be45740 1 --2- 172.21.15.19:0/2804286915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8854950 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.909+0000 7f726be45740 1 -- 172.21.15.19:0/2804286915 >> 172.21.15.19:0/2804286915 conn(0x556ac84a73c0 msgr2=0x556ac8855870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.909+0000 7f726be45740 1 -- 172.21.15.19:0/2804286915 shutdown_connections 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.909+0000 7f726be45740 1 -- 172.21.15.19:0/2804286915 wait complete. 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.910+0000 7f726be45740 1 -- start start 2024-04-25T11:32:03.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.910+0000 7f726be45740 1 Processor -- start 2024-04-25T11:32:03.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac848a3b0 0x556ac88a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac8492590 0x556ac889e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f726be45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8891d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f726be45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556ac8740f70 con 0x556ac84b3270 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556ac88929c0 con 0x556ac848a3b0 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556ac889d070 con 0x556ac8492590 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f726be45740 10 client.0 _ll_register_callbacks cb 0x556ac87af480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8891d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8891d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44812/0 (socket says 172.21.15.19:44812) 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725e800640 1 -- 172.21.15.19:0/4050742462 learned_addr learned my addr 172.21.15.19:0/4050742462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725de00640 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac848a3b0 0x556ac88a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f7255400640 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac8492590 0x556ac889e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725e800640 1 -- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac8492590 msgr2=0x556ac889e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725e800640 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac8492590 0x556ac889e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725e800640 1 -- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac848a3b0 msgr2=0x556ac88a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.912+0000 7f725e800640 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac848a3b0 0x556ac88a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.913+0000 7f725e800640 1 -- 172.21.15.19:0/4050742462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac85956b0 con 0x556ac84b3270 2024-04-25T11:32:03.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.913+0000 7f725e800640 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8891d80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f72580098c0 tx=0x7f7258004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3a64d9be8039de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:03.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.913+0000 7f7243400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:32:03.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.913+0000 7f7243400640 1 -- 172.21.15.19:0/4050742462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258042030 con 0x556ac84b3270 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.913+0000 7f7243400640 1 -- 172.21.15.19:0/4050742462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7f7258003610 con 0x556ac84b3270 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.913+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7250005480 con 0x556ac84b3270 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.913+0000 7f7243400640 1 -- 172.21.15.19:0/4050742462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f725804e050 con 0x556ac84b3270 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 10 client.6654 fetch_fsmap learned FSMap version 65 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x556ac8892700 con 0x556ac84b3270 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f7243400640 1 -- 172.21.15.19:0/4050742462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7250005480 con 0x556ac84b3270 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f7243400640 1 -- 172.21.15.19:0/4050742462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f7258051020 con 0x556ac84b3270 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 10 client.6654 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 10 client.6654 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 20 client.6654 populate_metadata read hostname 'smithi019' 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8471e40 0x556ac889bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 4 client.6654 mds_command: new command op to 5452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 --> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ac87c7210 con 0x556ac8471e40 2024-04-25T11:32:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.914+0000 7f725de00640 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8471e40 0x556ac889bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.915+0000 7f725de00640 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8471e40 0x556ac889bcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720672ecc2ae3cb7 server_cookie=d722a1cf68d361a5 in_seq=0 out_seq=0 2024-04-25T11:32:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.915+0000 7f7243400640 10 client.6654 ms_handle_connect on v2:172.21.15.169:6836/1885933205 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f7243400640 1 -- 172.21.15.19:0/4050742462 <== mds.0 v2:172.21.15.169:6836/1885933205 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ac87c7210 con 0x556ac8471e40 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f7243400640 10 client.6654 handle_command_reply: tid=0 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 client.6654 shutdown 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 msgr2=0x556ac8891d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8891d80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f72580098c0 tx=0x7f7258004730 comp rx=0 tx=0).stop 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 shutdown_connections 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8471e40 0x556ac889bcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac84b3270 0x556ac8891d80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac8492590 0x556ac889e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 --2- 172.21.15.19:0/4050742462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac848a3b0 0x556ac88a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 >> 172.21.15.19:0/4050742462 conn(0x556ac84a73c0 msgr2=0x556ac889af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f7243400640 0 client.6654 ms_handle_reset on v2:172.21.15.169:6836/1885933205 2024-04-25T11:32:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 shutdown_connections 2024-04-25T11:32:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.917+0000 7f726be45740 1 -- 172.21.15.19:0/4050742462 wait complete. 2024-04-25T11:32:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.918+0000 7f726be45740 20 client.6654 trim_cache size 0 max 16384 2024-04-25T11:32:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.939+0000 7f726be45740 1 Processor -- start 2024-04-25T11:32:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.939+0000 7f726be45740 1 -- start start 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac884f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f726be45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac884f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f726be45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72600c9c70 con 0x556ac8492590 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72600c8cf0 con 0x556ac84b3270 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72600cd7a0 con 0x556ac848a3b0 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac884f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac884f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44826/0 (socket says 172.21.15.19:44826) 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 -- 172.21.15.19:0/1754394492 learned_addr learned my addr 172.21.15.19:0/1754394492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f725de00640 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac884f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f725e800640 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 -- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 msgr2=0x556ac884f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac884f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 -- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 msgr2=0x556ac88956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f7255400640 1 -- 172.21.15.19:0/1754394492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac88570a0 con 0x556ac8492590 2024-04-25T11:32:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.941+0000 7f725e800640 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:32:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.942+0000 7f7255400640 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac884f7b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f724805af10 tx=0x7f724805b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666bb5c9392a1867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.942+0000 7f724e000640 1 -- 172.21.15.19:0/1754394492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248092070 con 0x556ac8492590 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.942+0000 7f724e000640 1 -- 172.21.15.19:0/1754394492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724805eee0 con 0x556ac8492590 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.942+0000 7f724e000640 1 -- 172.21.15.19:0/1754394492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248067af0 con 0x556ac8492590 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 -- 172.21.15.19:0/1754394492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 msgr2=0x556ac884f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac884f7b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f724805af10 tx=0x7f724805b660 comp rx=0 tx=0).stop 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 -- 172.21.15.19:0/1754394492 shutdown_connections 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac884f7b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 --2- 172.21.15.19:0/1754394492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac884f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 -- 172.21.15.19:0/1754394492 >> 172.21.15.19:0/1754394492 conn(0x556ac84a73c0 msgr2=0x556ac8850080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 -- 172.21.15.19:0/1754394492 shutdown_connections 2024-04-25T11:32:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.943+0000 7f726be45740 1 -- 172.21.15.19:0/1754394492 wait complete. 2024-04-25T11:32:03.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.944+0000 7f726be45740 1 -- start start 2024-04-25T11:32:03.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.944+0000 7f726be45740 1 Processor -- start 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac88b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f726be45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac88b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f726be45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f726be45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556ac889cbc0 con 0x556ac8492590 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72600c86a0 con 0x556ac84b3270 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f726be45740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556ac88923b0 con 0x556ac848a3b0 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f726be45740 10 client.0 _ll_register_callbacks cb 0x7f7258009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f7255400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac88b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f7255400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac88b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44840/0 (socket says 172.21.15.19:44840) 2024-04-25T11:32:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f7255400640 1 -- 172.21.15.19:0/2352813989 learned_addr learned my addr 172.21.15.19:0/2352813989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.946+0000 7f725de00640 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac88b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f725e800640 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7255400640 1 -- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 msgr2=0x556ac88b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7255400640 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac88b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7255400640 1 -- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 msgr2=0x556ac88b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7255400640 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7255400640 1 -- 172.21.15.19:0/2352813989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac8896100 con 0x556ac8492590 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7255400640 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac88b27d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f724805af60 tx=0x7f7248093250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58275dd9a069100a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:32:03.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7243400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7243400640 1 -- 172.21.15.19:0/2352813989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248092040 con 0x556ac8492590 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f7243400640 1 -- 172.21.15.19:0/2352813989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5881+0+0 (secure 0 0 0) 0x7f724805e340 con 0x556ac8492590 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.947+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724805eee0 con 0x556ac8492590 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f7243400640 1 -- 172.21.15.19:0/2352813989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f7248063290 con 0x556ac8492590 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 10 client.6660 fetch_fsmap learned FSMap version 65 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x556ac88b3000 con 0x556ac8492590 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f7243400640 1 -- 172.21.15.19:0/2352813989 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724805eee0 con 0x556ac8492590 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f7243400640 1 -- 172.21.15.19:0/2352813989 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f724805a850 con 0x556ac8492590 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 10 client.6660 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 10 client.6660 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 20 client.6660 populate_metadata read hostname 'smithi019' 2024-04-25T11:32:03.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8891aa0 0x556ac88b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:32:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 4 client.6660 mds_command: new command op to 5452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:32:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.948+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 --> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] -- command(tid 0: {"prefix": "status"}) -- 0x556ac88a7ef0 con 0x556ac8891aa0 2024-04-25T11:32:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.949+0000 7f725de00640 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8891aa0 0x556ac88b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f7243400640 10 client.6660 ms_handle_connect on v2:172.21.15.169:6836/1885933205 2024-04-25T11:32:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.949+0000 7f725de00640 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8891aa0 0x556ac88b5e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7e426e96801e2c server_cookie=228c2286b22d751d in_seq=0 out_seq=0 2024-04-25T11:32:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f7243400640 1 -- 172.21.15.19:0/2352813989 <== mds.0 v2:172.21.15.169:6836/1885933205 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x556ac88a7ef0 con 0x556ac8891aa0 2024-04-25T11:32:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f7243400640 10 client.6660 handle_command_reply: tid=0 2024-04-25T11:32:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 client.6660 shutdown 2024-04-25T11:32:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 msgr2=0x556ac88b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac88b27d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f724805af60 tx=0x7f7248093250 comp rx=0 tx=0).stop 2024-04-25T11:32:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 shutdown_connections 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x556ac8891aa0 0x556ac88b5e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556ac84b3270 0x556ac88b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556ac8492590 0x556ac88b27d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f726be45740 1 --2- 172.21.15.19:0/2352813989 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556ac848a3b0 0x556ac88b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.950+0000 7f7243400640 0 client.6660 ms_handle_reset on v2:172.21.15.169:6836/1885933205 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 >> 172.21.15.19:0/2352813989 conn(0x556ac84a73c0 msgr2=0x556ac8850080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 shutdown_connections 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f726be45740 1 -- 172.21.15.19:0/2352813989 wait complete. 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f726be45740 20 client.6660 trim_cache size 0 max 16384 2024-04-25T11:32:03.942 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5452, 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.6447118459999999, 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 65, 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 65, 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 65, 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 62.833728880999999 2024-04-25T11:32:03.943 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:32:03.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f723c0744b0 msgr2=0x7f723c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f725f200640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f723c0744b0 0x7f723c0769c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f72500012e0 tx=0x7f7250009040 comp rx=0 tx=0).stop 2024-04-25T11:32:03.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 msgr2=0x7f7260145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:32:03.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.951+0000 7f725f200640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f7260145750 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f724800c420 tx=0x7f724800c8e0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 shutdown_connections 2024-04-25T11:32:03.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f723c0744b0 0x7f723c0769c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72600cc150 0x7f72600cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7260144370 0x7f7260145750 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 --2- 172.21.15.19:0/1313214412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f726006c5c0 0x7f726006f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:32:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 >> 172.21.15.19:0/1313214412 conn(0x7f72600d1720 msgr2=0x7f72600b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:32:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 shutdown_connections 2024-04-25T11:32:03.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:03.952+0000 7f725f200640 1 -- 172.21.15.19:0/1313214412 wait complete. 2024-04-25T11:32:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.017+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.018+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-04-25T11:32:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.018+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-04-25T11:32:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.019+0000 7f1c87400640 0 client.6576 ms_handle_remote_reset on v2:172.21.15.19:6832/1883095429 2024-04-25T11:32:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.019+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:32:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.019+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:32:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.019+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:32:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:05.019+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-25T11:32:20.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:20.026+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:20.028+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:20.028+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:20.028+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:32:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:20.028+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-25T11:32:33.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:32:33.981 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:32:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:35.030+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:35.031+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:32:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:35.031+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:32:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:35.031+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:32:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:35.032+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-25T11:32:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:50.048+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:32:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:50.049+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:32:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:50.049+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:32:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:50.049+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:32:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:32:50.049+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-25T11:33:04.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:33:04.120 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:33:05.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:05.064+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:05.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:05.065+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:33:05.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:05.065+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:33:05.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:05.065+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:33:05.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:05.066+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-25T11:33:20.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:20.081+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:20.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:20.083+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:33:20.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:20.083+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:33:20.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:20.083+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:33:20.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:20.083+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-25T11:33:34.250 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:33:34.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:33:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:35.094+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:35.096+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-04-25T11:33:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:35.096+0000 7f1ca6800640 1 -- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 msgr2=0x5556eefd8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:33:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:35.096+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:33:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:35.096+0000 7f1ca6800640 1 --2- 172.21.15.19:0/1156846417 >> [v2:172.21.15.19:6832/1883095429,v1:172.21.15.19:6833/1883095429] conn(0x5556eef7f240 0x5556eefd8560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-04-25T11:33:39.079 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-25T11:33:39.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:33:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.387+0000 7f5fc2600640 1 Processor -- start 2024-04-25T11:33:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.387+0000 7f5fc2600640 1 -- start start 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.389+0000 7f5fc2600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5fbc140e00 0x7f5fbc143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.389+0000 7f5fc2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc143a90 0x7f5fbc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.389+0000 7f5fc2600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fbc06db10 0x7f5fbc161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.389+0000 7f5fc2600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5fbc141b40 con 0x7f5fbc06db10 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.389+0000 7f5fc2600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5fbc143ea0 con 0x7f5fbc140e00 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.389+0000 7f5fc2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fbc06df20 con 0x7f5fbc143a90 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fbc06db10 0x7f5fbc161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5fbc06db10 0x7f5fbc161a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48082/0 (socket says 172.21.15.19:48082) 2024-04-25T11:33:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 -- 172.21.15.19:0/3208229151 learned_addr learned my addr 172.21.15.19:0/3208229151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1200640 1 --2- 172.21.15.19:0/3208229151 >> v2:172.21.15.169:3300/0 conn(0x7f5fbc140e00 0x7f5fbc143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fbbe00640 1 --2- 172.21.15.19:0/3208229151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc143a90 0x7f5fbc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 -- 172.21.15.19:0/3208229151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc143a90 msgr2=0x7f5fbc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 --2- 172.21.15.19:0/3208229151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc143a90 0x7f5fbc06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 -- 172.21.15.19:0/3208229151 >> v2:172.21.15.169:3300/0 conn(0x7f5fbc140e00 msgr2=0x7f5fbc143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 --2- 172.21.15.19:0/3208229151 >> v2:172.21.15.169:3300/0 conn(0x7f5fbc140e00 0x7f5fbc143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1c00640 1 -- 172.21.15.19:0/3208229151 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc141210 con 0x7f5fbc06db10 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fbbe00640 1 --2- 172.21.15.19:0/3208229151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc143a90 0x7f5fbc06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.390+0000 7f5fc1200640 1 --2- 172.21.15.19:0/3208229151 >> v2:172.21.15.169:3300/0 conn(0x7f5fbc140e00 0x7f5fbc143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.391+0000 7f5fc1c00640 1 --2- 172.21.15.19:0/3208229151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc06db10 0x7f5fbc161a80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc072990 tx=0x7f5fb402a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efe2ba305cda935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.392+0000 7f5fbb400640 1 -- 172.21.15.19:0/3208229151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4036070 con 0x7f5fbc06db10 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.392+0000 7f5fbb400640 1 -- 172.21.15.19:0/3208229151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb4031600 con 0x7f5fbc06db10 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.392+0000 7f5fbb400640 1 -- 172.21.15.19:0/3208229151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40319d0 con 0x7f5fbc06db10 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.392+0000 7f5fc2600640 1 -- 172.21.15.19:0/3208229151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc06db10 msgr2=0x7f5fbc161a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.392+0000 7f5fc2600640 1 --2- 172.21.15.19:0/3208229151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc06db10 0x7f5fbc161a80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc072990 tx=0x7f5fb402a690 comp rx=0 tx=0).stop 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.393+0000 7f5fc2600640 1 -- 172.21.15.19:0/3208229151 shutdown_connections 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.393+0000 7f5fc2600640 1 --2- 172.21.15.19:0/3208229151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc06db10 0x7f5fbc161a80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.393+0000 7f5fc2600640 1 --2- 172.21.15.19:0/3208229151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc143a90 0x7f5fbc06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.393+0000 7f5fc2600640 1 --2- 172.21.15.19:0/3208229151 >> v2:172.21.15.169:3300/0 conn(0x7f5fbc140e00 0x7f5fbc143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.393+0000 7f5fc2600640 1 -- 172.21.15.19:0/3208229151 >> 172.21.15.19:0/3208229151 conn(0x7f5fbc072ee0 msgr2=0x7f5fbc0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.393+0000 7f5fc2600640 1 -- 172.21.15.19:0/3208229151 shutdown_connections 2024-04-25T11:33:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.393+0000 7f5fc2600640 1 -- 172.21.15.19:0/3208229151 wait complete. 2024-04-25T11:33:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.395+0000 7f5fc2600640 1 Processor -- start 2024-04-25T11:33:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.395+0000 7f5fc2600640 1 -- start start 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fbc06db10 0x7f5fbc1fa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc140e00 0x7f5fbc157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc143a90 0x7f5fbc13b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc1443d0 con 0x7f5fbc143a90 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc1440f0 con 0x7f5fbc06db10 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5fbc1417d0 con 0x7f5fbc140e00 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc143a90 0x7f5fbc13b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc143a90 0x7f5fbc13b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48098/0 (socket says 172.21.15.19:48098) 2024-04-25T11:33:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.397+0000 7f5fc1c00640 1 -- 172.21.15.19:0/535988393 learned_addr learned my addr 172.21.15.19:0/535988393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:39.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fbbe00640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc140e00 0x7f5fbc157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:39.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fc1200640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fbc06db10 0x7f5fbc1fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fc1c00640 1 -- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc140e00 msgr2=0x7f5fbc157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fc1c00640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc140e00 0x7f5fbc157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fc1c00640 1 -- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fbc06db10 msgr2=0x7f5fbc1fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fc1c00640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fbc06db10 0x7f5fbc1fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fc1c00640 1 -- 172.21.15.19:0/535988393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc14bf50 con 0x7f5fbc143a90 2024-04-25T11:33:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fbbe00640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc140e00 0x7f5fbc157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:33:39.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.398+0000 7f5fc1c00640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc143a90 0x7f5fbc13b060 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4034040 tx=0x7f5fb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.399+0000 7f5fb8c00640 1 -- 172.21.15.19:0/535988393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4036050 con 0x7f5fbc143a90 2024-04-25T11:33:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.399+0000 7f5fb8c00640 1 -- 172.21.15.19:0/535988393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb4042070 con 0x7f5fbc143a90 2024-04-25T11:33:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.399+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fbc14c480 con 0x7f5fbc143a90 2024-04-25T11:33:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.399+0000 7f5fb8c00640 1 -- 172.21.15.19:0/535988393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4009510 con 0x7f5fbc143a90 2024-04-25T11:33:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.399+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fbc14ca40 con 0x7f5fbc143a90 2024-04-25T11:33:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.400+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fbc050740 con 0x7f5fbc143a90 2024-04-25T11:33:39.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.401+0000 7f5fb8c00640 1 -- 172.21.15.19:0/535988393 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5fb4007bd0 con 0x7f5fbc143a90 2024-04-25T11:33:39.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.403+0000 7f5fb8c00640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f9c0744b0 0x7f5f9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:39.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.403+0000 7f5fb8c00640 1 -- 172.21.15.19:0/535988393 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f5fb40b6370 con 0x7f5fbc143a90 2024-04-25T11:33:39.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.404+0000 7f5fc1200640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f9c0744b0 0x7f5f9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:39.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.405+0000 7f5fc1200640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f9c0744b0 0x7f5f9c0769c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000a980 tx=0x7f5fb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.409+0000 7f5fb8c00640 1 -- 172.21.15.19:0/535988393 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fb4080970 con 0x7f5fbc143a90 2024-04-25T11:33:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.654+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5fbc14c210 con 0x7f5fbc143a90 2024-04-25T11:33:39.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.655+0000 7f5fb8c00640 1 -- 172.21.15.19:0/535988393 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7f5fb4080270 con 0x7f5fbc143a90 2024-04-25T11:33:39.643 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:33:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f9c0744b0 msgr2=0x7f5f9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f9c0744b0 0x7f5f9c0769c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000a980 tx=0x7f5fb0008040 comp rx=0 tx=0).stop 2024-04-25T11:33:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc143a90 msgr2=0x7f5fbc13b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc143a90 0x7f5fbc13b060 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4034040 tx=0x7f5fb4004a40 comp rx=0 tx=0).stop 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 shutdown_connections 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f9c0744b0 0x7f5f9c0769c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fbc143a90 0x7f5fbc13b060 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fbc140e00 0x7f5fbc157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.657+0000 7f5fc2600640 1 --2- 172.21.15.19:0/535988393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5fbc06db10 0x7f5fbc1fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.658+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 >> 172.21.15.19:0/535988393 conn(0x7f5fbc072ee0 msgr2=0x7f5fbc0b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.658+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 shutdown_connections 2024-04-25T11:33:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:39.658+0000 7f5fc2600640 1 -- 172.21.15.19:0/535988393 wait complete. 2024-04-25T11:33:39.673 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:33:39.674 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:33:39.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:39.674 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:33:39.730 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:33:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:33:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:33:39.730 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:33:39.731 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42254sec preferred_lft 42254sec 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:33:39.732 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:33:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:33:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:33:39.733 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:33:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:33:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:33:39.733 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:33:39.734 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:33:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:33:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:33:39.734 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:33:39.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:39.736 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:33:39.820 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:33:39.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:39.825 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:33:39.882 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:33:39.883 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:33:39.883 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:33:39.883 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:33:39.883 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:33:39.883 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:33:39.883 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:33:39.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:33:39.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:33:39.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:33:39.884 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:33:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:33:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:33:40.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:40.104 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:33:40.156 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:33:40.157 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:33:40.158 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:33:40.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:40.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:33:40.242 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:33:40.243 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:33:40.302 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:33:40.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:33:40.302 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:33:40.357 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:33:40.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:40.357 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:33:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:33:40.573 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:33:40.619 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:33:40.619 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:33:40.619 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:33:40.619 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:33:40.619 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:33:40.621 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:33:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:33:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:33:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:33:40.621 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:33:40.625 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:33:40.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:40.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:33:40.697 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:33:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:33:40.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:33:40.819 INFO:teuthology.orchestra.run.smithi019.stdout:client5297 2024-04-25T11:33:40.819 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:33:40.819 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:40.873 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-04-25T11:33:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: File: a/ 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 2ah/42d Inode: 1099511627776 Links: 4 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1316/ ubuntu) 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:31:16.555981233 +0000 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:31:16.567981111 +0000 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:31:16.567981111 +0000 2024-04-25T11:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:31:16.555981233 +0000 2024-04-25T11:33:41.009 INFO:teuthology.orchestra.run.smithi019.stdout:a/ 2024-04-25T11:33:41.009 INFO:teuthology.orchestra.run.smithi019.stdout:a/c 2024-04-25T11:33:41.009 INFO:teuthology.orchestra.run.smithi019.stdout:a/c/d 2024-04-25T11:33:41.009 INFO:teuthology.orchestra.run.smithi019.stdout:a/b 2024-04-25T11:33:41.009 INFO:teuthology.orchestra.run.smithi019.stdout:a/b/d 2024-04-25T11:33:41.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:33:41.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-04-25T11:33:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.321+0000 7f42eee00640 1 Processor -- start 2024-04-25T11:33:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.321+0000 7f42eee00640 1 -- start start 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42eee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42e8143660 0x7f42e8070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42eee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e806d5d0 0x7f42e806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42eee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42e80d04f0 0x7f42e80d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42eee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f42e806df20 con 0x7f42e8143660 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42eee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f42e8070c70 con 0x7f42e80d04f0 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42eee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42e80b4920 con 0x7f42e806d5d0 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42e7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42e8143660 0x7f42e8070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42e7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42e8143660 0x7f42e8070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48110/0 (socket says 172.21.15.19:48110) 2024-04-25T11:33:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.324+0000 7f42e7e00640 1 -- 172.21.15.19:0/2418073713 learned_addr learned my addr 172.21.15.19:0/2418073713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.325+0000 7f42e7e00640 1 -- 172.21.15.19:0/2418073713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e806d5d0 msgr2=0x7f42e806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.325+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2418073713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e806d5d0 0x7f42e806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.325+0000 7f42e7e00640 1 -- 172.21.15.19:0/2418073713 >> v2:172.21.15.169:3300/0 conn(0x7f42e80d04f0 msgr2=0x7f42e80d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.325+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2418073713 >> v2:172.21.15.169:3300/0 conn(0x7f42e80d04f0 0x7f42e80d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.325+0000 7f42e7e00640 1 -- 172.21.15.19:0/2418073713 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e80b4c00 con 0x7f42e8143660 2024-04-25T11:33:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.326+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2418073713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e8070290 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f42e80cf100 tx=0x7f42d802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208c8f73d60a3692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.326+0000 7f42e7400640 1 -- 172.21.15.19:0/2418073713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8037070 con 0x7f42e8143660 2024-04-25T11:33:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.326+0000 7f42e7400640 1 -- 172.21.15.19:0/2418073713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d8032440 con 0x7f42e8143660 2024-04-25T11:33:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.326+0000 7f42e7400640 1 -- 172.21.15.19:0/2418073713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8032810 con 0x7f42e8143660 2024-04-25T11:33:41.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.327+0000 7f42eee00640 1 -- 172.21.15.19:0/2418073713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 msgr2=0x7f42e8070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.327+0000 7f42eee00640 1 --2- 172.21.15.19:0/2418073713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e8070290 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f42e80cf100 tx=0x7f42d802a200 comp rx=0 tx=0).stop 2024-04-25T11:33:41.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.328+0000 7f42eee00640 1 -- 172.21.15.19:0/2418073713 shutdown_connections 2024-04-25T11:33:41.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.328+0000 7f42eee00640 1 --2- 172.21.15.19:0/2418073713 >> v2:172.21.15.169:3300/0 conn(0x7f42e80d04f0 0x7f42e80d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.328+0000 7f42eee00640 1 --2- 172.21.15.19:0/2418073713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e806d5d0 0x7f42e806d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.328+0000 7f42eee00640 1 --2- 172.21.15.19:0/2418073713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e8070290 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.328+0000 7f42eee00640 1 -- 172.21.15.19:0/2418073713 >> 172.21.15.19:0/2418073713 conn(0x7f42e8073120 msgr2=0x7f42e8073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.328+0000 7f42eee00640 1 -- 172.21.15.19:0/2418073713 shutdown_connections 2024-04-25T11:33:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.328+0000 7f42eee00640 1 -- 172.21.15.19:0/2418073713 wait complete. 2024-04-25T11:33:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.331+0000 7f42eee00640 1 Processor -- start 2024-04-25T11:33:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.331+0000 7f42eee00640 1 -- start start 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.333+0000 7f42eee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e806d5d0 0x7f42e815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.333+0000 7f42eee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d04f0 0x7f42e815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.333+0000 7f42eee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e813c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.333+0000 7f42eee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42e80ce0e0 con 0x7f42e8143660 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.333+0000 7f42eee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42e80b51c0 con 0x7f42e806d5d0 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.333+0000 7f42eee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42e80ce260 con 0x7f42e80d04f0 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e813c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e813c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48112/0 (socket says 172.21.15.19:48112) 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 -- 172.21.15.19:0/1532160972 learned_addr learned my addr 172.21.15.19:0/1532160972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42e7e00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e806d5d0 0x7f42e815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42dfe00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d04f0 0x7f42e815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 -- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d04f0 msgr2=0x7f42e815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d04f0 0x7f42e815a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 -- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e806d5d0 msgr2=0x7f42e815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e806d5d0 0x7f42e815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 -- 172.21.15.19:0/1532160972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81584f0 con 0x7f42e8143660 2024-04-25T11:33:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.334+0000 7f42ece00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e813c080 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f42e000ba10 tx=0x7f42e000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.335+0000 7f42e4c00640 1 -- 172.21.15.19:0/1532160972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0019070 con 0x7f42e8143660 2024-04-25T11:33:41.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.335+0000 7f42e4c00640 1 -- 172.21.15.19:0/1532160972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e000cb60 con 0x7f42e8143660 2024-04-25T11:33:41.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.335+0000 7f42e4c00640 1 -- 172.21.15.19:0/1532160972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0014440 con 0x7f42e8143660 2024-04-25T11:33:41.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.335+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81587d0 con 0x7f42e8143660 2024-04-25T11:33:41.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.335+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e8158d90 con 0x7f42e8143660 2024-04-25T11:33:41.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.338+0000 7f42e4c00640 1 -- 172.21.15.19:0/1532160972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f42e001c070 con 0x7f42e8143660 2024-04-25T11:33:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.339+0000 7f42e4c00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42c40744b0 0x7f42c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.339+0000 7f42e4c00640 1 -- 172.21.15.19:0/1532160972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f42e0094f60 con 0x7f42e8143660 2024-04-25T11:33:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.339+0000 7f42f5e71740 1 Processor -- start 2024-04-25T11:33:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.340+0000 7f42f5e71740 1 -- start start 2024-04-25T11:33:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.340+0000 7f42e7e00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42c40744b0 0x7f42c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.341+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 0x55e982e6f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.341+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e6bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.341+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e45250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.341+0000 7f42f5e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e982d55030 con 0x55e982a794d0 2024-04-25T11:33:41.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.341+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e982d707f0 con 0x55e982a816b0 2024-04-25T11:33:41.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.341+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e982d2b7e0 con 0x55e982aa2390 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.341+0000 7f42e7e00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42c40744b0 0x7f42c40769c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f42d8034000 tx=0x7f42d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e6bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e6bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48122/0 (socket says 172.21.15.19:48122) 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 -- 172.21.15.19:0/2660717845 learned_addr learned my addr 172.21.15.19:0/2660717845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 0x55e982e6f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42ece00640 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e45250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 -- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 msgr2=0x55e982e6f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 0x55e982e6f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 -- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 msgr2=0x55e982e45250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e45250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.342+0000 7f42dfe00640 1 -- 172.21.15.19:0/2660717845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e982d37050 con 0x55e982a794d0 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.343+0000 7f42dfe00640 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e6bf90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f42d000b750 tx=0x7f42d000bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af2125ae95fa5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:41.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.343+0000 7f42dcc00640 1 -- 172.21.15.19:0/2660717845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d000be60 con 0x55e982a794d0 2024-04-25T11:33:41.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.343+0000 7f42dcc00640 1 -- 172.21.15.19:0/2660717845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d0019070 con 0x55e982a794d0 2024-04-25T11:33:41.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.343+0000 7f42dcc00640 1 -- 172.21.15.19:0/2660717845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d000db30 con 0x55e982a794d0 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 -- 172.21.15.19:0/2660717845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 msgr2=0x55e982e6bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e6bf90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f42d000b750 tx=0x7f42d000bc10 comp rx=0 tx=0).stop 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 -- 172.21.15.19:0/2660717845 shutdown_connections 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e45250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e6bf90 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2660717845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 0x55e982e6f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 -- 172.21.15.19:0/2660717845 >> 172.21.15.19:0/2660717845 conn(0x55e982a964e0 msgr2=0x55e982e449e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 -- 172.21.15.19:0/2660717845 shutdown_connections 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.344+0000 7f42f5e71740 1 -- 172.21.15.19:0/2660717845 wait complete. 2024-04-25T11:33:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.345+0000 7f42f5e71740 1 -- start start 2024-04-25T11:33:41.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.345+0000 7f42f5e71740 1 Processor -- start 2024-04-25T11:33:41.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a794d0 0x55e982e63560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e63aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982aa2390 0x55e982e9c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42f5e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e982dad200 con 0x55e982aa2390 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e982da7ee0 con 0x55e982a794d0 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e982db4e70 con 0x55e982a816b0 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42f5e71740 10 client.0 _ll_register_callbacks cb 0x55e982d9e270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982aa2390 0x55e982e9c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982aa2390 0x55e982e9c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48136/0 (socket says 172.21.15.19:48136) 2024-04-25T11:33:41.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.347+0000 7f42ece00640 1 -- 172.21.15.19:0/2592484461 learned_addr learned my addr 172.21.15.19:0/2592484461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42dfe00640 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e63aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a794d0 0x55e982e63560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42ece00640 1 -- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 msgr2=0x55e982e63aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42ece00640 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e63aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42ece00640 1 -- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a794d0 msgr2=0x55e982e63560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42ece00640 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a794d0 0x55e982e63560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42ece00640 1 -- 172.21.15.19:0/2592484461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e982b847d0 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42ece00640 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982aa2390 0x55e982e9c180 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b9e0 tx=0x7f42e005f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d633519a3142e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42d5600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42d5600640 1 -- 172.21.15.19:0/2592484461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0063330 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42d5600640 1 -- 172.21.15.19:0/2592484461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f42e008dcd0 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d0019070 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42d5600640 1 -- 172.21.15.19:0/2592484461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f42e0067d50 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42f5e71740 10 client.6678 fetch_fsmap learned FSMap version 65 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.348+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55e982aba3f0 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42d5600640 1 -- 172.21.15.19:0/2592484461 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d0019070 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42d5600640 1 -- 172.21.15.19:0/2592484461 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f42e005a8f0 con 0x55e982aa2390 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42f5e71740 10 client.6678 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42f5e71740 10 client.6678 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-25T11:33:41.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42f5e71740 20 client.6678 populate_metadata read hostname 'smithi019' 2024-04-25T11:33:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982a60f60 0x55e982e9c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42f5e71740 4 client.6678 mds_command: new command op to 5452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:33:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.349+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 --> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e982db67e0 con 0x55e982a60f60 2024-04-25T11:33:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.350+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982a60f60 0x55e982e9c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.351+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982a60f60 0x55e982e9c6c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a90f1541965c5e5 server_cookie=22ce7c2eb93c57f1 in_seq=0 out_seq=0 2024-04-25T11:33:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.351+0000 7f42d5600640 10 client.6678 ms_handle_connect on v2:172.21.15.169:6836/1885933205 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.353+0000 7f42d5600640 1 -- 172.21.15.19:0/2592484461 <== mds.0 v2:172.21.15.169:6836/1885933205 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e982db67e0 con 0x55e982a60f60 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.353+0000 7f42d5600640 10 client.6678 handle_command_reply: tid=0 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 client.6678 shutdown 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982aa2390 msgr2=0x55e982e9c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982aa2390 0x55e982e9c180 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f42e000b9e0 tx=0x7f42e005f5a0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 shutdown_connections 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982a60f60 0x55e982e9c6c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982aa2390 0x55e982e9c180 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e63aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2592484461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a794d0 0x55e982e63560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42d5600640 0 client.6678 ms_handle_reset on v2:172.21.15.169:6836/1885933205 2024-04-25T11:33:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 >> 172.21.15.19:0/2592484461 conn(0x55e982a964e0 msgr2=0x55e982e43d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 shutdown_connections 2024-04-25T11:33:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 1 -- 172.21.15.19:0/2592484461 wait complete. 2024-04-25T11:33:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.354+0000 7f42f5e71740 20 client.6678 trim_cache size 0 max 16384 2024-04-25T11:33:41.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.372+0000 7f42f5e71740 1 Processor -- start 2024-04-25T11:33:41.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.372+0000 7f42f5e71740 1 -- start start 2024-04-25T11:33:41.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.373+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e8d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.373+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e8db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.373+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982aa2390 0x55e982e69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.373+0000 7f42f5e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42e82031a0 con 0x55e982a794d0 2024-04-25T11:33:41.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.373+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8202180 con 0x55e982aa2390 2024-04-25T11:33:41.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.373+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42e82014e0 con 0x55e982a816b0 2024-04-25T11:33:41.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e8d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e8d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48142/0 (socket says 172.21.15.19:48142) 2024-04-25T11:33:41.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 -- 172.21.15.19:0/2854946362 learned_addr learned my addr 172.21.15.19:0/2854946362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:41.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42ece00640 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982aa2390 0x55e982e69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42dfe00640 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e8db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 -- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 msgr2=0x55e982e8db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e8db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 -- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982aa2390 msgr2=0x55e982e69f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982aa2390 0x55e982e69f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.374+0000 7f42e7e00640 1 -- 172.21.15.19:0/2854946362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e982a7a0f0 con 0x55e982a794d0 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.375+0000 7f42e7e00640 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e8d5f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f42d802a7f0 tx=0x7f42d802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7e387ba7679f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.375+0000 7f42d7e00640 1 -- 172.21.15.19:0/2854946362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8052070 con 0x55e982a794d0 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.375+0000 7f42d7e00640 1 -- 172.21.15.19:0/2854946362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d803e660 con 0x55e982a794d0 2024-04-25T11:33:41.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.375+0000 7f42d7e00640 1 -- 172.21.15.19:0/2854946362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8037040 con 0x55e982a794d0 2024-04-25T11:33:41.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 -- 172.21.15.19:0/2854946362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 msgr2=0x55e982e8d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e8d5f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f42d802a7f0 tx=0x7f42d802c000 comp rx=0 tx=0).stop 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 -- 172.21.15.19:0/2854946362 shutdown_connections 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982aa2390 0x55e982e69f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982a816b0 0x55e982e8db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 --2- 172.21.15.19:0/2854946362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e8d5f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 -- 172.21.15.19:0/2854946362 >> 172.21.15.19:0/2854946362 conn(0x55e982a964e0 msgr2=0x55e982e3acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 -- 172.21.15.19:0/2854946362 shutdown_connections 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.376+0000 7f42f5e71740 1 -- 172.21.15.19:0/2854946362 wait complete. 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.377+0000 7f42f5e71740 1 -- start start 2024-04-25T11:33:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.377+0000 7f42f5e71740 1 Processor -- start 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e64440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e64980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42f5e71740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 0x55e982f097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42f5e71740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e982db5fd0 con 0x55e982a794d0 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8201b30 con 0x55e982a816b0 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42f5e71740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e982e62710 con 0x55e982aa2390 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42f5e71740 10 client.0 _ll_register_callbacks cb 0x7f42e005b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e64440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e64440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48158/0 (socket says 172.21.15.19:48158) 2024-04-25T11:33:41.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.379+0000 7f42e7e00640 1 -- 172.21.15.19:0/900462901 learned_addr learned my addr 172.21.15.19:0/900462901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42dfe00640 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e64980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42e7e00640 1 -- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 msgr2=0x55e982f097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42e7e00640 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 0x55e982f097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42e7e00640 1 -- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 msgr2=0x55e982e64980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42e7e00640 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e64980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42e7e00640 1 -- 172.21.15.19:0/900462901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e982d70a30 con 0x55e982a794d0 2024-04-25T11:33:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42e7e00640 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e64440 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f42d8032ac0 tx=0x7f42d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead44d9d05198965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42d5600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:33:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42d5600640 1 -- 172.21.15.19:0/900462901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8052030 con 0x55e982a794d0 2024-04-25T11:33:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42d5600640 1 -- 172.21.15.19:0/900462901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f42d803e8d0 con 0x55e982a794d0 2024-04-25T11:33:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.380+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d803e660 con 0x55e982a794d0 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42d5600640 1 -- 172.21.15.19:0/900462901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f42d803ead0 con 0x55e982a794d0 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 10 client.6684 fetch_fsmap learned FSMap version 65 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55e982e8e030 con 0x55e982a794d0 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42d5600640 1 -- 172.21.15.19:0/900462901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d803e660 con 0x55e982a794d0 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42d5600640 1 -- 172.21.15.19:0/900462901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f42d803ec70 con 0x55e982a794d0 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 10 client.6684 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 10 client.6684 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 20 client.6684 populate_metadata read hostname 'smithi019' 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982f09cf0 0x55e982e69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 4 client.6684 mds_command: new command op to 5452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.381+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 --> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e982e407a0 con 0x55e982f09cf0 2024-04-25T11:33:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.382+0000 7f42dfe00640 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982f09cf0 0x55e982e69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.383+0000 7f42dfe00640 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982f09cf0 0x55e982e69f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6944dd212b1182c3 server_cookie=6cb87d99d59bd448 in_seq=0 out_seq=0 2024-04-25T11:33:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.383+0000 7f42d5600640 10 client.6684 ms_handle_connect on v2:172.21.15.169:6836/1885933205 2024-04-25T11:33:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42d5600640 1 -- 172.21.15.19:0/900462901 <== mds.0 v2:172.21.15.169:6836/1885933205 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e982e407a0 con 0x55e982f09cf0 2024-04-25T11:33:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42d5600640 10 client.6684 handle_command_reply: tid=0 2024-04-25T11:33:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 client.6684 shutdown 2024-04-25T11:33:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 msgr2=0x55e982e64440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e64440 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f42d8032ac0 tx=0x7f42d8004730 comp rx=0 tx=0).stop 2024-04-25T11:33:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 shutdown_connections 2024-04-25T11:33:41.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:6836/1885933205,v1:172.21.15.169:6837/1885933205] conn(0x55e982f09cf0 0x55e982e69f80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e982a816b0 0x55e982e64980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e982aa2390 0x55e982f097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42d5600640 0 client.6684 ms_handle_reset on v2:172.21.15.169:6836/1885933205 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 --2- 172.21.15.19:0/900462901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e982a794d0 0x55e982e64440 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 >> 172.21.15.19:0/900462901 conn(0x55e982a964e0 msgr2=0x55e982e3acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 shutdown_connections 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.406+0000 7f42f5e71740 1 -- 172.21.15.19:0/900462901 wait complete. 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.407+0000 7f42f5e71740 20 client.6684 trim_cache size 0 max 16384 2024-04-25T11:33:41.396 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:33:41.397 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-04-25T11:33:41.397 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T11:33:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:33:41.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.407+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42c40744b0 msgr2=0x7f42c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42c40744b0 0x7f42c40769c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f42d8034000 tx=0x7f42d802e000 comp rx=0 tx=0).stop 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 msgr2=0x7f42e813c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e813c080 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f42e000ba10 tx=0x7f42e000bed0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 shutdown_connections 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42c40744b0 0x7f42c40769c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e8143660 0x7f42e813c080 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d04f0 0x7f42e815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 --2- 172.21.15.19:0/1532160972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e806d5d0 0x7f42e815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 >> 172.21.15.19:0/1532160972 conn(0x7f42e8073120 msgr2=0x7f42e806f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 shutdown_connections 2024-04-25T11:33:41.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:41.408+0000 7f42eee00640 1 -- 172.21.15.19:0/1532160972 wait complete. 2024-04-25T11:33:41.429 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:33:41.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:33:41.532 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:33:41.646 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:33:41.646 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:33:41.647 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:33:41.647 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:41.707 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:33:41.707 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:41.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:33:42.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.069+0000 7f73bac00640 1 Processor -- start 2024-04-25T11:33:42.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.070+0000 7f73bac00640 1 -- start start 2024-04-25T11:33:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.072+0000 7f73bac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73b40cf700 0x7f73b406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.072+0000 7f73bac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b4071180 con 0x7f73b4168720 2024-04-25T11:33:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.072+0000 7f73bac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b40700e0 con 0x7f73b40cf2f0 2024-04-25T11:33:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.072+0000 7f73bac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f73b4070cf0 con 0x7f73b40cf700 2024-04-25T11:33:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.073+0000 7f73ba200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f73b40cf700 0x7f73b406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.073+0000 7f73b8e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f73b4168720 0x7f73b406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36294/0 (socket says 172.21.15.19:36294) 2024-04-25T11:33:42.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.073+0000 7f73b8e00640 1 -- 172.21.15.19:0/523357761 learned_addr learned my addr 172.21.15.19:0/523357761 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.073+0000 7f73ba200640 1 -- 172.21.15.19:0/523357761 >> v1:172.21.15.169:6789/0 conn(0x7f73b40cf2f0 legacy=0x7f73b4168010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.073+0000 7f73ba200640 1 -- 172.21.15.19:0/523357761 >> v1:172.21.15.19:6789/0 conn(0x7f73b4168720 legacy=0x7f73b406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.073+0000 7f73ba200640 1 -- 172.21.15.19:0/523357761 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b40703c0 con 0x7f73b40cf700 2024-04-25T11:33:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.073+0000 7f73b3e00640 1 -- 172.21.15.19:0/523357761 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3089136137 0 0) 0x7f73b4071180 con 0x7f73b4168720 2024-04-25T11:33:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.074+0000 7f73ba200640 1 --2- 172.21.15.19:0/523357761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b406ccd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0097a0 tx=0x7f73ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797e165f23e71af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.074+0000 7f73b3e00640 1 -- 172.21.15.19:0/523357761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac037070 con 0x7f73b40cf700 2024-04-25T11:33:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.075+0000 7f73b3e00640 1 -- 172.21.15.19:0/523357761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac032440 con 0x7f73b40cf700 2024-04-25T11:33:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.075+0000 7f73b3e00640 1 -- 172.21.15.19:0/523357761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac032810 con 0x7f73b40cf700 2024-04-25T11:33:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.075+0000 7f73bac00640 1 -- 172.21.15.19:0/523357761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 msgr2=0x7f73b406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.075+0000 7f73bac00640 1 --2- 172.21.15.19:0/523357761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b406ccd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0097a0 tx=0x7f73ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.076+0000 7f73bac00640 1 -- 172.21.15.19:0/523357761 shutdown_connections 2024-04-25T11:33:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.076+0000 7f73bac00640 1 --2- 172.21.15.19:0/523357761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b406ccd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.076+0000 7f73bac00640 1 -- 172.21.15.19:0/523357761 >> 172.21.15.19:0/523357761 conn(0x7f73b40d1720 msgr2=0x7f73b40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.076+0000 7f73bac00640 1 -- 172.21.15.19:0/523357761 shutdown_connections 2024-04-25T11:33:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.076+0000 7f73bac00640 1 -- 172.21.15.19:0/523357761 wait complete. 2024-04-25T11:33:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.078+0000 7f73bac00640 1 Processor -- start 2024-04-25T11:33:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.078+0000 7f73bac00640 1 -- start start 2024-04-25T11:33:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73bac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b40cf2f0 0x7f73b4160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73bac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73bac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b4162470 0x7f73b414c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73bac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4168d00 con 0x7f73b40cf700 2024-04-25T11:33:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73bac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4070980 con 0x7f73b40cf2f0 2024-04-25T11:33:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73bac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f73b4168e80 con 0x7f73b4162470 2024-04-25T11:33:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73b8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73b8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b415dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48184/0 (socket says 172.21.15.19:48184) 2024-04-25T11:33:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.080+0000 7f73b8e00640 1 -- 172.21.15.19:0/3001189462 learned_addr learned my addr 172.21.15.19:0/3001189462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b9800640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b40cf2f0 0x7f73b4160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73ba200640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b4162470 0x7f73b414c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b8e00640 1 -- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b4162470 msgr2=0x7f73b414c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b8e00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b4162470 0x7f73b414c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b8e00640 1 -- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b40cf2f0 msgr2=0x7f73b4160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b8e00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b40cf2f0 0x7f73b4160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b8e00640 1 -- 172.21.15.19:0/3001189462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b41613d0 con 0x7f73b40cf700 2024-04-25T11:33:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73ba200640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b4162470 0x7f73b414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b9800640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b40cf2f0 0x7f73b4160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b8e00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b415dcc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f73a000c460 tx=0x7f73a000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.081+0000 7f73b1600640 1 -- 172.21.15.19:0/3001189462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0018070 con 0x7f73b40cf700 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.082+0000 7f73b1600640 1 -- 172.21.15.19:0/3001189462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0007500 con 0x7f73b40cf700 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.082+0000 7f73b1600640 1 -- 172.21.15.19:0/3001189462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004a20 con 0x7f73b40cf700 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.082+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b41616b0 con 0x7f73b40cf700 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.082+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b4161c70 con 0x7f73b40cf700 2024-04-25T11:33:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.083+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b4050740 con 0x7f73b40cf700 2024-04-25T11:33:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.084+0000 7f73b1600640 1 -- 172.21.15.19:0/3001189462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f73a0004bc0 con 0x7f73b40cf700 2024-04-25T11:33:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.085+0000 7f73b1600640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73940744b0 0x7f73940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.085+0000 7f73b1600640 1 -- 172.21.15.19:0/3001189462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f73a0094630 con 0x7f73b40cf700 2024-04-25T11:33:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.086+0000 7f73b9800640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73940744b0 0x7f73940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.087+0000 7f73b9800640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73940744b0 0x7f73940769c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f73b40d11e0 tx=0x7f73a80073a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:42.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.090+0000 7f73b1600640 1 -- 172.21.15.19:0/3001189462 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73a005ecc0 con 0x7f73b40cf700 2024-04-25T11:33:42.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.294+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73b4161160 con 0x7f73b40cf700 2024-04-25T11:33:42.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.297+0000 7f73b1600640 1 -- 172.21.15.19:0/3001189462 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17018 (secure 0 0 0) 0x7f73a005e5c0 con 0x7f73b40cf700 2024-04-25T11:33:42.287 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:42.287 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":67,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:32:07.391254+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.36","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6833/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6833/751039234":"2024-04-26T11:31:25.310192+0000","172.21.15.19:6832/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6832/751039234":"2024-04-26T11:31:25.310192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:33:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.300+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73940744b0 msgr2=0x7f73940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.300+0000 7f73bac00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73940744b0 0x7f73940769c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f73b40d11e0 tx=0x7f73a80073a0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.300+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 msgr2=0x7f73b415dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.300+0000 7f73bac00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b415dcc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f73a000c460 tx=0x7f73a000c920 comp rx=0 tx=0).stop 2024-04-25T11:33:42.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.300+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 shutdown_connections 2024-04-25T11:33:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.300+0000 7f73bac00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73940744b0 0x7f73940769c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.301+0000 7f73bac00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f73b4162470 0x7f73b414c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.301+0000 7f73bac00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b40cf700 0x7f73b415dcc0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.301+0000 7f73bac00640 1 --2- 172.21.15.19:0/3001189462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f73b40cf2f0 0x7f73b4160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.301+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 >> 172.21.15.19:0/3001189462 conn(0x7f73b40d1720 msgr2=0x7f73b406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.301+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 shutdown_connections 2024-04-25T11:33:42.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.301+0000 7f73bac00640 1 -- 172.21.15.19:0/3001189462 wait complete. 2024-04-25T11:33:42.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:31:05.039941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:31:05.784246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:33:42.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:33:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.621+0000 7fa08ae00640 1 Processor -- start 2024-04-25T11:33:42.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.621+0000 7fa08ae00640 1 -- start start 2024-04-25T11:33:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.624+0000 7fa08ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa084144340 0x7fa08406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.624+0000 7fa08ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 0x7fa08415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.624+0000 7fa08ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa08406df20 con 0x7fa08406db10 2024-04-25T11:33:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.624+0000 7fa08ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa08415f270 con 0x7fa084144340 2024-04-25T11:33:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.624+0000 7fa08ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa084144750 con 0x7fa08415ee60 2024-04-25T11:33:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.625+0000 7fa089000640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa08406db10 0x7fa08415e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:56904/0 (socket says 172.21.15.19:56904) 2024-04-25T11:33:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.625+0000 7fa089000640 1 -- 172.21.15.19:0/4088173709 learned_addr learned my addr 172.21.15.19:0/4088173709 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.625+0000 7fa089a00640 1 --2- 172.21.15.19:0/4088173709 >> v2:172.21.15.169:3300/0 conn(0x7fa084144340 0x7fa08406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.625+0000 7fa08a400640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 0x7fa08415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.626+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3838159550 0 0) 0x7fa070001110 con 0x7fa08406db10 2024-04-25T11:33:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.626+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa06c003520 con 0x7fa08406db10 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 <== mon.1 v1:172.21.15.169:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3465585761 0 0) 0x7fa06c003520 con 0x7fa08406db10 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 msgr2=0x7fa08415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 0x7fa08415f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 >> v2:172.21.15.169:3300/0 conn(0x7fa084144340 msgr2=0x7fa08406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 --2- 172.21.15.19:0/4088173709 >> v2:172.21.15.169:3300/0 conn(0x7fa084144340 0x7fa08406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 --> v1:172.21.15.169:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa084070000 con 0x7fa08406db10 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 <== mon.1 v1:172.21.15.169:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa070001840 con 0x7fa08406db10 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 >> v1:172.21.15.169:6789/0 conn(0x7fa08406db10 legacy=0x7fa08415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 0x7fa06c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c003f80 0x7fa06c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c004d40 0x7fa06c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa084144750 con 0x7fa06c004d40 2024-04-25T11:33:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c00ba70 con 0x7fa06c003f80 2024-04-25T11:33:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa06c00bbf0 con 0x7fa08415ee60 2024-04-25T11:33:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa08a400640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c004d40 0x7fa06c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.627+0000 7fa089a00640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 0x7fa06c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.628+0000 7fa089000640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c003f80 0x7fa06c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.628+0000 7fa08a400640 1 -- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 msgr2=0x7fa06c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.628+0000 7fa08a400640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415ee60 0x7fa06c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.628+0000 7fa08a400640 1 -- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c003f80 msgr2=0x7fa06c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.628+0000 7fa08a400640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa06c003f80 0x7fa06c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.628+0000 7fa08a400640 1 -- 172.21.15.19:0/4088173709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c003520 con 0x7fa06c004d40 2024-04-25T11:33:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.629+0000 7fa08a400640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c004d40 0x7fa06c005150 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009f90 tx=0x7fa06c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27e8bb95a181aff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.629+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c009c10 con 0x7fa06c004d40 2024-04-25T11:33:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.629+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07c036070 con 0x7fa06c004d40 2024-04-25T11:33:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.629+0000 7fa083e00640 1 -- 172.21.15.19:0/4088173709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c031850 con 0x7fa06c004d40 2024-04-25T11:33:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.630+0000 7fa08ae00640 1 -- 172.21.15.19:0/4088173709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c004d40 msgr2=0x7fa06c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.630+0000 7fa08ae00640 1 --2- 172.21.15.19:0/4088173709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa06c004d40 0x7fa06c005150 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009f90 tx=0x7fa06c005830 comp rx=0 tx=0).stop 2024-04-25T11:33:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.630+0000 7fa08a400640 1 -- 172.21.15.19:0/4088173709 reap_dead start 2024-04-25T11:33:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.630+0000 7fa08ae00640 1 -- 172.21.15.19:0/4088173709 shutdown_connections 2024-04-25T11:33:42.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.630+0000 7fa08ae00640 1 -- 172.21.15.19:0/4088173709 >> 172.21.15.19:0/4088173709 conn(0x7fa084072f40 msgr2=0x7fa084073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.630+0000 7fa08ae00640 1 -- 172.21.15.19:0/4088173709 shutdown_connections 2024-04-25T11:33:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.630+0000 7fa08ae00640 1 -- 172.21.15.19:0/4088173709 wait complete. 2024-04-25T11:33:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.633+0000 7fa08ae00640 1 Processor -- start 2024-04-25T11:33:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.633+0000 7fa08ae00640 1 -- start start 2024-04-25T11:33:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.634+0000 7fa08ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa08415b390 0x7fa0841eb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.634+0000 7fa08ae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa08415b7a0 0x7fa08415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.634+0000 7fa08ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415d5d0 0x7fa0841e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.634+0000 7fa08ae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa08406f800 con 0x7fa08415b7a0 2024-04-25T11:33:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.634+0000 7fa08ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa08406f520 con 0x7fa08415b390 2024-04-25T11:33:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa08ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa08406fae0 con 0x7fa08415d5d0 2024-04-25T11:33:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa08415b7a0 0x7fa08415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa08415b7a0 0x7fa08415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48208/0 (socket says 172.21.15.19:48208) 2024-04-25T11:33:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 -- 172.21.15.19:0/2195062850 learned_addr learned my addr 172.21.15.19:0/2195062850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa08a400640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415d5d0 0x7fa0841e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089a00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa08415b390 0x7fa0841eb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 -- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415d5d0 msgr2=0x7fa0841e78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415d5d0 0x7fa0841e78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 -- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa08415b390 msgr2=0x7fa0841eb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa08415b390 0x7fa0841eb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.635+0000 7fa089000640 1 -- 172.21.15.19:0/2195062850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08415d9e0 con 0x7fa08415b7a0 2024-04-25T11:33:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.636+0000 7fa089000640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa08415b7a0 0x7fa08415d090 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa070004840 tx=0x7fa07000ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.636+0000 7fa081600640 1 -- 172.21.15.19:0/2195062850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000c8a0 con 0x7fa08415b7a0 2024-04-25T11:33:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.636+0000 7fa081600640 1 -- 172.21.15.19:0/2195062850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa070018070 con 0x7fa08415b7a0 2024-04-25T11:33:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.636+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0841e8c10 con 0x7fa08415b7a0 2024-04-25T11:33:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.636+0000 7fa081600640 1 -- 172.21.15.19:0/2195062850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000cc40 con 0x7fa08415b7a0 2024-04-25T11:33:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.636+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0841e91d0 con 0x7fa08415b7a0 2024-04-25T11:33:42.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.637+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa084076c60 con 0x7fa08415b7a0 2024-04-25T11:33:42.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.639+0000 7fa081600640 1 -- 172.21.15.19:0/2195062850 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa07001d460 con 0x7fa08415b7a0 2024-04-25T11:33:42.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.640+0000 7fa081600640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0640744b0 0x7fa0640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:42.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.640+0000 7fa081600640 1 -- 172.21.15.19:0/2195062850 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7fa070095310 con 0x7fa08415b7a0 2024-04-25T11:33:42.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.640+0000 7fa089a00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0640744b0 0x7fa0640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:42.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.642+0000 7fa089a00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0640744b0 0x7fa0640769c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa0840b6540 tx=0x7fa078008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:42.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.645+0000 7fa081600640 1 -- 172.21.15.19:0/2195062850 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa08415d9e0 con 0x7fa08415b7a0 2024-04-25T11:33:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.852+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0840b7920 con 0x7fa08415b7a0 2024-04-25T11:33:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.853+0000 7fa081600640 1 -- 172.21.15.19:0/2195062850 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7fa08406df20 con 0x7fa08415b7a0 2024-04-25T11:33:42.841 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:33:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0640744b0 msgr2=0x7fa0640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0640744b0 0x7fa0640769c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa0840b6540 tx=0x7fa078008040 comp rx=0 tx=0).stop 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa08415b7a0 msgr2=0x7fa08415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa08415b7a0 0x7fa08415d090 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa070004840 tx=0x7fa07000ef80 comp rx=0 tx=0).stop 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 shutdown_connections 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0640744b0 0x7fa0640769c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa08415d5d0 0x7fa0841e78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa08415b7a0 0x7fa08415d090 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 --2- 172.21.15.19:0/2195062850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa08415b390 0x7fa0841eb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 >> 172.21.15.19:0/2195062850 conn(0x7fa084072f40 msgr2=0x7fa0840b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 shutdown_connections 2024-04-25T11:33:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:42.856+0000 7fa08ae00640 1 -- 172.21.15.19:0/2195062850 wait complete. 2024-04-25T11:33:42.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:33:43.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.178+0000 7f805fe00640 1 Processor -- start 2024-04-25T11:33:43.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.178+0000 7f805fe00640 1 -- start start 2024-04-25T11:33:43.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.180+0000 7f805fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8060144340 0x7f806006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.180+0000 7f805fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f806015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.181+0000 7f805fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806006df20 con 0x7f806015ec90 2024-04-25T11:33:43.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.181+0000 7f805fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8060144750 con 0x7f8060144340 2024-04-25T11:33:43.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.181+0000 7f805fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8060070000 con 0x7f806006db10 2024-04-25T11:33:43.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.181+0000 7f805e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f806015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.181+0000 7f805ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8060144340 0x7f806006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.181+0000 7f805ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8060144340 0x7f806006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:45166/0 (socket says 172.21.15.19:45166) 2024-04-25T11:33:43.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.181+0000 7f805ea00640 1 -- 172.21.15.19:0/4180789865 learned_addr learned my addr 172.21.15.19:0/4180789865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.182+0000 7f805ea00640 1 -- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 msgr2=0x7f806015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.182+0000 7f805ea00640 1 --2- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f806015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.182+0000 7f805ea00640 1 -- 172.21.15.19:0/4180789865 >> v1:172.21.15.169:6789/0 conn(0x7f806015ec90 legacy=0x7f8060161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.182+0000 7f805ea00640 1 -- 172.21.15.19:0/4180789865 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806006f240 con 0x7f8060144340 2024-04-25T11:33:43.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.182+0000 7f805e000640 1 --2- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f806015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:33:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.184+0000 7f805ea00640 1 --2- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 0x7f806006d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8050009870 tx=0x7f805002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=203f5ffdc0711c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:43.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.184+0000 7f805d600640 1 -- 172.21.15.19:0/4180789865 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050037070 con 0x7f8060144340 2024-04-25T11:33:43.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.184+0000 7f805d600640 1 -- 172.21.15.19:0/4180789865 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8050032440 con 0x7f8060144340 2024-04-25T11:33:43.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805d600640 1 -- 172.21.15.19:0/4180789865 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050032810 con 0x7f8060144340 2024-04-25T11:33:43.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805fe00640 1 -- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 msgr2=0x7f806006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805fe00640 1 --2- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 0x7f806006d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8050009870 tx=0x7f805002a250 comp rx=0 tx=0).stop 2024-04-25T11:33:43.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805fe00640 1 -- 172.21.15.19:0/4180789865 shutdown_connections 2024-04-25T11:33:43.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805fe00640 1 --2- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f806015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805fe00640 1 --2- 172.21.15.19:0/4180789865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 0x7f806006d5d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805fe00640 1 -- 172.21.15.19:0/4180789865 >> 172.21.15.19:0/4180789865 conn(0x7f8060072f40 msgr2=0x7f8060073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.185+0000 7f805fe00640 1 -- 172.21.15.19:0/4180789865 shutdown_connections 2024-04-25T11:33:43.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.186+0000 7f805fe00640 1 -- 172.21.15.19:0/4180789865 wait complete. 2024-04-25T11:33:43.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.188+0000 7f805fe00640 1 Processor -- start 2024-04-25T11:33:43.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.188+0000 7f805fe00640 1 -- start start 2024-04-25T11:33:43.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.189+0000 7f805fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f80601f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 0x7f80601461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806015ec90 0x7f8060146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f806006fae0 con 0x7f806015ec90 2024-04-25T11:33:43.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f806006f800 con 0x7f8060144340 2024-04-25T11:33:43.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f806015f0a0 con 0x7f806006db10 2024-04-25T11:33:43.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806015ec90 0x7f8060146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806015ec90 0x7f8060146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48222/0 (socket says 172.21.15.19:48222) 2024-04-25T11:33:43.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805f400640 1 -- 172.21.15.19:0/4188952872 learned_addr learned my addr 172.21.15.19:0/4188952872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:43.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805ea00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f80601f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.190+0000 7f805e000640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 0x7f80601461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f805f400640 1 -- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 msgr2=0x7f80601f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f805f400640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f80601f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f805f400640 1 -- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 msgr2=0x7f80601461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f805f400640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 0x7f80601461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f805f400640 1 -- 172.21.15.19:0/4188952872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80601f99d0 con 0x7f806015ec90 2024-04-25T11:33:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f805f400640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806015ec90 0x7f8060146720 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f805800c3c0 tx=0x7f805800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f8057400640 1 -- 172.21.15.19:0/4188952872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058017070 con 0x7f806015ec90 2024-04-25T11:33:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f8057400640 1 -- 172.21.15.19:0/4188952872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80580052d0 con 0x7f806015ec90 2024-04-25T11:33:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8060158cf0 con 0x7f806015ec90 2024-04-25T11:33:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.191+0000 7f8057400640 1 -- 172.21.15.19:0/4188952872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058003810 con 0x7f806015ec90 2024-04-25T11:33:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.192+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80601592b0 con 0x7f806015ec90 2024-04-25T11:33:43.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.193+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8060157df0 con 0x7f806015ec90 2024-04-25T11:33:43.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.194+0000 7f8057400640 1 -- 172.21.15.19:0/4188952872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f805801a070 con 0x7f806015ec90 2024-04-25T11:33:43.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.195+0000 7f8057400640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744b0 0x7f803c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.195+0000 7f8057400640 1 -- 172.21.15.19:0/4188952872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f8058094620 con 0x7f806015ec90 2024-04-25T11:33:43.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.196+0000 7f805ea00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744b0 0x7f803c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.197+0000 7f805ea00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744b0 0x7f803c0769c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8050034000 tx=0x7f805002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:43.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.200+0000 7f8057400640 1 -- 172.21.15.19:0/4188952872 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8060157df0 con 0x7f806015ec90 2024-04-25T11:33:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.403+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8060076c60 con 0x7f806015ec90 2024-04-25T11:33:43.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.404+0000 7f8057400640 1 -- 172.21.15.19:0/4188952872 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) ==== 104+0+11 (secure 0 0 0) 0x7f806006df20 con 0x7f806015ec90 2024-04-25T11:33:43.392 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:33:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.406+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744b0 msgr2=0x7f803c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.406+0000 7f805fe00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744b0 0x7f803c0769c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8050034000 tx=0x7f805002e000 comp rx=0 tx=0).stop 2024-04-25T11:33:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.406+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806015ec90 msgr2=0x7f8060146720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.406+0000 7f805fe00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806015ec90 0x7f8060146720 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f805800c3c0 tx=0x7f805800c880 comp rx=0 tx=0).stop 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 shutdown_connections 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744b0 0x7f803c0769c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8060144340 0x7f80601461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f806015ec90 0x7f8060146720 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 --2- 172.21.15.19:0/4188952872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f806006db10 0x7f80601f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 >> 172.21.15.19:0/4188952872 conn(0x7f8060072f40 msgr2=0x7f8060073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 shutdown_connections 2024-04-25T11:33:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.407+0000 7f805fe00640 1 -- 172.21.15.19:0/4188952872 wait complete. 2024-04-25T11:33:43.421 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:33:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.722+0000 7f4c61600640 1 Processor -- start 2024-04-25T11:33:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.723+0000 7f4c61600640 1 -- start start 2024-04-25T11:33:43.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.725+0000 7f4c61600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c15f360 0x7f4c5c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.725+0000 7f4c61600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5c0cdea0 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.725+0000 7f4c61600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5c0ce180 con 0x7f4c5c06dce0 2024-04-25T11:33:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.725+0000 7f4c61600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c0ceab0 con 0x7f4c5c15f360 2024-04-25T11:33:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.726+0000 7f4c5aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4c5c0cd9a0 0x7f4c5c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36296/0 (socket says 172.21.15.19:36296) 2024-04-25T11:33:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.726+0000 7f4c5aa00640 1 -- 172.21.15.19:0/2308065010 learned_addr learned my addr 172.21.15.19:0/2308065010 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.726+0000 7f4c5b400640 1 --2- 172.21.15.19:0/2308065010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c15f360 0x7f4c5c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.726+0000 7f4c5a000640 1 -- 172.21.15.19:0/2308065010 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114735364 0 0) 0x7f4c5c0cdea0 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.726+0000 7f4c5a000640 1 -- 172.21.15.19:0/2308065010 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c40003520 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.727+0000 7f4c5b400640 1 -- 172.21.15.19:0/2308065010 >> v1:172.21.15.169:6789/0 conn(0x7f4c5c06dce0 legacy=0x7f4c5c15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.727+0000 7f4c5b400640 1 -- 172.21.15.19:0/2308065010 >> v1:172.21.15.19:6789/0 conn(0x7f4c5c0cd9a0 legacy=0x7f4c5c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.727+0000 7f4c5b400640 1 -- 172.21.15.19:0/2308065010 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c0ce460 con 0x7f4c5c15f360 2024-04-25T11:33:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.728+0000 7f4c5b400640 1 --2- 172.21.15.19:0/2308065010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c15f360 0x7f4c5c0b0af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009dc0 tx=0x7f4c5402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beae66655cd25a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c5a000640 1 -- 172.21.15.19:0/2308065010 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54007740 con 0x7f4c5c15f360 2024-04-25T11:33:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c5a000640 1 -- 172.21.15.19:0/2308065010 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c540078e0 con 0x7f4c5c15f360 2024-04-25T11:33:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c5a000640 1 -- 172.21.15.19:0/2308065010 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54007cb0 con 0x7f4c5c15f360 2024-04-25T11:33:43.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c61600640 1 -- 172.21.15.19:0/2308065010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c15f360 msgr2=0x7f4c5c0b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c61600640 1 --2- 172.21.15.19:0/2308065010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c15f360 0x7f4c5c0b0af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009dc0 tx=0x7f4c5402a6d0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c61600640 1 -- 172.21.15.19:0/2308065010 shutdown_connections 2024-04-25T11:33:43.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c61600640 1 --2- 172.21.15.19:0/2308065010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c15f360 0x7f4c5c0b0af0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c61600640 1 -- 172.21.15.19:0/2308065010 >> 172.21.15.19:0/2308065010 conn(0x7f4c5c073090 msgr2=0x7f4c5c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.729+0000 7f4c61600640 1 -- 172.21.15.19:0/2308065010 shutdown_connections 2024-04-25T11:33:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.730+0000 7f4c61600640 1 -- 172.21.15.19:0/2308065010 wait complete. 2024-04-25T11:33:43.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.732+0000 7f4c61600640 1 Processor -- start 2024-04-25T11:33:43.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.732+0000 7f4c61600640 1 -- start start 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c61600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c5c0cd9a0 0x7f4c5c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c61600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c15f360 0x7f4c5c13c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c61600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c157c10 0x7f4c5c158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c61600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c0cef40 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c61600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c15f770 con 0x7f4c5c15f360 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c61600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c0cf0c0 con 0x7f4c5c157c10 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c5c0cd9a0 0x7f4c5c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c5c0cd9a0 0x7f4c5c155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48230/0 (socket says 172.21.15.19:48230) 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5aa00640 1 -- 172.21.15.19:0/36339605 learned_addr learned my addr 172.21.15.19:0/36339605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c53e00640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c15f360 0x7f4c5c13c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5b400640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c157c10 0x7f4c5c158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5aa00640 1 -- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c157c10 msgr2=0x7f4c5c158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5aa00640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c157c10 0x7f4c5c158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5aa00640 1 -- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c15f360 msgr2=0x7f4c5c13c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.734+0000 7f4c5aa00640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c15f360 0x7f4c5c13c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.735+0000 7f4c5aa00640 1 -- 172.21.15.19:0/36339605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c158560 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.735+0000 7f4c5aa00640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c5c0cd9a0 0x7f4c5c155a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00c410 tx=0x7f4c4c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.735+0000 7f4c52a00640 1 -- 172.21.15.19:0/36339605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c017070 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.735+0000 7f4c52a00640 1 -- 172.21.15.19:0/36339605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c005ce0 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.735+0000 7f4c52a00640 1 -- 172.21.15.19:0/36339605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c0072f0 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.735+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c0ab560 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.735+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1568c0 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.737+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c050740 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.738+0000 7f4c52a00640 1 -- 172.21.15.19:0/36339605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4c4c01a070 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.739+0000 7f4c52a00640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:43.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.739+0000 7f4c52a00640 1 -- 172.21.15.19:0/36339605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f4c4c094840 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.740+0000 7f4c53e00640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.742+0000 7f4c53e00640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4c44007fd0 tx=0x7f4c44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:43.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.744+0000 7f4c52a00640 1 -- 172.21.15.19:0/36339605 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c5c158560 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.953+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4c5c0b6290 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.954+0000 7f4c52a00640 1 -- 172.21.15.19:0/36339605 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) ==== 100+0+11 (secure 0 0 0) 0x7f4c4c05ef00 con 0x7f4c5c0cd9a0 2024-04-25T11:33:43.942 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.956+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 msgr2=0x7f4c380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.956+0000 7f4c61600640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4c44007fd0 tx=0x7f4c44009040 comp rx=0 tx=0).stop 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.956+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c5c0cd9a0 msgr2=0x7f4c5c155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.956+0000 7f4c61600640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c5c0cd9a0 0x7f4c5c155a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00c410 tx=0x7f4c4c00c8d0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 shutdown_connections 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c380744b0 0x7f4c380769c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c15f360 0x7f4c5c13c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c5c157c10 0x7f4c5c158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 --2- 172.21.15.19:0/36339605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c5c0cd9a0 0x7f4c5c155a00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 >> 172.21.15.19:0/36339605 conn(0x7f4c5c073090 msgr2=0x7f4c5c15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 shutdown_connections 2024-04-25T11:33:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:43.957+0000 7f4c61600640 1 -- 172.21.15.19:0/36339605 wait complete. 2024-04-25T11:33:43.971 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:33:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.263+0000 7f5a1c200640 1 Processor -- start 2024-04-25T11:33:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.263+0000 7f5a1c200640 1 -- start start 2024-04-25T11:33:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.265+0000 7f5a1c200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5a140b63a0 0x7f5a141647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.266+0000 7f5a1c200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1406c7d0 con 0x7f5a14070680 2024-04-25T11:33:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.266+0000 7f5a1c200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1406cab0 con 0x7f5a14160360 2024-04-25T11:33:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.266+0000 7f5a1c200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5a1406d100 con 0x7f5a140b63a0 2024-04-25T11:33:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.266+0000 7f5a19800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5a14070680 0x7f5a140af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58678/0 (socket says 172.21.15.19:58678) 2024-04-25T11:33:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.266+0000 7f5a19800640 1 -- 172.21.15.19:0/3158306286 learned_addr learned my addr 172.21.15.19:0/3158306286 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.266+0000 7f5a1a200640 1 --2- 172.21.15.19:0/3158306286 >> v2:172.21.15.19:3300/0 conn(0x7f5a140b63a0 0x7f5a141647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.267+0000 7f5a13e00640 1 -- 172.21.15.19:0/3158306286 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214884751 0 0) 0x7f5a1406c7d0 con 0x7f5a14070680 2024-04-25T11:33:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.267+0000 7f5a13e00640 1 -- 172.21.15.19:0/3158306286 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59fc003520 con 0x7f5a14070680 2024-04-25T11:33:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.267+0000 7f5a1a200640 1 -- 172.21.15.19:0/3158306286 >> v1:172.21.15.169:6789/0 conn(0x7f5a14160360 legacy=0x7f5a140b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.267+0000 7f5a1a200640 1 -- 172.21.15.19:0/3158306286 >> v1:172.21.15.19:6789/0 conn(0x7f5a14070680 legacy=0x7f5a140af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.267+0000 7f5a1a200640 1 -- 172.21.15.19:0/3158306286 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1406cd90 con 0x7f5a140b63a0 2024-04-25T11:33:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.268+0000 7f5a1a200640 1 --2- 172.21.15.19:0/3158306286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b63a0 0x7f5a141647f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5a1406d280 tx=0x7f5a0c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7845fb5493d8a78a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:44.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.269+0000 7f5a13e00640 1 -- 172.21.15.19:0/3158306286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c007740 con 0x7f5a140b63a0 2024-04-25T11:33:44.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.269+0000 7f5a13e00640 1 -- 172.21.15.19:0/3158306286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c0078e0 con 0x7f5a140b63a0 2024-04-25T11:33:44.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.269+0000 7f5a13e00640 1 -- 172.21.15.19:0/3158306286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c007cb0 con 0x7f5a140b63a0 2024-04-25T11:33:44.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.269+0000 7f5a1c200640 1 -- 172.21.15.19:0/3158306286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b63a0 msgr2=0x7f5a141647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:44.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.269+0000 7f5a1c200640 1 --2- 172.21.15.19:0/3158306286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b63a0 0x7f5a141647f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5a1406d280 tx=0x7f5a0c02a660 comp rx=0 tx=0).stop 2024-04-25T11:33:44.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.269+0000 7f5a1c200640 1 -- 172.21.15.19:0/3158306286 shutdown_connections 2024-04-25T11:33:44.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.269+0000 7f5a1c200640 1 --2- 172.21.15.19:0/3158306286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b63a0 0x7f5a141647f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:44.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.270+0000 7f5a1c200640 1 -- 172.21.15.19:0/3158306286 >> 172.21.15.19:0/3158306286 conn(0x7f5a140d1ae0 msgr2=0x7f5a140d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:44.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.270+0000 7f5a1c200640 1 -- 172.21.15.19:0/3158306286 shutdown_connections 2024-04-25T11:33:44.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.270+0000 7f5a1c200640 1 -- 172.21.15.19:0/3158306286 wait complete. 2024-04-25T11:33:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.272+0000 7f5a1c200640 1 Processor -- start 2024-04-25T11:33:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.272+0000 7f5a1c200640 1 -- start start 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1c200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a14070680 0x7f5a1415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1c200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a14160360 0x7f5a14150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1c200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b2010 0x7f5a141560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1c200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a140afd30 con 0x7f5a140b2010 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1c200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5a140afbb0 con 0x7f5a14160360 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1c200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1406d590 con 0x7f5a14070680 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b2010 0x7f5a141560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b2010 0x7f5a141560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33750/0 (socket says 172.21.15.19:33750) 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a1a200640 1 -- 172.21.15.19:0/71444240 learned_addr learned my addr 172.21.15.19:0/71444240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.274+0000 7f5a19800640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a14070680 0x7f5a1415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:44.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.275+0000 7f5a18e00640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a14160360 0x7f5a14150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.275+0000 7f5a1a200640 1 -- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a14070680 msgr2=0x7f5a1415c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.275+0000 7f5a1a200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a14070680 0x7f5a1415c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.275+0000 7f5a1a200640 1 -- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a14160360 msgr2=0x7f5a14150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.275+0000 7f5a1a200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a14160360 0x7f5a14150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.275+0000 7f5a1a200640 1 -- 172.21.15.19:0/71444240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a140b2a60 con 0x7f5a140b2010 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.275+0000 7f5a1a200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b2010 0x7f5a141560b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009e50 tx=0x7f5a0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.276+0000 7f5a11600640 1 -- 172.21.15.19:0/71444240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c03f070 con 0x7f5a140b2010 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.276+0000 7f5a11600640 1 -- 172.21.15.19:0/71444240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c031bf0 con 0x7f5a140b2010 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.276+0000 7f5a11600640 1 -- 172.21.15.19:0/71444240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c009450 con 0x7f5a140b2010 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.276+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a141555d0 con 0x7f5a140b2010 2024-04-25T11:33:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.276+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a14155b90 con 0x7f5a140b2010 2024-04-25T11:33:44.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.277+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a14050740 con 0x7f5a140b2010 2024-04-25T11:33:44.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.278+0000 7f5a11600640 1 -- 172.21.15.19:0/71444240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a0c03b410 con 0x7f5a140b2010 2024-04-25T11:33:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.279+0000 7f5a11600640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59f40744b0 0x7f59f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.280+0000 7f5a11600640 1 -- 172.21.15.19:0/71444240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f5a0c0b6be0 con 0x7f5a140b2010 2024-04-25T11:33:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.280+0000 7f5a19800640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59f40744b0 0x7f59f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.281+0000 7f5a19800640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59f40744b0 0x7f59f40769c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a080011e0 tx=0x7f5a08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.285+0000 7f5a11600640 1 -- 172.21.15.19:0/71444240 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a0c080c20 con 0x7f5a140b2010 2024-04-25T11:33:44.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.502+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a140b27f0 con 0x7f5a140b2010 2024-04-25T11:33:44.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.503+0000 7f5a11600640 1 -- 172.21.15.19:0/71444240 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v67) ==== 80+0+242 (secure 0 0 0) 0x7f5a0c084f20 con 0x7f5a140b2010 2024-04-25T11:33:44.491 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6833/1883095429 2024-04-26T11:31:58.298678+0000 2024-04-25T11:33:44.492 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6833/751039234 2024-04-26T11:31:25.310192+0000 2024-04-25T11:33:44.492 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6832/1883095429 2024-04-26T11:31:58.298678+0000 2024-04-25T11:33:44.492 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6832/751039234 2024-04-26T11:31:25.310192+0000 2024-04-25T11:33:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:listed 4 entries 2024-04-25T11:33:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59f40744b0 msgr2=0x7f59f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59f40744b0 0x7f59f40769c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a080011e0 tx=0x7f5a08009040 comp rx=0 tx=0).stop 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b2010 msgr2=0x7f5a141560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b2010 0x7f5a141560b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009e50 tx=0x7f5a0c004a40 comp rx=0 tx=0).stop 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 shutdown_connections 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59f40744b0 0x7f59f40769c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a14160360 0x7f5a14150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a140b2010 0x7f5a141560b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 --2- 172.21.15.19:0/71444240 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a14070680 0x7f5a1415c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.505+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 >> 172.21.15.19:0/71444240 conn(0x7f5a140d1ae0 msgr2=0x7f5a1415aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.506+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 shutdown_connections 2024-04-25T11:33:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:44.506+0000 7f5a1c200640 1 -- 172.21.15.19:0/71444240 wait complete. 2024-04-25T11:33:44.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:33:44.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:44.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:33:44.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:33:44.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:33:44.622 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:33:44.679 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:33:44.679 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:33:44.679 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:44.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:33:44.736 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:33:44.737 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:33:44.737 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:44.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:33:44.795 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:33:44.796 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:33:44.796 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:44.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:33:44.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:33:45.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.156+0000 7f7928a00640 1 Processor -- start 2024-04-25T11:33:45.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.156+0000 7f7928a00640 1 -- start start 2024-04-25T11:33:45.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.159+0000 7f7928a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 0x7f79240d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.159+0000 7f7928a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7924160be0 0x7f7924160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.159+0000 7f7928a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7924161530 con 0x7f79241434d0 2024-04-25T11:33:45.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.159+0000 7f7928a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79241438e0 con 0x7f7924160be0 2024-04-25T11:33:45.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.159+0000 7f7928a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f792406f3e0 con 0x7f79240d0ed0 2024-04-25T11:33:45.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.159+0000 7f7921e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f79241434d0 0x7f7924070480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58682/0 (socket says 172.21.15.19:58682) 2024-04-25T11:33:45.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.159+0000 7f7921e00640 1 -- 172.21.15.19:0/1170694674 learned_addr learned my addr 172.21.15.19:0/1170694674 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966586234 0 0) 0x7f7914001110 con 0x7f79241434d0 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7908003520 con 0x7f79241434d0 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7922800640 1 --2- 172.21.15.19:0/1170694674 >> v2:172.21.15.169:3300/0 conn(0x7f7924160be0 0x7f7924160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7921400640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 0x7f79240d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3420681425 0 0) 0x7f7908003520 con 0x7f79241434d0 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 msgr2=0x7f79240d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 0x7f79240d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 >> v2:172.21.15.169:3300/0 conn(0x7f7924160be0 msgr2=0x7f7924160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 --2- 172.21.15.19:0/1170694674 >> v2:172.21.15.169:3300/0 conn(0x7f7924160be0 0x7f7924160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792406f6c0 con 0x7f79241434d0 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7914001840 con 0x7f79241434d0 2024-04-25T11:33:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 >> v1:172.21.15.19:6789/0 conn(0x7f79241434d0 legacy=0x7f7924070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79080039b0 0x7f7908003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908004800 0x7f7908004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 0x7f7908005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79241438e0 con 0x7f79080039b0 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f792406f3e0 con 0x7f7908004800 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.160+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f790800bb40 con 0x7f79240d0ed0 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.161+0000 7f7921e00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79080039b0 0x7f7908003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.161+0000 7f7922800640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 0x7f7908005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.161+0000 7f7921e00640 1 -- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 msgr2=0x7f7908005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.161+0000 7f7921e00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79240d0ed0 0x7f7908005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.161+0000 7f7921e00640 1 -- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908004800 msgr2=0x7f7908004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.161+0000 7f7921e00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7908004800 0x7f7908004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.161+0000 7f7921e00640 1 -- 172.21.15.19:0/1170694674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7908003520 con 0x7f79080039b0 2024-04-25T11:33:45.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.162+0000 7f7921e00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79080039b0 0x7f7908003e20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f791400c2f0 tx=0x7f7908005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951c4b16949c9c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.162+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914035070 con 0x7f79080039b0 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.162+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79140058d0 con 0x7f79080039b0 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.162+0000 7f7920a00640 1 -- 172.21.15.19:0/1170694674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914007e30 con 0x7f79080039b0 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.163+0000 7f7928a00640 1 -- 172.21.15.19:0/1170694674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79080039b0 msgr2=0x7f7908003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.163+0000 7f7928a00640 1 --2- 172.21.15.19:0/1170694674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79080039b0 0x7f7908003e20 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f791400c2f0 tx=0x7f7908005830 comp rx=0 tx=0).stop 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.163+0000 7f7922800640 1 -- 172.21.15.19:0/1170694674 reap_dead start 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.163+0000 7f7928a00640 1 -- 172.21.15.19:0/1170694674 shutdown_connections 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.163+0000 7f7928a00640 1 -- 172.21.15.19:0/1170694674 >> 172.21.15.19:0/1170694674 conn(0x7f79240d1ae0 msgr2=0x7f79240d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.163+0000 7f7928a00640 1 -- 172.21.15.19:0/1170694674 shutdown_connections 2024-04-25T11:33:45.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.163+0000 7f7928a00640 1 -- 172.21.15.19:0/1170694674 wait complete. 2024-04-25T11:33:45.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.166+0000 7f7928a00640 1 Processor -- start 2024-04-25T11:33:45.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.166+0000 7f7928a00640 1 -- start start 2024-04-25T11:33:45.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.167+0000 7f7928a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79080039b0 0x7f79241400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.167+0000 7f7928a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792413e3f0 0x7f792414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7928a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7924145ff0 0x7f7924146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7928a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f792406f090 con 0x7f792413e3f0 2024-04-25T11:33:45.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7928a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f792406fc80 con 0x7f79080039b0 2024-04-25T11:33:45.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7928a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79240d12e0 con 0x7f7924145ff0 2024-04-25T11:33:45.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792413e3f0 0x7f792414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792413e3f0 0x7f792414f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33766/0 (socket says 172.21.15.19:33766) 2024-04-25T11:33:45.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 -- 172.21.15.19:0/3878672140 learned_addr learned my addr 172.21.15.19:0/3878672140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7922800640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7924145ff0 0x7f7924146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921e00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79080039b0 0x7f79241400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 -- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7924145ff0 msgr2=0x7f7924146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7924145ff0 0x7f7924146400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 -- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79080039b0 msgr2=0x7f79241400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79080039b0 0x7f79241400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921400640 1 -- 172.21.15.19:0/3878672140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79241405f0 con 0x7f792413e3f0 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7922800640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7924145ff0 0x7f7924146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:45.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.168+0000 7f7921e00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79080039b0 0x7f79241400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.169+0000 7f7921400640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792413e3f0 0x7f792414f000 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d800 tx=0x7f790c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.169+0000 7f791a000640 1 -- 172.21.15.19:0/3878672140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c018020 con 0x7f792413e3f0 2024-04-25T11:33:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.169+0000 7f791a000640 1 -- 172.21.15.19:0/3878672140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c00b6c0 con 0x7f792413e3f0 2024-04-25T11:33:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.169+0000 7f791a000640 1 -- 172.21.15.19:0/3878672140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c00ba90 con 0x7f792413e3f0 2024-04-25T11:33:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.169+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792413f010 con 0x7f792413e3f0 2024-04-25T11:33:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.169+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792413f5d0 con 0x7f792413e3f0 2024-04-25T11:33:45.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.171+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7924050740 con 0x7f792413e3f0 2024-04-25T11:33:45.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.172+0000 7f791a000640 1 -- 172.21.15.19:0/3878672140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f790c01b020 con 0x7f792413e3f0 2024-04-25T11:33:45.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.173+0000 7f791a000640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79000744b0 0x7f79000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.173+0000 7f791a000640 1 -- 172.21.15.19:0/3878672140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f790c0955c0 con 0x7f792413e3f0 2024-04-25T11:33:45.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.174+0000 7f7921e00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79000744b0 0x7f79000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.175+0000 7f7921e00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79000744b0 0x7f79000769c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7914032000 tx=0x7f7914009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:45.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.178+0000 7f791a000640 1 -- 172.21.15.19:0/3878672140 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f790c022030 con 0x7f792413e3f0 2024-04-25T11:33:45.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.429+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f792413eda0 con 0x7f792413e3f0 2024-04-25T11:33:45.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.431+0000 7f791a000640 1 -- 172.21.15.19:0/3878672140 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7f790c05fbc0 con 0x7f792413e3f0 2024-04-25T11:33:45.420 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:45.420 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3770220424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3770220424},{"type":"v1","addr":"172.21.15.19:6833","nonce":3770220424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:32:01.321084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5452":{"gid":5452,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":18,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5452,"qdb_cluster":[5452]},"id":4}]} 2024-04-25T11:33:45.420 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T11:33:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.433+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79000744b0 msgr2=0x7f79000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.433+0000 7f7928a00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79000744b0 0x7f79000769c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7914032000 tx=0x7f7914009030 comp rx=0 tx=0).stop 2024-04-25T11:33:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.433+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792413e3f0 msgr2=0x7f792414f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.433+0000 7f7928a00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792413e3f0 0x7f792414f000 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f790c00d800 tx=0x7f790c00dcc0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 shutdown_connections 2024-04-25T11:33:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79000744b0 0x7f79000769c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7924145ff0 0x7f7924146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f792413e3f0 0x7f792414f000 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 --2- 172.21.15.19:0/3878672140 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79080039b0 0x7f79241400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 >> 172.21.15.19:0/3878672140 conn(0x7f79240d1ae0 msgr2=0x7f79240d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 shutdown_connections 2024-04-25T11:33:45.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.434+0000 7f7928a00640 1 -- 172.21.15.19:0/3878672140 wait complete. 2024-04-25T11:33:45.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:33:45.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.747+0000 7fa707c00640 1 Processor -- start 2024-04-25T11:33:45.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.747+0000 7fa707c00640 1 -- start start 2024-04-25T11:33:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.750+0000 7fa707c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa70014ad60 0x7fa700070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.750+0000 7fa707c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7000b4af0 con 0x7fa70006c5c0 2024-04-25T11:33:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.750+0000 7fa707c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7000b4dd0 con 0x7fa7000b5e40 2024-04-25T11:33:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.750+0000 7fa707c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa7000b50b0 con 0x7fa70014ad60 2024-04-25T11:33:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.750+0000 7fa705200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa70014ad60 0x7fa700070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.750+0000 7fa705200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa70014ad60 0x7fa700070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33774/0 (socket says 172.21.15.19:33774) 2024-04-25T11:33:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.750+0000 7fa705200640 1 -- 172.21.15.19:0/2866123989 learned_addr learned my addr 172.21.15.19:0/2866123989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.751+0000 7fa705200640 1 -- 172.21.15.19:0/2866123989 >> v1:172.21.15.169:6789/0 conn(0x7fa7000b5e40 legacy=0x7fa7000b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.751+0000 7fa705200640 1 -- 172.21.15.19:0/2866123989 >> v1:172.21.15.19:6789/0 conn(0x7fa70006c5c0 legacy=0x7fa70006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.751+0000 7fa705200640 1 -- 172.21.15.19:0/2866123989 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7000b5390 con 0x7fa70014ad60 2024-04-25T11:33:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.752+0000 7fa705200640 1 --2- 172.21.15.19:0/2866123989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa700070010 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009e50 tx=0x7fa6f002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392237fa868a1bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.752+0000 7fa6ff400640 1 -- 172.21.15.19:0/2866123989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0036070 con 0x7fa70014ad60 2024-04-25T11:33:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.752+0000 7fa6ff400640 1 -- 172.21.15.19:0/2866123989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f0031600 con 0x7fa70014ad60 2024-04-25T11:33:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.752+0000 7fa6ff400640 1 -- 172.21.15.19:0/2866123989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f00319d0 con 0x7fa70014ad60 2024-04-25T11:33:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.753+0000 7fa707c00640 1 -- 172.21.15.19:0/2866123989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 msgr2=0x7fa700070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.753+0000 7fa707c00640 1 --2- 172.21.15.19:0/2866123989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa700070010 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009e50 tx=0x7fa6f002a6d0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.753+0000 7fa707c00640 1 -- 172.21.15.19:0/2866123989 shutdown_connections 2024-04-25T11:33:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.753+0000 7fa707c00640 1 --2- 172.21.15.19:0/2866123989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa700070010 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.753+0000 7fa707c00640 1 -- 172.21.15.19:0/2866123989 >> 172.21.15.19:0/2866123989 conn(0x7fa7000d1ae0 msgr2=0x7fa7000d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.753+0000 7fa707c00640 1 -- 172.21.15.19:0/2866123989 shutdown_connections 2024-04-25T11:33:45.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.753+0000 7fa707c00640 1 -- 172.21.15.19:0/2866123989 wait complete. 2024-04-25T11:33:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.756+0000 7fa707c00640 1 Processor -- start 2024-04-25T11:33:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.756+0000 7fa707c00640 1 -- start start 2024-04-25T11:33:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa707c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa70006c5c0 0x7fa700147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa707c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7000b5e40 0x7fa7001ed430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa707c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa7001ed970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa707c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7000706d0 con 0x7fa70014ad60 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa707c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa700070550 con 0x7fa7000b5e40 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa707c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa70006fb80 con 0x7fa70006c5c0 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa705c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa7001ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa705c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa7001ed970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33788/0 (socket says 172.21.15.19:33788) 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa705c00640 1 -- 172.21.15.19:0/3967691488 learned_addr learned my addr 172.21.15.19:0/3967691488 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa705200640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa70006c5c0 0x7fa700147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.758+0000 7fa6ffe00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7000b5e40 0x7fa7001ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa705c00640 1 -- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa70006c5c0 msgr2=0x7fa700147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa705c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa70006c5c0 0x7fa700147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa705c00640 1 -- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7000b5e40 msgr2=0x7fa7001ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa705c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7000b5e40 0x7fa7001ed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa705c00640 1 -- 172.21.15.19:0/3967691488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa700150a20 con 0x7fa70014ad60 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa705c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa7001ed970 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800c3c0 tx=0x7fa6f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa6fcc00640 1 -- 172.21.15.19:0/3967691488 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8017070 con 0x7fa70014ad60 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa6fcc00640 1 -- 172.21.15.19:0/3967691488 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f80052d0 con 0x7fa70014ad60 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa6fcc00640 1 -- 172.21.15.19:0/3967691488 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8003850 con 0x7fa70014ad60 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa700150d00 con 0x7fa70014ad60 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.759+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7001512c0 con 0x7fa70014ad60 2024-04-25T11:33:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.761+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa700050740 con 0x7fa70014ad60 2024-04-25T11:33:45.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.762+0000 7fa6fcc00640 1 -- 172.21.15.19:0/3967691488 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa6f801a070 con 0x7fa70014ad60 2024-04-25T11:33:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.763+0000 7fa6fcc00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6dc0744b0 0x7fa6dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.764+0000 7fa6fcc00640 1 -- 172.21.15.19:0/3967691488 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7fa6f8094610 con 0x7fa70014ad60 2024-04-25T11:33:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.764+0000 7fa705200640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6dc0744b0 0x7fa6dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.766+0000 7fa705200640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6dc0744b0 0x7fa6dc0769c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0033000 tx=0x7fa6f0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:45.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.769+0000 7fa6fcc00640 1 -- 172.21.15.19:0/3967691488 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6f805eca0 con 0x7fa70014ad60 2024-04-25T11:33:45.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.979+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa700160da0 con 0x7fa70014ad60 2024-04-25T11:33:45.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.982+0000 7fa6fcc00640 1 -- 172.21.15.19:0/3967691488 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17018 (secure 0 0 0) 0x7fa6f805e5a0 con 0x7fa70014ad60 2024-04-25T11:33:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:45.972 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":67,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:32:07.391254+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.36","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6833/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6833/751039234":"2024-04-26T11:31:25.310192+0000","172.21.15.19:6832/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6832/751039234":"2024-04-26T11:31:25.310192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6dc0744b0 msgr2=0x7fa6dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6dc0744b0 0x7fa6dc0769c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0033000 tx=0x7fa6f0009000 comp rx=0 tx=0).stop 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 msgr2=0x7fa7001ed970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa7001ed970 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800c3c0 tx=0x7fa6f800c880 comp rx=0 tx=0).stop 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 shutdown_connections 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6dc0744b0 0x7fa6dc0769c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7000b5e40 0x7fa7001ed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa70014ad60 0x7fa7001ed970 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 --2- 172.21.15.19:0/3967691488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa70006c5c0 0x7fa700147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:45.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 >> 172.21.15.19:0/3967691488 conn(0x7fa7000d1ae0 msgr2=0x7fa7001637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:45.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 shutdown_connections 2024-04-25T11:33:45.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:45.985+0000 7fa707c00640 1 -- 172.21.15.19:0/3967691488 wait complete. 2024-04-25T11:33:46.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:31:05.039941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:31:05.784246+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:33:46.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:33:46.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.309+0000 7f14dee00640 1 Processor -- start 2024-04-25T11:33:46.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.309+0000 7f14dee00640 1 -- start start 2024-04-25T11:33:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.311+0000 7f14dee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d81441d0 0x7f14d8161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.311+0000 7f14dee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d8162450 con 0x7f14d8162040 2024-04-25T11:33:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.311+0000 7f14dee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d806d3a0 con 0x7f14d806cd40 2024-04-25T11:33:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.311+0000 7f14dee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14d806d150 con 0x7f14d81441d0 2024-04-25T11:33:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.312+0000 7f14dd000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f14d8162040 0x7f14d806c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58700/0 (socket says 172.21.15.19:58700) 2024-04-25T11:33:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.312+0000 7f14dd000640 1 -- 172.21.15.19:0/3491250329 learned_addr learned my addr 172.21.15.19:0/3491250329 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.312+0000 7f14dda00640 1 --2- 172.21.15.19:0/3491250329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d81441d0 0x7f14d8161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.313+0000 7f14d7e00640 1 -- 172.21.15.19:0/3491250329 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 937275686 0 0) 0x7f14c4001110 con 0x7f14d8162040 2024-04-25T11:33:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.313+0000 7f14d7e00640 1 -- 172.21.15.19:0/3491250329 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14bc003520 con 0x7f14d8162040 2024-04-25T11:33:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.313+0000 7f14dda00640 1 -- 172.21.15.19:0/3491250329 >> v1:172.21.15.169:6789/0 conn(0x7f14d806cd40 legacy=0x7f14d815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.313+0000 7f14d7e00640 1 -- 172.21.15.19:0/3491250329 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599295375 0 0) 0x7f14d806d3a0 con 0x7f14d806cd40 2024-04-25T11:33:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.313+0000 7f14dda00640 1 -- 172.21.15.19:0/3491250329 >> v1:172.21.15.19:6789/0 conn(0x7f14d8162040 legacy=0x7f14d806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.313+0000 7f14dda00640 1 -- 172.21.15.19:0/3491250329 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d806d680 con 0x7f14d81441d0 2024-04-25T11:33:46.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.314+0000 7f14dda00640 1 --2- 172.21.15.19:0/3491250329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d81441d0 0x7f14d8161b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009dc0 tx=0x7f14c802a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb314c43d64aab74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.315+0000 7f14d7e00640 1 -- 172.21.15.19:0/3491250329 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8035020 con 0x7f14d81441d0 2024-04-25T11:33:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.315+0000 7f14d7e00640 1 -- 172.21.15.19:0/3491250329 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c8007540 con 0x7f14d81441d0 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.315+0000 7f14d7e00640 1 -- 172.21.15.19:0/3491250329 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4001840 con 0x7f14d81441d0 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.316+0000 7f14dee00640 1 -- 172.21.15.19:0/3491250329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d81441d0 msgr2=0x7f14d8161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.316+0000 7f14dee00640 1 --2- 172.21.15.19:0/3491250329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d81441d0 0x7f14d8161b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009dc0 tx=0x7f14c802a5b0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.316+0000 7f14dee00640 1 -- 172.21.15.19:0/3491250329 shutdown_connections 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.316+0000 7f14dee00640 1 --2- 172.21.15.19:0/3491250329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d81441d0 0x7f14d8161b00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.316+0000 7f14dee00640 1 -- 172.21.15.19:0/3491250329 >> 172.21.15.19:0/3491250329 conn(0x7f14d8072f40 msgr2=0x7f14d8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.316+0000 7f14dee00640 1 -- 172.21.15.19:0/3491250329 shutdown_connections 2024-04-25T11:33:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.316+0000 7f14dee00640 1 -- 172.21.15.19:0/3491250329 wait complete. 2024-04-25T11:33:46.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.318+0000 7f14dee00640 1 Processor -- start 2024-04-25T11:33:46.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.319+0000 7f14dee00640 1 -- start start 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d806cd40 0x7f14d815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81441d0 0x7f14d81348a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d8162040 0x7f14d8134de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81445e0 con 0x7f14d81441d0 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14d806dc40 con 0x7f14d806cd40 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14d8160150 con 0x7f14d8162040 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81441d0 0x7f14d81348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81441d0 0x7f14d81348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33792/0 (socket says 172.21.15.19:33792) 2024-04-25T11:33:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.320+0000 7f14dd000640 1 -- 172.21.15.19:0/2504580638 learned_addr learned my addr 172.21.15.19:0/2504580638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dda00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d806cd40 0x7f14d815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dd000640 1 -- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d8162040 msgr2=0x7f14d8134de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dd000640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d8162040 0x7f14d8134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dd000640 1 -- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d806cd40 msgr2=0x7f14d815d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dd000640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d806cd40 0x7f14d815d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dd000640 1 -- 172.21.15.19:0/2504580638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d815dce0 con 0x7f14d81441d0 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dda00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d806cd40 0x7f14d815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.321+0000 7f14dd000640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81441d0 0x7f14d81348a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f14c400c760 tx=0x7f14c400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.322+0000 7f14d5600640 1 -- 172.21.15.19:0/2504580638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4005a90 con 0x7f14d81441d0 2024-04-25T11:33:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.322+0000 7f14d5600640 1 -- 172.21.15.19:0/2504580638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c4005c30 con 0x7f14d81441d0 2024-04-25T11:33:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.322+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d80aafc0 con 0x7f14d81441d0 2024-04-25T11:33:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.322+0000 7f14d5600640 1 -- 172.21.15.19:0/2504580638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4007070 con 0x7f14d81441d0 2024-04-25T11:33:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.322+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d80ab580 con 0x7f14d81441d0 2024-04-25T11:33:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.323+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d8073350 con 0x7f14d81441d0 2024-04-25T11:33:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.324+0000 7f14d5600640 1 -- 172.21.15.19:0/2504580638 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14c4007210 con 0x7f14d81441d0 2024-04-25T11:33:46.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.325+0000 7f14d5600640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b80744e0 0x7f14b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.325+0000 7f14d5600640 1 -- 172.21.15.19:0/2504580638 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f14c40951f0 con 0x7f14d81441d0 2024-04-25T11:33:46.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.326+0000 7f14dda00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b80744e0 0x7f14b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.327+0000 7f14dda00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b80744e0 0x7f14b80769f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14c8033000 tx=0x7f14c802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:46.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.330+0000 7f14d5600640 1 -- 172.21.15.19:0/2504580638 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14d8073350 con 0x7f14d81441d0 2024-04-25T11:33:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.534+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14d8076c60 con 0x7f14d81441d0 2024-04-25T11:33:46.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.536+0000 7f14d5600640 1 -- 172.21.15.19:0/2504580638 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f14d8162450 con 0x7f14d81441d0 2024-04-25T11:33:46.524 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b80744e0 msgr2=0x7f14b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b80744e0 0x7f14b80769f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f14c8033000 tx=0x7f14c802f000 comp rx=0 tx=0).stop 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81441d0 msgr2=0x7f14d81348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81441d0 0x7f14d81348a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f14c400c760 tx=0x7f14c400cc20 comp rx=0 tx=0).stop 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 shutdown_connections 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b80744e0 0x7f14b80769f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14d8162040 0x7f14d8134de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d81441d0 0x7f14d81348a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 --2- 172.21.15.19:0/2504580638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14d806cd40 0x7f14d815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 >> 172.21.15.19:0/2504580638 conn(0x7f14d8072f40 msgr2=0x7f14d80b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.538+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 shutdown_connections 2024-04-25T11:33:46.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.539+0000 7f14dee00640 1 -- 172.21.15.19:0/2504580638 wait complete. 2024-04-25T11:33:46.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:33:46.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.845+0000 7f0f89c00640 1 Processor -- start 2024-04-25T11:33:46.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.845+0000 7f0f89c00640 1 -- start start 2024-04-25T11:33:46.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.847+0000 7f0f89c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0f840b77d0 0x7f0f84164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.847+0000 7f0f89c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 0x7f0f84160360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.847+0000 7f0f89c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f840cffa0 con 0x7f0f841608a0 2024-04-25T11:33:46.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.847+0000 7f0f89c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0f840b7be0 con 0x7f0f840b77d0 2024-04-25T11:33:46.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.847+0000 7f0f89c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f84165110 con 0x7f0f840d0d60 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.848+0000 7f0f83400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f841608a0 0x7f0f840b2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58712/0 (socket says 172.21.15.19:58712) 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.848+0000 7f0f83400640 1 -- 172.21.15.19:0/2260475275 learned_addr learned my addr 172.21.15.19:0/2260475275 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.848+0000 7f0f82a00640 1 --2- 172.21.15.19:0/2260475275 >> v2:172.21.15.169:3300/0 conn(0x7f0f840b77d0 0x7f0f84164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.848+0000 7f0f82000640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 0x7f0f84160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445987629 0 0) 0x7f0f840cffa0 con 0x7f0f841608a0 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f68003520 con 0x7f0f841608a0 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3316285720 0 0) 0x7f0f68003520 con 0x7f0f841608a0 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 msgr2=0x7f0f84160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 0x7f0f84160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 >> v2:172.21.15.169:3300/0 conn(0x7f0f840b77d0 msgr2=0x7f0f84164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:46.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 --2- 172.21.15.19:0/2260475275 >> v2:172.21.15.169:3300/0 conn(0x7f0f840b77d0 0x7f0f84164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f840d0280 con 0x7f0f841608a0 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f0f7c001670 con 0x7f0f841608a0 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 >> v1:172.21.15.19:6789/0 conn(0x7f0f841608a0 legacy=0x7f0f840b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f680039b0 0x7f0f68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68004800 0x7f0f68004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 0x7f0f68005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f840b7be0 con 0x7f0f680039b0 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84165110 con 0x7f0f68004800 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6800ba40 con 0x7f0f840d0d60 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.849+0000 7f0f82a00640 1 --2- 172.21.15.19:0/2260475275 >> v2:172.21.15.169:3300/0 conn(0x7f0f840b77d0 0x7f0f84164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f82a00640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f680039b0 0x7f0f68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f83400640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 0x7f0f68005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f82000640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68004800 0x7f0f68004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f82a00640 1 -- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 msgr2=0x7f0f68005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f82a00640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f840d0d60 0x7f0f68005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f82a00640 1 -- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68004800 msgr2=0x7f0f68004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f82a00640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f68004800 0x7f0f68004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.850+0000 7f0f82a00640 1 -- 172.21.15.19:0/2260475275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f68003520 con 0x7f0f680039b0 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.851+0000 7f0f82a00640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f680039b0 0x7f0f68003e20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009c70 tx=0x7f0f68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176c3f0ba98db726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.851+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c009840 con 0x7f0f680039b0 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.852+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6c004020 con 0x7f0f680039b0 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.852+0000 7f0f81600640 1 -- 172.21.15.19:0/2260475275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0043d0 con 0x7f0f680039b0 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.852+0000 7f0f89c00640 1 -- 172.21.15.19:0/2260475275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f680039b0 msgr2=0x7f0f68003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.852+0000 7f0f89c00640 1 --2- 172.21.15.19:0/2260475275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f680039b0 0x7f0f68003e20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009c70 tx=0x7f0f68005830 comp rx=0 tx=0).stop 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.852+0000 7f0f83400640 1 -- 172.21.15.19:0/2260475275 reap_dead start 2024-04-25T11:33:46.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.852+0000 7f0f89c00640 1 -- 172.21.15.19:0/2260475275 shutdown_connections 2024-04-25T11:33:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.852+0000 7f0f89c00640 1 -- 172.21.15.19:0/2260475275 >> 172.21.15.19:0/2260475275 conn(0x7f0f8409b7d0 msgr2=0x7f0f840b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.853+0000 7f0f89c00640 1 -- 172.21.15.19:0/2260475275 shutdown_connections 2024-04-25T11:33:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.853+0000 7f0f89c00640 1 -- 172.21.15.19:0/2260475275 wait complete. 2024-04-25T11:33:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.855+0000 7f0f89c00640 1 Processor -- start 2024-04-25T11:33:46.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.855+0000 7f0f89c00640 1 -- start start 2024-04-25T11:33:46.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f89c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680039b0 0x7f0f84163b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f89c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f840d1de0 0x7f0f840d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f89c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84160360 0x7f0f84160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f89c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f840d1170 con 0x7f0f84160360 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f89c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0f840d08d0 con 0x7f0f840d1de0 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f89c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0f840d12f0 con 0x7f0f680039b0 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f83400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84160360 0x7f0f84160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f83400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84160360 0x7f0f84160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33816/0 (socket says 172.21.15.19:33816) 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f83400640 1 -- 172.21.15.19:0/935097623 learned_addr learned my addr 172.21.15.19:0/935097623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f82a00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680039b0 0x7f0f84163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.857+0000 7f0f82000640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f840d1de0 0x7f0f840d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f83400640 1 -- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680039b0 msgr2=0x7f0f84163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f83400640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680039b0 0x7f0f84163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f83400640 1 -- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f840d1de0 msgr2=0x7f0f840d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f83400640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f840d1de0 0x7f0f840d21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f83400640 1 -- 172.21.15.19:0/935097623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f840d2730 con 0x7f0f84160360 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f83400640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84160360 0x7f0f84160770 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0f840d1d80 tx=0x7f0f7c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f7aa00640 1 -- 172.21.15.19:0/935097623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00c610 con 0x7f0f84160360 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f7aa00640 1 -- 172.21.15.19:0/935097623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c018070 con 0x7f0f84160360 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f7aa00640 1 -- 172.21.15.19:0/935097623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00ca30 con 0x7f0f84160360 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f841640b0 con 0x7f0f84160360 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.858+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841f93b0 con 0x7f0f84160360 2024-04-25T11:33:46.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.860+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84164280 con 0x7f0f84160360 2024-04-25T11:33:46.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.861+0000 7f0f7aa00640 1 -- 172.21.15.19:0/935097623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f7c00cbd0 con 0x7f0f84160360 2024-04-25T11:33:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.862+0000 7f0f7aa00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f60074500 0x7f0f60076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:46.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.862+0000 7f0f7aa00640 1 -- 172.21.15.19:0/935097623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f0f7c094ef0 con 0x7f0f84160360 2024-04-25T11:33:46.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.862+0000 7f0f82a00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f60074500 0x7f0f60076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:46.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.864+0000 7f0f82a00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f60074500 0x7f0f60076a10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009c70 tx=0x7f0f6c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:46.867+0000 7f0f7aa00640 1 -- 172.21.15.19:0/935097623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f840d2730 con 0x7f0f84160360 2024-04-25T11:33:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.081+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f840d2e10 con 0x7f0f84160360 2024-04-25T11:33:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.082+0000 7f0f7aa00640 1 -- 172.21.15.19:0/935097623 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) ==== 104+0+11 (secure 0 0 0) 0x7f0f7c05f570 con 0x7f0f84160360 2024-04-25T11:33:47.071 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:33:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f60074500 msgr2=0x7f0f60076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f60074500 0x7f0f60076a10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009c70 tx=0x7f0f6c008030 comp rx=0 tx=0).stop 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84160360 msgr2=0x7f0f84160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84160360 0x7f0f84160770 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0f840d1d80 tx=0x7f0f7c00eb90 comp rx=0 tx=0).stop 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 shutdown_connections 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f60074500 0x7f0f60076a10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0f840d1de0 0x7f0f840d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84160360 0x7f0f84160770 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 --2- 172.21.15.19:0/935097623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0f680039b0 0x7f0f84163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 >> 172.21.15.19:0/935097623 conn(0x7f0f8409b7d0 msgr2=0x7f0f8415f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 shutdown_connections 2024-04-25T11:33:47.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.085+0000 7f0f89c00640 1 -- 172.21.15.19:0/935097623 wait complete. 2024-04-25T11:33:47.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:33:47.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.403+0000 7f150b200640 1 Processor -- start 2024-04-25T11:33:47.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.403+0000 7f150b200640 1 -- start start 2024-04-25T11:33:47.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.405+0000 7f150b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f150c070d20 0x7f150c161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.405+0000 7f150b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06f930 0x7f150c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.405+0000 7f150b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f150c06fd40 0x7f150c06db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.405+0000 7f150b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f150c1623f0 con 0x7f150c06fd40 2024-04-25T11:33:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.405+0000 7f150b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f150c1626d0 con 0x7f150c070d20 2024-04-25T11:33:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.405+0000 7f150b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f150c06f130 con 0x7f150c06f930 2024-04-25T11:33:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.406+0000 7f150a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f150c06fd40 0x7f150c06db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.406+0000 7f150a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f150c06fd40 0x7f150c06db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33824/0 (socket says 172.21.15.19:33824) 2024-04-25T11:33:47.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.406+0000 7f150a800640 1 -- 172.21.15.19:0/3125994533 learned_addr learned my addr 172.21.15.19:0/3125994533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:47.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.406+0000 7f1509e00640 1 --2- 172.21.15.19:0/3125994533 >> v2:172.21.15.169:3300/0 conn(0x7f150c070d20 0x7f150c161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.406+0000 7f1509400640 1 --2- 172.21.15.19:0/3125994533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06f930 0x7f150c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.407+0000 7f150a800640 1 -- 172.21.15.19:0/3125994533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06f930 msgr2=0x7f150c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:47.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.407+0000 7f150a800640 1 --2- 172.21.15.19:0/3125994533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06f930 0x7f150c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.407+0000 7f150a800640 1 -- 172.21.15.19:0/3125994533 >> v2:172.21.15.169:3300/0 conn(0x7f150c070d20 msgr2=0x7f150c161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:47.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.407+0000 7f150a800640 1 --2- 172.21.15.19:0/3125994533 >> v2:172.21.15.169:3300/0 conn(0x7f150c070d20 0x7f150c161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.407+0000 7f150a800640 1 -- 172.21.15.19:0/3125994533 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c06f410 con 0x7f150c06fd40 2024-04-25T11:33:47.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.408+0000 7f150a800640 1 --2- 172.21.15.19:0/3125994533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c06fd40 0x7f150c06db10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f150c06e520 tx=0x7f150402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cec1cd4ef09863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:47.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.408+0000 7f1508a00640 1 -- 172.21.15.19:0/3125994533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504036070 con 0x7f150c06fd40 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.408+0000 7f1508a00640 1 -- 172.21.15.19:0/3125994533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504031600 con 0x7f150c06fd40 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.408+0000 7f1508a00640 1 -- 172.21.15.19:0/3125994533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040319d0 con 0x7f150c06fd40 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 -- 172.21.15.19:0/3125994533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c06fd40 msgr2=0x7f150c06db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 --2- 172.21.15.19:0/3125994533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c06fd40 0x7f150c06db10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f150c06e520 tx=0x7f150402a610 comp rx=0 tx=0).stop 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 -- 172.21.15.19:0/3125994533 shutdown_connections 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 --2- 172.21.15.19:0/3125994533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c06fd40 0x7f150c06db10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 --2- 172.21.15.19:0/3125994533 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06f930 0x7f150c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 --2- 172.21.15.19:0/3125994533 >> v2:172.21.15.169:3300/0 conn(0x7f150c070d20 0x7f150c161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 -- 172.21.15.19:0/3125994533 >> 172.21.15.19:0/3125994533 conn(0x7f150c073160 msgr2=0x7f150c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 -- 172.21.15.19:0/3125994533 shutdown_connections 2024-04-25T11:33:47.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.409+0000 7f150b200640 1 -- 172.21.15.19:0/3125994533 wait complete. 2024-04-25T11:33:47.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.411+0000 7f150b200640 1 Processor -- start 2024-04-25T11:33:47.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.412+0000 7f150b200640 1 -- start start 2024-04-25T11:33:47.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.413+0000 7f150b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c06f930 0x7f150c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.413+0000 7f150b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06fd40 0x7f150c1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.413+0000 7f150b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c070d20 0x7f150c1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.413+0000 7f150b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f150c15e2f0 con 0x7f150c070d20 2024-04-25T11:33:47.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.413+0000 7f150b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f150c162070 con 0x7f150c06f930 2024-04-25T11:33:47.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.413+0000 7f150b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f150c15e470 con 0x7f150c06fd40 2024-04-25T11:33:47.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c070d20 0x7f150c1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c070d20 0x7f150c1552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33834/0 (socket says 172.21.15.19:33834) 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 -- 172.21.15.19:0/3766139781 learned_addr learned my addr 172.21.15.19:0/3766139781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f1509400640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06fd40 0x7f150c1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f1509e00640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c06f930 0x7f150c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 -- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06fd40 msgr2=0x7f150c1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06fd40 0x7f150c1432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 -- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c06f930 msgr2=0x7f150c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c06f930 0x7f150c143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f150a800640 1 -- 172.21.15.19:0/3766139781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150c15e690 con 0x7f150c070d20 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f1509400640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06fd40 0x7f150c1432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.414+0000 7f1509e00640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c06f930 0x7f150c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.415+0000 7f150a800640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c070d20 0x7f150c1552e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1504034040 tx=0x7f1504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.415+0000 7f1502000640 1 -- 172.21.15.19:0/3766139781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504036050 con 0x7f150c070d20 2024-04-25T11:33:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.415+0000 7f1502000640 1 -- 172.21.15.19:0/3766139781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504042070 con 0x7f150c070d20 2024-04-25T11:33:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.415+0000 7f1502000640 1 -- 172.21.15.19:0/3766139781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504009510 con 0x7f150c070d20 2024-04-25T11:33:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.415+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150c15e970 con 0x7f150c070d20 2024-04-25T11:33:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.415+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150c143820 con 0x7f150c070d20 2024-04-25T11:33:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.416+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150c155d40 con 0x7f150c070d20 2024-04-25T11:33:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.417+0000 7f1502000640 1 -- 172.21.15.19:0/3766139781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1504007bd0 con 0x7f150c070d20 2024-04-25T11:33:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.418+0000 7f1502000640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14ec0744b0 0x7f14ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.419+0000 7f1502000640 1 -- 172.21.15.19:0/3766139781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f15040b6270 con 0x7f150c070d20 2024-04-25T11:33:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.419+0000 7f1509e00640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14ec0744b0 0x7f14ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.420+0000 7f1509e00640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14ec0744b0 0x7f14ec0769c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:47.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.423+0000 7f1502000640 1 -- 172.21.15.19:0/3766139781 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1504080870 con 0x7f150c070d20 2024-04-25T11:33:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.626+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f150c074070 con 0x7f150c070d20 2024-04-25T11:33:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.627+0000 7f1502000640 1 -- 172.21.15.19:0/3766139781 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) ==== 100+0+11 (secure 0 0 0) 0x7f1504080170 con 0x7f150c070d20 2024-04-25T11:33:47.615 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14ec0744b0 msgr2=0x7f14ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14ec0744b0 0x7f14ec0769c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc008040 comp rx=0 tx=0).stop 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c070d20 msgr2=0x7f150c1552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c070d20 0x7f150c1552e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1504034040 tx=0x7f1504004a40 comp rx=0 tx=0).stop 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 shutdown_connections 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14ec0744b0 0x7f14ec0769c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f150c070d20 0x7f150c1552e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f150c06fd40 0x7f150c1432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 --2- 172.21.15.19:0/3766139781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f150c06f930 0x7f150c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 >> 172.21.15.19:0/3766139781 conn(0x7f150c073160 msgr2=0x7f150c0aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 shutdown_connections 2024-04-25T11:33:47.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.630+0000 7f150b200640 1 -- 172.21.15.19:0/3766139781 wait complete. 2024-04-25T11:33:47.645 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:33:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.940+0000 7f8143e00640 1 Processor -- start 2024-04-25T11:33:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.941+0000 7f8143e00640 1 -- start start 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f8143e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81440b0800 0x7f8144070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f8143e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8144070be0 con 0x7f81440cf9e0 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f8143e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81440cffc0 con 0x7f81440707d0 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f8143e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f81440d02a0 con 0x7f81440b0800 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f813be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81440b0800 0x7f8144070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f813be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f81440b0800 0x7f8144070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33846/0 (socket says 172.21.15.19:33846) 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f813be00640 1 -- 172.21.15.19:0/3699664363 learned_addr learned my addr 172.21.15.19:0/3699664363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.943+0000 7f8142a00640 1 --1- 172.21.15.19:0/3699664363 >> v1:172.21.15.19:6789/0 conn(0x7f81440cf9e0 0x7f8144161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58718/0 (socket says 172.21.15.19:58718) 2024-04-25T11:33:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.944+0000 7f813be00640 1 -- 172.21.15.19:0/3699664363 >> v1:172.21.15.169:6789/0 conn(0x7f81440707d0 legacy=0x7f81441569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.944+0000 7f813be00640 1 -- 172.21.15.19:0/3699664363 >> v1:172.21.15.19:6789/0 conn(0x7f81440cf9e0 legacy=0x7f8144161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.944+0000 7f813be00640 1 -- 172.21.15.19:0/3699664363 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81440d0580 con 0x7f81440b0800 2024-04-25T11:33:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.945+0000 7f813be00640 1 --2- 172.21.15.19:0/3699664363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440b0800 0x7f8144070290 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f812c009870 tx=0x7f812c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3791ee75bd62ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.945+0000 7f8142000640 1 -- 172.21.15.19:0/3699664363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c037070 con 0x7f81440b0800 2024-04-25T11:33:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.945+0000 7f8142000640 1 -- 172.21.15.19:0/3699664363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812c032440 con 0x7f81440b0800 2024-04-25T11:33:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.945+0000 7f8142000640 1 -- 172.21.15.19:0/3699664363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c032810 con 0x7f81440b0800 2024-04-25T11:33:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.946+0000 7f8143e00640 1 -- 172.21.15.19:0/3699664363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440b0800 msgr2=0x7f8144070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.946+0000 7f8143e00640 1 --2- 172.21.15.19:0/3699664363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440b0800 0x7f8144070290 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f812c009870 tx=0x7f812c02a210 comp rx=0 tx=0).stop 2024-04-25T11:33:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.946+0000 7f8143e00640 1 -- 172.21.15.19:0/3699664363 shutdown_connections 2024-04-25T11:33:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.946+0000 7f8143e00640 1 --2- 172.21.15.19:0/3699664363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440b0800 0x7f8144070290 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.946+0000 7f8143e00640 1 -- 172.21.15.19:0/3699664363 >> 172.21.15.19:0/3699664363 conn(0x7f81440d1720 msgr2=0x7f81440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.947+0000 7f8143e00640 1 -- 172.21.15.19:0/3699664363 shutdown_connections 2024-04-25T11:33:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.947+0000 7f8143e00640 1 -- 172.21.15.19:0/3699664363 wait complete. 2024-04-25T11:33:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.949+0000 7f8143e00640 1 Processor -- start 2024-04-25T11:33:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.949+0000 7f8143e00640 1 -- start start 2024-04-25T11:33:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8143e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440707d0 0x7f8144148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8143e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81440b0800 0x7f814414ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8143e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81440cf9e0 0x7f81441fa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8143e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81440b0d90 con 0x7f81440707d0 2024-04-25T11:33:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8143e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f81440b0c10 con 0x7f81440cf9e0 2024-04-25T11:33:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8143e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f814406d9b0 con 0x7f81440b0800 2024-04-25T11:33:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440707d0 0x7f8144148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440707d0 0x7f8144148f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33854/0 (socket says 172.21.15.19:33854) 2024-04-25T11:33:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 -- 172.21.15.19:0/640792255 learned_addr learned my addr 172.21.15.19:0/640792255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f813be00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81440b0800 0x7f814414ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8143400640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81440cf9e0 0x7f81441fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 -- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81440b0800 msgr2=0x7f814414ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81440b0800 0x7f814414ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 -- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81440cf9e0 msgr2=0x7f81441fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81440cf9e0 0x7f81441fa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.951+0000 7f8142a00640 1 -- 172.21.15.19:0/640792255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81441fa8d0 con 0x7f81440707d0 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.952+0000 7f8142a00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440707d0 0x7f8144148f50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f813400c440 tx=0x7f813400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.952+0000 7f813aa00640 1 -- 172.21.15.19:0/640792255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340074e0 con 0x7f81440707d0 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.952+0000 7f813aa00640 1 -- 172.21.15.19:0/640792255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8134007680 con 0x7f81440707d0 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.952+0000 7f813aa00640 1 -- 172.21.15.19:0/640792255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134004a20 con 0x7f81440707d0 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.952+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81441590f0 con 0x7f81440707d0 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.952+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81441596b0 con 0x7f81440707d0 2024-04-25T11:33:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.953+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8144050740 con 0x7f81440707d0 2024-04-25T11:33:47.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.955+0000 7f813aa00640 1 -- 172.21.15.19:0/640792255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f813401a020 con 0x7f81440707d0 2024-04-25T11:33:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.956+0000 7f813aa00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8120074530 0x7f8120076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.956+0000 7f813aa00640 1 -- 172.21.15.19:0/640792255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f81340950e0 con 0x7f81440707d0 2024-04-25T11:33:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.957+0000 7f813be00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8120074530 0x7f8120076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:47.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.958+0000 7f813be00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8120074530 0x7f8120076a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f812c034000 tx=0x7f812c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:47.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:47.959+0000 7f813aa00640 1 -- 172.21.15.19:0/640792255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8134021070 con 0x7f81440707d0 2024-04-25T11:33:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.201+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f814406fb60 con 0x7f81440707d0 2024-04-25T11:33:48.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.203+0000 7f813aa00640 1 -- 172.21.15.19:0/640792255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7f81441fa8d0 con 0x7f81440707d0 2024-04-25T11:33:48.192 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:48.192 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3770220424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3770220424},{"type":"v1","addr":"172.21.15.19:6833","nonce":3770220424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:32:01.321084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5452":{"gid":5452,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":18,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5452,"qdb_cluster":[5452]},"id":4}]} 2024-04-25T11:33:48.192 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T11:33:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.205+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8120074530 msgr2=0x7f8120076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.205+0000 7f8143e00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8120074530 0x7f8120076a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f812c034000 tx=0x7f812c02e000 comp rx=0 tx=0).stop 2024-04-25T11:33:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.205+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440707d0 msgr2=0x7f8144148f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.205+0000 7f8143e00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440707d0 0x7f8144148f50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f813400c440 tx=0x7f813400c900 comp rx=0 tx=0).stop 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 shutdown_connections 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8120074530 0x7f8120076a40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81440cf9e0 0x7f81441fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81440b0800 0x7f814414ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 --2- 172.21.15.19:0/640792255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81440707d0 0x7f8144148f50 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 >> 172.21.15.19:0/640792255 conn(0x7f81440d1720 msgr2=0x7f81440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 shutdown_connections 2024-04-25T11:33:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.206+0000 7f8143e00640 1 -- 172.21.15.19:0/640792255 wait complete. 2024-04-25T11:33:48.220 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:33:48.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.514+0000 7f9a6a400640 1 Processor -- start 2024-04-25T11:33:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.514+0000 7f9a6a400640 1 -- start start 2024-04-25T11:33:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.516+0000 7f9a6a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 0x7f9a641611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a6a400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a640d1f70 0x7f9a640d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a6a400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a64161720 con 0x7f9a6408cd60 2024-04-25T11:33:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a6a400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9a6406e9d0 con 0x7f9a640d1f70 2024-04-25T11:33:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a6a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6406ddc0 con 0x7f9a640b4ff0 2024-04-25T11:33:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a63e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9a6408cd60 0x7f9a6408d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58726/0 (socket says 172.21.15.19:58726) 2024-04-25T11:33:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a63e00640 1 -- 172.21.15.19:0/3678929553 learned_addr learned my addr 172.21.15.19:0/3678929553 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a69000640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 0x7f9a641611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a69a00640 1 --2- 172.21.15.19:0/3678929553 >> v2:172.21.15.169:3300/0 conn(0x7f9a640d1f70 0x7f9a640d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.517+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660343435 0 0) 0x7f9a50001110 con 0x7f9a6408cd60 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a48003520 con 0x7f9a6408cd60 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1480158719 0 0) 0x7f9a48003520 con 0x7f9a6408cd60 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 msgr2=0x7f9a641611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 0x7f9a641611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 >> v2:172.21.15.169:3300/0 conn(0x7f9a640d1f70 msgr2=0x7f9a640d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 --2- 172.21.15.19:0/3678929553 >> v2:172.21.15.169:3300/0 conn(0x7f9a640d1f70 0x7f9a640d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6406e0a0 con 0x7f9a6408cd60 2024-04-25T11:33:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9a50001840 con 0x7f9a6408cd60 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 >> v1:172.21.15.19:6789/0 conn(0x7f9a6408cd60 legacy=0x7f9a6408d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 0x7f9a480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48003f30 0x7f9a48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48004d40 0x7f9a48005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6406ddc0 con 0x7f9a48003f30 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4800ba70 con 0x7f9a48004d40 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.518+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4800bc40 con 0x7f9a640b4ff0 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a63e00640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48003f30 0x7f9a48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a69000640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 0x7f9a480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a69a00640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48004d40 0x7f9a48005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a63e00640 1 -- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 msgr2=0x7f9a480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a63e00640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a640b4ff0 0x7f9a480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a63e00640 1 -- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48004d40 msgr2=0x7f9a48005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a63e00640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48004d40 0x7f9a48005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.519+0000 7f9a63e00640 1 -- 172.21.15.19:0/3678929553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a48003520 con 0x7f9a48003f30 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.520+0000 7f9a63e00640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48003f30 0x7f9a48004800 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c3e0 tx=0x7f9a48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8accf77b029915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.520+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50032020 con 0x7f9a48003f30 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.520+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a50005af0 con 0x7f9a48003f30 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.520+0000 7f9a63400640 1 -- 172.21.15.19:0/3678929553 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000c0e0 con 0x7f9a48003f30 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.521+0000 7f9a6a400640 1 -- 172.21.15.19:0/3678929553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48003f30 msgr2=0x7f9a48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.521+0000 7f9a6a400640 1 --2- 172.21.15.19:0/3678929553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48003f30 0x7f9a48004800 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c3e0 tx=0x7f9a48005830 comp rx=0 tx=0).stop 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.521+0000 7f9a69a00640 1 -- 172.21.15.19:0/3678929553 reap_dead start 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.521+0000 7f9a6a400640 1 -- 172.21.15.19:0/3678929553 shutdown_connections 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.521+0000 7f9a6a400640 1 -- 172.21.15.19:0/3678929553 >> 172.21.15.19:0/3678929553 conn(0x7f9a640b5460 msgr2=0x7f9a640b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.521+0000 7f9a6a400640 1 -- 172.21.15.19:0/3678929553 shutdown_connections 2024-04-25T11:33:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.521+0000 7f9a6a400640 1 -- 172.21.15.19:0/3678929553 wait complete. 2024-04-25T11:33:48.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.524+0000 7f9a6a400640 1 Processor -- start 2024-04-25T11:33:48.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.524+0000 7f9a6a400640 1 -- start start 2024-04-25T11:33:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.525+0000 7f9a6a400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48003f30 0x7f9a6414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.525+0000 7f9a6a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a6409fbc0 0x7f9a6409ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.525+0000 7f9a6a400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a6415ef60 0x7f9a6415f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.525+0000 7f9a6a400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a640b9180 con 0x7f9a6409fbc0 2024-04-25T11:33:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.525+0000 7f9a6a400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6406e660 con 0x7f9a48003f30 2024-04-25T11:33:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.525+0000 7f9a6a400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a6406da20 con 0x7f9a6415ef60 2024-04-25T11:33:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a6409fbc0 0x7f9a6409ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a6409fbc0 0x7f9a6409ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33874/0 (socket says 172.21.15.19:33874) 2024-04-25T11:33:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 -- 172.21.15.19:0/1839160673 learned_addr learned my addr 172.21.15.19:0/1839160673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a69000640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48003f30 0x7f9a6414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a69a00640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a6415ef60 0x7f9a6415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 -- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a6415ef60 msgr2=0x7f9a6415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a6415ef60 0x7f9a6415f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 -- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48003f30 msgr2=0x7f9a6414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48003f30 0x7f9a6414abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.526+0000 7f9a63e00640 1 -- 172.21.15.19:0/1839160673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6415f8b0 con 0x7f9a6409fbc0 2024-04-25T11:33:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.527+0000 7f9a63e00640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a6409fbc0 0x7f9a6409ffd0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c3e0 tx=0x7f9a50001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.527+0000 7f9a60c00640 1 -- 172.21.15.19:0/1839160673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50032030 con 0x7f9a6409fbc0 2024-04-25T11:33:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.527+0000 7f9a60c00640 1 -- 172.21.15.19:0/1839160673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a50007270 con 0x7f9a6409fbc0 2024-04-25T11:33:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.527+0000 7f9a60c00640 1 -- 172.21.15.19:0/1839160673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50007620 con 0x7f9a6409fbc0 2024-04-25T11:33:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.527+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6414b7f0 con 0x7f9a6409fbc0 2024-04-25T11:33:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.527+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a6414bdb0 con 0x7f9a6409fbc0 2024-04-25T11:33:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.528+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a640b4bf0 con 0x7f9a6409fbc0 2024-04-25T11:33:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.530+0000 7f9a60c00640 1 -- 172.21.15.19:0/1839160673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9a50035020 con 0x7f9a6409fbc0 2024-04-25T11:33:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.530+0000 7f9a60c00640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a440744e0 0x7f9a440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.531+0000 7f9a60c00640 1 -- 172.21.15.19:0/1839160673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f9a500b56b0 con 0x7f9a6409fbc0 2024-04-25T11:33:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.531+0000 7f9a69000640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a440744e0 0x7f9a440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:48.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.533+0000 7f9a69000640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a440744e0 0x7f9a440769f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a64164e10 tx=0x7f9a54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.536+0000 7f9a60c00640 1 -- 172.21.15.19:0/1839160673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a640b4bf0 con 0x7f9a6409fbc0 2024-04-25T11:33:48.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.734+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a6406f350 con 0x7f9a6409fbc0 2024-04-25T11:33:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.737+0000 7f9a60c00640 1 -- 172.21.15.19:0/1839160673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17018 (secure 0 0 0) 0x7f9a6415f8b0 con 0x7f9a6409fbc0 2024-04-25T11:33:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:48.726 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":67,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:32:07.391254+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.36","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6833/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6833/751039234":"2024-04-26T11:31:25.310192+0000","172.21.15.19:6832/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6832/751039234":"2024-04-26T11:31:25.310192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a440744e0 msgr2=0x7f9a440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a440744e0 0x7f9a440769f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9a64164e10 tx=0x7f9a54008040 comp rx=0 tx=0).stop 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a6409fbc0 msgr2=0x7f9a6409ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a6409fbc0 0x7f9a6409ffd0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000c3e0 tx=0x7f9a50001c10 comp rx=0 tx=0).stop 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 shutdown_connections 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a440744e0 0x7f9a440769f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a6415ef60 0x7f9a6415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a6409fbc0 0x7f9a6409ffd0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 --2- 172.21.15.19:0/1839160673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a48003f30 0x7f9a6414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 >> 172.21.15.19:0/1839160673 conn(0x7f9a640b5460 msgr2=0x7f9a640b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 shutdown_connections 2024-04-25T11:33:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:48.740+0000 7f9a6a400640 1 -- 172.21.15.19:0/1839160673 wait complete. 2024-04-25T11:33:48.753 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:33:48.753 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:33:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.044+0000 7f5467200640 1 Processor -- start 2024-04-25T11:33:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.044+0000 7f5467200640 1 -- start start 2024-04-25T11:33:49.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5467200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f546806c5c0 0x7f54680cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5467200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54680b4ea0 0x7f546815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5467200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54680d1230 con 0x7f5468144370 2024-04-25T11:33:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5467200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f54680d06b0 con 0x7f54680b4ea0 2024-04-25T11:33:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5467200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f54680d0990 con 0x7f546806c5c0 2024-04-25T11:33:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5466800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54680b4ea0 0x7f546815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5465400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5468144370 0x7f54680b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58732/0 (socket says 172.21.15.19:58732) 2024-04-25T11:33:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.047+0000 7f5465400640 1 -- 172.21.15.19:0/3037766522 learned_addr learned my addr 172.21.15.19:0/3037766522 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:49.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.048+0000 7f5466800640 1 -- 172.21.15.19:0/3037766522 >> v2:172.21.15.169:3300/0 conn(0x7f546806c5c0 msgr2=0x7f54680cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.048+0000 7f5466800640 1 --2- 172.21.15.19:0/3037766522 >> v2:172.21.15.169:3300/0 conn(0x7f546806c5c0 0x7f54680cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.048+0000 7f5466800640 1 -- 172.21.15.19:0/3037766522 >> v1:172.21.15.19:6789/0 conn(0x7f5468144370 legacy=0x7f54680b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.048+0000 7f5466800640 1 -- 172.21.15.19:0/3037766522 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54680d0c70 con 0x7f54680b4ea0 2024-04-25T11:33:49.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.049+0000 7f5466800640 1 --2- 172.21.15.19:0/3037766522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54680b4ea0 0x7f546815e770 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f54680726b0 tx=0x7f546002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a1dcd660cca3d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:49.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.049+0000 7f5464a00640 1 -- 172.21.15.19:0/3037766522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460007740 con 0x7f54680b4ea0 2024-04-25T11:33:49.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.049+0000 7f5464a00640 1 -- 172.21.15.19:0/3037766522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54600078e0 con 0x7f54680b4ea0 2024-04-25T11:33:49.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.049+0000 7f5464a00640 1 -- 172.21.15.19:0/3037766522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460007cb0 con 0x7f54680b4ea0 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 -- 172.21.15.19:0/3037766522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54680b4ea0 msgr2=0x7f546815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 --2- 172.21.15.19:0/3037766522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54680b4ea0 0x7f546815e770 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f54680726b0 tx=0x7f546002a680 comp rx=0 tx=0).stop 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 -- 172.21.15.19:0/3037766522 shutdown_connections 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 --2- 172.21.15.19:0/3037766522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54680b4ea0 0x7f546815e770 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 --2- 172.21.15.19:0/3037766522 >> v2:172.21.15.169:3300/0 conn(0x7f546806c5c0 0x7f54680cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 -- 172.21.15.19:0/3037766522 >> 172.21.15.19:0/3037766522 conn(0x7f54680d1720 msgr2=0x7f54680d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 -- 172.21.15.19:0/3037766522 shutdown_connections 2024-04-25T11:33:49.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.050+0000 7f5467200640 1 -- 172.21.15.19:0/3037766522 wait complete. 2024-04-25T11:33:49.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.052+0000 7f5467200640 1 Processor -- start 2024-04-25T11:33:49.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.052+0000 7f5467200640 1 -- start start 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5467200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f546806c5c0 0x7f5468146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5467200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468144370 0x7f5468161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5467200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f546815f910 0x7f546815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5467200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54680d0450 con 0x7f5468144370 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5467200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f54680d02d0 con 0x7f546806c5c0 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5467200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5468147760 con 0x7f546815f910 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468144370 0x7f5468161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468144370 0x7f5468161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33906/0 (socket says 172.21.15.19:33906) 2024-04-25T11:33:49.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465400640 1 -- 172.21.15.19:0/2309540417 learned_addr learned my addr 172.21.15.19:0/2309540417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465400640 1 -- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f546815f910 msgr2=0x7f546815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465400640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f546815f910 0x7f546815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465400640 1 -- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f546806c5c0 msgr2=0x7f5468146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465e00640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f546806c5c0 0x7f5468146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.054+0000 7f5465400640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f546806c5c0 0x7f5468146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.055+0000 7f5465400640 1 -- 172.21.15.19:0/2309540417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5468161660 con 0x7f5468144370 2024-04-25T11:33:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.055+0000 7f5465400640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468144370 0x7f5468161120 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f545400c3c0 tx=0x7f545400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.055+0000 7f545e000640 1 -- 172.21.15.19:0/2309540417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454017070 con 0x7f5468144370 2024-04-25T11:33:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.055+0000 7f545e000640 1 -- 172.21.15.19:0/2309540417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5454005ce0 con 0x7f5468144370 2024-04-25T11:33:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.055+0000 7f545e000640 1 -- 172.21.15.19:0/2309540417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54540072f0 con 0x7f5468144370 2024-04-25T11:33:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.055+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546815d890 con 0x7f5468144370 2024-04-25T11:33:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.055+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5468160360 con 0x7f5468144370 2024-04-25T11:33:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.056+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546815fdd0 con 0x7f5468144370 2024-04-25T11:33:49.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.058+0000 7f545e000640 1 -- 172.21.15.19:0/2309540417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f545401a070 con 0x7f5468144370 2024-04-25T11:33:49.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.059+0000 7f545e000640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f54480744b0 0x7f54480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.059+0000 7f545e000640 1 -- 172.21.15.19:0/2309540417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f5454094790 con 0x7f5468144370 2024-04-25T11:33:49.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.059+0000 7f5465e00640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f54480744b0 0x7f54480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.060+0000 7f5465e00640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f54480744b0 0x7f54480769c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f545800a770 tx=0x7f5458008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:49.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.064+0000 7f545e000640 1 -- 172.21.15.19:0/2309540417 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5468161660 con 0x7f5468144370 2024-04-25T11:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.299+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5468050740 con 0x7f5468144370 2024-04-25T11:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.300+0000 7f545e000640 1 -- 172.21.15.19:0/2309540417 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7f545405ed90 con 0x7f5468144370 2024-04-25T11:33:49.288 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:49.289 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3770220424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3770220424},{"type":"v1","addr":"172.21.15.19:6833","nonce":3770220424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:32:01.321084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5452":{"gid":5452,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":18,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5452,"qdb_cluster":[5452]},"id":4}]} 2024-04-25T11:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T11:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.302+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f54480744b0 msgr2=0x7f54480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.302+0000 7f5467200640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f54480744b0 0x7f54480769c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f545800a770 tx=0x7f5458008040 comp rx=0 tx=0).stop 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468144370 msgr2=0x7f5468161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468144370 0x7f5468161120 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f545400c3c0 tx=0x7f545400c880 comp rx=0 tx=0).stop 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 shutdown_connections 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f54480744b0 0x7f54480769c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f546815f910 0x7f546815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5468144370 0x7f5468161120 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 --2- 172.21.15.19:0/2309540417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f546806c5c0 0x7f5468146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 >> 172.21.15.19:0/2309540417 conn(0x7f54680d1720 msgr2=0x7f546806d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 shutdown_connections 2024-04-25T11:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.303+0000 7f5467200640 1 -- 172.21.15.19:0/2309540417 wait complete. 2024-04-25T11:33:49.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:33:49.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.611+0000 7fda9ea00640 1 Processor -- start 2024-04-25T11:33:49.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.611+0000 7fda9ea00640 1 -- start start 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.613+0000 7fda9ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda980b5420 0x7fda980700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.613+0000 7fda9ea00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda9806f7f0 0x7fda98163db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.613+0000 7fda9ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda98160840 con 0x7fda98070620 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.613+0000 7fda9ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fda98160ab0 con 0x7fda980b5420 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.613+0000 7fda9ea00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fda98160d90 con 0x7fda9806f7f0 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.614+0000 7fda9d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda980b5420 0x7fda980700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.614+0000 7fda9d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda980b5420 0x7fda980700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33916/0 (socket says 172.21.15.19:33916) 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.614+0000 7fda9d600640 1 -- 172.21.15.19:0/3096067529 learned_addr learned my addr 172.21.15.19:0/3096067529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.614+0000 7fda9e000640 1 --2- 172.21.15.19:0/3096067529 >> v2:172.21.15.169:3300/0 conn(0x7fda9806f7f0 0x7fda98163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.614+0000 7fda9d600640 1 -- 172.21.15.19:0/3096067529 >> v2:172.21.15.169:3300/0 conn(0x7fda9806f7f0 msgr2=0x7fda98163db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.614+0000 7fda9d600640 1 --2- 172.21.15.19:0/3096067529 >> v2:172.21.15.169:3300/0 conn(0x7fda9806f7f0 0x7fda98163db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.614+0000 7fda9d600640 1 -- 172.21.15.19:0/3096067529 >> v1:172.21.15.169:6789/0 conn(0x7fda98070620 legacy=0x7fda9806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:49.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.615+0000 7fda9d600640 1 -- 172.21.15.19:0/3096067529 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda98161070 con 0x7fda980b5420 2024-04-25T11:33:49.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.616+0000 7fda9d600640 1 --2- 172.21.15.19:0/3096067529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda980b5420 0x7fda980700e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fda980b13b0 tx=0x7fda8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31716ad95476e563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.617+0000 7fda97e00640 1 -- 172.21.15.19:0/3096067529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c037070 con 0x7fda980b5420 2024-04-25T11:33:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.617+0000 7fda97e00640 1 -- 172.21.15.19:0/3096067529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8c032440 con 0x7fda980b5420 2024-04-25T11:33:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.617+0000 7fda97e00640 1 -- 172.21.15.19:0/3096067529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c032810 con 0x7fda980b5420 2024-04-25T11:33:49.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 -- 172.21.15.19:0/3096067529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda980b5420 msgr2=0x7fda980700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3096067529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda980b5420 0x7fda980700e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fda980b13b0 tx=0x7fda8c02a230 comp rx=0 tx=0).stop 2024-04-25T11:33:49.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 -- 172.21.15.19:0/3096067529 shutdown_connections 2024-04-25T11:33:49.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3096067529 >> v2:172.21.15.169:3300/0 conn(0x7fda9806f7f0 0x7fda98163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3096067529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda980b5420 0x7fda980700e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 -- 172.21.15.19:0/3096067529 >> 172.21.15.19:0/3096067529 conn(0x7fda980d1720 msgr2=0x7fda980d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:49.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 -- 172.21.15.19:0/3096067529 shutdown_connections 2024-04-25T11:33:49.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.618+0000 7fda9ea00640 1 -- 172.21.15.19:0/3096067529 wait complete. 2024-04-25T11:33:49.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.620+0000 7fda9ea00640 1 Processor -- start 2024-04-25T11:33:49.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.621+0000 7fda9ea00640 1 -- start start 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda98070620 0x7fda980cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda980b5420 0x7fda981532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981e9b40 0x7fda98154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda980b0f60 con 0x7fda98070620 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda980b0de0 con 0x7fda980b5420 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda980b4790 con 0x7fda981e9b40 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda98070620 0x7fda980cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda98070620 0x7fda980cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33928/0 (socket says 172.21.15.19:33928) 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.622+0000 7fda9d600640 1 -- 172.21.15.19:0/3217764009 learned_addr learned my addr 172.21.15.19:0/3217764009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9cc00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda980b5420 0x7fda981532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9e000640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981e9b40 0x7fda98154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9d600640 1 -- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981e9b40 msgr2=0x7fda98154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9d600640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981e9b40 0x7fda98154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9d600640 1 -- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda980b5420 msgr2=0x7fda981532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9d600640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda980b5420 0x7fda981532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9d600640 1 -- 172.21.15.19:0/3217764009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda98153dd0 con 0x7fda98070620 2024-04-25T11:33:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9cc00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda980b5420 0x7fda981532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.623+0000 7fda9d600640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda98070620 0x7fda980cfb80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fda8c02a270 tx=0x7fda8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:49.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.624+0000 7fda95600640 1 -- 172.21.15.19:0/3217764009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c041020 con 0x7fda98070620 2024-04-25T11:33:49.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.624+0000 7fda95600640 1 -- 172.21.15.19:0/3217764009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8c0306c0 con 0x7fda98070620 2024-04-25T11:33:49.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.624+0000 7fda95600640 1 -- 172.21.15.19:0/3217764009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c037040 con 0x7fda98070620 2024-04-25T11:33:49.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.624+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda981540b0 con 0x7fda98070620 2024-04-25T11:33:49.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.624+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda98154670 con 0x7fda98070620 2024-04-25T11:33:49.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.625+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda98050740 con 0x7fda98070620 2024-04-25T11:33:49.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.626+0000 7fda95600640 1 -- 172.21.15.19:0/3217764009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda8c044020 con 0x7fda98070620 2024-04-25T11:33:49.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.628+0000 7fda95600640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda780744b0 0x7fda780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:49.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.628+0000 7fda95600640 1 -- 172.21.15.19:0/3217764009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7fda8c0b5c10 con 0x7fda98070620 2024-04-25T11:33:49.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.628+0000 7fda9cc00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda780744b0 0x7fda780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:49.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.630+0000 7fda9cc00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda780744b0 0x7fda780769c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fda98164850 tx=0x7fda84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:49.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.633+0000 7fda95600640 1 -- 172.21.15.19:0/3217764009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda8c04b030 con 0x7fda98070620 2024-04-25T11:33:49.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.880+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda980703a0 con 0x7fda98070620 2024-04-25T11:33:49.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.882+0000 7fda95600640 1 -- 172.21.15.19:0/3217764009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7fda8c0802a0 con 0x7fda98070620 2024-04-25T11:33:49.871 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:49.871 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3770220424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3770220424},{"type":"v1","addr":"172.21.15.19:6833","nonce":3770220424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:31:06.789413+0000","modified":"2024-04-25T11:32:01.321084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5452":{"gid":5452,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":18,"addr":"172.21.15.169:6837/1885933205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1885933205},{"type":"v1","addr":"172.21.15.169:6837","nonce":1885933205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5452,"qdb_cluster":[5452]},"id":4}]} 2024-04-25T11:33:49.871 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 65 2024-04-25T11:33:49.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda780744b0 msgr2=0x7fda780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda780744b0 0x7fda780769c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fda98164850 tx=0x7fda84008040 comp rx=0 tx=0).stop 2024-04-25T11:33:49.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda98070620 msgr2=0x7fda980cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:49.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda98070620 0x7fda980cfb80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fda8c02a270 tx=0x7fda8c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 shutdown_connections 2024-04-25T11:33:49.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda780744b0 0x7fda780769c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda980b5420 0x7fda981532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda981e9b40 0x7fda98154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 --2- 172.21.15.19:0/3217764009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda98070620 0x7fda980cfb80 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:49.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 >> 172.21.15.19:0/3217764009 conn(0x7fda980d1720 msgr2=0x7fda9806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:49.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 shutdown_connections 2024-04-25T11:33:49.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:49.885+0000 7fda9ea00640 1 -- 172.21.15.19:0/3217764009 wait complete. 2024-04-25T11:33:49.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:33:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.197+0000 7f589f200640 1 Processor -- start 2024-04-25T11:33:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.197+0000 7f589f200640 1 -- start start 2024-04-25T11:33:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.199+0000 7f589f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f58a00d50e0 0x7f58a00d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.199+0000 7f589f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58a0160d60 0x7f58a0161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.199+0000 7f589f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a00b7940 con 0x7f58a00d2ae0 2024-04-25T11:33:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.200+0000 7f589f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f58a00d2ef0 con 0x7f58a0160d60 2024-04-25T11:33:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.200+0000 7f589f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f58a0161280 con 0x7f58a00d50e0 2024-04-25T11:33:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.200+0000 7f589e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58a0160d60 0x7f58a0161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.200+0000 7f589e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58a0160d60 0x7f58a0161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33932/0 (socket says 172.21.15.19:33932) 2024-04-25T11:33:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.200+0000 7f589e800640 1 -- 172.21.15.19:0/4042462440 learned_addr learned my addr 172.21.15.19:0/4042462440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.200+0000 7f589de00640 1 --2- 172.21.15.19:0/4042462440 >> v2:172.21.15.169:3300/0 conn(0x7f58a00d50e0 0x7f58a00d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.201+0000 7f589e800640 1 -- 172.21.15.19:0/4042462440 >> v2:172.21.15.169:3300/0 conn(0x7f58a00d50e0 msgr2=0x7f58a00d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.201+0000 7f589e800640 1 --2- 172.21.15.19:0/4042462440 >> v2:172.21.15.169:3300/0 conn(0x7f58a00d50e0 0x7f58a00d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.201+0000 7f589e800640 1 -- 172.21.15.19:0/4042462440 >> v1:172.21.15.169:6789/0 conn(0x7f58a00d2ae0 legacy=0x7f58a0160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.201+0000 7f589e800640 1 -- 172.21.15.19:0/4042462440 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a00d4600 con 0x7f58a0160d60 2024-04-25T11:33:50.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.202+0000 7f589e800640 1 --2- 172.21.15.19:0/4042462440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a0160d60 0x7f58a0161810 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f58a00d6cc0 tx=0x7f589802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2518c6086a409f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.202+0000 7f589d400640 1 -- 172.21.15.19:0/4042462440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898036070 con 0x7f58a0160d60 2024-04-25T11:33:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589d400640 1 -- 172.21.15.19:0/4042462440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5898031600 con 0x7f58a0160d60 2024-04-25T11:33:50.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589d400640 1 -- 172.21.15.19:0/4042462440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58980319d0 con 0x7f58a0160d60 2024-04-25T11:33:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 -- 172.21.15.19:0/4042462440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a0160d60 msgr2=0x7f58a0161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 --2- 172.21.15.19:0/4042462440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a0160d60 0x7f58a0161810 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f58a00d6cc0 tx=0x7f589802a730 comp rx=0 tx=0).stop 2024-04-25T11:33:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 -- 172.21.15.19:0/4042462440 shutdown_connections 2024-04-25T11:33:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 --2- 172.21.15.19:0/4042462440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a0160d60 0x7f58a0161810 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 --2- 172.21.15.19:0/4042462440 >> v2:172.21.15.169:3300/0 conn(0x7f58a00d50e0 0x7f58a00d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 -- 172.21.15.19:0/4042462440 >> 172.21.15.19:0/4042462440 conn(0x7f58a006c5c0 msgr2=0x7f58a006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 -- 172.21.15.19:0/4042462440 shutdown_connections 2024-04-25T11:33:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.203+0000 7f589f200640 1 -- 172.21.15.19:0/4042462440 wait complete. 2024-04-25T11:33:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.206+0000 7f589f200640 1 Processor -- start 2024-04-25T11:33:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.206+0000 7f589f200640 1 -- start start 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58a00d2ae0 0x7f58a014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00d50e0 0x7f58a00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a0160d60 0x7f58a0162e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0161560 con 0x7f58a00d50e0 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00d4bc0 con 0x7f58a0160d60 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f58a00d54f0 con 0x7f58a00d2ae0 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f5897e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00d50e0 0x7f58a00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f5897e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00d50e0 0x7f58a00c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33934/0 (socket says 172.21.15.19:33934) 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f5897e00640 1 -- 172.21.15.19:0/843268721 learned_addr learned my addr 172.21.15.19:0/843268721 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589e800640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a0160d60 0x7f58a0162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.208+0000 7f589de00640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58a00d2ae0 0x7f58a014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5897e00640 1 -- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58a00d2ae0 msgr2=0x7f58a014b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5897e00640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58a00d2ae0 0x7f58a014b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5897e00640 1 -- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a0160d60 msgr2=0x7f58a0162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5897e00640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a0160d60 0x7f58a0162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5897e00640 1 -- 172.21.15.19:0/843268721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a014ba50 con 0x7f58a00d50e0 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5897e00640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00d50e0 0x7f58a00c8430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f588800c3c0 tx=0x7f588800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5896000640 1 -- 172.21.15.19:0/843268721 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888017070 con 0x7f58a00d50e0 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5896000640 1 -- 172.21.15.19:0/843268721 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58880052f0 con 0x7f58a00d50e0 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f5896000640 1 -- 172.21.15.19:0/843268721 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888003830 con 0x7f58a00d50e0 2024-04-25T11:33:50.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a00c8970 con 0x7f58a00d50e0 2024-04-25T11:33:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.209+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a00ca090 con 0x7f58a00d50e0 2024-04-25T11:33:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.210+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a0050740 con 0x7f58a00d50e0 2024-04-25T11:33:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.212+0000 7f5896000640 1 -- 172.21.15.19:0/843268721 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f588801a070 con 0x7f58a00d50e0 2024-04-25T11:33:50.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.213+0000 7f5896000640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f587c0744b0 0x7f587c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.213+0000 7f5896000640 1 -- 172.21.15.19:0/843268721 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7f58880946c0 con 0x7f58a00d50e0 2024-04-25T11:33:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.214+0000 7f589de00640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f587c0744b0 0x7f587c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.215+0000 7f589de00640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f587c0744b0 0x7f587c0769c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5890009a30 tx=0x7f5890008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:50.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.218+0000 7f5896000640 1 -- 172.21.15.19:0/843268721 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58a00b7940 con 0x7f58a00d50e0 2024-04-25T11:33:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.421+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58a01601f0 con 0x7f58a00d50e0 2024-04-25T11:33:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.424+0000 7f5896000640 1 -- 172.21.15.19:0/843268721 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+17018 (secure 0 0 0) 0x7f588805ecc0 con 0x7f58a00d50e0 2024-04-25T11:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":67,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:32:07.391254+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:31:05.039941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T11:31:05.784246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.36","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6833/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6833/751039234":"2024-04-26T11:31:25.310192+0000","172.21.15.19:6832/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6832/751039234":"2024-04-26T11:31:25.310192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:33:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f587c0744b0 msgr2=0x7f587c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f587c0744b0 0x7f587c0769c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5890009a30 tx=0x7f5890008040 comp rx=0 tx=0).stop 2024-04-25T11:33:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00d50e0 msgr2=0x7f58a00c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00d50e0 0x7f58a00c8430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f588800c3c0 tx=0x7f588800c880 comp rx=0 tx=0).stop 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 shutdown_connections 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f587c0744b0 0x7f587c0769c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a0160d60 0x7f58a0162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58a00d50e0 0x7f58a00c8430 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 --2- 172.21.15.19:0/843268721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58a00d2ae0 0x7f58a014b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 >> 172.21.15.19:0/843268721 conn(0x7f58a006c5c0 msgr2=0x7f58a00b9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 shutdown_connections 2024-04-25T11:33:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.427+0000 7f589f200640 1 -- 172.21.15.19:0/843268721 wait complete. 2024-04-25T11:33:50.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:33:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.732+0000 7fe8aaa00640 1 Processor -- start 2024-04-25T11:33:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.733+0000 7fe8aaa00640 1 -- start start 2024-04-25T11:33:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.735+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a40b4ea0 0x7fe8a40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.735+0000 7fe8aaa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe8a4144370 0x7fe8a415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.735+0000 7fe8aaa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a40d1230 con 0x7fe8a406c5c0 2024-04-25T11:33:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.735+0000 7fe8aaa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a40d06b0 con 0x7fe8a4144370 2024-04-25T11:33:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.735+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a40d0990 con 0x7fe8a40b4ea0 2024-04-25T11:33:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.735+0000 7fe8aa000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe8a4144370 0x7fe8a415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.735+0000 7fe8aa000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe8a4144370 0x7fe8a415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33946/0 (socket says 172.21.15.19:33946) 2024-04-25T11:33:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8aa000640 1 -- 172.21.15.19:0/2269555370 learned_addr learned my addr 172.21.15.19:0/2269555370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8a9600640 1 --1- 172.21.15.19:0/2269555370 >> v1:172.21.15.19:6789/0 conn(0x7fe8a406c5c0 0x7fe8a40b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58740/0 (socket says 172.21.15.19:58740) 2024-04-25T11:33:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8a8c00640 1 --2- 172.21.15.19:0/2269555370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a40b4ea0 0x7fe8a40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8aa000640 1 -- 172.21.15.19:0/2269555370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a40b4ea0 msgr2=0x7fe8a40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8aa000640 1 --2- 172.21.15.19:0/2269555370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a40b4ea0 0x7fe8a40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8aa000640 1 -- 172.21.15.19:0/2269555370 >> v1:172.21.15.19:6789/0 conn(0x7fe8a406c5c0 legacy=0x7fe8a40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8aa000640 1 -- 172.21.15.19:0/2269555370 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a40d0c70 con 0x7fe8a4144370 2024-04-25T11:33:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.736+0000 7fe8a3e00640 1 -- 172.21.15.19:0/2269555370 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1499171803 0 0) 0x7fe8a40d1230 con 0x7fe8a406c5c0 2024-04-25T11:33:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.737+0000 7fe8aa000640 1 --2- 172.21.15.19:0/2269555370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a415e770 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009e50 tx=0x7fe89c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd1a8d7bc052efb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8a3e00640 1 -- 172.21.15.19:0/2269555370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c036070 con 0x7fe8a4144370 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8a3e00640 1 -- 172.21.15.19:0/2269555370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe89c031600 con 0x7fe8a4144370 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8a3e00640 1 -- 172.21.15.19:0/2269555370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0319d0 con 0x7fe8a4144370 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8aaa00640 1 -- 172.21.15.19:0/2269555370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 msgr2=0x7fe8a415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/2269555370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a415e770 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009e50 tx=0x7fe89c02a710 comp rx=0 tx=0).stop 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8aaa00640 1 -- 172.21.15.19:0/2269555370 shutdown_connections 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/2269555370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a415e770 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/2269555370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a40b4ea0 0x7fe8a40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.738+0000 7fe8aaa00640 1 -- 172.21.15.19:0/2269555370 >> 172.21.15.19:0/2269555370 conn(0x7fe8a40d1720 msgr2=0x7fe8a40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.739+0000 7fe8aaa00640 1 -- 172.21.15.19:0/2269555370 shutdown_connections 2024-04-25T11:33:50.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.739+0000 7fe8aaa00640 1 -- 172.21.15.19:0/2269555370 wait complete. 2024-04-25T11:33:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.741+0000 7fe8aaa00640 1 Processor -- start 2024-04-25T11:33:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.741+0000 7fe8aaa00640 1 -- start start 2024-04-25T11:33:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a406c5c0 0x7fe8a4135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a4070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415d440 0x7fe8a4160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a40d0450 con 0x7fe8a4144370 2024-04-25T11:33:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a40d02d0 con 0x7fe8a415d440 2024-04-25T11:33:50.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a4147760 con 0x7fe8a406c5c0 2024-04-25T11:33:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8a8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a4070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8a8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a4070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33954/0 (socket says 172.21.15.19:33954) 2024-04-25T11:33:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.743+0000 7fe8a8c00640 1 -- 172.21.15.19:0/69287766 learned_addr learned my addr 172.21.15.19:0/69287766 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8aa000640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415d440 0x7fe8a4160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a9600640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a406c5c0 0x7fe8a4135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a8c00640 1 -- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a406c5c0 msgr2=0x7fe8a4135ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a8c00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a406c5c0 0x7fe8a4135ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a8c00640 1 -- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415d440 msgr2=0x7fe8a4160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a8c00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415d440 0x7fe8a4160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a8c00640 1 -- 172.21.15.19:0/69287766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a4071120 con 0x7fe8a4144370 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a8c00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a4070be0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe89000d620 tx=0x7fe89000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.744+0000 7fe8a1600640 1 -- 172.21.15.19:0/69287766 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000b620 con 0x7fe8a4144370 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.745+0000 7fe8a1600640 1 -- 172.21.15.19:0/69287766 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe890019070 con 0x7fe8a4144370 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.745+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a41624c0 con 0x7fe8a4144370 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.745+0000 7fe8a1600640 1 -- 172.21.15.19:0/69287766 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000ba00 con 0x7fe8a4144370 2024-04-25T11:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.745+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a4145a70 con 0x7fe8a4144370 2024-04-25T11:33:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.746+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a4050740 con 0x7fe8a4144370 2024-04-25T11:33:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.747+0000 7fe8a1600640 1 -- 172.21.15.19:0/69287766 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe89000bba0 con 0x7fe8a4144370 2024-04-25T11:33:50.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.748+0000 7fe8a1600640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8840744b0 0x7fe8840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.748+0000 7fe8a1600640 1 -- 172.21.15.19:0/69287766 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5939+0+0 (secure 0 0 0) 0x7fe890095820 con 0x7fe8a4144370 2024-04-25T11:33:50.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.749+0000 7fe8a9600640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8840744b0 0x7fe8840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:50.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.750+0000 7fe8a9600640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8840744b0 0x7fe8840769c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe898000f60 tx=0x7fe898007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:50.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:50.753+0000 7fe8a1600640 1 -- 172.21.15.19:0/69287766 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe89005fe20 con 0x7fe8a4144370 2024-04-25T11:33:50.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.003+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe8a4162250 con 0x7fe8a4144370 2024-04-25T11:33:50.993 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:33:51.005+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:33:50.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.009+0000 7fe8a1600640 1 -- 172.21.15.19:0/69287766 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) ==== 161+0+0 (secure 0 0 0) 0x7fe89005f720 con 0x7fe8a4144370 2024-04-25T11:33:50.997 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:33:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8840744b0 msgr2=0x7fe8840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8840744b0 0x7fe8840769c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe898000f60 tx=0x7fe898007c00 comp rx=0 tx=0).stop 2024-04-25T11:33:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 msgr2=0x7fe8a4070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a4070be0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe89000d620 tx=0x7fe89000dae0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 shutdown_connections 2024-04-25T11:33:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8840744b0 0x7fe8840769c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a415d440 0x7fe8a4160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8a4144370 0x7fe8a4070be0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 --2- 172.21.15.19:0/69287766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8a406c5c0 0x7fe8a4135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.011+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 >> 172.21.15.19:0/69287766 conn(0x7fe8a40d1720 msgr2=0x7fe8a40b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:51.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.012+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 shutdown_connections 2024-04-25T11:33:51.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.012+0000 7fe8aaa00640 1 -- 172.21.15.19:0/69287766 wait complete. 2024-04-25T11:33:51.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:33:51.055 INFO:tasks.ceph.mds.d.smithi169.stderr: -4559> 2024-04-25T11:32:02.188+0000 7ff844400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:33:51.195 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:33:51.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.355+0000 7fa614e00640 1 Processor -- start 2024-04-25T11:33:51.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.355+0000 7fa614e00640 1 -- start start 2024-04-25T11:33:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.357+0000 7fa614e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6100b2e40 0x7fa6100d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.357+0000 7fa614e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6100d1ed0 con 0x7fa6100b9100 2024-04-25T11:33:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.358+0000 7fa614e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6100d21b0 con 0x7fa6100b55c0 2024-04-25T11:33:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.358+0000 7fa614e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa6100b9510 con 0x7fa6100b2e40 2024-04-25T11:33:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.358+0000 7fa60f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa6100b9100 0x7fa6100b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58754/0 (socket says 172.21.15.19:58754) 2024-04-25T11:33:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.358+0000 7fa60f400640 1 -- 172.21.15.19:0/3078016442 learned_addr learned my addr 172.21.15.19:0/3078016442 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.358+0000 7fa60ea00640 1 --2- 172.21.15.19:0/3078016442 >> v2:172.21.15.169:3300/0 conn(0x7fa6100b2e40 0x7fa6100d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.358+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1908456273 0 0) 0x7fa6100d1ed0 con 0x7fa6100b9100 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.358+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5f4003520 con 0x7fa6100b9100 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 24295861 0 0) 0x7fa5f4003520 con 0x7fa6100b9100 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 >> v2:172.21.15.169:3300/0 conn(0x7fa6100b2e40 msgr2=0x7fa6100d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 --2- 172.21.15.19:0/3078016442 >> v2:172.21.15.169:3300/0 conn(0x7fa6100b2e40 0x7fa6100d2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 >> v1:172.21.15.169:6789/0 conn(0x7fa6100b55c0 legacy=0x7fa6100b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6100d2490 con 0x7fa6100b9100 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa600001670 con 0x7fa6100b9100 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 >> v1:172.21.15.19:6789/0 conn(0x7fa6100b9100 legacy=0x7fa6100b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f40039b0 0x7fa5f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f4004800 0x7fa5f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4005250 0x7fa5f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b9510 con 0x7fa5f4004800 2024-04-25T11:33:51.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f400c1e0 con 0x7fa5f4005250 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f400c3e0 con 0x7fa5f40039b0 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.359+0000 7fa60ea00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f4004800 0x7fa5f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60fe00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4005250 0x7fa5f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60f400640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f40039b0 0x7fa5f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60ea00640 1 -- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f40039b0 msgr2=0x7fa5f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60ea00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5f40039b0 0x7fa5f4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60ea00640 1 -- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4005250 msgr2=0x7fa5f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60ea00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4005250 0x7fa5f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60ea00640 1 -- 172.21.15.19:0/3078016442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f4003520 con 0x7fa5f4004800 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60fe00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4005250 0x7fa5f400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.360+0000 7fa60fe00640 1 -- 172.21.15.19:0/3078016442 reap_dead start 2024-04-25T11:33:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.361+0000 7fa60ea00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f4004800 0x7fa5f4004c10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa6100d1a90 tx=0x7fa5f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6647d2c4519ee9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.361+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc036070 con 0x7fa5f4004800 2024-04-25T11:33:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.361+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5fc031400 con 0x7fa5f4004800 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.361+0000 7fa60e000640 1 -- 172.21.15.19:0/3078016442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc009690 con 0x7fa5f4004800 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.362+0000 7fa614e00640 1 -- 172.21.15.19:0/3078016442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f4004800 msgr2=0x7fa5f4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.362+0000 7fa614e00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f4004800 0x7fa5f4004c10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa6100d1a90 tx=0x7fa5f4005830 comp rx=0 tx=0).stop 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.362+0000 7fa614e00640 1 -- 172.21.15.19:0/3078016442 shutdown_connections 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.362+0000 7fa614e00640 1 --2- 172.21.15.19:0/3078016442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f4004800 0x7fa5f4004c10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.362+0000 7fa614e00640 1 -- 172.21.15.19:0/3078016442 >> 172.21.15.19:0/3078016442 conn(0x7fa61008d180 msgr2=0x7fa61008d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.362+0000 7fa614e00640 1 -- 172.21.15.19:0/3078016442 shutdown_connections 2024-04-25T11:33:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.362+0000 7fa614e00640 1 -- 172.21.15.19:0/3078016442 wait complete. 2024-04-25T11:33:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.365+0000 7fa614e00640 1 Processor -- start 2024-04-25T11:33:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.366+0000 7fa614e00640 1 -- start start 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa614e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4004800 0x7fa6100ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa614e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100d0560 0x7fa6100d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa614e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610161a90 0x7fa610161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa614e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b5c20 con 0x7fa610161a90 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa614e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b3250 con 0x7fa5f4004800 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa614e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa6100b5da0 con 0x7fa6100d0560 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610161a90 0x7fa610161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610161a90 0x7fa610161ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33974/0 (socket says 172.21.15.19:33974) 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa60fe00640 1 -- 172.21.15.19:0/399221548 learned_addr learned my addr 172.21.15.19:0/399221548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa60f400640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4004800 0x7fa6100ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.368+0000 7fa60ea00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100d0560 0x7fa6100d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa60fe00640 1 -- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100d0560 msgr2=0x7fa6100d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:51.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa60fe00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100d0560 0x7fa6100d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa60fe00640 1 -- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4004800 msgr2=0x7fa6100ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa60fe00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4004800 0x7fa6100ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa60fe00640 1 -- 172.21.15.19:0/399221548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6100ce4e0 con 0x7fa610161a90 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa60fe00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610161a90 0x7fa610161ea0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa6100711e0 tx=0x7fa60800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa607400640 1 -- 172.21.15.19:0/399221548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608018070 con 0x7fa610161a90 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa607400640 1 -- 172.21.15.19:0/399221548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa608003da0 con 0x7fa610161a90 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa607400640 1 -- 172.21.15.19:0/399221548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608005330 con 0x7fa610161a90 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.369+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6100cbb90 con 0x7fa610161a90 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.370+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6100cd6f0 con 0x7fa610161a90 2024-04-25T11:33:51.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.371+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6100b6340 con 0x7fa610161a90 2024-04-25T11:33:51.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.372+0000 7fa607400640 1 -- 172.21.15.19:0/399221548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa60801b070 con 0x7fa610161a90 2024-04-25T11:33:51.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.373+0000 7fa607400640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5f00744e0 0x7fa5f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.373+0000 7fa607400640 1 -- 172.21.15.19:0/399221548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6025+0+0 (secure 0 0 0) 0x7fa608094b80 con 0x7fa610161a90 2024-04-25T11:33:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.374+0000 7fa60f400640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5f00744e0 0x7fa5f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.375+0000 7fa60f400640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5f00744e0 0x7fa5f00769f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa600000f60 tx=0x7fa600009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:51.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.378+0000 7fa607400640 1 -- 172.21.15.19:0/399221548 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa60805f1c0 con 0x7fa610161a90 2024-04-25T11:33:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:51.627+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa610076c60 con 0x7fa610161a90 2024-04-25T11:33:52.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.049+0000 7fa607400640 1 -- 172.21.15.19:0/399221548 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) ==== 104+0+0 (secure 0 0 0) 0x7fa60805eac0 con 0x7fa610161a90 2024-04-25T11:33:52.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5f00744e0 msgr2=0x7fa5f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5f00744e0 0x7fa5f00769f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa600000f60 tx=0x7fa600009040 comp rx=0 tx=0).stop 2024-04-25T11:33:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610161a90 msgr2=0x7fa610161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610161a90 0x7fa610161ea0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa6100711e0 tx=0x7fa60800d9f0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 shutdown_connections 2024-04-25T11:33:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5f00744e0 0x7fa5f00769f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa610161a90 0x7fa610161ea0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6100d0560 0x7fa6100d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 --2- 172.21.15.19:0/399221548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5f4004800 0x7fa6100ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.053+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 >> 172.21.15.19:0/399221548 conn(0x7fa61008d180 msgr2=0x7fa61008d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:52.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.054+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 shutdown_connections 2024-04-25T11:33:52.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.054+0000 7fa614e00640 1 -- 172.21.15.19:0/399221548 wait complete. 2024-04-25T11:33:52.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:33:52.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.374+0000 7fc516c00640 1 Processor -- start 2024-04-25T11:33:52.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.374+0000 7fc516c00640 1 -- start start 2024-04-25T11:33:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.375+0000 7fc516c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5100cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.375+0000 7fc516c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc510070480 0x7fc510070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.375+0000 7fc516c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51015d1b0 con 0x7fc510070dd0 2024-04-25T11:33:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.375+0000 7fc516c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc5100cbf80 con 0x7fc510070480 2024-04-25T11:33:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.375+0000 7fc516c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5100ce190 con 0x7fc51006c5c0 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc514e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc510070480 0x7fc510070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc514e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc510070480 0x7fc510070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33990/0 (socket says 172.21.15.19:33990) 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc514e00640 1 -- 172.21.15.19:0/3117301953 learned_addr learned my addr 172.21.15.19:0/3117301953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc515800640 1 --2- 172.21.15.19:0/3117301953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5100cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc514e00640 1 -- 172.21.15.19:0/3117301953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 msgr2=0x7fc5100cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc514e00640 1 --2- 172.21.15.19:0/3117301953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5100cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc514e00640 1 -- 172.21.15.19:0/3117301953 >> v1:172.21.15.169:6789/0 conn(0x7fc510070dd0 legacy=0x7fc5100cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc514e00640 1 -- 172.21.15.19:0/3117301953 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5100d05b0 con 0x7fc510070480 2024-04-25T11:33:52.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.376+0000 7fc515800640 1 --2- 172.21.15.19:0/3117301953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5100cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.377+0000 7fc514e00640 1 --2- 172.21.15.19:0/3117301953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070480 0x7fc510070890 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009870 tx=0x7fc4fc02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e460bf9d875281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc50fe00640 1 -- 172.21.15.19:0/3117301953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc037070 con 0x7fc510070480 2024-04-25T11:33:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc50fe00640 1 -- 172.21.15.19:0/3117301953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4fc032440 con 0x7fc510070480 2024-04-25T11:33:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc50fe00640 1 -- 172.21.15.19:0/3117301953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc032810 con 0x7fc510070480 2024-04-25T11:33:52.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc516c00640 1 -- 172.21.15.19:0/3117301953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070480 msgr2=0x7fc510070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc516c00640 1 --2- 172.21.15.19:0/3117301953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070480 0x7fc510070890 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009870 tx=0x7fc4fc02a200 comp rx=0 tx=0).stop 2024-04-25T11:33:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc516c00640 1 -- 172.21.15.19:0/3117301953 shutdown_connections 2024-04-25T11:33:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc516c00640 1 --2- 172.21.15.19:0/3117301953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070480 0x7fc510070890 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.378+0000 7fc516c00640 1 --2- 172.21.15.19:0/3117301953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5100cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.379+0000 7fc516c00640 1 -- 172.21.15.19:0/3117301953 >> 172.21.15.19:0/3117301953 conn(0x7fc5100ce3b0 msgr2=0x7fc5100ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.379+0000 7fc516c00640 1 -- 172.21.15.19:0/3117301953 shutdown_connections 2024-04-25T11:33:52.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.379+0000 7fc516c00640 1 -- 172.21.15.19:0/3117301953 wait complete. 2024-04-25T11:33:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.381+0000 7fc516c00640 1 Processor -- start 2024-04-25T11:33:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.381+0000 7fc516c00640 1 -- start start 2024-04-25T11:33:52.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5101f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:52.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc510070480 0x7fc5100ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070dd0 0x7fc51015f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5100d0e50 con 0x7fc510070dd0 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc5100d0b70 con 0x7fc510070480 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc510161920 con 0x7fc51006c5c0 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070dd0 0x7fc51015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070dd0 0x7fc51015f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33992/0 (socket says 172.21.15.19:33992) 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 -- 172.21.15.19:0/3335793811 learned_addr learned my addr 172.21.15.19:0/3335793811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc514e00640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc510070480 0x7fc5100ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc515800640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5101f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 -- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 msgr2=0x7fc5101f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5101f0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 -- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc510070480 msgr2=0x7fc5100ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc510070480 0x7fc5100ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.383+0000 7fc516200640 1 -- 172.21.15.19:0/3335793811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5101f07c0 con 0x7fc510070dd0 2024-04-25T11:33:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.384+0000 7fc516200640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070dd0 0x7fc51015f560 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc50800b9a0 tx=0x7fc50800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.384+0000 7fc50d600640 1 -- 172.21.15.19:0/3335793811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508019070 con 0x7fc510070dd0 2024-04-25T11:33:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.384+0000 7fc50d600640 1 -- 172.21.15.19:0/3335793811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50800cba0 con 0x7fc510070dd0 2024-04-25T11:33:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.384+0000 7fc50d600640 1 -- 172.21.15.19:0/3335793811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508014440 con 0x7fc510070dd0 2024-04-25T11:33:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.384+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5100c4830 con 0x7fc510070dd0 2024-04-25T11:33:52.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.384+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5100c4df0 con 0x7fc510070dd0 2024-04-25T11:33:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.385+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc510076c60 con 0x7fc510070dd0 2024-04-25T11:33:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.386+0000 7fc50d600640 1 -- 172.21.15.19:0/3335793811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc50801c070 con 0x7fc510070dd0 2024-04-25T11:33:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.387+0000 7fc50d600640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4f00744b0 0x7fc4f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.387+0000 7fc50d600640 1 -- 172.21.15.19:0/3335793811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6025+0+0 (secure 0 0 0) 0x7fc5080950e0 con 0x7fc510070dd0 2024-04-25T11:33:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.388+0000 7fc515800640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4f00744b0 0x7fc4f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.389+0000 7fc515800640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4f00744b0 0x7fc4f00769c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b840 tx=0x7fc504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:52.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.392+0000 7fc50d600640 1 -- 172.21.15.19:0/3335793811 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc50805f690 con 0x7fc510070dd0 2024-04-25T11:33:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:52.590+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc51006a160 con 0x7fc510070dd0 2024-04-25T11:33:53.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.059+0000 7fc50d600640 1 -- 172.21.15.19:0/3335793811 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) ==== 181+0+0 (secure 0 0 0) 0x7fc50805ef90 con 0x7fc510070dd0 2024-04-25T11:33:53.048 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:33:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.065+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4f00744b0 msgr2=0x7fc4f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.065+0000 7fc516c00640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4f00744b0 0x7fc4f00769c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b840 tx=0x7fc504008040 comp rx=0 tx=0).stop 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.065+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070dd0 msgr2=0x7fc51015f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.065+0000 7fc516c00640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070dd0 0x7fc51015f560 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc50800b9a0 tx=0x7fc50800be60 comp rx=0 tx=0).stop 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 shutdown_connections 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4f00744b0 0x7fc4f00769c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc510070480 0x7fc5100ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc510070dd0 0x7fc51015f560 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 --2- 172.21.15.19:0/3335793811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc51006c5c0 0x7fc5101f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 >> 172.21.15.19:0/3335793811 conn(0x7fc5100ce3b0 msgr2=0x7fc5100b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 shutdown_connections 2024-04-25T11:33:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.067+0000 7fc516c00640 1 -- 172.21.15.19:0/3335793811 wait complete. 2024-04-25T11:33:53.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:33:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.390+0000 7f7b8ba00640 1 Processor -- start 2024-04-25T11:33:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.390+0000 7f7b8ba00640 1 -- start start 2024-04-25T11:33:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.392+0000 7f7b8ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8415e0c0 0x7f7b8406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.392+0000 7f7b8ba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7b8406e590 0x7f7b840d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.392+0000 7f7b8ba00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b84161970 con 0x7f7b840d1270 2024-04-25T11:33:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.392+0000 7f7b8ba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7b841614e0 con 0x7f7b8406e590 2024-04-25T11:33:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.392+0000 7f7b8ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7b840d1a40 con 0x7f7b8415e0c0 2024-04-25T11:33:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.393+0000 7f7b89000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8415e0c0 0x7f7b8406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.393+0000 7f7b89000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8415e0c0 0x7f7b8406e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:47922/0 (socket says 172.21.15.19:47922) 2024-04-25T11:33:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.393+0000 7f7b89000640 1 -- 172.21.15.19:0/4068187310 learned_addr learned my addr 172.21.15.19:0/4068187310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.393+0000 7f7b83e00640 1 --2- 172.21.15.19:0/4068187310 >> v2:172.21.15.169:3300/0 conn(0x7f7b8406e590 0x7f7b840d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.394+0000 7f7b89000640 1 -- 172.21.15.19:0/4068187310 >> v2:172.21.15.169:3300/0 conn(0x7f7b8406e590 msgr2=0x7f7b840d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.394+0000 7f7b89000640 1 --2- 172.21.15.19:0/4068187310 >> v2:172.21.15.169:3300/0 conn(0x7f7b8406e590 0x7f7b840d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.394+0000 7f7b89000640 1 -- 172.21.15.19:0/4068187310 >> v1:172.21.15.169:6789/0 conn(0x7f7b840d1270 legacy=0x7f7b840b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.394+0000 7f7b89000640 1 -- 172.21.15.19:0/4068187310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8406e9a0 con 0x7f7b8415e0c0 2024-04-25T11:33:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.396+0000 7f7b89000640 1 --2- 172.21.15.19:0/4068187310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8415e0c0 0x7f7b8406e050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009e50 tx=0x7f7b7402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3012301233dbcfcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.396+0000 7f7b83400640 1 -- 172.21.15.19:0/4068187310 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74036070 con 0x7f7b8415e0c0 2024-04-25T11:33:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.396+0000 7f7b83400640 1 -- 172.21.15.19:0/4068187310 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b74031600 con 0x7f7b8415e0c0 2024-04-25T11:33:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.396+0000 7f7b83400640 1 -- 172.21.15.19:0/4068187310 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740319d0 con 0x7f7b8415e0c0 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 -- 172.21.15.19:0/4068187310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8415e0c0 msgr2=0x7f7b8406e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/4068187310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8415e0c0 0x7f7b8406e050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009e50 tx=0x7f7b7402a710 comp rx=0 tx=0).stop 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 -- 172.21.15.19:0/4068187310 shutdown_connections 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/4068187310 >> v2:172.21.15.169:3300/0 conn(0x7f7b8406e590 0x7f7b840d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/4068187310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8415e0c0 0x7f7b8406e050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 -- 172.21.15.19:0/4068187310 >> 172.21.15.19:0/4068187310 conn(0x7f7b840d2800 msgr2=0x7f7b840d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 -- 172.21.15.19:0/4068187310 shutdown_connections 2024-04-25T11:33:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.397+0000 7f7b8ba00640 1 -- 172.21.15.19:0/4068187310 wait complete. 2024-04-25T11:33:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.400+0000 7f7b8ba00640 1 Processor -- start 2024-04-25T11:33:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.400+0000 7f7b8ba00640 1 -- start start 2024-04-25T11:33:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.401+0000 7f7b8ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8406e590 0x7f7b841f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.401+0000 7f7b8ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b840d1270 0x7f7b840cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.401+0000 7f7b8ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b8415e0c0 0x7f7b841512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.401+0000 7f7b8ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84161170 con 0x7f7b8415e0c0 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.401+0000 7f7b8ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84160e90 con 0x7f7b840d1270 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.401+0000 7f7b8ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7b840d1680 con 0x7f7b8406e590 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b8415e0c0 0x7f7b841512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b8415e0c0 0x7f7b841512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34000/0 (socket says 172.21.15.19:34000) 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 -- 172.21.15.19:0/1788979011 learned_addr learned my addr 172.21.15.19:0/1788979011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b83e00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b840d1270 0x7f7b840cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89000640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8406e590 0x7f7b841f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 -- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8406e590 msgr2=0x7f7b841f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8406e590 0x7f7b841f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 -- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b840d1270 msgr2=0x7f7b840cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b840d1270 0x7f7b840cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89a00640 1 -- 172.21.15.19:0/1788979011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b840cb8c0 con 0x7f7b8415e0c0 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.402+0000 7f7b89000640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8406e590 0x7f7b841f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.403+0000 7f7b89a00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b8415e0c0 0x7f7b841512b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c3c0 tx=0x7f7b7c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.403+0000 7f7b80c00640 1 -- 172.21.15.19:0/1788979011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c017070 con 0x7f7b8415e0c0 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.403+0000 7f7b80c00640 1 -- 172.21.15.19:0/1788979011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c0052d0 con 0x7f7b8415e0c0 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.403+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b840cbba0 con 0x7f7b8415e0c0 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.403+0000 7f7b80c00640 1 -- 172.21.15.19:0/1788979011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c003830 con 0x7f7b8415e0c0 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.403+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b840cc160 con 0x7f7b8415e0c0 2024-04-25T11:33:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.404+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b840cb630 con 0x7f7b8415e0c0 2024-04-25T11:33:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.406+0000 7f7b80c00640 1 -- 172.21.15.19:0/1788979011 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7b7c01a070 con 0x7f7b8415e0c0 2024-04-25T11:33:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.408+0000 7f7b80c00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7b600744b0 0x7f7b600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.408+0000 7f7b80c00640 1 -- 172.21.15.19:0/1788979011 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5587+0+0 (secure 0 0 0) 0x7f7b7c0945a0 con 0x7f7b8415e0c0 2024-04-25T11:33:53.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.409+0000 7f7b89000640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7b600744b0 0x7f7b600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:53.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.410+0000 7f7b89000640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7b600744b0 0x7f7b600769c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7b74033000 tx=0x7f7b74009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:53.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.417+0000 7f7b80c00640 1 -- 172.21.15.19:0/1788979011 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b7c05ed90 con 0x7f7b8415e0c0 2024-04-25T11:33:53.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:53.631+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7b840d43b0 con 0x7f7b8415e0c0 2024-04-25T11:33:54.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.072+0000 7f7b80c00640 1 -- 172.21.15.19:0/1788979011 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) ==== 169+0+0 (secure 0 0 0) 0x7f7b7c05e690 con 0x7f7b8415e0c0 2024-04-25T11:33:54.062 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:33:54.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7b600744b0 msgr2=0x7f7b600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7b600744b0 0x7f7b600769c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7b74033000 tx=0x7f7b74009000 comp rx=0 tx=0).stop 2024-04-25T11:33:54.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b8415e0c0 msgr2=0x7f7b841512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b8415e0c0 0x7f7b841512b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c3c0 tx=0x7f7b7c00c7c0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 shutdown_connections 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7b600744b0 0x7f7b600769c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b840d1270 0x7f7b840cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b8415e0c0 0x7f7b841512b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 --2- 172.21.15.19:0/1788979011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b8406e590 0x7f7b841f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.077+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 >> 172.21.15.19:0/1788979011 conn(0x7f7b840d2800 msgr2=0x7f7b840d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.078+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 shutdown_connections 2024-04-25T11:33:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.079+0000 7f7b8ba00640 1 -- 172.21.15.19:0/1788979011 wait complete. 2024-04-25T11:33:54.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-04-25T11:33:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.391+0000 7f19fac00640 1 Processor -- start 2024-04-25T11:33:54.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.391+0000 7f19fac00640 1 -- start start 2024-04-25T11:33:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.394+0000 7f19fac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19f4144310 0x7f19f406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.394+0000 7f19fac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4161a80 0x7f19f4161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.394+0000 7f19fac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f4144720 con 0x7f19f40b6ba0 2024-04-25T11:33:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.394+0000 7f19fac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f19f406f640 con 0x7f19f4144310 2024-04-25T11:33:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.394+0000 7f19fac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19f41624d0 con 0x7f19f4161a80 2024-04-25T11:33:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.395+0000 7f19f8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4161a80 0x7f19f4161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.395+0000 7f19f9800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19f4144310 0x7f19f406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.395+0000 7f19f8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4161a80 0x7f19f4161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:41986/0 (socket says 172.21.15.19:41986) 2024-04-25T11:33:54.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.395+0000 7f19f8e00640 1 -- 172.21.15.19:0/4066511178 learned_addr learned my addr 172.21.15.19:0/4066511178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:54.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.396+0000 7f19f8e00640 1 -- 172.21.15.19:0/4066511178 >> v2:172.21.15.169:3300/0 conn(0x7f19f4144310 msgr2=0x7f19f406fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:54.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.396+0000 7f19f8e00640 1 --2- 172.21.15.19:0/4066511178 >> v2:172.21.15.169:3300/0 conn(0x7f19f4144310 0x7f19f406fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.396+0000 7f19f8e00640 1 -- 172.21.15.19:0/4066511178 >> v1:172.21.15.169:6789/0 conn(0x7f19f40b6ba0 legacy=0x7f19f40b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.396+0000 7f19f8e00640 1 -- 172.21.15.19:0/4066511178 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f41627b0 con 0x7f19f4161a80 2024-04-25T11:33:54.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.397+0000 7f19f8e00640 1 --2- 172.21.15.19:0/4066511178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4161a80 0x7f19f4161e90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed0eb6ff0e0184a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:54.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.397+0000 7f19f3e00640 1 -- 172.21.15.19:0/4066511178 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0037070 con 0x7f19f4161a80 2024-04-25T11:33:54.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.397+0000 7f19f3e00640 1 -- 172.21.15.19:0/4066511178 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0032440 con 0x7f19f4161a80 2024-04-25T11:33:54.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19f3e00640 1 -- 172.21.15.19:0/4066511178 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0032810 con 0x7f19f4161a80 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19fac00640 1 -- 172.21.15.19:0/4066511178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4161a80 msgr2=0x7f19f4161e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19fac00640 1 --2- 172.21.15.19:0/4066511178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4161a80 0x7f19f4161e90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009870 tx=0x7f19e002a250 comp rx=0 tx=0).stop 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19fac00640 1 -- 172.21.15.19:0/4066511178 shutdown_connections 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19fac00640 1 --2- 172.21.15.19:0/4066511178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4161a80 0x7f19f4161e90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19fac00640 1 --2- 172.21.15.19:0/4066511178 >> v2:172.21.15.169:3300/0 conn(0x7f19f4144310 0x7f19f406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19fac00640 1 -- 172.21.15.19:0/4066511178 >> 172.21.15.19:0/4066511178 conn(0x7f19f40b4260 msgr2=0x7f19f4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.398+0000 7f19fac00640 1 -- 172.21.15.19:0/4066511178 shutdown_connections 2024-04-25T11:33:54.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.399+0000 7f19fac00640 1 -- 172.21.15.19:0/4066511178 wait complete. 2024-04-25T11:33:54.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.401+0000 7f19fac00640 1 Processor -- start 2024-04-25T11:33:54.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.401+0000 7f19fac00640 1 -- start start 2024-04-25T11:33:54.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19fac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4144310 0x7f19f41f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19fac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f4161a80 0x7f19f415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19fac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f4131870 0x7f19f41e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19fac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19f41478d0 con 0x7f19f4161a80 2024-04-25T11:33:54.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19fac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4070010 con 0x7f19f4131870 2024-04-25T11:33:54.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19fac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19f4147a50 con 0x7f19f4144310 2024-04-25T11:33:54.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19f8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f4161a80 0x7f19f415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19f8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f4161a80 0x7f19f415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49854/0 (socket says 172.21.15.19:49854) 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19f8e00640 1 -- 172.21.15.19:0/3282029941 learned_addr learned my addr 172.21.15.19:0/3282029941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19f9800640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4144310 0x7f19f41f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.403+0000 7f19fa200640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f4131870 0x7f19f41e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f8e00640 1 -- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4144310 msgr2=0x7f19f41f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f8e00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4144310 0x7f19f41f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f8e00640 1 -- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f4131870 msgr2=0x7f19f41e77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f8e00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f4131870 0x7f19f41e77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f8e00640 1 -- 172.21.15.19:0/3282029941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f4131fc0 con 0x7f19f4161a80 2024-04-25T11:33:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f8e00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f4161a80 0x7f19f415d6e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f19e00090b0 tx=0x7f19e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f1600640 1 -- 172.21.15.19:0/3282029941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0041030 con 0x7f19f4161a80 2024-04-25T11:33:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f1600640 1 -- 172.21.15.19:0/3282029941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0030440 con 0x7f19f4161a80 2024-04-25T11:33:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19f1600640 1 -- 172.21.15.19:0/3282029941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0037040 con 0x7f19f4161a80 2024-04-25T11:33:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.404+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f41322a0 con 0x7f19f4161a80 2024-04-25T11:33:54.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.405+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f40b2190 con 0x7f19f4161a80 2024-04-25T11:33:54.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.406+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f41327c0 con 0x7f19f4161a80 2024-04-25T11:33:54.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.407+0000 7f19f1600640 1 -- 172.21.15.19:0/3282029941 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f19e0044070 con 0x7f19f4161a80 2024-04-25T11:33:54.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.408+0000 7f19f1600640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19d40744e0 0x7f19d40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.408+0000 7f19f1600640 1 -- 172.21.15.19:0/3282029941 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7f19e00b53f0 con 0x7f19f4161a80 2024-04-25T11:33:54.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.409+0000 7f19f9800640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19d40744e0 0x7f19d40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.410+0000 7f19f9800640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19d40744e0 0x7f19d40769f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19f40b7b20 tx=0x7f19e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:54.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.413+0000 7f19f1600640 1 -- 172.21.15.19:0/3282029941 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19f41327c0 con 0x7f19f4161a80 2024-04-25T11:33:54.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.620+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7f19f4131d50 con 0x7f19f4161a80 2024-04-25T11:33:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.620+0000 7f19f1600640 1 -- 172.21.15.19:0/3282029941 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f19e004b070 con 0x7f19f4161a80 2024-04-25T11:33:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19d40744e0 msgr2=0x7f19d40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19d40744e0 0x7f19d40769f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19f40b7b20 tx=0x7f19e8008040 comp rx=0 tx=0).stop 2024-04-25T11:33:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f4161a80 msgr2=0x7f19f415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f4161a80 0x7f19f415d6e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f19e00090b0 tx=0x7f19e00046c0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 shutdown_connections 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19d40744e0 0x7f19d40769f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19f4131870 0x7f19f41e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f4161a80 0x7f19f415d6e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 --2- 172.21.15.19:0/3282029941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19f4144310 0x7f19f41f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.623+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 >> 172.21.15.19:0/3282029941 conn(0x7f19f40b6ba0 msgr2=0x7f19f4141ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.624+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 shutdown_connections 2024-04-25T11:33:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.624+0000 7f19fac00640 1 -- 172.21.15.19:0/3282029941 wait complete. 2024-04-25T11:33:54.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:33:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.938+0000 7f49d5600640 1 Processor -- start 2024-04-25T11:33:54.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.938+0000 7f49d5600640 1 -- start start 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49d5600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49d0143660 0x7f49d0070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49d5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49d5600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d006df20 con 0x7f49d00d04f0 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49d5600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f49d0070c70 con 0x7f49d0143660 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49d5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f49d00b4920 con 0x7f49d006d5d0 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49cea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49d0143660 0x7f49d0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49cf400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f49d00d04f0 0x7f49d00d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37566/0 (socket says 172.21.15.19:37566) 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49cea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49d0143660 0x7f49d0070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49864/0 (socket says 172.21.15.19:49864) 2024-04-25T11:33:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.941+0000 7f49cf400640 1 -- 172.21.15.19:0/866634085 learned_addr learned my addr 172.21.15.19:0/866634085 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.942+0000 7f49c6000640 1 --2- 172.21.15.19:0/866634085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.942+0000 7f49cea00640 1 -- 172.21.15.19:0/866634085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 msgr2=0x7f49d006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.942+0000 7f49cea00640 1 --2- 172.21.15.19:0/866634085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d006d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.942+0000 7f49cea00640 1 -- 172.21.15.19:0/866634085 >> v1:172.21.15.19:6789/0 conn(0x7f49d00d04f0 legacy=0x7f49d00d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.942+0000 7f49cea00640 1 -- 172.21.15.19:0/866634085 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d00b4c00 con 0x7f49d0143660 2024-04-25T11:33:54.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.943+0000 7f49cea00640 1 --2- 172.21.15.19:0/866634085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d0143660 0x7f49d0070290 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f49c00097a0 tx=0x7f49c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277a8b5b9ebcc6be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:54.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.944+0000 7f49ce000640 1 -- 172.21.15.19:0/866634085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0037070 con 0x7f49d0143660 2024-04-25T11:33:54.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.944+0000 7f49ce000640 1 -- 172.21.15.19:0/866634085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c0032440 con 0x7f49d0143660 2024-04-25T11:33:54.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.944+0000 7f49ce000640 1 -- 172.21.15.19:0/866634085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0032810 con 0x7f49d0143660 2024-04-25T11:33:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.944+0000 7f49d5600640 1 -- 172.21.15.19:0/866634085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d0143660 msgr2=0x7f49d0070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.944+0000 7f49d5600640 1 --2- 172.21.15.19:0/866634085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d0143660 0x7f49d0070290 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f49c00097a0 tx=0x7f49c002a180 comp rx=0 tx=0).stop 2024-04-25T11:33:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.945+0000 7f49d5600640 1 -- 172.21.15.19:0/866634085 shutdown_connections 2024-04-25T11:33:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.945+0000 7f49d5600640 1 --2- 172.21.15.19:0/866634085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.945+0000 7f49d5600640 1 --2- 172.21.15.19:0/866634085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d0143660 0x7f49d0070290 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.945+0000 7f49d5600640 1 -- 172.21.15.19:0/866634085 >> 172.21.15.19:0/866634085 conn(0x7f49d0073120 msgr2=0x7f49d0073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.945+0000 7f49d5600640 1 -- 172.21.15.19:0/866634085 shutdown_connections 2024-04-25T11:33:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.945+0000 7f49d5600640 1 -- 172.21.15.19:0/866634085 wait complete. 2024-04-25T11:33:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.947+0000 7f49d5600640 1 Processor -- start 2024-04-25T11:33:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.947+0000 7f49d5600640 1 -- start start 2024-04-25T11:33:54.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49d5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49d5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d00d04f0 0x7f49d015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49d5600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0143660 0x7f49d00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49d5600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49d00ce0e0 con 0x7f49d00d04f0 2024-04-25T11:33:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49d5600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f49d00b51c0 con 0x7f49d0143660 2024-04-25T11:33:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49d5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f49d00ce260 con 0x7f49d006d5d0 2024-04-25T11:33:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49c6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d00d04f0 0x7f49d015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49c6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d00d04f0 0x7f49d015def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49876/0 (socket says 172.21.15.19:49876) 2024-04-25T11:33:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49c6000640 1 -- 172.21.15.19:0/1572786010 learned_addr learned my addr 172.21.15.19:0/1572786010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49cea00640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.949+0000 7f49c6000640 1 -- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 msgr2=0x7f49d015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c6000640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c6000640 1 -- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0143660 msgr2=0x7f49d00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49cf400640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0143660 0x7f49d00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c6000640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0143660 0x7f49d00b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c6000640 1 -- 172.21.15.19:0/1572786010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d0159810 con 0x7f49d00d04f0 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49cea00640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c6000640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d00d04f0 0x7f49d015def0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d5d0 tx=0x7f49b800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c7400640 1 -- 172.21.15.19:0/1572786010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004240 con 0x7f49d00d04f0 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c7400640 1 -- 172.21.15.19:0/1572786010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b80043e0 con 0x7f49d00d04f0 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49c7400640 1 -- 172.21.15.19:0/1572786010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8003830 con 0x7f49d00d04f0 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.950+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d0159af0 con 0x7f49d00d04f0 2024-04-25T11:33:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.951+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d015a0b0 con 0x7f49d00d04f0 2024-04-25T11:33:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.952+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d0050740 con 0x7f49d00d04f0 2024-04-25T11:33:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.953+0000 7f49c7400640 1 -- 172.21.15.19:0/1572786010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49b80039d0 con 0x7f49d00d04f0 2024-04-25T11:33:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.954+0000 7f49c7400640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49ac0744b0 0x7f49ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.954+0000 7f49c7400640 1 -- 172.21.15.19:0/1572786010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7f49b80951e0 con 0x7f49d00d04f0 2024-04-25T11:33:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.955+0000 7f49cea00640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49ac0744b0 0x7f49ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.956+0000 7f49cea00640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49ac0744b0 0x7f49ac0769c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f49c0034000 tx=0x7f49c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:54.960+0000 7f49c7400640 1 -- 172.21.15.19:0/1572786010 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49b805fc40 con 0x7f49d00d04f0 2024-04-25T11:33:55.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.164+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f49d00719d0 con 0x7f49d00d04f0 2024-04-25T11:33:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.172+0000 7f49c7400640 1 -- 172.21.15.19:0/1572786010 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f49b805f540 con 0x7f49d00d04f0 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49ac0744b0 msgr2=0x7f49ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49ac0744b0 0x7f49ac0769c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f49c0034000 tx=0x7f49c002e000 comp rx=0 tx=0).stop 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d00d04f0 msgr2=0x7f49d015def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d00d04f0 0x7f49d015def0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d5d0 tx=0x7f49b800da90 comp rx=0 tx=0).stop 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 shutdown_connections 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f49ac0744b0 0x7f49ac0769c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49d0143660 0x7f49d00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49d00d04f0 0x7f49d015def0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 --2- 172.21.15.19:0/1572786010 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49d006d5d0 0x7f49d015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 >> 172.21.15.19:0/1572786010 conn(0x7f49d0073120 msgr2=0x7f49d015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 shutdown_connections 2024-04-25T11:33:55.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.175+0000 7f49d5600640 1 -- 172.21.15.19:0/1572786010 wait complete. 2024-04-25T11:33:55.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-04-25T11:33:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.478+0000 7fe800a00640 1 Processor -- start 2024-04-25T11:33:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.478+0000 7fe800a00640 1 -- start start 2024-04-25T11:33:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.480+0000 7fe800a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7fc06dce0 0x7fe7fc142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.480+0000 7fe800a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7fc142c70 0x7fe7fc0d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.480+0000 7fe800a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7fc15d5e0 con 0x7fe7fc13fe10 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.480+0000 7fe800a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe7fc15d8c0 con 0x7fe7fc06dce0 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.480+0000 7fe800a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe7fc15dba0 con 0x7fe7fc142c70 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7f3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7fc06dce0 0x7fe7fc142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7f3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7fc06dce0 0x7fe7fc142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49884/0 (socket says 172.21.15.19:49884) 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7fb400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe7fc13fe10 0x7fe7fc06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37574/0 (socket says 172.21.15.19:37574) 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7f3e00640 1 -- 172.21.15.19:0/589527544 learned_addr learned my addr 172.21.15.19:0/589527544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7fbe00640 1 --2- 172.21.15.19:0/589527544 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc142c70 0x7fe7fc0d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7f3e00640 1 -- 172.21.15.19:0/589527544 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc142c70 msgr2=0x7fe7fc0d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7f3e00640 1 --2- 172.21.15.19:0/589527544 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc142c70 0x7fe7fc0d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7f3e00640 1 -- 172.21.15.19:0/589527544 >> v1:172.21.15.19:6789/0 conn(0x7fe7fc13fe10 legacy=0x7fe7fc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.481+0000 7fe7f3e00640 1 -- 172.21.15.19:0/589527544 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc15de80 con 0x7fe7fc06dce0 2024-04-25T11:33:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.482+0000 7fe7f3e00640 1 --2- 172.21.15.19:0/589527544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc06dce0 0x7fe7fc142730 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc161a10 tx=0x7fe7e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9a0220e99d3728 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.483+0000 7fe7faa00640 1 -- 172.21.15.19:0/589527544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4037070 con 0x7fe7fc06dce0 2024-04-25T11:33:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.483+0000 7fe7faa00640 1 -- 172.21.15.19:0/589527544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e4032440 con 0x7fe7fc06dce0 2024-04-25T11:33:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.483+0000 7fe7faa00640 1 -- 172.21.15.19:0/589527544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4032810 con 0x7fe7fc06dce0 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.483+0000 7fe800a00640 1 -- 172.21.15.19:0/589527544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc06dce0 msgr2=0x7fe7fc142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.483+0000 7fe800a00640 1 --2- 172.21.15.19:0/589527544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc06dce0 0x7fe7fc142730 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc161a10 tx=0x7fe7e402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.484+0000 7fe800a00640 1 -- 172.21.15.19:0/589527544 shutdown_connections 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.484+0000 7fe800a00640 1 --2- 172.21.15.19:0/589527544 >> v2:172.21.15.169:3300/0 conn(0x7fe7fc142c70 0x7fe7fc0d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.484+0000 7fe800a00640 1 --2- 172.21.15.19:0/589527544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc06dce0 0x7fe7fc142730 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.484+0000 7fe800a00640 1 -- 172.21.15.19:0/589527544 >> 172.21.15.19:0/589527544 conn(0x7fe7fc072ee0 msgr2=0x7fe7fc0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.484+0000 7fe800a00640 1 -- 172.21.15.19:0/589527544 shutdown_connections 2024-04-25T11:33:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.484+0000 7fe800a00640 1 -- 172.21.15.19:0/589527544 wait complete. 2024-04-25T11:33:55.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.486+0000 7fe800a00640 1 Processor -- start 2024-04-25T11:33:55.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.486+0000 7fe800a00640 1 -- start start 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe800a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc13fe10 0x7fe7fc145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe800a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc142c70 0x7fe7fc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe800a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0b70c0 0x7fe7fc1f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe800a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc143200 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe800a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc143080 con 0x7fe7fc142c70 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe800a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7fc0d0570 con 0x7fe7fc13fe10 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0b70c0 0x7fe7fc1f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe7fbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0b70c0 0x7fe7fc1f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49888/0 (socket says 172.21.15.19:49888) 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe7fbe00640 1 -- 172.21.15.19:0/8799189 learned_addr learned my addr 172.21.15.19:0/8799189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.488+0000 7fe7f3e00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc142c70 0x7fe7fc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fb400640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc13fe10 0x7fe7fc145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fbe00640 1 -- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc13fe10 msgr2=0x7fe7fc145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fbe00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc13fe10 0x7fe7fc145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fbe00640 1 -- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc142c70 msgr2=0x7fe7fc155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fbe00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc142c70 0x7fe7fc155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fbe00640 1 -- 172.21.15.19:0/8799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc146380 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fb400640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc13fe10 0x7fe7fc145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:55.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7fbe00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0b70c0 0x7fe7fc1f9b80 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400cc60 tx=0x7fe7f400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:55.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7f3400640 1 -- 172.21.15.19:0/8799189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4003990 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7f3400640 1 -- 172.21.15.19:0/8799189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f4003b30 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe7f3400640 1 -- 172.21.15.19:0/8799189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4004020 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.489+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc156420 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.490+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc0b6300 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.491+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7fc050740 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.492+0000 7fe7f3400640 1 -- 172.21.15.19:0/8799189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe7f40041c0 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.493+0000 7fe7f3400640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7d80744e0 0x7fe7d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.493+0000 7fe7f3400640 1 -- 172.21.15.19:0/8799189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7fe7f4095540 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.494+0000 7fe7fb400640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7d80744e0 0x7fe7d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.495+0000 7fe7fb400640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7d80744e0 0x7fe7d80769f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000f60 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.498+0000 7fe7f3400640 1 -- 172.21.15.19:0/8799189 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7fc146380 con 0x7fe7fc0b70c0 2024-04-25T11:33:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:55.781+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7fe7fc0b5db0 con 0x7fe7fc0b70c0 2024-04-25T11:33:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.090+0000 7fe7f3400640 1 -- 172.21.15.19:0/8799189 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v332) ==== 142+0+0 (secure 0 0 0) 0x7fe7f4014030 con 0x7fe7fc0b70c0 2024-04-25T11:33:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7d80744e0 msgr2=0x7fe7d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7d80744e0 0x7fe7d80769f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec000f60 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2024-04-25T11:33:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0b70c0 msgr2=0x7fe7fc1f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0b70c0 0x7fe7fc1f9b80 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400cc60 tx=0x7fe7f400a5c0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 shutdown_connections 2024-04-25T11:33:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7d80744e0 0x7fe7d80769f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7fc142c70 0x7fe7fc155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7fc0b70c0 0x7fe7fc1f9b80 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 --2- 172.21.15.19:0/8799189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7fc13fe10 0x7fe7fc145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.094+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 >> 172.21.15.19:0/8799189 conn(0x7fe7fc072ee0 msgr2=0x7fe7fc142500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.095+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 shutdown_connections 2024-04-25T11:33:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.095+0000 7fe800a00640 1 -- 172.21.15.19:0/8799189 wait complete. 2024-04-25T11:33:56.115 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-04-25T11:33:56.115 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-04-25T11:33:56.115 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2024-04-25T11:33:56.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:33:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.414+0000 7f94b0200640 1 Processor -- start 2024-04-25T11:33:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.415+0000 7f94b0200640 1 -- start start 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94b0200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81443c0 0x7f94a806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94b0200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f94a8070370 0x7f94a80cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94b0200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a80cff30 con 0x7f94a80cfb20 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94b0200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f94a80708d0 con 0x7f94a8070370 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94b0200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94a815f160 con 0x7f94a81443c0 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94ace00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f94a8070370 0x7f94a80cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94ace00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f94a8070370 0x7f94a80cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49900/0 (socket says 172.21.15.19:49900) 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.417+0000 7f94ace00640 1 -- 172.21.15.19:0/3052025662 learned_addr learned my addr 172.21.15.19:0/3052025662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.418+0000 7f94ad800640 1 --2- 172.21.15.19:0/3052025662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81443c0 0x7f94a806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.418+0000 7f94ace00640 1 -- 172.21.15.19:0/3052025662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81443c0 msgr2=0x7f94a806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.418+0000 7f94ace00640 1 --2- 172.21.15.19:0/3052025662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81443c0 0x7f94a806fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.418+0000 7f94ace00640 1 -- 172.21.15.19:0/3052025662 >> v1:172.21.15.169:6789/0 conn(0x7f94a80cfb20 legacy=0x7f94a80ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.418+0000 7f94ace00640 1 -- 172.21.15.19:0/3052025662 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a815f5f0 con 0x7f94a8070370 2024-04-25T11:33:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.418+0000 7f94ad800640 1 --2- 172.21.15.19:0/3052025662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81443c0 0x7f94a806fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.419+0000 7f94ace00640 1 --2- 172.21.15.19:0/3052025662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a80cf5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9494009870 tx=0x7f949402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b6c502a5a694a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.419+0000 7f94a7e00640 1 -- 172.21.15.19:0/3052025662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494037070 con 0x7f94a8070370 2024-04-25T11:33:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.419+0000 7f94a7e00640 1 -- 172.21.15.19:0/3052025662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9494032440 con 0x7f94a8070370 2024-04-25T11:33:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.419+0000 7f94a7e00640 1 -- 172.21.15.19:0/3052025662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494032810 con 0x7f94a8070370 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 -- 172.21.15.19:0/3052025662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 msgr2=0x7f94a80cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 --2- 172.21.15.19:0/3052025662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a80cf5e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9494009870 tx=0x7f949402a210 comp rx=0 tx=0).stop 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 -- 172.21.15.19:0/3052025662 shutdown_connections 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 --2- 172.21.15.19:0/3052025662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a80cf5e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 --2- 172.21.15.19:0/3052025662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a81443c0 0x7f94a806fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 -- 172.21.15.19:0/3052025662 >> 172.21.15.19:0/3052025662 conn(0x7f94a80d1ae0 msgr2=0x7f94a80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 -- 172.21.15.19:0/3052025662 shutdown_connections 2024-04-25T11:33:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.420+0000 7f94b0200640 1 -- 172.21.15.19:0/3052025662 wait complete. 2024-04-25T11:33:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.423+0000 7f94b0200640 1 Processor -- start 2024-04-25T11:33:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.423+0000 7f94b0200640 1 -- start start 2024-04-25T11:33:56.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.424+0000 7f94b0200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a813d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.424+0000 7f94b0200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80cfb20 0x7f94a813bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.424+0000 7f94b0200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81443c0 0x7f94a81f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.424+0000 7f94b0200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94a815edf0 con 0x7f94a8070370 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.424+0000 7f94b0200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94a815eb10 con 0x7f94a81443c0 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.424+0000 7f94b0200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94a806d5d0 con 0x7f94a80cfb20 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a813d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a813d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49916/0 (socket says 172.21.15.19:49916) 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 -- 172.21.15.19:0/272976441 learned_addr learned my addr 172.21.15.19:0/272976441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ae200640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81443c0 0x7f94a81f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 -- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80cfb20 msgr2=0x7f94a813bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80cfb20 0x7f94a813bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 -- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81443c0 msgr2=0x7f94a81f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81443c0 0x7f94a81f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 -- 172.21.15.19:0/272976441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81525e0 con 0x7f94a8070370 2024-04-25T11:33:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.425+0000 7f94ad800640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a813d9e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f949c00cde0 tx=0x7f949c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.426+0000 7f94a5600640 1 -- 172.21.15.19:0/272976441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c00f680 con 0x7f94a8070370 2024-04-25T11:33:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.426+0000 7f94a5600640 1 -- 172.21.15.19:0/272976441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949c00f820 con 0x7f94a8070370 2024-04-25T11:33:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.426+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a813c810 con 0x7f94a8070370 2024-04-25T11:33:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.426+0000 7f94a5600640 1 -- 172.21.15.19:0/272976441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c00fc10 con 0x7f94a8070370 2024-04-25T11:33:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.426+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a813cf00 con 0x7f94a8070370 2024-04-25T11:33:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.427+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a8050740 con 0x7f94a8070370 2024-04-25T11:33:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.428+0000 7f94a5600640 1 -- 172.21.15.19:0/272976441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f949c018400 con 0x7f94a8070370 2024-04-25T11:33:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.429+0000 7f94a5600640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.429+0000 7f94a5600640 1 -- 172.21.15.19:0/272976441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7f949c095f20 con 0x7f94a8070370 2024-04-25T11:33:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.430+0000 7f94ace00640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.431+0000 7f94ace00640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9494034000 tx=0x7f949402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.434+0000 7f94a5600640 1 -- 172.21.15.19:0/272976441 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f949c014030 con 0x7f94a8070370 2024-04-25T11:33:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.638+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94a806f950 con 0x7f94a8070370 2024-04-25T11:33:56.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.640+0000 7f94a5600640 1 -- 172.21.15.19:0/272976441 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+13280 (secure 0 0 0) 0x7f949c060860 con 0x7f94a8070370 2024-04-25T11:33:56.628 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:56.629 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":70,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:33:54.060688+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6837/1885933205":"2024-04-26T11:33:51.005421+0000","172.21.15.169:6836/1885933205":"2024-04-26T11:33:51.005421+0000","172.21.15.19:6833/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6833/751039234":"2024-04-26T11:31:25.310192+0000","172.21.15.19:6832/1883095429":"2024-04-26T11:31:58.298678+0000","172.21.15.19:6832/751039234":"2024-04-26T11:31:25.310192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:33:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 msgr2=0x7f94880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9494034000 tx=0x7f949402c000 comp rx=0 tx=0).stop 2024-04-25T11:33:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 msgr2=0x7f94a813d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a813d9e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f949c00cde0 tx=0x7f949c00a650 comp rx=0 tx=0).stop 2024-04-25T11:33:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 shutdown_connections 2024-04-25T11:33:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94880744b0 0x7f94880769c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94a81443c0 0x7f94a81f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a80cfb20 0x7f94a813bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 --2- 172.21.15.19:0/272976441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8070370 0x7f94a813d9e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.642+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 >> 172.21.15.19:0/272976441 conn(0x7f94a80d1ae0 msgr2=0x7f94a8142c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.643+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 shutdown_connections 2024-04-25T11:33:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.643+0000 7f94b0200640 1 -- 172.21.15.19:0/272976441 wait complete. 2024-04-25T11:33:56.660 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:33:56.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:33:56.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.962+0000 7f3289400640 1 Processor -- start 2024-04-25T11:33:56.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.962+0000 7f3289400640 1 -- start start 2024-04-25T11:33:56.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3289400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f32840ce930 0x7f328406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3289400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f328406db10 0x7f32840d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3289400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f328406df20 con 0x7f32840ce520 2024-04-25T11:33:56.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3289400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f32840cda60 con 0x7f328406db10 2024-04-25T11:33:56.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3289400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f328415dfe0 con 0x7f32840ce930 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3283e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f32840ce520 0x7f328415d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37580/0 (socket says 172.21.15.19:37580) 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3283e00640 1 -- 172.21.15.19:0/954421212 learned_addr learned my addr 172.21.15.19:0/954421212 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.964+0000 7f3288a00640 1 --2- 172.21.15.19:0/954421212 >> v2:172.21.15.19:3300/0 conn(0x7f328406db10 0x7f32840d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.965+0000 7f3283400640 1 --2- 172.21.15.19:0/954421212 >> v2:172.21.15.169:3300/0 conn(0x7f32840ce930 0x7f328406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.965+0000 7f3282a00640 1 -- 172.21.15.19:0/954421212 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 56162623 0 0) 0x7f3274001110 con 0x7f32840ce520 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.965+0000 7f3288a00640 1 -- 172.21.15.19:0/954421212 >> v2:172.21.15.169:3300/0 conn(0x7f32840ce930 msgr2=0x7f328406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.965+0000 7f3288a00640 1 --2- 172.21.15.19:0/954421212 >> v2:172.21.15.169:3300/0 conn(0x7f32840ce930 0x7f328406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.965+0000 7f3288a00640 1 -- 172.21.15.19:0/954421212 >> v1:172.21.15.19:6789/0 conn(0x7f32840ce520 legacy=0x7f328415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.965+0000 7f3288a00640 1 -- 172.21.15.19:0/954421212 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328415e2c0 con 0x7f328406db10 2024-04-25T11:33:56.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.965+0000 7f3283400640 1 --2- 172.21.15.19:0/954421212 >> v2:172.21.15.169:3300/0 conn(0x7f32840ce930 0x7f328406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:56.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.966+0000 7f3288a00640 1 --2- 172.21.15.19:0/954421212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f328406db10 0x7f32840d00e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f32840b5150 tx=0x7f327c02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fa54e25bcd2fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:56.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.966+0000 7f3282a00640 1 -- 172.21.15.19:0/954421212 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c007740 con 0x7f328406db10 2024-04-25T11:33:56.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.966+0000 7f3282a00640 1 -- 172.21.15.19:0/954421212 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327c0078e0 con 0x7f328406db10 2024-04-25T11:33:56.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.966+0000 7f3282a00640 1 -- 172.21.15.19:0/954421212 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c007cb0 con 0x7f328406db10 2024-04-25T11:33:56.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 -- 172.21.15.19:0/954421212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f328406db10 msgr2=0x7f32840d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 --2- 172.21.15.19:0/954421212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f328406db10 0x7f32840d00e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f32840b5150 tx=0x7f327c02c800 comp rx=0 tx=0).stop 2024-04-25T11:33:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 -- 172.21.15.19:0/954421212 shutdown_connections 2024-04-25T11:33:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 --2- 172.21.15.19:0/954421212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f328406db10 0x7f32840d00e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 --2- 172.21.15.19:0/954421212 >> v2:172.21.15.169:3300/0 conn(0x7f32840ce930 0x7f328406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 -- 172.21.15.19:0/954421212 >> 172.21.15.19:0/954421212 conn(0x7f3284072f40 msgr2=0x7f3284073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 -- 172.21.15.19:0/954421212 shutdown_connections 2024-04-25T11:33:56.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.967+0000 7f3289400640 1 -- 172.21.15.19:0/954421212 wait complete. 2024-04-25T11:33:56.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.969+0000 7f3289400640 1 Processor -- start 2024-04-25T11:33:56.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.970+0000 7f3289400640 1 -- start start 2024-04-25T11:33:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.971+0000 7f3289400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f328406db10 0x7f3284145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.971+0000 7f3289400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32840ce520 0x7f328415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.971+0000 7f3289400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840ce930 0x7f328415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.971+0000 7f3289400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3284160d30 con 0x7f32840ce520 2024-04-25T11:33:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.971+0000 7f3289400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32840ce000 con 0x7f328406db10 2024-04-25T11:33:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.971+0000 7f3289400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f328415dbc0 con 0x7f32840ce930 2024-04-25T11:33:56.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32840ce520 0x7f328415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32840ce520 0x7f328415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49942/0 (socket says 172.21.15.19:49942) 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 -- 172.21.15.19:0/1915176325 learned_addr learned my addr 172.21.15.19:0/1915176325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283e00640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f328406db10 0x7f3284145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 -- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840ce930 msgr2=0x7f328415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840ce930 0x7f328415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 -- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f328406db10 msgr2=0x7f3284145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f328406db10 0x7f3284145150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.972+0000 7f3283400640 1 -- 172.21.15.19:0/1915176325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32841500a0 con 0x7f32840ce520 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.973+0000 7f3283400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32840ce520 0x7f328415f450 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f327000c7f0 tx=0x7f327000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.973+0000 7f327be00640 1 -- 172.21.15.19:0/1915176325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270015070 con 0x7f32840ce520 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.973+0000 7f327be00640 1 -- 172.21.15.19:0/1915176325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3270007820 con 0x7f32840ce520 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.973+0000 7f327be00640 1 -- 172.21.15.19:0/1915176325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270007bd0 con 0x7f32840ce520 2024-04-25T11:33:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.973+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3284150380 con 0x7f32840ce520 2024-04-25T11:33:56.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.973+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3284150940 con 0x7f32840ce520 2024-04-25T11:33:56.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.974+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3284076c60 con 0x7f32840ce520 2024-04-25T11:33:56.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.976+0000 7f327be00640 1 -- 172.21.15.19:0/1915176325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3270018070 con 0x7f32840ce520 2024-04-25T11:33:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.976+0000 7f327be00640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32640744b0 0x7f32640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.977+0000 7f327be00640 1 -- 172.21.15.19:0/1915176325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7f3270095250 con 0x7f32840ce520 2024-04-25T11:33:56.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.977+0000 7f3283e00640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32640744b0 0x7f32640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:56.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.978+0000 7f3283e00640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32640744b0 0x7f32640769c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3274000f60 tx=0x7f3274007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:56.982+0000 7f327be00640 1 -- 172.21.15.19:0/1915176325 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f327005fb90 con 0x7f32840ce520 2024-04-25T11:33:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.195+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32840b5480 con 0x7f32840ce520 2024-04-25T11:33:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.196+0000 7f327be00640 1 -- 172.21.15.19:0/1915176325 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f327005f490 con 0x7f32840ce520 2024-04-25T11:33:57.185 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:33:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32640744b0 msgr2=0x7f32640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32640744b0 0x7f32640769c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3274000f60 tx=0x7f3274007c00 comp rx=0 tx=0).stop 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32840ce520 msgr2=0x7f328415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32840ce520 0x7f328415f450 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f327000c7f0 tx=0x7f327000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 shutdown_connections 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f32640744b0 0x7f32640769c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32840ce930 0x7f328415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32840ce520 0x7f328415f450 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.199+0000 7f3289400640 1 --2- 172.21.15.19:0/1915176325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f328406db10 0x7f3284145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.200+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 >> 172.21.15.19:0/1915176325 conn(0x7f3284072f40 msgr2=0x7f3284156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.200+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 shutdown_connections 2024-04-25T11:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.200+0000 7f3289400640 1 -- 172.21.15.19:0/1915176325 wait complete. 2024-04-25T11:33:57.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-04-25T11:33:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.516+0000 7f85f0600640 1 Processor -- start 2024-04-25T11:33:57.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.517+0000 7f85f0600640 1 -- start start 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.519+0000 7f85f0600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e814ad60 0x7f85e8070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.519+0000 7f85f0600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85e806c5c0 0x7f85e806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.519+0000 7f85f0600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e80b4af0 con 0x7f85e80b5e40 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.519+0000 7f85f0600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f85e80b4dd0 con 0x7f85e806c5c0 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.519+0000 7f85f0600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f85e80b50b0 con 0x7f85e814ad60 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.519+0000 7f85ee600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85e806c5c0 0x7f85e806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.520+0000 7f85ee600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85e806c5c0 0x7f85e806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49946/0 (socket says 172.21.15.19:49946) 2024-04-25T11:33:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.520+0000 7f85ee600640 1 -- 172.21.15.19:0/2350319996 learned_addr learned my addr 172.21.15.19:0/2350319996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.520+0000 7f85edc00640 1 --2- 172.21.15.19:0/2350319996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e814ad60 0x7f85e8070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.520+0000 7f85ee600640 1 -- 172.21.15.19:0/2350319996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e814ad60 msgr2=0x7f85e8070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.520+0000 7f85ee600640 1 --2- 172.21.15.19:0/2350319996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e814ad60 0x7f85e8070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.520+0000 7f85ee600640 1 -- 172.21.15.19:0/2350319996 >> v1:172.21.15.169:6789/0 conn(0x7f85e80b5e40 legacy=0x7f85e80b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:57.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.520+0000 7f85ee600640 1 -- 172.21.15.19:0/2350319996 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e80b5390 con 0x7f85e806c5c0 2024-04-25T11:33:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.521+0000 7f85ee600640 1 --2- 172.21.15.19:0/2350319996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e806c5c0 0x7f85e806c9d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f85e80ce2f0 tx=0x7f85e002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3d7acbbf9c5200 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.521+0000 7f85e7e00640 1 -- 172.21.15.19:0/2350319996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0036070 con 0x7f85e806c5c0 2024-04-25T11:33:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85e7e00640 1 -- 172.21.15.19:0/2350319996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e0031600 con 0x7f85e806c5c0 2024-04-25T11:33:57.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85e7e00640 1 -- 172.21.15.19:0/2350319996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00319d0 con 0x7f85e806c5c0 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85f0600640 1 -- 172.21.15.19:0/2350319996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e806c5c0 msgr2=0x7f85e806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85f0600640 1 --2- 172.21.15.19:0/2350319996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e806c5c0 0x7f85e806c9d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f85e80ce2f0 tx=0x7f85e002a630 comp rx=0 tx=0).stop 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85f0600640 1 -- 172.21.15.19:0/2350319996 shutdown_connections 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85f0600640 1 --2- 172.21.15.19:0/2350319996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e806c5c0 0x7f85e806c9d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85f0600640 1 --2- 172.21.15.19:0/2350319996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e814ad60 0x7f85e8070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.522+0000 7f85f0600640 1 -- 172.21.15.19:0/2350319996 >> 172.21.15.19:0/2350319996 conn(0x7f85e80d1ae0 msgr2=0x7f85e80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.523+0000 7f85f0600640 1 -- 172.21.15.19:0/2350319996 shutdown_connections 2024-04-25T11:33:57.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.523+0000 7f85f0600640 1 -- 172.21.15.19:0/2350319996 wait complete. 2024-04-25T11:33:57.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.525+0000 7f85f0600640 1 Processor -- start 2024-04-25T11:33:57.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.525+0000 7f85f0600640 1 -- start start 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85f0600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e806c5c0 0x7f85e81512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85f0600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85e80b5e40 0x7f85e8146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85f0600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e814ad60 0x7f85e81675c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85f0600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85e80706d0 con 0x7f85e814ad60 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85f0600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8070550 con 0x7f85e80b5e40 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85f0600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f85e806fb80 con 0x7f85e806c5c0 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e814ad60 0x7f85e81675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e814ad60 0x7f85e81675c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49958/0 (socket says 172.21.15.19:49958) 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 -- 172.21.15.19:0/1180626286 learned_addr learned my addr 172.21.15.19:0/1180626286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85edc00640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e806c5c0 0x7f85e81512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ed200640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85e80b5e40 0x7f85e8146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 -- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e806c5c0 msgr2=0x7f85e81512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:57.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e806c5c0 0x7f85e81512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 -- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85e80b5e40 msgr2=0x7f85e8146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:57.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85e80b5e40 0x7f85e8146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.527+0000 7f85ee600640 1 -- 172.21.15.19:0/1180626286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e81ed300 con 0x7f85e814ad60 2024-04-25T11:33:57.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.528+0000 7f85ee600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e814ad60 0x7f85e81675c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009f40 tx=0x7f85e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:57.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.528+0000 7f85e5600640 1 -- 172.21.15.19:0/1180626286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0036050 con 0x7f85e814ad60 2024-04-25T11:33:57.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.528+0000 7f85e5600640 1 -- 172.21.15.19:0/1180626286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e0042070 con 0x7f85e814ad60 2024-04-25T11:33:57.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.528+0000 7f85e5600640 1 -- 172.21.15.19:0/1180626286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0009510 con 0x7f85e814ad60 2024-04-25T11:33:57.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.528+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e81ed5e0 con 0x7f85e814ad60 2024-04-25T11:33:57.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.528+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e81edba0 con 0x7f85e814ad60 2024-04-25T11:33:57.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.529+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e8050740 con 0x7f85e814ad60 2024-04-25T11:33:57.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.531+0000 7f85e5600640 1 -- 172.21.15.19:0/1180626286 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f85e0007b90 con 0x7f85e814ad60 2024-04-25T11:33:57.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.532+0000 7f85e5600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85c80744b0 0x7f85c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.532+0000 7f85e5600640 1 -- 172.21.15.19:0/1180626286 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7f85e00b5ed0 con 0x7f85e814ad60 2024-04-25T11:33:57.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.533+0000 7f85edc00640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85c80744b0 0x7f85c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.534+0000 7f85edc00640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85c80744b0 0x7f85c80769c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006fd0 tx=0x7f85dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:57.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.537+0000 7f85e5600640 1 -- 172.21.15.19:0/1180626286 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85e00808a0 con 0x7f85e814ad60 2024-04-25T11:33:57.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:57.813+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7f85e80ce500 con 0x7f85e814ad60 2024-04-25T11:33:58.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.109+0000 7f85e5600640 1 -- 172.21.15.19:0/1180626286 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v334) ==== 144+0+0 (secure 0 0 0) 0x7f85e00801a0 con 0x7f85e814ad60 2024-04-25T11:33:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.112+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85c80744b0 msgr2=0x7f85c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.112+0000 7f85f0600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85c80744b0 0x7f85c80769c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006fd0 tx=0x7f85dc008040 comp rx=0 tx=0).stop 2024-04-25T11:33:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.112+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e814ad60 msgr2=0x7f85e81675c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.112+0000 7f85f0600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e814ad60 0x7f85e81675c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009f40 tx=0x7f85e0004a40 comp rx=0 tx=0).stop 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 shutdown_connections 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f85c80744b0 0x7f85c80769c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85e80b5e40 0x7f85e8146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e814ad60 0x7f85e81675c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 --2- 172.21.15.19:0/1180626286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f85e806c5c0 0x7f85e81512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 >> 172.21.15.19:0/1180626286 conn(0x7f85e80d1ae0 msgr2=0x7f85e806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 shutdown_connections 2024-04-25T11:33:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.113+0000 7f85f0600640 1 -- 172.21.15.19:0/1180626286 wait complete. 2024-04-25T11:33:58.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:33:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.445+0000 7f19abe00640 1 Processor -- start 2024-04-25T11:33:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.445+0000 7f19abe00640 1 -- start start 2024-04-25T11:33:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.448+0000 7f19abe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19ac06dce0 0x7f19ac0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.448+0000 7f19abe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19ac0cda60 con 0x7f19ac0ce930 2024-04-25T11:33:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.448+0000 7f19abe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19ac15dfe0 con 0x7f19ac0ce520 2024-04-25T11:33:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.448+0000 7f19abe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f19ac15e2c0 con 0x7f19ac06dce0 2024-04-25T11:33:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.448+0000 7f19aa000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f19ac0ce930 0x7f19ac06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37588/0 (socket says 172.21.15.19:37588) 2024-04-25T11:33:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.448+0000 7f19aa000640 1 -- 172.21.15.19:0/2018540413 learned_addr learned my addr 172.21.15.19:0/2018540413 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.448+0000 7f19ab400640 1 --2- 172.21.15.19:0/2018540413 >> v2:172.21.15.169:3300/0 conn(0x7f19ac06dce0 0x7f19ac0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4106077029 0 0) 0x7f19ac0cda60 con 0x7f19ac0ce930 2024-04-25T11:33:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1990003520 con 0x7f19ac0ce930 2024-04-25T11:33:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 858235872 0 0) 0x7f1990003520 con 0x7f19ac0ce930 2024-04-25T11:33:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 >> v2:172.21.15.169:3300/0 conn(0x7f19ac06dce0 msgr2=0x7f19ac0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 --2- 172.21.15.19:0/2018540413 >> v2:172.21.15.169:3300/0 conn(0x7f19ac06dce0 0x7f19ac0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 >> v1:172.21.15.169:6789/0 conn(0x7f19ac0ce520 legacy=0x7f19ac15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac0cdd20 con 0x7f19ac0ce930 2024-04-25T11:33:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1994001670 con 0x7f19ac0ce930 2024-04-25T11:33:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 >> v1:172.21.15.19:6789/0 conn(0x7f19ac0ce930 legacy=0x7f19ac06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19900039b0 0x7f1990003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1990004800 0x7f1990004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1990005250 0x7f199000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac15e2c0 con 0x7f1990005250 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f199000c1e0 con 0x7f19900039b0 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.449+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f199000e5f0 con 0x7f1990004800 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.450+0000 7f19ab400640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1990005250 0x7f199000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.450+0000 7f19aaa00640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19900039b0 0x7f1990003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.450+0000 7f19ab400640 1 -- 172.21.15.19:0/2018540413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1990004800 msgr2=0x7f1990004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.450+0000 7f19ab400640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1990004800 0x7f1990004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.450+0000 7f19ab400640 1 -- 172.21.15.19:0/2018540413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19900039b0 msgr2=0x7f1990003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:58.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.450+0000 7f19ab400640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19900039b0 0x7f1990003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.450+0000 7f19ab400640 1 -- 172.21.15.19:0/2018540413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990003520 con 0x7f1990005250 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.451+0000 7f19ab400640 1 -- 172.21.15.19:0/2018540413 reap_dead start 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.451+0000 7f19ab400640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1990005250 0x7f199000b9c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009fc0 tx=0x7f1990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b0ba80d686411d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.451+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4035020 con 0x7f1990005250 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.451+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a4003bd0 con 0x7f1990005250 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.451+0000 7f19a9600640 1 -- 172.21.15.19:0/2018540413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4009c10 con 0x7f1990005250 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.452+0000 7f19abe00640 1 -- 172.21.15.19:0/2018540413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1990005250 msgr2=0x7f199000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.452+0000 7f19abe00640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1990005250 0x7f199000b9c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009fc0 tx=0x7f1990005830 comp rx=0 tx=0).stop 2024-04-25T11:33:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.452+0000 7f19abe00640 1 -- 172.21.15.19:0/2018540413 shutdown_connections 2024-04-25T11:33:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.452+0000 7f19abe00640 1 --2- 172.21.15.19:0/2018540413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1990005250 0x7f199000b9c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.452+0000 7f19abe00640 1 -- 172.21.15.19:0/2018540413 >> 172.21.15.19:0/2018540413 conn(0x7f19ac072f40 msgr2=0x7f19ac073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.452+0000 7f19abe00640 1 -- 172.21.15.19:0/2018540413 shutdown_connections 2024-04-25T11:33:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.452+0000 7f19abe00640 1 -- 172.21.15.19:0/2018540413 wait complete. 2024-04-25T11:33:58.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.455+0000 7f19abe00640 1 Processor -- start 2024-04-25T11:33:58.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.455+0000 7f19abe00640 1 -- start start 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19abe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1990005250 0x7f19ac146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19abe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19ac145950 0x7f19ac148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19abe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19ac070620 0x7f19ac070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19abe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac0b73d0 con 0x7f19ac070620 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19abe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac160d30 con 0x7f19ac145950 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19abe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19ac0b7550 con 0x7f1990005250 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19ac070620 0x7f19ac070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19ac070620 0x7f19ac070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49972/0 (socket says 172.21.15.19:49972) 2024-04-25T11:33:58.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 -- 172.21.15.19:0/3376933543 learned_addr learned my addr 172.21.15.19:0/3376933543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19aa000640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19ac145950 0x7f19ac148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 -- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1990005250 msgr2=0x7f19ac146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1990005250 0x7f19ac146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 -- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19ac145950 msgr2=0x7f19ac148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19ac145950 0x7f19ac148b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.457+0000 7f19ab400640 1 -- 172.21.15.19:0/3376933543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac070f70 con 0x7f19ac070620 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.458+0000 7f19aa000640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19ac145950 0x7f19ac148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.458+0000 7f19ab400640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19ac070620 0x7f19ac070a30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e230 tx=0x7f19a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:58.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.458+0000 7f19a2a00640 1 -- 172.21.15.19:0/3376933543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4035030 con 0x7f19ac070620 2024-04-25T11:33:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.458+0000 7f19a2a00640 1 -- 172.21.15.19:0/3376933543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a4043070 con 0x7f19ac070620 2024-04-25T11:33:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.458+0000 7f19a2a00640 1 -- 172.21.15.19:0/3376933543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a402d070 con 0x7f19ac070620 2024-04-25T11:33:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.458+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ac157990 con 0x7f19ac070620 2024-04-25T11:33:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.458+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ac160770 con 0x7f19ac070620 2024-04-25T11:33:58.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.460+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ac076c60 con 0x7f19ac070620 2024-04-25T11:33:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.461+0000 7f19a2a00640 1 -- 172.21.15.19:0/3376933543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f19a4005d10 con 0x7f19ac070620 2024-04-25T11:33:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.462+0000 7f19a2a00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19880744b0 0x7f19880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.462+0000 7f19a2a00640 1 -- 172.21.15.19:0/3376933543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7f19a40b5fe0 con 0x7f19ac070620 2024-04-25T11:33:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.463+0000 7f19aaa00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19880744b0 0x7f19880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.464+0000 7f19aaa00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19880744b0 0x7f19880769c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f199c007fd0 tx=0x7f199c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.467+0000 7f19a2a00640 1 -- 172.21.15.19:0/3376933543 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19ac070f70 con 0x7f19ac070620 2024-04-25T11:33:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.669+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f19ac0b5480 con 0x7f19ac070620 2024-04-25T11:33:58.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.677+0000 7f19a2a00640 1 -- 172.21.15.19:0/3376933543 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f19a4080920 con 0x7f19ac070620 2024-04-25T11:33:58.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19880744b0 msgr2=0x7f19880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19880744b0 0x7f19880769c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f199c007fd0 tx=0x7f199c009040 comp rx=0 tx=0).stop 2024-04-25T11:33:58.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19ac070620 msgr2=0x7f19ac070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19ac070620 0x7f19ac070a30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f19a400e230 tx=0x7f19a4009750 comp rx=0 tx=0).stop 2024-04-25T11:33:58.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 shutdown_connections 2024-04-25T11:33:58.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19880744b0 0x7f19880769c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19ac145950 0x7f19ac148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19ac070620 0x7f19ac070a30 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 --2- 172.21.15.19:0/3376933543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1990005250 0x7f19ac146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.680+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 >> 172.21.15.19:0/3376933543 conn(0x7f19ac072f40 msgr2=0x7f19ac0b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:58.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.681+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 shutdown_connections 2024-04-25T11:33:58.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:58.681+0000 7f19abe00640 1 -- 172.21.15.19:0/3376933543 wait complete. 2024-04-25T11:33:58.696 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:33:58.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:33:58.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:33:58.906 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:33:58.907 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:33:58.907 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:58.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:33:58.964 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:33:58.964 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:59.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:33:59.022 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:33:59.022 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:59.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:33:59.084 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:33:59.084 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:33:59.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:33:59.142 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:33:59.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:33:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.444+0000 7f8839800640 1 Processor -- start 2024-04-25T11:33:59.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.444+0000 7f8839800640 1 -- start start 2024-04-25T11:33:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.446+0000 7f8839800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8834071e90 0x7f88340b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.446+0000 7f8839800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88341607c0 con 0x7f88340b6270 2024-04-25T11:33:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.446+0000 7f8839800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8834160a80 con 0x7f88340d06c0 2024-04-25T11:33:59.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.446+0000 7f8839800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f88340b6ca0 con 0x7f8834071e90 2024-04-25T11:33:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.447+0000 7f8833e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f88340b6270 0x7f8834071780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37594/0 (socket says 172.21.15.19:37594) 2024-04-25T11:33:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.447+0000 7f8833e00640 1 -- 172.21.15.19:0/968455703 learned_addr learned my addr 172.21.15.19:0/968455703 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:33:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.447+0000 7f8833400640 1 --2- 172.21.15.19:0/968455703 >> v2:172.21.15.169:3300/0 conn(0x7f8834071e90 0x7f88340b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.447+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750617135 0 0) 0x7f88341607c0 con 0x7f88340b6270 2024-04-25T11:33:59.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.447+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8818003520 con 0x7f88340b6270 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3071746153 0 0) 0x7f8818003520 con 0x7f88340b6270 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 >> v2:172.21.15.169:3300/0 conn(0x7f8834071e90 msgr2=0x7f88340b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 --2- 172.21.15.19:0/968455703 >> v2:172.21.15.169:3300/0 conn(0x7f8834071e90 0x7f88340b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 >> v1:172.21.15.169:6789/0 conn(0x7f88340d06c0 legacy=0x7f88340d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8834160d60 con 0x7f88340b6270 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8824001670 con 0x7f88340b6270 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 >> v1:172.21.15.19:6789/0 conn(0x7f88340b6270 legacy=0x7f8834071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8818003a00 0x7f8818003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818004800 0x7f8818004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8818005270 0x7f881800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88340b6ca0 con 0x7f8818005270 2024-04-25T11:33:59.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f881800c190 con 0x7f8818004800 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f881800c390 con 0x7f8818003a00 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.448+0000 7f8838e00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8818005270 0x7f881800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8833400640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818004800 0x7f8818004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8833e00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8818003a00 0x7f8818003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8838e00640 1 -- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8818003a00 msgr2=0x7f8818003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8838e00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8818003a00 0x7f8818003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8838e00640 1 -- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818004800 msgr2=0x7f8818004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8838e00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818004800 0x7f8818004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8838e00640 1 -- 172.21.15.19:0/968455703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8818003520 con 0x7f8818005270 2024-04-25T11:33:59.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.449+0000 7f8833400640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818004800 0x7f8818004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:33:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.450+0000 7f8838e00640 1 -- 172.21.15.19:0/968455703 reap_dead start 2024-04-25T11:33:59.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.450+0000 7f8838e00640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8818005270 0x7f881800b9c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f882c001e50 tx=0x7f8818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9860e4eba1295811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.451+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c001a70 con 0x7f8818005270 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.451+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882c003970 con 0x7f8818005270 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.451+0000 7f8832a00640 1 -- 172.21.15.19:0/968455703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c003d60 con 0x7f8818005270 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.451+0000 7f8839800640 1 -- 172.21.15.19:0/968455703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8818005270 msgr2=0x7f881800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.451+0000 7f8839800640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8818005270 0x7f881800b9c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f882c001e50 tx=0x7f8818005830 comp rx=0 tx=0).stop 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.452+0000 7f8839800640 1 -- 172.21.15.19:0/968455703 shutdown_connections 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.452+0000 7f8839800640 1 --2- 172.21.15.19:0/968455703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8818005270 0x7f881800b9c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.452+0000 7f8839800640 1 -- 172.21.15.19:0/968455703 >> 172.21.15.19:0/968455703 conn(0x7f8834070270 msgr2=0x7f8834070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.452+0000 7f8839800640 1 -- 172.21.15.19:0/968455703 shutdown_connections 2024-04-25T11:33:59.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.452+0000 7f8839800640 1 -- 172.21.15.19:0/968455703 wait complete. 2024-04-25T11:33:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.454+0000 7f8839800640 1 Processor -- start 2024-04-25T11:33:59.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.454+0000 7f8839800640 1 -- start start 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8839800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818005270 0x7f883414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8839800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88341fa870 0x7f88341f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8839800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88341fa1a0 0x7f88341e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8839800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88340b1c20 con 0x7f88341fa1a0 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8839800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88341613b0 con 0x7f8818005270 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8839800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88340b1da0 con 0x7f88341fa870 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8838e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88341fa1a0 0x7f88341e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8838e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88341fa1a0 0x7f88341e96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49992/0 (socket says 172.21.15.19:49992) 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.456+0000 7f8838e00640 1 -- 172.21.15.19:0/1533302436 learned_addr learned my addr 172.21.15.19:0/1533302436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:33:59.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f8833400640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88341fa870 0x7f88341f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f8838e00640 1 -- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88341fa870 msgr2=0x7f88341f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f8838e00640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88341fa870 0x7f88341f9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f8838e00640 1 -- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818005270 msgr2=0x7f883414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f8838e00640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818005270 0x7f883414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f8838e00640 1 -- 172.21.15.19:0/1533302436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88341fa5b0 con 0x7f88341fa1a0 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f8838e00640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88341fa1a0 0x7f88341e96a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f882c02a920 tx=0x7f882c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f882be00640 1 -- 172.21.15.19:0/1533302436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c001a70 con 0x7f88341fa1a0 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f882be00640 1 -- 172.21.15.19:0/1533302436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882c001250 con 0x7f88341fa1a0 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.457+0000 7f882be00640 1 -- 172.21.15.19:0/1533302436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c006d70 con 0x7f88341fa1a0 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.458+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883415b860 con 0x7f88341fa1a0 2024-04-25T11:33:59.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.458+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88341e8ea0 con 0x7f88341fa1a0 2024-04-25T11:33:59.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.459+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8834050740 con 0x7f88341fa1a0 2024-04-25T11:33:59.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.460+0000 7f882be00640 1 -- 172.21.15.19:0/1533302436 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f882c007030 con 0x7f88341fa1a0 2024-04-25T11:33:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.461+0000 7f882be00640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88140744b0 0x7f88140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.461+0000 7f882be00640 1 -- 172.21.15.19:0/1533302436 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7f882c0b4950 con 0x7f88341fa1a0 2024-04-25T11:33:59.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.462+0000 7f8833e00640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88140744b0 0x7f88140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.463+0000 7f8833e00640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88140744b0 0x7f88140769c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8824000fd0 tx=0x7f8824009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:59.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.466+0000 7f882be00640 1 -- 172.21.15.19:0/1533302436 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88341fa5b0 con 0x7f88341fa1a0 2024-04-25T11:33:59.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.712+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88340b1660 con 0x7f88341fa1a0 2024-04-25T11:33:59.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.713+0000 7f882be00640 1 -- 172.21.15.19:0/1533302436 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+3875 (secure 0 0 0) 0x7f882c07f320 con 0x7f88341fa1a0 2024-04-25T11:33:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:33:59.702 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1583075284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1583075284},{"type":"v1","addr":"172.21.15.169:6835","nonce":1583075284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5363,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1071356157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1071356157},{"type":"v1","addr":"172.21.15.169:6837","nonce":1071356157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":67},{"gid":5443,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2125434620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2125434620},{"type":"v1","addr":"172.21.15.19:6835","nonce":2125434620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3770220424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3770220424},{"type":"v1","addr":"172.21.15.19:6833","nonce":3770220424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[]} 2024-04-25T11:33:59.702 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 67 2024-04-25T11:33:59.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88140744b0 msgr2=0x7f88140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88140744b0 0x7f88140769c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8824000fd0 tx=0x7f8824009040 comp rx=0 tx=0).stop 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88341fa1a0 msgr2=0x7f88341e96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88341fa1a0 0x7f88341e96a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f882c02a920 tx=0x7f882c0014a0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 shutdown_connections 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88140744b0 0x7f88140769c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88341fa1a0 0x7f88341e96a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88341fa870 0x7f88341f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 --2- 172.21.15.19:0/1533302436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8818005270 0x7f883414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 >> 172.21.15.19:0/1533302436 conn(0x7f8834070270 msgr2=0x7f8834160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.716+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 shutdown_connections 2024-04-25T11:33:59.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:33:59.717+0000 7f8839800640 1 -- 172.21.15.19:0/1533302436 wait complete. 2024-04-25T11:33:59.735 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:33:59.735 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:33:59.736 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:33:59.736 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:33:59.736 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:33:59.736 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:33:59.736 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:33:59.736 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:33:59.736 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:33:59.736 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:33:59.737 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:33:59.737 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:33:59.737 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:33:59.737 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:33:59.737 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:33:59.737 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:33:59.764 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:33:59.765 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:33:59.765 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:33:59.766 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:33:59.768 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:33:59.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:33:59.779 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:33:59.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:33:59.810 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:33:59.810 DEBUG:teuthology.parallel:result is None 2024-04-25T11:33:59.813 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:33:59.814 DEBUG:teuthology.parallel:result is None 2024-04-25T11:33:59.826 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:33:59.826 DEBUG:teuthology.parallel:result is None 2024-04-25T11:33:59.832 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:33:59.833 DEBUG:teuthology.parallel:result is None 2024-04-25T11:33:59.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:34:00.038 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:34:00.038 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:34:00.052 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:34:00.055 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:34:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.103+0000 7fadb7200640 1 Processor -- start 2024-04-25T11:34:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.103+0000 7fadb7200640 1 -- start start 2024-04-25T11:34:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.105+0000 7fadb7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadb80d1070 0x7fadb8162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.105+0000 7fadb7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb8144e60 0x7fadb8145270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.105+0000 7fadb7200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb806fc70 con 0x7fadb8144350 2024-04-25T11:34:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.105+0000 7fadb7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fadb806ff50 con 0x7fadb80d1070 2024-04-25T11:34:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.105+0000 7fadb7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadb8070230 con 0x7fadb8144e60 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.106+0000 7fadafe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadb80d1070 0x7fadb8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.106+0000 7fadb6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb8144e60 0x7fadb8145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.106+0000 7fadb5e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fadb8144350 0x7fadb8161af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:60182/0 (socket says 172.21.15.19:60182) 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.106+0000 7fadb5e00640 1 -- 172.21.15.19:0/3388826469 learned_addr learned my addr 172.21.15.19:0/3388826469 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.106+0000 7fadb6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb8144e60 0x7fadb8145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:42140/0 (socket says 172.21.15.19:42140) 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.106+0000 7fadafe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fadb80d1070 0x7fadb8162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:53538/0 (socket says 172.21.15.19:53538) 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.107+0000 7fadb5400640 1 -- 172.21.15.19:0/3388826469 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618131068 0 0) 0x7fadb806fc70 con 0x7fadb8144350 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.107+0000 7fadb6800640 1 -- 172.21.15.19:0/3388826469 >> v2:172.21.15.169:3300/0 conn(0x7fadb80d1070 msgr2=0x7fadb8162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:00.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.107+0000 7fadb6800640 1 --2- 172.21.15.19:0/3388826469 >> v2:172.21.15.169:3300/0 conn(0x7fadb80d1070 0x7fadb8162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.107+0000 7fadb6800640 1 -- 172.21.15.19:0/3388826469 >> v1:172.21.15.169:6789/0 conn(0x7fadb8144350 legacy=0x7fadb8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.107+0000 7fadb6800640 1 -- 172.21.15.19:0/3388826469 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb8070510 con 0x7fadb8144e60 2024-04-25T11:34:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.108+0000 7fadb6800640 1 --2- 172.21.15.19:0/3388826469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb8144e60 0x7fadb8145270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009d90 tx=0x7fadb002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9346403f88d34d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb5400640 1 -- 172.21.15.19:0/3388826469 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0007740 con 0x7fadb8144e60 2024-04-25T11:34:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb5400640 1 -- 172.21.15.19:0/3388826469 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb00078e0 con 0x7fadb8144e60 2024-04-25T11:34:00.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb5400640 1 -- 172.21.15.19:0/3388826469 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0007cb0 con 0x7fadb8144e60 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb7200640 1 -- 172.21.15.19:0/3388826469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb8144e60 msgr2=0x7fadb8145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb7200640 1 --2- 172.21.15.19:0/3388826469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb8144e60 0x7fadb8145270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009d90 tx=0x7fadb002a660 comp rx=0 tx=0).stop 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb7200640 1 -- 172.21.15.19:0/3388826469 shutdown_connections 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb7200640 1 --2- 172.21.15.19:0/3388826469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb8144e60 0x7fadb8145270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb7200640 1 --2- 172.21.15.19:0/3388826469 >> v2:172.21.15.169:3300/0 conn(0x7fadb80d1070 0x7fadb8162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb7200640 1 -- 172.21.15.19:0/3388826469 >> 172.21.15.19:0/3388826469 conn(0x7fadb80d1720 msgr2=0x7fadb80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.109+0000 7fadb7200640 1 -- 172.21.15.19:0/3388826469 shutdown_connections 2024-04-25T11:34:00.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.110+0000 7fadb7200640 1 -- 172.21.15.19:0/3388826469 wait complete. 2024-04-25T11:34:00.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.112+0000 7fadb7200640 1 Processor -- start 2024-04-25T11:34:00.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.112+0000 7fadb7200640 1 -- start start 2024-04-25T11:34:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb80d1070 0x7fadb80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb8144350 0x7fadb8140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadb8144e60 0x7fadb81e9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8162740 con 0x7fadb8144350 2024-04-25T11:34:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8147910 con 0x7fadb8144e60 2024-04-25T11:34:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fadb81628c0 con 0x7fadb80d1070 2024-04-25T11:34:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb8144350 0x7fadb8140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb8144350 0x7fadb8140a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50008/0 (socket says 172.21.15.19:50008) 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadafe00640 1 -- 172.21.15.19:0/692101459 learned_addr learned my addr 172.21.15.19:0/692101459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb5e00640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb80d1070 0x7fadb80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadb6800640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadb8144e60 0x7fadb81e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadafe00640 1 -- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb80d1070 msgr2=0x7fadb80cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.114+0000 7fadafe00640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb80d1070 0x7fadb80cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadafe00640 1 -- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadb8144e60 msgr2=0x7fadb81e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadafe00640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadb8144e60 0x7fadb81e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadafe00640 1 -- 172.21.15.19:0/692101459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb80cc400 con 0x7fadb8144350 2024-04-25T11:34:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadafe00640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb8144350 0x7fadb8140a10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fada000e870 tx=0x7fada000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadae000640 1 -- 172.21.15.19:0/692101459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0017070 con 0x7fadb8144350 2024-04-25T11:34:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadae000640 1 -- 172.21.15.19:0/692101459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada000c940 con 0x7fadb8144350 2024-04-25T11:34:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb80ce750 con 0x7fadb8144350 2024-04-25T11:34:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadae000640 1 -- 172.21.15.19:0/692101459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada000cd30 con 0x7fadb8144350 2024-04-25T11:34:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.115+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb80ced10 con 0x7fadb8144350 2024-04-25T11:34:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.117+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb8050740 con 0x7fadb8144350 2024-04-25T11:34:00.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.118+0000 7fadae000640 1 -- 172.21.15.19:0/692101459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fada001a070 con 0x7fadb8144350 2024-04-25T11:34:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.119+0000 7fadae000640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad940744b0 0x7fad940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.120+0000 7fadae000640 1 -- 172.21.15.19:0/692101459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5115+0+0 (secure 0 0 0) 0x7fada0094e10 con 0x7fadb8144350 2024-04-25T11:34:00.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.120+0000 7fadb5e00640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad940744b0 0x7fad940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.121+0000 7fadb5e00640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad940744b0 0x7fad940769c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fada8000f60 tx=0x7fada8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:00.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.124+0000 7fadae000640 1 -- 172.21.15.19:0/692101459 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fada0098030 con 0x7fadb8144350 2024-04-25T11:34:00.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:00.338+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fadb806d910 con 0x7fadb8144350 2024-04-25T11:34:01.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.172+0000 7fadae000640 1 -- 172.21.15.19:0/692101459 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) ==== 97+0+0 (secure 0 0 0) 0x7fada005f190 con 0x7fadb8144350 2024-04-25T11:34:01.161 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad940744b0 msgr2=0x7fad940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad940744b0 0x7fad940769c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fada8000f60 tx=0x7fada8009040 comp rx=0 tx=0).stop 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb8144350 msgr2=0x7fadb8140a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb8144350 0x7fadb8140a10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fada000e870 tx=0x7fada000ed30 comp rx=0 tx=0).stop 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 shutdown_connections 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad940744b0 0x7fad940769c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fadb8144e60 0x7fadb81e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fadb8144350 0x7fadb8140a10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 --2- 172.21.15.19:0/692101459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fadb80d1070 0x7fadb80cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 >> 172.21.15.19:0/692101459 conn(0x7fadb80d1720 msgr2=0x7fadb80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.176+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 shutdown_connections 2024-04-25T11:34:01.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.177+0000 7fadb7200640 1 -- 172.21.15.19:0/692101459 wait complete. 2024-04-25T11:34:01.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:34:01.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.483+0000 7f8178600640 1 Processor -- start 2024-04-25T11:34:01.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.483+0000 7f8178600640 1 -- start start 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.485+0000 7f8178600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81701434d0 0x7f8170070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.485+0000 7f8178600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f81700d0ed0 0x7f81700d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.485+0000 7f8178600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8170160be0 0x7f8170160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.485+0000 7f8178600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f81701438e0 con 0x7f8170160be0 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.485+0000 7f8178600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8170161530 con 0x7f81700d0ed0 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.485+0000 7f8178600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f81700709c0 con 0x7f81701434d0 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8170160be0 0x7f8170160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8170160be0 0x7f8170160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50020/0 (socket says 172.21.15.19:50020) 2024-04-25T11:34:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 -- 172.21.15.19:0/2655131379 learned_addr learned my addr 172.21.15.19:0/2655131379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8175c00640 1 --2- 172.21.15.19:0/2655131379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81701434d0 0x7f8170070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8175200640 1 --2- 172.21.15.19:0/2655131379 >> v2:172.21.15.169:3300/0 conn(0x7f81700d0ed0 0x7f81700d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:01.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 -- 172.21.15.19:0/2655131379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81701434d0 msgr2=0x7f8170070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 --2- 172.21.15.19:0/2655131379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81701434d0 0x7f8170070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 -- 172.21.15.19:0/2655131379 >> v2:172.21.15.169:3300/0 conn(0x7f81700d0ed0 msgr2=0x7f81700d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 --2- 172.21.15.19:0/2655131379 >> v2:172.21.15.169:3300/0 conn(0x7f81700d0ed0 0x7f81700d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8176600640 1 -- 172.21.15.19:0/2655131379 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817006f3e0 con 0x7f8170160be0 2024-04-25T11:34:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.486+0000 7f8175200640 1 --2- 172.21.15.19:0/2655131379 >> v2:172.21.15.169:3300/0 conn(0x7f81700d0ed0 0x7f81700d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:01.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.488+0000 7f8176600640 1 --2- 172.21.15.19:0/2655131379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f8170160ff0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8170070cb0 tx=0x7f816802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1441b98b7d25a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.488+0000 7f816fe00640 1 -- 172.21.15.19:0/2655131379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168032790 con 0x7f8170160be0 2024-04-25T11:34:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.488+0000 7f816fe00640 1 -- 172.21.15.19:0/2655131379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168038070 con 0x7f8170160be0 2024-04-25T11:34:01.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.488+0000 7f816fe00640 1 -- 172.21.15.19:0/2655131379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168032b50 con 0x7f8170160be0 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 -- 172.21.15.19:0/2655131379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 msgr2=0x7f8170160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 --2- 172.21.15.19:0/2655131379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f8170160ff0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8170070cb0 tx=0x7f816802c890 comp rx=0 tx=0).stop 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 -- 172.21.15.19:0/2655131379 shutdown_connections 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 --2- 172.21.15.19:0/2655131379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f8170160ff0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 --2- 172.21.15.19:0/2655131379 >> v2:172.21.15.169:3300/0 conn(0x7f81700d0ed0 0x7f81700d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 --2- 172.21.15.19:0/2655131379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81701434d0 0x7f8170070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 -- 172.21.15.19:0/2655131379 >> 172.21.15.19:0/2655131379 conn(0x7f81700d1ae0 msgr2=0x7f81700d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 -- 172.21.15.19:0/2655131379 shutdown_connections 2024-04-25T11:34:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.489+0000 7f8178600640 1 -- 172.21.15.19:0/2655131379 wait complete. 2024-04-25T11:34:01.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.491+0000 7f8178600640 1 Processor -- start 2024-04-25T11:34:01.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.491+0000 7f8178600640 1 -- start start 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8178600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81700d0ed0 0x7f817015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8178600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81701434d0 0x7f817015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8178600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f817015ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8178600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f817006fc80 con 0x7f8170160be0 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8178600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f817006f9a0 con 0x7f81701434d0 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8178600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f817006f090 con 0x7f81700d0ed0 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8176600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f817015ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8176600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f817015ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50030/0 (socket says 172.21.15.19:50030) 2024-04-25T11:34:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.493+0000 7f8176600640 1 -- 172.21.15.19:0/3839226903 learned_addr learned my addr 172.21.15.19:0/3839226903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8175c00640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81700d0ed0 0x7f817015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8175200640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81701434d0 0x7f817015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8176600640 1 -- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81700d0ed0 msgr2=0x7f817015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8176600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81700d0ed0 0x7f817015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8176600640 1 -- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81701434d0 msgr2=0x7f817015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8176600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81701434d0 0x7f817015e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8176600640 1 -- 172.21.15.19:0/3839226903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81701e6ff0 con 0x7f8170160be0 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8175c00640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81700d0ed0 0x7f817015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.494+0000 7f8176600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f817015ec60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f816800b2c0 tx=0x7f8168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.495+0000 7f816d600640 1 -- 172.21.15.19:0/3839226903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168043070 con 0x7f8170160be0 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.495+0000 7f816d600640 1 -- 172.21.15.19:0/3839226903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816803c620 con 0x7f8170160be0 2024-04-25T11:34:01.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.495+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81701e7260 con 0x7f8170160be0 2024-04-25T11:34:01.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.495+0000 7f816d600640 1 -- 172.21.15.19:0/3839226903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816803ca10 con 0x7f8170160be0 2024-04-25T11:34:01.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.495+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701e8dc0 con 0x7f8170160be0 2024-04-25T11:34:01.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.496+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81700b52c0 con 0x7f8170160be0 2024-04-25T11:34:01.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.497+0000 7f816d600640 1 -- 172.21.15.19:0/3839226903 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8168046070 con 0x7f8170160be0 2024-04-25T11:34:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.498+0000 7f816d600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81540744b0 0x7f81540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.498+0000 7f816d600640 1 -- 172.21.15.19:0/3839226903 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7f81680b5710 con 0x7f8170160be0 2024-04-25T11:34:01.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.499+0000 7f8175c00640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81540744b0 0x7f81540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:01.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.500+0000 7f8175c00640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81540744b0 0x7f81540769c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8164006fd0 tx=0x7f8164008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:01.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.503+0000 7f816d600640 1 -- 172.21.15.19:0/3839226903 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81700b52c0 con 0x7f8170160be0 2024-04-25T11:34:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.787+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8170050740 con 0x7f8170160be0 2024-04-25T11:34:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.788+0000 7f816d600640 1 -- 172.21.15.19:0/3839226903 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f8168080150 con 0x7f8170160be0 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ==", 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw==", 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw==", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA==", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ==", 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw==", 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q==", 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw==", 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw==", 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw==", 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ==", 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA==", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w==", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T11:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q==", 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg==", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q==", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNPSpmHT+MHRAAPZNLFcl/fxDgWnMeN3nAHw==", 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA==", 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA==", 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81540744b0 msgr2=0x7f81540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81540744b0 0x7f81540769c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8164006fd0 tx=0x7f8164008040 comp rx=0 tx=0).stop 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 msgr2=0x7f817015ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f817015ec60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f816800b2c0 tx=0x7f8168004730 comp rx=0 tx=0).stop 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 shutdown_connections 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f81540744b0 0x7f81540769c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81701434d0 0x7f817015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8170160be0 0x7f817015ec60 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 --2- 172.21.15.19:0/3839226903 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81700d0ed0 0x7f817015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 >> 172.21.15.19:0/3839226903 conn(0x7f81700d1ae0 msgr2=0x7f81700d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 shutdown_connections 2024-04-25T11:34:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:01.791+0000 7f8178600640 1 -- 172.21.15.19:0/3839226903 wait complete. 2024-04-25T11:34:01.806 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.101+0000 7f3769400640 1 Processor -- start 2024-04-25T11:34:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.101+0000 7f3769400640 1 -- start start 2024-04-25T11:34:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.103+0000 7f3769400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f37640709a0 0x7f37641569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.103+0000 7f3769400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37640cffc0 con 0x7f37640b0800 2024-04-25T11:34:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.103+0000 7f3769400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37640d02a0 con 0x7f37640cf9e0 2024-04-25T11:34:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.103+0000 7f3769400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f37640d0580 con 0x7f37640709a0 2024-04-25T11:34:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.104+0000 7f375b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f37640b0800 0x7f3764070290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37610/0 (socket says 172.21.15.19:37610) 2024-04-25T11:34:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.104+0000 7f375b400640 1 -- 172.21.15.19:0/2797941445 learned_addr learned my addr 172.21.15.19:0/2797941445 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:02.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.104+0000 7f3768a00640 1 --2- 172.21.15.19:0/2797941445 >> v2:172.21.15.19:3300/0 conn(0x7f37640709a0 0x7f37641569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.104+0000 7f3763400640 1 -- 172.21.15.19:0/2797941445 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1987878624 0 0) 0x7f37640cffc0 con 0x7f37640b0800 2024-04-25T11:34:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.104+0000 7f3763400640 1 -- 172.21.15.19:0/2797941445 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3748003520 con 0x7f37640b0800 2024-04-25T11:34:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.104+0000 7f3768a00640 1 -- 172.21.15.19:0/2797941445 >> v1:172.21.15.169:6789/0 conn(0x7f37640cf9e0 legacy=0x7f3764161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.104+0000 7f3768a00640 1 -- 172.21.15.19:0/2797941445 >> v1:172.21.15.19:6789/0 conn(0x7f37640b0800 legacy=0x7f3764070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.105+0000 7f3768a00640 1 -- 172.21.15.19:0/2797941445 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37640d0860 con 0x7f37640709a0 2024-04-25T11:34:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.106+0000 7f3768a00640 1 --2- 172.21.15.19:0/2797941445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640709a0 0x7f37641569b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f375c009d90 tx=0x7f375c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dced5c7552f31fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.106+0000 7f3763400640 1 -- 172.21.15.19:0/2797941445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c007740 con 0x7f37640709a0 2024-04-25T11:34:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.106+0000 7f3763400640 1 -- 172.21.15.19:0/2797941445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c0078e0 con 0x7f37640709a0 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.106+0000 7f3763400640 1 -- 172.21.15.19:0/2797941445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c007c90 con 0x7f37640709a0 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.107+0000 7f3769400640 1 -- 172.21.15.19:0/2797941445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640709a0 msgr2=0x7f37641569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.107+0000 7f3769400640 1 --2- 172.21.15.19:0/2797941445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640709a0 0x7f37641569b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f375c009d90 tx=0x7f375c02a660 comp rx=0 tx=0).stop 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.107+0000 7f3769400640 1 -- 172.21.15.19:0/2797941445 shutdown_connections 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.107+0000 7f3769400640 1 --2- 172.21.15.19:0/2797941445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640709a0 0x7f37641569b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.107+0000 7f3769400640 1 -- 172.21.15.19:0/2797941445 >> 172.21.15.19:0/2797941445 conn(0x7f37640d1720 msgr2=0x7f37640d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.107+0000 7f3769400640 1 -- 172.21.15.19:0/2797941445 shutdown_connections 2024-04-25T11:34:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.107+0000 7f3769400640 1 -- 172.21.15.19:0/2797941445 wait complete. 2024-04-25T11:34:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.109+0000 7f3769400640 1 Processor -- start 2024-04-25T11:34:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.110+0000 7f3769400640 1 -- start start 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.111+0000 7f3769400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37640b0800 0x7f37641560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.111+0000 7f3769400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640cf9e0 0x7f37641590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.111+0000 7f3769400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3764157b20 0x7f37641620e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.111+0000 7f3769400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f376406d9b0 con 0x7f37640cf9e0 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.111+0000 7f3769400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f37640b0e60 con 0x7f37640b0800 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.111+0000 7f3769400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f376406db30 con 0x7f3764157b20 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.111+0000 7f375b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640cf9e0 0x7f37641590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f375b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640cf9e0 0x7f37641590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50048/0 (socket says 172.21.15.19:50048) 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f375b400640 1 -- 172.21.15.19:0/3742687302 learned_addr learned my addr 172.21.15.19:0/3742687302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f3763e00640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37640b0800 0x7f37641560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f3768a00640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3764157b20 0x7f37641620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f375b400640 1 -- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3764157b20 msgr2=0x7f37641620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f375b400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3764157b20 0x7f37641620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f375b400640 1 -- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37640b0800 msgr2=0x7f37641560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f375b400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37640b0800 0x7f37641560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f375b400640 1 -- 172.21.15.19:0/3742687302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3764158ab0 con 0x7f37640cf9e0 2024-04-25T11:34:02.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.112+0000 7f3768a00640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3764157b20 0x7f37641620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:02.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.113+0000 7f375b400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640cf9e0 0x7f37641590d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f374c00c440 tx=0x7f374c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.113+0000 7f3760c00640 1 -- 172.21.15.19:0/3742687302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c017070 con 0x7f37640cf9e0 2024-04-25T11:34:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.113+0000 7f3760c00640 1 -- 172.21.15.19:0/3742687302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374c005ce0 con 0x7f37640cf9e0 2024-04-25T11:34:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.113+0000 7f3760c00640 1 -- 172.21.15.19:0/3742687302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c0072b0 con 0x7f37640cf9e0 2024-04-25T11:34:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.113+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3764155330 con 0x7f37640cf9e0 2024-04-25T11:34:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.113+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641558f0 con 0x7f37640cf9e0 2024-04-25T11:34:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.114+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764050740 con 0x7f37640cf9e0 2024-04-25T11:34:02.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.116+0000 7f3760c00640 1 -- 172.21.15.19:0/3742687302 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f374c01a070 con 0x7f37640cf9e0 2024-04-25T11:34:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.117+0000 7f3760c00640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37400744b0 0x7f37400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.117+0000 7f3760c00640 1 -- 172.21.15.19:0/3742687302 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7f374c0942e0 con 0x7f37640cf9e0 2024-04-25T11:34:02.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.117+0000 7f3763e00640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37400744b0 0x7f37400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.119+0000 7f3763e00640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37400744b0 0x7f37400769c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3754007fd0 tx=0x7f3754009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.121+0000 7f3760c00640 1 -- 172.21.15.19:0/3742687302 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3764158ab0 con 0x7f37640cf9e0 2024-04-25T11:34:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.333+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37640cdc30 con 0x7f37640cf9e0 2024-04-25T11:34:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.335+0000 7f3760c00640 1 -- 172.21.15.19:0/3742687302 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+12891 (secure 0 0 0) 0x7f374c05ee70 con 0x7f37640cf9e0 2024-04-25T11:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":75,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:01.161073+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37400744b0 msgr2=0x7f37400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37400744b0 0x7f37400769c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3754007fd0 tx=0x7f3754009040 comp rx=0 tx=0).stop 2024-04-25T11:34:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640cf9e0 msgr2=0x7f37641590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640cf9e0 0x7f37641590d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f374c00c440 tx=0x7f374c00c900 comp rx=0 tx=0).stop 2024-04-25T11:34:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 shutdown_connections 2024-04-25T11:34:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f37400744b0 0x7f37400769c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3764157b20 0x7f37641620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37640cf9e0 0x7f37641590d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 --2- 172.21.15.19:0/3742687302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37640b0800 0x7f37641560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 >> 172.21.15.19:0/3742687302 conn(0x7f37640d1720 msgr2=0x7f3764164400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.338+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 shutdown_connections 2024-04-25T11:34:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.339+0000 7f3769400640 1 -- 172.21.15.19:0/3742687302 wait complete. 2024-04-25T11:34:02.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:34:02.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:34:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.661+0000 7fa1f7e00640 1 Processor -- start 2024-04-25T11:34:02.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.661+0000 7fa1f7e00640 1 -- start start 2024-04-25T11:34:02.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.664+0000 7fa1f7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 0x7fa1f8168dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.664+0000 7fa1f7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f808de70 con 0x7fa1f808d770 2024-04-25T11:34:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.664+0000 7fa1f7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f806d9f0 con 0x7fa1f8169310 2024-04-25T11:34:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.664+0000 7fa1f7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f80d14a0 con 0x7fa1f81689c0 2024-04-25T11:34:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.664+0000 7fa1f6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa1f808d770 0x7fa1f80c6a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37624/0 (socket says 172.21.15.19:37624) 2024-04-25T11:34:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.664+0000 7fa1f6a00640 1 -- 172.21.15.19:0/1939237103 learned_addr learned my addr 172.21.15.19:0/1939237103 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f6000640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 0x7fa1f8168dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308993729 0 0) 0x7fa1f808de70 con 0x7fa1f808d770 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1dc003520 con 0x7fa1f808d770 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2277648989 0 0) 0x7fa1dc003520 con 0x7fa1f808d770 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 msgr2=0x7fa1f8168dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 0x7fa1f8168dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 >> v1:172.21.15.169:6789/0 conn(0x7fa1f8169310 legacy=0x7fa1f80d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f806dcd0 con 0x7fa1f808d770 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa1e8001670 con 0x7fa1f808d770 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 >> v1:172.21.15.19:6789/0 conn(0x7fa1f808d770 legacy=0x7fa1f80c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc0039e0 0x7fa1dc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 0x7fa1dc0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f80d14a0 con 0x7fa1dc004800 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc00bb80 con 0x7fa1dc0039e0 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.665+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa1dc00bd80 con 0x7fa1f81689c0 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6000640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6000640 1 -- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 msgr2=0x7fa1dc0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f7400640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 0x7fa1dc0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6a00640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc0039e0 0x7fa1dc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6000640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 0x7fa1dc0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6000640 1 -- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc0039e0 msgr2=0x7fa1dc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6000640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc0039e0 0x7fa1dc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6000640 1 -- 172.21.15.19:0/1939237103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc003520 con 0x7fa1dc004800 2024-04-25T11:34:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f7400640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f81689c0 0x7fa1dc0052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.666+0000 7fa1f6a00640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1dc0039e0 0x7fa1dc003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.667+0000 7fa1f6000640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009a90 tx=0x7fa1dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edd2424d56c257a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.668+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4036070 con 0x7fa1dc004800 2024-04-25T11:34:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.668+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e4031400 con 0x7fa1dc004800 2024-04-25T11:34:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.668+0000 7fa1f5600640 1 -- 172.21.15.19:0/1939237103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4009690 con 0x7fa1dc004800 2024-04-25T11:34:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.668+0000 7fa1f7e00640 1 -- 172.21.15.19:0/1939237103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 msgr2=0x7fa1dc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.668+0000 7fa1f7e00640 1 --2- 172.21.15.19:0/1939237103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1dc004c10 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009a90 tx=0x7fa1dc005830 comp rx=0 tx=0).stop 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.668+0000 7fa1f7400640 1 -- 172.21.15.19:0/1939237103 reap_dead start 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.669+0000 7fa1f7e00640 1 -- 172.21.15.19:0/1939237103 shutdown_connections 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.669+0000 7fa1f7e00640 1 -- 172.21.15.19:0/1939237103 >> 172.21.15.19:0/1939237103 conn(0x7fa1f80b53c0 msgr2=0x7fa1f80b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.669+0000 7fa1f7e00640 1 -- 172.21.15.19:0/1939237103 shutdown_connections 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.669+0000 7fa1f7e00640 1 -- 172.21.15.19:0/1939237103 wait complete. 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.671+0000 7fa1f7e00640 1 Processor -- start 2024-04-25T11:34:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.671+0000 7fa1f7e00640 1 -- start start 2024-04-25T11:34:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1f8156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1f8153b90 0x7fa1f80bd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f8155460 0x7fa1f8164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f808db80 con 0x7fa1dc004800 2024-04-25T11:34:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f80c7190 con 0x7fa1f8153b90 2024-04-25T11:34:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f80cdf20 con 0x7fa1f8155460 2024-04-25T11:34:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1f8156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1f8156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50062/0 (socket says 172.21.15.19:50062) 2024-04-25T11:34:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f6a00640 1 -- 172.21.15.19:0/958643847 learned_addr learned my addr 172.21.15.19:0/958643847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f6000640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1f8153b90 0x7fa1f80bd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.673+0000 7fa1f7400640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f8155460 0x7fa1f8164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f6a00640 1 -- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f8155460 msgr2=0x7fa1f8164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f6a00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f8155460 0x7fa1f8164830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f6a00640 1 -- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1f8153b90 msgr2=0x7fa1f80bd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f6a00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1f8153b90 0x7fa1f80bd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f6a00640 1 -- 172.21.15.19:0/958643847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f81543c0 con 0x7fa1dc004800 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f7400640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f8155460 0x7fa1f8164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f6a00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1f8156800 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800d820 tx=0x7fa1e800dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1eea00640 1 -- 172.21.15.19:0/958643847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8019070 con 0x7fa1dc004800 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1eea00640 1 -- 172.21.15.19:0/958643847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e80074e0 con 0x7fa1dc004800 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f81546a0 con 0x7fa1dc004800 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.674+0000 7fa1eea00640 1 -- 172.21.15.19:0/958643847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8004a00 con 0x7fa1dc004800 2024-04-25T11:34:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.675+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f8154c60 con 0x7fa1dc004800 2024-04-25T11:34:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.676+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f8076c60 con 0x7fa1dc004800 2024-04-25T11:34:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.677+0000 7fa1eea00640 1 -- 172.21.15.19:0/958643847 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa1e801c070 con 0x7fa1dc004800 2024-04-25T11:34:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.678+0000 7fa1eea00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1d80744e0 0x7fa1d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.678+0000 7fa1eea00640 1 -- 172.21.15.19:0/958643847 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7fa1e8094b30 con 0x7fa1dc004800 2024-04-25T11:34:02.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.679+0000 7fa1f6000640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1d80744e0 0x7fa1d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.681+0000 7fa1f6000640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1d80744e0 0x7fa1d80769f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4033000 tx=0x7fa1e4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.684+0000 7fa1eea00640 1 -- 172.21.15.19:0/958643847 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1e805f570 con 0x7fa1dc004800 2024-04-25T11:34:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.893+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1f80b93e0 con 0x7fa1dc004800 2024-04-25T11:34:02.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.894+0000 7fa1eea00640 1 -- 172.21.15.19:0/958643847 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7fa1e805ee70 con 0x7fa1dc004800 2024-04-25T11:34:02.883 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.896+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1d80744e0 msgr2=0x7fa1d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1d80744e0 0x7fa1d80769f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4033000 tx=0x7fa1e4008030 comp rx=0 tx=0).stop 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 msgr2=0x7fa1f8156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1f8156800 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800d820 tx=0x7fa1e800dce0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 shutdown_connections 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1d80744e0 0x7fa1d80769f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1f8153b90 0x7fa1f80bd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa1f8155460 0x7fa1f8164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 --2- 172.21.15.19:0/958643847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1dc004800 0x7fa1f8156800 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 >> 172.21.15.19:0/958643847 conn(0x7fa1f80b53c0 msgr2=0x7fa1f8167920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 shutdown_connections 2024-04-25T11:34:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:02.897+0000 7fa1f7e00640 1 -- 172.21.15.19:0/958643847 wait complete. 2024-04-25T11:34:02.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:34:03.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.212+0000 7f8a99a00640 1 Processor -- start 2024-04-25T11:34:03.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.212+0000 7f8a99a00640 1 -- start start 2024-04-25T11:34:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.214+0000 7f8a99a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8a9413fe10 0x7f8a9406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.214+0000 7f8a99a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9406db10 0x7f8a94142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.214+0000 7f8a99a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8a94142c70 0x7f8a94070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.214+0000 7f8a99a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9406df20 con 0x7f8a94142c70 2024-04-25T11:34:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.214+0000 7f8a99a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9415d5e0 con 0x7f8a9413fe10 2024-04-25T11:34:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.214+0000 7f8a99a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9415d8c0 con 0x7f8a9406db10 2024-04-25T11:34:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8a94142c70 0x7f8a94070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8a94142c70 0x7f8a94070240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50068/0 (socket says 172.21.15.19:50068) 2024-04-25T11:34:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 -- 172.21.15.19:0/771833299 learned_addr learned my addr 172.21.15.19:0/771833299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:03.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a93e00640 1 --2- 172.21.15.19:0/771833299 >> v2:172.21.15.169:3300/0 conn(0x7f8a9413fe10 0x7f8a9406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a93400640 1 --2- 172.21.15.19:0/771833299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9406db10 0x7f8a94142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 -- 172.21.15.19:0/771833299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9406db10 msgr2=0x7f8a94142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 --2- 172.21.15.19:0/771833299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9406db10 0x7f8a94142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 -- 172.21.15.19:0/771833299 >> v2:172.21.15.169:3300/0 conn(0x7f8a9413fe10 msgr2=0x7f8a9406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 --2- 172.21.15.19:0/771833299 >> v2:172.21.15.169:3300/0 conn(0x7f8a9413fe10 0x7f8a9406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.215+0000 7f8a99000640 1 -- 172.21.15.19:0/771833299 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9415dba0 con 0x7f8a94142c70 2024-04-25T11:34:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.217+0000 7f8a99000640 1 --2- 172.21.15.19:0/771833299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a94142c70 0x7f8a94070240 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8a94073c30 tx=0x7f8a8c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9366c7d0342b204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.217+0000 7f8a92a00640 1 -- 172.21.15.19:0/771833299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c036070 con 0x7f8a94142c70 2024-04-25T11:34:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.217+0000 7f8a92a00640 1 -- 172.21.15.19:0/771833299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8c031600 con 0x7f8a94142c70 2024-04-25T11:34:03.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.217+0000 7f8a92a00640 1 -- 172.21.15.19:0/771833299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0319d0 con 0x7f8a94142c70 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 -- 172.21.15.19:0/771833299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a94142c70 msgr2=0x7f8a94070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 --2- 172.21.15.19:0/771833299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a94142c70 0x7f8a94070240 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8a94073c30 tx=0x7f8a8c02a610 comp rx=0 tx=0).stop 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 -- 172.21.15.19:0/771833299 shutdown_connections 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 --2- 172.21.15.19:0/771833299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a94142c70 0x7f8a94070240 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 --2- 172.21.15.19:0/771833299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a9406db10 0x7f8a94142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 --2- 172.21.15.19:0/771833299 >> v2:172.21.15.169:3300/0 conn(0x7f8a9413fe10 0x7f8a9406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 -- 172.21.15.19:0/771833299 >> 172.21.15.19:0/771833299 conn(0x7f8a94072ee0 msgr2=0x7f8a940732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 -- 172.21.15.19:0/771833299 shutdown_connections 2024-04-25T11:34:03.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.218+0000 7f8a99a00640 1 -- 172.21.15.19:0/771833299 wait complete. 2024-04-25T11:34:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.221+0000 7f8a99a00640 1 Processor -- start 2024-04-25T11:34:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.221+0000 7f8a99a00640 1 -- start start 2024-04-25T11:34:03.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.223+0000 7f8a99a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9406db10 0x7f8a94156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.223+0000 7f8a99a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a9413fe10 0x7f8a94144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.223+0000 7f8a99a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a94142c70 0x7f8a94145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a99a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94143080 con 0x7f8a9413fe10 2024-04-25T11:34:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a99a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9415d2d0 con 0x7f8a9406db10 2024-04-25T11:34:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a99a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8a94143200 con 0x7f8a94142c70 2024-04-25T11:34:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a9413fe10 0x7f8a94144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a9413fe10 0x7f8a94144ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50076/0 (socket says 172.21.15.19:50076) 2024-04-25T11:34:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93400640 1 -- 172.21.15.19:0/3851992552 learned_addr learned my addr 172.21.15.19:0/3851992552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93e00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9406db10 0x7f8a94156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a99000640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a94142c70 0x7f8a94145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93400640 1 -- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a94142c70 msgr2=0x7f8a94145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93400640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a94142c70 0x7f8a94145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93400640 1 -- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9406db10 msgr2=0x7f8a94156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.224+0000 7f8a93400640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9406db10 0x7f8a94156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.225+0000 7f8a93400640 1 -- 172.21.15.19:0/3851992552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a94157280 con 0x7f8a9413fe10 2024-04-25T11:34:03.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.225+0000 7f8a93400640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a9413fe10 0x7f8a94144ce0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000d830 tx=0x7f8a8000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.225+0000 7f8a8be00640 1 -- 172.21.15.19:0/3851992552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80018030 con 0x7f8a9413fe10 2024-04-25T11:34:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.225+0000 7f8a8be00640 1 -- 172.21.15.19:0/3851992552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8000be10 con 0x7f8a9413fe10 2024-04-25T11:34:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.225+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a94145dd0 con 0x7f8a9413fe10 2024-04-25T11:34:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.225+0000 7f8a8be00640 1 -- 172.21.15.19:0/3851992552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800098b0 con 0x7f8a9413fe10 2024-04-25T11:34:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.225+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a94154440 con 0x7f8a9413fe10 2024-04-25T11:34:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.226+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94050740 con 0x7f8a9413fe10 2024-04-25T11:34:03.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.228+0000 7f8a8be00640 1 -- 172.21.15.19:0/3851992552 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8a8001b070 con 0x7f8a9413fe10 2024-04-25T11:34:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.229+0000 7f8a8be00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a740744b0 0x7f8a740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.229+0000 7f8a8be00640 1 -- 172.21.15.19:0/3851992552 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7f8a80095140 con 0x7f8a9413fe10 2024-04-25T11:34:03.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.230+0000 7f8a93e00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a740744b0 0x7f8a740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.231+0000 7f8a93e00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a740744b0 0x7f8a740769c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a84006fd0 tx=0x7f8a84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.234+0000 7f8a8be00640 1 -- 172.21.15.19:0/3851992552 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a8005fb80 con 0x7f8a9413fe10 2024-04-25T11:34:03.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.444+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8a940b5350 con 0x7f8a9413fe10 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.444+0000 7f8a8be00640 1 -- 172.21.15.19:0/3851992552 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f8a8005f480 con 0x7f8a9413fe10 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:34:03.433 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:34:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a740744b0 msgr2=0x7f8a740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a740744b0 0x7f8a740769c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a84006fd0 tx=0x7f8a84008040 comp rx=0 tx=0).stop 2024-04-25T11:34:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a9413fe10 msgr2=0x7f8a94144ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a9413fe10 0x7f8a94144ce0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000d830 tx=0x7f8a8000dcf0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 shutdown_connections 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a740744b0 0x7f8a740769c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8a94142c70 0x7f8a94145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a9413fe10 0x7f8a94144ce0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 --2- 172.21.15.19:0/3851992552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a9406db10 0x7f8a94156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.447+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 >> 172.21.15.19:0/3851992552 conn(0x7f8a94072ee0 msgr2=0x7f8a940b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.448+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 shutdown_connections 2024-04-25T11:34:03.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.448+0000 7f8a99a00640 1 -- 172.21.15.19:0/3851992552 wait complete. 2024-04-25T11:34:03.467 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:34:03.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:34:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.774+0000 7f8f68200640 1 Processor -- start 2024-04-25T11:34:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.774+0000 7f8f68200640 1 -- start start 2024-04-25T11:34:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.777+0000 7f8f68200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f6015f360 0x7f8f600b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.777+0000 7f8f68200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f600cdea0 con 0x7f8f6006dce0 2024-04-25T11:34:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.777+0000 7f8f68200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f600ce180 con 0x7f8f600cd9a0 2024-04-25T11:34:03.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.777+0000 7f8f68200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8f600ceab0 con 0x7f8f6015f360 2024-04-25T11:34:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.777+0000 7f8f64e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8f6006dce0 0x7f8f6015faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37628/0 (socket says 172.21.15.19:37628) 2024-04-25T11:34:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.777+0000 7f8f64e00640 1 -- 172.21.15.19:0/2284662246 learned_addr learned my addr 172.21.15.19:0/2284662246 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.777+0000 7f8f66200640 1 --2- 172.21.15.19:0/2284662246 >> v2:172.21.15.169:3300/0 conn(0x7f8f6015f360 0x7f8f600b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544987285 0 0) 0x7f8f600cdea0 con 0x7f8f6006dce0 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f44003520 con 0x7f8f6006dce0 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 956055059 0 0) 0x7f8f44003520 con 0x7f8f6006dce0 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 >> v2:172.21.15.169:3300/0 conn(0x7f8f6015f360 msgr2=0x7f8f600b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 --2- 172.21.15.19:0/2284662246 >> v2:172.21.15.169:3300/0 conn(0x7f8f6015f360 0x7f8f600b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 >> v1:172.21.15.169:6789/0 conn(0x7f8f600cd9a0 legacy=0x7f8f6006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f600ce460 con 0x7f8f6006dce0 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8f4c001670 con 0x7f8f6006dce0 2024-04-25T11:34:03.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 >> v1:172.21.15.19:6789/0 conn(0x7f8f6006dce0 legacy=0x7f8f6015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f440039b0 0x7f8f44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f44004800 0x7f8f44004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f440051c0 0x7f8f4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f600ceab0 con 0x7f8f440039b0 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4400c160 con 0x7f8f440051c0 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4400c360 con 0x7f8f44004800 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.778+0000 7f8f65800640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f440039b0 0x7f8f44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f66200640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f440051c0 0x7f8f4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f64e00640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f44004800 0x7f8f44004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f65800640 1 -- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f44004800 msgr2=0x7f8f44004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f65800640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f44004800 0x7f8f44004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f65800640 1 -- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f440051c0 msgr2=0x7f8f4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f65800640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f440051c0 0x7f8f4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f65800640 1 -- 172.21.15.19:0/2284662246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f44003520 con 0x7f8f440039b0 2024-04-25T11:34:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f66200640 1 -- 172.21.15.19:0/2284662246 reap_dead start 2024-04-25T11:34:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.779+0000 7f8f64e00640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f44004800 0x7f8f44004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.780+0000 7f8f65800640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f440039b0 0x7f8f44003e20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8f50001750 tx=0x7f8f44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854f83ccb97012e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.780+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50001370 con 0x7f8f440039b0 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f500052d0 con 0x7f8f440039b0 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f5fe00640 1 -- 172.21.15.19:0/2284662246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5002ab00 con 0x7f8f440039b0 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f68200640 1 -- 172.21.15.19:0/2284662246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f440039b0 msgr2=0x7f8f44003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f68200640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f440039b0 0x7f8f44003e20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8f50001750 tx=0x7f8f44005830 comp rx=0 tx=0).stop 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f68200640 1 -- 172.21.15.19:0/2284662246 shutdown_connections 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f68200640 1 --2- 172.21.15.19:0/2284662246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f440039b0 0x7f8f44003e20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f68200640 1 -- 172.21.15.19:0/2284662246 >> 172.21.15.19:0/2284662246 conn(0x7f8f60073090 msgr2=0x7f8f600734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f68200640 1 -- 172.21.15.19:0/2284662246 shutdown_connections 2024-04-25T11:34:03.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.781+0000 7f8f68200640 1 -- 172.21.15.19:0/2284662246 wait complete. 2024-04-25T11:34:03.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.784+0000 7f8f68200640 1 Processor -- start 2024-04-25T11:34:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.784+0000 7f8f68200640 1 -- start start 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f68200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f440039b0 0x7f8f601f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f68200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f6013daa0 0x7f8f6013f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f68200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f6013cbb0 0x7f8f6013f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f68200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f600cef40 con 0x7f8f6013daa0 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f68200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6015f770 con 0x7f8f6013cbb0 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f68200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f600cf0c0 con 0x7f8f440039b0 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f6013daa0 0x7f8f6013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f6013daa0 0x7f8f6013f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43148/0 (socket says 172.21.15.19:43148) 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 -- 172.21.15.19:0/2114085164 learned_addr learned my addr 172.21.15.19:0/2114085164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f66200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f6013cbb0 0x7f8f6013f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f65800640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f440039b0 0x7f8f601f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 -- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f440039b0 msgr2=0x7f8f601f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f440039b0 0x7f8f601f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 -- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f6013cbb0 msgr2=0x7f8f6013f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f6013cbb0 0x7f8f6013f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.786+0000 7f8f64e00640 1 -- 172.21.15.19:0/2114085164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6013c810 con 0x7f8f6013daa0 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f65800640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f440039b0 0x7f8f601f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f66200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f6013cbb0 0x7f8f6013f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f64e00640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f6013daa0 0x7f8f6013f270 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00d7d0 tx=0x7f8f4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f5d600640 1 -- 172.21.15.19:0/2114085164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c019070 con 0x7f8f6013daa0 2024-04-25T11:34:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f5d600640 1 -- 172.21.15.19:0/2114085164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c0074e0 con 0x7f8f6013daa0 2024-04-25T11:34:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f5d600640 1 -- 172.21.15.19:0/2114085164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c004a00 con 0x7f8f6013daa0 2024-04-25T11:34:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6013e1d0 con 0x7f8f6013daa0 2024-04-25T11:34:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.787+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6013e790 con 0x7f8f6013daa0 2024-04-25T11:34:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.789+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6013df40 con 0x7f8f6013daa0 2024-04-25T11:34:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.790+0000 7f8f5d600640 1 -- 172.21.15.19:0/2114085164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f4c004ba0 con 0x7f8f6013daa0 2024-04-25T11:34:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.791+0000 7f8f5d600640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:03.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.791+0000 7f8f5d600640 1 -- 172.21.15.19:0/2114085164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7f8f4c0948c0 con 0x7f8f6013daa0 2024-04-25T11:34:03.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.792+0000 7f8f65800640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.793+0000 7f8f65800640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8f50032000 tx=0x7f8f5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.796+0000 7f8f5d600640 1 -- 172.21.15.19:0/2114085164 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f4c05f300 con 0x7f8f6013daa0 2024-04-25T11:34:03.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:03.999+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8f60050740 con 0x7f8f6013daa0 2024-04-25T11:34:04.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.182+0000 7f8f5d600640 1 -- 172.21.15.19:0/2114085164 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f8f4c05ec00 con 0x7f8f6013daa0 2024-04-25T11:34:04.170 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 msgr2=0x7f8f400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8f50032000 tx=0x7f8f5002e000 comp rx=0 tx=0).stop 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f6013daa0 msgr2=0x7f8f6013f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f6013daa0 0x7f8f6013f270 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00d7d0 tx=0x7f8f4c00dc90 comp rx=0 tx=0).stop 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 shutdown_connections 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f6013cbb0 0x7f8f6013f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f6013daa0 0x7f8f6013f270 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 --2- 172.21.15.19:0/2114085164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f440039b0 0x7f8f601f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.185+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 >> 172.21.15.19:0/2114085164 conn(0x7f8f60073090 msgr2=0x7f8f6006ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:04.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.186+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 shutdown_connections 2024-04-25T11:34:04.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.186+0000 7f8f68200640 1 -- 172.21.15.19:0/2114085164 wait complete. 2024-04-25T11:34:04.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:34:04.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:34:04.397 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:34:04.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.514+0000 7f76ba000640 1 Processor -- start 2024-04-25T11:34:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.514+0000 7f76ba000640 1 -- start start 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.516+0000 7f76ba000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4164ad0 0x7f76b40b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.516+0000 7f76ba000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76b4163f30 0x7f76b4164340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.516+0000 7f76ba000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b4164880 con 0x7f76b40d1f30 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.516+0000 7f76ba000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f76b4164ee0 con 0x7f76b4163f30 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.516+0000 7f76ba000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76b40b31e0 con 0x7f76b4164ad0 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76b4163f30 0x7f76b4164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76b4163f30 0x7f76b4164340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43154/0 (socket says 172.21.15.19:43154) 2024-04-25T11:34:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f76b40d1f30 0x7f76b40d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51470/0 (socket says 172.21.15.19:51470) 2024-04-25T11:34:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2a00640 1 -- 172.21.15.19:0/3384425574 learned_addr learned my addr 172.21.15.19:0/3384425574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b3400640 1 --2- 172.21.15.19:0/3384425574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4164ad0 0x7f76b40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2a00640 1 -- 172.21.15.19:0/3384425574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4164ad0 msgr2=0x7f76b40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2a00640 1 --2- 172.21.15.19:0/3384425574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4164ad0 0x7f76b40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2a00640 1 -- 172.21.15.19:0/3384425574 >> v1:172.21.15.19:6789/0 conn(0x7f76b40d1f30 legacy=0x7f76b40d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2a00640 1 -- 172.21.15.19:0/3384425574 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b40d3170 con 0x7f76b4163f30 2024-04-25T11:34:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.517+0000 7f76b2000640 1 -- 172.21.15.19:0/3384425574 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432422507 0 0) 0x7f76ac001490 con 0x7f76b40d1f30 2024-04-25T11:34:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.518+0000 7f76b2a00640 1 --2- 172.21.15.19:0/3384425574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4163f30 0x7f76b4164340 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009870 tx=0x7f76a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d5a354f0566af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.519+0000 7f76b2000640 1 -- 172.21.15.19:0/3384425574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0037070 con 0x7f76b4163f30 2024-04-25T11:34:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.519+0000 7f76b2000640 1 -- 172.21.15.19:0/3384425574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0032440 con 0x7f76b4163f30 2024-04-25T11:34:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.519+0000 7f76b2000640 1 -- 172.21.15.19:0/3384425574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0032810 con 0x7f76b4163f30 2024-04-25T11:34:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 -- 172.21.15.19:0/3384425574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4163f30 msgr2=0x7f76b4164340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 --2- 172.21.15.19:0/3384425574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4163f30 0x7f76b4164340 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009870 tx=0x7f76a002a270 comp rx=0 tx=0).stop 2024-04-25T11:34:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 -- 172.21.15.19:0/3384425574 shutdown_connections 2024-04-25T11:34:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 --2- 172.21.15.19:0/3384425574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4163f30 0x7f76b4164340 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 --2- 172.21.15.19:0/3384425574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4164ad0 0x7f76b40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 -- 172.21.15.19:0/3384425574 >> 172.21.15.19:0/3384425574 conn(0x7f76b40d3aa0 msgr2=0x7f76b40d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 -- 172.21.15.19:0/3384425574 shutdown_connections 2024-04-25T11:34:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.520+0000 7f76ba000640 1 -- 172.21.15.19:0/3384425574 wait complete. 2024-04-25T11:34:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.522+0000 7f76ba000640 1 Processor -- start 2024-04-25T11:34:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.522+0000 7f76ba000640 1 -- start start 2024-04-25T11:34:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76ba000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76b40d1f30 0x7f76b40b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76ba000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4163f30 0x7f76b4160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76ba000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4164ad0 0x7f76b4160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76ba000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76b406d9b0 con 0x7f76b4164ad0 2024-04-25T11:34:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76ba000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76b406d6d0 con 0x7f76b40d1f30 2024-04-25T11:34:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76ba000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76b40b9430 con 0x7f76b4163f30 2024-04-25T11:34:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4164ad0 0x7f76b4160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4164ad0 0x7f76b4160f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43164/0 (socket says 172.21.15.19:43164) 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.524+0000 7f76b3e00640 1 -- 172.21.15.19:0/3284904107 learned_addr learned my addr 172.21.15.19:0/3284904107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b3400640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76b40d1f30 0x7f76b40b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b2a00640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4163f30 0x7f76b4160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b3e00640 1 -- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4163f30 msgr2=0x7f76b4160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b3e00640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4163f30 0x7f76b4160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b3e00640 1 -- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76b40d1f30 msgr2=0x7f76b40b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b3e00640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76b40d1f30 0x7f76b40b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b3e00640 1 -- 172.21.15.19:0/3284904107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b40c7b50 con 0x7f76b4164ad0 2024-04-25T11:34:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76b3e00640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4164ad0 0x7f76b4160f50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c700 tx=0x7f76ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.525+0000 7f76ab400640 1 -- 172.21.15.19:0/3284904107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac018070 con 0x7f76b4164ad0 2024-04-25T11:34:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.526+0000 7f76ab400640 1 -- 172.21.15.19:0/3284904107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ac00ce10 con 0x7f76b4164ad0 2024-04-25T11:34:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.526+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b415d8f0 con 0x7f76b4164ad0 2024-04-25T11:34:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.526+0000 7f76ab400640 1 -- 172.21.15.19:0/3284904107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00a860 con 0x7f76b4164ad0 2024-04-25T11:34:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.526+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b415deb0 con 0x7f76b4164ad0 2024-04-25T11:34:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.527+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b40d4770 con 0x7f76b4164ad0 2024-04-25T11:34:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.529+0000 7f76ab400640 1 -- 172.21.15.19:0/3284904107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f76ac01b070 con 0x7f76b4164ad0 2024-04-25T11:34:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.530+0000 7f76ab400640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76940744b0 0x7f76940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.530+0000 7f76ab400640 1 -- 172.21.15.19:0/3284904107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5227+0+0 (secure 0 0 0) 0x7f76ac094910 con 0x7f76b4164ad0 2024-04-25T11:34:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.530+0000 7f76b3400640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76940744b0 0x7f76940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.532+0000 7f76b3400640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76940744b0 0x7f76940769c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f76a4006fd0 tx=0x7f76a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.534+0000 7f76ab400640 1 -- 172.21.15.19:0/3284904107 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76b4164880 con 0x7f76b4164ad0 2024-04-25T11:34:04.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:04.739+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f76b406ea30 con 0x7f76b4164ad0 2024-04-25T11:34:05.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.198+0000 7f76ab400640 1 -- 172.21.15.19:0/3284904107 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f76ac05f2f0 con 0x7f76b4164ad0 2024-04-25T11:34:05.187 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76940744b0 msgr2=0x7f76940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76940744b0 0x7f76940769c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f76a4006fd0 tx=0x7f76a4008040 comp rx=0 tx=0).stop 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4164ad0 msgr2=0x7f76b4160f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4164ad0 0x7f76b4160f50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c700 tx=0x7f76ac00cbc0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 shutdown_connections 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76940744b0 0x7f76940769c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76b4164ad0 0x7f76b4160f50 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76b4163f30 0x7f76b4160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 --2- 172.21.15.19:0/3284904107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76b40d1f30 0x7f76b40b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 >> 172.21.15.19:0/3284904107 conn(0x7f76b40d3aa0 msgr2=0x7f76b40d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 shutdown_connections 2024-04-25T11:34:05.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.202+0000 7f76ba000640 1 -- 172.21.15.19:0/3284904107 wait complete. 2024-04-25T11:34:05.237 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:34:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.556+0000 7fcb3da00640 1 Processor -- start 2024-04-25T11:34:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.556+0000 7fcb3da00640 1 -- start start 2024-04-25T11:34:05.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.559+0000 7fcb3da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb381441d0 0x7fcb38161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.559+0000 7fcb3da00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb38162450 con 0x7fcb38162040 2024-04-25T11:34:05.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.559+0000 7fcb3da00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3806d3a0 con 0x7fcb3806cd40 2024-04-25T11:34:05.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.559+0000 7fcb3da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3806d150 con 0x7fcb381441d0 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.559+0000 7fcb2f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcb38162040 0x7fcb3806c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51480/0 (socket says 172.21.15.19:51480) 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.559+0000 7fcb2f400640 1 -- 172.21.15.19:0/4151011959 learned_addr learned my addr 172.21.15.19:0/4151011959 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.559+0000 7fcb37e00640 1 --2- 172.21.15.19:0/4151011959 >> v2:172.21.15.169:3300/0 conn(0x7fcb381441d0 0x7fcb38161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415282154 0 0) 0x7fcb20001110 con 0x7fcb38162040 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb1c003520 con 0x7fcb38162040 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2749844130 0 0) 0x7fcb1c003520 con 0x7fcb38162040 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 >> v2:172.21.15.169:3300/0 conn(0x7fcb381441d0 msgr2=0x7fcb38161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 --2- 172.21.15.19:0/4151011959 >> v2:172.21.15.169:3300/0 conn(0x7fcb381441d0 0x7fcb38161b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 >> v1:172.21.15.169:6789/0 conn(0x7fcb3806cd40 legacy=0x7fcb3815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3806d680 con 0x7fcb38162040 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fcb20001840 con 0x7fcb38162040 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 >> v1:172.21.15.19:6789/0 conn(0x7fcb38162040 legacy=0x7fcb3806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c0039b0 0x7fcb1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.560+0000 7fcb37400640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c004800 0x7fcb1c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37400640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c005250 0x7fcb1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3806d150 con 0x7fcb1c0039b0 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c00c1e0 con 0x7fcb1c005250 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c00c3e0 con 0x7fcb1c004800 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37e00640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c0039b0 0x7fcb1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37e00640 1 -- 172.21.15.19:0/4151011959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c004800 msgr2=0x7fcb1c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37e00640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb1c004800 0x7fcb1c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37e00640 1 -- 172.21.15.19:0/4151011959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c005250 msgr2=0x7fcb1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37e00640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c005250 0x7fcb1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb37e00640 1 -- 172.21.15.19:0/4151011959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c003520 con 0x7fcb1c0039b0 2024-04-25T11:34:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.561+0000 7fcb3d000640 1 -- 172.21.15.19:0/4151011959 reap_dead start 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.563+0000 7fcb37e00640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c0039b0 0x7fcb1c003e20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009ac0 tx=0x7fcb1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3660f9e457e8e4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.563+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28036070 con 0x7fcb1c0039b0 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.563+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb28031400 con 0x7fcb1c0039b0 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.563+0000 7fcb37400640 1 -- 172.21.15.19:0/4151011959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28009690 con 0x7fcb1c0039b0 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.563+0000 7fcb3da00640 1 -- 172.21.15.19:0/4151011959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c0039b0 msgr2=0x7fcb1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.564+0000 7fcb3da00640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c0039b0 0x7fcb1c003e20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009ac0 tx=0x7fcb1c005830 comp rx=0 tx=0).stop 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.564+0000 7fcb3da00640 1 -- 172.21.15.19:0/4151011959 shutdown_connections 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.564+0000 7fcb3da00640 1 --2- 172.21.15.19:0/4151011959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb1c0039b0 0x7fcb1c003e20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.564+0000 7fcb3da00640 1 -- 172.21.15.19:0/4151011959 >> 172.21.15.19:0/4151011959 conn(0x7fcb38072f40 msgr2=0x7fcb38073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.564+0000 7fcb3da00640 1 -- 172.21.15.19:0/4151011959 shutdown_connections 2024-04-25T11:34:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.564+0000 7fcb3da00640 1 -- 172.21.15.19:0/4151011959 wait complete. 2024-04-25T11:34:05.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.566+0000 7fcb3da00640 1 Processor -- start 2024-04-25T11:34:05.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.566+0000 7fcb3da00640 1 -- start start 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0039b0 0x7fcb3815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb380ceef0 0x7fcb380cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb380cf840 0x7fcb3813e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381445e0 con 0x7fcb380cf840 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3806dc40 con 0x7fcb1c0039b0 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb38161b00 con 0x7fcb380ceef0 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb380cf840 0x7fcb3813e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb380cf840 0x7fcb3813e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43182/0 (socket says 172.21.15.19:43182) 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb3d000640 1 -- 172.21.15.19:0/2490174444 learned_addr learned my addr 172.21.15.19:0/2490174444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb37e00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0039b0 0x7fcb3815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:05.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.568+0000 7fcb2f400640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb380ceef0 0x7fcb380cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.569+0000 7fcb3d000640 1 -- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb380ceef0 msgr2=0x7fcb380cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.569+0000 7fcb3d000640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb380ceef0 0x7fcb380cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.569+0000 7fcb3d000640 1 -- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0039b0 msgr2=0x7fcb3815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.569+0000 7fcb3d000640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0039b0 0x7fcb3815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.569+0000 7fcb3d000640 1 -- 172.21.15.19:0/2490174444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb380cfc50 con 0x7fcb380cf840 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.569+0000 7fcb37e00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0039b0 0x7fcb3815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.569+0000 7fcb3d000640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb380cf840 0x7fcb3813e070 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcb3815f560 tx=0x7fcb3000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.570+0000 7fcb34c00640 1 -- 172.21.15.19:0/2490174444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30017070 con 0x7fcb380cf840 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.570+0000 7fcb34c00640 1 -- 172.21.15.19:0/2490174444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb30005d40 con 0x7fcb380cf840 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.570+0000 7fcb34c00640 1 -- 172.21.15.19:0/2490174444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000a880 con 0x7fcb380cf840 2024-04-25T11:34:05.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.570+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb381e94b0 con 0x7fcb380cf840 2024-04-25T11:34:05.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.570+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb380ce6f0 con 0x7fcb380cf840 2024-04-25T11:34:05.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.571+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb381e9220 con 0x7fcb380cf840 2024-04-25T11:34:05.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.573+0000 7fcb34c00640 1 -- 172.21.15.19:0/2490174444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb3001a070 con 0x7fcb380cf840 2024-04-25T11:34:05.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.574+0000 7fcb34c00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb140744b0 0x7fcb140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:05.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.574+0000 7fcb34c00640 1 -- 172.21.15.19:0/2490174444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5609+0+0 (secure 0 0 0) 0x7fcb300945b0 con 0x7fcb380cf840 2024-04-25T11:34:05.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.575+0000 7fcb37e00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb140744b0 0x7fcb140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:05.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.576+0000 7fcb37e00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb140744b0 0x7fcb140769c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcb28033000 tx=0x7fcb28008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.580+0000 7fcb34c00640 1 -- 172.21.15.19:0/2490174444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb3005ed00 con 0x7fcb380cf840 2024-04-25T11:34:05.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:05.830+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcb38076c60 con 0x7fcb380cf840 2024-04-25T11:34:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.224+0000 7fcb34c00640 1 -- 172.21.15.19:0/2490174444 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v70) ==== 174+0+0 (secure 0 0 0) 0x7fcb3005e600 con 0x7fcb380cf840 2024-04-25T11:34:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T11:34:06.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb140744b0 msgr2=0x7fcb140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:06.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb140744b0 0x7fcb140769c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcb28033000 tx=0x7fcb28008030 comp rx=0 tx=0).stop 2024-04-25T11:34:06.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb380cf840 msgr2=0x7fcb3813e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:06.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb380cf840 0x7fcb3813e070 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcb3815f560 tx=0x7fcb3000cb90 comp rx=0 tx=0).stop 2024-04-25T11:34:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 shutdown_connections 2024-04-25T11:34:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb140744b0 0x7fcb140769c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb380cf840 0x7fcb3813e070 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb380ceef0 0x7fcb380cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 --2- 172.21.15.19:0/2490174444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb1c0039b0 0x7fcb3815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.238+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 >> 172.21.15.19:0/2490174444 conn(0x7fcb38072f40 msgr2=0x7fcb38143850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:06.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.239+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 shutdown_connections 2024-04-25T11:34:06.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.239+0000 7fcb3da00640 1 -- 172.21.15.19:0/2490174444 wait complete. 2024-04-25T11:34:06.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:06.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.534+0000 7f9ba3200640 1 Processor -- start 2024-04-25T11:34:06.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.534+0000 7f9ba3200640 1 -- start start 2024-04-25T11:34:06.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ba40cf9e0 0x7f9ba4161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9ba40709a0 0x7f9ba41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba41615d0 con 0x7f9ba40b0800 2024-04-25T11:34:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba3200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba40cffc0 con 0x7f9ba40cf9e0 2024-04-25T11:34:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba40d02a0 con 0x7f9ba40709a0 2024-04-25T11:34:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ba40cf9e0 0x7f9ba4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ba40cf9e0 0x7f9ba4161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43198/0 (socket says 172.21.15.19:43198) 2024-04-25T11:34:06.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.537+0000 7f9ba1e00640 1 -- 172.21.15.19:0/1475544055 learned_addr learned my addr 172.21.15.19:0/1475544055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.538+0000 7f9ba2800640 1 --2- 172.21.15.19:0/1475544055 >> v2:172.21.15.169:3300/0 conn(0x7f9ba40709a0 0x7f9ba41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.538+0000 7f9ba1e00640 1 -- 172.21.15.19:0/1475544055 >> v2:172.21.15.169:3300/0 conn(0x7f9ba40709a0 msgr2=0x7f9ba41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.538+0000 7f9ba1e00640 1 --2- 172.21.15.19:0/1475544055 >> v2:172.21.15.169:3300/0 conn(0x7f9ba40709a0 0x7f9ba41569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.538+0000 7f9ba1e00640 1 -- 172.21.15.19:0/1475544055 >> v1:172.21.15.169:6789/0 conn(0x7f9ba40b0800 legacy=0x7f9ba4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.538+0000 7f9ba1e00640 1 -- 172.21.15.19:0/1475544055 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba40d0580 con 0x7f9ba40cf9e0 2024-04-25T11:34:06.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.539+0000 7f9ba1e00640 1 --2- 172.21.15.19:0/1475544055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cf9e0 0x7f9ba4161090 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4074610 tx=0x7f9b9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aa9191d7cee1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:06.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.540+0000 7f9ba1400640 1 -- 172.21.15.19:0/1475544055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94037070 con 0x7f9ba40cf9e0 2024-04-25T11:34:06.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.540+0000 7f9ba1400640 1 -- 172.21.15.19:0/1475544055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b94032440 con 0x7f9ba40cf9e0 2024-04-25T11:34:06.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.540+0000 7f9ba1400640 1 -- 172.21.15.19:0/1475544055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94032810 con 0x7f9ba40cf9e0 2024-04-25T11:34:06.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 -- 172.21.15.19:0/1475544055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cf9e0 msgr2=0x7f9ba4161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:06.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1475544055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cf9e0 0x7f9ba4161090 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4074610 tx=0x7f9b9402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 -- 172.21.15.19:0/1475544055 shutdown_connections 2024-04-25T11:34:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1475544055 >> v2:172.21.15.169:3300/0 conn(0x7f9ba40709a0 0x7f9ba41569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1475544055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cf9e0 0x7f9ba4161090 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 -- 172.21.15.19:0/1475544055 >> 172.21.15.19:0/1475544055 conn(0x7f9ba40d1720 msgr2=0x7f9ba40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 -- 172.21.15.19:0/1475544055 shutdown_connections 2024-04-25T11:34:06.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.541+0000 7f9ba3200640 1 -- 172.21.15.19:0/1475544055 wait complete. 2024-04-25T11:34:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.544+0000 7f9ba3200640 1 Processor -- start 2024-04-25T11:34:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.544+0000 7f9ba3200640 1 -- start start 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.546+0000 7f9ba3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba40b0800 0x7f9ba41f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.546+0000 7f9ba3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba40cf9e0 0x7f9ba4144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.546+0000 7f9ba3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cbb30 0x7f9ba40b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.546+0000 7f9ba3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba40b0d90 con 0x7f9ba40cbb30 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.546+0000 7f9ba3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba40b0c10 con 0x7f9ba40b0800 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.546+0000 7f9ba3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba4156ef0 con 0x7f9ba40cf9e0 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cbb30 0x7f9ba40b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cbb30 0x7f9ba40b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43214/0 (socket says 172.21.15.19:43214) 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 -- 172.21.15.19:0/1245605338 learned_addr learned my addr 172.21.15.19:0/1245605338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba1e00640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba40b0800 0x7f9ba41f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9b99400640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba40cf9e0 0x7f9ba4144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 -- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba40cf9e0 msgr2=0x7f9ba4144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba40cf9e0 0x7f9ba4144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 -- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba40b0800 msgr2=0x7f9ba41f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba40b0800 0x7f9ba41f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9ba2800640 1 -- 172.21.15.19:0/1245605338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba41fa310 con 0x7f9ba40cbb30 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.547+0000 7f9b99400640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba40cf9e0 0x7f9ba4144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.548+0000 7f9ba2800640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cbb30 0x7f9ba40b5b70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009fd0 tx=0x7f9b9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.548+0000 7f9b9aa00640 1 -- 172.21.15.19:0/1245605338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014420 con 0x7f9ba40cbb30 2024-04-25T11:34:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.548+0000 7f9b9aa00640 1 -- 172.21.15.19:0/1245605338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c019070 con 0x7f9ba40cbb30 2024-04-25T11:34:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.548+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba4145a40 con 0x7f9ba40cbb30 2024-04-25T11:34:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.548+0000 7f9b9aa00640 1 -- 172.21.15.19:0/1245605338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014800 con 0x7f9ba40cbb30 2024-04-25T11:34:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.548+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba4146000 con 0x7f9ba40cbb30 2024-04-25T11:34:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.549+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba40d1b30 con 0x7f9ba40cbb30 2024-04-25T11:34:06.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.551+0000 7f9b9aa00640 1 -- 172.21.15.19:0/1245605338 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9b9c0149a0 con 0x7f9ba40cbb30 2024-04-25T11:34:06.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.552+0000 7f9b9aa00640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b800744e0 0x7f9b800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:06.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.552+0000 7f9b9aa00640 1 -- 172.21.15.19:0/1245605338 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f9b9c095a30 con 0x7f9ba40cbb30 2024-04-25T11:34:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.552+0000 7f9ba1e00640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b800744e0 0x7f9b800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.554+0000 7f9ba1e00640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b800744e0 0x7f9b800769f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9b94034000 tx=0x7f9b9402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.557+0000 7f9b9aa00640 1 -- 172.21.15.19:0/1245605338 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ba40d1b30 con 0x7f9ba40cbb30 2024-04-25T11:34:06.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.763+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ba4050740 con 0x7f9ba40cbb30 2024-04-25T11:34:06.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.766+0000 7f9b9aa00640 1 -- 172.21.15.19:0/1245605338 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16621 (secure 0 0 0) 0x7f9b9c0601c0 con 0x7f9ba40cbb30 2024-04-25T11:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":78,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:05.831698+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:34:04.018735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T11:34:04.759136+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:06.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b800744e0 msgr2=0x7f9b800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:06.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b800744e0 0x7f9b800769f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9b94034000 tx=0x7f9b9402e000 comp rx=0 tx=0).stop 2024-04-25T11:34:06.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cbb30 msgr2=0x7f9ba40b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:06.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cbb30 0x7f9ba40b5b70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009fd0 tx=0x7f9b9c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 shutdown_connections 2024-04-25T11:34:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b800744e0 0x7f9b800769f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ba40cbb30 0x7f9ba40b5b70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ba40cf9e0 0x7f9ba4144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 --2- 172.21.15.19:0/1245605338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ba40b0800 0x7f9ba41f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.769+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 >> 172.21.15.19:0/1245605338 conn(0x7f9ba40d1720 msgr2=0x7f9ba4163b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.770+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 shutdown_connections 2024-04-25T11:34:06.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:06.770+0000 7f9ba3200640 1 -- 172.21.15.19:0/1245605338 wait complete. 2024-04-25T11:34:06.784 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:07.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.085+0000 7f66faa00640 1 Processor -- start 2024-04-25T11:34:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.085+0000 7f66faa00640 1 -- start start 2024-04-25T11:34:07.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66faa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66f40cf9e0 0x7f66f4161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66faa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66f40b0800 0x7f66f4070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66faa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f41615d0 con 0x7f66f40707d0 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66faa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f66f4070be0 con 0x7f66f40cf9e0 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66faa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f66f40cffc0 con 0x7f66f40b0800 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66f9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66f40cf9e0 0x7f66f4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66f9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66f40cf9e0 0x7f66f4161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43224/0 (socket says 172.21.15.19:43224) 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.088+0000 7f66f9600640 1 -- 172.21.15.19:0/2428329626 learned_addr learned my addr 172.21.15.19:0/2428329626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:07.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.089+0000 7f66f8c00640 1 --2- 172.21.15.19:0/2428329626 >> v2:172.21.15.169:3300/0 conn(0x7f66f40b0800 0x7f66f4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.089+0000 7f66f9600640 1 -- 172.21.15.19:0/2428329626 >> v2:172.21.15.169:3300/0 conn(0x7f66f40b0800 msgr2=0x7f66f4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:07.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.089+0000 7f66f9600640 1 --2- 172.21.15.19:0/2428329626 >> v2:172.21.15.169:3300/0 conn(0x7f66f40b0800 0x7f66f4070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.089+0000 7f66f9600640 1 -- 172.21.15.19:0/2428329626 >> v1:172.21.15.169:6789/0 conn(0x7f66f40707d0 legacy=0x7f66f41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:07.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.089+0000 7f66f9600640 1 -- 172.21.15.19:0/2428329626 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f40d02a0 con 0x7f66f40cf9e0 2024-04-25T11:34:07.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.090+0000 7f66f9600640 1 --2- 172.21.15.19:0/2428329626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f4161090 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f66f40d14c0 tx=0x7f66e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02432db50803ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.091+0000 7f66f3e00640 1 -- 172.21.15.19:0/2428329626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8037070 con 0x7f66f40cf9e0 2024-04-25T11:34:07.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.091+0000 7f66f3e00640 1 -- 172.21.15.19:0/2428329626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e8032440 con 0x7f66f40cf9e0 2024-04-25T11:34:07.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.091+0000 7f66f3e00640 1 -- 172.21.15.19:0/2428329626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8032810 con 0x7f66f40cf9e0 2024-04-25T11:34:07.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 -- 172.21.15.19:0/2428329626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 msgr2=0x7f66f4161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 --2- 172.21.15.19:0/2428329626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f4161090 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f66f40d14c0 tx=0x7f66e802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 -- 172.21.15.19:0/2428329626 shutdown_connections 2024-04-25T11:34:07.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 --2- 172.21.15.19:0/2428329626 >> v2:172.21.15.169:3300/0 conn(0x7f66f40b0800 0x7f66f4070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 --2- 172.21.15.19:0/2428329626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f4161090 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 -- 172.21.15.19:0/2428329626 >> 172.21.15.19:0/2428329626 conn(0x7f66f40d1720 msgr2=0x7f66f40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:07.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 -- 172.21.15.19:0/2428329626 shutdown_connections 2024-04-25T11:34:07.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.092+0000 7f66faa00640 1 -- 172.21.15.19:0/2428329626 wait complete. 2024-04-25T11:34:07.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.095+0000 7f66faa00640 1 Processor -- start 2024-04-25T11:34:07.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.095+0000 7f66faa00640 1 -- start start 2024-04-25T11:34:07.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.097+0000 7f66faa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66f40707d0 0x7f66f4145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66faa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66f40b0800 0x7f66f41f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66faa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f41f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66faa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40b0c10 con 0x7f66f40cf9e0 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66faa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40d0860 con 0x7f66f40b0800 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66faa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66f40b0d90 con 0x7f66f40707d0 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66fa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f41f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66fa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f41f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43230/0 (socket says 172.21.15.19:43230) 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66fa000640 1 -- 172.21.15.19:0/344106945 learned_addr learned my addr 172.21.15.19:0/344106945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66f9600640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66f40707d0 0x7f66f4145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.098+0000 7f66f8c00640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66f40b0800 0x7f66f41f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66fa000640 1 -- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66f40707d0 msgr2=0x7f66f4145160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66fa000640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66f40707d0 0x7f66f4145160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66fa000640 1 -- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66f40b0800 msgr2=0x7f66f41f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66fa000640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66f40b0800 0x7f66f41f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66fa000640 1 -- 172.21.15.19:0/344106945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f41f7960 con 0x7f66f40cf9e0 2024-04-25T11:34:07.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66fa000640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f41f6ee0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b9e0 tx=0x7f66ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66f1600640 1 -- 172.21.15.19:0/344106945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec003b70 con 0x7f66f40cf9e0 2024-04-25T11:34:07.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66f1600640 1 -- 172.21.15.19:0/344106945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ec003d10 con 0x7f66f40cf9e0 2024-04-25T11:34:07.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f41f7740 con 0x7f66f40cf9e0 2024-04-25T11:34:07.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.099+0000 7f66f1600640 1 -- 172.21.15.19:0/344106945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00cb60 con 0x7f66f40cf9e0 2024-04-25T11:34:07.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.100+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41591e0 con 0x7f66f40cf9e0 2024-04-25T11:34:07.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.101+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f41f7b30 con 0x7f66f40cf9e0 2024-04-25T11:34:07.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.103+0000 7f66f1600640 1 -- 172.21.15.19:0/344106945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66ec00cd00 con 0x7f66f40cf9e0 2024-04-25T11:34:07.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.104+0000 7f66f1600640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66d40744b0 0x7f66d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.104+0000 7f66f1600640 1 -- 172.21.15.19:0/344106945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f66ec0950c0 con 0x7f66f40cf9e0 2024-04-25T11:34:07.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.104+0000 7f66f9600640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66d40744b0 0x7f66d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.106+0000 7f66f9600640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66d40744b0 0x7f66d40769c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f66e8034000 tx=0x7f66e802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.113+0000 7f66f1600640 1 -- 172.21.15.19:0/344106945 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66ec05f840 con 0x7f66f40cf9e0 2024-04-25T11:34:07.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.326+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66f4050740 con 0x7f66f40cf9e0 2024-04-25T11:34:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.330+0000 7f66f1600640 1 -- 172.21.15.19:0/344106945 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16608 (secure 0 0 0) 0x7f66ec05f140 con 0x7f66f40cf9e0 2024-04-25T11:34:07.319 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:07.319 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:07.222707+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:34:04.018735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T11:34:04.759136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66d40744b0 msgr2=0x7f66d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66d40744b0 0x7f66d40769c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f66e8034000 tx=0x7f66e802e000 comp rx=0 tx=0).stop 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 msgr2=0x7f66f41f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f41f6ee0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b9e0 tx=0x7f66ec00bea0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 shutdown_connections 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66d40744b0 0x7f66d40769c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66f40b0800 0x7f66f41f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f40cf9e0 0x7f66f41f6ee0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 --2- 172.21.15.19:0/344106945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66f40707d0 0x7f66f4145160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 >> 172.21.15.19:0/344106945 conn(0x7f66f40d1720 msgr2=0x7f66f40afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 shutdown_connections 2024-04-25T11:34:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.333+0000 7f66faa00640 1 -- 172.21.15.19:0/344106945 wait complete. 2024-04-25T11:34:07.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:34:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.653+0000 7fd461e00640 1 Processor -- start 2024-04-25T11:34:07.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.654+0000 7fd461e00640 1 -- start start 2024-04-25T11:34:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.656+0000 7fd461e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd45c06dce0 0x7fd45c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.656+0000 7fd461e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd45c070000 con 0x7fd45c144340 2024-04-25T11:34:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.656+0000 7fd461e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd45c06f240 con 0x7fd45c15ec90 2024-04-25T11:34:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.656+0000 7fd461e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd45c144750 con 0x7fd45c06dce0 2024-04-25T11:34:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.657+0000 7fd460a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd45c144340 0x7fd45c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51486/0 (socket says 172.21.15.19:51486) 2024-04-25T11:34:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.657+0000 7fd460a00640 1 -- 172.21.15.19:0/3117620815 learned_addr learned my addr 172.21.15.19:0/3117620815 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.657+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638153153 0 0) 0x7fd45c070000 con 0x7fd45c144340 2024-04-25T11:34:07.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.657+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd440003520 con 0x7fd45c144340 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.657+0000 7fd45be00640 1 --2- 172.21.15.19:0/3117620815 >> v2:172.21.15.169:3300/0 conn(0x7fd45c06dce0 0x7fd45c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.657+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745459647 0 0) 0x7fd440003520 con 0x7fd45c144340 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 >> v2:172.21.15.169:3300/0 conn(0x7fd45c06dce0 msgr2=0x7fd45c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 --2- 172.21.15.19:0/3117620815 >> v2:172.21.15.169:3300/0 conn(0x7fd45c06dce0 0x7fd45c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 >> v1:172.21.15.169:6789/0 conn(0x7fd45c15ec90 legacy=0x7fd45c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c06f520 con 0x7fd45c144340 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fd44c001670 con 0x7fd45c144340 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 >> v1:172.21.15.19:6789/0 conn(0x7fd45c144340 legacy=0x7fd45c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4400039b0 0x7fd440003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440004800 0x7fd440004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd440005270 0x7fd44000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c144750 con 0x7fd440005270 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd44000c200 con 0x7fd4400039b0 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd44000c400 con 0x7fd440004800 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd461400640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd440005270 0x7fd44000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd460a00640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4400039b0 0x7fd440003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.658+0000 7fd45be00640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440004800 0x7fd440004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.659+0000 7fd461400640 1 -- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440004800 msgr2=0x7fd440004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.659+0000 7fd461400640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440004800 0x7fd440004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.659+0000 7fd461400640 1 -- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4400039b0 msgr2=0x7fd440003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.659+0000 7fd461400640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4400039b0 0x7fd440003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.659+0000 7fd461400640 1 -- 172.21.15.19:0/3117620815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd440003520 con 0x7fd440005270 2024-04-25T11:34:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.660+0000 7fd461400640 1 -- 172.21.15.19:0/3117620815 reap_dead start 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.660+0000 7fd461400640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd440005270 0x7fd44000b9c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd454001e20 tx=0x7fd440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16b2e9341d40661 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.660+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454001a70 con 0x7fd440005270 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.660+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45402ae40 con 0x7fd440005270 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.660+0000 7fd45b400640 1 -- 172.21.15.19:0/3117620815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454003b90 con 0x7fd440005270 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.661+0000 7fd461e00640 1 -- 172.21.15.19:0/3117620815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd440005270 msgr2=0x7fd44000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.661+0000 7fd461e00640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd440005270 0x7fd44000b9c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd454001e20 tx=0x7fd440005830 comp rx=0 tx=0).stop 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.661+0000 7fd461e00640 1 -- 172.21.15.19:0/3117620815 shutdown_connections 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.661+0000 7fd461e00640 1 --2- 172.21.15.19:0/3117620815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd440005270 0x7fd44000b9c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.661+0000 7fd461e00640 1 -- 172.21.15.19:0/3117620815 >> 172.21.15.19:0/3117620815 conn(0x7fd45c072f40 msgr2=0x7fd45c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.661+0000 7fd461e00640 1 -- 172.21.15.19:0/3117620815 shutdown_connections 2024-04-25T11:34:07.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.661+0000 7fd461e00640 1 -- 172.21.15.19:0/3117620815 wait complete. 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.664+0000 7fd461e00640 1 Processor -- start 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.664+0000 7fd461e00640 1 -- start start 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.665+0000 7fd461e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440005270 0x7fd45c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.665+0000 7fd461e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45c0b2970 0x7fd45c15c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.665+0000 7fd461e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45c15c580 0x7fd45c1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.665+0000 7fd461e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c15f0a0 con 0x7fd45c0b2970 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.665+0000 7fd461e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c06fae0 con 0x7fd45c15c580 2024-04-25T11:34:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.665+0000 7fd461e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd45c15f220 con 0x7fd440005270 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45c0b2970 0x7fd45c15c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45c0b2970 0x7fd45c15c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43246/0 (socket says 172.21.15.19:43246) 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 -- 172.21.15.19:0/810880789 learned_addr learned my addr 172.21.15.19:0/810880789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd460a00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440005270 0x7fd45c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd461400640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45c15c580 0x7fd45c1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 -- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440005270 msgr2=0x7fd45c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440005270 0x7fd45c0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 -- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45c15c580 msgr2=0x7fd45c1604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45c15c580 0x7fd45c1604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.666+0000 7fd45be00640 1 -- 172.21.15.19:0/810880789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c15a410 con 0x7fd45c0b2970 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.667+0000 7fd45be00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45c0b2970 0x7fd45c15c040 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd44800f580 tx=0x7fd44800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.667+0000 7fd458c00640 1 -- 172.21.15.19:0/810880789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448013810 con 0x7fd45c0b2970 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.667+0000 7fd458c00640 1 -- 172.21.15.19:0/810880789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44801a070 con 0x7fd45c0b2970 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.667+0000 7fd458c00640 1 -- 172.21.15.19:0/810880789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448013bb0 con 0x7fd45c0b2970 2024-04-25T11:34:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.667+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c0af650 con 0x7fd45c0b2970 2024-04-25T11:34:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.667+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c0afc10 con 0x7fd45c0b2970 2024-04-25T11:34:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.668+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c15a5e0 con 0x7fd45c0b2970 2024-04-25T11:34:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.670+0000 7fd458c00640 1 -- 172.21.15.19:0/810880789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd448012070 con 0x7fd45c0b2970 2024-04-25T11:34:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.671+0000 7fd458c00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd43c0744e0 0x7fd43c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.671+0000 7fd458c00640 1 -- 172.21.15.19:0/810880789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fd448095200 con 0x7fd45c0b2970 2024-04-25T11:34:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.671+0000 7fd460a00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd43c0744e0 0x7fd43c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.673+0000 7fd460a00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd43c0744e0 0x7fd43c0769f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd44c001050 tx=0x7fd44c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.675+0000 7fd458c00640 1 -- 172.21.15.19:0/810880789 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd45c15a5e0 con 0x7fd45c0b2970 2024-04-25T11:34:07.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:07.924+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd45c076c60 con 0x7fd45c0b2970 2024-04-25T11:34:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.239+0000 7fd458c00640 1 -- 172.21.15.19:0/810880789 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7fd44805f990 con 0x7fd45c0b2970 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.242+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd43c0744e0 msgr2=0x7fd43c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.242+0000 7fd461e00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd43c0744e0 0x7fd43c0769f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd44c001050 tx=0x7fd44c009040 comp rx=0 tx=0).stop 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.242+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45c0b2970 msgr2=0x7fd45c15c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.242+0000 7fd461e00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45c0b2970 0x7fd45c15c040 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd44800f580 tx=0x7fd44800fa40 comp rx=0 tx=0).stop 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.242+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 shutdown_connections 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.242+0000 7fd461e00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd43c0744e0 0x7fd43c0769f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.243+0000 7fd461e00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd45c15c580 0x7fd45c1604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.243+0000 7fd461e00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45c0b2970 0x7fd45c15c040 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.243+0000 7fd461e00640 1 --2- 172.21.15.19:0/810880789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd440005270 0x7fd45c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.243+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 >> 172.21.15.19:0/810880789 conn(0x7fd45c072f40 msgr2=0x7fd45c13f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:08.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.243+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 shutdown_connections 2024-04-25T11:34:08.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.243+0000 7fd461e00640 1 -- 172.21.15.19:0/810880789 wait complete. 2024-04-25T11:34:08.260 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:34:08.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:34:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.559+0000 7ff4a2000640 1 Processor -- start 2024-04-25T11:34:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.560+0000 7ff4a2000640 1 -- start start 2024-04-25T11:34:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.562+0000 7ff4a2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff49c06dce0 0x7ff49c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.562+0000 7ff4a2000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff49c070000 con 0x7ff49c15ec90 2024-04-25T11:34:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.562+0000 7ff4a2000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff49c06f240 con 0x7ff49c144340 2024-04-25T11:34:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.562+0000 7ff4a2000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff49c144750 con 0x7ff49c06dce0 2024-04-25T11:34:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.563+0000 7ff4a1600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff49c15ec90 0x7ff49c15f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51502/0 (socket says 172.21.15.19:51502) 2024-04-25T11:34:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.563+0000 7ff4a1600640 1 -- 172.21.15.19:0/3714915563 learned_addr learned my addr 172.21.15.19:0/3714915563 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.563+0000 7ff49be00640 1 --2- 172.21.15.19:0/3714915563 >> v2:172.21.15.169:3300/0 conn(0x7ff49c06dce0 0x7ff49c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.563+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 661613994 0 0) 0x7ff49c070000 con 0x7ff49c15ec90 2024-04-25T11:34:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.563+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff484003520 con 0x7ff49c15ec90 2024-04-25T11:34:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2326094731 0 0) 0x7ff484003520 con 0x7ff49c15ec90 2024-04-25T11:34:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 >> v2:172.21.15.169:3300/0 conn(0x7ff49c06dce0 msgr2=0x7ff49c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 --2- 172.21.15.19:0/3714915563 >> v2:172.21.15.169:3300/0 conn(0x7ff49c06dce0 0x7ff49c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 >> v1:172.21.15.169:6789/0 conn(0x7ff49c144340 legacy=0x7ff49c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c06f520 con 0x7ff49c15ec90 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ff494001670 con 0x7ff49c15ec90 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 >> v1:172.21.15.19:6789/0 conn(0x7ff49c15ec90 legacy=0x7ff49c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4840039b0 0x7ff484003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff484004800 0x7ff484004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4840051c0 0x7ff48400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c144750 con 0x7ff484004800 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff48400c160 con 0x7ff4840039b0 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff48400c360 con 0x7ff4840051c0 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.564+0000 7ff49be00640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff484004800 0x7ff484004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff4a1600640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4840051c0 0x7ff48400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff4a0c00640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4840039b0 0x7ff484003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff49be00640 1 -- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4840051c0 msgr2=0x7ff48400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff49be00640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4840051c0 0x7ff48400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff49be00640 1 -- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4840039b0 msgr2=0x7ff484003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff49be00640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4840039b0 0x7ff484003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff49be00640 1 -- 172.21.15.19:0/3714915563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff484003520 con 0x7ff484004800 2024-04-25T11:34:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff4a1600640 1 -- 172.21.15.19:0/3714915563 reap_dead start 2024-04-25T11:34:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.565+0000 7ff4a0c00640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4840039b0 0x7ff484003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.566+0000 7ff49be00640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff484004800 0x7ff484004c10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff488009a90 tx=0x7ff484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1ea02740d1a430 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.566+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488036070 con 0x7ff484004800 2024-04-25T11:34:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.566+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff488031400 con 0x7ff484004800 2024-04-25T11:34:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.566+0000 7ff49b400640 1 -- 172.21.15.19:0/3714915563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488009690 con 0x7ff484004800 2024-04-25T11:34:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.567+0000 7ff4a2000640 1 -- 172.21.15.19:0/3714915563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff484004800 msgr2=0x7ff484004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.567+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff484004800 0x7ff484004c10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff488009a90 tx=0x7ff484005830 comp rx=0 tx=0).stop 2024-04-25T11:34:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.567+0000 7ff4a2000640 1 -- 172.21.15.19:0/3714915563 shutdown_connections 2024-04-25T11:34:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.567+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3714915563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff484004800 0x7ff484004c10 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.567+0000 7ff4a2000640 1 -- 172.21.15.19:0/3714915563 >> 172.21.15.19:0/3714915563 conn(0x7ff49c072f40 msgr2=0x7ff49c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.567+0000 7ff4a2000640 1 -- 172.21.15.19:0/3714915563 shutdown_connections 2024-04-25T11:34:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.567+0000 7ff4a2000640 1 -- 172.21.15.19:0/3714915563 wait complete. 2024-04-25T11:34:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.569+0000 7ff4a2000640 1 Processor -- start 2024-04-25T11:34:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.570+0000 7ff4a2000640 1 -- start start 2024-04-25T11:34:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.571+0000 7ff4a2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff484004800 0x7ff49c0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.571+0000 7ff4a2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c138030 0x7ff49c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.571+0000 7ff4a2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff49c15aaf0 0x7ff49c0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.571+0000 7ff4a2000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c15f0a0 con 0x7ff49c15aaf0 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.571+0000 7ff4a2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c06fae0 con 0x7ff484004800 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.571+0000 7ff4a2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff49c15f220 con 0x7ff49c138030 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff49c15aaf0 0x7ff49c0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff49c15aaf0 0x7ff49c0aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43262/0 (socket says 172.21.15.19:43262) 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 -- 172.21.15.19:0/3105023946 learned_addr learned my addr 172.21.15.19:0/3105023946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a0c00640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff484004800 0x7ff49c0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff49be00640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c138030 0x7ff49c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 -- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c138030 msgr2=0x7ff49c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c138030 0x7ff49c15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 -- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff484004800 msgr2=0x7ff49c0b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff484004800 0x7ff49c0b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 -- 172.21.15.19:0/3105023946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c15af00 con 0x7ff49c15aaf0 2024-04-25T11:34:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff49be00640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c138030 0x7ff49c15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.572+0000 7ff4a1600640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff49c15aaf0 0x7ff49c0aee40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff494001200 tx=0x7ff49400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.573+0000 7ff498c00640 1 -- 172.21.15.19:0/3105023946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494019070 con 0x7ff49c15aaf0 2024-04-25T11:34:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.573+0000 7ff498c00640 1 -- 172.21.15.19:0/3105023946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff494007440 con 0x7ff49c15aaf0 2024-04-25T11:34:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.573+0000 7ff498c00640 1 -- 172.21.15.19:0/3105023946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940047e0 con 0x7ff49c15aaf0 2024-04-25T11:34:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.573+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c0b5c60 con 0x7ff49c15aaf0 2024-04-25T11:34:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.573+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c0b6220 con 0x7ff49c15aaf0 2024-04-25T11:34:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.574+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c076c60 con 0x7ff49c15aaf0 2024-04-25T11:34:08.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.575+0000 7ff498c00640 1 -- 172.21.15.19:0/3105023946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff494004980 con 0x7ff49c15aaf0 2024-04-25T11:34:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.576+0000 7ff498c00640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff47c0744b0 0x7ff47c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.577+0000 7ff498c00640 1 -- 172.21.15.19:0/3105023946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7ff4940948a0 con 0x7ff49c15aaf0 2024-04-25T11:34:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.577+0000 7ff4a0c00640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff47c0744b0 0x7ff47c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.578+0000 7ff4a0c00640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff47c0744b0 0x7ff47c0769c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff490007fd0 tx=0x7ff490009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.581+0000 7ff498c00640 1 -- 172.21.15.19:0/3105023946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff49c15af00 con 0x7ff49c15aaf0 2024-04-25T11:34:08.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:08.825+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff49c15f930 con 0x7ff49c15aaf0 2024-04-25T11:34:09.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.251+0000 7ff498c00640 1 -- 172.21.15.19:0/3105023946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7ff494004c70 con 0x7ff49c15aaf0 2024-04-25T11:34:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.254+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff47c0744b0 msgr2=0x7ff47c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.254+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff47c0744b0 0x7ff47c0769c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff490007fd0 tx=0x7ff490009040 comp rx=0 tx=0).stop 2024-04-25T11:34:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.254+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff49c15aaf0 msgr2=0x7ff49c0aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.254+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff49c15aaf0 0x7ff49c0aee40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff494001200 tx=0x7ff49400d9d0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 shutdown_connections 2024-04-25T11:34:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff47c0744b0 0x7ff47c0769c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff49c15aaf0 0x7ff49c0aee40 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff49c138030 0x7ff49c15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 --2- 172.21.15.19:0/3105023946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff484004800 0x7ff49c0b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 >> 172.21.15.19:0/3105023946 conn(0x7ff49c072f40 msgr2=0x7ff49c142220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 shutdown_connections 2024-04-25T11:34:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.255+0000 7ff4a2000640 1 -- 172.21.15.19:0/3105023946 wait complete. 2024-04-25T11:34:09.275 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:34:09.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.571+0000 7f9c5f000640 1 Processor -- start 2024-04-25T11:34:09.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.571+0000 7f9c5f000640 1 -- start start 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.573+0000 7f9c5f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c581400a0 0x7f9c580b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c5f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c58070980 0x7f9c58142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c5f000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c581431d0 con 0x7f9c58142730 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c5f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9c581434b0 con 0x7f9c581400a0 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c5f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c580b6090 con 0x7f9c58070980 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c57400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9c58142730 0x7f9c58070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51516/0 (socket says 172.21.15.19:51516) 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c57400640 1 -- 172.21.15.19:0/1669219557 learned_addr learned my addr 172.21.15.19:0/1669219557 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1669219557 >> v2:172.21.15.169:3300/0 conn(0x7f9c581400a0 0x7f9c580b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.574+0000 7f9c5d000640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c58070980 0x7f9c58142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385802512 0 0) 0x7f9c581431d0 con 0x7f9c58142730 2024-04-25T11:34:09.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c3c003520 con 0x7f9c58142730 2024-04-25T11:34:09.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3787429559 0 0) 0x7f9c3c003520 con 0x7f9c58142730 2024-04-25T11:34:09.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c58070980 msgr2=0x7f9c58142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c58070980 0x7f9c58142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 >> v2:172.21.15.169:3300/0 conn(0x7f9c581400a0 msgr2=0x7f9c580b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 --2- 172.21.15.19:0/1669219557 >> v2:172.21.15.169:3300/0 conn(0x7f9c581400a0 0x7f9c580b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c580b6370 con 0x7f9c58142730 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9c48001670 con 0x7f9c58142730 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 >> v1:172.21.15.19:6789/0 conn(0x7f9c58142730 legacy=0x7f9c58070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c0039b0 0x7f9c3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c004800 0x7f9c3c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c005150 0x7f9c3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581434b0 con 0x7f9c3c0039b0 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c580b6090 con 0x7f9c3c005150 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.575+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c3c00e110 con 0x7f9c3c004800 2024-04-25T11:34:09.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c0039b0 0x7f9c3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c5d000640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c005150 0x7f9c3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c57e00640 1 -- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c004800 msgr2=0x7f9c3c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c004800 0x7f9c3c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c57e00640 1 -- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c005150 msgr2=0x7f9c3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c005150 0x7f9c3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c57e00640 1 -- 172.21.15.19:0/1669219557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3c003520 con 0x7f9c3c0039b0 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.576+0000 7f9c5d000640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c3c005150 0x7f9c3c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.577+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c0039b0 0x7f9c3c003e20 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009a90 tx=0x7f9c3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612ad5aa81b9aeb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.577+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40036070 con 0x7f9c3c0039b0 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.577+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c40031400 con 0x7f9c3c0039b0 2024-04-25T11:34:09.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.577+0000 7f9c56a00640 1 -- 172.21.15.19:0/1669219557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40009690 con 0x7f9c3c0039b0 2024-04-25T11:34:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.578+0000 7f9c5f000640 1 -- 172.21.15.19:0/1669219557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c0039b0 msgr2=0x7f9c3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.578+0000 7f9c5f000640 1 --2- 172.21.15.19:0/1669219557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c3c0039b0 0x7f9c3c003e20 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009a90 tx=0x7f9c3c005830 comp rx=0 tx=0).stop 2024-04-25T11:34:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.578+0000 7f9c5d000640 1 -- 172.21.15.19:0/1669219557 reap_dead start 2024-04-25T11:34:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.578+0000 7f9c5f000640 1 -- 172.21.15.19:0/1669219557 shutdown_connections 2024-04-25T11:34:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.578+0000 7f9c5f000640 1 -- 172.21.15.19:0/1669219557 >> 172.21.15.19:0/1669219557 conn(0x7f9c58073050 msgr2=0x7f9c58073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.578+0000 7f9c5f000640 1 -- 172.21.15.19:0/1669219557 shutdown_connections 2024-04-25T11:34:09.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.578+0000 7f9c5f000640 1 -- 172.21.15.19:0/1669219557 wait complete. 2024-04-25T11:34:09.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.581+0000 7f9c5f000640 1 Processor -- start 2024-04-25T11:34:09.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.581+0000 7f9c5f000640 1 -- start start 2024-04-25T11:34:09.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.582+0000 7f9c5f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0039b0 0x7f9c581f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.582+0000 7f9c5f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5813eaa0 0x7f9c5813f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.582+0000 7f9c5f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c58160950 0x7f9c58160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.582+0000 7f9c5f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58070980 con 0x7f9c58160950 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.582+0000 7f9c5f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c580cddb0 con 0x7f9c5813eaa0 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.582+0000 7f9c5f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c58070b00 con 0x7f9c3c0039b0 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.583+0000 7f9c5d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c58160950 0x7f9c58160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.583+0000 7f9c5d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c58160950 0x7f9c58160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43286/0 (socket says 172.21.15.19:43286) 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.583+0000 7f9c5d000640 1 -- 172.21.15.19:0/1438867709 learned_addr learned my addr 172.21.15.19:0/1438867709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.583+0000 7f9c57400640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5813eaa0 0x7f9c5813f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.583+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0039b0 0x7f9c581f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5d000640 1 -- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0039b0 msgr2=0x7f9c581f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5d000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0039b0 0x7f9c581f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5d000640 1 -- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5813eaa0 msgr2=0x7f9c5813f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5d000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5813eaa0 0x7f9c5813f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5d000640 1 -- 172.21.15.19:0/1438867709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c581612a0 con 0x7f9c58160950 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c57400640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5813eaa0 0x7f9c5813f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5d000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c58160950 0x7f9c58160d60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9c58072e50 tx=0x7f9c5000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c4fe00640 1 -- 172.21.15.19:0/1438867709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5001c070 con 0x7f9c58160950 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c4fe00640 1 -- 172.21.15.19:0/1438867709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c5000fe90 con 0x7f9c58160950 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c4fe00640 1 -- 172.21.15.19:0/1438867709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50017600 con 0x7f9c58160950 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5813eeb0 con 0x7f9c58160950 2024-04-25T11:34:09.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.584+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c580aa670 con 0x7f9c58160950 2024-04-25T11:34:09.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.586+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c58050740 con 0x7f9c58160950 2024-04-25T11:34:09.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.587+0000 7f9c4fe00640 1 -- 172.21.15.19:0/1438867709 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c50005d10 con 0x7f9c58160950 2024-04-25T11:34:09.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.588+0000 7f9c4fe00640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c340744b0 0x7f9c340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:09.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.588+0000 7f9c4fe00640 1 -- 172.21.15.19:0/1438867709 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f9c50095650 con 0x7f9c58160950 2024-04-25T11:34:09.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.589+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c340744b0 0x7f9c340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:09.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.590+0000 7f9c57e00640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c340744b0 0x7f9c340769c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9c40033000 tx=0x7f9c40008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:09.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.593+0000 7f9c4fe00640 1 -- 172.21.15.19:0/1438867709 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c581612a0 con 0x7f9c58160950 2024-04-25T11:34:09.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.837+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c580b5c60 con 0x7f9c58160950 2024-04-25T11:34:09.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.838+0000 7f9c4fe00640 1 -- 172.21.15.19:0/1438867709 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5186 (secure 0 0 0) 0x7f9c50012070 con 0x7f9c58160950 2024-04-25T11:34:09.826 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:09.827 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:08.826699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5866":{"gid":5866,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/853310115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":853310115},{"type":"v1","addr":"172.21.15.19:6835","nonce":853310115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5866,"qdb_cluster":[5866]},"id":5}]} 2024-04-25T11:34:09.827 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c340744b0 msgr2=0x7f9c340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c340744b0 0x7f9c340769c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9c40033000 tx=0x7f9c40008030 comp rx=0 tx=0).stop 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c58160950 msgr2=0x7f9c58160d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c58160950 0x7f9c58160d60 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9c58072e50 tx=0x7f9c5000f4a0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 shutdown_connections 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c340744b0 0x7f9c340769c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5813eaa0 0x7f9c5813f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c58160950 0x7f9c58160d60 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.840+0000 7f9c5f000640 1 --2- 172.21.15.19:0/1438867709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c3c0039b0 0x7f9c581f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:09.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.841+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 >> 172.21.15.19:0/1438867709 conn(0x7f9c58073050 msgr2=0x7f9c580ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:09.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.841+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 shutdown_connections 2024-04-25T11:34:09.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:09.841+0000 7f9c5f000640 1 -- 172.21.15.19:0/1438867709 wait complete. 2024-04-25T11:34:09.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.150+0000 7fa355000640 1 Processor -- start 2024-04-25T11:34:10.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.150+0000 7fa355000640 1 -- start start 2024-04-25T11:34:10.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.152+0000 7fa355000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa3500b6a00 0x7fa35015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.152+0000 7fa355000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa35015a2f0 0x7fa35006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.153+0000 7fa355000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3501611e0 con 0x7fa35006d390 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.153+0000 7fa355000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa350160d50 con 0x7fa35015a2f0 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.153+0000 7fa355000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa35015a700 con 0x7fa3500b6a00 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.153+0000 7fa34e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa35006d390 0x7fa35006f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51526/0 (socket says 172.21.15.19:51526) 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.153+0000 7fa34e800640 1 -- 172.21.15.19:0/757927844 learned_addr learned my addr 172.21.15.19:0/757927844 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.153+0000 7fa345400640 1 --2- 172.21.15.19:0/757927844 >> v2:172.21.15.19:3300/0 conn(0x7fa35015a2f0 0x7fa35006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.153+0000 7fa34de00640 1 --2- 172.21.15.19:0/757927844 >> v2:172.21.15.169:3300/0 conn(0x7fa3500b6a00 0x7fa35015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.154+0000 7fa34d400640 1 -- 172.21.15.19:0/757927844 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857976571 0 0) 0x7fa3501611e0 con 0x7fa35006d390 2024-04-25T11:34:10.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.154+0000 7fa34d400640 1 -- 172.21.15.19:0/757927844 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa334003520 con 0x7fa35006d390 2024-04-25T11:34:10.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.154+0000 7fa345400640 1 -- 172.21.15.19:0/757927844 >> v2:172.21.15.169:3300/0 conn(0x7fa3500b6a00 msgr2=0x7fa35015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.154+0000 7fa345400640 1 --2- 172.21.15.19:0/757927844 >> v2:172.21.15.169:3300/0 conn(0x7fa3500b6a00 0x7fa35015d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.154+0000 7fa345400640 1 -- 172.21.15.19:0/757927844 >> v1:172.21.15.19:6789/0 conn(0x7fa35006d390 legacy=0x7fa35006f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.154+0000 7fa345400640 1 -- 172.21.15.19:0/757927844 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35006d7a0 con 0x7fa35015a2f0 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.155+0000 7fa345400640 1 --2- 172.21.15.19:0/757927844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35015a2f0 0x7fa35006ce50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa3500713e0 tx=0x7fa33802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567472c61465de83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.155+0000 7fa34d400640 1 -- 172.21.15.19:0/757927844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338037070 con 0x7fa35015a2f0 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.155+0000 7fa34d400640 1 -- 172.21.15.19:0/757927844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa338032440 con 0x7fa35015a2f0 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.155+0000 7fa34d400640 1 -- 172.21.15.19:0/757927844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33802ae60 con 0x7fa35015a2f0 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 -- 172.21.15.19:0/757927844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35015a2f0 msgr2=0x7fa35006ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 --2- 172.21.15.19:0/757927844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35015a2f0 0x7fa35006ce50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa3500713e0 tx=0x7fa33802a110 comp rx=0 tx=0).stop 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 -- 172.21.15.19:0/757927844 shutdown_connections 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 --2- 172.21.15.19:0/757927844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35015a2f0 0x7fa35006ce50 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 --2- 172.21.15.19:0/757927844 >> v2:172.21.15.169:3300/0 conn(0x7fa3500b6a00 0x7fa35015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 -- 172.21.15.19:0/757927844 >> 172.21.15.19:0/757927844 conn(0x7fa3500d1ae0 msgr2=0x7fa3500d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:10.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 -- 172.21.15.19:0/757927844 shutdown_connections 2024-04-25T11:34:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.156+0000 7fa355000640 1 -- 172.21.15.19:0/757927844 wait complete. 2024-04-25T11:34:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.159+0000 7fa355000640 1 Processor -- start 2024-04-25T11:34:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.159+0000 7fa355000640 1 -- start start 2024-04-25T11:34:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.160+0000 7fa355000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35006d390 0x7fa350141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.160+0000 7fa355000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500b6a00 0x7fa35013ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.160+0000 7fa355000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa35015a2f0 0x7fa3500c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.160+0000 7fa355000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501609e0 con 0x7fa35006d390 2024-04-25T11:34:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.160+0000 7fa355000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa350160700 con 0x7fa3500b6a00 2024-04-25T11:34:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.160+0000 7fa355000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa3500b5980 con 0x7fa35015a2f0 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35006d390 0x7fa350141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35006d390 0x7fa350141f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43304/0 (socket says 172.21.15.19:43304) 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 -- 172.21.15.19:0/2646934561 learned_addr learned my addr 172.21.15.19:0/2646934561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa345400640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500b6a00 0x7fa35013ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 -- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa35015a2f0 msgr2=0x7fa3500c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa35015a2f0 0x7fa3500c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 -- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500b6a00 msgr2=0x7fa35013ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500b6a00 0x7fa35013ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa34de00640 1 -- 172.21.15.19:0/2646934561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3501e7f80 con 0x7fa35006d390 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.161+0000 7fa345400640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500b6a00 0x7fa35013ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.162+0000 7fa34de00640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35006d390 0x7fa350141f10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa340009cd0 tx=0x7fa34000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.162+0000 7fa346a00640 1 -- 172.21.15.19:0/2646934561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340018020 con 0x7fa35006d390 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.162+0000 7fa346a00640 1 -- 172.21.15.19:0/2646934561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa340014480 con 0x7fa35006d390 2024-04-25T11:34:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.162+0000 7fa346a00640 1 -- 172.21.15.19:0/2646934561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340014850 con 0x7fa35006d390 2024-04-25T11:34:10.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.162+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3501e8260 con 0x7fa35006d390 2024-04-25T11:34:10.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.162+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501e8820 con 0x7fa35006d390 2024-04-25T11:34:10.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.163+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3500ce580 con 0x7fa35006d390 2024-04-25T11:34:10.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.164+0000 7fa346a00640 1 -- 172.21.15.19:0/2646934561 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa340005d10 con 0x7fa35006d390 2024-04-25T11:34:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.165+0000 7fa346a00640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa32c0744b0 0x7fa32c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.166+0000 7fa346a00640 1 -- 172.21.15.19:0/2646934561 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fa340096060 con 0x7fa35006d390 2024-04-25T11:34:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.166+0000 7fa345400640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa32c0744b0 0x7fa32c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.168+0000 7fa345400640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa32c0744b0 0x7fa32c0769c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa338034000 tx=0x7fa33802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:10.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.171+0000 7fa346a00640 1 -- 172.21.15.19:0/2646934561 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3500ce580 con 0x7fa35006d390 2024-04-25T11:34:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.381+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa350050740 con 0x7fa35006d390 2024-04-25T11:34:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.383+0000 7fa346a00640 1 -- 172.21.15.19:0/2646934561 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16608 (secure 0 0 0) 0x7fa340022030 con 0x7fa35006d390 2024-04-25T11:34:10.372 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:10.372 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":79,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:07.222707+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:34:04.018735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T11:34:04.759136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa32c0744b0 msgr2=0x7fa32c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa32c0744b0 0x7fa32c0769c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa338034000 tx=0x7fa33802e000 comp rx=0 tx=0).stop 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35006d390 msgr2=0x7fa350141f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35006d390 0x7fa350141f10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa340009cd0 tx=0x7fa34000c490 comp rx=0 tx=0).stop 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 shutdown_connections 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa32c0744b0 0x7fa32c0769c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa3500b6a00 0x7fa35013ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa35015a2f0 0x7fa3500c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 --2- 172.21.15.19:0/2646934561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa35006d390 0x7fa350141f10 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 >> 172.21.15.19:0/2646934561 conn(0x7fa3500d1ae0 msgr2=0x7fa3500b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 shutdown_connections 2024-04-25T11:34:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.386+0000 7fa355000640 1 -- 172.21.15.19:0/2646934561 wait complete. 2024-04-25T11:34:10.401 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:34:10.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:34:10.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.700+0000 7f4bc0800640 1 Processor -- start 2024-04-25T11:34:10.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.700+0000 7f4bc0800640 1 -- start start 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.702+0000 7f4bc0800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4bb8144500 0x7f4bb8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.702+0000 7f4bc0800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4bb8162310 0x7f4bb80ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.702+0000 7f4bc0800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb806f5f0 con 0x7f4bb80cdb90 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.702+0000 7f4bc0800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb8070200 con 0x7f4bb8144500 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.702+0000 7f4bc0800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb806f8d0 con 0x7f4bb8162310 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbde00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4bb8144500 0x7f4bb8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbde00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4bb8144500 0x7f4bb8070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43320/0 (socket says 172.21.15.19:43320) 2024-04-25T11:34:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbde00640 1 -- 172.21.15.19:0/582402144 learned_addr learned my addr 172.21.15.19:0/582402144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbe800640 1 --2- 172.21.15.19:0/582402144 >> v2:172.21.15.169:3300/0 conn(0x7f4bb8162310 0x7f4bb80ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbde00640 1 -- 172.21.15.19:0/582402144 >> v2:172.21.15.169:3300/0 conn(0x7f4bb8162310 msgr2=0x7f4bb80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbde00640 1 --2- 172.21.15.19:0/582402144 >> v2:172.21.15.169:3300/0 conn(0x7f4bb8162310 0x7f4bb80ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbde00640 1 -- 172.21.15.19:0/582402144 >> v1:172.21.15.19:6789/0 conn(0x7f4bb80cdb90 legacy=0x7f4bb8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbde00640 1 -- 172.21.15.19:0/582402144 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb806fbb0 con 0x7f4bb8144500 2024-04-25T11:34:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.703+0000 7f4bbca00640 1 -- 172.21.15.19:0/582402144 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2653130509 0 0) 0x7f4bb806f5f0 con 0x7f4bb80cdb90 2024-04-25T11:34:10.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.705+0000 7f4bbde00640 1 --2- 172.21.15.19:0/582402144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb8144500 0x7f4bb8070690 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80ce840 tx=0x7f4bac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759d1be364b13b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:10.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.705+0000 7f4bbca00640 1 -- 172.21.15.19:0/582402144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac037070 con 0x7f4bb8144500 2024-04-25T11:34:10.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.705+0000 7f4bbca00640 1 -- 172.21.15.19:0/582402144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac032440 con 0x7f4bb8144500 2024-04-25T11:34:10.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.705+0000 7f4bbca00640 1 -- 172.21.15.19:0/582402144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac032810 con 0x7f4bb8144500 2024-04-25T11:34:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 -- 172.21.15.19:0/582402144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb8144500 msgr2=0x7f4bb8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 --2- 172.21.15.19:0/582402144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb8144500 0x7f4bb8070690 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80ce840 tx=0x7f4bac02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 -- 172.21.15.19:0/582402144 shutdown_connections 2024-04-25T11:34:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 --2- 172.21.15.19:0/582402144 >> v2:172.21.15.169:3300/0 conn(0x7f4bb8162310 0x7f4bb80ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 --2- 172.21.15.19:0/582402144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb8144500 0x7f4bb8070690 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 -- 172.21.15.19:0/582402144 >> 172.21.15.19:0/582402144 conn(0x7f4bb8073050 msgr2=0x7f4bb8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 -- 172.21.15.19:0/582402144 shutdown_connections 2024-04-25T11:34:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.706+0000 7f4bc0800640 1 -- 172.21.15.19:0/582402144 wait complete. 2024-04-25T11:34:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.708+0000 7f4bc0800640 1 Processor -- start 2024-04-25T11:34:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.709+0000 7f4bc0800640 1 -- start start 2024-04-25T11:34:10.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.710+0000 7f4bc0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80cdb90 0x7f4bb8143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.710+0000 7f4bc0800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bb8144500 0x7f4bb80cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.710+0000 7f4bc0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80cc480 0x7f4bb81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.710+0000 7f4bc0800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb806f2e0 con 0x7f4bb80cc480 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.710+0000 7f4bc0800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb806f160 con 0x7f4bb8144500 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.710+0000 7f4bc0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb80cdfa0 con 0x7f4bb80cdb90 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80cc480 0x7f4bb81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80cc480 0x7f4bb81586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43334/0 (socket says 172.21.15.19:43334) 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 -- 172.21.15.19:0/853371719 learned_addr learned my addr 172.21.15.19:0/853371719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbd400640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bb8144500 0x7f4bb80cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 -- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80cdb90 msgr2=0x7f4bb8143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80cdb90 0x7f4bb8143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 -- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bb8144500 msgr2=0x7f4bb80cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bb8144500 0x7f4bb80cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 -- 172.21.15.19:0/853371719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb81f6f90 con 0x7f4bb80cc480 2024-04-25T11:34:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.711+0000 7f4bbe800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80cc480 0x7f4bb81586d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4007d40 tx=0x7f4bb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:10.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.712+0000 7f4bb2000640 1 -- 172.21.15.19:0/853371719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4014420 con 0x7f4bb80cc480 2024-04-25T11:34:10.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.712+0000 7f4bb2000640 1 -- 172.21.15.19:0/853371719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb4019070 con 0x7f4bb80cc480 2024-04-25T11:34:10.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.712+0000 7f4bb2000640 1 -- 172.21.15.19:0/853371719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4014820 con 0x7f4bb80cc480 2024-04-25T11:34:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.712+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81f7270 con 0x7f4bb80cc480 2024-04-25T11:34:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.712+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81f7830 con 0x7f4bb80cc480 2024-04-25T11:34:10.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.713+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb806d810 con 0x7f4bb80cc480 2024-04-25T11:34:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.715+0000 7f4bb2000640 1 -- 172.21.15.19:0/853371719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4bb40149c0 con 0x7f4bb80cc480 2024-04-25T11:34:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.716+0000 7f4bb2000640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b9c0744e0 0x7f4b9c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.716+0000 7f4bb2000640 1 -- 172.21.15.19:0/853371719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f4bb4095a70 con 0x7f4bb80cc480 2024-04-25T11:34:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.716+0000 7f4bbde00640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b9c0744e0 0x7f4b9c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:10.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.718+0000 7f4bbde00640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b9c0744e0 0x7f4b9c0769f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4bac034000 tx=0x7f4bac02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:10.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.720+0000 7f4bb2000640 1 -- 172.21.15.19:0/853371719 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bb806d810 con 0x7f4bb80cc480 2024-04-25T11:34:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.921+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bb80d0e10 con 0x7f4b9c0744e0 2024-04-25T11:34:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.942+0000 7f4bb2000640 1 -- 172.21.15.19:0/853371719 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252049 (secure 0 0 0) 0x7f4bb80d0e10 con 0x7f4b9c0744e0 2024-04-25T11:34:10.933 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:10.935 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b9c0744e0 msgr2=0x7f4b9c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b9c0744e0 0x7f4b9c0769f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4bac034000 tx=0x7f4bac02c000 comp rx=0 tx=0).stop 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80cc480 msgr2=0x7f4bb81586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80cc480 0x7f4bb81586d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4007d40 tx=0x7f4bb400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 shutdown_connections 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b9c0744e0 0x7f4b9c0769f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bb8144500 0x7f4bb80cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb80cc480 0x7f4bb81586d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 --2- 172.21.15.19:0/853371719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80cdb90 0x7f4bb8143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 >> 172.21.15.19:0/853371719 conn(0x7f4bb8073050 msgr2=0x7f4bb815fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.946+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 shutdown_connections 2024-04-25T11:34:10.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:10.947+0000 7f4bc0800640 1 -- 172.21.15.19:0/853371719 wait complete. 2024-04-25T11:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-04-25T11:34:10.610373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253880,"kb_used_data":8592,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720840,"statfs":{"total":773094113280,"available":772834140160,"internally_reserved":0,"allocated":8798208,"data_stored":4415059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":251055666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002399"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239887+0000","last_change":"2024-04-25T11:34:06.248049+0000","last_active":"2024-04-25T11:34:07.239887+0000","last_peered":"2024-04-25T11:34:07.239887+0000","last_clean":"2024-04-25T11:34:07.239887+0000","last_became_active":"2024-04-25T11:34:06.243041+0000","last_became_peered":"2024-04-25T11:34:06.243041+0000","last_unstale":"2024-04-25T11:34:07.239887+0000","last_undegraded":"2024-04-25T11:34:07.239887+0000","last_fullsized":"2024-04-25T11:34:07.239887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:33.036618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481795+0000","last_change":"2024-04-25T11:34:06.253171+0000","last_active":"2024-04-25T11:34:07.481795+0000","last_peered":"2024-04-25T11:34:07.481795+0000","last_clean":"2024-04-25T11:34:07.481795+0000","last_became_active":"2024-04-25T11:34:06.252442+0000","last_became_peered":"2024-04-25T11:34:06.252442+0000","last_unstale":"2024-04-25T11:34:07.481795+0000","last_undegraded":"2024-04-25T11:34:07.481795+0000","last_fullsized":"2024-04-25T11:34:07.481795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:43.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245574+0000","last_change":"2024-04-25T11:34:06.249591+0000","last_active":"2024-04-25T11:34:07.245574+0000","last_peered":"2024-04-25T11:34:07.245574+0000","last_clean":"2024-04-25T11:34:07.245574+0000","last_became_active":"2024-04-25T11:34:06.249210+0000","last_became_peered":"2024-04-25T11:34:06.249210+0000","last_unstale":"2024-04-25T11:34:07.245574+0000","last_undegraded":"2024-04-25T11:34:07.245574+0000","last_fullsized":"2024-04-25T11:34:07.245574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:48.469536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244324+0000","last_change":"2024-04-25T11:34:06.249607+0000","last_active":"2024-04-25T11:34:07.244324+0000","last_peered":"2024-04-25T11:34:07.244324+0000","last_clean":"2024-04-25T11:34:07.244324+0000","last_became_active":"2024-04-25T11:34:06.249210+0000","last_became_peered":"2024-04-25T11:34:06.249210+0000","last_unstale":"2024-04-25T11:34:07.244324+0000","last_undegraded":"2024-04-25T11:34:07.244324+0000","last_fullsized":"2024-04-25T11:34:07.244324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.416520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.259137+0000","last_change":"2024-04-25T11:34:06.255106+0000","last_active":"2024-04-25T11:34:06.259137+0000","last_peered":"2024-04-25T11:34:06.259137+0000","last_clean":"2024-04-25T11:34:06.259137+0000","last_became_active":"2024-04-25T11:34:06.254601+0000","last_became_peered":"2024-04-25T11:34:06.254601+0000","last_unstale":"2024-04-25T11:34:06.259137+0000","last_undegraded":"2024-04-25T11:34:06.259137+0000","last_fullsized":"2024-04-25T11:34:06.259137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:28.385384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483311+0000","last_change":"2024-04-25T11:34:06.258744+0000","last_active":"2024-04-25T11:34:07.483311+0000","last_peered":"2024-04-25T11:34:07.483311+0000","last_clean":"2024-04-25T11:34:07.483311+0000","last_became_active":"2024-04-25T11:34:06.258432+0000","last_became_peered":"2024-04-25T11:34:06.258432+0000","last_unstale":"2024-04-25T11:34:07.483311+0000","last_undegraded":"2024-04-25T11:34:07.483311+0000","last_fullsized":"2024-04-25T11:34:07.483311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:57.577377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246264+0000","last_change":"2024-04-25T11:34:06.224284+0000","last_active":"2024-04-25T11:34:07.246264+0000","last_peered":"2024-04-25T11:34:07.246264+0000","last_clean":"2024-04-25T11:34:07.246264+0000","last_became_active":"2024-04-25T11:34:06.223247+0000","last_became_peered":"2024-04-25T11:34:06.223247+0000","last_unstale":"2024-04-25T11:34:07.246264+0000","last_undegraded":"2024-04-25T11:34:07.246264+0000","last_fullsized":"2024-04-25T11:34:07.246264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.169600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239051+0000","last_change":"2024-04-25T11:34:06.246402+0000","last_active":"2024-04-25T11:34:07.239051+0000","last_peered":"2024-04-25T11:34:07.239051+0000","last_clean":"2024-04-25T11:34:07.239051+0000","last_became_active":"2024-04-25T11:34:06.245346+0000","last_became_peered":"2024-04-25T11:34:06.245346+0000","last_unstale":"2024-04-25T11:34:07.239051+0000","last_undegraded":"2024-04-25T11:34:07.239051+0000","last_fullsized":"2024-04-25T11:34:07.239051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:02.471215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245766+0000","last_change":"2024-04-25T11:34:06.245260+0000","last_active":"2024-04-25T11:34:07.245766+0000","last_peered":"2024-04-25T11:34:07.245766+0000","last_clean":"2024-04-25T11:34:07.245766+0000","last_became_active":"2024-04-25T11:34:06.244957+0000","last_became_peered":"2024-04-25T11:34:06.244957+0000","last_unstale":"2024-04-25T11:34:07.245766+0000","last_undegraded":"2024-04-25T11:34:07.245766+0000","last_fullsized":"2024-04-25T11:34:07.245766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:46.011420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247299+0000","last_change":"2024-04-25T11:34:06.228439+0000","last_active":"2024-04-25T11:34:07.247299+0000","last_peered":"2024-04-25T11:34:07.247299+0000","last_clean":"2024-04-25T11:34:07.247299+0000","last_became_active":"2024-04-25T11:34:06.227493+0000","last_became_peered":"2024-04-25T11:34:06.227493+0000","last_unstale":"2024-04-25T11:34:07.247299+0000","last_undegraded":"2024-04-25T11:34:07.247299+0000","last_fullsized":"2024-04-25T11:34:07.247299+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:03.772611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240070+0000","last_change":"2024-04-25T11:34:06.245966+0000","last_active":"2024-04-25T11:34:07.240070+0000","last_peered":"2024-04-25T11:34:07.240070+0000","last_clean":"2024-04-25T11:34:07.240070+0000","last_became_active":"2024-04-25T11:34:06.245433+0000","last_became_peered":"2024-04-25T11:34:06.245433+0000","last_unstale":"2024-04-25T11:34:07.240070+0000","last_undegraded":"2024-04-25T11:34:07.240070+0000","last_fullsized":"2024-04-25T11:34:07.240070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:37.989420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248585+0000","last_change":"2024-04-25T11:34:06.244291+0000","last_active":"2024-04-25T11:34:07.248585+0000","last_peered":"2024-04-25T11:34:07.248585+0000","last_clean":"2024-04-25T11:34:07.248585+0000","last_became_active":"2024-04-25T11:34:06.243951+0000","last_became_peered":"2024-04-25T11:34:06.243951+0000","last_unstale":"2024-04-25T11:34:07.248585+0000","last_undegraded":"2024-04-25T11:34:07.248585+0000","last_fullsized":"2024-04-25T11:34:07.248585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:01.685889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244822+0000","last_change":"2024-04-25T11:34:06.249832+0000","last_active":"2024-04-25T11:34:07.244822+0000","last_peered":"2024-04-25T11:34:07.244822+0000","last_clean":"2024-04-25T11:34:07.244822+0000","last_became_active":"2024-04-25T11:34:06.249222+0000","last_became_peered":"2024-04-25T11:34:06.249222+0000","last_unstale":"2024-04-25T11:34:07.244822+0000","last_undegraded":"2024-04-25T11:34:07.244822+0000","last_fullsized":"2024-04-25T11:34:07.244822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:00.980813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248311+0000","last_change":"2024-04-25T11:34:06.246909+0000","last_active":"2024-04-25T11:34:07.248311+0000","last_peered":"2024-04-25T11:34:07.248311+0000","last_clean":"2024-04-25T11:34:07.248311+0000","last_became_active":"2024-04-25T11:34:06.245458+0000","last_became_peered":"2024-04-25T11:34:06.245458+0000","last_unstale":"2024-04-25T11:34:07.248311+0000","last_undegraded":"2024-04-25T11:34:07.248311+0000","last_fullsized":"2024-04-25T11:34:07.248311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:59.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247758+0000","last_change":"2024-04-25T11:34:06.246646+0000","last_active":"2024-04-25T11:34:07.247758+0000","last_peered":"2024-04-25T11:34:07.247758+0000","last_clean":"2024-04-25T11:34:07.247758+0000","last_became_active":"2024-04-25T11:34:06.238961+0000","last_became_peered":"2024-04-25T11:34:06.238961+0000","last_unstale":"2024-04-25T11:34:07.247758+0000","last_undegraded":"2024-04-25T11:34:07.247758+0000","last_fullsized":"2024-04-25T11:34:07.247758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:07.353595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239272+0000","last_change":"2024-04-25T11:34:06.245697+0000","last_active":"2024-04-25T11:34:07.239272+0000","last_peered":"2024-04-25T11:34:07.239272+0000","last_clean":"2024-04-25T11:34:07.239272+0000","last_became_active":"2024-04-25T11:34:06.244897+0000","last_became_peered":"2024-04-25T11:34:06.244897+0000","last_unstale":"2024-04-25T11:34:07.239272+0000","last_undegraded":"2024-04-25T11:34:07.239272+0000","last_fullsized":"2024-04-25T11:34:07.239272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:52.212358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.240296+0000","last_change":"2024-04-25T11:34:06.235348+0000","last_active":"2024-04-25T11:34:06.240296+0000","last_peered":"2024-04-25T11:34:06.240296+0000","last_clean":"2024-04-25T11:34:06.240296+0000","last_became_active":"2024-04-25T11:34:06.235004+0000","last_became_peered":"2024-04-25T11:34:06.235004+0000","last_unstale":"2024-04-25T11:34:06.240296+0000","last_undegraded":"2024-04-25T11:34:06.240296+0000","last_fullsized":"2024-04-25T11:34:06.240296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:28.172502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.239584+0000","last_change":"2024-04-25T11:34:06.233392+0000","last_active":"2024-04-25T11:34:06.239584+0000","last_peered":"2024-04-25T11:34:06.239584+0000","last_clean":"2024-04-25T11:34:06.239584+0000","last_became_active":"2024-04-25T11:34:06.232959+0000","last_became_peered":"2024-04-25T11:34:06.232959+0000","last_unstale":"2024-04-25T11:34:06.239584+0000","last_undegraded":"2024-04-25T11:34:06.239584+0000","last_fullsized":"2024-04-25T11:34:06.239584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:08.909757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239820+0000","last_change":"2024-04-25T11:34:06.254370+0000","last_active":"2024-04-25T11:34:07.239820+0000","last_peered":"2024-04-25T11:34:07.239820+0000","last_clean":"2024-04-25T11:34:07.239820+0000","last_became_active":"2024-04-25T11:34:06.251958+0000","last_became_peered":"2024-04-25T11:34:06.251958+0000","last_unstale":"2024-04-25T11:34:07.239820+0000","last_undegraded":"2024-04-25T11:34:07.239820+0000","last_fullsized":"2024-04-25T11:34:07.239820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:17.742305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.256439+0000","last_change":"2024-04-25T11:34:06.253247+0000","last_active":"2024-04-25T11:34:06.256439+0000","last_peered":"2024-04-25T11:34:06.256439+0000","last_clean":"2024-04-25T11:34:06.256439+0000","last_became_active":"2024-04-25T11:34:06.252909+0000","last_became_peered":"2024-04-25T11:34:06.252909+0000","last_unstale":"2024-04-25T11:34:06.256439+0000","last_undegraded":"2024-04-25T11:34:06.256439+0000","last_fullsized":"2024-04-25T11:34:06.256439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:16.119265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481384+0000","last_change":"2024-04-25T11:34:06.247920+0000","last_active":"2024-04-25T11:34:07.481384+0000","last_peered":"2024-04-25T11:34:07.481384+0000","last_clean":"2024-04-25T11:34:07.481384+0000","last_became_active":"2024-04-25T11:34:06.244097+0000","last_became_peered":"2024-04-25T11:34:06.244097+0000","last_unstale":"2024-04-25T11:34:07.481384+0000","last_undegraded":"2024-04-25T11:34:07.481384+0000","last_fullsized":"2024-04-25T11:34:07.481384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:11.226289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483138+0000","last_change":"2024-04-25T11:34:06.249507+0000","last_active":"2024-04-25T11:34:07.483138+0000","last_peered":"2024-04-25T11:34:07.483138+0000","last_clean":"2024-04-25T11:34:07.483138+0000","last_became_active":"2024-04-25T11:34:06.249194+0000","last_became_peered":"2024-04-25T11:34:06.249194+0000","last_unstale":"2024-04-25T11:34:07.483138+0000","last_undegraded":"2024-04-25T11:34:07.483138+0000","last_fullsized":"2024-04-25T11:34:07.483138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:01.685889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246669+0000","last_change":"2024-04-25T11:34:06.246937+0000","last_active":"2024-04-25T11:34:07.246669+0000","last_peered":"2024-04-25T11:34:07.246669+0000","last_clean":"2024-04-25T11:34:07.246669+0000","last_became_active":"2024-04-25T11:34:06.246417+0000","last_became_peered":"2024-04-25T11:34:06.246417+0000","last_unstale":"2024-04-25T11:34:07.246669+0000","last_undegraded":"2024-04-25T11:34:07.246669+0000","last_fullsized":"2024-04-25T11:34:07.246669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:12.727113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239935+0000","last_change":"2024-04-25T11:34:06.246692+0000","last_active":"2024-04-25T11:34:07.239935+0000","last_peered":"2024-04-25T11:34:07.239935+0000","last_clean":"2024-04-25T11:34:07.239935+0000","last_became_active":"2024-04-25T11:34:06.246076+0000","last_became_peered":"2024-04-25T11:34:06.246076+0000","last_unstale":"2024-04-25T11:34:07.239935+0000","last_undegraded":"2024-04-25T11:34:07.239935+0000","last_fullsized":"2024-04-25T11:34:07.239935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:22.842505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247193+0000","last_change":"2024-04-25T11:34:06.242235+0000","last_active":"2024-04-25T11:34:07.247193+0000","last_peered":"2024-04-25T11:34:07.247193+0000","last_clean":"2024-04-25T11:34:07.247193+0000","last_became_active":"2024-04-25T11:34:06.241839+0000","last_became_peered":"2024-04-25T11:34:06.241839+0000","last_unstale":"2024-04-25T11:34:07.247193+0000","last_undegraded":"2024-04-25T11:34:07.247193+0000","last_fullsized":"2024-04-25T11:34:07.247193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:50.529668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239154+0000","last_change":"2024-04-25T11:34:06.249562+0000","last_active":"2024-04-25T11:34:07.239154+0000","last_peered":"2024-04-25T11:34:07.239154+0000","last_clean":"2024-04-25T11:34:07.239154+0000","last_became_active":"2024-04-25T11:34:06.249204+0000","last_became_peered":"2024-04-25T11:34:06.249204+0000","last_unstale":"2024-04-25T11:34:07.239154+0000","last_undegraded":"2024-04-25T11:34:07.239154+0000","last_fullsized":"2024-04-25T11:34:07.239154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:54.163242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.245055+0000","last_change":"2024-04-25T11:34:06.241681+0000","last_active":"2024-04-25T11:34:06.245055+0000","last_peered":"2024-04-25T11:34:06.245055+0000","last_clean":"2024-04-25T11:34:06.245055+0000","last_became_active":"2024-04-25T11:34:06.239306+0000","last_became_peered":"2024-04-25T11:34:06.239306+0000","last_unstale":"2024-04-25T11:34:06.245055+0000","last_undegraded":"2024-04-25T11:34:06.245055+0000","last_fullsized":"2024-04-25T11:34:06.245055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:23.928567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482454+0000","last_change":"2024-04-25T11:34:06.244295+0000","last_active":"2024-04-25T11:34:07.482454+0000","last_peered":"2024-04-25T11:34:07.482454+0000","last_clean":"2024-04-25T11:34:07.482454+0000","last_became_active":"2024-04-25T11:34:06.243951+0000","last_became_peered":"2024-04-25T11:34:06.243951+0000","last_unstale":"2024-04-25T11:34:07.482454+0000","last_undegraded":"2024-04-25T11:34:07.482454+0000","last_fullsized":"2024-04-25T11:34:07.482454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:44.452645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244764+0000","last_change":"2024-04-25T11:34:06.258335+0000","last_active":"2024-04-25T11:34:07.244764+0000","last_peered":"2024-04-25T11:34:07.244764+0000","last_clean":"2024-04-25T11:34:07.244764+0000","last_became_active":"2024-04-25T11:34:06.257759+0000","last_became_peered":"2024-04-25T11:34:06.257759+0000","last_unstale":"2024-04-25T11:34:07.244764+0000","last_undegraded":"2024-04-25T11:34:07.244764+0000","last_fullsized":"2024-04-25T11:34:07.244764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:48.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239162+0000","last_change":"2024-04-25T11:34:06.255435+0000","last_active":"2024-04-25T11:34:07.239162+0000","last_peered":"2024-04-25T11:34:07.239162+0000","last_clean":"2024-04-25T11:34:07.239162+0000","last_became_active":"2024-04-25T11:34:06.254549+0000","last_became_peered":"2024-04-25T11:34:06.254549+0000","last_unstale":"2024-04-25T11:34:07.239162+0000","last_undegraded":"2024-04-25T11:34:07.239162+0000","last_fullsized":"2024-04-25T11:34:07.239162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:19.046853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481308+0000","last_change":"2024-04-25T11:34:06.246191+0000","last_active":"2024-04-25T11:34:07.481308+0000","last_peered":"2024-04-25T11:34:07.481308+0000","last_clean":"2024-04-25T11:34:07.481308+0000","last_became_active":"2024-04-25T11:34:06.245858+0000","last_became_peered":"2024-04-25T11:34:06.245858+0000","last_unstale":"2024-04-25T11:34:07.481308+0000","last_undegraded":"2024-04-25T11:34:07.481308+0000","last_fullsized":"2024-04-25T11:34:07.481308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:03.271274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481038+0000","last_change":"2024-04-25T11:34:06.242072+0000","last_active":"2024-04-25T11:34:07.481038+0000","last_peered":"2024-04-25T11:34:07.481038+0000","last_clean":"2024-04-25T11:34:07.481038+0000","last_became_active":"2024-04-25T11:34:06.241659+0000","last_became_peered":"2024-04-25T11:34:06.241659+0000","last_unstale":"2024-04-25T11:34:07.481038+0000","last_undegraded":"2024-04-25T11:34:07.481038+0000","last_fullsized":"2024-04-25T11:34:07.481038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:29.542275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239456+0000","last_change":"2024-04-25T11:34:06.245505+0000","last_active":"2024-04-25T11:34:07.239456+0000","last_peered":"2024-04-25T11:34:07.239456+0000","last_clean":"2024-04-25T11:34:07.239456+0000","last_became_active":"2024-04-25T11:34:06.245121+0000","last_became_peered":"2024-04-25T11:34:06.245121+0000","last_unstale":"2024-04-25T11:34:07.239456+0000","last_undegraded":"2024-04-25T11:34:07.239456+0000","last_fullsized":"2024-04-25T11:34:07.239456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:14.438600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.484100+0000","last_change":"2024-04-25T11:34:05.243143+0000","last_active":"2024-04-25T11:34:07.484100+0000","last_peered":"2024-04-25T11:34:07.484100+0000","last_clean":"2024-04-25T11:34:07.484100+0000","last_became_active":"2024-04-25T11:34:05.242183+0000","last_became_peered":"2024-04-25T11:34:05.242183+0000","last_unstale":"2024-04-25T11:34:07.484100+0000","last_undegraded":"2024-04-25T11:34:07.484100+0000","last_fullsized":"2024-04-25T11:34:07.484100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:28.525172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245696+0000","last_change":"2024-04-25T11:34:06.244568+0000","last_active":"2024-04-25T11:34:07.245696+0000","last_peered":"2024-04-25T11:34:07.245696+0000","last_clean":"2024-04-25T11:34:07.245696+0000","last_became_active":"2024-04-25T11:34:06.244191+0000","last_became_peered":"2024-04-25T11:34:06.244191+0000","last_unstale":"2024-04-25T11:34:07.245696+0000","last_undegraded":"2024-04-25T11:34:07.245696+0000","last_fullsized":"2024-04-25T11:34:07.245696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:48.211622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.263488+0000","last_change":"2024-04-25T11:34:05.243981+0000","last_active":"2024-04-25T11:34:06.263488+0000","last_peered":"2024-04-25T11:34:06.263488+0000","last_clean":"2024-04-25T11:34:06.263488+0000","last_became_active":"2024-04-25T11:34:05.243334+0000","last_became_peered":"2024-04-25T11:34:05.243334+0000","last_unstale":"2024-04-25T11:34:06.263488+0000","last_undegraded":"2024-04-25T11:34:06.263488+0000","last_fullsized":"2024-04-25T11:34:06.263488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:47.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483505+0000","last_change":"2024-04-25T11:34:06.253060+0000","last_active":"2024-04-25T11:34:07.483505+0000","last_peered":"2024-04-25T11:34:07.483505+0000","last_clean":"2024-04-25T11:34:07.483505+0000","last_became_active":"2024-04-25T11:34:06.252733+0000","last_became_peered":"2024-04-25T11:34:06.252733+0000","last_unstale":"2024-04-25T11:34:07.483505+0000","last_undegraded":"2024-04-25T11:34:07.483505+0000","last_fullsized":"2024-04-25T11:34:07.483505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:54.924418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240215+0000","last_change":"2024-04-25T11:34:05.236597+0000","last_active":"2024-04-25T11:34:07.240215+0000","last_peered":"2024-04-25T11:34:07.240215+0000","last_clean":"2024-04-25T11:34:07.240215+0000","last_became_active":"2024-04-25T11:34:05.236205+0000","last_became_peered":"2024-04-25T11:34:05.236205+0000","last_unstale":"2024-04-25T11:34:07.240215+0000","last_undegraded":"2024-04-25T11:34:07.240215+0000","last_fullsized":"2024-04-25T11:34:07.240215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:05.340167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246689+0000","last_change":"2024-04-25T11:34:06.243440+0000","last_active":"2024-04-25T11:34:07.246689+0000","last_peered":"2024-04-25T11:34:07.246689+0000","last_clean":"2024-04-25T11:34:07.246689+0000","last_became_active":"2024-04-25T11:34:06.242919+0000","last_became_peered":"2024-04-25T11:34:06.242919+0000","last_unstale":"2024-04-25T11:34:07.246689+0000","last_undegraded":"2024-04-25T11:34:07.246689+0000","last_fullsized":"2024-04-25T11:34:07.246689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.279224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483695+0000","last_change":"2024-04-25T11:34:05.235526+0000","last_active":"2024-04-25T11:34:07.483695+0000","last_peered":"2024-04-25T11:34:07.483695+0000","last_clean":"2024-04-25T11:34:07.483695+0000","last_became_active":"2024-04-25T11:34:05.235143+0000","last_became_peered":"2024-04-25T11:34:05.235143+0000","last_unstale":"2024-04-25T11:34:07.483695+0000","last_undegraded":"2024-04-25T11:34:07.483695+0000","last_fullsized":"2024-04-25T11:34:07.483695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:10.209186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239828+0000","last_change":"2024-04-25T11:34:06.253944+0000","last_active":"2024-04-25T11:34:07.239828+0000","last_peered":"2024-04-25T11:34:07.239828+0000","last_clean":"2024-04-25T11:34:07.239828+0000","last_became_active":"2024-04-25T11:34:06.253319+0000","last_became_peered":"2024-04-25T11:34:06.253319+0000","last_unstale":"2024-04-25T11:34:07.239828+0000","last_undegraded":"2024-04-25T11:34:07.239828+0000","last_fullsized":"2024-04-25T11:34:07.239828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:25.239741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248217+0000","last_change":"2024-04-25T11:34:05.238928+0000","last_active":"2024-04-25T11:34:07.248217+0000","last_peered":"2024-04-25T11:34:07.248217+0000","last_clean":"2024-04-25T11:34:07.248217+0000","last_became_active":"2024-04-25T11:34:05.238522+0000","last_became_peered":"2024-04-25T11:34:05.238522+0000","last_unstale":"2024-04-25T11:34:07.248217+0000","last_undegraded":"2024-04-25T11:34:07.248217+0000","last_fullsized":"2024-04-25T11:34:07.248217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:09.855120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483684+0000","last_change":"2024-04-25T11:34:06.247101+0000","last_active":"2024-04-25T11:34:07.483684+0000","last_peered":"2024-04-25T11:34:07.483684+0000","last_clean":"2024-04-25T11:34:07.483684+0000","last_became_active":"2024-04-25T11:34:06.246792+0000","last_became_peered":"2024-04-25T11:34:06.246792+0000","last_unstale":"2024-04-25T11:34:07.483684+0000","last_undegraded":"2024-04-25T11:34:07.483684+0000","last_fullsized":"2024-04-25T11:34:07.483684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:25.458654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246353+0000","last_change":"2024-04-25T11:34:05.232685+0000","last_active":"2024-04-25T11:34:07.246353+0000","last_peered":"2024-04-25T11:34:07.246353+0000","last_clean":"2024-04-25T11:34:07.246353+0000","last_became_active":"2024-04-25T11:34:05.232166+0000","last_became_peered":"2024-04-25T11:34:05.232166+0000","last_unstale":"2024-04-25T11:34:07.246353+0000","last_undegraded":"2024-04-25T11:34:07.246353+0000","last_fullsized":"2024-04-25T11:34:07.246353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:36.105936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482838+0000","last_change":"2024-04-25T11:34:06.225391+0000","last_active":"2024-04-25T11:34:07.482838+0000","last_peered":"2024-04-25T11:34:07.482838+0000","last_clean":"2024-04-25T11:34:07.482838+0000","last_became_active":"2024-04-25T11:34:06.224578+0000","last_became_peered":"2024-04-25T11:34:06.224578+0000","last_unstale":"2024-04-25T11:34:07.482838+0000","last_undegraded":"2024-04-25T11:34:07.482838+0000","last_fullsized":"2024-04-25T11:34:07.482838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.279224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482760+0000","last_change":"2024-04-25T11:34:05.235110+0000","last_active":"2024-04-25T11:34:07.482760+0000","last_peered":"2024-04-25T11:34:07.482760+0000","last_clean":"2024-04-25T11:34:07.482760+0000","last_became_active":"2024-04-25T11:34:05.233394+0000","last_became_peered":"2024-04-25T11:34:05.233394+0000","last_unstale":"2024-04-25T11:34:07.482760+0000","last_undegraded":"2024-04-25T11:34:07.482760+0000","last_fullsized":"2024-04-25T11:34:07.482760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:28.352911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247766+0000","last_change":"2024-04-25T11:34:06.246976+0000","last_active":"2024-04-25T11:34:07.247766+0000","last_peered":"2024-04-25T11:34:07.247766+0000","last_clean":"2024-04-25T11:34:07.247766+0000","last_became_active":"2024-04-25T11:34:06.245224+0000","last_became_peered":"2024-04-25T11:34:06.245224+0000","last_unstale":"2024-04-25T11:34:07.247766+0000","last_undegraded":"2024-04-25T11:34:07.247766+0000","last_fullsized":"2024-04-25T11:34:07.247766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:44.452645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247568+0000","last_change":"2024-04-25T11:34:05.241365+0000","last_active":"2024-04-25T11:34:07.247568+0000","last_peered":"2024-04-25T11:34:07.247568+0000","last_clean":"2024-04-25T11:34:07.247568+0000","last_became_active":"2024-04-25T11:34:05.240957+0000","last_became_peered":"2024-04-25T11:34:05.240957+0000","last_unstale":"2024-04-25T11:34:07.247568+0000","last_undegraded":"2024-04-25T11:34:07.247568+0000","last_fullsized":"2024-04-25T11:34:07.247568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.664736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247212+0000","last_change":"2024-04-25T11:34:06.250293+0000","last_active":"2024-04-25T11:34:07.247212+0000","last_peered":"2024-04-25T11:34:07.247212+0000","last_clean":"2024-04-25T11:34:07.247212+0000","last_became_active":"2024-04-25T11:34:06.250010+0000","last_became_peered":"2024-04-25T11:34:06.250010+0000","last_unstale":"2024-04-25T11:34:07.247212+0000","last_undegraded":"2024-04-25T11:34:07.247212+0000","last_fullsized":"2024-04-25T11:34:07.247212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:38.681839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239438+0000","last_change":"2024-04-25T11:34:05.233427+0000","last_active":"2024-04-25T11:34:07.239438+0000","last_peered":"2024-04-25T11:34:07.239438+0000","last_clean":"2024-04-25T11:34:07.239438+0000","last_became_active":"2024-04-25T11:34:05.232747+0000","last_became_peered":"2024-04-25T11:34:05.232747+0000","last_unstale":"2024-04-25T11:34:07.239438+0000","last_undegraded":"2024-04-25T11:34:07.239438+0000","last_fullsized":"2024-04-25T11:34:07.239438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:16.602789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482229+0000","last_change":"2024-04-25T11:34:06.252767+0000","last_active":"2024-04-25T11:34:07.482229+0000","last_peered":"2024-04-25T11:34:07.482229+0000","last_clean":"2024-04-25T11:34:07.482229+0000","last_became_active":"2024-04-25T11:34:06.252124+0000","last_became_peered":"2024-04-25T11:34:06.252124+0000","last_unstale":"2024-04-25T11:34:07.482229+0000","last_undegraded":"2024-04-25T11:34:07.482229+0000","last_fullsized":"2024-04-25T11:34:07.482229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:07.353595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481986+0000","last_change":"2024-04-25T11:34:05.239323+0000","last_active":"2024-04-25T11:34:07.481986+0000","last_peered":"2024-04-25T11:34:07.481986+0000","last_clean":"2024-04-25T11:34:07.481986+0000","last_became_active":"2024-04-25T11:34:05.238821+0000","last_became_peered":"2024-04-25T11:34:05.238821+0000","last_unstale":"2024-04-25T11:34:07.481986+0000","last_undegraded":"2024-04-25T11:34:07.481986+0000","last_fullsized":"2024-04-25T11:34:07.481986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:47.194519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240000+0000","last_change":"2024-04-25T11:34:06.243408+0000","last_active":"2024-04-25T11:34:07.240000+0000","last_peered":"2024-04-25T11:34:07.240000+0000","last_clean":"2024-04-25T11:34:07.240000+0000","last_became_active":"2024-04-25T11:34:06.242107+0000","last_became_peered":"2024-04-25T11:34:06.242107+0000","last_unstale":"2024-04-25T11:34:07.240000+0000","last_undegraded":"2024-04-25T11:34:07.240000+0000","last_fullsized":"2024-04-25T11:34:07.240000+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:58.032333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248937+0000","last_change":"2024-04-25T11:34:05.245078+0000","last_active":"2024-04-25T11:34:07.248937+0000","last_peered":"2024-04-25T11:34:07.248937+0000","last_clean":"2024-04-25T11:34:07.248937+0000","last_became_active":"2024-04-25T11:34:05.244660+0000","last_became_peered":"2024-04-25T11:34:05.244660+0000","last_unstale":"2024-04-25T11:34:07.248937+0000","last_undegraded":"2024-04-25T11:34:07.248937+0000","last_fullsized":"2024-04-25T11:34:07.248937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:08.627975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.241897+0000","last_change":"2024-04-25T11:34:06.234301+0000","last_active":"2024-04-25T11:34:06.241897+0000","last_peered":"2024-04-25T11:34:06.241897+0000","last_clean":"2024-04-25T11:34:06.241897+0000","last_became_active":"2024-04-25T11:34:06.233914+0000","last_became_peered":"2024-04-25T11:34:06.233914+0000","last_unstale":"2024-04-25T11:34:06.241897+0000","last_undegraded":"2024-04-25T11:34:06.241897+0000","last_fullsized":"2024-04-25T11:34:06.241897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:34.242170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483810+0000","last_change":"2024-04-25T11:34:05.234357+0000","last_active":"2024-04-25T11:34:07.483810+0000","last_peered":"2024-04-25T11:34:07.483810+0000","last_clean":"2024-04-25T11:34:07.483810+0000","last_became_active":"2024-04-25T11:34:05.233927+0000","last_became_peered":"2024-04-25T11:34:05.233927+0000","last_unstale":"2024-04-25T11:34:07.483810+0000","last_undegraded":"2024-04-25T11:34:07.483810+0000","last_fullsized":"2024-04-25T11:34:07.483810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:47.452434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483698+0000","last_change":"2024-04-25T11:34:06.244448+0000","last_active":"2024-04-25T11:34:07.483698+0000","last_peered":"2024-04-25T11:34:07.483698+0000","last_clean":"2024-04-25T11:34:07.483698+0000","last_became_active":"2024-04-25T11:34:06.244099+0000","last_became_peered":"2024-04-25T11:34:06.244099+0000","last_unstale":"2024-04-25T11:34:07.483698+0000","last_undegraded":"2024-04-25T11:34:07.483698+0000","last_fullsized":"2024-04-25T11:34:07.483698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:21.764096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483798+0000","last_change":"2024-04-25T11:34:05.228788+0000","last_active":"2024-04-25T11:34:07.483798+0000","last_peered":"2024-04-25T11:34:07.483798+0000","last_clean":"2024-04-25T11:34:07.483798+0000","last_became_active":"2024-04-25T11:34:05.228353+0000","last_became_peered":"2024-04-25T11:34:05.228353+0000","last_unstale":"2024-04-25T11:34:07.483798+0000","last_undegraded":"2024-04-25T11:34:07.483798+0000","last_fullsized":"2024-04-25T11:34:07.483798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:47.194519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246802+0000","last_change":"2024-04-25T11:34:06.249325+0000","last_active":"2024-04-25T11:34:07.246802+0000","last_peered":"2024-04-25T11:34:07.246802+0000","last_clean":"2024-04-25T11:34:07.246802+0000","last_became_active":"2024-04-25T11:34:06.248971+0000","last_became_peered":"2024-04-25T11:34:06.248971+0000","last_unstale":"2024-04-25T11:34:07.246802+0000","last_undegraded":"2024-04-25T11:34:07.246802+0000","last_fullsized":"2024-04-25T11:34:07.246802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.689445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246398+0000","last_change":"2024-04-25T11:34:05.243888+0000","last_active":"2024-04-25T11:34:07.246398+0000","last_peered":"2024-04-25T11:34:07.246398+0000","last_clean":"2024-04-25T11:34:07.246398+0000","last_became_active":"2024-04-25T11:34:05.243528+0000","last_became_peered":"2024-04-25T11:34:05.243528+0000","last_unstale":"2024-04-25T11:34:07.246398+0000","last_undegraded":"2024-04-25T11:34:07.246398+0000","last_fullsized":"2024-04-25T11:34:07.246398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:20.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.247510+0000","last_change":"2024-04-25T11:34:06.243025+0000","last_active":"2024-04-25T11:34:06.247510+0000","last_peered":"2024-04-25T11:34:06.247510+0000","last_clean":"2024-04-25T11:34:06.247510+0000","last_became_active":"2024-04-25T11:34:06.242346+0000","last_became_peered":"2024-04-25T11:34:06.242346+0000","last_unstale":"2024-04-25T11:34:06.247510+0000","last_undegraded":"2024-04-25T11:34:06.247510+0000","last_fullsized":"2024-04-25T11:34:06.247510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:05.767705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246395+0000","last_change":"2024-04-25T11:34:05.241007+0000","last_active":"2024-04-25T11:34:07.246395+0000","last_peered":"2024-04-25T11:34:07.246395+0000","last_clean":"2024-04-25T11:34:07.246395+0000","last_became_active":"2024-04-25T11:34:05.240726+0000","last_became_peered":"2024-04-25T11:34:05.240726+0000","last_unstale":"2024-04-25T11:34:07.246395+0000","last_undegraded":"2024-04-25T11:34:07.246395+0000","last_fullsized":"2024-04-25T11:34:07.246395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:35.216257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244141+0000","last_change":"2024-04-25T11:34:06.243138+0000","last_active":"2024-04-25T11:34:07.244141+0000","last_peered":"2024-04-25T11:34:07.244141+0000","last_clean":"2024-04-25T11:34:07.244141+0000","last_became_active":"2024-04-25T11:34:06.242767+0000","last_became_peered":"2024-04-25T11:34:06.242767+0000","last_unstale":"2024-04-25T11:34:07.244141+0000","last_undegraded":"2024-04-25T11:34:07.244141+0000","last_fullsized":"2024-04-25T11:34:07.244141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:56.956581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246100+0000","last_change":"2024-04-25T11:34:05.232409+0000","last_active":"2024-04-25T11:34:07.246100+0000","last_peered":"2024-04-25T11:34:07.246100+0000","last_clean":"2024-04-25T11:34:07.246100+0000","last_became_active":"2024-04-25T11:34:05.232086+0000","last_became_peered":"2024-04-25T11:34:05.232086+0000","last_unstale":"2024-04-25T11:34:07.246100+0000","last_undegraded":"2024-04-25T11:34:07.246100+0000","last_fullsized":"2024-04-25T11:34:07.246100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:27.740735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481994+0000","last_change":"2024-04-25T11:34:06.245145+0000","last_active":"2024-04-25T11:34:07.481994+0000","last_peered":"2024-04-25T11:34:07.481994+0000","last_clean":"2024-04-25T11:34:07.481994+0000","last_became_active":"2024-04-25T11:34:06.244130+0000","last_became_peered":"2024-04-25T11:34:06.244130+0000","last_unstale":"2024-04-25T11:34:07.481994+0000","last_undegraded":"2024-04-25T11:34:07.481994+0000","last_fullsized":"2024-04-25T11:34:07.481994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:43.166051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239619+0000","last_change":"2024-04-25T11:34:05.239168+0000","last_active":"2024-04-25T11:34:07.239619+0000","last_peered":"2024-04-25T11:34:07.239619+0000","last_clean":"2024-04-25T11:34:07.239619+0000","last_became_active":"2024-04-25T11:34:05.238775+0000","last_became_peered":"2024-04-25T11:34:05.238775+0000","last_unstale":"2024-04-25T11:34:07.239619+0000","last_undegraded":"2024-04-25T11:34:07.239619+0000","last_fullsized":"2024-04-25T11:34:07.239619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:07.510487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482715+0000","last_change":"2024-04-25T11:34:06.250245+0000","last_active":"2024-04-25T11:34:07.482715+0000","last_peered":"2024-04-25T11:34:07.482715+0000","last_clean":"2024-04-25T11:34:07.482715+0000","last_became_active":"2024-04-25T11:34:06.247561+0000","last_became_peered":"2024-04-25T11:34:06.247561+0000","last_unstale":"2024-04-25T11:34:07.482715+0000","last_undegraded":"2024-04-25T11:34:07.482715+0000","last_fullsized":"2024-04-25T11:34:07.482715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:14.216115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482132+0000","last_change":"2024-04-25T11:34:05.224234+0000","last_active":"2024-04-25T11:34:07.482132+0000","last_peered":"2024-04-25T11:34:07.482132+0000","last_clean":"2024-04-25T11:34:07.482132+0000","last_became_active":"2024-04-25T11:34:05.221602+0000","last_became_peered":"2024-04-25T11:34:05.221602+0000","last_unstale":"2024-04-25T11:34:07.482132+0000","last_undegraded":"2024-04-25T11:34:07.482132+0000","last_fullsized":"2024-04-25T11:34:07.482132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:44.994317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245343+0000","last_change":"2024-04-25T11:34:06.247056+0000","last_active":"2024-04-25T11:34:07.245343+0000","last_peered":"2024-04-25T11:34:07.245343+0000","last_clean":"2024-04-25T11:34:07.245343+0000","last_became_active":"2024-04-25T11:34:06.246027+0000","last_became_peered":"2024-04-25T11:34:06.246027+0000","last_unstale":"2024-04-25T11:34:07.245343+0000","last_undegraded":"2024-04-25T11:34:07.245343+0000","last_fullsized":"2024-04-25T11:34:07.245343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:28.956060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481741+0000","last_change":"2024-04-25T11:34:05.227013+0000","last_active":"2024-04-25T11:34:07.481741+0000","last_peered":"2024-04-25T11:34:07.481741+0000","last_clean":"2024-04-25T11:34:07.481741+0000","last_became_active":"2024-04-25T11:34:05.226683+0000","last_became_peered":"2024-04-25T11:34:05.226683+0000","last_unstale":"2024-04-25T11:34:07.481741+0000","last_undegraded":"2024-04-25T11:34:07.481741+0000","last_fullsized":"2024-04-25T11:34:07.481741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:47.452434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245540+0000","last_change":"2024-04-25T11:34:06.252333+0000","last_active":"2024-04-25T11:34:07.245540+0000","last_peered":"2024-04-25T11:34:07.245540+0000","last_clean":"2024-04-25T11:34:07.245540+0000","last_became_active":"2024-04-25T11:34:06.251724+0000","last_became_peered":"2024-04-25T11:34:06.251724+0000","last_unstale":"2024-04-25T11:34:07.245540+0000","last_undegraded":"2024-04-25T11:34:07.245540+0000","last_fullsized":"2024-04-25T11:34:07.245540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:29.370014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483948+0000","last_change":"2024-04-25T11:34:05.238080+0000","last_active":"2024-04-25T11:34:07.483948+0000","last_peered":"2024-04-25T11:34:07.483948+0000","last_clean":"2024-04-25T11:34:07.483948+0000","last_became_active":"2024-04-25T11:34:05.237600+0000","last_became_peered":"2024-04-25T11:34:05.237600+0000","last_unstale":"2024-04-25T11:34:07.483948+0000","last_undegraded":"2024-04-25T11:34:07.483948+0000","last_fullsized":"2024-04-25T11:34:07.483948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:42.761950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.484121+0000","last_change":"2024-04-25T11:34:06.241010+0000","last_active":"2024-04-25T11:34:07.484121+0000","last_peered":"2024-04-25T11:34:07.484121+0000","last_clean":"2024-04-25T11:34:07.484121+0000","last_became_active":"2024-04-25T11:34:06.240653+0000","last_became_peered":"2024-04-25T11:34:06.240653+0000","last_unstale":"2024-04-25T11:34:07.484121+0000","last_undegraded":"2024-04-25T11:34:07.484121+0000","last_fullsized":"2024-04-25T11:34:07.484121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:14.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483971+0000","last_change":"2024-04-25T11:34:05.238126+0000","last_active":"2024-04-25T11:34:07.483971+0000","last_peered":"2024-04-25T11:34:07.483971+0000","last_clean":"2024-04-25T11:34:07.483971+0000","last_became_active":"2024-04-25T11:34:05.236401+0000","last_became_peered":"2024-04-25T11:34:05.236401+0000","last_unstale":"2024-04-25T11:34:07.483971+0000","last_undegraded":"2024-04-25T11:34:07.483971+0000","last_fullsized":"2024-04-25T11:34:07.483971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:02.254171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.484386+0000","last_change":"2024-04-25T11:34:06.240985+0000","last_active":"2024-04-25T11:34:07.484386+0000","last_peered":"2024-04-25T11:34:07.484386+0000","last_clean":"2024-04-25T11:34:07.484386+0000","last_became_active":"2024-04-25T11:34:06.240558+0000","last_became_peered":"2024-04-25T11:34:06.240558+0000","last_unstale":"2024-04-25T11:34:07.484386+0000","last_undegraded":"2024-04-25T11:34:07.484386+0000","last_fullsized":"2024-04-25T11:34:07.484386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:50.529668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481091+0000","last_change":"2024-04-25T11:34:05.241758+0000","last_active":"2024-04-25T11:34:07.481091+0000","last_peered":"2024-04-25T11:34:07.481091+0000","last_clean":"2024-04-25T11:34:07.481091+0000","last_became_active":"2024-04-25T11:34:05.241393+0000","last_became_peered":"2024-04-25T11:34:05.241393+0000","last_unstale":"2024-04-25T11:34:07.481091+0000","last_undegraded":"2024-04-25T11:34:07.481091+0000","last_fullsized":"2024-04-25T11:34:07.481091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:41.152497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248288+0000","last_change":"2024-04-25T11:34:06.243257+0000","last_active":"2024-04-25T11:34:07.248288+0000","last_peered":"2024-04-25T11:34:07.248288+0000","last_clean":"2024-04-25T11:34:07.248288+0000","last_became_active":"2024-04-25T11:34:06.239661+0000","last_became_peered":"2024-04-25T11:34:06.239661+0000","last_unstale":"2024-04-25T11:34:07.248288+0000","last_undegraded":"2024-04-25T11:34:07.248288+0000","last_fullsized":"2024-04-25T11:34:07.248288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:14.216115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.214784+0000","last_change":"2024-04-25T11:34:05.241495+0000","last_active":"2024-04-25T11:34:06.214784+0000","last_peered":"2024-04-25T11:34:06.214784+0000","last_clean":"2024-04-25T11:34:06.214784+0000","last_became_active":"2024-04-25T11:34:05.241121+0000","last_became_peered":"2024-04-25T11:34:05.241121+0000","last_unstale":"2024-04-25T11:34:06.214784+0000","last_undegraded":"2024-04-25T11:34:06.214784+0000","last_fullsized":"2024-04-25T11:34:06.214784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:54.245245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.241626+0000","last_change":"2024-04-25T11:34:06.235254+0000","last_active":"2024-04-25T11:34:06.241626+0000","last_peered":"2024-04-25T11:34:06.241626+0000","last_clean":"2024-04-25T11:34:06.241626+0000","last_became_active":"2024-04-25T11:34:06.233838+0000","last_became_peered":"2024-04-25T11:34:06.233838+0000","last_unstale":"2024-04-25T11:34:06.241626+0000","last_undegraded":"2024-04-25T11:34:06.241626+0000","last_fullsized":"2024-04-25T11:34:06.241626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:10.838501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240011+0000","last_change":"2024-04-25T11:34:05.239594+0000","last_active":"2024-04-25T11:34:07.240011+0000","last_peered":"2024-04-25T11:34:07.240011+0000","last_clean":"2024-04-25T11:34:07.240011+0000","last_became_active":"2024-04-25T11:34:05.239093+0000","last_became_peered":"2024-04-25T11:34:05.239093+0000","last_unstale":"2024-04-25T11:34:07.240011+0000","last_undegraded":"2024-04-25T11:34:07.240011+0000","last_fullsized":"2024-04-25T11:34:07.240011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:56.643526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247191+0000","last_change":"2024-04-25T11:34:06.233895+0000","last_active":"2024-04-25T11:34:07.247191+0000","last_peered":"2024-04-25T11:34:07.247191+0000","last_clean":"2024-04-25T11:34:07.247191+0000","last_became_active":"2024-04-25T11:34:06.232903+0000","last_became_peered":"2024-04-25T11:34:06.232903+0000","last_unstale":"2024-04-25T11:34:07.247191+0000","last_undegraded":"2024-04-25T11:34:07.247191+0000","last_fullsized":"2024-04-25T11:34:07.247191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:32.917368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.243454+0000","last_change":"2024-04-25T11:34:05.227704+0000","last_active":"2024-04-25T11:34:07.243454+0000","last_peered":"2024-04-25T11:34:07.243454+0000","last_clean":"2024-04-25T11:34:07.243454+0000","last_became_active":"2024-04-25T11:34:05.226528+0000","last_became_peered":"2024-04-25T11:34:05.226528+0000","last_unstale":"2024-04-25T11:34:07.243454+0000","last_undegraded":"2024-04-25T11:34:07.243454+0000","last_fullsized":"2024-04-25T11:34:07.243454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:17.374177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481867+0000","last_change":"2024-04-25T11:34:06.257328+0000","last_active":"2024-04-25T11:34:07.481867+0000","last_peered":"2024-04-25T11:34:07.481867+0000","last_clean":"2024-04-25T11:34:07.481867+0000","last_became_active":"2024-04-25T11:34:06.255389+0000","last_became_peered":"2024-04-25T11:34:06.255389+0000","last_unstale":"2024-04-25T11:34:07.481867+0000","last_undegraded":"2024-04-25T11:34:07.481867+0000","last_fullsized":"2024-04-25T11:34:07.481867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:27.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.249042+0000","last_change":"2024-04-25T11:34:05.242223+0000","last_active":"2024-04-25T11:34:07.249042+0000","last_peered":"2024-04-25T11:34:07.249042+0000","last_clean":"2024-04-25T11:34:07.249042+0000","last_became_active":"2024-04-25T11:34:05.241762+0000","last_became_peered":"2024-04-25T11:34:05.241762+0000","last_unstale":"2024-04-25T11:34:07.249042+0000","last_undegraded":"2024-04-25T11:34:07.249042+0000","last_fullsized":"2024-04-25T11:34:07.249042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:28.525172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246890+0000","last_change":"2024-04-25T11:34:06.242461+0000","last_active":"2024-04-25T11:34:07.246890+0000","last_peered":"2024-04-25T11:34:07.246890+0000","last_clean":"2024-04-25T11:34:07.246890+0000","last_became_active":"2024-04-25T11:34:06.241544+0000","last_became_peered":"2024-04-25T11:34:06.241544+0000","last_unstale":"2024-04-25T11:34:07.246890+0000","last_undegraded":"2024-04-25T11:34:07.246890+0000","last_fullsized":"2024-04-25T11:34:07.246890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:14.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248921+0000","last_change":"2024-04-25T11:34:05.244709+0000","last_active":"2024-04-25T11:34:07.248921+0000","last_peered":"2024-04-25T11:34:07.248921+0000","last_clean":"2024-04-25T11:34:07.248921+0000","last_became_active":"2024-04-25T11:34:05.244381+0000","last_became_peered":"2024-04-25T11:34:05.244381+0000","last_unstale":"2024-04-25T11:34:07.248921+0000","last_undegraded":"2024-04-25T11:34:07.248921+0000","last_fullsized":"2024-04-25T11:34:07.248921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:02.755508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.239584+0000","last_change":"2024-04-25T11:34:06.234951+0000","last_active":"2024-04-25T11:34:06.239584+0000","last_peered":"2024-04-25T11:34:06.239584+0000","last_clean":"2024-04-25T11:34:06.239584+0000","last_became_active":"2024-04-25T11:34:06.234553+0000","last_became_peered":"2024-04-25T11:34:06.234553+0000","last_unstale":"2024-04-25T11:34:06.239584+0000","last_undegraded":"2024-04-25T11:34:06.239584+0000","last_fullsized":"2024-04-25T11:34:06.239584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:05.022523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248558+0000","last_change":"2024-04-25T11:34:05.238934+0000","last_active":"2024-04-25T11:34:07.248558+0000","last_peered":"2024-04-25T11:34:07.248558+0000","last_clean":"2024-04-25T11:34:07.248558+0000","last_became_active":"2024-04-25T11:34:05.238531+0000","last_became_peered":"2024-04-25T11:34:05.238531+0000","last_unstale":"2024-04-25T11:34:07.248558+0000","last_undegraded":"2024-04-25T11:34:07.248558+0000","last_fullsized":"2024-04-25T11:34:07.248558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:31.942620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239549+0000","last_change":"2024-04-25T11:34:06.258078+0000","last_active":"2024-04-25T11:34:07.239549+0000","last_peered":"2024-04-25T11:34:07.239549+0000","last_clean":"2024-04-25T11:34:07.239549+0000","last_became_active":"2024-04-25T11:34:06.256780+0000","last_became_peered":"2024-04-25T11:34:06.256780+0000","last_unstale":"2024-04-25T11:34:07.239549+0000","last_undegraded":"2024-04-25T11:34:07.239549+0000","last_fullsized":"2024-04-25T11:34:07.239549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:35.627673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.218994+0000","last_change":"2024-04-25T11:34:05.245054+0000","last_active":"2024-04-25T11:34:06.218994+0000","last_peered":"2024-04-25T11:34:06.218994+0000","last_clean":"2024-04-25T11:34:06.218994+0000","last_became_active":"2024-04-25T11:34:05.244581+0000","last_became_peered":"2024-04-25T11:34:05.244581+0000","last_unstale":"2024-04-25T11:34:06.218994+0000","last_undegraded":"2024-04-25T11:34:06.218994+0000","last_fullsized":"2024-04-25T11:34:06.218994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:23.697530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483363+0000","last_change":"2024-04-25T11:34:06.244804+0000","last_active":"2024-04-25T11:34:07.483363+0000","last_peered":"2024-04-25T11:34:07.483363+0000","last_clean":"2024-04-25T11:34:07.483363+0000","last_became_active":"2024-04-25T11:34:06.244431+0000","last_became_peered":"2024-04-25T11:34:06.244431+0000","last_unstale":"2024-04-25T11:34:07.483363+0000","last_undegraded":"2024-04-25T11:34:07.483363+0000","last_fullsized":"2024-04-25T11:34:07.483363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.689445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481238+0000","last_change":"2024-04-25T11:34:05.232793+0000","last_active":"2024-04-25T11:34:07.481238+0000","last_peered":"2024-04-25T11:34:07.481238+0000","last_clean":"2024-04-25T11:34:07.481238+0000","last_became_active":"2024-04-25T11:34:05.232445+0000","last_became_peered":"2024-04-25T11:34:05.232445+0000","last_unstale":"2024-04-25T11:34:07.481238+0000","last_undegraded":"2024-04-25T11:34:07.481238+0000","last_fullsized":"2024-04-25T11:34:07.481238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:28.741533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.480926+0000","last_change":"2024-04-25T11:34:06.253004+0000","last_active":"2024-04-25T11:34:07.480926+0000","last_peered":"2024-04-25T11:34:07.480926+0000","last_clean":"2024-04-25T11:34:07.480926+0000","last_became_active":"2024-04-25T11:34:06.252687+0000","last_became_peered":"2024-04-25T11:34:06.252687+0000","last_unstale":"2024-04-25T11:34:07.480926+0000","last_undegraded":"2024-04-25T11:34:07.480926+0000","last_fullsized":"2024-04-25T11:34:07.480926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:32.959723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482999+0000","last_change":"2024-04-25T11:34:05.240378+0000","last_active":"2024-04-25T11:34:07.482999+0000","last_peered":"2024-04-25T11:34:07.482999+0000","last_clean":"2024-04-25T11:34:07.482999+0000","last_became_active":"2024-04-25T11:34:05.240032+0000","last_became_peered":"2024-04-25T11:34:05.240032+0000","last_unstale":"2024-04-25T11:34:07.482999+0000","last_undegraded":"2024-04-25T11:34:07.482999+0000","last_fullsized":"2024-04-25T11:34:07.482999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:31.942620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.255997+0000","last_change":"2024-04-25T11:34:06.252992+0000","last_active":"2024-04-25T11:34:06.255997+0000","last_peered":"2024-04-25T11:34:06.255997+0000","last_clean":"2024-04-25T11:34:06.255997+0000","last_became_active":"2024-04-25T11:34:06.252579+0000","last_became_peered":"2024-04-25T11:34:06.252579+0000","last_unstale":"2024-04-25T11:34:06.255997+0000","last_undegraded":"2024-04-25T11:34:06.255997+0000","last_fullsized":"2024-04-25T11:34:06.255997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:56.315581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246679+0000","last_change":"2024-04-25T11:34:05.236265+0000","last_active":"2024-04-25T11:34:07.246679+0000","last_peered":"2024-04-25T11:34:07.246679+0000","last_clean":"2024-04-25T11:34:07.246679+0000","last_became_active":"2024-04-25T11:34:05.235615+0000","last_became_peered":"2024-04-25T11:34:05.235615+0000","last_unstale":"2024-04-25T11:34:07.246679+0000","last_undegraded":"2024-04-25T11:34:07.246679+0000","last_fullsized":"2024-04-25T11:34:07.246679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:55.939478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":221,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246192+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:34:07.246192+0000","last_peered":"2024-04-25T11:34:07.246192+0000","last_clean":"2024-04-25T11:34:07.246192+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:34:07.246192+0000","last_undegraded":"2024-04-25T11:34:07.246192+0000","last_fullsized":"2024-04-25T11:34:07.246192+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":410883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":404617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":954368,"data_stored":403665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":954368,"data_stored":403097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":946176,"data_stored":403553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509527,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":999639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":946176,"data_stored":403041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542229,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":986564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:34:10.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:34:11.158 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:34:11.170+0000 7f71cda00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:34:11.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.311+0000 7f2b45000640 1 Processor -- start 2024-04-25T11:34:11.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.311+0000 7f2b45000640 1 -- start start 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.313+0000 7f2b45000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2b401443c0 0x7f2b4006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b45000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 0x7f2b400cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b45000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b400cff30 con 0x7f2b400cfb20 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b45000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2b400708d0 con 0x7f2b401443c0 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b45000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4015f160 con 0x7f2b40070370 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b3e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2b400cfb20 0x7f2b400ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51532/0 (socket says 172.21.15.19:51532) 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b3e800640 1 -- 172.21.15.19:0/4068642141 learned_addr learned my addr 172.21.15.19:0/4068642141 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b3de00640 1 --2- 172.21.15.19:0/4068642141 >> v2:172.21.15.169:3300/0 conn(0x7f2b401443c0 0x7f2b4006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.314+0000 7f2b35400640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 0x7f2b400cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028778537 0 0) 0x7f2b38001490 con 0x7f2b400cfb20 2024-04-25T11:34:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b24003520 con 0x7f2b400cfb20 2024-04-25T11:34:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1652722047 0 0) 0x7f2b24003520 con 0x7f2b400cfb20 2024-04-25T11:34:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 msgr2=0x7f2b400cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 0x7f2b400cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 >> v2:172.21.15.169:3300/0 conn(0x7f2b401443c0 msgr2=0x7f2b4006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 --2- 172.21.15.19:0/4068642141 >> v2:172.21.15.169:3300/0 conn(0x7f2b401443c0 0x7f2b4006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4015f5f0 con 0x7f2b400cfb20 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2b38001bc0 con 0x7f2b400cfb20 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 >> v1:172.21.15.19:6789/0 conn(0x7f2b400cfb20 legacy=0x7f2b400ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 0x7f2b240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b24003f30 0x7f2b24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b24005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b400708d0 con 0x7f2b24004d40 2024-04-25T11:34:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4015f160 con 0x7f2b24003f30 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.315+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2400ba10 con 0x7f2b40070370 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b3e800640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b24005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b35400640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b24003f30 0x7f2b24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b3de00640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 0x7f2b240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b3e800640 1 -- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 msgr2=0x7f2b240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b3e800640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b40070370 0x7f2b240039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b3e800640 1 -- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b24003f30 msgr2=0x7f2b24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b3e800640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b24003f30 0x7f2b24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.316+0000 7f2b3e800640 1 -- 172.21.15.19:0/4068642141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b24003520 con 0x7f2b24004d40 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.317+0000 7f2b3e800640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b24005150 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800aa40 tx=0x7f2b24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e65bab317f295b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.317+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800a690 con 0x7f2b24004d40 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.317+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3802f070 con 0x7f2b24004d40 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.317+0000 7f2b3d400640 1 -- 172.21.15.19:0/4068642141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38034470 con 0x7f2b24004d40 2024-04-25T11:34:11.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.318+0000 7f2b45000640 1 -- 172.21.15.19:0/4068642141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 msgr2=0x7f2b24005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.318+0000 7f2b45000640 1 --2- 172.21.15.19:0/4068642141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b24005150 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800aa40 tx=0x7f2b24005830 comp rx=0 tx=0).stop 2024-04-25T11:34:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.318+0000 7f2b3e800640 1 -- 172.21.15.19:0/4068642141 reap_dead start 2024-04-25T11:34:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.318+0000 7f2b45000640 1 -- 172.21.15.19:0/4068642141 shutdown_connections 2024-04-25T11:34:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.318+0000 7f2b45000640 1 -- 172.21.15.19:0/4068642141 >> 172.21.15.19:0/4068642141 conn(0x7f2b400d1ae0 msgr2=0x7f2b400d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.318+0000 7f2b45000640 1 -- 172.21.15.19:0/4068642141 shutdown_connections 2024-04-25T11:34:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.318+0000 7f2b45000640 1 -- 172.21.15.19:0/4068642141 wait complete. 2024-04-25T11:34:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.321+0000 7f2b45000640 1 Processor -- start 2024-04-25T11:34:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.321+0000 7f2b45000640 1 -- start start 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.322+0000 7f2b45000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b400cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.322+0000 7f2b45000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4015b2b0 0x7f2b400b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b45000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b400b5d30 0x7f2b4013a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b45000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4015edf0 con 0x7f2b24004d40 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b45000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4015eb10 con 0x7f2b4015b2b0 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b45000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4006d5d0 con 0x7f2b400b5d30 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b400cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b400cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43346/0 (socket says 172.21.15.19:43346) 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 -- 172.21.15.19:0/1245065589 learned_addr learned my addr 172.21.15.19:0/1245065589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b35400640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4015b2b0 0x7f2b400b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 -- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b400b5d30 msgr2=0x7f2b4013a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b400b5d30 0x7f2b4013a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 -- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4015b2b0 msgr2=0x7f2b400b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4015b2b0 0x7f2b400b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.323+0000 7f2b3de00640 1 -- 172.21.15.19:0/1245065589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b400b6140 con 0x7f2b24004d40 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.324+0000 7f2b3de00640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b400cf380 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2b30010900 tx=0x7f2b30010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.324+0000 7f2b36a00640 1 -- 172.21.15.19:0/1245065589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000f040 con 0x7f2b24004d40 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.324+0000 7f2b36a00640 1 -- 172.21.15.19:0/1245065589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3000ea80 con 0x7f2b24004d40 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.324+0000 7f2b36a00640 1 -- 172.21.15.19:0/1245065589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000ee50 con 0x7f2b24004d40 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.324+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b40140730 con 0x7f2b24004d40 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.324+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b40140cf0 con 0x7f2b24004d40 2024-04-25T11:34:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.325+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b40050740 con 0x7f2b24004d40 2024-04-25T11:34:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.327+0000 7f2b36a00640 1 -- 172.21.15.19:0/1245065589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b3001c400 con 0x7f2b24004d40 2024-04-25T11:34:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.328+0000 7f2b36a00640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b1c0744e0 0x7f2b1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.328+0000 7f2b36a00640 1 -- 172.21.15.19:0/1245065589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f2b30095880 con 0x7f2b24004d40 2024-04-25T11:34:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.329+0000 7f2b35400640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b1c0744e0 0x7f2b1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.330+0000 7f2b35400640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b1c0744e0 0x7f2b1c0769f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cef50 tx=0x7f2b28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.332+0000 7f2b36a00640 1 -- 172.21.15.19:0/1245065589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b400b6140 con 0x7f2b24004d40 2024-04-25T11:34:11.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.536+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b400cf8c0 con 0x7f2b1c0744e0 2024-04-25T11:34:11.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.557+0000 7f2b36a00640 1 -- 172.21.15.19:0/1245065589 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252049 (secure 0 0 0) 0x7f2b400cf8c0 con 0x7f2b1c0744e0 2024-04-25T11:34:11.549 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:11.550 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:34:11.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.563+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b1c0744e0 msgr2=0x7f2b1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.563+0000 7f2b45000640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b1c0744e0 0x7f2b1c0769f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2b400cef50 tx=0x7f2b28008040 comp rx=0 tx=0).stop 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 msgr2=0x7f2b400cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b400cf380 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2b30010900 tx=0x7f2b30010dc0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 shutdown_connections 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2b1c0744e0 0x7f2b1c0769f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2b4015b2b0 0x7f2b400b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2b400b5d30 0x7f2b4013a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 --2- 172.21.15.19:0/1245065589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b24004d40 0x7f2b400cf380 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 >> 172.21.15.19:0/1245065589 conn(0x7f2b400d1ae0 msgr2=0x7f2b40073050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 shutdown_connections 2024-04-25T11:34:11.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.564+0000 7f2b45000640 1 -- 172.21.15.19:0/1245065589 wait complete. 2024-04-25T11:34:11.579 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-04-25T11:34:10.610373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253880,"kb_used_data":8592,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720840,"statfs":{"total":773094113280,"available":772834140160,"internally_reserved":0,"allocated":8798208,"data_stored":4415059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":251055666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002399"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239887+0000","last_change":"2024-04-25T11:34:06.248049+0000","last_active":"2024-04-25T11:34:07.239887+0000","last_peered":"2024-04-25T11:34:07.239887+0000","last_clean":"2024-04-25T11:34:07.239887+0000","last_became_active":"2024-04-25T11:34:06.243041+0000","last_became_peered":"2024-04-25T11:34:06.243041+0000","last_unstale":"2024-04-25T11:34:07.239887+0000","last_undegraded":"2024-04-25T11:34:07.239887+0000","last_fullsized":"2024-04-25T11:34:07.239887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:33.036618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481795+0000","last_change":"2024-04-25T11:34:06.253171+0000","last_active":"2024-04-25T11:34:07.481795+0000","last_peered":"2024-04-25T11:34:07.481795+0000","last_clean":"2024-04-25T11:34:07.481795+0000","last_became_active":"2024-04-25T11:34:06.252442+0000","last_became_peered":"2024-04-25T11:34:06.252442+0000","last_unstale":"2024-04-25T11:34:07.481795+0000","last_undegraded":"2024-04-25T11:34:07.481795+0000","last_fullsized":"2024-04-25T11:34:07.481795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:43.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245574+0000","last_change":"2024-04-25T11:34:06.249591+0000","last_active":"2024-04-25T11:34:07.245574+0000","last_peered":"2024-04-25T11:34:07.245574+0000","last_clean":"2024-04-25T11:34:07.245574+0000","last_became_active":"2024-04-25T11:34:06.249210+0000","last_became_peered":"2024-04-25T11:34:06.249210+0000","last_unstale":"2024-04-25T11:34:07.245574+0000","last_undegraded":"2024-04-25T11:34:07.245574+0000","last_fullsized":"2024-04-25T11:34:07.245574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:48.469536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244324+0000","last_change":"2024-04-25T11:34:06.249607+0000","last_active":"2024-04-25T11:34:07.244324+0000","last_peered":"2024-04-25T11:34:07.244324+0000","last_clean":"2024-04-25T11:34:07.244324+0000","last_became_active":"2024-04-25T11:34:06.249210+0000","last_became_peered":"2024-04-25T11:34:06.249210+0000","last_unstale":"2024-04-25T11:34:07.244324+0000","last_undegraded":"2024-04-25T11:34:07.244324+0000","last_fullsized":"2024-04-25T11:34:07.244324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.416520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.259137+0000","last_change":"2024-04-25T11:34:06.255106+0000","last_active":"2024-04-25T11:34:06.259137+0000","last_peered":"2024-04-25T11:34:06.259137+0000","last_clean":"2024-04-25T11:34:06.259137+0000","last_became_active":"2024-04-25T11:34:06.254601+0000","last_became_peered":"2024-04-25T11:34:06.254601+0000","last_unstale":"2024-04-25T11:34:06.259137+0000","last_undegraded":"2024-04-25T11:34:06.259137+0000","last_fullsized":"2024-04-25T11:34:06.259137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:28.385384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483311+0000","last_change":"2024-04-25T11:34:06.258744+0000","last_active":"2024-04-25T11:34:07.483311+0000","last_peered":"2024-04-25T11:34:07.483311+0000","last_clean":"2024-04-25T11:34:07.483311+0000","last_became_active":"2024-04-25T11:34:06.258432+0000","last_became_peered":"2024-04-25T11:34:06.258432+0000","last_unstale":"2024-04-25T11:34:07.483311+0000","last_undegraded":"2024-04-25T11:34:07.483311+0000","last_fullsized":"2024-04-25T11:34:07.483311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:57.577377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246264+0000","last_change":"2024-04-25T11:34:06.224284+0000","last_active":"2024-04-25T11:34:07.246264+0000","last_peered":"2024-04-25T11:34:07.246264+0000","last_clean":"2024-04-25T11:34:07.246264+0000","last_became_active":"2024-04-25T11:34:06.223247+0000","last_became_peered":"2024-04-25T11:34:06.223247+0000","last_unstale":"2024-04-25T11:34:07.246264+0000","last_undegraded":"2024-04-25T11:34:07.246264+0000","last_fullsized":"2024-04-25T11:34:07.246264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.169600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239051+0000","last_change":"2024-04-25T11:34:06.246402+0000","last_active":"2024-04-25T11:34:07.239051+0000","last_peered":"2024-04-25T11:34:07.239051+0000","last_clean":"2024-04-25T11:34:07.239051+0000","last_became_active":"2024-04-25T11:34:06.245346+0000","last_became_peered":"2024-04-25T11:34:06.245346+0000","last_unstale":"2024-04-25T11:34:07.239051+0000","last_undegraded":"2024-04-25T11:34:07.239051+0000","last_fullsized":"2024-04-25T11:34:07.239051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:02.471215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245766+0000","last_change":"2024-04-25T11:34:06.245260+0000","last_active":"2024-04-25T11:34:07.245766+0000","last_peered":"2024-04-25T11:34:07.245766+0000","last_clean":"2024-04-25T11:34:07.245766+0000","last_became_active":"2024-04-25T11:34:06.244957+0000","last_became_peered":"2024-04-25T11:34:06.244957+0000","last_unstale":"2024-04-25T11:34:07.245766+0000","last_undegraded":"2024-04-25T11:34:07.245766+0000","last_fullsized":"2024-04-25T11:34:07.245766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:46.011420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247299+0000","last_change":"2024-04-25T11:34:06.228439+0000","last_active":"2024-04-25T11:34:07.247299+0000","last_peered":"2024-04-25T11:34:07.247299+0000","last_clean":"2024-04-25T11:34:07.247299+0000","last_became_active":"2024-04-25T11:34:06.227493+0000","last_became_peered":"2024-04-25T11:34:06.227493+0000","last_unstale":"2024-04-25T11:34:07.247299+0000","last_undegraded":"2024-04-25T11:34:07.247299+0000","last_fullsized":"2024-04-25T11:34:07.247299+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:03.772611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240070+0000","last_change":"2024-04-25T11:34:06.245966+0000","last_active":"2024-04-25T11:34:07.240070+0000","last_peered":"2024-04-25T11:34:07.240070+0000","last_clean":"2024-04-25T11:34:07.240070+0000","last_became_active":"2024-04-25T11:34:06.245433+0000","last_became_peered":"2024-04-25T11:34:06.245433+0000","last_unstale":"2024-04-25T11:34:07.240070+0000","last_undegraded":"2024-04-25T11:34:07.240070+0000","last_fullsized":"2024-04-25T11:34:07.240070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:37.989420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248585+0000","last_change":"2024-04-25T11:34:06.244291+0000","last_active":"2024-04-25T11:34:07.248585+0000","last_peered":"2024-04-25T11:34:07.248585+0000","last_clean":"2024-04-25T11:34:07.248585+0000","last_became_active":"2024-04-25T11:34:06.243951+0000","last_became_peered":"2024-04-25T11:34:06.243951+0000","last_unstale":"2024-04-25T11:34:07.248585+0000","last_undegraded":"2024-04-25T11:34:07.248585+0000","last_fullsized":"2024-04-25T11:34:07.248585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:01.685889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244822+0000","last_change":"2024-04-25T11:34:06.249832+0000","last_active":"2024-04-25T11:34:07.244822+0000","last_peered":"2024-04-25T11:34:07.244822+0000","last_clean":"2024-04-25T11:34:07.244822+0000","last_became_active":"2024-04-25T11:34:06.249222+0000","last_became_peered":"2024-04-25T11:34:06.249222+0000","last_unstale":"2024-04-25T11:34:07.244822+0000","last_undegraded":"2024-04-25T11:34:07.244822+0000","last_fullsized":"2024-04-25T11:34:07.244822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:00.980813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248311+0000","last_change":"2024-04-25T11:34:06.246909+0000","last_active":"2024-04-25T11:34:07.248311+0000","last_peered":"2024-04-25T11:34:07.248311+0000","last_clean":"2024-04-25T11:34:07.248311+0000","last_became_active":"2024-04-25T11:34:06.245458+0000","last_became_peered":"2024-04-25T11:34:06.245458+0000","last_unstale":"2024-04-25T11:34:07.248311+0000","last_undegraded":"2024-04-25T11:34:07.248311+0000","last_fullsized":"2024-04-25T11:34:07.248311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:59.126901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247758+0000","last_change":"2024-04-25T11:34:06.246646+0000","last_active":"2024-04-25T11:34:07.247758+0000","last_peered":"2024-04-25T11:34:07.247758+0000","last_clean":"2024-04-25T11:34:07.247758+0000","last_became_active":"2024-04-25T11:34:06.238961+0000","last_became_peered":"2024-04-25T11:34:06.238961+0000","last_unstale":"2024-04-25T11:34:07.247758+0000","last_undegraded":"2024-04-25T11:34:07.247758+0000","last_fullsized":"2024-04-25T11:34:07.247758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:07.353595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239272+0000","last_change":"2024-04-25T11:34:06.245697+0000","last_active":"2024-04-25T11:34:07.239272+0000","last_peered":"2024-04-25T11:34:07.239272+0000","last_clean":"2024-04-25T11:34:07.239272+0000","last_became_active":"2024-04-25T11:34:06.244897+0000","last_became_peered":"2024-04-25T11:34:06.244897+0000","last_unstale":"2024-04-25T11:34:07.239272+0000","last_undegraded":"2024-04-25T11:34:07.239272+0000","last_fullsized":"2024-04-25T11:34:07.239272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:52.212358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.240296+0000","last_change":"2024-04-25T11:34:06.235348+0000","last_active":"2024-04-25T11:34:06.240296+0000","last_peered":"2024-04-25T11:34:06.240296+0000","last_clean":"2024-04-25T11:34:06.240296+0000","last_became_active":"2024-04-25T11:34:06.235004+0000","last_became_peered":"2024-04-25T11:34:06.235004+0000","last_unstale":"2024-04-25T11:34:06.240296+0000","last_undegraded":"2024-04-25T11:34:06.240296+0000","last_fullsized":"2024-04-25T11:34:06.240296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:28.172502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.239584+0000","last_change":"2024-04-25T11:34:06.233392+0000","last_active":"2024-04-25T11:34:06.239584+0000","last_peered":"2024-04-25T11:34:06.239584+0000","last_clean":"2024-04-25T11:34:06.239584+0000","last_became_active":"2024-04-25T11:34:06.232959+0000","last_became_peered":"2024-04-25T11:34:06.232959+0000","last_unstale":"2024-04-25T11:34:06.239584+0000","last_undegraded":"2024-04-25T11:34:06.239584+0000","last_fullsized":"2024-04-25T11:34:06.239584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:08.909757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239820+0000","last_change":"2024-04-25T11:34:06.254370+0000","last_active":"2024-04-25T11:34:07.239820+0000","last_peered":"2024-04-25T11:34:07.239820+0000","last_clean":"2024-04-25T11:34:07.239820+0000","last_became_active":"2024-04-25T11:34:06.251958+0000","last_became_peered":"2024-04-25T11:34:06.251958+0000","last_unstale":"2024-04-25T11:34:07.239820+0000","last_undegraded":"2024-04-25T11:34:07.239820+0000","last_fullsized":"2024-04-25T11:34:07.239820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:17.742305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.256439+0000","last_change":"2024-04-25T11:34:06.253247+0000","last_active":"2024-04-25T11:34:06.256439+0000","last_peered":"2024-04-25T11:34:06.256439+0000","last_clean":"2024-04-25T11:34:06.256439+0000","last_became_active":"2024-04-25T11:34:06.252909+0000","last_became_peered":"2024-04-25T11:34:06.252909+0000","last_unstale":"2024-04-25T11:34:06.256439+0000","last_undegraded":"2024-04-25T11:34:06.256439+0000","last_fullsized":"2024-04-25T11:34:06.256439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:16.119265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481384+0000","last_change":"2024-04-25T11:34:06.247920+0000","last_active":"2024-04-25T11:34:07.481384+0000","last_peered":"2024-04-25T11:34:07.481384+0000","last_clean":"2024-04-25T11:34:07.481384+0000","last_became_active":"2024-04-25T11:34:06.244097+0000","last_became_peered":"2024-04-25T11:34:06.244097+0000","last_unstale":"2024-04-25T11:34:07.481384+0000","last_undegraded":"2024-04-25T11:34:07.481384+0000","last_fullsized":"2024-04-25T11:34:07.481384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:11.226289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483138+0000","last_change":"2024-04-25T11:34:06.249507+0000","last_active":"2024-04-25T11:34:07.483138+0000","last_peered":"2024-04-25T11:34:07.483138+0000","last_clean":"2024-04-25T11:34:07.483138+0000","last_became_active":"2024-04-25T11:34:06.249194+0000","last_became_peered":"2024-04-25T11:34:06.249194+0000","last_unstale":"2024-04-25T11:34:07.483138+0000","last_undegraded":"2024-04-25T11:34:07.483138+0000","last_fullsized":"2024-04-25T11:34:07.483138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:01.685889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246669+0000","last_change":"2024-04-25T11:34:06.246937+0000","last_active":"2024-04-25T11:34:07.246669+0000","last_peered":"2024-04-25T11:34:07.246669+0000","last_clean":"2024-04-25T11:34:07.246669+0000","last_became_active":"2024-04-25T11:34:06.246417+0000","last_became_peered":"2024-04-25T11:34:06.246417+0000","last_unstale":"2024-04-25T11:34:07.246669+0000","last_undegraded":"2024-04-25T11:34:07.246669+0000","last_fullsized":"2024-04-25T11:34:07.246669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:12.727113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239935+0000","last_change":"2024-04-25T11:34:06.246692+0000","last_active":"2024-04-25T11:34:07.239935+0000","last_peered":"2024-04-25T11:34:07.239935+0000","last_clean":"2024-04-25T11:34:07.239935+0000","last_became_active":"2024-04-25T11:34:06.246076+0000","last_became_peered":"2024-04-25T11:34:06.246076+0000","last_unstale":"2024-04-25T11:34:07.239935+0000","last_undegraded":"2024-04-25T11:34:07.239935+0000","last_fullsized":"2024-04-25T11:34:07.239935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:22.842505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247193+0000","last_change":"2024-04-25T11:34:06.242235+0000","last_active":"2024-04-25T11:34:07.247193+0000","last_peered":"2024-04-25T11:34:07.247193+0000","last_clean":"2024-04-25T11:34:07.247193+0000","last_became_active":"2024-04-25T11:34:06.241839+0000","last_became_peered":"2024-04-25T11:34:06.241839+0000","last_unstale":"2024-04-25T11:34:07.247193+0000","last_undegraded":"2024-04-25T11:34:07.247193+0000","last_fullsized":"2024-04-25T11:34:07.247193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:50.529668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239154+0000","last_change":"2024-04-25T11:34:06.249562+0000","last_active":"2024-04-25T11:34:07.239154+0000","last_peered":"2024-04-25T11:34:07.239154+0000","last_clean":"2024-04-25T11:34:07.239154+0000","last_became_active":"2024-04-25T11:34:06.249204+0000","last_became_peered":"2024-04-25T11:34:06.249204+0000","last_unstale":"2024-04-25T11:34:07.239154+0000","last_undegraded":"2024-04-25T11:34:07.239154+0000","last_fullsized":"2024-04-25T11:34:07.239154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:54.163242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.245055+0000","last_change":"2024-04-25T11:34:06.241681+0000","last_active":"2024-04-25T11:34:06.245055+0000","last_peered":"2024-04-25T11:34:06.245055+0000","last_clean":"2024-04-25T11:34:06.245055+0000","last_became_active":"2024-04-25T11:34:06.239306+0000","last_became_peered":"2024-04-25T11:34:06.239306+0000","last_unstale":"2024-04-25T11:34:06.245055+0000","last_undegraded":"2024-04-25T11:34:06.245055+0000","last_fullsized":"2024-04-25T11:34:06.245055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:23.928567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482454+0000","last_change":"2024-04-25T11:34:06.244295+0000","last_active":"2024-04-25T11:34:07.482454+0000","last_peered":"2024-04-25T11:34:07.482454+0000","last_clean":"2024-04-25T11:34:07.482454+0000","last_became_active":"2024-04-25T11:34:06.243951+0000","last_became_peered":"2024-04-25T11:34:06.243951+0000","last_unstale":"2024-04-25T11:34:07.482454+0000","last_undegraded":"2024-04-25T11:34:07.482454+0000","last_fullsized":"2024-04-25T11:34:07.482454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:44.452645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244764+0000","last_change":"2024-04-25T11:34:06.258335+0000","last_active":"2024-04-25T11:34:07.244764+0000","last_peered":"2024-04-25T11:34:07.244764+0000","last_clean":"2024-04-25T11:34:07.244764+0000","last_became_active":"2024-04-25T11:34:06.257759+0000","last_became_peered":"2024-04-25T11:34:06.257759+0000","last_unstale":"2024-04-25T11:34:07.244764+0000","last_undegraded":"2024-04-25T11:34:07.244764+0000","last_fullsized":"2024-04-25T11:34:07.244764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:48.544538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239162+0000","last_change":"2024-04-25T11:34:06.255435+0000","last_active":"2024-04-25T11:34:07.239162+0000","last_peered":"2024-04-25T11:34:07.239162+0000","last_clean":"2024-04-25T11:34:07.239162+0000","last_became_active":"2024-04-25T11:34:06.254549+0000","last_became_peered":"2024-04-25T11:34:06.254549+0000","last_unstale":"2024-04-25T11:34:07.239162+0000","last_undegraded":"2024-04-25T11:34:07.239162+0000","last_fullsized":"2024-04-25T11:34:07.239162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:19.046853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481308+0000","last_change":"2024-04-25T11:34:06.246191+0000","last_active":"2024-04-25T11:34:07.481308+0000","last_peered":"2024-04-25T11:34:07.481308+0000","last_clean":"2024-04-25T11:34:07.481308+0000","last_became_active":"2024-04-25T11:34:06.245858+0000","last_became_peered":"2024-04-25T11:34:06.245858+0000","last_unstale":"2024-04-25T11:34:07.481308+0000","last_undegraded":"2024-04-25T11:34:07.481308+0000","last_fullsized":"2024-04-25T11:34:07.481308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:03.271274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481038+0000","last_change":"2024-04-25T11:34:06.242072+0000","last_active":"2024-04-25T11:34:07.481038+0000","last_peered":"2024-04-25T11:34:07.481038+0000","last_clean":"2024-04-25T11:34:07.481038+0000","last_became_active":"2024-04-25T11:34:06.241659+0000","last_became_peered":"2024-04-25T11:34:06.241659+0000","last_unstale":"2024-04-25T11:34:07.481038+0000","last_undegraded":"2024-04-25T11:34:07.481038+0000","last_fullsized":"2024-04-25T11:34:07.481038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:29.542275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239456+0000","last_change":"2024-04-25T11:34:06.245505+0000","last_active":"2024-04-25T11:34:07.239456+0000","last_peered":"2024-04-25T11:34:07.239456+0000","last_clean":"2024-04-25T11:34:07.239456+0000","last_became_active":"2024-04-25T11:34:06.245121+0000","last_became_peered":"2024-04-25T11:34:06.245121+0000","last_unstale":"2024-04-25T11:34:07.239456+0000","last_undegraded":"2024-04-25T11:34:07.239456+0000","last_fullsized":"2024-04-25T11:34:07.239456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:14.438600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.484100+0000","last_change":"2024-04-25T11:34:05.243143+0000","last_active":"2024-04-25T11:34:07.484100+0000","last_peered":"2024-04-25T11:34:07.484100+0000","last_clean":"2024-04-25T11:34:07.484100+0000","last_became_active":"2024-04-25T11:34:05.242183+0000","last_became_peered":"2024-04-25T11:34:05.242183+0000","last_unstale":"2024-04-25T11:34:07.484100+0000","last_undegraded":"2024-04-25T11:34:07.484100+0000","last_fullsized":"2024-04-25T11:34:07.484100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:28.525172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245696+0000","last_change":"2024-04-25T11:34:06.244568+0000","last_active":"2024-04-25T11:34:07.245696+0000","last_peered":"2024-04-25T11:34:07.245696+0000","last_clean":"2024-04-25T11:34:07.245696+0000","last_became_active":"2024-04-25T11:34:06.244191+0000","last_became_peered":"2024-04-25T11:34:06.244191+0000","last_unstale":"2024-04-25T11:34:07.245696+0000","last_undegraded":"2024-04-25T11:34:07.245696+0000","last_fullsized":"2024-04-25T11:34:07.245696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:48.211622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.263488+0000","last_change":"2024-04-25T11:34:05.243981+0000","last_active":"2024-04-25T11:34:06.263488+0000","last_peered":"2024-04-25T11:34:06.263488+0000","last_clean":"2024-04-25T11:34:06.263488+0000","last_became_active":"2024-04-25T11:34:05.243334+0000","last_became_peered":"2024-04-25T11:34:05.243334+0000","last_unstale":"2024-04-25T11:34:06.263488+0000","last_undegraded":"2024-04-25T11:34:06.263488+0000","last_fullsized":"2024-04-25T11:34:06.263488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:47.163536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483505+0000","last_change":"2024-04-25T11:34:06.253060+0000","last_active":"2024-04-25T11:34:07.483505+0000","last_peered":"2024-04-25T11:34:07.483505+0000","last_clean":"2024-04-25T11:34:07.483505+0000","last_became_active":"2024-04-25T11:34:06.252733+0000","last_became_peered":"2024-04-25T11:34:06.252733+0000","last_unstale":"2024-04-25T11:34:07.483505+0000","last_undegraded":"2024-04-25T11:34:07.483505+0000","last_fullsized":"2024-04-25T11:34:07.483505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:54.924418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240215+0000","last_change":"2024-04-25T11:34:05.236597+0000","last_active":"2024-04-25T11:34:07.240215+0000","last_peered":"2024-04-25T11:34:07.240215+0000","last_clean":"2024-04-25T11:34:07.240215+0000","last_became_active":"2024-04-25T11:34:05.236205+0000","last_became_peered":"2024-04-25T11:34:05.236205+0000","last_unstale":"2024-04-25T11:34:07.240215+0000","last_undegraded":"2024-04-25T11:34:07.240215+0000","last_fullsized":"2024-04-25T11:34:07.240215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:05.340167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246689+0000","last_change":"2024-04-25T11:34:06.243440+0000","last_active":"2024-04-25T11:34:07.246689+0000","last_peered":"2024-04-25T11:34:07.246689+0000","last_clean":"2024-04-25T11:34:07.246689+0000","last_became_active":"2024-04-25T11:34:06.242919+0000","last_became_peered":"2024-04-25T11:34:06.242919+0000","last_unstale":"2024-04-25T11:34:07.246689+0000","last_undegraded":"2024-04-25T11:34:07.246689+0000","last_fullsized":"2024-04-25T11:34:07.246689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.279224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483695+0000","last_change":"2024-04-25T11:34:05.235526+0000","last_active":"2024-04-25T11:34:07.483695+0000","last_peered":"2024-04-25T11:34:07.483695+0000","last_clean":"2024-04-25T11:34:07.483695+0000","last_became_active":"2024-04-25T11:34:05.235143+0000","last_became_peered":"2024-04-25T11:34:05.235143+0000","last_unstale":"2024-04-25T11:34:07.483695+0000","last_undegraded":"2024-04-25T11:34:07.483695+0000","last_fullsized":"2024-04-25T11:34:07.483695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:10.209186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239828+0000","last_change":"2024-04-25T11:34:06.253944+0000","last_active":"2024-04-25T11:34:07.239828+0000","last_peered":"2024-04-25T11:34:07.239828+0000","last_clean":"2024-04-25T11:34:07.239828+0000","last_became_active":"2024-04-25T11:34:06.253319+0000","last_became_peered":"2024-04-25T11:34:06.253319+0000","last_unstale":"2024-04-25T11:34:07.239828+0000","last_undegraded":"2024-04-25T11:34:07.239828+0000","last_fullsized":"2024-04-25T11:34:07.239828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:25.239741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248217+0000","last_change":"2024-04-25T11:34:05.238928+0000","last_active":"2024-04-25T11:34:07.248217+0000","last_peered":"2024-04-25T11:34:07.248217+0000","last_clean":"2024-04-25T11:34:07.248217+0000","last_became_active":"2024-04-25T11:34:05.238522+0000","last_became_peered":"2024-04-25T11:34:05.238522+0000","last_unstale":"2024-04-25T11:34:07.248217+0000","last_undegraded":"2024-04-25T11:34:07.248217+0000","last_fullsized":"2024-04-25T11:34:07.248217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:09.855120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483684+0000","last_change":"2024-04-25T11:34:06.247101+0000","last_active":"2024-04-25T11:34:07.483684+0000","last_peered":"2024-04-25T11:34:07.483684+0000","last_clean":"2024-04-25T11:34:07.483684+0000","last_became_active":"2024-04-25T11:34:06.246792+0000","last_became_peered":"2024-04-25T11:34:06.246792+0000","last_unstale":"2024-04-25T11:34:07.483684+0000","last_undegraded":"2024-04-25T11:34:07.483684+0000","last_fullsized":"2024-04-25T11:34:07.483684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:25.458654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246353+0000","last_change":"2024-04-25T11:34:05.232685+0000","last_active":"2024-04-25T11:34:07.246353+0000","last_peered":"2024-04-25T11:34:07.246353+0000","last_clean":"2024-04-25T11:34:07.246353+0000","last_became_active":"2024-04-25T11:34:05.232166+0000","last_became_peered":"2024-04-25T11:34:05.232166+0000","last_unstale":"2024-04-25T11:34:07.246353+0000","last_undegraded":"2024-04-25T11:34:07.246353+0000","last_fullsized":"2024-04-25T11:34:07.246353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:36.105936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482838+0000","last_change":"2024-04-25T11:34:06.225391+0000","last_active":"2024-04-25T11:34:07.482838+0000","last_peered":"2024-04-25T11:34:07.482838+0000","last_clean":"2024-04-25T11:34:07.482838+0000","last_became_active":"2024-04-25T11:34:06.224578+0000","last_became_peered":"2024-04-25T11:34:06.224578+0000","last_unstale":"2024-04-25T11:34:07.482838+0000","last_undegraded":"2024-04-25T11:34:07.482838+0000","last_fullsized":"2024-04-25T11:34:07.482838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:24.279224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482760+0000","last_change":"2024-04-25T11:34:05.235110+0000","last_active":"2024-04-25T11:34:07.482760+0000","last_peered":"2024-04-25T11:34:07.482760+0000","last_clean":"2024-04-25T11:34:07.482760+0000","last_became_active":"2024-04-25T11:34:05.233394+0000","last_became_peered":"2024-04-25T11:34:05.233394+0000","last_unstale":"2024-04-25T11:34:07.482760+0000","last_undegraded":"2024-04-25T11:34:07.482760+0000","last_fullsized":"2024-04-25T11:34:07.482760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:28.352911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247766+0000","last_change":"2024-04-25T11:34:06.246976+0000","last_active":"2024-04-25T11:34:07.247766+0000","last_peered":"2024-04-25T11:34:07.247766+0000","last_clean":"2024-04-25T11:34:07.247766+0000","last_became_active":"2024-04-25T11:34:06.245224+0000","last_became_peered":"2024-04-25T11:34:06.245224+0000","last_unstale":"2024-04-25T11:34:07.247766+0000","last_undegraded":"2024-04-25T11:34:07.247766+0000","last_fullsized":"2024-04-25T11:34:07.247766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:44.452645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247568+0000","last_change":"2024-04-25T11:34:05.241365+0000","last_active":"2024-04-25T11:34:07.247568+0000","last_peered":"2024-04-25T11:34:07.247568+0000","last_clean":"2024-04-25T11:34:07.247568+0000","last_became_active":"2024-04-25T11:34:05.240957+0000","last_became_peered":"2024-04-25T11:34:05.240957+0000","last_unstale":"2024-04-25T11:34:07.247568+0000","last_undegraded":"2024-04-25T11:34:07.247568+0000","last_fullsized":"2024-04-25T11:34:07.247568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.664736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247212+0000","last_change":"2024-04-25T11:34:06.250293+0000","last_active":"2024-04-25T11:34:07.247212+0000","last_peered":"2024-04-25T11:34:07.247212+0000","last_clean":"2024-04-25T11:34:07.247212+0000","last_became_active":"2024-04-25T11:34:06.250010+0000","last_became_peered":"2024-04-25T11:34:06.250010+0000","last_unstale":"2024-04-25T11:34:07.247212+0000","last_undegraded":"2024-04-25T11:34:07.247212+0000","last_fullsized":"2024-04-25T11:34:07.247212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:38.681839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239438+0000","last_change":"2024-04-25T11:34:05.233427+0000","last_active":"2024-04-25T11:34:07.239438+0000","last_peered":"2024-04-25T11:34:07.239438+0000","last_clean":"2024-04-25T11:34:07.239438+0000","last_became_active":"2024-04-25T11:34:05.232747+0000","last_became_peered":"2024-04-25T11:34:05.232747+0000","last_unstale":"2024-04-25T11:34:07.239438+0000","last_undegraded":"2024-04-25T11:34:07.239438+0000","last_fullsized":"2024-04-25T11:34:07.239438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:16.602789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482229+0000","last_change":"2024-04-25T11:34:06.252767+0000","last_active":"2024-04-25T11:34:07.482229+0000","last_peered":"2024-04-25T11:34:07.482229+0000","last_clean":"2024-04-25T11:34:07.482229+0000","last_became_active":"2024-04-25T11:34:06.252124+0000","last_became_peered":"2024-04-25T11:34:06.252124+0000","last_unstale":"2024-04-25T11:34:07.482229+0000","last_undegraded":"2024-04-25T11:34:07.482229+0000","last_fullsized":"2024-04-25T11:34:07.482229+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:07.353595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481986+0000","last_change":"2024-04-25T11:34:05.239323+0000","last_active":"2024-04-25T11:34:07.481986+0000","last_peered":"2024-04-25T11:34:07.481986+0000","last_clean":"2024-04-25T11:34:07.481986+0000","last_became_active":"2024-04-25T11:34:05.238821+0000","last_became_peered":"2024-04-25T11:34:05.238821+0000","last_unstale":"2024-04-25T11:34:07.481986+0000","last_undegraded":"2024-04-25T11:34:07.481986+0000","last_fullsized":"2024-04-25T11:34:07.481986+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:47.194519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240000+0000","last_change":"2024-04-25T11:34:06.243408+0000","last_active":"2024-04-25T11:34:07.240000+0000","last_peered":"2024-04-25T11:34:07.240000+0000","last_clean":"2024-04-25T11:34:07.240000+0000","last_became_active":"2024-04-25T11:34:06.242107+0000","last_became_peered":"2024-04-25T11:34:06.242107+0000","last_unstale":"2024-04-25T11:34:07.240000+0000","last_undegraded":"2024-04-25T11:34:07.240000+0000","last_fullsized":"2024-04-25T11:34:07.240000+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:58.032333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248937+0000","last_change":"2024-04-25T11:34:05.245078+0000","last_active":"2024-04-25T11:34:07.248937+0000","last_peered":"2024-04-25T11:34:07.248937+0000","last_clean":"2024-04-25T11:34:07.248937+0000","last_became_active":"2024-04-25T11:34:05.244660+0000","last_became_peered":"2024-04-25T11:34:05.244660+0000","last_unstale":"2024-04-25T11:34:07.248937+0000","last_undegraded":"2024-04-25T11:34:07.248937+0000","last_fullsized":"2024-04-25T11:34:07.248937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:08.627975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.241897+0000","last_change":"2024-04-25T11:34:06.234301+0000","last_active":"2024-04-25T11:34:06.241897+0000","last_peered":"2024-04-25T11:34:06.241897+0000","last_clean":"2024-04-25T11:34:06.241897+0000","last_became_active":"2024-04-25T11:34:06.233914+0000","last_became_peered":"2024-04-25T11:34:06.233914+0000","last_unstale":"2024-04-25T11:34:06.241897+0000","last_undegraded":"2024-04-25T11:34:06.241897+0000","last_fullsized":"2024-04-25T11:34:06.241897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:34.242170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483810+0000","last_change":"2024-04-25T11:34:05.234357+0000","last_active":"2024-04-25T11:34:07.483810+0000","last_peered":"2024-04-25T11:34:07.483810+0000","last_clean":"2024-04-25T11:34:07.483810+0000","last_became_active":"2024-04-25T11:34:05.233927+0000","last_became_peered":"2024-04-25T11:34:05.233927+0000","last_unstale":"2024-04-25T11:34:07.483810+0000","last_undegraded":"2024-04-25T11:34:07.483810+0000","last_fullsized":"2024-04-25T11:34:07.483810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:47.452434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483698+0000","last_change":"2024-04-25T11:34:06.244448+0000","last_active":"2024-04-25T11:34:07.483698+0000","last_peered":"2024-04-25T11:34:07.483698+0000","last_clean":"2024-04-25T11:34:07.483698+0000","last_became_active":"2024-04-25T11:34:06.244099+0000","last_became_peered":"2024-04-25T11:34:06.244099+0000","last_unstale":"2024-04-25T11:34:07.483698+0000","last_undegraded":"2024-04-25T11:34:07.483698+0000","last_fullsized":"2024-04-25T11:34:07.483698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:21.764096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483798+0000","last_change":"2024-04-25T11:34:05.228788+0000","last_active":"2024-04-25T11:34:07.483798+0000","last_peered":"2024-04-25T11:34:07.483798+0000","last_clean":"2024-04-25T11:34:07.483798+0000","last_became_active":"2024-04-25T11:34:05.228353+0000","last_became_peered":"2024-04-25T11:34:05.228353+0000","last_unstale":"2024-04-25T11:34:07.483798+0000","last_undegraded":"2024-04-25T11:34:07.483798+0000","last_fullsized":"2024-04-25T11:34:07.483798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:47.194519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246802+0000","last_change":"2024-04-25T11:34:06.249325+0000","last_active":"2024-04-25T11:34:07.246802+0000","last_peered":"2024-04-25T11:34:07.246802+0000","last_clean":"2024-04-25T11:34:07.246802+0000","last_became_active":"2024-04-25T11:34:06.248971+0000","last_became_peered":"2024-04-25T11:34:06.248971+0000","last_unstale":"2024-04-25T11:34:07.246802+0000","last_undegraded":"2024-04-25T11:34:07.246802+0000","last_fullsized":"2024-04-25T11:34:07.246802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.689445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246398+0000","last_change":"2024-04-25T11:34:05.243888+0000","last_active":"2024-04-25T11:34:07.246398+0000","last_peered":"2024-04-25T11:34:07.246398+0000","last_clean":"2024-04-25T11:34:07.246398+0000","last_became_active":"2024-04-25T11:34:05.243528+0000","last_became_peered":"2024-04-25T11:34:05.243528+0000","last_unstale":"2024-04-25T11:34:07.246398+0000","last_undegraded":"2024-04-25T11:34:07.246398+0000","last_fullsized":"2024-04-25T11:34:07.246398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:20.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.247510+0000","last_change":"2024-04-25T11:34:06.243025+0000","last_active":"2024-04-25T11:34:06.247510+0000","last_peered":"2024-04-25T11:34:06.247510+0000","last_clean":"2024-04-25T11:34:06.247510+0000","last_became_active":"2024-04-25T11:34:06.242346+0000","last_became_peered":"2024-04-25T11:34:06.242346+0000","last_unstale":"2024-04-25T11:34:06.247510+0000","last_undegraded":"2024-04-25T11:34:06.247510+0000","last_fullsized":"2024-04-25T11:34:06.247510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:05.767705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246395+0000","last_change":"2024-04-25T11:34:05.241007+0000","last_active":"2024-04-25T11:34:07.246395+0000","last_peered":"2024-04-25T11:34:07.246395+0000","last_clean":"2024-04-25T11:34:07.246395+0000","last_became_active":"2024-04-25T11:34:05.240726+0000","last_became_peered":"2024-04-25T11:34:05.240726+0000","last_unstale":"2024-04-25T11:34:07.246395+0000","last_undegraded":"2024-04-25T11:34:07.246395+0000","last_fullsized":"2024-04-25T11:34:07.246395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:35.216257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.244141+0000","last_change":"2024-04-25T11:34:06.243138+0000","last_active":"2024-04-25T11:34:07.244141+0000","last_peered":"2024-04-25T11:34:07.244141+0000","last_clean":"2024-04-25T11:34:07.244141+0000","last_became_active":"2024-04-25T11:34:06.242767+0000","last_became_peered":"2024-04-25T11:34:06.242767+0000","last_unstale":"2024-04-25T11:34:07.244141+0000","last_undegraded":"2024-04-25T11:34:07.244141+0000","last_fullsized":"2024-04-25T11:34:07.244141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:56.956581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246100+0000","last_change":"2024-04-25T11:34:05.232409+0000","last_active":"2024-04-25T11:34:07.246100+0000","last_peered":"2024-04-25T11:34:07.246100+0000","last_clean":"2024-04-25T11:34:07.246100+0000","last_became_active":"2024-04-25T11:34:05.232086+0000","last_became_peered":"2024-04-25T11:34:05.232086+0000","last_unstale":"2024-04-25T11:34:07.246100+0000","last_undegraded":"2024-04-25T11:34:07.246100+0000","last_fullsized":"2024-04-25T11:34:07.246100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:27.740735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481994+0000","last_change":"2024-04-25T11:34:06.245145+0000","last_active":"2024-04-25T11:34:07.481994+0000","last_peered":"2024-04-25T11:34:07.481994+0000","last_clean":"2024-04-25T11:34:07.481994+0000","last_became_active":"2024-04-25T11:34:06.244130+0000","last_became_peered":"2024-04-25T11:34:06.244130+0000","last_unstale":"2024-04-25T11:34:07.481994+0000","last_undegraded":"2024-04-25T11:34:07.481994+0000","last_fullsized":"2024-04-25T11:34:07.481994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:43.166051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239619+0000","last_change":"2024-04-25T11:34:05.239168+0000","last_active":"2024-04-25T11:34:07.239619+0000","last_peered":"2024-04-25T11:34:07.239619+0000","last_clean":"2024-04-25T11:34:07.239619+0000","last_became_active":"2024-04-25T11:34:05.238775+0000","last_became_peered":"2024-04-25T11:34:05.238775+0000","last_unstale":"2024-04-25T11:34:07.239619+0000","last_undegraded":"2024-04-25T11:34:07.239619+0000","last_fullsized":"2024-04-25T11:34:07.239619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:07.510487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482715+0000","last_change":"2024-04-25T11:34:06.250245+0000","last_active":"2024-04-25T11:34:07.482715+0000","last_peered":"2024-04-25T11:34:07.482715+0000","last_clean":"2024-04-25T11:34:07.482715+0000","last_became_active":"2024-04-25T11:34:06.247561+0000","last_became_peered":"2024-04-25T11:34:06.247561+0000","last_unstale":"2024-04-25T11:34:07.482715+0000","last_undegraded":"2024-04-25T11:34:07.482715+0000","last_fullsized":"2024-04-25T11:34:07.482715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:14.216115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482132+0000","last_change":"2024-04-25T11:34:05.224234+0000","last_active":"2024-04-25T11:34:07.482132+0000","last_peered":"2024-04-25T11:34:07.482132+0000","last_clean":"2024-04-25T11:34:07.482132+0000","last_became_active":"2024-04-25T11:34:05.221602+0000","last_became_peered":"2024-04-25T11:34:05.221602+0000","last_unstale":"2024-04-25T11:34:07.482132+0000","last_undegraded":"2024-04-25T11:34:07.482132+0000","last_fullsized":"2024-04-25T11:34:07.482132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:44.994317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245343+0000","last_change":"2024-04-25T11:34:06.247056+0000","last_active":"2024-04-25T11:34:07.245343+0000","last_peered":"2024-04-25T11:34:07.245343+0000","last_clean":"2024-04-25T11:34:07.245343+0000","last_became_active":"2024-04-25T11:34:06.246027+0000","last_became_peered":"2024-04-25T11:34:06.246027+0000","last_unstale":"2024-04-25T11:34:07.245343+0000","last_undegraded":"2024-04-25T11:34:07.245343+0000","last_fullsized":"2024-04-25T11:34:07.245343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:28.956060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481741+0000","last_change":"2024-04-25T11:34:05.227013+0000","last_active":"2024-04-25T11:34:07.481741+0000","last_peered":"2024-04-25T11:34:07.481741+0000","last_clean":"2024-04-25T11:34:07.481741+0000","last_became_active":"2024-04-25T11:34:05.226683+0000","last_became_peered":"2024-04-25T11:34:05.226683+0000","last_unstale":"2024-04-25T11:34:07.481741+0000","last_undegraded":"2024-04-25T11:34:07.481741+0000","last_fullsized":"2024-04-25T11:34:07.481741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:47.452434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.245540+0000","last_change":"2024-04-25T11:34:06.252333+0000","last_active":"2024-04-25T11:34:07.245540+0000","last_peered":"2024-04-25T11:34:07.245540+0000","last_clean":"2024-04-25T11:34:07.245540+0000","last_became_active":"2024-04-25T11:34:06.251724+0000","last_became_peered":"2024-04-25T11:34:06.251724+0000","last_unstale":"2024-04-25T11:34:07.245540+0000","last_undegraded":"2024-04-25T11:34:07.245540+0000","last_fullsized":"2024-04-25T11:34:07.245540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:29.370014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483948+0000","last_change":"2024-04-25T11:34:05.238080+0000","last_active":"2024-04-25T11:34:07.483948+0000","last_peered":"2024-04-25T11:34:07.483948+0000","last_clean":"2024-04-25T11:34:07.483948+0000","last_became_active":"2024-04-25T11:34:05.237600+0000","last_became_peered":"2024-04-25T11:34:05.237600+0000","last_unstale":"2024-04-25T11:34:07.483948+0000","last_undegraded":"2024-04-25T11:34:07.483948+0000","last_fullsized":"2024-04-25T11:34:07.483948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:42.761950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.484121+0000","last_change":"2024-04-25T11:34:06.241010+0000","last_active":"2024-04-25T11:34:07.484121+0000","last_peered":"2024-04-25T11:34:07.484121+0000","last_clean":"2024-04-25T11:34:07.484121+0000","last_became_active":"2024-04-25T11:34:06.240653+0000","last_became_peered":"2024-04-25T11:34:06.240653+0000","last_unstale":"2024-04-25T11:34:07.484121+0000","last_undegraded":"2024-04-25T11:34:07.484121+0000","last_fullsized":"2024-04-25T11:34:07.484121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:14.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483971+0000","last_change":"2024-04-25T11:34:05.238126+0000","last_active":"2024-04-25T11:34:07.483971+0000","last_peered":"2024-04-25T11:34:07.483971+0000","last_clean":"2024-04-25T11:34:07.483971+0000","last_became_active":"2024-04-25T11:34:05.236401+0000","last_became_peered":"2024-04-25T11:34:05.236401+0000","last_unstale":"2024-04-25T11:34:07.483971+0000","last_undegraded":"2024-04-25T11:34:07.483971+0000","last_fullsized":"2024-04-25T11:34:07.483971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:02.254171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.484386+0000","last_change":"2024-04-25T11:34:06.240985+0000","last_active":"2024-04-25T11:34:07.484386+0000","last_peered":"2024-04-25T11:34:07.484386+0000","last_clean":"2024-04-25T11:34:07.484386+0000","last_became_active":"2024-04-25T11:34:06.240558+0000","last_became_peered":"2024-04-25T11:34:06.240558+0000","last_unstale":"2024-04-25T11:34:07.484386+0000","last_undegraded":"2024-04-25T11:34:07.484386+0000","last_fullsized":"2024-04-25T11:34:07.484386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:50.529668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481091+0000","last_change":"2024-04-25T11:34:05.241758+0000","last_active":"2024-04-25T11:34:07.481091+0000","last_peered":"2024-04-25T11:34:07.481091+0000","last_clean":"2024-04-25T11:34:07.481091+0000","last_became_active":"2024-04-25T11:34:05.241393+0000","last_became_peered":"2024-04-25T11:34:05.241393+0000","last_unstale":"2024-04-25T11:34:07.481091+0000","last_undegraded":"2024-04-25T11:34:07.481091+0000","last_fullsized":"2024-04-25T11:34:07.481091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:41.152497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248288+0000","last_change":"2024-04-25T11:34:06.243257+0000","last_active":"2024-04-25T11:34:07.248288+0000","last_peered":"2024-04-25T11:34:07.248288+0000","last_clean":"2024-04-25T11:34:07.248288+0000","last_became_active":"2024-04-25T11:34:06.239661+0000","last_became_peered":"2024-04-25T11:34:06.239661+0000","last_unstale":"2024-04-25T11:34:07.248288+0000","last_undegraded":"2024-04-25T11:34:07.248288+0000","last_fullsized":"2024-04-25T11:34:07.248288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:14.216115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.214784+0000","last_change":"2024-04-25T11:34:05.241495+0000","last_active":"2024-04-25T11:34:06.214784+0000","last_peered":"2024-04-25T11:34:06.214784+0000","last_clean":"2024-04-25T11:34:06.214784+0000","last_became_active":"2024-04-25T11:34:05.241121+0000","last_became_peered":"2024-04-25T11:34:05.241121+0000","last_unstale":"2024-04-25T11:34:06.214784+0000","last_undegraded":"2024-04-25T11:34:06.214784+0000","last_fullsized":"2024-04-25T11:34:06.214784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:54.245245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.241626+0000","last_change":"2024-04-25T11:34:06.235254+0000","last_active":"2024-04-25T11:34:06.241626+0000","last_peered":"2024-04-25T11:34:06.241626+0000","last_clean":"2024-04-25T11:34:06.241626+0000","last_became_active":"2024-04-25T11:34:06.233838+0000","last_became_peered":"2024-04-25T11:34:06.233838+0000","last_unstale":"2024-04-25T11:34:06.241626+0000","last_undegraded":"2024-04-25T11:34:06.241626+0000","last_fullsized":"2024-04-25T11:34:06.241626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:10.838501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.240011+0000","last_change":"2024-04-25T11:34:05.239594+0000","last_active":"2024-04-25T11:34:07.240011+0000","last_peered":"2024-04-25T11:34:07.240011+0000","last_clean":"2024-04-25T11:34:07.240011+0000","last_became_active":"2024-04-25T11:34:05.239093+0000","last_became_peered":"2024-04-25T11:34:05.239093+0000","last_unstale":"2024-04-25T11:34:07.240011+0000","last_undegraded":"2024-04-25T11:34:07.240011+0000","last_fullsized":"2024-04-25T11:34:07.240011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:56.643526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.247191+0000","last_change":"2024-04-25T11:34:06.233895+0000","last_active":"2024-04-25T11:34:07.247191+0000","last_peered":"2024-04-25T11:34:07.247191+0000","last_clean":"2024-04-25T11:34:07.247191+0000","last_became_active":"2024-04-25T11:34:06.232903+0000","last_became_peered":"2024-04-25T11:34:06.232903+0000","last_unstale":"2024-04-25T11:34:07.247191+0000","last_undegraded":"2024-04-25T11:34:07.247191+0000","last_fullsized":"2024-04-25T11:34:07.247191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:32.917368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.243454+0000","last_change":"2024-04-25T11:34:05.227704+0000","last_active":"2024-04-25T11:34:07.243454+0000","last_peered":"2024-04-25T11:34:07.243454+0000","last_clean":"2024-04-25T11:34:07.243454+0000","last_became_active":"2024-04-25T11:34:05.226528+0000","last_became_peered":"2024-04-25T11:34:05.226528+0000","last_unstale":"2024-04-25T11:34:07.243454+0000","last_undegraded":"2024-04-25T11:34:07.243454+0000","last_fullsized":"2024-04-25T11:34:07.243454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:17.374177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481867+0000","last_change":"2024-04-25T11:34:06.257328+0000","last_active":"2024-04-25T11:34:07.481867+0000","last_peered":"2024-04-25T11:34:07.481867+0000","last_clean":"2024-04-25T11:34:07.481867+0000","last_became_active":"2024-04-25T11:34:06.255389+0000","last_became_peered":"2024-04-25T11:34:06.255389+0000","last_unstale":"2024-04-25T11:34:07.481867+0000","last_undegraded":"2024-04-25T11:34:07.481867+0000","last_fullsized":"2024-04-25T11:34:07.481867+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:27.772725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.249042+0000","last_change":"2024-04-25T11:34:05.242223+0000","last_active":"2024-04-25T11:34:07.249042+0000","last_peered":"2024-04-25T11:34:07.249042+0000","last_clean":"2024-04-25T11:34:07.249042+0000","last_became_active":"2024-04-25T11:34:05.241762+0000","last_became_peered":"2024-04-25T11:34:05.241762+0000","last_unstale":"2024-04-25T11:34:07.249042+0000","last_undegraded":"2024-04-25T11:34:07.249042+0000","last_fullsized":"2024-04-25T11:34:07.249042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:28.525172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246890+0000","last_change":"2024-04-25T11:34:06.242461+0000","last_active":"2024-04-25T11:34:07.246890+0000","last_peered":"2024-04-25T11:34:07.246890+0000","last_clean":"2024-04-25T11:34:07.246890+0000","last_became_active":"2024-04-25T11:34:06.241544+0000","last_became_peered":"2024-04-25T11:34:06.241544+0000","last_unstale":"2024-04-25T11:34:07.246890+0000","last_undegraded":"2024-04-25T11:34:07.246890+0000","last_fullsized":"2024-04-25T11:34:07.246890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:14.800705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248921+0000","last_change":"2024-04-25T11:34:05.244709+0000","last_active":"2024-04-25T11:34:07.248921+0000","last_peered":"2024-04-25T11:34:07.248921+0000","last_clean":"2024-04-25T11:34:07.248921+0000","last_became_active":"2024-04-25T11:34:05.244381+0000","last_became_peered":"2024-04-25T11:34:05.244381+0000","last_unstale":"2024-04-25T11:34:07.248921+0000","last_undegraded":"2024-04-25T11:34:07.248921+0000","last_fullsized":"2024-04-25T11:34:07.248921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:02.755508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.239584+0000","last_change":"2024-04-25T11:34:06.234951+0000","last_active":"2024-04-25T11:34:06.239584+0000","last_peered":"2024-04-25T11:34:06.239584+0000","last_clean":"2024-04-25T11:34:06.239584+0000","last_became_active":"2024-04-25T11:34:06.234553+0000","last_became_peered":"2024-04-25T11:34:06.234553+0000","last_unstale":"2024-04-25T11:34:06.239584+0000","last_undegraded":"2024-04-25T11:34:06.239584+0000","last_fullsized":"2024-04-25T11:34:06.239584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:05.022523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.248558+0000","last_change":"2024-04-25T11:34:05.238934+0000","last_active":"2024-04-25T11:34:07.248558+0000","last_peered":"2024-04-25T11:34:07.248558+0000","last_clean":"2024-04-25T11:34:07.248558+0000","last_became_active":"2024-04-25T11:34:05.238531+0000","last_became_peered":"2024-04-25T11:34:05.238531+0000","last_unstale":"2024-04-25T11:34:07.248558+0000","last_undegraded":"2024-04-25T11:34:07.248558+0000","last_fullsized":"2024-04-25T11:34:07.248558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:31.942620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.239549+0000","last_change":"2024-04-25T11:34:06.258078+0000","last_active":"2024-04-25T11:34:07.239549+0000","last_peered":"2024-04-25T11:34:07.239549+0000","last_clean":"2024-04-25T11:34:07.239549+0000","last_became_active":"2024-04-25T11:34:06.256780+0000","last_became_peered":"2024-04-25T11:34:06.256780+0000","last_unstale":"2024-04-25T11:34:07.239549+0000","last_undegraded":"2024-04-25T11:34:07.239549+0000","last_fullsized":"2024-04-25T11:34:07.239549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:35.627673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.218994+0000","last_change":"2024-04-25T11:34:05.245054+0000","last_active":"2024-04-25T11:34:06.218994+0000","last_peered":"2024-04-25T11:34:06.218994+0000","last_clean":"2024-04-25T11:34:06.218994+0000","last_became_active":"2024-04-25T11:34:05.244581+0000","last_became_peered":"2024-04-25T11:34:05.244581+0000","last_unstale":"2024-04-25T11:34:06.218994+0000","last_undegraded":"2024-04-25T11:34:06.218994+0000","last_fullsized":"2024-04-25T11:34:06.218994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:23.697530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.483363+0000","last_change":"2024-04-25T11:34:06.244804+0000","last_active":"2024-04-25T11:34:07.483363+0000","last_peered":"2024-04-25T11:34:07.483363+0000","last_clean":"2024-04-25T11:34:07.483363+0000","last_became_active":"2024-04-25T11:34:06.244431+0000","last_became_peered":"2024-04-25T11:34:06.244431+0000","last_unstale":"2024-04-25T11:34:07.483363+0000","last_undegraded":"2024-04-25T11:34:07.483363+0000","last_fullsized":"2024-04-25T11:34:07.483363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:49.689445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.481238+0000","last_change":"2024-04-25T11:34:05.232793+0000","last_active":"2024-04-25T11:34:07.481238+0000","last_peered":"2024-04-25T11:34:07.481238+0000","last_clean":"2024-04-25T11:34:07.481238+0000","last_became_active":"2024-04-25T11:34:05.232445+0000","last_became_peered":"2024-04-25T11:34:05.232445+0000","last_unstale":"2024-04-25T11:34:07.481238+0000","last_undegraded":"2024-04-25T11:34:07.481238+0000","last_fullsized":"2024-04-25T11:34:07.481238+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:28.741533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.480926+0000","last_change":"2024-04-25T11:34:06.253004+0000","last_active":"2024-04-25T11:34:07.480926+0000","last_peered":"2024-04-25T11:34:07.480926+0000","last_clean":"2024-04-25T11:34:07.480926+0000","last_became_active":"2024-04-25T11:34:06.252687+0000","last_became_peered":"2024-04-25T11:34:06.252687+0000","last_unstale":"2024-04-25T11:34:07.480926+0000","last_undegraded":"2024-04-25T11:34:07.480926+0000","last_fullsized":"2024-04-25T11:34:07.480926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:32.959723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.482999+0000","last_change":"2024-04-25T11:34:05.240378+0000","last_active":"2024-04-25T11:34:07.482999+0000","last_peered":"2024-04-25T11:34:07.482999+0000","last_clean":"2024-04-25T11:34:07.482999+0000","last_became_active":"2024-04-25T11:34:05.240032+0000","last_became_peered":"2024-04-25T11:34:05.240032+0000","last_unstale":"2024-04-25T11:34:07.482999+0000","last_undegraded":"2024-04-25T11:34:07.482999+0000","last_fullsized":"2024-04-25T11:34:07.482999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:31.942620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T11:34:06.255997+0000","last_change":"2024-04-25T11:34:06.252992+0000","last_active":"2024-04-25T11:34:06.255997+0000","last_peered":"2024-04-25T11:34:06.255997+0000","last_clean":"2024-04-25T11:34:06.255997+0000","last_became_active":"2024-04-25T11:34:06.252579+0000","last_became_peered":"2024-04-25T11:34:06.252579+0000","last_unstale":"2024-04-25T11:34:06.255997+0000","last_undegraded":"2024-04-25T11:34:06.255997+0000","last_fullsized":"2024-04-25T11:34:06.255997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:05.183276+0000","last_clean_scrub_stamp":"2024-04-25T11:34:05.183276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:56.315581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246679+0000","last_change":"2024-04-25T11:34:05.236265+0000","last_active":"2024-04-25T11:34:07.246679+0000","last_peered":"2024-04-25T11:34:07.246679+0000","last_clean":"2024-04-25T11:34:07.246679+0000","last_became_active":"2024-04-25T11:34:05.235615+0000","last_became_peered":"2024-04-25T11:34:05.235615+0000","last_unstale":"2024-04-25T11:34:07.246679+0000","last_undegraded":"2024-04-25T11:34:07.246679+0000","last_fullsized":"2024-04-25T11:34:07.246679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:34:04.166173+0000","last_clean_scrub_stamp":"2024-04-25T11:34:04.166173+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:55.939478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":221,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:34:07.246192+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:34:07.246192+0000","last_peered":"2024-04-25T11:34:07.246192+0000","last_clean":"2024-04-25T11:34:07.246192+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:34:07.246192+0000","last_undegraded":"2024-04-25T11:34:07.246192+0000","last_fullsized":"2024-04-25T11:34:07.246192+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":410883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":404617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":954368,"data_stored":403665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":954368,"data_stored":403097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":946176,"data_stored":403553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509527,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":999639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":946176,"data_stored":403041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542229,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1523712,"data_stored":986564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:34:11.583 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:34:11.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:34:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.931+0000 7fcac3e00640 1 Processor -- start 2024-04-25T11:34:11.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.931+0000 7fcac3e00640 1 -- start start 2024-04-25T11:34:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.933+0000 7fcac3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcac40cf540 0x7fcac40b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.933+0000 7fcac3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac406ccf0 con 0x7fcac4164030 2024-04-25T11:34:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.934+0000 7fcac3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac406ca10 con 0x7fcac406cf60 2024-04-25T11:34:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.934+0000 7fcac3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcac4076290 con 0x7fcac40cf540 2024-04-25T11:34:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.934+0000 7fcac3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcac4164030 0x7fcac4164440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51546/0 (socket says 172.21.15.19:51546) 2024-04-25T11:34:11.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.934+0000 7fcac3400640 1 -- 172.21.15.19:0/260489818 learned_addr learned my addr 172.21.15.19:0/260489818 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:11.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.934+0000 7fcac2000640 1 --2- 172.21.15.19:0/260489818 >> v2:172.21.15.169:3300/0 conn(0x7fcac40cf540 0x7fcac40b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750835545 0 0) 0x7fcac406ccf0 con 0x7fcac4164030 2024-04-25T11:34:11.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaa8003520 con 0x7fcac4164030 2024-04-25T11:34:11.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2883301558 0 0) 0x7fcaa8003520 con 0x7fcac4164030 2024-04-25T11:34:11.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 >> v2:172.21.15.169:3300/0 conn(0x7fcac40cf540 msgr2=0x7fcac40b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:11.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 --2- 172.21.15.19:0/260489818 >> v2:172.21.15.169:3300/0 conn(0x7fcac40cf540 0x7fcac40b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 >> v1:172.21.15.169:6789/0 conn(0x7fcac406cf60 legacy=0x7fcac41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac4164c00 con 0x7fcac4164030 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fcabc001670 con 0x7fcac4164030 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 >> v1:172.21.15.19:6789/0 conn(0x7fcac4164030 legacy=0x7fcac4164440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa80039b0 0x7fcaa8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcaa8004800 0x7fcaa8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8005250 0x7fcaa800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa800c1b0 con 0x7fcaa8005250 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa800c360 con 0x7fcaa8004800 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.935+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa800c560 con 0x7fcaa80039b0 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac3400640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8005250 0x7fcaa800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac2000640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcaa8004800 0x7fcaa8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac2a00640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa80039b0 0x7fcaa8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac3400640 1 -- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa80039b0 msgr2=0x7fcaa8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac3400640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa80039b0 0x7fcaa8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac3400640 1 -- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcaa8004800 msgr2=0x7fcaa8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac3400640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcaa8004800 0x7fcaa8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac3400640 1 -- 172.21.15.19:0/260489818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa8003520 con 0x7fcaa8005250 2024-04-25T11:34:11.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.936+0000 7fcac2a00640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa80039b0 0x7fcaa8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:11.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.937+0000 7fcac3400640 1 -- 172.21.15.19:0/260489818 reap_dead start 2024-04-25T11:34:11.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.937+0000 7fcac3400640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8005250 0x7fcaa800b9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c210 tx=0x7fcaa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60c16ca2f2aabe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc007d10 con 0x7fcaa8005250 2024-04-25T11:34:11.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcabc0058d0 con 0x7fcaa8005250 2024-04-25T11:34:11.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac1600640 1 -- 172.21.15.19:0/260489818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc005cc0 con 0x7fcaa8005250 2024-04-25T11:34:11.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac3e00640 1 -- 172.21.15.19:0/260489818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8005250 msgr2=0x7fcaa800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac3e00640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8005250 0x7fcaa800b9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00c210 tx=0x7fcaa8005830 comp rx=0 tx=0).stop 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac3e00640 1 -- 172.21.15.19:0/260489818 shutdown_connections 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac3e00640 1 --2- 172.21.15.19:0/260489818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8005250 0x7fcaa800b9c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.938+0000 7fcac3e00640 1 -- 172.21.15.19:0/260489818 >> 172.21.15.19:0/260489818 conn(0x7fcac409b7d0 msgr2=0x7fcac40b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.939+0000 7fcac3e00640 1 -- 172.21.15.19:0/260489818 shutdown_connections 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.939+0000 7fcac3e00640 1 -- 172.21.15.19:0/260489818 wait complete. 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.941+0000 7fcac3e00640 1 Processor -- start 2024-04-25T11:34:11.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.941+0000 7fcac3e00640 1 -- start start 2024-04-25T11:34:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa8005250 0x7fcac41519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac415c780 0x7fcac40d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac40d2100 0x7fcac41602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4075f20 con 0x7fcac415c780 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcac406c850 con 0x7fcac40d2100 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcac40b7810 con 0x7fcaa8005250 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac415c780 0x7fcac40d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac415c780 0x7fcac40d1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43362/0 (socket says 172.21.15.19:43362) 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.943+0000 7fcac2000640 1 -- 172.21.15.19:0/745616897 learned_addr learned my addr 172.21.15.19:0/745616897 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac3400640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac40d2100 0x7fcac41602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac2a00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa8005250 0x7fcac41519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac2000640 1 -- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa8005250 msgr2=0x7fcac41519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac2000640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa8005250 0x7fcac41519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac2000640 1 -- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac40d2100 msgr2=0x7fcac41602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac2000640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac40d2100 0x7fcac41602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac2000640 1 -- 172.21.15.19:0/745616897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac40d2510 con 0x7fcac415c780 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac3400640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac40d2100 0x7fcac41602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:11.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.944+0000 7fcac2000640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac415c780 0x7fcac40d1bc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcaac011560 tx=0x7fcaac011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.945+0000 7fcabaa00640 1 -- 172.21.15.19:0/745616897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac015930 con 0x7fcac415c780 2024-04-25T11:34:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.945+0000 7fcabaa00640 1 -- 172.21.15.19:0/745616897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaac015ad0 con 0x7fcac415c780 2024-04-25T11:34:11.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.945+0000 7fcabaa00640 1 -- 172.21.15.19:0/745616897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac01d400 con 0x7fcac415c780 2024-04-25T11:34:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.945+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac415d240 con 0x7fcac415c780 2024-04-25T11:34:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.945+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac415d800 con 0x7fcac415c780 2024-04-25T11:34:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.946+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac4161a20 con 0x7fcac415c780 2024-04-25T11:34:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.947+0000 7fcabaa00640 1 -- 172.21.15.19:0/745616897 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcaac013400 con 0x7fcac415c780 2024-04-25T11:34:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.948+0000 7fcabaa00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcaa00744b0 0x7fcaa00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.948+0000 7fcabaa00640 1 -- 172.21.15.19:0/745616897 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fcaac095a10 con 0x7fcac415c780 2024-04-25T11:34:11.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.949+0000 7fcac2a00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcaa00744b0 0x7fcaa00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.950+0000 7fcac2a00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcaa00744b0 0x7fcaa00769c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcab4001330 tx=0x7fcab4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:11.953+0000 7fcabaa00640 1 -- 172.21.15.19:0/745616897 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcac40d2510 con 0x7fcac415c780 2024-04-25T11:34:12.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.237+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcac4076c60 con 0x7fcac415c780 2024-04-25T11:34:12.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.242+0000 7fcabaa00640 1 -- 172.21.15.19:0/745616897 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fcaac019030 con 0x7fcac415c780 2024-04-25T11:34:12.231 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T11:34:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:34:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T11:34:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T11:34:12.231 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:34:12.231 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T11:34:12.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcaa00744b0 msgr2=0x7fcaa00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcaa00744b0 0x7fcaa00769c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcab4001330 tx=0x7fcab4009040 comp rx=0 tx=0).stop 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac415c780 msgr2=0x7fcac40d1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac415c780 0x7fcac40d1bc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcaac011560 tx=0x7fcaac011a20 comp rx=0 tx=0).stop 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 shutdown_connections 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcaa00744b0 0x7fcaa00769c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcac40d2100 0x7fcac41602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcac415c780 0x7fcac40d1bc0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 --2- 172.21.15.19:0/745616897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcaa8005250 0x7fcac41519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 >> 172.21.15.19:0/745616897 conn(0x7fcac409b7d0 msgr2=0x7fcac40b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:12.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 shutdown_connections 2024-04-25T11:34:12.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.245+0000 7fcac3e00640 1 -- 172.21.15.19:0/745616897 wait complete. 2024-04-25T11:34:12.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.548+0000 7f067a000640 1 Processor -- start 2024-04-25T11:34:12.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.548+0000 7f067a000640 1 -- start start 2024-04-25T11:34:12.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f067a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f06741443c0 0x7f067406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f067a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0674070370 0x7f06740cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f067a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06740cfb20 0x7f06740ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f067a000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f06740708d0 con 0x7f0674070370 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f067a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f067415f160 con 0x7f06741443c0 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f067a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06740cff30 con 0x7f06740cfb20 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f066be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0674070370 0x7f06740cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f066be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0674070370 0x7f06740cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43376/0 (socket says 172.21.15.19:43376) 2024-04-25T11:34:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.551+0000 7f066be00640 1 -- 172.21.15.19:0/392447352 learned_addr learned my addr 172.21.15.19:0/392447352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f0673400640 1 --2- 172.21.15.19:0/392447352 >> v2:172.21.15.169:3300/0 conn(0x7f06741443c0 0x7f067406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f0673e00640 1 --2- 172.21.15.19:0/392447352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06740cfb20 0x7f06740ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f066be00640 1 -- 172.21.15.19:0/392447352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06740cfb20 msgr2=0x7f06740ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f066be00640 1 --2- 172.21.15.19:0/392447352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06740cfb20 0x7f06740ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f066be00640 1 -- 172.21.15.19:0/392447352 >> v2:172.21.15.169:3300/0 conn(0x7f06741443c0 msgr2=0x7f067406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f066be00640 1 --2- 172.21.15.19:0/392447352 >> v2:172.21.15.169:3300/0 conn(0x7f06741443c0 0x7f067406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f066be00640 1 -- 172.21.15.19:0/392447352 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067415f5f0 con 0x7f0674070370 2024-04-25T11:34:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.552+0000 7f0673400640 1 --2- 172.21.15.19:0/392447352 >> v2:172.21.15.169:3300/0 conn(0x7f06741443c0 0x7f067406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.553+0000 7f066be00640 1 --2- 172.21.15.19:0/392447352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0674070370 0x7f06740cf5e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0674072820 tx=0x7f065c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b585c0f26a68a4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.553+0000 7f0672a00640 1 -- 172.21.15.19:0/392447352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c037070 con 0x7f0674070370 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f0672a00640 1 -- 172.21.15.19:0/392447352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c032440 con 0x7f0674070370 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f0672a00640 1 -- 172.21.15.19:0/392447352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c032810 con 0x7f0674070370 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f067a000640 1 -- 172.21.15.19:0/392447352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0674070370 msgr2=0x7f06740cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f067a000640 1 --2- 172.21.15.19:0/392447352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0674070370 0x7f06740cf5e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0674072820 tx=0x7f065c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f067a000640 1 -- 172.21.15.19:0/392447352 shutdown_connections 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f067a000640 1 --2- 172.21.15.19:0/392447352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0674070370 0x7f06740cf5e0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f067a000640 1 --2- 172.21.15.19:0/392447352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06740cfb20 0x7f06740ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f067a000640 1 --2- 172.21.15.19:0/392447352 >> v2:172.21.15.169:3300/0 conn(0x7f06741443c0 0x7f067406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.554+0000 7f067a000640 1 -- 172.21.15.19:0/392447352 >> 172.21.15.19:0/392447352 conn(0x7f06740d1ae0 msgr2=0x7f06740d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.555+0000 7f067a000640 1 -- 172.21.15.19:0/392447352 shutdown_connections 2024-04-25T11:34:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.555+0000 7f067a000640 1 -- 172.21.15.19:0/392447352 wait complete. 2024-04-25T11:34:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.557+0000 7f067a000640 1 Processor -- start 2024-04-25T11:34:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.557+0000 7f067a000640 1 -- start start 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f067a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0674070370 0x7f0674143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f067a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06740cfb20 0x7f0674159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f067a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06741443c0 0x7f0674159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f067a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f067415edf0 con 0x7f06740cfb20 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f067a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f067415eb10 con 0x7f0674070370 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f067a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f06741447d0 con 0x7f06741443c0 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f066be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06740cfb20 0x7f0674159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f066be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06740cfb20 0x7f0674159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43390/0 (socket says 172.21.15.19:43390) 2024-04-25T11:34:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f066be00640 1 -- 172.21.15.19:0/160032644 learned_addr learned my addr 172.21.15.19:0/160032644 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f0673400640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0674070370 0x7f0674143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.559+0000 7f0673e00640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06741443c0 0x7f0674159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066be00640 1 -- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06741443c0 msgr2=0x7f0674159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066be00640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06741443c0 0x7f0674159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066be00640 1 -- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0674070370 msgr2=0x7f0674143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066be00640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0674070370 0x7f0674143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066be00640 1 -- 172.21.15.19:0/160032644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0674157ac0 con 0x7f06740cfb20 2024-04-25T11:34:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066be00640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06740cfb20 0x7f0674159770 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f065c02a220 tx=0x7f065c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066b400640 1 -- 172.21.15.19:0/160032644 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c041020 con 0x7f06740cfb20 2024-04-25T11:34:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f066b400640 1 -- 172.21.15.19:0/160032644 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c0306c0 con 0x7f06740cfb20 2024-04-25T11:34:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.560+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0674157da0 con 0x7f06740cfb20 2024-04-25T11:34:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.561+0000 7f066b400640 1 -- 172.21.15.19:0/160032644 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c037040 con 0x7f06740cfb20 2024-04-25T11:34:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.561+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0674158360 con 0x7f06740cfb20 2024-04-25T11:34:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.562+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0674157830 con 0x7f06740cfb20 2024-04-25T11:34:12.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.563+0000 7f066b400640 1 -- 172.21.15.19:0/160032644 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f065c044020 con 0x7f06740cfb20 2024-04-25T11:34:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.564+0000 7f066b400640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f06500744b0 0x7f06500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.564+0000 7f066b400640 1 -- 172.21.15.19:0/160032644 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f065c0b5a50 con 0x7f06740cfb20 2024-04-25T11:34:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.565+0000 7f0673400640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f06500744b0 0x7f06500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.566+0000 7f0673400640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f06500744b0 0x7f06500769c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0664006fd0 tx=0x7f0664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.570+0000 7f066b400640 1 -- 172.21.15.19:0/160032644 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f065c04b030 con 0x7f06740cfb20 2024-04-25T11:34:12.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.815+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0674050740 con 0x7f06740cfb20 2024-04-25T11:34:12.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.817+0000 7f066b400640 1 -- 172.21.15.19:0/160032644 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5186 (secure 0 0 0) 0x7f065c080260 con 0x7f06740cfb20 2024-04-25T11:34:12.805 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:08.826699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5866":{"gid":5866,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/853310115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":853310115},{"type":"v1","addr":"172.21.15.19:6835","nonce":853310115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5866,"qdb_cluster":[5866]},"id":5}]} 2024-04-25T11:34:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-04-25T11:34:12.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f06500744b0 msgr2=0x7f06500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:12.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f06500744b0 0x7f06500769c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0664006fd0 tx=0x7f0664008040 comp rx=0 tx=0).stop 2024-04-25T11:34:12.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06740cfb20 msgr2=0x7f0674159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:12.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06740cfb20 0x7f0674159770 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f065c02a220 tx=0x7f065c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 shutdown_connections 2024-04-25T11:34:12.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f06500744b0 0x7f06500769c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06741443c0 0x7f0674159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06740cfb20 0x7f0674159770 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 --2- 172.21.15.19:0/160032644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0674070370 0x7f0674143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:12.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 >> 172.21.15.19:0/160032644 conn(0x7f06740d1ae0 msgr2=0x7f06740b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:12.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 shutdown_connections 2024-04-25T11:34:12.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:12.820+0000 7f067a000640 1 -- 172.21.15.19:0/160032644 wait complete. 2024-04-25T11:34:13.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:34:05.831545+0000', 'modified': '2024-04-25T11:34:08.826699+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5866}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5866': {'gid': 5866, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/853310115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 853310115}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 853310115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5866, 'qdb_cluster': [5866]} 2024-04-25T11:34:13.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:34:13.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:34:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.544+0000 7f147ee00640 1 Processor -- start 2024-04-25T11:34:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.544+0000 7f147ee00640 1 -- start start 2024-04-25T11:34:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.546+0000 7f147ee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1478142730 0x7f1478142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.546+0000 7f147ee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14781431d0 con 0x7f14781400a0 2024-04-25T11:34:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.546+0000 7f147ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14781434b0 con 0x7f1478070270 2024-04-25T11:34:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.546+0000 7f147ee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f14780b6090 con 0x7f1478142730 2024-04-25T11:34:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.546+0000 7f1477e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f14781400a0 0x7f14780b6890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51556/0 (socket says 172.21.15.19:51556) 2024-04-25T11:34:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.546+0000 7f1477e00640 1 -- 172.21.15.19:0/2203260895 learned_addr learned my addr 172.21.15.19:0/2203260895 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f146fe00640 1 --2- 172.21.15.19:0/2203260895 >> v2:172.21.15.169:3300/0 conn(0x7f1478142730 0x7f1478142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992865039 0 0) 0x7f14781431d0 con 0x7f14781400a0 2024-04-25T11:34:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f145c003520 con 0x7f14781400a0 2024-04-25T11:34:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3752378898 0 0) 0x7f145c003520 con 0x7f14781400a0 2024-04-25T11:34:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 >> v2:172.21.15.169:3300/0 conn(0x7f1478142730 msgr2=0x7f1478142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 --2- 172.21.15.19:0/2203260895 >> v2:172.21.15.169:3300/0 conn(0x7f1478142730 0x7f1478142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 >> v1:172.21.15.169:6789/0 conn(0x7f1478070270 legacy=0x7f1478070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14780b6370 con 0x7f14781400a0 2024-04-25T11:34:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.547+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1468001670 con 0x7f14781400a0 2024-04-25T11:34:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 >> v1:172.21.15.19:6789/0 conn(0x7f14781400a0 legacy=0x7f14780b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477400640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f145c003a00 0x7f145c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477400640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f145c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477400640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f145c005270 0x7f145c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14780b6090 con 0x7f145c004800 2024-04-25T11:34:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f145c00e3a0 con 0x7f145c003a00 2024-04-25T11:34:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f145c00e5a0 con 0x7f145c005270 2024-04-25T11:34:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f146fe00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f145c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f1477e00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f145c003a00 0x7f145c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f146fe00640 1 -- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f145c005270 msgr2=0x7f145c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f146fe00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f145c005270 0x7f145c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f146fe00640 1 -- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f145c003a00 msgr2=0x7f145c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f146fe00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f145c003a00 0x7f145c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.548+0000 7f146fe00640 1 -- 172.21.15.19:0/2203260895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c003520 con 0x7f145c004800 2024-04-25T11:34:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.549+0000 7f1477e00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f145c003a00 0x7f145c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.549+0000 7f147ce00640 1 -- 172.21.15.19:0/2203260895 reap_dead start 2024-04-25T11:34:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.550+0000 7f146fe00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f145c004c10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1460009ac0 tx=0x7f145c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb3c886e996d742 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.550+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460036070 con 0x7f145c004800 2024-04-25T11:34:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.550+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1460031400 con 0x7f145c004800 2024-04-25T11:34:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.550+0000 7f1477400640 1 -- 172.21.15.19:0/2203260895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460009690 con 0x7f145c004800 2024-04-25T11:34:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.551+0000 7f147ee00640 1 -- 172.21.15.19:0/2203260895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 msgr2=0x7f145c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.551+0000 7f147ee00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f145c004c10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1460009ac0 tx=0x7f145c005830 comp rx=0 tx=0).stop 2024-04-25T11:34:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.551+0000 7f147ee00640 1 -- 172.21.15.19:0/2203260895 shutdown_connections 2024-04-25T11:34:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.551+0000 7f147ee00640 1 --2- 172.21.15.19:0/2203260895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f145c004c10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.551+0000 7f147ee00640 1 -- 172.21.15.19:0/2203260895 >> 172.21.15.19:0/2203260895 conn(0x7f1478073050 msgr2=0x7f1478073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:13.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.551+0000 7f147ee00640 1 -- 172.21.15.19:0/2203260895 shutdown_connections 2024-04-25T11:34:13.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.551+0000 7f147ee00640 1 -- 172.21.15.19:0/2203260895 wait complete. 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.553+0000 7f147ee00640 1 Processor -- start 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.553+0000 7f147ee00640 1 -- start start 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f147ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f147815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f147ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14781f6a70 0x7f1478155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f147ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478156240 0x7f14781527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f147ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1478142cc0 con 0x7f145c004800 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f147ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1478142b40 con 0x7f1478156240 2024-04-25T11:34:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f147ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1478142e40 con 0x7f14781f6a70 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f1477e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f147815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f1477e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f147815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43416/0 (socket says 172.21.15.19:43416) 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.555+0000 7f1477e00640 1 -- 172.21.15.19:0/2436392636 learned_addr learned my addr 172.21.15.19:0/2436392636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f147ce00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478156240 0x7f14781527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1477e00640 1 -- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14781f6a70 msgr2=0x7f1478155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1477e00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14781f6a70 0x7f1478155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1477e00640 1 -- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478156240 msgr2=0x7f14781527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1477e00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478156240 0x7f14781527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1477e00640 1 -- 172.21.15.19:0/2436392636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147815dfc0 con 0x7f145c004800 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1477e00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f147815d6c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f146800d940 tx=0x7f146800de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1474c00640 1 -- 172.21.15.19:0/2436392636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468019070 con 0x7f145c004800 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1474c00640 1 -- 172.21.15.19:0/2436392636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14680074e0 con 0x7f145c004800 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f1474c00640 1 -- 172.21.15.19:0/2436392636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004a00 con 0x7f145c004800 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.556+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147815e2a0 con 0x7f145c004800 2024-04-25T11:34:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.557+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14781f3c00 con 0x7f145c004800 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.559+0000 7f1474c00640 1 -- 172.21.15.19:0/2436392636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1468004ba0 con 0x7f145c004800 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.560+0000 7f1474c00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14540744b0 0x7f14540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.560+0000 7f1474c00640 1 -- 172.21.15.19:0/2436392636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f1468094db0 con 0x7f145c004800 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.561+0000 7f1485e67740 1 Processor -- start 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.561+0000 7f1485e67740 1 -- start start 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.561+0000 7f146fe00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14540744b0 0x7f14540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.562+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 0x5616e9d5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.562+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d75d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.562+0000 7f1485e67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e996d590 0x5616e9d762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.562+0000 7f1485e67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616e9c94600 con 0x5616e996d590 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.562+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5616e9c31100 con 0x5616e99653b0 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.562+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5616e9c905c0 con 0x5616e998e270 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f146fe00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14540744b0 0x7f14540769c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1460033000 tx=0x7f146002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e996d590 0x5616e9d762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e996d590 0x5616e9d762b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43420/0 (socket says 172.21.15.19:43420) 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 -- 172.21.15.19:0/1104445639 learned_addr learned my addr 172.21.15.19:0/1104445639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f146fe00640 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d75d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 -- 172.21.15.19:0/1104445639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 msgr2=0x5616e9d5b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 0x5616e9d5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 -- 172.21.15.19:0/1104445639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 msgr2=0x5616e9d75d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d75d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 -- 172.21.15.19:0/1104445639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e99a62d0 con 0x5616e996d590 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.563+0000 7f147ce00640 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e996d590 0x5616e9d762b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f14780b5c30 tx=0x7f1470001f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3db5160580cff23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.564+0000 7f146cc00640 1 -- 172.21.15.19:0/1104445639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470003970 con 0x5616e996d590 2024-04-25T11:34:13.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.564+0000 7f146cc00640 1 -- 172.21.15.19:0/1104445639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470003b10 con 0x5616e996d590 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.564+0000 7f146cc00640 1 -- 172.21.15.19:0/1104445639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000d730 con 0x5616e996d590 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 -- 172.21.15.19:0/1104445639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e996d590 msgr2=0x5616e9d762b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e996d590 0x5616e9d762b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f14780b5c30 tx=0x7f1470001f70 comp rx=0 tx=0).stop 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 -- 172.21.15.19:0/1104445639 shutdown_connections 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d75d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e996d590 0x5616e9d762b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 --2- 172.21.15.19:0/1104445639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 0x5616e9d5b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 -- 172.21.15.19:0/1104445639 >> 172.21.15.19:0/1104445639 conn(0x5616e99823c0 msgr2=0x5616e9d306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 -- 172.21.15.19:0/1104445639 shutdown_connections 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.565+0000 7f1485e67740 1 -- 172.21.15.19:0/1104445639 wait complete. 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.566+0000 7f1485e67740 1 -- start start 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.566+0000 7f1485e67740 1 Processor -- start 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1485e67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e99653b0 0x5616e9d858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d5e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 0x5616e9d72210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1485e67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616e9c1bf70 con 0x5616e99653b0 2024-04-25T11:34:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5616e9d5cfb0 con 0x5616e996d590 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5616e9d710b0 con 0x5616e998e270 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1485e67740 10 client.0 _ll_register_callbacks cb 0x5616e9c8a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1477e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e99653b0 0x5616e9d858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1477e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e99653b0 0x5616e9d858f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43426/0 (socket says 172.21.15.19:43426) 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.568+0000 7f1477e00640 1 -- 172.21.15.19:0/1270957740 learned_addr learned my addr 172.21.15.19:0/1270957740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f146fe00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d5e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f147ce00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 0x5616e9d72210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1477e00640 1 -- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 msgr2=0x5616e9d72210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1477e00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 0x5616e9d72210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1477e00640 1 -- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 msgr2=0x5616e9d5e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1477e00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d5e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1477e00640 1 -- 172.21.15.19:0/1270957740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e9a706b0 con 0x5616e99653b0 2024-04-25T11:34:13.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f146fe00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d5e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1477e00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e99653b0 0x5616e9d858f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f146805eb30 tx=0x7f146805e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf9550580983abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1465600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1465600640 1 -- 172.21.15.19:0/1270957740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146805ee30 con 0x5616e99653b0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1465600640 1 -- 172.21.15.19:0/1270957740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f146805e2a0 con 0x5616e99653b0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.569+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1470003b10 con 0x5616e99653b0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1465600640 1 -- 172.21.15.19:0/1270957740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f14680631e0 con 0x5616e99653b0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 10 client.7011 fetch_fsmap learned FSMap version 74 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5616e9d5cd80 con 0x5616e99653b0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1465600640 1 -- 172.21.15.19:0/1270957740 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470003b10 con 0x5616e99653b0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1465600640 1 -- 172.21.15.19:0/1270957740 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f1468077070 con 0x5616e99653b0 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 10 client.7011 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 10 client.7011 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 20 client.7011 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e994ce40 0x5616e9d88690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 4 client.7011 mds_command: new command op to 5866 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.570+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 --> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616e9ca2210 con 0x5616e994ce40 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.571+0000 7f146fe00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e994ce40 0x5616e9d88690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.571+0000 7f146fe00640 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e994ce40 0x5616e9d88690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00bcc3c4cee6ec server_cookie=2514a00445da807 in_seq=0 out_seq=0 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.571+0000 7f1465600640 10 client.7011 ms_handle_connect on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.572+0000 7f1465600640 1 -- 172.21.15.19:0/1270957740 <== mds.0 v2:172.21.15.19:6834/853310115 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5616e9ca2210 con 0x5616e994ce40 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.572+0000 7f1465600640 10 client.7011 handle_command_reply: tid=0 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.572+0000 7f1485e67740 1 client.7011 shutdown 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.572+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e99653b0 msgr2=0x5616e9d858f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.572+0000 7f1485e67740 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e99653b0 0x5616e9d858f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f146805eb30 tx=0x7f146805e840 comp rx=0 tx=0).stop 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 shutdown_connections 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e994ce40 0x5616e9d88690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d5e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e998e270 0x5616e9d72210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 --2- 172.21.15.19:0/1270957740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e99653b0 0x5616e9d858f0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1465600640 0 client.7011 ms_handle_reset on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 >> 172.21.15.19:0/1270957740 conn(0x5616e99823c0 msgr2=0x5616e9d306a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 shutdown_connections 2024-04-25T11:34:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 1 -- 172.21.15.19:0/1270957740 wait complete. 2024-04-25T11:34:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.573+0000 7f1485e67740 20 client.7011 trim_cache size 0 max 16384 2024-04-25T11:34:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.594+0000 7f1485e67740 1 Processor -- start 2024-04-25T11:34:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.594+0000 7f1485e67740 1 -- start start 2024-04-25T11:34:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d5b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e996d590 0x5616e9d98a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f1485e67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d30de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f1485e67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14781f3680 con 0x5616e998e270 2024-04-25T11:34:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1478133480 con 0x5616e99653b0 2024-04-25T11:34:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14781f3030 con 0x5616e996d590 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f147ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d30de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f147ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d30de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43432/0 (socket says 172.21.15.19:43432) 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f147ce00640 1 -- 172.21.15.19:0/108440970 learned_addr learned my addr 172.21.15.19:0/108440970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.596+0000 7f1477e00640 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d5b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f146fe00640 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e996d590 0x5616e9d98a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f147ce00640 1 -- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e996d590 msgr2=0x5616e9d98a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f147ce00640 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e996d590 0x5616e9d98a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f147ce00640 1 -- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 msgr2=0x5616e9d5b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f147ce00640 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d5b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f147ce00640 1 -- 172.21.15.19:0/108440970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e9d5c590 con 0x5616e998e270 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f147ce00640 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d30de0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x5616e9d2ca40 tx=0x7f147000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e80fbeb524b957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f1467e00640 1 -- 172.21.15.19:0/108440970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000d930 con 0x5616e998e270 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.597+0000 7f1467e00640 1 -- 172.21.15.19:0/108440970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470024070 con 0x5616e998e270 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1467e00640 1 -- 172.21.15.19:0/108440970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000dd30 con 0x5616e998e270 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 -- 172.21.15.19:0/108440970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 msgr2=0x5616e9d30de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d30de0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x5616e9d2ca40 tx=0x7f147000b000 comp rx=0 tx=0).stop 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 -- 172.21.15.19:0/108440970 shutdown_connections 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d30de0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e996d590 0x5616e9d98a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 --2- 172.21.15.19:0/108440970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e99653b0 0x5616e9d5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 -- 172.21.15.19:0/108440970 >> 172.21.15.19:0/108440970 conn(0x5616e99823c0 msgr2=0x5616e9d6f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 -- 172.21.15.19:0/108440970 shutdown_connections 2024-04-25T11:34:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.598+0000 7f1485e67740 1 -- 172.21.15.19:0/108440970 wait complete. 2024-04-25T11:34:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.599+0000 7f1485e67740 1 -- start start 2024-04-25T11:34:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.600+0000 7f1485e67740 1 Processor -- start 2024-04-25T11:34:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.601+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e99653b0 0x5616e9d83e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.601+0000 7f1485e67740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d84600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.601+0000 7f1485e67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d84b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.601+0000 7f1485e67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616e9d5ee80 con 0x5616e998e270 2024-04-25T11:34:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.601+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5616e9d715a0 con 0x5616e996d590 2024-04-25T11:34:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.601+0000 7f1485e67740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5616e9d5c340 con 0x5616e99653b0 2024-04-25T11:34:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.601+0000 7f1485e67740 10 client.0 _ll_register_callbacks cb 0x7f1468001de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d84b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d84b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43442/0 (socket says 172.21.15.19:43442) 2024-04-25T11:34:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 -- 172.21.15.19:0/1164709231 learned_addr learned my addr 172.21.15.19:0/1164709231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f146fe00640 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d84600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f1477e00640 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e99653b0 0x5616e9d83e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 -- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e99653b0 msgr2=0x5616e9d83e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e99653b0 0x5616e9d83e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 -- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 msgr2=0x5616e9d84600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d84600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.602+0000 7f147ce00640 1 -- 172.21.15.19:0/1164709231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e9df3630 con 0x5616e998e270 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f147ce00640 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d84b40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1470001fb0 tx=0x7f14700014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8ce144ee01c7a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f1465600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f1465600640 1 -- 172.21.15.19:0/1164709231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470034670 con 0x5616e998e270 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f1465600640 1 -- 172.21.15.19:0/1164709231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f1470034810 con 0x5616e998e270 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1470024070 con 0x5616e998e270 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f1465600640 1 -- 172.21.15.19:0/1164709231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f1470034a10 con 0x5616e998e270 2024-04-25T11:34:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f1485e67740 10 client.7017 fetch_fsmap learned FSMap version 74 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.603+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5616e9dd7c40 con 0x5616e998e270 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1465600640 1 -- 172.21.15.19:0/1164709231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1470024070 con 0x5616e998e270 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1465600640 1 -- 172.21.15.19:0/1164709231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f147003f020 con 0x5616e998e270 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1485e67740 10 client.7017 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1485e67740 10 client.7017 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1485e67740 20 client.7017 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1485e67740 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e9df2e60 0x5616e9d2a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1485e67740 4 client.7017 mds_command: new command op to 5866 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 --> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] -- command(tid 0: {"prefix": "status"}) -- 0x5616e9c9c080 con 0x5616e9df2e60 2024-04-25T11:34:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.604+0000 7f1477e00640 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e9df2e60 0x5616e9d2a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1477e00640 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e9df2e60 0x5616e9d2a270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed2274c92df48b8 server_cookie=9682857d4c0c34b5 in_seq=0 out_seq=0 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1465600640 10 client.7017 ms_handle_connect on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1465600640 1 -- 172.21.15.19:0/1164709231 <== mds.0 v2:172.21.15.19:6834/853310115 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5616e9c9c080 con 0x5616e9df2e60 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1465600640 10 client.7017 handle_command_reply: tid=0 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 client.7017 shutdown 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 msgr2=0x5616e9d84b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d84b40 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1470001fb0 tx=0x7f14700014a0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 shutdown_connections 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x5616e9df2e60 0x5616e9d2a270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5616e996d590 0x5616e9d84600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e998e270 0x5616e9d84b40 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 --2- 172.21.15.19:0/1164709231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5616e99653b0 0x5616e9d83e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 >> 172.21.15.19:0/1164709231 conn(0x5616e99823c0 msgr2=0x5616e9d2cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1465600640 0 client.7017 ms_handle_reset on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 shutdown_connections 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.605+0000 7f1485e67740 1 -- 172.21.15.19:0/1164709231 wait complete. 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f1485e67740 20 client.7017 trim_cache size 0 max 16384 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:34:13.597 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5866, 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3795557360000004, 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 74, 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 79, 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.532485396 2024-04-25T11:34:13.598 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:34:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14540744b0 msgr2=0x7f14540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14540744b0 0x7f14540769c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1460033000 tx=0x7f146002e000 comp rx=0 tx=0).stop 2024-04-25T11:34:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 msgr2=0x7f147815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f147815d6c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f146800d940 tx=0x7f146800de00 comp rx=0 tx=0).stop 2024-04-25T11:34:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 shutdown_connections 2024-04-25T11:34:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14540744b0 0x7f14540769c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478156240 0x7f14781527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14781f6a70 0x7f1478155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 --2- 172.21.15.19:0/2436392636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f145c004800 0x7f147815d6c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 >> 172.21.15.19:0/2436392636 conn(0x7f1478073050 msgr2=0x7f147806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:13.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 shutdown_connections 2024-04-25T11:34:13.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.606+0000 7f147ee00640 1 -- 172.21.15.19:0/2436392636 wait complete. 2024-04-25T11:34:13.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:34:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.919+0000 7f3dae200640 1 Processor -- start 2024-04-25T11:34:13.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.919+0000 7f3dae200640 1 -- start start 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dae200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3da81610f0 0x7f3da80b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dae200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da8163cd0 0x7f3da806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dae200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3da81640e0 0x7f3da806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dae200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3da8160c60 con 0x7f3da81610f0 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dae200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3da8160690 con 0x7f3da81640e0 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dae200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3da8164640 con 0x7f3da8163cd0 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dace00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3da81610f0 0x7f3da80b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dace00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3da81610f0 0x7f3da80b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37866/0 (socket says 172.21.15.19:37866) 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dace00640 1 -- 172.21.15.19:0/536229271 learned_addr learned my addr 172.21.15.19:0/536229271 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3da7e00640 1 --2- 172.21.15.19:0/536229271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da8163cd0 0x7f3da806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.921+0000 7f3dad800640 1 --2- 172.21.15.19:0/536229271 >> v2:172.21.15.169:3300/0 conn(0x7f3da81640e0 0x7f3da806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.922+0000 7f3dace00640 1 -- 172.21.15.19:0/536229271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da8163cd0 msgr2=0x7f3da806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.922+0000 7f3dace00640 1 --2- 172.21.15.19:0/536229271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da8163cd0 0x7f3da806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.922+0000 7f3dace00640 1 -- 172.21.15.19:0/536229271 >> v2:172.21.15.169:3300/0 conn(0x7f3da81640e0 msgr2=0x7f3da806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.922+0000 7f3dace00640 1 --2- 172.21.15.19:0/536229271 >> v2:172.21.15.169:3300/0 conn(0x7f3da81640e0 0x7f3da806db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.922+0000 7f3dace00640 1 -- 172.21.15.19:0/536229271 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da8164920 con 0x7f3da81610f0 2024-04-25T11:34:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.922+0000 7f3dad800640 1 --2- 172.21.15.19:0/536229271 >> v2:172.21.15.169:3300/0 conn(0x7f3da81640e0 0x7f3da806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:13.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.923+0000 7f3dace00640 1 --2- 172.21.15.19:0/536229271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81610f0 0x7f3da80b0830 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3da8072e70 tx=0x7f3d9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bff9391406884f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.923+0000 7f3da7400640 1 -- 172.21.15.19:0/536229271 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c037070 con 0x7f3da81610f0 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.923+0000 7f3da7400640 1 -- 172.21.15.19:0/536229271 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9c032440 con 0x7f3da81610f0 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.923+0000 7f3da7400640 1 -- 172.21.15.19:0/536229271 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c032810 con 0x7f3da81610f0 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 -- 172.21.15.19:0/536229271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81610f0 msgr2=0x7f3da80b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 --2- 172.21.15.19:0/536229271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81610f0 0x7f3da80b0830 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3da8072e70 tx=0x7f3d9c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 -- 172.21.15.19:0/536229271 shutdown_connections 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 --2- 172.21.15.19:0/536229271 >> v2:172.21.15.169:3300/0 conn(0x7f3da81640e0 0x7f3da806db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 --2- 172.21.15.19:0/536229271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da8163cd0 0x7f3da806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 --2- 172.21.15.19:0/536229271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81610f0 0x7f3da80b0830 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 -- 172.21.15.19:0/536229271 >> 172.21.15.19:0/536229271 conn(0x7f3da8072ee0 msgr2=0x7f3da80732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:13.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 -- 172.21.15.19:0/536229271 shutdown_connections 2024-04-25T11:34:13.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.924+0000 7f3dae200640 1 -- 172.21.15.19:0/536229271 wait complete. 2024-04-25T11:34:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.927+0000 7f3dae200640 1 Processor -- start 2024-04-25T11:34:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.927+0000 7f3dae200640 1 -- start start 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.928+0000 7f3dae200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da81610f0 0x7f3da815c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.928+0000 7f3dae200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3da8163cd0 0x7f3da8153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.928+0000 7f3dae200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81640e0 0x7f3da80b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.928+0000 7f3dae200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8161500 con 0x7f3da81640e0 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.928+0000 7f3dae200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3da80b0d70 con 0x7f3da8163cd0 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.928+0000 7f3dae200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3da80b2240 con 0x7f3da81610f0 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81640e0 0x7f3da80b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81640e0 0x7f3da80b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37878/0 (socket says 172.21.15.19:37878) 2024-04-25T11:34:13.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 -- 172.21.15.19:0/2757186235 learned_addr learned my addr 172.21.15.19:0/2757186235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dace00640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da81610f0 0x7f3da815c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3da7e00640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3da8163cd0 0x7f3da8153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 -- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da81610f0 msgr2=0x7f3da815c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da81610f0 0x7f3da815c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 -- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3da8163cd0 msgr2=0x7f3da8153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3da8163cd0 0x7f3da8153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.929+0000 7f3dad800640 1 -- 172.21.15.19:0/2757186235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da815c830 con 0x7f3da81640e0 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.930+0000 7f3dad800640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81640e0 0x7f3da80b1520 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3da000cd80 tx=0x7f3da000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.930+0000 7f3da4c00640 1 -- 172.21.15.19:0/2757186235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da000f680 con 0x7f3da81640e0 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.930+0000 7f3da4c00640 1 -- 172.21.15.19:0/2757186235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da000f840 con 0x7f3da81640e0 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.930+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da815fec0 con 0x7f3da81640e0 2024-04-25T11:34:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.930+0000 7f3da4c00640 1 -- 172.21.15.19:0/2757186235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da000fc30 con 0x7f3da81640e0 2024-04-25T11:34:13.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.930+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da815b850 con 0x7f3da81640e0 2024-04-25T11:34:13.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.931+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da8050740 con 0x7f3da81640e0 2024-04-25T11:34:13.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.933+0000 7f3da4c00640 1 -- 172.21.15.19:0/2757186235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3da000e3e0 con 0x7f3da81640e0 2024-04-25T11:34:13.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.933+0000 7f3da4c00640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d880744e0 0x7f3d880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:13.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.934+0000 7f3da4c00640 1 -- 172.21.15.19:0/2757186235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f3da0095ff0 con 0x7f3da81640e0 2024-04-25T11:34:13.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.934+0000 7f3dace00640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d880744e0 0x7f3d880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:13.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.935+0000 7f3dace00640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d880744e0 0x7f3d880769f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c034000 tx=0x7f3d9c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:13.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:13.938+0000 7f3da4c00640 1 -- 172.21.15.19:0/2757186235 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3da0014030 con 0x7f3da81640e0 2024-04-25T11:34:14.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.187+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3da815fbe0 con 0x7f3da81640e0 2024-04-25T11:34:14.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.188+0000 7f3da4c00640 1 -- 172.21.15.19:0/2757186235 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f3da0060730 con 0x7f3da81640e0 2024-04-25T11:34:14.177 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:34:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.191+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d880744e0 msgr2=0x7f3d880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.191+0000 7f3dae200640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d880744e0 0x7f3d880769f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c034000 tx=0x7f3d9c02e000 comp rx=0 tx=0).stop 2024-04-25T11:34:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.191+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81640e0 msgr2=0x7f3da80b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.191+0000 7f3dae200640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81640e0 0x7f3da80b1520 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3da000cd80 tx=0x7f3da000a650 comp rx=0 tx=0).stop 2024-04-25T11:34:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.191+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 shutdown_connections 2024-04-25T11:34:14.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.191+0000 7f3dae200640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d880744e0 0x7f3d880769f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.192+0000 7f3dae200640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3da8163cd0 0x7f3da8153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.192+0000 7f3dae200640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3da81640e0 0x7f3da80b1520 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.192+0000 7f3dae200640 1 --2- 172.21.15.19:0/2757186235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3da81610f0 0x7f3da815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.192+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 >> 172.21.15.19:0/2757186235 conn(0x7f3da8072ee0 msgr2=0x7f3da81621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.192+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 shutdown_connections 2024-04-25T11:34:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14.192+0000 7f3dae200640 1 -- 172.21.15.19:0/2757186235 wait complete. 2024-04-25T11:34:14.210 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:34:14.210 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:34:14.210 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:34:14.210 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:34:14.210 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:34:14.210 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:34:14.210 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:34:14.211 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:34:14.211 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:34:14.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:14.211 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:34:14.264 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:34:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:34:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:34:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42219sec preferred_lft 42219sec 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:34:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:34:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:34:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:34:14.267 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:34:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:34:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:34:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:34:14.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:14.267 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:34:14.342 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:34:14.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:14.346 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:34:14.396 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:34:14.396 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:34:14.397 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:34:14.397 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:34:14.397 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:34:14.398 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:34:14.398 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:34:14.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:34:14.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:34:14.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:34:14.398 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:34:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:34:14.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:34:14.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:14.625 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:34:14.675 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:34:14.675 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:34:14.675 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:34:14.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:14.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:34:14.753 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:34:14.753 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:34:14.810 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:34:14.810 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:34:14.810 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:34:14.867 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:34:14.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:14.867 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:34:15.078 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:34:15.079 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:34:15.124 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:34:15.124 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:34:15.124 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:34:15.124 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:34:15.124 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:34:15.127 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:34:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:34:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:34:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:34:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:34:15.132 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:34:15.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:15.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:34:15.211 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:34:15.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:34:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:34:15.340 INFO:teuthology.orchestra.run.smithi019.stdout:client7026 2024-04-25T11:34:15.340 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:34:15.340 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:34:15.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:15.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.686+0000 7fd396e00640 1 Processor -- start 2024-04-25T11:34:15.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.686+0000 7fd396e00640 1 -- start start 2024-04-25T11:34:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.688+0000 7fd396e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd39006f180 0x7fd390168190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.688+0000 7fd396e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd390071230 con 0x7fd39015c800 2024-04-25T11:34:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.688+0000 7fd396e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3900d0110 con 0x7fd390070e20 2024-04-25T11:34:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.688+0000 7fd396e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd39006f8f0 con 0x7fd39006f180 2024-04-25T11:34:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.688+0000 7fd395000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd39015c800 0x7fd3900cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40220/0 (socket says 172.21.15.19:40220) 2024-04-25T11:34:15.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd395000640 1 -- 172.21.15.19:0/3468122018 learned_addr learned my addr 172.21.15.19:0/3468122018 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd396400640 1 --2- 172.21.15.19:0/3468122018 >> v2:172.21.15.169:3300/0 conn(0x7fd39006f180 0x7fd390168190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361757948 0 0) 0x7fd390071230 con 0x7fd39015c800 2024-04-25T11:34:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd378003520 con 0x7fd39015c800 2024-04-25T11:34:15.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 543914509 0 0) 0x7fd378003520 con 0x7fd39015c800 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 >> v2:172.21.15.169:3300/0 conn(0x7fd39006f180 msgr2=0x7fd390168190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd38fe00640 1 --2- 172.21.15.19:0/3468122018 >> v2:172.21.15.169:3300/0 conn(0x7fd39006f180 0x7fd390168190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.689+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 >> v1:172.21.15.169:6789/0 conn(0x7fd390070e20 legacy=0x7fd3900cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39006fbd0 con 0x7fd39015c800 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fd37c001670 con 0x7fd39015c800 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 >> v1:172.21.15.19:6789/0 conn(0x7fd39015c800 legacy=0x7fd3900cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3780039b0 0x7fd378003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378004800 0x7fd378004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780051c0 0x7fd37800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd39006f8f0 con 0x7fd3780039b0 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd37800e370 con 0x7fd378004800 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd37800e570 con 0x7fd3780051c0 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd395a00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3780039b0 0x7fd378003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd395000640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378004800 0x7fd378004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd395a00640 1 -- 172.21.15.19:0/3468122018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780051c0 msgr2=0x7fd37800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:15.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd395a00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780051c0 0x7fd37800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd395a00640 1 -- 172.21.15.19:0/3468122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378004800 msgr2=0x7fd378004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:15.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd395a00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd378004800 0x7fd378004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.690+0000 7fd395a00640 1 -- 172.21.15.19:0/3468122018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd378003520 con 0x7fd3780039b0 2024-04-25T11:34:15.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.691+0000 7fd396400640 1 -- 172.21.15.19:0/3468122018 reap_dead start 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.692+0000 7fd395a00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3780039b0 0x7fd378003e20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd384001750 tx=0x7fd378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a2624d0d408a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.692+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384001370 con 0x7fd3780039b0 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.692+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3840052d0 con 0x7fd3780039b0 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.692+0000 7fd38fe00640 1 -- 172.21.15.19:0/3468122018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38402ab00 con 0x7fd3780039b0 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.692+0000 7fd396e00640 1 -- 172.21.15.19:0/3468122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3780039b0 msgr2=0x7fd378003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.692+0000 7fd396e00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3780039b0 0x7fd378003e20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd384001750 tx=0x7fd378005830 comp rx=0 tx=0).stop 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.693+0000 7fd396e00640 1 -- 172.21.15.19:0/3468122018 shutdown_connections 2024-04-25T11:34:15.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.693+0000 7fd396e00640 1 --2- 172.21.15.19:0/3468122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3780039b0 0x7fd378003e20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.693+0000 7fd396e00640 1 -- 172.21.15.19:0/3468122018 >> 172.21.15.19:0/3468122018 conn(0x7fd3900d1720 msgr2=0x7fd3900d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:15.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.693+0000 7fd396e00640 1 -- 172.21.15.19:0/3468122018 shutdown_connections 2024-04-25T11:34:15.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.693+0000 7fd396e00640 1 -- 172.21.15.19:0/3468122018 wait complete. 2024-04-25T11:34:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.695+0000 7fd396e00640 1 Processor -- start 2024-04-25T11:34:15.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.696+0000 7fd396e00640 1 -- start start 2024-04-25T11:34:15.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.697+0000 7fd396e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780039b0 0x7fd3900cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.697+0000 7fd396e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd390152280 0x7fd39015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.697+0000 7fd396e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd39015e320 0x7fd3901f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.697+0000 7fd396e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd39015cc10 con 0x7fd390152280 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.697+0000 7fd396e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd390070190 con 0x7fd39015e320 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.697+0000 7fd396e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd39006f590 con 0x7fd3780039b0 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd390152280 0x7fd39015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd390152280 0x7fd39015dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37882/0 (socket says 172.21.15.19:37882) 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 -- 172.21.15.19:0/1145342314 learned_addr learned my addr 172.21.15.19:0/1145342314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395a00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780039b0 0x7fd3900cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd396400640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd39015e320 0x7fd3901f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 -- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780039b0 msgr2=0x7fd3900cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780039b0 0x7fd3900cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 -- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd39015e320 msgr2=0x7fd3901f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd39015e320 0x7fd3901f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.698+0000 7fd395000640 1 -- 172.21.15.19:0/1145342314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3901f6130 con 0x7fd390152280 2024-04-25T11:34:15.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.699+0000 7fd395000640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd390152280 0x7fd39015dde0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d7a0 tx=0x7fd37c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:15.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.699+0000 7fd38d600640 1 -- 172.21.15.19:0/1145342314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c019070 con 0x7fd390152280 2024-04-25T11:34:15.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.699+0000 7fd38d600640 1 -- 172.21.15.19:0/1145342314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37c0074e0 con 0x7fd390152280 2024-04-25T11:34:15.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.699+0000 7fd38d600640 1 -- 172.21.15.19:0/1145342314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c004a20 con 0x7fd390152280 2024-04-25T11:34:15.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.699+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39015e730 con 0x7fd390152280 2024-04-25T11:34:15.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.699+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd390152a80 con 0x7fd390152280 2024-04-25T11:34:15.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.700+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3900cc870 con 0x7fd390152280 2024-04-25T11:34:15.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.702+0000 7fd38d600640 1 -- 172.21.15.19:0/1145342314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd37c004bc0 con 0x7fd390152280 2024-04-25T11:34:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.703+0000 7fd38d600640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3700744b0 0x7fd3700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.703+0000 7fd38d600640 1 -- 172.21.15.19:0/1145342314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fd37c094c60 con 0x7fd390152280 2024-04-25T11:34:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.703+0000 7fd395a00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3700744b0 0x7fd3700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.705+0000 7fd395a00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3700744b0 0x7fd3700769c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd384032000 tx=0x7fd38402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.711+0000 7fd38d600640 1 -- 172.21.15.19:0/1145342314 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd39015e730 con 0x7fd390152280 2024-04-25T11:34:15.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.961+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd390050740 con 0x7fd390152280 2024-04-25T11:34:15.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.962+0000 7fd38d600640 1 -- 172.21.15.19:0/1145342314 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5186 (secure 0 0 0) 0x7fd3901f6130 con 0x7fd390152280 2024-04-25T11:34:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:15.951 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:08.826699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5866":{"gid":5866,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/853310115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":853310115},{"type":"v1","addr":"172.21.15.19:6835","nonce":853310115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5866,"qdb_cluster":[5866]},"id":5}]} 2024-04-25T11:34:15.951 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3700744b0 msgr2=0x7fd3700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3700744b0 0x7fd3700769c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd384032000 tx=0x7fd38402e000 comp rx=0 tx=0).stop 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd390152280 msgr2=0x7fd39015dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd390152280 0x7fd39015dde0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d7a0 tx=0x7fd37c00dc60 comp rx=0 tx=0).stop 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 shutdown_connections 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3700744b0 0x7fd3700769c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd39015e320 0x7fd3901f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd390152280 0x7fd39015dde0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 --2- 172.21.15.19:0/1145342314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3780039b0 0x7fd3900cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.965+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 >> 172.21.15.19:0/1145342314 conn(0x7fd3900d1720 msgr2=0x7fd39006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:15.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.966+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 shutdown_connections 2024-04-25T11:34:15.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:15.966+0000 7fd396e00640 1 -- 172.21.15.19:0/1145342314 wait complete. 2024-04-25T11:34:15.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2024-04-25T11:34:16.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.283+0000 7f3493200640 1 Processor -- start 2024-04-25T11:34:16.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.283+0000 7f3493200640 1 -- start start 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.285+0000 7f3493200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3494144310 0x7f349406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.285+0000 7f3493200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3494161a80 0x7f3494161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.285+0000 7f3493200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3494144720 con 0x7f34940b6ba0 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.285+0000 7f3493200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f349406f640 con 0x7f3494144310 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.285+0000 7f3493200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f34941624d0 con 0x7f3494161a80 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f3491e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3494144310 0x7f349406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f3491e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3494144310 0x7f349406fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37890/0 (socket says 172.21.15.19:37890) 2024-04-25T11:34:16.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f3491e00640 1 -- 172.21.15.19:0/1199804810 learned_addr learned my addr 172.21.15.19:0/1199804810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:16.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f3491e00640 1 -- 172.21.15.19:0/1199804810 >> v2:172.21.15.169:3300/0 conn(0x7f3494161a80 msgr2=0x7f3494161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:16.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f348be00640 1 --2- 172.21.15.19:0/1199804810 >> v2:172.21.15.169:3300/0 conn(0x7f3494161a80 0x7f3494161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f3491e00640 1 --2- 172.21.15.19:0/1199804810 >> v2:172.21.15.169:3300/0 conn(0x7f3494161a80 0x7f3494161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f3491e00640 1 -- 172.21.15.19:0/1199804810 >> v1:172.21.15.169:6789/0 conn(0x7f34940b6ba0 legacy=0x7f34940b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:16.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.286+0000 7f3491e00640 1 -- 172.21.15.19:0/1199804810 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34941627b0 con 0x7f3494144310 2024-04-25T11:34:16.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.287+0000 7f3491e00640 1 --2- 172.21.15.19:0/1199804810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f349406fad0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f34941623d0 tx=0x7f348402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781a289e9a5a9cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:16.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.288+0000 7f3491400640 1 -- 172.21.15.19:0/1199804810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484037070 con 0x7f3494144310 2024-04-25T11:34:16.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.288+0000 7f3491400640 1 -- 172.21.15.19:0/1199804810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3484032440 con 0x7f3494144310 2024-04-25T11:34:16.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.288+0000 7f3491400640 1 -- 172.21.15.19:0/1199804810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484032810 con 0x7f3494144310 2024-04-25T11:34:16.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 -- 172.21.15.19:0/1199804810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 msgr2=0x7f349406fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:16.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 --2- 172.21.15.19:0/1199804810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f349406fad0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f34941623d0 tx=0x7f348402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 -- 172.21.15.19:0/1199804810 shutdown_connections 2024-04-25T11:34:16.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 --2- 172.21.15.19:0/1199804810 >> v2:172.21.15.169:3300/0 conn(0x7f3494161a80 0x7f3494161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 --2- 172.21.15.19:0/1199804810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f349406fad0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 -- 172.21.15.19:0/1199804810 >> 172.21.15.19:0/1199804810 conn(0x7f34940b4260 msgr2=0x7f3494072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:16.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 -- 172.21.15.19:0/1199804810 shutdown_connections 2024-04-25T11:34:16.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.289+0000 7f3493200640 1 -- 172.21.15.19:0/1199804810 wait complete. 2024-04-25T11:34:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.291+0000 7f3493200640 1 Processor -- start 2024-04-25T11:34:16.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.292+0000 7f3493200640 1 -- start start 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.293+0000 7f3493200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f34940ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.293+0000 7f3493200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3494161a80 0x7f34940b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.293+0000 7f3493200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34940aeb80 0x7f34940b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.293+0000 7f3493200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34941478d0 con 0x7f3494144310 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.293+0000 7f3493200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3494070010 con 0x7f3494161a80 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.293+0000 7f3493200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3494147a50 con 0x7f34940aeb80 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f34940ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f34940ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37894/0 (socket says 172.21.15.19:37894) 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 -- 172.21.15.19:0/543687054 learned_addr learned my addr 172.21.15.19:0/543687054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:16.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f348be00640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3494161a80 0x7f34940b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3492800640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34940aeb80 0x7f34940b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 -- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34940aeb80 msgr2=0x7f34940b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34940aeb80 0x7f34940b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 -- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3494161a80 msgr2=0x7f34940b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3494161a80 0x7f34940b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3491e00640 1 -- 172.21.15.19:0/543687054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34940af300 con 0x7f3494144310 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.294+0000 7f3492800640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34940aeb80 0x7f34940b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.295+0000 7f3491e00640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f34940ae640 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3484000c00 tx=0x7f34840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.295+0000 7f348a000640 1 -- 172.21.15.19:0/543687054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484041070 con 0x7f3494144310 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.295+0000 7f348a000640 1 -- 172.21.15.19:0/543687054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348403c870 con 0x7f3494144310 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.295+0000 7f348a000640 1 -- 172.21.15.19:0/543687054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484037040 con 0x7f3494144310 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.295+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34940af5e0 con 0x7f3494144310 2024-04-25T11:34:16.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.295+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34940afba0 con 0x7f3494144310 2024-04-25T11:34:16.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.296+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3494068c30 con 0x7f3494144310 2024-04-25T11:34:16.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.298+0000 7f348a000640 1 -- 172.21.15.19:0/543687054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3484044070 con 0x7f3494144310 2024-04-25T11:34:16.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.299+0000 7f348a000640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34700744e0 0x7f34700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.299+0000 7f348a000640 1 -- 172.21.15.19:0/543687054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f34840b5f20 con 0x7f3494144310 2024-04-25T11:34:16.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.299+0000 7f348be00640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34700744e0 0x7f34700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.300+0000 7f348be00640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34700744e0 0x7f34700769f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f347c006fd0 tx=0x7f347c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:16.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.304+0000 7f348a000640 1 -- 172.21.15.19:0/543687054 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3494068c30 con 0x7f3494144310 2024-04-25T11:34:16.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:16.595+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) -- 0x7f3494076c60 con 0x7f3494144310 2024-04-25T11:34:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.302+0000 7f348a000640 1 -- 172.21.15.19:0/543687054 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.5866 v75) ==== 112+0+0 (secure 0 0 0) 0x7f3484080610 con 0x7f3494144310 2024-04-25T11:34:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:freezing mds.5866 2024-04-25T11:34:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.306+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34700744e0 msgr2=0x7f34700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.306+0000 7f3493200640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34700744e0 0x7f34700769f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f347c006fd0 tx=0x7f347c008040 comp rx=0 tx=0).stop 2024-04-25T11:34:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 msgr2=0x7f34940ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f34940ae640 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3484000c00 tx=0x7f34840046c0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 shutdown_connections 2024-04-25T11:34:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34700744e0 0x7f34700769f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3494161a80 0x7f34940b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34940aeb80 0x7f34940b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 --2- 172.21.15.19:0/543687054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3494144310 0x7f34940ae640 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 >> 172.21.15.19:0/543687054 conn(0x7f34940b6ba0 msgr2=0x7f3494146c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 shutdown_connections 2024-04-25T11:34:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.307+0000 7f3493200640 1 -- 172.21.15.19:0/543687054 wait complete. 2024-04-25T11:34:17.332 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d') 2024-04-25T11:34:17.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:34:17.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:34:17.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-04-25T11:34:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.801+0000 7fcf56200640 1 Processor -- start 2024-04-25T11:34:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.801+0000 7fcf56200640 1 -- start start 2024-04-25T11:34:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.803+0000 7fcf56200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf50161ce0 0x7fcf5006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.803+0000 7fcf56200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcf501620f0 0x7fcf5006db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.803+0000 7fcf56200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf50162650 con 0x7fcf50155350 2024-04-25T11:34:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf56200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcf50162c10 con 0x7fcf501620f0 2024-04-25T11:34:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf56200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf50162930 con 0x7fcf50161ce0 2024-04-25T11:34:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf4f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcf50155350 0x7fcf5015f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40228/0 (socket says 172.21.15.19:40228) 2024-04-25T11:34:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf4f400640 1 -- 172.21.15.19:0/3580904499 learned_addr learned my addr 172.21.15.19:0/3580904499 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf50161ce0 0x7fcf5006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3580904499 >> v2:172.21.15.169:3300/0 conn(0x7fcf501620f0 0x7fcf5006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2967249364 0 0) 0x7fcf50162650 con 0x7fcf50155350 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.804+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf34003520 con 0x7fcf50155350 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3968451131 0 0) 0x7fcf34003520 con 0x7fcf50155350 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf50161ce0 msgr2=0x7fcf5006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf50161ce0 0x7fcf5006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 >> v2:172.21.15.169:3300/0 conn(0x7fcf501620f0 msgr2=0x7fcf5006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 --2- 172.21.15.19:0/3580904499 >> v2:172.21.15.169:3300/0 conn(0x7fcf501620f0 0x7fcf5006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5015e890 con 0x7fcf50155350 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fcf40001670 con 0x7fcf50155350 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 >> v1:172.21.15.19:6789/0 conn(0x7fcf50155350 legacy=0x7fcf5015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340039b0 0x7fcf34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf50161ce0 0x7fcf34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf34004d40 0x7fcf34005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50162c10 con 0x7fcf340039b0 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50162930 con 0x7fcf34004d40 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3400ba40 con 0x7fcf50161ce0 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.805+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340039b0 0x7fcf34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.806+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf34004d40 0x7fcf34005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.806+0000 7fcf4f400640 1 -- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf50161ce0 msgr2=0x7fcf34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.806+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf50161ce0 0x7fcf34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.806+0000 7fcf4f400640 1 -- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf34004d40 msgr2=0x7fcf34005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.806+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf34004d40 0x7fcf34005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.806+0000 7fcf4f400640 1 -- 172.21.15.19:0/3580904499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf34003520 con 0x7fcf340039b0 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.807+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340039b0 0x7fcf34003e20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fcf50072050 tx=0x7fcf34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e39b6e3a578e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.807+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40007c80 con 0x7fcf340039b0 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.807+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf40005ad0 con 0x7fcf340039b0 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.807+0000 7fcf4ea00640 1 -- 172.21.15.19:0/3580904499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40007070 con 0x7fcf340039b0 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.808+0000 7fcf56200640 1 -- 172.21.15.19:0/3580904499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340039b0 msgr2=0x7fcf34003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.808+0000 7fcf56200640 1 --2- 172.21.15.19:0/3580904499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340039b0 0x7fcf34003e20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fcf50072050 tx=0x7fcf34005830 comp rx=0 tx=0).stop 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.808+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3580904499 reap_dead start 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.808+0000 7fcf56200640 1 -- 172.21.15.19:0/3580904499 shutdown_connections 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.808+0000 7fcf56200640 1 -- 172.21.15.19:0/3580904499 >> 172.21.15.19:0/3580904499 conn(0x7fcf50073120 msgr2=0x7fcf50073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.808+0000 7fcf56200640 1 -- 172.21.15.19:0/3580904499 shutdown_connections 2024-04-25T11:34:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.808+0000 7fcf56200640 1 -- 172.21.15.19:0/3580904499 wait complete. 2024-04-25T11:34:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.811+0000 7fcf56200640 1 Processor -- start 2024-04-25T11:34:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.811+0000 7fcf56200640 1 -- start start 2024-04-25T11:34:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.812+0000 7fcf56200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf340039b0 0x7fcf501404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.812+0000 7fcf56200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf50157020 0x7fcf5014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.812+0000 7fcf56200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf50140cf0 0x7fcf50141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.812+0000 7fcf56200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50070930 con 0x7fcf50140cf0 2024-04-25T11:34:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.812+0000 7fcf56200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf500707b0 con 0x7fcf50157020 2024-04-25T11:34:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.812+0000 7fcf56200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5006d5d0 con 0x7fcf340039b0 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf50140cf0 0x7fcf50141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf50140cf0 0x7fcf50141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37910/0 (socket says 172.21.15.19:37910) 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3113256600 learned_addr learned my addr 172.21.15.19:0/3113256600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf340039b0 0x7fcf501404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf340039b0 msgr2=0x7fcf501404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf50157020 0x7fcf5014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf340039b0 0x7fcf501404c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf50157020 msgr2=0x7fcf5014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf50157020 0x7fcf5014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3113256600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5014c370 con 0x7fcf50140cf0 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf340039b0 0x7fcf501404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf50157020 0x7fcf5014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.813+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf50140cf0 0x7fcf50141100 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fcf50072e10 tx=0x7fcf4800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.814+0000 7fcf47e00640 1 -- 172.21.15.19:0/3113256600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48016620 con 0x7fcf50140cf0 2024-04-25T11:34:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.814+0000 7fcf47e00640 1 -- 172.21.15.19:0/3113256600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf4801b070 con 0x7fcf50140cf0 2024-04-25T11:34:17.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.814+0000 7fcf47e00640 1 -- 172.21.15.19:0/3113256600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48016a20 con 0x7fcf50140cf0 2024-04-25T11:34:17.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.814+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5014c870 con 0x7fcf50140cf0 2024-04-25T11:34:17.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.814+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5014ce30 con 0x7fcf50140cf0 2024-04-25T11:34:17.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.816+0000 7fcf47e00640 1 -- 172.21.15.19:0/3113256600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf48005d10 con 0x7fcf50140cf0 2024-04-25T11:34:17.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.817+0000 7fcf47e00640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf2c0744b0 0x7fcf2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.818+0000 7fcf47e00640 1 -- 172.21.15.19:0/3113256600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fcf48095a20 con 0x7fcf50140cf0 2024-04-25T11:34:17.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.818+0000 7fcf5d227740 1 Processor -- start 2024-04-25T11:34:17.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.818+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf2c0744b0 0x7fcf2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.818+0000 7fcf5d227740 1 -- start start 2024-04-25T11:34:17.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.819+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf2c0744b0 0x7fcf2c0769c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcf40032000 tx=0x7fcf40009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:17.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6a4390 0x560f1aa715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1aa47250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf5d227740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f1a957030 con 0x560f1a6836b0 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560f1a9727f0 con 0x560f1a6a4390 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560f1a92d7e0 con 0x560f1a67b4d0 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1aa47250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1aa47250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37924/0 (socket says 172.21.15.19:37924) 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf4fe00640 1 -- 172.21.15.19:0/1650488052 learned_addr learned my addr 172.21.15.19:0/1650488052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.820+0000 7fcf4f400640 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6a4390 0x560f1aa715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf46a00640 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf4fe00640 1 -- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 msgr2=0x560f1aa6df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf4fe00640 1 -- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6a4390 msgr2=0x560f1aa715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6a4390 0x560f1aa715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf4fe00640 1 -- 172.21.15.19:0/1650488052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1a939050 con 0x560f1a6836b0 2024-04-25T11:34:17.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1aa47250 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcf48002230 tx=0x7fcf4805f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f10ea5de30fa58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:17.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf3f400640 1 -- 172.21.15.19:0/1650488052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48074e60 con 0x560f1a6836b0 2024-04-25T11:34:17.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf3f400640 1 -- 172.21.15.19:0/1650488052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf480681c0 con 0x560f1a6836b0 2024-04-25T11:34:17.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.821+0000 7fcf3f400640 1 -- 172.21.15.19:0/1650488052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48068620 con 0x560f1a6836b0 2024-04-25T11:34:17.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 -- 172.21.15.19:0/1650488052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 msgr2=0x560f1aa47250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1aa47250 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcf48002230 tx=0x7fcf4805f9c0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 -- 172.21.15.19:0/1650488052 shutdown_connections 2024-04-25T11:34:17.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1aa47250 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 --2- 172.21.15.19:0/1650488052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6a4390 0x560f1aa715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 -- 172.21.15.19:0/1650488052 >> 172.21.15.19:0/1650488052 conn(0x560f1a6984e0 msgr2=0x560f1aa469e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 -- 172.21.15.19:0/1650488052 shutdown_connections 2024-04-25T11:34:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.822+0000 7fcf5d227740 1 -- 172.21.15.19:0/1650488052 wait complete. 2024-04-25T11:34:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.823+0000 7fcf5d227740 1 -- start start 2024-04-25T11:34:17.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.824+0000 7fcf5d227740 1 Processor -- start 2024-04-25T11:34:17.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.825+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.825+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa8c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.825+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6a4390 0x560f1aa8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.825+0000 7fcf5d227740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f1a9af200 con 0x560f1a6a4390 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.825+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560f1a9a9ee0 con 0x560f1a6836b0 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.825+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560f1a9b6e70 con 0x560f1a67b4d0 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.825+0000 7fcf5d227740 10 client.0 _ll_register_callbacks cb 0x560f1a9a0270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6a4390 0x560f1aa8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6a4390 0x560f1aa8c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37930/0 (socket says 172.21.15.19:37930) 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3339138289 learned_addr learned my addr 172.21.15.19:0/3339138289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa8c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 msgr2=0x560f1aa6f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 msgr2=0x560f1aa8c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa8c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 -- 172.21.15.19:0/3339138289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1a7867d0 con 0x560f1a6a4390 2024-04-25T11:34:17.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:17.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa8c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:17.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6a4390 0x560f1aa8c660 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fcf48008010 tx=0x7fcf48015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a404c6f1db49a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:17.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.826+0000 7fcf33e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:17.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf33e00640 1 -- 172.21.15.19:0/3339138289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48074e60 con 0x560f1a6a4390 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf480681c0 con 0x560f1a6a4390 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf33e00640 1 -- 172.21.15.19:0/3339138289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fcf480683e0 con 0x560f1a6a4390 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf33e00640 1 -- 172.21.15.19:0/3339138289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fcf48086050 con 0x560f1a6a4390 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf5d227740 10 client.7050 fetch_fsmap learned FSMap version 75 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x560f1a6bc3f0 con 0x560f1a6a4390 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf33e00640 1 -- 172.21.15.19:0/3339138289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf480681c0 con 0x560f1a6a4390 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf33e00640 1 -- 172.21.15.19:0/3339138289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7fcf48016bc0 con 0x560f1a6a4390 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf5d227740 10 client.7050 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf5d227740 10 client.7050 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf5d227740 20 client.7050 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.827+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1a662f60 0x560f1aa6df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.828+0000 7fcf5d227740 4 client.7050 mds_command: new command op to 5866 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:34:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.828+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 --> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f1a9b87e0 con 0x560f1a662f60 2024-04-25T11:34:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.828+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1a662f60 0x560f1aa6df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.828+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1a662f60 0x560f1aa6df00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5486edc57d9b38 server_cookie=7613d2a2c6860d7b in_seq=0 out_seq=0 2024-04-25T11:34:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.828+0000 7fcf33e00640 10 client.7050 ms_handle_connect on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.829+0000 7fcf33e00640 1 -- 172.21.15.19:0/3339138289 <== mds.0 v2:172.21.15.19:6834/853310115 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560f1a9b87e0 con 0x560f1a662f60 2024-04-25T11:34:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.829+0000 7fcf33e00640 10 client.7050 handle_command_reply: tid=0 2024-04-25T11:34:17.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.829+0000 7fcf5d227740 1 client.7050 shutdown 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6a4390 msgr2=0x560f1aa8c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6a4390 0x560f1aa8c660 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fcf48008010 tx=0x7fcf48015000 comp rx=0 tx=0).stop 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 shutdown_connections 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1a662f60 0x560f1aa6df00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa8c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6a4390 0x560f1aa8c660 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3339138289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a67b4d0 0x560f1aa6f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf33e00640 0 client.7050 ms_handle_reset on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:17.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 >> 172.21.15.19:0/3339138289 conn(0x560f1a6984e0 msgr2=0x560f1aa45d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 shutdown_connections 2024-04-25T11:34:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 1 -- 172.21.15.19:0/3339138289 wait complete. 2024-04-25T11:34:17.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.830+0000 7fcf5d227740 20 client.7050 trim_cache size 0 max 16384 2024-04-25T11:34:17.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.847+0000 7fcf5d227740 1 Processor -- start 2024-04-25T11:34:17.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.847+0000 7fcf5d227740 1 -- start start 2024-04-25T11:34:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a67b4d0 0x560f1aa9da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 0x560f1aa747e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf5d227740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50203370 con 0x560f1a67b4d0 2024-04-25T11:34:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50202350 con 0x560f1a6836b0 2024-04-25T11:34:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf50156220 con 0x560f1a6a4390 2024-04-25T11:34:17.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a67b4d0 0x560f1aa9da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a67b4d0 0x560f1aa9da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37940/0 (socket says 172.21.15.19:37940) 2024-04-25T11:34:17.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 -- 172.21.15.19:0/3631815082 learned_addr learned my addr 172.21.15.19:0/3631815082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 -- 172.21.15.19:0/3631815082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 msgr2=0x560f1aa747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3631815082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 0x560f1aa747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 -- 172.21.15.19:0/3631815082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 msgr2=0x560f1aa742a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3631815082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.849+0000 7fcf4f400640 1 -- 172.21.15.19:0/3631815082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1a67c0f0 con 0x560f1a67b4d0 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.850+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3631815082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a67b4d0 0x560f1aa9da70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fcf40001eb0 tx=0x7fcf40007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784c2fd94283e503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.850+0000 7fcf3ea00640 1 -- 172.21.15.19:0/3631815082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf400076c0 con 0x560f1a67b4d0 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.850+0000 7fcf3ea00640 1 -- 172.21.15.19:0/3631815082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf40038040 con 0x560f1a67b4d0 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.850+0000 7fcf3ea00640 1 -- 172.21.15.19:0/3631815082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4004f3d0 con 0x560f1a67b4d0 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 -- 172.21.15.19:0/3631815082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a67b4d0 msgr2=0x560f1aa9da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3631815082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a67b4d0 0x560f1aa9da70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fcf40001eb0 tx=0x7fcf40007000 comp rx=0 tx=0).stop 2024-04-25T11:34:17.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 -- 172.21.15.19:0/3631815082 shutdown_connections 2024-04-25T11:34:17.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3631815082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a6836b0 0x560f1aa742a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3631815082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 0x560f1aa747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3631815082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a67b4d0 0x560f1aa9da70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 -- 172.21.15.19:0/3631815082 >> 172.21.15.19:0/3631815082 conn(0x560f1a6984e0 msgr2=0x560f1aa8d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 -- 172.21.15.19:0/3631815082 shutdown_connections 2024-04-25T11:34:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.851+0000 7fcf5d227740 1 -- 172.21.15.19:0/3631815082 wait complete. 2024-04-25T11:34:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.852+0000 7fcf5d227740 1 -- start start 2024-04-25T11:34:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.852+0000 7fcf5d227740 1 Processor -- start 2024-04-25T11:34:17.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a67b4d0 0x560f1aa9d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1ab0b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf5d227740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 0x560f1aa8b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf5d227740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f1a9b7fd0 con 0x560f1a6836b0 2024-04-25T11:34:17.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50201d60 con 0x560f1a67b4d0 2024-04-25T11:34:17.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf5d227740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560f1aa6fc40 con 0x560f1a6a4390 2024-04-25T11:34:17.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf5d227740 10 client.0 _ll_register_callbacks cb 0x7fcf48077040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf46a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1ab0b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf46a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1ab0b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37942/0 (socket says 172.21.15.19:37942) 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf46a00640 1 -- 172.21.15.19:0/3072381485 learned_addr learned my addr 172.21.15.19:0/3072381485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.854+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a67b4d0 0x560f1aa9d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf4fe00640 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 0x560f1aa8b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf46a00640 1 -- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 msgr2=0x560f1aa8b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 0x560f1aa8b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf46a00640 1 -- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a67b4d0 msgr2=0x560f1aa9d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a67b4d0 0x560f1aa9d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf46a00640 1 -- 172.21.15.19:0/3072381485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1a972a30 con 0x560f1a6836b0 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf46a00640 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1ab0b540 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcf5006db60 tx=0x7fcf38014a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935bb94f08596088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf33e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf33e00640 1 -- 172.21.15.19:0/3072381485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3801b070 con 0x560f1a6836b0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf33e00640 1 -- 172.21.15.19:0/3072381485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fcf38004440 con 0x560f1a6836b0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.855+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf40038040 con 0x560f1a6836b0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf33e00640 1 -- 172.21.15.19:0/3072381485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fcf38026050 con 0x560f1a6836b0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 10 client.7056 fetch_fsmap learned FSMap version 75 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x560f1aa9c9f0 con 0x560f1a6836b0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf33e00640 1 -- 172.21.15.19:0/3072381485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf40038040 con 0x560f1a6836b0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf33e00640 1 -- 172.21.15.19:0/3072381485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 75) ==== 3494+0+0 (secure 0 0 0) 0x7fcf38029020 con 0x560f1a6836b0 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 10 client.7056 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T11:34:17.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 10 client.7056 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T11:34:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 20 client.7056 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1aa6fdc0 0x560f1aa8bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 4 client.7056 mds_command: new command op to 5866 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T11:34:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 --> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f1a9b2650 con 0x560f1aa6fdc0 2024-04-25T11:34:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.856+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1aa6fdc0 0x560f1aa8bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.857+0000 7fcf4f400640 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1aa6fdc0 0x560f1aa8bf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e624bd01551aea server_cookie=a5db8e0b4b29677f in_seq=0 out_seq=0 2024-04-25T11:34:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.857+0000 7fcf33e00640 10 client.7056 ms_handle_connect on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:17.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf33e00640 1 -- 172.21.15.19:0/3072381485 <== mds.0 v2:172.21.15.19:6834/853310115 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f1a9b2650 con 0x560f1aa6fdc0 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf33e00640 10 client.7056 handle_command_reply: tid=0 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 client.7056 shutdown 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 msgr2=0x560f1ab0b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1ab0b540 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcf5006db60 tx=0x7fcf38014a00 comp rx=0 tx=0).stop 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 shutdown_connections 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:6834/853310115,v1:172.21.15.19:6835/853310115] conn(0x560f1aa6fdc0 0x560f1aa8bf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560f1a6a4390 0x560f1aa8b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1a6836b0 0x560f1ab0b540 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 --2- 172.21.15.19:0/3072381485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560f1a67b4d0 0x560f1aa9d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf33e00640 0 client.7056 ms_handle_reset on v2:172.21.15.19:6834/853310115 2024-04-25T11:34:17.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.888+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 >> 172.21.15.19:0/3072381485 conn(0x560f1a6984e0 msgr2=0x560f1aaaec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:17.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.889+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 shutdown_connections 2024-04-25T11:34:17.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.889+0000 7fcf5d227740 1 -- 172.21.15.19:0/3072381485 wait complete. 2024-04-25T11:34:17.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.889+0000 7fcf5d227740 20 client.7056 trim_cache size 0 max 16384 2024-04-25T11:34:17.879 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:34:17.879 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-04-25T11:34:17.879 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T11:34:17.879 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf2c0744b0 msgr2=0x7fcf2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf2c0744b0 0x7fcf2c0769c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcf40032000 tx=0x7fcf40009030 comp rx=0 tx=0).stop 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf50140cf0 msgr2=0x7fcf50141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf50140cf0 0x7fcf50141100 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fcf50072e10 tx=0x7fcf4800f520 comp rx=0 tx=0).stop 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 shutdown_connections 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf2c0744b0 0x7fcf2c0769c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf50157020 0x7fcf5014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf50140cf0 0x7fcf50141100 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 --2- 172.21.15.19:0/3113256600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf340039b0 0x7fcf501404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 >> 172.21.15.19:0/3113256600 conn(0x7fcf50073120 msgr2=0x7fcf500b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 shutdown_connections 2024-04-25T11:34:17.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:17.890+0000 7fcf56200640 1 -- 172.21.15.19:0/3113256600 wait complete. 2024-04-25T11:34:17.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-04-25T11:34:18.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.208+0000 7f4babe00640 1 Processor -- start 2024-04-25T11:34:18.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.209+0000 7f4babe00640 1 -- start start 2024-04-25T11:34:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.211+0000 7f4babe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 0x7f4bac1655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.211+0000 7f4babe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4bac0b50b0 0x7f4bac0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.211+0000 7f4babe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac165ae0 con 0x7f4bac06d5d0 2024-04-25T11:34:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.211+0000 7f4babe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4bac14aed0 con 0x7f4bac0b50b0 2024-04-25T11:34:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.211+0000 7f4babe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac06f460 con 0x7f4bac14aac0 2024-04-25T11:34:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.212+0000 7f4ba3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4bac06d5d0 0x7f4bac06d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40236/0 (socket says 172.21.15.19:40236) 2024-04-25T11:34:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.212+0000 7f4ba3e00640 1 -- 172.21.15.19:0/1641496816 learned_addr learned my addr 172.21.15.19:0/1641496816 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.212+0000 7f4bab400640 1 --2- 172.21.15.19:0/1641496816 >> v2:172.21.15.169:3300/0 conn(0x7f4bac0b50b0 0x7f4bac0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.212+0000 7f4baaa00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 0x7f4bac1655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.212+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 831226043 0 0) 0x7f4b94001110 con 0x7f4bac06d5d0 2024-04-25T11:34:18.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.212+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b90003520 con 0x7f4bac06d5d0 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2349989717 0 0) 0x7f4b90003520 con 0x7f4bac06d5d0 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 msgr2=0x7f4bac1655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 0x7f4bac1655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 >> v2:172.21.15.169:3300/0 conn(0x7f4bac0b50b0 msgr2=0x7f4bac0b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 --2- 172.21.15.19:0/1641496816 >> v2:172.21.15.169:3300/0 conn(0x7f4bac0b50b0 0x7f4bac0b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac06f740 con 0x7f4bac06d5d0 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4b94001840 con 0x7f4bac06d5d0 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 >> v1:172.21.15.19:6789/0 conn(0x7f4bac06d5d0 legacy=0x7f4bac06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b900039b0 0x7f4b90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90004800 0x7f4b90004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 0x7f4b90005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac14aed0 con 0x7f4b900039b0 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac06f460 con 0x7f4b90004800 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9000baa0 con 0x7f4bac14aac0 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.213+0000 7f4baaa00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b900039b0 0x7f4b90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4ba3e00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90004800 0x7f4b90004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4bab400640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 0x7f4b90005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4baaa00640 1 -- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 msgr2=0x7f4b90005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4baaa00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bac14aac0 0x7f4b90005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4baaa00640 1 -- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90004800 msgr2=0x7f4b90004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4baaa00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90004800 0x7f4b90004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4baaa00640 1 -- 172.21.15.19:0/1641496816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b90003520 con 0x7f4b900039b0 2024-04-25T11:34:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.214+0000 7f4ba3e00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90004800 0x7f4b90004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.215+0000 7f4baaa00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b900039b0 0x7f4b90003e20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009a90 tx=0x7f4b90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d70e843497e0ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.215+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c036070 con 0x7f4b900039b0 2024-04-25T11:34:18.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.215+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9c031400 con 0x7f4b900039b0 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.215+0000 7f4baa000640 1 -- 172.21.15.19:0/1641496816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c009690 con 0x7f4b900039b0 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.216+0000 7f4babe00640 1 -- 172.21.15.19:0/1641496816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b900039b0 msgr2=0x7f4b90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.216+0000 7f4babe00640 1 --2- 172.21.15.19:0/1641496816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b900039b0 0x7f4b90003e20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009a90 tx=0x7f4b90005830 comp rx=0 tx=0).stop 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.216+0000 7f4bab400640 1 -- 172.21.15.19:0/1641496816 reap_dead start 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.216+0000 7f4babe00640 1 -- 172.21.15.19:0/1641496816 shutdown_connections 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.216+0000 7f4babe00640 1 -- 172.21.15.19:0/1641496816 >> 172.21.15.19:0/1641496816 conn(0x7f4bac072f40 msgr2=0x7f4bac073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.217+0000 7f4babe00640 1 -- 172.21.15.19:0/1641496816 shutdown_connections 2024-04-25T11:34:18.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.217+0000 7f4babe00640 1 -- 172.21.15.19:0/1641496816 wait complete. 2024-04-25T11:34:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.219+0000 7f4babe00640 1 Processor -- start 2024-04-25T11:34:18.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.219+0000 7f4babe00640 1 -- start start 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4babe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b900039b0 0x7f4bac161a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4babe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bac1edf50 0x7f4bac1ee360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4babe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bac14c9e0 0x7f4bac14cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4babe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac06ffe0 con 0x7f4bac14c9e0 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4babe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac06fd00 con 0x7f4bac1edf50 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4babe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac0b4a90 con 0x7f4b900039b0 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4bab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bac14c9e0 0x7f4bac14cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4bab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bac14c9e0 0x7f4bac14cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37966/0 (socket says 172.21.15.19:37966) 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4bab400640 1 -- 172.21.15.19:0/4658421 learned_addr learned my addr 172.21.15.19:0/4658421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4ba3e00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bac1edf50 0x7f4bac1ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4baaa00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b900039b0 0x7f4bac161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.221+0000 7f4bab400640 1 -- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b900039b0 msgr2=0x7f4bac161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4bab400640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b900039b0 0x7f4bac161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4bab400640 1 -- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bac1edf50 msgr2=0x7f4bac1ee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4bab400640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bac1edf50 0x7f4bac1ee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4bab400640 1 -- 172.21.15.19:0/4658421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac14d330 con 0x7f4bac14c9e0 2024-04-25T11:34:18.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4bab400640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bac14c9e0 0x7f4bac14cdf0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4b90004780 tx=0x7f4ba400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:18.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4ba2a00640 1 -- 172.21.15.19:0/4658421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba401c070 con 0x7f4bac14c9e0 2024-04-25T11:34:18.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4ba2a00640 1 -- 172.21.15.19:0/4658421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba4017400 con 0x7f4bac14c9e0 2024-04-25T11:34:18.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4ba2a00640 1 -- 172.21.15.19:0/4658421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40177f0 con 0x7f4bac14c9e0 2024-04-25T11:34:18.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac1eec80 con 0x7f4bac14c9e0 2024-04-25T11:34:18.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.222+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac1ef240 con 0x7f4bac14c9e0 2024-04-25T11:34:18.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.224+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac076c60 con 0x7f4bac14c9e0 2024-04-25T11:34:18.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.225+0000 7f4ba2a00640 1 -- 172.21.15.19:0/4658421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ba401f070 con 0x7f4bac14c9e0 2024-04-25T11:34:18.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.226+0000 7f4ba2a00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b880744b0 0x7f4b880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.226+0000 7f4ba2a00640 1 -- 172.21.15.19:0/4658421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f4ba40952d0 con 0x7f4bac14c9e0 2024-04-25T11:34:18.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.227+0000 7f4baaa00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b880744b0 0x7f4b880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.228+0000 7f4baaa00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b880744b0 0x7f4b880769c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c033000 tx=0x7f4b9c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:18.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.231+0000 7f4ba2a00640 1 -- 172.21.15.19:0/4658421 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bac14d330 con 0x7f4bac14c9e0 2024-04-25T11:34:18.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.440+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f4bac070e70 con 0x7f4bac14c9e0 2024-04-25T11:34:18.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.445+0000 7f4ba2a00640 1 -- 172.21.15.19:0/4658421 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v23)=0 v23) ==== 138+0+0 (secure 0 0 0) 0x7f4ba4012070 con 0x7f4bac14c9e0 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b880744b0 msgr2=0x7f4b880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b880744b0 0x7f4b880769c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c033000 tx=0x7f4b9c008030 comp rx=0 tx=0).stop 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bac14c9e0 msgr2=0x7f4bac14cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bac14c9e0 0x7f4bac14cdf0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4b90004780 tx=0x7f4ba400f4a0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 shutdown_connections 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b880744b0 0x7f4b880769c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bac1edf50 0x7f4bac1ee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bac14c9e0 0x7f4bac14cdf0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 --2- 172.21.15.19:0/4658421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b900039b0 0x7f4bac161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 >> 172.21.15.19:0/4658421 conn(0x7f4bac072f40 msgr2=0x7f4bac0b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 shutdown_connections 2024-04-25T11:34:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:18.448+0000 7f4babe00640 1 -- 172.21.15.19:0/4658421 wait complete. 2024-04-25T11:34:23.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T11:34:23.472 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'timeout 60 mv a/b a/z') 2024-04-25T11:34:23.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T11:34:23.582 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:34:23.594+0000 7f71d0c00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7026:47 nref=7 cr=0x5628730dbc00)) (dversion lock w=1 last_client=7026) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562873fe1680] 2024-04-25T11:34:23.583 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:34:23.594+0000 7f71d0c00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7026:47 nref=7 cr=0x5628730dbc00)) (dversion lock w=1 last_client=7026) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562873fe1680] 2024-04-25T11:34:23.584 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f71d0c00640 time 2024-04-25T11:34:23.595336+0000 2024-04-25T11:34:23.584 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:34:23.584 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.584 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f71d7c4bfcd] 2024-04-25T11:34:23.584 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.584 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.584 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.585 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:34:23.596+0000 7f71d0c00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f71d0c00640 time 2024-04-25T11:34:23.595336+0000 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f71d7c4bfcd] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.586 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.587 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr:*** Caught signal (Aborted) ** 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f71d0c00640 thread_name:ms_dispatch 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f71d723e6f0] 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f71d728b94c] 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:34:23.588 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f71d7c4c13f] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.589 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:34:23.598+0000 7f71d0c00640 -1 *** Caught signal (Aborted) ** 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f71d0c00640 thread_name:ms_dispatch 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f71d723e6f0] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f71d728b94c] 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:34:23.590 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f71d7c4c13f] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.591 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.592 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.592 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.592 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.592 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.592 INFO:tasks.ceph.mds.c.smithi019.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T11:34:23.592 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.607 INFO:tasks.ceph.mds.c.smithi019.stderr: -4895> 2024-04-25T11:34:11.170+0000 7f71cda00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:34:23.624 INFO:tasks.ceph.mds.c.smithi019.stderr: -8> 2024-04-25T11:34:23.594+0000 7f71d0c00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7026:47 nref=7 cr=0x5628730dbc00)) (dversion lock w=1 last_client=7026) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562873fe1680] 2024-04-25T11:34:23.624 INFO:tasks.ceph.mds.c.smithi019.stderr: -6> 2024-04-25T11:34:23.594+0000 7f71d0c00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7026:47 nref=7 cr=0x5628730dbc00)) (dversion lock w=1 last_client=7026) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562873fe1680] 2024-04-25T11:34:23.624 INFO:tasks.ceph.mds.c.smithi019.stderr: -1> 2024-04-25T11:34:23.596+0000 7f71d0c00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f71d0c00640 time 2024-04-25T11:34:23.595336+0000 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f71d7c4bfcd] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.625 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.626 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 0> 2024-04-25T11:34:23.598+0000 7f71d0c00640 -1 *** Caught signal (Aborted) ** 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f71d0c00640 thread_name:ms_dispatch 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f71d723e6f0] 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f71d728b94c] 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f71d7c4c13f] 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.627 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.628 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.629 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.629 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.629 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.629 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.629 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.629 INFO:tasks.ceph.mds.c.smithi019.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T11:34:23.629 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.631 INFO:tasks.ceph.mds.c.smithi019.stderr: -4895> 2024-04-25T11:34:11.170+0000 7f71cda00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:34:23.640 INFO:tasks.ceph.mds.c.smithi019.stderr: -8> 2024-04-25T11:34:23.594+0000 7f71d0c00640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7026:47 nref=7 cr=0x5628730dbc00)) (dversion lock w=1 last_client=7026) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562873fe1680] 2024-04-25T11:34:23.640 INFO:tasks.ceph.mds.c.smithi019.stderr: -6> 2024-04-25T11:34:23.594+0000 7f71d0c00640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7026:47 nref=7 cr=0x5628730dbc00)) (dversion lock w=1 last_client=7026) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562873fe1680] 2024-04-25T11:34:23.640 INFO:tasks.ceph.mds.c.smithi019.stderr: -1> 2024-04-25T11:34:23.596+0000 7f71d0c00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f71d0c00640 time 2024-04-25T11:34:23.595336+0000 2024-04-25T11:34:23.640 INFO:tasks.ceph.mds.c.smithi019.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3291-g994e2f20/rpm/el9/BUILD/ceph-19.0.0-3291-g994e2f20/src/mds/MDSRank.cc: 951: ceph_abort_msg("abort() called") 2024-04-25T11:34:23.640 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.640 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xcf) [0x7f71d7c4bfcd] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.641 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 0> 2024-04-25T11:34:23.598+0000 7f71d0c00640 -1 *** Caught signal (Aborted) ** 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: in thread 7f71d0c00640 thread_name:ms_dispatch 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:34:23.642 INFO:tasks.ceph.mds.c.smithi019.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f71d723e6f0] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f71d728b94c] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 3: raise() 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 4: abort() 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x241) [0x7f71d7c4c13f] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x5f) [0x5628439aa937] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 7: (CDentry::check_corruption(bool)+0x530) [0x562843c6fdf6] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 8: ceph-mds(+0x17e876) [0x562843a0f876] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 9: ceph-mds(+0x17ec76) [0x562843a0fc76] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 10: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x249b) [0x562843ab0dbd] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 11: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x2fab) [0x562843ab4669] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 12: (Server::dispatch_client_request(boost::intrusive_ptr const&)+0xeb1) [0x562843a7754f] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 13: (Server::handle_client_request(boost::intrusive_ptr const&)+0xa2a) [0x562843a78258] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 14: (Server::dispatch(boost::intrusive_ptr const&)+0xc0) [0x562843a62c6e] 2024-04-25T11:34:23.643 INFO:tasks.ceph.mds.c.smithi019.stderr: 15: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x5ac) [0x5628439b50c6] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 16: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5628439b9a21] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 17: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5628439ba348] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 18: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x2c4) [0x56284399c4f2] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 19: /usr/lib64/ceph/libceph-common.so.2(+0x394bd5) [0x7f71d7d94bd5] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 20: (DispatchQueue::entry()+0x837) [0x7f71d7d9573f] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 21: /usr/lib64/ceph/libceph-common.so.2(+0x474c67) [0x7f71d7e74c67] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 22: (Thread::entry_wrapper()+0x43) [0x7f71d7c2487f] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 23: (Thread::_entry_func(void*)+0xd) [0x7f71d7c2489b] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 24: /lib64/libc.so.6(+0x89c02) [0x7f71d7289c02] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 25: /lib64/libc.so.6(+0x10ec40) [0x7f71d730ec40] 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-25T11:34:23.644 INFO:tasks.ceph.mds.c.smithi019.stderr: 2024-04-25T11:34:23.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.774+0000 7f9c62200640 1 Processor -- start 2024-04-25T11:34:23.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.774+0000 7f9c62200640 1 -- start start 2024-04-25T11:34:23.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.776+0000 7f9c62200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c073960 0x7f9c5c0d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.776+0000 7f9c62200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c5c06f180 0x7f9c5c0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.776+0000 7f9c62200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c06c760 con 0x7f9c5c161430 2024-04-25T11:34:23.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.776+0000 7f9c62200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5c06d090 con 0x7f9c5c06f180 2024-04-25T11:34:23.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.776+0000 7f9c62200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c06ca40 con 0x7f9c5c073960 2024-04-25T11:34:23.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.777+0000 7f9c5b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9c5c161430 0x7f9c5c073250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40250/0 (socket says 172.21.15.19:40250) 2024-04-25T11:34:23.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.777+0000 7f9c5b400640 1 -- 172.21.15.19:0/3505970411 learned_addr learned my addr 172.21.15.19:0/3505970411 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.777+0000 7f9c5be00640 1 --2- 172.21.15.19:0/3505970411 >> v2:172.21.15.169:3300/0 conn(0x7f9c5c06f180 0x7f9c5c0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.777+0000 7f9c5aa00640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c073960 0x7f9c5c0d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.777+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883635501 0 0) 0x7f9c5c06c760 con 0x7f9c5c161430 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.777+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c40003520 con 0x7f9c5c161430 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3248483443 0 0) 0x7f9c40003520 con 0x7f9c5c161430 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c073960 msgr2=0x7f9c5c0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c073960 0x7f9c5c0d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 >> v2:172.21.15.169:3300/0 conn(0x7f9c5c06f180 msgr2=0x7f9c5c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 --2- 172.21.15.19:0/3505970411 >> v2:172.21.15.169:3300/0 conn(0x7f9c5c06f180 0x7f9c5c0cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c06cd20 con 0x7f9c5c161430 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9c4c001670 con 0x7f9c5c161430 2024-04-25T11:34:23.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 >> v1:172.21.15.19:6789/0 conn(0x7f9c5c161430 legacy=0x7f9c5c073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c40004800 0x7f9c40004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c40005150 0x7f9c4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c06d090 con 0x7f9c400039b0 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c06ca40 con 0x7f9c40005150 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4000bf00 con 0x7f9c40004800 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.778+0000 7f9c5b400640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.779+0000 7f9c5be00640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c40005150 0x7f9c4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.779+0000 7f9c5aa00640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c40004800 0x7f9c40004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.779+0000 7f9c5b400640 1 -- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c40004800 msgr2=0x7f9c40004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.779+0000 7f9c5b400640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c40004800 0x7f9c40004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.779+0000 7f9c5b400640 1 -- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c40005150 msgr2=0x7f9c4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.779+0000 7f9c5b400640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c40005150 0x7f9c4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.779+0000 7f9c5b400640 1 -- 172.21.15.19:0/3505970411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c40003520 con 0x7f9c400039b0 2024-04-25T11:34:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.780+0000 7f9c5b400640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c40003e20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c240 tx=0x7f9c40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0d2118a9beddd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.780+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c02e070 con 0x7f9c400039b0 2024-04-25T11:34:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.780+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c005af0 con 0x7f9c400039b0 2024-04-25T11:34:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.781+0000 7f9c5a000640 1 -- 172.21.15.19:0/3505970411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c007d30 con 0x7f9c400039b0 2024-04-25T11:34:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.782+0000 7f9c62200640 1 -- 172.21.15.19:0/3505970411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 msgr2=0x7f9c40003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:23.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.782+0000 7f9c62200640 1 --2- 172.21.15.19:0/3505970411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c40003e20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c240 tx=0x7f9c40005830 comp rx=0 tx=0).stop 2024-04-25T11:34:23.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.782+0000 7f9c5be00640 1 -- 172.21.15.19:0/3505970411 reap_dead start 2024-04-25T11:34:23.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.782+0000 7f9c62200640 1 -- 172.21.15.19:0/3505970411 shutdown_connections 2024-04-25T11:34:23.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.782+0000 7f9c62200640 1 -- 172.21.15.19:0/3505970411 >> 172.21.15.19:0/3505970411 conn(0x7f9c5c0d1e60 msgr2=0x7f9c5c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:23.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.782+0000 7f9c62200640 1 -- 172.21.15.19:0/3505970411 shutdown_connections 2024-04-25T11:34:23.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.782+0000 7f9c62200640 1 -- 172.21.15.19:0/3505970411 wait complete. 2024-04-25T11:34:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.786+0000 7f9c62200640 1 Processor -- start 2024-04-25T11:34:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.787+0000 7f9c62200640 1 -- start start 2024-04-25T11:34:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.789+0000 7f9c62200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c5c0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.789+0000 7f9c62200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0aef30 0x7f9c5c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.789+0000 7f9c62200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c1585f0 0x7f9c5c1f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.789+0000 7f9c62200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c070fa0 con 0x7f9c400039b0 2024-04-25T11:34:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.789+0000 7f9c62200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c06d520 con 0x7f9c5c1585f0 2024-04-25T11:34:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.789+0000 7f9c62200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c071120 con 0x7f9c5c0aef30 2024-04-25T11:34:23.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c5c0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c5c0b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43422/0 (socket says 172.21.15.19:43422) 2024-04-25T11:34:23.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 -- 172.21.15.19:0/1499479439 learned_addr learned my addr 172.21.15.19:0/1499479439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:23.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5aa00640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0aef30 0x7f9c5c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5be00640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c1585f0 0x7f9c5c1f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 -- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0aef30 msgr2=0x7f9c5c14e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c5c0aef30 0x7f9c5c14e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 -- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c1585f0 msgr2=0x7f9c5c1f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c5c1585f0 0x7f9c5c1f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.790+0000 7f9c5b400640 1 -- 172.21.15.19:0/1499479439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c0af710 con 0x7f9c400039b0 2024-04-25T11:34:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.791+0000 7f9c5b400640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c400039b0 0x7f9c5c0b07b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c034040 tx=0x7f9c5c06cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.791+0000 7f9c53400640 1 -- 172.21.15.19:0/1499479439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c042070 con 0x7f9c400039b0 2024-04-25T11:34:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.791+0000 7f9c53400640 1 -- 172.21.15.19:0/1499479439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c007480 con 0x7f9c400039b0 2024-04-25T11:34:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.791+0000 7f9c53400640 1 -- 172.21.15.19:0/1499479439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c001c70 con 0x7f9c400039b0 2024-04-25T11:34:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.791+0000 7f9c62200640 1 -- 172.21.15.19:0/1499479439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c0af9f0 con 0x7f9c400039b0 2024-04-25T11:34:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.791+0000 7f9c62200640 1 -- 172.21.15.19:0/1499479439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c0affb0 con 0x7f9c400039b0 2024-04-25T11:34:23.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.792+0000 7f9c50c00640 1 -- 172.21.15.19:0/1499479439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9c30005270 con 0x7f9c400039b0 2024-04-25T11:34:23.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.794+0000 7f9c53400640 1 -- 172.21.15.19:0/1499479439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c4c045070 con 0x7f9c400039b0 2024-04-25T11:34:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.796+0000 7f9c53400640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c3c0744b0 0x7f9c3c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:23.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.796+0000 7f9c53400640 1 -- 172.21.15.19:0/1499479439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f9c4c0b6030 con 0x7f9c400039b0 2024-04-25T11:34:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.796+0000 7f9c53400640 1 -- 172.21.15.19:0/1499479439 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f9c4c0b86b0 con 0x7f9c400039b0 2024-04-25T11:34:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.796+0000 7f9c5aa00640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c3c0744b0 0x7f9c3c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.797+0000 7f9c50c00640 1 -- 172.21.15.19:0/1499479439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f9c30005550 con 0x7f9c400039b0 2024-04-25T11:34:23.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.797+0000 7f9c53400640 1 -- 172.21.15.19:0/1499479439 <== mon.0 v2:172.21.15.19:3300/0 7 ==== log(1 entries from seq 1 at 2024-04-25T11:34:23.595287+0000) ==== 468+0+0 (secure 0 0 0) 0x7f9c4c0b4670 con 0x7f9c400039b0 2024-04-25T11:34:23.785 INFO:teuthology.orchestra.run.smithi019.stdout: cluster: 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: id: 23a56573-5965-4bbb-bd60-741a44babbed 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: health: HEALTH_OK 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: services: 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: mon: 3 daemons, quorum a,b,c (age 7m) 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: mgr: x(active, since 7m), standbys: y 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: osd: 8 osds: 8 up (since 7m), 8 in (since 7m) 2024-04-25T11:34:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: data: 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: volumes: 1/1 healthy 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: pools: 3 pools, 97 pgs 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: objects: 29 objects, 596 KiB 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: usage: 252 MiB used, 720 GiB / 720 GiB avail 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: pgs: 97 active+clean 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: io: 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: client: 0 B/s rd, 2.7 KiB/s wr, 0 op/s rd, 3 op/s wr 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:23.787 INFO:teuthology.orchestra.run.smithi019.stdout:2024-04-25T11:34:23.595287+0000 mds.c [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.7026:47 nref=7 cr=0x5628730dbc00)) (dversion lock w=1 last_client=7026) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x562873fe1680] 2024-04-25T11:34:23.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:23.798+0000 7f9c5aa00640 1 --2- 172.21.15.19:0/1499479439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c3c0744b0 0x7f9c3c0769c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c1585a0 tx=0x7f9c48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:23.798 INFO:teuthology.orchestra.run.smithi019.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T11:34:23.798 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T11:34:23.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:24.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.061+0000 7fc7b1e00640 1 Processor -- start 2024-04-25T11:34:24.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.061+0000 7fc7b1e00640 1 -- start start 2024-04-25T11:34:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7b1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7b1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc7ac0cfcf0 0x7fc7ac06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7b1e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7ac15f5f0 con 0x7fc7ac070370 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7b1e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ac0708d0 con 0x7fc7ac0cfcf0 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7b1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac15f160 con 0x7fc7ac1443c0 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7ab400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc7ac0cfcf0 0x7fc7ac06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7ab400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc7ac0cfcf0 0x7fc7ac06c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43424/0 (socket says 172.21.15.19:43424) 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7ab400640 1 -- 172.21.15.19:0/2005624750 learned_addr learned my addr 172.21.15.19:0/2005624750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7aaa00640 1 --2- 172.21.15.19:0/2005624750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7ab400640 1 -- 172.21.15.19:0/2005624750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 msgr2=0x7fc7ac06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.064+0000 7fc7ab400640 1 --2- 172.21.15.19:0/2005624750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac06fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.065+0000 7fc7ab400640 1 -- 172.21.15.19:0/2005624750 >> v1:172.21.15.19:6789/0 conn(0x7fc7ac070370 legacy=0x7fc7ac0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.065+0000 7fc7ab400640 1 -- 172.21.15.19:0/2005624750 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac15e830 con 0x7fc7ac0cfcf0 2024-04-25T11:34:24.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.066+0000 7fc7ab400640 1 --2- 172.21.15.19:0/2005624750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0cfcf0 0x7fc7ac06c5c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009e50 tx=0x7fc7a402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf1730715ab38c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:24.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.066+0000 7fc7a9600640 1 -- 172.21.15.19:0/2005624750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4036070 con 0x7fc7ac0cfcf0 2024-04-25T11:34:24.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.066+0000 7fc7a9600640 1 -- 172.21.15.19:0/2005624750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a4031600 con 0x7fc7ac0cfcf0 2024-04-25T11:34:24.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.066+0000 7fc7a9600640 1 -- 172.21.15.19:0/2005624750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40319d0 con 0x7fc7ac0cfcf0 2024-04-25T11:34:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2005624750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0cfcf0 msgr2=0x7fc7ac06c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2005624750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0cfcf0 0x7fc7ac06c5c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009e50 tx=0x7fc7a402a710 comp rx=0 tx=0).stop 2024-04-25T11:34:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2005624750 shutdown_connections 2024-04-25T11:34:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2005624750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0cfcf0 0x7fc7ac06c5c0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2005624750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac06fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2005624750 >> 172.21.15.19:0/2005624750 conn(0x7fc7ac0d1ae0 msgr2=0x7fc7ac0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2005624750 shutdown_connections 2024-04-25T11:34:24.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.067+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2005624750 wait complete. 2024-04-25T11:34:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.069+0000 7fc7b1e00640 1 Processor -- start 2024-04-25T11:34:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.069+0000 7fc7b1e00640 1 -- start start 2024-04-25T11:34:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7b1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac070370 0x7fc7ac136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7b1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac0aa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7b1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0b05a0 0x7fc7ac06cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7b1e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac1447d0 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7b1e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac15edf0 con 0x7fc7ac070370 2024-04-25T11:34:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7b1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac162b00 con 0x7fc7ac1443c0 2024-04-25T11:34:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7ab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0b05a0 0x7fc7ac06cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7ab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0b05a0 0x7fc7ac06cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43430/0 (socket says 172.21.15.19:43430) 2024-04-25T11:34:24.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.071+0000 7fc7ab400640 1 -- 172.21.15.19:0/2393208236 learned_addr learned my addr 172.21.15.19:0/2393208236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:24.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7aa000640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac0aa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7ab400640 1 -- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 msgr2=0x7fc7ac0aa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7ab400640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac0aa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7ab400640 1 -- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac070370 msgr2=0x7fc7ac136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7ab400640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac070370 0x7fc7ac136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7ab400640 1 -- 172.21.15.19:0/2393208236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac157ee0 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7aa000640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac0aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7ab400640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0b05a0 0x7fc7ac06cd80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4034040 tx=0x7fc7a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7a3400640 1 -- 172.21.15.19:0/2393208236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4036050 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.072+0000 7fc7a3400640 1 -- 172.21.15.19:0/2393208236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a4042070 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.073+0000 7fc7a3400640 1 -- 172.21.15.19:0/2393208236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4009510 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.073+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1581c0 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.073+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac158780 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.074+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac137190 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.075+0000 7fc7a3400640 1 -- 172.21.15.19:0/2393208236 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc7a4007bd0 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.076+0000 7fc7a3400640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc78c0744e0 0x7fc78c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.077+0000 7fc7a3400640 1 -- 172.21.15.19:0/2393208236 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fc7a40b6160 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.077+0000 7fc7aaa00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc78c0744e0 0x7fc78c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.079+0000 7fc7aaa00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc78c0744e0 0x7fc78c0769f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006fd0 tx=0x7fc79c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:24.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.082+0000 7fc7a3400640 1 -- 172.21.15.19:0/2393208236 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7ac137190 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.333+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7ac06f950 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.334+0000 7fc7a3400640 1 -- 172.21.15.19:0/2393208236 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5186 (secure 0 0 0) 0x7fc7a4080850 con 0x7fc7ac0b05a0 2024-04-25T11:34:24.323 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:24.323 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:16.597206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5866":{"gid":5866,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/853310115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":853310115},{"type":"v1","addr":"172.21.15.19:6835","nonce":853310115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5866,"qdb_cluster":[5866]},"id":5}]} 2024-04-25T11:34:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 75 2024-04-25T11:34:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc78c0744e0 msgr2=0x7fc78c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc78c0744e0 0x7fc78c0769f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006fd0 tx=0x7fc79c008040 comp rx=0 tx=0).stop 2024-04-25T11:34:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0b05a0 msgr2=0x7fc7ac06cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0b05a0 0x7fc7ac06cd80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4034040 tx=0x7fc7a4004a40 comp rx=0 tx=0).stop 2024-04-25T11:34:24.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 shutdown_connections 2024-04-25T11:34:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc78c0744e0 0x7fc78c0769f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac0b05a0 0x7fc7ac06cd80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7ac1443c0 0x7fc7ac0aa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 --2- 172.21.15.19:0/2393208236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac070370 0x7fc7ac136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 >> 172.21.15.19:0/2393208236 conn(0x7fc7ac0d1ae0 msgr2=0x7fc7ac0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.337+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 shutdown_connections 2024-04-25T11:34:24.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:24.338+0000 7fc7b1e00640 1 -- 172.21.15.19:0/2393208236 wait complete. 2024-04-25T11:34:24.353 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T11:34:24.379 INFO:tasks.ceph.mds.c.smithi019.stderr:daemon-helper: command crashed with signal 6 2024-04-25T11:34:28.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T11:34:29.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.661+0000 7ff803200640 1 Processor -- start 2024-04-25T11:34:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.661+0000 7ff803200640 1 -- start start 2024-04-25T11:34:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.663+0000 7ff803200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040b32d0 0x7ff8040b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.663+0000 7ff803200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff8040b3c20 0x7ff8040b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.664+0000 7ff803200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8040b2c00 con 0x7ff8040d0f10 2024-04-25T11:34:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.664+0000 7ff803200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff8040b9520 con 0x7ff8040b3c20 2024-04-25T11:34:29.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.664+0000 7ff803200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff80409cce0 con 0x7ff8040b32d0 2024-04-25T11:34:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.664+0000 7ff801e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff8040d0f10 0x7ff80406d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:44794/0 (socket says 172.21.15.19:44794) 2024-04-25T11:34:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.664+0000 7ff7f9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040b32d0 0x7ff8040b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.664+0000 7ff801e00640 1 -- 172.21.15.19:0/3064312429 learned_addr learned my addr 172.21.15.19:0/3064312429 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.665+0000 7ff802800640 1 --2- 172.21.15.19:0/3064312429 >> v2:172.21.15.169:3300/0 conn(0x7ff8040b3c20 0x7ff8040b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.665+0000 7ff801400640 1 -- 172.21.15.19:0/3064312429 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481248151 0 0) 0x7ff8040b2c00 con 0x7ff8040d0f10 2024-04-25T11:34:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.665+0000 7ff801400640 1 -- 172.21.15.19:0/3064312429 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7e8003520 con 0x7ff8040d0f10 2024-04-25T11:34:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.665+0000 7ff802800640 1 -- 172.21.15.19:0/3064312429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040b32d0 msgr2=0x7ff8040b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.665+0000 7ff802800640 1 --2- 172.21.15.19:0/3064312429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040b32d0 0x7ff8040b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.665+0000 7ff802800640 1 -- 172.21.15.19:0/3064312429 >> v1:172.21.15.169:6789/0 conn(0x7ff8040d0f10 legacy=0x7ff80406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.665+0000 7ff802800640 1 -- 172.21.15.19:0/3064312429 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8040b9240 con 0x7ff8040b3c20 2024-04-25T11:34:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.667+0000 7ff802800640 1 --2- 172.21.15.19:0/3064312429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b3c20 0x7ff8040b4030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009d90 tx=0x7ff7fc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dce9602147b66ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.667+0000 7ff801400640 1 -- 172.21.15.19:0/3064312429 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007740 con 0x7ff8040b3c20 2024-04-25T11:34:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.667+0000 7ff801400640 1 -- 172.21.15.19:0/3064312429 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7fc0078e0 con 0x7ff8040b3c20 2024-04-25T11:34:29.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.667+0000 7ff801400640 1 -- 172.21.15.19:0/3064312429 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007cb0 con 0x7ff8040b3c20 2024-04-25T11:34:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 -- 172.21.15.19:0/3064312429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b3c20 msgr2=0x7ff8040b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 --2- 172.21.15.19:0/3064312429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b3c20 0x7ff8040b4030 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009d90 tx=0x7ff7fc02a660 comp rx=0 tx=0).stop 2024-04-25T11:34:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 -- 172.21.15.19:0/3064312429 shutdown_connections 2024-04-25T11:34:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 --2- 172.21.15.19:0/3064312429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b3c20 0x7ff8040b4030 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 --2- 172.21.15.19:0/3064312429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040b32d0 0x7ff8040b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 -- 172.21.15.19:0/3064312429 >> 172.21.15.19:0/3064312429 conn(0x7ff8040d03f0 msgr2=0x7ff8040d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 -- 172.21.15.19:0/3064312429 shutdown_connections 2024-04-25T11:34:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.668+0000 7ff803200640 1 -- 172.21.15.19:0/3064312429 wait complete. 2024-04-25T11:34:29.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.671+0000 7ff803200640 1 Processor -- start 2024-04-25T11:34:29.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.671+0000 7ff803200640 1 -- start start 2024-04-25T11:34:29.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.672+0000 7ff803200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b32d0 0x7ff8040c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.672+0000 7ff803200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8040b3c20 0x7ff804159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.672+0000 7ff803200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040d0f10 0x7ff8040a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.672+0000 7ff803200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8040b8c00 con 0x7ff8040b3c20 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.672+0000 7ff803200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff80409d170 con 0x7ff8040b32d0 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.672+0000 7ff803200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8040b8d80 con 0x7ff8040d0f10 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8040b3c20 0x7ff804159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8040b3c20 0x7ff804159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43444/0 (socket says 172.21.15.19:43444) 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 -- 172.21.15.19:0/4099828339 learned_addr learned my addr 172.21.15.19:0/4099828339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff801e00640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b32d0 0x7ff8040c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff802800640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040d0f10 0x7ff8040a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 -- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040d0f10 msgr2=0x7ff8040a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040d0f10 0x7ff8040a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 -- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b32d0 msgr2=0x7ff8040c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b32d0 0x7ff8040c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.673+0000 7ff7f9400640 1 -- 172.21.15.19:0/4099828339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8040c9440 con 0x7ff8040b3c20 2024-04-25T11:34:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.674+0000 7ff7f9400640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8040b3c20 0x7ff804159cb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e870 tx=0x7ff7ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.674+0000 7ff7faa00640 1 -- 172.21.15.19:0/4099828339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec017070 con 0x7ff8040b3c20 2024-04-25T11:34:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.674+0000 7ff7faa00640 1 -- 172.21.15.19:0/4099828339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec00c940 con 0x7ff8040b3c20 2024-04-25T11:34:29.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.674+0000 7ff7faa00640 1 -- 172.21.15.19:0/4099828339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00cd30 con 0x7ff8040b3c20 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.674+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80415a1f0 con 0x7ff8040b3c20 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.674+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8040c9c30 con 0x7ff8040b3c20 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.675+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8040d48e0 con 0x7ff8040b3c20 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.677+0000 7ff7faa00640 1 -- 172.21.15.19:0/4099828339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff7ec01a070 con 0x7ff8040b3c20 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.678+0000 7ff7faa00640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7e00744b0 0x7ff7e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.678+0000 7ff7faa00640 1 -- 172.21.15.19:0/4099828339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7ff7ec0950f0 con 0x7ff8040b3c20 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.678+0000 7ff801e00640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7e00744b0 0x7ff7e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.680+0000 7ff801e00640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7e00744b0 0x7ff7e00769c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40058d0 tx=0x7ff7f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.683+0000 7ff7faa00640 1 -- 172.21.15.19:0/4099828339 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8040d48e0 con 0x7ff8040b3c20 2024-04-25T11:34:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.928+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff80409ce90 con 0x7ff8040b3c20 2024-04-25T11:34:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.929+0000 7ff7faa00640 1 -- 172.21.15.19:0/4099828339 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5186 (secure 0 0 0) 0x7ff80409ce90 con 0x7ff8040b3c20 2024-04-25T11:34:29.919 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:29.919 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:16.597206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5866":{"gid":5866,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/853310115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":853310115},{"type":"v1","addr":"172.21.15.19:6835","nonce":853310115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5866,"qdb_cluster":[5866]},"id":5}]} 2024-04-25T11:34:29.919 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 75 2024-04-25T11:34:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7e00744b0 msgr2=0x7ff7e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7e00744b0 0x7ff7e00769c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40058d0 tx=0x7ff7f4009040 comp rx=0 tx=0).stop 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8040b3c20 msgr2=0x7ff804159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8040b3c20 0x7ff804159cb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e870 tx=0x7ff7ec00ed30 comp rx=0 tx=0).stop 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 shutdown_connections 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7e00744b0 0x7ff7e00769c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8040d0f10 0x7ff8040a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8040b3c20 0x7ff804159cb0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 --2- 172.21.15.19:0/4099828339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8040b32d0 0x7ff8040c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 >> 172.21.15.19:0/4099828339 conn(0x7ff8040d03f0 msgr2=0x7ff804072590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 shutdown_connections 2024-04-25T11:34:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:29.932+0000 7ff803200640 1 -- 172.21.15.19:0/4099828339 wait complete. 2024-04-25T11:34:29.947 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T11:34:34.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:34:34.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:34:34.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:35.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-04-25T11:34:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.252+0000 7f7433200640 1 Processor -- start 2024-04-25T11:34:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.252+0000 7f7433200640 1 -- start start 2024-04-25T11:34:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.254+0000 7f7433200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0b0800 0x7f742c070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.254+0000 7f7433200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f742c0707d0 0x7f742c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:35.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.254+0000 7f7433200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f742c070be0 con 0x7f742c0cf9e0 2024-04-25T11:34:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.255+0000 7f7433200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f742c0cffc0 con 0x7f742c0707d0 2024-04-25T11:34:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.255+0000 7f7433200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f742c0d02a0 con 0x7f742c0b0800 2024-04-25T11:34:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.255+0000 7f7432800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f742c0707d0 0x7f742c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.255+0000 7f7431e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f742c0cf9e0 0x7f742c161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35136/0 (socket says 172.21.15.19:35136) 2024-04-25T11:34:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.255+0000 7f7431e00640 1 -- 172.21.15.19:0/307561217 learned_addr learned my addr 172.21.15.19:0/307561217 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:35.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.255+0000 7f7431400640 1 --2- 172.21.15.19:0/307561217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0b0800 0x7f742c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.256+0000 7f7432800640 1 -- 172.21.15.19:0/307561217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0b0800 msgr2=0x7f742c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.256+0000 7f7430a00640 1 -- 172.21.15.19:0/307561217 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877916413 0 0) 0x7f7420001110 con 0x7f742c0cf9e0 2024-04-25T11:34:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.256+0000 7f7432800640 1 --2- 172.21.15.19:0/307561217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0b0800 0x7f742c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.256+0000 7f7432800640 1 -- 172.21.15.19:0/307561217 >> v1:172.21.15.19:6789/0 conn(0x7f742c0cf9e0 legacy=0x7f742c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.256+0000 7f7432800640 1 -- 172.21.15.19:0/307561217 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c0d0580 con 0x7f742c0707d0 2024-04-25T11:34:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.257+0000 7f7432800640 1 --2- 172.21.15.19:0/307561217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c1569b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7428009d90 tx=0x7f742802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd651ec8be2c56d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.257+0000 7f7430a00640 1 -- 172.21.15.19:0/307561217 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428031600 con 0x7f742c0707d0 2024-04-25T11:34:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.257+0000 7f7430a00640 1 -- 172.21.15.19:0/307561217 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74280317a0 con 0x7f742c0707d0 2024-04-25T11:34:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.257+0000 7f7430a00640 1 -- 172.21.15.19:0/307561217 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428031b70 con 0x7f742c0707d0 2024-04-25T11:34:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 -- 172.21.15.19:0/307561217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 msgr2=0x7f742c1569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 --2- 172.21.15.19:0/307561217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c1569b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7428009d90 tx=0x7f742802a5d0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 -- 172.21.15.19:0/307561217 shutdown_connections 2024-04-25T11:34:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 --2- 172.21.15.19:0/307561217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c1569b0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 --2- 172.21.15.19:0/307561217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0b0800 0x7f742c070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 -- 172.21.15.19:0/307561217 >> 172.21.15.19:0/307561217 conn(0x7f742c0d1720 msgr2=0x7f742c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 -- 172.21.15.19:0/307561217 shutdown_connections 2024-04-25T11:34:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.258+0000 7f7433200640 1 -- 172.21.15.19:0/307561217 wait complete. 2024-04-25T11:34:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.260+0000 7f7433200640 1 Processor -- start 2024-04-25T11:34:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.261+0000 7f7433200640 1 -- start start 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7433200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7433200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f742c0b0800 0x7f742c1620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7433200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0cf9e0 0x7f742c1625f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7433200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f742c0b0d90 con 0x7f742c0707d0 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7433200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f742c0b0c10 con 0x7f742c0b0800 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7433200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f742c156ef0 con 0x7f742c0cf9e0 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7431e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7431e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33922/0 (socket says 172.21.15.19:33922) 2024-04-25T11:34:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.262+0000 7f7431e00640 1 -- 172.21.15.19:0/260510297 learned_addr learned my addr 172.21.15.19:0/260510297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.263+0000 7f7431e00640 1 -- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0cf9e0 msgr2=0x7f742c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.263+0000 7f7431400640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f742c0b0800 0x7f742c1620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.263+0000 7f7431e00640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0cf9e0 0x7f742c1625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.263+0000 7f7431e00640 1 -- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f742c0b0800 msgr2=0x7f742c1620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.263+0000 7f7431e00640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f742c0b0800 0x7f742c1620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.263+0000 7f7431e00640 1 -- 172.21.15.19:0/260510297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c1f73d0 con 0x7f742c0707d0 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.263+0000 7f7431e00640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c150070 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f742000c3e0 tx=0x7f742000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.264+0000 7f7426000640 1 -- 172.21.15.19:0/260510297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420017070 con 0x7f742c0707d0 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.264+0000 7f7426000640 1 -- 172.21.15.19:0/260510297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7420005ce0 con 0x7f742c0707d0 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.264+0000 7f7426000640 1 -- 172.21.15.19:0/260510297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420007310 con 0x7f742c0707d0 2024-04-25T11:34:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.264+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c1f9bf0 con 0x7f742c0707d0 2024-04-25T11:34:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.264+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1fa1b0 con 0x7f742c0707d0 2024-04-25T11:34:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.265+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c1eaf40 con 0x7f742c0707d0 2024-04-25T11:34:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.266+0000 7f7426000640 1 -- 172.21.15.19:0/260510297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f742001a070 con 0x7f742c0707d0 2024-04-25T11:34:35.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.268+0000 7f7426000640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f74100744e0 0x7f74100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:35.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.268+0000 7f7426000640 1 -- 172.21.15.19:0/260510297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f7420094520 con 0x7f742c0707d0 2024-04-25T11:34:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.269+0000 7f7431400640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f74100744e0 0x7f74100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.270+0000 7f7431400640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f74100744e0 0x7f74100769f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f741c006fd0 tx=0x7f741c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.274+0000 7f7426000640 1 -- 172.21.15.19:0/260510297 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f742c070be0 con 0x7f742c0707d0 2024-04-25T11:34:35.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.524+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f742c050740 con 0x7f742c0707d0 2024-04-25T11:34:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.526+0000 7f7426000640 1 -- 172.21.15.19:0/260510297 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5186 (secure 0 0 0) 0x7f742005ed30 con 0x7f742c0707d0 2024-04-25T11:34:35.515 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:35.515 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":75,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:16.597206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5866":{"gid":5866,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/853310115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":853310115},{"type":"v1","addr":"172.21.15.19:6835","nonce":853310115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5866,"qdb_cluster":[5866]},"id":5}]} 2024-04-25T11:34:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 75 2024-04-25T11:34:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.528+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f74100744e0 msgr2=0x7f74100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.528+0000 7f7433200640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f74100744e0 0x7f74100769f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f741c006fd0 tx=0x7f741c008040 comp rx=0 tx=0).stop 2024-04-25T11:34:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.528+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 msgr2=0x7f742c150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.528+0000 7f7433200640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c150070 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f742000c3e0 tx=0x7f742000c8a0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 shutdown_connections 2024-04-25T11:34:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f74100744e0 0x7f74100769f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f742c0b0800 0x7f742c1620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f742c0cf9e0 0x7f742c1625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 --2- 172.21.15.19:0/260510297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f742c0707d0 0x7f742c150070 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 >> 172.21.15.19:0/260510297 conn(0x7f742c0d1720 msgr2=0x7f742c0aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 shutdown_connections 2024-04-25T11:34:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:35.529+0000 7f7433200640 1 -- 172.21.15.19:0/260510297 wait complete. 2024-04-25T11:34:35.546 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T11:34:40.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.853+0000 7fcf8ce00640 1 Processor -- start 2024-04-25T11:34:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.853+0000 7fcf8ce00640 1 -- start start 2024-04-25T11:34:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.855+0000 7fcf8ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf881434d0 0x7fcf88070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.855+0000 7fcf8ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf880709c0 con 0x7fcf880d0ed0 2024-04-25T11:34:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.855+0000 7fcf8ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8806f3e0 con 0x7fcf881612f0 2024-04-25T11:34:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.855+0000 7fcf8ce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcf881438e0 con 0x7fcf881434d0 2024-04-25T11:34:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf85e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf881434d0 0x7fcf88070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf85e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf881434d0 0x7fcf88070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33936/0 (socket says 172.21.15.19:33936) 2024-04-25T11:34:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf7fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcf880d0ed0 0x7fcf88160be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35144/0 (socket says 172.21.15.19:35144) 2024-04-25T11:34:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf85e00640 1 -- 172.21.15.19:0/3487778702 learned_addr learned my addr 172.21.15.19:0/3487778702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf85e00640 1 -- 172.21.15.19:0/3487778702 >> v1:172.21.15.169:6789/0 conn(0x7fcf881612f0 legacy=0x7fcf880b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf85e00640 1 -- 172.21.15.19:0/3487778702 >> v1:172.21.15.19:6789/0 conn(0x7fcf880d0ed0 legacy=0x7fcf88160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf85e00640 1 -- 172.21.15.19:0/3487778702 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8806f6c0 con 0x7fcf881434d0 2024-04-25T11:34:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.856+0000 7fcf85400640 1 -- 172.21.15.19:0/3487778702 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1154752366 0 0) 0x7fcf880709c0 con 0x7fcf880d0ed0 2024-04-25T11:34:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.857+0000 7fcf85e00640 1 --2- 172.21.15.19:0/3487778702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf881434d0 0x7fcf88070480 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fcf780098a0 tx=0x7fcf7802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a84f92515fff1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.858+0000 7fcf85400640 1 -- 172.21.15.19:0/3487778702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78037070 con 0x7fcf881434d0 2024-04-25T11:34:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.858+0000 7fcf85400640 1 -- 172.21.15.19:0/3487778702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78032440 con 0x7fcf881434d0 2024-04-25T11:34:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.858+0000 7fcf85400640 1 -- 172.21.15.19:0/3487778702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78032810 con 0x7fcf881434d0 2024-04-25T11:34:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.858+0000 7fcf8ce00640 1 -- 172.21.15.19:0/3487778702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf881434d0 msgr2=0x7fcf88070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.858+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/3487778702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf881434d0 0x7fcf88070480 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fcf780098a0 tx=0x7fcf7802a2e0 comp rx=0 tx=0).stop 2024-04-25T11:34:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.859+0000 7fcf8ce00640 1 -- 172.21.15.19:0/3487778702 shutdown_connections 2024-04-25T11:34:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.859+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/3487778702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf881434d0 0x7fcf88070480 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.859+0000 7fcf8ce00640 1 -- 172.21.15.19:0/3487778702 >> 172.21.15.19:0/3487778702 conn(0x7fcf880d1ae0 msgr2=0x7fcf880d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.859+0000 7fcf8ce00640 1 -- 172.21.15.19:0/3487778702 shutdown_connections 2024-04-25T11:34:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.859+0000 7fcf8ce00640 1 -- 172.21.15.19:0/3487778702 wait complete. 2024-04-25T11:34:40.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.861+0000 7fcf8ce00640 1 Processor -- start 2024-04-25T11:34:40.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.861+0000 7fcf8ce00640 1 -- start start 2024-04-25T11:34:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf8ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf880d0ed0 0x7fcf88146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf8ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf881434d0 0x7fcf8815a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf8ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf880af7c0 0x7fcf880afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf8ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8806f090 con 0x7fcf880d0ed0 2024-04-25T11:34:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf8ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8806fc80 con 0x7fcf881434d0 2024-04-25T11:34:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf8ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf880d12e0 con 0x7fcf880af7c0 2024-04-25T11:34:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf85e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf880d0ed0 0x7fcf88146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf85e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf880d0ed0 0x7fcf88146190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33938/0 (socket says 172.21.15.19:33938) 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.863+0000 7fcf85e00640 1 -- 172.21.15.19:0/673518644 learned_addr learned my addr 172.21.15.19:0/673518644 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf7fe00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf881434d0 0x7fcf8815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf85e00640 1 -- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf880af7c0 msgr2=0x7fcf880afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf86800640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf880af7c0 0x7fcf880afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf85e00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf880af7c0 0x7fcf880afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf85e00640 1 -- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf881434d0 msgr2=0x7fcf8815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf85e00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf881434d0 0x7fcf8815a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf85e00640 1 -- 172.21.15.19:0/673518644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf880b0110 con 0x7fcf880d0ed0 2024-04-25T11:34:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf85e00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf880d0ed0 0x7fcf88146190 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.864+0000 7fcf7e000640 1 -- 172.21.15.19:0/673518644 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78037040 con 0x7fcf880d0ed0 2024-04-25T11:34:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.865+0000 7fcf7e000640 1 -- 172.21.15.19:0/673518644 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78004250 con 0x7fcf880d0ed0 2024-04-25T11:34:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.865+0000 7fcf7e000640 1 -- 172.21.15.19:0/673518644 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78003610 con 0x7fcf880d0ed0 2024-04-25T11:34:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.865+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8815e4b0 con 0x7fcf880d0ed0 2024-04-25T11:34:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.865+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8815ecc0 con 0x7fcf880d0ed0 2024-04-25T11:34:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.866+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf880d0a10 con 0x7fcf880d0ed0 2024-04-25T11:34:40.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.867+0000 7fcf7e000640 1 -- 172.21.15.19:0/673518644 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf7803a070 con 0x7fcf880d0ed0 2024-04-25T11:34:40.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.868+0000 7fcf7e000640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf640744e0 0x7fcf640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:40.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.868+0000 7fcf7e000640 1 -- 172.21.15.19:0/673518644 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fcf780b5910 con 0x7fcf880d0ed0 2024-04-25T11:34:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.869+0000 7fcf7fe00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf640744e0 0x7fcf640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:40.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.870+0000 7fcf7fe00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf640744e0 0x7fcf640769f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcf8806da80 tx=0x7fcf70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:40.873+0000 7fcf7e000640 1 -- 172.21.15.19:0/673518644 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf880d0a10 con 0x7fcf880d0ed0 2024-04-25T11:34:41.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.119+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf880b58e0 con 0x7fcf880d0ed0 2024-04-25T11:34:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.119+0000 7fcf7e000640 1 -- 172.21.15.19:0/673518644 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5234 (secure 0 0 0) 0x7fcf880b0110 con 0x7fcf880d0ed0 2024-04-25T11:34:41.109 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:41.109 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":76,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5845,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:39.057611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5866},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5866":{"gid":5866,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/853310115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":853310115},{"type":"v1","addr":"172.21.15.19:6835","nonce":853310115}]},"join_fscid":-1,"laggy_since":"2024-04-25T11:34:39.057611+0000","export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5866,"qdb_cluster":[5866]},"id":5}]} 2024-04-25T11:34:41.109 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 76 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf640744e0 msgr2=0x7fcf640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf640744e0 0x7fcf640769f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcf8806da80 tx=0x7fcf70009040 comp rx=0 tx=0).stop 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf880d0ed0 msgr2=0x7fcf88146190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf880d0ed0 0x7fcf88146190 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fcf78000c00 tx=0x7fcf780046c0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 shutdown_connections 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf640744e0 0x7fcf640769f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf881434d0 0x7fcf8815a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf880af7c0 0x7fcf880afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 --2- 172.21.15.19:0/673518644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf880d0ed0 0x7fcf88146190 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.122+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 >> 172.21.15.19:0/673518644 conn(0x7fcf880d1ae0 msgr2=0x7fcf880b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:41.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.123+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 shutdown_connections 2024-04-25T11:34:41.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.123+0000 7fcf8ce00640 1 -- 172.21.15.19:0/673518644 wait complete. 2024-04-25T11:34:41.142 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-04-25T11:34:41.243 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-25T11:34:41.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.537+0000 7f8e66200640 1 Processor -- start 2024-04-25T11:34:41.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.537+0000 7f8e66200640 1 -- start start 2024-04-25T11:34:41.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e66200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e6006fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e66200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8e60161a80 0x7f8e60161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e66200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8e600b6ba0 0x7f8e600b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e66200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8e6006f640 con 0x7f8e60161a80 2024-04-25T11:34:41.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e66200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8e60144720 con 0x7f8e600b6ba0 2024-04-25T11:34:41.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e66200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8e601624d0 con 0x7f8e60144310 2024-04-25T11:34:41.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e57e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8e60161a80 0x7f8e60161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e57e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8e60161a80 0x7f8e60161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33952/0 (socket says 172.21.15.19:33952) 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.539+0000 7f8e57e00640 1 -- 172.21.15.19:0/3435927510 learned_addr learned my addr 172.21.15.19:0/3435927510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.540+0000 7f8e65800640 1 --2- 172.21.15.19:0/3435927510 >> v2:172.21.15.169:3300/0 conn(0x7f8e600b6ba0 0x7f8e600b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.540+0000 7f8e64e00640 1 --2- 172.21.15.19:0/3435927510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e6006fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.540+0000 7f8e57e00640 1 -- 172.21.15.19:0/3435927510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 msgr2=0x7f8e6006fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.540+0000 7f8e57e00640 1 --2- 172.21.15.19:0/3435927510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e6006fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.540+0000 7f8e57e00640 1 -- 172.21.15.19:0/3435927510 >> v2:172.21.15.169:3300/0 conn(0x7f8e600b6ba0 msgr2=0x7f8e600b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.540+0000 7f8e57e00640 1 --2- 172.21.15.19:0/3435927510 >> v2:172.21.15.169:3300/0 conn(0x7f8e600b6ba0 0x7f8e600b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.540+0000 7f8e57e00640 1 -- 172.21.15.19:0/3435927510 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e601627b0 con 0x7f8e60161a80 2024-04-25T11:34:41.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.541+0000 7f8e57e00640 1 --2- 172.21.15.19:0/3435927510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e60161a80 0x7f8e60161e90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8e6015f5e0 tx=0x7f8e4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0506ac39de3a90e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:41.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.541+0000 7f8e5fe00640 1 -- 172.21.15.19:0/3435927510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48037070 con 0x7f8e60161a80 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.541+0000 7f8e5fe00640 1 -- 172.21.15.19:0/3435927510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e48032440 con 0x7f8e60161a80 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.541+0000 7f8e5fe00640 1 -- 172.21.15.19:0/3435927510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48032810 con 0x7f8e60161a80 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 -- 172.21.15.19:0/3435927510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e60161a80 msgr2=0x7f8e60161e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 --2- 172.21.15.19:0/3435927510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e60161a80 0x7f8e60161e90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8e6015f5e0 tx=0x7f8e4802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 -- 172.21.15.19:0/3435927510 shutdown_connections 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 --2- 172.21.15.19:0/3435927510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e60161a80 0x7f8e60161e90 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 --2- 172.21.15.19:0/3435927510 >> v2:172.21.15.169:3300/0 conn(0x7f8e600b6ba0 0x7f8e600b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 --2- 172.21.15.19:0/3435927510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e6006fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 -- 172.21.15.19:0/3435927510 >> 172.21.15.19:0/3435927510 conn(0x7f8e600b4260 msgr2=0x7f8e60072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 -- 172.21.15.19:0/3435927510 shutdown_connections 2024-04-25T11:34:41.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.542+0000 7f8e66200640 1 -- 172.21.15.19:0/3435927510 wait complete. 2024-04-25T11:34:41.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.545+0000 7f8e66200640 1 Processor -- start 2024-04-25T11:34:41.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.545+0000 7f8e66200640 1 -- start start 2024-04-25T11:34:41.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.546+0000 7f8e66200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e600b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.546+0000 7f8e66200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e60161a80 0x7f8e6015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.546+0000 7f8e66200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e600c6aa0 0x7f8e600b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.546+0000 7f8e66200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e60070010 con 0x7f8e600c6aa0 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.546+0000 7f8e66200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e600b74f0 con 0x7f8e60161a80 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.546+0000 7f8e66200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8e601478d0 con 0x7f8e60144310 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e600c6aa0 0x7f8e600b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e600c6aa0 0x7f8e600b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33958/0 (socket says 172.21.15.19:33958) 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 -- 172.21.15.19:0/354473080 learned_addr learned my addr 172.21.15.19:0/354473080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e64e00640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e600b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e57e00640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e60161a80 0x7f8e6015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 -- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 msgr2=0x7f8e600b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e600b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 -- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e60161a80 msgr2=0x7f8e6015d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e60161a80 0x7f8e6015d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e65800640 1 -- 172.21.15.19:0/354473080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e600b1fe0 con 0x7f8e600c6aa0 2024-04-25T11:34:41.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.547+0000 7f8e64e00640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e600b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:41.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.548+0000 7f8e65800640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e600c6aa0 0x7f8e600b4fc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009fd0 tx=0x7f8e5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.548+0000 7f8e5d600640 1 -- 172.21.15.19:0/354473080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58014420 con 0x7f8e600c6aa0 2024-04-25T11:34:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.548+0000 7f8e5d600640 1 -- 172.21.15.19:0/354473080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e58019070 con 0x7f8e600c6aa0 2024-04-25T11:34:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.548+0000 7f8e5d600640 1 -- 172.21.15.19:0/354473080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58014800 con 0x7f8e600c6aa0 2024-04-25T11:34:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.548+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e600b22c0 con 0x7f8e600c6aa0 2024-04-25T11:34:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.548+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e600b2880 con 0x7f8e600c6aa0 2024-04-25T11:34:41.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.550+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e600b1d30 con 0x7f8e600c6aa0 2024-04-25T11:34:41.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.551+0000 7f8e5d600640 1 -- 172.21.15.19:0/354473080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e580149a0 con 0x7f8e600c6aa0 2024-04-25T11:34:41.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.552+0000 7f8e5d600640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e3c0744e0 0x7f8e3c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.553+0000 7f8e5d600640 1 -- 172.21.15.19:0/354473080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f8e58095ad0 con 0x7f8e600c6aa0 2024-04-25T11:34:41.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.553+0000 7f8e64e00640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e3c0744e0 0x7f8e3c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.555+0000 7f8e64e00640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e3c0744e0 0x7f8e3c0769f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:41.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.558+0000 7f8e5d600640 1 -- 172.21.15.19:0/354473080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e580601c0 con 0x7f8e600c6aa0 2024-04-25T11:34:41.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-04-25T11:34:41.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.762+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f8e601610c0 con 0x7f8e600c6aa0 2024-04-25T11:34:41.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.770+0000 7f8e5d600640 1 -- 172.21.15.19:0/354473080 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f8e5805fac0 con 0x7f8e600c6aa0 2024-04-25T11:34:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e3c0744e0 msgr2=0x7f8e3c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e3c0744e0 0x7f8e3c0769f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8e50006fd0 tx=0x7f8e50008040 comp rx=0 tx=0).stop 2024-04-25T11:34:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e600c6aa0 msgr2=0x7f8e600b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e600c6aa0 0x7f8e600b4fc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009fd0 tx=0x7f8e5800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 shutdown_connections 2024-04-25T11:34:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e3c0744e0 0x7f8e3c0769f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e60161a80 0x7f8e6015d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e600c6aa0 0x7f8e600b4fc0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 --2- 172.21.15.19:0/354473080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e60144310 0x7f8e600b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 >> 172.21.15.19:0/354473080 conn(0x7f8e600b6ba0 msgr2=0x7f8e60160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 shutdown_connections 2024-04-25T11:34:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:41.773+0000 7f8e66200640 1 -- 172.21.15.19:0/354473080 wait complete. 2024-04-25T11:34:41.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2024-04-25T11:34:42.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.088+0000 7eff0d600640 1 Processor -- start 2024-04-25T11:34:42.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.088+0000 7eff0d600640 1 -- start start 2024-04-25T11:34:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.091+0000 7eff0d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 0x7eff08167cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.091+0000 7eff0d600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff080709a0 con 0x7eff0806d5d0 2024-04-25T11:34:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.091+0000 7eff0d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff08070c70 con 0x7eff08143660 2024-04-25T11:34:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.091+0000 7eff0d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff080b4920 con 0x7eff080b66f0 2024-04-25T11:34:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.091+0000 7eff06000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7eff0806d5d0 0x7eff0806d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35160/0 (socket says 172.21.15.19:35160) 2024-04-25T11:34:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.091+0000 7eff06000640 1 -- 172.21.15.19:0/2043563694 learned_addr learned my addr 172.21.15.19:0/2043563694 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:42.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff07400640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 0x7eff08167cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3583797313 0 0) 0x7eff080709a0 con 0x7eff0806d5d0 2024-04-25T11:34:42.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeec003520 con 0x7eff0806d5d0 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2196041760 0 0) 0x7efeec003520 con 0x7eff0806d5d0 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 msgr2=0x7eff08167cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 0x7eff08167cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 >> v1:172.21.15.169:6789/0 conn(0x7eff08143660 legacy=0x7eff08070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff080b4c00 con 0x7eff0806d5d0 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efef4001670 con 0x7eff0806d5d0 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 >> v1:172.21.15.19:6789/0 conn(0x7eff0806d5d0 legacy=0x7eff0806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeec0039e0 0x7efeec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec004800 0x7efeec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 0x7efeec005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff080b4920 con 0x7efeec0039e0 2024-04-25T11:34:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efeec00dd60 con 0x7efeec004800 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.092+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efeec00df60 con 0x7eff080b66f0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06a00640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeec0039e0 0x7efeec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff07400640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 0x7efeec005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06000640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec004800 0x7efeec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06a00640 1 -- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 msgr2=0x7efeec005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06a00640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff080b66f0 0x7efeec005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06a00640 1 -- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec004800 msgr2=0x7efeec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06a00640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec004800 0x7efeec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06a00640 1 -- 172.21.15.19:0/2043563694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec003520 con 0x7efeec0039e0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.093+0000 7eff06000640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec004800 0x7efeec004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.094+0000 7eff06a00640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeec0039e0 0x7efeec003e50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efef8001720 tx=0x7efeec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760b1460caa3485e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.094+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8001370 con 0x7efeec0039e0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.095+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef80052d0 con 0x7efeec0039e0 2024-04-25T11:34:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.095+0000 7eff05600640 1 -- 172.21.15.19:0/2043563694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef802abc0 con 0x7efeec0039e0 2024-04-25T11:34:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.095+0000 7eff0d600640 1 -- 172.21.15.19:0/2043563694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeec0039e0 msgr2=0x7efeec003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.095+0000 7eff0d600640 1 --2- 172.21.15.19:0/2043563694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeec0039e0 0x7efeec003e50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efef8001720 tx=0x7efeec005830 comp rx=0 tx=0).stop 2024-04-25T11:34:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.095+0000 7eff07400640 1 -- 172.21.15.19:0/2043563694 reap_dead start 2024-04-25T11:34:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.095+0000 7eff0d600640 1 -- 172.21.15.19:0/2043563694 shutdown_connections 2024-04-25T11:34:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.095+0000 7eff0d600640 1 -- 172.21.15.19:0/2043563694 >> 172.21.15.19:0/2043563694 conn(0x7eff08073120 msgr2=0x7eff08073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.096+0000 7eff0d600640 1 -- 172.21.15.19:0/2043563694 shutdown_connections 2024-04-25T11:34:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.096+0000 7eff0d600640 1 -- 172.21.15.19:0/2043563694 wait complete. 2024-04-25T11:34:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.098+0000 7eff0d600640 1 Processor -- start 2024-04-25T11:34:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.098+0000 7eff0d600640 1 -- start start 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff0d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec0039e0 0x7eff081581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff0d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff0815b400 0x7eff0813f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff0d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff0815b810 0x7eff0813fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff0d600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff080ce0e0 con 0x7eff0815b810 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff0d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7eff080b51c0 con 0x7efeec0039e0 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff0d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7eff080ce260 con 0x7eff0815b400 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff0815b810 0x7eff0813fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff0815b810 0x7eff0813fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33978/0 (socket says 172.21.15.19:33978) 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 -- 172.21.15.19:0/4122592327 learned_addr learned my addr 172.21.15.19:0/4122592327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff06a00640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec0039e0 0x7eff081581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff06000640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff0815b400 0x7eff0813f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 -- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff0815b400 msgr2=0x7eff0813f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff0815b400 0x7eff0813f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 -- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec0039e0 msgr2=0x7eff081581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec0039e0 0x7eff081581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff07400640 1 -- 172.21.15.19:0/4122592327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff081569c0 con 0x7eff0815b810 2024-04-25T11:34:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.100+0000 7eff06000640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff0815b400 0x7eff0813f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.101+0000 7eff07400640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff0815b810 0x7eff0813fe30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7eff080cf100 tx=0x7eff000115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.101+0000 7efefea00640 1 -- 172.21.15.19:0/4122592327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00011de0 con 0x7eff0815b810 2024-04-25T11:34:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.101+0000 7efefea00640 1 -- 172.21.15.19:0/4122592327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0001b070 con 0x7eff0815b810 2024-04-25T11:34:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.101+0000 7efefea00640 1 -- 172.21.15.19:0/4122592327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00003990 con 0x7eff0815b810 2024-04-25T11:34:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.101+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff08156ca0 con 0x7eff0815b810 2024-04-25T11:34:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.101+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff08157260 con 0x7eff0815b810 2024-04-25T11:34:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.102+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff08050740 con 0x7eff0815b810 2024-04-25T11:34:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.104+0000 7efefea00640 1 -- 172.21.15.19:0/4122592327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7eff00005d10 con 0x7eff0815b810 2024-04-25T11:34:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.105+0000 7efefea00640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efee80744e0 0x7efee80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.105+0000 7efefea00640 1 -- 172.21.15.19:0/4122592327 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7eff00095740 con 0x7eff0815b810 2024-04-25T11:34:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.106+0000 7eff06a00640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efee80744e0 0x7efee80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.107+0000 7eff06a00640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efee80744e0 0x7efee80769f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efef8032000 tx=0x7efef802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:42.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.110+0000 7efefea00640 1 -- 172.21.15.19:0/4122592327 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff00025020 con 0x7eff0815b810 2024-04-25T11:34:42.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:42.401+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) -- 0x7eff08140d30 con 0x7eff0815b810 2024-04-25T11:34:43.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.084+0000 7efefea00640 1 -- 172.21.15.19:0/4122592327 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.5866 v77) ==== 115+0+0 (secure 0 0 0) 0x7eff0005fec0 con 0x7eff0815b810 2024-04-25T11:34:43.073 INFO:teuthology.orchestra.run.smithi019.stderr:unfreezing mds.5866 2024-04-25T11:34:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.087+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efee80744e0 msgr2=0x7efee80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.087+0000 7eff0d600640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efee80744e0 0x7efee80769f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efef8032000 tx=0x7efef802e000 comp rx=0 tx=0).stop 2024-04-25T11:34:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.087+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff0815b810 msgr2=0x7eff0813fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.087+0000 7eff0d600640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff0815b810 0x7eff0813fe30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7eff080cf100 tx=0x7eff000115a0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 shutdown_connections 2024-04-25T11:34:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efee80744e0 0x7efee80769f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff0815b810 0x7eff0813fe30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7eff0815b400 0x7eff0813f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 --2- 172.21.15.19:0/4122592327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeec0039e0 0x7eff081581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 >> 172.21.15.19:0/4122592327 conn(0x7eff08073120 msgr2=0x7eff080b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 shutdown_connections 2024-04-25T11:34:43.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.088+0000 7eff0d600640 1 -- 172.21.15.19:0/4122592327 wait complete. 2024-04-25T11:34:43.120 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -n kernel.core_pattern 2024-04-25T11:34:43.197 INFO:teuthology.orchestra.run.smithi019.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:34:43.200 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:34:43.200 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-04-25T11:34:43.286 INFO:teuthology.orchestra.run.smithi019.stdout:1714044863.57743.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-04-25T11:34:43.291 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1714044863.57743.core'] 2024-04-25T11:34:43.291 INFO:tasks.cephfs.cephfs_test_case:Found core file 1714044863.57743.core, deleting it 2024-04-25T11:34:43.291 DEBUG:teuthology.orchestra.run.smithi019:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1714044863.57743.core 2024-04-25T11:34:43.413 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:34:43.414 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:34:43.414 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:34:43.414 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:34:43.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:34:43.414 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi019 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-04-25T11:34:43.415 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:34:43.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:34:43.422 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:34:43.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:43.653 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:34:43.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.692+0000 7f8cc5e00640 1 Processor -- start 2024-04-25T11:34:43.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.692+0000 7f8cc5e00640 1 -- start start 2024-04-25T11:34:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cc5e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8cc01443c0 0x7f8cc006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cc5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cc00cfcf0 0x7f8cc006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cc5e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cc015f5f0 con 0x7f8cc0070370 2024-04-25T11:34:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cc5e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc00708d0 con 0x7f8cc00cfcf0 2024-04-25T11:34:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cc5e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc015f160 con 0x7f8cc01443c0 2024-04-25T11:34:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cbf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cc00cfcf0 0x7f8cc006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cb7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8cc0070370 0x7f8cc00cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35178/0 (socket says 172.21.15.19:35178) 2024-04-25T11:34:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cb7e00640 1 -- 172.21.15.19:0/241297054 learned_addr learned my addr 172.21.15.19:0/241297054 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cbf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cc00cfcf0 0x7f8cc006c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34008/0 (socket says 172.21.15.19:34008) 2024-04-25T11:34:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.694+0000 7f8cbea00640 1 --2- 172.21.15.19:0/241297054 >> v2:172.21.15.169:3300/0 conn(0x7f8cc01443c0 0x7f8cc006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.695+0000 7f8cbf400640 1 -- 172.21.15.19:0/241297054 >> v2:172.21.15.169:3300/0 conn(0x7f8cc01443c0 msgr2=0x7f8cc006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.695+0000 7f8cbf400640 1 --2- 172.21.15.19:0/241297054 >> v2:172.21.15.169:3300/0 conn(0x7f8cc01443c0 0x7f8cc006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.695+0000 7f8cbf400640 1 -- 172.21.15.19:0/241297054 >> v1:172.21.15.19:6789/0 conn(0x7f8cc0070370 legacy=0x7f8cc00cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.695+0000 7f8cbf400640 1 -- 172.21.15.19:0/241297054 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc015e830 con 0x7f8cc00cfcf0 2024-04-25T11:34:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.695+0000 7f8cbe000640 1 -- 172.21.15.19:0/241297054 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3961334449 0 0) 0x7f8cc015f5f0 con 0x7f8cc0070370 2024-04-25T11:34:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.696+0000 7f8cbf400640 1 --2- 172.21.15.19:0/241297054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00cfcf0 0x7f8cc006c5c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0072820 tx=0x7f8cb802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5572fd4e8b047b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.696+0000 7f8cbe000640 1 -- 172.21.15.19:0/241297054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007740 con 0x7f8cc00cfcf0 2024-04-25T11:34:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.696+0000 7f8cbe000640 1 -- 172.21.15.19:0/241297054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb80078e0 con 0x7f8cc00cfcf0 2024-04-25T11:34:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.696+0000 7f8cbe000640 1 -- 172.21.15.19:0/241297054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007cb0 con 0x7f8cc00cfcf0 2024-04-25T11:34:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 -- 172.21.15.19:0/241297054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00cfcf0 msgr2=0x7f8cc006c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/241297054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00cfcf0 0x7f8cc006c5c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0072820 tx=0x7f8cb802a680 comp rx=0 tx=0).stop 2024-04-25T11:34:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 -- 172.21.15.19:0/241297054 shutdown_connections 2024-04-25T11:34:43.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/241297054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc00cfcf0 0x7f8cc006c5c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/241297054 >> v2:172.21.15.169:3300/0 conn(0x7f8cc01443c0 0x7f8cc006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 -- 172.21.15.19:0/241297054 >> 172.21.15.19:0/241297054 conn(0x7f8cc00d1ae0 msgr2=0x7f8cc00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 -- 172.21.15.19:0/241297054 shutdown_connections 2024-04-25T11:34:43.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.697+0000 7f8cc5e00640 1 -- 172.21.15.19:0/241297054 wait complete. 2024-04-25T11:34:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.699+0000 7f8cc5e00640 1 Processor -- start 2024-04-25T11:34:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.699+0000 7f8cc5e00640 1 -- start start 2024-04-25T11:34:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cc5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0070370 0x7f8cc014e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cc5e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cc01443c0 0x7f8cc013d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cc5e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cc0140a50 0x7f8cc0140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cc5e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01447d0 con 0x7f8cc0070370 2024-04-25T11:34:43.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cc5e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc015edf0 con 0x7f8cc0140a50 2024-04-25T11:34:43.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cc5e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc0162b00 con 0x7f8cc01443c0 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cbea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0070370 0x7f8cc014e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cbea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0070370 0x7f8cc014e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34014/0 (socket says 172.21.15.19:34014) 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.701+0000 7f8cbea00640 1 -- 172.21.15.19:0/3026906439 learned_addr learned my addr 172.21.15.19:0/3026906439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbf400640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cc0140a50 0x7f8cc0140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cb7e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cc01443c0 0x7f8cc013d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbea00640 1 -- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cc01443c0 msgr2=0x7f8cc013d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbea00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cc01443c0 0x7f8cc013d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbea00640 1 -- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cc0140a50 msgr2=0x7f8cc0140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbea00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cc0140a50 0x7f8cc0140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbea00640 1 -- 172.21.15.19:0/3026906439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc014eaa0 con 0x7f8cc0070370 2024-04-25T11:34:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbf400640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cc0140a50 0x7f8cc0140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:43.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cbea00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0070370 0x7f8cc014e560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d630 tx=0x7f8cb000daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:43.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cb7400640 1 -- 172.21.15.19:0/3026906439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0004240 con 0x7f8cc0070370 2024-04-25T11:34:43.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cb7400640 1 -- 172.21.15.19:0/3026906439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb00043e0 con 0x7f8cc0070370 2024-04-25T11:34:43.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.702+0000 7f8cb7400640 1 -- 172.21.15.19:0/3026906439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0003830 con 0x7f8cc0070370 2024-04-25T11:34:43.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.703+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc01413a0 con 0x7f8cc0070370 2024-04-25T11:34:43.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.703+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc01f7fc0 con 0x7f8cc0070370 2024-04-25T11:34:43.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.704+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc0050740 con 0x7f8cc0070370 2024-04-25T11:34:43.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.706+0000 7f8cb7400640 1 -- 172.21.15.19:0/3026906439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8cb00039d0 con 0x7f8cc0070370 2024-04-25T11:34:43.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.707+0000 7f8cb7400640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c9c0744e0 0x7f8c9c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.708+0000 7f8cb7400640 1 -- 172.21.15.19:0/3026906439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f8cb00954d0 con 0x7f8cc0070370 2024-04-25T11:34:43.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.708+0000 7f8cb7e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c9c0744e0 0x7f8c9c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.710+0000 7f8cb7e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c9c0744e0 0x7f8c9c0769f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b4980 tx=0x7f8ca8007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:43.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.712+0000 7f8cb7400640 1 -- 172.21.15.19:0/3026906439 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cc01413a0 con 0x7f8cc0070370 2024-04-25T11:34:43.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.964+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cc006f950 con 0x7f8cc0070370 2024-04-25T11:34:43.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.965+0000 7f8cb7400640 1 -- 172.21.15.19:0/3026906439 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+4358 (secure 0 0 0) 0x7f8cc014eaa0 con 0x7f8cc0070370 2024-04-25T11:34:43.955 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:43.955 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":78,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:43.092138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5845":{"gid":5845,"name":"b","rank":0,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-25T11:34:43.955 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 78 2024-04-25T11:34:43.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c9c0744e0 msgr2=0x7f8c9c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c9c0744e0 0x7f8c9c0769f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b4980 tx=0x7f8ca8007c00 comp rx=0 tx=0).stop 2024-04-25T11:34:43.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0070370 msgr2=0x7f8cc014e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0070370 0x7f8cc014e560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d630 tx=0x7f8cb000daf0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 shutdown_connections 2024-04-25T11:34:43.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c9c0744e0 0x7f8c9c0769f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8cc0140a50 0x7f8cc0140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8cc01443c0 0x7f8cc013d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 --2- 172.21.15.19:0/3026906439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0070370 0x7f8cc014e560 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.968+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 >> 172.21.15.19:0/3026906439 conn(0x7f8cc00d1ae0 msgr2=0x7f8cc006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.969+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 shutdown_connections 2024-04-25T11:34:43.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:43.969+0000 7f8cc5e00640 1 -- 172.21.15.19:0/3026906439 wait complete. 2024-04-25T11:34:44.389 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:34:05.831545+0000', 'modified': '2024-04-25T11:34:43.092138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5845': {'gid': 5845, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.169:6836/2118499768', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2118499768}, {'type': 'v1', 'addr': '172.21.15.169:6836', 'nonce': 2118499768}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:34:44.389 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5845:up:replay 2024-04-25T11:34:45.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:45.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.695+0000 7fae8e000640 1 Processor -- start 2024-04-25T11:34:45.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.695+0000 7fae8e000640 1 -- start start 2024-04-25T11:34:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.698+0000 7fae8e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fae8814ad60 0x7fae88070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.698+0000 7fae8e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fae8806c5c0 0x7fae8806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.698+0000 7fae8e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae880b4af0 con 0x7fae880b5e40 2024-04-25T11:34:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.698+0000 7fae8e000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fae880b4dd0 con 0x7fae8806c5c0 2024-04-25T11:34:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.698+0000 7fae8e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fae880b50b0 con 0x7fae8814ad60 2024-04-25T11:34:45.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fae8806c5c0 0x7fae8806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fae8806c5c0 0x7fae8806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35346/0 (socket says 172.21.15.19:35346) 2024-04-25T11:34:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87e00640 1 -- 172.21.15.19:0/2702081048 learned_addr learned my addr 172.21.15.19:0/2702081048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87400640 1 --2- 172.21.15.19:0/2702081048 >> v2:172.21.15.169:3300/0 conn(0x7fae8814ad60 0x7fae88070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87e00640 1 -- 172.21.15.19:0/2702081048 >> v2:172.21.15.169:3300/0 conn(0x7fae8814ad60 msgr2=0x7fae88070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87e00640 1 --2- 172.21.15.19:0/2702081048 >> v2:172.21.15.169:3300/0 conn(0x7fae8814ad60 0x7fae88070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87e00640 1 -- 172.21.15.19:0/2702081048 >> v1:172.21.15.169:6789/0 conn(0x7fae880b5e40 legacy=0x7fae880b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87e00640 1 -- 172.21.15.19:0/2702081048 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae880b5390 con 0x7fae8806c5c0 2024-04-25T11:34:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.699+0000 7fae87400640 1 --2- 172.21.15.19:0/2702081048 >> v2:172.21.15.169:3300/0 conn(0x7fae8814ad60 0x7fae88070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:45.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.700+0000 7fae87e00640 1 --2- 172.21.15.19:0/2702081048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8806c5c0 0x7fae8806c9d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fae880ce320 tx=0x7fae8002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0c94e329d53ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:45.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.700+0000 7fae86a00640 1 -- 172.21.15.19:0/2702081048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80036070 con 0x7fae8806c5c0 2024-04-25T11:34:45.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.701+0000 7fae86a00640 1 -- 172.21.15.19:0/2702081048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae80031600 con 0x7fae8806c5c0 2024-04-25T11:34:45.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.701+0000 7fae86a00640 1 -- 172.21.15.19:0/2702081048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae800319d0 con 0x7fae8806c5c0 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.701+0000 7fae8e000640 1 -- 172.21.15.19:0/2702081048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8806c5c0 msgr2=0x7fae8806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.701+0000 7fae8e000640 1 --2- 172.21.15.19:0/2702081048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8806c5c0 0x7fae8806c9d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fae880ce320 tx=0x7fae8002a630 comp rx=0 tx=0).stop 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.702+0000 7fae8e000640 1 -- 172.21.15.19:0/2702081048 shutdown_connections 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.702+0000 7fae8e000640 1 --2- 172.21.15.19:0/2702081048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8806c5c0 0x7fae8806c9d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.702+0000 7fae8e000640 1 --2- 172.21.15.19:0/2702081048 >> v2:172.21.15.169:3300/0 conn(0x7fae8814ad60 0x7fae88070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.702+0000 7fae8e000640 1 -- 172.21.15.19:0/2702081048 >> 172.21.15.19:0/2702081048 conn(0x7fae880d1ae0 msgr2=0x7fae880d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.702+0000 7fae8e000640 1 -- 172.21.15.19:0/2702081048 shutdown_connections 2024-04-25T11:34:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.702+0000 7fae8e000640 1 -- 172.21.15.19:0/2702081048 wait complete. 2024-04-25T11:34:45.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.705+0000 7fae8e000640 1 Processor -- start 2024-04-25T11:34:45.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.705+0000 7fae8e000640 1 -- start start 2024-04-25T11:34:45.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae8e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae8806c5c0 0x7fae881608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae8e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae880b5e40 0x7fae881fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae8e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8814ad60 0x7fae881650b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae8e000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae880706d0 con 0x7fae8814ad60 2024-04-25T11:34:45.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae8e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fae88070550 con 0x7fae8806c5c0 2024-04-25T11:34:45.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae8e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fae8806fb80 con 0x7fae880b5e40 2024-04-25T11:34:45.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae87e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8814ad60 0x7fae881650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae87e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8814ad60 0x7fae881650b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35354/0 (socket says 172.21.15.19:35354) 2024-04-25T11:34:45.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.707+0000 7fae87e00640 1 -- 172.21.15.19:0/1153312996 learned_addr learned my addr 172.21.15.19:0/1153312996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:45.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae87400640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae8806c5c0 0x7fae881608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae7fe00640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae880b5e40 0x7fae881fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae87e00640 1 -- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae880b5e40 msgr2=0x7fae881fb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:45.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae87e00640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae880b5e40 0x7fae881fb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae87e00640 1 -- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae8806c5c0 msgr2=0x7fae881608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:45.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae87e00640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae8806c5c0 0x7fae881608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae87e00640 1 -- 172.21.15.19:0/1153312996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae881fb900 con 0x7fae8814ad60 2024-04-25T11:34:45.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae7fe00640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae880b5e40 0x7fae881fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.708+0000 7fae87e00640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8814ad60 0x7fae881650b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fae80009f10 tx=0x7fae80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.709+0000 7fae7f400640 1 -- 172.21.15.19:0/1153312996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80036050 con 0x7fae8814ad60 2024-04-25T11:34:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.709+0000 7fae7f400640 1 -- 172.21.15.19:0/1153312996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae80042070 con 0x7fae8814ad60 2024-04-25T11:34:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.709+0000 7fae7f400640 1 -- 172.21.15.19:0/1153312996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80009510 con 0x7fae8814ad60 2024-04-25T11:34:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.709+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881ed2e0 con 0x7fae8814ad60 2024-04-25T11:34:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.709+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881ef2e0 con 0x7fae8814ad60 2024-04-25T11:34:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.710+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae880b2480 con 0x7fae8814ad60 2024-04-25T11:34:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.711+0000 7fae7f400640 1 -- 172.21.15.19:0/1153312996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fae80007bd0 con 0x7fae8814ad60 2024-04-25T11:34:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.712+0000 7fae7f400640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fae640744e0 0x7fae640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.713+0000 7fae7f400640 1 -- 172.21.15.19:0/1153312996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fae800b5c90 con 0x7fae8814ad60 2024-04-25T11:34:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.713+0000 7fae87400640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fae640744e0 0x7fae640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.715+0000 7fae87400640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fae640744e0 0x7fae640769f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae880b1590 tx=0x7fae78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.718+0000 7fae7f400640 1 -- 172.21.15.19:0/1153312996 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae880b2480 con 0x7fae8814ad60 2024-04-25T11:34:45.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.961+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae88162220 con 0x7fae8814ad60 2024-04-25T11:34:45.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.962+0000 7fae7f400640 1 -- 172.21.15.19:0/1153312996 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5184 (secure 0 0 0) 0x7fae88162220 con 0x7fae8814ad60 2024-04-25T11:34:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:45.953 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3655604895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3655604895},{"type":"v1","addr":"172.21.15.19:6835","nonce":3655604895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:44.123315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5845":{"gid":5845,"name":"b","rank":0,"incarnation":78,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-25T11:34:45.953 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 80 2024-04-25T11:34:45.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fae640744e0 msgr2=0x7fae640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:45.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fae640744e0 0x7fae640769f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fae880b1590 tx=0x7fae78008040 comp rx=0 tx=0).stop 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8814ad60 msgr2=0x7fae881650b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8814ad60 0x7fae881650b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fae80009f10 tx=0x7fae80004a40 comp rx=0 tx=0).stop 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 shutdown_connections 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fae640744e0 0x7fae640769f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae8814ad60 0x7fae881650b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae880b5e40 0x7fae881fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 --2- 172.21.15.19:0/1153312996 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae8806c5c0 0x7fae881608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 >> 172.21.15.19:0/1153312996 conn(0x7fae880d1ae0 msgr2=0x7fae880b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 shutdown_connections 2024-04-25T11:34:45.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:45.965+0000 7fae8e000640 1 -- 172.21.15.19:0/1153312996 wait complete. 2024-04-25T11:34:46.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:34:46.385 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:34:05.831545+0000', 'modified': '2024-04-25T11:34:44.123315+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5845': {'gid': 5845, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:rejoin', 'state_seq': 13, 'addr': '172.21.15.169:6836/2118499768', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2118499768}, {'type': 'v1', 'addr': '172.21.15.169:6836', 'nonce': 2118499768}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:34:46.385 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5845:up:rejoin 2024-04-25T11:34:47.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.689+0000 7f089b400640 1 Processor -- start 2024-04-25T11:34:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.689+0000 7f089b400640 1 -- start start 2024-04-25T11:34:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f089b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08941616a0 0x7f089415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f089b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f089406e030 0x7f089406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f089b400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f089406e980 con 0x7f0894163fc0 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f089b400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0894160980 con 0x7f08941616a0 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f089b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0894164c80 con 0x7f089406e030 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f0898a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08941616a0 0x7f089415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f0898a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08941616a0 0x7f089415ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35366/0 (socket says 172.21.15.19:35366) 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f0898a00640 1 -- 172.21.15.19:0/567547609 learned_addr learned my addr 172.21.15.19:0/567547609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.692+0000 7f0899400640 1 --1- 172.21.15.19:0/567547609 >> v1:172.21.15.19:6789/0 conn(0x7f0894163fc0 0x7f08941643d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56576/0 (socket says 172.21.15.19:56576) 2024-04-25T11:34:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.693+0000 7f0893e00640 1 --2- 172.21.15.19:0/567547609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f089406e030 0x7f089406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.693+0000 7f0898a00640 1 -- 172.21.15.19:0/567547609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f089406e030 msgr2=0x7f089406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.693+0000 7f0898a00640 1 --2- 172.21.15.19:0/567547609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f089406e030 0x7f089406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.693+0000 7f0898a00640 1 -- 172.21.15.19:0/567547609 >> v1:172.21.15.19:6789/0 conn(0x7f0894163fc0 legacy=0x7f08941643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.693+0000 7f0898a00640 1 -- 172.21.15.19:0/567547609 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0894160bc0 con 0x7f08941616a0 2024-04-25T11:34:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.694+0000 7f0898a00640 1 --2- 172.21.15.19:0/567547609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08941616a0 0x7f089415ded0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0884009e50 tx=0x7f088402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fa5d474e1af4c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.694+0000 7f0893400640 1 -- 172.21.15.19:0/567547609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884036070 con 0x7f08941616a0 2024-04-25T11:34:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f0893400640 1 -- 172.21.15.19:0/567547609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0884031600 con 0x7f08941616a0 2024-04-25T11:34:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f0893400640 1 -- 172.21.15.19:0/567547609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08840319d0 con 0x7f08941616a0 2024-04-25T11:34:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f089b400640 1 -- 172.21.15.19:0/567547609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08941616a0 msgr2=0x7f089415ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f089b400640 1 --2- 172.21.15.19:0/567547609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08941616a0 0x7f089415ded0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0884009e50 tx=0x7f088402a710 comp rx=0 tx=0).stop 2024-04-25T11:34:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f089b400640 1 -- 172.21.15.19:0/567547609 shutdown_connections 2024-04-25T11:34:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f089b400640 1 --2- 172.21.15.19:0/567547609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f089406e030 0x7f089406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f089b400640 1 --2- 172.21.15.19:0/567547609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08941616a0 0x7f089415ded0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.695+0000 7f089b400640 1 -- 172.21.15.19:0/567547609 >> 172.21.15.19:0/567547609 conn(0x7f08940d27c0 msgr2=0x7f08940d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.696+0000 7f089b400640 1 -- 172.21.15.19:0/567547609 shutdown_connections 2024-04-25T11:34:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.696+0000 7f089b400640 1 -- 172.21.15.19:0/567547609 wait complete. 2024-04-25T11:34:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.698+0000 7f089b400640 1 Processor -- start 2024-04-25T11:34:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.698+0000 7f089b400640 1 -- start start 2024-04-25T11:34:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f089b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f089406e030 0x7f08941f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f089b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08941616a0 0x7f089415f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f089b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0894163fc0 0x7f089415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f089b400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08940b77c0 con 0x7f089406e030 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f089b400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0894161180 con 0x7f0894163fc0 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f089b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08940b7940 con 0x7f08941616a0 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f0898a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f089406e030 0x7f08941f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f0898a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f089406e030 0x7f08941f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35376/0 (socket says 172.21.15.19:35376) 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f0898a00640 1 -- 172.21.15.19:0/2488971864 learned_addr learned my addr 172.21.15.19:0/2488971864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f0893e00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08941616a0 0x7f089415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.700+0000 7f0899400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0894163fc0 0x7f089415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0898a00640 1 -- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08941616a0 msgr2=0x7f089415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0898a00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08941616a0 0x7f089415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0898a00640 1 -- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0894163fc0 msgr2=0x7f089415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0898a00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0894163fc0 0x7f089415f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0898a00640 1 -- 172.21.15.19:0/2488971864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08941452b0 con 0x7f089406e030 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0893e00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08941616a0 0x7f089415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0899400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0894163fc0 0x7f089415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0898a00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f089406e030 0x7f08941f98f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f088402ad00 tx=0x7f0884004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0890c00640 1 -- 172.21.15.19:0/2488971864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088403f020 con 0x7f089406e030 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0890c00640 1 -- 172.21.15.19:0/2488971864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0884037040 con 0x7f089406e030 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f0890c00640 1 -- 172.21.15.19:0/2488971864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884036070 con 0x7f089406e030 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0894145590 con 0x7f089406e030 2024-04-25T11:34:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.701+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0894145b50 con 0x7f089406e030 2024-04-25T11:34:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.703+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0894050740 con 0x7f089406e030 2024-04-25T11:34:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.704+0000 7f0890c00640 1 -- 172.21.15.19:0/2488971864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0884042020 con 0x7f089406e030 2024-04-25T11:34:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.705+0000 7f0890c00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08740744b0 0x7f08740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.705+0000 7f0890c00640 1 -- 172.21.15.19:0/2488971864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f08840b58e0 con 0x7f089406e030 2024-04-25T11:34:47.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.706+0000 7f0893e00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08740744b0 0x7f08740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.707+0000 7f0893e00640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08740744b0 0x7f08740769c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0880006fd0 tx=0x7f0880008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:47.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.711+0000 7f0890c00640 1 -- 172.21.15.19:0/2488971864 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0884049030 con 0x7f089406e030 2024-04-25T11:34:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.956+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08940c8d70 con 0x7f089406e030 2024-04-25T11:34:47.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.957+0000 7f0890c00640 1 -- 172.21.15.19:0/2488971864 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5191 (secure 0 0 0) 0x7f088407ff70 con 0x7f089406e030 2024-04-25T11:34:47.948 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:47.948 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3655604895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3655604895},{"type":"v1","addr":"172.21.15.19:6835","nonce":3655604895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:46.124891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5845":{"gid":5845,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":14,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5845,"qdb_cluster":[5845]},"id":5}]} 2024-04-25T11:34:47.948 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 81 2024-04-25T11:34:47.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08740744b0 msgr2=0x7f08740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:47.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08740744b0 0x7f08740769c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0880006fd0 tx=0x7f0880008040 comp rx=0 tx=0).stop 2024-04-25T11:34:47.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f089406e030 msgr2=0x7f08941f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:47.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f089406e030 0x7f08941f98f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f088402ad00 tx=0x7f0884004a40 comp rx=0 tx=0).stop 2024-04-25T11:34:47.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 shutdown_connections 2024-04-25T11:34:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08740744b0 0x7f08740769c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0894163fc0 0x7f089415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f08941616a0 0x7f089415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 --2- 172.21.15.19:0/2488971864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f089406e030 0x7f08941f98f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 >> 172.21.15.19:0/2488971864 conn(0x7f08940d27c0 msgr2=0x7f08940d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 shutdown_connections 2024-04-25T11:34:47.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:47.960+0000 7f089b400640 1 -- 172.21.15.19:0/2488971864 wait complete. 2024-04-25T11:34:48.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T11:34:48.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:34:05.831545+0000', 'modified': '2024-04-25T11:34:46.124891+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5845': {'gid': 5845, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.169:6836/2118499768', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2118499768}, {'type': 'v1', 'addr': '172.21.15.169:6836', 'nonce': 2118499768}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5845, 'qdb_cluster': [5845]} 2024-04-25T11:34:48.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:34:48.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:34:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.675+0000 7f34f5a00640 1 Processor -- start 2024-04-25T11:34:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.675+0000 7f34f5a00640 1 -- start start 2024-04-25T11:34:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.677+0000 7f34f5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34f00b4ea0 0x7f34f00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.677+0000 7f34f5a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f00d1230 con 0x7f34f006c5c0 2024-04-25T11:34:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.677+0000 7f34f5a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f00d06b0 con 0x7f34f0144370 2024-04-25T11:34:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.677+0000 7f34f5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34f00d0990 con 0x7f34f00b4ea0 2024-04-25T11:34:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34efe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f34f006c5c0 0x7f34f00b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56580/0 (socket says 172.21.15.19:56580) 2024-04-25T11:34:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34efe00640 1 -- 172.21.15.19:0/880423230 learned_addr learned my addr 172.21.15.19:0/880423230 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34ef400640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34f00b4ea0 0x7f34f00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3629943415 0 0) 0x7f34f00d1230 con 0x7f34f006c5c0 2024-04-25T11:34:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34d4003520 con 0x7f34f006c5c0 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4198729316 0 0) 0x7f34d4003520 con 0x7f34f006c5c0 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34f00b4ea0 msgr2=0x7f34f00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34eea00640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34f00b4ea0 0x7f34f00cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.678+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 >> v1:172.21.15.169:6789/0 conn(0x7f34f0144370 legacy=0x7f34f015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f00d0c70 con 0x7f34f006c5c0 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f34e0001670 con 0x7f34f006c5c0 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 >> v1:172.21.15.19:6789/0 conn(0x7f34f006c5c0 legacy=0x7f34f00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d40039e0 0x7f34d4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34d4004800 0x7f34d4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d4005150 0x7f34d400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34f00d0990 con 0x7f34d4005150 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34d40103c0 con 0x7f34d4004800 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34d40105c0 con 0x7f34d40039e0 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34f5000640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d4005150 0x7f34d400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34ef400640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34d4004800 0x7f34d4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.679+0000 7f34efe00640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d40039e0 0x7f34d4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.680+0000 7f34f5000640 1 -- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d40039e0 msgr2=0x7f34d4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.680+0000 7f34f5000640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d40039e0 0x7f34d4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.680+0000 7f34f5000640 1 -- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34d4004800 msgr2=0x7f34d4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.680+0000 7f34f5000640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34d4004800 0x7f34d4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.680+0000 7f34f5000640 1 -- 172.21.15.19:0/880423230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d4003520 con 0x7f34d4005150 2024-04-25T11:34:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.681+0000 7f34f5000640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d4005150 0x7f34d400fdd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f34e8001e50 tx=0x7f34d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b730e0cef7366a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.681+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8001a70 con 0x7f34d4005150 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.681+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8003970 con 0x7f34d4005150 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.681+0000 7f34eea00640 1 -- 172.21.15.19:0/880423230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8003d60 con 0x7f34d4005150 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.682+0000 7f34f5a00640 1 -- 172.21.15.19:0/880423230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d4005150 msgr2=0x7f34d400fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.682+0000 7f34f5a00640 1 --2- 172.21.15.19:0/880423230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d4005150 0x7f34d400fdd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f34e8001e50 tx=0x7f34d4005830 comp rx=0 tx=0).stop 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.682+0000 7f34f5000640 1 -- 172.21.15.19:0/880423230 reap_dead start 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.682+0000 7f34f5a00640 1 -- 172.21.15.19:0/880423230 shutdown_connections 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.682+0000 7f34f5a00640 1 -- 172.21.15.19:0/880423230 >> 172.21.15.19:0/880423230 conn(0x7f34f00d1720 msgr2=0x7f34f00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.682+0000 7f34f5a00640 1 -- 172.21.15.19:0/880423230 shutdown_connections 2024-04-25T11:34:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.682+0000 7f34f5a00640 1 -- 172.21.15.19:0/880423230 wait complete. 2024-04-25T11:34:48.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.684+0000 7f34f5a00640 1 Processor -- start 2024-04-25T11:34:48.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.684+0000 7f34f5a00640 1 -- start start 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34f5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d4005150 0x7f34f015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34f5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0162660 0x7f34f01f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34f5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34f01f5bc0 0x7f34f013b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34f5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34f00d0450 con 0x7f34f0162660 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34f5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34f00d02d0 con 0x7f34f01f5bc0 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34f5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34f0147760 con 0x7f34d4005150 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0162660 0x7f34f01f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0162660 0x7f34f01f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35400/0 (socket says 172.21.15.19:35400) 2024-04-25T11:34:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.686+0000 7f34ef400640 1 -- 172.21.15.19:0/2325519910 learned_addr learned my addr 172.21.15.19:0/2325519910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34f5000640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34f01f5bc0 0x7f34f013b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34efe00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d4005150 0x7f34f015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34ef400640 1 -- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d4005150 msgr2=0x7f34f015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34ef400640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d4005150 0x7f34f015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34ef400640 1 -- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34f01f5bc0 msgr2=0x7f34f013b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34ef400640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34f01f5bc0 0x7f34f013b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34ef400640 1 -- 172.21.15.19:0/2325519910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f01f5fd0 con 0x7f34f0162660 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34efe00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d4005150 0x7f34f015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34ef400640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0162660 0x7f34f01f5680 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0119d0 tx=0x7f34dc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34e7e00640 1 -- 172.21.15.19:0/2325519910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00fb50 con 0x7f34f0162660 2024-04-25T11:34:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34e7e00640 1 -- 172.21.15.19:0/2325519910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34dc00fcf0 con 0x7f34f0162660 2024-04-25T11:34:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34e7e00640 1 -- 172.21.15.19:0/2325519910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00fea0 con 0x7f34f0162660 2024-04-25T11:34:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.687+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f0144e70 con 0x7f34f0162660 2024-04-25T11:34:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.688+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f015c990 con 0x7f34f0162660 2024-04-25T11:34:48.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.690+0000 7f34e7e00640 1 -- 172.21.15.19:0/2325519910 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34dc012040 con 0x7f34f0162660 2024-04-25T11:34:48.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.691+0000 7f34e7e00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34d00744b0 0x7f34d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.691+0000 7f34e7e00640 1 -- 172.21.15.19:0/2325519910 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f34dc095a50 con 0x7f34f0162660 2024-04-25T11:34:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.692+0000 7f34fdfa4740 1 Processor -- start 2024-04-25T11:34:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.692+0000 7f34efe00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34d00744b0 0x7f34d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.692+0000 7f34fdfa4740 1 -- start start 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.693+0000 7f34efe00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34d00744b0 0x7f34d00769c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f34e0001bf0 tx=0x7f34e0000f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ac5ffb600 con 0x564ac5cd4590 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ac5f98100 con 0x564ac5ccc3b0 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564ac5ff75c0 con 0x564ac5cf5270 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60dd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35416/0 (socket says 172.21.15.19:35416) 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34f5000640 1 -- 172.21.15.19:0/2319829672 learned_addr learned my addr 172.21.15.19:0/2319829672 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34ef400640 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.694+0000 7f34efe00640 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34f5000640 1 -- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 msgr2=0x564ac60c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34f5000640 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34f5000640 1 -- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 msgr2=0x564ac60dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34f5000640 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34f5000640 1 -- 172.21.15.19:0/2319829672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac5d0d2d0 con 0x564ac5cd4590 2024-04-25T11:34:48.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34f5000640 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60dd480 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f34e8005040 tx=0x7f34e802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b265a8e7983840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34e4c00640 1 -- 172.21.15.19:0/2319829672 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8037040 con 0x564ac5cd4590 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34e4c00640 1 -- 172.21.15.19:0/2319829672 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8003970 con 0x564ac5cd4590 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.695+0000 7f34e4c00640 1 -- 172.21.15.19:0/2319829672 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8003d40 con 0x564ac5cd4590 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 -- 172.21.15.19:0/2319829672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 msgr2=0x564ac60dd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60dd480 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f34e8005040 tx=0x7f34e802c000 comp rx=0 tx=0).stop 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 -- 172.21.15.19:0/2319829672 shutdown_connections 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60dd480 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/2319829672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 -- 172.21.15.19:0/2319829672 >> 172.21.15.19:0/2319829672 conn(0x564ac5ce93c0 msgr2=0x564ac6097870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 -- 172.21.15.19:0/2319829672 shutdown_connections 2024-04-25T11:34:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.696+0000 7f34fdfa4740 1 -- 172.21.15.19:0/2319829672 wait complete. 2024-04-25T11:34:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.697+0000 7f34fdfa4740 1 -- start start 2024-04-25T11:34:48.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.697+0000 7f34fdfa4740 1 Processor -- start 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60ecab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ac5f82f70 con 0x564ac5cd4590 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ac60c4230 con 0x564ac5ccc3b0 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564ac60d79a0 con 0x564ac5cf5270 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34fdfa4740 10 client.0 _ll_register_callbacks cb 0x564ac5ff1480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35420/0 (socket says 172.21.15.19:35420) 2024-04-25T11:34:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.699+0000 7f34ef400640 1 -- 172.21.15.19:0/1035822184 learned_addr learned my addr 172.21.15.19:0/1035822184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34efe00640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60ecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34f5000640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34ef400640 1 -- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 msgr2=0x564ac60d9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34ef400640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60d9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34ef400640 1 -- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 msgr2=0x564ac60ecab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34ef400640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60ecab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34ef400640 1 -- 172.21.15.19:0/1035822184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac5dd76b0 con 0x564ac5cd4590 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34efe00640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60ecab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34ef400640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60c5b30 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00f610 tx=0x7f34dc068180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a6095658c65ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34d9600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34d9600640 1 -- 172.21.15.19:0/1035822184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc091060 con 0x564ac5cd4590 2024-04-25T11:34:48.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34d9600640 1 -- 172.21.15.19:0/1035822184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f34dc0660c0 con 0x564ac5cd4590 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.700+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e8003970 con 0x564ac5cd4590 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34d9600640 1 -- 172.21.15.19:0/1035822184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f34dc078020 con 0x564ac5cd4590 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 10 client.7152 fetch_fsmap learned FSMap version 81 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x564ac60c3f70 con 0x564ac5cd4590 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34d9600640 1 -- 172.21.15.19:0/1035822184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8003970 con 0x564ac5cd4590 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34d9600640 1 -- 172.21.15.19:0/1035822184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f34dc025030 con 0x564ac5cd4590 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 10 client.7152 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 10 client.7152 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 20 client.7152 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac5cb3e40 0x564ac60ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 4 client.7152 mds_command: new command op to 5845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.701+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 --> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ac6009210 con 0x564ac5cb3e40 2024-04-25T11:34:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.702+0000 7f34efe00640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac5cb3e40 0x564ac60ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.703+0000 7f34efe00640 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac5cb3e40 0x564ac60ef9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da16b6ccaa38d35 server_cookie=f3ac11927cbed7ee in_seq=0 out_seq=0 2024-04-25T11:34:48.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.703+0000 7f34d9600640 10 client.7152 ms_handle_connect on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:48.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.705+0000 7f34d9600640 1 -- 172.21.15.19:0/1035822184 <== mds.0 v2:172.21.15.169:6834/2118499768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ac6009210 con 0x564ac5cb3e40 2024-04-25T11:34:48.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.705+0000 7f34d9600640 10 client.7152 handle_command_reply: tid=0 2024-04-25T11:34:48.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.705+0000 7f34fdfa4740 1 client.7152 shutdown 2024-04-25T11:34:48.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.705+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 msgr2=0x564ac60c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.705+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60c5b30 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00f610 tx=0x7f34dc068180 comp rx=0 tx=0).stop 2024-04-25T11:34:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 shutdown_connections 2024-04-25T11:34:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac5cb3e40 0x564ac60ef9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cf5270 0x564ac60d9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60c5b30 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/1035822184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5ccc3b0 0x564ac60ecab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34d9600640 0 client.7152 ms_handle_reset on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 >> 172.21.15.19:0/1035822184 conn(0x564ac5ce93c0 msgr2=0x564ac60c2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 shutdown_connections 2024-04-25T11:34:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 1 -- 172.21.15.19:0/1035822184 wait complete. 2024-04-25T11:34:48.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.706+0000 7f34fdfa4740 20 client.7152 trim_cache size 0 max 16384 2024-04-25T11:34:48.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.726+0000 7f34fdfa4740 1 Processor -- start 2024-04-25T11:34:48.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.726+0000 7f34fdfa4740 1 -- start start 2024-04-25T11:34:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5ccc3b0 0x564ac60f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cd4590 0x564ac60c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 0x564ac60c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34f013d8f0 con 0x564ac5ccc3b0 2024-04-25T11:34:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34f01605d0 con 0x564ac5cf5270 2024-04-25T11:34:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34f013d2a0 con 0x564ac5cd4590 2024-04-25T11:34:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5ccc3b0 0x564ac60f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5ccc3b0 0x564ac60f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35434/0 (socket says 172.21.15.19:35434) 2024-04-25T11:34:48.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34efe00640 1 -- 172.21.15.19:0/3745218278 learned_addr learned my addr 172.21.15.19:0/3745218278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:48.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34f5000640 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 0x564ac60c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.728+0000 7f34ef400640 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cd4590 0x564ac60c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34efe00640 1 -- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cd4590 msgr2=0x564ac60c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34efe00640 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cd4590 0x564ac60c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34efe00640 1 -- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 msgr2=0x564ac60c09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34efe00640 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 0x564ac60c09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34efe00640 1 -- 172.21.15.19:0/3745218278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac60958c0 con 0x564ac5ccc3b0 2024-04-25T11:34:48.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34ef400640 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cd4590 0x564ac60c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:48.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34efe00640 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5ccc3b0 0x564ac60f5af0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x564ac60c53f0 tx=0x7f34e0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19a3a8e82918f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:48.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34dbe00640 1 -- 172.21.15.19:0/3745218278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e001f040 con 0x564ac5ccc3b0 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34dbe00640 1 -- 172.21.15.19:0/3745218278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e0014c00 con 0x564ac5ccc3b0 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.729+0000 7f34dbe00640 1 -- 172.21.15.19:0/3745218278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0010040 con 0x564ac5ccc3b0 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3745218278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5ccc3b0 msgr2=0x564ac60f5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5ccc3b0 0x564ac60f5af0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x564ac60c53f0 tx=0x7f34e0009000 comp rx=0 tx=0).stop 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3745218278 shutdown_connections 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 0x564ac60c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5cd4590 0x564ac60c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3745218278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5ccc3b0 0x564ac60f5af0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3745218278 >> 172.21.15.19:0/3745218278 conn(0x564ac5ce93c0 msgr2=0x564ac60c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3745218278 shutdown_connections 2024-04-25T11:34:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.730+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3745218278 wait complete. 2024-04-25T11:34:48.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.731+0000 7f34fdfa4740 1 -- start start 2024-04-25T11:34:48.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.731+0000 7f34fdfa4740 1 Processor -- start 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5ccc3b0 0x564ac6097f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34fdfa4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 0x564ac60ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ac60c32b0 con 0x564ac5cd4590 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ac60c60d0 con 0x564ac5cf5270 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34fdfa4740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564ac60ef490 con 0x564ac5ccc3b0 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34fdfa4740 10 client.0 _ll_register_callbacks cb 0x7f34dc05c0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60de830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35448/0 (socket says 172.21.15.19:35448) 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 -- 172.21.15.19:0/3187108463 learned_addr learned my addr 172.21.15.19:0/3187108463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34efe00640 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5ccc3b0 0x564ac6097f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 -- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5ccc3b0 msgr2=0x564ac6097f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5ccc3b0 0x564ac6097f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 -- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 msgr2=0x564ac60ded70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 0x564ac60ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.733+0000 7f34ef400640 1 -- 172.21.15.19:0/3187108463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac6098810 con 0x564ac5cd4590 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34ef400640 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60de830 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000c00 tx=0x7f34dc00f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f3b82467ae3c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34d9600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34d9600640 1 -- 172.21.15.19:0/3187108463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc086060 con 0x564ac5cd4590 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34d9600640 1 -- 172.21.15.19:0/3187108463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f34dc05ecd0 con 0x564ac5cd4590 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e0014c00 con 0x564ac5cd4590 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34d9600640 1 -- 172.21.15.19:0/3187108463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f34dc081170 con 0x564ac5cd4590 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34fdfa4740 10 client.7158 fetch_fsmap learned FSMap version 81 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x564ac60f62a0 con 0x564ac5cd4590 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34d9600640 1 -- 172.21.15.19:0/3187108463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e0014c00 con 0x564ac5cd4590 2024-04-25T11:34:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34d9600640 1 -- 172.21.15.19:0/3187108463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f34dc05b3e0 con 0x564ac5cd4590 2024-04-25T11:34:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34fdfa4740 10 client.7158 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T11:34:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.734+0000 7f34fdfa4740 10 client.7158 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:34:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.735+0000 7f34fdfa4740 20 client.7158 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.735+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac60bdf10 0x564ac60be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.735+0000 7f34fdfa4740 4 client.7158 mds_command: new command op to 5845 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:34:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.735+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 --> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] -- command(tid 0: {"prefix": "status"}) -- 0x564ac6003080 con 0x564ac60bdf10 2024-04-25T11:34:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.735+0000 7f34efe00640 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac60bdf10 0x564ac60be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.737+0000 7f34efe00640 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac60bdf10 0x564ac60be320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc223c2486c53e1 server_cookie=bddea5ba512d275c in_seq=0 out_seq=0 2024-04-25T11:34:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.737+0000 7f34d9600640 10 client.7158 ms_handle_connect on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.737+0000 7f34d9600640 1 -- 172.21.15.19:0/3187108463 <== mds.0 v2:172.21.15.169:6834/2118499768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564ac6003080 con 0x564ac60bdf10 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.737+0000 7f34d9600640 10 client.7158 handle_command_reply: tid=0 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 client.7158 shutdown 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 msgr2=0x564ac60de830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60de830 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000c00 tx=0x7f34dc00f7e0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 shutdown_connections 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ac5cf5270 0x564ac60ded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x564ac60bdf10 0x564ac60be320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ac5cd4590 0x564ac60de830 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 --2- 172.21.15.19:0/3187108463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ac5ccc3b0 0x564ac6097f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 >> 172.21.15.19:0/3187108463 conn(0x564ac5ce93c0 msgr2=0x564ac60e5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34d9600640 0 client.7158 ms_handle_reset on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 shutdown_connections 2024-04-25T11:34:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 1 -- 172.21.15.19:0/3187108463 wait complete. 2024-04-25T11:34:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.738+0000 7f34fdfa4740 20 client.7158 trim_cache size 0 max 16384 2024-04-25T11:34:48.730 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:34:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:34:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:34:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5845, 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.6402225279999998, 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 81, 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 80, 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 48.685225311000004 2024-04-25T11:34:48.731 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34d00744b0 msgr2=0x7f34d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34d00744b0 0x7f34d00769c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f34e0001bf0 tx=0x7f34e0000f90 comp rx=0 tx=0).stop 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0162660 msgr2=0x7f34f01f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0162660 0x7f34f01f5680 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0119d0 tx=0x7f34dc011e90 comp rx=0 tx=0).stop 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 shutdown_connections 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34d00744b0 0x7f34d00769c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34f01f5bc0 0x7f34f013b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34f0162660 0x7f34f01f5680 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 --2- 172.21.15.19:0/2325519910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34d4005150 0x7f34f015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 >> 172.21.15.19:0/2325519910 conn(0x7f34f00d1720 msgr2=0x7f34f00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 shutdown_connections 2024-04-25T11:34:48.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48.739+0000 7f34f5a00640 1 -- 172.21.15.19:0/2325519910 wait complete. 2024-04-25T11:34:48.754 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-04-25T11:34:48.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:34:48.863 INFO:teuthology.orchestra.run.smithi019.stdout: File: a/ 2024-04-25T11:34:48.863 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-04-25T11:34:48.863 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 2ah/42d Inode: 1099511627776 Links: 4 2024-04-25T11:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1316/ ubuntu) 2024-04-25T11:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-25T11:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:34:17.479117284 +0000 2024-04-25T11:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:34:23.592053844 +0000 2024-04-25T11:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:34:23.592053844 +0000 2024-04-25T11:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:34:17.479117284 +0000 2024-04-25T11:34:48.883 INFO:teuthology.orchestra.run.smithi019.stdout:a/ 2024-04-25T11:34:48.883 INFO:teuthology.orchestra.run.smithi019.stdout:a/z 2024-04-25T11:34:48.883 INFO:teuthology.orchestra.run.smithi019.stdout:a/z/d 2024-04-25T11:34:48.883 INFO:teuthology.orchestra.run.smithi019.stdout:a/c 2024-04-25T11:34:48.883 INFO:teuthology.orchestra.run.smithi019.stdout:a/c/d 2024-04-25T11:34:48.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:34:48.894 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-04-25T11:34:49.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.190+0000 7f1c05600640 1 Processor -- start 2024-04-25T11:34:49.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.191+0000 7f1c05600640 1 -- start start 2024-04-25T11:34:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.193+0000 7f1c05600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 0x7f1c000af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.193+0000 7f1c05600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1c00163c50 0x7f1c00164060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.193+0000 7f1c05600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c000b0110 con 0x7f1c001645a0 2024-04-25T11:34:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.193+0000 7f1c05600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1c000739b0 con 0x7f1c00163c50 2024-04-25T11:34:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.193+0000 7f1c05600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c000b03f0 con 0x7f1c00070b50 2024-04-25T11:34:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.194+0000 7f1c04c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1c001645a0 0x7f1c00160660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56594/0 (socket says 172.21.15.19:56594) 2024-04-25T11:34:49.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.194+0000 7f1c04c00640 1 -- 172.21.15.19:0/3922162560 learned_addr learned my addr 172.21.15.19:0/3922162560 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:49.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.194+0000 7f1bff400640 1 --2- 172.21.15.19:0/3922162560 >> v2:172.21.15.169:3300/0 conn(0x7f1c00163c50 0x7f1c00164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.194+0000 7f1bffe00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 0x7f1c000af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.194+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1278522122 0 0) 0x7f1c000b0110 con 0x7f1c001645a0 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.194+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1be4003520 con 0x7f1c001645a0 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2089588031 0 0) 0x7f1be4003520 con 0x7f1c001645a0 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 msgr2=0x7f1c000af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 0x7f1c000af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 >> v2:172.21.15.169:3300/0 conn(0x7f1c00163c50 msgr2=0x7f1c00164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 --2- 172.21.15.19:0/3922162560 >> v2:172.21.15.169:3300/0 conn(0x7f1c00163c50 0x7f1c00164060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c000b06d0 con 0x7f1c001645a0 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1bf80019f0 con 0x7f1c001645a0 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 >> v1:172.21.15.19:6789/0 conn(0x7f1c001645a0 legacy=0x7f1c00160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 0x7f1be40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4003f80 0x7f1be4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be4004d40 0x7f1be4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000739b0 con 0x7f1be4003f80 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000b03f0 con 0x7f1be4004d40 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1be400ba70 con 0x7f1c00070b50 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.195+0000 7f1bff400640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4003f80 0x7f1be4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1bffe00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 0x7f1be40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1c04c00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be4004d40 0x7f1be4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1bff400640 1 -- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 msgr2=0x7f1be40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1bff400640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 0x7f1be40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1bff400640 1 -- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be4004d40 msgr2=0x7f1be4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1bff400640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1be4004d40 0x7f1be4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1bff400640 1 -- 172.21.15.19:0/3922162560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be4003520 con 0x7f1be4003f80 2024-04-25T11:34:49.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.196+0000 7f1bffe00640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00070b50 0x7f1be40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.197+0000 7f1bff400640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4003f80 0x7f1be4004800 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009a90 tx=0x7f1be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9afc61c1f33abfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.198+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec036070 con 0x7f1be4003f80 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.198+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bec031400 con 0x7f1be4003f80 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.198+0000 7f1bfea00640 1 -- 172.21.15.19:0/3922162560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec009690 con 0x7f1be4003f80 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.198+0000 7f1c05600640 1 -- 172.21.15.19:0/3922162560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4003f80 msgr2=0x7f1be4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.198+0000 7f1c05600640 1 --2- 172.21.15.19:0/3922162560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be4003f80 0x7f1be4004800 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009a90 tx=0x7f1be4005830 comp rx=0 tx=0).stop 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.199+0000 7f1c04c00640 1 -- 172.21.15.19:0/3922162560 reap_dead start 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.199+0000 7f1c05600640 1 -- 172.21.15.19:0/3922162560 shutdown_connections 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.199+0000 7f1c05600640 1 -- 172.21.15.19:0/3922162560 >> 172.21.15.19:0/3922162560 conn(0x7f1c000b4260 msgr2=0x7f1c00072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:49.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.199+0000 7f1c05600640 1 -- 172.21.15.19:0/3922162560 shutdown_connections 2024-04-25T11:34:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.199+0000 7f1c05600640 1 -- 172.21.15.19:0/3922162560 wait complete. 2024-04-25T11:34:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.201+0000 7f1c05600640 1 Processor -- start 2024-04-25T11:34:49.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.201+0000 7f1c05600640 1 -- start start 2024-04-25T11:34:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c05600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00163350 0x7f1c0014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c05600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00150bf0 0x7f1c00151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c05600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015b8d0 0x7f1c0015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c05600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000b6a30 con 0x7f1c0015b8d0 2024-04-25T11:34:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c05600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001649b0 con 0x7f1c00163350 2024-04-25T11:34:49.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c05600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c000b6bb0 con 0x7f1c00150bf0 2024-04-25T11:34:49.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c04c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015b8d0 0x7f1c0015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c04c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015b8d0 0x7f1c0015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35464/0 (socket says 172.21.15.19:35464) 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.203+0000 7f1c04c00640 1 -- 172.21.15.19:0/3828603597 learned_addr learned my addr 172.21.15.19:0/3828603597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1bffe00640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00163350 0x7f1c0014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1bff400640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00150bf0 0x7f1c00151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1c04c00640 1 -- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00150bf0 msgr2=0x7f1c00151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1c04c00640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00150bf0 0x7f1c00151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1c04c00640 1 -- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00163350 msgr2=0x7f1c0014ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1c04c00640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00163350 0x7f1c0014ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1c04c00640 1 -- 172.21.15.19:0/3828603597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c00151540 con 0x7f1c0015b8d0 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.204+0000 7f1c04c00640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015b8d0 0x7f1c0015bce0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8001fd0 tx=0x7f1bf800e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.205+0000 7f1bf7e00640 1 -- 172.21.15.19:0/3828603597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800ea90 con 0x7f1c0015b8d0 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.205+0000 7f1bf7e00640 1 -- 172.21.15.19:0/3828603597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf8018070 con 0x7f1c0015b8d0 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.205+0000 7f1bf7e00640 1 -- 172.21.15.19:0/3828603597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800ee70 con 0x7f1c0015b8d0 2024-04-25T11:34:49.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.205+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c00163760 con 0x7f1c0015b8d0 2024-04-25T11:34:49.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.205+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c00157ea0 con 0x7f1c0015b8d0 2024-04-25T11:34:49.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.208+0000 7f1bf7e00640 1 -- 172.21.15.19:0/3828603597 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1bf801d460 con 0x7f1c0015b8d0 2024-04-25T11:34:49.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.209+0000 7f1bf7e00640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1be00744b0 0x7f1be00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.209+0000 7f1bf7e00640 1 -- 172.21.15.19:0/3828603597 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f1bf8095410 con 0x7f1c0015b8d0 2024-04-25T11:34:49.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.209+0000 7f1c0dd7e740 1 Processor -- start 2024-04-25T11:34:49.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.209+0000 7f1bffe00640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1be00744b0 0x7f1be00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.209+0000 7f1c0dd7e740 1 -- start start 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1bffe00640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1be00744b0 0x7f1be00769c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1c00073430 tx=0x7f1bf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaad10390 0x55bbab0eec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaacef6b0 0x55bbab0f9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bbaafc3030 con 0x55bbaad10390 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bbaafde7f0 con 0x55bbaacef6b0 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bbaaf997e0 con 0x55bbaace74d0 2024-04-25T11:34:49.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1bffe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaad10390 0x55bbab0eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1bffe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaad10390 0x55bbab0eec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35478/0 (socket says 172.21.15.19:35478) 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.211+0000 7f1bffe00640 1 -- 172.21.15.19:0/2667805086 learned_addr learned my addr 172.21.15.19:0/2667805086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1c04c00640 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaacef6b0 0x55bbab0f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bff400640 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bffe00640 1 -- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 msgr2=0x55bbab0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bffe00640 1 -- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaacef6b0 msgr2=0x55bbab0f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaacef6b0 0x55bbab0f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bffe00640 1 -- 172.21.15.19:0/2667805086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbaafa5050 con 0x55bbaad10390 2024-04-25T11:34:49.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaad10390 0x55bbab0eec30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0013d90 tx=0x7f1bf0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c01e868f285fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.212+0000 7f1bebe00640 1 -- 172.21.15.19:0/2667805086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0013dc0 con 0x55bbaad10390 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1bebe00640 1 -- 172.21.15.19:0/2667805086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf0028070 con 0x55bbaad10390 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1bebe00640 1 -- 172.21.15.19:0/2667805086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0023640 con 0x55bbaad10390 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2667805086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaad10390 msgr2=0x55bbab0eec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaad10390 0x55bbab0eec30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0013d90 tx=0x7f1bf0016000 comp rx=0 tx=0).stop 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2667805086 shutdown_connections 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaacef6b0 0x55bbab0f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2667805086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaad10390 0x55bbab0eec30 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.213+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2667805086 >> 172.21.15.19:0/2667805086 conn(0x55bbaad044e0 msgr2=0x55bbab0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.214+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2667805086 shutdown_connections 2024-04-25T11:34:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.214+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2667805086 wait complete. 2024-04-25T11:34:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.215+0000 7f1c0dd7e740 1 -- start start 2024-04-25T11:34:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.215+0000 7f1c0dd7e740 1 Processor -- start 2024-04-25T11:34:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0e0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bbab01b200 con 0x55bbaacef6b0 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bbab015ee0 con 0x55bbaad10390 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bbab022e70 con 0x55bbaace74d0 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c0dd7e740 10 client.0 _ll_register_callbacks cb 0x55bbab00c270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0e0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35490/0 (socket says 172.21.15.19:35490) 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 -- 172.21.15.19:0/2778333520 learned_addr learned my addr 172.21.15.19:0/2778333520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1c04c00640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 -- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 msgr2=0x55bbab0d9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0d9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 -- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 msgr2=0x55bbab0e0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0e0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bff400640 1 -- 172.21.15.19:0/2778333520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbaadf27d0 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.217+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.218+0000 7f1bff400640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0e0910 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1bec02abb0 tx=0x7f1bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38390e3f72ff75b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.218+0000 7f1be8c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.218+0000 7f1be8c00640 1 -- 172.21.15.19:0/2778333520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec041070 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.218+0000 7f1be8c00640 1 -- 172.21.15.19:0/2778333520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f1bec003810 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.218+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf0028070 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.218+0000 7f1be8c00640 1 -- 172.21.15.19:0/2778333520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1bec04c050 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 10 client.7170 fetch_fsmap learned FSMap version 81 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55bbaad283f0 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1be8c00640 1 -- 172.21.15.19:0/2778333520 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf0028070 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1be8c00640 1 -- 172.21.15.19:0/2778333520 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f1bec04f020 con 0x55bbaacef6b0 2024-04-25T11:34:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 10 client.7170 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T11:34:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 10 client.7170 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T11:34:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 20 client.7170 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbaaccef60 0x55bbab0f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 4 client.7170 mds_command: new command op to 5845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:34:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.219+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 --> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbab0247e0 con 0x55bbaaccef60 2024-04-25T11:34:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.220+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbaaccef60 0x55bbab0f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.221+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbaaccef60 0x55bbab0f5930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116c6ab283f0829f server_cookie=c29dfbaf197ab2f in_seq=0 out_seq=0 2024-04-25T11:34:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.221+0000 7f1be8c00640 10 client.7170 ms_handle_connect on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.223+0000 7f1be8c00640 1 -- 172.21.15.19:0/2778333520 <== mds.0 v2:172.21.15.169:6834/2118499768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbab0247e0 con 0x55bbaaccef60 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.223+0000 7f1be8c00640 10 client.7170 handle_command_reply: tid=0 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.223+0000 7f1c0dd7e740 1 client.7170 shutdown 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 msgr2=0x55bbab0e0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0e0910 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1bec02abb0 tx=0x7f1bec0046c0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 shutdown_connections 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0e0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbaaccef60 0x55bbab0f5930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0e0910 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2778333520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab0d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1be8c00640 0 client.7170 ms_handle_reset on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 >> 172.21.15.19:0/2778333520 conn(0x55bbaad044e0 msgr2=0x55bbab0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 shutdown_connections 2024-04-25T11:34:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2778333520 wait complete. 2024-04-25T11:34:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.224+0000 7f1c0dd7e740 20 client.7170 trim_cache size 0 max 16384 2024-04-25T11:34:49.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.241+0000 7f1c0dd7e740 1 Processor -- start 2024-04-25T11:34:49.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.241+0000 7f1c0dd7e740 1 -- start start 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaace74d0 0x55bbab0f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaad10390 0x55bbab0f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000ad0e0 con 0x55bbaacef6b0 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000ac0e0 con 0x55bbaace74d0 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c000ab440 con 0x55bbaad10390 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35506/0 (socket says 172.21.15.19:35506) 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 -- 172.21.15.19:0/2696431028 learned_addr learned my addr 172.21.15.19:0/2696431028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaace74d0 0x55bbab0f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 -- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaad10390 msgr2=0x55bbab0f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaad10390 0x55bbab0f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 -- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaace74d0 msgr2=0x55bbab0f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaace74d0 0x55bbab0f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bff400640 1 -- 172.21.15.19:0/2696431028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab0f5280 con 0x55bbaacef6b0 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.243+0000 7f1bffe00640 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaace74d0 0x55bbab0f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.244+0000 7f1bff400640 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0f5eb0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00f210 tx=0x7f1bec03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cd92774961e346 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.244+0000 7f1beb400640 1 -- 172.21.15.19:0/2696431028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec041040 con 0x55bbaacef6b0 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.244+0000 7f1beb400640 1 -- 172.21.15.19:0/2696431028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bec003610 con 0x55bbaacef6b0 2024-04-25T11:34:49.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.244+0000 7f1beb400640 1 -- 172.21.15.19:0/2696431028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec036040 con 0x55bbaacef6b0 2024-04-25T11:34:49.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2696431028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 msgr2=0x55bbab0f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0f5eb0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00f210 tx=0x7f1bec03e000 comp rx=0 tx=0).stop 2024-04-25T11:34:49.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2696431028 shutdown_connections 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaad10390 0x55bbab0f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab0f5eb0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/2696431028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaace74d0 0x55bbab0f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2696431028 >> 172.21.15.19:0/2696431028 conn(0x55bbaad044e0 msgr2=0x55bbab119b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2696431028 shutdown_connections 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.245+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/2696431028 wait complete. 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.246+0000 7f1c0dd7e740 1 -- start start 2024-04-25T11:34:49.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.246+0000 7f1c0dd7e740 1 Processor -- start 2024-04-25T11:34:49.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab122ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab1230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1c0dd7e740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bbab023fd0 con 0x55bbaacef6b0 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000aba90 con 0x55bbaad10390 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1c0dd7e740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bbab0dab90 con 0x55bbaace74d0 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1c0dd7e740 10 client.0 _ll_register_callbacks cb 0x7f1bec034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1bff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab1230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1bff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab1230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35522/0 (socket says 172.21.15.19:35522) 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.248+0000 7f1bff400640 1 -- 172.21.15.19:0/761379223 learned_addr learned my addr 172.21.15.19:0/761379223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bffe00640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab122ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1c04c00640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bff400640 1 -- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 msgr2=0x55bbab122ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bff400640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab122ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bff400640 1 -- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 msgr2=0x55bbab0f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bff400640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bff400640 1 -- 172.21.15.19:0/761379223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbaace80f0 con 0x55bbaacef6b0 2024-04-25T11:34:49.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1c04c00640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:49.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bffe00640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab122ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:49.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1bff400640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab1230e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009160 tx=0x7f1bec009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633e3984876d1bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1be8c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:34:49.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1be8c00640 1 -- 172.21.15.19:0/761379223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec055060 con 0x55bbaacef6b0 2024-04-25T11:34:49.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1be8c00640 1 -- 172.21.15.19:0/761379223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f1bec006030 con 0x55bbaacef6b0 2024-04-25T11:34:49.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.249+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bec003610 con 0x55bbaacef6b0 2024-04-25T11:34:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1be8c00640 1 -- 172.21.15.19:0/761379223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1bec05b050 con 0x55bbaacef6b0 2024-04-25T11:34:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 10 client.7176 fetch_fsmap learned FSMap version 81 2024-04-25T11:34:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55bbaafdea30 con 0x55bbaacef6b0 2024-04-25T11:34:49.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1be8c00640 1 -- 172.21.15.19:0/761379223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bec003610 con 0x55bbaacef6b0 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1be8c00640 1 -- 172.21.15.19:0/761379223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f1bec04f070 con 0x55bbaacef6b0 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 10 client.7176 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 10 client.7176 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 20 client.7176 populate_metadata read hostname 'smithi019' 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbab0aceb0 0x55bbab0f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 4 client.7176 mds_command: new command op to 5845 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.250+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 --> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbab01e650 con 0x55bbab0aceb0 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.251+0000 7f1bffe00640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbab0aceb0 0x55bbab0f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.252+0000 7f1bffe00640 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbab0aceb0 0x55bbab0f61b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb60644afd53c41 server_cookie=fa09fbff62da68f8 in_seq=0 out_seq=0 2024-04-25T11:34:49.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.252+0000 7f1be8c00640 10 client.7176 ms_handle_connect on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1be8c00640 1 -- 172.21.15.19:0/761379223 <== mds.0 v2:172.21.15.169:6834/2118499768 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbab01e650 con 0x55bbab0aceb0 2024-04-25T11:34:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1be8c00640 10 client.7176 handle_command_reply: tid=0 2024-04-25T11:34:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 client.7176 shutdown 2024-04-25T11:34:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 msgr2=0x55bbab1230e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab1230e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009160 tx=0x7f1bec009190 comp rx=0 tx=0).stop 2024-04-25T11:34:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 shutdown_connections 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bbaad10390 0x55bbab0f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:6834/2118499768,v1:172.21.15.169:6836/2118499768] conn(0x55bbab0aceb0 0x55bbab0f61b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbaacef6b0 0x55bbab1230e0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 --2- 172.21.15.19:0/761379223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bbaace74d0 0x55bbab122ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 >> 172.21.15.19:0/761379223 conn(0x55bbaad044e0 msgr2=0x55bbab0f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1be8c00640 0 client.7176 ms_handle_reset on v2:172.21.15.169:6834/2118499768 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 shutdown_connections 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 1 -- 172.21.15.19:0/761379223 wait complete. 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.281+0000 7f1c0dd7e740 20 client.7176 trim_cache size 0 max 16384 2024-04-25T11:34:49.271 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:34:49.272 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-04-25T11:34:49.272 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-04-25T11:34:49.272 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:34:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1be00744b0 msgr2=0x7f1be00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1be00744b0 0x7f1be00769c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1c00073430 tx=0x7f1bf0008040 comp rx=0 tx=0).stop 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015b8d0 msgr2=0x7f1c0015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015b8d0 0x7f1c0015bce0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8001fd0 tx=0x7f1bf800e160 comp rx=0 tx=0).stop 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 shutdown_connections 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1be00744b0 0x7f1be00769c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0015b8d0 0x7f1c0015bce0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c00150bf0 0x7f1c00151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 --2- 172.21.15.19:0/3828603597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c00163350 0x7f1c0014ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 >> 172.21.15.19:0/3828603597 conn(0x7f1c000b1500 msgr2=0x7f1c0006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 shutdown_connections 2024-04-25T11:34:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.282+0000 7f1c05600640 1 -- 172.21.15.19:0/3828603597 wait complete. 2024-04-25T11:34:49.300 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:34:49.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:34:49.411 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:34:49.525 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:34:49.526 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:34:49.526 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:34:49.526 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:34:49.586 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:34:49.586 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:34:49.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.936+0000 7f253f200640 1 Processor -- start 2024-04-25T11:34:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.936+0000 7f253f200640 1 -- start start 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.938+0000 7f253f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380cfb90 0x7f25380b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.938+0000 7f253f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25380d2030 0x7f25380b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.938+0000 7f253f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25380b3ef0 con 0x7f2538164aa0 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.938+0000 7f253f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2538164610 con 0x7f25380d2030 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.938+0000 7f253f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2538163fc0 con 0x7f25380cfb90 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.939+0000 7f253de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2538164aa0 0x7f25380d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56598/0 (socket says 172.21.15.19:56598) 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.939+0000 7f253de00640 1 -- 172.21.15.19:0/850109516 learned_addr learned my addr 172.21.15.19:0/850109516 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.939+0000 7f253e800640 1 --2- 172.21.15.19:0/850109516 >> v2:172.21.15.19:3300/0 conn(0x7f25380d2030 0x7f25380b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.939+0000 7f253d400640 1 --2- 172.21.15.19:0/850109516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380cfb90 0x7f25380b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.939+0000 7f253ca00640 1 -- 172.21.15.19:0/850109516 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3200373592 0 0) 0x7f25380b3ef0 con 0x7f2538164aa0 2024-04-25T11:34:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.939+0000 7f253ca00640 1 -- 172.21.15.19:0/850109516 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2520003520 con 0x7f2538164aa0 2024-04-25T11:34:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.940+0000 7f253e800640 1 -- 172.21.15.19:0/850109516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380cfb90 msgr2=0x7f25380b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.940+0000 7f253e800640 1 --2- 172.21.15.19:0/850109516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380cfb90 0x7f25380b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.940+0000 7f253e800640 1 -- 172.21.15.19:0/850109516 >> v1:172.21.15.19:6789/0 conn(0x7f2538164aa0 legacy=0x7f25380d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.940+0000 7f253e800640 1 -- 172.21.15.19:0/850109516 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25381642a0 con 0x7f25380d2030 2024-04-25T11:34:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.941+0000 7f253e800640 1 --2- 172.21.15.19:0/850109516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25380d2030 0x7f25380b3090 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2534009d90 tx=0x7f253402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f571d98a37f91a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.941+0000 7f253ca00640 1 -- 172.21.15.19:0/850109516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534007740 con 0x7f25380d2030 2024-04-25T11:34:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.941+0000 7f253ca00640 1 -- 172.21.15.19:0/850109516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25340078e0 con 0x7f25380d2030 2024-04-25T11:34:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.941+0000 7f253ca00640 1 -- 172.21.15.19:0/850109516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534007cb0 con 0x7f25380d2030 2024-04-25T11:34:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 -- 172.21.15.19:0/850109516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25380d2030 msgr2=0x7f25380b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 --2- 172.21.15.19:0/850109516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25380d2030 0x7f25380b3090 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2534009d90 tx=0x7f253402a660 comp rx=0 tx=0).stop 2024-04-25T11:34:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 -- 172.21.15.19:0/850109516 shutdown_connections 2024-04-25T11:34:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 --2- 172.21.15.19:0/850109516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25380d2030 0x7f25380b3090 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 --2- 172.21.15.19:0/850109516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380cfb90 0x7f25380b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 -- 172.21.15.19:0/850109516 >> 172.21.15.19:0/850109516 conn(0x7f253809b7d0 msgr2=0x7f253809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 -- 172.21.15.19:0/850109516 shutdown_connections 2024-04-25T11:34:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.942+0000 7f253f200640 1 -- 172.21.15.19:0/850109516 wait complete. 2024-04-25T11:34:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.945+0000 7f253f200640 1 Processor -- start 2024-04-25T11:34:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.945+0000 7f253f200640 1 -- start start 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.946+0000 7f253f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25380cfb90 0x7f25381eace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.946+0000 7f253f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380d2030 0x7f253814cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.946+0000 7f253f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2538164aa0 0x7f25381eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.946+0000 7f253f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25380b1a30 con 0x7f2538164aa0 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.946+0000 7f253f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25380b18b0 con 0x7f25380cfb90 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.946+0000 7f253f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f25380b1bb0 con 0x7f25380d2030 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2538164aa0 0x7f25381eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2538164aa0 0x7f25381eb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35554/0 (socket says 172.21.15.19:35554) 2024-04-25T11:34:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 -- 172.21.15.19:0/3159867124 learned_addr learned my addr 172.21.15.19:0/3159867124 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 -- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380d2030 msgr2=0x7f253814cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380d2030 0x7f253814cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 -- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25380cfb90 msgr2=0x7f25381eace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25380cfb90 0x7f25381eace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 -- 172.21.15.19:0/3159867124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253806d070 con 0x7f2538164aa0 2024-04-25T11:34:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.947+0000 7f253e800640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2538164aa0 0x7f25381eb8e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2534009eb0 tx=0x7f2534004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.948+0000 7f2532000640 1 -- 172.21.15.19:0/3159867124 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253403f070 con 0x7f2538164aa0 2024-04-25T11:34:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.948+0000 7f2532000640 1 -- 172.21.15.19:0/3159867124 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534031bc0 con 0x7f2538164aa0 2024-04-25T11:34:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.948+0000 7f2532000640 1 -- 172.21.15.19:0/3159867124 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534009450 con 0x7f2538164aa0 2024-04-25T11:34:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.948+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253806d350 con 0x7f2538164aa0 2024-04-25T11:34:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.948+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253806d910 con 0x7f2538164aa0 2024-04-25T11:34:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.949+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538050740 con 0x7f2538164aa0 2024-04-25T11:34:49.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.951+0000 7f2532000640 1 -- 172.21.15.19:0/3159867124 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f253403b410 con 0x7f2538164aa0 2024-04-25T11:34:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.952+0000 7f2532000640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25180744b0 0x7f25180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:49.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.952+0000 7f2532000640 1 -- 172.21.15.19:0/3159867124 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f25340b6750 con 0x7f2538164aa0 2024-04-25T11:34:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.953+0000 7f253de00640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25180744b0 0x7f25180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:49.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.955+0000 7f253de00640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25180744b0 0x7f25180769c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f252c000f60 tx=0x7f252c0078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:49.958+0000 7f2532000640 1 -- 172.21.15.19:0/3159867124 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f253404b030 con 0x7f2538164aa0 2024-04-25T11:34:50.131 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-25T11:34:50.143+0000 7f9eca600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:34:50.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.163+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f253806e840 con 0x7f2538164aa0 2024-04-25T11:34:50.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.171+0000 7f2532000640 1 -- 172.21.15.19:0/3159867124 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16735 (secure 0 0 0) 0x7f2534080790 con 0x7f2538164aa0 2024-04-25T11:34:50.161 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:50.161 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:43.091776+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:34:04.018735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T11:34:04.759136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/853310115":"2024-04-26T11:34:43.091612+0000","172.21.15.19:6834/853310115":"2024-04-26T11:34:43.091612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:50.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25180744b0 msgr2=0x7f25180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25180744b0 0x7f25180769c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f252c000f60 tx=0x7f252c0078f0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2538164aa0 msgr2=0x7f25381eb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2538164aa0 0x7f25381eb8e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2534009eb0 tx=0x7f2534004a40 comp rx=0 tx=0).stop 2024-04-25T11:34:50.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 shutdown_connections 2024-04-25T11:34:50.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25180744b0 0x7f25180769c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2538164aa0 0x7f25381eb8e0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25380d2030 0x7f253814cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 --2- 172.21.15.19:0/3159867124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25380cfb90 0x7f25381eace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 >> 172.21.15.19:0/3159867124 conn(0x7f253809b7d0 msgr2=0x7f2538070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:50.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 shutdown_connections 2024-04-25T11:34:50.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.175+0000 7f253f200640 1 -- 172.21.15.19:0/3159867124 wait complete. 2024-04-25T11:34:50.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:34:04.018735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:34:04.759136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:34:50.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:34:50.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.497+0000 7fdc79000640 1 Processor -- start 2024-04-25T11:34:50.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.497+0000 7fdc79000640 1 -- start start 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.499+0000 7fdc79000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc740ce520 0x7fdc7415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc79000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc7406dce0 0x7fdc7415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc79000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc740cda60 con 0x7fdc740ce930 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc79000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdc7415dfe0 con 0x7fdc740ce520 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc79000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7415e2c0 con 0x7fdc7406dce0 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc73400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc740ce520 0x7fdc7415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc73400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc740ce520 0x7fdc7415d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35558/0 (socket says 172.21.15.19:35558) 2024-04-25T11:34:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc73400640 1 -- 172.21.15.19:0/1806523353 learned_addr learned my addr 172.21.15.19:0/1806523353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc72a00640 1 --1- 172.21.15.19:0/1806523353 >> v1:172.21.15.19:6789/0 conn(0x7fdc740ce930 0x7fdc7406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56610/0 (socket says 172.21.15.19:56610) 2024-04-25T11:34:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc73400640 1 -- 172.21.15.19:0/1806523353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc7406dce0 msgr2=0x7fdc7415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc73400640 1 --2- 172.21.15.19:0/1806523353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc7406dce0 0x7fdc7415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc73400640 1 -- 172.21.15.19:0/1806523353 >> v1:172.21.15.19:6789/0 conn(0x7fdc740ce930 legacy=0x7fdc7406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.500+0000 7fdc73400640 1 -- 172.21.15.19:0/1806523353 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc740cdd20 con 0x7fdc740ce520 2024-04-25T11:34:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.502+0000 7fdc73400640 1 --2- 172.21.15.19:0/1806523353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc7415d4b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009870 tx=0x7fdc6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eab050e610642cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.502+0000 7fdc72000640 1 -- 172.21.15.19:0/1806523353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64037070 con 0x7fdc740ce520 2024-04-25T11:34:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.502+0000 7fdc72000640 1 -- 172.21.15.19:0/1806523353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc64032440 con 0x7fdc740ce520 2024-04-25T11:34:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.502+0000 7fdc72000640 1 -- 172.21.15.19:0/1806523353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64032810 con 0x7fdc740ce520 2024-04-25T11:34:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 -- 172.21.15.19:0/1806523353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 msgr2=0x7fdc7415d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 --2- 172.21.15.19:0/1806523353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc7415d4b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009870 tx=0x7fdc6402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 -- 172.21.15.19:0/1806523353 shutdown_connections 2024-04-25T11:34:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 --2- 172.21.15.19:0/1806523353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc7406dce0 0x7fdc7415d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 --2- 172.21.15.19:0/1806523353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc7415d4b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 -- 172.21.15.19:0/1806523353 >> 172.21.15.19:0/1806523353 conn(0x7fdc74072f40 msgr2=0x7fdc74073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 -- 172.21.15.19:0/1806523353 shutdown_connections 2024-04-25T11:34:50.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.503+0000 7fdc79000640 1 -- 172.21.15.19:0/1806523353 wait complete. 2024-04-25T11:34:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.505+0000 7fdc79000640 1 Processor -- start 2024-04-25T11:34:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.506+0000 7fdc79000640 1 -- start start 2024-04-25T11:34:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.507+0000 7fdc79000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc74144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.507+0000 7fdc79000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc740ce930 0x7fdc740d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.507+0000 7fdc79000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc740d11e0 0x7fdc74158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.507+0000 7fdc79000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc740b73d0 con 0x7fdc740ce520 2024-04-25T11:34:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.507+0000 7fdc79000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74160d30 con 0x7fdc740d11e0 2024-04-25T11:34:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.507+0000 7fdc79000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdc740b7550 con 0x7fdc740ce930 2024-04-25T11:34:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc74144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc74144cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35560/0 (socket says 172.21.15.19:35560) 2024-04-25T11:34:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 -- 172.21.15.19:0/4047487430 learned_addr learned my addr 172.21.15.19:0/4047487430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc72a00640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc740ce930 0x7fdc740d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73e00640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc740d11e0 0x7fdc74158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 -- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc740ce930 msgr2=0x7fdc740d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc740ce930 0x7fdc740d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 -- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc740d11e0 msgr2=0x7fdc74158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc740d11e0 0x7fdc74158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.508+0000 7fdc73400640 1 -- 172.21.15.19:0/4047487430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc74151550 con 0x7fdc740ce520 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.509+0000 7fdc73400640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc74144cb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009870 tx=0x7fdc640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.509+0000 7fdc6b400640 1 -- 172.21.15.19:0/4047487430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64041070 con 0x7fdc740ce520 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.509+0000 7fdc6b400640 1 -- 172.21.15.19:0/4047487430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6403c870 con 0x7fdc740ce520 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.509+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc74150f90 con 0x7fdc740ce520 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.509+0000 7fdc6b400640 1 -- 172.21.15.19:0/4047487430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64037040 con 0x7fdc740ce520 2024-04-25T11:34:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.509+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7414d570 con 0x7fdc740ce520 2024-04-25T11:34:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.510+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc74076c60 con 0x7fdc740ce520 2024-04-25T11:34:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.512+0000 7fdc6b400640 1 -- 172.21.15.19:0/4047487430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc64044070 con 0x7fdc740ce520 2024-04-25T11:34:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.513+0000 7fdc6b400640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc500744e0 0x7fdc500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.513+0000 7fdc6b400640 1 -- 172.21.15.19:0/4047487430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fdc640b5eb0 con 0x7fdc740ce520 2024-04-25T11:34:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.513+0000 7fdc72a00640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc500744e0 0x7fdc500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:50.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.515+0000 7fdc72a00640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc500744e0 0x7fdc500769f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdc74157ad0 tx=0x7fdc5c007c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:50.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.521+0000 7fdc6b400640 1 -- 172.21.15.19:0/4047487430 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc640805d0 con 0x7fdc740ce520 2024-04-25T11:34:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.727+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc740b5480 con 0x7fdc740ce520 2024-04-25T11:34:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.728+0000 7fdc6b400640 1 -- 172.21.15.19:0/4047487430 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fdc6407fed0 con 0x7fdc740ce520 2024-04-25T11:34:50.717 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:34:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc500744e0 msgr2=0x7fdc500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc500744e0 0x7fdc500769f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdc74157ad0 tx=0x7fdc5c007c40 comp rx=0 tx=0).stop 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 msgr2=0x7fdc74144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc74144cb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009870 tx=0x7fdc640046c0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 shutdown_connections 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdc500744e0 0x7fdc500769f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdc740d11e0 0x7fdc74158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdc740ce930 0x7fdc740d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 --2- 172.21.15.19:0/4047487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc740ce520 0x7fdc74144cb0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 >> 172.21.15.19:0/4047487430 conn(0x7fdc74072f40 msgr2=0x7fdc7406d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 shutdown_connections 2024-04-25T11:34:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:50.730+0000 7fdc79000640 1 -- 172.21.15.19:0/4047487430 wait complete. 2024-04-25T11:34:50.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:34:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.023+0000 7fd45d600640 1 Processor -- start 2024-04-25T11:34:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.023+0000 7fd45d600640 1 -- start start 2024-04-25T11:34:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.025+0000 7fd45d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd458161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd45d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd458162040 0x7fd45806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd45d600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd458162450 con 0x7fd45806cb70 2024-04-25T11:34:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd45d600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd45806d3a0 con 0x7fd458162040 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd45d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd45806d680 con 0x7fd4581441d0 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd457400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd458162040 0x7fd45806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd457400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd458162040 0x7fd45806c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35564/0 (socket says 172.21.15.19:35564) 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd457400640 1 -- 172.21.15.19:0/293263209 learned_addr learned my addr 172.21.15.19:0/293263209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.026+0000 7fd457e00640 1 --2- 172.21.15.19:0/293263209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd458161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.027+0000 7fd457400640 1 -- 172.21.15.19:0/293263209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 msgr2=0x7fd458161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.027+0000 7fd457400640 1 --2- 172.21.15.19:0/293263209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd458161b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.027+0000 7fd457400640 1 -- 172.21.15.19:0/293263209 >> v1:172.21.15.169:6789/0 conn(0x7fd45806cb70 legacy=0x7fd45815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.027+0000 7fd457400640 1 -- 172.21.15.19:0/293263209 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45806d960 con 0x7fd458162040 2024-04-25T11:34:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.027+0000 7fd457e00640 1 --2- 172.21.15.19:0/293263209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd458161b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:34:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.028+0000 7fd457400640 1 --2- 172.21.15.19:0/293263209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd458162040 0x7fd45806c630 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c07674203fc5622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.028+0000 7fd456a00640 1 -- 172.21.15.19:0/293263209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440037070 con 0x7fd458162040 2024-04-25T11:34:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.028+0000 7fd456a00640 1 -- 172.21.15.19:0/293263209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440032440 con 0x7fd458162040 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.028+0000 7fd456a00640 1 -- 172.21.15.19:0/293263209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440032810 con 0x7fd458162040 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.029+0000 7fd45d600640 1 -- 172.21.15.19:0/293263209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd458162040 msgr2=0x7fd45806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.029+0000 7fd45d600640 1 --2- 172.21.15.19:0/293263209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd458162040 0x7fd45806c630 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a180 comp rx=0 tx=0).stop 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.029+0000 7fd45d600640 1 -- 172.21.15.19:0/293263209 shutdown_connections 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.029+0000 7fd45d600640 1 --2- 172.21.15.19:0/293263209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd458162040 0x7fd45806c630 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.029+0000 7fd45d600640 1 --2- 172.21.15.19:0/293263209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd458161b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.029+0000 7fd45d600640 1 -- 172.21.15.19:0/293263209 >> 172.21.15.19:0/293263209 conn(0x7fd458072f40 msgr2=0x7fd458073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.029+0000 7fd45d600640 1 -- 172.21.15.19:0/293263209 shutdown_connections 2024-04-25T11:34:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.030+0000 7fd45d600640 1 -- 172.21.15.19:0/293263209 wait complete. 2024-04-25T11:34:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.032+0000 7fd45d600640 1 Processor -- start 2024-04-25T11:34:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.033+0000 7fd45d600640 1 -- start start 2024-04-25T11:34:51.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd45d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45806cb70 0x7fd4580afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd45d600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd45815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd45d600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd458162040 0x7fd4581f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd45d600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd45806cf80 con 0x7fd45806cb70 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd45d600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4581445e0 con 0x7fd458162040 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd45d600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd45806d100 con 0x7fd4581441d0 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45806cb70 0x7fd4580afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45806cb70 0x7fd4580afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35566/0 (socket says 172.21.15.19:35566) 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 -- 172.21.15.19:0/1489993068 learned_addr learned my addr 172.21.15.19:0/1489993068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 -- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 msgr2=0x7fd45815ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457400640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd45815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd45815ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 -- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd458162040 msgr2=0x7fd4581f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd458162040 0x7fd4581f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.035+0000 7fd457e00640 1 -- 172.21.15.19:0/1489993068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45815b3a0 con 0x7fd45806cb70 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.036+0000 7fd457e00640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45806cb70 0x7fd4580afd10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd44800d7d0 tx=0x7fd44800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.036+0000 7fd44fe00640 1 -- 172.21.15.19:0/1489993068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448004030 con 0x7fd45806cb70 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.036+0000 7fd44fe00640 1 -- 172.21.15.19:0/1489993068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4480041d0 con 0x7fd45806cb70 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.036+0000 7fd44fe00640 1 -- 172.21.15.19:0/1489993068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448009640 con 0x7fd45806cb70 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.036+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4580b0560 con 0x7fd45806cb70 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.036+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4580b0b20 con 0x7fd45806cb70 2024-04-25T11:34:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.037+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd458076c60 con 0x7fd45806cb70 2024-04-25T11:34:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.039+0000 7fd44fe00640 1 -- 172.21.15.19:0/1489993068 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4480097e0 con 0x7fd45806cb70 2024-04-25T11:34:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.040+0000 7fd44fe00640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4340744e0 0x7fd4340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.040+0000 7fd44fe00640 1 -- 172.21.15.19:0/1489993068 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fd448095490 con 0x7fd45806cb70 2024-04-25T11:34:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.040+0000 7fd457400640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4340744e0 0x7fd4340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.042+0000 7fd457400640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4340744e0 0x7fd4340769f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd44002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.045+0000 7fd44fe00640 1 -- 172.21.15.19:0/1489993068 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd44805fbb0 con 0x7fd45806cb70 2024-04-25T11:34:51.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.242+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4580707e0 con 0x7fd45806cb70 2024-04-25T11:34:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.243+0000 7fd44fe00640 1 -- 172.21.15.19:0/1489993068 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7fd44805f4b0 con 0x7fd45806cb70 2024-04-25T11:34:51.233 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:34:51.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4340744e0 msgr2=0x7fd4340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4340744e0 0x7fd4340769f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd440034000 tx=0x7fd44002e000 comp rx=0 tx=0).stop 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45806cb70 msgr2=0x7fd4580afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45806cb70 0x7fd4580afd10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd44800d7d0 tx=0x7fd44800dc90 comp rx=0 tx=0).stop 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 shutdown_connections 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4340744e0 0x7fd4340769f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd458162040 0x7fd4581f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4581441d0 0x7fd45815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 --2- 172.21.15.19:0/1489993068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd45806cb70 0x7fd4580afd10 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 >> 172.21.15.19:0/1489993068 conn(0x7fd458072f40 msgr2=0x7fd4580b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 shutdown_connections 2024-04-25T11:34:51.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.246+0000 7fd45d600640 1 -- 172.21.15.19:0/1489993068 wait complete. 2024-04-25T11:34:51.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:34:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.543+0000 7fd313e00640 1 Processor -- start 2024-04-25T11:34:51.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.543+0000 7fd313e00640 1 -- start start 2024-04-25T11:34:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd313e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd314070d20 0x7fd314161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd313e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd31406dce0 0x7fd3140ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd313e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3141623f0 con 0x7fd31406f930 2024-04-25T11:34:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd313e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd3141626d0 con 0x7fd31406dce0 2024-04-25T11:34:51.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd313e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd31406f130 con 0x7fd314070d20 2024-04-25T11:34:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd312a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd314070d20 0x7fd314161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd312a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd314070d20 0x7fd314161b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:45270/0 (socket says 172.21.15.19:45270) 2024-04-25T11:34:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.546+0000 7fd312a00640 1 -- 172.21.15.19:0/4160296907 learned_addr learned my addr 172.21.15.19:0/4160296907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.547+0000 7fd313400640 1 --2- 172.21.15.19:0/4160296907 >> v2:172.21.15.169:3300/0 conn(0x7fd31406dce0 0x7fd3140ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.547+0000 7fd313400640 1 -- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd314070d20 msgr2=0x7fd314161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.547+0000 7fd313400640 1 --2- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd314070d20 0x7fd314161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.547+0000 7fd313400640 1 -- 172.21.15.19:0/4160296907 >> v1:172.21.15.169:6789/0 conn(0x7fd31406f930 legacy=0x7fd31406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.547+0000 7fd313400640 1 -- 172.21.15.19:0/4160296907 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31406f410 con 0x7fd31406dce0 2024-04-25T11:34:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.547+0000 7fd312a00640 1 --2- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd314070d20 0x7fd314161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:34:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.549+0000 7fd313400640 1 --2- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31406dce0 0x7fd3140ce7e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009d90 tx=0x7fd30c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0993e2cd7506c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.549+0000 7fd311600640 1 -- 172.21.15.19:0/4160296907 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c007740 con 0x7fd31406dce0 2024-04-25T11:34:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.549+0000 7fd311600640 1 -- 172.21.15.19:0/4160296907 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30c0078e0 con 0x7fd31406dce0 2024-04-25T11:34:51.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.549+0000 7fd311600640 1 -- 172.21.15.19:0/4160296907 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c007cb0 con 0x7fd31406dce0 2024-04-25T11:34:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 -- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31406dce0 msgr2=0x7fd3140ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 --2- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31406dce0 0x7fd3140ce7e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009d90 tx=0x7fd30c02a5f0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 -- 172.21.15.19:0/4160296907 shutdown_connections 2024-04-25T11:34:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 --2- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd31406dce0 0x7fd3140ce7e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 --2- 172.21.15.19:0/4160296907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd314070d20 0x7fd314161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 -- 172.21.15.19:0/4160296907 >> 172.21.15.19:0/4160296907 conn(0x7fd314073160 msgr2=0x7fd314073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 -- 172.21.15.19:0/4160296907 shutdown_connections 2024-04-25T11:34:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.550+0000 7fd313e00640 1 -- 172.21.15.19:0/4160296907 wait complete. 2024-04-25T11:34:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.553+0000 7fd313e00640 1 Processor -- start 2024-04-25T11:34:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.553+0000 7fd313e00640 1 -- start start 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.554+0000 7fd313e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd31406f930 0x7fd3140caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.554+0000 7fd313e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd314070d20 0x7fd3141f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.554+0000 7fd313e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3141f6370 0x7fd3140ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.554+0000 7fd313e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd31415e2f0 con 0x7fd31406f930 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.554+0000 7fd313e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd314162070 con 0x7fd314070d20 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.554+0000 7fd313e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd31415e470 con 0x7fd3141f6370 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd31406f930 0x7fd3140caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd31406f930 0x7fd3140caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35568/0 (socket says 172.21.15.19:35568) 2024-04-25T11:34:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 -- 172.21.15.19:0/2917314846 learned_addr learned my addr 172.21.15.19:0/2917314846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd313400640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3141f6370 0x7fd3140ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312000640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd314070d20 0x7fd3141f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 -- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3141f6370 msgr2=0x7fd3140ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3141f6370 0x7fd3140ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 -- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd314070d20 msgr2=0x7fd3141f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd314070d20 0x7fd3141f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.555+0000 7fd312a00640 1 -- 172.21.15.19:0/2917314846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3141f6b20 con 0x7fd31406f930 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.556+0000 7fd312a00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd31406f930 0x7fd3140caeb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd30400e900 tx=0x7fd30400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.556+0000 7fd30aa00640 1 -- 172.21.15.19:0/2917314846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304017070 con 0x7fd31406f930 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.556+0000 7fd30aa00640 1 -- 172.21.15.19:0/2917314846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30400ca00 con 0x7fd31406f930 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.556+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31415f7e0 con 0x7fd31406f930 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.556+0000 7fd30aa00640 1 -- 172.21.15.19:0/2917314846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30400cdf0 con 0x7fd31406f930 2024-04-25T11:34:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.556+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31415fda0 con 0x7fd31406f930 2024-04-25T11:34:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.558+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd314076c60 con 0x7fd31406f930 2024-04-25T11:34:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.559+0000 7fd30aa00640 1 -- 172.21.15.19:0/2917314846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd30401a070 con 0x7fd31406f930 2024-04-25T11:34:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.560+0000 7fd30aa00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2f40744b0 0x7fd2f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.561+0000 7fd30aa00640 1 -- 172.21.15.19:0/2917314846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fd304095560 con 0x7fd31406f930 2024-04-25T11:34:51.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.561+0000 7fd312000640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2f40744b0 0x7fd2f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.563+0000 7fd312000640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2f40744b0 0x7fd2f40769c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd300007fd0 tx=0x7fd300009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.565+0000 7fd30aa00640 1 -- 172.21.15.19:0/2917314846 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd30405fc70 con 0x7fd31406f930 2024-04-25T11:34:51.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.767+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd314074070 con 0x7fd31406f930 2024-04-25T11:34:51.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.768+0000 7fd30aa00640 1 -- 172.21.15.19:0/2917314846 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fd30405f570 con 0x7fd31406f930 2024-04-25T11:34:51.757 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:34:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2f40744b0 msgr2=0x7fd2f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2f40744b0 0x7fd2f40769c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd300007fd0 tx=0x7fd300009040 comp rx=0 tx=0).stop 2024-04-25T11:34:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd31406f930 msgr2=0x7fd3140caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd31406f930 0x7fd3140caeb0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd30400e900 tx=0x7fd30400edc0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 shutdown_connections 2024-04-25T11:34:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd2f40744b0 0x7fd2f40769c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd314070d20 0x7fd3141f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3141f6370 0x7fd3140ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 --2- 172.21.15.19:0/2917314846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd31406f930 0x7fd3140caeb0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 >> 172.21.15.19:0/2917314846 conn(0x7fd314073160 msgr2=0x7fd3140b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 shutdown_connections 2024-04-25T11:34:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:51.771+0000 7fd313e00640 1 -- 172.21.15.19:0/2917314846 wait complete. 2024-04-25T11:34:51.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:34:52.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.085+0000 7fbbf5600640 1 Processor -- start 2024-04-25T11:34:52.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.085+0000 7fbbf5600640 1 -- start start 2024-04-25T11:34:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbf5600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbf0144370 0x7fbbf00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbf5600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf00d1230 con 0x7fbbf00b4ea0 2024-04-25T11:34:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbf5600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf00d06b0 con 0x7fbbf006c5c0 2024-04-25T11:34:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbf5600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf00d0990 con 0x7fbbf0144370 2024-04-25T11:34:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbef400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbbf00b4ea0 0x7fbbf015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56626/0 (socket says 172.21.15.19:56626) 2024-04-25T11:34:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbef400640 1 -- 172.21.15.19:0/3261770608 learned_addr learned my addr 172.21.15.19:0/3261770608 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbf4c00640 1 --2- 172.21.15.19:0/3261770608 >> v2:172.21.15.19:3300/0 conn(0x7fbbf0144370 0x7fbbf00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:52.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.088+0000 7fbbeea00640 1 -- 172.21.15.19:0/3261770608 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181274536 0 0) 0x7fbbf00d1230 con 0x7fbbf00b4ea0 2024-04-25T11:34:52.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.089+0000 7fbbeea00640 1 -- 172.21.15.19:0/3261770608 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbd4003520 con 0x7fbbf00b4ea0 2024-04-25T11:34:52.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.089+0000 7fbbf4c00640 1 -- 172.21.15.19:0/3261770608 >> v1:172.21.15.169:6789/0 conn(0x7fbbf006c5c0 legacy=0x7fbbf00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:52.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.089+0000 7fbbf4c00640 1 -- 172.21.15.19:0/3261770608 >> v1:172.21.15.19:6789/0 conn(0x7fbbf00b4ea0 legacy=0x7fbbf015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.089+0000 7fbbf4c00640 1 -- 172.21.15.19:0/3261770608 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf00d0c70 con 0x7fbbf0144370 2024-04-25T11:34:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.090+0000 7fbbf4c00640 1 --2- 172.21.15.19:0/3261770608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf00cfa70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009d90 tx=0x7fbbe802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23faa4e1a8d3b5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.090+0000 7fbbeea00640 1 -- 172.21.15.19:0/3261770608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8007740 con 0x7fbbf0144370 2024-04-25T11:34:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.090+0000 7fbbeea00640 1 -- 172.21.15.19:0/3261770608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe80078e0 con 0x7fbbf0144370 2024-04-25T11:34:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.090+0000 7fbbeea00640 1 -- 172.21.15.19:0/3261770608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8007cb0 con 0x7fbbf0144370 2024-04-25T11:34:52.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.091+0000 7fbbf5600640 1 -- 172.21.15.19:0/3261770608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 msgr2=0x7fbbf00cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:52.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.091+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3261770608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf00cfa70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009d90 tx=0x7fbbe802a6f0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.091+0000 7fbbf5600640 1 -- 172.21.15.19:0/3261770608 shutdown_connections 2024-04-25T11:34:52.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.091+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3261770608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf00cfa70 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.091+0000 7fbbf5600640 1 -- 172.21.15.19:0/3261770608 >> 172.21.15.19:0/3261770608 conn(0x7fbbf00d1720 msgr2=0x7fbbf00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:52.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.091+0000 7fbbf5600640 1 -- 172.21.15.19:0/3261770608 shutdown_connections 2024-04-25T11:34:52.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.091+0000 7fbbf5600640 1 -- 172.21.15.19:0/3261770608 wait complete. 2024-04-25T11:34:52.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.093+0000 7fbbf5600640 1 Processor -- start 2024-04-25T11:34:52.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.094+0000 7fbbf5600640 1 -- start start 2024-04-25T11:34:52.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.095+0000 7fbbf5600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbf006c5c0 0x7fbbf00b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.095+0000 7fbbf5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.095+0000 7fbbf5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbf0161db0 0x7fbbf01621c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.095+0000 7fbbf5600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf00d0450 con 0x7fbbf0144370 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.095+0000 7fbbf5600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf00d02d0 con 0x7fbbf006c5c0 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.095+0000 7fbbf5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf0147760 con 0x7fbbf0161db0 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.095+0000 7fbbef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf015d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35576/0 (socket says 172.21.15.19:35576) 2024-04-25T11:34:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 -- 172.21.15.19:0/3544416379 learned_addr learned my addr 172.21.15.19:0/3544416379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:52.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 -- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbf0161db0 msgr2=0x7fbbf01621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:52.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbf0161db0 0x7fbbf01621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 -- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbf006c5c0 msgr2=0x7fbbf00b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:52.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbf006c5c0 0x7fbbf00b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 -- 172.21.15.19:0/3544416379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf0162700 con 0x7fbbf0144370 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.096+0000 7fbbef400640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf015d5a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c410 tx=0x7fbbdc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.097+0000 7fbbe7e00640 1 -- 172.21.15.19:0/3544416379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc017070 con 0x7fbbf0144370 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.097+0000 7fbbe7e00640 1 -- 172.21.15.19:0/3544416379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbdc005ce0 con 0x7fbbf0144370 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.097+0000 7fbbe7e00640 1 -- 172.21.15.19:0/3544416379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0072f0 con 0x7fbbf0144370 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.097+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf01f9890 con 0x7fbbf0144370 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.097+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01f9e50 con 0x7fbbf0144370 2024-04-25T11:34:52.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.098+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf0050740 con 0x7fbbf0144370 2024-04-25T11:34:52.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.099+0000 7fbbe7e00640 1 -- 172.21.15.19:0/3544416379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbbdc01a070 con 0x7fbbf0144370 2024-04-25T11:34:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.100+0000 7fbbe7e00640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbbd00744b0 0x7fbbd00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:52.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.100+0000 7fbbe7e00640 1 -- 172.21.15.19:0/3544416379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fbbdc0947f0 con 0x7fbbf0144370 2024-04-25T11:34:52.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.101+0000 7fbbefe00640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbbd00744b0 0x7fbbd00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:52.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.102+0000 7fbbefe00640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbbd00744b0 0x7fbbd00769c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0000f60 tx=0x7fbbe00079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:52.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.107+0000 7fbbe7e00640 1 -- 172.21.15.19:0/3544416379 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbbf0162700 con 0x7fbbf0144370 2024-04-25T11:34:52.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.326+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbbf01fa020 con 0x7fbbf0144370 2024-04-25T11:34:52.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.327+0000 7fbbe7e00640 1 -- 172.21.15.19:0/3544416379 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) ==== 80+0+120 (secure 0 0 0) 0x7fbbdc05ef40 con 0x7fbbf0144370 2024-04-25T11:34:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/853310115 2024-04-26T11:34:43.091612+0000 2024-04-25T11:34:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/853310115 2024-04-26T11:34:43.091612+0000 2024-04-25T11:34:52.317 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbbd00744b0 msgr2=0x7fbbd00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbbd00744b0 0x7fbbd00769c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0000f60 tx=0x7fbbe00079d0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 msgr2=0x7fbbf015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf015d5a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c410 tx=0x7fbbdc00c8d0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 shutdown_connections 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbbd00744b0 0x7fbbd00769c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbf0161db0 0x7fbbf01621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbf0144370 0x7fbbf015d5a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 --2- 172.21.15.19:0/3544416379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbf006c5c0 0x7fbbf00b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:52.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 >> 172.21.15.19:0/3544416379 conn(0x7fbbf00d1720 msgr2=0x7fbbf00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:52.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 shutdown_connections 2024-04-25T11:34:52.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:52.330+0000 7fbbf5600640 1 -- 172.21.15.19:0/3544416379 wait complete. 2024-04-25T11:34:52.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:34:52.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:52.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:34:52.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:34:52.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:34:52.481 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:34:52.538 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:34:52.539 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:34:52.539 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:34:52.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:34:52.596 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:34:52.596 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:34:52.596 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:34:52.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:34:52.654 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:34:52.654 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:34:52.655 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:34:52.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:34:52.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:53.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.014+0000 7f7ab3200640 1 Processor -- start 2024-04-25T11:34:53.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.014+0000 7f7ab3200640 1 -- start start 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.016+0000 7f7ab3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7ab40cf2f0 0x7f7ab40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.016+0000 7f7ab3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ab41608d0 0x7f7ab4160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.016+0000 7f7ab3200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab4161220 con 0x7f7ab415a810 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.016+0000 7f7ab3200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab406f870 con 0x7f7ab41608d0 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.016+0000 7f7ab3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab40b74d0 con 0x7f7ab40cf2f0 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ab41608d0 0x7f7ab4160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ab41608d0 0x7f7ab4160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35580/0 (socket says 172.21.15.19:35580) 2024-04-25T11:34:53.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab2800640 1 -- 172.21.15.19:0/227328470 learned_addr learned my addr 172.21.15.19:0/227328470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab1e00640 1 --2- 172.21.15.19:0/227328470 >> v2:172.21.15.169:3300/0 conn(0x7f7ab40cf2f0 0x7f7ab40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab2800640 1 -- 172.21.15.19:0/227328470 >> v2:172.21.15.169:3300/0 conn(0x7f7ab40cf2f0 msgr2=0x7f7ab40b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab2800640 1 --2- 172.21.15.19:0/227328470 >> v2:172.21.15.169:3300/0 conn(0x7f7ab40cf2f0 0x7f7ab40b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab2800640 1 -- 172.21.15.19:0/227328470 >> v1:172.21.15.19:6789/0 conn(0x7f7ab415a810 legacy=0x7f7ab40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab2800640 1 -- 172.21.15.19:0/227328470 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab40b77b0 con 0x7f7ab41608d0 2024-04-25T11:34:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.017+0000 7f7ab0a00640 1 -- 172.21.15.19:0/227328470 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3514560970 0 0) 0x7f7aa0001110 con 0x7f7ab415a810 2024-04-25T11:34:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.018+0000 7f7ab2800640 1 --2- 172.21.15.19:0/227328470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab41608d0 0x7f7ab4160ce0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40d26d0 tx=0x7f7aac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ae32c18657401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.019+0000 7f7ab0a00640 1 -- 172.21.15.19:0/227328470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac036070 con 0x7f7ab41608d0 2024-04-25T11:34:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.019+0000 7f7ab0a00640 1 -- 172.21.15.19:0/227328470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aac031600 con 0x7f7ab41608d0 2024-04-25T11:34:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.019+0000 7f7ab0a00640 1 -- 172.21.15.19:0/227328470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0319d0 con 0x7f7ab41608d0 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 -- 172.21.15.19:0/227328470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab41608d0 msgr2=0x7f7ab4160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 --2- 172.21.15.19:0/227328470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab41608d0 0x7f7ab4160ce0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40d26d0 tx=0x7f7aac02a730 comp rx=0 tx=0).stop 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 -- 172.21.15.19:0/227328470 shutdown_connections 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 --2- 172.21.15.19:0/227328470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab41608d0 0x7f7ab4160ce0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 --2- 172.21.15.19:0/227328470 >> v2:172.21.15.169:3300/0 conn(0x7f7ab40cf2f0 0x7f7ab40b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 -- 172.21.15.19:0/227328470 >> 172.21.15.19:0/227328470 conn(0x7f7ab40d1720 msgr2=0x7f7ab40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 -- 172.21.15.19:0/227328470 shutdown_connections 2024-04-25T11:34:53.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.020+0000 7f7ab3200640 1 -- 172.21.15.19:0/227328470 wait complete. 2024-04-25T11:34:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.022+0000 7f7ab3200640 1 Processor -- start 2024-04-25T11:34:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.022+0000 7f7ab3200640 1 -- start start 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ab40cf2f0 0x7f7ab4153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab415a810 0x7f7ab415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab41608d0 0x7f7ab415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab415ac20 con 0x7f7ab415a810 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab406f500 con 0x7f7ab41608d0 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab406fd00 con 0x7f7ab40cf2f0 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab415a810 0x7f7ab415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab415a810 0x7f7ab415d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35582/0 (socket says 172.21.15.19:35582) 2024-04-25T11:34:53.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab1400640 1 -- 172.21.15.19:0/2282450048 learned_addr learned my addr 172.21.15.19:0/2282450048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.024+0000 7f7ab2800640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab41608d0 0x7f7ab415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1e00640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ab40cf2f0 0x7f7ab4153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1400640 1 -- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ab40cf2f0 msgr2=0x7f7ab4153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1400640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ab40cf2f0 0x7f7ab4153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1400640 1 -- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab41608d0 msgr2=0x7f7ab415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1400640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab41608d0 0x7f7ab415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1400640 1 -- 172.21.15.19:0/2282450048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab4152860 con 0x7f7ab415a810 2024-04-25T11:34:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1e00640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ab40cf2f0 0x7f7ab4153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab1400640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab415a810 0x7f7ab415d430 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c440 tx=0x7f7aa000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7aaa000640 1 -- 172.21.15.19:0/2282450048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0017070 con 0x7f7ab415a810 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7aaa000640 1 -- 172.21.15.19:0/2282450048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0005480 con 0x7f7ab415a810 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7aaa000640 1 -- 172.21.15.19:0/2282450048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0003830 con 0x7f7ab415a810 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.025+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab4152b40 con 0x7f7ab415a810 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.026+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab4153b60 con 0x7f7ab415a810 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.027+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab4050740 con 0x7f7ab415a810 2024-04-25T11:34:53.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.028+0000 7f7aaa000640 1 -- 172.21.15.19:0/2282450048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7aa001a070 con 0x7f7ab415a810 2024-04-25T11:34:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.029+0000 7f7aaa000640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a940744b0 0x7f7a940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.029+0000 7f7aaa000640 1 -- 172.21.15.19:0/2282450048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f7aa0094580 con 0x7f7ab415a810 2024-04-25T11:34:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.030+0000 7f7ab1e00640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a940744b0 0x7f7a940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.031+0000 7f7ab1e00640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a940744b0 0x7f7a940769c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006fd0 tx=0x7f7aa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.034+0000 7f7aaa000640 1 -- 172.21.15.19:0/2282450048 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ab4161220 con 0x7f7ab415a810 2024-04-25T11:34:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.285+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ab415c9e0 con 0x7f7ab415a810 2024-04-25T11:34:53.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.287+0000 7f7aaa000640 1 -- 172.21.15.19:0/2282450048 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5191 (secure 0 0 0) 0x7f7aa005eca0 con 0x7f7ab415a810 2024-04-25T11:34:53.276 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:53.276 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3655604895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3655604895},{"type":"v1","addr":"172.21.15.19:6835","nonce":3655604895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:46.124891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5845":{"gid":5845,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":14,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5845,"qdb_cluster":[5845]},"id":5}]} 2024-04-25T11:34:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 81 2024-04-25T11:34:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a940744b0 msgr2=0x7f7a940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a940744b0 0x7f7a940769c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4006fd0 tx=0x7f7aa4008040 comp rx=0 tx=0).stop 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab415a810 msgr2=0x7f7ab415d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab415a810 0x7f7ab415d430 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c440 tx=0x7f7aa000c900 comp rx=0 tx=0).stop 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 shutdown_connections 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a940744b0 0x7f7a940769c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ab41608d0 0x7f7ab415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab415a810 0x7f7ab415d430 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 --2- 172.21.15.19:0/2282450048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ab40cf2f0 0x7f7ab4153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 >> 172.21.15.19:0/2282450048 conn(0x7f7ab40d1720 msgr2=0x7f7ab40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.290+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 shutdown_connections 2024-04-25T11:34:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.291+0000 7f7ab3200640 1 -- 172.21.15.19:0/2282450048 wait complete. 2024-04-25T11:34:53.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:53.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.585+0000 7fc037200640 1 Processor -- start 2024-04-25T11:34:53.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.585+0000 7fc037200640 1 -- start start 2024-04-25T11:34:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.587+0000 7fc037200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d5370 0x7fc038142e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.587+0000 7fc037200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0380d3580 0x7fc0380d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.587+0000 7fc037200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc038145860 con 0x7fc03806dc80 2024-04-25T11:34:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.588+0000 7fc037200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc0381616f0 con 0x7fc0380d3580 2024-04-25T11:34:53.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.588+0000 7fc037200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc038145b40 con 0x7fc0380d5370 2024-04-25T11:34:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.588+0000 7fc035400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0380d3580 0x7fc0380d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.588+0000 7fc035400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0380d3580 0x7fc0380d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35598/0 (socket says 172.21.15.19:35598) 2024-04-25T11:34:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.588+0000 7fc035400640 1 -- 172.21.15.19:0/967025060 learned_addr learned my addr 172.21.15.19:0/967025060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.588+0000 7fc035e00640 1 --2- 172.21.15.19:0/967025060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d5370 0x7fc038142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.589+0000 7fc035400640 1 -- 172.21.15.19:0/967025060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d5370 msgr2=0x7fc038142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:53.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.589+0000 7fc035400640 1 --2- 172.21.15.19:0/967025060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d5370 0x7fc038142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.589+0000 7fc035400640 1 -- 172.21.15.19:0/967025060 >> v1:172.21.15.169:6789/0 conn(0x7fc03806dc80 legacy=0x7fc03806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.589+0000 7fc035400640 1 -- 172.21.15.19:0/967025060 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc038145e20 con 0x7fc0380d3580 2024-04-25T11:34:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.590+0000 7fc035400640 1 --2- 172.21.15.19:0/967025060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0380d3580 0x7fc0380d3990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc03806ed50 tx=0x7fc02402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3ac70072d5326a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.590+0000 7fc034a00640 1 -- 172.21.15.19:0/967025060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024037070 con 0x7fc0380d3580 2024-04-25T11:34:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.590+0000 7fc034a00640 1 -- 172.21.15.19:0/967025060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02402ae50 con 0x7fc0380d3580 2024-04-25T11:34:53.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.590+0000 7fc034a00640 1 -- 172.21.15.19:0/967025060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024032660 con 0x7fc0380d3580 2024-04-25T11:34:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 -- 172.21.15.19:0/967025060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0380d3580 msgr2=0x7fc0380d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 --2- 172.21.15.19:0/967025060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0380d3580 0x7fc0380d3990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc03806ed50 tx=0x7fc02402a140 comp rx=0 tx=0).stop 2024-04-25T11:34:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 -- 172.21.15.19:0/967025060 shutdown_connections 2024-04-25T11:34:53.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 --2- 172.21.15.19:0/967025060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0380d3580 0x7fc0380d3990 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 --2- 172.21.15.19:0/967025060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d5370 0x7fc038142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 -- 172.21.15.19:0/967025060 >> 172.21.15.19:0/967025060 conn(0x7fc03806c5c0 msgr2=0x7fc03806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 -- 172.21.15.19:0/967025060 shutdown_connections 2024-04-25T11:34:53.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.591+0000 7fc037200640 1 -- 172.21.15.19:0/967025060 wait complete. 2024-04-25T11:34:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.594+0000 7fc037200640 1 Processor -- start 2024-04-25T11:34:53.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.594+0000 7fc037200640 1 -- start start 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.595+0000 7fc037200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03806dc80 0x7fc03815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.595+0000 7fc037200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d3580 0x7fc038156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc037200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0380d5370 0x7fc0381599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc037200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc038164ae0 con 0x7fc03806dc80 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc037200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0381433c0 con 0x7fc0380d5370 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc037200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc038164c60 con 0x7fc0380d3580 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03806dc80 0x7fc03815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03806dc80 0x7fc03815c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35604/0 (socket says 172.21.15.19:35604) 2024-04-25T11:34:53.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 -- 172.21.15.19:0/3121360674 learned_addr learned my addr 172.21.15.19:0/3121360674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035400640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d3580 0x7fc038156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc036800640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0380d5370 0x7fc0381599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 -- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d3580 msgr2=0x7fc038156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d3580 0x7fc038156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 -- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0380d5370 msgr2=0x7fc0381599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0380d5370 0x7fc0381599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035e00640 1 -- 172.21.15.19:0/3121360674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc038156880 con 0x7fc03806dc80 2024-04-25T11:34:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc035400640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d3580 0x7fc038156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.596+0000 7fc036800640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0380d5370 0x7fc0381599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.597+0000 7fc035e00640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03806dc80 0x7fc03815c2b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc028007e70 tx=0x7fc02800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.597+0000 7fc02e000640 1 -- 172.21.15.19:0/3121360674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028014420 con 0x7fc03806dc80 2024-04-25T11:34:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.597+0000 7fc02e000640 1 -- 172.21.15.19:0/3121360674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028019070 con 0x7fc03806dc80 2024-04-25T11:34:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.597+0000 7fc02e000640 1 -- 172.21.15.19:0/3121360674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028014800 con 0x7fc03806dc80 2024-04-25T11:34:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.597+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03815ada0 con 0x7fc03806dc80 2024-04-25T11:34:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.597+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03815b4f0 con 0x7fc03806dc80 2024-04-25T11:34:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.599+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0380b91e0 con 0x7fc03806dc80 2024-04-25T11:34:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.600+0000 7fc02e000640 1 -- 172.21.15.19:0/3121360674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc0280149a0 con 0x7fc03806dc80 2024-04-25T11:34:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.601+0000 7fc02e000640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0180744b0 0x7fc0180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.602+0000 7fc02e000640 1 -- 172.21.15.19:0/3121360674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fc028095b90 con 0x7fc03806dc80 2024-04-25T11:34:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.602+0000 7fc035400640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0180744b0 0x7fc0180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.604+0000 7fc035400640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0180744b0 0x7fc0180769c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc02402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.606+0000 7fc02e000640 1 -- 172.21.15.19:0/3121360674 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0380b91e0 con 0x7fc03806dc80 2024-04-25T11:34:53.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.811+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc038050740 con 0x7fc03806dc80 2024-04-25T11:34:53.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.814+0000 7fc02e000640 1 -- 172.21.15.19:0/3121360674 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16735 (secure 0 0 0) 0x7fc038156880 con 0x7fc03806dc80 2024-04-25T11:34:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:53.804 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:43.091776+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:34:04.018735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T11:34:04.759136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/853310115":"2024-04-26T11:34:43.091612+0000","172.21.15.19:6834/853310115":"2024-04-26T11:34:43.091612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0180744b0 msgr2=0x7fc0180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0180744b0 0x7fc0180769c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc02402e000 comp rx=0 tx=0).stop 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03806dc80 msgr2=0x7fc03815c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03806dc80 0x7fc03815c2b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc028007e70 tx=0x7fc02800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 shutdown_connections 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0180744b0 0x7fc0180769c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0380d5370 0x7fc0381599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0380d3580 0x7fc038156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 --2- 172.21.15.19:0/3121360674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03806dc80 0x7fc03815c2b0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 >> 172.21.15.19:0/3121360674 conn(0x7fc03806c5c0 msgr2=0x7fc0380b7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 shutdown_connections 2024-04-25T11:34:53.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:53.817+0000 7fc037200640 1 -- 172.21.15.19:0/3121360674 wait complete. 2024-04-25T11:34:53.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:34:04.018735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:34:04.759136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:34:53.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.141+0000 7fe813200640 1 Processor -- start 2024-04-25T11:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.141+0000 7fe813200640 1 -- start start 2024-04-25T11:34:54.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.143+0000 7fe813200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe8140ce930 0x7fe81406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.143+0000 7fe813200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe81406db10 0x7fe8140d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.143+0000 7fe813200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81406df20 con 0x7fe8140ce520 2024-04-25T11:34:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.143+0000 7fe813200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe8140cda60 con 0x7fe8140ce930 2024-04-25T11:34:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.143+0000 7fe813200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe81415dfe0 con 0x7fe81406db10 2024-04-25T11:34:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.144+0000 7fe811e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe8140ce520 0x7fe81415d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:51608/0 (socket says 172.21.15.19:51608) 2024-04-25T11:34:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.144+0000 7fe811e00640 1 -- 172.21.15.19:0/2487552857 learned_addr learned my addr 172.21.15.19:0/2487552857 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.144+0000 7fe80be00640 1 --2- 172.21.15.19:0/2487552857 >> v2:172.21.15.169:3300/0 conn(0x7fe8140ce930 0x7fe81406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.144+0000 7fe812800640 1 --2- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe81406db10 0x7fe8140d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.145+0000 7fe80be00640 1 -- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe81406db10 msgr2=0x7fe8140d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.145+0000 7fe80be00640 1 --2- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe81406db10 0x7fe8140d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.145+0000 7fe80be00640 1 -- 172.21.15.19:0/2487552857 >> v1:172.21.15.169:6789/0 conn(0x7fe8140ce520 legacy=0x7fe81415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.145+0000 7fe80be00640 1 -- 172.21.15.19:0/2487552857 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81415e2c0 con 0x7fe8140ce930 2024-04-25T11:34:54.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.146+0000 7fe80be00640 1 --2- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce930 0x7fe81406d5d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0097a0 tx=0x7fe7fc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed5bf1f8a986999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:54.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.147+0000 7fe811400640 1 -- 172.21.15.19:0/2487552857 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc037070 con 0x7fe8140ce930 2024-04-25T11:34:54.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.147+0000 7fe811400640 1 -- 172.21.15.19:0/2487552857 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7fc02ae50 con 0x7fe8140ce930 2024-04-25T11:34:54.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.147+0000 7fe811400640 1 -- 172.21.15.19:0/2487552857 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc032660 con 0x7fe8140ce930 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 -- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce930 msgr2=0x7fe81406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 --2- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce930 0x7fe81406d5d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0097a0 tx=0x7fe7fc02a100 comp rx=0 tx=0).stop 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 -- 172.21.15.19:0/2487552857 shutdown_connections 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 --2- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe81406db10 0x7fe8140d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 --2- 172.21.15.19:0/2487552857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce930 0x7fe81406d5d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 -- 172.21.15.19:0/2487552857 >> 172.21.15.19:0/2487552857 conn(0x7fe814072f40 msgr2=0x7fe814073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 -- 172.21.15.19:0/2487552857 shutdown_connections 2024-04-25T11:34:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.148+0000 7fe813200640 1 -- 172.21.15.19:0/2487552857 wait complete. 2024-04-25T11:34:54.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.150+0000 7fe813200640 1 Processor -- start 2024-04-25T11:34:54.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.151+0000 7fe813200640 1 -- start start 2024-04-25T11:34:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.152+0000 7fe813200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81406db10 0x7fe8141451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.152+0000 7fe813200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce520 0x7fe8141e6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.152+0000 7fe813200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8140ce930 0x7fe8140cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.152+0000 7fe813200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe814160d30 con 0x7fe81406db10 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.152+0000 7fe813200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe8140ce000 con 0x7fe8140ce520 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.152+0000 7fe813200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe81415dbc0 con 0x7fe8140ce930 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.152+0000 7fe811e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81406db10 0x7fe8141451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81406db10 0x7fe8141451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38990/0 (socket says 172.21.15.19:38990) 2024-04-25T11:34:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 -- 172.21.15.19:0/4082313297 learned_addr learned my addr 172.21.15.19:0/4082313297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe812800640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8140ce930 0x7fe8140cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 -- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8140ce930 msgr2=0x7fe8140cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe80be00640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce520 0x7fe8141e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8140ce930 0x7fe8140cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 -- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce520 msgr2=0x7fe8141e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce520 0x7fe8141e6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 -- 172.21.15.19:0/4082313297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8141456e0 con 0x7fe81406db10 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.153+0000 7fe811e00640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81406db10 0x7fe8141451a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c470 tx=0x7fe80400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.154+0000 7fe80a000640 1 -- 172.21.15.19:0/4082313297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804017070 con 0x7fe81406db10 2024-04-25T11:34:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.154+0000 7fe80a000640 1 -- 172.21.15.19:0/4082313297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe804005d00 con 0x7fe81406db10 2024-04-25T11:34:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.154+0000 7fe80a000640 1 -- 172.21.15.19:0/4082313297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804007310 con 0x7fe81406db10 2024-04-25T11:34:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.154+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8141e7350 con 0x7fe81406db10 2024-04-25T11:34:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.154+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8141e7910 con 0x7fe81406db10 2024-04-25T11:34:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.155+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe814076c60 con 0x7fe81406db10 2024-04-25T11:34:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.156+0000 7fe80a000640 1 -- 172.21.15.19:0/4082313297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe80401a070 con 0x7fe81406db10 2024-04-25T11:34:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.157+0000 7fe80a000640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7f00744b0 0x7fe7f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.158+0000 7fe80a000640 1 -- 172.21.15.19:0/4082313297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fe8040945b0 con 0x7fe81406db10 2024-04-25T11:34:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.158+0000 7fe80be00640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7f00744b0 0x7fe7f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.159+0000 7fe80be00640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7f00744b0 0x7fe7f00769c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc034000 tx=0x7fe7fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.162+0000 7fe80a000640 1 -- 172.21.15.19:0/4082313297 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe81406df20 con 0x7fe81406db10 2024-04-25T11:34:54.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.372+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe8140b5480 con 0x7fe81406db10 2024-04-25T11:34:54.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.373+0000 7fe80a000640 1 -- 172.21.15.19:0/4082313297 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fe80405ecd0 con 0x7fe81406db10 2024-04-25T11:34:54.362 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:34:54.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7f00744b0 msgr2=0x7fe7f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7f00744b0 0x7fe7f00769c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc034000 tx=0x7fe7fc02e000 comp rx=0 tx=0).stop 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81406db10 msgr2=0x7fe8141451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81406db10 0x7fe8141451a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c470 tx=0x7fe80400c930 comp rx=0 tx=0).stop 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 shutdown_connections 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7f00744b0 0x7fe7f00769c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8140ce520 0x7fe8141e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe8140ce930 0x7fe8140cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 --2- 172.21.15.19:0/4082313297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe81406db10 0x7fe8141451a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 >> 172.21.15.19:0/4082313297 conn(0x7fe814072f40 msgr2=0x7fe814070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 shutdown_connections 2024-04-25T11:34:54.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.376+0000 7fe813200640 1 -- 172.21.15.19:0/4082313297 wait complete. 2024-04-25T11:34:54.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:34:54.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.692+0000 7ff1d9e00640 1 Processor -- start 2024-04-25T11:34:54.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.692+0000 7ff1d9e00640 1 -- start start 2024-04-25T11:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.694+0000 7ff1d9e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff1d40ce520 0x7ff1d415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.695+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.695+0000 7ff1d9e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d40cda60 con 0x7ff1d406d5d0 2024-04-25T11:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.695+0000 7ff1d9e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d415dfe0 con 0x7ff1d40ce520 2024-04-25T11:34:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.695+0000 7ff1d9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d415e2c0 con 0x7ff1d40ce930 2024-04-25T11:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.695+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.695+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52490/0 (socket says 172.21.15.19:52490) 2024-04-25T11:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.695+0000 7ff1d3e00640 1 -- 172.21.15.19:0/1414813998 learned_addr learned my addr 172.21.15.19:0/1414813998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.696+0000 7ff1d8a00640 1 --2- 172.21.15.19:0/1414813998 >> v2:172.21.15.169:3300/0 conn(0x7ff1d40ce520 0x7ff1d415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.696+0000 7ff1d3e00640 1 -- 172.21.15.19:0/1414813998 >> v2:172.21.15.169:3300/0 conn(0x7ff1d40ce520 msgr2=0x7ff1d415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.696+0000 7ff1d3e00640 1 --2- 172.21.15.19:0/1414813998 >> v2:172.21.15.169:3300/0 conn(0x7ff1d40ce520 0x7ff1d415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.696+0000 7ff1d3e00640 1 -- 172.21.15.19:0/1414813998 >> v1:172.21.15.169:6789/0 conn(0x7ff1d406d5d0 legacy=0x7ff1d406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.696+0000 7ff1d3e00640 1 -- 172.21.15.19:0/1414813998 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d40cdd20 con 0x7ff1d40ce930 2024-04-25T11:34:54.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.698+0000 7ff1d3e00640 1 --2- 172.21.15.19:0/1414813998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415d9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c002c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28c496c66228333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.699+0000 7ff1d3400640 1 -- 172.21.15.19:0/1414813998 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0038070 con 0x7ff1d40ce930 2024-04-25T11:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.699+0000 7ff1d3400640 1 -- 172.21.15.19:0/1414813998 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c0031710 con 0x7ff1d40ce930 2024-04-25T11:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.699+0000 7ff1d3400640 1 -- 172.21.15.19:0/1414813998 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0031ae0 con 0x7ff1d40ce930 2024-04-25T11:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.699+0000 7ff1d9e00640 1 -- 172.21.15.19:0/1414813998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 msgr2=0x7ff1d415d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.699+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/1414813998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415d9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009870 tx=0x7ff1c002c3d0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.700+0000 7ff1d9e00640 1 -- 172.21.15.19:0/1414813998 shutdown_connections 2024-04-25T11:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.700+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/1414813998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415d9f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.700+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/1414813998 >> v2:172.21.15.169:3300/0 conn(0x7ff1d40ce520 0x7ff1d415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.700+0000 7ff1d9e00640 1 -- 172.21.15.19:0/1414813998 >> 172.21.15.19:0/1414813998 conn(0x7ff1d4072f40 msgr2=0x7ff1d4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.700+0000 7ff1d9e00640 1 -- 172.21.15.19:0/1414813998 shutdown_connections 2024-04-25T11:34:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.700+0000 7ff1d9e00640 1 -- 172.21.15.19:0/1414813998 wait complete. 2024-04-25T11:34:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.702+0000 7ff1d9e00640 1 Processor -- start 2024-04-25T11:34:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.702+0000 7ff1d9e00640 1 -- start start 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1d406d5d0 0x7ff1d41e8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1d40ce520 0x7ff1d4156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d9e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d40b73d0 con 0x7ff1d406d5d0 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d9e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4160d30 con 0x7ff1d40ce520 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d40b7550 con 0x7ff1d40ce930 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1d406d5d0 0x7ff1d41e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1d406d5d0 0x7ff1d41e8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38998/0 (socket says 172.21.15.19:38998) 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d8a00640 1 -- 172.21.15.19:0/2729883716 learned_addr learned my addr 172.21.15.19:0/2729883716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.704+0000 7ff1d3e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1d40ce520 0x7ff1d4156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d9400640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d8a00640 1 -- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 msgr2=0x7ff1d415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d8a00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d8a00640 1 -- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1d40ce520 msgr2=0x7ff1d4156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d8a00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1d40ce520 0x7ff1d4156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d8a00640 1 -- 172.21.15.19:0/2729883716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d41e8d00 con 0x7ff1d406d5d0 2024-04-25T11:34:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d8a00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1d406d5d0 0x7ff1d41e8460 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400e870 tx=0x7ff1c400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d0c00640 1 -- 172.21.15.19:0/2729883716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4017070 con 0x7ff1d406d5d0 2024-04-25T11:34:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.705+0000 7ff1d0c00640 1 -- 172.21.15.19:0/2729883716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c400c940 con 0x7ff1d406d5d0 2024-04-25T11:34:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.706+0000 7ff1d0c00640 1 -- 172.21.15.19:0/2729883716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400cd10 con 0x7ff1d406d5d0 2024-04-25T11:34:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.706+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d40cad90 con 0x7ff1d406d5d0 2024-04-25T11:34:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.706+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d40cb350 con 0x7ff1d406d5d0 2024-04-25T11:34:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.707+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d40cee60 con 0x7ff1d406d5d0 2024-04-25T11:34:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.708+0000 7ff1d0c00640 1 -- 172.21.15.19:0/2729883716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff1c401a070 con 0x7ff1d406d5d0 2024-04-25T11:34:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.710+0000 7ff1d0c00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1b40744e0 0x7ff1b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.710+0000 7ff1d0c00640 1 -- 172.21.15.19:0/2729883716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7ff1c4095560 con 0x7ff1d406d5d0 2024-04-25T11:34:54.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.711+0000 7ff1d3e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1b40744e0 0x7ff1b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.712+0000 7ff1d3e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1b40744e0 0x7ff1b40769f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0035000 tx=0x7ff1c0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:54.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.715+0000 7ff1d0c00640 1 -- 172.21.15.19:0/2729883716 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1d40cee60 con 0x7ff1d406d5d0 2024-04-25T11:34:54.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.913+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff1d4076c60 con 0x7ff1d406d5d0 2024-04-25T11:34:54.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.914+0000 7ff1d0c00640 1 -- 172.21.15.19:0/2729883716 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7ff1c405fbf0 con 0x7ff1d406d5d0 2024-04-25T11:34:54.904 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:34:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1b40744e0 msgr2=0x7ff1b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1b40744e0 0x7ff1b40769f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0035000 tx=0x7ff1c0008030 comp rx=0 tx=0).stop 2024-04-25T11:34:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1d406d5d0 msgr2=0x7ff1d41e8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1d406d5d0 0x7ff1d41e8460 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400e870 tx=0x7ff1c400ed30 comp rx=0 tx=0).stop 2024-04-25T11:34:54.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 shutdown_connections 2024-04-25T11:34:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1b40744e0 0x7ff1b40769f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1d40ce520 0x7ff1d4156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1d40ce930 0x7ff1d415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 --2- 172.21.15.19:0/2729883716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1d406d5d0 0x7ff1d41e8460 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 >> 172.21.15.19:0/2729883716 conn(0x7ff1d4072f40 msgr2=0x7ff1d415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 shutdown_connections 2024-04-25T11:34:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:54.917+0000 7ff1d9e00640 1 -- 172.21.15.19:0/2729883716 wait complete. 2024-04-25T11:34:54.930 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:34:55.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.230+0000 7f35ae600640 1 Processor -- start 2024-04-25T11:34:55.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.231+0000 7f35ae600640 1 -- start start 2024-04-25T11:34:55.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.233+0000 7f35ae600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a806dce0 0x7f35a80cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.233+0000 7f35ae600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35a815adf0 0x7f35a8160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.233+0000 7f35ae600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a8071110 con 0x7f35a8070630 2024-04-25T11:34:55.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.233+0000 7f35ae600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f35a80713f0 con 0x7f35a815adf0 2024-04-25T11:34:55.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.233+0000 7f35ae600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35a80701a0 con 0x7f35a806dce0 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35a815adf0 0x7f35a8160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35a815adf0 0x7f35a8160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39006/0 (socket says 172.21.15.19:39006) 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a7e00640 1 -- 172.21.15.19:0/347487430 learned_addr learned my addr 172.21.15.19:0/347487430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a6a00640 1 --2- 172.21.15.19:0/347487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a806dce0 0x7f35a80cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a7e00640 1 -- 172.21.15.19:0/347487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a806dce0 msgr2=0x7f35a80cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a7e00640 1 --2- 172.21.15.19:0/347487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a806dce0 0x7f35a80cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a7e00640 1 -- 172.21.15.19:0/347487430 >> v1:172.21.15.169:6789/0 conn(0x7f35a8070630 legacy=0x7f35a806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.234+0000 7f35a7e00640 1 -- 172.21.15.19:0/347487430 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a806fb50 con 0x7f35a815adf0 2024-04-25T11:34:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.235+0000 7f35a7e00640 1 --2- 172.21.15.19:0/347487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a815adf0 0x7f35a8160d80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009870 tx=0x7f35a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8bff2b5a0ccc0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.236+0000 7f35a6000640 1 -- 172.21.15.19:0/347487430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0037070 con 0x7f35a815adf0 2024-04-25T11:34:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.236+0000 7f35a6000640 1 -- 172.21.15.19:0/347487430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a0032440 con 0x7f35a815adf0 2024-04-25T11:34:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.236+0000 7f35a6000640 1 -- 172.21.15.19:0/347487430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0032810 con 0x7f35a815adf0 2024-04-25T11:34:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 -- 172.21.15.19:0/347487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a815adf0 msgr2=0x7f35a8160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 --2- 172.21.15.19:0/347487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a815adf0 0x7f35a8160d80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009870 tx=0x7f35a002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 -- 172.21.15.19:0/347487430 shutdown_connections 2024-04-25T11:34:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 --2- 172.21.15.19:0/347487430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a815adf0 0x7f35a8160d80 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 --2- 172.21.15.19:0/347487430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a806dce0 0x7f35a80cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 -- 172.21.15.19:0/347487430 >> 172.21.15.19:0/347487430 conn(0x7f35a8073090 msgr2=0x7f35a80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:55.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 -- 172.21.15.19:0/347487430 shutdown_connections 2024-04-25T11:34:55.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.237+0000 7f35ae600640 1 -- 172.21.15.19:0/347487430 wait complete. 2024-04-25T11:34:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.239+0000 7f35ae600640 1 Processor -- start 2024-04-25T11:34:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.239+0000 7f35ae600640 1 -- start start 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35ae600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a8070630 0x7f35a8145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35ae600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a815adf0 0x7f35a80b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35ae600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a8159a80 0x7f35a80b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35ae600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35a80b0e80 con 0x7f35a8070630 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35ae600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35a80b0d00 con 0x7f35a815adf0 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35ae600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35a8070c80 con 0x7f35a8159a80 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a8070630 0x7f35a8145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a8070630 0x7f35a8145b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39022/0 (socket says 172.21.15.19:39022) 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.241+0000 7f35a7400640 1 -- 172.21.15.19:0/2083579100 learned_addr learned my addr 172.21.15.19:0/2083579100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:55.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a7e00640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a8159a80 0x7f35a80b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a6a00640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a815adf0 0x7f35a80b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a7400640 1 -- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a8159a80 msgr2=0x7f35a80b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a7400640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a8159a80 0x7f35a80b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a7400640 1 -- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a815adf0 msgr2=0x7f35a80b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a7400640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a815adf0 0x7f35a80b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a7400640 1 -- 172.21.15.19:0/2083579100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a815ccc0 con 0x7f35a8070630 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f35a7400640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a8070630 0x7f35a8145b20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f359800c700 tx=0x7f359800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.242+0000 7f359f400640 1 -- 172.21.15.19:0/2083579100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598003b70 con 0x7f35a8070630 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.243+0000 7f359f400640 1 -- 172.21.15.19:0/2083579100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3598003d10 con 0x7f35a8070630 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.243+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a815c1a0 con 0x7f35a8070630 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.243+0000 7f359f400640 1 -- 172.21.15.19:0/2083579100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359800aa30 con 0x7f35a8070630 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.243+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81574d0 con 0x7f35a8070630 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.244+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a8072d90 con 0x7f35a8070630 2024-04-25T11:34:55.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.245+0000 7f359f400640 1 -- 172.21.15.19:0/2083579100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f359800abd0 con 0x7f35a8070630 2024-04-25T11:34:55.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.246+0000 7f359f400640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35840744b0 0x7f35840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.246+0000 7f359f400640 1 -- 172.21.15.19:0/2083579100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f3598095200 con 0x7f35a8070630 2024-04-25T11:34:55.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.247+0000 7f35a6a00640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35840744b0 0x7f35840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.248+0000 7f35a6a00640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35840744b0 0x7f35840769c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3590006fd0 tx=0x7f3590008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.251+0000 7f359f400640 1 -- 172.21.15.19:0/2083579100 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35a8072d90 con 0x7f35a8070630 2024-04-25T11:34:55.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.457+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35a815c770 con 0x7f35a8070630 2024-04-25T11:34:55.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.458+0000 7f359f400640 1 -- 172.21.15.19:0/2083579100 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f359805f890 con 0x7f35a8070630 2024-04-25T11:34:55.447 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:34:55.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35840744b0 msgr2=0x7f35840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35840744b0 0x7f35840769c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3590006fd0 tx=0x7f3590008040 comp rx=0 tx=0).stop 2024-04-25T11:34:55.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a8070630 msgr2=0x7f35a8145b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a8070630 0x7f35a8145b20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f359800c700 tx=0x7f359800cbc0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 shutdown_connections 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35840744b0 0x7f35840769c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35a815adf0 0x7f35a80b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35a8159a80 0x7f35a80b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 --2- 172.21.15.19:0/2083579100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35a8070630 0x7f35a8145b20 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 >> 172.21.15.19:0/2083579100 conn(0x7f35a8073090 msgr2=0x7f35a806d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 shutdown_connections 2024-04-25T11:34:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.461+0000 7f35ae600640 1 -- 172.21.15.19:0/2083579100 wait complete. 2024-04-25T11:34:55.477 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:55.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.761+0000 7f0fbb400640 1 Processor -- start 2024-04-25T11:34:55.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.761+0000 7f0fbb400640 1 -- start start 2024-04-25T11:34:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb41434d0 0x7f0fb4070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fbb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fb40d0ed0 0x7f0fb40d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fbb400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fb40709c0 con 0x7f0fb4160be0 2024-04-25T11:34:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fbb400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb41438e0 con 0x7f0fb40d0ed0 2024-04-25T11:34:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fbb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb406f3e0 con 0x7f0fb41434d0 2024-04-25T11:34:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fb3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fb40d0ed0 0x7f0fb40d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fb3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0fb40d0ed0 0x7f0fb40d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39038/0 (socket says 172.21.15.19:39038) 2024-04-25T11:34:55.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fb3e00640 1 -- 172.21.15.19:0/456079918 learned_addr learned my addr 172.21.15.19:0/456079918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:55.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fb9400640 1 --1- 172.21.15.19:0/456079918 >> v1:172.21.15.19:6789/0 conn(0x7f0fb4160be0 0x7f0fb4160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34838/0 (socket says 172.21.15.19:34838) 2024-04-25T11:34:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.764+0000 7f0fb8a00640 1 --2- 172.21.15.19:0/456079918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb41434d0 0x7f0fb4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.765+0000 7f0fb3e00640 1 -- 172.21.15.19:0/456079918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb41434d0 msgr2=0x7f0fb4070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.765+0000 7f0fb3e00640 1 --2- 172.21.15.19:0/456079918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb41434d0 0x7f0fb4070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.765+0000 7f0fb3e00640 1 -- 172.21.15.19:0/456079918 >> v1:172.21.15.19:6789/0 conn(0x7f0fb4160be0 legacy=0x7f0fb4160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.765+0000 7f0fb3e00640 1 -- 172.21.15.19:0/456079918 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb406f6c0 con 0x7f0fb40d0ed0 2024-04-25T11:34:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.765+0000 7f0fb3400640 1 -- 172.21.15.19:0/456079918 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867736684 0 0) 0x7f0fb40709c0 con 0x7f0fb4160be0 2024-04-25T11:34:55.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.766+0000 7f0fb3e00640 1 --2- 172.21.15.19:0/456079918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb40d0ed0 0x7f0fb40d12e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009870 tx=0x7f0fa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb751b4f1cded76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.766+0000 7f0fb3400640 1 -- 172.21.15.19:0/456079918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0037070 con 0x7f0fb40d0ed0 2024-04-25T11:34:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.766+0000 7f0fb3400640 1 -- 172.21.15.19:0/456079918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa0032440 con 0x7f0fb40d0ed0 2024-04-25T11:34:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.766+0000 7f0fb3400640 1 -- 172.21.15.19:0/456079918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0032810 con 0x7f0fb40d0ed0 2024-04-25T11:34:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 -- 172.21.15.19:0/456079918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb40d0ed0 msgr2=0x7f0fb40d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 --2- 172.21.15.19:0/456079918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb40d0ed0 0x7f0fb40d12e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009870 tx=0x7f0fa002a270 comp rx=0 tx=0).stop 2024-04-25T11:34:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 -- 172.21.15.19:0/456079918 shutdown_connections 2024-04-25T11:34:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 --2- 172.21.15.19:0/456079918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb40d0ed0 0x7f0fb40d12e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 --2- 172.21.15.19:0/456079918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb41434d0 0x7f0fb4070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 -- 172.21.15.19:0/456079918 >> 172.21.15.19:0/456079918 conn(0x7f0fb40d1ae0 msgr2=0x7f0fb40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 -- 172.21.15.19:0/456079918 shutdown_connections 2024-04-25T11:34:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.767+0000 7f0fbb400640 1 -- 172.21.15.19:0/456079918 wait complete. 2024-04-25T11:34:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.770+0000 7f0fbb400640 1 Processor -- start 2024-04-25T11:34:55.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.770+0000 7f0fbb400640 1 -- start start 2024-04-25T11:34:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.771+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb40d0ed0 0x7f0fb4146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.771+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41434d0 0x7f0fb41f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.771+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4160be0 0x7f0fb4158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.771+0000 7f0fbb400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb406f090 con 0x7f0fb41434d0 2024-04-25T11:34:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.771+0000 7f0fbb400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb406fc80 con 0x7f0fb40d0ed0 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.771+0000 7f0fbb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb40b51a0 con 0x7f0fb4160be0 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41434d0 0x7f0fb41f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41434d0 0x7f0fb41f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39044/0 (socket says 172.21.15.19:39044) 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 -- 172.21.15.19:0/3359104975 learned_addr learned my addr 172.21.15.19:0/3359104975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 -- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4160be0 msgr2=0x7f0fb4158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4160be0 0x7f0fb4158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 -- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb40d0ed0 msgr2=0x7f0fb4146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb40d0ed0 0x7f0fb4146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 -- 172.21.15.19:0/3359104975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb41f2550 con 0x7f0fb41434d0 2024-04-25T11:34:55.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.772+0000 7f0fb3e00640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41434d0 0x7f0fb41f2010 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000c00 tx=0x7f0fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.773+0000 7f0fb0c00640 1 -- 172.21.15.19:0/3359104975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0041020 con 0x7f0fb41434d0 2024-04-25T11:34:55.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.773+0000 7f0fb0c00640 1 -- 172.21.15.19:0/3359104975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa00306c0 con 0x7f0fb41434d0 2024-04-25T11:34:55.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.773+0000 7f0fb0c00640 1 -- 172.21.15.19:0/3359104975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0037040 con 0x7f0fb41434d0 2024-04-25T11:34:55.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.773+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb41446c0 con 0x7f0fb41434d0 2024-04-25T11:34:55.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.773+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb4144c80 con 0x7f0fb41434d0 2024-04-25T11:34:55.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.774+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb4050740 con 0x7f0fb41434d0 2024-04-25T11:34:55.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.775+0000 7f0fb0c00640 1 -- 172.21.15.19:0/3359104975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0fa0044020 con 0x7f0fb41434d0 2024-04-25T11:34:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.777+0000 7f0fb0c00640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f940744b0 0x7f0f940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.777+0000 7f0fb0c00640 1 -- 172.21.15.19:0/3359104975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f0fa00b5b70 con 0x7f0fb41434d0 2024-04-25T11:34:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.778+0000 7f0fb8a00640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f940744b0 0x7f0f940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.779+0000 7f0fb8a00640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f940744b0 0x7f0f940769c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8006fd0 tx=0x7f0fa8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:55.782+0000 7f0fb0c00640 1 -- 172.21.15.19:0/3359104975 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fa004b030 con 0x7f0fb41434d0 2024-04-25T11:34:56.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.026+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fb4073d40 con 0x7f0fb41434d0 2024-04-25T11:34:56.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.028+0000 7f0fb0c00640 1 -- 172.21.15.19:0/3359104975 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5191 (secure 0 0 0) 0x7f0fa0080200 con 0x7f0fb41434d0 2024-04-25T11:34:56.017 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:56.017 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3655604895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3655604895},{"type":"v1","addr":"172.21.15.19:6835","nonce":3655604895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:46.124891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5845":{"gid":5845,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":14,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5845,"qdb_cluster":[5845]},"id":5}]} 2024-04-25T11:34:56.018 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 81 2024-04-25T11:34:56.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.030+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f940744b0 msgr2=0x7f0f940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.030+0000 7f0fbb400640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f940744b0 0x7f0f940769c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8006fd0 tx=0x7f0fa8008040 comp rx=0 tx=0).stop 2024-04-25T11:34:56.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.030+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41434d0 msgr2=0x7f0fb41f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.030+0000 7f0fbb400640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41434d0 0x7f0fb41f2010 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000c00 tx=0x7f0fa00046c0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 shutdown_connections 2024-04-25T11:34:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f940744b0 0x7f0f940769c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0fb4160be0 0x7f0fb4158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fb41434d0 0x7f0fb41f2010 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 --2- 172.21.15.19:0/3359104975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0fb40d0ed0 0x7f0fb4146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 >> 172.21.15.19:0/3359104975 conn(0x7f0fb40d1ae0 msgr2=0x7f0fb40b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:56.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 shutdown_connections 2024-04-25T11:34:56.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.031+0000 7f0fbb400640 1 -- 172.21.15.19:0/3359104975 wait complete. 2024-04-25T11:34:56.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:56.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.329+0000 7f2534600640 1 Processor -- start 2024-04-25T11:34:56.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.329+0000 7f2534600640 1 -- start start 2024-04-25T11:34:56.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.331+0000 7f2534600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f252c0cf5e0 0x7f252c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.331+0000 7f2534600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c06c5c0 0x7f252c06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.332+0000 7f2534600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f252c0cff30 con 0x7f252c1443c0 2024-04-25T11:34:56.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.332+0000 7f2534600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f252c0708d0 con 0x7f252c0cf5e0 2024-04-25T11:34:56.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.332+0000 7f2534600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f252c15f160 con 0x7f252c06c5c0 2024-04-25T11:34:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.332+0000 7f2532600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c06c5c0 0x7f252c06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.332+0000 7f2532600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c06c5c0 0x7f252c06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52530/0 (socket says 172.21.15.19:52530) 2024-04-25T11:34:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.332+0000 7f2532600640 1 -- 172.21.15.19:0/1678737045 learned_addr learned my addr 172.21.15.19:0/1678737045 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.333+0000 7f2531200640 1 --2- 172.21.15.19:0/1678737045 >> v2:172.21.15.169:3300/0 conn(0x7f252c0cf5e0 0x7f252c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.333+0000 7f2532600640 1 -- 172.21.15.19:0/1678737045 >> v2:172.21.15.169:3300/0 conn(0x7f252c0cf5e0 msgr2=0x7f252c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.333+0000 7f2532600640 1 --2- 172.21.15.19:0/1678737045 >> v2:172.21.15.169:3300/0 conn(0x7f252c0cf5e0 0x7f252c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.333+0000 7f2532600640 1 -- 172.21.15.19:0/1678737045 >> v1:172.21.15.169:6789/0 conn(0x7f252c1443c0 legacy=0x7f252c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.333+0000 7f2532600640 1 -- 172.21.15.19:0/1678737045 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c15f5f0 con 0x7f252c06c5c0 2024-04-25T11:34:56.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.334+0000 7f2532600640 1 --2- 172.21.15.19:0/1678737045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c06c5c0 0x7f252c06c9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2524009e50 tx=0x7f252402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6942315d4ddd39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.335+0000 7f252be00640 1 -- 172.21.15.19:0/1678737045 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524036070 con 0x7f252c06c5c0 2024-04-25T11:34:56.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.335+0000 7f252be00640 1 -- 172.21.15.19:0/1678737045 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2524031600 con 0x7f252c06c5c0 2024-04-25T11:34:56.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.335+0000 7f252be00640 1 -- 172.21.15.19:0/1678737045 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25240319d0 con 0x7f252c06c5c0 2024-04-25T11:34:56.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 -- 172.21.15.19:0/1678737045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c06c5c0 msgr2=0x7f252c06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 --2- 172.21.15.19:0/1678737045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c06c5c0 0x7f252c06c9d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2524009e50 tx=0x7f252402a710 comp rx=0 tx=0).stop 2024-04-25T11:34:56.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 -- 172.21.15.19:0/1678737045 shutdown_connections 2024-04-25T11:34:56.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 --2- 172.21.15.19:0/1678737045 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c06c5c0 0x7f252c06c9d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 --2- 172.21.15.19:0/1678737045 >> v2:172.21.15.169:3300/0 conn(0x7f252c0cf5e0 0x7f252c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 -- 172.21.15.19:0/1678737045 >> 172.21.15.19:0/1678737045 conn(0x7f252c0d1ae0 msgr2=0x7f252c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:56.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 -- 172.21.15.19:0/1678737045 shutdown_connections 2024-04-25T11:34:56.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.336+0000 7f2534600640 1 -- 172.21.15.19:0/1678737045 wait complete. 2024-04-25T11:34:56.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.338+0000 7f2534600640 1 Processor -- start 2024-04-25T11:34:56.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.339+0000 7f2534600640 1 -- start start 2024-04-25T11:34:56.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.340+0000 7f2534600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f252c06c5c0 0x7f252c1e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.340+0000 7f2534600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f252c0cf5e0 0x7f252c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.340+0000 7f2534600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c1443c0 0x7f252c1fa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.340+0000 7f2534600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f252c15edf0 con 0x7f252c06c5c0 2024-04-25T11:34:56.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.340+0000 7f2534600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f252c15eb10 con 0x7f252c0cf5e0 2024-04-25T11:34:56.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.340+0000 7f2534600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f252c1447d0 con 0x7f252c1443c0 2024-04-25T11:34:56.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.340+0000 7f2531c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f252c06c5c0 0x7f252c1e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f252c06c5c0 0x7f252c1e8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39054/0 (socket says 172.21.15.19:39054) 2024-04-25T11:34:56.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 -- 172.21.15.19:0/551371476 learned_addr learned my addr 172.21.15.19:0/551371476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:56.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2532600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c1443c0 0x7f252c1fa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 -- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c1443c0 msgr2=0x7f252c1fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c1443c0 0x7f252c1fa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 -- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f252c0cf5e0 msgr2=0x7f252c15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f252c0cf5e0 0x7f252c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 -- 172.21.15.19:0/551371476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c1e8dc0 con 0x7f252c06c5c0 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.341+0000 7f2531c00640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f252c06c5c0 0x7f252c1e8880 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f252000c440 tx=0x7f252000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.342+0000 7f2529600640 1 -- 172.21.15.19:0/551371476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25200052d0 con 0x7f252c06c5c0 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.342+0000 7f2529600640 1 -- 172.21.15.19:0/551371476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2520005470 con 0x7f252c06c5c0 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.342+0000 7f2529600640 1 -- 172.21.15.19:0/551371476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520007260 con 0x7f252c06c5c0 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.342+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252c1f6d90 con 0x7f252c06c5c0 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.342+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252c1e9570 con 0x7f252c06c5c0 2024-04-25T11:34:56.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.343+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252c143fb0 con 0x7f252c06c5c0 2024-04-25T11:34:56.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.344+0000 7f2529600640 1 -- 172.21.15.19:0/551371476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2520007400 con 0x7f252c06c5c0 2024-04-25T11:34:56.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.345+0000 7f2529600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25100744e0 0x7f25100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.346+0000 7f2529600640 1 -- 172.21.15.19:0/551371476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f2520094720 con 0x7f252c06c5c0 2024-04-25T11:34:56.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.346+0000 7f2531200640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25100744e0 0x7f25100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.347+0000 7f2531200640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25100744e0 0x7f25100769f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f251c006a90 tx=0x7f251c006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.350+0000 7f2529600640 1 -- 172.21.15.19:0/551371476 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f252c143fb0 con 0x7f252c06c5c0 2024-04-25T11:34:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.555+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f252c050740 con 0x7f252c06c5c0 2024-04-25T11:34:56.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.557+0000 7f2529600640 1 -- 172.21.15.19:0/551371476 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16735 (secure 0 0 0) 0x7f252c0cff30 con 0x7f252c06c5c0 2024-04-25T11:34:56.547 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:56.547 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:43.091776+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:34:04.018735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T11:34:04.759136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/853310115":"2024-04-26T11:34:43.091612+0000","172.21.15.19:6834/853310115":"2024-04-26T11:34:43.091612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25100744e0 msgr2=0x7f25100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25100744e0 0x7f25100769f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f251c006a90 tx=0x7f251c006a20 comp rx=0 tx=0).stop 2024-04-25T11:34:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f252c06c5c0 msgr2=0x7f252c1e8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f252c06c5c0 0x7f252c1e8880 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f252000c440 tx=0x7f252000c900 comp rx=0 tx=0).stop 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 shutdown_connections 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f25100744e0 0x7f25100769f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f252c0cf5e0 0x7f252c15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f252c1443c0 0x7f252c1fa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 --2- 172.21.15.19:0/551371476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f252c06c5c0 0x7f252c1e8880 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 >> 172.21.15.19:0/551371476 conn(0x7f252c0d1ae0 msgr2=0x7f252c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 shutdown_connections 2024-04-25T11:34:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.560+0000 7f2534600640 1 -- 172.21.15.19:0/551371476 wait complete. 2024-04-25T11:34:56.575 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:34:56.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:56.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.862+0000 7f515b200640 1 Processor -- start 2024-04-25T11:34:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.862+0000 7f515b200640 1 -- start start 2024-04-25T11:34:56.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f515b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f515c0cf2f0 0x7f515c0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f515b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c15a810 0x7f515c0b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f515b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515c0b4e90 con 0x7f515c1608d0 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f515b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f515c06f870 con 0x7f515c0cf2f0 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f515b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f515c0b74d0 con 0x7f515c15a810 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f5159e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f515c0cf2f0 0x7f515c0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f5159e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f515c0cf2f0 0x7f515c0b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39068/0 (socket says 172.21.15.19:39068) 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.865+0000 7f5159e00640 1 -- 172.21.15.19:0/3529911088 learned_addr learned my addr 172.21.15.19:0/3529911088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.866+0000 7f5159400640 1 --2- 172.21.15.19:0/3529911088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c15a810 0x7f515c0b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.866+0000 7f5159e00640 1 -- 172.21.15.19:0/3529911088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c15a810 msgr2=0x7f515c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.866+0000 7f5159e00640 1 --2- 172.21.15.19:0/3529911088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c15a810 0x7f515c0b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.866+0000 7f5159e00640 1 -- 172.21.15.19:0/3529911088 >> v1:172.21.15.169:6789/0 conn(0x7f515c1608d0 legacy=0x7f515c160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:56.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.866+0000 7f5159e00640 1 -- 172.21.15.19:0/3529911088 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c0b77b0 con 0x7f515c0cf2f0 2024-04-25T11:34:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.867+0000 7f5159e00640 1 --2- 172.21.15.19:0/3529911088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c0b6e10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f515c0d26d0 tx=0x7f514c02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132cf61a8cef087b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.867+0000 7f5158a00640 1 -- 172.21.15.19:0/3529911088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c039070 con 0x7f515c0cf2f0 2024-04-25T11:34:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.867+0000 7f5158a00640 1 -- 172.21.15.19:0/3529911088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c034680 con 0x7f515c0cf2f0 2024-04-25T11:34:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.867+0000 7f5158a00640 1 -- 172.21.15.19:0/3529911088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c034a50 con 0x7f515c0cf2f0 2024-04-25T11:34:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 -- 172.21.15.19:0/3529911088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 msgr2=0x7f515c0b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 --2- 172.21.15.19:0/3529911088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c0b6e10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f515c0d26d0 tx=0x7f514c02c8d0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 -- 172.21.15.19:0/3529911088 shutdown_connections 2024-04-25T11:34:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 --2- 172.21.15.19:0/3529911088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c15a810 0x7f515c0b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 --2- 172.21.15.19:0/3529911088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c0b6e10 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 -- 172.21.15.19:0/3529911088 >> 172.21.15.19:0/3529911088 conn(0x7f515c0d1720 msgr2=0x7f515c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 -- 172.21.15.19:0/3529911088 shutdown_connections 2024-04-25T11:34:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.868+0000 7f515b200640 1 -- 172.21.15.19:0/3529911088 wait complete. 2024-04-25T11:34:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.870+0000 7f515b200640 1 Processor -- start 2024-04-25T11:34:56.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.871+0000 7f515b200640 1 -- start start 2024-04-25T11:34:56.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f515b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f515b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515c15a810 0x7f515c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f515b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c1608d0 0x7f515c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f515b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f515c15ac20 con 0x7f515c0cf2f0 2024-04-25T11:34:56.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f515b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f515c06f500 con 0x7f515c15a810 2024-04-25T11:34:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f515b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f515c06fd00 con 0x7f515c1608d0 2024-04-25T11:34:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f5159e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f5159e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39078/0 (socket says 172.21.15.19:39078) 2024-04-25T11:34:56.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.872+0000 7f5159e00640 1 -- 172.21.15.19:0/1457761371 learned_addr learned my addr 172.21.15.19:0/1457761371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f515a800640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c1608d0 0x7f515c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f5159400640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515c15a810 0x7f515c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f5159e00640 1 -- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c1608d0 msgr2=0x7f515c157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f5159e00640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c1608d0 0x7f515c157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f5159e00640 1 -- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515c15a810 msgr2=0x7f515c0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f5159e00640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515c15a810 0x7f515c0b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f5159e00640 1 -- 172.21.15.19:0/1457761371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c0af180 con 0x7f515c0cf2f0 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f515a800640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c1608d0 0x7f515c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.873+0000 7f5159e00640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c151f00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f514c037040 tx=0x7f514c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.874+0000 7f5152000640 1 -- 172.21.15.19:0/1457761371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c039040 con 0x7f515c0cf2f0 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.874+0000 7f5152000640 1 -- 172.21.15.19:0/1457761371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c042070 con 0x7f515c0cf2f0 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.874+0000 7f5152000640 1 -- 172.21.15.19:0/1457761371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c03d420 con 0x7f515c0cf2f0 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.874+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c0af460 con 0x7f515c0cf2f0 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.874+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c0afa20 con 0x7f515c0cf2f0 2024-04-25T11:34:56.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.875+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c050740 con 0x7f515c0cf2f0 2024-04-25T11:34:56.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.876+0000 7f5152000640 1 -- 172.21.15.19:0/1457761371 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f514c03d5c0 con 0x7f515c0cf2f0 2024-04-25T11:34:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.877+0000 7f5152000640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f513c0744b0 0x7f513c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.877+0000 7f5152000640 1 -- 172.21.15.19:0/1457761371 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f514c0b5730 con 0x7f515c0cf2f0 2024-04-25T11:34:56.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.878+0000 7f5159400640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f513c0744b0 0x7f513c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.879+0000 7f5159400640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f513c0744b0 0x7f513c0769c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5148006fd0 tx=0x7f5148008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:56.882+0000 7f5152000640 1 -- 172.21.15.19:0/1457761371 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f514c07fdc0 con 0x7f515c0cf2f0 2024-04-25T11:34:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.117+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f515c06c770 con 0x7f515c0cf2f0 2024-04-25T11:34:57.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.118+0000 7f5152000640 1 -- 172.21.15.19:0/1457761371 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5191 (secure 0 0 0) 0x7f514c07f6c0 con 0x7f515c0cf2f0 2024-04-25T11:34:57.107 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:57.108 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3655604895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3655604895},{"type":"v1","addr":"172.21.15.19:6835","nonce":3655604895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:46.124891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5845":{"gid":5845,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":14,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5845,"qdb_cluster":[5845]},"id":5}]} 2024-04-25T11:34:57.108 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 81 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.120+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f513c0744b0 msgr2=0x7f513c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.120+0000 7f515b200640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f513c0744b0 0x7f513c0769c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5148006fd0 tx=0x7f5148008040 comp rx=0 tx=0).stop 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 msgr2=0x7f515c151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c151f00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f514c037040 tx=0x7f514c004a40 comp rx=0 tx=0).stop 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 shutdown_connections 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f513c0744b0 0x7f513c0769c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f515c15a810 0x7f515c0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f515c1608d0 0x7f515c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 --2- 172.21.15.19:0/1457761371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c0cf2f0 0x7f515c151f00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 >> 172.21.15.19:0/1457761371 conn(0x7f515c0d1720 msgr2=0x7f515c072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 shutdown_connections 2024-04-25T11:34:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.121+0000 7f515b200640 1 -- 172.21.15.19:0/1457761371 wait complete. 2024-04-25T11:34:57.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:34:57.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.430+0000 7fd019e00640 1 Processor -- start 2024-04-25T11:34:57.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.431+0000 7fd019e00640 1 -- start start 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.433+0000 7fd019e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd0141443c0 0x7fd01406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.433+0000 7fd019e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0140cfcf0 0x7fd01406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.433+0000 7fd019e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd01415f5f0 con 0x7fd014070370 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.433+0000 7fd019e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd0140708d0 con 0x7fd0141443c0 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.433+0000 7fd019e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd01415f160 con 0x7fd0140cfcf0 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.433+0000 7fd012a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd0141443c0 0x7fd01406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd012a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd0141443c0 0x7fd01406fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39092/0 (socket says 172.21.15.19:39092) 2024-04-25T11:34:57.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd012a00640 1 -- 172.21.15.19:0/2767458463 learned_addr learned my addr 172.21.15.19:0/2767458463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd013400640 1 --2- 172.21.15.19:0/2767458463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0140cfcf0 0x7fd01406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd012a00640 1 -- 172.21.15.19:0/2767458463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0140cfcf0 msgr2=0x7fd01406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd012a00640 1 --2- 172.21.15.19:0/2767458463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0140cfcf0 0x7fd01406c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd012a00640 1 -- 172.21.15.19:0/2767458463 >> v1:172.21.15.19:6789/0 conn(0x7fd014070370 legacy=0x7fd0140cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd012000640 1 -- 172.21.15.19:0/2767458463 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3138817225 0 0) 0x7fd01415f5f0 con 0x7fd014070370 2024-04-25T11:34:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd012a00640 1 -- 172.21.15.19:0/2767458463 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01415e830 con 0x7fd0141443c0 2024-04-25T11:34:57.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.434+0000 7fd013400640 1 --2- 172.21.15.19:0/2767458463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0140cfcf0 0x7fd01406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:57.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.435+0000 7fd012a00640 1 --2- 172.21.15.19:0/2767458463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0141443c0 0x7fd01406fe30 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd004009870 tx=0x7fd00402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a745d4d91699ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd012000640 1 -- 172.21.15.19:0/2767458463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004037070 con 0x7fd0141443c0 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd012000640 1 -- 172.21.15.19:0/2767458463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd004032440 con 0x7fd0141443c0 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd012000640 1 -- 172.21.15.19:0/2767458463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004032810 con 0x7fd0141443c0 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd019e00640 1 -- 172.21.15.19:0/2767458463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0141443c0 msgr2=0x7fd01406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd019e00640 1 --2- 172.21.15.19:0/2767458463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0141443c0 0x7fd01406fe30 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd004009870 tx=0x7fd00402a250 comp rx=0 tx=0).stop 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd019e00640 1 -- 172.21.15.19:0/2767458463 shutdown_connections 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd019e00640 1 --2- 172.21.15.19:0/2767458463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0140cfcf0 0x7fd01406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.436+0000 7fd019e00640 1 --2- 172.21.15.19:0/2767458463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0141443c0 0x7fd01406fe30 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.437+0000 7fd019e00640 1 -- 172.21.15.19:0/2767458463 >> 172.21.15.19:0/2767458463 conn(0x7fd0140d1ae0 msgr2=0x7fd0140d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.437+0000 7fd019e00640 1 -- 172.21.15.19:0/2767458463 shutdown_connections 2024-04-25T11:34:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.437+0000 7fd019e00640 1 -- 172.21.15.19:0/2767458463 wait complete. 2024-04-25T11:34:57.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.439+0000 7fd019e00640 1 Processor -- start 2024-04-25T11:34:57.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.439+0000 7fd019e00640 1 -- start start 2024-04-25T11:34:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.440+0000 7fd019e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd014070370 0x7fd0141361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.440+0000 7fd019e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0141443c0 0x7fd0140af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.440+0000 7fd019e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0140af7d0 0x7fd0140aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd019e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141447d0 con 0x7fd0140af7d0 2024-04-25T11:34:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd019e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd01415edf0 con 0x7fd0141443c0 2024-04-25T11:34:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd019e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd014162b00 con 0x7fd014070370 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0140af7d0 0x7fd0140aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0140af7d0 0x7fd0140aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39108/0 (socket says 172.21.15.19:39108) 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 -- 172.21.15.19:0/2415915504 learned_addr learned my addr 172.21.15.19:0/2415915504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd012a00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd014070370 0x7fd0141361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd00be00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0141443c0 0x7fd0140af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 -- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd014070370 msgr2=0x7fd0141361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd014070370 0x7fd0141361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 -- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0141443c0 msgr2=0x7fd0140af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0141443c0 0x7fd0140af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd013400640 1 -- 172.21.15.19:0/2415915504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd014136720 con 0x7fd0140af7d0 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.441+0000 7fd00be00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0141443c0 0x7fd0140af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.442+0000 7fd013400640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0140af7d0 0x7fd0140aabb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00cd80 tx=0x7fd00c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.442+0000 7fd00b400640 1 -- 172.21.15.19:0/2415915504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c00f680 con 0x7fd0140af7d0 2024-04-25T11:34:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.442+0000 7fd00b400640 1 -- 172.21.15.19:0/2415915504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00c00f840 con 0x7fd0140af7d0 2024-04-25T11:34:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.442+0000 7fd00b400640 1 -- 172.21.15.19:0/2415915504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c00fc30 con 0x7fd0140af7d0 2024-04-25T11:34:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.442+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0140b00a0 con 0x7fd0140af7d0 2024-04-25T11:34:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.442+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0140b0660 con 0x7fd0140af7d0 2024-04-25T11:34:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.443+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd014142700 con 0x7fd0140af7d0 2024-04-25T11:34:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.445+0000 7fd00b400640 1 -- 172.21.15.19:0/2415915504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd00c00e3e0 con 0x7fd0140af7d0 2024-04-25T11:34:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.446+0000 7fd00b400640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcff00744b0 0x7fcff00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.446+0000 7fd00b400640 1 -- 172.21.15.19:0/2415915504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fd00c095ff0 con 0x7fd0140af7d0 2024-04-25T11:34:57.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.447+0000 7fd012a00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcff00744b0 0x7fcff00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.448+0000 7fd012a00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcff00744b0 0x7fcff00769c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd004034000 tx=0x7fd00402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.451+0000 7fd00b400640 1 -- 172.21.15.19:0/2415915504 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd00c014030 con 0x7fd0140af7d0 2024-04-25T11:34:57.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.703+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd014050740 con 0x7fd0140af7d0 2024-04-25T11:34:57.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.705+0000 7fd00b400640 1 -- 172.21.15.19:0/2415915504 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5191 (secure 0 0 0) 0x7fd00c0606c0 con 0x7fd0140af7d0 2024-04-25T11:34:57.694 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:57.694 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3655604895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3655604895},{"type":"v1","addr":"172.21.15.19:6835","nonce":3655604895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:34:05.831545+0000","modified":"2024-04-25T11:34:46.124891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5845":{"gid":5845,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":14,"addr":"172.21.15.169:6836/2118499768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2118499768},{"type":"v1","addr":"172.21.15.169:6836","nonce":2118499768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5845,"qdb_cluster":[5845]},"id":5}]} 2024-04-25T11:34:57.694 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 81 2024-04-25T11:34:57.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcff00744b0 msgr2=0x7fcff00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcff00744b0 0x7fcff00769c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd004034000 tx=0x7fd00402e000 comp rx=0 tx=0).stop 2024-04-25T11:34:57.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0140af7d0 msgr2=0x7fd0140aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0140af7d0 0x7fd0140aabb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00cd80 tx=0x7fd00c00a650 comp rx=0 tx=0).stop 2024-04-25T11:34:57.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 shutdown_connections 2024-04-25T11:34:57.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcff00744b0 0x7fcff00769c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd0141443c0 0x7fd0140af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0140af7d0 0x7fd0140aabb0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 --2- 172.21.15.19:0/2415915504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd014070370 0x7fd0141361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 >> 172.21.15.19:0/2415915504 conn(0x7fd0140d1ae0 msgr2=0x7fd01415d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:57.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 shutdown_connections 2024-04-25T11:34:57.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:57.708+0000 7fd019e00640 1 -- 172.21.15.19:0/2415915504 wait complete. 2024-04-25T11:34:57.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:34:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.022+0000 7fac53200640 1 Processor -- start 2024-04-25T11:34:58.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.022+0000 7fac53200640 1 -- start start 2024-04-25T11:34:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.024+0000 7fac53200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540b6e10 0x7fac5415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.024+0000 7fac53200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fac541608d0 0x7fac54160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.024+0000 7fac53200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac54161220 con 0x7fac540cf2f0 2024-04-25T11:34:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.024+0000 7fac53200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fac5406f870 con 0x7fac541608d0 2024-04-25T11:34:58.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.024+0000 7fac53200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac540b7220 con 0x7fac540b6e10 2024-04-25T11:34:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.024+0000 7fac51400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540b6e10 0x7fac5415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.025+0000 7fac51400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540b6e10 0x7fac5415a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52602/0 (socket says 172.21.15.19:52602) 2024-04-25T11:34:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.025+0000 7fac51400640 1 -- 172.21.15.19:0/729122665 learned_addr learned my addr 172.21.15.19:0/729122665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:58.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.025+0000 7fac52800640 1 --2- 172.21.15.19:0/729122665 >> v2:172.21.15.169:3300/0 conn(0x7fac541608d0 0x7fac54160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.025+0000 7fac51400640 1 -- 172.21.15.19:0/729122665 >> v2:172.21.15.169:3300/0 conn(0x7fac541608d0 msgr2=0x7fac54160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.025+0000 7fac51400640 1 --2- 172.21.15.19:0/729122665 >> v2:172.21.15.169:3300/0 conn(0x7fac541608d0 0x7fac54160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.025+0000 7fac51400640 1 -- 172.21.15.19:0/729122665 >> v1:172.21.15.169:6789/0 conn(0x7fac540cf2f0 legacy=0x7fac540b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.025+0000 7fac51400640 1 -- 172.21.15.19:0/729122665 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac540b74d0 con 0x7fac540b6e10 2024-04-25T11:34:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.027+0000 7fac51400640 1 --2- 172.21.15.19:0/729122665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540b6e10 0x7fac5415a810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac40009870 tx=0x7fac4002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86b59687dcbc030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.027+0000 7fac50a00640 1 -- 172.21.15.19:0/729122665 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40037070 con 0x7fac540b6e10 2024-04-25T11:34:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.027+0000 7fac50a00640 1 -- 172.21.15.19:0/729122665 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac40032440 con 0x7fac540b6e10 2024-04-25T11:34:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.027+0000 7fac50a00640 1 -- 172.21.15.19:0/729122665 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40032810 con 0x7fac540b6e10 2024-04-25T11:34:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 -- 172.21.15.19:0/729122665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540b6e10 msgr2=0x7fac5415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 --2- 172.21.15.19:0/729122665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540b6e10 0x7fac5415a810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac40009870 tx=0x7fac4002a250 comp rx=0 tx=0).stop 2024-04-25T11:34:58.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 -- 172.21.15.19:0/729122665 shutdown_connections 2024-04-25T11:34:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 --2- 172.21.15.19:0/729122665 >> v2:172.21.15.169:3300/0 conn(0x7fac541608d0 0x7fac54160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 --2- 172.21.15.19:0/729122665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540b6e10 0x7fac5415a810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 -- 172.21.15.19:0/729122665 >> 172.21.15.19:0/729122665 conn(0x7fac540d1720 msgr2=0x7fac540d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 -- 172.21.15.19:0/729122665 shutdown_connections 2024-04-25T11:34:58.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.028+0000 7fac53200640 1 -- 172.21.15.19:0/729122665 wait complete. 2024-04-25T11:34:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.031+0000 7fac53200640 1 Processor -- start 2024-04-25T11:34:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.031+0000 7fac53200640 1 -- start start 2024-04-25T11:34:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.032+0000 7fac53200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac540b6e10 0x7fac5414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac53200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540cf2f0 0x7fac541e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac53200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac541608d0 0x7fac540afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac53200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac5406f500 con 0x7fac540b6e10 2024-04-25T11:34:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac53200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac5406f220 con 0x7fac541608d0 2024-04-25T11:34:58.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac53200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac5406fd00 con 0x7fac540cf2f0 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac540b6e10 0x7fac5414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac540b6e10 0x7fac5414a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39120/0 (socket says 172.21.15.19:39120) 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 -- 172.21.15.19:0/4118420573 learned_addr learned my addr 172.21.15.19:0/4118420573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51400640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540cf2f0 0x7fac541e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac52800640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac541608d0 0x7fac540afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 -- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540cf2f0 msgr2=0x7fac541e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540cf2f0 0x7fac541e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 -- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac541608d0 msgr2=0x7fac540afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac541608d0 0x7fac540afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51e00640 1 -- 172.21.15.19:0/4118420573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac541e73b0 con 0x7fac540b6e10 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac51400640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540cf2f0 0x7fac541e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.033+0000 7fac52800640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac541608d0 0x7fac540afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:58.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.034+0000 7fac51e00640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac540b6e10 0x7fac5414a3f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fac4400c3c0 tx=0x7fac4400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.034+0000 7fac4a000640 1 -- 172.21.15.19:0/4118420573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac440052d0 con 0x7fac540b6e10 2024-04-25T11:34:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.034+0000 7fac4a000640 1 -- 172.21.15.19:0/4118420573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac44005490 con 0x7fac540b6e10 2024-04-25T11:34:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.034+0000 7fac4a000640 1 -- 172.21.15.19:0/4118420573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44007260 con 0x7fac540b6e10 2024-04-25T11:34:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.034+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac541e7690 con 0x7fac540b6e10 2024-04-25T11:34:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.034+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac541e7c50 con 0x7fac540b6e10 2024-04-25T11:34:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.035+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac54050740 con 0x7fac540b6e10 2024-04-25T11:34:58.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.037+0000 7fac4a000640 1 -- 172.21.15.19:0/4118420573 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac44007400 con 0x7fac540b6e10 2024-04-25T11:34:58.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.038+0000 7fac4a000640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac340744b0 0x7fac340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.039+0000 7fac4a000640 1 -- 172.21.15.19:0/4118420573 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7fac440946d0 con 0x7fac540b6e10 2024-04-25T11:34:58.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.039+0000 7fac51400640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac340744b0 0x7fac340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.040+0000 7fac51400640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac340744b0 0x7fac340769c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fac40034000 tx=0x7fac4002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:58.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.044+0000 7fac4a000640 1 -- 172.21.15.19:0/4118420573 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac54161220 con 0x7fac540b6e10 2024-04-25T11:34:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.251+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac540d0130 con 0x7fac540b6e10 2024-04-25T11:34:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.252+0000 7fac4a000640 1 -- 172.21.15.19:0/4118420573 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16735 (secure 0 0 0) 0x7fac44065e20 con 0x7fac540b6e10 2024-04-25T11:34:58.242 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:34:58.242 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:34:43.091776+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:34:04.018735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T11:34:04.759136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/853310115":"2024-04-26T11:34:43.091612+0000","172.21.15.19:6834/853310115":"2024-04-26T11:34:43.091612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:34:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac340744b0 msgr2=0x7fac340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac340744b0 0x7fac340769c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fac40034000 tx=0x7fac4002e000 comp rx=0 tx=0).stop 2024-04-25T11:34:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac540b6e10 msgr2=0x7fac5414a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac540b6e10 0x7fac5414a3f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fac4400c3c0 tx=0x7fac4400c880 comp rx=0 tx=0).stop 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 shutdown_connections 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac340744b0 0x7fac340769c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac541608d0 0x7fac540afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac540cf2f0 0x7fac541e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 --2- 172.21.15.19:0/4118420573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac540b6e10 0x7fac5414a3f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 >> 172.21.15.19:0/4118420573 conn(0x7fac540d1720 msgr2=0x7fac5406ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 shutdown_connections 2024-04-25T11:34:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.255+0000 7fac53200640 1 -- 172.21.15.19:0/4118420573 wait complete. 2024-04-25T11:34:58.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:34:58.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.570+0000 7f2785400640 1 Processor -- start 2024-04-25T11:34:58.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.570+0000 7f2785400640 1 -- start start 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.572+0000 7f2785400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27800b90e0 0x7f27800b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.572+0000 7f2785400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f278008cd60 0x7f278008d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.572+0000 7f2785400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278008d6b0 con 0x7f278006e680 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.572+0000 7f2785400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f27800d1d70 con 0x7f278008cd60 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.572+0000 7f2785400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27800ce6d0 con 0x7f27800b90e0 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f2784a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f278008cd60 0x7f278008d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f277fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f278006e680 0x7f27800cdfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34856/0 (socket says 172.21.15.19:34856) 2024-04-25T11:34:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f277fe00640 1 -- 172.21.15.19:0/3351843555 learned_addr learned my addr 172.21.15.19:0/3351843555 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f2777400640 1 --2- 172.21.15.19:0/3351843555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27800b90e0 0x7f27800b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f2784a00640 1 -- 172.21.15.19:0/3351843555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27800b90e0 msgr2=0x7f27800b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f2784a00640 1 --2- 172.21.15.19:0/3351843555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27800b90e0 0x7f27800b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f2784a00640 1 -- 172.21.15.19:0/3351843555 >> v1:172.21.15.19:6789/0 conn(0x7f278006e680 legacy=0x7f27800cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.574+0000 7f2784a00640 1 -- 172.21.15.19:0/3351843555 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27800d1fa0 con 0x7f278008cd60 2024-04-25T11:34:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.573+0000 7f277f400640 1 -- 172.21.15.19:0/3351843555 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746316891 0 0) 0x7f2770001290 con 0x7f278006e680 2024-04-25T11:34:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.575+0000 7f2784a00640 1 --2- 172.21.15.19:0/3351843555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278008cd60 0x7f278008d170 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2778009d90 tx=0x7f277802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5950ab6619081fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.575+0000 7f277f400640 1 -- 172.21.15.19:0/3351843555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778031600 con 0x7f278008cd60 2024-04-25T11:34:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.575+0000 7f277f400640 1 -- 172.21.15.19:0/3351843555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27780317a0 con 0x7f278008cd60 2024-04-25T11:34:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.575+0000 7f277f400640 1 -- 172.21.15.19:0/3351843555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778031b70 con 0x7f278008cd60 2024-04-25T11:34:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 -- 172.21.15.19:0/3351843555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278008cd60 msgr2=0x7f278008d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 --2- 172.21.15.19:0/3351843555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278008cd60 0x7f278008d170 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2778009d90 tx=0x7f277802a5d0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 -- 172.21.15.19:0/3351843555 shutdown_connections 2024-04-25T11:34:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 --2- 172.21.15.19:0/3351843555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278008cd60 0x7f278008d170 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 --2- 172.21.15.19:0/3351843555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27800b90e0 0x7f27800b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 -- 172.21.15.19:0/3351843555 >> 172.21.15.19:0/3351843555 conn(0x7f27800b5460 msgr2=0x7f27800b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 -- 172.21.15.19:0/3351843555 shutdown_connections 2024-04-25T11:34:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.576+0000 7f2785400640 1 -- 172.21.15.19:0/3351843555 wait complete. 2024-04-25T11:34:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.578+0000 7f2785400640 1 Processor -- start 2024-04-25T11:34:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.578+0000 7f2785400640 1 -- start start 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2785400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278006e680 0x7f27800ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2785400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278008cd60 0x7f27800cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2785400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27800b90e0 0x7f27800cf580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2785400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27800d2560 con 0x7f27800b90e0 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2785400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27800b94f0 con 0x7f278006e680 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2785400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27800d2840 con 0x7f278008cd60 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2784a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27800b90e0 0x7f27800cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2784a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27800b90e0 0x7f27800cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39130/0 (socket says 172.21.15.19:39130) 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.580+0000 7f2784a00640 1 -- 172.21.15.19:0/3746598905 learned_addr learned my addr 172.21.15.19:0/3746598905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2777400640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278008cd60 0x7f27800cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f277fe00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278006e680 0x7f27800ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2784a00640 1 -- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278008cd60 msgr2=0x7f27800cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2784a00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278008cd60 0x7f27800cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2784a00640 1 -- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278006e680 msgr2=0x7f27800ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2784a00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278006e680 0x7f27800ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2784a00640 1 -- 172.21.15.19:0/3746598905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27800c7940 con 0x7f27800b90e0 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f277fe00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278006e680 0x7f27800ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2784a00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27800b90e0 0x7f27800cf580 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2778009d90 tx=0x7f2778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f277cc00640 1 -- 172.21.15.19:0/3746598905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778040070 con 0x7f27800b90e0 2024-04-25T11:34:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f277cc00640 1 -- 172.21.15.19:0/3746598905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2778030b60 con 0x7f27800b90e0 2024-04-25T11:34:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f277cc00640 1 -- 172.21.15.19:0/3746598905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778009450 con 0x7f27800b90e0 2024-04-25T11:34:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.581+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27800c7c20 con 0x7f27800b90e0 2024-04-25T11:34:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.582+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27800c81e0 con 0x7f27800b90e0 2024-04-25T11:34:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.583+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2780050740 con 0x7f27800b90e0 2024-04-25T11:34:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.584+0000 7f277cc00640 1 -- 172.21.15.19:0/3746598905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2778039040 con 0x7f27800b90e0 2024-04-25T11:34:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.585+0000 7f277cc00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f275c0744e0 0x7f275c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.585+0000 7f277cc00640 1 -- 172.21.15.19:0/3746598905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5795+0+0 (secure 0 0 0) 0x7f27780b61e0 con 0x7f27800b90e0 2024-04-25T11:34:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.586+0000 7f277fe00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f275c0744e0 0x7f275c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.587+0000 7f277fe00640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f275c0744e0 0x7f275c0769f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2780074b10 tx=0x7f2770009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.590+0000 7f277cc00640 1 -- 172.21.15.19:0/3746598905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2778080900 con 0x7f27800b90e0 2024-04-25T11:34:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.835+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27800b85c0 con 0x7f27800b90e0 2024-04-25T11:34:58.825 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:34:58.836+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:34:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.841+0000 7f277cc00640 1 -- 172.21.15.19:0/3746598905 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 161+0+0 (secure 0 0 0) 0x7f2778080200 con 0x7f27800b90e0 2024-04-25T11:34:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.843+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f275c0744e0 msgr2=0x7f275c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.843+0000 7f2785400640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f275c0744e0 0x7f275c0769f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2780074b10 tx=0x7f2770009040 comp rx=0 tx=0).stop 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.843+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27800b90e0 msgr2=0x7f27800cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.843+0000 7f2785400640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27800b90e0 0x7f27800cf580 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2778009d90 tx=0x7f2778004a40 comp rx=0 tx=0).stop 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 shutdown_connections 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f275c0744e0 0x7f275c0769f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27800b90e0 0x7f27800cf580 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278008cd60 0x7f27800cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 --2- 172.21.15.19:0/3746598905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278006e680 0x7f27800ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 >> 172.21.15.19:0/3746598905 conn(0x7f27800b5460 msgr2=0x7f27800d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 shutdown_connections 2024-04-25T11:34:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:58.844+0000 7f2785400640 1 -- 172.21.15.19:0/3746598905 wait complete. 2024-04-25T11:34:58.860 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:34:58.872 INFO:tasks.ceph.mds.b.smithi169.stderr: -360> 2024-04-25T11:34:50.143+0000 7f9eca600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:34:59.002 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:34:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.158+0000 7fc058a00640 1 Processor -- start 2024-04-25T11:34:59.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.159+0000 7fc058a00640 1 -- start start 2024-04-25T11:34:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc058a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0540d1340 0x7fc054070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc058a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0540707a0 0x7fc054140490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc058a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc054070bb0 con 0x7fc05413ff60 2024-04-25T11:34:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc058a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc05415e340 con 0x7fc0540d1340 2024-04-25T11:34:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc058a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc0540d02a0 con 0x7fc0540707a0 2024-04-25T11:34:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc053400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc05413ff60 0x7fc05415dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34860/0 (socket says 172.21.15.19:34860) 2024-04-25T11:34:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc053400640 1 -- 172.21.15.19:0/2957987904 learned_addr learned my addr 172.21.15.19:0/2957987904 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:34:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.161+0000 7fc052a00640 1 --2- 172.21.15.19:0/2957987904 >> v2:172.21.15.19:3300/0 conn(0x7fc0540d1340 0x7fc054070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.162+0000 7fc053e00640 1 --2- 172.21.15.19:0/2957987904 >> v2:172.21.15.169:3300/0 conn(0x7fc0540707a0 0x7fc054140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.162+0000 7fc052a00640 1 -- 172.21.15.19:0/2957987904 >> v2:172.21.15.169:3300/0 conn(0x7fc0540707a0 msgr2=0x7fc054140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:34:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.162+0000 7fc052a00640 1 --2- 172.21.15.19:0/2957987904 >> v2:172.21.15.169:3300/0 conn(0x7fc0540707a0 0x7fc054140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.162+0000 7fc052a00640 1 -- 172.21.15.19:0/2957987904 >> v1:172.21.15.19:6789/0 conn(0x7fc05413ff60 legacy=0x7fc05415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.162+0000 7fc052a00640 1 -- 172.21.15.19:0/2957987904 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0540d0580 con 0x7fc0540d1340 2024-04-25T11:34:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.162+0000 7fc053e00640 1 --2- 172.21.15.19:0/2957987904 >> v2:172.21.15.169:3300/0 conn(0x7fc0540707a0 0x7fc054140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:34:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.162+0000 7fc052000640 1 -- 172.21.15.19:0/2957987904 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1215476957 0 0) 0x7fc044001110 con 0x7fc05413ff60 2024-04-25T11:34:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.163+0000 7fc052a00640 1 --2- 172.21.15.19:0/2957987904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0540d1340 0x7fc054070260 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc0540714e0 tx=0x7fc04002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d52a10b424ddcbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.163+0000 7fc052000640 1 -- 172.21.15.19:0/2957987904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040037070 con 0x7fc0540d1340 2024-04-25T11:34:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.163+0000 7fc052000640 1 -- 172.21.15.19:0/2957987904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc040032440 con 0x7fc0540d1340 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.163+0000 7fc052000640 1 -- 172.21.15.19:0/2957987904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04002ae60 con 0x7fc0540d1340 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 -- 172.21.15.19:0/2957987904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0540d1340 msgr2=0x7fc054070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 --2- 172.21.15.19:0/2957987904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0540d1340 0x7fc054070260 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc0540714e0 tx=0x7fc04002a110 comp rx=0 tx=0).stop 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 -- 172.21.15.19:0/2957987904 shutdown_connections 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 --2- 172.21.15.19:0/2957987904 >> v2:172.21.15.169:3300/0 conn(0x7fc0540707a0 0x7fc054140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 --2- 172.21.15.19:0/2957987904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0540d1340 0x7fc054070260 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 -- 172.21.15.19:0/2957987904 >> 172.21.15.19:0/2957987904 conn(0x7fc0540b4260 msgr2=0x7fc054072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 -- 172.21.15.19:0/2957987904 shutdown_connections 2024-04-25T11:34:59.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.164+0000 7fc058a00640 1 -- 172.21.15.19:0/2957987904 wait complete. 2024-04-25T11:34:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.167+0000 7fc058a00640 1 Processor -- start 2024-04-25T11:34:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.167+0000 7fc058a00640 1 -- start start 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc058a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0540d1340 0x7fc0540b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc058a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc05413ff60 0x7fc054142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc058a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc054142a80 0x7fc054160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc058a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0540d0e20 con 0x7fc05413ff60 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc058a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0540d0b40 con 0x7fc054142a80 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc058a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc05415d600 con 0x7fc0540d1340 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc05413ff60 0x7fc054142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc05413ff60 0x7fc054142540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39146/0 (socket says 172.21.15.19:39146) 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 -- 172.21.15.19:0/2638204553 learned_addr learned my addr 172.21.15.19:0/2638204553 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 -- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0540d1340 msgr2=0x7fc0540b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc053400640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0540d1340 0x7fc0540b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc053e00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc054142a80 0x7fc054160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0540d1340 0x7fc0540b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 -- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc054142a80 msgr2=0x7fc054160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc054142a80 0x7fc054160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.169+0000 7fc052a00640 1 -- 172.21.15.19:0/2638204553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc054142e90 con 0x7fc05413ff60 2024-04-25T11:34:59.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.170+0000 7fc052a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc05413ff60 0x7fc054142540 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc0400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.170+0000 7fc04b400640 1 -- 172.21.15.19:0/2638204553 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040037040 con 0x7fc05413ff60 2024-04-25T11:34:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.170+0000 7fc04b400640 1 -- 172.21.15.19:0/2638204553 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc040042070 con 0x7fc05413ff60 2024-04-25T11:34:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.170+0000 7fc04b400640 1 -- 172.21.15.19:0/2638204553 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040030bd0 con 0x7fc05413ff60 2024-04-25T11:34:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.170+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05413b430 con 0x7fc05413ff60 2024-04-25T11:34:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.170+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05415efb0 con 0x7fc05413ff60 2024-04-25T11:34:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.172+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0540b77e0 con 0x7fc05413ff60 2024-04-25T11:34:59.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.173+0000 7fc04b400640 1 -- 172.21.15.19:0/2638204553 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc040047460 con 0x7fc05413ff60 2024-04-25T11:34:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.174+0000 7fc04b400640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0340744b0 0x7fc0340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.175+0000 7fc04b400640 1 -- 172.21.15.19:0/2638204553 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5881+0+0 (secure 0 0 0) 0x7fc0400b60e0 con 0x7fc05413ff60 2024-04-25T11:34:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.175+0000 7fc053400640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0340744b0 0x7fc0340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.176+0000 7fc053400640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0340744b0 0x7fc0340769c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc044000f60 tx=0x7fc044009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:59.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.180+0000 7fc04b400640 1 -- 172.21.15.19:0/2638204553 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc040032e20 con 0x7fc05413ff60 2024-04-25T11:34:59.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.425+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc054076c60 con 0x7fc05413ff60 2024-04-25T11:34:59.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.845+0000 7fc04b400640 1 -- 172.21.15.19:0/2638204553 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) ==== 104+0+0 (secure 0 0 0) 0x7fc040080720 con 0x7fc05413ff60 2024-04-25T11:34:59.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0340744b0 msgr2=0x7fc0340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0340744b0 0x7fc0340769c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc044000f60 tx=0x7fc044009040 comp rx=0 tx=0).stop 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc05413ff60 msgr2=0x7fc054142540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc05413ff60 0x7fc054142540 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc0400046c0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 shutdown_connections 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc0340744b0 0x7fc0340769c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc054142a80 0x7fc054160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc05413ff60 0x7fc054142540 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 --2- 172.21.15.19:0/2638204553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0540d1340 0x7fc0540b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 >> 172.21.15.19:0/2638204553 conn(0x7fc0540707a0 msgr2=0x7fc054153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.848+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 shutdown_connections 2024-04-25T11:34:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:34:59.849+0000 7fc058a00640 1 -- 172.21.15.19:0/2638204553 wait complete. 2024-04-25T11:34:59.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:35:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.168+0000 7f0263e00640 1 Processor -- start 2024-04-25T11:35:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.168+0000 7f0263e00640 1 -- start start 2024-04-25T11:35:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.170+0000 7f0263e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f026415e160 0x7f02640ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.170+0000 7f0263e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f026406f370 0x7f026406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.170+0000 7f0263e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 0x7f026406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.170+0000 7f0263e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f026406cab0 con 0x7f026406f370 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.170+0000 7f0263e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f02640714a0 con 0x7f026415e160 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.170+0000 7f0263e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f026406fcc0 con 0x7f026406cf40 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f026406f370 0x7f026406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f026406f370 0x7f026406f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39162/0 (socket says 172.21.15.19:39162) 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 -- 172.21.15.19:0/3834638017 learned_addr learned my addr 172.21.15.19:0/3834638017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f0262a00640 1 --2- 172.21.15.19:0/3834638017 >> v2:172.21.15.169:3300/0 conn(0x7f026415e160 0x7f02640ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f0263400640 1 --2- 172.21.15.19:0/3834638017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 0x7f026406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 -- 172.21.15.19:0/3834638017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 msgr2=0x7f026406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 --2- 172.21.15.19:0/3834638017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 0x7f026406d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 -- 172.21.15.19:0/3834638017 >> v2:172.21.15.169:3300/0 conn(0x7f026415e160 msgr2=0x7f02640ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 --2- 172.21.15.19:0/3834638017 >> v2:172.21.15.169:3300/0 conn(0x7f026415e160 0x7f02640ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f025be00640 1 -- 172.21.15.19:0/3834638017 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02640cd440 con 0x7f026406f370 2024-04-25T11:35:00.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.171+0000 7f0262a00640 1 --2- 172.21.15.19:0/3834638017 >> v2:172.21.15.169:3300/0 conn(0x7f026415e160 0x7f02640ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:00.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.173+0000 7f025be00640 1 --2- 172.21.15.19:0/3834638017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026406f370 0x7f026406f780 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0264070990 tx=0x7f024c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2198c69063c5bb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:00.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.173+0000 7f0262000640 1 -- 172.21.15.19:0/3834638017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c037070 con 0x7f026406f370 2024-04-25T11:35:00.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.173+0000 7f0262000640 1 -- 172.21.15.19:0/3834638017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c032440 con 0x7f026406f370 2024-04-25T11:35:00.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.173+0000 7f0262000640 1 -- 172.21.15.19:0/3834638017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c032810 con 0x7f026406f370 2024-04-25T11:35:00.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 -- 172.21.15.19:0/3834638017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026406f370 msgr2=0x7f026406f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:00.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 --2- 172.21.15.19:0/3834638017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026406f370 0x7f026406f780 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0264070990 tx=0x7f024c02a140 comp rx=0 tx=0).stop 2024-04-25T11:35:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 -- 172.21.15.19:0/3834638017 shutdown_connections 2024-04-25T11:35:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 --2- 172.21.15.19:0/3834638017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026406f370 0x7f026406f780 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 --2- 172.21.15.19:0/3834638017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 0x7f026406d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 --2- 172.21.15.19:0/3834638017 >> v2:172.21.15.169:3300/0 conn(0x7f026415e160 0x7f02640ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 -- 172.21.15.19:0/3834638017 >> 172.21.15.19:0/3834638017 conn(0x7f02640ce3b0 msgr2=0x7f02640ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 -- 172.21.15.19:0/3834638017 shutdown_connections 2024-04-25T11:35:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.174+0000 7f0263e00640 1 -- 172.21.15.19:0/3834638017 wait complete. 2024-04-25T11:35:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.176+0000 7f0263e00640 1 Processor -- start 2024-04-25T11:35:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.176+0000 7f0263e00640 1 -- start start 2024-04-25T11:35:00.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 0x7f02640cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:00.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026406f370 0x7f02640b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:00.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02640b6a40 0x7f02640b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:00.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f026406c740 con 0x7f02640b6a40 2024-04-25T11:35:00.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02640cdec0 con 0x7f026406f370 2024-04-25T11:35:00.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02640b4820 con 0x7f026406cf40 2024-04-25T11:35:00.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02640b6a40 0x7f02640b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:00.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02640b6a40 0x7f02640b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39178/0 (socket says 172.21.15.19:39178) 2024-04-25T11:35:00.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.178+0000 7f0263400640 1 -- 172.21.15.19:0/2152913499 learned_addr learned my addr 172.21.15.19:0/2152913499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263400640 1 -- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 msgr2=0x7f02640cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263400640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 0x7f02640cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263400640 1 -- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026406f370 msgr2=0x7f02640b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:00.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263400640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026406f370 0x7f02640b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263400640 1 -- 172.21.15.19:0/2152913499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02640c7590 con 0x7f02640b6a40 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263400640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02640b6a40 0x7f02640b2930 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f025c00cb80 tx=0x7f025c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f025aa00640 1 -- 172.21.15.19:0/2152913499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c015070 con 0x7f02640b6a40 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f025aa00640 1 -- 172.21.15.19:0/2152913499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c00abd0 con 0x7f02640b6a40 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02640c7870 con 0x7f02640b6a40 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f025aa00640 1 -- 172.21.15.19:0/2152913499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c00d040 con 0x7f02640b6a40 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.179+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02640c7e30 con 0x7f02640b6a40 2024-04-25T11:35:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.180+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0264076c60 con 0x7f02640b6a40 2024-04-25T11:35:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.182+0000 7f025aa00640 1 -- 172.21.15.19:0/2152913499 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f025c018070 con 0x7f02640b6a40 2024-04-25T11:35:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.183+0000 7f025aa00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0240074500 0x7f0240076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:00.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.183+0000 7f025aa00640 1 -- 172.21.15.19:0/2152913499 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5881+0+0 (secure 0 0 0) 0x7f025c094f70 con 0x7f02640b6a40 2024-04-25T11:35:00.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.183+0000 7f0262a00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0240074500 0x7f0240076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.185+0000 7f0262a00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0240074500 0x7f0240076a10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0254006fd0 tx=0x7f0254008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.187+0000 7f025aa00640 1 -- 172.21.15.19:0/2152913499 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f025c05f630 con 0x7f02640b6a40 2024-04-25T11:35:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:00.392+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f02640b4e90 con 0x7f02640b6a40 2024-04-25T11:35:01.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.256+0000 7f025aa00640 1 -- 172.21.15.19:0/2152913499 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f025c05ef30 con 0x7f02640b6a40 2024-04-25T11:35:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:35:01.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0240074500 msgr2=0x7f0240076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0240074500 0x7f0240076a10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0254006fd0 tx=0x7f0254008040 comp rx=0 tx=0).stop 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02640b6a40 msgr2=0x7f02640b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02640b6a40 0x7f02640b2930 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f025c00cb80 tx=0x7f025c00a4c0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 shutdown_connections 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0240074500 0x7f0240076a10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f026406f370 0x7f02640b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02640b6a40 0x7f02640b2930 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 --2- 172.21.15.19:0/2152913499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f026406cf40 0x7f02640cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 >> 172.21.15.19:0/2152913499 conn(0x7f02640ce3b0 msgr2=0x7f02640b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.260+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 shutdown_connections 2024-04-25T11:35:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.261+0000 7f0263e00640 1 -- 172.21.15.19:0/2152913499 wait complete. 2024-04-25T11:35:01.287 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:35:01.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.581+0000 7f554be00640 1 Processor -- start 2024-04-25T11:35:01.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.581+0000 7f554be00640 1 -- start start 2024-04-25T11:35:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.583+0000 7f554be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f554c15ee60 0x7f554c168e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.583+0000 7f554be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c070000 con 0x7f554c144340 2024-04-25T11:35:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.583+0000 7f554be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c06f240 con 0x7f554c06dce0 2024-04-25T11:35:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.583+0000 7f554be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f554c15f270 con 0x7f554c15ee60 2024-04-25T11:35:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.584+0000 7f554aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f554c144340 0x7f554c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34864/0 (socket says 172.21.15.19:34864) 2024-04-25T11:35:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.584+0000 7f554aa00640 1 -- 172.21.15.19:0/139273298 learned_addr learned my addr 172.21.15.19:0/139273298 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.584+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897597358 0 0) 0x7f554c070000 con 0x7f554c144340 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.584+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5530003520 con 0x7f554c144340 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.584+0000 7f554b400640 1 --2- 172.21.15.19:0/139273298 >> v2:172.21.15.169:3300/0 conn(0x7f554c15ee60 0x7f554c168e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1960819838 0 0) 0x7f5530003520 con 0x7f554c144340 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 >> v2:172.21.15.169:3300/0 conn(0x7f554c15ee60 msgr2=0x7f554c168e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 --2- 172.21.15.19:0/139273298 >> v2:172.21.15.169:3300/0 conn(0x7f554c15ee60 0x7f554c168e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 >> v1:172.21.15.169:6789/0 conn(0x7f554c06dce0 legacy=0x7f554c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c06f520 con 0x7f554c144340 2024-04-25T11:35:01.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f553c001670 con 0x7f554c144340 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 >> v1:172.21.15.19:6789/0 conn(0x7f554c144340 legacy=0x7f554c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5530003a00 0x7f5530003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f5530004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5530005270 0x7f553000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f553000c160 con 0x7f5530004800 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f553000c310 con 0x7f5530003a00 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f553000c510 con 0x7f5530005270 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.585+0000 7f554a000640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f5530004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554aa00640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5530003a00 0x7f5530003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554b400640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5530005270 0x7f553000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554a000640 1 -- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5530005270 msgr2=0x7f553000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554a000640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5530005270 0x7f553000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554a000640 1 -- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5530003a00 msgr2=0x7f5530003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:01.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554a000640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5530003a00 0x7f5530003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554a000640 1 -- 172.21.15.19:0/139273298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5530003520 con 0x7f5530004800 2024-04-25T11:35:01.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.586+0000 7f554b400640 1 -- 172.21.15.19:0/139273298 reap_dead start 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.587+0000 7f554a000640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f5530004c10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5538001720 tx=0x7f5530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cbe49042e66bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.587+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538001370 con 0x7f5530004800 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.587+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55380052d0 con 0x7f5530004800 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.587+0000 7f5549600640 1 -- 172.21.15.19:0/139273298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553802ab20 con 0x7f5530004800 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.588+0000 7f554be00640 1 -- 172.21.15.19:0/139273298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 msgr2=0x7f5530004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.588+0000 7f554be00640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f5530004c10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5538001720 tx=0x7f5530005830 comp rx=0 tx=0).stop 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.588+0000 7f554be00640 1 -- 172.21.15.19:0/139273298 shutdown_connections 2024-04-25T11:35:01.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.588+0000 7f554be00640 1 --2- 172.21.15.19:0/139273298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f5530004c10 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.588+0000 7f554be00640 1 -- 172.21.15.19:0/139273298 >> 172.21.15.19:0/139273298 conn(0x7f554c072f40 msgr2=0x7f554c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:01.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.588+0000 7f554be00640 1 -- 172.21.15.19:0/139273298 shutdown_connections 2024-04-25T11:35:01.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.588+0000 7f554be00640 1 -- 172.21.15.19:0/139273298 wait complete. 2024-04-25T11:35:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.591+0000 7f554be00640 1 Processor -- start 2024-04-25T11:35:01.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.591+0000 7f554be00640 1 -- start start 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f554c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f554c1448d0 0x7f554c142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f554c144ce0 0x7f554c142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f554c06fae0 con 0x7f5530004800 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f554c144750 con 0x7f554c1448d0 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f554c15e750 con 0x7f554c144ce0 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f554c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f554c15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39188/0 (socket says 172.21.15.19:39188) 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554aa00640 1 -- 172.21.15.19:0/1401021890 learned_addr learned my addr 172.21.15.19:0/1401021890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554a000640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f554c1448d0 0x7f554c142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.593+0000 7f554b400640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f554c144ce0 0x7f554c142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f554aa00640 1 -- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f554c144ce0 msgr2=0x7f554c142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f554aa00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f554c144ce0 0x7f554c142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f554aa00640 1 -- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f554c1448d0 msgr2=0x7f554c142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f554aa00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f554c1448d0 0x7f554c142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f554aa00640 1 -- 172.21.15.19:0/1401021890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c15a960 con 0x7f5530004800 2024-04-25T11:35:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f554aa00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f554c15a420 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f553c00e780 tx=0x7f553c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f5542a00640 1 -- 172.21.15.19:0/1401021890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c00c660 con 0x7f5530004800 2024-04-25T11:35:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f5542a00640 1 -- 172.21.15.19:0/1401021890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c018070 con 0x7f5530004800 2024-04-25T11:35:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c146f60 con 0x7f5530004800 2024-04-25T11:35:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.594+0000 7f5542a00640 1 -- 172.21.15.19:0/1401021890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c00ca60 con 0x7f5530004800 2024-04-25T11:35:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.595+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c1476e0 con 0x7f5530004800 2024-04-25T11:35:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.596+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c076c60 con 0x7f5530004800 2024-04-25T11:35:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.597+0000 7f5542a00640 1 -- 172.21.15.19:0/1401021890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f553c00cc00 con 0x7f5530004800 2024-04-25T11:35:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.598+0000 7f5542a00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f552c0744b0 0x7f552c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.598+0000 7f5542a00640 1 -- 172.21.15.19:0/1401021890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5443+0+0 (secure 0 0 0) 0x7f553c094ea0 con 0x7f5530004800 2024-04-25T11:35:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.599+0000 7f554a000640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f552c0744b0 0x7f552c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.600+0000 7f554a000640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f552c0744b0 0x7f552c0769c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5538032000 tx=0x7f553802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.603+0000 7f5542a00640 1 -- 172.21.15.19:0/1401021890 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f553c001a60 con 0x7f5530004800 2024-04-25T11:35:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:01.807+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f554c0b7920 con 0x7f5530004800 2024-04-25T11:35:02.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.260+0000 7f5542a00640 1 -- 172.21.15.19:0/1401021890 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f553c05f720 con 0x7f5530004800 2024-04-25T11:35:02.250 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:35:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f552c0744b0 msgr2=0x7f552c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f552c0744b0 0x7f552c0769c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5538032000 tx=0x7f553802e000 comp rx=0 tx=0).stop 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 msgr2=0x7f554c15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f554c15a420 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f553c00e780 tx=0x7f553c00ec40 comp rx=0 tx=0).stop 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 shutdown_connections 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f552c0744b0 0x7f552c0769c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f554c1448d0 0x7f554c142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f554c144ce0 0x7f554c142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 --2- 172.21.15.19:0/1401021890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5530004800 0x7f554c15a420 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.265+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 >> 172.21.15.19:0/1401021890 conn(0x7f554c072f40 msgr2=0x7f554c15b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.267+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 shutdown_connections 2024-04-25T11:35:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.268+0000 7f554be00640 1 -- 172.21.15.19:0/1401021890 wait complete. 2024-04-25T11:35:02.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-04-25T11:35:02.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.584+0000 7fada3e00640 1 Processor -- start 2024-04-25T11:35:02.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.584+0000 7fada3e00640 1 -- start start 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada40d56c0 0x7fada40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fada4160440 0x7fada406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fada406e0a0 0x7fada40d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fada40d43f0 con 0x7fada406e0a0 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fada406e4b0 con 0x7fada4160440 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fada40d37e0 con 0x7fada40d56c0 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fada406e0a0 0x7fada40d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fada406e0a0 0x7fada40d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39204/0 (socket says 172.21.15.19:39204) 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.586+0000 7fada3400640 1 -- 172.21.15.19:0/3013939541 learned_addr learned my addr 172.21.15.19:0/3013939541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:02.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.587+0000 7fada2000640 1 --2- 172.21.15.19:0/3013939541 >> v2:172.21.15.169:3300/0 conn(0x7fada4160440 0x7fada406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.587+0000 7fada2a00640 1 --2- 172.21.15.19:0/3013939541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada40d56c0 0x7fada40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.587+0000 7fada3400640 1 -- 172.21.15.19:0/3013939541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada40d56c0 msgr2=0x7fada40b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.587+0000 7fada3400640 1 --2- 172.21.15.19:0/3013939541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada40d56c0 0x7fada40b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.587+0000 7fada3400640 1 -- 172.21.15.19:0/3013939541 >> v2:172.21.15.169:3300/0 conn(0x7fada4160440 msgr2=0x7fada406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.587+0000 7fada3400640 1 --2- 172.21.15.19:0/3013939541 >> v2:172.21.15.169:3300/0 conn(0x7fada4160440 0x7fada406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.587+0000 7fada3400640 1 -- 172.21.15.19:0/3013939541 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada40d3ac0 con 0x7fada406e0a0 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.588+0000 7fada3400640 1 --2- 172.21.15.19:0/3013939541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada406e0a0 0x7fada40d0900 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fada40d3680 tx=0x7fad9402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42b2056a6066280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:02.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.588+0000 7fada1600640 1 -- 172.21.15.19:0/3013939541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94036070 con 0x7fada406e0a0 2024-04-25T11:35:02.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.588+0000 7fada1600640 1 -- 172.21.15.19:0/3013939541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad94031600 con 0x7fada406e0a0 2024-04-25T11:35:02.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.588+0000 7fada1600640 1 -- 172.21.15.19:0/3013939541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad940319d0 con 0x7fada406e0a0 2024-04-25T11:35:02.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 -- 172.21.15.19:0/3013939541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada406e0a0 msgr2=0x7fada40d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 --2- 172.21.15.19:0/3013939541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada406e0a0 0x7fada40d0900 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fada40d3680 tx=0x7fad9402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 -- 172.21.15.19:0/3013939541 shutdown_connections 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 --2- 172.21.15.19:0/3013939541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada406e0a0 0x7fada40d0900 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 --2- 172.21.15.19:0/3013939541 >> v2:172.21.15.169:3300/0 conn(0x7fada4160440 0x7fada406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 --2- 172.21.15.19:0/3013939541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada40d56c0 0x7fada40b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 -- 172.21.15.19:0/3013939541 >> 172.21.15.19:0/3013939541 conn(0x7fada40d2c20 msgr2=0x7fada40d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 -- 172.21.15.19:0/3013939541 shutdown_connections 2024-04-25T11:35:02.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.589+0000 7fada3e00640 1 -- 172.21.15.19:0/3013939541 wait complete. 2024-04-25T11:35:02.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.592+0000 7fada3e00640 1 Processor -- start 2024-04-25T11:35:02.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.592+0000 7fada3e00640 1 -- start start 2024-04-25T11:35:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.593+0000 7fada3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada406e0a0 0x7fada414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.593+0000 7fada3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4160440 0x7fada414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.593+0000 7fada3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada41630c0 0x7fada40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.593+0000 7fada3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fada40d4880 con 0x7fada4160440 2024-04-25T11:35:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.593+0000 7fada3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fada40d4080 con 0x7fada406e0a0 2024-04-25T11:35:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.593+0000 7fada3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fada40d4a00 con 0x7fada41630c0 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.593+0000 7fada2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4160440 0x7fada414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4160440 0x7fada414aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39212/0 (socket says 172.21.15.19:39212) 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 -- 172.21.15.19:0/1999810369 learned_addr learned my addr 172.21.15.19:0/1999810369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2a00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada406e0a0 0x7fada414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 -- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada41630c0 msgr2=0x7fada40c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada41630c0 0x7fada40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 -- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada406e0a0 msgr2=0x7fada414b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada406e0a0 0x7fada414b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 -- 172.21.15.19:0/1999810369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada414bef0 con 0x7fada4160440 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2a00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada406e0a0 0x7fada414b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.594+0000 7fada2000640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4160440 0x7fada414aed0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d620 tx=0x7fad8c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.595+0000 7fad9aa00640 1 -- 172.21.15.19:0/1999810369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00b620 con 0x7fada4160440 2024-04-25T11:35:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.595+0000 7fad9aa00640 1 -- 172.21.15.19:0/1999810369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c019070 con 0x7fada4160440 2024-04-25T11:35:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.595+0000 7fad9aa00640 1 -- 172.21.15.19:0/1999810369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00ba00 con 0x7fada4160440 2024-04-25T11:35:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.595+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada4162020 con 0x7fada4160440 2024-04-25T11:35:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.595+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada41625e0 con 0x7fada4160440 2024-04-25T11:35:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.596+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada4069b30 con 0x7fada4160440 2024-04-25T11:35:02.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.597+0000 7fad9aa00640 1 -- 172.21.15.19:0/1999810369 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad8c00bba0 con 0x7fada4160440 2024-04-25T11:35:02.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.598+0000 7fad9aa00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad80074500 0x7fad80076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.598+0000 7fad9aa00640 1 -- 172.21.15.19:0/1999810369 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7fad8c0952c0 con 0x7fada4160440 2024-04-25T11:35:02.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.599+0000 7fada2a00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad80074500 0x7fad80076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.600+0000 7fada2a00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad80074500 0x7fad80076a10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fada40b8b30 tx=0x7fad9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:02.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.603+0000 7fad9aa00640 1 -- 172.21.15.19:0/1999810369 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fada4069b30 con 0x7fada4160440 2024-04-25T11:35:02.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.807+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7fada4076c60 con 0x7fada4160440 2024-04-25T11:35:02.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.808+0000 7fad9aa00640 1 -- 172.21.15.19:0/1999810369 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fad8c05fdf0 con 0x7fada4160440 2024-04-25T11:35:02.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad80074500 msgr2=0x7fad80076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad80074500 0x7fad80076a10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fada40b8b30 tx=0x7fad9c008040 comp rx=0 tx=0).stop 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4160440 msgr2=0x7fada414aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4160440 0x7fada414aed0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d620 tx=0x7fad8c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 shutdown_connections 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad80074500 0x7fad80076a10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fada41630c0 0x7fada40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fada4160440 0x7fada414aed0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 --2- 172.21.15.19:0/1999810369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fada406e0a0 0x7fada414b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 >> 172.21.15.19:0/1999810369 conn(0x7fada40d2c20 msgr2=0x7fada415f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 shutdown_connections 2024-04-25T11:35:02.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:02.810+0000 7fada3e00640 1 -- 172.21.15.19:0/1999810369 wait complete. 2024-04-25T11:35:02.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:35:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.117+0000 7f36bd000640 1 Processor -- start 2024-04-25T11:35:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.117+0000 7f36bd000640 1 -- start start 2024-04-25T11:35:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.118+0000 7f36bd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 0x7f36b80b7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36bd000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f36b806ccd0 0x7f36b80b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36bd000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36b80b7e70 con 0x7f36b80b7a60 2024-04-25T11:35:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36bd000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f36b80b9a70 con 0x7f36b806ccd0 2024-04-25T11:35:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36bd000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f36b806d260 con 0x7f36b8160260 2024-04-25T11:35:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36afe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f36b80b7a60 0x7f36b806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34876/0 (socket says 172.21.15.19:34876) 2024-04-25T11:35:03.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36afe00640 1 -- 172.21.15.19:0/142233293 learned_addr learned my addr 172.21.15.19:0/142233293 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36b7400640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 0x7f36b80b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.119+0000 7f36b7e00640 1 --2- 172.21.15.19:0/142233293 >> v2:172.21.15.169:3300/0 conn(0x7f36b806ccd0 0x7f36b80b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2501082360 0 0) 0x7f36a0001110 con 0x7f36b80b7a60 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f369c003520 con 0x7f36b80b7a60 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3923276976 0 0) 0x7f369c003520 con 0x7f36b80b7a60 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 msgr2=0x7f36b80b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 0x7f36b80b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 >> v2:172.21.15.169:3300/0 conn(0x7f36b806ccd0 msgr2=0x7f36b80b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 --2- 172.21.15.19:0/142233293 >> v2:172.21.15.169:3300/0 conn(0x7f36b806ccd0 0x7f36b80b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b80b9420 con 0x7f36b80b7a60 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f36a0001840 con 0x7f36b80b7a60 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 >> v1:172.21.15.19:6789/0 conn(0x7f36b80b7a60 legacy=0x7f36b806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 0x7f369c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c003f80 0x7f369c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f369c004d40 0x7f369c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36b80b9a70 con 0x7f369c004d40 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36b806d260 con 0x7f369c003f80 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.120+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f369c00bb20 con 0x7f36b8160260 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36b7e00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f369c004d40 0x7f369c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36afe00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c003f80 0x7f369c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36b7400640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 0x7f369c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36b7e00640 1 -- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 msgr2=0x7f369c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36b7e00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8160260 0x7f369c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36b7e00640 1 -- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c003f80 msgr2=0x7f369c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36b7e00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c003f80 0x7f369c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.121+0000 7f36b7e00640 1 -- 172.21.15.19:0/142233293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c003520 con 0x7f369c004d40 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.122+0000 7f36b7e00640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f369c004d40 0x7f369c005150 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f36b808d210 tx=0x7f369c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78d1393bfcf2e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.123+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0037070 con 0x7f369c004d40 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.123+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b002e050 con 0x7f369c004d40 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.123+0000 7f36b6a00640 1 -- 172.21.15.19:0/142233293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0009c10 con 0x7f369c004d40 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.123+0000 7f36bd000640 1 -- 172.21.15.19:0/142233293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f369c004d40 msgr2=0x7f369c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.123+0000 7f36bd000640 1 --2- 172.21.15.19:0/142233293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f369c004d40 0x7f369c005150 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f36b808d210 tx=0x7f369c005830 comp rx=0 tx=0).stop 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.123+0000 7f36b7e00640 1 -- 172.21.15.19:0/142233293 reap_dead start 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.124+0000 7f36bd000640 1 -- 172.21.15.19:0/142233293 shutdown_connections 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.124+0000 7f36bd000640 1 -- 172.21.15.19:0/142233293 >> 172.21.15.19:0/142233293 conn(0x7f36b80d2520 msgr2=0x7f36b80d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.124+0000 7f36bd000640 1 -- 172.21.15.19:0/142233293 shutdown_connections 2024-04-25T11:35:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.124+0000 7f36bd000640 1 -- 172.21.15.19:0/142233293 wait complete. 2024-04-25T11:35:03.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.126+0000 7f36bd000640 1 Processor -- start 2024-04-25T11:35:03.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.126+0000 7f36bd000640 1 -- start start 2024-04-25T11:35:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36bd000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c004d40 0x7f36b8161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36bd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8153010 0x7f36b80d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36bd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b80d0f80 0x7f36b8160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36bd000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36b80b7520 con 0x7f36b80d0f80 2024-04-25T11:35:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36bd000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f36b80b90f0 con 0x7f369c004d40 2024-04-25T11:35:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36bd000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f36b80b76a0 con 0x7f36b8153010 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b80d0f80 0x7f36b8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b80d0f80 0x7f36b8160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39234/0 (socket says 172.21.15.19:39234) 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.128+0000 7f36b7e00640 1 -- 172.21.15.19:0/3448109382 learned_addr learned my addr 172.21.15.19:0/3448109382 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7400640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c004d40 0x7f36b8161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36afe00640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8153010 0x7f36b80d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7e00640 1 -- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8153010 msgr2=0x7f36b80d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7e00640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8153010 0x7f36b80d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7e00640 1 -- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c004d40 msgr2=0x7f36b8161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7e00640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c004d40 0x7f36b8161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7e00640 1 -- 172.21.15.19:0/3448109382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b80d1390 con 0x7f36b80d0f80 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7400640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c004d40 0x7f36b8161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.129+0000 7f36b7e00640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b80d0f80 0x7f36b8160670 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f36b0035040 tx=0x7f36b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.130+0000 7f36af400640 1 -- 172.21.15.19:0/3448109382 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0037040 con 0x7f36b80d0f80 2024-04-25T11:35:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.130+0000 7f36af400640 1 -- 172.21.15.19:0/3448109382 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b0044070 con 0x7f36b80d0f80 2024-04-25T11:35:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.130+0000 7f36af400640 1 -- 172.21.15.19:0/3448109382 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0009c10 con 0x7f36b80d0f80 2024-04-25T11:35:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.130+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b8161c70 con 0x7f36b80d0f80 2024-04-25T11:35:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.130+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b8156590 con 0x7f36b80d0f80 2024-04-25T11:35:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.131+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b8076c60 con 0x7f36b80d0f80 2024-04-25T11:35:03.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.132+0000 7f36af400640 1 -- 172.21.15.19:0/3448109382 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f36b003a070 con 0x7f36b80d0f80 2024-04-25T11:35:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.133+0000 7f36af400640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36940744b0 0x7f36940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.133+0000 7f36af400640 1 -- 172.21.15.19:0/3448109382 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7f36b00b55b0 con 0x7f36b80d0f80 2024-04-25T11:35:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.134+0000 7f36b7400640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36940744b0 0x7f36940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.137+0000 7f36b7400640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36940744b0 0x7f36940769c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f36a80099e0 tx=0x7f36a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.139+0000 7f36af400640 1 -- 172.21.15.19:0/3448109382 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36b80d1390 con 0x7f36b80d0f80 2024-04-25T11:35:03.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.343+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f36b80d2e20 con 0x7f36b80d0f80 2024-04-25T11:35:03.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.351+0000 7f36af400640 1 -- 172.21.15.19:0/3448109382 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f36b007ff40 con 0x7f36b80d0f80 2024-04-25T11:35:03.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36940744b0 msgr2=0x7f36940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36940744b0 0x7f36940769c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f36a80099e0 tx=0x7f36a8008040 comp rx=0 tx=0).stop 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b80d0f80 msgr2=0x7f36b8160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b80d0f80 0x7f36b8160670 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f36b0035040 tx=0x7f36b0009750 comp rx=0 tx=0).stop 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 shutdown_connections 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f36940744b0 0x7f36940769c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36b80d0f80 0x7f36b8160670 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f36b8153010 0x7f36b80d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 --2- 172.21.15.19:0/3448109382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f369c004d40 0x7f36b8161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 >> 172.21.15.19:0/3448109382 conn(0x7f36b80d2520 msgr2=0x7f36b80d56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 shutdown_connections 2024-04-25T11:35:03.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.354+0000 7f36bd000640 1 -- 172.21.15.19:0/3448109382 wait complete. 2024-04-25T11:35:03.369 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-04-25T11:35:03.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.669+0000 7fbd53200640 1 Processor -- start 2024-04-25T11:35:03.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.669+0000 7fbd53200640 1 -- start start 2024-04-25T11:35:03.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.671+0000 7fbd53200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd540d1c50 0x7fbd540ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.671+0000 7fbd53200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5415ecb0 con 0x7fbd540cf830 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.671+0000 7fbd53200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5415ef90 con 0x7fbd5415e6d0 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.671+0000 7fbd53200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbd540cec00 con 0x7fbd540d1c50 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.671+0000 7fbd52800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd540d1c50 0x7fbd540ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.672+0000 7fbd52800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd540d1c50 0x7fbd540ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39244/0 (socket says 172.21.15.19:39244) 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.672+0000 7fbd52800640 1 -- 172.21.15.19:0/2353832789 learned_addr learned my addr 172.21.15.19:0/2353832789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.672+0000 7fbd52800640 1 -- 172.21.15.19:0/2353832789 >> v1:172.21.15.169:6789/0 conn(0x7fbd5415e6d0 legacy=0x7fbd540d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:03.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.672+0000 7fbd52800640 1 -- 172.21.15.19:0/2353832789 >> v1:172.21.15.19:6789/0 conn(0x7fbd540cf830 legacy=0x7fbd540b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.672+0000 7fbd52800640 1 -- 172.21.15.19:0/2353832789 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5415f270 con 0x7fbd540d1c50 2024-04-25T11:35:03.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.673+0000 7fbd52800640 1 --2- 172.21.15.19:0/2353832789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540d1c50 0x7fbd540ce6c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009e50 tx=0x7fbd4c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332faa82b008c906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:03.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.674+0000 7fbd51400640 1 -- 172.21.15.19:0/2353832789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c036070 con 0x7fbd540d1c50 2024-04-25T11:35:03.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.674+0000 7fbd51400640 1 -- 172.21.15.19:0/2353832789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4c031600 con 0x7fbd540d1c50 2024-04-25T11:35:03.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.674+0000 7fbd51400640 1 -- 172.21.15.19:0/2353832789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0319d0 con 0x7fbd540d1c50 2024-04-25T11:35:03.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.674+0000 7fbd53200640 1 -- 172.21.15.19:0/2353832789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540d1c50 msgr2=0x7fbd540ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.674+0000 7fbd53200640 1 --2- 172.21.15.19:0/2353832789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540d1c50 0x7fbd540ce6c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009e50 tx=0x7fbd4c02a760 comp rx=0 tx=0).stop 2024-04-25T11:35:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.675+0000 7fbd53200640 1 -- 172.21.15.19:0/2353832789 shutdown_connections 2024-04-25T11:35:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.675+0000 7fbd53200640 1 --2- 172.21.15.19:0/2353832789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540d1c50 0x7fbd540ce6c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.675+0000 7fbd53200640 1 -- 172.21.15.19:0/2353832789 >> 172.21.15.19:0/2353832789 conn(0x7fbd540b5460 msgr2=0x7fbd540b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.675+0000 7fbd53200640 1 -- 172.21.15.19:0/2353832789 shutdown_connections 2024-04-25T11:35:03.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.675+0000 7fbd53200640 1 -- 172.21.15.19:0/2353832789 wait complete. 2024-04-25T11:35:03.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.677+0000 7fbd53200640 1 Processor -- start 2024-04-25T11:35:03.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.677+0000 7fbd53200640 1 -- start start 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd53200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540cf830 0x7fbd540b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd53200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd5415e6d0 0x7fbd5415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd53200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd540b39e0 0x7fbd540b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd53200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5408d280 con 0x7fbd540cf830 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd53200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5408cd60 con 0x7fbd540b39e0 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd53200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5408d400 con 0x7fbd5415e6d0 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd51e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540cf830 0x7fbd540b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd51e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540cf830 0x7fbd540b74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39254/0 (socket says 172.21.15.19:39254) 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd51e00640 1 -- 172.21.15.19:0/3146647260 learned_addr learned my addr 172.21.15.19:0/3146647260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:03.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd52800640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd540b39e0 0x7fbd540b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.679+0000 7fbd49400640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd5415e6d0 0x7fbd5415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd51e00640 1 -- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd5415e6d0 msgr2=0x7fbd5415cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd51e00640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd5415e6d0 0x7fbd5415cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd51e00640 1 -- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd540b39e0 msgr2=0x7fbd540b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd51e00640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd540b39e0 0x7fbd540b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd51e00640 1 -- 172.21.15.19:0/3146647260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd540b68c0 con 0x7fbd540cf830 2024-04-25T11:35:03.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd51e00640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540cf830 0x7fbd540b74b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c3f0 tx=0x7fbd4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd4aa00640 1 -- 172.21.15.19:0/3146647260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44017070 con 0x7fbd540cf830 2024-04-25T11:35:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd4aa00640 1 -- 172.21.15.19:0/3146647260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd440035f0 con 0x7fbd540cf830 2024-04-25T11:35:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd4aa00640 1 -- 172.21.15.19:0/3146647260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440039e0 con 0x7fbd540cf830 2024-04-25T11:35:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.680+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd540b4330 con 0x7fbd540cf830 2024-04-25T11:35:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.681+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd540b82b0 con 0x7fbd540cf830 2024-04-25T11:35:03.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.682+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd54050740 con 0x7fbd540cf830 2024-04-25T11:35:03.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.683+0000 7fbd4aa00640 1 -- 172.21.15.19:0/3146647260 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd44003b80 con 0x7fbd540cf830 2024-04-25T11:35:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.684+0000 7fbd4aa00640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd300744b0 0x7fbd300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.684+0000 7fbd4aa00640 1 -- 172.21.15.19:0/3146647260 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7fbd44094250 con 0x7fbd540cf830 2024-04-25T11:35:03.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.685+0000 7fbd49400640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd300744b0 0x7fbd300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:03.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.686+0000 7fbd49400640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd300744b0 0x7fbd300769c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006fd0 tx=0x7fbd3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:03.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.690+0000 7fbd4aa00640 1 -- 172.21.15.19:0/3146647260 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd540b4330 con 0x7fbd540cf830 2024-04-25T11:35:03.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:03.966+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7fbd54161960 con 0x7fbd540cf830 2024-04-25T11:35:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.291+0000 7fbd4aa00640 1 -- 172.21.15.19:0/3146647260 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v388) ==== 141+0+0 (secure 0 0 0) 0x7fbd540b68c0 con 0x7fbd540cf830 2024-04-25T11:35:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd300744b0 msgr2=0x7fbd300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd300744b0 0x7fbd300769c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006fd0 tx=0x7fbd3c008040 comp rx=0 tx=0).stop 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540cf830 msgr2=0x7fbd540b74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540cf830 0x7fbd540b74b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c3f0 tx=0x7fbd4400c8b0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 shutdown_connections 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbd300744b0 0x7fbd300769c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd540b39e0 0x7fbd540b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd5415e6d0 0x7fbd5415cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 --2- 172.21.15.19:0/3146647260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd540cf830 0x7fbd540b74b0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 >> 172.21.15.19:0/3146647260 conn(0x7fbd540b5460 msgr2=0x7fbd540d2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 shutdown_connections 2024-04-25T11:35:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.293+0000 7fbd53200640 1 -- 172.21.15.19:0/3146647260 wait complete. 2024-04-25T11:35:04.313 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-04-25T11:35:04.313 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2024-04-25T11:35:04.313 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2024-04-25T11:35:04.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:35:04.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.613+0000 7fec02e00640 1 Processor -- start 2024-04-25T11:35:04.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.613+0000 7fec02e00640 1 -- start start 2024-04-25T11:35:04.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.616+0000 7fec02e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febfc0b3f80 0x7febfc0d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.616+0000 7fec02e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7febfc0d2240 0x7febfc0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.616+0000 7fec02e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febfc0b4390 con 0x7febfc160870 2024-04-25T11:35:04.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.616+0000 7fec02e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7febfc0d00f0 con 0x7febfc0b3f80 2024-04-25T11:35:04.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.616+0000 7fec02e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7febfc0b2470 con 0x7febfc0d2240 2024-04-25T11:35:04.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.616+0000 7febfb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febfc0b3f80 0x7febfc0d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.616+0000 7febfb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febfc0b3f80 0x7febfc0d1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50380/0 (socket says 172.21.15.19:50380) 2024-04-25T11:35:04.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.617+0000 7febfb400640 1 -- 172.21.15.19:0/1149378333 learned_addr learned my addr 172.21.15.19:0/1149378333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:04.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.617+0000 7fec00e00640 1 --2- 172.21.15.19:0/1149378333 >> v2:172.21.15.169:3300/0 conn(0x7febfc0d2240 0x7febfc0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.617+0000 7febfb400640 1 -- 172.21.15.19:0/1149378333 >> v2:172.21.15.169:3300/0 conn(0x7febfc0d2240 msgr2=0x7febfc0b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:04.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.617+0000 7febfb400640 1 --2- 172.21.15.19:0/1149378333 >> v2:172.21.15.169:3300/0 conn(0x7febfc0d2240 0x7febfc0b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.617+0000 7febfb400640 1 -- 172.21.15.19:0/1149378333 >> v1:172.21.15.169:6789/0 conn(0x7febfc160870 legacy=0x7febfc0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.617+0000 7febfb400640 1 -- 172.21.15.19:0/1149378333 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc0cfb30 con 0x7febfc0b3f80 2024-04-25T11:35:04.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.619+0000 7febfb400640 1 --2- 172.21.15.19:0/1149378333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0b3f80 0x7febfc0d1d00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7febfc0c64e0 tx=0x7febe402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58068988ca80321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.619+0000 7febfaa00640 1 -- 172.21.15.19:0/1149378333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4036070 con 0x7febfc0b3f80 2024-04-25T11:35:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.619+0000 7febfaa00640 1 -- 172.21.15.19:0/1149378333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4031600 con 0x7febfc0b3f80 2024-04-25T11:35:04.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.619+0000 7febfaa00640 1 -- 172.21.15.19:0/1149378333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe40319d0 con 0x7febfc0b3f80 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.620+0000 7fec02e00640 1 -- 172.21.15.19:0/1149378333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0b3f80 msgr2=0x7febfc0d1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.620+0000 7fec02e00640 1 --2- 172.21.15.19:0/1149378333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0b3f80 0x7febfc0d1d00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7febfc0c64e0 tx=0x7febe402a730 comp rx=0 tx=0).stop 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.620+0000 7fec02e00640 1 -- 172.21.15.19:0/1149378333 shutdown_connections 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.620+0000 7fec02e00640 1 --2- 172.21.15.19:0/1149378333 >> v2:172.21.15.169:3300/0 conn(0x7febfc0d2240 0x7febfc0b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.620+0000 7fec02e00640 1 --2- 172.21.15.19:0/1149378333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0b3f80 0x7febfc0d1d00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.620+0000 7fec02e00640 1 -- 172.21.15.19:0/1149378333 >> 172.21.15.19:0/1149378333 conn(0x7febfc09b7d0 msgr2=0x7febfc09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.621+0000 7fec02e00640 1 -- 172.21.15.19:0/1149378333 shutdown_connections 2024-04-25T11:35:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.621+0000 7fec02e00640 1 -- 172.21.15.19:0/1149378333 wait complete. 2024-04-25T11:35:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.623+0000 7fec02e00640 1 Processor -- start 2024-04-25T11:35:04.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.623+0000 7fec02e00640 1 -- start start 2024-04-25T11:35:04.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.624+0000 7fec02e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febfc0b3f80 0x7febfc1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.624+0000 7fec02e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0d2240 0x7febfc0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7fec02e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febfc160870 0x7febfc09c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7fec02e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febfc0d2650 con 0x7febfc0d2240 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7fec02e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febfc0b2100 con 0x7febfc0b3f80 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7fec02e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7febfc0af850 con 0x7febfc160870 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0d2240 0x7febfc0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0d2240 0x7febfc0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50388/0 (socket says 172.21.15.19:50388) 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 -- 172.21.15.19:0/207926370 learned_addr learned my addr 172.21.15.19:0/207926370 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7fec00e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febfc160870 0x7febfc09c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 -- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febfc160870 msgr2=0x7febfc09c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febfc160870 0x7febfc09c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfbe00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febfc0b3f80 0x7febfc1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 -- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febfc0b3f80 msgr2=0x7febfc1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febfc0b3f80 0x7febfc1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7febfb400640 1 -- 172.21.15.19:0/207926370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febfc0cb3a0 con 0x7febfc0d2240 2024-04-25T11:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.625+0000 7fec00e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febfc160870 0x7febfc09c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:04.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.626+0000 7febfb400640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0d2240 0x7febfc0cae60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7febe4034040 tx=0x7febe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.626+0000 7febf3e00640 1 -- 172.21.15.19:0/207926370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4036050 con 0x7febfc0d2240 2024-04-25T11:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.626+0000 7febf3e00640 1 -- 172.21.15.19:0/207926370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4042070 con 0x7febfc0d2240 2024-04-25T11:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.626+0000 7febf3e00640 1 -- 172.21.15.19:0/207926370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4009510 con 0x7febfc0d2240 2024-04-25T11:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.626+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febfc0c9dc0 con 0x7febfc0d2240 2024-04-25T11:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.626+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febfc0ca380 con 0x7febfc0d2240 2024-04-25T11:35:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.627+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febfc0c9b30 con 0x7febfc0d2240 2024-04-25T11:35:04.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.629+0000 7febf3e00640 1 -- 172.21.15.19:0/207926370 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febe4007c50 con 0x7febfc0d2240 2024-04-25T11:35:04.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.630+0000 7febf3e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd80744e0 0x7febd80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.631+0000 7febf3e00640 1 -- 172.21.15.19:0/207926370 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7febe40b6a10 con 0x7febfc0d2240 2024-04-25T11:35:04.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.631+0000 7febfbe00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd80744e0 0x7febd80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.632+0000 7febfbe00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd80744e0 0x7febd80769f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7febfc073a00 tx=0x7febec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:04.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.634+0000 7febf3e00640 1 -- 172.21.15.19:0/207926370 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febe40814e0 con 0x7febfc0d2240 2024-04-25T11:35:04.666 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:35:04.674 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:35:04.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.842+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febfc050740 con 0x7febfc0d2240 2024-04-25T11:35:04.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.844+0000 7febf3e00640 1 -- 172.21.15.19:0/207926370 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13151 (secure 0 0 0) 0x7febe4080de0 con 0x7febfc0d2240 2024-04-25T11:35:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:04.833 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:35:02.257309+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6834/2118499768":"2024-04-26T11:34:58.837713+0000","172.21.15.169:6836/2118499768":"2024-04-26T11:34:58.837713+0000","172.21.15.19:6835/853310115":"2024-04-26T11:34:43.091612+0000","172.21.15.19:6834/853310115":"2024-04-26T11:34:43.091612+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd80744e0 msgr2=0x7febd80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd80744e0 0x7febd80769f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7febfc073a00 tx=0x7febec009040 comp rx=0 tx=0).stop 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0d2240 msgr2=0x7febfc0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0d2240 0x7febfc0cae60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7febe4034040 tx=0x7febe4004a40 comp rx=0 tx=0).stop 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 shutdown_connections 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd80744e0 0x7febd80769f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febfc160870 0x7febfc09c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febfc0d2240 0x7febfc0cae60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 --2- 172.21.15.19:0/207926370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febfc0b3f80 0x7febfc1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 >> 172.21.15.19:0/207926370 conn(0x7febfc09b7d0 msgr2=0x7febfc071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.847+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 shutdown_connections 2024-04-25T11:35:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:04.848+0000 7fec02e00640 1 -- 172.21.15.19:0/207926370 wait complete. 2024-04-25T11:35:04.862 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:35:04.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:35:05.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.162+0000 7f8888800640 1 Processor -- start 2024-04-25T11:35:05.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.163+0000 7f8888800640 1 -- start start 2024-04-25T11:35:05.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8888800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f888006dce0 0x7f888015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8888800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 0x7f888015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8888800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88800cdea0 con 0x7f88800cd9a0 2024-04-25T11:35:05.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8888800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f88800ceab0 con 0x7f888006dce0 2024-04-25T11:35:05.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8888800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88800ce180 con 0x7f88800b0af0 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8885e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f88800cd9a0 0x7f888006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53498/0 (socket says 172.21.15.19:53498) 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8885e00640 1 -- 172.21.15.19:0/1199023862 learned_addr learned my addr 172.21.15.19:0/1199023862 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8886800640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 0x7f888015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.165+0000 7f8885400640 1 --2- 172.21.15.19:0/1199023862 >> v2:172.21.15.169:3300/0 conn(0x7f888006dce0 0x7f888015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998161777 0 0) 0x7f88800cdea0 con 0x7f88800cd9a0 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8868003520 con 0x7f88800cd9a0 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2097198530 0 0) 0x7f8868003520 con 0x7f88800cd9a0 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 msgr2=0x7f888015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:05.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 0x7f888015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 >> v2:172.21.15.169:3300/0 conn(0x7f888006dce0 msgr2=0x7f888015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 --2- 172.21.15.19:0/1199023862 >> v2:172.21.15.169:3300/0 conn(0x7f888006dce0 0x7f888015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88800ce460 con 0x7f88800cd9a0 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8874001670 con 0x7f88800cd9a0 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 >> v1:172.21.15.19:6789/0 conn(0x7f88800cd9a0 legacy=0x7f888006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 0x7f88680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8868003f30 0x7f8868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8868004d40 0x7f8868005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88800ceab0 con 0x7f8868004d40 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88800ce180 con 0x7f8868003f30 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.166+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f886800ba10 con 0x7f88800b0af0 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8886800640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8868004d40 0x7f8868005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8885e00640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 0x7f88680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8886800640 1 -- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 msgr2=0x7f88680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8886800640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 0x7f88680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8886800640 1 -- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8868003f30 msgr2=0x7f8868004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:05.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8886800640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8868003f30 0x7f8868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8886800640 1 -- 172.21.15.19:0/1199023862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8868003520 con 0x7f8868004d40 2024-04-25T11:35:05.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.167+0000 7f8885e00640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b0af0 0x7f88680039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.168+0000 7f8886800640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8868004d40 0x7f8868005150 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8880148040 tx=0x7f8868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79ba094362c388c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c037070 con 0x7f8868004d40 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887c032600 con 0x7f8868004d40 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8884a00640 1 -- 172.21.15.19:0/1199023862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c009c10 con 0x7f8868004d40 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8888800640 1 -- 172.21.15.19:0/1199023862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8868004d40 msgr2=0x7f8868005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8888800640 1 --2- 172.21.15.19:0/1199023862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8868004d40 0x7f8868005150 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8880148040 tx=0x7f8868005830 comp rx=0 tx=0).stop 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8886800640 1 -- 172.21.15.19:0/1199023862 reap_dead start 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8888800640 1 -- 172.21.15.19:0/1199023862 shutdown_connections 2024-04-25T11:35:05.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.169+0000 7f8888800640 1 -- 172.21.15.19:0/1199023862 >> 172.21.15.19:0/1199023862 conn(0x7f8880073090 msgr2=0x7f88800734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:05.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.170+0000 7f8888800640 1 -- 172.21.15.19:0/1199023862 shutdown_connections 2024-04-25T11:35:05.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.170+0000 7f8888800640 1 -- 172.21.15.19:0/1199023862 wait complete. 2024-04-25T11:35:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.172+0000 7f8888800640 1 Processor -- start 2024-04-25T11:35:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.172+0000 7f8888800640 1 -- start start 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8888800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b2140 0x7f88801f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8888800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88801573b0 0x7f888013fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8888800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8880070120 0x7f8880070530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8888800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88800cef40 con 0x7f88801573b0 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8888800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88800b0f00 con 0x7f8880070120 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8888800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f88800cf0c0 con 0x7f88800b2140 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8885400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88801573b0 0x7f888013fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8885400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88801573b0 0x7f888013fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50410/0 (socket says 172.21.15.19:50410) 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8885400640 1 -- 172.21.15.19:0/1595953732 learned_addr learned my addr 172.21.15.19:0/1595953732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.174+0000 7f8885e00640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b2140 0x7f88801f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8886800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8880070120 0x7f8880070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8885400640 1 -- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b2140 msgr2=0x7f88801f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8885400640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b2140 0x7f88801f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8885400640 1 -- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8880070120 msgr2=0x7f8880070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8885400640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8880070120 0x7f8880070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8885400640 1 -- 172.21.15.19:0/1595953732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8880070a70 con 0x7f88801573b0 2024-04-25T11:35:05.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8886800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8880070120 0x7f8880070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:05.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8885400640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88801573b0 0x7f888013fbd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f887000e7f0 tx=0x7f887000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:05.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f887a000640 1 -- 172.21.15.19:0/1595953732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870017070 con 0x7f88801573b0 2024-04-25T11:35:05.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f887a000640 1 -- 172.21.15.19:0/1595953732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887000c940 con 0x7f88801573b0 2024-04-25T11:35:05.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88801f6a00 con 0x7f88801573b0 2024-04-25T11:35:05.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.175+0000 7f887a000640 1 -- 172.21.15.19:0/1595953732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887000cd30 con 0x7f88801573b0 2024-04-25T11:35:05.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.176+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88800b2720 con 0x7f88801573b0 2024-04-25T11:35:05.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.177+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8880050740 con 0x7f88801573b0 2024-04-25T11:35:05.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.178+0000 7f887a000640 1 -- 172.21.15.19:0/1595953732 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f887001a070 con 0x7f88801573b0 2024-04-25T11:35:05.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.180+0000 7f887a000640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88640744b0 0x7f88640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.180+0000 7f887a000640 1 -- 172.21.15.19:0/1595953732 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7f8870094ef0 con 0x7f88801573b0 2024-04-25T11:35:05.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.180+0000 7f8885e00640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88640744b0 0x7f88640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.182+0000 7f8885e00640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88640744b0 0x7f88640769c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8880161740 tx=0x7f8874009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:05.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.185+0000 7f887a000640 1 -- 172.21.15.19:0/1595953732 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8880070a70 con 0x7f88801573b0 2024-04-25T11:35:05.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.392+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f888006ca20 con 0x7f88801573b0 2024-04-25T11:35:05.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.393+0000 7f887a000640 1 -- 172.21.15.19:0/1595953732 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f887005f910 con 0x7f88801573b0 2024-04-25T11:35:05.381 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:35:05.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88640744b0 msgr2=0x7f88640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88640744b0 0x7f88640769c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8880161740 tx=0x7f8874009040 comp rx=0 tx=0).stop 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88801573b0 msgr2=0x7f888013fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88801573b0 0x7f888013fbd0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f887000e7f0 tx=0x7f887000ecb0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 shutdown_connections 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88640744b0 0x7f88640769c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8880070120 0x7f8880070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88801573b0 0x7f888013fbd0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 --2- 172.21.15.19:0/1595953732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88800b2140 0x7f88801f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 >> 172.21.15.19:0/1595953732 conn(0x7f8880073090 msgr2=0x7f888015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 shutdown_connections 2024-04-25T11:35:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.396+0000 7f8888800640 1 -- 172.21.15.19:0/1595953732 wait complete. 2024-04-25T11:35:05.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-04-25T11:35:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.708+0000 7f012c000640 1 Processor -- start 2024-04-25T11:35:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.708+0000 7f012c000640 1 -- start start 2024-04-25T11:35:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f012c000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f012414ad60 0x7f0124070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f012c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01240b5e40 0x7f01240b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f012c000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01240b6790 con 0x7f01241681d0 2024-04-25T11:35:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f012c000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f01240b4af0 con 0x7f012414ad60 2024-04-25T11:35:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f012c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01240b4dd0 con 0x7f01240b5e40 2024-04-25T11:35:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f0129600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f012414ad60 0x7f0124070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f0129600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f012414ad60 0x7f0124070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50416/0 (socket says 172.21.15.19:50416) 2024-04-25T11:35:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.710+0000 7f0129600640 1 -- 172.21.15.19:0/430648607 learned_addr learned my addr 172.21.15.19:0/430648607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.711+0000 7f0128c00640 1 --2- 172.21.15.19:0/430648607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01240b5e40 0x7f01240b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.711+0000 7f0129600640 1 -- 172.21.15.19:0/430648607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01240b5e40 msgr2=0x7f01240b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.711+0000 7f0129600640 1 --2- 172.21.15.19:0/430648607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01240b5e40 0x7f01240b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.711+0000 7f0129600640 1 -- 172.21.15.19:0/430648607 >> v1:172.21.15.169:6789/0 conn(0x7f01241681d0 legacy=0x7f01241685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.711+0000 7f0129600640 1 -- 172.21.15.19:0/430648607 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01240b50b0 con 0x7f012414ad60 2024-04-25T11:35:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.712+0000 7f0129600640 1 --2- 172.21.15.19:0/430648607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012414ad60 0x7f0124070010 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0118009870 tx=0x7f011802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d07a8432f534892 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.713+0000 7f0123e00640 1 -- 172.21.15.19:0/430648607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118037070 con 0x7f012414ad60 2024-04-25T11:35:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.713+0000 7f0123e00640 1 -- 172.21.15.19:0/430648607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0118032440 con 0x7f012414ad60 2024-04-25T11:35:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.713+0000 7f0123e00640 1 -- 172.21.15.19:0/430648607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118032810 con 0x7f012414ad60 2024-04-25T11:35:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 -- 172.21.15.19:0/430648607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012414ad60 msgr2=0x7f0124070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 --2- 172.21.15.19:0/430648607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012414ad60 0x7f0124070010 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0118009870 tx=0x7f011802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 -- 172.21.15.19:0/430648607 shutdown_connections 2024-04-25T11:35:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 --2- 172.21.15.19:0/430648607 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01240b5e40 0x7f01240b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 --2- 172.21.15.19:0/430648607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f012414ad60 0x7f0124070010 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 -- 172.21.15.19:0/430648607 >> 172.21.15.19:0/430648607 conn(0x7f01240d1ae0 msgr2=0x7f01240d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 -- 172.21.15.19:0/430648607 shutdown_connections 2024-04-25T11:35:05.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.714+0000 7f012c000640 1 -- 172.21.15.19:0/430648607 wait complete. 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.716+0000 7f012c000640 1 Processor -- start 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.717+0000 7f012c000640 1 -- start start 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f012c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01240b5e40 0x7f01240c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f012c000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f012414ad60 0x7f01240c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f012c000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01241681d0 0x7f01240c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f012c000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0124070550 con 0x7f01240b5e40 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f012c000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01240b5670 con 0x7f01241681d0 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f012c000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01240706d0 con 0x7f012414ad60 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f0129600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01240b5e40 0x7f01240c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f0129600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01240b5e40 0x7f01240c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50428/0 (socket says 172.21.15.19:50428) 2024-04-25T11:35:05.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.718+0000 7f0129600640 1 -- 172.21.15.19:0/1861015195 learned_addr learned my addr 172.21.15.19:0/1861015195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.719+0000 7f0129600640 1 -- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f012414ad60 msgr2=0x7f01240c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.719+0000 7f0129600640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f012414ad60 0x7f01240c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.719+0000 7f0129600640 1 -- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01241681d0 msgr2=0x7f01240c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.719+0000 7f0129600640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01241681d0 0x7f01240c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.719+0000 7f0129600640 1 -- 172.21.15.19:0/1861015195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01240c6c10 con 0x7f01240b5e40 2024-04-25T11:35:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.719+0000 7f0129600640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01240b5e40 0x7f01240c7af0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f011802a730 tx=0x7f01180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.720+0000 7f0121600640 1 -- 172.21.15.19:0/1861015195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118041020 con 0x7f01240b5e40 2024-04-25T11:35:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.720+0000 7f0121600640 1 -- 172.21.15.19:0/1861015195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0118030740 con 0x7f01240b5e40 2024-04-25T11:35:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.720+0000 7f0121600640 1 -- 172.21.15.19:0/1861015195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118037040 con 0x7f01240b5e40 2024-04-25T11:35:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.720+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0124167220 con 0x7f01240b5e40 2024-04-25T11:35:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.720+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01240c72f0 con 0x7f01240b5e40 2024-04-25T11:35:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.721+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0124050740 con 0x7f01240b5e40 2024-04-25T11:35:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.722+0000 7f0121600640 1 -- 172.21.15.19:0/1861015195 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0118044020 con 0x7f01240b5e40 2024-04-25T11:35:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.723+0000 7f0121600640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01040744e0 0x7f01040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.723+0000 7f0121600640 1 -- 172.21.15.19:0/1861015195 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7f01180b5880 con 0x7f01240b5e40 2024-04-25T11:35:05.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.724+0000 7f0128c00640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01040744e0 0x7f01040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.725+0000 7f0128c00640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01040744e0 0x7f01040769f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0110006fd0 tx=0x7f0110008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:05.730+0000 7f0121600640 1 -- 172.21.15.19:0/1861015195 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f011804b030 con 0x7f01240b5e40 2024-04-25T11:35:06.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.016+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7f01240d1ef0 con 0x7f01240b5e40 2024-04-25T11:35:06.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.305+0000 7f0121600640 1 -- 172.21.15.19:0/1861015195 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v390) ==== 138+0+0 (secure 0 0 0) 0x7f01180802a0 con 0x7f01240b5e40 2024-04-25T11:35:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01040744e0 msgr2=0x7f01040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:06.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01040744e0 0x7f01040769f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0110006fd0 tx=0x7f0110008040 comp rx=0 tx=0).stop 2024-04-25T11:35:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01240b5e40 msgr2=0x7f01240c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01240b5e40 0x7f01240c7af0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f011802a730 tx=0x7f01180046c0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 shutdown_connections 2024-04-25T11:35:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01040744e0 0x7f01040769f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01241681d0 0x7f01240c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f012414ad60 0x7f01240c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 --2- 172.21.15.19:0/1861015195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01240b5e40 0x7f01240c7af0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 >> 172.21.15.19:0/1861015195 conn(0x7f01240d1ae0 msgr2=0x7f0124162800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 shutdown_connections 2024-04-25T11:35:06.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.309+0000 7f012c000640 1 -- 172.21.15.19:0/1861015195 wait complete. 2024-04-25T11:35:06.334 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:35:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.638+0000 7ffad8c00640 1 Processor -- start 2024-04-25T11:35:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.638+0000 7ffad8c00640 1 -- start start 2024-04-25T11:35:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad8c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffad40d4250 0x7ffad40bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad40b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad8c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffad40b7580 0x7ffad40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad8c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ffad40b9730 con 0x7ffad40d4250 2024-04-25T11:35:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad8c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ffad40b7ed0 con 0x7ffad40b7580 2024-04-25T11:35:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffad40d4900 con 0x7ffad40b4db0 2024-04-25T11:35:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffad40d4250 0x7ffad40bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffad40d4250 0x7ffad40bb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50434/0 (socket says 172.21.15.19:50434) 2024-04-25T11:35:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.640+0000 7ffad3400640 1 -- 172.21.15.19:0/407943962 learned_addr learned my addr 172.21.15.19:0/407943962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.641+0000 7ffad2a00640 1 --2- 172.21.15.19:0/407943962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad40b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.641+0000 7ffad3e00640 1 --2- 172.21.15.19:0/407943962 >> v2:172.21.15.169:3300/0 conn(0x7ffad40b7580 0x7ffad40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.641+0000 7ffad3400640 1 -- 172.21.15.19:0/407943962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 msgr2=0x7ffad40b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.641+0000 7ffad3400640 1 --2- 172.21.15.19:0/407943962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad40b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.641+0000 7ffad3400640 1 -- 172.21.15.19:0/407943962 >> v2:172.21.15.169:3300/0 conn(0x7ffad40b7580 msgr2=0x7ffad40b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.641+0000 7ffad3400640 1 --2- 172.21.15.19:0/407943962 >> v2:172.21.15.169:3300/0 conn(0x7ffad40b7580 0x7ffad40b7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.641+0000 7ffad3400640 1 -- 172.21.15.19:0/407943962 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad40d4be0 con 0x7ffad40d4250 2024-04-25T11:35:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.642+0000 7ffad3400640 1 --2- 172.21.15.19:0/407943962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40bb2e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ffad408ea70 tx=0x7ffabc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6288c210f6d8793b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.642+0000 7ffad2000640 1 -- 172.21.15.19:0/407943962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc037070 con 0x7ffad40d4250 2024-04-25T11:35:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.642+0000 7ffad2000640 1 -- 172.21.15.19:0/407943962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffabc032440 con 0x7ffad40d4250 2024-04-25T11:35:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.642+0000 7ffad2000640 1 -- 172.21.15.19:0/407943962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc032810 con 0x7ffad40d4250 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 -- 172.21.15.19:0/407943962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 msgr2=0x7ffad40bb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 --2- 172.21.15.19:0/407943962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40bb2e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ffad408ea70 tx=0x7ffabc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 -- 172.21.15.19:0/407943962 shutdown_connections 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 --2- 172.21.15.19:0/407943962 >> v2:172.21.15.169:3300/0 conn(0x7ffad40b7580 0x7ffad40b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 --2- 172.21.15.19:0/407943962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad40b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 --2- 172.21.15.19:0/407943962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40bb2e0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 -- 172.21.15.19:0/407943962 >> 172.21.15.19:0/407943962 conn(0x7ffad40d2760 msgr2=0x7ffad40d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 -- 172.21.15.19:0/407943962 shutdown_connections 2024-04-25T11:35:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.643+0000 7ffad8c00640 1 -- 172.21.15.19:0/407943962 wait complete. 2024-04-25T11:35:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.646+0000 7ffad8c00640 1 Processor -- start 2024-04-25T11:35:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.646+0000 7ffad8c00640 1 -- start start 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.647+0000 7ffad8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.647+0000 7ffad8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad40b7580 0x7ffad40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.647+0000 7ffad8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.647+0000 7ffad8c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffad40d5480 con 0x7ffad40d4250 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.647+0000 7ffad8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffad40d51a0 con 0x7ffad40b7580 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.647+0000 7ffad8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffad40d1700 con 0x7ffad40b4db0 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50442/0 (socket says 172.21.15.19:50442) 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 -- 172.21.15.19:0/2242145797 learned_addr learned my addr 172.21.15.19:0/2242145797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3400640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad2a00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad40b7580 0x7ffad40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 -- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 msgr2=0x7ffad415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad415a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 -- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad40b7580 msgr2=0x7ffad40c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad40b7580 0x7ffad40c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3e00640 1 -- 172.21.15.19:0/2242145797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad40ca090 con 0x7ffad40d4250 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.648+0000 7ffad3400640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.649+0000 7ffad3e00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40cbbb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00ccb0 tx=0x7ffacc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.649+0000 7ffacb400640 1 -- 172.21.15.19:0/2242145797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00ce60 con 0x7ffad40d4250 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.649+0000 7ffacb400640 1 -- 172.21.15.19:0/2242145797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffacc003990 con 0x7ffad40d4250 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.649+0000 7ffacb400640 1 -- 172.21.15.19:0/2242145797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc003da0 con 0x7ffad40d4250 2024-04-25T11:35:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.649+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad40ca370 con 0x7ffad40d4250 2024-04-25T11:35:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.649+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad40ca930 con 0x7ffad40d4250 2024-04-25T11:35:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.650+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad40c9e00 con 0x7ffad40d4250 2024-04-25T11:35:06.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.652+0000 7ffacb400640 1 -- 172.21.15.19:0/2242145797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffacc003f40 con 0x7ffad40d4250 2024-04-25T11:35:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.653+0000 7ffacb400640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffab00744b0 0x7ffab00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.653+0000 7ffacb400640 1 -- 172.21.15.19:0/2242145797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7ffacc0954f0 con 0x7ffad40d4250 2024-04-25T11:35:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.654+0000 7ffad3400640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffab00744b0 0x7ffab00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:06.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.655+0000 7ffad3400640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffab00744b0 0x7ffab00769c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffabc034000 tx=0x7ffabc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.657+0000 7ffacb400640 1 -- 172.21.15.19:0/2242145797 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffacc014030 con 0x7ffad40d4250 2024-04-25T11:35:06.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.865+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffad4076c60 con 0x7ffad40d4250 2024-04-25T11:35:06.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.872+0000 7ffacb400640 1 -- 172.21.15.19:0/2242145797 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7ffacc05fe80 con 0x7ffad40d4250 2024-04-25T11:35:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffab00744b0 msgr2=0x7ffab00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffab00744b0 0x7ffab00769c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ffabc034000 tx=0x7ffabc02e000 comp rx=0 tx=0).stop 2024-04-25T11:35:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 msgr2=0x7ffad40cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40cbbb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00ccb0 tx=0x7ffacc00a5c0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 shutdown_connections 2024-04-25T11:35:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffab00744b0 0x7ffab00769c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffad40b7580 0x7ffad40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffad40d4250 0x7ffad40cbbb0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 --2- 172.21.15.19:0/2242145797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffad40b4db0 0x7ffad415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 >> 172.21.15.19:0/2242145797 conn(0x7ffad40d2760 msgr2=0x7ffad40b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 shutdown_connections 2024-04-25T11:35:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:06.875+0000 7ffad8c00640 1 -- 172.21.15.19:0/2242145797 wait complete. 2024-04-25T11:35:06.892 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:35:06.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:35:07.010 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:35:07.121 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:35:07.121 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:07.122 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:07.183 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:35:07.183 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:35:07.183 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:07.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:35:07.239 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:07.240 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:07.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:35:07.299 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:35:07.299 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:07.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:35:07.356 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:35:07.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:35:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.663+0000 7f322ca00640 1 Processor -- start 2024-04-25T11:35:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.663+0000 7f322ca00640 1 -- start start 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.665+0000 7f322ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32280d50e0 0x7f32280d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.665+0000 7f322ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f32280d2ae0 0x7f3228160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.665+0000 7f322ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3228160b90 0x7f3228161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.665+0000 7f322ca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f32280b7940 con 0x7f32280d50e0 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.665+0000 7f322ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f32280d2ef0 con 0x7f32280d2ae0 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.665+0000 7f322ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3228161280 con 0x7f3228160b90 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32280d50e0 0x7f32280d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32280d50e0 0x7f32280d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50446/0 (socket says 172.21.15.19:50446) 2024-04-25T11:35:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 -- 172.21.15.19:0/3272413041 learned_addr learned my addr 172.21.15.19:0/3272413041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3225e00640 1 --2- 172.21.15.19:0/3272413041 >> v2:172.21.15.169:3300/0 conn(0x7f32280d2ae0 0x7f3228160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3227200640 1 --2- 172.21.15.19:0/3272413041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3228160b90 0x7f3228161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 -- 172.21.15.19:0/3272413041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3228160b90 msgr2=0x7f3228161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 --2- 172.21.15.19:0/3272413041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3228160b90 0x7f3228161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 -- 172.21.15.19:0/3272413041 >> v2:172.21.15.169:3300/0 conn(0x7f32280d2ae0 msgr2=0x7f3228160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 --2- 172.21.15.19:0/3272413041 >> v2:172.21.15.169:3300/0 conn(0x7f32280d2ae0 0x7f3228160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.666+0000 7f3226800640 1 -- 172.21.15.19:0/3272413041 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32280d4600 con 0x7f32280d50e0 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.667+0000 7f3226800640 1 --2- 172.21.15.19:0/3272413041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d50e0 0x7f32280d25a0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f3228163b80 tx=0x7f321802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefb7d8865f2faa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.668+0000 7f3225400640 1 -- 172.21.15.19:0/3272413041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218037070 con 0x7f32280d50e0 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.668+0000 7f3225400640 1 -- 172.21.15.19:0/3272413041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3218032440 con 0x7f32280d50e0 2024-04-25T11:35:07.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.668+0000 7f3225400640 1 -- 172.21.15.19:0/3272413041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218032810 con 0x7f32280d50e0 2024-04-25T11:35:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 -- 172.21.15.19:0/3272413041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d50e0 msgr2=0x7f32280d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 --2- 172.21.15.19:0/3272413041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d50e0 0x7f32280d25a0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f3228163b80 tx=0x7f321802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 -- 172.21.15.19:0/3272413041 shutdown_connections 2024-04-25T11:35:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 --2- 172.21.15.19:0/3272413041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3228160b90 0x7f3228161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 --2- 172.21.15.19:0/3272413041 >> v2:172.21.15.169:3300/0 conn(0x7f32280d2ae0 0x7f3228160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 --2- 172.21.15.19:0/3272413041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d50e0 0x7f32280d25a0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 -- 172.21.15.19:0/3272413041 >> 172.21.15.19:0/3272413041 conn(0x7f322806c5c0 msgr2=0x7f322806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 -- 172.21.15.19:0/3272413041 shutdown_connections 2024-04-25T11:35:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.669+0000 7f322ca00640 1 -- 172.21.15.19:0/3272413041 wait complete. 2024-04-25T11:35:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.671+0000 7f322ca00640 1 Processor -- start 2024-04-25T11:35:07.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.671+0000 7f322ca00640 1 -- start start 2024-04-25T11:35:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f322ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d2ae0 0x7f322814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f322ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32280d50e0 0x7f32281eb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f322ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3228160b90 0x7f32281eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f322ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3228160fa0 con 0x7f32280d2ae0 2024-04-25T11:35:07.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f322ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32280d4bc0 con 0x7f3228160b90 2024-04-25T11:35:07.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f322ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3228161560 con 0x7f32280d50e0 2024-04-25T11:35:07.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f3226800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d2ae0 0x7f322814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f3226800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d2ae0 0x7f322814d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50450/0 (socket says 172.21.15.19:50450) 2024-04-25T11:35:07.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.673+0000 7f3226800640 1 -- 172.21.15.19:0/2004069352 learned_addr learned my addr 172.21.15.19:0/2004069352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.674+0000 7f3226800640 1 -- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32280d50e0 msgr2=0x7f32281eb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.674+0000 7f3226800640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32280d50e0 0x7f32281eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.674+0000 7f3226800640 1 -- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3228160b90 msgr2=0x7f32281eb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.674+0000 7f3227200640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3228160b90 0x7f32281eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.674+0000 7f3226800640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3228160b90 0x7f32281eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.674+0000 7f3226800640 1 -- 172.21.15.19:0/2004069352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32281e9020 con 0x7f32280d2ae0 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.674+0000 7f3226800640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d2ae0 0x7f322814d6f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f321802a220 tx=0x7f32180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.675+0000 7f321ea00640 1 -- 172.21.15.19:0/2004069352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218041020 con 0x7f32280d2ae0 2024-04-25T11:35:07.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.675+0000 7f321ea00640 1 -- 172.21.15.19:0/2004069352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32180306c0 con 0x7f32280d2ae0 2024-04-25T11:35:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.675+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32281e9300 con 0x7f32280d2ae0 2024-04-25T11:35:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.675+0000 7f321ea00640 1 -- 172.21.15.19:0/2004069352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218037040 con 0x7f32280d2ae0 2024-04-25T11:35:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.675+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281e98c0 con 0x7f32280d2ae0 2024-04-25T11:35:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.676+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3228050740 con 0x7f32280d2ae0 2024-04-25T11:35:07.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.677+0000 7f321ea00640 1 -- 172.21.15.19:0/2004069352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3218044020 con 0x7f32280d2ae0 2024-04-25T11:35:07.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.678+0000 7f321ea00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3208074500 0x7f3208076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.678+0000 7f321ea00640 1 -- 172.21.15.19:0/2004069352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7f32180b5890 con 0x7f32280d2ae0 2024-04-25T11:35:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.679+0000 7f3225e00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3208074500 0x7f3208076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.680+0000 7f3225e00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3208074500 0x7f3208076a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f32281e8b80 tx=0x7f3214008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.683+0000 7f321ea00640 1 -- 172.21.15.19:0/2004069352 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f321804b030 con 0x7f32280d2ae0 2024-04-25T11:35:07.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.926+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32280b8aa0 con 0x7f32280d2ae0 2024-04-25T11:35:07.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.928+0000 7f321ea00640 1 -- 172.21.15.19:0/2004069352 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+3875 (secure 0 0 0) 0x7f32180802a0 con 0x7f32280d2ae0 2024-04-25T11:35:07.916 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:07.916 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5396,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/3617792443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6835","nonce":3617792443},{"type":"v1","addr":"172.21.15.169:6837","nonce":3617792443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":5405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/1928977171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1928977171},{"type":"v1","addr":"172.21.15.19:6833","nonce":1928977171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6836/2651865644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2651865644},{"type":"v1","addr":"172.21.15.169:6836","nonce":2651865644}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3655604895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3655604895},{"type":"v1","addr":"172.21.15.19:6835","nonce":3655604895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-04-25T11:35:07.916 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.930+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3208074500 msgr2=0x7f3208076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.930+0000 7f322ca00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3208074500 0x7f3208076a10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f32281e8b80 tx=0x7f3214008040 comp rx=0 tx=0).stop 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.930+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d2ae0 msgr2=0x7f322814d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.930+0000 7f322ca00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d2ae0 0x7f322814d6f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f321802a220 tx=0x7f32180046c0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 shutdown_connections 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3208074500 0x7f3208076a10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3228160b90 0x7f32281eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32280d50e0 0x7f32281eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 --2- 172.21.15.19:0/2004069352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32280d2ae0 0x7f322814d6f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 >> 172.21.15.19:0/2004069352 conn(0x7f322806c5c0 msgr2=0x7f32280b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:07.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 shutdown_connections 2024-04-25T11:35:07.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:07.931+0000 7f322ca00640 1 -- 172.21.15.19:0/2004069352 wait complete. 2024-04-25T11:35:07.948 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:35:07.948 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:35:07.948 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:35:07.948 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:35:07.948 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:35:07.948 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:35:07.949 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:35:07.949 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:35:07.949 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:35:07.949 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:35:07.949 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:35:07.949 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:35:07.949 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:35:07.950 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:35:07.950 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:35:07.950 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:35:07.991 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:35:07.991 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:35:07.993 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:35:07.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:35:07.996 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:35:07.996 DEBUG:teuthology.parallel:result is None 2024-04-25T11:35:07.999 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:35:07.999 DEBUG:teuthology.parallel:result is None 2024-04-25T11:35:08.002 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:35:08.002 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:35:08.004 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:35:08.004 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:35:08.047 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:35:08.048 DEBUG:teuthology.parallel:result is None 2024-04-25T11:35:08.051 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:35:08.052 DEBUG:teuthology.parallel:result is None 2024-04-25T11:35:08.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:35:08.226 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:35:08.232 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:35:08.273 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:35:08.279 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:35:08.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T11:35:08.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T11:35:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.326+0000 7f79d6e00640 1 Processor -- start 2024-04-25T11:35:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.326+0000 7f79d6e00640 1 -- start start 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.328+0000 7f79d6e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79d0164860 0x7f79d00b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.328+0000 7f79d6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00b7540 0x7f79d00b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.328+0000 7f79d6e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79d00d44a0 0x7f79d00d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.328+0000 7f79d6e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f79d01643d0 con 0x7f79d00d44a0 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.328+0000 7f79d6e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79d00b5270 con 0x7f79d0164860 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.328+0000 7f79d6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79d0161650 con 0x7f79d00b7540 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79d00d44a0 0x7f79d00d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79d00d44a0 0x7f79d00d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50480/0 (socket says 172.21.15.19:50480) 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 -- 172.21.15.19:0/3011777033 learned_addr learned my addr 172.21.15.19:0/3011777033 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d5000640 1 --2- 172.21.15.19:0/3011777033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00b7540 0x7f79d00b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d5a00640 1 --2- 172.21.15.19:0/3011777033 >> v2:172.21.15.169:3300/0 conn(0x7f79d0164860 0x7f79d00b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 -- 172.21.15.19:0/3011777033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00b7540 msgr2=0x7f79d00b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 --2- 172.21.15.19:0/3011777033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00b7540 0x7f79d00b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 -- 172.21.15.19:0/3011777033 >> v2:172.21.15.169:3300/0 conn(0x7f79d0164860 msgr2=0x7f79d00b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 --2- 172.21.15.19:0/3011777033 >> v2:172.21.15.169:3300/0 conn(0x7f79d0164860 0x7f79d00b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d6400640 1 -- 172.21.15.19:0/3011777033 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d00b7e90 con 0x7f79d00d44a0 2024-04-25T11:35:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.329+0000 7f79d5a00640 1 --2- 172.21.15.19:0/3011777033 >> v2:172.21.15.169:3300/0 conn(0x7f79d0164860 0x7f79d00b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:08.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.331+0000 7f79d6400640 1 --2- 172.21.15.19:0/3011777033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00d44a0 0x7f79d00d48b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f79d00d3820 tx=0x7f79c802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a77728be665ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:08.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.331+0000 7f79cfe00640 1 -- 172.21.15.19:0/3011777033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8036070 con 0x7f79d00d44a0 2024-04-25T11:35:08.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.331+0000 7f79cfe00640 1 -- 172.21.15.19:0/3011777033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c8031600 con 0x7f79d00d44a0 2024-04-25T11:35:08.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.331+0000 7f79cfe00640 1 -- 172.21.15.19:0/3011777033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c80319d0 con 0x7f79d00d44a0 2024-04-25T11:35:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 -- 172.21.15.19:0/3011777033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00d44a0 msgr2=0x7f79d00d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 --2- 172.21.15.19:0/3011777033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00d44a0 0x7f79d00d48b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f79d00d3820 tx=0x7f79c802a6a0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 -- 172.21.15.19:0/3011777033 shutdown_connections 2024-04-25T11:35:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 --2- 172.21.15.19:0/3011777033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00d44a0 0x7f79d00d48b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 --2- 172.21.15.19:0/3011777033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00b7540 0x7f79d00b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 --2- 172.21.15.19:0/3011777033 >> v2:172.21.15.169:3300/0 conn(0x7f79d0164860 0x7f79d00b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 -- 172.21.15.19:0/3011777033 >> 172.21.15.19:0/3011777033 conn(0x7f79d00d2720 msgr2=0x7f79d00d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.332+0000 7f79d6e00640 1 -- 172.21.15.19:0/3011777033 shutdown_connections 2024-04-25T11:35:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.333+0000 7f79d6e00640 1 -- 172.21.15.19:0/3011777033 wait complete. 2024-04-25T11:35:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.335+0000 7f79d6e00640 1 Processor -- start 2024-04-25T11:35:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.335+0000 7f79d6e00640 1 -- start start 2024-04-25T11:35:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.337+0000 7f79d6e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00b7540 0x7f79d015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.337+0000 7f79d6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00d44a0 0x7f79d0148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.337+0000 7f79d6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d0164860 0x7f79d015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.337+0000 7f79d6e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0164060 con 0x7f79d00b7540 2024-04-25T11:35:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.337+0000 7f79d6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0163d80 con 0x7f79d0164860 2024-04-25T11:35:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.337+0000 7f79d6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79d00d4df0 con 0x7f79d00d44a0 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00b7540 0x7f79d015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00b7540 0x7f79d015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50494/0 (socket says 172.21.15.19:50494) 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 -- 172.21.15.19:0/2812948169 learned_addr learned my addr 172.21.15.19:0/2812948169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5000640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00d44a0 0x7f79d0148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d6400640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d0164860 0x7f79d015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 -- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00d44a0 msgr2=0x7f79d0148c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00d44a0 0x7f79d0148c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 -- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d0164860 msgr2=0x7f79d015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d0164860 0x7f79d015e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:08.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.338+0000 7f79d5a00640 1 -- 172.21.15.19:0/2812948169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d015ab90 con 0x7f79d00b7540 2024-04-25T11:35:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.339+0000 7f79d5a00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00b7540 0x7f79d015a650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f79c400d830 tx=0x7f79c400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.339+0000 7f79cd600640 1 -- 172.21.15.19:0/2812948169 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40096a0 con 0x7f79d00b7540 2024-04-25T11:35:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.339+0000 7f79cd600640 1 -- 172.21.15.19:0/2812948169 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c4019070 con 0x7f79d00b7540 2024-04-25T11:35:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.339+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d015ccc0 con 0x7f79d00b7540 2024-04-25T11:35:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.339+0000 7f79cd600640 1 -- 172.21.15.19:0/2812948169 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4009a80 con 0x7f79d00b7540 2024-04-25T11:35:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.339+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d01480a0 con 0x7f79d00b7540 2024-04-25T11:35:08.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.341+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d0050740 con 0x7f79d00b7540 2024-04-25T11:35:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.342+0000 7f79cd600640 1 -- 172.21.15.19:0/2812948169 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79c4009c20 con 0x7f79d00b7540 2024-04-25T11:35:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.343+0000 7f79cd600640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79b00744b0 0x7f79b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:08.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.344+0000 7f79cd600640 1 -- 172.21.15.19:0/2812948169 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5029+0+0 (secure 0 0 0) 0x7f79c40951f0 con 0x7f79d00b7540 2024-04-25T11:35:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.344+0000 7f79d5000640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79b00744b0 0x7f79b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:08.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.345+0000 7f79d5000640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79b00744b0 0x7f79b00769c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f79d00cf540 tx=0x7f79bc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.351+0000 7f79cd600640 1 -- 172.21.15.19:0/2812948169 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79c405f660 con 0x7f79d00b7540 2024-04-25T11:35:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:08.569+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f79d00d3f30 con 0x7f79d00b7540 2024-04-25T11:35:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.356+0000 7f79cd600640 1 -- 172.21.15.19:0/2812948169 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7f79c4063960 con 0x7f79d00b7540 2024-04-25T11:35:09.347 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T11:35:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79b00744b0 msgr2=0x7f79b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79b00744b0 0x7f79b00769c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f79d00cf540 tx=0x7f79bc009190 comp rx=0 tx=0).stop 2024-04-25T11:35:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00b7540 msgr2=0x7f79d015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00b7540 0x7f79d015a650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f79c400d830 tx=0x7f79c400dcf0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 shutdown_connections 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79b00744b0 0x7f79b00769c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d0164860 0x7f79d015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79d00d44a0 0x7f79d0148c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 --2- 172.21.15.19:0/2812948169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d00b7540 0x7f79d015a650 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.361+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 >> 172.21.15.19:0/2812948169 conn(0x7f79d00d2720 msgr2=0x7f79d00d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.362+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 shutdown_connections 2024-04-25T11:35:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.362+0000 7f79d6e00640 1 -- 172.21.15.19:0/2812948169 wait complete. 2024-04-25T11:35:09.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:35:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.672+0000 7f0b73200640 1 Processor -- start 2024-04-25T11:35:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.672+0000 7f0b73200640 1 -- start start 2024-04-25T11:35:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.675+0000 7f0b73200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 0x7f0b6c169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.675+0000 7f0b73200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b6c15f5f0 con 0x7f0b6c1443c0 2024-04-25T11:35:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.675+0000 7f0b73200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b6c15e830 con 0x7f0b6c0cf5e0 2024-04-25T11:35:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.675+0000 7f0b73200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c0708d0 con 0x7f0b6c167000 2024-04-25T11:35:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.675+0000 7f0b6be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b6c1443c0 0x7f0b6c06fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53508/0 (socket says 172.21.15.19:53508) 2024-04-25T11:35:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.675+0000 7f0b6be00640 1 -- 172.21.15.19:0/4281134697 learned_addr learned my addr 172.21.15.19:0/4281134697 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b71200640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 0x7f0b6c169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073412890 0 0) 0x7f0b6c15f5f0 con 0x7f0b6c1443c0 2024-04-25T11:35:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b50003520 con 0x7f0b6c1443c0 2024-04-25T11:35:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2129978906 0 0) 0x7f0b50003520 con 0x7f0b6c1443c0 2024-04-25T11:35:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 msgr2=0x7f0b6c169420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 0x7f0b6c169420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 >> v1:172.21.15.169:6789/0 conn(0x7f0b6c0cf5e0 legacy=0x7f0b6c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c15eb10 con 0x7f0b6c1443c0 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f0b5c001670 con 0x7f0b6c1443c0 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 >> v1:172.21.15.19:6789/0 conn(0x7f0b6c1443c0 legacy=0x7f0b6c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 0x7f0b500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b50003f60 0x7f0b50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b50005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c0708d0 con 0x7f0b50004d40 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5000bb50 con 0x7f0b50003f60 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.676+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5000bd50 con 0x7f0b6c167000 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b71200640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b50005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b6b400640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b50003f60 0x7f0b50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b6be00640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 0x7f0b500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b71200640 1 -- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 msgr2=0x7f0b500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b71200640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c167000 0x7f0b500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b71200640 1 -- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b50003f60 msgr2=0x7f0b50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b71200640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b50003f60 0x7f0b50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.677+0000 7f0b71200640 1 -- 172.21.15.19:0/4281134697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b50003520 con 0x7f0b50004d40 2024-04-25T11:35:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.678+0000 7f0b71200640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b50005150 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009fc0 tx=0x7f0b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19313c777a6460a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.679+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64034020 con 0x7f0b50004d40 2024-04-25T11:35:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.679+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b640305f0 con 0x7f0b50004d40 2024-04-25T11:35:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.679+0000 7f0b6aa00640 1 -- 172.21.15.19:0/4281134697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64009c10 con 0x7f0b50004d40 2024-04-25T11:35:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.679+0000 7f0b73200640 1 -- 172.21.15.19:0/4281134697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 msgr2=0x7f0b50005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.679+0000 7f0b73200640 1 --2- 172.21.15.19:0/4281134697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b50005150 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009fc0 tx=0x7f0b50005830 comp rx=0 tx=0).stop 2024-04-25T11:35:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.679+0000 7f0b71200640 1 -- 172.21.15.19:0/4281134697 reap_dead start 2024-04-25T11:35:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.680+0000 7f0b73200640 1 -- 172.21.15.19:0/4281134697 shutdown_connections 2024-04-25T11:35:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.680+0000 7f0b73200640 1 -- 172.21.15.19:0/4281134697 >> 172.21.15.19:0/4281134697 conn(0x7f0b6c0d1ae0 msgr2=0x7f0b6c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.680+0000 7f0b73200640 1 -- 172.21.15.19:0/4281134697 shutdown_connections 2024-04-25T11:35:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.680+0000 7f0b73200640 1 -- 172.21.15.19:0/4281134697 wait complete. 2024-04-25T11:35:09.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.682+0000 7f0b73200640 1 Processor -- start 2024-04-25T11:35:09.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.682+0000 7f0b73200640 1 -- start start 2024-04-25T11:35:09.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b73200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b6c13e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b73200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c143670 0x7f0b6c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b73200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c15a040 0x7f0b6c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b73200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c06d5d0 con 0x7f0b50004d40 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b73200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c15f160 con 0x7f0b6c15a040 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b73200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c1447d0 con 0x7f0b6c143670 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b6c13e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b6c13e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50518/0 (socket says 172.21.15.19:50518) 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 -- 172.21.15.19:0/1713227444 learned_addr learned my addr 172.21.15.19:0/1713227444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6b400640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c143670 0x7f0b6c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 -- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c143670 msgr2=0x7f0b6c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c143670 0x7f0b6c159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 -- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c15a040 msgr2=0x7f0b6c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c15a040 0x7f0b6c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.684+0000 7f0b6be00640 1 -- 172.21.15.19:0/1713227444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c1553c0 con 0x7f0b50004d40 2024-04-25T11:35:09.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.685+0000 7f0b6be00640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b6c13e700 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e7f0 tx=0x7f0b5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:09.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.685+0000 7f0b63e00640 1 -- 172.21.15.19:0/1713227444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00c7f0 con 0x7f0b50004d40 2024-04-25T11:35:09.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.685+0000 7f0b63e00640 1 -- 172.21.15.19:0/1713227444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b5c018070 con 0x7f0b50004d40 2024-04-25T11:35:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.685+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c13ec40 con 0x7f0b50004d40 2024-04-25T11:35:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.685+0000 7f0b63e00640 1 -- 172.21.15.19:0/1713227444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00cbd0 con 0x7f0b50004d40 2024-04-25T11:35:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.685+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c155b10 con 0x7f0b50004d40 2024-04-25T11:35:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.686+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c050740 con 0x7f0b50004d40 2024-04-25T11:35:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.688+0000 7f0b63e00640 1 -- 172.21.15.19:0/1713227444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0b5c013420 con 0x7f0b50004d40 2024-04-25T11:35:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.688+0000 7f0b63e00640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b480744e0 0x7f0b480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.689+0000 7f0b63e00640 1 -- 172.21.15.19:0/1713227444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f0b5c0949e0 con 0x7f0b50004d40 2024-04-25T11:35:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.689+0000 7f0b6b400640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b480744e0 0x7f0b480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.690+0000 7f0b6b400640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b480744e0 0x7f0b480769f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0b54007fd0 tx=0x7f0b54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:09.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.694+0000 7f0b63e00640 1 -- 172.21.15.19:0/1713227444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b5c05f4c0 con 0x7f0b50004d40 2024-04-25T11:35:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.961+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0b6c0b3450 con 0x7f0b50004d40 2024-04-25T11:35:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.962+0000 7f0b63e00640 1 -- 172.21.15.19:0/1713227444 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f0b5c05edc0 con 0x7f0b50004d40 2024-04-25T11:35:09.950 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ==", 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw==", 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw==", 2024-04-25T11:35:09.952 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA==", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:35:09.953 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ==", 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.954 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw==", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q==", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw==", 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw==", 2024-04-25T11:35:09.956 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw==", 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ==", 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA==", 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.958 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w==", 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q==", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg==", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.960 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q==", 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNPSpmHT+MHRAAPZNLFcl/fxDgWnMeN3nAHw==", 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA==", 2024-04-25T11:35:09.962 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA==", 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.963 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:09.964 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:35:09.964 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b480744e0 msgr2=0x7f0b480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b480744e0 0x7f0b480769f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0b54007fd0 tx=0x7f0b54009040 comp rx=0 tx=0).stop 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 msgr2=0x7f0b6c13e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b6c13e700 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e7f0 tx=0x7f0b5c00ecb0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 shutdown_connections 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b480744e0 0x7f0b480769f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c15a040 0x7f0b6c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0b6c143670 0x7f0b6c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 --2- 172.21.15.19:0/1713227444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b50004d40 0x7f0b6c13e700 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 >> 172.21.15.19:0/1713227444 conn(0x7f0b6c0d1ae0 msgr2=0x7f0b6c1610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 shutdown_connections 2024-04-25T11:35:09.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:09.965+0000 7f0b73200640 1 -- 172.21.15.19:0/1713227444 wait complete. 2024-04-25T11:35:09.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:35:10.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.272+0000 7f314ec00640 1 Processor -- start 2024-04-25T11:35:10.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.273+0000 7f314ec00640 1 -- start start 2024-04-25T11:35:10.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.275+0000 7f314ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f31481443c0 0x7f314806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.275+0000 7f314ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3148070370 0x7f31480cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.275+0000 7f314ec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31480cfb20 0x7f31480ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.275+0000 7f314ec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f31480708d0 con 0x7f31480cfb20 2024-04-25T11:35:10.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.275+0000 7f314ec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f314815f160 con 0x7f31481443c0 2024-04-25T11:35:10.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.275+0000 7f314ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f31480cff30 con 0x7f3148070370 2024-04-25T11:35:10.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31480cfb20 0x7f31480ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31480cfb20 0x7f31480ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50522/0 (socket says 172.21.15.19:50522) 2024-04-25T11:35:10.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 -- 172.21.15.19:0/1249816889 learned_addr learned my addr 172.21.15.19:0/1249816889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:10.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f313fe00640 1 --2- 172.21.15.19:0/1249816889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3148070370 0x7f31480cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f3147e00640 1 --2- 172.21.15.19:0/1249816889 >> v2:172.21.15.169:3300/0 conn(0x7f31481443c0 0x7f314806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 -- 172.21.15.19:0/1249816889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3148070370 msgr2=0x7f31480cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:10.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 --2- 172.21.15.19:0/1249816889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3148070370 0x7f31480cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 -- 172.21.15.19:0/1249816889 >> v2:172.21.15.169:3300/0 conn(0x7f31481443c0 msgr2=0x7f314806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:10.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 --2- 172.21.15.19:0/1249816889 >> v2:172.21.15.169:3300/0 conn(0x7f31481443c0 0x7f314806fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.276+0000 7f314cc00640 1 -- 172.21.15.19:0/1249816889 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314815f5f0 con 0x7f31480cfb20 2024-04-25T11:35:10.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.277+0000 7f314cc00640 1 --2- 172.21.15.19:0/1249816889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f31480ce330 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3148072820 tx=0x7f314002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ce587b42f03514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:10.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.278+0000 7f3147400640 1 -- 172.21.15.19:0/1249816889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140036070 con 0x7f31480cfb20 2024-04-25T11:35:10.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.278+0000 7f3147400640 1 -- 172.21.15.19:0/1249816889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3140031600 con 0x7f31480cfb20 2024-04-25T11:35:10.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.278+0000 7f3147400640 1 -- 172.21.15.19:0/1249816889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400319d0 con 0x7f31480cfb20 2024-04-25T11:35:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.278+0000 7f314ec00640 1 -- 172.21.15.19:0/1249816889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 msgr2=0x7f31480ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.278+0000 7f314ec00640 1 --2- 172.21.15.19:0/1249816889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f31480ce330 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3148072820 tx=0x7f314002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.279+0000 7f314ec00640 1 -- 172.21.15.19:0/1249816889 shutdown_connections 2024-04-25T11:35:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.279+0000 7f314ec00640 1 --2- 172.21.15.19:0/1249816889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f31480ce330 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.279+0000 7f314ec00640 1 --2- 172.21.15.19:0/1249816889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3148070370 0x7f31480cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.279+0000 7f314ec00640 1 --2- 172.21.15.19:0/1249816889 >> v2:172.21.15.169:3300/0 conn(0x7f31481443c0 0x7f314806fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.279+0000 7f314ec00640 1 -- 172.21.15.19:0/1249816889 >> 172.21.15.19:0/1249816889 conn(0x7f31480d1ae0 msgr2=0x7f31480d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.279+0000 7f314ec00640 1 -- 172.21.15.19:0/1249816889 shutdown_connections 2024-04-25T11:35:10.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.279+0000 7f314ec00640 1 -- 172.21.15.19:0/1249816889 wait complete. 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.281+0000 7f314ec00640 1 Processor -- start 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.282+0000 7f314ec00640 1 -- start start 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f314ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3148070370 0x7f3148159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f314ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f3148153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f314ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31481443c0 0x7f31480b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f314ec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f314815edf0 con 0x7f31480cfb20 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f314ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f314815eb10 con 0x7f3148070370 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f314ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f31481447d0 con 0x7f31481443c0 2024-04-25T11:35:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f313fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f3148153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.283+0000 7f313fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f3148153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50524/0 (socket says 172.21.15.19:50524) 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f313fe00640 1 -- 172.21.15.19:0/3310530429 learned_addr learned my addr 172.21.15.19:0/3310530429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f3147e00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3148070370 0x7f3148159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f314cc00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31481443c0 0x7f31480b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f313fe00640 1 -- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31481443c0 msgr2=0x7f31480b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f313fe00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31481443c0 0x7f31480b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f313fe00640 1 -- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3148070370 msgr2=0x7f3148159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f313fe00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3148070370 0x7f3148159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f313fe00640 1 -- 172.21.15.19:0/3310530429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314815bb80 con 0x7f31480cfb20 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f314cc00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31481443c0 0x7f31480b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.284+0000 7f313fe00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f3148153710 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f313000d7d0 tx=0x7f313000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.285+0000 7f3144c00640 1 -- 172.21.15.19:0/3310530429 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130018020 con 0x7f31480cfb20 2024-04-25T11:35:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.285+0000 7f3144c00640 1 -- 172.21.15.19:0/3310530429 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31300096a0 con 0x7f31480cfb20 2024-04-25T11:35:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.285+0000 7f3144c00640 1 -- 172.21.15.19:0/3310530429 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130009a90 con 0x7f31480cfb20 2024-04-25T11:35:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.285+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314815be60 con 0x7f31480cfb20 2024-04-25T11:35:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.285+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31481fa0a0 con 0x7f31480cfb20 2024-04-25T11:35:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.286+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3148050740 con 0x7f31480cfb20 2024-04-25T11:35:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.287+0000 7f3144c00640 1 -- 172.21.15.19:0/3310530429 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f313001b020 con 0x7f31480cfb20 2024-04-25T11:35:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.289+0000 7f3144c00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31240744b0 0x7f31240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.289+0000 7f3144c00640 1 -- 172.21.15.19:0/3310530429 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f3130095570 con 0x7f31480cfb20 2024-04-25T11:35:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.290+0000 7f3147e00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31240744b0 0x7f31240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.291+0000 7f3147e00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31240744b0 0x7f31240769c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.294+0000 7f3144c00640 1 -- 172.21.15.19:0/3310530429 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3130022020 con 0x7f31480cfb20 2024-04-25T11:35:10.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.502+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f314815b910 con 0x7f31480cfb20 2024-04-25T11:35:10.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.504+0000 7f3144c00640 1 -- 172.21.15.19:0/3310530429 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+12892 (secure 0 0 0) 0x7f313005ffb0 con 0x7f31480cfb20 2024-04-25T11:35:10.493 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:10.493 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:35:09.351890+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:35:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.507+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31240744b0 msgr2=0x7f31240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.507+0000 7f314ec00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31240744b0 0x7f31240769c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3138006fd0 tx=0x7f3138008040 comp rx=0 tx=0).stop 2024-04-25T11:35:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.507+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 msgr2=0x7f3148153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.507+0000 7f314ec00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f3148153710 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f313000d7d0 tx=0x7f313000dc90 comp rx=0 tx=0).stop 2024-04-25T11:35:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 shutdown_connections 2024-04-25T11:35:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f31240744b0 0x7f31240769c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f31481443c0 0x7f31480b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31480cfb20 0x7f3148153710 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 --2- 172.21.15.19:0/3310530429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3148070370 0x7f3148159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 >> 172.21.15.19:0/3310530429 conn(0x7f31480d1ae0 msgr2=0x7f31480b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 shutdown_connections 2024-04-25T11:35:10.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.508+0000 7f314ec00640 1 -- 172.21.15.19:0/3310530429 wait complete. 2024-04-25T11:35:10.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:35:10.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:35:10.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.824+0000 7fad3ca00640 1 Processor -- start 2024-04-25T11:35:10.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.824+0000 7fad3ca00640 1 -- start start 2024-04-25T11:35:10.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad3ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad3815c960 0x7fad380ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad3ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad380cf080 0x7fad3806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad3ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad380cf490 con 0x7fad3815c550 2024-04-25T11:35:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad3ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fad3806c930 con 0x7fad3815c960 2024-04-25T11:35:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad3ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad380b6710 con 0x7fad380cf080 2024-04-25T11:35:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad37200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad380cf080 0x7fad3806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad35e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad3815c960 0x7fad380ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.827+0000 7fad36800640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fad3815c550 0x7fad380b6000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:58988/0 (socket says 172.21.15.19:58988) 2024-04-25T11:35:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.828+0000 7fad36800640 1 -- 172.21.15.19:0/3181488633 learned_addr learned my addr 172.21.15.19:0/3181488633 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.828+0000 7fad37200640 1 -- 172.21.15.19:0/3181488633 >> v2:172.21.15.169:3300/0 conn(0x7fad3815c960 msgr2=0x7fad380ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.828+0000 7fad37200640 1 --2- 172.21.15.19:0/3181488633 >> v2:172.21.15.169:3300/0 conn(0x7fad3815c960 0x7fad380ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.828+0000 7fad37200640 1 -- 172.21.15.19:0/3181488633 >> v1:172.21.15.169:6789/0 conn(0x7fad3815c550 legacy=0x7fad380b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.828+0000 7fad37200640 1 -- 172.21.15.19:0/3181488633 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad380d0ec0 con 0x7fad380cf080 2024-04-25T11:35:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.828+0000 7fad35e00640 1 --2- 172.21.15.19:0/3181488633 >> v2:172.21.15.169:3300/0 conn(0x7fad3815c960 0x7fad380ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:35:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.830+0000 7fad37200640 1 --2- 172.21.15.19:0/3181488633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad380cf080 0x7fad3806cdc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad300097a0 tx=0x7fad3002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1543e60de486e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:10.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.830+0000 7fad35400640 1 -- 172.21.15.19:0/3181488633 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30037070 con 0x7fad380cf080 2024-04-25T11:35:10.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.830+0000 7fad35400640 1 -- 172.21.15.19:0/3181488633 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad30032440 con 0x7fad380cf080 2024-04-25T11:35:10.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.830+0000 7fad35400640 1 -- 172.21.15.19:0/3181488633 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30032810 con 0x7fad380cf080 2024-04-25T11:35:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.831+0000 7fad3ca00640 1 -- 172.21.15.19:0/3181488633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad380cf080 msgr2=0x7fad3806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.831+0000 7fad3ca00640 1 --2- 172.21.15.19:0/3181488633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad380cf080 0x7fad3806cdc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad300097a0 tx=0x7fad3002a180 comp rx=0 tx=0).stop 2024-04-25T11:35:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.832+0000 7fad3ca00640 1 -- 172.21.15.19:0/3181488633 shutdown_connections 2024-04-25T11:35:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.832+0000 7fad3ca00640 1 --2- 172.21.15.19:0/3181488633 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad380cf080 0x7fad3806cdc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.832+0000 7fad3ca00640 1 --2- 172.21.15.19:0/3181488633 >> v2:172.21.15.169:3300/0 conn(0x7fad3815c960 0x7fad380ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.832+0000 7fad3ca00640 1 -- 172.21.15.19:0/3181488633 >> 172.21.15.19:0/3181488633 conn(0x7fad38072f40 msgr2=0x7fad38073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:10.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.832+0000 7fad3ca00640 1 -- 172.21.15.19:0/3181488633 shutdown_connections 2024-04-25T11:35:10.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.832+0000 7fad3ca00640 1 -- 172.21.15.19:0/3181488633 wait complete. 2024-04-25T11:35:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.835+0000 7fad3ca00640 1 Processor -- start 2024-04-25T11:35:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.835+0000 7fad3ca00640 1 -- start start 2024-04-25T11:35:10.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad3ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380cf080 0x7fad3813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad3ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3815c550 0x7fad380afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad3ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3815c960 0x7fad380b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad3ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad3806c5c0 con 0x7fad380cf080 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad3ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad380d1480 con 0x7fad3815c550 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad3ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad380d0a20 con 0x7fad3815c960 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad36800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380cf080 0x7fad3813cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad36800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380cf080 0x7fad3813cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50538/0 (socket says 172.21.15.19:50538) 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.837+0000 7fad36800640 1 -- 172.21.15.19:0/2705376150 learned_addr learned my addr 172.21.15.19:0/2705376150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad35e00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3815c550 0x7fad380afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad37200640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3815c960 0x7fad380b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad36800640 1 -- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3815c960 msgr2=0x7fad380b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad36800640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3815c960 0x7fad380b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad36800640 1 -- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3815c550 msgr2=0x7fad380afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad36800640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3815c550 0x7fad380afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad36800640 1 -- 172.21.15.19:0/2705376150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad38152f10 con 0x7fad380cf080 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad35e00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3815c550 0x7fad380afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.838+0000 7fad36800640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380cf080 0x7fad3813cbe0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fad2800cb00 tx=0x7fad2800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.839+0000 7fad2ea00640 1 -- 172.21.15.19:0/2705376150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28017070 con 0x7fad380cf080 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.839+0000 7fad2ea00640 1 -- 172.21.15.19:0/2705376150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2800ac40 con 0x7fad380cf080 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.839+0000 7fad2ea00640 1 -- 172.21.15.19:0/2705376150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28012400 con 0x7fad380cf080 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.839+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3813d6a0 con 0x7fad380cf080 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.839+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3813dc60 con 0x7fad380cf080 2024-04-25T11:35:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.840+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad38076c60 con 0x7fad380cf080 2024-04-25T11:35:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.841+0000 7fad2ea00640 1 -- 172.21.15.19:0/2705376150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad2801a070 con 0x7fad380cf080 2024-04-25T11:35:10.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.842+0000 7fad2ea00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad140744e0 0x7fad140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:10.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.842+0000 7fad2ea00640 1 -- 172.21.15.19:0/2705376150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7fad28094c40 con 0x7fad380cf080 2024-04-25T11:35:10.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.843+0000 7fad35e00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad140744e0 0x7fad140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:10.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.844+0000 7fad35e00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad140744e0 0x7fad140769f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fad380b5f40 tx=0x7fad20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:10.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:10.847+0000 7fad2ea00640 1 -- 172.21.15.19:0/2705376150 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad380cf490 con 0x7fad380cf080 2024-04-25T11:35:11.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.051+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad3815b9b0 con 0x7fad380cf080 2024-04-25T11:35:11.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.052+0000 7fad2ea00640 1 -- 172.21.15.19:0/2705376150 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fad2805f710 con 0x7fad380cf080 2024-04-25T11:35:11.043 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:35:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad140744e0 msgr2=0x7fad140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad140744e0 0x7fad140769f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fad380b5f40 tx=0x7fad20008040 comp rx=0 tx=0).stop 2024-04-25T11:35:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380cf080 msgr2=0x7fad3813cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380cf080 0x7fad3813cbe0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fad2800cb00 tx=0x7fad2800a060 comp rx=0 tx=0).stop 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 shutdown_connections 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad140744e0 0x7fad140769f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad3815c550 0x7fad380afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad3815c960 0x7fad380b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 --2- 172.21.15.19:0/2705376150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380cf080 0x7fad3813cbe0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 >> 172.21.15.19:0/2705376150 conn(0x7fad38072f40 msgr2=0x7fad380b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 shutdown_connections 2024-04-25T11:35:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.056+0000 7fad3ca00640 1 -- 172.21.15.19:0/2705376150 wait complete. 2024-04-25T11:35:11.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:35:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.369+0000 7f2200000640 1 Processor -- start 2024-04-25T11:35:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.369+0000 7f2200000640 1 -- start start 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f2200000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21f8142730 0x7f21f8142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f2200000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21f8070270 0x7f21f8070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f2200000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f8070bc0 con 0x7f21f81400a0 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f2200000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f21f81431d0 con 0x7f21f8142730 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f2200000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f21f81434b0 con 0x7f21f8070270 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f21fcc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21f8142730 0x7f21f8142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f21fcc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21f8142730 0x7f21f8142b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50544/0 (socket says 172.21.15.19:50544) 2024-04-25T11:35:11.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f21fcc00640 1 -- 172.21.15.19:0/1243339550 learned_addr learned my addr 172.21.15.19:0/1243339550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.372+0000 7f21fe000640 1 --2- 172.21.15.19:0/1243339550 >> v2:172.21.15.169:3300/0 conn(0x7f21f8070270 0x7f21f8070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.373+0000 7f21fcc00640 1 -- 172.21.15.19:0/1243339550 >> v2:172.21.15.169:3300/0 conn(0x7f21f8070270 msgr2=0x7f21f8070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.373+0000 7f21fcc00640 1 --2- 172.21.15.19:0/1243339550 >> v2:172.21.15.169:3300/0 conn(0x7f21f8070270 0x7f21f8070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.373+0000 7f21fcc00640 1 -- 172.21.15.19:0/1243339550 >> v1:172.21.15.169:6789/0 conn(0x7f21f81400a0 legacy=0x7f21f80b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:11.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.373+0000 7f21fcc00640 1 -- 172.21.15.19:0/1243339550 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f80b6090 con 0x7f21f8142730 2024-04-25T11:35:11.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.374+0000 7f21fcc00640 1 --2- 172.21.15.19:0/1243339550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8142730 0x7f21f8142b40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f21f80b15c0 tx=0x7f21e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cf30fb21f54eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:11.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.374+0000 7f21f7e00640 1 -- 172.21.15.19:0/1243339550 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4037070 con 0x7f21f8142730 2024-04-25T11:35:11.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.374+0000 7f21f7e00640 1 -- 172.21.15.19:0/1243339550 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4032440 con 0x7f21f8142730 2024-04-25T11:35:11.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.374+0000 7f21f7e00640 1 -- 172.21.15.19:0/1243339550 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4032810 con 0x7f21f8142730 2024-04-25T11:35:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.375+0000 7f2200000640 1 -- 172.21.15.19:0/1243339550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8142730 msgr2=0x7f21f8142b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.375+0000 7f2200000640 1 --2- 172.21.15.19:0/1243339550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8142730 0x7f21f8142b40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f21f80b15c0 tx=0x7f21e402a230 comp rx=0 tx=0).stop 2024-04-25T11:35:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.375+0000 7f2200000640 1 -- 172.21.15.19:0/1243339550 shutdown_connections 2024-04-25T11:35:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.375+0000 7f2200000640 1 --2- 172.21.15.19:0/1243339550 >> v2:172.21.15.169:3300/0 conn(0x7f21f8070270 0x7f21f8070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.375+0000 7f2200000640 1 --2- 172.21.15.19:0/1243339550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8142730 0x7f21f8142b40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.375+0000 7f2200000640 1 -- 172.21.15.19:0/1243339550 >> 172.21.15.19:0/1243339550 conn(0x7f21f8073050 msgr2=0x7f21f8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.375+0000 7f2200000640 1 -- 172.21.15.19:0/1243339550 shutdown_connections 2024-04-25T11:35:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.376+0000 7f2200000640 1 -- 172.21.15.19:0/1243339550 wait complete. 2024-04-25T11:35:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.378+0000 7f2200000640 1 Processor -- start 2024-04-25T11:35:11.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.378+0000 7f2200000640 1 -- start start 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f2200000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8070270 0x7f21f813edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f2200000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f81400a0 0x7f21f81e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f2200000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f8142730 0x7f21f81e9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f2200000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21f80b49a0 con 0x7f21f8070270 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f2200000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21f80cddb0 con 0x7f21f81400a0 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f2200000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21f80b4b20 con 0x7f21f8142730 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f21fd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8070270 0x7f21f813edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f21fd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8070270 0x7f21f813edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50554/0 (socket says 172.21.15.19:50554) 2024-04-25T11:35:11.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f21fd600640 1 -- 172.21.15.19:0/1913659772 learned_addr learned my addr 172.21.15.19:0/1913659772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f21fcc00640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f81400a0 0x7f21f81e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.380+0000 7f21fe000640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f8142730 0x7f21f81e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21fd600640 1 -- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f8142730 msgr2=0x7f21f81e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21fd600640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f8142730 0x7f21f81e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21fd600640 1 -- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f81400a0 msgr2=0x7f21f81e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21fd600640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f81400a0 0x7f21f81e9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21fd600640 1 -- 172.21.15.19:0/1913659772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f813f310 con 0x7f21f8070270 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21fd600640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8070270 0x7f21f813edd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b3c0 tx=0x7f21ec00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:11.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21f5600640 1 -- 172.21.15.19:0/1913659772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec011830 con 0x7f21f8070270 2024-04-25T11:35:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f21f5600640 1 -- 172.21.15.19:0/1913659772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec016070 con 0x7f21f8070270 2024-04-25T11:35:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.381+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f81e83d0 con 0x7f21f8070270 2024-04-25T11:35:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.382+0000 7f21f5600640 1 -- 172.21.15.19:0/1913659772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec011c10 con 0x7f21f8070270 2024-04-25T11:35:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.382+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f81e8990 con 0x7f21f8070270 2024-04-25T11:35:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.383+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f8050740 con 0x7f21f8070270 2024-04-25T11:35:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.384+0000 7f21f5600640 1 -- 172.21.15.19:0/1913659772 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f21ec01b460 con 0x7f21f8070270 2024-04-25T11:35:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.385+0000 7f21f5600640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d80744e0 0x7f21d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.385+0000 7f21f5600640 1 -- 172.21.15.19:0/1913659772 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f21ec094950 con 0x7f21f8070270 2024-04-25T11:35:11.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.386+0000 7f21fcc00640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d80744e0 0x7f21d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.387+0000 7f21fcc00640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d80744e0 0x7f21d80769f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f21e4034000 tx=0x7f21e402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:11.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.390+0000 7f21f5600640 1 -- 172.21.15.19:0/1913659772 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21ec020020 con 0x7f21f8070270 2024-04-25T11:35:11.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.595+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f21f80b5c60 con 0x7f21f8070270 2024-04-25T11:35:11.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.596+0000 7f21f5600640 1 -- 172.21.15.19:0/1913659772 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7f21f813f310 con 0x7f21f8070270 2024-04-25T11:35:11.584 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:11.584 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:35:11.584 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:35:11.584 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T11:35:11.584 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T11:35:11.584 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:35:11.585 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d80744e0 msgr2=0x7f21d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d80744e0 0x7f21d80769f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f21e4034000 tx=0x7f21e402e000 comp rx=0 tx=0).stop 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8070270 msgr2=0x7f21f813edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8070270 0x7f21f813edd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b3c0 tx=0x7f21ec00b880 comp rx=0 tx=0).stop 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 shutdown_connections 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21d80744e0 0x7f21d80769f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21f81400a0 0x7f21f81e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21f8142730 0x7f21f81e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 --2- 172.21.15.19:0/1913659772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f8070270 0x7f21f813edd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.598+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 >> 172.21.15.19:0/1913659772 conn(0x7f21f8073050 msgr2=0x7f21f80cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:11.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.599+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 shutdown_connections 2024-04-25T11:35:11.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.599+0000 7f2200000640 1 -- 172.21.15.19:0/1913659772 wait complete. 2024-04-25T11:35:11.612 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:35:11.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:35:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.910+0000 7f9856200640 1 Processor -- start 2024-04-25T11:35:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.911+0000 7f9856200640 1 -- start start 2024-04-25T11:35:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.912+0000 7f9856200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98500ce520 0x7f985015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.913+0000 7f9856200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98500cda60 con 0x7f985006dce0 2024-04-25T11:35:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.913+0000 7f9856200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f985015dfe0 con 0x7f98500ce930 2024-04-25T11:35:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.913+0000 7f9856200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f985015e2c0 con 0x7f98500ce520 2024-04-25T11:35:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.913+0000 7f9854e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98500ce520 0x7f985015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.913+0000 7f9855800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f985006dce0 0x7f98500d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53516/0 (socket says 172.21.15.19:53516) 2024-04-25T11:35:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.913+0000 7f9855800640 1 -- 172.21.15.19:0/2741351219 learned_addr learned my addr 172.21.15.19:0/2741351219 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:11.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.914+0000 7f9854e00640 1 -- 172.21.15.19:0/2741351219 >> v1:172.21.15.169:6789/0 conn(0x7f98500ce930 legacy=0x7f985006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:11.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.914+0000 7f9854e00640 1 -- 172.21.15.19:0/2741351219 >> v1:172.21.15.19:6789/0 conn(0x7f985006dce0 legacy=0x7f98500d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.914+0000 7f9854e00640 1 -- 172.21.15.19:0/2741351219 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98500cdd20 con 0x7f98500ce520 2024-04-25T11:35:11.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.914+0000 7f984f400640 1 -- 172.21.15.19:0/2741351219 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325150401 0 0) 0x7f98500cda60 con 0x7f985006dce0 2024-04-25T11:35:11.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.915+0000 7f9854e00640 1 --2- 172.21.15.19:0/2741351219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98500ce520 0x7f985015d4b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f98440097a0 tx=0x7f984402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34378f1b8673ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:11.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.915+0000 7f984f400640 1 -- 172.21.15.19:0/2741351219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844037070 con 0x7f98500ce520 2024-04-25T11:35:11.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.915+0000 7f984f400640 1 -- 172.21.15.19:0/2741351219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984402ae50 con 0x7f98500ce520 2024-04-25T11:35:11.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.915+0000 7f984f400640 1 -- 172.21.15.19:0/2741351219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844032660 con 0x7f98500ce520 2024-04-25T11:35:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.916+0000 7f9856200640 1 -- 172.21.15.19:0/2741351219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98500ce520 msgr2=0x7f985015d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.916+0000 7f9856200640 1 --2- 172.21.15.19:0/2741351219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98500ce520 0x7f985015d4b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f98440097a0 tx=0x7f984402a140 comp rx=0 tx=0).stop 2024-04-25T11:35:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.916+0000 7f9856200640 1 -- 172.21.15.19:0/2741351219 shutdown_connections 2024-04-25T11:35:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.916+0000 7f9856200640 1 --2- 172.21.15.19:0/2741351219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98500ce520 0x7f985015d4b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.916+0000 7f9856200640 1 -- 172.21.15.19:0/2741351219 >> 172.21.15.19:0/2741351219 conn(0x7f9850072f40 msgr2=0x7f9850073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.916+0000 7f9856200640 1 -- 172.21.15.19:0/2741351219 shutdown_connections 2024-04-25T11:35:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.916+0000 7f9856200640 1 -- 172.21.15.19:0/2741351219 wait complete. 2024-04-25T11:35:11.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.918+0000 7f9856200640 1 Processor -- start 2024-04-25T11:35:11.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.919+0000 7f9856200640 1 -- start start 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.920+0000 7f9856200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98500ce520 0x7f98500cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.920+0000 7f9856200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98500ce930 0x7f985015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.920+0000 7f9856200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985015ff30 0x7f98500cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.920+0000 7f9856200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98500b73d0 con 0x7f985015ff30 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.920+0000 7f9856200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9850160d30 con 0x7f98500ce930 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.920+0000 7f9856200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f98500b7550 con 0x7f98500ce520 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985015ff30 0x7f98500cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985015ff30 0x7f98500cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50558/0 (socket says 172.21.15.19:50558) 2024-04-25T11:35:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 -- 172.21.15.19:0/1081832659 learned_addr learned my addr 172.21.15.19:0/1081832659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9854e00640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98500ce520 0x7f98500cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 -- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98500ce520 msgr2=0x7f98500cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98500ce520 0x7f98500cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f984fe00640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98500ce930 0x7f985015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 -- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98500ce930 msgr2=0x7f985015f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98500ce930 0x7f985015f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 -- 172.21.15.19:0/1081832659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9850160340 con 0x7f985015ff30 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f984fe00640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98500ce930 0x7f985015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.921+0000 7f9855800640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985015ff30 0x7f98500cabf0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f984800c780 tx=0x7f984800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.922+0000 7f984cc00640 1 -- 172.21.15.19:0/1081832659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848018070 con 0x7f985015ff30 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.922+0000 7f984cc00640 1 -- 172.21.15.19:0/1081832659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984800a640 con 0x7f985015ff30 2024-04-25T11:35:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.922+0000 7f984cc00640 1 -- 172.21.15.19:0/1081832659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984800aa30 con 0x7f985015ff30 2024-04-25T11:35:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.922+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98500cc890 con 0x7f985015ff30 2024-04-25T11:35:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.922+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98500cce50 con 0x7f985015ff30 2024-04-25T11:35:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.923+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98500cc600 con 0x7f985015ff30 2024-04-25T11:35:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.925+0000 7f984cc00640 1 -- 172.21.15.19:0/1081832659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f984801b070 con 0x7f985015ff30 2024-04-25T11:35:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.926+0000 7f984cc00640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98300744b0 0x7f98300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.926+0000 7f984cc00640 1 -- 172.21.15.19:0/1081832659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f9848094540 con 0x7f985015ff30 2024-04-25T11:35:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.926+0000 7f9854e00640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98300744b0 0x7f98300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.928+0000 7f9854e00640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98300744b0 0x7f98300769c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9844034000 tx=0x7f984402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:11.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:11.930+0000 7f984cc00640 1 -- 172.21.15.19:0/1081832659 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9850160340 con 0x7f985015ff30 2024-04-25T11:35:12.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.131+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f98500b5480 con 0x7f985015ff30 2024-04-25T11:35:12.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.376+0000 7f984cc00640 1 -- 172.21.15.19:0/1081832659 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7f984805ef80 con 0x7f985015ff30 2024-04-25T11:35:12.365 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T11:35:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.380+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98300744b0 msgr2=0x7f98300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.380+0000 7f9856200640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98300744b0 0x7f98300769c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9844034000 tx=0x7f984402e000 comp rx=0 tx=0).stop 2024-04-25T11:35:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.380+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985015ff30 msgr2=0x7f98500cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.380+0000 7f9856200640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985015ff30 0x7f98500cabf0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f984800c780 tx=0x7f984800cc40 comp rx=0 tx=0).stop 2024-04-25T11:35:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 shutdown_connections 2024-04-25T11:35:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98300744b0 0x7f98300769c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98500ce930 0x7f985015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f985015ff30 0x7f98500cabf0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 --2- 172.21.15.19:0/1081832659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98500ce520 0x7f98500cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 >> 172.21.15.19:0/1081832659 conn(0x7f9850072f40 msgr2=0x7f98500d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 shutdown_connections 2024-04-25T11:35:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.381+0000 7f9856200640 1 -- 172.21.15.19:0/1081832659 wait complete. 2024-04-25T11:35:12.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:35:12.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.690+0000 7fb992800640 1 Processor -- start 2024-04-25T11:35:12.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.691+0000 7fb992800640 1 -- start start 2024-04-25T11:35:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb992800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 0x7fb98c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb992800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb98c15ee60 0x7fb98c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb992800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98c06df20 con 0x7fb98c06db10 2024-04-25T11:35:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb992800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c15f270 con 0x7fb98c15ee60 2024-04-25T11:35:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb992800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c144750 con 0x7fb98c144340 2024-04-25T11:35:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb990a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb98c06db10 0x7fb98c15e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53522/0 (socket says 172.21.15.19:53522) 2024-04-25T11:35:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb990a00640 1 -- 172.21.15.19:0/1353524058 learned_addr learned my addr 172.21.15.19:0/1353524058 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb991400640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 0x7fb98c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.693+0000 7fb991e00640 1 --2- 172.21.15.19:0/1353524058 >> v2:172.21.15.169:3300/0 conn(0x7fb98c15ee60 0x7fb98c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343963633 0 0) 0x7fb978001110 con 0x7fb98c06db10 2024-04-25T11:35:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb974003520 con 0x7fb98c06db10 2024-04-25T11:35:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1433906688 0 0) 0x7fb974003520 con 0x7fb98c06db10 2024-04-25T11:35:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 msgr2=0x7fb98c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 0x7fb98c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 >> v2:172.21.15.169:3300/0 conn(0x7fb98c15ee60 msgr2=0x7fb98c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 --2- 172.21.15.19:0/1353524058 >> v2:172.21.15.169:3300/0 conn(0x7fb98c15ee60 0x7fb98c15f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c070000 con 0x7fb98c06db10 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb978001840 con 0x7fb98c06db10 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 >> v1:172.21.15.19:6789/0 conn(0x7fb98c06db10 legacy=0x7fb98c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9740039b0 0x7fb974003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974004800 0x7fb974004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 0x7fb974005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c144750 con 0x7fb974004800 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb97400b9c0 con 0x7fb9740039b0 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.694+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb97400bb40 con 0x7fb98c144340 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb990a00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974004800 0x7fb974004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb991400640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9740039b0 0x7fb974003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb991e00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 0x7fb974005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb990a00640 1 -- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 msgr2=0x7fb974005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb990a00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb98c144340 0x7fb974005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb990a00640 1 -- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9740039b0 msgr2=0x7fb974003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb990a00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9740039b0 0x7fb974003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb990a00640 1 -- 172.21.15.19:0/1353524058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb974003520 con 0x7fb974004800 2024-04-25T11:35:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.695+0000 7fb991400640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9740039b0 0x7fb974003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.696+0000 7fb990a00640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974004800 0x7fb974004c10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb98c073ef0 tx=0x7fb974005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5ec5c5d09b4821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97800c0e0 con 0x7fb974004800 2024-04-25T11:35:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb978034070 con 0x7fb974004800 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb98be00640 1 -- 172.21.15.19:0/1353524058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978005d30 con 0x7fb974004800 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb992800640 1 -- 172.21.15.19:0/1353524058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974004800 msgr2=0x7fb974004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb992800640 1 --2- 172.21.15.19:0/1353524058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb974004800 0x7fb974004c10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb98c073ef0 tx=0x7fb974005830 comp rx=0 tx=0).stop 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb991e00640 1 -- 172.21.15.19:0/1353524058 reap_dead start 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb992800640 1 -- 172.21.15.19:0/1353524058 shutdown_connections 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb992800640 1 -- 172.21.15.19:0/1353524058 >> 172.21.15.19:0/1353524058 conn(0x7fb98c072f40 msgr2=0x7fb98c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb992800640 1 -- 172.21.15.19:0/1353524058 shutdown_connections 2024-04-25T11:35:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.697+0000 7fb992800640 1 -- 172.21.15.19:0/1353524058 wait complete. 2024-04-25T11:35:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.700+0000 7fb992800640 1 Processor -- start 2024-04-25T11:35:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.700+0000 7fb992800640 1 -- start start 2024-04-25T11:35:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb992800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb974004800 0x7fb98c141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb992800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb98c1f4f80 0x7fb98c1f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb992800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1f8060 0x7fb98c145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb992800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c06f800 con 0x7fb98c1f8060 2024-04-25T11:35:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb992800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c06f520 con 0x7fb98c1f4f80 2024-04-25T11:35:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb992800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c06fae0 con 0x7fb974004800 2024-04-25T11:35:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb991e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1f8060 0x7fb98c145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb991e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1f8060 0x7fb98c145920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50584/0 (socket says 172.21.15.19:50584) 2024-04-25T11:35:12.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.702+0000 7fb991e00640 1 -- 172.21.15.19:0/832720333 learned_addr learned my addr 172.21.15.19:0/832720333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb991400640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb974004800 0x7fb98c141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb990a00640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb98c1f4f80 0x7fb98c1f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb991e00640 1 -- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb974004800 msgr2=0x7fb98c141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb991e00640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb974004800 0x7fb98c141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb991e00640 1 -- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb98c1f4f80 msgr2=0x7fb98c1f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb991e00640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb98c1f4f80 0x7fb98c1f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb991e00640 1 -- 172.21.15.19:0/832720333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c1f8470 con 0x7fb98c1f8060 2024-04-25T11:35:12.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.703+0000 7fb991e00640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1f8060 0x7fb98c145920 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb98c147c30 tx=0x7fb98400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.704+0000 7fb989600640 1 -- 172.21.15.19:0/832720333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984016620 con 0x7fb98c1f8060 2024-04-25T11:35:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.704+0000 7fb989600640 1 -- 172.21.15.19:0/832720333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98401b070 con 0x7fb98c1f8060 2024-04-25T11:35:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.704+0000 7fb989600640 1 -- 172.21.15.19:0/832720333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984016a20 con 0x7fb98c1f8060 2024-04-25T11:35:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.704+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c0aad60 con 0x7fb98c1f8060 2024-04-25T11:35:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.704+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c0ab320 con 0x7fb98c1f8060 2024-04-25T11:35:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.705+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c076c60 con 0x7fb98c1f8060 2024-04-25T11:35:12.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.707+0000 7fb989600640 1 -- 172.21.15.19:0/832720333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb984005d10 con 0x7fb98c1f8060 2024-04-25T11:35:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.708+0000 7fb989600640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb96c0744b0 0x7fb96c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.708+0000 7fb989600640 1 -- 172.21.15.19:0/832720333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5227+0+0 (secure 0 0 0) 0x7fb984096280 con 0x7fb98c1f8060 2024-04-25T11:35:12.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.709+0000 7fb991400640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb96c0744b0 0x7fb96c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:12.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.710+0000 7fb991400640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb96c0744b0 0x7fb96c0769c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb9800090e0 tx=0x7fb980008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:12.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.712+0000 7fb989600640 1 -- 172.21.15.19:0/832720333 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb984024020 con 0x7fb98c1f8060 2024-04-25T11:35:12.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:12.918+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb98c0b7920 con 0x7fb98c1f8060 2024-04-25T11:35:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.395+0000 7fb989600640 1 -- 172.21.15.19:0/832720333 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7fb98c1f8470 con 0x7fb98c1f8060 2024-04-25T11:35:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T11:35:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.399+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb96c0744b0 msgr2=0x7fb96c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.399+0000 7fb992800640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb96c0744b0 0x7fb96c0769c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb9800090e0 tx=0x7fb980008040 comp rx=0 tx=0).stop 2024-04-25T11:35:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.399+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1f8060 msgr2=0x7fb98c145920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.399+0000 7fb992800640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1f8060 0x7fb98c145920 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb98c147c30 tx=0x7fb98400f520 comp rx=0 tx=0).stop 2024-04-25T11:35:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 shutdown_connections 2024-04-25T11:35:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb96c0744b0 0x7fb96c0769c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb98c1f4f80 0x7fb98c1f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb98c1f8060 0x7fb98c145920 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 --2- 172.21.15.19:0/832720333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb974004800 0x7fb98c141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 >> 172.21.15.19:0/832720333 conn(0x7fb98c072f40 msgr2=0x7fb98c160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 shutdown_connections 2024-04-25T11:35:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.400+0000 7fb992800640 1 -- 172.21.15.19:0/832720333 wait complete. 2024-04-25T11:35:13.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:35:13.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.697+0000 7f5043200640 1 Processor -- start 2024-04-25T11:35:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.697+0000 7f5043200640 1 -- start start 2024-04-25T11:35:13.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.700+0000 7f5043200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f503c144340 0x7f503c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.700+0000 7f5043200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 0x7f503c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.700+0000 7f5043200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f503c06df20 con 0x7f503c06db10 2024-04-25T11:35:13.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.700+0000 7f5043200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f503c15f270 con 0x7f503c144340 2024-04-25T11:35:13.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.700+0000 7f5043200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f503c144750 con 0x7f503c15ee60 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.700+0000 7f5041400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f503c06db10 0x7f503c15e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53538/0 (socket says 172.21.15.19:53538) 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.700+0000 7f5041400640 1 -- 172.21.15.19:0/679045998 learned_addr learned my addr 172.21.15.19:0/679045998 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.701+0000 7f5041e00640 1 --2- 172.21.15.19:0/679045998 >> v2:172.21.15.169:3300/0 conn(0x7f503c144340 0x7f503c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.701+0000 7f5042800640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 0x7f503c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.701+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2901185500 0 0) 0x7f502c001110 con 0x7f503c06db10 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.701+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5024003520 con 0x7f503c06db10 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 333183108 0 0) 0x7f5024003520 con 0x7f503c06db10 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 msgr2=0x7f503c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 0x7f503c15f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 >> v2:172.21.15.169:3300/0 conn(0x7f503c144340 msgr2=0x7f503c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 --2- 172.21.15.19:0/679045998 >> v2:172.21.15.169:3300/0 conn(0x7f503c144340 0x7f503c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c070000 con 0x7f503c06db10 2024-04-25T11:35:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f502c001840 con 0x7f503c06db10 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 >> v1:172.21.15.19:6789/0 conn(0x7f503c06db10 legacy=0x7f503c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 0x7f50240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024003f30 0x7f5024004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5024004d40 0x7f5024005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f503c144750 con 0x7f5024004d40 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f502400b9c0 con 0x7f5024003f30 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f502400bb90 con 0x7f503c15ee60 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5042800640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5024004d40 0x7f5024005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5041400640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024003f30 0x7f5024004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5041e00640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 0x7f50240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5042800640 1 -- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 msgr2=0x7f50240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.702+0000 7f5042800640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c15ee60 0x7f50240039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.703+0000 7f5042800640 1 -- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024003f30 msgr2=0x7f5024004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:13.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.703+0000 7f5042800640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024003f30 0x7f5024004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.703+0000 7f5042800640 1 -- 172.21.15.19:0/679045998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5024003520 con 0x7f5024004d40 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.704+0000 7f5042800640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5024004d40 0x7f5024005150 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f503c0d1790 tx=0x7f5024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9dfe0b3eacc7d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.704+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038009c10 con 0x7f5024004d40 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.704+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038036070 con 0x7f5024004d40 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.704+0000 7f5040a00640 1 -- 172.21.15.19:0/679045998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038031830 con 0x7f5024004d40 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.705+0000 7f5043200640 1 -- 172.21.15.19:0/679045998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5024004d40 msgr2=0x7f5024005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.705+0000 7f5043200640 1 --2- 172.21.15.19:0/679045998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5024004d40 0x7f5024005150 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f503c0d1790 tx=0x7f5024005830 comp rx=0 tx=0).stop 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.705+0000 7f5042800640 1 -- 172.21.15.19:0/679045998 reap_dead start 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.705+0000 7f5043200640 1 -- 172.21.15.19:0/679045998 shutdown_connections 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.705+0000 7f5043200640 1 -- 172.21.15.19:0/679045998 >> 172.21.15.19:0/679045998 conn(0x7f503c072f40 msgr2=0x7f503c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.705+0000 7f5043200640 1 -- 172.21.15.19:0/679045998 shutdown_connections 2024-04-25T11:35:13.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.705+0000 7f5043200640 1 -- 172.21.15.19:0/679045998 wait complete. 2024-04-25T11:35:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.707+0000 7f5043200640 1 Processor -- start 2024-04-25T11:35:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.707+0000 7f5043200640 1 -- start start 2024-04-25T11:35:13.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5043200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024004d40 0x7f503c0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5043200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c1fa490 0x7f503c13ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5043200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f503c1f98f0 0x7f503c1f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5043200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f503c06f800 con 0x7f503c1f98f0 2024-04-25T11:35:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5043200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f503c06f520 con 0x7f5024004d40 2024-04-25T11:35:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5043200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f503c06fae0 con 0x7f503c1fa490 2024-04-25T11:35:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5042800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f503c1f98f0 0x7f503c1f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5042800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f503c1f98f0 0x7f503c1f9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50592/0 (socket says 172.21.15.19:50592) 2024-04-25T11:35:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5042800640 1 -- 172.21.15.19:0/1653472336 learned_addr learned my addr 172.21.15.19:0/1653472336 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.709+0000 7f5041e00640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024004d40 0x7f503c0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5041400640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c1fa490 0x7f503c13ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5042800640 1 -- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c1fa490 msgr2=0x7f503c13ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5042800640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c1fa490 0x7f503c13ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5042800640 1 -- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024004d40 msgr2=0x7f503c0c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5042800640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024004d40 0x7f503c0c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5042800640 1 -- 172.21.15.19:0/1653472336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c1fa240 con 0x7f503c1f98f0 2024-04-25T11:35:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5042800640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f503c1f98f0 0x7f503c1f9d00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5038033040 tx=0x7f5038009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:13.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5036000640 1 -- 172.21.15.19:0/1653472336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038009c10 con 0x7f503c1f98f0 2024-04-25T11:35:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5036000640 1 -- 172.21.15.19:0/1653472336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038003990 con 0x7f503c1f98f0 2024-04-25T11:35:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5036000640 1 -- 172.21.15.19:0/1653472336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003d80 con 0x7f503c1f98f0 2024-04-25T11:35:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.710+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503c15dc90 con 0x7f503c1f98f0 2024-04-25T11:35:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.711+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503c15e250 con 0x7f503c1f98f0 2024-04-25T11:35:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.712+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503c076c60 con 0x7f503c1f98f0 2024-04-25T11:35:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.713+0000 7f5036000640 1 -- 172.21.15.19:0/1653472336 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5038005d10 con 0x7f503c1f98f0 2024-04-25T11:35:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.714+0000 7f5036000640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5020074500 0x7f5020076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:13.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.714+0000 7f5036000640 1 -- 172.21.15.19:0/1653472336 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5609+0+0 (secure 0 0 0) 0x7f50380b6860 con 0x7f503c1f98f0 2024-04-25T11:35:13.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.715+0000 7f5041e00640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5020074500 0x7f5020076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.716+0000 7f5041e00640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5020074500 0x7f5020076a10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5030009a90 tx=0x7f5030008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:13.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.720+0000 7f5036000640 1 -- 172.21.15.19:0/1653472336 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f503804a060 con 0x7f503c1f98f0 2024-04-25T11:35:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:13.962+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f503c15e420 con 0x7f503c1f98f0 2024-04-25T11:35:14.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.399+0000 7f5036000640 1 -- 172.21.15.19:0/1653472336 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v86) ==== 174+0+0 (secure 0 0 0) 0x7f503c1fa240 con 0x7f503c1f98f0 2024-04-25T11:35:14.387 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T11:35:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5020074500 msgr2=0x7f5020076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5020074500 0x7f5020076a10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5030009a90 tx=0x7f5030008040 comp rx=0 tx=0).stop 2024-04-25T11:35:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f503c1f98f0 msgr2=0x7f503c1f9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f503c1f98f0 0x7f503c1f9d00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5038033040 tx=0x7f5038009750 comp rx=0 tx=0).stop 2024-04-25T11:35:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 shutdown_connections 2024-04-25T11:35:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5020074500 0x7f5020076a10 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5030009a90 tx=0x7f5030008040 comp rx=0 tx=0).stop 2024-04-25T11:35:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f503c1f98f0 0x7f503c1f9d00 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f503c1fa490 0x7f503c13ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 --2- 172.21.15.19:0/1653472336 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5024004d40 0x7f503c0c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.404+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 >> 172.21.15.19:0/1653472336 conn(0x7f503c072f40 msgr2=0x7f503c160660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.405+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 shutdown_connections 2024-04-25T11:35:14.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.406+0000 7f5043200640 1 -- 172.21.15.19:0/1653472336 wait complete. 2024-04-25T11:35:14.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:35:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.694+0000 7f4ccca00640 1 Processor -- start 2024-04-25T11:35:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.695+0000 7f4ccca00640 1 -- start start 2024-04-25T11:35:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.697+0000 7f4ccca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4cc8160fe0 0x7f4cc815a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.697+0000 7f4ccca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc80b7220 con 0x7f4cc80cf2f0 2024-04-25T11:35:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.697+0000 7f4ccca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc80b74d0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.697+0000 7f4ccca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4cc806f870 con 0x7f4cc8160fe0 2024-04-25T11:35:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.697+0000 7f4cc7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4cc8160fe0 0x7f4cc815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.697+0000 7f4cc7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4cc80cf2f0 0x7f4cc80b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35864/0 (socket says 172.21.15.19:35864) 2024-04-25T11:35:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.697+0000 7f4cc7400640 1 -- 172.21.15.19:0/4009595939 learned_addr learned my addr 172.21.15.19:0/4009595939 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.698+0000 7f4cc6000640 1 -- 172.21.15.19:0/4009595939 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1583095764 0 0) 0x7f4cc80b7220 con 0x7f4cc80cf2f0 2024-04-25T11:35:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.698+0000 7f4cc7e00640 1 -- 172.21.15.19:0/4009595939 >> v1:172.21.15.169:6789/0 conn(0x7f4cc80b6e10 legacy=0x7f4cc81608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.698+0000 7f4cc7e00640 1 -- 172.21.15.19:0/4009595939 >> v1:172.21.15.19:6789/0 conn(0x7f4cc80cf2f0 legacy=0x7f4cc80b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.698+0000 7f4cc7e00640 1 -- 172.21.15.19:0/4009595939 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc80b77b0 con 0x7f4cc8160fe0 2024-04-25T11:35:14.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.699+0000 7f4cc7e00640 1 --2- 172.21.15.19:0/4009595939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc8160fe0 0x7f4cc815a810 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009d90 tx=0x7f4cc002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceacf10e3d5dcd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.700+0000 7f4cc6000640 1 -- 172.21.15.19:0/4009595939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0007740 con 0x7f4cc8160fe0 2024-04-25T11:35:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.700+0000 7f4cc6000640 1 -- 172.21.15.19:0/4009595939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc00078e0 con 0x7f4cc8160fe0 2024-04-25T11:35:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.700+0000 7f4cc6000640 1 -- 172.21.15.19:0/4009595939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0007cb0 con 0x7f4cc8160fe0 2024-04-25T11:35:14.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.700+0000 7f4ccca00640 1 -- 172.21.15.19:0/4009595939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc8160fe0 msgr2=0x7f4cc815a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.700+0000 7f4ccca00640 1 --2- 172.21.15.19:0/4009595939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc8160fe0 0x7f4cc815a810 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009d90 tx=0x7f4cc002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.701+0000 7f4ccca00640 1 -- 172.21.15.19:0/4009595939 shutdown_connections 2024-04-25T11:35:14.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.701+0000 7f4ccca00640 1 --2- 172.21.15.19:0/4009595939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc8160fe0 0x7f4cc815a810 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.701+0000 7f4ccca00640 1 -- 172.21.15.19:0/4009595939 >> 172.21.15.19:0/4009595939 conn(0x7f4cc80d1720 msgr2=0x7f4cc80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:14.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.701+0000 7f4ccca00640 1 -- 172.21.15.19:0/4009595939 shutdown_connections 2024-04-25T11:35:14.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.701+0000 7f4ccca00640 1 -- 172.21.15.19:0/4009595939 wait complete. 2024-04-25T11:35:14.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.703+0000 7f4ccca00640 1 Processor -- start 2024-04-25T11:35:14.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.703+0000 7f4ccca00640 1 -- start start 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4ccca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc80b6e10 0x7f4cc815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4ccca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cc80cf2f0 0x7f4cc813e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4ccca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cc815f890 0x7f4cc80b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4ccca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc806fd00 con 0x7f4cc80b6e10 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4ccca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc806f500 con 0x7f4cc815f890 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4ccca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc806fe80 con 0x7f4cc80cf2f0 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc80b6e10 0x7f4cc815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc80b6e10 0x7f4cc815c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36442/0 (socket says 172.21.15.19:36442) 2024-04-25T11:35:14.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 -- 172.21.15.19:0/580479099 learned_addr learned my addr 172.21.15.19:0/580479099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:14.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 -- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cc80cf2f0 msgr2=0x7f4cc813e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cc80cf2f0 0x7f4cc813e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 -- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cc815f890 msgr2=0x7f4cc80b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cc815f890 0x7f4cc80b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.705+0000 7f4cc7400640 1 -- 172.21.15.19:0/580479099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc815e7f0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.706+0000 7f4cc7400640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc80b6e10 0x7f4cc815c960 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800c420 tx=0x7f4cb800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.706+0000 7f4cbf400640 1 -- 172.21.15.19:0/580479099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80074e0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.706+0000 7f4cbf400640 1 -- 172.21.15.19:0/580479099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb80047e0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.706+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc815ead0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.706+0000 7f4cbf400640 1 -- 172.21.15.19:0/580479099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8004bd0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.706+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc815f090 con 0x7f4cc80b6e10 2024-04-25T11:35:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.707+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc8050740 con 0x7f4cc80b6e10 2024-04-25T11:35:14.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.708+0000 7f4cbf400640 1 -- 172.21.15.19:0/580479099 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4cb8011070 con 0x7f4cc80b6e10 2024-04-25T11:35:14.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.709+0000 7f4cbf400640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ca40744e0 0x7f4ca40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:14.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.709+0000 7f4cbf400640 1 -- 172.21.15.19:0/580479099 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f4cb8094e30 con 0x7f4cc80b6e10 2024-04-25T11:35:14.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.710+0000 7f4cc6a00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ca40744e0 0x7f4ca40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.711+0000 7f4cc6a00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ca40744e0 0x7f4ca40769f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0000f60 tx=0x7f4cb00079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:14.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.714+0000 7f4cbf400640 1 -- 172.21.15.19:0/580479099 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4cb805f5c0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.908+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cc806c770 con 0x7f4cc80b6e10 2024-04-25T11:35:14.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.910+0000 7f4cbf400640 1 -- 172.21.15.19:0/580479099 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16685 (secure 0 0 0) 0x7f4cb805eec0 con 0x7f4cc80b6e10 2024-04-25T11:35:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:14.899 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:35:13.963982+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:35:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ca40744e0 msgr2=0x7f4ca40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ca40744e0 0x7f4ca40769f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0000f60 tx=0x7f4cb00079d0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc80b6e10 msgr2=0x7f4cc815c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc80b6e10 0x7f4cc815c960 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800c420 tx=0x7f4cb800c8e0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 shutdown_connections 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ca40744e0 0x7f4ca40769f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cc815f890 0x7f4cc80b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4cc80cf2f0 0x7f4cc813e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 --2- 172.21.15.19:0/580479099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc80b6e10 0x7f4cc815c960 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 >> 172.21.15.19:0/580479099 conn(0x7f4cc80d1720 msgr2=0x7f4cc80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 shutdown_connections 2024-04-25T11:35:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:14.913+0000 7f4ccca00640 1 -- 172.21.15.19:0/580479099 wait complete. 2024-04-25T11:35:14.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:35:15.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T11:35:15.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T11:35:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.205+0000 7f8e4b200640 1 Processor -- start 2024-04-25T11:35:15.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.205+0000 7f8e4b200640 1 -- start start 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e4b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8e4c06c5c0 0x7f8e4c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e4b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8e4c144370 0x7f8e4c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e4b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0b4cd0 0x7f8e4c161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e4b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c0b50e0 con 0x7f8e4c06c5c0 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e4b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8e4c0d1230 con 0x7f8e4c144370 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e4b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c0d06b0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e49e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8e4c06c5c0 0x7f8e4c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e49e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8e4c06c5c0 0x7f8e4c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36456/0 (socket says 172.21.15.19:36456) 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.207+0000 7f8e49e00640 1 -- 172.21.15.19:0/3332946260 learned_addr learned my addr 172.21.15.19:0/3332946260 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e49400640 1 --2- 172.21.15.19:0/3332946260 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c144370 0x7f8e4c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e4a800640 1 --2- 172.21.15.19:0/3332946260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0b4cd0 0x7f8e4c161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e49e00640 1 -- 172.21.15.19:0/3332946260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0b4cd0 msgr2=0x7f8e4c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e49e00640 1 --2- 172.21.15.19:0/3332946260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0b4cd0 0x7f8e4c161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e49e00640 1 -- 172.21.15.19:0/3332946260 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c144370 msgr2=0x7f8e4c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e49e00640 1 --2- 172.21.15.19:0/3332946260 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c144370 0x7f8e4c0b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e49e00640 1 -- 172.21.15.19:0/3332946260 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c0d0990 con 0x7f8e4c06c5c0 2024-04-25T11:35:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.208+0000 7f8e49400640 1 --2- 172.21.15.19:0/3332946260 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c144370 0x7f8e4c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.209+0000 7f8e49e00640 1 --2- 172.21.15.19:0/3332946260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c06c5c0 0x7f8e4c0cfa70 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0b5550 tx=0x7f8e3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2d056ee4cdc9cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.209+0000 7f8e48a00640 1 -- 172.21.15.19:0/3332946260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c037070 con 0x7f8e4c06c5c0 2024-04-25T11:35:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.209+0000 7f8e48a00640 1 -- 172.21.15.19:0/3332946260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c032440 con 0x7f8e4c06c5c0 2024-04-25T11:35:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.209+0000 7f8e48a00640 1 -- 172.21.15.19:0/3332946260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c032810 con 0x7f8e4c06c5c0 2024-04-25T11:35:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 -- 172.21.15.19:0/3332946260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c06c5c0 msgr2=0x7f8e4c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3332946260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c06c5c0 0x7f8e4c0cfa70 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0b5550 tx=0x7f8e3c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 -- 172.21.15.19:0/3332946260 shutdown_connections 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3332946260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c0b4cd0 0x7f8e4c161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3332946260 >> v2:172.21.15.169:3300/0 conn(0x7f8e4c144370 0x7f8e4c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3332946260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c06c5c0 0x7f8e4c0cfa70 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 -- 172.21.15.19:0/3332946260 >> 172.21.15.19:0/3332946260 conn(0x7f8e4c0d1720 msgr2=0x7f8e4c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 -- 172.21.15.19:0/3332946260 shutdown_connections 2024-04-25T11:35:15.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.210+0000 7f8e4b200640 1 -- 172.21.15.19:0/3332946260 wait complete. 2024-04-25T11:35:15.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.213+0000 7f8e4b200640 1 Processor -- start 2024-04-25T11:35:15.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.213+0000 7f8e4b200640 1 -- start start 2024-04-25T11:35:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.214+0000 7f8e4b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c06c5c0 0x7f8e4c137980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.214+0000 7f8e4b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c0b4cd0 0x7f8e4c1f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e4b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c144370 0x7f8e4c1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e4b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c0d02d0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e4b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c0d0f50 con 0x7f8e4c144370 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e4b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c0d0450 con 0x7f8e4c06c5c0 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c0b4cd0 0x7f8e4c1f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c0b4cd0 0x7f8e4c1f9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36464/0 (socket says 172.21.15.19:36464) 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 -- 172.21.15.19:0/3342227226 learned_addr learned my addr 172.21.15.19:0/3342227226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e4a800640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c144370 0x7f8e4c1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49e00640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c06c5c0 0x7f8e4c137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 -- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c06c5c0 msgr2=0x7f8e4c137980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c06c5c0 0x7f8e4c137980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 -- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c144370 msgr2=0x7f8e4c1f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c144370 0x7f8e4c1f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.215+0000 7f8e49400640 1 -- 172.21.15.19:0/3342227226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1fa4d0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.216+0000 7f8e49400640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c0b4cd0 0x7f8e4c1f9a50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400d7d0 tx=0x7f8e3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.216+0000 7f8e42000640 1 -- 172.21.15.19:0/3342227226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34004030 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.216+0000 7f8e42000640 1 -- 172.21.15.19:0/3342227226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e340041d0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.216+0000 7f8e42000640 1 -- 172.21.15.19:0/3342227226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400b6c0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.216+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c1fa7b0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.216+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1f9f90 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.217+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c1fa980 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.219+0000 7f8e42000640 1 -- 172.21.15.19:0/3342227226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e3400b860 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.220+0000 7f8e42000640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e280744e0 0x7f8e280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.220+0000 7f8e42000640 1 -- 172.21.15.19:0/3342227226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f8e340953e0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.220+0000 7f8e49e00640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e280744e0 0x7f8e280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.221+0000 7f8e49e00640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e280744e0 0x7f8e280769f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c034000 tx=0x7f8e3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:15.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.224+0000 7f8e42000640 1 -- 172.21.15.19:0/3342227226 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e3405fbf0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.434+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e4c1fa260 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.444+0000 7f8e42000640 1 -- 172.21.15.19:0/3342227226 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16672 (secure 0 0 0) 0x7f8e3405f4f0 con 0x7f8e4c0b4cd0 2024-04-25T11:35:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:15.433 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":92,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:35:15.422148+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:35:15.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e280744e0 msgr2=0x7f8e280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e280744e0 0x7f8e280769f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c034000 tx=0x7f8e3c02e000 comp rx=0 tx=0).stop 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c0b4cd0 msgr2=0x7f8e4c1f9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c0b4cd0 0x7f8e4c1f9a50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400d7d0 tx=0x7f8e3400dc90 comp rx=0 tx=0).stop 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 shutdown_connections 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8e280744e0 0x7f8e280769f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e4c144370 0x7f8e4c1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e4c0b4cd0 0x7f8e4c1f9a50 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 --2- 172.21.15.19:0/3342227226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e4c06c5c0 0x7f8e4c137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 >> 172.21.15.19:0/3342227226 conn(0x7f8e4c0d1720 msgr2=0x7f8e4c070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 shutdown_connections 2024-04-25T11:35:15.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.447+0000 7f8e4b200640 1 -- 172.21.15.19:0/3342227226 wait complete. 2024-04-25T11:35:15.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:35:15.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.748+0000 7fcdfa600640 1 Processor -- start 2024-04-25T11:35:15.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.748+0000 7fcdfa600640 1 -- start start 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.750+0000 7fcdfa600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdf40ce520 0x7fcdf415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.750+0000 7fcdfa600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcdf406dce0 0x7fcdf415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.750+0000 7fcdfa600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdf40cda60 con 0x7fcdf40ce930 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.750+0000 7fcdfa600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf415dfe0 con 0x7fcdf40ce520 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.750+0000 7fcdfa600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf415e2c0 con 0x7fcdf406dce0 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.751+0000 7fcdf9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdf40ce520 0x7fcdf415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.751+0000 7fcdf9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcdf40ce520 0x7fcdf415d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36468/0 (socket says 172.21.15.19:36468) 2024-04-25T11:35:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.751+0000 7fcdf9200640 1 -- 172.21.15.19:0/3821515702 learned_addr learned my addr 172.21.15.19:0/3821515702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.751+0000 7fcdf9200640 1 -- 172.21.15.19:0/3821515702 >> v2:172.21.15.169:3300/0 conn(0x7fcdf406dce0 msgr2=0x7fcdf415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.751+0000 7fcdf9200640 1 --2- 172.21.15.19:0/3821515702 >> v2:172.21.15.169:3300/0 conn(0x7fcdf406dce0 0x7fcdf415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.751+0000 7fcdf9200640 1 -- 172.21.15.19:0/3821515702 >> v1:172.21.15.169:6789/0 conn(0x7fcdf40ce930 legacy=0x7fcdf406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.751+0000 7fcdf9200640 1 -- 172.21.15.19:0/3821515702 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf40cdd20 con 0x7fcdf40ce520 2024-04-25T11:35:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.752+0000 7fcdf9200640 1 --2- 172.21.15.19:0/3821515702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce520 0x7fcdf415d4b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcdf415aff0 tx=0x7fcddc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7f327369980a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.753+0000 7fcdf3e00640 1 -- 172.21.15.19:0/3821515702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc037070 con 0x7fcdf40ce520 2024-04-25T11:35:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.753+0000 7fcdf3e00640 1 -- 172.21.15.19:0/3821515702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcddc032440 con 0x7fcdf40ce520 2024-04-25T11:35:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.753+0000 7fcdf3e00640 1 -- 172.21.15.19:0/3821515702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc032810 con 0x7fcdf40ce520 2024-04-25T11:35:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.753+0000 7fcdfa600640 1 -- 172.21.15.19:0/3821515702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce520 msgr2=0x7fcdf415d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.753+0000 7fcdfa600640 1 --2- 172.21.15.19:0/3821515702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce520 0x7fcdf415d4b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcdf415aff0 tx=0x7fcddc02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.754+0000 7fcdfa600640 1 -- 172.21.15.19:0/3821515702 shutdown_connections 2024-04-25T11:35:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.754+0000 7fcdfa600640 1 --2- 172.21.15.19:0/3821515702 >> v2:172.21.15.169:3300/0 conn(0x7fcdf406dce0 0x7fcdf415d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.754+0000 7fcdfa600640 1 --2- 172.21.15.19:0/3821515702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce520 0x7fcdf415d4b0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.754+0000 7fcdfa600640 1 -- 172.21.15.19:0/3821515702 >> 172.21.15.19:0/3821515702 conn(0x7fcdf4072f40 msgr2=0x7fcdf4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.754+0000 7fcdfa600640 1 -- 172.21.15.19:0/3821515702 shutdown_connections 2024-04-25T11:35:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.754+0000 7fcdfa600640 1 -- 172.21.15.19:0/3821515702 wait complete. 2024-04-25T11:35:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.757+0000 7fcdfa600640 1 Processor -- start 2024-04-25T11:35:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.757+0000 7fcdfa600640 1 -- start start 2024-04-25T11:35:15.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf40ce520 0x7fcdf4145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce930 0x7fcdf4145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf41f9b10 0x7fcdf41f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdfa600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf40b73d0 con 0x7fcdf40ce930 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdfa600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf4160d30 con 0x7fcdf41f9b10 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdfa600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf40b7550 con 0x7fcdf40ce520 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce930 0x7fcdf4145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce930 0x7fcdf4145540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36484/0 (socket says 172.21.15.19:36484) 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.759+0000 7fcdebe00640 1 -- 172.21.15.19:0/1561296561 learned_addr learned my addr 172.21.15.19:0/1561296561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdf9200640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf40ce520 0x7fcdf4145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdebe00640 1 -- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf40ce520 msgr2=0x7fcdf4145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdebe00640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf40ce520 0x7fcdf4145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdebe00640 1 -- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf41f9b10 msgr2=0x7fcdf41f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdf9c00640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf41f9b10 0x7fcdf41f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdebe00640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf41f9b10 0x7fcdf41f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdebe00640 1 -- 172.21.15.19:0/1561296561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf41e7ac0 con 0x7fcdf40ce930 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdf9c00640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf41f9b10 0x7fcdf41f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.760+0000 7fcdebe00640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce930 0x7fcdf4145540 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b3e0 tx=0x7fcde400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.761+0000 7fcdf1600640 1 -- 172.21.15.19:0/1561296561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4015020 con 0x7fcdf40ce930 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.761+0000 7fcdf1600640 1 -- 172.21.15.19:0/1561296561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde4011850 con 0x7fcdf40ce930 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.761+0000 7fcdf1600640 1 -- 172.21.15.19:0/1561296561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4011c40 con 0x7fcdf40ce930 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.761+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf41e6e70 con 0x7fcdf40ce930 2024-04-25T11:35:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.761+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf41e7500 con 0x7fcdf40ce930 2024-04-25T11:35:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.762+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf41e6be0 con 0x7fcdf40ce930 2024-04-25T11:35:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.763+0000 7fcdf1600640 1 -- 172.21.15.19:0/1561296561 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcde4018020 con 0x7fcdf40ce930 2024-04-25T11:35:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.764+0000 7fcdf1600640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcdd00744e0 0x7fcdd00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.764+0000 7fcdf1600640 1 -- 172.21.15.19:0/1561296561 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fcde4095200 con 0x7fcdf40ce930 2024-04-25T11:35:15.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.765+0000 7fcdf9200640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcdd00744e0 0x7fcdd00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:15.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.766+0000 7fcdf9200640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcdd00744e0 0x7fcdd00769f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcddc034000 tx=0x7fcddc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:15.769+0000 7fcdf1600640 1 -- 172.21.15.19:0/1561296561 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcde401f020 con 0x7fcdf40ce930 2024-04-25T11:35:16.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.016+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcdf4076c60 con 0x7fcdf40ce930 2024-04-25T11:35:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.447+0000 7fcdf1600640 1 -- 172.21.15.19:0/1561296561 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) ==== 118+0+0 (secure 0 0 0) 0x7fcde405f8f0 con 0x7fcdf40ce930 2024-04-25T11:35:16.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcdd00744e0 msgr2=0x7fcdd00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcdd00744e0 0x7fcdd00769f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcddc034000 tx=0x7fcddc02e000 comp rx=0 tx=0).stop 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce930 msgr2=0x7fcdf4145540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce930 0x7fcdf4145540 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b3e0 tx=0x7fcde400b8a0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 shutdown_connections 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcdd00744e0 0x7fcdd00769f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdf41f9b10 0x7fcdf41f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdf40ce930 0x7fcdf4145540 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 --2- 172.21.15.19:0/1561296561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdf40ce520 0x7fcdf4145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 >> 172.21.15.19:0/1561296561 conn(0x7fcdf4072f40 msgr2=0x7fcdf406d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.451+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 shutdown_connections 2024-04-25T11:35:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.452+0000 7fcdfa600640 1 -- 172.21.15.19:0/1561296561 wait complete. 2024-04-25T11:35:16.474 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:35:16.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:35:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.778+0000 7f1741e00640 1 Processor -- start 2024-04-25T11:35:16.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.778+0000 7f1741e00640 1 -- start start 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f1741e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f1741e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f173c06dce0 0x7f173c070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f1741e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173c164fc0 con 0x7f173c160390 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f1741e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f173c0afb20 con 0x7f173c06dce0 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f1741e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f173c06f440 con 0x7f173c0af710 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f173b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f173c06dce0 0x7f173c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f173b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f173c06dce0 0x7f173c070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36500/0 (socket says 172.21.15.19:36500) 2024-04-25T11:35:16.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f173b400640 1 -- 172.21.15.19:0/3053364528 learned_addr learned my addr 172.21.15.19:0/3053364528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.781+0000 7f173aa00640 1 --2- 172.21.15.19:0/3053364528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.782+0000 7f173b400640 1 -- 172.21.15.19:0/3053364528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 msgr2=0x7f173c164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.782+0000 7f173b400640 1 --2- 172.21.15.19:0/3053364528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c164a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.782+0000 7f173b400640 1 -- 172.21.15.19:0/3053364528 >> v1:172.21.15.169:6789/0 conn(0x7f173c160390 legacy=0x7f173c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:16.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.782+0000 7f173b400640 1 -- 172.21.15.19:0/3053364528 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c06f720 con 0x7f173c06dce0 2024-04-25T11:35:16.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.783+0000 7f173b400640 1 --2- 172.21.15.19:0/3053364528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c06dce0 0x7f173c070270 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1734009e50 tx=0x7f173402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28ea90a1dc0c5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:16.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.783+0000 7f173a000640 1 -- 172.21.15.19:0/3053364528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734036070 con 0x7f173c06dce0 2024-04-25T11:35:16.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.784+0000 7f173a000640 1 -- 172.21.15.19:0/3053364528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1734031600 con 0x7f173c06dce0 2024-04-25T11:35:16.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.784+0000 7f173a000640 1 -- 172.21.15.19:0/3053364528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17340319d0 con 0x7f173c06dce0 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.784+0000 7f1741e00640 1 -- 172.21.15.19:0/3053364528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c06dce0 msgr2=0x7f173c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.785+0000 7f1741e00640 1 --2- 172.21.15.19:0/3053364528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c06dce0 0x7f173c070270 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1734009e50 tx=0x7f173402a680 comp rx=0 tx=0).stop 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.785+0000 7f1741e00640 1 -- 172.21.15.19:0/3053364528 shutdown_connections 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.785+0000 7f1741e00640 1 --2- 172.21.15.19:0/3053364528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c06dce0 0x7f173c070270 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.785+0000 7f1741e00640 1 --2- 172.21.15.19:0/3053364528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.785+0000 7f1741e00640 1 -- 172.21.15.19:0/3053364528 >> 172.21.15.19:0/3053364528 conn(0x7f173c073090 msgr2=0x7f173c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.785+0000 7f1741e00640 1 -- 172.21.15.19:0/3053364528 shutdown_connections 2024-04-25T11:35:16.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.785+0000 7f1741e00640 1 -- 172.21.15.19:0/3053364528 wait complete. 2024-04-25T11:35:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.787+0000 7f1741e00640 1 Processor -- start 2024-04-25T11:35:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.787+0000 7f1741e00640 1 -- start start 2024-04-25T11:35:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f1741e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c14c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f1741e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173c160390 0x7f173c1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f1741e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c1f6b20 0x7f173c1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f1741e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f173c06f0d0 con 0x7f173c1f6b20 2024-04-25T11:35:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f1741e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f173c06ffc0 con 0x7f173c160390 2024-04-25T11:35:16.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f1741e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f173c0b1110 con 0x7f173c0af710 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f173b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c1f6b20 0x7f173c1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.789+0000 7f173b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c1f6b20 0x7f173c1629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36502/0 (socket says 172.21.15.19:36502) 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173b400640 1 -- 172.21.15.19:0/1436856194 learned_addr learned my addr 172.21.15.19:0/1436856194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173aa00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c14c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f1733e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173c160390 0x7f173c1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173b400640 1 -- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 msgr2=0x7f173c14c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173b400640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c14c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173b400640 1 -- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173c160390 msgr2=0x7f173c1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173b400640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173c160390 0x7f173c1f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173b400640 1 -- 172.21.15.19:0/1436856194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c0d0f70 con 0x7f173c1f6b20 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f1733e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173c160390 0x7f173c1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.790+0000 7f173b400640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c1f6b20 0x7f173c1629e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f1734009e50 tx=0x7f1734004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.791+0000 7f1733400640 1 -- 172.21.15.19:0/1436856194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734036050 con 0x7f173c1f6b20 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.791+0000 7f1733400640 1 -- 172.21.15.19:0/1436856194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1734042070 con 0x7f173c1f6b20 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.791+0000 7f1733400640 1 -- 172.21.15.19:0/1436856194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734009510 con 0x7f173c1f6b20 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.791+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173c1f7230 con 0x7f173c1f6b20 2024-04-25T11:35:16.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.791+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c160840 con 0x7f173c1f6b20 2024-04-25T11:35:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.792+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173c050740 con 0x7f173c1f6b20 2024-04-25T11:35:16.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.794+0000 7f1733400640 1 -- 172.21.15.19:0/1436856194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1734007bf0 con 0x7f173c1f6b20 2024-04-25T11:35:16.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.795+0000 7f1733400640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17180744b0 0x7f17180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:16.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.796+0000 7f1733400640 1 -- 172.21.15.19:0/1436856194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f17340b6090 con 0x7f173c1f6b20 2024-04-25T11:35:16.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.796+0000 7f173aa00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17180744b0 0x7f17180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:16.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.798+0000 7f173aa00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17180744b0 0x7f17180769c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f172c009aa0 tx=0x7f172c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:16.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:16.801+0000 7f1733400640 1 -- 172.21.15.19:0/1436856194 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f173c0d0f70 con 0x7f173c1f6b20 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.057+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f173c0b5d40 con 0x7f173c1f6b20 2024-04-25T11:35:17.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.462+0000 7f1733400640 1 -- 172.21.15.19:0/1436856194 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) ==== 122+0+0 (secure 0 0 0) 0x7f1734080780 con 0x7f173c1f6b20 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.466+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17180744b0 msgr2=0x7f17180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.466+0000 7f1741e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17180744b0 0x7f17180769c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f172c009aa0 tx=0x7f172c009190 comp rx=0 tx=0).stop 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.466+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c1f6b20 msgr2=0x7f173c1629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.466+0000 7f1741e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c1f6b20 0x7f173c1629e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f1734009e50 tx=0x7f1734004a40 comp rx=0 tx=0).stop 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 shutdown_connections 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17180744b0 0x7f17180769c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f173c160390 0x7f173c1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173c1f6b20 0x7f173c1629e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 --2- 172.21.15.19:0/1436856194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f173c0af710 0x7f173c14c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 >> 172.21.15.19:0/1436856194 conn(0x7f173c073090 msgr2=0x7f173c0b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:17.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 shutdown_connections 2024-04-25T11:35:17.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.467+0000 7f1741e00640 1 -- 172.21.15.19:0/1436856194 wait complete. 2024-04-25T11:35:17.493 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:35:17.494 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:35:17.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.794+0000 7f75d9400640 1 Processor -- start 2024-04-25T11:35:17.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.794+0000 7f75d9400640 1 -- start start 2024-04-25T11:35:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75d406d560 0x7f75d40b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d9400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d41611e0 con 0x7f75d415a2f0 2024-04-25T11:35:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d9400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f75d4160d50 con 0x7f75d406d560 2024-04-25T11:35:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75d415a700 con 0x7f75d40b6a00 2024-04-25T11:35:17.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75d406d560 0x7f75d40b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75d406d560 0x7f75d40b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36504/0 (socket says 172.21.15.19:36504) 2024-04-25T11:35:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.797+0000 7f75d3200640 1 -- 172.21.15.19:0/2978824808 learned_addr learned my addr 172.21.15.19:0/2978824808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.798+0000 7f75d2800640 1 --2- 172.21.15.19:0/2978824808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.798+0000 7f75d3200640 1 -- 172.21.15.19:0/2978824808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 msgr2=0x7f75d415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.798+0000 7f75d3200640 1 --2- 172.21.15.19:0/2978824808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.798+0000 7f75d3200640 1 -- 172.21.15.19:0/2978824808 >> v1:172.21.15.19:6789/0 conn(0x7f75d415a2f0 legacy=0x7f75d406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:17.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.798+0000 7f75d3200640 1 -- 172.21.15.19:0/2978824808 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d4160420 con 0x7f75d406d560 2024-04-25T11:35:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.799+0000 7f75d3200640 1 --2- 172.21.15.19:0/2978824808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d406d560 0x7f75d40b4970 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009e50 tx=0x7f75cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228cd7fa0f3c0301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.799+0000 7f75d1e00640 1 -- 172.21.15.19:0/2978824808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc036070 con 0x7f75d406d560 2024-04-25T11:35:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.799+0000 7f75d1e00640 1 -- 172.21.15.19:0/2978824808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc031600 con 0x7f75d406d560 2024-04-25T11:35:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.799+0000 7f75d1e00640 1 -- 172.21.15.19:0/2978824808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0319d0 con 0x7f75d406d560 2024-04-25T11:35:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.800+0000 7f75d9400640 1 -- 172.21.15.19:0/2978824808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d406d560 msgr2=0x7f75d40b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.800+0000 7f75d9400640 1 --2- 172.21.15.19:0/2978824808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d406d560 0x7f75d40b4970 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009e50 tx=0x7f75cc02a710 comp rx=0 tx=0).stop 2024-04-25T11:35:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.800+0000 7f75d9400640 1 -- 172.21.15.19:0/2978824808 shutdown_connections 2024-04-25T11:35:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.800+0000 7f75d9400640 1 --2- 172.21.15.19:0/2978824808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d406d560 0x7f75d40b4970 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.800+0000 7f75d9400640 1 --2- 172.21.15.19:0/2978824808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.800+0000 7f75d9400640 1 -- 172.21.15.19:0/2978824808 >> 172.21.15.19:0/2978824808 conn(0x7f75d40d1ae0 msgr2=0x7f75d40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.800+0000 7f75d9400640 1 -- 172.21.15.19:0/2978824808 shutdown_connections 2024-04-25T11:35:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.801+0000 7f75d9400640 1 -- 172.21.15.19:0/2978824808 wait complete. 2024-04-25T11:35:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.803+0000 7f75d9400640 1 Processor -- start 2024-04-25T11:35:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.803+0000 7f75d9400640 1 -- start start 2024-04-25T11:35:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d406d560 0x7f75d41512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d415a2f0 0x7f75d415ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d9400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75d40ce580 con 0x7f75d415a2f0 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d9400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41609e0 con 0x7f75d406d560 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75d40ce700 con 0x7f75d40b6a00 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d415a2f0 0x7f75d415ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d415a2f0 0x7f75d415ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36518/0 (socket says 172.21.15.19:36518) 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 -- 172.21.15.19:0/2650540442 learned_addr learned my addr 172.21.15.19:0/2650540442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d2800640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d406d560 0x7f75d41512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75c9e00640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 -- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 msgr2=0x7f75d4156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d4156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 -- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d406d560 msgr2=0x7f75d41512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d406d560 0x7f75d41512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d3200640 1 -- 172.21.15.19:0/2650540442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d4155890 con 0x7f75d415a2f0 2024-04-25T11:35:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.805+0000 7f75d2800640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d406d560 0x7f75d41512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:17.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.806+0000 7f75d3200640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d415a2f0 0x7f75d415ae10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f75cc034040 tx=0x7f75cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.806+0000 7f75cb400640 1 -- 172.21.15.19:0/2650540442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc036050 con 0x7f75d415a2f0 2024-04-25T11:35:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.806+0000 7f75cb400640 1 -- 172.21.15.19:0/2650540442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc042070 con 0x7f75d415a2f0 2024-04-25T11:35:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.806+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d4155b70 con 0x7f75d415a2f0 2024-04-25T11:35:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.806+0000 7f75cb400640 1 -- 172.21.15.19:0/2650540442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc009510 con 0x7f75d415a2f0 2024-04-25T11:35:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.806+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d4156130 con 0x7f75d415a2f0 2024-04-25T11:35:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.808+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d40afe10 con 0x7f75d415a2f0 2024-04-25T11:35:17.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.809+0000 7f75cb400640 1 -- 172.21.15.19:0/2650540442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f75cc007bd0 con 0x7f75d415a2f0 2024-04-25T11:35:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.810+0000 7f75cb400640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f75b00744e0 0x7f75b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.810+0000 7f75cb400640 1 -- 172.21.15.19:0/2650540442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f75cc0b6160 con 0x7f75d415a2f0 2024-04-25T11:35:17.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.811+0000 7f75d2800640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f75b00744e0 0x7f75b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.812+0000 7f75d2800640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f75b00744e0 0x7f75b00769f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75c4006fd0 tx=0x7f75c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:17.815+0000 7f75cb400640 1 -- 172.21.15.19:0/2650540442 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75cc080850 con 0x7f75d415a2f0 2024-04-25T11:35:18.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.060+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d40705a0 con 0x7f75d415a2f0 2024-04-25T11:35:18.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.062+0000 7f75cb400640 1 -- 172.21.15.19:0/2650540442 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5189 (secure 0 0 0) 0x7f75cc080150 con 0x7f75d415a2f0 2024-04-25T11:35:18.050 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:18.050 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:35:17.059328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2716758396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2716758396},{"type":"v1","addr":"172.21.15.19:6835","nonce":2716758396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7374,"qdb_cluster":[7374]},"id":6}]} 2024-04-25T11:35:18.051 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 90 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.064+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f75b00744e0 msgr2=0x7f75b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.064+0000 7f75d9400640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f75b00744e0 0x7f75b00769f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f75c4006fd0 tx=0x7f75c4008040 comp rx=0 tx=0).stop 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.064+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d415a2f0 msgr2=0x7f75d415ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.064+0000 7f75d9400640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d415a2f0 0x7f75d415ae10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f75cc034040 tx=0x7f75cc004a40 comp rx=0 tx=0).stop 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 shutdown_connections 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f75b00744e0 0x7f75b00769f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75d415a2f0 0x7f75d415ae10 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75d40b6a00 0x7f75d4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 --2- 172.21.15.19:0/2650540442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75d406d560 0x7f75d41512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 >> 172.21.15.19:0/2650540442 conn(0x7f75d40d1ae0 msgr2=0x7f75d40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:18.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 shutdown_connections 2024-04-25T11:35:18.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.065+0000 7f75d9400640 1 -- 172.21.15.19:0/2650540442 wait complete. 2024-04-25T11:35:18.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:35:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.377+0000 7fb18b200640 1 Processor -- start 2024-04-25T11:35:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.377+0000 7fb18b200640 1 -- start start 2024-04-25T11:35:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.379+0000 7fb18b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb1840b4ea0 0x7fb1840cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.379+0000 7fb18b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb184144370 0x7fb18415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.379+0000 7fb18b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1840d1230 con 0x7fb18406c5c0 2024-04-25T11:35:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.379+0000 7fb18b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb1840d06b0 con 0x7fb1840b4ea0 2024-04-25T11:35:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.379+0000 7fb18b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb1840d0990 con 0x7fb184144370 2024-04-25T11:35:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.380+0000 7fb189e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb18406c5c0 0x7fb1840b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35882/0 (socket says 172.21.15.19:35882) 2024-04-25T11:35:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.380+0000 7fb189e00640 1 -- 172.21.15.19:0/3911429076 learned_addr learned my addr 172.21.15.19:0/3911429076 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.380+0000 7fb189400640 1 --2- 172.21.15.19:0/3911429076 >> v2:172.21.15.19:3300/0 conn(0x7fb1840b4ea0 0x7fb1840cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.380+0000 7fb188a00640 1 -- 172.21.15.19:0/3911429076 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 719155172 0 0) 0x7fb1840d1230 con 0x7fb18406c5c0 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.380+0000 7fb188a00640 1 -- 172.21.15.19:0/3911429076 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb16c003520 con 0x7fb18406c5c0 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.380+0000 7fb18a800640 1 --2- 172.21.15.19:0/3911429076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb184144370 0x7fb18415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.381+0000 7fb189400640 1 -- 172.21.15.19:0/3911429076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb184144370 msgr2=0x7fb18415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.381+0000 7fb189400640 1 --2- 172.21.15.19:0/3911429076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb184144370 0x7fb18415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.381+0000 7fb189400640 1 -- 172.21.15.19:0/3911429076 >> v1:172.21.15.19:6789/0 conn(0x7fb18406c5c0 legacy=0x7fb1840b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.381+0000 7fb189400640 1 -- 172.21.15.19:0/3911429076 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1840d0c70 con 0x7fb1840b4ea0 2024-04-25T11:35:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.382+0000 7fb189400640 1 --2- 172.21.15.19:0/3911429076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1840b4ea0 0x7fb1840cfa70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb1740097a0 tx=0x7fb17402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3808938815f9fa16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.382+0000 7fb188a00640 1 -- 172.21.15.19:0/3911429076 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174037070 con 0x7fb1840b4ea0 2024-04-25T11:35:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.382+0000 7fb188a00640 1 -- 172.21.15.19:0/3911429076 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb174032440 con 0x7fb1840b4ea0 2024-04-25T11:35:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.382+0000 7fb188a00640 1 -- 172.21.15.19:0/3911429076 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174032810 con 0x7fb1840b4ea0 2024-04-25T11:35:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 -- 172.21.15.19:0/3911429076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1840b4ea0 msgr2=0x7fb1840cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 --2- 172.21.15.19:0/3911429076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1840b4ea0 0x7fb1840cfa70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb1740097a0 tx=0x7fb17402a180 comp rx=0 tx=0).stop 2024-04-25T11:35:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 -- 172.21.15.19:0/3911429076 shutdown_connections 2024-04-25T11:35:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 --2- 172.21.15.19:0/3911429076 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb184144370 0x7fb18415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 --2- 172.21.15.19:0/3911429076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1840b4ea0 0x7fb1840cfa70 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 -- 172.21.15.19:0/3911429076 >> 172.21.15.19:0/3911429076 conn(0x7fb1840d1720 msgr2=0x7fb1840d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 -- 172.21.15.19:0/3911429076 shutdown_connections 2024-04-25T11:35:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.383+0000 7fb18b200640 1 -- 172.21.15.19:0/3911429076 wait complete. 2024-04-25T11:35:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.385+0000 7fb18b200640 1 Processor -- start 2024-04-25T11:35:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.386+0000 7fb18b200640 1 -- start start 2024-04-25T11:35:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb18b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb18406c5c0 0x7fb1841622b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb18b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb184144370 0x7fb184142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb18b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1840caea0 0x7fb184143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb18b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1840d0450 con 0x7fb184144370 2024-04-25T11:35:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb18b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1840d02d0 con 0x7fb18406c5c0 2024-04-25T11:35:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb18b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb184147760 con 0x7fb1840caea0 2024-04-25T11:35:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb189400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb184144370 0x7fb184142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb189400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb184144370 0x7fb184142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36524/0 (socket says 172.21.15.19:36524) 2024-04-25T11:35:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.387+0000 7fb189400640 1 -- 172.21.15.19:0/4037100207 learned_addr learned my addr 172.21.15.19:0/4037100207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:18.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb18a800640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1840caea0 0x7fb184143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb189e00640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb18406c5c0 0x7fb1841622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb189400640 1 -- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1840caea0 msgr2=0x7fb184143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb189400640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1840caea0 0x7fb184143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb189400640 1 -- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb18406c5c0 msgr2=0x7fb1841622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb189400640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb18406c5c0 0x7fb1841622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb189400640 1 -- 172.21.15.19:0/4037100207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1841e7ce0 con 0x7fb184144370 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.388+0000 7fb189400640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb184144370 0x7fb184142fe0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb174035040 tx=0x7fb1740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.389+0000 7fb17e000640 1 -- 172.21.15.19:0/4037100207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174037040 con 0x7fb184144370 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.389+0000 7fb17e000640 1 -- 172.21.15.19:0/4037100207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb174030500 con 0x7fb184144370 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.389+0000 7fb17e000640 1 -- 172.21.15.19:0/4037100207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174030910 con 0x7fb184144370 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.389+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1841e7fc0 con 0x7fb184144370 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.389+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1841e8580 con 0x7fb184144370 2024-04-25T11:35:18.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.390+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1841e7a50 con 0x7fb184144370 2024-04-25T11:35:18.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.391+0000 7fb17e000640 1 -- 172.21.15.19:0/4037100207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb174044020 con 0x7fb184144370 2024-04-25T11:35:18.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.393+0000 7fb17e000640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1680744e0 0x7fb1680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.393+0000 7fb17e000640 1 -- 172.21.15.19:0/4037100207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fb1740b5390 con 0x7fb184144370 2024-04-25T11:35:18.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.393+0000 7fb189e00640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1680744e0 0x7fb1680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.395+0000 7fb189e00640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1680744e0 0x7fb1680769f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb178005880 tx=0x7fb178009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:18.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.397+0000 7fb17e000640 1 -- 172.21.15.19:0/4037100207 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb17404c060 con 0x7fb184144370 2024-04-25T11:35:18.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.605+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb184050740 con 0x7fb184144370 2024-04-25T11:35:18.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.609+0000 7fb17e000640 1 -- 172.21.15.19:0/4037100207 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16672 (secure 0 0 0) 0x7fb17407fa80 con 0x7fb184144370 2024-04-25T11:35:18.598 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:18.598 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":92,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:35:15.422148+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:35:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1680744e0 msgr2=0x7fb1680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1680744e0 0x7fb1680769f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb178005880 tx=0x7fb178009040 comp rx=0 tx=0).stop 2024-04-25T11:35:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb184144370 msgr2=0x7fb184142fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb184144370 0x7fb184142fe0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb174035040 tx=0x7fb1740046c0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 shutdown_connections 2024-04-25T11:35:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb1680744e0 0x7fb1680769f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1840caea0 0x7fb184143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb184144370 0x7fb184142fe0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 --2- 172.21.15.19:0/4037100207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb18406c5c0 0x7fb1841622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 >> 172.21.15.19:0/4037100207 conn(0x7fb1840d1720 msgr2=0x7fb18406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 shutdown_connections 2024-04-25T11:35:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.612+0000 7fb18b200640 1 -- 172.21.15.19:0/4037100207 wait complete. 2024-04-25T11:35:18.628 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:35:18.628 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:35:18.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.928+0000 7fc155800640 1 Processor -- start 2024-04-25T11:35:18.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.928+0000 7fc155800640 1 -- start start 2024-04-25T11:35:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc155800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1500cf5e0 0x7fc1500cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc155800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc15006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc155800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1500cff30 con 0x7fc1501443c0 2024-04-25T11:35:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc155800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc1500708d0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc155800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc15015f160 con 0x7fc15006c5c0 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc14e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1500cf5e0 0x7fc1500cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc14e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1500cf5e0 0x7fc1500cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36528/0 (socket says 172.21.15.19:36528) 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc14e000640 1 -- 172.21.15.19:0/3343490694 learned_addr learned my addr 172.21.15.19:0/3343490694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.931+0000 7fc14f400640 1 --2- 172.21.15.19:0/3343490694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc15006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.932+0000 7fc14e000640 1 -- 172.21.15.19:0/3343490694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 msgr2=0x7fc15006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.932+0000 7fc14e000640 1 --2- 172.21.15.19:0/3343490694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc15006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.932+0000 7fc14e000640 1 -- 172.21.15.19:0/3343490694 >> v1:172.21.15.169:6789/0 conn(0x7fc1501443c0 legacy=0x7fc15006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.932+0000 7fc14e000640 1 -- 172.21.15.19:0/3343490694 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15015f5f0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.933+0000 7fc14e000640 1 --2- 172.21.15.19:0/3343490694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500cf9f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc138009e50 tx=0x7fc13802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c72fe97718a0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.933+0000 7fc14d600640 1 -- 172.21.15.19:0/3343490694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138036070 con 0x7fc1500cf5e0 2024-04-25T11:35:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.933+0000 7fc14d600640 1 -- 172.21.15.19:0/3343490694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138031600 con 0x7fc1500cf5e0 2024-04-25T11:35:18.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.933+0000 7fc14d600640 1 -- 172.21.15.19:0/3343490694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380319d0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.934+0000 7fc155800640 1 -- 172.21.15.19:0/3343490694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 msgr2=0x7fc1500cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.934+0000 7fc155800640 1 --2- 172.21.15.19:0/3343490694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500cf9f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc138009e50 tx=0x7fc13802a710 comp rx=0 tx=0).stop 2024-04-25T11:35:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.934+0000 7fc155800640 1 -- 172.21.15.19:0/3343490694 shutdown_connections 2024-04-25T11:35:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.934+0000 7fc155800640 1 --2- 172.21.15.19:0/3343490694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc15006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.934+0000 7fc155800640 1 --2- 172.21.15.19:0/3343490694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500cf9f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.934+0000 7fc155800640 1 -- 172.21.15.19:0/3343490694 >> 172.21.15.19:0/3343490694 conn(0x7fc1500d1ae0 msgr2=0x7fc1500d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.935+0000 7fc155800640 1 -- 172.21.15.19:0/3343490694 shutdown_connections 2024-04-25T11:35:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.935+0000 7fc155800640 1 -- 172.21.15.19:0/3343490694 wait complete. 2024-04-25T11:35:18.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.938+0000 7fc155800640 1 Processor -- start 2024-04-25T11:35:18.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.938+0000 7fc155800640 1 -- start start 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.940+0000 7fc155800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc1501432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.940+0000 7fc155800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.940+0000 7fc155800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1501443c0 0x7fc1500ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.940+0000 7fc155800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc15015edf0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.940+0000 7fc155800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc15015eb10 con 0x7fc1501443c0 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.940+0000 7fc155800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc1501447d0 con 0x7fc15006c5c0 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500b6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36536/0 (socket says 172.21.15.19:36536) 2024-04-25T11:35:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 -- 172.21.15.19:0/3783620133 learned_addr learned my addr 172.21.15.19:0/3783620133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14ea00640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc1501432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 -- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 msgr2=0x7fc1501432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14f400640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1501443c0 0x7fc1500ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc1501432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 -- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1501443c0 msgr2=0x7fc1500ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1501443c0 0x7fc1500ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14e000640 1 -- 172.21.15.19:0/3783620133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1500b54b0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14ea00640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc1501432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.941+0000 7fc14f400640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1501443c0 0x7fc1500ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.942+0000 7fc14e000640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500b6380 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc138034040 tx=0x7fc138004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.942+0000 7fc146a00640 1 -- 172.21.15.19:0/3783620133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138036050 con 0x7fc1500cf5e0 2024-04-25T11:35:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.942+0000 7fc146a00640 1 -- 172.21.15.19:0/3783620133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138042070 con 0x7fc1500cf5e0 2024-04-25T11:35:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.942+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1500b59b0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.942+0000 7fc146a00640 1 -- 172.21.15.19:0/3783620133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138009510 con 0x7fc1500cf5e0 2024-04-25T11:35:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.942+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1501ea300 con 0x7fc1500cf5e0 2024-04-25T11:35:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.943+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1500d04b0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.945+0000 7fc146a00640 1 -- 172.21.15.19:0/3783620133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc138007c50 con 0x7fc1500cf5e0 2024-04-25T11:35:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.946+0000 7fc146a00640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc12c0744b0 0x7fc12c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.946+0000 7fc146a00640 1 -- 172.21.15.19:0/3783620133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fc1380b61b0 con 0x7fc1500cf5e0 2024-04-25T11:35:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.947+0000 7fc14ea00640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc12c0744b0 0x7fc12c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.948+0000 7fc14ea00640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc12c0744b0 0x7fc12c0769c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc1500d1860 tx=0x7fc140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:18.952+0000 7fc146a00640 1 -- 172.21.15.19:0/3783620133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1500d04b0 con 0x7fc1500cf5e0 2024-04-25T11:35:19.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.146+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1500b5740 con 0x7fc12c0744b0 2024-04-25T11:35:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.168+0000 7fc146a00640 1 -- 172.21.15.19:0/3783620133 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7fc1500b5740 con 0x7fc12c0744b0 2024-04-25T11:35:19.157 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc12c0744b0 msgr2=0x7fc12c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc12c0744b0 0x7fc12c0769c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc1500d1860 tx=0x7fc140009040 comp rx=0 tx=0).stop 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 msgr2=0x7fc1500b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500b6380 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc138034040 tx=0x7fc138004a40 comp rx=0 tx=0).stop 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 shutdown_connections 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc12c0744b0 0x7fc12c0769c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1501443c0 0x7fc1500ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1500cf5e0 0x7fc1500b6380 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 --2- 172.21.15.19:0/3783620133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc15006c5c0 0x7fc1501432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 >> 172.21.15.19:0/3783620133 conn(0x7fc1500d1ae0 msgr2=0x7fc15015f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:19.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 shutdown_connections 2024-04-25T11:35:19.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.175+0000 7fc155800640 1 -- 172.21.15.19:0/3783620133 wait complete. 2024-04-25T11:35:19.189 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-04-25T11:35:18.638328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259428,"kb_used_data":9980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715292,"statfs":{"total":773094113280,"available":772828459008,"internally_reserved":0,"allocated":10219520,"data_stored":5055568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002308"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671702+0000","last_change":"2024-04-25T11:35:14.437721+0000","last_active":"2024-04-25T11:35:15.671702+0000","last_peered":"2024-04-25T11:35:15.671702+0000","last_clean":"2024-04-25T11:35:15.671702+0000","last_became_active":"2024-04-25T11:35:14.437402+0000","last_became_peered":"2024-04-25T11:35:14.437402+0000","last_unstale":"2024-04-25T11:35:15.671702+0000","last_undegraded":"2024-04-25T11:35:15.671702+0000","last_fullsized":"2024-04-25T11:35:15.671702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:00.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.700506+0000","last_change":"2024-04-25T11:35:14.449924+0000","last_active":"2024-04-25T11:35:15.700506+0000","last_peered":"2024-04-25T11:35:15.700506+0000","last_clean":"2024-04-25T11:35:15.700506+0000","last_became_active":"2024-04-25T11:35:14.449468+0000","last_became_peered":"2024-04-25T11:35:14.449468+0000","last_unstale":"2024-04-25T11:35:15.700506+0000","last_undegraded":"2024-04-25T11:35:15.700506+0000","last_fullsized":"2024-04-25T11:35:15.700506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.822426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441139+0000","last_change":"2024-04-25T11:35:14.424226+0000","last_active":"2024-04-25T11:35:15.441139+0000","last_peered":"2024-04-25T11:35:15.441139+0000","last_clean":"2024-04-25T11:35:15.441139+0000","last_became_active":"2024-04-25T11:35:14.423890+0000","last_became_peered":"2024-04-25T11:35:14.423890+0000","last_unstale":"2024-04-25T11:35:15.441139+0000","last_undegraded":"2024-04-25T11:35:15.441139+0000","last_fullsized":"2024-04-25T11:35:15.441139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.350949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.975380+0000","last_change":"2024-04-25T11:35:14.425627+0000","last_active":"2024-04-25T11:35:15.975380+0000","last_peered":"2024-04-25T11:35:15.975380+0000","last_clean":"2024-04-25T11:35:15.975380+0000","last_became_active":"2024-04-25T11:35:14.424819+0000","last_became_peered":"2024-04-25T11:35:14.424819+0000","last_unstale":"2024-04-25T11:35:15.975380+0000","last_undegraded":"2024-04-25T11:35:15.975380+0000","last_fullsized":"2024-04-25T11:35:15.975380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:32.983110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.439446+0000","last_change":"2024-04-25T11:35:14.445094+0000","last_active":"2024-04-25T11:35:15.439446+0000","last_peered":"2024-04-25T11:35:15.439446+0000","last_clean":"2024-04-25T11:35:15.439446+0000","last_became_active":"2024-04-25T11:35:14.444736+0000","last_became_peered":"2024-04-25T11:35:14.444736+0000","last_unstale":"2024-04-25T11:35:15.439446+0000","last_undegraded":"2024-04-25T11:35:15.439446+0000","last_fullsized":"2024-04-25T11:35:15.439446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:45.259815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440360+0000","last_change":"2024-04-25T11:35:14.426610+0000","last_active":"2024-04-25T11:35:15.440360+0000","last_peered":"2024-04-25T11:35:15.440360+0000","last_clean":"2024-04-25T11:35:15.440360+0000","last_became_active":"2024-04-25T11:35:14.426299+0000","last_became_peered":"2024-04-25T11:35:14.426299+0000","last_unstale":"2024-04-25T11:35:15.440360+0000","last_undegraded":"2024-04-25T11:35:15.440360+0000","last_fullsized":"2024-04-25T11:35:15.440360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:21.772732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699659+0000","last_change":"2024-04-25T11:35:14.446356+0000","last_active":"2024-04-25T11:35:15.699659+0000","last_peered":"2024-04-25T11:35:15.699659+0000","last_clean":"2024-04-25T11:35:15.699659+0000","last_became_active":"2024-04-25T11:35:14.445518+0000","last_became_peered":"2024-04-25T11:35:14.445518+0000","last_unstale":"2024-04-25T11:35:15.699659+0000","last_undegraded":"2024-04-25T11:35:15.699659+0000","last_fullsized":"2024-04-25T11:35:15.699659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:27.241606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973462+0000","last_change":"2024-04-25T11:35:14.448845+0000","last_active":"2024-04-25T11:35:15.973462+0000","last_peered":"2024-04-25T11:35:15.973462+0000","last_clean":"2024-04-25T11:35:15.973462+0000","last_became_active":"2024-04-25T11:35:14.448052+0000","last_became_peered":"2024-04-25T11:35:14.448052+0000","last_unstale":"2024-04-25T11:35:15.973462+0000","last_undegraded":"2024-04-25T11:35:15.973462+0000","last_fullsized":"2024-04-25T11:35:15.973462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:22.633353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672155+0000","last_change":"2024-04-25T11:35:14.442376+0000","last_active":"2024-04-25T11:35:15.672155+0000","last_peered":"2024-04-25T11:35:15.672155+0000","last_clean":"2024-04-25T11:35:15.672155+0000","last_became_active":"2024-04-25T11:35:14.442052+0000","last_became_peered":"2024-04-25T11:35:14.442052+0000","last_unstale":"2024-04-25T11:35:15.672155+0000","last_undegraded":"2024-04-25T11:35:15.672155+0000","last_fullsized":"2024-04-25T11:35:15.672155+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.822426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441188+0000","last_change":"2024-04-25T11:35:14.433269+0000","last_active":"2024-04-25T11:35:15.441188+0000","last_peered":"2024-04-25T11:35:15.441188+0000","last_clean":"2024-04-25T11:35:15.441188+0000","last_became_active":"2024-04-25T11:35:14.432939+0000","last_became_peered":"2024-04-25T11:35:14.432939+0000","last_unstale":"2024-04-25T11:35:15.441188+0000","last_undegraded":"2024-04-25T11:35:15.441188+0000","last_fullsized":"2024-04-25T11:35:15.441188+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:53.280142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669180+0000","last_change":"2024-04-25T11:35:14.452859+0000","last_active":"2024-04-25T11:35:15.669180+0000","last_peered":"2024-04-25T11:35:15.669180+0000","last_clean":"2024-04-25T11:35:15.669180+0000","last_became_active":"2024-04-25T11:35:14.452431+0000","last_became_peered":"2024-04-25T11:35:14.452431+0000","last_unstale":"2024-04-25T11:35:15.669180+0000","last_undegraded":"2024-04-25T11:35:15.669180+0000","last_fullsized":"2024-04-25T11:35:15.669180+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:05.855382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973859+0000","last_change":"2024-04-25T11:35:14.447933+0000","last_active":"2024-04-25T11:35:15.973859+0000","last_peered":"2024-04-25T11:35:15.973859+0000","last_clean":"2024-04-25T11:35:15.973859+0000","last_became_active":"2024-04-25T11:35:14.447574+0000","last_became_peered":"2024-04-25T11:35:14.447574+0000","last_unstale":"2024-04-25T11:35:15.973859+0000","last_undegraded":"2024-04-25T11:35:15.973859+0000","last_fullsized":"2024-04-25T11:35:15.973859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:25.937058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.439499+0000","last_change":"2024-04-25T11:35:14.431316+0000","last_active":"2024-04-25T11:35:15.439499+0000","last_peered":"2024-04-25T11:35:15.439499+0000","last_clean":"2024-04-25T11:35:15.439499+0000","last_became_active":"2024-04-25T11:35:14.429092+0000","last_became_peered":"2024-04-25T11:35:14.429092+0000","last_unstale":"2024-04-25T11:35:15.439499+0000","last_undegraded":"2024-04-25T11:35:15.439499+0000","last_fullsized":"2024-04-25T11:35:15.439499+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:58.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.444791+0000","last_change":"2024-04-25T11:35:14.456382+0000","last_active":"2024-04-25T11:35:15.444791+0000","last_peered":"2024-04-25T11:35:15.444791+0000","last_clean":"2024-04-25T11:35:15.444791+0000","last_became_active":"2024-04-25T11:35:14.456086+0000","last_became_peered":"2024-04-25T11:35:14.456086+0000","last_unstale":"2024-04-25T11:35:15.444791+0000","last_undegraded":"2024-04-25T11:35:15.444791+0000","last_fullsized":"2024-04-25T11:35:15.444791+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:34.847504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699839+0000","last_change":"2024-04-25T11:35:14.446623+0000","last_active":"2024-04-25T11:35:15.699839+0000","last_peered":"2024-04-25T11:35:15.699839+0000","last_clean":"2024-04-25T11:35:15.699839+0000","last_became_active":"2024-04-25T11:35:14.445973+0000","last_became_peered":"2024-04-25T11:35:14.445973+0000","last_unstale":"2024-04-25T11:35:15.699839+0000","last_undegraded":"2024-04-25T11:35:15.699839+0000","last_fullsized":"2024-04-25T11:35:15.699839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:22.633353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440516+0000","last_change":"2024-04-25T11:35:14.434368+0000","last_active":"2024-04-25T11:35:15.440516+0000","last_peered":"2024-04-25T11:35:15.440516+0000","last_clean":"2024-04-25T11:35:15.440516+0000","last_became_active":"2024-04-25T11:35:14.434038+0000","last_became_peered":"2024-04-25T11:35:14.434038+0000","last_unstale":"2024-04-25T11:35:15.440516+0000","last_undegraded":"2024-04-25T11:35:15.440516+0000","last_fullsized":"2024-04-25T11:35:15.440516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:41.353736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974397+0000","last_change":"2024-04-25T11:35:14.447167+0000","last_active":"2024-04-25T11:35:15.974397+0000","last_peered":"2024-04-25T11:35:15.974397+0000","last_clean":"2024-04-25T11:35:15.974397+0000","last_became_active":"2024-04-25T11:35:14.446570+0000","last_became_peered":"2024-04-25T11:35:14.446570+0000","last_unstale":"2024-04-25T11:35:15.974397+0000","last_undegraded":"2024-04-25T11:35:15.974397+0000","last_fullsized":"2024-04-25T11:35:15.974397+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:47.735930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669389+0000","last_change":"2024-04-25T11:35:14.457086+0000","last_active":"2024-04-25T11:35:15.669389+0000","last_peered":"2024-04-25T11:35:15.669389+0000","last_clean":"2024-04-25T11:35:15.669389+0000","last_became_active":"2024-04-25T11:35:14.456750+0000","last_became_peered":"2024-04-25T11:35:14.456750+0000","last_unstale":"2024-04-25T11:35:15.669389+0000","last_undegraded":"2024-04-25T11:35:15.669389+0000","last_fullsized":"2024-04-25T11:35:15.669389+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:02.357995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.444255+0000","last_change":"2024-04-25T11:35:14.455058+0000","last_active":"2024-04-25T11:35:15.444255+0000","last_peered":"2024-04-25T11:35:15.444255+0000","last_clean":"2024-04-25T11:35:15.444255+0000","last_became_active":"2024-04-25T11:35:14.454688+0000","last_became_peered":"2024-04-25T11:35:14.454688+0000","last_unstale":"2024-04-25T11:35:15.444255+0000","last_undegraded":"2024-04-25T11:35:15.444255+0000","last_fullsized":"2024-04-25T11:35:15.444255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:05.855382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443783+0000","last_change":"2024-04-25T11:35:14.460394+0000","last_active":"2024-04-25T11:35:15.443783+0000","last_peered":"2024-04-25T11:35:15.443783+0000","last_clean":"2024-04-25T11:35:15.443783+0000","last_became_active":"2024-04-25T11:35:14.458496+0000","last_became_peered":"2024-04-25T11:35:14.458496+0000","last_unstale":"2024-04-25T11:35:15.443783+0000","last_undegraded":"2024-04-25T11:35:15.443783+0000","last_fullsized":"2024-04-25T11:35:15.443783+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:25.814645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974603+0000","last_change":"2024-04-25T11:35:14.448650+0000","last_active":"2024-04-25T11:35:15.974603+0000","last_peered":"2024-04-25T11:35:15.974603+0000","last_clean":"2024-04-25T11:35:15.974603+0000","last_became_active":"2024-04-25T11:35:14.448330+0000","last_became_peered":"2024-04-25T11:35:14.448330+0000","last_unstale":"2024-04-25T11:35:15.974603+0000","last_undegraded":"2024-04-25T11:35:15.974603+0000","last_fullsized":"2024-04-25T11:35:15.974603+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:40.477803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443508+0000","last_change":"2024-04-25T11:35:14.461184+0000","last_active":"2024-04-25T11:35:15.443508+0000","last_peered":"2024-04-25T11:35:15.443508+0000","last_clean":"2024-04-25T11:35:15.443508+0000","last_became_active":"2024-04-25T11:35:14.460843+0000","last_became_peered":"2024-04-25T11:35:14.460843+0000","last_unstale":"2024-04-25T11:35:15.443508+0000","last_undegraded":"2024-04-25T11:35:15.443508+0000","last_fullsized":"2024-04-25T11:35:15.443508+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:26.586033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973497+0000","last_change":"2024-04-25T11:35:14.436174+0000","last_active":"2024-04-25T11:35:15.973497+0000","last_peered":"2024-04-25T11:35:15.973497+0000","last_clean":"2024-04-25T11:35:15.973497+0000","last_became_active":"2024-04-25T11:35:14.435858+0000","last_became_peered":"2024-04-25T11:35:14.435858+0000","last_unstale":"2024-04-25T11:35:15.973497+0000","last_undegraded":"2024-04-25T11:35:15.973497+0000","last_fullsized":"2024-04-25T11:35:15.973497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.822426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668524+0000","last_change":"2024-04-25T11:35:14.458906+0000","last_active":"2024-04-25T11:35:15.668524+0000","last_peered":"2024-04-25T11:35:15.668524+0000","last_clean":"2024-04-25T11:35:15.668524+0000","last_became_active":"2024-04-25T11:35:14.457429+0000","last_became_peered":"2024-04-25T11:35:14.457429+0000","last_unstale":"2024-04-25T11:35:15.668524+0000","last_undegraded":"2024-04-25T11:35:15.668524+0000","last_fullsized":"2024-04-25T11:35:15.668524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:26.586033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669171+0000","last_change":"2024-04-25T11:35:14.448530+0000","last_active":"2024-04-25T11:35:15.669171+0000","last_peered":"2024-04-25T11:35:15.669171+0000","last_clean":"2024-04-25T11:35:15.669171+0000","last_became_active":"2024-04-25T11:35:14.448220+0000","last_became_peered":"2024-04-25T11:35:14.448220+0000","last_unstale":"2024-04-25T11:35:15.669171+0000","last_undegraded":"2024-04-25T11:35:15.669171+0000","last_fullsized":"2024-04-25T11:35:15.669171+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:34.847504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442046+0000","last_change":"2024-04-25T11:35:14.450485+0000","last_active":"2024-04-25T11:35:15.442046+0000","last_peered":"2024-04-25T11:35:15.442046+0000","last_clean":"2024-04-25T11:35:15.442046+0000","last_became_active":"2024-04-25T11:35:14.450082+0000","last_became_peered":"2024-04-25T11:35:14.450082+0000","last_unstale":"2024-04-25T11:35:15.442046+0000","last_undegraded":"2024-04-25T11:35:15.442046+0000","last_fullsized":"2024-04-25T11:35:15.442046+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:17.516432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441718+0000","last_change":"2024-04-25T11:35:14.426613+0000","last_active":"2024-04-25T11:35:15.441718+0000","last_peered":"2024-04-25T11:35:15.441718+0000","last_clean":"2024-04-25T11:35:15.441718+0000","last_became_active":"2024-04-25T11:35:14.426304+0000","last_became_peered":"2024-04-25T11:35:14.426304+0000","last_unstale":"2024-04-25T11:35:15.441718+0000","last_undegraded":"2024-04-25T11:35:15.441718+0000","last_fullsized":"2024-04-25T11:35:15.441718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:34.975141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671238+0000","last_change":"2024-04-25T11:35:14.429109+0000","last_active":"2024-04-25T11:35:15.671238+0000","last_peered":"2024-04-25T11:35:15.671238+0000","last_clean":"2024-04-25T11:35:15.671238+0000","last_became_active":"2024-04-25T11:35:14.428779+0000","last_became_peered":"2024-04-25T11:35:14.428779+0000","last_unstale":"2024-04-25T11:35:15.671238+0000","last_undegraded":"2024-04-25T11:35:15.671238+0000","last_fullsized":"2024-04-25T11:35:15.671238+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:34.847504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974540+0000","last_change":"2024-04-25T11:35:14.443789+0000","last_active":"2024-04-25T11:35:15.974540+0000","last_peered":"2024-04-25T11:35:15.974540+0000","last_clean":"2024-04-25T11:35:15.974540+0000","last_became_active":"2024-04-25T11:35:14.439806+0000","last_became_peered":"2024-04-25T11:35:14.439806+0000","last_unstale":"2024-04-25T11:35:15.974540+0000","last_undegraded":"2024-04-25T11:35:15.974540+0000","last_fullsized":"2024-04-25T11:35:15.974540+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:42.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443986+0000","last_change":"2024-04-25T11:35:14.457363+0000","last_active":"2024-04-25T11:35:15.443986+0000","last_peered":"2024-04-25T11:35:15.443986+0000","last_clean":"2024-04-25T11:35:15.443986+0000","last_became_active":"2024-04-25T11:35:14.457035+0000","last_became_peered":"2024-04-25T11:35:14.457035+0000","last_unstale":"2024-04-25T11:35:15.443986+0000","last_undegraded":"2024-04-25T11:35:15.443986+0000","last_fullsized":"2024-04-25T11:35:15.443986+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:16.722343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440203+0000","last_change":"2024-04-25T11:35:14.434057+0000","last_active":"2024-04-25T11:35:15.440203+0000","last_peered":"2024-04-25T11:35:15.440203+0000","last_clean":"2024-04-25T11:35:15.440203+0000","last_became_active":"2024-04-25T11:35:14.429795+0000","last_became_peered":"2024-04-25T11:35:14.429795+0000","last_unstale":"2024-04-25T11:35:15.440203+0000","last_undegraded":"2024-04-25T11:35:15.440203+0000","last_fullsized":"2024-04-25T11:35:15.440203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:48.823179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440331+0000","last_change":"2024-04-25T11:35:14.436818+0000","last_active":"2024-04-25T11:35:15.440331+0000","last_peered":"2024-04-25T11:35:15.440331+0000","last_clean":"2024-04-25T11:35:15.440331+0000","last_became_active":"2024-04-25T11:35:14.435526+0000","last_became_peered":"2024-04-25T11:35:14.435526+0000","last_unstale":"2024-04-25T11:35:15.440331+0000","last_undegraded":"2024-04-25T11:35:15.440331+0000","last_fullsized":"2024-04-25T11:35:15.440331+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.415414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974007+0000","last_change":"2024-04-25T11:35:14.434385+0000","last_active":"2024-04-25T11:35:15.974007+0000","last_peered":"2024-04-25T11:35:15.974007+0000","last_clean":"2024-04-25T11:35:15.974007+0000","last_became_active":"2024-04-25T11:35:14.434075+0000","last_became_peered":"2024-04-25T11:35:14.434075+0000","last_unstale":"2024-04-25T11:35:15.974007+0000","last_undegraded":"2024-04-25T11:35:15.974007+0000","last_fullsized":"2024-04-25T11:35:15.974007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:31.037258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442045+0000","last_change":"2024-04-25T11:35:13.437396+0000","last_active":"2024-04-25T11:35:15.442045+0000","last_peered":"2024-04-25T11:35:15.442045+0000","last_clean":"2024-04-25T11:35:15.442045+0000","last_became_active":"2024-04-25T11:35:13.434939+0000","last_became_peered":"2024-04-25T11:35:13.434939+0000","last_unstale":"2024-04-25T11:35:15.442045+0000","last_undegraded":"2024-04-25T11:35:15.442045+0000","last_fullsized":"2024-04-25T11:35:15.442045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:45.138962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672019+0000","last_change":"2024-04-25T11:35:14.437779+0000","last_active":"2024-04-25T11:35:15.672019+0000","last_peered":"2024-04-25T11:35:15.672019+0000","last_clean":"2024-04-25T11:35:15.672019+0000","last_became_active":"2024-04-25T11:35:14.437486+0000","last_became_peered":"2024-04-25T11:35:14.437486+0000","last_unstale":"2024-04-25T11:35:15.672019+0000","last_undegraded":"2024-04-25T11:35:15.672019+0000","last_fullsized":"2024-04-25T11:35:15.672019+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:22.633353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441254+0000","last_change":"2024-04-25T11:35:13.427281+0000","last_active":"2024-04-25T11:35:15.441254+0000","last_peered":"2024-04-25T11:35:15.441254+0000","last_clean":"2024-04-25T11:35:15.441254+0000","last_became_active":"2024-04-25T11:35:13.426949+0000","last_became_peered":"2024-04-25T11:35:13.426949+0000","last_unstale":"2024-04-25T11:35:15.441254+0000","last_undegraded":"2024-04-25T11:35:15.441254+0000","last_fullsized":"2024-04-25T11:35:15.441254+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:32.416300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974260+0000","last_change":"2024-04-25T11:35:14.446064+0000","last_active":"2024-04-25T11:35:15.974260+0000","last_peered":"2024-04-25T11:35:15.974260+0000","last_clean":"2024-04-25T11:35:15.974260+0000","last_became_active":"2024-04-25T11:35:14.445687+0000","last_became_peered":"2024-04-25T11:35:14.445687+0000","last_unstale":"2024-04-25T11:35:15.974260+0000","last_undegraded":"2024-04-25T11:35:15.974260+0000","last_fullsized":"2024-04-25T11:35:15.974260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:06.227086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669047+0000","last_change":"2024-04-25T11:35:13.430166+0000","last_active":"2024-04-25T11:35:15.669047+0000","last_peered":"2024-04-25T11:35:15.669047+0000","last_clean":"2024-04-25T11:35:15.669047+0000","last_became_active":"2024-04-25T11:35:13.426917+0000","last_became_peered":"2024-04-25T11:35:13.426917+0000","last_unstale":"2024-04-25T11:35:15.669047+0000","last_undegraded":"2024-04-25T11:35:15.669047+0000","last_fullsized":"2024-04-25T11:35:15.669047+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:12.994234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671141+0000","last_change":"2024-04-25T11:35:14.456199+0000","last_active":"2024-04-25T11:35:15.671141+0000","last_peered":"2024-04-25T11:35:15.671141+0000","last_clean":"2024-04-25T11:35:15.671141+0000","last_became_active":"2024-04-25T11:35:14.455892+0000","last_became_peered":"2024-04-25T11:35:14.455892+0000","last_unstale":"2024-04-25T11:35:15.671141+0000","last_undegraded":"2024-04-25T11:35:15.671141+0000","last_fullsized":"2024-04-25T11:35:15.671141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:05.855382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.975280+0000","last_change":"2024-04-25T11:35:13.430002+0000","last_active":"2024-04-25T11:35:15.975280+0000","last_peered":"2024-04-25T11:35:15.975280+0000","last_clean":"2024-04-25T11:35:15.975280+0000","last_became_active":"2024-04-25T11:35:13.427079+0000","last_became_peered":"2024-04-25T11:35:13.427079+0000","last_unstale":"2024-04-25T11:35:15.975280+0000","last_undegraded":"2024-04-25T11:35:15.975280+0000","last_fullsized":"2024-04-25T11:35:15.975280+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.387249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.975389+0000","last_change":"2024-04-25T11:35:14.443427+0000","last_active":"2024-04-25T11:35:15.975389+0000","last_peered":"2024-04-25T11:35:15.975389+0000","last_clean":"2024-04-25T11:35:15.975389+0000","last_became_active":"2024-04-25T11:35:14.442447+0000","last_became_peered":"2024-04-25T11:35:14.442447+0000","last_unstale":"2024-04-25T11:35:15.975389+0000","last_undegraded":"2024-04-25T11:35:15.975389+0000","last_fullsized":"2024-04-25T11:35:15.975389+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:21.444901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973317+0000","last_change":"2024-04-25T11:35:13.433743+0000","last_active":"2024-04-25T11:35:15.973317+0000","last_peered":"2024-04-25T11:35:15.973317+0000","last_clean":"2024-04-25T11:35:15.973317+0000","last_became_active":"2024-04-25T11:35:13.432855+0000","last_became_peered":"2024-04-25T11:35:13.432855+0000","last_unstale":"2024-04-25T11:35:15.973317+0000","last_undegraded":"2024-04-25T11:35:15.973317+0000","last_fullsized":"2024-04-25T11:35:15.973317+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:02.978409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440841+0000","last_change":"2024-04-25T11:35:14.447155+0000","last_active":"2024-04-25T11:35:15.440841+0000","last_peered":"2024-04-25T11:35:15.440841+0000","last_clean":"2024-04-25T11:35:15.440841+0000","last_became_active":"2024-04-25T11:35:14.446489+0000","last_became_peered":"2024-04-25T11:35:14.446489+0000","last_unstale":"2024-04-25T11:35:15.440841+0000","last_undegraded":"2024-04-25T11:35:15.440841+0000","last_fullsized":"2024-04-25T11:35:15.440841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:18.826261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440440+0000","last_change":"2024-04-25T11:35:13.420146+0000","last_active":"2024-04-25T11:35:15.440440+0000","last_peered":"2024-04-25T11:35:15.440440+0000","last_clean":"2024-04-25T11:35:15.440440+0000","last_became_active":"2024-04-25T11:35:13.419810+0000","last_became_peered":"2024-04-25T11:35:13.419810+0000","last_unstale":"2024-04-25T11:35:15.440440+0000","last_undegraded":"2024-04-25T11:35:15.440440+0000","last_fullsized":"2024-04-25T11:35:15.440440+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:59.388917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671381+0000","last_change":"2024-04-25T11:35:14.457536+0000","last_active":"2024-04-25T11:35:15.671381+0000","last_peered":"2024-04-25T11:35:15.671381+0000","last_clean":"2024-04-25T11:35:15.671381+0000","last_became_active":"2024-04-25T11:35:14.457137+0000","last_became_peered":"2024-04-25T11:35:14.457137+0000","last_unstale":"2024-04-25T11:35:15.671381+0000","last_undegraded":"2024-04-25T11:35:15.671381+0000","last_fullsized":"2024-04-25T11:35:15.671381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:27.241606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442805+0000","last_change":"2024-04-25T11:35:13.434918+0000","last_active":"2024-04-25T11:35:15.442805+0000","last_peered":"2024-04-25T11:35:15.442805+0000","last_clean":"2024-04-25T11:35:15.442805+0000","last_became_active":"2024-04-25T11:35:13.434553+0000","last_became_peered":"2024-04-25T11:35:13.434553+0000","last_unstale":"2024-04-25T11:35:15.442805+0000","last_undegraded":"2024-04-25T11:35:15.442805+0000","last_fullsized":"2024-04-25T11:35:15.442805+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:52.649863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442631+0000","last_change":"2024-04-25T11:35:14.444864+0000","last_active":"2024-04-25T11:35:15.442631+0000","last_peered":"2024-04-25T11:35:15.442631+0000","last_clean":"2024-04-25T11:35:15.442631+0000","last_became_active":"2024-04-25T11:35:14.444515+0000","last_became_peered":"2024-04-25T11:35:14.444515+0000","last_unstale":"2024-04-25T11:35:15.442631+0000","last_undegraded":"2024-04-25T11:35:15.442631+0000","last_fullsized":"2024-04-25T11:35:15.442631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:47.831889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442190+0000","last_change":"2024-04-25T11:35:13.437418+0000","last_active":"2024-04-25T11:35:15.442190+0000","last_peered":"2024-04-25T11:35:15.442190+0000","last_clean":"2024-04-25T11:35:15.442190+0000","last_became_active":"2024-04-25T11:35:13.436540+0000","last_became_peered":"2024-04-25T11:35:13.436540+0000","last_unstale":"2024-04-25T11:35:15.442190+0000","last_undegraded":"2024-04-25T11:35:15.442190+0000","last_fullsized":"2024-04-25T11:35:15.442190+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:09.768799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443072+0000","last_change":"2024-04-25T11:35:14.454311+0000","last_active":"2024-04-25T11:35:15.443072+0000","last_peered":"2024-04-25T11:35:15.443072+0000","last_clean":"2024-04-25T11:35:15.443072+0000","last_became_active":"2024-04-25T11:35:14.453982+0000","last_became_peered":"2024-04-25T11:35:14.453982+0000","last_unstale":"2024-04-25T11:35:15.443072+0000","last_undegraded":"2024-04-25T11:35:15.443072+0000","last_fullsized":"2024-04-25T11:35:15.443072+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:14.552023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672265+0000","last_change":"2024-04-25T11:35:13.426296+0000","last_active":"2024-04-25T11:35:15.672265+0000","last_peered":"2024-04-25T11:35:15.672265+0000","last_clean":"2024-04-25T11:35:15.672265+0000","last_became_active":"2024-04-25T11:35:13.424459+0000","last_became_peered":"2024-04-25T11:35:13.424459+0000","last_unstale":"2024-04-25T11:35:15.672265+0000","last_undegraded":"2024-04-25T11:35:15.672265+0000","last_fullsized":"2024-04-25T11:35:15.672265+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:26.930066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672261+0000","last_change":"2024-04-25T11:35:14.456394+0000","last_active":"2024-04-25T11:35:15.672261+0000","last_peered":"2024-04-25T11:35:15.672261+0000","last_clean":"2024-04-25T11:35:15.672261+0000","last_became_active":"2024-04-25T11:35:14.455379+0000","last_became_peered":"2024-04-25T11:35:14.455379+0000","last_unstale":"2024-04-25T11:35:15.672261+0000","last_undegraded":"2024-04-25T11:35:15.672261+0000","last_fullsized":"2024-04-25T11:35:15.672261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:33.434494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441263+0000","last_change":"2024-04-25T11:35:13.432537+0000","last_active":"2024-04-25T11:35:15.441263+0000","last_peered":"2024-04-25T11:35:15.441263+0000","last_clean":"2024-04-25T11:35:15.441263+0000","last_became_active":"2024-04-25T11:35:13.431989+0000","last_became_peered":"2024-04-25T11:35:13.431989+0000","last_unstale":"2024-04-25T11:35:15.441263+0000","last_undegraded":"2024-04-25T11:35:15.441263+0000","last_fullsized":"2024-04-25T11:35:15.441263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:24.918864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669184+0000","last_change":"2024-04-25T11:35:14.451867+0000","last_active":"2024-04-25T11:35:15.669184+0000","last_peered":"2024-04-25T11:35:15.669184+0000","last_clean":"2024-04-25T11:35:15.669184+0000","last_became_active":"2024-04-25T11:35:14.451535+0000","last_became_peered":"2024-04-25T11:35:14.451535+0000","last_unstale":"2024-04-25T11:35:15.669184+0000","last_undegraded":"2024-04-25T11:35:15.669184+0000","last_fullsized":"2024-04-25T11:35:15.669184+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:10.665968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671330+0000","last_change":"2024-04-25T11:35:13.428207+0000","last_active":"2024-04-25T11:35:15.671330+0000","last_peered":"2024-04-25T11:35:15.671330+0000","last_clean":"2024-04-25T11:35:15.671330+0000","last_became_active":"2024-04-25T11:35:13.427902+0000","last_became_peered":"2024-04-25T11:35:13.427902+0000","last_unstale":"2024-04-25T11:35:15.671330+0000","last_undegraded":"2024-04-25T11:35:15.671330+0000","last_fullsized":"2024-04-25T11:35:15.671330+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.156013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441320+0000","last_change":"2024-04-25T11:35:14.454313+0000","last_active":"2024-04-25T11:35:15.441320+0000","last_peered":"2024-04-25T11:35:15.441320+0000","last_clean":"2024-04-25T11:35:15.441320+0000","last_became_active":"2024-04-25T11:35:14.453891+0000","last_became_peered":"2024-04-25T11:35:14.453891+0000","last_unstale":"2024-04-25T11:35:15.441320+0000","last_undegraded":"2024-04-25T11:35:15.441320+0000","last_fullsized":"2024-04-25T11:35:15.441320+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:09.121728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974562+0000","last_change":"2024-04-25T11:35:13.437399+0000","last_active":"2024-04-25T11:35:15.974562+0000","last_peered":"2024-04-25T11:35:15.974562+0000","last_clean":"2024-04-25T11:35:15.974562+0000","last_became_active":"2024-04-25T11:35:13.436580+0000","last_became_peered":"2024-04-25T11:35:13.436580+0000","last_unstale":"2024-04-25T11:35:15.974562+0000","last_undegraded":"2024-04-25T11:35:15.974562+0000","last_fullsized":"2024-04-25T11:35:15.974562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.567839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668510+0000","last_change":"2024-04-25T11:35:14.447867+0000","last_active":"2024-04-25T11:35:15.668510+0000","last_peered":"2024-04-25T11:35:15.668510+0000","last_clean":"2024-04-25T11:35:15.668510+0000","last_became_active":"2024-04-25T11:35:14.447416+0000","last_became_peered":"2024-04-25T11:35:14.447416+0000","last_unstale":"2024-04-25T11:35:15.668510+0000","last_undegraded":"2024-04-25T11:35:15.668510+0000","last_fullsized":"2024-04-25T11:35:15.668510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:14.552023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974873+0000","last_change":"2024-04-25T11:35:13.428944+0000","last_active":"2024-04-25T11:35:15.974873+0000","last_peered":"2024-04-25T11:35:15.974873+0000","last_clean":"2024-04-25T11:35:15.974873+0000","last_became_active":"2024-04-25T11:35:13.428595+0000","last_became_peered":"2024-04-25T11:35:13.428595+0000","last_unstale":"2024-04-25T11:35:15.974873+0000","last_undegraded":"2024-04-25T11:35:15.974873+0000","last_fullsized":"2024-04-25T11:35:15.974873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:05.329123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671365+0000","last_change":"2024-04-25T11:35:14.444313+0000","last_active":"2024-04-25T11:35:15.671365+0000","last_peered":"2024-04-25T11:35:15.671365+0000","last_clean":"2024-04-25T11:35:15.671365+0000","last_became_active":"2024-04-25T11:35:14.443696+0000","last_became_peered":"2024-04-25T11:35:14.443696+0000","last_unstale":"2024-04-25T11:35:15.671365+0000","last_undegraded":"2024-04-25T11:35:15.671365+0000","last_fullsized":"2024-04-25T11:35:15.671365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:02.357995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699623+0000","last_change":"2024-04-25T11:35:13.435321+0000","last_active":"2024-04-25T11:35:15.699623+0000","last_peered":"2024-04-25T11:35:15.699623+0000","last_clean":"2024-04-25T11:35:15.699623+0000","last_became_active":"2024-04-25T11:35:13.434947+0000","last_became_peered":"2024-04-25T11:35:13.434947+0000","last_unstale":"2024-04-25T11:35:15.699623+0000","last_undegraded":"2024-04-25T11:35:15.699623+0000","last_fullsized":"2024-04-25T11:35:15.699623+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:13.533829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442549+0000","last_change":"2024-04-25T11:35:14.458234+0000","last_active":"2024-04-25T11:35:15.442549+0000","last_peered":"2024-04-25T11:35:15.442549+0000","last_clean":"2024-04-25T11:35:15.442549+0000","last_became_active":"2024-04-25T11:35:14.457944+0000","last_became_peered":"2024-04-25T11:35:14.457944+0000","last_unstale":"2024-04-25T11:35:15.442549+0000","last_undegraded":"2024-04-25T11:35:15.442549+0000","last_fullsized":"2024-04-25T11:35:15.442549+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:03.996603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440371+0000","last_change":"2024-04-25T11:35:13.426786+0000","last_active":"2024-04-25T11:35:15.440371+0000","last_peered":"2024-04-25T11:35:15.440371+0000","last_clean":"2024-04-25T11:35:15.440371+0000","last_became_active":"2024-04-25T11:35:13.424232+0000","last_became_peered":"2024-04-25T11:35:13.424232+0000","last_unstale":"2024-04-25T11:35:15.440371+0000","last_undegraded":"2024-04-25T11:35:15.440371+0000","last_fullsized":"2024-04-25T11:35:15.440371+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:26.223412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668021+0000","last_change":"2024-04-25T11:35:14.458052+0000","last_active":"2024-04-25T11:35:15.668021+0000","last_peered":"2024-04-25T11:35:15.668021+0000","last_clean":"2024-04-25T11:35:15.668021+0000","last_became_active":"2024-04-25T11:35:14.455359+0000","last_became_peered":"2024-04-25T11:35:14.455359+0000","last_unstale":"2024-04-25T11:35:15.668021+0000","last_undegraded":"2024-04-25T11:35:15.668021+0000","last_fullsized":"2024-04-25T11:35:15.668021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:47.831889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668091+0000","last_change":"2024-04-25T11:35:13.435283+0000","last_active":"2024-04-25T11:35:15.668091+0000","last_peered":"2024-04-25T11:35:15.668091+0000","last_clean":"2024-04-25T11:35:15.668091+0000","last_became_active":"2024-04-25T11:35:13.434976+0000","last_became_peered":"2024-04-25T11:35:13.434976+0000","last_unstale":"2024-04-25T11:35:15.668091+0000","last_undegraded":"2024-04-25T11:35:15.668091+0000","last_fullsized":"2024-04-25T11:35:15.668091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:52.649863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973753+0000","last_change":"2024-04-25T11:35:14.438682+0000","last_active":"2024-04-25T11:35:15.973753+0000","last_peered":"2024-04-25T11:35:15.973753+0000","last_clean":"2024-04-25T11:35:15.973753+0000","last_became_active":"2024-04-25T11:35:14.438334+0000","last_became_peered":"2024-04-25T11:35:14.438334+0000","last_unstale":"2024-04-25T11:35:15.973753+0000","last_undegraded":"2024-04-25T11:35:15.973753+0000","last_fullsized":"2024-04-25T11:35:15.973753+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:33.434494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442117+0000","last_change":"2024-04-25T11:35:13.437779+0000","last_active":"2024-04-25T11:35:15.442117+0000","last_peered":"2024-04-25T11:35:15.442117+0000","last_clean":"2024-04-25T11:35:15.442117+0000","last_became_active":"2024-04-25T11:35:13.437350+0000","last_became_peered":"2024-04-25T11:35:13.437350+0000","last_unstale":"2024-04-25T11:35:15.442117+0000","last_undegraded":"2024-04-25T11:35:15.442117+0000","last_fullsized":"2024-04-25T11:35:15.442117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.619380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441580+0000","last_change":"2024-04-25T11:35:14.435207+0000","last_active":"2024-04-25T11:35:15.441580+0000","last_peered":"2024-04-25T11:35:15.441580+0000","last_clean":"2024-04-25T11:35:15.441580+0000","last_became_active":"2024-04-25T11:35:14.434485+0000","last_became_peered":"2024-04-25T11:35:14.434485+0000","last_unstale":"2024-04-25T11:35:15.441580+0000","last_undegraded":"2024-04-25T11:35:15.441580+0000","last_fullsized":"2024-04-25T11:35:15.441580+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.524923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443505+0000","last_change":"2024-04-25T11:35:13.436647+0000","last_active":"2024-04-25T11:35:15.443505+0000","last_peered":"2024-04-25T11:35:15.443505+0000","last_clean":"2024-04-25T11:35:15.443505+0000","last_became_active":"2024-04-25T11:35:13.436262+0000","last_became_peered":"2024-04-25T11:35:13.436262+0000","last_unstale":"2024-04-25T11:35:15.443505+0000","last_undegraded":"2024-04-25T11:35:15.443505+0000","last_fullsized":"2024-04-25T11:35:15.443505+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:14.404919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974374+0000","last_change":"2024-04-25T11:35:14.439044+0000","last_active":"2024-04-25T11:35:15.974374+0000","last_peered":"2024-04-25T11:35:15.974374+0000","last_clean":"2024-04-25T11:35:15.974374+0000","last_became_active":"2024-04-25T11:35:14.438731+0000","last_became_peered":"2024-04-25T11:35:14.438731+0000","last_unstale":"2024-04-25T11:35:15.974374+0000","last_undegraded":"2024-04-25T11:35:15.974374+0000","last_fullsized":"2024-04-25T11:35:15.974374+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:46.184173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.444116+0000","last_change":"2024-04-25T11:35:13.436802+0000","last_active":"2024-04-25T11:35:15.444116+0000","last_peered":"2024-04-25T11:35:15.444116+0000","last_clean":"2024-04-25T11:35:15.444116+0000","last_became_active":"2024-04-25T11:35:13.436423+0000","last_became_peered":"2024-04-25T11:35:13.436423+0000","last_unstale":"2024-04-25T11:35:15.444116+0000","last_undegraded":"2024-04-25T11:35:15.444116+0000","last_fullsized":"2024-04-25T11:35:15.444116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.058527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699717+0000","last_change":"2024-04-25T11:35:14.457201+0000","last_active":"2024-04-25T11:35:15.699717+0000","last_peered":"2024-04-25T11:35:15.699717+0000","last_clean":"2024-04-25T11:35:15.699717+0000","last_became_active":"2024-04-25T11:35:14.456824+0000","last_became_peered":"2024-04-25T11:35:14.456824+0000","last_unstale":"2024-04-25T11:35:15.699717+0000","last_undegraded":"2024-04-25T11:35:15.699717+0000","last_fullsized":"2024-04-25T11:35:15.699717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:00.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442866+0000","last_change":"2024-04-25T11:35:13.427560+0000","last_active":"2024-04-25T11:35:15.442866+0000","last_peered":"2024-04-25T11:35:15.442866+0000","last_clean":"2024-04-25T11:35:15.442866+0000","last_became_active":"2024-04-25T11:35:13.427177+0000","last_became_peered":"2024-04-25T11:35:13.427177+0000","last_unstale":"2024-04-25T11:35:15.442866+0000","last_undegraded":"2024-04-25T11:35:15.442866+0000","last_fullsized":"2024-04-25T11:35:15.442866+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:17.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.439614+0000","last_change":"2024-04-25T11:35:14.443854+0000","last_active":"2024-04-25T11:35:15.439614+0000","last_peered":"2024-04-25T11:35:15.439614+0000","last_clean":"2024-04-25T11:35:15.439614+0000","last_became_active":"2024-04-25T11:35:14.443441+0000","last_became_peered":"2024-04-25T11:35:14.443441+0000","last_unstale":"2024-04-25T11:35:15.439614+0000","last_undegraded":"2024-04-25T11:35:15.439614+0000","last_fullsized":"2024-04-25T11:35:15.439614+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:59.530010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699873+0000","last_change":"2024-04-25T11:35:13.430809+0000","last_active":"2024-04-25T11:35:15.699873+0000","last_peered":"2024-04-25T11:35:15.699873+0000","last_clean":"2024-04-25T11:35:15.699873+0000","last_became_active":"2024-04-25T11:35:13.430489+0000","last_became_peered":"2024-04-25T11:35:13.430489+0000","last_unstale":"2024-04-25T11:35:15.699873+0000","last_undegraded":"2024-04-25T11:35:15.699873+0000","last_fullsized":"2024-04-25T11:35:15.699873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.567839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974521+0000","last_change":"2024-04-25T11:35:14.429807+0000","last_active":"2024-04-25T11:35:15.974521+0000","last_peered":"2024-04-25T11:35:15.974521+0000","last_clean":"2024-04-25T11:35:15.974521+0000","last_became_active":"2024-04-25T11:35:14.429360+0000","last_became_peered":"2024-04-25T11:35:14.429360+0000","last_unstale":"2024-04-25T11:35:15.974521+0000","last_undegraded":"2024-04-25T11:35:15.974521+0000","last_fullsized":"2024-04-25T11:35:15.974521+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.355878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.667846+0000","last_change":"2024-04-25T11:35:13.430997+0000","last_active":"2024-04-25T11:35:15.667846+0000","last_peered":"2024-04-25T11:35:15.667846+0000","last_clean":"2024-04-25T11:35:15.667846+0000","last_became_active":"2024-04-25T11:35:13.428031+0000","last_became_peered":"2024-04-25T11:35:13.428031+0000","last_unstale":"2024-04-25T11:35:15.667846+0000","last_undegraded":"2024-04-25T11:35:15.667846+0000","last_fullsized":"2024-04-25T11:35:15.667846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:00.428630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973879+0000","last_change":"2024-04-25T11:35:14.426072+0000","last_active":"2024-04-25T11:35:15.973879+0000","last_peered":"2024-04-25T11:35:15.973879+0000","last_clean":"2024-04-25T11:35:15.973879+0000","last_became_active":"2024-04-25T11:35:14.425752+0000","last_became_peered":"2024-04-25T11:35:14.425752+0000","last_unstale":"2024-04-25T11:35:15.973879+0000","last_undegraded":"2024-04-25T11:35:15.973879+0000","last_fullsized":"2024-04-25T11:35:15.973879+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:41.231371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442533+0000","last_change":"2024-04-25T11:35:13.431423+0000","last_active":"2024-04-25T11:35:15.442533+0000","last_peered":"2024-04-25T11:35:15.442533+0000","last_clean":"2024-04-25T11:35:15.442533+0000","last_became_active":"2024-04-25T11:35:13.430960+0000","last_became_peered":"2024-04-25T11:35:13.430960+0000","last_unstale":"2024-04-25T11:35:15.442533+0000","last_undegraded":"2024-04-25T11:35:15.442533+0000","last_fullsized":"2024-04-25T11:35:15.442533+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:09.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442047+0000","last_change":"2024-04-25T11:35:14.431519+0000","last_active":"2024-04-25T11:35:15.442047+0000","last_peered":"2024-04-25T11:35:15.442047+0000","last_clean":"2024-04-25T11:35:15.442047+0000","last_became_active":"2024-04-25T11:35:14.431178+0000","last_became_peered":"2024-04-25T11:35:14.431178+0000","last_unstale":"2024-04-25T11:35:15.442047+0000","last_undegraded":"2024-04-25T11:35:15.442047+0000","last_fullsized":"2024-04-25T11:35:15.442047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:02.950848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443031+0000","last_change":"2024-04-25T11:35:13.431541+0000","last_active":"2024-04-25T11:35:15.443031+0000","last_peered":"2024-04-25T11:35:15.443031+0000","last_clean":"2024-04-25T11:35:15.443031+0000","last_became_active":"2024-04-25T11:35:13.431197+0000","last_became_peered":"2024-04-25T11:35:13.431197+0000","last_unstale":"2024-04-25T11:35:15.443031+0000","last_undegraded":"2024-04-25T11:35:15.443031+0000","last_fullsized":"2024-04-25T11:35:15.443031+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.156013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441981+0000","last_change":"2024-04-25T11:35:14.424889+0000","last_active":"2024-04-25T11:35:15.441981+0000","last_peered":"2024-04-25T11:35:15.441981+0000","last_clean":"2024-04-25T11:35:15.441981+0000","last_became_active":"2024-04-25T11:35:14.424564+0000","last_became_peered":"2024-04-25T11:35:14.424564+0000","last_unstale":"2024-04-25T11:35:15.441981+0000","last_undegraded":"2024-04-25T11:35:15.441981+0000","last_fullsized":"2024-04-25T11:35:15.441981+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:58.700206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668814+0000","last_change":"2024-04-25T11:35:13.431620+0000","last_active":"2024-04-25T11:35:15.668814+0000","last_peered":"2024-04-25T11:35:15.668814+0000","last_clean":"2024-04-25T11:35:15.668814+0000","last_became_active":"2024-04-25T11:35:13.431300+0000","last_became_peered":"2024-04-25T11:35:13.431300+0000","last_unstale":"2024-04-25T11:35:15.668814+0000","last_undegraded":"2024-04-25T11:35:15.668814+0000","last_fullsized":"2024-04-25T11:35:15.668814+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:14.404919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440167+0000","last_change":"2024-04-25T11:35:14.434307+0000","last_active":"2024-04-25T11:35:15.440167+0000","last_peered":"2024-04-25T11:35:15.440167+0000","last_clean":"2024-04-25T11:35:15.440167+0000","last_became_active":"2024-04-25T11:35:14.432356+0000","last_became_peered":"2024-04-25T11:35:14.432356+0000","last_unstale":"2024-04-25T11:35:15.440167+0000","last_undegraded":"2024-04-25T11:35:15.440167+0000","last_fullsized":"2024-04-25T11:35:15.440167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:21.662801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973591+0000","last_change":"2024-04-25T11:35:13.436194+0000","last_active":"2024-04-25T11:35:15.973591+0000","last_peered":"2024-04-25T11:35:15.973591+0000","last_clean":"2024-04-25T11:35:15.973591+0000","last_became_active":"2024-04-25T11:35:13.434670+0000","last_became_peered":"2024-04-25T11:35:13.434670+0000","last_unstale":"2024-04-25T11:35:15.973591+0000","last_undegraded":"2024-04-25T11:35:15.973591+0000","last_fullsized":"2024-04-25T11:35:15.973591+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:46.813695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441132+0000","last_change":"2024-04-25T11:35:14.441216+0000","last_active":"2024-04-25T11:35:15.441132+0000","last_peered":"2024-04-25T11:35:15.441132+0000","last_clean":"2024-04-25T11:35:15.441132+0000","last_became_active":"2024-04-25T11:35:14.440903+0000","last_became_peered":"2024-04-25T11:35:14.440903+0000","last_unstale":"2024-04-25T11:35:15.441132+0000","last_undegraded":"2024-04-25T11:35:15.441132+0000","last_fullsized":"2024-04-25T11:35:15.441132+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:31.972290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440835+0000","last_change":"2024-04-25T11:35:13.429491+0000","last_active":"2024-04-25T11:35:15.440835+0000","last_peered":"2024-04-25T11:35:15.440835+0000","last_clean":"2024-04-25T11:35:15.440835+0000","last_became_active":"2024-04-25T11:35:13.427659+0000","last_became_peered":"2024-04-25T11:35:13.427659+0000","last_unstale":"2024-04-25T11:35:15.440835+0000","last_undegraded":"2024-04-25T11:35:15.440835+0000","last_fullsized":"2024-04-25T11:35:15.440835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:42.804232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671105+0000","last_change":"2024-04-25T11:35:14.431959+0000","last_active":"2024-04-25T11:35:15.671105+0000","last_peered":"2024-04-25T11:35:15.671105+0000","last_clean":"2024-04-25T11:35:15.671105+0000","last_became_active":"2024-04-25T11:35:14.428698+0000","last_became_peered":"2024-04-25T11:35:14.428698+0000","last_unstale":"2024-04-25T11:35:15.671105+0000","last_undegraded":"2024-04-25T11:35:15.671105+0000","last_fullsized":"2024-04-25T11:35:15.671105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:16.722343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.700050+0000","last_change":"2024-04-25T11:35:13.425327+0000","last_active":"2024-04-25T11:35:15.700050+0000","last_peered":"2024-04-25T11:35:15.700050+0000","last_clean":"2024-04-25T11:35:15.700050+0000","last_became_active":"2024-04-25T11:35:13.424970+0000","last_became_peered":"2024-04-25T11:35:13.424970+0000","last_unstale":"2024-04-25T11:35:15.700050+0000","last_undegraded":"2024-04-25T11:35:15.700050+0000","last_fullsized":"2024-04-25T11:35:15.700050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:24.796451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668640+0000","last_change":"2024-04-25T11:35:14.441685+0000","last_active":"2024-04-25T11:35:15.668640+0000","last_peered":"2024-04-25T11:35:15.668640+0000","last_clean":"2024-04-25T11:35:15.668640+0000","last_became_active":"2024-04-25T11:35:14.441168+0000","last_became_peered":"2024-04-25T11:35:14.441168+0000","last_unstale":"2024-04-25T11:35:15.668640+0000","last_undegraded":"2024-04-25T11:35:15.668640+0000","last_fullsized":"2024-04-25T11:35:15.668640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:25.814645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440527+0000","last_change":"2024-04-25T11:35:13.420152+0000","last_active":"2024-04-25T11:35:15.440527+0000","last_peered":"2024-04-25T11:35:15.440527+0000","last_clean":"2024-04-25T11:35:15.440527+0000","last_became_active":"2024-04-25T11:35:13.419769+0000","last_became_peered":"2024-04-25T11:35:13.419769+0000","last_unstale":"2024-04-25T11:35:15.440527+0000","last_undegraded":"2024-04-25T11:35:15.440527+0000","last_fullsized":"2024-04-25T11:35:15.440527+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:21.615159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671336+0000","last_change":"2024-04-25T11:35:14.427179+0000","last_active":"2024-04-25T11:35:15.671336+0000","last_peered":"2024-04-25T11:35:15.671336+0000","last_clean":"2024-04-25T11:35:15.671336+0000","last_became_active":"2024-04-25T11:35:14.426803+0000","last_became_peered":"2024-04-25T11:35:14.426803+0000","last_unstale":"2024-04-25T11:35:15.671336+0000","last_undegraded":"2024-04-25T11:35:15.671336+0000","last_fullsized":"2024-04-25T11:35:15.671336+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:10.665968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.670703+0000","last_change":"2024-04-25T11:35:13.420223+0000","last_active":"2024-04-25T11:35:15.670703+0000","last_peered":"2024-04-25T11:35:15.670703+0000","last_clean":"2024-04-25T11:35:15.670703+0000","last_became_active":"2024-04-25T11:35:13.419888+0000","last_became_peered":"2024-04-25T11:35:13.419888+0000","last_unstale":"2024-04-25T11:35:15.670703+0000","last_undegraded":"2024-04-25T11:35:15.670703+0000","last_fullsized":"2024-04-25T11:35:15.670703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.058527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.670804+0000","last_change":"2024-04-25T11:35:14.444665+0000","last_active":"2024-04-25T11:35:15.670804+0000","last_peered":"2024-04-25T11:35:15.670804+0000","last_clean":"2024-04-25T11:35:15.670804+0000","last_became_active":"2024-04-25T11:35:14.444348+0000","last_became_peered":"2024-04-25T11:35:14.444348+0000","last_unstale":"2024-04-25T11:35:15.670804+0000","last_undegraded":"2024-04-25T11:35:15.670804+0000","last_fullsized":"2024-04-25T11:35:15.670804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:25.814645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973146+0000","last_change":"2024-04-25T11:35:13.426511+0000","last_active":"2024-04-25T11:35:15.973146+0000","last_peered":"2024-04-25T11:35:15.973146+0000","last_clean":"2024-04-25T11:35:15.973146+0000","last_became_active":"2024-04-25T11:35:13.426183+0000","last_became_peered":"2024-04-25T11:35:13.426183+0000","last_unstale":"2024-04-25T11:35:15.973146+0000","last_undegraded":"2024-04-25T11:35:15.973146+0000","last_fullsized":"2024-04-25T11:35:15.973146+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:34.938031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668638+0000","last_change":"2024-04-25T11:35:14.451135+0000","last_active":"2024-04-25T11:35:15.668638+0000","last_peered":"2024-04-25T11:35:15.668638+0000","last_clean":"2024-04-25T11:35:15.668638+0000","last_became_active":"2024-04-25T11:35:14.446640+0000","last_became_peered":"2024-04-25T11:35:14.446640+0000","last_unstale":"2024-04-25T11:35:15.668638+0000","last_undegraded":"2024-04-25T11:35:15.668638+0000","last_fullsized":"2024-04-25T11:35:15.668638+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:16.722343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441712+0000","last_change":"2024-04-25T11:35:13.428876+0000","last_active":"2024-04-25T11:35:15.441712+0000","last_peered":"2024-04-25T11:35:15.441712+0000","last_clean":"2024-04-25T11:35:15.441712+0000","last_became_active":"2024-04-25T11:35:13.422642+0000","last_became_peered":"2024-04-25T11:35:13.422642+0000","last_unstale":"2024-04-25T11:35:15.441712+0000","last_undegraded":"2024-04-25T11:35:15.441712+0000","last_fullsized":"2024-04-25T11:35:15.441712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:40.031617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671725+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:35:15.671725+0000","last_peered":"2024-04-25T11:35:15.671725+0000","last_clean":"2024-04-25T11:35:15.671725+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:35:15.671725+0000","last_undegraded":"2024-04-25T11:35:15.671725+0000","last_fullsized":"2024-04-25T11:35:15.671725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":484609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1130496,"data_stored":485211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":484573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":485213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509541,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":484539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509541,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":1074997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":486173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1720320,"data_stored":1070253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:35:19.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:35:19.351 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:35:19.363+0000 7f3e62a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:35:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.529+0000 7fac3ae00640 1 Processor -- start 2024-04-25T11:35:19.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.529+0000 7fac3ae00640 1 -- start start 2024-04-25T11:35:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.531+0000 7fac3ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fac3406c5c0 0x7fac340cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.531+0000 7fac3ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac34144370 0x7fac340b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.531+0000 7fac3ae00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac340b50e0 con 0x7fac340b4cd0 2024-04-25T11:35:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.531+0000 7fac3ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fac340d1230 con 0x7fac3406c5c0 2024-04-25T11:35:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.531+0000 7fac3ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac340d06b0 con 0x7fac34144370 2024-04-25T11:35:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.532+0000 7fac3a400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fac340b4cd0 0x7fac34161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35888/0 (socket says 172.21.15.19:35888) 2024-04-25T11:35:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.532+0000 7fac3a400640 1 -- 172.21.15.19:0/1262929828 learned_addr learned my addr 172.21.15.19:0/1262929828 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.532+0000 7fac39000640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac34144370 0x7fac340b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.532+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292674653 0 0) 0x7fac2c001490 con 0x7fac340b4cd0 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.532+0000 7fac39a00640 1 --2- 172.21.15.19:0/1262929828 >> v2:172.21.15.169:3300/0 conn(0x7fac3406c5c0 0x7fac340cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.532+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac1c003520 con 0x7fac340b4cd0 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.532+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2654749582 0 0) 0x7fac1c003520 con 0x7fac340b4cd0 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac34144370 msgr2=0x7fac340b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac34144370 0x7fac340b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 >> v2:172.21.15.169:3300/0 conn(0x7fac3406c5c0 msgr2=0x7fac340cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 --2- 172.21.15.19:0/1262929828 >> v2:172.21.15.169:3300/0 conn(0x7fac3406c5c0 0x7fac340cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac340d0990 con 0x7fac340b4cd0 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fac2c001bc0 con 0x7fac340b4cd0 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 >> v1:172.21.15.19:6789/0 conn(0x7fac340b4cd0 legacy=0x7fac34161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c0039b0 0x7fac1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c004800 0x7fac1c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac34144370 0x7fac1c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac340d1230 con 0x7fac1c0039b0 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac340d06b0 con 0x7fac1c004800 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c00b9c0 con 0x7fac34144370 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.533+0000 7fac39a00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c0039b0 0x7fac1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.534+0000 7fac39a00640 1 -- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac34144370 msgr2=0x7fac1c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.534+0000 7fac39a00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac34144370 0x7fac1c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.534+0000 7fac39a00640 1 -- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c004800 msgr2=0x7fac1c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.534+0000 7fac39a00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c004800 0x7fac1c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.534+0000 7fac39a00640 1 -- 172.21.15.19:0/1262929828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c003520 con 0x7fac1c0039b0 2024-04-25T11:35:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.535+0000 7fac39a00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c0039b0 0x7fac1c003e20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fac28009ac0 tx=0x7fac1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4a6abeb623ad78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:19.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.535+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28036070 con 0x7fac1c0039b0 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.535+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac28031400 con 0x7fac1c0039b0 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.535+0000 7fac33e00640 1 -- 172.21.15.19:0/1262929828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28009690 con 0x7fac1c0039b0 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.536+0000 7fac3ae00640 1 -- 172.21.15.19:0/1262929828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c0039b0 msgr2=0x7fac1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.536+0000 7fac3ae00640 1 --2- 172.21.15.19:0/1262929828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac1c0039b0 0x7fac1c003e20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fac28009ac0 tx=0x7fac1c005830 comp rx=0 tx=0).stop 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.536+0000 7fac3a400640 1 -- 172.21.15.19:0/1262929828 reap_dead start 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.536+0000 7fac3ae00640 1 -- 172.21.15.19:0/1262929828 shutdown_connections 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.536+0000 7fac3ae00640 1 -- 172.21.15.19:0/1262929828 >> 172.21.15.19:0/1262929828 conn(0x7fac340d1720 msgr2=0x7fac340d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.536+0000 7fac3ae00640 1 -- 172.21.15.19:0/1262929828 shutdown_connections 2024-04-25T11:35:19.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.536+0000 7fac3ae00640 1 -- 172.21.15.19:0/1262929828 wait complete. 2024-04-25T11:35:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.538+0000 7fac3ae00640 1 Processor -- start 2024-04-25T11:35:19.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.539+0000 7fac3ae00640 1 -- start start 2024-04-25T11:35:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.540+0000 7fac3ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c0039b0 0x7fac3413f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.540+0000 7fac3ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac3413c5c0 0x7fac3413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.540+0000 7fac3ae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac340cc320 0x7fac340cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.540+0000 7fac3ae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac340d02d0 con 0x7fac340cc320 2024-04-25T11:35:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.540+0000 7fac3ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac340d0f50 con 0x7fac1c0039b0 2024-04-25T11:35:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.540+0000 7fac3ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac340d0450 con 0x7fac3413c5c0 2024-04-25T11:35:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac340cc320 0x7fac340cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac340cc320 0x7fac340cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36540/0 (socket says 172.21.15.19:36540) 2024-04-25T11:35:19.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 -- 172.21.15.19:0/2184384688 learned_addr learned my addr 172.21.15.19:0/2184384688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac39000640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac3413c5c0 0x7fac3413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:19.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 -- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac3413c5c0 msgr2=0x7fac3413f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac3413c5c0 0x7fac3413f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 -- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c0039b0 msgr2=0x7fac3413f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac39a00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c0039b0 0x7fac3413f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c0039b0 0x7fac3413f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 -- 172.21.15.19:0/2184384688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac34160cd0 con 0x7fac340cc320 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.541+0000 7fac3a400640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac340cc320 0x7fac340cc730 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fac340b5550 tx=0x7fac2c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.542+0000 7fac31600640 1 -- 172.21.15.19:0/2184384688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c00a610 con 0x7fac340cc320 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.542+0000 7fac31600640 1 -- 172.21.15.19:0/2184384688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac2c00a7b0 con 0x7fac340cc320 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.542+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac34142fe0 con 0x7fac340cc320 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.542+0000 7fac31600640 1 -- 172.21.15.19:0/2184384688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c00aba0 con 0x7fac340cc320 2024-04-25T11:35:19.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.542+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341432c0 con 0x7fac340cc320 2024-04-25T11:35:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.543+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac34143520 con 0x7fac340cc320 2024-04-25T11:35:19.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.544+0000 7fac31600640 1 -- 172.21.15.19:0/2184384688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac2c00ad40 con 0x7fac340cc320 2024-04-25T11:35:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.546+0000 7fac31600640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744e0 0x7fac140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.546+0000 7fac31600640 1 -- 172.21.15.19:0/2184384688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fac2c094b90 con 0x7fac340cc320 2024-04-25T11:35:19.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.547+0000 7fac39a00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744e0 0x7fac140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:19.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.548+0000 7fac39a00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744e0 0x7fac140769f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac28033000 tx=0x7fac28008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:19.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.551+0000 7fac31600640 1 -- 172.21.15.19:0/2184384688 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac340b50e0 con 0x7fac340cc320 2024-04-25T11:35:19.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.748+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac34068db0 con 0x7fac140744e0 2024-04-25T11:35:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.767+0000 7fac31600640 1 -- 172.21.15.19:0/2184384688 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7fac34068db0 con 0x7fac140744e0 2024-04-25T11:35:19.759 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744e0 msgr2=0x7fac140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744e0 0x7fac140769f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac28033000 tx=0x7fac28008030 comp rx=0 tx=0).stop 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac340cc320 msgr2=0x7fac340cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac340cc320 0x7fac340cc730 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fac340b5550 tx=0x7fac2c00cf00 comp rx=0 tx=0).stop 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 shutdown_connections 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744e0 0x7fac140769f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac340cc320 0x7fac340cc730 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac3413c5c0 0x7fac3413f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 --2- 172.21.15.19:0/2184384688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac1c0039b0 0x7fac3413f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.773+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 >> 172.21.15.19:0/2184384688 conn(0x7fac340d1720 msgr2=0x7fac34070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.774+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 shutdown_connections 2024-04-25T11:35:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:19.774+0000 7fac3ae00640 1 -- 172.21.15.19:0/2184384688 wait complete. 2024-04-25T11:35:19.789 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-04-25T11:35:18.638328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259428,"kb_used_data":9980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715292,"statfs":{"total":773094113280,"available":772828459008,"internally_reserved":0,"allocated":10219520,"data_stored":5055568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002308"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671702+0000","last_change":"2024-04-25T11:35:14.437721+0000","last_active":"2024-04-25T11:35:15.671702+0000","last_peered":"2024-04-25T11:35:15.671702+0000","last_clean":"2024-04-25T11:35:15.671702+0000","last_became_active":"2024-04-25T11:35:14.437402+0000","last_became_peered":"2024-04-25T11:35:14.437402+0000","last_unstale":"2024-04-25T11:35:15.671702+0000","last_undegraded":"2024-04-25T11:35:15.671702+0000","last_fullsized":"2024-04-25T11:35:15.671702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:00.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.700506+0000","last_change":"2024-04-25T11:35:14.449924+0000","last_active":"2024-04-25T11:35:15.700506+0000","last_peered":"2024-04-25T11:35:15.700506+0000","last_clean":"2024-04-25T11:35:15.700506+0000","last_became_active":"2024-04-25T11:35:14.449468+0000","last_became_peered":"2024-04-25T11:35:14.449468+0000","last_unstale":"2024-04-25T11:35:15.700506+0000","last_undegraded":"2024-04-25T11:35:15.700506+0000","last_fullsized":"2024-04-25T11:35:15.700506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.822426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441139+0000","last_change":"2024-04-25T11:35:14.424226+0000","last_active":"2024-04-25T11:35:15.441139+0000","last_peered":"2024-04-25T11:35:15.441139+0000","last_clean":"2024-04-25T11:35:15.441139+0000","last_became_active":"2024-04-25T11:35:14.423890+0000","last_became_peered":"2024-04-25T11:35:14.423890+0000","last_unstale":"2024-04-25T11:35:15.441139+0000","last_undegraded":"2024-04-25T11:35:15.441139+0000","last_fullsized":"2024-04-25T11:35:15.441139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.350949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.975380+0000","last_change":"2024-04-25T11:35:14.425627+0000","last_active":"2024-04-25T11:35:15.975380+0000","last_peered":"2024-04-25T11:35:15.975380+0000","last_clean":"2024-04-25T11:35:15.975380+0000","last_became_active":"2024-04-25T11:35:14.424819+0000","last_became_peered":"2024-04-25T11:35:14.424819+0000","last_unstale":"2024-04-25T11:35:15.975380+0000","last_undegraded":"2024-04-25T11:35:15.975380+0000","last_fullsized":"2024-04-25T11:35:15.975380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:32.983110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.439446+0000","last_change":"2024-04-25T11:35:14.445094+0000","last_active":"2024-04-25T11:35:15.439446+0000","last_peered":"2024-04-25T11:35:15.439446+0000","last_clean":"2024-04-25T11:35:15.439446+0000","last_became_active":"2024-04-25T11:35:14.444736+0000","last_became_peered":"2024-04-25T11:35:14.444736+0000","last_unstale":"2024-04-25T11:35:15.439446+0000","last_undegraded":"2024-04-25T11:35:15.439446+0000","last_fullsized":"2024-04-25T11:35:15.439446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:45.259815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440360+0000","last_change":"2024-04-25T11:35:14.426610+0000","last_active":"2024-04-25T11:35:15.440360+0000","last_peered":"2024-04-25T11:35:15.440360+0000","last_clean":"2024-04-25T11:35:15.440360+0000","last_became_active":"2024-04-25T11:35:14.426299+0000","last_became_peered":"2024-04-25T11:35:14.426299+0000","last_unstale":"2024-04-25T11:35:15.440360+0000","last_undegraded":"2024-04-25T11:35:15.440360+0000","last_fullsized":"2024-04-25T11:35:15.440360+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:21.772732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699659+0000","last_change":"2024-04-25T11:35:14.446356+0000","last_active":"2024-04-25T11:35:15.699659+0000","last_peered":"2024-04-25T11:35:15.699659+0000","last_clean":"2024-04-25T11:35:15.699659+0000","last_became_active":"2024-04-25T11:35:14.445518+0000","last_became_peered":"2024-04-25T11:35:14.445518+0000","last_unstale":"2024-04-25T11:35:15.699659+0000","last_undegraded":"2024-04-25T11:35:15.699659+0000","last_fullsized":"2024-04-25T11:35:15.699659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:27.241606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973462+0000","last_change":"2024-04-25T11:35:14.448845+0000","last_active":"2024-04-25T11:35:15.973462+0000","last_peered":"2024-04-25T11:35:15.973462+0000","last_clean":"2024-04-25T11:35:15.973462+0000","last_became_active":"2024-04-25T11:35:14.448052+0000","last_became_peered":"2024-04-25T11:35:14.448052+0000","last_unstale":"2024-04-25T11:35:15.973462+0000","last_undegraded":"2024-04-25T11:35:15.973462+0000","last_fullsized":"2024-04-25T11:35:15.973462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:22.633353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672155+0000","last_change":"2024-04-25T11:35:14.442376+0000","last_active":"2024-04-25T11:35:15.672155+0000","last_peered":"2024-04-25T11:35:15.672155+0000","last_clean":"2024-04-25T11:35:15.672155+0000","last_became_active":"2024-04-25T11:35:14.442052+0000","last_became_peered":"2024-04-25T11:35:14.442052+0000","last_unstale":"2024-04-25T11:35:15.672155+0000","last_undegraded":"2024-04-25T11:35:15.672155+0000","last_fullsized":"2024-04-25T11:35:15.672155+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.822426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441188+0000","last_change":"2024-04-25T11:35:14.433269+0000","last_active":"2024-04-25T11:35:15.441188+0000","last_peered":"2024-04-25T11:35:15.441188+0000","last_clean":"2024-04-25T11:35:15.441188+0000","last_became_active":"2024-04-25T11:35:14.432939+0000","last_became_peered":"2024-04-25T11:35:14.432939+0000","last_unstale":"2024-04-25T11:35:15.441188+0000","last_undegraded":"2024-04-25T11:35:15.441188+0000","last_fullsized":"2024-04-25T11:35:15.441188+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:53.280142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669180+0000","last_change":"2024-04-25T11:35:14.452859+0000","last_active":"2024-04-25T11:35:15.669180+0000","last_peered":"2024-04-25T11:35:15.669180+0000","last_clean":"2024-04-25T11:35:15.669180+0000","last_became_active":"2024-04-25T11:35:14.452431+0000","last_became_peered":"2024-04-25T11:35:14.452431+0000","last_unstale":"2024-04-25T11:35:15.669180+0000","last_undegraded":"2024-04-25T11:35:15.669180+0000","last_fullsized":"2024-04-25T11:35:15.669180+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:05.855382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973859+0000","last_change":"2024-04-25T11:35:14.447933+0000","last_active":"2024-04-25T11:35:15.973859+0000","last_peered":"2024-04-25T11:35:15.973859+0000","last_clean":"2024-04-25T11:35:15.973859+0000","last_became_active":"2024-04-25T11:35:14.447574+0000","last_became_peered":"2024-04-25T11:35:14.447574+0000","last_unstale":"2024-04-25T11:35:15.973859+0000","last_undegraded":"2024-04-25T11:35:15.973859+0000","last_fullsized":"2024-04-25T11:35:15.973859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:25.937058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.439499+0000","last_change":"2024-04-25T11:35:14.431316+0000","last_active":"2024-04-25T11:35:15.439499+0000","last_peered":"2024-04-25T11:35:15.439499+0000","last_clean":"2024-04-25T11:35:15.439499+0000","last_became_active":"2024-04-25T11:35:14.429092+0000","last_became_peered":"2024-04-25T11:35:14.429092+0000","last_unstale":"2024-04-25T11:35:15.439499+0000","last_undegraded":"2024-04-25T11:35:15.439499+0000","last_fullsized":"2024-04-25T11:35:15.439499+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:58.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.444791+0000","last_change":"2024-04-25T11:35:14.456382+0000","last_active":"2024-04-25T11:35:15.444791+0000","last_peered":"2024-04-25T11:35:15.444791+0000","last_clean":"2024-04-25T11:35:15.444791+0000","last_became_active":"2024-04-25T11:35:14.456086+0000","last_became_peered":"2024-04-25T11:35:14.456086+0000","last_unstale":"2024-04-25T11:35:15.444791+0000","last_undegraded":"2024-04-25T11:35:15.444791+0000","last_fullsized":"2024-04-25T11:35:15.444791+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:34.847504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699839+0000","last_change":"2024-04-25T11:35:14.446623+0000","last_active":"2024-04-25T11:35:15.699839+0000","last_peered":"2024-04-25T11:35:15.699839+0000","last_clean":"2024-04-25T11:35:15.699839+0000","last_became_active":"2024-04-25T11:35:14.445973+0000","last_became_peered":"2024-04-25T11:35:14.445973+0000","last_unstale":"2024-04-25T11:35:15.699839+0000","last_undegraded":"2024-04-25T11:35:15.699839+0000","last_fullsized":"2024-04-25T11:35:15.699839+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:22.633353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440516+0000","last_change":"2024-04-25T11:35:14.434368+0000","last_active":"2024-04-25T11:35:15.440516+0000","last_peered":"2024-04-25T11:35:15.440516+0000","last_clean":"2024-04-25T11:35:15.440516+0000","last_became_active":"2024-04-25T11:35:14.434038+0000","last_became_peered":"2024-04-25T11:35:14.434038+0000","last_unstale":"2024-04-25T11:35:15.440516+0000","last_undegraded":"2024-04-25T11:35:15.440516+0000","last_fullsized":"2024-04-25T11:35:15.440516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:41.353736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974397+0000","last_change":"2024-04-25T11:35:14.447167+0000","last_active":"2024-04-25T11:35:15.974397+0000","last_peered":"2024-04-25T11:35:15.974397+0000","last_clean":"2024-04-25T11:35:15.974397+0000","last_became_active":"2024-04-25T11:35:14.446570+0000","last_became_peered":"2024-04-25T11:35:14.446570+0000","last_unstale":"2024-04-25T11:35:15.974397+0000","last_undegraded":"2024-04-25T11:35:15.974397+0000","last_fullsized":"2024-04-25T11:35:15.974397+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:47.735930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669389+0000","last_change":"2024-04-25T11:35:14.457086+0000","last_active":"2024-04-25T11:35:15.669389+0000","last_peered":"2024-04-25T11:35:15.669389+0000","last_clean":"2024-04-25T11:35:15.669389+0000","last_became_active":"2024-04-25T11:35:14.456750+0000","last_became_peered":"2024-04-25T11:35:14.456750+0000","last_unstale":"2024-04-25T11:35:15.669389+0000","last_undegraded":"2024-04-25T11:35:15.669389+0000","last_fullsized":"2024-04-25T11:35:15.669389+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:02.357995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.444255+0000","last_change":"2024-04-25T11:35:14.455058+0000","last_active":"2024-04-25T11:35:15.444255+0000","last_peered":"2024-04-25T11:35:15.444255+0000","last_clean":"2024-04-25T11:35:15.444255+0000","last_became_active":"2024-04-25T11:35:14.454688+0000","last_became_peered":"2024-04-25T11:35:14.454688+0000","last_unstale":"2024-04-25T11:35:15.444255+0000","last_undegraded":"2024-04-25T11:35:15.444255+0000","last_fullsized":"2024-04-25T11:35:15.444255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:05.855382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443783+0000","last_change":"2024-04-25T11:35:14.460394+0000","last_active":"2024-04-25T11:35:15.443783+0000","last_peered":"2024-04-25T11:35:15.443783+0000","last_clean":"2024-04-25T11:35:15.443783+0000","last_became_active":"2024-04-25T11:35:14.458496+0000","last_became_peered":"2024-04-25T11:35:14.458496+0000","last_unstale":"2024-04-25T11:35:15.443783+0000","last_undegraded":"2024-04-25T11:35:15.443783+0000","last_fullsized":"2024-04-25T11:35:15.443783+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:25.814645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974603+0000","last_change":"2024-04-25T11:35:14.448650+0000","last_active":"2024-04-25T11:35:15.974603+0000","last_peered":"2024-04-25T11:35:15.974603+0000","last_clean":"2024-04-25T11:35:15.974603+0000","last_became_active":"2024-04-25T11:35:14.448330+0000","last_became_peered":"2024-04-25T11:35:14.448330+0000","last_unstale":"2024-04-25T11:35:15.974603+0000","last_undegraded":"2024-04-25T11:35:15.974603+0000","last_fullsized":"2024-04-25T11:35:15.974603+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:40.477803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443508+0000","last_change":"2024-04-25T11:35:14.461184+0000","last_active":"2024-04-25T11:35:15.443508+0000","last_peered":"2024-04-25T11:35:15.443508+0000","last_clean":"2024-04-25T11:35:15.443508+0000","last_became_active":"2024-04-25T11:35:14.460843+0000","last_became_peered":"2024-04-25T11:35:14.460843+0000","last_unstale":"2024-04-25T11:35:15.443508+0000","last_undegraded":"2024-04-25T11:35:15.443508+0000","last_fullsized":"2024-04-25T11:35:15.443508+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:26.586033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973497+0000","last_change":"2024-04-25T11:35:14.436174+0000","last_active":"2024-04-25T11:35:15.973497+0000","last_peered":"2024-04-25T11:35:15.973497+0000","last_clean":"2024-04-25T11:35:15.973497+0000","last_became_active":"2024-04-25T11:35:14.435858+0000","last_became_peered":"2024-04-25T11:35:14.435858+0000","last_unstale":"2024-04-25T11:35:15.973497+0000","last_undegraded":"2024-04-25T11:35:15.973497+0000","last_fullsized":"2024-04-25T11:35:15.973497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:43.822426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668524+0000","last_change":"2024-04-25T11:35:14.458906+0000","last_active":"2024-04-25T11:35:15.668524+0000","last_peered":"2024-04-25T11:35:15.668524+0000","last_clean":"2024-04-25T11:35:15.668524+0000","last_became_active":"2024-04-25T11:35:14.457429+0000","last_became_peered":"2024-04-25T11:35:14.457429+0000","last_unstale":"2024-04-25T11:35:15.668524+0000","last_undegraded":"2024-04-25T11:35:15.668524+0000","last_fullsized":"2024-04-25T11:35:15.668524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:26.586033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669171+0000","last_change":"2024-04-25T11:35:14.448530+0000","last_active":"2024-04-25T11:35:15.669171+0000","last_peered":"2024-04-25T11:35:15.669171+0000","last_clean":"2024-04-25T11:35:15.669171+0000","last_became_active":"2024-04-25T11:35:14.448220+0000","last_became_peered":"2024-04-25T11:35:14.448220+0000","last_unstale":"2024-04-25T11:35:15.669171+0000","last_undegraded":"2024-04-25T11:35:15.669171+0000","last_fullsized":"2024-04-25T11:35:15.669171+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:34.847504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442046+0000","last_change":"2024-04-25T11:35:14.450485+0000","last_active":"2024-04-25T11:35:15.442046+0000","last_peered":"2024-04-25T11:35:15.442046+0000","last_clean":"2024-04-25T11:35:15.442046+0000","last_became_active":"2024-04-25T11:35:14.450082+0000","last_became_peered":"2024-04-25T11:35:14.450082+0000","last_unstale":"2024-04-25T11:35:15.442046+0000","last_undegraded":"2024-04-25T11:35:15.442046+0000","last_fullsized":"2024-04-25T11:35:15.442046+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:17.516432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441718+0000","last_change":"2024-04-25T11:35:14.426613+0000","last_active":"2024-04-25T11:35:15.441718+0000","last_peered":"2024-04-25T11:35:15.441718+0000","last_clean":"2024-04-25T11:35:15.441718+0000","last_became_active":"2024-04-25T11:35:14.426304+0000","last_became_peered":"2024-04-25T11:35:14.426304+0000","last_unstale":"2024-04-25T11:35:15.441718+0000","last_undegraded":"2024-04-25T11:35:15.441718+0000","last_fullsized":"2024-04-25T11:35:15.441718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:34.975141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671238+0000","last_change":"2024-04-25T11:35:14.429109+0000","last_active":"2024-04-25T11:35:15.671238+0000","last_peered":"2024-04-25T11:35:15.671238+0000","last_clean":"2024-04-25T11:35:15.671238+0000","last_became_active":"2024-04-25T11:35:14.428779+0000","last_became_peered":"2024-04-25T11:35:14.428779+0000","last_unstale":"2024-04-25T11:35:15.671238+0000","last_undegraded":"2024-04-25T11:35:15.671238+0000","last_fullsized":"2024-04-25T11:35:15.671238+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:34.847504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974540+0000","last_change":"2024-04-25T11:35:14.443789+0000","last_active":"2024-04-25T11:35:15.974540+0000","last_peered":"2024-04-25T11:35:15.974540+0000","last_clean":"2024-04-25T11:35:15.974540+0000","last_became_active":"2024-04-25T11:35:14.439806+0000","last_became_peered":"2024-04-25T11:35:14.439806+0000","last_unstale":"2024-04-25T11:35:15.974540+0000","last_undegraded":"2024-04-25T11:35:15.974540+0000","last_fullsized":"2024-04-25T11:35:15.974540+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:42.939974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443986+0000","last_change":"2024-04-25T11:35:14.457363+0000","last_active":"2024-04-25T11:35:15.443986+0000","last_peered":"2024-04-25T11:35:15.443986+0000","last_clean":"2024-04-25T11:35:15.443986+0000","last_became_active":"2024-04-25T11:35:14.457035+0000","last_became_peered":"2024-04-25T11:35:14.457035+0000","last_unstale":"2024-04-25T11:35:15.443986+0000","last_undegraded":"2024-04-25T11:35:15.443986+0000","last_fullsized":"2024-04-25T11:35:15.443986+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:16.722343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440203+0000","last_change":"2024-04-25T11:35:14.434057+0000","last_active":"2024-04-25T11:35:15.440203+0000","last_peered":"2024-04-25T11:35:15.440203+0000","last_clean":"2024-04-25T11:35:15.440203+0000","last_became_active":"2024-04-25T11:35:14.429795+0000","last_became_peered":"2024-04-25T11:35:14.429795+0000","last_unstale":"2024-04-25T11:35:15.440203+0000","last_undegraded":"2024-04-25T11:35:15.440203+0000","last_fullsized":"2024-04-25T11:35:15.440203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:48.823179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440331+0000","last_change":"2024-04-25T11:35:14.436818+0000","last_active":"2024-04-25T11:35:15.440331+0000","last_peered":"2024-04-25T11:35:15.440331+0000","last_clean":"2024-04-25T11:35:15.440331+0000","last_became_active":"2024-04-25T11:35:14.435526+0000","last_became_peered":"2024-04-25T11:35:14.435526+0000","last_unstale":"2024-04-25T11:35:15.440331+0000","last_undegraded":"2024-04-25T11:35:15.440331+0000","last_fullsized":"2024-04-25T11:35:15.440331+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.415414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974007+0000","last_change":"2024-04-25T11:35:14.434385+0000","last_active":"2024-04-25T11:35:15.974007+0000","last_peered":"2024-04-25T11:35:15.974007+0000","last_clean":"2024-04-25T11:35:15.974007+0000","last_became_active":"2024-04-25T11:35:14.434075+0000","last_became_peered":"2024-04-25T11:35:14.434075+0000","last_unstale":"2024-04-25T11:35:15.974007+0000","last_undegraded":"2024-04-25T11:35:15.974007+0000","last_fullsized":"2024-04-25T11:35:15.974007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:31.037258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442045+0000","last_change":"2024-04-25T11:35:13.437396+0000","last_active":"2024-04-25T11:35:15.442045+0000","last_peered":"2024-04-25T11:35:15.442045+0000","last_clean":"2024-04-25T11:35:15.442045+0000","last_became_active":"2024-04-25T11:35:13.434939+0000","last_became_peered":"2024-04-25T11:35:13.434939+0000","last_unstale":"2024-04-25T11:35:15.442045+0000","last_undegraded":"2024-04-25T11:35:15.442045+0000","last_fullsized":"2024-04-25T11:35:15.442045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:45.138962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672019+0000","last_change":"2024-04-25T11:35:14.437779+0000","last_active":"2024-04-25T11:35:15.672019+0000","last_peered":"2024-04-25T11:35:15.672019+0000","last_clean":"2024-04-25T11:35:15.672019+0000","last_became_active":"2024-04-25T11:35:14.437486+0000","last_became_peered":"2024-04-25T11:35:14.437486+0000","last_unstale":"2024-04-25T11:35:15.672019+0000","last_undegraded":"2024-04-25T11:35:15.672019+0000","last_fullsized":"2024-04-25T11:35:15.672019+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:22.633353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441254+0000","last_change":"2024-04-25T11:35:13.427281+0000","last_active":"2024-04-25T11:35:15.441254+0000","last_peered":"2024-04-25T11:35:15.441254+0000","last_clean":"2024-04-25T11:35:15.441254+0000","last_became_active":"2024-04-25T11:35:13.426949+0000","last_became_peered":"2024-04-25T11:35:13.426949+0000","last_unstale":"2024-04-25T11:35:15.441254+0000","last_undegraded":"2024-04-25T11:35:15.441254+0000","last_fullsized":"2024-04-25T11:35:15.441254+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:32.416300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974260+0000","last_change":"2024-04-25T11:35:14.446064+0000","last_active":"2024-04-25T11:35:15.974260+0000","last_peered":"2024-04-25T11:35:15.974260+0000","last_clean":"2024-04-25T11:35:15.974260+0000","last_became_active":"2024-04-25T11:35:14.445687+0000","last_became_peered":"2024-04-25T11:35:14.445687+0000","last_unstale":"2024-04-25T11:35:15.974260+0000","last_undegraded":"2024-04-25T11:35:15.974260+0000","last_fullsized":"2024-04-25T11:35:15.974260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:06.227086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669047+0000","last_change":"2024-04-25T11:35:13.430166+0000","last_active":"2024-04-25T11:35:15.669047+0000","last_peered":"2024-04-25T11:35:15.669047+0000","last_clean":"2024-04-25T11:35:15.669047+0000","last_became_active":"2024-04-25T11:35:13.426917+0000","last_became_peered":"2024-04-25T11:35:13.426917+0000","last_unstale":"2024-04-25T11:35:15.669047+0000","last_undegraded":"2024-04-25T11:35:15.669047+0000","last_fullsized":"2024-04-25T11:35:15.669047+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:12.994234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671141+0000","last_change":"2024-04-25T11:35:14.456199+0000","last_active":"2024-04-25T11:35:15.671141+0000","last_peered":"2024-04-25T11:35:15.671141+0000","last_clean":"2024-04-25T11:35:15.671141+0000","last_became_active":"2024-04-25T11:35:14.455892+0000","last_became_peered":"2024-04-25T11:35:14.455892+0000","last_unstale":"2024-04-25T11:35:15.671141+0000","last_undegraded":"2024-04-25T11:35:15.671141+0000","last_fullsized":"2024-04-25T11:35:15.671141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:05.855382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.975280+0000","last_change":"2024-04-25T11:35:13.430002+0000","last_active":"2024-04-25T11:35:15.975280+0000","last_peered":"2024-04-25T11:35:15.975280+0000","last_clean":"2024-04-25T11:35:15.975280+0000","last_became_active":"2024-04-25T11:35:13.427079+0000","last_became_peered":"2024-04-25T11:35:13.427079+0000","last_unstale":"2024-04-25T11:35:15.975280+0000","last_undegraded":"2024-04-25T11:35:15.975280+0000","last_fullsized":"2024-04-25T11:35:15.975280+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.387249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.975389+0000","last_change":"2024-04-25T11:35:14.443427+0000","last_active":"2024-04-25T11:35:15.975389+0000","last_peered":"2024-04-25T11:35:15.975389+0000","last_clean":"2024-04-25T11:35:15.975389+0000","last_became_active":"2024-04-25T11:35:14.442447+0000","last_became_peered":"2024-04-25T11:35:14.442447+0000","last_unstale":"2024-04-25T11:35:15.975389+0000","last_undegraded":"2024-04-25T11:35:15.975389+0000","last_fullsized":"2024-04-25T11:35:15.975389+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:21.444901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973317+0000","last_change":"2024-04-25T11:35:13.433743+0000","last_active":"2024-04-25T11:35:15.973317+0000","last_peered":"2024-04-25T11:35:15.973317+0000","last_clean":"2024-04-25T11:35:15.973317+0000","last_became_active":"2024-04-25T11:35:13.432855+0000","last_became_peered":"2024-04-25T11:35:13.432855+0000","last_unstale":"2024-04-25T11:35:15.973317+0000","last_undegraded":"2024-04-25T11:35:15.973317+0000","last_fullsized":"2024-04-25T11:35:15.973317+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:02.978409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440841+0000","last_change":"2024-04-25T11:35:14.447155+0000","last_active":"2024-04-25T11:35:15.440841+0000","last_peered":"2024-04-25T11:35:15.440841+0000","last_clean":"2024-04-25T11:35:15.440841+0000","last_became_active":"2024-04-25T11:35:14.446489+0000","last_became_peered":"2024-04-25T11:35:14.446489+0000","last_unstale":"2024-04-25T11:35:15.440841+0000","last_undegraded":"2024-04-25T11:35:15.440841+0000","last_fullsized":"2024-04-25T11:35:15.440841+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:18.826261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440440+0000","last_change":"2024-04-25T11:35:13.420146+0000","last_active":"2024-04-25T11:35:15.440440+0000","last_peered":"2024-04-25T11:35:15.440440+0000","last_clean":"2024-04-25T11:35:15.440440+0000","last_became_active":"2024-04-25T11:35:13.419810+0000","last_became_peered":"2024-04-25T11:35:13.419810+0000","last_unstale":"2024-04-25T11:35:15.440440+0000","last_undegraded":"2024-04-25T11:35:15.440440+0000","last_fullsized":"2024-04-25T11:35:15.440440+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:33:59.388917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671381+0000","last_change":"2024-04-25T11:35:14.457536+0000","last_active":"2024-04-25T11:35:15.671381+0000","last_peered":"2024-04-25T11:35:15.671381+0000","last_clean":"2024-04-25T11:35:15.671381+0000","last_became_active":"2024-04-25T11:35:14.457137+0000","last_became_peered":"2024-04-25T11:35:14.457137+0000","last_unstale":"2024-04-25T11:35:15.671381+0000","last_undegraded":"2024-04-25T11:35:15.671381+0000","last_fullsized":"2024-04-25T11:35:15.671381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:27.241606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442805+0000","last_change":"2024-04-25T11:35:13.434918+0000","last_active":"2024-04-25T11:35:15.442805+0000","last_peered":"2024-04-25T11:35:15.442805+0000","last_clean":"2024-04-25T11:35:15.442805+0000","last_became_active":"2024-04-25T11:35:13.434553+0000","last_became_peered":"2024-04-25T11:35:13.434553+0000","last_unstale":"2024-04-25T11:35:15.442805+0000","last_undegraded":"2024-04-25T11:35:15.442805+0000","last_fullsized":"2024-04-25T11:35:15.442805+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:52.649863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442631+0000","last_change":"2024-04-25T11:35:14.444864+0000","last_active":"2024-04-25T11:35:15.442631+0000","last_peered":"2024-04-25T11:35:15.442631+0000","last_clean":"2024-04-25T11:35:15.442631+0000","last_became_active":"2024-04-25T11:35:14.444515+0000","last_became_peered":"2024-04-25T11:35:14.444515+0000","last_unstale":"2024-04-25T11:35:15.442631+0000","last_undegraded":"2024-04-25T11:35:15.442631+0000","last_fullsized":"2024-04-25T11:35:15.442631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:47.831889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442190+0000","last_change":"2024-04-25T11:35:13.437418+0000","last_active":"2024-04-25T11:35:15.442190+0000","last_peered":"2024-04-25T11:35:15.442190+0000","last_clean":"2024-04-25T11:35:15.442190+0000","last_became_active":"2024-04-25T11:35:13.436540+0000","last_became_peered":"2024-04-25T11:35:13.436540+0000","last_unstale":"2024-04-25T11:35:15.442190+0000","last_undegraded":"2024-04-25T11:35:15.442190+0000","last_fullsized":"2024-04-25T11:35:15.442190+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:09.768799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443072+0000","last_change":"2024-04-25T11:35:14.454311+0000","last_active":"2024-04-25T11:35:15.443072+0000","last_peered":"2024-04-25T11:35:15.443072+0000","last_clean":"2024-04-25T11:35:15.443072+0000","last_became_active":"2024-04-25T11:35:14.453982+0000","last_became_peered":"2024-04-25T11:35:14.453982+0000","last_unstale":"2024-04-25T11:35:15.443072+0000","last_undegraded":"2024-04-25T11:35:15.443072+0000","last_fullsized":"2024-04-25T11:35:15.443072+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:14.552023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672265+0000","last_change":"2024-04-25T11:35:13.426296+0000","last_active":"2024-04-25T11:35:15.672265+0000","last_peered":"2024-04-25T11:35:15.672265+0000","last_clean":"2024-04-25T11:35:15.672265+0000","last_became_active":"2024-04-25T11:35:13.424459+0000","last_became_peered":"2024-04-25T11:35:13.424459+0000","last_unstale":"2024-04-25T11:35:15.672265+0000","last_undegraded":"2024-04-25T11:35:15.672265+0000","last_fullsized":"2024-04-25T11:35:15.672265+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:26.930066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.672261+0000","last_change":"2024-04-25T11:35:14.456394+0000","last_active":"2024-04-25T11:35:15.672261+0000","last_peered":"2024-04-25T11:35:15.672261+0000","last_clean":"2024-04-25T11:35:15.672261+0000","last_became_active":"2024-04-25T11:35:14.455379+0000","last_became_peered":"2024-04-25T11:35:14.455379+0000","last_unstale":"2024-04-25T11:35:15.672261+0000","last_undegraded":"2024-04-25T11:35:15.672261+0000","last_fullsized":"2024-04-25T11:35:15.672261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:33.434494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441263+0000","last_change":"2024-04-25T11:35:13.432537+0000","last_active":"2024-04-25T11:35:15.441263+0000","last_peered":"2024-04-25T11:35:15.441263+0000","last_clean":"2024-04-25T11:35:15.441263+0000","last_became_active":"2024-04-25T11:35:13.431989+0000","last_became_peered":"2024-04-25T11:35:13.431989+0000","last_unstale":"2024-04-25T11:35:15.441263+0000","last_undegraded":"2024-04-25T11:35:15.441263+0000","last_fullsized":"2024-04-25T11:35:15.441263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:24.918864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.669184+0000","last_change":"2024-04-25T11:35:14.451867+0000","last_active":"2024-04-25T11:35:15.669184+0000","last_peered":"2024-04-25T11:35:15.669184+0000","last_clean":"2024-04-25T11:35:15.669184+0000","last_became_active":"2024-04-25T11:35:14.451535+0000","last_became_peered":"2024-04-25T11:35:14.451535+0000","last_unstale":"2024-04-25T11:35:15.669184+0000","last_undegraded":"2024-04-25T11:35:15.669184+0000","last_fullsized":"2024-04-25T11:35:15.669184+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:10.665968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671330+0000","last_change":"2024-04-25T11:35:13.428207+0000","last_active":"2024-04-25T11:35:15.671330+0000","last_peered":"2024-04-25T11:35:15.671330+0000","last_clean":"2024-04-25T11:35:15.671330+0000","last_became_active":"2024-04-25T11:35:13.427902+0000","last_became_peered":"2024-04-25T11:35:13.427902+0000","last_unstale":"2024-04-25T11:35:15.671330+0000","last_undegraded":"2024-04-25T11:35:15.671330+0000","last_fullsized":"2024-04-25T11:35:15.671330+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.156013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441320+0000","last_change":"2024-04-25T11:35:14.454313+0000","last_active":"2024-04-25T11:35:15.441320+0000","last_peered":"2024-04-25T11:35:15.441320+0000","last_clean":"2024-04-25T11:35:15.441320+0000","last_became_active":"2024-04-25T11:35:14.453891+0000","last_became_peered":"2024-04-25T11:35:14.453891+0000","last_unstale":"2024-04-25T11:35:15.441320+0000","last_undegraded":"2024-04-25T11:35:15.441320+0000","last_fullsized":"2024-04-25T11:35:15.441320+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:09.121728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974562+0000","last_change":"2024-04-25T11:35:13.437399+0000","last_active":"2024-04-25T11:35:15.974562+0000","last_peered":"2024-04-25T11:35:15.974562+0000","last_clean":"2024-04-25T11:35:15.974562+0000","last_became_active":"2024-04-25T11:35:13.436580+0000","last_became_peered":"2024-04-25T11:35:13.436580+0000","last_unstale":"2024-04-25T11:35:15.974562+0000","last_undegraded":"2024-04-25T11:35:15.974562+0000","last_fullsized":"2024-04-25T11:35:15.974562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.567839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668510+0000","last_change":"2024-04-25T11:35:14.447867+0000","last_active":"2024-04-25T11:35:15.668510+0000","last_peered":"2024-04-25T11:35:15.668510+0000","last_clean":"2024-04-25T11:35:15.668510+0000","last_became_active":"2024-04-25T11:35:14.447416+0000","last_became_peered":"2024-04-25T11:35:14.447416+0000","last_unstale":"2024-04-25T11:35:15.668510+0000","last_undegraded":"2024-04-25T11:35:15.668510+0000","last_fullsized":"2024-04-25T11:35:15.668510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:14.552023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974873+0000","last_change":"2024-04-25T11:35:13.428944+0000","last_active":"2024-04-25T11:35:15.974873+0000","last_peered":"2024-04-25T11:35:15.974873+0000","last_clean":"2024-04-25T11:35:15.974873+0000","last_became_active":"2024-04-25T11:35:13.428595+0000","last_became_peered":"2024-04-25T11:35:13.428595+0000","last_unstale":"2024-04-25T11:35:15.974873+0000","last_undegraded":"2024-04-25T11:35:15.974873+0000","last_fullsized":"2024-04-25T11:35:15.974873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:05.329123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671365+0000","last_change":"2024-04-25T11:35:14.444313+0000","last_active":"2024-04-25T11:35:15.671365+0000","last_peered":"2024-04-25T11:35:15.671365+0000","last_clean":"2024-04-25T11:35:15.671365+0000","last_became_active":"2024-04-25T11:35:14.443696+0000","last_became_peered":"2024-04-25T11:35:14.443696+0000","last_unstale":"2024-04-25T11:35:15.671365+0000","last_undegraded":"2024-04-25T11:35:15.671365+0000","last_fullsized":"2024-04-25T11:35:15.671365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:02.357995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699623+0000","last_change":"2024-04-25T11:35:13.435321+0000","last_active":"2024-04-25T11:35:15.699623+0000","last_peered":"2024-04-25T11:35:15.699623+0000","last_clean":"2024-04-25T11:35:15.699623+0000","last_became_active":"2024-04-25T11:35:13.434947+0000","last_became_peered":"2024-04-25T11:35:13.434947+0000","last_unstale":"2024-04-25T11:35:15.699623+0000","last_undegraded":"2024-04-25T11:35:15.699623+0000","last_fullsized":"2024-04-25T11:35:15.699623+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:13.533829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442549+0000","last_change":"2024-04-25T11:35:14.458234+0000","last_active":"2024-04-25T11:35:15.442549+0000","last_peered":"2024-04-25T11:35:15.442549+0000","last_clean":"2024-04-25T11:35:15.442549+0000","last_became_active":"2024-04-25T11:35:14.457944+0000","last_became_peered":"2024-04-25T11:35:14.457944+0000","last_unstale":"2024-04-25T11:35:15.442549+0000","last_undegraded":"2024-04-25T11:35:15.442549+0000","last_fullsized":"2024-04-25T11:35:15.442549+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:03.996603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440371+0000","last_change":"2024-04-25T11:35:13.426786+0000","last_active":"2024-04-25T11:35:15.440371+0000","last_peered":"2024-04-25T11:35:15.440371+0000","last_clean":"2024-04-25T11:35:15.440371+0000","last_became_active":"2024-04-25T11:35:13.424232+0000","last_became_peered":"2024-04-25T11:35:13.424232+0000","last_unstale":"2024-04-25T11:35:15.440371+0000","last_undegraded":"2024-04-25T11:35:15.440371+0000","last_fullsized":"2024-04-25T11:35:15.440371+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:26.223412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668021+0000","last_change":"2024-04-25T11:35:14.458052+0000","last_active":"2024-04-25T11:35:15.668021+0000","last_peered":"2024-04-25T11:35:15.668021+0000","last_clean":"2024-04-25T11:35:15.668021+0000","last_became_active":"2024-04-25T11:35:14.455359+0000","last_became_peered":"2024-04-25T11:35:14.455359+0000","last_unstale":"2024-04-25T11:35:15.668021+0000","last_undegraded":"2024-04-25T11:35:15.668021+0000","last_fullsized":"2024-04-25T11:35:15.668021+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:47.831889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668091+0000","last_change":"2024-04-25T11:35:13.435283+0000","last_active":"2024-04-25T11:35:15.668091+0000","last_peered":"2024-04-25T11:35:15.668091+0000","last_clean":"2024-04-25T11:35:15.668091+0000","last_became_active":"2024-04-25T11:35:13.434976+0000","last_became_peered":"2024-04-25T11:35:13.434976+0000","last_unstale":"2024-04-25T11:35:15.668091+0000","last_undegraded":"2024-04-25T11:35:15.668091+0000","last_fullsized":"2024-04-25T11:35:15.668091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:52.649863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973753+0000","last_change":"2024-04-25T11:35:14.438682+0000","last_active":"2024-04-25T11:35:15.973753+0000","last_peered":"2024-04-25T11:35:15.973753+0000","last_clean":"2024-04-25T11:35:15.973753+0000","last_became_active":"2024-04-25T11:35:14.438334+0000","last_became_peered":"2024-04-25T11:35:14.438334+0000","last_unstale":"2024-04-25T11:35:15.973753+0000","last_undegraded":"2024-04-25T11:35:15.973753+0000","last_fullsized":"2024-04-25T11:35:15.973753+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:33.434494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442117+0000","last_change":"2024-04-25T11:35:13.437779+0000","last_active":"2024-04-25T11:35:15.442117+0000","last_peered":"2024-04-25T11:35:15.442117+0000","last_clean":"2024-04-25T11:35:15.442117+0000","last_became_active":"2024-04-25T11:35:13.437350+0000","last_became_peered":"2024-04-25T11:35:13.437350+0000","last_unstale":"2024-04-25T11:35:15.442117+0000","last_undegraded":"2024-04-25T11:35:15.442117+0000","last_fullsized":"2024-04-25T11:35:15.442117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:25.619380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441580+0000","last_change":"2024-04-25T11:35:14.435207+0000","last_active":"2024-04-25T11:35:15.441580+0000","last_peered":"2024-04-25T11:35:15.441580+0000","last_clean":"2024-04-25T11:35:15.441580+0000","last_became_active":"2024-04-25T11:35:14.434485+0000","last_became_peered":"2024-04-25T11:35:14.434485+0000","last_unstale":"2024-04-25T11:35:15.441580+0000","last_undegraded":"2024-04-25T11:35:15.441580+0000","last_fullsized":"2024-04-25T11:35:15.441580+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.524923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443505+0000","last_change":"2024-04-25T11:35:13.436647+0000","last_active":"2024-04-25T11:35:15.443505+0000","last_peered":"2024-04-25T11:35:15.443505+0000","last_clean":"2024-04-25T11:35:15.443505+0000","last_became_active":"2024-04-25T11:35:13.436262+0000","last_became_peered":"2024-04-25T11:35:13.436262+0000","last_unstale":"2024-04-25T11:35:15.443505+0000","last_undegraded":"2024-04-25T11:35:15.443505+0000","last_fullsized":"2024-04-25T11:35:15.443505+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:14.404919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974374+0000","last_change":"2024-04-25T11:35:14.439044+0000","last_active":"2024-04-25T11:35:15.974374+0000","last_peered":"2024-04-25T11:35:15.974374+0000","last_clean":"2024-04-25T11:35:15.974374+0000","last_became_active":"2024-04-25T11:35:14.438731+0000","last_became_peered":"2024-04-25T11:35:14.438731+0000","last_unstale":"2024-04-25T11:35:15.974374+0000","last_undegraded":"2024-04-25T11:35:15.974374+0000","last_fullsized":"2024-04-25T11:35:15.974374+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:46.184173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.444116+0000","last_change":"2024-04-25T11:35:13.436802+0000","last_active":"2024-04-25T11:35:15.444116+0000","last_peered":"2024-04-25T11:35:15.444116+0000","last_clean":"2024-04-25T11:35:15.444116+0000","last_became_active":"2024-04-25T11:35:13.436423+0000","last_became_peered":"2024-04-25T11:35:13.436423+0000","last_unstale":"2024-04-25T11:35:15.444116+0000","last_undegraded":"2024-04-25T11:35:15.444116+0000","last_fullsized":"2024-04-25T11:35:15.444116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.058527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699717+0000","last_change":"2024-04-25T11:35:14.457201+0000","last_active":"2024-04-25T11:35:15.699717+0000","last_peered":"2024-04-25T11:35:15.699717+0000","last_clean":"2024-04-25T11:35:15.699717+0000","last_became_active":"2024-04-25T11:35:14.456824+0000","last_became_peered":"2024-04-25T11:35:14.456824+0000","last_unstale":"2024-04-25T11:35:15.699717+0000","last_undegraded":"2024-04-25T11:35:15.699717+0000","last_fullsized":"2024-04-25T11:35:15.699717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:00.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442866+0000","last_change":"2024-04-25T11:35:13.427560+0000","last_active":"2024-04-25T11:35:15.442866+0000","last_peered":"2024-04-25T11:35:15.442866+0000","last_clean":"2024-04-25T11:35:15.442866+0000","last_became_active":"2024-04-25T11:35:13.427177+0000","last_became_peered":"2024-04-25T11:35:13.427177+0000","last_unstale":"2024-04-25T11:35:15.442866+0000","last_undegraded":"2024-04-25T11:35:15.442866+0000","last_fullsized":"2024-04-25T11:35:15.442866+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:17.312616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.439614+0000","last_change":"2024-04-25T11:35:14.443854+0000","last_active":"2024-04-25T11:35:15.439614+0000","last_peered":"2024-04-25T11:35:15.439614+0000","last_clean":"2024-04-25T11:35:15.439614+0000","last_became_active":"2024-04-25T11:35:14.443441+0000","last_became_peered":"2024-04-25T11:35:14.443441+0000","last_unstale":"2024-04-25T11:35:15.439614+0000","last_undegraded":"2024-04-25T11:35:15.439614+0000","last_fullsized":"2024-04-25T11:35:15.439614+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:59.530010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.699873+0000","last_change":"2024-04-25T11:35:13.430809+0000","last_active":"2024-04-25T11:35:15.699873+0000","last_peered":"2024-04-25T11:35:15.699873+0000","last_clean":"2024-04-25T11:35:15.699873+0000","last_became_active":"2024-04-25T11:35:13.430489+0000","last_became_peered":"2024-04-25T11:35:13.430489+0000","last_unstale":"2024-04-25T11:35:15.699873+0000","last_undegraded":"2024-04-25T11:35:15.699873+0000","last_fullsized":"2024-04-25T11:35:15.699873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.567839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.974521+0000","last_change":"2024-04-25T11:35:14.429807+0000","last_active":"2024-04-25T11:35:15.974521+0000","last_peered":"2024-04-25T11:35:15.974521+0000","last_clean":"2024-04-25T11:35:15.974521+0000","last_became_active":"2024-04-25T11:35:14.429360+0000","last_became_peered":"2024-04-25T11:35:14.429360+0000","last_unstale":"2024-04-25T11:35:15.974521+0000","last_undegraded":"2024-04-25T11:35:15.974521+0000","last_fullsized":"2024-04-25T11:35:15.974521+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.355878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.667846+0000","last_change":"2024-04-25T11:35:13.430997+0000","last_active":"2024-04-25T11:35:15.667846+0000","last_peered":"2024-04-25T11:35:15.667846+0000","last_clean":"2024-04-25T11:35:15.667846+0000","last_became_active":"2024-04-25T11:35:13.428031+0000","last_became_peered":"2024-04-25T11:35:13.428031+0000","last_unstale":"2024-04-25T11:35:15.667846+0000","last_undegraded":"2024-04-25T11:35:15.667846+0000","last_fullsized":"2024-04-25T11:35:15.667846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:00.428630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973879+0000","last_change":"2024-04-25T11:35:14.426072+0000","last_active":"2024-04-25T11:35:15.973879+0000","last_peered":"2024-04-25T11:35:15.973879+0000","last_clean":"2024-04-25T11:35:15.973879+0000","last_became_active":"2024-04-25T11:35:14.425752+0000","last_became_peered":"2024-04-25T11:35:14.425752+0000","last_unstale":"2024-04-25T11:35:15.973879+0000","last_undegraded":"2024-04-25T11:35:15.973879+0000","last_fullsized":"2024-04-25T11:35:15.973879+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:41.231371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442533+0000","last_change":"2024-04-25T11:35:13.431423+0000","last_active":"2024-04-25T11:35:15.442533+0000","last_peered":"2024-04-25T11:35:15.442533+0000","last_clean":"2024-04-25T11:35:15.442533+0000","last_became_active":"2024-04-25T11:35:13.430960+0000","last_became_peered":"2024-04-25T11:35:13.430960+0000","last_unstale":"2024-04-25T11:35:15.442533+0000","last_undegraded":"2024-04-25T11:35:15.442533+0000","last_fullsized":"2024-04-25T11:35:15.442533+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:09.962598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.442047+0000","last_change":"2024-04-25T11:35:14.431519+0000","last_active":"2024-04-25T11:35:15.442047+0000","last_peered":"2024-04-25T11:35:15.442047+0000","last_clean":"2024-04-25T11:35:15.442047+0000","last_became_active":"2024-04-25T11:35:14.431178+0000","last_became_peered":"2024-04-25T11:35:14.431178+0000","last_unstale":"2024-04-25T11:35:15.442047+0000","last_undegraded":"2024-04-25T11:35:15.442047+0000","last_fullsized":"2024-04-25T11:35:15.442047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:02.950848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.443031+0000","last_change":"2024-04-25T11:35:13.431541+0000","last_active":"2024-04-25T11:35:15.443031+0000","last_peered":"2024-04-25T11:35:15.443031+0000","last_clean":"2024-04-25T11:35:15.443031+0000","last_became_active":"2024-04-25T11:35:13.431197+0000","last_became_peered":"2024-04-25T11:35:13.431197+0000","last_unstale":"2024-04-25T11:35:15.443031+0000","last_undegraded":"2024-04-25T11:35:15.443031+0000","last_fullsized":"2024-04-25T11:35:15.443031+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:37.156013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441981+0000","last_change":"2024-04-25T11:35:14.424889+0000","last_active":"2024-04-25T11:35:15.441981+0000","last_peered":"2024-04-25T11:35:15.441981+0000","last_clean":"2024-04-25T11:35:15.441981+0000","last_became_active":"2024-04-25T11:35:14.424564+0000","last_became_peered":"2024-04-25T11:35:14.424564+0000","last_unstale":"2024-04-25T11:35:15.441981+0000","last_undegraded":"2024-04-25T11:35:15.441981+0000","last_fullsized":"2024-04-25T11:35:15.441981+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:58.700206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668814+0000","last_change":"2024-04-25T11:35:13.431620+0000","last_active":"2024-04-25T11:35:15.668814+0000","last_peered":"2024-04-25T11:35:15.668814+0000","last_clean":"2024-04-25T11:35:15.668814+0000","last_became_active":"2024-04-25T11:35:13.431300+0000","last_became_peered":"2024-04-25T11:35:13.431300+0000","last_unstale":"2024-04-25T11:35:15.668814+0000","last_undegraded":"2024-04-25T11:35:15.668814+0000","last_fullsized":"2024-04-25T11:35:15.668814+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:14.404919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440167+0000","last_change":"2024-04-25T11:35:14.434307+0000","last_active":"2024-04-25T11:35:15.440167+0000","last_peered":"2024-04-25T11:35:15.440167+0000","last_clean":"2024-04-25T11:35:15.440167+0000","last_became_active":"2024-04-25T11:35:14.432356+0000","last_became_peered":"2024-04-25T11:35:14.432356+0000","last_unstale":"2024-04-25T11:35:15.440167+0000","last_undegraded":"2024-04-25T11:35:15.440167+0000","last_fullsized":"2024-04-25T11:35:15.440167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:21.662801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973591+0000","last_change":"2024-04-25T11:35:13.436194+0000","last_active":"2024-04-25T11:35:15.973591+0000","last_peered":"2024-04-25T11:35:15.973591+0000","last_clean":"2024-04-25T11:35:15.973591+0000","last_became_active":"2024-04-25T11:35:13.434670+0000","last_became_peered":"2024-04-25T11:35:13.434670+0000","last_unstale":"2024-04-25T11:35:15.973591+0000","last_undegraded":"2024-04-25T11:35:15.973591+0000","last_fullsized":"2024-04-25T11:35:15.973591+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:46.813695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441132+0000","last_change":"2024-04-25T11:35:14.441216+0000","last_active":"2024-04-25T11:35:15.441132+0000","last_peered":"2024-04-25T11:35:15.441132+0000","last_clean":"2024-04-25T11:35:15.441132+0000","last_became_active":"2024-04-25T11:35:14.440903+0000","last_became_peered":"2024-04-25T11:35:14.440903+0000","last_unstale":"2024-04-25T11:35:15.441132+0000","last_undegraded":"2024-04-25T11:35:15.441132+0000","last_fullsized":"2024-04-25T11:35:15.441132+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:31.972290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440835+0000","last_change":"2024-04-25T11:35:13.429491+0000","last_active":"2024-04-25T11:35:15.440835+0000","last_peered":"2024-04-25T11:35:15.440835+0000","last_clean":"2024-04-25T11:35:15.440835+0000","last_became_active":"2024-04-25T11:35:13.427659+0000","last_became_peered":"2024-04-25T11:35:13.427659+0000","last_unstale":"2024-04-25T11:35:15.440835+0000","last_undegraded":"2024-04-25T11:35:15.440835+0000","last_fullsized":"2024-04-25T11:35:15.440835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:42.804232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671105+0000","last_change":"2024-04-25T11:35:14.431959+0000","last_active":"2024-04-25T11:35:15.671105+0000","last_peered":"2024-04-25T11:35:15.671105+0000","last_clean":"2024-04-25T11:35:15.671105+0000","last_became_active":"2024-04-25T11:35:14.428698+0000","last_became_peered":"2024-04-25T11:35:14.428698+0000","last_unstale":"2024-04-25T11:35:15.671105+0000","last_undegraded":"2024-04-25T11:35:15.671105+0000","last_fullsized":"2024-04-25T11:35:15.671105+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:16.722343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.700050+0000","last_change":"2024-04-25T11:35:13.425327+0000","last_active":"2024-04-25T11:35:15.700050+0000","last_peered":"2024-04-25T11:35:15.700050+0000","last_clean":"2024-04-25T11:35:15.700050+0000","last_became_active":"2024-04-25T11:35:13.424970+0000","last_became_peered":"2024-04-25T11:35:13.424970+0000","last_unstale":"2024-04-25T11:35:15.700050+0000","last_undegraded":"2024-04-25T11:35:15.700050+0000","last_fullsized":"2024-04-25T11:35:15.700050+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:24.796451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668640+0000","last_change":"2024-04-25T11:35:14.441685+0000","last_active":"2024-04-25T11:35:15.668640+0000","last_peered":"2024-04-25T11:35:15.668640+0000","last_clean":"2024-04-25T11:35:15.668640+0000","last_became_active":"2024-04-25T11:35:14.441168+0000","last_became_peered":"2024-04-25T11:35:14.441168+0000","last_unstale":"2024-04-25T11:35:15.668640+0000","last_undegraded":"2024-04-25T11:35:15.668640+0000","last_fullsized":"2024-04-25T11:35:15.668640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:25.814645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.440527+0000","last_change":"2024-04-25T11:35:13.420152+0000","last_active":"2024-04-25T11:35:15.440527+0000","last_peered":"2024-04-25T11:35:15.440527+0000","last_clean":"2024-04-25T11:35:15.440527+0000","last_became_active":"2024-04-25T11:35:13.419769+0000","last_became_peered":"2024-04-25T11:35:13.419769+0000","last_unstale":"2024-04-25T11:35:15.440527+0000","last_undegraded":"2024-04-25T11:35:15.440527+0000","last_fullsized":"2024-04-25T11:35:15.440527+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:21.615159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671336+0000","last_change":"2024-04-25T11:35:14.427179+0000","last_active":"2024-04-25T11:35:15.671336+0000","last_peered":"2024-04-25T11:35:15.671336+0000","last_clean":"2024-04-25T11:35:15.671336+0000","last_became_active":"2024-04-25T11:35:14.426803+0000","last_became_peered":"2024-04-25T11:35:14.426803+0000","last_unstale":"2024-04-25T11:35:15.671336+0000","last_undegraded":"2024-04-25T11:35:15.671336+0000","last_fullsized":"2024-04-25T11:35:15.671336+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:10.665968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.670703+0000","last_change":"2024-04-25T11:35:13.420223+0000","last_active":"2024-04-25T11:35:15.670703+0000","last_peered":"2024-04-25T11:35:15.670703+0000","last_clean":"2024-04-25T11:35:15.670703+0000","last_became_active":"2024-04-25T11:35:13.419888+0000","last_became_peered":"2024-04-25T11:35:13.419888+0000","last_unstale":"2024-04-25T11:35:15.670703+0000","last_undegraded":"2024-04-25T11:35:15.670703+0000","last_fullsized":"2024-04-25T11:35:15.670703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.058527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.670804+0000","last_change":"2024-04-25T11:35:14.444665+0000","last_active":"2024-04-25T11:35:15.670804+0000","last_peered":"2024-04-25T11:35:15.670804+0000","last_clean":"2024-04-25T11:35:15.670804+0000","last_became_active":"2024-04-25T11:35:14.444348+0000","last_became_peered":"2024-04-25T11:35:14.444348+0000","last_unstale":"2024-04-25T11:35:15.670804+0000","last_undegraded":"2024-04-25T11:35:15.670804+0000","last_fullsized":"2024-04-25T11:35:15.670804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:25.814645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.973146+0000","last_change":"2024-04-25T11:35:13.426511+0000","last_active":"2024-04-25T11:35:15.973146+0000","last_peered":"2024-04-25T11:35:15.973146+0000","last_clean":"2024-04-25T11:35:15.973146+0000","last_became_active":"2024-04-25T11:35:13.426183+0000","last_became_peered":"2024-04-25T11:35:13.426183+0000","last_unstale":"2024-04-25T11:35:15.973146+0000","last_undegraded":"2024-04-25T11:35:15.973146+0000","last_fullsized":"2024-04-25T11:35:15.973146+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:34.938031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.668638+0000","last_change":"2024-04-25T11:35:14.451135+0000","last_active":"2024-04-25T11:35:15.668638+0000","last_peered":"2024-04-25T11:35:15.668638+0000","last_clean":"2024-04-25T11:35:15.668638+0000","last_became_active":"2024-04-25T11:35:14.446640+0000","last_became_peered":"2024-04-25T11:35:14.446640+0000","last_unstale":"2024-04-25T11:35:15.668638+0000","last_undegraded":"2024-04-25T11:35:15.668638+0000","last_fullsized":"2024-04-25T11:35:15.668638+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:13.378029+0000","last_clean_scrub_stamp":"2024-04-25T11:35:13.378029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:16.722343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.441712+0000","last_change":"2024-04-25T11:35:13.428876+0000","last_active":"2024-04-25T11:35:15.441712+0000","last_peered":"2024-04-25T11:35:15.441712+0000","last_clean":"2024-04-25T11:35:15.441712+0000","last_became_active":"2024-04-25T11:35:13.422642+0000","last_became_peered":"2024-04-25T11:35:13.422642+0000","last_unstale":"2024-04-25T11:35:15.441712+0000","last_undegraded":"2024-04-25T11:35:15.441712+0000","last_fullsized":"2024-04-25T11:35:15.441712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:35:12.359835+0000","last_clean_scrub_stamp":"2024-04-25T11:35:12.359835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:40.031617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:35:15.671725+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:35:15.671725+0000","last_peered":"2024-04-25T11:35:15.671725+0000","last_clean":"2024-04-25T11:35:15.671725+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:35:15.671725+0000","last_undegraded":"2024-04-25T11:35:15.671725+0000","last_fullsized":"2024-04-25T11:35:15.671725+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":484609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1130496,"data_stored":485211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":484573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509541,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":485213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509541,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":484539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509541,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":1074997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":486173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1720320,"data_stored":1070253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:35:19.793 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:35:19.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:35:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.137+0000 7fd35d200640 1 Processor -- start 2024-04-25T11:35:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.137+0000 7fd35d200640 1 -- start start 2024-04-25T11:35:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd35d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd35806dc30 0x7fd3580af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd35d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd35806d7c0 0x7fd3580b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd35d200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35809b870 con 0x7fd358160390 2024-04-25T11:35:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd35d200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd35809bb50 con 0x7fd35806dc30 2024-04-25T11:35:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd35d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd3580b1a70 con 0x7fd35806d7c0 2024-04-25T11:35:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd357400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd35806dc30 0x7fd3580af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd356a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd358160390 0x7fd35806d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35892/0 (socket says 172.21.15.19:35892) 2024-04-25T11:35:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.140+0000 7fd356a00640 1 -- 172.21.15.19:0/502296900 learned_addr learned my addr 172.21.15.19:0/502296900 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.141+0000 7fd357e00640 1 --2- 172.21.15.19:0/502296900 >> v2:172.21.15.169:3300/0 conn(0x7fd35806d7c0 0x7fd3580b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.141+0000 7fd356000640 1 -- 172.21.15.19:0/502296900 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3550211936 0 0) 0x7fd35809b870 con 0x7fd358160390 2024-04-25T11:35:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.141+0000 7fd356000640 1 -- 172.21.15.19:0/502296900 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd33c003520 con 0x7fd358160390 2024-04-25T11:35:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.141+0000 7fd357400640 1 -- 172.21.15.19:0/502296900 >> v2:172.21.15.169:3300/0 conn(0x7fd35806d7c0 msgr2=0x7fd3580b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.141+0000 7fd357400640 1 --2- 172.21.15.19:0/502296900 >> v2:172.21.15.169:3300/0 conn(0x7fd35806d7c0 0x7fd3580b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.141+0000 7fd357400640 1 -- 172.21.15.19:0/502296900 >> v1:172.21.15.19:6789/0 conn(0x7fd358160390 legacy=0x7fd35806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.141+0000 7fd357400640 1 -- 172.21.15.19:0/502296900 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3580b1d50 con 0x7fd35806dc30 2024-04-25T11:35:20.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.142+0000 7fd357400640 1 --2- 172.21.15.19:0/502296900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35806dc30 0x7fd3580af1d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd3580b1760 tx=0x7fd35002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26af5d18eb555f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:20.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.142+0000 7fd356000640 1 -- 172.21.15.19:0/502296900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350007740 con 0x7fd35806dc30 2024-04-25T11:35:20.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.142+0000 7fd356000640 1 -- 172.21.15.19:0/502296900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3500078e0 con 0x7fd35806dc30 2024-04-25T11:35:20.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.142+0000 7fd356000640 1 -- 172.21.15.19:0/502296900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350007cb0 con 0x7fd35806dc30 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 -- 172.21.15.19:0/502296900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35806dc30 msgr2=0x7fd3580af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 --2- 172.21.15.19:0/502296900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35806dc30 0x7fd3580af1d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd3580b1760 tx=0x7fd35002a560 comp rx=0 tx=0).stop 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 -- 172.21.15.19:0/502296900 shutdown_connections 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 --2- 172.21.15.19:0/502296900 >> v2:172.21.15.169:3300/0 conn(0x7fd35806d7c0 0x7fd3580b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 --2- 172.21.15.19:0/502296900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35806dc30 0x7fd3580af1d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 -- 172.21.15.19:0/502296900 >> 172.21.15.19:0/502296900 conn(0x7fd358073920 msgr2=0x7fd3580b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 -- 172.21.15.19:0/502296900 shutdown_connections 2024-04-25T11:35:20.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.143+0000 7fd35d200640 1 -- 172.21.15.19:0/502296900 wait complete. 2024-04-25T11:35:20.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.146+0000 7fd35d200640 1 Processor -- start 2024-04-25T11:35:20.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.146+0000 7fd35d200640 1 -- start start 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.147+0000 7fd35d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35806d7c0 0x7fd3580cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.147+0000 7fd35d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd358160390 0x7fd358162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.147+0000 7fd35d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35814b2a0 0x7fd35814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.147+0000 7fd35d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580af710 con 0x7fd35814b2a0 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.147+0000 7fd35d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd358076420 con 0x7fd358160390 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.147+0000 7fd35d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd3580af890 con 0x7fd35806d7c0 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35814b2a0 0x7fd35814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35814b2a0 0x7fd35814b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36554/0 (socket says 172.21.15.19:36554) 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 -- 172.21.15.19:0/517391927 learned_addr learned my addr 172.21.15.19:0/517391927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357400640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35806d7c0 0x7fd3580cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd356a00640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd358160390 0x7fd358162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 -- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35806d7c0 msgr2=0x7fd3580cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35806d7c0 0x7fd3580cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 -- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd358160390 msgr2=0x7fd358162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd358160390 0x7fd358162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 -- 172.21.15.19:0/517391927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3580ce190 con 0x7fd35814b2a0 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd356a00640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd358160390 0x7fd358162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.148+0000 7fd357e00640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35814b2a0 0x7fd35814b6b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d5d0 tx=0x7fd34800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.149+0000 7fd34f400640 1 -- 172.21.15.19:0/517391927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480125f0 con 0x7fd35814b2a0 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.149+0000 7fd34f400640 1 -- 172.21.15.19:0/517391927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd348019070 con 0x7fd35814b2a0 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.149+0000 7fd34f400640 1 -- 172.21.15.19:0/517391927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480129f0 con 0x7fd35814b2a0 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.149+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3580cb7c0 con 0x7fd35814b2a0 2024-04-25T11:35:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.149+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3580cbd80 con 0x7fd35814b2a0 2024-04-25T11:35:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.150+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35809d5a0 con 0x7fd35814b2a0 2024-04-25T11:35:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.151+0000 7fd34f400640 1 -- 172.21.15.19:0/517391927 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd348012b90 con 0x7fd35814b2a0 2024-04-25T11:35:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.152+0000 7fd34f400640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3340744b0 0x7fd3340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.152+0000 7fd34f400640 1 -- 172.21.15.19:0/517391927 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fd3480956b0 con 0x7fd35814b2a0 2024-04-25T11:35:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.153+0000 7fd357400640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3340744b0 0x7fd3340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.155+0000 7fd357400640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3340744b0 0x7fd3340769c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd350034000 tx=0x7fd35002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.157+0000 7fd34f400640 1 -- 172.21.15.19:0/517391927 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd34805fdb0 con 0x7fd35814b2a0 2024-04-25T11:35:20.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.434+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd358076c60 con 0x7fd35814b2a0 2024-04-25T11:35:20.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.440+0000 7fd34f400640 1 -- 172.21.15.19:0/517391927 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fd34805f6b0 con 0x7fd35814b2a0 2024-04-25T11:35:20.429 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T11:35:20.429 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:35:20.429 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T11:35:20.429 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T11:35:20.429 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:35:20.429 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T11:35:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3340744b0 msgr2=0x7fd3340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3340744b0 0x7fd3340769c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd350034000 tx=0x7fd35002f000 comp rx=0 tx=0).stop 2024-04-25T11:35:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35814b2a0 msgr2=0x7fd35814b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35814b2a0 0x7fd35814b6b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d5d0 tx=0x7fd34800da90 comp rx=0 tx=0).stop 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 shutdown_connections 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd3340744b0 0x7fd3340769c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd358160390 0x7fd358162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd35814b2a0 0x7fd35814b6b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 --2- 172.21.15.19:0/517391927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35806d7c0 0x7fd3580cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 >> 172.21.15.19:0/517391927 conn(0x7fd358073920 msgr2=0x7fd358070760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 shutdown_connections 2024-04-25T11:35:20.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.444+0000 7fd35d200640 1 -- 172.21.15.19:0/517391927 wait complete. 2024-04-25T11:35:20.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:35:20.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.741+0000 7f506fe00640 1 Processor -- start 2024-04-25T11:35:20.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.741+0000 7f506fe00640 1 -- start start 2024-04-25T11:35:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.743+0000 7f506fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f507006c5c0 0x7f50700cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.743+0000 7f506fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50700d1230 con 0x7f50700b4ea0 2024-04-25T11:35:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.743+0000 7f506fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50700d06b0 con 0x7f5070144370 2024-04-25T11:35:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.743+0000 7f506fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f50700d0990 con 0x7f507006c5c0 2024-04-25T11:35:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.743+0000 7f506f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f50700b4ea0 0x7f507015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35896/0 (socket says 172.21.15.19:35896) 2024-04-25T11:35:20.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.743+0000 7f506f400640 1 -- 172.21.15.19:0/1736240329 learned_addr learned my addr 172.21.15.19:0/1736240329 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2849571793 0 0) 0x7f50700d1230 con 0x7f50700b4ea0 2024-04-25T11:35:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5054003520 con 0x7f50700b4ea0 2024-04-25T11:35:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506ea00640 1 --2- 172.21.15.19:0/1736240329 >> v2:172.21.15.169:3300/0 conn(0x7f507006c5c0 0x7f50700cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 788023105 0 0) 0x7f5054003520 con 0x7f50700b4ea0 2024-04-25T11:35:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 >> v2:172.21.15.169:3300/0 conn(0x7f507006c5c0 msgr2=0x7f50700cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 --2- 172.21.15.19:0/1736240329 >> v2:172.21.15.169:3300/0 conn(0x7f507006c5c0 0x7f50700cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 >> v1:172.21.15.169:6789/0 conn(0x7f5070144370 legacy=0x7f50700b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50700d0c70 con 0x7f50700b4ea0 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f50680019f0 con 0x7f50700b4ea0 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 >> v1:172.21.15.19:6789/0 conn(0x7f50700b4ea0 legacy=0x7f507015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50540039b0 0x7f5054003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5054004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5054005270 0x7f505400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50700d0990 con 0x7f5054004800 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f505400c200 con 0x7f5054005270 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.744+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f505400c400 con 0x7f50540039b0 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f5067e00640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5054004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f506f400640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5054005270 0x7f505400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f5067e00640 1 -- 172.21.15.19:0/1736240329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50540039b0 msgr2=0x7f5054003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f5067e00640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50540039b0 0x7f5054003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f5067e00640 1 -- 172.21.15.19:0/1736240329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5054005270 msgr2=0x7f505400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f5067e00640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5054005270 0x7f505400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f5067e00640 1 -- 172.21.15.19:0/1736240329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5054003520 con 0x7f5054004800 2024-04-25T11:35:20.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.745+0000 7f506f400640 1 -- 172.21.15.19:0/1736240329 reap_dead start 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.746+0000 7f5067e00640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5054004c10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5058001750 tx=0x7f5054005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadcc51edcb39b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.746+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058001370 con 0x7f5054004800 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.746+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50580052d0 con 0x7f5054004800 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.746+0000 7f506e000640 1 -- 172.21.15.19:0/1736240329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505802abc0 con 0x7f5054004800 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.747+0000 7f506fe00640 1 -- 172.21.15.19:0/1736240329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 msgr2=0x7f5054004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.747+0000 7f506fe00640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5054004c10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5058001750 tx=0x7f5054005830 comp rx=0 tx=0).stop 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.747+0000 7f506fe00640 1 -- 172.21.15.19:0/1736240329 shutdown_connections 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.747+0000 7f506fe00640 1 --2- 172.21.15.19:0/1736240329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5054004c10 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.747+0000 7f506fe00640 1 -- 172.21.15.19:0/1736240329 >> 172.21.15.19:0/1736240329 conn(0x7f50700d1720 msgr2=0x7f50700d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.747+0000 7f506fe00640 1 -- 172.21.15.19:0/1736240329 shutdown_connections 2024-04-25T11:35:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.747+0000 7f506fe00640 1 -- 172.21.15.19:0/1736240329 wait complete. 2024-04-25T11:35:20.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.750+0000 7f506fe00640 1 Processor -- start 2024-04-25T11:35:20.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.750+0000 7f506fe00640 1 -- start start 2024-04-25T11:35:20.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.751+0000 7f506fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5070070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.751+0000 7f506fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50701f5d40 0x7f5070141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.751+0000 7f506fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50701f6150 0x7f5070141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.751+0000 7f506fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50700d0450 con 0x7f5054004800 2024-04-25T11:35:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.751+0000 7f506fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50700d02d0 con 0x7f50701f6150 2024-04-25T11:35:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.751+0000 7f506fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5070147760 con 0x7f50701f5d40 2024-04-25T11:35:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.751+0000 7f506ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5070070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5070070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36562/0 (socket says 172.21.15.19:36562) 2024-04-25T11:35:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 -- 172.21.15.19:0/3582912314 learned_addr learned my addr 172.21.15.19:0/3582912314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 -- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50701f5d40 msgr2=0x7f5070141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50701f5d40 0x7f5070141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 -- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50701f6150 msgr2=0x7f5070141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50701f6150 0x7f5070141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 -- 172.21.15.19:0/3582912314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507006f660 con 0x7f5054004800 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f506ea00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5070070640 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f506000d560 tx=0x7f506000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.752+0000 7f5066a00640 1 -- 172.21.15.19:0/3582912314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50600125f0 con 0x7f5054004800 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.753+0000 7f5066a00640 1 -- 172.21.15.19:0/3582912314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5060019070 con 0x7f5054004800 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.753+0000 7f5066a00640 1 -- 172.21.15.19:0/3582912314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50600129d0 con 0x7f5054004800 2024-04-25T11:35:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.753+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507006f940 con 0x7f5054004800 2024-04-25T11:35:20.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.753+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507006ff00 con 0x7f5054004800 2024-04-25T11:35:20.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.754+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5070050740 con 0x7f5054004800 2024-04-25T11:35:20.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.755+0000 7f5066a00640 1 -- 172.21.15.19:0/3582912314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5060012b70 con 0x7f5054004800 2024-04-25T11:35:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.756+0000 7f5066a00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f504c0744b0 0x7f504c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.756+0000 7f5066a00640 1 -- 172.21.15.19:0/3582912314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f50600956a0 con 0x7f5054004800 2024-04-25T11:35:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.757+0000 7f5067e00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f504c0744b0 0x7f504c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:20.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.758+0000 7f5067e00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f504c0744b0 0x7f504c0769c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5058032000 tx=0x7f505802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:20.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.760+0000 7f5066a00640 1 -- 172.21.15.19:0/3582912314 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f506005fd90 con 0x7f5054004800 2024-04-25T11:35:20.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.994+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f507015ddd0 con 0x7f5054004800 2024-04-25T11:35:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.996+0000 7f5066a00640 1 -- 172.21.15.19:0/3582912314 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5189 (secure 0 0 0) 0x7f506005f690 con 0x7f5054004800 2024-04-25T11:35:20.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:20.984 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:35:17.059328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2716758396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2716758396},{"type":"v1","addr":"172.21.15.19:6835","nonce":2716758396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7374,"qdb_cluster":[7374]},"id":6}]} 2024-04-25T11:35:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 90 2024-04-25T11:35:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f504c0744b0 msgr2=0x7f504c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f504c0744b0 0x7f504c0769c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5058032000 tx=0x7f505802c000 comp rx=0 tx=0).stop 2024-04-25T11:35:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 msgr2=0x7f5070070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5070070640 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f506000d560 tx=0x7f506000da20 comp rx=0 tx=0).stop 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 shutdown_connections 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f504c0744b0 0x7f504c0769c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50701f6150 0x7f5070141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50701f5d40 0x7f5070141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 --2- 172.21.15.19:0/3582912314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054004800 0x7f5070070640 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 >> 172.21.15.19:0/3582912314 conn(0x7f50700d1720 msgr2=0x7f5070072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.998+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 shutdown_connections 2024-04-25T11:35:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:20.999+0000 7f506fe00640 1 -- 172.21.15.19:0/3582912314 wait complete. 2024-04-25T11:35:21.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:35:17.059328+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7374}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7374': {'gid': 7374, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/2716758396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2716758396}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2716758396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7374, 'qdb_cluster': [7374]} 2024-04-25T11:35:21.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:35:21.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:35:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.715+0000 7f799d000640 1 Processor -- start 2024-04-25T11:35:21.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.716+0000 7f799d000640 1 -- start start 2024-04-25T11:35:21.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.718+0000 7f799d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f7998070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.718+0000 7f799d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7998162310 0x7f79980ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.718+0000 7f799d000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f799806f5f0 con 0x7f79980cdb90 2024-04-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.718+0000 7f799d000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7998070200 con 0x7f7998162310 2024-04-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.718+0000 7f799d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f799806f8d0 con 0x7f7998144500 2024-04-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.718+0000 7f7996800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7998162310 0x7f79980ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.719+0000 7f7996800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7998162310 0x7f79980ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36572/0 (socket says 172.21.15.19:36572) 2024-04-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.719+0000 7f7996800640 1 -- 172.21.15.19:0/3460809278 learned_addr learned my addr 172.21.15.19:0/3460809278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.719+0000 7f7995e00640 1 --2- 172.21.15.19:0/3460809278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f7998070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.719+0000 7f7996800640 1 -- 172.21.15.19:0/3460809278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 msgr2=0x7f7998070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.719+0000 7f7996800640 1 --2- 172.21.15.19:0/3460809278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f7998070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.719+0000 7f7996800640 1 -- 172.21.15.19:0/3460809278 >> v1:172.21.15.169:6789/0 conn(0x7f79980cdb90 legacy=0x7f7998161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.719+0000 7f7996800640 1 -- 172.21.15.19:0/3460809278 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799806fbb0 con 0x7f7998162310 2024-04-25T11:35:21.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.720+0000 7f7996800640 1 --2- 172.21.15.19:0/3460809278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7998162310 0x7f79980ce8c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f7990009e50 tx=0x7f799002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13112cabccd342ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.720+0000 7f7995400640 1 -- 172.21.15.19:0/3460809278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990036070 con 0x7f7998162310 2024-04-25T11:35:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f7995400640 1 -- 172.21.15.19:0/3460809278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7990031600 con 0x7f7998162310 2024-04-25T11:35:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f7995400640 1 -- 172.21.15.19:0/3460809278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79900319d0 con 0x7f7998162310 2024-04-25T11:35:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f799d000640 1 -- 172.21.15.19:0/3460809278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7998162310 msgr2=0x7f79980ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f799d000640 1 --2- 172.21.15.19:0/3460809278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7998162310 0x7f79980ce8c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f7990009e50 tx=0x7f799002a6c0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f799d000640 1 -- 172.21.15.19:0/3460809278 shutdown_connections 2024-04-25T11:35:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f799d000640 1 --2- 172.21.15.19:0/3460809278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7998162310 0x7f79980ce8c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f799d000640 1 --2- 172.21.15.19:0/3460809278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f7998070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f799d000640 1 -- 172.21.15.19:0/3460809278 >> 172.21.15.19:0/3460809278 conn(0x7f7998073050 msgr2=0x7f7998073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.721+0000 7f799d000640 1 -- 172.21.15.19:0/3460809278 shutdown_connections 2024-04-25T11:35:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.722+0000 7f799d000640 1 -- 172.21.15.19:0/3460809278 wait complete. 2024-04-25T11:35:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.724+0000 7f799d000640 1 Processor -- start 2024-04-25T11:35:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.724+0000 7f799d000640 1 -- start start 2024-04-25T11:35:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.725+0000 7f799d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79980cdb90 0x7f799813f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f799d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f79981f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f799d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79981f6db0 0x7f79980cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f799d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f799806f2e0 con 0x7f79980cdb90 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f799d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f799806f160 con 0x7f79981f6db0 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f799d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79980cdfa0 con 0x7f7998144500 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79980cdb90 0x7f799813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79980cdb90 0x7f799813f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36588/0 (socket says 172.21.15.19:36588) 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 -- 172.21.15.19:0/3202942780 learned_addr learned my addr 172.21.15.19:0/3202942780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f798fe00640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f79981f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 -- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 msgr2=0x7f79981f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f79981f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 -- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79981f6db0 msgr2=0x7f79980cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79981f6db0 0x7f79980cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f7995e00640 1 -- 172.21.15.19:0/3202942780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7998145a90 con 0x7f79980cdb90 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.726+0000 7f798fe00640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f79981f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:21.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.727+0000 7f7995e00640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79980cdb90 0x7f799813f190 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f798800c820 tx=0x7f798800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.727+0000 7f798e000640 1 -- 172.21.15.19:0/3202942780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988015070 con 0x7f79980cdb90 2024-04-25T11:35:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.727+0000 7f798e000640 1 -- 172.21.15.19:0/3202942780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7988004020 con 0x7f79980cdb90 2024-04-25T11:35:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.727+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7998145d70 con 0x7f79980cdb90 2024-04-25T11:35:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.727+0000 7f798e000640 1 -- 172.21.15.19:0/3202942780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880043f0 con 0x7f79980cdb90 2024-04-25T11:35:21.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.727+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7998146330 con 0x7f79980cdb90 2024-04-25T11:35:21.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.730+0000 7f798e000640 1 -- 172.21.15.19:0/3202942780 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79880077d0 con 0x7f79980cdb90 2024-04-25T11:35:21.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.731+0000 7f798e000640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79740744b0 0x7f79740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.731+0000 7f798e000640 1 -- 172.21.15.19:0/3202942780 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f7988094f70 con 0x7f79980cdb90 2024-04-25T11:35:21.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.731+0000 7f79a4050740 1 Processor -- start 2024-04-25T11:35:21.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.731+0000 7f79a4050740 1 -- start start 2024-04-25T11:35:21.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.731+0000 7f798fe00640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79740744b0 0x7f79740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f79a4050740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d4b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d65d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 0x55fe13d662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f79a4050740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fe13c84600 con 0x55fe1397e270 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fe13c21100 con 0x55fe1395d590 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fe13c805c0 con 0x55fe139553b0 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f798fe00640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79740744b0 0x7f79740769c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7998147910 tx=0x7f7980008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d4b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d4b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36598/0 (socket says 172.21.15.19:36598) 2024-04-25T11:35:21.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 -- 172.21.15.19:0/4190704150 learned_addr learned my addr 172.21.15.19:0/4190704150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 -- 172.21.15.19:0/4190704150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 msgr2=0x55fe13d65d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 --2- 172.21.15.19:0/4190704150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d65d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 -- 172.21.15.19:0/4190704150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 msgr2=0x55fe13d662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 --2- 172.21.15.19:0/4190704150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 0x55fe13d662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.733+0000 7f7995e00640 1 -- 172.21.15.19:0/4190704150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe139962d0 con 0x55fe1397e270 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.734+0000 7f7995e00640 1 --2- 172.21.15.19:0/4190704150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d4b040 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f798805b100 tx=0x7f798805eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5acd31f940180dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.734+0000 7f7986a00640 1 -- 172.21.15.19:0/4190704150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798801e040 con 0x55fe1397e270 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.734+0000 7f7986a00640 1 -- 172.21.15.19:0/4190704150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798805f080 con 0x55fe1397e270 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.734+0000 7f7986a00640 1 -- 172.21.15.19:0/4190704150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988063430 con 0x55fe1397e270 2024-04-25T11:35:21.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 -- 172.21.15.19:0/4190704150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 msgr2=0x55fe13d4b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 --2- 172.21.15.19:0/4190704150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d4b040 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f798805b100 tx=0x7f798805eb90 comp rx=0 tx=0).stop 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 -- 172.21.15.19:0/4190704150 shutdown_connections 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 --2- 172.21.15.19:0/4190704150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 0x55fe13d662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 --2- 172.21.15.19:0/4190704150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d65d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 --2- 172.21.15.19:0/4190704150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d4b040 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 -- 172.21.15.19:0/4190704150 >> 172.21.15.19:0/4190704150 conn(0x55fe139723c0 msgr2=0x55fe13d206a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 -- 172.21.15.19:0/4190704150 shutdown_connections 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.735+0000 7f79a4050740 1 -- 172.21.15.19:0/4190704150 wait complete. 2024-04-25T11:35:21.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.736+0000 7f79a4050740 1 -- start start 2024-04-25T11:35:21.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.736+0000 7f79a4050740 1 Processor -- start 2024-04-25T11:35:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d46a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 0x55fe13d5f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f79a4050740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f79a4050740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fe13c0bf70 con 0x55fe1397e270 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fe13d4e3e0 con 0x55fe1395d590 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fe13d79aa0 con 0x55fe139553b0 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f79a4050740 10 client.0 _ll_register_callbacks cb 0x55fe13c7a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d78ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36614/0 (socket says 172.21.15.19:36614) 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 -- 172.21.15.19:0/323696709 learned_addr learned my addr 172.21.15.19:0/323696709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7995e00640 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d46a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 -- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 msgr2=0x55fe13d46a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d46a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 -- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 msgr2=0x55fe13d5f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 0x55fe13d5f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7996800640 1 -- 172.21.15.19:0/323696709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe13a606b0 con 0x55fe1397e270 2024-04-25T11:35:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.738+0000 7f7995e00640 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d46a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f7996800640 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d78ba0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7990034040 tx=0x7f7990031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdc8be029551119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:21.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f797b400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:35:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f797b400640 1 -- 172.21.15.19:0/323696709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990036050 con 0x55fe1397e270 2024-04-25T11:35:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798805f080 con 0x55fe1397e270 2024-04-25T11:35:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f797b400640 1 -- 172.21.15.19:0/323696709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f7990031db0 con 0x55fe1397e270 2024-04-25T11:35:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f797b400640 1 -- 172.21.15.19:0/323696709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f799004a050 con 0x55fe1397e270 2024-04-25T11:35:21.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f79a4050740 10 client.7506 fetch_fsmap learned FSMap version 90 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.739+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55fe13d795b0 con 0x55fe1397e270 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f797b400640 1 -- 172.21.15.19:0/323696709 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798805f080 con 0x55fe1397e270 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f797b400640 1 -- 172.21.15.19:0/323696709 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f799003d450 con 0x55fe1397e270 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f79a4050740 10 client.7506 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f79a4050740 10 client.7506 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f79a4050740 20 client.7506 populate_metadata read hostname 'smithi019' 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f79a4050740 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe1393ce40 0x55fe13d75bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f79a4050740 4 client.7506 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 --> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe13c92210 con 0x55fe1393ce40 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.740+0000 7f7995e00640 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe1393ce40 0x55fe13d75bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.741+0000 7f7995e00640 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe1393ce40 0x55fe13d75bc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d0464d5a306c6d server_cookie=33155150acc66a77 in_seq=0 out_seq=0 2024-04-25T11:35:21.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.741+0000 7f797b400640 10 client.7506 ms_handle_connect on v2:172.21.15.19:6834/2716758396 2024-04-25T11:35:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f797b400640 1 -- 172.21.15.19:0/323696709 <== mds.0 v2:172.21.15.19:6834/2716758396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe13c92210 con 0x55fe1393ce40 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f797b400640 10 client.7506 handle_command_reply: tid=0 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 client.7506 shutdown 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 msgr2=0x55fe13d78ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d78ba0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7990034040 tx=0x7f7990031ac0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 shutdown_connections 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe1393ce40 0x55fe13d75bc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1395d590 0x55fe13d5f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1397e270 0x55fe13d78ba0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 --2- 172.21.15.19:0/323696709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe139553b0 0x55fe13d46a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f797b400640 0 client.7506 ms_handle_reset on v2:172.21.15.19:6834/2716758396 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 >> 172.21.15.19:0/323696709 conn(0x55fe139723c0 msgr2=0x55fe13d47c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 shutdown_connections 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.742+0000 7f79a4050740 1 -- 172.21.15.19:0/323696709 wait complete. 2024-04-25T11:35:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.743+0000 7f79a4050740 20 client.7506 trim_cache size 0 max 16384 2024-04-25T11:35:21.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.763+0000 7f79a4050740 1 Processor -- start 2024-04-25T11:35:21.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.763+0000 7f79a4050740 1 -- start start 2024-04-25T11:35:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe139553b0 0x55fe13d4c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f79a4050740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1395d590 0x55fe13d62650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1397e270 0x55fe13d749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f79a4050740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7998201fb0 con 0x55fe1395d590 2024-04-25T11:35:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79981fa8a0 con 0x55fe139553b0 2024-04-25T11:35:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7998140f90 con 0x55fe1397e270 2024-04-25T11:35:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f798fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1395d590 0x55fe13d62650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f798fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1395d590 0x55fe13d62650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36618/0 (socket says 172.21.15.19:36618) 2024-04-25T11:35:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f798fe00640 1 -- 172.21.15.19:0/3310053325 learned_addr learned my addr 172.21.15.19:0/3310053325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f798fe00640 1 -- 172.21.15.19:0/3310053325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1397e270 msgr2=0x55fe13d749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f798fe00640 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1397e270 0x55fe13d749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f7995e00640 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe139553b0 0x55fe13d4c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.765+0000 7f798fe00640 1 -- 172.21.15.19:0/3310053325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe139553b0 msgr2=0x55fe13d4c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.766+0000 7f798fe00640 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe139553b0 0x55fe13d4c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.766+0000 7f798fe00640 1 -- 172.21.15.19:0/3310053325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe13d1e6f0 con 0x55fe1395d590 2024-04-25T11:35:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.766+0000 7f798fe00640 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1395d590 0x55fe13d62650 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f798000ca30 tx=0x7f79800046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154f322d2f92212a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.766+0000 7f7986000640 1 -- 172.21.15.19:0/3310053325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980023070 con 0x55fe1395d590 2024-04-25T11:35:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.766+0000 7f7986000640 1 -- 172.21.15.19:0/3310053325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7980013d30 con 0x55fe1395d590 2024-04-25T11:35:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.766+0000 7f7986000640 1 -- 172.21.15.19:0/3310053325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798001e640 con 0x55fe1395d590 2024-04-25T11:35:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 -- 172.21.15.19:0/3310053325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1395d590 msgr2=0x55fe13d62650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1395d590 0x55fe13d62650 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f798000ca30 tx=0x7f79800046f0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 -- 172.21.15.19:0/3310053325 shutdown_connections 2024-04-25T11:35:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1397e270 0x55fe13d749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe1395d590 0x55fe13d62650 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 --2- 172.21.15.19:0/3310053325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe139553b0 0x55fe13d4c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 -- 172.21.15.19:0/3310053325 >> 172.21.15.19:0/3310053325 conn(0x55fe139723c0 msgr2=0x55fe13d60de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 -- 172.21.15.19:0/3310053325 shutdown_connections 2024-04-25T11:35:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.767+0000 7f79a4050740 1 -- 172.21.15.19:0/3310053325 wait complete. 2024-04-25T11:35:21.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.768+0000 7f79a4050740 1 -- start start 2024-04-25T11:35:21.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.768+0000 7f79a4050740 1 Processor -- start 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f79a4050740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe139553b0 0x55fe13d456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1395d590 0x55fe13d45bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f79a4050740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1397e270 0x55fe13d77c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f79a4050740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fe13d4d4f0 con 0x55fe139553b0 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fe13d4d0a0 con 0x55fe1397e270 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f79a4050740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fe13d4dae0 con 0x55fe1395d590 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f79a4050740 10 client.0 _ll_register_callbacks cb 0x7f7990034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f7995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe139553b0 0x55fe13d456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f7995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe139553b0 0x55fe13d456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36626/0 (socket says 172.21.15.19:36626) 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f7995e00640 1 -- 172.21.15.19:0/1971055987 learned_addr learned my addr 172.21.15.19:0/1971055987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.770+0000 7f798fe00640 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1395d590 0x55fe13d45bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f7995e00640 1 -- 172.21.15.19:0/1971055987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1395d590 msgr2=0x55fe13d45bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f7995e00640 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1395d590 0x55fe13d45bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f7995e00640 1 -- 172.21.15.19:0/1971055987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1397e270 msgr2=0x55fe13d77c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f7995e00640 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1397e270 0x55fe13d77c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f7995e00640 1 -- 172.21.15.19:0/1971055987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe13de6990 con 0x55fe139553b0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f7995e00640 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe139553b0 0x55fe13d456b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f798805b670 tx=0x7f7988004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abec22fb882b8cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f797b400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f797b400640 1 -- 172.21.15.19:0/1971055987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798801e040 con 0x55fe139553b0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f797b400640 1 -- 172.21.15.19:0/1971055987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f798805e2e0 con 0x55fe139553b0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7980013d30 con 0x55fe139553b0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f797b400640 1 -- 172.21.15.19:0/1971055987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f798805f080 con 0x55fe139553b0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f79a4050740 10 client.7512 fetch_fsmap learned FSMap version 90 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.771+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55fe13d496a0 con 0x55fe139553b0 2024-04-25T11:35:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f797b400640 1 -- 172.21.15.19:0/1971055987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7980013d30 con 0x55fe139553b0 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f797b400640 1 -- 172.21.15.19:0/1971055987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f798805a9f0 con 0x55fe139553b0 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f79a4050740 10 client.7512 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f79a4050740 10 client.7512 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f79a4050740 20 client.7512 populate_metadata read hostname 'smithi019' 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f79a4050740 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe13d78260 0x55fe13d5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f79a4050740 4 client.7512 mds_command: new command op to 7374 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 --> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] -- command(tid 0: {"prefix": "status"}) -- 0x55fe13c8c080 con 0x55fe13d78260 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.772+0000 7f798fe00640 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe13d78260 0x55fe13d5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f798fe00640 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe13d78260 0x55fe13d5d7f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d46a9d77f72a1c server_cookie=37907a86fe773c51 in_seq=0 out_seq=0 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f797b400640 10 client.7512 ms_handle_connect on v2:172.21.15.19:6834/2716758396 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f797b400640 1 -- 172.21.15.19:0/1971055987 <== mds.0 v2:172.21.15.19:6834/2716758396 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55fe13c8c080 con 0x55fe13d78260 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f797b400640 10 client.7512 handle_command_reply: tid=0 2024-04-25T11:35:21.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 client.7512 shutdown 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe139553b0 msgr2=0x55fe13d456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe139553b0 0x55fe13d456b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f798805b670 tx=0x7f7988004590 comp rx=0 tx=0).stop 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 shutdown_connections 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:6834/2716758396,v1:172.21.15.19:6835/2716758396] conn(0x55fe13d78260 0x55fe13d5d7f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fe1397e270 0x55fe13d77c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fe1395d590 0x55fe13d45bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 --2- 172.21.15.19:0/1971055987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fe139553b0 0x55fe13d456b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f797b400640 0 client.7512 ms_handle_reset on v2:172.21.15.19:6834/2716758396 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 >> 172.21.15.19:0/1971055987 conn(0x55fe139723c0 msgr2=0x55fe13d46850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 shutdown_connections 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.773+0000 7f79a4050740 1 -- 172.21.15.19:0/1971055987 wait complete. 2024-04-25T11:35:21.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f79a4050740 20 client.7512 trim_cache size 0 max 16384 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 7374, 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.372489345, 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 90, 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 92, 2024-04-25T11:35:21.766 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T11:35:21.767 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.476767835 2024-04-25T11:35:21.767 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:35:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79740744b0 msgr2=0x7f79740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79740744b0 0x7f79740769c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7998147910 tx=0x7f7980008040 comp rx=0 tx=0).stop 2024-04-25T11:35:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79980cdb90 msgr2=0x7f799813f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79980cdb90 0x7f799813f190 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f798800c820 tx=0x7f798800cce0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 shutdown_connections 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79740744b0 0x7f79740769c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79981f6db0 0x7f79980cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7998144500 0x7f79981f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 --2- 172.21.15.19:0/3202942780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79980cdb90 0x7f799813f190 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 >> 172.21.15.19:0/3202942780 conn(0x7f7998073050 msgr2=0x7f799815e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 shutdown_connections 2024-04-25T11:35:21.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:21.774+0000 7f799d000640 1 -- 172.21.15.19:0/3202942780 wait complete. 2024-04-25T11:35:21.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:35:22.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.087+0000 7f29bce00640 1 Processor -- start 2024-04-25T11:35:22.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.087+0000 7f29bce00640 1 -- start start 2024-04-25T11:35:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.090+0000 7f29bce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f29b8161060 0x7f29b8161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.090+0000 7f29bce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b80cdb20 con 0x7f29b8143170 2024-04-25T11:35:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.090+0000 7f29bce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b80cde00 con 0x7f29b80cff30 2024-04-25T11:35:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.090+0000 7f29bce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f29b8146970 con 0x7f29b8161060 2024-04-25T11:35:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.090+0000 7f29b7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f29b8143170 0x7f29b8160950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35912/0 (socket says 172.21.15.19:35912) 2024-04-25T11:35:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.090+0000 7f29b7400640 1 -- 172.21.15.19:0/3055948330 learned_addr learned my addr 172.21.15.19:0/3055948330 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.090+0000 7f29b6a00640 1 --2- 172.21.15.19:0/3055948330 >> v2:172.21.15.19:3300/0 conn(0x7f29b8161060 0x7f29b8161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.091+0000 7f29b6a00640 1 -- 172.21.15.19:0/3055948330 >> v1:172.21.15.169:6789/0 conn(0x7f29b80cff30 legacy=0x7f29b80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.091+0000 7f29b6a00640 1 -- 172.21.15.19:0/3055948330 >> v1:172.21.15.19:6789/0 conn(0x7f29b8143170 legacy=0x7f29b8160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.091+0000 7f29b6a00640 1 -- 172.21.15.19:0/3055948330 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b806f160 con 0x7f29b8161060 2024-04-25T11:35:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.091+0000 7f29b6000640 1 -- 172.21.15.19:0/3055948330 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702046831 0 0) 0x7f29b80cdb20 con 0x7f29b8143170 2024-04-25T11:35:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.092+0000 7f29b6a00640 1 --2- 172.21.15.19:0/3055948330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8161060 0x7f29b8161470 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f29a40097a0 tx=0x7f29a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d04129078afbbdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.092+0000 7f29b6000640 1 -- 172.21.15.19:0/3055948330 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4037070 con 0x7f29b8161060 2024-04-25T11:35:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.092+0000 7f29b6000640 1 -- 172.21.15.19:0/3055948330 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a4032440 con 0x7f29b8161060 2024-04-25T11:35:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.092+0000 7f29b6000640 1 -- 172.21.15.19:0/3055948330 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4032810 con 0x7f29b8161060 2024-04-25T11:35:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.093+0000 7f29bce00640 1 -- 172.21.15.19:0/3055948330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8161060 msgr2=0x7f29b8161470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.093+0000 7f29bce00640 1 --2- 172.21.15.19:0/3055948330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8161060 0x7f29b8161470 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f29a40097a0 tx=0x7f29a402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.093+0000 7f29bce00640 1 -- 172.21.15.19:0/3055948330 shutdown_connections 2024-04-25T11:35:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.093+0000 7f29bce00640 1 --2- 172.21.15.19:0/3055948330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b8161060 0x7f29b8161470 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.093+0000 7f29bce00640 1 -- 172.21.15.19:0/3055948330 >> 172.21.15.19:0/3055948330 conn(0x7f29b80d1b40 msgr2=0x7f29b80d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.093+0000 7f29bce00640 1 -- 172.21.15.19:0/3055948330 shutdown_connections 2024-04-25T11:35:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.093+0000 7f29bce00640 1 -- 172.21.15.19:0/3055948330 wait complete. 2024-04-25T11:35:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.096+0000 7f29bce00640 1 Processor -- start 2024-04-25T11:35:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.096+0000 7f29bce00640 1 -- start start 2024-04-25T11:35:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.097+0000 7f29bce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b80cff30 0x7f29b806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29bce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b8143170 0x7f29b815f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29bce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29b8161060 0x7f29b815bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29bce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8143580 con 0x7f29b80cff30 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29bce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8070950 con 0x7f29b8143170 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29bce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f29b8070ad0 con 0x7f29b8161060 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29b7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b80cff30 0x7f29b806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29b7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b80cff30 0x7f29b806ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36658/0 (socket says 172.21.15.19:36658) 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29b7400640 1 -- 172.21.15.19:0/3127125208 learned_addr learned my addr 172.21.15.19:0/3127125208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29b6a00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b8143170 0x7f29b815f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.098+0000 7f29b7e00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29b8161060 0x7f29b815bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29b7400640 1 -- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29b8161060 msgr2=0x7f29b815bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29b7400640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29b8161060 0x7f29b815bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29b7400640 1 -- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b8143170 msgr2=0x7f29b815f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29b7400640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b8143170 0x7f29b815f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29b7400640 1 -- 172.21.15.19:0/3127125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b815e0c0 con 0x7f29b80cff30 2024-04-25T11:35:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29b7400640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b80cff30 0x7f29b806ff20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c420 tx=0x7f29a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29af400640 1 -- 172.21.15.19:0/3127125208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a80074e0 con 0x7f29b80cff30 2024-04-25T11:35:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29af400640 1 -- 172.21.15.19:0/3127125208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a80047e0 con 0x7f29b80cff30 2024-04-25T11:35:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b815e3a0 con 0x7f29b80cff30 2024-04-25T11:35:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29af400640 1 -- 172.21.15.19:0/3127125208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8004c10 con 0x7f29b80cff30 2024-04-25T11:35:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.099+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b815e960 con 0x7f29b80cff30 2024-04-25T11:35:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.101+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b8050740 con 0x7f29b80cff30 2024-04-25T11:35:22.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.102+0000 7f29af400640 1 -- 172.21.15.19:0/3127125208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f29a8011070 con 0x7f29b80cff30 2024-04-25T11:35:22.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.103+0000 7f29af400640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29980744e0 0x7f29980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:22.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.103+0000 7f29af400640 1 -- 172.21.15.19:0/3127125208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f29a8094ea0 con 0x7f29b80cff30 2024-04-25T11:35:22.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.104+0000 7f29b6a00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29980744e0 0x7f29980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:22.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.105+0000 7f29b6a00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29980744e0 0x7f29980769f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.108+0000 7f29af400640 1 -- 172.21.15.19:0/3127125208 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29a805f620 con 0x7f29b80cff30 2024-04-25T11:35:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.342+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f29b80b4e60 con 0x7f29b80cff30 2024-04-25T11:35:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.343+0000 7f29af400640 1 -- 172.21.15.19:0/3127125208 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) ==== 53+0+73 (secure 0 0 0) 0x7f29a805ef20 con 0x7f29b80cff30 2024-04-25T11:35:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:35:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29980744e0 msgr2=0x7f29980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29980744e0 0x7f29980769f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).stop 2024-04-25T11:35:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b80cff30 msgr2=0x7f29b806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b80cff30 0x7f29b806ff20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c420 tx=0x7f29a800c8e0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 shutdown_connections 2024-04-25T11:35:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29980744e0 0x7f29980769f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29b8143170 0x7f29b815f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29b8161060 0x7f29b815bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 --2- 172.21.15.19:0/3127125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29b80cff30 0x7f29b806ff20 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:22.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 >> 172.21.15.19:0/3127125208 conn(0x7f29b80d1b40 msgr2=0x7f29b80ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:22.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 shutdown_connections 2024-04-25T11:35:22.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22.346+0000 7f29bce00640 1 -- 172.21.15.19:0/3127125208 wait complete. 2024-04-25T11:35:22.359 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:35:22.359 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:35:22.359 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:35:22.359 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:35:22.359 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:22.359 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:35:22.360 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:35:22.360 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:35:22.360 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:35:22.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:22.360 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:35:22.416 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42151sec preferred_lft 42151sec 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:22.417 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:35:22.418 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:22.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:22.419 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:35:22.502 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:35:22.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:22.507 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:35:22.556 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:35:22.556 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:35:22.556 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:35:22.556 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:35:22.556 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:35:22.557 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:35:22.557 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:35:22.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:35:22.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:35:22.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:35:22.558 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:35:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:35:22.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:35:22.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:22.769 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:35:22.823 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:35:22.823 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:35:22.823 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:35:22.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:22.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:35:22.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:35:22.906 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:22.963 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:35:22.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:35:22.964 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:23.016 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:35:23.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:23.016 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:35:23.227 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:35:23.227 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:35:23.271 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:35:23.271 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:35:23.271 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:35:23.271 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:35:23.271 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:35:23.273 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:35:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:35:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:35:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:35:23.273 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:35:23.278 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:35:23.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:23.278 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:23.347 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:35:23.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:35:23.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:35:23.473 INFO:teuthology.orchestra.run.smithi019.stdout:client5693 2024-04-25T11:35:23.473 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:23.473 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:23.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:23.528 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T11:35:23.597 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:23.597 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:23.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:23.651 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T11:35:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records in 2024-04-25T11:35:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records out 2024-04-25T11:35:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.132397 s, 47.5 MB/s 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T11:35:23.838 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T11:35:23.839 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T11:35:23.839 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T11:35:23.839 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:35:23.984 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mode": 33188, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ino": 1099511627777, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_dev": 42, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_nlink": 1, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_uid": 1000, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_gid": 1316, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_size": 6291456, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_atime": 1714044923.71343, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_mtime": 1714044923.7554295, 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout: "st_ctime": 1714044923.7554295 2024-04-25T11:35:23.985 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:35:23.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:35:23.998 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:23.999 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:24.099 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:24.099 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:24.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:35:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.451+0000 7f7dfc000640 1 Processor -- start 2024-04-25T11:35:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.451+0000 7f7dfc000640 1 -- start start 2024-04-25T11:35:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.453+0000 7f7dfc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df40cf5e0 0x7f7df40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.453+0000 7f7dfc000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7df406c5c0 0x7f7df406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.453+0000 7f7dfc000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df40cff30 con 0x7f7df41443c0 2024-04-25T11:35:24.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.453+0000 7f7dfc000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7df40708d0 con 0x7f7df406c5c0 2024-04-25T11:35:24.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.453+0000 7f7dfc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7df415f160 con 0x7f7df40cf5e0 2024-04-25T11:35:24.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.454+0000 7f7df8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df40cf5e0 0x7f7df40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.454+0000 7f7df8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df40cf5e0 0x7f7df40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:50050/0 (socket says 172.21.15.19:50050) 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.454+0000 7f7df9600640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7df41443c0 0x7f7df406fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:34226/0 (socket says 172.21.15.19:34226) 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.454+0000 7f7df8c00640 1 -- 172.21.15.19:0/2732921626 learned_addr learned my addr 172.21.15.19:0/2732921626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.454+0000 7f7dfa000640 1 --2- 172.21.15.19:0/2732921626 >> v2:172.21.15.169:3300/0 conn(0x7f7df406c5c0 0x7f7df406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.455+0000 7f7df8c00640 1 -- 172.21.15.19:0/2732921626 >> v2:172.21.15.169:3300/0 conn(0x7f7df406c5c0 msgr2=0x7f7df406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.455+0000 7f7df8c00640 1 --2- 172.21.15.19:0/2732921626 >> v2:172.21.15.169:3300/0 conn(0x7f7df406c5c0 0x7f7df406c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.455+0000 7f7df8c00640 1 -- 172.21.15.19:0/2732921626 >> v1:172.21.15.169:6789/0 conn(0x7f7df41443c0 legacy=0x7f7df406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.455+0000 7f7df8c00640 1 -- 172.21.15.19:0/2732921626 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df415f5f0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.455+0000 7f7dfa000640 1 --2- 172.21.15.19:0/2732921626 >> v2:172.21.15.169:3300/0 conn(0x7f7df406c5c0 0x7f7df406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:35:24.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.457+0000 7f7df8c00640 1 --2- 172.21.15.19:0/2732921626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df40cf5e0 0x7f7df40cf9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009870 tx=0x7f7de002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=278419e90572642f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:24.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.458+0000 7f7df3e00640 1 -- 172.21.15.19:0/2732921626 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0037070 con 0x7f7df40cf5e0 2024-04-25T11:35:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.458+0000 7f7df3e00640 1 -- 172.21.15.19:0/2732921626 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de0032440 con 0x7f7df40cf5e0 2024-04-25T11:35:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.458+0000 7f7df3e00640 1 -- 172.21.15.19:0/2732921626 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0032810 con 0x7f7df40cf5e0 2024-04-25T11:35:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.458+0000 7f7dfc000640 1 -- 172.21.15.19:0/2732921626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df40cf5e0 msgr2=0x7f7df40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.458+0000 7f7dfc000640 1 --2- 172.21.15.19:0/2732921626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df40cf5e0 0x7f7df40cf9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009870 tx=0x7f7de002a250 comp rx=0 tx=0).stop 2024-04-25T11:35:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.459+0000 7f7dfc000640 1 -- 172.21.15.19:0/2732921626 shutdown_connections 2024-04-25T11:35:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.459+0000 7f7dfc000640 1 --2- 172.21.15.19:0/2732921626 >> v2:172.21.15.169:3300/0 conn(0x7f7df406c5c0 0x7f7df406c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.459+0000 7f7dfc000640 1 --2- 172.21.15.19:0/2732921626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df40cf5e0 0x7f7df40cf9f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.459+0000 7f7dfc000640 1 -- 172.21.15.19:0/2732921626 >> 172.21.15.19:0/2732921626 conn(0x7f7df40d1ae0 msgr2=0x7f7df40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.459+0000 7f7dfc000640 1 -- 172.21.15.19:0/2732921626 shutdown_connections 2024-04-25T11:35:24.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.459+0000 7f7dfc000640 1 -- 172.21.15.19:0/2732921626 wait complete. 2024-04-25T11:35:24.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.461+0000 7f7dfc000640 1 Processor -- start 2024-04-25T11:35:24.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.461+0000 7f7dfc000640 1 -- start start 2024-04-25T11:35:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7dfc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df406c5c0 0x7f7df415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7dfc000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40cf5e0 0x7f7df4135e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7dfc000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df41443c0 0x7f7df41e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7dfc000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7df415edf0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7dfc000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7df415eb10 con 0x7f7df41443c0 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7dfc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7df41447d0 con 0x7f7df406c5c0 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7df8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40cf5e0 0x7f7df4135e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7df8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40cf5e0 0x7f7df4135e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41108/0 (socket says 172.21.15.19:41108) 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.463+0000 7f7df8c00640 1 -- 172.21.15.19:0/3050116476 learned_addr learned my addr 172.21.15.19:0/3050116476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7dfa000640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df41443c0 0x7f7df41e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df9600640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df406c5c0 0x7f7df415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df8c00640 1 -- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df406c5c0 msgr2=0x7f7df415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df8c00640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df406c5c0 0x7f7df415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df8c00640 1 -- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df41443c0 msgr2=0x7f7df41e87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df8c00640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df41443c0 0x7f7df41e87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df8c00640 1 -- 172.21.15.19:0/3050116476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df41363d0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df8c00640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40cf5e0 0x7f7df4135e90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009990 tx=0x7f7de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:24.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df1600640 1 -- 172.21.15.19:0/3050116476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0037040 con 0x7f7df40cf5e0 2024-04-25T11:35:24.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df1600640 1 -- 172.21.15.19:0/3050116476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de0042070 con 0x7f7df40cf5e0 2024-04-25T11:35:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.464+0000 7f7df1600640 1 -- 172.21.15.19:0/3050116476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0030940 con 0x7f7df40cf5e0 2024-04-25T11:35:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.465+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df415d630 con 0x7f7df40cf5e0 2024-04-25T11:35:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.465+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df415dbf0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.466+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df415d3a0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.467+0000 7f7df1600640 1 -- 172.21.15.19:0/3050116476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7de0030ae0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.469+0000 7f7df1600640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dd40744b0 0x7f7dd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.469+0000 7f7df1600640 1 -- 172.21.15.19:0/3050116476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f7de00b5ad0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.470+0000 7f7df9600640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dd40744b0 0x7f7dd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:24.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.471+0000 7f7df9600640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dd40744b0 0x7f7dd40769c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7de8000f60 tx=0x7f7de8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:24.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.474+0000 7f7df1600640 1 -- 172.21.15.19:0/3050116476 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7de00802e0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.685+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7df4050740 con 0x7f7df40cf5e0 2024-04-25T11:35:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.688+0000 7f7df1600640 1 -- 172.21.15.19:0/3050116476 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16672 (secure 0 0 0) 0x7f7de00882d0 con 0x7f7df40cf5e0 2024-04-25T11:35:24.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:24.677 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":92,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:35:15.422148+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:35:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dd40744b0 msgr2=0x7f7dd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dd40744b0 0x7f7dd40769c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7de8000f60 tx=0x7f7de8009040 comp rx=0 tx=0).stop 2024-04-25T11:35:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40cf5e0 msgr2=0x7f7df4135e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40cf5e0 0x7f7df4135e90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009990 tx=0x7f7de00046c0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 shutdown_connections 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7dd40744b0 0x7f7dd40769c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df41443c0 0x7f7df41e87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df40cf5e0 0x7f7df4135e90 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 --2- 172.21.15.19:0/3050116476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df406c5c0 0x7f7df415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 >> 172.21.15.19:0/3050116476 conn(0x7f7df40d1ae0 msgr2=0x7f7df415f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 shutdown_connections 2024-04-25T11:35:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:24.691+0000 7f7dfc000640 1 -- 172.21.15.19:0/3050116476 wait complete. 2024-04-25T11:35:24.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:35:24.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:35:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.008+0000 7f2396800640 1 Processor -- start 2024-04-25T11:35:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.008+0000 7f2396800640 1 -- start start 2024-04-25T11:35:24.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.010+0000 7f2396800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23900ce930 0x7f239006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.010+0000 7f2396800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f239006db10 0x7f23900d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.010+0000 7f2396800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239006df20 con 0x7f23900ce520 2024-04-25T11:35:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.010+0000 7f2396800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f23900cda60 con 0x7f239006db10 2024-04-25T11:35:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.010+0000 7f2396800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f239015dfe0 con 0x7f23900ce930 2024-04-25T11:35:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.011+0000 7f2395400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f23900ce520 0x7f239015d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38538/0 (socket says 172.21.15.19:38538) 2024-04-25T11:35:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.011+0000 7f2395400640 1 -- 172.21.15.19:0/3843207750 learned_addr learned my addr 172.21.15.19:0/3843207750 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.011+0000 7f2395e00640 1 --2- 172.21.15.19:0/3843207750 >> v2:172.21.15.169:3300/0 conn(0x7f239006db10 0x7f23900d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.011+0000 7f2394a00640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23900ce930 0x7f239006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.011+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581135918 0 0) 0x7f2384001110 con 0x7f23900ce520 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2378003520 con 0x7f23900ce520 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2302533816 0 0) 0x7f2378003520 con 0x7f23900ce520 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23900ce930 msgr2=0x7f239006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23900ce930 0x7f239006d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 >> v2:172.21.15.169:3300/0 conn(0x7f239006db10 msgr2=0x7f23900d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 --2- 172.21.15.19:0/3843207750 >> v2:172.21.15.169:3300/0 conn(0x7f239006db10 0x7f23900d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239015e2c0 con 0x7f23900ce520 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2384001840 con 0x7f23900ce520 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 >> v1:172.21.15.19:6789/0 conn(0x7f23900ce520 legacy=0x7f239015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f2378003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23900ce930 0x7f2378004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2378004d40 0x7f2378005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23900cda60 con 0x7f23780039b0 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f239015dfe0 con 0x7f2378004d40 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f237800baf0 con 0x7f23900ce930 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.012+0000 7f2395400640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f2378003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.013+0000 7f2395400640 1 -- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23900ce930 msgr2=0x7f2378004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.013+0000 7f2395400640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23900ce930 0x7f2378004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.013+0000 7f2395400640 1 -- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2378004d40 msgr2=0x7f2378005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.013+0000 7f2395400640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2378004d40 0x7f2378005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.013+0000 7f2395400640 1 -- 172.21.15.19:0/3843207750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2378003520 con 0x7f23780039b0 2024-04-25T11:35:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.014+0000 7f2395400640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f2378003e20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f238400c3b0 tx=0x7f2378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253a35720c2d7270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.014+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238400c0e0 con 0x7f23780039b0 2024-04-25T11:35:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.014+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2384034070 con 0x7f23780039b0 2024-04-25T11:35:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.014+0000 7f238fe00640 1 -- 172.21.15.19:0/3843207750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384005d30 con 0x7f23780039b0 2024-04-25T11:35:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.015+0000 7f2396800640 1 -- 172.21.15.19:0/3843207750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 msgr2=0x7f2378003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.015+0000 7f2396800640 1 --2- 172.21.15.19:0/3843207750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f2378003e20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f238400c3b0 tx=0x7f2378005830 comp rx=0 tx=0).stop 2024-04-25T11:35:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.015+0000 7f2395e00640 1 -- 172.21.15.19:0/3843207750 reap_dead start 2024-04-25T11:35:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.015+0000 7f2396800640 1 -- 172.21.15.19:0/3843207750 shutdown_connections 2024-04-25T11:35:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.015+0000 7f2396800640 1 -- 172.21.15.19:0/3843207750 >> 172.21.15.19:0/3843207750 conn(0x7f2390072f40 msgr2=0x7f2390073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.015+0000 7f2396800640 1 -- 172.21.15.19:0/3843207750 shutdown_connections 2024-04-25T11:35:25.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.015+0000 7f2396800640 1 -- 172.21.15.19:0/3843207750 wait complete. 2024-04-25T11:35:25.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.018+0000 7f2396800640 1 Processor -- start 2024-04-25T11:35:25.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.018+0000 7f2396800640 1 -- start start 2024-04-25T11:35:25.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.019+0000 7f2396800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f23901fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.019+0000 7f2396800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239014fb10 0x7f2390157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.019+0000 7f2396800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2390158090 0x7f239015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2396800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2390160d30 con 0x7f23780039b0 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2396800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f23900ce000 con 0x7f2390158090 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2396800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f239015dbc0 con 0x7f239014fb10 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f23901fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f23901fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41122/0 (socket says 172.21.15.19:41122) 2024-04-25T11:35:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 -- 172.21.15.19:0/3532923039 learned_addr learned my addr 172.21.15.19:0/3532923039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395e00640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2390158090 0x7f239015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 -- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239014fb10 msgr2=0x7f2390157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2394a00640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239014fb10 0x7f2390157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239014fb10 0x7f2390157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 -- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2390158090 msgr2=0x7f239015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2390158090 0x7f239015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395400640 1 -- 172.21.15.19:0/3532923039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23901588c0 con 0x7f23780039b0 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.020+0000 7f2395e00640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2390158090 0x7f239015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.021+0000 7f2395400640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f23901fbcd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f2384031040 tx=0x7f2384001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.021+0000 7f238d600640 1 -- 172.21.15.19:0/3532923039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384007270 con 0x7f23780039b0 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.021+0000 7f238d600640 1 -- 172.21.15.19:0/3532923039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2384007410 con 0x7f23780039b0 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.021+0000 7f238d600640 1 -- 172.21.15.19:0/3532923039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23840077e0 con 0x7f23780039b0 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.021+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2390159a00 con 0x7f23780039b0 2024-04-25T11:35:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.021+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901fc2a0 con 0x7f23780039b0 2024-04-25T11:35:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.022+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2390157210 con 0x7f23780039b0 2024-04-25T11:35:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.024+0000 7f238d600640 1 -- 172.21.15.19:0/3532923039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f238402f480 con 0x7f23780039b0 2024-04-25T11:35:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.025+0000 7f238d600640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23700744e0 0x7f23700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.025+0000 7f238d600640 1 -- 172.21.15.19:0/3532923039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f23840b5b20 con 0x7f23780039b0 2024-04-25T11:35:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.026+0000 7f2394a00640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23700744e0 0x7f23700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.027+0000 7f2394a00640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23700744e0 0x7f23700769f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f237c006a90 tx=0x7f237c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:25.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.030+0000 7f238d600640 1 -- 172.21.15.19:0/3532923039 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2390157210 con 0x7f23780039b0 2024-04-25T11:35:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.237+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2390076c60 con 0x7f23780039b0 2024-04-25T11:35:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.238+0000 7f238d600640 1 -- 172.21.15.19:0/3532923039 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f2384049030 con 0x7f23780039b0 2024-04-25T11:35:25.226 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:35:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23700744e0 msgr2=0x7f23700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23700744e0 0x7f23700769f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f237c006a90 tx=0x7f237c008040 comp rx=0 tx=0).stop 2024-04-25T11:35:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 msgr2=0x7f23901fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f23901fbcd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f2384031040 tx=0x7f2384001c10 comp rx=0 tx=0).stop 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 shutdown_connections 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23700744e0 0x7f23700769f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2390158090 0x7f239015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f239014fb10 0x7f2390157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 --2- 172.21.15.19:0/3532923039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23780039b0 0x7f23901fbcd0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 >> 172.21.15.19:0/3532923039 conn(0x7f2390072f40 msgr2=0x7f239015d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 shutdown_connections 2024-04-25T11:35:25.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.241+0000 7f2396800640 1 -- 172.21.15.19:0/3532923039 wait complete. 2024-04-25T11:35:25.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:35:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.552+0000 7f5a77e00640 1 Processor -- start 2024-04-25T11:35:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.552+0000 7f5a77e00640 1 -- start start 2024-04-25T11:35:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.555+0000 7f5a77e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a78162210 0x7f5a7806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.555+0000 7f5a77e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a7806d3a0 con 0x7f5a7806cb70 2024-04-25T11:35:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.555+0000 7f5a77e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a7806d680 con 0x7f5a781441d0 2024-04-25T11:35:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.555+0000 7f5a77e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5a7806d960 con 0x7f5a78162210 2024-04-25T11:35:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.555+0000 7f5a77400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5a7806cb70 0x7f5a7815fa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38548/0 (socket says 172.21.15.19:38548) 2024-04-25T11:35:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.555+0000 7f5a77400640 1 -- 172.21.15.19:0/1507624361 learned_addr learned my addr 172.21.15.19:0/1507624361 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:25.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a76000640 1 --2- 172.21.15.19:0/1507624361 >> v2:172.21.15.169:3300/0 conn(0x7f5a78162210 0x7f5a7806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 418958269 0 0) 0x7f5a7806d3a0 con 0x7f5a7806cb70 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a5c003520 con 0x7f5a7806cb70 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4088910621 0 0) 0x7f5a5c003520 con 0x7f5a7806cb70 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 >> v2:172.21.15.169:3300/0 conn(0x7f5a78162210 msgr2=0x7f5a7806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 --2- 172.21.15.19:0/1507624361 >> v2:172.21.15.169:3300/0 conn(0x7f5a78162210 0x7f5a7806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 >> v1:172.21.15.169:6789/0 conn(0x7f5a781441d0 legacy=0x7f5a78161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7806dc40 con 0x7f5a7806cb70 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f5a700019f0 con 0x7f5a7806cb70 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.556+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 >> v1:172.21.15.19:6789/0 conn(0x7f5a7806cb70 legacy=0x7f5a7815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a75600640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c0039b0 0x7f5a5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a75600640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a5c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a75600640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a5c0051c0 0x7f5a5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7806d960 con 0x7f5a5c004800 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c00c160 con 0x7f5a5c0039b0 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5c00c360 con 0x7f5a5c0051c0 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a76000640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a5c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a76000640 1 -- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a5c0051c0 msgr2=0x7f5a5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a77400640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a5c0051c0 0x7f5a5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a76a00640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c0039b0 0x7f5a5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a76000640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a5c0051c0 0x7f5a5c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a76000640 1 -- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c0039b0 msgr2=0x7f5a5c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.557+0000 7f5a76000640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c0039b0 0x7f5a5c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.558+0000 7f5a76000640 1 -- 172.21.15.19:0/1507624361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c003520 con 0x7f5a5c004800 2024-04-25T11:35:25.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.558+0000 7f5a77400640 1 -- 172.21.15.19:0/1507624361 reap_dead start 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.559+0000 7f5a76000640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a5c004c10 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009ac0 tx=0x7f5a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b534308695eda170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.559+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64036070 con 0x7f5a5c004800 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.559+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a64031400 con 0x7f5a5c004800 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.559+0000 7f5a75600640 1 -- 172.21.15.19:0/1507624361 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64009690 con 0x7f5a5c004800 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.560+0000 7f5a77e00640 1 -- 172.21.15.19:0/1507624361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 msgr2=0x7f5a5c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.560+0000 7f5a77e00640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a5c004c10 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009ac0 tx=0x7f5a5c005830 comp rx=0 tx=0).stop 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.560+0000 7f5a77e00640 1 -- 172.21.15.19:0/1507624361 shutdown_connections 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.560+0000 7f5a77e00640 1 --2- 172.21.15.19:0/1507624361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a5c004c10 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.560+0000 7f5a77e00640 1 -- 172.21.15.19:0/1507624361 >> 172.21.15.19:0/1507624361 conn(0x7f5a78072f40 msgr2=0x7f5a78073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.560+0000 7f5a77e00640 1 -- 172.21.15.19:0/1507624361 shutdown_connections 2024-04-25T11:35:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.560+0000 7f5a77e00640 1 -- 172.21.15.19:0/1507624361 wait complete. 2024-04-25T11:35:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.562+0000 7f5a77e00640 1 Processor -- start 2024-04-25T11:35:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.563+0000 7f5a77e00640 1 -- start start 2024-04-25T11:35:25.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.564+0000 7f5a77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a780cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.564+0000 7f5a77e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a780ce210 0x7f5a7815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.564+0000 7f5a77e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a780b52a0 0x7f5a7815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.564+0000 7f5a77e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7806d100 con 0x7f5a5c004800 2024-04-25T11:35:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.564+0000 7f5a77e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7806cf80 con 0x7f5a780b52a0 2024-04-25T11:35:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.564+0000 7f5a77e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7806c630 con 0x7f5a780ce210 2024-04-25T11:35:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a780cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a780cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41140/0 (socket says 172.21.15.19:41140) 2024-04-25T11:35:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 -- 172.21.15.19:0/3143940925 learned_addr learned my addr 172.21.15.19:0/3143940925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76000640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a780ce210 0x7f5a7815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a77400640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a780b52a0 0x7f5a7815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 -- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a780ce210 msgr2=0x7f5a7815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a780ce210 0x7f5a7815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 -- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a780b52a0 msgr2=0x7f5a7815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a780b52a0 0x7f5a7815c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 -- 172.21.15.19:0/3143940925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a780b62c0 con 0x7f5a5c004800 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76000640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a780ce210 0x7f5a7815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.565+0000 7f5a76a00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a780cdcd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800d7c0 tx=0x7f5a6800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.566+0000 7f5a6f400640 1 -- 172.21.15.19:0/3143940925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68005270 con 0x7f5a5c004800 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.566+0000 7f5a6f400640 1 -- 172.21.15.19:0/3143940925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a68017070 con 0x7f5a5c004800 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.566+0000 7f5a6f400640 1 -- 172.21.15.19:0/3143940925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800b6b0 con 0x7f5a5c004800 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.566+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a780b5a20 con 0x7f5a5c004800 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.566+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a780b5fe0 con 0x7f5a5c004800 2024-04-25T11:35:25.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.567+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a78073580 con 0x7f5a5c004800 2024-04-25T11:35:25.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.568+0000 7f5a6f400640 1 -- 172.21.15.19:0/3143940925 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a6800b850 con 0x7f5a5c004800 2024-04-25T11:35:25.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.569+0000 7f5a6f400640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a580744b0 0x7f5a580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:25.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.569+0000 7f5a6f400640 1 -- 172.21.15.19:0/3143940925 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f5a680951b0 con 0x7f5a5c004800 2024-04-25T11:35:25.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.570+0000 7f5a76000640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a580744b0 0x7f5a580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.571+0000 7f5a76000640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a580744b0 0x7f5a580769c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5a64033000 tx=0x7f5a64008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.575+0000 7f5a6f400640 1 -- 172.21.15.19:0/3143940925 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a78073580 con 0x7f5a5c004800 2024-04-25T11:35:25.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.779+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5a78076c60 con 0x7f5a5c004800 2024-04-25T11:35:25.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.780+0000 7f5a6f400640 1 -- 172.21.15.19:0/3143940925 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f5a6805f930 con 0x7f5a5c004800 2024-04-25T11:35:25.768 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a580744b0 msgr2=0x7f5a580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a580744b0 0x7f5a580769c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5a64033000 tx=0x7f5a64008030 comp rx=0 tx=0).stop 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 msgr2=0x7f5a780cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a780cdcd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800d7c0 tx=0x7f5a6800dc80 comp rx=0 tx=0).stop 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 shutdown_connections 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a580744b0 0x7f5a580769c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a780b52a0 0x7f5a7815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a780ce210 0x7f5a7815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 --2- 172.21.15.19:0/3143940925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a5c004800 0x7f5a780cdcd0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:25.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 >> 172.21.15.19:0/3143940925 conn(0x7f5a78072f40 msgr2=0x7f5a780b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:25.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 shutdown_connections 2024-04-25T11:35:25.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:25.783+0000 7f5a77e00640 1 -- 172.21.15.19:0/3143940925 wait complete. 2024-04-25T11:35:25.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:35:26.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.097+0000 7f460f200640 1 Processor -- start 2024-04-25T11:35:26.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.097+0000 7f460f200640 1 -- start start 2024-04-25T11:35:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f460f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4610070d20 0x7f4610161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f460f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f461006dce0 0x7f46100ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f460f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46101623f0 con 0x7f461006f930 2024-04-25T11:35:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f460f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f46101626d0 con 0x7f461006dce0 2024-04-25T11:35:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f460f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f461006f130 con 0x7f4610070d20 2024-04-25T11:35:26.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f460e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f461006dce0 0x7f46100ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f4607e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f461006f930 0x7f461006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38554/0 (socket says 172.21.15.19:38554) 2024-04-25T11:35:26.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.100+0000 7f4607e00640 1 -- 172.21.15.19:0/3785616054 learned_addr learned my addr 172.21.15.19:0/3785616054 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:26.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.101+0000 7f460de00640 1 --2- 172.21.15.19:0/3785616054 >> v2:172.21.15.169:3300/0 conn(0x7f4610070d20 0x7f4610161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.101+0000 7f460e800640 1 -- 172.21.15.19:0/3785616054 >> v2:172.21.15.169:3300/0 conn(0x7f4610070d20 msgr2=0x7f4610161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:26.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.101+0000 7f460e800640 1 --2- 172.21.15.19:0/3785616054 >> v2:172.21.15.169:3300/0 conn(0x7f4610070d20 0x7f4610161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.101+0000 7f460e800640 1 -- 172.21.15.19:0/3785616054 >> v1:172.21.15.19:6789/0 conn(0x7f461006f930 legacy=0x7f461006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.101+0000 7f460e800640 1 -- 172.21.15.19:0/3785616054 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461006f410 con 0x7f461006dce0 2024-04-25T11:35:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.102+0000 7f460e800640 1 --2- 172.21.15.19:0/3785616054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461006dce0 0x7f46100ce7e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f461006e520 tx=0x7f460802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c14822a9c92765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.102+0000 7f460d400640 1 -- 172.21.15.19:0/3785616054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608007740 con 0x7f461006dce0 2024-04-25T11:35:26.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.103+0000 7f460d400640 1 -- 172.21.15.19:0/3785616054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46080078e0 con 0x7f461006dce0 2024-04-25T11:35:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.103+0000 7f460d400640 1 -- 172.21.15.19:0/3785616054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608007cb0 con 0x7f461006dce0 2024-04-25T11:35:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.103+0000 7f460f200640 1 -- 172.21.15.19:0/3785616054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461006dce0 msgr2=0x7f46100ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.103+0000 7f460f200640 1 --2- 172.21.15.19:0/3785616054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461006dce0 0x7f46100ce7e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f461006e520 tx=0x7f460802a5f0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.104+0000 7f460f200640 1 -- 172.21.15.19:0/3785616054 shutdown_connections 2024-04-25T11:35:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.104+0000 7f460f200640 1 --2- 172.21.15.19:0/3785616054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461006dce0 0x7f46100ce7e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.104+0000 7f460f200640 1 --2- 172.21.15.19:0/3785616054 >> v2:172.21.15.169:3300/0 conn(0x7f4610070d20 0x7f4610161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.104+0000 7f460f200640 1 -- 172.21.15.19:0/3785616054 >> 172.21.15.19:0/3785616054 conn(0x7f4610073160 msgr2=0x7f4610073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.104+0000 7f460f200640 1 -- 172.21.15.19:0/3785616054 shutdown_connections 2024-04-25T11:35:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.104+0000 7f460f200640 1 -- 172.21.15.19:0/3785616054 wait complete. 2024-04-25T11:35:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.106+0000 7f460f200640 1 Processor -- start 2024-04-25T11:35:26.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.106+0000 7f460f200640 1 -- start start 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461006f930 0x7f4610158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4610070d20 0x7f46101f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46101f80b0 0x7f461015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f461015e2f0 con 0x7f46101f80b0 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4610162070 con 0x7f4610070d20 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f461015e470 con 0x7f461006f930 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46101f80b0 0x7f461015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46101f80b0 0x7f461015bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41160/0 (socket says 172.21.15.19:41160) 2024-04-25T11:35:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460e800640 1 -- 172.21.15.19:0/2638530 learned_addr learned my addr 172.21.15.19:0/2638530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f4607e00640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4610070d20 0x7f46101f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460de00640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461006f930 0x7f4610158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.108+0000 7f460e800640 1 -- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461006f930 msgr2=0x7f4610158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f460e800640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461006f930 0x7f4610158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f460e800640 1 -- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4610070d20 msgr2=0x7f46101f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f460e800640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4610070d20 0x7f46101f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f460e800640 1 -- 172.21.15.19:0/2638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46101f8580 con 0x7f46101f80b0 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f460e800640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46101f80b0 0x7f461015bd30 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f460802abe0 tx=0x7f4608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f4606000640 1 -- 172.21.15.19:0/2638530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460803f070 con 0x7f46101f80b0 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f4606000640 1 -- 172.21.15.19:0/2638530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4608031bc0 con 0x7f46101f80b0 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46101f8860 con 0x7f46101f80b0 2024-04-25T11:35:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f4606000640 1 -- 172.21.15.19:0/2638530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608044460 con 0x7f46101f80b0 2024-04-25T11:35:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.109+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101f4860 con 0x7f46101f80b0 2024-04-25T11:35:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.111+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46100b0ee0 con 0x7f46101f80b0 2024-04-25T11:35:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.112+0000 7f4606000640 1 -- 172.21.15.19:0/2638530 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4608007790 con 0x7f46101f80b0 2024-04-25T11:35:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.113+0000 7f4606000640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45ec0744b0 0x7f45ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.113+0000 7f4606000640 1 -- 172.21.15.19:0/2638530 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f46080b6c20 con 0x7f46101f80b0 2024-04-25T11:35:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.114+0000 7f460de00640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45ec0744b0 0x7f45ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.115+0000 7f460de00640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45ec0744b0 0x7f45ec0769c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4600006fd0 tx=0x7f4600008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.118+0000 7f4606000640 1 -- 172.21.15.19:0/2638530 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46100b0ee0 con 0x7f46101f80b0 2024-04-25T11:35:26.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.323+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46101f4ac0 con 0x7f46101f80b0 2024-04-25T11:35:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.324+0000 7f4606000640 1 -- 172.21.15.19:0/2638530 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f4608089390 con 0x7f46101f80b0 2024-04-25T11:35:26.313 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:35:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45ec0744b0 msgr2=0x7f45ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45ec0744b0 0x7f45ec0769c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4600006fd0 tx=0x7f4600008040 comp rx=0 tx=0).stop 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46101f80b0 msgr2=0x7f461015bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46101f80b0 0x7f461015bd30 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f460802abe0 tx=0x7f4608004a40 comp rx=0 tx=0).stop 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 shutdown_connections 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45ec0744b0 0x7f45ec0769c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4610070d20 0x7f46101f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46101f80b0 0x7f461015bd30 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 --2- 172.21.15.19:0/2638530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461006f930 0x7f4610158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.327+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 >> 172.21.15.19:0/2638530 conn(0x7f4610073160 msgr2=0x7f46100d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.328+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 shutdown_connections 2024-04-25T11:35:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.328+0000 7f460f200640 1 -- 172.21.15.19:0/2638530 wait complete. 2024-04-25T11:35:26.343 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:35:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.634+0000 7f8288c00640 1 Processor -- start 2024-04-25T11:35:26.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.634+0000 7f8288c00640 1 -- start start 2024-04-25T11:35:26.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.636+0000 7f8288c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f82841612f0 0x7f82840d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.636+0000 7f8288c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f828406f3e0 con 0x7f82840d0ed0 2024-04-25T11:35:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8288c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f828406f6c0 con 0x7f82841434d0 2024-04-25T11:35:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8288c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f82841438e0 con 0x7f82841612f0 2024-04-25T11:35:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8282800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f82841612f0 0x7f82840d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8282800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f82841612f0 0x7f82840d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41162/0 (socket says 172.21.15.19:41162) 2024-04-25T11:35:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f827be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f82840d0ed0 0x7f8284160be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38568/0 (socket says 172.21.15.19:38568) 2024-04-25T11:35:26.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8282800640 1 -- 172.21.15.19:0/216165078 learned_addr learned my addr 172.21.15.19:0/216165078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8282800640 1 -- 172.21.15.19:0/216165078 >> v1:172.21.15.169:6789/0 conn(0x7f82841434d0 legacy=0x7f8284070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8282800640 1 -- 172.21.15.19:0/216165078 >> v1:172.21.15.19:6789/0 conn(0x7f82840d0ed0 legacy=0x7f8284160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.637+0000 7f8282800640 1 -- 172.21.15.19:0/216165078 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828406f9a0 con 0x7f82841612f0 2024-04-25T11:35:26.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.638+0000 7f8282800640 1 --2- 172.21.15.19:0/216165078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841612f0 0x7f82840d12e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f827c009e80 tx=0x7f827c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0274235823d45d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.639+0000 7f8281400640 1 -- 172.21.15.19:0/216165078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c036070 con 0x7f82841612f0 2024-04-25T11:35:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.639+0000 7f8281400640 1 -- 172.21.15.19:0/216165078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827c031600 con 0x7f82841612f0 2024-04-25T11:35:26.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.639+0000 7f8281400640 1 -- 172.21.15.19:0/216165078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c0319d0 con 0x7f82841612f0 2024-04-25T11:35:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.640+0000 7f8288c00640 1 -- 172.21.15.19:0/216165078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841612f0 msgr2=0x7f82840d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.640+0000 7f8288c00640 1 --2- 172.21.15.19:0/216165078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841612f0 0x7f82840d12e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f827c009e80 tx=0x7f827c02a7a0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.640+0000 7f8288c00640 1 -- 172.21.15.19:0/216165078 shutdown_connections 2024-04-25T11:35:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.640+0000 7f8288c00640 1 --2- 172.21.15.19:0/216165078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841612f0 0x7f82840d12e0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.640+0000 7f8288c00640 1 -- 172.21.15.19:0/216165078 >> 172.21.15.19:0/216165078 conn(0x7f82840d1ae0 msgr2=0x7f82840d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.640+0000 7f8288c00640 1 -- 172.21.15.19:0/216165078 shutdown_connections 2024-04-25T11:35:26.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.640+0000 7f8288c00640 1 -- 172.21.15.19:0/216165078 wait complete. 2024-04-25T11:35:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.642+0000 7f8288c00640 1 Processor -- start 2024-04-25T11:35:26.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.642+0000 7f8288c00640 1 -- start start 2024-04-25T11:35:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f8288c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82840d0ed0 0x7f8284144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f8288c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841434d0 0x7f828415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f8288c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82841f9830 0x7f82841f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f8288c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82840b51a0 con 0x7f82841434d0 2024-04-25T11:35:26.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f8288c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f828406f090 con 0x7f82840d0ed0 2024-04-25T11:35:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f8288c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f82840b5320 con 0x7f82841f9830 2024-04-25T11:35:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f827be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841434d0 0x7f828415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f827be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841434d0 0x7f828415eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41164/0 (socket says 172.21.15.19:41164) 2024-04-25T11:35:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.644+0000 7f827be00640 1 -- 172.21.15.19:0/3407273364 learned_addr learned my addr 172.21.15.19:0/3407273364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:26.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f8281e00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82840d0ed0 0x7f8284144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f8282800640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82841f9830 0x7f82841f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f827be00640 1 -- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82841f9830 msgr2=0x7f82841f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f827be00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82841f9830 0x7f82841f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f827be00640 1 -- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82840d0ed0 msgr2=0x7f8284144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f827be00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82840d0ed0 0x7f8284144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f827be00640 1 -- 172.21.15.19:0/3407273364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82841fa180 con 0x7f82841434d0 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f827be00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841434d0 0x7f828415eef0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c470 tx=0x7f826c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.645+0000 7f827a000640 1 -- 172.21.15.19:0/3407273364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0074e0 con 0x7f82841434d0 2024-04-25T11:35:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.646+0000 7f827a000640 1 -- 172.21.15.19:0/3407273364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c007680 con 0x7f82841434d0 2024-04-25T11:35:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.646+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82841453b0 con 0x7f82841434d0 2024-04-25T11:35:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.646+0000 7f827a000640 1 -- 172.21.15.19:0/3407273364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c004a00 con 0x7f82841434d0 2024-04-25T11:35:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.646+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8284145970 con 0x7f82841434d0 2024-04-25T11:35:26.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.647+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f828406d260 con 0x7f82841434d0 2024-04-25T11:35:26.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.648+0000 7f827a000640 1 -- 172.21.15.19:0/3407273364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f826c01a020 con 0x7f82841434d0 2024-04-25T11:35:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.649+0000 7f827a000640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82600744b0 0x7f82600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.649+0000 7f827a000640 1 -- 172.21.15.19:0/3407273364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f826c094a00 con 0x7f82841434d0 2024-04-25T11:35:26.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.650+0000 7f8281e00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82600744b0 0x7f82600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.651+0000 7f8281e00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82600744b0 0x7f82600769c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.654+0000 7f827a000640 1 -- 172.21.15.19:0/3407273364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82841fa180 con 0x7f82841434d0 2024-04-25T11:35:26.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.882+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8284145140 con 0x7f82841434d0 2024-04-25T11:35:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.883+0000 7f827a000640 1 -- 172.21.15.19:0/3407273364 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) ==== 80+0+0 (secure 0 0 0) 0x7f826c05f180 con 0x7f82841434d0 2024-04-25T11:35:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:35:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82600744b0 msgr2=0x7f82600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82600744b0 0x7f82600769c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).stop 2024-04-25T11:35:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841434d0 msgr2=0x7f828415eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841434d0 0x7f828415eef0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c470 tx=0x7f826c00c930 comp rx=0 tx=0).stop 2024-04-25T11:35:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 shutdown_connections 2024-04-25T11:35:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82600744b0 0x7f82600769c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f82841f9830 0x7f82841f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82841434d0 0x7f828415eef0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 --2- 172.21.15.19:0/3407273364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82840d0ed0 0x7f8284144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 >> 172.21.15.19:0/3407273364 conn(0x7f82840d1ae0 msgr2=0x7f828415b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 shutdown_connections 2024-04-25T11:35:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:26.886+0000 7f8288c00640 1 -- 172.21.15.19:0/3407273364 wait complete. 2024-04-25T11:35:26.904 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:35:26.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:26.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:27.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:35:27.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:27.058 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:35:27.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:35:27.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.368+0000 7f5c99a00640 1 Processor -- start 2024-04-25T11:35:27.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.368+0000 7f5c99a00640 1 -- start start 2024-04-25T11:35:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.370+0000 7f5c99a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c940b7c60 0x7f5c940d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.370+0000 7f5c99a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5c9406c8a0 0x7f5c9406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c99a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c940d1f40 con 0x7f5c9406ecb0 2024-04-25T11:35:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c99a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5c940d0430 con 0x7f5c9406c8a0 2024-04-25T11:35:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c99a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5c940d21e0 con 0x7f5c940b7c60 2024-04-25T11:35:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c93400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c940b7c60 0x7f5c940d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c99000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5c9406c8a0 0x7f5c9406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c93400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c940b7c60 0x7f5c940d3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:50114/0 (socket says 172.21.15.19:50114) 2024-04-25T11:35:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c93e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5c9406ecb0 0x7f5c940b7550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:34242/0 (socket says 172.21.15.19:34242) 2024-04-25T11:35:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.371+0000 7f5c93400640 1 -- 172.21.15.19:0/2594539812 learned_addr learned my addr 172.21.15.19:0/2594539812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.372+0000 7f5c99000640 1 -- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c940b7c60 msgr2=0x7f5c940d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.372+0000 7f5c99000640 1 --2- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c940b7c60 0x7f5c940d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.372+0000 7f5c99000640 1 -- 172.21.15.19:0/2594539812 >> v1:172.21.15.169:6789/0 conn(0x7f5c9406ecb0 legacy=0x7f5c940b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.372+0000 7f5c99000640 1 -- 172.21.15.19:0/2594539812 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c940d06d0 con 0x7f5c9406c8a0 2024-04-25T11:35:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.372+0000 7f5c93400640 1 --2- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c940b7c60 0x7f5c940d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:35:27.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.374+0000 7f5c99000640 1 --2- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c9406c8a0 0x7f5c9406ccb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0097a0 tx=0x7f5c8c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9f1aa1ce9f5d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.374+0000 7f5c92a00640 1 -- 172.21.15.19:0/2594539812 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c037070 con 0x7f5c9406c8a0 2024-04-25T11:35:27.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c92a00640 1 -- 172.21.15.19:0/2594539812 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c032440 con 0x7f5c9406c8a0 2024-04-25T11:35:27.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c92a00640 1 -- 172.21.15.19:0/2594539812 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c032810 con 0x7f5c9406c8a0 2024-04-25T11:35:27.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c99a00640 1 -- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c9406c8a0 msgr2=0x7f5c9406ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:27.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c9406c8a0 0x7f5c9406ccb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0097a0 tx=0x7f5c8c02a180 comp rx=0 tx=0).stop 2024-04-25T11:35:27.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c99a00640 1 -- 172.21.15.19:0/2594539812 shutdown_connections 2024-04-25T11:35:27.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c9406c8a0 0x7f5c9406ccb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2594539812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c940b7c60 0x7f5c940d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.375+0000 7f5c99a00640 1 -- 172.21.15.19:0/2594539812 >> 172.21.15.19:0/2594539812 conn(0x7f5c9409b7d0 msgr2=0x7f5c9409bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:27.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.376+0000 7f5c99a00640 1 -- 172.21.15.19:0/2594539812 shutdown_connections 2024-04-25T11:35:27.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.376+0000 7f5c99a00640 1 -- 172.21.15.19:0/2594539812 wait complete. 2024-04-25T11:35:27.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.378+0000 7f5c99a00640 1 Processor -- start 2024-04-25T11:35:27.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.378+0000 7f5c99a00640 1 -- start start 2024-04-25T11:35:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c99a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9406c8a0 0x7f5c940c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c99a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c9406ecb0 0x7f5c940c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c99a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c940b0c10 0x7f5c940b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c99a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94163bb0 con 0x7f5c9406c8a0 2024-04-25T11:35:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c99a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5c940d24c0 con 0x7f5c940b0c10 2024-04-25T11:35:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c99a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5c94163d30 con 0x7f5c9406ecb0 2024-04-25T11:35:27.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9406c8a0 0x7f5c940c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:27.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9406c8a0 0x7f5c940c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41178/0 (socket says 172.21.15.19:41178) 2024-04-25T11:35:27.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c93e00640 1 -- 172.21.15.19:0/2754024082 learned_addr learned my addr 172.21.15.19:0/2754024082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:27.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c93400640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c9406ecb0 0x7f5c940c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:27.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.380+0000 7f5c99000640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c940b0c10 0x7f5c940b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c93e00640 1 -- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c9406ecb0 msgr2=0x7f5c940c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c93e00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c9406ecb0 0x7f5c940c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c93e00640 1 -- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c940b0c10 msgr2=0x7f5c940b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c93e00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c940b0c10 0x7f5c940b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c93e00640 1 -- 172.21.15.19:0/2754024082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c940c8890 con 0x7f5c9406c8a0 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c93400640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c9406ecb0 0x7f5c940c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c93e00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9406c8a0 0x7f5c940c8350 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400cad0 tx=0x7f5c8400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c8be00640 1 -- 172.21.15.19:0/2754024082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84017070 con 0x7f5c9406c8a0 2024-04-25T11:35:27.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c8be00640 1 -- 172.21.15.19:0/2754024082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8400a820 con 0x7f5c9406c8a0 2024-04-25T11:35:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c940b1560 con 0x7f5c9406c8a0 2024-04-25T11:35:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.381+0000 7f5c8be00640 1 -- 172.21.15.19:0/2754024082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400abf0 con 0x7f5c9406c8a0 2024-04-25T11:35:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.382+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9415ad10 con 0x7f5c9406c8a0 2024-04-25T11:35:27.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.383+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c94050740 con 0x7f5c9406c8a0 2024-04-25T11:35:27.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.384+0000 7f5c8be00640 1 -- 172.21.15.19:0/2754024082 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5c8401a070 con 0x7f5c9406c8a0 2024-04-25T11:35:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.385+0000 7f5c8be00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c700744b0 0x7f5c700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:27.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.385+0000 7f5c8be00640 1 -- 172.21.15.19:0/2754024082 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f5c84094cf0 con 0x7f5c9406c8a0 2024-04-25T11:35:27.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.386+0000 7f5c93400640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c700744b0 0x7f5c700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.387+0000 7f5c93400640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c700744b0 0x7f5c700769c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5c941f9c30 tx=0x7f5c7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.390+0000 7f5c8be00640 1 -- 172.21.15.19:0/2754024082 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c940b1560 con 0x7f5c9406c8a0 2024-04-25T11:35:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.636+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c940cfb50 con 0x7f5c9406c8a0 2024-04-25T11:35:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.637+0000 7f5c8be00640 1 -- 172.21.15.19:0/2754024082 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5189 (secure 0 0 0) 0x7f5c8405f3e0 con 0x7f5c9406c8a0 2024-04-25T11:35:27.625 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:27.625 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:35:17.059328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7374},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7374":{"gid":7374,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/2716758396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2716758396},{"type":"v1","addr":"172.21.15.19:6835","nonce":2716758396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7374,"qdb_cluster":[7374]},"id":6}]} 2024-04-25T11:35:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 90 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.639+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c700744b0 msgr2=0x7f5c700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.639+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c700744b0 0x7f5c700769c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5c941f9c30 tx=0x7f5c7c008040 comp rx=0 tx=0).stop 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.639+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9406c8a0 msgr2=0x7f5c940c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.639+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9406c8a0 0x7f5c940c8350 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400cad0 tx=0x7f5c8400cf90 comp rx=0 tx=0).stop 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 shutdown_connections 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c700744b0 0x7f5c700769c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c940b0c10 0x7f5c940b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c9406ecb0 0x7f5c940c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 --2- 172.21.15.19:0/2754024082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c9406c8a0 0x7f5c940c8350 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 >> 172.21.15.19:0/2754024082 conn(0x7f5c9409b7d0 msgr2=0x7f5c9406d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:27.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 shutdown_connections 2024-04-25T11:35:27.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:27.640+0000 7f5c99a00640 1 -- 172.21.15.19:0/2754024082 wait complete. 2024-04-25T11:35:27.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T11:35:27.976 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-04-25T11:35:27.976 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T11:35:27.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:35:28.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.269+0000 7f155fe00640 1 Processor -- start 2024-04-25T11:35:28.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.269+0000 7f155fe00640 1 -- start start 2024-04-25T11:35:28.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.272+0000 7f155fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600b6e10 0x7f156015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.272+0000 7f155fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15601608d0 0x7f1560160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.272+0000 7f155fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1560161220 con 0x7f15600cf2f0 2024-04-25T11:35:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.272+0000 7f155fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f156006f870 con 0x7f15601608d0 2024-04-25T11:35:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.272+0000 7f155fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15600b7220 con 0x7f15600b6e10 2024-04-25T11:35:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600b6e10 0x7f156015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15601608d0 0x7f1560160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600b6e10 0x7f156015a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:50134/0 (socket says 172.21.15.19:50134) 2024-04-25T11:35:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155e000640 1 -- 172.21.15.19:0/3238106502 learned_addr learned my addr 172.21.15.19:0/3238106502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155ea00640 1 --1- 172.21.15.19:0/3238106502 >> v1:172.21.15.169:6789/0 conn(0x7f15600cf2f0 0x7f15600b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:34246/0 (socket says 172.21.15.19:34246) 2024-04-25T11:35:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155f400640 1 -- 172.21.15.19:0/3238106502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600b6e10 msgr2=0x7f156015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155f400640 1 --2- 172.21.15.19:0/3238106502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600b6e10 0x7f156015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155f400640 1 -- 172.21.15.19:0/3238106502 >> v1:172.21.15.169:6789/0 conn(0x7f15600cf2f0 legacy=0x7f15600b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.273+0000 7f155f400640 1 -- 172.21.15.19:0/3238106502 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15600b74d0 con 0x7f15601608d0 2024-04-25T11:35:28.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.275+0000 7f155f400640 1 --2- 172.21.15.19:0/3238106502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601608d0 0x7f1560160ce0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1558009d90 tx=0x7f155802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae75cd2cb613ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.275+0000 7f155d600640 1 -- 172.21.15.19:0/3238106502 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558007740 con 0x7f15601608d0 2024-04-25T11:35:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155d600640 1 -- 172.21.15.19:0/3238106502 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15580078e0 con 0x7f15601608d0 2024-04-25T11:35:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155d600640 1 -- 172.21.15.19:0/3238106502 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558007cb0 con 0x7f15601608d0 2024-04-25T11:35:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155fe00640 1 -- 172.21.15.19:0/3238106502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601608d0 msgr2=0x7f1560160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155fe00640 1 --2- 172.21.15.19:0/3238106502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601608d0 0x7f1560160ce0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1558009d90 tx=0x7f155802a660 comp rx=0 tx=0).stop 2024-04-25T11:35:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155fe00640 1 -- 172.21.15.19:0/3238106502 shutdown_connections 2024-04-25T11:35:28.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155fe00640 1 --2- 172.21.15.19:0/3238106502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15601608d0 0x7f1560160ce0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155fe00640 1 --2- 172.21.15.19:0/3238106502 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600b6e10 0x7f156015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155fe00640 1 -- 172.21.15.19:0/3238106502 >> 172.21.15.19:0/3238106502 conn(0x7f15600d1720 msgr2=0x7f15600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:28.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.276+0000 7f155fe00640 1 -- 172.21.15.19:0/3238106502 shutdown_connections 2024-04-25T11:35:28.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.277+0000 7f155fe00640 1 -- 172.21.15.19:0/3238106502 wait complete. 2024-04-25T11:35:28.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.279+0000 7f155fe00640 1 Processor -- start 2024-04-25T11:35:28.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.279+0000 7f155fe00640 1 -- start start 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15600b6e10 0x7f15601532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600cf2f0 0x7f1560158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15601608d0 0x7f15600afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f156006f500 con 0x7f15601608d0 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f156006f220 con 0x7f15600b6e10 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f156006fd00 con 0x7f15600cf2f0 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15601608d0 0x7f15600afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15601608d0 0x7f15600afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41188/0 (socket says 172.21.15.19:41188) 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155f400640 1 -- 172.21.15.19:0/124943790 learned_addr learned my addr 172.21.15.19:0/124943790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155ea00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15600b6e10 0x7f15601532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.281+0000 7f155e000640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600cf2f0 0x7f1560158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f155f400640 1 -- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600cf2f0 msgr2=0x7f1560158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f155f400640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600cf2f0 0x7f1560158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f155f400640 1 -- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15600b6e10 msgr2=0x7f15601532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f155f400640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15600b6e10 0x7f15601532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f155f400640 1 -- 172.21.15.19:0/124943790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1560153810 con 0x7f15601608d0 2024-04-25T11:35:28.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f155f400640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15601608d0 0x7f15600afdd0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1558009eb0 tx=0x7f1558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f1556a00640 1 -- 172.21.15.19:0/124943790 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155803f070 con 0x7f15601608d0 2024-04-25T11:35:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f1556a00640 1 -- 172.21.15.19:0/124943790 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1558031b40 con 0x7f15601608d0 2024-04-25T11:35:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f1556a00640 1 -- 172.21.15.19:0/124943790 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558044460 con 0x7f15601608d0 2024-04-25T11:35:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.282+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1560157390 con 0x7f15601608d0 2024-04-25T11:35:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.283+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1560157950 con 0x7f15601608d0 2024-04-25T11:35:28.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.284+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1560050740 con 0x7f15601608d0 2024-04-25T11:35:28.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.285+0000 7f1556a00640 1 -- 172.21.15.19:0/124943790 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f155803b410 con 0x7f15601608d0 2024-04-25T11:35:28.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.286+0000 7f1556a00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15400744e0 0x7f15400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.286+0000 7f1556a00640 1 -- 172.21.15.19:0/124943790 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f15580b6a10 con 0x7f15601608d0 2024-04-25T11:35:28.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.287+0000 7f155ea00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15400744e0 0x7f15400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.288+0000 7f155ea00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15400744e0 0x7f15400769f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1550000f60 tx=0x7f1550007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.290+0000 7f1556a00640 1 -- 172.21.15.19:0/124943790 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1560153810 con 0x7f15601608d0 2024-04-25T11:35:28.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.535+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1560073df0 con 0x7f15601608d0 2024-04-25T11:35:28.524 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:35:28.536+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:35:28.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.540+0000 7f1556a00640 1 -- 172.21.15.19:0/124943790 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f1558081190 con 0x7f15601608d0 2024-04-25T11:35:28.528 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:35:28.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15400744e0 msgr2=0x7f15400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15400744e0 0x7f15400769f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1550000f60 tx=0x7f1550007ce0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15601608d0 msgr2=0x7f15600afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15601608d0 0x7f15600afdd0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1558009eb0 tx=0x7f1558004a40 comp rx=0 tx=0).stop 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 shutdown_connections 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15400744e0 0x7f15400769f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15601608d0 0x7f15600afdd0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15600cf2f0 0x7f1560158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 --2- 172.21.15.19:0/124943790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15600b6e10 0x7f15601532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.545+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 >> 172.21.15.19:0/124943790 conn(0x7f15600d1720 msgr2=0x7f15600d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.546+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 shutdown_connections 2024-04-25T11:35:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.546+0000 7f155fe00640 1 -- 172.21.15.19:0/124943790 wait complete. 2024-04-25T11:35:28.540 INFO:tasks.ceph.mds.a.smithi019.stderr: -4487> 2024-04-25T11:35:19.363+0000 7f3e62a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:35:28.560 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2024-04-25T11:35:28.680 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:35:28.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.734+0000 7f8fba40bf40 1 Processor -- start 2024-04-25T11:35:28.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.734+0000 7f8fba40bf40 1 -- start start 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.735+0000 7f8fba40bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f98003c40 0x560e235dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.735+0000 7f8fba40bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e235dc480 0x560e2367c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.735+0000 7f8fba40bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 0x560e235b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fba40bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e23559d20 con 0x560e235dc480 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fba40bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560e2355dd60 con 0x560e23684990 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fba40bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560e235fc5c0 con 0x7f8f98003c40 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e235dc480 0x560e2367c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e235dc480 0x560e2367c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41232/0 (socket says 172.21.15.19:41232) 2024-04-25T11:35:28.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 -- 172.21.15.19:0/1044711810 learned_addr learned my addr 172.21.15.19:0/1044711810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f98003c40 0x560e235dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb3400640 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 0x560e235b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 -- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f98003c40 msgr2=0x560e235dd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f98003c40 0x560e235dd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 -- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 msgr2=0x560e235b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 0x560e235b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2000640 1 -- 172.21.15.19:0/1044711810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e235b0c00 con 0x560e235dc480 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.736+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f98003c40 0x560e235dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.737+0000 7f8fb2000640 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e235dc480 0x560e2367c320 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40065b0 tx=0x7f8fa4006a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59d7e6517380645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.737+0000 7f8fb0c00640 1 -- 172.21.15.19:0/1044711810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40062e0 con 0x560e235dc480 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.737+0000 7f8fb0c00640 1 -- 172.21.15.19:0/1044711810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa4003670 con 0x560e235dc480 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.737+0000 7f8fb0c00640 1 -- 172.21.15.19:0/1044711810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4003aa0 con 0x560e235dc480 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 -- 172.21.15.19:0/1044711810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e235dc480 msgr2=0x560e2367c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e235dc480 0x560e2367c320 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40065b0 tx=0x7f8fa4006a70 comp rx=0 tx=0).stop 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 -- 172.21.15.19:0/1044711810 shutdown_connections 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 0x560e235b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560e235dc480 0x560e2367c320 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/1044711810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f98003c40 0x560e235dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 -- 172.21.15.19:0/1044711810 >> 172.21.15.19:0/1044711810 conn(0x560e234d5ac0 msgr2=0x560e23685430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 -- 172.21.15.19:0/1044711810 shutdown_connections 2024-04-25T11:35:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.738+0000 7f8fba40bf40 1 -- 172.21.15.19:0/1044711810 wait complete. 2024-04-25T11:35:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.739+0000 7f8fba40bf40 1 Processor -- start 2024-04-25T11:35:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.739+0000 7f8fba40bf40 1 -- start start 2024-04-25T11:35:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.740+0000 7f8fba40bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560e235dc480 0x560e23677620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.740+0000 7f8fba40bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 0x560e23684e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.740+0000 7f8fba40bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f98003c40 0x560e236853c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.740+0000 7f8fba40bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560e235b1510 con 0x7f8f98003c40 2024-04-25T11:35:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.740+0000 7f8fba40bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560e235b3900 con 0x560e23684990 2024-04-25T11:35:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.740+0000 7f8fba40bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560e235b3be0 con 0x560e235dc480 2024-04-25T11:35:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f98003c40 0x560e236853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f98003c40 0x560e236853c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41236/0 (socket says 172.21.15.19:41236) 2024-04-25T11:35:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 learned_addr learned my addr 172.21.15.19:0/2614629688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560e235dc480 msgr2=0x560e23677620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560e235dc480 0x560e23677620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 msgr2=0x560e23684e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560e23684990 0x560e23684e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e23677e70 con 0x7f8f98003c40 2024-04-25T11:35:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.741+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f98003c40 0x560e236853c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0258a0 tx=0x7f8fac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.742+0000 7f8fab400640 1 -- 172.21.15.19:0/2614629688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac062070 con 0x7f8f98003c40 2024-04-25T11:35:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.742+0000 7f8fab400640 1 -- 172.21.15.19:0/2614629688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac000ed0 con 0x7f8f98003c40 2024-04-25T11:35:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.742+0000 7f8fab400640 1 -- 172.21.15.19:0/2614629688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0014e0 con 0x7f8f98003c40 2024-04-25T11:35:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.742+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560e23678150 con 0x7f8f98003c40 2024-04-25T11:35:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.742+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e23679a10 con 0x7f8f98003c40 2024-04-25T11:35:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.744+0000 7f8fab400640 1 -- 172.21.15.19:0/2614629688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8fac065070 con 0x7f8f98003c40 2024-04-25T11:35:28.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.745+0000 7f8fab400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f90072ec0 0x7f8f900753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.745+0000 7f8fab400640 1 -- 172.21.15.19:0/2614629688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f8fac0d7430 con 0x7f8f98003c40 2024-04-25T11:35:28.733 INFO:teuthology.orchestra.run.smithi019.stdout:���� 2024-04-25T11:35:28.733 INFO:teuthology.orchestra.run.smithi019.stdout: �� 2024-04-25T11:35:28.733 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.745+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560e2367a440 0x560e2367a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.745+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.0 12:00000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e2367a440 2024-04-25T11:35:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.745+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f90072ec0 0x7f8f900753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.745+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560e2367a440 0x560e2367a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.746+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560e2367a440 0x560e2367a850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.747+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f90072ec0 0x7f8f900753d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005fd0 tx=0x7f8f9c0058c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.749+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa400f030 con 0x560e2367a440 2024-04-25T11:35:28.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.749+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f8fa4013930 0x7f8fa40123c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.749+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:08000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4013930 2024-04-25T11:35:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.750+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f8fa4013930 0x7f8fa40123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.751+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f8fa4013930 0x7f8fa40123c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.759+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=177b] v92'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.759+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f8f9c017940 0x7f8f9c0073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.759+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:3 12.8 12:10000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f9c017940 2024-04-25T11:35:28.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.760+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f8f9c017940 0x7f8f9c0073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.762+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f8f9c017940 0x7f8f9c0073f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.762+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa400f030 con 0x7f8f9c017940 2024-04-25T11:35:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.763+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:4 12.18 12:18000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f9c017940 2024-04-25T11:35:28.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.763+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa400f030 con 0x7f8f9c017940 2024-04-25T11:35:28.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.764+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f8fa4021950 0x7f8fa4023400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.764+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:5 12.4 12:20000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4021950 2024-04-25T11:35:28.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.764+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f8fa4021950 0x7f8fa4023400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.765+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f8fa4021950 0x7f8fa4023400 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.765+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.2 v2:172.21.15.19:6813/2392070031 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x7f8fa4021950 2024-04-25T11:35:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.765+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:6 12.14 12:28000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e2367a440 2024-04-25T11:35:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.766+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(6 [pgnls start_epoch 93 out=149b] v92'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f8fa400f030 con 0x560e2367a440 2024-04-25T11:35:28.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.766+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:7 12.c 12:30000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f9c017940 2024-04-25T11:35:28.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.767+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.767+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7f8fa40203d0 0x7f8fa40207e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.767+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:8 12.1c 12:38000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa40203d0 2024-04-25T11:35:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.767+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7f8fa40203d0 0x7f8fa40207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.769+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7f8fa40203d0 0x7f8fa40207e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.777+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(8 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa40203d0 2024-04-25T11:35:28.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.777+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:9 12.2 12:40000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4021950 2024-04-25T11:35:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.777+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.2 v2:172.21.15.19:6813/2392070031 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x7f8fa4021950 2024-04-25T11:35:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.777+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x560e235b4a50 0x7f8fac0a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.777+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:10 12.12 12:48000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e235b4a50 2024-04-25T11:35:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.778+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x560e235b4a50 0x7f8fac0a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.779+0000 7f8fb2000640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x560e235b4a50 0x7f8fac0a01b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.780+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.4 v2:172.21.15.169:6816/3656468664 1 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa4031040 con 0x560e235b4a50 2024-04-25T11:35:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.780+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:11 12.a 12:50000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e235b4a50 2024-04-25T11:35:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.781+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.4 v2:172.21.15.169:6816/3656468664 2 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa4031040 con 0x560e235b4a50 2024-04-25T11:35:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.782+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:12 12.1a 12:58000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4021950 2024-04-25T11:35:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.782+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.2 v2:172.21.15.19:6813/2392070031 3 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x7f8fa4021950 2024-04-25T11:35:28.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.782+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.6 12:60000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4013930 2024-04-25T11:35:28.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.783+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(13 [pgnls start_epoch 93 out=77b] v92'4 uv4 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.783+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:14 12.16 12:68000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4021950 2024-04-25T11:35:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.784+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.2 v2:172.21.15.19:6813/2392070031 4 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x7f8fa4021950 2024-04-25T11:35:28.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.784+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:15 12.e 12:70000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f9c017940 2024-04-25T11:35:28.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.785+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(15 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.785+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:16 12.1e 12:78000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4021950 2024-04-25T11:35:28.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.785+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.2 v2:172.21.15.19:6813/2392070031 5 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x7f8fa4021950 2024-04-25T11:35:28.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.786+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560e235fca50 0x7f8fac0d6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.786+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:17 12.1 12:80000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e235fca50 2024-04-25T11:35:28.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.786+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560e235fca50 0x7f8fac0d6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.787+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560e235fca50 0x7f8fac0d6cd0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.790+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.790+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:18 12.11 12:88000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4021950 2024-04-25T11:35:28.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.791+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.2 v2:172.21.15.19:6813/2392070031 6 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x7f8fa4021950 2024-04-25T11:35:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.791+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.9 12:90000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4013930 2024-04-25T11:35:28.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.792+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.792+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:20 12.19 12:98000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f9c017940 2024-04-25T11:35:28.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.793+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.793+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:21 12.5 12:a0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e2367a440 2024-04-25T11:35:28.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.794+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.794+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:22 12.15 12:a8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e235fca50 2024-04-25T11:35:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.794+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(22 [pgnls start_epoch 93 out=97b] v91'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.794+0000 7f8fb3400640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f8f980022f0 0x7f8fac0d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.794+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:23 12.d 12:b0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f980022f0 2024-04-25T11:35:28.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.795+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f8f980022f0 0x7f8fac0d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:28.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.796+0000 7f8fb2a00640 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f8f980022f0 0x7f8fac0d50c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.796+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(23 [pgnls start_epoch 93 out=74b] v91'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8f980022f0 2024-04-25T11:35:28.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.796+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.1d 12:b8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa4013930 2024-04-25T11:35:28.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.797+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(24 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.797+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:25 12.3 12:c0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e235b4a50 2024-04-25T11:35:28.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.798+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.4 v2:172.21.15.169:6816/3656468664 3 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa4031040 con 0x560e235b4a50 2024-04-25T11:35:28.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.799+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:26 12.13 12:c8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8fa40203d0 2024-04-25T11:35:28.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.800+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(26 [pgnls start_epoch 93 out=98b] v92'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa40203d0 2024-04-25T11:35:28.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.800+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:27 12.b 12:d0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f9c017940 2024-04-25T11:35:28.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.801+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(27 [pgnls start_epoch 93 out=76b] v92'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.801+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:28 12.1b 12:d8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e2367a440 2024-04-25T11:35:28.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.802+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.802+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:29 12.7 12:e0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f9c017940 2024-04-25T11:35:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.803+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.803+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:30 12.17 12:e8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x560e235fca50 2024-04-25T11:35:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.803+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.804+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:31 12.f 12:f0000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f980022f0 2024-04-25T11:35:28.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.804+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(31 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8f980022f0 2024-04-25T11:35:28.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.804+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:32 12.1f 12:f8000000:%01:::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x560e23672410 con 0x7f8f980022f0 2024-04-25T11:35:28.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.805+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(32 [pgnls start_epoch 93 out=101b] v92'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8f980022f0 2024-04-25T11:35:28.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.805+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2367e6f0 con 0x7f8fa4013930 2024-04-25T11:35:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.807+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(33 601.00000000 [read 0~0] v0'0 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.807+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:34 12.10 12:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2367eae0 con 0x7f8fa4013930 2024-04-25T11:35:28.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.808+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(34 601.00000000 [getxattrs out=124b] v0'0 uv4 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.808+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:35 12.10 12:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2367ef00 con 0x7f8fa4013930 2024-04-25T11:35:28.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.809+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(35 601.00000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.809+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:36 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2367f2f0 con 0x7f8fa4013930 2024-04-25T11:35:28.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.810+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(36 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.810+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:37 12.10 12:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2367afb0 con 0x7f8fa4013930 2024-04-25T11:35:28.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.811+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(37 602.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.811+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:38 12.10 12:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372ccb0 con 0x7f8fa4013930 2024-04-25T11:35:28.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.813+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(38 602.00000000 [getxattrs out=124b] v0'0 uv6 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.813+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:39 12.10 12:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.814+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(39 602.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.814+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:40 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372e060 con 0x7f8fa4013930 2024-04-25T11:35:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.815+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(40 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.815+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:41 12.10 12:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.817+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(41 10000000000.00000000 [read 0~0] v0'0 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.817+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:42 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.818+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(42 10000000000.00000000 [getxattrs out=124b] v0'0 uv10 ondisk = 0) ==== 164+0+124 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.818+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:43 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.819+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(43 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.819+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:44 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372e060 con 0x7f8fa4013930 2024-04-25T11:35:28.808 INFO:teuthology.orchestra.run.smithi019.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1 ���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2 ���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir ���� ��  (�?*f��*(�?*f��*8`�?*f��-8`�?*f��-���� ��WQ sixmegs_head9i* ��?*f��-���$@@ `���������?*f��-�?*f��*(8`�?*f��-8`�?*f��- 2024-04-25T11:35:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.820+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals out=594b] v0'0 uv10 ondisk = 0) ==== 164+0+594 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.820+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:45 12.10 12:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.821+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(45 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c012030 con 0x7f8fa4013930 2024-04-25T11:35:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.822+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:46 12.10 12:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.823+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(46 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.823+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:47 12.10 12:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.824+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(47 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.824+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:48 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372e060 con 0x7f8fa4013930 2024-04-25T11:35:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.825+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(48 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.825+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:49 12.10 12:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.826+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(49 603.00000000 [read 0~0] v0'0 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.827+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:50 12.10 12:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.828+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(50 603.00000000 [getxattrs out=124b] v0'0 uv8 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.828+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:51 12.10 12:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x7f8fa4013930 2024-04-25T11:35:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.829+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(51 603.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.829+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:52 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372e060 con 0x7f8fa4013930 2024-04-25T11:35:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.830+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(52 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.830+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:53 12.14 12:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d050 con 0x560e2367a440 2024-04-25T11:35:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.831+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(53 1.00000000.inode [read 0~582 out=582b] v0'0 uv15 ondisk = 0) ==== 160+0+582 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.831+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:54 12.14 12:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d3f0 con 0x560e2367a440 2024-04-25T11:35:28.820 INFO:teuthology.orchestra.run.smithi019.stdout:���������?*f��*����`security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0 ���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3 ���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)��?*fï—º#�C@@ ���������?*fï—º#(�?*fï—º#8`�?*f��-8�?*f�n �?*f�n ����NH�?*f�n ������������ �� 2024-04-25T11:35:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.832+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(54 1.00000000.inode [getxattrs out=4b] v0'0 uv15 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.832+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:55 12.14 12:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372d790 con 0x560e2367a440 2024-04-25T11:35:28.821 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.833+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(55 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.833+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:56 12.14 12:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372eb40 con 0x560e2367a440 2024-04-25T11:35:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.834+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(56 1.00000000.inode [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.834+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:57 12.14 12:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372db30 con 0x560e2367a440 2024-04-25T11:35:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.835+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(57 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.836+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:58 12.14 12:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.824 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@�>@�>@�F@@@ ���� �� 2024-04-25T11:35:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.836+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(58 200.00000000 [getxattrs out=4b] v0'0 uv19 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.837+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:59 12.14 12:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.825 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.837+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(59 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.837+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:60 12.14 12:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.838+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(60 200.00000000 [omap-get-vals out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.838+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:61 12.14 12:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.828 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ���F�F�F�F��XS&YA1e@��XS&YA1h^T�?*f�QG2 ((88 2 ( 2024-04-25T11:35:28.828 INFO:teuthology.orchestra.run.smithi019.stdout:( 2024-04-25T11:35:28.828 INFO:teuthology.orchestra.run.smithi019.stdout:8 2024-04-25T11:35:28.828 INFO:teuthology.orchestra.run.smithi019.stdout:�?*f=8 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:�?*f= ��������@0@��XS&YA1�� 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:��?*f�=�͸��� entity_id0hostname smithi019kernel_version6.9.0-rc3-g806ec36b2112root/��7 ����@��XS&YA1����?*f�6n���������@��XS&YA1�?*f��=setattr� C����������?*f�: �C@@ ���������?*f�n (8�?*f�: 8�?*f�n �?*f�n ����NH�?*f�n ��������=��������Z@��XS&YA1���?*f� �#mkdir�p (�?*fï—º#(�?*fï—º#8�?*fï—º#8�?*fï—º#J 8subdir����������?*fï—º#�A�$�����������������?*fï—º#�?*fï—º#(88���������?*fï—º#����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������2 ((88  C����������?*fï—º#�C@@ ���������?*fï—º#(�?*fï—º#8�?*fï—º#8�?*f�n �?*f�n ����NH�?*f�n ���������==���������@��XS&YA1��?*f��*openc�p (�?*fï—º#(�?*fï—º#8�?*f��*8�?*fï—º#J 8subdir����������?*f��*�A�$�����������������?*f��*�?*fï—º#(�?*f��*8�?*f��*8�?*f��*���������?*fï—º#����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (�?*f��*(�?*f��*8�?*f��*8�?*f��*m [sixmegs����������?*f��*���$@@ ���������?*f��*�?*f��*=@(88���������?*f��*����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������==!!���������@��XS&YA1�?*f�R, 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:cap update�p (�?*fï—º#(�?*fï—º#8 �?*f���+8�?*fï—º#J 8subdir����������?*f��*�A�$�����������������?*f��*�?*fï—º#(�?*f��*8 �?*f���+8 �?*f���+���������?*fï—º#����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (�?*f��*(�?*f��*8 �?*f���+8 �?*f���+u csixmegs����������?*f���+���$@@ ���������?*f���+�?*f��*=�(8 �?*f���+8 �?*f���+���������?*f��*���� security.selinux%unconfined_u:object_r:unlabeled_t:s0����������������=�@��XS&YA1�?*fܻ�, 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:cap update�p  (�?*fï—º#(�?*fï—º#8�:�?*fN_,8�?*fï—º#J 8subdir����������?*f��*�A�$�����������������?*f��*�?*fï—º#(�?*f��*8�:�?*fN_,8�:�?*fN_,���������?*fï—º#����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� (�?*f��*(�?*f��*8�:�?*fN_,8�:�?*fN_,u csixmegs����������?*fN_,���$@@ �:���������?*fN_,�?*f��*=�(8�:�?*fN_,8�:�?*fN_,���������?*f��*�����:security.selinux%unconfined_u:object_r:unlabeled_t:s0����������������=� @��XS&YA1e 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:Q 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:�?*fu�' 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:cap update 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:p  (�?*fï—º#(�?*fï—º#8`�?*f��-8`�?*f��-J 8subdir�������� 2024-04-25T11:35:28.829 INFO:teuthology.orchestra.run.smithi019.stdout:��?*f��*�A�$�����������������?*f��*�?*fï—º#(�?*f��*8`�?*f��-8`�?*f��- 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout:���������?*fï—º#����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  (�?*f��*(�?*f��*8`�?*f��-8`�?*f��-u csixmegs����������?*f��-���$@@ `���������?*f��-�?*f��*=(8`�?*f��-8`�?*f��-���������?*f��*����`security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� C����������?*fï—º#�C@@ ���������?*fï—º#(�?*fï—º#8`�?*f��-8�?*f�n �?*f�n ����NH�?*f�n ����������������=')@��XS&YA1�� 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout:��?*f��=�͸������3@��XS&YA1* 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout:�?*f���check_inode_max_size� p  (�?*fï—º#(�?*fï—º#8`�?*f��-8`�?*f��-J 8subdir�������� ��?*f��*�A�$�����������������?*f��*�?*fï—º#(�?*f��*8`�?*f��-8`�?*f��- ���������?*fï—º#����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  (�?*f��*(�?*f��*8`�?*f��-8`�?*f��-� =sixmegs�������� 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout:��?*f��-���$@@ `���������?*f��-�?*f��*(8`�?*f��-8`�?*f��- 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout:���������?*f��*����`security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� =sixmegs�������� 2024-04-25T11:35:28.830 INFO:teuthology.orchestra.run.smithi019.stdout:��?*f��-���$@@ `���������?*f��-�?*f��*(8`�?*f��-8`�?*f��- 2024-04-25T11:35:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.840+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(61 200.00000001 [read 0~18117 out=18117b] v0'0 uv16 ondisk = 0) ==== 156+0+18117 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.840+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:62 12.14 12:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.831 INFO:teuthology.orchestra.run.smithi019.stdout:���������?*f��*����`security.selinux%unconfined_u:object_r:unlabeled_t:s0����������������M4@��XS&YA18.$��>@��XS&YA1h^T�?*f�+S82  (�?*fï—º#(�?*fï—º#8`�?*f��-8`�?*f��- 2 ( 2024-04-25T11:35:28.831 INFO:teuthology.orchestra.run.smithi019.stdout:( 2024-04-25T11:35:28.831 INFO:teuthology.orchestra.run.smithi019.stdout:8 2024-04-25T11:35:28.832 INFO:teuthology.orchestra.run.smithi019.stdout:�?*f=8 2024-04-25T11:35:28.832 INFO:teuthology.orchestra.run.smithi019.stdout:�?*f= ��������@�>@��XS&YA1^TJ�?*f*�8scatter_writebehind2 (�?*fï—º#(�?*fï—º#8`�?*f��-8`�?*f��-  C����������?*fï—º#�C@@ ���������?*fï—º#(�?*fï—º#8`�?*f��-8�?*f�n �?*f�n ����NH�?*f�n ����������������SB@���� �� 2024-04-25T11:35:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.841+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(62 200.00000001 [getxattrs out=4b] v0'0 uv16 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.841+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:63 12.14 12:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.832 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.842+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(63 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.842+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:64 12.14 12:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.843+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(64 200.00000001 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.843+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:65 12.14 12:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.844+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(65 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.844+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:66 12.14 12:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.845+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(66 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.845+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:67 12.14 12:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.846+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(67 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.846+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:68 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e2367a440 2024-04-25T11:35:28.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.847+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(68 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8fa4031040 con 0x560e2367a440 2024-04-25T11:35:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.847+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:69 12.1c 12:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa40203d0 2024-04-25T11:35:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.849+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(69 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa40203d0 2024-04-25T11:35:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.849+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:70 12.1c 12:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa40203d0 2024-04-25T11:35:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.850+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(70 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa40203d0 2024-04-25T11:35:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.850+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:71 12.1c 12:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa40203d0 2024-04-25T11:35:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.851+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(71 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa40203d0 2024-04-25T11:35:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.852+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:72 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa40203d0 2024-04-25T11:35:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.853+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(72 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa40203d0 2024-04-25T11:35:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.853+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:73 12.6 12:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa4013930 2024-04-25T11:35:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.854+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(73 mds0_openfiles.0 [read 0~0] v0'0 uv4 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.854+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:74 12.6 12:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa4013930 2024-04-25T11:35:28.843 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6 ���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7 ���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2024-04-25T11:35:28.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.856+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(74 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv4 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.856+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:75 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa4013930 2024-04-25T11:35:28.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.857+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(75 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv4 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.857+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:76 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8fa4013930 2024-04-25T11:35:28.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.858+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(76 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.859+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:77 12.e 12:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8f9c017940 2024-04-25T11:35:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.860+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(77 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.860+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:78 12.e 12:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8f9c017940 2024-04-25T11:35:28.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.861+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(78 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.861+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:79 12.e 12:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8f9c017940 2024-04-25T11:35:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.862+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(79 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.862+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:80 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x7f8f9c017940 2024-04-25T11:35:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.864+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(80 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.864+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:81 12.15 12:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372df20 con 0x560e235fca50 2024-04-25T11:35:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.865+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(81 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.865+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:82 12.15 12:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372e310 con 0x560e235fca50 2024-04-25T11:35:28.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.866+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(82 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.866+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:83 12.15 12:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372e730 con 0x560e235fca50 2024-04-25T11:35:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.867+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(83 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.867+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:84 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fef0 con 0x560e235fca50 2024-04-25T11:35:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.868+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(84 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.868+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:85 12.15 12:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372eee0 con 0x560e235fca50 2024-04-25T11:35:28.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.870+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(85 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.870+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:86 12.15 12:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372f2d0 con 0x560e235fca50 2024-04-25T11:35:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.871+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(86 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.859 INFO:teuthology.orchestra.run.smithi019.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8 ���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4 ���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@ ���� �� 2024-04-25T11:35:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.871+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:87 12.15 12:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372f6f0 con 0x560e235fca50 2024-04-25T11:35:28.860 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.872+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(87 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.872+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:88 12.15 12:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x560e235fca50 2024-04-25T11:35:28.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.873+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(88 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8fac06e070 con 0x560e235fca50 2024-04-25T11:35:28.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.873+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:89 12.d 12:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.874+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(89 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8f980022f0 2024-04-25T11:35:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.874+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:90 12.d 12:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.863 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2024-04-25T11:35:28.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.875+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(90 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8f980022f0 2024-04-25T11:35:28.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.875+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:91 12.d 12:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.864 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.876+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(91 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8f980022f0 2024-04-25T11:35:28.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.876+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:92 12.d 12:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.877+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(92 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8f980022f0 2024-04-25T11:35:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.877+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:93 12.1d 12:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa4013930 2024-04-25T11:35:28.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.879+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(93 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.879+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:94 12.1d 12:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa4013930 2024-04-25T11:35:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.880+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(94 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.880+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:95 12.1d 12:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa4013930 2024-04-25T11:35:28.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.881+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(95 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8f9c049090 con 0x7f8fa4013930 2024-04-25T11:35:28.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.882+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:96 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa4013930 2024-04-25T11:35:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.883+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(96 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa4013930 2024-04-25T11:35:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.883+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa40203d0 2024-04-25T11:35:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.884+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.885+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa40203d0 2024-04-25T11:35:28.874 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5 ���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2024-04-25T11:35:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.886+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(98 mds0_inotable [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.886+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8fa40203d0 2024-04-25T11:35:28.875 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.887+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(99 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.887+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e237312a0 con 0x7f8fa40203d0 2024-04-25T11:35:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.888+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(100 mds0_inotable [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.888+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:101 12.13 12:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8fa40203d0 2024-04-25T11:35:28.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.889+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(101 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.890+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:102 12.13 12:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8fa40203d0 2024-04-25T11:35:28.879 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2024-04-25T11:35:28.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.891+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(102 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.891+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:103 12.13 12:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa40203d0 2024-04-25T11:35:28.880 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ( 2024-04-25T11:35:28.880 INFO:teuthology.orchestra.run.smithi019.stdout:( 2024-04-25T11:35:28.880 INFO:teuthology.orchestra.run.smithi019.stdout:8 2024-04-25T11:35:28.880 INFO:teuthology.orchestra.run.smithi019.stdout:�?*f=8 2024-04-25T11:35:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.892+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(103 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.892+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:104 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8fa40203d0 2024-04-25T11:35:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.894+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(104 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8fa40203d0 2024-04-25T11:35:28.882 INFO:teuthology.orchestra.run.smithi019.stdout:�?*f=���� ��PJ 2024-04-25T11:35:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.894+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.b 12:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8f9c017940 2024-04-25T11:35:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.895+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(105 mds0_sessionmap [read 0~0] v0'0 uv3 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.895+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.b 12:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8f9c017940 2024-04-25T11:35:28.884 INFO:teuthology.orchestra.run.smithi019.stdout: stray0_head�i����?*f@i@A�����������������?*f@i(8�?*f@i8�?*f@i�?*f@i������������ stray1_head�i����?*f��@A�����������������?*f��(8�?*f��8�?*f���?*f�������������� stray2_head�i����?*f=@A�����������������?*f=(8�?*f=8�?*f=�?*f=������������ stray3_head�i����?*f�@A�����������������?*f�(8�?*f�8�?*f��?*f������������� stray4_head�i����?*f�!@A�����������������?*f�!(8�?*f�!8�?*f�!�?*f�!������������ stray5_head�i����?*fe�-@A�����������������?*fe�-(8�?*fe�-8�?*fe�-�?*fe�-������������ stray6_head�i����?*fHo1@A�����������������?*fHo1(8�?*fHo18�?*fHo1�?*fHo1������������ stray7_head�i����?*fx�5@A�����������������?*fx�5(8�?*fx�58�?*fx�5�?*fx�5������������ stray8_head�i����?*f �:@A�����������������?*f �:(8�?*f �:8�?*f �:�?*f �:������������ stray9_head�i��� �?*f=@A�����������������?*f=(8�?*f=8�?*f=�?*f=���������������� �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2024-04-25T11:35:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.896+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(106 mds0_sessionmap [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 159+0+4 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.897+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8f9c017940 2024-04-25T11:35:28.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.898+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(107 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.898+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e237312a0 con 0x7f8f9c017940 2024-04-25T11:35:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.899+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(108 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.899+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8f9c017940 2024-04-25T11:35:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.900+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(109 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.900+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8f9c017940 2024-04-25T11:35:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.902+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 17 ==== osd_op_reply(110 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.902+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:111 12.7 12:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e23730290 con 0x7f8f9c017940 2024-04-25T11:35:28.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.903+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 18 ==== osd_op_reply(111 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.903+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:112 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f9c017940 2024-04-25T11:35:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.904+0000 7f8fb2000640 1 -- 172.21.15.19:0/2614629688 <== osd.7 v2:172.21.15.169:6800/2355270624 19 ==== osd_op_reply(112 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8fa4031040 con 0x7f8f9c017940 2024-04-25T11:35:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.905+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.906+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.906+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.895 INFO:teuthology.orchestra.run.smithi019.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9 ���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2024-04-25T11:35:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.907+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.907+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.896 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.908+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.908+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.f 12:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.909+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.909+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.898 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)��?*fAe@A�����������������?*fAe( 2024-04-25T11:35:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.910+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [read 0~582 out=582b] v0'0 uv3 ondisk = 0) ==== 162+0+582 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.910+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.899 INFO:teuthology.orchestra.run.smithi019.stdout:8 �?*f=8 �?*f=�?*fAe����NH������������ �� 2024-04-25T11:35:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.911+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 162+0+4 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.912+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.900 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� 2024-04-25T11:35:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.912+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.913+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.901 INFO:teuthology.orchestra.run.smithi019.stdout:���� �� ��("+ 2024-04-25T11:35:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.913+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.914+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.914+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.915+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.915+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [getxattrs out=92b] v0'0 uv6 ondisk = 0) ==== 154+0+92 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.916+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.916+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.916+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x560e2372fae0 con 0x7f8f980022f0 2024-04-25T11:35:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fb2a00640 1 -- 172.21.15.19:0/2614629688 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [omap-get-vals out=585b] v0'0 uv6 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x7f8f9c05e020 con 0x7f8f980022f0 2024-04-25T11:35:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560e2367a440 msgr2=0x560e2367a850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560e2367a440 0x560e2367a850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560e235fca50 msgr2=0x7f8fac0d6cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560e235fca50 0x7f8fac0d6cd0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f8fa4021950 msgr2=0x7f8fa4023400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f8fa4021950 0x7f8fa4023400 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f8f980022f0 msgr2=0x7f8fac0d50c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f8f980022f0 0x7f8fac0d50c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x560e235b4a50 msgr2=0x7f8fac0a01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x560e235b4a50 0x7f8fac0a01b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7f8fa40203d0 msgr2=0x7f8fa40207e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7f8fa40203d0 0x7f8fa40207e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f8fa4013930 msgr2=0x7f8fa40123c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7f8fa4013930 0x7f8fa40123c0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f8f9c017940 msgr2=0x7f8f9c0073f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f8f9c017940 0x7f8f9c0073f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 reap_dead start 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.918+0000 7f8fb3400640 1 -- 172.21.15.19:0/2614629688 reap_dead start 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f90072ec0 msgr2=0x7f8f900753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f90072ec0 0x7f8f900753d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c005fd0 tx=0x7f8f9c0058c0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f98003c40 msgr2=0x560e236853c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f98003c40 0x560e236853c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0258a0 tx=0x7f8fac001120 comp rx=0 tx=0).stop 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 shutdown_connections 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f90072ec0 0x7f8f900753d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 --2- 172.21.15.19:0/2614629688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f98003c40 0x560e236853c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.919+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 >> 172.21.15.19:0/2614629688 conn(0x560e234d5ac0 msgr2=0x560e235b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.920+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 shutdown_connections 2024-04-25T11:35:28.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:28.920+0000 7f8fba40bf40 1 -- 172.21.15.19:0/2614629688 wait complete. 2024-04-25T11:35:28.929 INFO:teuthology.orchestra.run.smithi019.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@ _parent ���� ��  (�?*fï—º#(�?*fï—º#8`�?*f��-8`�?*f��-���� ��NH subdir_head1i"��?*f��*�A�$�����������������?*f��*�?*fï—º#(�?*f��*8`�?*f��-8`�?*f��- ���������?*fï—º#����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� �� 2024-04-25T11:35:28.930 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2024-04-25T11:35:29.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.129+0000 7fbb01083f40 1 Processor -- start 2024-04-25T11:35:29.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.129+0000 7fbb01083f40 1 -- start start 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.130+0000 7fbb01083f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbae00030f0 0x559bbbb8df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.130+0000 7fbb01083f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559bbbaed8d0 0x559bbbb924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.130+0000 7fbb01083f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbb90620 0x559bbbb90a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.130+0000 7fbb01083f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bbba65d20 con 0x559bbbb90620 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.130+0000 7fbb01083f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559bbba69d60 con 0x7fbae00030f0 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.130+0000 7fbb01083f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559bbbb08100 con 0x559bbbaed8d0 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbb90620 0x559bbbb90a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbb90620 0x559bbbb90a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41256/0 (socket says 172.21.15.19:41256) 2024-04-25T11:35:29.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 -- 172.21.15.19:0/756794673 learned_addr learned my addr 172.21.15.19:0/756794673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbaf9800640 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559bbbaed8d0 0x559bbbb924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafa200640 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbae00030f0 0x559bbbb8df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 -- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559bbbaed8d0 msgr2=0x559bbbb924f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559bbbaed8d0 0x559bbbb924f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 -- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbae00030f0 msgr2=0x559bbbb8df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbae00030f0 0x559bbbb8df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.131+0000 7fbafac00640 1 -- 172.21.15.19:0/756794673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbbaf0d70 con 0x559bbbb90620 2024-04-25T11:35:29.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.132+0000 7fbafac00640 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbb90620 0x559bbbb90a30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4001050 tx=0x7fbaf400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b6b91ae78abb24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.132+0000 7fbaf3e00640 1 -- 172.21.15.19:0/756794673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4008070 con 0x559bbbb90620 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.132+0000 7fbaf3e00640 1 -- 172.21.15.19:0/756794673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf400be60 con 0x559bbbb90620 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.132+0000 7fbaf3e00640 1 -- 172.21.15.19:0/756794673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400d740 con 0x559bbbb90620 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 -- 172.21.15.19:0/756794673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbb90620 msgr2=0x559bbbb90a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbb90620 0x559bbbb90a30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4001050 tx=0x7fbaf400b720 comp rx=0 tx=0).stop 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 -- 172.21.15.19:0/756794673 shutdown_connections 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbb90620 0x559bbbb90a30 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559bbbaed8d0 0x559bbbb924f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 --2- 172.21.15.19:0/756794673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbae00030f0 0x559bbbb8df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 -- 172.21.15.19:0/756794673 >> 172.21.15.19:0/756794673 conn(0x559bbb9e1ac0 msgr2=0x559bbbaef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 -- 172.21.15.19:0/756794673 shutdown_connections 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.133+0000 7fbb01083f40 1 -- 172.21.15.19:0/756794673 wait complete. 2024-04-25T11:35:29.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.134+0000 7fbb01083f40 1 Processor -- start 2024-04-25T11:35:29.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.134+0000 7fbb01083f40 1 -- start start 2024-04-25T11:35:29.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.135+0000 7fbb01083f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbaed8d0 0x559bbbb92460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.135+0000 7fbb01083f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bbbb90620 0x559bbbb8a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.135+0000 7fbb01083f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbae00030f0 0x559bbbaef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.135+0000 7fbb01083f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bbbb0c960 con 0x559bbbaed8d0 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.135+0000 7fbb01083f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559bbbaf1050 con 0x559bbbb90620 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.135+0000 7fbb01083f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559bbbb0cc20 con 0x7fbae00030f0 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbaed8d0 0x559bbbb92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbaed8d0 0x559bbbb92460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41264/0 (socket says 172.21.15.19:41264) 2024-04-25T11:35:29.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 learned_addr learned my addr 172.21.15.19:0/3040818097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbae00030f0 msgr2=0x559bbbaef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbae00030f0 0x559bbbaef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bbbb90620 msgr2=0x559bbbb8a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559bbbb90620 0x559bbbb8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbbb94130 con 0x559bbbaed8d0 2024-04-25T11:35:29.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.136+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbaed8d0 0x559bbbb92460 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d440 tx=0x7fbaec00d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.137+0000 7fbaf2a00640 1 -- 172.21.15.19:0/3040818097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011590 con 0x559bbbaed8d0 2024-04-25T11:35:29.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.137+0000 7fbaf2a00640 1 -- 172.21.15.19:0/3040818097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaec017070 con 0x559bbbaed8d0 2024-04-25T11:35:29.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.137+0000 7fbaf2a00640 1 -- 172.21.15.19:0/3040818097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec011970 con 0x559bbbaed8d0 2024-04-25T11:35:29.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.137+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559bbbb94410 con 0x559bbbaed8d0 2024-04-25T11:35:29.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.137+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbbb949d0 con 0x559bbbaed8d0 2024-04-25T11:35:29.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.139+0000 7fbaf2a00640 1 -- 172.21.15.19:0/3040818097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbaec011b10 con 0x559bbbaed8d0 2024-04-25T11:35:29.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.140+0000 7fbaf2a00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbadc072e90 0x7fbadc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.140+0000 7fbaf2a00640 1 -- 172.21.15.19:0/3040818097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fbaec093f10 con 0x559bbbaed8d0 2024-04-25T11:35:29.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.140+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559bbbc2cb40 0x559bbbc2cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.140+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x559bbbc2d490 con 0x559bbbc2cb40 2024-04-25T11:35:29.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.141+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbadc072e90 0x7fbadc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.141+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559bbbc2cb40 0x559bbbc2cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.141+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559bbbc2cb40 0x559bbbc2cf50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.142+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x559bbbc2cb40 2024-04-25T11:35:29.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.142+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x559bbbabf1d0 0x7fbaf400fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.142+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x559bbbc2d490 con 0x559bbbabf1d0 2024-04-25T11:35:29.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.142+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbadc072e90 0x7fbadc0753a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009810 tx=0x7fbae8051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.143+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x559bbbabf1d0 0x7fbaf400fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.144+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x559bbbabf1d0 0x7fbaf400fa70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.146+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=177b] v92'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7fbaec05a250 con 0x559bbbabf1d0 2024-04-25T11:35:29.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.146+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7fbaec05e220 0x7fbaec05cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.146+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x559bbbc2d490 con 0x7fbaec05e220 2024-04-25T11:35:29.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.146+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7fbaec05e220 0x7fbaec05cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.148+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7fbaec05e220 0x7fbaec05cc90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.148+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae8061030 con 0x7fbaec05e220 2024-04-25T11:35:29.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.148+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x559bbbc2d490 con 0x7fbaec05e220 2024-04-25T11:35:29.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.149+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae8061030 con 0x7fbaec05e220 2024-04-25T11:35:29.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.149+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7fbae8022e30 0x7fbae8066940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.149+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x559bbbc2d490 con 0x7fbae8022e30 2024-04-25T11:35:29.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.150+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7fbae8022e30 0x7fbae8066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.151+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7fbae8022e30 0x7fbae8066940 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.151+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.2 v2:172.21.15.19:6813/2392070031 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x7fbae8022e30 2024-04-25T11:35:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.151+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x559bbbc2d490 con 0x559bbbc2cb40 2024-04-25T11:35:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.151+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(6 [pgnls start_epoch 93 out=149b] v92'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7fbaf400d800 con 0x559bbbc2cb40 2024-04-25T11:35:29.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.151+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x559bbbc2d490 con 0x7fbaec05e220 2024-04-25T11:35:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.152+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae8061030 con 0x7fbaec05e220 2024-04-25T11:35:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.152+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fbae801f7b0 0x7fbae8058910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.152+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae801f7b0 2024-04-25T11:35:29.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.153+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fbae801f7b0 0x7fbae8058910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.155+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fbae801f7b0 0x7fbae8058910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.156+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(8 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fbaec05a250 con 0x7fbae801f7b0 2024-04-25T11:35:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.156+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae8022e30 2024-04-25T11:35:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.156+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.2 v2:172.21.15.19:6813/2392070031 2 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x7fbae8022e30 2024-04-25T11:35:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.156+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x559bbbabf5e0 0x7fbaf401e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.156+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbabf5e0 2024-04-25T11:35:29.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.157+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x559bbbabf5e0 0x7fbaf401e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.159+0000 7fbaf9800640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x559bbbabf5e0 0x7fbaf401e450 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.159+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.4 v2:172.21.15.169:6816/3656468664 1 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae8061030 con 0x559bbbabf5e0 2024-04-25T11:35:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.160+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbabf5e0 2024-04-25T11:35:29.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.160+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.4 v2:172.21.15.169:6816/3656468664 2 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae8061030 con 0x559bbbabf5e0 2024-04-25T11:35:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.161+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae8022e30 2024-04-25T11:35:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.161+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.2 v2:172.21.15.19:6813/2392070031 3 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x7fbae8022e30 2024-04-25T11:35:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.161+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbabf1d0 2024-04-25T11:35:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.163+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(13 [pgnls start_epoch 93 out=77b] v92'4 uv4 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7fbaec05a250 con 0x559bbbabf1d0 2024-04-25T11:35:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.163+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae8022e30 2024-04-25T11:35:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.164+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.2 v2:172.21.15.19:6813/2392070031 4 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x7fbae8022e30 2024-04-25T11:35:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.164+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbaec05e220 2024-04-25T11:35:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.165+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(15 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fbae8061030 con 0x7fbaec05e220 2024-04-25T11:35:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.165+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae8022e30 2024-04-25T11:35:29.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.166+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.2 v2:172.21.15.19:6813/2392070031 5 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x7fbae8022e30 2024-04-25T11:35:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.166+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559bbbb98dc0 0x7fbaf4031430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.166+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbb98dc0 2024-04-25T11:35:29.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.166+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559bbbb98dc0 0x7fbaf4031430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.167+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559bbbb98dc0 0x7fbaf4031430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.168+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x559bbbb98dc0 2024-04-25T11:35:29.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.168+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae8022e30 2024-04-25T11:35:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.168+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.2 v2:172.21.15.19:6813/2392070031 6 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x7fbae8022e30 2024-04-25T11:35:29.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.168+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbabf1d0 2024-04-25T11:35:29.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.169+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaec05a250 con 0x559bbbabf1d0 2024-04-25T11:35:29.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.170+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbaec05e220 2024-04-25T11:35:29.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.171+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae8061030 con 0x7fbaec05e220 2024-04-25T11:35:29.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.171+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbc2cb40 2024-04-25T11:35:29.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.171+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x559bbbc2cb40 2024-04-25T11:35:29.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.171+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbb98dc0 2024-04-25T11:35:29.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.172+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(22 [pgnls start_epoch 93 out=97b] v91'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7fbaf400d800 con 0x559bbbb98dc0 2024-04-25T11:35:29.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.172+0000 7fbafac00640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7fbae00022d0 0x7fbaf40423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.172+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae00022d0 2024-04-25T11:35:29.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.172+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7fbae00022d0 0x7fbaf40423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.173+0000 7fbafa200640 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7fbae00022d0 0x7fbaf40423d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.174+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(23 [pgnls start_epoch 93 out=74b] v91'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbaec05a250 con 0x7fbae00022d0 2024-04-25T11:35:29.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.174+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbabf1d0 2024-04-25T11:35:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.175+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(24 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fbaec05a250 con 0x559bbbabf1d0 2024-04-25T11:35:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.175+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbabf5e0 2024-04-25T11:35:29.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.176+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.4 v2:172.21.15.169:6816/3656468664 3 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae8061030 con 0x559bbbabf5e0 2024-04-25T11:35:29.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.176+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae801f7b0 2024-04-25T11:35:29.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.178+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(26 [pgnls start_epoch 93 out=98b] v92'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7fbaec05a250 con 0x7fbae801f7b0 2024-04-25T11:35:29.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.178+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbaec05e220 2024-04-25T11:35:29.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.179+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(27 [pgnls start_epoch 93 out=76b] v92'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7fbae8061030 con 0x7fbaec05e220 2024-04-25T11:35:29.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.179+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbc2cb40 2024-04-25T11:35:29.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.180+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v91'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x559bbbc2cb40 2024-04-25T11:35:29.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.180+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbaec05e220 2024-04-25T11:35:29.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.181+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v91'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fbae8061030 con 0x7fbaec05e220 2024-04-25T11:35:29.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.181+0000 7fbaf9800640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x559bbbb98dc0 2024-04-25T11:35:29.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.182+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbaf400d800 con 0x559bbbb98dc0 2024-04-25T11:35:29.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.182+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae00022d0 2024-04-25T11:35:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.182+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(31 [pgnls start_epoch 93 out=73b] v91'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fbaec05a250 con 0x7fbae00022d0 2024-04-25T11:35:29.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.182+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) -- 0x7fbae8073040 con 0x7fbae00022d0 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:601.00000000 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:602.00000000 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:10000000000.00000000 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:600.00000000 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:603.00000000 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:1.00000000.inode 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:200.00000000 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:200.00000001 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:606.00000000 2024-04-25T11:35:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:607.00000000 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:mds0_openfiles.0 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:608.00000000 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:604.00000000 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:500.00000000 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:mds_snaptable 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:605.00000000 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:mds0_inotable 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:100.00000000 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:mds0_sessionmap 2024-04-25T11:35:29.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbafa200640 1 -- 172.21.15.19:0/3040818097 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(32 [pgnls start_epoch 93 out=101b] v92'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7fbaec05a250 con 0x7fbae00022d0 2024-04-25T11:35:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:609.00000000 2024-04-25T11:35:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:400.00000000 2024-04-25T11:35:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:100.00000000.inode 2024-04-25T11:35:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:1.00000000 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559bbbc2cb40 msgr2=0x559bbbc2cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559bbbc2cb40 0x559bbbc2cf50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559bbbb98dc0 msgr2=0x7fbaf4031430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559bbbb98dc0 0x7fbaf4031430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7fbae8022e30 msgr2=0x7fbae8066940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7fbae8022e30 0x7fbae8066940 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7fbae00022d0 msgr2=0x7fbaf40423d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7fbae00022d0 0x7fbaf40423d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x559bbbabf5e0 msgr2=0x7fbaf401e450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x559bbbabf5e0 0x7fbaf401e450 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fbae801f7b0 msgr2=0x7fbae8058910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fbae801f7b0 0x7fbae8058910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x559bbbabf1d0 msgr2=0x7fbaf400fa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x559bbbabf1d0 0x7fbaf400fa70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7fbaec05e220 msgr2=0x7fbaec05cc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7fbaec05e220 0x7fbaec05cc90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.183+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 reap_dead start 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbadc072e90 msgr2=0x7fbadc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbadc072e90 0x7fbadc0753a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009810 tx=0x7fbae8051000 comp rx=0 tx=0).stop 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbaed8d0 msgr2=0x559bbbb92460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 --2- 172.21.15.19:0/3040818097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bbbaed8d0 0x559bbbb92460 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d440 tx=0x7fbaec00d900 comp rx=0 tx=0).stop 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 reap_dead start 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbafac00640 1 -- 172.21.15.19:0/3040818097 reap_dead start 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 shutdown_connections 2024-04-25T11:35:29.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 >> 172.21.15.19:0/3040818097 conn(0x559bbb9e1ac0 msgr2=0x559bbbb95530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 shutdown_connections 2024-04-25T11:35:29.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.184+0000 7fbb01083f40 1 -- 172.21.15.19:0/3040818097 wait complete. 2024-04-25T11:35:29.191 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2024-04-25T11:35:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.375+0000 7f7faf708f40 1 Processor -- start 2024-04-25T11:35:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.375+0000 7f7faf708f40 1 -- start start 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7faf708f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd200a1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7faf708f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 0x55fd20059560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7faf708f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2012ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7faf708f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fd200a3e70 con 0x55fd2005b1e0 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7faf708f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fd20001d20 con 0x55fd20134dd0 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7faf708f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fd200a4110 con 0x55fd20059aa0 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7fa8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd200a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7fa8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd200a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41270/0 (socket says 172.21.15.19:41270) 2024-04-25T11:35:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7fa8800640 1 -- 172.21.15.19:0/1950091389 learned_addr learned my addr 172.21.15.19:0/1950091389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7fa7e00640 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 0x55fd20059560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.377+0000 7f7fa9200640 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2012ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa8800640 1 -- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 msgr2=0x55fd2012ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa8800640 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2012ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa8800640 1 -- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 msgr2=0x55fd20059560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa8800640 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 0x55fd20059560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa8800640 1 -- 172.21.15.19:0/1950091389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd20059eb0 con 0x55fd2005b1e0 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa8800640 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd200a1910 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7f98001610 tx=0x7f7f98001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a873a433dfaa88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa6a00640 1 -- 172.21.15.19:0/1950091389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98004520 con 0x55fd2005b1e0 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa6a00640 1 -- 172.21.15.19:0/1950091389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f980046e0 con 0x55fd2005b1e0 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.378+0000 7f7fa6a00640 1 -- 172.21.15.19:0/1950091389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98004ad0 con 0x55fd2005b1e0 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 -- 172.21.15.19:0/1950091389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 msgr2=0x55fd200a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd200a1910 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7f98001610 tx=0x7f7f98001ad0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 -- 172.21.15.19:0/1950091389 shutdown_connections 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 0x55fd20059560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2012ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 --2- 172.21.15.19:0/1950091389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd200a1910 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 -- 172.21.15.19:0/1950091389 >> 172.21.15.19:0/1950091389 conn(0x55fd1ff7dac0 msgr2=0x55fd20133dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.379+0000 7f7faf708f40 1 -- 172.21.15.19:0/1950091389 shutdown_connections 2024-04-25T11:35:29.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.380+0000 7f7faf708f40 1 -- 172.21.15.19:0/1950091389 wait complete. 2024-04-25T11:35:29.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.381+0000 7f7faf708f40 1 Processor -- start 2024-04-25T11:35:29.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.381+0000 7f7faf708f40 1 -- start start 2024-04-25T11:35:29.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.382+0000 7f7faf708f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2009d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.382+0000 7f7faf708f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd20117500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.382+0000 7f7faf708f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 0x55fd20117a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.382+0000 7f7faf708f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fd200a1350 con 0x55fd2005b1e0 2024-04-25T11:35:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.382+0000 7f7faf708f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fd20129710 con 0x55fd20134dd0 2024-04-25T11:35:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.382+0000 7f7faf708f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fd2009e780 con 0x55fd20059aa0 2024-04-25T11:35:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.382+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd20117500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd20117500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41276/0 (socket says 172.21.15.19:41276) 2024-04-25T11:35:29.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 -- 172.21.15.19:0/12600631 learned_addr learned my addr 172.21.15.19:0/12600631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa8800640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2009d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 -- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 msgr2=0x55fd2009d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2009d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 -- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 msgr2=0x55fd20117a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa9200640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 0x55fd20117a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fd20134dd0 0x55fd20117a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 -- 172.21.15.19:0/12600631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2009e190 con 0x55fd2005b1e0 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa8800640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fd20059aa0 0x55fd2009d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.383+0000 7f7fa7e00640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd20117500 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7f90025b20 tx=0x7f7f90002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.384+0000 7f7fa5600640 1 -- 172.21.15.19:0/12600631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90063030 con 0x55fd2005b1e0 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.384+0000 7f7fa5600640 1 -- 172.21.15.19:0/12600631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f900547f0 con 0x55fd2005b1e0 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.384+0000 7f7fa5600640 1 -- 172.21.15.19:0/12600631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90059040 con 0x55fd2005b1e0 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.384+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fd20089b50 con 0x55fd2005b1e0 2024-04-25T11:35:29.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.384+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd200a1090 con 0x55fd2005b1e0 2024-04-25T11:35:29.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.386+0000 7f7fa5600640 1 -- 172.21.15.19:0/12600631 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f90066030 con 0x55fd2005b1e0 2024-04-25T11:35:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.387+0000 7f7fa5600640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f84072e90 0x7f7f840753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.387+0000 7f7fa5600640 1 -- 172.21.15.19:0/12600631 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f7f900d74d0 con 0x55fd2005b1e0 2024-04-25T11:35:29.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.388+0000 7f7faf708f40 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fd20117f80 0x55fd201cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.388+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55fd2009d8b0 con 0x55fd20117f80 2024-04-25T11:35:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.388+0000 7f7fa8800640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f84072e90 0x7f7f840753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.388+0000 7f7fa9200640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fd20117f80 0x55fd201cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.389+0000 7f7fa8800640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f84072e90 0x7f7f840753a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7f98001610 tx=0x7f7f98008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.390+0000 7f7fa9200640 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fd20117f80 0x55fd201cbdc0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7fa9200640 1 -- 172.21.15.19:0/12600631 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f7fa0009910 con 0x55fd20117f80 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:header (274 bytes) : 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000000 04 03 0c 01 00 00 0b 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000020 00 00 00 00 fb 3f 2a 66 d0 13 86 2a 01 00 00 00 |.....?*f...*....| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000050 00 00 fb 3f 2a 66 d0 13 86 2a 01 00 00 00 00 00 |...?*f...*......| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:000000a0 00 00 00 00 00 00 00 00 fb 3f 2a 66 9c f0 06 2d |.........?*f...-| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-25T11:35:29.379 INFO:teuthology.orchestra.run.smithi019.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2024-04-25T11:35:29.380 INFO:teuthology.orchestra.run.smithi019.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.380 INFO:teuthology.orchestra.run.smithi019.stdout:000000e0 00 00 00 00 00 00 fb 3f 2a 66 9c f0 06 2d 00 00 |.......?*f...-..| 2024-04-25T11:35:29.380 INFO:teuthology.orchestra.run.smithi019.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.380 INFO:teuthology.orchestra.run.smithi019.stdout:* 2024-04-25T11:35:29.380 INFO:teuthology.orchestra.run.smithi019.stdout:00000110 00 00 |..| 2024-04-25T11:35:29.380 INFO:teuthology.orchestra.run.smithi019.stdout:00000112 2024-04-25T11:35:29.380 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fd20117f80 msgr2=0x55fd201cbdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fd20117f80 0x55fd201cbdc0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f84072e90 msgr2=0x7f7f840753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f84072e90 0x7f7f840753a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7f98001610 tx=0x7f7f98008000 comp rx=0 tx=0).stop 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 msgr2=0x55fd20117500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 --2- 172.21.15.19:0/12600631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd2005b1e0 0x55fd20117500 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7f90025b20 tx=0x7f7f90002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7fa9200640 1 -- 172.21.15.19:0/12600631 reap_dead start 2024-04-25T11:35:29.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 shutdown_connections 2024-04-25T11:35:29.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 >> 172.21.15.19:0/12600631 conn(0x55fd1ff7dac0 msgr2=0x55fd200a1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 shutdown_connections 2024-04-25T11:35:29.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.391+0000 7f7faf708f40 1 -- 172.21.15.19:0/12600631 wait complete. 2024-04-25T11:35:29.394 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2024-04-25T11:35:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.587+0000 7f36a35fef40 1 Processor -- start 2024-04-25T11:35:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.587+0000 7f36a35fef40 1 -- start start 2024-04-25T11:35:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f36a35fef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3680003bf0 0x5635f05aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f36a35fef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5635f0521fe0 0x5635f0508280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f36a35fef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f051df60 0x5635f05074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f36a35fef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5635f047dd20 con 0x5635f051df60 2024-04-25T11:35:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f36a35fef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5635f0481d60 con 0x5635f0521fe0 2024-04-25T11:35:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f36a35fef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5635f05203f0 con 0x7f3680003bf0 2024-04-25T11:35:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f369d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f051df60 0x5635f05074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f369d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f051df60 0x5635f05074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41298/0 (socket says 172.21.15.19:41298) 2024-04-25T11:35:29.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.589+0000 7f369d200640 1 -- 172.21.15.19:0/3919878241 learned_addr learned my addr 172.21.15.19:0/3919878241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369c800640 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3680003bf0 0x5635f05aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369be00640 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5635f0521fe0 0x5635f0508280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369d200640 1 -- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3680003bf0 msgr2=0x5635f05aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369d200640 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3680003bf0 0x5635f05aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369d200640 1 -- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5635f0521fe0 msgr2=0x5635f0508280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369d200640 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5635f0521fe0 0x5635f0508280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369d200640 1 -- 172.21.15.19:0/3919878241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f05acf30 con 0x5635f051df60 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369d200640 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f051df60 0x5635f05074a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3694046aa0 tx=0x7f3694050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0bcc1b18dd2e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369aa00640 1 -- 172.21.15.19:0/3919878241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694066070 con 0x5635f051df60 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369aa00640 1 -- 172.21.15.19:0/3919878241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3694061450 con 0x5635f051df60 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.590+0000 7f369aa00640 1 -- 172.21.15.19:0/3919878241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694001670 con 0x5635f051df60 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 -- 172.21.15.19:0/3919878241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f051df60 msgr2=0x5635f05074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f051df60 0x5635f05074a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f3694046aa0 tx=0x7f3694050000 comp rx=0 tx=0).stop 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 -- 172.21.15.19:0/3919878241 shutdown_connections 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5635f0521fe0 0x5635f0508280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f051df60 0x5635f05074a0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 --2- 172.21.15.19:0/3919878241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3680003bf0 0x5635f05aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 -- 172.21.15.19:0/3919878241 >> 172.21.15.19:0/3919878241 conn(0x5635f03f9ac0 msgr2=0x5635f0509190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 -- 172.21.15.19:0/3919878241 shutdown_connections 2024-04-25T11:35:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.591+0000 7f36a35fef40 1 -- 172.21.15.19:0/3919878241 wait complete. 2024-04-25T11:35:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.592+0000 7f36a35fef40 1 Processor -- start 2024-04-25T11:35:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.593+0000 7f36a35fef40 1 -- start start 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f36a35fef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5635f051df60 0x5635f05a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f36a35fef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3680003bf0 0x5635f0508000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f36a35fef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f0523b10 0x5635f0508540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f36a35fef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5635f05ae350 con 0x5635f0523b10 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f36a35fef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5635f051d3a0 con 0x7f3680003bf0 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f36a35fef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5635f05ae070 con 0x5635f051df60 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f369d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f0523b10 0x5635f0508540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f369d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f0523b10 0x5635f0508540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41314/0 (socket says 172.21.15.19:41314) 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.594+0000 7f369d200640 1 -- 172.21.15.19:0/2649315837 learned_addr learned my addr 172.21.15.19:0/2649315837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f369be00640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3680003bf0 0x5635f0508000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f369d200640 1 -- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5635f051df60 msgr2=0x5635f05a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f369d200640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5635f051df60 0x5635f05a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f369d200640 1 -- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3680003bf0 msgr2=0x5635f0508000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f369d200640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3680003bf0 0x5635f0508000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f369d200640 1 -- 172.21.15.19:0/2649315837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f05a6a40 con 0x5635f0523b10 2024-04-25T11:35:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f369d200640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f0523b10 0x5635f0508540 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x5635f05af940 tx=0x7f3694001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f3699600640 1 -- 172.21.15.19:0/2649315837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694073070 con 0x5635f0523b10 2024-04-25T11:35:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f3699600640 1 -- 172.21.15.19:0/2649315837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3694061820 con 0x5635f0523b10 2024-04-25T11:35:29.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f3699600640 1 -- 172.21.15.19:0/2649315837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694066040 con 0x5635f0523b10 2024-04-25T11:35:29.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.595+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5635f0509330 con 0x5635f0523b10 2024-04-25T11:35:29.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.596+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f063c9f0 con 0x5635f0523b10 2024-04-25T11:35:29.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.598+0000 7f3699600640 1 -- 172.21.15.19:0/2649315837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3694076070 con 0x5635f0523b10 2024-04-25T11:35:29.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.599+0000 7f3699600640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f367c072e90 0x7f367c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.599+0000 7f3699600640 1 -- 172.21.15.19:0/2649315837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f36940d99e0 con 0x5635f0523b10 2024-04-25T11:35:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.599+0000 7f36a35fef40 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5635f05ab460 0x5635f05ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.599+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x5635f05a8720 con 0x5635f05ab460 2024-04-25T11:35:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.600+0000 7f369c800640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f367c072e90 0x7f367c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.600+0000 7f369be00640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5635f05ab460 0x5635f05ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.601+0000 7f369c800640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f367c072e90 0x7f367c0753a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x5635f0525340 tx=0x7f368c005730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.601+0000 7f369be00640 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5635f05ab460 0x5635f05ab870 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.602+0000 7f369be00640 1 -- 172.21.15.19:0/2649315837 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f36880015e0 con 0x5635f05ab460 2024-04-25T11:35:29.590 INFO:teuthology.orchestra.run.smithi019.stdout:header (274 bytes) : 2024-04-25T11:35:29.590 INFO:teuthology.orchestra.run.smithi019.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:* 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:* 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:* 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:* 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000110 00 00 |..| 2024-04-25T11:35:29.591 INFO:teuthology.orchestra.run.smithi019.stdout:00000112 2024-04-25T11:35:29.592 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:29.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.602+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5635f05ab460 msgr2=0x5635f05ab870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.602+0000 7f36a35fef40 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5635f05ab460 0x5635f05ab870 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.602+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f367c072e90 msgr2=0x7f367c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.602+0000 7f36a35fef40 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f367c072e90 0x7f367c0753a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x5635f0525340 tx=0x7f368c005730 comp rx=0 tx=0).stop 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.602+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f0523b10 msgr2=0x5635f0508540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.602+0000 7f36a35fef40 1 --2- 172.21.15.19:0/2649315837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635f0523b10 0x5635f0508540 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x5635f05af940 tx=0x7f3694001970 comp rx=0 tx=0).stop 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.603+0000 7f369d200640 1 -- 172.21.15.19:0/2649315837 reap_dead start 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.603+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 shutdown_connections 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.603+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 >> 172.21.15.19:0/2649315837 conn(0x5635f03f9ac0 msgr2=0x5635f0522870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.603+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 shutdown_connections 2024-04-25T11:35:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.603+0000 7f36a35fef40 1 -- 172.21.15.19:0/2649315837 wait complete. 2024-04-25T11:35:29.605 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2024-04-25T11:35:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.799+0000 7f047ad08f40 1 Processor -- start 2024-04-25T11:35:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.799+0000 7f047ad08f40 1 -- start start 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f047ad08f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 0x55bce3fce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f047ad08f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3fe8da0 0x55bce4076d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f047ad08f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3febf60 0x55bce407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f047ad08f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bce3fe96f0 con 0x55bce3fe8da0 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f047ad08f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bce3f46d20 con 0x55bce3febf60 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f047ad08f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bce3fecab0 con 0x55bce3fe8670 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f0473400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3fe8da0 0x55bce4076d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f0473400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3fe8da0 0x55bce4076d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41330/0 (socket says 172.21.15.19:41330) 2024-04-25T11:35:29.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.801+0000 7f0473400640 1 -- 172.21.15.19:0/1029600744 learned_addr learned my addr 172.21.15.19:0/1029600744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0474800640 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3febf60 0x55bce407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0473e00640 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 0x55bce3fce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0473400640 1 -- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 msgr2=0x55bce3fce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0473400640 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 0x55bce3fce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0473400640 1 -- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3febf60 msgr2=0x55bce407d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0473400640 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3febf60 0x55bce407d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0473400640 1 -- 172.21.15.19:0/1029600744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce3fcec10 con 0x55bce3fe8da0 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0474800640 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3febf60 0x55bce407d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:29.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0473400640 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3fe8da0 0x55bce4076d20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0460000c00 tx=0x7f0460051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e68ecda7b9ca82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0472000640 1 -- 172.21.15.19:0/1029600744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460056660 con 0x55bce3fe8da0 2024-04-25T11:35:29.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0472000640 1 -- 172.21.15.19:0/1029600744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0460056820 con 0x55bce3fe8da0 2024-04-25T11:35:29.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.802+0000 7f0472000640 1 -- 172.21.15.19:0/1029600744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460056c10 con 0x55bce3fe8da0 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 -- 172.21.15.19:0/1029600744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3fe8da0 msgr2=0x55bce4076d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3fe8da0 0x55bce4076d20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0460000c00 tx=0x7f0460051000 comp rx=0 tx=0).stop 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 -- 172.21.15.19:0/1029600744 shutdown_connections 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3febf60 0x55bce407d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3fe8da0 0x55bce4076d20 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 --2- 172.21.15.19:0/1029600744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 0x55bce3fce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 -- 172.21.15.19:0/1029600744 >> 172.21.15.19:0/1029600744 conn(0x55bce3ec2ac0 msgr2=0x55bce4079210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 -- 172.21.15.19:0/1029600744 shutdown_connections 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.803+0000 7f047ad08f40 1 -- 172.21.15.19:0/1029600744 wait complete. 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.804+0000 7f047ad08f40 1 Processor -- start 2024-04-25T11:35:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.805+0000 7f047ad08f40 1 -- start start 2024-04-25T11:35:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f047ad08f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 0x55bce4071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f047ad08f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3fe8da0 0x55bce4073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f047ad08f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3febf60 0x55bce4073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f047ad08f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bce407a0a0 con 0x55bce3febf60 2024-04-25T11:35:29.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f047ad08f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bce4078d40 con 0x55bce3fe8da0 2024-04-25T11:35:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f047ad08f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bce3fe6870 con 0x55bce3fe8670 2024-04-25T11:35:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f0474800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3febf60 0x55bce4073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f0474800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3febf60 0x55bce4073940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41340/0 (socket says 172.21.15.19:41340) 2024-04-25T11:35:29.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.806+0000 7f0474800640 1 -- 172.21.15.19:0/2345954048 learned_addr learned my addr 172.21.15.19:0/2345954048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0473e00640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 0x55bce4071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0473400640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3fe8da0 0x55bce4073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0474800640 1 -- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 msgr2=0x55bce4071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0474800640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bce3fe8670 0x55bce4071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0474800640 1 -- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3fe8da0 msgr2=0x55bce4073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0474800640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3fe8da0 0x55bce4073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0474800640 1 -- 172.21.15.19:0/2345954048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce3fa4ef0 con 0x55bce3febf60 2024-04-25T11:35:29.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0473400640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bce3fe8da0 0x55bce4073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.807+0000 7f0474800640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3febf60 0x55bce4073940 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f046c008220 tx=0x7f046c0086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.808+0000 7f0470c00640 1 -- 172.21.15.19:0/2345954048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c016020 con 0x55bce3febf60 2024-04-25T11:35:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.808+0000 7f0470c00640 1 -- 172.21.15.19:0/2345954048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c012610 con 0x55bce3febf60 2024-04-25T11:35:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.808+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bce4074a00 con 0x55bce3febf60 2024-04-25T11:35:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.808+0000 7f0470c00640 1 -- 172.21.15.19:0/2345954048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c00e040 con 0x55bce3febf60 2024-04-25T11:35:29.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.808+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bce3fe65b0 con 0x55bce3febf60 2024-04-25T11:35:29.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.810+0000 7f0470c00640 1 -- 172.21.15.19:0/2345954048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f046c019020 con 0x55bce3febf60 2024-04-25T11:35:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.811+0000 7f0470c00640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0454072e90 0x7f04540753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.811+0000 7f0470c00640 1 -- 172.21.15.19:0/2345954048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f046c093d20 con 0x55bce3febf60 2024-04-25T11:35:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.812+0000 7f047ad08f40 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bce407ab90 0x55bce3fe2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.812+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55bce3fe2bd0 con 0x55bce407ab90 2024-04-25T11:35:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.812+0000 7f0473e00640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0454072e90 0x7f04540753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.812+0000 7f0473400640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bce407ab90 0x55bce3fe2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f0473400640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bce407ab90 0x55bce3fe2690 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f0473400640 1 -- 172.21.15.19:0/2345954048 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04600569d0 con 0x55bce407ab90 2024-04-25T11:35:29.801 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:29.801 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bce407ab90 msgr2=0x55bce3fe2690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f047ad08f40 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bce407ab90 0x55bce3fe2690 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f0473e00640 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0454072e90 0x7f04540753a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55bce3fa6d20 tx=0x7f0464006770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0454072e90 msgr2=0x7f04540753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f047ad08f40 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0454072e90 0x7f04540753a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55bce3fa6d20 tx=0x7f0464006770 comp rx=0 tx=0).stop 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3febf60 msgr2=0x55bce4073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.813+0000 7f047ad08f40 1 --2- 172.21.15.19:0/2345954048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bce3febf60 0x55bce4073940 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f046c008220 tx=0x7f046c0086e0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.814+0000 7f0474800640 1 -- 172.21.15.19:0/2345954048 reap_dead start 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.814+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 shutdown_connections 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.814+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 >> 172.21.15.19:0/2345954048 conn(0x55bce3ec2ac0 msgr2=0x55bce4077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.814+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 shutdown_connections 2024-04-25T11:35:29.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:29.814+0000 7f047ad08f40 1 -- 172.21.15.19:0/2345954048 wait complete. 2024-04-25T11:35:29.818 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2024-04-25T11:35:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.006+0000 7f59e9786f40 1 Processor -- start 2024-04-25T11:35:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.006+0000 7f59e9786f40 1 -- start start 2024-04-25T11:35:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.007+0000 7f59e9786f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e91e0 0x5649c94bf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.007+0000 7f59e9786f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c93e95f0 0x5649c94b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.007+0000 7f59e9786f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c94c2dd0 0x5649c94bb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.007+0000 7f59e9786f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5649c9431e70 con 0x5649c93e95f0 2024-04-25T11:35:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.007+0000 7f59e9786f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649c938fd20 con 0x5649c93e91e0 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.007+0000 7f59e9786f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5649c94323f0 con 0x5649c94c2dd0 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c93e95f0 0x5649c94b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c93e95f0 0x5649c94b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41352/0 (socket says 172.21.15.19:41352) 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 -- 172.21.15.19:0/2915790063 learned_addr learned my addr 172.21.15.19:0/2915790063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2a00640 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e91e0 0x5649c94bf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e3400640 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c94c2dd0 0x5649c94bb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 -- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c94c2dd0 msgr2=0x5649c94bb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c94c2dd0 0x5649c94bb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 -- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e91e0 msgr2=0x5649c94bf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e91e0 0x5649c94bf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2000640 1 -- 172.21.15.19:0/2915790063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c94b6320 con 0x5649c93e95f0 2024-04-25T11:35:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.008+0000 7f59e2a00640 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e91e0 0x5649c94bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.009+0000 7f59e2000640 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c93e95f0 0x5649c94b5de0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f59d0025980 tx=0x7f59d0050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2081b837c6c44b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.009+0000 7f59e0c00640 1 -- 172.21.15.19:0/2915790063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d006a070 con 0x5649c93e95f0 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.009+0000 7f59e0c00640 1 -- 172.21.15.19:0/2915790063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d005d9c0 con 0x5649c93e95f0 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.009+0000 7f59e0c00640 1 -- 172.21.15.19:0/2915790063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d005ddb0 con 0x5649c93e95f0 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 -- 172.21.15.19:0/2915790063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c93e95f0 msgr2=0x5649c94b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c93e95f0 0x5649c94b5de0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f59d0025980 tx=0x7f59d0050000 comp rx=0 tx=0).stop 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 -- 172.21.15.19:0/2915790063 shutdown_connections 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c94c2dd0 0x5649c94bb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c93e95f0 0x5649c94b5de0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 --2- 172.21.15.19:0/2915790063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e91e0 0x5649c94bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 -- 172.21.15.19:0/2915790063 >> 172.21.15.19:0/2915790063 conn(0x5649c930bac0 msgr2=0x5649c94bffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 -- 172.21.15.19:0/2915790063 shutdown_connections 2024-04-25T11:35:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.010+0000 7f59e9786f40 1 -- 172.21.15.19:0/2915790063 wait complete. 2024-04-25T11:35:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.011+0000 7f59e9786f40 1 Processor -- start 2024-04-25T11:35:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.011+0000 7f59e9786f40 1 -- start start 2024-04-25T11:35:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e9786f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c93e91e0 0x5649c94bc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e9786f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e95f0 0x5649c9550bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e9786f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c94c2dd0 0x5649c95510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e9786f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5649c941b620 con 0x5649c94c2dd0 2024-04-25T11:35:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e9786f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649c93e87c0 con 0x5649c93e95f0 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e9786f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5649c94b4dd0 con 0x5649c93e91e0 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c94c2dd0 0x5649c95510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c94c2dd0 0x5649c95510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41354/0 (socket says 172.21.15.19:41354) 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 -- 172.21.15.19:0/1062962719 learned_addr learned my addr 172.21.15.19:0/1062962719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e2000640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e95f0 0x5649c9550bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 -- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c93e91e0 msgr2=0x5649c94bc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e2a00640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c93e91e0 0x5649c94bc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649c93e91e0 0x5649c94bc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 -- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e95f0 msgr2=0x5649c9550bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e95f0 0x5649c9550bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.013+0000 7f59e3400640 1 -- 172.21.15.19:0/1062962719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c94c0680 con 0x5649c94c2dd0 2024-04-25T11:35:30.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.014+0000 7f59e2000640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649c93e95f0 0x5649c9550bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.014+0000 7f59e3400640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c94c2dd0 0x5649c95510f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00bfb0 tx=0x7f59dc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.014+0000 7f59db400640 1 -- 172.21.15.19:0/1062962719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc007760 con 0x5649c94c2dd0 2024-04-25T11:35:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.014+0000 7f59db400640 1 -- 172.21.15.19:0/1062962719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59dc007920 con 0x5649c94c2dd0 2024-04-25T11:35:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.014+0000 7f59db400640 1 -- 172.21.15.19:0/1062962719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc007d10 con 0x5649c94c2dd0 2024-04-25T11:35:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.014+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5649c9433a00 con 0x5649c94c2dd0 2024-04-25T11:35:30.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.014+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c94bff50 con 0x5649c94c2dd0 2024-04-25T11:35:30.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.017+0000 7f59db400640 1 -- 172.21.15.19:0/1062962719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f59dc01d440 con 0x5649c94c2dd0 2024-04-25T11:35:30.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.018+0000 7f59db400640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59c4072e90 0x7f59c40753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.018+0000 7f59db400640 1 -- 172.21.15.19:0/1062962719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f59dc094100 con 0x5649c94c2dd0 2024-04-25T11:35:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.018+0000 7f59e9786f40 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5649c954e8c0 0x5649c955a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.018+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x5649c94b5220 con 0x5649c954e8c0 2024-04-25T11:35:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.018+0000 7f59e2a00640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59c4072e90 0x7f59c40753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.018+0000 7f59e2000640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5649c954e8c0 0x5649c955a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.019+0000 7f59e2000640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5649c954e8c0 0x5649c955a650 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e2000640 1 -- 172.21.15.19:0/1062962719 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f59d005db70 con 0x5649c954e8c0 2024-04-25T11:35:30.008 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:30.008 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5649c954e8c0 msgr2=0x5649c955a650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5649c954e8c0 0x5649c955a650 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e2a00640 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59c4072e90 0x7f59c40753a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f59d40095b0 tx=0x7f59d4005820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59c4072e90 msgr2=0x7f59c40753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59c4072e90 0x7f59c40753a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f59d40095b0 tx=0x7f59d4005820 comp rx=0 tx=0).stop 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c94c2dd0 msgr2=0x5649c95510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 --2- 172.21.15.19:0/1062962719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649c94c2dd0 0x5649c95510f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00bfb0 tx=0x7f59dc009800 comp rx=0 tx=0).stop 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e3400640 1 -- 172.21.15.19:0/1062962719 reap_dead start 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 shutdown_connections 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 >> 172.21.15.19:0/1062962719 conn(0x5649c930bac0 msgr2=0x5649c9433250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 shutdown_connections 2024-04-25T11:35:30.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.020+0000 7f59e9786f40 1 -- 172.21.15.19:0/1062962719 wait complete. 2024-04-25T11:35:30.020 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2024-04-25T11:35:30.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.211+0000 7f0c8d75bf40 1 Processor -- start 2024-04-25T11:35:30.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.211+0000 7f0c8d75bf40 1 -- start start 2024-04-25T11:35:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c8d75bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6670 0x556b6b31aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c8d75bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6da0 0x556b6b2843c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c8d75bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b284900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c8d75bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b6b1f76f0 con 0x556b6b1f6670 2024-04-25T11:35:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c8d75bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556b6b154d20 con 0x556b6b1f9f60 2024-04-25T11:35:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c8d75bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556b6b1faab0 con 0x556b6b1f6da0 2024-04-25T11:35:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c86800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6670 0x556b6b31aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c86800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6670 0x556b6b31aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41368/0 (socket says 172.21.15.19:41368) 2024-04-25T11:35:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c86800640 1 -- 172.21.15.19:0/3199051674 learned_addr learned my addr 172.21.15.19:0/3199051674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.213+0000 7f0c85e00640 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6da0 0x556b6b2843c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c87200640 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b284900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c86800640 1 -- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6da0 msgr2=0x556b6b2843c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c86800640 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6da0 0x556b6b2843c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c86800640 1 -- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 msgr2=0x556b6b284900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c86800640 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b284900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c86800640 1 -- 172.21.15.19:0/3199051674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6b317c90 con 0x556b6b1f6670 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c86800640 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6670 0x556b6b31aa60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800a430 tx=0x7f0c7800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e39037b295b4d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.214+0000 7f0c84a00640 1 -- 172.21.15.19:0/3199051674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78015070 con 0x556b6b1f6670 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c84a00640 1 -- 172.21.15.19:0/3199051674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7800e730 con 0x556b6b1f6670 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c84a00640 1 -- 172.21.15.19:0/3199051674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7800eb20 con 0x556b6b1f6670 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/3199051674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6670 msgr2=0x556b6b31aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6670 0x556b6b31aa60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800a430 tx=0x7f0c7800a8f0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/3199051674 shutdown_connections 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b284900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6da0 0x556b6b2843c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/3199051674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6670 0x556b6b31aa60 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/3199051674 >> 172.21.15.19:0/3199051674 conn(0x556b6b0d0ac0 msgr2=0x556b6b1f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/3199051674 shutdown_connections 2024-04-25T11:35:30.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.215+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/3199051674 wait complete. 2024-04-25T11:35:30.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.216+0000 7f0c8d75bf40 1 Processor -- start 2024-04-25T11:35:30.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.217+0000 7f0c8d75bf40 1 -- start start 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c8d75bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6670 0x556b6b28bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c8d75bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6da0 0x556b6b1f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c8d75bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b286090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c8d75bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556b6b285aa0 con 0x556b6b1f6da0 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c8d75bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556b6b31afa0 con 0x556b6b1f9f60 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c8d75bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556b6b1f2200 con 0x556b6b1f6670 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c85e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6da0 0x556b6b1f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c85e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6da0 0x556b6b1f1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41378/0 (socket says 172.21.15.19:41378) 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.218+0000 7f0c85e00640 1 -- 172.21.15.19:0/682252267 learned_addr learned my addr 172.21.15.19:0/682252267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c86800640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6670 0x556b6b28bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c87200640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b286090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c85e00640 1 -- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6670 msgr2=0x556b6b28bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c85e00640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556b6b1f6670 0x556b6b28bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c85e00640 1 -- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 msgr2=0x556b6b286090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c85e00640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b286090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c85e00640 1 -- 172.21.15.19:0/682252267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6b1f5610 con 0x556b6b1f6da0 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c87200640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556b6b1f9f60 0x556b6b286090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:30.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.219+0000 7f0c85e00640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6da0 0x556b6b1f1710 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0c70007d20 tx=0x7f0c70006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.220+0000 7f0c7f400640 1 -- 172.21.15.19:0/682252267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70018070 con 0x556b6b1f6da0 2024-04-25T11:35:30.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.220+0000 7f0c7f400640 1 -- 172.21.15.19:0/682252267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c70011700 con 0x556b6b1f6da0 2024-04-25T11:35:30.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.220+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b6b31ad40 con 0x556b6b1f6da0 2024-04-25T11:35:30.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.220+0000 7f0c7f400640 1 -- 172.21.15.19:0/682252267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011af0 con 0x556b6b1f6da0 2024-04-25T11:35:30.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.220+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6b1f4d00 con 0x556b6b1f6da0 2024-04-25T11:35:30.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.222+0000 7f0c7f400640 1 -- 172.21.15.19:0/682252267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c7001b070 con 0x556b6b1f6da0 2024-04-25T11:35:30.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.223+0000 7f0c7f400640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c64072e90 0x7f0c640753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.223+0000 7f0c7f400640 1 -- 172.21.15.19:0/682252267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f0c700931a0 con 0x556b6b1f6da0 2024-04-25T11:35:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.224+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556b6b2865d0 0x556b6b28acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.224+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x556b6b1f4000 con 0x556b6b2865d0 2024-04-25T11:35:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.224+0000 7f0c86800640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c64072e90 0x7f0c640753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.224+0000 7f0c87200640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556b6b2865d0 0x556b6b28acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.225+0000 7f0c87200640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556b6b2865d0 0x556b6b28acf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.225+0000 7f0c86800640 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c64072e90 0x7f0c640753a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0c78013040 tx=0x7f0c78008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.225+0000 7f0c87200640 1 -- 172.21.15.19:0/682252267 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0c80050050 con 0x556b6b2865d0 2024-04-25T11:35:30.214 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.225+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556b6b2865d0 msgr2=0x556b6b28acf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.225+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556b6b2865d0 0x556b6b28acf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c64072e90 msgr2=0x7f0c640753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0c64072e90 0x7f0c640753a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0c78013040 tx=0x7f0c78008000 comp rx=0 tx=0).stop 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6da0 msgr2=0x556b6b1f1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 --2- 172.21.15.19:0/682252267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556b6b1f6da0 0x556b6b1f1710 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0c70007d20 tx=0x7f0c70006750 comp rx=0 tx=0).stop 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c87200640 1 -- 172.21.15.19:0/682252267 reap_dead start 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 shutdown_connections 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 >> 172.21.15.19:0/682252267 conn(0x556b6b0d0ac0 msgr2=0x556b6b1dd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 shutdown_connections 2024-04-25T11:35:30.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.226+0000 7f0c8d75bf40 1 -- 172.21.15.19:0/682252267 wait complete. 2024-04-25T11:35:30.225 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2024-04-25T11:35:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.418+0000 7f0108ddbf40 1 Processor -- start 2024-04-25T11:35:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.418+0000 7f0108ddbf40 1 -- start start 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.419+0000 7f0108ddbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f1e0 0x55ea40407830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.419+0000 7f0108ddbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f5f0 0x55ea40360980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.419+0000 7f0108ddbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea403fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.419+0000 7f0108ddbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea40377e70 con 0x55ea40408fa0 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.419+0000 7f0108ddbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ea402d5d20 con 0x55ea4032f5f0 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.419+0000 7f0108ddbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ea403783f0 con 0x55ea4032f1e0 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea403fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea403fdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41400/0 (socket says 172.21.15.19:41400) 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 -- 172.21.15.19:0/1346765792 learned_addr learned my addr 172.21.15.19:0/1346765792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102000640 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f1e0 0x55ea40407830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 -- 172.21.15.19:0/1346765792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f1e0 msgr2=0x55ea40407830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f1e0 0x55ea40407830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 -- 172.21.15.19:0/1346765792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f5f0 msgr2=0x55ea40360980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f5f0 0x55ea40360980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.420+0000 7f0102a00640 1 -- 172.21.15.19:0/1346765792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea40407d70 con 0x55ea40408fa0 2024-04-25T11:35:30.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.421+0000 7f0102a00640 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea403fdaa0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f00fc025ca0 tx=0x7f00fc051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec4ac7aa91d3e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.421+0000 7f00fbe00640 1 -- 172.21.15.19:0/1346765792 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc062070 con 0x55ea40408fa0 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.421+0000 7f00fbe00640 1 -- 172.21.15.19:0/1346765792 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00fc054970 con 0x55ea40408fa0 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.421+0000 7f00fbe00640 1 -- 172.21.15.19:0/1346765792 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc054d20 con 0x55ea40408fa0 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.421+0000 7f0108ddbf40 1 -- 172.21.15.19:0/1346765792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 msgr2=0x55ea403fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.421+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea403fdaa0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f00fc025ca0 tx=0x7f00fc051000 comp rx=0 tx=0).stop 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.422+0000 7f0108ddbf40 1 -- 172.21.15.19:0/1346765792 shutdown_connections 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.422+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f5f0 0x55ea40360980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.422+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea403fdaa0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.422+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/1346765792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f1e0 0x55ea40407830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.422+0000 7f0108ddbf40 1 -- 172.21.15.19:0/1346765792 >> 172.21.15.19:0/1346765792 conn(0x55ea40251ac0 msgr2=0x55ea40361500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.422+0000 7f0108ddbf40 1 -- 172.21.15.19:0/1346765792 shutdown_connections 2024-04-25T11:35:30.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.422+0000 7f0108ddbf40 1 -- 172.21.15.19:0/1346765792 wait complete. 2024-04-25T11:35:30.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.423+0000 7f0108ddbf40 1 Processor -- start 2024-04-25T11:35:30.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.423+0000 7f0108ddbf40 1 -- start start 2024-04-25T11:35:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0108ddbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f1e0 0x55ea40407810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0108ddbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f5f0 0x55ea403f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0108ddbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea40400920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0108ddbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ea40408350 con 0x55ea40408fa0 2024-04-25T11:35:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0108ddbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ea40403010 con 0x55ea4032f1e0 2024-04-25T11:35:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0108ddbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ea40371210 con 0x55ea4032f5f0 2024-04-25T11:35:30.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0102a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea40400920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0102a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea40400920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41404/0 (socket says 172.21.15.19:41404) 2024-04-25T11:35:30.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.425+0000 7f0102a00640 1 -- 172.21.15.19:0/3450865774 learned_addr learned my addr 172.21.15.19:0/3450865774 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0101600640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f5f0 0x55ea403f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102000640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f1e0 0x55ea40407810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102a00640 1 -- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f5f0 msgr2=0x55ea403f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102a00640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ea4032f5f0 0x55ea403f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102a00640 1 -- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f1e0 msgr2=0x55ea40407810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102a00640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f1e0 0x55ea40407810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102a00640 1 -- 172.21.15.19:0/3450865774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea40371460 con 0x55ea40408fa0 2024-04-25T11:35:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102000640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ea4032f1e0 0x55ea40407810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.426+0000 7f0102a00640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea40400920 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00a5a0 tx=0x7f00fc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.427+0000 7f00faa00640 1 -- 172.21.15.19:0/3450865774 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc074070 con 0x55ea40408fa0 2024-04-25T11:35:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.427+0000 7f00faa00640 1 -- 172.21.15.19:0/3450865774 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00fc067450 con 0x55ea40408fa0 2024-04-25T11:35:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.427+0000 7f00faa00640 1 -- 172.21.15.19:0/3450865774 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc062040 con 0x55ea40408fa0 2024-04-25T11:35:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.427+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea40371d00 con 0x55ea40408fa0 2024-04-25T11:35:30.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.427+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea40371a20 con 0x55ea40408fa0 2024-04-25T11:35:30.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.429+0000 7f00faa00640 1 -- 172.21.15.19:0/3450865774 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00fc077070 con 0x55ea40408fa0 2024-04-25T11:35:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.430+0000 7f00faa00640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00e0072ec0 0x7f00e00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.431+0000 7f00faa00640 1 -- 172.21.15.19:0/3450865774 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f00fc0d9300 con 0x55ea40408fa0 2024-04-25T11:35:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.431+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ea403fc150 0x55ea403fc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.431+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55ea403fcaa0 con 0x55ea403fc150 2024-04-25T11:35:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.431+0000 7f0102000640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00e0072ec0 0x7f00e00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.432+0000 7f0101600640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ea403fc150 0x55ea403fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.433+0000 7f0102000640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00e0072ec0 0x7f00e00753d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f00f400c550 tx=0x7f00f4007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.433+0000 7f0101600640 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ea403fc150 0x55ea403fc560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0101600640 1 -- 172.21.15.19:0/3450865774 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv4 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f00ec00e040 con 0x55ea403fc150 2024-04-25T11:35:30.422 INFO:teuthology.orchestra.run.smithi019.stdout:header (42 bytes) : 2024-04-25T11:35:30.422 INFO:teuthology.orchestra.run.smithi019.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2024-04-25T11:35:30.422 INFO:teuthology.orchestra.run.smithi019.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 04 00 00 |me v011.........| 2024-04-25T11:35:30.422 INFO:teuthology.orchestra.run.smithi019.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2024-04-25T11:35:30.422 INFO:teuthology.orchestra.run.smithi019.stdout:0000002a 2024-04-25T11:35:30.422 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ea403fc150 msgr2=0x55ea403fc560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ea403fc150 0x55ea403fc560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00e0072ec0 msgr2=0x7f00e00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f00e0072ec0 0x7f00e00753d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f00f400c550 tx=0x7f00f4007040 comp rx=0 tx=0).stop 2024-04-25T11:35:30.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 msgr2=0x55ea40400920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 --2- 172.21.15.19:0/3450865774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ea40408fa0 0x55ea40400920 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00a5a0 tx=0x7f00fc002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0102a00640 1 -- 172.21.15.19:0/3450865774 reap_dead start 2024-04-25T11:35:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 shutdown_connections 2024-04-25T11:35:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 >> 172.21.15.19:0/3450865774 conn(0x55ea40251ac0 msgr2=0x55ea40360940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 shutdown_connections 2024-04-25T11:35:30.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.434+0000 7f0108ddbf40 1 -- 172.21.15.19:0/3450865774 wait complete. 2024-04-25T11:35:30.437 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2024-04-25T11:35:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.627+0000 7fabaa78cf40 1 Processor -- start 2024-04-25T11:35:30.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.627+0000 7fabaa78cf40 1 -- start start 2024-04-25T11:35:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.628+0000 7fabaa78cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d45d1e0 0x55917d529e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.628+0000 7fabaa78cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 0x55917d531b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.628+0000 7fabaa78cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d536dd0 0x55917d5320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.628+0000 7fabaa78cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55917d4a5e70 con 0x55917d536dd0 2024-04-25T11:35:30.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.628+0000 7fabaa78cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55917d403d20 con 0x55917d45d5f0 2024-04-25T11:35:30.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.628+0000 7fabaa78cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55917d4a6110 con 0x55917d45d1e0 2024-04-25T11:35:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d536dd0 0x55917d5320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d536dd0 0x55917d5320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41422/0 (socket says 172.21.15.19:41422) 2024-04-25T11:35:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 -- 172.21.15.19:0/4284172882 learned_addr learned my addr 172.21.15.19:0/4284172882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba3a00640 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d45d1e0 0x55917d529e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba3000640 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 0x55917d531b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 -- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d45d1e0 msgr2=0x55917d529e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d45d1e0 0x55917d529e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 -- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 msgr2=0x55917d531b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 0x55917d531b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.629+0000 7faba4400640 1 -- 172.21.15.19:0/4284172882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917d52a3d0 con 0x55917d536dd0 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.630+0000 7faba4400640 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d536dd0 0x55917d5320c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a590 tx=0x7fab9c00a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeea4a4273bc3f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.630+0000 7faba1c00640 1 -- 172.21.15.19:0/4284172882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c019070 con 0x55917d536dd0 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.630+0000 7faba1c00640 1 -- 172.21.15.19:0/4284172882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c007600 con 0x55917d536dd0 2024-04-25T11:35:30.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.630+0000 7faba1c00640 1 -- 172.21.15.19:0/4284172882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0079f0 con 0x55917d536dd0 2024-04-25T11:35:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 -- 172.21.15.19:0/4284172882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d536dd0 msgr2=0x55917d5320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d536dd0 0x55917d5320c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a590 tx=0x7fab9c00a680 comp rx=0 tx=0).stop 2024-04-25T11:35:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 -- 172.21.15.19:0/4284172882 shutdown_connections 2024-04-25T11:35:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 0x55917d531b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d536dd0 0x55917d5320c0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/4284172882 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d45d1e0 0x55917d529e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 -- 172.21.15.19:0/4284172882 >> 172.21.15.19:0/4284172882 conn(0x55917d37fac0 msgr2=0x55917d535b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 -- 172.21.15.19:0/4284172882 shutdown_connections 2024-04-25T11:35:30.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.631+0000 7fabaa78cf40 1 -- 172.21.15.19:0/4284172882 wait complete. 2024-04-25T11:35:30.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.632+0000 7fabaa78cf40 1 Processor -- start 2024-04-25T11:35:30.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.632+0000 7fabaa78cf40 1 -- start start 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7fabaa78cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d45d1e0 0x55917d529190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7fabaa78cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 0x55917d52f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7fabaa78cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d536dd0 0x55917d52fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7fabaa78cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55917d5337f0 con 0x55917d45d1e0 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7fabaa78cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55917d4a7370 con 0x55917d45d5f0 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7fabaa78cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55917d5cb330 con 0x55917d536dd0 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d45d1e0 0x55917d529190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d45d1e0 0x55917d529190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41426/0 (socket says 172.21.15.19:41426) 2024-04-25T11:35:30.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 -- 172.21.15.19:0/1665524333 learned_addr learned my addr 172.21.15.19:0/1665524333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba4400640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d536dd0 0x55917d52fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 -- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d536dd0 msgr2=0x55917d52fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55917d536dd0 0x55917d52fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 -- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 msgr2=0x55917d52f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55917d45d5f0 0x55917d52f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.634+0000 7faba3a00640 1 -- 172.21.15.19:0/1665524333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917d52a200 con 0x55917d45d1e0 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.635+0000 7faba3a00640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d45d1e0 0x55917d529190 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00ce10 tx=0x7fab8c00a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.635+0000 7fab9be00640 1 -- 172.21.15.19:0/1665524333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0085e0 con 0x55917d45d1e0 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.635+0000 7fab9be00640 1 -- 172.21.15.19:0/1665524333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c008780 con 0x55917d45d1e0 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.635+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55917d45ce20 con 0x55917d45d1e0 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.635+0000 7fab9be00640 1 -- 172.21.15.19:0/1665524333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c008b70 con 0x55917d45d1e0 2024-04-25T11:35:30.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.635+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917d45cb40 con 0x55917d45d1e0 2024-04-25T11:35:30.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.638+0000 7fab9be00640 1 -- 172.21.15.19:0/1665524333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab8c008d10 con 0x55917d45d1e0 2024-04-25T11:35:30.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.639+0000 7fab9be00640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fab80072ec0 0x7fab800753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.639+0000 7fab9be00640 1 -- 172.21.15.19:0/1665524333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fab8c0942a0 con 0x55917d45d1e0 2024-04-25T11:35:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.639+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55917d532ff0 0x55917d5ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.639+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55917d5cac00 con 0x55917d532ff0 2024-04-25T11:35:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.639+0000 7faba3000640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fab80072ec0 0x7fab800753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.640+0000 7faba4400640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55917d532ff0 0x55917d5ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.640+0000 7faba4400640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55917d532ff0 0x55917d5ca6c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7faba4400640 1 -- 172.21.15.19:0/1665524333 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fab9c01c040 con 0x55917d532ff0 2024-04-25T11:35:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:30.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55917d532ff0 msgr2=0x55917d5ca6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55917d532ff0 0x55917d5ca6c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7faba3000640 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fab80072ec0 0x7fab800753d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fab94056000 tx=0x7fab94050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fab80072ec0 msgr2=0x7fab800753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fab80072ec0 0x7fab800753d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fab94056000 tx=0x7fab94050000 comp rx=0 tx=0).stop 2024-04-25T11:35:30.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d45d1e0 msgr2=0x55917d529190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 --2- 172.21.15.19:0/1665524333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55917d45d1e0 0x55917d529190 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00ce10 tx=0x7fab8c00a750 comp rx=0 tx=0).stop 2024-04-25T11:35:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7faba4400640 1 -- 172.21.15.19:0/1665524333 reap_dead start 2024-04-25T11:35:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 shutdown_connections 2024-04-25T11:35:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 >> 172.21.15.19:0/1665524333 conn(0x55917d37fac0 msgr2=0x55917d52b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 shutdown_connections 2024-04-25T11:35:30.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.641+0000 7fabaa78cf40 1 -- 172.21.15.19:0/1665524333 wait complete. 2024-04-25T11:35:30.643 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2024-04-25T11:35:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.845+0000 7f7865a3bf40 1 Processor -- start 2024-04-25T11:35:30.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.845+0000 7f7865a3bf40 1 -- start start 2024-04-25T11:35:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f7865a3bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440030f0 0x55bb1674a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f7865a3bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb166c17b0 0x55bb1674cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f7865a3bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb16743540 0x55bb1674d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f7865a3bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb166bde70 con 0x55bb16743540 2024-04-25T11:35:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f7865a3bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bb1661bd20 con 0x55bb166c17b0 2024-04-25T11:35:30.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f7865a3bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bb166be3f0 con 0x7f78440030f0 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f785f600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb16743540 0x55bb1674d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f785f600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb16743540 0x55bb1674d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41440/0 (socket says 172.21.15.19:41440) 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f785f600640 1 -- 172.21.15.19:0/1997925389 learned_addr learned my addr 172.21.15.19:0/1997925389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.847+0000 7f785ec00640 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440030f0 0x55bb1674a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785e200640 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb166c17b0 0x55bb1674cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785f600640 1 -- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440030f0 msgr2=0x55bb1674a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785f600640 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440030f0 0x55bb1674a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785f600640 1 -- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb166c17b0 msgr2=0x55bb1674cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785f600640 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb166c17b0 0x55bb1674cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785f600640 1 -- 172.21.15.19:0/1997925389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1674bab0 con 0x55bb16743540 2024-04-25T11:35:30.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785e200640 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb166c17b0 0x55bb1674cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785f600640 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb16743540 0x55bb1674d420 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x55bb166befb0 tx=0x7f7858005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac92346ab32eb3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785ce00640 1 -- 172.21.15.19:0/1997925389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858016020 con 0x55bb16743540 2024-04-25T11:35:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785ce00640 1 -- 172.21.15.19:0/1997925389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7858012610 con 0x55bb16743540 2024-04-25T11:35:30.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.848+0000 7f785ce00640 1 -- 172.21.15.19:0/1997925389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785800e040 con 0x55bb16743540 2024-04-25T11:35:30.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 -- 172.21.15.19:0/1997925389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb16743540 msgr2=0x55bb1674d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb16743540 0x55bb1674d420 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x55bb166befb0 tx=0x7f7858005f90 comp rx=0 tx=0).stop 2024-04-25T11:35:30.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 -- 172.21.15.19:0/1997925389 shutdown_connections 2024-04-25T11:35:30.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb166c17b0 0x55bb1674cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bb16743540 0x55bb1674d420 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/1997925389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78440030f0 0x55bb1674a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 -- 172.21.15.19:0/1997925389 >> 172.21.15.19:0/1997925389 conn(0x55bb16597ac0 msgr2=0x55bb166c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 -- 172.21.15.19:0/1997925389 shutdown_connections 2024-04-25T11:35:30.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.849+0000 7f7865a3bf40 1 -- 172.21.15.19:0/1997925389 wait complete. 2024-04-25T11:35:30.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.850+0000 7f7865a3bf40 1 Processor -- start 2024-04-25T11:35:30.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.850+0000 7f7865a3bf40 1 -- start start 2024-04-25T11:35:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f7865a3bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb16743540 0x55bb167463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f7865a3bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440030f0 0x55bb16733cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f7865a3bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bb1673f010 0x55bb166743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f7865a3bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bb166bbda0 con 0x7f78440030f0 2024-04-25T11:35:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f7865a3bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bb16742c20 con 0x55bb16743540 2024-04-25T11:35:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f7865a3bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bb167469f0 con 0x55bb1673f010 2024-04-25T11:35:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f785e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440030f0 0x55bb16733cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f785e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440030f0 0x55bb16733cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41454/0 (socket says 172.21.15.19:41454) 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.852+0000 7f785e200640 1 -- 172.21.15.19:0/2169315255 learned_addr learned my addr 172.21.15.19:0/2169315255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785ec00640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb16743540 0x55bb167463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785f600640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bb1673f010 0x55bb166743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785e200640 1 -- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bb1673f010 msgr2=0x55bb166743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785e200640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bb1673f010 0x55bb166743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785e200640 1 -- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb16743540 msgr2=0x55bb167463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785e200640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bb16743540 0x55bb167463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785e200640 1 -- 172.21.15.19:0/2169315255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb167341f0 con 0x7f78440030f0 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785f600640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bb1673f010 0x55bb166743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.853+0000 7f785e200640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440030f0 0x55bb16733cb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f784c026300 tx=0x7f784c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.854+0000 7f7857400640 1 -- 172.21.15.19:0/2169315255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c065020 con 0x7f78440030f0 2024-04-25T11:35:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.854+0000 7f7857400640 1 -- 172.21.15.19:0/2169315255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784c001570 con 0x7f78440030f0 2024-04-25T11:35:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.854+0000 7f7857400640 1 -- 172.21.15.19:0/2169315255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c061650 con 0x7f78440030f0 2024-04-25T11:35:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.854+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bb16746c40 con 0x7f78440030f0 2024-04-25T11:35:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.854+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb167410e0 con 0x7f78440030f0 2024-04-25T11:35:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.856+0000 7f7857400640 1 -- 172.21.15.19:0/2169315255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f784c068020 con 0x7f78440030f0 2024-04-25T11:35:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.857+0000 7f7857400640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.857+0000 7f7857400640 1 -- 172.21.15.19:0/2169315255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f784c0d8710 con 0x7f78440030f0 2024-04-25T11:35:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.857+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bb16747f10 0x55bb16748320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.857+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55bb167418e0 con 0x55bb16747f10 2024-04-25T11:35:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.858+0000 7f785f600640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bb16747f10 0x55bb16748320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.858+0000 7f785ec00640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f785f600640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bb16747f10 0x55bb16748320 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f785f600640 1 -- 172.21.15.19:0/2169315255 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7858012a80 con 0x55bb16747f10 2024-04-25T11:35:30.847 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:30.847 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bb16747f10 msgr2=0x55bb16748320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bb16747f10 0x55bb16748320 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f785ec00640 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7850009520 tx=0x7f785000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 msgr2=0x7f78400753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7850009520 tx=0x7f785000b040 comp rx=0 tx=0).stop 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440030f0 msgr2=0x55bb16733cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f7865a3bf40 1 --2- 172.21.15.19:0/2169315255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440030f0 0x55bb16733cb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f784c026300 tx=0x7f784c001970 comp rx=0 tx=0).stop 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.859+0000 7f785f600640 1 -- 172.21.15.19:0/2169315255 reap_dead start 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.860+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 shutdown_connections 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.860+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 >> 172.21.15.19:0/2169315255 conn(0x55bb16597ac0 msgr2=0x55bb166c27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.860+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 shutdown_connections 2024-04-25T11:35:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:30.860+0000 7f7865a3bf40 1 -- 172.21.15.19:0/2169315255 wait complete. 2024-04-25T11:35:30.861 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2024-04-25T11:35:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.047+0000 7f416d71ef40 1 Processor -- start 2024-04-25T11:35:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.047+0000 7f416d71ef40 1 -- start start 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f416d71ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 0x564e8d5d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f416d71ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d65e030 0x564e8d5bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f416d71ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e8d5d8fe0 0x564e8d5d93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f416d71ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e8d534d20 con 0x564e8d65e030 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f416d71ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564e8d538d60 con 0x7f414c0030f0 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f416d71ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564e8d5d73f0 con 0x564e8d5d8fe0 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f4165400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d65e030 0x564e8d5bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f4165400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d65e030 0x564e8d5bf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41478/0 (socket says 172.21.15.19:41478) 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.048+0000 7f4165400640 1 -- 172.21.15.19:0/1651201767 learned_addr learned my addr 172.21.15.19:0/1651201767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f4165e00640 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 0x564e8d5d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f4165400640 1 -- 172.21.15.19:0/1651201767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e8d5d8fe0 msgr2=0x564e8d5d93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f4165400640 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e8d5d8fe0 0x564e8d5d93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f4165400640 1 -- 172.21.15.19:0/1651201767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 msgr2=0x564e8d5d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f4165400640 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 0x564e8d5d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f4165400640 1 -- 172.21.15.19:0/1651201767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e8d65e820 con 0x564e8d65e030 2024-04-25T11:35:31.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f4165400640 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d65e030 0x564e8d5bf3f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4154057040 tx=0x7f4154050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f685d935c0dd898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.049+0000 7f415fe00640 1 -- 172.21.15.19:0/1651201767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154061450 con 0x564e8d65e030 2024-04-25T11:35:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f415fe00640 1 -- 172.21.15.19:0/1651201767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4154066070 con 0x564e8d65e030 2024-04-25T11:35:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f415fe00640 1 -- 172.21.15.19:0/1651201767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154061810 con 0x564e8d65e030 2024-04-25T11:35:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 -- 172.21.15.19:0/1651201767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d65e030 msgr2=0x564e8d5bf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d65e030 0x564e8d5bf3f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4154057040 tx=0x7f4154050000 comp rx=0 tx=0).stop 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 -- 172.21.15.19:0/1651201767 shutdown_connections 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e8d5d8fe0 0x564e8d5d93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d65e030 0x564e8d5bf3f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 --2- 172.21.15.19:0/1651201767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 0x564e8d5d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 -- 172.21.15.19:0/1651201767 >> 172.21.15.19:0/1651201767 conn(0x564e8d4b0ac0 msgr2=0x564e8d5c0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 -- 172.21.15.19:0/1651201767 shutdown_connections 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.050+0000 7f416d71ef40 1 -- 172.21.15.19:0/1651201767 wait complete. 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.051+0000 7f416d71ef40 1 Processor -- start 2024-04-25T11:35:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.052+0000 7f416d71ef40 1 -- start start 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f416d71ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d5d8fe0 0x564e8d651de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f416d71ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e8d65e030 0x564e8d6f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f416d71ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 0x564e8d6f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f416d71ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564e8d5d4f50 con 0x564e8d5d8fe0 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f416d71ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564e8d65ea90 con 0x7f414c0030f0 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f416d71ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564e8d6fbd50 con 0x564e8d65e030 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f4165e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d5d8fe0 0x564e8d651de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f4165e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d5d8fe0 0x564e8d651de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41484/0 (socket says 172.21.15.19:41484) 2024-04-25T11:35:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.053+0000 7f4165e00640 1 -- 172.21.15.19:0/3975818241 learned_addr learned my addr 172.21.15.19:0/3975818241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f4166800640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 0x564e8d6f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f4165e00640 1 -- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e8d65e030 msgr2=0x564e8d6f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f4165e00640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564e8d65e030 0x564e8d6f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f4165e00640 1 -- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 msgr2=0x564e8d6f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f4165e00640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f414c0030f0 0x564e8d6f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f4165e00640 1 -- 172.21.15.19:0/3975818241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e8d666d10 con 0x564e8d5d8fe0 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f4165e00640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d5d8fe0 0x564e8d651de0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4158009e90 tx=0x7f4158009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.054+0000 7f415ea00640 1 -- 172.21.15.19:0/3975818241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158019070 con 0x564e8d5d8fe0 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.055+0000 7f415ea00640 1 -- 172.21.15.19:0/3975818241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4158014400 con 0x564e8d5d8fe0 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.055+0000 7f415ea00640 1 -- 172.21.15.19:0/3975818241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580147f0 con 0x564e8d5d8fe0 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.055+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e8d5bf7d0 con 0x564e8d5d8fe0 2024-04-25T11:35:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.055+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e8d65ecc0 con 0x564e8d5d8fe0 2024-04-25T11:35:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.057+0000 7f415ea00640 1 -- 172.21.15.19:0/3975818241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f415801c070 con 0x564e8d5d8fe0 2024-04-25T11:35:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.058+0000 7f415ea00640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4148072ec0 0x7f41480753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.058+0000 7f415ea00640 1 -- 172.21.15.19:0/3975818241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f4158093490 con 0x564e8d5d8fe0 2024-04-25T11:35:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.058+0000 7f416d71ef40 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564e8d6655b0 0x564e8d6659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.058+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x564e8d6f48d0 con 0x564e8d6655b0 2024-04-25T11:35:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.059+0000 7f4165400640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4148072ec0 0x7f41480753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.059+0000 7f4166800640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564e8d6655b0 0x564e8d6659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.060+0000 7f4165400640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4148072ec0 0x7f41480753d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4154046630 tx=0x7f4154009270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.060+0000 7f4166800640 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564e8d6655b0 0x564e8d6659c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.061+0000 7f4166800640 1 -- 172.21.15.19:0/3975818241 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4160006d30 con 0x564e8d6655b0 2024-04-25T11:35:31.049 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:31.050 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564e8d6655b0 msgr2=0x564e8d6659c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564e8d6655b0 0x564e8d6659c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4148072ec0 msgr2=0x7f41480753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4148072ec0 0x7f41480753d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4154046630 tx=0x7f4154009270 comp rx=0 tx=0).stop 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d5d8fe0 msgr2=0x564e8d651de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 --2- 172.21.15.19:0/3975818241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564e8d5d8fe0 0x564e8d651de0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4158009e90 tx=0x7f4158009f80 comp rx=0 tx=0).stop 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f4166800640 1 -- 172.21.15.19:0/3975818241 reap_dead start 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 shutdown_connections 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 >> 172.21.15.19:0/3975818241 conn(0x564e8d4b0ac0 msgr2=0x564e8d5d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 shutdown_connections 2024-04-25T11:35:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.062+0000 7f416d71ef40 1 -- 172.21.15.19:0/3975818241 wait complete. 2024-04-25T11:35:31.065 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2024-04-25T11:35:31.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.263+0000 7f34be197f40 1 Processor -- start 2024-04-25T11:35:31.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.263+0000 7f34be197f40 1 -- start start 2024-04-25T11:35:31.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.264+0000 7f34be197f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750d911e0 0x563750e5dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.264+0000 7f34be197f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d915f0 0x563750d8f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.264+0000 7f34be197f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750e6add0 0x563750d8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.264+0000 7f34be197f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563750dd9e70 con 0x563750e6add0 2024-04-25T11:35:31.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.264+0000 7f34be197f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563750d37d20 con 0x563750d915f0 2024-04-25T11:35:31.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.264+0000 7f34be197f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563750dda3f0 con 0x563750d911e0 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750e6add0 0x563750d8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750e6add0 0x563750d8faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41500/0 (socket says 172.21.15.19:41500) 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 -- 172.21.15.19:0/3712836880 learned_addr learned my addr 172.21.15.19:0/3712836880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7400640 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750d911e0 0x563750e5dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34afe00640 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d915f0 0x563750d8f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 -- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750d911e0 msgr2=0x563750e5dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750d911e0 0x563750e5dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 -- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d915f0 msgr2=0x563750d8f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d915f0 0x563750d8f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34b7e00640 1 -- 172.21.15.19:0/3712836880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563750dc3180 con 0x563750e6add0 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.265+0000 7f34afe00640 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d915f0 0x563750d8f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.266+0000 7f34b7e00640 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750e6add0 0x563750d8faa0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f34b00015c0 tx=0x7f34b0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8939b7c733bc4cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.266+0000 7f34b6000640 1 -- 172.21.15.19:0/3712836880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0003650 con 0x563750e6add0 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.266+0000 7f34b6000640 1 -- 172.21.15.19:0/3712836880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b00037f0 con 0x563750e6add0 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.266+0000 7f34b6000640 1 -- 172.21.15.19:0/3712836880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0003c20 con 0x563750e6add0 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 -- 172.21.15.19:0/3712836880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750e6add0 msgr2=0x563750d8faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750e6add0 0x563750d8faa0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f34b00015c0 tx=0x7f34b0001a80 comp rx=0 tx=0).stop 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 -- 172.21.15.19:0/3712836880 shutdown_connections 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d915f0 0x563750d8f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750e6add0 0x563750d8faa0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 --2- 172.21.15.19:0/3712836880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750d911e0 0x563750e5dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 -- 172.21.15.19:0/3712836880 >> 172.21.15.19:0/3712836880 conn(0x563750cb3ac0 msgr2=0x563750d90530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 -- 172.21.15.19:0/3712836880 shutdown_connections 2024-04-25T11:35:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.267+0000 7f34be197f40 1 -- 172.21.15.19:0/3712836880 wait complete. 2024-04-25T11:35:31.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.268+0000 7f34be197f40 1 Processor -- start 2024-04-25T11:35:31.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.268+0000 7f34be197f40 1 -- start start 2024-04-25T11:35:31.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.269+0000 7f34be197f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d911e0 0x563750ef7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.269+0000 7f34be197f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750d915f0 0x563750e5f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34be197f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750e6add0 0x563750e5fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34be197f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563750e62960 con 0x563750d915f0 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34be197f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563750e605a0 con 0x563750d911e0 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34be197f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563750d8f880 con 0x563750e6add0 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750d915f0 0x563750e5f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750d915f0 0x563750e5f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41514/0 (socket says 172.21.15.19:41514) 2024-04-25T11:35:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 -- 172.21.15.19:0/3132730417 learned_addr learned my addr 172.21.15.19:0/3132730417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34b7e00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750e6add0 0x563750e5fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34b7400640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d911e0 0x563750ef7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 -- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750e6add0 msgr2=0x563750e5fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750e6add0 0x563750e5fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 -- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d911e0 msgr2=0x563750ef7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563750d911e0 0x563750ef7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34afe00640 1 -- 172.21.15.19:0/3132730417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563750ef7360 con 0x563750d915f0 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.270+0000 7f34b7e00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563750e6add0 0x563750e5fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.271+0000 7f34afe00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750d915f0 0x563750e5f810 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f34a000be30 tx=0x7f34a000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.271+0000 7f34b4c00640 1 -- 172.21.15.19:0/3132730417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0007b50 con 0x563750d915f0 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.271+0000 7f34b4c00640 1 -- 172.21.15.19:0/3132730417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0007cf0 con 0x563750d915f0 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.271+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563750e5d160 con 0x563750d915f0 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.271+0000 7f34b4c00640 1 -- 172.21.15.19:0/3132730417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0015640 con 0x563750d915f0 2024-04-25T11:35:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.271+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563750e69230 con 0x563750d915f0 2024-04-25T11:35:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.273+0000 7f34b4c00640 1 -- 172.21.15.19:0/3132730417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34a001e440 con 0x563750d915f0 2024-04-25T11:35:31.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.274+0000 7f34b4c00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3494072e90 0x7f34940753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.275+0000 7f34b4c00640 1 -- 172.21.15.19:0/3132730417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f34a00934f0 con 0x563750d915f0 2024-04-25T11:35:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.275+0000 7f34be197f40 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563750e5d370 0x563750ef62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.275+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x563750ef8f00 con 0x563750e5d370 2024-04-25T11:35:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.275+0000 7f34b7400640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3494072e90 0x7f34940753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.276+0000 7f34b7e00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563750e5d370 0x563750ef62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.277+0000 7f34b7400640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3494072e90 0x7f34940753a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f34a8056000 tx=0x7f34a8050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.277+0000 7f34b7e00640 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563750e5d370 0x563750ef62b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34b7e00640 1 -- 172.21.15.19:0/3132730417 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f34b000f030 con 0x563750e5d370 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:header (274 bytes) : 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-25T11:35:31.266 INFO:teuthology.orchestra.run.smithi019.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:000000a0 00 00 00 00 00 00 00 00 f2 3f 2a 66 15 13 3d 18 |.........?*f..=.| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:000000e0 00 00 00 00 00 00 f2 3f 2a 66 15 13 3d 18 00 00 |.......?*f..=...| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:* 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:00000110 00 00 |..| 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout:00000112 2024-04-25T11:35:31.267 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563750e5d370 msgr2=0x563750ef62b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563750e5d370 0x563750ef62b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3494072e90 msgr2=0x7f34940753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3494072e90 0x7f34940753a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f34a8056000 tx=0x7f34a8050000 comp rx=0 tx=0).stop 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750d915f0 msgr2=0x563750e5f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 --2- 172.21.15.19:0/3132730417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563750d915f0 0x563750e5f810 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f34a000be30 tx=0x7f34a000bf20 comp rx=0 tx=0).stop 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34b7e00640 1 -- 172.21.15.19:0/3132730417 reap_dead start 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 shutdown_connections 2024-04-25T11:35:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 >> 172.21.15.19:0/3132730417 conn(0x563750cb3ac0 msgr2=0x563750d8fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 shutdown_connections 2024-04-25T11:35:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.278+0000 7f34be197f40 1 -- 172.21.15.19:0/3132730417 wait complete. 2024-04-25T11:35:31.278 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2024-04-25T11:35:31.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.455+0000 7faf51926f40 1 Processor -- start 2024-04-25T11:35:31.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.455+0000 7faf51926f40 1 -- start start 2024-04-25T11:35:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.456+0000 7faf51926f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd231e0 0x56022dd551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.456+0000 7faf51926f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd235f0 0x56022dd542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.456+0000 7faf51926f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.456+0000 7faf51926f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56022dd6be70 con 0x56022ddfcfa0 2024-04-25T11:35:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.456+0000 7faf51926f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56022dcc9d20 con 0x56022dd235f0 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.456+0000 7faf51926f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56022dd6c3f0 con 0x56022dd231e0 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.456+0000 7faf4aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd22180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41536/0 (socket says 172.21.15.19:41536) 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 -- 172.21.15.19:0/2817410267 learned_addr learned my addr 172.21.15.19:0/2817410267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4a000640 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd231e0 0x56022dd551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf49600640 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd235f0 0x56022dd542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 -- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd231e0 msgr2=0x56022dd551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd231e0 0x56022dd551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 -- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd235f0 msgr2=0x56022dd542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd235f0 0x56022dd542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 -- 172.21.15.19:0/2817410267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022ddf6670 con 0x56022ddfcfa0 2024-04-25T11:35:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.457+0000 7faf4aa00640 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd22180 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7faf4400a3c0 tx=0x7faf4400a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38110b8f6dd882ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf43e00640 1 -- 172.21.15.19:0/2817410267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400b8d0 con 0x56022ddfcfa0 2024-04-25T11:35:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf43e00640 1 -- 172.21.15.19:0/2817410267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4400f030 con 0x56022ddfcfa0 2024-04-25T11:35:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf43e00640 1 -- 172.21.15.19:0/2817410267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400bc90 con 0x56022ddfcfa0 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 -- 172.21.15.19:0/2817410267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 msgr2=0x56022dd22180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd22180 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7faf4400a3c0 tx=0x7faf4400a880 comp rx=0 tx=0).stop 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 -- 172.21.15.19:0/2817410267 shutdown_connections 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd235f0 0x56022dd542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd22180 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 --2- 172.21.15.19:0/2817410267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd231e0 0x56022dd551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 -- 172.21.15.19:0/2817410267 >> 172.21.15.19:0/2817410267 conn(0x56022dc45ac0 msgr2=0x56022dd70720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.458+0000 7faf51926f40 1 -- 172.21.15.19:0/2817410267 shutdown_connections 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.459+0000 7faf51926f40 1 -- 172.21.15.19:0/2817410267 wait complete. 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.460+0000 7faf51926f40 1 Processor -- start 2024-04-25T11:35:31.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.460+0000 7faf51926f40 1 -- start start 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.461+0000 7faf51926f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd231e0 0x56022ddf4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.461+0000 7faf51926f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd235f0 0x56022dd69710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.461+0000 7faf51926f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd54db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.461+0000 7faf51926f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56022ddf9bf0 con 0x56022ddfcfa0 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.461+0000 7faf51926f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56022ddef730 con 0x56022dd231e0 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.461+0000 7faf51926f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56022de91790 con 0x56022dd235f0 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.461+0000 7faf4aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd54db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41540/0 (socket says 172.21.15.19:41540) 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 -- 172.21.15.19:0/564557004 learned_addr learned my addr 172.21.15.19:0/564557004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf49600640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd235f0 0x56022dd69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 -- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd235f0 msgr2=0x56022dd69710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4a000640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd231e0 0x56022ddf4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56022dd235f0 0x56022dd69710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 -- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd231e0 msgr2=0x56022ddf4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56022dd231e0 0x56022ddf4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 -- 172.21.15.19:0/564557004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56022dd70860 con 0x56022ddfcfa0 2024-04-25T11:35:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.462+0000 7faf4aa00640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd54db0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7faf4400f000 tx=0x7faf44002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.463+0000 7faf42a00640 1 -- 172.21.15.19:0/564557004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44024070 con 0x56022ddfcfa0 2024-04-25T11:35:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.463+0000 7faf42a00640 1 -- 172.21.15.19:0/564557004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf44014630 con 0x56022ddfcfa0 2024-04-25T11:35:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.463+0000 7faf42a00640 1 -- 172.21.15.19:0/564557004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44014a00 con 0x56022ddfcfa0 2024-04-25T11:35:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.463+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56022ddf2000 con 0x56022ddfcfa0 2024-04-25T11:35:31.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.463+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56022ddf25c0 con 0x56022ddfcfa0 2024-04-25T11:35:31.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.465+0000 7faf42a00640 1 -- 172.21.15.19:0/564557004 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf44027070 con 0x56022ddfcfa0 2024-04-25T11:35:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.466+0000 7faf42a00640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf2c072ec0 0x7faf2c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.466+0000 7faf42a00640 1 -- 172.21.15.19:0/564557004 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7faf44094f20 con 0x56022ddfcfa0 2024-04-25T11:35:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.467+0000 7faf51926f40 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56022ddf03d0 0x56022ddf07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.467+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x56022ddf0d20 con 0x56022ddf03d0 2024-04-25T11:35:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.467+0000 7faf4a000640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf2c072ec0 0x7faf2c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.467+0000 7faf49600640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56022ddf03d0 0x56022ddf07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.469+0000 7faf4a000640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf2c072ec0 0x7faf2c0753d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faf3c056000 tx=0x7faf3c007030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.469+0000 7faf49600640 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56022ddf03d0 0x56022ddf07e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf49600640 1 -- 172.21.15.19:0/564557004 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7faf3800c640 con 0x56022ddf03d0 2024-04-25T11:35:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:header (14 bytes) : 2024-04-25T11:35:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2024-04-25T11:35:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:0000000e 2024-04-25T11:35:31.458 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56022ddf03d0 msgr2=0x56022ddf07e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56022ddf03d0 0x56022ddf07e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf2c072ec0 msgr2=0x7faf2c0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faf2c072ec0 0x7faf2c0753d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faf3c056000 tx=0x7faf3c007030 comp rx=0 tx=0).stop 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 msgr2=0x56022dd54db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 --2- 172.21.15.19:0/564557004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56022ddfcfa0 0x56022dd54db0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7faf4400f000 tx=0x7faf44002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf4aa00640 1 -- 172.21.15.19:0/564557004 reap_dead start 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 shutdown_connections 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 >> 172.21.15.19:0/564557004 conn(0x56022dc45ac0 msgr2=0x56022dd70b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 shutdown_connections 2024-04-25T11:35:31.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.470+0000 7faf51926f40 1 -- 172.21.15.19:0/564557004 wait complete. 2024-04-25T11:35:31.469 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2024-04-25T11:35:31.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.660+0000 7f05fab1af40 1 Processor -- start 2024-04-25T11:35:31.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.661+0000 7f05fab1af40 1 -- start start 2024-04-25T11:35:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05fab1af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64811e0 0x556dc654dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05fab1af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556dc64815f0 0x556dc64cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05fab1af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc64ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05fab1af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556dc64c9e70 con 0x556dc655add0 2024-04-25T11:35:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05fab1af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556dc6427d20 con 0x556dc64815f0 2024-04-25T11:35:31.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05fab1af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556dc64ca110 con 0x556dc64811e0 2024-04-25T11:35:31.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc64ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc64ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41568/0 (socket says 172.21.15.19:41568) 2024-04-25T11:35:31.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.662+0000 7f05f3e00640 1 -- 172.21.15.19:0/3391985243 learned_addr learned my addr 172.21.15.19:0/3391985243 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f3e00640 1 -- 172.21.15.19:0/3391985243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64811e0 msgr2=0x556dc654dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f3e00640 1 --2- 172.21.15.19:0/3391985243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64811e0 0x556dc654dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f3e00640 1 -- 172.21.15.19:0/3391985243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556dc64815f0 msgr2=0x556dc64cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f3e00640 1 --2- 172.21.15.19:0/3391985243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556dc64815f0 0x556dc64cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f3e00640 1 -- 172.21.15.19:0/3391985243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc65552e0 con 0x556dc655add0 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f3e00640 1 --2- 172.21.15.19:0/3391985243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc64ce300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f05ec025ca0 tx=0x7f05ec051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457405baf279a6e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f1600640 1 -- 172.21.15.19:0/3391985243 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec062070 con 0x556dc655add0 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.663+0000 7f05f1600640 1 -- 172.21.15.19:0/3391985243 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ec054970 con 0x556dc655add0 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05f1600640 1 -- 172.21.15.19:0/3391985243 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec054d20 con 0x556dc655add0 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 -- 172.21.15.19:0/3391985243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 msgr2=0x556dc64ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 --2- 172.21.15.19:0/3391985243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc64ce300 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f05ec025ca0 tx=0x7f05ec051000 comp rx=0 tx=0).stop 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 -- 172.21.15.19:0/3391985243 shutdown_connections 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 --2- 172.21.15.19:0/3391985243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556dc64815f0 0x556dc64cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 --2- 172.21.15.19:0/3391985243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc64ce300 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 --2- 172.21.15.19:0/3391985243 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64811e0 0x556dc654dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 -- 172.21.15.19:0/3391985243 >> 172.21.15.19:0/3391985243 conn(0x556dc63a3ac0 msgr2=0x556dc64b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 -- 172.21.15.19:0/3391985243 shutdown_connections 2024-04-25T11:35:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.664+0000 7f05fab1af40 1 -- 172.21.15.19:0/3391985243 wait complete. 2024-04-25T11:35:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.665+0000 7f05fab1af40 1 Processor -- start 2024-04-25T11:35:31.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.666+0000 7f05fab1af40 1 -- start start 2024-04-25T11:35:31.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05fab1af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556dc64811e0 0x556dc654fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05fab1af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64815f0 0x556dc65424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05fab1af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc6542a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05fab1af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556dc65575a0 con 0x556dc655add0 2024-04-25T11:35:31.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05fab1af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556dc6555030 con 0x556dc64811e0 2024-04-25T11:35:31.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05fab1af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556dc6556ca0 con 0x556dc64815f0 2024-04-25T11:35:31.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc6542a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc6542a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41584/0 (socket says 172.21.15.19:41584) 2024-04-25T11:35:31.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.667+0000 7f05f3e00640 1 -- 172.21.15.19:0/2375448779 learned_addr learned my addr 172.21.15.19:0/2375448779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05f2a00640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64815f0 0x556dc65424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05f3e00640 1 -- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64815f0 msgr2=0x556dc65424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05f3e00640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556dc64815f0 0x556dc65424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05f3e00640 1 -- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556dc64811e0 msgr2=0x556dc654fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05f3e00640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556dc64811e0 0x556dc654fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05f3e00640 1 -- 172.21.15.19:0/2375448779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc6557840 con 0x556dc655add0 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05f3e00640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc6542a20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f05ec025ca0 tx=0x7f05ec002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05ebe00640 1 -- 172.21.15.19:0/2375448779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec062040 con 0x556dc655add0 2024-04-25T11:35:31.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05ebe00640 1 -- 172.21.15.19:0/2375448779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05ec06f070 con 0x556dc655add0 2024-04-25T11:35:31.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05ebe00640 1 -- 172.21.15.19:0/2375448779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec067650 con 0x556dc655add0 2024-04-25T11:35:31.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.668+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556dc6550990 con 0x556dc655add0 2024-04-25T11:35:31.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.669+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc6542fb0 con 0x556dc655add0 2024-04-25T11:35:31.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.671+0000 7f05ebe00640 1 -- 172.21.15.19:0/2375448779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f05ec065070 con 0x556dc655add0 2024-04-25T11:35:31.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.672+0000 7f05ebe00640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05d4072ec0 0x7f05d40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.672+0000 7f05ebe00640 1 -- 172.21.15.19:0/2375448779 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f05ec0d89b0 con 0x556dc655add0 2024-04-25T11:35:31.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.672+0000 7f05fab1af40 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556dc6544bf0 0x556dc64ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.672+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x556dc6554aa0 con 0x556dc6544bf0 2024-04-25T11:35:31.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.672+0000 7f05f2a00640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556dc6544bf0 0x556dc64ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.672+0000 7f05f3400640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05d4072ec0 0x7f05d40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.673+0000 7f05f2a00640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556dc6544bf0 0x556dc64ce2d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05f2a00640 1 -- 172.21.15.19:0/2375448779 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f05e000b680 con 0x556dc6544bf0 2024-04-25T11:35:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05f3400640 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05d4072ec0 0x7f05d40753d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006fb0 tx=0x7f05e4006640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556dc6544bf0 msgr2=0x556dc64ce2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556dc6544bf0 0x556dc64ce2d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05d4072ec0 msgr2=0x7f05d40753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05d4072ec0 0x7f05d40753d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f05e4006fb0 tx=0x7f05e4006640 comp rx=0 tx=0).stop 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 msgr2=0x556dc6542a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 --2- 172.21.15.19:0/2375448779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556dc655add0 0x556dc6542a20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f05ec025ca0 tx=0x7f05ec002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05f3e00640 1 -- 172.21.15.19:0/2375448779 reap_dead start 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 shutdown_connections 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 >> 172.21.15.19:0/2375448779 conn(0x556dc63a3ac0 msgr2=0x556dc6551180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 shutdown_connections 2024-04-25T11:35:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.674+0000 7f05fab1af40 1 -- 172.21.15.19:0/2375448779 wait complete. 2024-04-25T11:35:31.677 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2024-04-25T11:35:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.865+0000 7fd9d01b7f40 1 Processor -- start 2024-04-25T11:35:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.865+0000 7fd9d01b7f40 1 -- start start 2024-04-25T11:35:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9d01b7f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b00030f0 0x5624226bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9d01b7f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242275b840 0x56242275bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9d01b7f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275c520 0x56242275c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9d01b7f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562422635d20 con 0x56242275b840 2024-04-25T11:35:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9d01b7f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562422639d60 con 0x7fd9b00030f0 2024-04-25T11:35:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9d01b7f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5624226d85c0 con 0x56242275c520 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242275b840 0x56242275bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242275b840 0x56242275bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41600/0 (socket says 172.21.15.19:41600) 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9c3e00640 1 -- 172.21.15.19:0/3486868072 learned_addr learned my addr 172.21.15.19:0/3486868072 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9c9e00640 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275c520 0x56242275c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9c9400640 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b00030f0 0x5624226bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9c3e00640 1 -- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275c520 msgr2=0x56242275c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.867+0000 7fd9c3e00640 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275c520 0x56242275c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.868+0000 7fd9c3e00640 1 -- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b00030f0 msgr2=0x5624226bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.868+0000 7fd9c3e00640 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b00030f0 0x5624226bcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.868+0000 7fd9c3e00640 1 -- 172.21.15.19:0/3486868072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56242268d890 con 0x56242275b840 2024-04-25T11:35:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.868+0000 7fd9c3e00640 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242275b840 0x56242275bc50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4045e40 tx=0x7fd9b404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cc110921b79505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.868+0000 7fd9c3400640 1 -- 172.21.15.19:0/3486868072 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4059040 con 0x56242275b840 2024-04-25T11:35:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.868+0000 7fd9c3400640 1 -- 172.21.15.19:0/3486868072 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b4063070 con 0x56242275b840 2024-04-25T11:35:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.868+0000 7fd9c3400640 1 -- 172.21.15.19:0/3486868072 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40549a0 con 0x56242275b840 2024-04-25T11:35:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/3486868072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242275b840 msgr2=0x56242275bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242275b840 0x56242275bc50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4045e40 tx=0x7fd9b404e000 comp rx=0 tx=0).stop 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/3486868072 shutdown_connections 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275c520 0x56242275c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56242275b840 0x56242275bc50 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/3486868072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9b00030f0 0x5624226bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/3486868072 >> 172.21.15.19:0/3486868072 conn(0x5624225b1ac0 msgr2=0x562422761550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/3486868072 shutdown_connections 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.869+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/3486868072 wait complete. 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.870+0000 7fd9d01b7f40 1 Processor -- start 2024-04-25T11:35:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.870+0000 7fd9d01b7f40 1 -- start start 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9d01b7f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275b840 0x56242275af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9d01b7f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56242275c520 0x5624227662f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9d01b7f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9b00030f0 0x5624226bce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9d01b7f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562422764c70 con 0x7fd9b00030f0 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9d01b7f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5624227646b0 con 0x56242275c520 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9d01b7f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562422764990 con 0x56242275b840 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9b00030f0 0x5624226bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9b00030f0 0x5624226bce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41608/0 (socket says 172.21.15.19:41608) 2024-04-25T11:35:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9c9e00640 1 -- 172.21.15.19:0/734085184 learned_addr learned my addr 172.21.15.19:0/734085184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9c3e00640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56242275c520 0x5624227662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.872+0000 7fd9c9400640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275b840 0x56242275af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c9e00640 1 -- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275b840 msgr2=0x56242275af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c9e00640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275b840 0x56242275af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c9e00640 1 -- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56242275c520 msgr2=0x5624227662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c9e00640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56242275c520 0x5624227662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c9e00640 1 -- 172.21.15.19:0/734085184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562422766830 con 0x7fd9b00030f0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c9400640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56242275b840 0x56242275af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c9e00640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9b00030f0 0x5624226bce50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400ee40 tx=0x7fd9c400c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c2000640 1 -- 172.21.15.19:0/734085184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4017070 con 0x7fd9b00030f0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c2000640 1 -- 172.21.15.19:0/734085184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c4013480 con 0x7fd9b00030f0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9c2000640 1 -- 172.21.15.19:0/734085184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4013870 con 0x7fd9b00030f0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.873+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5624226d47b0 con 0x7fd9b00030f0 2024-04-25T11:35:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.874+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624226d4f90 con 0x7fd9b00030f0 2024-04-25T11:35:31.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.876+0000 7fd9c2000640 1 -- 172.21.15.19:0/734085184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9c401a070 con 0x7fd9b00030f0 2024-04-25T11:35:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.877+0000 7fd9c2000640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9a8072ec0 0x7fd9a80753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.877+0000 7fd9c2000640 1 -- 172.21.15.19:0/734085184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fd9c40941c0 con 0x7fd9b00030f0 2024-04-25T11:35:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.877+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56242275e490 0x56242275ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.877+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x562422766a00 con 0x56242275e490 2024-04-25T11:35:31.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.878+0000 7fd9c9400640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9a8072ec0 0x7fd9a80753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.878+0000 7fd9c3e00640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56242275e490 0x56242275ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.878+0000 7fd9c3e00640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56242275e490 0x56242275ce40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9c9400640 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9a8072ec0 0x7fd9a80753d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0017e0 tx=0x7fd9bc000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9c3e00640 1 -- 172.21.15.19:0/734085184 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd9b405c040 con 0x56242275e490 2024-04-25T11:35:31.868 INFO:teuthology.orchestra.run.smithi019.stdout:header (0 bytes) : 2024-04-25T11:35:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56242275e490 msgr2=0x56242275ce40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56242275e490 0x56242275ce40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9a8072ec0 msgr2=0x7fd9a80753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9a8072ec0 0x7fd9a80753d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0017e0 tx=0x7fd9bc000ed0 comp rx=0 tx=0).stop 2024-04-25T11:35:31.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9b00030f0 msgr2=0x5624226bce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 --2- 172.21.15.19:0/734085184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9b00030f0 0x5624226bce50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400ee40 tx=0x7fd9c400c530 comp rx=0 tx=0).stop 2024-04-25T11:35:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9c9e00640 1 -- 172.21.15.19:0/734085184 reap_dead start 2024-04-25T11:35:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 shutdown_connections 2024-04-25T11:35:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 >> 172.21.15.19:0/734085184 conn(0x5624225b1ac0 msgr2=0x56242268f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 shutdown_connections 2024-04-25T11:35:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:31.879+0000 7fd9d01b7f40 1 -- 172.21.15.19:0/734085184 wait complete. 2024-04-25T11:35:31.879 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2024-04-25T11:35:32.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.075+0000 7f09af995f40 1 Processor -- start 2024-04-25T11:35:32.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.075+0000 7f09af995f40 1 -- start start 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09af995f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc881e0 0x56137bd5e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09af995f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc885f0 0x56137bd5df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09af995f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bccea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09af995f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56137bcd0e70 con 0x56137bd61dd0 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09af995f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56137bc2ed20 con 0x56137bc885f0 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09af995f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56137bcd1110 con 0x56137bc881e0 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09a9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bccea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09a9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bccea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41622/0 (socket says 172.21.15.19:41622) 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09a9600640 1 -- 172.21.15.19:0/1236984854 learned_addr learned my addr 172.21.15.19:0/1236984854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09a8c00640 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc881e0 0x56137bd5e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.077+0000 7f09a8200640 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc885f0 0x56137bd5df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a9600640 1 -- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc881e0 msgr2=0x56137bd5e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a9600640 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc881e0 0x56137bd5e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a9600640 1 -- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc885f0 msgr2=0x56137bd5df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a9600640 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc885f0 0x56137bd5df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a9600640 1 -- 172.21.15.19:0/1236984854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137bccdf60 con 0x56137bd61dd0 2024-04-25T11:35:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a9600640 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bccea20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f09a000b6f0 tx=0x7f09a000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7d21314e28af2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a6e00640 1 -- 172.21.15.19:0/1236984854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0016020 con 0x56137bd61dd0 2024-04-25T11:35:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a6e00640 1 -- 172.21.15.19:0/1236984854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a000be00 con 0x56137bd61dd0 2024-04-25T11:35:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.078+0000 7f09a6e00640 1 -- 172.21.15.19:0/1236984854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a000dc00 con 0x56137bd61dd0 2024-04-25T11:35:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 -- 172.21.15.19:0/1236984854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 msgr2=0x56137bccea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bccea20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f09a000b6f0 tx=0x7f09a000bbb0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 -- 172.21.15.19:0/1236984854 shutdown_connections 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc885f0 0x56137bd5df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bccea20 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 --2- 172.21.15.19:0/1236984854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc881e0 0x56137bd5e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 -- 172.21.15.19:0/1236984854 >> 172.21.15.19:0/1236984854 conn(0x56137bbaaac0 msgr2=0x56137bcd2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 -- 172.21.15.19:0/1236984854 shutdown_connections 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.079+0000 7f09af995f40 1 -- 172.21.15.19:0/1236984854 wait complete. 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.080+0000 7f09af995f40 1 Processor -- start 2024-04-25T11:35:32.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.081+0000 7f09af995f40 1 -- start start 2024-04-25T11:35:32.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09af995f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc881e0 0x56137bc87440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09af995f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc885f0 0x56137bde8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09af995f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bde85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09af995f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56137bcce590 con 0x56137bd61dd0 2024-04-25T11:35:32.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09af995f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56137bcd3d90 con 0x56137bc881e0 2024-04-25T11:35:32.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09af995f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56137bdeaef0 con 0x56137bc885f0 2024-04-25T11:35:32.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09a9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bde85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09a9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bde85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41634/0 (socket says 172.21.15.19:41634) 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.082+0000 7f09a9600640 1 -- 172.21.15.19:0/1815456036 learned_addr learned my addr 172.21.15.19:0/1815456036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a8200640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc885f0 0x56137bde8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a8c00640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc881e0 0x56137bc87440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a9600640 1 -- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc885f0 msgr2=0x56137bde8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a9600640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56137bc885f0 0x56137bde8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a9600640 1 -- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc881e0 msgr2=0x56137bc87440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a9600640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc881e0 0x56137bc87440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a9600640 1 -- 172.21.15.19:0/1815456036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137bcd3350 con 0x56137bd61dd0 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a8c00640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56137bc881e0 0x56137bc87440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.083+0000 7f09a9600640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bde85b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x56137bcb96e0 tx=0x7f09a00033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.084+0000 7f09a5a00640 1 -- 172.21.15.19:0/1815456036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0021070 con 0x56137bd61dd0 2024-04-25T11:35:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.084+0000 7f09a5a00640 1 -- 172.21.15.19:0/1815456036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09a0013c60 con 0x56137bd61dd0 2024-04-25T11:35:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.084+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56137bcceb30 con 0x56137bd61dd0 2024-04-25T11:35:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.084+0000 7f09a5a00640 1 -- 172.21.15.19:0/1815456036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a001b660 con 0x56137bd61dd0 2024-04-25T11:35:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.084+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137bd5d900 con 0x56137bd61dd0 2024-04-25T11:35:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.086+0000 7f09a5a00640 1 -- 172.21.15.19:0/1815456036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09a0019020 con 0x56137bd61dd0 2024-04-25T11:35:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.087+0000 7f09a5a00640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0988072e90 0x7f09880753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.087+0000 7f09a5a00640 1 -- 172.21.15.19:0/1815456036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f09a0095a00 con 0x56137bd61dd0 2024-04-25T11:35:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.088+0000 7f09af995f40 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56137bdf8e70 0x56137bdf9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.088+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x56137bc87980 con 0x56137bdf8e70 2024-04-25T11:35:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.088+0000 7f09a8c00640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0988072e90 0x7f09880753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.088+0000 7f09a8200640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56137bdf8e70 0x56137bdf9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.089+0000 7f09a8200640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56137bdf8e70 0x56137bdf9280 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.089+0000 7f09a8200640 1 -- 172.21.15.19:0/1815456036 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f0994054970 con 0x56137bdf8e70 2024-04-25T11:35:32.077 INFO:teuthology.orchestra.run.smithi019.stdout:header (274 bytes) : 2024-04-25T11:35:32.077 INFO:teuthology.orchestra.run.smithi019.stdout:00000000 04 03 0c 01 00 00 0d 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:32.077 INFO:teuthology.orchestra.run.smithi019.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-04-25T11:35:32.077 INFO:teuthology.orchestra.run.smithi019.stdout:00000020 00 00 00 00 fb 3f 2a 66 ef 97 ba 23 00 00 00 00 |.....?*f...#....| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:00000050 00 00 fb 3f 2a 66 ef 97 ba 23 00 00 00 00 00 00 |...?*f...#......| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:000000a0 00 00 00 00 00 00 00 00 fb 3f 2a 66 9c f0 06 2d |.........?*f...-| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 |`...............| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:000000e0 00 00 00 00 00 00 fb 3f 2a 66 9c f0 06 2d 00 00 |.......?*f...-..| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-04-25T11:35:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:* 2024-04-25T11:35:32.079 INFO:teuthology.orchestra.run.smithi019.stdout:00000110 00 00 |..| 2024-04-25T11:35:32.079 INFO:teuthology.orchestra.run.smithi019.stdout:00000112 2024-04-25T11:35:32.079 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.089+0000 7f09a8c00640 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0988072e90 0x7f09880753a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0998002f10 tx=0x7f0998002510 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.089+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56137bdf8e70 msgr2=0x56137bdf9280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.089+0000 7f09af995f40 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56137bdf8e70 0x56137bdf9280 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0988072e90 msgr2=0x7f09880753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0988072e90 0x7f09880753a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0998002f10 tx=0x7f0998002510 comp rx=0 tx=0).stop 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 msgr2=0x56137bde85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 --2- 172.21.15.19:0/1815456036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56137bd61dd0 0x56137bde85b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x56137bcb96e0 tx=0x7f09a00033a0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09a9600640 1 -- 172.21.15.19:0/1815456036 reap_dead start 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 shutdown_connections 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 >> 172.21.15.19:0/1815456036 conn(0x56137bbaaac0 msgr2=0x56137bcb6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 shutdown_connections 2024-04-25T11:35:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.090+0000 7f09af995f40 1 -- 172.21.15.19:0/1815456036 wait complete. 2024-04-25T11:35:32.088 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-04-25T11:35:32.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.269+0000 7ff9e5d26f40 1 Processor -- start 2024-04-25T11:35:32.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.270+0000 7ff9e5d26f40 1 -- start start 2024-04-25T11:35:32.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.271+0000 7ff9e5d26f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8101e0 0x55758d8dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.271+0000 7ff9e5d26f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8105f0 0x55758d80ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.271+0000 7ff9e5d26f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8e9fa0 0x55758d80f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.271+0000 7ff9e5d26f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55758d7b6d20 con 0x55758d8105f0 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.271+0000 7ff9e5d26f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55758d7bad60 con 0x55758d8e9fa0 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.271+0000 7ff9e5d26f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55758d8593f0 con 0x55758d8101e0 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8105f0 0x55758d80ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8105f0 0x55758d80ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41646/0 (socket says 172.21.15.19:41646) 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 -- 172.21.15.19:0/2258135571 learned_addr learned my addr 172.21.15.19:0/2258135571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9de400640 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8101e0 0x55758d8dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 -- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8101e0 msgr2=0x55758d8dcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9dee00640 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8e9fa0 0x55758d80f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8101e0 0x55758d8dcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 -- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8e9fa0 msgr2=0x55758d80f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8e9fa0 0x55758d80f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9d5a00640 1 -- 172.21.15.19:0/2258135571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758d83f850 con 0x55758d8105f0 2024-04-25T11:35:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.272+0000 7ff9dee00640 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8e9fa0 0x55758d80f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.273+0000 7ff9d5a00640 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8105f0 0x55758d80ef90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b240 tx=0x7ff9c800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a848301795b13a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.273+0000 7ff9dd000640 1 -- 172.21.15.19:0/2258135571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8008070 con 0x55758d8105f0 2024-04-25T11:35:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.273+0000 7ff9dd000640 1 -- 172.21.15.19:0/2258135571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c800d500 con 0x55758d8105f0 2024-04-25T11:35:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.273+0000 7ff9dd000640 1 -- 172.21.15.19:0/2258135571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800d8d0 con 0x55758d8105f0 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.273+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2258135571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8105f0 msgr2=0x55758d80ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.273+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8105f0 0x55758d80ef90 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b240 tx=0x7ff9c800b700 comp rx=0 tx=0).stop 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.274+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2258135571 shutdown_connections 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.274+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8e9fa0 0x55758d80f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.274+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8105f0 0x55758d80ef90 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.274+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2258135571 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8101e0 0x55758d8dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.274+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2258135571 >> 172.21.15.19:0/2258135571 conn(0x55758d732ac0 msgr2=0x55758d8e6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.274+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2258135571 shutdown_connections 2024-04-25T11:35:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.274+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2258135571 wait complete. 2024-04-25T11:35:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.275+0000 7ff9e5d26f40 1 Processor -- start 2024-04-25T11:35:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.275+0000 7ff9e5d26f40 1 -- start start 2024-04-25T11:35:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.276+0000 7ff9e5d26f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8101e0 0x55758d8dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.276+0000 7ff9e5d26f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8105f0 0x55758d8e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.276+0000 7ff9e5d26f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8e9fa0 0x55758d8e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.276+0000 7ff9e5d26f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55758d841d20 con 0x55758d8101e0 2024-04-25T11:35:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.276+0000 7ff9e5d26f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55758d8420a0 con 0x55758d8105f0 2024-04-25T11:35:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.276+0000 7ff9e5d26f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55758d842380 con 0x55758d8e9fa0 2024-04-25T11:35:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8101e0 0x55758d8dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8101e0 0x55758d8dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41658/0 (socket says 172.21.15.19:41658) 2024-04-25T11:35:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 -- 172.21.15.19:0/2425512159 learned_addr learned my addr 172.21.15.19:0/2425512159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9dee00640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8e9fa0 0x55758d8e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9d5a00640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8105f0 0x55758d8e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 -- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8e9fa0 msgr2=0x55758d8e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55758d8e9fa0 0x55758d8e1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 -- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8105f0 msgr2=0x55758d8e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55758d8105f0 0x55758d8e17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.277+0000 7ff9de400640 1 -- 172.21.15.19:0/2425512159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758d8e0000 con 0x55758d8101e0 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.278+0000 7ff9de400640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8101e0 0x55758d8dbdd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c00 tx=0x7ff9d0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.278+0000 7ff9d7400640 1 -- 172.21.15.19:0/2425512159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0054b50 con 0x55758d8101e0 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.278+0000 7ff9d7400640 1 -- 172.21.15.19:0/2425512159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d0054cf0 con 0x55758d8101e0 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.278+0000 7ff9d7400640 1 -- 172.21.15.19:0/2425512159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0068650 con 0x55758d8101e0 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.278+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55758d8dfdf0 con 0x55758d8101e0 2024-04-25T11:35:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.278+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758d856ed0 con 0x55758d8101e0 2024-04-25T11:35:32.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.281+0000 7ff9d7400640 1 -- 172.21.15.19:0/2425512159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9d0071440 con 0x55758d8101e0 2024-04-25T11:35:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.281+0000 7ff9d7400640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc072ec0 0x7ff9bc0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.282+0000 7ff9d7400640 1 -- 172.21.15.19:0/2425512159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7ff9d00d8a70 con 0x55758d8101e0 2024-04-25T11:35:32.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.282+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55758d8e2470 0x55758d989350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.282+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55758d989890 con 0x55758d8e2470 2024-04-25T11:35:32.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.282+0000 7ff9d5a00640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc072ec0 0x7ff9bc0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.283+0000 7ff9dee00640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55758d8e2470 0x55758d989350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.284+0000 7ff9d5a00640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc072ec0 0x7ff9bc0753d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55758d8e5e00 tx=0x7ff9c8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.284+0000 7ff9dee00640 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55758d8e2470 0x55758d989350 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9dee00640 1 -- 172.21.15.19:0/2425512159 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv10 ondisk = 0) ==== 164+0+21 (crc 0 0 0) 0x7ff9d80037b0 con 0x55758d8e2470 2024-04-25T11:35:32.273 INFO:teuthology.orchestra.run.smithi019.stdout:sixmegs_head 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55758d8e2470 msgr2=0x55758d989350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55758d8e2470 0x55758d989350 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc072ec0 msgr2=0x7ff9bc0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc072ec0 0x7ff9bc0753d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55758d8e5e00 tx=0x7ff9c8009000 comp rx=0 tx=0).stop 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8101e0 msgr2=0x55758d8dbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 --2- 172.21.15.19:0/2425512159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55758d8101e0 0x55758d8dbdd0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0000c00 tx=0x7ff9d0002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9dee00640 1 -- 172.21.15.19:0/2425512159 reap_dead start 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 shutdown_connections 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 >> 172.21.15.19:0/2425512159 conn(0x55758d732ac0 msgr2=0x55758d8e3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 shutdown_connections 2024-04-25T11:35:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.285+0000 7ff9e5d26f40 1 -- 172.21.15.19:0/2425512159 wait complete. 2024-04-25T11:35:32.285 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2024-04-25T11:35:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.469+0000 7f36ad56ff40 1 Processor -- start 2024-04-25T11:35:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.469+0000 7f36ad56ff40 1 -- start start 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36ad56ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec889310 0x5590ec7f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36ad56ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec7550e0 0x5590ec7554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36ad56ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36ad56ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590ec76ce70 con 0x5590ec889310 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36ad56ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5590ec6cad20 con 0x5590ec7550e0 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36ad56ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5590ec76d110 con 0x5590ec755a30 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36a6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec889310 0x5590ec7f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36a6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec889310 0x5590ec7f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41686/0 (socket says 172.21.15.19:41686) 2024-04-25T11:35:32.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.471+0000 7f36a6600640 1 -- 172.21.15.19:0/2251305460 learned_addr learned my addr 172.21.15.19:0/2251305460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a7000640 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a6600640 1 -- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 msgr2=0x5590ec7f8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a6600640 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a5c00640 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec7550e0 0x5590ec7554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a6600640 1 -- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec7550e0 msgr2=0x5590ec7554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a6600640 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec7550e0 0x5590ec7554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a6600640 1 -- 172.21.15.19:0/2251305460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ec754b80 con 0x5590ec889310 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f36a6600640 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec889310 0x5590ec7f2d20 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f3698026020 tx=0x7f369804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5016fa2968a8e53f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f369fe00640 1 -- 172.21.15.19:0/2251305460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698001a30 con 0x5590ec889310 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f369fe00640 1 -- 172.21.15.19:0/2251305460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698006070 con 0x5590ec889310 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.472+0000 7f369fe00640 1 -- 172.21.15.19:0/2251305460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369804d8b0 con 0x5590ec889310 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 -- 172.21.15.19:0/2251305460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec889310 msgr2=0x5590ec7f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec889310 0x5590ec7f2d20 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f3698026020 tx=0x7f369804f000 comp rx=0 tx=0).stop 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 -- 172.21.15.19:0/2251305460 shutdown_connections 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec7550e0 0x5590ec7554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7f8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/2251305460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec889310 0x5590ec7f2d20 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 -- 172.21.15.19:0/2251305460 >> 172.21.15.19:0/2251305460 conn(0x5590ec646ac0 msgr2=0x5590ec756320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 -- 172.21.15.19:0/2251305460 shutdown_connections 2024-04-25T11:35:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.473+0000 7f36ad56ff40 1 -- 172.21.15.19:0/2251305460 wait complete. 2024-04-25T11:35:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.474+0000 7f36ad56ff40 1 Processor -- start 2024-04-25T11:35:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.474+0000 7f36ad56ff40 1 -- start start 2024-04-25T11:35:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36ad56ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec7550e0 0x5590ec7f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36ad56ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7671d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36ad56ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec889310 0x5590ec767710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36ad56ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590ec76a7a0 con 0x5590ec7550e0 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36ad56ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5590ec754930 con 0x5590ec889310 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36ad56ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5590ec76ad60 con 0x5590ec755a30 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36a6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec7550e0 0x5590ec7f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36a6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec7550e0 0x5590ec7f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41702/0 (socket says 172.21.15.19:41702) 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.476+0000 7f36a6600640 1 -- 172.21.15.19:0/4142722650 learned_addr learned my addr 172.21.15.19:0/4142722650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a5c00640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a6600640 1 -- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 msgr2=0x5590ec7671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a6600640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7671d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a6600640 1 -- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec889310 msgr2=0x5590ec767710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a6600640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5590ec889310 0x5590ec767710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a6600640 1 -- 172.21.15.19:0/4142722650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ec7f2350 con 0x5590ec7550e0 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a5c00640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5590ec755a30 0x5590ec7671d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36a6600640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec7550e0 0x5590ec7f1e10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f369804f040 tx=0x7f3698001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f369ea00640 1 -- 172.21.15.19:0/4142722650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698070070 con 0x5590ec7550e0 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f369ea00640 1 -- 172.21.15.19:0/4142722650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698006220 con 0x5590ec7550e0 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f369ea00640 1 -- 172.21.15.19:0/4142722650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698003510 con 0x5590ec7550e0 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.477+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5590ec755740 con 0x5590ec7550e0 2024-04-25T11:35:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.478+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ec7690e0 con 0x5590ec7550e0 2024-04-25T11:35:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.480+0000 7f369ea00640 1 -- 172.21.15.19:0/4142722650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3698073070 con 0x5590ec7550e0 2024-04-25T11:35:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.481+0000 7f369ea00640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3684072e90 0x7f36840753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.481+0000 7f369ea00640 1 -- 172.21.15.19:0/4142722650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f36980d7e60 con 0x5590ec7550e0 2024-04-25T11:35:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.481+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5590ec88c190 0x5590ec892900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.481+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x5590ec88b850 con 0x5590ec88c190 2024-04-25T11:35:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.481+0000 7f36a5c00640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3684072e90 0x7f36840753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.482+0000 7f36a7000640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5590ec88c190 0x5590ec892900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.483+0000 7f36a5c00640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3684072e90 0x7f36840753a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f369000c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.483+0000 7f36a7000640 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5590ec88c190 0x5590ec892900 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.484+0000 7f36a7000640 1 -- 172.21.15.19:0/4142722650 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f36a0009490 con 0x5590ec88c190 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.484+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5590ec88c190 msgr2=0x5590ec892900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.484+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5590ec88c190 0x5590ec892900 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3684072e90 msgr2=0x7f36840753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3684072e90 0x7f36840753a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f369000c040 comp rx=0 tx=0).stop 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec7550e0 msgr2=0x5590ec7f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 --2- 172.21.15.19:0/4142722650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590ec7550e0 0x5590ec7f1e10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f369804f040 tx=0x7f3698001360 comp rx=0 tx=0).stop 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36a7000640 1 -- 172.21.15.19:0/4142722650 reap_dead start 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 shutdown_connections 2024-04-25T11:35:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 >> 172.21.15.19:0/4142722650 conn(0x5590ec646ac0 msgr2=0x5590ec756170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 shutdown_connections 2024-04-25T11:35:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.485+0000 7f36ad56ff40 1 -- 172.21.15.19:0/4142722650 wait complete. 2024-04-25T11:35:32.484 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2024-04-25T11:35:32.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.681+0000 7facf8954f40 1 Processor -- start 2024-04-25T11:35:32.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.681+0000 7facf8954f40 1 -- start start 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.682+0000 7facf8954f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d51e0 0x55fc806a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.682+0000 7facf8954f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d55f0 0x55fc805d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.682+0000 7facf8954f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc805d44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.682+0000 7facf8954f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc8057bd20 con 0x55fc806aefa0 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.682+0000 7facf8954f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc8057fd60 con 0x55fc805d51e0 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.682+0000 7facf8954f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fc8061e3f0 con 0x55fc805d55f0 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc805d44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc805d44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41728/0 (socket says 172.21.15.19:41728) 2024-04-25T11:35:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 -- 172.21.15.19:0/444254987 learned_addr learned my addr 172.21.15.19:0/444254987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 -- 172.21.15.19:0/444254987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d55f0 msgr2=0x55fc805d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 --2- 172.21.15.19:0/444254987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d55f0 0x55fc805d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 -- 172.21.15.19:0/444254987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d51e0 msgr2=0x55fc806a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 --2- 172.21.15.19:0/444254987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d51e0 0x55fc806a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.683+0000 7facf2400640 1 -- 172.21.15.19:0/444254987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc806a7a30 con 0x55fc806aefa0 2024-04-25T11:35:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.684+0000 7facf2400640 1 --2- 172.21.15.19:0/444254987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc805d44d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7facec00a4b0 tx=0x7facec00a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e1e0a797414bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.684+0000 7faceb400640 1 -- 172.21.15.19:0/444254987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec015070 con 0x55fc806aefa0 2024-04-25T11:35:32.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.684+0000 7faceb400640 1 -- 172.21.15.19:0/444254987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facec00e730 con 0x55fc806aefa0 2024-04-25T11:35:32.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.684+0000 7faceb400640 1 -- 172.21.15.19:0/444254987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec00eb20 con 0x55fc806aefa0 2024-04-25T11:35:32.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.684+0000 7facf8954f40 1 -- 172.21.15.19:0/444254987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 msgr2=0x55fc805d44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.684+0000 7facf8954f40 1 --2- 172.21.15.19:0/444254987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc805d44d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7facec00a4b0 tx=0x7facec00a970 comp rx=0 tx=0).stop 2024-04-25T11:35:32.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.685+0000 7facf8954f40 1 -- 172.21.15.19:0/444254987 shutdown_connections 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.685+0000 7facf8954f40 1 --2- 172.21.15.19:0/444254987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc805d44d0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.685+0000 7facf8954f40 1 --2- 172.21.15.19:0/444254987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d55f0 0x55fc805d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.685+0000 7facf8954f40 1 --2- 172.21.15.19:0/444254987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d51e0 0x55fc806a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.685+0000 7facf8954f40 1 -- 172.21.15.19:0/444254987 >> 172.21.15.19:0/444254987 conn(0x55fc804f7ac0 msgr2=0x55fc806226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.685+0000 7facf8954f40 1 -- 172.21.15.19:0/444254987 shutdown_connections 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.685+0000 7facf8954f40 1 -- 172.21.15.19:0/444254987 wait complete. 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.686+0000 7facf8954f40 1 Processor -- start 2024-04-25T11:35:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.686+0000 7facf8954f40 1 -- start start 2024-04-25T11:35:32.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf8954f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d51e0 0x55fc8068fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf8954f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d55f0 0x55fc80605400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf8954f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc80605940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf8954f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc80623110 con 0x55fc806aefa0 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf8954f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fc806a87b0 con 0x55fc805d55f0 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf8954f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fc806a0360 con 0x55fc805d51e0 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc80605940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc80605940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41744/0 (socket says 172.21.15.19:41744) 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf2400640 1 -- 172.21.15.19:0/4153159783 learned_addr learned my addr 172.21.15.19:0/4153159783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf1a00640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d51e0 0x55fc8068fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf2400640 1 -- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d51e0 msgr2=0x55fc8068fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf2400640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fc805d51e0 0x55fc8068fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf1000640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d55f0 0x55fc80605400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.688+0000 7facf2400640 1 -- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d55f0 msgr2=0x55fc80605400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facf2400640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d55f0 0x55fc80605400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facf2400640 1 -- 172.21.15.19:0/4153159783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc806902b0 con 0x55fc806aefa0 2024-04-25T11:35:32.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facf1000640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fc805d55f0 0x55fc80605400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:32.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facf2400640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc80605940 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x55fc806aad40 tx=0x7facec0032f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facea000640 1 -- 172.21.15.19:0/4153159783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec020020 con 0x55fc806aefa0 2024-04-25T11:35:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facea000640 1 -- 172.21.15.19:0/4153159783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facec016040 con 0x55fc806aefa0 2024-04-25T11:35:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facea000640 1 -- 172.21.15.19:0/4153159783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec015070 con 0x55fc806aefa0 2024-04-25T11:35:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fc806a4d00 con 0x55fc806aefa0 2024-04-25T11:35:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.689+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc806a50d0 con 0x55fc806aefa0 2024-04-25T11:35:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.692+0000 7facea000640 1 -- 172.21.15.19:0/4153159783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7facec023020 con 0x55fc806aefa0 2024-04-25T11:35:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.693+0000 7facea000640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7facd4072ec0 0x7facd40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.693+0000 7facea000640 1 -- 172.21.15.19:0/4153159783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7facec095440 con 0x55fc806aefa0 2024-04-25T11:35:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.693+0000 7facf8954f40 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55fc8069f5a0 0x55fc8069f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.693+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55fc8074eb70 con 0x55fc8069f5a0 2024-04-25T11:35:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.693+0000 7facf1a00640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7facd4072ec0 0x7facd40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.694+0000 7facf1000640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55fc8069f5a0 0x55fc8069f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.694+0000 7facf1000640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55fc8069f5a0 0x55fc8069f9b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf1000640 1 -- 172.21.15.19:0/4153159783 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7face000f3a0 con 0x55fc8069f5a0 2024-04-25T11:35:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf1a00640 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7facd4072ec0 0x7facd40753d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7face4057000 tx=0x7face4051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55fc8069f5a0 msgr2=0x55fc8069f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55fc8069f5a0 0x55fc8069f9b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7facd4072ec0 msgr2=0x7facd40753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7facd4072ec0 0x7facd40753d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7face4057000 tx=0x7face4051000 comp rx=0 tx=0).stop 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 msgr2=0x55fc80605940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 --2- 172.21.15.19:0/4153159783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc806aefa0 0x55fc80605940 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x55fc806aad40 tx=0x7facec0032f0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf2400640 1 -- 172.21.15.19:0/4153159783 reap_dead start 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 shutdown_connections 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 >> 172.21.15.19:0/4153159783 conn(0x55fc804f7ac0 msgr2=0x55fc806ab890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 shutdown_connections 2024-04-25T11:35:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.695+0000 7facf8954f40 1 -- 172.21.15.19:0/4153159783 wait complete. 2024-04-25T11:35:32.694 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2024-04-25T11:35:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.883+0000 7fd7d4337f40 1 Processor -- start 2024-04-25T11:35:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.883+0000 7fd7d4337f40 1 -- start start 2024-04-25T11:35:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7d4337f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6401a41e0 0x55e64027b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7d4337f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a45f0 0x55e640270ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7d4337f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e64027ddd0 0x55e640278f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7d4337f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6401ece70 con 0x55e64027ddd0 2024-04-25T11:35:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7d4337f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e64014ad20 con 0x55e6401a41e0 2024-04-25T11:35:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7d4337f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e6401ed3f0 con 0x55e6401a45f0 2024-04-25T11:35:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7cde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e64027ddd0 0x55e640278f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7cde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e64027ddd0 0x55e640278f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41762/0 (socket says 172.21.15.19:41762) 2024-04-25T11:35:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7cde00640 1 -- 172.21.15.19:0/2488963296 learned_addr learned my addr 172.21.15.19:0/2488963296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7cca00640 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a45f0 0x55e640270ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.885+0000 7fd7cd400640 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6401a41e0 0x55e64027b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.886+0000 7fd7cde00640 1 -- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a45f0 msgr2=0x55e640270ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.886+0000 7fd7cde00640 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a45f0 0x55e640270ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.886+0000 7fd7cde00640 1 -- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6401a41e0 msgr2=0x55e64027b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.886+0000 7fd7cde00640 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6401a41e0 0x55e64027b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.886+0000 7fd7cde00640 1 -- 172.21.15.19:0/2488963296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e64027aba0 con 0x55e64027ddd0 2024-04-25T11:35:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.886+0000 7fd7cde00640 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e64027ddd0 0x55e640278f20 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8001630 tx=0x7fd7c8001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50704324b308983 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.886+0000 7fd7c7400640 1 -- 172.21.15.19:0/2488963296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8003650 con 0x55e64027ddd0 2024-04-25T11:35:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7c7400640 1 -- 172.21.15.19:0/2488963296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c80037f0 con 0x55e64027ddd0 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7c7400640 1 -- 172.21.15.19:0/2488963296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8003ba0 con 0x55e64027ddd0 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 -- 172.21.15.19:0/2488963296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e64027ddd0 msgr2=0x55e640278f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e64027ddd0 0x55e640278f20 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8001630 tx=0x7fd7c8001af0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 -- 172.21.15.19:0/2488963296 shutdown_connections 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e64027ddd0 0x55e640278f20 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a45f0 0x55e640270ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/2488963296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e6401a41e0 0x55e64027b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 -- 172.21.15.19:0/2488963296 >> 172.21.15.19:0/2488963296 conn(0x55e6400c6ac0 msgr2=0x55e64027afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 -- 172.21.15.19:0/2488963296 shutdown_connections 2024-04-25T11:35:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.887+0000 7fd7d4337f40 1 -- 172.21.15.19:0/2488963296 wait complete. 2024-04-25T11:35:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.889+0000 7fd7d4337f40 1 Processor -- start 2024-04-25T11:35:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.889+0000 7fd7d4337f40 1 -- start start 2024-04-25T11:35:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.890+0000 7fd7d4337f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a41e0 0x55e6401e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.890+0000 7fd7d4337f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6401a45f0 0x55e6402735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.890+0000 7fd7d4337f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e64027ddd0 0x55e640272420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.890+0000 7fd7d4337f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e64027a950 con 0x55e6401a45f0 2024-04-25T11:35:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.890+0000 7fd7d4337f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e6401a3780 con 0x55e64027ddd0 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.890+0000 7fd7d4337f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e640271f90 con 0x55e6401a41e0 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.890+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6401a45f0 0x55e6402735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6401a45f0 0x55e6402735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41778/0 (socket says 172.21.15.19:41778) 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 -- 172.21.15.19:0/1491387047 learned_addr learned my addr 172.21.15.19:0/1491387047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cd400640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a41e0 0x55e6401e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cde00640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e64027ddd0 0x55e640272420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 -- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a41e0 msgr2=0x55e6401e75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a41e0 0x55e6401e75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 -- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e64027ddd0 msgr2=0x55e640272420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e64027ddd0 0x55e640272420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 -- 172.21.15.19:0/1491387047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6402716b0 con 0x55e6401a45f0 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cd400640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e6401a41e0 0x55e6401e75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cde00640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e64027ddd0 0x55e640272420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.891+0000 7fd7cca00640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6401a45f0 0x55e6402735d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc000c00 tx=0x7fd7bc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.892+0000 7fd7c6000640 1 -- 172.21.15.19:0/1491387047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc063030 con 0x55e6401a45f0 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.892+0000 7fd7c6000640 1 -- 172.21.15.19:0/1491387047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc054a30 con 0x55e6401a45f0 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.892+0000 7fd7c6000640 1 -- 172.21.15.19:0/1491387047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc059040 con 0x55e6401a45f0 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.892+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6401ee990 con 0x55e6401a45f0 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.892+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e640271c20 con 0x55e6401a45f0 2024-04-25T11:35:32.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.894+0000 7fd7c6000640 1 -- 172.21.15.19:0/1491387047 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd7bc066030 con 0x55e6401a45f0 2024-04-25T11:35:32.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.895+0000 7fd7c6000640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7b0072e90 0x7fd7b00753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.895+0000 7fd7c6000640 1 -- 172.21.15.19:0/1491387047 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fd7bc0d77b0 con 0x55e6401a45f0 2024-04-25T11:35:32.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.896+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e6402702c0 0x55e640314af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:32.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.896+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55e640272e70 con 0x55e6402702c0 2024-04-25T11:35:32.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.896+0000 7fd7cde00640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e6402702c0 0x55e640314af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.896+0000 7fd7cd400640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7b0072e90 0x7fd7b00753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:32.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7cde00640 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e6402702c0 0x55e640314af0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7cde00640 1 -- 172.21.15.19:0/1491387047 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fd7c800f030 con 0x55e6402702c0 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e6402702c0 msgr2=0x55e640314af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e6402702c0 0x55e640314af0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7b0072e90 msgr2=0x7fd7b00753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd7b0072e90 0x7fd7b00753a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0005fd0 tx=0x7fd7c000c040 comp rx=0 tx=0).stop 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6401a45f0 msgr2=0x55e6402735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7d4337f40 1 --2- 172.21.15.19:0/1491387047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6401a45f0 0x55e6402735d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc000c00 tx=0x7fd7bc002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.897+0000 7fd7cde00640 1 -- 172.21.15.19:0/1491387047 reap_dead start 2024-04-25T11:35:32.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.898+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 shutdown_connections 2024-04-25T11:35:32.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.898+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 >> 172.21.15.19:0/1491387047 conn(0x55e6400c6ac0 msgr2=0x55e64027b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:32.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.898+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 shutdown_connections 2024-04-25T11:35:32.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:32.898+0000 7fd7d4337f40 1 -- 172.21.15.19:0/1491387047 wait complete. 2024-04-25T11:35:32.898 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2024-04-25T11:35:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.072+0000 7f9b29428f40 1 Processor -- start 2024-04-25T11:35:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.072+0000 7f9b29428f40 1 -- start start 2024-04-25T11:35:33.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.073+0000 7f9b29428f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c1e0 0x559fc2e9bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.073+0000 7f9b29428f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2e9c5f0 0x559fc2f68e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.073+0000 7f9b29428f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2f75dd0 0x559fc2f74100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.073+0000 7f9b29428f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fc2ee53f0 con 0x559fc2e9c5f0 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.073+0000 7f9b29428f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559fc2ee4e70 con 0x559fc2e9c1e0 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.073+0000 7f9b29428f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559fc2ee56d0 con 0x559fc2f75dd0 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2e9c5f0 0x559fc2f68e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2e9c5f0 0x559fc2f68e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41800/0 (socket says 172.21.15.19:41800) 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 -- 172.21.15.19:0/2709392137 learned_addr learned my addr 172.21.15.19:0/2709392137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b22600640 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c1e0 0x559fc2e9bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b23000640 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2f75dd0 0x559fc2f74100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 -- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2f75dd0 msgr2=0x559fc2f74100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2f75dd0 0x559fc2f74100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 -- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c1e0 msgr2=0x559fc2e9bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c1e0 0x559fc2e9bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.074+0000 7f9b21c00640 1 -- 172.21.15.19:0/2709392137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc2dc5b30 con 0x559fc2e9c5f0 2024-04-25T11:35:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.075+0000 7f9b21c00640 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2e9c5f0 0x559fc2f68e00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c4b0 tx=0x7f9b1400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11865ac351489aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.075+0000 7f9b1be00640 1 -- 172.21.15.19:0/2709392137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140116b0 con 0x559fc2e9c5f0 2024-04-25T11:35:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.075+0000 7f9b1be00640 1 -- 172.21.15.19:0/2709392137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b14017070 con 0x559fc2e9c5f0 2024-04-25T11:35:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.075+0000 7f9b1be00640 1 -- 172.21.15.19:0/2709392137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14011a90 con 0x559fc2e9c5f0 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.075+0000 7f9b29428f40 1 -- 172.21.15.19:0/2709392137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2e9c5f0 msgr2=0x559fc2f68e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.075+0000 7f9b29428f40 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2e9c5f0 0x559fc2f68e00 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c4b0 tx=0x7f9b1400c970 comp rx=0 tx=0).stop 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.076+0000 7f9b29428f40 1 -- 172.21.15.19:0/2709392137 shutdown_connections 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.076+0000 7f9b29428f40 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2f75dd0 0x559fc2f74100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.076+0000 7f9b29428f40 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2e9c5f0 0x559fc2f68e00 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.076+0000 7f9b29428f40 1 --2- 172.21.15.19:0/2709392137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c1e0 0x559fc2e9bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.076+0000 7f9b29428f40 1 -- 172.21.15.19:0/2709392137 >> 172.21.15.19:0/2709392137 conn(0x559fc2dbeac0 msgr2=0x559fc2ece360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.076+0000 7f9b29428f40 1 -- 172.21.15.19:0/2709392137 shutdown_connections 2024-04-25T11:35:33.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.076+0000 7f9b29428f40 1 -- 172.21.15.19:0/2709392137 wait complete. 2024-04-25T11:35:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.077+0000 7f9b29428f40 1 Processor -- start 2024-04-25T11:35:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.077+0000 7f9b29428f40 1 -- start start 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.078+0000 7f9b29428f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2e9c1e0 0x559fc2f67920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.078+0000 7f9b29428f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c5f0 0x559fc2ee6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.078+0000 7f9b29428f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2f75dd0 0x559fc2ee73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b29428f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fc2f71fe0 con 0x559fc2f75dd0 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b29428f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559fc2f69340 con 0x559fc2e9c5f0 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b29428f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559fc2f6b630 con 0x559fc2e9c1e0 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2f75dd0 0x559fc2ee73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2f75dd0 0x559fc2ee73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41802/0 (socket says 172.21.15.19:41802) 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 -- 172.21.15.19:0/3648875572 learned_addr learned my addr 172.21.15.19:0/3648875572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b21c00640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c5f0 0x559fc2ee6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b22600640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2e9c1e0 0x559fc2f67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 -- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2e9c1e0 msgr2=0x559fc2f67920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559fc2e9c1e0 0x559fc2f67920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 -- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c5f0 msgr2=0x559fc2ee6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559fc2e9c5f0 0x559fc2ee6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.079+0000 7f9b23000640 1 -- 172.21.15.19:0/3648875572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc30043b0 con 0x559fc2f75dd0 2024-04-25T11:35:33.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.080+0000 7f9b23000640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2f75dd0 0x559fc2ee73c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00a520 tx=0x7f9b1c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.080+0000 7f9b1aa00640 1 -- 172.21.15.19:0/3648875572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c062070 con 0x559fc2f75dd0 2024-04-25T11:35:33.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.080+0000 7f9b1aa00640 1 -- 172.21.15.19:0/3648875572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0549f0 con 0x559fc2f75dd0 2024-04-25T11:35:33.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.080+0000 7f9b1aa00640 1 -- 172.21.15.19:0/3648875572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c059040 con 0x559fc2f75dd0 2024-04-25T11:35:33.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.080+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559fc3004690 con 0x559fc2f75dd0 2024-04-25T11:35:33.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.080+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc2f6b370 con 0x559fc2f75dd0 2024-04-25T11:35:33.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.083+0000 7f9b1aa00640 1 -- 172.21.15.19:0/3648875572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9b1c065070 con 0x559fc2f75dd0 2024-04-25T11:35:33.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.084+0000 7f9b1aa00640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b00072ec0 0x7f9b000753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.084+0000 7f9b1aa00640 1 -- 172.21.15.19:0/3648875572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f9b1c0d7690 con 0x559fc2f75dd0 2024-04-25T11:35:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.085+0000 7f9b29428f40 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559fc2f6aec0 0x559fc300d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.085+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x559fc2ee6980 con 0x559fc2f6aec0 2024-04-25T11:35:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.085+0000 7f9b22600640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b00072ec0 0x7f9b000753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.085+0000 7f9b21c00640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559fc2f6aec0 0x559fc300d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.086+0000 7f9b21c00640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559fc2f6aec0 0x559fc300d120 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.086+0000 7f9b22600640 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b00072ec0 0x7f9b000753d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005400 tx=0x7f9b0c005390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b21c00640 1 -- 172.21.15.19:0/3648875572 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f9b14011830 con 0x559fc2f6aec0 2024-04-25T11:35:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559fc2f6aec0 msgr2=0x559fc300d120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x559fc2f6aec0 0x559fc300d120 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b00072ec0 msgr2=0x7f9b000753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b00072ec0 0x7f9b000753d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005400 tx=0x7f9b0c005390 comp rx=0 tx=0).stop 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2f75dd0 msgr2=0x559fc2ee73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 --2- 172.21.15.19:0/3648875572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc2f75dd0 0x559fc2ee73c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00a520 tx=0x7f9b1c002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b23000640 1 -- 172.21.15.19:0/3648875572 reap_dead start 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 shutdown_connections 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 >> 172.21.15.19:0/3648875572 conn(0x559fc2dbeac0 msgr2=0x559fc2f70f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 shutdown_connections 2024-04-25T11:35:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.087+0000 7f9b29428f40 1 -- 172.21.15.19:0/3648875572 wait complete. 2024-04-25T11:35:33.087 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2024-04-25T11:35:33.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.260+0000 7f484f07af40 1 Processor -- start 2024-04-25T11:35:33.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.260+0000 7f484f07af40 1 -- start start 2024-04-25T11:35:33.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f484f07af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e19328f1e0 0x55e193365fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f484f07af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f5f0 0x55e193366510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f484f07af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e193368dd0 0x55e193367100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f484f07af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e1932d7e70 con 0x55e19328f5f0 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f484f07af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e193235d20 con 0x55e193368dd0 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f484f07af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e1932d83f0 con 0x55e19328f1e0 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f4847800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f5f0 0x55e193366510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f4847800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f5f0 0x55e193366510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41834/0 (socket says 172.21.15.19:41834) 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f4847800640 1 -- 172.21.15.19:0/2749206628 learned_addr learned my addr 172.21.15.19:0/2749206628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f4848200640 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e19328f1e0 0x55e193365fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f4848c00640 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e193368dd0 0x55e193367100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f4847800640 1 -- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e19328f1e0 msgr2=0x55e193365fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.262+0000 7f4847800640 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e19328f1e0 0x55e193365fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4847800640 1 -- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e193368dd0 msgr2=0x55e193367100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4847800640 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e193368dd0 0x55e193367100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4847800640 1 -- 172.21.15.19:0/2749206628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1933f5310 con 0x55e19328f5f0 2024-04-25T11:35:33.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4848c00640 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e193368dd0 0x55e193367100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:33.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4847800640 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f5f0 0x55e193366510 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4830025c20 tx=0x7f483004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4453b10c9475e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4846400640 1 -- 172.21.15.19:0/2749206628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830062070 con 0x55e19328f5f0 2024-04-25T11:35:33.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4846400640 1 -- 172.21.15.19:0/2749206628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48300549f0 con 0x55e19328f5f0 2024-04-25T11:35:33.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.263+0000 7f4846400640 1 -- 172.21.15.19:0/2749206628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830059040 con 0x55e19328f5f0 2024-04-25T11:35:33.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 -- 172.21.15.19:0/2749206628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f5f0 msgr2=0x55e193366510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f5f0 0x55e193366510 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4830025c20 tx=0x7f483004e000 comp rx=0 tx=0).stop 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 -- 172.21.15.19:0/2749206628 shutdown_connections 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e193368dd0 0x55e193367100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f5f0 0x55e193366510 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 --2- 172.21.15.19:0/2749206628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e19328f1e0 0x55e193365fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 -- 172.21.15.19:0/2749206628 >> 172.21.15.19:0/2749206628 conn(0x55e1931b1ac0 msgr2=0x55e1932c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 -- 172.21.15.19:0/2749206628 shutdown_connections 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.264+0000 7f484f07af40 1 -- 172.21.15.19:0/2749206628 wait complete. 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.265+0000 7f484f07af40 1 Processor -- start 2024-04-25T11:35:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.265+0000 7f484f07af40 1 -- start start 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f484f07af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f1e0 0x55e193344650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f484f07af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e19328f5f0 0x55e193364970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f484f07af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e193368dd0 0x55e1933f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f484f07af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e193365ba0 con 0x55e19328f1e0 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f484f07af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e193365550 con 0x55e19328f5f0 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f484f07af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e193344b90 con 0x55e193368dd0 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f4848200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f1e0 0x55e193344650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f4848200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f1e0 0x55e193344650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41840/0 (socket says 172.21.15.19:41840) 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f4848200640 1 -- 172.21.15.19:0/1854130924 learned_addr learned my addr 172.21.15.19:0/1854130924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.267+0000 7f4848c00640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e193368dd0 0x55e1933f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4847800640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e19328f5f0 0x55e193364970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4848200640 1 -- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e193368dd0 msgr2=0x55e1933f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4848200640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e193368dd0 0x55e1933f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4848200640 1 -- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e19328f5f0 msgr2=0x55e193364970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4848200640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e19328f5f0 0x55e193364970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4848200640 1 -- 172.21.15.19:0/1854130924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1932c10e0 con 0x55e19328f1e0 2024-04-25T11:35:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4848200640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f1e0 0x55e193344650 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f483800e2e0 tx=0x7f483800e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4845000640 1 -- 172.21.15.19:0/1854130924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838018020 con 0x55e19328f1e0 2024-04-25T11:35:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4845000640 1 -- 172.21.15.19:0/1854130924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4838013870 con 0x55e19328f1e0 2024-04-25T11:35:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f4845000640 1 -- 172.21.15.19:0/1854130924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838013c40 con 0x55e19328f1e0 2024-04-25T11:35:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.268+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1932d12b0 con 0x55e19328f1e0 2024-04-25T11:35:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.269+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1933f7650 con 0x55e19328f1e0 2024-04-25T11:35:33.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.271+0000 7f4845000640 1 -- 172.21.15.19:0/1854130924 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f483801b020 con 0x55e19328f1e0 2024-04-25T11:35:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.272+0000 7f4845000640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4824072ec0 0x7f48240753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.272+0000 7f4845000640 1 -- 172.21.15.19:0/1854130924 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f48380946a0 con 0x55e19328f1e0 2024-04-25T11:35:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.272+0000 7f484f07af40 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e193343f00 0x55e1932d9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.272+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55e19335cf70 con 0x55e193343f00 2024-04-25T11:35:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.273+0000 7f4847800640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4824072ec0 0x7f48240753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.273+0000 7f4848c00640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e193343f00 0x55e1932d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.274+0000 7f4847800640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4824072ec0 0x7f48240753d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f483000a5a0 tx=0x7f4830051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.274+0000 7f4848c00640 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e193343f00 0x55e1932d9760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f4848c00640 1 -- 172.21.15.19:0/1854130924 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv4 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f484000b590 con 0x55e193343f00 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e193343f00 msgr2=0x55e1932d9760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e193343f00 0x55e1932d9760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4824072ec0 msgr2=0x7f48240753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4824072ec0 0x7f48240753d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f483000a5a0 tx=0x7f4830051000 comp rx=0 tx=0).stop 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f1e0 msgr2=0x55e193344650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 --2- 172.21.15.19:0/1854130924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e19328f1e0 0x55e193344650 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f483800e2e0 tx=0x7f483800e7a0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f4848c00640 1 -- 172.21.15.19:0/1854130924 reap_dead start 2024-04-25T11:35:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 shutdown_connections 2024-04-25T11:35:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 >> 172.21.15.19:0/1854130924 conn(0x55e1931b1ac0 msgr2=0x55e19328eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 shutdown_connections 2024-04-25T11:35:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.276+0000 7f484f07af40 1 -- 172.21.15.19:0/1854130924 wait complete. 2024-04-25T11:35:33.275 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2024-04-25T11:35:33.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.472+0000 7fa982a13f40 1 Processor -- start 2024-04-25T11:35:33.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.472+0000 7fa982a13f40 1 -- start start 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.473+0000 7fa982a13f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9600030f0 0x559c2b5854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.473+0000 7fa982a13f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b60bac0 0x559c2b607da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.473+0000 7fa982a13f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2b608760 0x559c2b608b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa982a13f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c2b586ab0 con 0x559c2b608760 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa982a13f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559c2b4e0d20 con 0x7fa9600030f0 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa982a13f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559c2b5818b0 con 0x559c2b60bac0 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2b608760 0x559c2b608b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2b608760 0x559c2b608b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41856/0 (socket says 172.21.15.19:41856) 2024-04-25T11:35:33.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 -- 172.21.15.19:0/3643303872 learned_addr learned my addr 172.21.15.19:0/3643303872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97a600640 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b60bac0 0x559c2b607da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97b000640 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9600030f0 0x559c2b5854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 -- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b60bac0 msgr2=0x559c2b607da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b60bac0 0x559c2b607da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 -- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9600030f0 msgr2=0x559c2b5854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9600030f0 0x559c2b5854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97ba00640 1 -- 172.21.15.19:0/3643303872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2b60b6a0 con 0x559c2b608760 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97b000640 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9600030f0 0x559c2b5854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.474+0000 7fa97a600640 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b60bac0 0x559c2b607da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.475+0000 7fa97ba00640 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2b608760 0x559c2b608b70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x559c2b56d4d0 tx=0x7fa97400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a431989d577129fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.475+0000 7fa979200640 1 -- 172.21.15.19:0/3643303872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97400e700 con 0x559c2b608760 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.475+0000 7fa979200640 1 -- 172.21.15.19:0/3643303872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97400e8a0 con 0x559c2b608760 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.475+0000 7fa979200640 1 -- 172.21.15.19:0/3643303872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97400ec90 con 0x559c2b608760 2024-04-25T11:35:33.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 -- 172.21.15.19:0/3643303872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2b608760 msgr2=0x559c2b608b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2b608760 0x559c2b608b70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x559c2b56d4d0 tx=0x7fa97400c7f0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 -- 172.21.15.19:0/3643303872 shutdown_connections 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c2b608760 0x559c2b608b70 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b60bac0 0x559c2b607da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 --2- 172.21.15.19:0/3643303872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9600030f0 0x559c2b5854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 -- 172.21.15.19:0/3643303872 >> 172.21.15.19:0/3643303872 conn(0x559c2b45cac0 msgr2=0x559c2b617960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 -- 172.21.15.19:0/3643303872 shutdown_connections 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.476+0000 7fa982a13f40 1 -- 172.21.15.19:0/3643303872 wait complete. 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.477+0000 7fa982a13f40 1 Processor -- start 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.477+0000 7fa982a13f40 1 -- start start 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa982a13f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b608760 0x559c2b606d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa982a13f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c2b60bac0 0x559c2b568090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa982a13f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9600030f0 0x559c2b5685d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa982a13f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c2b614170 con 0x7fa9600030f0 2024-04-25T11:35:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa982a13f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x559c2b584fd0 con 0x559c2b60bac0 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa982a13f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x559c2b6082e0 con 0x559c2b608760 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9600030f0 0x559c2b5685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9600030f0 0x559c2b5685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41868/0 (socket says 172.21.15.19:41868) 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 -- 172.21.15.19:0/1696847369 learned_addr learned my addr 172.21.15.19:0/1696847369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97a600640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c2b60bac0 0x559c2b568090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97b000640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b608760 0x559c2b606d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 -- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b608760 msgr2=0x559c2b606d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x559c2b608760 0x559c2b606d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 -- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c2b60bac0 msgr2=0x559c2b568090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c2b60bac0 0x559c2b568090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97ba00640 1 -- 172.21.15.19:0/1696847369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2b60a6d0 con 0x7fa9600030f0 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.479+0000 7fa97a600640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x559c2b60bac0 0x559c2b568090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.480+0000 7fa97ba00640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9600030f0 0x559c2b5685d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa97401c660 tx=0x7fa9740019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.480+0000 7fa973400640 1 -- 172.21.15.19:0/1696847369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97401cc50 con 0x7fa9600030f0 2024-04-25T11:35:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.480+0000 7fa973400640 1 -- 172.21.15.19:0/1696847369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97401cdf0 con 0x7fa9600030f0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.480+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559c2b60ab40 con 0x7fa9600030f0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.480+0000 7fa973400640 1 -- 172.21.15.19:0/1696847369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974019650 con 0x7fa9600030f0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.480+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2b6af530 con 0x7fa9600030f0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.482+0000 7fa973400640 1 -- 172.21.15.19:0/1696847369 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa974009070 con 0x7fa9600030f0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.483+0000 7fa973400640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa958072e90 0x7fa9580753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.484+0000 7fa973400640 1 -- 172.21.15.19:0/1696847369 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fa974095e60 con 0x7fa9600030f0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.484+0000 7fa982a13f40 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559c2b6afd30 0x559c2b57c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.484+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x559c2b617da0 con 0x559c2b6afd30 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.484+0000 7fa97b000640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa958072e90 0x7fa9580753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.484+0000 7fa97a600640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559c2b6afd30 0x559c2b57c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa97a600640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559c2b6afd30 0x559c2b57c690 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa97a600640 1 -- 172.21.15.19:0/1696847369 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7fa96c0050c0 con 0x559c2b6afd30 2024-04-25T11:35:33.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559c2b6afd30 msgr2=0x559c2b57c690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa982a13f40 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x559c2b6afd30 0x559c2b57c690 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa97b000640 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa958072e90 0x7fa9580753a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x559c2b540d20 tx=0x7fa96400c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa958072e90 msgr2=0x7fa9580753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa982a13f40 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa958072e90 0x7fa9580753a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x559c2b540d20 tx=0x7fa96400c040 comp rx=0 tx=0).stop 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9600030f0 msgr2=0x559c2b5685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa982a13f40 1 --2- 172.21.15.19:0/1696847369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9600030f0 0x559c2b5685d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa97401c660 tx=0x7fa9740019e0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa97ba00640 1 -- 172.21.15.19:0/1696847369 reap_dead start 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.485+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 shutdown_connections 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.486+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 >> 172.21.15.19:0/1696847369 conn(0x559c2b45cac0 msgr2=0x559c2b614c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.486+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 shutdown_connections 2024-04-25T11:35:33.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.486+0000 7fa982a13f40 1 -- 172.21.15.19:0/1696847369 wait complete. 2024-04-25T11:35:33.492 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2024-04-25T11:35:33.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.688+0000 7f160ee0bf40 1 Processor -- start 2024-04-25T11:35:33.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.689+0000 7f160ee0bf40 1 -- start start 2024-04-25T11:35:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.690+0000 7f160ee0bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 0x56550b708180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.690+0000 7f160ee0bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.690+0000 7f160ee0bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b7da1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.690+0000 7f160ee0bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56550b751e70 con 0x56550b7e2fa0 2024-04-25T11:35:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.690+0000 7f160ee0bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56550b6afd20 con 0x56550b7091e0 2024-04-25T11:35:33.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.690+0000 7f160ee0bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56550b7523f0 con 0x56550b7095f0 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b7da1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b7da1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41880/0 (socket says 172.21.15.19:41880) 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 -- 172.21.15.19:0/4026168068 learned_addr learned my addr 172.21.15.19:0/4026168068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1606800640 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607200640 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 0x56550b708180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 -- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 msgr2=0x56550b7086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 -- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 msgr2=0x56550b708180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 0x56550b708180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 -- 172.21.15.19:0/4026168068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550b756510 con 0x56550b7e2fa0 2024-04-25T11:35:33.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1606800640 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.691+0000 7f1607c00640 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b7da1b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f1600009820 tx=0x7f1600005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcca5ac41f1532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.692+0000 7f1605400640 1 -- 172.21.15.19:0/4026168068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16000088d0 con 0x56550b7e2fa0 2024-04-25T11:35:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.692+0000 7f1605400640 1 -- 172.21.15.19:0/4026168068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1600018070 con 0x56550b7e2fa0 2024-04-25T11:35:33.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.692+0000 7f1605400640 1 -- 172.21.15.19:0/4026168068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600008cb0 con 0x56550b7e2fa0 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 -- 172.21.15.19:0/4026168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 msgr2=0x56550b7da1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b7da1b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f1600009820 tx=0x7f1600005a00 comp rx=0 tx=0).stop 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 -- 172.21.15.19:0/4026168068 shutdown_connections 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b7da1b0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/4026168068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 0x56550b708180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 -- 172.21.15.19:0/4026168068 >> 172.21.15.19:0/4026168068 conn(0x56550b62bac0 msgr2=0x56550b7df5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 -- 172.21.15.19:0/4026168068 shutdown_connections 2024-04-25T11:35:33.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.693+0000 7f160ee0bf40 1 -- 172.21.15.19:0/4026168068 wait complete. 2024-04-25T11:35:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.694+0000 7f160ee0bf40 1 Processor -- start 2024-04-25T11:35:33.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.694+0000 7f160ee0bf40 1 -- start start 2024-04-25T11:35:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f160ee0bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 0x56550b7dd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f160ee0bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7d1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f160ee0bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b74ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f160ee0bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56550b708c00 con 0x56550b7e2fa0 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f160ee0bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56550b7de070 con 0x56550b7091e0 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f160ee0bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56550b7082a0 con 0x56550b7095f0 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b74ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b74ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41892/0 (socket says 172.21.15.19:41892) 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 -- 172.21.15.19:0/2502313625 learned_addr learned my addr 172.21.15.19:0/2502313625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1606800640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607200640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 0x56550b7dd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 -- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 msgr2=0x56550b7d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 -- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 msgr2=0x56550b7dd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56550b7091e0 0x56550b7dd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1607c00640 1 -- 172.21.15.19:0/2502313625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550b74e1f0 con 0x56550b7e2fa0 2024-04-25T11:35:33.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.696+0000 7f1606800640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56550b7095f0 0x56550b7d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.697+0000 7f1607c00640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b74ecb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f1600009da0 tx=0x7f1600003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.697+0000 7f15ffe00640 1 -- 172.21.15.19:0/2502313625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600026670 con 0x56550b7e2fa0 2024-04-25T11:35:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.697+0000 7f15ffe00640 1 -- 172.21.15.19:0/2502313625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1600021070 con 0x56550b7e2fa0 2024-04-25T11:35:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.697+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56550b7084f0 con 0x56550b7e2fa0 2024-04-25T11:35:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.697+0000 7f15ffe00640 1 -- 172.21.15.19:0/2502313625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600026a30 con 0x56550b7e2fa0 2024-04-25T11:35:33.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.697+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550b74e4b0 con 0x56550b7e2fa0 2024-04-25T11:35:33.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.700+0000 7f15ffe00640 1 -- 172.21.15.19:0/2502313625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1600026bd0 con 0x56550b7e2fa0 2024-04-25T11:35:33.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.701+0000 7f15ffe00640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15e8072e90 0x7f15e80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.702+0000 7f15ffe00640 1 -- 172.21.15.19:0/2502313625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f1600095cd0 con 0x56550b7e2fa0 2024-04-25T11:35:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.702+0000 7f1607200640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15e8072e90 0x7f15e80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.703+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56550b7dc4b0 0x56550b7da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.703+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x56550b74ffb0 con 0x56550b7dc4b0 2024-04-25T11:35:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.703+0000 7f1606800640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56550b7dc4b0 0x56550b7da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.703+0000 7f1607200640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15e8072e90 0x7f15e80753a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f15f8056000 tx=0x7f15f8050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.704+0000 7f1606800640 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56550b7dc4b0 0x56550b7da6c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f1606800640 1 -- 172.21.15.19:0/2502313625 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f15f4005020 con 0x56550b7dc4b0 2024-04-25T11:35:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56550b7dc4b0 msgr2=0x56550b7da6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56550b7dc4b0 0x56550b7da6c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15e8072e90 msgr2=0x7f15e80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15e8072e90 0x7f15e80753a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f15f8056000 tx=0x7f15f8050000 comp rx=0 tx=0).stop 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 msgr2=0x56550b74ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 --2- 172.21.15.19:0/2502313625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56550b7e2fa0 0x56550b74ecb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f1600009da0 tx=0x7f1600003020 comp rx=0 tx=0).stop 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f1607c00640 1 -- 172.21.15.19:0/2502313625 reap_dead start 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 shutdown_connections 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 >> 172.21.15.19:0/2502313625 conn(0x56550b62bac0 msgr2=0x56550b7d91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.705+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 shutdown_connections 2024-04-25T11:35:33.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.706+0000 7f160ee0bf40 1 -- 172.21.15.19:0/2502313625 wait complete. 2024-04-25T11:35:33.707 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2024-04-25T11:35:33.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.907+0000 7fc2d1c0ff40 1 Processor -- start 2024-04-25T11:35:33.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.907+0000 7fc2d1c0ff40 1 -- start start 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.908+0000 7fc2d1c0ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8132a8250 0x55e81332cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.908+0000 7fc2d1c0ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e81332d240 0x55e813336100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.908+0000 7fc2d1c0ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e81328ea10 0x55e813336640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.908+0000 7fc2d1c0ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e813204d20 con 0x55e8132a8250 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.908+0000 7fc2d1c0ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e813208d60 con 0x55e81328ea10 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.908+0000 7fc2d1c0ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e8132a7110 con 0x55e81332d240 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8132a8250 0x55e81332cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8132a8250 0x55e81332cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39752/0 (socket says 172.21.15.19:39752) 2024-04-25T11:35:33.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 -- 172.21.15.19:0/3051879457 learned_addr learned my addr 172.21.15.19:0/3051879457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 -- 172.21.15.19:0/3051879457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e81332d240 msgr2=0x55e813336100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 --2- 172.21.15.19:0/3051879457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e81332d240 0x55e813336100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 -- 172.21.15.19:0/3051879457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e81328ea10 msgr2=0x55e813336640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:33.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 --2- 172.21.15.19:0/3051879457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e81328ea10 0x55e813336640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 -- 172.21.15.19:0/3051879457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81328c8f0 con 0x55e8132a8250 2024-04-25T11:35:33.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.909+0000 7fc2ca000640 1 --2- 172.21.15.19:0/3051879457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8132a8250 0x55e81332cd00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4001d80 tx=0x7fc2b4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30e5ba89b107d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2c3e00640 1 -- 172.21.15.19:0/3051879457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400bcf0 con 0x55e8132a8250 2024-04-25T11:35:33.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2c3e00640 1 -- 172.21.15.19:0/3051879457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b400f400 con 0x55e8132a8250 2024-04-25T11:35:33.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2c3e00640 1 -- 172.21.15.19:0/3051879457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400f7f0 con 0x55e8132a8250 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/3051879457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8132a8250 msgr2=0x55e81332cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/3051879457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8132a8250 0x55e81332cd00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4001d80 tx=0x7fc2b4001e70 comp rx=0 tx=0).stop 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/3051879457 shutdown_connections 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/3051879457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e81328ea10 0x55e813336640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/3051879457 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e81332d240 0x55e813336100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/3051879457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e8132a8250 0x55e81332cd00 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.910+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/3051879457 >> 172.21.15.19:0/3051879457 conn(0x55e813180ac0 msgr2=0x55e813290320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.911+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/3051879457 shutdown_connections 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.911+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/3051879457 wait complete. 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.912+0000 7fc2d1c0ff40 1 Processor -- start 2024-04-25T11:35:33.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.912+0000 7fc2d1c0ff40 1 -- start start 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.913+0000 7fc2d1c0ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81328ea10 0x55e813330b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.913+0000 7fc2d1c0ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e8132a8250 0x55e8133209c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.913+0000 7fc2d1c0ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e81332d240 0x55e81332a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.913+0000 7fc2d1c0ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e81328f170 con 0x55e81328ea10 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.913+0000 7fc2d1c0ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e81328cbd0 con 0x55e81332d240 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.913+0000 7fc2d1c0ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e81328f450 con 0x55e8132a8250 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81328ea10 0x55e813330b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81328ea10 0x55e813330b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39766/0 (socket says 172.21.15.19:39766) 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 -- 172.21.15.19:0/156354477 learned_addr learned my addr 172.21.15.19:0/156354477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 -- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e8132a8250 msgr2=0x55e8133209c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e8132a8250 0x55e8133209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 -- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e81332d240 msgr2=0x55e81332a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e81332d240 0x55e81332a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 -- 172.21.15.19:0/156354477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8133c5a60 con 0x55e81328ea10 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.914+0000 7fc2ca000640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81328ea10 0x55e813330b00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x55e8132a8690 tx=0x7fc2b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.915+0000 7fc2c2a00640 1 -- 172.21.15.19:0/156354477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400be70 con 0x55e81328ea10 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.915+0000 7fc2c2a00640 1 -- 172.21.15.19:0/156354477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b4019680 con 0x55e81328ea10 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.915+0000 7fc2c2a00640 1 -- 172.21.15.19:0/156354477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4019a30 con 0x55e81328ea10 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.915+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e8133c5d40 con 0x55e81328ea10 2024-04-25T11:35:33.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.915+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8133c6300 con 0x55e81328ea10 2024-04-25T11:35:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.917+0000 7fc2c2a00640 1 -- 172.21.15.19:0/156354477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc2b4017040 con 0x55e81328ea10 2024-04-25T11:35:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.918+0000 7fc2c2a00640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2a8072ec0 0x7fc2a80753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.918+0000 7fc2c2a00640 1 -- 172.21.15.19:0/156354477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fc2b4095d00 con 0x55e81328ea10 2024-04-25T11:35:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.919+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e8133c3bc0 0x55e8133c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.919+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55e8133d8560 con 0x55e8133c3bc0 2024-04-25T11:35:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.919+0000 7fc2c9600640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2a8072ec0 0x7fc2a80753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.919+0000 7fc2caa00640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e8133c3bc0 0x55e8133c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.921+0000 7fc2c9600640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2a8072ec0 0x7fc2a80753d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55e813329d10 tx=0x7fc2bc001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.921+0000 7fc2caa00640 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e8133c3bc0 0x55e8133c3fd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2caa00640 1 -- 172.21.15.19:0/156354477 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7fc2c4005040 con 0x55e8133c3bc0 2024-04-25T11:35:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e8133c3bc0 msgr2=0x55e8133c3fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e8133c3bc0 0x55e8133c3fd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2a8072ec0 msgr2=0x7fc2a80753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc2a8072ec0 0x7fc2a80753d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55e813329d10 tx=0x7fc2bc001520 comp rx=0 tx=0).stop 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81328ea10 msgr2=0x55e813330b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 --2- 172.21.15.19:0/156354477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e81328ea10 0x55e813330b00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x55e8132a8690 tx=0x7fc2b4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2caa00640 1 -- 172.21.15.19:0/156354477 reap_dead start 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 shutdown_connections 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 >> 172.21.15.19:0/156354477 conn(0x55e813180ac0 msgr2=0x55e813335180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 shutdown_connections 2024-04-25T11:35:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:33.922+0000 7fc2d1c0ff40 1 -- 172.21.15.19:0/156354477 wait complete. 2024-04-25T11:35:33.923 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2024-04-25T11:35:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.103+0000 7fe8cd528f40 1 Processor -- start 2024-04-25T11:35:34.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.103+0000 7fe8cd528f40 1 -- start start 2024-04-25T11:35:34.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.104+0000 7fe8cd528f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.104+0000 7fe8cd528f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba977d85f0 0x55ba978aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.104+0000 7fe8cd528f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba978b1dd0 0x55ba978af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.104+0000 7fe8cd528f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba97820e70 con 0x55ba977d85f0 2024-04-25T11:35:34.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.104+0000 7fe8cd528f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba9777ed20 con 0x55ba978b1dd0 2024-04-25T11:35:34.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.104+0000 7fe8cd528f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba978213f0 con 0x55ba977d81e0 2024-04-25T11:35:34.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba977d85f0 0x55ba978aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba977d85f0 0x55ba978aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39778/0 (socket says 172.21.15.19:39778) 2024-04-25T11:35:34.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 -- 172.21.15.19:0/4232682583 learned_addr learned my addr 172.21.15.19:0/4232682583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5c00640 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 -- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 msgr2=0x55ba978a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c6600640 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba978b1dd0 0x55ba978af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 -- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba978b1dd0 msgr2=0x55ba978af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba978b1dd0 0x55ba978af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 -- 172.21.15.19:0/4232682583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba978a87a0 con 0x55ba977d85f0 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.105+0000 7fe8c5200640 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba977d85f0 0x55ba978aefd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009780 tx=0x7fe8b4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715fa21e359bf335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.106+0000 7fe8bf400640 1 -- 172.21.15.19:0/4232682583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4008870 con 0x55ba977d85f0 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.106+0000 7fe8bf400640 1 -- 172.21.15.19:0/4232682583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b4018070 con 0x55ba977d85f0 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.106+0000 7fe8bf400640 1 -- 172.21.15.19:0/4232682583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4008c30 con 0x55ba977d85f0 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.106+0000 7fe8cd528f40 1 -- 172.21.15.19:0/4232682583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba977d85f0 msgr2=0x55ba978aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.106+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba977d85f0 0x55ba978aefd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009780 tx=0x7fe8b4005a00 comp rx=0 tx=0).stop 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.107+0000 7fe8cd528f40 1 -- 172.21.15.19:0/4232682583 shutdown_connections 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.107+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba978b1dd0 0x55ba978af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.107+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba977d85f0 0x55ba978aefd0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.107+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/4232682583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.107+0000 7fe8cd528f40 1 -- 172.21.15.19:0/4232682583 >> 172.21.15.19:0/4232682583 conn(0x55ba976faac0 msgr2=0x55ba9780a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.107+0000 7fe8cd528f40 1 -- 172.21.15.19:0/4232682583 shutdown_connections 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.107+0000 7fe8cd528f40 1 -- 172.21.15.19:0/4232682583 wait complete. 2024-04-25T11:35:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.108+0000 7fe8cd528f40 1 Processor -- start 2024-04-25T11:35:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.108+0000 7fe8cd528f40 1 -- start start 2024-04-25T11:35:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.109+0000 7fe8cd528f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.109+0000 7fe8cd528f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba977d85f0 0x55ba978a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.109+0000 7fe8cd528f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba978b1dd0 0x55ba978a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.109+0000 7fe8cd528f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba977d7260 con 0x55ba978b1dd0 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.109+0000 7fe8cd528f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba978ae5e0 con 0x55ba977d85f0 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.109+0000 7fe8cd528f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba978ae840 con 0x55ba977d81e0 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba978b1dd0 0x55ba978a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba978b1dd0 0x55ba978a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39780/0 (socket says 172.21.15.19:39780) 2024-04-25T11:35:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 -- 172.21.15.19:0/1214105504 learned_addr learned my addr 172.21.15.19:0/1214105504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c5c00640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 -- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 msgr2=0x55ba978a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 -- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba977d85f0 msgr2=0x55ba978a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c5200640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba977d85f0 0x55ba978a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba977d85f0 0x55ba978a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c6600640 1 -- 172.21.15.19:0/1214105504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba978ae1b0 con 0x55ba978b1dd0 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c5c00640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba977d81e0 0x55ba978a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.110+0000 7fe8c5200640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba977d85f0 0x55ba978a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.111+0000 7fe8c6600640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba978b1dd0 0x55ba978a42d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0006e70 tx=0x7fe8c000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.111+0000 7fe8be000640 1 -- 172.21.15.19:0/1214105504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0017070 con 0x55ba978b1dd0 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.111+0000 7fe8be000640 1 -- 172.21.15.19:0/1214105504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c00062d0 con 0x55ba978b1dd0 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.111+0000 7fe8be000640 1 -- 172.21.15.19:0/1214105504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004740 con 0x55ba978b1dd0 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.111+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba9793e850 con 0x55ba978b1dd0 2024-04-25T11:35:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.111+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9780ab30 con 0x55ba978b1dd0 2024-04-25T11:35:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.113+0000 7fe8be000640 1 -- 172.21.15.19:0/1214105504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe8c001a070 con 0x55ba978b1dd0 2024-04-25T11:35:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.114+0000 7fe8be000640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8a8072e90 0x7fe8a80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.114+0000 7fe8be000640 1 -- 172.21.15.19:0/1214105504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fe8c0093680 con 0x55ba978b1dd0 2024-04-25T11:35:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.115+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ba978a70d0 0x55ba9793f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.115+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55ba9793fd80 con 0x55ba978a70d0 2024-04-25T11:35:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.115+0000 7fe8c5c00640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8a8072e90 0x7fe8a80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.116+0000 7fe8c5200640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ba978a70d0 0x55ba9793f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.117+0000 7fe8c5c00640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8a8072e90 0x7fe8a80753a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8056000 tx=0x7fe8b8050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.117+0000 7fe8c5200640 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ba978a70d0 0x55ba9793f840 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8c5200640 1 -- 172.21.15.19:0/1214105504 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) ==== 156+0+155 (crc 0 0 0) 0x7fe8b4008810 con 0x55ba978a70d0 2024-04-25T11:35:34.106 INFO:teuthology.orchestra.run.smithi019.stdout:stray0_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray1_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray2_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray3_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray4_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray5_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray6_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray7_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray8_head 2024-04-25T11:35:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:stray9_head 2024-04-25T11:35:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ba978a70d0 msgr2=0x55ba9793f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ba978a70d0 0x55ba9793f840 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8a8072e90 msgr2=0x7fe8a80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe8a8072e90 0x7fe8a80753a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8056000 tx=0x7fe8b8050000 comp rx=0 tx=0).stop 2024-04-25T11:35:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba978b1dd0 msgr2=0x55ba978a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 --2- 172.21.15.19:0/1214105504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba978b1dd0 0x55ba978a42d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0006e70 tx=0x7fe8c000c520 comp rx=0 tx=0).stop 2024-04-25T11:35:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8c6600640 1 -- 172.21.15.19:0/1214105504 reap_dead start 2024-04-25T11:35:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 shutdown_connections 2024-04-25T11:35:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.118+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 >> 172.21.15.19:0/1214105504 conn(0x55ba976faac0 msgr2=0x55ba978a4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.119+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 shutdown_connections 2024-04-25T11:35:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.119+0000 7fe8cd528f40 1 -- 172.21.15.19:0/1214105504 wait complete. 2024-04-25T11:35:34.118 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2024-04-25T11:35:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.305+0000 7f63fe3def40 1 Processor -- start 2024-04-25T11:35:34.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.305+0000 7f63fe3def40 1 -- start start 2024-04-25T11:35:34.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63fe3def40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63fe3def40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e8d85f0 0x55c77ea3dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63fe3def40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e9b1fa0 0x55c77e9087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63fe3def40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c77e920e70 con 0x55c77e8d85f0 2024-04-25T11:35:34.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63fe3def40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c77e87ed20 con 0x55c77e9b1fa0 2024-04-25T11:35:34.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63fe3def40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c77e9213f0 con 0x55c77e8d81e0 2024-04-25T11:35:34.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f6c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e8d85f0 0x55c77ea3dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f6c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e8d85f0 0x55c77ea3dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39790/0 (socket says 172.21.15.19:39790) 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f6c00640 1 -- 172.21.15.19:0/4259625420 learned_addr learned my addr 172.21.15.19:0/4259625420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f7600640 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f6c00640 1 -- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 msgr2=0x55c77ea3d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f6c00640 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f6c00640 1 -- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e9b1fa0 msgr2=0x55c77e9087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.307+0000 7f63f8000640 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e9b1fa0 0x55c77e9087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.308+0000 7f63f6c00640 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e9b1fa0 0x55c77e9087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.308+0000 7f63f6c00640 1 -- 172.21.15.19:0/4259625420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77e9af3b0 con 0x55c77e8d85f0 2024-04-25T11:35:34.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.308+0000 7f63f7600640 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:34.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.308+0000 7f63f6c00640 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e8d85f0 0x55c77ea3dcd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f63e4001ce0 tx=0x7f63e400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae952d7ccd6e81b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.308+0000 7f63f5800640 1 -- 172.21.15.19:0/4259625420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e400bd20 con 0x55c77e8d85f0 2024-04-25T11:35:34.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.308+0000 7f63f5800640 1 -- 172.21.15.19:0/4259625420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e400f400 con 0x55c77e8d85f0 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.308+0000 7f63f5800640 1 -- 172.21.15.19:0/4259625420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e400f7f0 con 0x55c77e8d85f0 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 -- 172.21.15.19:0/4259625420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e8d85f0 msgr2=0x55c77ea3dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e8d85f0 0x55c77ea3dcd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f63e4001ce0 tx=0x7f63e400b5b0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 -- 172.21.15.19:0/4259625420 shutdown_connections 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e9b1fa0 0x55c77e9087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e8d85f0 0x55c77ea3dcd0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4259625420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 -- 172.21.15.19:0/4259625420 >> 172.21.15.19:0/4259625420 conn(0x55c77e7faac0 msgr2=0x55c77e90a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 -- 172.21.15.19:0/4259625420 shutdown_connections 2024-04-25T11:35:34.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.309+0000 7f63fe3def40 1 -- 172.21.15.19:0/4259625420 wait complete. 2024-04-25T11:35:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.310+0000 7f63fe3def40 1 Processor -- start 2024-04-25T11:35:34.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.310+0000 7f63fe3def40 1 -- start start 2024-04-25T11:35:34.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63fe3def40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63fe3def40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e8d85f0 0x55c77e909660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63fe3def40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e9b1fa0 0x55c77e909ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63fe3def40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c77e9ae610 con 0x55c77e9b1fa0 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63fe3def40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c77e9af690 con 0x55c77e8d85f0 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63fe3def40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c77e9a3050 con 0x55c77e8d81e0 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63f8000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e9b1fa0 0x55c77e909ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63f8000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e9b1fa0 0x55c77e909ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39794/0 (socket says 172.21.15.19:39794) 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63f8000640 1 -- 172.21.15.19:0/4051941829 learned_addr learned my addr 172.21.15.19:0/4051941829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.312+0000 7f63f7600640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f6c00640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e8d85f0 0x55c77e909660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f8000640 1 -- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 msgr2=0x55c77ea3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f8000640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c77e8d81e0 0x55c77ea3d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f8000640 1 -- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e8d85f0 msgr2=0x55c77e909660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f8000640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e8d85f0 0x55c77e909660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f8000640 1 -- 172.21.15.19:0/4051941829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c77e91d960 con 0x55c77e9b1fa0 2024-04-25T11:35:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f6c00640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c77e8d85f0 0x55c77e909660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63f8000640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e9b1fa0 0x55c77e909ba0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f63f0025c70 tx=0x7f63f0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63efe00640 1 -- 172.21.15.19:0/4051941829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f0062070 con 0x55c77e9b1fa0 2024-04-25T11:35:34.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63efe00640 1 -- 172.21.15.19:0/4051941829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f0054b70 con 0x55c77e9b1fa0 2024-04-25T11:35:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63efe00640 1 -- 172.21.15.19:0/4051941829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00545e0 con 0x55c77e9b1fa0 2024-04-25T11:35:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.313+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c77e91dc40 con 0x55c77e9b1fa0 2024-04-25T11:35:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.314+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c77e9a2d90 con 0x55c77e9b1fa0 2024-04-25T11:35:34.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.316+0000 7f63efe00640 1 -- 172.21.15.19:0/4051941829 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f63f005e410 con 0x55c77e9b1fa0 2024-04-25T11:35:34.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.317+0000 7f63efe00640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63d8072ec0 0x7f63d80753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.317+0000 7f63efe00640 1 -- 172.21.15.19:0/4051941829 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f63f00d81e0 con 0x55c77e9b1fa0 2024-04-25T11:35:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.317+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c77e9a2220 0x55c77e9a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.317+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55c77ea51c30 con 0x55c77e9a2220 2024-04-25T11:35:34.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.317+0000 7f63f7600640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63d8072ec0 0x7f63d80753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.318+0000 7f63f6c00640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c77e9a2220 0x55c77e9a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.319+0000 7f63f7600640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63d8072ec0 0x7f63d80753d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005fd0 tx=0x7f63e8007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.319+0000 7f63f6c00640 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c77e9a2220 0x55c77e9a2630 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63f6c00640 1 -- 172.21.15.19:0/4051941829 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7f63e400f6b0 con 0x55c77e9a2220 2024-04-25T11:35:34.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c77e9a2220 msgr2=0x55c77e9a2630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c77e9a2220 0x55c77e9a2630 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63d8072ec0 msgr2=0x7f63d80753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63d8072ec0 0x7f63d80753d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005fd0 tx=0x7f63e8007040 comp rx=0 tx=0).stop 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e9b1fa0 msgr2=0x55c77e909ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 --2- 172.21.15.19:0/4051941829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c77e9b1fa0 0x55c77e909ba0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f63f0025c70 tx=0x7f63f0002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63f8000640 1 -- 172.21.15.19:0/4051941829 reap_dead start 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 shutdown_connections 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.320+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 >> 172.21.15.19:0/4051941829 conn(0x55c77e7faac0 msgr2=0x55c77e90a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.321+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 shutdown_connections 2024-04-25T11:35:34.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.321+0000 7f63fe3def40 1 -- 172.21.15.19:0/4051941829 wait complete. 2024-04-25T11:35:34.320 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2024-04-25T11:35:34.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.511+0000 7f5617906f40 1 Processor -- start 2024-04-25T11:35:34.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.511+0000 7f5617906f40 1 -- start start 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5617906f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 0x563ae2b19740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5617906f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5617906f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 0x563ae2ba2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5617906f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ae2b163f0 con 0x563ae2acd5f0 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5617906f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ae2b15e70 con 0x563ae2acd1e0 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5617906f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563ae2b166d0 con 0x563ae2ba6dd0 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5610000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5610000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39808/0 (socket says 172.21.15.19:39808) 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5610000640 1 -- 172.21.15.19:0/2626940778 learned_addr learned my addr 172.21.15.19:0/2626940778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.513+0000 7f5611400640 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 0x563ae2ba2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610a00640 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 0x563ae2b19740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610000640 1 -- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 msgr2=0x563ae2ba2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610000640 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 0x563ae2ba2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610000640 1 -- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 msgr2=0x563ae2b19740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610000640 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 0x563ae2b19740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610000640 1 -- 172.21.15.19:0/2626940778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae29f6b30 con 0x563ae2acd5f0 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5611400640 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 0x563ae2ba2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610a00640 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 0x563ae2b19740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f5610000640 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9d210 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f55f8045f30 tx=0x7f55f8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3787da3011ec5e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f560ec00640 1 -- 172.21.15.19:0/2626940778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8053860 con 0x563ae2acd5f0 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f560ec00640 1 -- 172.21.15.19:0/2626940778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f8053a00 con 0x563ae2acd5f0 2024-04-25T11:35:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.514+0000 7f560ec00640 1 -- 172.21.15.19:0/2626940778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8053df0 con 0x563ae2acd5f0 2024-04-25T11:35:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 -- 172.21.15.19:0/2626940778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 msgr2=0x563ae2b9d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9d210 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f55f8045f30 tx=0x7f55f8051000 comp rx=0 tx=0).stop 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 -- 172.21.15.19:0/2626940778 shutdown_connections 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 0x563ae2ba2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9d210 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 --2- 172.21.15.19:0/2626940778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 0x563ae2b19740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 -- 172.21.15.19:0/2626940778 >> 172.21.15.19:0/2626940778 conn(0x563ae29efac0 msgr2=0x563ae2ba1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 -- 172.21.15.19:0/2626940778 shutdown_connections 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.515+0000 7f5617906f40 1 -- 172.21.15.19:0/2626940778 wait complete. 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.516+0000 7f5617906f40 1 Processor -- start 2024-04-25T11:35:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.517+0000 7f5617906f40 1 -- start start 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5617906f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 0x563ae2afd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5617906f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5617906f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 0x563ae2b9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5617906f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ae2accdd0 con 0x563ae2acd5f0 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5617906f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563ae2b9d750 con 0x563ae2acd1e0 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5617906f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563ae2acc810 con 0x563ae2ba6dd0 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5610000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5610000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39814/0 (socket says 172.21.15.19:39814) 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.518+0000 7f5610000640 1 -- 172.21.15.19:0/1939432603 learned_addr learned my addr 172.21.15.19:0/1939432603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f5610000640 1 -- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 msgr2=0x563ae2b9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f5610000640 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563ae2ba6dd0 0x563ae2b9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f5610000640 1 -- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 msgr2=0x563ae2afd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f5610000640 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563ae2acd1e0 0x563ae2afd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f5610000640 1 -- 172.21.15.19:0/1939432603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae2b99aa0 con 0x563ae2acd5f0 2024-04-25T11:35:34.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f5610000640 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9a740 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f55f8005570 tx=0x7f55f8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f560d800640 1 -- 172.21.15.19:0/1939432603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8053cf0 con 0x563ae2acd5f0 2024-04-25T11:35:34.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f560d800640 1 -- 172.21.15.19:0/1939432603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f8075070 con 0x563ae2acd5f0 2024-04-25T11:35:34.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563ae2afde70 con 0x563ae2acd5f0 2024-04-25T11:35:34.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.519+0000 7f560d800640 1 -- 172.21.15.19:0/1939432603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8070680 con 0x563ae2acd5f0 2024-04-25T11:35:34.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.520+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae2b9a020 con 0x563ae2acd5f0 2024-04-25T11:35:34.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.522+0000 7f560d800640 1 -- 172.21.15.19:0/1939432603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f55f8070820 con 0x563ae2acd5f0 2024-04-25T11:35:34.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.523+0000 7f560d800640 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55ec072e90 0x7f55ec0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.523+0000 7f560d800640 1 -- 172.21.15.19:0/1939432603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f55f80da560 con 0x563ae2acd5f0 2024-04-25T11:35:34.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.523+0000 7f5617906f40 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563ae2b9b1c0 0x563ae2c3e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.523+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x563ae2b96390 con 0x563ae2b9b1c0 2024-04-25T11:35:34.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.524+0000 7f5611400640 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563ae2b9b1c0 0x563ae2c3e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.524+0000 7f5610a00640 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55ec072e90 0x7f55ec0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.524+0000 7f5611400640 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563ae2b9b1c0 0x563ae2c3e8d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5611400640 1 -- 172.21.15.19:0/1939432603 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f56080074d0 con 0x563ae2b9b1c0 2024-04-25T11:35:34.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563ae2b9b1c0 msgr2=0x563ae2c3e8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563ae2b9b1c0 0x563ae2c3e8d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55ec072e90 msgr2=0x7f55ec0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55ec072e90 0x7f55ec0753a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f560000b4a0 tx=0x7f5600007040 comp rx=0 tx=0).stop 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 msgr2=0x563ae2b9a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 --2- 172.21.15.19:0/1939432603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ae2acd5f0 0x563ae2b9a740 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f55f8005570 tx=0x7f55f8002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5611400640 1 -- 172.21.15.19:0/1939432603 reap_dead start 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 shutdown_connections 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 >> 172.21.15.19:0/1939432603 conn(0x563ae29efac0 msgr2=0x563ae2ba0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.525+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 shutdown_connections 2024-04-25T11:35:34.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.526+0000 7f5617906f40 1 -- 172.21.15.19:0/1939432603 wait complete. 2024-04-25T11:35:34.526 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2024-04-25T11:35:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.721+0000 7f61f6cf6f40 1 Processor -- start 2024-04-25T11:35:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.721+0000 7f61f6cf6f40 1 -- start start 2024-04-25T11:35:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f6cf6f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d40030f0 0x55dc7cb1d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f6cf6f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc7cba7860 0x55dc7cba7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f6cf6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cb2eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f6cf6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc7ca92d20 con 0x55dc7cb2eab0 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f6cf6f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dc7ca96d60 con 0x55dc7cba7860 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f6cf6f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55dc7cb353f0 con 0x7f61d40030f0 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cb2eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cb2eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39828/0 (socket says 172.21.15.19:39828) 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.723+0000 7f61f0800640 1 -- 172.21.15.19:0/1669265254 learned_addr learned my addr 172.21.15.19:0/1669265254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61efe00640 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d40030f0 0x55dc7cb1d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61ef400640 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc7cba7860 0x55dc7cba7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61f0800640 1 -- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d40030f0 msgr2=0x55dc7cb1d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61f0800640 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d40030f0 0x55dc7cb1d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61f0800640 1 -- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc7cba7860 msgr2=0x55dc7cba7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61f0800640 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc7cba7860 0x55dc7cba7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61f0800640 1 -- 172.21.15.19:0/1669265254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7cba81b0 con 0x55dc7cb2eab0 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61f0800640 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cb2eec0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x55dc7cb35fb0 tx=0x7f61e8003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a5bed492935290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61ee000640 1 -- 172.21.15.19:0/1669265254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8003480 con 0x55dc7cb2eab0 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61ee000640 1 -- 172.21.15.19:0/1669265254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e8001610 con 0x55dc7cb2eab0 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.724+0000 7f61ee000640 1 -- 172.21.15.19:0/1669265254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e80019e0 con 0x55dc7cb2eab0 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/1669265254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 msgr2=0x55dc7cb2eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cb2eec0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x55dc7cb35fb0 tx=0x7f61e8003a00 comp rx=0 tx=0).stop 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/1669265254 shutdown_connections 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dc7cba7860 0x55dc7cba7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cb2eec0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/1669265254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61d40030f0 0x55dc7cb1d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/1669265254 >> 172.21.15.19:0/1669265254 conn(0x55dc7ca0eac0 msgr2=0x55dc7cb1e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/1669265254 shutdown_connections 2024-04-25T11:35:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.725+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/1669265254 wait complete. 2024-04-25T11:35:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.727+0000 7f61f6cf6f40 1 Processor -- start 2024-04-25T11:35:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.727+0000 7f61f6cf6f40 1 -- start start 2024-04-25T11:35:34.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.728+0000 7f61f6cf6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cbc48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.728+0000 7f61f6cf6f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dc7cba7860 0x55dc7cba6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.728+0000 7f61f6cf6f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40030f0 0x55dc7cb2eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.728+0000 7f61f6cf6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dc7cb1cba0 con 0x55dc7cb2eab0 2024-04-25T11:35:34.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.728+0000 7f61f6cf6f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dc7cb1c5e0 con 0x7f61d40030f0 2024-04-25T11:35:34.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.728+0000 7f61f6cf6f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55dc7cb1c8c0 con 0x55dc7cba7860 2024-04-25T11:35:34.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.728+0000 7f61efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cbc48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cbc48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39842/0 (socket says 172.21.15.19:39842) 2024-04-25T11:35:34.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 -- 172.21.15.19:0/45662696 learned_addr learned my addr 172.21.15.19:0/45662696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 -- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dc7cba7860 msgr2=0x55dc7cba6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dc7cba7860 0x55dc7cba6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 -- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40030f0 msgr2=0x55dc7cb2eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61d40030f0 0x55dc7cb2eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 -- 172.21.15.19:0/45662696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7cb32620 con 0x55dc7cb2eab0 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.729+0000 7f61efe00640 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cbc48e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009dd0 tx=0x7f61e0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.730+0000 7f61ecc00640 1 -- 172.21.15.19:0/45662696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0016440 con 0x55dc7cb2eab0 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.730+0000 7f61ecc00640 1 -- 172.21.15.19:0/45662696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e00165e0 con 0x55dc7cb2eab0 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.730+0000 7f61ecc00640 1 -- 172.21.15.19:0/45662696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00169b0 con 0x55dc7cb2eab0 2024-04-25T11:35:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.730+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dc7cb32900 con 0x55dc7cb2eab0 2024-04-25T11:35:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.730+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7cb1d330 con 0x55dc7cb2eab0 2024-04-25T11:35:34.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.732+0000 7f61ecc00640 1 -- 172.21.15.19:0/45662696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61e001f440 con 0x55dc7cb2eab0 2024-04-25T11:35:34.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.733+0000 7f61ecc00640 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61d0072e90 0x7f61d00753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.733+0000 7f61ecc00640 1 -- 172.21.15.19:0/45662696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f61e0093a50 con 0x55dc7cb2eab0 2024-04-25T11:35:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.734+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dc7cbc2590 0x55dc7cbc29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.734+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55dc7cba6f20 con 0x55dc7cbc2590 2024-04-25T11:35:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.734+0000 7f61f0800640 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dc7cbc2590 0x55dc7cbc29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.734+0000 7f61ef400640 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61d0072e90 0x7f61d00753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.734+0000 7f61f0800640 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dc7cbc2590 0x55dc7cbc29a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f0800640 1 -- 172.21.15.19:0/45662696 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f61e801c040 con 0x55dc7cbc2590 2024-04-25T11:35:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dc7cbc2590 msgr2=0x55dc7cbc29a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dc7cbc2590 0x55dc7cbc29a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61d0072e90 msgr2=0x7f61d00753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61d0072e90 0x7f61d00753a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x55dc7cb35cb0 tx=0x7f61dc050000 comp rx=0 tx=0).stop 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 msgr2=0x55dc7cbc48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 --2- 172.21.15.19:0/45662696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dc7cb2eab0 0x55dc7cbc48e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009dd0 tx=0x7f61e0009ec0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f0800640 1 -- 172.21.15.19:0/45662696 reap_dead start 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 shutdown_connections 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 >> 172.21.15.19:0/45662696 conn(0x55dc7ca0eac0 msgr2=0x55dc7cb1e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 shutdown_connections 2024-04-25T11:35:34.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.735+0000 7f61f6cf6f40 1 -- 172.21.15.19:0/45662696 wait complete. 2024-04-25T11:35:34.736 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-04-25T11:35:34.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:35:34.801 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:35:34.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.917+0000 7f4de8428f40 1 Processor -- start 2024-04-25T11:35:34.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.917+0000 7f4de8428f40 1 -- start start 2024-04-25T11:35:34.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de8428f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad48ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de8428f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad48f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de8428f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 0x55a2ad3f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de8428f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a2ad4093f0 con 0x55a2ad3c05f0 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de8428f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a2ad408e70 con 0x55a2ad499dd0 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de8428f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a2ad4096d0 con 0x55a2ad3c01e0 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad48f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad48f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39854/0 (socket says 172.21.15.19:39854) 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3914761536 learned_addr learned my addr 172.21.15.19:0/3914761536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de1600640 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad48ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 msgr2=0x55a2ad48ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad48ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 msgr2=0x55a2ad3f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 0x55a2ad3f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4ddbe00640 1 -- 172.21.15.19:0/3914761536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2ad2e9b30 con 0x55a2ad3c05f0 2024-04-25T11:35:34.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.919+0000 7f4de1600640 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad48ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:34.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.920+0000 7f4ddbe00640 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad48f970 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc025c20 tx=0x7f4dcc04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fae5e32229cdd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.920+0000 7f4ddb400640 1 -- 172.21.15.19:0/3914761536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc062070 con 0x55a2ad3c05f0 2024-04-25T11:35:34.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.920+0000 7f4ddb400640 1 -- 172.21.15.19:0/3914761536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0547f0 con 0x55a2ad3c05f0 2024-04-25T11:35:34.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.920+0000 7f4ddb400640 1 -- 172.21.15.19:0/3914761536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc059040 con 0x55a2ad3c05f0 2024-04-25T11:35:34.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 -- 172.21.15.19:0/3914761536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 msgr2=0x55a2ad48f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad48f970 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc025c20 tx=0x7f4dcc04e000 comp rx=0 tx=0).stop 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 -- 172.21.15.19:0/3914761536 shutdown_connections 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 0x55a2ad3f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad48f970 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 --2- 172.21.15.19:0/3914761536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad48ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 -- 172.21.15.19:0/3914761536 >> 172.21.15.19:0/3914761536 conn(0x55a2ad2e2ac0 msgr2=0x55a2ad3f05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 -- 172.21.15.19:0/3914761536 shutdown_connections 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.921+0000 7f4de8428f40 1 -- 172.21.15.19:0/3914761536 wait complete. 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.922+0000 7f4de8428f40 1 Processor -- start 2024-04-25T11:35:34.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.922+0000 7f4de8428f40 1 -- start start 2024-04-25T11:35:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de8428f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad495150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de8428f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad485e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de8428f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 0x55a2ad3f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de8428f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a2ad48f3d0 con 0x55a2ad3c05f0 2024-04-25T11:35:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de8428f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a2ad3f2bb0 con 0x55a2ad499dd0 2024-04-25T11:35:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de8428f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a2ad3f28d0 con 0x55a2ad3c01e0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad485e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad485e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39868/0 (socket says 172.21.15.19:39868) 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 -- 172.21.15.19:0/1105656515 learned_addr learned my addr 172.21.15.19:0/1105656515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de1600640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad495150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 -- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 msgr2=0x55a2ad495150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a2ad3c01e0 0x55a2ad495150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 -- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 msgr2=0x55a2ad3f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4de2000640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 0x55a2ad3f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a2ad499dd0 0x55a2ad3f1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.924+0000 7f4ddbe00640 1 -- 172.21.15.19:0/1105656515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2ad498230 con 0x55a2ad3c05f0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.925+0000 7f4ddbe00640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad485e20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc025bf0 tx=0x7f4dcc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.925+0000 7f4dda000640 1 -- 172.21.15.19:0/1105656515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc06a070 con 0x55a2ad3c05f0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.925+0000 7f4dda000640 1 -- 172.21.15.19:0/1105656515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc054d90 con 0x55a2ad3c05f0 2024-04-25T11:35:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.925+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a2ad494af0 con 0x55a2ad3c05f0 2024-04-25T11:35:34.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.925+0000 7f4dda000640 1 -- 172.21.15.19:0/1105656515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc062070 con 0x55a2ad3c05f0 2024-04-25T11:35:34.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.925+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ad486360 con 0x55a2ad3c05f0 2024-04-25T11:35:34.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.927+0000 7f4dda000640 1 -- 172.21.15.19:0/1105656515 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4dcc06d070 con 0x55a2ad3c05f0 2024-04-25T11:35:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.928+0000 7f4dda000640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dc0072ec0 0x7f4dc00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.928+0000 7f4dda000640 1 -- 172.21.15.19:0/1105656515 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f4dcc0d7950 con 0x55a2ad3c05f0 2024-04-25T11:35:34.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.929+0000 7f4de8428f40 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a2ad52d390 0x55a2ad52d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:34.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.929+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55a2ad40c8b0 con 0x55a2ad52d390 2024-04-25T11:35:34.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.929+0000 7f4de2000640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a2ad52d390 0x55a2ad52d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.929+0000 7f4de1600640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dc0072ec0 0x7f4dc00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:34.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.930+0000 7f4de2000640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a2ad52d390 0x55a2ad52d7a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.930+0000 7f4de2000640 1 -- 172.21.15.19:0/1105656515 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv6 ondisk = 0) ==== 154+0+20 (crc 0 0 0) 0x7f4ddc00d490 con 0x55a2ad52d390 2024-04-25T11:35:34.918 INFO:teuthology.orchestra.run.smithi019.stdout:subdir_head 2024-04-25T11:35:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.930+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a2ad52d390 msgr2=0x55a2ad52d7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.930+0000 7f4de8428f40 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a2ad52d390 0x55a2ad52d7a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.930+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dc0072ec0 msgr2=0x7f4dc00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.930+0000 7f4de1600640 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dc0072ec0 0x7f4dc00753d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4005fd0 tx=0x7f4dd4005a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.930+0000 7f4de8428f40 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4dc0072ec0 0x7f4dc00753d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4005fd0 tx=0x7f4dd4005a00 comp rx=0 tx=0).stop 2024-04-25T11:35:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.931+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 msgr2=0x55a2ad485e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.931+0000 7f4de8428f40 1 --2- 172.21.15.19:0/1105656515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2ad3c05f0 0x55a2ad485e20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc025bf0 tx=0x7f4dcc002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.931+0000 7f4de2000640 1 -- 172.21.15.19:0/1105656515 reap_dead start 2024-04-25T11:35:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.931+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 shutdown_connections 2024-04-25T11:35:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.931+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 >> 172.21.15.19:0/1105656515 conn(0x55a2ad2e2ac0 msgr2=0x55a2ad48f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.931+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 shutdown_connections 2024-04-25T11:35:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:34.931+0000 7f4de8428f40 1 -- 172.21.15.19:0/1105656515 wait complete. 2024-04-25T11:35:34.933 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-04-25T11:35:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.168+0000 7febd0c49f40 1 Processor -- start 2024-04-25T11:35:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.168+0000 7febd0c49f40 1 -- start start 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febd0c49f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 0x55924f1a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febd0c49f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febd0c49f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 0x55924f19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febd0c49f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55924f076d20 con 0x55924f0cf830 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febd0c49f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55924f07ad60 con 0x55924f1a7700 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febd0c49f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55924f1193f0 con 0x7febb00030f0 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39878/0 (socket says 172.21.15.19:39878) 2024-04-25T11:35:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 -- 172.21.15.19:0/1722027309 learned_addr learned my addr 172.21.15.19:0/1722027309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febca800640 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 0x55924f19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9e00640 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 0x55924f1a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 -- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 msgr2=0x55924f1a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 0x55924f1a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 -- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 msgr2=0x55924f19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 0x55924f19dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.170+0000 7febc9400640 1 -- 172.21.15.19:0/1722027309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924f0ce840 con 0x55924f0cf830 2024-04-25T11:35:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.171+0000 7febc9400640 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a71c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7febb4003040 tx=0x7febb404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfc8251b7b01eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.171+0000 7febc3e00640 1 -- 172.21.15.19:0/1722027309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4063070 con 0x55924f0cf830 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.171+0000 7febc3e00640 1 -- 172.21.15.19:0/1722027309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb40549a0 con 0x55924f0cf830 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.171+0000 7febc3e00640 1 -- 172.21.15.19:0/1722027309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4059040 con 0x55924f0cf830 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 -- 172.21.15.19:0/1722027309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 msgr2=0x55924f1a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a71c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7febb4003040 tx=0x7febb404e000 comp rx=0 tx=0).stop 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 -- 172.21.15.19:0/1722027309 shutdown_connections 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 0x55924f19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a71c0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 --2- 172.21.15.19:0/1722027309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 0x55924f1a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 -- 172.21.15.19:0/1722027309 >> 172.21.15.19:0/1722027309 conn(0x55924eff2ac0 msgr2=0x55924f0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 -- 172.21.15.19:0/1722027309 shutdown_connections 2024-04-25T11:35:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.172+0000 7febd0c49f40 1 -- 172.21.15.19:0/1722027309 wait complete. 2024-04-25T11:35:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.173+0000 7febd0c49f40 1 Processor -- start 2024-04-25T11:35:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.173+0000 7febd0c49f40 1 -- start start 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febd0c49f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febd0c49f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 0x55924f195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febd0c49f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 0x55924f11c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febd0c49f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55924f1a0c40 con 0x55924f0cf830 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febd0c49f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55924f115860 con 0x55924f1a7700 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febd0c49f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55924f1a4b60 con 0x7febb00030f0 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39888/0 (socket says 172.21.15.19:39888) 2024-04-25T11:35:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 -- 172.21.15.19:0/2960896933 learned_addr learned my addr 172.21.15.19:0/2960896933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9400640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 0x55924f195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febca800640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 0x55924f11c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 -- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 msgr2=0x55924f11c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febb00030f0 0x55924f11c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 -- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 msgr2=0x55924f195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55924f1a7700 0x55924f195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.175+0000 7febc9e00640 1 -- 172.21.15.19:0/2960896933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924f1a4db0 con 0x55924f0cf830 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.176+0000 7febc9e00640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a42f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7febbc00d4a0 tx=0x7febbc00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.176+0000 7febc2a00640 1 -- 172.21.15.19:0/2960896933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc018070 con 0x55924f0cf830 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.176+0000 7febc2a00640 1 -- 172.21.15.19:0/2960896933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc003670 con 0x55924f0cf830 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.176+0000 7febc2a00640 1 -- 172.21.15.19:0/2960896933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc0062e0 con 0x55924f0cf830 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.176+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55924f1a5090 con 0x55924f0cf830 2024-04-25T11:35:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.176+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924f1a5650 con 0x55924f0cf830 2024-04-25T11:35:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.179+0000 7febc2a00640 1 -- 172.21.15.19:0/2960896933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febbc01b070 con 0x55924f0cf830 2024-04-25T11:35:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.180+0000 7febc2a00640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feba8072e90 0x7feba80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.180+0000 7febc2a00640 1 -- 172.21.15.19:0/2960896933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7febbc093880 con 0x55924f0cf830 2024-04-25T11:35:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.180+0000 7febd0c49f40 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55924f1a2b60 0x55924f1a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.180+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55924f114520 con 0x55924f1a2b60 2024-04-25T11:35:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.180+0000 7febc9400640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feba8072e90 0x7feba80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.181+0000 7febca800640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55924f1a2b60 0x55924f1a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.182+0000 7febc9400640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feba8072e90 0x7feba80753a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7febb4000f60 tx=0x7febb4051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.182+0000 7febca800640 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55924f1a2b60 0x55924f1a2f70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febca800640 1 -- 172.21.15.19:0/2960896933 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x7febc40073f0 con 0x55924f1a2b60 2024-04-25T11:35:35.171 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/10000000000.00000000 mtime 2024-04-25T11:35:27.000000+0000, size 0 2024-04-25T11:35:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55924f1a2b60 msgr2=0x55924f1a2f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febd0c49f40 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55924f1a2b60 0x55924f1a2f70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feba8072e90 msgr2=0x7feba80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febd0c49f40 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feba8072e90 0x7feba80753a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7febb4000f60 tx=0x7febb4051000 comp rx=0 tx=0).stop 2024-04-25T11:35:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 msgr2=0x55924f1a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febd0c49f40 1 --2- 172.21.15.19:0/2960896933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55924f0cf830 0x55924f1a42f0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7febbc00d4a0 tx=0x7febbc00d960 comp rx=0 tx=0).stop 2024-04-25T11:35:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.183+0000 7febca800640 1 -- 172.21.15.19:0/2960896933 reap_dead start 2024-04-25T11:35:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.184+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 shutdown_connections 2024-04-25T11:35:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.184+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 >> 172.21.15.19:0/2960896933 conn(0x55924eff2ac0 msgr2=0x55924f0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.184+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 shutdown_connections 2024-04-25T11:35:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.184+0000 7febd0c49f40 1 -- 172.21.15.19:0/2960896933 wait complete. 2024-04-25T11:35:35.183 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2024-04-25T11:35:35.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.380+0000 7fb6e5320f40 1 Processor -- start 2024-04-25T11:35:35.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.380+0000 7fb6e5320f40 1 -- start start 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6e5320f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4003630 0x5557caca54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6e5320f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557cad399a0 0x5557cad380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6e5320f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557caca6840 0x5557caca6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6e5320f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557cac0cd20 con 0x5557cad399a0 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6e5320f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5557cac67af0 con 0x5557caca6840 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6e5320f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5557caca5e60 con 0x7fb6c4003630 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6dd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557cad399a0 0x5557cad380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6dd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557cad399a0 0x5557cad380f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39906/0 (socket says 172.21.15.19:39906) 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.382+0000 7fb6dd200640 1 -- 172.21.15.19:0/1911710095 learned_addr learned my addr 172.21.15.19:0/1911710095 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6ddc00640 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4003630 0x5557caca54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6dd200640 1 -- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4003630 msgr2=0x5557caca54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6de600640 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557caca6840 0x5557caca6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6dd200640 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4003630 0x5557caca54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6dd200640 1 -- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557caca6840 msgr2=0x5557caca6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6dd200640 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557caca6840 0x5557caca6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6dd200640 1 -- 172.21.15.19:0/1911710095 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557cad38920 con 0x5557cad399a0 2024-04-25T11:35:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6dd200640 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557cad399a0 0x5557cad380f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009760 tx=0x7fb6c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297b1a31fbfdd64d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6d7400640 1 -- 172.21.15.19:0/1911710095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8008870 con 0x5557cad399a0 2024-04-25T11:35:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6d7400640 1 -- 172.21.15.19:0/1911710095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c8018070 con 0x5557cad399a0 2024-04-25T11:35:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.383+0000 7fb6d7400640 1 -- 172.21.15.19:0/1911710095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8008c50 con 0x5557cad399a0 2024-04-25T11:35:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1911710095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557cad399a0 msgr2=0x5557cad380f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557cad399a0 0x5557cad380f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009760 tx=0x7fb6c8005f90 comp rx=0 tx=0).stop 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1911710095 shutdown_connections 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5557caca6840 0x5557caca6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557cad399a0 0x5557cad380f0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1911710095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4003630 0x5557caca54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1911710095 >> 172.21.15.19:0/1911710095 conn(0x5557cab88ac0 msgr2=0x5557cac92e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1911710095 shutdown_connections 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.384+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1911710095 wait complete. 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.385+0000 7fb6e5320f40 1 Processor -- start 2024-04-25T11:35:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.386+0000 7fb6e5320f40 1 -- start start 2024-04-25T11:35:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6e5320f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557caca6840 0x5557cad30430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6e5320f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557cad399a0 0x5557caca94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6e5320f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6c4003630 0x5557caca9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6e5320f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557cad38f10 con 0x5557caca6840 2024-04-25T11:35:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6e5320f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5557cad395d0 con 0x7fb6c4003630 2024-04-25T11:35:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6e5320f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5557cad38bc0 con 0x5557cad399a0 2024-04-25T11:35:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6ddc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557caca6840 0x5557cad30430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6ddc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557caca6840 0x5557cad30430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39918/0 (socket says 172.21.15.19:39918) 2024-04-25T11:35:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.387+0000 7fb6ddc00640 1 -- 172.21.15.19:0/1824291991 learned_addr learned my addr 172.21.15.19:0/1824291991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6dd200640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557cad399a0 0x5557caca94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6ddc00640 1 -- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557cad399a0 msgr2=0x5557caca94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6ddc00640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5557cad399a0 0x5557caca94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6ddc00640 1 -- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6c4003630 msgr2=0x5557caca9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6ddc00640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6c4003630 0x5557caca9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6ddc00640 1 -- 172.21.15.19:0/1824291991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557cad37d00 con 0x5557caca6840 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6ddc00640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557caca6840 0x5557cad30430 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0010310 tx=0x7fb6d00107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.388+0000 7fb6d6000640 1 -- 172.21.15.19:0/1824291991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0018020 con 0x5557caca6840 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.389+0000 7fb6d6000640 1 -- 172.21.15.19:0/1824291991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d0014690 con 0x5557caca6840 2024-04-25T11:35:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.389+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5557caca7de0 con 0x5557caca6840 2024-04-25T11:35:35.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.389+0000 7fb6d6000640 1 -- 172.21.15.19:0/1824291991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0014a60 con 0x5557caca6840 2024-04-25T11:35:35.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.389+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557cadd2d00 con 0x5557caca6840 2024-04-25T11:35:35.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.391+0000 7fb6d6000640 1 -- 172.21.15.19:0/1824291991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb6d001b020 con 0x5557caca6840 2024-04-25T11:35:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.392+0000 7fb6d6000640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6bc072e90 0x7fb6bc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.392+0000 7fb6d6000640 1 -- 172.21.15.19:0/1824291991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fb6d0094350 con 0x5557caca6840 2024-04-25T11:35:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.392+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5557cad30c50 0x5557cadd3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.392+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x5557cacaaa30 con 0x5557cad30c50 2024-04-25T11:35:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.393+0000 7fb6dd200640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6bc072e90 0x7fb6bc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.393+0000 7fb6de600640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5557cad30c50 0x5557cadd3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.394+0000 7fb6dd200640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6bc072e90 0x7fb6bc0753a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80030a0 tx=0x7fb6c8012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.394+0000 7fb6de600640 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5557cad30c50 0x5557cadd3ec0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.383 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/600.00000000 mtime 2024-04-25T11:35:14.000000+0000, size 0 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.395+0000 7fb6de600640 1 -- 172.21.15.19:0/1824291991 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fb6d8001430 con 0x5557cad30c50 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.395+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5557cad30c50 msgr2=0x5557cadd3ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.395+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5557cad30c50 0x5557cadd3ec0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6bc072e90 msgr2=0x7fb6bc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6bc072e90 0x7fb6bc0753a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80030a0 tx=0x7fb6c8012000 comp rx=0 tx=0).stop 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557caca6840 msgr2=0x5557cad30430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 --2- 172.21.15.19:0/1824291991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557caca6840 0x5557cad30430 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0010310 tx=0x7fb6d00107d0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6de600640 1 -- 172.21.15.19:0/1824291991 reap_dead start 2024-04-25T11:35:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 shutdown_connections 2024-04-25T11:35:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 >> 172.21.15.19:0/1824291991 conn(0x5557cab88ac0 msgr2=0x5557cac913c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 shutdown_connections 2024-04-25T11:35:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.396+0000 7fb6e5320f40 1 -- 172.21.15.19:0/1824291991 wait complete. 2024-04-25T11:35:35.398 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2024-04-25T11:35:35.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.587+0000 7f379acaff40 1 Processor -- start 2024-04-25T11:35:35.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.587+0000 7f379acaff40 1 -- start start 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f379acaff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd81e0 0x558f43cae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f379acaff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43bd85f0 0x558f43cab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f379acaff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43cb1dd0 0x558f43caefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f379acaff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f43c20e70 con 0x558f43cb1dd0 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f379acaff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558f43b7ed20 con 0x558f43bd81e0 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f379acaff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x558f43c213f0 con 0x558f43bd85f0 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f3794800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43cb1dd0 0x558f43caefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f3794800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43cb1dd0 0x558f43caefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39936/0 (socket says 172.21.15.19:39936) 2024-04-25T11:35:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f3794800640 1 -- 172.21.15.19:0/1803131506 learned_addr learned my addr 172.21.15.19:0/1803131506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f3793e00640 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd81e0 0x558f43cae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f3794800640 1 -- 172.21.15.19:0/1803131506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43bd85f0 msgr2=0x558f43cab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f3794800640 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43bd85f0 0x558f43cab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.589+0000 7f3794800640 1 -- 172.21.15.19:0/1803131506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd81e0 msgr2=0x558f43cae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.590+0000 7f3794800640 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd81e0 0x558f43cae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.590+0000 7f3794800640 1 -- 172.21.15.19:0/1803131506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f43cad1b0 con 0x558f43cb1dd0 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.590+0000 7f3794800640 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43cb1dd0 0x558f43caefd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f378c00a4b0 tx=0x7f378c00a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9720f86c0ea979c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.590+0000 7f3792a00640 1 -- 172.21.15.19:0/1803131506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c015070 con 0x558f43cb1dd0 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.590+0000 7f3792a00640 1 -- 172.21.15.19:0/1803131506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c00e730 con 0x558f43cb1dd0 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.590+0000 7f3792a00640 1 -- 172.21.15.19:0/1803131506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c00eb20 con 0x558f43cb1dd0 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 -- 172.21.15.19:0/1803131506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43cb1dd0 msgr2=0x558f43caefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43cb1dd0 0x558f43caefd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f378c00a4b0 tx=0x7f378c00a970 comp rx=0 tx=0).stop 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 -- 172.21.15.19:0/1803131506 shutdown_connections 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43cb1dd0 0x558f43caefd0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43bd85f0 0x558f43cab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 --2- 172.21.15.19:0/1803131506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd81e0 0x558f43cae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 -- 172.21.15.19:0/1803131506 >> 172.21.15.19:0/1803131506 conn(0x558f43afaac0 msgr2=0x558f43c07570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 -- 172.21.15.19:0/1803131506 shutdown_connections 2024-04-25T11:35:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.591+0000 7f379acaff40 1 -- 172.21.15.19:0/1803131506 wait complete. 2024-04-25T11:35:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.592+0000 7f379acaff40 1 Processor -- start 2024-04-25T11:35:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.592+0000 7f379acaff40 1 -- start start 2024-04-25T11:35:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.594+0000 7f379acaff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43bd81e0 0x558f43ca51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.594+0000 7f379acaff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd85f0 0x558f43caf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.594+0000 7f379acaff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43cb1dd0 0x558f43ca95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.594+0000 7f379acaff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f43cadf50 con 0x558f43bd81e0 2024-04-25T11:35:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.594+0000 7f379acaff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558f43c0a640 con 0x558f43bd85f0 2024-04-25T11:35:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.594+0000 7f379acaff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x558f43caa500 con 0x558f43cb1dd0 2024-04-25T11:35:35.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.594+0000 7f3793e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43bd81e0 0x558f43ca51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43bd81e0 0x558f43ca51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39950/0 (socket says 172.21.15.19:39950) 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 -- 172.21.15.19:0/4236278670 learned_addr learned my addr 172.21.15.19:0/4236278670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3794800640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43cb1dd0 0x558f43ca95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f378b400640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd85f0 0x558f43caf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 -- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43cb1dd0 msgr2=0x558f43ca95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558f43cb1dd0 0x558f43ca95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 -- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd85f0 msgr2=0x558f43caf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd85f0 0x558f43caf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 -- 172.21.15.19:0/4236278670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f43cad550 con 0x558f43bd81e0 2024-04-25T11:35:35.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f378b400640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558f43bd85f0 0x558f43caf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:35.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.595+0000 7f3793e00640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43bd81e0 0x558f43ca51b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3784025d20 tx=0x7f3784002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.596+0000 7f3791600640 1 -- 172.21.15.19:0/4236278670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378406b070 con 0x558f43bd81e0 2024-04-25T11:35:35.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.596+0000 7f3791600640 1 -- 172.21.15.19:0/4236278670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3784053920 con 0x558f43bd81e0 2024-04-25T11:35:35.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.596+0000 7f3791600640 1 -- 172.21.15.19:0/4236278670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784053cf0 con 0x558f43bd81e0 2024-04-25T11:35:35.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.596+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f43ca6d20 con 0x558f43bd81e0 2024-04-25T11:35:35.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.596+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f43cadad0 con 0x558f43bd81e0 2024-04-25T11:35:35.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.599+0000 7f3791600640 1 -- 172.21.15.19:0/4236278670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f378406e070 con 0x558f43bd81e0 2024-04-25T11:35:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.599+0000 7f3791600640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3770072ee0 0x7f37700753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.600+0000 7f3791600640 1 -- 172.21.15.19:0/4236278670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f37840d7c20 con 0x558f43bd81e0 2024-04-25T11:35:35.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.600+0000 7f379acaff40 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558f43ca88b0 0x558f43d3f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.600+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x558f43bd7b60 con 0x558f43ca88b0 2024-04-25T11:35:35.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.600+0000 7f3794800640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558f43ca88b0 0x558f43d3f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.600+0000 7f378b400640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3770072ee0 0x7f37700753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f3794800640 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558f43ca88b0 0x558f43d3f9a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f3794800640 1 -- 172.21.15.19:0/4236278670 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv15 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f378c00e070 con 0x558f43ca88b0 2024-04-25T11:35:35.589 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/1.00000000.inode mtime 2024-04-25T11:35:27.000000+0000, size 582 2024-04-25T11:35:35.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558f43ca88b0 msgr2=0x558f43d3f9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f379acaff40 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558f43ca88b0 0x558f43d3f9a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3770072ee0 msgr2=0x7f37700753f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f379acaff40 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3770072ee0 0x7f37700753f0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x558f43ca8ff0 tx=0x7f377c005690 comp rx=0 tx=0).stop 2024-04-25T11:35:35.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43bd81e0 msgr2=0x558f43ca51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f379acaff40 1 --2- 172.21.15.19:0/4236278670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f43bd81e0 0x558f43ca51b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3784025d20 tx=0x7f3784002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.601+0000 7f3794800640 1 -- 172.21.15.19:0/4236278670 reap_dead start 2024-04-25T11:35:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.602+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 shutdown_connections 2024-04-25T11:35:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.602+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 >> 172.21.15.19:0/4236278670 conn(0x558f43afaac0 msgr2=0x558f43c07570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.602+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 shutdown_connections 2024-04-25T11:35:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.602+0000 7f379acaff40 1 -- 172.21.15.19:0/4236278670 wait complete. 2024-04-25T11:35:35.600 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2024-04-25T11:35:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.770+0000 7f95e9778f40 1 Processor -- start 2024-04-25T11:35:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.770+0000 7f95e9778f40 1 -- start start 2024-04-25T11:35:35.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.771+0000 7f95e9778f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9361e0 0x555e1aa0a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.771+0000 7f95e9778f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9365f0 0x555e1aa0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.771+0000 7f95e9778f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1aa0e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.771+0000 7f95e9778f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555e1a97ee70 con 0x555e1aa0fdd0 2024-04-25T11:35:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.771+0000 7f95e9778f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555e1a8dcd20 con 0x555e1a9361e0 2024-04-25T11:35:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.771+0000 7f95e9778f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x555e1a97f3f0 con 0x555e1a9365f0 2024-04-25T11:35:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1aa0e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1aa0e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39968/0 (socket says 172.21.15.19:39968) 2024-04-25T11:35:35.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 -- 172.21.15.19:0/1336032305 learned_addr learned my addr 172.21.15.19:0/1336032305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 -- 172.21.15.19:0/1336032305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9365f0 msgr2=0x555e1aa0e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 --2- 172.21.15.19:0/1336032305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9365f0 0x555e1aa0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 -- 172.21.15.19:0/1336032305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9361e0 msgr2=0x555e1aa0a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 --2- 172.21.15.19:0/1336032305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9361e0 0x555e1aa0a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 -- 172.21.15.19:0/1336032305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1a964980 con 0x555e1aa0fdd0 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.772+0000 7f95e3200640 1 --2- 172.21.15.19:0/1336032305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1aa0e640 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00a370 tx=0x7f95dc00a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d060224eed5bbe11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e0a00640 1 -- 172.21.15.19:0/1336032305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc010070 con 0x555e1aa0fdd0 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e0a00640 1 -- 172.21.15.19:0/1336032305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc00c8f0 con 0x555e1aa0fdd0 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e0a00640 1 -- 172.21.15.19:0/1336032305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00cce0 con 0x555e1aa0fdd0 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 -- 172.21.15.19:0/1336032305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 msgr2=0x555e1aa0e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 --2- 172.21.15.19:0/1336032305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1aa0e640 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00a370 tx=0x7f95dc00a830 comp rx=0 tx=0).stop 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 -- 172.21.15.19:0/1336032305 shutdown_connections 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 --2- 172.21.15.19:0/1336032305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1aa0e640 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 --2- 172.21.15.19:0/1336032305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9365f0 0x555e1aa0e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 --2- 172.21.15.19:0/1336032305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9361e0 0x555e1aa0a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 -- 172.21.15.19:0/1336032305 >> 172.21.15.19:0/1336032305 conn(0x555e1a858ac0 msgr2=0x555e1aa0b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.773+0000 7f95e9778f40 1 -- 172.21.15.19:0/1336032305 shutdown_connections 2024-04-25T11:35:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.774+0000 7f95e9778f40 1 -- 172.21.15.19:0/1336032305 wait complete. 2024-04-25T11:35:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.775+0000 7f95e9778f40 1 Processor -- start 2024-04-25T11:35:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.775+0000 7f95e9778f40 1 -- start start 2024-04-25T11:35:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.776+0000 7f95e9778f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9361e0 0x555e1a9829e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.776+0000 7f95e9778f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9365f0 0x555e1aa06260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.776+0000 7f95e9778f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1a97acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.776+0000 7f95e9778f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555e1a964730 con 0x555e1aa0fdd0 2024-04-25T11:35:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.776+0000 7f95e9778f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555e1a968180 con 0x555e1a9365f0 2024-04-25T11:35:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.776+0000 7f95e9778f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x555e1aa0a890 con 0x555e1a9361e0 2024-04-25T11:35:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.776+0000 7f95e3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1a97acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1a97acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39972/0 (socket says 172.21.15.19:39972) 2024-04-25T11:35:35.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 -- 172.21.15.19:0/3134528807 learned_addr learned my addr 172.21.15.19:0/3134528807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e1e00640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9365f0 0x555e1aa06260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 -- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9361e0 msgr2=0x555e1a9829e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e2800640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9361e0 0x555e1a9829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9361e0 0x555e1a9829e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 -- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9365f0 msgr2=0x555e1aa06260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9365f0 0x555e1aa06260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 -- 172.21.15.19:0/3134528807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1a982f20 con 0x555e1aa0fdd0 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e2800640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555e1a9361e0 0x555e1a9829e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e1e00640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555e1a9365f0 0x555e1aa06260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.777+0000 7f95e3200640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1a97acc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00cfd0 tx=0x7f95dc002f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.778+0000 7f95db400640 1 -- 172.21.15.19:0/3134528807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc025070 con 0x555e1aa0fdd0 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.778+0000 7f95db400640 1 -- 172.21.15.19:0/3134528807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc015c10 con 0x555e1aa0fdd0 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.778+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e1aa0a5a0 con 0x555e1aa0fdd0 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.778+0000 7f95db400640 1 -- 172.21.15.19:0/3134528807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc010050 con 0x555e1aa0fdd0 2024-04-25T11:35:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.778+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1aa9e940 con 0x555e1aa0fdd0 2024-04-25T11:35:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.780+0000 7f95db400640 1 -- 172.21.15.19:0/3134528807 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f95dc028070 con 0x555e1aa0fdd0 2024-04-25T11:35:35.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.781+0000 7f95db400640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95c0072ec0 0x7f95c00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.781+0000 7f95db400640 1 -- 172.21.15.19:0/3134528807 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f95dc095780 con 0x555e1aa0fdd0 2024-04-25T11:35:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.782+0000 7f95e9778f40 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x555e1aa067a0 0x555e1aa9b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.782+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x555e1a9782a0 con 0x555e1aa067a0 2024-04-25T11:35:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.782+0000 7f95e1e00640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x555e1aa067a0 0x555e1aa9b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.782+0000 7f95e2800640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95c0072ec0 0x7f95c00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.783+0000 7f95e1e00640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x555e1aa067a0 0x555e1aa9b9f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e1e00640 1 -- 172.21.15.19:0/3134528807 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f95cc0518e0 con 0x555e1aa067a0 2024-04-25T11:35:35.772 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/200.00000000 mtime 2024-04-25T11:35:27.000000+0000, size 90 2024-04-25T11:35:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x555e1aa067a0 msgr2=0x555e1aa9b9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x555e1aa067a0 0x555e1aa9b9f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e2800640 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95c0072ec0 0x7f95c00753d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b450 tx=0x7f95d4005580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95c0072ec0 msgr2=0x7f95c00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95c0072ec0 0x7f95c00753d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b450 tx=0x7f95d4005580 comp rx=0 tx=0).stop 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 msgr2=0x555e1a97acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 --2- 172.21.15.19:0/3134528807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e1aa0fdd0 0x555e1a97acc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00cfd0 tx=0x7f95dc002f30 comp rx=0 tx=0).stop 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e3200640 1 -- 172.21.15.19:0/3134528807 reap_dead start 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 shutdown_connections 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 >> 172.21.15.19:0/3134528807 conn(0x555e1a858ac0 msgr2=0x555e1a9836c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 shutdown_connections 2024-04-25T11:35:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.784+0000 7f95e9778f40 1 -- 172.21.15.19:0/3134528807 wait complete. 2024-04-25T11:35:35.783 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2024-04-25T11:35:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.971+0000 7fc405dd6f40 1 Processor -- start 2024-04-25T11:35:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.971+0000 7fc405dd6f40 1 -- start start 2024-04-25T11:35:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc405dd6f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b1e0 0x5610c3afaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc405dd6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b5f0 0x5610c3a29560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc405dd6f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 0x5610c3a29aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc405dd6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610c3a73e70 con 0x5610c3a2b5f0 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc405dd6f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5610c39d1d20 con 0x5610c3b04fa0 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc405dd6f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5610c3a743f0 con 0x5610c3a2b1e0 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc3fe600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b5f0 0x5610c3a29560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc3fe600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b5f0 0x5610c3a29560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39986/0 (socket says 172.21.15.19:39986) 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc3fe600640 1 -- 172.21.15.19:0/848702043 learned_addr learned my addr 172.21.15.19:0/848702043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc3ff000640 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b1e0 0x5610c3afaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.973+0000 7fc3ffa00640 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 0x5610c3a29aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fe600640 1 -- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b1e0 msgr2=0x5610c3afaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fe600640 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b1e0 0x5610c3afaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fe600640 1 -- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 msgr2=0x5610c3a29aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fe600640 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 0x5610c3a29aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fe600640 1 -- 172.21.15.19:0/848702043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c3a5a870 con 0x5610c3a2b5f0 2024-04-25T11:35:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fe600640 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b5f0 0x5610c3a29560 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0001640 tx=0x7fc3f0001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182d652eb7e6a70a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fd200640 1 -- 172.21.15.19:0/848702043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0017070 con 0x5610c3a2b5f0 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fd200640 1 -- 172.21.15.19:0/848702043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f00026b0 con 0x5610c3a2b5f0 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.974+0000 7fc3fd200640 1 -- 172.21.15.19:0/848702043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0002aa0 con 0x5610c3a2b5f0 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 -- 172.21.15.19:0/848702043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b5f0 msgr2=0x5610c3a29560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b5f0 0x5610c3a29560 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0001640 tx=0x7fc3f0001b00 comp rx=0 tx=0).stop 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 -- 172.21.15.19:0/848702043 shutdown_connections 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 0x5610c3a29aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b5f0 0x5610c3a29560 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/848702043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b1e0 0x5610c3afaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 -- 172.21.15.19:0/848702043 >> 172.21.15.19:0/848702043 conn(0x5610c394dac0 msgr2=0x5610c3a2a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 -- 172.21.15.19:0/848702043 shutdown_connections 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.975+0000 7fc405dd6f40 1 -- 172.21.15.19:0/848702043 wait complete. 2024-04-25T11:35:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.976+0000 7fc405dd6f40 1 Processor -- start 2024-04-25T11:35:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.976+0000 7fc405dd6f40 1 -- start start 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc405dd6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b1e0 0x5610c3af3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc405dd6f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b5f0 0x5610c3af9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc405dd6f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 0x5610c3af6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc405dd6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610c3a5d120 con 0x5610c3a2b1e0 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc405dd6f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5610c3a5d400 con 0x5610c3b04fa0 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc405dd6f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5610c3b952f0 con 0x5610c3a2b5f0 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc3ff000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b1e0 0x5610c3af3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc3ff000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b1e0 0x5610c3af3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39996/0 (socket says 172.21.15.19:39996) 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.978+0000 7fc3ff000640 1 -- 172.21.15.19:0/4017021352 learned_addr learned my addr 172.21.15.19:0/4017021352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3ffa00640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 0x5610c3af6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3ff000640 1 -- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b5f0 msgr2=0x5610c3af9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3ff000640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5610c3a2b5f0 0x5610c3af9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3ff000640 1 -- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 msgr2=0x5610c3af6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3ff000640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5610c3b04fa0 0x5610c3af6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3ff000640 1 -- 172.21.15.19:0/4017021352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c3af83c0 con 0x5610c3a2b1e0 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3ff000640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b1e0 0x5610c3af3e90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8025ca0 tx=0x7fc3e8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3f7400640 1 -- 172.21.15.19:0/4017021352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8054c30 con 0x5610c3a2b1e0 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3f7400640 1 -- 172.21.15.19:0/4017021352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e8054dd0 con 0x5610c3a2b1e0 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.979+0000 7fc3f7400640 1 -- 172.21.15.19:0/4017021352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e805f660 con 0x5610c3a2b1e0 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.980+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610c3af6140 con 0x5610c3a2b1e0 2024-04-25T11:35:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.980+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c3af6700 con 0x5610c3a2b1e0 2024-04-25T11:35:35.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.982+0000 7fc3f7400640 1 -- 172.21.15.19:0/4017021352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc3e805c040 con 0x5610c3a2b1e0 2024-04-25T11:35:35.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.983+0000 7fc3f7400640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3dc072e90 0x7fc3dc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.983+0000 7fc3f7400640 1 -- 172.21.15.19:0/4017021352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fc3e80d8190 con 0x5610c3a2b1e0 2024-04-25T11:35:35.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.983+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5610c3af51a0 0x5610c3af55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:35.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.983+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x5610c3afa3f0 con 0x5610c3af51a0 2024-04-25T11:35:35.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.983+0000 7fc3fe600640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3dc072e90 0x7fc3dc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.984+0000 7fc3ffa00640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5610c3af51a0 0x5610c3af55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:35.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.984+0000 7fc3ffa00640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5610c3af51a0 0x5610c3af55b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc3ffa00640 1 -- 172.21.15.19:0/4017021352 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fc3f80097a0 con 0x5610c3af51a0 2024-04-25T11:35:35.973 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/200.00000001 mtime 2024-04-25T11:35:27.000000+0000, size 18117 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5610c3af51a0 msgr2=0x5610c3af55b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5610c3af51a0 0x5610c3af55b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc3fe600640 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3dc072e90 0x7fc3dc0753a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0001640 tx=0x7fc3f0008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3dc072e90 msgr2=0x7fc3dc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3dc072e90 0x7fc3dc0753a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0001640 tx=0x7fc3f0008000 comp rx=0 tx=0).stop 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b1e0 msgr2=0x5610c3af3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 --2- 172.21.15.19:0/4017021352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610c3a2b1e0 0x5610c3af3e90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8025ca0 tx=0x7fc3e8002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:35.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc3ffa00640 1 -- 172.21.15.19:0/4017021352 reap_dead start 2024-04-25T11:35:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 shutdown_connections 2024-04-25T11:35:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 >> 172.21.15.19:0/4017021352 conn(0x5610c394dac0 msgr2=0x5610c3a2a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 shutdown_connections 2024-04-25T11:35:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:35.985+0000 7fc405dd6f40 1 -- 172.21.15.19:0/4017021352 wait complete. 2024-04-25T11:35:35.985 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2024-04-25T11:35:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.159+0000 7f8441839f40 1 Processor -- start 2024-04-25T11:35:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.159+0000 7f8441839f40 1 -- start start 2024-04-25T11:35:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f8441839f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689ea9a6c0 0x55689eb62e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f8441839f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689eac6180 0x55689eac2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f8441839f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 0x55689eac2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f8441839f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55689eadfce0 con 0x55689ea9a6c0 2024-04-25T11:35:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f8441839f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55689ea3dd20 con 0x55689eac7180 2024-04-25T11:35:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f8441839f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55689eae02a0 con 0x55689eac6180 2024-04-25T11:35:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689ea9a6c0 0x55689eb62e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689ea9a6c0 0x55689eb62e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40022/0 (socket says 172.21.15.19:40022) 2024-04-25T11:35:36.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 -- 172.21.15.19:0/2424266040 learned_addr learned my addr 172.21.15.19:0/2424266040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 -- 172.21.15.19:0/2424266040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689eac6180 msgr2=0x55689eac2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 --2- 172.21.15.19:0/2424266040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689eac6180 0x55689eac2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 -- 172.21.15.19:0/2424266040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 msgr2=0x55689eac2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 --2- 172.21.15.19:0/2424266040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 0x55689eac2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.161+0000 7f843aa00640 1 -- 172.21.15.19:0/2424266040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55689eb633c0 con 0x55689ea9a6c0 2024-04-25T11:35:36.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.162+0000 7f843aa00640 1 --2- 172.21.15.19:0/2424266040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689ea9a6c0 0x55689eb62e80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f842c054040 tx=0x7f842c050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d834d077860e3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.162+0000 7f8438c00640 1 -- 172.21.15.19:0/2424266040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c051ca0 con 0x55689ea9a6c0 2024-04-25T11:35:36.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.162+0000 7f8438c00640 1 -- 172.21.15.19:0/2424266040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842c051e40 con 0x55689ea9a6c0 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.162+0000 7f8438c00640 1 -- 172.21.15.19:0/2424266040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c065680 con 0x55689ea9a6c0 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 -- 172.21.15.19:0/2424266040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689ea9a6c0 msgr2=0x55689eb62e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 --2- 172.21.15.19:0/2424266040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689ea9a6c0 0x55689eb62e80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f842c054040 tx=0x7f842c050000 comp rx=0 tx=0).stop 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 -- 172.21.15.19:0/2424266040 shutdown_connections 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 --2- 172.21.15.19:0/2424266040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 0x55689eac2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 --2- 172.21.15.19:0/2424266040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689eac6180 0x55689eac2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 --2- 172.21.15.19:0/2424266040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689ea9a6c0 0x55689eb62e80 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 -- 172.21.15.19:0/2424266040 >> 172.21.15.19:0/2424266040 conn(0x55689e9b9ac0 msgr2=0x55689eadcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 -- 172.21.15.19:0/2424266040 shutdown_connections 2024-04-25T11:35:36.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.163+0000 7f8441839f40 1 -- 172.21.15.19:0/2424266040 wait complete. 2024-04-25T11:35:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.164+0000 7f8441839f40 1 Processor -- start 2024-04-25T11:35:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.164+0000 7f8441839f40 1 -- start start 2024-04-25T11:35:36.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f8441839f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689ea9a6c0 0x55689eb66120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f8441839f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689eac6180 0x55689eb68550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f8441839f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 0x55689eb706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f8441839f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55689eb69420 con 0x55689eac6180 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f8441839f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55689eb698b0 con 0x55689eac7180 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f8441839f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55689eac1550 con 0x55689ea9a6c0 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f843a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689eac6180 0x55689eb68550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f843a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689eac6180 0x55689eb68550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40028/0 (socket says 172.21.15.19:40028) 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f843a000640 1 -- 172.21.15.19:0/3058441923 learned_addr learned my addr 172.21.15.19:0/3058441923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.166+0000 7f843aa00640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689ea9a6c0 0x55689eb66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f843b400640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 0x55689eb706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f843a000640 1 -- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689ea9a6c0 msgr2=0x55689eb66120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f843a000640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55689ea9a6c0 0x55689eb66120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f843a000640 1 -- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 msgr2=0x55689eb706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f843a000640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55689eac7180 0x55689eb706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f843a000640 1 -- 172.21.15.19:0/3058441923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55689eb66660 con 0x55689eac6180 2024-04-25T11:35:36.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f843a000640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689eac6180 0x55689eb68550 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f842400ab60 tx=0x7f84240086d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f8433400640 1 -- 172.21.15.19:0/3058441923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842400ae60 con 0x55689eac6180 2024-04-25T11:35:36.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f8433400640 1 -- 172.21.15.19:0/3058441923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842400f070 con 0x55689eac6180 2024-04-25T11:35:36.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55689eb5f6a0 con 0x55689eac6180 2024-04-25T11:35:36.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.167+0000 7f8433400640 1 -- 172.21.15.19:0/3058441923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84240148b0 con 0x55689eac6180 2024-04-25T11:35:36.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.168+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55689eb6fcb0 con 0x55689eac6180 2024-04-25T11:35:36.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.170+0000 7f8433400640 1 -- 172.21.15.19:0/3058441923 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f842401d3f0 con 0x55689eac6180 2024-04-25T11:35:36.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.171+0000 7f8433400640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8418072ec0 0x7f84180753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.171+0000 7f8433400640 1 -- 172.21.15.19:0/3058441923 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f84240939f0 con 0x55689eac6180 2024-04-25T11:35:36.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.171+0000 7f8441839f40 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55689eb6bb60 0x55689eb6bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.171+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55689eb70c00 con 0x55689eb6bb60 2024-04-25T11:35:36.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.172+0000 7f843aa00640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8418072ec0 0x7f84180753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.172+0000 7f843b400640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55689eb6bb60 0x55689eb6bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.173+0000 7f843aa00640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8418072ec0 0x7f84180753d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f842c061000 tx=0x7f842c067040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.173+0000 7f843b400640 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55689eb6bb60 0x55689eb6bf70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.162 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2024-04-25T11:35:27.000000+0000, size 0 2024-04-25T11:35:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f843b400640 1 -- 172.21.15.19:0/3058441923 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv4 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f843400c570 con 0x55689eb6bb60 2024-04-25T11:35:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55689eb6bb60 msgr2=0x55689eb6bf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f8441839f40 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55689eb6bb60 0x55689eb6bf70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8418072ec0 msgr2=0x7f84180753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f8441839f40 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8418072ec0 0x7f84180753d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f842c061000 tx=0x7f842c067040 comp rx=0 tx=0).stop 2024-04-25T11:35:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689eac6180 msgr2=0x55689eb68550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f8441839f40 1 --2- 172.21.15.19:0/3058441923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55689eac6180 0x55689eb68550 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f842400ab60 tx=0x7f84240086d0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.174+0000 7f843b400640 1 -- 172.21.15.19:0/3058441923 reap_dead start 2024-04-25T11:35:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.175+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 shutdown_connections 2024-04-25T11:35:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.175+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 >> 172.21.15.19:0/3058441923 conn(0x55689e9b9ac0 msgr2=0x55689eadd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.175+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 shutdown_connections 2024-04-25T11:35:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.175+0000 7f8441839f40 1 -- 172.21.15.19:0/3058441923 wait complete. 2024-04-25T11:35:36.174 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2024-04-25T11:35:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.351+0000 7f38f1863f40 1 Processor -- start 2024-04-25T11:35:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.351+0000 7f38f1863f40 1 -- start start 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.352+0000 7f38f1863f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31e8eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.352+0000 7f38f1863f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 0x55e31e9b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.352+0000 7f38f1863f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31e91d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.352+0000 7f38f1863f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e31e892d20 con 0x55e31e9b52c0 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.352+0000 7f38f1863f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e31e896d60 con 0x55e31e91e3d0 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.352+0000 7f38f1863f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e31e935110 con 0x7f38d0002970 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31e91d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31e91d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40040/0 (socket says 172.21.15.19:40040) 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 -- 172.21.15.19:0/2526591568 learned_addr learned my addr 172.21.15.19:0/2526591568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38e2000640 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 0x55e31e9b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eaa00640 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31e8eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 -- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 msgr2=0x55e31e8eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31e8eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 -- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 msgr2=0x55e31e9b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 0x55e31e9b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 -- 172.21.15.19:0/2526591568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31e9bb6e0 con 0x55e31e9b52c0 2024-04-25T11:35:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eaa00640 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31e8eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.353+0000 7f38eb400640 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31e91d110 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x55e31e934810 tx=0x7f38e400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2447e32e4608ddcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38e9600640 1 -- 172.21.15.19:0/2526591568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e400b030 con 0x55e31e9b52c0 2024-04-25T11:35:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38e9600640 1 -- 172.21.15.19:0/2526591568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e400b1d0 con 0x55e31e9b52c0 2024-04-25T11:35:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38e9600640 1 -- 172.21.15.19:0/2526591568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4009860 con 0x55e31e9b52c0 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 -- 172.21.15.19:0/2526591568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 msgr2=0x55e31e91d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31e91d110 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x55e31e934810 tx=0x7f38e400b430 comp rx=0 tx=0).stop 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 -- 172.21.15.19:0/2526591568 shutdown_connections 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 0x55e31e9b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31e91d110 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 --2- 172.21.15.19:0/2526591568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31e8eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 -- 172.21.15.19:0/2526591568 >> 172.21.15.19:0/2526591568 conn(0x55e31e80eac0 msgr2=0x55e31e937730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.354+0000 7f38f1863f40 1 -- 172.21.15.19:0/2526591568 shutdown_connections 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.355+0000 7f38f1863f40 1 -- 172.21.15.19:0/2526591568 wait complete. 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.356+0000 7f38f1863f40 1 Processor -- start 2024-04-25T11:35:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.356+0000 7f38f1863f40 1 -- start start 2024-04-25T11:35:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.357+0000 7f38f1863f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 0x55e31e9b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.357+0000 7f38f1863f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31ea53260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.357+0000 7f38f1863f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31ea537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.357+0000 7f38f1863f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e31e91ba60 con 0x55e31e9b52c0 2024-04-25T11:35:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.357+0000 7f38f1863f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e31e9c2280 con 0x55e31e91e3d0 2024-04-25T11:35:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.357+0000 7f38f1863f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e31e9c1a60 con 0x7f38d0002970 2024-04-25T11:35:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.357+0000 7f38e2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31ea53260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31ea53260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40056/0 (socket says 172.21.15.19:40056) 2024-04-25T11:35:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 -- 172.21.15.19:0/1131891895 learned_addr learned my addr 172.21.15.19:0/1131891895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38eaa00640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 0x55e31e9b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38eb400640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31ea537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 -- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 msgr2=0x55e31ea537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38d0002970 0x55e31ea537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 -- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 msgr2=0x55e31e9b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e31e91e3d0 0x55e31e9b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 -- 172.21.15.19:0/1131891895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31e9b41b0 con 0x55e31e9b52c0 2024-04-25T11:35:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.358+0000 7f38e2000640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31ea53260 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f38d4046a60 tx=0x7f38d4001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.359+0000 7f38e3e00640 1 -- 172.21.15.19:0/1131891895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4065020 con 0x55e31e9b52c0 2024-04-25T11:35:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.359+0000 7f38e3e00640 1 -- 172.21.15.19:0/1131891895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d40015b0 con 0x55e31e9b52c0 2024-04-25T11:35:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.359+0000 7f38e3e00640 1 -- 172.21.15.19:0/1131891895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4061650 con 0x55e31e9b52c0 2024-04-25T11:35:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.359+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e31e9b4450 con 0x55e31e9b52c0 2024-04-25T11:35:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.359+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31e9b3030 con 0x55e31e9b52c0 2024-04-25T11:35:36.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.361+0000 7f38e3e00640 1 -- 172.21.15.19:0/1131891895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f38d4068020 con 0x55e31e9b52c0 2024-04-25T11:35:36.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.362+0000 7f38e3e00640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38c8072e90 0x7f38c80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.362+0000 7f38e3e00640 1 -- 172.21.15.19:0/1131891895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f38d40d8700 con 0x55e31e9b52c0 2024-04-25T11:35:36.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.362+0000 7f38f1863f40 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e31e91d080 0x55e31e91d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.363+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55e31e9bcd50 con 0x55e31e91d080 2024-04-25T11:35:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.363+0000 7f38eaa00640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38c8072e90 0x7f38c80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.363+0000 7f38eb400640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e31e91d080 0x55e31e91d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.364+0000 7f38eb400640 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e31e91d080 0x55e31e91d490 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.364+0000 7f38eb400640 1 -- 172.21.15.19:0/1131891895 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f38e4009800 con 0x55e31e91d080 2024-04-25T11:35:36.352 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/500.00000000 mtime 2024-04-25T11:35:14.000000+0000, size 90 2024-04-25T11:35:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.364+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e31e91d080 msgr2=0x55e31e91d490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.364+0000 7f38f1863f40 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e31e91d080 0x55e31e91d490 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38c8072e90 msgr2=0x7f38c80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38c8072e90 0x7f38c80753a0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f38dc006a40 tx=0x7f38dc008040 comp rx=0 tx=0).stop 2024-04-25T11:35:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 msgr2=0x55e31ea53260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 --2- 172.21.15.19:0/1131891895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e31e9b52c0 0x55e31ea53260 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f38d4046a60 tx=0x7f38d4001970 comp rx=0 tx=0).stop 2024-04-25T11:35:36.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38eb400640 1 -- 172.21.15.19:0/1131891895 reap_dead start 2024-04-25T11:35:36.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 shutdown_connections 2024-04-25T11:35:36.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 >> 172.21.15.19:0/1131891895 conn(0x55e31e80eac0 msgr2=0x55e31e9b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 shutdown_connections 2024-04-25T11:35:36.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.365+0000 7f38f1863f40 1 -- 172.21.15.19:0/1131891895 wait complete. 2024-04-25T11:35:36.365 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2024-04-25T11:35:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.539+0000 7f78653edf40 1 Processor -- start 2024-04-25T11:35:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.539+0000 7f78653edf40 1 -- start start 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.540+0000 7f78653edf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e500920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.540+0000 7f78653edf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 0x561c6e634420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.540+0000 7f78653edf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 0x561c6e51bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.540+0000 7f78653edf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c6e475d20 con 0x7f78440028c0 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.540+0000 7f78653edf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561c6e479d60 con 0x561c6e59e230 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.540+0000 7f78653edf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561c6e518110 con 0x561c6e59de20 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e500920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e500920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40078/0 (socket says 172.21.15.19:40078) 2024-04-25T11:35:36.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 -- 172.21.15.19:0/146383997 learned_addr learned my addr 172.21.15.19:0/146383997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 -- 172.21.15.19:0/146383997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 msgr2=0x561c6e634420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 --2- 172.21.15.19:0/146383997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 0x561c6e634420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 -- 172.21.15.19:0/146383997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 msgr2=0x561c6e51bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 --2- 172.21.15.19:0/146383997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 0x561c6e51bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 -- 172.21.15.19:0/146383997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6e4fdb30 con 0x7f78440028c0 2024-04-25T11:35:36.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.541+0000 7f785e600640 1 --2- 172.21.15.19:0/146383997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e500920 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f785000c6d0 tx=0x7f785000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72189a0791f019f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f7857e00640 1 -- 172.21.15.19:0/146383997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850012070 con 0x7f78440028c0 2024-04-25T11:35:36.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f7857e00640 1 -- 172.21.15.19:0/146383997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785000a840 con 0x7f78440028c0 2024-04-25T11:35:36.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f7857e00640 1 -- 172.21.15.19:0/146383997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000abf0 con 0x7f78440028c0 2024-04-25T11:35:36.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f78653edf40 1 -- 172.21.15.19:0/146383997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 msgr2=0x561c6e500920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f78653edf40 1 --2- 172.21.15.19:0/146383997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e500920 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f785000c6d0 tx=0x7f785000cb90 comp rx=0 tx=0).stop 2024-04-25T11:35:36.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f78653edf40 1 -- 172.21.15.19:0/146383997 shutdown_connections 2024-04-25T11:35:36.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f78653edf40 1 --2- 172.21.15.19:0/146383997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 0x561c6e51bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f78653edf40 1 --2- 172.21.15.19:0/146383997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 0x561c6e634420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f78653edf40 1 --2- 172.21.15.19:0/146383997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e500920 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.542+0000 7f78653edf40 1 -- 172.21.15.19:0/146383997 >> 172.21.15.19:0/146383997 conn(0x561c6e3f1ac0 msgr2=0x561c6e5011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.543+0000 7f78653edf40 1 -- 172.21.15.19:0/146383997 shutdown_connections 2024-04-25T11:35:36.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.543+0000 7f78653edf40 1 -- 172.21.15.19:0/146383997 wait complete. 2024-04-25T11:35:36.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.544+0000 7f78653edf40 1 Processor -- start 2024-04-25T11:35:36.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.544+0000 7f78653edf40 1 -- start start 2024-04-25T11:35:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.545+0000 7f78653edf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 0x561c6e519360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.545+0000 7f78653edf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 0x561c6e59ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.545+0000 7f78653edf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e51bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.545+0000 7f78653edf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561c6e502590 con 0x7f78440028c0 2024-04-25T11:35:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.545+0000 7f78653edf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561c6e500490 con 0x561c6e59e230 2024-04-25T11:35:36.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.545+0000 7f78653edf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561c6e6325b0 con 0x561c6e59de20 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e51bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e51bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40086/0 (socket says 172.21.15.19:40086) 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 -- 172.21.15.19:0/745665343 learned_addr learned my addr 172.21.15.19:0/745665343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785dc00640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 0x561c6e59ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785e600640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 0x561c6e519360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 -- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 msgr2=0x561c6e519360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561c6e59de20 0x561c6e519360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 -- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 msgr2=0x561c6e59ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561c6e59e230 0x561c6e59ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.546+0000 7f785f000640 1 -- 172.21.15.19:0/745665343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6e633e50 con 0x7f78440028c0 2024-04-25T11:35:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.547+0000 7f785f000640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e51bdc0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x561c6e518fb0 tx=0x7f785800f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.547+0000 7f7856a00640 1 -- 172.21.15.19:0/745665343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858017070 con 0x7f78440028c0 2024-04-25T11:35:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.547+0000 7f7856a00640 1 -- 172.21.15.19:0/745665343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7858013700 con 0x7f78440028c0 2024-04-25T11:35:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.547+0000 7f7856a00640 1 -- 172.21.15.19:0/745665343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858013ad0 con 0x7f78440028c0 2024-04-25T11:35:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.547+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561c6e634130 con 0x7f78440028c0 2024-04-25T11:35:36.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.547+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6e515620 con 0x7f78440028c0 2024-04-25T11:35:36.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.549+0000 7f7856a00640 1 -- 172.21.15.19:0/745665343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f785801a070 con 0x7f78440028c0 2024-04-25T11:35:36.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.550+0000 7f7856a00640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.551+0000 7f7856a00640 1 -- 172.21.15.19:0/745665343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f7858094240 con 0x7f78440028c0 2024-04-25T11:35:36.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.551+0000 7f78653edf40 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561c6e59ce70 0x561c6e59d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.551+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x561c6e6302c0 con 0x561c6e59ce70 2024-04-25T11:35:36.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.551+0000 7f785dc00640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561c6e59ce70 0x561c6e59d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.551+0000 7f785e600640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.552+0000 7f785dc00640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561c6e59ce70 0x561c6e59d280 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.552+0000 7f785dc00640 1 -- 172.21.15.19:0/745665343 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f784c0547c0 con 0x561c6e59ce70 2024-04-25T11:35:36.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.552+0000 7f785e600640 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f785000c6d0 tx=0x7f785001a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.541 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/mds_snaptable mtime 2024-04-25T11:35:14.000000+0000, size 70 2024-04-25T11:35:36.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561c6e59ce70 msgr2=0x561c6e59d280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561c6e59ce70 0x561c6e59d280 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 msgr2=0x7f78400753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7840072e90 0x7f78400753a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f785000c6d0 tx=0x7f785001a040 comp rx=0 tx=0).stop 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 msgr2=0x561c6e51bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 --2- 172.21.15.19:0/745665343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78440028c0 0x561c6e51bdc0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x561c6e518fb0 tx=0x7f785800f910 comp rx=0 tx=0).stop 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f785f000640 1 -- 172.21.15.19:0/745665343 reap_dead start 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 shutdown_connections 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 >> 172.21.15.19:0/745665343 conn(0x561c6e3f1ac0 msgr2=0x561c6e5a7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 shutdown_connections 2024-04-25T11:35:36.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.553+0000 7f78653edf40 1 -- 172.21.15.19:0/745665343 wait complete. 2024-04-25T11:35:36.551 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2024-04-25T11:35:36.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.723+0000 7ff9b1261f40 1 Processor -- start 2024-04-25T11:35:36.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.723+0000 7ff9b1261f40 1 -- start start 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9b1261f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce9179ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9b1261f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce916f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9b1261f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce916f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9b1261f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce9166ed20 con 0x55ce9179c6d0 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9b1261f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ce916c9af0 con 0x55ce9179d960 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9b1261f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ce916ea340 con 0x55ce916c96e0 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9a9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce916f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9a9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce916f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40102/0 (socket says 172.21.15.19:40102) 2024-04-25T11:35:36.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.724+0000 7ff9a9a00640 1 -- 172.21.15.19:0/2985531965 learned_addr learned my addr 172.21.15.19:0/2985531965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9aa400640 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce9179ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9aae00640 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce916f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9a9a00640 1 -- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 msgr2=0x55ce9179ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9a9a00640 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce9179ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9a9a00640 1 -- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 msgr2=0x55ce916f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9a9a00640 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce916f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9a9a00640 1 -- 172.21.15.19:0/2985531965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce916c92d0 con 0x55ce9179c6d0 2024-04-25T11:35:36.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9a9a00640 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce916f5870 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff99400a770 tx=0x7ff99400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15652932f387eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.725+0000 7ff9a3e00640 1 -- 172.21.15.19:0/2985531965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994019070 con 0x55ce9179c6d0 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9a3e00640 1 -- 172.21.15.19:0/2985531965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff994007600 con 0x55ce9179c6d0 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9a3e00640 1 -- 172.21.15.19:0/2985531965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940079d0 con 0x55ce9179c6d0 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 -- 172.21.15.19:0/2985531965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 msgr2=0x55ce916f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce916f5870 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff99400a770 tx=0x7ff99400a860 comp rx=0 tx=0).stop 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 -- 172.21.15.19:0/2985531965 shutdown_connections 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce916f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce916f5870 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/2985531965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce9179ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 -- 172.21.15.19:0/2985531965 >> 172.21.15.19:0/2985531965 conn(0x55ce915eaac0 msgr2=0x55ce9179d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 -- 172.21.15.19:0/2985531965 shutdown_connections 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.726+0000 7ff9b1261f40 1 -- 172.21.15.19:0/2985531965 wait complete. 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.727+0000 7ff9b1261f40 1 Processor -- start 2024-04-25T11:35:36.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.728+0000 7ff9b1261f40 1 -- start start 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9b1261f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce916e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9b1261f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce917888e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9b1261f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce9178f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9b1261f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ce916c8ff0 con 0x55ce9179c6d0 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9b1261f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ce91710870 con 0x55ce9179d960 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9b1261f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ce91836830 con 0x55ce916c96e0 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9a9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce917888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9a9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce917888e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40108/0 (socket says 172.21.15.19:40108) 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.729+0000 7ff9a9a00640 1 -- 172.21.15.19:0/103419404 learned_addr learned my addr 172.21.15.19:0/103419404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9aa400640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce916e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9aae00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce9178f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9a9a00640 1 -- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 msgr2=0x55ce916e93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9a9a00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce916e93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9a9a00640 1 -- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 msgr2=0x55ce9178f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9a9a00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce9178f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9a9a00640 1 -- 172.21.15.19:0/103419404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce91799420 con 0x55ce9179c6d0 2024-04-25T11:35:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9aa400640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ce916c96e0 0x55ce916e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:36.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9aae00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ce9179d960 0x55ce9178f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.730+0000 7ff9a9a00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce917888e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff994013000 tx=0x7ff994003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.731+0000 7ff9a2a00640 1 -- 172.21.15.19:0/103419404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994019070 con 0x55ce9179c6d0 2024-04-25T11:35:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.731+0000 7ff9a2a00640 1 -- 172.21.15.19:0/103419404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99401a040 con 0x55ce9179c6d0 2024-04-25T11:35:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.731+0000 7ff9a2a00640 1 -- 172.21.15.19:0/103419404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940146b0 con 0x55ce9179c6d0 2024-04-25T11:35:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.731+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce91799700 con 0x55ce9179c6d0 2024-04-25T11:35:36.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.731+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce91799cc0 con 0x55ce9179c6d0 2024-04-25T11:35:36.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.733+0000 7ff9a2a00640 1 -- 172.21.15.19:0/103419404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff994014850 con 0x55ce9179c6d0 2024-04-25T11:35:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.734+0000 7ff9a2a00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff988072e90 0x7ff9880753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.734+0000 7ff9a2a00640 1 -- 172.21.15.19:0/103419404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7ff994094f40 con 0x55ce9179c6d0 2024-04-25T11:35:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.734+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ce9178fd10 0x55ce918359e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.735+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55ce91835f20 con 0x55ce9178fd10 2024-04-25T11:35:36.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.735+0000 7ff9aa400640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff988072e90 0x7ff9880753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.735+0000 7ff9aae00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ce9178fd10 0x55ce918359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.736+0000 7ff9aa400640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff988072e90 0x7ff9880753a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55ce916f1c90 tx=0x7ff99c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.736+0000 7ff9aae00640 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ce9178fd10 0x55ce918359e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.725 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/mds0_inotable mtime 2024-04-25T11:35:27.000000+0000, size 34 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.737+0000 7ff9aae00640 1 -- 172.21.15.19:0/103419404 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv3 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7ff9a4054970 con 0x55ce9178fd10 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.737+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ce9178fd10 msgr2=0x55ce918359e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ce9178fd10 0x55ce918359e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff988072e90 msgr2=0x7ff9880753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff988072e90 0x7ff9880753a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55ce916f1c90 tx=0x7ff99c008040 comp rx=0 tx=0).stop 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 msgr2=0x55ce917888e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 --2- 172.21.15.19:0/103419404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ce9179c6d0 0x55ce917888e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff994013000 tx=0x7ff994003020 comp rx=0 tx=0).stop 2024-04-25T11:35:36.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9aae00640 1 -- 172.21.15.19:0/103419404 reap_dead start 2024-04-25T11:35:36.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 shutdown_connections 2024-04-25T11:35:36.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 >> 172.21.15.19:0/103419404 conn(0x55ce915eaac0 msgr2=0x55ce9170bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 shutdown_connections 2024-04-25T11:35:36.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.738+0000 7ff9b1261f40 1 -- 172.21.15.19:0/103419404 wait complete. 2024-04-25T11:35:36.738 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2024-04-25T11:35:36.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.920+0000 7f971831ff40 1 Processor -- start 2024-04-25T11:35:36.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.920+0000 7f971831ff40 1 -- start start 2024-04-25T11:35:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f971831ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac317a50 0x55cbac3e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f971831ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac35cac0 0x55cbac343b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f971831ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35fa50 0x55cbac3440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f971831ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbac35f5c0 con 0x55cbac35fa50 2024-04-25T11:35:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f971831ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cbac2bcd20 con 0x55cbac35cac0 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f971831ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55cbac35ef70 con 0x55cbac317a50 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35fa50 0x55cbac3440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35fa50 0x55cbac3440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40126/0 (socket says 172.21.15.19:40126) 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 -- 172.21.15.19:0/3290361965 learned_addr learned my addr 172.21.15.19:0/3290361965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9710a00640 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac317a50 0x55cbac3e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9710000640 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac35cac0 0x55cbac343b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 -- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac317a50 msgr2=0x55cbac3e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac317a50 0x55cbac3e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 -- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac35cac0 msgr2=0x55cbac343b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac35cac0 0x55cbac343b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.922+0000 7f9711400640 1 -- 172.21.15.19:0/3290361965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbac3ec810 con 0x55cbac35fa50 2024-04-25T11:35:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.923+0000 7f9711400640 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35fa50 0x55cbac3440c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9708001e80 tx=0x7f970800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4248fcad0266db99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.923+0000 7f970ec00640 1 -- 172.21.15.19:0/3290361965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708008070 con 0x55cbac35fa50 2024-04-25T11:35:36.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.923+0000 7f970ec00640 1 -- 172.21.15.19:0/3290361965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970800d500 con 0x55cbac35fa50 2024-04-25T11:35:36.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.923+0000 7f970ec00640 1 -- 172.21.15.19:0/3290361965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800d8d0 con 0x55cbac35fa50 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 -- 172.21.15.19:0/3290361965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35fa50 msgr2=0x55cbac3440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35fa50 0x55cbac3440c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9708001e80 tx=0x7f970800b6a0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 -- 172.21.15.19:0/3290361965 shutdown_connections 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac35cac0 0x55cbac343b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35fa50 0x55cbac3440c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 --2- 172.21.15.19:0/3290361965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac317a50 0x55cbac3e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 -- 172.21.15.19:0/3290361965 >> 172.21.15.19:0/3290361965 conn(0x55cbac238ac0 msgr2=0x55cbac3414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 -- 172.21.15.19:0/3290361965 shutdown_connections 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.924+0000 7f971831ff40 1 -- 172.21.15.19:0/3290361965 wait complete. 2024-04-25T11:35:36.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.925+0000 7f971831ff40 1 Processor -- start 2024-04-25T11:35:36.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.925+0000 7f971831ff40 1 -- start start 2024-04-25T11:35:36.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f971831ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac317a50 0x55cbac35b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f971831ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35cac0 0x55cbac33e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f971831ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac35fa50 0x55cbac486b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f971831ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cbac35ddc0 con 0x55cbac35cac0 2024-04-25T11:35:36.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f971831ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cbac3146b0 con 0x55cbac317a50 2024-04-25T11:35:36.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f971831ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55cbac3ee330 con 0x55cbac35fa50 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35cac0 0x55cbac33e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35cac0 0x55cbac33e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40138/0 (socket says 172.21.15.19:40138) 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710000640 1 -- 172.21.15.19:0/4257436171 learned_addr learned my addr 172.21.15.19:0/4257436171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9711400640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac35fa50 0x55cbac486b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710a00640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac317a50 0x55cbac35b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710000640 1 -- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac35fa50 msgr2=0x55cbac486b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710000640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cbac35fa50 0x55cbac486b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710000640 1 -- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac317a50 msgr2=0x55cbac35b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.927+0000 7f9710000640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cbac317a50 0x55cbac35b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.928+0000 7f9710000640 1 -- 172.21.15.19:0/4257436171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbac3e60d0 con 0x55cbac35cac0 2024-04-25T11:35:36.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.928+0000 7f9710000640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35cac0 0x55cbac33e720 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f97000081a0 tx=0x7f9700008660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.928+0000 7f970d800640 1 -- 172.21.15.19:0/4257436171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700016020 con 0x55cbac35cac0 2024-04-25T11:35:36.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.928+0000 7f970d800640 1 -- 172.21.15.19:0/4257436171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9700012610 con 0x55cbac35cac0 2024-04-25T11:35:36.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.928+0000 7f970d800640 1 -- 172.21.15.19:0/4257436171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970000e040 con 0x55cbac35cac0 2024-04-25T11:35:36.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.928+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cbac3ef610 con 0x55cbac35cac0 2024-04-25T11:35:36.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.928+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbac3595e0 con 0x55cbac35cac0 2024-04-25T11:35:36.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.931+0000 7f970d800640 1 -- 172.21.15.19:0/4257436171 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9700019020 con 0x55cbac35cac0 2024-04-25T11:35:36.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.932+0000 7f970d800640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96ec072e90 0x7f96ec0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.932+0000 7f970d800640 1 -- 172.21.15.19:0/4257436171 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f9700093ba0 con 0x55cbac35cac0 2024-04-25T11:35:36.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.932+0000 7f971831ff40 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cbac3e48a0 0x55cbac3ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:36.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.932+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55cbac3e4df0 con 0x55cbac3e48a0 2024-04-25T11:35:36.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.932+0000 7f9710a00640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96ec072e90 0x7f96ec0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.933+0000 7f9711400640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cbac3e48a0 0x55cbac3ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:36.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.934+0000 7f9710a00640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96ec072e90 0x7f96ec0753a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f96f8057000 tx=0x7f96f8056040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.934+0000 7f9711400640 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cbac3e48a0 0x55cbac3ecbb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:36.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f9711400640 1 -- 172.21.15.19:0/4257436171 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f970800d730 con 0x55cbac3e48a0 2024-04-25T11:35:36.923 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/100.00000000 mtime 2024-04-25T11:35:14.000000+0000, size 0 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cbac3e48a0 msgr2=0x55cbac3ecbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cbac3e48a0 0x55cbac3ecbb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96ec072e90 msgr2=0x7f96ec0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96ec072e90 0x7f96ec0753a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f96f8057000 tx=0x7f96f8056040 comp rx=0 tx=0).stop 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35cac0 msgr2=0x55cbac33e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 --2- 172.21.15.19:0/4257436171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cbac35cac0 0x55cbac33e720 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f97000081a0 tx=0x7f9700008660 comp rx=0 tx=0).stop 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f9711400640 1 -- 172.21.15.19:0/4257436171 reap_dead start 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 shutdown_connections 2024-04-25T11:35:36.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.935+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 >> 172.21.15.19:0/4257436171 conn(0x55cbac238ac0 msgr2=0x55cbac3ebb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.936+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 shutdown_connections 2024-04-25T11:35:36.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:36.936+0000 7f971831ff40 1 -- 172.21.15.19:0/4257436171 wait complete. 2024-04-25T11:35:36.935 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2024-04-25T11:35:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.135+0000 7f44f361bf40 1 Processor -- start 2024-04-25T11:35:37.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.135+0000 7f44f361bf40 1 -- start start 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.136+0000 7f44f361bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee287f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.136+0000 7f44f361bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee286945f0 0x55ee2876a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.136+0000 7f44f361bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee2876ddd0 0x55ee286c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.136+0000 7f44f361bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee286dce70 con 0x55ee286941e0 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.136+0000 7f44f361bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ee2863ad20 con 0x55ee286945f0 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.136+0000 7f44f361bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ee286dd3f0 con 0x55ee2876ddd0 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.136+0000 7f44ec800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee287f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee287f94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40152/0 (socket says 172.21.15.19:40152) 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 -- 172.21.15.19:0/1677890984 learned_addr learned my addr 172.21.15.19:0/1677890984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 -- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee2876ddd0 msgr2=0x55ee286c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ebe00640 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee286945f0 0x55ee2876a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ed200640 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee2876ddd0 0x55ee286c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee2876ddd0 0x55ee286c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 -- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee286945f0 msgr2=0x55ee2876a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee286945f0 0x55ee2876a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 -- 172.21.15.19:0/1677890984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee286d8780 con 0x55ee286941e0 2024-04-25T11:35:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ebe00640 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee286945f0 0x55ee2876a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.137+0000 7f44ec800640 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee287f94f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00a7a0 tx=0x7f44dc00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd27cc414b0a02c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.138+0000 7f44eaa00640 1 -- 172.21.15.19:0/1677890984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc019070 con 0x55ee286941e0 2024-04-25T11:35:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.138+0000 7f44eaa00640 1 -- 172.21.15.19:0/1677890984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc007600 con 0x55ee286941e0 2024-04-25T11:35:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.138+0000 7f44eaa00640 1 -- 172.21.15.19:0/1677890984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc0079d0 con 0x55ee286941e0 2024-04-25T11:35:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.138+0000 7f44f361bf40 1 -- 172.21.15.19:0/1677890984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 msgr2=0x55ee287f94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.138+0000 7f44f361bf40 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee287f94f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00a7a0 tx=0x7f44dc00a890 comp rx=0 tx=0).stop 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.139+0000 7f44f361bf40 1 -- 172.21.15.19:0/1677890984 shutdown_connections 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.139+0000 7f44f361bf40 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee286945f0 0x55ee2876a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.139+0000 7f44f361bf40 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee2876ddd0 0x55ee286c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.139+0000 7f44f361bf40 1 --2- 172.21.15.19:0/1677890984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee287f94f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.139+0000 7f44f361bf40 1 -- 172.21.15.19:0/1677890984 >> 172.21.15.19:0/1677890984 conn(0x55ee285b6ac0 msgr2=0x55ee28693540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.139+0000 7f44f361bf40 1 -- 172.21.15.19:0/1677890984 shutdown_connections 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.139+0000 7f44f361bf40 1 -- 172.21.15.19:0/1677890984 wait complete. 2024-04-25T11:35:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.140+0000 7f44f361bf40 1 Processor -- start 2024-04-25T11:35:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.140+0000 7f44f361bf40 1 -- start start 2024-04-25T11:35:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.141+0000 7f44f361bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee28766ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.141+0000 7f44f361bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee286945f0 0x55ee2876c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.141+0000 7f44f361bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee2876ddd0 0x55ee286c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.141+0000 7f44f361bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee286d8a60 con 0x55ee286941e0 2024-04-25T11:35:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.141+0000 7f44f361bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ee286c5160 con 0x55ee2876ddd0 2024-04-25T11:35:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.141+0000 7f44f361bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ee287fbf30 con 0x55ee286945f0 2024-04-25T11:35:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee28766ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee28766ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40156/0 (socket says 172.21.15.19:40156) 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 -- 172.21.15.19:0/37383666 learned_addr learned my addr 172.21.15.19:0/37383666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ebe00640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee286945f0 0x55ee2876c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ed200640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee2876ddd0 0x55ee286c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 -- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee286945f0 msgr2=0x55ee2876c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee286945f0 0x55ee2876c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 -- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee2876ddd0 msgr2=0x55ee286c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee2876ddd0 0x55ee286c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ec800640 1 -- 172.21.15.19:0/37383666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee286c1500 con 0x55ee286941e0 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ebe00640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ee286945f0 0x55ee2876c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.142+0000 7f44ed200640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ee2876ddd0 0x55ee286c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.143+0000 7f44ec800640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee28766ef0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f44dc000c00 tx=0x7f44dc003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.143+0000 7f44e9600640 1 -- 172.21.15.19:0/37383666 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc019040 con 0x55ee286941e0 2024-04-25T11:35:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.143+0000 7f44e9600640 1 -- 172.21.15.19:0/37383666 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc00a030 con 0x55ee286941e0 2024-04-25T11:35:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.143+0000 7f44e9600640 1 -- 172.21.15.19:0/37383666 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc014470 con 0x55ee286941e0 2024-04-25T11:35:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.143+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ee286c17e0 con 0x55ee286941e0 2024-04-25T11:35:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.143+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee286c1da0 con 0x55ee286941e0 2024-04-25T11:35:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.145+0000 7f44e9600640 1 -- 172.21.15.19:0/37383666 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f44dc014610 con 0x55ee286941e0 2024-04-25T11:35:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.146+0000 7f44e9600640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44cc072e90 0x7f44cc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.146+0000 7f44e9600640 1 -- 172.21.15.19:0/37383666 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f44dc0947a0 con 0x55ee286941e0 2024-04-25T11:35:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.147+0000 7f44f361bf40 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ee28768e20 0x55ee28769230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.147+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55ee28769770 con 0x55ee28768e20 2024-04-25T11:35:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.147+0000 7f44ebe00640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44cc072e90 0x7f44cc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.147+0000 7f44ed200640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ee28768e20 0x55ee28769230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.149+0000 7f44ebe00640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44cc072e90 0x7f44cc0753a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f44d8056000 tx=0x7f44d8050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.149+0000 7f44ed200640 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ee28768e20 0x55ee28769230 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44ed200640 1 -- 172.21.15.19:0/37383666 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv3 ondisk = 0) ==== 159+0+16 (crc 0 0 0) 0x7f44e4006d70 con 0x55ee28768e20 2024-04-25T11:35:37.138 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/mds0_sessionmap mtime 2024-04-25T11:35:27.000000+0000, size 0 2024-04-25T11:35:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ee28768e20 msgr2=0x55ee28769230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44f361bf40 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ee28768e20 0x55ee28769230 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44cc072e90 msgr2=0x7f44cc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44f361bf40 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44cc072e90 0x7f44cc0753a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f44d8056000 tx=0x7f44d8050000 comp rx=0 tx=0).stop 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 msgr2=0x55ee28766ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44f361bf40 1 --2- 172.21.15.19:0/37383666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee286941e0 0x55ee28766ef0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f44dc000c00 tx=0x7f44dc003020 comp rx=0 tx=0).stop 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44ed200640 1 -- 172.21.15.19:0/37383666 reap_dead start 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.150+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 shutdown_connections 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.151+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 >> 172.21.15.19:0/37383666 conn(0x55ee285b6ac0 msgr2=0x55ee2876a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.151+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 shutdown_connections 2024-04-25T11:35:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.151+0000 7f44f361bf40 1 -- 172.21.15.19:0/37383666 wait complete. 2024-04-25T11:35:37.152 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2024-04-25T11:35:37.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.352+0000 7f5092ce4f40 1 Processor -- start 2024-04-25T11:35:37.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.353+0000 7f5092ce4f40 1 -- start start 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f5092ce4f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50700030f0 0x563eaec96050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f5092ce4f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f5092ce4f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eaebf1380 0x563eaebf1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f5092ce4f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563eaec08e70 con 0x563eaebf1380 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f5092ce4f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563eaeb66d20 con 0x563eaebf24c0 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f5092ce4f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563eaec09110 con 0x7f50700030f0 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f508c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eaebf1380 0x563eaebf1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f508c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eaebf1380 0x563eaebf1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40162/0 (socket says 172.21.15.19:40162) 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.354+0000 7f508c800640 1 -- 172.21.15.19:0/1221120196 learned_addr learned my addr 172.21.15.19:0/1221120196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508be00640 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50700030f0 0x563eaec96050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f5083400640 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508c800640 1 -- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50700030f0 msgr2=0x563eaec96050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508c800640 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50700030f0 0x563eaec96050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508c800640 1 -- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 msgr2=0x563eaec8cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508c800640 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508c800640 1 -- 172.21.15.19:0/1221120196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eaebf1cd0 con 0x563eaebf1380 2024-04-25T11:35:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508be00640 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50700030f0 0x563eaec96050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.355+0000 7f508c800640 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eaebf1380 0x563eaebf1790 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5070003070 tx=0x7f508400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7ec5c991df1cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f508aa00640 1 -- 172.21.15.19:0/1221120196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508400b280 con 0x563eaebf1380 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f508aa00640 1 -- 172.21.15.19:0/1221120196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508400bbb0 con 0x563eaebf1380 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f508aa00640 1 -- 172.21.15.19:0/1221120196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084015420 con 0x563eaebf1380 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 -- 172.21.15.19:0/1221120196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eaebf1380 msgr2=0x563eaebf1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eaebf1380 0x563eaebf1790 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5070003070 tx=0x7f508400b430 comp rx=0 tx=0).stop 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 -- 172.21.15.19:0/1221120196 shutdown_connections 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563eaebf1380 0x563eaebf1790 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/1221120196 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50700030f0 0x563eaec96050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 -- 172.21.15.19:0/1221120196 >> 172.21.15.19:0/1221120196 conn(0x563eaeae2ac0 msgr2=0x563eaec0a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.356+0000 7f5092ce4f40 1 -- 172.21.15.19:0/1221120196 shutdown_connections 2024-04-25T11:35:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.357+0000 7f5092ce4f40 1 -- 172.21.15.19:0/1221120196 wait complete. 2024-04-25T11:35:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.358+0000 7f5092ce4f40 1 Processor -- start 2024-04-25T11:35:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.358+0000 7f5092ce4f40 1 -- start start 2024-04-25T11:35:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f5092ce4f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563eaebf1380 0x563eaec8b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f5092ce4f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f5092ce4f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50700030f0 0x563eaec8dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f5092ce4f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563eaebf0e70 con 0x7f50700030f0 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f5092ce4f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563eaebf0b90 con 0x563eaebf24c0 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f5092ce4f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563eaec91650 con 0x563eaebf1380 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f508c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50700030f0 0x563eaec8dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f508c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50700030f0 0x563eaec8dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40174/0 (socket says 172.21.15.19:40174) 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.359+0000 7f508c800640 1 -- 172.21.15.19:0/723689597 learned_addr learned my addr 172.21.15.19:0/723689597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f508be00640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563eaebf1380 0x563eaec8b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f508c800640 1 -- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563eaebf1380 msgr2=0x563eaec8b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f5083400640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f508c800640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563eaebf1380 0x563eaec8b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f508c800640 1 -- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 msgr2=0x563eaec8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f508c800640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f508c800640 1 -- 172.21.15.19:0/723689597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eaec95900 con 0x7f50700030f0 2024-04-25T11:35:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f5083400640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563eaebf24c0 0x563eaec8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.360+0000 7f508c800640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50700030f0 0x563eaec8dc90 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f5084011000 tx=0x7f5084002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.361+0000 7f5089600640 1 -- 172.21.15.19:0/723689597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084009820 con 0x7f50700030f0 2024-04-25T11:35:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.361+0000 7f5089600640 1 -- 172.21.15.19:0/723689597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50840099e0 con 0x7f50700030f0 2024-04-25T11:35:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.361+0000 7f5089600640 1 -- 172.21.15.19:0/723689597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084009e10 con 0x7f50700030f0 2024-04-25T11:35:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.361+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563eaebf0640 con 0x7f50700030f0 2024-04-25T11:35:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.361+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eaec95e80 con 0x7f50700030f0 2024-04-25T11:35:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.363+0000 7f5089600640 1 -- 172.21.15.19:0/723689597 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f50840283f0 con 0x7f50700030f0 2024-04-25T11:35:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.364+0000 7f5089600640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5068072e90 0x7f50680753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.364+0000 7f5089600640 1 -- 172.21.15.19:0/723689597 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f5084094f90 con 0x7f50700030f0 2024-04-25T11:35:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.364+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563eaec946e0 0x563eaec94af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.364+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x563eaec95030 con 0x563eaec946e0 2024-04-25T11:35:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.365+0000 7f508be00640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5068072e90 0x7f50680753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.365+0000 7f5083400640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563eaec946e0 0x563eaec94af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5083400640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563eaec946e0 0x563eaec94af0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5083400640 1 -- 172.21.15.19:0/723689597 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f507404f860 con 0x563eaec946e0 2024-04-25T11:35:37.355 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/400.00000000 mtime 2024-04-25T11:35:14.000000+0000, size 22 2024-04-25T11:35:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f508be00640 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5068072e90 0x7f50680753a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f507c002250 tx=0x7f507c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563eaec946e0 msgr2=0x563eaec94af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563eaec946e0 0x563eaec94af0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5068072e90 msgr2=0x7f50680753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5068072e90 0x7f50680753a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f507c002250 tx=0x7f507c008040 comp rx=0 tx=0).stop 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50700030f0 msgr2=0x563eaec8dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 --2- 172.21.15.19:0/723689597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50700030f0 0x563eaec8dc90 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f5084011000 tx=0x7f5084002fb0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f508c800640 1 -- 172.21.15.19:0/723689597 reap_dead start 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 shutdown_connections 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.366+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 >> 172.21.15.19:0/723689597 conn(0x563eaeae2ac0 msgr2=0x563eaec0a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.367+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 shutdown_connections 2024-04-25T11:35:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.367+0000 7f5092ce4f40 1 -- 172.21.15.19:0/723689597 wait complete. 2024-04-25T11:35:37.369 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2024-04-25T11:35:37.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.552+0000 7f2301be2f40 1 Processor -- start 2024-04-25T11:35:37.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.552+0000 7f2301be2f40 1 -- start start 2024-04-25T11:35:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f2301be2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f2301be2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c4513b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f2301be2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c4513b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f2301be2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c451326e70 con 0x55c4513b7dd0 2024-04-25T11:35:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f2301be2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c451284d20 con 0x55c4512de1e0 2024-04-25T11:35:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f2301be2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c4513273f0 con 0x55c4512de5f0 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f22fb800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c4513b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f22fb800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c4513b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40176/0 (socket says 172.21.15.19:40176) 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f22fb800640 1 -- 172.21.15.19:0/245176165 learned_addr learned my addr 172.21.15.19:0/245176165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.554+0000 7f22fae00640 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fa400640 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c4513b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fb800640 1 -- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 msgr2=0x55c4513b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fb800640 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c4513b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fb800640 1 -- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 msgr2=0x55c4513aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fb800640 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fb800640 1 -- 172.21.15.19:0/245176165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4512dd340 con 0x55c4513b7dd0 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fa400640 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c4513b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22fb800640 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c4513b5510 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f22f4001580 tx=0x7f22f4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5fc824351e0e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22f9000640 1 -- 172.21.15.19:0/245176165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f400f710 con 0x55c4513b7dd0 2024-04-25T11:35:37.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22f9000640 1 -- 172.21.15.19:0/245176165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f400f8b0 con 0x55c4513b7dd0 2024-04-25T11:35:37.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.555+0000 7f22f9000640 1 -- 172.21.15.19:0/245176165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f400fce0 con 0x55c4513b7dd0 2024-04-25T11:35:37.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 -- 172.21.15.19:0/245176165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 msgr2=0x55c4513b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c4513b5510 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f22f4001580 tx=0x7f22f4001a40 comp rx=0 tx=0).stop 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 -- 172.21.15.19:0/245176165 shutdown_connections 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c4513b5510 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c4513b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 --2- 172.21.15.19:0/245176165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 -- 172.21.15.19:0/245176165 >> 172.21.15.19:0/245176165 conn(0x55c451200ac0 msgr2=0x55c4513103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 -- 172.21.15.19:0/245176165 shutdown_connections 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.556+0000 7f2301be2f40 1 -- 172.21.15.19:0/245176165 wait complete. 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.557+0000 7f2301be2f40 1 Processor -- start 2024-04-25T11:35:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.558+0000 7f2301be2f40 1 -- start start 2024-04-25T11:35:37.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f2301be2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f2301be2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c451323ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f2301be2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c451321280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f2301be2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c4513b4d30 con 0x55c4513b7dd0 2024-04-25T11:35:37.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f2301be2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c4513ab6a0 con 0x55c4512de1e0 2024-04-25T11:35:37.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f2301be2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c4513aa560 con 0x55c4512de5f0 2024-04-25T11:35:37.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f22fb800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c451321280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f22fb800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c451321280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40192/0 (socket says 172.21.15.19:40192) 2024-04-25T11:35:37.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.559+0000 7f22fb800640 1 -- 172.21.15.19:0/1370431695 learned_addr learned my addr 172.21.15.19:0/1370431695 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fa400640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c451323ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fae00640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fb800640 1 -- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 msgr2=0x55c451323ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fb800640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c451323ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fb800640 1 -- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 msgr2=0x55c4513aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fb800640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fb800640 1 -- 172.21.15.19:0/1370431695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c451322e20 con 0x55c4513b7dd0 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fae00640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c4512de1e0 0x55c4513aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fa400640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c4512de5f0 0x55c451323ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.560+0000 7f22fb800640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c451321280 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f22f4006620 tx=0x7f22f4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.561+0000 7f22f3400640 1 -- 172.21.15.19:0/1370431695 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4022020 con 0x55c4513b7dd0 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.561+0000 7f22f3400640 1 -- 172.21.15.19:0/1370431695 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f401e860 con 0x55c4513b7dd0 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.561+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c451323100 con 0x55c4513b7dd0 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.561+0000 7f22f3400640 1 -- 172.21.15.19:0/1370431695 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f401ec30 con 0x55c4513b7dd0 2024-04-25T11:35:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.561+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4513236c0 con 0x55c4513b7dd0 2024-04-25T11:35:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.563+0000 7f22f3400640 1 -- 172.21.15.19:0/1370431695 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f22f4025020 con 0x55c4513b7dd0 2024-04-25T11:35:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.564+0000 7f22f3400640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22d8072e90 0x7f22d80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.564+0000 7f22f3400640 1 -- 172.21.15.19:0/1370431695 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f22f4096130 con 0x55c4513b7dd0 2024-04-25T11:35:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.565+0000 7f2301be2f40 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c451321cd0 0x55c4514565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.565+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x55c4513b0d60 con 0x55c451321cd0 2024-04-25T11:35:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.565+0000 7f22fa400640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c451321cd0 0x55c4514565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.565+0000 7f22fae00640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22d8072e90 0x7f22d80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f22fa400640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c451321cd0 0x55c4514565a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f22fa400640 1 -- 172.21.15.19:0/1370431695 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x7f22e4009890 con 0x55c451321cd0 2024-04-25T11:35:37.554 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/100.00000000.inode mtime 2024-04-25T11:35:14.000000+0000, size 582 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c451321cd0 msgr2=0x55c4514565a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c451321cd0 0x55c4514565a0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f22fae00640 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22d8072e90 0x7f22d80753a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f22ec056000 tx=0x7f22ec050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22d8072e90 msgr2=0x7f22d80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22d8072e90 0x7f22d80753a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f22ec056000 tx=0x7f22ec050000 comp rx=0 tx=0).stop 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 msgr2=0x55c451321280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 --2- 172.21.15.19:0/1370431695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4513b7dd0 0x55c451321280 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f22f4006620 tx=0x7f22f4001250 comp rx=0 tx=0).stop 2024-04-25T11:35:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f22fb800640 1 -- 172.21.15.19:0/1370431695 reap_dead start 2024-04-25T11:35:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 shutdown_connections 2024-04-25T11:35:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 >> 172.21.15.19:0/1370431695 conn(0x55c451200ac0 msgr2=0x55c451310240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.566+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 shutdown_connections 2024-04-25T11:35:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.567+0000 7f2301be2f40 1 -- 172.21.15.19:0/1370431695 wait complete. 2024-04-25T11:35:37.566 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2024-04-25T11:35:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.755+0000 7f0f91275f40 1 Processor -- start 2024-04-25T11:35:37.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.755+0000 7f0f91275f40 1 -- start start 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f91275f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x5608576b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f91275f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608576b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f91275f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x560857742760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f91275f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560857583d20 con 0x5608575dd1e0 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f91275f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560857587d60 con 0x5608576b6fa0 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f91275f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560857626110 con 0x5608575dd5f0 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x5608576b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x5608576b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40210/0 (socket says 172.21.15.19:40210) 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 -- 172.21.15.19:0/1266618342 learned_addr learned my addr 172.21.15.19:0/1266618342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f89a00640 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608576b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8ae00640 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x560857742760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 -- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 msgr2=0x5608576b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608576b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 -- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 msgr2=0x560857742760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x560857742760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8a400640 1 -- 172.21.15.19:0/1266618342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560857623ea0 con 0x5608575dd1e0 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f8ae00640 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x560857742760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.757+0000 7f0f89a00640 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608576b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.758+0000 7f0f8a400640 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x5608576b3300 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0016a0 tx=0x7f0f7c001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8df0f9d9595d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.758+0000 7f0f83e00640 1 -- 172.21.15.19:0/1266618342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c003650 con 0x5608575dd1e0 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.758+0000 7f0f83e00640 1 -- 172.21.15.19:0/1266618342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c0037f0 con 0x5608575dd1e0 2024-04-25T11:35:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.758+0000 7f0f83e00640 1 -- 172.21.15.19:0/1266618342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c003c60 con 0x5608575dd1e0 2024-04-25T11:35:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 -- 172.21.15.19:0/1266618342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 msgr2=0x5608576b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x5608576b3300 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0016a0 tx=0x7f0f7c001b60 comp rx=0 tx=0).stop 2024-04-25T11:35:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 -- 172.21.15.19:0/1266618342 shutdown_connections 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x560857742760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608576b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1266618342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x5608576b3300 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 -- 172.21.15.19:0/1266618342 >> 172.21.15.19:0/1266618342 conn(0x5608574ffac0 msgr2=0x56085760f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 -- 172.21.15.19:0/1266618342 shutdown_connections 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.759+0000 7f0f91275f40 1 -- 172.21.15.19:0/1266618342 wait complete. 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.760+0000 7f0f91275f40 1 Processor -- start 2024-04-25T11:35:37.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.760+0000 7f0f91275f40 1 -- start start 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f91275f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x560857743ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f91275f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608575dbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f91275f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x5608576a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f91275f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608576b12d0 con 0x5608575dd1e0 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f91275f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5608576b15b0 con 0x5608576b6fa0 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f91275f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5608576b0590 con 0x5608575dd5f0 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x560857743ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x560857743ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40214/0 (socket says 172.21.15.19:40214) 2024-04-25T11:35:37.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 -- 172.21.15.19:0/1298370525 learned_addr learned my addr 172.21.15.19:0/1298370525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8ae00640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x5608576a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f89a00640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608575dbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 -- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 msgr2=0x5608575dbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608575dbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 -- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 msgr2=0x5608576a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608576b6fa0 0x5608576a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.762+0000 7f0f8a400640 1 -- 172.21.15.19:0/1298370525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560857623910 con 0x5608575dd1e0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.763+0000 7f0f89a00640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608575dd5f0 0x5608575dbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.763+0000 7f0f8a400640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x560857743ff0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c001330 tx=0x7f0f7c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.763+0000 7f0f82a00640 1 -- 172.21.15.19:0/1298370525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c025070 con 0x5608575dd1e0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.763+0000 7f0f82a00640 1 -- 172.21.15.19:0/1298370525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c019040 con 0x5608575dd1e0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.763+0000 7f0f82a00640 1 -- 172.21.15.19:0/1298370525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c013a90 con 0x5608575dd1e0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.763+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608576b3b50 con 0x5608575dd1e0 2024-04-25T11:35:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.763+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608577446d0 con 0x5608575dd1e0 2024-04-25T11:35:37.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.766+0000 7f0f82a00640 1 -- 172.21.15.19:0/1298370525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f7c010070 con 0x5608575dd1e0 2024-04-25T11:35:37.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.766+0000 7f0f82a00640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f68072ec0 0x7f0f680753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.767+0000 7f0f82a00640 1 -- 172.21.15.19:0/1298370525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f0f7c0952c0 con 0x5608575dd1e0 2024-04-25T11:35:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.767+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608576ad540 0x560857621da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.767+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x5608575dc2c0 con 0x5608576ad540 2024-04-25T11:35:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.767+0000 7f0f8ae00640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608576ad540 0x560857621da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.767+0000 7f0f89a00640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f68072ec0 0x7f0f680753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.768+0000 7f0f8ae00640 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608576ad540 0x560857621da0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:37.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.768+0000 7f0f8ae00640 1 -- 172.21.15.19:0/1298370525 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 154+0+16 (crc 0 0 0) 0x7f0f84054630 con 0x5608576ad540 2024-04-25T11:35:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:cephfs_metadata/1.00000000 mtime 2024-04-25T11:35:27.000000+0000, size 0 2024-04-25T11:35:37.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608576ad540 msgr2=0x560857621da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608576ad540 0x560857621da0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f68072ec0 msgr2=0x7f0f680753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f68072ec0 0x7f0f680753d0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x560857623240 tx=0x7f0f74001530 comp rx=0 tx=0).stop 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 msgr2=0x560857743ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 --2- 172.21.15.19:0/1298370525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608575dd1e0 0x560857743ff0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c001330 tx=0x7f0f7c001250 comp rx=0 tx=0).stop 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f8ae00640 1 -- 172.21.15.19:0/1298370525 reap_dead start 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 shutdown_connections 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.769+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 >> 172.21.15.19:0/1298370525 conn(0x5608574ffac0 msgr2=0x5608576af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.770+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 shutdown_connections 2024-04-25T11:35:37.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:37.770+0000 7f0f91275f40 1 -- 172.21.15.19:0/1298370525 wait complete. 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2024-04-25T11:35:37.770 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2024-04-25T11:35:37.771 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2024-04-25T11:35:37.771 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:37.771 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:37.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:35:37.816 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:35:37.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:37.817 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:35:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:35:37.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:35:37.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:35:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.214+0000 7f8066000640 1 Processor -- start 2024-04-25T11:35:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.215+0000 7f8066000640 1 -- start start 2024-04-25T11:35:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.217+0000 7f8066000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f80600d0ed0 0x7f80600d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:38.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.217+0000 7f8066000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8060160be0 0x7f8060160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.217+0000 7f8066000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8060161530 con 0x7f80601434d0 2024-04-25T11:35:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.217+0000 7f8066000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f80601438e0 con 0x7f80600d0ed0 2024-04-25T11:35:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.217+0000 7f8066000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f806006f3e0 con 0x7f8060160be0 2024-04-25T11:35:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.217+0000 7f8057e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f80600d0ed0 0x7f80600d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f8057e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f80600d0ed0 0x7f80600d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40216/0 (socket says 172.21.15.19:40216) 2024-04-25T11:35:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f8057e00640 1 -- 172.21.15.19:0/3420091074 learned_addr learned my addr 172.21.15.19:0/3420091074 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f8057e00640 1 -- 172.21.15.19:0/3420091074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8060160be0 msgr2=0x7f8060160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f805fe00640 1 --2- 172.21.15.19:0/3420091074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8060160be0 0x7f8060160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f8057e00640 1 --2- 172.21.15.19:0/3420091074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8060160be0 0x7f8060160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f8057e00640 1 -- 172.21.15.19:0/3420091074 >> v1:172.21.15.169:6789/0 conn(0x7f80601434d0 legacy=0x7f8060070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f8057e00640 1 -- 172.21.15.19:0/3420091074 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806006f6c0 con 0x7f80600d0ed0 2024-04-25T11:35:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.218+0000 7f805fe00640 1 --2- 172.21.15.19:0/3420091074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8060160be0 0x7f8060160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.219+0000 7f8057e00640 1 --2- 172.21.15.19:0/3420091074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80600d0ed0 0x7f80600d12e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8048009870 tx=0x7f804802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fef96c015a29a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.220+0000 7f805ea00640 1 -- 172.21.15.19:0/3420091074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048037070 con 0x7f80600d0ed0 2024-04-25T11:35:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.220+0000 7f805ea00640 1 -- 172.21.15.19:0/3420091074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8048032440 con 0x7f80600d0ed0 2024-04-25T11:35:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.220+0000 7f805ea00640 1 -- 172.21.15.19:0/3420091074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048032810 con 0x7f80600d0ed0 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 -- 172.21.15.19:0/3420091074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80600d0ed0 msgr2=0x7f80600d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 --2- 172.21.15.19:0/3420091074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80600d0ed0 0x7f80600d12e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8048009870 tx=0x7f804802a200 comp rx=0 tx=0).stop 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 -- 172.21.15.19:0/3420091074 shutdown_connections 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 --2- 172.21.15.19:0/3420091074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8060160be0 0x7f8060160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 --2- 172.21.15.19:0/3420091074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80600d0ed0 0x7f80600d12e0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 -- 172.21.15.19:0/3420091074 >> 172.21.15.19:0/3420091074 conn(0x7f80600d1ae0 msgr2=0x7f80600d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 -- 172.21.15.19:0/3420091074 shutdown_connections 2024-04-25T11:35:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.221+0000 7f8066000640 1 -- 172.21.15.19:0/3420091074 wait complete. 2024-04-25T11:35:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.225+0000 7f8066000640 1 Processor -- start 2024-04-25T11:35:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.225+0000 7f8066000640 1 -- start start 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f8066000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80600d0ed0 0x7f806013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f8066000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80601434d0 0x7f8060141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f8066000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8060160be0 0x7f80601f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f8066000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f806006f090 con 0x7f8060160be0 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f8066000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f806006fc80 con 0x7f80601434d0 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f8066000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f80600b51a0 con 0x7f80600d0ed0 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f805fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8060160be0 0x7f80601f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f805fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8060160be0 0x7f80601f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40226/0 (socket says 172.21.15.19:40226) 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.227+0000 7f805fe00640 1 -- 172.21.15.19:0/1897744066 learned_addr learned my addr 172.21.15.19:0/1897744066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f805f400640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80600d0ed0 0x7f806013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f8057e00640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80601434d0 0x7f8060141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:38.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f805fe00640 1 -- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80600d0ed0 msgr2=0x7f806013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f805fe00640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80600d0ed0 0x7f806013f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f805fe00640 1 -- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80601434d0 msgr2=0x7f8060141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f805fe00640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80601434d0 0x7f8060141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f805fe00640 1 -- 172.21.15.19:0/1897744066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8060140a20 con 0x7f8060160be0 2024-04-25T11:35:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.228+0000 7f805fe00640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8060160be0 0x7f80601f62c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f805800cdb0 tx=0x7f805800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.229+0000 7f8057400640 1 -- 172.21.15.19:0/1897744066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058003b90 con 0x7f8060160be0 2024-04-25T11:35:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.229+0000 7f8057400640 1 -- 172.21.15.19:0/1897744066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8058003d50 con 0x7f8060160be0 2024-04-25T11:35:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.229+0000 7f8057400640 1 -- 172.21.15.19:0/1897744066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058004260 con 0x7f8060160be0 2024-04-25T11:35:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.229+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8060140d00 con 0x7f8060160be0 2024-04-25T11:35:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.229+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80601412c0 con 0x7f8060160be0 2024-04-25T11:35:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.230+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8060050740 con 0x7f8060160be0 2024-04-25T11:35:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.232+0000 7f8057400640 1 -- 172.21.15.19:0/1897744066 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8058004400 con 0x7f8060160be0 2024-04-25T11:35:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.233+0000 7f8057400640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.233+0000 7f8057400640 1 -- 172.21.15.19:0/1897744066 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f8058095d00 con 0x7f8060160be0 2024-04-25T11:35:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.234+0000 7f805f400640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.236+0000 7f805f400640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744e0 0x7f803c0769f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8050006fd0 tx=0x7f8050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:38.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.240+0000 7f8057400640 1 -- 172.21.15.19:0/1897744066 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8058014030 con 0x7f8060160be0 2024-04-25T11:35:38.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.503+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f80600d0a10 con 0x7f8060160be0 2024-04-25T11:35:38.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.803+0000 7f8057400640 1 -- 172.21.15.19:0/1897744066 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f8058060420 con 0x7f8060160be0 2024-04-25T11:35:38.791 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:35:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744e0 msgr2=0x7f803c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744e0 0x7f803c0769f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8050006fd0 tx=0x7f8050008040 comp rx=0 tx=0).stop 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8060160be0 msgr2=0x7f80601f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8060160be0 0x7f80601f62c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f805800cdb0 tx=0x7f805800a650 comp rx=0 tx=0).stop 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 shutdown_connections 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80601434d0 0x7f8060141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8060160be0 0x7f80601f62c0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 --2- 172.21.15.19:0/1897744066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80600d0ed0 0x7f806013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:38.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 >> 172.21.15.19:0/1897744066 conn(0x7f80600d1ae0 msgr2=0x7f80600d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:38.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 shutdown_connections 2024-04-25T11:35:38.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:38.807+0000 7f8066000640 1 -- 172.21.15.19:0/1897744066 wait complete. 2024-04-25T11:35:38.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:35:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.116+0000 7f0eaf400640 1 Processor -- start 2024-04-25T11:35:39.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.116+0000 7f0eaf400640 1 -- start start 2024-04-25T11:35:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.119+0000 7f0eaf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 0x7f0ea80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.119+0000 7f0eaf400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80b6790 con 0x7f0ea806c5c0 2024-04-25T11:35:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.119+0000 7f0eaf400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80b4af0 con 0x7f0ea814ad60 2024-04-25T11:35:39.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.119+0000 7f0eaf400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea80b4dd0 con 0x7f0ea80b5e40 2024-04-25T11:35:39.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.119+0000 7f0ead400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0ea806c5c0 0x7f0ea806c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51516/0 (socket says 172.21.15.19:51516) 2024-04-25T11:35:39.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.119+0000 7f0ead400640 1 -- 172.21.15.19:0/3926034284 learned_addr learned my addr 172.21.15.19:0/3926034284 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:39.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.119+0000 7f0ea7e00640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 0x7f0ea80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548697997 0 0) 0x7f0ea0001490 con 0x7f0ea806c5c0 2024-04-25T11:35:39.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e90003520 con 0x7f0ea806c5c0 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2271365583 0 0) 0x7f0e90003520 con 0x7f0ea806c5c0 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 msgr2=0x7f0ea80b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 0x7f0ea80b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 >> v1:172.21.15.169:6789/0 conn(0x7f0ea814ad60 legacy=0x7f0ea8070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea80b50b0 con 0x7f0ea806c5c0 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f0ea0001bc0 con 0x7f0ea806c5c0 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.120+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 >> v1:172.21.15.19:6789/0 conn(0x7f0ea806c5c0 legacy=0x7f0ea806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ea7400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 0x7f0e900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ea7400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90003fb0 0x7f0e90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ea7400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e90004d40 0x7f0e900051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea80b4dd0 con 0x7f0e90004d40 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9000bc50 con 0x7f0e90003fb0 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9000be50 con 0x7f0ea80b5e40 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ead400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e90004d40 0x7f0e900051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0eaca00640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 0x7f0e900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ea7e00640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90003fb0 0x7f0e90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ead400640 1 -- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 msgr2=0x7f0e900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ead400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 0x7f0e900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ead400640 1 -- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90003fb0 msgr2=0x7f0e90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:39.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.121+0000 7f0ead400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90003fb0 0x7f0e90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:39.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.122+0000 7f0ead400640 1 -- 172.21.15.19:0/3926034284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e90003520 con 0x7f0e90004d40 2024-04-25T11:35:39.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.122+0000 7f0eaca00640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ea80b5e40 0x7f0e900039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:39.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.122+0000 7f0ea7e00640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e90003fb0 0x7f0e90004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:39.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.123+0000 7f0ead400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e90004d40 0x7f0e900051b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0ea806fa10 tx=0x7f0e90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c547e474d0b23d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:39.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.123+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000a570 con 0x7f0e90004d40 2024-04-25T11:35:39.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.123+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea002f070 con 0x7f0e90004d40 2024-04-25T11:35:39.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.123+0000 7f0ea7400640 1 -- 172.21.15.19:0/3926034284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea002bda0 con 0x7f0e90004d40 2024-04-25T11:35:39.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.124+0000 7f0eaf400640 1 -- 172.21.15.19:0/3926034284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e90004d40 msgr2=0x7f0e900051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:39.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.124+0000 7f0eaf400640 1 --2- 172.21.15.19:0/3926034284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e90004d40 0x7f0e900051b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0ea806fa10 tx=0x7f0e90005830 comp rx=0 tx=0).stop 2024-04-25T11:35:39.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.124+0000 7f0ead400640 1 -- 172.21.15.19:0/3926034284 reap_dead start 2024-04-25T11:35:39.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.124+0000 7f0eaf400640 1 -- 172.21.15.19:0/3926034284 shutdown_connections 2024-04-25T11:35:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.124+0000 7f0eaf400640 1 -- 172.21.15.19:0/3926034284 >> 172.21.15.19:0/3926034284 conn(0x7f0ea80d1ae0 msgr2=0x7f0ea80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.124+0000 7f0eaf400640 1 -- 172.21.15.19:0/3926034284 shutdown_connections 2024-04-25T11:35:39.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.124+0000 7f0eaf400640 1 -- 172.21.15.19:0/3926034284 wait complete. 2024-04-25T11:35:39.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.127+0000 7f0eaf400640 1 Processor -- start 2024-04-25T11:35:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.127+0000 7f0eaf400640 1 -- start start 2024-04-25T11:35:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.128+0000 7f0eaf400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90004d40 0x7f0ea8160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.128+0000 7f0eaf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8165c70 0x7f0ea8149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.128+0000 7f0eaf400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ea8149aa0 0x7f0ea81ed710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.128+0000 7f0eaf400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea806fb80 con 0x7f0ea8165c70 2024-04-25T11:35:39.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.128+0000 7f0eaf400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea80b5670 con 0x7f0ea8149aa0 2024-04-25T11:35:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.128+0000 7f0eaf400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea806fd00 con 0x7f0e90004d40 2024-04-25T11:35:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8165c70 0x7f0ea8149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8165c70 0x7f0ea8149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40256/0 (socket says 172.21.15.19:40256) 2024-04-25T11:35:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 -- 172.21.15.19:0/1715641959 learned_addr learned my addr 172.21.15.19:0/1715641959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0eaca00640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90004d40 0x7f0ea8160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 -- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90004d40 msgr2=0x7f0ea8160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ead400640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ea8149aa0 0x7f0ea81ed710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90004d40 0x7f0ea8160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 -- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ea8149aa0 msgr2=0x7f0ea81ed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ea8149aa0 0x7f0ea81ed710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.129+0000 7f0ea7e00640 1 -- 172.21.15.19:0/1715641959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea8149eb0 con 0x7f0ea8165c70 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.130+0000 7f0ea7e00640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8165c70 0x7f0ea8149560 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0e940108e0 tx=0x7f0e94010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.130+0000 7f0ea4c00640 1 -- 172.21.15.19:0/1715641959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9400f040 con 0x7f0ea8165c70 2024-04-25T11:35:39.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.130+0000 7f0ea4c00640 1 -- 172.21.15.19:0/1715641959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9400ebc0 con 0x7f0ea8165c70 2024-04-25T11:35:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.130+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea81fa410 con 0x7f0ea8165c70 2024-04-25T11:35:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.130+0000 7f0ea4c00640 1 -- 172.21.15.19:0/1715641959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9401c400 con 0x7f0ea8165c70 2024-04-25T11:35:39.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.130+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea8166980 con 0x7f0ea8165c70 2024-04-25T11:35:39.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.131+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea8160ea0 con 0x7f0ea8165c70 2024-04-25T11:35:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.133+0000 7f0ea4c00640 1 -- 172.21.15.19:0/1715641959 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e9401c5a0 con 0x7f0ea8165c70 2024-04-25T11:35:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.134+0000 7f0ea4c00640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e880744b0 0x7f0e880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.134+0000 7f0ea4c00640 1 -- 172.21.15.19:0/1715641959 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f0e94095a10 con 0x7f0ea8165c70 2024-04-25T11:35:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.134+0000 7f0eaca00640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e880744b0 0x7f0e880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:39.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.136+0000 7f0eaca00640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e880744b0 0x7f0e880769c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80b1590 tx=0x7f0e9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.139+0000 7f0ea4c00640 1 -- 172.21.15.19:0/1715641959 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ea8160ea0 con 0x7f0ea8165c70 2024-04-25T11:35:39.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:39.419+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f0ea8164480 con 0x7f0ea8165c70 2024-04-25T11:35:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.094+0000 7f0ea4c00640 1 -- 172.21.15.19:0/1715641959 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v94) ==== 107+0+0 (secure 0 0 0) 0x7f0ea8149eb0 con 0x7f0ea8165c70 2024-04-25T11:35:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:35:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e880744b0 msgr2=0x7f0e880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e880744b0 0x7f0e880769c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80b1590 tx=0x7f0e9c009040 comp rx=0 tx=0).stop 2024-04-25T11:35:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8165c70 msgr2=0x7f0ea8149560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8165c70 0x7f0ea8149560 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0e940108e0 tx=0x7f0e94010da0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 shutdown_connections 2024-04-25T11:35:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e880744b0 0x7f0e880769c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ea8149aa0 0x7f0ea81ed710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ea8165c70 0x7f0ea8149560 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 --2- 172.21.15.19:0/1715641959 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e90004d40 0x7f0ea8160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 >> 172.21.15.19:0/1715641959 conn(0x7f0ea80d1ae0 msgr2=0x7f0ea80b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 shutdown_connections 2024-04-25T11:35:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.098+0000 7f0eaf400640 1 -- 172.21.15.19:0/1715641959 wait complete. 2024-04-25T11:35:40.114 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:35:40.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.312+0000 7f891bd2af40 1 Processor -- start 2024-04-25T11:35:40.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.312+0000 7f891bd2af40 1 -- start start 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f891bd2af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab1e0 0x5598cb27fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f891bd2af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb1ab5f0 0x5598cb1f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f891bd2af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb284dd0 0x5598cb27a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f891bd2af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598cb1f3e70 con 0x5598cb1ab5f0 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f891bd2af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5598cb151d20 con 0x5598cb284dd0 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f891bd2af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5598cb1f4110 con 0x5598cb1ab1e0 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb1ab5f0 0x5598cb1f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb1ab5f0 0x5598cb1f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40262/0 (socket says 172.21.15.19:40262) 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 -- 172.21.15.19:0/325706043 learned_addr learned my addr 172.21.15.19:0/325706043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8914e00640 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb284dd0 0x5598cb27a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 -- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab1e0 msgr2=0x5598cb27fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab1e0 0x5598cb27fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 -- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb284dd0 msgr2=0x5598cb27a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb284dd0 0x5598cb27a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8913a00640 1 -- 172.21.15.19:0/325706043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598cb1aa610 con 0x5598cb1ab5f0 2024-04-25T11:35:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.314+0000 7f8914e00640 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb284dd0 0x5598cb27a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.315+0000 7f8913a00640 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb1ab5f0 0x5598cb1f8a10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f890000a560 tx=0x7f8900050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6eab3b599f5db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.315+0000 7f8912600640 1 -- 172.21.15.19:0/325706043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900051bc0 con 0x5598cb1ab5f0 2024-04-25T11:35:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.315+0000 7f8912600640 1 -- 172.21.15.19:0/325706043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8900051d60 con 0x5598cb1ab5f0 2024-04-25T11:35:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.315+0000 7f8912600640 1 -- 172.21.15.19:0/325706043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900065650 con 0x5598cb1ab5f0 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 -- 172.21.15.19:0/325706043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb1ab5f0 msgr2=0x5598cb1f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb1ab5f0 0x5598cb1f8a10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f890000a560 tx=0x7f8900050000 comp rx=0 tx=0).stop 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 -- 172.21.15.19:0/325706043 shutdown_connections 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb284dd0 0x5598cb27a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb1ab5f0 0x5598cb1f8a10 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 --2- 172.21.15.19:0/325706043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab1e0 0x5598cb27fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 -- 172.21.15.19:0/325706043 >> 172.21.15.19:0/325706043 conn(0x5598cb0cdac0 msgr2=0x5598cb283d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 -- 172.21.15.19:0/325706043 shutdown_connections 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.316+0000 7f891bd2af40 1 -- 172.21.15.19:0/325706043 wait complete. 2024-04-25T11:35:40.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.317+0000 7f891bd2af40 1 Processor -- start 2024-04-25T11:35:40.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.317+0000 7f891bd2af40 1 -- start start 2024-04-25T11:35:40.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f891bd2af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb1ab1e0 0x5598cb31b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f891bd2af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab5f0 0x5598cb1dcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f891bd2af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb284dd0 0x5598cb283100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f891bd2af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598cb27b7d0 con 0x5598cb284dd0 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f891bd2af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5598cb1aa8f0 con 0x5598cb1ab1e0 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f891bd2af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5598cb1aab60 con 0x5598cb1ab5f0 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f8914e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb284dd0 0x5598cb283100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f8914e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb284dd0 0x5598cb283100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40268/0 (socket says 172.21.15.19:40268) 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 learned_addr learned my addr 172.21.15.19:0/2111856547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f8913a00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab5f0 0x5598cb1dcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.319+0000 7f8914400640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb1ab1e0 0x5598cb31b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab5f0 msgr2=0x5598cb1dcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8914e00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5598cb1ab5f0 0x5598cb1dcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb1ab1e0 msgr2=0x5598cb31b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8914e00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5598cb1ab1e0 0x5598cb31b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598cb275f10 con 0x5598cb284dd0 2024-04-25T11:35:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8914e00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb284dd0 0x5598cb283100 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f890c00be10 tx=0x7f890c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8911200640 1 -- 172.21.15.19:0/2111856547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c007570 con 0x5598cb284dd0 2024-04-25T11:35:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8911200640 1 -- 172.21.15.19:0/2111856547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890c007710 con 0x5598cb284dd0 2024-04-25T11:35:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f8911200640 1 -- 172.21.15.19:0/2111856547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c007b40 con 0x5598cb284dd0 2024-04-25T11:35:40.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.320+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598cb2761f0 con 0x5598cb284dd0 2024-04-25T11:35:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.321+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598cb2767b0 con 0x5598cb284dd0 2024-04-25T11:35:40.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.323+0000 7f8911200640 1 -- 172.21.15.19:0/2111856547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f890c007ce0 con 0x5598cb284dd0 2024-04-25T11:35:40.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.324+0000 7f8911200640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88f4072e90 0x7f88f40753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.324+0000 7f8911200640 1 -- 172.21.15.19:0/2111856547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f890c093e20 con 0x5598cb284dd0 2024-04-25T11:35:40.312 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:35:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.324+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5598cb276fb0 0x5598cb2773c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.324+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb277900 con 0x5598cb276fb0 2024-04-25T11:35:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.325+0000 7f8914400640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88f4072e90 0x7f88f40753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.325+0000 7f8913a00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5598cb276fb0 0x5598cb2773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.326+0000 7f8914400640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88f4072e90 0x7f88f40753a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f890400c4d0 tx=0x7f8904005620 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.326+0000 7f8913a00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5598cb276fb0 0x5598cb2773c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.329+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v93'11 uv4 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.329+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb324f50 con 0x5598cb276fb0 2024-04-25T11:35:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.333+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v93'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.333+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3252f0 con 0x5598cb276fb0 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:35:40.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.336+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v93'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.336+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3266a0 con 0x5598cb276fb0 2024-04-25T11:35:40.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.340+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v93'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.340+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb326a40 con 0x5598cb276fb0 2024-04-25T11:35:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.343+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v93'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.343+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325690 con 0x5598cb276fb0 2024-04-25T11:35:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.346+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v93'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.347+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325a80 con 0x5598cb276fb0 2024-04-25T11:35:40.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.350+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v93'17 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.350+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.354+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v93'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.354+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.346 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:35:40.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.358+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v93'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.358+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.362+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v93'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.362+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.365+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v93'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.366+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.369+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v93'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.369+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.374+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v93'23 uv10 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.374+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.378+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v93'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.378+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.369 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:35:40.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.382+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v93'25 uv25 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.382+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.386+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v93'26 uv26 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.386+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.391+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v93'27 uv27 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.391+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb325e20 con 0x5598cb276fb0 2024-04-25T11:35:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.395+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v93'28 uv28 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.395+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb25f740 con 0x5598cb276fb0 2024-04-25T11:35:40.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.400+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v93'29 uv29 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.400+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb25f740 con 0x5598cb276fb0 2024-04-25T11:35:40.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.404+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v93'30 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.404+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.408+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v93'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.408+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.399 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:35:40.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.412+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v93'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.412+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.416+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v93'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.416+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.420+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v93'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.420+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.424+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v93'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.424+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.428+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v93'36 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.429+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.432+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v93'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.433+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:35:40.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.438+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v93'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.438+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.442+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v93'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.442+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.446+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v93'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.446+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb276fb0 2024-04-25T11:35:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.450+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v93'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.450+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5598cb3272f0 0x5598cb329710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.450+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3260e0 con 0x5598cb3272f0 2024-04-25T11:35:40.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.451+0000 7f8914400640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5598cb3272f0 0x5598cb329710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.453+0000 7f8914400640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5598cb3272f0 0x5598cb329710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.456+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v93'20 uv15 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.456+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb329c50 con 0x5598cb3272f0 2024-04-25T11:35:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.461+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v93'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.461+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb329ff0 con 0x5598cb3272f0 2024-04-25T11:35:40.452 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:35:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.465+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v93'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.465+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32a390 con 0x5598cb3272f0 2024-04-25T11:35:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.469+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v93'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.469+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32a730 con 0x5598cb3272f0 2024-04-25T11:35:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.473+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v93'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.474+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32aad0 con 0x5598cb3272f0 2024-04-25T11:35:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.478+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v93'25 uv19 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.478+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb32ae70 con 0x5598cb3272f0 2024-04-25T11:35:40.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.482+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v93'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.482+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.473 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:35:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.485+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v93'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.486+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.490+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v93'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.490+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.494+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v93'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.494+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.497+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v93'30 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.498+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.501+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v93'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.501+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.493 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:35:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.505+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v93'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.505+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32f900 con 0x5598cb3272f0 2024-04-25T11:35:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.510+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v93'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.511+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32f900 con 0x5598cb3272f0 2024-04-25T11:35:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.514+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v93'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.514+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.518+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v93'35 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.518+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.523+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v93'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.523+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.514 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:35:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.526+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v93'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.526+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32c220 con 0x5598cb3272f0 2024-04-25T11:35:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.530+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v93'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.530+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32c220 con 0x5598cb3272f0 2024-04-25T11:35:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.534+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v93'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.534+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32b210 con 0x5598cb3272f0 2024-04-25T11:35:40.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.538+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v93'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb3272f0 2024-04-25T11:35:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.539+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5598cb32b720 0x5598cb32db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.539+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb32b720 2024-04-25T11:35:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.540+0000 7f8913a00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5598cb32b720 0x5598cb32db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.541+0000 7f8913a00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5598cb32b720 0x5598cb32db40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.545+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.545+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb32b720 2024-04-25T11:35:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.549+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v93'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.549+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb32b720 2024-04-25T11:35:40.541 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:35:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.553+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v93'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.554+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32f090 con 0x5598cb32b720 2024-04-25T11:35:40.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.557+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.558+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32f090 con 0x5598cb32b720 2024-04-25T11:35:40.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.561+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.562+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb32b720 2024-04-25T11:35:40.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.566+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.566+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb276fb0 2024-04-25T11:35:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.570+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v93'5 uv4 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.570+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb276fb0 2024-04-25T11:35:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.575+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v93'6 uv6 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.575+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb276fb0 2024-04-25T11:35:40.567 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:35:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.579+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v93'7 uv7 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.579+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb32e080 con 0x5598cb276fb0 2024-04-25T11:35:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.584+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v93'8 uv8 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.584+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5598cb32e980 0x5598cb330da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.584+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3312e0 con 0x5598cb32e980 2024-04-25T11:35:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.585+0000 7f8914e00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5598cb32e980 0x5598cb330da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.586+0000 7f8914e00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5598cb32e980 0x5598cb330da0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.590+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.590+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb331680 con 0x5598cb32e980 2024-04-25T11:35:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.594+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v93'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.594+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb331b30 con 0x5598cb32e980 2024-04-25T11:35:40.586 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:35:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.598+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v93'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.599+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3333e0 con 0x5598cb32e980 2024-04-25T11:35:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.602+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.602+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb333780 con 0x5598cb32e980 2024-04-25T11:35:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.606+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.606+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3323d0 con 0x5598cb32e980 2024-04-25T11:35:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.610+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.611+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5598cb332cf0 0x5598cb335b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.611+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb336070 con 0x5598cb332cf0 2024-04-25T11:35:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.612+0000 7f8914400640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5598cb332cf0 0x5598cb335b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.613+0000 7f8914400640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5598cb332cf0 0x5598cb335b30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.617+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v93'4 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.617+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb336070 con 0x5598cb332cf0 2024-04-25T11:35:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.622+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v93'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.622+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3360f0 con 0x5598cb332cf0 2024-04-25T11:35:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:35:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.626+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v93'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.626+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3374f0 con 0x5598cb332cf0 2024-04-25T11:35:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.630+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v93'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.630+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3374f0 con 0x5598cb332cf0 2024-04-25T11:35:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.634+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v93'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.635+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3364e0 con 0x5598cb332cf0 2024-04-25T11:35:40.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.639+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v93'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.639+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3364e0 con 0x5598cb332cf0 2024-04-25T11:35:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.643+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v93'10 uv3 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.643+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb3364e0 con 0x5598cb332cf0 2024-04-25T11:35:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.647+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v93'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.647+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3364e0 con 0x5598cb332cf0 2024-04-25T11:35:40.639 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:35:40.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.651+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v93'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.651+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3364e0 con 0x5598cb332cf0 2024-04-25T11:35:40.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.656+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v93'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.656+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3364e0 con 0x5598cb332cf0 2024-04-25T11:35:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.660+0000 7f8914400640 1 -- 172.21.15.19:0/2111856547 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v93'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f89040193c0 con 0x5598cb332cf0 2024-04-25T11:35:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.660+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5598cb3369f0 0x5598cb338e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.661+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339470 con 0x5598cb3369f0 2024-04-25T11:35:40.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.661+0000 7f8913a00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5598cb3369f0 0x5598cb338e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.663+0000 7f8913a00640 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5598cb3369f0 0x5598cb338e10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.667+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v93'2 uv1 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.667+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb339470 con 0x5598cb3369f0 2024-04-25T11:35:40.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.671+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v93'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.671+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb3369f0 2024-04-25T11:35:40.663 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:35:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.675+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v93'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.675+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb3369f0 2024-04-25T11:35:40.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.680+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v93'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.680+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb3369f0 2024-04-25T11:35:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.684+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v93'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.684+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb276fb0 2024-04-25T11:35:40.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.688+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.688+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb276fb0 2024-04-25T11:35:40.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.692+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v93'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.692+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb276fb0 2024-04-25T11:35:40.684 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:35:40.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.697+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v93'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.697+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb33a8d0 con 0x5598cb276fb0 2024-04-25T11:35:40.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.701+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.701+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb33a8d0 con 0x5598cb276fb0 2024-04-25T11:35:40.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.704+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.705+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb276fb0 2024-04-25T11:35:40.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.709+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb276fb0 2024-04-25T11:35:40.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.709+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.713+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v93'4 uv3 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.713+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.717+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v93'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.718+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.721+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v93'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.710 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:35:40.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.722+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.726+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v93'7 uv7 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.727+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.732+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v93'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.732+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.736+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v93'9 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.736+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.740+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v93'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.740+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.732 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:35:40.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.744+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v93'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.744+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb3398c0 con 0x5598cb32b720 2024-04-25T11:35:40.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.749+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v93'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.749+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb33c670 con 0x5598cb32b720 2024-04-25T11:35:40.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.754+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v93'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb32b720 2024-04-25T11:35:40.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.754+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.759+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v93'4 uv3 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.759+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.763+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v93'5 uv5 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.763+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.755 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:35:40.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.767+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v93'6 uv6 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.767+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.772+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v93'7 uv7 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.772+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.777+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v93'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.778+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.782+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v93'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.782+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.775 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:35:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.787+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v93'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.787+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb33acc0 con 0x5598cb32e980 2024-04-25T11:35:40.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.792+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v93'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.792+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb33acc0 con 0x5598cb32e980 2024-04-25T11:35:40.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.796+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v93'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.797+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb32e980 2024-04-25T11:35:40.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.801+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v93'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890c01a020 con 0x5598cb32e980 2024-04-25T11:35:40.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.801+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339cb0 con 0x5598cb3369f0 2024-04-25T11:35:40.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.804+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v93'2 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.804+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb339c60 con 0x5598cb3369f0 2024-04-25T11:35:40.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.808+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v93'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.809+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c60 con 0x5598cb3369f0 2024-04-25T11:35:40.800 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:35:40.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.812+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v93'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.812+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c60 con 0x5598cb3369f0 2024-04-25T11:35:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.816+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v93'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.816+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c60 con 0x5598cb3369f0 2024-04-25T11:35:40.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.821+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v93'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.821+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.825+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v93'7 uv3 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.825+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.829+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v93'8 uv8 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.829+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.821 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:35:40.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.834+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v93'9 uv9 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.834+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.838+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v93'10 uv10 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.838+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.843+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v93'11 uv11 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.843+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.847+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v93'12 uv6 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.847+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.851+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v93'13 uv13 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.851+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.844 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:35:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.856+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v93'14 uv14 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.856+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c30 con 0x5598cb3369f0 2024-04-25T11:35:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.860+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v93'15 uv15 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.860+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb33ac20 con 0x5598cb3369f0 2024-04-25T11:35:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.865+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v93'16 uv16 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.865+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb339c10 con 0x5598cb3369f0 2024-04-25T11:35:40.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.872+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v93'17 uv17 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.872+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x5598cb33a370 con 0x5598cb3369f0 2024-04-25T11:35:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f8913a00640 1 -- 172.21.15.19:0/2111856547 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v93'18 uv18 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f890005b430 con 0x5598cb3369f0 2024-04-25T11:35:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5598cb3272f0 msgr2=0x5598cb329710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5598cb3272f0 0x5598cb329710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5598cb332cf0 msgr2=0x5598cb335b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5598cb332cf0 0x5598cb335b30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5598cb3369f0 msgr2=0x5598cb338e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5598cb3369f0 0x5598cb338e10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5598cb32b720 msgr2=0x5598cb32db40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5598cb32b720 0x5598cb32db40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5598cb276fb0 msgr2=0x5598cb2773c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5598cb276fb0 0x5598cb2773c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 reap_dead start 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5598cb32e980 msgr2=0x5598cb330da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5598cb32e980 0x5598cb330da0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.877+0000 7f8914e00640 1 -- 172.21.15.19:0/2111856547 reap_dead start 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88f4072e90 msgr2=0x7f88f40753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88f4072e90 0x7f88f40753a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f890400c4d0 tx=0x7f8904005620 comp rx=0 tx=0).stop 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb284dd0 msgr2=0x5598cb283100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb284dd0 0x5598cb283100 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f890c00be10 tx=0x7f890c009750 comp rx=0 tx=0).stop 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 shutdown_connections 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88f4072e90 0x7f88f40753a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 --2- 172.21.15.19:0/2111856547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598cb284dd0 0x5598cb283100 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 >> 172.21.15.19:0/2111856547 conn(0x5598cb0cdac0 msgr2=0x5598cb27fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 shutdown_connections 2024-04-25T11:35:40.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:40.878+0000 7f891bd2af40 1 -- 172.21.15.19:0/2111856547 wait complete. 2024-04-25T11:35:40.894 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T11:35:41.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.140+0000 7f5cb91d2f40 1 Processor -- start 2024-04-25T11:35:41.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.140+0000 7f5cb91d2f40 1 -- start start 2024-04-25T11:35:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb91d2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 0x562d64eb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb91d2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 0x562d64f2b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb91d2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64f29db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb91d2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d64e12d20 con 0x562d64f45fa0 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb91d2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d64e16d60 con 0x562d64e6c1e0 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb91d2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562d64eb53f0 con 0x562d64e6c5f0 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64f29db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64f29db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40280/0 (socket says 172.21.15.19:40280) 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2e00640 1 -- 172.21.15.19:0/988349711 learned_addr learned my addr 172.21.15.19:0/988349711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2400640 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 0x562d64eb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb1a00640 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 0x562d64f2b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2e00640 1 -- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 msgr2=0x562d64f2b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2e00640 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 0x562d64f2b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2e00640 1 -- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 msgr2=0x562d64eb9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.142+0000 7f5cb2e00640 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 0x562d64eb9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.143+0000 7f5cb2e00640 1 -- 172.21.15.19:0/988349711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d64e9e830 con 0x562d64f45fa0 2024-04-25T11:35:41.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.143+0000 7f5cb2e00640 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64f29db0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001e60 tx=0x7f5cac00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a214b99af4fa699a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.143+0000 7f5cabe00640 1 -- 172.21.15.19:0/988349711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac008070 con 0x562d64f45fa0 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.143+0000 7f5cabe00640 1 -- 172.21.15.19:0/988349711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac00d500 con 0x562d64f45fa0 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.143+0000 7f5cabe00640 1 -- 172.21.15.19:0/988349711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00d910 con 0x562d64f45fa0 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/988349711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 msgr2=0x562d64f29db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64f29db0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001e60 tx=0x7f5cac00b6a0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/988349711 shutdown_connections 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64f29db0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 0x562d64f2b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/988349711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 0x562d64eb9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/988349711 >> 172.21.15.19:0/988349711 conn(0x562d64d8eac0 msgr2=0x562d64e6af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/988349711 shutdown_connections 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.144+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/988349711 wait complete. 2024-04-25T11:35:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.145+0000 7f5cb91d2f40 1 Processor -- start 2024-04-25T11:35:41.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.145+0000 7f5cb91d2f40 1 -- start start 2024-04-25T11:35:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb91d2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 0x562d64f27f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb91d2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 0x562d64eb1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb91d2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64eb2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb91d2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d64e9df50 con 0x562d64f45fa0 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb91d2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562d64e9dc70 con 0x562d64e6c1e0 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb91d2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562d64e9e550 con 0x562d64e6c5f0 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64eb2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64eb2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40290/0 (socket says 172.21.15.19:40290) 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb2e00640 1 -- 172.21.15.19:0/2192742034 learned_addr learned my addr 172.21.15.19:0/2192742034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb2400640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 0x562d64f27f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.147+0000 7f5cb1a00640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 0x562d64eb1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5cb2e00640 1 -- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 msgr2=0x562d64eb1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5cb2e00640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562d64e6c5f0 0x562d64eb1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5cb2e00640 1 -- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 msgr2=0x562d64f27f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5cb2e00640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562d64e6c1e0 0x562d64f27f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5cb2e00640 1 -- 172.21.15.19:0/2192742034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d64eb0ba0 con 0x562d64f45fa0 2024-04-25T11:35:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5cb2e00640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64eb2310 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001ed0 tx=0x7f5cac000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5caaa00640 1 -- 172.21.15.19:0/2192742034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac020020 con 0x562d64f45fa0 2024-04-25T11:35:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5caaa00640 1 -- 172.21.15.19:0/2192742034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac01c890 con 0x562d64f45fa0 2024-04-25T11:35:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5caaa00640 1 -- 172.21.15.19:0/2192742034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac01ccc0 con 0x562d64f45fa0 2024-04-25T11:35:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.148+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d64eb0e80 con 0x562d64f45fa0 2024-04-25T11:35:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.149+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d64eb1440 con 0x562d64f45fa0 2024-04-25T11:35:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.151+0000 7f5caaa00640 1 -- 172.21.15.19:0/2192742034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5cac023020 con 0x562d64f45fa0 2024-04-25T11:35:41.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.152+0000 7f5caaa00640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c90072ec0 0x7f5c900753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.152+0000 7f5caaa00640 1 -- 172.21.15.19:0/2192742034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f5cac095b60 con 0x562d64f45fa0 2024-04-25T11:35:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.152+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562d64fd96b0 0x562d64e9d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.152+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x562d64e9d590 con 0x562d64fd96b0 2024-04-25T11:35:41.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.152+0000 7f5cb2400640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c90072ec0 0x7f5c900753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.153+0000 7f5cb1a00640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562d64fd96b0 0x562d64e9d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.154+0000 7f5cb2400640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c90072ec0 0x7f5c900753d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4056000 tx=0x7f5ca4007030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:41.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.154+0000 7f5cb1a00640 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562d64fd96b0 0x562d64e9d050 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.157+0000 7f5cb1a00640 1 -- 172.21.15.19:0/2192742034 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v93'42 uv42 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5c9c00c440 con 0x562d64fd96b0 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.157+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562d64fd96b0 msgr2=0x562d64e9d050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.157+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562d64fd96b0 0x562d64e9d050 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.157+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c90072ec0 msgr2=0x7f5c900753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.157+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c90072ec0 0x7f5c900753d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4056000 tx=0x7f5ca4007030 comp rx=0 tx=0).stop 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.158+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 msgr2=0x562d64eb2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.158+0000 7f5cb91d2f40 1 --2- 172.21.15.19:0/2192742034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d64f45fa0 0x562d64eb2310 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5cac001ed0 tx=0x7f5cac000ed0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.158+0000 7f5cb2e00640 1 -- 172.21.15.19:0/2192742034 reap_dead start 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.158+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 shutdown_connections 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.158+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 >> 172.21.15.19:0/2192742034 conn(0x562d64d8eac0 msgr2=0x562d64e6a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.158+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 shutdown_connections 2024-04-25T11:35:41.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.158+0000 7f5cb91d2f40 1 -- 172.21.15.19:0/2192742034 wait complete. 2024-04-25T11:35:41.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:35:41.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.445+0000 7ff496a00640 1 Processor -- start 2024-04-25T11:35:41.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.445+0000 7ff496a00640 1 -- start start 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.447+0000 7ff496a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b5260 0x7ff4900b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.447+0000 7ff496a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff4900b80a0 0x7ff4900d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.447+0000 7ff496a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900b84b0 con 0x7ff4900b4d80 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.447+0000 7ff496a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900d4420 con 0x7ff4900b80a0 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff496a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900d0b10 con 0x7ff4900b5260 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff496000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff4900b80a0 0x7ff4900d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff496000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff4900b80a0 0x7ff4900d51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40294/0 (socket says 172.21.15.19:40294) 2024-04-25T11:35:41.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff496000640 1 -- 172.21.15.19:0/919556544 learned_addr learned my addr 172.21.15.19:0/919556544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff494c00640 1 --2- 172.21.15.19:0/919556544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b5260 0x7ff4900b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff496000640 1 -- 172.21.15.19:0/919556544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b5260 msgr2=0x7ff4900b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff496000640 1 --2- 172.21.15.19:0/919556544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b5260 0x7ff4900b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.448+0000 7ff496000640 1 -- 172.21.15.19:0/919556544 >> v1:172.21.15.169:6789/0 conn(0x7ff4900b4d80 legacy=0x7ff4900b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.449+0000 7ff496000640 1 -- 172.21.15.19:0/919556544 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900d46c0 con 0x7ff4900b80a0 2024-04-25T11:35:41.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.450+0000 7ff496000640 1 --2- 172.21.15.19:0/919556544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b80a0 0x7ff4900d51a0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff488009870 tx=0x7ff48802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6ac082b5611def server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.450+0000 7ff48fe00640 1 -- 172.21.15.19:0/919556544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488037070 con 0x7ff4900b80a0 2024-04-25T11:35:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.450+0000 7ff48fe00640 1 -- 172.21.15.19:0/919556544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff488032440 con 0x7ff4900b80a0 2024-04-25T11:35:41.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.450+0000 7ff48fe00640 1 -- 172.21.15.19:0/919556544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488032810 con 0x7ff4900b80a0 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 -- 172.21.15.19:0/919556544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b80a0 msgr2=0x7ff4900d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 --2- 172.21.15.19:0/919556544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b80a0 0x7ff4900d51a0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff488009870 tx=0x7ff48802a270 comp rx=0 tx=0).stop 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 -- 172.21.15.19:0/919556544 shutdown_connections 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 --2- 172.21.15.19:0/919556544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b80a0 0x7ff4900d51a0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 --2- 172.21.15.19:0/919556544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b5260 0x7ff4900b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 -- 172.21.15.19:0/919556544 >> 172.21.15.19:0/919556544 conn(0x7ff4900d2bd0 msgr2=0x7ff4900d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 -- 172.21.15.19:0/919556544 shutdown_connections 2024-04-25T11:35:41.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.451+0000 7ff496a00640 1 -- 172.21.15.19:0/919556544 wait complete. 2024-04-25T11:35:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.453+0000 7ff496a00640 1 Processor -- start 2024-04-25T11:35:41.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.454+0000 7ff496a00640 1 -- start start 2024-04-25T11:35:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.455+0000 7ff496a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b4d80 0x7ff4900cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.455+0000 7ff496a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4900b5260 0x7ff4901e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.455+0000 7ff496a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b80a0 0x7ff4900cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.455+0000 7ff496a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900d0770 con 0x7ff4900b4d80 2024-04-25T11:35:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.455+0000 7ff496a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900d4c80 con 0x7ff4900b5260 2024-04-25T11:35:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.455+0000 7ff496a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900d08f0 con 0x7ff4900b80a0 2024-04-25T11:35:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.455+0000 7ff495600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b4d80 0x7ff4900cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b4d80 0x7ff4900cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40300/0 (socket says 172.21.15.19:40300) 2024-04-25T11:35:41.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 -- 172.21.15.19:0/4139585351 learned_addr learned my addr 172.21.15.19:0/4139585351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff496000640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b80a0 0x7ff4900cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff494c00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4900b5260 0x7ff4901e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 -- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b80a0 msgr2=0x7ff4900cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b80a0 0x7ff4900cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 -- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4900b5260 msgr2=0x7ff4901e89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4900b5260 0x7ff4901e89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 -- 172.21.15.19:0/4139585351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900ceb90 con 0x7ff4900b4d80 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff494c00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4900b5260 0x7ff4901e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.456+0000 7ff495600640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b4d80 0x7ff4900cfc30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c730 tx=0x7ff48000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.457+0000 7ff48d600640 1 -- 172.21.15.19:0/4139585351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480003b70 con 0x7ff4900b4d80 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.457+0000 7ff48d600640 1 -- 172.21.15.19:0/4139585351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff480003d30 con 0x7ff4900b4d80 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.457+0000 7ff48d600640 1 -- 172.21.15.19:0/4139585351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48000aa90 con 0x7ff4900b4d80 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.457+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4900cee70 con 0x7ff4900b4d80 2024-04-25T11:35:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.457+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4900cf430 con 0x7ff4900b4d80 2024-04-25T11:35:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.458+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff490076c60 con 0x7ff4900b4d80 2024-04-25T11:35:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.459+0000 7ff48d600640 1 -- 172.21.15.19:0/4139585351 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff48000ac30 con 0x7ff4900b4d80 2024-04-25T11:35:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.460+0000 7ff48d600640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4700744b0 0x7ff4700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.461+0000 7ff48d600640 1 -- 172.21.15.19:0/4139585351 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7ff480095240 con 0x7ff4900b4d80 2024-04-25T11:35:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.461+0000 7ff494c00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4700744b0 0x7ff4700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.463+0000 7ff494c00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4700744b0 0x7ff4700769c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.465+0000 7ff48d600640 1 -- 172.21.15.19:0/4139585351 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4900b84b0 con 0x7ff4900b4d80 2024-04-25T11:35:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:41.710+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff4900d6bc0 con 0x7ff4900b4d80 2024-04-25T11:35:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.114+0000 7ff48d600640 1 -- 172.21.15.19:0/4139585351 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v95) ==== 162+0+0 (secure 0 0 0) 0x7ff48005f8d0 con 0x7ff4900b4d80 2024-04-25T11:35:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:35:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4700744b0 msgr2=0x7ff4700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4700744b0 0x7ff4700769c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c008040 comp rx=0 tx=0).stop 2024-04-25T11:35:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b4d80 msgr2=0x7ff4900cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b4d80 0x7ff4900cfc30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c730 tx=0x7ff48000cbf0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 shutdown_connections 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff4700744b0 0x7ff4700769c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4900b5260 0x7ff4901e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4900b80a0 0x7ff4900cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 --2- 172.21.15.19:0/4139585351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4900b4d80 0x7ff4900cfc30 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 >> 172.21.15.19:0/4139585351 conn(0x7ff4900d2bd0 msgr2=0x7ff4900b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.118+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 shutdown_connections 2024-04-25T11:35:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.119+0000 7ff496a00640 1 -- 172.21.15.19:0/4139585351 wait complete. 2024-04-25T11:35:42.543 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:35:42.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.844+0000 7f2791800640 1 Processor -- start 2024-04-25T11:35:42.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.844+0000 7f2791800640 1 -- start start 2024-04-25T11:35:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f2791800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f278c06c5c0 0x7f278c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f2791800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278c0d1230 con 0x7f278c0b4ea0 2024-04-25T11:35:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f2791800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278c0d06b0 con 0x7f278c144370 2024-04-25T11:35:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f2791800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f278c0d0990 con 0x7f278c06c5c0 2024-04-25T11:35:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f278be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f278c06c5c0 0x7f278c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f278be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f278c06c5c0 0x7f278c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40310/0 (socket says 172.21.15.19:40310) 2024-04-25T11:35:42.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f278be00640 1 -- 172.21.15.19:0/922077283 learned_addr learned my addr 172.21.15.19:0/922077283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.847+0000 7f2790e00640 1 --1- 172.21.15.19:0/922077283 >> v1:172.21.15.19:6789/0 conn(0x7f278c0b4ea0 0x7f278c15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51532/0 (socket says 172.21.15.19:51532) 2024-04-25T11:35:42.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.848+0000 7f278be00640 1 -- 172.21.15.19:0/922077283 >> v1:172.21.15.169:6789/0 conn(0x7f278c144370 legacy=0x7f278c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.848+0000 7f278be00640 1 -- 172.21.15.19:0/922077283 >> v1:172.21.15.19:6789/0 conn(0x7f278c0b4ea0 legacy=0x7f278c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.848+0000 7f278be00640 1 -- 172.21.15.19:0/922077283 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c0d0c70 con 0x7f278c06c5c0 2024-04-25T11:35:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.848+0000 7f278aa00640 1 -- 172.21.15.19:0/922077283 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251718182 0 0) 0x7f278c0d1230 con 0x7f278c0b4ea0 2024-04-25T11:35:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.849+0000 7f278be00640 1 --2- 172.21.15.19:0/922077283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c06c5c0 0x7f278c0cfa70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f277c009870 tx=0x7f277c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c29ea1756e978e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.849+0000 7f278aa00640 1 -- 172.21.15.19:0/922077283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c037070 con 0x7f278c06c5c0 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.849+0000 7f278aa00640 1 -- 172.21.15.19:0/922077283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277c032440 con 0x7f278c06c5c0 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.849+0000 7f278aa00640 1 -- 172.21.15.19:0/922077283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c032810 con 0x7f278c06c5c0 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.850+0000 7f2791800640 1 -- 172.21.15.19:0/922077283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c06c5c0 msgr2=0x7f278c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.850+0000 7f2791800640 1 --2- 172.21.15.19:0/922077283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c06c5c0 0x7f278c0cfa70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f277c009870 tx=0x7f277c02a2a0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.850+0000 7f2791800640 1 -- 172.21.15.19:0/922077283 shutdown_connections 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.850+0000 7f2791800640 1 --2- 172.21.15.19:0/922077283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c06c5c0 0x7f278c0cfa70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.850+0000 7f2791800640 1 -- 172.21.15.19:0/922077283 >> 172.21.15.19:0/922077283 conn(0x7f278c0d1720 msgr2=0x7f278c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.850+0000 7f2791800640 1 -- 172.21.15.19:0/922077283 shutdown_connections 2024-04-25T11:35:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.850+0000 7f2791800640 1 -- 172.21.15.19:0/922077283 wait complete. 2024-04-25T11:35:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.852+0000 7f2791800640 1 Processor -- start 2024-04-25T11:35:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.853+0000 7f2791800640 1 -- start start 2024-04-25T11:35:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.854+0000 7f2791800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278c06c5c0 0x7f278c13ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.854+0000 7f2791800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c144370 0x7f278c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.854+0000 7f2791800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278c15da30 0x7f278c070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.854+0000 7f2791800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f278c0d0450 con 0x7f278c144370 2024-04-25T11:35:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.854+0000 7f2791800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f278c0d02d0 con 0x7f278c06c5c0 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.854+0000 7f2791800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f278c147760 con 0x7f278c15da30 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c144370 0x7f278c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c144370 0x7f278c0702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40326/0 (socket says 172.21.15.19:40326) 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 -- 172.21.15.19:0/4244080320 learned_addr learned my addr 172.21.15.19:0/4244080320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 -- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278c15da30 msgr2=0x7f278c070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278c15da30 0x7f278c070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 -- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278c06c5c0 msgr2=0x7f278c13ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278c06c5c0 0x7f278c13ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 -- 172.21.15.19:0/4244080320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c13e000 con 0x7f278c144370 2024-04-25T11:35:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.855+0000 7f278b400640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c144370 0x7f278c0702e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f277800b3f0 tx=0x7f277800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.856+0000 7f2783e00640 1 -- 172.21.15.19:0/4244080320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778015020 con 0x7f278c144370 2024-04-25T11:35:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.856+0000 7f2783e00640 1 -- 172.21.15.19:0/4244080320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2778011830 con 0x7f278c144370 2024-04-25T11:35:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.856+0000 7f2783e00640 1 -- 172.21.15.19:0/4244080320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778011c20 con 0x7f278c144370 2024-04-25T11:35:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.856+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c13e2e0 con 0x7f278c144370 2024-04-25T11:35:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.856+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c13e8a0 con 0x7f278c144370 2024-04-25T11:35:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.857+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c050740 con 0x7f278c144370 2024-04-25T11:35:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.859+0000 7f2783e00640 1 -- 172.21.15.19:0/4244080320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2778018020 con 0x7f278c144370 2024-04-25T11:35:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.859+0000 7f2783e00640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f276c0744b0 0x7f276c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.860+0000 7f2783e00640 1 -- 172.21.15.19:0/4244080320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f2778094ec0 con 0x7f278c144370 2024-04-25T11:35:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.860+0000 7f278be00640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f276c0744b0 0x7f276c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.862+0000 7f278be00640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f276c0744b0 0x7f276c0769c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f277c034000 tx=0x7f277c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:42.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:42.864+0000 7f2783e00640 1 -- 172.21.15.19:0/4244080320 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f277801f030 con 0x7f278c144370 2024-04-25T11:35:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.109+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f278c0d1d80 con 0x7f278c144370 2024-04-25T11:35:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.110+0000 7f2783e00640 1 -- 172.21.15.19:0/4244080320 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5177 (secure 0 0 0) 0x7f277805f670 con 0x7f278c144370 2024-04-25T11:35:43.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:43.098 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":96,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:35:42.119730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7566":{"gid":7566,"name":"a","rank":0,"incarnation":96,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/46202637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":46202637},{"type":"v1","addr":"172.21.15.19:6835","nonce":46202637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:35:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 96 2024-04-25T11:35:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f276c0744b0 msgr2=0x7f276c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f276c0744b0 0x7f276c0769c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f277c034000 tx=0x7f277c02c000 comp rx=0 tx=0).stop 2024-04-25T11:35:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c144370 msgr2=0x7f278c0702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c144370 0x7f278c0702e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f277800b3f0 tx=0x7f277800b8b0 comp rx=0 tx=0).stop 2024-04-25T11:35:43.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 shutdown_connections 2024-04-25T11:35:43.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f276c0744b0 0x7f276c0769c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:43.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f278c15da30 0x7f278c070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:43.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f278c144370 0x7f278c0702e0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:43.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 --2- 172.21.15.19:0/4244080320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f278c06c5c0 0x7f278c13ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:43.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 >> 172.21.15.19:0/4244080320 conn(0x7f278c0d1720 msgr2=0x7f278c0b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:43.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 shutdown_connections 2024-04-25T11:35:43.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:43.113+0000 7f2791800640 1 -- 172.21.15.19:0/4244080320 wait complete. 2024-04-25T11:35:43.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:35:42.119730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7566': {'gid': 7566, 'name': 'a', 'rank': 0, 'incarnation': 96, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/46202637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 46202637}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 46202637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:35:43.130 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7566:up:replay 2024-04-25T11:35:43.131 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:35:45.548 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:35:45.560+0000 7f0ea5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:35:48.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:35:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.837+0000 7f4bbd200640 1 Processor -- start 2024-04-25T11:35:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.837+0000 7f4bbd200640 1 -- start start 2024-04-25T11:35:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.839+0000 7f4bbd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 0x7f4bb806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.840+0000 7f4bbd200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb806c7d0 con 0x7f4bb8160aa0 2024-04-25T11:35:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.840+0000 7f4bbd200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb806cab0 con 0x7f4bb80d0bd0 2024-04-25T11:35:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.840+0000 7f4bbd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb806cd90 con 0x7f4bb80b6430 2024-04-25T11:35:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.840+0000 7f4bb6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4bb8160aa0 0x7f4bb8160eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53496/0 (socket says 172.21.15.19:53496) 2024-04-25T11:35:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.840+0000 7f4bb6a00640 1 -- 172.21.15.19:0/1480383294 learned_addr learned my addr 172.21.15.19:0/1480383294 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.840+0000 7f4bb7400640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 0x7f4bb806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91796773 0 0) 0x7f4bb806c7d0 con 0x7f4bb8160aa0 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b9c005730 con 0x7f4bb8160aa0 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3215772787 0 0) 0x7f4b9c005730 con 0x7f4bb8160aa0 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 msgr2=0x7f4bb806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 0x7f4bb806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 >> v1:172.21.15.169:6789/0 conn(0x7f4bb80d0bd0 legacy=0x7f4bb80d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb806d070 con 0x7f4bb8160aa0 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4ba80019f0 con 0x7f4bb8160aa0 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 >> v1:172.21.15.19:6789/0 conn(0x7f4bb8160aa0 legacy=0x7f4bb8160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c005bf0 0x7f4b9c006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b9c006a10 0x7f4b9c006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 0x7f4b9c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb806cd90 con 0x7f4b9c006a10 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c0077a0 con 0x7f4b9c005bf0 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.841+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9c00e110 con 0x7f4bb80b6430 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb6a00640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b9c006a10 0x7f4b9c006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb7400640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c005bf0 0x7f4b9c006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb7e00640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 0x7f4b9c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb6a00640 1 -- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 msgr2=0x7f4b9c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb6a00640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b6430 0x7f4b9c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb6a00640 1 -- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c005bf0 msgr2=0x7f4b9c006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb6a00640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c005bf0 0x7f4b9c006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.842+0000 7f4bb6a00640 1 -- 172.21.15.19:0/1480383294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c005730 con 0x7f4b9c006a10 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.843+0000 7f4bb6a00640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b9c006a10 0x7f4b9c006e20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800a740 tx=0x7f4b9c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559986464e1800b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.844+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800a390 con 0x7f4b9c006a10 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.844+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba802f070 con 0x7f4b9c006a10 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.844+0000 7f4bb6000640 1 -- 172.21.15.19:0/1480383294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba802bba0 con 0x7f4b9c006a10 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.844+0000 7f4bbd200640 1 -- 172.21.15.19:0/1480383294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b9c006a10 msgr2=0x7f4b9c006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.844+0000 7f4bbd200640 1 --2- 172.21.15.19:0/1480383294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b9c006a10 0x7f4b9c006e20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800a740 tx=0x7f4b9c007a40 comp rx=0 tx=0).stop 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.844+0000 7f4bb7e00640 1 -- 172.21.15.19:0/1480383294 reap_dead start 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.845+0000 7f4bbd200640 1 -- 172.21.15.19:0/1480383294 shutdown_connections 2024-04-25T11:35:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.845+0000 7f4bbd200640 1 -- 172.21.15.19:0/1480383294 >> 172.21.15.19:0/1480383294 conn(0x7f4bb80d1720 msgr2=0x7f4bb80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.845+0000 7f4bbd200640 1 -- 172.21.15.19:0/1480383294 shutdown_connections 2024-04-25T11:35:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.845+0000 7f4bbd200640 1 -- 172.21.15.19:0/1480383294 wait complete. 2024-04-25T11:35:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.847+0000 7f4bbd200640 1 Processor -- start 2024-04-25T11:35:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.847+0000 7f4bbd200640 1 -- start start 2024-04-25T11:35:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bbd200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c006a10 0x7f4bb8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bbd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb813a470 0x7f4bb81f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bbd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b1c00 0x7f4bb80b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bbd200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb806d9f0 con 0x7f4bb813a470 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bbd200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb806d870 con 0x7f4b9c006a10 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bbd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb806db70 con 0x7f4bb80b1c00 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bb6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb813a470 0x7f4bb81f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bb6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb813a470 0x7f4bb81f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51302/0 (socket says 172.21.15.19:51302) 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.849+0000 7f4bb6a00640 1 -- 172.21.15.19:0/26082085 learned_addr learned my addr 172.21.15.19:0/26082085 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb7e00640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b1c00 0x7f4bb80b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb7400640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c006a10 0x7f4bb8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb6a00640 1 -- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b1c00 msgr2=0x7f4bb80b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb6a00640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b1c00 0x7f4bb80b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb6a00640 1 -- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c006a10 msgr2=0x7f4bb8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb6a00640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c006a10 0x7f4bb8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb6a00640 1 -- 172.21.15.19:0/26082085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb80b2550 con 0x7f4bb813a470 2024-04-25T11:35:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb7400640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c006a10 0x7f4bb8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.850+0000 7f4bb6a00640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb813a470 0x7f4bb81f5b90 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f4ba802b410 tx=0x7f4bb806d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.851+0000 7f4baf400640 1 -- 172.21.15.19:0/26082085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba802b760 con 0x7f4bb813a470 2024-04-25T11:35:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.851+0000 7f4baf400640 1 -- 172.21.15.19:0/26082085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba802b900 con 0x7f4bb813a470 2024-04-25T11:35:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.851+0000 7f4baf400640 1 -- 172.21.15.19:0/26082085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba802bcb0 con 0x7f4bb813a470 2024-04-25T11:35:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.851+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb815de60 con 0x7f4bb813a470 2024-04-25T11:35:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.851+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb815e3e0 con 0x7f4bb813a470 2024-04-25T11:35:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.852+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb8050740 con 0x7f4bb813a470 2024-04-25T11:35:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.853+0000 7f4baf400640 1 -- 172.21.15.19:0/26082085 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ba8034470 con 0x7f4bb813a470 2024-04-25T11:35:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.855+0000 7f4baf400640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b940744b0 0x7f4b940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.855+0000 7f4baf400640 1 -- 172.21.15.19:0/26082085 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f4ba80b5ba0 con 0x7f4bb813a470 2024-04-25T11:35:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.856+0000 7f4bb7400640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b940744b0 0x7f4b940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.857+0000 7f4bb7400640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b940744b0 0x7f4b940769c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009ab0 tx=0x7f4ba0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:48.861+0000 7f4baf400640 1 -- 172.21.15.19:0/26082085 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bb80b2550 con 0x7f4bb813a470 2024-04-25T11:35:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.110+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bb80cea10 con 0x7f4bb813a470 2024-04-25T11:35:49.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.112+0000 7f4baf400640 1 -- 172.21.15.19:0/26082085 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5184 (secure 0 0 0) 0x7f4ba8080230 con 0x7f4bb813a470 2024-04-25T11:35:49.100 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:49.100 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:35:45.148791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7566},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7566":{"gid":7566,"name":"a","rank":0,"incarnation":96,"state":"up:active","state_seq":7,"addr":"172.21.15.19:6835/46202637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":46202637},{"type":"v1","addr":"172.21.15.19:6835","nonce":46202637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7566,"qdb_cluster":[7566]},"id":6}]} 2024-04-25T11:35:49.100 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 99 2024-04-25T11:35:49.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b940744b0 msgr2=0x7f4b940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b940744b0 0x7f4b940769c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009ab0 tx=0x7f4ba0008040 comp rx=0 tx=0).stop 2024-04-25T11:35:49.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb813a470 msgr2=0x7f4bb81f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb813a470 0x7f4bb81f5b90 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f4ba802b410 tx=0x7f4bb806d240 comp rx=0 tx=0).stop 2024-04-25T11:35:49.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 shutdown_connections 2024-04-25T11:35:49.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b940744b0 0x7f4b940769c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4bb80b1c00 0x7f4bb80b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bb813a470 0x7f4bb81f5b90 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 --2- 172.21.15.19:0/26082085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b9c006a10 0x7f4bb8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 >> 172.21.15.19:0/26082085 conn(0x7f4bb80d1720 msgr2=0x7f4bb8146300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 shutdown_connections 2024-04-25T11:35:49.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.115+0000 7f4bbd200640 1 -- 172.21.15.19:0/26082085 wait complete. 2024-04-25T11:35:49.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:35:45.148791+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7566}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7566': {'gid': 7566, 'name': 'a', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.19:6835/46202637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 46202637}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 46202637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7566, 'qdb_cluster': [7566]} 2024-04-25T11:35:49.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:35:49.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:35:49.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.422+0000 7f15fee00640 1 Processor -- start 2024-04-25T11:35:49.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.422+0000 7f15fee00640 1 -- start start 2024-04-25T11:35:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.424+0000 7f15fee00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15f8144350 0x7f15f80d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.424+0000 7f15fee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 0x7f15f8144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.424+0000 7f15fee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f806fc70 con 0x7f15f8161af0 2024-04-25T11:35:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.424+0000 7f15fee00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f15f806ff50 con 0x7f15f8144350 2024-04-25T11:35:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.424+0000 7f15fee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15f8070230 con 0x7f15f8162610 2024-04-25T11:35:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.425+0000 7f15fd000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f15f8161af0 0x7f15f8161f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53510/0 (socket says 172.21.15.19:53510) 2024-04-25T11:35:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.425+0000 7f15fd000640 1 -- 172.21.15.19:0/3001049370 learned_addr learned my addr 172.21.15.19:0/3001049370 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.425+0000 7f15fe400640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 0x7f15f8144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.425+0000 7f15fda00640 1 --2- 172.21.15.19:0/3001049370 >> v2:172.21.15.169:3300/0 conn(0x7f15f8144350 0x7f15f80d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.425+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2616898404 0 0) 0x7f15f806fc70 con 0x7f15f8161af0 2024-04-25T11:35:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.425+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e0003520 con 0x7f15f8161af0 2024-04-25T11:35:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1976812958 0 0) 0x7f15e0003520 con 0x7f15f8161af0 2024-04-25T11:35:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 msgr2=0x7f15f8144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 0x7f15f8144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 >> v2:172.21.15.169:3300/0 conn(0x7f15f8144350 msgr2=0x7f15f80d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 --2- 172.21.15.19:0/3001049370 >> v2:172.21.15.169:3300/0 conn(0x7f15f8144350 0x7f15f80d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f8070510 con 0x7f15f8161af0 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f15e4001690 con 0x7f15f8161af0 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 >> v1:172.21.15.19:6789/0 conn(0x7f15f8161af0 legacy=0x7f15f8161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15e00039b0 0x7f15e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e0004800 0x7f15e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 0x7f15e0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15f806ff50 con 0x7f15e00039b0 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8070230 con 0x7f15e0004800 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15e000b9f0 con 0x7f15f8162610 2024-04-25T11:35:49.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.426+0000 7f15fda00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15e00039b0 0x7f15e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.427+0000 7f15fd000640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e0004800 0x7f15e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.427+0000 7f15fe400640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 0x7f15e0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.427+0000 7f15fda00640 1 -- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 msgr2=0x7f15e0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.427+0000 7f15fda00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8162610 0x7f15e0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.427+0000 7f15fda00640 1 -- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e0004800 msgr2=0x7f15e0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.427+0000 7f15fda00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e0004800 0x7f15e0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.427+0000 7f15fda00640 1 -- 172.21.15.19:0/3001049370 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e0003520 con 0x7f15e00039b0 2024-04-25T11:35:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.428+0000 7f15fda00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15e00039b0 0x7f15e0003e20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009a90 tx=0x7f15e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30cdd2276e3ec09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.429+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec036070 con 0x7f15e00039b0 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.429+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec031400 con 0x7f15e00039b0 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.429+0000 7f15f7e00640 1 -- 172.21.15.19:0/3001049370 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec009690 con 0x7f15e00039b0 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.430+0000 7f15fee00640 1 -- 172.21.15.19:0/3001049370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15e00039b0 msgr2=0x7f15e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.430+0000 7f15fee00640 1 --2- 172.21.15.19:0/3001049370 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15e00039b0 0x7f15e0003e20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009a90 tx=0x7f15e0005830 comp rx=0 tx=0).stop 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.430+0000 7f15fe400640 1 -- 172.21.15.19:0/3001049370 reap_dead start 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.430+0000 7f15fee00640 1 -- 172.21.15.19:0/3001049370 shutdown_connections 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.430+0000 7f15fee00640 1 -- 172.21.15.19:0/3001049370 >> 172.21.15.19:0/3001049370 conn(0x7f15f80d1720 msgr2=0x7f15f80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.430+0000 7f15fee00640 1 -- 172.21.15.19:0/3001049370 shutdown_connections 2024-04-25T11:35:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.430+0000 7f15fee00640 1 -- 172.21.15.19:0/3001049370 wait complete. 2024-04-25T11:35:49.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.432+0000 7f15fee00640 1 Processor -- start 2024-04-25T11:35:49.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.433+0000 7f15fee00640 1 -- start start 2024-04-25T11:35:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.434+0000 7f15fee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e00039b0 0x7f15f81e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.434+0000 7f15fee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15f815d7b0 0x7f15f8160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.434+0000 7f15fee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8132d40 0x7f15f8133150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.434+0000 7f15fee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8144fe0 con 0x7f15f815d7b0 2024-04-25T11:35:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.434+0000 7f15fee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8144e60 con 0x7f15e00039b0 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.434+0000 7f15fee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15f8145160 con 0x7f15f8132d40 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.434+0000 7f15fd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15f815d7b0 0x7f15f8160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15f815d7b0 0x7f15f8160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51328/0 (socket says 172.21.15.19:51328) 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fd000640 1 -- 172.21.15.19:0/2404387537 learned_addr learned my addr 172.21.15.19:0/2404387537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fda00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e00039b0 0x7f15f81e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fd000640 1 -- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8132d40 msgr2=0x7f15f8133150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fe400640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8132d40 0x7f15f8133150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fd000640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8132d40 0x7f15f8133150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fd000640 1 -- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e00039b0 msgr2=0x7f15f81e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fd000640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e00039b0 0x7f15f81e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fd000640 1 -- 172.21.15.19:0/2404387537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f81e7d10 con 0x7f15f815d7b0 2024-04-25T11:35:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fda00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e00039b0 0x7f15f81e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.435+0000 7f15fe400640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8132d40 0x7f15f8133150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.436+0000 7f15fd000640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15f815d7b0 0x7f15f8160b10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f15e4004e60 tx=0x7f15e4007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.436+0000 7f15f5600640 1 -- 172.21.15.19:0/2404387537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e400f7b0 con 0x7f15f815d7b0 2024-04-25T11:35:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.436+0000 7f15f5600640 1 -- 172.21.15.19:0/2404387537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e4018070 con 0x7f15f815d7b0 2024-04-25T11:35:49.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.436+0000 7f15f5600640 1 -- 172.21.15.19:0/2404387537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e400fb70 con 0x7f15f815d7b0 2024-04-25T11:35:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.436+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f81e7ff0 con 0x7f15f815d7b0 2024-04-25T11:35:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.436+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f81e85b0 con 0x7f15f815d7b0 2024-04-25T11:35:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.439+0000 7f15f5600640 1 -- 172.21.15.19:0/2404387537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15e400fd10 con 0x7f15f815d7b0 2024-04-25T11:35:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.440+0000 7f15f5600640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15d8074530 0x7f15d8076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.440+0000 7f15f5600640 1 -- 172.21.15.19:0/2404387537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f15e4095190 con 0x7f15f815d7b0 2024-04-25T11:35:49.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.440+0000 7f160741c740 1 Processor -- start 2024-04-25T11:35:49.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.440+0000 7f15fda00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15d8074530 0x7f15d8076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.440+0000 7f160741c740 1 -- start start 2024-04-25T11:35:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f15fda00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15d8074530 0x7f15d8076a40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f15ec033000 tx=0x7f15ec008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8847d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f160741c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb88497f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb88498480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f160741c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb883b6600 con 0x55fb880873b0 2024-04-25T11:35:49.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fb88353100 con 0x55fb8808f590 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fb883b25c0 con 0x55fb880b0270 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f15fd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb88497f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f15fd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb88497f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51344/0 (socket says 172.21.15.19:51344) 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.442+0000 7f15fd000640 1 -- 172.21.15.19:0/4151005787 learned_addr learned my addr 172.21.15.19:0/4151005787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fda00640 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8847d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fe400640 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb88498480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fd000640 1 -- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 msgr2=0x55fb8847d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fd000640 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8847d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fd000640 1 -- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 msgr2=0x55fb88498480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fd000640 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb88498480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fd000640 1 -- 172.21.15.19:0/4151005787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb880c82d0 con 0x55fb880873b0 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fe400640 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb88498480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.443+0000 7f15fd000640 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb88497f40 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f15e405b2b0 tx=0x7f15e405f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea2a2b95448b9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.444+0000 7f15ea000640 1 -- 172.21.15.19:0/4151005787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e40635e0 con 0x55fb880873b0 2024-04-25T11:35:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.444+0000 7f15ea000640 1 -- 172.21.15.19:0/4151005787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e4068000 con 0x55fb880873b0 2024-04-25T11:35:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.444+0000 7f15ea000640 1 -- 172.21.15.19:0/4151005787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4063370 con 0x55fb880873b0 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.444+0000 7f160741c740 1 -- 172.21.15.19:0/4151005787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 msgr2=0x55fb88497f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.444+0000 7f160741c740 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb88497f40 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f15e405b2b0 tx=0x7f15e405f130 comp rx=0 tx=0).stop 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.445+0000 7f160741c740 1 -- 172.21.15.19:0/4151005787 shutdown_connections 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.445+0000 7f160741c740 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb88498480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.445+0000 7f160741c740 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb88497f40 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.445+0000 7f160741c740 1 --2- 172.21.15.19:0/4151005787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8847d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.445+0000 7f160741c740 1 -- 172.21.15.19:0/4151005787 >> 172.21.15.19:0/4151005787 conn(0x55fb880a43c0 msgr2=0x55fb88452870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.445+0000 7f160741c740 1 -- 172.21.15.19:0/4151005787 shutdown_connections 2024-04-25T11:35:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.445+0000 7f160741c740 1 -- 172.21.15.19:0/4151005787 wait complete. 2024-04-25T11:35:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.446+0000 7f160741c740 1 -- start start 2024-04-25T11:35:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.446+0000 7f160741c740 1 Processor -- start 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f160741c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb8847e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb8849c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8849c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f160741c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb8833df70 con 0x55fb880873b0 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fb884a8af0 con 0x55fb8808f590 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fb8847e840 con 0x55fb880b0270 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f160741c740 10 client.0 _ll_register_callbacks cb 0x55fb883ac480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:35:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb8847e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb8847e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51354/0 (socket says 172.21.15.19:51354) 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 -- 172.21.15.19:0/160502707 learned_addr learned my addr 172.21.15.19:0/160502707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fd000640 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb8849c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fe400640 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8849c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 -- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 msgr2=0x55fb8849c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8849c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 -- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 msgr2=0x55fb8849c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb8849c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.448+0000 7f15fda00640 1 -- 172.21.15.19:0/160502707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb881926b0 con 0x55fb880873b0 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f15fda00640 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb8847e120 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009bb0 tx=0x7f15ec02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e710781a3861fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f15dea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:35:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f15dea00640 1 -- 172.21.15.19:0/160502707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec036040 con 0x55fb880873b0 2024-04-25T11:35:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f15dea00640 1 -- 172.21.15.19:0/160502707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f15ec03cc50 con 0x55fb880873b0 2024-04-25T11:35:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e4068000 con 0x55fb880873b0 2024-04-25T11:35:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f15dea00640 1 -- 172.21.15.19:0/160502707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f15ec056050 con 0x55fb880873b0 2024-04-25T11:35:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f160741c740 10 client.8031 fetch_fsmap learned FSMap version 99 2024-04-25T11:35:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.449+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55fb884a8890 con 0x55fb880873b0 2024-04-25T11:35:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f15dea00640 1 -- 172.21.15.19:0/160502707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e4068000 con 0x55fb880873b0 2024-04-25T11:35:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f15dea00640 1 -- 172.21.15.19:0/160502707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f15ec059020 con 0x55fb880873b0 2024-04-25T11:35:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f160741c740 10 client.8031 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f160741c740 10 client.8031 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f160741c740 20 client.8031 populate_metadata read hostname 'smithi019' 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f160741c740 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8806ee40 0x55fb884a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f160741c740 4 client.8031 mds_command: new command op to 7566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 --> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb883c4210 con 0x55fb8806ee40 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.450+0000 7f15fd000640 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8806ee40 0x55fb884a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.451+0000 7f15fd000640 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8806ee40 0x55fb884a7be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6993b64720f99 server_cookie=6f9e6b3fd4ae7b3d in_seq=0 out_seq=0 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.451+0000 7f15dea00640 10 client.8031 ms_handle_connect on v2:172.21.15.19:6834/46202637 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f15dea00640 1 -- 172.21.15.19:0/160502707 <== mds.0 v2:172.21.15.19:6834/46202637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb883c4210 con 0x55fb8806ee40 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f15dea00640 10 client.8031 handle_command_reply: tid=0 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 client.8031 shutdown 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 msgr2=0x55fb8847e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb8847e120 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009bb0 tx=0x7f15ec02c000 comp rx=0 tx=0).stop 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 shutdown_connections 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8806ee40 0x55fb884a7be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb8808f590 0x55fb8849c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880b0270 0x55fb8849c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f15dea00640 0 client.8031 ms_handle_reset on v2:172.21.15.19:6834/46202637 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 --2- 172.21.15.19:0/160502707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880873b0 0x55fb8847e120 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.452+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 >> 172.21.15.19:0/160502707 conn(0x55fb880a43c0 msgr2=0x55fb88452870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.453+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 shutdown_connections 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.453+0000 7f160741c740 1 -- 172.21.15.19:0/160502707 wait complete. 2024-04-25T11:35:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.453+0000 7f160741c740 20 client.8031 trim_cache size 0 max 16384 2024-04-25T11:35:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.474+0000 7f160741c740 1 Processor -- start 2024-04-25T11:35:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.474+0000 7f160741c740 1 -- start start 2024-04-25T11:35:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.475+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880873b0 0x55fb8844cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.475+0000 7f160741c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb8808f590 0x55fb88490df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.475+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880b0270 0x55fb884a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.475+0000 7f160741c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8202220 con 0x55fb8808f590 2024-04-25T11:35:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.475+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15f815f150 con 0x55fb880b0270 2024-04-25T11:35:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.475+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15f8201bd0 con 0x55fb880873b0 2024-04-25T11:35:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.475+0000 7f15fd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb8808f590 0x55fb88490df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb8808f590 0x55fb88490df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51358/0 (socket says 172.21.15.19:51358) 2024-04-25T11:35:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 -- 172.21.15.19:0/1745991889 learned_addr learned my addr 172.21.15.19:0/1745991889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fda00640 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880873b0 0x55fb8844cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fe400640 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880b0270 0x55fb884a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 -- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880873b0 msgr2=0x55fb8844cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880873b0 0x55fb8844cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 -- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880b0270 msgr2=0x55fb884a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880b0270 0x55fb884a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 -- 172.21.15.19:0/1745991889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb88478cd0 con 0x55fb8808f590 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fe400640 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880b0270 0x55fb884a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.476+0000 7f15fd000640 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb8808f590 0x55fb88490df0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f15e405b820 tx=0x7f15e4094000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bbaba4c5161b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f15e9600640 1 -- 172.21.15.19:0/1745991889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4068000 con 0x55fb8808f590 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f15e9600640 1 -- 172.21.15.19:0/1745991889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e4063170 con 0x55fb8808f590 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f15e9600640 1 -- 172.21.15.19:0/1745991889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e405e490 con 0x55fb8808f590 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f160741c740 1 -- 172.21.15.19:0/1745991889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb8808f590 msgr2=0x55fb88490df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f160741c740 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb8808f590 0x55fb88490df0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f15e405b820 tx=0x7f15e4094000 comp rx=0 tx=0).stop 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f160741c740 1 -- 172.21.15.19:0/1745991889 shutdown_connections 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f160741c740 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880b0270 0x55fb884a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f160741c740 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb8808f590 0x55fb88490df0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f160741c740 1 --2- 172.21.15.19:0/1745991889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb880873b0 0x55fb8844cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.477+0000 7f160741c740 1 -- 172.21.15.19:0/1745991889 >> 172.21.15.19:0/1745991889 conn(0x55fb880a43c0 msgr2=0x55fb8844d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.478+0000 7f160741c740 1 -- 172.21.15.19:0/1745991889 shutdown_connections 2024-04-25T11:35:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.478+0000 7f160741c740 1 -- 172.21.15.19:0/1745991889 wait complete. 2024-04-25T11:35:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.479+0000 7f160741c740 1 -- start start 2024-04-25T11:35:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.479+0000 7f160741c740 1 Processor -- start 2024-04-25T11:35:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.480+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880873b0 0x55fb884a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f160741c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb8808f590 0x55fb884b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f160741c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880b0270 0x55fb884b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f160741c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fb8848f1c0 con 0x55fb880b0270 2024-04-25T11:35:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fb8847ff30 con 0x55fb880873b0 2024-04-25T11:35:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f160741c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fb8848f7e0 con 0x55fb8808f590 2024-04-25T11:35:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f160741c740 10 client.0 _ll_register_callbacks cb 0x7f15ec034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f15fe400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880b0270 0x55fb884b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f15fe400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880b0270 0x55fb884b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51364/0 (socket says 172.21.15.19:51364) 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f15fe400640 1 -- 172.21.15.19:0/2752443696 learned_addr learned my addr 172.21.15.19:0/2752443696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f15fda00640 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880873b0 0x55fb884a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.481+0000 7f15fd000640 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb8808f590 0x55fb884b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15fe400640 1 -- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb8808f590 msgr2=0x55fb884b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15fe400640 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb8808f590 0x55fb884b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15fe400640 1 -- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880873b0 msgr2=0x55fb884a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15fe400640 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880873b0 0x55fb884a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15fe400640 1 -- 172.21.15.19:0/2752443696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb8847f410 con 0x55fb880b0270 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15fe400640 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880b0270 0x55fb884b0fa0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f15f80b13b0 tx=0x7f15f00136e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5879ca8f65dd5d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15dea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15dea00640 1 -- 172.21.15.19:0/2752443696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0009e00 con 0x55fb880b0270 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f15dea00640 1 -- 172.21.15.19:0/2752443696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f15f0003990 con 0x55fb880b0270 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.482+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e4063170 con 0x55fb880b0270 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f15dea00640 1 -- 172.21.15.19:0/2752443696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f15f0013e70 con 0x55fb880b0270 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 10 client.8037 fetch_fsmap learned FSMap version 99 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55fb884518a0 con 0x55fb880b0270 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f15dea00640 1 -- 172.21.15.19:0/2752443696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e4063170 con 0x55fb880b0270 2024-04-25T11:35:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f15dea00640 1 -- 172.21.15.19:0/2752443696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f15f002b080 con 0x55fb880b0270 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 10 client.8037 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 10 client.8037 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 20 client.8037 populate_metadata read hostname 'smithi019' 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8848d9b0 0x55fb8848ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 4 client.8037 mds_command: new command op to 7566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 --> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] -- command(tid 0: {"prefix": "status"}) -- 0x55fb883be080 con 0x55fb8848d9b0 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.483+0000 7f15fda00640 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8848d9b0 0x55fb8848ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.484+0000 7f15dea00640 10 client.8037 ms_handle_connect on v2:172.21.15.19:6834/46202637 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.484+0000 7f15fda00640 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8848d9b0 0x55fb8848ddc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec641b920f0f72d1 server_cookie=7e3a1504b58d6c2b in_seq=0 out_seq=0 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.484+0000 7f15dea00640 1 -- 172.21.15.19:0/2752443696 <== mds.0 v2:172.21.15.19:6834/46202637 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fb883be080 con 0x55fb8848d9b0 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.484+0000 7f15dea00640 10 client.8037 handle_command_reply: tid=0 2024-04-25T11:35:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.484+0000 7f160741c740 1 client.8037 shutdown 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.484+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880b0270 msgr2=0x55fb884b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.484+0000 7f160741c740 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880b0270 0x55fb884b0fa0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f15f80b13b0 tx=0x7f15f00136e0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 shutdown_connections 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:6834/46202637,v1:172.21.15.19:6835/46202637] conn(0x55fb8848d9b0 0x55fb8848ddc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fb880b0270 0x55fb884b0fa0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fb8808f590 0x55fb884b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 --2- 172.21.15.19:0/2752443696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fb880873b0 0x55fb884a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f15dea00640 0 client.8037 ms_handle_reset on v2:172.21.15.19:6834/46202637 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 >> 172.21.15.19:0/2752443696 conn(0x55fb880a43c0 msgr2=0x55fb88490df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 shutdown_connections 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 1 -- 172.21.15.19:0/2752443696 wait complete. 2024-04-25T11:35:49.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.485+0000 7f160741c740 20 client.8037 trim_cache size 0 max 16384 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 7566, 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3603131619999997, 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 99, 2024-04-25T11:35:49.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-04-25T11:35:49.478 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T11:35:49.478 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 20.786369625999999 2024-04-25T11:35:49.478 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15d8074530 msgr2=0x7f15d8076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15d8074530 0x7f15d8076a40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f15ec033000 tx=0x7f15ec008030 comp rx=0 tx=0).stop 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15f815d7b0 msgr2=0x7f15f8160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15f815d7b0 0x7f15f8160b10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f15e4004e60 tx=0x7f15e4007430 comp rx=0 tx=0).stop 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 shutdown_connections 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15d8074530 0x7f15d8076a40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15f8132d40 0x7f15f8133150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15f815d7b0 0x7f15f8160b10 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 --2- 172.21.15.19:0/2404387537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15e00039b0 0x7f15f81e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 >> 172.21.15.19:0/2404387537 conn(0x7f15f80d1720 msgr2=0x7f15f8161f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 shutdown_connections 2024-04-25T11:35:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.486+0000 7f15fee00640 1 -- 172.21.15.19:0/2404387537 wait complete. 2024-04-25T11:35:49.500 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:35:49.500 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2024-04-25T11:35:49.501 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:35:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.800+0000 7fea9ea00640 1 Processor -- start 2024-04-25T11:35:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.800+0000 7fea9ea00640 1 -- start start 2024-04-25T11:35:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.802+0000 7fea9ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fea98144500 0x7fea98070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.802+0000 7fea9ea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea9806f5f0 con 0x7fea98162310 2024-04-25T11:35:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.802+0000 7fea9ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea9806f8d0 con 0x7fea980cdb90 2024-04-25T11:35:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.802+0000 7fea9ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fea98070200 con 0x7fea98144500 2024-04-25T11:35:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.803+0000 7fea97e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fea98144500 0x7fea98070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.803+0000 7fea9ca00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fea98162310 0x7fea980ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53518/0 (socket says 172.21.15.19:53518) 2024-04-25T11:35:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.803+0000 7fea9ca00640 1 -- 172.21.15.19:0/1675754351 learned_addr learned my addr 172.21.15.19:0/1675754351 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.803+0000 7fea97e00640 1 -- 172.21.15.19:0/1675754351 >> v1:172.21.15.169:6789/0 conn(0x7fea980cdb90 legacy=0x7fea98161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.803+0000 7fea97e00640 1 -- 172.21.15.19:0/1675754351 >> v1:172.21.15.19:6789/0 conn(0x7fea98162310 legacy=0x7fea980ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.803+0000 7fea97e00640 1 -- 172.21.15.19:0/1675754351 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9806fbb0 con 0x7fea98144500 2024-04-25T11:35:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.803+0000 7fea96a00640 1 -- 172.21.15.19:0/1675754351 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66512191 0 0) 0x7fea9806f5f0 con 0x7fea98162310 2024-04-25T11:35:49.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.804+0000 7fea97e00640 1 --2- 172.21.15.19:0/1675754351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea98144500 0x7fea98070690 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fea90009d90 tx=0x7fea9002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29809a264e0eaeda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.805+0000 7fea96a00640 1 -- 172.21.15.19:0/1675754351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90007740 con 0x7fea98144500 2024-04-25T11:35:49.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.805+0000 7fea96a00640 1 -- 172.21.15.19:0/1675754351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea900078e0 con 0x7fea98144500 2024-04-25T11:35:49.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.805+0000 7fea96a00640 1 -- 172.21.15.19:0/1675754351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90007cb0 con 0x7fea98144500 2024-04-25T11:35:49.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.806+0000 7fea9ea00640 1 -- 172.21.15.19:0/1675754351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea98144500 msgr2=0x7fea98070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.806+0000 7fea9ea00640 1 --2- 172.21.15.19:0/1675754351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea98144500 0x7fea98070690 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fea90009d90 tx=0x7fea9002a660 comp rx=0 tx=0).stop 2024-04-25T11:35:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.806+0000 7fea9ea00640 1 -- 172.21.15.19:0/1675754351 shutdown_connections 2024-04-25T11:35:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.806+0000 7fea9ea00640 1 --2- 172.21.15.19:0/1675754351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea98144500 0x7fea98070690 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.806+0000 7fea9ea00640 1 -- 172.21.15.19:0/1675754351 >> 172.21.15.19:0/1675754351 conn(0x7fea98073050 msgr2=0x7fea98073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.806+0000 7fea9ea00640 1 -- 172.21.15.19:0/1675754351 shutdown_connections 2024-04-25T11:35:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.806+0000 7fea9ea00640 1 -- 172.21.15.19:0/1675754351 wait complete. 2024-04-25T11:35:49.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.808+0000 7fea9ea00640 1 Processor -- start 2024-04-25T11:35:49.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.808+0000 7fea9ea00640 1 -- start start 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea980cdb90 0x7fea980cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea98144500 0x7fea98137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea981f6550 0x7fea981f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea9806f2e0 con 0x7fea981f6550 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea9806f160 con 0x7fea98144500 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fea980cdfa0 con 0x7fea980cdb90 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea981f6550 0x7fea981f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea981f6550 0x7fea981f6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51382/0 (socket says 172.21.15.19:51382) 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.810+0000 7fea9ca00640 1 -- 172.21.15.19:0/3808458892 learned_addr learned my addr 172.21.15.19:0/3808458892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea97e00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea980cdb90 0x7fea980cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea97400640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea98144500 0x7fea98137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea9ca00640 1 -- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea980cdb90 msgr2=0x7fea980cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea9ca00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea980cdb90 0x7fea980cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea9ca00640 1 -- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea98144500 msgr2=0x7fea98137e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea9ca00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea98144500 0x7fea98137e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea9ca00640 1 -- 172.21.15.19:0/3808458892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea98138a00 con 0x7fea981f6550 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea97400640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea98144500 0x7fea98137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea97e00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea980cdb90 0x7fea980cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:49.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea9ca00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea981f6550 0x7fea981f6960 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c440 tx=0x7fea8800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.811+0000 7fea8fe00640 1 -- 172.21.15.19:0/3808458892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880074e0 con 0x7fea981f6550 2024-04-25T11:35:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.812+0000 7fea8fe00640 1 -- 172.21.15.19:0/3808458892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea880047e0 con 0x7fea981f6550 2024-04-25T11:35:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.812+0000 7fea8fe00640 1 -- 172.21.15.19:0/3808458892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88004bd0 con 0x7fea981f6550 2024-04-25T11:35:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.812+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea980b4b40 con 0x7fea981f6550 2024-04-25T11:35:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.812+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea980b5100 con 0x7fea981f6550 2024-04-25T11:35:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.813+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea98050740 con 0x7fea981f6550 2024-04-25T11:35:49.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.814+0000 7fea8fe00640 1 -- 172.21.15.19:0/3808458892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea88011070 con 0x7fea981f6550 2024-04-25T11:35:49.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.815+0000 7fea8fe00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fea740744b0 0x7fea740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:49.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.815+0000 7fea8fe00640 1 -- 172.21.15.19:0/3808458892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fea88094e50 con 0x7fea981f6550 2024-04-25T11:35:49.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.816+0000 7fea97e00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fea740744b0 0x7fea740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:49.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.817+0000 7fea97e00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fea740744b0 0x7fea740769c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fea90034000 tx=0x7fea9002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:49.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:49.821+0000 7fea8fe00640 1 -- 172.21.15.19:0/3808458892 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea8805f5a0 con 0x7fea981f6550 2024-04-25T11:35:50.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.064+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea98138790 con 0x7fea981f6550 2024-04-25T11:35:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.065+0000 7fea8fe00640 1 -- 172.21.15.19:0/3808458892 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7fea8805eea0 con 0x7fea981f6550 2024-04-25T11:35:50.053 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fea740744b0 msgr2=0x7fea740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fea740744b0 0x7fea740769c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fea90034000 tx=0x7fea9002f000 comp rx=0 tx=0).stop 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea981f6550 msgr2=0x7fea981f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea981f6550 0x7fea981f6960 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c440 tx=0x7fea8800c900 comp rx=0 tx=0).stop 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 shutdown_connections 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fea740744b0 0x7fea740769c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea98144500 0x7fea98137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea981f6550 0x7fea981f6960 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 --2- 172.21.15.19:0/3808458892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea980cdb90 0x7fea980cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 >> 172.21.15.19:0/3808458892 conn(0x7fea98073050 msgr2=0x7fea981609b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 shutdown_connections 2024-04-25T11:35:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50.068+0000 7fea9ea00640 1 -- 172.21.15.19:0/3808458892 wait complete. 2024-04-25T11:35:50.080 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:35:50.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:35:50.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:35:50.080 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:35:50.080 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:50.081 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:35:50.081 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:35:50.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:35:50.081 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:35:50.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:50.081 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:35:50.132 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:35:50.132 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:35:50.132 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:35:50.132 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:50.132 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:35:50.132 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42123sec preferred_lft 42123sec 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:35:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:50.134 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:35:50.135 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:35:50.135 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:35:50.135 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:35:50.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:50.135 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:35:50.211 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:35:50.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:50.215 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:35:50.261 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:35:50.261 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:35:50.261 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:35:50.261 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:35:50.261 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:35:50.262 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:35:50.262 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:35:50.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:35:50.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:35:50.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:35:50.263 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:35:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:35:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:35:50.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:50.524 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:35:50.577 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:35:50.577 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:35:50.578 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:35:50.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:50.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:35:50.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:35:50.667 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:50.723 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:35:50.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:35:50.724 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:50.777 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:35:50.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:50.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:35:50.996 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:35:50.997 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:35:51.038 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:35:51.039 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:35:51.039 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:35:51.039 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:35:51.039 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:35:51.040 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:35:51.040 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:35:51.040 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:35:51.041 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:35:51.041 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:35:51.045 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:35:51.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:51.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:35:51.117 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:35:51.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:35:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:35:51.243 INFO:teuthology.orchestra.run.smithi019.stdout:client8049 2024-04-25T11:35:51.243 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:51.243 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:51.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:51.292 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:35:51.340 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:35:51.395 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:35:51.407+0000 7f0ea0e00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-04-25T11:35:51.396 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:35:51.396 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:35:51.396 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:51.396 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:35:51.397 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2024-04-25T11:35:51.397 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2024-04-25T11:35:51.397 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:51.397 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:51.453 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:35:51.453 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:35:51.512 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:35:51.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.811+0000 7f11ab000640 1 Processor -- start 2024-04-25T11:35:51.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.811+0000 7f11ab000640 1 -- start start 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11ab000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a40cf5e0 0x7f11a40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11ab000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11a406c5c0 0x7f11a406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11ab000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a40cff30 con 0x7f11a41443c0 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11ab000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f11a40708d0 con 0x7f11a406c5c0 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11ab000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11a415f160 con 0x7f11a40cf5e0 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11a9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11a406c5c0 0x7f11a406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11a3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f11a41443c0 0x7f11a406fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53528/0 (socket says 172.21.15.19:53528) 2024-04-25T11:35:51.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11a3e00640 1 -- 172.21.15.19:0/3731985044 learned_addr learned my addr 172.21.15.19:0/3731985044 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.814+0000 7f11a3400640 1 --2- 172.21.15.19:0/3731985044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a40cf5e0 0x7f11a40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.815+0000 7f11a9000640 1 -- 172.21.15.19:0/3731985044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a40cf5e0 msgr2=0x7f11a40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.815+0000 7f11a9000640 1 --2- 172.21.15.19:0/3731985044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a40cf5e0 0x7f11a40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.815+0000 7f11a9000640 1 -- 172.21.15.19:0/3731985044 >> v1:172.21.15.19:6789/0 conn(0x7f11a41443c0 legacy=0x7f11a406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.815+0000 7f11a2a00640 1 -- 172.21.15.19:0/3731985044 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877896113 0 0) 0x7f1194001110 con 0x7f11a41443c0 2024-04-25T11:35:51.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.815+0000 7f11a9000640 1 -- 172.21.15.19:0/3731985044 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a415f5f0 con 0x7f11a406c5c0 2024-04-25T11:35:51.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.816+0000 7f11a9000640 1 --2- 172.21.15.19:0/3731985044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a406c5c0 0x7f11a406c9d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c4de4b50a950d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:51.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.817+0000 7f11a2a00640 1 -- 172.21.15.19:0/3731985044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c007740 con 0x7f11a406c5c0 2024-04-25T11:35:51.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.817+0000 7f11a2a00640 1 -- 172.21.15.19:0/3731985044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c0078e0 con 0x7f11a406c5c0 2024-04-25T11:35:51.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.817+0000 7f11a2a00640 1 -- 172.21.15.19:0/3731985044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c007cb0 con 0x7f11a406c5c0 2024-04-25T11:35:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.817+0000 7f11ab000640 1 -- 172.21.15.19:0/3731985044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a406c5c0 msgr2=0x7f11a406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.817+0000 7f11ab000640 1 --2- 172.21.15.19:0/3731985044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a406c5c0 0x7f11a406c9d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c02a660 comp rx=0 tx=0).stop 2024-04-25T11:35:51.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.818+0000 7f11ab000640 1 -- 172.21.15.19:0/3731985044 shutdown_connections 2024-04-25T11:35:51.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.818+0000 7f11ab000640 1 --2- 172.21.15.19:0/3731985044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a406c5c0 0x7f11a406c9d0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.818+0000 7f11ab000640 1 --2- 172.21.15.19:0/3731985044 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a40cf5e0 0x7f11a40cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.818+0000 7f11ab000640 1 -- 172.21.15.19:0/3731985044 >> 172.21.15.19:0/3731985044 conn(0x7f11a40d1ae0 msgr2=0x7f11a40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:51.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.818+0000 7f11ab000640 1 -- 172.21.15.19:0/3731985044 shutdown_connections 2024-04-25T11:35:51.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.818+0000 7f11ab000640 1 -- 172.21.15.19:0/3731985044 wait complete. 2024-04-25T11:35:51.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.820+0000 7f11ab000640 1 Processor -- start 2024-04-25T11:35:51.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.820+0000 7f11ab000640 1 -- start start 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11ab000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a406c5c0 0x7f11a41f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11ab000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a40cf5e0 0x7f11a4143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11ab000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a41443c0 0x7f11a4150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11ab000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11a415edf0 con 0x7f11a40cf5e0 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11ab000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11a415eb10 con 0x7f11a41443c0 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11ab000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11a41447d0 con 0x7f11a406c5c0 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a40cf5e0 0x7f11a4143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a40cf5e0 0x7f11a4143010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51406/0 (socket says 172.21.15.19:51406) 2024-04-25T11:35:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11a3400640 1 -- 172.21.15.19:0/203953371 learned_addr learned my addr 172.21.15.19:0/203953371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.822+0000 7f11a3e00640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a406c5c0 0x7f11a41f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f11a3400640 1 -- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a406c5c0 msgr2=0x7f11a41f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f11a3400640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a406c5c0 0x7f11a41f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f11a3400640 1 -- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a41443c0 msgr2=0x7f11a4150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f11a3400640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a41443c0 0x7f11a4150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f11a3400640 1 -- 172.21.15.19:0/203953371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a414f3f0 con 0x7f11a40cf5e0 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f11a3400640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a40cf5e0 0x7f11a4143010 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f119000d600 tx=0x7f119000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f119be00640 1 -- 172.21.15.19:0/203953371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190004240 con 0x7f11a40cf5e0 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f119be00640 1 -- 172.21.15.19:0/203953371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11900043e0 con 0x7f11a40cf5e0 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f119be00640 1 -- 172.21.15.19:0/203953371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190003830 con 0x7f11a40cf5e0 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.823+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a414f6d0 con 0x7f11a40cf5e0 2024-04-25T11:35:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.824+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a414fc90 con 0x7f11a40cf5e0 2024-04-25T11:35:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.825+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a4068f70 con 0x7f11a40cf5e0 2024-04-25T11:35:51.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.826+0000 7f119be00640 1 -- 172.21.15.19:0/203953371 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f11900039d0 con 0x7f11a40cf5e0 2024-04-25T11:35:51.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.827+0000 7f119be00640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11840744b0 0x7f11840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.827+0000 7f119be00640 1 -- 172.21.15.19:0/203953371 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f1190095580 con 0x7f11a40cf5e0 2024-04-25T11:35:51.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.828+0000 7f11a3e00640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11840744b0 0x7f11840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.829+0000 7f11a3e00640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11840744b0 0x7f11840769c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1194000f60 tx=0x7f1194007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:51.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:51.832+0000 7f119be00640 1 -- 172.21.15.19:0/203953371 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11a4068f70 con 0x7f11a40cf5e0 2024-04-25T11:35:52.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.036+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11a414f180 con 0x7f11a40cf5e0 2024-04-25T11:35:52.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.039+0000 7f119be00640 1 -- 172.21.15.19:0/203953371 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16801 (secure 0 0 0) 0x7f119005fca0 con 0x7f11a40cf5e0 2024-04-25T11:35:52.028 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:52.028 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":93,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:35:28.536947+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:35:52.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11840744b0 msgr2=0x7f11840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11840744b0 0x7f11840769c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1194000f60 tx=0x7f1194007c00 comp rx=0 tx=0).stop 2024-04-25T11:35:52.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a40cf5e0 msgr2=0x7f11a4143010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a40cf5e0 0x7f11a4143010 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f119000d600 tx=0x7f119000dac0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 shutdown_connections 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11840744b0 0x7f11840769c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11a41443c0 0x7f11a4150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11a40cf5e0 0x7f11a4143010 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 --2- 172.21.15.19:0/203953371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11a406c5c0 0x7f11a41f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 >> 172.21.15.19:0/203953371 conn(0x7f11a40d1ae0 msgr2=0x7f11a4160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 shutdown_connections 2024-04-25T11:35:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.042+0000 7f11ab000640 1 -- 172.21.15.19:0/203953371 wait complete. 2024-04-25T11:35:52.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:35:52.060 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:35:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.355+0000 7f9f32800640 1 Processor -- start 2024-04-25T11:35:52.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.355+0000 7f9f32800640 1 -- start start 2024-04-25T11:35:52.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.357+0000 7f9f32800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9f2c06dce0 0x7f9f2c0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.357+0000 7f9f32800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 0x7f9f2c160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.358+0000 7f9f32800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2c071110 con 0x7f9f2c070630 2024-04-25T11:35:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.358+0000 7f9f32800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2c0713f0 con 0x7f9f2c06dce0 2024-04-25T11:35:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.358+0000 7f9f32800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c0701a0 con 0x7f9f2c15adf0 2024-04-25T11:35:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.358+0000 7f9f2b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9f2c070630 0x7f9f2c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53540/0 (socket says 172.21.15.19:53540) 2024-04-25T11:35:52.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.358+0000 7f9f2b400640 1 -- 172.21.15.19:0/3741211176 learned_addr learned my addr 172.21.15.19:0/3741211176 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.358+0000 7f9f2be00640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 0x7f9f2c160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2aa00640 1 --2- 172.21.15.19:0/3741211176 >> v2:172.21.15.169:3300/0 conn(0x7f9f2c06dce0 0x7f9f2c0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188244402 0 0) 0x7f9f2c071110 con 0x7f9f2c070630 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f10003520 con 0x7f9f2c070630 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4059519457 0 0) 0x7f9f10003520 con 0x7f9f2c070630 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 msgr2=0x7f9f2c160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 0x7f9f2c160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 >> v2:172.21.15.169:3300/0 conn(0x7f9f2c06dce0 msgr2=0x7f9f2c0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 --2- 172.21.15.19:0/3741211176 >> v2:172.21.15.169:3300/0 conn(0x7f9f2c06dce0 0x7f9f2c0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c06fb50 con 0x7f9f2c070630 2024-04-25T11:35:52.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9f1c001670 con 0x7f9f2c070630 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 >> v1:172.21.15.19:6789/0 conn(0x7f9f2c070630 legacy=0x7f9f2c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f100039b0 0x7f9f10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 0x7f9f10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10004d40 0x7f9f10005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c0713f0 con 0x7f9f10004d40 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c0701a0 con 0x7f9f100039b0 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.359+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1000baf0 con 0x7f9f2c15adf0 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.360+0000 7f9f2be00640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10004d40 0x7f9f10005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.360+0000 7f9f2aa00640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 0x7f9f10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.360+0000 7f9f2be00640 1 -- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 msgr2=0x7f9f10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.360+0000 7f9f2be00640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f2c15adf0 0x7f9f10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.360+0000 7f9f2be00640 1 -- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f100039b0 msgr2=0x7f9f10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.360+0000 7f9f2be00640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f100039b0 0x7f9f10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.360+0000 7f9f2be00640 1 -- 172.21.15.19:0/3741211176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f10003520 con 0x7f9f10004d40 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.361+0000 7f9f2be00640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10004d40 0x7f9f10005150 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c073040 tx=0x7f9f10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbf4b9532fabf73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24037070 con 0x7f9f10004d40 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2402e050 con 0x7f9f10004d40 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f2a000640 1 -- 172.21.15.19:0/3741211176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24009c10 con 0x7f9f10004d40 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f32800640 1 -- 172.21.15.19:0/3741211176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10004d40 msgr2=0x7f9f10005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f32800640 1 --2- 172.21.15.19:0/3741211176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f10004d40 0x7f9f10005150 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c073040 tx=0x7f9f10005830 comp rx=0 tx=0).stop 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f2be00640 1 -- 172.21.15.19:0/3741211176 reap_dead start 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f32800640 1 -- 172.21.15.19:0/3741211176 shutdown_connections 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.362+0000 7f9f32800640 1 -- 172.21.15.19:0/3741211176 >> 172.21.15.19:0/3741211176 conn(0x7f9f2c073090 msgr2=0x7f9f2c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.363+0000 7f9f32800640 1 -- 172.21.15.19:0/3741211176 shutdown_connections 2024-04-25T11:35:52.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.363+0000 7f9f32800640 1 -- 172.21.15.19:0/3741211176 wait complete. 2024-04-25T11:35:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.365+0000 7f9f32800640 1 Processor -- start 2024-04-25T11:35:52.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.365+0000 7f9f32800640 1 -- start start 2024-04-25T11:35:52.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f32800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f10004d40 0x7f9f2c14efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f32800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2c13b260 0x7f9f2c137de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f32800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f2c0b2490 0x7f9f2c0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f32800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c0b0e80 con 0x7f9f2c0b2490 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f32800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c0b0d00 con 0x7f9f2c13b260 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f32800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c070c80 con 0x7f9f10004d40 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f2c0b2490 0x7f9f2c0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f2c0b2490 0x7f9f2c0b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51430/0 (socket says 172.21.15.19:51430) 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f2be00640 1 -- 172.21.15.19:0/1566547711 learned_addr learned my addr 172.21.15.19:0/1566547711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.367+0000 7f9f2b400640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f10004d40 0x7f9f2c14efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f2aa00640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2c13b260 0x7f9f2c137de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f2be00640 1 -- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f10004d40 msgr2=0x7f9f2c14efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f2be00640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f10004d40 0x7f9f2c14efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f2be00640 1 -- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2c13b260 msgr2=0x7f9f2c137de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f2be00640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2c13b260 0x7f9f2c137de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f2be00640 1 -- 172.21.15.19:0/1566547711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c0b2de0 con 0x7f9f2c0b2490 2024-04-25T11:35:52.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f2be00640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f2c0b2490 0x7f9f2c0b28a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400e230 tx=0x7f9f24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:52.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f23400640 1 -- 172.21.15.19:0/1566547711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24037040 con 0x7f9f2c0b2490 2024-04-25T11:35:52.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f23400640 1 -- 172.21.15.19:0/1566547711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24042070 con 0x7f9f2c0b2490 2024-04-25T11:35:52.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f23400640 1 -- 172.21.15.19:0/1566547711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24009c10 con 0x7f9f2c0b2490 2024-04-25T11:35:52.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.368+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c150f40 con 0x7f9f2c0b2490 2024-04-25T11:35:52.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.369+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c151500 con 0x7f9f2c0b2490 2024-04-25T11:35:52.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.370+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c150ca0 con 0x7f9f2c0b2490 2024-04-25T11:35:52.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.371+0000 7f9f23400640 1 -- 172.21.15.19:0/1566547711 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9f2403a070 con 0x7f9f2c0b2490 2024-04-25T11:35:52.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.372+0000 7f9f23400640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f080744e0 0x7f9f080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.372+0000 7f9f23400640 1 -- 172.21.15.19:0/1566547711 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f9f240b5410 con 0x7f9f2c0b2490 2024-04-25T11:35:52.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.373+0000 7f9f2b400640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f080744e0 0x7f9f080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.374+0000 7f9f2b400640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f080744e0 0x7f9f080769f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c001eb0 tx=0x7f9f1c0016b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:52.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.377+0000 7f9f23400640 1 -- 172.21.15.19:0/1566547711 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f2c0b2de0 con 0x7f9f2c0b2490 2024-04-25T11:35:52.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.578+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f2c050740 con 0x7f9f2c0b2490 2024-04-25T11:35:52.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.580+0000 7f9f23400640 1 -- 172.21.15.19:0/1566547711 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f9f2407fb30 con 0x7f9f2c0b2490 2024-04-25T11:35:52.568 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:35:52.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f080744e0 msgr2=0x7f9f080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f080744e0 0x7f9f080769f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c001eb0 tx=0x7f9f1c0016b0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f2c0b2490 msgr2=0x7f9f2c0b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f2c0b2490 0x7f9f2c0b28a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400e230 tx=0x7f9f24009750 comp rx=0 tx=0).stop 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 shutdown_connections 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f080744e0 0x7f9f080769f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f2c13b260 0x7f9f2c137de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f2c0b2490 0x7f9f2c0b28a0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 --2- 172.21.15.19:0/1566547711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f10004d40 0x7f9f2c14efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.582+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 >> 172.21.15.19:0/1566547711 conn(0x7f9f2c073090 msgr2=0x7f9f2c152610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.583+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 shutdown_connections 2024-04-25T11:35:52.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.583+0000 7f9f32800640 1 -- 172.21.15.19:0/1566547711 wait complete. 2024-04-25T11:35:52.599 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:35:52.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.898+0000 7fccee200640 1 Processor -- start 2024-04-25T11:35:52.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.898+0000 7fccee200640 1 -- start start 2024-04-25T11:35:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.900+0000 7fccee200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 0x7fcce80b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.900+0000 7fccee200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcce80b7e90 0x7fcce80d44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.900+0000 7fccee200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce80b82a0 con 0x7fcce80d49e0 2024-04-25T11:35:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.900+0000 7fccee200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcce8164330 con 0x7fcce80b7e90 2024-04-25T11:35:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.900+0000 7fccee200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcce80d0b40 con 0x7fcce81647c0 2024-04-25T11:35:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccdfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcce80b7e90 0x7fcce80d44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccdfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcce80b7e90 0x7fcce80d44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51434/0 (socket says 172.21.15.19:51434) 2024-04-25T11:35:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fcced800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcce80d49e0 0x7fcce806d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53544/0 (socket says 172.21.15.19:53544) 2024-04-25T11:35:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccdfe00640 1 -- 172.21.15.19:0/224137179 learned_addr learned my addr 172.21.15.19:0/224137179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccece00640 1 --2- 172.21.15.19:0/224137179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 0x7fcce80b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccdfe00640 1 -- 172.21.15.19:0/224137179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 msgr2=0x7fcce80b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccdfe00640 1 --2- 172.21.15.19:0/224137179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 0x7fcce80b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccdfe00640 1 -- 172.21.15.19:0/224137179 >> v1:172.21.15.19:6789/0 conn(0x7fcce80d49e0 legacy=0x7fcce806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fccdfe00640 1 -- 172.21.15.19:0/224137179 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce81612a0 con 0x7fcce80b7e90 2024-04-25T11:35:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.901+0000 7fcce7e00640 1 -- 172.21.15.19:0/224137179 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 526728251 0 0) 0x7fcce0001110 con 0x7fcce80d49e0 2024-04-25T11:35:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.902+0000 7fccdfe00640 1 --2- 172.21.15.19:0/224137179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce80d44a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009870 tx=0x7fccd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed53d8bc8228e190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.903+0000 7fcce7e00640 1 -- 172.21.15.19:0/224137179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0037070 con 0x7fcce80b7e90 2024-04-25T11:35:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.903+0000 7fcce7e00640 1 -- 172.21.15.19:0/224137179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd0032440 con 0x7fcce80b7e90 2024-04-25T11:35:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.903+0000 7fcce7e00640 1 -- 172.21.15.19:0/224137179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd0032810 con 0x7fcce80b7e90 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.903+0000 7fccee200640 1 -- 172.21.15.19:0/224137179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 msgr2=0x7fcce80d44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.903+0000 7fccee200640 1 --2- 172.21.15.19:0/224137179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce80d44a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009870 tx=0x7fccd002a270 comp rx=0 tx=0).stop 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.904+0000 7fccee200640 1 -- 172.21.15.19:0/224137179 shutdown_connections 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.904+0000 7fccee200640 1 --2- 172.21.15.19:0/224137179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce80d44a0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.904+0000 7fccee200640 1 --2- 172.21.15.19:0/224137179 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 0x7fcce80b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.904+0000 7fccee200640 1 -- 172.21.15.19:0/224137179 >> 172.21.15.19:0/224137179 conn(0x7fcce80d2a10 msgr2=0x7fcce80d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.904+0000 7fccee200640 1 -- 172.21.15.19:0/224137179 shutdown_connections 2024-04-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.904+0000 7fccee200640 1 -- 172.21.15.19:0/224137179 wait complete. 2024-04-25T11:35:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.906+0000 7fccee200640 1 Processor -- start 2024-04-25T11:35:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.906+0000 7fccee200640 1 -- start start 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccee200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce81ea540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccee200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce80d49e0 0x7fcce8156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccee200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 0x7fcce8159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccee200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8163ce0 con 0x7fcce80b7e90 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccee200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8164bd0 con 0x7fcce80d49e0 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccee200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcce8163fc0 con 0x7fcce81647c0 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce81ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce81ea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51450/0 (socket says 172.21.15.19:51450) 2024-04-25T11:35:52.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.908+0000 7fccece00640 1 -- 172.21.15.19:0/1762115363 learned_addr learned my addr 172.21.15.19:0/1762115363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fccece00640 1 -- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 msgr2=0x7fcce8159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fccece00640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 0x7fcce8159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fccece00640 1 -- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce80d49e0 msgr2=0x7fcce8156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fccdfe00640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce80d49e0 0x7fcce8156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fccece00640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce80d49e0 0x7fcce8156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fccece00640 1 -- 172.21.15.19:0/1762115363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce815aaf0 con 0x7fcce80b7e90 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fccece00640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce81ea540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fccd8007e70 tx=0x7fccd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fcce5600640 1 -- 172.21.15.19:0/1762115363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8014620 con 0x7fcce80b7e90 2024-04-25T11:35:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fcce5600640 1 -- 172.21.15.19:0/1762115363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd8019070 con 0x7fcce80b7e90 2024-04-25T11:35:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.909+0000 7fcce5600640 1 -- 172.21.15.19:0/1762115363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8014a00 con 0x7fcce80b7e90 2024-04-25T11:35:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.910+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce815add0 con 0x7fcce80b7e90 2024-04-25T11:35:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.910+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81e9d40 con 0x7fcce80b7e90 2024-04-25T11:35:52.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.911+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce80b8920 con 0x7fcce80b7e90 2024-04-25T11:35:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.912+0000 7fcce5600640 1 -- 172.21.15.19:0/1762115363 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fccd8014ba0 con 0x7fcce80b7e90 2024-04-25T11:35:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.914+0000 7fcce5600640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc40744e0 0x7fccc40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.914+0000 7fcce5600640 1 -- 172.21.15.19:0/1762115363 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fccd8095ba0 con 0x7fcce80b7e90 2024-04-25T11:35:52.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.914+0000 7fccdfe00640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc40744e0 0x7fccc40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:52.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.916+0000 7fccdfe00640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc40744e0 0x7fccc40769f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fccd0034000 tx=0x7fccd002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:52.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:52.919+0000 7fcce5600640 1 -- 172.21.15.19:0/1762115363 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcce80b8920 con 0x7fcce80b7e90 2024-04-25T11:35:53.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.123+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcce8076c60 con 0x7fcce80b7e90 2024-04-25T11:35:53.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.124+0000 7fcce5600640 1 -- 172.21.15.19:0/1762115363 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fccd8060230 con 0x7fcce80b7e90 2024-04-25T11:35:53.112 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.126+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc40744e0 msgr2=0x7fccc40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.126+0000 7fccee200640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc40744e0 0x7fccc40769f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fccd0034000 tx=0x7fccd002e000 comp rx=0 tx=0).stop 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.126+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 msgr2=0x7fcce81ea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.126+0000 7fccee200640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce81ea540 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fccd8007e70 tx=0x7fccd800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 shutdown_connections 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc40744e0 0x7fccc40769f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce80d49e0 0x7fcce8156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce81647c0 0x7fcce8159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 --2- 172.21.15.19:0/1762115363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce80b7e90 0x7fcce81ea540 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 >> 172.21.15.19:0/1762115363 conn(0x7fcce80d2a10 msgr2=0x7fcce80b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:53.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 shutdown_connections 2024-04-25T11:35:53.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.127+0000 7fccee200640 1 -- 172.21.15.19:0/1762115363 wait complete. 2024-04-25T11:35:53.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:35:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.433+0000 7fa2e3e00640 1 Processor -- start 2024-04-25T11:35:53.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.433+0000 7fa2e3e00640 1 -- start start 2024-04-25T11:35:53.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.436+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e41646c0 0x7fa2e40d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.436+0000 7fa2e3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e40b5350 con 0x7fa2e40b7ff0 2024-04-25T11:35:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.436+0000 7fa2e3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e41610f0 con 0x7fa2e40b4f40 2024-04-25T11:35:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.436+0000 7fa2e3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e4163c70 con 0x7fa2e41646c0 2024-04-25T11:35:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.436+0000 7fa2da000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa2e40b7ff0 0x7fa2e40d4190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53550/0 (socket says 172.21.15.19:53550) 2024-04-25T11:35:53.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.436+0000 7fa2da000640 1 -- 172.21.15.19:0/1316158837 learned_addr learned my addr 172.21.15.19:0/1316158837 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.436+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e41646c0 0x7fa2e40d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756146045 0 0) 0x7fa2e40b5350 con 0x7fa2e40b7ff0 2024-04-25T11:35:53.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2c4003520 con 0x7fa2e40b7ff0 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3026222294 0 0) 0x7fa2c4003520 con 0x7fa2e40b7ff0 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e41646c0 msgr2=0x7fa2e40d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e41646c0 0x7fa2e40d48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 >> v1:172.21.15.169:6789/0 conn(0x7fa2e40b4f40 legacy=0x7fa2e40b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e41613d0 con 0x7fa2e40b7ff0 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa2cc001670 con 0x7fa2e40b7ff0 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 >> v1:172.21.15.19:6789/0 conn(0x7fa2e40b7ff0 legacy=0x7fa2e40d4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e41646c0 0x7fa2c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4004d40 0x7fa2c4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4163c70 con 0x7fa2c4004d40 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c400bb70 con 0x7fa2c40039e0 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.437+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c400bd70 con 0x7fa2e41646c0 2024-04-25T11:35:53.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4004d40 0x7fa2c4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e2a00640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e3400640 1 -- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e41646c0 msgr2=0x7fa2c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2e41646c0 0x7fa2c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e3400640 1 -- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c40039e0 msgr2=0x7fa2c4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e3400640 1 -- 172.21.15.19:0/1316158837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c4003520 con 0x7fa2c4004d40 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.438+0000 7fa2e2a00640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2c40039e0 0x7fa2c4003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:53.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.439+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4004d40 0x7fa2c4005210 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc007330 tx=0x7fa2c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e407053f81c376f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.439+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc009be0 con 0x7fa2c4004d40 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.439+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2dc036600 con 0x7fa2c4004d40 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.439+0000 7fa2e2000640 1 -- 172.21.15.19:0/1316158837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0369d0 con 0x7fa2c4004d40 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.440+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1316158837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4004d40 msgr2=0x7fa2c4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.440+0000 7fa2e3e00640 1 --2- 172.21.15.19:0/1316158837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2c4004d40 0x7fa2c4005210 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc007330 tx=0x7fa2c4005830 comp rx=0 tx=0).stop 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.440+0000 7fa2e3400640 1 -- 172.21.15.19:0/1316158837 reap_dead start 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.440+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1316158837 shutdown_connections 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.440+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1316158837 >> 172.21.15.19:0/1316158837 conn(0x7fa2e40d2980 msgr2=0x7fa2e40d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.440+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1316158837 shutdown_connections 2024-04-25T11:35:53.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.440+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1316158837 wait complete. 2024-04-25T11:35:53.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.443+0000 7fa2e3e00640 1 Processor -- start 2024-04-25T11:35:53.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.443+0000 7fa2e3e00640 1 -- start start 2024-04-25T11:35:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c4004d40 0x7fa2e40cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e415f8b0 0x7fa2e414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e40ba0b0 0x7fa2e40ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4164ad0 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4164230 con 0x7fa2e415f8b0 2024-04-25T11:35:53.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e40d0940 con 0x7fa2c4004d40 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e40ba0b0 0x7fa2e40ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e40ba0b0 0x7fa2e40ba4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51480/0 (socket says 172.21.15.19:51480) 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e3400640 1 -- 172.21.15.19:0/1149273126 learned_addr learned my addr 172.21.15.19:0/1149273126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2e2a00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c4004d40 0x7fa2e40cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.445+0000 7fa2da000640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e415f8b0 0x7fa2e414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3400640 1 -- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c4004d40 msgr2=0x7fa2e40cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c4004d40 0x7fa2e40cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3400640 1 -- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e415f8b0 msgr2=0x7fa2e414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e415f8b0 0x7fa2e414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3400640 1 -- 172.21.15.19:0/1149273126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e40baa00 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3400640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e40ba0b0 0x7fa2e40ba4c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc002230 tx=0x7fa2dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2dbe00640 1 -- 172.21.15.19:0/1149273126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc009be0 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2dbe00640 1 -- 172.21.15.19:0/1149273126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2dc036400 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e40cc060 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2dbe00640 1 -- 172.21.15.19:0/1149273126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc038520 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.446+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e414fbb0 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.448+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e40d2fc0 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.449+0000 7fa2dbe00640 1 -- 172.21.15.19:0/1149273126 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa2dc02e620 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.450+0000 7fa2dbe00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2c00744b0 0x7fa2c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.451+0000 7fa2dbe00640 1 -- 172.21.15.19:0/1149273126 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fa2dc0b5c80 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.451+0000 7fa2e2a00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2c00744b0 0x7fa2c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.453+0000 7fa2e2a00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2c00744b0 0x7fa2c00769c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40d3f40 tx=0x7fa2d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:53.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.455+0000 7fa2dbe00640 1 -- 172.21.15.19:0/1149273126 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2e40d2fc0 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.649+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2e4076c60 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.650+0000 7fa2dbe00640 1 -- 172.21.15.19:0/1149273126 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fa2dc048060 con 0x7fa2e40ba0b0 2024-04-25T11:35:53.638 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.652+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2c00744b0 msgr2=0x7fa2c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.652+0000 7fa2e3e00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2c00744b0 0x7fa2c00769c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40d3f40 tx=0x7fa2d4009040 comp rx=0 tx=0).stop 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.652+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e40ba0b0 msgr2=0x7fa2e40ba4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.652+0000 7fa2e3e00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e40ba0b0 0x7fa2e40ba4c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc002230 tx=0x7fa2dc004ab0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 shutdown_connections 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2c00744b0 0x7fa2c00769c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2e415f8b0 0x7fa2e414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e40ba0b0 0x7fa2e40ba4c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 --2- 172.21.15.19:0/1149273126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2c4004d40 0x7fa2e40cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 >> 172.21.15.19:0/1149273126 conn(0x7fa2e40d2980 msgr2=0x7fa2e41604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 shutdown_connections 2024-04-25T11:35:53.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.653+0000 7fa2e3e00640 1 -- 172.21.15.19:0/1149273126 wait complete. 2024-04-25T11:35:53.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:35:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.957+0000 7efd0d800640 1 Processor -- start 2024-04-25T11:35:53.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.957+0000 7efd0d800640 1 -- start start 2024-04-25T11:35:53.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.959+0000 7efd0d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 0x7efd08070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.959+0000 7efd0d800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd081612f0 0x7efd080d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efd0d800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd080709c0 con 0x7efd080d0ed0 2024-04-25T11:35:53.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efd0d800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efd081438e0 con 0x7efd081612f0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efd0d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd0806f3e0 con 0x7efd081434d0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efcfe000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd080d0ed0 0x7efd08160be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38224/0 (socket says 172.21.15.19:38224) 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efcfe000640 1 -- 172.21.15.19:0/2496156015 learned_addr learned my addr 172.21.15.19:0/2496156015 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efd06a00640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 0x7efd08070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efd07400640 1 --2- 172.21.15.19:0/2496156015 >> v2:172.21.15.169:3300/0 conn(0x7efd081612f0 0x7efd080d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916494612 0 0) 0x7efd080709c0 con 0x7efd080d0ed0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.960+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcec003520 con 0x7efd080d0ed0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2820271056 0 0) 0x7efcec003520 con 0x7efd080d0ed0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 msgr2=0x7efd08070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 0x7efd08070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 >> v2:172.21.15.169:3300/0 conn(0x7efd081612f0 msgr2=0x7efd080d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 --2- 172.21.15.19:0/2496156015 >> v2:172.21.15.169:3300/0 conn(0x7efd081612f0 0x7efd080d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0806f6c0 con 0x7efd080d0ed0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efcf0001670 con 0x7efd080d0ed0 2024-04-25T11:35:53.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 >> v1:172.21.15.19:6789/0 conn(0x7efd080d0ed0 legacy=0x7efd08160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcec0039b0 0x7efcec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 0x7efcec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efcec005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd081438e0 con 0x7efcec004d40 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd0806f3e0 con 0x7efcec0039b0 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efcec00b9c0 con 0x7efd081434d0 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.961+0000 7efd07400640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efcec005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.962+0000 7efcfe000640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 0x7efcec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.962+0000 7efd06a00640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcec0039b0 0x7efcec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.962+0000 7efd07400640 1 -- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 msgr2=0x7efcec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.962+0000 7efd07400640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd081434d0 0x7efcec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.962+0000 7efd07400640 1 -- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcec0039b0 msgr2=0x7efcec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.962+0000 7efd07400640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efcec0039b0 0x7efcec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.962+0000 7efd07400640 1 -- 172.21.15.19:0/2496156015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcec003520 con 0x7efcec004d40 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.963+0000 7efd07400640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efcec005150 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efd00009ac0 tx=0x7efcec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e732250b81402b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:53.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.963+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00036070 con 0x7efcec004d40 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.963+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd00031400 con 0x7efcec004d40 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.963+0000 7efd06000640 1 -- 172.21.15.19:0/2496156015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00009690 con 0x7efcec004d40 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.964+0000 7efd0d800640 1 -- 172.21.15.19:0/2496156015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 msgr2=0x7efcec005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.964+0000 7efd0d800640 1 --2- 172.21.15.19:0/2496156015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efcec005150 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efd00009ac0 tx=0x7efcec005830 comp rx=0 tx=0).stop 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.964+0000 7efd07400640 1 -- 172.21.15.19:0/2496156015 reap_dead start 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.964+0000 7efd0d800640 1 -- 172.21.15.19:0/2496156015 shutdown_connections 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.964+0000 7efd0d800640 1 -- 172.21.15.19:0/2496156015 >> 172.21.15.19:0/2496156015 conn(0x7efd080d1ae0 msgr2=0x7efd080d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.964+0000 7efd0d800640 1 -- 172.21.15.19:0/2496156015 shutdown_connections 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.964+0000 7efd0d800640 1 -- 172.21.15.19:0/2496156015 wait complete. 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.967+0000 7efd0d800640 1 Processor -- start 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.968+0000 7efd0d800640 1 -- start start 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd0d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efd0815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd0d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd0806cfa0 0x7efd0806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd0d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd080c7d20 0x7efd080c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd0d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd0806f090 con 0x7efcec004d40 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd0d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd0806fc80 con 0x7efd080c7d20 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd0d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd08070480 con 0x7efd0806cfa0 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd06a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efd0815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd06a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efd0815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56380/0 (socket says 172.21.15.19:56380) 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efd06a00640 1 -- 172.21.15.19:0/3021707002 learned_addr learned my addr 172.21.15.19:0/3021707002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.970+0000 7efcfe000640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd0806cfa0 0x7efd0806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd07400640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd080c7d20 0x7efd080c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd06a00640 1 -- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd0806cfa0 msgr2=0x7efd0806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd06a00640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd0806cfa0 0x7efd0806d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd06a00640 1 -- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd080c7d20 msgr2=0x7efd080c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd06a00640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd080c7d20 0x7efd080c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd06a00640 1 -- 172.21.15.19:0/3021707002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0815aba0 con 0x7efcec004d40 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd07400640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd080c7d20 0x7efd080c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efd06a00640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efd0815a660 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009fb0 tx=0x7efcf80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:53.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.971+0000 7efcff400640 1 -- 172.21.15.19:0/3021707002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf801a020 con 0x7efcec004d40 2024-04-25T11:35:53.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.972+0000 7efcff400640 1 -- 172.21.15.19:0/3021707002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf8016470 con 0x7efcec004d40 2024-04-25T11:35:53.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.972+0000 7efcff400640 1 -- 172.21.15.19:0/3021707002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8016860 con 0x7efcec004d40 2024-04-25T11:35:53.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.972+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd080c6c80 con 0x7efcec004d40 2024-04-25T11:35:53.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.972+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd080c7240 con 0x7efcec004d40 2024-04-25T11:35:53.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.973+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd08050740 con 0x7efcec004d40 2024-04-25T11:35:53.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.974+0000 7efcff400640 1 -- 172.21.15.19:0/3021707002 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efcf8005d10 con 0x7efcec004d40 2024-04-25T11:35:53.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.975+0000 7efcff400640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efce4074390 0x7efce40768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:53.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.975+0000 7efcff400640 1 -- 172.21.15.19:0/3021707002 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7efcf8096210 con 0x7efcec004d40 2024-04-25T11:35:53.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.976+0000 7efcfe000640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efce4074390 0x7efce40768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:53.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.977+0000 7efcfe000640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efce4074390 0x7efce40768a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efd080b5400 tx=0x7efcf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:53.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:53.980+0000 7efcff400640 1 -- 172.21.15.19:0/3021707002 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcf8024030 con 0x7efcec004d40 2024-04-25T11:35:54.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.200+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd0806e100 con 0x7efcec004d40 2024-04-25T11:35:54.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.201+0000 7efcff400640 1 -- 172.21.15.19:0/3021707002 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) ==== 80+0+122 (secure 0 0 0) 0x7efd0815aba0 con 0x7efcec004d40 2024-04-25T11:35:54.189 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:35:54.189 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:35:54.189 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-04-25T11:35:54.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efce4074390 msgr2=0x7efce40768a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efce4074390 0x7efce40768a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efd080b5400 tx=0x7efcf0009040 comp rx=0 tx=0).stop 2024-04-25T11:35:54.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 msgr2=0x7efd0815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efd0815a660 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009fb0 tx=0x7efcf80115a0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 shutdown_connections 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efce4074390 0x7efce40768a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd080c7d20 0x7efd080c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd0806cfa0 0x7efd0806d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 --2- 172.21.15.19:0/3021707002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcec004d40 0x7efd0815a660 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.203+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 >> 172.21.15.19:0/3021707002 conn(0x7efd080d1ae0 msgr2=0x7efd080d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.204+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 shutdown_connections 2024-04-25T11:35:54.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.204+0000 7efd0d800640 1 -- 172.21.15.19:0/3021707002 wait complete. 2024-04-25T11:35:54.218 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:35:54.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:54.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:35:54.309 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:35:54.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:35:54.310 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:35:54.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:35:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.695+0000 7f9649000640 1 Processor -- start 2024-04-25T11:35:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.695+0000 7f9649000640 1 -- start start 2024-04-25T11:35:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9649000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 0x7f96440b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9649000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96440d1fa0 con 0x7f964408cd60 2024-04-25T11:35:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9649000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96440d2280 con 0x7f964406e680 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9649000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96440d1d70 con 0x7f96440b90e0 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9643e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f964408cd60 0x7f964408d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38238/0 (socket says 172.21.15.19:38238) 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9643e00640 1 -- 172.21.15.19:0/2961162733 learned_addr learned my addr 172.21.15.19:0/2961162733 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9642a00640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 0x7f96440b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.697+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 916772045 0 0) 0x7f96440d1fa0 con 0x7f964408cd60 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9624003520 con 0x7f964408cd60 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1763468414 0 0) 0x7f9624003520 con 0x7f964408cd60 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 msgr2=0x7f96440b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 0x7f96440b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 >> v1:172.21.15.169:6789/0 conn(0x7f964406e680 legacy=0x7f96440cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440d2560 con 0x7f964408cd60 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f963c001670 con 0x7f964408cd60 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 >> v1:172.21.15.19:6789/0 conn(0x7f964408cd60 legacy=0x7f964408d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96240039e0 0x7f9624003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9624004800 0x7f9624004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 0x7f962400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9624005590 con 0x7f9624004800 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f962400bf00 con 0x7f96240039e0 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f962400c080 con 0x7f96440b90e0 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.698+0000 7f9642a00640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9624004800 0x7f9624004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9643e00640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 0x7f962400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9643400640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96240039e0 0x7f9624003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9642a00640 1 -- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 msgr2=0x7f962400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9642a00640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 0x7f962400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9642a00640 1 -- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96240039e0 msgr2=0x7f9624003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9642a00640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96240039e0 0x7f9624003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9642a00640 1 -- 172.21.15.19:0/2961162733 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9624003520 con 0x7f9624004800 2024-04-25T11:35:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.699+0000 7f9643e00640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440b90e0 0x7f962400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:54.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.700+0000 7f9642a00640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9624004800 0x7f9624004c10 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f962c007330 tx=0x7f9624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2587ff1070d5d611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.700+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c033070 con 0x7f9624004800 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.700+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962c02f670 con 0x7f9624004800 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.700+0000 7f9642000640 1 -- 172.21.15.19:0/2961162733 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c009be0 con 0x7f9624004800 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.701+0000 7f9649000640 1 -- 172.21.15.19:0/2961162733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9624004800 msgr2=0x7f9624004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.701+0000 7f9649000640 1 --2- 172.21.15.19:0/2961162733 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9624004800 0x7f9624004c10 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f962c007330 tx=0x7f9624005830 comp rx=0 tx=0).stop 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.701+0000 7f9643e00640 1 -- 172.21.15.19:0/2961162733 reap_dead start 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.701+0000 7f9649000640 1 -- 172.21.15.19:0/2961162733 shutdown_connections 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.701+0000 7f9649000640 1 -- 172.21.15.19:0/2961162733 >> 172.21.15.19:0/2961162733 conn(0x7f96440b5460 msgr2=0x7f96440b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.701+0000 7f9649000640 1 -- 172.21.15.19:0/2961162733 shutdown_connections 2024-04-25T11:35:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.701+0000 7f9649000640 1 -- 172.21.15.19:0/2961162733 wait complete. 2024-04-25T11:35:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.704+0000 7f9649000640 1 Processor -- start 2024-04-25T11:35:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.704+0000 7f9649000640 1 -- start start 2024-04-25T11:35:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.705+0000 7f9649000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9624004800 0x7f964414deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.705+0000 7f9649000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96440c9570 0x7f96440c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.705+0000 7f9649000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440c73d0 0x7f96440c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.705+0000 7f9649000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96440b94f0 con 0x7f96440c9570 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.705+0000 7f9649000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f96440ce6d0 con 0x7f9624004800 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.705+0000 7f9649000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f964406ec40 con 0x7f96440c73d0 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96440c9570 0x7f96440c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96440c9570 0x7f96440c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56394/0 (socket says 172.21.15.19:56394) 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 -- 172.21.15.19:0/2972840930 learned_addr learned my addr 172.21.15.19:0/2972840930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9643e00640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440c73d0 0x7f96440c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 -- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440c73d0 msgr2=0x7f96440c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9643400640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9624004800 0x7f964414deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440c73d0 0x7f96440c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 -- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9624004800 msgr2=0x7f964414deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9624004800 0x7f964414deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.706+0000 7f9642a00640 1 -- 172.21.15.19:0/2972840930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440c7d20 con 0x7f96440c9570 2024-04-25T11:35:54.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.707+0000 7f9642a00640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96440c9570 0x7f96440c9980 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f962c007450 tx=0x7f962c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.707+0000 7f963b400640 1 -- 172.21.15.19:0/2972840930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c033040 con 0x7f96440c9570 2024-04-25T11:35:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.707+0000 7f963b400640 1 -- 172.21.15.19:0/2972840930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962c02f470 con 0x7f96440c9570 2024-04-25T11:35:54.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.707+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96440ca2a0 con 0x7f96440c9570 2024-04-25T11:35:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.707+0000 7f963b400640 1 -- 172.21.15.19:0/2972840930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c009be0 con 0x7f96440c9570 2024-04-25T11:35:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.707+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9644161b60 con 0x7f96440c9570 2024-04-25T11:35:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.708+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96440ca010 con 0x7f96440c9570 2024-04-25T11:35:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.710+0000 7f963b400640 1 -- 172.21.15.19:0/2972840930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f962c005d10 con 0x7f96440c9570 2024-04-25T11:35:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.711+0000 7f963b400640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96200744e0 0x7f96200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.711+0000 7f963b400640 1 -- 172.21.15.19:0/2972840930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f962c0b64f0 con 0x7f96440c9570 2024-04-25T11:35:54.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.712+0000 7f9643400640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96200744e0 0x7f96200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:54.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.713+0000 7f9643400640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96200744e0 0x7f96200769f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f9634009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:54.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.716+0000 7f963b400640 1 -- 172.21.15.19:0/2972840930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96440c7d20 con 0x7f96440c9570 2024-04-25T11:35:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.965+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9644050740 con 0x7f96440c9570 2024-04-25T11:35:54.954 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:35:54.966+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:35:54.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.970+0000 7f963b400640 1 -- 172.21.15.19:0/2972840930 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) ==== 161+0+0 (secure 0 0 0) 0x7f962c041e90 con 0x7f96440c9570 2024-04-25T11:35:54.958 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96200744e0 msgr2=0x7f96200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96200744e0 0x7f96200769f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f9634009040 comp rx=0 tx=0).stop 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96440c9570 msgr2=0x7f96440c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96440c9570 0x7f96440c9980 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f962c007450 tx=0x7f962c004ab0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 shutdown_connections 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96200744e0 0x7f96200769f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96440c73d0 0x7f96440c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96440c9570 0x7f96440c9980 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 --2- 172.21.15.19:0/2972840930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9624004800 0x7f964414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 >> 172.21.15.19:0/2972840930 conn(0x7f96440b5460 msgr2=0x7f96440729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:54.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 shutdown_connections 2024-04-25T11:35:54.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:54.973+0000 7f9649000640 1 -- 172.21.15.19:0/2972840930 wait complete. 2024-04-25T11:35:54.982 INFO:tasks.ceph.mds.a.smithi019.stderr: -795> 2024-04-25T11:35:45.560+0000 7f0ea5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:35:54.986 INFO:tasks.ceph.mds.a.smithi019.stderr: -226> 2024-04-25T11:35:51.407+0000 7f0ea0e00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-04-25T11:35:54.991 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:35:55.118 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:35:55.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.258+0000 7f19b6c00640 1 Processor -- start 2024-04-25T11:35:55.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.258+0000 7f19b6c00640 1 -- start start 2024-04-25T11:35:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.260+0000 7f19b6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b00d0ed0 0x7f19b00d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.260+0000 7f19b6c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19b0160be0 0x7f19b0160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.260+0000 7f19b6c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19b0161530 con 0x7f19b01434d0 2024-04-25T11:35:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.260+0000 7f19b6c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f19b01438e0 con 0x7f19b0160be0 2024-04-25T11:35:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.261+0000 7f19b6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19b006f3e0 con 0x7f19b00d0ed0 2024-04-25T11:35:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.261+0000 7f19afe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f19b01434d0 0x7f19b0070480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38262/0 (socket says 172.21.15.19:38262) 2024-04-25T11:35:55.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.261+0000 7f19afe00640 1 -- 172.21.15.19:0/2521109503 learned_addr learned my addr 172.21.15.19:0/2521109503 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.261+0000 7f19af400640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b00d0ed0 0x7f19b00d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.261+0000 7f19b4c00640 1 --2- 172.21.15.19:0/2521109503 >> v2:172.21.15.169:3300/0 conn(0x7f19b0160be0 0x7f19b0160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.261+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427123953 0 0) 0x7f19a0001490 con 0x7f19b01434d0 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1994005730 con 0x7f19b01434d0 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 179492881 0 0) 0x7f1994005730 con 0x7f19b01434d0 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b00d0ed0 msgr2=0x7f19b00d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b00d0ed0 0x7f19b00d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 >> v2:172.21.15.169:3300/0 conn(0x7f19b0160be0 msgr2=0x7f19b0160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 --2- 172.21.15.19:0/2521109503 >> v2:172.21.15.169:3300/0 conn(0x7f19b0160be0 0x7f19b0160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b006f6c0 con 0x7f19b01434d0 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f19a0001bc0 con 0x7f19b01434d0 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 >> v1:172.21.15.19:6789/0 conn(0x7f19b01434d0 legacy=0x7f19b0070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1994005bc0 0x7f1994006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f1994006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b00d0ed0 0x7f1994007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19b01438e0 con 0x7f1994006a10 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19b006f3e0 con 0x7f1994005bc0 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.262+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f199400dc80 con 0x7f19b00d0ed0 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.263+0000 7f19af400640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f1994006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.263+0000 7f19af400640 1 -- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b00d0ed0 msgr2=0x7f1994007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.263+0000 7f19af400640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b00d0ed0 0x7f1994007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.263+0000 7f19af400640 1 -- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1994005bc0 msgr2=0x7f1994006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.263+0000 7f19af400640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1994005bc0 0x7f1994006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.263+0000 7f19af400640 1 -- 172.21.15.19:0/2521109503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1994005730 con 0x7f1994006a10 2024-04-25T11:35:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.264+0000 7f19af400640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f1994006e20 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f199c009a90 tx=0x7f1994007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a861311a69d68e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.264+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c036070 con 0x7f1994006a10 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f199c031400 con 0x7f1994006a10 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19aea00640 1 -- 172.21.15.19:0/2521109503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c009690 con 0x7f1994006a10 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19b6c00640 1 -- 172.21.15.19:0/2521109503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 msgr2=0x7f1994006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19b6c00640 1 --2- 172.21.15.19:0/2521109503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f1994006e20 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f199c009a90 tx=0x7f1994007a40 comp rx=0 tx=0).stop 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19b4c00640 1 -- 172.21.15.19:0/2521109503 reap_dead start 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19b6c00640 1 -- 172.21.15.19:0/2521109503 shutdown_connections 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19b6c00640 1 -- 172.21.15.19:0/2521109503 >> 172.21.15.19:0/2521109503 conn(0x7f19b00d1ae0 msgr2=0x7f19b00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.265+0000 7f19b6c00640 1 -- 172.21.15.19:0/2521109503 shutdown_connections 2024-04-25T11:35:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.266+0000 7f19b6c00640 1 -- 172.21.15.19:0/2521109503 wait complete. 2024-04-25T11:35:55.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.268+0000 7f19b6c00640 1 Processor -- start 2024-04-25T11:35:55.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.268+0000 7f19b6c00640 1 -- start start 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19b6c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f19b013f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19b6c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b01fa790 0x7f19b01f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19b6c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19b0153150 0x7f19b01f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19b6c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19b006f090 con 0x7f1994006a10 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19b6c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19b006fc80 con 0x7f19b0153150 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19b6c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f19b00d12e0 con 0x7f19b01fa790 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f19b013f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f19b013f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56438/0 (socket says 172.21.15.19:56438) 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.270+0000 7f19afe00640 1 -- 172.21.15.19:0/4228814624 learned_addr learned my addr 172.21.15.19:0/4228814624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19afe00640 1 -- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b01fa790 msgr2=0x7f19b01f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19afe00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b01fa790 0x7f19b01f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19afe00640 1 -- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19b0153150 msgr2=0x7f19b01f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19afe00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19b0153150 0x7f19b01f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19afe00640 1 -- 172.21.15.19:0/4228814624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b0143b90 con 0x7f1994006a10 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19afe00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f19b013f090 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f19a0007fd0 tx=0x7f19a000ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19a7e00640 1 -- 172.21.15.19:0/4228814624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0018070 con 0x7f1994006a10 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19a7e00640 1 -- 172.21.15.19:0/4228814624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a000a700 con 0x7f1994006a10 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19a7e00640 1 -- 172.21.15.19:0/4228814624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000aab0 con 0x7f1994006a10 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b0143e70 con 0x7f1994006a10 2024-04-25T11:35:55.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.271+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b013eb70 con 0x7f1994006a10 2024-04-25T11:35:55.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.273+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b015e680 con 0x7f1994006a10 2024-04-25T11:35:55.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.274+0000 7f19a7e00640 1 -- 172.21.15.19:0/4228814624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f19a000ac50 con 0x7f1994006a10 2024-04-25T11:35:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.275+0000 7f19a7e00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19900744e0 0x7f19900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.275+0000 7f19a7e00640 1 -- 172.21.15.19:0/4228814624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f19a0094c30 con 0x7f1994006a10 2024-04-25T11:35:55.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.276+0000 7f19af400640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19900744e0 0x7f19900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.277+0000 7f19af400640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19900744e0 0x7f19900769f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f199c033000 tx=0x7f199c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.280+0000 7f19a7e00640 1 -- 172.21.15.19:0/4228814624 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19b015e680 con 0x7f1994006a10 2024-04-25T11:35:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.566+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f19b0050740 con 0x7f1994006a10 2024-04-25T11:35:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.974+0000 7f19a7e00640 1 -- 172.21.15.19:0/4228814624 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v101) ==== 107+0+0 (secure 0 0 0) 0x7f19b0161530 con 0x7f1994006a10 2024-04-25T11:35:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:35:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19900744e0 msgr2=0x7f19900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19900744e0 0x7f19900769f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f199c033000 tx=0x7f199c02e000 comp rx=0 tx=0).stop 2024-04-25T11:35:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 msgr2=0x7f19b013f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f19b013f090 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f19a0007fd0 tx=0x7f19a000ce40 comp rx=0 tx=0).stop 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 shutdown_connections 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19900744e0 0x7f19900769f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19b0153150 0x7f19b01f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f19b01fa790 0x7f19b01f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 --2- 172.21.15.19:0/4228814624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994006a10 0x7f19b013f090 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 >> 172.21.15.19:0/4228814624 conn(0x7f19b00d1ae0 msgr2=0x7f19b01411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 shutdown_connections 2024-04-25T11:35:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:55.981+0000 7f19b6c00640 1 -- 172.21.15.19:0/4228814624 wait complete. 2024-04-25T11:35:56.002 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:35:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.201+0000 7f566621af40 1 Processor -- start 2024-04-25T11:35:56.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.201+0000 7f566621af40 1 -- start start 2024-04-25T11:35:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.202+0000 7f566621af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d664570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.202+0000 7f566621af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 0x56292d5de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.202+0000 7f566621af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 0x56292d65c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.202+0000 7f566621af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56292d537d20 con 0x56292d65fee0 2024-04-25T11:35:56.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.202+0000 7f566621af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56292d53bd60 con 0x56292d5ddfb0 2024-04-25T11:35:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.202+0000 7f566621af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56292d5da110 con 0x56292d65bd60 2024-04-25T11:35:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.202+0000 7f565e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d664570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d664570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56454/0 (socket says 172.21.15.19:56454) 2024-04-25T11:35:56.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 -- 172.21.15.19:0/4074221583 learned_addr learned my addr 172.21.15.19:0/4074221583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 -- 172.21.15.19:0/4074221583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 msgr2=0x56292d65c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 --2- 172.21.15.19:0/4074221583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 0x56292d65c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 -- 172.21.15.19:0/4074221583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 msgr2=0x56292d5de3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 --2- 172.21.15.19:0/4074221583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 0x56292d5de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 -- 172.21.15.19:0/4074221583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292d5dcc20 con 0x56292d65fee0 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.203+0000 7f565e800640 1 --2- 172.21.15.19:0/4074221583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d664570 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f565000c6b0 tx=0x7f565000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92d6bc2454878dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f565ca00640 1 -- 172.21.15.19:0/4074221583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650012070 con 0x56292d65fee0 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f565ca00640 1 -- 172.21.15.19:0/4074221583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565000a8c0 con 0x56292d65fee0 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f565ca00640 1 -- 172.21.15.19:0/4074221583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565000ac90 con 0x56292d65fee0 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 -- 172.21.15.19:0/4074221583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 msgr2=0x56292d664570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 --2- 172.21.15.19:0/4074221583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d664570 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f565000c6b0 tx=0x7f565000cb70 comp rx=0 tx=0).stop 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 -- 172.21.15.19:0/4074221583 shutdown_connections 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 --2- 172.21.15.19:0/4074221583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 0x56292d5de3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 --2- 172.21.15.19:0/4074221583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 0x56292d65c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 --2- 172.21.15.19:0/4074221583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d664570 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 -- 172.21.15.19:0/4074221583 >> 172.21.15.19:0/4074221583 conn(0x56292d4b3ac0 msgr2=0x56292d5c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.204+0000 7f566621af40 1 -- 172.21.15.19:0/4074221583 shutdown_connections 2024-04-25T11:35:56.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.205+0000 7f566621af40 1 -- 172.21.15.19:0/4074221583 wait complete. 2024-04-25T11:35:56.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.206+0000 7f566621af40 1 Processor -- start 2024-04-25T11:35:56.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.206+0000 7f566621af40 1 -- start start 2024-04-25T11:35:56.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.207+0000 7f566621af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 0x56292d662ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.207+0000 7f566621af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 0x56292d666740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.207+0000 7f566621af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d666c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.207+0000 7f566621af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56292d5bfb00 con 0x56292d65fee0 2024-04-25T11:35:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.207+0000 7f566621af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56292d5c36d0 con 0x56292d5ddfb0 2024-04-25T11:35:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.207+0000 7f566621af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56292d5db700 con 0x56292d65bd60 2024-04-25T11:35:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d666c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d666c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56458/0 (socket says 172.21.15.19:56458) 2024-04-25T11:35:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 learned_addr learned my addr 172.21.15.19:0/1920391593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565de00640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 0x56292d666740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 0x56292d662ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 msgr2=0x56292d666740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56292d65bd60 0x56292d666740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 msgr2=0x56292d662ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 0x56292d662ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292d649000 con 0x56292d65fee0 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.208+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56292d5ddfb0 0x56292d662ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.209+0000 7f565f200640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d666c80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x56292d5d8940 tx=0x7f565800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.209+0000 7f5657400640 1 -- 172.21.15.19:0/1920391593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800ab10 con 0x56292d65fee0 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.209+0000 7f5657400640 1 -- 172.21.15.19:0/1920391593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5658015020 con 0x56292d65fee0 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.209+0000 7f5657400640 1 -- 172.21.15.19:0/1920391593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565800acf0 con 0x56292d65fee0 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.209+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56292d649270 con 0x56292d65fee0 2024-04-25T11:35:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.209+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292d649b30 con 0x56292d65fee0 2024-04-25T11:35:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.211+0000 7f5657400640 1 -- 172.21.15.19:0/1920391593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f565800e040 con 0x56292d65fee0 2024-04-25T11:35:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.212+0000 7f5657400640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5640072e90 0x7f56400753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.212+0000 7f5657400640 1 -- 172.21.15.19:0/1920391593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f5658093af0 con 0x56292d65fee0 2024-04-25T11:35:56.200 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:35:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.213+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56292d64a560 0x56292d64a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.213+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d64aeb0 con 0x56292d64a560 2024-04-25T11:35:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.213+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5640072e90 0x7f56400753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.214+0000 7f565de00640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56292d64a560 0x56292d64a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.214+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5640072e90 0x7f56400753a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f565000c6b0 tx=0x7f565001a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.215+0000 7f565de00640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56292d64a560 0x56292d64a970 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.218+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v94'43 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.218+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d70baa0 con 0x56292d64a560 2024-04-25T11:35:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.221+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v94'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.221+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70be40 con 0x56292d64a560 2024-04-25T11:35:56.212 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:35:56.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.224+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v94'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.225+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70d1f0 con 0x56292d64a560 2024-04-25T11:35:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.228+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v94'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.228+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70d590 con 0x56292d64a560 2024-04-25T11:35:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.232+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v94'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.232+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c1e0 con 0x56292d64a560 2024-04-25T11:35:56.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.235+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v94'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.235+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c580 con 0x56292d64a560 2024-04-25T11:35:56.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.238+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v94'49 uv22 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.238+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d70c970 con 0x56292d64a560 2024-04-25T11:35:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.241+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v94'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.241+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c970 con 0x56292d64a560 2024-04-25T11:35:56.232 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:35:56.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.245+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v94'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.245+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.248+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v94'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.249+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.252+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v94'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.252+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.256+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v94'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.256+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.247 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:35:56.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.259+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v94'55 uv55 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.259+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.263+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v94'56 uv56 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.263+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.267+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v94'57 uv57 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.267+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.271+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v94'58 uv58 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.272+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.275+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v94'59 uv59 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.276+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.280+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 600.00000000 [create] v94'60 uv35 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.280+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.285+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 600.00000000 [delete] v94'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.285+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.276 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:35:56.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.289+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v94'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.289+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.293+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v94'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.293+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.297+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v94'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.298+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.302+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v94'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.302+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.306+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 603.00000000 [create] v94'66 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.306+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.310+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 603.00000000 [delete] v94'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.310+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.302 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:35:56.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.314+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v94'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.314+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.318+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v94'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.319+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.323+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v94'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.323+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d64a560 2024-04-25T11:35:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.327+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v94'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.327+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56292d5c1330 0x56292d70fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.328+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:30 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70c920 con 0x56292d5c1330 2024-04-25T11:35:56.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.328+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56292d5c1330 0x56292d70fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.330+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56292d5c1330 0x56292d70fe50 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.334+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(30 1.00000000.inode [create] v94'45 uv24 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.334+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:31 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d710390 con 0x56292d5c1330 2024-04-25T11:35:56.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.339+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(31 1.00000000.inode [delete] v94'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.339+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d710730 con 0x56292d5c1330 2024-04-25T11:35:56.331 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:35:56.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.343+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(32 1.00000000.inode [create] v94'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.343+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d710ad0 con 0x56292d5c1330 2024-04-25T11:35:56.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.348+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v94'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.348+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d710e70 con 0x56292d5c1330 2024-04-25T11:35:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.352+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v94'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.352+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.356+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(35 200.00000000 [create] v94'50 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.356+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.360+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(36 200.00000000 [delete] v94'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.360+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.352 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:35:56.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.364+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(37 200.00000000 [create] v94'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.364+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.369+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v94'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.369+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.373+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v94'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.373+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.377+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(40 200.00000001 [create] v94'55 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.377+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.381+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(41 200.00000001 [delete] v94'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.381+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.373 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:35:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.385+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(42 200.00000001 [create] v94'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.386+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d715900 con 0x56292d5c1330 2024-04-25T11:35:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.390+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(43 200.00000001 [write 0~18117] v94'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.390+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d715900 con 0x56292d5c1330 2024-04-25T11:35:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.394+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v94'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.394+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.398+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(45 606.00000000 [create] v94'60 uv40 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.398+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.403+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(46 606.00000000 [delete] v94'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.403+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.394 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:35:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.406+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(47 606.00000000 [create] v94'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.407+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d712220 con 0x56292d5c1330 2024-04-25T11:35:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.415+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v94'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.415+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d712220 con 0x56292d5c1330 2024-04-25T11:35:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.419+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v94'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.419+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d711210 con 0x56292d5c1330 2024-04-25T11:35:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.424+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v94'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d5c1330 2024-04-25T11:35:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.424+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56292d711890 0x56292d711ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.424+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d7145f0 con 0x56292d711890 2024-04-25T11:35:56.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.425+0000 7f565de00640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56292d711890 0x56292d711ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.427+0000 7f565de00640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56292d711890 0x56292d711ca0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.431+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(51 607.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.431+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d7145f0 con 0x56292d711890 2024-04-25T11:35:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.435+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(52 607.00000000 [delete] v94'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.436+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d7145f0 con 0x56292d711890 2024-04-25T11:35:56.427 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:35:56.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.440+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(53 607.00000000 [create] v94'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.440+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d7156a0 con 0x56292d711890 2024-04-25T11:35:56.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.444+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.445+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d715660 con 0x56292d711890 2024-04-25T11:35:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.449+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.449+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d715660 con 0x56292d711890 2024-04-25T11:35:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.454+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.454+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d715640 con 0x56292d64a560 2024-04-25T11:35:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.459+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(57 mds0_openfiles.0 [create] v94'11 uv10 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.459+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d7145f0 con 0x56292d64a560 2024-04-25T11:35:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.464+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v94'12 uv12 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.464+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d7145f0 con 0x56292d64a560 2024-04-25T11:35:56.456 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:35:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.468+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v94'13 uv13 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.468+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d7145f0 con 0x56292d64a560 2024-04-25T11:35:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.473+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v94'14 uv14 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.473+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56292d714ec0 0x56292d717a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.473+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d7152d0 con 0x56292d714ec0 2024-04-25T11:35:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.474+0000 7f565f200640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56292d714ec0 0x56292d717a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.475+0000 7f565f200640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56292d714ec0 0x56292d717a50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.479+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(61 608.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.479+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d717f90 con 0x56292d714ec0 2024-04-25T11:35:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.483+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(62 608.00000000 [delete] v94'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.483+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d718330 con 0x56292d714ec0 2024-04-25T11:35:56.475 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:35:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.487+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(63 608.00000000 [create] v94'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.487+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d70ce00 con 0x56292d714ec0 2024-04-25T11:35:56.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.491+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.491+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d719c70 con 0x56292d714ec0 2024-04-25T11:35:56.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.495+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.496+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d718b80 con 0x56292d714ec0 2024-04-25T11:35:56.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.499+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.499+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56292d719640 0x56292d71c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.499+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71c680 con 0x56292d719640 2024-04-25T11:35:56.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.500+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56292d719640 0x56292d71c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.502+0000 7f565e800640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56292d719640 0x56292d71c0b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.506+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(67 604.00000000 [create] v94'15 uv9 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.507+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d71c740 con 0x56292d719640 2024-04-25T11:35:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.511+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(68 604.00000000 [delete] v94'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.511+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71cb60 con 0x56292d719640 2024-04-25T11:35:56.503 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:35:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.515+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(69 604.00000000 [create] v94'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.515+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71db70 con 0x56292d719640 2024-04-25T11:35:56.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.519+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v94'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.519+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71db70 con 0x56292d719640 2024-04-25T11:35:56.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.523+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v94'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.523+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71cb60 con 0x56292d719640 2024-04-25T11:35:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.528+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v94'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.528+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71cb60 con 0x56292d719640 2024-04-25T11:35:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.532+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(73 500.00000000 [create] v94'21 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.532+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d71cb60 con 0x56292d719640 2024-04-25T11:35:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.536+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(74 500.00000000 [delete] v94'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.537+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71cb60 con 0x56292d719640 2024-04-25T11:35:56.529 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:35:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.541+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(75 500.00000000 [create] v94'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.541+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71cb60 con 0x56292d719640 2024-04-25T11:35:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.545+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v94'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.546+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71cb60 con 0x56292d719640 2024-04-25T11:35:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.549+0000 7f565e800640 1 -- 172.21.15.19:0/1920391593 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v94'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5650018460 con 0x56292d719640 2024-04-25T11:35:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.550+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56292d71d1e0 0x56292d71f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.550+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d71fc10 con 0x56292d71d1e0 2024-04-25T11:35:56.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.550+0000 7f565de00640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56292d71d1e0 0x56292d71f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.552+0000 7f565de00640 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56292d71d1e0 0x56292d71f600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.556+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(78 mds_snaptable [create] v94'7 uv6 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.556+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d71fc10 con 0x56292d71d1e0 2024-04-25T11:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.561+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(79 mds_snaptable [delete] v94'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.561+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d71d1e0 2024-04-25T11:35:56.553 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:35:56.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.565+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(80 mds_snaptable [create] v94'9 uv9 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.565+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d71d1e0 2024-04-25T11:35:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.570+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v94'10 uv10 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.570+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d71d1e0 2024-04-25T11:35:56.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.575+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v94'11 uv11 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.575+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d64a560 2024-04-25T11:35:56.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.579+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(83 605.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.579+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d64a560 2024-04-25T11:35:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.583+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(84 605.00000000 [delete] v94'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.583+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d64a560 2024-04-25T11:35:56.574 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:35:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.587+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v94'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.587+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721070 con 0x56292d64a560 2024-04-25T11:35:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.590+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.591+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721070 con 0x56292d64a560 2024-04-25T11:35:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.594+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.594+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d64a560 2024-04-25T11:35:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.598+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d64a560 2024-04-25T11:35:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.599+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d711890 2024-04-25T11:35:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.603+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(89 mds0_inotable [create] v94'14 uv8 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.603+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d711890 2024-04-25T11:35:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.607+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(90 mds0_inotable [delete] v94'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.607+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720060 con 0x56292d711890 2024-04-25T11:35:56.599 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:35:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.612+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(91 mds0_inotable [create] v94'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.612+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720120 con 0x56292d711890 2024-04-25T11:35:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.616+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v94'17 uv17 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.616+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720120 con 0x56292d711890 2024-04-25T11:35:56.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.622+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v94'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.622+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720120 con 0x56292d711890 2024-04-25T11:35:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.626+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(94 100.00000000 [create] v94'19 uv13 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.626+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d720120 con 0x56292d711890 2024-04-25T11:35:56.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.630+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(95 100.00000000 [delete] v94'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.630+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720100 con 0x56292d711890 2024-04-25T11:35:56.623 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:35:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.635+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(96 100.00000000 [create] v94'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.635+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720100 con 0x56292d711890 2024-04-25T11:35:56.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.639+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v94'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.639+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d722f60 con 0x56292d711890 2024-04-25T11:35:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.644+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v94'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d711890 2024-04-25T11:35:56.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.645+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d722f20 con 0x56292d714ec0 2024-04-25T11:35:56.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.649+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v94'9 uv7 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.649+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d720580 con 0x56292d714ec0 2024-04-25T11:35:56.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.654+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v94'10 uv10 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.654+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d714ec0 2024-04-25T11:35:56.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.658+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v94'11 uv11 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.646 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:35:56.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.658+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d714ec0 2024-04-25T11:35:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.663+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v94'12 uv12 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.663+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d714ec0 2024-04-25T11:35:56.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.667+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v94'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.667+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d714ec0 2024-04-25T11:35:56.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.672+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v94'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.672+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d714ec0 2024-04-25T11:35:56.664 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:35:56.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.676+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v94'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.676+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721930 con 0x56292d714ec0 2024-04-25T11:35:56.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.680+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v94'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.681+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d714ec0 2024-04-25T11:35:56.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.685+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v94'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.685+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d714ec0 2024-04-25T11:35:56.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.689+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v94'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5658021070 con 0x56292d714ec0 2024-04-25T11:35:56.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.690+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.693+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v94'7 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.693+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.698+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v94'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.698+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.689 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:35:56.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.702+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v94'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.702+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.706+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v94'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.706+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.710+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v94'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.710+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.714+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v94'20 uv11 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.714+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.720+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v94'21 uv21 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.720+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.724+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v94'22 uv22 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.712 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:35:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.724+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.729+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v94'23 uv23 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.729+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.733+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v94'24 uv24 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.734+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.738+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v94'25 uv18 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.738+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.742+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v94'26 uv26 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.743+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d721d30 con 0x56292d71d1e0 2024-04-25T11:35:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.747+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v94'27 uv27 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.735 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:35:56.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.747+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d71d1e0 2024-04-25T11:35:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.751+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v94'28 uv28 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.751+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d71d1e0 2024-04-25T11:35:56.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.756+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v94'29 uv29 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.756+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d720920 con 0x56292d71d1e0 2024-04-25T11:35:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.760+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v94'30 uv30 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.760+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x56292d7211d0 con 0x56292d71d1e0 2024-04-25T11:35:56.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.764+0000 7f565de00640 1 -- 172.21.15.19:0/1920391593 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v94'31 uv31 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f564c005040 con 0x56292d71d1e0 2024-04-25T11:35:56.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.764+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56292d5c1330 msgr2=0x56292d70fe50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56292d5c1330 0x56292d70fe50 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56292d719640 msgr2=0x56292d71c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56292d719640 0x56292d71c0b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56292d71d1e0 msgr2=0x56292d71f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56292d71d1e0 0x56292d71f600 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56292d711890 msgr2=0x56292d711ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56292d711890 0x56292d711ca0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56292d64a560 msgr2=0x56292d64a970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56292d64a560 0x56292d64a970 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 reap_dead start 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56292d714ec0 msgr2=0x56292d717a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56292d714ec0 0x56292d717a50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f565f200640 1 -- 172.21.15.19:0/1920391593 reap_dead start 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5640072e90 msgr2=0x7f56400753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5640072e90 0x7f56400753a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f565000c6b0 tx=0x7f565001a040 comp rx=0 tx=0).stop 2024-04-25T11:35:56.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 msgr2=0x56292d666c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.765+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d666c80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x56292d5d8940 tx=0x7f565800cf30 comp rx=0 tx=0).stop 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.766+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 shutdown_connections 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.766+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56292d714ec0 0x56292d717a50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.766+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5640072e90 0x7f56400753a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.766+0000 7f566621af40 1 --2- 172.21.15.19:0/1920391593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56292d65fee0 0x56292d666c80 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.766+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 >> 172.21.15.19:0/1920391593 conn(0x56292d4b3ac0 msgr2=0x56292d5c22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.766+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 shutdown_connections 2024-04-25T11:35:56.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.766+0000 7f566621af40 1 -- 172.21.15.19:0/1920391593 wait complete. 2024-04-25T11:35:56.784 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2024-04-25T11:35:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.990+0000 7f3127120f40 1 Processor -- start 2024-04-25T11:35:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.990+0000 7f3127120f40 1 -- start start 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f3127120f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e1e0 0x55fef29fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f3127120f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef292e5f0 0x55fef2a034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f3127120f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef2a07fa0 0x55fef2a039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f3127120f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fef2976e70 con 0x55fef292e1e0 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f3127120f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fef28d4d20 con 0x55fef292e5f0 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f3127120f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fef29773f0 con 0x55fef2a07fa0 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f311fa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e1e0 0x55fef29fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f311fa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e1e0 0x55fef29fde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56464/0 (socket says 172.21.15.19:56464) 2024-04-25T11:35:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f311fa00640 1 -- 172.21.15.19:0/3184923509 learned_addr learned my addr 172.21.15.19:0/3184923509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f3120400640 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef2a07fa0 0x55fef2a039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f311fa00640 1 -- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef2a07fa0 msgr2=0x55fef2a039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.992+0000 7f311f000640 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef292e5f0 0x55fef2a034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311fa00640 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef2a07fa0 0x55fef2a039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311fa00640 1 -- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef292e5f0 msgr2=0x55fef2a034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311fa00640 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef292e5f0 0x55fef2a034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311fa00640 1 -- 172.21.15.19:0/3184923509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef2960190 con 0x55fef292e1e0 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311f000640 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef292e5f0 0x55fef2a034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311fa00640 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e1e0 0x55fef29fde10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f311000b600 tx=0x7f311000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a268b9ba8d65d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311dc00640 1 -- 172.21.15.19:0/3184923509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110014030 con 0x55fef292e1e0 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311dc00640 1 -- 172.21.15.19:0/3184923509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3110010620 con 0x55fef292e1e0 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.993+0000 7f311dc00640 1 -- 172.21.15.19:0/3184923509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100109d0 con 0x55fef292e1e0 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 -- 172.21.15.19:0/3184923509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e1e0 msgr2=0x55fef29fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e1e0 0x55fef29fde10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f311000b600 tx=0x7f311000bac0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 -- 172.21.15.19:0/3184923509 shutdown_connections 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef292e5f0 0x55fef2a034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef2a07fa0 0x55fef2a039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 --2- 172.21.15.19:0/3184923509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e1e0 0x55fef29fde10 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 -- 172.21.15.19:0/3184923509 >> 172.21.15.19:0/3184923509 conn(0x55fef2850ac0 msgr2=0x55fef2a02d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 -- 172.21.15.19:0/3184923509 shutdown_connections 2024-04-25T11:35:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.994+0000 7f3127120f40 1 -- 172.21.15.19:0/3184923509 wait complete. 2024-04-25T11:35:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.995+0000 7f3127120f40 1 Processor -- start 2024-04-25T11:35:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.995+0000 7f3127120f40 1 -- start start 2024-04-25T11:35:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f3127120f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef292e1e0 0x55fef29f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f3127120f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e5f0 0x55fef29fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f3127120f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef2a07fa0 0x55fef29fb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f3127120f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fef2a04780 con 0x55fef292e5f0 2024-04-25T11:35:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f3127120f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55fef2a04cb0 con 0x55fef2a07fa0 2024-04-25T11:35:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f3127120f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55fef29fa300 con 0x55fef292e1e0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f311f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e5f0 0x55fef29fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f311f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e5f0 0x55fef29fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56480/0 (socket says 172.21.15.19:56480) 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.997+0000 7f311f000640 1 -- 172.21.15.19:0/1874371554 learned_addr learned my addr 172.21.15.19:0/1874371554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f3120400640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef2a07fa0 0x55fef29fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f311fa00640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef292e1e0 0x55fef29f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f311f000640 1 -- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef292e1e0 msgr2=0x55fef29f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f311f000640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fef292e1e0 0x55fef29f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f311f000640 1 -- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef2a07fa0 msgr2=0x55fef29fb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f311f000640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef2a07fa0 0x55fef29fb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f311f000640 1 -- 172.21.15.19:0/1874371554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef29f7830 con 0x55fef292e5f0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f3120400640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fef2a07fa0 0x55fef29fb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f311f000640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e5f0 0x55fef29fac00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3108009470 tx=0x7f3108009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f3117e00640 1 -- 172.21.15.19:0/1874371554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108006810 con 0x55fef292e5f0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f3117e00640 1 -- 172.21.15.19:0/1874371554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31080069b0 con 0x55fef292e5f0 2024-04-25T11:35:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f3117e00640 1 -- 172.21.15.19:0/1874371554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108006d60 con 0x55fef292e5f0 2024-04-25T11:35:56.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.998+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fef29fcb10 con 0x55fef292e5f0 2024-04-25T11:35:56.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:56.999+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef29fb980 con 0x55fef292e5f0 2024-04-25T11:35:56.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.001+0000 7f3117e00640 1 -- 172.21.15.19:0/1874371554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f310801d440 con 0x55fef292e5f0 2024-04-25T11:35:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.002+0000 7f3117e00640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30fc072ec0 0x7f30fc0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.002+0000 7f3117e00640 1 -- 172.21.15.19:0/1874371554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f3108094230 con 0x55fef292e5f0 2024-04-25T11:35:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.002+0000 7f3127120f40 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fef2a01d20 0x55fef29fe320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.002+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e94) -- 0x55fef29fe860 con 0x55fef2a01d20 2024-04-25T11:35:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.002+0000 7f311fa00640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30fc072ec0 0x7f30fc0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.003+0000 7f3120400640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fef2a01d20 0x55fef29fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.004+0000 7f311fa00640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30fc072ec0 0x7f30fc0753d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3110000f80 tx=0x7f3110010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.004+0000 7f3120400640 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fef2a01d20 0x55fef29fe320 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.007+0000 7f3120400640 1 -- 172.21.15.19:0/1874371554 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 600.00000000 [delete] v94'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3118051070 con 0x55fef2a01d20 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.007+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fef2a01d20 msgr2=0x55fef29fe320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.007+0000 7f3127120f40 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55fef2a01d20 0x55fef29fe320 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30fc072ec0 msgr2=0x7f30fc0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30fc072ec0 0x7f30fc0753d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3110000f80 tx=0x7f3110010000 comp rx=0 tx=0).stop 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e5f0 msgr2=0x55fef29fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 --2- 172.21.15.19:0/1874371554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fef292e5f0 0x55fef29fac00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3108009470 tx=0x7f3108009930 comp rx=0 tx=0).stop 2024-04-25T11:35:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3120400640 1 -- 172.21.15.19:0/1874371554 reap_dead start 2024-04-25T11:35:56.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 shutdown_connections 2024-04-25T11:35:56.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 >> 172.21.15.19:0/1874371554 conn(0x55fef2850ac0 msgr2=0x55fef292c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:56.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 shutdown_connections 2024-04-25T11:35:56.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.008+0000 7f3127120f40 1 -- 172.21.15.19:0/1874371554 wait complete. 2024-04-25T11:35:57.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:35:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.311+0000 7f50e6200640 1 Processor -- start 2024-04-25T11:35:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.311+0000 7f50e6200640 1 -- start start 2024-04-25T11:35:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.313+0000 7f50e6200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50e0144340 0x7f50e006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.313+0000 7f50e6200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f50e006db10 0x7f50e015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.313+0000 7f50e6200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50e006df20 con 0x7f50e015ec90 2024-04-25T11:35:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.313+0000 7f50e6200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f50e0144750 con 0x7f50e0144340 2024-04-25T11:35:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.313+0000 7f50e6200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f50e0070000 con 0x7f50e006db10 2024-04-25T11:35:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50e4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50e0144340 0x7f50e006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50e5800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f50e015ec90 0x7f50e0161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38276/0 (socket says 172.21.15.19:38276) 2024-04-25T11:35:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50e5800640 1 -- 172.21.15.19:0/130608871 learned_addr learned my addr 172.21.15.19:0/130608871 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50dfe00640 1 --2- 172.21.15.19:0/130608871 >> v2:172.21.15.169:3300/0 conn(0x7f50e006db10 0x7f50e015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50e4e00640 1 -- 172.21.15.19:0/130608871 >> v2:172.21.15.169:3300/0 conn(0x7f50e006db10 msgr2=0x7f50e015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50e4e00640 1 --2- 172.21.15.19:0/130608871 >> v2:172.21.15.169:3300/0 conn(0x7f50e006db10 0x7f50e015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50e4e00640 1 -- 172.21.15.19:0/130608871 >> v1:172.21.15.19:6789/0 conn(0x7f50e015ec90 legacy=0x7f50e0161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50e4e00640 1 -- 172.21.15.19:0/130608871 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e006f240 con 0x7f50e0144340 2024-04-25T11:35:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.314+0000 7f50df400640 1 -- 172.21.15.19:0/130608871 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2950910827 0 0) 0x7f50d8001490 con 0x7f50e015ec90 2024-04-25T11:35:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.315+0000 7f50e4e00640 1 --2- 172.21.15.19:0/130608871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e0144340 0x7f50e006d5d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f50e0073ef0 tx=0x7f50d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b161a0bbbb7b6d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.316+0000 7f50df400640 1 -- 172.21.15.19:0/130608871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4037070 con 0x7f50e0144340 2024-04-25T11:35:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.316+0000 7f50df400640 1 -- 172.21.15.19:0/130608871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d4032440 con 0x7f50e0144340 2024-04-25T11:35:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.316+0000 7f50df400640 1 -- 172.21.15.19:0/130608871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4032810 con 0x7f50e0144340 2024-04-25T11:35:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.316+0000 7f50e6200640 1 -- 172.21.15.19:0/130608871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e0144340 msgr2=0x7f50e006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.316+0000 7f50e6200640 1 --2- 172.21.15.19:0/130608871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e0144340 0x7f50e006d5d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f50e0073ef0 tx=0x7f50d402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:35:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.317+0000 7f50e6200640 1 -- 172.21.15.19:0/130608871 shutdown_connections 2024-04-25T11:35:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.317+0000 7f50e6200640 1 --2- 172.21.15.19:0/130608871 >> v2:172.21.15.169:3300/0 conn(0x7f50e006db10 0x7f50e015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.317+0000 7f50e6200640 1 --2- 172.21.15.19:0/130608871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e0144340 0x7f50e006d5d0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.317+0000 7f50e6200640 1 -- 172.21.15.19:0/130608871 >> 172.21.15.19:0/130608871 conn(0x7f50e0072f40 msgr2=0x7f50e0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.317+0000 7f50e6200640 1 -- 172.21.15.19:0/130608871 shutdown_connections 2024-04-25T11:35:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.317+0000 7f50e6200640 1 -- 172.21.15.19:0/130608871 wait complete. 2024-04-25T11:35:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.319+0000 7f50e6200640 1 Processor -- start 2024-04-25T11:35:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.319+0000 7f50e6200640 1 -- start start 2024-04-25T11:35:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50e006db10 0x7f50e0145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50e0144340 0x7f50e013d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e015ec90 0x7f50e01f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e6200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50e006fae0 con 0x7f50e015ec90 2024-04-25T11:35:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f50e006f800 con 0x7f50e0144340 2024-04-25T11:35:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50e015f0a0 con 0x7f50e006db10 2024-04-25T11:35:57.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e015ec90 0x7f50e01f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e015ec90 0x7f50e01f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56496/0 (socket says 172.21.15.19:56496) 2024-04-25T11:35:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.321+0000 7f50e5800640 1 -- 172.21.15.19:0/495242683 learned_addr learned my addr 172.21.15.19:0/495242683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e4e00640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50e006db10 0x7f50e0145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:57.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50dfe00640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50e0144340 0x7f50e013d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e5800640 1 -- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50e006db10 msgr2=0x7f50e0145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e5800640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50e006db10 0x7f50e0145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e5800640 1 -- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50e0144340 msgr2=0x7f50e013d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e5800640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50e0144340 0x7f50e013d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e5800640 1 -- 172.21.15.19:0/495242683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e0147380 con 0x7f50e015ec90 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e4e00640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50e006db10 0x7f50e0145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50dfe00640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50e0144340 0x7f50e013d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.322+0000 7f50e5800640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e015ec90 0x7f50e01f9ca0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c700 tx=0x7f50d800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.323+0000 7f50dcc00640 1 -- 172.21.15.19:0/495242683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8018070 con 0x7f50e015ec90 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.323+0000 7f50dcc00640 1 -- 172.21.15.19:0/495242683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d800ce10 con 0x7f50e015ec90 2024-04-25T11:35:57.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.323+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e0147660 con 0x7f50e015ec90 2024-04-25T11:35:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.323+0000 7f50dcc00640 1 -- 172.21.15.19:0/495242683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d800a880 con 0x7f50e015ec90 2024-04-25T11:35:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.323+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e00af560 con 0x7f50e015ec90 2024-04-25T11:35:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.324+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e0076c60 con 0x7f50e015ec90 2024-04-25T11:35:57.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.325+0000 7f50dcc00640 1 -- 172.21.15.19:0/495242683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f50d800aa20 con 0x7f50e015ec90 2024-04-25T11:35:57.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.327+0000 7f50dcc00640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50c00744e0 0x7f50c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:57.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.327+0000 7f50dcc00640 1 -- 172.21.15.19:0/495242683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f50d80948e0 con 0x7f50e015ec90 2024-04-25T11:35:57.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.327+0000 7f50e4e00640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50c00744e0 0x7f50c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:57.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.328+0000 7f50e4e00640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50c00744e0 0x7f50c00769f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:57.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.332+0000 7f50dcc00640 1 -- 172.21.15.19:0/495242683 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50e006df20 con 0x7f50e015ec90 2024-04-25T11:35:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:57.580+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f50e015c7a0 con 0x7f50e015ec90 2024-04-25T11:35:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.240+0000 7f50dcc00640 1 -- 172.21.15.19:0/495242683 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v102) ==== 162+0+0 (secure 0 0 0) 0x7f50d805eea0 con 0x7f50e015ec90 2024-04-25T11:35:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:35:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50c00744e0 msgr2=0x7f50c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50c00744e0 0x7f50c00769f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f50d4034000 tx=0x7f50d402e000 comp rx=0 tx=0).stop 2024-04-25T11:35:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e015ec90 msgr2=0x7f50e01f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e015ec90 0x7f50e01f9ca0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c700 tx=0x7f50d800cbc0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 shutdown_connections 2024-04-25T11:35:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50c00744e0 0x7f50c00769f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50e0144340 0x7f50e013d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50e015ec90 0x7f50e01f9ca0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 --2- 172.21.15.19:0/495242683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f50e006db10 0x7f50e0145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.243+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 >> 172.21.15.19:0/495242683 conn(0x7f50e0072f40 msgr2=0x7f50e0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.244+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 shutdown_connections 2024-04-25T11:35:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.244+0000 7f50e6200640 1 -- 172.21.15.19:0/495242683 wait complete. 2024-04-25T11:35:58.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:35:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.548+0000 7fd797200640 1 Processor -- start 2024-04-25T11:35:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.548+0000 7fd797200640 1 -- start start 2024-04-25T11:35:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.550+0000 7fd797200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd798145e70 0x7fd798168e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.550+0000 7fd797200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79806fc70 con 0x7fd7980d1070 2024-04-25T11:35:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.550+0000 7fd797200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79806ff50 con 0x7fd798144350 2024-04-25T11:35:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.551+0000 7fd797200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd798070230 con 0x7fd798145e70 2024-04-25T11:35:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.551+0000 7fd795400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd7980d1070 0x7fd798162200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38278/0 (socket says 172.21.15.19:38278) 2024-04-25T11:35:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.551+0000 7fd795400640 1 -- 172.21.15.19:0/3374388309 learned_addr learned my addr 172.21.15.19:0/3374388309 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:35:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.551+0000 7fd796800640 1 --2- 172.21.15.19:0/3374388309 >> v2:172.21.15.169:3300/0 conn(0x7fd798145e70 0x7fd798168e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:58.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.551+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2217291291 0 0) 0x7fd79806fc70 con 0x7fd7980d1070 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.551+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd77c003520 con 0x7fd7980d1070 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2653113440 0 0) 0x7fd77c003520 con 0x7fd7980d1070 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 >> v2:172.21.15.169:3300/0 conn(0x7fd798145e70 msgr2=0x7fd798168e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 --2- 172.21.15.19:0/3374388309 >> v2:172.21.15.169:3300/0 conn(0x7fd798145e70 0x7fd798168e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 >> v1:172.21.15.169:6789/0 conn(0x7fd798144350 legacy=0x7fd798161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd798070510 con 0x7fd7980d1070 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fd784001670 con 0x7fd7980d1070 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 >> v1:172.21.15.19:6789/0 conn(0x7fd7980d1070 legacy=0x7fd798162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c003a00 0x7fd77c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77c004800 0x7fd77c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c005270 0x7fd77c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd798070230 con 0x7fd77c005270 2024-04-25T11:35:58.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c00c190 con 0x7fd77c003a00 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd77c00c390 con 0x7fd77c004800 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.552+0000 7fd796800640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c005270 0x7fd77c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.553+0000 7fd795400640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77c004800 0x7fd77c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.553+0000 7fd796800640 1 -- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77c004800 msgr2=0x7fd77c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.553+0000 7fd796800640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77c004800 0x7fd77c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.553+0000 7fd796800640 1 -- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c003a00 msgr2=0x7fd77c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.553+0000 7fd796800640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c003a00 0x7fd77c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.553+0000 7fd796800640 1 -- 172.21.15.19:0/3374388309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c003520 con 0x7fd77c005270 2024-04-25T11:35:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.553+0000 7fd795400640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd77c004800 0x7fd77c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:58.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.554+0000 7fd796800640 1 -- 172.21.15.19:0/3374388309 reap_dead start 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.554+0000 7fd796800640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c005270 0x7fd77c00b9c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd798072e90 tx=0x7fd77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcee61b758ec98a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.554+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790035020 con 0x7fd77c005270 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.554+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790003b90 con 0x7fd77c005270 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.554+0000 7fd794a00640 1 -- 172.21.15.19:0/3374388309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790009c10 con 0x7fd77c005270 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.555+0000 7fd797200640 1 -- 172.21.15.19:0/3374388309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c005270 msgr2=0x7fd77c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.555+0000 7fd797200640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c005270 0x7fd77c00b9c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd798072e90 tx=0x7fd77c005830 comp rx=0 tx=0).stop 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.555+0000 7fd797200640 1 -- 172.21.15.19:0/3374388309 shutdown_connections 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.555+0000 7fd797200640 1 --2- 172.21.15.19:0/3374388309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd77c005270 0x7fd77c00b9c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.555+0000 7fd797200640 1 -- 172.21.15.19:0/3374388309 >> 172.21.15.19:0/3374388309 conn(0x7fd7980d1720 msgr2=0x7fd7980d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.555+0000 7fd797200640 1 -- 172.21.15.19:0/3374388309 shutdown_connections 2024-04-25T11:35:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.555+0000 7fd797200640 1 -- 172.21.15.19:0/3374388309 wait complete. 2024-04-25T11:35:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.558+0000 7fd797200640 1 Processor -- start 2024-04-25T11:35:58.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.558+0000 7fd797200640 1 -- start start 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.559+0000 7fd797200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c005270 0x7fd79815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.559+0000 7fd797200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79815edb0 0x7fd798140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.559+0000 7fd797200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7981607b0 0x7fd798160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.559+0000 7fd797200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd798161af0 con 0x7fd79815edb0 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.559+0000 7fd797200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd798147910 con 0x7fd77c005270 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd797200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd798161c70 con 0x7fd7981607b0 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79815edb0 0x7fd798140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79815edb0 0x7fd798140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56516/0 (socket says 172.21.15.19:56516) 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 -- 172.21.15.19:0/2504572326 learned_addr learned my addr 172.21.15.19:0/2504572326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795e00640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c005270 0x7fd79815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd796800640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7981607b0 0x7fd798160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 -- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7981607b0 msgr2=0x7fd798160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7981607b0 0x7fd798160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 -- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c005270 msgr2=0x7fd79815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c005270 0x7fd79815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.560+0000 7fd795400640 1 -- 172.21.15.19:0/2504572326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd798161100 con 0x7fd79815edb0 2024-04-25T11:35:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd795e00640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c005270 0x7fd79815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd796800640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7981607b0 0x7fd798160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:35:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd795400640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79815edb0 0x7fd798140560 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd784001200 tx=0x7fd78400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd78e000640 1 -- 172.21.15.19:0/2504572326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784019070 con 0x7fd79815edb0 2024-04-25T11:35:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd78e000640 1 -- 172.21.15.19:0/2504572326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78400de50 con 0x7fd79815edb0 2024-04-25T11:35:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd78e000640 1 -- 172.21.15.19:0/2504572326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7840047e0 con 0x7fd79815edb0 2024-04-25T11:35:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7981f65a0 con 0x7fd79815edb0 2024-04-25T11:35:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.561+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7980af2b0 con 0x7fd79815edb0 2024-04-25T11:35:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.563+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd798050740 con 0x7fd79815edb0 2024-04-25T11:35:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.564+0000 7fd78e000640 1 -- 172.21.15.19:0/2504572326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd78401c070 con 0x7fd79815edb0 2024-04-25T11:35:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.565+0000 7fd78e000640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd778074500 0x7fd778076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.565+0000 7fd78e000640 1 -- 172.21.15.19:0/2504572326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fd784094b90 con 0x7fd79815edb0 2024-04-25T11:35:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.566+0000 7fd795e00640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd778074500 0x7fd778076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.567+0000 7fd795e00640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd778074500 0x7fd778076a10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd788007fd0 tx=0x7fd788009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:58.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.570+0000 7fd78e000640 1 -- 172.21.15.19:0/2504572326 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd798161100 con 0x7fd79815edb0 2024-04-25T11:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.815+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd79806d910 con 0x7fd79815edb0 2024-04-25T11:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.816+0000 7fd78e000640 1 -- 172.21.15.19:0/2504572326 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5186 (secure 0 0 0) 0x7fd78405ec90 con 0x7fd79815edb0 2024-04-25T11:35:58.804 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:35:58.804 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":103,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:35:58.246014+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8097":{"gid":8097,"name":"a","rank":0,"incarnation":103,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/1743540779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1743540779},{"type":"v1","addr":"172.21.15.19:6835","nonce":1743540779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 103 2024-04-25T11:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd778074500 msgr2=0x7fd778076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd778074500 0x7fd778076a10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd788007fd0 tx=0x7fd788009040 comp rx=0 tx=0).stop 2024-04-25T11:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79815edb0 msgr2=0x7fd798140560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79815edb0 0x7fd798140560 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd784001200 tx=0x7fd78400dc00 comp rx=0 tx=0).stop 2024-04-25T11:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 shutdown_connections 2024-04-25T11:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd778074500 0x7fd778076a10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd7981607b0 0x7fd798160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd79815edb0 0x7fd798140560 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 --2- 172.21.15.19:0/2504572326 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd77c005270 0x7fd79815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 >> 172.21.15.19:0/2504572326 conn(0x7fd7980d1720 msgr2=0x7fd798144d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.819+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 shutdown_connections 2024-04-25T11:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:35:58.820+0000 7fd797200640 1 -- 172.21.15.19:0/2504572326 wait complete. 2024-04-25T11:35:58.837 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:36:01.285 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:01.297+0000 7f3db3400640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-04-25T11:36:01.285 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:01.297+0000 7f3db3400640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-25T11:36:01.285 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:01.298+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:36:01.286 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:01.298+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:36:01.307 INFO:tasks.ceph.mds.a.smithi019.stderr: -16> 2024-04-25T11:36:01.297+0000 7f3db3400640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-04-25T11:36:01.307 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:36:01.297+0000 7f3db3400640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-25T11:36:01.452 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:36:03.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.143+0000 7f7f2e400640 1 Processor -- start 2024-04-25T11:36:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.143+0000 7f7f2e400640 1 -- start start 2024-04-25T11:36:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.145+0000 7f7f2e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 0x7f7f28161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.145+0000 7f7f2e400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7f280709a0 0x7f7f281569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.145+0000 7f7f2e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f281615d0 con 0x7f7f280b0800 2024-04-25T11:36:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.145+0000 7f7f2e400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7f280cffc0 con 0x7f7f280709a0 2024-04-25T11:36:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.145+0000 7f7f2e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7f280d02a0 con 0x7f7f280cf9e0 2024-04-25T11:36:04.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.146+0000 7f7f27e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7f280b0800 0x7f7f28070290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56624/0 (socket says 172.21.15.19:56624) 2024-04-25T11:36:04.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.146+0000 7f7f27e00640 1 -- 172.21.15.19:0/2107836454 learned_addr learned my addr 172.21.15.19:0/2107836454 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.146+0000 7f7f2da00640 1 --2- 172.21.15.19:0/2107836454 >> v2:172.21.15.19:3300/0 conn(0x7f7f280709a0 0x7f7f281569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.146+0000 7f7f2d000640 1 --2- 172.21.15.19:0/2107836454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 0x7f7f28161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.146+0000 7f7f27400640 1 -- 172.21.15.19:0/2107836454 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230745592 0 0) 0x7f7f14001110 con 0x7f7f280b0800 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.146+0000 7f7f27400640 1 -- 172.21.15.19:0/2107836454 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f10003520 con 0x7f7f280b0800 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.147+0000 7f7f2da00640 1 -- 172.21.15.19:0/2107836454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 msgr2=0x7f7f28161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.147+0000 7f7f2da00640 1 --2- 172.21.15.19:0/2107836454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 0x7f7f28161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.147+0000 7f7f2da00640 1 -- 172.21.15.19:0/2107836454 >> v1:172.21.15.19:6789/0 conn(0x7f7f280b0800 legacy=0x7f7f28070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.147+0000 7f7f2da00640 1 -- 172.21.15.19:0/2107836454 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f280d0580 con 0x7f7f280709a0 2024-04-25T11:36:04.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.148+0000 7f7f2da00640 1 --2- 172.21.15.19:0/2107836454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280709a0 0x7f7f281569b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009d90 tx=0x7f7f2002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81201c4bc1c52051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:04.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.148+0000 7f7f27400640 1 -- 172.21.15.19:0/2107836454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20031600 con 0x7f7f280709a0 2024-04-25T11:36:04.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.148+0000 7f7f27400640 1 -- 172.21.15.19:0/2107836454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f200317a0 con 0x7f7f280709a0 2024-04-25T11:36:04.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.148+0000 7f7f27400640 1 -- 172.21.15.19:0/2107836454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20031b70 con 0x7f7f280709a0 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 -- 172.21.15.19:0/2107836454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280709a0 msgr2=0x7f7f281569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2107836454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280709a0 0x7f7f281569b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009d90 tx=0x7f7f2002a5d0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 -- 172.21.15.19:0/2107836454 shutdown_connections 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2107836454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280709a0 0x7f7f281569b0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2107836454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 0x7f7f28161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 -- 172.21.15.19:0/2107836454 >> 172.21.15.19:0/2107836454 conn(0x7f7f280d1720 msgr2=0x7f7f280d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 -- 172.21.15.19:0/2107836454 shutdown_connections 2024-04-25T11:36:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.149+0000 7f7f2e400640 1 -- 172.21.15.19:0/2107836454 wait complete. 2024-04-25T11:36:04.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.151+0000 7f7f2e400640 1 Processor -- start 2024-04-25T11:36:04.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.151+0000 7f7f2e400640 1 -- start start 2024-04-25T11:36:04.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280b0800 0x7f7f28146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 0x7f7f281f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f28155200 0x7f7f28155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f280b0d90 con 0x7f7f280b0800 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7f280b0c10 con 0x7f7f28155200 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7f28156ef0 con 0x7f7f280cf9e0 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280b0800 0x7f7f28146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280b0800 0x7f7f28146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59344/0 (socket says 172.21.15.19:59344) 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.153+0000 7f7f2d000640 1 -- 172.21.15.19:0/2964884273 learned_addr learned my addr 172.21.15.19:0/2964884273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.154+0000 7f7f2d000640 1 -- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 msgr2=0x7f7f281f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.154+0000 7f7f2d000640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 0x7f7f281f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.154+0000 7f7f2d000640 1 -- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f28155200 msgr2=0x7f7f28155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.154+0000 7f7f2d000640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f28155200 0x7f7f28155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.154+0000 7f7f2d000640 1 -- 172.21.15.19:0/2964884273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f28155b50 con 0x7f7f280b0800 2024-04-25T11:36:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.154+0000 7f7f2d000640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280b0800 0x7f7f28146100 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e870 tx=0x7f7f1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.155+0000 7f7f24c00640 1 -- 172.21.15.19:0/2964884273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c017070 con 0x7f7f280b0800 2024-04-25T11:36:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.155+0000 7f7f24c00640 1 -- 172.21.15.19:0/2964884273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c00c940 con 0x7f7f280b0800 2024-04-25T11:36:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.155+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f281fa2a0 con 0x7f7f280b0800 2024-04-25T11:36:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.155+0000 7f7f24c00640 1 -- 172.21.15.19:0/2964884273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00cd10 con 0x7f7f280b0800 2024-04-25T11:36:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.155+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f280b4700 con 0x7f7f280b0800 2024-04-25T11:36:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.156+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f28050740 con 0x7f7f280b0800 2024-04-25T11:36:04.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.157+0000 7f7f24c00640 1 -- 172.21.15.19:0/2964884273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f1c01a070 con 0x7f7f280b0800 2024-04-25T11:36:04.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.158+0000 7f7f24c00640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f080744e0 0x7f7f080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.158+0000 7f7f24c00640 1 -- 172.21.15.19:0/2964884273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7f7f1c095120 con 0x7f7f280b0800 2024-04-25T11:36:04.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.159+0000 7f7f27e00640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f080744e0 0x7f7f080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.160+0000 7f7f27e00640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f080744e0 0x7f7f080769f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000f60 tx=0x7f7f14001110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:04.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.166+0000 7f7f24c00640 1 -- 172.21.15.19:0/2964884273 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f28155b50 con 0x7f7f280b0800 2024-04-25T11:36:04.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.410+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f2806fb60 con 0x7f7f280b0800 2024-04-25T11:36:04.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.412+0000 7f7f24c00640 1 -- 172.21.15.19:0/2964884273 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5177 (secure 0 0 0) 0x7f7f1c05f830 con 0x7f7f280b0800 2024-04-25T11:36:04.400 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:04.400 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3516161211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3516161211},{"type":"v1","addr":"172.21.15.19:6835","nonce":3516161211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:01.298712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:04.400 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 108 2024-04-25T11:36:04.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.414+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f080744e0 msgr2=0x7f7f080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.414+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f080744e0 0x7f7f080769f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000f60 tx=0x7f7f14001110 comp rx=0 tx=0).stop 2024-04-25T11:36:04.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.414+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280b0800 msgr2=0x7f7f28146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.414+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280b0800 0x7f7f28146100 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e870 tx=0x7f7f1c00ed30 comp rx=0 tx=0).stop 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 shutdown_connections 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f080744e0 0x7f7f080769f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f28155200 0x7f7f28155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7f280cf9e0 0x7f7f281f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 --2- 172.21.15.19:0/2964884273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f280b0800 0x7f7f28146100 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 >> 172.21.15.19:0/2964884273 conn(0x7f7f280d1720 msgr2=0x7f7f2806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 shutdown_connections 2024-04-25T11:36:04.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.415+0000 7f7f2e400640 1 -- 172.21.15.19:0/2964884273 wait complete. 2024-04-25T11:36:04.431 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:36:04.431 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:04.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.722+0000 7f14b7e00640 1 Processor -- start 2024-04-25T11:36:04.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.722+0000 7f14b7e00640 1 -- start start 2024-04-25T11:36:04.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.724+0000 7f14b7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14b80d1070 0x7f14b8162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.724+0000 7f14b7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14b8144e60 0x7f14b8145270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.725+0000 7f14b7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b806fc70 con 0x7f14b8144350 2024-04-25T11:36:04.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.725+0000 7f14b7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f14b806ff50 con 0x7f14b80d1070 2024-04-25T11:36:04.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.725+0000 7f14b7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f14b8070230 con 0x7f14b8144e60 2024-04-25T11:36:04.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.725+0000 7f14b6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14b80d1070 0x7f14b8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.725+0000 7f14b6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14b80d1070 0x7f14b8162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59356/0 (socket says 172.21.15.19:59356) 2024-04-25T11:36:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.725+0000 7f14b6000640 1 -- 172.21.15.19:0/2931994864 learned_addr learned my addr 172.21.15.19:0/2931994864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.726+0000 7f14b7400640 1 --2- 172.21.15.19:0/2931994864 >> v2:172.21.15.169:3300/0 conn(0x7f14b8144e60 0x7f14b8145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.726+0000 7f14b6000640 1 -- 172.21.15.19:0/2931994864 >> v2:172.21.15.169:3300/0 conn(0x7f14b8144e60 msgr2=0x7f14b8145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.726+0000 7f14b6000640 1 --2- 172.21.15.19:0/2931994864 >> v2:172.21.15.169:3300/0 conn(0x7f14b8144e60 0x7f14b8145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.726+0000 7f14b6000640 1 -- 172.21.15.19:0/2931994864 >> v1:172.21.15.169:6789/0 conn(0x7f14b8144350 legacy=0x7f14b8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:04.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.726+0000 7f14b6000640 1 -- 172.21.15.19:0/2931994864 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b8070510 con 0x7f14b80d1070 2024-04-25T11:36:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.727+0000 7f14b6000640 1 --2- 172.21.15.19:0/2931994864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b80d1070 0x7f14b8162200 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b4820 tx=0x7f14a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14dea077f4a2f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.727+0000 7f14b5600640 1 -- 172.21.15.19:0/2931994864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8037070 con 0x7f14b80d1070 2024-04-25T11:36:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.727+0000 7f14b5600640 1 -- 172.21.15.19:0/2931994864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a8032440 con 0x7f14b80d1070 2024-04-25T11:36:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.727+0000 7f14b5600640 1 -- 172.21.15.19:0/2931994864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8032810 con 0x7f14b80d1070 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 -- 172.21.15.19:0/2931994864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b80d1070 msgr2=0x7f14b8162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 --2- 172.21.15.19:0/2931994864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b80d1070 0x7f14b8162200 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b4820 tx=0x7f14a802a230 comp rx=0 tx=0).stop 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 -- 172.21.15.19:0/2931994864 shutdown_connections 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 --2- 172.21.15.19:0/2931994864 >> v2:172.21.15.169:3300/0 conn(0x7f14b8144e60 0x7f14b8145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 --2- 172.21.15.19:0/2931994864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b80d1070 0x7f14b8162200 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 -- 172.21.15.19:0/2931994864 >> 172.21.15.19:0/2931994864 conn(0x7f14b80d1720 msgr2=0x7f14b80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 -- 172.21.15.19:0/2931994864 shutdown_connections 2024-04-25T11:36:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.728+0000 7f14b7e00640 1 -- 172.21.15.19:0/2931994864 wait complete. 2024-04-25T11:36:04.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.731+0000 7f14b7e00640 1 Processor -- start 2024-04-25T11:36:04.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.731+0000 7f14b7e00640 1 -- start start 2024-04-25T11:36:04.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.732+0000 7f14b7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14b80d1070 0x7f14b81fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.732+0000 7f14b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b8144350 0x7f14b815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.732+0000 7f14b7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14b8144e60 0x7f14b80aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.732+0000 7f14b7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8162740 con 0x7f14b8144350 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.732+0000 7f14b7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8147910 con 0x7f14b8144e60 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.732+0000 7f14b7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14b81628c0 con 0x7f14b80d1070 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b8144350 0x7f14b815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b8144350 0x7f14b815baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59364/0 (socket says 172.21.15.19:59364) 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 -- 172.21.15.19:0/3479109625 learned_addr learned my addr 172.21.15.19:0/3479109625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 -- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14b80d1070 msgr2=0x7f14b81fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b7400640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14b8144e60 0x7f14b80aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14b80d1070 0x7f14b81fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 -- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14b8144e60 msgr2=0x7f14b80aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14b8144e60 0x7f14b80aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.733+0000 7f14b6000640 1 -- 172.21.15.19:0/3479109625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b81fa7c0 con 0x7f14b8144350 2024-04-25T11:36:04.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.734+0000 7f14b6000640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b8144350 0x7f14b815baf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f14a8000c00 tx=0x7f14a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:04.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.734+0000 7f14af400640 1 -- 172.21.15.19:0/3479109625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8041070 con 0x7f14b8144350 2024-04-25T11:36:04.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.734+0000 7f14af400640 1 -- 172.21.15.19:0/3479109625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a803c870 con 0x7f14b8144350 2024-04-25T11:36:04.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.734+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b81faaa0 con 0x7f14b8144350 2024-04-25T11:36:04.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.734+0000 7f14af400640 1 -- 172.21.15.19:0/3479109625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8037040 con 0x7f14b8144350 2024-04-25T11:36:04.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.734+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b80ae850 con 0x7f14b8144350 2024-04-25T11:36:04.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.735+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b80ae640 con 0x7f14b8144350 2024-04-25T11:36:04.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.736+0000 7f14af400640 1 -- 172.21.15.19:0/3479109625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14a8044070 con 0x7f14b8144350 2024-04-25T11:36:04.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.737+0000 7f14af400640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14940744b0 0x7f14940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:04.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.738+0000 7f14af400640 1 -- 172.21.15.19:0/3479109625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7f14a80b5e50 con 0x7f14b8144350 2024-04-25T11:36:04.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.738+0000 7f14b6a00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14940744b0 0x7f14940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:04.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.739+0000 7f14b6a00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14940744b0 0x7f14940769c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006fd0 tx=0x7f14a0006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:04.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.742+0000 7f14af400640 1 -- 172.21.15.19:0/3479109625 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14a8080440 con 0x7f14b8144350 2024-04-25T11:36:04.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.990+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b8050740 con 0x7f14b8144350 2024-04-25T11:36:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.991+0000 7f14af400640 1 -- 172.21.15.19:0/3479109625 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5177 (secure 0 0 0) 0x7f14a807fd40 con 0x7f14b8144350 2024-04-25T11:36:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:04.979 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3516161211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3516161211},{"type":"v1","addr":"172.21.15.19:6835","nonce":3516161211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:01.298712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:04.980 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 108 2024-04-25T11:36:04.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14940744b0 msgr2=0x7f14940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14940744b0 0x7f14940769c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006fd0 tx=0x7f14a0006bc0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b8144350 msgr2=0x7f14b815baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:04.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b8144350 0x7f14b815baf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f14a8000c00 tx=0x7f14a80046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 shutdown_connections 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14940744b0 0x7f14940769c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14b8144e60 0x7f14b80aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14b8144350 0x7f14b815baf0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 --2- 172.21.15.19:0/3479109625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14b80d1070 0x7f14b81fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 >> 172.21.15.19:0/3479109625 conn(0x7f14b80d1720 msgr2=0x7f14b81617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 shutdown_connections 2024-04-25T11:36:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:04.994+0000 7f14b7e00640 1 -- 172.21.15.19:0/3479109625 wait complete. 2024-04-25T11:36:04.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:36:04.999 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:36:05.010 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:36:05.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:05.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.333+0000 7f1b8c800640 1 Processor -- start 2024-04-25T11:36:05.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.334+0000 7f1b8c800640 1 -- start start 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b8c800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b840cf5e0 0x7f1b840cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b8c800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b840cff30 con 0x7f1b841443c0 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b8c800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b8415f5f0 con 0x7f1b8406c5c0 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b8c800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1b840708d0 con 0x7f1b840cf5e0 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b89400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b840cf5e0 0x7f1b840cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b89400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b840cf5e0 0x7f1b840cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59374/0 (socket says 172.21.15.19:59374) 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b89400640 1 -- 172.21.15.19:0/503449888 learned_addr learned my addr 172.21.15.19:0/503449888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.337+0000 7f1b89e00640 1 --1- 172.21.15.19:0/503449888 >> v1:172.21.15.19:6789/0 conn(0x7f1b841443c0 0x7f1b8406fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56630/0 (socket says 172.21.15.19:56630) 2024-04-25T11:36:05.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.338+0000 7f1b89400640 1 -- 172.21.15.19:0/503449888 >> v1:172.21.15.169:6789/0 conn(0x7f1b8406c5c0 legacy=0x7f1b8406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:05.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.338+0000 7f1b89400640 1 -- 172.21.15.19:0/503449888 >> v1:172.21.15.19:6789/0 conn(0x7f1b841443c0 legacy=0x7f1b8406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:05.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.338+0000 7f1b89400640 1 -- 172.21.15.19:0/503449888 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8415e830 con 0x7f1b840cf5e0 2024-04-25T11:36:05.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.339+0000 7f1b89400640 1 --2- 172.21.15.19:0/503449888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b840cf9f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009870 tx=0x7f1b7402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b602b05efd0a6059 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:05.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.339+0000 7f1b88a00640 1 -- 172.21.15.19:0/503449888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74037070 con 0x7f1b840cf5e0 2024-04-25T11:36:05.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.340+0000 7f1b88a00640 1 -- 172.21.15.19:0/503449888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b74032440 con 0x7f1b840cf5e0 2024-04-25T11:36:05.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.340+0000 7f1b88a00640 1 -- 172.21.15.19:0/503449888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74032810 con 0x7f1b840cf5e0 2024-04-25T11:36:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.340+0000 7f1b8c800640 1 -- 172.21.15.19:0/503449888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 msgr2=0x7f1b840cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.340+0000 7f1b8c800640 1 --2- 172.21.15.19:0/503449888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b840cf9f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009870 tx=0x7f1b7402a2a0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.341+0000 7f1b8c800640 1 -- 172.21.15.19:0/503449888 shutdown_connections 2024-04-25T11:36:05.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.341+0000 7f1b8c800640 1 --2- 172.21.15.19:0/503449888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b840cf9f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.341+0000 7f1b8c800640 1 -- 172.21.15.19:0/503449888 >> 172.21.15.19:0/503449888 conn(0x7f1b840d1ae0 msgr2=0x7f1b840d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:05.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.341+0000 7f1b8c800640 1 -- 172.21.15.19:0/503449888 shutdown_connections 2024-04-25T11:36:05.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.341+0000 7f1b8c800640 1 -- 172.21.15.19:0/503449888 wait complete. 2024-04-25T11:36:05.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.343+0000 7f1b8c800640 1 Processor -- start 2024-04-25T11:36:05.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.343+0000 7f1b8c800640 1 -- start start 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b8c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b8406c5c0 0x7f1b84142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b8c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b84157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b8c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b841443c0 0x7f1b84159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b8c800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8415edf0 con 0x7f1b840cf5e0 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b8c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8415f160 con 0x7f1b841443c0 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b8c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1b841447d0 con 0x7f1b8406c5c0 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b89400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b84157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b89400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b84157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59376/0 (socket says 172.21.15.19:59376) 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b89400640 1 -- 172.21.15.19:0/3449699856 learned_addr learned my addr 172.21.15.19:0/3449699856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:05.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.345+0000 7f1b89e00640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b8406c5c0 0x7f1b84142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b8a800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b841443c0 0x7f1b84159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b89400640 1 -- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b8406c5c0 msgr2=0x7f1b84142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b89400640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b8406c5c0 0x7f1b84142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b89400640 1 -- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b841443c0 msgr2=0x7f1b84159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b89400640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b841443c0 0x7f1b84159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b89400640 1 -- 172.21.15.19:0/3449699856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b84157e70 con 0x7f1b840cf5e0 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b8a800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b841443c0 0x7f1b84159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b89400640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b84157930 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009870 tx=0x7f1b740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:05.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b7e000640 1 -- 172.21.15.19:0/3449699856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74041070 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b7e000640 1 -- 172.21.15.19:0/3449699856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7403c870 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b7e000640 1 -- 172.21.15.19:0/3449699856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74037040 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.346+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b841f7ba0 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.347+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b841f8160 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.348+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b84050740 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.349+0000 7f1b7e000640 1 -- 172.21.15.19:0/3449699856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1b74044070 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.350+0000 7f1b7e000640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1b680744b0 0x7f1b680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.350+0000 7f1b7e000640 1 -- 172.21.15.19:0/3449699856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7f1b740b5f00 con 0x7f1b840cf5e0 2024-04-25T11:36:05.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.351+0000 7f1b89e00640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1b680744b0 0x7f1b680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:05.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.352+0000 7f1b89e00640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1b680744b0 0x7f1b680769c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000f60 tx=0x7f1b78007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.357+0000 7f1b7e000640 1 -- 172.21.15.19:0/3449699856 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b74080580 con 0x7f1b840cf5e0 2024-04-25T11:36:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.610+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b8406f950 con 0x7f1b840cf5e0 2024-04-25T11:36:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.611+0000 7f1b7e000640 1 -- 172.21.15.19:0/3449699856 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5177 (secure 0 0 0) 0x7f1b7407fe80 con 0x7f1b840cf5e0 2024-04-25T11:36:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:05.600 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3516161211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3516161211},{"type":"v1","addr":"172.21.15.19:6835","nonce":3516161211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:01.298712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 108 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1b680744b0 msgr2=0x7f1b680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1b680744b0 0x7f1b680769c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1b78000f60 tx=0x7f1b78007ce0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 msgr2=0x7f1b84157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b84157930 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009870 tx=0x7f1b740046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 shutdown_connections 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1b680744b0 0x7f1b680769c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b841443c0 0x7f1b84159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b840cf5e0 0x7f1b84157930 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 --2- 172.21.15.19:0/3449699856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b8406c5c0 0x7f1b84142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 >> 172.21.15.19:0/3449699856 conn(0x7f1b840d1ae0 msgr2=0x7f1b8406d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 shutdown_connections 2024-04-25T11:36:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:05.614+0000 7f1b8c800640 1 -- 172.21.15.19:0/3449699856 wait complete. 2024-04-25T11:36:05.629 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2024-04-25T11:36:05.629 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2024-04-25T11:36:05.629 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:05.630 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:05.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:36:05.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:36:05.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:05.681 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:36:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:36:05.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:36:05.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:36:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.072+0000 7fc226000640 1 Processor -- start 2024-04-25T11:36:06.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.072+0000 7fc226000640 1 -- start start 2024-04-25T11:36:06.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.074+0000 7fc226000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc2201443c0 0x7fc22006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc226000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cfcf0 0x7fc22006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc226000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22015f5f0 con 0x7fc220070370 2024-04-25T11:36:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc226000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc2200708d0 con 0x7fc2201443c0 2024-04-25T11:36:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc226000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc22015f160 con 0x7fc2200cfcf0 2024-04-25T11:36:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc217e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc220070370 0x7fc2200cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56634/0 (socket says 172.21.15.19:56634) 2024-04-25T11:36:06.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc217e00640 1 -- 172.21.15.19:0/2400394251 learned_addr learned my addr 172.21.15.19:0/2400394251 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc21f400640 1 --2- 172.21.15.19:0/2400394251 >> v2:172.21.15.169:3300/0 conn(0x7fc2201443c0 0x7fc22006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.075+0000 7fc21fe00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cfcf0 0x7fc22006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2872658692 0 0) 0x7fc22015f5f0 con 0x7fc220070370 2024-04-25T11:36:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc200003520 con 0x7fc220070370 2024-04-25T11:36:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1538999772 0 0) 0x7fc200003520 con 0x7fc220070370 2024-04-25T11:36:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cfcf0 msgr2=0x7fc22006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:06.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cfcf0 0x7fc22006c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 >> v2:172.21.15.169:3300/0 conn(0x7fc2201443c0 msgr2=0x7fc22006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 --2- 172.21.15.19:0/2400394251 >> v2:172.21.15.169:3300/0 conn(0x7fc2201443c0 0x7fc22006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22015e830 con 0x7fc220070370 2024-04-25T11:36:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fc208001670 con 0x7fc220070370 2024-04-25T11:36:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 >> v1:172.21.15.19:6789/0 conn(0x7fc220070370 legacy=0x7fc2200cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2000039b0 0x7fc200003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc200004340 0x7fc200004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004ed0 0x7fc20000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2200708d0 con 0x7fc200004340 2024-04-25T11:36:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc22015f160 con 0x7fc200004ed0 2024-04-25T11:36:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.076+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc20000bf00 con 0x7fc2000039b0 2024-04-25T11:36:06.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc217e00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc200004340 0x7fc200004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc21f400640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2000039b0 0x7fc200003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc21fe00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004ed0 0x7fc20000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc217e00640 1 -- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2000039b0 msgr2=0x7fc200003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc217e00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2000039b0 0x7fc200003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc217e00640 1 -- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004ed0 msgr2=0x7fc20000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc217e00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004ed0 0x7fc20000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:06.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc217e00640 1 -- 172.21.15.19:0/2400394251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc200003520 con 0x7fc200004340 2024-04-25T11:36:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.077+0000 7fc21fe00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004ed0 0x7fc20000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.078+0000 7fc217e00640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc200004340 0x7fc200004860 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc22006f410 tx=0x7fc200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d131f32507f943a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.079+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208007d30 con 0x7fc200004340 2024-04-25T11:36:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.079+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc208005af0 con 0x7fc200004340 2024-04-25T11:36:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.079+0000 7fc21ea00640 1 -- 172.21.15.19:0/2400394251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208007070 con 0x7fc200004340 2024-04-25T11:36:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.079+0000 7fc226000640 1 -- 172.21.15.19:0/2400394251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc200004340 msgr2=0x7fc200004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:06.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.079+0000 7fc226000640 1 --2- 172.21.15.19:0/2400394251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc200004340 0x7fc200004860 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc22006f410 tx=0x7fc200005830 comp rx=0 tx=0).stop 2024-04-25T11:36:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.079+0000 7fc21fe00640 1 -- 172.21.15.19:0/2400394251 reap_dead start 2024-04-25T11:36:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.080+0000 7fc226000640 1 -- 172.21.15.19:0/2400394251 shutdown_connections 2024-04-25T11:36:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.080+0000 7fc226000640 1 -- 172.21.15.19:0/2400394251 >> 172.21.15.19:0/2400394251 conn(0x7fc2200d1ae0 msgr2=0x7fc2200d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.080+0000 7fc226000640 1 -- 172.21.15.19:0/2400394251 shutdown_connections 2024-04-25T11:36:06.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.080+0000 7fc226000640 1 -- 172.21.15.19:0/2400394251 wait complete. 2024-04-25T11:36:06.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.082+0000 7fc226000640 1 Processor -- start 2024-04-25T11:36:06.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.082+0000 7fc226000640 1 -- start start 2024-04-25T11:36:06.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc226000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004340 0x7fc220151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc226000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2200cd370 0x7fc2200cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc226000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cdcc0 0x7fc2200ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc226000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201447d0 con 0x7fc2200cd370 2024-04-25T11:36:06.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc226000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc22015edf0 con 0x7fc200004340 2024-04-25T11:36:06.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc226000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc220162b00 con 0x7fc2200cdcc0 2024-04-25T11:36:06.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc217e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2200cd370 0x7fc2200cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc217e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2200cd370 0x7fc2200cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59404/0 (socket says 172.21.15.19:59404) 2024-04-25T11:36:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.084+0000 7fc217e00640 1 -- 172.21.15.19:0/4235786015 learned_addr learned my addr 172.21.15.19:0/4235786015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217e00640 1 -- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cdcc0 msgr2=0x7fc2200ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217e00640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cdcc0 0x7fc2200ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217e00640 1 -- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004340 msgr2=0x7fc220151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217e00640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004340 0x7fc220151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217e00640 1 -- 172.21.15.19:0/4235786015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2200ccb80 con 0x7fc2200cd370 2024-04-25T11:36:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217e00640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2200cd370 0x7fc2200cd780 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c300 tx=0x7fc22015ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217400640 1 -- 172.21.15.19:0/4235786015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080015b0 con 0x7fc2200cd370 2024-04-25T11:36:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217400640 1 -- 172.21.15.19:0/4235786015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc208042070 con 0x7fc2200cd370 2024-04-25T11:36:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc217400640 1 -- 172.21.15.19:0/4235786015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208007270 con 0x7fc2200cd370 2024-04-25T11:36:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2200cce60 con 0x7fc2200cd370 2024-04-25T11:36:06.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.085+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22015b8f0 con 0x7fc2200cd370 2024-04-25T11:36:06.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.087+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2200cf570 con 0x7fc2200cd370 2024-04-25T11:36:06.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.088+0000 7fc217400640 1 -- 172.21.15.19:0/4235786015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc208007410 con 0x7fc2200cd370 2024-04-25T11:36:06.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.089+0000 7fc217400640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1fc0744b0 0x7fc1fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:06.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.089+0000 7fc217400640 1 -- 172.21.15.19:0/4235786015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7fc2080b5350 con 0x7fc2200cd370 2024-04-25T11:36:06.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.090+0000 7fc21f400640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1fc0744b0 0x7fc1fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:06.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.091+0000 7fc21f400640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1fc0744b0 0x7fc1fc0769c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc2200d1860 tx=0x7fc210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:06.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.094+0000 7fc217400640 1 -- 172.21.15.19:0/4235786015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc20807f9d0 con 0x7fc2200cd370 2024-04-25T11:36:06.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:06.346+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc220050740 con 0x7fc2200cd370 2024-04-25T11:36:07.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.352+0000 7fc217400640 1 -- 172.21.15.19:0/4235786015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v109) ==== 161+0+0 (secure 0 0 0) 0x7fc20807f2d0 con 0x7fc2200cd370 2024-04-25T11:36:07.341 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:36:07.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1fc0744b0 msgr2=0x7fc1fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:07.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1fc0744b0 0x7fc1fc0769c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc2200d1860 tx=0x7fc210008040 comp rx=0 tx=0).stop 2024-04-25T11:36:07.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2200cd370 msgr2=0x7fc2200cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:07.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2200cd370 0x7fc2200cd780 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc20800c300 tx=0x7fc22015ea00 comp rx=0 tx=0).stop 2024-04-25T11:36:07.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 shutdown_connections 2024-04-25T11:36:07.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc1fc0744b0 0x7fc1fc0769c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2200cdcc0 0x7fc2200ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2200cd370 0x7fc2200cd780 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 --2- 172.21.15.19:0/4235786015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc200004340 0x7fc220151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 >> 172.21.15.19:0/4235786015 conn(0x7fc2200d1ae0 msgr2=0x7fc22006c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:07.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.356+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 shutdown_connections 2024-04-25T11:36:07.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.357+0000 7fc226000640 1 -- 172.21.15.19:0/4235786015 wait complete. 2024-04-25T11:36:07.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:36:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.675+0000 7fa66b800640 1 Processor -- start 2024-04-25T11:36:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.675+0000 7fa66b800640 1 -- start start 2024-04-25T11:36:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.677+0000 7fa66b800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6640b6a00 0x7fa66415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.677+0000 7fa66b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66415a2f0 0x7fa66406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa66b800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6641611e0 con 0x7fa66406d390 2024-04-25T11:36:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa66b800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa664160d50 con 0x7fa6640b6a00 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa66b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa66415a700 con 0x7fa66415a2f0 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa668e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6640b6a00 0x7fa66415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa668e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6640b6a00 0x7fa66415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59408/0 (socket says 172.21.15.19:59408) 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa668e00640 1 -- 172.21.15.19:0/617695868 learned_addr learned my addr 172.21.15.19:0/617695868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa669800640 1 --1- 172.21.15.19:0/617695868 >> v1:172.21.15.19:6789/0 conn(0x7fa66406d390 0x7fa66406f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56638/0 (socket says 172.21.15.19:56638) 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.678+0000 7fa663e00640 1 --2- 172.21.15.19:0/617695868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66415a2f0 0x7fa66406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.679+0000 7fa668e00640 1 -- 172.21.15.19:0/617695868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66415a2f0 msgr2=0x7fa66406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.679+0000 7fa668e00640 1 --2- 172.21.15.19:0/617695868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66415a2f0 0x7fa66406ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.679+0000 7fa668e00640 1 -- 172.21.15.19:0/617695868 >> v1:172.21.15.19:6789/0 conn(0x7fa66406d390 legacy=0x7fa66406f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.679+0000 7fa668e00640 1 -- 172.21.15.19:0/617695868 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66406d7a0 con 0x7fa6640b6a00 2024-04-25T11:36:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.679+0000 7fa663400640 1 -- 172.21.15.19:0/617695868 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 927232637 0 0) 0x7fa6641611e0 con 0x7fa66406d390 2024-04-25T11:36:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.680+0000 7fa668e00640 1 --2- 172.21.15.19:0/617695868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa66415d6c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa65800bb00 tx=0x7fa65802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33a291f91ddc6a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.680+0000 7fa663400640 1 -- 172.21.15.19:0/617695868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65803a070 con 0x7fa6640b6a00 2024-04-25T11:36:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.680+0000 7fa663400640 1 -- 172.21.15.19:0/617695868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa658032480 con 0x7fa6640b6a00 2024-04-25T11:36:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.680+0000 7fa663400640 1 -- 172.21.15.19:0/617695868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658032850 con 0x7fa6640b6a00 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 -- 172.21.15.19:0/617695868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 msgr2=0x7fa66415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 --2- 172.21.15.19:0/617695868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa66415d6c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fa65800bb00 tx=0x7fa65802c6e0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 -- 172.21.15.19:0/617695868 shutdown_connections 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 --2- 172.21.15.19:0/617695868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66415a2f0 0x7fa66406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 --2- 172.21.15.19:0/617695868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa66415d6c0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 -- 172.21.15.19:0/617695868 >> 172.21.15.19:0/617695868 conn(0x7fa6640d1ae0 msgr2=0x7fa6640d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 -- 172.21.15.19:0/617695868 shutdown_connections 2024-04-25T11:36:07.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.681+0000 7fa66b800640 1 -- 172.21.15.19:0/617695868 wait complete. 2024-04-25T11:36:07.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.684+0000 7fa66b800640 1 Processor -- start 2024-04-25T11:36:07.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.684+0000 7fa66b800640 1 -- start start 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.685+0000 7fa66b800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66406d390 0x7fa66414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.685+0000 7fa66b800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa664157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.685+0000 7fa66b800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa66415a2f0 0x7fa664158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.685+0000 7fa66b800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641609e0 con 0x7fa6640b6a00 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.685+0000 7fa66b800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa664160700 con 0x7fa66415a2f0 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.685+0000 7fa66b800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa6640b5980 con 0x7fa66406d390 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa664157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:07.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa664157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59416/0 (socket says 172.21.15.19:59416) 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 -- 172.21.15.19:0/3346279496 learned_addr learned my addr 172.21.15.19:0/3346279496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa668e00640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66406d390 0x7fa66414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa669800640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa66415a2f0 0x7fa664158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 -- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66406d390 msgr2=0x7fa66414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66406d390 0x7fa66414f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 -- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa66415a2f0 msgr2=0x7fa664158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa66415a2f0 0x7fa664158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.686+0000 7fa663e00640 1 -- 172.21.15.19:0/3346279496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa664158d60 con 0x7fa6640b6a00 2024-04-25T11:36:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.687+0000 7fa663e00640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa664157fc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa65000d620 tx=0x7fa65000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.687+0000 7fa660c00640 1 -- 172.21.15.19:0/3346279496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65000b620 con 0x7fa6640b6a00 2024-04-25T11:36:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.687+0000 7fa660c00640 1 -- 172.21.15.19:0/3346279496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa650019070 con 0x7fa6640b6a00 2024-04-25T11:36:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.687+0000 7fa660c00640 1 -- 172.21.15.19:0/3346279496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65000ba00 con 0x7fa6640b6a00 2024-04-25T11:36:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.687+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa664155ef0 con 0x7fa6640b6a00 2024-04-25T11:36:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.687+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66414f8b0 con 0x7fa6640b6a00 2024-04-25T11:36:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.688+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa664050740 con 0x7fa6640b6a00 2024-04-25T11:36:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.690+0000 7fa660c00640 1 -- 172.21.15.19:0/3346279496 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa65000bba0 con 0x7fa6640b6a00 2024-04-25T11:36:07.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.691+0000 7fa660c00640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6440744b0 0x7fa6440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:07.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.691+0000 7fa660c00640 1 -- 172.21.15.19:0/3346279496 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7fa650095770 con 0x7fa6640b6a00 2024-04-25T11:36:07.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.691+0000 7fa668e00640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6440744b0 0x7fa6440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:07.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.693+0000 7fa668e00640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6440744b0 0x7fa6440769c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa658037000 tx=0x7fa658030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:07.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.696+0000 7fa660c00640 1 -- 172.21.15.19:0/3346279496 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa65005fdf0 con 0x7fa6640b6a00 2024-04-25T11:36:07.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:07.973+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa664158af0 con 0x7fa6640b6a00 2024-04-25T11:36:08.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.356+0000 7fa660c00640 1 -- 172.21.15.19:0/3346279496 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v110) ==== 101+0+0 (secure 0 0 0) 0x7fa65005f6f0 con 0x7fa6640b6a00 2024-04-25T11:36:08.345 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:36:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6440744b0 msgr2=0x7fa6440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6440744b0 0x7fa6440769c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa658037000 tx=0x7fa658030000 comp rx=0 tx=0).stop 2024-04-25T11:36:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 msgr2=0x7fa664157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa664157fc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa65000d620 tx=0x7fa65000dae0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 shutdown_connections 2024-04-25T11:36:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa6440744b0 0x7fa6440769c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa66415a2f0 0x7fa664158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6640b6a00 0x7fa664157fc0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 --2- 172.21.15.19:0/3346279496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa66406d390 0x7fa66414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 >> 172.21.15.19:0/3346279496 conn(0x7fa6640d1ae0 msgr2=0x7fa6640b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 shutdown_connections 2024-04-25T11:36:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.359+0000 7fa66b800640 1 -- 172.21.15.19:0/3346279496 wait complete. 2024-04-25T11:36:08.380 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:36:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.577+0000 7f4f6fd64f40 1 Processor -- start 2024-04-25T11:36:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.578+0000 7f4f6fd64f40 1 -- start start 2024-04-25T11:36:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f6fd64f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086666d6c0 0x56086672fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f6fd64f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560866699180 0x5608667353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f6fd64f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56086669a180 0x560866735930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f6fd64f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608666b2ce0 con 0x56086669a180 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f6fd64f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560866610d20 con 0x560866699180 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f6fd64f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5608666b32a0 con 0x56086666d6c0 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f69800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56086669a180 0x560866735930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f69800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56086669a180 0x560866735930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59438/0 (socket says 172.21.15.19:59438) 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.579+0000 7f4f69800640 1 -- 172.21.15.19:0/475880910 learned_addr learned my addr 172.21.15.19:0/475880910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.580+0000 7f4f69800640 1 -- 172.21.15.19:0/475880910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086666d6c0 msgr2=0x56086672fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.580+0000 7f4f69800640 1 --2- 172.21.15.19:0/475880910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086666d6c0 0x56086672fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.580+0000 7f4f69800640 1 -- 172.21.15.19:0/475880910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560866699180 msgr2=0x5608667353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.580+0000 7f4f69800640 1 --2- 172.21.15.19:0/475880910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560866699180 0x5608667353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.580+0000 7f4f69800640 1 -- 172.21.15.19:0/475880910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608666b0a00 con 0x56086669a180 2024-04-25T11:36:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.580+0000 7f4f69800640 1 --2- 172.21.15.19:0/475880910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56086669a180 0x560866735930 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4f60025da0 tx=0x7f4f60051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac9df608ba95086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f67000640 1 -- 172.21.15.19:0/475880910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60069070 con 0x56086669a180 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f67000640 1 -- 172.21.15.19:0/475880910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f60052b10 con 0x56086669a180 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f67000640 1 -- 172.21.15.19:0/475880910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60056070 con 0x56086669a180 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/475880910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56086669a180 msgr2=0x560866735930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/475880910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56086669a180 0x560866735930 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4f60025da0 tx=0x7f4f60051000 comp rx=0 tx=0).stop 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/475880910 shutdown_connections 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/475880910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560866699180 0x5608667353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/475880910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56086669a180 0x560866735930 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/475880910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086666d6c0 0x56086672fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/475880910 >> 172.21.15.19:0/475880910 conn(0x56086658cac0 msgr2=0x560866695660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/475880910 shutdown_connections 2024-04-25T11:36:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.581+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/475880910 wait complete. 2024-04-25T11:36:08.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.583+0000 7f4f6fd64f40 1 Processor -- start 2024-04-25T11:36:08.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.583+0000 7f4f6fd64f40 1 -- start start 2024-04-25T11:36:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f6fd64f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56086666d6c0 0x56086672f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f6fd64f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560866699180 0x56086672a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f6fd64f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086669a180 0x560866725560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f6fd64f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56086673fe30 con 0x560866699180 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f6fd64f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5608667d3630 con 0x56086666d6c0 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f6fd64f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560866725ed0 con 0x56086669a180 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f68400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560866699180 0x56086672a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f68400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560866699180 0x56086672a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59448/0 (socket says 172.21.15.19:59448) 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.584+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 learned_addr learned my addr 172.21.15.19:0/1703778477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f69800640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086669a180 0x560866725560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086669a180 msgr2=0x560866725560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f68400640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56086669a180 0x560866725560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f68e00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56086666d6c0 0x56086672f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56086666d6c0 msgr2=0x56086672f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f68400640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56086666d6c0 0x56086672f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086673f3f0 con 0x560866699180 2024-04-25T11:36:08.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.585+0000 7f4f68400640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560866699180 0x56086672a350 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000bd40 tx=0x7f4f50009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.586+0000 7f4f65c00640 1 -- 172.21.15.19:0/1703778477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50008070 con 0x560866699180 2024-04-25T11:36:08.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.586+0000 7f4f65c00640 1 -- 172.21.15.19:0/1703778477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5000f650 con 0x560866699180 2024-04-25T11:36:08.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.586+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56086673f6d0 con 0x560866699180 2024-04-25T11:36:08.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.586+0000 7f4f65c00640 1 -- 172.21.15.19:0/1703778477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5000fa80 con 0x560866699180 2024-04-25T11:36:08.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.586+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560866725c10 con 0x560866699180 2024-04-25T11:36:08.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.588+0000 7f4f65c00640 1 -- 172.21.15.19:0/1703778477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f5000fc20 con 0x560866699180 2024-04-25T11:36:08.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.589+0000 7f4f65c00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f44072e90 0x7f4f440753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.589+0000 7f4f65c00640 1 -- 172.21.15.19:0/1703778477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7f4f50094030 con 0x560866699180 2024-04-25T11:36:08.578 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:36:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.590+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5608667d28d0 0x5608667d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.590+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e3c80 con 0x5608667d28d0 2024-04-25T11:36:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.590+0000 7f4f68e00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f44072e90 0x7f4f440753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.590+0000 7f4f69800640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5608667d28d0 0x5608667d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.591+0000 7f4f68e00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f44072e90 0x7f4f440753a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x560866656f70 tx=0x7f4f58007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.592+0000 7f4f69800640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5608667d28d0 0x5608667d2ce0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.595+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v95'73 uv48 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.595+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667e4020 con 0x5608667d28d0 2024-04-25T11:36:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.598+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v95'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.598+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e43c0 con 0x5608667d28d0 2024-04-25T11:36:08.590 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:36:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.601+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v95'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.601+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e5770 con 0x5608667d28d0 2024-04-25T11:36:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.604+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v95'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.604+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e5b10 con 0x5608667d28d0 2024-04-25T11:36:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.607+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v95'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.608+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4760 con 0x5608667d28d0 2024-04-25T11:36:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.611+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v95'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.611+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4b50 con 0x5608667d28d0 2024-04-25T11:36:08.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.613+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v95'79 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.613+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.617+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v95'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.617+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.608 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:36:08.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.620+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v95'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.620+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.623+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v95'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.623+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.626+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v95'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.626+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.630+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v95'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.630+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.633+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v95'85 uv59 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.633+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.637+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v95'86 uv86 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.637+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.629 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:36:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.640+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v95'87 uv87 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.640+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.643+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v95'88 uv88 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.643+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.647+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v95'89 uv89 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.647+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e4ef0 con 0x5608667d28d0 2024-04-25T11:36:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.650+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v95'90 uv90 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.651+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e5310 con 0x5608667d28d0 2024-04-25T11:36:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.654+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v95'91 uv91 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.654+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e5310 con 0x5608667d28d0 2024-04-25T11:36:08.646 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:36:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.657+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v95'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.657+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.661+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v95'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.661+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.665+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v95'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.665+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.669+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v95'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.669+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.672+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 603.00000000 [create] v95'96 uv71 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.672+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.677+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 603.00000000 [delete] v95'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.677+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.681+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v95'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.669 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:36:08.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.681+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.685+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v95'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.685+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.688+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v95'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.688+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e51b0 con 0x5608667d28d0 2024-04-25T11:36:08.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.693+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v95'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.693+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608667e5eb0 0x5608667e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.693+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:30 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e8810 con 0x5608667e5eb0 2024-04-25T11:36:08.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.694+0000 7f4f68e00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608667e5eb0 0x5608667e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.695+0000 7f4f68e00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608667e5eb0 0x5608667e82d0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.699+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(30 1.00000000.inode [create] v95'67 uv49 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.699+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:31 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667e8bb0 con 0x5608667e5eb0 2024-04-25T11:36:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.703+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(31 1.00000000.inode [delete] v95'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.704+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e8f50 con 0x5608667e5eb0 2024-04-25T11:36:08.696 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:36:08.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.707+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(32 1.00000000.inode [create] v95'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.707+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e92f0 con 0x5608667e5eb0 2024-04-25T11:36:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.712+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v95'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.713+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e9690 con 0x5608667e5eb0 2024-04-25T11:36:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.717+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v95'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.717+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667e9a30 con 0x5608667e5eb0 2024-04-25T11:36:08.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.720+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(35 200.00000000 [create] v95'72 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.720+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667e9dd0 con 0x5608667e5eb0 2024-04-25T11:36:08.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.724+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(36 200.00000000 [delete] v95'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.724+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.717 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:36:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.729+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(37 200.00000000 [create] v95'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.729+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.733+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v95'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.734+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.738+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v95'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.738+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.742+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(40 200.00000001 [create] v95'77 uv59 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.742+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.746+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(41 200.00000001 [delete] v95'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.746+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.750+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(42 200.00000001 [create] v95'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.739 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:36:08.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.750+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ee860 con 0x5608667e5eb0 2024-04-25T11:36:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.755+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(43 200.00000001 [write 0~18117] v95'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.755+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ee860 con 0x5608667e5eb0 2024-04-25T11:36:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.760+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v95'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.760+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.764+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(45 606.00000000 [create] v95'82 uv65 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.764+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.768+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(46 606.00000000 [delete] v95'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.768+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.761 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:36:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.772+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(47 606.00000000 [create] v95'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.772+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667eb180 con 0x5608667e5eb0 2024-04-25T11:36:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.777+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v95'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.777+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667eb180 con 0x5608667e5eb0 2024-04-25T11:36:08.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.781+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v95'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.781+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ea170 con 0x5608667e5eb0 2024-04-25T11:36:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.785+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v95'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667e5eb0 2024-04-25T11:36:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.785+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608667ea680 0x5608667ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.785+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ecfe0 con 0x5608667ea680 2024-04-25T11:36:08.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.786+0000 7f4f68400640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608667ea680 0x5608667ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.788+0000 7f4f68400640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608667ea680 0x5608667ecaa0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.792+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(51 607.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.792+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667ed380 con 0x5608667ea680 2024-04-25T11:36:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.796+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(52 607.00000000 [delete] v95'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.796+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ed760 con 0x5608667ea680 2024-04-25T11:36:08.789 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:36:08.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.800+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(53 607.00000000 [create] v95'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.801+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ef010 con 0x5608667ea680 2024-04-25T11:36:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.805+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.805+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ef3b0 con 0x5608667ea680 2024-04-25T11:36:08.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.809+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.809+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ee000 con 0x5608667ea680 2024-04-25T11:36:08.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.813+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.813+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ee000 con 0x5608667d28d0 2024-04-25T11:36:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.818+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(57 mds0_openfiles.0 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.818+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667ee000 con 0x5608667d28d0 2024-04-25T11:36:08.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.822+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v95'16 uv16 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.822+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ee000 con 0x5608667d28d0 2024-04-25T11:36:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.827+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v95'17 uv17 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.816 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:36:08.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.827+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667ee000 con 0x5608667d28d0 2024-04-25T11:36:08.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.832+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v95'18 uv18 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.832+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608667ee950 0x5608667f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.832+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f1ca0 con 0x5608667ee950 2024-04-25T11:36:08.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.833+0000 7f4f69800640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608667ee950 0x5608667f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.835+0000 7f4f69800640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608667ee950 0x5608667f1760 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.839+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(61 608.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.839+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667f1ca0 con 0x5608667ee950 2024-04-25T11:36:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.842+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(62 608.00000000 [delete] v95'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.842+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f1ca0 con 0x5608667ee950 2024-04-25T11:36:08.835 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:36:08.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.846+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(63 608.00000000 [create] v95'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.846+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f2cb0 con 0x5608667ee950 2024-04-25T11:36:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.851+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.851+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f2cb0 con 0x5608667ee950 2024-04-25T11:36:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.855+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.855+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f1ca0 con 0x5608667ee950 2024-04-25T11:36:08.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.859+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.859+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608667f2320 0x5608667f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.859+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f4da0 con 0x5608667f2320 2024-04-25T11:36:08.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.860+0000 7f4f68e00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608667f2320 0x5608667f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.862+0000 7f4f68e00640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608667f2320 0x5608667f4740 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.866+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(67 604.00000000 [create] v95'26 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.866+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.870+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(68 604.00000000 [delete] v95'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.871+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.875+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(69 604.00000000 [create] v95'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.863 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:36:08.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.875+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f6180 con 0x5608667f2320 2024-04-25T11:36:08.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.879+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v95'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.879+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f6180 con 0x5608667f2320 2024-04-25T11:36:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.883+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v95'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.883+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.887+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v95'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.888+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.891+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(73 500.00000000 [create] v95'32 uv25 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.891+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.895+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(74 500.00000000 [delete] v95'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.895+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.899+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(75 500.00000000 [create] v95'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.889 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:36:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.900+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.904+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v95'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.904+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f5170 con 0x5608667f2320 2024-04-25T11:36:08.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.908+0000 7f4f68e00640 1 -- 172.21.15.19:0/1703778477 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v95'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f58013730 con 0x5608667f2320 2024-04-25T11:36:08.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.908+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608667f57f0 0x5608667f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:08.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.908+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8370 con 0x5608667f57f0 2024-04-25T11:36:08.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.909+0000 7f4f68400640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608667f57f0 0x5608667f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.910+0000 7f4f68400640 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608667f57f0 0x5608667f7c70 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.914+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(78 mds_snaptable [create] v95'12 uv11 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.914+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667f8370 con 0x5608667f57f0 2024-04-25T11:36:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.919+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(79 mds_snaptable [delete] v95'13 uv13 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.919+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667f57f0 2024-04-25T11:36:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:36:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.923+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(80 mds_snaptable [create] v95'14 uv14 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.923+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667f57f0 2024-04-25T11:36:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.928+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v95'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.928+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667f57f0 2024-04-25T11:36:08.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.932+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v95'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:08.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.932+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667d28d0 2024-04-25T11:36:08.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.936+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(83 605.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.936+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667d28d0 2024-04-25T11:36:08.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.940+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(84 605.00000000 [delete] v95'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.940+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667d28d0 2024-04-25T11:36:08.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.944+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v95'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.933 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:36:08.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.944+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f97a0 con 0x5608667d28d0 2024-04-25T11:36:08.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.948+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.949+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f97a0 con 0x5608667d28d0 2024-04-25T11:36:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.952+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.952+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667d28d0 2024-04-25T11:36:08.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.956+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667d28d0 2024-04-25T11:36:08.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.956+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.960+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(89 mds0_inotable [create] v95'24 uv18 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.960+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.964+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(90 mds0_inotable [delete] v95'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.964+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.957 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:36:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.968+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(91 mds0_inotable [create] v95'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.968+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.973+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v95'27 uv27 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.973+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.977+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v95'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.977+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.981+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(94 100.00000000 [create] v95'29 uv23 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.981+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.985+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(95 100.00000000 [delete] v95'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.985+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.988+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(96 100.00000000 [create] v95'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.977 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:36:08.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.988+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8790 con 0x5608667ea680 2024-04-25T11:36:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.992+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v95'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.992+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb5d0 con 0x5608667ea680 2024-04-25T11:36:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.997+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v95'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667ea680 2024-04-25T11:36:08.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:08.997+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:08.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.001+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v95'14 uv12 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.001+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:08.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.007+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v95'15 uv15 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:08.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.007+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:08.999 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:36:09.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.011+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v95'16 uv16 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.011+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:09.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.016+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v95'17 uv17 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.016+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.019+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v95'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.019+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:09.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.023+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v95'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.023+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:09.016 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:36:09.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.028+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v95'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.028+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:09.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.031+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v95'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.032+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.035+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v95'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.035+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667ee950 2024-04-25T11:36:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.039+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v95'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f6006c040 con 0x5608667ee950 2024-04-25T11:36:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.040+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.043+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v95'12 uv11 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.043+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.048+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v95'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.048+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.051+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v95'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.040 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:36:09.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.051+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.055+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v95'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.055+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.059+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v95'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.059+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.063+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v95'33 uv24 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.063+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.067+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v95'34 uv34 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.068+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.060 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:36:09.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.071+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v95'35 uv35 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.071+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.076+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v95'36 uv36 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.076+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.080+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v95'37 uv37 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.080+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.084+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v95'38 uv31 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.084+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.088+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v95'39 uv39 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.088+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.081 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:36:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.092+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v95'40 uv40 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.092+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.096+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v95'41 uv41 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.096+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.101+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v95'42 uv42 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.101+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667fb590 con 0x5608667f57f0 2024-04-25T11:36:09.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.105+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v95'43 uv43 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.105+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5608667f8f50 con 0x5608667f57f0 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f68400640 1 -- 172.21.15.19:0/1703778477 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v95'44 uv44 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4f5005a2d0 con 0x5608667f57f0 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608667e5eb0 msgr2=0x5608667e82d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608667e5eb0 0x5608667e82d0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608667f2320 msgr2=0x5608667f4740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608667f2320 0x5608667f4740 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608667f57f0 msgr2=0x5608667f7c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608667f57f0 0x5608667f7c70 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608667ea680 msgr2=0x5608667ecaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608667ea680 0x5608667ecaa0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 reap_dead start 2024-04-25T11:36:09.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5608667d28d0 msgr2=0x5608667d2ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5608667d28d0 0x5608667d2ce0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608667ee950 msgr2=0x5608667f1760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f69800640 1 -- 172.21.15.19:0/1703778477 reap_dead start 2024-04-25T11:36:09.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.109+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608667ee950 0x5608667f1760 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f44072e90 msgr2=0x7f4f440753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f44072e90 0x7f4f440753a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x560866656f70 tx=0x7f4f58007040 comp rx=0 tx=0).stop 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560866699180 msgr2=0x56086672a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560866699180 0x56086672a350 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000bd40 tx=0x7f4f50009560 comp rx=0 tx=0).stop 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 shutdown_connections 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608667ee950 0x5608667f1760 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f44072e90 0x7f4f440753a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 --2- 172.21.15.19:0/1703778477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560866699180 0x56086672a350 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 >> 172.21.15.19:0/1703778477 conn(0x56086658cac0 msgr2=0x5608666afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.110+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 shutdown_connections 2024-04-25T11:36:09.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.111+0000 7f4f6fd64f40 1 -- 172.21.15.19:0/1703778477 wait complete. 2024-04-25T11:36:09.117 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2024-04-25T11:36:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.358+0000 7f46b948cf40 1 Processor -- start 2024-04-25T11:36:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.358+0000 7f46b948cf40 1 -- start start 2024-04-25T11:36:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b948cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a613710710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b948cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a613678d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b948cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 0x55a613645560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b948cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a6135edd20 con 0x7f4698003b40 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b948cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6135f1d60 con 0x55a6136783e0 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b948cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a6136903f0 con 0x55a613677fd0 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a613710710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a613710710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59474/0 (socket says 172.21.15.19:59474) 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b2600640 1 -- 172.21.15.19:0/2086512183 learned_addr learned my addr 172.21.15.19:0/2086512183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b3000640 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 0x55a613645560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b1c00640 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a613678d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b2600640 1 -- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 msgr2=0x55a613678d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.360+0000 7f46b2600640 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a613678d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46b2600640 1 -- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 msgr2=0x55a613645560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46b2600640 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 0x55a613645560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46b2600640 1 -- 172.21.15.19:0/2086512183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a613710c50 con 0x7f4698003b40 2024-04-25T11:36:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46b1c00640 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a613678d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:09.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46b2600640 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a613710710 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f46a4001690 tx=0x7f46a4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d93a6017aa31db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46abe00640 1 -- 172.21.15.19:0/2086512183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4018070 con 0x7f4698003b40 2024-04-25T11:36:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46abe00640 1 -- 172.21.15.19:0/2086512183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a400e700 con 0x7f4698003b40 2024-04-25T11:36:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.361+0000 7f46abe00640 1 -- 172.21.15.19:0/2086512183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a400eb10 con 0x7f4698003b40 2024-04-25T11:36:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 -- 172.21.15.19:0/2086512183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 msgr2=0x55a613710710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a613710710 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f46a4001690 tx=0x7f46a4001b50 comp rx=0 tx=0).stop 2024-04-25T11:36:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 -- 172.21.15.19:0/2086512183 shutdown_connections 2024-04-25T11:36:09.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 0x55a613645560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a613678d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 --2- 172.21.15.19:0/2086512183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a613710710 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 -- 172.21.15.19:0/2086512183 >> 172.21.15.19:0/2086512183 conn(0x55a613569ac0 msgr2=0x55a613646810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 -- 172.21.15.19:0/2086512183 shutdown_connections 2024-04-25T11:36:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.362+0000 7f46b948cf40 1 -- 172.21.15.19:0/2086512183 wait complete. 2024-04-25T11:36:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.363+0000 7f46b948cf40 1 Processor -- start 2024-04-25T11:36:09.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.363+0000 7f46b948cf40 1 -- start start 2024-04-25T11:36:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b948cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a61370cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b948cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 0x55a61370b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b948cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a61370d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b948cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a6137173f0 con 0x7f4698003b40 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b948cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a6137176d0 con 0x55a6136783e0 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b948cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a613677ab0 con 0x55a613677fd0 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a61370d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a61370d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59478/0 (socket says 172.21.15.19:59478) 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.365+0000 7f46b3000640 1 -- 172.21.15.19:0/1596705442 learned_addr learned my addr 172.21.15.19:0/1596705442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b2600640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a61370cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b1c00640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 0x55a61370b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b3000640 1 -- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 msgr2=0x55a61370cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b3000640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a613677fd0 0x55a61370cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b3000640 1 -- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 msgr2=0x55a61370b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b3000640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a6136783e0 0x55a61370b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b3000640 1 -- 172.21.15.19:0/1596705442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a61370bcb0 con 0x7f4698003b40 2024-04-25T11:36:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b3000640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a61370d550 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f46ac025e00 tx=0x7f46ac003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46aaa00640 1 -- 172.21.15.19:0/1596705442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac009410 con 0x7f4698003b40 2024-04-25T11:36:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46aaa00640 1 -- 172.21.15.19:0/1596705442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac04f4b0 con 0x7f4698003b40 2024-04-25T11:36:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46aaa00640 1 -- 172.21.15.19:0/1596705442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac04faa0 con 0x7f4698003b40 2024-04-25T11:36:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.366+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a61370bf90 con 0x7f4698003b40 2024-04-25T11:36:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.367+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a61370c550 con 0x7f4698003b40 2024-04-25T11:36:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.372+0000 7f46aaa00640 1 -- 172.21.15.19:0/1596705442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46ac062030 con 0x7f4698003b40 2024-04-25T11:36:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.373+0000 7f46aaa00640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4690072ec0 0x7f46900753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.373+0000 7f46aaa00640 1 -- 172.21.15.19:0/1596705442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7f46ac0d6bf0 con 0x7f4698003b40 2024-04-25T11:36:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.374+0000 7f46b948cf40 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a613679260 0x55a613714000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.374+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x55a613714540 con 0x55a613679260 2024-04-25T11:36:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.374+0000 7f46b2600640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4690072ec0 0x7f46900753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.374+0000 7f46b1c00640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a613679260 0x55a613714000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.375+0000 7f46b1c00640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a613679260 0x55a613714000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.375+0000 7f46b2600640 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4690072ec0 0x7f46900753d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f46a4001690 tx=0x7f46a4014040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b1c00640 1 -- 172.21.15.19:0/1596705442 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 1.00000000.inode [delete] v95'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f469c009330 con 0x55a613679260 2024-04-25T11:36:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a613679260 msgr2=0x55a613714000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a613679260 0x55a613714000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4690072ec0 msgr2=0x7f46900753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4690072ec0 0x7f46900753d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f46a4001690 tx=0x7f46a4014040 comp rx=0 tx=0).stop 2024-04-25T11:36:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 msgr2=0x55a61370d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 --2- 172.21.15.19:0/1596705442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4698003b40 0x55a61370d550 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f46ac025e00 tx=0x7f46ac003020 comp rx=0 tx=0).stop 2024-04-25T11:36:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b3000640 1 -- 172.21.15.19:0/1596705442 reap_dead start 2024-04-25T11:36:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 shutdown_connections 2024-04-25T11:36:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 >> 172.21.15.19:0/1596705442 conn(0x55a613569ac0 msgr2=0x55a6137a67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 shutdown_connections 2024-04-25T11:36:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.378+0000 7f46b948cf40 1 -- 172.21.15.19:0/1596705442 wait complete. 2024-04-25T11:36:09.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:36:09.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.683+0000 7f8dc2c00640 1 Processor -- start 2024-04-25T11:36:09.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.683+0000 7f8dc2c00640 1 -- start start 2024-04-25T11:36:09.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.685+0000 7f8dc2c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8dbc06dce0 0x7f8dbc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.685+0000 7f8dc2c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dbc0cdea0 con 0x7f8dbc0b0af0 2024-04-25T11:36:09.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.686+0000 7f8dc2c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dbc0ce180 con 0x7f8dbc0cd9a0 2024-04-25T11:36:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.686+0000 7f8dc2c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8dbc0ceab0 con 0x7f8dbc06dce0 2024-04-25T11:36:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.686+0000 7f8dc0c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8dbc0b0af0 0x7f8dbc15faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56664/0 (socket says 172.21.15.19:56664) 2024-04-25T11:36:09.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.686+0000 7f8dc0c00640 1 -- 172.21.15.19:0/2626622344 learned_addr learned my addr 172.21.15.19:0/2626622344 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.686+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2626622344 >> v2:172.21.15.169:3300/0 conn(0x7f8dbc06dce0 0x7f8dbc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.686+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 402962106 0 0) 0x7f8dbc0cdea0 con 0x7f8dbc0b0af0 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.686+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8da0003520 con 0x7f8dbc0b0af0 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382795069 0 0) 0x7f8da0003520 con 0x7f8dbc0b0af0 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 >> v2:172.21.15.169:3300/0 conn(0x7f8dbc06dce0 msgr2=0x7f8dbc15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 --2- 172.21.15.19:0/2626622344 >> v2:172.21.15.169:3300/0 conn(0x7f8dbc06dce0 0x7f8dbc15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 >> v1:172.21.15.169:6789/0 conn(0x7f8dbc0cd9a0 legacy=0x7f8dbc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc0ce460 con 0x7f8dbc0b0af0 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f8db40019f0 con 0x7f8dbc0b0af0 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 >> v1:172.21.15.19:6789/0 conn(0x7f8dbc0b0af0 legacy=0x7f8dbc15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da00039b0 0x7f8da0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0004800 0x7f8da0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da00051c0 0x7f8da000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc0ceab0 con 0x7f8da0004800 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8da000c160 con 0x7f8da00039b0 2024-04-25T11:36:09.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8da000c360 con 0x7f8da00051c0 2024-04-25T11:36:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.687+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0004800 0x7f8da0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.688+0000 7f8db3e00640 1 -- 172.21.15.19:0/2626622344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da00051c0 msgr2=0x7f8da000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.688+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da00051c0 0x7f8da000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.688+0000 7f8db3e00640 1 -- 172.21.15.19:0/2626622344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da00039b0 msgr2=0x7f8da0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.688+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da00039b0 0x7f8da0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.688+0000 7f8db3e00640 1 -- 172.21.15.19:0/2626622344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da0003520 con 0x7f8da0004800 2024-04-25T11:36:09.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.688+0000 7f8dc0c00640 1 -- 172.21.15.19:0/2626622344 reap_dead start 2024-04-25T11:36:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.689+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0004800 0x7f8da0004c10 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009ac0 tx=0x7f8da0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e6063f7db2a3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.689+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4036070 con 0x7f8da0004800 2024-04-25T11:36:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.689+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da4031400 con 0x7f8da0004800 2024-04-25T11:36:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.689+0000 7f8dbb400640 1 -- 172.21.15.19:0/2626622344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4009690 con 0x7f8da0004800 2024-04-25T11:36:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.690+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2626622344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0004800 msgr2=0x7f8da0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.690+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0004800 0x7f8da0004c10 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009ac0 tx=0x7f8da0005830 comp rx=0 tx=0).stop 2024-04-25T11:36:09.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.690+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2626622344 shutdown_connections 2024-04-25T11:36:09.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.690+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2626622344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8da0004800 0x7f8da0004c10 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.690+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2626622344 >> 172.21.15.19:0/2626622344 conn(0x7f8dbc073090 msgr2=0x7f8dbc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:09.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.690+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2626622344 shutdown_connections 2024-04-25T11:36:09.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.690+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2626622344 wait complete. 2024-04-25T11:36:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.693+0000 7f8dc2c00640 1 Processor -- start 2024-04-25T11:36:09.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.693+0000 7f8dc2c00640 1 -- start start 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.694+0000 7f8dc2c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da0004800 0x7f8dbc1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.694+0000 7f8dc2c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dbc142600 0x7f8dbc15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.694+0000 7f8dc2c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc15dcc0 0x7f8dbc159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.694+0000 7f8dc2c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc0cef40 con 0x7f8dbc142600 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.694+0000 7f8dc2c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc0b0f00 con 0x7f8dbc15dcc0 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.694+0000 7f8dc2c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8dbc0cf0c0 con 0x7f8da0004800 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dbc142600 0x7f8dbc15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dbc142600 0x7f8dbc15d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59486/0 (socket says 172.21.15.19:59486) 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 -- 172.21.15.19:0/2697191012 learned_addr learned my addr 172.21.15.19:0/2697191012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8dbbe00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da0004800 0x7f8dbc1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8dc0c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc15dcc0 0x7f8dbc159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 -- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da0004800 msgr2=0x7f8dbc1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da0004800 0x7f8dbc1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 -- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc15dcc0 msgr2=0x7f8dbc159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc15dcc0 0x7f8dbc159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8db3e00640 1 -- 172.21.15.19:0/2697191012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc15e0d0 con 0x7f8dbc142600 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.695+0000 7f8dc0c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc15dcc0 0x7f8dbc159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.696+0000 7f8db3e00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dbc142600 0x7f8dbc15d780 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009be0 tx=0x7f8da4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.696+0000 7f8db8c00640 1 -- 172.21.15.19:0/2697191012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da403f020 con 0x7f8dbc142600 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.696+0000 7f8db8c00640 1 -- 172.21.15.19:0/2697191012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da4037040 con 0x7f8dbc142600 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.696+0000 7f8db8c00640 1 -- 172.21.15.19:0/2697191012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4036070 con 0x7f8dbc142600 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.696+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc157910 con 0x7f8dbc142600 2024-04-25T11:36:09.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.696+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc1594d0 con 0x7f8dbc142600 2024-04-25T11:36:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.698+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dbc050740 con 0x7f8dbc142600 2024-04-25T11:36:09.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.699+0000 7f8db8c00640 1 -- 172.21.15.19:0/2697191012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8da4042020 con 0x7f8dbc142600 2024-04-25T11:36:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.700+0000 7f8db8c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d980744e0 0x7f8d980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.700+0000 7f8db8c00640 1 -- 172.21.15.19:0/2697191012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7f8da40b54a0 con 0x7f8dbc142600 2024-04-25T11:36:09.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.701+0000 7f8dbbe00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d980744e0 0x7f8d980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:09.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.702+0000 7f8dbbe00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d980744e0 0x7f8d980769f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8dac007fd0 tx=0x7f8dac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:09.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.705+0000 7f8db8c00640 1 -- 172.21.15.19:0/2697191012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8da4049030 con 0x7f8dbc142600 2024-04-25T11:36:09.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:09.951+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8dbc157660 con 0x7f8dbc142600 2024-04-25T11:36:10.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.384+0000 7f8db8c00640 1 -- 172.21.15.19:0/2697191012 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v111) ==== 162+0+0 (secure 0 0 0) 0x7f8dbc15e0d0 con 0x7f8dbc142600 2024-04-25T11:36:10.373 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:36:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d980744e0 msgr2=0x7f8d980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d980744e0 0x7f8d980769f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8dac007fd0 tx=0x7f8dac009040 comp rx=0 tx=0).stop 2024-04-25T11:36:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dbc142600 msgr2=0x7f8dbc15d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dbc142600 0x7f8dbc15d780 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009be0 tx=0x7f8da4004730 comp rx=0 tx=0).stop 2024-04-25T11:36:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 shutdown_connections 2024-04-25T11:36:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d980744e0 0x7f8d980769f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dbc15dcc0 0x7f8dbc159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8dbc142600 0x7f8dbc15d780 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 --2- 172.21.15.19:0/2697191012 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da0004800 0x7f8dbc1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.388+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 >> 172.21.15.19:0/2697191012 conn(0x7f8dbc073090 msgr2=0x7f8dbc143320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.389+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 shutdown_connections 2024-04-25T11:36:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.389+0000 7f8dc2c00640 1 -- 172.21.15.19:0/2697191012 wait complete. 2024-04-25T11:36:10.393 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:10.405+0000 7f60cce00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-25T11:36:10.393 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:10.405+0000 7f60cce00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:36:10.394 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:10.405+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:36:10.394 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:10.405+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:36:10.404 INFO:tasks.ceph.mds.a.smithi019.stderr: -16> 2024-04-25T11:36:10.405+0000 7f60cce00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-25T11:36:10.404 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:36:10.405+0000 7f60cce00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:36:10.412 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:10.539 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:36:10.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.692+0000 7fbfbbe00640 1 Processor -- start 2024-04-25T11:36:10.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.693+0000 7fbfbbe00640 1 -- start start 2024-04-25T11:36:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.695+0000 7fbfbbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbfbc072190 0x7fbfbc06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.695+0000 7fbfbbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfbc0b1bc0 con 0x7fbfbc15bbe0 2024-04-25T11:36:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.695+0000 7fbfbbe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfbc0b1ea0 con 0x7fbfbc06d510 2024-04-25T11:36:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.695+0000 7fbfbbe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbfbc06cdc0 con 0x7fbfbc072190 2024-04-25T11:36:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.695+0000 7fbfb3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbfbc072190 0x7fbfbc06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.695+0000 7fbfbaa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbfbc15bbe0 0x7fbfbc0d06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56688/0 (socket says 172.21.15.19:56688) 2024-04-25T11:36:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.695+0000 7fbfbaa00640 1 -- 172.21.15.19:0/762121835 learned_addr learned my addr 172.21.15.19:0/762121835 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.696+0000 7fbfba000640 1 -- 172.21.15.19:0/762121835 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167688760 0 0) 0x7fbfbc0b1bc0 con 0x7fbfbc15bbe0 2024-04-25T11:36:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.696+0000 7fbfb3e00640 1 -- 172.21.15.19:0/762121835 >> v1:172.21.15.169:6789/0 conn(0x7fbfbc06d510 legacy=0x7fbfbc15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.696+0000 7fbfb3e00640 1 -- 172.21.15.19:0/762121835 >> v1:172.21.15.19:6789/0 conn(0x7fbfbc15bbe0 legacy=0x7fbfbc0d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.696+0000 7fbfb3e00640 1 -- 172.21.15.19:0/762121835 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc06c5c0 con 0x7fbfbc072190 2024-04-25T11:36:10.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.697+0000 7fbfb3e00640 1 --2- 172.21.15.19:0/762121835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc06cfd0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40097a0 tx=0x7fbfa402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82a7afb793a0fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:10.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfba000640 1 -- 172.21.15.19:0/762121835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4037070 con 0x7fbfbc072190 2024-04-25T11:36:10.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfba000640 1 -- 172.21.15.19:0/762121835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa402ae50 con 0x7fbfbc072190 2024-04-25T11:36:10.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfba000640 1 -- 172.21.15.19:0/762121835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4032660 con 0x7fbfbc072190 2024-04-25T11:36:10.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfbbe00640 1 -- 172.21.15.19:0/762121835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 msgr2=0x7fbfbc06cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/762121835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc06cfd0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40097a0 tx=0x7fbfa402a140 comp rx=0 tx=0).stop 2024-04-25T11:36:10.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfbbe00640 1 -- 172.21.15.19:0/762121835 shutdown_connections 2024-04-25T11:36:10.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/762121835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc06cfd0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.698+0000 7fbfbbe00640 1 -- 172.21.15.19:0/762121835 >> 172.21.15.19:0/762121835 conn(0x7fbfbc070270 msgr2=0x7fbfbc070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:10.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.699+0000 7fbfbbe00640 1 -- 172.21.15.19:0/762121835 shutdown_connections 2024-04-25T11:36:10.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.699+0000 7fbfbbe00640 1 -- 172.21.15.19:0/762121835 wait complete. 2024-04-25T11:36:10.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.701+0000 7fbfbbe00640 1 Processor -- start 2024-04-25T11:36:10.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.701+0000 7fbfbbe00640 1 -- start start 2024-04-25T11:36:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.702+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfbc06d510 0x7fbfbc15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.702+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.702+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfbc15bbe0 0x7fbfbc1f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfbbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc06c930 con 0x7fbfbc072190 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfbbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc0b1970 con 0x7fbfbc15bbe0 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfbbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc06ef30 con 0x7fbfbc06d510 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59530/0 (socket says 172.21.15.19:59530) 2024-04-25T11:36:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 -- 172.21.15.19:0/2953619824 learned_addr learned my addr 172.21.15.19:0/2953619824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:10.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 -- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfbc06d510 msgr2=0x7fbfbc15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfbc06d510 0x7fbfbc15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 -- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfbc15bbe0 msgr2=0x7fbfbc1f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfbb400640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfbc15bbe0 0x7fbfbc1f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfbc15bbe0 0x7fbfbc1f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.703+0000 7fbfb3e00640 1 -- 172.21.15.19:0/2953619824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc15b020 con 0x7fbfbc072190 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.704+0000 7fbfb3e00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc156c00 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098c0 tx=0x7fbfa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.704+0000 7fbfb2a00640 1 -- 172.21.15.19:0/2953619824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4042070 con 0x7fbfbc072190 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.704+0000 7fbfb2a00640 1 -- 172.21.15.19:0/2953619824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa403e670 con 0x7fbfbc072190 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.704+0000 7fbfb2a00640 1 -- 172.21.15.19:0/2953619824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4037040 con 0x7fbfbc072190 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.704+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc157140 con 0x7fbfbc072190 2024-04-25T11:36:10.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.704+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc1591d0 con 0x7fbfbc072190 2024-04-25T11:36:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.705+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfbc050740 con 0x7fbfbc072190 2024-04-25T11:36:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.707+0000 7fbfb2a00640 1 -- 172.21.15.19:0/2953619824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbfa4045070 con 0x7fbfbc072190 2024-04-25T11:36:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.708+0000 7fbfb2a00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf980744b0 0x7fbf980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.708+0000 7fbfb2a00640 1 -- 172.21.15.19:0/2953619824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7fbfa40b5cb0 con 0x7fbfbc072190 2024-04-25T11:36:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.708+0000 7fbfbaa00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf980744b0 0x7fbf980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.710+0000 7fbfbaa00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf980744b0 0x7fbf980769c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbfac007fd0 tx=0x7fbfac007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:10.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.713+0000 7fbfb2a00640 1 -- 172.21.15.19:0/2953619824 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfa4080240 con 0x7fbfbc072190 2024-04-25T11:36:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.961+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfbc158c80 con 0x7fbfbc072190 2024-04-25T11:36:10.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.962+0000 7fbfb2a00640 1 -- 172.21.15.19:0/2953619824 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+4351 (secure 0 0 0) 0x7fbfa407fb40 con 0x7fbfbc072190 2024-04-25T11:36:10.951 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:10.951 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":113,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:10.407286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 113 2024-04-25T11:36:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf980744b0 msgr2=0x7fbf980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf980744b0 0x7fbf980769c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbfac007fd0 tx=0x7fbfac007960 comp rx=0 tx=0).stop 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 msgr2=0x7fbfbc156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc156c00 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098c0 tx=0x7fbfa40046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 shutdown_connections 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbf980744b0 0x7fbf980769c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfbc15bbe0 0x7fbfbc1f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfbc072190 0x7fbfbc156c00 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 --2- 172.21.15.19:0/2953619824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfbc06d510 0x7fbfbc15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.964+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 >> 172.21.15.19:0/2953619824 conn(0x7fbfbc070270 msgr2=0x7fbfbc1605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.965+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 shutdown_connections 2024-04-25T11:36:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:10.965+0000 7fbfbbe00640 1 -- 172.21.15.19:0/2953619824 wait complete. 2024-04-25T11:36:10.982 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:36:10.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.271+0000 7f4423e00640 1 Processor -- start 2024-04-25T11:36:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.272+0000 7f4423e00640 1 -- start start 2024-04-25T11:36:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.274+0000 7f4423e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4424160fe0 0x7f442415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.274+0000 7f4423e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44240b7220 con 0x7f44240b6e10 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.274+0000 7f4423e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44240b74d0 con 0x7f44240cf2f0 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.274+0000 7f4423e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f442406f870 con 0x7f4424160fe0 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.274+0000 7f4423400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4424160fe0 0x7f442415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.274+0000 7f4423400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4424160fe0 0x7f442415a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59546/0 (socket says 172.21.15.19:59546) 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.274+0000 7f4423400640 1 -- 172.21.15.19:0/148444914 learned_addr learned my addr 172.21.15.19:0/148444914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.275+0000 7f4423400640 1 -- 172.21.15.19:0/148444914 >> v1:172.21.15.169:6789/0 conn(0x7f44240cf2f0 legacy=0x7f44240b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.275+0000 7f4423400640 1 -- 172.21.15.19:0/148444914 >> v1:172.21.15.19:6789/0 conn(0x7f44240b6e10 legacy=0x7f44241608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.275+0000 7f4423400640 1 -- 172.21.15.19:0/148444914 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44240b77b0 con 0x7f4424160fe0 2024-04-25T11:36:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.276+0000 7f4423400640 1 --2- 172.21.15.19:0/148444914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424160fe0 0x7f442415a810 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f441c009e50 tx=0x7f441c02c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaa48b71870b19d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.276+0000 7f4422000640 1 -- 172.21.15.19:0/148444914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c032620 con 0x7f4424160fe0 2024-04-25T11:36:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.276+0000 7f4422000640 1 -- 172.21.15.19:0/148444914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441c037070 con 0x7f4424160fe0 2024-04-25T11:36:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.276+0000 7f4422000640 1 -- 172.21.15.19:0/148444914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0329e0 con 0x7f4424160fe0 2024-04-25T11:36:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.277+0000 7f4423e00640 1 -- 172.21.15.19:0/148444914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424160fe0 msgr2=0x7f442415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.277+0000 7f4423e00640 1 --2- 172.21.15.19:0/148444914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424160fe0 0x7f442415a810 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f441c009e50 tx=0x7f441c02c970 comp rx=0 tx=0).stop 2024-04-25T11:36:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.277+0000 7f4423e00640 1 -- 172.21.15.19:0/148444914 shutdown_connections 2024-04-25T11:36:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.277+0000 7f4423e00640 1 --2- 172.21.15.19:0/148444914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424160fe0 0x7f442415a810 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.277+0000 7f4423e00640 1 -- 172.21.15.19:0/148444914 >> 172.21.15.19:0/148444914 conn(0x7f44240d1720 msgr2=0x7f44240d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.277+0000 7f4423e00640 1 -- 172.21.15.19:0/148444914 shutdown_connections 2024-04-25T11:36:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.277+0000 7f4423e00640 1 -- 172.21.15.19:0/148444914 wait complete. 2024-04-25T11:36:11.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.280+0000 7f4423e00640 1 Processor -- start 2024-04-25T11:36:11.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.280+0000 7f4423e00640 1 -- start start 2024-04-25T11:36:11.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.281+0000 7f4423e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44240b6e10 0x7f4424160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.281+0000 7f4423e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44240cf2f0 0x7f4424149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.281+0000 7f4423e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f442414a5b0 0x7f442415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.281+0000 7f4423e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f442406fd00 con 0x7f442414a5b0 2024-04-25T11:36:11.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.281+0000 7f4423e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f442406f500 con 0x7f44240b6e10 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.281+0000 7f4423e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f442406fe80 con 0x7f44240cf2f0 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f442414a5b0 0x7f442415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f442414a5b0 0x7f442415b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59556/0 (socket says 172.21.15.19:59556) 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 -- 172.21.15.19:0/1106234934 learned_addr learned my addr 172.21.15.19:0/1106234934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4422a00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44240b6e10 0x7f4424160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f441be00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44240cf2f0 0x7f4424149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 -- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44240cf2f0 msgr2=0x7f4424149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44240cf2f0 0x7f4424149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 -- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44240b6e10 msgr2=0x7f4424160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44240b6e10 0x7f4424160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.282+0000 7f4423400640 1 -- 172.21.15.19:0/1106234934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442415aa60 con 0x7f442414a5b0 2024-04-25T11:36:11.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.283+0000 7f4423400640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f442414a5b0 0x7f442415b820 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f70 tx=0x7f441c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:11.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.283+0000 7f441aa00640 1 -- 172.21.15.19:0/1106234934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c041070 con 0x7f442414a5b0 2024-04-25T11:36:11.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.283+0000 7f441aa00640 1 -- 172.21.15.19:0/1106234934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f441c03b420 con 0x7f442414a5b0 2024-04-25T11:36:11.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.283+0000 7f441aa00640 1 -- 172.21.15.19:0/1106234934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c032e60 con 0x7f442414a5b0 2024-04-25T11:36:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.283+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442415ad40 con 0x7f442414a5b0 2024-04-25T11:36:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.283+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442415b300 con 0x7f442414a5b0 2024-04-25T11:36:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.284+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4424050740 con 0x7f442414a5b0 2024-04-25T11:36:11.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.286+0000 7f441aa00640 1 -- 172.21.15.19:0/1106234934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f441c005d10 con 0x7f442414a5b0 2024-04-25T11:36:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.286+0000 7f441aa00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44000744b0 0x7f44000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.287+0000 7f441aa00640 1 -- 172.21.15.19:0/1106234934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7f441c0b5de0 con 0x7f442414a5b0 2024-04-25T11:36:11.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.287+0000 7f4422a00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44000744b0 0x7f44000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.288+0000 7f4422a00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44000744b0 0x7f44000769c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4414006fd0 tx=0x7f4414008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:11.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.291+0000 7f441aa00640 1 -- 172.21.15.19:0/1106234934 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f441c04b020 con 0x7f442414a5b0 2024-04-25T11:36:11.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.528+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44241614e0 con 0x7f442414a5b0 2024-04-25T11:36:11.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.529+0000 7f441aa00640 1 -- 172.21.15.19:0/1106234934 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5177 (secure 0 0 0) 0x7f441c080400 con 0x7f442414a5b0 2024-04-25T11:36:11.519 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:11.519 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":114,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3555498549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3555498549},{"type":"v1","addr":"172.21.15.19:6835","nonce":3555498549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:10.407286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:11.519 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 114 2024-04-25T11:36:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44000744b0 msgr2=0x7f44000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44000744b0 0x7f44000769c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4414006fd0 tx=0x7f4414008040 comp rx=0 tx=0).stop 2024-04-25T11:36:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f442414a5b0 msgr2=0x7f442415b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f442414a5b0 0x7f442415b820 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f441c009f70 tx=0x7f441c004a40 comp rx=0 tx=0).stop 2024-04-25T11:36:11.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 shutdown_connections 2024-04-25T11:36:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44000744b0 0x7f44000769c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f442414a5b0 0x7f442415b820 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44240cf2f0 0x7f4424149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 --2- 172.21.15.19:0/1106234934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44240b6e10 0x7f4424160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 >> 172.21.15.19:0/1106234934 conn(0x7f44240d1720 msgr2=0x7f4424072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:11.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 shutdown_connections 2024-04-25T11:36:11.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.532+0000 7f4423e00640 1 -- 172.21.15.19:0/1106234934 wait complete. 2024-04-25T11:36:11.545 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:36:11.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:11.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.844+0000 7fd6fca00640 1 Processor -- start 2024-04-25T11:36:11.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.844+0000 7fd6fca00640 1 -- start start 2024-04-25T11:36:11.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.846+0000 7fd6fca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd6f80cfab0 0x7fd6f80b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6fca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80d2220 0x7fd6f80d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6fca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6f80b9100 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6fca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f80d1580 con 0x7fd6f80cfab0 2024-04-25T11:36:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6fca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f80b9450 con 0x7fd6f80d2220 2024-04-25T11:36:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6eea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd6f80cfab0 0x7fd6f80b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6eea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd6f80cfab0 0x7fd6f80b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59564/0 (socket says 172.21.15.19:59564) 2024-04-25T11:36:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6eea00640 1 -- 172.21.15.19:0/443870545 learned_addr learned my addr 172.21.15.19:0/443870545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6f7e00640 1 --2- 172.21.15.19:0/443870545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80d2220 0x7fd6f80d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6eea00640 1 -- 172.21.15.19:0/443870545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80d2220 msgr2=0x7fd6f80d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6eea00640 1 --2- 172.21.15.19:0/443870545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80d2220 0x7fd6f80d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6eea00640 1 -- 172.21.15.19:0/443870545 >> v1:172.21.15.169:6789/0 conn(0x7fd6f80d0ee0 legacy=0x7fd6f809ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.847+0000 7fd6eea00640 1 -- 172.21.15.19:0/443870545 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f809d150 con 0x7fd6f80cfab0 2024-04-25T11:36:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.849+0000 7fd6eea00640 1 --2- 172.21.15.19:0/443870545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80cfab0 0x7fd6f80b71a0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbcd4f5a58fcf37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.849+0000 7fd6f6a00640 1 -- 172.21.15.19:0/443870545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0037070 con 0x7fd6f80cfab0 2024-04-25T11:36:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.849+0000 7fd6f6a00640 1 -- 172.21.15.19:0/443870545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e002ae10 con 0x7fd6f80cfab0 2024-04-25T11:36:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.849+0000 7fd6f6a00640 1 -- 172.21.15.19:0/443870545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0032660 con 0x7fd6f80cfab0 2024-04-25T11:36:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 -- 172.21.15.19:0/443870545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80cfab0 msgr2=0x7fd6f80b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 --2- 172.21.15.19:0/443870545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80cfab0 0x7fd6f80b71a0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e002a100 comp rx=0 tx=0).stop 2024-04-25T11:36:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 -- 172.21.15.19:0/443870545 shutdown_connections 2024-04-25T11:36:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 --2- 172.21.15.19:0/443870545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80d2220 0x7fd6f80d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 --2- 172.21.15.19:0/443870545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80cfab0 0x7fd6f80b71a0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 -- 172.21.15.19:0/443870545 >> 172.21.15.19:0/443870545 conn(0x7fd6f80d03c0 msgr2=0x7fd6f80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 -- 172.21.15.19:0/443870545 shutdown_connections 2024-04-25T11:36:11.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.850+0000 7fd6fca00640 1 -- 172.21.15.19:0/443870545 wait complete. 2024-04-25T11:36:11.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.852+0000 7fd6fca00640 1 Processor -- start 2024-04-25T11:36:11.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.852+0000 7fd6fca00640 1 -- start start 2024-04-25T11:36:11.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6fca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80cfab0 0x7fd6f80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6fca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80d0ee0 0x7fd6f809ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6fca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f80d2220 0x7fd6f8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6fca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f80d3590 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6fca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f80d17d0 con 0x7fd6f80d2220 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6fca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f80d3710 con 0x7fd6f80cfab0 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6eea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80d0ee0 0x7fd6f809ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6eea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80d0ee0 0x7fd6f809ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59572/0 (socket says 172.21.15.19:59572) 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.854+0000 7fd6eea00640 1 -- 172.21.15.19:0/884773307 learned_addr learned my addr 172.21.15.19:0/884773307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6f7e00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f80d2220 0x7fd6f8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6f7400640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80cfab0 0x7fd6f80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6eea00640 1 -- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80cfab0 msgr2=0x7fd6f80c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6eea00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80cfab0 0x7fd6f80c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6eea00640 1 -- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f80d2220 msgr2=0x7fd6f8160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:11.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6eea00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f80d2220 0x7fd6f8160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:11.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6eea00640 1 -- 172.21.15.19:0/884773307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f80c7470 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6f7400640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80cfab0 0x7fd6f80c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:11.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6eea00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80d0ee0 0x7fd6f809ef40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6efe00640 1 -- 172.21.15.19:0/884773307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0041070 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6efe00640 1 -- 172.21.15.19:0/884773307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e003c670 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6efe00640 1 -- 172.21.15.19:0/884773307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0037040 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.855+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f80c8db0 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.856+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f80c79f0 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.857+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f8050740 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.858+0000 7fd6efe00640 1 -- 172.21.15.19:0/884773307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6e0044070 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.859+0000 7fd6efe00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6d40744b0 0x7fd6d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:11.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.859+0000 7fd6efe00640 1 -- 172.21.15.19:0/884773307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7fd6e00b5be0 con 0x7fd6f80d0ee0 2024-04-25T11:36:11.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.860+0000 7fd6f7400640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6d40744b0 0x7fd6d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.861+0000 7fd6f7400640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6d40744b0 0x7fd6d40769c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd6f806d5a0 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:11.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:11.864+0000 7fd6efe00640 1 -- 172.21.15.19:0/884773307 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6e0080200 con 0x7fd6f80d0ee0 2024-04-25T11:36:12.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.110+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6f808cd60 con 0x7fd6f80d0ee0 2024-04-25T11:36:12.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.111+0000 7fd6efe00640 1 -- 172.21.15.19:0/884773307 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5177 (secure 0 0 0) 0x7fd6e007fb00 con 0x7fd6f80d0ee0 2024-04-25T11:36:12.100 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:12.101 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":114,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3555498549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3555498549},{"type":"v1","addr":"172.21.15.19:6835","nonce":3555498549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:10.407286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:12.101 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 114 2024-04-25T11:36:12.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6d40744b0 msgr2=0x7fd6d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:12.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6d40744b0 0x7fd6d40769c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd6f806d5a0 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80d0ee0 msgr2=0x7fd6f809ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80d0ee0 0x7fd6f809ef40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e00046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 shutdown_connections 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6d40744b0 0x7fd6d40769c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6f80d2220 0x7fd6f8160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6f80d0ee0 0x7fd6f809ef40 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 --2- 172.21.15.19:0/884773307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6f80cfab0 0x7fd6f80c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 >> 172.21.15.19:0/884773307 conn(0x7fd6f80d03c0 msgr2=0x7fd6f806e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 shutdown_connections 2024-04-25T11:36:12.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.114+0000 7fd6fca00640 1 -- 172.21.15.19:0/884773307 wait complete. 2024-04-25T11:36:12.130 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2024-04-25T11:36:12.130 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2024-04-25T11:36:12.131 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:12.131 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:12.186 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:36:12.186 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:36:12.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:12.186 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:36:12.244 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:36:12.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:36:12.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:36:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.551+0000 7f1493e00640 1 Processor -- start 2024-04-25T11:36:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.551+0000 7f1493e00640 1 -- start start 2024-04-25T11:36:12.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.553+0000 7f1493e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f149406c5c0 0x7f14940cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.553+0000 7f1493e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 0x7f14940b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.553+0000 7f1493e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14940b50e0 con 0x7f14940b4cd0 2024-04-25T11:36:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.554+0000 7f1493e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f14940d1230 con 0x7f149406c5c0 2024-04-25T11:36:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.554+0000 7f1493e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14940d06b0 con 0x7f1494144370 2024-04-25T11:36:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.554+0000 7f1493400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f14940b4cd0 0x7f1494161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56716/0 (socket says 172.21.15.19:56716) 2024-04-25T11:36:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.554+0000 7f1493400640 1 -- 172.21.15.19:0/3974699136 learned_addr learned my addr 172.21.15.19:0/3974699136 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.554+0000 7f1492a00640 1 --2- 172.21.15.19:0/3974699136 >> v2:172.21.15.169:3300/0 conn(0x7f149406c5c0 0x7f14940cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.554+0000 7f1492000640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 0x7f14940b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789330568 0 0) 0x7f148c001110 con 0x7f14940b4cd0 2024-04-25T11:36:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1478003520 con 0x7f14940b4cd0 2024-04-25T11:36:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1693858748 0 0) 0x7f1478003520 con 0x7f14940b4cd0 2024-04-25T11:36:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 msgr2=0x7f14940b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 0x7f14940b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 >> v2:172.21.15.169:3300/0 conn(0x7f149406c5c0 msgr2=0x7f14940cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 --2- 172.21.15.19:0/3974699136 >> v2:172.21.15.169:3300/0 conn(0x7f149406c5c0 0x7f14940cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14940d0990 con 0x7f14940b4cd0 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f148c001840 con 0x7f14940b4cd0 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 >> v1:172.21.15.19:6789/0 conn(0x7f14940b4cd0 legacy=0x7f1494161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14780039b0 0x7f1478003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478004800 0x7f1478004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 0x7f1478005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14940d1230 con 0x7f14780039b0 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14940d06b0 con 0x7f1478004800 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.555+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f147800b9f0 con 0x7f1494144370 2024-04-25T11:36:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1492a00640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14780039b0 0x7f1478003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1493400640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 0x7f1478005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1492a00640 1 -- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 msgr2=0x7f1478005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:12.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1492a00640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 0x7f1478005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1492a00640 1 -- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478004800 msgr2=0x7f1478004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1492000640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478004800 0x7f1478004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1492a00640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1478004800 0x7f1478004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1492a00640 1 -- 172.21.15.19:0/3974699136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1478003520 con 0x7f14780039b0 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.556+0000 7f1493400640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1494144370 0x7f1478005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.557+0000 7f1492a00640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14780039b0 0x7f1478003e20 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f1484009ac0 tx=0x7f1478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ee53de17d12b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484036070 con 0x7f14780039b0 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1484031400 con 0x7f14780039b0 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1491600640 1 -- 172.21.15.19:0/3974699136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484009690 con 0x7f14780039b0 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1493e00640 1 -- 172.21.15.19:0/3974699136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14780039b0 msgr2=0x7f1478003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1493e00640 1 --2- 172.21.15.19:0/3974699136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14780039b0 0x7f1478003e20 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f1484009ac0 tx=0x7f1478005830 comp rx=0 tx=0).stop 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1493400640 1 -- 172.21.15.19:0/3974699136 reap_dead start 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1493e00640 1 -- 172.21.15.19:0/3974699136 shutdown_connections 2024-04-25T11:36:12.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.558+0000 7f1493e00640 1 -- 172.21.15.19:0/3974699136 >> 172.21.15.19:0/3974699136 conn(0x7f14940d1720 msgr2=0x7f14940d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.559+0000 7f1493e00640 1 -- 172.21.15.19:0/3974699136 shutdown_connections 2024-04-25T11:36:12.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.559+0000 7f1493e00640 1 -- 172.21.15.19:0/3974699136 wait complete. 2024-04-25T11:36:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.561+0000 7f1493e00640 1 Processor -- start 2024-04-25T11:36:12.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.561+0000 7f1493e00640 1 -- start start 2024-04-25T11:36:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1493e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14780039b0 0x7f14941e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1493e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14940cc480 0x7f149415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1493e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1494161ae0 0x7f1494161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1493e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14940d02d0 con 0x7f14940cc480 2024-04-25T11:36:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1493e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14940d0f50 con 0x7f1494161ae0 2024-04-25T11:36:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1493e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14940d0450 con 0x7f14780039b0 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1492000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14940cc480 0x7f149415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1492000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14940cc480 0x7f149415b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59600/0 (socket says 172.21.15.19:59600) 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1492000640 1 -- 172.21.15.19:0/3762323193 learned_addr learned my addr 172.21.15.19:0/3762323193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.563+0000 7f1492a00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14780039b0 0x7f14941e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1493400640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1494161ae0 0x7f1494161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1492000640 1 -- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14780039b0 msgr2=0x7f14941e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1492000640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14780039b0 0x7f14941e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1492000640 1 -- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1494161ae0 msgr2=0x7f1494161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1492000640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1494161ae0 0x7f1494161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1492000640 1 -- 172.21.15.19:0/3762323193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494162430 con 0x7f14940cc480 2024-04-25T11:36:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1493400640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1494161ae0 0x7f1494161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1492000640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14940cc480 0x7f149415b7b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f148000f380 tx=0x7f148000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f148aa00640 1 -- 172.21.15.19:0/3762323193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148001c070 con 0x7f14940cc480 2024-04-25T11:36:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f148aa00640 1 -- 172.21.15.19:0/3762323193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1480017620 con 0x7f14940cc480 2024-04-25T11:36:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f148aa00640 1 -- 172.21.15.19:0/3762323193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480017a10 con 0x7f14940cc480 2024-04-25T11:36:12.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.564+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14941fa7f0 con 0x7f14940cc480 2024-04-25T11:36:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.565+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14941fa230 con 0x7f14940cc480 2024-04-25T11:36:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.566+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1494050740 con 0x7f14940cc480 2024-04-25T11:36:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.567+0000 7f148aa00640 1 -- 172.21.15.19:0/3762323193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1480005d10 con 0x7f14940cc480 2024-04-25T11:36:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.569+0000 7f148aa00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14740744e0 0x7f14740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.569+0000 7f148aa00640 1 -- 172.21.15.19:0/3762323193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7f1480095a00 con 0x7f14940cc480 2024-04-25T11:36:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.569+0000 7f1492a00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14740744e0 0x7f14740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:12.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.571+0000 7f1492a00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14740744e0 0x7f14740769f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1484033000 tx=0x7f1484008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:12.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.574+0000 7f148aa00640 1 -- 172.21.15.19:0/3762323193 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1494162430 con 0x7f14940cc480 2024-04-25T11:36:12.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:12.817+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f149406f160 con 0x7f14940cc480 2024-04-25T11:36:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.414+0000 7f148aa00640 1 -- 172.21.15.19:0/3762323193 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) ==== 161+0+0 (secure 0 0 0) 0x7f1480012070 con 0x7f14940cc480 2024-04-25T11:36:13.403 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:36:13.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14740744e0 msgr2=0x7f14740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:13.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14740744e0 0x7f14740769f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1484033000 tx=0x7f1484008030 comp rx=0 tx=0).stop 2024-04-25T11:36:13.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14940cc480 msgr2=0x7f149415b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:13.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14940cc480 0x7f149415b7b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f148000f380 tx=0x7f148000f840 comp rx=0 tx=0).stop 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 shutdown_connections 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14740744e0 0x7f14740769f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1494161ae0 0x7f1494161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14940cc480 0x7f149415b7b0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 --2- 172.21.15.19:0/3762323193 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14780039b0 0x7f14941e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 >> 172.21.15.19:0/3762323193 conn(0x7f14940d1720 msgr2=0x7f14940684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 shutdown_connections 2024-04-25T11:36:13.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.418+0000 7f1493e00640 1 -- 172.21.15.19:0/3762323193 wait complete. 2024-04-25T11:36:13.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:36:13.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.730+0000 7efc83000640 1 Processor -- start 2024-04-25T11:36:13.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.730+0000 7efc83000640 1 -- start start 2024-04-25T11:36:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc83000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1443c0 0x7efc7c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc83000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efc7c0cfcf0 0x7efc7c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc83000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7c15f5f0 con 0x7efc7c070370 2024-04-25T11:36:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc83000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efc7c0708d0 con 0x7efc7c0cfcf0 2024-04-25T11:36:13.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc83000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c15f160 con 0x7efc7c1443c0 2024-04-25T11:36:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc73e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efc7c070370 0x7efc7c0cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56730/0 (socket says 172.21.15.19:56730) 2024-04-25T11:36:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc73e00640 1 -- 172.21.15.19:0/2754882625 learned_addr learned my addr 172.21.15.19:0/2754882625 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.732+0000 7efc81000640 1 --2- 172.21.15.19:0/2754882625 >> v2:172.21.15.19:3300/0 conn(0x7efc7c0cfcf0 0x7efc7c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.733+0000 7efc7be00640 1 --2- 172.21.15.19:0/2754882625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1443c0 0x7efc7c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.733+0000 7efc7b400640 1 -- 172.21.15.19:0/2754882625 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314648216 0 0) 0x7efc7c15f5f0 con 0x7efc7c070370 2024-04-25T11:36:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.733+0000 7efc7b400640 1 -- 172.21.15.19:0/2754882625 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc60003520 con 0x7efc7c070370 2024-04-25T11:36:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.733+0000 7efc81000640 1 -- 172.21.15.19:0/2754882625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1443c0 msgr2=0x7efc7c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.733+0000 7efc81000640 1 --2- 172.21.15.19:0/2754882625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1443c0 0x7efc7c06fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.733+0000 7efc81000640 1 -- 172.21.15.19:0/2754882625 >> v1:172.21.15.19:6789/0 conn(0x7efc7c070370 legacy=0x7efc7c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.733+0000 7efc81000640 1 -- 172.21.15.19:0/2754882625 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c15e830 con 0x7efc7c0cfcf0 2024-04-25T11:36:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.735+0000 7efc81000640 1 --2- 172.21.15.19:0/2754882625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0cfcf0 0x7efc7c06c5c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7efc74009d90 tx=0x7efc7402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d342a3762af5c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.735+0000 7efc7b400640 1 -- 172.21.15.19:0/2754882625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74007740 con 0x7efc7c0cfcf0 2024-04-25T11:36:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.735+0000 7efc7b400640 1 -- 172.21.15.19:0/2754882625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc740078e0 con 0x7efc7c0cfcf0 2024-04-25T11:36:13.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.735+0000 7efc7b400640 1 -- 172.21.15.19:0/2754882625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74007c90 con 0x7efc7c0cfcf0 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 -- 172.21.15.19:0/2754882625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0cfcf0 msgr2=0x7efc7c06c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 --2- 172.21.15.19:0/2754882625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0cfcf0 0x7efc7c06c5c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7efc74009d90 tx=0x7efc7402a660 comp rx=0 tx=0).stop 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 -- 172.21.15.19:0/2754882625 shutdown_connections 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 --2- 172.21.15.19:0/2754882625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0cfcf0 0x7efc7c06c5c0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 --2- 172.21.15.19:0/2754882625 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1443c0 0x7efc7c06fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 -- 172.21.15.19:0/2754882625 >> 172.21.15.19:0/2754882625 conn(0x7efc7c0d1ae0 msgr2=0x7efc7c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 -- 172.21.15.19:0/2754882625 shutdown_connections 2024-04-25T11:36:13.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.736+0000 7efc83000640 1 -- 172.21.15.19:0/2754882625 wait complete. 2024-04-25T11:36:13.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.738+0000 7efc83000640 1 Processor -- start 2024-04-25T11:36:13.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.739+0000 7efc83000640 1 -- start start 2024-04-25T11:36:13.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.740+0000 7efc83000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c070370 0x7efc7c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:13.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.740+0000 7efc83000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc7c1443c0 0x7efc7c13be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.740+0000 7efc83000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0b16a0 0x7efc7c13c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.740+0000 7efc83000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c1447d0 con 0x7efc7c0b16a0 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.740+0000 7efc83000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c15edf0 con 0x7efc7c1443c0 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.740+0000 7efc83000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c162b00 con 0x7efc7c070370 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0b16a0 0x7efc7c13c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0b16a0 0x7efc7c13c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59618/0 (socket says 172.21.15.19:59618) 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 -- 172.21.15.19:0/1407880269 learned_addr learned my addr 172.21.15.19:0/1407880269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc7be00640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c070370 0x7efc7c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc73e00640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc7c1443c0 0x7efc7c13be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 -- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c070370 msgr2=0x7efc7c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c070370 0x7efc7c15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 -- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc7c1443c0 msgr2=0x7efc7c13be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc7c1443c0 0x7efc7c13be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc81000640 1 -- 172.21.15.19:0/1407880269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c0b1e80 con 0x7efc7c0b16a0 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.741+0000 7efc73e00640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc7c1443c0 0x7efc7c13be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.742+0000 7efc81000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0b16a0 0x7efc7c13c3c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7efc7402ac50 tx=0x7efc74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.742+0000 7efc78c00640 1 -- 172.21.15.19:0/1407880269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7403f070 con 0x7efc7c0b16a0 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.742+0000 7efc78c00640 1 -- 172.21.15.19:0/1407880269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc74031c60 con 0x7efc7c0b16a0 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.742+0000 7efc78c00640 1 -- 172.21.15.19:0/1407880269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74009450 con 0x7efc7c0b16a0 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.742+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c13f590 con 0x7efc7c0b16a0 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.742+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c13fb50 con 0x7efc7c0b16a0 2024-04-25T11:36:13.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.744+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7c0b1bf0 con 0x7efc7c0b16a0 2024-04-25T11:36:13.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.745+0000 7efc78c00640 1 -- 172.21.15.19:0/1407880269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc7403b410 con 0x7efc7c0b16a0 2024-04-25T11:36:13.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.746+0000 7efc78c00640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc580744b0 0x7efc580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:13.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.746+0000 7efc78c00640 1 -- 172.21.15.19:0/1407880269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7efc740b5fe0 con 0x7efc7c0b16a0 2024-04-25T11:36:13.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.747+0000 7efc7be00640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc580744b0 0x7efc580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:13.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.748+0000 7efc7be00640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc580744b0 0x7efc580769c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0d1860 tx=0x7efc6c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:13.751+0000 7efc78c00640 1 -- 172.21.15.19:0/1407880269 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc7404a070 con 0x7efc7c0b16a0 2024-04-25T11:36:14.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.027+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7efc7c050740 con 0x7efc7c0b16a0 2024-04-25T11:36:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.418+0000 7efc78c00640 1 -- 172.21.15.19:0/1407880269 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v116) ==== 101+0+0 (secure 0 0 0) 0x7efc740805b0 con 0x7efc7c0b16a0 2024-04-25T11:36:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:36:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc580744b0 msgr2=0x7efc580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc580744b0 0x7efc580769c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efc7c0d1860 tx=0x7efc6c009190 comp rx=0 tx=0).stop 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0b16a0 msgr2=0x7efc7c13c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0b16a0 0x7efc7c13c3c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7efc7402ac50 tx=0x7efc74004a40 comp rx=0 tx=0).stop 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 shutdown_connections 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc580744b0 0x7efc580769c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc7c1443c0 0x7efc7c13be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c0b16a0 0x7efc7c13c3c0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 --2- 172.21.15.19:0/1407880269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c070370 0x7efc7c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 >> 172.21.15.19:0/1407880269 conn(0x7efc7c0d1ae0 msgr2=0x7efc7c15f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.421+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 shutdown_connections 2024-04-25T11:36:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.422+0000 7efc83000640 1 -- 172.21.15.19:0/1407880269 wait complete. 2024-04-25T11:36:14.441 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:36:14.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.639+0000 7fed87f2df40 1 Processor -- start 2024-04-25T11:36:14.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.639+0000 7fed87f2df40 1 -- start start 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.640+0000 7fed87f2df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385107990 0x5623851e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.640+0000 7fed87f2df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562385150790 0x5623851db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.640+0000 7fed87f2df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385152af0 0x5623851dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.640+0000 7fed87f2df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562385107da0 con 0x562385152af0 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.640+0000 7fed87f2df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562385108f90 con 0x562385107990 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.640+0000 7fed87f2df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562385153390 con 0x562385150790 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385152af0 0x5623851dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385152af0 0x5623851dbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40136/0 (socket says 172.21.15.19:40136) 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 -- 172.21.15.19:0/3143597499 learned_addr learned my addr 172.21.15.19:0/3143597499 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 -- 172.21.15.19:0/3143597499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562385150790 msgr2=0x5623851db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 --2- 172.21.15.19:0/3143597499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562385150790 0x5623851db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 -- 172.21.15.19:0/3143597499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385107990 msgr2=0x5623851e3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 --2- 172.21.15.19:0/3143597499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385107990 0x5623851e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.641+0000 7fed81000640 1 -- 172.21.15.19:0/3143597499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562385033b30 con 0x562385152af0 2024-04-25T11:36:14.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed81000640 1 --2- 172.21.15.19:0/3143597499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385152af0 0x5623851dbb10 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fed7800a550 tx=0x7fed7800a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5f19f15de9a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed7e800640 1 -- 172.21.15.19:0/3143597499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78018070 con 0x562385152af0 2024-04-25T11:36:14.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed7e800640 1 -- 172.21.15.19:0/3143597499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed78007600 con 0x562385152af0 2024-04-25T11:36:14.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed7e800640 1 -- 172.21.15.19:0/3143597499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780079d0 con 0x562385152af0 2024-04-25T11:36:14.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed87f2df40 1 -- 172.21.15.19:0/3143597499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385152af0 msgr2=0x5623851dbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:14.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed87f2df40 1 --2- 172.21.15.19:0/3143597499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385152af0 0x5623851dbb10 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fed7800a550 tx=0x7fed7800a640 comp rx=0 tx=0).stop 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed87f2df40 1 -- 172.21.15.19:0/3143597499 shutdown_connections 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed87f2df40 1 --2- 172.21.15.19:0/3143597499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385152af0 0x5623851dbb10 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed87f2df40 1 --2- 172.21.15.19:0/3143597499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562385150790 0x5623851db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed87f2df40 1 --2- 172.21.15.19:0/3143597499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385107990 0x5623851e3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.642+0000 7fed87f2df40 1 -- 172.21.15.19:0/3143597499 >> 172.21.15.19:0/3143597499 conn(0x56238502cac0 msgr2=0x562385137cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.643+0000 7fed87f2df40 1 -- 172.21.15.19:0/3143597499 shutdown_connections 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.643+0000 7fed87f2df40 1 -- 172.21.15.19:0/3143597499 wait complete. 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.644+0000 7fed87f2df40 1 Processor -- start 2024-04-25T11:36:14.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.644+0000 7fed87f2df40 1 -- start start 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.645+0000 7fed87f2df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385107990 0x56238514be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.645+0000 7fed87f2df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385150790 0x5623852736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.645+0000 7fed87f2df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562385152af0 0x562385275570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.645+0000 7fed87f2df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56238510a440 con 0x562385107990 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.645+0000 7fed87f2df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562385150310 con 0x562385150790 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.645+0000 7fed87f2df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56238514c840 con 0x562385152af0 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385107990 0x56238514be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385107990 0x56238514be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40150/0 (socket says 172.21.15.19:40150) 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 learned_addr learned my addr 172.21.15.19:0/748749668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:14.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562385152af0 msgr2=0x562385275570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562385152af0 0x562385275570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385150790 msgr2=0x5623852736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed7fc00640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385150790 0x5623852736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562385150790 0x5623852736d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238514c340 con 0x562385107990 2024-04-25T11:36:14.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.646+0000 7fed80600640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385107990 0x56238514be00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fed7000a440 tx=0x7fed7000a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.647+0000 7fed7d400640 1 -- 172.21.15.19:0/748749668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70011630 con 0x562385107990 2024-04-25T11:36:14.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.647+0000 7fed7d400640 1 -- 172.21.15.19:0/748749668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed700117f0 con 0x562385107990 2024-04-25T11:36:14.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.647+0000 7fed7d400640 1 -- 172.21.15.19:0/748749668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70011be0 con 0x562385107990 2024-04-25T11:36:14.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.647+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562385138510 con 0x562385107990 2024-04-25T11:36:14.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.647+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623851da210 con 0x562385107990 2024-04-25T11:36:14.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.649+0000 7fed7d400640 1 -- 172.21.15.19:0/748749668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed7001c400 con 0x562385107990 2024-04-25T11:36:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.650+0000 7fed7d400640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed60072e90 0x7fed600753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.650+0000 7fed7d400640 1 -- 172.21.15.19:0/748749668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7fed70094320 con 0x562385107990 2024-04-25T11:36:14.639 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:36:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.650+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56238527fd40 0x562385288650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.650+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623851dac10 con 0x56238527fd40 2024-04-25T11:36:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.651+0000 7fed7fc00640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed60072e90 0x7fed600753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.651+0000 7fed81000640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56238527fd40 0x562385288650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.652+0000 7fed7fc00640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed60072e90 0x7fed600753a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed6c057000 tx=0x7fed6c051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.653+0000 7fed81000640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56238527fd40 0x562385288650 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.655+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v96'102 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.655+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x562385275ab0 con 0x56238527fd40 2024-04-25T11:36:14.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.659+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v96'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.659+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385288b90 con 0x56238527fd40 2024-04-25T11:36:14.651 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:36:14.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.662+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v96'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.662+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385289f40 con 0x56238527fd40 2024-04-25T11:36:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.665+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v96'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.665+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528a2e0 con 0x56238527fd40 2024-04-25T11:36:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.668+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v96'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.668+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385288f30 con 0x56238527fd40 2024-04-25T11:36:14.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.671+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v96'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.671+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385289340 con 0x56238527fd40 2024-04-25T11:36:14.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.674+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v96'108 uv84 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.674+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x562385289340 con 0x56238527fd40 2024-04-25T11:36:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.677+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v96'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.677+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892f0 con 0x56238527fd40 2024-04-25T11:36:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.680+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v96'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.669 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:36:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.680+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892f0 con 0x56238527fd40 2024-04-25T11:36:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.683+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v96'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.683+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.687+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v96'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.687+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.690+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v96'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.690+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.693+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v96'114 uv91 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.693+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.696+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v96'115 uv115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.696+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.688 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:36:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.699+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v96'116 uv116 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.700+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.703+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v96'117 uv117 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.703+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.707+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v96'118 uv118 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.707+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.710+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v96'119 uv119 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.711+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.714+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v96'120 uv120 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.714+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.718+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v96'121 uv95 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.718+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.722+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v96'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.722+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.726+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v96'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.715 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:36:14.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.726+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.730+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v96'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.730+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.734+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v96'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.734+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.739+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v96'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.739+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.743+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v96'127 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.743+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.747+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v96'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.747+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.740 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:36:14.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.751+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v96'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.751+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.755+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v96'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.755+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.760+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v96'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.760+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238527fd40 2024-04-25T11:36:14.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.764+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v96'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.764+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56238528a680 0x56238528caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.764+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852892d0 con 0x56238528a680 2024-04-25T11:36:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.765+0000 7fed80600640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56238528a680 0x56238528caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.767+0000 7fed80600640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56238528a680 0x56238528caa0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.759 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:36:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.771+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v96'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.771+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528cfe0 con 0x56238528a680 2024-04-25T11:36:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.775+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v96'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.775+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528d380 con 0x56238528a680 2024-04-25T11:36:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.779+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v96'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.779+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528d720 con 0x56238528a680 2024-04-25T11:36:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.783+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 200.00000000 [create] v96'92 uv76 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.783+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238528dac0 con 0x56238528a680 2024-04-25T11:36:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.788+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 200.00000000 [delete] v96'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.788+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528de60 con 0x56238528a680 2024-04-25T11:36:14.780 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:36:14.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.792+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v96'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.792+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528e200 con 0x56238528a680 2024-04-25T11:36:14.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.796+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v96'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.797+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.801+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v96'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.801+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.805+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000001 [create] v96'97 uv81 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.805+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.809+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000001 [delete] v96'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.809+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.801 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:36:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.812+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v96'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.813+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385292c90 con 0x56238528a680 2024-04-25T11:36:14.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.818+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [write 0~18117] v96'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.818+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385292c90 con 0x56238528a680 2024-04-25T11:36:14.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.822+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v96'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.822+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.826+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 606.00000000 [create] v96'102 uv87 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.826+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.830+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 606.00000000 [delete] v96'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.830+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.823 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:36:14.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.834+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v96'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.834+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528f5b0 con 0x56238528a680 2024-04-25T11:36:14.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.838+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v96'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.838+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528f5b0 con 0x56238528a680 2024-04-25T11:36:14.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.842+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v96'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.842+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238528e5a0 con 0x56238528a680 2024-04-25T11:36:14.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.846+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v96'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x56238528a680 2024-04-25T11:36:14.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.846+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56238528eab0 0x562385290ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.846+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385291410 con 0x56238528eab0 2024-04-25T11:36:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.847+0000 7fed7fc00640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56238528eab0 0x562385290ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.849+0000 7fed7fc00640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56238528eab0 0x562385290ed0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.853+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(51 607.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:14.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.853+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5623852917b0 con 0x56238528eab0 2024-04-25T11:36:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.858+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(52 607.00000000 [delete] v96'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.858+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385291b50 con 0x56238528eab0 2024-04-25T11:36:14.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.861+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(53 607.00000000 [create] v96'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:14.850 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:36:14.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.862+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385292f00 con 0x56238528eab0 2024-04-25T11:36:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.866+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.866+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852932a0 con 0x56238528eab0 2024-04-25T11:36:14.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.869+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:14.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.870+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385291ef0 con 0x56238528eab0 2024-04-25T11:36:14.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.874+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:14.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.874+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852922e0 con 0x56238527fd40 2024-04-25T11:36:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.879+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(57 mds0_openfiles.0 [create] v96'19 uv18 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.879+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x5623852922e0 con 0x56238527fd40 2024-04-25T11:36:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.883+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v96'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.883+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852922e0 con 0x56238527fd40 2024-04-25T11:36:14.876 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:36:14.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.888+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(59 mds0_openfiles.0 [create] v96'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.888+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852922e0 con 0x56238527fd40 2024-04-25T11:36:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.892+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v96'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.892+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562385293640 0x562385295a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.892+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385295fa0 con 0x562385293640 2024-04-25T11:36:14.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.894+0000 7fed81000640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562385293640 0x562385295a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.895+0000 7fed81000640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562385293640 0x562385295a60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.899+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(61 608.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:14.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.899+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x562385295fa0 con 0x562385293640 2024-04-25T11:36:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.904+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(62 608.00000000 [delete] v96'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.904+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385295fa0 con 0x562385293640 2024-04-25T11:36:14.897 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:36:14.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.908+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(63 608.00000000 [create] v96'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:14.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.908+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385296fb0 con 0x562385293640 2024-04-25T11:36:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.913+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.913+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385296fb0 con 0x562385293640 2024-04-25T11:36:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.918+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:14.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.918+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385295fa0 con 0x562385293640 2024-04-25T11:36:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.923+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.923+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562385296620 0x562385298a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.924+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x5623852990a0 con 0x562385296620 2024-04-25T11:36:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.924+0000 7fed80600640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562385296620 0x562385298a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.926+0000 7fed80600640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562385296620 0x562385298a40 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.929+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(67 604.00000000 [create] v96'37 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.930+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x562385299160 con 0x562385296620 2024-04-25T11:36:14.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.934+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(68 604.00000000 [delete] v96'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.934+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385299580 con 0x562385296620 2024-04-25T11:36:14.926 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:36:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.938+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(69 604.00000000 [create] v96'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.938+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529a590 con 0x562385296620 2024-04-25T11:36:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.942+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v96'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.942+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529a590 con 0x562385296620 2024-04-25T11:36:14.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.945+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v96'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.946+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385299580 con 0x562385296620 2024-04-25T11:36:14.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.950+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v96'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.950+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385299580 con 0x562385296620 2024-04-25T11:36:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.954+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(73 500.00000000 [create] v96'43 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.954+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x562385299580 con 0x562385296620 2024-04-25T11:36:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.958+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(74 500.00000000 [delete] v96'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.958+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385299580 con 0x562385296620 2024-04-25T11:36:14.950 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:36:14.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.962+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(75 500.00000000 [create] v96'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.962+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385299580 con 0x562385296620 2024-04-25T11:36:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.966+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v96'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.966+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562385299580 con 0x562385296620 2024-04-25T11:36:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.970+0000 7fed80600640 1 -- 172.21.15.19:0/748749668 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v96'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed70094050 con 0x562385296620 2024-04-25T11:36:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.970+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562385299a90 0x56238529bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.970+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529c620 con 0x562385299a90 2024-04-25T11:36:14.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.971+0000 7fed7fc00640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562385299a90 0x56238529bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:14.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.972+0000 7fed7fc00640 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562385299a90 0x56238529bed0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.977+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(78 mds_snaptable [create] v96'17 uv16 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.977+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529c9f0 con 0x562385299a90 2024-04-25T11:36:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.981+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(79 mds_snaptable [delete] v96'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:14.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.981+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x562385299a90 2024-04-25T11:36:14.974 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:36:14.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.985+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(80 mds_snaptable [create] v96'19 uv19 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:14.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.985+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x562385299a90 2024-04-25T11:36:14.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.989+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v96'20 uv20 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:14.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.990+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x562385299a90 2024-04-25T11:36:14.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.994+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v96'21 uv21 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:14.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.994+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x56238527fd40 2024-04-25T11:36:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.998+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(83 605.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:14.998+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529ce10 con 0x56238527fd40 2024-04-25T11:36:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.002+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(84 605.00000000 [delete] v96'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.002+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x56238527fd40 2024-04-25T11:36:14.994 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:36:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.006+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(85 605.00000000 [create] v96'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.006+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529de20 con 0x56238527fd40 2024-04-25T11:36:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.010+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:14.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.010+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529de20 con 0x56238527fd40 2024-04-25T11:36:15.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.014+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:15.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.014+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x56238527fd40 2024-04-25T11:36:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.018+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x56238527fd40 2024-04-25T11:36:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.018+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x56238528eab0 2024-04-25T11:36:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.023+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(89 mds0_inotable [create] v96'34 uv28 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.023+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529ce10 con 0x56238528eab0 2024-04-25T11:36:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.028+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(90 mds0_inotable [delete] v96'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.028+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ce10 con 0x56238528eab0 2024-04-25T11:36:15.021 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:36:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.032+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(91 mds0_inotable [create] v96'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.032+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d200 con 0x56238528eab0 2024-04-25T11:36:15.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.037+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v96'37 uv37 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.037+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d200 con 0x56238528eab0 2024-04-25T11:36:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.042+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v96'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.042+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ceb0 con 0x56238528eab0 2024-04-25T11:36:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.046+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(94 100.00000000 [create] v96'39 uv33 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.046+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529ceb0 con 0x56238528eab0 2024-04-25T11:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.051+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(95 100.00000000 [delete] v96'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.051+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ceb0 con 0x56238528eab0 2024-04-25T11:36:15.043 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:36:15.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.055+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(96 100.00000000 [create] v96'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.055+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529ceb0 con 0x56238528eab0 2024-04-25T11:36:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.059+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v96'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.059+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529fcc0 con 0x56238528eab0 2024-04-25T11:36:15.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.064+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v96'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x56238528eab0 2024-04-25T11:36:15.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.064+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.068+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v96'18 uv17 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.068+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.073+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v96'19 uv19 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.073+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.067 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:36:15.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.078+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v96'20 uv20 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.078+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.083+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v96'21 uv21 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.083+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.087+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v96'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.087+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.091+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v96'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.091+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.084 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:36:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.096+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v96'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.096+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529e370 con 0x562385293640 2024-04-25T11:36:15.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.100+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v96'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.100+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529e370 con 0x562385293640 2024-04-25T11:36:15.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.104+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v96'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.104+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385293640 2024-04-25T11:36:15.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.109+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v96'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed7801b040 con 0x562385293640 2024-04-25T11:36:15.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.109+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.113+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v96'17 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.113+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.117+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v96'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.117+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.109 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:36:15.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.120+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v96'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.121+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.125+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v96'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.125+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.129+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v96'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.130+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.133+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v96'45 uv37 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.133+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.138+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v96'46 uv46 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.138+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.130 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:36:15.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.141+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v96'47 uv47 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.141+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.145+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v96'48 uv48 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.145+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.149+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v96'49 uv49 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.149+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.154+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v96'50 uv44 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.154+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.158+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v96'51 uv51 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.158+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.150 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:36:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.161+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v96'52 uv52 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.162+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529e370 con 0x562385299a90 2024-04-25T11:36:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.165+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v96'53 uv53 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.165+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529e370 con 0x562385299a90 2024-04-25T11:36:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.170+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v96'54 uv54 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.170+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d360 con 0x562385299a90 2024-04-25T11:36:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.174+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v96'55 uv55 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.174+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56238529d8c0 con 0x562385299a90 2024-04-25T11:36:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.178+0000 7fed7fc00640 1 -- 172.21.15.19:0/748749668 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v96'56 uv56 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fed6c052070 con 0x562385299a90 2024-04-25T11:36:15.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.178+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56238528a680 msgr2=0x56238528caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.178+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56238528a680 0x56238528caa0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.178+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562385296620 msgr2=0x562385298a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.178+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562385296620 0x562385298a40 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.178+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562385299a90 msgr2=0x56238529bed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.178+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562385299a90 0x56238529bed0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56238528eab0 msgr2=0x562385290ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56238528eab0 0x562385290ed0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 reap_dead start 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56238527fd40 msgr2=0x562385288650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56238527fd40 0x562385288650 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562385293640 msgr2=0x562385295a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562385293640 0x562385295a60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed60072e90 msgr2=0x7fed600753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed60072e90 0x7fed600753a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed6c057000 tx=0x7fed6c051000 comp rx=0 tx=0).stop 2024-04-25T11:36:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385107990 msgr2=0x56238514be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed87f2df40 1 --2- 172.21.15.19:0/748749668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562385107990 0x56238514be00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fed7000a440 tx=0x7fed7000a900 comp rx=0 tx=0).stop 2024-04-25T11:36:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.179+0000 7fed81000640 1 -- 172.21.15.19:0/748749668 reap_dead start 2024-04-25T11:36:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.180+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 shutdown_connections 2024-04-25T11:36:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.180+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 >> 172.21.15.19:0/748749668 conn(0x56238502cac0 msgr2=0x562385137cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.180+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 shutdown_connections 2024-04-25T11:36:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.180+0000 7fed87f2df40 1 -- 172.21.15.19:0/748749668 wait complete. 2024-04-25T11:36:15.192 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2024-04-25T11:36:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.396+0000 7fe0ed7c7f40 1 Processor -- start 2024-04-25T11:36:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.396+0000 7fe0ed7c7f40 1 -- start start 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0ed7c7f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43cd466e0 0x55b43cd84be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0ed7c7f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43ce1a960 0x55b43ce13470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0ed7c7f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce14d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0ed7c7f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b43ccebd20 con 0x55b43cd8d050 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0ed7c7f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b43ccefd60 con 0x55b43ce1a960 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0ed7c7f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55b43ce15ab0 con 0x55b43cd466e0 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce14d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce14d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40172/0 (socket says 172.21.15.19:40172) 2024-04-25T11:36:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 -- 172.21.15.19:0/45062125 learned_addr learned my addr 172.21.15.19:0/45062125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e6a00640 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43cd466e0 0x55b43cd84be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e6000640 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43ce1a960 0x55b43ce13470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 -- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43cd466e0 msgr2=0x55b43cd84be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43cd466e0 0x55b43cd84be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 -- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43ce1a960 msgr2=0x55b43ce13470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43ce1a960 0x55b43ce13470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.398+0000 7fe0e7400640 1 -- 172.21.15.19:0/45062125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43cd85d40 con 0x55b43cd8d050 2024-04-25T11:36:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.399+0000 7fe0e6000640 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43ce1a960 0x55b43ce13470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.399+0000 7fe0e7400640 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce14d90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000a4e0 tx=0x7fe0e000a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0480ed4346e7d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.399+0000 7fe0e4c00640 1 -- 172.21.15.19:0/45062125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0015070 con 0x55b43cd8d050 2024-04-25T11:36:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.399+0000 7fe0e4c00640 1 -- 172.21.15.19:0/45062125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e000e730 con 0x55b43cd8d050 2024-04-25T11:36:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.399+0000 7fe0e4c00640 1 -- 172.21.15.19:0/45062125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000eb20 con 0x55b43cd8d050 2024-04-25T11:36:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/45062125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 msgr2=0x55b43ce14d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce14d90 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000a4e0 tx=0x7fe0e000a9a0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/45062125 shutdown_connections 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43ce1a960 0x55b43ce13470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce14d90 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/45062125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43cd466e0 0x55b43cd84be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/45062125 >> 172.21.15.19:0/45062125 conn(0x55b43cc67ac0 msgr2=0x55b43ce1a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/45062125 shutdown_connections 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.400+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/45062125 wait complete. 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.401+0000 7fe0ed7c7f40 1 Processor -- start 2024-04-25T11:36:15.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.401+0000 7fe0ed7c7f40 1 -- start start 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0ed7c7f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43cd466e0 0x55b43cd850f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0ed7c7f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce0fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0ed7c7f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43ce1a960 0x55b43cd43e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0ed7c7f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b43cd85fe0 con 0x55b43cd8d050 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0ed7c7f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55b43cd8d850 con 0x55b43cd466e0 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0ed7c7f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55b43cd862c0 con 0x55b43ce1a960 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0e6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce0fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0e6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce0fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40180/0 (socket says 172.21.15.19:40180) 2024-04-25T11:36:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.403+0000 7fe0e6000640 1 -- 172.21.15.19:0/2824809893 learned_addr learned my addr 172.21.15.19:0/2824809893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6a00640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43cd466e0 0x55b43cd850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e7400640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43ce1a960 0x55b43cd43e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6000640 1 -- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43ce1a960 msgr2=0x55b43cd43e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6000640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43ce1a960 0x55b43cd43e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6000640 1 -- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43cd466e0 msgr2=0x55b43cd850f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6000640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43cd466e0 0x55b43cd850f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6000640 1 -- 172.21.15.19:0/2824809893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43ce12840 con 0x55b43cd8d050 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6a00640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b43cd466e0 0x55b43cd850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e7400640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b43ce1a960 0x55b43cd43e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0e6000640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce0fe40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4055040 tx=0x7fe0d4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.404+0000 7fe0df400640 1 -- 172.21.15.19:0/2824809893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d406b070 con 0x55b43cd8d050 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.405+0000 7fe0df400640 1 -- 172.21.15.19:0/2824809893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d40528d0 con 0x55b43cd8d050 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.405+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b43ce12b20 con 0x55b43cd8d050 2024-04-25T11:36:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.405+0000 7fe0df400640 1 -- 172.21.15.19:0/2824809893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4052c80 con 0x55b43cd8d050 2024-04-25T11:36:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.405+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43cd87a90 con 0x55b43cd8d050 2024-04-25T11:36:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.407+0000 7fe0df400640 1 -- 172.21.15.19:0/2824809893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe0d406e070 con 0x55b43cd8d050 2024-04-25T11:36:15.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.408+0000 7fe0df400640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c8072e90 0x7fe0c80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.408+0000 7fe0df400640 1 -- 172.21.15.19:0/2824809893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7fe0d40d7cc0 con 0x55b43cd8d050 2024-04-25T11:36:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.408+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55b43ce109c0 0x55b43ce10dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.409+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96) -- 0x55b43ce11310 con 0x55b43ce109c0 2024-04-25T11:36:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.409+0000 7fe0e6a00640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c8072e90 0x7fe0c80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.409+0000 7fe0e7400640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55b43ce109c0 0x55b43ce10dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.410+0000 7fe0e7400640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55b43ce109c0 0x55b43ce10dd0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:15.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.410+0000 7fe0e6a00640 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c8072e90 0x7fe0c80753a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000f60 tx=0x7fe0d8001500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:15.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0e7400640 1 -- 172.21.15.19:0/2824809893 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000000 [delete] v96'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe0e000e070 con 0x55b43ce109c0 2024-04-25T11:36:15.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55b43ce109c0 msgr2=0x55b43ce10dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55b43ce109c0 0x55b43ce10dd0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c8072e90 msgr2=0x7fe0c80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c8072e90 0x7fe0c80753a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8000f60 tx=0x7fe0d8001500 comp rx=0 tx=0).stop 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 msgr2=0x55b43ce0fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0ed7c7f40 1 --2- 172.21.15.19:0/2824809893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b43cd8d050 0x55b43ce0fe40 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4055040 tx=0x7fe0d4002fb0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.413+0000 7fe0e7400640 1 -- 172.21.15.19:0/2824809893 reap_dead start 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.414+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 shutdown_connections 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.414+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 >> 172.21.15.19:0/2824809893 conn(0x55b43cc67ac0 msgr2=0x55b43ce13470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.414+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 shutdown_connections 2024-04-25T11:36:15.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.414+0000 7fe0ed7c7f40 1 -- 172.21.15.19:0/2824809893 wait complete. 2024-04-25T11:36:15.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:36:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.703+0000 7f9b7d400640 1 Processor -- start 2024-04-25T11:36:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.703+0000 7f9b7d400640 1 -- start start 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.706+0000 7f9b7d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b781441d0 0x7f9b78161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.706+0000 7f9b7d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7806cd40 0x7f9b7815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.706+0000 7f9b7d400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b78162450 con 0x7f9b78162040 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.706+0000 7f9b7d400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7806d150 con 0x7f9b781441d0 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.706+0000 7f9b7d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7806d3a0 con 0x7f9b7806cd40 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.706+0000 7f9b77e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b781441d0 0x7f9b78161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b6fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9b78162040 0x7f9b7806c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46576/0 (socket says 172.21.15.19:46576) 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b77e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9b781441d0 0x7f9b78161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40186/0 (socket says 172.21.15.19:40186) 2024-04-25T11:36:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b6fe00640 1 -- 172.21.15.19:0/3624389291 learned_addr learned my addr 172.21.15.19:0/3624389291 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b7ca00640 1 --2- 172.21.15.19:0/3624389291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7806cd40 0x7f9b7815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b77e00640 1 -- 172.21.15.19:0/3624389291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7806cd40 msgr2=0x7f9b7815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b77e00640 1 --2- 172.21.15.19:0/3624389291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7806cd40 0x7f9b7815fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b77e00640 1 -- 172.21.15.19:0/3624389291 >> v1:172.21.15.19:6789/0 conn(0x7f9b78162040 legacy=0x7f9b7806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b77e00640 1 -- 172.21.15.19:0/3624389291 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7806d680 con 0x7f9b781441d0 2024-04-25T11:36:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.707+0000 7f9b77400640 1 -- 172.21.15.19:0/3624389291 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976673536 0 0) 0x7f9b60001110 con 0x7f9b78162040 2024-04-25T11:36:15.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.709+0000 7f9b77e00640 1 --2- 172.21.15.19:0/3624389291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b78161b00 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f9b680097a0 tx=0x7f9b6802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe15a928880dda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.709+0000 7f9b77400640 1 -- 172.21.15.19:0/3624389291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68037070 con 0x7f9b781441d0 2024-04-25T11:36:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.709+0000 7f9b77400640 1 -- 172.21.15.19:0/3624389291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6802ae50 con 0x7f9b781441d0 2024-04-25T11:36:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.709+0000 7f9b77400640 1 -- 172.21.15.19:0/3624389291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68032660 con 0x7f9b781441d0 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 -- 172.21.15.19:0/3624389291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 msgr2=0x7f9b78161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 --2- 172.21.15.19:0/3624389291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b78161b00 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f9b680097a0 tx=0x7f9b6802a100 comp rx=0 tx=0).stop 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 -- 172.21.15.19:0/3624389291 shutdown_connections 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 --2- 172.21.15.19:0/3624389291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b7806cd40 0x7f9b7815fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 --2- 172.21.15.19:0/3624389291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b78161b00 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 -- 172.21.15.19:0/3624389291 >> 172.21.15.19:0/3624389291 conn(0x7f9b78072f40 msgr2=0x7f9b78073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 -- 172.21.15.19:0/3624389291 shutdown_connections 2024-04-25T11:36:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.710+0000 7f9b7d400640 1 -- 172.21.15.19:0/3624389291 wait complete. 2024-04-25T11:36:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.713+0000 7f9b7d400640 1 Processor -- start 2024-04-25T11:36:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.713+0000 7f9b7d400640 1 -- start start 2024-04-25T11:36:15.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.714+0000 7f9b7d400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7806cd40 0x7f9b7815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.714+0000 7f9b7d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b780b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.714+0000 7f9b7d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b78162040 0x7f9b78140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.714+0000 7f9b7d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781445e0 con 0x7f9b781441d0 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.714+0000 7f9b7d400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7806dc40 con 0x7f9b7806cd40 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.714+0000 7f9b7d400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7815ff80 con 0x7f9b78162040 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b780b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b780b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40200/0 (socket says 172.21.15.19:40200) 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 -- 172.21.15.19:0/747420684 learned_addr learned my addr 172.21.15.19:0/747420684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:15.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b77e00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7806cd40 0x7f9b7815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b7ca00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b78162040 0x7f9b78140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 -- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b78162040 msgr2=0x7f9b78140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b78162040 0x7f9b78140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 -- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7806cd40 msgr2=0x7f9b7815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7806cd40 0x7f9b7815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.715+0000 7f9b6fe00640 1 -- 172.21.15.19:0/747420684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b780b5350 con 0x7f9b781441d0 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.716+0000 7f9b6fe00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b780b4690 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000c420 tx=0x7f9b6000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.716+0000 7f9b74c00640 1 -- 172.21.15.19:0/747420684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600074e0 con 0x7f9b781441d0 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.716+0000 7f9b74c00640 1 -- 172.21.15.19:0/747420684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b600047e0 con 0x7f9b781441d0 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.716+0000 7f9b74c00640 1 -- 172.21.15.19:0/747420684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60004c10 con 0x7f9b781441d0 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.716+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b781f9d00 con 0x7f9b781441d0 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.716+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b780b65a0 con 0x7f9b781441d0 2024-04-25T11:36:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.717+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b78076c60 con 0x7f9b781441d0 2024-04-25T11:36:15.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.719+0000 7f9b74c00640 1 -- 172.21.15.19:0/747420684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9b60011070 con 0x7f9b781441d0 2024-04-25T11:36:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.720+0000 7f9b74c00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b540744b0 0x7f9b540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:15.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.720+0000 7f9b74c00640 1 -- 172.21.15.19:0/747420684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6053+0+0 (secure 0 0 0) 0x7f9b60095070 con 0x7f9b781441d0 2024-04-25T11:36:15.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.721+0000 7f9b77e00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b540744b0 0x7f9b540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:15.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.722+0000 7f9b77e00640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b540744b0 0x7f9b540769c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b68034000 tx=0x7f9b6802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:15.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.726+0000 7f9b74c00640 1 -- 172.21.15.19:0/747420684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b78162450 con 0x7f9b781441d0 2024-04-25T11:36:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:15.970+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9b780b33e0 con 0x7f9b781441d0 2024-04-25T11:36:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.423+0000 7f9b74c00640 1 -- 172.21.15.19:0/747420684 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v117) ==== 162+0+0 (secure 0 0 0) 0x7f9b6005f600 con 0x7f9b781441d0 2024-04-25T11:36:16.412 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b540744b0 msgr2=0x7f9b540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b540744b0 0x7f9b540769c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9b68034000 tx=0x7f9b6802e000 comp rx=0 tx=0).stop 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 msgr2=0x7f9b780b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b780b4690 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000c420 tx=0x7f9b6000c8e0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 shutdown_connections 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9b540744b0 0x7f9b540769c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b78162040 0x7f9b78140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b781441d0 0x7f9b780b4690 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 --2- 172.21.15.19:0/747420684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b7806cd40 0x7f9b7815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 >> 172.21.15.19:0/747420684 conn(0x7f9b78072f40 msgr2=0x7f9b781603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:16.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 shutdown_connections 2024-04-25T11:36:16.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.428+0000 7f9b7d400640 1 -- 172.21.15.19:0/747420684 wait complete. 2024-04-25T11:36:16.431 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:16.442+0000 7fa01aa00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-04-25T11:36:16.431 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:16.443+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:36:16.432 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:16.443+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:36:16.441 INFO:tasks.ceph.mds.a.smithi019.stderr: -13> 2024-04-25T11:36:16.442+0000 7fa01aa00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-04-25T11:36:16.455 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:16.572 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:36:16.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.731+0000 7f07e8a00640 1 Processor -- start 2024-04-25T11:36:16.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.731+0000 7f07e8a00640 1 -- start start 2024-04-25T11:36:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.733+0000 7f07e8a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f07e40cf830 0x7f07e415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.733+0000 7f07e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d1540 0x7f07e40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.733+0000 7f07e8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e40d1e90 con 0x7f07e40ce6c0 2024-04-25T11:36:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.733+0000 7f07e8a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f07e415ecb0 con 0x7f07e40cf830 2024-04-25T11:36:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.733+0000 7f07e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07e415ef90 con 0x7f07e40d1540 2024-04-25T11:36:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.734+0000 7f07e3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d1540 0x7f07e40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.734+0000 7f07e3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d1540 0x7f07e40d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:36292/0 (socket says 172.21.15.19:36292) 2024-04-25T11:36:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.734+0000 7f07e3200640 1 -- 172.21.15.19:0/2485481999 learned_addr learned my addr 172.21.15.19:0/2485481999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.734+0000 7f07e2800640 1 --2- 172.21.15.19:0/2485481999 >> v2:172.21.15.169:3300/0 conn(0x7f07e40cf830 0x7f07e415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.735+0000 7f07e3200640 1 -- 172.21.15.19:0/2485481999 >> v2:172.21.15.169:3300/0 conn(0x7f07e40cf830 msgr2=0x7f07e415e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.735+0000 7f07e3200640 1 --2- 172.21.15.19:0/2485481999 >> v2:172.21.15.169:3300/0 conn(0x7f07e40cf830 0x7f07e415e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.735+0000 7f07e3200640 1 -- 172.21.15.19:0/2485481999 >> v1:172.21.15.169:6789/0 conn(0x7f07e40ce6c0 legacy=0x7f07e40b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.735+0000 7f07e3200640 1 -- 172.21.15.19:0/2485481999 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e415f270 con 0x7f07e40d1540 2024-04-25T11:36:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.736+0000 7f07e3200640 1 --2- 172.21.15.19:0/2485481999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d1540 0x7f07e40d1950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009e50 tx=0x7f07dc02a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e105b11af54fa1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.737+0000 7f07e1e00640 1 -- 172.21.15.19:0/2485481999 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc036070 con 0x7f07e40d1540 2024-04-25T11:36:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.737+0000 7f07e1e00640 1 -- 172.21.15.19:0/2485481999 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc031600 con 0x7f07e40d1540 2024-04-25T11:36:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.737+0000 7f07e1e00640 1 -- 172.21.15.19:0/2485481999 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0319d0 con 0x7f07e40d1540 2024-04-25T11:36:16.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.737+0000 7f07e8a00640 1 -- 172.21.15.19:0/2485481999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d1540 msgr2=0x7f07e40d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.737+0000 7f07e8a00640 1 --2- 172.21.15.19:0/2485481999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d1540 0x7f07e40d1950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009e50 tx=0x7f07dc02a730 comp rx=0 tx=0).stop 2024-04-25T11:36:16.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.738+0000 7f07e8a00640 1 -- 172.21.15.19:0/2485481999 shutdown_connections 2024-04-25T11:36:16.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.738+0000 7f07e8a00640 1 --2- 172.21.15.19:0/2485481999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d1540 0x7f07e40d1950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.738+0000 7f07e8a00640 1 --2- 172.21.15.19:0/2485481999 >> v2:172.21.15.169:3300/0 conn(0x7f07e40cf830 0x7f07e415e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.738+0000 7f07e8a00640 1 -- 172.21.15.19:0/2485481999 >> 172.21.15.19:0/2485481999 conn(0x7f07e40b5460 msgr2=0x7f07e40b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:16.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.738+0000 7f07e8a00640 1 -- 172.21.15.19:0/2485481999 shutdown_connections 2024-04-25T11:36:16.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.738+0000 7f07e8a00640 1 -- 172.21.15.19:0/2485481999 wait complete. 2024-04-25T11:36:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.740+0000 7f07e8a00640 1 Processor -- start 2024-04-25T11:36:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.740+0000 7f07e8a00640 1 -- start start 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40ce6c0 0x7f07e406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e40cf830 0x7f07e40b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d1540 0x7f07e40c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e8a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40cead0 con 0x7f07e40d1540 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07e408cd60 con 0x7f07e40cf830 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07e408d280 con 0x7f07e40ce6c0 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d1540 0x7f07e40c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d1540 0x7f07e40c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40224/0 (socket says 172.21.15.19:40224) 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.742+0000 7f07e3200640 1 -- 172.21.15.19:0/328864820 learned_addr learned my addr 172.21.15.19:0/328864820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07e2800640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40ce6c0 0x7f07e406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07d9e00640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e40cf830 0x7f07e40b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07e3200640 1 -- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40ce6c0 msgr2=0x7f07e406f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07e3200640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40ce6c0 0x7f07e406f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07e3200640 1 -- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e40cf830 msgr2=0x7f07e40b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07e3200640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e40cf830 0x7f07e40b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07e3200640 1 -- 172.21.15.19:0/328864820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e406f6d0 con 0x7f07e40d1540 2024-04-25T11:36:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.743+0000 7f07e3200640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d1540 0x7f07e40c9440 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f07dc034040 tx=0x7f07dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.744+0000 7f07db400640 1 -- 172.21.15.19:0/328864820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc03f020 con 0x7f07e40d1540 2024-04-25T11:36:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.744+0000 7f07db400640 1 -- 172.21.15.19:0/328864820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc037040 con 0x7f07e40d1540 2024-04-25T11:36:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.744+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e40b2ca0 con 0x7f07e40d1540 2024-04-25T11:36:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.744+0000 7f07db400640 1 -- 172.21.15.19:0/328864820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc036070 con 0x7f07e40d1540 2024-04-25T11:36:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.744+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e40b3260 con 0x7f07e40d1540 2024-04-25T11:36:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.745+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e4050740 con 0x7f07e40d1540 2024-04-25T11:36:16.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.746+0000 7f07db400640 1 -- 172.21.15.19:0/328864820 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f07dc042020 con 0x7f07e40d1540 2024-04-25T11:36:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.747+0000 7f07db400640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.748+0000 7f07db400640 1 -- 172.21.15.19:0/328864820 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7f07dc0b5810 con 0x7f07e40d1540 2024-04-25T11:36:16.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.748+0000 7f07e2800640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:16.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.750+0000 7f07e2800640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f07e40d34a0 tx=0x7f07d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:16.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.753+0000 7f07db400640 1 -- 172.21.15.19:0/328864820 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07dc049030 con 0x7f07e40d1540 2024-04-25T11:36:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:16.998+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07e40b2500 con 0x7f07e40d1540 2024-04-25T11:36:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.000+0000 7f07db400640 1 -- 172.21.15.19:0/328864820 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+4351 (secure 0 0 0) 0x7f07dc07fde0 con 0x7f07e40d1540 2024-04-25T11:36:16.988 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:16.989 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":119,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:16.445088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 119 2024-04-25T11:36:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.002+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 msgr2=0x7f07c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.002+0000 7f07e8a00640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f07e40d34a0 tx=0x7f07d4008040 comp rx=0 tx=0).stop 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.002+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d1540 msgr2=0x7f07e40c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.002+0000 7f07e8a00640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d1540 0x7f07e40c9440 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f07dc034040 tx=0x7f07dc004a40 comp rx=0 tx=0).stop 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 shutdown_connections 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e40cf830 0x7f07e40b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d1540 0x7f07e40c9440 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 --2- 172.21.15.19:0/328864820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40ce6c0 0x7f07e406f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 >> 172.21.15.19:0/328864820 conn(0x7f07e40b5460 msgr2=0x7f07e40b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 shutdown_connections 2024-04-25T11:36:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.003+0000 7f07e8a00640 1 -- 172.21.15.19:0/328864820 wait complete. 2024-04-25T11:36:17.019 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:36:17.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.302+0000 7f12cb200640 1 Processor -- start 2024-04-25T11:36:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.303+0000 7f12cb200640 1 -- start start 2024-04-25T11:36:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.305+0000 7f12cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b0800 0x7f12cc070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.305+0000 7f12cb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12cc0707d0 0x7f12cc1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.305+0000 7f12cb200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12cc070be0 con 0x7f12cc0cf9e0 2024-04-25T11:36:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.305+0000 7f12cb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f12cc0cffc0 con 0x7f12cc0707d0 2024-04-25T11:36:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.305+0000 7f12cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc0d02a0 con 0x7f12cc0b0800 2024-04-25T11:36:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.306+0000 7f12c9e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f12cc0cf9e0 0x7f12cc161090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46602/0 (socket says 172.21.15.19:46602) 2024-04-25T11:36:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.306+0000 7f12c9e00640 1 -- 172.21.15.19:0/277097298 learned_addr learned my addr 172.21.15.19:0/277097298 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.306+0000 7f12c1400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b0800 0x7f12cc070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.306+0000 7f12ca800640 1 --2- 172.21.15.19:0/277097298 >> v2:172.21.15.169:3300/0 conn(0x7f12cc0707d0 0x7f12cc1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.306+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826667749 0 0) 0x7f12bc001490 con 0x7f12cc0cf9e0 2024-04-25T11:36:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.306+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12b0003520 con 0x7f12cc0cf9e0 2024-04-25T11:36:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3165749131 0 0) 0x7f12b0003520 con 0x7f12cc0cf9e0 2024-04-25T11:36:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b0800 msgr2=0x7f12cc070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b0800 0x7f12cc070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 >> v2:172.21.15.169:3300/0 conn(0x7f12cc0707d0 msgr2=0x7f12cc1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 --2- 172.21.15.19:0/277097298 >> v2:172.21.15.169:3300/0 conn(0x7f12cc0707d0 0x7f12cc1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc0d0580 con 0x7f12cc0cf9e0 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f12bc001be0 con 0x7f12cc0cf9e0 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 >> v1:172.21.15.19:6789/0 conn(0x7f12cc0cf9e0 legacy=0x7f12cc161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b00039b0 0x7f12b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12b0004800 0x7f12b0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b0800 0x7f12b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc0cffc0 con 0x7f12b0004800 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc0d02a0 con 0x7f12b00039b0 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12b000bb50 con 0x7f12cc0b0800 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12ca800640 1 --2- 172.21.15.19:0/277097298 >> v2:172.21.15.169:3300/0 conn(0x7f12cc0707d0 0x7f12cc1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:36:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c1400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12b0004800 0x7f12b0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c1400640 1 -- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b0800 msgr2=0x7f12b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c1400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b0800 0x7f12b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c1400640 1 -- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b00039b0 msgr2=0x7f12b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c1400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b00039b0 0x7f12b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.307+0000 7f12c1400640 1 -- 172.21.15.19:0/277097298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b0003520 con 0x7f12b0004800 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.308+0000 7f12c1400640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12b0004800 0x7f12b0004c10 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009ac0 tx=0x7f12b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3df72313609d10a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.309+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4036070 con 0x7f12b0004800 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.309+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b4031400 con 0x7f12b0004800 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.309+0000 7f12c9400640 1 -- 172.21.15.19:0/277097298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4009690 con 0x7f12b0004800 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.309+0000 7f12cb200640 1 -- 172.21.15.19:0/277097298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12b0004800 msgr2=0x7f12b0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.309+0000 7f12cb200640 1 --2- 172.21.15.19:0/277097298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12b0004800 0x7f12b0004c10 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009ac0 tx=0x7f12b0005830 comp rx=0 tx=0).stop 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.309+0000 7f12ca800640 1 -- 172.21.15.19:0/277097298 reap_dead start 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.310+0000 7f12cb200640 1 -- 172.21.15.19:0/277097298 shutdown_connections 2024-04-25T11:36:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.310+0000 7f12cb200640 1 -- 172.21.15.19:0/277097298 >> 172.21.15.19:0/277097298 conn(0x7f12cc0d1720 msgr2=0x7f12cc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:17.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.310+0000 7f12cb200640 1 -- 172.21.15.19:0/277097298 shutdown_connections 2024-04-25T11:36:17.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.310+0000 7f12cb200640 1 -- 172.21.15.19:0/277097298 wait complete. 2024-04-25T11:36:17.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.312+0000 7f12cb200640 1 Processor -- start 2024-04-25T11:36:17.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.312+0000 7f12cb200640 1 -- start start 2024-04-25T11:36:17.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12cb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b0004800 0x7f12cc1472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b4f30 0x7f12cc0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12cb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12cc1482d0 0x7f12cc1486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12cb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc0b0d90 con 0x7f12cc1482d0 2024-04-25T11:36:17.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12cb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc0b0c10 con 0x7f12b0004800 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc070290 con 0x7f12cc0b4f30 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12cc1482d0 0x7f12cc1486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12cc1482d0 0x7f12cc1486e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40236/0 (socket says 172.21.15.19:40236) 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12ca800640 1 -- 172.21.15.19:0/3483571170 learned_addr learned my addr 172.21.15.19:0/3483571170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12c1400640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b4f30 0x7f12cc0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.314+0000 7f12c9e00640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b0004800 0x7f12cc1472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12ca800640 1 -- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b4f30 msgr2=0x7f12cc0b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12ca800640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b4f30 0x7f12cc0b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12ca800640 1 -- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b0004800 msgr2=0x7f12cc1472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12ca800640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b0004800 0x7f12cc1472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12ca800640 1 -- 172.21.15.19:0/3483571170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc0b5c60 con 0x7f12cc1482d0 2024-04-25T11:36:17.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12ca800640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12cc1482d0 0x7f12cc1486e0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006fd0 tx=0x7f12c400e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12c2a00640 1 -- 172.21.15.19:0/3483571170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c400f040 con 0x7f12cc1482d0 2024-04-25T11:36:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12c2a00640 1 -- 172.21.15.19:0/3483571170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c4019070 con 0x7f12cc1482d0 2024-04-25T11:36:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12c2a00640 1 -- 172.21.15.19:0/3483571170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4013870 con 0x7f12cc1482d0 2024-04-25T11:36:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc0b5f40 con 0x7f12cc1482d0 2024-04-25T11:36:17.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.315+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc147ad0 con 0x7f12cc1482d0 2024-04-25T11:36:17.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.317+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc0aeb70 con 0x7f12cc1482d0 2024-04-25T11:36:17.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.318+0000 7f12c2a00640 1 -- 172.21.15.19:0/3483571170 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f12c4013a10 con 0x7f12cc1482d0 2024-04-25T11:36:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.319+0000 7f12c2a00640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a8074530 0x7f12a8076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.319+0000 7f12c2a00640 1 -- 172.21.15.19:0/3483571170 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7f12c4094e70 con 0x7f12cc1482d0 2024-04-25T11:36:17.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.320+0000 7f12c9e00640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a8074530 0x7f12a8076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.321+0000 7f12c9e00640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a8074530 0x7f12a8076a40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f12bc001670 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:17.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.324+0000 7f12c2a00640 1 -- 172.21.15.19:0/3483571170 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12cc0aeb70 con 0x7f12cc1482d0 2024-04-25T11:36:17.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.574+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12cc06fb60 con 0x7f12cc1482d0 2024-04-25T11:36:17.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.576+0000 7f12c2a00640 1 -- 172.21.15.19:0/3483571170 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5177 (secure 0 0 0) 0x7f12c405f430 con 0x7f12cc1482d0 2024-04-25T11:36:17.565 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:17.565 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2645472884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2645472884},{"type":"v1","addr":"172.21.15.19:6835","nonce":2645472884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:16.445088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:17.565 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 120 2024-04-25T11:36:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a8074530 msgr2=0x7f12a8076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a8074530 0x7f12a8076a40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f12bc001670 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12cc1482d0 msgr2=0x7f12cc1486e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12cc1482d0 0x7f12cc1486e0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f12c4006fd0 tx=0x7f12c400e830 comp rx=0 tx=0).stop 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 shutdown_connections 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a8074530 0x7f12a8076a40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12cc1482d0 0x7f12cc1486e0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12cc0b4f30 0x7f12cc0b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 --2- 172.21.15.19:0/3483571170 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12b0004800 0x7f12cc1472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 >> 172.21.15.19:0/3483571170 conn(0x7f12cc0d1720 msgr2=0x7f12cc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 shutdown_connections 2024-04-25T11:36:17.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.579+0000 7f12cb200640 1 -- 172.21.15.19:0/3483571170 wait complete. 2024-04-25T11:36:17.598 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:36:17.599 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.878+0000 7f8f23200640 1 Processor -- start 2024-04-25T11:36:17.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.878+0000 7f8f23200640 1 -- start start 2024-04-25T11:36:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.880+0000 7f8f23200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f240d2cb0 0x7f8f24160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.880+0000 7f8f23200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.880+0000 7f8f23200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f240b7940 con 0x7f8f240d50e0 2024-04-25T11:36:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.880+0000 7f8f23200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8f24161280 con 0x7f8f240d2cb0 2024-04-25T11:36:17.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.880+0000 7f8f23200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f240d4600 con 0x7f8f24160b90 2024-04-25T11:36:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.881+0000 7f8f21400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f240d2cb0 0x7f8f24160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.881+0000 7f8f22800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.881+0000 7f8f21e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f8f240d50e0 0x7f8f240d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:33668/0 (socket says 172.21.15.19:33668) 2024-04-25T11:36:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.881+0000 7f8f21e00640 1 -- 172.21.15.19:0/3643794308 learned_addr learned my addr 172.21.15.19:0/3643794308 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:17.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.881+0000 7f8f21400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8f240d2cb0 0x7f8f24160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:46416/0 (socket says 172.21.15.19:46416) 2024-04-25T11:36:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.882+0000 7f8f22800640 1 -- 172.21.15.19:0/3643794308 >> v2:172.21.15.169:3300/0 conn(0x7f8f240d2cb0 msgr2=0x7f8f24160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.882+0000 7f8f22800640 1 --2- 172.21.15.19:0/3643794308 >> v2:172.21.15.169:3300/0 conn(0x7f8f240d2cb0 0x7f8f24160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.882+0000 7f8f22800640 1 -- 172.21.15.19:0/3643794308 >> v1:172.21.15.169:6789/0 conn(0x7f8f240d50e0 legacy=0x7f8f240d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.882+0000 7f8f22800640 1 -- 172.21.15.19:0/3643794308 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f240d48e0 con 0x7f8f24160b90 2024-04-25T11:36:17.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.882+0000 7f8f21400640 1 --2- 172.21.15.19:0/3643794308 >> v2:172.21.15.169:3300/0 conn(0x7f8f240d2cb0 0x7f8f24160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:36:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.883+0000 7f8f22800640 1 --2- 172.21.15.19:0/3643794308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24161810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097a0 tx=0x7f8f1402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82064fd5d0afa521 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.884+0000 7f8f20a00640 1 -- 172.21.15.19:0/3643794308 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14037070 con 0x7f8f24160b90 2024-04-25T11:36:17.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.884+0000 7f8f20a00640 1 -- 172.21.15.19:0/3643794308 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14032440 con 0x7f8f24160b90 2024-04-25T11:36:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.884+0000 7f8f20a00640 1 -- 172.21.15.19:0/3643794308 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14032810 con 0x7f8f24160b90 2024-04-25T11:36:17.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 -- 172.21.15.19:0/3643794308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 msgr2=0x7f8f24161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 --2- 172.21.15.19:0/3643794308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24161810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f140097a0 tx=0x7f8f1402a180 comp rx=0 tx=0).stop 2024-04-25T11:36:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 -- 172.21.15.19:0/3643794308 shutdown_connections 2024-04-25T11:36:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 --2- 172.21.15.19:0/3643794308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24161810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 --2- 172.21.15.19:0/3643794308 >> v2:172.21.15.169:3300/0 conn(0x7f8f240d2cb0 0x7f8f24160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 -- 172.21.15.19:0/3643794308 >> 172.21.15.19:0/3643794308 conn(0x7f8f2406c5c0 msgr2=0x7f8f2406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 -- 172.21.15.19:0/3643794308 shutdown_connections 2024-04-25T11:36:17.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.885+0000 7f8f23200640 1 -- 172.21.15.19:0/3643794308 wait complete. 2024-04-25T11:36:17.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.887+0000 7f8f23200640 1 Processor -- start 2024-04-25T11:36:17.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.887+0000 7f8f23200640 1 -- start start 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f23200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f240d2cb0 0x7f8f241e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f23200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f240d50e0 0x7f8f24153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f23200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f23200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24161560 con 0x7f8f240d2cb0 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f23200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24160fa0 con 0x7f8f240d50e0 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f23200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f240d54f0 con 0x7f8f24160b90 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f240d2cb0 0x7f8f241e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f240d2cb0 0x7f8f241e7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40244/0 (socket says 172.21.15.19:40244) 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 -- 172.21.15.19:0/40980406 learned_addr learned my addr 172.21.15.19:0/40980406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21400640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f240d50e0 0x7f8f24153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 -- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 msgr2=0x7f8f24153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 -- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f240d50e0 msgr2=0x7f8f24153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f240d50e0 0x7f8f24153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.889+0000 7f8f21e00640 1 -- 172.21.15.19:0/40980406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f24155ac0 con 0x7f8f240d2cb0 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.890+0000 7f8f21e00640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f240d2cb0 0x7f8f241e7ca0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c3f0 tx=0x7f8f1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.890+0000 7f8f1a000640 1 -- 172.21.15.19:0/40980406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c017070 con 0x7f8f240d2cb0 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.890+0000 7f8f1a000640 1 -- 172.21.15.19:0/40980406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c005ce0 con 0x7f8f240d2cb0 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.890+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f241f67b0 con 0x7f8f240d2cb0 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.890+0000 7f8f1a000640 1 -- 172.21.15.19:0/40980406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0072f0 con 0x7f8f240d2cb0 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.890+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241f6d70 con 0x7f8f240d2cb0 2024-04-25T11:36:17.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.891+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24050740 con 0x7f8f240d2cb0 2024-04-25T11:36:17.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.892+0000 7f8f1a000640 1 -- 172.21.15.19:0/40980406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f1c01a070 con 0x7f8f240d2cb0 2024-04-25T11:36:17.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.893+0000 7f8f1a000640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f000744b0 0x7f8f000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:17.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.894+0000 7f8f1a000640 1 -- 172.21.15.19:0/40980406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7f8f1c094750 con 0x7f8f240d2cb0 2024-04-25T11:36:17.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.894+0000 7f8f21400640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f000744b0 0x7f8f000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:17.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.895+0000 7f8f21400640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f000744b0 0x7f8f000769c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8f241f7830 tx=0x7f8f0c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:17.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:17.898+0000 7f8f1a000640 1 -- 172.21.15.19:0/40980406 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f240b7940 con 0x7f8f240d2cb0 2024-04-25T11:36:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.137+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f240b8aa0 con 0x7f8f240d2cb0 2024-04-25T11:36:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.138+0000 7f8f1a000640 1 -- 172.21.15.19:0/40980406 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5177 (secure 0 0 0) 0x7f8f1c05ed20 con 0x7f8f240d2cb0 2024-04-25T11:36:18.127 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:18.127 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2645472884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2645472884},{"type":"v1","addr":"172.21.15.19:6835","nonce":2645472884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:16.445088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 120 2024-04-25T11:36:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f000744b0 msgr2=0x7f8f000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f000744b0 0x7f8f000769c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8f241f7830 tx=0x7f8f0c00b040 comp rx=0 tx=0).stop 2024-04-25T11:36:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f240d2cb0 msgr2=0x7f8f241e7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f240d2cb0 0x7f8f241e7ca0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c3f0 tx=0x7f8f1c00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 shutdown_connections 2024-04-25T11:36:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f000744b0 0x7f8f000769c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f240d50e0 0x7f8f24153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f24160b90 0x7f8f24153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 --2- 172.21.15.19:0/40980406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f240d2cb0 0x7f8f241e7ca0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 >> 172.21.15.19:0/40980406 conn(0x7f8f2406c5c0 msgr2=0x7f8f240b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 shutdown_connections 2024-04-25T11:36:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.141+0000 7f8f23200640 1 -- 172.21.15.19:0/40980406 wait complete. 2024-04-25T11:36:18.155 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2024-04-25T11:36:18.155 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2024-04-25T11:36:18.155 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:18.155 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:18.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:36:18.207 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:36:18.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:18.207 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:36:18.306 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:36:18.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:36:18.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:36:18.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.609+0000 7f92eae00640 1 Processor -- start 2024-04-25T11:36:18.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.610+0000 7f92eae00640 1 -- start start 2024-04-25T11:36:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92eae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e406c5c0 0x7f92e40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92eae00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92e4144370 0x7f92e40b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92eae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e40b50e0 con 0x7f92e40b4cd0 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92eae00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f92e40d1230 con 0x7f92e4144370 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92eae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92e40d06b0 con 0x7f92e406c5c0 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92e9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92e4144370 0x7f92e40b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92e9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f92e4144370 0x7f92e40b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40254/0 (socket says 172.21.15.19:40254) 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.612+0000 7f92e9000640 1 -- 172.21.15.19:0/3084898509 learned_addr learned my addr 172.21.15.19:0/3084898509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.613+0000 7f92e9000640 1 -- 172.21.15.19:0/3084898509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e406c5c0 msgr2=0x7f92e40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.613+0000 7f92e9000640 1 --2- 172.21.15.19:0/3084898509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e406c5c0 0x7f92e40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.613+0000 7f92e9000640 1 -- 172.21.15.19:0/3084898509 >> v1:172.21.15.169:6789/0 conn(0x7f92e40b4cd0 legacy=0x7f92e4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:18.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.613+0000 7f92e9000640 1 -- 172.21.15.19:0/3084898509 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e40d0990 con 0x7f92e4144370 2024-04-25T11:36:18.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.614+0000 7f92e9000640 1 --2- 172.21.15.19:0/3084898509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e4144370 0x7f92e40b4790 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31726c33d874230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:18.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.614+0000 7f92e3e00640 1 -- 172.21.15.19:0/3084898509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0037070 con 0x7f92e4144370 2024-04-25T11:36:18.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.614+0000 7f92e3e00640 1 -- 172.21.15.19:0/3084898509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d0032440 con 0x7f92e4144370 2024-04-25T11:36:18.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.614+0000 7f92e3e00640 1 -- 172.21.15.19:0/3084898509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0032810 con 0x7f92e4144370 2024-04-25T11:36:18.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 -- 172.21.15.19:0/3084898509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e4144370 msgr2=0x7f92e40b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:18.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 --2- 172.21.15.19:0/3084898509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e4144370 0x7f92e40b4790 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d002a210 comp rx=0 tx=0).stop 2024-04-25T11:36:18.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 -- 172.21.15.19:0/3084898509 shutdown_connections 2024-04-25T11:36:18.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 --2- 172.21.15.19:0/3084898509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e4144370 0x7f92e40b4790 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 --2- 172.21.15.19:0/3084898509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e406c5c0 0x7f92e40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 -- 172.21.15.19:0/3084898509 >> 172.21.15.19:0/3084898509 conn(0x7f92e40d1720 msgr2=0x7f92e40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:18.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 -- 172.21.15.19:0/3084898509 shutdown_connections 2024-04-25T11:36:18.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.615+0000 7f92eae00640 1 -- 172.21.15.19:0/3084898509 wait complete. 2024-04-25T11:36:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.617+0000 7f92eae00640 1 Processor -- start 2024-04-25T11:36:18.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.618+0000 7f92eae00640 1 -- start start 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92eae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92e406c5c0 0x7f92e41f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92eae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40b4cd0 0x7f92e40cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92eae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e4144370 0x7f92e415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92eae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92e40d02d0 con 0x7f92e40b4cd0 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92eae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f92e40d0f50 con 0x7f92e406c5c0 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92eae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92e40d0450 con 0x7f92e4144370 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92e9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40b4cd0 0x7f92e40cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92e9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40b4cd0 0x7f92e40cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40260/0 (socket says 172.21.15.19:40260) 2024-04-25T11:36:18.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.619+0000 7f92e9000640 1 -- 172.21.15.19:0/2788058001 learned_addr learned my addr 172.21.15.19:0/2788058001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e9000640 1 -- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e4144370 msgr2=0x7f92e415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e9000640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e4144370 0x7f92e415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e9000640 1 -- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92e406c5c0 msgr2=0x7f92e41f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e9a00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92e406c5c0 0x7f92e41f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e9000640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92e406c5c0 0x7f92e41f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e9000640 1 -- 172.21.15.19:0/2788058001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e41f65a0 con 0x7f92e40b4cd0 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e9000640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40b4cd0 0x7f92e40cab10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f92d00090b0 tx=0x7f92d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:18.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e1600640 1 -- 172.21.15.19:0/2788058001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0041070 con 0x7f92e40b4cd0 2024-04-25T11:36:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e1600640 1 -- 172.21.15.19:0/2788058001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d003c890 con 0x7f92e40b4cd0 2024-04-25T11:36:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e4135e00 con 0x7f92e40b4cd0 2024-04-25T11:36:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.620+0000 7f92e1600640 1 -- 172.21.15.19:0/2788058001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0037040 con 0x7f92e40b4cd0 2024-04-25T11:36:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.621+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e41363c0 con 0x7f92e40b4cd0 2024-04-25T11:36:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.622+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e4135b70 con 0x7f92e40b4cd0 2024-04-25T11:36:18.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.623+0000 7f92e1600640 1 -- 172.21.15.19:0/2788058001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f92d0044070 con 0x7f92e40b4cd0 2024-04-25T11:36:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.624+0000 7f92e1600640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92c40744e0 0x7f92c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:18.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.624+0000 7f92e1600640 1 -- 172.21.15.19:0/2788058001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7f92d00b5f60 con 0x7f92e40b4cd0 2024-04-25T11:36:18.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.624+0000 7f92e9a00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92c40744e0 0x7f92c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:18.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.626+0000 7f92e9a00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92c40744e0 0x7f92c40769f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f92d800a770 tx=0x7f92d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:18.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.628+0000 7f92e1600640 1 -- 172.21.15.19:0/2788058001 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92e40b50e0 con 0x7f92e40b4cd0 2024-04-25T11:36:18.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:18.864+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f92e4050740 con 0x7f92e40b4cd0 2024-04-25T11:36:19.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.466+0000 7f92e1600640 1 -- 172.21.15.19:0/2788058001 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v121) ==== 161+0+0 (secure 0 0 0) 0x7f92d00805c0 con 0x7f92e40b4cd0 2024-04-25T11:36:19.458 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:36:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92c40744e0 msgr2=0x7f92c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92c40744e0 0x7f92c40769f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f92d800a770 tx=0x7f92d8008040 comp rx=0 tx=0).stop 2024-04-25T11:36:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40b4cd0 msgr2=0x7f92e40cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40b4cd0 0x7f92e40cab10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f92d00090b0 tx=0x7f92d00046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 shutdown_connections 2024-04-25T11:36:19.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92c40744e0 0x7f92c40769f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f92e4144370 0x7f92e415acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92e40b4cd0 0x7f92e40cab10 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 --2- 172.21.15.19:0/2788058001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92e406c5c0 0x7f92e41f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 >> 172.21.15.19:0/2788058001 conn(0x7f92e40d1720 msgr2=0x7f92e415c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.471+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 shutdown_connections 2024-04-25T11:36:19.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.472+0000 7f92eae00640 1 -- 172.21.15.19:0/2788058001 wait complete. 2024-04-25T11:36:19.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:36:19.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.776+0000 7f3a62200640 1 Processor -- start 2024-04-25T11:36:19.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.777+0000 7f3a62200640 1 -- start start 2024-04-25T11:36:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a62200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a5c0b4ea0 0x7f3a5c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a62200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a5c144370 0x7f3a5c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a62200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a5c0d1230 con 0x7f3a5c06c5c0 2024-04-25T11:36:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a62200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c0d06b0 con 0x7f3a5c144370 2024-04-25T11:36:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a62200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c0d0990 con 0x7f3a5c0b4ea0 2024-04-25T11:36:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a61800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a5c144370 0x7f3a5c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:19.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a60e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3a5c06c5c0 0x7f3a5c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46610/0 (socket says 172.21.15.19:46610) 2024-04-25T11:36:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.779+0000 7f3a60e00640 1 -- 172.21.15.19:0/1112927573 learned_addr learned my addr 172.21.15.19:0/1112927573 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.780+0000 7f3a61800640 1 -- 172.21.15.19:0/1112927573 >> v2:172.21.15.169:3300/0 conn(0x7f3a5c0b4ea0 msgr2=0x7f3a5c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.780+0000 7f3a5be00640 1 --2- 172.21.15.19:0/1112927573 >> v2:172.21.15.169:3300/0 conn(0x7f3a5c0b4ea0 0x7f3a5c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.780+0000 7f3a61800640 1 --2- 172.21.15.19:0/1112927573 >> v2:172.21.15.169:3300/0 conn(0x7f3a5c0b4ea0 0x7f3a5c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.780+0000 7f3a5b400640 1 -- 172.21.15.19:0/1112927573 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815489224 0 0) 0x7f3a5c0d1230 con 0x7f3a5c06c5c0 2024-04-25T11:36:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.780+0000 7f3a61800640 1 -- 172.21.15.19:0/1112927573 >> v1:172.21.15.19:6789/0 conn(0x7f3a5c06c5c0 legacy=0x7f3a5c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.780+0000 7f3a61800640 1 -- 172.21.15.19:0/1112927573 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c0d0c70 con 0x7f3a5c144370 2024-04-25T11:36:19.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.780+0000 7f3a5be00640 1 --2- 172.21.15.19:0/1112927573 >> v2:172.21.15.169:3300/0 conn(0x7f3a5c0b4ea0 0x7f3a5c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.781+0000 7f3a61800640 1 --2- 172.21.15.19:0/1112927573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15e770 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0b5550 tx=0x7f3a5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e170f1bbf677bad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.781+0000 7f3a5b400640 1 -- 172.21.15.19:0/1112927573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54037070 con 0x7f3a5c144370 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.781+0000 7f3a5b400640 1 -- 172.21.15.19:0/1112927573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54032440 con 0x7f3a5c144370 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.781+0000 7f3a5b400640 1 -- 172.21.15.19:0/1112927573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54032810 con 0x7f3a5c144370 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 -- 172.21.15.19:0/1112927573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 msgr2=0x7f3a5c15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 --2- 172.21.15.19:0/1112927573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15e770 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0b5550 tx=0x7f3a5402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 -- 172.21.15.19:0/1112927573 shutdown_connections 2024-04-25T11:36:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 --2- 172.21.15.19:0/1112927573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15e770 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 --2- 172.21.15.19:0/1112927573 >> v2:172.21.15.169:3300/0 conn(0x7f3a5c0b4ea0 0x7f3a5c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 -- 172.21.15.19:0/1112927573 >> 172.21.15.19:0/1112927573 conn(0x7f3a5c0d1720 msgr2=0x7f3a5c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 -- 172.21.15.19:0/1112927573 shutdown_connections 2024-04-25T11:36:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.782+0000 7f3a62200640 1 -- 172.21.15.19:0/1112927573 wait complete. 2024-04-25T11:36:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.785+0000 7f3a62200640 1 Processor -- start 2024-04-25T11:36:19.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.785+0000 7f3a62200640 1 -- start start 2024-04-25T11:36:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.786+0000 7f3a62200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a5c06c5c0 0x7f3a5c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.786+0000 7f3a62200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.786+0000 7f3a62200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a5c160440 0x7f3a5c142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.786+0000 7f3a62200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d0450 con 0x7f3a5c144370 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.786+0000 7f3a62200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d02d0 con 0x7f3a5c160440 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.786+0000 7f3a62200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c147760 con 0x7f3a5c06c5c0 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40272/0 (socket says 172.21.15.19:40272) 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 -- 172.21.15.19:0/4005466580 learned_addr learned my addr 172.21.15.19:0/4005466580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a61800640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a5c160440 0x7f3a5c142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a60e00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a5c06c5c0 0x7f3a5c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 -- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a5c06c5c0 msgr2=0x7f3a5c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a5c06c5c0 0x7f3a5c15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 -- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a5c160440 msgr2=0x7f3a5c142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a5c160440 0x7f3a5c142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:19.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a5be00640 1 -- 172.21.15.19:0/4005466580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c1f8ef0 con 0x7f3a5c144370 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.787+0000 7f3a60e00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a5c06c5c0 0x7f3a5c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.788+0000 7f3a5be00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15ff00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c820 tx=0x7f3a4800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.788+0000 7f3a58c00640 1 -- 172.21.15.19:0/4005466580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48015070 con 0x7f3a5c144370 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.788+0000 7f3a58c00640 1 -- 172.21.15.19:0/4005466580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a48007a30 con 0x7f3a5c144370 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.788+0000 7f3a58c00640 1 -- 172.21.15.19:0/4005466580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48007de0 con 0x7f3a5c144370 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.788+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c1f91d0 con 0x7f3a5c144370 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.788+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1f9790 con 0x7f3a5c144370 2024-04-25T11:36:19.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.789+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c160930 con 0x7f3a5c144370 2024-04-25T11:36:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.790+0000 7f3a58c00640 1 -- 172.21.15.19:0/4005466580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a48018070 con 0x7f3a5c144370 2024-04-25T11:36:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.791+0000 7f3a58c00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a3c0744e0 0x7f3a3c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.792+0000 7f3a58c00640 1 -- 172.21.15.19:0/4005466580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7f3a48095370 con 0x7f3a5c144370 2024-04-25T11:36:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.792+0000 7f3a60e00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a3c0744e0 0x7f3a3c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.793+0000 7f3a60e00640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a3c0744e0 0x7f3a3c0769f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c007fd0 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:19.797+0000 7f3a58c00640 1 -- 172.21.15.19:0/4005466580 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a4805f8b0 con 0x7f3a5c144370 2024-04-25T11:36:20.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.071+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3a5c050740 con 0x7f3a5c144370 2024-04-25T11:36:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.470+0000 7f3a58c00640 1 -- 172.21.15.19:0/4005466580 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v122) ==== 101+0+0 (secure 0 0 0) 0x7f3a4805f1b0 con 0x7f3a5c144370 2024-04-25T11:36:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.473+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a3c0744e0 msgr2=0x7f3a3c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.473+0000 7f3a62200640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a3c0744e0 0x7f3a3c0769f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c007fd0 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.473+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 msgr2=0x7f3a5c15ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.473+0000 7f3a62200640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15ff00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800c820 tx=0x7f3a4800cce0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 shutdown_connections 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a3c0744e0 0x7f3a3c0769f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a5c160440 0x7f3a5c142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a5c144370 0x7f3a5c15ff00 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 --2- 172.21.15.19:0/4005466580 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a5c06c5c0 0x7f3a5c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 >> 172.21.15.19:0/4005466580 conn(0x7f3a5c0d1720 msgr2=0x7f3a5c0b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 shutdown_connections 2024-04-25T11:36:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.474+0000 7f3a62200640 1 -- 172.21.15.19:0/4005466580 wait complete. 2024-04-25T11:36:20.491 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:36:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.689+0000 7fb86b849f40 1 Processor -- start 2024-04-25T11:36:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.689+0000 7fb86b849f40 1 -- start start 2024-04-25T11:36:20.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.690+0000 7fb86b849f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 0x561853075f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.690+0000 7fb86b849f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fd75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.690+0000 7fb86b849f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561852fd7b30 0x56185310c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.690+0000 7fb86b849f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561852f4dd20 con 0x561852fd5730 2024-04-25T11:36:20.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.690+0000 7fb86b849f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561852f51d60 con 0x7fb8480030f0 2024-04-25T11:36:20.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.690+0000 7fb86b849f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561852ff03f0 con 0x561852fd7b30 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.690+0000 7fb864000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fd75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fd75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40296/0 (socket says 172.21.15.19:40296) 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 -- 172.21.15.19:0/2448006539 learned_addr learned my addr 172.21.15.19:0/2448006539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 -- 172.21.15.19:0/2448006539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561852fd7b30 msgr2=0x56185310c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 --2- 172.21.15.19:0/2448006539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561852fd7b30 0x56185310c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 -- 172.21.15.19:0/2448006539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 msgr2=0x561853075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 --2- 172.21.15.19:0/2448006539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 0x561853075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 -- 172.21.15.19:0/2448006539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561852fd7f40 con 0x561852fd5730 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.691+0000 7fb864000640 1 --2- 172.21.15.19:0/2448006539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fd75f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb854046750 tx=0x7fb854006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121e8d44df1f792f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.692+0000 7fb862c00640 1 -- 172.21.15.19:0/2448006539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8540017b0 con 0x561852fd5730 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.692+0000 7fb862c00640 1 -- 172.21.15.19:0/2448006539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb854001950 con 0x561852fd5730 2024-04-25T11:36:20.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.692+0000 7fb862c00640 1 -- 172.21.15.19:0/2448006539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85404d8b0 con 0x561852fd5730 2024-04-25T11:36:20.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.692+0000 7fb86b849f40 1 -- 172.21.15.19:0/2448006539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 msgr2=0x561852fd75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.692+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2448006539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fd75f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb854046750 tx=0x7fb854006000 comp rx=0 tx=0).stop 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.693+0000 7fb86b849f40 1 -- 172.21.15.19:0/2448006539 shutdown_connections 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.693+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2448006539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561852fd7b30 0x56185310c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.693+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2448006539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fd75f0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.693+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2448006539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 0x561853075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.693+0000 7fb86b849f40 1 -- 172.21.15.19:0/2448006539 >> 172.21.15.19:0/2448006539 conn(0x561852ec9ac0 msgr2=0x561852fd9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.693+0000 7fb86b849f40 1 -- 172.21.15.19:0/2448006539 shutdown_connections 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.693+0000 7fb86b849f40 1 -- 172.21.15.19:0/2448006539 wait complete. 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.694+0000 7fb86b849f40 1 Processor -- start 2024-04-25T11:36:20.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.694+0000 7fb86b849f40 1 -- start start 2024-04-25T11:36:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.695+0000 7fb86b849f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fa5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.695+0000 7fb86b849f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561852fd7b30 0x5618531177a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.695+0000 7fb86b849f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 0x561852ff3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.695+0000 7fb86b849f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561852fd8bc0 con 0x561852fd5730 2024-04-25T11:36:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.695+0000 7fb86b849f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561852fd8420 con 0x7fb8480030f0 2024-04-25T11:36:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.695+0000 7fb86b849f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5618531160c0 con 0x561852fd7b30 2024-04-25T11:36:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fa5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fa5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40302/0 (socket says 172.21.15.19:40302) 2024-04-25T11:36:20.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 learned_addr learned my addr 172.21.15.19:0/2941681536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb865400640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 0x561852ff3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561852fd7b30 msgr2=0x5618531177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561852fd7b30 0x5618531177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 msgr2=0x561852ff3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8480030f0 0x561852ff3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185306b660 con 0x561852fd5730 2024-04-25T11:36:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.696+0000 7fb864a00640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fa5e20 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00ccb0 tx=0x7fb84c00a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.697+0000 7fb861800640 1 -- 172.21.15.19:0/2941681536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c010470 con 0x561852fd5730 2024-04-25T11:36:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.697+0000 7fb861800640 1 -- 172.21.15.19:0/2941681536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c015020 con 0x561852fd5730 2024-04-25T11:36:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.697+0000 7fb861800640 1 -- 172.21.15.19:0/2941681536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c010670 con 0x561852fd5730 2024-04-25T11:36:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.697+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56185306b940 con 0x561852fd5730 2024-04-25T11:36:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.697+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185306bf00 con 0x561852fd5730 2024-04-25T11:36:20.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.699+0000 7fb861800640 1 -- 172.21.15.19:0/2941681536 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb84c00e040 con 0x561852fd5730 2024-04-25T11:36:20.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.700+0000 7fb861800640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb840072e90 0x7fb8400753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.700+0000 7fb861800640 1 -- 172.21.15.19:0/2941681536 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7fb84c0940a0 con 0x561852fd5730 2024-04-25T11:36:20.689 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:36:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.701+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56185307af70 0x56185307b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.701+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185307b8c0 con 0x56185307af70 2024-04-25T11:36:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.701+0000 7fb864000640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb840072e90 0x7fb8400753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.701+0000 7fb865400640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56185307af70 0x56185307b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.703+0000 7fb864000640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb840072e90 0x7fb8400753a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb854026050 tx=0x7fb8540021f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.703+0000 7fb865400640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56185307af70 0x56185307b380 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.705+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v97'133 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.706+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x56185306a520 con 0x56185307af70 2024-04-25T11:36:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.709+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v97'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.709+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185306a8f0 con 0x56185307af70 2024-04-25T11:36:20.701 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:36:20.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.712+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v97'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.712+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185306ac90 con 0x56185307af70 2024-04-25T11:36:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.715+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v97'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.715+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185306b090 con 0x56185307af70 2024-04-25T11:36:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.718+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v97'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.718+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122130 con 0x56185307af70 2024-04-25T11:36:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.722+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v97'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.722+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531224d0 con 0x56185307af70 2024-04-25T11:36:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.725+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v97'139 uv113 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.725+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.728+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v97'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.728+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.720 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:36:20.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.731+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v97'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.731+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.734+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v97'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.734+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.737+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v97'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.738+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.741+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v97'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.741+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.744+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v97'145 uv120 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.744+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.748+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v97'146 uv146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.748+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.740 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:36:20.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.752+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v97'147 uv147 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.752+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.755+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v97'148 uv148 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.755+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.759+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v97'149 uv149 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.759+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.763+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v97'150 uv150 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.763+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.767+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v97'151 uv151 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.767+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.771+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v97'152 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.771+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.775+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v97'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.775+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.768 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:36:20.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.779+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v97'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.779+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.783+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v97'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.783+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.787+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v97'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.787+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.791+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v97'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.792+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.796+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v97'158 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.796+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.799+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v97'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.800+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.792 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:36:20.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.803+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v97'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.803+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.808+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v97'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.808+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853123880 con 0x56185307af70 2024-04-25T11:36:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.812+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v97'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.812+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853122870 con 0x56185307af70 2024-04-25T11:36:20.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.816+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v97'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.816+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561853122d80 0x5618531251a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.816+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531256e0 con 0x561853122d80 2024-04-25T11:36:20.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.817+0000 7fb864a00640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561853122d80 0x5618531251a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.818+0000 7fb864a00640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561853122d80 0x5618531251a0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.822+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v97'109 uv91 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.822+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853125a80 con 0x561853122d80 2024-04-25T11:36:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.826+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v97'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.826+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853125e20 con 0x561853122d80 2024-04-25T11:36:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.830+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v97'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.819 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:36:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.831+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531261c0 con 0x561853122d80 2024-04-25T11:36:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.835+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v97'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.835+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185307e8c0 con 0x561853122d80 2024-04-25T11:36:20.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.839+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v97'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.839+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853126aa0 con 0x561853122d80 2024-04-25T11:36:20.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.843+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v97'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.832 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:36:20.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.843+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531266d0 con 0x561853122d80 2024-04-25T11:36:20.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.848+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v97'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.848+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127390 con 0x561853122d80 2024-04-25T11:36:20.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.852+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v97'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.852+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127390 con 0x561853122d80 2024-04-25T11:36:20.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.856+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000001 [create] v97'117 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.857+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.861+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000001 [delete] v97'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.861+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.866+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v97'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.855 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:36:20.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.866+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.871+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [write 0~18117] v97'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.871+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.875+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v97'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.875+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.879+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 606.00000000 [create] v97'122 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.879+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.883+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 606.00000000 [delete] v97'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.883+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.887+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v97'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.876 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:36:20.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.887+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853128350 con 0x561853122d80 2024-04-25T11:36:20.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.891+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v97'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.891+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853128350 con 0x561853122d80 2024-04-25T11:36:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.895+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v97'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.896+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853127340 con 0x561853122d80 2024-04-25T11:36:20.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.900+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v97'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x561853122d80 2024-04-25T11:36:20.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.900+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561853127c10 0x561853128020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.900+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312a7d0 con 0x561853127c10 2024-04-25T11:36:20.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.901+0000 7fb864000640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561853127c10 0x561853128020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.903+0000 7fb864000640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561853127c10 0x561853128020 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.907+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(51 607.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:20.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.907+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x56185312ab70 con 0x561853127c10 2024-04-25T11:36:20.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.911+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(52 607.00000000 [delete] v97'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:20.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.911+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312af10 con 0x561853127c10 2024-04-25T11:36:20.904 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:36:20.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.915+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(53 607.00000000 [create] v97'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:20.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.915+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312c2c0 con 0x561853127c10 2024-04-25T11:36:20.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.919+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:20.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.920+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312c6b0 con 0x561853127c10 2024-04-25T11:36:20.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.924+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:20.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.924+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312b2b0 con 0x561853127c10 2024-04-25T11:36:20.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.928+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:20.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.928+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312b6c0 con 0x56185307af70 2024-04-25T11:36:20.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.932+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(57 mds0_openfiles.0 [create] v97'23 uv22 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.932+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x56185312b6c0 con 0x56185307af70 2024-04-25T11:36:20.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.936+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v97'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.936+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312b6c0 con 0x56185307af70 2024-04-25T11:36:20.929 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:36:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.941+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(59 mds0_openfiles.0 [create] v97'25 uv25 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.941+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312ba80 con 0x56185307af70 2024-04-25T11:36:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.945+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v97'26 uv26 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.945+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56185312beb0 0x56185312ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.945+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312ba80 con 0x56185312beb0 2024-04-25T11:36:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.946+0000 7fb865400640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56185312beb0 0x56185312ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.948+0000 7fb865400640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56185312beb0 0x56185312ef70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.952+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(61 608.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:20.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.952+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x56185312ba80 con 0x56185312beb0 2024-04-25T11:36:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.957+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(62 608.00000000 [delete] v97'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.957+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312f500 con 0x56185312beb0 2024-04-25T11:36:20.949 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:36:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.960+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(63 608.00000000 [create] v97'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:20.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.961+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853130510 con 0x56185312beb0 2024-04-25T11:36:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.964+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:20.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.964+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853130510 con 0x56185312beb0 2024-04-25T11:36:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.968+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:20.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.968+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x56185312f500 con 0x56185312beb0 2024-04-25T11:36:20.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.973+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.973+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56185312fb80 0x561853131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.973+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531325b0 con 0x56185312fb80 2024-04-25T11:36:20.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.974+0000 7fb864a00640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56185312fb80 0x561853131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:20.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.975+0000 7fb864a00640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56185312fb80 0x561853131fa0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:20.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.979+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(67 604.00000000 [create] v97'48 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.979+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:20.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.984+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(68 604.00000000 [delete] v97'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.984+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.988+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(69 604.00000000 [create] v97'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.977 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:36:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.988+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853133990 con 0x56185312fb80 2024-04-25T11:36:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.992+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v97'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.993+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853133990 con 0x56185312fb80 2024-04-25T11:36:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.997+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v97'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:20.997+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.001+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v97'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.002+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.006+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(73 500.00000000 [create] v97'54 uv47 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.006+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:20.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.010+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(74 500.00000000 [delete] v97'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:20.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.010+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:21.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.015+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(75 500.00000000 [create] v97'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:21.004 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:36:21.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.015+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:21.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.020+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v97'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:21.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.020+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853132980 con 0x56185312fb80 2024-04-25T11:36:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.024+0000 7fb864a00640 1 -- 172.21.15.19:0/2941681536 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v97'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb84c021070 con 0x56185312fb80 2024-04-25T11:36:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.024+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561853133280 0x5618531356a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.024+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853135be0 con 0x561853133280 2024-04-25T11:36:21.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.025+0000 7fb864000640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561853133280 0x5618531356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.027+0000 7fb864000640 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561853133280 0x5618531356a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.031+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(78 mds_snaptable [create] v97'22 uv21 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.031+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853135be0 con 0x561853133280 2024-04-25T11:36:21.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.036+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(79 mds_snaptable [delete] v97'23 uv23 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.036+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x561853133280 2024-04-25T11:36:21.029 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:36:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.040+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(80 mds_snaptable [create] v97'24 uv24 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.040+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x561853133280 2024-04-25T11:36:21.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.045+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v97'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.045+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x561853133280 2024-04-25T11:36:21.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.050+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v97'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.050+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x56185307af70 2024-04-25T11:36:21.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.054+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(83 605.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:21.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.054+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853136030 con 0x56185307af70 2024-04-25T11:36:21.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.058+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(84 605.00000000 [delete] v97'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:21.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.058+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x56185307af70 2024-04-25T11:36:21.051 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:36:21.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.062+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(85 605.00000000 [create] v97'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:21.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.063+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853137040 con 0x56185307af70 2024-04-25T11:36:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.067+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.067+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853137040 con 0x56185307af70 2024-04-25T11:36:21.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.071+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:21.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.071+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x56185307af70 2024-04-25T11:36:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.076+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185307af70 2024-04-25T11:36:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.076+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x561853127c10 2024-04-25T11:36:21.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.080+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(89 mds0_inotable [create] v97'44 uv38 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.080+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853136030 con 0x561853127c10 2024-04-25T11:36:21.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.085+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(90 mds0_inotable [delete] v97'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.085+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136030 con 0x561853127c10 2024-04-25T11:36:21.078 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:36:21.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.089+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(91 mds0_inotable [create] v97'46 uv46 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.089+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853126f30 con 0x561853127c10 2024-04-25T11:36:21.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.094+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v97'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.094+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853126f30 con 0x561853127c10 2024-04-25T11:36:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.099+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v97'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.099+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853126f30 con 0x561853127c10 2024-04-25T11:36:21.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.103+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(94 100.00000000 [create] v97'49 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.103+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x561853126f30 con 0x561853127c10 2024-04-25T11:36:21.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.107+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(95 100.00000000 [delete] v97'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.107+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853126f30 con 0x561853127c10 2024-04-25T11:36:21.100 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:36:21.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.112+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(96 100.00000000 [create] v97'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.112+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853126f30 con 0x561853127c10 2024-04-25T11:36:21.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.116+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v97'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.116+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853138e40 con 0x561853127c10 2024-04-25T11:36:21.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.120+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v97'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853127c10 2024-04-25T11:36:21.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.120+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.124+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v97'22 uv21 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.124+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.128+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v97'23 uv23 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.128+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.121 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:36:21.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.133+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v97'24 uv24 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.133+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.137+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v97'25 uv25 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.137+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.140+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v97'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.140+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.144+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v97'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.144+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.137 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:36:21.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.148+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v97'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.148+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.152+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v97'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.152+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.156+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v97'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.156+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x56185312beb0 2024-04-25T11:36:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.160+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v97'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85c0073f0 con 0x56185312beb0 2024-04-25T11:36:21.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.160+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.164+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v97'22 uv21 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.165+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.169+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v97'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.169+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.174+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v97'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.163 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:36:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.174+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.178+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v97'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.178+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.182+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v97'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.182+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.185+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v97'57 uv49 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.186+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.190+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v97'58 uv58 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.190+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.182 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:36:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.194+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v97'59 uv59 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.194+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.198+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v97'60 uv60 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.198+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.202+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v97'61 uv61 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.202+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.206+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v97'62 uv56 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.206+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.210+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v97'63 uv63 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.210+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.214+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v97'64 uv64 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.203 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:36:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.214+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.218+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v97'65 uv65 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.218+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.222+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v97'66 uv66 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.222+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x5618531364e0 con 0x561853133280 2024-04-25T11:36:21.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.226+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v97'67 uv67 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.226+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97) -- 0x561853136d50 con 0x561853133280 2024-04-25T11:36:21.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.230+0000 7fb864000640 1 -- 172.21.15.19:0/2941681536 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v97'68 uv68 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb85404d8b0 con 0x561853133280 2024-04-25T11:36:21.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.230+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561853122d80 msgr2=0x5618531251a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.230+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561853122d80 0x5618531251a0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.230+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56185312fb80 msgr2=0x561853131fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.230+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56185312fb80 0x561853131fa0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.230+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561853133280 msgr2=0x5618531356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.230+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561853133280 0x5618531356a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561853127c10 msgr2=0x561853128020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561853127c10 0x561853128020 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 reap_dead start 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56185307af70 msgr2=0x56185307b380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56185307af70 0x56185307b380 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56185312beb0 msgr2=0x56185312ef70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56185312beb0 0x56185312ef70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb840072e90 msgr2=0x7fb8400753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb840072e90 0x7fb8400753a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb854026050 tx=0x7fb8540021f0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 msgr2=0x561852fa5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb86b849f40 1 --2- 172.21.15.19:0/2941681536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561852fd5730 0x561852fa5e20 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00ccb0 tx=0x7fb84c00a5a0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.231+0000 7fb865400640 1 -- 172.21.15.19:0/2941681536 reap_dead start 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.232+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 shutdown_connections 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.232+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 >> 172.21.15.19:0/2941681536 conn(0x561852ec9ac0 msgr2=0x561852fd90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.232+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 shutdown_connections 2024-04-25T11:36:21.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.232+0000 7fb86b849f40 1 -- 172.21.15.19:0/2941681536 wait complete. 2024-04-25T11:36:21.249 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2024-04-25T11:36:21.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.497+0000 7fddabc7af40 1 Processor -- start 2024-04-25T11:36:21.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.497+0000 7fddabc7af40 1 -- start start 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fddabc7af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6131e0 0x5589ec6df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fddabc7af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6135f0 0x5589ec611f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fddabc7af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6ecfa0 0x5589ec6124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fddabc7af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5589ec65be70 con 0x5589ec6ecfa0 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fddabc7af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5589ec5b9d20 con 0x5589ec6135f0 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fddabc7af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5589ec65c3f0 con 0x5589ec6131e0 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fdda5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6ecfa0 0x5589ec6124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fdda5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6ecfa0 0x5589ec6124d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40308/0 (socket says 172.21.15.19:40308) 2024-04-25T11:36:21.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.499+0000 7fdda5800640 1 -- 172.21.15.19:0/4076886941 learned_addr learned my addr 172.21.15.19:0/4076886941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:21.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda4e00640 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6131e0 0x5589ec6df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdd9fe00640 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6135f0 0x5589ec611f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda5800640 1 -- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6131e0 msgr2=0x5589ec6df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda5800640 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6131e0 0x5589ec6df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda5800640 1 -- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6135f0 msgr2=0x5589ec611f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda5800640 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6135f0 0x5589ec611f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda5800640 1 -- 172.21.15.19:0/4076886941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ec643fd0 con 0x5589ec6ecfa0 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdd9fe00640 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6135f0 0x5589ec611f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda4e00640 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6131e0 0x5589ec6df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdda5800640 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6ecfa0 0x5589ec6124d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdda000a860 tx=0x7fdda000ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2319afc44e4f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.500+0000 7fdd9ea00640 1 -- 172.21.15.19:0/4076886941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda000d9c0 con 0x5589ec6ecfa0 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fdd9ea00640 1 -- 172.21.15.19:0/4076886941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda000db60 con 0x5589ec6ecfa0 2024-04-25T11:36:21.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fdd9ea00640 1 -- 172.21.15.19:0/4076886941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0012470 con 0x5589ec6ecfa0 2024-04-25T11:36:21.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fddabc7af40 1 -- 172.21.15.19:0/4076886941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6ecfa0 msgr2=0x5589ec6124d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fddabc7af40 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6ecfa0 0x5589ec6124d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdda000a860 tx=0x7fdda000ad20 comp rx=0 tx=0).stop 2024-04-25T11:36:21.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fddabc7af40 1 -- 172.21.15.19:0/4076886941 shutdown_connections 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fddabc7af40 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6135f0 0x5589ec611f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fddabc7af40 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6ecfa0 0x5589ec6124d0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.501+0000 7fddabc7af40 1 --2- 172.21.15.19:0/4076886941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6131e0 0x5589ec6df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.502+0000 7fddabc7af40 1 -- 172.21.15.19:0/4076886941 >> 172.21.15.19:0/4076886941 conn(0x5589ec535ac0 msgr2=0x5589ec641730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.502+0000 7fddabc7af40 1 -- 172.21.15.19:0/4076886941 shutdown_connections 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.502+0000 7fddabc7af40 1 -- 172.21.15.19:0/4076886941 wait complete. 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.503+0000 7fddabc7af40 1 Processor -- start 2024-04-25T11:36:21.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.503+0000 7fddabc7af40 1 -- start start 2024-04-25T11:36:21.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.504+0000 7fddabc7af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6131e0 0x5589ec644390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.504+0000 7fddabc7af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6135f0 0x5589ec6e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fddabc7af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6ecfa0 0x5589ec6e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fddabc7af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5589ec65e110 con 0x5589ec6131e0 2024-04-25T11:36:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fddabc7af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5589ec6e8e50 con 0x5589ec6ecfa0 2024-04-25T11:36:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fddabc7af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5589ec6e5ef0 con 0x5589ec6135f0 2024-04-25T11:36:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6131e0 0x5589ec644390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6131e0 0x5589ec644390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40324/0 (socket says 172.21.15.19:40324) 2024-04-25T11:36:21.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 -- 172.21.15.19:0/730600693 learned_addr learned my addr 172.21.15.19:0/730600693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:21.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdd9fe00640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6135f0 0x5589ec6e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda5800640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6ecfa0 0x5589ec6e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 -- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6135f0 msgr2=0x5589ec6e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5589ec6135f0 0x5589ec6e4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 -- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6ecfa0 msgr2=0x5589ec6e51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6ecfa0 0x5589ec6e51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda4e00640 1 -- 172.21.15.19:0/730600693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ec6dbed0 con 0x5589ec6131e0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.505+0000 7fdda5800640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5589ec6ecfa0 0x5589ec6e51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.506+0000 7fdda4e00640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6131e0 0x5589ec644390 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fdd98025b20 tx=0x7fdd98002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.506+0000 7fdd9d600640 1 -- 172.21.15.19:0/730600693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98064070 con 0x5589ec6131e0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.506+0000 7fdd9d600640 1 -- 172.21.15.19:0/730600693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd98054b30 con 0x5589ec6131e0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.506+0000 7fdd9d600640 1 -- 172.21.15.19:0/730600693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98059040 con 0x5589ec6131e0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.506+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5589ec77ad60 con 0x5589ec6131e0 2024-04-25T11:36:21.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.506+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ec77b040 con 0x5589ec6131e0 2024-04-25T11:36:21.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.509+0000 7fdd9d600640 1 -- 172.21.15.19:0/730600693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd98067070 con 0x5589ec6131e0 2024-04-25T11:36:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.509+0000 7fdd9d600640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd84072ec0 0x7fdd840753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.510+0000 7fdd9d600640 1 -- 172.21.15.19:0/730600693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7fdd980d7d40 con 0x5589ec6131e0 2024-04-25T11:36:21.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.510+0000 7fddabc7af40 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5589ec6e86d0 0x5589ec783bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.510+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97) -- 0x5589ec6448d0 con 0x5589ec6e86d0 2024-04-25T11:36:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.510+0000 7fdda5800640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5589ec6e86d0 0x5589ec783bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.510+0000 7fdd9fe00640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd84072ec0 0x7fdd840753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.511+0000 7fdda5800640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5589ec6e86d0 0x5589ec783bf0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.511+0000 7fdd9fe00640 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd84072ec0 0x7fdd840753d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x5589ec6dc860 tx=0x7fdd90001400 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fdda5800640 1 -- 172.21.15.19:0/730600693 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000001 [delete] v97'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdda0018020 con 0x5589ec6e86d0 2024-04-25T11:36:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5589ec6e86d0 msgr2=0x5589ec783bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5589ec6e86d0 0x5589ec783bf0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd84072ec0 msgr2=0x7fdd840753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd84072ec0 0x7fdd840753d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x5589ec6dc860 tx=0x7fdd90001400 comp rx=0 tx=0).stop 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6131e0 msgr2=0x5589ec644390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 --2- 172.21.15.19:0/730600693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589ec6131e0 0x5589ec644390 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fdd98025b20 tx=0x7fdd98002fb0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fdda5800640 1 -- 172.21.15.19:0/730600693 reap_dead start 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 shutdown_connections 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.514+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 >> 172.21.15.19:0/730600693 conn(0x5589ec535ac0 msgr2=0x5589ec6e36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.515+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 shutdown_connections 2024-04-25T11:36:21.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.515+0000 7fddabc7af40 1 -- 172.21.15.19:0/730600693 wait complete. 2024-04-25T11:36:21.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:36:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.801+0000 7f42ac400640 1 Processor -- start 2024-04-25T11:36:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.801+0000 7f42ac400640 1 -- start start 2024-04-25T11:36:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.803+0000 7f42ac400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42a40cd9a0 0x7f42a415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42ac400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a406dce0 0x7f42a415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42ac400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a40cdea0 con 0x7f42a40b0af0 2024-04-25T11:36:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42ac400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f42a40ceab0 con 0x7f42a40cd9a0 2024-04-25T11:36:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42ac400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42a40ce180 con 0x7f42a406dce0 2024-04-25T11:36:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42a9000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f42a40b0af0 0x7f42a406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46618/0 (socket says 172.21.15.19:46618) 2024-04-25T11:36:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42a9000640 1 -- 172.21.15.19:0/3803190565 learned_addr learned my addr 172.21.15.19:0/3803190565 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42aa400640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a406dce0 0x7f42a415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.804+0000 7f42a9a00640 1 --2- 172.21.15.19:0/3803190565 >> v2:172.21.15.169:3300/0 conn(0x7f42a40cd9a0 0x7f42a415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209464350 0 0) 0x7f42a40cdea0 con 0x7f42a40b0af0 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4288003520 con 0x7f42a40b0af0 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2191283309 0 0) 0x7f4288003520 con 0x7f42a40b0af0 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a406dce0 msgr2=0x7f42a415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42a406dce0 0x7f42a415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 >> v2:172.21.15.169:3300/0 conn(0x7f42a40cd9a0 msgr2=0x7f42a415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 --2- 172.21.15.19:0/3803190565 >> v2:172.21.15.169:3300/0 conn(0x7f42a40cd9a0 0x7f42a415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a40ce460 con 0x7f42a40b0af0 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4290001670 con 0x7f42a40b0af0 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 >> v1:172.21.15.19:6789/0 conn(0x7f42a40b0af0 legacy=0x7f42a406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42880039b0 0x7f4288003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4288004800 0x7f4288004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288005150 0x7f428800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40ceab0 con 0x7f42880039b0 2024-04-25T11:36:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40ce180 con 0x7f4288005150 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.805+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f428800bf00 con 0x7f4288004800 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9a00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42880039b0 0x7f4288003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9000640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4288004800 0x7f4288004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42aa400640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288005150 0x7f428800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9a00640 1 -- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4288004800 msgr2=0x7f4288004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9a00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4288004800 0x7f4288004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9a00640 1 -- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288005150 msgr2=0x7f428800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9a00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4288005150 0x7f428800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9a00640 1 -- 172.21.15.19:0/3803190565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4288003520 con 0x7f42880039b0 2024-04-25T11:36:21.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.806+0000 7f42a9000640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4288004800 0x7f4288004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.807+0000 7f42a9a00640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42880039b0 0x7f4288003e20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4294009fc0 tx=0x7f4288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713c5a729dbc78fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294009c10 con 0x7f42880039b0 2024-04-25T11:36:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4294036070 con 0x7f42880039b0 2024-04-25T11:36:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42a3e00640 1 -- 172.21.15.19:0/3803190565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429402d070 con 0x7f42880039b0 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42ac400640 1 -- 172.21.15.19:0/3803190565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42880039b0 msgr2=0x7f4288003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42ac400640 1 --2- 172.21.15.19:0/3803190565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42880039b0 0x7f4288003e20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4294009fc0 tx=0x7f4288005830 comp rx=0 tx=0).stop 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42aa400640 1 -- 172.21.15.19:0/3803190565 reap_dead start 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42ac400640 1 -- 172.21.15.19:0/3803190565 shutdown_connections 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42ac400640 1 -- 172.21.15.19:0/3803190565 >> 172.21.15.19:0/3803190565 conn(0x7f42a4073090 msgr2=0x7f42a40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42ac400640 1 -- 172.21.15.19:0/3803190565 shutdown_connections 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.808+0000 7f42ac400640 1 -- 172.21.15.19:0/3803190565 wait complete. 2024-04-25T11:36:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.811+0000 7f42ac400640 1 Processor -- start 2024-04-25T11:36:21.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.811+0000 7f42ac400640 1 -- start start 2024-04-25T11:36:21.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.812+0000 7f42ac400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42880039b0 0x7f42a41583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42ac400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a41319e0 0x7f42a415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42ac400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a415dad0 0x7f42a4070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42ac400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40cef40 con 0x7f42a415dad0 2024-04-25T11:36:21.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42ac400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40b0f00 con 0x7f42a41319e0 2024-04-25T11:36:21.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42ac400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42a40cf0c0 con 0x7f42880039b0 2024-04-25T11:36:21.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a415dad0 0x7f42a4070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a415dad0 0x7f42a4070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40338/0 (socket says 172.21.15.19:40338) 2024-04-25T11:36:21.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 -- 172.21.15.19:0/1984869567 learned_addr learned my addr 172.21.15.19:0/1984869567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:21.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42a9a00640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42880039b0 0x7f42a41583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42a9000640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a41319e0 0x7f42a415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 -- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42880039b0 msgr2=0x7f42a41583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42880039b0 0x7f42a41583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 -- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a41319e0 msgr2=0x7f42a415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a41319e0 0x7f42a415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.813+0000 7f42aa400640 1 -- 172.21.15.19:0/1984869567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a4141d70 con 0x7f42a415dad0 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.814+0000 7f42aa400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a415dad0 0x7f42a4070980 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f42a4148040 tx=0x7f429c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.814+0000 7f42a1600640 1 -- 172.21.15.19:0/1984869567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c00f040 con 0x7f42a415dad0 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.814+0000 7f42a1600640 1 -- 172.21.15.19:0/1984869567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c01c070 con 0x7f42a415dad0 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.814+0000 7f42a1600640 1 -- 172.21.15.19:0/1984869567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c00eae0 con 0x7f42a415dad0 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.814+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a4142050 con 0x7f42a415dad0 2024-04-25T11:36:21.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.814+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a4070180 con 0x7f42a415dad0 2024-04-25T11:36:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.816+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a4141ae0 con 0x7f42a415dad0 2024-04-25T11:36:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.817+0000 7f42a1600640 1 -- 172.21.15.19:0/1984869567 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f429c00ec80 con 0x7f42a415dad0 2024-04-25T11:36:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.818+0000 7f42a1600640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4284074500 0x7f4284076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.818+0000 7f42a1600640 1 -- 172.21.15.19:0/1984869567 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6139+0+0 (secure 0 0 0) 0x7f429c094f30 con 0x7f42a415dad0 2024-04-25T11:36:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.819+0000 7f42a9a00640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4284074500 0x7f4284076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:21.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.820+0000 7f42a9a00640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4284074500 0x7f4284076a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4294035000 tx=0x7f429402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:21.823+0000 7f42a1600640 1 -- 172.21.15.19:0/1984869567 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f429c026020 con 0x7f42a415dad0 2024-04-25T11:36:22.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.069+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f42a4068600 con 0x7f42a415dad0 2024-04-25T11:36:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.510+0000 7f42a1600640 1 -- 172.21.15.19:0/1984869567 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v123) ==== 162+0+0 (secure 0 0 0) 0x7f429c05f580 con 0x7f42a415dad0 2024-04-25T11:36:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:36:22.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4284074500 msgr2=0x7f4284076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:22.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4284074500 0x7f4284076a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4294035000 tx=0x7f429402f000 comp rx=0 tx=0).stop 2024-04-25T11:36:22.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a415dad0 msgr2=0x7f42a4070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:22.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a415dad0 0x7f42a4070980 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f42a4148040 tx=0x7f429c010c20 comp rx=0 tx=0).stop 2024-04-25T11:36:22.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 shutdown_connections 2024-04-25T11:36:22.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4284074500 0x7f4284076a10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42a41319e0 0x7f42a415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42a415dad0 0x7f42a4070980 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 --2- 172.21.15.19:0/1984869567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42880039b0 0x7f42a41583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 >> 172.21.15.19:0/1984869567 conn(0x7f42a4073090 msgr2=0x7f42a406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:22.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.513+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 shutdown_connections 2024-04-25T11:36:22.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.514+0000 7f42ac400640 1 -- 172.21.15.19:0/1984869567 wait complete. 2024-04-25T11:36:22.522 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:22.533+0000 7f08cce00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-04-25T11:36:22.523 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:22.534+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:36:22.523 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:22.534+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:36:22.537 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:36:22.533+0000 7f08cce00640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-04-25T11:36:22.541 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:22.666 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:36:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.815+0000 7f187f200640 1 Processor -- start 2024-04-25T11:36:22.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.815+0000 7f187f200640 1 -- start start 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f18800b75f0 0x7f18800ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f18801605c0 0x7f18800ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18800cfb40 con 0x7f18800cf270 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f18800cf680 con 0x7f18800b75f0 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f188006c6f0 con 0x7f18801605c0 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f18800b75f0 0x7f18800ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f18800cf270 0x7f1880169000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46644/0 (socket says 172.21.15.19:46644) 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.817+0000 7f187e800640 1 -- 172.21.15.19:0/2348788377 learned_addr learned my addr 172.21.15.19:0/2348788377 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.818+0000 7f187d400640 1 --2- 172.21.15.19:0/2348788377 >> v2:172.21.15.169:3300/0 conn(0x7f18801605c0 0x7f18800ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.818+0000 7f187de00640 1 -- 172.21.15.19:0/2348788377 >> v2:172.21.15.169:3300/0 conn(0x7f18801605c0 msgr2=0x7f18800ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.818+0000 7f187de00640 1 --2- 172.21.15.19:0/2348788377 >> v2:172.21.15.169:3300/0 conn(0x7f18801605c0 0x7f18800ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.818+0000 7f187de00640 1 -- 172.21.15.19:0/2348788377 >> v1:172.21.15.19:6789/0 conn(0x7f18800cf270 legacy=0x7f1880169000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.818+0000 7f187ca00640 1 -- 172.21.15.19:0/2348788377 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583372901 0 0) 0x7f18800cfb40 con 0x7f18800cf270 2024-04-25T11:36:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.818+0000 7f187de00640 1 -- 172.21.15.19:0/2348788377 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18800764e0 con 0x7f18800b75f0 2024-04-25T11:36:22.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.819+0000 7f187de00640 1 --2- 172.21.15.19:0/2348788377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800b75f0 0x7f18800ce7f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1880071c30 tx=0x7f186802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aca35a9d80d93e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:22.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.820+0000 7f187ca00640 1 -- 172.21.15.19:0/2348788377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868037070 con 0x7f18800b75f0 2024-04-25T11:36:22.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.820+0000 7f187ca00640 1 -- 172.21.15.19:0/2348788377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1868032440 con 0x7f18800b75f0 2024-04-25T11:36:22.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.820+0000 7f187ca00640 1 -- 172.21.15.19:0/2348788377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868032810 con 0x7f18800b75f0 2024-04-25T11:36:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.820+0000 7f187f200640 1 -- 172.21.15.19:0/2348788377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800b75f0 msgr2=0x7f18800ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.820+0000 7f187f200640 1 --2- 172.21.15.19:0/2348788377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800b75f0 0x7f18800ce7f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1880071c30 tx=0x7f186802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.821+0000 7f187f200640 1 -- 172.21.15.19:0/2348788377 shutdown_connections 2024-04-25T11:36:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.821+0000 7f187f200640 1 --2- 172.21.15.19:0/2348788377 >> v2:172.21.15.169:3300/0 conn(0x7f18801605c0 0x7f18800ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.821+0000 7f187f200640 1 --2- 172.21.15.19:0/2348788377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800b75f0 0x7f18800ce7f0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.821+0000 7f187f200640 1 -- 172.21.15.19:0/2348788377 >> 172.21.15.19:0/2348788377 conn(0x7f188009b7d0 msgr2=0x7f18800b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.821+0000 7f187f200640 1 -- 172.21.15.19:0/2348788377 shutdown_connections 2024-04-25T11:36:22.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.821+0000 7f187f200640 1 -- 172.21.15.19:0/2348788377 wait complete. 2024-04-25T11:36:22.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.823+0000 7f187f200640 1 Processor -- start 2024-04-25T11:36:22.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.824+0000 7f187f200640 1 -- start start 2024-04-25T11:36:22.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18800b75f0 0x7f1880147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:22.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800cf270 0x7f18800b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:22.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18801605c0 0x7f1880147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:22.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1880076200 con 0x7f18800cf270 2024-04-25T11:36:22.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1880075f20 con 0x7f18801605c0 2024-04-25T11:36:22.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f188006cb80 con 0x7f18800b75f0 2024-04-25T11:36:22.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800cf270 0x7f18800b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800cf270 0x7f18800b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40378/0 (socket says 172.21.15.19:40378) 2024-04-25T11:36:22.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.825+0000 7f187d400640 1 -- 172.21.15.19:0/79358843 learned_addr learned my addr 172.21.15.19:0/79358843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187de00640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18800b75f0 0x7f1880147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187d400640 1 -- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18800b75f0 msgr2=0x7f1880147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187d400640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18800b75f0 0x7f1880147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187d400640 1 -- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18801605c0 msgr2=0x7f1880147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187e800640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18801605c0 0x7f1880147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187d400640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18801605c0 0x7f1880147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187d400640 1 -- 172.21.15.19:0/79358843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188014ae60 con 0x7f18800cf270 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187e800640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18801605c0 0x7f1880147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:22.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.826+0000 7f187d400640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800cf270 0x7f18800b41d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f187000d5f0 tx=0x7f187000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.827+0000 7f1876000640 1 -- 172.21.15.19:0/79358843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700125f0 con 0x7f18800cf270 2024-04-25T11:36:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.827+0000 7f1876000640 1 -- 172.21.15.19:0/79358843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1870019070 con 0x7f18800cf270 2024-04-25T11:36:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.827+0000 7f1876000640 1 -- 172.21.15.19:0/79358843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700129d0 con 0x7f18800cf270 2024-04-25T11:36:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.827+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18800b4710 con 0x7f18800cf270 2024-04-25T11:36:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.827+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18800b36f0 con 0x7f18800cf270 2024-04-25T11:36:22.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.828+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18800b3500 con 0x7f18800cf270 2024-04-25T11:36:22.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.829+0000 7f1876000640 1 -- 172.21.15.19:0/79358843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1870012b70 con 0x7f18800cf270 2024-04-25T11:36:22.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.830+0000 7f1876000640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f185c0744e0 0x7f185c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:22.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.831+0000 7f1876000640 1 -- 172.21.15.19:0/79358843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f1870095810 con 0x7f18800cf270 2024-04-25T11:36:22.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.831+0000 7f187de00640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f185c0744e0 0x7f185c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:22.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.833+0000 7f187de00640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f185c0744e0 0x7f185c0769f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1868034000 tx=0x7f186802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:22.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:22.835+0000 7f1876000640 1 -- 172.21.15.19:0/79358843 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18800b4710 con 0x7f18800cf270 2024-04-25T11:36:23.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.080+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1880050740 con 0x7f18800cf270 2024-04-25T11:36:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.082+0000 7f1876000640 1 -- 172.21.15.19:0/79358843 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+4351 (secure 0 0 0) 0x7f187005fcf0 con 0x7f18800cf270 2024-04-25T11:36:23.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:23.071 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":125,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:22.536166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 125 2024-04-25T11:36:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.084+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f185c0744e0 msgr2=0x7f185c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.084+0000 7f187f200640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f185c0744e0 0x7f185c0769f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1868034000 tx=0x7f186802e000 comp rx=0 tx=0).stop 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.084+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800cf270 msgr2=0x7f18800b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.084+0000 7f187f200640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800cf270 0x7f18800b41d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f187000d5f0 tx=0x7f187000dab0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 shutdown_connections 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f185c0744e0 0x7f185c0769f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18801605c0 0x7f1880147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18800cf270 0x7f18800b41d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 --2- 172.21.15.19:0/79358843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18800b75f0 0x7f1880147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 >> 172.21.15.19:0/79358843 conn(0x7f188009b7d0 msgr2=0x7f1880070200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 shutdown_connections 2024-04-25T11:36:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.085+0000 7f187f200640 1 -- 172.21.15.19:0/79358843 wait complete. 2024-04-25T11:36:23.098 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:36:23.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:23.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.382+0000 7fb4e2800640 1 Processor -- start 2024-04-25T11:36:23.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.382+0000 7fb4e2800640 1 -- start start 2024-04-25T11:36:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4dc0b9050 0x7fb4dc0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e2800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4dc0cfa70 con 0x7fb4dc0b71d0 2024-04-25T11:36:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4dc0d28c0 con 0x7fb4dc0cf660 2024-04-25T11:36:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb4dc0d2ba0 con 0x7fb4dc0b9050 2024-04-25T11:36:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4dc0b9050 0x7fb4dc0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4dc0b9050 0x7fb4dc0b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40390/0 (socket says 172.21.15.19:40390) 2024-04-25T11:36:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e1e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb4dc0b71d0 0x7fb4dc0b75e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46648/0 (socket says 172.21.15.19:46648) 2024-04-25T11:36:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.384+0000 7fb4e1400640 1 -- 172.21.15.19:0/2953662219 learned_addr learned my addr 172.21.15.19:0/2953662219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.385+0000 7fb4e1400640 1 -- 172.21.15.19:0/2953662219 >> v1:172.21.15.169:6789/0 conn(0x7fb4dc0cf660 legacy=0x7fb4dc0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.385+0000 7fb4e1400640 1 -- 172.21.15.19:0/2953662219 >> v1:172.21.15.19:6789/0 conn(0x7fb4dc0b71d0 legacy=0x7fb4dc0b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.385+0000 7fb4e1400640 1 -- 172.21.15.19:0/2953662219 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc09cc50 con 0x7fb4dc0b9050 2024-04-25T11:36:23.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.385+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2953662219 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732892534 0 0) 0x7fb4dc0cfa70 con 0x7fb4dc0b71d0 2024-04-25T11:36:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.386+0000 7fb4e1400640 1 --2- 172.21.15.19:0/2953662219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0b9050 0x7fb4dc0b8990 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009860 tx=0x7fb4cc02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d0ecb78b871018 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.386+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2953662219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc037070 con 0x7fb4dc0b9050 2024-04-25T11:36:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.386+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2953662219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc032440 con 0x7fb4dc0b9050 2024-04-25T11:36:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.386+0000 7fb4dbe00640 1 -- 172.21.15.19:0/2953662219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc032810 con 0x7fb4dc0b9050 2024-04-25T11:36:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.387+0000 7fb4e2800640 1 -- 172.21.15.19:0/2953662219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0b9050 msgr2=0x7fb4dc0b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.387+0000 7fb4e2800640 1 --2- 172.21.15.19:0/2953662219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0b9050 0x7fb4dc0b8990 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009860 tx=0x7fb4cc02a290 comp rx=0 tx=0).stop 2024-04-25T11:36:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.387+0000 7fb4e2800640 1 -- 172.21.15.19:0/2953662219 shutdown_connections 2024-04-25T11:36:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.387+0000 7fb4e2800640 1 --2- 172.21.15.19:0/2953662219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0b9050 0x7fb4dc0b8990 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.387+0000 7fb4e2800640 1 -- 172.21.15.19:0/2953662219 >> 172.21.15.19:0/2953662219 conn(0x7fb4dc0d03f0 msgr2=0x7fb4dc0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.387+0000 7fb4e2800640 1 -- 172.21.15.19:0/2953662219 shutdown_connections 2024-04-25T11:36:23.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.387+0000 7fb4e2800640 1 -- 172.21.15.19:0/2953662219 wait complete. 2024-04-25T11:36:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.390+0000 7fb4e2800640 1 Processor -- start 2024-04-25T11:36:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.390+0000 7fb4e2800640 1 -- start start 2024-04-25T11:36:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b71d0 0x7fb4dc1db490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc0b9050 0x7fb4dc0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0cf660 0x7fb4dc0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0d2550 con 0x7fb4dc0cf660 2024-04-25T11:36:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0cf0b0 con 0x7fb4dc0b71d0 2024-04-25T11:36:23.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4dc0b7cf0 con 0x7fb4dc0b9050 2024-04-25T11:36:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0cf660 0x7fb4dc0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0cf660 0x7fb4dc0c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40396/0 (socket says 172.21.15.19:40396) 2024-04-25T11:36:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e1e00640 1 -- 172.21.15.19:0/711103246 learned_addr learned my addr 172.21.15.19:0/711103246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e1400640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b71d0 0x7fb4dc1db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e0a00640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc0b9050 0x7fb4dc0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.392+0000 7fb4e1e00640 1 -- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc0b9050 msgr2=0x7fb4dc0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e1e00640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc0b9050 0x7fb4dc0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e1e00640 1 -- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b71d0 msgr2=0x7fb4dc1db490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e1e00640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b71d0 0x7fb4dc1db490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e1e00640 1 -- 172.21.15.19:0/711103246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc09f9a0 con 0x7fb4dc0cf660 2024-04-25T11:36:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e0a00640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc0b9050 0x7fb4dc0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e1400640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b71d0 0x7fb4dc1db490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e1e00640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0cf660 0x7fb4dc0c81f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c700 tx=0x7fb4d400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4d9600640 1 -- 172.21.15.19:0/711103246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4018070 con 0x7fb4dc0cf660 2024-04-25T11:36:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4d9600640 1 -- 172.21.15.19:0/711103246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d400ce10 con 0x7fb4dc0cf660 2024-04-25T11:36:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4d9600640 1 -- 172.21.15.19:0/711103246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400a880 con 0x7fb4dc0cf660 2024-04-25T11:36:23.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc09fc80 con 0x7fb4dc0cf660 2024-04-25T11:36:23.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.393+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc0c7790 con 0x7fb4dc0cf660 2024-04-25T11:36:23.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.395+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4dc0a01a0 con 0x7fb4dc0cf660 2024-04-25T11:36:23.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.396+0000 7fb4d9600640 1 -- 172.21.15.19:0/711103246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb4d400aa20 con 0x7fb4dc0cf660 2024-04-25T11:36:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.397+0000 7fb4d9600640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4bc0744b0 0x7fb4bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.397+0000 7fb4d9600640 1 -- 172.21.15.19:0/711103246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fb4d4094b00 con 0x7fb4dc0cf660 2024-04-25T11:36:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.398+0000 7fb4e1400640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4bc0744b0 0x7fb4bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.399+0000 7fb4e1400640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4bc0744b0 0x7fb4bc0769c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.402+0000 7fb4d9600640 1 -- 172.21.15.19:0/711103246 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4dc0a01a0 con 0x7fb4dc0cf660 2024-04-25T11:36:23.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.652+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4dc050740 con 0x7fb4dc0cf660 2024-04-25T11:36:23.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.653+0000 7fb4d9600640 1 -- 172.21.15.19:0/711103246 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) ==== 95+0+5174 (secure 0 0 0) 0x7fb4d4094e20 con 0x7fb4dc0cf660 2024-04-25T11:36:23.642 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:23.642 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/432323448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":432323448},{"type":"v1","addr":"172.21.15.19:6835","nonce":432323448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:22.536166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:23.643 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 126 2024-04-25T11:36:23.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.656+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4bc0744b0 msgr2=0x7fb4bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.656+0000 7fb4e2800640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4bc0744b0 0x7fb4bc0769c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc034000 tx=0x7fb4cc02e000 comp rx=0 tx=0).stop 2024-04-25T11:36:23.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.656+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0cf660 msgr2=0x7fb4dc0c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.656+0000 7fb4e2800640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0cf660 0x7fb4dc0c81f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c700 tx=0x7fb4d400cbc0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 shutdown_connections 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4bc0744b0 0x7fb4bc0769c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4dc0cf660 0x7fb4dc0c81f0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4dc0b9050 0x7fb4dc0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 --2- 172.21.15.19:0/711103246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4dc0b71d0 0x7fb4dc1db490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 >> 172.21.15.19:0/711103246 conn(0x7fb4dc0d03f0 msgr2=0x7fb4dc06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 shutdown_connections 2024-04-25T11:36:23.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.657+0000 7fb4e2800640 1 -- 172.21.15.19:0/711103246 wait complete. 2024-04-25T11:36:23.669 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:36:23.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:23.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.949+0000 7f8f63200640 1 Processor -- start 2024-04-25T11:36:23.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.949+0000 7f8f63200640 1 -- start start 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f63200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f640cf2f0 0x7f8f640b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f63200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f641608d0 0x7f8f64160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f63200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f64161220 con 0x7f8f6415a810 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f63200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6406f870 con 0x7f8f640cf2f0 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f63200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f640b74d0 con 0x7f8f641608d0 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f61e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f640cf2f0 0x7f8f640b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f59400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8f6415a810 0x7f8f640b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40636/0 (socket says 172.21.15.19:40636) 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f59400640 1 -- 172.21.15.19:0/3360909391 learned_addr learned my addr 172.21.15.19:0/3360909391 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.951+0000 7f8f61e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f640cf2f0 0x7f8f640b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33212/0 (socket says 172.21.15.19:33212) 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.952+0000 7f8f61e00640 1 -- 172.21.15.19:0/3360909391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f641608d0 msgr2=0x7f8f64160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.952+0000 7f8f62800640 1 --2- 172.21.15.19:0/3360909391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f641608d0 0x7f8f64160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.952+0000 7f8f61400640 1 -- 172.21.15.19:0/3360909391 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721874657 0 0) 0x7f8f4c001110 con 0x7f8f6415a810 2024-04-25T11:36:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.952+0000 7f8f61e00640 1 --2- 172.21.15.19:0/3360909391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f641608d0 0x7f8f64160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.952+0000 7f8f61e00640 1 -- 172.21.15.19:0/3360909391 >> v1:172.21.15.19:6789/0 conn(0x7f8f6415a810 legacy=0x7f8f640b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.952+0000 7f8f61e00640 1 -- 172.21.15.19:0/3360909391 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f640b77b0 con 0x7f8f640cf2f0 2024-04-25T11:36:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.953+0000 7f8f61e00640 1 --2- 172.21.15.19:0/3360909391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f640b6e10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8f540097a0 tx=0x7f8f5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901403ab63ac3168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.953+0000 7f8f61400640 1 -- 172.21.15.19:0/3360909391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54037070 con 0x7f8f640cf2f0 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.953+0000 7f8f61400640 1 -- 172.21.15.19:0/3360909391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54032440 con 0x7f8f640cf2f0 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.953+0000 7f8f61400640 1 -- 172.21.15.19:0/3360909391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54032810 con 0x7f8f640cf2f0 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 -- 172.21.15.19:0/3360909391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 msgr2=0x7f8f640b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 --2- 172.21.15.19:0/3360909391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f640b6e10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8f540097a0 tx=0x7f8f5402a180 comp rx=0 tx=0).stop 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 -- 172.21.15.19:0/3360909391 shutdown_connections 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 --2- 172.21.15.19:0/3360909391 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f641608d0 0x7f8f64160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 --2- 172.21.15.19:0/3360909391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f640b6e10 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 -- 172.21.15.19:0/3360909391 >> 172.21.15.19:0/3360909391 conn(0x7f8f640d1720 msgr2=0x7f8f640d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:23.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 -- 172.21.15.19:0/3360909391 shutdown_connections 2024-04-25T11:36:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.954+0000 7f8f63200640 1 -- 172.21.15.19:0/3360909391 wait complete. 2024-04-25T11:36:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.957+0000 7f8f63200640 1 Processor -- start 2024-04-25T11:36:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.957+0000 7f8f63200640 1 -- start start 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.958+0000 7f8f63200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f6414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.958+0000 7f8f63200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f6415a810 0x7f8f6414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.958+0000 7f8f63200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f641608d0 0x7f8f640b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.958+0000 7f8f63200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6415ac20 con 0x7f8f640cf2f0 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.958+0000 7f8f63200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6406f500 con 0x7f8f641608d0 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.958+0000 7f8f63200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6406fd00 con 0x7f8f6415a810 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f6414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f6414dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33220/0 (socket says 172.21.15.19:33220) 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 -- 172.21.15.19:0/2379441302 learned_addr learned my addr 172.21.15.19:0/2379441302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 -- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f6415a810 msgr2=0x7f8f6414ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f6415a810 0x7f8f6414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 -- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f641608d0 msgr2=0x7f8f640b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f641608d0 0x7f8f640b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 -- 172.21.15.19:0/2379441302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f640afd40 con 0x7f8f640cf2f0 2024-04-25T11:36:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.959+0000 7f8f61e00640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f6414dd50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.960+0000 7f8f5aa00640 1 -- 172.21.15.19:0/2379441302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54037040 con 0x7f8f640cf2f0 2024-04-25T11:36:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.960+0000 7f8f5aa00640 1 -- 172.21.15.19:0/2379441302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54042070 con 0x7f8f640cf2f0 2024-04-25T11:36:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.960+0000 7f8f5aa00640 1 -- 172.21.15.19:0/2379441302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54030a70 con 0x7f8f640cf2f0 2024-04-25T11:36:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.960+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f640b0020 con 0x7f8f640cf2f0 2024-04-25T11:36:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.960+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6414b240 con 0x7f8f640cf2f0 2024-04-25T11:36:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.961+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f64160ce0 con 0x7f8f640cf2f0 2024-04-25T11:36:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.962+0000 7f8f5aa00640 1 -- 172.21.15.19:0/2379441302 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f54030c10 con 0x7f8f640cf2f0 2024-04-25T11:36:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.963+0000 7f8f5aa00640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.963+0000 7f8f5aa00640 1 -- 172.21.15.19:0/2379441302 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f8f540b5f20 con 0x7f8f640cf2f0 2024-04-25T11:36:23.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.964+0000 7f8f59400640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:23.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.965+0000 7f8f59400640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c005880 tx=0x7f8f4c007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:23.968+0000 7f8f5aa00640 1 -- 172.21.15.19:0/2379441302 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f64160ce0 con 0x7f8f640cf2f0 2024-04-25T11:36:24.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.211+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f64050740 con 0x7f8f640cf2f0 2024-04-25T11:36:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.212+0000 7f8f5aa00640 1 -- 172.21.15.19:0/2379441302 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) ==== 95+0+5174 (secure 0 0 0) 0x7f8f54080490 con 0x7f8f640cf2f0 2024-04-25T11:36:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:24.201 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":126,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/432323448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":432323448},{"type":"v1","addr":"172.21.15.19:6835","nonce":432323448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:22.536166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 126 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 msgr2=0x7f8f400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c005880 tx=0x7f8f4c007a00 comp rx=0 tx=0).stop 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 msgr2=0x7f8f6414dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f6414dd50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f540046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 shutdown_connections 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f400744e0 0x7f8f400769f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f641608d0 0x7f8f640b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f6415a810 0x7f8f6414ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 --2- 172.21.15.19:0/2379441302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f640cf2f0 0x7f8f6414dd50 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 >> 172.21.15.19:0/2379441302 conn(0x7f8f640d1720 msgr2=0x7f8f6415d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:24.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 shutdown_connections 2024-04-25T11:36:24.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.215+0000 7f8f63200640 1 -- 172.21.15.19:0/2379441302 wait complete. 2024-04-25T11:36:24.232 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2024-04-25T11:36:24.232 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2024-04-25T11:36:24.232 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:24.232 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:24.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:36:24.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:36:24.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:24.286 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:36:24.340 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:36:24.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:36:24.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:36:24.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.646+0000 7f9f97e00640 1 Processor -- start 2024-04-25T11:36:24.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.646+0000 7f9f97e00640 1 -- start start 2024-04-25T11:36:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.648+0000 7f9f97e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9f980d1dd0 0x7f9f980d3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.648+0000 7f9f97e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9f98163e10 0x7f9f980cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.648+0000 7f9f97e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f980d22b0 con 0x7f9f980ce530 2024-04-25T11:36:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.648+0000 7f9f97e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9f980d2ec0 con 0x7f9f98163e10 2024-04-25T11:36:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.648+0000 7f9f97e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9f980d2590 con 0x7f9f980d1dd0 2024-04-25T11:36:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f8fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9f98163e10 0x7f9f980cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f97400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9f980ce530 0x7f9f9806dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40640/0 (socket says 172.21.15.19:40640) 2024-04-25T11:36:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f97400640 1 -- 172.21.15.19:0/1013753545 learned_addr learned my addr 172.21.15.19:0/1013753545 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f96a00640 1 --2- 172.21.15.19:0/1013753545 >> v2:172.21.15.169:3300/0 conn(0x7f9f980d1dd0 0x7f9f980d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f8fe00640 1 -- 172.21.15.19:0/1013753545 >> v2:172.21.15.169:3300/0 conn(0x7f9f980d1dd0 msgr2=0x7f9f980d3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f8fe00640 1 --2- 172.21.15.19:0/1013753545 >> v2:172.21.15.169:3300/0 conn(0x7f9f980d1dd0 0x7f9f980d3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f8fe00640 1 -- 172.21.15.19:0/1013753545 >> v1:172.21.15.19:6789/0 conn(0x7f9f980ce530 legacy=0x7f9f9806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f8fe00640 1 -- 172.21.15.19:0/1013753545 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f980d2870 con 0x7f9f98163e10 2024-04-25T11:36:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.649+0000 7f9f96a00640 1 --2- 172.21.15.19:0/1013753545 >> v2:172.21.15.169:3300/0 conn(0x7f9f980d1dd0 0x7f9f980d3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.650+0000 7f9f8fe00640 1 --2- 172.21.15.19:0/1013753545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98163e10 0x7f9f980cdff0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9f98074580 tx=0x7f9f8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904018b4ff2e22d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.651+0000 7f9f96000640 1 -- 172.21.15.19:0/1013753545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80037070 con 0x7f9f98163e10 2024-04-25T11:36:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.651+0000 7f9f96000640 1 -- 172.21.15.19:0/1013753545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f80032440 con 0x7f9f98163e10 2024-04-25T11:36:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.651+0000 7f9f96000640 1 -- 172.21.15.19:0/1013753545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80032810 con 0x7f9f98163e10 2024-04-25T11:36:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.651+0000 7f9f97e00640 1 -- 172.21.15.19:0/1013753545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98163e10 msgr2=0x7f9f980cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.651+0000 7f9f97e00640 1 --2- 172.21.15.19:0/1013753545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98163e10 0x7f9f980cdff0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9f98074580 tx=0x7f9f8002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.652+0000 7f9f97e00640 1 -- 172.21.15.19:0/1013753545 shutdown_connections 2024-04-25T11:36:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.652+0000 7f9f97e00640 1 --2- 172.21.15.19:0/1013753545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98163e10 0x7f9f980cdff0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.652+0000 7f9f97e00640 1 --2- 172.21.15.19:0/1013753545 >> v2:172.21.15.169:3300/0 conn(0x7f9f980d1dd0 0x7f9f980d3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.652+0000 7f9f97e00640 1 -- 172.21.15.19:0/1013753545 >> 172.21.15.19:0/1013753545 conn(0x7f9f980b5490 msgr2=0x7f9f980b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.652+0000 7f9f97e00640 1 -- 172.21.15.19:0/1013753545 shutdown_connections 2024-04-25T11:36:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.652+0000 7f9f97e00640 1 -- 172.21.15.19:0/1013753545 wait complete. 2024-04-25T11:36:24.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.654+0000 7f9f97e00640 1 Processor -- start 2024-04-25T11:36:24.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.654+0000 7f9f97e00640 1 -- start start 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f97e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f980ce530 0x7f9f981f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f97e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f980d1dd0 0x7f9f980cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f97e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f98163e10 0x7f9f980ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f97e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f980ce940 con 0x7f9f980d1dd0 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f97e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f980b9330 con 0x7f9f980ce530 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f97e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9f980ceac0 con 0x7f9f98163e10 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f980d1dd0 0x7f9f980cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f980d1dd0 0x7f9f980cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33240/0 (socket says 172.21.15.19:33240) 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 -- 172.21.15.19:0/3425096009 learned_addr learned my addr 172.21.15.19:0/3425096009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 -- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f98163e10 msgr2=0x7f9f980ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f98163e10 0x7f9f980ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 -- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f980ce530 msgr2=0x7f9f981f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f980ce530 0x7f9f981f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.656+0000 7f9f8fe00640 1 -- 172.21.15.19:0/3425096009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f98161bb0 con 0x7f9f980d1dd0 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.657+0000 7f9f8fe00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f980d1dd0 0x7f9f980cb2b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.657+0000 7f9f8ea00640 1 -- 172.21.15.19:0/3425096009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80037040 con 0x7f9f980d1dd0 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.657+0000 7f9f8ea00640 1 -- 172.21.15.19:0/3425096009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f80042070 con 0x7f9f980d1dd0 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.657+0000 7f9f8ea00640 1 -- 172.21.15.19:0/3425096009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80030a50 con 0x7f9f980d1dd0 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.657+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f98161e90 con 0x7f9f980d1dd0 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.657+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f98162450 con 0x7f9f980d1dd0 2024-04-25T11:36:24.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.659+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98050740 con 0x7f9f980d1dd0 2024-04-25T11:36:24.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.660+0000 7f9f8ea00640 1 -- 172.21.15.19:0/3425096009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9f80030bf0 con 0x7f9f980d1dd0 2024-04-25T11:36:24.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.661+0000 7f9f8ea00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f740744b0 0x7f9f740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:24.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.661+0000 7f9f8ea00640 1 -- 172.21.15.19:0/3425096009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f9f800b5f00 con 0x7f9f980d1dd0 2024-04-25T11:36:24.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.662+0000 7f9f96a00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f740744b0 0x7f9f740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:24.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.663+0000 7f9f96a00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f740744b0 0x7f9f740769c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9f980b9a10 tx=0x7f9f88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:24.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.666+0000 7f9f8ea00640 1 -- 172.21.15.19:0/3425096009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f80080500 con 0x7f9f980d1dd0 2024-04-25T11:36:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:24.907+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9f980d16f0 con 0x7f9f980d1dd0 2024-04-25T11:36:25.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.567+0000 7f9f8ea00640 1 -- 172.21.15.19:0/3425096009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v127) ==== 161+0+0 (secure 0 0 0) 0x7f9f8007fe00 con 0x7f9f980d1dd0 2024-04-25T11:36:25.556 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.570+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f740744b0 msgr2=0x7f9f740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.570+0000 7f9f97e00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f740744b0 0x7f9f740769c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9f980b9a10 tx=0x7f9f88008040 comp rx=0 tx=0).stop 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f980d1dd0 msgr2=0x7f9f980cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f980d1dd0 0x7f9f980cb2b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f800046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 shutdown_connections 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9f740744b0 0x7f9f740769c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f98163e10 0x7f9f980ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f980d1dd0 0x7f9f980cb2b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 --2- 172.21.15.19:0/3425096009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f980ce530 0x7f9f981f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 >> 172.21.15.19:0/3425096009 conn(0x7f9f980b5490 msgr2=0x7f9f980b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 shutdown_connections 2024-04-25T11:36:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.571+0000 7f9f97e00640 1 -- 172.21.15.19:0/3425096009 wait complete. 2024-04-25T11:36:25.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:36:25.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.882+0000 7fd563200640 1 Processor -- start 2024-04-25T11:36:25.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.882+0000 7fd563200640 1 -- start start 2024-04-25T11:36:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.884+0000 7fd563200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5640cf9e0 0x7fd564161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.884+0000 7fd563200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5640b0800 0x7fd564070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.884+0000 7fd563200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5641569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.884+0000 7fd563200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd5641615d0 con 0x7fd5640cf9e0 2024-04-25T11:36:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.884+0000 7fd563200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd564070be0 con 0x7fd5640b0800 2024-04-25T11:36:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.884+0000 7fd563200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5640cffc0 con 0x7fd5640707d0 2024-04-25T11:36:25.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.884+0000 7fd561e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5640cf9e0 0x7fd564161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:25.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd5640cf9e0 0x7fd564161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33250/0 (socket says 172.21.15.19:33250) 2024-04-25T11:36:25.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561e00640 1 -- 172.21.15.19:0/636020626 learned_addr learned my addr 172.21.15.19:0/636020626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:25.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd562800640 1 --2- 172.21.15.19:0/636020626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5641569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:25.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561e00640 1 -- 172.21.15.19:0/636020626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 msgr2=0x7fd5641569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:25.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561e00640 1 --2- 172.21.15.19:0/636020626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5641569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561400640 1 --2- 172.21.15.19:0/636020626 >> v2:172.21.15.169:3300/0 conn(0x7fd5640b0800 0x7fd564070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:25.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561e00640 1 -- 172.21.15.19:0/636020626 >> v2:172.21.15.169:3300/0 conn(0x7fd5640b0800 msgr2=0x7fd564070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:25.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561e00640 1 --2- 172.21.15.19:0/636020626 >> v2:172.21.15.169:3300/0 conn(0x7fd5640b0800 0x7fd564070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.885+0000 7fd561e00640 1 -- 172.21.15.19:0/636020626 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5640d02a0 con 0x7fd5640cf9e0 2024-04-25T11:36:25.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.886+0000 7fd561e00640 1 --2- 172.21.15.19:0/636020626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640cf9e0 0x7fd564161090 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd5640d14c0 tx=0x7fd55402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292694e8235a7092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:25.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.887+0000 7fd560a00640 1 -- 172.21.15.19:0/636020626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554036070 con 0x7fd5640cf9e0 2024-04-25T11:36:25.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.887+0000 7fd560a00640 1 -- 172.21.15.19:0/636020626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd554031600 con 0x7fd5640cf9e0 2024-04-25T11:36:25.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.887+0000 7fd560a00640 1 -- 172.21.15.19:0/636020626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5540319d0 con 0x7fd5640cf9e0 2024-04-25T11:36:25.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 -- 172.21.15.19:0/636020626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640cf9e0 msgr2=0x7fd564161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 --2- 172.21.15.19:0/636020626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640cf9e0 0x7fd564161090 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd5640d14c0 tx=0x7fd55402a610 comp rx=0 tx=0).stop 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 -- 172.21.15.19:0/636020626 shutdown_connections 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 --2- 172.21.15.19:0/636020626 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5641569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 --2- 172.21.15.19:0/636020626 >> v2:172.21.15.169:3300/0 conn(0x7fd5640b0800 0x7fd564070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 --2- 172.21.15.19:0/636020626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640cf9e0 0x7fd564161090 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 -- 172.21.15.19:0/636020626 >> 172.21.15.19:0/636020626 conn(0x7fd5640d1720 msgr2=0x7fd5640d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 -- 172.21.15.19:0/636020626 shutdown_connections 2024-04-25T11:36:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.888+0000 7fd563200640 1 -- 172.21.15.19:0/636020626 wait complete. 2024-04-25T11:36:25.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.891+0000 7fd563200640 1 Processor -- start 2024-04-25T11:36:25.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.891+0000 7fd563200640 1 -- start start 2024-04-25T11:36:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd563200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5640b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd563200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640b0800 0x7fd564158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd563200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5640cf9e0 0x7fd564158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd563200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5640b0c10 con 0x7fd5640b0800 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd563200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5640d0860 con 0x7fd5640cf9e0 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd563200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5640b0d90 con 0x7fd5640707d0 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd561400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640b0800 0x7fd564158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd561400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640b0800 0x7fd564158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33260/0 (socket says 172.21.15.19:33260) 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd561400640 1 -- 172.21.15.19:0/2831266603 learned_addr learned my addr 172.21.15.19:0/2831266603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.893+0000 7fd561e00640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5640b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd562800640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5640cf9e0 0x7fd564158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd561400640 1 -- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 msgr2=0x7fd5640b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd561400640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5640b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd561400640 1 -- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5640cf9e0 msgr2=0x7fd564158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd561400640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5640cf9e0 0x7fd564158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:25.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd561400640 1 -- 172.21.15.19:0/2831266603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5641f9ad0 con 0x7fd5640b0800 2024-04-25T11:36:25.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd561400640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640b0800 0x7fd564158000 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c820 tx=0x7fd55000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:25.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.894+0000 7fd55a000640 1 -- 172.21.15.19:0/2831266603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550015070 con 0x7fd5640b0800 2024-04-25T11:36:25.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.895+0000 7fd55a000640 1 -- 172.21.15.19:0/2831266603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd550004020 con 0x7fd5640b0800 2024-04-25T11:36:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.895+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5641f9d70 con 0x7fd5640b0800 2024-04-25T11:36:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.895+0000 7fd55a000640 1 -- 172.21.15.19:0/2831266603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500043f0 con 0x7fd5640b0800 2024-04-25T11:36:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.895+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641fa330 con 0x7fd5640b0800 2024-04-25T11:36:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.896+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd564158ac0 con 0x7fd5640b0800 2024-04-25T11:36:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.897+0000 7fd55a000640 1 -- 172.21.15.19:0/2831266603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd550007830 con 0x7fd5640b0800 2024-04-25T11:36:25.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.898+0000 7fd55a000640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5440744b0 0x7fd5440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:25.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.898+0000 7fd55a000640 1 -- 172.21.15.19:0/2831266603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fd550095260 con 0x7fd5640b0800 2024-04-25T11:36:25.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.899+0000 7fd561e00640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5440744b0 0x7fd5440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:25.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.900+0000 7fd561e00640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5440744b0 0x7fd5440769c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd554033000 tx=0x7fd554009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:25.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:25.903+0000 7fd55a000640 1 -- 172.21.15.19:0/2831266603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd55005f750 con 0x7fd5640b0800 2024-04-25T11:36:26.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.189+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd564050740 con 0x7fd5640b0800 2024-04-25T11:36:26.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.582+0000 7fd55a000640 1 -- 172.21.15.19:0/2831266603 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v128) ==== 101+0+0 (secure 0 0 0) 0x7fd55005f050 con 0x7fd5640b0800 2024-04-25T11:36:26.571 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:36:26.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5440744b0 msgr2=0x7fd5440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5440744b0 0x7fd5440769c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd554033000 tx=0x7fd554009000 comp rx=0 tx=0).stop 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640b0800 msgr2=0x7fd564158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640b0800 0x7fd564158000 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c820 tx=0x7fd55000cce0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 shutdown_connections 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5440744b0 0x7fd5440769c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5640cf9e0 0x7fd564158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5640b0800 0x7fd564158000 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 --2- 172.21.15.19:0/2831266603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5640707d0 0x7fd5640b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.585+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 >> 172.21.15.19:0/2831266603 conn(0x7fd5640d1720 msgr2=0x7fd5640d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.586+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 shutdown_connections 2024-04-25T11:36:26.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.586+0000 7fd563200640 1 -- 172.21.15.19:0/2831266603 wait complete. 2024-04-25T11:36:26.611 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:36:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.814+0000 7fa1d0d3ef40 1 Processor -- start 2024-04-25T11:36:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.814+0000 7fa1d0d3ef40 1 -- start start 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1d0d3ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b0003c70 0x55bfe558c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1d0d3ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 0x55bfe5512230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1d0d3ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe5593050 0x55bfe5593460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1d0d3ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bfe546cd20 con 0x55bfe5593050 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1d0d3ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bfe5470d60 con 0x7fa1b0003c70 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1d0d3ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bfe550f3f0 con 0x55bfe5512a00 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe5593050 0x55bfe5593460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe5593050 0x55bfe5593460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33276/0 (socket says 172.21.15.19:33276) 2024-04-25T11:36:26.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.816+0000 7fa1ca800640 1 -- 172.21.15.19:0/489127660 learned_addr learned my addr 172.21.15.19:0/489127660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:26.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b0003c70 0x55bfe558c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1c9400640 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 0x55bfe5512230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1ca800640 1 -- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 msgr2=0x55bfe5512230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1ca800640 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 0x55bfe5512230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1ca800640 1 -- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b0003c70 msgr2=0x55bfe558c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1ca800640 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b0003c70 0x55bfe558c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1ca800640 1 -- 172.21.15.19:0/489127660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe54f7830 con 0x55bfe5593050 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1ca800640 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe5593050 0x55bfe5593460 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0464f0 tx=0x7fa1bc04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8d70e52baae277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.817+0000 7fa1c3e00640 1 -- 172.21.15.19:0/489127660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc000ed0 con 0x55bfe5593050 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1c3e00640 1 -- 172.21.15.19:0/489127660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1bc001430 con 0x55bfe5593050 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1c3e00640 1 -- 172.21.15.19:0/489127660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc04db40 con 0x55bfe5593050 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/489127660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe5593050 msgr2=0x55bfe5593460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe5593050 0x55bfe5593460 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0464f0 tx=0x7fa1bc04f000 comp rx=0 tx=0).stop 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/489127660 shutdown_connections 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bfe5593050 0x55bfe5593460 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 0x55bfe5512230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/489127660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa1b0003c70 0x55bfe558c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/489127660 >> 172.21.15.19:0/489127660 conn(0x55bfe53e8ac0 msgr2=0x55bfe54f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/489127660 shutdown_connections 2024-04-25T11:36:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.818+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/489127660 wait complete. 2024-04-25T11:36:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.820+0000 7fa1d0d3ef40 1 Processor -- start 2024-04-25T11:36:26.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.820+0000 7fa1d0d3ef40 1 -- start start 2024-04-25T11:36:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.821+0000 7fa1d0d3ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 0x55bfe5588df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.821+0000 7fa1d0d3ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe5593050 0x55bfe558add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.821+0000 7fa1d0d3ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0003c70 0x55bfe558b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.821+0000 7fa1d0d3ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bfe5512770 con 0x7fa1b0003c70 2024-04-25T11:36:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.821+0000 7fa1d0d3ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bfe5597070 con 0x55bfe5593050 2024-04-25T11:36:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.821+0000 7fa1d0d3ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55bfe558e400 con 0x55bfe5512a00 2024-04-25T11:36:26.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0003c70 0x55bfe558b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0003c70 0x55bfe558b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33288/0 (socket says 172.21.15.19:33288) 2024-04-25T11:36:26.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 learned_addr learned my addr 172.21.15.19:0/2900769503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 0x55bfe5588df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 msgr2=0x55bfe5588df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55bfe5512a00 0x55bfe5588df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1c9400640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe5593050 0x55bfe558add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe5593050 msgr2=0x55bfe558add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bfe5593050 0x55bfe558add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe5592cd0 con 0x7fa1b0003c70 2024-04-25T11:36:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.822+0000 7fa1ca800640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0003c70 0x55bfe558b310 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc000c00 tx=0x7fa1bc0018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.823+0000 7fa1c2a00640 1 -- 172.21.15.19:0/2900769503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0015e0 con 0x7fa1b0003c70 2024-04-25T11:36:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.823+0000 7fa1c2a00640 1 -- 172.21.15.19:0/2900769503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1bc05e470 con 0x7fa1b0003c70 2024-04-25T11:36:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.823+0000 7fa1c2a00640 1 -- 172.21.15.19:0/2900769503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc05e660 con 0x7fa1b0003c70 2024-04-25T11:36:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.823+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bfe55896e0 con 0x7fa1b0003c70 2024-04-25T11:36:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.823+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe558bbe0 con 0x7fa1b0003c70 2024-04-25T11:36:26.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.825+0000 7fa1c2a00640 1 -- 172.21.15.19:0/2900769503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa1bc05c040 con 0x7fa1b0003c70 2024-04-25T11:36:26.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.826+0000 7fa1c2a00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1a8072e90 0x7fa1a80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.826+0000 7fa1c2a00640 1 -- 172.21.15.19:0/2900769503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fa1bc0d9050 con 0x7fa1b0003c70 2024-04-25T11:36:26.815 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:36:26.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.827+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55bfe559cc30 0x55bfe558d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.827+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe558dc90 con 0x55bfe559cc30 2024-04-25T11:36:26.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.827+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1a8072e90 0x7fa1a80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.827+0000 7fa1c9400640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55bfe559cc30 0x55bfe558d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.828+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1a8072e90 0x7fa1a80753a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000f80 tx=0x7fa1c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:26.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.829+0000 7fa1c9400640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55bfe559cc30 0x55bfe558d750 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:26.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.832+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v98'164 uv138 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.832+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe563fc80 con 0x55bfe559cc30 2024-04-25T11:36:26.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.835+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v98'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.835+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640020 con 0x55bfe559cc30 2024-04-25T11:36:26.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.838+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v98'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.827 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:36:26.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.839+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe56413d0 con 0x55bfe559cc30 2024-04-25T11:36:26.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.842+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v98'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.842+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5641770 con 0x55bfe559cc30 2024-04-25T11:36:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.845+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v98'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.845+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe56403c0 con 0x55bfe559cc30 2024-04-25T11:36:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.849+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v98'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.849+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe56407b0 con 0x55bfe559cc30 2024-04-25T11:36:26.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.852+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v98'170 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.853+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.856+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v98'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.856+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.860+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v98'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.848 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:36:26.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.860+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.863+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v98'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.863+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.867+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v98'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.867+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.871+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v98'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.871+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.874+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v98'176 uv151 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.874+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.879+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v98'177 uv177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.879+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.882+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v98'178 uv178 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.871 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:36:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.883+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.887+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v98'179 uv179 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.887+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.891+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v98'180 uv180 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.892+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.897+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v98'181 uv181 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.897+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.901+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v98'182 uv182 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.901+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.905+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v98'183 uv157 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.905+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.910+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v98'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.910+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.903 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:36:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.914+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v98'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.914+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.918+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v98'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.918+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.922+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v98'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.922+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.927+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v98'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.927+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.931+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v98'189 uv163 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.931+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.936+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v98'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.936+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.929 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:36:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.940+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v98'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.940+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.945+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v98'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.945+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.949+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v98'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.949+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe559cc30 2024-04-25T11:36:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.954+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v98'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.954+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bfe5640f60 0x55bfe5644030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.954+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5640b50 con 0x55bfe5640f60 2024-04-25T11:36:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.955+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bfe5640f60 0x55bfe5644030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:26.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.956+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bfe5640f60 0x55bfe5644030 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:26.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.961+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v98'129 uv113 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.961+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5644570 con 0x55bfe5640f60 2024-04-25T11:36:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.966+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v98'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.966+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5644910 con 0x55bfe5640f60 2024-04-25T11:36:26.959 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:36:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.970+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v98'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.970+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5644cb0 con 0x55bfe5640f60 2024-04-25T11:36:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.975+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v98'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.976+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe54f8140 con 0x55bfe5640f60 2024-04-25T11:36:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.980+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v98'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.980+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645500 con 0x55bfe5640f60 2024-04-25T11:36:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.985+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v98'134 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.985+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe56458a0 con 0x55bfe5640f60 2024-04-25T11:36:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.990+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v98'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.990+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645c90 con 0x55bfe5640f60 2024-04-25T11:36:26.983 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:36:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.994+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v98'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.994+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.999+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v98'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:26.999+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:26.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.002+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v98'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.003+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:26.995 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:36:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.007+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v98'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.007+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.012+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [write 0~18117] v98'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.012+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.016+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v98'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.016+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.020+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 606.00000000 [create] v98'142 uv127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.020+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.024+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 606.00000000 [delete] v98'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.025+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.028+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v98'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.017 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:36:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.029+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.033+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v98'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.033+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.037+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v98'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.037+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645130 con 0x55bfe5640f60 2024-04-25T11:36:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.041+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v98'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe5640f60 2024-04-25T11:36:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.041+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55bfe5646600 0x55bfe5646a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.042+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:51 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5649180 con 0x55bfe5646600 2024-04-25T11:36:27.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.043+0000 7fa1c9400640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55bfe5646600 0x55bfe5646a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.044+0000 7fa1c9400640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55bfe5646600 0x55bfe5646a10 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.048+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(51 607.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.048+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:52 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5649180 con 0x55bfe5646600 2024-04-25T11:36:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.052+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(52 607.00000000 [delete] v98'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.052+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5649180 con 0x55bfe5646600 2024-04-25T11:36:27.045 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:36:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.056+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(53 607.00000000 [create] v98'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.056+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645cf0 con 0x55bfe5646600 2024-04-25T11:36:27.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.061+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.061+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645cb0 con 0x55bfe5646600 2024-04-25T11:36:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.065+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.065+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645cb0 con 0x55bfe5646600 2024-04-25T11:36:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.069+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.069+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:57 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645cb0 con 0x55bfe559cc30 2024-04-25T11:36:27.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.073+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(57 mds0_openfiles.0 [create] v98'27 uv26 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.073+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:58 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5645cb0 con 0x55bfe559cc30 2024-04-25T11:36:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.077+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v98'28 uv28 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.077+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645c70 con 0x55bfe559cc30 2024-04-25T11:36:27.069 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:36:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.081+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(59 mds0_openfiles.0 [create] v98'29 uv29 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.081+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5645c90 con 0x55bfe559cc30 2024-04-25T11:36:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.085+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v98'30 uv30 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.085+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55bfe5649c80 0x55bfe564c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.085+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:61 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe564c5e0 con 0x55bfe5649c80 2024-04-25T11:36:27.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.086+0000 7fa1ca800640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55bfe5649c80 0x55bfe564c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.087+0000 7fa1ca800640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55bfe5649c80 0x55bfe564c0a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.091+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(61 608.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.091+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:62 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe564c980 con 0x55bfe5649c80 2024-04-25T11:36:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.096+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(62 608.00000000 [delete] v98'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.096+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe564ce30 con 0x55bfe5649c80 2024-04-25T11:36:27.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.100+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(63 608.00000000 [create] v98'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.089 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:36:27.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.100+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe564e6e0 con 0x55bfe5649c80 2024-04-25T11:36:27.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.104+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.104+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe564ea80 con 0x55bfe5649c80 2024-04-25T11:36:27.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.108+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.108+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe564d6d0 con 0x55bfe5649c80 2024-04-25T11:36:27.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.112+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(66 608.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.112+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55bfe564dff0 0x55bfe5650e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.112+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:67 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe564d6d0 con 0x55bfe564dff0 2024-04-25T11:36:27.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.113+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55bfe564dff0 0x55bfe5650e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.115+0000 7fa1c9e00640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55bfe564dff0 0x55bfe5650e30 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.119+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(67 604.00000000 [create] v98'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.119+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:68 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe564d6d0 con 0x55bfe564dff0 2024-04-25T11:36:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.123+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(68 604.00000000 [delete] v98'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.123+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5651370 con 0x55bfe564dff0 2024-04-25T11:36:27.116 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:36:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.127+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(69 604.00000000 [create] v98'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.127+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5652380 con 0x55bfe564dff0 2024-04-25T11:36:27.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.131+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v98'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.131+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5652380 con 0x55bfe564dff0 2024-04-25T11:36:27.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.135+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v98'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.135+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5651370 con 0x55bfe564dff0 2024-04-25T11:36:27.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.139+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(72 604.00000000 [omap-set-header] v98'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.139+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5651370 con 0x55bfe564dff0 2024-04-25T11:36:27.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.143+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(73 500.00000000 [create] v98'65 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.143+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5651370 con 0x55bfe564dff0 2024-04-25T11:36:27.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.147+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(74 500.00000000 [delete] v98'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.147+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5651370 con 0x55bfe564dff0 2024-04-25T11:36:27.140 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:36:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.151+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(75 500.00000000 [create] v98'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.152+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5651370 con 0x55bfe564dff0 2024-04-25T11:36:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.156+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(76 500.00000000 [write 0~90] v98'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.156+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5651370 con 0x55bfe564dff0 2024-04-25T11:36:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.160+0000 7fa1c9e00640 1 -- 172.21.15.19:0/2900769503 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(77 500.00000000 [omap-set-header] v98'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1c40137f0 con 0x55bfe564dff0 2024-04-25T11:36:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.160+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bfe5651880 0x55bfe5653ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.160+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:78 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe56543b0 con 0x55bfe5651880 2024-04-25T11:36:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.161+0000 7fa1c9400640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bfe5651880 0x55bfe5653ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.163+0000 7fa1c9400640 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bfe5651880 0x55bfe5653ce0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.167+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(78 mds_snaptable [create] v98'27 uv26 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.167+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:79 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe56543b0 con 0x55bfe5651880 2024-04-25T11:36:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.171+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(79 mds_snaptable [delete] v98'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.171+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5651880 2024-04-25T11:36:27.164 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:36:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.176+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(80 mds_snaptable [create] v98'29 uv29 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.176+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5651880 2024-04-25T11:36:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.180+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(81 mds_snaptable [write 0~70] v98'30 uv30 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.180+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5651880 2024-04-25T11:36:27.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.184+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v98'31 uv31 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.184+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe559cc30 2024-04-25T11:36:27.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.188+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(83 605.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.188+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe559cc30 2024-04-25T11:36:27.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.192+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(84 605.00000000 [delete] v98'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.193+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe559cc30 2024-04-25T11:36:27.185 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:36:27.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.196+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(85 605.00000000 [create] v98'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.196+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5655810 con 0x55bfe559cc30 2024-04-25T11:36:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.200+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.200+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5655810 con 0x55bfe559cc30 2024-04-25T11:36:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.204+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.204+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe559cc30 2024-04-25T11:36:27.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.209+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe559cc30 2024-04-25T11:36:27.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.209+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.213+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(89 mds0_inotable [create] v98'54 uv48 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.213+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.218+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(90 mds0_inotable [delete] v98'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.218+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.210 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:36:27.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.221+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(91 mds0_inotable [create] v98'56 uv56 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.222+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.226+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v98'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.226+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.230+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v98'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.231+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.235+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(94 100.00000000 [create] v98'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.235+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.239+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(95 100.00000000 [delete] v98'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.239+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.231 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:36:27.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.242+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(96 100.00000000 [create] v98'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.242+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654800 con 0x55bfe5646600 2024-04-25T11:36:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.247+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v98'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.247+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5657610 con 0x55bfe5646600 2024-04-25T11:36:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.251+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v98'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5646600 2024-04-25T11:36:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.251+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.255+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v98'26 uv25 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.255+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.259+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v98'27 uv27 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.260+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.252 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:36:27.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.264+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v98'28 uv28 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.264+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.268+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v98'29 uv29 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.268+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.273+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v98'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.273+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.277+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v98'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.277+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.270 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:36:27.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.282+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v98'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.282+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5655cc0 con 0x55bfe5649c80 2024-04-25T11:36:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.286+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v98'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.287+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5655cc0 con 0x55bfe5649c80 2024-04-25T11:36:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.291+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v98'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.291+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5649c80 2024-04-25T11:36:27.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.296+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v98'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1bc076070 con 0x55bfe5649c80 2024-04-25T11:36:27.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.296+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.300+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v98'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.300+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.305+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v98'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.305+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.297 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:36:27.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.309+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v98'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.309+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.313+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v98'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.313+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.317+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v98'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.317+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.321+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v98'69 uv61 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.322+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.326+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v98'70 uv70 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.326+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.319 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:36:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.330+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v98'71 uv71 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.330+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.335+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v98'72 uv72 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.335+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.338+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v98'73 uv73 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.339+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.343+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v98'74 uv68 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.343+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.347+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v98'75 uv75 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.347+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.339 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:36:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.351+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v98'76 uv76 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.351+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.355+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v98'77 uv77 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.355+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5655cc0 con 0x55bfe5651880 2024-04-25T11:36:27.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.359+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v98'78 uv78 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.359+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5654cb0 con 0x55bfe5651880 2024-04-25T11:36:27.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.363+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v98'79 uv79 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.363+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x55bfe5655210 con 0x55bfe5651880 2024-04-25T11:36:27.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1c9400640 1 -- 172.21.15.19:0/2900769503 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v98'80 uv80 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa1b4009770 con 0x55bfe5651880 2024-04-25T11:36:27.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bfe5640f60 msgr2=0x55bfe5644030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55bfe5640f60 0x55bfe5644030 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55bfe564dff0 msgr2=0x55bfe5650e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55bfe564dff0 0x55bfe5650e30 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bfe5651880 msgr2=0x55bfe5653ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55bfe5651880 0x55bfe5653ce0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55bfe5646600 msgr2=0x55bfe5646a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55bfe5646600 0x55bfe5646a10 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55bfe559cc30 msgr2=0x55bfe558d750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55bfe559cc30 0x55bfe558d750 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 reap_dead start 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55bfe5649c80 msgr2=0x55bfe564c0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55bfe5649c80 0x55bfe564c0a0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.368+0000 7fa1ca800640 1 -- 172.21.15.19:0/2900769503 reap_dead start 2024-04-25T11:36:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1a8072e90 msgr2=0x7fa1a80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1a8072e90 0x7fa1a80753a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000f80 tx=0x7fa1c4008040 comp rx=0 tx=0).stop 2024-04-25T11:36:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0003c70 msgr2=0x55bfe558b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0003c70 0x55bfe558b310 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc000c00 tx=0x7fa1bc0018b0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 shutdown_connections 2024-04-25T11:36:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa1a8072e90 0x7fa1a80753a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 --2- 172.21.15.19:0/2900769503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0003c70 0x55bfe558b310 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 >> 172.21.15.19:0/2900769503 conn(0x55bfe53e8ac0 msgr2=0x55bfe5593460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 shutdown_connections 2024-04-25T11:36:27.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.369+0000 7fa1d0d3ef40 1 -- 172.21.15.19:0/2900769503 wait complete. 2024-04-25T11:36:27.377 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2024-04-25T11:36:27.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.623+0000 7f1ae8b2cf40 1 Processor -- start 2024-04-25T11:36:27.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.624+0000 7f1ae8b2cf40 1 -- start start 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae8b2cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x56178259adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae8b2cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 0x56178259b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae8b2cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 0x56178261d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae8b2cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561782596e70 con 0x56178254e1e0 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae8b2cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5617824f4d20 con 0x561782627dd0 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae8b2cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561782597110 con 0x56178254e5f0 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x56178259adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x56178259adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33314/0 (socket says 172.21.15.19:33314) 2024-04-25T11:36:27.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.625+0000 7f1ae1200640 1 -- 172.21.15.19:0/998172080 learned_addr learned my addr 172.21.15.19:0/998172080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1ae1200640 1 -- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 msgr2=0x56178259b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1adbe00640 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 0x56178259b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1ae1c00640 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 0x56178261d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1ae1200640 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 0x56178259b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1ae1200640 1 -- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 msgr2=0x56178261d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1ae1200640 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 0x56178261d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1ae1200640 1 -- 172.21.15.19:0/998172080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56178261ba40 con 0x56178254e1e0 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1ae1200640 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x56178259adc0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4006ec0 tx=0x7f1ad400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0144ca2a917ffdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.626+0000 7f1adaa00640 1 -- 172.21.15.19:0/998172080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4012070 con 0x56178254e1e0 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1adaa00640 1 -- 172.21.15.19:0/998172080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad400e690 con 0x56178254e1e0 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1adaa00640 1 -- 172.21.15.19:0/998172080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400eb00 con 0x56178254e1e0 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/998172080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 msgr2=0x56178259adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x56178259adc0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4006ec0 tx=0x7f1ad400b6c0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/998172080 shutdown_connections 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 0x56178261d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 0x56178259b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/998172080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x56178259adc0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/998172080 >> 172.21.15.19:0/998172080 conn(0x561782470ac0 msgr2=0x5617825803d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/998172080 shutdown_connections 2024-04-25T11:36:27.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.627+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/998172080 wait complete. 2024-04-25T11:36:27.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.628+0000 7f1ae8b2cf40 1 Processor -- start 2024-04-25T11:36:27.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.629+0000 7f1ae8b2cf40 1 -- start start 2024-04-25T11:36:27.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.630+0000 7f1ae8b2cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x5617826b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.630+0000 7f1ae8b2cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 0x5617826175c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.630+0000 7f1ae8b2cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 0x561782598fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.630+0000 7f1ae8b2cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56178261b470 con 0x56178254e1e0 2024-04-25T11:36:27.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.630+0000 7f1ae8b2cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56178254d850 con 0x561782627dd0 2024-04-25T11:36:27.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.630+0000 7f1ae8b2cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561782624740 con 0x56178254e5f0 2024-04-25T11:36:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x5617826b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 0x561782598fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x5617826b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33324/0 (socket says 172.21.15.19:33324) 2024-04-25T11:36:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 -- 172.21.15.19:0/3967842225 learned_addr learned my addr 172.21.15.19:0/3967842225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1adbe00640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 0x5617826175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 -- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 msgr2=0x5617826175c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56178254e5f0 0x5617826175c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 -- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 msgr2=0x561782598fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561782627dd0 0x561782598fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.631+0000 7f1ae1200640 1 -- 172.21.15.19:0/3967842225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617826b51a0 con 0x56178254e1e0 2024-04-25T11:36:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.632+0000 7f1ae1200640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x5617826b4c60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x56178257ce70 tx=0x7f1ad4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.632+0000 7f1ad9600640 1 -- 172.21.15.19:0/3967842225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4012050 con 0x56178254e1e0 2024-04-25T11:36:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.632+0000 7f1ad9600640 1 -- 172.21.15.19:0/3967842225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad4017860 con 0x56178254e1e0 2024-04-25T11:36:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.632+0000 7f1ad9600640 1 -- 172.21.15.19:0/3967842225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4017c50 con 0x56178254e1e0 2024-04-25T11:36:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.632+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56178261ae70 con 0x56178254e1e0 2024-04-25T11:36:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.632+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617826b4460 con 0x56178254e1e0 2024-04-25T11:36:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.635+0000 7f1ad9600640 1 -- 172.21.15.19:0/3967842225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ad4021070 con 0x56178254e1e0 2024-04-25T11:36:27.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.636+0000 7f1ad9600640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ac4072ec0 0x7f1ac40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.636+0000 7f1ad9600640 1 -- 172.21.15.19:0/3967842225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f1ad4096290 con 0x56178254e1e0 2024-04-25T11:36:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.636+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561782599520 0x56178261ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.636+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98) -- 0x56178261f0f0 con 0x561782599520 2024-04-25T11:36:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.636+0000 7f1adbe00640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ac4072ec0 0x7f1ac40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.637+0000 7f1ae1c00640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561782599520 0x56178261ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.638+0000 7f1adbe00640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ac4072ec0 0x7f1ac40753d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005fd0 tx=0x7f1ad000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.638+0000 7f1ae1c00640 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561782599520 0x56178261ebb0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae1c00640 1 -- 172.21.15.19:0/3967842225 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v98'31 uv31 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1adc0535c0 con 0x561782599520 2024-04-25T11:36:27.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561782599520 msgr2=0x56178261ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561782599520 0x56178261ebb0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ac4072ec0 msgr2=0x7f1ac40753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ac4072ec0 0x7f1ac40753d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005fd0 tx=0x7f1ad000d040 comp rx=0 tx=0).stop 2024-04-25T11:36:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 msgr2=0x5617826b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae8b2cf40 1 --2- 172.21.15.19:0/3967842225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56178254e1e0 0x5617826b4c60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x56178257ce70 tx=0x7f1ad4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.642+0000 7f1ae1c00640 1 -- 172.21.15.19:0/3967842225 reap_dead start 2024-04-25T11:36:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.643+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 shutdown_connections 2024-04-25T11:36:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.643+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 >> 172.21.15.19:0/3967842225 conn(0x561782470ac0 msgr2=0x561782599b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.643+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 shutdown_connections 2024-04-25T11:36:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.643+0000 7f1ae8b2cf40 1 -- 172.21.15.19:0/3967842225 wait complete. 2024-04-25T11:36:27.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:36:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.925+0000 7fd537e00640 1 Processor -- start 2024-04-25T11:36:27.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.925+0000 7fd537e00640 1 -- start start 2024-04-25T11:36:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.927+0000 7fd537e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381441d0 0x7fd538161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.927+0000 7fd537e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd538162040 0x7fd53806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.927+0000 7fd537e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd53806cb70 0x7fd53815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.927+0000 7fd537e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd538162450 con 0x7fd538162040 2024-04-25T11:36:27.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.927+0000 7fd537e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd53806d3a0 con 0x7fd53806cb70 2024-04-25T11:36:27.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.927+0000 7fd537e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd53806d680 con 0x7fd5381441d0 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.928+0000 7fd52e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd538162040 0x7fd53806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.928+0000 7fd52e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd538162040 0x7fd53806c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33340/0 (socket says 172.21.15.19:33340) 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.928+0000 7fd52e000640 1 -- 172.21.15.19:0/2326996183 learned_addr learned my addr 172.21.15.19:0/2326996183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.928+0000 7fd536a00640 1 --2- 172.21.15.19:0/2326996183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381441d0 0x7fd538161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.928+0000 7fd537400640 1 --2- 172.21.15.19:0/2326996183 >> v2:172.21.15.169:3300/0 conn(0x7fd53806cb70 0x7fd53815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.929+0000 7fd52e000640 1 -- 172.21.15.19:0/2326996183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381441d0 msgr2=0x7fd538161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.929+0000 7fd52e000640 1 --2- 172.21.15.19:0/2326996183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381441d0 0x7fd538161b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.929+0000 7fd52e000640 1 -- 172.21.15.19:0/2326996183 >> v2:172.21.15.169:3300/0 conn(0x7fd53806cb70 msgr2=0x7fd53815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.929+0000 7fd52e000640 1 --2- 172.21.15.19:0/2326996183 >> v2:172.21.15.169:3300/0 conn(0x7fd53806cb70 0x7fd53815e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.929+0000 7fd52e000640 1 -- 172.21.15.19:0/2326996183 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53806d960 con 0x7fd538162040 2024-04-25T11:36:27.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.930+0000 7fd52e000640 1 --2- 172.21.15.19:0/2326996183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd53806c630 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd53815c0d0 tx=0x7fd52002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07a9d8aa21321ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.931+0000 7fd536000640 1 -- 172.21.15.19:0/2326996183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520037070 con 0x7fd538162040 2024-04-25T11:36:27.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.931+0000 7fd536000640 1 -- 172.21.15.19:0/2326996183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd520032440 con 0x7fd538162040 2024-04-25T11:36:27.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.931+0000 7fd536000640 1 -- 172.21.15.19:0/2326996183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520032810 con 0x7fd538162040 2024-04-25T11:36:27.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 -- 172.21.15.19:0/2326996183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 msgr2=0x7fd53806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 --2- 172.21.15.19:0/2326996183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd53806c630 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd53815c0d0 tx=0x7fd52002a140 comp rx=0 tx=0).stop 2024-04-25T11:36:27.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 -- 172.21.15.19:0/2326996183 shutdown_connections 2024-04-25T11:36:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 --2- 172.21.15.19:0/2326996183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd53806c630 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 --2- 172.21.15.19:0/2326996183 >> v2:172.21.15.169:3300/0 conn(0x7fd53806cb70 0x7fd53815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 --2- 172.21.15.19:0/2326996183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5381441d0 0x7fd538161b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 -- 172.21.15.19:0/2326996183 >> 172.21.15.19:0/2326996183 conn(0x7fd538072f40 msgr2=0x7fd538073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 -- 172.21.15.19:0/2326996183 shutdown_connections 2024-04-25T11:36:27.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.932+0000 7fd537e00640 1 -- 172.21.15.19:0/2326996183 wait complete. 2024-04-25T11:36:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.934+0000 7fd537e00640 1 Processor -- start 2024-04-25T11:36:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.934+0000 7fd537e00640 1 -- start start 2024-04-25T11:36:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53806cb70 0x7fd5380ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5381441d0 0x7fd538143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd538160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd53806cf80 con 0x7fd538162040 2024-04-25T11:36:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381445e0 con 0x7fd5381441d0 2024-04-25T11:36:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd53806d100 con 0x7fd53806cb70 2024-04-25T11:36:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd538160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd538160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33356/0 (socket says 172.21.15.19:33356) 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.936+0000 7fd537400640 1 -- 172.21.15.19:0/624206842 learned_addr learned my addr 172.21.15.19:0/624206842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd536a00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53806cb70 0x7fd5380ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd52e000640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5381441d0 0x7fd538143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd537400640 1 -- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53806cb70 msgr2=0x7fd5380ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd537400640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53806cb70 0x7fd5380ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd537400640 1 -- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5381441d0 msgr2=0x7fd538143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd537400640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5381441d0 0x7fd538143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd537400640 1 -- 172.21.15.19:0/624206842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53815d750 con 0x7fd538162040 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd537400640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd538160990 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd53000eed0 tx=0x7fd53000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd52f400640 1 -- 172.21.15.19:0/624206842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530018020 con 0x7fd538162040 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd52f400640 1 -- 172.21.15.19:0/624206842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53000bd30 con 0x7fd538162040 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd52f400640 1 -- 172.21.15.19:0/624206842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530014640 con 0x7fd538162040 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.937+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53815da30 con 0x7fd538162040 2024-04-25T11:36:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.938+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53815dff0 con 0x7fd538162040 2024-04-25T11:36:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.939+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd538076c60 con 0x7fd538162040 2024-04-25T11:36:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.940+0000 7fd52f400640 1 -- 172.21.15.19:0/624206842 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd53001b020 con 0x7fd538162040 2024-04-25T11:36:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.941+0000 7fd52f400640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5140744b0 0x7fd5140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.941+0000 7fd52f400640 1 -- 172.21.15.19:0/624206842 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fd530095e10 con 0x7fd538162040 2024-04-25T11:36:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.942+0000 7fd536a00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5140744b0 0x7fd5140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:27.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.943+0000 7fd536a00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5140744b0 0x7fd5140769c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd528009970 tx=0x7fd528008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:27.946+0000 7fd52f400640 1 -- 172.21.15.19:0/624206842 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd530099030 con 0x7fd538162040 2024-04-25T11:36:28.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.191+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd53813dab0 con 0x7fd538162040 2024-04-25T11:36:28.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.599+0000 7fd52f400640 1 -- 172.21.15.19:0/624206842 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v129) ==== 162+0+0 (secure 0 0 0) 0x7fd530022020 con 0x7fd538162040 2024-04-25T11:36:28.588 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:36:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.602+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5140744b0 msgr2=0x7fd5140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5140744b0 0x7fd5140769c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd528009970 tx=0x7fd528008040 comp rx=0 tx=0).stop 2024-04-25T11:36:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 msgr2=0x7fd538160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd538160990 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd53000eed0 tx=0x7fd53000b480 comp rx=0 tx=0).stop 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 shutdown_connections 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd5140744b0 0x7fd5140769c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5381441d0 0x7fd538143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd538162040 0x7fd538160990 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 --2- 172.21.15.19:0/624206842 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd53806cb70 0x7fd5380ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 >> 172.21.15.19:0/624206842 conn(0x7fd538072f40 msgr2=0x7fd5380b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 shutdown_connections 2024-04-25T11:36:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:28.603+0000 7fd537e00640 1 -- 172.21.15.19:0/624206842 wait complete. 2024-04-25T11:36:28.606 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:28.617+0000 7fc0c8000640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-04-25T11:36:29.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.321+0000 7f0d5f200640 1 Processor -- start 2024-04-25T11:36:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.321+0000 7f0d5f200640 1 -- start start 2024-04-25T11:36:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.323+0000 7f0d5f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6006c5c0 0x7f0d600cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.323+0000 7f0d5f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d600b4ea0 0x7f0d6015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.323+0000 7f0d5f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d600d1230 con 0x7f0d60144370 2024-04-25T11:36:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.323+0000 7f0d5f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0d600d06b0 con 0x7f0d600b4ea0 2024-04-25T11:36:29.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.323+0000 7f0d5f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0d600d0990 con 0x7f0d6006c5c0 2024-04-25T11:36:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.324+0000 7f0d5de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6006c5c0 0x7f0d600cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.324+0000 7f0d55400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f0d60144370 0x7f0d600b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:50818/0 (socket says 172.21.15.19:50818) 2024-04-25T11:36:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.324+0000 7f0d55400640 1 -- 172.21.15.19:0/1035800225 learned_addr learned my addr 172.21.15.19:0/1035800225 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:29.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.324+0000 7f0d5e800640 1 --2- 172.21.15.19:0/1035800225 >> v2:172.21.15.169:3300/0 conn(0x7f0d600b4ea0 0x7f0d6015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.325+0000 7f0d5d400640 1 -- 172.21.15.19:0/1035800225 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873151401 0 0) 0x7f0d600d1230 con 0x7f0d60144370 2024-04-25T11:36:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.325+0000 7f0d5d400640 1 -- 172.21.15.19:0/1035800225 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d44003520 con 0x7f0d60144370 2024-04-25T11:36:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.325+0000 7f0d5de00640 1 -- 172.21.15.19:0/1035800225 >> v2:172.21.15.169:3300/0 conn(0x7f0d600b4ea0 msgr2=0x7f0d6015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.325+0000 7f0d5de00640 1 --2- 172.21.15.19:0/1035800225 >> v2:172.21.15.169:3300/0 conn(0x7f0d600b4ea0 0x7f0d6015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.325+0000 7f0d5de00640 1 -- 172.21.15.19:0/1035800225 >> v1:172.21.15.169:6789/0 conn(0x7f0d60144370 legacy=0x7f0d600b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.325+0000 7f0d5de00640 1 -- 172.21.15.19:0/1035800225 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d600d0c70 con 0x7f0d6006c5c0 2024-04-25T11:36:29.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.327+0000 7f0d5de00640 1 --2- 172.21.15.19:0/1035800225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6006c5c0 0x7f0d600cfa70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d90 tx=0x7f0d5002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e01812291b87b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:29.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.328+0000 7f0d5d400640 1 -- 172.21.15.19:0/1035800225 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50007740 con 0x7f0d6006c5c0 2024-04-25T11:36:29.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.328+0000 7f0d5d400640 1 -- 172.21.15.19:0/1035800225 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d500078e0 con 0x7f0d6006c5c0 2024-04-25T11:36:29.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.328+0000 7f0d5d400640 1 -- 172.21.15.19:0/1035800225 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50007cb0 con 0x7f0d6006c5c0 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 -- 172.21.15.19:0/1035800225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6006c5c0 msgr2=0x7f0d600cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 --2- 172.21.15.19:0/1035800225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6006c5c0 0x7f0d600cfa70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d90 tx=0x7f0d5002a660 comp rx=0 tx=0).stop 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 -- 172.21.15.19:0/1035800225 shutdown_connections 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 --2- 172.21.15.19:0/1035800225 >> v2:172.21.15.169:3300/0 conn(0x7f0d600b4ea0 0x7f0d6015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 --2- 172.21.15.19:0/1035800225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d6006c5c0 0x7f0d600cfa70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 -- 172.21.15.19:0/1035800225 >> 172.21.15.19:0/1035800225 conn(0x7f0d600d1720 msgr2=0x7f0d600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 -- 172.21.15.19:0/1035800225 shutdown_connections 2024-04-25T11:36:29.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.329+0000 7f0d5f200640 1 -- 172.21.15.19:0/1035800225 wait complete. 2024-04-25T11:36:29.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.331+0000 7f0d5f200640 1 Processor -- start 2024-04-25T11:36:29.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.332+0000 7f0d5f200640 1 -- start start 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.333+0000 7f0d5f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6006c5c0 0x7f0d600cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.333+0000 7f0d5f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d60144370 0x7f0d60140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.333+0000 7f0d5f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d60145f50 0x7f0d60146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.333+0000 7f0d5f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600d0450 con 0x7f0d60144370 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.333+0000 7f0d5f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600d02d0 con 0x7f0d6006c5c0 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.333+0000 7f0d5f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0d60147760 con 0x7f0d60145f50 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d60144370 0x7f0d60140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d60144370 0x7f0d60140d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33372/0 (socket says 172.21.15.19:33372) 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 -- 172.21.15.19:0/4166902133 learned_addr learned my addr 172.21.15.19:0/4166902133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d5e800640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d60145f50 0x7f0d60146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d5de00640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6006c5c0 0x7f0d600cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 -- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d60145f50 msgr2=0x7f0d60146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d60145f50 0x7f0d60146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 -- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6006c5c0 msgr2=0x7f0d600cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6006c5c0 0x7f0d600cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 -- 172.21.15.19:0/4166902133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d601468a0 con 0x7f0d60144370 2024-04-25T11:36:29.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d5de00640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6006c5c0 0x7f0d600cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:29.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.334+0000 7f0d55400640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d60144370 0x7f0d60140d80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800c3c0 tx=0x7f0d4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:29.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.335+0000 7f0d56a00640 1 -- 172.21.15.19:0/4166902133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48017070 con 0x7f0d60144370 2024-04-25T11:36:29.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.335+0000 7f0d56a00640 1 -- 172.21.15.19:0/4166902133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d48005ce0 con 0x7f0d60144370 2024-04-25T11:36:29.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.335+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6006f460 con 0x7f0d60144370 2024-04-25T11:36:29.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.335+0000 7f0d56a00640 1 -- 172.21.15.19:0/4166902133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48007310 con 0x7f0d60144370 2024-04-25T11:36:29.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.335+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6006f740 con 0x7f0d60144370 2024-04-25T11:36:29.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.336+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d60050740 con 0x7f0d60144370 2024-04-25T11:36:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.338+0000 7f0d56a00640 1 -- 172.21.15.19:0/4166902133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0d4801a070 con 0x7f0d60144370 2024-04-25T11:36:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.339+0000 7f0d56a00640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d3c0744b0 0x7f0d3c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.339+0000 7f0d56a00640 1 -- 172.21.15.19:0/4166902133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f0d48094850 con 0x7f0d60144370 2024-04-25T11:36:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.340+0000 7f0d5de00640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d3c0744b0 0x7f0d3c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.342+0000 7f0d5de00640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d3c0744b0 0x7f0d3c0769c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0d50034000 tx=0x7f0d50033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.345+0000 7f0d56a00640 1 -- 172.21.15.19:0/4166902133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d601468a0 con 0x7f0d60144370 2024-04-25T11:36:29.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.593+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d6006ccb0 con 0x7f0d60144370 2024-04-25T11:36:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.594+0000 7f0d56a00640 1 -- 172.21.15.19:0/4166902133 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) ==== 95+0+5183 (secure 0 0 0) 0x7f0d4805ed30 con 0x7f0d60144370 2024-04-25T11:36:29.583 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:29.583 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":130,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:28.606820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8346},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8346":{"gid":8346,"name":"a","rank":0,"incarnation":130,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/432323448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":432323448},{"type":"v1","addr":"172.21.15.19:6835","nonce":432323448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 130 2024-04-25T11:36:29.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d3c0744b0 msgr2=0x7f0d3c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d3c0744b0 0x7f0d3c0769c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0d50034000 tx=0x7f0d50033040 comp rx=0 tx=0).stop 2024-04-25T11:36:29.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d60144370 msgr2=0x7f0d60140d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d60144370 0x7f0d60140d80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800c3c0 tx=0x7f0d4800c880 comp rx=0 tx=0).stop 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 shutdown_connections 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d3c0744b0 0x7f0d3c0769c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d60145f50 0x7f0d60146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d60144370 0x7f0d60140d80 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.597+0000 7f0d5f200640 1 --2- 172.21.15.19:0/4166902133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d6006c5c0 0x7f0d600cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.598+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 >> 172.21.15.19:0/4166902133 conn(0x7f0d600d1720 msgr2=0x7f0d6006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.598+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 shutdown_connections 2024-04-25T11:36:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:29.598+0000 7f0d5f200640 1 -- 172.21.15.19:0/4166902133 wait complete. 2024-04-25T11:36:29.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 130, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:36:28.606820+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8346}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8346': {'gid': 8346, 'name': 'a', 'rank': 0, 'incarnation': 130, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/432323448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 432323448}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 432323448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:36:29.616 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8346:up:replay 2024-04-25T11:36:29.616 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:36:33.556 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:33.567+0000 7fc0cd000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:36:35.021 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:35.140 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:36:35.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:36:35.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.331+0000 7f3a4ca00640 1 Processor -- start 2024-04-25T11:36:35.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.331+0000 7f3a4ca00640 1 -- start start 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.333+0000 7f3a4ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a480b9050 0x7f3a480b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.333+0000 7f3a4ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480cf660 0x7f3a480ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.333+0000 7f3a4ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3a480ceee0 0x7f3a480b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.333+0000 7f3a4ca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3a480d2ba0 con 0x7f3a480b9050 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.333+0000 7f3a4ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3a480cfa70 con 0x7f3a480ceee0 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.333+0000 7f3a4ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a480d28c0 con 0x7f3a480cf660 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a480b9050 0x7f3a480b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a480b9050 0x7f3a480b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40516/0 (socket says 172.21.15.19:40516) 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 -- 172.21.15.19:0/2908882186 learned_addr learned my addr 172.21.15.19:0/2908882186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a46a00640 1 --2- 172.21.15.19:0/2908882186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480cf660 0x7f3a480ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 -- 172.21.15.19:0/2908882186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480cf660 msgr2=0x7f3a480ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47e00640 1 --2- 172.21.15.19:0/2908882186 >> v2:172.21.15.169:3300/0 conn(0x7f3a480ceee0 0x7f3a480b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 --2- 172.21.15.19:0/2908882186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480cf660 0x7f3a480ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 -- 172.21.15.19:0/2908882186 >> v2:172.21.15.169:3300/0 conn(0x7f3a480ceee0 msgr2=0x7f3a480b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 --2- 172.21.15.19:0/2908882186 >> v2:172.21.15.169:3300/0 conn(0x7f3a480ceee0 0x7f3a480b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47400640 1 -- 172.21.15.19:0/2908882186 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4809cc50 con 0x7f3a480b9050 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a46a00640 1 --2- 172.21.15.19:0/2908882186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480cf660 0x7f3a480ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.334+0000 7f3a47e00640 1 --2- 172.21.15.19:0/2908882186 >> v2:172.21.15.169:3300/0 conn(0x7f3a480ceee0 0x7f3a480b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.335+0000 7f3a47400640 1 --2- 172.21.15.19:0/2908882186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9050 0x7f3a480b8990 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3a48073df0 tx=0x7f3a3802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8897a04b1309eed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.336+0000 7f3a46000640 1 -- 172.21.15.19:0/2908882186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38037070 con 0x7f3a480b9050 2024-04-25T11:36:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.336+0000 7f3a46000640 1 -- 172.21.15.19:0/2908882186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a38032440 con 0x7f3a480b9050 2024-04-25T11:36:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.336+0000 7f3a46000640 1 -- 172.21.15.19:0/2908882186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38032810 con 0x7f3a480b9050 2024-04-25T11:36:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 -- 172.21.15.19:0/2908882186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9050 msgr2=0x7f3a480b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/2908882186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9050 0x7f3a480b8990 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3a48073df0 tx=0x7f3a3802a140 comp rx=0 tx=0).stop 2024-04-25T11:36:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 -- 172.21.15.19:0/2908882186 shutdown_connections 2024-04-25T11:36:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/2908882186 >> v2:172.21.15.169:3300/0 conn(0x7f3a480ceee0 0x7f3a480b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/2908882186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480cf660 0x7f3a480ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/2908882186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480b9050 0x7f3a480b8990 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 -- 172.21.15.19:0/2908882186 >> 172.21.15.19:0/2908882186 conn(0x7f3a480d03f0 msgr2=0x7f3a480d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 -- 172.21.15.19:0/2908882186 shutdown_connections 2024-04-25T11:36:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.337+0000 7f3a4ca00640 1 -- 172.21.15.19:0/2908882186 wait complete. 2024-04-25T11:36:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.340+0000 7f3a4ca00640 1 Processor -- start 2024-04-25T11:36:35.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.340+0000 7f3a4ca00640 1 -- start start 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.341+0000 7f3a4ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480b9050 0x7f3a481e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.341+0000 7f3a4ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a480ceee0 0x7f3a480a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.341+0000 7f3a4ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480cf660 0x7f3a4806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.341+0000 7f3a4ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a480b7890 con 0x7f3a480cf660 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.341+0000 7f3a4ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3a480b7710 con 0x7f3a480ceee0 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.341+0000 7f3a4ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3a480b7a10 con 0x7f3a480b9050 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480cf660 0x7f3a4806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480cf660 0x7f3a4806eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40526/0 (socket says 172.21.15.19:40526) 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 -- 172.21.15.19:0/1865544674 learned_addr learned my addr 172.21.15.19:0/1865544674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47400640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480b9050 0x7f3a481e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a46a00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a480ceee0 0x7f3a480a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 -- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480b9050 msgr2=0x7f3a481e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480b9050 0x7f3a481e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 -- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a480ceee0 msgr2=0x7f3a480a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a480ceee0 0x7f3a480a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47e00640 1 -- 172.21.15.19:0/1865544674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a480a01f0 con 0x7f3a480cf660 2024-04-25T11:36:35.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.342+0000 7f3a47400640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480b9050 0x7f3a481e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.343+0000 7f3a47e00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480cf660 0x7f3a4806eac0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000cc90 tx=0x7f3a4000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.343+0000 7f3a3f400640 1 -- 172.21.15.19:0/1865544674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000f680 con 0x7f3a480cf660 2024-04-25T11:36:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.343+0000 7f3a3f400640 1 -- 172.21.15.19:0/1865544674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4000f820 con 0x7f3a480cf660 2024-04-25T11:36:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.343+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a480cd260 con 0x7f3a480cf660 2024-04-25T11:36:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.343+0000 7f3a3f400640 1 -- 172.21.15.19:0/1865544674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000fc10 con 0x7f3a480cf660 2024-04-25T11:36:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.343+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4809ff10 con 0x7f3a480cf660 2024-04-25T11:36:35.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.344+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a48050740 con 0x7f3a480cf660 2024-04-25T11:36:35.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.346+0000 7f3a3f400640 1 -- 172.21.15.19:0/1865544674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a400213f0 con 0x7f3a480cf660 2024-04-25T11:36:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.347+0000 7f3a3f400640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a240744b0 0x7f3a240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.347+0000 7f3a3f400640 1 -- 172.21.15.19:0/1865544674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f3a400961b0 con 0x7f3a480cf660 2024-04-25T11:36:35.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.348+0000 7f3a47400640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a240744b0 0x7f3a240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.349+0000 7f3a47400640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a240744b0 0x7f3a240769c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a38034000 tx=0x7f3a3802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.352+0000 7f3a3f400640 1 -- 172.21.15.19:0/1865544674 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a480a01f0 con 0x7f3a480cf660 2024-04-25T11:36:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.602+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a480b81c0 con 0x7f3a480cf660 2024-04-25T11:36:35.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.604+0000 7f3a3f400640 1 -- 172.21.15.19:0/1865544674 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) ==== 95+0+5190 (secure 0 0 0) 0x7f3a40014030 con 0x7f3a480cf660 2024-04-25T11:36:35.592 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:35.593 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":133,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:31.634052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8346},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8346":{"gid":8346,"name":"a","rank":0,"incarnation":130,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6835/432323448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":432323448},{"type":"v1","addr":"172.21.15.19:6835","nonce":432323448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8346,"qdb_cluster":[8346]},"id":6}]} 2024-04-25T11:36:35.593 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 133 2024-04-25T11:36:35.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.606+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a240744b0 msgr2=0x7f3a240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.606+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a240744b0 0x7f3a240769c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a38034000 tx=0x7f3a3802e000 comp rx=0 tx=0).stop 2024-04-25T11:36:35.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.606+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480cf660 msgr2=0x7f3a4806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.606+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480cf660 0x7f3a4806eac0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000cc90 tx=0x7f3a4000a650 comp rx=0 tx=0).stop 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 shutdown_connections 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3a240744b0 0x7f3a240769c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3a480ceee0 0x7f3a480a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a480cf660 0x7f3a4806eac0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 --2- 172.21.15.19:0/1865544674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3a480b9050 0x7f3a481e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 >> 172.21.15.19:0/1865544674 conn(0x7f3a480d03f0 msgr2=0x7f3a481e7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 shutdown_connections 2024-04-25T11:36:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.607+0000 7f3a4ca00640 1 -- 172.21.15.19:0/1865544674 wait complete. 2024-04-25T11:36:35.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 133, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:36:31.634052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8346}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8346': {'gid': 8346, 'name': 'a', 'rank': 0, 'incarnation': 130, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6835/432323448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 432323448}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 432323448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8346, 'qdb_cluster': [8346]} 2024-04-25T11:36:35.622 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:36:35.623 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:36:35.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.913+0000 7f01bca00640 1 Processor -- start 2024-04-25T11:36:35.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.914+0000 7f01bca00640 1 -- start start 2024-04-25T11:36:35.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.916+0000 7f01bca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01b806d280 0x7f01b8161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.916+0000 7f01bca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b80b4790 0x7f01b80b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.916+0000 7f01bca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b8162460 con 0x7f01b8168090 2024-04-25T11:36:35.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.916+0000 7f01bca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f01b80b54c0 con 0x7f01b806d280 2024-04-25T11:36:35.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.916+0000 7f01bca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01b806c7a0 con 0x7f01b80b4790 2024-04-25T11:36:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.917+0000 7f01b7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b80b4790 0x7f01b80b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.917+0000 7f01b7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b80b4790 0x7f01b80b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:40020/0 (socket says 172.21.15.19:40020) 2024-04-25T11:36:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.917+0000 7f01b7200640 1 -- 172.21.15.19:0/1289905390 learned_addr learned my addr 172.21.15.19:0/1289905390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.917+0000 7f01b6800640 1 --2- 172.21.15.19:0/1289905390 >> v2:172.21.15.169:3300/0 conn(0x7f01b806d280 0x7f01b8161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.918+0000 7f01b5400640 1 -- 172.21.15.19:0/1289905390 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154757020 0 0) 0x7f01b8162460 con 0x7f01b8168090 2024-04-25T11:36:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.918+0000 7f01b5400640 1 -- 172.21.15.19:0/1289905390 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0198003520 con 0x7f01b8168090 2024-04-25T11:36:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.918+0000 7f01b7200640 1 -- 172.21.15.19:0/1289905390 >> v2:172.21.15.169:3300/0 conn(0x7f01b806d280 msgr2=0x7f01b8161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.918+0000 7f01b7200640 1 --2- 172.21.15.19:0/1289905390 >> v2:172.21.15.169:3300/0 conn(0x7f01b806d280 0x7f01b8161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.918+0000 7f01b7200640 1 -- 172.21.15.19:0/1289905390 >> v1:172.21.15.169:6789/0 conn(0x7f01b8168090 legacy=0x7f01b81684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.918+0000 7f01b7200640 1 -- 172.21.15.19:0/1289905390 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b806ca80 con 0x7f01b80b4790 2024-04-25T11:36:35.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.920+0000 7f01b7200640 1 --2- 172.21.15.19:0/1289905390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b80b4790 0x7f01b80b4ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009e50 tx=0x7f01b002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548ecc9703f9c5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.920+0000 7f01b5400640 1 -- 172.21.15.19:0/1289905390 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0036070 con 0x7f01b80b4790 2024-04-25T11:36:35.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.920+0000 7f01b5400640 1 -- 172.21.15.19:0/1289905390 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b0031600 con 0x7f01b80b4790 2024-04-25T11:36:35.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.920+0000 7f01b5400640 1 -- 172.21.15.19:0/1289905390 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b00319d0 con 0x7f01b80b4790 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 -- 172.21.15.19:0/1289905390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b80b4790 msgr2=0x7f01b80b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 --2- 172.21.15.19:0/1289905390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b80b4790 0x7f01b80b4ba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009e50 tx=0x7f01b002a710 comp rx=0 tx=0).stop 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 -- 172.21.15.19:0/1289905390 shutdown_connections 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 --2- 172.21.15.19:0/1289905390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b80b4790 0x7f01b80b4ba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 --2- 172.21.15.19:0/1289905390 >> v2:172.21.15.169:3300/0 conn(0x7f01b806d280 0x7f01b8161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 -- 172.21.15.19:0/1289905390 >> 172.21.15.19:0/1289905390 conn(0x7f01b80d1720 msgr2=0x7f01b80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 -- 172.21.15.19:0/1289905390 shutdown_connections 2024-04-25T11:36:35.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.921+0000 7f01bca00640 1 -- 172.21.15.19:0/1289905390 wait complete. 2024-04-25T11:36:35.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.924+0000 7f01bca00640 1 Processor -- start 2024-04-25T11:36:35.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.924+0000 7f01bca00640 1 -- start start 2024-04-25T11:36:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.925+0000 7f01bca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01b806d280 0x7f01b814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.925+0000 7f01bca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b80b4790 0x7f01b814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.925+0000 7f01bca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b8168090 0x7f01b8164860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01bca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01b80b50e0 con 0x7f01b806d280 2024-04-25T11:36:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01bca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01b806d690 con 0x7f01b80b4790 2024-04-25T11:36:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01bca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01b80b5260 con 0x7f01b8168090 2024-04-25T11:36:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01b806d280 0x7f01b814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01b806d280 0x7f01b814f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40540/0 (socket says 172.21.15.19:40540) 2024-04-25T11:36:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 -- 172.21.15.19:0/597919715 learned_addr learned my addr 172.21.15.19:0/597919715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b7200640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b8168090 0x7f01b8164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 -- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b8168090 msgr2=0x7f01b8164860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b8168090 0x7f01b8164860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 -- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b80b4790 msgr2=0x7f01b814f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b5e00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b80b4790 0x7f01b814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b80b4790 0x7f01b814f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b6800640 1 -- 172.21.15.19:0/597919715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b81efad0 con 0x7f01b806d280 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.926+0000 7f01b7200640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b8168090 0x7f01b8164860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.927+0000 7f01b5e00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b80b4790 0x7f01b814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.927+0000 7f01b6800640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01b806d280 0x7f01b814f110 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e8a0 tx=0x7f01a800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.927+0000 7f01aea00640 1 -- 172.21.15.19:0/597919715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8017070 con 0x7f01b806d280 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.927+0000 7f01aea00640 1 -- 172.21.15.19:0/597919715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a800c940 con 0x7f01b806d280 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.927+0000 7f01aea00640 1 -- 172.21.15.19:0/597919715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a800cd30 con 0x7f01b806d280 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.927+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b81efdb0 con 0x7f01b806d280 2024-04-25T11:36:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.927+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b81f0370 con 0x7f01b806d280 2024-04-25T11:36:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.930+0000 7f01aea00640 1 -- 172.21.15.19:0/597919715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f01a801a070 con 0x7f01b806d280 2024-04-25T11:36:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.931+0000 7f01aea00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01940744b0 0x7f01940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.931+0000 7f01aea00640 1 -- 172.21.15.19:0/597919715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f01a80957b0 con 0x7f01b806d280 2024-04-25T11:36:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.932+0000 7f01b5e00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01940744b0 0x7f01940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.932+0000 7f01c43df740 1 Processor -- start 2024-04-25T11:36:35.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.932+0000 7f01c43df740 1 -- start start 2024-04-25T11:36:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.933+0000 7f01b5e00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01940744b0 0x7f01940769c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f01b80b6070 tx=0x7f01a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.933+0000 7f01c43df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9dc9270 0x55d0fa196210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.933+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9da8590 0x55d0fa1b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01c43df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0fa0cf600 con 0x55d0f9dc9270 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d0fa06c100 con 0x55d0f9da8590 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d0fa0cb5c0 con 0x55d0f9da03b0 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9dc9270 0x55d0fa196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9dc9270 0x55d0fa196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40544/0 (socket says 172.21.15.19:40544) 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 -- 172.21.15.19:0/306310909 learned_addr learned my addr 172.21.15.19:0/306310909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b5e00640 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 -- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 msgr2=0x55d0fa1b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 -- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9da8590 msgr2=0x55d0fa1b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9da8590 0x55d0fa1b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b6800640 1 -- 172.21.15.19:0/306310909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f9de12d0 con 0x55d0f9dc9270 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.934+0000 7f01b5e00640 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.935+0000 7f01b6800640 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9dc9270 0x55d0fa196210 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f01a805bb80 tx=0x7f01a805f0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600a1ddf70376a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.935+0000 7f01a7000640 1 -- 172.21.15.19:0/306310909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8021030 con 0x55d0f9dc9270 2024-04-25T11:36:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.935+0000 7f01a7000640 1 -- 172.21.15.19:0/306310909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a805f590 con 0x55d0f9dc9270 2024-04-25T11:36:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.935+0000 7f01a7000640 1 -- 172.21.15.19:0/306310909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a805ea10 con 0x55d0f9dc9270 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 -- 172.21.15.19:0/306310909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9dc9270 msgr2=0x55d0fa196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9dc9270 0x55d0fa196210 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f01a805bb80 tx=0x7f01a805f0a0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 -- 172.21.15.19:0/306310909 shutdown_connections 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9da8590 0x55d0fa1b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 --2- 172.21.15.19:0/306310909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9dc9270 0x55d0fa196210 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 -- 172.21.15.19:0/306310909 >> 172.21.15.19:0/306310909 conn(0x55d0f9dbd3c0 msgr2=0x55d0fa16b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 -- 172.21.15.19:0/306310909 shutdown_connections 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.936+0000 7f01c43df740 1 -- 172.21.15.19:0/306310909 wait complete. 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.937+0000 7f01c43df740 1 -- start start 2024-04-25T11:36:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.937+0000 7f01c43df740 1 Processor -- start 2024-04-25T11:36:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa191940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01c43df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa191e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa1abb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01c43df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0fa056f70 con 0x55d0f9da8590 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d0fa198420 con 0x55d0f9dc9270 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d0fa198880 con 0x55d0f9da03b0 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01c43df740 10 client.0 _ll_register_callbacks cb 0x55d0fa0c5480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa191e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa191e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40552/0 (socket says 172.21.15.19:40552) 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01b5e00640 1 -- 172.21.15.19:0/2109334097 learned_addr learned my addr 172.21.15.19:0/2109334097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.939+0000 7f01b6800640 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa191940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01b5e00640 1 -- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 msgr2=0x55d0fa191940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01b7200640 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa1abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01b5e00640 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa191940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01b5e00640 1 -- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 msgr2=0x55d0fa1abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01b5e00640 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa1abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01b5e00640 1 -- 172.21.15.19:0/2109334097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f9eab6b0 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01b5e00640 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa191e80 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f01a000dd20 tx=0x7f01a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1c648393a6c222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f019f400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f019f400640 1 -- 172.21.15.19:0/2109334097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a001f610 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a805f590 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.940+0000 7f019f400640 1 -- 172.21.15.19:0/2109334097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f01a001f7b0 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f019f400640 1 -- 172.21.15.19:0/2109334097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) ==== 24+0+0 (secure 0 0 0) 0x7f01a002c050 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 10 client.8433 fetch_fsmap learned FSMap version 133 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) -- 0x55d0fa1985c0 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f019f400640 1 -- 172.21.15.19:0/2109334097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a805f590 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f019f400640 1 -- 172.21.15.19:0/2109334097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 133) ==== 3494+0+0 (secure 0 0 0) 0x7f01a002f020 con 0x55d0f9da8590 2024-04-25T11:36:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 10 client.8433 fetch_fsmap finished waiting for FSMap version 133 2024-04-25T11:36:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 10 client.8433 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:36:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 20 client.8433 populate_metadata read hostname 'smithi019' 2024-04-25T11:36:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0f9d87e40 0x55d0fa1c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 4 client.8433 mds_command: new command op to 8346 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:36:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 --> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0fa0dd210 con 0x55d0f9d87e40 2024-04-25T11:36:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.941+0000 7f01b6800640 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0f9d87e40 0x55d0fa1c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.942+0000 7f01b6800640 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0f9d87e40 0x55d0fa1c3d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696abcb3176bd91e server_cookie=bb37c3f5a7685457 in_seq=0 out_seq=0 2024-04-25T11:36:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.942+0000 7f019f400640 10 client.8433 ms_handle_connect on v2:172.21.15.19:6834/432323448 2024-04-25T11:36:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.943+0000 7f019f400640 1 -- 172.21.15.19:0/2109334097 <== mds.0 v2:172.21.15.19:6834/432323448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0fa0dd210 con 0x55d0f9d87e40 2024-04-25T11:36:35.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.943+0000 7f019f400640 10 client.8433 handle_command_reply: tid=0 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.943+0000 7f01c43df740 1 client.8433 shutdown 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.943+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 msgr2=0x55d0fa191e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.943+0000 7f01c43df740 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa191e80 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f01a000dd20 tx=0x7f01a0000ed0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 shutdown_connections 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0f9d87e40 0x55d0fa1c3d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa1abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa191e80 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 --2- 172.21.15.19:0/2109334097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa191940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f019f400640 0 client.8433 ms_handle_reset on v2:172.21.15.19:6834/432323448 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 >> 172.21.15.19:0/2109334097 conn(0x55d0f9dbd3c0 msgr2=0x55d0fa16b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 shutdown_connections 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 1 -- 172.21.15.19:0/2109334097 wait complete. 2024-04-25T11:36:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.944+0000 7f01c43df740 20 client.8433 trim_cache size 0 max 16384 2024-04-25T11:36:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.965+0000 7f01c43df740 1 Processor -- start 2024-04-25T11:36:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.965+0000 7f01c43df740 1 -- start start 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.966+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa196400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.966+0000 7f01c43df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.966+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa1acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.966+0000 7f01c43df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01b814c120 con 0x55d0f9da8590 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.966+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01b81fec30 con 0x55d0f9dc9270 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.966+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01b814bad0 con 0x55d0f9da03b0 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40566/0 (socket says 172.21.15.19:40566) 2024-04-25T11:36:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 -- 172.21.15.19:0/3011158544 learned_addr learned my addr 172.21.15.19:0/3011158544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 -- 172.21.15.19:0/3011158544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 msgr2=0x55d0fa196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b6800640 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 -- 172.21.15.19:0/3011158544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 msgr2=0x55d0fa1acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa1acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.967+0000 7f01b5e00640 1 -- 172.21.15.19:0/3011158544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fa1b2660 con 0x55d0f9da8590 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.968+0000 7f01b5e00640 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1aca10 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f01a0016010 tx=0x7f01a001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d716534a9916a026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.968+0000 7f01a6600640 1 -- 172.21.15.19:0/3011158544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0012710 con 0x55d0f9da8590 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.968+0000 7f01a6600640 1 -- 172.21.15.19:0/3011158544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a0024070 con 0x55d0f9da8590 2024-04-25T11:36:35.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.968+0000 7f01a6600640 1 -- 172.21.15.19:0/3011158544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0012ad0 con 0x55d0f9da8590 2024-04-25T11:36:35.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 -- 172.21.15.19:0/3011158544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 msgr2=0x55d0fa1aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1aca10 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f01a0016010 tx=0x7f01a001d000 comp rx=0 tx=0).stop 2024-04-25T11:36:35.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 -- 172.21.15.19:0/3011158544 shutdown_connections 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa1acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1aca10 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 --2- 172.21.15.19:0/3011158544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 -- 172.21.15.19:0/3011158544 >> 172.21.15.19:0/3011158544 conn(0x55d0f9dbd3c0 msgr2=0x55d0fa195650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 -- 172.21.15.19:0/3011158544 shutdown_connections 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.969+0000 7f01c43df740 1 -- 172.21.15.19:0/3011158544 wait complete. 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.970+0000 7f01c43df740 1 -- start start 2024-04-25T11:36:35.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.970+0000 7f01c43df740 1 Processor -- start 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01c43df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01c43df740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa198d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01c43df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d0fa18f480 con 0x55d0f9da8590 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d0fa18ff30 con 0x55d0f9dc9270 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01c43df740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d0fa1ac240 con 0x55d0f9da03b0 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01c43df740 10 client.0 _ll_register_callbacks cb 0x7f01a0001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40568/0 (socket says 172.21.15.19:40568) 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b5e00640 1 -- 172.21.15.19:0/1090467700 learned_addr learned my addr 172.21.15.19:0/1090467700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b6800640 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b5e00640 1 -- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 msgr2=0x55d0fa1930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b5e00640 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b5e00640 1 -- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 msgr2=0x55d0fa198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.972+0000 7f01b5e00640 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.973+0000 7f01b5e00640 1 -- 172.21.15.19:0/1090467700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fa16a640 con 0x55d0f9da8590 2024-04-25T11:36:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.973+0000 7f01b6800640 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.973+0000 7f01b5e00640 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1987f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f01a000d590 tx=0x7f01a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1582ffcc40fcdc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.973+0000 7f019f400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.973+0000 7f019f400640 1 -- 172.21.15.19:0/1090467700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0039680 con 0x55d0f9da8590 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.973+0000 7f019f400640 1 -- 172.21.15.19:0/1090467700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f01a0021d10 con 0x55d0f9da8590 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.973+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a0024070 con 0x55d0f9da8590 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f019f400640 1 -- 172.21.15.19:0/1090467700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=133) ==== 24+0+0 (secure 0 0 0) 0x7f01a0039820 con 0x55d0f9da8590 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 10 client.8439 fetch_fsmap learned FSMap version 133 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=133,monmap=2+}) -- 0x55d0fa199700 con 0x55d0f9da8590 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f019f400640 1 -- 172.21.15.19:0/1090467700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a0024070 con 0x55d0f9da8590 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f019f400640 1 -- 172.21.15.19:0/1090467700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 133) ==== 3494+0+0 (secure 0 0 0) 0x7f01a002f040 con 0x55d0f9da8590 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 10 client.8439 fetch_fsmap finished waiting for FSMap version 133 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 10 client.8439 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 20 client.8439 populate_metadata read hostname 'smithi019' 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0fa22e9f0 0x55d0fa1c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 4 client.8439 mds_command: new command op to 8346 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 --> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] -- command(tid 0: {"prefix": "status"}) -- 0x55d0fa0d7080 con 0x55d0fa22e9f0 2024-04-25T11:36:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.974+0000 7f01b6800640 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0fa22e9f0 0x55d0fa1c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.975+0000 7f019f400640 10 client.8439 ms_handle_connect on v2:172.21.15.19:6834/432323448 2024-04-25T11:36:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.975+0000 7f01b6800640 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0fa22e9f0 0x55d0fa1c22e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a9ba533169f728 server_cookie=cf1d906a7f75a339 in_seq=0 out_seq=0 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.975+0000 7f019f400640 1 -- 172.21.15.19:0/1090467700 <== mds.0 v2:172.21.15.19:6834/432323448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d0fa0d7080 con 0x55d0fa22e9f0 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.975+0000 7f019f400640 10 client.8439 handle_command_reply: tid=0 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.975+0000 7f01c43df740 1 client.8439 shutdown 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 msgr2=0x55d0fa1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1987f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f01a000d590 tx=0x7f01a0012000 comp rx=0 tx=0).stop 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 shutdown_connections 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:6834/432323448,v1:172.21.15.19:6835/432323448] conn(0x55d0fa22e9f0 0x55d0fa1c22e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d0f9dc9270 0x55d0fa198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d0f9da8590 0x55d0fa1987f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f019f400640 0 client.8439 ms_handle_reset on v2:172.21.15.19:6834/432323448 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 --2- 172.21.15.19:0/1090467700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d0f9da03b0 0x55d0fa1930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 >> 172.21.15.19:0/1090467700 conn(0x55d0f9dbd3c0 msgr2=0x55d0fa1bf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 shutdown_connections 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 1 -- 172.21.15.19:0/1090467700 wait complete. 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.976+0000 7f01c43df740 20 client.8439 trim_cache size 0 max 16384 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8346, 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3659525209999996, 2024-04-25T11:36:35.967 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 133, 2024-04-25T11:36:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 98, 2024-04-25T11:36:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 98, 2024-04-25T11:36:35.968 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.29440917 2024-04-25T11:36:35.968 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:36:35.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01940744b0 msgr2=0x7f01940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01940744b0 0x7f01940769c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f01b80b6070 tx=0x7f01a0009040 comp rx=0 tx=0).stop 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01b806d280 msgr2=0x7f01b814f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01b806d280 0x7f01b814f110 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e8a0 tx=0x7f01a800ed60 comp rx=0 tx=0).stop 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 shutdown_connections 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01940744b0 0x7f01940769c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b80b4790 0x7f01b814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01b8168090 0x7f01b8164860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 --2- 172.21.15.19:0/597919715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01b806d280 0x7f01b814f110 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 >> 172.21.15.19:0/597919715 conn(0x7f01b80d1720 msgr2=0x7f01b80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 shutdown_connections 2024-04-25T11:36:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:35.977+0000 7f01bca00640 1 -- 172.21.15.19:0/597919715 wait complete. 2024-04-25T11:36:35.993 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:36:35.993 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2024-04-25T11:36:35.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:36:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.290+0000 7f3b47600640 1 Processor -- start 2024-04-25T11:36:36.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.290+0000 7f3b47600640 1 -- start start 2024-04-25T11:36:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.292+0000 7f3b47600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3b40161760 0x7f3b400b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.292+0000 7f3b47600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 0x7f3b400b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.292+0000 7f3b47600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b400cf4b0 con 0x7f3b400b82d0 2024-04-25T11:36:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.292+0000 7f3b47600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400d00c0 con 0x7f3b40161760 2024-04-25T11:36:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.292+0000 7f3b47600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400cf790 con 0x7f3b400d2200 2024-04-25T11:36:36.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.293+0000 7f3b3fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3b400b82d0 0x7f3b400d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34218/0 (socket says 172.21.15.19:34218) 2024-04-25T11:36:36.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.293+0000 7f3b3fe00640 1 -- 172.21.15.19:0/3210164233 learned_addr learned my addr 172.21.15.19:0/3210164233 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.293+0000 7f3b44c00640 1 --2- 172.21.15.19:0/3210164233 >> v2:172.21.15.169:3300/0 conn(0x7f3b40161760 0x7f3b400b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.293+0000 7f3b45600640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 0x7f3b400b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.293+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3975288382 0 0) 0x7f3b400cf4b0 con 0x7f3b400b82d0 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.293+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b24003520 con 0x7f3b400b82d0 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3151301658 0 0) 0x7f3b24003520 con 0x7f3b400b82d0 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 msgr2=0x7f3b400b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 0x7f3b400b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 >> v2:172.21.15.169:3300/0 conn(0x7f3b40161760 msgr2=0x7f3b400b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 --2- 172.21.15.19:0/3210164233 >> v2:172.21.15.169:3300/0 conn(0x7f3b40161760 0x7f3b400b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b400cfa70 con 0x7f3b400b82d0 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f3b2c001670 con 0x7f3b400b82d0 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 >> v1:172.21.15.19:6789/0 conn(0x7f3b400b82d0 legacy=0x7f3b400d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b240039b0 0x7f3b24003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 0x7f3b24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b24004d40 0x7f3b24005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400d00c0 con 0x7f3b24004d40 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400cf790 con 0x7f3b240039b0 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2400b9c0 con 0x7f3b400d2200 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b45600640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b24004d40 0x7f3b24005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.294+0000 7f3b3fe00640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 0x7f3b24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.295+0000 7f3b45600640 1 -- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 msgr2=0x7f3b24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.295+0000 7f3b45600640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b400d2200 0x7f3b24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.295+0000 7f3b44c00640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b240039b0 0x7f3b24003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.295+0000 7f3b45600640 1 -- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b240039b0 msgr2=0x7f3b24003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.295+0000 7f3b45600640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b240039b0 0x7f3b24003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.295+0000 7f3b45600640 1 -- 172.21.15.19:0/3210164233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b24003520 con 0x7f3b24004d40 2024-04-25T11:36:36.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.295+0000 7f3b44c00640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b240039b0 0x7f3b24003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.296+0000 7f3b45600640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b24004d40 0x7f3b24005150 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009f90 tx=0x7f3b24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56894ed1c7079e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.296+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38009c10 con 0x7f3b24004d40 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.296+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38036070 con 0x7f3b24004d40 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.296+0000 7f3b3f400640 1 -- 172.21.15.19:0/3210164233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38031850 con 0x7f3b24004d40 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.297+0000 7f3b47600640 1 -- 172.21.15.19:0/3210164233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b24004d40 msgr2=0x7f3b24005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.297+0000 7f3b47600640 1 --2- 172.21.15.19:0/3210164233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b24004d40 0x7f3b24005150 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009f90 tx=0x7f3b24005830 comp rx=0 tx=0).stop 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.297+0000 7f3b45600640 1 -- 172.21.15.19:0/3210164233 reap_dead start 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.297+0000 7f3b47600640 1 -- 172.21.15.19:0/3210164233 shutdown_connections 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.297+0000 7f3b47600640 1 -- 172.21.15.19:0/3210164233 >> 172.21.15.19:0/3210164233 conn(0x7f3b4009b7d0 msgr2=0x7f3b4009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.297+0000 7f3b47600640 1 -- 172.21.15.19:0/3210164233 shutdown_connections 2024-04-25T11:36:36.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.297+0000 7f3b47600640 1 -- 172.21.15.19:0/3210164233 wait complete. 2024-04-25T11:36:36.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.300+0000 7f3b47600640 1 Processor -- start 2024-04-25T11:36:36.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.300+0000 7f3b47600640 1 -- start start 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.301+0000 7f3b47600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b24004d40 0x7f3b4015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.301+0000 7f3b47600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b4015adc0 0x7f3b4014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.301+0000 7f3b47600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b40151fa0 0x7f3b401523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b47600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400b7940 con 0x7f3b40151fa0 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b47600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40161b70 con 0x7f3b24004d40 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b47600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400b23e0 con 0x7f3b4015adc0 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b40151fa0 0x7f3b401523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b40151fa0 0x7f3b401523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40586/0 (socket says 172.21.15.19:40586) 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 -- 172.21.15.19:0/661684273 learned_addr learned my addr 172.21.15.19:0/661684273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:36.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b3fe00640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b4015adc0 0x7f3b4014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b44c00640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b24004d40 0x7f3b4015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 -- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b4015adc0 msgr2=0x7f3b4014ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b4015adc0 0x7f3b4014ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 -- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b24004d40 msgr2=0x7f3b4015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b24004d40 0x7f3b4015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.302+0000 7f3b45600640 1 -- 172.21.15.19:0/661684273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b401528f0 con 0x7f3b40151fa0 2024-04-25T11:36:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.303+0000 7f3b45600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b40151fa0 0x7f3b401523b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009f90 tx=0x7f3b38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.303+0000 7f3b3cc00640 1 -- 172.21.15.19:0/661684273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38009c10 con 0x7f3b40151fa0 2024-04-25T11:36:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.303+0000 7f3b3cc00640 1 -- 172.21.15.19:0/661684273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38003990 con 0x7f3b40151fa0 2024-04-25T11:36:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.303+0000 7f3b3cc00640 1 -- 172.21.15.19:0/661684273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38003d80 con 0x7f3b40151fa0 2024-04-25T11:36:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.303+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4015b560 con 0x7f3b40151fa0 2024-04-25T11:36:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.303+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4009e920 con 0x7f3b40151fa0 2024-04-25T11:36:36.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.304+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b40050740 con 0x7f3b40151fa0 2024-04-25T11:36:36.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.306+0000 7f3b3cc00640 1 -- 172.21.15.19:0/661684273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3b38005d10 con 0x7f3b40151fa0 2024-04-25T11:36:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.307+0000 7f3b3cc00640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3b200744b0 0x7f3b200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:36.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.307+0000 7f3b3cc00640 1 -- 172.21.15.19:0/661684273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f3b380b6840 con 0x7f3b40151fa0 2024-04-25T11:36:36.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.307+0000 7f3b44c00640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3b200744b0 0x7f3b200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:36.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.309+0000 7f3b44c00640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3b200744b0 0x7f3b200769c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b20 tx=0x7f3b30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:36.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.312+0000 7f3b3cc00640 1 -- 172.21.15.19:0/661684273 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b401528f0 con 0x7f3b40151fa0 2024-04-25T11:36:36.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.565+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3b400b21c0 con 0x7f3b40151fa0 2024-04-25T11:36:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.566+0000 7f3b3cc00640 1 -- 172.21.15.19:0/661684273 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v133) ==== 53+0+73 (secure 0 0 0) 0x7f3b3802e050 con 0x7f3b40151fa0 2024-04-25T11:36:36.555 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:36:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.569+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3b200744b0 msgr2=0x7f3b200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.569+0000 7f3b47600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3b200744b0 0x7f3b200769c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b20 tx=0x7f3b30008040 comp rx=0 tx=0).stop 2024-04-25T11:36:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.569+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b40151fa0 msgr2=0x7f3b401523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.569+0000 7f3b47600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b40151fa0 0x7f3b401523b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009f90 tx=0x7f3b38009750 comp rx=0 tx=0).stop 2024-04-25T11:36:36.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 shutdown_connections 2024-04-25T11:36:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3b200744b0 0x7f3b200769c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b40151fa0 0x7f3b401523b0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b4015adc0 0x7f3b4014ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 --2- 172.21.15.19:0/661684273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b24004d40 0x7f3b4015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 >> 172.21.15.19:0/661684273 conn(0x7f3b4009b7d0 msgr2=0x7f3b400d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 shutdown_connections 2024-04-25T11:36:36.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36.570+0000 7f3b47600640 1 -- 172.21.15.19:0/661684273 wait complete. 2024-04-25T11:36:36.587 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:36:36.588 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:36:36.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:36.588 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:36:36.642 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:36:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:36:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:36:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:36:36.642 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42077sec preferred_lft 42077sec 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:36:36.643 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:36:36.644 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:36:36.645 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:36.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:36.646 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:36:36.728 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:36:36.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:36.734 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:36:36.786 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:36:36.787 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:36:36.787 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:36:36.787 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:36:36.787 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:36:36.787 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:36:36.787 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:36:36.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:36:36.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:36:36.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:36:36.788 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:36:36.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:36:37.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:36:37.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:37.010 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:36:37.061 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:36:37.061 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:36:37.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:36:37.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:37.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:36:37.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:36:37.143 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:37.194 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:36:37.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:36:37.194 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:37.246 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:36:37.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:37.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:36:37.465 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:36:37.465 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:36:37.511 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:36:37.511 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:36:37.511 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:36:37.511 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:36:37.511 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:36:37.513 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:36:37.513 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:36:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:36:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:36:37.514 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:36:37.519 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:36:37.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:37.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:37.600 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:36:37.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:36:37.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:36:37.720 INFO:teuthology.orchestra.run.smithi019.stdout:client8448 2024-04-25T11:36:37.720 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:37.720 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:37.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:37.777 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:36:37.782 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:36:37.835 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:36:37.835 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:36:37.835 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:37.835 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:36:37.835 INFO:teuthology.orchestra.run.smithi019.stdout:sixmegs 2024-04-25T11:36:37.836 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2024-04-25T11:36:37.836 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2024-04-25T11:36:37.837 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:37.837 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:37.888 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:37.889 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:37.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:36:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.228+0000 7fb57fe00640 1 Processor -- start 2024-04-25T11:36:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.230+0000 7fb57fe00640 1 -- start start 2024-04-25T11:36:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.231+0000 7fb57fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb58006c5c0 0x7fb5800cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.231+0000 7fb57fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb580144370 0x7fb5800b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.231+0000 7fb57fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb580161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.231+0000 7fb57fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb5800b50e0 con 0x7fb580144370 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.231+0000 7fb57fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb5800d1230 con 0x7fb58006c5c0 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.231+0000 7fb57fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800d06b0 con 0x7fb5800b4cd0 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb580144370 0x7fb5800b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb580144370 0x7fb5800b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40602/0 (socket says 172.21.15.19:40602) 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 -- 172.21.15.19:0/2201716649 learned_addr learned my addr 172.21.15.19:0/2201716649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb57ea00640 1 --2- 172.21.15.19:0/2201716649 >> v2:172.21.15.169:3300/0 conn(0x7fb58006c5c0 0x7fb5800cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb57f400640 1 --2- 172.21.15.19:0/2201716649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb580161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 -- 172.21.15.19:0/2201716649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 msgr2=0x7fb580161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 --2- 172.21.15.19:0/2201716649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb580161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 -- 172.21.15.19:0/2201716649 >> v2:172.21.15.169:3300/0 conn(0x7fb58006c5c0 msgr2=0x7fb5800cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 --2- 172.21.15.19:0/2201716649 >> v2:172.21.15.169:3300/0 conn(0x7fb58006c5c0 0x7fb5800cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.232+0000 7fb577e00640 1 -- 172.21.15.19:0/2201716649 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5800d0990 con 0x7fb580144370 2024-04-25T11:36:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.234+0000 7fb577e00640 1 --2- 172.21.15.19:0/2201716649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb580144370 0x7fb5800b4790 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb5800726b0 tx=0x7fb56802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905fa2e42c02218a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.234+0000 7fb57e000640 1 -- 172.21.15.19:0/2201716649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568037070 con 0x7fb580144370 2024-04-25T11:36:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.234+0000 7fb57e000640 1 -- 172.21.15.19:0/2201716649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb568032440 con 0x7fb580144370 2024-04-25T11:36:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.234+0000 7fb57e000640 1 -- 172.21.15.19:0/2201716649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568032810 con 0x7fb580144370 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 -- 172.21.15.19:0/2201716649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb580144370 msgr2=0x7fb5800b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 --2- 172.21.15.19:0/2201716649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb580144370 0x7fb5800b4790 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb5800726b0 tx=0x7fb56802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 -- 172.21.15.19:0/2201716649 shutdown_connections 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 --2- 172.21.15.19:0/2201716649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb580144370 0x7fb5800b4790 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 --2- 172.21.15.19:0/2201716649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb580161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 --2- 172.21.15.19:0/2201716649 >> v2:172.21.15.169:3300/0 conn(0x7fb58006c5c0 0x7fb5800cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 -- 172.21.15.19:0/2201716649 >> 172.21.15.19:0/2201716649 conn(0x7fb5800d1720 msgr2=0x7fb5800d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 -- 172.21.15.19:0/2201716649 shutdown_connections 2024-04-25T11:36:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.235+0000 7fb57fe00640 1 -- 172.21.15.19:0/2201716649 wait complete. 2024-04-25T11:36:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.238+0000 7fb57fe00640 1 Processor -- start 2024-04-25T11:36:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.238+0000 7fb57fe00640 1 -- start start 2024-04-25T11:36:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb58006c5c0 0x7fb5800c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb5800c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb580144370 0x7fb580161920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800d02d0 con 0x7fb58006c5c0 2024-04-25T11:36:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb5800d0f50 con 0x7fb580144370 2024-04-25T11:36:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb5800d0450 con 0x7fb5800b4cd0 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb58006c5c0 0x7fb5800c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb58006c5c0 0x7fb5800c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40616/0 (socket says 172.21.15.19:40616) 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 -- 172.21.15.19:0/3566061159 learned_addr learned my addr 172.21.15.19:0/3566061159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb577e00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb5800c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57f400640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb580144370 0x7fb580161920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 -- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 msgr2=0x7fb5800c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb5800c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 -- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb580144370 msgr2=0x7fb580161920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb580144370 0x7fb580161920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 -- 172.21.15.19:0/3566061159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb580160880 con 0x7fb58006c5c0 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57ea00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb58006c5c0 0x7fb5800c6700 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb57000d620 tx=0x7fb57000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb577400640 1 -- 172.21.15.19:0/3566061159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000b620 con 0x7fb58006c5c0 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb577400640 1 -- 172.21.15.19:0/3566061159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570019070 con 0x7fb58006c5c0 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb577400640 1 -- 172.21.15.19:0/3566061159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000ba00 con 0x7fb58006c5c0 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.241+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb580160b60 con 0x7fb58006c5c0 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.242+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb580161120 con 0x7fb58006c5c0 2024-04-25T11:36:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.243+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb580050740 con 0x7fb58006c5c0 2024-04-25T11:36:38.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.244+0000 7fb577400640 1 -- 172.21.15.19:0/3566061159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb57000bba0 con 0x7fb58006c5c0 2024-04-25T11:36:38.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.245+0000 7fb577400640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb55c0744e0 0x7fb55c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.245+0000 7fb577400640 1 -- 172.21.15.19:0/3566061159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fb570095820 con 0x7fb58006c5c0 2024-04-25T11:36:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.246+0000 7fb577e00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb55c0744e0 0x7fb55c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.247+0000 7fb577e00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb55c0744e0 0x7fb55c0769f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb568034000 tx=0x7fb56802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.250+0000 7fb577400640 1 -- 172.21.15.19:0/3566061159 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb57005fd00 con 0x7fb58006c5c0 2024-04-25T11:36:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.457+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb58006d8c0 con 0x7fb58006c5c0 2024-04-25T11:36:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.461+0000 7fb577400640 1 -- 172.21.15.19:0/3566061159 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+17447 (secure 0 0 0) 0x7fb57005f600 con 0x7fb58006c5c0 2024-04-25T11:36:38.450 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:38.450 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":98,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:36:22.535997+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:36:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb55c0744e0 msgr2=0x7fb55c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb55c0744e0 0x7fb55c0769f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb568034000 tx=0x7fb56802e000 comp rx=0 tx=0).stop 2024-04-25T11:36:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb58006c5c0 msgr2=0x7fb5800c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb58006c5c0 0x7fb5800c6700 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb57000d620 tx=0x7fb57000dae0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 shutdown_connections 2024-04-25T11:36:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb55c0744e0 0x7fb55c0769f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb580144370 0x7fb580161920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5800b4cd0 0x7fb5800c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 --2- 172.21.15.19:0/3566061159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb58006c5c0 0x7fb5800c6700 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.464+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 >> 172.21.15.19:0/3566061159 conn(0x7fb5800d1720 msgr2=0x7fb58006f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.465+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 shutdown_connections 2024-04-25T11:36:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.465+0000 7fb57fe00640 1 -- 172.21.15.19:0/3566061159 wait complete. 2024-04-25T11:36:38.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:36:38.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:36:38.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.785+0000 7f0365600640 1 Processor -- start 2024-04-25T11:36:38.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.786+0000 7f0365600640 1 -- start start 2024-04-25T11:36:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.788+0000 7f0365600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f03600cd9a0 0x7f036015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.788+0000 7f0365600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03600b0af0 0x7f036006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.788+0000 7f0365600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036006db10 0x7f0360161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.788+0000 7f0365600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f03600ceab0 con 0x7f03600b0af0 2024-04-25T11:36:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.788+0000 7f0365600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f036006df20 con 0x7f03600cd9a0 2024-04-25T11:36:38.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.788+0000 7f0365600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03600cdea0 con 0x7f036006db10 2024-04-25T11:36:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.788+0000 7f0357e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03600b0af0 0x7f036006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f0357e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03600b0af0 0x7f036006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40628/0 (socket says 172.21.15.19:40628) 2024-04-25T11:36:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f0357e00640 1 -- 172.21.15.19:0/3012751851 learned_addr learned my addr 172.21.15.19:0/3012751851 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f035f400640 1 --2- 172.21.15.19:0/3012751851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036006db10 0x7f0360161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f0357e00640 1 -- 172.21.15.19:0/3012751851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036006db10 msgr2=0x7f0360161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f035ea00640 1 --2- 172.21.15.19:0/3012751851 >> v2:172.21.15.169:3300/0 conn(0x7f03600cd9a0 0x7f036015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f0357e00640 1 --2- 172.21.15.19:0/3012751851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036006db10 0x7f0360161ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f0357e00640 1 -- 172.21.15.19:0/3012751851 >> v2:172.21.15.169:3300/0 conn(0x7f03600cd9a0 msgr2=0x7f036015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f0357e00640 1 --2- 172.21.15.19:0/3012751851 >> v2:172.21.15.169:3300/0 conn(0x7f03600cd9a0 0x7f036015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f0357e00640 1 -- 172.21.15.19:0/3012751851 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03600ce180 con 0x7f03600b0af0 2024-04-25T11:36:38.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.789+0000 7f035ea00640 1 --2- 172.21.15.19:0/3012751851 >> v2:172.21.15.169:3300/0 conn(0x7f03600cd9a0 0x7f036015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.790+0000 7f0357e00640 1 --2- 172.21.15.19:0/3012751851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03600b0af0 0x7f036006d5d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0360072860 tx=0x7f034802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f71731564b10faf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.790+0000 7f035e000640 1 -- 172.21.15.19:0/3012751851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348037070 con 0x7f03600b0af0 2024-04-25T11:36:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f035e000640 1 -- 172.21.15.19:0/3012751851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0348032440 con 0x7f03600b0af0 2024-04-25T11:36:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f035e000640 1 -- 172.21.15.19:0/3012751851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348032810 con 0x7f03600b0af0 2024-04-25T11:36:38.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f0365600640 1 -- 172.21.15.19:0/3012751851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03600b0af0 msgr2=0x7f036006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f0365600640 1 --2- 172.21.15.19:0/3012751851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03600b0af0 0x7f036006d5d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0360072860 tx=0x7f034802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f0365600640 1 -- 172.21.15.19:0/3012751851 shutdown_connections 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f0365600640 1 --2- 172.21.15.19:0/3012751851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03600b0af0 0x7f036006d5d0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f0365600640 1 --2- 172.21.15.19:0/3012751851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f036006db10 0x7f0360161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f0365600640 1 --2- 172.21.15.19:0/3012751851 >> v2:172.21.15.169:3300/0 conn(0x7f03600cd9a0 0x7f036015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.791+0000 7f0365600640 1 -- 172.21.15.19:0/3012751851 >> 172.21.15.19:0/3012751851 conn(0x7f0360073090 msgr2=0x7f03600734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.792+0000 7f0365600640 1 -- 172.21.15.19:0/3012751851 shutdown_connections 2024-04-25T11:36:38.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.792+0000 7f0365600640 1 -- 172.21.15.19:0/3012751851 wait complete. 2024-04-25T11:36:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.795+0000 7f0365600640 1 Processor -- start 2024-04-25T11:36:38.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.795+0000 7f0365600640 1 -- start start 2024-04-25T11:36:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.795+0000 7f0365600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036006db10 0x7f03601453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f0365600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03600b0af0 0x7f0360158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f0365600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03600cd9a0 0x7f0360138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f0365600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03600b0f00 con 0x7f036006db10 2024-04-25T11:36:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f0365600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03600ce740 con 0x7f03600cd9a0 2024-04-25T11:36:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f0365600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03600cef40 con 0x7f03600b0af0 2024-04-25T11:36:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036006db10 0x7f03601453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036006db10 0x7f03601453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40642/0 (socket says 172.21.15.19:40642) 2024-04-25T11:36:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 -- 172.21.15.19:0/1100552123 learned_addr learned my addr 172.21.15.19:0/1100552123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035f400640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03600cd9a0 0x7f0360138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f0357e00640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03600b0af0 0x7f0360158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 -- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03600b0af0 msgr2=0x7f0360158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03600b0af0 0x7f0360158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 -- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03600cd9a0 msgr2=0x7f0360138fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03600cd9a0 0x7f0360138fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.796+0000 7f035ea00640 1 -- 172.21.15.19:0/1100552123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0360156ff0 con 0x7f036006db10 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.797+0000 7f035ea00640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036006db10 0x7f03601453c0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f035000c7f0 tx=0x7f035000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.797+0000 7f0356a00640 1 -- 172.21.15.19:0/1100552123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350015070 con 0x7f036006db10 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.797+0000 7f0356a00640 1 -- 172.21.15.19:0/1100552123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0350007820 con 0x7f036006db10 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.797+0000 7f0356a00640 1 -- 172.21.15.19:0/1100552123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350007bd0 con 0x7f036006db10 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.797+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03601572d0 con 0x7f036006db10 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.797+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0360157890 con 0x7f036006db10 2024-04-25T11:36:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.798+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0360050740 con 0x7f036006db10 2024-04-25T11:36:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.800+0000 7f0356a00640 1 -- 172.21.15.19:0/1100552123 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0350018070 con 0x7f036006db10 2024-04-25T11:36:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.801+0000 7f0356a00640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.801+0000 7f0356a00640 1 -- 172.21.15.19:0/1100552123 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7f03500955c0 con 0x7f036006db10 2024-04-25T11:36:38.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.802+0000 7f0357e00640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:38.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.803+0000 7f0357e00640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f033c0744e0 0x7f033c0769f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0348034000 tx=0x7f034802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:38.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:38.806+0000 7f0356a00640 1 -- 172.21.15.19:0/1100552123 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f035005faa0 con 0x7f036006db10 2024-04-25T11:36:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.009+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03600b63d0 con 0x7f036006db10 2024-04-25T11:36:38.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.011+0000 7f0356a00640 1 -- 172.21.15.19:0/1100552123 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f035005f3a0 con 0x7f036006db10 2024-04-25T11:36:38.999 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.013+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f033c0744e0 msgr2=0x7f033c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f033c0744e0 0x7f033c0769f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0348034000 tx=0x7f034802e000 comp rx=0 tx=0).stop 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036006db10 msgr2=0x7f03601453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036006db10 0x7f03601453c0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f035000c7f0 tx=0x7f035000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 shutdown_connections 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03600cd9a0 0x7f0360138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03600b0af0 0x7f0360158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 --2- 172.21.15.19:0/1100552123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036006db10 0x7f03601453c0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 >> 172.21.15.19:0/1100552123 conn(0x7f0360073090 msgr2=0x7f036006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 shutdown_connections 2024-04-25T11:36:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.014+0000 7f0365600640 1 -- 172.21.15.19:0/1100552123 wait complete. 2024-04-25T11:36:39.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:36:39.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.331+0000 7fe4b0a00640 1 Processor -- start 2024-04-25T11:36:39.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.332+0000 7fe4b0a00640 1 -- start start 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4b0a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4ac15ee60 0x7fe4ac168e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4b0a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ac070000 con 0x7fe4ac144340 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4b0a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ac06f240 con 0x7fe4ac06dce0 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4b0a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe4ac15f270 con 0x7fe4ac15ee60 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4ab200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4ac15ee60 0x7fe4ac168e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4ab200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4ac15ee60 0x7fe4ac168e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40652/0 (socket says 172.21.15.19:40652) 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4aa800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe4ac144340 0x7fe4ac06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34228/0 (socket says 172.21.15.19:34228) 2024-04-25T11:36:39.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.334+0000 7fe4ab200640 1 -- 172.21.15.19:0/839134263 learned_addr learned my addr 172.21.15.19:0/839134263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:39.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.335+0000 7fe4ab200640 1 -- 172.21.15.19:0/839134263 >> v1:172.21.15.169:6789/0 conn(0x7fe4ac06dce0 legacy=0x7fe4ac15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.335+0000 7fe4ab200640 1 -- 172.21.15.19:0/839134263 >> v1:172.21.15.19:6789/0 conn(0x7fe4ac144340 legacy=0x7fe4ac06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.335+0000 7fe4ab200640 1 -- 172.21.15.19:0/839134263 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac06f520 con 0x7fe4ac15ee60 2024-04-25T11:36:39.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.336+0000 7fe4ab200640 1 --2- 172.21.15.19:0/839134263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac15ee60 0x7fe4ac168e30 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009870 tx=0x7fe4a402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678cd664006cd75e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:39.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.336+0000 7fe4a9400640 1 -- 172.21.15.19:0/839134263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4037070 con 0x7fe4ac15ee60 2024-04-25T11:36:39.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.336+0000 7fe4a9400640 1 -- 172.21.15.19:0/839134263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a402ae50 con 0x7fe4ac15ee60 2024-04-25T11:36:39.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.336+0000 7fe4a9400640 1 -- 172.21.15.19:0/839134263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4032860 con 0x7fe4ac15ee60 2024-04-25T11:36:39.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.337+0000 7fe4b0a00640 1 -- 172.21.15.19:0/839134263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac15ee60 msgr2=0x7fe4ac168e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.337+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/839134263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac15ee60 0x7fe4ac168e30 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009870 tx=0x7fe4a402a300 comp rx=0 tx=0).stop 2024-04-25T11:36:39.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.337+0000 7fe4b0a00640 1 -- 172.21.15.19:0/839134263 shutdown_connections 2024-04-25T11:36:39.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.337+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/839134263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac15ee60 0x7fe4ac168e30 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.337+0000 7fe4b0a00640 1 -- 172.21.15.19:0/839134263 >> 172.21.15.19:0/839134263 conn(0x7fe4ac072f40 msgr2=0x7fe4ac073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:39.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.337+0000 7fe4b0a00640 1 -- 172.21.15.19:0/839134263 shutdown_connections 2024-04-25T11:36:39.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.337+0000 7fe4b0a00640 1 -- 172.21.15.19:0/839134263 wait complete. 2024-04-25T11:36:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.340+0000 7fe4b0a00640 1 Processor -- start 2024-04-25T11:36:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.340+0000 7fe4b0a00640 1 -- start start 2024-04-25T11:36:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.341+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac144340 0x7fe4ac158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.341+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ac15ee60 0x7fe4ac0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.341+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ac142e90 0x7fe4ac1432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.341+0000 7fe4b0a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac06fae0 con 0x7fe4ac144340 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.341+0000 7fe4b0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac144750 con 0x7fe4ac142e90 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.341+0000 7fe4b0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe4ac1604b0 con 0x7fe4ac15ee60 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.341+0000 7fe4aa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac144340 0x7fe4ac158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac144340 0x7fe4ac158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40660/0 (socket says 172.21.15.19:40660) 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 -- 172.21.15.19:0/3013781629 learned_addr learned my addr 172.21.15.19:0/3013781629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 -- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ac15ee60 msgr2=0x7fe4ac0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ac15ee60 0x7fe4ac0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 -- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ac142e90 msgr2=0x7fe4ac1432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ac142e90 0x7fe4ac1432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 -- 172.21.15.19:0/3013781629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac1437e0 con 0x7fe4ac144340 2024-04-25T11:36:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.342+0000 7fe4aa800640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac144340 0x7fe4ac158130 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c730 tx=0x7fe49c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.343+0000 7fe4a2a00640 1 -- 172.21.15.19:0/3013781629 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c017070 con 0x7fe4ac144340 2024-04-25T11:36:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.343+0000 7fe4a2a00640 1 -- 172.21.15.19:0/3013781629 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49c00ce40 con 0x7fe4ac144340 2024-04-25T11:36:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.343+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac1f9bd0 con 0x7fe4ac144340 2024-04-25T11:36:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.343+0000 7fe4a2a00640 1 -- 172.21.15.19:0/3013781629 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00a860 con 0x7fe4ac144340 2024-04-25T11:36:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.343+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac1fa190 con 0x7fe4ac144340 2024-04-25T11:36:39.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.345+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ac076c60 con 0x7fe4ac144340 2024-04-25T11:36:39.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.345+0000 7fe4a2a00640 1 -- 172.21.15.19:0/3013781629 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe49c01a070 con 0x7fe4ac144340 2024-04-25T11:36:39.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.346+0000 7fe4a2a00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe48c0744e0 0x7fe48c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.346+0000 7fe4a2a00640 1 -- 172.21.15.19:0/3013781629 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fe49c0949e0 con 0x7fe4ac144340 2024-04-25T11:36:39.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.348+0000 7fe4a9e00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe48c0744e0 0x7fe48c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.349+0000 7fe4a9e00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe48c0744e0 0x7fe48c0769f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe498000f60 tx=0x7fe4980079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:39.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.352+0000 7fe4a2a00640 1 -- 172.21.15.19:0/3013781629 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe49c05efe0 con 0x7fe4ac144340 2024-04-25T11:36:39.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.553+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe4ac0b7920 con 0x7fe4ac144340 2024-04-25T11:36:39.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.554+0000 7fe4a2a00640 1 -- 172.21.15.19:0/3013781629 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fe49c063e20 con 0x7fe4ac144340 2024-04-25T11:36:39.543 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe48c0744e0 msgr2=0x7fe48c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe48c0744e0 0x7fe48c0769f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe498000f60 tx=0x7fe4980079d0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac144340 msgr2=0x7fe4ac158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac144340 0x7fe4ac158130 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c730 tx=0x7fe49c00cbf0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 shutdown_connections 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe48c0744e0 0x7fe48c0769f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4ac142e90 0x7fe4ac1432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4ac15ee60 0x7fe4ac0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 --2- 172.21.15.19:0/3013781629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4ac144340 0x7fe4ac158130 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 >> 172.21.15.19:0/3013781629 conn(0x7fe4ac072f40 msgr2=0x7fe4ac160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 shutdown_connections 2024-04-25T11:36:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.557+0000 7fe4b0a00640 1 -- 172.21.15.19:0/3013781629 wait complete. 2024-04-25T11:36:39.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:36:39.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.865+0000 7fccfde00640 1 Processor -- start 2024-04-25T11:36:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.865+0000 7fccfde00640 1 -- start start 2024-04-25T11:36:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.867+0000 7fccfde00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fccf8160260 0x7fccf80b7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.867+0000 7fccfde00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf80b7e70 con 0x7fccf80b7a60 2024-04-25T11:36:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.867+0000 7fccfde00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccf806d260 con 0x7fccf806ccd0 2024-04-25T11:36:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.867+0000 7fccfde00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fccf80b9a70 con 0x7fccf8160260 2024-04-25T11:36:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.868+0000 7fccf7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fccf80b7a60 0x7fccf806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34232/0 (socket says 172.21.15.19:34232) 2024-04-25T11:36:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.868+0000 7fccf7e00640 1 -- 172.21.15.19:0/4019879333 learned_addr learned my addr 172.21.15.19:0/4019879333 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:39.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.868+0000 7fccfca00640 1 --2- 172.21.15.19:0/4019879333 >> v2:172.21.15.169:3300/0 conn(0x7fccf8160260 0x7fccf80b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.868+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1522789651 0 0) 0x7fcce0001110 con 0x7fccf80b7a60 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.868+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccdc003520 con 0x7fccf80b7a60 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4127410545 0 0) 0x7fccdc003520 con 0x7fccf80b7a60 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 >> v2:172.21.15.169:3300/0 conn(0x7fccf8160260 msgr2=0x7fccf80b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 --2- 172.21.15.19:0/4019879333 >> v2:172.21.15.169:3300/0 conn(0x7fccf8160260 0x7fccf80b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 >> v1:172.21.15.169:6789/0 conn(0x7fccf806ccd0 legacy=0x7fccf80b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf80b9420 con 0x7fccf80b7a60 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fcce0001840 con 0x7fccf80b7a60 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 >> v1:172.21.15.19:6789/0 conn(0x7fccf80b7a60 legacy=0x7fccf806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc003a00 0x7fccdc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc004800 0x7fccdc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccdc0052a0 0x7fccdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccf80b9a70 con 0x7fccdc0052a0 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc00c230 con 0x7fccdc004800 2024-04-25T11:36:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fccdc00c430 con 0x7fccdc003a00 2024-04-25T11:36:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.869+0000 7fccfd400640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccdc0052a0 0x7fccdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.870+0000 7fccfca00640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc003a00 0x7fccdc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.870+0000 7fccfd400640 1 -- 172.21.15.19:0/4019879333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc003a00 msgr2=0x7fccdc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.870+0000 7fccfd400640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccdc003a00 0x7fccdc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.870+0000 7fccfd400640 1 -- 172.21.15.19:0/4019879333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc004800 msgr2=0x7fccdc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.870+0000 7fccfd400640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc004800 0x7fccdc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.870+0000 7fccfd400640 1 -- 172.21.15.19:0/4019879333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc003520 con 0x7fccdc0052a0 2024-04-25T11:36:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.871+0000 7fccfd400640 1 -- 172.21.15.19:0/4019879333 reap_dead start 2024-04-25T11:36:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.871+0000 7fccfd400640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccdc0052a0 0x7fccdc00b9c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fccf80d00a0 tx=0x7fccdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ead67517a230b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.871+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0001370 con 0x7fccdc0052a0 2024-04-25T11:36:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.871+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf00052d0 con 0x7fccdc0052a0 2024-04-25T11:36:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.871+0000 7fccf7400640 1 -- 172.21.15.19:0/4019879333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf002ab90 con 0x7fccdc0052a0 2024-04-25T11:36:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.872+0000 7fccfde00640 1 -- 172.21.15.19:0/4019879333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccdc0052a0 msgr2=0x7fccdc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.872+0000 7fccfde00640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccdc0052a0 0x7fccdc00b9c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fccf80d00a0 tx=0x7fccdc005830 comp rx=0 tx=0).stop 2024-04-25T11:36:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.872+0000 7fccfde00640 1 -- 172.21.15.19:0/4019879333 shutdown_connections 2024-04-25T11:36:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.872+0000 7fccfde00640 1 --2- 172.21.15.19:0/4019879333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccdc0052a0 0x7fccdc00b9c0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.872+0000 7fccfde00640 1 -- 172.21.15.19:0/4019879333 >> 172.21.15.19:0/4019879333 conn(0x7fccf80d2520 msgr2=0x7fccf80d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.872+0000 7fccfde00640 1 -- 172.21.15.19:0/4019879333 shutdown_connections 2024-04-25T11:36:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.872+0000 7fccfde00640 1 -- 172.21.15.19:0/4019879333 wait complete. 2024-04-25T11:36:39.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.874+0000 7fccfde00640 1 Processor -- start 2024-04-25T11:36:39.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.874+0000 7fccfde00640 1 -- start start 2024-04-25T11:36:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccfde00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc0052a0 0x7fccf81523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccfde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccf81f6530 0x7fccf8162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccfde00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccf8152cf0 0x7fccf81630f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccfde00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccf80b7520 con 0x7fccf81f6530 2024-04-25T11:36:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccfde00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccf80b90f0 con 0x7fccdc0052a0 2024-04-25T11:36:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccfde00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fccf80b76a0 con 0x7fccf8152cf0 2024-04-25T11:36:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccf81f6530 0x7fccf8162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccf81f6530 0x7fccf8162bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40684/0 (socket says 172.21.15.19:40684) 2024-04-25T11:36:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 -- 172.21.15.19:0/3284267863 learned_addr learned my addr 172.21.15.19:0/3284267863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 -- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccf8152cf0 msgr2=0x7fccf81630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccf8152cf0 0x7fccf81630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 -- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc0052a0 msgr2=0x7fccf81523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc0052a0 0x7fccf81523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 -- 172.21.15.19:0/3284267863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf8157790 con 0x7fccf81f6530 2024-04-25T11:36:39.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf7e00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccf81f6530 0x7fccf8162bb0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c730 tx=0x7fcce000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:39.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf4c00640 1 -- 172.21.15.19:0/3284267863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0018070 con 0x7fccf81f6530 2024-04-25T11:36:39.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf4c00640 1 -- 172.21.15.19:0/3284267863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce00072d0 con 0x7fccf81f6530 2024-04-25T11:36:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf8157a70 con 0x7fccf81f6530 2024-04-25T11:36:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.877+0000 7fccf4c00640 1 -- 172.21.15.19:0/3284267863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce00047e0 con 0x7fccf81f6530 2024-04-25T11:36:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.878+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf8158030 con 0x7fccf81f6530 2024-04-25T11:36:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.879+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf8157500 con 0x7fccf81f6530 2024-04-25T11:36:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.880+0000 7fccf4c00640 1 -- 172.21.15.19:0/3284267863 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcce001b070 con 0x7fccf81f6530 2024-04-25T11:36:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.881+0000 7fccf4c00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccd40744b0 0x7fccd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.881+0000 7fccf4c00640 1 -- 172.21.15.19:0/3284267863 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fcce0094a40 con 0x7fccf81f6530 2024-04-25T11:36:39.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.882+0000 7fccfca00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccd40744b0 0x7fccd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.883+0000 7fccfca00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccd40744b0 0x7fccd40769c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009fa0 tx=0x7fcce8009750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:39.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:39.887+0000 7fccf4c00640 1 -- 172.21.15.19:0/3284267863 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fccf80b7e70 con 0x7fccf81f6530 2024-04-25T11:36:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.078+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccf8076c60 con 0x7fccf81f6530 2024-04-25T11:36:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.079+0000 7fccf4c00640 1 -- 172.21.15.19:0/3284267863 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7fcce005efb0 con 0x7fccf81f6530 2024-04-25T11:36:40.067 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccd40744b0 msgr2=0x7fccd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccd40744b0 0x7fccd40769c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009fa0 tx=0x7fcce8009750 comp rx=0 tx=0).stop 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccf81f6530 msgr2=0x7fccf8162bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccf81f6530 0x7fccf8162bb0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fcce000c730 tx=0x7fcce000cbf0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 shutdown_connections 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccd40744b0 0x7fccd40769c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccf8152cf0 0x7fccf81630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccf81f6530 0x7fccf8162bb0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 --2- 172.21.15.19:0/3284267863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccdc0052a0 0x7fccf81523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 >> 172.21.15.19:0/3284267863 conn(0x7fccf80d2520 msgr2=0x7fccf815d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.081+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 shutdown_connections 2024-04-25T11:36:40.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.082+0000 7fccfde00640 1 -- 172.21.15.19:0/3284267863 wait complete. 2024-04-25T11:36:40.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:36:40.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.392+0000 7fa5bfe00640 1 Processor -- start 2024-04-25T11:36:40.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.392+0000 7fa5bfe00640 1 -- start start 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.393+0000 7fa5bfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5c00b6e10 0x7fa5c015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.393+0000 7fa5bfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa5c01608d0 0x7fa5c0160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.393+0000 7fa5bfe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c0161220 con 0x7fa5c00cf2f0 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.393+0000 7fa5bfe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c006f870 con 0x7fa5c00b6e10 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.393+0000 7fa5bfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c00b7220 con 0x7fa5c01608d0 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.394+0000 7fa5b6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5c00b6e10 0x7fa5c015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.394+0000 7fa5b6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5c00b6e10 0x7fa5c015a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40700/0 (socket says 172.21.15.19:40700) 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.394+0000 7fa5b6000640 1 -- 172.21.15.19:0/3221860346 learned_addr learned my addr 172.21.15.19:0/3221860346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.394+0000 7fa5b6000640 1 -- 172.21.15.19:0/3221860346 >> v2:172.21.15.169:3300/0 conn(0x7fa5c01608d0 msgr2=0x7fa5c0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:40.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.394+0000 7fa5b6000640 1 --2- 172.21.15.19:0/3221860346 >> v2:172.21.15.169:3300/0 conn(0x7fa5c01608d0 0x7fa5c0160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.394+0000 7fa5b6000640 1 -- 172.21.15.19:0/3221860346 >> v1:172.21.15.169:6789/0 conn(0x7fa5c00cf2f0 legacy=0x7fa5c00b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.394+0000 7fa5b6000640 1 -- 172.21.15.19:0/3221860346 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c00b74d0 con 0x7fa5c00b6e10 2024-04-25T11:36:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.395+0000 7fa5b6000640 1 --2- 172.21.15.19:0/3221860346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c015a810 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00d27a0 tx=0x7fa5a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ba3708f000f266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.396+0000 7fa5be000640 1 -- 172.21.15.19:0/3221860346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8037070 con 0x7fa5c00b6e10 2024-04-25T11:36:40.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.396+0000 7fa5be000640 1 -- 172.21.15.19:0/3221860346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a8032440 con 0x7fa5c00b6e10 2024-04-25T11:36:40.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.396+0000 7fa5be000640 1 -- 172.21.15.19:0/3221860346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8032810 con 0x7fa5c00b6e10 2024-04-25T11:36:40.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.396+0000 7fa5bfe00640 1 -- 172.21.15.19:0/3221860346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 msgr2=0x7fa5c015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:40.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.396+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/3221860346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c015a810 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00d27a0 tx=0x7fa5a802a230 comp rx=0 tx=0).stop 2024-04-25T11:36:40.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.397+0000 7fa5bfe00640 1 -- 172.21.15.19:0/3221860346 shutdown_connections 2024-04-25T11:36:40.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.397+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/3221860346 >> v2:172.21.15.169:3300/0 conn(0x7fa5c01608d0 0x7fa5c0160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.397+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/3221860346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c015a810 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.397+0000 7fa5bfe00640 1 -- 172.21.15.19:0/3221860346 >> 172.21.15.19:0/3221860346 conn(0x7fa5c00d1720 msgr2=0x7fa5c00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:40.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.397+0000 7fa5bfe00640 1 -- 172.21.15.19:0/3221860346 shutdown_connections 2024-04-25T11:36:40.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.397+0000 7fa5bfe00640 1 -- 172.21.15.19:0/3221860346 wait complete. 2024-04-25T11:36:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.399+0000 7fa5bfe00640 1 Processor -- start 2024-04-25T11:36:40.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.399+0000 7fa5bfe00640 1 -- start start 2024-04-25T11:36:40.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c01591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c00cf2f0 0x7fa5c015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5c01608d0 0x7fa5c00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bfe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c006f500 con 0x7fa5c00b6e10 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c006f220 con 0x7fa5c00cf2f0 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c006fd00 con 0x7fa5c01608d0 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c01591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c01591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40714/0 (socket says 172.21.15.19:40714) 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.401+0000 7fa5bea00640 1 -- 172.21.15.19:0/2782109281 learned_addr learned my addr 172.21.15.19:0/2782109281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5b6000640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c00cf2f0 0x7fa5c015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5bea00640 1 -- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5c01608d0 msgr2=0x7fa5c00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5bf400640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5c01608d0 0x7fa5c00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5bea00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5c01608d0 0x7fa5c00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5bea00640 1 -- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c00cf2f0 msgr2=0x7fa5c015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5bea00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c00cf2f0 0x7fa5c015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5bea00640 1 -- 172.21.15.19:0/2782109281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c00ae850 con 0x7fa5c00b6e10 2024-04-25T11:36:40.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.402+0000 7fa5bea00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c01591c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000b3f0 tx=0x7fa5b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:40.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.403+0000 7fa5b7400640 1 -- 172.21.15.19:0/2782109281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0015020 con 0x7fa5c00b6e10 2024-04-25T11:36:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.403+0000 7fa5b7400640 1 -- 172.21.15.19:0/2782109281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b0011830 con 0x7fa5c00b6e10 2024-04-25T11:36:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.403+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c00aeb30 con 0x7fa5c00b6e10 2024-04-25T11:36:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.403+0000 7fa5b7400640 1 -- 172.21.15.19:0/2782109281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0011c60 con 0x7fa5c00b6e10 2024-04-25T11:36:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.403+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c00af0f0 con 0x7fa5c00b6e10 2024-04-25T11:36:40.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.404+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c0050740 con 0x7fa5c00b6e10 2024-04-25T11:36:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.405+0000 7fa5b7400640 1 -- 172.21.15.19:0/2782109281 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa5b0018020 con 0x7fa5c00b6e10 2024-04-25T11:36:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.406+0000 7fa5b7400640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa59c0744e0 0x7fa59c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.407+0000 7fa5b7400640 1 -- 172.21.15.19:0/2782109281 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fa5b0095370 con 0x7fa5c00b6e10 2024-04-25T11:36:40.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.407+0000 7fa5b6000640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa59c0744e0 0x7fa59c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:40.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.408+0000 7fa5b6000640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa59c0744e0 0x7fa59c0769f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8034000 tx=0x7fa5a802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:40.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.411+0000 7fa5b7400640 1 -- 172.21.15.19:0/2782109281 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5b001f020 con 0x7fa5c00b6e10 2024-04-25T11:36:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.624+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5c00cd890 con 0x7fa5c00b6e10 2024-04-25T11:36:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.626+0000 7fa5b7400640 1 -- 172.21.15.19:0/2782109281 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v98) ==== 81+0+728 (secure 0 0 0) 0x7fa5b005f8e0 con 0x7fa5c00b6e10 2024-04-25T11:36:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:36:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:36:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:36:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:36:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:36:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:36:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:listed 12 entries 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa59c0744e0 msgr2=0x7fa59c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa59c0744e0 0x7fa59c0769f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8034000 tx=0x7fa5a802e000 comp rx=0 tx=0).stop 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 msgr2=0x7fa5c01591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c01591c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000b3f0 tx=0x7fa5b000b8b0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 shutdown_connections 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa59c0744e0 0x7fa59c0769f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5c00cf2f0 0x7fa5c015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5c01608d0 0x7fa5c00cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 --2- 172.21.15.19:0/2782109281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5c00b6e10 0x7fa5c01591c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 >> 172.21.15.19:0/2782109281 conn(0x7fa5c00d1720 msgr2=0x7fa5c00b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 shutdown_connections 2024-04-25T11:36:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:40.628+0000 7fa5bfe00640 1 -- 172.21.15.19:0/2782109281 wait complete. 2024-04-25T11:36:40.646 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:36:40.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:40.646 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:36:40.748 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:36:40.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:40.748 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:36:40.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:36:41.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.080+0000 7fd5b5400640 1 Processor -- start 2024-04-25T11:36:41.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.080+0000 7fd5b5400640 1 -- start start 2024-04-25T11:36:41.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.082+0000 7fd5b5400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5b006d560 0x7fd5b015c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.083+0000 7fd5b5400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b0071230 con 0x7fd5b006f180 2024-04-25T11:36:41.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.083+0000 7fd5b5400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b006f8f0 con 0x7fd5b0070e20 2024-04-25T11:36:41.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.083+0000 7fd5b5400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b006fbd0 con 0x7fd5b006d560 2024-04-25T11:36:41.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.083+0000 7fd5b4a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd5b006f180 0x7fd5b00b5e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34236/0 (socket says 172.21.15.19:34236) 2024-04-25T11:36:41.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.083+0000 7fd5b4a00640 1 -- 172.21.15.19:0/3346717061 learned_addr learned my addr 172.21.15.19:0/3346717061 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.083+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094231794 0 0) 0x7fd5b0071230 con 0x7fd5b006f180 2024-04-25T11:36:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd594003440 con 0x7fd5b006f180 2024-04-25T11:36:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5a7e00640 1 --2- 172.21.15.19:0/3346717061 >> v2:172.21.15.169:3300/0 conn(0x7fd5b006d560 0x7fd5b015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2061329416 0 0) 0x7fd594003440 con 0x7fd5b006f180 2024-04-25T11:36:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 >> v2:172.21.15.169:3300/0 conn(0x7fd5b006d560 msgr2=0x7fd5b015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:41.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 --2- 172.21.15.19:0/3346717061 >> v2:172.21.15.169:3300/0 conn(0x7fd5b006d560 0x7fd5b015c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 >> v1:172.21.15.169:6789/0 conn(0x7fd5b0070e20 legacy=0x7fd5b006ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b006feb0 con 0x7fd5b006f180 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fd5a80019f0 con 0x7fd5b006f180 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 >> v1:172.21.15.19:6789/0 conn(0x7fd5b006f180 legacy=0x7fd5b00b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594003aa0 0x7fd594003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd594008950 0x7fd59400ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd59400b610 0x7fd59400db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b006fbd0 con 0x7fd594003aa0 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd59400e400 con 0x7fd594008950 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.084+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd59400e600 con 0x7fd59400b610 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5afe00640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594003aa0 0x7fd594003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5a7e00640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd594008950 0x7fd59400ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5afe00640 1 -- 172.21.15.19:0/3346717061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd59400b610 msgr2=0x7fd59400db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5afe00640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd59400b610 0x7fd59400db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5afe00640 1 -- 172.21.15.19:0/3346717061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd594008950 msgr2=0x7fd59400ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5afe00640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd594008950 0x7fd59400ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5afe00640 1 -- 172.21.15.19:0/3346717061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd594003440 con 0x7fd594003aa0 2024-04-25T11:36:41.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.085+0000 7fd5b4a00640 1 -- 172.21.15.19:0/3346717061 reap_dead start 2024-04-25T11:36:41.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.086+0000 7fd5afe00640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594003aa0 0x7fd594003f10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0001750 tx=0x7fd594003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8485f2eada40f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:41.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.086+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0001370 con 0x7fd594003aa0 2024-04-25T11:36:41.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.086+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a00052d0 con 0x7fd594003aa0 2024-04-25T11:36:41.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.086+0000 7fd5af400640 1 -- 172.21.15.19:0/3346717061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a002abe0 con 0x7fd594003aa0 2024-04-25T11:36:41.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.087+0000 7fd5b5400640 1 -- 172.21.15.19:0/3346717061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594003aa0 msgr2=0x7fd594003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.087+0000 7fd5b5400640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594003aa0 0x7fd594003f10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0001750 tx=0x7fd594003730 comp rx=0 tx=0).stop 2024-04-25T11:36:41.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.087+0000 7fd5b5400640 1 -- 172.21.15.19:0/3346717061 shutdown_connections 2024-04-25T11:36:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.087+0000 7fd5b5400640 1 --2- 172.21.15.19:0/3346717061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd594003aa0 0x7fd594003f10 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.087+0000 7fd5b5400640 1 -- 172.21.15.19:0/3346717061 >> 172.21.15.19:0/3346717061 conn(0x7fd5b00d1720 msgr2=0x7fd5b00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.087+0000 7fd5b5400640 1 -- 172.21.15.19:0/3346717061 shutdown_connections 2024-04-25T11:36:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.087+0000 7fd5b5400640 1 -- 172.21.15.19:0/3346717061 wait complete. 2024-04-25T11:36:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.090+0000 7fd5b5400640 1 Processor -- start 2024-04-25T11:36:41.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.090+0000 7fd5b5400640 1 -- start start 2024-04-25T11:36:41.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.091+0000 7fd5b5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594003aa0 0x7fd5b01483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.091+0000 7fd5b5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b01541e0 0x7fd5b015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b01545f0 0x7fd5b015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b5400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b015c800 con 0x7fd5b01545f0 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b006f590 con 0x7fd5b01541e0 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b015c980 con 0x7fd594003aa0 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b01545f0 0x7fd5b015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b01545f0 0x7fd5b015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40740/0 (socket says 172.21.15.19:40740) 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 -- 172.21.15.19:0/1701968408 learned_addr learned my addr 172.21.15.19:0/1701968408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5afe00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594003aa0 0x7fd5b01483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5a7e00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b01541e0 0x7fd5b015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 -- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594003aa0 msgr2=0x7fd5b01483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594003aa0 0x7fd5b01483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 -- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b01541e0 msgr2=0x7fd5b015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b01541e0 0x7fd5b015f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.092+0000 7fd5b4a00640 1 -- 172.21.15.19:0/1701968408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b0154d50 con 0x7fd5b01545f0 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.093+0000 7fd5b4a00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b01545f0 0x7fd5b015f660 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00d0650 tx=0x7fd5a800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.093+0000 7fd5acc00640 1 -- 172.21.15.19:0/1701968408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800a5b0 con 0x7fd5b01545f0 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.093+0000 7fd5acc00640 1 -- 172.21.15.19:0/1701968408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a800a750 con 0x7fd5b01545f0 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.093+0000 7fd5acc00640 1 -- 172.21.15.19:0/1701968408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a800ab80 con 0x7fd5b01545f0 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.093+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b0155030 con 0x7fd5b01545f0 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.093+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b0155850 con 0x7fd5b01545f0 2024-04-25T11:36:41.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.095+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b0154ac0 con 0x7fd5b01545f0 2024-04-25T11:36:41.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.096+0000 7fd5acc00640 1 -- 172.21.15.19:0/1701968408 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd5a800ad20 con 0x7fd5b01545f0 2024-04-25T11:36:41.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.097+0000 7fd5acc00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd58c0744e0 0x7fd58c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.097+0000 7fd5acc00640 1 -- 172.21.15.19:0/1701968408 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6225+0+0 (secure 0 0 0) 0x7fd5a8094950 con 0x7fd5b01545f0 2024-04-25T11:36:41.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.097+0000 7fd5afe00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd58c0744e0 0x7fd58c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.099+0000 7fd5afe00640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd58c0744e0 0x7fd58c0769f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0032000 tx=0x7fd5a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:41.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.102+0000 7fd5acc00640 1 -- 172.21.15.19:0/1701968408 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5a801a030 con 0x7fd5b01545f0 2024-04-25T11:36:41.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.346+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd5b0050740 con 0x7fd5b01545f0 2024-04-25T11:36:41.335 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:41.348+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:36:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.352+0000 7fd5acc00640 1 -- 172.21.15.19:0/1701968408 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v134) ==== 161+0+0 (secure 0 0 0) 0x7fd5a805eed0 con 0x7fd5b01545f0 2024-04-25T11:36:41.340 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:36:41.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd58c0744e0 msgr2=0x7fd58c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd58c0744e0 0x7fd58c0769f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0032000 tx=0x7fd5a002e000 comp rx=0 tx=0).stop 2024-04-25T11:36:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b01545f0 msgr2=0x7fd5b015f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b01545f0 0x7fd5b015f660 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00d0650 tx=0x7fd5a800ccd0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 shutdown_connections 2024-04-25T11:36:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd58c0744e0 0x7fd58c0769f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5b01541e0 0x7fd5b015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5b01545f0 0x7fd5b015f660 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 --2- 172.21.15.19:0/1701968408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd594003aa0 0x7fd5b01483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.356+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 >> 172.21.15.19:0/1701968408 conn(0x7fd5b00d1720 msgr2=0x7fd5b0159810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:41.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.357+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 shutdown_connections 2024-04-25T11:36:41.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.357+0000 7fd5b5400640 1 -- 172.21.15.19:0/1701968408 wait complete. 2024-04-25T11:36:41.349 INFO:tasks.ceph.mds.a.smithi019.stderr: -1904> 2024-04-25T11:36:28.617+0000 7fc0c8000640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-04-25T11:36:41.353 INFO:tasks.ceph.mds.a.smithi019.stderr: -767> 2024-04-25T11:36:33.567+0000 7fc0cd000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:36:41.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:36:41.485 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:36:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.653+0000 7f01ab200640 1 Processor -- start 2024-04-25T11:36:41.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.653+0000 7f01ab200640 1 -- start start 2024-04-25T11:36:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01ab200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f01ac06c5c0 0x7f01ac0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01ab200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01ac0b4ea0 0x7f01ac15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01ab200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0d1230 con 0x7f01ac144370 2024-04-25T11:36:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01ab200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac0d06b0 con 0x7f01ac06c5c0 2024-04-25T11:36:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01ab200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac0d0990 con 0x7f01ac0b4ea0 2024-04-25T11:36:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01a9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f01ac06c5c0 0x7f01ac0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01a9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f01ac06c5c0 0x7f01ac0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40756/0 (socket says 172.21.15.19:40756) 2024-04-25T11:36:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.655+0000 7f01a9e00640 1 -- 172.21.15.19:0/338402618 learned_addr learned my addr 172.21.15.19:0/338402618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.656+0000 7f01a9e00640 1 -- 172.21.15.19:0/338402618 >> v2:172.21.15.169:3300/0 conn(0x7f01ac0b4ea0 msgr2=0x7f01ac15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.656+0000 7f01aa800640 1 --2- 172.21.15.19:0/338402618 >> v2:172.21.15.169:3300/0 conn(0x7f01ac0b4ea0 0x7f01ac15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.656+0000 7f01a9e00640 1 --2- 172.21.15.19:0/338402618 >> v2:172.21.15.169:3300/0 conn(0x7f01ac0b4ea0 0x7f01ac15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.656+0000 7f01a9e00640 1 -- 172.21.15.19:0/338402618 >> v1:172.21.15.169:6789/0 conn(0x7f01ac144370 legacy=0x7f01ac0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.656+0000 7f01a9e00640 1 -- 172.21.15.19:0/338402618 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac0d0c70 con 0x7f01ac06c5c0 2024-04-25T11:36:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.656+0000 7f01aa800640 1 --2- 172.21.15.19:0/338402618 >> v2:172.21.15.169:3300/0 conn(0x7f01ac0b4ea0 0x7f01ac15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.657+0000 7f01a9e00640 1 --2- 172.21.15.19:0/338402618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac06c5c0 0x7f01ac0cfa70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0d26c0 tx=0x7f019c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702df10cefa31012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.657+0000 7f01a9400640 1 -- 172.21.15.19:0/338402618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c037070 con 0x7f01ac06c5c0 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.657+0000 7f01a9400640 1 -- 172.21.15.19:0/338402618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c032440 con 0x7f01ac06c5c0 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.657+0000 7f01a9400640 1 -- 172.21.15.19:0/338402618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c032810 con 0x7f01ac06c5c0 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 -- 172.21.15.19:0/338402618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac06c5c0 msgr2=0x7f01ac0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 --2- 172.21.15.19:0/338402618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac06c5c0 0x7f01ac0cfa70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0d26c0 tx=0x7f019c02a230 comp rx=0 tx=0).stop 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 -- 172.21.15.19:0/338402618 shutdown_connections 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 --2- 172.21.15.19:0/338402618 >> v2:172.21.15.169:3300/0 conn(0x7f01ac0b4ea0 0x7f01ac15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 --2- 172.21.15.19:0/338402618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac06c5c0 0x7f01ac0cfa70 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 -- 172.21.15.19:0/338402618 >> 172.21.15.19:0/338402618 conn(0x7f01ac0d1720 msgr2=0x7f01ac0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 -- 172.21.15.19:0/338402618 shutdown_connections 2024-04-25T11:36:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.658+0000 7f01ab200640 1 -- 172.21.15.19:0/338402618 wait complete. 2024-04-25T11:36:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.660+0000 7f01ab200640 1 Processor -- start 2024-04-25T11:36:41.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.661+0000 7f01ab200640 1 -- start start 2024-04-25T11:36:41.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.662+0000 7f01ab200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ac06c5c0 0x7f01ac146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.662+0000 7f01ab200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac144370 0x7f01ac133fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.662+0000 7f01ab200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ac13f230 0x7f01ac0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.662+0000 7f01ab200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac0d0450 con 0x7f01ac144370 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.662+0000 7f01ab200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac0d02d0 con 0x7f01ac06c5c0 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.662+0000 7f01ab200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac147760 con 0x7f01ac13f230 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.662+0000 7f01a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac144370 0x7f01ac133fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac144370 0x7f01ac133fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40772/0 (socket says 172.21.15.19:40772) 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 -- 172.21.15.19:0/1491045379 learned_addr learned my addr 172.21.15.19:0/1491045379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a9e00640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ac06c5c0 0x7f01ac146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 -- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ac13f230 msgr2=0x7f01ac0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ac13f230 0x7f01ac0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 -- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ac06c5c0 msgr2=0x7f01ac146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ac06c5c0 0x7f01ac146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 -- 172.21.15.19:0/1491045379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac15c760 con 0x7f01ac144370 2024-04-25T11:36:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.663+0000 7f01a3e00640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac144370 0x7f01ac133fe0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f019400b3f0 tx=0x7f019400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.664+0000 7f01a2000640 1 -- 172.21.15.19:0/1491045379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194015020 con 0x7f01ac144370 2024-04-25T11:36:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.664+0000 7f01a2000640 1 -- 172.21.15.19:0/1491045379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0194011830 con 0x7f01ac144370 2024-04-25T11:36:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.664+0000 7f01a2000640 1 -- 172.21.15.19:0/1491045379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194011c20 con 0x7f01ac144370 2024-04-25T11:36:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.664+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac145370 con 0x7f01ac144370 2024-04-25T11:36:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.664+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac15cd00 con 0x7f01ac144370 2024-04-25T11:36:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.665+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac050740 con 0x7f01ac144370 2024-04-25T11:36:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.666+0000 7f01a2000640 1 -- 172.21.15.19:0/1491045379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0194018020 con 0x7f01ac144370 2024-04-25T11:36:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.667+0000 7f01a2000640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01880744e0 0x7f01880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.667+0000 7f01a2000640 1 -- 172.21.15.19:0/1491045379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f0194095140 con 0x7f01ac144370 2024-04-25T11:36:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.668+0000 7f01a9e00640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01880744e0 0x7f01880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.669+0000 7f01a9e00640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01880744e0 0x7f01880769f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.672+0000 7f01a2000640 1 -- 172.21.15.19:0/1491045379 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f019401f030 con 0x7f01ac144370 2024-04-25T11:36:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:41.953+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f01ac073810 con 0x7f01ac144370 2024-04-25T11:36:42.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.356+0000 7f01a2000640 1 -- 172.21.15.19:0/1491045379 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v135) ==== 107+0+0 (secure 0 0 0) 0x7f019405f5d0 con 0x7f01ac144370 2024-04-25T11:36:42.344 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01880744e0 msgr2=0x7f01880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01880744e0 0x7f01880769f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f019c034000 tx=0x7f019c02e000 comp rx=0 tx=0).stop 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac144370 msgr2=0x7f01ac133fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac144370 0x7f01ac133fe0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f019400b3f0 tx=0x7f019400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 shutdown_connections 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f01880744e0 0x7f01880769f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01ac13f230 0x7f01ac0afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01ac144370 0x7f01ac133fe0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 --2- 172.21.15.19:0/1491045379 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01ac06c5c0 0x7f01ac146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 >> 172.21.15.19:0/1491045379 conn(0x7f01ac0d1720 msgr2=0x7f01ac0b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 shutdown_connections 2024-04-25T11:36:42.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.360+0000 7f01ab200640 1 -- 172.21.15.19:0/1491045379 wait complete. 2024-04-25T11:36:42.380 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:36:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.569+0000 7f5c81f08f40 1 Processor -- start 2024-04-25T11:36:42.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.569+0000 7f5c81f08f40 1 -- start start 2024-04-25T11:36:42.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c81f08f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c60002820 0x55e0811aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c81f08f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0811241b0 0x55e0810de470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c81f08f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e0810de9b0 0x55e08110c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c81f08f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e081085d20 con 0x7f5c60002820 2024-04-25T11:36:42.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c81f08f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e0810e0af0 con 0x55e0811241b0 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c81f08f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e0811b2570 con 0x55e0810de9b0 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c7b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c60002820 0x55e0811aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c7b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c60002820 0x55e0811aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40792/0 (socket says 172.21.15.19:40792) 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.570+0000 7f5c7b000640 1 -- 172.21.15.19:0/157400019 learned_addr learned my addr 172.21.15.19:0/157400019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c7b000640 1 -- 172.21.15.19:0/157400019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e0810de9b0 msgr2=0x55e08110c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c7b000640 1 --2- 172.21.15.19:0/157400019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e0810de9b0 0x55e08110c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c7b000640 1 -- 172.21.15.19:0/157400019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0811241b0 msgr2=0x55e0810de470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c7b000640 1 --2- 172.21.15.19:0/157400019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0811241b0 0x55e0810de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c7b000640 1 -- 172.21.15.19:0/157400019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0810dedc0 con 0x7f5c60002820 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c7b000640 1 --2- 172.21.15.19:0/157400019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c60002820 0x55e0811aab40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00d720 tx=0x7f5c6c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4251daf10f69fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c79c00640 1 -- 172.21.15.19:0/157400019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c013070 con 0x7f5c60002820 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c79c00640 1 -- 172.21.15.19:0/157400019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c00b850 con 0x7f5c60002820 2024-04-25T11:36:42.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.571+0000 7f5c79c00640 1 -- 172.21.15.19:0/157400019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00bc40 con 0x7f5c60002820 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 -- 172.21.15.19:0/157400019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c60002820 msgr2=0x55e0811aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/157400019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c60002820 0x55e0811aab40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00d720 tx=0x7f5c6c00dbe0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 -- 172.21.15.19:0/157400019 shutdown_connections 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/157400019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0811241b0 0x55e0810de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/157400019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e0810de9b0 0x55e08110c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/157400019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c60002820 0x55e0811aab40 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 -- 172.21.15.19:0/157400019 >> 172.21.15.19:0/157400019 conn(0x55e081001ac0 msgr2=0x55e0811b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 -- 172.21.15.19:0/157400019 shutdown_connections 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.572+0000 7f5c81f08f40 1 -- 172.21.15.19:0/157400019 wait complete. 2024-04-25T11:36:42.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.573+0000 7f5c81f08f40 1 Processor -- start 2024-04-25T11:36:42.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.574+0000 7f5c81f08f40 1 -- start start 2024-04-25T11:36:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c81f08f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0810de9b0 0x55e0811a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c81f08f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0811241b0 0x55e08119da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c81f08f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c60002820 0x55e0811a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c81f08f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e081123d20 con 0x55e0811241b0 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c81f08f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e0811b6bf0 con 0x55e0810de9b0 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c81f08f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e0811b22c0 con 0x7f5c60002820 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0811241b0 0x55e08119da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0811241b0 0x55e08119da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40804/0 (socket says 172.21.15.19:40804) 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.575+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 learned_addr learned my addr 172.21.15.19:0/2989122420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0810de9b0 0x55e0811a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c60002820 msgr2=0x55e0811a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73e00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5c60002820 0x55e0811a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0810de9b0 msgr2=0x55e0811a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73e00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0810de9b0 0x55e0811a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0811a8ca0 con 0x55e0811241b0 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e0810de9b0 0x55e0811a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73e00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0811241b0 0x55e08119da70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5c64025f70 tx=0x7f5c640058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73400640 1 -- 172.21.15.19:0/2989122420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64065030 con 0x55e0811241b0 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73400640 1 -- 172.21.15.19:0/2989122420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6400a040 con 0x55e0811241b0 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e0811a8f80 con 0x55e0811241b0 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.576+0000 7f5c73400640 1 -- 172.21.15.19:0/2989122420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c640577b0 con 0x55e0811241b0 2024-04-25T11:36:42.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.577+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0811a9540 con 0x55e0811241b0 2024-04-25T11:36:42.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.579+0000 7f5c73400640 1 -- 172.21.15.19:0/2989122420 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5c64068070 con 0x55e0811241b0 2024-04-25T11:36:42.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.580+0000 7f5c73400640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c58072e90 0x7f5c580753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.580+0000 7f5c73400640 1 -- 172.21.15.19:0/2989122420 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f5c640d7730 con 0x55e0811241b0 2024-04-25T11:36:42.568 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:36:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.580+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e08119e320 0x55e0811a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.580+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e0811aa280 con 0x55e08119e320 2024-04-25T11:36:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.580+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c58072e90 0x7f5c580753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.581+0000 7f5c7ba00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e08119e320 0x55e0811a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.582+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c58072e90 0x7f5c580753a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00dc20 tx=0x7f5c6c01a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.582+0000 7f5c7ba00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e08119e320 0x55e0811a9d40 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.585+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v99'195 uv169 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.585+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e0812590c0 con 0x55e08119e320 2024-04-25T11:36:42.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.588+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v99'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.588+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259460 con 0x55e08119e320 2024-04-25T11:36:42.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.591+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v99'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.579 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:36:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.591+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a810 con 0x55e08119e320 2024-04-25T11:36:42.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.594+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v99'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.594+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125abb0 con 0x55e08119e320 2024-04-25T11:36:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.597+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v99'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.597+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259800 con 0x55e08119e320 2024-04-25T11:36:42.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.600+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v99'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.601+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259bf0 con 0x55e08119e320 2024-04-25T11:36:42.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.604+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v99'201 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.604+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.608+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v99'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.608+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.599 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:36:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.612+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v99'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.612+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.616+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v99'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.616+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.620+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v99'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.620+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.625+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v99'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.625+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.628+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v99'207 uv182 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.629+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.632+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v99'208 uv208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.632+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.624 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:36:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.636+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v99'209 uv209 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.636+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.640+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v99'210 uv210 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.640+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.645+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v99'211 uv211 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.645+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081259f90 con 0x55e08119e320 2024-04-25T11:36:42.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.649+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v99'212 uv212 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.650+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.655+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v99'213 uv213 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.655+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.659+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v99'214 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.659+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.663+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v99'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.663+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.667+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v99'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.655 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:36:42.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.667+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.672+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v99'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.672+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.676+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v99'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.676+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.680+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v99'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.680+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.684+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v99'220 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.684+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.689+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v99'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.689+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.693+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v99'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.680 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:36:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.693+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.697+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v99'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.697+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.701+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v99'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.701+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125a370 con 0x55e08119e320 2024-04-25T11:36:42.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.705+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v99'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.706+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e08125a250 0x55e08125d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.706+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125d9b0 con 0x55e08125a250 2024-04-25T11:36:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.706+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e08125a250 0x55e08125d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.708+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e08125a250 0x55e08125d470 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.711+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v99'152 uv133 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.712+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08125dd50 con 0x55e08125a250 2024-04-25T11:36:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.717+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v99'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.717+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e0f0 con 0x55e08125a250 2024-04-25T11:36:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.721+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v99'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.709 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:36:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.721+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e490 con 0x55e08125a250 2024-04-25T11:36:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.726+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v99'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.726+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.730+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v99'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.730+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.734+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v99'157 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.734+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.738+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v99'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.739+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.742+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v99'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.730 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:36:42.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.742+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.746+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v99'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.746+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.750+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v99'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.751+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.755+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v99'162 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.755+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.759+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v99'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.759+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.751 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:36:42.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.763+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v99'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.764+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262f20 con 0x55e08125a250 2024-04-25T11:36:42.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.768+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v99'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.769+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262f20 con 0x55e08125a250 2024-04-25T11:36:42.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.773+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v99'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.773+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.777+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v99'167 uv147 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.777+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.781+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v99'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.781+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.773 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:36:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.785+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v99'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.785+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125f840 con 0x55e08125a250 2024-04-25T11:36:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.789+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v99'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.789+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125f840 con 0x55e08125a250 2024-04-25T11:36:42.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.793+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v99'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.794+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08125e830 con 0x55e08125a250 2024-04-25T11:36:42.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.798+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v99'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e08125a250 2024-04-25T11:36:42.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.798+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e08125eeb0 0x55e0812612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.798+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081261810 con 0x55e08125eeb0 2024-04-25T11:36:42.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.799+0000 7f5c73e00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e08125eeb0 0x55e0812612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.800+0000 7f5c73e00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e08125eeb0 0x55e0812612d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.805+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.805+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e081261bb0 con 0x55e08125eeb0 2024-04-25T11:36:42.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.809+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v99'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.809+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262060 con 0x55e08125eeb0 2024-04-25T11:36:42.801 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:36:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.813+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v99'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.813+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081263910 con 0x55e08125eeb0 2024-04-25T11:36:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.817+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.817+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081263cb0 con 0x55e08125eeb0 2024-04-25T11:36:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.821+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.821+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262900 con 0x55e08125eeb0 2024-04-25T11:36:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.826+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.826+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262cf0 con 0x55e08119e320 2024-04-25T11:36:42.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.830+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v99'34 uv33 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.830+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e081262cf0 con 0x55e08119e320 2024-04-25T11:36:42.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.834+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v99'35 uv35 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.835+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262cf0 con 0x55e08119e320 2024-04-25T11:36:42.826 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:36:42.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.838+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v99'36 uv36 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.839+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262cf0 con 0x55e08119e320 2024-04-25T11:36:42.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.842+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v99'37 uv37 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.843+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e081264050 0x55e081266470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.843+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262cf0 con 0x55e081264050 2024-04-25T11:36:42.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.844+0000 7f5c7ba00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e081264050 0x55e081266470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.845+0000 7f5c7ba00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e081264050 0x55e081266470 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.849+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.849+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e081262cf0 con 0x55e081264050 2024-04-25T11:36:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.853+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v99'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.853+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081262cf0 con 0x55e081264050 2024-04-25T11:36:42.844 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:36:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.857+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v99'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.857+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e0812679c0 con 0x55e081264050 2024-04-25T11:36:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.861+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.861+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e0812679c0 con 0x55e081264050 2024-04-25T11:36:42.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.865+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:42.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.865+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e0812669b0 con 0x55e081264050 2024-04-25T11:36:42.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.869+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.869+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e081266ec0 0x55e0812692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.869+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e0812698f0 con 0x55e081266ec0 2024-04-25T11:36:42.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.870+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e081266ec0 0x55e0812692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.871+0000 7f5c7b000640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e081266ec0 0x55e0812692e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.876+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v99'70 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.876+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e0812699b0 con 0x55e081266ec0 2024-04-25T11:36:42.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.880+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v99'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.880+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081269dd0 con 0x55e081266ec0 2024-04-25T11:36:42.871 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:36:42.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.884+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v99'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.884+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126ade0 con 0x55e081266ec0 2024-04-25T11:36:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.888+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v99'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.888+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126ade0 con 0x55e081266ec0 2024-04-25T11:36:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.892+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v99'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.892+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081269dd0 con 0x55e081266ec0 2024-04-25T11:36:42.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.896+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v99'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.896+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081269dd0 con 0x55e081266ec0 2024-04-25T11:36:42.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.900+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v99'76 uv69 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.900+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e081269dd0 con 0x55e081266ec0 2024-04-25T11:36:42.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.905+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v99'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.905+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081269dd0 con 0x55e081266ec0 2024-04-25T11:36:42.896 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:36:42.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.909+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v99'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.909+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081269dd0 con 0x55e081266ec0 2024-04-25T11:36:42.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.913+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v99'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.913+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e081269dd0 con 0x55e081266ec0 2024-04-25T11:36:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.917+0000 7f5c7b000640 1 -- 172.21.15.19:0/2989122420 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v99'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6c015030 con 0x55e081266ec0 2024-04-25T11:36:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.917+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e08126a450 0x55e08126c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:42.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.917+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126ced0 con 0x55e08126a450 2024-04-25T11:36:42.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.918+0000 7f5c73e00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e08126a450 0x55e08126c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.919+0000 7f5c73e00640 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e08126a450 0x55e08126c870 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.923+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v99'32 uv31 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.923+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126d2a0 con 0x55e08126a450 2024-04-25T11:36:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.927+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v99'33 uv33 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.928+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08126a450 2024-04-25T11:36:42.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.932+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v99'34 uv34 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:42.920 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:36:42.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.932+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08126a450 2024-04-25T11:36:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.937+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v99'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:42.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.937+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08126a450 2024-04-25T11:36:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.941+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v99'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:42.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.941+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08119e320 2024-04-25T11:36:42.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.945+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.945+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08119e320 2024-04-25T11:36:42.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.949+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v99'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.949+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08119e320 2024-04-25T11:36:42.940 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:36:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.953+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v99'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.953+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126e6d0 con 0x55e08119e320 2024-04-25T11:36:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.957+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.957+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126e6d0 con 0x55e08119e320 2024-04-25T11:36:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.961+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.961+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08119e320 2024-04-25T11:36:42.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.965+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e08119e320 2024-04-25T11:36:42.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.965+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.969+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v99'64 uv58 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.969+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.974+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v99'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.974+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.965 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:36:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.978+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v99'66 uv66 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.978+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.982+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v99'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.982+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.986+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v99'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.987+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.991+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v99'69 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.991+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.995+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v99'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.995+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.986 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:36:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.999+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v99'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:42.999+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d6c0 con 0x55e08125eeb0 2024-04-25T11:36:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.003+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v99'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.003+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e08125eeb0 2024-04-25T11:36:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.008+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v99'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08125eeb0 2024-04-25T11:36:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.008+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.012+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v99'31 uv29 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.013+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.017+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v99'32 uv32 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.017+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.009 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:36:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.021+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v99'33 uv33 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.021+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.026+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v99'34 uv34 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.026+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.030+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v99'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.030+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.035+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v99'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.035+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.038+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v99'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.026 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:36:43.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.039+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.042+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v99'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.043+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.047+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v99'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.047+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e081264050 2024-04-25T11:36:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.051+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v99'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c74002660 con 0x55e081264050 2024-04-25T11:36:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.051+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126dab0 con 0x55e08126a450 2024-04-25T11:36:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.055+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v99'32 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.055+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.059+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v99'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.060+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.063+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v99'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.051 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:36:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.063+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.067+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v99'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.067+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.072+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v99'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.072+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.076+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v99'82 uv73 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.076+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.080+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v99'83 uv83 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.081+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.072 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:36:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.084+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v99'84 uv84 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.085+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.089+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v99'85 uv85 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.089+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.093+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v99'86 uv86 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.093+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.097+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v99'87 uv80 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.097+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.101+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v99'88 uv88 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.101+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.105+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v99'89 uv89 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.092 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:36:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.105+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.109+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v99'90 uv90 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.109+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.113+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v99'91 uv91 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.113+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126d9e0 con 0x55e08126a450 2024-04-25T11:36:43.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.117+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v99'92 uv92 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.117+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) -- 0x55e08126e350 con 0x55e08126a450 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c73e00640 1 -- 172.21.15.19:0/2989122420 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v99'93 uv93 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5c6409d870 con 0x55e08126a450 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e08125a250 msgr2=0x55e08125d470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e08125a250 0x55e08125d470 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e081266ec0 msgr2=0x55e0812692e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e081266ec0 0x55e0812692e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e08126a450 msgr2=0x55e08126c870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e08126a450 0x55e08126c870 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e08125eeb0 msgr2=0x55e0812612d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e08125eeb0 0x55e0812612d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e08119e320 msgr2=0x55e0811a9d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e08119e320 0x55e0811a9d40 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c7ba00640 1 -- 172.21.15.19:0/2989122420 reap_dead start 2024-04-25T11:36:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e081264050 msgr2=0x55e081266470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.121+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e081264050 0x55e081266470 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c58072e90 msgr2=0x7f5c580753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c58072e90 0x7f5c580753a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00dc20 tx=0x7f5c6c01a040 comp rx=0 tx=0).stop 2024-04-25T11:36:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0811241b0 msgr2=0x55e08119da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0811241b0 0x55e08119da70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5c64025f70 tx=0x7f5c640058e0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 shutdown_connections 2024-04-25T11:36:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e081264050 0x55e081266470 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e08119e320 0x55e0811a9d40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5c58072e90 0x7f5c580753a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 --2- 172.21.15.19:0/2989122420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e0811241b0 0x55e08119da70 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 >> 172.21.15.19:0/2989122420 conn(0x55e081001ac0 msgr2=0x55e0811b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.122+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 shutdown_connections 2024-04-25T11:36:43.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.123+0000 7f5c81f08f40 1 -- 172.21.15.19:0/2989122420 wait complete. 2024-04-25T11:36:43.147 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2024-04-25T11:36:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.393+0000 7f4f75b1ff40 1 Processor -- start 2024-04-25T11:36:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.394+0000 7f4f75b1ff40 1 -- start start 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.395+0000 7f4f75b1ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb771e0 0x561f1ec4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.395+0000 7f4f75b1ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1eb775f0 0x561f1ebb9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.395+0000 7f4f75b1ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1ec50dd0 0x561f1ebb96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.395+0000 7f4f75b1ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561f1ebbfe70 con 0x561f1ec50dd0 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.395+0000 7f4f75b1ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561f1eb1dd20 con 0x561f1eb771e0 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.395+0000 7f4f75b1ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561f1ebc0110 con 0x561f1eb775f0 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.395+0000 7f4f6ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1ec50dd0 0x561f1ebb96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1ec50dd0 0x561f1ebb96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40834/0 (socket says 172.21.15.19:40834) 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 -- 172.21.15.19:0/522814924 learned_addr learned my addr 172.21.15.19:0/522814924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:43.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6e200640 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb771e0 0x561f1ec4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6d800640 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1eb775f0 0x561f1ebb9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 -- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1eb775f0 msgr2=0x561f1ebb9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1eb775f0 0x561f1ebb9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 -- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb771e0 msgr2=0x561f1ec4b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb771e0 0x561f1ec4b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 -- 172.21.15.19:0/522814924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f1ec4dd80 con 0x561f1ec50dd0 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6d800640 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1eb775f0 0x561f1ebb9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.396+0000 7f4f6ec00640 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1ec50dd0 0x561f1ebb96c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f4f68005fd0 tx=0x7f4f68050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb82e1256d43cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.397+0000 7f4f67e00640 1 -- 172.21.15.19:0/522814924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68051c20 con 0x561f1ec50dd0 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.397+0000 7f4f67e00640 1 -- 172.21.15.19:0/522814924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f68051dc0 con 0x561f1ec50dd0 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.397+0000 7f4f67e00640 1 -- 172.21.15.19:0/522814924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68065670 con 0x561f1ec50dd0 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.397+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/522814924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1ec50dd0 msgr2=0x561f1ebb96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.397+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1ec50dd0 0x561f1ebb96c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f4f68005fd0 tx=0x7f4f68050000 comp rx=0 tx=0).stop 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.398+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/522814924 shutdown_connections 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.398+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1ec50dd0 0x561f1ebb96c0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.398+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1eb775f0 0x561f1ebb9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.398+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/522814924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb771e0 0x561f1ec4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.398+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/522814924 >> 172.21.15.19:0/522814924 conn(0x561f1ea99ac0 msgr2=0x561f1ebc3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.398+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/522814924 shutdown_connections 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.398+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/522814924 wait complete. 2024-04-25T11:36:43.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.399+0000 7f4f75b1ff40 1 Processor -- start 2024-04-25T11:36:43.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.399+0000 7f4f75b1ff40 1 -- start start 2024-04-25T11:36:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.400+0000 7f4f75b1ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1eb771e0 0x561f1ec4d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.400+0000 7f4f75b1ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb775f0 0x561f1ec41b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f75b1ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1ec50dd0 0x561f1ece8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f75b1ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561f1ec4fd80 con 0x561f1eb771e0 2024-04-25T11:36:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f75b1ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561f1ec50060 con 0x561f1eb775f0 2024-04-25T11:36:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f75b1ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561f1ebb9f90 con 0x561f1ec50dd0 2024-04-25T11:36:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1eb771e0 0x561f1ec4d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1eb771e0 0x561f1ec4d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40848/0 (socket says 172.21.15.19:40848) 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 -- 172.21.15.19:0/3309999422 learned_addr learned my addr 172.21.15.19:0/3309999422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6ec00640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1ec50dd0 0x561f1ece8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 -- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1ec50dd0 msgr2=0x561f1ece8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6d800640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb775f0 0x561f1ec41b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561f1ec50dd0 0x561f1ece8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 -- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb775f0 msgr2=0x561f1ec41b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561f1eb775f0 0x561f1ec41b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.401+0000 7f4f6e200640 1 -- 172.21.15.19:0/3309999422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f1ec4c2b0 con 0x561f1eb771e0 2024-04-25T11:36:43.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.402+0000 7f4f6e200640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1eb771e0 0x561f1ec4d0e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000a3f0 tx=0x7f4f6000a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.402+0000 7f4f66a00640 1 -- 172.21.15.19:0/3309999422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000d960 con 0x561f1eb771e0 2024-04-25T11:36:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.402+0000 7f4f66a00640 1 -- 172.21.15.19:0/3309999422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f60011070 con 0x561f1eb771e0 2024-04-25T11:36:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.402+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f1ec4ca60 con 0x561f1eb771e0 2024-04-25T11:36:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.402+0000 7f4f66a00640 1 -- 172.21.15.19:0/3309999422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000dd60 con 0x561f1eb771e0 2024-04-25T11:36:43.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.402+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f1ec42080 con 0x561f1eb771e0 2024-04-25T11:36:43.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.404+0000 7f4f66a00640 1 -- 172.21.15.19:0/3309999422 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f60009040 con 0x561f1eb771e0 2024-04-25T11:36:43.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.405+0000 7f4f66a00640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f50072ec0 0x7f4f500753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.406+0000 7f4f66a00640 1 -- 172.21.15.19:0/3309999422 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f4f600944b0 con 0x561f1eb771e0 2024-04-25T11:36:43.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.406+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561f1ec4b000 0x561f1ec4b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.406+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e99) -- 0x561f1ec4b950 con 0x561f1ec4b000 2024-04-25T11:36:43.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.406+0000 7f4f6d800640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f50072ec0 0x7f4f500753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.406+0000 7f4f6ec00640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561f1ec4b000 0x561f1ec4b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.407+0000 7f4f6ec00640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561f1ec4b000 0x561f1ec4b410 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:43.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.407+0000 7f4f6d800640 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f50072ec0 0x7f4f500753d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x561f1ebc3ef0 tx=0x7f4f5c0055e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f6ec00640 1 -- 172.21.15.19:0/3309999422 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(1 500.00000000 [delete] v99'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4f68065430 con 0x561f1ec4b000 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561f1ec4b000 msgr2=0x561f1ec4b410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561f1ec4b000 0x561f1ec4b410 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f50072ec0 msgr2=0x7f4f500753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4f50072ec0 0x7f4f500753d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x561f1ebc3ef0 tx=0x7f4f5c0055e0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1eb771e0 msgr2=0x561f1ec4d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 --2- 172.21.15.19:0/3309999422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561f1eb771e0 0x561f1ec4d0e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000a3f0 tx=0x7f4f6000a8b0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f6ec00640 1 -- 172.21.15.19:0/3309999422 reap_dead start 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 shutdown_connections 2024-04-25T11:36:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 >> 172.21.15.19:0/3309999422 conn(0x561f1ea99ac0 msgr2=0x561f1ebb9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 shutdown_connections 2024-04-25T11:36:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.410+0000 7f4f75b1ff40 1 -- 172.21.15.19:0/3309999422 wait complete. 2024-04-25T11:36:43.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:36:43.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.710+0000 7f1fe6800640 1 Processor -- start 2024-04-25T11:36:43.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.711+0000 7f1fe6800640 1 -- start start 2024-04-25T11:36:43.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.713+0000 7f1fe6800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1fe0144340 0x7f1fe006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.713+0000 7f1fe6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fe015ee60 0x7f1fe015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.713+0000 7f1fe6800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe006df20 con 0x7f1fe006db10 2024-04-25T11:36:43.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.713+0000 7f1fe6800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe015f270 con 0x7f1fe015ee60 2024-04-25T11:36:43.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.713+0000 7f1fe6800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe0144750 con 0x7f1fe0144340 2024-04-25T11:36:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.713+0000 7f1fe5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fe015ee60 0x7f1fe015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.714+0000 7f1fe5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1fe015ee60 0x7f1fe015f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40860/0 (socket says 172.21.15.19:40860) 2024-04-25T11:36:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.714+0000 7f1fe5e00640 1 -- 172.21.15.19:0/4076780717 learned_addr learned my addr 172.21.15.19:0/4076780717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.714+0000 7f1fe5e00640 1 -- 172.21.15.19:0/4076780717 >> v2:172.21.15.169:3300/0 conn(0x7f1fe0144340 msgr2=0x7f1fe006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.714+0000 7f1fe5e00640 1 --2- 172.21.15.19:0/4076780717 >> v2:172.21.15.169:3300/0 conn(0x7f1fe0144340 0x7f1fe006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.714+0000 7f1fe5e00640 1 -- 172.21.15.19:0/4076780717 >> v1:172.21.15.169:6789/0 conn(0x7f1fe006db10 legacy=0x7f1fe015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.714+0000 7f1fe5e00640 1 -- 172.21.15.19:0/4076780717 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe0070000 con 0x7f1fe015ee60 2024-04-25T11:36:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.715+0000 7f1fe5e00640 1 --2- 172.21.15.19:0/4076780717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe015ee60 0x7f1fe015f4a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00b6e30 tx=0x7f1fd802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877adf4c89cb0820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.716+0000 7f1fdfe00640 1 -- 172.21.15.19:0/4076780717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8036070 con 0x7f1fe015ee60 2024-04-25T11:36:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.716+0000 7f1fdfe00640 1 -- 172.21.15.19:0/4076780717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd8031600 con 0x7f1fe015ee60 2024-04-25T11:36:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.716+0000 7f1fdfe00640 1 -- 172.21.15.19:0/4076780717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80319d0 con 0x7f1fe015ee60 2024-04-25T11:36:43.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.716+0000 7f1fe6800640 1 -- 172.21.15.19:0/4076780717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe015ee60 msgr2=0x7f1fe015f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.716+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4076780717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe015ee60 0x7f1fe015f4a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00b6e30 tx=0x7f1fd802a6f0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.717+0000 7f1fe6800640 1 -- 172.21.15.19:0/4076780717 shutdown_connections 2024-04-25T11:36:43.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.717+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4076780717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe015ee60 0x7f1fe015f4a0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.717+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4076780717 >> v2:172.21.15.169:3300/0 conn(0x7f1fe0144340 0x7f1fe006d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.717+0000 7f1fe6800640 1 -- 172.21.15.19:0/4076780717 >> 172.21.15.19:0/4076780717 conn(0x7f1fe0072f40 msgr2=0x7f1fe0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:43.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.717+0000 7f1fe6800640 1 -- 172.21.15.19:0/4076780717 shutdown_connections 2024-04-25T11:36:43.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.717+0000 7f1fe6800640 1 -- 172.21.15.19:0/4076780717 wait complete. 2024-04-25T11:36:43.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.719+0000 7f1fe6800640 1 Processor -- start 2024-04-25T11:36:43.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.719+0000 7f1fe6800640 1 -- start start 2024-04-25T11:36:43.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe6800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fe006db10 0x7f1fe0158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe0144340 0x7f1fe01364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe6800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fe015ee60 0x7f1fe0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe6800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe006f800 con 0x7f1fe0144340 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe6800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe006f520 con 0x7f1fe015ee60 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe6800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe006fae0 con 0x7f1fe006db10 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe0144340 0x7f1fe01364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe0144340 0x7f1fe01364a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40862/0 (socket says 172.21.15.19:40862) 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.721+0000 7f1fe4a00640 1 -- 172.21.15.19:0/4027874093 learned_addr learned my addr 172.21.15.19:0/4027874093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe5400640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fe006db10 0x7f1fe0158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe4a00640 1 -- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fe006db10 msgr2=0x7f1fe0158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe5e00640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fe015ee60 0x7f1fe0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe4a00640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fe006db10 0x7f1fe0158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe4a00640 1 -- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fe015ee60 msgr2=0x7f1fe0157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe4a00640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fe015ee60 0x7f1fe0157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe4a00640 1 -- 172.21.15.19:0/4027874093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe01356e0 con 0x7f1fe0144340 2024-04-25T11:36:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fe4a00640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe0144340 0x7f1fe01364a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00b3c0 tx=0x7f1fcc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.722+0000 7f1fdd600640 1 -- 172.21.15.19:0/4027874093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc011830 con 0x7f1fe0144340 2024-04-25T11:36:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.723+0000 7f1fdd600640 1 -- 172.21.15.19:0/4027874093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fcc016070 con 0x7f1fe0144340 2024-04-25T11:36:43.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.723+0000 7f1fdd600640 1 -- 172.21.15.19:0/4027874093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc011c50 con 0x7f1fe0144340 2024-04-25T11:36:43.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.723+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe01359c0 con 0x7f1fe0144340 2024-04-25T11:36:43.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.723+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe0135f80 con 0x7f1fe0144340 2024-04-25T11:36:43.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.724+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe01602f0 con 0x7f1fe0144340 2024-04-25T11:36:43.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.725+0000 7f1fdd600640 1 -- 172.21.15.19:0/4027874093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1fcc01b460 con 0x7f1fe0144340 2024-04-25T11:36:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.726+0000 7f1fdd600640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1fc00744b0 0x7f1fc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:43.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.726+0000 7f1fdd600640 1 -- 172.21.15.19:0/4027874093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f1fcc094ff0 con 0x7f1fe0144340 2024-04-25T11:36:43.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.727+0000 7f1fe5400640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1fc00744b0 0x7f1fc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:43.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.728+0000 7f1fe5400640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1fc00744b0 0x7f1fc00769c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400b810 tx=0x7f1fd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:43.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.732+0000 7f1fdd600640 1 -- 172.21.15.19:0/4027874093 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fcc020020 con 0x7f1fe0144340 2024-04-25T11:36:43.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:43.978+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1fe0076c60 con 0x7f1fe0144340 2024-04-25T11:36:44.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.110+0000 7f1fdd600640 1 -- 172.21.15.19:0/4027874093 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v136) ==== 162+0+0 (secure 0 0 0) 0x7f1fcc05f4c0 con 0x7f1fe0144340 2024-04-25T11:36:44.098 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:36:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.113+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1fc00744b0 msgr2=0x7f1fc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.113+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1fc00744b0 0x7f1fc00769c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400b810 tx=0x7f1fd4008040 comp rx=0 tx=0).stop 2024-04-25T11:36:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.113+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe0144340 msgr2=0x7f1fe01364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.113+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe0144340 0x7f1fe01364a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00b3c0 tx=0x7f1fcc00b880 comp rx=0 tx=0).stop 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 shutdown_connections 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1fc00744b0 0x7f1fc00769c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1fe015ee60 0x7f1fe0157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fe0144340 0x7f1fe01364a0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 --2- 172.21.15.19:0/4027874093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1fe006db10 0x7f1fe0158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 >> 172.21.15.19:0/4027874093 conn(0x7f1fe0072f40 msgr2=0x7f1fe0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 shutdown_connections 2024-04-25T11:36:44.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.114+0000 7f1fe6800640 1 -- 172.21.15.19:0/4027874093 wait complete. 2024-04-25T11:36:44.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:44.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.844+0000 7f79f8c00640 1 Processor -- start 2024-04-25T11:36:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.844+0000 7f79f8c00640 1 -- start start 2024-04-25T11:36:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.846+0000 7f79f8c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79f408cd60 0x7f79f408d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.846+0000 7f79f8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 0x7f79f40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.846+0000 7f79f8c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f408d6b0 con 0x7f79f40b4ff0 2024-04-25T11:36:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.846+0000 7f79f8c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f79f406e9d0 con 0x7f79f408cd60 2024-04-25T11:36:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.846+0000 7f79f8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79f406ddc0 con 0x7f79f40d1f70 2024-04-25T11:36:44.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.847+0000 7f79f3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f79f40b4ff0 0x7f79f41611e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42824/0 (socket says 172.21.15.19:42824) 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.847+0000 7f79f3400640 1 -- 172.21.15.19:0/3083669189 learned_addr learned my addr 172.21.15.19:0/3083669189 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.847+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 0x7f79f40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.847+0000 7f79f2a00640 1 --2- 172.21.15.19:0/3083669189 >> v2:172.21.15.169:3300/0 conn(0x7f79f408cd60 0x7f79f408d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.847+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1015277755 0 0) 0x7f79e4001290 con 0x7f79f40b4ff0 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79d8003520 con 0x7f79f40b4ff0 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2872519073 0 0) 0x7f79d8003520 con 0x7f79f40b4ff0 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 msgr2=0x7f79f40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 0x7f79f40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 >> v2:172.21.15.169:3300/0 conn(0x7f79f408cd60 msgr2=0x7f79f408d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 --2- 172.21.15.19:0/3083669189 >> v2:172.21.15.169:3300/0 conn(0x7f79f408cd60 0x7f79f408d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f406e0a0 con 0x7f79f40b4ff0 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f79e40019c0 con 0x7f79f40b4ff0 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 >> v1:172.21.15.19:6789/0 conn(0x7f79f40b4ff0 legacy=0x7f79f41611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 0x7f79d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d8003f80 0x7f79d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d8004d40 0x7f79d8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79f406ddc0 con 0x7f79d8004d40 2024-04-25T11:36:44.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79d800baa0 con 0x7f79d8003f80 2024-04-25T11:36:44.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.848+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79d800bca0 con 0x7f79f40d1f70 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d8004d40 0x7f79d8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f3400640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 0x7f79d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f2a00640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d8003f80 0x7f79d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f3e00640 1 -- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 msgr2=0x7f79d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40d1f70 0x7f79d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f3e00640 1 -- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d8003f80 msgr2=0x7f79d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79d8003f80 0x7f79d8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.849+0000 7f79f3e00640 1 -- 172.21.15.19:0/3083669189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d8003520 con 0x7f79d8004d40 2024-04-25T11:36:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.850+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d8004d40 0x7f79d8005150 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f79f40b5430 tx=0x7f79d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e63f1d38da3d485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.850+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec037070 con 0x7f79d8004d40 2024-04-25T11:36:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.850+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ec032620 con 0x7f79d8004d40 2024-04-25T11:36:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.850+0000 7f79f2000640 1 -- 172.21.15.19:0/3083669189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec009c10 con 0x7f79d8004d40 2024-04-25T11:36:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.851+0000 7f79f8c00640 1 -- 172.21.15.19:0/3083669189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d8004d40 msgr2=0x7f79d8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.851+0000 7f79f8c00640 1 --2- 172.21.15.19:0/3083669189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79d8004d40 0x7f79d8005150 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f79f40b5430 tx=0x7f79d8005830 comp rx=0 tx=0).stop 2024-04-25T11:36:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.851+0000 7f79f3e00640 1 -- 172.21.15.19:0/3083669189 reap_dead start 2024-04-25T11:36:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.851+0000 7f79f8c00640 1 -- 172.21.15.19:0/3083669189 shutdown_connections 2024-04-25T11:36:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.851+0000 7f79f8c00640 1 -- 172.21.15.19:0/3083669189 >> 172.21.15.19:0/3083669189 conn(0x7f79f40b5460 msgr2=0x7f79f40b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.851+0000 7f79f8c00640 1 -- 172.21.15.19:0/3083669189 shutdown_connections 2024-04-25T11:36:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.851+0000 7f79f8c00640 1 -- 172.21.15.19:0/3083669189 wait complete. 2024-04-25T11:36:44.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.854+0000 7f79f8c00640 1 Processor -- start 2024-04-25T11:36:44.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.854+0000 7f79f8c00640 1 -- start start 2024-04-25T11:36:44.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.855+0000 7f79f8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f41f67b0 0x7f79f4154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.855+0000 7f79f8c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f41eb940 0x7f79f41ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.855+0000 7f79f8c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f41562f0 0x7f79f4156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.855+0000 7f79f8c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79f40b9180 con 0x7f79f41f67b0 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.855+0000 7f79f8c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79f406e660 con 0x7f79f41562f0 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.855+0000 7f79f8c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79f406da20 con 0x7f79f41eb940 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f41f67b0 0x7f79f4154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f41f67b0 0x7f79f4154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42952/0 (socket says 172.21.15.19:42952) 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 -- 172.21.15.19:0/1783325924 learned_addr learned my addr 172.21.15.19:0/1783325924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f2a00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f41eb940 0x7f79f41ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 -- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f41eb940 msgr2=0x7f79f41ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f41eb940 0x7f79f41ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 -- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f41562f0 msgr2=0x7f79f4156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:44.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f41562f0 0x7f79f4156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.856+0000 7f79f3400640 1 -- 172.21.15.19:0/1783325924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f41ec290 con 0x7f79f41f67b0 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.857+0000 7f79f3400640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f41f67b0 0x7f79f4154bf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f79e400eaf0 tx=0x7f79e4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.857+0000 7f79eb400640 1 -- 172.21.15.19:0/1783325924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40072e0 con 0x7f79f41f67b0 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.857+0000 7f79eb400640 1 -- 172.21.15.19:0/1783325924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e4018070 con 0x7f79f41f67b0 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.857+0000 7f79eb400640 1 -- 172.21.15.19:0/1783325924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40057a0 con 0x7f79f41f67b0 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.857+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f415a9f0 con 0x7f79f41f67b0 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.857+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f415acb0 con 0x7f79f41f67b0 2024-04-25T11:36:44.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.859+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f4050740 con 0x7f79f41f67b0 2024-04-25T11:36:44.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.860+0000 7f79eb400640 1 -- 172.21.15.19:0/1783325924 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79e4005940 con 0x7f79f41f67b0 2024-04-25T11:36:44.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.861+0000 7f79eb400640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d40744b0 0x7f79d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:44.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.861+0000 7f79eb400640 1 -- 172.21.15.19:0/1783325924 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f79e40954f0 con 0x7f79f41f67b0 2024-04-25T11:36:44.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.862+0000 7f79f2a00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d40744b0 0x7f79d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:44.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.863+0000 7f79f2a00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d40744b0 0x7f79d40769c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f79f408d4c0 tx=0x7f79e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:44.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:44.866+0000 7f79eb400640 1 -- 172.21.15.19:0/1783325924 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79f41ec290 con 0x7f79f41f67b0 2024-04-25T11:36:45.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.110+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79f40b6370 con 0x7f79f41f67b0 2024-04-25T11:36:45.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.112+0000 7f79eb400640 1 -- 172.21.15.19:0/1783325924 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+5186 (secure 0 0 0) 0x7f79f408d6b0 con 0x7f79f41f67b0 2024-04-25T11:36:45.099 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:45.100 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":136,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:44.107205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8496":{"gid":8496,"name":"a","rank":0,"incarnation":136,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/2855003271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2855003271},{"type":"v1","addr":"172.21.15.19:6835","nonce":2855003271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:36:45.100 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 136 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d40744b0 msgr2=0x7f79d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d40744b0 0x7f79d40769c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f79f408d4c0 tx=0x7f79e0008040 comp rx=0 tx=0).stop 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f41f67b0 msgr2=0x7f79f4154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f41f67b0 0x7f79f4154bf0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f79e400eaf0 tx=0x7f79e4007ea0 comp rx=0 tx=0).stop 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 shutdown_connections 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d40744b0 0x7f79d40769c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f41562f0 0x7f79f4156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:45.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f41eb940 0x7f79f41ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:45.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 --2- 172.21.15.19:0/1783325924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f41f67b0 0x7f79f4154bf0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:45.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 >> 172.21.15.19:0/1783325924 conn(0x7f79f40b5460 msgr2=0x7f79f40711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:45.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 shutdown_connections 2024-04-25T11:36:45.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:45.115+0000 7f79f8c00640 1 -- 172.21.15.19:0/1783325924 wait complete. 2024-04-25T11:36:45.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 136, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:36:44.107205+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8496': {'gid': 8496, 'name': 'a', 'rank': 0, 'incarnation': 136, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/2855003271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2855003271}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2855003271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:36:45.131 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8496:up:replay 2024-04-25T11:36:45.131 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:36:47.359 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:36:47.371+0000 7f5e17000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:36:50.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:36:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.840+0000 7f05df200640 1 Processor -- start 2024-04-25T11:36:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.841+0000 7f05df200640 1 -- start start 2024-04-25T11:36:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.843+0000 7f05df200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f05e0144370 0x7f05e00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.843+0000 7f05df200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e00d1230 con 0x7f05e006c5c0 2024-04-25T11:36:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.843+0000 7f05df200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e00d06b0 con 0x7f05e00b4ea0 2024-04-25T11:36:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.843+0000 7f05df200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f05e00d0990 con 0x7f05e0144370 2024-04-25T11:36:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.844+0000 7f05dde00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f05e006c5c0 0x7f05e00b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42836/0 (socket says 172.21.15.19:42836) 2024-04-25T11:36:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.844+0000 7f05dde00640 1 -- 172.21.15.19:0/902535791 learned_addr learned my addr 172.21.15.19:0/902535791 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.844+0000 7f05de800640 1 --2- 172.21.15.19:0/902535791 >> v2:172.21.15.19:3300/0 conn(0x7f05e0144370 0x7f05e00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.844+0000 7f05de800640 1 -- 172.21.15.19:0/902535791 >> v1:172.21.15.169:6789/0 conn(0x7f05e00b4ea0 legacy=0x7f05e015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.844+0000 7f05de800640 1 -- 172.21.15.19:0/902535791 >> v1:172.21.15.19:6789/0 conn(0x7f05e006c5c0 legacy=0x7f05e00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.844+0000 7f05de800640 1 -- 172.21.15.19:0/902535791 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e00d0c70 con 0x7f05e0144370 2024-04-25T11:36:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.845+0000 7f05de800640 1 --2- 172.21.15.19:0/902535791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e0144370 0x7f05e00cfa70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009d90 tx=0x7f05d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2940e3bca14bf79a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.846+0000 7f05dd400640 1 -- 172.21.15.19:0/902535791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8007740 con 0x7f05e0144370 2024-04-25T11:36:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.846+0000 7f05dd400640 1 -- 172.21.15.19:0/902535791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d80078e0 con 0x7f05e0144370 2024-04-25T11:36:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.846+0000 7f05dd400640 1 -- 172.21.15.19:0/902535791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8007cb0 con 0x7f05e0144370 2024-04-25T11:36:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.847+0000 7f05df200640 1 -- 172.21.15.19:0/902535791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e0144370 msgr2=0x7f05e00cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.847+0000 7f05df200640 1 --2- 172.21.15.19:0/902535791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e0144370 0x7f05e00cfa70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009d90 tx=0x7f05d802a6f0 comp rx=0 tx=0).stop 2024-04-25T11:36:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.847+0000 7f05df200640 1 -- 172.21.15.19:0/902535791 shutdown_connections 2024-04-25T11:36:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.847+0000 7f05df200640 1 --2- 172.21.15.19:0/902535791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e0144370 0x7f05e00cfa70 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.847+0000 7f05df200640 1 -- 172.21.15.19:0/902535791 >> 172.21.15.19:0/902535791 conn(0x7f05e00d1720 msgr2=0x7f05e00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.847+0000 7f05df200640 1 -- 172.21.15.19:0/902535791 shutdown_connections 2024-04-25T11:36:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.847+0000 7f05df200640 1 -- 172.21.15.19:0/902535791 wait complete. 2024-04-25T11:36:50.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.849+0000 7f05df200640 1 Processor -- start 2024-04-25T11:36:50.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.850+0000 7f05df200640 1 -- start start 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05df200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e006c5c0 0x7f05e00b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05df200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e0144370 0x7f05e00b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05df200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e0138b60 0x7f05e01f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05df200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05e00d0450 con 0x7f05e006c5c0 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05df200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f05e00d02d0 con 0x7f05e0138b60 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05df200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f05e0147760 con 0x7f05e0144370 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05dde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e006c5c0 0x7f05e00b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05dde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e006c5c0 0x7f05e00b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42960/0 (socket says 172.21.15.19:42960) 2024-04-25T11:36:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.851+0000 7f05dde00640 1 -- 172.21.15.19:0/1754546409 learned_addr learned my addr 172.21.15.19:0/1754546409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05dde00640 1 -- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e0144370 msgr2=0x7f05e00b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05dde00640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e0144370 0x7f05e00b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05dde00640 1 -- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e0138b60 msgr2=0x7f05e01f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05dde00640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e0138b60 0x7f05e01f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05dde00640 1 -- 172.21.15.19:0/1754546409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e015fa40 con 0x7f05e006c5c0 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05dde00640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e006c5c0 0x7f05e00b0780 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c480 tx=0x7f05d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05d6000640 1 -- 172.21.15.19:0/1754546409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0017070 con 0x7f05e006c5c0 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05d6000640 1 -- 172.21.15.19:0/1754546409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d0005ce0 con 0x7f05e006c5c0 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05d6000640 1 -- 172.21.15.19:0/1754546409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d00072d0 con 0x7f05e006c5c0 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.852+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e015fd20 con 0x7f05e006c5c0 2024-04-25T11:36:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.853+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e0162170 con 0x7f05e006c5c0 2024-04-25T11:36:50.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.854+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e006d300 con 0x7f05e006c5c0 2024-04-25T11:36:50.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.855+0000 7f05d6000640 1 -- 172.21.15.19:0/1754546409 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f05d001a070 con 0x7f05e006c5c0 2024-04-25T11:36:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.857+0000 7f05d6000640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05bc0744b0 0x7f05bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.857+0000 7f05d6000640 1 -- 172.21.15.19:0/1754546409 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f05d0094750 con 0x7f05e006c5c0 2024-04-25T11:36:50.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.857+0000 7f05d7e00640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05bc0744b0 0x7f05bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.859+0000 7f05d7e00640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05bc0744b0 0x7f05bc0769c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f05c8000f60 tx=0x7f05c80079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:50.862+0000 7f05d6000640 1 -- 172.21.15.19:0/1754546409 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05e006d300 con 0x7f05e006c5c0 2024-04-25T11:36:51.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.108+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05e0050740 con 0x7f05e006c5c0 2024-04-25T11:36:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.110+0000 7f05d6000640 1 -- 172.21.15.19:0/1754546409 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) ==== 95+0+5193 (secure 0 0 0) 0x7f05d005e6d0 con 0x7f05e006c5c0 2024-04-25T11:36:51.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:51.098 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":139,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:36:47.118070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8496":{"gid":8496,"name":"a","rank":0,"incarnation":136,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/2855003271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2855003271},{"type":"v1","addr":"172.21.15.19:6835","nonce":2855003271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8496,"qdb_cluster":[8496]},"id":6}]} 2024-04-25T11:36:51.098 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 139 2024-04-25T11:36:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.112+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05bc0744b0 msgr2=0x7f05bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.112+0000 7f05df200640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05bc0744b0 0x7f05bc0769c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f05c8000f60 tx=0x7f05c80079d0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.112+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e006c5c0 msgr2=0x7f05e00b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.112+0000 7f05df200640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e006c5c0 0x7f05e00b0780 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c480 tx=0x7f05d000c940 comp rx=0 tx=0).stop 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 shutdown_connections 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05bc0744b0 0x7f05bc0769c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05e0138b60 0x7f05e01f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05e0144370 0x7f05e00b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 --2- 172.21.15.19:0/1754546409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05e006c5c0 0x7f05e00b0780 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 >> 172.21.15.19:0/1754546409 conn(0x7f05e00d1720 msgr2=0x7f05e00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 shutdown_connections 2024-04-25T11:36:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.113+0000 7f05df200640 1 -- 172.21.15.19:0/1754546409 wait complete. 2024-04-25T11:36:51.127 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:36:47.118070+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8496': {'gid': 8496, 'name': 'a', 'rank': 0, 'incarnation': 136, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/2855003271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2855003271}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2855003271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8496, 'qdb_cluster': [8496]} 2024-04-25T11:36:51.127 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:36:51.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:36:51.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.417+0000 7ff225600640 1 Processor -- start 2024-04-25T11:36:51.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.418+0000 7ff225600640 1 -- start start 2024-04-25T11:36:51.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.419+0000 7ff225600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff2200b6e10 0x7ff22015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.419+0000 7ff225600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2201608d0 0x7ff220160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff225600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff220161220 con 0x7ff2200cf2f0 2024-04-25T11:36:51.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff225600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff22006f870 con 0x7ff2201608d0 2024-04-25T11:36:51.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff225600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff2200b7220 con 0x7ff2200b6e10 2024-04-25T11:36:51.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff224c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2201608d0 0x7ff220160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff224c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2201608d0 0x7ff220160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42974/0 (socket says 172.21.15.19:42974) 2024-04-25T11:36:51.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff224c00640 1 -- 172.21.15.19:0/265369108 learned_addr learned my addr 172.21.15.19:0/265369108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff21f400640 1 --2- 172.21.15.19:0/265369108 >> v2:172.21.15.169:3300/0 conn(0x7ff2200b6e10 0x7ff22015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff224c00640 1 -- 172.21.15.19:0/265369108 >> v2:172.21.15.169:3300/0 conn(0x7ff2200b6e10 msgr2=0x7ff22015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff224c00640 1 --2- 172.21.15.19:0/265369108 >> v2:172.21.15.169:3300/0 conn(0x7ff2200b6e10 0x7ff22015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff224c00640 1 -- 172.21.15.19:0/265369108 >> v1:172.21.15.169:6789/0 conn(0x7ff2200cf2f0 legacy=0x7ff2200b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.420+0000 7ff224c00640 1 -- 172.21.15.19:0/265369108 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2200b74d0 con 0x7ff2201608d0 2024-04-25T11:36:51.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.422+0000 7ff224c00640 1 --2- 172.21.15.19:0/265369108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff220160ce0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff220070dd0 tx=0x7ff21802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4519b2fe2ee18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.422+0000 7ff21ea00640 1 -- 172.21.15.19:0/265369108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2180325a0 con 0x7ff2201608d0 2024-04-25T11:36:51.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.422+0000 7ff21ea00640 1 -- 172.21.15.19:0/265369108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff218038070 con 0x7ff2201608d0 2024-04-25T11:36:51.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.422+0000 7ff21ea00640 1 -- 172.21.15.19:0/265369108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218032960 con 0x7ff2201608d0 2024-04-25T11:36:51.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 -- 172.21.15.19:0/265369108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 msgr2=0x7ff220160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 --2- 172.21.15.19:0/265369108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff220160ce0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff220070dd0 tx=0x7ff21802c8e0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 -- 172.21.15.19:0/265369108 shutdown_connections 2024-04-25T11:36:51.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 --2- 172.21.15.19:0/265369108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff220160ce0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 --2- 172.21.15.19:0/265369108 >> v2:172.21.15.169:3300/0 conn(0x7ff2200b6e10 0x7ff22015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 -- 172.21.15.19:0/265369108 >> 172.21.15.19:0/265369108 conn(0x7ff2200d1720 msgr2=0x7ff2200d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 -- 172.21.15.19:0/265369108 shutdown_connections 2024-04-25T11:36:51.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.423+0000 7ff225600640 1 -- 172.21.15.19:0/265369108 wait complete. 2024-04-25T11:36:51.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.425+0000 7ff225600640 1 Processor -- start 2024-04-25T11:36:51.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.425+0000 7ff225600640 1 -- start start 2024-04-25T11:36:51.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff225600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2200b6e10 0x7ff2201e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff225600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2200cf2f0 0x7ff22015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff225600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff22015f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff225600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff22006f500 con 0x7ff2201608d0 2024-04-25T11:36:51.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff225600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff22006f220 con 0x7ff2200cf2f0 2024-04-25T11:36:51.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff225600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff22006fd00 con 0x7ff2200b6e10 2024-04-25T11:36:51.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff22015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff22015f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42980/0 (socket says 172.21.15.19:42980) 2024-04-25T11:36:51.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 -- 172.21.15.19:0/66409521 learned_addr learned my addr 172.21.15.19:0/66409521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 -- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2200b6e10 msgr2=0x7ff2201e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff21fe00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2200b6e10 0x7ff2201e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff21f400640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2200cf2f0 0x7ff22015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2200b6e10 0x7ff2201e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 -- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2200cf2f0 msgr2=0x7ff22015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2200cf2f0 0x7ff22015b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff224c00640 1 -- 172.21.15.19:0/66409521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2201e6c30 con 0x7ff2201608d0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.427+0000 7ff21fe00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2200b6e10 0x7ff2201e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.428+0000 7ff21f400640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2200cf2f0 0x7ff22015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.428+0000 7ff224c00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff22015f460 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7ff21802c920 tx=0x7ff218004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.428+0000 7ff217e00640 1 -- 172.21.15.19:0/66409521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218043070 con 0x7ff2201608d0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.428+0000 7ff217e00640 1 -- 172.21.15.19:0/66409521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2180339e0 con 0x7ff2201608d0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.428+0000 7ff217e00640 1 -- 172.21.15.19:0/66409521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218033dd0 con 0x7ff2201608d0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.428+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2201f68d0 con 0x7ff2201608d0 2024-04-25T11:36:51.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.428+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2201f6e90 con 0x7ff2201608d0 2024-04-25T11:36:51.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.432+0000 7ff217e00640 1 -- 172.21.15.19:0/66409521 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff218046070 con 0x7ff2201608d0 2024-04-25T11:36:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.433+0000 7ff217e00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1fc0744e0 0x7ff1fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.433+0000 7ff217e00640 1 -- 172.21.15.19:0/66409521 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7ff218030020 con 0x7ff2201608d0 2024-04-25T11:36:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.433+0000 7ff22dc08740 1 Processor -- start 2024-04-25T11:36:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.433+0000 7ff21fe00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1fc0744e0 0x7ff1fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.434+0000 7ff22dc08740 1 -- start start 2024-04-25T11:36:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff21fe00640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1fc0744e0 0x7ff1fc0769f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff20800a830 tx=0x7ff208008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d24529210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24543f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d24544480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff22dc08740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d24462600 con 0x556d2415c270 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d243ff100 con 0x556d241333b0 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556d2445e5c0 con 0x556d2413b590 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d24529210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d24529210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42996/0 (socket says 172.21.15.19:42996) 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.435+0000 7ff21fe00640 1 -- 172.21.15.19:0/2670955732 learned_addr learned my addr 172.21.15.19:0/2670955732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff21f400640 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24543f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff21fe00640 1 -- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 msgr2=0x556d24544480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff224c00640 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d24544480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff21fe00640 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d24544480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff21fe00640 1 -- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 msgr2=0x556d24543f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff21fe00640 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24543f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff21fe00640 1 -- 172.21.15.19:0/2670955732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d241742d0 con 0x556d2415c270 2024-04-25T11:36:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff21fe00640 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d24529210 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7ff20800ca90 tx=0x7ff20800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f6314ffe698db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff20fe00640 1 -- 172.21.15.19:0/2670955732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800ae60 con 0x556d2415c270 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff20fe00640 1 -- 172.21.15.19:0/2670955732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff208013dd0 con 0x556d2415c270 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.436+0000 7ff20fe00640 1 -- 172.21.15.19:0/2670955732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208004260 con 0x556d2415c270 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 -- 172.21.15.19:0/2670955732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 msgr2=0x556d24529210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d24529210 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7ff20800ca90 tx=0x7ff20800f000 comp rx=0 tx=0).stop 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 -- 172.21.15.19:0/2670955732 shutdown_connections 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24543f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d24544480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2670955732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d24529210 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 -- 172.21.15.19:0/2670955732 >> 172.21.15.19:0/2670955732 conn(0x556d241503c0 msgr2=0x556d244fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 -- 172.21.15.19:0/2670955732 shutdown_connections 2024-04-25T11:36:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.437+0000 7ff22dc08740 1 -- 172.21.15.19:0/2670955732 wait complete. 2024-04-25T11:36:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.438+0000 7ff22dc08740 1 -- start start 2024-04-25T11:36:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.439+0000 7ff22dc08740 1 Processor -- start 2024-04-25T11:36:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d241333b0 0x556d24522470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2413b590 0x556d2452ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2415c270 0x556d24524710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff22dc08740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d243e9f70 con 0x556d2413b590 2024-04-25T11:36:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d2453c410 con 0x556d2415c270 2024-04-25T11:36:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556d2452c290 con 0x556d241333b0 2024-04-25T11:36:51.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff22dc08740 10 client.0 _ll_register_callbacks cb 0x556d24458480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.440+0000 7ff21f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2413b590 0x556d2452ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2413b590 0x556d2452ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43004/0 (socket says 172.21.15.19:43004) 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 -- 172.21.15.19:0/1280608213 learned_addr learned my addr 172.21.15.19:0/1280608213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21fe00640 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d241333b0 0x556d24522470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 -- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d241333b0 msgr2=0x556d24522470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d241333b0 0x556d24522470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 -- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2415c270 msgr2=0x556d24524710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2415c270 0x556d24524710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 -- 172.21.15.19:0/1280608213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2423e6b0 con 0x556d2413b590 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21fe00640 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d241333b0 0x556d24522470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff21f400640 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2413b590 0x556d2452ae00 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ff210007a00 tx=0x7ff210007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6901b066381ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff20cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff20cc00640 1 -- 172.21.15.19:0/1280608213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210019030 con 0x556d2413b590 2024-04-25T11:36:51.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff20cc00640 1 -- 172.21.15.19:0/1280608213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7ff210010a70 con 0x556d2413b590 2024-04-25T11:36:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.441+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff208013dd0 con 0x556d2413b590 2024-04-25T11:36:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff20cc00640 1 -- 172.21.15.19:0/1280608213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) ==== 24+0+0 (secure 0 0 0) 0x7ff210022050 con 0x556d2413b590 2024-04-25T11:36:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 10 client.8553 fetch_fsmap learned FSMap version 139 2024-04-25T11:36:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) -- 0x556d2453c150 con 0x556d2413b590 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff20cc00640 1 -- 172.21.15.19:0/1280608213 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff208013dd0 con 0x556d2413b590 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff20cc00640 1 -- 172.21.15.19:0/1280608213 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 139) ==== 3494+0+0 (secure 0 0 0) 0x7ff210025020 con 0x556d2413b590 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 10 client.8553 fetch_fsmap finished waiting for FSMap version 139 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 10 client.8553 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 20 client.8553 populate_metadata read hostname 'smithi019' 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d2411ae40 0x556d2453b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 4 client.8553 mds_command: new command op to 8496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.442+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 --> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d24470210 con 0x556d2411ae40 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.443+0000 7ff21fe00640 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d2411ae40 0x556d2453b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.443+0000 7ff21fe00640 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d2411ae40 0x556d2453b510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f58de1d59597563a server_cookie=8d712c7b4c7c6ad4 in_seq=0 out_seq=0 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.443+0000 7ff20cc00640 10 client.8553 ms_handle_connect on v2:172.21.15.19:6834/2855003271 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff20cc00640 1 -- 172.21.15.19:0/1280608213 <== mds.0 v2:172.21.15.19:6834/2855003271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d24470210 con 0x556d2411ae40 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff20cc00640 10 client.8553 handle_command_reply: tid=0 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 client.8553 shutdown 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2413b590 msgr2=0x556d2452ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2413b590 0x556d2452ae00 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7ff210007a00 tx=0x7ff210007ec0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 shutdown_connections 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d2411ae40 0x556d2453b510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2415c270 0x556d24524710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2413b590 0x556d2452ae00 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1280608213 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d241333b0 0x556d24522470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff20cc00640 0 client.8553 ms_handle_reset on v2:172.21.15.19:6834/2855003271 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.444+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 >> 172.21.15.19:0/1280608213 conn(0x556d241503c0 msgr2=0x556d244fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.445+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 shutdown_connections 2024-04-25T11:36:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.445+0000 7ff22dc08740 1 -- 172.21.15.19:0/1280608213 wait complete. 2024-04-25T11:36:51.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.445+0000 7ff22dc08740 20 client.8553 trim_cache size 0 max 16384 2024-04-25T11:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.465+0000 7ff22dc08740 1 Processor -- start 2024-04-25T11:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.466+0000 7ff22dc08740 1 -- start start 2024-04-25T11:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24524330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d2453b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d2453ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff22dc08740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2202021b0 con 0x556d2415c270 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff22014a990 con 0x556d241333b0 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff220201b90 con 0x556d2413b590 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff224c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d2453ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff224c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d2453ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43018/0 (socket says 172.21.15.19:43018) 2024-04-25T11:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.467+0000 7ff224c00640 1 -- 172.21.15.19:0/2260096905 learned_addr learned my addr 172.21.15.19:0/2260096905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff21f400640 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d2453b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff224c00640 1 -- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 msgr2=0x556d2453b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff224c00640 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d2453b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff21fe00640 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24524330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff224c00640 1 -- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 msgr2=0x556d24524330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff224c00640 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24524330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff224c00640 1 -- 172.21.15.19:0/2260096905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2452a470 con 0x556d2415c270 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff224c00640 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d2453ba50 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff21807c3e0 tx=0x7ff21807f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d00c622079d6db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff20f400640 1 -- 172.21.15.19:0/2260096905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2180804f0 con 0x556d2415c270 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff20f400640 1 -- 172.21.15.19:0/2260096905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff218038040 con 0x556d2415c270 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.468+0000 7ff20f400640 1 -- 172.21.15.19:0/2260096905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21807f250 con 0x556d2415c270 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 -- 172.21.15.19:0/2260096905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 msgr2=0x556d2453ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d2453ba50 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff21807c3e0 tx=0x7ff21807f740 comp rx=0 tx=0).stop 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 -- 172.21.15.19:0/2260096905 shutdown_connections 2024-04-25T11:36:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2415c270 0x556d2453ba50 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2413b590 0x556d2453b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 --2- 172.21.15.19:0/2260096905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d241333b0 0x556d24524330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 -- 172.21.15.19:0/2260096905 >> 172.21.15.19:0/2260096905 conn(0x556d241503c0 msgr2=0x556d2452abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 -- 172.21.15.19:0/2260096905 shutdown_connections 2024-04-25T11:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.469+0000 7ff22dc08740 1 -- 172.21.15.19:0/2260096905 wait complete. 2024-04-25T11:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.470+0000 7ff22dc08740 1 -- start start 2024-04-25T11:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.471+0000 7ff22dc08740 1 Processor -- start 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d241333b0 0x556d245c0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2413b590 0x556d2453b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff22dc08740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2415c270 0x556d2453b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff22dc08740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d24556c20 con 0x556d241333b0 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556d2452b340 con 0x556d2413b590 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff22dc08740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556d24557270 con 0x556d2415c270 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff22dc08740 10 client.0 _ll_register_callbacks cb 0x7ff210009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d241333b0 0x556d245c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.472+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d241333b0 0x556d245c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43020/0 (socket says 172.21.15.19:43020) 2024-04-25T11:36:51.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff21fe00640 1 -- 172.21.15.19:0/1637117308 learned_addr learned my addr 172.21.15.19:0/1637117308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff21fe00640 1 -- 172.21.15.19:0/1637117308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2415c270 msgr2=0x556d2453b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff21fe00640 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2415c270 0x556d2453b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff21fe00640 1 -- 172.21.15.19:0/1637117308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2413b590 msgr2=0x556d2453b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff21fe00640 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2413b590 0x556d2453b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff21fe00640 1 -- 172.21.15.19:0/1637117308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d245c0d50 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff21fe00640 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d241333b0 0x556d245c0810 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff20801e000 tx=0x7ff208013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1320db7b583273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff20cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.473+0000 7ff20cc00640 1 -- 172.21.15.19:0/1637117308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208004040 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff20cc00640 1 -- 172.21.15.19:0/1637117308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7ff2080041e0 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff218038040 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff20cc00640 1 -- 172.21.15.19:0/1637117308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) ==== 24+0+0 (secure 0 0 0) 0x7ff208039050 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 10 client.8559 fetch_fsmap learned FSMap version 139 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) -- 0x556d244fd0a0 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff20cc00640 1 -- 172.21.15.19:0/1637117308 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff218038040 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff20cc00640 1 -- 172.21.15.19:0/1637117308 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 139) ==== 3494+0+0 (secure 0 0 0) 0x7ff20803c020 con 0x556d241333b0 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 10 client.8559 fetch_fsmap finished waiting for FSMap version 139 2024-04-25T11:36:51.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 10 client.8559 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:36:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 20 client.8559 populate_metadata read hostname 'smithi019' 2024-04-25T11:36:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d245c45a0 0x556d245c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 4 client.8559 mds_command: new command op to 8496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:36:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.474+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 --> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] -- command(tid 0: {"prefix": "status"}) -- 0x556d2446a080 con 0x556d245c45a0 2024-04-25T11:36:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.475+0000 7ff21f400640 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d245c45a0 0x556d245c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.475+0000 7ff20cc00640 10 client.8559 ms_handle_connect on v2:172.21.15.19:6834/2855003271 2024-04-25T11:36:51.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.475+0000 7ff21f400640 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d245c45a0 0x556d245c49b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648fc9e0a5dfebca server_cookie=87ce3d45cafa1a3 in_seq=0 out_seq=0 2024-04-25T11:36:51.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.475+0000 7ff20cc00640 1 -- 172.21.15.19:0/1637117308 <== mds.0 v2:172.21.15.19:6834/2855003271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556d2446a080 con 0x556d245c45a0 2024-04-25T11:36:51.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.475+0000 7ff20cc00640 10 client.8559 handle_command_reply: tid=0 2024-04-25T11:36:51.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.475+0000 7ff22dc08740 1 client.8559 shutdown 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d241333b0 msgr2=0x556d245c0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d241333b0 0x556d245c0810 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff20801e000 tx=0x7ff208013000 comp rx=0 tx=0).stop 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 shutdown_connections 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:6834/2855003271,v1:172.21.15.19:6835/2855003271] conn(0x556d245c45a0 0x556d245c49b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556d2413b590 0x556d2453b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556d2415c270 0x556d2453b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 --2- 172.21.15.19:0/1637117308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d241333b0 0x556d245c0810 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff20cc00640 0 client.8559 ms_handle_reset on v2:172.21.15.19:6834/2855003271 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 >> 172.21.15.19:0/1637117308 conn(0x556d241503c0 msgr2=0x556d245399b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 shutdown_connections 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 1 -- 172.21.15.19:0/1637117308 wait complete. 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.476+0000 7ff22dc08740 20 client.8559 trim_cache size 0 max 16384 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8496, 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3635444310000002, 2024-04-25T11:36:51.467 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 139, 2024-04-25T11:36:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 99, 2024-04-25T11:36:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 99, 2024-04-25T11:36:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 9.9734230270000008 2024-04-25T11:36:51.468 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:36:51.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1fc0744e0 msgr2=0x7ff1fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1fc0744e0 0x7ff1fc0769f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff20800a830 tx=0x7ff208008040 comp rx=0 tx=0).stop 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 msgr2=0x7ff22015f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff22015f460 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7ff21802c920 tx=0x7ff218004730 comp rx=0 tx=0).stop 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 shutdown_connections 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1fc0744e0 0x7ff1fc0769f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2200cf2f0 0x7ff22015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2201608d0 0x7ff22015f460 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 --2- 172.21.15.19:0/66409521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2200b6e10 0x7ff2201e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 >> 172.21.15.19:0/66409521 conn(0x7ff2200d1720 msgr2=0x7ff2200d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 shutdown_connections 2024-04-25T11:36:51.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.477+0000 7ff225600640 1 -- 172.21.15.19:0/66409521 wait complete. 2024-04-25T11:36:51.488 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:36:51.488 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2024-04-25T11:36:51.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:36:51.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.779+0000 7f30f7e00640 1 Processor -- start 2024-04-25T11:36:51.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.779+0000 7f30f7e00640 1 -- start start 2024-04-25T11:36:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.781+0000 7f30f7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30f8073d90 0x7f30f8163630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.781+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.781+0000 7f30f7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f806fc10 con 0x7f30f80afe40 2024-04-25T11:36:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.781+0000 7f30f7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f30f80cc560 con 0x7f30f8073d90 2024-04-25T11:36:51.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.781+0000 7f30f7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30f806f650 con 0x7f30f8164630 2024-04-25T11:36:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30f8073d90 0x7f30f8163630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30f8073d90 0x7f30f8163630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43028/0 (socket says 172.21.15.19:43028) 2024-04-25T11:36:51.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f6000640 1 -- 172.21.15.19:0/3048337318 learned_addr learned my addr 172.21.15.19:0/3048337318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f7400640 1 --2- 172.21.15.19:0/3048337318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f6000640 1 -- 172.21.15.19:0/3048337318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 msgr2=0x7f30f806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f6000640 1 --2- 172.21.15.19:0/3048337318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f806c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f6000640 1 -- 172.21.15.19:0/3048337318 >> v1:172.21.15.169:6789/0 conn(0x7f30f80afe40 legacy=0x7f30f8073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:51.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f6000640 1 -- 172.21.15.19:0/3048337318 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f806f930 con 0x7f30f8073d90 2024-04-25T11:36:51.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.782+0000 7f30f7400640 1 --2- 172.21.15.19:0/3048337318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:51.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.784+0000 7f30f6000640 1 --2- 172.21.15.19:0/3048337318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f8073d90 0x7f30f8163630 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009870 tx=0x7f30e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d74ce8ca976f029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.784+0000 7f30f5600640 1 -- 172.21.15.19:0/3048337318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4037070 con 0x7f30f8073d90 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.784+0000 7f30f5600640 1 -- 172.21.15.19:0/3048337318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e4032440 con 0x7f30f8073d90 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.784+0000 7f30f5600640 1 -- 172.21.15.19:0/3048337318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4032810 con 0x7f30f8073d90 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 -- 172.21.15.19:0/3048337318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f8073d90 msgr2=0x7f30f8163630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 --2- 172.21.15.19:0/3048337318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f8073d90 0x7f30f8163630 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009870 tx=0x7f30e402a210 comp rx=0 tx=0).stop 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 -- 172.21.15.19:0/3048337318 shutdown_connections 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 --2- 172.21.15.19:0/3048337318 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 --2- 172.21.15.19:0/3048337318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f8073d90 0x7f30f8163630 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 -- 172.21.15.19:0/3048337318 >> 172.21.15.19:0/3048337318 conn(0x7f30f80d1f10 msgr2=0x7f30f80d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 -- 172.21.15.19:0/3048337318 shutdown_connections 2024-04-25T11:36:51.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.785+0000 7f30f7e00640 1 -- 172.21.15.19:0/3048337318 wait complete. 2024-04-25T11:36:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.787+0000 7f30f7e00640 1 Processor -- start 2024-04-25T11:36:51.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.788+0000 7f30f7e00640 1 -- start start 2024-04-25T11:36:51.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.789+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f80afe40 0x7f30f80b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.789+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.789+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30f80d0750 0x7f30f80d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.789+0000 7f30f7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8164a40 con 0x7f30f80afe40 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.789+0000 7f30f7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80b0370 con 0x7f30f80d0750 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.789+0000 7f30f7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30f806cb00 con 0x7f30f8164630 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.789+0000 7f30f6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f80afe40 0x7f30f80b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f80afe40 0x7f30f80b6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43040/0 (socket says 172.21.15.19:43040) 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 -- 172.21.15.19:0/1654593720 learned_addr learned my addr 172.21.15.19:0/1654593720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 -- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 msgr2=0x7f30f815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 -- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30f80d0750 msgr2=0x7f30f80d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:51.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30f80d0750 0x7f30f80d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:51.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 -- 172.21.15.19:0/1654593720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f80d10a0 con 0x7f30f80afe40 2024-04-25T11:36:51.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.790+0000 7f30f6a00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f80afe40 0x7f30f80b6c60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b420 tx=0x7f30e800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.791+0000 7f30eea00640 1 -- 172.21.15.19:0/1654593720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8011830 con 0x7f30f80afe40 2024-04-25T11:36:51.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.791+0000 7f30eea00640 1 -- 172.21.15.19:0/1654593720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8016070 con 0x7f30f80afe40 2024-04-25T11:36:51.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.791+0000 7f30eea00640 1 -- 172.21.15.19:0/1654593720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8011c10 con 0x7f30f80afe40 2024-04-25T11:36:51.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.791+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f80b4b00 con 0x7f30f80afe40 2024-04-25T11:36:51.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.791+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f80b5000 con 0x7f30f80afe40 2024-04-25T11:36:51.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.792+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f8075f70 con 0x7f30f80afe40 2024-04-25T11:36:51.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.793+0000 7f30eea00640 1 -- 172.21.15.19:0/1654593720 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f30e801b460 con 0x7f30f80afe40 2024-04-25T11:36:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.795+0000 7f30eea00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30d80744b0 0x7f30d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.796+0000 7f30eea00640 1 -- 172.21.15.19:0/1654593720 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f30e8094fa0 con 0x7f30f80afe40 2024-04-25T11:36:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.796+0000 7f30f6000640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30d80744b0 0x7f30d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:51.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.797+0000 7f30f6000640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30d80744b0 0x7f30d80769c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f30e4034000 tx=0x7f30e402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:51.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:51.801+0000 7f30eea00640 1 -- 172.21.15.19:0/1654593720 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30e8020020 con 0x7f30f80afe40 2024-04-25T11:36:52.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.041+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f30f80741b0 con 0x7f30f80afe40 2024-04-25T11:36:52.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.041+0000 7f30eea00640 1 -- 172.21.15.19:0/1654593720 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v139) ==== 53+0+73 (secure 0 0 0) 0x7f30f80d10a0 con 0x7f30f80afe40 2024-04-25T11:36:52.029 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30d80744b0 msgr2=0x7f30d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30d80744b0 0x7f30d80769c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f30e4034000 tx=0x7f30e402c000 comp rx=0 tx=0).stop 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f80afe40 msgr2=0x7f30f80b6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f80afe40 0x7f30f80b6c60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b420 tx=0x7f30e800b8e0 comp rx=0 tx=0).stop 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 shutdown_connections 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30d80744b0 0x7f30d80769c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30f80d0750 0x7f30f80d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30f8164630 0x7f30f815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 --2- 172.21.15.19:0/1654593720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30f80afe40 0x7f30f80b6c60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:52.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 >> 172.21.15.19:0/1654593720 conn(0x7f30f80d1f10 msgr2=0x7f30f80b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:52.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 shutdown_connections 2024-04-25T11:36:52.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52.044+0000 7f30f7e00640 1 -- 172.21.15.19:0/1654593720 wait complete. 2024-04-25T11:36:52.057 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:36:52.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:36:52.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:36:52.057 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:36:52.057 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:52.057 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:36:52.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:36:52.058 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:36:52.058 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:36:52.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:52.058 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:36:52.110 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:36:52.110 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:36:52.110 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:36:52.111 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42061sec preferred_lft 42061sec 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:36:52.112 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:36:52.113 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:36:52.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:52.115 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:36:52.192 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:36:52.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:52.198 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:36:52.255 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:36:52.255 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:36:52.255 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:36:52.255 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:36:52.255 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:36:52.256 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:36:52.256 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:36:52.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:36:52.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:36:52.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:36:52.256 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:36:52.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:36:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:36:52.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:52.484 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:36:52.535 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:36:52.536 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:36:52.536 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:36:52.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:52.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:36:52.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:36:52.622 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:52.678 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:36:52.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:36:52.678 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:52.734 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:36:52.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:52.734 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:36:52.953 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:36:52.953 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:36:52.996 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:36:52.997 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:36:52.997 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:36:52.997 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:36:52.997 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:36:52.999 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:36:52.999 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:36:52.999 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:36:52.999 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:36:52.999 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:36:53.002 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:36:53.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:53.002 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:36:53.077 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:36:53.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:36:53.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:36:53.207 INFO:teuthology.orchestra.run.smithi019.stdout:client6077 2024-04-25T11:36:53.208 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:53.208 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:53.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:53.261 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:36:53.266 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:36:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:36:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:36:53.320 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:36:53.320 INFO:teuthology.orchestra.run.smithi019.stdout:sixmegs 2024-04-25T11:36:53.321 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2024-04-25T11:36:53.321 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2024-04-25T11:36:53.321 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:53.321 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:53.376 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:36:53.376 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:36:53.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:36:53.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.702+0000 7f9487e00640 1 Processor -- start 2024-04-25T11:36:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.702+0000 7f9487e00640 1 -- start start 2024-04-25T11:36:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.704+0000 7f9487e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9488144370 0x7f94880cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.704+0000 7f9487e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94880d1230 con 0x7f948806c5c0 2024-04-25T11:36:53.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.704+0000 7f9487e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94880d06b0 con 0x7f94880b4ea0 2024-04-25T11:36:53.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.704+0000 7f9487e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f94880d0990 con 0x7f9488144370 2024-04-25T11:36:53.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.704+0000 7f9486a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f948806c5c0 0x7f94880b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42846/0 (socket says 172.21.15.19:42846) 2024-04-25T11:36:53.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.704+0000 7f9486a00640 1 -- 172.21.15.19:0/3164600077 learned_addr learned my addr 172.21.15.19:0/3164600077 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9487400640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9488144370 0x7f94880cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546651849 0 0) 0x7f94880d1230 con 0x7f948806c5c0 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f946c003520 con 0x7f948806c5c0 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1176130019 0 0) 0x7f946c003520 con 0x7f948806c5c0 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9488144370 msgr2=0x7f94880cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9488144370 0x7f94880cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 >> v1:172.21.15.169:6789/0 conn(0x7f94880b4ea0 legacy=0x7f948815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94880d0c70 con 0x7f948806c5c0 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9470001670 con 0x7f948806c5c0 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 >> v1:172.21.15.19:6789/0 conn(0x7f948806c5c0 legacy=0x7f94880b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c0039e0 0x7f946c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946c004800 0x7f946c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9488144370 0x7f946c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94880d0990 con 0x7f946c004800 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f946c00baf0 con 0x7f946c0039e0 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.705+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f946c00bcf0 con 0x7f9488144370 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486000640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946c004800 0x7f946c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486a00640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c0039e0 0x7f946c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486000640 1 -- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9488144370 msgr2=0x7f946c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486000640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9488144370 0x7f946c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486000640 1 -- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c0039e0 msgr2=0x7f946c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486000640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c0039e0 0x7f946c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486000640 1 -- 172.21.15.19:0/3164600077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c003520 con 0x7f946c004800 2024-04-25T11:36:53.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.706+0000 7f9486a00640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c0039e0 0x7f946c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.707+0000 7f9486000640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946c004800 0x7f946c004c10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9478001720 tx=0x7f946c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fae00e1b2e4305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.708+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478001370 con 0x7f946c004800 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.708+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94780052d0 con 0x7f946c004800 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.708+0000 7f9485600640 1 -- 172.21.15.19:0/3164600077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947802abc0 con 0x7f946c004800 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.708+0000 7f9487e00640 1 -- 172.21.15.19:0/3164600077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946c004800 msgr2=0x7f946c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.708+0000 7f9487e00640 1 --2- 172.21.15.19:0/3164600077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f946c004800 0x7f946c004c10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9478001720 tx=0x7f946c005830 comp rx=0 tx=0).stop 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.708+0000 7f9487400640 1 -- 172.21.15.19:0/3164600077 reap_dead start 2024-04-25T11:36:53.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.709+0000 7f9487e00640 1 -- 172.21.15.19:0/3164600077 shutdown_connections 2024-04-25T11:36:53.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.709+0000 7f9487e00640 1 -- 172.21.15.19:0/3164600077 >> 172.21.15.19:0/3164600077 conn(0x7f94880d1720 msgr2=0x7f94880d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:53.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.709+0000 7f9487e00640 1 -- 172.21.15.19:0/3164600077 shutdown_connections 2024-04-25T11:36:53.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.709+0000 7f9487e00640 1 -- 172.21.15.19:0/3164600077 wait complete. 2024-04-25T11:36:53.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.711+0000 7f9487e00640 1 Processor -- start 2024-04-25T11:36:53.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.711+0000 7f9487e00640 1 -- start start 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c004800 0x7f9488135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948815b840 0x7f94880cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948815f910 0x7f948815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94880d0450 con 0x7f948815f910 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f94880d02d0 con 0x7f946c004800 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9488147760 con 0x7f948815b840 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948815f910 0x7f948815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948815f910 0x7f948815fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43062/0 (socket says 172.21.15.19:43062) 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9487400640 1 -- 172.21.15.19:0/1172782562 learned_addr learned my addr 172.21.15.19:0/1172782562 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:53.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.713+0000 7f9486a00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c004800 0x7f9488135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9486000640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948815b840 0x7f94880cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9487400640 1 -- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948815b840 msgr2=0x7f94880cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9487400640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948815b840 0x7f94880cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9487400640 1 -- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c004800 msgr2=0x7f9488135ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9487400640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c004800 0x7f9488135ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9487400640 1 -- 172.21.15.19:0/1172782562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9488136010 con 0x7f948815f910 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9487400640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948815f910 0x7f948815fd20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f94880b6ca0 tx=0x7f948000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f947f400640 1 -- 172.21.15.19:0/1172782562 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948000bae0 con 0x7f948815f910 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f947f400640 1 -- 172.21.15.19:0/1172782562 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948001a070 con 0x7f948815f910 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f947f400640 1 -- 172.21.15.19:0/1172782562 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480014400 con 0x7f948815f910 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.714+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94880cc020 con 0x7f948815f910 2024-04-25T11:36:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.715+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94880cc5e0 con 0x7f948815f910 2024-04-25T11:36:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.716+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9488050740 con 0x7f948815f910 2024-04-25T11:36:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.717+0000 7f947f400640 1 -- 172.21.15.19:0/1172782562 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9480005d10 con 0x7f948815f910 2024-04-25T11:36:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.718+0000 7f947f400640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94640744b0 0x7f94640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.718+0000 7f947f400640 1 -- 172.21.15.19:0/1172782562 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f9480095730 con 0x7f948815f910 2024-04-25T11:36:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.719+0000 7f9486a00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94640744b0 0x7f94640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.720+0000 7f9486a00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94640744b0 0x7f94640769c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9470005b50 tx=0x7f94700015b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:53.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.723+0000 7f947f400640 1 -- 172.21.15.19:0/1172782562 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9480024020 con 0x7f948815f910 2024-04-25T11:36:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.926+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94880cfb10 con 0x7f948815f910 2024-04-25T11:36:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.927+0000 7f947f400640 1 -- 172.21.15.19:0/1172782562 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+17575 (secure 0 0 0) 0x7f948005fbc0 con 0x7f948815f910 2024-04-25T11:36:53.915 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:36:53.916 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":99,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:36:41.348259+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94640744b0 msgr2=0x7f94640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94640744b0 0x7f94640769c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9470005b50 tx=0x7f94700015b0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948815f910 msgr2=0x7f948815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948815f910 0x7f948815fd20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f94880b6ca0 tx=0x7f948000b480 comp rx=0 tx=0).stop 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 shutdown_connections 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f94640744b0 0x7f94640769c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f948815f910 0x7f948815fd20 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f948815b840 0x7f94880cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 --2- 172.21.15.19:0/1172782562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f946c004800 0x7f9488135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 >> 172.21.15.19:0/1172782562 conn(0x7f94880d1720 msgr2=0x7f9488073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 shutdown_connections 2024-04-25T11:36:53.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:53.930+0000 7f9487e00640 1 -- 172.21.15.19:0/1172782562 wait complete. 2024-04-25T11:36:53.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:36:53.946 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:36:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.243+0000 7fcca8600640 1 Processor -- start 2024-04-25T11:36:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.243+0000 7fcca8600640 1 -- start start 2024-04-25T11:36:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.245+0000 7fcca8600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcca0161710 0x7fcca015e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0163fa0 0x7fcca01643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca8600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca01609b0 con 0x7fcca006c5c0 2024-04-25T11:36:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca8600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcca0164cd0 con 0x7fcca0161710 2024-04-25T11:36:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcca0160c30 con 0x7fcca0163fa0 2024-04-25T11:36:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcca0161710 0x7fcca015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca5200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcca006c5c0 0x7fcca006c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46748/0 (socket says 172.21.15.19:46748) 2024-04-25T11:36:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca5200640 1 -- 172.21.15.19:0/1611470037 learned_addr learned my addr 172.21.15.19:0/1611470037 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:54.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.246+0000 7fcca5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcca0161710 0x7fcca015e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36530/0 (socket says 172.21.15.19:36530) 2024-04-25T11:36:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.247+0000 7fcca5c00640 1 -- 172.21.15.19:0/1611470037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0163fa0 msgr2=0x7fcca01643b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:36:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.247+0000 7fcca5c00640 1 --2- 172.21.15.19:0/1611470037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0163fa0 0x7fcca01643b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.247+0000 7fcca5c00640 1 -- 172.21.15.19:0/1611470037 >> v1:172.21.15.19:6789/0 conn(0x7fcca006c5c0 legacy=0x7fcca006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.247+0000 7fcca5c00640 1 -- 172.21.15.19:0/1611470037 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca0160f10 con 0x7fcca0161710 2024-04-25T11:36:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.248+0000 7fcca5c00640 1 --2- 172.21.15.19:0/1611470037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0161710 0x7fcca015e080 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097a0 tx=0x7fcc9402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec6f274d395aeb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.248+0000 7fcc9fe00640 1 -- 172.21.15.19:0/1611470037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94037070 con 0x7fcca0161710 2024-04-25T11:36:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.248+0000 7fcc9fe00640 1 -- 172.21.15.19:0/1611470037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc94032440 con 0x7fcca0161710 2024-04-25T11:36:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.248+0000 7fcc9fe00640 1 -- 172.21.15.19:0/1611470037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9402ae60 con 0x7fcca0161710 2024-04-25T11:36:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 -- 172.21.15.19:0/1611470037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0161710 msgr2=0x7fcca015e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 --2- 172.21.15.19:0/1611470037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0161710 0x7fcca015e080 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097a0 tx=0x7fcc9402a110 comp rx=0 tx=0).stop 2024-04-25T11:36:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 -- 172.21.15.19:0/1611470037 shutdown_connections 2024-04-25T11:36:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 --2- 172.21.15.19:0/1611470037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0163fa0 0x7fcca01643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 --2- 172.21.15.19:0/1611470037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0161710 0x7fcca015e080 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 -- 172.21.15.19:0/1611470037 >> 172.21.15.19:0/1611470037 conn(0x7fcca00d5d90 msgr2=0x7fcca00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 -- 172.21.15.19:0/1611470037 shutdown_connections 2024-04-25T11:36:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.249+0000 7fcca8600640 1 -- 172.21.15.19:0/1611470037 wait complete. 2024-04-25T11:36:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.252+0000 7fcca8600640 1 Processor -- start 2024-04-25T11:36:54.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.252+0000 7fcca8600640 1 -- start start 2024-04-25T11:36:54.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.253+0000 7fcca8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcca006c5c0 0x7fcca00ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.253+0000 7fcca8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0161710 0x7fcca00ba160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.253+0000 7fcca8600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0163fa0 0x7fcca00ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.253+0000 7fcca8600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcca0164a70 con 0x7fcca0163fa0 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.253+0000 7fcca8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcca01648f0 con 0x7fcca006c5c0 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.253+0000 7fcca8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcca00d49f0 con 0x7fcca0161710 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0163fa0 0x7fcca00ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0163fa0 0x7fcca00ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36540/0 (socket says 172.21.15.19:36540) 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 -- 172.21.15.19:0/2407629682 learned_addr learned my addr 172.21.15.19:0/2407629682 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca5c00640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcca006c5c0 0x7fcca00ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca5200640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0161710 0x7fcca00ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 -- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0161710 msgr2=0x7fcca00ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0161710 0x7fcca00ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 -- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcca006c5c0 msgr2=0x7fcca00ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcca006c5c0 0x7fcca00ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.254+0000 7fcca6600640 1 -- 172.21.15.19:0/2407629682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca00ca0b0 con 0x7fcca0163fa0 2024-04-25T11:36:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.255+0000 7fcca6600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0163fa0 0x7fcca00ba6a0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800b410 tx=0x7fcc9800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.255+0000 7fcc9d600640 1 -- 172.21.15.19:0/2407629682 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98015020 con 0x7fcca0163fa0 2024-04-25T11:36:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.255+0000 7fcc9d600640 1 -- 172.21.15.19:0/2407629682 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc98011850 con 0x7fcca0163fa0 2024-04-25T11:36:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.255+0000 7fcc9d600640 1 -- 172.21.15.19:0/2407629682 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98011c40 con 0x7fcca0163fa0 2024-04-25T11:36:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.255+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca00cb3f0 con 0x7fcca0163fa0 2024-04-25T11:36:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.255+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca00ca630 con 0x7fcca0163fa0 2024-04-25T11:36:54.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.256+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca0050740 con 0x7fcca0163fa0 2024-04-25T11:36:54.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.258+0000 7fcc9d600640 1 -- 172.21.15.19:0/2407629682 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcc98018020 con 0x7fcca0163fa0 2024-04-25T11:36:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.259+0000 7fcc9d600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc840744b0 0x7fcc840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.259+0000 7fcc9d600640 1 -- 172.21.15.19:0/2407629682 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7fcc98095280 con 0x7fcca0163fa0 2024-04-25T11:36:54.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.260+0000 7fcca5c00640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc840744b0 0x7fcc840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.261+0000 7fcca5c00640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc840744b0 0x7fcc840769c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc94034000 tx=0x7fcc9402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:54.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.264+0000 7fcc9d600640 1 -- 172.21.15.19:0/2407629682 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc9801f020 con 0x7fcca0163fa0 2024-04-25T11:36:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.470+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcca015ff00 con 0x7fcca0163fa0 2024-04-25T11:36:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.471+0000 7fcc9d600640 1 -- 172.21.15.19:0/2407629682 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fcca00ca0b0 con 0x7fcca0163fa0 2024-04-25T11:36:54.458 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:36:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.473+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc840744b0 msgr2=0x7fcc840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.473+0000 7fcca8600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc840744b0 0x7fcc840769c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc94034000 tx=0x7fcc9402e000 comp rx=0 tx=0).stop 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.473+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0163fa0 msgr2=0x7fcca00ba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.473+0000 7fcca8600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0163fa0 0x7fcca00ba6a0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800b410 tx=0x7fcc9800b8d0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 shutdown_connections 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcc840744b0 0x7fcc840769c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcca0163fa0 0x7fcca00ba6a0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcca0161710 0x7fcca00ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 --2- 172.21.15.19:0/2407629682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcca006c5c0 0x7fcca00ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 >> 172.21.15.19:0/2407629682 conn(0x7fcca00d5d90 msgr2=0x7fcca006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 shutdown_connections 2024-04-25T11:36:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.474+0000 7fcca8600640 1 -- 172.21.15.19:0/2407629682 wait complete. 2024-04-25T11:36:54.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:36:54.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.793+0000 7fd47fe00640 1 Processor -- start 2024-04-25T11:36:54.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.793+0000 7fd47fe00640 1 -- start start 2024-04-25T11:36:54.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.795+0000 7fd47fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd48009ccc0 0x7fd48006d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.795+0000 7fd47fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd48006ddf0 0x7fd4801646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.795+0000 7fd47fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4800d31d0 con 0x7fd480164c00 2024-04-25T11:36:54.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.795+0000 7fd47fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd4800d34b0 con 0x7fd48006ddf0 2024-04-25T11:36:54.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.795+0000 7fd47fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800d3790 con 0x7fd48009ccc0 2024-04-25T11:36:54.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.796+0000 7fd47f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd480164c00 0x7fd4800d1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46760/0 (socket says 172.21.15.19:46760) 2024-04-25T11:36:54.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.796+0000 7fd47f400640 1 -- 172.21.15.19:0/1343104566 learned_addr learned my addr 172.21.15.19:0/1343104566 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.796+0000 7fd47ea00640 1 --2- 172.21.15.19:0/1343104566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd48009ccc0 0x7fd48006d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.796+0000 7fd47e000640 1 --2- 172.21.15.19:0/1343104566 >> v2:172.21.15.169:3300/0 conn(0x7fd48006ddf0 0x7fd4801646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.797+0000 7fd47d600640 1 -- 172.21.15.19:0/1343104566 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117063483 0 0) 0x7fd4800d31d0 con 0x7fd480164c00 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.797+0000 7fd47d600640 1 -- 172.21.15.19:0/1343104566 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd464003520 con 0x7fd480164c00 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.797+0000 7fd47d600640 1 -- 172.21.15.19:0/1343104566 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3386710343 0 0) 0x7fd464003520 con 0x7fd480164c00 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.797+0000 7fd47ea00640 1 -- 172.21.15.19:0/1343104566 >> v2:172.21.15.169:3300/0 conn(0x7fd48006ddf0 msgr2=0x7fd4801646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.797+0000 7fd47ea00640 1 --2- 172.21.15.19:0/1343104566 >> v2:172.21.15.169:3300/0 conn(0x7fd48006ddf0 0x7fd4801646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.797+0000 7fd47ea00640 1 -- 172.21.15.19:0/1343104566 >> v1:172.21.15.19:6789/0 conn(0x7fd480164c00 legacy=0x7fd4800d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.797+0000 7fd47ea00640 1 -- 172.21.15.19:0/1343104566 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4800ce0f0 con 0x7fd48009ccc0 2024-04-25T11:36:54.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.798+0000 7fd47ea00640 1 --2- 172.21.15.19:0/1343104566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd48009ccc0 0x7fd48006d8b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd468009790 tx=0x7fd46802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37688cd20448e6d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:54.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.799+0000 7fd47d600640 1 -- 172.21.15.19:0/1343104566 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468037070 con 0x7fd48009ccc0 2024-04-25T11:36:54.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.799+0000 7fd47d600640 1 -- 172.21.15.19:0/1343104566 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46802ab10 con 0x7fd48009ccc0 2024-04-25T11:36:54.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.799+0000 7fd47d600640 1 -- 172.21.15.19:0/1343104566 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478001670 con 0x7fd48009ccc0 2024-04-25T11:36:54.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 -- 172.21.15.19:0/1343104566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd48009ccc0 msgr2=0x7fd48006d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 --2- 172.21.15.19:0/1343104566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd48009ccc0 0x7fd48006d8b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd468009790 tx=0x7fd46802a080 comp rx=0 tx=0).stop 2024-04-25T11:36:54.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 -- 172.21.15.19:0/1343104566 shutdown_connections 2024-04-25T11:36:54.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 --2- 172.21.15.19:0/1343104566 >> v2:172.21.15.169:3300/0 conn(0x7fd48006ddf0 0x7fd4801646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 --2- 172.21.15.19:0/1343104566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd48009ccc0 0x7fd48006d8b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 -- 172.21.15.19:0/1343104566 >> 172.21.15.19:0/1343104566 conn(0x7fd4800d0480 msgr2=0x7fd4800d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:54.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 -- 172.21.15.19:0/1343104566 shutdown_connections 2024-04-25T11:36:54.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.800+0000 7fd47fe00640 1 -- 172.21.15.19:0/1343104566 wait complete. 2024-04-25T11:36:54.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.803+0000 7fd47fe00640 1 Processor -- start 2024-04-25T11:36:54.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.803+0000 7fd47fe00640 1 -- start start 2024-04-25T11:36:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.805+0000 7fd47fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48006ddf0 0x7fd480151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.805+0000 7fd47fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd48009ccc0 0x7fd480155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.805+0000 7fd47fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd480164c00 0x7fd4800c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.805+0000 7fd47fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800d1190 con 0x7fd48006ddf0 2024-04-25T11:36:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.805+0000 7fd47fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800d1010 con 0x7fd48009ccc0 2024-04-25T11:36:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.805+0000 7fd47fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800cf900 con 0x7fd480164c00 2024-04-25T11:36:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48006ddf0 0x7fd480151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48006ddf0 0x7fd480151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36554/0 (socket says 172.21.15.19:36554) 2024-04-25T11:36:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 -- 172.21.15.19:0/2559761736 learned_addr learned my addr 172.21.15.19:0/2559761736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47e000640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd48009ccc0 0x7fd480155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 -- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd480164c00 msgr2=0x7fd4800c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd480164c00 0x7fd4800c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 -- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd48009ccc0 msgr2=0x7fd480155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd48009ccc0 0x7fd480155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.806+0000 7fd47ea00640 1 -- 172.21.15.19:0/2559761736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd480155680 con 0x7fd48006ddf0 2024-04-25T11:36:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.807+0000 7fd47ea00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48006ddf0 0x7fd480151f80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd468035040 tx=0x7fd4680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:54.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.807+0000 7fd476a00640 1 -- 172.21.15.19:0/2559761736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468037040 con 0x7fd48006ddf0 2024-04-25T11:36:54.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.807+0000 7fd476a00640 1 -- 172.21.15.19:0/2559761736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd468005b90 con 0x7fd48006ddf0 2024-04-25T11:36:54.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.807+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4800c9c40 con 0x7fd48006ddf0 2024-04-25T11:36:54.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.807+0000 7fd476a00640 1 -- 172.21.15.19:0/2559761736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468006db0 con 0x7fd48006ddf0 2024-04-25T11:36:54.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.807+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4800ca200 con 0x7fd48006ddf0 2024-04-25T11:36:54.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.808+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4800d17b0 con 0x7fd48006ddf0 2024-04-25T11:36:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.810+0000 7fd476a00640 1 -- 172.21.15.19:0/2559761736 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd468006030 con 0x7fd48006ddf0 2024-04-25T11:36:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.811+0000 7fd476a00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd45c074500 0x7fd45c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.811+0000 7fd476a00640 1 -- 172.21.15.19:0/2559761736 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7fd4680b62c0 con 0x7fd48006ddf0 2024-04-25T11:36:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.811+0000 7fd47e000640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd45c074500 0x7fd45c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.813+0000 7fd47e000640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd45c074500 0x7fd45c076a10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd4800cb340 tx=0x7fd47000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:54.816+0000 7fd476a00640 1 -- 172.21.15.19:0/2559761736 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4800d17b0 con 0x7fd48006ddf0 2024-04-25T11:36:55.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.026+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd480076c60 con 0x7fd48006ddf0 2024-04-25T11:36:55.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.028+0000 7fd476a00640 1 -- 172.21.15.19:0/2559761736 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fd480155680 con 0x7fd48006ddf0 2024-04-25T11:36:55.015 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:36:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.030+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd45c074500 msgr2=0x7fd45c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.030+0000 7fd47fe00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd45c074500 0x7fd45c076a10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd4800cb340 tx=0x7fd47000b040 comp rx=0 tx=0).stop 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.030+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48006ddf0 msgr2=0x7fd480151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.030+0000 7fd47fe00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48006ddf0 0x7fd480151f80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fd468035040 tx=0x7fd4680046c0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 shutdown_connections 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd45c074500 0x7fd45c076a10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd48009ccc0 0x7fd480155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd480164c00 0x7fd4800c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 --2- 172.21.15.19:0/2559761736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd48006ddf0 0x7fd480151f80 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 >> 172.21.15.19:0/2559761736 conn(0x7fd4800d0480 msgr2=0x7fd480163c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 shutdown_connections 2024-04-25T11:36:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.031+0000 7fd47fe00640 1 -- 172.21.15.19:0/2559761736 wait complete. 2024-04-25T11:36:55.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:36:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.351+0000 7fb9af200640 1 Processor -- start 2024-04-25T11:36:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.351+0000 7fb9af200640 1 -- start start 2024-04-25T11:36:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.353+0000 7fb9af200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a80d1f80 0x7fb9a80d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.353+0000 7fb9af200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb9a809cab0 0x7fb9a809cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.353+0000 7fb9af200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9a80ce990 0x7fb9a80ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.353+0000 7fb9af200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80d2ce0 con 0x7fb9a809cab0 2024-04-25T11:36:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.353+0000 7fb9af200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80b2fd0 con 0x7fb9a80ce990 2024-04-25T11:36:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.353+0000 7fb9af200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a806e780 con 0x7fb9a80d1f80 2024-04-25T11:36:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb9a809cab0 0x7fb9a809cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb9a809cab0 0x7fb9a809cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36562/0 (socket says 172.21.15.19:36562) 2024-04-25T11:36:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 -- 172.21.15.19:0/1905773566 learned_addr learned my addr 172.21.15.19:0/1905773566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 -- 172.21.15.19:0/1905773566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a80d1f80 msgr2=0x7fb9a80d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 --2- 172.21.15.19:0/1905773566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a80d1f80 0x7fb9a80d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 -- 172.21.15.19:0/1905773566 >> v2:172.21.15.169:3300/0 conn(0x7fb9a80ce990 msgr2=0x7fb9a80ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 --2- 172.21.15.19:0/1905773566 >> v2:172.21.15.169:3300/0 conn(0x7fb9a80ce990 0x7fb9a80ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.354+0000 7fb9ad400640 1 -- 172.21.15.19:0/1905773566 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80d2970 con 0x7fb9a809cab0 2024-04-25T11:36:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.356+0000 7fb9ad400640 1 --2- 172.21.15.19:0/1905773566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a809cab0 0x7fb9a809cec0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80d1290 tx=0x7fb99802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc932e03531aecba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.356+0000 7fb9aca00640 1 -- 172.21.15.19:0/1905773566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998037070 con 0x7fb9a809cab0 2024-04-25T11:36:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.356+0000 7fb9aca00640 1 -- 172.21.15.19:0/1905773566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb998032440 con 0x7fb9a809cab0 2024-04-25T11:36:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.356+0000 7fb9aca00640 1 -- 172.21.15.19:0/1905773566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998032810 con 0x7fb9a809cab0 2024-04-25T11:36:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.356+0000 7fb9af200640 1 -- 172.21.15.19:0/1905773566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a809cab0 msgr2=0x7fb9a809cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.356+0000 7fb9af200640 1 --2- 172.21.15.19:0/1905773566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a809cab0 0x7fb9a809cec0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80d1290 tx=0x7fb99802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.357+0000 7fb9af200640 1 -- 172.21.15.19:0/1905773566 shutdown_connections 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.357+0000 7fb9af200640 1 --2- 172.21.15.19:0/1905773566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a809cab0 0x7fb9a809cec0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.357+0000 7fb9af200640 1 --2- 172.21.15.19:0/1905773566 >> v2:172.21.15.169:3300/0 conn(0x7fb9a80ce990 0x7fb9a80ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.357+0000 7fb9af200640 1 --2- 172.21.15.19:0/1905773566 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a80d1f80 0x7fb9a80d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.357+0000 7fb9af200640 1 -- 172.21.15.19:0/1905773566 >> 172.21.15.19:0/1905773566 conn(0x7fb9a80d03b0 msgr2=0x7fb9a80d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.357+0000 7fb9af200640 1 -- 172.21.15.19:0/1905773566 shutdown_connections 2024-04-25T11:36:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.357+0000 7fb9af200640 1 -- 172.21.15.19:0/1905773566 wait complete. 2024-04-25T11:36:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.359+0000 7fb9af200640 1 Processor -- start 2024-04-25T11:36:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.359+0000 7fb9af200640 1 -- start start 2024-04-25T11:36:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9af200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a809cab0 0x7fb9a80ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9af200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a80ce990 0x7fb9a81daa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9af200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a80d1f80 0x7fb9a80b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9af200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80b9480 con 0x7fb9a80d1f80 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9af200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80b9300 con 0x7fb9a80ce990 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9af200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a80b9760 con 0x7fb9a809cab0 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9ae800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a80d1f80 0x7fb9a80b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9ae800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a80d1f80 0x7fb9a80b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36572/0 (socket says 172.21.15.19:36572) 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.361+0000 7fb9ae800640 1 -- 172.21.15.19:0/1695938516 learned_addr learned my addr 172.21.15.19:0/1695938516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ad400640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a80ce990 0x7fb9a81daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ade00640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a809cab0 0x7fb9a80ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ae800640 1 -- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a809cab0 msgr2=0x7fb9a80ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ae800640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a809cab0 0x7fb9a80ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ae800640 1 -- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a80ce990 msgr2=0x7fb9a81daa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ae800640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a80ce990 0x7fb9a81daa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ae800640 1 -- 172.21.15.19:0/1695938516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80ca7b0 con 0x7fb9a80d1f80 2024-04-25T11:36:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ad400640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a80ce990 0x7fb9a81daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9ae800640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a80d1f80 0x7fb9a80b75a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ba10 tx=0x7fb9a400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.362+0000 7fb9a2000640 1 -- 172.21.15.19:0/1695938516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4019070 con 0x7fb9a80d1f80 2024-04-25T11:36:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.363+0000 7fb9a2000640 1 -- 172.21.15.19:0/1695938516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a400cb40 con 0x7fb9a80d1f80 2024-04-25T11:36:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.363+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a80cc010 con 0x7fb9a80d1f80 2024-04-25T11:36:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.363+0000 7fb9a2000640 1 -- 172.21.15.19:0/1695938516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4014440 con 0x7fb9a80d1f80 2024-04-25T11:36:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.363+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a80c87c0 con 0x7fb9a80d1f80 2024-04-25T11:36:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.364+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a80c8530 con 0x7fb9a80d1f80 2024-04-25T11:36:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.365+0000 7fb9a2000640 1 -- 172.21.15.19:0/1695938516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb9a401c070 con 0x7fb9a80d1f80 2024-04-25T11:36:55.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.366+0000 7fb9a2000640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb98c0744b0 0x7fb98c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.366+0000 7fb9a2000640 1 -- 172.21.15.19:0/1695938516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7fb9a4095180 con 0x7fb9a80d1f80 2024-04-25T11:36:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.367+0000 7fb9ade00640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb98c0744b0 0x7fb98c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.368+0000 7fb9ade00640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb98c0744b0 0x7fb98c0769c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb99c006fd0 tx=0x7fb99c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:55.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.371+0000 7fb9a2000640 1 -- 172.21.15.19:0/1695938516 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9a80ca7b0 con 0x7fb9a80d1f80 2024-04-25T11:36:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.576+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9a8076c60 con 0x7fb9a80d1f80 2024-04-25T11:36:55.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.577+0000 7fb9a2000640 1 -- 172.21.15.19:0/1695938516 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fb9a405f6a0 con 0x7fb9a80d1f80 2024-04-25T11:36:55.565 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:36:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb98c0744b0 msgr2=0x7fb98c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb98c0744b0 0x7fb98c0769c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb99c006fd0 tx=0x7fb99c008040 comp rx=0 tx=0).stop 2024-04-25T11:36:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a80d1f80 msgr2=0x7fb9a80b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a80d1f80 0x7fb9a80b75a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400ba10 tx=0x7fb9a400bed0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 shutdown_connections 2024-04-25T11:36:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb98c0744b0 0x7fb98c0769c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9a80ce990 0x7fb9a81daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9a80d1f80 0x7fb9a80b75a0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 --2- 172.21.15.19:0/1695938516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9a809cab0 0x7fb9a80ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 >> 172.21.15.19:0/1695938516 conn(0x7fb9a80d03b0 msgr2=0x7fb9a80b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.580+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 shutdown_connections 2024-04-25T11:36:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.581+0000 7fb9af200640 1 -- 172.21.15.19:0/1695938516 wait complete. 2024-04-25T11:36:55.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:36:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.891+0000 7f8ad9e00640 1 Processor -- start 2024-04-25T11:36:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.891+0000 7f8ad9e00640 1 -- start start 2024-04-25T11:36:55.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ad406c5c0 0x7f8ad40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad40b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad9e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ad40b4cd0 0x7f8ad4161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad9e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40b50e0 con 0x7f8ad406c5c0 2024-04-25T11:36:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad9e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40d1230 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad40d06b0 con 0x7f8ad4144370 2024-04-25T11:36:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ad406c5c0 0x7f8ad40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ad406c5c0 0x7f8ad40cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36576/0 (socket says 172.21.15.19:36576) 2024-04-25T11:36:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.894+0000 7f8ad8a00640 1 -- 172.21.15.19:0/854768194 learned_addr learned my addr 172.21.15.19:0/854768194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:55.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.895+0000 7f8ad9400640 1 --2- 172.21.15.19:0/854768194 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40b4cd0 0x7f8ad4161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.895+0000 7f8ad3e00640 1 --2- 172.21.15.19:0/854768194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad40b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.895+0000 7f8ad8a00640 1 -- 172.21.15.19:0/854768194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 msgr2=0x7f8ad40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:55.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.895+0000 7f8ad8a00640 1 --2- 172.21.15.19:0/854768194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad40b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.895+0000 7f8ad8a00640 1 -- 172.21.15.19:0/854768194 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40b4cd0 msgr2=0x7f8ad4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.895+0000 7f8ad8a00640 1 --2- 172.21.15.19:0/854768194 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40b4cd0 0x7f8ad4161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.895+0000 7f8ad8a00640 1 -- 172.21.15.19:0/854768194 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40d0990 con 0x7f8ad406c5c0 2024-04-25T11:36:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.896+0000 7f8ad8a00640 1 --2- 172.21.15.19:0/854768194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad406c5c0 0x7f8ad40cfa70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40b6cd0 tx=0x7f8ac802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3b2f84b06de1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.896+0000 7f8ad3400640 1 -- 172.21.15.19:0/854768194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8037070 con 0x7f8ad406c5c0 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.896+0000 7f8ad3400640 1 -- 172.21.15.19:0/854768194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac8032440 con 0x7f8ad406c5c0 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.896+0000 7f8ad3400640 1 -- 172.21.15.19:0/854768194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8032810 con 0x7f8ad406c5c0 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 -- 172.21.15.19:0/854768194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad406c5c0 msgr2=0x7f8ad40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/854768194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad406c5c0 0x7f8ad40cfa70 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40b6cd0 tx=0x7f8ac802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 -- 172.21.15.19:0/854768194 shutdown_connections 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/854768194 >> v2:172.21.15.169:3300/0 conn(0x7f8ad40b4cd0 0x7f8ad4161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/854768194 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/854768194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad406c5c0 0x7f8ad40cfa70 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 -- 172.21.15.19:0/854768194 >> 172.21.15.19:0/854768194 conn(0x7f8ad40d1720 msgr2=0x7f8ad40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 -- 172.21.15.19:0/854768194 shutdown_connections 2024-04-25T11:36:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.897+0000 7f8ad9e00640 1 -- 172.21.15.19:0/854768194 wait complete. 2024-04-25T11:36:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.900+0000 7f8ad9e00640 1 Processor -- start 2024-04-25T11:36:55.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.900+0000 7f8ad9e00640 1 -- start start 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.901+0000 7f8ad9e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad406c5c0 0x7f8ad413b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.901+0000 7f8ad9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad40b4cd0 0x7f8ad415f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.901+0000 7f8ad9e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.901+0000 7f8ad9e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40d02d0 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.901+0000 7f8ad9e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40d0f50 con 0x7f8ad406c5c0 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.901+0000 7f8ad9e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad40d0450 con 0x7f8ad4144370 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad40b4cd0 0x7f8ad415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad40b4cd0 0x7f8ad415f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36588/0 (socket says 172.21.15.19:36588) 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 -- 172.21.15.19:0/109440411 learned_addr learned my addr 172.21.15.19:0/109440411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad8a00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad406c5c0 0x7f8ad413b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad9400640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 -- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 msgr2=0x7f8ad415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad415fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 -- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad406c5c0 msgr2=0x7f8ad413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad406c5c0 0x7f8ad413b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.902+0000 7f8ad3e00640 1 -- 172.21.15.19:0/109440411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad4146250 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.903+0000 7f8ad3e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad40b4cd0 0x7f8ad415f910 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000d5f0 tx=0x7f8ac000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.903+0000 7f8ad0c00640 1 -- 172.21.15.19:0/109440411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac00125f0 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.903+0000 7f8ad0c00640 1 -- 172.21.15.19:0/109440411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac0019070 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.903+0000 7f8ad0c00640 1 -- 172.21.15.19:0/109440411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac00129d0 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.903+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad4160570 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.903+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad4160b30 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.905+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad4143a00 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.906+0000 7f8ad0c00640 1 -- 172.21.15.19:0/109440411 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8ac0012b70 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.907+0000 7f8ad0c00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ab40744b0 0x7f8ab40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.907+0000 7f8ad0c00640 1 -- 172.21.15.19:0/109440411 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7f8ac0095860 con 0x7f8ad40b4cd0 2024-04-25T11:36:55.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.907+0000 7f8ad8a00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ab40744b0 0x7f8ab40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:55.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.909+0000 7f8ad8a00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ab40744b0 0x7f8ab40769c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8034000 tx=0x7f8ac802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:55.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:55.911+0000 7f8ad0c00640 1 -- 172.21.15.19:0/109440411 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ad4143a00 con 0x7f8ad40b4cd0 2024-04-25T11:36:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.131+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8ad4145fe0 con 0x7f8ad40b4cd0 2024-04-25T11:36:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.132+0000 7f8ad0c00640 1 -- 172.21.15.19:0/109440411 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v99) ==== 81+0+848 (secure 0 0 0) 0x7f8ac005fd90 con 0x7f8ad40b4cd0 2024-04-25T11:36:56.119 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:36:56.119 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:36:56.120 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:36:56.121 INFO:teuthology.orchestra.run.smithi019.stderr:listed 14 entries 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.134+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ab40744b0 msgr2=0x7f8ab40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.134+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ab40744b0 0x7f8ab40769c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8034000 tx=0x7f8ac802e000 comp rx=0 tx=0).stop 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.134+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad40b4cd0 msgr2=0x7f8ad415f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.134+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad40b4cd0 0x7f8ad415f910 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000d5f0 tx=0x7f8ac000dab0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 shutdown_connections 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ab40744b0 0x7f8ab40769c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ad4144370 0x7f8ad415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad40b4cd0 0x7f8ad415f910 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 --2- 172.21.15.19:0/109440411 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad406c5c0 0x7f8ad413b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 >> 172.21.15.19:0/109440411 conn(0x7f8ad40d1720 msgr2=0x7f8ad4068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 shutdown_connections 2024-04-25T11:36:56.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.135+0000 7f8ad9e00640 1 -- 172.21.15.19:0/109440411 wait complete. 2024-04-25T11:36:56.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:36:56.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:56.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:36:56.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:36:56.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:36:56.262 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:36:56.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:36:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.610+0000 7fa580c00640 1 Processor -- start 2024-04-25T11:36:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.610+0000 7fa580c00640 1 -- start start 2024-04-25T11:36:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.612+0000 7fa580c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa57c167000 0x7fa57c169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.612+0000 7fa580c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c15f5f0 con 0x7fa57c1443c0 2024-04-25T11:36:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.613+0000 7fa580c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c15e830 con 0x7fa57c0cf5e0 2024-04-25T11:36:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.613+0000 7fa580c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa57c0708d0 con 0x7fa57c167000 2024-04-25T11:36:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.613+0000 7fa579e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa57c1443c0 0x7fa57c06fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46776/0 (socket says 172.21.15.19:46776) 2024-04-25T11:36:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.613+0000 7fa579e00640 1 -- 172.21.15.19:0/4278576113 learned_addr learned my addr 172.21.15.19:0/4278576113 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.613+0000 7fa57a800640 1 --2- 172.21.15.19:0/4278576113 >> v2:172.21.15.169:3300/0 conn(0x7fa57c167000 0x7fa57c169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.613+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139621832 0 0) 0x7fa57c15f5f0 con 0x7fa57c1443c0 2024-04-25T11:36:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.613+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa560003520 con 0x7fa57c1443c0 2024-04-25T11:36:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1940458214 0 0) 0x7fa560003520 con 0x7fa57c1443c0 2024-04-25T11:36:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 >> v2:172.21.15.169:3300/0 conn(0x7fa57c167000 msgr2=0x7fa57c169420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 --2- 172.21.15.19:0/4278576113 >> v2:172.21.15.169:3300/0 conn(0x7fa57c167000 0x7fa57c169420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 >> v1:172.21.15.169:6789/0 conn(0x7fa57c0cf5e0 legacy=0x7fa57c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c15eb10 con 0x7fa57c1443c0 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa56c001670 con 0x7fa57c1443c0 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 >> v1:172.21.15.19:6789/0 conn(0x7fa57c1443c0 legacy=0x7fa57c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5600039b0 0x7fa560003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa560004800 0x7fa560004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa560005270 0x7fa56000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c0708d0 con 0x7fa560004800 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa56000c1e0 con 0x7fa5600039b0 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa56000c3e0 con 0x7fa560005270 2024-04-25T11:36:56.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.614+0000 7fa573e00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa560004800 0x7fa560004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa573e00640 1 -- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa560005270 msgr2=0x7fa56000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa573e00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa560005270 0x7fa56000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa573e00640 1 -- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5600039b0 msgr2=0x7fa560003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa579e00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5600039b0 0x7fa560003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa573e00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5600039b0 0x7fa560003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa57a800640 1 -- 172.21.15.19:0/4278576113 reap_dead start 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa573e00640 1 -- 172.21.15.19:0/4278576113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa560003520 con 0x7fa560004800 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.615+0000 7fa579e00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5600039b0 0x7fa560003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.616+0000 7fa573e00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa560004800 0x7fa560004c10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa564001640 tx=0x7fa560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705699b249b08093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.616+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564001240 con 0x7fa560004800 2024-04-25T11:36:56.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.616+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56402e070 con 0x7fa560004800 2024-04-25T11:36:56.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.616+0000 7fa579400640 1 -- 172.21.15.19:0/4278576113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56402a7c0 con 0x7fa560004800 2024-04-25T11:36:56.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.617+0000 7fa580c00640 1 -- 172.21.15.19:0/4278576113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa560004800 msgr2=0x7fa560004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.617+0000 7fa580c00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa560004800 0x7fa560004c10 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa564001640 tx=0x7fa560005830 comp rx=0 tx=0).stop 2024-04-25T11:36:56.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.617+0000 7fa580c00640 1 -- 172.21.15.19:0/4278576113 shutdown_connections 2024-04-25T11:36:56.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.617+0000 7fa580c00640 1 --2- 172.21.15.19:0/4278576113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa560004800 0x7fa560004c10 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.617+0000 7fa580c00640 1 -- 172.21.15.19:0/4278576113 >> 172.21.15.19:0/4278576113 conn(0x7fa57c0d1ae0 msgr2=0x7fa57c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.617+0000 7fa580c00640 1 -- 172.21.15.19:0/4278576113 shutdown_connections 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.617+0000 7fa580c00640 1 -- 172.21.15.19:0/4278576113 wait complete. 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.620+0000 7fa580c00640 1 Processor -- start 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.620+0000 7fa580c00640 1 -- start start 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa580c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa560004800 0x7fa57c140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa580c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57c15b3a0 0x7fa57c1f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa580c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa57c15b7b0 0x7fa57c1f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa580c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c1447d0 con 0x7fa57c15b3a0 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa580c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c15f160 con 0x7fa560004800 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa580c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c162b00 con 0x7fa57c15b7b0 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa573e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57c15b3a0 0x7fa57c1f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:56.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.621+0000 7fa573e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57c15b3a0 0x7fa57c1f95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36602/0 (socket says 172.21.15.19:36602) 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa573e00640 1 -- 172.21.15.19:0/2860607192 learned_addr learned my addr 172.21.15.19:0/2860607192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa573e00640 1 -- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa57c15b7b0 msgr2=0x7fa57c1f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa573e00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa57c15b7b0 0x7fa57c1f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa579e00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa560004800 0x7fa57c140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa573e00640 1 -- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa560004800 msgr2=0x7fa57c140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa573e00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa560004800 0x7fa57c140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa573e00640 1 -- 172.21.15.19:0/2860607192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c0b0230 con 0x7fa57c15b3a0 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa579e00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa560004800 0x7fa57c140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa573e00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57c15b3a0 0x7fa57c1f95a0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa564001730 tx=0x7fa564000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.622+0000 7fa572000640 1 -- 172.21.15.19:0/2860607192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564043070 con 0x7fa57c15b3a0 2024-04-25T11:36:56.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.623+0000 7fa572000640 1 -- 172.21.15.19:0/2860607192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa564003610 con 0x7fa57c15b3a0 2024-04-25T11:36:56.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.623+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c141440 con 0x7fa57c15b3a0 2024-04-25T11:36:56.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.623+0000 7fa572000640 1 -- 172.21.15.19:0/2860607192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564007a50 con 0x7fa57c15b3a0 2024-04-25T11:36:56.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.623+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c141a00 con 0x7fa57c15b3a0 2024-04-25T11:36:56.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.624+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c1411b0 con 0x7fa57c15b3a0 2024-04-25T11:36:56.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.625+0000 7fa572000640 1 -- 172.21.15.19:0/2860607192 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa564033470 con 0x7fa57c15b3a0 2024-04-25T11:36:56.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.626+0000 7fa572000640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5580744e0 0x7fa5580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:56.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.626+0000 7fa572000640 1 -- 172.21.15.19:0/2860607192 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6311+0+0 (secure 0 0 0) 0x7fa5640b4870 con 0x7fa57c15b3a0 2024-04-25T11:36:56.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.627+0000 7fa579e00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5580744e0 0x7fa5580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:56.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.628+0000 7fa579e00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5580744e0 0x7fa5580769f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa56c000fd0 tx=0x7fa56c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.631+0000 7fa572000640 1 -- 172.21.15.19:0/2860607192 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa56407ed00 con 0x7fa57c15b3a0 2024-04-25T11:36:56.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.883+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa57c0aea80 con 0x7fa57c15b3a0 2024-04-25T11:36:56.872 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:36:56.885+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:36:56.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.890+0000 7fa572000640 1 -- 172.21.15.19:0/2860607192 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v140) ==== 161+0+0 (secure 0 0 0) 0x7fa56407e600 con 0x7fa57c15b3a0 2024-04-25T11:36:56.878 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5580744e0 msgr2=0x7fa5580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5580744e0 0x7fa5580769f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa56c000fd0 tx=0x7fa56c009040 comp rx=0 tx=0).stop 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57c15b3a0 msgr2=0x7fa57c1f95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57c15b3a0 0x7fa57c1f95a0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa564001730 tx=0x7fa564000fb0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 shutdown_connections 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5580744e0 0x7fa5580769f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa57c15b7b0 0x7fa57c1f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57c15b3a0 0x7fa57c1f95a0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 --2- 172.21.15.19:0/2860607192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa560004800 0x7fa57c140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 >> 172.21.15.19:0/2860607192 conn(0x7fa57c0d1ae0 msgr2=0x7fa57c15d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:56.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 shutdown_connections 2024-04-25T11:36:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:56.893+0000 7fa580c00640 1 -- 172.21.15.19:0/2860607192 wait complete. 2024-04-25T11:36:56.892 INFO:tasks.ceph.mds.a.smithi019.stderr: -823> 2024-04-25T11:36:47.371+0000 7f5e17000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:36:56.913 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:36:57.026 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:36:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.188+0000 7ff435400640 1 Processor -- start 2024-04-25T11:36:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.188+0000 7ff435400640 1 -- start start 2024-04-25T11:36:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.190+0000 7ff435400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff4300b8990 0x7ff4300cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.190+0000 7ff435400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4300b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.190+0000 7ff435400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4300d28c0 con 0x7ff4300b9050 2024-04-25T11:36:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.190+0000 7ff435400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff4300b8da0 con 0x7ff4300b8990 2024-04-25T11:36:57.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.190+0000 7ff435400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4300d2ba0 con 0x7ff4300b71d0 2024-04-25T11:36:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff42fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff4300b9050 0x7ff4300ce9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46786/0 (socket says 172.21.15.19:46786) 2024-04-25T11:36:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff42fe00640 1 -- 172.21.15.19:0/905304257 learned_addr learned my addr 172.21.15.19:0/905304257 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:36:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff427e00640 1 --2- 172.21.15.19:0/905304257 >> v2:172.21.15.19:3300/0 conn(0x7ff4300b8990 0x7ff4300cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff434a00640 1 --2- 172.21.15.19:0/905304257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4300b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff42f400640 1 -- 172.21.15.19:0/905304257 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020775789 0 0) 0x7ff4300d28c0 con 0x7ff4300b9050 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff42f400640 1 -- 172.21.15.19:0/905304257 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff414003520 con 0x7ff4300b9050 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff427e00640 1 -- 172.21.15.19:0/905304257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 msgr2=0x7ff4300b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff427e00640 1 --2- 172.21.15.19:0/905304257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4300b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff427e00640 1 -- 172.21.15.19:0/905304257 >> v1:172.21.15.19:6789/0 conn(0x7ff4300b9050 legacy=0x7ff4300ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff427e00640 1 -- 172.21.15.19:0/905304257 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43009cc50 con 0x7ff4300b8990 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.191+0000 7ff434a00640 1 --2- 172.21.15.19:0/905304257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4300b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:36:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.193+0000 7ff427e00640 1 --2- 172.21.15.19:0/905304257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b8990 0x7ff4300cf660 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff4180097a0 tx=0x7ff41802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f47049e01e4e377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.193+0000 7ff42f400640 1 -- 172.21.15.19:0/905304257 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418037070 con 0x7ff4300b8990 2024-04-25T11:36:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.193+0000 7ff42f400640 1 -- 172.21.15.19:0/905304257 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff418032440 con 0x7ff4300b8990 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.193+0000 7ff42f400640 1 -- 172.21.15.19:0/905304257 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180327f0 con 0x7ff4300b8990 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.193+0000 7ff435400640 1 -- 172.21.15.19:0/905304257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b8990 msgr2=0x7ff4300cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.193+0000 7ff435400640 1 --2- 172.21.15.19:0/905304257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b8990 0x7ff4300cf660 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff4180097a0 tx=0x7ff41802a180 comp rx=0 tx=0).stop 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.194+0000 7ff435400640 1 -- 172.21.15.19:0/905304257 shutdown_connections 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.194+0000 7ff435400640 1 --2- 172.21.15.19:0/905304257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4300b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.194+0000 7ff435400640 1 --2- 172.21.15.19:0/905304257 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b8990 0x7ff4300cf660 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.194+0000 7ff435400640 1 -- 172.21.15.19:0/905304257 >> 172.21.15.19:0/905304257 conn(0x7ff4300d03f0 msgr2=0x7ff4300d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.194+0000 7ff435400640 1 -- 172.21.15.19:0/905304257 shutdown_connections 2024-04-25T11:36:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.194+0000 7ff435400640 1 -- 172.21.15.19:0/905304257 wait complete. 2024-04-25T11:36:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.196+0000 7ff435400640 1 Processor -- start 2024-04-25T11:36:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.196+0000 7ff435400640 1 -- start start 2024-04-25T11:36:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff435400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4301d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff435400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4300b8990 0x7ff4301d5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff435400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b9050 0x7ff43009fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff435400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4300d2550 con 0x7ff4300b9050 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff435400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff4300cf0b0 con 0x7ff4300b8990 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff435400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff4300b7b20 con 0x7ff4300b71d0 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff434a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b9050 0x7ff43009fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff434a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b9050 0x7ff43009fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36630/0 (socket says 172.21.15.19:36630) 2024-04-25T11:36:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.198+0000 7ff434a00640 1 -- 172.21.15.19:0/2660661256 learned_addr learned my addr 172.21.15.19:0/2660661256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff42fe00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4301d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff427e00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4300b8990 0x7ff4301d5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff434a00640 1 -- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 msgr2=0x7ff4301d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff434a00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4301d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff434a00640 1 -- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4300b8990 msgr2=0x7ff4301d5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff434a00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4300b8990 0x7ff4301d5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff434a00640 1 -- 172.21.15.19:0/2660661256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4301d9110 con 0x7ff4300b9050 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff427e00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4300b8990 0x7ff4301d5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff434a00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b9050 0x7ff43009fe00 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff428007e20 tx=0x7ff42800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.199+0000 7ff42cc00640 1 -- 172.21.15.19:0/2660661256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42801a070 con 0x7ff4300b9050 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.200+0000 7ff42cc00640 1 -- 172.21.15.19:0/2660661256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428003b90 con 0x7ff4300b9050 2024-04-25T11:36:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.200+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4301f6a80 con 0x7ff4300b9050 2024-04-25T11:36:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.200+0000 7ff42cc00640 1 -- 172.21.15.19:0/2660661256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42800de80 con 0x7ff4300b9050 2024-04-25T11:36:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.200+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4301f7040 con 0x7ff4300b9050 2024-04-25T11:36:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.201+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff430050740 con 0x7ff4300b9050 2024-04-25T11:36:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.202+0000 7ff42cc00640 1 -- 172.21.15.19:0/2660661256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff42801d070 con 0x7ff4300b9050 2024-04-25T11:36:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.203+0000 7ff42cc00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff40c0744b0 0x7ff40c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.203+0000 7ff42cc00640 1 -- 172.21.15.19:0/2660661256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6397+0+0 (secure 0 0 0) 0x7ff428095770 con 0x7ff4300b9050 2024-04-25T11:36:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.204+0000 7ff42fe00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff40c0744b0 0x7ff40c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.205+0000 7ff42fe00640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff40c0744b0 0x7ff40c0769c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff420001360 tx=0x7ff420009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.208+0000 7ff42cc00640 1 -- 172.21.15.19:0/2660661256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff428010070 con 0x7ff4300b9050 2024-04-25T11:36:57.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.498+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff4301f6810 con 0x7ff4300b9050 2024-04-25T11:36:57.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.894+0000 7ff42cc00640 1 -- 172.21.15.19:0/2660661256 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v141) ==== 107+0+0 (secure 0 0 0) 0x7ff42805fc50 con 0x7ff4300b9050 2024-04-25T11:36:57.882 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff40c0744b0 msgr2=0x7ff40c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff40c0744b0 0x7ff40c0769c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff420001360 tx=0x7ff420009040 comp rx=0 tx=0).stop 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b9050 msgr2=0x7ff43009fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b9050 0x7ff43009fe00 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff428007e20 tx=0x7ff42800d4a0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 shutdown_connections 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff40c0744b0 0x7ff40c0769c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4300b8990 0x7ff4301d5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4300b9050 0x7ff43009fe00 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 --2- 172.21.15.19:0/2660661256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff4300b71d0 0x7ff4301d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 >> 172.21.15.19:0/2660661256 conn(0x7ff4300d03f0 msgr2=0x7ff4300720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 shutdown_connections 2024-04-25T11:36:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:57.898+0000 7ff435400640 1 -- 172.21.15.19:0/2660661256 wait complete. 2024-04-25T11:36:57.916 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:36:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.110+0000 7fe68431af40 1 Processor -- start 2024-04-25T11:36:58.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.110+0000 7fe68431af40 1 -- start start 2024-04-25T11:36:58.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe68431af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x56036173a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe68431af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x5603616a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe68431af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x5603616a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe68431af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5603616aee70 con 0x56036173fdd0 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe68431af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56036160cd20 con 0x5603616661e0 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe68431af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5603616af110 con 0x5603616665f0 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe67d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x5603616a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe67d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x5603616a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36648/0 (socket says 172.21.15.19:36648) 2024-04-25T11:36:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.112+0000 7fe67d400640 1 -- 172.21.15.19:0/1085102544 learned_addr learned my addr 172.21.15.19:0/1085102544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67c000640 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x5603616a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67ca00640 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x56036173a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67d400640 1 -- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 msgr2=0x5603616a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67d400640 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x5603616a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67d400640 1 -- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 msgr2=0x56036173a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67d400640 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x56036173a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67d400640 1 -- 172.21.15.19:0/1085102544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036173cd80 con 0x56036173fdd0 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67ca00640 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x56036173a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67d400640 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x5603616a86c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe674055040 tx=0x7fe674051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cc4653630a681d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.113+0000 7fe67ac00640 1 -- 172.21.15.19:0/1085102544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674053880 con 0x56036173fdd0 2024-04-25T11:36:58.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe67ac00640 1 -- 172.21.15.19:0/1085102544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe674053a20 con 0x56036173fdd0 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe67ac00640 1 -- 172.21.15.19:0/1085102544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674053e10 con 0x56036173fdd0 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 -- 172.21.15.19:0/1085102544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 msgr2=0x5603616a86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x5603616a86c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe674055040 tx=0x7fe674051000 comp rx=0 tx=0).stop 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 -- 172.21.15.19:0/1085102544 shutdown_connections 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x5603616a86c0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x5603616a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 --2- 172.21.15.19:0/1085102544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x56036173a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 -- 172.21.15.19:0/1085102544 >> 172.21.15.19:0/1085102544 conn(0x560361588ac0 msgr2=0x5603616b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 -- 172.21.15.19:0/1085102544 shutdown_connections 2024-04-25T11:36:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.114+0000 7fe68431af40 1 -- 172.21.15.19:0/1085102544 wait complete. 2024-04-25T11:36:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.115+0000 7fe68431af40 1 Processor -- start 2024-04-25T11:36:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.116+0000 7fe68431af40 1 -- start start 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe68431af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x5603617d3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe68431af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x56036173a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe68431af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x56036173a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe68431af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56036173ed80 con 0x56036173fdd0 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe68431af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56036173f060 con 0x5603616661e0 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe68431af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5603616a8f90 con 0x5603616665f0 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe67d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x56036173a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe67d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x56036173a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36662/0 (socket says 172.21.15.19:36662) 2024-04-25T11:36:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.117+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 learned_addr learned my addr 172.21.15.19:0/2613606739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x56036173a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 msgr2=0x56036173a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67ca00640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x5603617d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67d400640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x56036173a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 msgr2=0x5603617d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67d400640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5603616661e0 0x5603617d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603617c53b0 con 0x56036173fdd0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5603616665f0 0x56036173a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe67d400640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x56036173a540 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe674025d20 tx=0x7fe674002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe679800640 1 -- 172.21.15.19:0/2613606739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674053c50 con 0x56036173fdd0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe679800640 1 -- 172.21.15.19:0/2613606739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe674074070 con 0x56036173fdd0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe679800640 1 -- 172.21.15.19:0/2613606739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674067670 con 0x56036173fdd0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.118+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603617c5690 con 0x56036173fdd0 2024-04-25T11:36:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.119+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603616981c0 con 0x56036173fdd0 2024-04-25T11:36:58.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.121+0000 7fe679800640 1 -- 172.21.15.19:0/2613606739 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe674067810 con 0x56036173fdd0 2024-04-25T11:36:58.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.122+0000 7fe679800640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe65c072f10 0x7fe65c075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.122+0000 7fe679800640 1 -- 172.21.15.19:0/2613606739 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6397+0+0 (secure 0 0 0) 0x7fe6740da210 con 0x56036173fdd0 2024-04-25T11:36:58.109 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:36:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.122+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560361697650 0x560361697a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.122+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603616a8630 con 0x560361697650 2024-04-25T11:36:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.122+0000 7fe67ca00640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe65c072f10 0x7fe65c075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.123+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560361697650 0x560361697a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.124+0000 7fe67ca00640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe65c072f10 0x7fe65c075420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c530 tx=0x7fe66c005590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.124+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560361697650 0x560361697a60 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.127+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v100'226 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.127+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617c4ef0 con 0x560361697650 2024-04-25T11:36:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.130+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v100'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.131+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0460 con 0x560361697650 2024-04-25T11:36:58.121 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:36:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.134+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v100'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.134+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1810 con 0x560361697650 2024-04-25T11:36:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.137+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.137+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1bb0 con 0x560361697650 2024-04-25T11:36:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.141+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.141+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0800 con 0x560361697650 2024-04-25T11:36:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.145+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.145+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0ba0 con 0x560361697650 2024-04-25T11:36:58.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.148+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v100'232 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.148+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.152+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v100'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.152+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.143 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:36:58.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.155+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v100'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.155+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.159+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.159+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.163+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.163+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.167+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.167+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.171+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v100'238 uv213 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.171+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.175+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'239 uv239 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.175+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.166 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:36:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.179+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v100'240 uv240 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.179+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.183+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'241 uv241 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.183+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.187+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'242 uv242 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.187+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e0f40 con 0x560361697650 2024-04-25T11:36:58.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.191+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'243 uv243 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.191+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1340 con 0x560361697650 2024-04-25T11:36:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.195+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'244 uv244 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.195+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1320 con 0x560361697650 2024-04-25T11:36:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.199+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v100'245 uv219 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.199+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.203+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v100'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.203+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.195 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:36:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.207+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v100'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.208+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.211+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.212+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.215+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.216+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.220+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.220+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.224+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v100'251 uv225 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.224+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.228+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v100'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.228+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.220 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:36:58.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.232+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v100'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.232+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.236+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.236+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.240+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.240+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x560361697650 2024-04-25T11:36:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.245+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.245+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5603617e2460 0x5603617e4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.245+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e1200 con 0x5603617e2460 2024-04-25T11:36:58.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.246+0000 7fe67ca00640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5603617e2460 0x5603617e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.247+0000 7fe67ca00640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5603617e2460 0x5603617e4880 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.251+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'177 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.251+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e4dc0 con 0x5603617e2460 2024-04-25T11:36:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.256+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v100'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.256+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e5160 con 0x5603617e2460 2024-04-25T11:36:58.247 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:36:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.260+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v100'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.260+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e5500 con 0x5603617e2460 2024-04-25T11:36:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.264+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v100'180 uv180 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.264+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e58a0 con 0x5603617e2460 2024-04-25T11:36:58.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.268+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v100'181 uv181 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.268+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e5c40 con 0x5603617e2460 2024-04-25T11:36:58.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.272+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v100'182 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.272+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e5fe0 con 0x5603617e2460 2024-04-25T11:36:58.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.276+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v100'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.276+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.267 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:36:58.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.280+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v100'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.280+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.284+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v100'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.284+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.288+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v100'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.288+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.292+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v100'187 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.292+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.296+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v100'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.296+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.287 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:36:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.300+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v100'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.300+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617eaa70 con 0x5603617e2460 2024-04-25T11:36:58.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.305+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v100'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.305+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617eaa70 con 0x5603617e2460 2024-04-25T11:36:58.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.309+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v100'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.309+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.313+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v100'192 uv172 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.313+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.317+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v100'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.317+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.309 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:36:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.321+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v100'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.321+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e7390 con 0x5603617e2460 2024-04-25T11:36:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.325+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v100'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.325+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e7390 con 0x5603617e2460 2024-04-25T11:36:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.329+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v100'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.329+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e6380 con 0x5603617e2460 2024-04-25T11:36:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.334+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v100'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617e2460 2024-04-25T11:36:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.334+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5603617e6a00 0x5603617e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.334+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e9360 con 0x5603617e6a00 2024-04-25T11:36:58.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.335+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5603617e6a00 0x5603617e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.336+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5603617e6a00 0x5603617e8e20 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.341+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.341+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e9730 con 0x5603617e6a00 2024-04-25T11:36:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.345+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v100'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.345+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e9730 con 0x5603617e6a00 2024-04-25T11:36:58.336 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:36:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.349+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v100'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.349+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617ea740 con 0x5603617e6a00 2024-04-25T11:36:58.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.353+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.353+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617ea740 con 0x5603617e6a00 2024-04-25T11:36:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.359+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.359+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e9730 con 0x5603617e6a00 2024-04-25T11:36:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.363+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.363+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e9730 con 0x560361697650 2024-04-25T11:36:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.367+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'40 uv39 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.367+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617e9730 con 0x560361697650 2024-04-25T11:36:58.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.372+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v100'41 uv41 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.372+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e9730 con 0x560361697650 2024-04-25T11:36:58.366 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:36:58.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.376+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v100'42 uv42 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.376+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617e9730 con 0x560361697650 2024-04-25T11:36:58.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.381+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v100'43 uv43 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.381+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5603617ea0a0 0x5603617ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.381+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617eca00 con 0x5603617ea0a0 2024-04-25T11:36:58.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.382+0000 7fe67d400640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5603617ea0a0 0x5603617ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.383+0000 7fe67d400640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5603617ea0a0 0x5603617ec4c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.387+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.387+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617ecda0 con 0x5603617ea0a0 2024-04-25T11:36:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.392+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v100'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.392+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617ed250 con 0x5603617ea0a0 2024-04-25T11:36:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.395+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v100'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.383 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:36:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.396+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617eeb00 con 0x5603617ea0a0 2024-04-25T11:36:58.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.399+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.400+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617eeea0 con 0x5603617ea0a0 2024-04-25T11:36:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.403+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.404+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617edaf0 con 0x5603617ea0a0 2024-04-25T11:36:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.408+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.408+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5603617ee410 0x5603617f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.408+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f1790 con 0x5603617ee410 2024-04-25T11:36:58.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.409+0000 7fe67ca00640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5603617ee410 0x5603617f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.411+0000 7fe67ca00640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5603617ee410 0x5603617f1250 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.415+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v100'83 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.415+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f1790 con 0x5603617ee410 2024-04-25T11:36:58.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.419+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v100'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.419+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f1810 con 0x5603617ee410 2024-04-25T11:36:58.410 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:36:58.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.423+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v100'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.423+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f2c10 con 0x5603617ee410 2024-04-25T11:36:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.427+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v100'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.428+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f2c10 con 0x5603617ee410 2024-04-25T11:36:58.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.432+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v100'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.433+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f1c00 con 0x5603617ee410 2024-04-25T11:36:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.437+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v100'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.437+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f1c00 con 0x5603617ee410 2024-04-25T11:36:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.442+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v100'89 uv82 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.442+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f1c00 con 0x5603617ee410 2024-04-25T11:36:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.449+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v100'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.449+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f1c00 con 0x5603617ee410 2024-04-25T11:36:58.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.454+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v100'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.441 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:36:58.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.454+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f1c00 con 0x5603617ee410 2024-04-25T11:36:58.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.460+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v100'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.460+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f1c00 con 0x5603617ee410 2024-04-25T11:36:58.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.465+0000 7fe67ca00640 1 -- 172.21.15.19:0/2613606739 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v100'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66c00a750 con 0x5603617ee410 2024-04-25T11:36:58.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.465+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5603617f2110 0x5603617f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.465+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4c20 con 0x5603617f2110 2024-04-25T11:36:58.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.466+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5603617f2110 0x5603617f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.468+0000 7fe67c000640 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5603617f2110 0x5603617f4530 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.472+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v100'37 uv36 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.472+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617f2110 2024-04-25T11:36:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.476+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v100'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.476+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617f2110 2024-04-25T11:36:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.480+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v100'39 uv39 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.468 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:36:58.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.480+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617f2110 2024-04-25T11:36:58.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.485+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v100'40 uv40 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.485+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617f2110 2024-04-25T11:36:58.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.489+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v100'41 uv41 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.489+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x560361697650 2024-04-25T11:36:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.493+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.493+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x560361697650 2024-04-25T11:36:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.497+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v100'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.498+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x560361697650 2024-04-25T11:36:58.489 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:36:58.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.502+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v100'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.502+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f6020 con 0x560361697650 2024-04-25T11:36:58.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.506+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.506+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f6020 con 0x560361697650 2024-04-25T11:36:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.510+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.511+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f6020 con 0x560361697650 2024-04-25T11:36:58.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.515+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x560361697650 2024-04-25T11:36:58.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.515+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f6020 con 0x5603617e6a00 2024-04-25T11:36:58.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.519+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v100'74 uv68 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.519+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f6020 con 0x5603617e6a00 2024-04-25T11:36:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.524+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v100'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.524+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f6020 con 0x5603617e6a00 2024-04-25T11:36:58.515 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:36:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.527+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v100'76 uv76 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.528+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617e6a00 2024-04-25T11:36:58.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.532+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v100'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.532+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617e6a00 2024-04-25T11:36:58.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.536+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v100'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.536+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617e6a00 2024-04-25T11:36:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.540+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v100'79 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.540+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617e6a00 2024-04-25T11:36:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.544+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v100'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.544+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617e6a00 2024-04-25T11:36:58.535 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:36:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.548+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v100'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.548+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f4ff0 con 0x5603617e6a00 2024-04-25T11:36:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.552+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v100'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.552+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617ed750 con 0x5603617e6a00 2024-04-25T11:36:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.556+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v100'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617e6a00 2024-04-25T11:36:58.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.557+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617ed710 con 0x5603617ea0a0 2024-04-25T11:36:58.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.561+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v100'36 uv34 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.561+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.565+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v100'37 uv37 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.565+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.556 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:36:58.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.569+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v100'38 uv38 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.569+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.573+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v100'39 uv39 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.574+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.578+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v100'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.578+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.582+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v100'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.582+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.586+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v100'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.574 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:36:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.586+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.590+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v100'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.591+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.594+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v100'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.595+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617ea0a0 2024-04-25T11:36:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.599+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v100'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe674051070 con 0x5603617ea0a0 2024-04-25T11:36:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.599+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617f2110 2024-04-25T11:36:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.603+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v100'37 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.603+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f5b80 con 0x5603617f2110 2024-04-25T11:36:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.607+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v100'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.607+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.598 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:36:58.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.611+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v100'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.611+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.615+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v100'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.615+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.619+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v100'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.619+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.623+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v100'95 uv86 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.623+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.628+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v100'96 uv96 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.628+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.619 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:36:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.631+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v100'97 uv97 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.632+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.637+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v100'98 uv98 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.637+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.642+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v100'99 uv99 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.642+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.647+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v100'100 uv93 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.647+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.651+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v100'101 uv101 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.651+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.642 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:36:58.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.655+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v100'102 uv102 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.655+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.660+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v100'103 uv103 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.660+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.664+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v100'104 uv104 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.664+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.668+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v100'105 uv105 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.668+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x5603617f5840 con 0x5603617f2110 2024-04-25T11:36:58.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.673+0000 7fe67c000640 1 -- 172.21.15.19:0/2613606739 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v100'106 uv106 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe66800b6f0 con 0x5603617f2110 2024-04-25T11:36:58.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.673+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5603617e2460 msgr2=0x5603617e4880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.673+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5603617e2460 0x5603617e4880 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5603617ee410 msgr2=0x5603617f1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5603617ee410 0x5603617f1250 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5603617f2110 msgr2=0x5603617f4530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5603617f2110 0x5603617f4530 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5603617e6a00 msgr2=0x5603617e8e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5603617e6a00 0x5603617e8e20 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560361697650 msgr2=0x560361697a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560361697650 0x560361697a60 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5603617ea0a0 msgr2=0x5603617ec4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5603617ea0a0 0x5603617ec4c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 reap_dead start 2024-04-25T11:36:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe67d400640 1 -- 172.21.15.19:0/2613606739 reap_dead start 2024-04-25T11:36:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe65c072f10 msgr2=0x7fe65c075420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe65c072f10 0x7fe65c075420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c530 tx=0x7fe66c005590 comp rx=0 tx=0).stop 2024-04-25T11:36:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 msgr2=0x56036173a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.674+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x56036173a540 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe674025d20 tx=0x7fe674002fb0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.675+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 shutdown_connections 2024-04-25T11:36:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.675+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe65c072f10 0x7fe65c075420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.675+0000 7fe68431af40 1 --2- 172.21.15.19:0/2613606739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56036173fdd0 0x56036173a540 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.675+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 >> 172.21.15.19:0/2613606739 conn(0x560361588ac0 msgr2=0x5603616ab4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.675+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 shutdown_connections 2024-04-25T11:36:58.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.675+0000 7fe68431af40 1 -- 172.21.15.19:0/2613606739 wait complete. 2024-04-25T11:36:58.690 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2024-04-25T11:36:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.898+0000 7f14dc756f40 1 Processor -- start 2024-04-25T11:36:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.898+0000 7f14dc756f40 1 -- start start 2024-04-25T11:36:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14dc756f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5421e0 0x56113b572a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14dc756f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56113b5425f0 0x56113b572f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14dc756f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56113b61bdd0 0x56113b6a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14dc756f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56113b58b3f0 con 0x56113b5421e0 2024-04-25T11:36:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14dc756f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56113b58ae70 con 0x56113b5425f0 2024-04-25T11:36:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14dc756f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56113b58b6d0 con 0x56113b61bdd0 2024-04-25T11:36:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5421e0 0x56113b572a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5421e0 0x56113b572a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36682/0 (socket says 172.21.15.19:36682) 2024-04-25T11:36:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 -- 172.21.15.19:0/3774522034 learned_addr learned my addr 172.21.15.19:0/3774522034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 -- 172.21.15.19:0/3774522034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56113b61bdd0 msgr2=0x56113b6a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 --2- 172.21.15.19:0/3774522034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56113b61bdd0 0x56113b6a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 -- 172.21.15.19:0/3774522034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56113b5425f0 msgr2=0x56113b572f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 --2- 172.21.15.19:0/3774522034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56113b5425f0 0x56113b572f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.900+0000 7f14d5800640 1 -- 172.21.15.19:0/3774522034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113b46bb30 con 0x56113b5421e0 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.901+0000 7f14d5800640 1 --2- 172.21.15.19:0/3774522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5421e0 0x56113b572a10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f14c80095d0 tx=0x7f14c8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec6c94d3e7c5e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.901+0000 7f14cf400640 1 -- 172.21.15.19:0/3774522034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8017070 con 0x56113b5421e0 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.901+0000 7f14cf400640 1 -- 172.21.15.19:0/3774522034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c8006600 con 0x56113b5421e0 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.901+0000 7f14cf400640 1 -- 172.21.15.19:0/3774522034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80069d0 con 0x56113b5421e0 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 -- 172.21.15.19:0/3774522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5421e0 msgr2=0x56113b572a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 --2- 172.21.15.19:0/3774522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5421e0 0x56113b572a10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f14c80095d0 tx=0x7f14c8009a90 comp rx=0 tx=0).stop 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 -- 172.21.15.19:0/3774522034 shutdown_connections 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 --2- 172.21.15.19:0/3774522034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56113b5425f0 0x56113b572f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 --2- 172.21.15.19:0/3774522034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56113b61bdd0 0x56113b6a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 --2- 172.21.15.19:0/3774522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5421e0 0x56113b572a10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 -- 172.21.15.19:0/3774522034 >> 172.21.15.19:0/3774522034 conn(0x56113b464ac0 msgr2=0x56113b619360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 -- 172.21.15.19:0/3774522034 shutdown_connections 2024-04-25T11:36:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.902+0000 7f14dc756f40 1 -- 172.21.15.19:0/3774522034 wait complete. 2024-04-25T11:36:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.903+0000 7f14dc756f40 1 Processor -- start 2024-04-25T11:36:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.903+0000 7f14dc756f40 1 -- start start 2024-04-25T11:36:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14dc756f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56113b5421e0 0x56113b610960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14dc756f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5425f0 0x56113b5734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14dc756f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56113b61bdd0 0x56113b573a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14dc756f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56113b540560 con 0x56113b5425f0 2024-04-25T11:36:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14dc756f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56113b617430 con 0x56113b5421e0 2024-04-25T11:36:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14dc756f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56113b584510 con 0x56113b61bdd0 2024-04-25T11:36:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5425f0 0x56113b5734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5425f0 0x56113b5734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36690/0 (socket says 172.21.15.19:36690) 2024-04-25T11:36:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 -- 172.21.15.19:0/1668556805 learned_addr learned my addr 172.21.15.19:0/1668556805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 -- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56113b61bdd0 msgr2=0x56113b573a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56113b61bdd0 0x56113b573a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 -- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56113b5421e0 msgr2=0x56113b610960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56113b5421e0 0x56113b610960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.905+0000 7f14d4e00640 1 -- 172.21.15.19:0/1668556805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113b584760 con 0x56113b5425f0 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.906+0000 7f14d4e00640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5425f0 0x56113b5734f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f14c4055040 tx=0x7f14c4053440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.906+0000 7f14ce000640 1 -- 172.21.15.19:0/1668556805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c40538f0 con 0x56113b5425f0 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.906+0000 7f14ce000640 1 -- 172.21.15.19:0/1668556805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c4053a90 con 0x56113b5425f0 2024-04-25T11:36:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.906+0000 7f14ce000640 1 -- 172.21.15.19:0/1668556805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4067430 con 0x56113b5425f0 2024-04-25T11:36:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.906+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56113b584a40 con 0x56113b5425f0 2024-04-25T11:36:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.906+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113b615bd0 con 0x56113b5425f0 2024-04-25T11:36:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.909+0000 7f14ce000640 1 -- 172.21.15.19:0/1668556805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14c4070440 con 0x56113b5425f0 2024-04-25T11:36:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.910+0000 7f14ce000640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b8072e90 0x7f14b80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.910+0000 7f14ce000640 1 -- 172.21.15.19:0/1668556805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6397+0+0 (secure 0 0 0) 0x7f14c40d88b0 con 0x56113b5425f0 2024-04-25T11:36:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.910+0000 7f14dc756f40 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56113b61ac90 0x56113b586cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.910+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100) -- 0x56113b6160f0 con 0x56113b61ac90 2024-04-25T11:36:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.911+0000 7f14d5800640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b8072e90 0x7f14b80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.911+0000 7f14d6200640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56113b61ac90 0x56113b586cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.911+0000 7f14d6200640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56113b61ac90 0x56113b586cc0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.912+0000 7f14d5800640 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b8072e90 0x7f14b80753a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f14c8000c00 tx=0x7f14c800e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14d6200640 1 -- 172.21.15.19:0/1668556805 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 mds_snaptable [delete] v100'42 uv42 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f14d000b4f0 con 0x56113b61ac90 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56113b61ac90 msgr2=0x56113b586cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56113b61ac90 0x56113b586cc0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b8072e90 msgr2=0x7f14b80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14b8072e90 0x7f14b80753a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f14c8000c00 tx=0x7f14c800e000 comp rx=0 tx=0).stop 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5425f0 msgr2=0x56113b5734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 --2- 172.21.15.19:0/1668556805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56113b5425f0 0x56113b5734f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f14c4055040 tx=0x7f14c4053440 comp rx=0 tx=0).stop 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14d6200640 1 -- 172.21.15.19:0/1668556805 reap_dead start 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 shutdown_connections 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 >> 172.21.15.19:0/1668556805 conn(0x56113b464ac0 msgr2=0x56113b619320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 shutdown_connections 2024-04-25T11:36:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:58.915+0000 7f14dc756f40 1 -- 172.21.15.19:0/1668556805 wait complete. 2024-04-25T11:36:58.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:36:59.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.203+0000 7fd9f2800640 1 Processor -- start 2024-04-25T11:36:59.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.203+0000 7fd9f2800640 1 -- start start 2024-04-25T11:36:59.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.205+0000 7fd9f2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9ec070630 0x7fd9ec0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:59.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9f2800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ec071110 con 0x7fd9ec06dce0 2024-04-25T11:36:59.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9f2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ec06fb50 con 0x7fd9ec15adf0 2024-04-25T11:36:59.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9f2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ec0713f0 con 0x7fd9ec070630 2024-04-25T11:36:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9eb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9ec070630 0x7fd9ec0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9eb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9ec070630 0x7fd9ec0cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36704/0 (socket says 172.21.15.19:36704) 2024-04-25T11:36:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9eb400640 1 -- 172.21.15.19:0/3510444145 learned_addr learned my addr 172.21.15.19:0/3510444145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:59.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9eb400640 1 -- 172.21.15.19:0/3510444145 >> v1:172.21.15.169:6789/0 conn(0x7fd9ec15adf0 legacy=0x7fd9ec06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:36:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.206+0000 7fd9eb400640 1 -- 172.21.15.19:0/3510444145 >> v1:172.21.15.19:6789/0 conn(0x7fd9ec06dce0 legacy=0x7fd9ec160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.207+0000 7fd9eaa00640 1 -- 172.21.15.19:0/3510444145 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2117311002 0 0) 0x7fd9ec071110 con 0x7fd9ec06dce0 2024-04-25T11:36:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.207+0000 7fd9eb400640 1 -- 172.21.15.19:0/3510444145 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec06fe30 con 0x7fd9ec070630 2024-04-25T11:36:59.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.208+0000 7fd9eb400640 1 --2- 172.21.15.19:0/3510444145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec0cec50 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009870 tx=0x7fd9dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50549afc83c55b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.208+0000 7fd9eaa00640 1 -- 172.21.15.19:0/3510444145 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc037070 con 0x7fd9ec070630 2024-04-25T11:36:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.208+0000 7fd9eaa00640 1 -- 172.21.15.19:0/3510444145 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc032440 con 0x7fd9ec070630 2024-04-25T11:36:59.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.208+0000 7fd9eaa00640 1 -- 172.21.15.19:0/3510444145 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc032810 con 0x7fd9ec070630 2024-04-25T11:36:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.209+0000 7fd9f2800640 1 -- 172.21.15.19:0/3510444145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 msgr2=0x7fd9ec0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.209+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3510444145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec0cec50 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009870 tx=0x7fd9dc02a210 comp rx=0 tx=0).stop 2024-04-25T11:36:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.209+0000 7fd9f2800640 1 -- 172.21.15.19:0/3510444145 shutdown_connections 2024-04-25T11:36:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.209+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3510444145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec0cec50 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.209+0000 7fd9f2800640 1 -- 172.21.15.19:0/3510444145 >> 172.21.15.19:0/3510444145 conn(0x7fd9ec073090 msgr2=0x7fd9ec0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.209+0000 7fd9f2800640 1 -- 172.21.15.19:0/3510444145 shutdown_connections 2024-04-25T11:36:59.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.209+0000 7fd9f2800640 1 -- 172.21.15.19:0/3510444145 wait complete. 2024-04-25T11:36:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.212+0000 7fd9f2800640 1 Processor -- start 2024-04-25T11:36:59.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.212+0000 7fd9f2800640 1 -- start start 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec1e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9ec15adf0 0x7fd9ec150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9f2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9ec15b290 0x7fd9ec0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9f2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec070c80 con 0x7fd9ec070630 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9f2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec0701a0 con 0x7fd9ec15b290 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9f2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ec070e00 con 0x7fd9ec15adf0 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9eb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec1e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:59.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9eb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec1e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36716/0 (socket says 172.21.15.19:36716) 2024-04-25T11:36:59.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9eb400640 1 -- 172.21.15.19:0/3691915877 learned_addr learned my addr 172.21.15.19:0/3691915877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:36:59.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9eb400640 1 -- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9ec15adf0 msgr2=0x7fd9ec150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:59.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9e3e00640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9ec15adf0 0x7fd9ec150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:59.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9eb400640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9ec15adf0 0x7fd9ec150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:59.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9eb400640 1 -- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9ec15b290 msgr2=0x7fd9ec0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:59.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.214+0000 7fd9eb400640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9ec15b290 0x7fd9ec0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:59.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.215+0000 7fd9eb400640 1 -- 172.21.15.19:0/3691915877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec1e8500 con 0x7fd9ec070630 2024-04-25T11:36:59.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.215+0000 7fd9eb400640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec1e6b30 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009870 tx=0x7fd9dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:59.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.215+0000 7fd9e3400640 1 -- 172.21.15.19:0/3691915877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc041070 con 0x7fd9ec070630 2024-04-25T11:36:59.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.215+0000 7fd9e3400640 1 -- 172.21.15.19:0/3691915877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc03c870 con 0x7fd9ec070630 2024-04-25T11:36:59.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.215+0000 7fd9e3400640 1 -- 172.21.15.19:0/3691915877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc037040 con 0x7fd9ec070630 2024-04-25T11:36:59.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.215+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec1e8a00 con 0x7fd9ec070630 2024-04-25T11:36:59.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.215+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec15f1e0 con 0x7fd9ec070630 2024-04-25T11:36:59.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.217+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ec050740 con 0x7fd9ec070630 2024-04-25T11:36:59.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.218+0000 7fd9e3400640 1 -- 172.21.15.19:0/3691915877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9dc044070 con 0x7fd9ec070630 2024-04-25T11:36:59.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.219+0000 7fd9e3400640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9c80744e0 0x7fd9c80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:59.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.219+0000 7fd9e3400640 1 -- 172.21.15.19:0/3691915877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6397+0+0 (secure 0 0 0) 0x7fd9dc0b5ff0 con 0x7fd9ec070630 2024-04-25T11:36:59.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.220+0000 7fd9e3e00640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9c80744e0 0x7fd9c80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:59.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.221+0000 7fd9e3e00640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9c80744e0 0x7fd9c80769f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4006fd0 tx=0x7fd9d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:59.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.225+0000 7fd9e3400640 1 -- 172.21.15.19:0/3691915877 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9dc080500 con 0x7fd9ec070630 2024-04-25T11:36:59.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:36:59.471+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd9ec0cd790 con 0x7fd9ec070630 2024-04-25T11:37:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.504+0000 7fd9e3400640 1 -- 172.21.15.19:0/3691915877 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v142) ==== 162+0+0 (secure 0 0 0) 0x7fd9dc07fe00 con 0x7fd9ec070630 2024-04-25T11:37:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:37:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9c80744e0 msgr2=0x7fd9c80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9c80744e0 0x7fd9c80769f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4006fd0 tx=0x7fd9d4008040 comp rx=0 tx=0).stop 2024-04-25T11:37:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 msgr2=0x7fd9ec1e6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec1e6b30 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009870 tx=0x7fd9dc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 shutdown_connections 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd9c80744e0 0x7fd9c80769f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9ec15b290 0x7fd9ec0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.508+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9ec15adf0 0x7fd9ec150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.509+0000 7fd9f2800640 1 --2- 172.21.15.19:0/3691915877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9ec070630 0x7fd9ec1e6b30 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.509+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 >> 172.21.15.19:0/3691915877 conn(0x7fd9ec073090 msgr2=0x7fd9ec1614e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.509+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 shutdown_connections 2024-04-25T11:37:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.509+0000 7fd9f2800640 1 -- 172.21.15.19:0/3691915877 wait complete. 2024-04-25T11:37:00.512 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:00.525+0000 7fe33f800640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-04-25T11:37:00.513 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:00.525+0000 7fe33f800640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-04-25T11:37:00.513 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:00.526+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:37:00.514 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:00.526+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:37:00.525 INFO:tasks.ceph.mds.a.smithi019.stderr: -30> 2024-04-25T11:37:00.525+0000 7fe33f800640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-04-25T11:37:00.525 INFO:tasks.ceph.mds.a.smithi019.stderr: -29> 2024-04-25T11:37:00.525+0000 7fe33f800640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-04-25T11:37:00.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:00.650 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:37:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.801+0000 7f03ece00640 1 Processor -- start 2024-04-25T11:37:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.801+0000 7f03ece00640 1 -- start start 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.803+0000 7f03ece00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03e80b7c60 0x7f03e80d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.803+0000 7f03ece00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e80d1f40 con 0x7f03e806c8a0 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.803+0000 7f03ece00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e80d0430 con 0x7f03e806ecb0 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.803+0000 7f03ece00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f03e80d21e0 con 0x7f03e80b7c60 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.803+0000 7f03e6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03e80b7c60 0x7f03e80d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.804+0000 7f03e6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03e80b7c60 0x7f03e80d3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36730/0 (socket says 172.21.15.19:36730) 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.804+0000 7f03e6a00640 1 -- 172.21.15.19:0/76507506 learned_addr learned my addr 172.21.15.19:0/76507506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.804+0000 7f03e7e00640 1 --1- 172.21.15.19:0/76507506 >> v1:172.21.15.19:6789/0 conn(0x7f03e806c8a0 0x7f03e806ccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46804/0 (socket says 172.21.15.19:46804) 2024-04-25T11:37:00.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.804+0000 7f03e6a00640 1 -- 172.21.15.19:0/76507506 >> v1:172.21.15.169:6789/0 conn(0x7f03e806ecb0 legacy=0x7f03e80b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:00.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.804+0000 7f03e6a00640 1 -- 172.21.15.19:0/76507506 >> v1:172.21.15.19:6789/0 conn(0x7f03e806c8a0 legacy=0x7f03e806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:00.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.804+0000 7f03e6a00640 1 -- 172.21.15.19:0/76507506 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e80d06d0 con 0x7f03e80b7c60 2024-04-25T11:37:00.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.804+0000 7f03e6000640 1 -- 172.21.15.19:0/76507506 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930885591 0 0) 0x7f03e80d1f40 con 0x7f03e806c8a0 2024-04-25T11:37:00.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.805+0000 7f03e6a00640 1 --2- 172.21.15.19:0/76507506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e80b7c60 0x7f03e80d3160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009870 tx=0x7f03d002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c213d83ec167d521 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:00.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.806+0000 7f03e6000640 1 -- 172.21.15.19:0/76507506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0037070 con 0x7f03e80b7c60 2024-04-25T11:37:00.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.806+0000 7f03e6000640 1 -- 172.21.15.19:0/76507506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d0032440 con 0x7f03e80b7c60 2024-04-25T11:37:00.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.806+0000 7f03e6000640 1 -- 172.21.15.19:0/76507506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0032810 con 0x7f03e80b7c60 2024-04-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.806+0000 7f03ece00640 1 -- 172.21.15.19:0/76507506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e80b7c60 msgr2=0x7f03e80d3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.806+0000 7f03ece00640 1 --2- 172.21.15.19:0/76507506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e80b7c60 0x7f03e80d3160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009870 tx=0x7f03d002a2a0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.807+0000 7f03ece00640 1 -- 172.21.15.19:0/76507506 shutdown_connections 2024-04-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.807+0000 7f03ece00640 1 --2- 172.21.15.19:0/76507506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e80b7c60 0x7f03e80d3160 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.807+0000 7f03ece00640 1 -- 172.21.15.19:0/76507506 >> 172.21.15.19:0/76507506 conn(0x7f03e809b7d0 msgr2=0x7f03e809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.807+0000 7f03ece00640 1 -- 172.21.15.19:0/76507506 shutdown_connections 2024-04-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.807+0000 7f03ece00640 1 -- 172.21.15.19:0/76507506 wait complete. 2024-04-25T11:37:00.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.809+0000 7f03ece00640 1 Processor -- start 2024-04-25T11:37:00.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.809+0000 7f03ece00640 1 -- start start 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.810+0000 7f03ece00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e806c8a0 0x7f03e814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03ece00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e806ecb0 0x7f03e81f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e81fa2f0 0x7f03e81fa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03ece00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8163bb0 con 0x7f03e81fa2f0 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03ece00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03e80d24c0 con 0x7f03e806c8a0 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03ece00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03e8163d30 con 0x7f03e806ecb0 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e81fa2f0 0x7f03e81fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e81fa2f0 0x7f03e81fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36732/0 (socket says 172.21.15.19:36732) 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 -- 172.21.15.19:0/2502710129 learned_addr learned my addr 172.21.15.19:0/2502710129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 -- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e806ecb0 msgr2=0x7f03e81f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e806ecb0 0x7f03e81f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 -- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e806c8a0 msgr2=0x7f03e814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e806c8a0 0x7f03e814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.811+0000 7f03e7e00640 1 -- 172.21.15.19:0/2502710129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e80b0350 con 0x7f03e81fa2f0 2024-04-25T11:37:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.812+0000 7f03e7e00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e81fa2f0 0x7f03e81fa700 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c6d0 tx=0x7f03e000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.812+0000 7f03df400640 1 -- 172.21.15.19:0/2502710129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0018070 con 0x7f03e81fa2f0 2024-04-25T11:37:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.812+0000 7f03df400640 1 -- 172.21.15.19:0/2502710129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e0004d60 con 0x7f03e81fa2f0 2024-04-25T11:37:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.812+0000 7f03df400640 1 -- 172.21.15.19:0/2502710129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e000a880 con 0x7f03e81fa2f0 2024-04-25T11:37:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.812+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e80b0630 con 0x7f03e81fa2f0 2024-04-25T11:37:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.812+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e80b0bf0 con 0x7f03e81fa2f0 2024-04-25T11:37:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.814+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e80b00a0 con 0x7f03e81fa2f0 2024-04-25T11:37:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.815+0000 7f03df400640 1 -- 172.21.15.19:0/2502710129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f03e001b070 con 0x7f03e81fa2f0 2024-04-25T11:37:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.816+0000 7f03df400640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03c40744b0 0x7f03c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.816+0000 7f03df400640 1 -- 172.21.15.19:0/2502710129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7f03e0094d70 con 0x7f03e81fa2f0 2024-04-25T11:37:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.817+0000 7f03e7400640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03c40744b0 0x7f03c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:00.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.818+0000 7f03e7400640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03c40744b0 0x7f03c40769c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03e8073a30 tx=0x7f03d8006bb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:00.821+0000 7f03df400640 1 -- 172.21.15.19:0/2502710129 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03e005f150 con 0x7f03e81fa2f0 2024-04-25T11:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.068+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e8050740 con 0x7f03e81fa2f0 2024-04-25T11:37:01.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.070+0000 7f03df400640 1 -- 172.21.15.19:0/2502710129 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) ==== 95+0+4352 (secure 0 0 0) 0x7f03e005ea50 con 0x7f03e81fa2f0 2024-04-25T11:37:01.057 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:01.058 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":144,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:00.527003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:01.058 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 144 2024-04-25T11:37:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.072+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03c40744b0 msgr2=0x7f03c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.072+0000 7f03ece00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03c40744b0 0x7f03c40769c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03e8073a30 tx=0x7f03d8006bb0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.072+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e81fa2f0 msgr2=0x7f03e81fa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.072+0000 7f03ece00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e81fa2f0 0x7f03e81fa700 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f03e000c6d0 tx=0x7f03e000cb90 comp rx=0 tx=0).stop 2024-04-25T11:37:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.072+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 shutdown_connections 2024-04-25T11:37:01.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.073+0000 7f03ece00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03c40744b0 0x7f03c40769c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.073+0000 7f03ece00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03e81fa2f0 0x7f03e81fa700 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.073+0000 7f03ece00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03e806ecb0 0x7f03e81f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.073+0000 7f03ece00640 1 --2- 172.21.15.19:0/2502710129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03e806c8a0 0x7f03e814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.073+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 >> 172.21.15.19:0/2502710129 conn(0x7f03e809b7d0 msgr2=0x7f03e8163ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:01.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.073+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 shutdown_connections 2024-04-25T11:37:01.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.073+0000 7f03ece00640 1 -- 172.21.15.19:0/2502710129 wait complete. 2024-04-25T11:37:01.089 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:01.089 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:01.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.387+0000 7f451ae00640 1 Processor -- start 2024-04-25T11:37:01.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.387+0000 7f451ae00640 1 -- start start 2024-04-25T11:37:01.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f451ae00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45140d0ed0 0x7f45140d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f451ae00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451406f3e0 con 0x7f4514160be0 2024-04-25T11:37:01.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f451ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451406f6c0 con 0x7f45141434d0 2024-04-25T11:37:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f451ae00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f45141438e0 con 0x7f45140d0ed0 2024-04-25T11:37:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f4513400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45140d0ed0 0x7f45140d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f4518e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4514160be0 0x7f4514160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46820/0 (socket says 172.21.15.19:46820) 2024-04-25T11:37:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f4513400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45140d0ed0 0x7f45140d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36748/0 (socket says 172.21.15.19:36748) 2024-04-25T11:37:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.390+0000 7f4518e00640 1 -- 172.21.15.19:0/462653623 learned_addr learned my addr 172.21.15.19:0/462653623 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.391+0000 7f4513400640 1 -- 172.21.15.19:0/462653623 >> v1:172.21.15.169:6789/0 conn(0x7f45141434d0 legacy=0x7f4514070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:01.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.391+0000 7f4513400640 1 -- 172.21.15.19:0/462653623 >> v1:172.21.15.19:6789/0 conn(0x7f4514160be0 legacy=0x7f4514160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.391+0000 7f4513400640 1 -- 172.21.15.19:0/462653623 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451406f9a0 con 0x7f45140d0ed0 2024-04-25T11:37:01.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.391+0000 7f4512a00640 1 -- 172.21.15.19:0/462653623 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2905154900 0 0) 0x7f451406f3e0 con 0x7f4514160be0 2024-04-25T11:37:01.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.392+0000 7f4513400640 1 --2- 172.21.15.19:0/462653623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f45140d12e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f45000097d0 tx=0x7f450002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b560fd350f9e178f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.392+0000 7f4512a00640 1 -- 172.21.15.19:0/462653623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500037070 con 0x7f45140d0ed0 2024-04-25T11:37:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.392+0000 7f4512a00640 1 -- 172.21.15.19:0/462653623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4500032440 con 0x7f45140d0ed0 2024-04-25T11:37:01.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.392+0000 7f4512a00640 1 -- 172.21.15.19:0/462653623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500032810 con 0x7f45140d0ed0 2024-04-25T11:37:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.393+0000 7f451ae00640 1 -- 172.21.15.19:0/462653623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 msgr2=0x7f45140d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.393+0000 7f451ae00640 1 --2- 172.21.15.19:0/462653623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f45140d12e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f45000097d0 tx=0x7f450002a210 comp rx=0 tx=0).stop 2024-04-25T11:37:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.393+0000 7f451ae00640 1 -- 172.21.15.19:0/462653623 shutdown_connections 2024-04-25T11:37:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.393+0000 7f451ae00640 1 --2- 172.21.15.19:0/462653623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f45140d12e0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.393+0000 7f451ae00640 1 -- 172.21.15.19:0/462653623 >> 172.21.15.19:0/462653623 conn(0x7f45140d1ae0 msgr2=0x7f45140d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.393+0000 7f451ae00640 1 -- 172.21.15.19:0/462653623 shutdown_connections 2024-04-25T11:37:01.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.393+0000 7f451ae00640 1 -- 172.21.15.19:0/462653623 wait complete. 2024-04-25T11:37:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.396+0000 7f451ae00640 1 Processor -- start 2024-04-25T11:37:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.396+0000 7f451ae00640 1 -- start start 2024-04-25T11:37:01.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f451ae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f451415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f451ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45141434d0 0x7f4514146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f451ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4514160be0 0x7f4514146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f451ae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45140b51a0 con 0x7f45140d0ed0 2024-04-25T11:37:01.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f451ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f451406f090 con 0x7f45141434d0 2024-04-25T11:37:01.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f451ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45140b5320 con 0x7f4514160be0 2024-04-25T11:37:01.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f4513e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f451415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f4513e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f451415caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36764/0 (socket says 172.21.15.19:36764) 2024-04-25T11:37:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.398+0000 7f4513e00640 1 -- 172.21.15.19:0/677662554 learned_addr learned my addr 172.21.15.19:0/677662554 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f4513e00640 1 -- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4514160be0 msgr2=0x7f4514146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f4513e00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4514160be0 0x7f4514146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f4513e00640 1 -- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45141434d0 msgr2=0x7f4514146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f4513e00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45141434d0 0x7f4514146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f4513e00640 1 -- 172.21.15.19:0/677662554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451415cfe0 con 0x7f45140d0ed0 2024-04-25T11:37:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f4513e00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f451415caa0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f450400ba50 tx=0x7f450400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f450be00640 1 -- 172.21.15.19:0/677662554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504019070 con 0x7f45140d0ed0 2024-04-25T11:37:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f450be00640 1 -- 172.21.15.19:0/677662554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450400cc10 con 0x7f45140d0ed0 2024-04-25T11:37:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f450be00640 1 -- 172.21.15.19:0/677662554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504014440 con 0x7f45140d0ed0 2024-04-25T11:37:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.399+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4514144c30 con 0x7f45140d0ed0 2024-04-25T11:37:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.400+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45141451f0 con 0x7f45140d0ed0 2024-04-25T11:37:01.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.401+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45141605f0 con 0x7f45140d0ed0 2024-04-25T11:37:01.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.402+0000 7f450be00640 1 -- 172.21.15.19:0/677662554 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4504007370 con 0x7f45140d0ed0 2024-04-25T11:37:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.403+0000 7f450be00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44f40744e0 0x7f44f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.403+0000 7f450be00640 1 -- 172.21.15.19:0/677662554 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7f4504095260 con 0x7f45140d0ed0 2024-04-25T11:37:01.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.404+0000 7f4513400640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44f40744e0 0x7f44f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.406+0000 7f4513400640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44f40744e0 0x7f44f40769f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4500034000 tx=0x7f450002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:01.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.408+0000 7f450be00640 1 -- 172.21.15.19:0/677662554 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f450405f640 con 0x7f45140d0ed0 2024-04-25T11:37:01.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.665+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f451406e100 con 0x7f45140d0ed0 2024-04-25T11:37:01.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.666+0000 7f450be00640 1 -- 172.21.15.19:0/677662554 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) ==== 95+0+5175 (secure 0 0 0) 0x7f450405ef40 con 0x7f45140d0ed0 2024-04-25T11:37:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:01.654 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":145,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/845688948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":845688948},{"type":"v1","addr":"172.21.15.19:6835","nonce":845688948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":145}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:00.527003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:01.655 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 145 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44f40744e0 msgr2=0x7f44f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44f40744e0 0x7f44f40769f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4500034000 tx=0x7f450002c000 comp rx=0 tx=0).stop 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 msgr2=0x7f451415caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f451415caa0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f450400ba50 tx=0x7f450400bf10 comp rx=0 tx=0).stop 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 shutdown_connections 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44f40744e0 0x7f44f40769f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45141434d0 0x7f4514146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4514160be0 0x7f4514146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 --2- 172.21.15.19:0/677662554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45140d0ed0 0x7f451415caa0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 >> 172.21.15.19:0/677662554 conn(0x7f45140d1ae0 msgr2=0x7f4514160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 shutdown_connections 2024-04-25T11:37:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.669+0000 7f451ae00640 1 -- 172.21.15.19:0/677662554 wait complete. 2024-04-25T11:37:01.688 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:01.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:01.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.985+0000 7fe544a00640 1 Processor -- start 2024-04-25T11:37:01.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.985+0000 7fe544a00640 1 -- start start 2024-04-25T11:37:01.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe544a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5401647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe544a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5400b5c90 0x7fe5400b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe544a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5400b65e0 con 0x7fe540160360 2024-04-25T11:37:01.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe544a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe54006d100 con 0x7fe5400b5c90 2024-04-25T11:37:01.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe544a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe54006c7d0 con 0x7fe540070680 2024-04-25T11:37:01.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe53e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5400b5c90 0x7fe5400b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe53e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe5400b5c90 0x7fe5400b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36776/0 (socket says 172.21.15.19:36776) 2024-04-25T11:37:01.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.988+0000 7fe53e800640 1 -- 172.21.15.19:0/1163203677 learned_addr learned my addr 172.21.15.19:0/1163203677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:01.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.989+0000 7fe53de00640 1 --2- 172.21.15.19:0/1163203677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5401647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.989+0000 7fe53e800640 1 -- 172.21.15.19:0/1163203677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 msgr2=0x7fe5401647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:01.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.989+0000 7fe53e800640 1 --2- 172.21.15.19:0/1163203677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5401647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.989+0000 7fe53e800640 1 -- 172.21.15.19:0/1163203677 >> v1:172.21.15.169:6789/0 conn(0x7fe540160360 legacy=0x7fe5400af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:01.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.989+0000 7fe53e800640 1 -- 172.21.15.19:0/1163203677 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54006cab0 con 0x7fe5400b5c90 2024-04-25T11:37:01.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.989+0000 7fe53de00640 1 --2- 172.21.15.19:0/1163203677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5401647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:01.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.990+0000 7fe53e800640 1 --2- 172.21.15.19:0/1163203677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe5400b60a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe538009870 tx=0x7fe53802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519c19777828dd5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:01.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.990+0000 7fe53ca00640 1 -- 172.21.15.19:0/1163203677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538037070 con 0x7fe5400b5c90 2024-04-25T11:37:01.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.990+0000 7fe53ca00640 1 -- 172.21.15.19:0/1163203677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53802ae10 con 0x7fe5400b5c90 2024-04-25T11:37:01.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.990+0000 7fe53ca00640 1 -- 172.21.15.19:0/1163203677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538032660 con 0x7fe5400b5c90 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 -- 172.21.15.19:0/1163203677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 msgr2=0x7fe5400b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 --2- 172.21.15.19:0/1163203677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe5400b60a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe538009870 tx=0x7fe53802a100 comp rx=0 tx=0).stop 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 -- 172.21.15.19:0/1163203677 shutdown_connections 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 --2- 172.21.15.19:0/1163203677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe5400b60a0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 --2- 172.21.15.19:0/1163203677 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5401647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 -- 172.21.15.19:0/1163203677 >> 172.21.15.19:0/1163203677 conn(0x7fe5400d1ae0 msgr2=0x7fe5400d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 -- 172.21.15.19:0/1163203677 shutdown_connections 2024-04-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.991+0000 7fe544a00640 1 -- 172.21.15.19:0/1163203677 wait complete. 2024-04-25T11:37:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.994+0000 7fe544a00640 1 Processor -- start 2024-04-25T11:37:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.994+0000 7fe544a00640 1 -- start start 2024-04-25T11:37:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.995+0000 7fe544a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5400c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.995+0000 7fe544a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe54014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.995+0000 7fe544a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe540160360 0x7fe54014be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.995+0000 7fe544a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400afbb0 con 0x7fe5400b5c90 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.995+0000 7fe544a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe54006fcc0 con 0x7fe540160360 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.995+0000 7fe544a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400afd30 con 0x7fe540070680 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe54014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe54014e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36790/0 (socket says 172.21.15.19:36790) 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 -- 172.21.15.19:0/1774624264 learned_addr learned my addr 172.21.15.19:0/1774624264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53de00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5400c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53e800640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe540160360 0x7fe54014be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 -- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 msgr2=0x7fe5400c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5400c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 -- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe540160360 msgr2=0x7fe54014be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe540160360 0x7fe54014be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53d400640 1 -- 172.21.15.19:0/1774624264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54014e7d0 con 0x7fe5400b5c90 2024-04-25T11:37:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.996+0000 7fe53e800640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe540160360 0x7fe54014be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.997+0000 7fe53d400640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe54014e290 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe52800ba50 tx=0x7fe52800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.997+0000 7fe536000640 1 -- 172.21.15.19:0/1774624264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528019070 con 0x7fe5400b5c90 2024-04-25T11:37:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.997+0000 7fe536000640 1 -- 172.21.15.19:0/1774624264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52800cc20 con 0x7fe5400b5c90 2024-04-25T11:37:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.997+0000 7fe536000640 1 -- 172.21.15.19:0/1774624264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528014680 con 0x7fe5400b5c90 2024-04-25T11:37:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.997+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54015e480 con 0x7fe5400b5c90 2024-04-25T11:37:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.997+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54015ea40 con 0x7fe5400b5c90 2024-04-25T11:37:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:01.998+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe540050740 con 0x7fe5400b5c90 2024-04-25T11:37:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.000+0000 7fe536000640 1 -- 172.21.15.19:0/1774624264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe52801c070 con 0x7fe5400b5c90 2024-04-25T11:37:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.001+0000 7fe536000640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe51c0744e0 0x7fe51c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.001+0000 7fe536000640 1 -- 172.21.15.19:0/1774624264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7fe528095250 con 0x7fe5400b5c90 2024-04-25T11:37:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.002+0000 7fe53de00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe51c0744e0 0x7fe51c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.004+0000 7fe53de00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe51c0744e0 0x7fe51c0769f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe540071250 tx=0x7fe530008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:01.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.006+0000 7fe536000640 1 -- 172.21.15.19:0/1774624264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe52805efe0 con 0x7fe5400b5c90 2024-04-25T11:37:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.252+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5400d0b90 con 0x7fe5400b5c90 2024-04-25T11:37:02.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.254+0000 7fe536000640 1 -- 172.21.15.19:0/1774624264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) ==== 95+0+5175 (secure 0 0 0) 0x7fe5280632e0 con 0x7fe5400b5c90 2024-04-25T11:37:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:02.243 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":145,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/845688948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":845688948},{"type":"v1","addr":"172.21.15.19:6835","nonce":845688948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":145}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:00.527003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 145 2024-04-25T11:37:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.256+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe51c0744e0 msgr2=0x7fe51c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.256+0000 7fe544a00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe51c0744e0 0x7fe51c0769f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe540071250 tx=0x7fe530008040 comp rx=0 tx=0).stop 2024-04-25T11:37:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.256+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 msgr2=0x7fe54014e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.256+0000 7fe544a00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe54014e290 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe52800ba50 tx=0x7fe52800bf10 comp rx=0 tx=0).stop 2024-04-25T11:37:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 shutdown_connections 2024-04-25T11:37:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe51c0744e0 0x7fe51c0769f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe540160360 0x7fe54014be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5400b5c90 0x7fe54014e290 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 --2- 172.21.15.19:0/1774624264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe540070680 0x7fe5400c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 >> 172.21.15.19:0/1774624264 conn(0x7fe5400d1ae0 msgr2=0x7fe5400b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 shutdown_connections 2024-04-25T11:37:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.257+0000 7fe544a00640 1 -- 172.21.15.19:0/1774624264 wait complete. 2024-04-25T11:37:02.272 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2024-04-25T11:37:02.273 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2024-04-25T11:37:02.273 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:02.273 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:02.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:37:02.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:37:02.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:02.329 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:37:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:37:02.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:37:02.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:37:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.679+0000 7f4683e00640 1 Processor -- start 2024-04-25T11:37:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.679+0000 7f4683e00640 1 -- start start 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.681+0000 7f4683e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f468406c5c0 0x7f46840cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.681+0000 7f4683e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4684144370 0x7f46840b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.681+0000 7f4683e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46840b50e0 con 0x7f46840b4cd0 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.681+0000 7f4683e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f46840d1230 con 0x7f4684144370 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.681+0000 7f4683e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f46840d06b0 con 0x7f468406c5c0 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.681+0000 7f4682000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4684144370 0x7f46840b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.682+0000 7f4682000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4684144370 0x7f46840b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36804/0 (socket says 172.21.15.19:36804) 2024-04-25T11:37:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.682+0000 7f4682000640 1 -- 172.21.15.19:0/2316229694 learned_addr learned my addr 172.21.15.19:0/2316229694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.682+0000 7f4682000640 1 -- 172.21.15.19:0/2316229694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f468406c5c0 msgr2=0x7f46840cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.682+0000 7f4682a00640 1 --2- 172.21.15.19:0/2316229694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f468406c5c0 0x7f46840cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.682+0000 7f4682000640 1 --2- 172.21.15.19:0/2316229694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f468406c5c0 0x7f46840cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.682+0000 7f4682000640 1 -- 172.21.15.19:0/2316229694 >> v1:172.21.15.169:6789/0 conn(0x7f46840b4cd0 legacy=0x7f4684161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.682+0000 7f4682000640 1 -- 172.21.15.19:0/2316229694 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46840d0990 con 0x7f4684144370 2024-04-25T11:37:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.683+0000 7f4682000640 1 --2- 172.21.15.19:0/2316229694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4684144370 0x7f46840b4790 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4670009870 tx=0x7f467002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c74a11ce85bd05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.683+0000 7f4681600640 1 -- 172.21.15.19:0/2316229694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670037070 con 0x7f4684144370 2024-04-25T11:37:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4681600640 1 -- 172.21.15.19:0/2316229694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4670032440 con 0x7f4684144370 2024-04-25T11:37:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4681600640 1 -- 172.21.15.19:0/2316229694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670032810 con 0x7f4684144370 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4683e00640 1 -- 172.21.15.19:0/2316229694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4684144370 msgr2=0x7f46840b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4683e00640 1 --2- 172.21.15.19:0/2316229694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4684144370 0x7f46840b4790 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4670009870 tx=0x7f467002a210 comp rx=0 tx=0).stop 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4683e00640 1 -- 172.21.15.19:0/2316229694 shutdown_connections 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4683e00640 1 --2- 172.21.15.19:0/2316229694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4684144370 0x7f46840b4790 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4683e00640 1 --2- 172.21.15.19:0/2316229694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f468406c5c0 0x7f46840cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4683e00640 1 -- 172.21.15.19:0/2316229694 >> 172.21.15.19:0/2316229694 conn(0x7f46840d1720 msgr2=0x7f46840d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.684+0000 7f4683e00640 1 -- 172.21.15.19:0/2316229694 shutdown_connections 2024-04-25T11:37:02.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.685+0000 7f4683e00640 1 -- 172.21.15.19:0/2316229694 wait complete. 2024-04-25T11:37:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.687+0000 7f4683e00640 1 Processor -- start 2024-04-25T11:37:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.687+0000 7f4683e00640 1 -- start start 2024-04-25T11:37:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4683e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468406c5c0 0x7f4684158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4683e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46840b4cd0 0x7f46841e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4683e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4684144370 0x7f46841e8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4683e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46840d02d0 con 0x7f468406c5c0 2024-04-25T11:37:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4683e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46840d0f50 con 0x7f46840b4cd0 2024-04-25T11:37:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4683e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f46840d0450 con 0x7f4684144370 2024-04-25T11:37:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468406c5c0 0x7f4684158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468406c5c0 0x7f4684158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36814/0 (socket says 172.21.15.19:36814) 2024-04-25T11:37:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682a00640 1 -- 172.21.15.19:0/1506164673 learned_addr learned my addr 172.21.15.19:0/1506164673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4683400640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4684144370 0x7f46841e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682a00640 1 -- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4684144370 msgr2=0x7f46841e8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682000640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46840b4cd0 0x7f46841e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682a00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4684144370 0x7f46841e8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682a00640 1 -- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46840b4cd0 msgr2=0x7f46841e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.689+0000 7f4682a00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46840b4cd0 0x7f46841e7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.690+0000 7f4682a00640 1 -- 172.21.15.19:0/1506164673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4684159b80 con 0x7f468406c5c0 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.690+0000 7f4682a00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468406c5c0 0x7f4684158fb0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f467400c7f0 tx=0x7f467400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.690+0000 7f467aa00640 1 -- 172.21.15.19:0/1506164673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674015070 con 0x7f468406c5c0 2024-04-25T11:37:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.690+0000 7f467aa00640 1 -- 172.21.15.19:0/1506164673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4674004220 con 0x7f468406c5c0 2024-04-25T11:37:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.690+0000 7f467aa00640 1 -- 172.21.15.19:0/1506164673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674007620 con 0x7f468406c5c0 2024-04-25T11:37:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.690+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4684161e40 con 0x7f468406c5c0 2024-04-25T11:37:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.690+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4684162400 con 0x7f468406c5c0 2024-04-25T11:37:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.691+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4684050740 con 0x7f468406c5c0 2024-04-25T11:37:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.693+0000 7f467aa00640 1 -- 172.21.15.19:0/1506164673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46740079d0 con 0x7f468406c5c0 2024-04-25T11:37:02.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.694+0000 7f467aa00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46640744e0 0x7f46640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:02.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.694+0000 7f467aa00640 1 -- 172.21.15.19:0/1506164673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7f4674095910 con 0x7f468406c5c0 2024-04-25T11:37:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.695+0000 7f4682000640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46640744e0 0x7f46640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:02.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.696+0000 7f4682000640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46640744e0 0x7f46640769f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4670034000 tx=0x7f467002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:02.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.698+0000 7f467aa00640 1 -- 172.21.15.19:0/1506164673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f467405fe00 con 0x7f468406c5c0 2024-04-25T11:37:02.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:02.948+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f46840b5850 con 0x7f468406c5c0 2024-04-25T11:37:03.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.550+0000 7f467aa00640 1 -- 172.21.15.19:0/1506164673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v146) ==== 161+0+0 (secure 0 0 0) 0x7f467405f700 con 0x7f468406c5c0 2024-04-25T11:37:03.539 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46640744e0 msgr2=0x7f46640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46640744e0 0x7f46640769f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4670034000 tx=0x7f467002e000 comp rx=0 tx=0).stop 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468406c5c0 msgr2=0x7f4684158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468406c5c0 0x7f4684158fb0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f467400c7f0 tx=0x7f467400ccb0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 shutdown_connections 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f46640744e0 0x7f46640769f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46840b4cd0 0x7f46841e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4684144370 0x7f46841e8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 --2- 172.21.15.19:0/1506164673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f468406c5c0 0x7f4684158fb0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.554+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 >> 172.21.15.19:0/1506164673 conn(0x7f46840d1720 msgr2=0x7f4684142850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:03.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.555+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 shutdown_connections 2024-04-25T11:37:03.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.555+0000 7f4683e00640 1 -- 172.21.15.19:0/1506164673 wait complete. 2024-04-25T11:37:03.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:37:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.875+0000 7f42bd800640 1 Processor -- start 2024-04-25T11:37:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.875+0000 7f42bd800640 1 -- start start 2024-04-25T11:37:03.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.878+0000 7f42bd800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42b8144500 0x7f42b8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.878+0000 7f42bd800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b806f5f0 con 0x7f42b8162310 2024-04-25T11:37:03.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.878+0000 7f42bd800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42b806f8d0 con 0x7f42b80cdb90 2024-04-25T11:37:03.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.878+0000 7f42bd800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f42b8070200 con 0x7f42b8144500 2024-04-25T11:37:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.878+0000 7f42b7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f42b8162310 0x7f42b80ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33654/0 (socket says 172.21.15.19:33654) 2024-04-25T11:37:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.878+0000 7f42b7400640 1 -- 172.21.15.19:0/2176594335 learned_addr learned my addr 172.21.15.19:0/2176594335 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295643687 0 0) 0x7f42b806f5f0 con 0x7f42b8162310 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f429c003520 con 0x7f42b8162310 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b6a00640 1 --2- 172.21.15.19:0/2176594335 >> v2:172.21.15.169:3300/0 conn(0x7f42b8144500 0x7f42b8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1531067032 0 0) 0x7f429c003520 con 0x7f42b8162310 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 >> v2:172.21.15.169:3300/0 conn(0x7f42b8144500 msgr2=0x7f42b8070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 --2- 172.21.15.19:0/2176594335 >> v2:172.21.15.169:3300/0 conn(0x7f42b8144500 0x7f42b8070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 >> v1:172.21.15.169:6789/0 conn(0x7f42b80cdb90 legacy=0x7f42b8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b806fbb0 con 0x7f42b8162310 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f42b0001670 con 0x7f42b8162310 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 >> v1:172.21.15.19:6789/0 conn(0x7f42b8162310 legacy=0x7f42b80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c003a00 0x7f429c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c004800 0x7f429c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c00ddc0 0x7f429c0101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42b8070200 con 0x7f429c004800 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f429c010800 con 0x7f429c00ddc0 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.879+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f429c010a00 con 0x7f429c003a00 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b6000640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c004800 0x7f429c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b6a00640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c003a00 0x7f429c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b7400640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c00ddc0 0x7f429c0101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b6000640 1 -- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c003a00 msgr2=0x7f429c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b6000640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f429c003a00 0x7f429c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b6000640 1 -- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c00ddc0 msgr2=0x7f429c0101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b6000640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c00ddc0 0x7f429c0101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b6000640 1 -- 172.21.15.19:0/2176594335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c003520 con 0x7f429c004800 2024-04-25T11:37:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.880+0000 7f42b7400640 1 -- 172.21.15.19:0/2176594335 reap_dead start 2024-04-25T11:37:03.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.881+0000 7f42b6000640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c004800 0x7f429c004c10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f42a0001720 tx=0x7f429c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7766f594336fbd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:03.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.882+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0001370 con 0x7f429c004800 2024-04-25T11:37:03.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.882+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a00052d0 con 0x7f429c004800 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.882+0000 7f42b5600640 1 -- 172.21.15.19:0/2176594335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a002ab90 con 0x7f429c004800 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.883+0000 7f42bd800640 1 -- 172.21.15.19:0/2176594335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c004800 msgr2=0x7f429c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.883+0000 7f42bd800640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c004800 0x7f429c004c10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f42a0001720 tx=0x7f429c005830 comp rx=0 tx=0).stop 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.883+0000 7f42bd800640 1 -- 172.21.15.19:0/2176594335 shutdown_connections 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.883+0000 7f42bd800640 1 --2- 172.21.15.19:0/2176594335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f429c004800 0x7f429c004c10 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.883+0000 7f42bd800640 1 -- 172.21.15.19:0/2176594335 >> 172.21.15.19:0/2176594335 conn(0x7f42b8073050 msgr2=0x7f42b8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.883+0000 7f42bd800640 1 -- 172.21.15.19:0/2176594335 shutdown_connections 2024-04-25T11:37:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.883+0000 7f42bd800640 1 -- 172.21.15.19:0/2176594335 wait complete. 2024-04-25T11:37:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.885+0000 7f42bd800640 1 Processor -- start 2024-04-25T11:37:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.885+0000 7f42bd800640 1 -- start start 2024-04-25T11:37:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42bd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c004800 0x7f42b8136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42bd800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42b81f2b90 0x7f42b813e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42bd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b813ed20 0x7f42b8136c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42bd800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42b806f2e0 con 0x7f42b81f2b90 2024-04-25T11:37:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42bd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42b806f160 con 0x7f429c004800 2024-04-25T11:37:03.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42bd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f42b80cdfa0 con 0x7f42b813ed20 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42b6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42b81f2b90 0x7f42b813e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42b6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42b81f2b90 0x7f42b813e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60546/0 (socket says 172.21.15.19:60546) 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.887+0000 7f42b6000640 1 -- 172.21.15.19:0/4218705811 learned_addr learned my addr 172.21.15.19:0/4218705811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b7400640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b813ed20 0x7f42b8136c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b6a00640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c004800 0x7f42b8136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b6000640 1 -- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b813ed20 msgr2=0x7f42b8136c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b6000640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b813ed20 0x7f42b8136c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b6000640 1 -- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c004800 msgr2=0x7f42b8136670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b6000640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c004800 0x7f42b8136670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:03.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b6000640 1 -- 172.21.15.19:0/4218705811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b813f130 con 0x7f42b81f2b90 2024-04-25T11:37:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.888+0000 7f42b6000640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42b81f2b90 0x7f42b813e7e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f42a0000c00 tx=0x7f42a0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.889+0000 7f42aea00640 1 -- 172.21.15.19:0/4218705811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0037040 con 0x7f42b81f2b90 2024-04-25T11:37:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.889+0000 7f42aea00640 1 -- 172.21.15.19:0/4218705811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a0005480 con 0x7f42b81f2b90 2024-04-25T11:37:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.889+0000 7f42aea00640 1 -- 172.21.15.19:0/4218705811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0007a30 con 0x7f42b81f2b90 2024-04-25T11:37:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.889+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b81f5910 con 0x7f42b81f2b90 2024-04-25T11:37:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.889+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b81f5ed0 con 0x7f42b81f2b90 2024-04-25T11:37:03.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.890+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b8050740 con 0x7f42b81f2b90 2024-04-25T11:37:03.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.891+0000 7f42aea00640 1 -- 172.21.15.19:0/4218705811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f42a0044020 con 0x7f42b81f2b90 2024-04-25T11:37:03.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.892+0000 7f42aea00640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42940744b0 0x7f42940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:03.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.892+0000 7f42aea00640 1 -- 172.21.15.19:0/4218705811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7f42a00b53e0 con 0x7f42b81f2b90 2024-04-25T11:37:03.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.893+0000 7f42b6a00640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42940744b0 0x7f42940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:03.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.894+0000 7f42b6a00640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42940744b0 0x7f42940769c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f42a80070b0 tx=0x7f42a8007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:03.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:03.897+0000 7f42aea00640 1 -- 172.21.15.19:0/4218705811 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42b813f130 con 0x7f42b81f2b90 2024-04-25T11:37:04.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.188+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f42b806dae0 con 0x7f42b81f2b90 2024-04-25T11:37:04.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.567+0000 7f42aea00640 1 -- 172.21.15.19:0/4218705811 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v147) ==== 101+0+0 (secure 0 0 0) 0x7f42a007f7c0 con 0x7f42b81f2b90 2024-04-25T11:37:04.555 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:37:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42940744b0 msgr2=0x7f42940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42940744b0 0x7f42940769c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f42a80070b0 tx=0x7f42a8007040 comp rx=0 tx=0).stop 2024-04-25T11:37:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42b81f2b90 msgr2=0x7f42b813e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42b81f2b90 0x7f42b813e7e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f42a0000c00 tx=0x7f42a0001120 comp rx=0 tx=0).stop 2024-04-25T11:37:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 shutdown_connections 2024-04-25T11:37:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f42940744b0 0x7f42940769c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42b813ed20 0x7f42b8136c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42b81f2b90 0x7f42b813e7e0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 --2- 172.21.15.19:0/4218705811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f429c004800 0x7f42b8136670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.572+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 >> 172.21.15.19:0/4218705811 conn(0x7f42b8073050 msgr2=0x7f42b80cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.573+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 shutdown_connections 2024-04-25T11:37:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.573+0000 7f42bd800640 1 -- 172.21.15.19:0/4218705811 wait complete. 2024-04-25T11:37:04.590 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:37:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.783+0000 7f1126f97f40 1 Processor -- start 2024-04-25T11:37:04.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.783+0000 7f1126f97f40 1 -- start start 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.784+0000 7f1126f97f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec081e0 0x5646becdbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.784+0000 7f1126f97f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec085f0 0x5646bec06560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.784+0000 7f1126f97f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec06aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.784+0000 7f1126f97f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646bec50e70 con 0x5646bece1dd0 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.784+0000 7f1126f97f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646bebaed20 con 0x5646bec081e0 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.784+0000 7f1126f97f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5646bec513f0 con 0x5646bec085f0 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.784+0000 7f1120c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec06aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec06aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60566/0 (socket says 172.21.15.19:60566) 2024-04-25T11:37:04.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 -- 172.21.15.19:0/3120681743 learned_addr learned my addr 172.21.15.19:0/3120681743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:04.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 -- 172.21.15.19:0/3120681743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec085f0 msgr2=0x5646bec06560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:04.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 --2- 172.21.15.19:0/3120681743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec085f0 0x5646bec06560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 -- 172.21.15.19:0/3120681743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec081e0 msgr2=0x5646becdbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:04.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 --2- 172.21.15.19:0/3120681743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec081e0 0x5646becdbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 -- 172.21.15.19:0/3120681743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646becdc4d0 con 0x5646bece1dd0 2024-04-25T11:37:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.785+0000 7f1120c00640 1 --2- 172.21.15.19:0/3120681743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec06aa0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f111800a3b0 tx=0x7f111800a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941595e5d9f469ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f111e400640 1 -- 172.21.15.19:0/3120681743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800b8c0 con 0x5646bece1dd0 2024-04-25T11:37:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f111e400640 1 -- 172.21.15.19:0/3120681743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f111800f030 con 0x5646bece1dd0 2024-04-25T11:37:04.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f111e400640 1 -- 172.21.15.19:0/3120681743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800bcc0 con 0x5646bece1dd0 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f1126f97f40 1 -- 172.21.15.19:0/3120681743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 msgr2=0x5646bec06aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3120681743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec06aa0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f111800a3b0 tx=0x7f111800a870 comp rx=0 tx=0).stop 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f1126f97f40 1 -- 172.21.15.19:0/3120681743 shutdown_connections 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3120681743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec06aa0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3120681743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec085f0 0x5646bec06560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.786+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3120681743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec081e0 0x5646becdbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.787+0000 7f1126f97f40 1 -- 172.21.15.19:0/3120681743 >> 172.21.15.19:0/3120681743 conn(0x5646beb2aac0 msgr2=0x5646bec07350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.787+0000 7f1126f97f40 1 -- 172.21.15.19:0/3120681743 shutdown_connections 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.787+0000 7f1126f97f40 1 -- 172.21.15.19:0/3120681743 wait complete. 2024-04-25T11:37:04.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.788+0000 7f1126f97f40 1 Processor -- start 2024-04-25T11:37:04.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.788+0000 7f1126f97f40 1 -- start start 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.789+0000 7f1126f97f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec081e0 0x5646bec38c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.789+0000 7f1126f97f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec085f0 0x5646becdd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.789+0000 7f1126f97f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec4bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.789+0000 7f1126f97f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646bec4a610 con 0x5646bece1dd0 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.789+0000 7f1126f97f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646bec39530 con 0x5646bec085f0 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.789+0000 7f1126f97f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5646bed702a0 con 0x5646bec081e0 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec4bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:04.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec4bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60572/0 (socket says 172.21.15.19:60572) 2024-04-25T11:37:04.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 learned_addr learned my addr 172.21.15.19:0/3580920832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec081e0 0x5646bec38c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec085f0 0x5646becdd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec081e0 msgr2=0x5646bec38c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec081e0 0x5646bec38c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec085f0 msgr2=0x5646becdd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec085f0 0x5646becdd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646bec4eef0 con 0x5646bece1dd0 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646bec081e0 0x5646bec38c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.790+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646bec085f0 0x5646becdd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.791+0000 7f1120c00640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec4bc10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f111800ad70 tx=0x7f1118002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.791+0000 7f111d000640 1 -- 172.21.15.19:0/3580920832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118024070 con 0x5646bece1dd0 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.791+0000 7f111d000640 1 -- 172.21.15.19:0/3580920832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1118014420 con 0x5646bece1dd0 2024-04-25T11:37:04.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.791+0000 7f111d000640 1 -- 172.21.15.19:0/3580920832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118014830 con 0x5646bece1dd0 2024-04-25T11:37:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.791+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646becd94e0 con 0x5646bece1dd0 2024-04-25T11:37:04.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.791+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646becd9890 con 0x5646bece1dd0 2024-04-25T11:37:04.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.793+0000 7f111d000640 1 -- 172.21.15.19:0/3580920832 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1118027070 con 0x5646bece1dd0 2024-04-25T11:37:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.794+0000 7f111d000640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1100072e90 0x7f11000753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.794+0000 7f111d000640 1 -- 172.21.15.19:0/3580920832 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7f1118095120 con 0x5646bece1dd0 2024-04-25T11:37:04.782 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:37:04.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.795+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646bec4c150 0x5646bec4e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.795+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bec39190 con 0x5646bec4c150 2024-04-25T11:37:04.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.795+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1100072e90 0x7f11000753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:04.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.795+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646bec4c150 0x5646bec4e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.797+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646bec4c150 0x5646bec4e2b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:04.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.797+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1100072e90 0x7f11000753a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1110057000 tx=0x7f1110051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:04.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.800+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v101'257 uv231 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.800+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646becdc6a0 con 0x5646bec4c150 2024-04-25T11:37:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.803+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v101'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.803+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646becd9de0 con 0x5646bec4c150 2024-04-25T11:37:04.793 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:37:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.805+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v101'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.805+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646becdd5b0 con 0x5646bec4c150 2024-04-25T11:37:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.808+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.808+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed830c0 con 0x5646bec4c150 2024-04-25T11:37:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.811+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.811+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed820b0 con 0x5646bec4c150 2024-04-25T11:37:04.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.815+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.815+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82450 con 0x5646bec4c150 2024-04-25T11:37:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.818+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v101'263 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.818+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed827f0 con 0x5646bec4c150 2024-04-25T11:37:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.822+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v101'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.822+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82be0 con 0x5646bec4c150 2024-04-25T11:37:04.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.825+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v101'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.813 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:37:04.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.825+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82be0 con 0x5646bec4c150 2024-04-25T11:37:04.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.829+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'266 uv266 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.829+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.832+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.833+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.836+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.836+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.840+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v101'269 uv244 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.840+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.843+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v101'270 uv270 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.843+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.834 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:37:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.847+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v101'271 uv271 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.847+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.851+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v101'272 uv272 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.851+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.855+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v101'273 uv273 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.856+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed82b90 con 0x5646bec4c150 2024-04-25T11:37:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.860+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v101'274 uv274 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.860+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.864+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v101'275 uv275 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.864+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.869+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v101'276 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.869+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.873+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v101'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.873+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.865 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:37:04.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.877+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v101'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.877+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed84920 con 0x5646bec4c150 2024-04-25T11:37:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.882+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v101'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.882+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed84920 con 0x5646bec4c150 2024-04-25T11:37:04.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.887+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v101'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.887+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.891+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v101'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.891+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.896+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v101'282 uv256 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.896+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.901+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v101'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.901+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.905+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v101'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.893 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:37:04.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.905+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed84920 con 0x5646bec4c150 2024-04-25T11:37:04.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.910+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v101'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.910+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed84920 con 0x5646bec4c150 2024-04-25T11:37:04.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.914+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v101'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.914+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed83910 con 0x5646bec4c150 2024-04-25T11:37:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.919+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v101'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:04.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.919+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646bed83e20 0x5646bed86240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.919+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed86780 con 0x5646bed83e20 2024-04-25T11:37:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.920+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646bed83e20 0x5646bed86240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.922+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646bed83e20 0x5646bed86240 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.926+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v101'198 uv181 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.926+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed86b20 con 0x5646bed83e20 2024-04-25T11:37:04.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.930+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v101'199 uv199 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.931+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed86ec0 con 0x5646bed83e20 2024-04-25T11:37:04.922 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:37:04.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.935+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v101'200 uv200 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.935+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed87260 con 0x5646bed83e20 2024-04-25T11:37:04.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.939+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v101'201 uv201 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.939+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed87600 con 0x5646bed83e20 2024-04-25T11:37:04.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.944+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v101'202 uv202 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.944+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed879a0 con 0x5646bed83e20 2024-04-25T11:37:04.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.948+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v101'203 uv186 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.948+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed87d40 con 0x5646bed83e20 2024-04-25T11:37:04.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.952+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v101'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.952+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.944 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:37:04.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.957+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v101'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.957+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.962+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v101'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.962+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.966+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v101'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.966+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.970+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v101'208 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.970+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.974+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v101'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.974+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.966 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:37:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.979+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v101'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.979+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8c7d0 con 0x5646bed83e20 2024-04-25T11:37:04.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.983+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v101'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.984+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8c7d0 con 0x5646bed83e20 2024-04-25T11:37:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.987+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v101'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.988+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.991+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v101'213 uv197 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.991+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.996+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v101'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:04.996+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.987 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:37:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.000+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v101'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.000+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed890f0 con 0x5646bed83e20 2024-04-25T11:37:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.004+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v101'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.004+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed890f0 con 0x5646bed83e20 2024-04-25T11:37:04.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.008+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v101'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:04.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.008+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed880e0 con 0x5646bed83e20 2024-04-25T11:37:04.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.012+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v101'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed83e20 2024-04-25T11:37:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.012+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646bed885f0 0x5646bed8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.012+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8b070 con 0x5646bed885f0 2024-04-25T11:37:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.013+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646bed885f0 0x5646bed8aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.015+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646bed885f0 0x5646bed8aa10 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.019+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.019+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed8b490 con 0x5646bed885f0 2024-04-25T11:37:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.023+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.024+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8b490 con 0x5646bed885f0 2024-04-25T11:37:05.015 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:37:05.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.028+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.028+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8c4a0 con 0x5646bed885f0 2024-04-25T11:37:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.032+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.032+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8c4a0 con 0x5646bed885f0 2024-04-25T11:37:05.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.036+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.036+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8b490 con 0x5646bed885f0 2024-04-25T11:37:05.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.041+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.041+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8b490 con 0x5646bec4c150 2024-04-25T11:37:05.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.045+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'44 uv43 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.045+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed8b490 con 0x5646bec4c150 2024-04-25T11:37:05.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.050+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v101'45 uv45 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.050+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8b490 con 0x5646bec4c150 2024-04-25T11:37:05.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.054+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v101'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.041 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:37:05.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.054+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8b490 con 0x5646bec4c150 2024-04-25T11:37:05.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.058+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v101'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.058+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646bed8bd90 0x5646bed8e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.058+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8e6f0 con 0x5646bed8bd90 2024-04-25T11:37:05.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.059+0000 7f1120c00640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646bed8bd90 0x5646bed8e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.060+0000 7f1120c00640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646bed8bd90 0x5646bed8e1b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.064+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.064+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed8ea90 con 0x5646bed8bd90 2024-04-25T11:37:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.068+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.068+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8ef30 con 0x5646bed8bd90 2024-04-25T11:37:05.060 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:37:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.072+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.072+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed907e0 con 0x5646bed8bd90 2024-04-25T11:37:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.077+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.077+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed90b80 con 0x5646bed8bd90 2024-04-25T11:37:05.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.081+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.081+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed8f7d0 con 0x5646bed8bd90 2024-04-25T11:37:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.085+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.085+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646bed900f0 0x5646bed92f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.085+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed93470 con 0x5646bed900f0 2024-04-25T11:37:05.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.086+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646bed900f0 0x5646bed92f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.088+0000 7f1120200640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646bed900f0 0x5646bed92f30 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.092+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v101'94 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.092+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed93470 con 0x5646bed900f0 2024-04-25T11:37:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.096+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v101'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.096+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed934f0 con 0x5646bed900f0 2024-04-25T11:37:05.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.100+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v101'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.087 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:37:05.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.100+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed948f0 con 0x5646bed900f0 2024-04-25T11:37:05.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.104+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v101'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.104+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed948f0 con 0x5646bed900f0 2024-04-25T11:37:05.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.109+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v101'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.109+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed938e0 con 0x5646bed900f0 2024-04-25T11:37:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.113+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v101'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.113+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed938e0 con 0x5646bed900f0 2024-04-25T11:37:05.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.117+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v101'100 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.117+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed938e0 con 0x5646bed900f0 2024-04-25T11:37:05.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.121+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v101'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.121+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed938e0 con 0x5646bed900f0 2024-04-25T11:37:05.112 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:37:05.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.125+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v101'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.125+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed938e0 con 0x5646bed900f0 2024-04-25T11:37:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.129+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v101'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.129+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed938e0 con 0x5646bed900f0 2024-04-25T11:37:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.133+0000 7f1120200640 1 -- 172.21.15.19:0/3580920832 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v101'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1110051070 con 0x5646bed900f0 2024-04-25T11:37:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.133+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646bed93df0 0x5646bed96210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.133+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed968c0 con 0x5646bed93df0 2024-04-25T11:37:05.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.134+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646bed93df0 0x5646bed96210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.136+0000 7f111f800640 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646bed93df0 0x5646bed96210 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.127 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:37:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.140+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v101'43 uv43 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.140+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed93df0 2024-04-25T11:37:05.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.144+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [write 0~70] v101'44 uv44 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.145+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed93df0 2024-04-25T11:37:05.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.148+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v101'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.148+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:83 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bec4c150 2024-04-25T11:37:05.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.153+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(83 605.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.153+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:84 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bec4c150 2024-04-25T11:37:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.157+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(84 605.00000000 [delete] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.157+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bec4c150 2024-04-25T11:37:05.148 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:37:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.160+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(85 605.00000000 [create] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.161+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97ca0 con 0x5646bec4c150 2024-04-25T11:37:05.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.165+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.165+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97ca0 con 0x5646bec4c150 2024-04-25T11:37:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.169+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.169+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bec4c150 2024-04-25T11:37:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.173+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(88 605.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bec4c150 2024-04-25T11:37:05.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.173+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:89 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.177+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(89 mds0_inotable [create] v101'84 uv78 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.177+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:90 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.181+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(90 mds0_inotable [delete] v101'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.181+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.172 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:37:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.185+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(91 mds0_inotable [create] v101'86 uv86 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.185+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.189+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v101'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.190+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.194+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v101'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.194+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.198+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(94 100.00000000 [create] v101'89 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.198+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.202+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(95 100.00000000 [delete] v101'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.202+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.193 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:37:05.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.206+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(96 100.00000000 [create] v101'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.206+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed96c90 con 0x5646bed885f0 2024-04-25T11:37:05.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.210+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v101'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.210+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed99a80 con 0x5646bed885f0 2024-04-25T11:37:05.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.215+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v101'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed885f0 2024-04-25T11:37:05.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.215+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed99a40 con 0x5646bed8bd90 2024-04-25T11:37:05.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.219+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v101'40 uv39 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.219+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed8bd90 2024-04-25T11:37:05.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.223+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v101'41 uv41 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.223+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed8bd90 2024-04-25T11:37:05.215 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:37:05.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.227+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v101'42 uv42 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.227+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed8bd90 2024-04-25T11:37:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.232+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v101'43 uv43 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.232+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed8bd90 2024-04-25T11:37:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.236+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v101'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.236+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed8bd90 2024-04-25T11:37:05.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.240+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.240+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed8bd90 2024-04-25T11:37:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.244+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.232 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:37:05.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.244+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed98090 con 0x5646bed8bd90 2024-04-25T11:37:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.248+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v101'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.248+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed98090 con 0x5646bed8bd90 2024-04-25T11:37:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.252+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v101'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.252+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed8bd90 2024-04-25T11:37:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.257+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v101'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1118026030 con 0x5646bed8bd90 2024-04-25T11:37:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.257+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97080 con 0x5646bed93df0 2024-04-25T11:37:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.260+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(109 400.00000000 [create] v101'42 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.261+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed97030 con 0x5646bed93df0 2024-04-25T11:37:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.264+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(110 400.00000000 [delete] v101'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.264+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97030 con 0x5646bed93df0 2024-04-25T11:37:05.255 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:37:05.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.268+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v101'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.268+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97030 con 0x5646bed93df0 2024-04-25T11:37:05.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.272+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [write 0~22] v101'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.272+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.276+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [omap-set-header] v101'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.276+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:05.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.281+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 100.00000000.inode [create] v101'107 uv99 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.281+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.277 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:05.278 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:37:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.285+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 100.00000000.inode [delete] v101'108 uv108 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.285+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.289+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v101'109 uv109 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.289+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.294+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v101'110 uv110 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.294+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.298+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v101'111 uv111 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.298+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.301+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 1.00000000 [create] v101'112 uv106 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.302+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.305+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 1.00000000 [delete] v101'113 uv113 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.305+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.308+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v101'114 uv114 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.296 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:37:05.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.308+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.312+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v101'115 uv115 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.312+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.315+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v101'116 uv116 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.315+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed97000 con 0x5646bed93df0 2024-04-25T11:37:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.317+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [omap-set-header] v101'117 uv117 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.317+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x5646bed978f0 con 0x5646bed93df0 2024-04-25T11:37:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f111f800640 1 -- 172.21.15.19:0/3580920832 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v101'118 uv118 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f110c00b5d0 con 0x5646bed93df0 2024-04-25T11:37:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646bed83e20 msgr2=0x5646bed86240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646bed83e20 0x5646bed86240 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646bed900f0 msgr2=0x5646bed92f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646bed900f0 0x5646bed92f30 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646bed93df0 msgr2=0x5646bed96210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646bed93df0 0x5646bed96210 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646bed885f0 msgr2=0x5646bed8aa10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646bed885f0 0x5646bed8aa10 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 reap_dead start 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646bec4c150 msgr2=0x5646bec4e2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646bec4c150 0x5646bec4e2b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1120c00640 1 -- 172.21.15.19:0/3580920832 reap_dead start 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646bed8bd90 msgr2=0x5646bed8e1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646bed8bd90 0x5646bed8e1b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1100072e90 msgr2=0x7f11000753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1100072e90 0x7f11000753a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1110057000 tx=0x7f1110051000 comp rx=0 tx=0).stop 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 msgr2=0x5646bec4bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.320+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec4bc10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f111800ad70 tx=0x7f1118002f70 comp rx=0 tx=0).stop 2024-04-25T11:37:05.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.321+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 shutdown_connections 2024-04-25T11:37:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.321+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646bed8bd90 0x5646bed8e1b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.321+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1100072e90 0x7f11000753a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.321+0000 7f1126f97f40 1 --2- 172.21.15.19:0/3580920832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646bece1dd0 0x5646bec4bc10 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.321+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 >> 172.21.15.19:0/3580920832 conn(0x5646beb2aac0 msgr2=0x5646bec07330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.321+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 shutdown_connections 2024-04-25T11:37:05.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.321+0000 7f1126f97f40 1 -- 172.21.15.19:0/3580920832 wait complete. 2024-04-25T11:37:05.327 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2024-04-25T11:37:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.559+0000 7f7daa335f40 1 Processor -- start 2024-04-25T11:37:05.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.559+0000 7f7daa335f40 1 -- start start 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7daa335f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 0x564f0ab33e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7daa335f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0aa655f0 0x564f0ab3bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7daa335f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0ab3edd0 0x564f0ab3c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7daa335f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f0aaade70 con 0x564f0aa655f0 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7daa335f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564f0aa0bd20 con 0x564f0ab3edd0 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7daa335f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564f0aaae3f0 con 0x564f0aa651e0 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7da2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0aa655f0 0x564f0ab3bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7da2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0aa655f0 0x564f0ab3bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60588/0 (socket says 172.21.15.19:60588) 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.561+0000 7f7da2a00640 1 -- 172.21.15.19:0/2492528052 learned_addr learned my addr 172.21.15.19:0/2492528052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da3400640 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 0x564f0ab33e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da3e00640 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0ab3edd0 0x564f0ab3c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da2a00640 1 -- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 msgr2=0x564f0ab33e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da2a00640 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 0x564f0ab33e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da2a00640 1 -- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0ab3edd0 msgr2=0x564f0ab3c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da2a00640 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0ab3edd0 0x564f0ab3c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da2a00640 1 -- 172.21.15.19:0/2492528052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0ab3bd90 con 0x564f0aa655f0 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da2a00640 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0aa655f0 0x564f0ab3bfd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00a7a0 tx=0x7f7d8c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76b6e855e9fc0f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.562+0000 7f7da1600640 1 -- 172.21.15.19:0/2492528052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c018070 con 0x564f0aa655f0 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7da1600640 1 -- 172.21.15.19:0/2492528052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8c007600 con 0x564f0aa655f0 2024-04-25T11:37:05.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7da1600640 1 -- 172.21.15.19:0/2492528052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0079b0 con 0x564f0aa655f0 2024-04-25T11:37:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 -- 172.21.15.19:0/2492528052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0aa655f0 msgr2=0x564f0ab3bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0aa655f0 0x564f0ab3bfd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00a7a0 tx=0x7f7d8c00a890 comp rx=0 tx=0).stop 2024-04-25T11:37:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 -- 172.21.15.19:0/2492528052 shutdown_connections 2024-04-25T11:37:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0ab3edd0 0x564f0ab3c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0aa655f0 0x564f0ab3bfd0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 --2- 172.21.15.19:0/2492528052 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 0x564f0ab33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 -- 172.21.15.19:0/2492528052 >> 172.21.15.19:0/2492528052 conn(0x564f0a987ac0 msgr2=0x564f0aa974c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.563+0000 7f7daa335f40 1 -- 172.21.15.19:0/2492528052 shutdown_connections 2024-04-25T11:37:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.564+0000 7f7daa335f40 1 -- 172.21.15.19:0/2492528052 wait complete. 2024-04-25T11:37:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.565+0000 7f7daa335f40 1 Processor -- start 2024-04-25T11:37:05.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.565+0000 7f7daa335f40 1 -- start start 2024-04-25T11:37:05.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.566+0000 7f7daa335f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 0x564f0ab32e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.566+0000 7f7daa335f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0aa655f0 0x564f0aaa7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.566+0000 7f7daa335f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0ab3edd0 0x564f0aaaffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.566+0000 7f7daa335f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564f0ab39dd0 con 0x564f0ab3edd0 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.566+0000 7f7daa335f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564f0ab3a0b0 con 0x564f0aa655f0 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.566+0000 7f7daa335f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564f0ab3a390 con 0x564f0aa651e0 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0ab3edd0 0x564f0aaaffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0ab3edd0 0x564f0aaaffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60594/0 (socket says 172.21.15.19:60594) 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 -- 172.21.15.19:0/3842271836 learned_addr learned my addr 172.21.15.19:0/3842271836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3400640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 0x564f0ab32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 -- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 msgr2=0x564f0ab32e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564f0aa651e0 0x564f0ab32e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 -- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0aa655f0 msgr2=0x564f0aaa7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564f0aa655f0 0x564f0aaa7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.567+0000 7f7da3e00640 1 -- 172.21.15.19:0/3842271836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f0aaa9980 con 0x564f0ab3edd0 2024-04-25T11:37:05.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.568+0000 7f7da3e00640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0ab3edd0 0x564f0aaaffe0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c002200 tx=0x7f7d9c00d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.568+0000 7f7d9be00640 1 -- 172.21.15.19:0/3842271836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c018070 con 0x564f0ab3edd0 2024-04-25T11:37:05.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.568+0000 7f7d9be00640 1 -- 172.21.15.19:0/3842271836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c0060d0 con 0x564f0ab3edd0 2024-04-25T11:37:05.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.568+0000 7f7d9be00640 1 -- 172.21.15.19:0/3842271836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c004500 con 0x564f0ab3edd0 2024-04-25T11:37:05.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.568+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564f0ab349c0 con 0x564f0ab3edd0 2024-04-25T11:37:05.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.568+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f0ab34f80 con 0x564f0ab3edd0 2024-04-25T11:37:05.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.571+0000 7f7d9be00640 1 -- 172.21.15.19:0/3842271836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d9c01b070 con 0x564f0ab3edd0 2024-04-25T11:37:05.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.573+0000 7f7d9be00640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d80072ec0 0x7f7d800753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.573+0000 7f7d9be00640 1 -- 172.21.15.19:0/3842271836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7f7d9c093aa0 con 0x564f0ab3edd0 2024-04-25T11:37:05.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.573+0000 7f7daa335f40 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564f0aaa80e0 0x564f0ab39140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:05.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.573+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101) -- 0x564f0ab39680 con 0x564f0aaa80e0 2024-04-25T11:37:05.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.573+0000 7f7da3400640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d80072ec0 0x7f7d800753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.574+0000 7f7da2a00640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564f0aaa80e0 0x564f0ab39140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:05.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.575+0000 7f7da3400640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d80072ec0 0x7f7d800753d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7d94056000 tx=0x7f7d94050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.576+0000 7f7da2a00640 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564f0aaa80e0 0x564f0ab39140 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7da2a00640 1 -- 172.21.15.19:0/3842271836 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 mds0_inotable [delete] v101'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7d8c01b040 con 0x564f0aaa80e0 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564f0aaa80e0 msgr2=0x564f0ab39140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564f0aaa80e0 0x564f0ab39140 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d80072ec0 msgr2=0x7f7d800753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d80072ec0 0x7f7d800753d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7d94056000 tx=0x7f7d94050000 comp rx=0 tx=0).stop 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0ab3edd0 msgr2=0x564f0aaaffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 --2- 172.21.15.19:0/3842271836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564f0ab3edd0 0x564f0aaaffe0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c002200 tx=0x7f7d9c00d5f0 comp rx=0 tx=0).stop 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7da3e00640 1 -- 172.21.15.19:0/3842271836 reap_dead start 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 shutdown_connections 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 >> 172.21.15.19:0/3842271836 conn(0x564f0a987ac0 msgr2=0x564f0ab33e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 shutdown_connections 2024-04-25T11:37:05.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:05.580+0000 7f7daa335f40 1 -- 172.21.15.19:0/3842271836 wait complete. 2024-04-25T11:37:05.582 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:37:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.158+0000 7f2bf2e00640 1 Processor -- start 2024-04-25T11:37:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.158+0000 7f2bf2e00640 1 -- start start 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.160+0000 7f2bf2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec161710 0x7f2bec15e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.160+0000 7f2bf2e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2bec06c5c0 0x7f2bec06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.160+0000 7f2bf2e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bec1609b0 con 0x7f2bec163fa0 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.160+0000 7f2bf2e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2bec06cf10 con 0x7f2bec06c5c0 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.160+0000 7f2bf2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2bec164cd0 con 0x7f2bec161710 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.161+0000 7f2bebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec161710 0x7f2bec15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.161+0000 7f2bebe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec161710 0x7f2bec15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52640/0 (socket says 172.21.15.19:52640) 2024-04-25T11:37:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.161+0000 7f2bebe00640 1 -- 172.21.15.19:0/2954463496 learned_addr learned my addr 172.21.15.19:0/2954463496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.161+0000 7f2be3400640 1 --2- 172.21.15.19:0/2954463496 >> v2:172.21.15.169:3300/0 conn(0x7f2bec06c5c0 0x7f2bec06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.162+0000 7f2beb400640 1 -- 172.21.15.19:0/2954463496 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698637141 0 0) 0x7f2be4001110 con 0x7f2bec163fa0 2024-04-25T11:37:06.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.162+0000 7f2be3400640 1 -- 172.21.15.19:0/2954463496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec161710 msgr2=0x7f2bec15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:06.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.162+0000 7f2be3400640 1 --2- 172.21.15.19:0/2954463496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec161710 0x7f2bec15e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.162+0000 7f2be3400640 1 -- 172.21.15.19:0/2954463496 >> v1:172.21.15.169:6789/0 conn(0x7f2bec163fa0 legacy=0x7f2bec1643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.162+0000 7f2be3400640 1 -- 172.21.15.19:0/2954463496 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec160c30 con 0x7f2bec06c5c0 2024-04-25T11:37:06.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.164+0000 7f2be3400640 1 --2- 172.21.15.19:0/2954463496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec06c5c0 0x7f2bec06c9d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097a0 tx=0x7f2bd402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a8efdfff96563d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:06.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.164+0000 7f2beb400640 1 -- 172.21.15.19:0/2954463496 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4037070 con 0x7f2bec06c5c0 2024-04-25T11:37:06.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.164+0000 7f2beb400640 1 -- 172.21.15.19:0/2954463496 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd402ae50 con 0x7f2bec06c5c0 2024-04-25T11:37:06.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.164+0000 7f2beb400640 1 -- 172.21.15.19:0/2954463496 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4032660 con 0x7f2bec06c5c0 2024-04-25T11:37:06.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2954463496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec06c5c0 msgr2=0x7f2bec06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2954463496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec06c5c0 0x7f2bec06c9d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097a0 tx=0x7f2bd402a100 comp rx=0 tx=0).stop 2024-04-25T11:37:06.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2954463496 shutdown_connections 2024-04-25T11:37:06.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2954463496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec06c5c0 0x7f2bec06c9d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2954463496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec161710 0x7f2bec15e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2954463496 >> 172.21.15.19:0/2954463496 conn(0x7f2bec0d5d90 msgr2=0x7f2bec0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:06.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2954463496 shutdown_connections 2024-04-25T11:37:06.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.165+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2954463496 wait complete. 2024-04-25T11:37:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.168+0000 7f2bf2e00640 1 Processor -- start 2024-04-25T11:37:06.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.168+0000 7f2bf2e00640 1 -- start start 2024-04-25T11:37:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.169+0000 7f2bf2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec06c5c0 0x7f2bec152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.169+0000 7f2bf2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bec161710 0x7f2bec1f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.169+0000 7f2bf2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec163fa0 0x7f2bec0bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.169+0000 7f2bf2e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec0d49f0 con 0x7f2bec161710 2024-04-25T11:37:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.169+0000 7f2bf2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1611f0 con 0x7f2bec163fa0 2024-04-25T11:37:06.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.169+0000 7f2bf2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2bec0d4b70 con 0x7f2bec06c5c0 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bec161710 0x7f2bec1f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bec161710 0x7f2bec1f9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60608/0 (socket says 172.21.15.19:60608) 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 -- 172.21.15.19:0/2578521757 learned_addr learned my addr 172.21.15.19:0/2578521757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2bebe00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec06c5c0 0x7f2bec152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2bf0e00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec163fa0 0x7f2bec0bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 -- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec06c5c0 msgr2=0x7f2bec152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec06c5c0 0x7f2bec152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 -- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec163fa0 msgr2=0x7f2bec0bafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec163fa0 0x7f2bec0bafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.170+0000 7f2be3400640 1 -- 172.21.15.19:0/2578521757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec152bb0 con 0x7f2bec161710 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.171+0000 7f2be3400640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bec161710 0x7f2bec1f9380 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40098c0 tx=0x7f2bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.171+0000 7f2be8c00640 1 -- 172.21.15.19:0/2578521757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4042020 con 0x7f2bec161710 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.171+0000 7f2be8c00640 1 -- 172.21.15.19:0/2578521757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd403d470 con 0x7f2bec161710 2024-04-25T11:37:06.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.171+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec152e90 con 0x7f2bec161710 2024-04-25T11:37:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.171+0000 7f2be8c00640 1 -- 172.21.15.19:0/2578521757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4037040 con 0x7f2bec161710 2024-04-25T11:37:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.171+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec153450 con 0x7f2bec161710 2024-04-25T11:37:06.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.173+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bec0cf790 con 0x7f2bec161710 2024-04-25T11:37:06.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.174+0000 7f2be8c00640 1 -- 172.21.15.19:0/2578521757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2bd4045020 con 0x7f2bec161710 2024-04-25T11:37:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.175+0000 7f2be8c00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bc80744e0 0x7f2bc80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.175+0000 7f2be8c00640 1 -- 172.21.15.19:0/2578521757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6483+0+0 (secure 0 0 0) 0x7f2bd40b5cf0 con 0x7f2bec161710 2024-04-25T11:37:06.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.175+0000 7f2bebe00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bc80744e0 0x7f2bc80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.177+0000 7f2bebe00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bc80744e0 0x7f2bc80769f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2bec08e250 tx=0x7f2bdc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.180+0000 7f2be8c00640 1 -- 172.21.15.19:0/2578521757 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bd404c030 con 0x7f2bec161710 2024-04-25T11:37:06.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.425+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2bec050740 con 0x7f2bec161710 2024-04-25T11:37:06.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.583+0000 7f2be8c00640 1 -- 172.21.15.19:0/2578521757 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v148) ==== 162+0+0 (secure 0 0 0) 0x7f2bd40800d0 con 0x7f2bec161710 2024-04-25T11:37:06.572 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:37:06.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bc80744e0 msgr2=0x7f2bc80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bc80744e0 0x7f2bc80769f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2bec08e250 tx=0x7f2bdc00b040 comp rx=0 tx=0).stop 2024-04-25T11:37:06.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bec161710 msgr2=0x7f2bec1f9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bec161710 0x7f2bec1f9380 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40098c0 tx=0x7f2bd40046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 shutdown_connections 2024-04-25T11:37:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2bc80744e0 0x7f2bc80769f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2bec163fa0 0x7f2bec0bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bec161710 0x7f2bec1f9380 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 --2- 172.21.15.19:0/2578521757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2bec06c5c0 0x7f2bec152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.587+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 >> 172.21.15.19:0/2578521757 conn(0x7f2bec0d5d90 msgr2=0x7f2bec15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.588+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 shutdown_connections 2024-04-25T11:37:06.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.588+0000 7f2bf2e00640 1 -- 172.21.15.19:0/2578521757 wait complete. 2024-04-25T11:37:06.589 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:06.602+0000 7ff923000640 -1 mds.0.inotable: load_2 could not read table: -2 2024-04-25T11:37:06.589 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:06.602+0000 7ff923000640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-04-25T11:37:06.590 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:06.603+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:37:06.590 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:06.603+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:37:06.601 INFO:tasks.ceph.mds.a.smithi019.stderr: -25> 2024-04-25T11:37:06.602+0000 7ff923000640 -1 mds.0.inotable: load_2 could not read table: -2 2024-04-25T11:37:06.602 INFO:tasks.ceph.mds.a.smithi019.stderr: -24> 2024-04-25T11:37:06.602+0000 7ff923000640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-04-25T11:37:06.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:06.735 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:37:06.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.888+0000 7f1142a00640 1 Processor -- start 2024-04-25T11:37:06.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.889+0000 7f1142a00640 1 -- start start 2024-04-25T11:37:06.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.891+0000 7f1142a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c070680 0x7f113c1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.891+0000 7f1142a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f113c160360 0x7f113c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.891+0000 7f1142a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c06c7d0 con 0x7f113c0af9e0 2024-04-25T11:37:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.891+0000 7f1142a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f113c06d100 con 0x7f113c160360 2024-04-25T11:37:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.891+0000 7f1142a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f113c06cab0 con 0x7f113c070680 2024-04-25T11:37:06.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.892+0000 7f113be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c070680 0x7f113c1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.892+0000 7f113be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c070680 0x7f113c1647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52674/0 (socket says 172.21.15.19:52674) 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.892+0000 7f113be00640 1 -- 172.21.15.19:0/3622375339 learned_addr learned my addr 172.21.15.19:0/3622375339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.892+0000 7f1140a00640 1 --1- 172.21.15.19:0/3622375339 >> v1:172.21.15.169:6789/0 conn(0x7f113c0af9e0 0x7f113c0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:44508/0 (socket says 172.21.15.19:44508) 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.892+0000 7f1133e00640 1 --2- 172.21.15.19:0/3622375339 >> v2:172.21.15.169:3300/0 conn(0x7f113c160360 0x7f113c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.893+0000 7f113be00640 1 -- 172.21.15.19:0/3622375339 >> v2:172.21.15.169:3300/0 conn(0x7f113c160360 msgr2=0x7f113c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.893+0000 7f113be00640 1 --2- 172.21.15.19:0/3622375339 >> v2:172.21.15.169:3300/0 conn(0x7f113c160360 0x7f113c0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.893+0000 7f113be00640 1 -- 172.21.15.19:0/3622375339 >> v1:172.21.15.169:6789/0 conn(0x7f113c0af9e0 legacy=0x7f113c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.893+0000 7f113be00640 1 -- 172.21.15.19:0/3622375339 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c06cd90 con 0x7f113c070680 2024-04-25T11:37:06.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.894+0000 7f113be00640 1 --2- 172.21.15.19:0/3622375339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c070680 0x7f113c1647f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f112c009870 tx=0x7f112c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a4abca3d822d007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.894+0000 7f113b400640 1 -- 172.21.15.19:0/3622375339 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c037070 con 0x7f113c070680 2024-04-25T11:37:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f113b400640 1 -- 172.21.15.19:0/3622375339 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112c032440 con 0x7f113c070680 2024-04-25T11:37:06.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f113b400640 1 -- 172.21.15.19:0/3622375339 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c032810 con 0x7f113c070680 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f1142a00640 1 -- 172.21.15.19:0/3622375339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c070680 msgr2=0x7f113c1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f1142a00640 1 --2- 172.21.15.19:0/3622375339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c070680 0x7f113c1647f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f112c009870 tx=0x7f112c02a130 comp rx=0 tx=0).stop 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f1142a00640 1 -- 172.21.15.19:0/3622375339 shutdown_connections 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f1142a00640 1 --2- 172.21.15.19:0/3622375339 >> v2:172.21.15.169:3300/0 conn(0x7f113c160360 0x7f113c0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f1142a00640 1 --2- 172.21.15.19:0/3622375339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c070680 0x7f113c1647f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.895+0000 7f1142a00640 1 -- 172.21.15.19:0/3622375339 >> 172.21.15.19:0/3622375339 conn(0x7f113c0d1ae0 msgr2=0x7f113c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.896+0000 7f1142a00640 1 -- 172.21.15.19:0/3622375339 shutdown_connections 2024-04-25T11:37:06.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.896+0000 7f1142a00640 1 -- 172.21.15.19:0/3622375339 wait complete. 2024-04-25T11:37:06.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.898+0000 7f1142a00640 1 Processor -- start 2024-04-25T11:37:06.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.898+0000 7f1142a00640 1 -- start start 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1142a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f113c070680 0x7f113c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1142a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c0af9e0 0x7f113c14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1142a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113c160360 0x7f113c14df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1142a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f113c0b6520 con 0x7f113c160360 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1142a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f113c0b63a0 con 0x7f113c070680 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1142a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f113c06d590 con 0x7f113c0af9e0 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1140a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113c160360 0x7f113c14df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1140a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113c160360 0x7f113c14df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60634/0 (socket says 172.21.15.19:60634) 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1140a00640 1 -- 172.21.15.19:0/3966832773 learned_addr learned my addr 172.21.15.19:0/3966832773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f113be00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f113c070680 0x7f113c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.900+0000 7f1133e00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c0af9e0 0x7f113c14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.901+0000 7f1140a00640 1 -- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c0af9e0 msgr2=0x7f113c14bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.901+0000 7f1140a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c0af9e0 0x7f113c14bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.901+0000 7f1140a00640 1 -- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f113c070680 msgr2=0x7f113c15b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.901+0000 7f1140a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f113c070680 0x7f113c15b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.901+0000 7f1140a00640 1 -- 172.21.15.19:0/3966832773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c15e050 con 0x7f113c160360 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.901+0000 7f1140a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113c160360 0x7f113c14df30 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f113400c6c0 tx=0x7f113400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.902+0000 7f1138c00640 1 -- 172.21.15.19:0/3966832773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134018070 con 0x7f113c160360 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.902+0000 7f1138c00640 1 -- 172.21.15.19:0/3966832773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1134004d60 con 0x7f113c160360 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.902+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c15e330 con 0x7f113c160360 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.902+0000 7f1138c00640 1 -- 172.21.15.19:0/3966832773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113400a840 con 0x7f113c160360 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.902+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c15e8f0 con 0x7f113c160360 2024-04-25T11:37:06.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.903+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113c14ea60 con 0x7f113c160360 2024-04-25T11:37:06.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.904+0000 7f1138c00640 1 -- 172.21.15.19:0/3966832773 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f113401b070 con 0x7f113c160360 2024-04-25T11:37:06.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.905+0000 7f1138c00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1118074530 0x7f1118076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:06.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.905+0000 7f1138c00640 1 -- 172.21.15.19:0/3966832773 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f1134094a80 con 0x7f113c160360 2024-04-25T11:37:06.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.906+0000 7f113be00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1118074530 0x7f1118076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:06.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.907+0000 7f113be00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1118074530 0x7f1118076a40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f112c034000 tx=0x7f112c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:06.910+0000 7f1138c00640 1 -- 172.21.15.19:0/3966832773 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1134096e20 con 0x7f113c160360 2024-04-25T11:37:07.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.158+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f113c050740 con 0x7f113c160360 2024-04-25T11:37:07.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.159+0000 7f1138c00640 1 -- 172.21.15.19:0/3966832773 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) ==== 95+0+4352 (secure 0 0 0) 0x7f113405e840 con 0x7f113c160360 2024-04-25T11:37:07.146 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:07.146 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":150,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:06.603724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 150 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1118074530 msgr2=0x7f1118076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1118074530 0x7f1118076a40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f112c034000 tx=0x7f112c02e000 comp rx=0 tx=0).stop 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113c160360 msgr2=0x7f113c14df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113c160360 0x7f113c14df30 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f113400c6c0 tx=0x7f113400cb80 comp rx=0 tx=0).stop 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 shutdown_connections 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1118074530 0x7f1118076a40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f113c160360 0x7f113c14df30 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f113c0af9e0 0x7f113c14bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 --2- 172.21.15.19:0/3966832773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f113c070680 0x7f113c15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 >> 172.21.15.19:0/3966832773 conn(0x7f113c0d1ae0 msgr2=0x7f113c163d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:07.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.161+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 shutdown_connections 2024-04-25T11:37:07.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.162+0000 7f1142a00640 1 -- 172.21.15.19:0/3966832773 wait complete. 2024-04-25T11:37:07.174 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:07.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:07.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.468+0000 7f28a6a00640 1 Processor -- start 2024-04-25T11:37:07.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.468+0000 7f28a6a00640 1 -- start start 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f28a0164860 0x7f28a00b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f28a00d44a0 0x7f28a00d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28a0161650 con 0x7f28a00b7540 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a6a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f28a01643d0 con 0x7f28a0164860 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a6a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f28a00b5270 con 0x7f28a00d44a0 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a5600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f28a0164860 0x7f28a00b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a5600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f28a0164860 0x7f28a00b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60638/0 (socket says 172.21.15.19:60638) 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.470+0000 7f28a5600640 1 -- 172.21.15.19:0/3450869474 learned_addr learned my addr 172.21.15.19:0/3450869474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.471+0000 7f28a5600640 1 -- 172.21.15.19:0/3450869474 >> v2:172.21.15.169:3300/0 conn(0x7f28a00d44a0 msgr2=0x7f28a00d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:07.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.471+0000 7f28a5600640 1 --2- 172.21.15.19:0/3450869474 >> v2:172.21.15.169:3300/0 conn(0x7f28a00d44a0 0x7f28a00d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.471+0000 7f28a5600640 1 -- 172.21.15.19:0/3450869474 >> v1:172.21.15.169:6789/0 conn(0x7f28a00b7540 legacy=0x7f28a00b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.471+0000 7f28a5600640 1 -- 172.21.15.19:0/3450869474 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a0161370 con 0x7f28a0164860 2024-04-25T11:37:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.472+0000 7f28a5600640 1 --2- 172.21.15.19:0/3450869474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a0164860 0x7f28a00b4d30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f28a0164340 tx=0x7f289402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ecc2add60fa456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:07.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.472+0000 7f289fe00640 1 -- 172.21.15.19:0/3450869474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894037070 con 0x7f28a0164860 2024-04-25T11:37:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.472+0000 7f289fe00640 1 -- 172.21.15.19:0/3450869474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2894032440 con 0x7f28a0164860 2024-04-25T11:37:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.472+0000 7f289fe00640 1 -- 172.21.15.19:0/3450869474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894032810 con 0x7f28a0164860 2024-04-25T11:37:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 -- 172.21.15.19:0/3450869474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a0164860 msgr2=0x7f28a00b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:07.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 --2- 172.21.15.19:0/3450869474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a0164860 0x7f28a00b4d30 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f28a0164340 tx=0x7f289402a230 comp rx=0 tx=0).stop 2024-04-25T11:37:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 -- 172.21.15.19:0/3450869474 shutdown_connections 2024-04-25T11:37:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 --2- 172.21.15.19:0/3450869474 >> v2:172.21.15.169:3300/0 conn(0x7f28a00d44a0 0x7f28a00d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 --2- 172.21.15.19:0/3450869474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a0164860 0x7f28a00b4d30 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 -- 172.21.15.19:0/3450869474 >> 172.21.15.19:0/3450869474 conn(0x7f28a00d2720 msgr2=0x7f28a00d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 -- 172.21.15.19:0/3450869474 shutdown_connections 2024-04-25T11:37:07.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.473+0000 7f28a6a00640 1 -- 172.21.15.19:0/3450869474 wait complete. 2024-04-25T11:37:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.475+0000 7f28a6a00640 1 Processor -- start 2024-04-25T11:37:07.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.476+0000 7f28a6a00640 1 -- start start 2024-04-25T11:37:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00b7540 0x7f28a01540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a00d44a0 0x7f28a01f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a0164860 0x7f28a0159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a6a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28a00d4df0 con 0x7f28a00b7540 2024-04-25T11:37:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0164060 con 0x7f28a0164860 2024-04-25T11:37:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f28a00d4f70 con 0x7f28a00d44a0 2024-04-25T11:37:07.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00b7540 0x7f28a01540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00b7540 0x7f28a01540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60644/0 (socket says 172.21.15.19:60644) 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.477+0000 7f28a5600640 1 -- 172.21.15.19:0/2622384429 learned_addr learned my addr 172.21.15.19:0/2622384429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a5600640 1 -- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a00d44a0 msgr2=0x7f28a01f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a4c00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a00d44a0 0x7f28a01f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a6000640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a0164860 0x7f28a0159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a5600640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a00d44a0 0x7f28a01f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a5600640 1 -- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a0164860 msgr2=0x7f28a0159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a5600640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a0164860 0x7f28a0159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a5600640 1 -- 172.21.15.19:0/2622384429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a015ba10 con 0x7f28a00b7540 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a4c00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a00d44a0 0x7f28a01f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a6000640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a0164860 0x7f28a0159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.478+0000 7f28a5600640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00b7540 0x7f28a01540d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f2894000c00 tx=0x7f28940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:07.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.479+0000 7f289d600640 1 -- 172.21.15.19:0/2622384429 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894041070 con 0x7f28a00b7540 2024-04-25T11:37:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.479+0000 7f289d600640 1 -- 172.21.15.19:0/2622384429 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289403c870 con 0x7f28a00b7540 2024-04-25T11:37:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.479+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a015bcf0 con 0x7f28a00b7540 2024-04-25T11:37:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.479+0000 7f289d600640 1 -- 172.21.15.19:0/2622384429 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894037040 con 0x7f28a00b7540 2024-04-25T11:37:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.479+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a015c2b0 con 0x7f28a00b7540 2024-04-25T11:37:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.480+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a00d4ac0 con 0x7f28a00b7540 2024-04-25T11:37:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.481+0000 7f289d600640 1 -- 172.21.15.19:0/2622384429 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2894044070 con 0x7f28a00b7540 2024-04-25T11:37:07.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.482+0000 7f289d600640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28800744b0 0x7f28800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:07.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.482+0000 7f289d600640 1 -- 172.21.15.19:0/2622384429 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f28940b60d0 con 0x7f28a00b7540 2024-04-25T11:37:07.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.483+0000 7f28a4c00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28800744b0 0x7f28800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:07.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.484+0000 7f28a4c00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28800744b0 0x7f28800769c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f28a00cfd40 tx=0x7f288c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.487+0000 7f289d600640 1 -- 172.21.15.19:0/2622384429 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28a00d4ac0 con 0x7f28a00b7540 2024-04-25T11:37:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.728+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28a0050740 con 0x7f28a00b7540 2024-04-25T11:37:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.729+0000 7f289d600640 1 -- 172.21.15.19:0/2622384429 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) ==== 95+0+5175 (secure 0 0 0) 0x7f2894080590 con 0x7f28a00b7540 2024-04-25T11:37:07.717 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:07.717 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":151,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/390026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":390026662},{"type":"v1","addr":"172.21.15.19:6835","nonce":390026662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:06.603724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:07.717 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 151 2024-04-25T11:37:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28800744b0 msgr2=0x7f28800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28800744b0 0x7f28800769c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f28a00cfd40 tx=0x7f288c008040 comp rx=0 tx=0).stop 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00b7540 msgr2=0x7f28a01540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00b7540 0x7f28a01540d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f2894000c00 tx=0x7f28940046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 shutdown_connections 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28800744b0 0x7f28800769c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a0164860 0x7f28a0159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a00d44a0 0x7f28a01f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 --2- 172.21.15.19:0/2622384429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28a00b7540 0x7f28a01540d0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 >> 172.21.15.19:0/2622384429 conn(0x7f28a00d2720 msgr2=0x7f28a00684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 shutdown_connections 2024-04-25T11:37:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:07.732+0000 7f28a6a00640 1 -- 172.21.15.19:0/2622384429 wait complete. 2024-04-25T11:37:07.743 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:07.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:08.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.017+0000 7f2dc9000640 1 Processor -- start 2024-04-25T11:37:08.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.017+0000 7f2dc9000640 1 -- start start 2024-04-25T11:37:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc9000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2dc40b6a00 0x7f2dc415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc415a2f0 0x7f2dc406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dc406d390 0x7f2dc406f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc9000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc4160d50 con 0x7f2dc406d390 2024-04-25T11:37:08.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc9000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc415a700 con 0x7f2dc40b6a00 2024-04-25T11:37:08.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc41611e0 con 0x7f2dc415a2f0 2024-04-25T11:37:08.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dc406d390 0x7f2dc406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2dc406d390 0x7f2dc406f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60656/0 (socket says 172.21.15.19:60656) 2024-04-25T11:37:08.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc2800640 1 -- 172.21.15.19:0/2059726478 learned_addr learned my addr 172.21.15.19:0/2059726478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:08.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2db9400640 1 --2- 172.21.15.19:0/2059726478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc415a2f0 0x7f2dc406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.020+0000 7f2dc1e00640 1 --2- 172.21.15.19:0/2059726478 >> v2:172.21.15.169:3300/0 conn(0x7f2dc40b6a00 0x7f2dc415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.021+0000 7f2dc2800640 1 -- 172.21.15.19:0/2059726478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc415a2f0 msgr2=0x7f2dc406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:08.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.021+0000 7f2dc2800640 1 --2- 172.21.15.19:0/2059726478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc415a2f0 0x7f2dc406ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.021+0000 7f2dc2800640 1 -- 172.21.15.19:0/2059726478 >> v2:172.21.15.169:3300/0 conn(0x7f2dc40b6a00 msgr2=0x7f2dc415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:08.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.021+0000 7f2dc2800640 1 --2- 172.21.15.19:0/2059726478 >> v2:172.21.15.169:3300/0 conn(0x7f2dc40b6a00 0x7f2dc415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.021+0000 7f2dc2800640 1 -- 172.21.15.19:0/2059726478 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc406d7a0 con 0x7f2dc406d390 2024-04-25T11:37:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.022+0000 7f2dc2800640 1 --2- 172.21.15.19:0/2059726478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc406d390 0x7f2dc406f090 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40713e0 tx=0x7f2dbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d084e49647b1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.022+0000 7f2dc1400640 1 -- 172.21.15.19:0/2059726478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc037070 con 0x7f2dc406d390 2024-04-25T11:37:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.022+0000 7f2dc1400640 1 -- 172.21.15.19:0/2059726478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dbc032440 con 0x7f2dc406d390 2024-04-25T11:37:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.022+0000 7f2dc1400640 1 -- 172.21.15.19:0/2059726478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc032810 con 0x7f2dc406d390 2024-04-25T11:37:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 -- 172.21.15.19:0/2059726478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc406d390 msgr2=0x7f2dc406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 --2- 172.21.15.19:0/2059726478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc406d390 0x7f2dc406f090 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40713e0 tx=0x7f2dbc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 -- 172.21.15.19:0/2059726478 shutdown_connections 2024-04-25T11:37:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 --2- 172.21.15.19:0/2059726478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc406d390 0x7f2dc406f090 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 --2- 172.21.15.19:0/2059726478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc415a2f0 0x7f2dc406ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 --2- 172.21.15.19:0/2059726478 >> v2:172.21.15.169:3300/0 conn(0x7f2dc40b6a00 0x7f2dc415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 -- 172.21.15.19:0/2059726478 >> 172.21.15.19:0/2059726478 conn(0x7f2dc40d1ae0 msgr2=0x7f2dc40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 -- 172.21.15.19:0/2059726478 shutdown_connections 2024-04-25T11:37:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.023+0000 7f2dc9000640 1 -- 172.21.15.19:0/2059726478 wait complete. 2024-04-25T11:37:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.025+0000 7f2dc9000640 1 Processor -- start 2024-04-25T11:37:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.025+0000 7f2dc9000640 1 -- start start 2024-04-25T11:37:08.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dc406d390 0x7f2dc41512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc40b6a00 0x7f2dc41f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc415a2f0 0x7f2dc41f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc9000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41609e0 con 0x7f2dc415a2f0 2024-04-25T11:37:08.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4160700 con 0x7f2dc406d390 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc40ce580 con 0x7f2dc40b6a00 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc415a2f0 0x7f2dc41f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc415a2f0 0x7f2dc41f6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60672/0 (socket says 172.21.15.19:60672) 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.027+0000 7f2dc2800640 1 -- 172.21.15.19:0/3813963292 learned_addr learned my addr 172.21.15.19:0/3813963292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dc1e00640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dc406d390 0x7f2dc41512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2db9400640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc40b6a00 0x7f2dc41f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dc2800640 1 -- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc40b6a00 msgr2=0x7f2dc41f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dc2800640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc40b6a00 0x7f2dc41f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dc2800640 1 -- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dc406d390 msgr2=0x7f2dc41512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dc2800640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dc406d390 0x7f2dc41512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dc2800640 1 -- 172.21.15.19:0/3813963292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc4149bf0 con 0x7f2dc415a2f0 2024-04-25T11:37:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dc2800640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc415a2f0 0x7f2dc41f6c30 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0090b0 tx=0x7f2dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:08.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.028+0000 7f2dbaa00640 1 -- 172.21.15.19:0/3813963292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc037040 con 0x7f2dc415a2f0 2024-04-25T11:37:08.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.029+0000 7f2dbaa00640 1 -- 172.21.15.19:0/3813963292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dbc042070 con 0x7f2dc415a2f0 2024-04-25T11:37:08.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.029+0000 7f2dbaa00640 1 -- 172.21.15.19:0/3813963292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc030920 con 0x7f2dc415a2f0 2024-04-25T11:37:08.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.029+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc41f7170 con 0x7f2dc415a2f0 2024-04-25T11:37:08.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.029+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc4151e80 con 0x7f2dc415a2f0 2024-04-25T11:37:08.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.030+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc41518f0 con 0x7f2dc415a2f0 2024-04-25T11:37:08.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.031+0000 7f2dbaa00640 1 -- 172.21.15.19:0/3813963292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2dbc030ac0 con 0x7f2dc415a2f0 2024-04-25T11:37:08.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.032+0000 7f2dbaa00640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2da00744b0 0x7f2da00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.032+0000 7f2dbaa00640 1 -- 172.21.15.19:0/3813963292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f2dbc0b5ec0 con 0x7f2dc415a2f0 2024-04-25T11:37:08.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.033+0000 7f2dc1e00640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2da00744b0 0x7f2da00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.034+0000 7f2dc1e00640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2da00744b0 0x7f2da00769c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4070570 tx=0x7f2db4003a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:08.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.037+0000 7f2dbaa00640 1 -- 172.21.15.19:0/3813963292 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2dc4149bf0 con 0x7f2dc415a2f0 2024-04-25T11:37:08.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.272+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dc4050740 con 0x7f2dc415a2f0 2024-04-25T11:37:08.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.273+0000 7f2dbaa00640 1 -- 172.21.15.19:0/3813963292 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) ==== 95+0+5175 (secure 0 0 0) 0x7f2dbc07fc90 con 0x7f2dc415a2f0 2024-04-25T11:37:08.260 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:08.260 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":151,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/390026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":390026662},{"type":"v1","addr":"172.21.15.19:6835","nonce":390026662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:06.603724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:08.260 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 151 2024-04-25T11:37:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2da00744b0 msgr2=0x7f2da00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2da00744b0 0x7f2da00769c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4070570 tx=0x7f2db4003a30 comp rx=0 tx=0).stop 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc415a2f0 msgr2=0x7f2dc41f6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc415a2f0 0x7f2dc41f6c30 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0090b0 tx=0x7f2dbc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 shutdown_connections 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2da00744b0 0x7f2da00769c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2dc415a2f0 0x7f2dc41f6c30 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2dc40b6a00 0x7f2dc41f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 --2- 172.21.15.19:0/3813963292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2dc406d390 0x7f2dc41512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.275+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 >> 172.21.15.19:0/3813963292 conn(0x7f2dc40d1ae0 msgr2=0x7f2dc414d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.276+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 shutdown_connections 2024-04-25T11:37:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.276+0000 7f2dc9000640 1 -- 172.21.15.19:0/3813963292 wait complete. 2024-04-25T11:37:08.290 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2024-04-25T11:37:08.290 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2024-04-25T11:37:08.290 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:08.290 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:08.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:37:08.340 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:37:08.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:08.340 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:37:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:37:08.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:37:08.438 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:37:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.736+0000 7f7242400640 1 Processor -- start 2024-04-25T11:37:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.737+0000 7f7242400640 1 -- start start 2024-04-25T11:37:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.739+0000 7f7242400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f723c1434d0 0x7f723c070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.739+0000 7f7242400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.739+0000 7f7242400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723c0709c0 con 0x7f723c160be0 2024-04-25T11:37:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.739+0000 7f7242400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f723c1438e0 con 0x7f723c1434d0 2024-04-25T11:37:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.739+0000 7f7242400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f723c06f3e0 con 0x7f723c0d0ed0 2024-04-25T11:37:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.740+0000 7f7233e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.740+0000 7f7233e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c0d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52708/0 (socket says 172.21.15.19:52708) 2024-04-25T11:37:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.740+0000 7f7233e00640 1 -- 172.21.15.19:0/3253805659 learned_addr learned my addr 172.21.15.19:0/3253805659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.741+0000 7f723b400640 1 --2- 172.21.15.19:0/3253805659 >> v2:172.21.15.169:3300/0 conn(0x7f723c1434d0 0x7f723c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.741+0000 7f7233e00640 1 -- 172.21.15.19:0/3253805659 >> v2:172.21.15.169:3300/0 conn(0x7f723c1434d0 msgr2=0x7f723c070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.741+0000 7f7233e00640 1 --2- 172.21.15.19:0/3253805659 >> v2:172.21.15.169:3300/0 conn(0x7f723c1434d0 0x7f723c070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.741+0000 7f7233e00640 1 -- 172.21.15.19:0/3253805659 >> v1:172.21.15.169:6789/0 conn(0x7f723c160be0 legacy=0x7f723c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.741+0000 7f7233e00640 1 -- 172.21.15.19:0/3253805659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c06f6c0 con 0x7f723c0d0ed0 2024-04-25T11:37:08.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.742+0000 7f7233e00640 1 --2- 172.21.15.19:0/3253805659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c0d12e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722402c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61574b31cee1cb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.743+0000 7f723aa00640 1 -- 172.21.15.19:0/3253805659 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224038070 con 0x7f723c0d0ed0 2024-04-25T11:37:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.743+0000 7f723aa00640 1 -- 172.21.15.19:0/3253805659 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7224031710 con 0x7f723c0d0ed0 2024-04-25T11:37:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.743+0000 7f723aa00640 1 -- 172.21.15.19:0/3253805659 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224031ae0 con 0x7f723c0d0ed0 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.743+0000 7f7242400640 1 -- 172.21.15.19:0/3253805659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 msgr2=0x7f723c0d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.743+0000 7f7242400640 1 --2- 172.21.15.19:0/3253805659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c0d12e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722402c460 comp rx=0 tx=0).stop 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.744+0000 7f7242400640 1 -- 172.21.15.19:0/3253805659 shutdown_connections 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.744+0000 7f7242400640 1 --2- 172.21.15.19:0/3253805659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c0d12e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.744+0000 7f7242400640 1 --2- 172.21.15.19:0/3253805659 >> v2:172.21.15.169:3300/0 conn(0x7f723c1434d0 0x7f723c070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.744+0000 7f7242400640 1 -- 172.21.15.19:0/3253805659 >> 172.21.15.19:0/3253805659 conn(0x7f723c0d1ae0 msgr2=0x7f723c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.744+0000 7f7242400640 1 -- 172.21.15.19:0/3253805659 shutdown_connections 2024-04-25T11:37:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.744+0000 7f7242400640 1 -- 172.21.15.19:0/3253805659 wait complete. 2024-04-25T11:37:08.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.746+0000 7f7242400640 1 Processor -- start 2024-04-25T11:37:08.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.746+0000 7f7242400640 1 -- start start 2024-04-25T11:37:08.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7242400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c14fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7242400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f723c1434d0 0x7f723c1464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7242400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723c160be0 0x7f723c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7242400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f723c06f090 con 0x7f723c1434d0 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7242400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f723c06fc80 con 0x7f723c160be0 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7242400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0b51a0 con 0x7f723c0d0ed0 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7233e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f723c1434d0 0x7f723c1464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7233e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f723c1434d0 0x7f723c1464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60680/0 (socket says 172.21.15.19:60680) 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.748+0000 7f7233e00640 1 -- 172.21.15.19:0/1659283729 learned_addr learned my addr 172.21.15.19:0/1659283729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f723b400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c14fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f7233e00640 1 -- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 msgr2=0x7f723c14fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f7233e00640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c14fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f7233e00640 1 -- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723c160be0 msgr2=0x7f723c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f723be00640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723c160be0 0x7f723c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f7233e00640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723c160be0 0x7f723c151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f7233e00640 1 -- 172.21.15.19:0/1659283729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c14ff90 con 0x7f723c1434d0 2024-04-25T11:37:08.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f7233e00640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f723c1434d0 0x7f723c1464f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f722402c4a0 tx=0x7f72240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:08.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.749+0000 7f7233400640 1 -- 172.21.15.19:0/1659283729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224038040 con 0x7f723c1434d0 2024-04-25T11:37:08.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.750+0000 7f7233400640 1 -- 172.21.15.19:0/1659283729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7224033830 con 0x7f723c1434d0 2024-04-25T11:37:08.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.750+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c152810 con 0x7f723c1434d0 2024-04-25T11:37:08.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.750+0000 7f7233400640 1 -- 172.21.15.19:0/1659283729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224033c40 con 0x7f723c1434d0 2024-04-25T11:37:08.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.750+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c145cf0 con 0x7f723c1434d0 2024-04-25T11:37:08.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.751+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c050740 con 0x7f723c1434d0 2024-04-25T11:37:08.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.752+0000 7f7233400640 1 -- 172.21.15.19:0/1659283729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f722403e430 con 0x7f723c1434d0 2024-04-25T11:37:08.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.753+0000 7f7233400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72180744e0 0x7f72180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:08.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.753+0000 7f7233400640 1 -- 172.21.15.19:0/1659283729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f72240b5f70 con 0x7f723c1434d0 2024-04-25T11:37:08.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.754+0000 7f723b400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72180744e0 0x7f72180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:08.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.755+0000 7f723b400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72180744e0 0x7f72180769f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f722c009a30 tx=0x7f722c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:08.758+0000 7f7233400640 1 -- 172.21.15.19:0/1659283729 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7224080420 con 0x7f723c1434d0 2024-04-25T11:37:08.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.009+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f723c1525a0 con 0x7f723c1434d0 2024-04-25T11:37:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.626+0000 7f7233400640 1 -- 172.21.15.19:0/1659283729 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v152) ==== 161+0+0 (secure 0 0 0) 0x7f722407fd20 con 0x7f723c1434d0 2024-04-25T11:37:09.614 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:37:09.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.629+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72180744e0 msgr2=0x7f72180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:09.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.629+0000 7f7242400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72180744e0 0x7f72180769f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f722c009a30 tx=0x7f722c008040 comp rx=0 tx=0).stop 2024-04-25T11:37:09.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.629+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f723c1434d0 msgr2=0x7f723c1464f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:09.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.629+0000 7f7242400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f723c1434d0 0x7f723c1464f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f722402c4a0 tx=0x7f72240046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 shutdown_connections 2024-04-25T11:37:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72180744e0 0x7f72180769f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f723c160be0 0x7f723c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f723c1434d0 0x7f723c1464f0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 --2- 172.21.15.19:0/1659283729 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f723c0d0ed0 0x7f723c14fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 >> 172.21.15.19:0/1659283729 conn(0x7f723c0d1ae0 msgr2=0x7f723c0b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 shutdown_connections 2024-04-25T11:37:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.630+0000 7f7242400640 1 -- 172.21.15.19:0/1659283729 wait complete. 2024-04-25T11:37:09.645 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:37:09.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.943+0000 7f2503200640 1 Processor -- start 2024-04-25T11:37:09.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.944+0000 7f2503200640 1 -- start start 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2503200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f250406c5c0 0x7f25040cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2503200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25040b4ea0 0x7f250415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2503200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25040d1230 con 0x7f2504144370 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2503200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f25040d06b0 con 0x7f25040b4ea0 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2503200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f25040d0990 con 0x7f250406c5c0 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2501400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2504144370 0x7f25040b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33708/0 (socket says 172.21.15.19:33708) 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2501400640 1 -- 172.21.15.19:0/1904447095 learned_addr learned my addr 172.21.15.19:0/1904447095 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:09.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.946+0000 7f2502800640 1 --2- 172.21.15.19:0/1904447095 >> v2:172.21.15.19:3300/0 conn(0x7f25040b4ea0 0x7f250415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:09.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.947+0000 7f2501e00640 1 --2- 172.21.15.19:0/1904447095 >> v2:172.21.15.169:3300/0 conn(0x7f250406c5c0 0x7f25040cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:09.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.947+0000 7f2500a00640 1 -- 172.21.15.19:0/1904447095 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4047751521 0 0) 0x7f25040d1230 con 0x7f2504144370 2024-04-25T11:37:09.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.947+0000 7f2500a00640 1 -- 172.21.15.19:0/1904447095 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24e8003520 con 0x7f2504144370 2024-04-25T11:37:09.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.947+0000 7f2502800640 1 -- 172.21.15.19:0/1904447095 >> v2:172.21.15.169:3300/0 conn(0x7f250406c5c0 msgr2=0x7f25040cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:09.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.947+0000 7f2502800640 1 --2- 172.21.15.19:0/1904447095 >> v2:172.21.15.169:3300/0 conn(0x7f250406c5c0 0x7f25040cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.947+0000 7f2502800640 1 -- 172.21.15.19:0/1904447095 >> v1:172.21.15.19:6789/0 conn(0x7f2504144370 legacy=0x7f25040b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:09.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.947+0000 7f2502800640 1 -- 172.21.15.19:0/1904447095 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040d0c70 con 0x7f25040b4ea0 2024-04-25T11:37:09.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.948+0000 7f2502800640 1 --2- 172.21.15.19:0/1904447095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25040b4ea0 0x7f250415e770 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f25040726b0 tx=0x7f24fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcc28109587bbea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:09.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2500a00640 1 -- 172.21.15.19:0/1904447095 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc037070 con 0x7f25040b4ea0 2024-04-25T11:37:09.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2500a00640 1 -- 172.21.15.19:0/1904447095 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24fc032440 con 0x7f25040b4ea0 2024-04-25T11:37:09.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2500a00640 1 -- 172.21.15.19:0/1904447095 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0327f0 con 0x7f25040b4ea0 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2503200640 1 -- 172.21.15.19:0/1904447095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25040b4ea0 msgr2=0x7f250415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2503200640 1 --2- 172.21.15.19:0/1904447095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25040b4ea0 0x7f250415e770 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f25040726b0 tx=0x7f24fc02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2503200640 1 -- 172.21.15.19:0/1904447095 shutdown_connections 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2503200640 1 --2- 172.21.15.19:0/1904447095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25040b4ea0 0x7f250415e770 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2503200640 1 --2- 172.21.15.19:0/1904447095 >> v2:172.21.15.169:3300/0 conn(0x7f250406c5c0 0x7f25040cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.949+0000 7f2503200640 1 -- 172.21.15.19:0/1904447095 >> 172.21.15.19:0/1904447095 conn(0x7f25040d1720 msgr2=0x7f25040d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.950+0000 7f2503200640 1 -- 172.21.15.19:0/1904447095 shutdown_connections 2024-04-25T11:37:09.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.950+0000 7f2503200640 1 -- 172.21.15.19:0/1904447095 wait complete. 2024-04-25T11:37:09.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.952+0000 7f2503200640 1 Processor -- start 2024-04-25T11:37:09.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.952+0000 7f2503200640 1 -- start start 2024-04-25T11:37:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2503200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f250406c5c0 0x7f25041604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2503200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2504144370 0x7f250415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2503200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250406f160 0x7f25040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2503200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25040d0450 con 0x7f250406f160 2024-04-25T11:37:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2503200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25040d02d0 con 0x7f2504144370 2024-04-25T11:37:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2503200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2504147760 con 0x7f250406c5c0 2024-04-25T11:37:09.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2502800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250406f160 0x7f25040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2502800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250406f160 0x7f25040709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60704/0 (socket says 172.21.15.19:60704) 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2502800640 1 -- 172.21.15.19:0/1093523402 learned_addr learned my addr 172.21.15.19:0/1093523402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2501400640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2504144370 0x7f250415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.954+0000 7f2501e00640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f250406c5c0 0x7f25041604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2502800640 1 -- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f250406c5c0 msgr2=0x7f25041604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2502800640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f250406c5c0 0x7f25041604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2502800640 1 -- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2504144370 msgr2=0x7f250415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2502800640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2504144370 0x7f250415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2502800640 1 -- 172.21.15.19:0/1093523402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250406f940 con 0x7f250406f160 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2501400640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2504144370 0x7f250415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2501e00640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f250406c5c0 0x7f25041604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:09.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2502800640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250406f160 0x7f25040709e0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f24fc035040 tx=0x7f24fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f24fa000640 1 -- 172.21.15.19:0/1093523402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc037040 con 0x7f250406f160 2024-04-25T11:37:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f24fa000640 1 -- 172.21.15.19:0/1093523402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24fc042070 con 0x7f250406f160 2024-04-25T11:37:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f24fa000640 1 -- 172.21.15.19:0/1093523402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc030840 con 0x7f250406f160 2024-04-25T11:37:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.955+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250406fc20 con 0x7f250406f160 2024-04-25T11:37:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.956+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25040701e0 con 0x7f250406f160 2024-04-25T11:37:09.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.957+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250406f6b0 con 0x7f250406f160 2024-04-25T11:37:09.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.958+0000 7f24fa000640 1 -- 172.21.15.19:0/1093523402 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f24fc0309e0 con 0x7f250406f160 2024-04-25T11:37:09.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.959+0000 7f24fa000640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:09.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.959+0000 7f24fa000640 1 -- 172.21.15.19:0/1093523402 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f24fc0b5e50 con 0x7f250406f160 2024-04-25T11:37:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.960+0000 7f2501e00640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.961+0000 7f2501e00640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009f50 tx=0x7f24f400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:09.965+0000 7f24fa000640 1 -- 172.21.15.19:0/1093523402 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24fc04c020 con 0x7f250406f160 2024-04-25T11:37:10.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.254+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f25040b61c0 con 0x7f250406f160 2024-04-25T11:37:10.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.642+0000 7f24fa000640 1 -- 172.21.15.19:0/1093523402 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v153) ==== 101+0+0 (secure 0 0 0) 0x7f25040b61c0 con 0x7f250406f160 2024-04-25T11:37:10.630 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:37:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 msgr2=0x7f24e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:10.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009f50 tx=0x7f24f400b040 comp rx=0 tx=0).stop 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250406f160 msgr2=0x7f25040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250406f160 0x7f25040709e0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f24fc035040 tx=0x7f24fc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 shutdown_connections 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2504144370 0x7f250415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f250406f160 0x7f25040709e0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 --2- 172.21.15.19:0/1093523402 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f250406c5c0 0x7f25041604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 >> 172.21.15.19:0/1093523402 conn(0x7f25040d1720 msgr2=0x7f25040d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 shutdown_connections 2024-04-25T11:37:10.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.646+0000 7f2503200640 1 -- 172.21.15.19:0/1093523402 wait complete. 2024-04-25T11:37:10.667 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:37:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.869+0000 7fece5f5df40 1 Processor -- start 2024-04-25T11:37:10.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.869+0000 7fece5f5df40 1 -- start start 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fece5f5df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d3d6c0 0x562376d65850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fece5f5df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d69180 0x562376e9f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fece5f5df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562376d6a180 0x562376e9f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fece5f5df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562376d82ce0 con 0x562376d69180 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fece5f5df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562376ce0d20 con 0x562376d6a180 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fece5f5df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562376d832a0 con 0x562376d3d6c0 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fecde600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d69180 0x562376e9f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:10.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fecde600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d69180 0x562376e9f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60718/0 (socket says 172.21.15.19:60718) 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.871+0000 7fecde600640 1 -- 172.21.15.19:0/571452929 learned_addr learned my addr 172.21.15.19:0/571452929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecdf000640 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d3d6c0 0x562376d65850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecde600640 1 -- 172.21.15.19:0/571452929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d3d6c0 msgr2=0x562376d65850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecde600640 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d3d6c0 0x562376d65850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecde600640 1 -- 172.21.15.19:0/571452929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562376d6a180 msgr2=0x562376e9f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecde600640 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562376d6a180 0x562376e9f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecde600640 1 -- 172.21.15.19:0/571452929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562376e0fee0 con 0x562376d69180 2024-04-25T11:37:10.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecde600640 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d69180 0x562376e9f3c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7feccc00a5e0 tx=0x7feccc00a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb32c3b716dfc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecdd200640 1 -- 172.21.15.19:0/571452929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc019070 con 0x562376d69180 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecdd200640 1 -- 172.21.15.19:0/571452929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feccc007600 con 0x562376d69180 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.872+0000 7fecdd200640 1 -- 172.21.15.19:0/571452929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0079f0 con 0x562376d69180 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 -- 172.21.15.19:0/571452929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d69180 msgr2=0x562376e9f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d69180 0x562376e9f3c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7feccc00a5e0 tx=0x7feccc00a6d0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 -- 172.21.15.19:0/571452929 shutdown_connections 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562376d6a180 0x562376e9f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d69180 0x562376e9f3c0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 --2- 172.21.15.19:0/571452929 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d3d6c0 0x562376d65850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 -- 172.21.15.19:0/571452929 >> 172.21.15.19:0/571452929 conn(0x562376c5cac0 msgr2=0x562376d66390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 -- 172.21.15.19:0/571452929 shutdown_connections 2024-04-25T11:37:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.873+0000 7fece5f5df40 1 -- 172.21.15.19:0/571452929 wait complete. 2024-04-25T11:37:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.874+0000 7fece5f5df40 1 Processor -- start 2024-04-25T11:37:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.875+0000 7fece5f5df40 1 -- start start 2024-04-25T11:37:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fece5f5df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562376d3d6c0 0x562376d39b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fece5f5df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d69180 0x562376e00f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fece5f5df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d6a180 0x562376e014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fece5f5df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562376e13ab0 con 0x562376d6a180 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fece5f5df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562376d80600 con 0x562376d3d6c0 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fece5f5df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562376ea9c70 con 0x562376d69180 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fecdfa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d6a180 0x562376e014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fecdfa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d6a180 0x562376e014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60728/0 (socket says 172.21.15.19:60728) 2024-04-25T11:37:10.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.876+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 learned_addr learned my addr 172.21.15.19:0/3727800868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.877+0000 7fecde600640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d69180 0x562376e00f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.877+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d69180 msgr2=0x562376e00f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.877+0000 7fecdfa00640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562376d69180 0x562376e00f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.877+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562376d3d6c0 msgr2=0x562376d39b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.877+0000 7fecdfa00640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562376d3d6c0 0x562376d39b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.877+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562376d3c6c0 con 0x562376d6a180 2024-04-25T11:37:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.877+0000 7fecdfa00640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d6a180 0x562376e014d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fecd8006fd0 tx=0x7fecd800b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.878+0000 7fecd7400640 1 -- 172.21.15.19:0/3727800868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8016070 con 0x562376d6a180 2024-04-25T11:37:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.878+0000 7fecd7400640 1 -- 172.21.15.19:0/3727800868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd800bb60 con 0x562376d6a180 2024-04-25T11:37:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.878+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562376d3c9a0 con 0x562376d6a180 2024-04-25T11:37:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.878+0000 7fecd7400640 1 -- 172.21.15.19:0/3727800868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8004520 con 0x562376d6a180 2024-04-25T11:37:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.878+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562376ea99b0 con 0x562376d6a180 2024-04-25T11:37:10.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.880+0000 7fecd7400640 1 -- 172.21.15.19:0/3727800868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fecd8019070 con 0x562376d6a180 2024-04-25T11:37:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.881+0000 7fecd7400640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc0072ec0 0x7fecc00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.881+0000 7fecd7400640 1 -- 172.21.15.19:0/3727800868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7fecd80932f0 con 0x562376d6a180 2024-04-25T11:37:10.869 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:37:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.882+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562376e02050 0x562376ea8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.882+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376d3a0d0 con 0x562376e02050 2024-04-25T11:37:10.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.882+0000 7fecdf000640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc0072ec0 0x7fecc00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:10.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.883+0000 7fecde600640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562376e02050 0x562376ea8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.884+0000 7fecdf000640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc0072ec0 0x7fecc00753d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fecd0056000 tx=0x7fecd004e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:10.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.884+0000 7fecde600640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562376e02050 0x562376ea8e40 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.887+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v102'288 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.888+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ea9380 con 0x562376e02050 2024-04-25T11:37:10.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.891+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v102'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.891+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb3980 con 0x562376e02050 2024-04-25T11:37:10.881 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:37:10.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.894+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v102'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.894+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4d30 con 0x562376e02050 2024-04-25T11:37:10.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.897+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.897+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb50d0 con 0x562376e02050 2024-04-25T11:37:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.900+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.900+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb3d20 con 0x562376e02050 2024-04-25T11:37:10.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.904+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.904+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4130 con 0x562376e02050 2024-04-25T11:37:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.908+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v102'294 uv268 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.908+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376eb4130 con 0x562376e02050 2024-04-25T11:37:10.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.911+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v102'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.911+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40e0 con 0x562376e02050 2024-04-25T11:37:10.902 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:37:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.915+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v102'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.915+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40e0 con 0x562376e02050 2024-04-25T11:37:10.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.918+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.918+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.922+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.922+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.925+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.925+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.929+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v102'300 uv275 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.929+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.933+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'301 uv301 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.933+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.924 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:37:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.937+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v102'302 uv302 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.937+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.941+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'303 uv303 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.941+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.945+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'304 uv304 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.945+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb40c0 con 0x562376e02050 2024-04-25T11:37:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.948+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'305 uv305 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.949+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4480 con 0x562376e02050 2024-04-25T11:37:10.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.952+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'306 uv306 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.952+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4480 con 0x562376e02050 2024-04-25T11:37:10.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.956+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v102'307 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.956+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376eb4480 con 0x562376e02050 2024-04-25T11:37:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.960+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v102'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.960+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:10.952 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:37:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.964+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v102'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.964+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:10.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.968+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v102'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:10.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.968+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:10.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.973+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v102'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.973+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:11.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.977+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v102'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.977+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:11.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.981+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v102'313 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.981+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:11.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.986+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v102'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.986+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:11.018 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.990+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v102'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.991+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.995+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v102'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:10.995+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.000+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v102'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.000+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376e02050 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.004+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v102'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.005+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562376eb5470 0x562376eb7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.005+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb4380 con 0x562376eb5470 2024-04-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.006+0000 7fecdf000640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562376eb5470 0x562376eb7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.007+0000 7fecdf000640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562376eb5470 0x562376eb7890 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.011+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v102'219 uv202 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.011+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376eb7dd0 con 0x562376eb5470 2024-04-25T11:37:11.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.016+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v102'220 uv220 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.016+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb8170 con 0x562376eb5470 2024-04-25T11:37:11.021 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:37:11.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.020+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v102'221 uv221 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.021+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb8510 con 0x562376eb5470 2024-04-25T11:37:11.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.025+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v102'222 uv222 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.025+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb88b0 con 0x562376eb5470 2024-04-25T11:37:11.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.029+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v102'223 uv223 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.029+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb8c50 con 0x562376eb5470 2024-04-25T11:37:11.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.033+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v102'224 uv207 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.034+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376eb8ff0 con 0x562376eb5470 2024-04-25T11:37:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.038+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v102'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.039+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376eb9390 con 0x562376eb5470 2024-04-25T11:37:11.032 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:37:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.043+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v102'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.043+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.047+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v102'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.047+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.052+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v102'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.052+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.056+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v102'229 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.056+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.060+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v102'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.060+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.052 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:37:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.064+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v102'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.064+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.069+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v102'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.070+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.074+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v102'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.074+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.078+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v102'234 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.078+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.082+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v102'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.082+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.074 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:37:11.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.087+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v102'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.087+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.091+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v102'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.091+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.095+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v102'238 uv238 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.095+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376e01af0 con 0x562376eb5470 2024-04-25T11:37:11.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.100+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v102'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376eb5470 2024-04-25T11:37:11.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.100+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562376eb9d50 0x562376ebc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.100+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebc6b0 con 0x562376eb9d50 2024-04-25T11:37:11.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.101+0000 7fecde600640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562376eb9d50 0x562376ebc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.103+0000 7fecde600640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562376eb9d50 0x562376ebc170 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.107+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.107+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ebc6b0 con 0x562376eb9d50 2024-04-25T11:37:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.111+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v102'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.111+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebcad0 con 0x562376eb9d50 2024-04-25T11:37:11.102 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:37:11.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.115+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v102'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.115+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebdae0 con 0x562376eb9d50 2024-04-25T11:37:11.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.120+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.120+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebdae0 con 0x562376eb9d50 2024-04-25T11:37:11.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.124+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.124+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebcad0 con 0x562376eb9d50 2024-04-25T11:37:11.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.128+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.128+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebcad0 con 0x562376e02050 2024-04-25T11:37:11.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.132+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'48 uv47 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.132+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ebcad0 con 0x562376e02050 2024-04-25T11:37:11.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.137+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v102'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.137+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebcad0 con 0x562376e02050 2024-04-25T11:37:11.128 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:37:11.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.141+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v102'50 uv50 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.141+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebcad0 con 0x562376e02050 2024-04-25T11:37:11.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.145+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v102'51 uv51 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.146+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562376ebd150 0x562376ebf570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.146+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ebfb40 con 0x562376ebd150 2024-04-25T11:37:11.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.147+0000 7fecdfa00640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562376ebd150 0x562376ebf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.148+0000 7fecdfa00640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562376ebd150 0x562376ebf570 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.153+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.153+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ebffe0 con 0x562376ebd150 2024-04-25T11:37:11.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.157+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v102'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.157+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec04c0 con 0x562376ebd150 2024-04-25T11:37:11.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.161+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v102'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.148 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:37:11.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.161+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec1d70 con 0x562376ebd150 2024-04-25T11:37:11.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.165+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.165+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec2110 con 0x562376ebd150 2024-04-25T11:37:11.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.169+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.169+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec0d60 con 0x562376ebd150 2024-04-25T11:37:11.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.173+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.173+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562376ec1680 0x562376ec44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.173+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec4a00 con 0x562376ec1680 2024-04-25T11:37:11.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.174+0000 7fecdf000640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562376ec1680 0x562376ec44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.176+0000 7fecdf000640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562376ec1680 0x562376ec44c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.180+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v102'105 uv99 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.180+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec4a00 con 0x562376ec1680 2024-04-25T11:37:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.184+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v102'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.184+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec4a00 con 0x562376ec1680 2024-04-25T11:37:11.175 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:37:11.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.188+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v102'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.188+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec5e00 con 0x562376ec1680 2024-04-25T11:37:11.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.192+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v102'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.192+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec5e00 con 0x562376ec1680 2024-04-25T11:37:11.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.196+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v102'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.196+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec4df0 con 0x562376ec1680 2024-04-25T11:37:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.200+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v102'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.200+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec4df0 con 0x562376ec1680 2024-04-25T11:37:11.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.204+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v102'111 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.204+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec4df0 con 0x562376ec1680 2024-04-25T11:37:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.208+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v102'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.208+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec4df0 con 0x562376ec1680 2024-04-25T11:37:11.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.212+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v102'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.200 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:37:11.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.212+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec4df0 con 0x562376ec1680 2024-04-25T11:37:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.216+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v102'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.217+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec4df0 con 0x562376ec1680 2024-04-25T11:37:11.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.221+0000 7fecdf000640 1 -- 172.21.15.19:0/3727800868 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v102'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd0060480 con 0x562376ec1680 2024-04-25T11:37:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.221+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562376ec5300 0x562376ec7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.221+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec7e20 con 0x562376ec5300 2024-04-25T11:37:11.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.222+0000 7fecde600640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562376ec5300 0x562376ec7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.223+0000 7fecde600640 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562376ec5300 0x562376ec7720 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.228+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v102'46 uv45 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.228+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec7e20 con 0x562376ec5300 2024-04-25T11:37:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.232+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v102'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.232+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376ec5300 2024-04-25T11:37:11.224 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:37:11.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.236+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v102'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.236+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376ec5300 2024-04-25T11:37:11.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.241+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v102'49 uv49 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.241+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376ec5300 2024-04-25T11:37:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.246+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v102'50 uv50 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.246+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376e02050 2024-04-25T11:37:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.250+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.250+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376e02050 2024-04-25T11:37:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.255+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v102'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.255+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376e02050 2024-04-25T11:37:11.246 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:37:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.259+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v102'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.259+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec9280 con 0x562376e02050 2024-04-25T11:37:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.263+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.263+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec9280 con 0x562376e02050 2024-04-25T11:37:11.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.267+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.267+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376e02050 2024-04-25T11:37:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.272+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376e02050 2024-04-25T11:37:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.272+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8270 con 0x562376eb9d50 2024-04-25T11:37:11.264 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:37:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.277+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v102'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.277+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec0950 con 0x562376eb9d50 2024-04-25T11:37:11.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.281+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [write 0~34] v102'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.281+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec0950 con 0x562376eb9d50 2024-04-25T11:37:11.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.285+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v102'97 uv97 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.285+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec0950 con 0x562376eb9d50 2024-04-25T11:37:11.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.290+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 100.00000000 [create] v102'98 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.290+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec0950 con 0x562376eb9d50 2024-04-25T11:37:11.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.295+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 100.00000000 [delete] v102'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.295+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec0950 con 0x562376eb9d50 2024-04-25T11:37:11.286 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:37:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.299+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v102'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.299+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec0950 con 0x562376eb9d50 2024-04-25T11:37:11.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.303+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v102'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.303+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ecb080 con 0x562376eb9d50 2024-04-25T11:37:11.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.308+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v102'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376eb9d50 2024-04-25T11:37:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.308+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8720 con 0x562376ebd150 2024-04-25T11:37:11.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.312+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v102'44 uv43 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.312+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec8b10 con 0x562376ebd150 2024-04-25T11:37:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.317+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v102'45 uv45 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.317+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8b10 con 0x562376ebd150 2024-04-25T11:37:11.309 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:37:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.321+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v102'46 uv46 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.321+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ebd150 2024-04-25T11:37:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.326+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v102'47 uv47 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.326+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ebd150 2024-04-25T11:37:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.330+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v102'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.330+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ebd150 2024-04-25T11:37:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.334+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v102'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.334+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ebd150 2024-04-25T11:37:11.326 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:37:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.338+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v102'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.338+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ebd150 2024-04-25T11:37:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.342+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v102'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.342+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ebd150 2024-04-25T11:37:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.346+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v102'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.346+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ebd150 2024-04-25T11:37:11.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.351+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v102'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecd8018030 con 0x562376ebd150 2024-04-25T11:37:11.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.351+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.355+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v102'47 uv46 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.355+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.359+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v102'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.359+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.350 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:37:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.363+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v102'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.363+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.367+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v102'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.367+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.371+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v102'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.371+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.376+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v102'119 uv111 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.376+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.380+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v102'120 uv120 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.380+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.371 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:37:11.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.384+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v102'121 uv121 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.384+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.389+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v102'122 uv122 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.389+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.394+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v102'123 uv123 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.394+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.398+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v102'124 uv118 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.398+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.402+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v102'125 uv125 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.402+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.393 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:37:11.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.405+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v102'126 uv126 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.406+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec9ad0 con 0x562376ec5300 2024-04-25T11:37:11.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.410+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v102'127 uv127 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.410+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec9ad0 con 0x562376ec5300 2024-04-25T11:37:11.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.414+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v102'128 uv128 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.414+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec8ac0 con 0x562376ec5300 2024-04-25T11:37:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.418+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v102'129 uv129 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.418+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x562376ec9220 con 0x562376ec5300 2024-04-25T11:37:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fecde600640 1 -- 172.21.15.19:0/3727800868 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v102'130 uv130 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feccc01c040 con 0x562376ec5300 2024-04-25T11:37:11.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562376eb5470 msgr2=0x562376eb7890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562376eb5470 0x562376eb7890 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562376ec1680 msgr2=0x562376ec44c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562376ec1680 0x562376ec44c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562376ec5300 msgr2=0x562376ec7720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562376ec5300 0x562376ec7720 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562376eb9d50 msgr2=0x562376ebc170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562376eb9d50 0x562376ebc170 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562376e02050 msgr2=0x562376ea8e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fecdfa00640 1 -- 172.21.15.19:0/3727800868 reap_dead start 2024-04-25T11:37:11.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.423+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562376e02050 0x562376ea8e40 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562376ebd150 msgr2=0x562376ebf570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562376ebd150 0x562376ebf570 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc0072ec0 msgr2=0x7fecc00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc0072ec0 0x7fecc00753d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fecd0056000 tx=0x7fecd004e000 comp rx=0 tx=0).stop 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d6a180 msgr2=0x562376e014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d6a180 0x562376e014d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fecd8006fd0 tx=0x7fecd800b4b0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 shutdown_connections 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562376ebd150 0x562376ebf570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562376e02050 0x562376ea8e40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fecc0072ec0 0x7fecc00753d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 --2- 172.21.15.19:0/3727800868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562376d6a180 0x562376e014d0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.424+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 >> 172.21.15.19:0/3727800868 conn(0x562376c5cac0 msgr2=0x562376d66390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.425+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 shutdown_connections 2024-04-25T11:37:11.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.425+0000 7fece5f5df40 1 -- 172.21.15.19:0/3727800868 wait complete. 2024-04-25T11:37:11.443 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2024-04-25T11:37:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.690+0000 7f0f6eb20f40 1 Processor -- start 2024-04-25T11:37:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.690+0000 7f0f6eb20f40 1 -- start start 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f6eb20f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d441e0 0x562b66d43180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f6eb20f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d445f0 0x562b66d75cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f6eb20f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 0x562b66d761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f6eb20f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b66cead20 con 0x562b66d445f0 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f6eb20f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562b66ceed60 con 0x562b66d441e0 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f6eb20f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562b66d8d3f0 con 0x562b66e1dfa0 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f66800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d445f0 0x562b66d75cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f66800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d445f0 0x562b66d75cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60744/0 (socket says 172.21.15.19:60744) 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f66800640 1 -- 172.21.15.19:0/550234459 learned_addr learned my addr 172.21.15.19:0/550234459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f67200640 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d441e0 0x562b66d43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.692+0000 7f0f67c00640 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 0x562b66d761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f66800640 1 -- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 msgr2=0x562b66d761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f66800640 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 0x562b66d761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f66800640 1 -- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d441e0 msgr2=0x562b66d43180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f66800640 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d441e0 0x562b66d43180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f66800640 1 -- 172.21.15.19:0/550234459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b66e10f80 con 0x562b66d445f0 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f67200640 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d441e0 0x562b66d43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f66800640 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d445f0 0x562b66d75cb0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000a460 tx=0x7f0f5000a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c050ce5ef1bf93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f65400640 1 -- 172.21.15.19:0/550234459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000b920 con 0x562b66d445f0 2024-04-25T11:37:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f65400640 1 -- 172.21.15.19:0/550234459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5000f070 con 0x562b66d445f0 2024-04-25T11:37:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.693+0000 7f0f65400640 1 -- 172.21.15.19:0/550234459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000bd20 con 0x562b66d445f0 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/550234459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d445f0 msgr2=0x562b66d75cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d445f0 0x562b66d75cb0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000a460 tx=0x7f0f5000a920 comp rx=0 tx=0).stop 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/550234459 shutdown_connections 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 0x562b66d761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d445f0 0x562b66d75cb0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/550234459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d441e0 0x562b66d43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/550234459 >> 172.21.15.19:0/550234459 conn(0x562b66c66ac0 msgr2=0x562b66e1a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/550234459 shutdown_connections 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.694+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/550234459 wait complete. 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.695+0000 7f0f6eb20f40 1 Processor -- start 2024-04-25T11:37:11.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.695+0000 7f0f6eb20f40 1 -- start start 2024-04-25T11:37:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f6eb20f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d441e0 0x562b66d43370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f6eb20f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d445f0 0x562b66eaa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f6eb20f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 0x562b66eaaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f6eb20f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b66e11220 con 0x562b66d441e0 2024-04-25T11:37:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f6eb20f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562b66e17ae0 con 0x562b66d445f0 2024-04-25T11:37:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f6eb20f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562b66d8f210 con 0x562b66e1dfa0 2024-04-25T11:37:11.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f67200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d441e0 0x562b66d43370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f67200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d441e0 0x562b66d43370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60750/0 (socket says 172.21.15.19:60750) 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f67200640 1 -- 172.21.15.19:0/3978504688 learned_addr learned my addr 172.21.15.19:0/3978504688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.697+0000 7f0f67c00640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 0x562b66eaaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f66800640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d445f0 0x562b66eaa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f67200640 1 -- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 msgr2=0x562b66eaaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f67200640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562b66e1dfa0 0x562b66eaaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f67200640 1 -- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d445f0 msgr2=0x562b66eaa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f67200640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562b66d445f0 0x562b66eaa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f67200640 1 -- 172.21.15.19:0/3978504688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b66eac200 con 0x562b66d441e0 2024-04-25T11:37:11.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f67200640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d441e0 0x562b66d43370 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f0f58025b20 tx=0x7f0f58003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3978504688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58068070 con 0x562b66d441e0 2024-04-25T11:37:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3978504688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f58050510 con 0x562b66d441e0 2024-04-25T11:37:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562b66e13f10 con 0x562b66d441e0 2024-04-25T11:37:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.698+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3978504688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58054040 con 0x562b66d441e0 2024-04-25T11:37:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.699+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b66e1ad80 con 0x562b66d441e0 2024-04-25T11:37:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.701+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3978504688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f5806b070 con 0x562b66d441e0 2024-04-25T11:37:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.702+0000 7f0f5fe00640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f44072e90 0x7f0f440753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.702+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3978504688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f0f580d7bf0 con 0x562b66d441e0 2024-04-25T11:37:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.702+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562b66eac6c0 0x562b66eb5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.702+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e102) -- 0x562b66d438b0 con 0x562b66eac6c0 2024-04-25T11:37:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.703+0000 7f0f66800640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f44072e90 0x7f0f440753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.703+0000 7f0f67c00640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562b66eac6c0 0x562b66eb5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.704+0000 7f0f66800640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f44072e90 0x7f0f440753a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000a460 tx=0x7f0f50002000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.704+0000 7f0f67c00640 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562b66eac6c0 0x562b66eb5760 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f67c00640 1 -- 172.21.15.19:0/3978504688 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 100.00000000 [delete] v102'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f60011410 con 0x562b66eac6c0 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562b66eac6c0 msgr2=0x562b66eb5760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562b66eac6c0 0x562b66eb5760 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f44072e90 msgr2=0x7f0f440753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0f44072e90 0x7f0f440753a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000a460 tx=0x7f0f50002000 comp rx=0 tx=0).stop 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d441e0 msgr2=0x562b66d43370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 --2- 172.21.15.19:0/3978504688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b66d441e0 0x562b66d43370 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f0f58025b20 tx=0x7f0f58003330 comp rx=0 tx=0).stop 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f67c00640 1 -- 172.21.15.19:0/3978504688 reap_dead start 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 shutdown_connections 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 >> 172.21.15.19:0/3978504688 conn(0x562b66c66ac0 msgr2=0x562b66d91640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.709+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 shutdown_connections 2024-04-25T11:37:11.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:11.710+0000 7f0f6eb20f40 1 -- 172.21.15.19:0/3978504688 wait complete. 2024-04-25T11:37:11.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:37:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.013+0000 7f2af4800640 1 Processor -- start 2024-04-25T11:37:12.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.013+0000 7f2af4800640 1 -- start start 2024-04-25T11:37:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.015+0000 7f2af4800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 0x7f2aec0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.015+0000 7f2af4800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0cdea0 con 0x7f2aec06dce0 2024-04-25T11:37:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.015+0000 7f2af4800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0ce180 con 0x7f2aec0cd9a0 2024-04-25T11:37:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.015+0000 7f2af4800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec0ceab0 con 0x7f2aec15f360 2024-04-25T11:37:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.016+0000 7f2af1400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2aec06dce0 0x7f2aec15faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33730/0 (socket says 172.21.15.19:33730) 2024-04-25T11:37:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.016+0000 7f2af1400640 1 -- 172.21.15.19:0/1355852189 learned_addr learned my addr 172.21.15.19:0/1355852189 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:12.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.016+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337791883 0 0) 0x7f2aec0cdea0 con 0x7f2aec06dce0 2024-04-25T11:37:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.016+0000 7f2af2800640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 0x7f2aec0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.016+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad4003520 con 0x7f2aec06dce0 2024-04-25T11:37:12.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 717194689 0 0) 0x7f2ad4003520 con 0x7f2aec06dce0 2024-04-25T11:37:12.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 msgr2=0x7f2aec0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 0x7f2aec0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 >> v1:172.21.15.169:6789/0 conn(0x7f2aec0cd9a0 legacy=0x7f2aec06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec0ce460 con 0x7f2aec06dce0 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2adc001670 con 0x7f2aec06dce0 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 >> v1:172.21.15.19:6789/0 conn(0x7f2aec06dce0 legacy=0x7f2aec15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 0x7f2ad40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4003ff0 0x7f2ad4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad4004d40 0x7f2ad4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0ceab0 con 0x7f2ad4004d40 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad400bbd0 con 0x7f2ad4003ff0 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad400bdd0 con 0x7f2aec15f360 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.017+0000 7f2af2800640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad4004d40 0x7f2ad4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.018+0000 7f2af1e00640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 0x7f2ad40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.018+0000 7f2af1400640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4003ff0 0x7f2ad4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.018+0000 7f2af2800640 1 -- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 msgr2=0x7f2ad40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.018+0000 7f2af2800640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec15f360 0x7f2ad40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.018+0000 7f2af2800640 1 -- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4003ff0 msgr2=0x7f2ad4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.018+0000 7f2af2800640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4003ff0 0x7f2ad4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.018+0000 7f2af2800640 1 -- 172.21.15.19:0/1355852189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad4003520 con 0x7f2ad4004d40 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.019+0000 7f2af2800640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad4004d40 0x7f2ad4005210 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0cddd0 tx=0x7f2ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4b368016a3aacf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.019+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8033070 con 0x7f2ad4004d40 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.019+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae802f630 con 0x7f2ad4004d40 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.019+0000 7f2af0a00640 1 -- 172.21.15.19:0/1355852189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8009be0 con 0x7f2ad4004d40 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.020+0000 7f2af4800640 1 -- 172.21.15.19:0/1355852189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad4004d40 msgr2=0x7f2ad4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.020+0000 7f2af4800640 1 --2- 172.21.15.19:0/1355852189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad4004d40 0x7f2ad4005210 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0cddd0 tx=0x7f2ad4005830 comp rx=0 tx=0).stop 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.020+0000 7f2af2800640 1 -- 172.21.15.19:0/1355852189 reap_dead start 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.020+0000 7f2af4800640 1 -- 172.21.15.19:0/1355852189 shutdown_connections 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.020+0000 7f2af4800640 1 -- 172.21.15.19:0/1355852189 >> 172.21.15.19:0/1355852189 conn(0x7f2aec073090 msgr2=0x7f2aec0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.020+0000 7f2af4800640 1 -- 172.21.15.19:0/1355852189 shutdown_connections 2024-04-25T11:37:12.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.020+0000 7f2af4800640 1 -- 172.21.15.19:0/1355852189 wait complete. 2024-04-25T11:37:12.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.023+0000 7f2af4800640 1 Processor -- start 2024-04-25T11:37:12.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.023+0000 7f2af4800640 1 -- start start 2024-04-25T11:37:12.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.024+0000 7f2af4800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4004d40 0x7f2aec141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.024+0000 7f2af4800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec144d20 0x7f2aec0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.024+0000 7f2af4800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec145a60 0x7f2aec0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.024+0000 7f2af4800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0cef40 con 0x7f2aec145a60 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.024+0000 7f2af4800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec15f770 con 0x7f2ad4004d40 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.024+0000 7f2af4800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec0cf0c0 con 0x7f2aec144d20 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec145a60 0x7f2aec0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec145a60 0x7f2aec0b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60762/0 (socket says 172.21.15.19:60762) 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 -- 172.21.15.19:0/737449482 learned_addr learned my addr 172.21.15.19:0/737449482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af1400640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec144d20 0x7f2aec0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af1e00640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4004d40 0x7f2aec141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 -- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec144d20 msgr2=0x7f2aec0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec144d20 0x7f2aec0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 -- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4004d40 msgr2=0x7f2aec141450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4004d40 0x7f2aec141450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.025+0000 7f2af2800640 1 -- 172.21.15.19:0/737449482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec1f7000 con 0x7f2aec145a60 2024-04-25T11:37:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.026+0000 7f2af2800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec145a60 0x7f2aec0b1030 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f2ae802b010 tx=0x7f2ae8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.026+0000 7f2ae6000640 1 -- 172.21.15.19:0/737449482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8033040 con 0x7f2aec145a60 2024-04-25T11:37:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.026+0000 7f2ae6000640 1 -- 172.21.15.19:0/737449482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae802f430 con 0x7f2aec145a60 2024-04-25T11:37:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.026+0000 7f2ae6000640 1 -- 172.21.15.19:0/737449482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8009be0 con 0x7f2aec145a60 2024-04-25T11:37:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.026+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec0b2fa0 con 0x7f2aec145a60 2024-04-25T11:37:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.026+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec0b1750 con 0x7f2aec145a60 2024-04-25T11:37:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.027+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec0b1570 con 0x7f2aec145a60 2024-04-25T11:37:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.029+0000 7f2ae6000640 1 -- 172.21.15.19:0/737449482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ae8036070 con 0x7f2aec145a60 2024-04-25T11:37:12.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.030+0000 7f2ae6000640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744b0 0x7f2ad00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.030+0000 7f2ae6000640 1 -- 172.21.15.19:0/737449482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f2ae80b5b90 con 0x7f2aec145a60 2024-04-25T11:37:12.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.030+0000 7f2af1e00640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744b0 0x7f2ad00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.032+0000 7f2af1e00640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744b0 0x7f2ad00769c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00055a0 tx=0x7f2ae0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:12.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.034+0000 7f2ae6000640 1 -- 172.21.15.19:0/737449482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2aec0b1570 con 0x7f2aec145a60 2024-04-25T11:37:12.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.281+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2aec050740 con 0x7f2aec145a60 2024-04-25T11:37:12.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.659+0000 7f2ae6000640 1 -- 172.21.15.19:0/737449482 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v154) ==== 162+0+0 (secure 0 0 0) 0x7f2ae807ffc0 con 0x7f2aec145a60 2024-04-25T11:37:12.648 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744b0 msgr2=0x7f2ad00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744b0 0x7f2ad00769c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00055a0 tx=0x7f2ae0009040 comp rx=0 tx=0).stop 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec145a60 msgr2=0x7f2aec0b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec145a60 0x7f2aec0b1030 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f2ae802b010 tx=0x7f2ae8004ab0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 shutdown_connections 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744b0 0x7f2ad00769c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec145a60 0x7f2aec0b1030 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec144d20 0x7f2aec0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 --2- 172.21.15.19:0/737449482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad4004d40 0x7f2aec141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.663+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 >> 172.21.15.19:0/737449482 conn(0x7f2aec073090 msgr2=0x7f2aec0707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:12.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.664+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 shutdown_connections 2024-04-25T11:37:12.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.664+0000 7f2af4800640 1 -- 172.21.15.19:0/737449482 wait complete. 2024-04-25T11:37:12.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:12.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.979+0000 7f68d9600640 1 Processor -- start 2024-04-25T11:37:12.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.979+0000 7f68d9600640 1 -- start start 2024-04-25T11:37:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.981+0000 7f68d9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68d41646d0 0x7f68d415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.981+0000 7f68d9600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d4164c80 con 0x7f68d406e740 2024-04-25T11:37:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.981+0000 7f68d9600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d4160bc0 con 0x7f68d41616a0 2024-04-25T11:37:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.981+0000 7f68d9600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f68d4160980 con 0x7f68d41646d0 2024-04-25T11:37:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.982+0000 7f68d3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f68d41646d0 0x7f68d415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.982+0000 7f68cbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f68d406e740 0x7f68d4163fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33746/0 (socket says 172.21.15.19:33746) 2024-04-25T11:37:12.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.982+0000 7f68cbe00640 1 -- 172.21.15.19:0/4091373105 learned_addr learned my addr 172.21.15.19:0/4091373105 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:12.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.982+0000 7f68d3400640 1 -- 172.21.15.19:0/4091373105 >> v1:172.21.15.169:6789/0 conn(0x7f68d41616a0 legacy=0x7f68d406e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:12.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.982+0000 7f68d3400640 1 -- 172.21.15.19:0/4091373105 >> v1:172.21.15.19:6789/0 conn(0x7f68d406e740 legacy=0x7f68d4163fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.982+0000 7f68d2000640 1 -- 172.21.15.19:0/4091373105 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383319212 0 0) 0x7f68d4164c80 con 0x7f68d406e740 2024-04-25T11:37:12.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.982+0000 7f68d3400640 1 -- 172.21.15.19:0/4091373105 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d4160ea0 con 0x7f68d41646d0 2024-04-25T11:37:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.984+0000 7f68d3400640 1 --2- 172.21.15.19:0/4091373105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41646d0 0x7f68d415ded0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009dc0 tx=0x7f68cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d33831e6cfd33fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.984+0000 7f68d2000640 1 -- 172.21.15.19:0/4091373105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc007740 con 0x7f68d41646d0 2024-04-25T11:37:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.984+0000 7f68d2000640 1 -- 172.21.15.19:0/4091373105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68cc0078e0 con 0x7f68d41646d0 2024-04-25T11:37:12.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.984+0000 7f68d2000640 1 -- 172.21.15.19:0/4091373105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68cc007cb0 con 0x7f68d41646d0 2024-04-25T11:37:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.985+0000 7f68d9600640 1 -- 172.21.15.19:0/4091373105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41646d0 msgr2=0x7f68d415ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.985+0000 7f68d9600640 1 --2- 172.21.15.19:0/4091373105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41646d0 0x7f68d415ded0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009dc0 tx=0x7f68cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.985+0000 7f68d9600640 1 -- 172.21.15.19:0/4091373105 shutdown_connections 2024-04-25T11:37:12.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.985+0000 7f68d9600640 1 --2- 172.21.15.19:0/4091373105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41646d0 0x7f68d415ded0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.985+0000 7f68d9600640 1 -- 172.21.15.19:0/4091373105 >> 172.21.15.19:0/4091373105 conn(0x7f68d40d27c0 msgr2=0x7f68d40d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.985+0000 7f68d9600640 1 -- 172.21.15.19:0/4091373105 shutdown_connections 2024-04-25T11:37:12.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.985+0000 7f68d9600640 1 -- 172.21.15.19:0/4091373105 wait complete. 2024-04-25T11:37:12.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.987+0000 7f68d9600640 1 Processor -- start 2024-04-25T11:37:12.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.987+0000 7f68d9600640 1 -- start start 2024-04-25T11:37:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41616a0 0x7f68d415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d41646d0 0x7f68d415a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d414e380 0x7f68d40c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d9600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68d416b280 con 0x7f68d41616a0 2024-04-25T11:37:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f68d416b100 con 0x7f68d41646d0 2024-04-25T11:37:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f68d416b400 con 0x7f68d414e380 2024-04-25T11:37:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41616a0 0x7f68d415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41616a0 0x7f68d415d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60786/0 (socket says 172.21.15.19:60786) 2024-04-25T11:37:12.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.989+0000 7f68d2a00640 1 -- 172.21.15.19:0/4057753413 learned_addr learned my addr 172.21.15.19:0/4057753413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:12.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68cbe00640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d41646d0 0x7f68d415a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68d3400640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d414e380 0x7f68d40c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68d2a00640 1 -- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d414e380 msgr2=0x7f68d40c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68d2a00640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d414e380 0x7f68d40c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68d2a00640 1 -- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d41646d0 msgr2=0x7f68d415a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68d2a00640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d41646d0 0x7f68d415a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68d2a00640 1 -- 172.21.15.19:0/4057753413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d415c390 con 0x7f68d41616a0 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68d2a00640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41616a0 0x7f68d415d410 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f68c400c420 tx=0x7f68c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.990+0000 7f68caa00640 1 -- 172.21.15.19:0/4057753413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4017070 con 0x7f68d41616a0 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.991+0000 7f68caa00640 1 -- 172.21.15.19:0/4057753413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c40052f0 con 0x7f68d41616a0 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.991+0000 7f68caa00640 1 -- 172.21.15.19:0/4057753413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4003830 con 0x7f68d41616a0 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.991+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d415c650 con 0x7f68d41616a0 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.991+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d415cc10 con 0x7f68d41616a0 2024-04-25T11:37:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.992+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d415c110 con 0x7f68d41616a0 2024-04-25T11:37:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.993+0000 7f68caa00640 1 -- 172.21.15.19:0/4057753413 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f68c401a070 con 0x7f68d41616a0 2024-04-25T11:37:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.994+0000 7f68caa00640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68b00744e0 0x7f68b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.994+0000 7f68caa00640 1 -- 172.21.15.19:0/4057753413 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6569+0+0 (secure 0 0 0) 0x7f68c4094980 con 0x7f68d41616a0 2024-04-25T11:37:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.995+0000 7f68cbe00640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68b00744e0 0x7f68b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:12.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.996+0000 7f68cbe00640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68b00744e0 0x7f68b00769f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f68d41604b0 tx=0x7f68bc007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:12.999+0000 7f68caa00640 1 -- 172.21.15.19:0/4057753413 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68d415c110 con 0x7f68d41616a0 2024-04-25T11:37:13.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.248+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68d40d3980 con 0x7f68d41616a0 2024-04-25T11:37:13.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.249+0000 7f68caa00640 1 -- 172.21.15.19:0/4057753413 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+5184 (secure 0 0 0) 0x7f68c405ed10 con 0x7f68d41616a0 2024-04-25T11:37:13.237 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:13.237 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":155,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:12.663722+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"a","rank":0,"incarnation":155,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/390026662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":390026662},{"type":"v1","addr":"172.21.15.19:6835","nonce":390026662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:13.237 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 155 2024-04-25T11:37:13.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.252+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68b00744e0 msgr2=0x7f68b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:13.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.252+0000 7f68d9600640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68b00744e0 0x7f68b00769f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f68d41604b0 tx=0x7f68bc007d20 comp rx=0 tx=0).stop 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.252+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41616a0 msgr2=0x7f68d415d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.252+0000 7f68d9600640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41616a0 0x7f68d415d410 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f68c400c420 tx=0x7f68c400c8e0 comp rx=0 tx=0).stop 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 shutdown_connections 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f68b00744e0 0x7f68b00769f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68d41646d0 0x7f68d415a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68d414e380 0x7f68d40c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 --2- 172.21.15.19:0/4057753413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68d41616a0 0x7f68d415d410 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 >> 172.21.15.19:0/4057753413 conn(0x7f68d40d27c0 msgr2=0x7f68d40d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 shutdown_connections 2024-04-25T11:37:13.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:13.253+0000 7f68d9600640 1 -- 172.21.15.19:0/4057753413 wait complete. 2024-04-25T11:37:13.273 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:37:15.680 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:15.692+0000 7fe6cd600640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-04-25T11:37:15.685 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:15.697+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:37:15.685 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:15.697+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:37:15.717 INFO:tasks.ceph.mds.a.smithi019.stderr: -30> 2024-04-25T11:37:15.692+0000 7fe6cd600640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-04-25T11:37:15.851 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:37:18.274 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:18.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.580+0000 7f027f200640 1 Processor -- start 2024-04-25T11:37:18.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.580+0000 7f027f200640 1 -- start start 2024-04-25T11:37:18.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.582+0000 7f027f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02800b7c60 0x7f02800d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:18.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.582+0000 7f027f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f028006c8a0 0x7f028006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:18.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.582+0000 7f027f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02800d1f40 con 0x7f028006ecb0 2024-04-25T11:37:18.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.582+0000 7f027f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f02800d0430 con 0x7f028006c8a0 2024-04-25T11:37:18.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.582+0000 7f027f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02800d21e0 con 0x7f02800b7c60 2024-04-25T11:37:18.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.583+0000 7f027d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02800b7c60 0x7f02800d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:18.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.583+0000 7f027e800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f028006c8a0 0x7f028006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:18.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.583+0000 7f027d400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02800b7c60 0x7f02800d3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:50822/0 (socket says 172.21.15.19:50822) 2024-04-25T11:37:18.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.583+0000 7f027d400640 1 -- 172.21.15.19:0/659120422 learned_addr learned my addr 172.21.15.19:0/659120422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:18.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.583+0000 7f027de00640 1 --1- 172.21.15.19:0/659120422 >> v1:172.21.15.169:6789/0 conn(0x7f028006ecb0 0x7f02800b7550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:42038/0 (socket says 172.21.15.19:42038) 2024-04-25T11:37:18.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.584+0000 7f027d400640 1 -- 172.21.15.19:0/659120422 >> v2:172.21.15.169:3300/0 conn(0x7f028006c8a0 msgr2=0x7f028006ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:18.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.584+0000 7f027d400640 1 --2- 172.21.15.19:0/659120422 >> v2:172.21.15.169:3300/0 conn(0x7f028006c8a0 0x7f028006ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.584+0000 7f027d400640 1 -- 172.21.15.19:0/659120422 >> v1:172.21.15.169:6789/0 conn(0x7f028006ecb0 legacy=0x7f02800b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:18.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.584+0000 7f027d400640 1 -- 172.21.15.19:0/659120422 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02800d06d0 con 0x7f02800b7c60 2024-04-25T11:37:18.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.585+0000 7f027d400640 1 --2- 172.21.15.19:0/659120422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02800b7c60 0x7f02800d3160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0268009870 tx=0x7f026802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd372fb80f62562d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.586+0000 7f027ca00640 1 -- 172.21.15.19:0/659120422 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268037070 con 0x7f02800b7c60 2024-04-25T11:37:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.586+0000 7f027ca00640 1 -- 172.21.15.19:0/659120422 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0268032440 con 0x7f02800b7c60 2024-04-25T11:37:18.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.586+0000 7f027ca00640 1 -- 172.21.15.19:0/659120422 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268032810 con 0x7f02800b7c60 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.586+0000 7f027f200640 1 -- 172.21.15.19:0/659120422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02800b7c60 msgr2=0x7f02800d3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.586+0000 7f027f200640 1 --2- 172.21.15.19:0/659120422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02800b7c60 0x7f02800d3160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0268009870 tx=0x7f026802a250 comp rx=0 tx=0).stop 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.586+0000 7f027f200640 1 -- 172.21.15.19:0/659120422 shutdown_connections 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.587+0000 7f027f200640 1 --2- 172.21.15.19:0/659120422 >> v2:172.21.15.169:3300/0 conn(0x7f028006c8a0 0x7f028006ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.587+0000 7f027f200640 1 --2- 172.21.15.19:0/659120422 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02800b7c60 0x7f02800d3160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.587+0000 7f027f200640 1 -- 172.21.15.19:0/659120422 >> 172.21.15.19:0/659120422 conn(0x7f028009b7d0 msgr2=0x7f028009bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.587+0000 7f027f200640 1 -- 172.21.15.19:0/659120422 shutdown_connections 2024-04-25T11:37:18.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.587+0000 7f027f200640 1 -- 172.21.15.19:0/659120422 wait complete. 2024-04-25T11:37:18.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.589+0000 7f027f200640 1 Processor -- start 2024-04-25T11:37:18.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.589+0000 7f027f200640 1 -- start start 2024-04-25T11:37:18.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f028006c8a0 0x7f028015b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028006ecb0 0x7f02800c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02800c6dd0 0x7f02800b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0280163bb0 con 0x7f028006c8a0 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02800d24c0 con 0x7f02800c6dd0 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0280163d30 con 0x7f028006ecb0 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f028006c8a0 0x7f028015b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f028006c8a0 0x7f028015b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47232/0 (socket says 172.21.15.19:47232) 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027de00640 1 -- 172.21.15.19:0/685961303 learned_addr learned my addr 172.21.15.19:0/685961303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.591+0000 7f027e800640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02800c6dd0 0x7f02800b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:18.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027d400640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028006ecb0 0x7f02800c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027de00640 1 -- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028006ecb0 msgr2=0x7f02800c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027de00640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028006ecb0 0x7f02800c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027de00640 1 -- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02800c6dd0 msgr2=0x7f02800b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027de00640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02800c6dd0 0x7f02800b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027de00640 1 -- 172.21.15.19:0/685961303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02800a0130 con 0x7f028006c8a0 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027de00640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f028006c8a0 0x7f028015b660 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f027000c3c0 tx=0x7f027000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f0276000640 1 -- 172.21.15.19:0/685961303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270017070 con 0x7f028006c8a0 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f0276000640 1 -- 172.21.15.19:0/685961303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0270005c70 con 0x7f028006c8a0 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f0276000640 1 -- 172.21.15.19:0/685961303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700072f0 con 0x7f028006c8a0 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.592+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02800a0410 con 0x7f028006c8a0 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.593+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02800c65d0 con 0x7f028006c8a0 2024-04-25T11:37:18.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.594+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0280050740 con 0x7f028006c8a0 2024-04-25T11:37:18.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.595+0000 7f0276000640 1 -- 172.21.15.19:0/685961303 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f027001a070 con 0x7f028006c8a0 2024-04-25T11:37:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.596+0000 7f0276000640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f025c0744b0 0x7f025c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:18.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.596+0000 7f0276000640 1 -- 172.21.15.19:0/685961303 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7f02700948d0 con 0x7f028006c8a0 2024-04-25T11:37:18.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.597+0000 7f027d400640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f025c0744b0 0x7f025c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:18.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.598+0000 7f027d400640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f025c0744b0 0x7f025c0769c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0268034000 tx=0x7f026802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:18.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.601+0000 7f0276000640 1 -- 172.21.15.19:0/685961303 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0270005e20 con 0x7f028006c8a0 2024-04-25T11:37:18.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.849+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02800d1830 con 0x7f028006c8a0 2024-04-25T11:37:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.851+0000 7f0276000640 1 -- 172.21.15.19:0/685961303 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) ==== 95+0+5178 (secure 0 0 0) 0x7f027005ed30 con 0x7f028006c8a0 2024-04-25T11:37:18.838 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:18.838 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":160,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3307408746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3307408746},{"type":"v1","addr":"172.21.15.19:6835","nonce":3307408746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:15.698366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 160 2024-04-25T11:37:18.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f025c0744b0 msgr2=0x7f025c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:18.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f025c0744b0 0x7f025c0769c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0268034000 tx=0x7f026802e000 comp rx=0 tx=0).stop 2024-04-25T11:37:18.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f028006c8a0 msgr2=0x7f028015b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f028006c8a0 0x7f028015b660 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f027000c3c0 tx=0x7f027000c880 comp rx=0 tx=0).stop 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 shutdown_connections 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f025c0744b0 0x7f025c0769c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02800c6dd0 0x7f02800b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f028006ecb0 0x7f02800c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 --2- 172.21.15.19:0/685961303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f028006c8a0 0x7f028015b660 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 >> 172.21.15.19:0/685961303 conn(0x7f028009b7d0 msgr2=0x7f02800b7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:18.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 shutdown_connections 2024-04-25T11:37:18.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:18.854+0000 7f027f200640 1 -- 172.21.15.19:0/685961303 wait complete. 2024-04-25T11:37:18.869 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:37:18.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.167+0000 7fe77c600640 1 Processor -- start 2024-04-25T11:37:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.167+0000 7fe77c600640 1 -- start start 2024-04-25T11:37:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.170+0000 7fe77c600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7741443c0 0x7fe77406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.170+0000 7fe77c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe774070370 0x7fe7740cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.170+0000 7fe77c600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7740cff30 con 0x7fe7740cfb20 2024-04-25T11:37:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.170+0000 7fe77c600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe7740708d0 con 0x7fe7741443c0 2024-04-25T11:37:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.170+0000 7fe77c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe77415f160 con 0x7fe774070370 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.170+0000 7fe779c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7741443c0 0x7fe77406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe779c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7741443c0 0x7fe77406fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47238/0 (socket says 172.21.15.19:47238) 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe779c00640 1 -- 172.21.15.19:0/4042850437 learned_addr learned my addr 172.21.15.19:0/4042850437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe779200640 1 --2- 172.21.15.19:0/4042850437 >> v2:172.21.15.169:3300/0 conn(0x7fe774070370 0x7fe7740cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe779c00640 1 -- 172.21.15.19:0/4042850437 >> v2:172.21.15.169:3300/0 conn(0x7fe774070370 msgr2=0x7fe7740cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe779c00640 1 --2- 172.21.15.19:0/4042850437 >> v2:172.21.15.169:3300/0 conn(0x7fe774070370 0x7fe7740cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe779c00640 1 -- 172.21.15.19:0/4042850437 >> v1:172.21.15.19:6789/0 conn(0x7fe7740cfb20 legacy=0x7fe7740ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe779c00640 1 -- 172.21.15.19:0/4042850437 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77415f5f0 con 0x7fe7741443c0 2024-04-25T11:37:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.171+0000 7fe773e00640 1 -- 172.21.15.19:0/4042850437 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2913081506 0 0) 0x7fe76c001490 con 0x7fe7740cfb20 2024-04-25T11:37:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.172+0000 7fe779c00640 1 --2- 172.21.15.19:0/4042850437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe77406fe30 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe77406f920 tx=0x7fe76802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d433820db2742c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.173+0000 7fe773e00640 1 -- 172.21.15.19:0/4042850437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768037070 con 0x7fe7741443c0 2024-04-25T11:37:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.173+0000 7fe773e00640 1 -- 172.21.15.19:0/4042850437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe768032440 con 0x7fe7741443c0 2024-04-25T11:37:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.173+0000 7fe773e00640 1 -- 172.21.15.19:0/4042850437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768032810 con 0x7fe7741443c0 2024-04-25T11:37:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.173+0000 7fe77c600640 1 -- 172.21.15.19:0/4042850437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 msgr2=0x7fe77406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.173+0000 7fe77c600640 1 --2- 172.21.15.19:0/4042850437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe77406fe30 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe77406f920 tx=0x7fe76802a270 comp rx=0 tx=0).stop 2024-04-25T11:37:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.174+0000 7fe77c600640 1 -- 172.21.15.19:0/4042850437 shutdown_connections 2024-04-25T11:37:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.174+0000 7fe77c600640 1 --2- 172.21.15.19:0/4042850437 >> v2:172.21.15.169:3300/0 conn(0x7fe774070370 0x7fe7740cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.174+0000 7fe77c600640 1 --2- 172.21.15.19:0/4042850437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe77406fe30 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.174+0000 7fe77c600640 1 -- 172.21.15.19:0/4042850437 >> 172.21.15.19:0/4042850437 conn(0x7fe7740d1ae0 msgr2=0x7fe7740d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.174+0000 7fe77c600640 1 -- 172.21.15.19:0/4042850437 shutdown_connections 2024-04-25T11:37:19.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.174+0000 7fe77c600640 1 -- 172.21.15.19:0/4042850437 wait complete. 2024-04-25T11:37:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.176+0000 7fe77c600640 1 Processor -- start 2024-04-25T11:37:19.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.176+0000 7fe77c600640 1 -- start start 2024-04-25T11:37:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe774070370 0x7fe77413bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cfb20 0x7fe774157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77c600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe7741584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77c600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe77415edf0 con 0x7fe7741443c0 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe77415eb10 con 0x7fe7740cfb20 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe77406d5d0 con 0x7fe774070370 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe7741584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe7741584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47242/0 (socket says 172.21.15.19:47242) 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe77a600640 1 -- 172.21.15.19:0/3435034373 learned_addr learned my addr 172.21.15.19:0/3435034373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe779c00640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe774070370 0x7fe77413bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.178+0000 7fe779200640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cfb20 0x7fe774157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe77a600640 1 -- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe774070370 msgr2=0x7fe77413bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe77a600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe774070370 0x7fe77413bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe77a600640 1 -- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cfb20 msgr2=0x7fe774157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe77a600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cfb20 0x7fe774157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe77a600640 1 -- 172.21.15.19:0/3435034373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe774157490 con 0x7fe7741443c0 2024-04-25T11:37:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe779c00640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe774070370 0x7fe77413bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:37:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe77a600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe7741584a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c780 tx=0x7fe76c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe771600640 1 -- 172.21.15.19:0/3435034373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c003b70 con 0x7fe7741443c0 2024-04-25T11:37:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe771600640 1 -- 172.21.15.19:0/3435034373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe76c003d10 con 0x7fe7741443c0 2024-04-25T11:37:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe771600640 1 -- 172.21.15.19:0/3435034373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c005270 con 0x7fe7741443c0 2024-04-25T11:37:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.179+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741e8870 con 0x7fe7741443c0 2024-04-25T11:37:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.180+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741e8e30 con 0x7fe7741443c0 2024-04-25T11:37:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.181+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7741e85e0 con 0x7fe7741443c0 2024-04-25T11:37:19.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.182+0000 7fe771600640 1 -- 172.21.15.19:0/3435034373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe76c005410 con 0x7fe7741443c0 2024-04-25T11:37:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.184+0000 7fe771600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7540744b0 0x7fe7540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.184+0000 7fe771600640 1 -- 172.21.15.19:0/3435034373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7fe76c0955a0 con 0x7fe7741443c0 2024-04-25T11:37:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.184+0000 7fe779c00640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7540744b0 0x7fe7540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.186+0000 7fe779c00640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7540744b0 0x7fe7540769c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe768034000 tx=0x7fe76802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:19.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.189+0000 7fe771600640 1 -- 172.21.15.19:0/3435034373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe774157490 con 0x7fe7741443c0 2024-04-25T11:37:19.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.434+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe774050740 con 0x7fe7741443c0 2024-04-25T11:37:19.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.436+0000 7fe771600640 1 -- 172.21.15.19:0/3435034373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) ==== 95+0+5178 (secure 0 0 0) 0x7fe7740cff30 con 0x7fe7741443c0 2024-04-25T11:37:19.423 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:19.423 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":160,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3307408746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3307408746},{"type":"v1","addr":"172.21.15.19:6835","nonce":3307408746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:15.698366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:19.423 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 160 2024-04-25T11:37:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.438+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7540744b0 msgr2=0x7fe7540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.438+0000 7fe77c600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7540744b0 0x7fe7540769c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe768034000 tx=0x7fe76802e000 comp rx=0 tx=0).stop 2024-04-25T11:37:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.438+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 msgr2=0x7fe7741584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.438+0000 7fe77c600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe7741584a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00c780 tx=0x7fe76c00cb80 comp rx=0 tx=0).stop 2024-04-25T11:37:19.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.438+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 shutdown_connections 2024-04-25T11:37:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.438+0000 7fe77c600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7540744b0 0x7fe7540769c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.438+0000 7fe77c600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7740cfb20 0x7fe774157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.439+0000 7fe77c600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7741443c0 0x7fe7741584a0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.439+0000 7fe77c600640 1 --2- 172.21.15.19:0/3435034373 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe774070370 0x7fe77413bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.439+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 >> 172.21.15.19:0/3435034373 conn(0x7fe7740d1ae0 msgr2=0x7fe774143a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.439+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 shutdown_connections 2024-04-25T11:37:19.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.439+0000 7fe77c600640 1 -- 172.21.15.19:0/3435034373 wait complete. 2024-04-25T11:37:19.456 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:19.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.756+0000 7efd3e200640 1 Processor -- start 2024-04-25T11:37:19.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.756+0000 7efd3e200640 1 -- start start 2024-04-25T11:37:19.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.758+0000 7efd3e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 0x7efd380d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.758+0000 7efd3e200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd380d1530 con 0x7efd38164bc0 2024-04-25T11:37:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.758+0000 7efd3e200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd381640e0 con 0x7efd380d2ad0 2024-04-25T11:37:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.758+0000 7efd3e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd38164730 con 0x7efd380d0ae0 2024-04-25T11:37:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.758+0000 7efd37400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd38164bc0 0x7efd381612f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35862/0 (socket says 172.21.15.19:35862) 2024-04-25T11:37:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.758+0000 7efd37400640 1 -- 172.21.15.19:0/3609173971 learned_addr learned my addr 172.21.15.19:0/3609173971 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.759+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892195448 0 0) 0x7efd380d1530 con 0x7efd38164bc0 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.759+0000 7efd36a00640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 0x7efd380d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.759+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd1c003520 con 0x7efd38164bc0 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.759+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3878991631 0 0) 0x7efd1c003520 con 0x7efd38164bc0 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.759+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 msgr2=0x7efd380d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.759+0000 7efd36000640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 0x7efd380d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.759+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 >> v1:172.21.15.169:6789/0 conn(0x7efd380d2ad0 legacy=0x7efd380d2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd381643c0 con 0x7efd38164bc0 2024-04-25T11:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efd28001670 con 0x7efd38164bc0 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 >> v1:172.21.15.19:6789/0 conn(0x7efd38164bc0 legacy=0x7efd381612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd1c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd1c004800 0x7efd1c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 0x7efd1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd38164730 con 0x7efd1c0039e0 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0055c0 con 0x7efd1c004800 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c00bf00 con 0x7efd380d0ae0 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd37400640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd1c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.760+0000 7efd36a00640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd1c004800 0x7efd1c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.761+0000 7efd37e00640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 0x7efd1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.761+0000 7efd37400640 1 -- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 msgr2=0x7efd1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.761+0000 7efd37400640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 0x7efd1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.761+0000 7efd37400640 1 -- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd1c004800 msgr2=0x7efd1c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.761+0000 7efd37400640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd1c004800 0x7efd1c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.761+0000 7efd37400640 1 -- 172.21.15.19:0/3609173971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c003520 con 0x7efd1c0039e0 2024-04-25T11:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.761+0000 7efd37e00640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd380d0ae0 0x7efd1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.762+0000 7efd37400640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd1c003e50 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c160 tx=0x7efd1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d042572825dd09b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.762+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28007c60 con 0x7efd1c0039e0 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.762+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd28005af0 con 0x7efd1c0039e0 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.762+0000 7efd36000640 1 -- 172.21.15.19:0/3609173971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28007070 con 0x7efd1c0039e0 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.763+0000 7efd3e200640 1 -- 172.21.15.19:0/3609173971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 msgr2=0x7efd1c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.763+0000 7efd3e200640 1 --2- 172.21.15.19:0/3609173971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd1c003e50 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c160 tx=0x7efd1c005830 comp rx=0 tx=0).stop 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.763+0000 7efd37e00640 1 -- 172.21.15.19:0/3609173971 reap_dead start 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.763+0000 7efd3e200640 1 -- 172.21.15.19:0/3609173971 shutdown_connections 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.763+0000 7efd3e200640 1 -- 172.21.15.19:0/3609173971 >> 172.21.15.19:0/3609173971 conn(0x7efd380b79f0 msgr2=0x7efd380b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.763+0000 7efd3e200640 1 -- 172.21.15.19:0/3609173971 shutdown_connections 2024-04-25T11:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.763+0000 7efd3e200640 1 -- 172.21.15.19:0/3609173971 wait complete. 2024-04-25T11:37:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.765+0000 7efd3e200640 1 Processor -- start 2024-04-25T11:37:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.766+0000 7efd3e200640 1 -- start start 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd3e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd380cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd3e200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd380cfeb0 0x7efd380ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd3e200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3815dbc0 0x7efd380d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd3e200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd380d4610 con 0x7efd1c0039e0 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd3e200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd380d1140 con 0x7efd380cfeb0 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd3e200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd380d4790 con 0x7efd3815dbc0 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd37400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd380cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.767+0000 7efd37400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd380cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47270/0 (socket says 172.21.15.19:47270) 2024-04-25T11:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37400640 1 -- 172.21.15.19:0/3733420951 learned_addr learned my addr 172.21.15.19:0/3733420951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd36a00640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd380cfeb0 0x7efd380ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37e00640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3815dbc0 0x7efd380d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37400640 1 -- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3815dbc0 msgr2=0x7efd380d5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37400640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3815dbc0 0x7efd380d5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37400640 1 -- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd380cfeb0 msgr2=0x7efd380ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37400640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd380cfeb0 0x7efd380ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37400640 1 -- 172.21.15.19:0/3733420951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd380cd2e0 con 0x7efd1c0039e0 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.768+0000 7efd37400640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd380cb0f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c280 tx=0x7efd28001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.769+0000 7efd2f400640 1 -- 172.21.15.19:0/3733420951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28044070 con 0x7efd1c0039e0 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.769+0000 7efd2f400640 1 -- 172.21.15.19:0/3733420951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd28038040 con 0x7efd1c0039e0 2024-04-25T11:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.769+0000 7efd2f400640 1 -- 172.21.15.19:0/3733420951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28004c20 con 0x7efd1c0039e0 2024-04-25T11:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.769+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd380cd5c0 con 0x7efd1c0039e0 2024-04-25T11:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.769+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd380cdb80 con 0x7efd1c0039e0 2024-04-25T11:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.770+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd38050740 con 0x7efd1c0039e0 2024-04-25T11:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.771+0000 7efd2f400640 1 -- 172.21.15.19:0/3733420951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd28007070 con 0x7efd1c0039e0 2024-04-25T11:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.772+0000 7efd2f400640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd140744b0 0x7efd140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.772+0000 7efd2f400640 1 -- 172.21.15.19:0/3733420951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7efd280b51d0 con 0x7efd1c0039e0 2024-04-25T11:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.773+0000 7efd36a00640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd140744b0 0x7efd140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:19.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.775+0000 7efd36a00640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd140744b0 0x7efd140769c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7efd380b4b70 tx=0x7efd20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:19.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:19.777+0000 7efd2f400640 1 -- 172.21.15.19:0/3733420951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd2807f510 con 0x7efd1c0039e0 2024-04-25T11:37:20.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.026+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd380d40a0 con 0x7efd1c0039e0 2024-04-25T11:37:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.027+0000 7efd2f400640 1 -- 172.21.15.19:0/3733420951 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) ==== 95+0+5178 (secure 0 0 0) 0x7efd2807ee10 con 0x7efd1c0039e0 2024-04-25T11:37:20.014 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:20.015 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":160,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3307408746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3307408746},{"type":"v1","addr":"172.21.15.19:6835","nonce":3307408746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":160}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:15.698366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 160 2024-04-25T11:37:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd140744b0 msgr2=0x7efd140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd140744b0 0x7efd140769c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7efd380b4b70 tx=0x7efd20008040 comp rx=0 tx=0).stop 2024-04-25T11:37:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 msgr2=0x7efd380cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd380cb0f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c280 tx=0x7efd28001970 comp rx=0 tx=0).stop 2024-04-25T11:37:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 shutdown_connections 2024-04-25T11:37:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd140744b0 0x7efd140769c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd380cfeb0 0x7efd380ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd3815dbc0 0x7efd380d5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 --2- 172.21.15.19:0/3733420951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd1c0039e0 0x7efd380cb0f0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 >> 172.21.15.19:0/3733420951 conn(0x7efd380b79f0 msgr2=0x7efd380b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 shutdown_connections 2024-04-25T11:37:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.030+0000 7efd3e200640 1 -- 172.21.15.19:0/3733420951 wait complete. 2024-04-25T11:37:20.047 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2024-04-25T11:37:20.047 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2024-04-25T11:37:20.048 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:20.048 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:20.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:37:20.102 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:37:20.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:20.103 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:37:20.158 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:37:20.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:37:20.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:37:20.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.464+0000 7f2260a00640 1 Processor -- start 2024-04-25T11:37:20.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.464+0000 7f2260a00640 1 -- start start 2024-04-25T11:37:20.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.466+0000 7f2260a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c06ecb0 0x7f225c0d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:20.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f2260a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f225c06c8a0 0x7f225c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:20.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f2260a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225c0d1f40 con 0x7f225c0b7550 2024-04-25T11:37:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f2260a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f225c0d0430 con 0x7f225c06c8a0 2024-04-25T11:37:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f2260a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f225c0d21e0 con 0x7f225c06ecb0 2024-04-25T11:37:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f225be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f225c06c8a0 0x7f225c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f225be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f225c06c8a0 0x7f225c06ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47280/0 (socket says 172.21.15.19:47280) 2024-04-25T11:37:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f225be00640 1 -- 172.21.15.19:0/3665334663 learned_addr learned my addr 172.21.15.19:0/3665334663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f225aa00640 1 --1- 172.21.15.19:0/3665334663 >> v1:172.21.15.19:6789/0 conn(0x7f225c0b7550 0x7f225c0b7960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35870/0 (socket says 172.21.15.19:35870) 2024-04-25T11:37:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.467+0000 7f225b400640 1 --2- 172.21.15.19:0/3665334663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c06ecb0 0x7f225c0d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.468+0000 7f225be00640 1 -- 172.21.15.19:0/3665334663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c06ecb0 msgr2=0x7f225c0d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.468+0000 7f225be00640 1 --2- 172.21.15.19:0/3665334663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c06ecb0 0x7f225c0d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.468+0000 7f225be00640 1 -- 172.21.15.19:0/3665334663 >> v1:172.21.15.19:6789/0 conn(0x7f225c0b7550 legacy=0x7f225c0b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.468+0000 7f225be00640 1 -- 172.21.15.19:0/3665334663 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c0d06d0 con 0x7f225c06c8a0 2024-04-25T11:37:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.469+0000 7f225be00640 1 --2- 172.21.15.19:0/3665334663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06c8a0 0x7f225c06ccb0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2254009e50 tx=0x7f225402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b1e13e03b12d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.469+0000 7f225a000640 1 -- 172.21.15.19:0/3665334663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254036070 con 0x7f225c06c8a0 2024-04-25T11:37:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.469+0000 7f225a000640 1 -- 172.21.15.19:0/3665334663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2254031600 con 0x7f225c06c8a0 2024-04-25T11:37:20.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.469+0000 7f225a000640 1 -- 172.21.15.19:0/3665334663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540319d0 con 0x7f225c06c8a0 2024-04-25T11:37:20.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 -- 172.21.15.19:0/3665334663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06c8a0 msgr2=0x7f225c06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 --2- 172.21.15.19:0/3665334663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06c8a0 0x7f225c06ccb0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2254009e50 tx=0x7f225402a730 comp rx=0 tx=0).stop 2024-04-25T11:37:20.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 -- 172.21.15.19:0/3665334663 shutdown_connections 2024-04-25T11:37:20.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 --2- 172.21.15.19:0/3665334663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06c8a0 0x7f225c06ccb0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 --2- 172.21.15.19:0/3665334663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c06ecb0 0x7f225c0d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 -- 172.21.15.19:0/3665334663 >> 172.21.15.19:0/3665334663 conn(0x7f225c09b7d0 msgr2=0x7f225c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 -- 172.21.15.19:0/3665334663 shutdown_connections 2024-04-25T11:37:20.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.470+0000 7f2260a00640 1 -- 172.21.15.19:0/3665334663 wait complete. 2024-04-25T11:37:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.472+0000 7f2260a00640 1 Processor -- start 2024-04-25T11:37:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.472+0000 7f2260a00640 1 -- start start 2024-04-25T11:37:20.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f2260a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c06c8a0 0x7f225c1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f2260a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06ecb0 0x7f225c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f2260a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c0b7550 0x7f225c0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f2260a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f225c163bb0 con 0x7f225c06ecb0 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f2260a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f225c0d24c0 con 0x7f225c06c8a0 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f2260a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f225c163d30 con 0x7f225c0b7550 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f225aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06ecb0 0x7f225c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f225aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06ecb0 0x7f225c0c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47286/0 (socket says 172.21.15.19:47286) 2024-04-25T11:37:20.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.474+0000 7f225aa00640 1 -- 172.21.15.19:0/3601501856 learned_addr learned my addr 172.21.15.19:0/3601501856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225b400640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c06c8a0 0x7f225c1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:20.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225be00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c0b7550 0x7f225c0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225aa00640 1 -- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c0b7550 msgr2=0x7f225c0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225aa00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c0b7550 0x7f225c0b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225aa00640 1 -- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c06c8a0 msgr2=0x7f225c1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225aa00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c06c8a0 0x7f225c1599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225aa00640 1 -- 172.21.15.19:0/3601501856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c1fa2b0 con 0x7f225c06ecb0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.475+0000 7f225aa00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06ecb0 0x7f225c0c8500 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f225c0d1cb0 tx=0x7f224400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.476+0000 7f2253400640 1 -- 172.21.15.19:0/3601501856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440074e0 con 0x7f225c06ecb0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.476+0000 7f2253400640 1 -- 172.21.15.19:0/3601501856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2244007680 con 0x7f225c06ecb0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.476+0000 7f2253400640 1 -- 172.21.15.19:0/3601501856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440049e0 con 0x7f225c06ecb0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.476+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c160ca0 con 0x7f225c06ecb0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.476+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c1fa520 con 0x7f225c06ecb0 2024-04-25T11:37:20.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.477+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c0c8e50 con 0x7f225c06ecb0 2024-04-25T11:37:20.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.478+0000 7f2253400640 1 -- 172.21.15.19:0/3601501856 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f224401a020 con 0x7f225c06ecb0 2024-04-25T11:37:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.479+0000 7f2253400640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22380744b0 0x7f22380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.480+0000 7f2253400640 1 -- 172.21.15.19:0/3601501856 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7f2244094c90 con 0x7f225c06ecb0 2024-04-25T11:37:20.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.480+0000 7f225b400640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22380744b0 0x7f22380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.482+0000 7f225b400640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22380744b0 0x7f22380769c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f225c073a30 tx=0x7f224c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.484+0000 7f2253400640 1 -- 172.21.15.19:0/3601501856 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f224405f070 con 0x7f225c06ecb0 2024-04-25T11:37:20.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.728+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f225c050740 con 0x7f225c06ecb0 2024-04-25T11:37:20.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.739+0000 7f2253400640 1 -- 172.21.15.19:0/3601501856 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v161) ==== 161+0+0 (secure 0 0 0) 0x7f224405e970 con 0x7f225c06ecb0 2024-04-25T11:37:20.726 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:37:20.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22380744b0 msgr2=0x7f22380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22380744b0 0x7f22380769c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f225c073a30 tx=0x7f224c008040 comp rx=0 tx=0).stop 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06ecb0 msgr2=0x7f225c0c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06ecb0 0x7f225c0c8500 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f225c0d1cb0 tx=0x7f224400c820 comp rx=0 tx=0).stop 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 shutdown_connections 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22380744b0 0x7f22380769c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c0b7550 0x7f225c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c06ecb0 0x7f225c0c8500 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 --2- 172.21.15.19:0/3601501856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c06c8a0 0x7f225c1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 >> 172.21.15.19:0/3601501856 conn(0x7f225c09b7d0 msgr2=0x7f225c0d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 shutdown_connections 2024-04-25T11:37:20.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:20.742+0000 7f2260a00640 1 -- 172.21.15.19:0/3601501856 wait complete. 2024-04-25T11:37:20.759 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:37:21.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.065+0000 7f38c0a00640 1 Processor -- start 2024-04-25T11:37:21.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.065+0000 7f38c0a00640 1 -- start start 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38c0a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38bc1443c0 0x7f38bc06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38c0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc070370 0x7f38bc0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38c0a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc0cff30 con 0x7f38bc0cfb20 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38c0a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc0708d0 con 0x7f38bc1443c0 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38c0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc15f160 con 0x7f38bc070370 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38b9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38bc1443c0 0x7f38bc06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38b9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f38bc1443c0 0x7f38bc06fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47298/0 (socket says 172.21.15.19:47298) 2024-04-25T11:37:21.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.067+0000 7f38b9e00640 1 -- 172.21.15.19:0/3739294693 learned_addr learned my addr 172.21.15.19:0/3739294693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:21.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.068+0000 7f38b9e00640 1 -- 172.21.15.19:0/3739294693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc070370 msgr2=0x7f38bc0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:21.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.068+0000 7f38b9e00640 1 --2- 172.21.15.19:0/3739294693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc070370 0x7f38bc0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.068+0000 7f38b9e00640 1 -- 172.21.15.19:0/3739294693 >> v1:172.21.15.19:6789/0 conn(0x7f38bc0cfb20 legacy=0x7f38bc0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.068+0000 7f38b9e00640 1 -- 172.21.15.19:0/3739294693 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc15f5f0 con 0x7f38bc1443c0 2024-04-25T11:37:21.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.069+0000 7f38b9e00640 1 --2- 172.21.15.19:0/3739294693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1443c0 0x7f38bc06fe30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009870 tx=0x7f38ac02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b656fe21b933a40e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:21.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.069+0000 7f38b9400640 1 -- 172.21.15.19:0/3739294693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac037070 con 0x7f38bc1443c0 2024-04-25T11:37:21.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.069+0000 7f38b9400640 1 -- 172.21.15.19:0/3739294693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac032440 con 0x7f38bc1443c0 2024-04-25T11:37:21.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.069+0000 7f38b9400640 1 -- 172.21.15.19:0/3739294693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac032810 con 0x7f38bc1443c0 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 -- 172.21.15.19:0/3739294693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1443c0 msgr2=0x7f38bc06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 --2- 172.21.15.19:0/3739294693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1443c0 0x7f38bc06fe30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009870 tx=0x7f38ac02a250 comp rx=0 tx=0).stop 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 -- 172.21.15.19:0/3739294693 shutdown_connections 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 --2- 172.21.15.19:0/3739294693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc070370 0x7f38bc0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 --2- 172.21.15.19:0/3739294693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1443c0 0x7f38bc06fe30 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 -- 172.21.15.19:0/3739294693 >> 172.21.15.19:0/3739294693 conn(0x7f38bc0d1ae0 msgr2=0x7f38bc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 -- 172.21.15.19:0/3739294693 shutdown_connections 2024-04-25T11:37:21.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.070+0000 7f38c0a00640 1 -- 172.21.15.19:0/3739294693 wait complete. 2024-04-25T11:37:21.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.073+0000 7f38c0a00640 1 Processor -- start 2024-04-25T11:37:21.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.073+0000 7f38c0a00640 1 -- start start 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.074+0000 7f38c0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38bc070370 0x7f38bc1e9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.074+0000 7f38c0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc0cfb20 0x7f38bc15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.074+0000 7f38c0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1f9b70 0x7f38bc1f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.074+0000 7f38c0a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc15edf0 con 0x7f38bc1f9b70 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.074+0000 7f38c0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc15eb10 con 0x7f38bc070370 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.074+0000 7f38c0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc06d5d0 con 0x7f38bc0cfb20 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.075+0000 7f38ba800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1f9b70 0x7f38bc1f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.075+0000 7f38ba800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1f9b70 0x7f38bc1f9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47310/0 (socket says 172.21.15.19:47310) 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.075+0000 7f38ba800640 1 -- 172.21.15.19:0/4240817034 learned_addr learned my addr 172.21.15.19:0/4240817034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.075+0000 7f38b9e00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38bc070370 0x7f38bc1e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.075+0000 7f38b1400640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc0cfb20 0x7f38bc15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38ba800640 1 -- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc0cfb20 msgr2=0x7f38bc15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38ba800640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc0cfb20 0x7f38bc15aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38ba800640 1 -- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38bc070370 msgr2=0x7f38bc1e9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38ba800640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38bc070370 0x7f38bc1e9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38ba800640 1 -- 172.21.15.19:0/4240817034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc1ea950 con 0x7f38bc1f9b70 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38b1400640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc0cfb20 0x7f38bc15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:37:21.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38ba800640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1f9b70 0x7f38bc1f9f80 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c6b0 tx=0x7f38b400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38b2a00640 1 -- 172.21.15.19:0/4240817034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4003b70 con 0x7f38bc1f9b70 2024-04-25T11:37:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38b2a00640 1 -- 172.21.15.19:0/4240817034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b4003d30 con 0x7f38bc1f9b70 2024-04-25T11:37:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38b2a00640 1 -- 172.21.15.19:0/4240817034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4005310 con 0x7f38bc1f9b70 2024-04-25T11:37:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc1fa4c0 con 0x7f38bc1f9b70 2024-04-25T11:37:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.076+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc1ea170 con 0x7f38bc1f9b70 2024-04-25T11:37:21.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.078+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc1ea3e0 con 0x7f38bc1f9b70 2024-04-25T11:37:21.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.079+0000 7f38b2a00640 1 -- 172.21.15.19:0/4240817034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f38b40054b0 con 0x7f38bc1f9b70 2024-04-25T11:37:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.080+0000 7f38b2a00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38980744b0 0x7f38980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.080+0000 7f38b2a00640 1 -- 172.21.15.19:0/4240817034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7f38b4095340 con 0x7f38bc1f9b70 2024-04-25T11:37:21.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.081+0000 7f38b9e00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38980744b0 0x7f38980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.082+0000 7f38b9e00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38980744b0 0x7f38980769c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f38ac034000 tx=0x7f38ac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:21.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.085+0000 7f38b2a00640 1 -- 172.21.15.19:0/4240817034 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38bc1fa4c0 con 0x7f38bc1f9b70 2024-04-25T11:37:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.368+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f38bc06f950 con 0x7f38bc1f9b70 2024-04-25T11:37:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.765+0000 7f38b2a00640 1 -- 172.21.15.19:0/4240817034 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v162) ==== 101+0+0 (secure 0 0 0) 0x7f38bc1ea950 con 0x7f38bc1f9b70 2024-04-25T11:37:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:37:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38980744b0 msgr2=0x7f38980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38980744b0 0x7f38980769c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f38ac034000 tx=0x7f38ac02e000 comp rx=0 tx=0).stop 2024-04-25T11:37:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1f9b70 msgr2=0x7f38bc1f9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1f9b70 0x7f38bc1f9f80 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c6b0 tx=0x7f38b400cb70 comp rx=0 tx=0).stop 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 shutdown_connections 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38980744b0 0x7f38980769c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38bc1f9b70 0x7f38bc1f9f80 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38bc0cfb20 0x7f38bc15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 --2- 172.21.15.19:0/4240817034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38bc070370 0x7f38bc1e9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 >> 172.21.15.19:0/4240817034 conn(0x7f38bc0d1ae0 msgr2=0x7f38bc142350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.769+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 shutdown_connections 2024-04-25T11:37:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.770+0000 7f38c0a00640 1 -- 172.21.15.19:0/4240817034 wait complete. 2024-04-25T11:37:21.787 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:37:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.982+0000 7f928291cf40 1 Processor -- start 2024-04-25T11:37:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.982+0000 7f928291cf40 1 -- start start 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.983+0000 7f928291cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928321e0 0x560892905790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.983+0000 7f928291cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608928325f0 0x560892863a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.983+0000 7f928291cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56089290bdd0 0x560892830f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.983+0000 7f928291cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56089287b3f0 con 0x5608928325f0 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.983+0000 7f928291cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56089287ae70 con 0x5608928321e0 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.983+0000 7f928291cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56089287b6d0 con 0x56089290bdd0 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608928325f0 0x560892863a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608928325f0 0x560892863a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47326/0 (socket says 172.21.15.19:47326) 2024-04-25T11:37:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 -- 172.21.15.19:0/1253303765 learned_addr learned my addr 172.21.15.19:0/1253303765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927b000640 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928321e0 0x560892905790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927ba00640 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56089290bdd0 0x560892830f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 -- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56089290bdd0 msgr2=0x560892830f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56089290bdd0 0x560892830f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 -- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928321e0 msgr2=0x560892905790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928321e0 0x560892905790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927a600640 1 -- 172.21.15.19:0/1253303765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089275bb30 con 0x5608928325f0 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.984+0000 7f927b000640 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928321e0 0x560892905790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.985+0000 7f927a600640 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608928325f0 0x560892863a20 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f9268009570 tx=0x7f9268009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe67084c4325079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.985+0000 7f9279200640 1 -- 172.21.15.19:0/1253303765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268017070 con 0x5608928325f0 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.985+0000 7f9279200640 1 -- 172.21.15.19:0/1253303765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9268006600 con 0x5608928325f0 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.985+0000 7f9279200640 1 -- 172.21.15.19:0/1253303765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92680069b0 con 0x5608928325f0 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.985+0000 7f928291cf40 1 -- 172.21.15.19:0/1253303765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608928325f0 msgr2=0x560892863a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.985+0000 7f928291cf40 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608928325f0 0x560892863a20 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f9268009570 tx=0x7f9268009a30 comp rx=0 tx=0).stop 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.986+0000 7f928291cf40 1 -- 172.21.15.19:0/1253303765 shutdown_connections 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.986+0000 7f928291cf40 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56089290bdd0 0x560892830f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.986+0000 7f928291cf40 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608928325f0 0x560892863a20 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.986+0000 7f928291cf40 1 --2- 172.21.15.19:0/1253303765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928321e0 0x560892905790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.986+0000 7f928291cf40 1 -- 172.21.15.19:0/1253303765 >> 172.21.15.19:0/1253303765 conn(0x560892754ac0 msgr2=0x560892907d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.986+0000 7f928291cf40 1 -- 172.21.15.19:0/1253303765 shutdown_connections 2024-04-25T11:37:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.986+0000 7f928291cf40 1 -- 172.21.15.19:0/1253303765 wait complete. 2024-04-25T11:37:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.987+0000 7f928291cf40 1 Processor -- start 2024-04-25T11:37:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.987+0000 7f928291cf40 1 -- start start 2024-04-25T11:37:21.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.988+0000 7f928291cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608928321e0 0x560892901ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.988+0000 7f928291cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928325f0 0x5608929998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.988+0000 7f928291cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56089290bdd0 0x560892999de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.988+0000 7f928291cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560892908830 con 0x56089290bdd0 2024-04-25T11:37:21.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f928291cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560892905ed0 con 0x5608928325f0 2024-04-25T11:37:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f928291cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560892876460 con 0x5608928321e0 2024-04-25T11:37:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56089290bdd0 0x560892999de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56089290bdd0 0x560892999de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47342/0 (socket says 172.21.15.19:47342) 2024-04-25T11:37:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 learned_addr learned my addr 172.21.15.19:0/870735668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927b000640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608928321e0 0x560892901ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927a600640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928325f0 0x5608929998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608928321e0 msgr2=0x560892901ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5608928321e0 0x560892901ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928325f0 msgr2=0x5608929998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5608928325f0 0x5608929998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.989+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560892998ae0 con 0x56089290bdd0 2024-04-25T11:37:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.990+0000 7f927ba00640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56089290bdd0 0x560892999de0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f9274007e00 tx=0x7f9274006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.990+0000 7f9273400640 1 -- 172.21.15.19:0/870735668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274017070 con 0x56089290bdd0 2024-04-25T11:37:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.990+0000 7f9273400640 1 -- 172.21.15.19:0/870735668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927400f470 con 0x56089290bdd0 2024-04-25T11:37:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.990+0000 7f9273400640 1 -- 172.21.15.19:0/870735668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927400f840 con 0x56089290bdd0 2024-04-25T11:37:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.990+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560892998dc0 con 0x56089290bdd0 2024-04-25T11:37:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.990+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560892875ee0 con 0x56089290bdd0 2024-04-25T11:37:21.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.993+0000 7f9273400640 1 -- 172.21.15.19:0/870735668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f927401a070 con 0x56089290bdd0 2024-04-25T11:37:21.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.994+0000 7f9273400640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f925c072e90 0x7f925c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.994+0000 7f9273400640 1 -- 172.21.15.19:0/870735668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7f9274093770 con 0x56089290bdd0 2024-04-25T11:37:21.981 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:37:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.994+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560892863bc0 0x5608929a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.994+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929a35f0 con 0x560892863bc0 2024-04-25T11:37:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.994+0000 7f927b000640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f925c072e90 0x7f925c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.995+0000 7f927a600640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560892863bc0 0x5608929a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.996+0000 7f927b000640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f925c072e90 0x7f925c0753a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f926c056000 tx=0x7f926c007030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:21.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.996+0000 7f927a600640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560892863bc0 0x5608929a30b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:21.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.999+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v103'319 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:21.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:21.999+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929a39b0 con 0x560892863bc0 2024-04-25T11:37:21.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.002+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v103'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:21.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.002+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac190 con 0x560892863bc0 2024-04-25T11:37:21.992 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:37:21.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.005+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v103'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:21.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.005+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ad540 con 0x560892863bc0 2024-04-25T11:37:21.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.007+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:21.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.007+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ad8e0 con 0x560892863bc0 2024-04-25T11:37:21.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.010+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:21.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.010+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac530 con 0x560892863bc0 2024-04-25T11:37:22.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.014+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.014+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac940 con 0x560892863bc0 2024-04-25T11:37:22.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.017+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v103'325 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.017+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929ac940 con 0x560892863bc0 2024-04-25T11:37:22.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.020+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v103'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.020+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.011 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:37:22.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.024+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v103'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.024+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.027+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.027+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.030+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.030+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.033+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.033+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.037+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v103'331 uv306 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.038+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.041+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'332 uv332 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.041+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.032 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:37:22.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.045+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v103'333 uv333 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.045+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.049+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'334 uv334 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.049+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.053+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'335 uv335 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.053+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.056+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'336 uv336 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.057+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.061+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'337 uv337 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.061+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.065+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v103'338 uv312 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.065+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.068+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v103'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.068+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.059 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:37:22.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.072+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v103'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.072+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.077+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.077+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.080+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.081+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.084+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.085+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.089+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v103'344 uv318 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.089+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.093+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v103'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.093+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.085 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:37:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.097+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v103'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.098+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.102+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.102+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.106+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.106+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x560892863bc0 2024-04-25T11:37:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.111+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.111+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608929adc80 0x5608929b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.111+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ac8d0 con 0x5608929adc80 2024-04-25T11:37:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.112+0000 7f927b000640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608929adc80 0x5608929b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.113+0000 7f927b000640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608929adc80 0x5608929b00a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.117+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'241 uv223 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.117+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929ad1a0 con 0x5608929adc80 2024-04-25T11:37:22.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.122+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v103'242 uv242 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.122+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b05e0 con 0x5608929adc80 2024-04-25T11:37:22.113 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:37:22.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.125+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v103'243 uv243 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.126+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b0980 con 0x5608929adc80 2024-04-25T11:37:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.130+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v103'244 uv244 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.130+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b0d20 con 0x5608929adc80 2024-04-25T11:37:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.134+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v103'245 uv245 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.135+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b10c0 con 0x5608929adc80 2024-04-25T11:37:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.139+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v103'246 uv228 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.139+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929b1460 con 0x5608929adc80 2024-04-25T11:37:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.143+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v103'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.143+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.148+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v103'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.135 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:37:22.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.148+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.153+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v103'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.153+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.157+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v103'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.158+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.162+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v103'251 uv233 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.162+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.167+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v103'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.167+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.171+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v103'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.159 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:37:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.171+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b5ef0 con 0x5608929adc80 2024-04-25T11:37:22.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.176+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v103'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.176+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b5ef0 con 0x5608929adc80 2024-04-25T11:37:22.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.181+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v103'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.181+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.186+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v103'256 uv239 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.186+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.190+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v103'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.190+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.193+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v103'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.181 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:37:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.194+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b2810 con 0x5608929adc80 2024-04-25T11:37:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.197+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v103'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.198+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b2810 con 0x5608929adc80 2024-04-25T11:37:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.202+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v103'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.202+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b1800 con 0x5608929adc80 2024-04-25T11:37:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.207+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v103'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929adc80 2024-04-25T11:37:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.207+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608929b1d10 0x5608929b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.207+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b4670 con 0x5608929b1d10 2024-04-25T11:37:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.208+0000 7f927a600640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608929b1d10 0x5608929b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.209+0000 7f927a600640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608929b1d10 0x5608929b4130 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.213+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.213+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929b4a40 con 0x5608929b1d10 2024-04-25T11:37:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.218+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v103'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.218+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b4a40 con 0x5608929b1d10 2024-04-25T11:37:22.209 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:37:22.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.221+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v103'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.222+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b5a50 con 0x5608929b1d10 2024-04-25T11:37:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.225+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.225+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b5a50 con 0x5608929b1d10 2024-04-25T11:37:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.229+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.229+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b4a40 con 0x5608929b1d10 2024-04-25T11:37:22.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.234+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.234+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b4a40 con 0x560892863bc0 2024-04-25T11:37:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.237+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'52 uv51 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.238+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929b4a40 con 0x560892863bc0 2024-04-25T11:37:22.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.243+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v103'53 uv53 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.243+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b4a40 con 0x560892863bc0 2024-04-25T11:37:22.234 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:37:22.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.247+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v103'54 uv54 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.247+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b4a40 con 0x560892863bc0 2024-04-25T11:37:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.251+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v103'55 uv55 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.251+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608929b5400 0x5608929b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.251+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b7d60 con 0x5608929b5400 2024-04-25T11:37:22.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.252+0000 7f927ba00640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608929b5400 0x5608929b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.254+0000 7f927ba00640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608929b5400 0x5608929b7820 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.257+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.258+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929b8100 con 0x5608929b5400 2024-04-25T11:37:22.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.262+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v103'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.262+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b8620 con 0x5608929b5400 2024-04-25T11:37:22.253 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:37:22.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.266+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v103'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.266+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b9ed0 con 0x5608929b5400 2024-04-25T11:37:22.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.271+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.271+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929ba270 con 0x5608929b5400 2024-04-25T11:37:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.276+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.276+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929b8ec0 con 0x5608929b5400 2024-04-25T11:37:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.280+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.280+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608929b97c0 0x5608929bc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.280+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bcb60 con 0x5608929b97c0 2024-04-25T11:37:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.281+0000 7f927b000640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608929b97c0 0x5608929bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.283+0000 7f927b000640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608929b97c0 0x5608929bc620 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.287+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v103'116 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.287+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929bcb60 con 0x5608929b97c0 2024-04-25T11:37:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.291+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v103'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.291+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bcbe0 con 0x5608929b97c0 2024-04-25T11:37:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.294+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v103'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.282 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:37:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.295+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bdfe0 con 0x5608929b97c0 2024-04-25T11:37:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.299+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v103'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.299+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bdfe0 con 0x5608929b97c0 2024-04-25T11:37:22.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.303+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v103'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.303+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bcfd0 con 0x5608929b97c0 2024-04-25T11:37:22.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.307+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v103'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.307+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bcfd0 con 0x5608929b97c0 2024-04-25T11:37:22.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.311+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v103'122 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.311+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929bcfd0 con 0x5608929b97c0 2024-04-25T11:37:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.315+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v103'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.315+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bcfd0 con 0x5608929b97c0 2024-04-25T11:37:22.306 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:37:22.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.319+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v103'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.319+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bcfd0 con 0x5608929b97c0 2024-04-25T11:37:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.324+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v103'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.324+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bcfd0 con 0x5608929b97c0 2024-04-25T11:37:22.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.329+0000 7f927b000640 1 -- 172.21.15.19:0/870735668 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v103'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f926c05c440 con 0x5608929b97c0 2024-04-25T11:37:22.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.329+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608929bd4e0 0x5608929bf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.329+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929bffb0 con 0x5608929bd4e0 2024-04-25T11:37:22.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.330+0000 7f927a600640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608929bd4e0 0x5608929bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.332+0000 7f927a600640 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608929bd4e0 0x5608929bf900 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.336+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v103'51 uv50 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.336+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929bffb0 con 0x5608929bd4e0 2024-04-25T11:37:22.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.340+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v103'52 uv52 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.340+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929bd4e0 2024-04-25T11:37:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:37:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.343+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v103'53 uv53 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.344+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929bd4e0 2024-04-25T11:37:22.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.348+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v103'54 uv54 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.348+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929bd4e0 2024-04-25T11:37:22.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.352+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v103'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.352+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x560892863bc0 2024-04-25T11:37:22.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.356+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.356+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929c0400 con 0x560892863bc0 2024-04-25T11:37:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.361+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v103'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.361+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x560892863bc0 2024-04-25T11:37:22.352 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:37:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.365+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v103'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.365+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c1410 con 0x560892863bc0 2024-04-25T11:37:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.369+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.370+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c1410 con 0x560892863bc0 2024-04-25T11:37:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.374+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.374+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x560892863bc0 2024-04-25T11:37:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.378+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x560892863bc0 2024-04-25T11:37:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.378+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929b1d10 2024-04-25T11:37:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.383+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v103'104 uv97 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.383+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929b1d10 2024-04-25T11:37:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.388+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v103'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.388+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929b1d10 2024-04-25T11:37:22.379 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:37:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.392+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v103'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.392+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929b1d10 2024-04-25T11:37:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.396+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v103'107 uv107 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.396+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929b1d10 2024-04-25T11:37:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.401+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v103'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.401+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929b1d10 2024-04-25T11:37:22.392 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:37:22.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.405+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v103'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.405+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0400 con 0x5608929b1d10 2024-04-25T11:37:22.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.409+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v103'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.409+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c3210 con 0x5608929b1d10 2024-04-25T11:37:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.414+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v103'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929b1d10 2024-04-25T11:37:22.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.414+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:99 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c08b0 con 0x5608929b5400 2024-04-25T11:37:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.418+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(99 mds0_sessionmap [create] v103'49 uv47 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.418+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:100 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929c0ca0 con 0x5608929b5400 2024-04-25T11:37:22.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.423+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(100 mds0_sessionmap [delete] v103'50 uv50 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.423+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0ca0 con 0x5608929b5400 2024-04-25T11:37:22.415 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:37:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.427+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(101 mds0_sessionmap [create] v103'51 uv51 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.427+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929b5400 2024-04-25T11:37:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.432+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v103'52 uv52 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.432+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929b5400 2024-04-25T11:37:22.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.436+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(103 609.00000000 [create] v103'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.436+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929b5400 2024-04-25T11:37:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.441+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(104 609.00000000 [delete] v103'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.441+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929b5400 2024-04-25T11:37:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.445+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(105 609.00000000 [create] v103'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.432 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:37:22.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.445+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929b5400 2024-04-25T11:37:22.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.450+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v103'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.450+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929b5400 2024-04-25T11:37:22.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.454+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v103'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.454+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929b5400 2024-04-25T11:37:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.458+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(108 609.00000000 [omap-set-header] v103'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9274019030 con 0x5608929b5400 2024-04-25T11:37:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.459+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.463+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v103'52 uv51 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.463+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.468+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v103'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.468+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.472+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v103'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.460 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:37:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.472+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.477+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v103'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.477+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.480+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v103'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.480+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.484+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v103'132 uv123 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.484+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.488+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v103'133 uv133 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.488+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.492+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v103'134 uv134 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:37:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.493+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.497+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v103'135 uv135 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.497+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.501+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v103'136 uv136 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.501+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.505+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v103'137 uv130 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.505+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.510+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v103'138 uv138 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.510+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.514+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v103'139 uv139 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.502 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:37:22.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.514+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.518+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v103'140 uv140 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.518+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.522+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v103'141 uv141 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.522+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c0c50 con 0x5608929bd4e0 2024-04-25T11:37:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.527+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v103'142 uv142 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.527+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x5608929c14f0 con 0x5608929bd4e0 2024-04-25T11:37:22.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f927a600640 1 -- 172.21.15.19:0/870735668 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v103'143 uv143 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9268006830 con 0x5608929bd4e0 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608929adc80 msgr2=0x5608929b00a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5608929adc80 0x5608929b00a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608929b97c0 msgr2=0x5608929bc620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5608929b97c0 0x5608929bc620 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608929bd4e0 msgr2=0x5608929bf900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5608929bd4e0 0x5608929bf900 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608929b1d10 msgr2=0x5608929b4130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5608929b1d10 0x5608929b4130 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560892863bc0 msgr2=0x5608929a30b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f927ba00640 1 -- 172.21.15.19:0/870735668 reap_dead start 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560892863bc0 0x5608929a30b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608929b5400 msgr2=0x5608929b7820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.531+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608929b5400 0x5608929b7820 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f925c072e90 msgr2=0x7f925c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f925c072e90 0x7f925c0753a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f926c056000 tx=0x7f926c007030 comp rx=0 tx=0).stop 2024-04-25T11:37:22.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56089290bdd0 msgr2=0x560892999de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56089290bdd0 0x560892999de0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f9274007e00 tx=0x7f9274006750 comp rx=0 tx=0).stop 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 shutdown_connections 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5608929b5400 0x5608929b7820 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560892863bc0 0x5608929a30b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f925c072e90 0x7f925c0753a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 --2- 172.21.15.19:0/870735668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56089290bdd0 0x560892999de0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 >> 172.21.15.19:0/870735668 conn(0x560892754ac0 msgr2=0x560892907d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.532+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 shutdown_connections 2024-04-25T11:37:22.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.533+0000 7f928291cf40 1 -- 172.21.15.19:0/870735668 wait complete. 2024-04-25T11:37:22.551 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2024-04-25T11:37:22.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.800+0000 7f3e4c159f40 1 Processor -- start 2024-04-25T11:37:22.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.800+0000 7f3e4c159f40 1 -- start start 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.801+0000 7f3e4c159f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b1e0 0x5626e3d616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.801+0000 7f3e4c159f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b5f0 0x5626e3d59c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.801+0000 7f3e4c159f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 0x5626e3d57e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.801+0000 7f3e4c159f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5626e3cd3e70 con 0x5626e3c8b1e0 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.801+0000 7f3e4c159f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5626e3c31d20 con 0x5626e3d64dd0 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.801+0000 7f3e4c159f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5626e3cd43f0 con 0x5626e3c8b5f0 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b1e0 0x5626e3d616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b1e0 0x5626e3d616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47368/0 (socket says 172.21.15.19:47368) 2024-04-25T11:37:22.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 -- 172.21.15.19:0/4136169237 learned_addr learned my addr 172.21.15.19:0/4136169237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e44800640 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b5f0 0x5626e3d59c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45c00640 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 0x5626e3d57e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 -- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b5f0 msgr2=0x5626e3d59c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b5f0 0x5626e3d59c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 -- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 msgr2=0x5626e3d57e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:22.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 0x5626e3d57e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.802+0000 7f3e45200640 1 -- 172.21.15.19:0/4136169237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e3d5a190 con 0x5626e3c8b1e0 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.803+0000 7f3e45200640 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b1e0 0x5626e3d616c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400a770 tx=0x7f3e3400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3144ec9b03006ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.803+0000 7f3e43400640 1 -- 172.21.15.19:0/4136169237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34018070 con 0x5626e3c8b1e0 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.803+0000 7f3e43400640 1 -- 172.21.15.19:0/4136169237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e34007600 con 0x5626e3c8b1e0 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.803+0000 7f3e43400640 1 -- 172.21.15.19:0/4136169237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e340079b0 con 0x5626e3c8b1e0 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 -- 172.21.15.19:0/4136169237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b1e0 msgr2=0x5626e3d616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b1e0 0x5626e3d616c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400a770 tx=0x7f3e3400a860 comp rx=0 tx=0).stop 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 -- 172.21.15.19:0/4136169237 shutdown_connections 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 0x5626e3d57e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b5f0 0x5626e3d59c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/4136169237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b1e0 0x5626e3d616c0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 -- 172.21.15.19:0/4136169237 >> 172.21.15.19:0/4136169237 conn(0x5626e3badac0 msgr2=0x5626e3d61fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 -- 172.21.15.19:0/4136169237 shutdown_connections 2024-04-25T11:37:22.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.804+0000 7f3e4c159f40 1 -- 172.21.15.19:0/4136169237 wait complete. 2024-04-25T11:37:22.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.805+0000 7f3e4c159f40 1 Processor -- start 2024-04-25T11:37:22.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.805+0000 7f3e4c159f40 1 -- start start 2024-04-25T11:37:22.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e4c159f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b1e0 0x5626e3ccd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e4c159f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b5f0 0x5626e3ccf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e4c159f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 0x5626e3ccfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e4c159f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5626e3d61c00 con 0x5626e3c8b5f0 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e4c159f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5626e3c8aac0 con 0x5626e3d64dd0 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e4c159f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5626e3c8afc0 con 0x5626e3c8b1e0 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e44800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b5f0 0x5626e3ccf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e44800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b5f0 0x5626e3ccf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47374/0 (socket says 172.21.15.19:47374) 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e44800640 1 -- 172.21.15.19:0/1951798302 learned_addr learned my addr 172.21.15.19:0/1951798302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e45200640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b1e0 0x5626e3ccd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.807+0000 7f3e45c00640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 0x5626e3ccfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.808+0000 7f3e44800640 1 -- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b1e0 msgr2=0x5626e3ccd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.808+0000 7f3e44800640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5626e3c8b1e0 0x5626e3ccd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.808+0000 7f3e44800640 1 -- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 msgr2=0x5626e3ccfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.808+0000 7f3e44800640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5626e3d64dd0 0x5626e3ccfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.808+0000 7f3e44800640 1 -- 172.21.15.19:0/1951798302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e3d5e8b0 con 0x5626e3c8b5f0 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.808+0000 7f3e44800640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b5f0 0x5626e3ccf560 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f3e30000c00 tx=0x7f3e30002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.809+0000 7f3e42000640 1 -- 172.21.15.19:0/1951798302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30063070 con 0x5626e3c8b5f0 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.809+0000 7f3e42000640 1 -- 172.21.15.19:0/1951798302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e30054bb0 con 0x5626e3c8b5f0 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.809+0000 7f3e42000640 1 -- 172.21.15.19:0/1951798302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30059040 con 0x5626e3c8b5f0 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.809+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5626e3cced60 con 0x5626e3c8b5f0 2024-04-25T11:37:22.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.809+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e3c89ec0 con 0x5626e3c8b5f0 2024-04-25T11:37:22.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.811+0000 7f3e42000640 1 -- 172.21.15.19:0/1951798302 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e30066070 con 0x5626e3c8b5f0 2024-04-25T11:37:22.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.812+0000 7f3e42000640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e24072e90 0x7f3e240753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.812+0000 7f3e42000640 1 -- 172.21.15.19:0/1951798302 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7f3e300d7ed0 con 0x5626e3c8b5f0 2024-04-25T11:37:22.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.813+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5626e3d57460 0x5626e3d5cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:22.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.813+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e103) -- 0x5626e3d5d2b0 con 0x5626e3d57460 2024-04-25T11:37:22.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.813+0000 7f3e45200640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e24072e90 0x7f3e240753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.813+0000 7f3e45c00640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5626e3d57460 0x5626e3d5cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:22.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.815+0000 7f3e45c00640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5626e3d57460 0x5626e3d5cd70 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.815+0000 7f3e45200640 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e24072e90 0x7f3e240753a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3e34000c00 tx=0x7f3e3400c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.818+0000 7f3e45c00640 1 -- 172.21.15.19:0/1951798302 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v103'53 uv53 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3e3c011040 con 0x5626e3d57460 2024-04-25T11:37:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5626e3d57460 msgr2=0x5626e3d5cd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5626e3d57460 0x5626e3d5cd70 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e24072e90 msgr2=0x7f3e240753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e24072e90 0x7f3e240753a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3e34000c00 tx=0x7f3e3400c000 comp rx=0 tx=0).stop 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b5f0 msgr2=0x5626e3ccf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 --2- 172.21.15.19:0/1951798302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5626e3c8b5f0 0x5626e3ccf560 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f3e30000c00 tx=0x7f3e30002fb0 comp rx=0 tx=0).stop 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e45c00640 1 -- 172.21.15.19:0/1951798302 reap_dead start 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 shutdown_connections 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 >> 172.21.15.19:0/1951798302 conn(0x5626e3badac0 msgr2=0x5626e3cd7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 shutdown_connections 2024-04-25T11:37:22.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:22.819+0000 7f3e4c159f40 1 -- 172.21.15.19:0/1951798302 wait complete. 2024-04-25T11:37:22.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:37:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.114+0000 7f5d87000640 1 Processor -- start 2024-04-25T11:37:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.114+0000 7f5d87000640 1 -- start start 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.116+0000 7f5d87000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d800b55c0 0x7f5d800b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.116+0000 7f5d87000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d800b5f10 con 0x7f5d800b2e40 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.116+0000 7f5d87000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d800d33a0 con 0x7f5d800b9100 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.116+0000 7f5d87000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5d800b9510 con 0x7f5d800b55c0 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.116+0000 7f5d85200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5d800b2e40 0x7f5d800d2c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35878/0 (socket says 172.21.15.19:35878) 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.116+0000 7f5d85200640 1 -- 172.21.15.19:0/3098270166 learned_addr learned my addr 172.21.15.19:0/3098270166 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.116+0000 7f5d86600640 1 --2- 172.21.15.19:0/3098270166 >> v2:172.21.15.169:3300/0 conn(0x7f5d800b55c0 0x7f5d800b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2249059973 0 0) 0x7f5d6c001110 con 0x7f5d800b2e40 2024-04-25T11:37:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d68003520 con 0x7f5d800b2e40 2024-04-25T11:37:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4120006155 0 0) 0x7f5d68003520 con 0x7f5d800b2e40 2024-04-25T11:37:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 >> v2:172.21.15.169:3300/0 conn(0x7f5d800b55c0 msgr2=0x7f5d800b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 --2- 172.21.15.19:0/3098270166 >> v2:172.21.15.169:3300/0 conn(0x7f5d800b55c0 0x7f5d800b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 >> v1:172.21.15.169:6789/0 conn(0x7f5d800b9100 legacy=0x7f5d800b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d800d1ed0 con 0x7f5d800b2e40 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f5d6c001840 con 0x7f5d800b2e40 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 >> v1:172.21.15.19:6789/0 conn(0x7f5d800b2e40 legacy=0x7f5d800d2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d680039b0 0x7f5d68003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d68004800 0x7f5d68004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68005250 0x7f5d6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d800b9510 con 0x7f5d68005250 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6800c1e0 con 0x7f5d680039b0 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.117+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6800c3e0 con 0x7f5d68004800 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d86600640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68005250 0x7f5d6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d85c00640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d680039b0 0x7f5d68003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d85200640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d68004800 0x7f5d68004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d86600640 1 -- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d68004800 msgr2=0x7f5d68004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d86600640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d68004800 0x7f5d68004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d86600640 1 -- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d680039b0 msgr2=0x7f5d68003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:23.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d86600640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d680039b0 0x7f5d68003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.118+0000 7f5d86600640 1 -- 172.21.15.19:0/3098270166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d68003520 con 0x7f5d68005250 2024-04-25T11:37:23.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.119+0000 7f5d86600640 1 -- 172.21.15.19:0/3098270166 reap_dead start 2024-04-25T11:37:23.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.119+0000 7f5d86600640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68005250 0x7f5d6800b9c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009f90 tx=0x7f5d68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1ccb2f324f8be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78009c10 con 0x7f5d68005250 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d78031440 con 0x7f5d68005250 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d7fe00640 1 -- 172.21.15.19:0/3098270166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78031810 con 0x7f5d68005250 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d87000640 1 -- 172.21.15.19:0/3098270166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68005250 msgr2=0x7f5d6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d87000640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68005250 0x7f5d6800b9c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009f90 tx=0x7f5d68005830 comp rx=0 tx=0).stop 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d87000640 1 -- 172.21.15.19:0/3098270166 shutdown_connections 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d87000640 1 --2- 172.21.15.19:0/3098270166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d68005250 0x7f5d6800b9c0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.120+0000 7f5d87000640 1 -- 172.21.15.19:0/3098270166 >> 172.21.15.19:0/3098270166 conn(0x7f5d8008d180 msgr2=0x7f5d8008d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.121+0000 7f5d87000640 1 -- 172.21.15.19:0/3098270166 shutdown_connections 2024-04-25T11:37:23.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.121+0000 7f5d87000640 1 -- 172.21.15.19:0/3098270166 wait complete. 2024-04-25T11:37:23.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.123+0000 7f5d87000640 1 Processor -- start 2024-04-25T11:37:23.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.123+0000 7f5d87000640 1 -- start start 2024-04-25T11:37:23.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d87000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d68005250 0x7f5d801f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d87000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800cd6d0 0x7f5d801f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d87000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8014ebc0 0x7f5d8014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d87000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d800b3250 con 0x7f5d800cd6d0 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d87000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d800b52a0 con 0x7f5d68005250 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d87000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8014fe40 con 0x7f5d8014ebc0 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d85200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800cd6d0 0x7f5d801f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d85200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800cd6d0 0x7f5d801f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47396/0 (socket says 172.21.15.19:47396) 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d85200640 1 -- 172.21.15.19:0/1489514524 learned_addr learned my addr 172.21.15.19:0/1489514524 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.125+0000 7f5d85c00640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d68005250 0x7f5d801f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d86600640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8014ebc0 0x7f5d8014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d85200640 1 -- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8014ebc0 msgr2=0x7f5d8014efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d85200640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8014ebc0 0x7f5d8014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d85200640 1 -- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d68005250 msgr2=0x7f5d801f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d85200640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d68005250 0x7f5d801f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d85200640 1 -- 172.21.15.19:0/1489514524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d80161760 con 0x7f5d800cd6d0 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d85200640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800cd6d0 0x7f5d801f61e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00e760 tx=0x7f5d6c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d7d600640 1 -- 172.21.15.19:0/1489514524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00c6c0 con 0x7f5d800cd6d0 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d7d600640 1 -- 172.21.15.19:0/1489514524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d6c018070 con 0x7f5d800cd6d0 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d7d600640 1 -- 172.21.15.19:0/1489514524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00caa0 con 0x7f5d800cd6d0 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d80161a40 con 0x7f5d800cd6d0 2024-04-25T11:37:23.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.126+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d80162000 con 0x7f5d800cd6d0 2024-04-25T11:37:23.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.128+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d80076c60 con 0x7f5d800cd6d0 2024-04-25T11:37:23.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.129+0000 7f5d7d600640 1 -- 172.21.15.19:0/1489514524 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d6c00cc40 con 0x7f5d800cd6d0 2024-04-25T11:37:23.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.130+0000 7f5d7d600640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5d600744e0 0x7f5d600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:23.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.130+0000 7f5d7d600640 1 -- 172.21.15.19:0/1489514524 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6655+0+0 (secure 0 0 0) 0x7f5d6c0950f0 con 0x7f5d800cd6d0 2024-04-25T11:37:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.131+0000 7f5d85c00640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5d600744e0 0x7f5d600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.132+0000 7f5d85c00640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5d600744e0 0x7f5d600769f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5d800b3570 tx=0x7f5d74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:23.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.135+0000 7f5d7d600640 1 -- 172.21.15.19:0/1489514524 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d800b5f10 con 0x7f5d800cd6d0 2024-04-25T11:37:23.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.388+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f5d800d3750 con 0x7f5d800cd6d0 2024-04-25T11:37:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.782+0000 7f5d7d600640 1 -- 172.21.15.19:0/1489514524 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v163) ==== 162+0+0 (secure 0 0 0) 0x7f5d6c05f4c0 con 0x7f5d800cd6d0 2024-04-25T11:37:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:37:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.787+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5d600744e0 msgr2=0x7f5d600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.787+0000 7f5d87000640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5d600744e0 0x7f5d600769f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5d800b3570 tx=0x7f5d74009040 comp rx=0 tx=0).stop 2024-04-25T11:37:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800cd6d0 msgr2=0x7f5d801f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800cd6d0 0x7f5d801f61e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00e760 tx=0x7f5d6c00ec20 comp rx=0 tx=0).stop 2024-04-25T11:37:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 shutdown_connections 2024-04-25T11:37:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5d600744e0 0x7f5d600769f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d8014ebc0 0x7f5d8014efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d800cd6d0 0x7f5d801f61e0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 --2- 172.21.15.19:0/1489514524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d68005250 0x7f5d801f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 >> 172.21.15.19:0/1489514524 conn(0x7f5d8008d180 msgr2=0x7f5d80072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 shutdown_connections 2024-04-25T11:37:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:23.788+0000 7f5d87000640 1 -- 172.21.15.19:0/1489514524 wait complete. 2024-04-25T11:37:23.788 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:23.801+0000 7f383b800640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-04-25T11:37:23.788 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:23.801+0000 7f383b800640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-04-25T11:37:23.789 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:23.802+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:37:23.789 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:23.802+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:37:23.800 INFO:tasks.ceph.mds.a.smithi019.stderr: -28> 2024-04-25T11:37:23.801+0000 7f383b800640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-04-25T11:37:23.800 INFO:tasks.ceph.mds.a.smithi019.stderr: -27> 2024-04-25T11:37:23.801+0000 7f383b800640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-04-25T11:37:23.815 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:23.932 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:37:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.100+0000 7fed93800640 1 Processor -- start 2024-04-25T11:37:24.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.100+0000 7fed93800640 1 -- start start 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.102+0000 7fed93800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed8c1434d0 0x7fed8c070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.102+0000 7fed93800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c0d0ed0 0x7fed8c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.102+0000 7fed93800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8c0709c0 con 0x7fed8c160be0 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.102+0000 7fed93800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fed8c1438e0 con 0x7fed8c1434d0 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.102+0000 7fed93800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c06f3e0 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed90e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed8c1434d0 0x7fed8c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed90e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed8c1434d0 0x7fed8c070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37610/0 (socket says 172.21.15.19:37610) 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed90e00640 1 -- 172.21.15.19:0/3964087214 learned_addr learned my addr 172.21.15.19:0/3964087214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed91800640 1 --1- 172.21.15.19:0/3964087214 >> v1:172.21.15.19:6789/0 conn(0x7fed8c160be0 0x7fed8c160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35890/0 (socket says 172.21.15.19:35890) 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed8be00640 1 --2- 172.21.15.19:0/3964087214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c0d0ed0 0x7fed8c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed90e00640 1 -- 172.21.15.19:0/3964087214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c0d0ed0 msgr2=0x7fed8c0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:24.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed90e00640 1 --2- 172.21.15.19:0/3964087214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c0d0ed0 0x7fed8c0d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed90e00640 1 -- 172.21.15.19:0/3964087214 >> v1:172.21.15.19:6789/0 conn(0x7fed8c160be0 legacy=0x7fed8c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed8b400640 1 -- 172.21.15.19:0/3964087214 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1600823875 0 0) 0x7fed8c0709c0 con 0x7fed8c160be0 2024-04-25T11:37:24.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.103+0000 7fed90e00640 1 -- 172.21.15.19:0/3964087214 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c06f6c0 con 0x7fed8c1434d0 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.104+0000 7fed90e00640 1 --2- 172.21.15.19:0/3964087214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c1434d0 0x7fed8c070480 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fed80009870 tx=0x7fed8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1e4b1a2b310727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.105+0000 7fed8b400640 1 -- 172.21.15.19:0/3964087214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80037070 con 0x7fed8c1434d0 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.105+0000 7fed8b400640 1 -- 172.21.15.19:0/3964087214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed80032440 con 0x7fed8c1434d0 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.105+0000 7fed8b400640 1 -- 172.21.15.19:0/3964087214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80032810 con 0x7fed8c1434d0 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.105+0000 7fed93800640 1 -- 172.21.15.19:0/3964087214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c1434d0 msgr2=0x7fed8c070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.105+0000 7fed93800640 1 --2- 172.21.15.19:0/3964087214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c1434d0 0x7fed8c070480 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fed80009870 tx=0x7fed8002a270 comp rx=0 tx=0).stop 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.106+0000 7fed93800640 1 -- 172.21.15.19:0/3964087214 shutdown_connections 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.106+0000 7fed93800640 1 --2- 172.21.15.19:0/3964087214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c0d0ed0 0x7fed8c0d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.106+0000 7fed93800640 1 --2- 172.21.15.19:0/3964087214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c1434d0 0x7fed8c070480 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.106+0000 7fed93800640 1 -- 172.21.15.19:0/3964087214 >> 172.21.15.19:0/3964087214 conn(0x7fed8c0d1ae0 msgr2=0x7fed8c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.106+0000 7fed93800640 1 -- 172.21.15.19:0/3964087214 shutdown_connections 2024-04-25T11:37:24.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.106+0000 7fed93800640 1 -- 172.21.15.19:0/3964087214 wait complete. 2024-04-25T11:37:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.108+0000 7fed93800640 1 Processor -- start 2024-04-25T11:37:24.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.108+0000 7fed93800640 1 -- start start 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed93800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c0d0ed0 0x7fed8c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed93800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c1434d0 0x7fed8c0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed93800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed8c160be0 0x7fed8c0aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed93800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c06f090 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed93800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c06fc80 con 0x7fed8c160be0 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed93800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c0b51a0 con 0x7fed8c1434d0 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c0d0ed0 0x7fed8c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c0d0ed0 0x7fed8c15a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37624/0 (socket says 172.21.15.19:37624) 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed90e00640 1 -- 172.21.15.19:0/69730921 learned_addr learned my addr 172.21.15.19:0/69730921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:24.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed8be00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c1434d0 0x7fed8c0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.110+0000 7fed91800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed8c160be0 0x7fed8c0aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed90e00640 1 -- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c1434d0 msgr2=0x7fed8c0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed90e00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c1434d0 0x7fed8c0ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed90e00640 1 -- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed8c160be0 msgr2=0x7fed8c0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed90e00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed8c160be0 0x7fed8c0aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed90e00640 1 -- 172.21.15.19:0/69730921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c146700 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed8be00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c1434d0 0x7fed8c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed91800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed8c160be0 0x7fed8c0aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed90e00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c0d0ed0 0x7fed8c15a9a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fed80009870 tx=0x7fed800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed88c00640 1 -- 172.21.15.19:0/69730921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80041020 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed88c00640 1 -- 172.21.15.19:0/69730921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed800306c0 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed88c00640 1 -- 172.21.15.19:0/69730921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80037040 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.111+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c135ee0 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.112+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c15aee0 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.113+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c050740 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.114+0000 7fed88c00640 1 -- 172.21.15.19:0/69730921 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed80044020 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.115+0000 7fed88c00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed6c0744e0 0x7fed6c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.115+0000 7fed88c00640 1 -- 172.21.15.19:0/69730921 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7fed800b5b70 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.117+0000 7fed8be00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed6c0744e0 0x7fed6c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.120+0000 7fed88c00640 1 -- 172.21.15.19:0/69730921 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed8004b070 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.120+0000 7fed8be00640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed6c0744e0 0x7fed6c0769f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fed78006fd0 tx=0x7fed78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:24.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.369+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed8c1428a0 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.371+0000 7fed88c00640 1 -- 172.21.15.19:0/69730921 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) ==== 95+0+4352 (secure 0 0 0) 0x7fed8007fe50 con 0x7fed8c0d0ed0 2024-04-25T11:37:24.358 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:24.358 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":165,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:23.802911+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:24.359 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 165 2024-04-25T11:37:24.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.373+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed6c0744e0 msgr2=0x7fed6c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed6c0744e0 0x7fed6c0769f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fed78006fd0 tx=0x7fed78008040 comp rx=0 tx=0).stop 2024-04-25T11:37:24.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c0d0ed0 msgr2=0x7fed8c15a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c0d0ed0 0x7fed8c15a9a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fed80009870 tx=0x7fed800046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 shutdown_connections 2024-04-25T11:37:24.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed6c0744e0 0x7fed6c0769f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed8c160be0 0x7fed8c0aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed8c1434d0 0x7fed8c0ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 --2- 172.21.15.19:0/69730921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed8c0d0ed0 0x7fed8c15a9a0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 >> 172.21.15.19:0/69730921 conn(0x7fed8c0d1ae0 msgr2=0x7fed8c0b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:24.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 shutdown_connections 2024-04-25T11:37:24.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.374+0000 7fed93800640 1 -- 172.21.15.19:0/69730921 wait complete. 2024-04-25T11:37:24.388 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:24.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.690+0000 7f262fe00640 1 Processor -- start 2024-04-25T11:37:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.690+0000 7f262fe00640 1 -- start start 2024-04-25T11:37:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.692+0000 7f262fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2630164860 0x7f26300b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.693+0000 7f262fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f26300d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.693+0000 7f262fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2630161650 con 0x7f26300b7540 2024-04-25T11:37:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.693+0000 7f262fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f26301643d0 con 0x7f2630164860 2024-04-25T11:37:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.693+0000 7f262fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26300b5270 con 0x7f26300d44a0 2024-04-25T11:37:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.693+0000 7f262ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2630164860 0x7f26300b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.693+0000 7f262e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f26300b7540 0x7f26300b7950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35894/0 (socket says 172.21.15.19:35894) 2024-04-25T11:37:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.693+0000 7f262e000640 1 -- 172.21.15.19:0/2855657409 learned_addr learned my addr 172.21.15.19:0/2855657409 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.694+0000 7f262f400640 1 --2- 172.21.15.19:0/2855657409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f26300d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.694+0000 7f262ea00640 1 -- 172.21.15.19:0/2855657409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 msgr2=0x7f26300d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.694+0000 7f262ea00640 1 --2- 172.21.15.19:0/2855657409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f26300d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.694+0000 7f262ea00640 1 -- 172.21.15.19:0/2855657409 >> v1:172.21.15.19:6789/0 conn(0x7f26300b7540 legacy=0x7f26300b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.694+0000 7f262d600640 1 -- 172.21.15.19:0/2855657409 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210679267 0 0) 0x7f2630161650 con 0x7f26300b7540 2024-04-25T11:37:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.694+0000 7f262ea00640 1 -- 172.21.15.19:0/2855657409 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2630161370 con 0x7f2630164860 2024-04-25T11:37:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.694+0000 7f262f400640 1 --2- 172.21.15.19:0/2855657409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f26300d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.695+0000 7f262ea00640 1 --2- 172.21.15.19:0/2855657409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630164860 0x7f26300b4d30 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f26200097a0 tx=0x7f262002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efda4e2a8b1e1ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.695+0000 7f262d600640 1 -- 172.21.15.19:0/2855657409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620037070 con 0x7f2630164860 2024-04-25T11:37:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.695+0000 7f262d600640 1 -- 172.21.15.19:0/2855657409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2620032440 con 0x7f2630164860 2024-04-25T11:37:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.695+0000 7f262d600640 1 -- 172.21.15.19:0/2855657409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620032810 con 0x7f2630164860 2024-04-25T11:37:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.696+0000 7f262fe00640 1 -- 172.21.15.19:0/2855657409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630164860 msgr2=0x7f26300b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.696+0000 7f262fe00640 1 --2- 172.21.15.19:0/2855657409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630164860 0x7f26300b4d30 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f26200097a0 tx=0x7f262002a180 comp rx=0 tx=0).stop 2024-04-25T11:37:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.696+0000 7f262fe00640 1 -- 172.21.15.19:0/2855657409 shutdown_connections 2024-04-25T11:37:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.696+0000 7f262fe00640 1 --2- 172.21.15.19:0/2855657409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f26300d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.696+0000 7f262fe00640 1 --2- 172.21.15.19:0/2855657409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2630164860 0x7f26300b4d30 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.696+0000 7f262fe00640 1 -- 172.21.15.19:0/2855657409 >> 172.21.15.19:0/2855657409 conn(0x7f26300d2720 msgr2=0x7f26300d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.696+0000 7f262fe00640 1 -- 172.21.15.19:0/2855657409 shutdown_connections 2024-04-25T11:37:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.697+0000 7f262fe00640 1 -- 172.21.15.19:0/2855657409 wait complete. 2024-04-25T11:37:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.699+0000 7f262fe00640 1 Processor -- start 2024-04-25T11:37:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.699+0000 7f262fe00640 1 -- start start 2024-04-25T11:37:24.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300b7540 0x7f263015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f2630155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26301eb3f0 0x7f263015e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26300d4df0 con 0x7f26300b7540 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2630164060 con 0x7f26301eb3f0 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26300d4f70 con 0x7f26300d44a0 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300b7540 0x7f263015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300b7540 0x7f263015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37644/0 (socket says 172.21.15.19:37644) 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 -- 172.21.15.19:0/2274512387 learned_addr learned my addr 172.21.15.19:0/2274512387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 -- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 msgr2=0x7f2630155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f2630155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 -- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26301eb3f0 msgr2=0x7f263015e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26301eb3f0 0x7f263015e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.701+0000 7f262ea00640 1 -- 172.21.15.19:0/2274512387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26301ebb40 con 0x7f26300b7540 2024-04-25T11:37:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.702+0000 7f262ea00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300b7540 0x7f263015b2b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f262002a6f0 tx=0x7f26200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.702+0000 7f2627400640 1 -- 172.21.15.19:0/2274512387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620032e60 con 0x7f26300b7540 2024-04-25T11:37:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.702+0000 7f2627400640 1 -- 172.21.15.19:0/2274512387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2620030770 con 0x7f26300b7540 2024-04-25T11:37:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.702+0000 7f2627400640 1 -- 172.21.15.19:0/2274512387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620037040 con 0x7f26300b7540 2024-04-25T11:37:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.702+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26301ea870 con 0x7f26300b7540 2024-04-25T11:37:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.702+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26301e9d90 con 0x7f26300b7540 2024-04-25T11:37:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.703+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2630050740 con 0x7f26300b7540 2024-04-25T11:37:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.705+0000 7f2627400640 1 -- 172.21.15.19:0/2274512387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2620044020 con 0x7f26300b7540 2024-04-25T11:37:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.706+0000 7f2627400640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f260c0744b0 0x7f260c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.706+0000 7f2627400640 1 -- 172.21.15.19:0/2274512387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f26200b5700 con 0x7f26300b7540 2024-04-25T11:37:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.707+0000 7f262e000640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f260c0744b0 0x7f260c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.708+0000 7f262e000640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f260c0744b0 0x7f260c0769c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2618007fd0 tx=0x7f2618007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.712+0000 7f2627400640 1 -- 172.21.15.19:0/2274512387 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f262007fa70 con 0x7f26300b7540 2024-04-25T11:37:24.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.957+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26300cfe80 con 0x7f26300b7540 2024-04-25T11:37:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.959+0000 7f2627400640 1 -- 172.21.15.19:0/2274512387 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+5178 (secure 0 0 0) 0x7f262007f370 con 0x7f26300b7540 2024-04-25T11:37:24.947 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:24.947 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":166,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2103355299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2103355299},{"type":"v1","addr":"172.21.15.19:6835","nonce":2103355299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:23.802911+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 166 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f260c0744b0 msgr2=0x7f260c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f260c0744b0 0x7f260c0769c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2618007fd0 tx=0x7f2618007960 comp rx=0 tx=0).stop 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300b7540 msgr2=0x7f263015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300b7540 0x7f263015b2b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f262002a6f0 tx=0x7f26200046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 shutdown_connections 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f260c0744b0 0x7f260c0769c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26301eb3f0 0x7f263015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26300d44a0 0x7f2630155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 --2- 172.21.15.19:0/2274512387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26300b7540 0x7f263015b2b0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:24.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.961+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 >> 172.21.15.19:0/2274512387 conn(0x7f26300d2720 msgr2=0x7f26301617d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:24.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.962+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 shutdown_connections 2024-04-25T11:37:24.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:24.962+0000 7f262fe00640 1 -- 172.21.15.19:0/2274512387 wait complete. 2024-04-25T11:37:24.974 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:24.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:25.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.269+0000 7f8316600640 1 Processor -- start 2024-04-25T11:37:25.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.269+0000 7f8316600640 1 -- start start 2024-04-25T11:37:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.271+0000 7f8316600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83100cf2f0 0x7f83100b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.271+0000 7f8316600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831015a810 0x7f83100b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.271+0000 7f8316600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83100b4e90 con 0x7f83101608d0 2024-04-25T11:37:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.271+0000 7f8316600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f831006f870 con 0x7f83100cf2f0 2024-04-25T11:37:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.271+0000 7f8316600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83100b74d0 con 0x7f831015a810 2024-04-25T11:37:25.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.272+0000 7f830fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831015a810 0x7f83100b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.272+0000 7f830fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831015a810 0x7f83100b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:45388/0 (socket says 172.21.15.19:45388) 2024-04-25T11:37:25.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.272+0000 7f830fe00640 1 -- 172.21.15.19:0/3338444105 learned_addr learned my addr 172.21.15.19:0/3338444105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:25.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.272+0000 7f8315200640 1 --2- 172.21.15.19:0/3338444105 >> v2:172.21.15.169:3300/0 conn(0x7f83100cf2f0 0x7f83100b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.273+0000 7f830fe00640 1 -- 172.21.15.19:0/3338444105 >> v2:172.21.15.169:3300/0 conn(0x7f83100cf2f0 msgr2=0x7f83100b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:25.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.273+0000 7f830fe00640 1 --2- 172.21.15.19:0/3338444105 >> v2:172.21.15.169:3300/0 conn(0x7f83100cf2f0 0x7f83100b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.273+0000 7f830fe00640 1 -- 172.21.15.19:0/3338444105 >> v1:172.21.15.169:6789/0 conn(0x7f83101608d0 legacy=0x7f8310160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.273+0000 7f830fe00640 1 -- 172.21.15.19:0/3338444105 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83100b77b0 con 0x7f831015a810 2024-04-25T11:37:25.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.274+0000 7f830fe00640 1 --2- 172.21.15.19:0/3338444105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831015a810 0x7f83100b4950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009870 tx=0x7f82f802c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8d510b7b5301d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.275+0000 7f830f400640 1 -- 172.21.15.19:0/3338444105 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8038070 con 0x7f831015a810 2024-04-25T11:37:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.275+0000 7f830f400640 1 -- 172.21.15.19:0/3338444105 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f8031710 con 0x7f831015a810 2024-04-25T11:37:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.275+0000 7f830f400640 1 -- 172.21.15.19:0/3338444105 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8031ae0 con 0x7f831015a810 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.275+0000 7f8316600640 1 -- 172.21.15.19:0/3338444105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831015a810 msgr2=0x7f83100b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.275+0000 7f8316600640 1 --2- 172.21.15.19:0/3338444105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831015a810 0x7f83100b4950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009870 tx=0x7f82f802c460 comp rx=0 tx=0).stop 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.276+0000 7f8316600640 1 -- 172.21.15.19:0/3338444105 shutdown_connections 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.276+0000 7f8316600640 1 --2- 172.21.15.19:0/3338444105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f831015a810 0x7f83100b4950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.276+0000 7f8316600640 1 --2- 172.21.15.19:0/3338444105 >> v2:172.21.15.169:3300/0 conn(0x7f83100cf2f0 0x7f83100b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.276+0000 7f8316600640 1 -- 172.21.15.19:0/3338444105 >> 172.21.15.19:0/3338444105 conn(0x7f83100d1720 msgr2=0x7f83100d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.276+0000 7f8316600640 1 -- 172.21.15.19:0/3338444105 shutdown_connections 2024-04-25T11:37:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.276+0000 7f8316600640 1 -- 172.21.15.19:0/3338444105 wait complete. 2024-04-25T11:37:25.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.278+0000 7f8316600640 1 Processor -- start 2024-04-25T11:37:25.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.278+0000 7f8316600640 1 -- start start 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f8316600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83100cf2f0 0x7f831014a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f8316600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f831015a810 0x7f831014ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f8316600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83101608d0 0x7f8310160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f8316600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f831015ac20 con 0x7f831015a810 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f8316600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f831006f500 con 0x7f83101608d0 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f8316600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f831006fd00 con 0x7f83100cf2f0 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f830fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f831015a810 0x7f831014ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f830fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f831015a810 0x7f831014ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37658/0 (socket says 172.21.15.19:37658) 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.280+0000 7f830fe00640 1 -- 172.21.15.19:0/691099690 learned_addr learned my addr 172.21.15.19:0/691099690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:25.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f8315200640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83100cf2f0 0x7f831014a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f830fe00640 1 -- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83100cf2f0 msgr2=0x7f831014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f830fe00640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83100cf2f0 0x7f831014a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f830fe00640 1 -- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83101608d0 msgr2=0x7f8310160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f8315c00640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83101608d0 0x7f8310160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f830fe00640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83101608d0 0x7f8310160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f830fe00640 1 -- 172.21.15.19:0/691099690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831014b280 con 0x7f831015a810 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.281+0000 7f830fe00640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f831015a810 0x7f831014ad40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f82f802c4a0 tx=0x7f82f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.282+0000 7f830cc00640 1 -- 172.21.15.19:0/691099690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8038040 con 0x7f831015a810 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.282+0000 7f830cc00640 1 -- 172.21.15.19:0/691099690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f8033830 con 0x7f831015a810 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.282+0000 7f830cc00640 1 -- 172.21.15.19:0/691099690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8033c20 con 0x7f831015a810 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.282+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83101e6c40 con 0x7f831015a810 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.282+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101e7200 con 0x7f831015a810 2024-04-25T11:37:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.283+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8310050740 con 0x7f831015a810 2024-04-25T11:37:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.285+0000 7f830cc00640 1 -- 172.21.15.19:0/691099690 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82f8051430 con 0x7f831015a810 2024-04-25T11:37:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.286+0000 7f830cc00640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82ec0744b0 0x7f82ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.286+0000 7f830cc00640 1 -- 172.21.15.19:0/691099690 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f82f80b6b50 con 0x7f831015a810 2024-04-25T11:37:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.286+0000 7f8315200640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82ec0744b0 0x7f82ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.288+0000 7f8315200640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82ec0744b0 0x7f82ec0769c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8300009a30 tx=0x7f8300008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.290+0000 7f830cc00640 1 -- 172.21.15.19:0/691099690 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82f8081160 con 0x7f831015a810 2024-04-25T11:37:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.533+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f831006c770 con 0x7f831015a810 2024-04-25T11:37:25.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.534+0000 7f830cc00640 1 -- 172.21.15.19:0/691099690 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+5178 (secure 0 0 0) 0x7f82f8080a60 con 0x7f831015a810 2024-04-25T11:37:25.522 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:25.522 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":166,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8859,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2103355299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2103355299},{"type":"v1","addr":"172.21.15.19:6835","nonce":2103355299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:23.802911+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:25.522 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 166 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82ec0744b0 msgr2=0x7f82ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82ec0744b0 0x7f82ec0769c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8300009a30 tx=0x7f8300008040 comp rx=0 tx=0).stop 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f831015a810 msgr2=0x7f831014ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f831015a810 0x7f831014ad40 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f82f802c4a0 tx=0x7f82f80046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 shutdown_connections 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82ec0744b0 0x7f82ec0769c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83101608d0 0x7f8310160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f831015a810 0x7f831014ad40 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 --2- 172.21.15.19:0/691099690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83100cf2f0 0x7f831014a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.537+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 >> 172.21.15.19:0/691099690 conn(0x7f83100d1720 msgr2=0x7f8310072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.538+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 shutdown_connections 2024-04-25T11:37:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:25.538+0000 7f8316600640 1 -- 172.21.15.19:0/691099690 wait complete. 2024-04-25T11:37:25.551 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2024-04-25T11:37:25.552 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2024-04-25T11:37:25.552 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:25.552 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:25.600 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:37:25.600 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:37:25.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:25.600 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:37:25.695 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:37:25.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:37:25.697 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:37:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.000+0000 7fbf01200640 1 Processor -- start 2024-04-25T11:37:25.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.000+0000 7fbf01200640 1 -- start start 2024-04-25T11:37:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbf01200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbefc06ecb0 0x7fbefc0d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbf01200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbefc0b7550 0x7fbefc0b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbf01200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbefc0b7ea0 con 0x7fbefc163fc0 2024-04-25T11:37:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbf01200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc0d1f40 con 0x7fbefc0b7550 2024-04-25T11:37:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbf01200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc0d0430 con 0x7fbefc06ecb0 2024-04-25T11:37:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbef2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbefc0b7550 0x7fbefc0b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbef2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbefc0b7550 0x7fbefc0b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37674/0 (socket says 172.21.15.19:37674) 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbefbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbefc163fc0 0x7fbefc1643d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35910/0 (socket says 172.21.15.19:35910) 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.003+0000 7fbef2a00640 1 -- 172.21.15.19:0/770361213 learned_addr learned my addr 172.21.15.19:0/770361213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.004+0000 7fbefb400640 1 --2- 172.21.15.19:0/770361213 >> v2:172.21.15.169:3300/0 conn(0x7fbefc06ecb0 0x7fbefc0d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.004+0000 7fbef2a00640 1 -- 172.21.15.19:0/770361213 >> v2:172.21.15.169:3300/0 conn(0x7fbefc06ecb0 msgr2=0x7fbefc0d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.004+0000 7fbef2a00640 1 --2- 172.21.15.19:0/770361213 >> v2:172.21.15.169:3300/0 conn(0x7fbefc06ecb0 0x7fbefc0d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.004+0000 7fbef2a00640 1 -- 172.21.15.19:0/770361213 >> v1:172.21.15.19:6789/0 conn(0x7fbefc163fc0 legacy=0x7fbefc1643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.004+0000 7fbefaa00640 1 -- 172.21.15.19:0/770361213 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274992144 0 0) 0x7fbef4001490 con 0x7fbefc163fc0 2024-04-25T11:37:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.004+0000 7fbef2a00640 1 -- 172.21.15.19:0/770361213 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc0d21e0 con 0x7fbefc0b7550 2024-04-25T11:37:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.005+0000 7fbef2a00640 1 --2- 172.21.15.19:0/770361213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc0b7960 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fbefc06f3e0 tx=0x7fbee402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14db19ff25a357c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.005+0000 7fbefaa00640 1 -- 172.21.15.19:0/770361213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4037070 con 0x7fbefc0b7550 2024-04-25T11:37:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.005+0000 7fbefaa00640 1 -- 172.21.15.19:0/770361213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee4032440 con 0x7fbefc0b7550 2024-04-25T11:37:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.005+0000 7fbefaa00640 1 -- 172.21.15.19:0/770361213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4032810 con 0x7fbefc0b7550 2024-04-25T11:37:25.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 -- 172.21.15.19:0/770361213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 msgr2=0x7fbefc0b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 --2- 172.21.15.19:0/770361213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc0b7960 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fbefc06f3e0 tx=0x7fbee402a270 comp rx=0 tx=0).stop 2024-04-25T11:37:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 -- 172.21.15.19:0/770361213 shutdown_connections 2024-04-25T11:37:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 --2- 172.21.15.19:0/770361213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc0b7960 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 --2- 172.21.15.19:0/770361213 >> v2:172.21.15.169:3300/0 conn(0x7fbefc06ecb0 0x7fbefc0d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 -- 172.21.15.19:0/770361213 >> 172.21.15.19:0/770361213 conn(0x7fbefc09b7d0 msgr2=0x7fbefc09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 -- 172.21.15.19:0/770361213 shutdown_connections 2024-04-25T11:37:25.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.006+0000 7fbf01200640 1 -- 172.21.15.19:0/770361213 wait complete. 2024-04-25T11:37:25.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.009+0000 7fbf01200640 1 Processor -- start 2024-04-25T11:37:25.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.009+0000 7fbf01200640 1 -- start start 2024-04-25T11:37:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.010+0000 7fbf01200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbefc06ecb0 0x7fbefc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbf01200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbf01200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbefc163fc0 0x7fbefc1627d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbf01200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc0d24c0 con 0x7fbefc0b7550 2024-04-25T11:37:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbf01200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc0d09b0 con 0x7fbefc06ecb0 2024-04-25T11:37:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbf01200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc163bb0 con 0x7fbefc163fc0 2024-04-25T11:37:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc162290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37684/0 (socket says 172.21.15.19:37684) 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 -- 172.21.15.19:0/716236430 learned_addr learned my addr 172.21.15.19:0/716236430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbefb400640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbefc06ecb0 0x7fbefc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbefbe00640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbefc163fc0 0x7fbefc1627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 -- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbefc163fc0 msgr2=0x7fbefc1627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbefc163fc0 0x7fbefc1627d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 -- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbefc06ecb0 msgr2=0x7fbefc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbefc06ecb0 0x7fbefc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbef2a00640 1 -- 172.21.15.19:0/716236430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc0c86e0 con 0x7fbefc0b7550 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.011+0000 7fbefbe00640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbefc163fc0 0x7fbefc1627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.012+0000 7fbef2a00640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc162290 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fbee402a2b0 tx=0x7fbee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.012+0000 7fbef3e00640 1 -- 172.21.15.19:0/716236430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4041020 con 0x7fbefc0b7550 2024-04-25T11:37:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.012+0000 7fbef3e00640 1 -- 172.21.15.19:0/716236430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee40306c0 con 0x7fbefc0b7550 2024-04-25T11:37:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.012+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc0c89c0 con 0x7fbefc0b7550 2024-04-25T11:37:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.012+0000 7fbef3e00640 1 -- 172.21.15.19:0/716236430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4037040 con 0x7fbefc0b7550 2024-04-25T11:37:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.012+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc0c8f80 con 0x7fbefc0b7550 2024-04-25T11:37:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.014+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc050740 con 0x7fbefc0b7550 2024-04-25T11:37:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.015+0000 7fbef3e00640 1 -- 172.21.15.19:0/716236430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbee4044020 con 0x7fbefc0b7550 2024-04-25T11:37:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.016+0000 7fbef3e00640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbed80744e0 0x7fbed80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.016+0000 7fbef3e00640 1 -- 172.21.15.19:0/716236430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7fbee40b5f70 con 0x7fbefc0b7550 2024-04-25T11:37:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.017+0000 7fbefb400640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbed80744e0 0x7fbed80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.018+0000 7fbefb400640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbed80744e0 0x7fbed80769f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbefc073a30 tx=0x7fbeec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.021+0000 7fbef3e00640 1 -- 172.21.15.19:0/716236430 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbee404b030 con 0x7fbefc0b7550 2024-04-25T11:37:26.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.271+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbefc0cfb50 con 0x7fbefc0b7550 2024-04-25T11:37:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.834+0000 7fbef3e00640 1 -- 172.21.15.19:0/716236430 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v167) ==== 161+0+0 (secure 0 0 0) 0x7fbee4080250 con 0x7fbefc0b7550 2024-04-25T11:37:26.822 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbed80744e0 msgr2=0x7fbed80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbed80744e0 0x7fbed80769f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbefc073a30 tx=0x7fbeec008040 comp rx=0 tx=0).stop 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 msgr2=0x7fbefc162290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc162290 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fbee402a2b0 tx=0x7fbee40046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 shutdown_connections 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbed80744e0 0x7fbed80769f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbefc163fc0 0x7fbefc1627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:26.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbefc0b7550 0x7fbefc162290 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:26.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 --2- 172.21.15.19:0/716236430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbefc06ecb0 0x7fbefc15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:26.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.838+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 >> 172.21.15.19:0/716236430 conn(0x7fbefc09b7d0 msgr2=0x7fbefc070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:26.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.839+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 shutdown_connections 2024-04-25T11:37:26.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:26.839+0000 7fbf01200640 1 -- 172.21.15.19:0/716236430 wait complete. 2024-04-25T11:37:26.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:37:27.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.149+0000 7fe0dfe00640 1 Processor -- start 2024-04-25T11:37:27.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.149+0000 7fe0dfe00640 1 -- start start 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.151+0000 7fe0dfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe0e006c5c0 0x7fe0e00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.151+0000 7fe0dfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 0x7fe0e00b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.151+0000 7fe0dfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe0e00b4cd0 0x7fe0e0161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.151+0000 7fe0dfe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e00b50e0 con 0x7fe0e006c5c0 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.151+0000 7fe0dfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e00d1230 con 0x7fe0e00b4cd0 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.151+0000 7fe0dfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e00d06b0 con 0x7fe0e0144370 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.151+0000 7fe0dea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe0e006c5c0 0x7fe0e00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:27.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.152+0000 7fe0dea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe0e006c5c0 0x7fe0e00cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37690/0 (socket says 172.21.15.19:37690) 2024-04-25T11:37:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.152+0000 7fe0dea00640 1 -- 172.21.15.19:0/3925465171 learned_addr learned my addr 172.21.15.19:0/3925465171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.152+0000 7fe0dea00640 1 -- 172.21.15.19:0/3925465171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 msgr2=0x7fe0e00b4790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:37:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.152+0000 7fe0dea00640 1 --2- 172.21.15.19:0/3925465171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 0x7fe0e00b4790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.152+0000 7fe0dea00640 1 -- 172.21.15.19:0/3925465171 >> v2:172.21.15.169:3300/0 conn(0x7fe0e00b4cd0 msgr2=0x7fe0e0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.152+0000 7fe0dea00640 1 --2- 172.21.15.19:0/3925465171 >> v2:172.21.15.169:3300/0 conn(0x7fe0e00b4cd0 0x7fe0e0161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.152+0000 7fe0dea00640 1 -- 172.21.15.19:0/3925465171 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e00d0990 con 0x7fe0e006c5c0 2024-04-25T11:37:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.153+0000 7fe0dea00640 1 --2- 172.21.15.19:0/3925465171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cfa70 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00b6cd0 tx=0x7fe0d002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c3d3da08072a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:27.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.153+0000 7fe0dd600640 1 -- 172.21.15.19:0/3925465171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0036070 con 0x7fe0e006c5c0 2024-04-25T11:37:27.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.153+0000 7fe0dd600640 1 -- 172.21.15.19:0/3925465171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d0031600 con 0x7fe0e006c5c0 2024-04-25T11:37:27.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.153+0000 7fe0dd600640 1 -- 172.21.15.19:0/3925465171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00319d0 con 0x7fe0e006c5c0 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 -- 172.21.15.19:0/3925465171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 msgr2=0x7fe0e00cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/3925465171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cfa70 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe0e00b6cd0 tx=0x7fe0d002a6c0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 -- 172.21.15.19:0/3925465171 shutdown_connections 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/3925465171 >> v2:172.21.15.169:3300/0 conn(0x7fe0e00b4cd0 0x7fe0e0161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/3925465171 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 0x7fe0e00b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/3925465171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cfa70 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 -- 172.21.15.19:0/3925465171 >> 172.21.15.19:0/3925465171 conn(0x7fe0e00d1720 msgr2=0x7fe0e00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 -- 172.21.15.19:0/3925465171 shutdown_connections 2024-04-25T11:37:27.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.154+0000 7fe0dfe00640 1 -- 172.21.15.19:0/3925465171 wait complete. 2024-04-25T11:37:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.156+0000 7fe0dfe00640 1 Processor -- start 2024-04-25T11:37:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.157+0000 7fe0dfe00640 1 -- start start 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e00b4cd0 0x7fe0e01f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 0x7fe0e01f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dfe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e00d02d0 con 0x7fe0e006c5c0 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e00d0f50 con 0x7fe0e00b4cd0 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e00d0450 con 0x7fe0e0144370 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37692/0 (socket says 172.21.15.19:37692) 2024-04-25T11:37:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.158+0000 7fe0dea00640 1 -- 172.21.15.19:0/1865071863 learned_addr learned my addr 172.21.15.19:0/1865071863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:27.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0dea00640 1 -- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 msgr2=0x7fe0e01f5e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:37:27.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0dea00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 0x7fe0e01f5e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0dea00640 1 -- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e00b4cd0 msgr2=0x7fe0e01f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:27.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0dea00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e00b4cd0 0x7fe0e01f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0dea00640 1 -- 172.21.15.19:0/1865071863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e013f750 con 0x7fe0e006c5c0 2024-04-25T11:37:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0dea00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cd170 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00093e0 tx=0x7fe0d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0d6a00640 1 -- 172.21.15.19:0/1865071863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0041020 con 0x7fe0e006c5c0 2024-04-25T11:37:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0d6a00640 1 -- 172.21.15.19:0/1865071863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d0007820 con 0x7fe0e006c5c0 2024-04-25T11:37:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.159+0000 7fe0d6a00640 1 -- 172.21.15.19:0/1865071863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0036040 con 0x7fe0e006c5c0 2024-04-25T11:37:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.160+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e0070470 con 0x7fe0e006c5c0 2024-04-25T11:37:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.160+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e0070a30 con 0x7fe0e006c5c0 2024-04-25T11:37:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.161+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e0070c70 con 0x7fe0e006c5c0 2024-04-25T11:37:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.162+0000 7fe0d6a00640 1 -- 172.21.15.19:0/1865071863 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe0d0044020 con 0x7fe0e006c5c0 2024-04-25T11:37:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.163+0000 7fe0d6a00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c00744b0 0x7fe0c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.163+0000 7fe0d6a00640 1 -- 172.21.15.19:0/1865071863 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7fe0d00b6480 con 0x7fe0e006c5c0 2024-04-25T11:37:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.165+0000 7fe0de000640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c00744b0 0x7fe0c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.168+0000 7fe0d6a00640 1 -- 172.21.15.19:0/1865071863 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe0e0070c70 con 0x7fe0e006c5c0 2024-04-25T11:37:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.170+0000 7fe0de000640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c00744b0 0x7fe0c00769c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe0e013f300 tx=0x7fe0cc006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:27.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.441+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe0e0050740 con 0x7fe0e006c5c0 2024-04-25T11:37:27.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.837+0000 7fe0d6a00640 1 -- 172.21.15.19:0/1865071863 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v168) ==== 101+0+0 (secure 0 0 0) 0x7fe0d004b030 con 0x7fe0e006c5c0 2024-04-25T11:37:27.825 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:37:27.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.841+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c00744b0 msgr2=0x7fe0c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:27.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.841+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c00744b0 0x7fe0c00769c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe0e013f300 tx=0x7fe0cc006ba0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.841+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 msgr2=0x7fe0e00cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:27.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.841+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cd170 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00093e0 tx=0x7fe0d0004a40 comp rx=0 tx=0).stop 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 shutdown_connections 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe0c00744b0 0x7fe0c00769c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0e00b4cd0 0x7fe0e01f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0e0144370 0x7fe0e01f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 --2- 172.21.15.19:0/1865071863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe0e006c5c0 0x7fe0e00cd170 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 >> 172.21.15.19:0/1865071863 conn(0x7fe0e00d1720 msgr2=0x7fe0e00b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 shutdown_connections 2024-04-25T11:37:27.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:27.842+0000 7fe0dfe00640 1 -- 172.21.15.19:0/1865071863 wait complete. 2024-04-25T11:37:27.863 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:37:28.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.064+0000 7f4f07cb1f40 1 Processor -- start 2024-04-25T11:37:28.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.065+0000 7f4f07cb1f40 1 -- start start 2024-04-25T11:37:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f07cb1f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc64ac0 0x55af2fd35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f07cb1f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc8f0a0 0x55af2fc8f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f07cb1f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f9f0 0x55af2fdc6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f07cb1f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af2fc04d20 con 0x55af2fc8f9f0 2024-04-25T11:37:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f07cb1f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55af2fc08d60 con 0x55af2fc64ac0 2024-04-25T11:37:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f07cb1f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55af2fca7110 con 0x55af2fc8f0a0 2024-04-25T11:37:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f01800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f9f0 0x55af2fdc6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f01800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f9f0 0x55af2fdc6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37712/0 (socket says 172.21.15.19:37712) 2024-04-25T11:37:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.066+0000 7f4f01800640 1 -- 172.21.15.19:0/1869344639 learned_addr learned my addr 172.21.15.19:0/1869344639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:28.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4f01800640 1 -- 172.21.15.19:0/1869344639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc8f0a0 msgr2=0x55af2fc8f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:28.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4f01800640 1 --2- 172.21.15.19:0/1869344639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc8f0a0 0x55af2fc8f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4f01800640 1 -- 172.21.15.19:0/1869344639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc64ac0 msgr2=0x55af2fd35560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:28.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4f01800640 1 --2- 172.21.15.19:0/1869344639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc64ac0 0x55af2fd35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4f01800640 1 -- 172.21.15.19:0/1869344639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2fd36d30 con 0x55af2fc8f9f0 2024-04-25T11:37:28.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4f01800640 1 --2- 172.21.15.19:0/1869344639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f9f0 0x55af2fdc6cf0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8025750 tx=0x7f4ef804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4869cdf812fcc290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4eff000640 1 -- 172.21.15.19:0/1869344639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8005d70 con 0x55af2fc8f9f0 2024-04-25T11:37:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4eff000640 1 -- 172.21.15.19:0/1869344639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef80080d0 con 0x55af2fc8f9f0 2024-04-25T11:37:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.067+0000 7f4eff000640 1 -- 172.21.15.19:0/1869344639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8005650 con 0x55af2fc8f9f0 2024-04-25T11:37:28.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1869344639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f9f0 msgr2=0x55af2fdc6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1869344639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f9f0 0x55af2fdc6cf0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8025750 tx=0x7f4ef804e000 comp rx=0 tx=0).stop 2024-04-25T11:37:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1869344639 shutdown_connections 2024-04-25T11:37:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1869344639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f9f0 0x55af2fdc6cf0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1869344639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc8f0a0 0x55af2fc8f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1869344639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc64ac0 0x55af2fd35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1869344639 >> 172.21.15.19:0/1869344639 conn(0x55af2fb80ac0 msgr2=0x55af2fc5cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1869344639 shutdown_connections 2024-04-25T11:37:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.068+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1869344639 wait complete. 2024-04-25T11:37:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.069+0000 7f4f07cb1f40 1 Processor -- start 2024-04-25T11:37:28.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.069+0000 7f4f07cb1f40 1 -- start start 2024-04-25T11:37:28.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f07cb1f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc64ac0 0x55af2fd31190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f07cb1f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f0a0 0x55af2fdce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f07cb1f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc8f9f0 0x55af2fdce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f07cb1f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af2fd365d0 con 0x55af2fc8f0a0 2024-04-25T11:37:28.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f07cb1f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55af2fd368b0 con 0x55af2fc8f9f0 2024-04-25T11:37:28.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f07cb1f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55af2fc5d740 con 0x55af2fc64ac0 2024-04-25T11:37:28.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f00400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f0a0 0x55af2fdce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f00400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f0a0 0x55af2fdce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37722/0 (socket says 172.21.15.19:37722) 2024-04-25T11:37:28.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.071+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 learned_addr learned my addr 172.21.15.19:0/1639832253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:28.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f01800640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc8f9f0 0x55af2fdce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f00e00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc64ac0 0x55af2fd31190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc64ac0 msgr2=0x55af2fd31190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f00400640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55af2fc64ac0 0x55af2fd31190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc8f9f0 msgr2=0x55af2fdce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f00400640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55af2fc8f9f0 0x55af2fdce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af2fdcd3f0 con 0x55af2fc8f0a0 2024-04-25T11:37:28.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f00400640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f0a0 0x55af2fdce490 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8001c60 tx=0x7f4ee800a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4efdc00640 1 -- 172.21.15.19:0/1639832253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8012070 con 0x55af2fc8f0a0 2024-04-25T11:37:28.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4efdc00640 1 -- 172.21.15.19:0/1639832253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee800e640 con 0x55af2fc8f0a0 2024-04-25T11:37:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4efdc00640 1 -- 172.21.15.19:0/1639832253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800ea30 con 0x55af2fc8f0a0 2024-04-25T11:37:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55af2fdcd6d0 con 0x55af2fc8f0a0 2024-04-25T11:37:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.072+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af2fdcdc90 con 0x55af2fc8f0a0 2024-04-25T11:37:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.075+0000 7f4efdc00640 1 -- 172.21.15.19:0/1639832253 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ee8009070 con 0x55af2fc8f0a0 2024-04-25T11:37:28.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.076+0000 7f4efdc00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4edc072e90 0x7f4edc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.076+0000 7f4efdc00640 1 -- 172.21.15.19:0/1639832253 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f4ee8094a00 con 0x55af2fc8f0a0 2024-04-25T11:37:28.071 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:37:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.076+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55af2fdd5ca0 0x55af2fdd80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.076+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd8600 con 0x55af2fdd5ca0 2024-04-25T11:37:28.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.077+0000 7f4f00e00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4edc072e90 0x7f4edc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.077+0000 7f4f01800640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55af2fdd5ca0 0x55af2fdd80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.078+0000 7f4f00e00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4edc072e90 0x7f4edc0753a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00035d0 tx=0x7f4ef0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.078+0000 7f4f01800640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55af2fdd5ca0 0x55af2fdd80c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.082+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v104'350 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.082+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdd89a0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.086+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v104'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.086+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd8d40 con 0x55af2fdd5ca0 2024-04-25T11:37:28.076 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:37:28.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.089+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v104'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.089+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdda0f0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.093+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.093+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdda490 con 0x55af2fdd5ca0 2024-04-25T11:37:28.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.096+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.096+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd90e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.099+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'355 uv355 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.099+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd94d0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.103+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v104'356 uv330 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.103+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdd98c0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.107+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v104'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.107+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd98c0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.098 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:37:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.111+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v104'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.111+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.115+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.115+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.118+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.118+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.122+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.122+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.126+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v104'362 uv337 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.126+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.130+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'363 uv363 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.130+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.122 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:37:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.135+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v104'364 uv364 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.135+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.139+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'365 uv365 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.139+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.143+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'366 uv366 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.143+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.148+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'367 uv367 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.148+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.152+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'368 uv368 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.152+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.157+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v104'369 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.157+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.161+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v104'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.161+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.153 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:37:28.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.165+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v104'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.165+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.169+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.169+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.173+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.174+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.178+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.178+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.182+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v104'375 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.182+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.186+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v104'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.187+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:37:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.190+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v104'377 uv377 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.191+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.195+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.195+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.199+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.199+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd5ca0 2024-04-25T11:37:28.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.203+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.204+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55af2fdd52e0 0x55af2fddcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.204+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdd9870 con 0x55af2fdd52e0 2024-04-25T11:37:28.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.205+0000 7f4f00e00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55af2fdd52e0 0x55af2fddcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.206+0000 7f4f00e00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55af2fdd52e0 0x55af2fddcd50 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.211+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'262 uv245 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.211+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fddd290 con 0x55af2fdd52e0 2024-04-25T11:37:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.215+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'263 uv263 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.216+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fddd630 con 0x55af2fdd52e0 2024-04-25T11:37:28.207 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:37:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.220+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'264 uv264 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.220+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fddd9d0 con 0x55af2fdd52e0 2024-04-25T11:37:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.224+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v104'265 uv265 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.225+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdddd70 con 0x55af2fdd52e0 2024-04-25T11:37:28.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.228+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'266 uv266 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.229+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde110 con 0x55af2fdd52e0 2024-04-25T11:37:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.233+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v104'267 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.233+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdde4b0 con 0x55af2fdd52e0 2024-04-25T11:37:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.237+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v104'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.237+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.228 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:37:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.241+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v104'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.241+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.245+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v104'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.245+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.250+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v104'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.250+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.254+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v104'272 uv255 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.255+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.259+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v104'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.259+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.251 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:37:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.263+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v104'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.263+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde2f40 con 0x55af2fdd52e0 2024-04-25T11:37:28.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.269+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v104'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.269+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde2f40 con 0x55af2fdd52e0 2024-04-25T11:37:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.273+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v104'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.273+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.277+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v104'277 uv261 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.277+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.282+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v104'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.282+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.273 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:37:28.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.286+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v104'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.286+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fddf860 con 0x55af2fdd52e0 2024-04-25T11:37:28.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.291+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v104'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.291+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fddf860 con 0x55af2fdd52e0 2024-04-25T11:37:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.295+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v104'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.295+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdde850 con 0x55af2fdd52e0 2024-04-25T11:37:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.300+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v104'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fdd52e0 2024-04-25T11:37:28.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.300+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55af2fdded60 0x55af2fde1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.300+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde16c0 con 0x55af2fdded60 2024-04-25T11:37:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.301+0000 7f4f00400640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55af2fdded60 0x55af2fde1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.304+0000 7f4f00400640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55af2fdded60 0x55af2fde1180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.308+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.308+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fde1ae0 con 0x55af2fdded60 2024-04-25T11:37:28.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.313+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v104'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.313+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde1fc0 con 0x55af2fdded60 2024-04-25T11:37:28.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.317+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v104'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.304 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:37:28.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.317+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde3870 con 0x55af2fdded60 2024-04-25T11:37:28.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.321+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.321+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde3c10 con 0x55af2fdded60 2024-04-25T11:37:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.325+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.325+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde2860 con 0x55af2fdded60 2024-04-25T11:37:28.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.329+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.330+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde2c50 con 0x55af2fdd5ca0 2024-04-25T11:37:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.333+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'56 uv55 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.334+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fde2c50 con 0x55af2fdd5ca0 2024-04-25T11:37:28.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.337+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v104'57 uv57 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.338+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde2c50 con 0x55af2fdd5ca0 2024-04-25T11:37:28.329 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:37:28.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.341+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v104'58 uv58 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.342+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde2c50 con 0x55af2fdd5ca0 2024-04-25T11:37:28.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.346+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v104'59 uv59 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.346+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55af2fde3fb0 0x55af2fde63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.346+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde6910 con 0x55af2fde3fb0 2024-04-25T11:37:28.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.347+0000 7f4f01800640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55af2fde3fb0 0x55af2fde63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.349+0000 7f4f01800640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55af2fde3fb0 0x55af2fde63d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.352+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.352+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fde6910 con 0x55af2fde3fb0 2024-04-25T11:37:28.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.356+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v104'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.356+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde6910 con 0x55af2fde3fb0 2024-04-25T11:37:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.361+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v104'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.348 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:37:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.361+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde7920 con 0x55af2fde3fb0 2024-04-25T11:37:28.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.365+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.365+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde7920 con 0x55af2fde3fb0 2024-04-25T11:37:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.369+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.369+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde6910 con 0x55af2fde3fb0 2024-04-25T11:37:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.373+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.374+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55af2fde6e20 0x55af2fde9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.374+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde9900 con 0x55af2fde6e20 2024-04-25T11:37:28.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.374+0000 7f4f00e00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55af2fde6e20 0x55af2fde9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.376+0000 7f4f00e00640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55af2fde6e20 0x55af2fde9240 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.380+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v104'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.380+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.384+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v104'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.384+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.387+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v104'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.375 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:37:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.388+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdeace0 con 0x55af2fde6e20 2024-04-25T11:37:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.392+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v104'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.392+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdeace0 con 0x55af2fde6e20 2024-04-25T11:37:28.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.396+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v104'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.396+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.400+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v104'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.400+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.404+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v104'133 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.404+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.408+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v104'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.408+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.412+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v104'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.400 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:37:28.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.412+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.417+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v104'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.417+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fde9cd0 con 0x55af2fde6e20 2024-04-25T11:37:28.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.421+0000 7f4f00e00640 1 -- 172.21.15.19:0/1639832253 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v104'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef00014a0 con 0x55af2fde6e20 2024-04-25T11:37:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.421+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55af2fdea350 0x55af2fdec770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.421+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdece70 con 0x55af2fdea350 2024-04-25T11:37:28.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.422+0000 7f4f00400640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55af2fdea350 0x55af2fdec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.424+0000 7f4f00400640 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55af2fdea350 0x55af2fdec770 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.428+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v104'56 uv55 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.428+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fdece70 con 0x55af2fdea350 2024-04-25T11:37:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.432+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v104'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.432+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded290 con 0x55af2fdea350 2024-04-25T11:37:28.423 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:37:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.436+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v104'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.436+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded290 con 0x55af2fdea350 2024-04-25T11:37:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.440+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v104'59 uv59 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.440+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded290 con 0x55af2fdea350 2024-04-25T11:37:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.444+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v104'60 uv60 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.445+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded290 con 0x55af2fdd5ca0 2024-04-25T11:37:28.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.449+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.449+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fded290 con 0x55af2fdd5ca0 2024-04-25T11:37:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.454+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v104'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.454+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded290 con 0x55af2fdd5ca0 2024-04-25T11:37:28.445 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:37:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.458+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v104'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.458+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdee690 con 0x55af2fdd5ca0 2024-04-25T11:37:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.463+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.463+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdee690 con 0x55af2fdd5ca0 2024-04-25T11:37:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.467+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.467+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded3b0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.471+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fdd5ca0 2024-04-25T11:37:28.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.471+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded330 con 0x55af2fdded60 2024-04-25T11:37:28.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.476+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v104'112 uv108 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.476+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fded330 con 0x55af2fdded60 2024-04-25T11:37:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.480+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v104'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.480+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded330 con 0x55af2fdded60 2024-04-25T11:37:28.472 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:37:28.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.484+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v104'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.484+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded3d0 con 0x55af2fdded60 2024-04-25T11:37:28.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.489+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v104'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.489+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded3d0 con 0x55af2fdded60 2024-04-25T11:37:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.493+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v104'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.493+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded3d0 con 0x55af2fdded60 2024-04-25T11:37:28.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.497+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v104'117 uv111 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.497+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fded3d0 con 0x55af2fdded60 2024-04-25T11:37:28.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.501+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v104'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.501+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded3d0 con 0x55af2fdded60 2024-04-25T11:37:28.492 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:37:28.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.505+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v104'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.505+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded3d0 con 0x55af2fdded60 2024-04-25T11:37:28.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.509+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v104'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.509+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdf01e0 con 0x55af2fdded60 2024-04-25T11:37:28.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.514+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v104'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdded60 2024-04-25T11:37:28.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.514+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.505 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:37:28.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.518+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v104'54 uv54 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.518+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.522+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v104'55 uv55 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.522+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.526+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 609.00000000 [create] v104'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.526+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.530+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 609.00000000 [delete] v104'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.530+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.521 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:37:28.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.534+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v104'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.534+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.539+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v104'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.539+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.543+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v104'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.543+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fde3fb0 2024-04-25T11:37:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.546+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [omap-set-header] v104'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ef80065e0 con 0x55af2fde3fb0 2024-04-25T11:37:28.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.546+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:109 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.550+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(109 400.00000000 [create] v104'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.551+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:110 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.555+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(110 400.00000000 [delete] v104'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.555+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.546 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:37:28.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.559+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(111 400.00000000 [create] v104'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.559+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.563+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(112 400.00000000 [write 0~22] v104'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.564+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.567+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(113 400.00000000 [omap-set-header] v104'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.567+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.571+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(114 100.00000000.inode [create] v104'144 uv136 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.571+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.575+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(115 100.00000000.inode [delete] v104'145 uv145 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.575+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.566 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:37:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.579+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(116 100.00000000.inode [create] v104'146 uv146 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.579+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.583+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v104'147 uv147 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.584+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.588+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v104'148 uv148 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.588+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.592+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(119 1.00000000 [create] v104'149 uv143 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.592+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.596+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(120 1.00000000 [delete] v104'150 uv150 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.596+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.587 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:37:28.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.600+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(121 1.00000000 [create] v104'151 uv151 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.600+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdee890 con 0x55af2fdea350 2024-04-25T11:37:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.604+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v104'152 uv152 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.604+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdee890 con 0x55af2fdea350 2024-04-25T11:37:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.608+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v104'153 uv153 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.608+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fded880 con 0x55af2fdea350 2024-04-25T11:37:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.612+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(124 1.00000000 [omap-set-header] v104'154 uv154 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.613+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55af2fdedb30 con 0x55af2fdea350 2024-04-25T11:37:28.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f00400640 1 -- 172.21.15.19:0/1639832253 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v104'155 uv155 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ee801f070 con 0x55af2fdea350 2024-04-25T11:37:28.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55af2fdd52e0 msgr2=0x55af2fddcd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55af2fdd52e0 0x55af2fddcd50 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55af2fde6e20 msgr2=0x55af2fde9240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55af2fde6e20 0x55af2fde9240 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55af2fdea350 msgr2=0x55af2fdec770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55af2fdea350 0x55af2fdec770 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55af2fdded60 msgr2=0x55af2fde1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55af2fdded60 0x55af2fde1180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55af2fdd5ca0 msgr2=0x55af2fdd80c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f01800640 1 -- 172.21.15.19:0/1639832253 reap_dead start 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55af2fdd5ca0 0x55af2fdd80c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55af2fde3fb0 msgr2=0x55af2fde63d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.617+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55af2fde3fb0 0x55af2fde63d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4edc072e90 msgr2=0x7f4edc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4edc072e90 0x7f4edc0753a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00035d0 tx=0x7f4ef0008040 comp rx=0 tx=0).stop 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f0a0 msgr2=0x55af2fdce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f0a0 0x55af2fdce490 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8001c60 tx=0x7f4ee800a6b0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 shutdown_connections 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55af2fde3fb0 0x55af2fde63d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55af2fdd5ca0 0x55af2fdd80c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4edc072e90 0x7f4edc0753a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 --2- 172.21.15.19:0/1639832253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af2fc8f0a0 0x55af2fdce490 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 >> 172.21.15.19:0/1639832253 conn(0x55af2fb80ac0 msgr2=0x55af2fc8fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.618+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 shutdown_connections 2024-04-25T11:37:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.619+0000 7f4f07cb1f40 1 -- 172.21.15.19:0/1639832253 wait complete. 2024-04-25T11:37:28.643 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2024-04-25T11:37:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.892+0000 7f517a51cf40 1 Processor -- start 2024-04-25T11:37:28.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.892+0000 7f517a51cf40 1 -- start start 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f517a51cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51580022d0 0x55c8b3af5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f517a51cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3a73230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f517a51cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8b3a582e0 0x55c8b3a586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f517a51cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c8b39cdd20 con 0x55c8b3af9500 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f517a51cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c8b39d1d60 con 0x7f51580022d0 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f517a51cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c8b3a703f0 con 0x55c8b3a582e0 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f5172400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3a73230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f5172400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3a73230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37746/0 (socket says 172.21.15.19:37746) 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f5172400640 1 -- 172.21.15.19:0/1919119937 learned_addr learned my addr 172.21.15.19:0/1919119937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.894+0000 7f5173800640 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8b3a582e0 0x55c8b3a586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5172e00640 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51580022d0 0x55c8b3af5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5172400640 1 -- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8b3a582e0 msgr2=0x55c8b3a586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5172400640 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8b3a582e0 0x55c8b3a586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5172400640 1 -- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51580022d0 msgr2=0x55c8b3af5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5172400640 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51580022d0 0x55c8b3af5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5172400640 1 -- 172.21.15.19:0/1919119937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b3a58c30 con 0x55c8b3af9500 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5173800640 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8b3a582e0 0x55c8b3a586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:28.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5172400640 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3a73230 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f5160001e30 tx=0x7f516000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dadf1bfb732e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5171000640 1 -- 172.21.15.19:0/1919119937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516000be20 con 0x55c8b3af9500 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5171000640 1 -- 172.21.15.19:0/1919119937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516000f400 con 0x55c8b3af9500 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.895+0000 7f5171000640 1 -- 172.21.15.19:0/1919119937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516000f7d0 con 0x55c8b3af9500 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 -- 172.21.15.19:0/1919119937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 msgr2=0x55c8b3a73230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3a73230 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f5160001e30 tx=0x7f516000b670 comp rx=0 tx=0).stop 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 -- 172.21.15.19:0/1919119937 shutdown_connections 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8b3a582e0 0x55c8b3a586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3a73230 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 --2- 172.21.15.19:0/1919119937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f51580022d0 0x55c8b3af5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 -- 172.21.15.19:0/1919119937 >> 172.21.15.19:0/1919119937 conn(0x55c8b3949ac0 msgr2=0x55c8b3a57900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 -- 172.21.15.19:0/1919119937 shutdown_connections 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.896+0000 7f517a51cf40 1 -- 172.21.15.19:0/1919119937 wait complete. 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.897+0000 7f517a51cf40 1 Processor -- start 2024-04-25T11:37:28.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.897+0000 7f517a51cf40 1 -- start start 2024-04-25T11:37:28.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f517a51cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8b3a582e0 0x55c8b3af1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f517a51cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3af4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f517a51cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51580022d0 0x55c8b3af4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f517a51cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c8b3aff090 con 0x55c8b3af9500 2024-04-25T11:37:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f517a51cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c8b3a56ef0 con 0x55c8b3a582e0 2024-04-25T11:37:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f517a51cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c8b3a571d0 con 0x7f51580022d0 2024-04-25T11:37:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f5172400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3af4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f5172400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3af4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37752/0 (socket says 172.21.15.19:37752) 2024-04-25T11:37:28.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.899+0000 7f5172400640 1 -- 172.21.15.19:0/3279956945 learned_addr learned my addr 172.21.15.19:0/3279956945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5173800640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51580022d0 0x55c8b3af4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5172e00640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8b3a582e0 0x55c8b3af1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5172400640 1 -- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51580022d0 msgr2=0x55c8b3af4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5172400640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f51580022d0 0x55c8b3af4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5172400640 1 -- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8b3a582e0 msgr2=0x55c8b3af1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5172400640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8b3a582e0 0x55c8b3af1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5172400640 1 -- 172.21.15.19:0/3279956945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b3a594a0 con 0x55c8b3af9500 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f5172400640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3af4680 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f516000bb70 tx=0x7f5160001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f516b400640 1 -- 172.21.15.19:0/3279956945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160019680 con 0x55c8b3af9500 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f516b400640 1 -- 172.21.15.19:0/3279956945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5160019820 con 0x55c8b3af9500 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f516b400640 1 -- 172.21.15.19:0/3279956945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160019bf0 con 0x55c8b3af9500 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.900+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8b3af7210 con 0x55c8b3af9500 2024-04-25T11:37:28.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.901+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b3af2420 con 0x55c8b3af9500 2024-04-25T11:37:28.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.903+0000 7f516b400640 1 -- 172.21.15.19:0/3279956945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5160017040 con 0x55c8b3af9500 2024-04-25T11:37:28.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.904+0000 7f516b400640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5154072e90 0x7f51540753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.904+0000 7f516b400640 1 -- 172.21.15.19:0/3279956945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f5160096110 con 0x55c8b3af9500 2024-04-25T11:37:28.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.904+0000 7f517a51cf40 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c8b3b950e0 0x55c8b3af5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.904+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104) -- 0x55c8b3a586f0 con 0x55c8b3b950e0 2024-04-25T11:37:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.905+0000 7f5172e00640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5154072e90 0x7f51540753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.905+0000 7f5173800640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c8b3b950e0 0x55c8b3af5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:28.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.905+0000 7f5173800640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c8b3b950e0 0x55c8b3af5fb0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.906+0000 7f5172e00640 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5154072e90 0x7f51540753a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5164009810 tx=0x7f516405a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f5173800640 1 -- 172.21.15.19:0/3279956945 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 400.00000000 [delete] v104'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f516c00d1c0 con 0x55c8b3b950e0 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c8b3b950e0 msgr2=0x55c8b3af5fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f517a51cf40 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c8b3b950e0 0x55c8b3af5fb0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5154072e90 msgr2=0x7f51540753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f517a51cf40 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5154072e90 0x7f51540753a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5164009810 tx=0x7f516405a040 comp rx=0 tx=0).stop 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 msgr2=0x55c8b3af4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f517a51cf40 1 --2- 172.21.15.19:0/3279956945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8b3af9500 0x55c8b3af4680 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f516000bb70 tx=0x7f5160001190 comp rx=0 tx=0).stop 2024-04-25T11:37:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.907+0000 7f5173800640 1 -- 172.21.15.19:0/3279956945 reap_dead start 2024-04-25T11:37:28.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.908+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 shutdown_connections 2024-04-25T11:37:28.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.908+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 >> 172.21.15.19:0/3279956945 conn(0x55c8b3949ac0 msgr2=0x55c8b3af27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:28.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.908+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 shutdown_connections 2024-04-25T11:37:28.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:28.908+0000 7f517a51cf40 1 -- 172.21.15.19:0/3279956945 wait complete. 2024-04-25T11:37:28.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:37:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.204+0000 7f3e69a00640 1 Processor -- start 2024-04-25T11:37:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.204+0000 7f3e69a00640 1 -- start start 2024-04-25T11:37:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.207+0000 7f3e69a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e6415ee60 0x7f3e64168e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.207+0000 7f3e69a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e64070000 con 0x7f3e64144340 2024-04-25T11:37:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.207+0000 7f3e69a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e6406f240 con 0x7f3e6406dce0 2024-04-25T11:37:29.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.207+0000 7f3e69a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6415f270 con 0x7f3e6415ee60 2024-04-25T11:37:29.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.207+0000 7f3e63e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3e64144340 0x7f3e6406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35934/0 (socket says 172.21.15.19:35934) 2024-04-25T11:37:29.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.207+0000 7f3e63e00640 1 -- 172.21.15.19:0/359567819 learned_addr learned my addr 172.21.15.19:0/359567819 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e69000640 1 --2- 172.21.15.19:0/359567819 >> v2:172.21.15.169:3300/0 conn(0x7f3e6415ee60 0x7f3e64168e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179675088 0 0) 0x7f3e64070000 con 0x7f3e64144340 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e48003520 con 0x7f3e64144340 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 305830351 0 0) 0x7f3e48003520 con 0x7f3e64144340 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 >> v2:172.21.15.169:3300/0 conn(0x7f3e6415ee60 msgr2=0x7f3e64168e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 --2- 172.21.15.19:0/359567819 >> v2:172.21.15.169:3300/0 conn(0x7f3e6415ee60 0x7f3e64168e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 >> v1:172.21.15.169:6789/0 conn(0x7f3e6406dce0 legacy=0x7f3e6415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6406f520 con 0x7f3e64144340 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f3e54001670 con 0x7f3e64144340 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 >> v1:172.21.15.19:6789/0 conn(0x7f3e64144340 legacy=0x7f3e6406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e480039b0 0x7f3e48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e48004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e48005270 0x7f3e4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4800c1d0 con 0x7f3e48004800 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4800c380 con 0x7f3e480039b0 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.208+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4800c580 con 0x7f3e48005270 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e5be00640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e48004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e63e00640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e480039b0 0x7f3e48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e69000640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e48005270 0x7f3e4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e5be00640 1 -- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e48005270 msgr2=0x7f3e4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e5be00640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e48005270 0x7f3e4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e5be00640 1 -- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e480039b0 msgr2=0x7f3e48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e5be00640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e480039b0 0x7f3e48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e5be00640 1 -- 172.21.15.19:0/359567819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e48003520 con 0x7f3e48004800 2024-04-25T11:37:29.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.209+0000 7f3e69000640 1 -- 172.21.15.19:0/359567819 reap_dead start 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.210+0000 7f3e5be00640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e48004c10 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c001720 tx=0x7f3e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c2c83a62c52ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.211+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c001370 con 0x7f3e48004800 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.211+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4c0052d0 con 0x7f3e48004800 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.211+0000 7f3e63400640 1 -- 172.21.15.19:0/359567819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c02ab90 con 0x7f3e48004800 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.211+0000 7f3e69a00640 1 -- 172.21.15.19:0/359567819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 msgr2=0x7f3e48004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.211+0000 7f3e69a00640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e48004c10 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c001720 tx=0x7f3e48005830 comp rx=0 tx=0).stop 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.212+0000 7f3e69a00640 1 -- 172.21.15.19:0/359567819 shutdown_connections 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.212+0000 7f3e69a00640 1 --2- 172.21.15.19:0/359567819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e48004c10 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.212+0000 7f3e69a00640 1 -- 172.21.15.19:0/359567819 >> 172.21.15.19:0/359567819 conn(0x7f3e64072f40 msgr2=0x7f3e64073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.212+0000 7f3e69a00640 1 -- 172.21.15.19:0/359567819 shutdown_connections 2024-04-25T11:37:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.212+0000 7f3e69a00640 1 -- 172.21.15.19:0/359567819 wait complete. 2024-04-25T11:37:29.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.214+0000 7f3e69a00640 1 Processor -- start 2024-04-25T11:37:29.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.215+0000 7f3e69a00640 1 -- start start 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e69a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e641448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e69a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e64144e10 0x7f3e64133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e69a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e64133dc0 0x7f3e641341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e69a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6406fae0 con 0x7f3e48004800 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e69a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e64144750 con 0x7f3e64144e10 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e69a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6415e750 con 0x7f3e64133dc0 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e641448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e641448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37778/0 (socket says 172.21.15.19:37778) 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.216+0000 7f3e63e00640 1 -- 172.21.15.19:0/4232379041 learned_addr learned my addr 172.21.15.19:0/4232379041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.217+0000 7f3e63e00640 1 -- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e64133dc0 msgr2=0x7f3e641341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.217+0000 7f3e63e00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e64133dc0 0x7f3e641341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.217+0000 7f3e63e00640 1 -- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e64144e10 msgr2=0x7f3e64133880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.217+0000 7f3e63e00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e64144e10 0x7f3e64133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.217+0000 7f3e63e00640 1 -- 172.21.15.19:0/4232379041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e64145220 con 0x7f3e48004800 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.217+0000 7f3e63e00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e641448d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400e730 tx=0x7f3e5400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.217+0000 7f3e60c00640 1 -- 172.21.15.19:0/4232379041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400c6c0 con 0x7f3e48004800 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.218+0000 7f3e60c00640 1 -- 172.21.15.19:0/4232379041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e54018070 con 0x7f3e48004800 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.218+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e64134ad0 con 0x7f3e48004800 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.218+0000 7f3e60c00640 1 -- 172.21.15.19:0/4232379041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400caa0 con 0x7f3e48004800 2024-04-25T11:37:29.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.218+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e640b4b10 con 0x7f3e48004800 2024-04-25T11:37:29.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.219+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e64134ff0 con 0x7f3e48004800 2024-04-25T11:37:29.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.220+0000 7f3e60c00640 1 -- 172.21.15.19:0/4232379041 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e5401d460 con 0x7f3e48004800 2024-04-25T11:37:29.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.221+0000 7f3e60c00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e400744e0 0x7f3e400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:29.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.221+0000 7f3e60c00640 1 -- 172.21.15.19:0/4232379041 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f3e54095690 con 0x7f3e48004800 2024-04-25T11:37:29.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.222+0000 7f3e5be00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e400744e0 0x7f3e400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:29.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.223+0000 7f3e5be00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e400744e0 0x7f3e400769f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c032000 tx=0x7f3e4c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:29.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.226+0000 7f3e60c00640 1 -- 172.21.15.19:0/4232379041 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e64134ff0 con 0x7f3e48004800 2024-04-25T11:37:29.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.482+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f3e64076c60 con 0x7f3e48004800 2024-04-25T11:37:29.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.865+0000 7f3e60c00640 1 -- 172.21.15.19:0/4232379041 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v169) ==== 162+0+0 (secure 0 0 0) 0x7f3e64145220 con 0x7f3e48004800 2024-04-25T11:37:29.853 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:37:29.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e400744e0 msgr2=0x7f3e400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:29.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e400744e0 0x7f3e400769f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c032000 tx=0x7f3e4c02c000 comp rx=0 tx=0).stop 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 msgr2=0x7f3e641448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e641448d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400e730 tx=0x7f3e5400ebf0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 shutdown_connections 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e400744e0 0x7f3e400769f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e64144e10 0x7f3e64133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e64133dc0 0x7f3e641341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 --2- 172.21.15.19:0/4232379041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e48004800 0x7f3e641448d0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 >> 172.21.15.19:0/4232379041 conn(0x7f3e64072f40 msgr2=0x7f3e640b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 shutdown_connections 2024-04-25T11:37:29.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:29.869+0000 7f3e69a00640 1 -- 172.21.15.19:0/4232379041 wait complete. 2024-04-25T11:37:30.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:30.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.603+0000 7f0ea2e00640 1 Processor -- start 2024-04-25T11:37:30.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.603+0000 7f0ea2e00640 1 -- start start 2024-04-25T11:37:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0ea2e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e9c070680 0x7f0e9c1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0ea2e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e9c0b5c90 0x7f0e9c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0ea2e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9c0b65e0 con 0x7f0e9c160360 2024-04-25T11:37:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0ea2e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c06d100 con 0x7f0e9c070680 2024-04-25T11:37:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0ea2e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c06c7d0 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0e9be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e9c070680 0x7f0e9c1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0e9be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0e9c070680 0x7f0e9c1647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37786/0 (socket says 172.21.15.19:37786) 2024-04-25T11:37:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.606+0000 7f0e9be00640 1 -- 172.21.15.19:0/1964570446 learned_addr learned my addr 172.21.15.19:0/1964570446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.607+0000 7f0ea0e00640 1 --2- 172.21.15.19:0/1964570446 >> v2:172.21.15.169:3300/0 conn(0x7f0e9c0b5c90 0x7f0e9c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.607+0000 7f0e9be00640 1 -- 172.21.15.19:0/1964570446 >> v2:172.21.15.169:3300/0 conn(0x7f0e9c0b5c90 msgr2=0x7f0e9c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.607+0000 7f0e9be00640 1 --2- 172.21.15.19:0/1964570446 >> v2:172.21.15.169:3300/0 conn(0x7f0e9c0b5c90 0x7f0e9c0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.607+0000 7f0e9be00640 1 -- 172.21.15.19:0/1964570446 >> v1:172.21.15.19:6789/0 conn(0x7f0e9c160360 legacy=0x7f0e9c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.607+0000 7f0e9be00640 1 -- 172.21.15.19:0/1964570446 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c06cab0 con 0x7f0e9c070680 2024-04-25T11:37:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.607+0000 7f0ea0e00640 1 --2- 172.21.15.19:0/1964570446 >> v2:172.21.15.169:3300/0 conn(0x7f0e9c0b5c90 0x7f0e9c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.607+0000 7f0e9b400640 1 -- 172.21.15.19:0/1964570446 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811897003 0 0) 0x7f0e84001110 con 0x7f0e9c160360 2024-04-25T11:37:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.608+0000 7f0e9be00640 1 --2- 172.21.15.19:0/1964570446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c070680 0x7f0e9c1647f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0b11b0 tx=0x7f0e8c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75e743476c7d1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.608+0000 7f0e9b400640 1 -- 172.21.15.19:0/1964570446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c037070 con 0x7f0e9c070680 2024-04-25T11:37:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.608+0000 7f0e9b400640 1 -- 172.21.15.19:0/1964570446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e8c032440 con 0x7f0e9c070680 2024-04-25T11:37:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.608+0000 7f0e9b400640 1 -- 172.21.15.19:0/1964570446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c032810 con 0x7f0e9c070680 2024-04-25T11:37:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1964570446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c070680 msgr2=0x7f0e9c1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1964570446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c070680 0x7f0e9c1647f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0b11b0 tx=0x7f0e8c02a160 comp rx=0 tx=0).stop 2024-04-25T11:37:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1964570446 shutdown_connections 2024-04-25T11:37:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1964570446 >> v2:172.21.15.169:3300/0 conn(0x7f0e9c0b5c90 0x7f0e9c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1964570446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c070680 0x7f0e9c1647f0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1964570446 >> 172.21.15.19:0/1964570446 conn(0x7f0e9c0d1ae0 msgr2=0x7f0e9c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1964570446 shutdown_connections 2024-04-25T11:37:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.609+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1964570446 wait complete. 2024-04-25T11:37:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.612+0000 7f0ea2e00640 1 Processor -- start 2024-04-25T11:37:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.612+0000 7f0ea2e00640 1 -- start start 2024-04-25T11:37:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.613+0000 7f0ea2e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9c070680 0x7f0e9c148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.613+0000 7f0ea2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c0b5c90 0x7f0e9c144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.613+0000 7f0ea2e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e9c160360 0x7f0e9c145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.613+0000 7f0ea2e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c0afbb0 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.613+0000 7f0ea2e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c06fcc0 con 0x7f0e9c070680 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.613+0000 7f0ea2e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c0afd30 con 0x7f0e9c160360 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c0b5c90 0x7f0e9c144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c0b5c90 0x7f0e9c144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37802/0 (socket says 172.21.15.19:37802) 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 -- 172.21.15.19:0/1328672752 learned_addr learned my addr 172.21.15.19:0/1328672752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 -- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e9c160360 msgr2=0x7f0e9c145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0ea0e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e9c160360 0x7f0e9c145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e9c160360 0x7f0e9c145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e9be00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9c070680 0x7f0e9c148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 -- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9c070680 msgr2=0x7f0e9c148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9c070680 0x7f0e9c148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0e93400640 1 -- 172.21.15.19:0/1328672752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c1eb6d0 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.614+0000 7f0ea0e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e9c160360 0x7f0e9c145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.615+0000 7f0e93400640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c0b5c90 0x7f0e9c144be0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400c440 tx=0x7f0e8400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:30.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.615+0000 7f0e98c00640 1 -- 172.21.15.19:0/1328672752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84017070 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.615+0000 7f0e98c00640 1 -- 172.21.15.19:0/1328672752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e840035f0 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.615+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c142da0 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.615+0000 7f0e98c00640 1 -- 172.21.15.19:0/1328672752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e840039e0 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.615+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c144900 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.616+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9c1eb440 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.618+0000 7f0e98c00640 1 -- 172.21.15.19:0/1328672752 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e8401a070 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.619+0000 7f0e98c00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e780744e0 0x7f0e780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:30.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.619+0000 7f0e98c00640 1 -- 172.21.15.19:0/1328672752 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f0e84094940 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.619+0000 7f0e9be00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e780744e0 0x7f0e780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:30.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.621+0000 7f0e9be00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e780744e0 0x7f0e780769f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c034000 tx=0x7f0e8c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.624+0000 7f0e98c00640 1 -- 172.21.15.19:0/1328672752 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e9c0b65e0 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.868+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e9c050740 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.870+0000 7f0e98c00640 1 -- 172.21.15.19:0/1328672752 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) ==== 95+0+5187 (secure 0 0 0) 0x7f0e8405ee10 con 0x7f0e9c0b5c90 2024-04-25T11:37:30.857 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:30.857 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":170,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:29.871989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8859":{"gid":8859,"name":"a","rank":0,"incarnation":170,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/2103355299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2103355299},{"type":"v1","addr":"172.21.15.19:6835","nonce":2103355299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:30.858 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 170 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e780744e0 msgr2=0x7f0e780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e780744e0 0x7f0e780769f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c034000 tx=0x7f0e8c02e000 comp rx=0 tx=0).stop 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c0b5c90 msgr2=0x7f0e9c144be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c0b5c90 0x7f0e9c144be0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400c440 tx=0x7f0e8400c900 comp rx=0 tx=0).stop 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 shutdown_connections 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e780744e0 0x7f0e780769f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e9c160360 0x7f0e9c145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e9c0b5c90 0x7f0e9c144be0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 --2- 172.21.15.19:0/1328672752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e9c070680 0x7f0e9c148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 >> 172.21.15.19:0/1328672752 conn(0x7f0e9c0d1ae0 msgr2=0x7f0e9c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 shutdown_connections 2024-04-25T11:37:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:30.872+0000 7f0ea2e00640 1 -- 172.21.15.19:0/1328672752 wait complete. 2024-04-25T11:37:30.889 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 170, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:37:29.871989+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 104, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8859}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8859': {'gid': 8859, 'name': 'a', 'rank': 0, 'incarnation': 170, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/2103355299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2103355299}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2103355299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:37:30.890 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8859:up:replay 2024-04-25T11:37:30.890 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:37:34.822 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:34.835+0000 7efc5d200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:37:35.457 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:35.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:37:36.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.602+0000 7f837ae00640 1 Processor -- start 2024-04-25T11:37:36.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.602+0000 7f837ae00640 1 -- start start 2024-04-25T11:37:36.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f837ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8374160370 0x7f83740d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f837ae00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83740d2df0 0x7f83740d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f837ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83740d14c0 con 0x7f83740d10b0 2024-04-25T11:37:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f837ae00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f83740d3200 con 0x7f83740d2df0 2024-04-25T11:37:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f837ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8374164840 con 0x7f8374160370 2024-04-25T11:37:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f8379000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83740d2df0 0x7f83740d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f8379000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f83740d2df0 0x7f83740d0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50152/0 (socket says 172.21.15.19:50152) 2024-04-25T11:37:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.604+0000 7f8379000640 1 -- 172.21.15.19:0/799649060 learned_addr learned my addr 172.21.15.19:0/799649060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.605+0000 7f8379000640 1 -- 172.21.15.19:0/799649060 >> v2:172.21.15.169:3300/0 conn(0x7f8374160370 msgr2=0x7f83740d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.605+0000 7f8379000640 1 --2- 172.21.15.19:0/799649060 >> v2:172.21.15.169:3300/0 conn(0x7f8374160370 0x7f83740d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.605+0000 7f8379000640 1 -- 172.21.15.19:0/799649060 >> v1:172.21.15.169:6789/0 conn(0x7f83740d10b0 legacy=0x7f83740d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.605+0000 7f8379000640 1 -- 172.21.15.19:0/799649060 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8374160780 con 0x7f83740d2df0 2024-04-25T11:37:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.606+0000 7f8379000640 1 --2- 172.21.15.19:0/799649060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d2df0 0x7f83740d0b70 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f837408e090 tx=0x7f836802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76791e98b9ace4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.606+0000 7f8373e00640 1 -- 172.21.15.19:0/799649060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368037070 con 0x7f83740d2df0 2024-04-25T11:37:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f8373e00640 1 -- 172.21.15.19:0/799649060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8368032440 con 0x7f83740d2df0 2024-04-25T11:37:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f8373e00640 1 -- 172.21.15.19:0/799649060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368032810 con 0x7f83740d2df0 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 -- 172.21.15.19:0/799649060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d2df0 msgr2=0x7f83740d0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 --2- 172.21.15.19:0/799649060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d2df0 0x7f83740d0b70 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f837408e090 tx=0x7f836802a230 comp rx=0 tx=0).stop 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 -- 172.21.15.19:0/799649060 shutdown_connections 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 --2- 172.21.15.19:0/799649060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d2df0 0x7f83740d0b70 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 --2- 172.21.15.19:0/799649060 >> v2:172.21.15.169:3300/0 conn(0x7f8374160370 0x7f83740d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 -- 172.21.15.19:0/799649060 >> 172.21.15.19:0/799649060 conn(0x7f837406dfe0 msgr2=0x7f837406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 -- 172.21.15.19:0/799649060 shutdown_connections 2024-04-25T11:37:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.607+0000 7f837ae00640 1 -- 172.21.15.19:0/799649060 wait complete. 2024-04-25T11:37:36.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.610+0000 7f837ae00640 1 Processor -- start 2024-04-25T11:37:36.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.610+0000 7f837ae00640 1 -- start start 2024-04-25T11:37:36.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.611+0000 7f837ae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d10b0 0x7f83741fa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:36.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.611+0000 7f837ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83740d2df0 0x7f837414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:36.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.611+0000 7f837ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8374160370 0x7f83740b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:36.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.611+0000 7f837ae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83741641f0 con 0x7f83740d10b0 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.611+0000 7f837ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8374163f10 con 0x7f8374160370 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.611+0000 7f837ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83741644d0 con 0x7f83740d2df0 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d10b0 0x7f83741fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d10b0 0x7f83741fa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50164/0 (socket says 172.21.15.19:50164) 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 -- 172.21.15.19:0/1720963912 learned_addr learned my addr 172.21.15.19:0/1720963912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 -- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83740d2df0 msgr2=0x7f837414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83740d2df0 0x7f837414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 -- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8374160370 msgr2=0x7f83740b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8374160370 0x7f83740b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 -- 172.21.15.19:0/1720963912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83740cdc20 con 0x7f83740d10b0 2024-04-25T11:37:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.612+0000 7f8379a00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d10b0 0x7f83741fa950 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f836400b440 tx=0x7f836400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.613+0000 7f8371600640 1 -- 172.21.15.19:0/1720963912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364015020 con 0x7f83740d10b0 2024-04-25T11:37:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.613+0000 7f8371600640 1 -- 172.21.15.19:0/1720963912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8364011890 con 0x7f83740d10b0 2024-04-25T11:37:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.613+0000 7f8371600640 1 -- 172.21.15.19:0/1720963912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364011c80 con 0x7f83740d10b0 2024-04-25T11:37:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.613+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83741fae90 con 0x7f83740d10b0 2024-04-25T11:37:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.613+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83740cd0a0 con 0x7f83740d10b0 2024-04-25T11:37:36.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.614+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83740d4050 con 0x7f83740d10b0 2024-04-25T11:37:36.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.615+0000 7f8371600640 1 -- 172.21.15.19:0/1720963912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8364018020 con 0x7f83740d10b0 2024-04-25T11:37:36.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.617+0000 7f8371600640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83580744e0 0x7f83580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:36.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.617+0000 7f8371600640 1 -- 172.21.15.19:0/1720963912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f8364095730 con 0x7f83740d10b0 2024-04-25T11:37:36.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.617+0000 7f8379000640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83580744e0 0x7f83580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:36.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.619+0000 7f8379000640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83580744e0 0x7f83580769f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8368034000 tx=0x7f836802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:36.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.621+0000 7f8371600640 1 -- 172.21.15.19:0/1720963912 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f836401f020 con 0x7f83740d10b0 2024-04-25T11:37:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.865+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8374159460 con 0x7f83740d10b0 2024-04-25T11:37:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.866+0000 7f8371600640 1 -- 172.21.15.19:0/1720963912 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) ==== 95+0+5194 (secure 0 0 0) 0x7f83741fae90 con 0x7f83740d10b0 2024-04-25T11:37:36.854 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:36.854 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":173,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:32.905551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8859":{"gid":8859,"name":"a","rank":0,"incarnation":170,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6835/2103355299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2103355299},{"type":"v1","addr":"172.21.15.19:6835","nonce":2103355299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8859,"qdb_cluster":[8859]},"id":6}]} 2024-04-25T11:37:36.854 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 173 2024-04-25T11:37:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.869+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83580744e0 msgr2=0x7f83580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.869+0000 7f837ae00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83580744e0 0x7f83580769f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8368034000 tx=0x7f836802c000 comp rx=0 tx=0).stop 2024-04-25T11:37:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.869+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d10b0 msgr2=0x7f83741fa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.869+0000 7f837ae00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d10b0 0x7f83741fa950 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f836400b440 tx=0x7f836400b900 comp rx=0 tx=0).stop 2024-04-25T11:37:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 shutdown_connections 2024-04-25T11:37:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83580744e0 0x7f83580769f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8374160370 0x7f83740b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83740d2df0 0x7f837414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 --2- 172.21.15.19:0/1720963912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83740d10b0 0x7f83741fa950 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 >> 172.21.15.19:0/1720963912 conn(0x7f837406dfe0 msgr2=0x7f837415af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 shutdown_connections 2024-04-25T11:37:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:36.870+0000 7f837ae00640 1 -- 172.21.15.19:0/1720963912 wait complete. 2024-04-25T11:37:36.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 173, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:37:32.905551+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 104, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8859}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8859': {'gid': 8859, 'name': 'a', 'rank': 0, 'incarnation': 170, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6835/2103355299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2103355299}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2103355299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8859, 'qdb_cluster': [8859]} 2024-04-25T11:37:36.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:37:36.885 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:37:37.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.184+0000 7f7553e00640 1 Processor -- start 2024-04-25T11:37:37.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.184+0000 7f7553e00640 1 -- start start 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7553e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f754c0d0ed0 0x7f754c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7553e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7553e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754c161530 con 0x7f754c1434d0 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7553e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f754c1438e0 con 0x7f754c0d0ed0 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7553e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f754c06f3e0 con 0x7f754c160be0 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7550a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f754c0d0ed0 0x7f754c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7550a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f754c0d0ed0 0x7f754c0d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50166/0 (socket says 172.21.15.19:50166) 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.187+0000 7f7550a00640 1 -- 172.21.15.19:0/921824154 learned_addr learned my addr 172.21.15.19:0/921824154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:37.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.188+0000 7f7551e00640 1 --2- 172.21.15.19:0/921824154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.188+0000 7f7550a00640 1 -- 172.21.15.19:0/921824154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 msgr2=0x7f754c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.188+0000 7f7550a00640 1 --2- 172.21.15.19:0/921824154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.188+0000 7f7550a00640 1 -- 172.21.15.19:0/921824154 >> v1:172.21.15.169:6789/0 conn(0x7f754c1434d0 legacy=0x7f754c070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.188+0000 7f7550a00640 1 -- 172.21.15.19:0/921824154 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c06f6c0 con 0x7f754c0d0ed0 2024-04-25T11:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.189+0000 7f7550a00640 1 --2- 172.21.15.19:0/921824154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c0d12e0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7538009870 tx=0x7f753802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7e9253957abcff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.189+0000 7f754be00640 1 -- 172.21.15.19:0/921824154 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538037070 con 0x7f754c0d0ed0 2024-04-25T11:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.190+0000 7f754be00640 1 -- 172.21.15.19:0/921824154 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7538032440 con 0x7f754c0d0ed0 2024-04-25T11:37:37.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.190+0000 7f754be00640 1 -- 172.21.15.19:0/921824154 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538032810 con 0x7f754c0d0ed0 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.190+0000 7f7553e00640 1 -- 172.21.15.19:0/921824154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 msgr2=0x7f754c0d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.190+0000 7f7553e00640 1 --2- 172.21.15.19:0/921824154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c0d12e0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7538009870 tx=0x7f753802a270 comp rx=0 tx=0).stop 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.191+0000 7f7553e00640 1 -- 172.21.15.19:0/921824154 shutdown_connections 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.191+0000 7f7553e00640 1 --2- 172.21.15.19:0/921824154 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.191+0000 7f7553e00640 1 --2- 172.21.15.19:0/921824154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c0d12e0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.191+0000 7f7553e00640 1 -- 172.21.15.19:0/921824154 >> 172.21.15.19:0/921824154 conn(0x7f754c0d1ae0 msgr2=0x7f754c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.191+0000 7f7553e00640 1 -- 172.21.15.19:0/921824154 shutdown_connections 2024-04-25T11:37:37.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.191+0000 7f7553e00640 1 -- 172.21.15.19:0/921824154 wait complete. 2024-04-25T11:37:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.193+0000 7f7553e00640 1 Processor -- start 2024-04-25T11:37:37.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.193+0000 7f7553e00640 1 -- start start 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7553e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c1fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7553e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f754c1434d0 0x7f754c15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7553e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7553e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f754c06f090 con 0x7f754c0d0ed0 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7553e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f754c06fc80 con 0x7f754c1434d0 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7553e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f754c0b51a0 con 0x7f754c160be0 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7551400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c1fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7551400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c1fe8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50172/0 (socket says 172.21.15.19:50172) 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.195+0000 7f7551400640 1 -- 172.21.15.19:0/2961644710 learned_addr learned my addr 172.21.15.19:0/2961644710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7550a00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f754c1434d0 0x7f754c15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551400640 1 -- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 msgr2=0x7f754c145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551400640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551400640 1 -- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f754c1434d0 msgr2=0x7f754c15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551400640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f754c1434d0 0x7f754c15acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551400640 1 -- 172.21.15.19:0/2961644710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c140450 con 0x7f754c0d0ed0 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7551400640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c1fe8a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f754000c3f0 tx=0x7f754000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7549600640 1 -- 172.21.15.19:0/2961644710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75400052d0 con 0x7f754c0d0ed0 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7549600640 1 -- 172.21.15.19:0/2961644710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7540005490 con 0x7f754c0d0ed0 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.196+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c140730 con 0x7f754c0d0ed0 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.197+0000 7f7549600640 1 -- 172.21.15.19:0/2961644710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540007260 con 0x7f754c0d0ed0 2024-04-25T11:37:37.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.197+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c15b210 con 0x7f754c0d0ed0 2024-04-25T11:37:37.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.200+0000 7f7549600640 1 -- 172.21.15.19:0/2961644710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7540007400 con 0x7f754c0d0ed0 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.200+0000 7f7549600640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f752c0744e0 0x7f752c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.201+0000 7f7549600640 1 -- 172.21.15.19:0/2961644710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f7540094b90 con 0x7f754c0d0ed0 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.201+0000 7f755adad740 1 Processor -- start 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.201+0000 7f755adad740 1 -- start start 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.201+0000 7f7550a00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f752c0744e0 0x7f752c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.202+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd33270 0x55873e100040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.202+0000 7f755adad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd0a3b0 0x55873e11ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.202+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd12590 0x55873e11b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.202+0000 7f755adad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55873e039600 con 0x55873dd0a3b0 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.202+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55873dfd6100 con 0x55873dd12590 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.202+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55873e0355c0 con 0x55873dd33270 2024-04-25T11:37:37.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f752c0744e0 0x7f752c0769f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7538034000 tx=0x7f753802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd0a3b0 0x55873e11ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd0a3b0 0x55873e11ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50186/0 (socket says 172.21.15.19:50186) 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 -- 172.21.15.19:0/3974218014 learned_addr learned my addr 172.21.15.19:0/3974218014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7551400640 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd33270 0x55873e100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7551e00640 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd12590 0x55873e11b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 -- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd33270 msgr2=0x55873e100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd33270 0x55873e100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 -- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd12590 msgr2=0x55873e11b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd12590 0x55873e11b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7550a00640 1 -- 172.21.15.19:0/3974218014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873dd4b2d0 con 0x55873dd0a3b0 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.203+0000 7f7551e00640 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd12590 0x55873e11b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.204+0000 7f7550a00640 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd0a3b0 0x55873e11ad70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7538006f20 tx=0x7f753802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764fb9c27f1bfb53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.204+0000 7f753dc00640 1 -- 172.21.15.19:0/3974218014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538037040 con 0x55873dd0a3b0 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.204+0000 7f753dc00640 1 -- 172.21.15.19:0/3974218014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753804c070 con 0x55873dd0a3b0 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.204+0000 7f753dc00640 1 -- 172.21.15.19:0/3974218014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753803c860 con 0x55873dd0a3b0 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 -- 172.21.15.19:0/3974218014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd0a3b0 msgr2=0x55873e11ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd0a3b0 0x55873e11ad70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7538006f20 tx=0x7f753802c000 comp rx=0 tx=0).stop 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 -- 172.21.15.19:0/3974218014 shutdown_connections 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd12590 0x55873e11b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd0a3b0 0x55873e11ad70 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 --2- 172.21.15.19:0/3974218014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd33270 0x55873e100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 -- 172.21.15.19:0/3974218014 >> 172.21.15.19:0/3974218014 conn(0x55873dd273c0 msgr2=0x55873e0d56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 -- 172.21.15.19:0/3974218014 shutdown_connections 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.205+0000 7f755adad740 1 -- 172.21.15.19:0/3974218014 wait complete. 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.206+0000 7f755adad740 1 -- start start 2024-04-25T11:37:37.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.206+0000 7f755adad740 1 Processor -- start 2024-04-25T11:37:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd0a3b0 0x55873e101860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f755adad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd12590 0x55873e0fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd33270 0x55873e0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f755adad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55873dfc0f70 con 0x55873dd12590 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55873e117d00 con 0x55873dd33270 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55873e101f10 con 0x55873dd0a3b0 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f755adad740 10 client.0 _ll_register_callbacks cb 0x55873e02f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f7550a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd12590 0x55873e0fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f7550a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd12590 0x55873e0fa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50192/0 (socket says 172.21.15.19:50192) 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.208+0000 7f7550a00640 1 -- 172.21.15.19:0/3548797955 learned_addr learned my addr 172.21.15.19:0/3548797955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7551400640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd0a3b0 0x55873e101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7551e00640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd33270 0x55873e0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7550a00640 1 -- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd0a3b0 msgr2=0x55873e101860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7550a00640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd0a3b0 0x55873e101860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7550a00640 1 -- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd33270 msgr2=0x55873e0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7550a00640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd33270 0x55873e0cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7550a00640 1 -- 172.21.15.19:0/3548797955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873de156b0 con 0x55873dd12590 2024-04-25T11:37:37.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7551400640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd0a3b0 0x55873e101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7551e00640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd33270 0x55873e0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7550a00640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd12590 0x55873e0fa790 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f753802a860 tx=0x7f7538004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2c8322ebfa3643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7532000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:37:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7532000640 1 -- 172.21.15.19:0/3548797955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538037040 con 0x55873dd12590 2024-04-25T11:37:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f7532000640 1 -- 172.21.15.19:0/3548797955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f75380503f0 con 0x55873dd12590 2024-04-25T11:37:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.209+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f753804c070 con 0x55873dd12590 2024-04-25T11:37:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f7532000640 1 -- 172.21.15.19:0/3548797955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f7538061050 con 0x55873dd12590 2024-04-25T11:37:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 10 client.8946 fetch_fsmap learned FSMap version 173 2024-04-25T11:37:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=2+}) -- 0x55873e117ad0 con 0x55873dd12590 2024-04-25T11:37:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f7532000640 1 -- 172.21.15.19:0/3548797955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753804c070 con 0x55873dd12590 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f7532000640 1 -- 172.21.15.19:0/3548797955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 173) ==== 3494+0+0 (secure 0 0 0) 0x7f7538064020 con 0x55873dd12590 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 10 client.8946 fetch_fsmap finished waiting for FSMap version 173 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 10 client.8946 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 20 client.8946 populate_metadata read hostname 'smithi019' 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873dcf1e40 0x55873e116da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 4 client.8946 mds_command: new command op to 8859 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.210+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 --> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873e047210 con 0x55873dcf1e40 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.211+0000 7f7551400640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873dcf1e40 0x55873e116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.211+0000 7f7551400640 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873dcf1e40 0x55873e116da0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c6b57c9851f636 server_cookie=400fa6aa06d07373 in_seq=0 out_seq=0 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.211+0000 7f7532000640 10 client.8946 ms_handle_connect on v2:172.21.15.19:6834/2103355299 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.212+0000 7f7532000640 1 -- 172.21.15.19:0/3548797955 <== mds.0 v2:172.21.15.19:6834/2103355299 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55873e047210 con 0x55873dcf1e40 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.212+0000 7f7532000640 10 client.8946 handle_command_reply: tid=0 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.212+0000 7f755adad740 1 client.8946 shutdown 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.212+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd12590 msgr2=0x55873e0fa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.212+0000 7f755adad740 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd12590 0x55873e0fa790 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f753802a860 tx=0x7f7538004730 comp rx=0 tx=0).stop 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 shutdown_connections 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873dcf1e40 0x55873e116da0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd33270 0x55873e0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd12590 0x55873e0fa790 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 --2- 172.21.15.19:0/3548797955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd0a3b0 0x55873e101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f7532000640 0 client.8946 ms_handle_reset on v2:172.21.15.19:6834/2103355299 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 >> 172.21.15.19:0/3548797955 conn(0x55873dd273c0 msgr2=0x55873e0d56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 shutdown_connections 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 1 -- 172.21.15.19:0/3548797955 wait complete. 2024-04-25T11:37:37.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.213+0000 7f755adad740 20 client.8946 trim_cache size 0 max 16384 2024-04-25T11:37:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.236+0000 7f755adad740 1 Processor -- start 2024-04-25T11:37:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.236+0000 7f755adad740 1 -- start start 2024-04-25T11:37:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e117190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 0x55873e12c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f755adad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f755adad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f754c207bf0 con 0x55873dd33270 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f754c206c60 con 0x55873dd0a3b0 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f754c206610 con 0x55873dd12590 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f7551e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f7551e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50196/0 (socket says 172.21.15.19:50196) 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.238+0000 7f7551e00640 1 -- 172.21.15.19:0/2351558646 learned_addr learned my addr 172.21.15.19:0/2351558646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7551400640 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e117190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7550a00640 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 0x55873e12c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7551e00640 1 -- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 msgr2=0x55873e12c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7551e00640 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 0x55873e12c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7551e00640 1 -- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 msgr2=0x55873e117190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7551e00640 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e117190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7551e00640 1 -- 172.21.15.19:0/2351558646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873e0cf5e0 con 0x55873dd33270 2024-04-25T11:37:37.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.239+0000 7f7551e00640 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12ccc0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7544009c80 tx=0x7f7544007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f862893eb8baba21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f753d200640 1 -- 172.21.15.19:0/2351558646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544019020 con 0x55873dd33270 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f753d200640 1 -- 172.21.15.19:0/2351558646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7544015650 con 0x55873dd33270 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f753d200640 1 -- 172.21.15.19:0/2351558646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544015a40 con 0x55873dd33270 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f755adad740 1 -- 172.21.15.19:0/2351558646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 msgr2=0x55873e12ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f755adad740 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12ccc0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7544009c80 tx=0x7f7544007700 comp rx=0 tx=0).stop 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f755adad740 1 -- 172.21.15.19:0/2351558646 shutdown_connections 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f755adad740 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12ccc0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f755adad740 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 0x55873e12c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f755adad740 1 --2- 172.21.15.19:0/2351558646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e117190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.240+0000 7f755adad740 1 -- 172.21.15.19:0/2351558646 >> 172.21.15.19:0/2351558646 conn(0x55873dd273c0 msgr2=0x55873e1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.241+0000 7f755adad740 1 -- 172.21.15.19:0/2351558646 shutdown_connections 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.241+0000 7f755adad740 1 -- 172.21.15.19:0/2351558646 wait complete. 2024-04-25T11:37:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.241+0000 7f755adad740 1 -- start start 2024-04-25T11:37:37.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.242+0000 7f755adad740 1 Processor -- start 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.243+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e13dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.243+0000 7f755adad740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 0x55873e197600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.243+0000 7f755adad740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.243+0000 7f755adad740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55873e0fe3f0 con 0x55873dd33270 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.243+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55873e0d0330 con 0x55873dd0a3b0 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.243+0000 7f755adad740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55873e0fea40 con 0x55873dd12590 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.243+0000 7f755adad740 10 client.0 _ll_register_callbacks cb 0x7f75380096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50206/0 (socket says 172.21.15.19:50206) 2024-04-25T11:37:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 -- 172.21.15.19:0/2323811027 learned_addr learned my addr 172.21.15.19:0/2323811027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 -- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 msgr2=0x55873e197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 0x55873e197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 -- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 msgr2=0x55873e13dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551400640 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e13dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e13dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 -- 172.21.15.19:0/2323811027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873e117850 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551400640 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e13dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7551e00640 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12c780 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f754c146e50 tx=0x7f7544004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dd6db29925b9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.244+0000 7f7532000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f7532000640 1 -- 172.21.15.19:0/2323811027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544003b90 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f7532000640 1 -- 172.21.15.19:0/2323811027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f7544003d30 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7544015650 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f7532000640 1 -- 172.21.15.19:0/2323811027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f7544030050 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f755adad740 10 client.8952 fetch_fsmap learned FSMap version 173 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=2+}) -- 0x55873e13d420 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f7532000640 1 -- 172.21.15.19:0/2323811027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7544015650 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f7532000640 1 -- 172.21.15.19:0/2323811027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 173) ==== 3494+0+0 (secure 0 0 0) 0x7f7544033020 con 0x55873dd33270 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f755adad740 10 client.8952 fetch_fsmap finished waiting for FSMap version 173 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f755adad740 10 client.8952 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:37:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.245+0000 7f755adad740 20 client.8952 populate_metadata read hostname 'smithi019' 2024-04-25T11:37:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.246+0000 7f755adad740 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873e13cff0 0x55873e11ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.246+0000 7f755adad740 4 client.8952 mds_command: new command op to 8859 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:37:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.246+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 --> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] -- command(tid 0: {"prefix": "status"}) -- 0x55873e041080 con 0x55873e13cff0 2024-04-25T11:37:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.246+0000 7f7551400640 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873e13cff0 0x55873e11ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.246+0000 7f7551400640 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873e13cff0 0x55873e11ad10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cdbdda6d9fbed1f server_cookie=f124c9b187dbeb4c in_seq=0 out_seq=0 2024-04-25T11:37:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.246+0000 7f7532000640 10 client.8952 ms_handle_connect on v2:172.21.15.19:6834/2103355299 2024-04-25T11:37:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f7532000640 1 -- 172.21.15.19:0/2323811027 <== mds.0 v2:172.21.15.19:6834/2103355299 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55873e041080 con 0x55873e13cff0 2024-04-25T11:37:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f7532000640 10 client.8952 handle_command_reply: tid=0 2024-04-25T11:37:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 client.8952 shutdown 2024-04-25T11:37:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 msgr2=0x55873e12c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12c780 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f754c146e50 tx=0x7f7544004ab0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 shutdown_connections 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:6834/2103355299,v1:172.21.15.19:6835/2103355299] conn(0x55873e13cff0 0x55873e11ad10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55873dd33270 0x55873e12c780 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55873dd12590 0x55873e197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 --2- 172.21.15.19:0/2323811027 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55873dd0a3b0 0x55873e13dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 >> 172.21.15.19:0/2323811027 conn(0x55873dd273c0 msgr2=0x55873e116540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f7532000640 0 client.8952 ms_handle_reset on v2:172.21.15.19:6834/2103355299 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 shutdown_connections 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 1 -- 172.21.15.19:0/2323811027 wait complete. 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.247+0000 7f755adad740 20 client.8952 trim_cache size 0 max 16384 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:37:37.237 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8859, 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3690632310000002, 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 173, 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 104, 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 104, 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.297629577 2024-04-25T11:37:37.238 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:37:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f752c0744e0 msgr2=0x7f752c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f752c0744e0 0x7f752c0769f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7538034000 tx=0x7f753802e000 comp rx=0 tx=0).stop 2024-04-25T11:37:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 msgr2=0x7f754c1fe8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c1fe8a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f754000c3f0 tx=0x7f754000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 shutdown_connections 2024-04-25T11:37:37.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f752c0744e0 0x7f752c0769f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f754c1434d0 0x7f754c15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f754c160be0 0x7f754c145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 --2- 172.21.15.19:0/2961644710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c0d0ed0 0x7f754c1fe8a0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 >> 172.21.15.19:0/2961644710 conn(0x7f754c0d1ae0 msgr2=0x7f754c15fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 shutdown_connections 2024-04-25T11:37:37.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.248+0000 7f7553e00640 1 -- 172.21.15.19:0/2961644710 wait complete. 2024-04-25T11:37:37.265 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:37:37.265 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2024-04-25T11:37:37.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:37:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.569+0000 7f13d3e00640 1 Processor -- start 2024-04-25T11:37:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.570+0000 7f13d3e00640 1 -- start start 2024-04-25T11:37:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.572+0000 7f13d3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f13d4140e00 0x7f13d4143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.572+0000 7f13d3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d4141210 con 0x7f13d406dce0 2024-04-25T11:37:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.572+0000 7f13d3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d4141510 con 0x7f13d4143a90 2024-04-25T11:37:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.572+0000 7f13d3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f13d4141b40 con 0x7f13d4140e00 2024-04-25T11:37:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.572+0000 7f13d3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f13d406dce0 0x7f13d4141fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46982/0 (socket says 172.21.15.19:46982) 2024-04-25T11:37:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.572+0000 7f13d3400640 1 -- 172.21.15.19:0/994150275 learned_addr learned my addr 172.21.15.19:0/994150275 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.573+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545446474 0 0) 0x7f13d4141210 con 0x7f13d406dce0 2024-04-25T11:37:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.573+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b8003520 con 0x7f13d406dce0 2024-04-25T11:37:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.573+0000 7f13d2a00640 1 --2- 172.21.15.19:0/994150275 >> v2:172.21.15.169:3300/0 conn(0x7f13d4140e00 0x7f13d4143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.573+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2100647758 0 0) 0x7f13b8003520 con 0x7f13d406dce0 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.573+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 >> v2:172.21.15.169:3300/0 conn(0x7f13d4140e00 msgr2=0x7f13d4143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 --2- 172.21.15.19:0/994150275 >> v2:172.21.15.169:3300/0 conn(0x7f13d4140e00 0x7f13d4143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 >> v1:172.21.15.169:6789/0 conn(0x7f13d4143a90 legacy=0x7f13d406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d41440f0 con 0x7f13d406dce0 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f13cc0019f0 con 0x7f13d406dce0 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 >> v1:172.21.15.19:6789/0 conn(0x7f13d406dce0 legacy=0x7f13d4141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b8003a00 0x7f13b8003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13b8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b8005350 0x7f13b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4141b40 con 0x7f13b8004800 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f13b800c280 con 0x7f13b8005350 2024-04-25T11:37:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f13b800c480 con 0x7f13b8003a00 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13cbe00640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13b8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d2a00640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b8003a00 0x7f13b8003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.574+0000 7f13d3400640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b8005350 0x7f13b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.575+0000 7f13cbe00640 1 -- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b8003a00 msgr2=0x7f13b8003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.575+0000 7f13cbe00640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13b8003a00 0x7f13b8003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.575+0000 7f13cbe00640 1 -- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b8005350 msgr2=0x7f13b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.575+0000 7f13cbe00640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13b8005350 0x7f13b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.575+0000 7f13d3400640 1 -- 172.21.15.19:0/994150275 reap_dead start 2024-04-25T11:37:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.575+0000 7f13cbe00640 1 -- 172.21.15.19:0/994150275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b8003520 con 0x7f13b8004800 2024-04-25T11:37:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.576+0000 7f13cbe00640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13b8004c10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f13bc001750 tx=0x7f13b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a68cdd2e3a02d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.576+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc001370 con 0x7f13b8004800 2024-04-25T11:37:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.576+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc0052d0 con 0x7f13b8004800 2024-04-25T11:37:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.576+0000 7f13d2000640 1 -- 172.21.15.19:0/994150275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc02abc0 con 0x7f13b8004800 2024-04-25T11:37:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.577+0000 7f13d3e00640 1 -- 172.21.15.19:0/994150275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 msgr2=0x7f13b8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.577+0000 7f13d3e00640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13b8004c10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f13bc001750 tx=0x7f13b8005830 comp rx=0 tx=0).stop 2024-04-25T11:37:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.577+0000 7f13d3e00640 1 -- 172.21.15.19:0/994150275 shutdown_connections 2024-04-25T11:37:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.577+0000 7f13d3e00640 1 --2- 172.21.15.19:0/994150275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13b8004c10 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.577+0000 7f13d3e00640 1 -- 172.21.15.19:0/994150275 >> 172.21.15.19:0/994150275 conn(0x7f13d4072ee0 msgr2=0x7f13d40732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.577+0000 7f13d3e00640 1 -- 172.21.15.19:0/994150275 shutdown_connections 2024-04-25T11:37:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.577+0000 7f13d3e00640 1 -- 172.21.15.19:0/994150275 wait complete. 2024-04-25T11:37:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.580+0000 7f13d3e00640 1 Processor -- start 2024-04-25T11:37:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.580+0000 7f13d3e00640 1 -- start start 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.581+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13d413d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.581+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d4157c40 0x7f13d40ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.581+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d40aeb80 0x7f13d40b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.581+0000 7f13d3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41417d0 con 0x7f13b8004800 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.581+0000 7f13d3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4143ea0 con 0x7f13d40aeb80 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.581+0000 7f13d3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f13d40d1490 con 0x7f13d4157c40 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13d413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13d413d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50226/0 (socket says 172.21.15.19:50226) 2024-04-25T11:37:37.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 -- 172.21.15.19:0/731100604 learned_addr learned my addr 172.21.15.19:0/731100604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d3400640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d40aeb80 0x7f13d40b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13cbe00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d4157c40 0x7f13d40ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 -- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d4157c40 msgr2=0x7f13d40ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d4157c40 0x7f13d40ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 -- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d40aeb80 msgr2=0x7f13d40b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d40aeb80 0x7f13d40b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.582+0000 7f13d2a00640 1 -- 172.21.15.19:0/731100604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d4156ba0 con 0x7f13b8004800 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.583+0000 7f13d2a00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13d413d960 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f13c400f600 tx=0x7f13c400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.583+0000 7f13caa00640 1 -- 172.21.15.19:0/731100604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4019020 con 0x7f13b8004800 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.583+0000 7f13caa00640 1 -- 172.21.15.19:0/731100604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c4014870 con 0x7f13b8004800 2024-04-25T11:37:37.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.583+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d4156e80 con 0x7f13b8004800 2024-04-25T11:37:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.583+0000 7f13caa00640 1 -- 172.21.15.19:0/731100604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4014c20 con 0x7f13b8004800 2024-04-25T11:37:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.583+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d4157440 con 0x7f13b8004800 2024-04-25T11:37:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.584+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d4050740 con 0x7f13b8004800 2024-04-25T11:37:37.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.586+0000 7f13caa00640 1 -- 172.21.15.19:0/731100604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f13c401c020 con 0x7f13b8004800 2024-04-25T11:37:37.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.587+0000 7f13caa00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13b00744b0 0x7f13b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:37.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.588+0000 7f13caa00640 1 -- 172.21.15.19:0/731100604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f13c4095f30 con 0x7f13b8004800 2024-04-25T11:37:37.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.588+0000 7f13cbe00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13b00744b0 0x7f13b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:37.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.590+0000 7f13cbe00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13b00744b0 0x7f13b00769c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f13bc032000 tx=0x7f13bc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.592+0000 7f13caa00640 1 -- 172.21.15.19:0/731100604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13c4023020 con 0x7f13b8004800 2024-04-25T11:37:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.835+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f13d40b7710 con 0x7f13b8004800 2024-04-25T11:37:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.836+0000 7f13caa00640 1 -- 172.21.15.19:0/731100604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v173) ==== 53+0+73 (secure 0 0 0) 0x7f13c4060330 con 0x7f13b8004800 2024-04-25T11:37:37.824 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:37:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13b00744b0 msgr2=0x7f13b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13b00744b0 0x7f13b00769c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f13bc032000 tx=0x7f13bc02e000 comp rx=0 tx=0).stop 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 msgr2=0x7f13d413d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13d413d960 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f13c400f600 tx=0x7f13c400fac0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 shutdown_connections 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13b00744b0 0x7f13b00769c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f13d40aeb80 0x7f13d40b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f13d4157c40 0x7f13d40ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 --2- 172.21.15.19:0/731100604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13b8004800 0x7f13d413d960 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 >> 172.21.15.19:0/731100604 conn(0x7f13d4072ee0 msgr2=0x7f13d40701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 shutdown_connections 2024-04-25T11:37:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:37.839+0000 7f13d3e00640 1 -- 172.21.15.19:0/731100604 wait complete. 2024-04-25T11:37:37.854 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:37:37.854 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:37:37.855 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:37:37.855 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:37:37.855 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:37:37.855 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:37:37.855 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:37:37.855 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:37:37.855 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:37:37.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:37.855 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:37:37.903 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:37:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:37:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:37:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:37:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42016sec preferred_lft 42016sec 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:37:37.904 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:37:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:37:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:37:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:37:37.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:37.907 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:37:37.982 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:37:37.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:37.987 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:37:38.038 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:37:38.038 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:37:38.039 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:37:38.039 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:37:38.039 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:37:38.039 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:37:38.039 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:37:38.039 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:37:38.039 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:37:38.039 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:37:38.039 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:37:38.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:37:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:37:38.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:38.258 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:37:38.308 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:37:38.308 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:37:38.309 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:37:38.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:38.309 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:37:38.396 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:37:38.396 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:37:38.456 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:37:38.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:37:38.457 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:37:38.509 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:37:38.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:38.509 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:37:38.722 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:37:38.722 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:37:38.769 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:37:38.769 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:37:38.770 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:37:38.770 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:37:38.770 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:37:38.772 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:37:38.772 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:37:38.772 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:37:38.772 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:37:38.772 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:37:38.777 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:37:38.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:38.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:37:38.856 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:37:38.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:37:38.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:37:38.980 INFO:teuthology.orchestra.run.smithi019.stdout:client6236 2024-04-25T11:37:38.980 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:38.980 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:39.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:39.036 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:37:39.040 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:37:39.093 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:37:39.093 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:37:39.093 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:39.093 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:37:39.093 INFO:teuthology.orchestra.run.smithi019.stdout:sixmegs 2024-04-25T11:37:39.094 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2024-04-25T11:37:39.094 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2024-04-25T11:37:39.095 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:39.095 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:39.149 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:39.149 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:39.206 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:37:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.505+0000 7ff9f6e00640 1 Processor -- start 2024-04-25T11:37:39.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.505+0000 7ff9f6e00640 1 -- start start 2024-04-25T11:37:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.507+0000 7ff9f6e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9f01443c0 0x7ff9f006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.507+0000 7ff9f6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f00cfcf0 0x7ff9f006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.507+0000 7ff9f6e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f015f5f0 con 0x7ff9f0070370 2024-04-25T11:37:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.507+0000 7ff9f6e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f00708d0 con 0x7ff9f01443c0 2024-04-25T11:37:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.507+0000 7ff9f6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f015f160 con 0x7ff9f00cfcf0 2024-04-25T11:37:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.508+0000 7ff9efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9f01443c0 0x7ff9f006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.508+0000 7ff9efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9f01443c0 0x7ff9f006fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50232/0 (socket says 172.21.15.19:50232) 2024-04-25T11:37:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.508+0000 7ff9efe00640 1 -- 172.21.15.19:0/589673032 learned_addr learned my addr 172.21.15.19:0/589673032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.508+0000 7ff9f4e00640 1 --2- 172.21.15.19:0/589673032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f00cfcf0 0x7ff9f006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.509+0000 7ff9efe00640 1 -- 172.21.15.19:0/589673032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f00cfcf0 msgr2=0x7ff9f006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.509+0000 7ff9efe00640 1 --2- 172.21.15.19:0/589673032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f00cfcf0 0x7ff9f006c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.509+0000 7ff9efe00640 1 -- 172.21.15.19:0/589673032 >> v1:172.21.15.169:6789/0 conn(0x7ff9f0070370 legacy=0x7ff9f00cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.509+0000 7ff9efe00640 1 -- 172.21.15.19:0/589673032 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f015e830 con 0x7ff9f01443c0 2024-04-25T11:37:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.510+0000 7ff9efe00640 1 --2- 172.21.15.19:0/589673032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f006fe30 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009870 tx=0x7ff9e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8381eded8088826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.510+0000 7ff9ef400640 1 -- 172.21.15.19:0/589673032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0037070 con 0x7ff9f01443c0 2024-04-25T11:37:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.511+0000 7ff9ef400640 1 -- 172.21.15.19:0/589673032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e0032440 con 0x7ff9f01443c0 2024-04-25T11:37:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.511+0000 7ff9ef400640 1 -- 172.21.15.19:0/589673032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0032810 con 0x7ff9f01443c0 2024-04-25T11:37:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.511+0000 7ff9f6e00640 1 -- 172.21.15.19:0/589673032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 msgr2=0x7ff9f006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.511+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/589673032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f006fe30 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009870 tx=0x7ff9e002a270 comp rx=0 tx=0).stop 2024-04-25T11:37:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.512+0000 7ff9f6e00640 1 -- 172.21.15.19:0/589673032 shutdown_connections 2024-04-25T11:37:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.512+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/589673032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f00cfcf0 0x7ff9f006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.512+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/589673032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f006fe30 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.512+0000 7ff9f6e00640 1 -- 172.21.15.19:0/589673032 >> 172.21.15.19:0/589673032 conn(0x7ff9f00d1ae0 msgr2=0x7ff9f00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.512+0000 7ff9f6e00640 1 -- 172.21.15.19:0/589673032 shutdown_connections 2024-04-25T11:37:39.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.512+0000 7ff9f6e00640 1 -- 172.21.15.19:0/589673032 wait complete. 2024-04-25T11:37:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.514+0000 7ff9f6e00640 1 Processor -- start 2024-04-25T11:37:39.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.514+0000 7ff9f6e00640 1 -- start start 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9f6e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f0070370 0x7ff9f00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9f6e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f0157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9f6e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f00af690 0x7ff9f00afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9f6e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f01447d0 con 0x7ff9f01443c0 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9f6e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f015edf0 con 0x7ff9f00af690 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9f6e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f0162b00 con 0x7ff9f0070370 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f0157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f0157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50238/0 (socket says 172.21.15.19:50238) 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.516+0000 7ff9e7e00640 1 -- 172.21.15.19:0/3341603810 learned_addr learned my addr 172.21.15.19:0/3341603810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9efe00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f0070370 0x7ff9f00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9f4e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f00af690 0x7ff9f00afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9e7e00640 1 -- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f0070370 msgr2=0x7ff9f00b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9e7e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f0070370 0x7ff9f00b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9e7e00640 1 -- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f00af690 msgr2=0x7ff9f00afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9e7e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f00af690 0x7ff9f00afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9e7e00640 1 -- 172.21.15.19:0/3341603810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f0140300 con 0x7ff9f01443c0 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9e7e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f0157e90 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c3c0 tx=0x7ff9d800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.517+0000 7ff9ecc00640 1 -- 172.21.15.19:0/3341603810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80052d0 con 0x7ff9f01443c0 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.518+0000 7ff9ecc00640 1 -- 172.21.15.19:0/3341603810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d8005490 con 0x7ff9f01443c0 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.518+0000 7ff9ecc00640 1 -- 172.21.15.19:0/3341603810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8007260 con 0x7ff9f01443c0 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.518+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f00affe0 con 0x7ff9f01443c0 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.518+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f0140020 con 0x7ff9f01443c0 2024-04-25T11:37:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.519+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f013fd90 con 0x7ff9f01443c0 2024-04-25T11:37:39.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.520+0000 7ff9ecc00640 1 -- 172.21.15.19:0/3341603810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9d8007400 con 0x7ff9f01443c0 2024-04-25T11:37:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.521+0000 7ff9ecc00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9cc0744b0 0x7ff9cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.521+0000 7ff9ecc00640 1 -- 172.21.15.19:0/3341603810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7ff9d8094b90 con 0x7ff9f01443c0 2024-04-25T11:37:39.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.522+0000 7ff9efe00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9cc0744b0 0x7ff9cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.523+0000 7ff9efe00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9cc0744b0 0x7ff9cc0769c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0034000 tx=0x7ff9e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.527+0000 7ff9ecc00640 1 -- 172.21.15.19:0/3341603810 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9f00affe0 con 0x7ff9f01443c0 2024-04-25T11:37:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.737+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9f0050740 con 0x7ff9f01443c0 2024-04-25T11:37:39.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.741+0000 7ff9ecc00640 1 -- 172.21.15.19:0/3341603810 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+18222 (secure 0 0 0) 0x7ff9f0140300 con 0x7ff9f01443c0 2024-04-25T11:37:39.729 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:39.729 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":104,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:37:23.802663+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:37:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9cc0744b0 msgr2=0x7ff9cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9cc0744b0 0x7ff9cc0769c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0034000 tx=0x7ff9e002e000 comp rx=0 tx=0).stop 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 msgr2=0x7ff9f0157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f0157e90 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c3c0 tx=0x7ff9d800c7c0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 shutdown_connections 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9cc0744b0 0x7ff9cc0769c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f00af690 0x7ff9f00afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f01443c0 0x7ff9f0157e90 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 --2- 172.21.15.19:0/3341603810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f0070370 0x7ff9f00b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 >> 172.21.15.19:0/3341603810 conn(0x7ff9f00d1ae0 msgr2=0x7ff9f00b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 shutdown_connections 2024-04-25T11:37:39.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:39.744+0000 7ff9f6e00640 1 -- 172.21.15.19:0/3341603810 wait complete. 2024-04-25T11:37:39.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:37:39.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:37:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.063+0000 7f8694e00640 1 Processor -- start 2024-04-25T11:37:40.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.064+0000 7f8694e00640 1 -- start start 2024-04-25T11:37:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.066+0000 7f8694e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8690144340 0x7f869006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.066+0000 7f8694e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f869006db10 0x7f869015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.066+0000 7f8694e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f869006df20 con 0x7f869015ec90 2024-04-25T11:37:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.066+0000 7f8694e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8690144750 con 0x7f869006db10 2024-04-25T11:37:40.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.066+0000 7f8694e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8690070000 con 0x7f8690144340 2024-04-25T11:37:40.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f868f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8690144340 0x7f869006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f868f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8690144340 0x7f869006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:33258/0 (socket says 172.21.15.19:33258) 2024-04-25T11:37:40.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f868f400640 1 -- 172.21.15.19:0/1116364992 learned_addr learned my addr 172.21.15.19:0/1116364992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:40.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f8687e00640 1 --2- 172.21.15.19:0/1116364992 >> v2:172.21.15.169:3300/0 conn(0x7f869006db10 0x7f869015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f868f400640 1 -- 172.21.15.19:0/1116364992 >> v2:172.21.15.169:3300/0 conn(0x7f869006db10 msgr2=0x7f869015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:40.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f868f400640 1 --2- 172.21.15.19:0/1116364992 >> v2:172.21.15.169:3300/0 conn(0x7f869006db10 0x7f869015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f868f400640 1 -- 172.21.15.19:0/1116364992 >> v1:172.21.15.169:6789/0 conn(0x7f869015ec90 legacy=0x7f8690161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.067+0000 7f868f400640 1 -- 172.21.15.19:0/1116364992 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869006f240 con 0x7f8690144340 2024-04-25T11:37:40.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.069+0000 7f868f400640 1 --2- 172.21.15.19:0/1116364992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8690144340 0x7f869006d5d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8680009870 tx=0x7f868002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d37be47787cfd62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.069+0000 7f868ea00640 1 -- 172.21.15.19:0/1116364992 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680037070 con 0x7f8690144340 2024-04-25T11:37:40.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.069+0000 7f868ea00640 1 -- 172.21.15.19:0/1116364992 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8680032440 con 0x7f8690144340 2024-04-25T11:37:40.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.069+0000 7f868ea00640 1 -- 172.21.15.19:0/1116364992 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680032810 con 0x7f8690144340 2024-04-25T11:37:40.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 -- 172.21.15.19:0/1116364992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8690144340 msgr2=0x7f869006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 --2- 172.21.15.19:0/1116364992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8690144340 0x7f869006d5d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8680009870 tx=0x7f868002a250 comp rx=0 tx=0).stop 2024-04-25T11:37:40.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 -- 172.21.15.19:0/1116364992 shutdown_connections 2024-04-25T11:37:40.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 --2- 172.21.15.19:0/1116364992 >> v2:172.21.15.169:3300/0 conn(0x7f869006db10 0x7f869015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 --2- 172.21.15.19:0/1116364992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8690144340 0x7f869006d5d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 -- 172.21.15.19:0/1116364992 >> 172.21.15.19:0/1116364992 conn(0x7f8690072f40 msgr2=0x7f8690073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 -- 172.21.15.19:0/1116364992 shutdown_connections 2024-04-25T11:37:40.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.070+0000 7f8694e00640 1 -- 172.21.15.19:0/1116364992 wait complete. 2024-04-25T11:37:40.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.073+0000 7f8694e00640 1 Processor -- start 2024-04-25T11:37:40.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.073+0000 7f8694e00640 1 -- start start 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.074+0000 7f8694e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f869006db10 0x7f8690141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.074+0000 7f8694e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8690144340 0x7f86901448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.074+0000 7f8694e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f869015ec90 0x7f8690144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.074+0000 7f8694e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f869006fae0 con 0x7f8690144340 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.074+0000 7f8694e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f869006f800 con 0x7f869015ec90 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.074+0000 7f8694e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f869015f0a0 con 0x7f869006db10 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8690144340 0x7f86901448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8690144340 0x7f86901448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50250/0 (socket says 172.21.15.19:50250) 2024-04-25T11:37:40.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 -- 172.21.15.19:0/1480451778 learned_addr learned my addr 172.21.15.19:0/1480451778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f868f400640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f869006db10 0x7f8690141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f868fe00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f869015ec90 0x7f8690144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 -- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f869006db10 msgr2=0x7f8690141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f869006db10 0x7f8690141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 -- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f869015ec90 msgr2=0x7f8690144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f869015ec90 0x7f8690144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 -- 172.21.15.19:0/1480451778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869015b030 con 0x7f8690144340 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f868fe00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f869015ec90 0x7f8690144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.075+0000 7f8687e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8690144340 0x7f86901448d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f867800e8a0 tx=0x7f867800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.076+0000 7f8687400640 1 -- 172.21.15.19:0/1480451778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678017070 con 0x7f8690144340 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.076+0000 7f8687400640 1 -- 172.21.15.19:0/1480451778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867800ee90 con 0x7f8690144340 2024-04-25T11:37:40.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.076+0000 7f8687400640 1 -- 172.21.15.19:0/1480451778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867800cb60 con 0x7f8690144340 2024-04-25T11:37:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.076+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869015b310 con 0x7f8690144340 2024-04-25T11:37:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.076+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869015b8d0 con 0x7f8690144340 2024-04-25T11:37:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.077+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8690076c60 con 0x7f8690144340 2024-04-25T11:37:40.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.078+0000 7f8687400640 1 -- 172.21.15.19:0/1480451778 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f867801a070 con 0x7f8690144340 2024-04-25T11:37:40.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.079+0000 7f8687400640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f866c0744e0 0x7f866c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.080+0000 7f8687400640 1 -- 172.21.15.19:0/1480451778 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f8678095810 con 0x7f8690144340 2024-04-25T11:37:40.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.080+0000 7f868f400640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f866c0744e0 0x7f866c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.082+0000 7f868f400640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f866c0744e0 0x7f866c0769f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8680034000 tx=0x7f868002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.084+0000 7f8687400640 1 -- 172.21.15.19:0/1480451778 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f867805fb80 con 0x7f8690144340 2024-04-25T11:37:40.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.291+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f869006d210 con 0x7f8690144340 2024-04-25T11:37:40.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.292+0000 7f8687400640 1 -- 172.21.15.19:0/1480451778 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f867805f480 con 0x7f8690144340 2024-04-25T11:37:40.279 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:37:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f866c0744e0 msgr2=0x7f866c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f866c0744e0 0x7f866c0769f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8680034000 tx=0x7f868002e000 comp rx=0 tx=0).stop 2024-04-25T11:37:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8690144340 msgr2=0x7f86901448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8690144340 0x7f86901448d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f867800e8a0 tx=0x7f867800ed60 comp rx=0 tx=0).stop 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 shutdown_connections 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f866c0744e0 0x7f866c0769f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f869015ec90 0x7f8690144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8690144340 0x7f86901448d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 --2- 172.21.15.19:0/1480451778 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f869006db10 0x7f8690141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 >> 172.21.15.19:0/1480451778 conn(0x7f8690072f40 msgr2=0x7f86900b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 shutdown_connections 2024-04-25T11:37:40.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.295+0000 7f8694e00640 1 -- 172.21.15.19:0/1480451778 wait complete. 2024-04-25T11:37:40.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:37:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.615+0000 7f553d000640 1 Processor -- start 2024-04-25T11:37:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.615+0000 7f553d000640 1 -- start start 2024-04-25T11:37:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f553d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5538158850 0x7f5538070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f553d000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f553806d5d0 0x7f553806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f553d000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553806df20 con 0x7f553815e780 2024-04-25T11:37:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f553d000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5538158c60 con 0x7f5538158850 2024-04-25T11:37:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f553d000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f553815ef90 con 0x7f553806d5d0 2024-04-25T11:37:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f5535e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5538158850 0x7f5538070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f5535e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5538158850 0x7f5538070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50256/0 (socket says 172.21.15.19:50256) 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.618+0000 7f5535e00640 1 -- 172.21.15.19:0/4210436235 learned_addr learned my addr 172.21.15.19:0/4210436235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.619+0000 7f5536800640 1 --2- 172.21.15.19:0/4210436235 >> v2:172.21.15.169:3300/0 conn(0x7f553806d5d0 0x7f553806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.619+0000 7f5535e00640 1 -- 172.21.15.19:0/4210436235 >> v2:172.21.15.169:3300/0 conn(0x7f553806d5d0 msgr2=0x7f553806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.619+0000 7f5535e00640 1 --2- 172.21.15.19:0/4210436235 >> v2:172.21.15.169:3300/0 conn(0x7f553806d5d0 0x7f553806d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.619+0000 7f5535e00640 1 -- 172.21.15.19:0/4210436235 >> v1:172.21.15.169:6789/0 conn(0x7f553815e780 legacy=0x7f553815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.619+0000 7f5535e00640 1 -- 172.21.15.19:0/4210436235 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553815f270 con 0x7f5538158850 2024-04-25T11:37:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.619+0000 7f5536800640 1 --2- 172.21.15.19:0/4210436235 >> v2:172.21.15.169:3300/0 conn(0x7f553806d5d0 0x7f553806d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.620+0000 7f5535e00640 1 --2- 172.21.15.19:0/4210436235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538070820 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f55380d1930 tx=0x7f552802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc022497f15e7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.620+0000 7f5534a00640 1 -- 172.21.15.19:0/4210436235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528037070 con 0x7f5538158850 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.620+0000 7f5534a00640 1 -- 172.21.15.19:0/4210436235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5528032440 con 0x7f5538158850 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.620+0000 7f5534a00640 1 -- 172.21.15.19:0/4210436235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528032810 con 0x7f5538158850 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 -- 172.21.15.19:0/4210436235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 msgr2=0x7f5538070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 --2- 172.21.15.19:0/4210436235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538070820 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f55380d1930 tx=0x7f552802a270 comp rx=0 tx=0).stop 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 -- 172.21.15.19:0/4210436235 shutdown_connections 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 --2- 172.21.15.19:0/4210436235 >> v2:172.21.15.169:3300/0 conn(0x7f553806d5d0 0x7f553806d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 --2- 172.21.15.19:0/4210436235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538070820 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 -- 172.21.15.19:0/4210436235 >> 172.21.15.19:0/4210436235 conn(0x7f55380730b0 msgr2=0x7f55380734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 -- 172.21.15.19:0/4210436235 shutdown_connections 2024-04-25T11:37:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.621+0000 7f553d000640 1 -- 172.21.15.19:0/4210436235 wait complete. 2024-04-25T11:37:40.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.624+0000 7f553d000640 1 Processor -- start 2024-04-25T11:37:40.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.624+0000 7f553d000640 1 -- start start 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.625+0000 7f553d000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f553806d5d0 0x7f55380cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.625+0000 7f553d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f553d000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f553815e780 0x7f55381e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f553d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5538160010 con 0x7f5538158850 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f553d000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f553806f040 con 0x7f553815e780 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f553d000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f553815eb90 con 0x7f553806d5d0 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50272/0 (socket says 172.21.15.19:50272) 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 -- 172.21.15.19:0/3520972061 learned_addr learned my addr 172.21.15.19:0/3520972061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:40.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535e00640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f553806d5d0 0x7f55380cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5536800640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f553815e780 0x7f55381e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 -- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f553806d5d0 msgr2=0x7f55380cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f553806d5d0 0x7f55380cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 -- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f553815e780 msgr2=0x7f55381e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f553815e780 0x7f55381e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5535400640 1 -- 172.21.15.19:0/3520972061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55380ccf70 con 0x7f5538158850 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.626+0000 7f5536800640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f553815e780 0x7f55381e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.627+0000 7f5535400640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538148480 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f552400c3c0 tx=0x7f552400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.627+0000 7f552e000640 1 -- 172.21.15.19:0/3520972061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240052d0 con 0x7f5538158850 2024-04-25T11:37:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.627+0000 7f552e000640 1 -- 172.21.15.19:0/3520972061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5524005490 con 0x7f5538158850 2024-04-25T11:37:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.627+0000 7f552e000640 1 -- 172.21.15.19:0/3520972061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524007260 con 0x7f5538158850 2024-04-25T11:37:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.627+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55380cd250 con 0x7f5538158850 2024-04-25T11:37:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.627+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55381598a0 con 0x7f5538158850 2024-04-25T11:37:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.628+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55380761c0 con 0x7f5538158850 2024-04-25T11:37:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.630+0000 7f552e000640 1 -- 172.21.15.19:0/3520972061 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5524007400 con 0x7f5538158850 2024-04-25T11:37:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.631+0000 7f552e000640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55180744e0 0x7f55180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.631+0000 7f552e000640 1 -- 172.21.15.19:0/3520972061 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f5524094b10 con 0x7f5538158850 2024-04-25T11:37:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.632+0000 7f5535e00640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55180744e0 0x7f55180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.633+0000 7f5535e00640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55180744e0 0x7f55180769f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5528034000 tx=0x7f552802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.636+0000 7f552e000640 1 -- 172.21.15.19:0/3520972061 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f553806df20 con 0x7f5538158850 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.851+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55380008d0 con 0x7f5538158850 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.852+0000 7f552e000640 1 -- 172.21.15.19:0/3520972061 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7f5524065e20 con 0x7f5538158850 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55180744e0 msgr2=0x7f55180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55180744e0 0x7f55180769f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5528034000 tx=0x7f552802e000 comp rx=0 tx=0).stop 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 msgr2=0x7f5538148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538148480 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f552400c3c0 tx=0x7f552400c880 comp rx=0 tx=0).stop 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 shutdown_connections 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f55180744e0 0x7f55180769f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f553815e780 0x7f55381e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5538158850 0x7f5538148480 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 --2- 172.21.15.19:0/3520972061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f553806d5d0 0x7f55380cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 >> 172.21.15.19:0/3520972061 conn(0x7f55380730b0 msgr2=0x7f55380cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 shutdown_connections 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:40.855+0000 7f553d000640 1 -- 172.21.15.19:0/3520972061 wait complete. 2024-04-25T11:37:40.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.166+0000 7ff9ff000640 1 Processor -- start 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.166+0000 7ff9ff000640 1 -- start start 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9ff000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80d0950 0x7ff9f80d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9ff000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9f80b90d0 0x7ff9f80b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9ff000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9f80d12a0 con 0x7ff9f80d23e0 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9ff000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f80b83f0 con 0x7ff9f80b90d0 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9ff000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f80d1530 con 0x7ff9f80d0950 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9fe600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9f80b90d0 0x7ff9f80b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9fe600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9f80b90d0 0x7ff9f80b94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50274/0 (socket says 172.21.15.19:50274) 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.169+0000 7ff9fe600640 1 -- 172.21.15.19:0/3469619655 learned_addr learned my addr 172.21.15.19:0/3469619655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.170+0000 7ff9fdc00640 1 --2- 172.21.15.19:0/3469619655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80d0950 0x7ff9f80d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.170+0000 7ff9fe600640 1 -- 172.21.15.19:0/3469619655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80d0950 msgr2=0x7ff9f80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.170+0000 7ff9fe600640 1 --2- 172.21.15.19:0/3469619655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80d0950 0x7ff9f80d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.170+0000 7ff9fe600640 1 -- 172.21.15.19:0/3469619655 >> v1:172.21.15.169:6789/0 conn(0x7ff9f80d23e0 legacy=0x7ff9f80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.170+0000 7ff9fe600640 1 -- 172.21.15.19:0/3469619655 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8163cb0 con 0x7ff9f80b90d0 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.171+0000 7ff9fe600640 1 --2- 172.21.15.19:0/3469619655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80b90d0 0x7ff9f80b94e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009e50 tx=0x7ff9f002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db131e5f1c151750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.171+0000 7ff9f7e00640 1 -- 172.21.15.19:0/3469619655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0036070 con 0x7ff9f80b90d0 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.171+0000 7ff9f7e00640 1 -- 172.21.15.19:0/3469619655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f0031600 con 0x7ff9f80b90d0 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.171+0000 7ff9f7e00640 1 -- 172.21.15.19:0/3469619655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00319d0 con 0x7ff9f80b90d0 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 -- 172.21.15.19:0/3469619655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80b90d0 msgr2=0x7ff9f80b94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 --2- 172.21.15.19:0/3469619655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80b90d0 0x7ff9f80b94e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009e50 tx=0x7ff9f002a630 comp rx=0 tx=0).stop 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 -- 172.21.15.19:0/3469619655 shutdown_connections 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 --2- 172.21.15.19:0/3469619655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80b90d0 0x7ff9f80b94e0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 --2- 172.21.15.19:0/3469619655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80d0950 0x7ff9f80d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 -- 172.21.15.19:0/3469619655 >> 172.21.15.19:0/3469619655 conn(0x7ff9f806de00 msgr2=0x7ff9f806e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 -- 172.21.15.19:0/3469619655 shutdown_connections 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.172+0000 7ff9ff000640 1 -- 172.21.15.19:0/3469619655 wait complete. 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.175+0000 7ff9ff000640 1 Processor -- start 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.175+0000 7ff9ff000640 1 -- start start 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.176+0000 7ff9ff000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80b90d0 0x7ff9f815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.176+0000 7ff9ff000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80d0950 0x7ff9f80ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.176+0000 7ff9ff000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80d23e0 0x7ff9f81eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.176+0000 7ff9ff000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f8164550 con 0x7ff9f80d23e0 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.176+0000 7ff9ff000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f8164270 con 0x7ff9f80d0950 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.176+0000 7ff9ff000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f81638f0 con 0x7ff9f80b90d0 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80d23e0 0x7ff9f81eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80d23e0 0x7ff9f81eb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50286/0 (socket says 172.21.15.19:50286) 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 -- 172.21.15.19:0/4244319315 learned_addr learned my addr 172.21.15.19:0/4244319315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fdc00640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80b90d0 0x7ff9f815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fd200640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80d0950 0x7ff9f80ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 -- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80b90d0 msgr2=0x7ff9f815cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80b90d0 0x7ff9f815cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 -- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80d0950 msgr2=0x7ff9f80ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80d0950 0x7ff9f80ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fe600640 1 -- 172.21.15.19:0/4244319315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f80cd810 con 0x7ff9f80d23e0 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.177+0000 7ff9fdc00640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80b90d0 0x7ff9f815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.178+0000 7ff9fe600640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80d23e0 0x7ff9f81eb5b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009f70 tx=0x7ff9f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.178+0000 7ff9f5600640 1 -- 172.21.15.19:0/4244319315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0036050 con 0x7ff9f80d23e0 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.178+0000 7ff9f5600640 1 -- 172.21.15.19:0/4244319315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f0042070 con 0x7ff9f80d23e0 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.178+0000 7ff9f5600640 1 -- 172.21.15.19:0/4244319315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0009510 con 0x7ff9f80d23e0 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.178+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f80cdaf0 con 0x7ff9f80d23e0 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.178+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f80cc4f0 con 0x7ff9f80d23e0 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.179+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f80cd580 con 0x7ff9f80d23e0 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.181+0000 7ff9f5600640 1 -- 172.21.15.19:0/4244319315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9f0007bd0 con 0x7ff9f80d23e0 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.182+0000 7ff9f5600640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9d80744b0 0x7ff9d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.182+0000 7ff9f5600640 1 -- 172.21.15.19:0/4244319315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7ff9f00b6470 con 0x7ff9f80d23e0 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.182+0000 7ff9fdc00640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9d80744b0 0x7ff9d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.184+0000 7ff9fdc00640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9d80744b0 0x7ff9d80769c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8006fd0 tx=0x7ff9e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.187+0000 7ff9f5600640 1 -- 172.21.15.19:0/4244319315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9f0080870 con 0x7ff9f80d23e0 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.388+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9f8076c60 con 0x7ff9f80d23e0 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.389+0000 7ff9f5600640 1 -- 172.21.15.19:0/4244319315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7ff9f0080170 con 0x7ff9f80d23e0 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9d80744b0 msgr2=0x7ff9d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9d80744b0 0x7ff9d80769c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8006fd0 tx=0x7ff9e8008040 comp rx=0 tx=0).stop 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80d23e0 msgr2=0x7ff9f81eb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80d23e0 0x7ff9f81eb5b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009f70 tx=0x7ff9f0004a40 comp rx=0 tx=0).stop 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 shutdown_connections 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9d80744b0 0x7ff9d80769c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f80d0950 0x7ff9f80ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f80d23e0 0x7ff9f81eb5b0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 --2- 172.21.15.19:0/4244319315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f80b90d0 0x7ff9f815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 >> 172.21.15.19:0/4244319315 conn(0x7ff9f806de00 msgr2=0x7ff9f8161c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 shutdown_connections 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.392+0000 7ff9ff000640 1 -- 172.21.15.19:0/4244319315 wait complete. 2024-04-25T11:37:41.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.705+0000 7f4efb200640 1 Processor -- start 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.705+0000 7f4efb200640 1 -- start start 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.707+0000 7f4efb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4efc0b4ff0 0x7f4efc1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.707+0000 7f4efb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc08cd60 0x7f4efc08d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.707+0000 7f4efb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4efc161720 con 0x7f4efc163d20 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.707+0000 7f4efb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4efc06e9d0 con 0x7f4efc0b4ff0 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.708+0000 7f4efb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4efc08d6b0 con 0x7f4efc08cd60 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.708+0000 7f4ef9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4efc0b4ff0 0x7f4efc1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.708+0000 7f4ef9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4efc0b4ff0 0x7f4efc1611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50296/0 (socket says 172.21.15.19:50296) 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.708+0000 7f4ef9e00640 1 -- 172.21.15.19:0/3192063615 learned_addr learned my addr 172.21.15.19:0/3192063615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.708+0000 7f4ef9400640 1 --2- 172.21.15.19:0/3192063615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc08cd60 0x7f4efc08d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.709+0000 7f4ef9e00640 1 -- 172.21.15.19:0/3192063615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc08cd60 msgr2=0x7f4efc08d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.709+0000 7f4ef9e00640 1 --2- 172.21.15.19:0/3192063615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc08cd60 0x7f4efc08d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.709+0000 7f4ef9e00640 1 -- 172.21.15.19:0/3192063615 >> v1:172.21.15.169:6789/0 conn(0x7f4efc163d20 legacy=0x7f4efc164130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.709+0000 7f4ef9e00640 1 -- 172.21.15.19:0/3192063615 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc06ddc0 con 0x7f4efc0b4ff0 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.710+0000 7f4ef9e00640 1 --2- 172.21.15.19:0/3192063615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc0b4ff0 0x7f4efc1611e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009e40 tx=0x7f4eec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9d2448c4feea9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.711+0000 7f4ef8a00640 1 -- 172.21.15.19:0/3192063615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec036070 con 0x7f4efc0b4ff0 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.711+0000 7f4ef8a00640 1 -- 172.21.15.19:0/3192063615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eec031600 con 0x7f4efc0b4ff0 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.711+0000 7f4ef8a00640 1 -- 172.21.15.19:0/3192063615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0319d0 con 0x7f4efc0b4ff0 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.711+0000 7f4efb200640 1 -- 172.21.15.19:0/3192063615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc0b4ff0 msgr2=0x7f4efc1611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.711+0000 7f4efb200640 1 --2- 172.21.15.19:0/3192063615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc0b4ff0 0x7f4efc1611e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009e40 tx=0x7f4eec02a680 comp rx=0 tx=0).stop 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.712+0000 7f4efb200640 1 -- 172.21.15.19:0/3192063615 shutdown_connections 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.712+0000 7f4efb200640 1 --2- 172.21.15.19:0/3192063615 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc08cd60 0x7f4efc08d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.712+0000 7f4efb200640 1 --2- 172.21.15.19:0/3192063615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc0b4ff0 0x7f4efc1611e0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.712+0000 7f4efb200640 1 -- 172.21.15.19:0/3192063615 >> 172.21.15.19:0/3192063615 conn(0x7f4efc0b5460 msgr2=0x7f4efc0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.712+0000 7f4efb200640 1 -- 172.21.15.19:0/3192063615 shutdown_connections 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.712+0000 7f4efb200640 1 -- 172.21.15.19:0/3192063615 wait complete. 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.714+0000 7f4efb200640 1 Processor -- start 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.714+0000 7f4efb200640 1 -- start start 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4efb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc08cd60 0x7f4efc150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4efb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0b4ff0 0x7f4efc1f9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4efb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc163d20 0x7f4efc0d2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4efb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc06e660 con 0x7f4efc08cd60 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4efb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc06e380 con 0x7f4efc163d20 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4efb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4efc0b9180 con 0x7f4efc0b4ff0 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4ef9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc08cd60 0x7f4efc150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4ef9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc08cd60 0x7f4efc150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50300/0 (socket says 172.21.15.19:50300) 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.716+0000 7f4ef9e00640 1 -- 172.21.15.19:0/4225085938 learned_addr learned my addr 172.21.15.19:0/4225085938 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4ef9400640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0b4ff0 0x7f4efc1f9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4efa800640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc163d20 0x7f4efc0d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4ef9e00640 1 -- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0b4ff0 msgr2=0x7f4efc1f9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4ef9e00640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0b4ff0 0x7f4efc1f9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4ef9e00640 1 -- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc163d20 msgr2=0x7f4efc0d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4ef9e00640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc163d20 0x7f4efc0d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4ef9e00640 1 -- 172.21.15.19:0/4225085938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc1fac40 con 0x7f4efc08cd60 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.717+0000 7f4ef9e00640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc08cd60 0x7f4efc150dc0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009f60 tx=0x7f4eec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.718+0000 7f4ef2000640 1 -- 172.21.15.19:0/4225085938 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec03f020 con 0x7f4efc08cd60 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.718+0000 7f4ef2000640 1 -- 172.21.15.19:0/4225085938 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eec037040 con 0x7f4efc08cd60 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.718+0000 7f4ef2000640 1 -- 172.21.15.19:0/4225085938 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec036070 con 0x7f4efc08cd60 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.718+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4efc09f9d0 con 0x7f4efc08cd60 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.718+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4efc1f6d40 con 0x7f4efc08cd60 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.719+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4efc050740 con 0x7f4efc08cd60 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.720+0000 7f4ef2000640 1 -- 172.21.15.19:0/4225085938 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4eec042020 con 0x7f4efc08cd60 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.721+0000 7f4ef2000640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ed80744b0 0x7f4ed80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.721+0000 7f4ef2000640 1 -- 172.21.15.19:0/4225085938 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f4eec0b5c10 con 0x7f4efc08cd60 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.722+0000 7f4ef9400640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ed80744b0 0x7f4ed80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.724+0000 7f4ef9400640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ed80744b0 0x7f4ed80769c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4006fd0 tx=0x7f4ee4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.726+0000 7f4ef2000640 1 -- 172.21.15.19:0/4225085938 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4eec049030 con 0x7f4efc08cd60 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.948+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4efc0b6370 con 0x7f4efc08cd60 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.949+0000 7f4ef2000640 1 -- 172.21.15.19:0/4225085938 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 24 entries v104) ==== 81+0+1454 (secure 0 0 0) 0x7f4eec07ff90 con 0x7f4efc08cd60 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi019.stderr:listed 24 entries 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ed80744b0 msgr2=0x7f4ed80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ed80744b0 0x7f4ed80769c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4006fd0 tx=0x7f4ee4008040 comp rx=0 tx=0).stop 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc08cd60 msgr2=0x7f4efc150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc08cd60 0x7f4efc150dc0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009f60 tx=0x7f4eec004a40 comp rx=0 tx=0).stop 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 shutdown_connections 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ed80744b0 0x7f4ed80769c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc163d20 0x7f4efc0d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0b4ff0 0x7f4efc1f9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 --2- 172.21.15.19:0/4225085938 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4efc08cd60 0x7f4efc150dc0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 >> 172.21.15.19:0/4225085938 conn(0x7f4efc0b5460 msgr2=0x7f4efc15eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 shutdown_connections 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:41.952+0000 7f4efb200640 1 -- 172.21.15.19:0/4225085938 wait complete. 2024-04-25T11:37:41.968 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:41.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:37:42.081 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:42.081 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:37:42.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:37:42.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.428+0000 7f357c800640 1 Processor -- start 2024-04-25T11:37:42.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.428+0000 7f357c800640 1 -- start start 2024-04-25T11:37:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.430+0000 7f357c800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f35741443c0 0x7f357406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.430+0000 7f357c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 0x7f35740cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.430+0000 7f357c800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35740cff30 con 0x7f35740cfb20 2024-04-25T11:37:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.430+0000 7f357c800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f35740708d0 con 0x7f35741443c0 2024-04-25T11:37:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.430+0000 7f357c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f357415f160 con 0x7f3574070370 2024-04-25T11:37:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.431+0000 7f357a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f35740cfb20 0x7f35740ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46992/0 (socket says 172.21.15.19:46992) 2024-04-25T11:37:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.431+0000 7f357a800640 1 -- 172.21.15.19:0/2080253578 learned_addr learned my addr 172.21.15.19:0/2080253578 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:42.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.431+0000 7f3579400640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 0x7f35740cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.431+0000 7f3579e00640 1 --2- 172.21.15.19:0/2080253578 >> v2:172.21.15.169:3300/0 conn(0x7f35741443c0 0x7f357406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.431+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665703042 0 0) 0x7f3570001110 con 0x7f35740cfb20 2024-04-25T11:37:42.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.431+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3560003520 con 0x7f35740cfb20 2024-04-25T11:37:42.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1943624226 0 0) 0x7f3560003520 con 0x7f35740cfb20 2024-04-25T11:37:42.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 msgr2=0x7f35740cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 0x7f35740cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 >> v2:172.21.15.169:3300/0 conn(0x7f35741443c0 msgr2=0x7f357406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 --2- 172.21.15.19:0/2080253578 >> v2:172.21.15.169:3300/0 conn(0x7f35741443c0 0x7f357406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357415f5f0 con 0x7f35740cfb20 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f3570001840 con 0x7f35740cfb20 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 >> v1:172.21.15.19:6789/0 conn(0x7f35740cfb20 legacy=0x7f35740ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 0x7f35600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3560003f80 0x7f3560004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560004d40 0x7f3560005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35740708d0 con 0x7f3560003f80 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f357415f160 con 0x7f3560004d40 2024-04-25T11:37:42.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f356000ba70 con 0x7f3574070370 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.432+0000 7f3579400640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3560003f80 0x7f3560004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f3579e00640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 0x7f35600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f357a800640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560004d40 0x7f3560005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f3579400640 1 -- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 msgr2=0x7f35600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f3579400640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 0x7f35600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f3579400640 1 -- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560004d40 msgr2=0x7f3560005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f3579400640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3560004d40 0x7f3560005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f3579400640 1 -- 172.21.15.19:0/2080253578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3560003520 con 0x7f3560003f80 2024-04-25T11:37:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.433+0000 7f3579e00640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3574070370 0x7f35600039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.434+0000 7f3579400640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3560003f80 0x7f3560004800 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3564009a90 tx=0x7f3560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e4cdeccb869233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.434+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564036070 con 0x7f3560003f80 2024-04-25T11:37:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.434+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3564031400 con 0x7f3560003f80 2024-04-25T11:37:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.434+0000 7f3578a00640 1 -- 172.21.15.19:0/2080253578 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564009690 con 0x7f3560003f80 2024-04-25T11:37:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.435+0000 7f357c800640 1 -- 172.21.15.19:0/2080253578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3560003f80 msgr2=0x7f3560004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.435+0000 7f357c800640 1 --2- 172.21.15.19:0/2080253578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3560003f80 0x7f3560004800 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3564009a90 tx=0x7f3560005830 comp rx=0 tx=0).stop 2024-04-25T11:37:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.435+0000 7f357a800640 1 -- 172.21.15.19:0/2080253578 reap_dead start 2024-04-25T11:37:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.435+0000 7f357c800640 1 -- 172.21.15.19:0/2080253578 shutdown_connections 2024-04-25T11:37:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.435+0000 7f357c800640 1 -- 172.21.15.19:0/2080253578 >> 172.21.15.19:0/2080253578 conn(0x7f35740d1ae0 msgr2=0x7f35740d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.435+0000 7f357c800640 1 -- 172.21.15.19:0/2080253578 shutdown_connections 2024-04-25T11:37:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.435+0000 7f357c800640 1 -- 172.21.15.19:0/2080253578 wait complete. 2024-04-25T11:37:42.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.438+0000 7f357c800640 1 Processor -- start 2024-04-25T11:37:42.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.438+0000 7f357c800640 1 -- start start 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.439+0000 7f357c800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3560003f80 0x7f357415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.439+0000 7f357c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35740cea50 0x7f35740b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.439+0000 7f357c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35740b6410 0x7f357413cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.439+0000 7f357c800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f357415edf0 con 0x7f35740cea50 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.439+0000 7f357c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f357415eb10 con 0x7f35740b6410 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.439+0000 7f357c800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f357406d5d0 con 0x7f3560003f80 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35740cea50 0x7f35740b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35740cea50 0x7f35740b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50316/0 (socket says 172.21.15.19:50316) 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 -- 172.21.15.19:0/1638150855 learned_addr learned my addr 172.21.15.19:0/1638150855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579e00640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3560003f80 0x7f357415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f357a800640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35740b6410 0x7f357413cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 -- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3560003f80 msgr2=0x7f357415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3560003f80 0x7f357415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 -- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35740b6410 msgr2=0x7f357413cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35740b6410 0x7f357413cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.440+0000 7f3579400640 1 -- 172.21.15.19:0/1638150855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35740b6820 con 0x7f35740cea50 2024-04-25T11:37:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.441+0000 7f3579400640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35740cea50 0x7f35740b5ed0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3564009bb0 tx=0x7f3564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.441+0000 7f356a000640 1 -- 172.21.15.19:0/1638150855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564040020 con 0x7f35740cea50 2024-04-25T11:37:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.441+0000 7f356a000640 1 -- 172.21.15.19:0/1638150855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3564003c40 con 0x7f35740cea50 2024-04-25T11:37:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.441+0000 7f356a000640 1 -- 172.21.15.19:0/1638150855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564036040 con 0x7f35740cea50 2024-04-25T11:37:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.441+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357413f140 con 0x7f35740cea50 2024-04-25T11:37:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.441+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357413f700 con 0x7f35740cea50 2024-04-25T11:37:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.442+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357406f950 con 0x7f35740cea50 2024-04-25T11:37:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.444+0000 7f356a000640 1 -- 172.21.15.19:0/1638150855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3564043020 con 0x7f35740cea50 2024-04-25T11:37:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.445+0000 7f356a000640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35580744b0 0x7f35580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.445+0000 7f356a000640 1 -- 172.21.15.19:0/1638150855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6741+0+0 (secure 0 0 0) 0x7f35640b5970 con 0x7f35740cea50 2024-04-25T11:37:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.445+0000 7f3579e00640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35580744b0 0x7f35580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.447+0000 7f3579e00640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35580744b0 0x7f35580769c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f35740d1860 tx=0x7f356c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.450+0000 7f356a000640 1 -- 172.21.15.19:0/1638150855 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f356404a070 con 0x7f35740cea50 2024-04-25T11:37:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.700+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3574143d60 con 0x7f35740cea50 2024-04-25T11:37:42.690 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:42.702+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:37:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.706+0000 7f356a000640 1 -- 172.21.15.19:0/1638150855 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v174) ==== 161+0+0 (secure 0 0 0) 0x7f35740b6820 con 0x7f35740cea50 2024-04-25T11:37:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35580744b0 msgr2=0x7f35580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35580744b0 0x7f35580769c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f35740d1860 tx=0x7f356c00b040 comp rx=0 tx=0).stop 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35740cea50 msgr2=0x7f35740b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35740cea50 0x7f35740b5ed0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3564009bb0 tx=0x7f3564004730 comp rx=0 tx=0).stop 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 shutdown_connections 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35580744b0 0x7f35580769c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35740b6410 0x7f357413cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35740cea50 0x7f35740b5ed0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 --2- 172.21.15.19:0/1638150855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3560003f80 0x7f357415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.710+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 >> 172.21.15.19:0/1638150855 conn(0x7f35740d1ae0 msgr2=0x7f35740d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.711+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 shutdown_connections 2024-04-25T11:37:42.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:42.711+0000 7f357c800640 1 -- 172.21.15.19:0/1638150855 wait complete. 2024-04-25T11:37:42.712 INFO:tasks.ceph.mds.a.smithi019.stderr: -780> 2024-04-25T11:37:34.835+0000 7efc5d200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:37:42.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:37:42.864 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:37:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.003+0000 7f60c5400640 1 Processor -- start 2024-04-25T11:37:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.004+0000 7f60c5400640 1 -- start start 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.005+0000 7f60c5400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60c00cf2f0 0x7f60c00b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.005+0000 7f60c5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c01608d0 0x7f60c0160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.005+0000 7f60c5400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60c0161220 con 0x7f60c015a810 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.005+0000 7f60c5400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f60c006f870 con 0x7f60c00cf2f0 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60c5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60c00b74d0 con 0x7f60c01608d0 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60bfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60c00cf2f0 0x7f60c00b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60bfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60c00cf2f0 0x7f60c00b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50350/0 (socket says 172.21.15.19:50350) 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60bfe00640 1 -- 172.21.15.19:0/3463395621 learned_addr learned my addr 172.21.15.19:0/3463395621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60bfe00640 1 -- 172.21.15.19:0/3463395621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c01608d0 msgr2=0x7f60c0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60c4a00640 1 --2- 172.21.15.19:0/3463395621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c01608d0 0x7f60c0160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60bfe00640 1 --2- 172.21.15.19:0/3463395621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c01608d0 0x7f60c0160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60bfe00640 1 -- 172.21.15.19:0/3463395621 >> v1:172.21.15.169:6789/0 conn(0x7f60c015a810 legacy=0x7f60c00b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60bfe00640 1 -- 172.21.15.19:0/3463395621 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c00b77b0 con 0x7f60c00cf2f0 2024-04-25T11:37:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.006+0000 7f60c4a00640 1 --2- 172.21.15.19:0/3463395621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c01608d0 0x7f60c0160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.007+0000 7f60bfe00640 1 --2- 172.21.15.19:0/3463395621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c00cf2f0 0x7f60c00b6e10 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f60c00d26d0 tx=0x7f60b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c1e60364c0fee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.008+0000 7f60bea00640 1 -- 172.21.15.19:0/3463395621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0037070 con 0x7f60c00cf2f0 2024-04-25T11:37:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.008+0000 7f60bea00640 1 -- 172.21.15.19:0/3463395621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b0032440 con 0x7f60c00cf2f0 2024-04-25T11:37:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.008+0000 7f60bea00640 1 -- 172.21.15.19:0/3463395621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0032810 con 0x7f60c00cf2f0 2024-04-25T11:37:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 -- 172.21.15.19:0/3463395621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c00cf2f0 msgr2=0x7f60c00b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 --2- 172.21.15.19:0/3463395621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c00cf2f0 0x7f60c00b6e10 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f60c00d26d0 tx=0x7f60b002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 -- 172.21.15.19:0/3463395621 shutdown_connections 2024-04-25T11:37:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 --2- 172.21.15.19:0/3463395621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c01608d0 0x7f60c0160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 --2- 172.21.15.19:0/3463395621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c00cf2f0 0x7f60c00b6e10 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 -- 172.21.15.19:0/3463395621 >> 172.21.15.19:0/3463395621 conn(0x7f60c00d1720 msgr2=0x7f60c00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 -- 172.21.15.19:0/3463395621 shutdown_connections 2024-04-25T11:37:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.009+0000 7f60c5400640 1 -- 172.21.15.19:0/3463395621 wait complete. 2024-04-25T11:37:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.011+0000 7f60c5400640 1 Processor -- start 2024-04-25T11:37:43.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.011+0000 7f60c5400640 1 -- start start 2024-04-25T11:37:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c00cf2f0 0x7f60c015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c015a810 0x7f60c015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c01608d0 0x7f60c015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c5400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60c015ac20 con 0x7f60c01608d0 2024-04-25T11:37:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f60c006f500 con 0x7f60c00cf2f0 2024-04-25T11:37:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f60c006fd00 con 0x7f60c015a810 2024-04-25T11:37:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c01608d0 0x7f60c015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c01608d0 0x7f60c015f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50354/0 (socket says 172.21.15.19:50354) 2024-04-25T11:37:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.013+0000 7f60c4a00640 1 -- 172.21.15.19:0/2302426523 learned_addr learned my addr 172.21.15.19:0/2302426523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60bfe00640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c00cf2f0 0x7f60c015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60bf400640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c015a810 0x7f60c015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60c4a00640 1 -- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c015a810 msgr2=0x7f60c015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60c4a00640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c015a810 0x7f60c015ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60c4a00640 1 -- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c00cf2f0 msgr2=0x7f60c015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60c4a00640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c00cf2f0 0x7f60c015ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60c4a00640 1 -- 172.21.15.19:0/2302426523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c015bad0 con 0x7f60c01608d0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.014+0000 7f60c4a00640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c01608d0 0x7f60c015f390 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ccf0 tx=0x7f60b800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.015+0000 7f60b7e00640 1 -- 172.21.15.19:0/2302426523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b800f680 con 0x7f60c01608d0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.015+0000 7f60b7e00640 1 -- 172.21.15.19:0/2302426523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b800f840 con 0x7f60c01608d0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.015+0000 7f60b7e00640 1 -- 172.21.15.19:0/2302426523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b800fc30 con 0x7f60c01608d0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.015+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c015bdb0 con 0x7f60c01608d0 2024-04-25T11:37:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.015+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c015c370 con 0x7f60c01608d0 2024-04-25T11:37:43.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.016+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c0050740 con 0x7f60c01608d0 2024-04-25T11:37:43.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.018+0000 7f60b7e00640 1 -- 172.21.15.19:0/2302426523 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f60b800e3e0 con 0x7f60c01608d0 2024-04-25T11:37:43.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.019+0000 7f60b7e00640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60a00744b0 0x7f60a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.019+0000 7f60b7e00640 1 -- 172.21.15.19:0/2302426523 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6827+0+0 (secure 0 0 0) 0x7f60b8096420 con 0x7f60c01608d0 2024-04-25T11:37:43.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.019+0000 7f60bfe00640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60a00744b0 0x7f60a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.021+0000 7f60bfe00640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60a00744b0 0x7f60a00769c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f60b0034000 tx=0x7f60b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:43.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.024+0000 7f60b7e00640 1 -- 172.21.15.19:0/2302426523 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60b8014030 con 0x7f60c01608d0 2024-04-25T11:37:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.301+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f60c00d0f10 con 0x7f60c01608d0 2024-04-25T11:37:43.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.710+0000 7f60b7e00640 1 -- 172.21.15.19:0/2302426523 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v175) ==== 107+0+0 (secure 0 0 0) 0x7f60b8060740 con 0x7f60c01608d0 2024-04-25T11:37:43.698 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60a00744b0 msgr2=0x7f60a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60a00744b0 0x7f60a00769c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f60b0034000 tx=0x7f60b002e000 comp rx=0 tx=0).stop 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c01608d0 msgr2=0x7f60c015f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c01608d0 0x7f60c015f390 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f60b800ccf0 tx=0x7f60b800a650 comp rx=0 tx=0).stop 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 shutdown_connections 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60a00744b0 0x7f60a00769c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60c01608d0 0x7f60c015f390 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60c015a810 0x7f60c015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 --2- 172.21.15.19:0/2302426523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60c00cf2f0 0x7f60c015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 >> 172.21.15.19:0/2302426523 conn(0x7f60c00d1720 msgr2=0x7f60c00d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 shutdown_connections 2024-04-25T11:37:43.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.713+0000 7f60c5400640 1 -- 172.21.15.19:0/2302426523 wait complete. 2024-04-25T11:37:43.731 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:37:43.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.907+0000 7f741420bf40 1 Processor -- start 2024-04-25T11:37:43.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.908+0000 7f741420bf40 1 -- start start 2024-04-25T11:37:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f741420bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e801f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f741420bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e7f7b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f741420bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 0x5561e8019cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f741420bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561e7ef0d20 con 0x7f73ec0036a0 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f741420bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5561e7ef4d60 con 0x5561e7f79830 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f741420bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5561e7f93110 con 0x5561e8022fe0 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f740c600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e801f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f740c600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e801f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48762/0 (socket says 172.21.15.19:48762) 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.909+0000 7f740c600640 1 -- 172.21.15.19:0/666265434 learned_addr learned my addr 172.21.15.19:0/666265434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740bc00640 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e7f7b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740c600640 1 -- 172.21.15.19:0/666265434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 msgr2=0x5561e8019cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740c600640 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 0x5561e8019cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740c600640 1 -- 172.21.15.19:0/666265434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 msgr2=0x5561e7f7b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740c600640 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e7f7b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740c600640 1 -- 172.21.15.19:0/666265434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e8020690 con 0x7f73ec0036a0 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740c600640 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e801f070 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00b560 tx=0x7f73fc00ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afe1c83b60de2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740a800640 1 -- 172.21.15.19:0/666265434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc015070 con 0x7f73ec0036a0 2024-04-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740a800640 1 -- 172.21.15.19:0/666265434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc00d7c0 con 0x7f73ec0036a0 2024-04-25T11:37:43.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.910+0000 7f740a800640 1 -- 172.21.15.19:0/666265434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc008070 con 0x7f73ec0036a0 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 -- 172.21.15.19:0/666265434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 msgr2=0x5561e801f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e801f070 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00b560 tx=0x7f73fc00ba20 comp rx=0 tx=0).stop 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 -- 172.21.15.19:0/666265434 shutdown_connections 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e7f7b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 0x5561e8019cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 --2- 172.21.15.19:0/666265434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e801f070 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 -- 172.21.15.19:0/666265434 >> 172.21.15.19:0/666265434 conn(0x5561e7e6cac0 msgr2=0x5561e7f7a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 -- 172.21.15.19:0/666265434 shutdown_connections 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.911+0000 7f741420bf40 1 -- 172.21.15.19:0/666265434 wait complete. 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.912+0000 7f741420bf40 1 Processor -- start 2024-04-25T11:37:43.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.912+0000 7f741420bf40 1 -- start start 2024-04-25T11:37:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f741420bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e800f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f741420bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 0x5561e80188d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f741420bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e8018e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f741420bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561e801e900 con 0x7f73ec0036a0 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f741420bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5561e7f78aa0 con 0x5561e7f79830 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f741420bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5561e801c310 con 0x5561e8022fe0 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e8018e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e8018e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48774/0 (socket says 172.21.15.19:48774) 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 learned_addr learned my addr 172.21.15.19:0/2414922131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e800f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 msgr2=0x5561e80188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740bc00640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 0x5561e80188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561e8022fe0 0x5561e80188d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 msgr2=0x5561e800f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e800f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e80ba370 con 0x7f73ec0036a0 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.914+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561e7f79830 0x5561e800f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.915+0000 7f740d000640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e8018e10 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f7404045de0 tx=0x7f74040458b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.915+0000 7f7409400640 1 -- 172.21.15.19:0/2414922131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404059040 con 0x7f73ec0036a0 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.915+0000 7f7409400640 1 -- 172.21.15.19:0/2414922131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7404064070 con 0x7f73ec0036a0 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.915+0000 7f7409400640 1 -- 172.21.15.19:0/2414922131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404001c20 con 0x7f73ec0036a0 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.915+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5561e80ba650 con 0x7f73ec0036a0 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.915+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e80bac10 con 0x7f73ec0036a0 2024-04-25T11:37:43.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.917+0000 7f7409400640 1 -- 172.21.15.19:0/2414922131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7404069460 con 0x7f73ec0036a0 2024-04-25T11:37:43.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.918+0000 7f7409400640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73e8072ec0 0x7f73e80753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.919+0000 7f7409400640 1 -- 172.21.15.19:0/2414922131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6827+0+0 (secure 0 0 0) 0x7f74040d7700 con 0x7f73ec0036a0 2024-04-25T11:37:43.907 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:37:43.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.919+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5561e7f7c260 0x5561e801b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.919+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e801ba20 con 0x5561e7f7c260 2024-04-25T11:37:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.919+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73e8072ec0 0x7f73e80753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.920+0000 7f740bc00640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5561e7f7c260 0x5561e801b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.920+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73e8072ec0 0x7f73e80753d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f73fc013040 tx=0x7f73fc005000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:43.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.921+0000 7f740bc00640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5561e7f7c260 0x5561e801b4e0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:43.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.930+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v105'381 uv355 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.930+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e801bdc0 con 0x5561e7f7c260 2024-04-25T11:37:43.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.933+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v105'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.933+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4140 con 0x5561e7f7c260 2024-04-25T11:37:43.924 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:37:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.936+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v105'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.936+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c54f0 con 0x5561e7f7c260 2024-04-25T11:37:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.939+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.939+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5890 con 0x5561e7f7c260 2024-04-25T11:37:43.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.943+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.943+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c44e0 con 0x5561e7f7c260 2024-04-25T11:37:43.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.946+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.946+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4880 con 0x5561e7f7c260 2024-04-25T11:37:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.950+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v105'387 uv361 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.950+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80c4c70 con 0x5561e7f7c260 2024-04-25T11:37:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.954+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v105'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.955+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c70 con 0x5561e7f7c260 2024-04-25T11:37:43.946 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:37:43.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.958+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v105'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.958+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.962+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.962+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.965+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.965+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.969+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.969+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.973+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v105'393 uv368 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.973+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.976+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'394 uv394 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.976+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.980+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v105'395 uv395 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.969 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:37:43.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.981+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.984+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'396 uv396 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.985+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.988+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'397 uv397 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.989+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c4c20 con 0x5561e7f7c260 2024-04-25T11:37:43.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.993+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'398 uv398 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.993+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:43.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.997+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'399 uv399 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:43.998+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:43.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.002+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v105'400 uv374 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.002+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:43.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.006+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v105'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.006+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:43.999 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:37:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.011+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v105'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.011+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c6c40 con 0x5561e7f7c260 2024-04-25T11:37:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.014+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.015+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c6c40 con 0x5561e7f7c260 2024-04-25T11:37:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.019+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.019+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.024+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.024+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.029+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v105'406 uv380 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.029+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.034+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v105'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.034+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.038+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v105'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.026 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:37:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.038+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c6c40 con 0x5561e7f7c260 2024-04-25T11:37:44.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.042+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.042+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c6c40 con 0x5561e7f7c260 2024-04-25T11:37:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.047+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.047+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c5c30 con 0x5561e7f7c260 2024-04-25T11:37:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.051+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.052+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5561e80c6140 0x5561e80c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.052+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c8aa0 con 0x5561e80c6140 2024-04-25T11:37:44.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.052+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5561e80c6140 0x5561e80c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.054+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5561e80c6140 0x5561e80c8560 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.064+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'287 uv266 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.064+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80c8e40 con 0x5561e80c6140 2024-04-25T11:37:44.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.068+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'288 uv288 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.068+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c91e0 con 0x5561e80c6140 2024-04-25T11:37:44.060 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:37:44.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.072+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'289 uv289 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.072+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c9580 con 0x5561e80c6140 2024-04-25T11:37:44.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.076+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v105'290 uv290 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.076+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c9920 con 0x5561e80c6140 2024-04-25T11:37:44.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.081+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'291 uv291 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.081+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80c9cc0 con 0x5561e80c6140 2024-04-25T11:37:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.085+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v105'292 uv285 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.085+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80ca060 con 0x5561e80c6140 2024-04-25T11:37:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.089+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v105'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.089+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.081 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:37:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.093+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v105'294 uv294 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.093+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.097+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.097+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.101+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.102+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.105+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v105'297 uv286 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.106+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.110+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v105'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.110+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.102 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:37:44.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.114+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v105'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.114+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ceaf0 con 0x5561e80c6140 2024-04-25T11:37:44.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.119+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v105'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.119+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ceaf0 con 0x5561e80c6140 2024-04-25T11:37:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.123+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v105'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.124+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.128+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v105'302 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.128+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.132+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v105'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.132+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.124 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:37:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.136+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v105'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.136+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cb410 con 0x5561e80c6140 2024-04-25T11:37:44.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.140+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v105'305 uv305 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.140+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cb410 con 0x5561e80c6140 2024-04-25T11:37:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.144+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v105'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.144+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ca400 con 0x5561e80c6140 2024-04-25T11:37:44.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.148+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v105'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80c6140 2024-04-25T11:37:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.148+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5561e80ca910 0x5561e80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.148+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cd390 con 0x5561e80ca910 2024-04-25T11:37:44.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.150+0000 7f740bc00640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5561e80ca910 0x5561e80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.151+0000 7f740bc00640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5561e80ca910 0x5561e80ccd30 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.155+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.155+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80cd760 con 0x5561e80ca910 2024-04-25T11:37:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.159+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v105'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.160+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cd760 con 0x5561e80ca910 2024-04-25T11:37:44.152 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:37:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.163+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v105'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.163+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ce770 con 0x5561e80ca910 2024-04-25T11:37:44.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.167+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.167+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80ce770 con 0x5561e80ca910 2024-04-25T11:37:44.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.171+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.171+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cd760 con 0x5561e80ca910 2024-04-25T11:37:44.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.176+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.176+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cd760 con 0x5561e7f7c260 2024-04-25T11:37:44.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.180+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'62 uv61 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.180+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80cd760 con 0x5561e7f7c260 2024-04-25T11:37:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.184+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v105'63 uv63 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.184+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cd760 con 0x5561e7f7c260 2024-04-25T11:37:44.176 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:37:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.188+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v105'64 uv64 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.188+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80cd760 con 0x5561e7f7c260 2024-04-25T11:37:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.193+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v105'65 uv65 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.193+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5561e80ce0d0 0x5561e80d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.193+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d0a30 con 0x5561e80ce0d0 2024-04-25T11:37:44.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.194+0000 7f740d000640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5561e80ce0d0 0x5561e80d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.195+0000 7f740d000640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5561e80ce0d0 0x5561e80d04f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.199+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.199+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d0dd0 con 0x5561e80ce0d0 2024-04-25T11:37:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.204+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v105'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.204+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d1280 con 0x5561e80ce0d0 2024-04-25T11:37:44.196 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:37:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.208+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v105'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.208+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d2b30 con 0x5561e80ce0d0 2024-04-25T11:37:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.212+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.212+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d2ed0 con 0x5561e80ce0d0 2024-04-25T11:37:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.216+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.216+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d1b20 con 0x5561e80ce0d0 2024-04-25T11:37:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.220+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.221+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5561e80d2420 0x5561e80d5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.221+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d57c0 con 0x5561e80d2420 2024-04-25T11:37:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.221+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5561e80d2420 0x5561e80d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.223+0000 7f740c600640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5561e80d2420 0x5561e80d5280 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.227+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v105'138 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.227+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d57c0 con 0x5561e80d2420 2024-04-25T11:37:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.232+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v105'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.232+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d5840 con 0x5561e80d2420 2024-04-25T11:37:44.224 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:37:44.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.235+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v105'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.236+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d6c70 con 0x5561e80d2420 2024-04-25T11:37:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.239+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v105'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.240+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d6c70 con 0x5561e80d2420 2024-04-25T11:37:44.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.243+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v105'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.243+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d5c60 con 0x5561e80d2420 2024-04-25T11:37:44.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.248+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v105'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.248+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d5c60 con 0x5561e80d2420 2024-04-25T11:37:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.252+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v105'144 uv137 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.252+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d5c60 con 0x5561e80d2420 2024-04-25T11:37:44.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.255+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v105'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.256+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d5c60 con 0x5561e80d2420 2024-04-25T11:37:44.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.260+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v105'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.248 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:37:44.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.260+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d5c60 con 0x5561e80d2420 2024-04-25T11:37:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.264+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v105'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.264+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d5c60 con 0x5561e80d2420 2024-04-25T11:37:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.268+0000 7f740c600640 1 -- 172.21.15.19:0/2414922131 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v105'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73fc018070 con 0x5561e80d2420 2024-04-25T11:37:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.268+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561e80d6170 0x5561e80d8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.268+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d8bf0 con 0x5561e80d6170 2024-04-25T11:37:44.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.269+0000 7f740bc00640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561e80d6170 0x5561e80d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.271+0000 7f740bc00640 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561e80d6170 0x5561e80d8590 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.275+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v105'61 uv60 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.275+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d8bf0 con 0x5561e80d6170 2024-04-25T11:37:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.279+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v105'62 uv62 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.279+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80d6170 2024-04-25T11:37:44.272 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:37:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.284+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v105'63 uv63 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.284+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80d6170 2024-04-25T11:37:44.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.289+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v105'64 uv64 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.289+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80d6170 2024-04-25T11:37:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.293+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v105'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.293+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e7f7c260 2024-04-25T11:37:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.297+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.297+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e7f7c260 2024-04-25T11:37:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.301+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v105'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.302+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e7f7c260 2024-04-25T11:37:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.305+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v105'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.294 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:37:44.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.306+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80da050 con 0x5561e7f7c260 2024-04-25T11:37:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.310+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.310+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80da050 con 0x5561e7f7c260 2024-04-25T11:37:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.313+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.314+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e7f7c260 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.317+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e7f7c260 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.317+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.322+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v105'122 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.322+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.326+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v105'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.326+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.331+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v105'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.331+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.336+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v105'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.336+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.340+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v105'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.340+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.343+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v105'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.343+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.348+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v105'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.348+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9040 con 0x5561e80ca910 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.352+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v105'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.352+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9430 con 0x5561e80ca910 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.356+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v105'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.356+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80dbdd0 con 0x5561e80ca910 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.360+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v105'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80ca910 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.360+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.365+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v105'57 uv55 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.365+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.368+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v105'58 uv58 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.368+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.373+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v105'59 uv59 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.361 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:37:44.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.373+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.377+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v105'60 uv60 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.377+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.381+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v105'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.381+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.385+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v105'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.386+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.378 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:37:44.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.389+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v105'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.390+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80da3f0 con 0x5561e80ce0d0 2024-04-25T11:37:44.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.393+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.394+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80da3f0 con 0x5561e80ce0d0 2024-04-25T11:37:44.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.398+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v105'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.398+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80ce0d0 2024-04-25T11:37:44.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.402+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v105'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f740405c040 con 0x5561e80ce0d0 2024-04-25T11:37:44.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.402+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.405+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v105'64 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.405+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.409+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v105'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.409+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.402 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:37:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.413+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v105'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.414+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.418+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v105'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.418+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.422+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v105'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.422+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.426+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v105'157 uv148 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.427+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.431+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v105'158 uv158 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.431+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.423 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:37:44.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.435+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v105'159 uv159 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.435+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.439+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v105'160 uv160 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.440+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.444+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v105'161 uv161 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.444+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.448+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v105'162 uv155 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.448+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.452+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v105'163 uv163 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.452+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.444 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:37:44.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.456+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v105'164 uv164 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.456+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80da3f0 con 0x5561e80d6170 2024-04-25T11:37:44.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.460+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v105'165 uv165 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.460+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80da3f0 con 0x5561e80d6170 2024-04-25T11:37:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.465+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v105'166 uv166 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.465+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d93e0 con 0x5561e80d6170 2024-04-25T11:37:44.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.469+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v105'167 uv167 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.469+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5561e80d9b40 con 0x5561e80d6170 2024-04-25T11:37:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.473+0000 7f740bc00640 1 -- 172.21.15.19:0/2414922131 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v105'168 uv168 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f73f400d730 con 0x5561e80d6170 2024-04-25T11:37:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5561e80c6140 msgr2=0x5561e80c8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5561e80c6140 0x5561e80c8560 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5561e80d2420 msgr2=0x5561e80d5280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5561e80d2420 0x5561e80d5280 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561e80d6170 msgr2=0x5561e80d8590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561e80d6170 0x5561e80d8590 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5561e80ca910 msgr2=0x5561e80ccd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5561e80ca910 0x5561e80ccd30 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5561e7f7c260 msgr2=0x5561e801b4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5561e7f7c260 0x5561e801b4e0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5561e80ce0d0 msgr2=0x5561e80d04f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5561e80ce0d0 0x5561e80d04f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 reap_dead start 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f740d000640 1 -- 172.21.15.19:0/2414922131 reap_dead start 2024-04-25T11:37:44.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73e8072ec0 msgr2=0x7f73e80753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73e8072ec0 0x7f73e80753d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f73fc013040 tx=0x7f73fc005000 comp rx=0 tx=0).stop 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 msgr2=0x5561e8018e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.474+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e8018e10 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f7404045de0 tx=0x7f74040458b0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.475+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 shutdown_connections 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.475+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73e8072ec0 0x7f73e80753d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.475+0000 7f741420bf40 1 --2- 172.21.15.19:0/2414922131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73ec0036a0 0x5561e8018e10 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.475+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 >> 172.21.15.19:0/2414922131 conn(0x5561e7e6cac0 msgr2=0x5561e7f7b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.475+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 shutdown_connections 2024-04-25T11:37:44.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.475+0000 7f741420bf40 1 -- 172.21.15.19:0/2414922131 wait complete. 2024-04-25T11:37:44.483 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2024-04-25T11:37:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.688+0000 7f45a1d3af40 1 Processor -- start 2024-04-25T11:37:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.688+0000 7f45a1d3af40 1 -- start start 2024-04-25T11:37:44.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f45a1d3af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4580003630 0x5636c77adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f45a1d3af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c7723000 0x5636c7723410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f45a1d3af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636c773dfe0 0x5636c773e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f45a1d3af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636c773be70 con 0x5636c7723000 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f45a1d3af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636c7699d20 con 0x5636c773dfe0 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f45a1d3af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5636c773c3f0 con 0x7f4580003630 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c7723000 0x5636c7723410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c7723000 0x5636c7723410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48788/0 (socket says 172.21.15.19:48788) 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 -- 172.21.15.19:0/3716191388 learned_addr learned my addr 172.21.15.19:0/3716191388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 -- 172.21.15.19:0/3716191388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4580003630 msgr2=0x5636c77adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 --2- 172.21.15.19:0/3716191388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4580003630 0x5636c77adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 -- 172.21.15.19:0/3716191388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636c773dfe0 msgr2=0x5636c773e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 --2- 172.21.15.19:0/3716191388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636c773dfe0 0x5636c773e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.690+0000 7f459a400640 1 -- 172.21.15.19:0/3716191388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c773fcc0 con 0x5636c7723000 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.691+0000 7f459a400640 1 --2- 172.21.15.19:0/3716191388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c7723000 0x5636c7723410 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4588001050 tx=0x7f458800db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5de214ddd4c9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.691+0000 7f4599000640 1 -- 172.21.15.19:0/3716191388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588013070 con 0x5636c7723000 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.691+0000 7f4599000640 1 -- 172.21.15.19:0/3716191388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458800b850 con 0x5636c7723000 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.691+0000 7f4599000640 1 -- 172.21.15.19:0/3716191388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458800bc40 con 0x5636c7723000 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 -- 172.21.15.19:0/3716191388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c7723000 msgr2=0x5636c7723410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/3716191388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c7723000 0x5636c7723410 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4588001050 tx=0x7f458800db50 comp rx=0 tx=0).stop 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 -- 172.21.15.19:0/3716191388 shutdown_connections 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/3716191388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5636c773dfe0 0x5636c773e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/3716191388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c7723000 0x5636c7723410 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/3716191388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4580003630 0x5636c77adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 -- 172.21.15.19:0/3716191388 >> 172.21.15.19:0/3716191388 conn(0x5636c7615ac0 msgr2=0x5636c77215b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 -- 172.21.15.19:0/3716191388 shutdown_connections 2024-04-25T11:37:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.692+0000 7f45a1d3af40 1 -- 172.21.15.19:0/3716191388 wait complete. 2024-04-25T11:37:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.693+0000 7f45a1d3af40 1 Processor -- start 2024-04-25T11:37:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.693+0000 7f45a1d3af40 1 -- start start 2024-04-25T11:37:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f45a1d3af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5636c7723000 0x5636c7723410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f45a1d3af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c773dfe0 0x5636c7739e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f45a1d3af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4580003630 0x5636c77c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f45a1d3af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5636c7725210 con 0x5636c773dfe0 2024-04-25T11:37:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f45a1d3af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5636c7723950 con 0x7f4580003630 2024-04-25T11:37:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f45a1d3af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5636c77c8e50 con 0x5636c7723000 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f459a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c773dfe0 0x5636c7739e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f459a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c773dfe0 0x5636c7739e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48796/0 (socket says 172.21.15.19:48796) 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.695+0000 7f459a400640 1 -- 172.21.15.19:0/1672697528 learned_addr learned my addr 172.21.15.19:0/1672697528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.696+0000 7f459ae00640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5636c7723000 0x5636c7723410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.696+0000 7f459a400640 1 -- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5636c7723000 msgr2=0x5636c7723410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.696+0000 7f459a400640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5636c7723000 0x5636c7723410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.696+0000 7f459a400640 1 -- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4580003630 msgr2=0x5636c77c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.696+0000 7f459a400640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4580003630 0x5636c77c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.696+0000 7f459a400640 1 -- 172.21.15.19:0/1672697528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c77afe60 con 0x5636c773dfe0 2024-04-25T11:37:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.696+0000 7f459a400640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c773dfe0 0x5636c7739e00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f458800b6c0 tx=0x7f4588001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.697+0000 7f4593400640 1 -- 172.21.15.19:0/1672697528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588028070 con 0x5636c773dfe0 2024-04-25T11:37:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.697+0000 7f4593400640 1 -- 172.21.15.19:0/1672697528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4588018c40 con 0x5636c773dfe0 2024-04-25T11:37:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.697+0000 7f4593400640 1 -- 172.21.15.19:0/1672697528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588013050 con 0x5636c773dfe0 2024-04-25T11:37:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.697+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5636c77c2140 con 0x5636c773dfe0 2024-04-25T11:37:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.697+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c77c8160 con 0x5636c773dfe0 2024-04-25T11:37:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.699+0000 7f4593400640 1 -- 172.21.15.19:0/1672697528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f458802b070 con 0x5636c773dfe0 2024-04-25T11:37:44.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.700+0000 7f4593400640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f457c072ec0 0x7f457c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.700+0000 7f4593400640 1 -- 172.21.15.19:0/1672697528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6827+0+0 (secure 0 0 0) 0x7f4588096400 con 0x5636c773dfe0 2024-04-25T11:37:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.700+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5636c77c3f40 0x5636c77aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.700+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e105) -- 0x5636c77af170 con 0x5636c77c3f40 2024-04-25T11:37:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.701+0000 7f459b800640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5636c77c3f40 0x5636c77aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.701+0000 7f459ae00640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f457c072ec0 0x7f457c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:44.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.701+0000 7f459b800640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5636c77c3f40 0x5636c77aec30 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.702+0000 7f459ae00640 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f457c072ec0 0x7f457c0753d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f458c002ff0 tx=0x7f458c005a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.704+0000 7f459b800640 1 -- 172.21.15.19:0/1672697528 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 100.00000000.inode [delete] v105'169 uv169 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4594005040 con 0x5636c77c3f40 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5636c77c3f40 msgr2=0x5636c77aec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5636c77c3f40 0x5636c77aec30 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f457c072ec0 msgr2=0x7f457c0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f457c072ec0 0x7f457c0753d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f458c002ff0 tx=0x7f458c005a00 comp rx=0 tx=0).stop 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c773dfe0 msgr2=0x5636c7739e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 --2- 172.21.15.19:0/1672697528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5636c773dfe0 0x5636c7739e00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f458800b6c0 tx=0x7f4588001970 comp rx=0 tx=0).stop 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f459b800640 1 -- 172.21.15.19:0/1672697528 reap_dead start 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 shutdown_connections 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 >> 172.21.15.19:0/1672697528 conn(0x5636c7615ac0 msgr2=0x5636c76f2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 shutdown_connections 2024-04-25T11:37:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:44.705+0000 7f45a1d3af40 1 -- 172.21.15.19:0/1672697528 wait complete. 2024-04-25T11:37:44.708 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:37:44.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.009+0000 7fb6c3e00640 1 Processor -- start 2024-04-25T11:37:44.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.009+0000 7fb6c3e00640 1 -- start start 2024-04-25T11:37:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.011+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c406dce0 0x7fb6c415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.011+0000 7fb6c3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb6c415ec90 0x7fb6c415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.011+0000 7fb6c3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c4070000 con 0x7fb6c4144340 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.011+0000 7fb6c3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c4144750 con 0x7fb6c415ec90 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.011+0000 7fb6c3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c406f240 con 0x7fb6c406dce0 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c2a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb6c4144340 0x7fb6c406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47306/0 (socket says 172.21.15.19:47306) 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c2a00640 1 -- 172.21.15.19:0/1764513545 learned_addr learned my addr 172.21.15.19:0/1764513545 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c3400640 1 --2- 172.21.15.19:0/1764513545 >> v2:172.21.15.19:3300/0 conn(0x7fb6c415ec90 0x7fb6c415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c2000640 1 --2- 172.21.15.19:0/1764513545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c406dce0 0x7fb6c415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c1600640 1 -- 172.21.15.19:0/1764513545 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1195357828 0 0) 0x7fb6c4070000 con 0x7fb6c4144340 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c3400640 1 -- 172.21.15.19:0/1764513545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c406dce0 msgr2=0x7fb6c415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c3400640 1 --2- 172.21.15.19:0/1764513545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c406dce0 0x7fb6c415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c3400640 1 -- 172.21.15.19:0/1764513545 >> v1:172.21.15.19:6789/0 conn(0x7fb6c4144340 legacy=0x7fb6c406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.012+0000 7fb6c3400640 1 -- 172.21.15.19:0/1764513545 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c406f520 con 0x7fb6c415ec90 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.013+0000 7fb6c3400640 1 --2- 172.21.15.19:0/1764513545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c415ec90 0x7fb6c415f4a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009d90 tx=0x7fb6bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e23aaeed6ca136e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c1600640 1 -- 172.21.15.19:0/1764513545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc007740 con 0x7fb6c415ec90 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c1600640 1 -- 172.21.15.19:0/1764513545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6bc0078e0 con 0x7fb6c415ec90 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c1600640 1 -- 172.21.15.19:0/1764513545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc007cb0 con 0x7fb6c415ec90 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c3e00640 1 -- 172.21.15.19:0/1764513545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c415ec90 msgr2=0x7fb6c415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/1764513545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c415ec90 0x7fb6c415f4a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009d90 tx=0x7fb6bc02a660 comp rx=0 tx=0).stop 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c3e00640 1 -- 172.21.15.19:0/1764513545 shutdown_connections 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/1764513545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c415ec90 0x7fb6c415f4a0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/1764513545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c406dce0 0x7fb6c415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.014+0000 7fb6c3e00640 1 -- 172.21.15.19:0/1764513545 >> 172.21.15.19:0/1764513545 conn(0x7fb6c4072f40 msgr2=0x7fb6c4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:45.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.015+0000 7fb6c3e00640 1 -- 172.21.15.19:0/1764513545 shutdown_connections 2024-04-25T11:37:45.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.015+0000 7fb6c3e00640 1 -- 172.21.15.19:0/1764513545 wait complete. 2024-04-25T11:37:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.017+0000 7fb6c3e00640 1 Processor -- start 2024-04-25T11:37:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.017+0000 7fb6c3e00640 1 -- start start 2024-04-25T11:37:45.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c4144340 0x7fb6c4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:45.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6c415ec90 0x7fb6c4160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4160640 0x7fb6c41f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c415f0a0 con 0x7fb6c4144340 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c406fae0 con 0x7fb6c415ec90 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c415f220 con 0x7fb6c4160640 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c4144340 0x7fb6c4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c4144340 0x7fb6c4158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48822/0 (socket says 172.21.15.19:48822) 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 -- 172.21.15.19:0/2954007036 learned_addr learned my addr 172.21.15.19:0/2954007036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 -- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4160640 msgr2=0x7fb6c41f9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4160640 0x7fb6c41f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 -- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6c415ec90 msgr2=0x7fb6c4160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6c415ec90 0x7fb6c4160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.019+0000 7fb6c2a00640 1 -- 172.21.15.19:0/2954007036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c41faa10 con 0x7fb6c4144340 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.020+0000 7fb6c2a00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c4144340 0x7fb6c4158360 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c460 tx=0x7fb6b400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.020+0000 7fb6bb400640 1 -- 172.21.15.19:0/2954007036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4018070 con 0x7fb6c4144340 2024-04-25T11:37:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.020+0000 7fb6bb400640 1 -- 172.21.15.19:0/2954007036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b4007500 con 0x7fb6c4144340 2024-04-25T11:37:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.020+0000 7fb6bb400640 1 -- 172.21.15.19:0/2954007036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4004a20 con 0x7fb6c4144340 2024-04-25T11:37:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.020+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c40b5c20 con 0x7fb6c4144340 2024-04-25T11:37:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.020+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c40b61e0 con 0x7fb6c4144340 2024-04-25T11:37:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.022+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4076c60 con 0x7fb6c4144340 2024-04-25T11:37:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.023+0000 7fb6bb400640 1 -- 172.21.15.19:0/2954007036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb6b4004bc0 con 0x7fb6c4144340 2024-04-25T11:37:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.024+0000 7fb6bb400640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6a40744b0 0x7fb6a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.024+0000 7fb6bb400640 1 -- 172.21.15.19:0/2954007036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6827+0+0 (secure 0 0 0) 0x7fb6b40949a0 con 0x7fb6c4144340 2024-04-25T11:37:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.025+0000 7fb6c2000640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6a40744b0 0x7fb6a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.026+0000 7fb6c2000640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6a40744b0 0x7fb6a40769c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0006fd0 tx=0x7fb6b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.035+0000 7fb6bb400640 1 -- 172.21.15.19:0/2954007036 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6b405ecc0 con 0x7fb6c4144340 2024-04-25T11:37:45.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.284+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb6c40b7920 con 0x7fb6c4144340 2024-04-25T11:37:45.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:45.995+0000 7fb6bb400640 1 -- 172.21.15.19:0/2954007036 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v176) ==== 162+0+0 (secure 0 0 0) 0x7fb6b405e5c0 con 0x7fb6c4144340 2024-04-25T11:37:45.984 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:37:45.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6a40744b0 msgr2=0x7fb6a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6a40744b0 0x7fb6a40769c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0006fd0 tx=0x7fb6b0008040 comp rx=0 tx=0).stop 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c4144340 msgr2=0x7fb6c4158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c4144340 0x7fb6c4158360 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c460 tx=0x7fb6b400c920 comp rx=0 tx=0).stop 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 shutdown_connections 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb6a40744b0 0x7fb6a40769c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6c415ec90 0x7fb6c4160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6c4160640 0x7fb6c41f9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 --2- 172.21.15.19:0/2954007036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c4144340 0x7fb6c4158360 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:45.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.001+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 >> 172.21.15.19:0/2954007036 conn(0x7fb6c4072f40 msgr2=0x7fb6c406d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:45.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.002+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 shutdown_connections 2024-04-25T11:37:45.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.002+0000 7fb6c3e00640 1 -- 172.21.15.19:0/2954007036 wait complete. 2024-04-25T11:37:46.003 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:46.015+0000 7ff4a5a00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-25T11:37:46.003 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:46.015+0000 7ff4a5a00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:37:46.005 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:46.016+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:37:46.005 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:46.016+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:37:46.019 INFO:tasks.ceph.mds.a.smithi019.stderr: -23> 2024-04-25T11:37:46.015+0000 7ff4a5a00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-25T11:37:46.019 INFO:tasks.ceph.mds.a.smithi019.stderr: -13> 2024-04-25T11:37:46.015+0000 7ff4a5a00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:37:46.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:46.161 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:37:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.308+0000 7faa83000640 1 Processor -- start 2024-04-25T11:37:46.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.308+0000 7faa83000640 1 -- start start 2024-04-25T11:37:46.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa83000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c15e2b0 0x7faa7c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa83000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faa7c0cc5c0 0x7faa7c06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa83000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa7c0b14d0 0x7faa7c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa83000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faa7c0760b0 con 0x7faa7c0cc5c0 2024-04-25T11:37:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa83000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faa7c0b7720 con 0x7faa7c0b14d0 2024-04-25T11:37:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa83000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c0b7a00 con 0x7faa7c15e2b0 2024-04-25T11:37:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa81200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faa7c0cc5c0 0x7faa7c06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa81200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faa7c0cc5c0 0x7faa7c06d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48848/0 (socket says 172.21.15.19:48848) 2024-04-25T11:37:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa81200640 1 -- 172.21.15.19:0/2967485613 learned_addr learned my addr 172.21.15.19:0/2967485613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.311+0000 7faa81c00640 1 --2- 172.21.15.19:0/2967485613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c15e2b0 0x7faa7c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.312+0000 7faa82600640 1 --2- 172.21.15.19:0/2967485613 >> v2:172.21.15.169:3300/0 conn(0x7faa7c0b14d0 0x7faa7c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.312+0000 7faa81200640 1 -- 172.21.15.19:0/2967485613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c15e2b0 msgr2=0x7faa7c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.312+0000 7faa81200640 1 --2- 172.21.15.19:0/2967485613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c15e2b0 0x7faa7c06d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.312+0000 7faa81200640 1 -- 172.21.15.19:0/2967485613 >> v2:172.21.15.169:3300/0 conn(0x7faa7c0b14d0 msgr2=0x7faa7c0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.312+0000 7faa81200640 1 --2- 172.21.15.19:0/2967485613 >> v2:172.21.15.169:3300/0 conn(0x7faa7c0b14d0 0x7faa7c0b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.312+0000 7faa81200640 1 -- 172.21.15.19:0/2967485613 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c0b2ec0 con 0x7faa7c0cc5c0 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.313+0000 7faa81200640 1 --2- 172.21.15.19:0/2967485613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0cc5c0 0x7faa7c06d730 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0736e0 tx=0x7faa6802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb97bfd8e124685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.313+0000 7faa7be00640 1 -- 172.21.15.19:0/2967485613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68037070 con 0x7faa7c0cc5c0 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.313+0000 7faa7be00640 1 -- 172.21.15.19:0/2967485613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6802ae00 con 0x7faa7c0cc5c0 2024-04-25T11:37:46.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.313+0000 7faa7be00640 1 -- 172.21.15.19:0/2967485613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68032660 con 0x7faa7c0cc5c0 2024-04-25T11:37:46.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 -- 172.21.15.19:0/2967485613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0cc5c0 msgr2=0x7faa7c06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 --2- 172.21.15.19:0/2967485613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0cc5c0 0x7faa7c06d730 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0736e0 tx=0x7faa6802a0b0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 -- 172.21.15.19:0/2967485613 shutdown_connections 2024-04-25T11:37:46.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 --2- 172.21.15.19:0/2967485613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0cc5c0 0x7faa7c06d730 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 --2- 172.21.15.19:0/2967485613 >> v2:172.21.15.169:3300/0 conn(0x7faa7c0b14d0 0x7faa7c0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 --2- 172.21.15.19:0/2967485613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c15e2b0 0x7faa7c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 -- 172.21.15.19:0/2967485613 >> 172.21.15.19:0/2967485613 conn(0x7faa7c09b7d0 msgr2=0x7faa7c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 -- 172.21.15.19:0/2967485613 shutdown_connections 2024-04-25T11:37:46.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.314+0000 7faa83000640 1 -- 172.21.15.19:0/2967485613 wait complete. 2024-04-25T11:37:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.316+0000 7faa83000640 1 Processor -- start 2024-04-25T11:37:46.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.317+0000 7faa83000640 1 -- start start 2024-04-25T11:37:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.318+0000 7faa83000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0b14d0 0x7faa7c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.318+0000 7faa83000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c0cc5c0 0x7faa7c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.318+0000 7faa83000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7c15e2b0 0x7faa7c1508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.318+0000 7faa83000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c0b3c00 con 0x7faa7c0b14d0 2024-04-25T11:37:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.318+0000 7faa83000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c0b2b70 con 0x7faa7c15e2b0 2024-04-25T11:37:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.318+0000 7faa83000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c0b3d80 con 0x7faa7c0cc5c0 2024-04-25T11:37:46.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0b14d0 0x7faa7c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0b14d0 0x7faa7c0d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48856/0 (socket says 172.21.15.19:48856) 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 -- 172.21.15.19:0/352053024 learned_addr learned my addr 172.21.15.19:0/352053024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81200640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c0cc5c0 0x7faa7c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 -- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c0cc5c0 msgr2=0x7faa7c1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c0cc5c0 0x7faa7c1503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa82600640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7c15e2b0 0x7faa7c1508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 -- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7c15e2b0 msgr2=0x7faa7c1508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7c15e2b0 0x7faa7c1508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 -- 172.21.15.19:0/352053024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c153440 con 0x7faa7c0b14d0 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.319+0000 7faa81c00640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0b14d0 0x7faa7c0d1380 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c7f0 tx=0x7faa7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.320+0000 7faa79600640 1 -- 172.21.15.19:0/352053024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70015070 con 0x7faa7c0b14d0 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.320+0000 7faa79600640 1 -- 172.21.15.19:0/352053024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa70004220 con 0x7faa7c0b14d0 2024-04-25T11:37:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.320+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c1609a0 con 0x7faa7c0b14d0 2024-04-25T11:37:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.320+0000 7faa79600640 1 -- 172.21.15.19:0/352053024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70007620 con 0x7faa7c0b14d0 2024-04-25T11:37:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.320+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c1512d0 con 0x7faa7c0b14d0 2024-04-25T11:37:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.321+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7c15ff70 con 0x7faa7c0b14d0 2024-04-25T11:37:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.323+0000 7faa79600640 1 -- 172.21.15.19:0/352053024 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faa700077c0 con 0x7faa7c0b14d0 2024-04-25T11:37:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.324+0000 7faa79600640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa5c0744b0 0x7faa5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.324+0000 7faa79600640 1 -- 172.21.15.19:0/352053024 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7faa70095900 con 0x7faa7c0b14d0 2024-04-25T11:37:46.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.325+0000 7faa81200640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa5c0744b0 0x7faa5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.326+0000 7faa81200640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa5c0744b0 0x7faa5c0769c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7faa68034000 tx=0x7faa6802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:46.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.329+0000 7faa79600640 1 -- 172.21.15.19:0/352053024 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa7005fb30 con 0x7faa7c0b14d0 2024-04-25T11:37:46.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.583+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa7c050740 con 0x7faa7c0b14d0 2024-04-25T11:37:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.585+0000 7faa79600640 1 -- 172.21.15.19:0/352053024 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) ==== 95+0+4352 (secure 0 0 0) 0x7faa7005f430 con 0x7faa7c0b14d0 2024-04-25T11:37:46.573 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:46.573 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":178,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:46.018203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:46.573 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 178 2024-04-25T11:37:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa5c0744b0 msgr2=0x7faa5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa5c0744b0 0x7faa5c0769c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7faa68034000 tx=0x7faa6802e000 comp rx=0 tx=0).stop 2024-04-25T11:37:46.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0b14d0 msgr2=0x7faa7c0d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0b14d0 0x7faa7c0d1380 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c7f0 tx=0x7faa7000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 shutdown_connections 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faa5c0744b0 0x7faa5c0769c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa7c15e2b0 0x7faa7c1508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa7c0cc5c0 0x7faa7c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.587+0000 7faa83000640 1 --2- 172.21.15.19:0/352053024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa7c0b14d0 0x7faa7c0d1380 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.588+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 >> 172.21.15.19:0/352053024 conn(0x7faa7c09b7d0 msgr2=0x7faa7c0ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.588+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 shutdown_connections 2024-04-25T11:37:46.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.588+0000 7faa83000640 1 -- 172.21.15.19:0/352053024 wait complete. 2024-04-25T11:37:46.603 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:46.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.896+0000 7f3508a00640 1 Processor -- start 2024-04-25T11:37:46.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.896+0000 7f3508a00640 1 -- start start 2024-04-25T11:37:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.898+0000 7f3508a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35041434d0 0x7f3504070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3508a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35040709c0 con 0x7f35041612f0 2024-04-25T11:37:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3508a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350406f3e0 con 0x7f35040d0ed0 2024-04-25T11:37:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3508a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f35041438e0 con 0x7f35041434d0 2024-04-25T11:37:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3501e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35041434d0 0x7f3504070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3502800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f35041612f0 0x7f35040b51a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47322/0 (socket says 172.21.15.19:47322) 2024-04-25T11:37:46.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3502800640 1 -- 172.21.15.19:0/2015367187 learned_addr learned my addr 172.21.15.19:0/2015367187 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:46.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3501e00640 1 -- 172.21.15.19:0/2015367187 >> v1:172.21.15.169:6789/0 conn(0x7f35040d0ed0 legacy=0x7f3504160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:46.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3501e00640 1 -- 172.21.15.19:0/2015367187 >> v1:172.21.15.19:6789/0 conn(0x7f35041612f0 legacy=0x7f35040b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.899+0000 7f3500a00640 1 -- 172.21.15.19:0/2015367187 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729249488 0 0) 0x7f35040709c0 con 0x7f35041612f0 2024-04-25T11:37:46.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.900+0000 7f3501e00640 1 -- 172.21.15.19:0/2015367187 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350406f6c0 con 0x7f35041434d0 2024-04-25T11:37:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.901+0000 7f3501e00640 1 --2- 172.21.15.19:0/2015367187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f3504070480 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097d0 tx=0x7f34f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ff42177c9c6c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.901+0000 7f3500a00640 1 -- 172.21.15.19:0/2015367187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4037070 con 0x7f35041434d0 2024-04-25T11:37:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.901+0000 7f3500a00640 1 -- 172.21.15.19:0/2015367187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f4032440 con 0x7f35041434d0 2024-04-25T11:37:46.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.901+0000 7f3500a00640 1 -- 172.21.15.19:0/2015367187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4032810 con 0x7f35041434d0 2024-04-25T11:37:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.902+0000 7f3508a00640 1 -- 172.21.15.19:0/2015367187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 msgr2=0x7f3504070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.902+0000 7f3508a00640 1 --2- 172.21.15.19:0/2015367187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f3504070480 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097d0 tx=0x7f34f402a270 comp rx=0 tx=0).stop 2024-04-25T11:37:46.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.902+0000 7f3508a00640 1 -- 172.21.15.19:0/2015367187 shutdown_connections 2024-04-25T11:37:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.902+0000 7f3508a00640 1 --2- 172.21.15.19:0/2015367187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f3504070480 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.902+0000 7f3508a00640 1 -- 172.21.15.19:0/2015367187 >> 172.21.15.19:0/2015367187 conn(0x7f35040d1ae0 msgr2=0x7f35040d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.902+0000 7f3508a00640 1 -- 172.21.15.19:0/2015367187 shutdown_connections 2024-04-25T11:37:46.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.902+0000 7f3508a00640 1 -- 172.21.15.19:0/2015367187 wait complete. 2024-04-25T11:37:46.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.904+0000 7f3508a00640 1 Processor -- start 2024-04-25T11:37:46.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.904+0000 7f3508a00640 1 -- start start 2024-04-25T11:37:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3508a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35040d0ed0 0x7f35041600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3508a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f350415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3508a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3504141730 0x7f35041e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3508a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f350406f090 con 0x7f35041434d0 2024-04-25T11:37:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3508a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f350406fc80 con 0x7f3504141730 2024-04-25T11:37:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3508a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35040d12e0 con 0x7f35040d0ed0 2024-04-25T11:37:46.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3501400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f350415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3501400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f350415a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48872/0 (socket says 172.21.15.19:48872) 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.906+0000 7f3501400640 1 -- 172.21.15.19:0/206068829 learned_addr learned my addr 172.21.15.19:0/206068829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3502800640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3504141730 0x7f35041e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3501e00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35040d0ed0 0x7f35041600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3501400640 1 -- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35040d0ed0 msgr2=0x7f35041600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3501400640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35040d0ed0 0x7f35041600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3501400640 1 -- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3504141730 msgr2=0x7f35041e7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3501400640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3504141730 0x7f35041e7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3501400640 1 -- 172.21.15.19:0/206068829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35040b0890 con 0x7f35041434d0 2024-04-25T11:37:46.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3501400640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f350415a8a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c730 tx=0x7f34ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f34fa000640 1 -- 172.21.15.19:0/206068829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec003b70 con 0x7f35041434d0 2024-04-25T11:37:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f34fa000640 1 -- 172.21.15.19:0/206068829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec003d10 con 0x7f35041434d0 2024-04-25T11:37:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f34fa000640 1 -- 172.21.15.19:0/206068829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00aa90 con 0x7f35041434d0 2024-04-25T11:37:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.907+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35040b0b70 con 0x7f35041434d0 2024-04-25T11:37:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.908+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35040b1130 con 0x7f35041434d0 2024-04-25T11:37:46.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.909+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3504050740 con 0x7f35041434d0 2024-04-25T11:37:46.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.910+0000 7f34fa000640 1 -- 172.21.15.19:0/206068829 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34ec00ac30 con 0x7f35041434d0 2024-04-25T11:37:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.912+0000 7f34fa000640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e00744b0 0x7f34e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:46.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.912+0000 7f34fa000640 1 -- 172.21.15.19:0/206068829 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7f34ec0956d0 con 0x7f35041434d0 2024-04-25T11:37:46.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.912+0000 7f3501e00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e00744b0 0x7f34e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:46.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.914+0000 7f3501e00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e00744b0 0x7f34e00769c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f34f4034000 tx=0x7f34f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:46.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:46.917+0000 7f34fa000640 1 -- 172.21.15.19:0/206068829 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34ec05f900 con 0x7f35041434d0 2024-04-25T11:37:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.164+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35040b67a0 con 0x7f35041434d0 2024-04-25T11:37:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.165+0000 7f34fa000640 1 -- 172.21.15.19:0/206068829 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) ==== 95+0+5178 (secure 0 0 0) 0x7f34ec05f200 con 0x7f35041434d0 2024-04-25T11:37:47.153 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:47.154 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":179,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9039,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3471202301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3471202301},{"type":"v1","addr":"172.21.15.19:6835","nonce":3471202301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:46.018203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 179 2024-04-25T11:37:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.167+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e00744b0 msgr2=0x7f34e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.167+0000 7f3508a00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e00744b0 0x7f34e00769c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f34f4034000 tx=0x7f34f402e000 comp rx=0 tx=0).stop 2024-04-25T11:37:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.167+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 msgr2=0x7f350415a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.167+0000 7f3508a00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f350415a8a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c730 tx=0x7f34ec00cbf0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 shutdown_connections 2024-04-25T11:37:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34e00744b0 0x7f34e00769c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3504141730 0x7f35041e7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35041434d0 0x7f350415a8a0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 --2- 172.21.15.19:0/206068829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35040d0ed0 0x7f35041600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 >> 172.21.15.19:0/206068829 conn(0x7f35040d1ae0 msgr2=0x7f35040b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 shutdown_connections 2024-04-25T11:37:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.168+0000 7f3508a00640 1 -- 172.21.15.19:0/206068829 wait complete. 2024-04-25T11:37:47.183 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:47.183 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:47.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.478+0000 7f077ba00640 1 Processor -- start 2024-04-25T11:37:47.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.478+0000 7f077ba00640 1 -- start start 2024-04-25T11:37:47.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f077ba00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07741443c0 0x7f077406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:47.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f077ba00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f07740cfcf0 0x7f077406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:47.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f077ba00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f077415f5f0 con 0x7f0774070370 2024-04-25T11:37:47.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f077ba00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f07740708d0 con 0x7f07741443c0 2024-04-25T11:37:47.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f077ba00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f077415f160 con 0x7f07740cfcf0 2024-04-25T11:37:47.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f0779000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07741443c0 0x7f077406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:47.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f0773e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0774070370 0x7f07740cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47326/0 (socket says 172.21.15.19:47326) 2024-04-25T11:37:47.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.481+0000 7f0773e00640 1 -- 172.21.15.19:0/3718013916 learned_addr learned my addr 172.21.15.19:0/3718013916 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.482+0000 7f0779a00640 1 --2- 172.21.15.19:0/3718013916 >> v2:172.21.15.169:3300/0 conn(0x7f07740cfcf0 0x7f077406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.482+0000 7f0779000640 1 -- 172.21.15.19:0/3718013916 >> v2:172.21.15.169:3300/0 conn(0x7f07740cfcf0 msgr2=0x7f077406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.482+0000 7f0779000640 1 --2- 172.21.15.19:0/3718013916 >> v2:172.21.15.169:3300/0 conn(0x7f07740cfcf0 0x7f077406c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.482+0000 7f0779000640 1 -- 172.21.15.19:0/3718013916 >> v1:172.21.15.19:6789/0 conn(0x7f0774070370 legacy=0x7f07740cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.482+0000 7f0779000640 1 -- 172.21.15.19:0/3718013916 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077415e830 con 0x7f07741443c0 2024-04-25T11:37:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.482+0000 7f0773400640 1 -- 172.21.15.19:0/3718013916 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271382748 0 0) 0x7f077415f5f0 con 0x7f0774070370 2024-04-25T11:37:47.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.482+0000 7f0779a00640 1 --2- 172.21.15.19:0/3718013916 >> v2:172.21.15.169:3300/0 conn(0x7f07740cfcf0 0x7f077406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:47.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.483+0000 7f0779000640 1 --2- 172.21.15.19:0/3718013916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07741443c0 0x7f077406fe30 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f077415c280 tx=0x7f076402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a91ba9a609daf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:47.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.483+0000 7f0773400640 1 -- 172.21.15.19:0/3718013916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764037070 con 0x7f07741443c0 2024-04-25T11:37:47.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.483+0000 7f0773400640 1 -- 172.21.15.19:0/3718013916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0764032440 con 0x7f07741443c0 2024-04-25T11:37:47.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f0773400640 1 -- 172.21.15.19:0/3718013916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764032810 con 0x7f07741443c0 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f077ba00640 1 -- 172.21.15.19:0/3718013916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07741443c0 msgr2=0x7f077406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f077ba00640 1 --2- 172.21.15.19:0/3718013916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07741443c0 0x7f077406fe30 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f077415c280 tx=0x7f076402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f077ba00640 1 -- 172.21.15.19:0/3718013916 shutdown_connections 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f077ba00640 1 --2- 172.21.15.19:0/3718013916 >> v2:172.21.15.169:3300/0 conn(0x7f07740cfcf0 0x7f077406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f077ba00640 1 --2- 172.21.15.19:0/3718013916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07741443c0 0x7f077406fe30 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f077ba00640 1 -- 172.21.15.19:0/3718013916 >> 172.21.15.19:0/3718013916 conn(0x7f07740d1ae0 msgr2=0x7f07740d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.484+0000 7f077ba00640 1 -- 172.21.15.19:0/3718013916 shutdown_connections 2024-04-25T11:37:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.485+0000 7f077ba00640 1 -- 172.21.15.19:0/3718013916 wait complete. 2024-04-25T11:37:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.487+0000 7f077ba00640 1 Processor -- start 2024-04-25T11:37:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.487+0000 7f077ba00640 1 -- start start 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f077ba00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774070370 0x7f07740c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f077ba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07741443c0 0x7f07740b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f077ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07740b1b50 0x7f077413f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f077ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07741447d0 con 0x7f07740b1b50 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f077ba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f077415edf0 con 0x7f07741443c0 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f077ba00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0774162b00 con 0x7f0774070370 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07740b1b50 0x7f077413f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07740b1b50 0x7f077413f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48894/0 (socket says 172.21.15.19:48894) 2024-04-25T11:37:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 -- 172.21.15.19:0/3416039545 learned_addr learned my addr 172.21.15.19:0/3416039545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0773e00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07741443c0 0x7f07740b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779000640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774070370 0x7f07740c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 -- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774070370 msgr2=0x7f07740c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774070370 0x7f07740c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 -- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07741443c0 msgr2=0x7f07740b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07741443c0 0x7f07740b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.489+0000 7f0779a00640 1 -- 172.21.15.19:0/3416039545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07740c6e20 con 0x7f07740b1b50 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.490+0000 7f0779a00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07740b1b50 0x7f077413f090 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c5c0 tx=0x7f076c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.490+0000 7f0770c00640 1 -- 172.21.15.19:0/3416039545 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c015070 con 0x7f07740b1b50 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.490+0000 7f0770c00640 1 -- 172.21.15.19:0/3416039545 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076c011820 con 0x7f07740b1b50 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.490+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0774141c50 con 0x7f07740b1b50 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.490+0000 7f0770c00640 1 -- 172.21.15.19:0/3416039545 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c011c10 con 0x7f07740b1b50 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.490+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0774142210 con 0x7f07740b1b50 2024-04-25T11:37:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.491+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0774050740 con 0x7f07740b1b50 2024-04-25T11:37:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.493+0000 7f0770c00640 1 -- 172.21.15.19:0/3416039545 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f076c018070 con 0x7f07740b1b50 2024-04-25T11:37:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.494+0000 7f0770c00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07540744b0 0x7f07540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.494+0000 7f0770c00640 1 -- 172.21.15.19:0/3416039545 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7f076c0955d0 con 0x7f07740b1b50 2024-04-25T11:37:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.494+0000 7f0779000640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07540744b0 0x7f07540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.496+0000 7f0779000640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07540744b0 0x7f07540769c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0764034000 tx=0x7f076402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.499+0000 7f0770c00640 1 -- 172.21.15.19:0/3416039545 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f076c05f8a0 con 0x7f07740b1b50 2024-04-25T11:37:47.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.746+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f077406f950 con 0x7f07740b1b50 2024-04-25T11:37:47.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.748+0000 7f0770c00640 1 -- 172.21.15.19:0/3416039545 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) ==== 95+0+5178 (secure 0 0 0) 0x7f076c05f1a0 con 0x7f07740b1b50 2024-04-25T11:37:47.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:47.736 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":179,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9039,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3471202301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3471202301},{"type":"v1","addr":"172.21.15.19:6835","nonce":3471202301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":179}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:46.018203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:47.737 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 179 2024-04-25T11:37:47.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.750+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07540744b0 msgr2=0x7f07540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.750+0000 7f077ba00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07540744b0 0x7f07540769c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0764034000 tx=0x7f076402e000 comp rx=0 tx=0).stop 2024-04-25T11:37:47.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.750+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07740b1b50 msgr2=0x7f077413f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:47.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.750+0000 7f077ba00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07740b1b50 0x7f077413f090 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c5c0 tx=0x7f076c00ca80 comp rx=0 tx=0).stop 2024-04-25T11:37:47.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 shutdown_connections 2024-04-25T11:37:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07540744b0 0x7f07540769c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07741443c0 0x7f07740b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07740b1b50 0x7f077413f090 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 --2- 172.21.15.19:0/3416039545 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0774070370 0x7f07740c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 >> 172.21.15.19:0/3416039545 conn(0x7f07740d1ae0 msgr2=0x7f07740b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 shutdown_connections 2024-04-25T11:37:47.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:47.751+0000 7f077ba00640 1 -- 172.21.15.19:0/3416039545 wait complete. 2024-04-25T11:37:47.766 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2024-04-25T11:37:47.767 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2024-04-25T11:37:47.767 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:47.767 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:47.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:37:47.816 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:37:47.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:47.817 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:37:47.913 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:37:47.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:37:47.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:37:48.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.215+0000 7f0d22c00640 1 Processor -- start 2024-04-25T11:37:48.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.215+0000 7f0d22c00640 1 -- start start 2024-04-25T11:37:48.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.217+0000 7f0d22c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d1c0cfd40 0x7f0d1c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:48.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.217+0000 7f0d22c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0d1c0b3e00 0x7f0d1c0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:48.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.217+0000 7f0d22c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1c0b8190 con 0x7f0d1c0b7d80 2024-04-25T11:37:48.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.217+0000 7f0d22c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1c0d27e0 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.217+0000 7f0d22c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1c0d1c10 con 0x7f0d1c0cfd40 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.218+0000 7f0d1b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0d1c0b3e00 0x7f0d1c0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.218+0000 7f0d1b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0d1c0b3e00 0x7f0d1c0b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48908/0 (socket says 172.21.15.19:48908) 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.218+0000 7f0d1b400640 1 -- 172.21.15.19:0/1022440900 learned_addr learned my addr 172.21.15.19:0/1022440900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.218+0000 7f0d1be00640 1 --2- 172.21.15.19:0/1022440900 >> v2:172.21.15.169:3300/0 conn(0x7f0d1c0cfd40 0x7f0d1c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.219+0000 7f0d1b400640 1 -- 172.21.15.19:0/1022440900 >> v2:172.21.15.169:3300/0 conn(0x7f0d1c0cfd40 msgr2=0x7f0d1c0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.219+0000 7f0d1b400640 1 --2- 172.21.15.19:0/1022440900 >> v2:172.21.15.169:3300/0 conn(0x7f0d1c0cfd40 0x7f0d1c0b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.219+0000 7f0d1b400640 1 -- 172.21.15.19:0/1022440900 >> v1:172.21.15.169:6789/0 conn(0x7f0d1c0b7d80 legacy=0x7f0d1c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.219+0000 7f0d1b400640 1 -- 172.21.15.19:0/1022440900 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c0d1eb0 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.221+0000 7f0d1b400640 1 --2- 172.21.15.19:0/1022440900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c0b7840 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b8620 tx=0x7f0d0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71135333935845c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.221+0000 7f0d1aa00640 1 -- 172.21.15.19:0/1022440900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08037070 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.221+0000 7f0d1aa00640 1 -- 172.21.15.19:0/1022440900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d08032440 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.221+0000 7f0d1aa00640 1 -- 172.21.15.19:0/1022440900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08032810 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 -- 172.21.15.19:0/1022440900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 msgr2=0x7f0d1c0b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 --2- 172.21.15.19:0/1022440900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c0b7840 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b8620 tx=0x7f0d0802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:37:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 -- 172.21.15.19:0/1022440900 shutdown_connections 2024-04-25T11:37:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 --2- 172.21.15.19:0/1022440900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c0b7840 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 --2- 172.21.15.19:0/1022440900 >> v2:172.21.15.169:3300/0 conn(0x7f0d1c0cfd40 0x7f0d1c0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 -- 172.21.15.19:0/1022440900 >> 172.21.15.19:0/1022440900 conn(0x7f0d1c09b7d0 msgr2=0x7f0d1c09bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 -- 172.21.15.19:0/1022440900 shutdown_connections 2024-04-25T11:37:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.222+0000 7f0d22c00640 1 -- 172.21.15.19:0/1022440900 wait complete. 2024-04-25T11:37:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.225+0000 7f0d22c00640 1 Processor -- start 2024-04-25T11:37:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.225+0000 7f0d22c00640 1 -- start start 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.226+0000 7f0d22c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c09d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.226+0000 7f0d22c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d1c0b7d80 0x7f0d1c162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.226+0000 7f0d22c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d1c0cfd40 0x7f0d1c162680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.226+0000 7f0d22c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0b4210 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.226+0000 7f0d22c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0d2470 con 0x7f0d1c0b7d80 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.226+0000 7f0d22c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c0b4390 con 0x7f0d1c0cfd40 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c09d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c09d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48916/0 (socket says 172.21.15.19:48916) 2024-04-25T11:37:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 -- 172.21.15.19:0/2528543316 learned_addr learned my addr 172.21.15.19:0/2528543316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 -- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d1c0cfd40 msgr2=0x7f0d1c162680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d1c0cfd40 0x7f0d1c162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 -- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d1c0b7d80 msgr2=0x7f0d1c162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d1c0b7d80 0x7f0d1c162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.227+0000 7f0d1be00640 1 -- 172.21.15.19:0/2528543316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c0c96c0 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.228+0000 7f0d1be00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c09d740 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00d5f0 tx=0x7f0d0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.228+0000 7f0d13e00640 1 -- 172.21.15.19:0/2528543316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0125f0 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.228+0000 7f0d13e00640 1 -- 172.21.15.19:0/2528543316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c019070 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.228+0000 7f0d13e00640 1 -- 172.21.15.19:0/2528543316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0129d0 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.228+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1610a0 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.228+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c161660 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.229+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c050740 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.230+0000 7f0d13e00640 1 -- 172.21.15.19:0/2528543316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0d0c012b70 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.231+0000 7f0d13e00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0cfc0744b0 0x7f0cfc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.232+0000 7f0d13e00640 1 -- 172.21.15.19:0/2528543316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7f0d0c095c20 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.232+0000 7f0d1b400640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0cfc0744b0 0x7f0cfc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.234+0000 7f0d1b400640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0cfc0744b0 0x7f0cfc0769c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0d08034000 tx=0x7f0d0802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:48.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.237+0000 7f0d13e00640 1 -- 172.21.15.19:0/2528543316 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d0c05fe50 con 0x7f0d1c0b3e00 2024-04-25T11:37:48.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:48.478+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0d1c070350 con 0x7f0d1c0b3e00 2024-04-25T11:37:49.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.041+0000 7f0d13e00640 1 -- 172.21.15.19:0/2528543316 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v180) ==== 161+0+0 (secure 0 0 0) 0x7f0d0c05f750 con 0x7f0d1c0b3e00 2024-04-25T11:37:49.030 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:37:49.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0cfc0744b0 msgr2=0x7f0cfc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:49.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0cfc0744b0 0x7f0cfc0769c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0d08034000 tx=0x7f0d0802c000 comp rx=0 tx=0).stop 2024-04-25T11:37:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 msgr2=0x7f0d1c09d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c09d740 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00d5f0 tx=0x7f0d0c00dab0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 shutdown_connections 2024-04-25T11:37:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0cfc0744b0 0x7f0cfc0769c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d1c0b7d80 0x7f0d1c162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d1c0cfd40 0x7f0d1c162680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 --2- 172.21.15.19:0/2528543316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d1c0b3e00 0x7f0d1c09d740 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.044+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 >> 172.21.15.19:0/2528543316 conn(0x7f0d1c09b7d0 msgr2=0x7f0d1c15fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:49.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.045+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 shutdown_connections 2024-04-25T11:37:49.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.045+0000 7f0d22c00640 1 -- 172.21.15.19:0/2528543316 wait complete. 2024-04-25T11:37:49.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:37:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.363+0000 7fd4d4a00640 1 Processor -- start 2024-04-25T11:37:49.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.363+0000 7fd4d4a00640 1 -- start start 2024-04-25T11:37:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.365+0000 7fd4d4a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4d006c5c0 0x7fd4d00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.365+0000 7fd4d4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d0144370 0x7fd4d00b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.365+0000 7fd4d4a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d00b50e0 con 0x7fd4d00b4cd0 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.365+0000 7fd4d4a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d00d1230 con 0x7fd4d006c5c0 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.365+0000 7fd4d4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d00d06b0 con 0x7fd4d0144370 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4ce800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4d006c5c0 0x7fd4d00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4ce800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd4d006c5c0 0x7fd4d00cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48926/0 (socket says 172.21.15.19:48926) 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4ce800640 1 -- 172.21.15.19:0/2681931515 learned_addr learned my addr 172.21.15.19:0/2681931515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4c7e00640 1 --2- 172.21.15.19:0/2681931515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d0144370 0x7fd4d00b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4ce800640 1 -- 172.21.15.19:0/2681931515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d0144370 msgr2=0x7fd4d00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4ce800640 1 --2- 172.21.15.19:0/2681931515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d0144370 0x7fd4d00b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4ce800640 1 -- 172.21.15.19:0/2681931515 >> v1:172.21.15.169:6789/0 conn(0x7fd4d00b4cd0 legacy=0x7fd4d0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4ce800640 1 -- 172.21.15.19:0/2681931515 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d00d0990 con 0x7fd4d006c5c0 2024-04-25T11:37:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.366+0000 7fd4c7e00640 1 --2- 172.21.15.19:0/2681931515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d0144370 0x7fd4d00b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.367+0000 7fd4ce800640 1 --2- 172.21.15.19:0/2681931515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d00cfa70 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009870 tx=0x7fd4c002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb3f8ce4257f95d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.368+0000 7fd4cde00640 1 -- 172.21.15.19:0/2681931515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0037070 con 0x7fd4d006c5c0 2024-04-25T11:37:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.368+0000 7fd4cde00640 1 -- 172.21.15.19:0/2681931515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0032440 con 0x7fd4d006c5c0 2024-04-25T11:37:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.368+0000 7fd4cde00640 1 -- 172.21.15.19:0/2681931515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0032810 con 0x7fd4d006c5c0 2024-04-25T11:37:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2681931515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 msgr2=0x7fd4d00cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2681931515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d00cfa70 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009870 tx=0x7fd4c002a200 comp rx=0 tx=0).stop 2024-04-25T11:37:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2681931515 shutdown_connections 2024-04-25T11:37:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2681931515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d0144370 0x7fd4d00b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2681931515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d00cfa70 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2681931515 >> 172.21.15.19:0/2681931515 conn(0x7fd4d00d1720 msgr2=0x7fd4d00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2681931515 shutdown_connections 2024-04-25T11:37:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.369+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2681931515 wait complete. 2024-04-25T11:37:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.371+0000 7fd4d4a00640 1 Processor -- start 2024-04-25T11:37:49.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.371+0000 7fd4d4a00640 1 -- start start 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4d4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d01467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4d4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d00b4cd0 0x7fd4d0070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4d4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d0144370 0x7fd4d00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4d4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d00d02d0 con 0x7fd4d006c5c0 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4d4a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d00d0f50 con 0x7fd4d0144370 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4d4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d00d0450 con 0x7fd4d00b4cd0 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4ce800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d01467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4ce800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d01467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48934/0 (socket says 172.21.15.19:48934) 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4ce800640 1 -- 172.21.15.19:0/2640916602 learned_addr learned my addr 172.21.15.19:0/2640916602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.373+0000 7fd4c7e00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d00b4cd0 0x7fd4d0070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4cf200640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d0144370 0x7fd4d00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4ce800640 1 -- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d00b4cd0 msgr2=0x7fd4d0070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4ce800640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d00b4cd0 0x7fd4d0070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4ce800640 1 -- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d0144370 msgr2=0x7fd4d00cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4ce800640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d0144370 0x7fd4d00cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4ce800640 1 -- 172.21.15.19:0/2640916602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d00cd010 con 0x7fd4d006c5c0 2024-04-25T11:37:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4cf200640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d0144370 0x7fd4d00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4ce800640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d01467d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009870 tx=0x7fd4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4c6a00640 1 -- 172.21.15.19:0/2640916602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0041020 con 0x7fd4d006c5c0 2024-04-25T11:37:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4c6a00640 1 -- 172.21.15.19:0/2640916602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0030970 con 0x7fd4d006c5c0 2024-04-25T11:37:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4c6a00640 1 -- 172.21.15.19:0/2640916602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0037040 con 0x7fd4d006c5c0 2024-04-25T11:37:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.374+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d00cd2f0 con 0x7fd4d006c5c0 2024-04-25T11:37:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.375+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d01459c0 con 0x7fd4d006c5c0 2024-04-25T11:37:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.376+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d0050740 con 0x7fd4d006c5c0 2024-04-25T11:37:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.377+0000 7fd4c6a00640 1 -- 172.21.15.19:0/2640916602 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4c0044020 con 0x7fd4d006c5c0 2024-04-25T11:37:49.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.378+0000 7fd4c6a00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4ac0744b0 0x7fd4ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:49.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.378+0000 7fd4c6a00640 1 -- 172.21.15.19:0/2640916602 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7fd4c00b6180 con 0x7fd4d006c5c0 2024-04-25T11:37:49.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.379+0000 7fd4c7e00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4ac0744b0 0x7fd4ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.380+0000 7fd4c7e00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4ac0744b0 0x7fd4ac0769c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd4d01448a0 tx=0x7fd4b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:49.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.383+0000 7fd4c6a00640 1 -- 172.21.15.19:0/2640916602 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4c004b030 con 0x7fd4d006c5c0 2024-04-25T11:37:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:49.664+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd4d0147710 con 0x7fd4d006c5c0 2024-04-25T11:37:50.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.045+0000 7fd4c6a00640 1 -- 172.21.15.19:0/2640916602 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v181) ==== 101+0+0 (secure 0 0 0) 0x7fd4d00b50e0 con 0x7fd4d006c5c0 2024-04-25T11:37:50.034 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:37:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.048+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4ac0744b0 msgr2=0x7fd4ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.048+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4ac0744b0 0x7fd4ac0769c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd4d01448a0 tx=0x7fd4b8008040 comp rx=0 tx=0).stop 2024-04-25T11:37:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.048+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 msgr2=0x7fd4d01467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.048+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d01467d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009870 tx=0x7fd4c00046c0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 shutdown_connections 2024-04-25T11:37:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd4ac0744b0 0x7fd4ac0769c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4d0144370 0x7fd4d00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd4d00b4cd0 0x7fd4d0070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 --2- 172.21.15.19:0/2640916602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4d006c5c0 0x7fd4d01467d0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 >> 172.21.15.19:0/2640916602 conn(0x7fd4d00d1720 msgr2=0x7fd4d015c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 shutdown_connections 2024-04-25T11:37:50.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.049+0000 7fd4d4a00640 1 -- 172.21.15.19:0/2640916602 wait complete. 2024-04-25T11:37:50.067 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:37:50.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.259+0000 7fc27a32cf40 1 Processor -- start 2024-04-25T11:37:50.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.259+0000 7fc27a32cf40 1 -- start start 2024-04-25T11:37:50.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc27a32cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2580022d0 0x55c56cc736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc27a32cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 0x55c56cd10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc27a32cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c56cd18250 0x55c56cc8cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc27a32cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c56cbe8d20 con 0x7fc2580022d0 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc27a32cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c56cbecd60 con 0x55c56cd18250 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc27a32cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c56cc8b3f0 con 0x55c56cd187a0 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc272a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2580022d0 0x55c56cc736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc272a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2580022d0 0x55c56cc736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48944/0 (socket says 172.21.15.19:48944) 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.260+0000 7fc272a00640 1 -- 172.21.15.19:0/3217650404 learned_addr learned my addr 172.21.15.19:0/3217650404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc272a00640 1 -- 172.21.15.19:0/3217650404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 msgr2=0x55c56cd10eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc272a00640 1 --2- 172.21.15.19:0/3217650404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 0x55c56cd10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc272a00640 1 -- 172.21.15.19:0/3217650404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c56cd18250 msgr2=0x55c56cc8cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc272a00640 1 --2- 172.21.15.19:0/3217650404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c56cd18250 0x55c56cc8cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc272a00640 1 -- 172.21.15.19:0/3217650404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c56cc8eaa0 con 0x7fc2580022d0 2024-04-25T11:37:50.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc272a00640 1 --2- 172.21.15.19:0/3217650404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2580022d0 0x55c56cc736c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc264046300 tx=0x7fc26404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610869235a3e413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc271600640 1 -- 172.21.15.19:0/3217650404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26400a210 con 0x7fc2580022d0 2024-04-25T11:37:50.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc271600640 1 -- 172.21.15.19:0/3217650404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26400a3b0 con 0x7fc2580022d0 2024-04-25T11:37:50.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.261+0000 7fc271600640 1 -- 172.21.15.19:0/3217650404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26404da30 con 0x7fc2580022d0 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 -- 172.21.15.19:0/3217650404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2580022d0 msgr2=0x55c56cc736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/3217650404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2580022d0 0x55c56cc736c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc264046300 tx=0x7fc26404f000 comp rx=0 tx=0).stop 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 -- 172.21.15.19:0/3217650404 shutdown_connections 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/3217650404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c56cd18250 0x55c56cc8cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/3217650404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 0x55c56cd10eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/3217650404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2580022d0 0x55c56cc736c0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 -- 172.21.15.19:0/3217650404 >> 172.21.15.19:0/3217650404 conn(0x55c56cb64ac0 msgr2=0x55c56cc72900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 -- 172.21.15.19:0/3217650404 shutdown_connections 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.262+0000 7fc27a32cf40 1 -- 172.21.15.19:0/3217650404 wait complete. 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.263+0000 7fc27a32cf40 1 Processor -- start 2024-04-25T11:37:50.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.264+0000 7fc27a32cf40 1 -- start start 2024-04-25T11:37:50.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc27a32cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c56cd18250 0x55c56cda9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc27a32cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 0x55c56cd0dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc27a32cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2580022d0 0x55c56cd1a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc27a32cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c56cd19290 con 0x55c56cd18250 2024-04-25T11:37:50.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc27a32cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c56cd113f0 con 0x7fc2580022d0 2024-04-25T11:37:50.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc27a32cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c56cdb0e70 con 0x55c56cd187a0 2024-04-25T11:37:50.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc272a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c56cd18250 0x55c56cda9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc272a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c56cd18250 0x55c56cda9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48956/0 (socket says 172.21.15.19:48956) 2024-04-25T11:37:50.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.265+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 learned_addr learned my addr 172.21.15.19:0/2010391683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:50.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 0x55c56cd0dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 msgr2=0x55c56cd0dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc273400640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2580022d0 0x55c56cd1a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc272a00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 0x55c56cd0dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2580022d0 msgr2=0x55c56cd1a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc272a00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2580022d0 0x55c56cd1a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c56cc86750 con 0x55c56cd18250 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c56cd187a0 0x55c56cd0dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc272a00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c56cd18250 0x55c56cda9fa0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc264025fa0 tx=0x7fc264003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.266+0000 7fc26be00640 1 -- 172.21.15.19:0/2010391683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264071020 con 0x55c56cd18250 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.267+0000 7fc26be00640 1 -- 172.21.15.19:0/2010391683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26404d810 con 0x55c56cd18250 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.267+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c56cc86a30 con 0x55c56cd18250 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.267+0000 7fc26be00640 1 -- 172.21.15.19:0/2010391683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26404dbe0 con 0x55c56cd18250 2024-04-25T11:37:50.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.267+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c56cda97a0 con 0x55c56cd18250 2024-04-25T11:37:50.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.269+0000 7fc26be00640 1 -- 172.21.15.19:0/2010391683 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc264074020 con 0x55c56cd18250 2024-04-25T11:37:50.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.270+0000 7fc26be00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc250072e90 0x7fc2500753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.270+0000 7fc26be00640 1 -- 172.21.15.19:0/2010391683 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7fc2640d9230 con 0x55c56cd18250 2024-04-25T11:37:50.258 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:37:50.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.270+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c56cc8cfe0 0x55c56cc8d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.270+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbc6a0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.271+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc250072e90 0x7fc2500753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.271+0000 7fc273400640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c56cc8cfe0 0x55c56cc8d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.272+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc250072e90 0x7fc2500753a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x55c56cc715c0 tx=0x7fc25c002580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.272+0000 7fc273400640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c56cc8cfe0 0x55c56cc8d3f0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.275+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v106'412 uv386 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.275+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdbca40 con 0x55c56cc8cfe0 2024-04-25T11:37:50.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.278+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v106'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.278+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbcde0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.281+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v106'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.270 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:37:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.281+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbe190 con 0x55c56cc8cfe0 2024-04-25T11:37:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.285+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v106'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.285+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbe530 con 0x55c56cc8cfe0 2024-04-25T11:37:50.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.288+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v106'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.288+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd180 con 0x55c56cc8cfe0 2024-04-25T11:37:50.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.291+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v106'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.291+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd520 con 0x55c56cc8cfe0 2024-04-25T11:37:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.294+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v106'418 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.294+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.297+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v106'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.298+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.301+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v106'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.289 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:37:50.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.301+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.304+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v106'421 uv421 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.304+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.308+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v106'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.308+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.311+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v106'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.312+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.315+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v106'424 uv399 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.315+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.320+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v106'425 uv425 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.320+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.323+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v106'426 uv426 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.312 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:37:50.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.324+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.327+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v106'427 uv427 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.327+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.331+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v106'428 uv428 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.331+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.336+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v106'429 uv429 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.336+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.340+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v106'430 uv430 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.340+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.344+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v106'431 uv405 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.344+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.349+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v106'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.349+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.352+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v106'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.341 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:37:50.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.353+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.357+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v106'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.357+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.361+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v106'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.361+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.365+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v106'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.365+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.369+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v106'437 uv411 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.369+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.373+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v106'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.373+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.366 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:37:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.378+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v106'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.378+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.382+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v106'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.382+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.386+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v106'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.386+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.390+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v106'442 uv442 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.390+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c56cdbdcc0 0x55c56cdc0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.390+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdbd8c0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.391+0000 7fc272a00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c56cdbdcc0 0x55c56cdc0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.392+0000 7fc272a00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c56cdbdcc0 0x55c56cdc0df0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.396+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v106'308 uv291 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.396+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdc1330 con 0x55c56cdbdcc0 2024-04-25T11:37:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.400+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v106'309 uv309 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.400+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc16d0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.393 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:37:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.404+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v106'310 uv310 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.405+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc1a70 con 0x55c56cdbdcc0 2024-04-25T11:37:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.409+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v106'311 uv311 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.409+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc1e10 con 0x55c56cdbdcc0 2024-04-25T11:37:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.413+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v106'312 uv312 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.414+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc21b0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.418+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v106'313 uv296 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.418+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdc2550 con 0x55c56cdbdcc0 2024-04-25T11:37:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.422+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v106'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.422+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.415 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:37:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.426+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v106'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.427+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.431+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v106'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.431+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.435+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v106'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.435+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.439+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v106'318 uv301 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.439+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.444+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v106'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.444+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.436 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:37:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.448+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v106'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.449+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc6fe0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.454+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v106'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.454+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc6fe0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.458+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v106'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.458+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.462+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v106'323 uv307 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.463+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.466+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v106'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.467+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.470+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v106'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.459 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:37:50.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.471+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc3900 con 0x55c56cdbdcc0 2024-04-25T11:37:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.475+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v106'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.475+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc3900 con 0x55c56cdbdcc0 2024-04-25T11:37:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.479+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v106'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.479+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc28f0 con 0x55c56cdbdcc0 2024-04-25T11:37:50.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.483+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v106'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdbdcc0 2024-04-25T11:37:50.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.484+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c56cdc2e00 0x55c56cdc5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.484+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc5760 con 0x55c56cdc2e00 2024-04-25T11:37:50.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.485+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c56cdc2e00 0x55c56cdc5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.486+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c56cdc2e00 0x55c56cdc5220 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.490+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.490+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdc5b00 con 0x55c56cdc2e00 2024-04-25T11:37:50.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.494+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v106'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.494+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdb0170 con 0x55c56cdc2e00 2024-04-25T11:37:50.487 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:37:50.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.499+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v106'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.499+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc7890 con 0x55c56cdc2e00 2024-04-25T11:37:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.503+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.503+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc7c30 con 0x55c56cdc2e00 2024-04-25T11:37:50.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.507+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.507+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc6880 con 0x55c56cdc2e00 2024-04-25T11:37:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.511+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.511+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc6c70 con 0x55c56cc8cfe0 2024-04-25T11:37:50.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.516+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v106'66 uv65 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.516+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdc6c70 con 0x55c56cc8cfe0 2024-04-25T11:37:50.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.520+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v106'67 uv67 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.520+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc6c70 con 0x55c56cc8cfe0 2024-04-25T11:37:50.512 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:37:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.524+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v106'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.524+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdc6c70 con 0x55c56cc8cfe0 2024-04-25T11:37:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.529+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v106'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.529+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c56cdc7fd0 0x55c56cdca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.529+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdca930 con 0x55c56cdc7fd0 2024-04-25T11:37:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.530+0000 7fc273400640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c56cdc7fd0 0x55c56cdca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.531+0000 7fc273400640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c56cdc7fd0 0x55c56cdca3f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.536+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.536+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdca930 con 0x55c56cdc7fd0 2024-04-25T11:37:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.541+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v106'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.541+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdca930 con 0x55c56cdc7fd0 2024-04-25T11:37:50.534 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:37:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.545+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v106'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.546+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcb940 con 0x55c56cdc7fd0 2024-04-25T11:37:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.549+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.550+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcb940 con 0x55c56cdc7fd0 2024-04-25T11:37:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.554+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.554+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdca930 con 0x55c56cdc7fd0 2024-04-25T11:37:50.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.559+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.559+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c56cdcae40 0x55c56cdcd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.559+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcd960 con 0x55c56cdcae40 2024-04-25T11:37:50.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.560+0000 7fc272a00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c56cdcae40 0x55c56cdcd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.561+0000 7fc272a00640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c56cdcae40 0x55c56cdcd260 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.565+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v106'149 uv143 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.565+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.569+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v106'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.569+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.561 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:37:50.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.573+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v106'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.573+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdced40 con 0x55c56cdcae40 2024-04-25T11:37:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.577+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v106'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.577+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdced40 con 0x55c56cdcae40 2024-04-25T11:37:50.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.581+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v106'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.581+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.585+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v106'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.585+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.589+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v106'155 uv148 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.589+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.594+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v106'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.594+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.586 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:37:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.598+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v106'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.598+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.602+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v106'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.602+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdcdd30 con 0x55c56cdcae40 2024-04-25T11:37:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.606+0000 7fc272a00640 1 -- 172.21.15.19:0/2010391683 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v106'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26409f110 con 0x55c56cdcae40 2024-04-25T11:37:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.606+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c56cdce630 0x55c56cdd0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.607+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd0f90 con 0x55c56cdce630 2024-04-25T11:37:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.607+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c56cdce630 0x55c56cdd0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.609+0000 7fc26a000640 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c56cdce630 0x55c56cdd0a50 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.612+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v106'66 uv65 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.613+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1330 con 0x55c56cdce630 2024-04-25T11:37:50.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.616+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v106'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.616+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdce630 2024-04-25T11:37:50.608 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:37:50.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.620+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v106'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.620+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdce630 2024-04-25T11:37:50.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.624+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v106'69 uv69 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.625+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdce630 2024-04-25T11:37:50.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.628+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v106'70 uv70 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.628+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cc8cfe0 2024-04-25T11:37:50.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.632+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.633+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cc8cfe0 2024-04-25T11:37:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.636+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v106'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.637+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cc8cfe0 2024-04-25T11:37:50.629 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:37:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.641+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v106'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.641+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd2730 con 0x55c56cc8cfe0 2024-04-25T11:37:50.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.645+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.645+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd2730 con 0x55c56cc8cfe0 2024-04-25T11:37:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.649+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.649+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cc8cfe0 2024-04-25T11:37:50.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.653+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cc8cfe0 2024-04-25T11:37:50.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.654+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.658+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v106'132 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.658+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.662+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v106'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.662+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.654 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:37:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.665+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v106'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.666+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.670+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v106'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.670+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.674+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v106'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.675+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.679+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v106'137 uv131 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.679+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.683+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v106'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.683+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.675 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:37:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.687+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v106'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.687+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1720 con 0x55c56cdc2e00 2024-04-25T11:37:50.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.691+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v106'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.691+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd4530 con 0x55c56cdc2e00 2024-04-25T11:37:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.695+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v106'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdc2e00 2024-04-25T11:37:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.695+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.699+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v106'61 uv60 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.699+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.704+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v106'62 uv62 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.704+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.695 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:37:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.708+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v106'63 uv63 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.708+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.712+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v106'64 uv64 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.712+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.717+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v106'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.717+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.721+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v106'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.721+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.714 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:37:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.725+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v106'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.726+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd2be0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.729+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v106'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.730+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd2be0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.734+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v106'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.734+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.738+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v106'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc26c00d1c0 con 0x55c56cdc7fd0 2024-04-25T11:37:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.739+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.742+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v106'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.742+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.746+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v106'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.746+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.739 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:37:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.750+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v106'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.751+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.755+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v106'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.755+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.759+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v106'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.759+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.751 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:37:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.763+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v106'170 uv170 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.763+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.767+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v106'171 uv171 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.768+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.772+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v106'172 uv172 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.772+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.776+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 1.00000000 [create] v106'173 uv168 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.776+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.780+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 1.00000000 [delete] v106'174 uv174 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.780+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.772 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:37:50.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.784+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v106'175 uv175 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.784+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd2be0 con 0x55c56cdce630 2024-04-25T11:37:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.788+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v106'176 uv176 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.788+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd2be0 con 0x55c56cdce630 2024-04-25T11:37:50.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.792+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v106'177 uv177 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.792+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd1bd0 con 0x55c56cdce630 2024-04-25T11:37:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.796+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [omap-set-header] v106'178 uv178 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.796+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55c56cdd2130 con 0x55c56cdce630 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.800+0000 7fc26a000640 1 -- 172.21.15.19:0/2010391683 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v106'179 uv179 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc25c0106d0 con 0x55c56cdce630 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c56cdbdcc0 msgr2=0x55c56cdc0df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c56cdbdcc0 0x55c56cdc0df0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c56cdcae40 msgr2=0x55c56cdcd260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c56cdcae40 0x55c56cdcd260 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c56cdce630 msgr2=0x55c56cdd0a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c56cdce630 0x55c56cdd0a50 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c56cdc2e00 msgr2=0x55c56cdc5220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c56cdc2e00 0x55c56cdc5220 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c56cc8cfe0 msgr2=0x55c56cc8d3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c56cc8cfe0 0x55c56cc8d3f0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c56cdc7fd0 msgr2=0x55c56cdca3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c56cdc7fd0 0x55c56cdca3f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 reap_dead start 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 reap_dead start 2024-04-25T11:37:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc273400640 1 -- 172.21.15.19:0/2010391683 reap_dead start 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc250072e90 msgr2=0x7fc2500753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc250072e90 0x7fc2500753a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x55c56cc715c0 tx=0x7fc25c002580 comp rx=0 tx=0).stop 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c56cd18250 msgr2=0x55c56cda9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.801+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c56cd18250 0x55c56cda9fa0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc264025fa0 tx=0x7fc264003020 comp rx=0 tx=0).stop 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.802+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 shutdown_connections 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.802+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc250072e90 0x7fc2500753a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.802+0000 7fc27a32cf40 1 --2- 172.21.15.19:0/2010391683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c56cd18250 0x55c56cda9fa0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.802+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 >> 172.21.15.19:0/2010391683 conn(0x55c56cb64ac0 msgr2=0x55c56cd19980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.802+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 shutdown_connections 2024-04-25T11:37:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:50.802+0000 7fc27a32cf40 1 -- 172.21.15.19:0/2010391683 wait complete. 2024-04-25T11:37:50.818 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2024-04-25T11:37:51.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.067+0000 7f9e5760cf40 1 Processor -- start 2024-04-25T11:37:51.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.067+0000 7f9e5760cf40 1 -- start start 2024-04-25T11:37:51.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e5760cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c6981e0 0x55659c76e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e5760cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6985f0 0x55659c6dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e5760cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c771dd0 0x55659c6ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e5760cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55659c6e0e70 con 0x55659c6985f0 2024-04-25T11:37:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e5760cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55659c63ed20 con 0x55659c771dd0 2024-04-25T11:37:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e5760cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55659c6e13f0 con 0x55659c6981e0 2024-04-25T11:37:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e4f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6985f0 0x55659c6dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e4f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6985f0 0x55659c6dea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48968/0 (socket says 172.21.15.19:48968) 2024-04-25T11:37:51.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.069+0000 7f9e4f200640 1 -- 172.21.15.19:0/2162762245 learned_addr learned my addr 172.21.15.19:0/2162762245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4fc00640 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c6981e0 0x55659c76e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e50600640 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c771dd0 0x55659c6ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4f200640 1 -- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c6981e0 msgr2=0x55659c76e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4f200640 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c6981e0 0x55659c76e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4f200640 1 -- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c771dd0 msgr2=0x55659c6ddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4f200640 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c771dd0 0x55659c6ddeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4f200640 1 -- 172.21.15.19:0/2162762245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55659c6cabb0 con 0x55659c6985f0 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4fc00640 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c6981e0 0x55659c76e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e50600640 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c771dd0 0x55659c6ddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:51.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4f200640 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6985f0 0x55659c6dea70 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c025ca0 tx=0x7f9e3c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ccdefd9957fb04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4de00640 1 -- 172.21.15.19:0/2162762245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c062070 con 0x55659c6985f0 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4de00640 1 -- 172.21.15.19:0/2162762245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c054970 con 0x55659c6985f0 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.070+0000 7f9e4de00640 1 -- 172.21.15.19:0/2162762245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c054d20 con 0x55659c6985f0 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.071+0000 7f9e5760cf40 1 -- 172.21.15.19:0/2162762245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6985f0 msgr2=0x55659c6dea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.071+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6985f0 0x55659c6dea70 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c025ca0 tx=0x7f9e3c051000 comp rx=0 tx=0).stop 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.071+0000 7f9e5760cf40 1 -- 172.21.15.19:0/2162762245 shutdown_connections 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.071+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c771dd0 0x55659c6ddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.071+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6985f0 0x55659c6dea70 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.071+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/2162762245 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c6981e0 0x55659c76e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.071+0000 7f9e5760cf40 1 -- 172.21.15.19:0/2162762245 >> 172.21.15.19:0/2162762245 conn(0x55659c5baac0 msgr2=0x55659c6c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.072+0000 7f9e5760cf40 1 -- 172.21.15.19:0/2162762245 shutdown_connections 2024-04-25T11:37:51.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.072+0000 7f9e5760cf40 1 -- 172.21.15.19:0/2162762245 wait complete. 2024-04-25T11:37:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.073+0000 7f9e5760cf40 1 Processor -- start 2024-04-25T11:37:51.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.073+0000 7f9e5760cf40 1 -- start start 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.074+0000 7f9e5760cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6981e0 0x55659c766930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e5760cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c6985f0 0x55659c765100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e5760cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c771dd0 0x55659c768900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e5760cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55659c76eb60 con 0x55659c6981e0 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e5760cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55659c6ca8d0 con 0x55659c6985f0 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e5760cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55659c6dc2c0 con 0x55659c771dd0 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6981e0 0x55659c766930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6981e0 0x55659c766930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48978/0 (socket says 172.21.15.19:48978) 2024-04-25T11:37:51.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 -- 172.21.15.19:0/3645702305 learned_addr learned my addr 172.21.15.19:0/3645702305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e50600640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c771dd0 0x55659c768900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 -- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c771dd0 msgr2=0x55659c768900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55659c771dd0 0x55659c768900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 -- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c6985f0 msgr2=0x55659c765100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659c6985f0 0x55659c765100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.075+0000 7f9e4fc00640 1 -- 172.21.15.19:0/3645702305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55659c7670f0 con 0x55659c6981e0 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.076+0000 7f9e4fc00640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6981e0 0x55659c766930 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009f30 tx=0x7f9e400084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.076+0000 7f9e4ca00640 1 -- 172.21.15.19:0/3645702305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e400143f0 con 0x55659c6981e0 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.076+0000 7f9e4ca00640 1 -- 172.21.15.19:0/3645702305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e40014590 con 0x55659c6981e0 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.076+0000 7f9e4ca00640 1 -- 172.21.15.19:0/3645702305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40014940 con 0x55659c6981e0 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.076+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55659c696690 con 0x55659c6981e0 2024-04-25T11:37:51.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.076+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55659c696c50 con 0x55659c6981e0 2024-04-25T11:37:51.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.079+0000 7f9e4ca00640 1 -- 172.21.15.19:0/3645702305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9e4001b020 con 0x55659c6981e0 2024-04-25T11:37:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.080+0000 7f9e4ca00640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e30072e90 0x7f9e300753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.080+0000 7f9e4ca00640 1 -- 172.21.15.19:0/3645702305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7f9e40093df0 con 0x55659c6981e0 2024-04-25T11:37:51.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.080+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55659c76f460 0x55659c76f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.080+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e106) -- 0x55659c76fdb0 con 0x55659c76f460 2024-04-25T11:37:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.080+0000 7f9e4f200640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e30072e90 0x7f9e300753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.080+0000 7f9e50600640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55659c76f460 0x55659c76f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.081+0000 7f9e50600640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55659c76f460 0x55659c76f870 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:51.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.082+0000 7f9e4f200640 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e30072e90 0x7f9e300753a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c005ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e50600640 1 -- 172.21.15.19:0/3645702305 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 1.00000000 [delete] v106'180 uv180 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9e48001400 con 0x55659c76f460 2024-04-25T11:37:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55659c76f460 msgr2=0x55659c76f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55659c76f460 0x55659c76f870 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e30072e90 msgr2=0x7f9e300753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e30072e90 0x7f9e300753a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c005ac0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6981e0 msgr2=0x55659c766930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e5760cf40 1 --2- 172.21.15.19:0/3645702305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55659c6981e0 0x55659c766930 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009f30 tx=0x7f9e400084c0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.084+0000 7f9e50600640 1 -- 172.21.15.19:0/3645702305 reap_dead start 2024-04-25T11:37:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.085+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 shutdown_connections 2024-04-25T11:37:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.085+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 >> 172.21.15.19:0/3645702305 conn(0x55659c5baac0 msgr2=0x55659c6c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.085+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 shutdown_connections 2024-04-25T11:37:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.085+0000 7f9e5760cf40 1 -- 172.21.15.19:0/3645702305 wait complete. 2024-04-25T11:37:51.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:37:51.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.383+0000 7feaefe00640 1 Processor -- start 2024-04-25T11:37:51.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.384+0000 7feaefe00640 1 -- start start 2024-04-25T11:37:51.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.386+0000 7feaefe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feaf0144340 0x7feaf006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.386+0000 7feaefe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf015ee60 0x7feaf015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.386+0000 7feaefe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf006df20 con 0x7feaf006db10 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.386+0000 7feaefe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7feaf015f270 con 0x7feaf0144340 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.386+0000 7feaefe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feaf0144750 con 0x7feaf015ee60 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaeea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feaf0144340 0x7feaf006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaeea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feaf0144340 0x7feaf006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48988/0 (socket says 172.21.15.19:48988) 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaeea00640 1 -- 172.21.15.19:0/1414813806 learned_addr learned my addr 172.21.15.19:0/1414813806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaef400640 1 --2- 172.21.15.19:0/1414813806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf015ee60 0x7feaf015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaeea00640 1 -- 172.21.15.19:0/1414813806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf015ee60 msgr2=0x7feaf015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaeea00640 1 --2- 172.21.15.19:0/1414813806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf015ee60 0x7feaf015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaeea00640 1 -- 172.21.15.19:0/1414813806 >> v1:172.21.15.169:6789/0 conn(0x7feaf006db10 legacy=0x7feaf015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaeea00640 1 -- 172.21.15.19:0/1414813806 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf0070000 con 0x7feaf0144340 2024-04-25T11:37:51.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.387+0000 7feaef400640 1 --2- 172.21.15.19:0/1414813806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf015ee60 0x7feaf015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:51.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.388+0000 7feaeea00640 1 --2- 172.21.15.19:0/1414813806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf0144340 0x7feaf006d5d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7feae0009870 tx=0x7feae002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa25a5f272f1635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:51.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.389+0000 7feaee000640 1 -- 172.21.15.19:0/1414813806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0037070 con 0x7feaf0144340 2024-04-25T11:37:51.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.389+0000 7feaee000640 1 -- 172.21.15.19:0/1414813806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae0032440 con 0x7feaf0144340 2024-04-25T11:37:51.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.389+0000 7feaee000640 1 -- 172.21.15.19:0/1414813806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0032810 con 0x7feaf0144340 2024-04-25T11:37:51.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.389+0000 7feaefe00640 1 -- 172.21.15.19:0/1414813806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf0144340 msgr2=0x7feaf006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.389+0000 7feaefe00640 1 --2- 172.21.15.19:0/1414813806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf0144340 0x7feaf006d5d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7feae0009870 tx=0x7feae002a200 comp rx=0 tx=0).stop 2024-04-25T11:37:51.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.390+0000 7feaefe00640 1 -- 172.21.15.19:0/1414813806 shutdown_connections 2024-04-25T11:37:51.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.390+0000 7feaefe00640 1 --2- 172.21.15.19:0/1414813806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf015ee60 0x7feaf015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.390+0000 7feaefe00640 1 --2- 172.21.15.19:0/1414813806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf0144340 0x7feaf006d5d0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.390+0000 7feaefe00640 1 -- 172.21.15.19:0/1414813806 >> 172.21.15.19:0/1414813806 conn(0x7feaf0072f40 msgr2=0x7feaf0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:51.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.390+0000 7feaefe00640 1 -- 172.21.15.19:0/1414813806 shutdown_connections 2024-04-25T11:37:51.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.390+0000 7feaefe00640 1 -- 172.21.15.19:0/1414813806 wait complete. 2024-04-25T11:37:51.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.392+0000 7feaefe00640 1 Processor -- start 2024-04-25T11:37:51.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.392+0000 7feaefe00640 1 -- start start 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaefe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf006db10 0x7feaf015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaefe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaf0144340 0x7feaf013f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaefe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf015ee60 0x7feaf0158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaefe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feaf006f800 con 0x7feaf015ee60 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaefe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feaf006f520 con 0x7feaf0144340 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaefe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feaf006fae0 con 0x7feaf006db10 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf015ee60 0x7feaf0158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf015ee60 0x7feaf0158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48994/0 (socket says 172.21.15.19:48994) 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaef400640 1 -- 172.21.15.19:0/1993215598 learned_addr learned my addr 172.21.15.19:0/1993215598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.394+0000 7feaeea00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf006db10 0x7feaf015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feae7e00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaf0144340 0x7feaf013f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feaef400640 1 -- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf006db10 msgr2=0x7feaf015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feaef400640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf006db10 0x7feaf015b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feaef400640 1 -- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaf0144340 msgr2=0x7feaf013f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feaef400640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaf0144340 0x7feaf013f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feaef400640 1 -- 172.21.15.19:0/1993215598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf01458e0 con 0x7feaf015ee60 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feae7e00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaf0144340 0x7feaf013f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feaef400640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf015ee60 0x7feaf0158570 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7feae800cde0 tx=0x7feae800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feae6a00640 1 -- 172.21.15.19:0/1993215598 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae800f680 con 0x7feaf015ee60 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feae6a00640 1 -- 172.21.15.19:0/1993215598 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae800f840 con 0x7feaf015ee60 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feae6a00640 1 -- 172.21.15.19:0/1993215598 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae800fc30 con 0x7feaf015ee60 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.395+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf0145bc0 con 0x7feaf015ee60 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.396+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf0159f60 con 0x7feaf015ee60 2024-04-25T11:37:51.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.397+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf015a1c0 con 0x7feaf015ee60 2024-04-25T11:37:51.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.398+0000 7feae6a00640 1 -- 172.21.15.19:0/1993215598 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feae800e3e0 con 0x7feaf015ee60 2024-04-25T11:37:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.399+0000 7feae6a00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feacc0744e0 0x7feacc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.399+0000 7feae6a00640 1 -- 172.21.15.19:0/1993215598 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7feae8096480 con 0x7feaf015ee60 2024-04-25T11:37:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.400+0000 7feaeea00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feacc0744e0 0x7feacc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:51.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.401+0000 7feaeea00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feacc0744e0 0x7feacc0769f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7feae0034000 tx=0x7feae002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:51.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.404+0000 7feae6a00640 1 -- 172.21.15.19:0/1993215598 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feae8014030 con 0x7feaf015ee60 2024-04-25T11:37:51.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:51.648+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7feaf015b7e0 con 0x7feaf015ee60 2024-04-25T11:37:52.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.069+0000 7feae6a00640 1 -- 172.21.15.19:0/1993215598 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v182) ==== 162+0+0 (secure 0 0 0) 0x7feae80607d0 con 0x7feaf015ee60 2024-04-25T11:37:52.058 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:37:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feacc0744e0 msgr2=0x7feacc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feacc0744e0 0x7feacc0769f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7feae0034000 tx=0x7feae002e000 comp rx=0 tx=0).stop 2024-04-25T11:37:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf015ee60 msgr2=0x7feaf0158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf015ee60 0x7feaf0158570 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7feae800cde0 tx=0x7feae800a650 comp rx=0 tx=0).stop 2024-04-25T11:37:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 shutdown_connections 2024-04-25T11:37:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feacc0744e0 0x7feacc0769f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaf0144340 0x7feaf013f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaf015ee60 0x7feaf0158570 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.073+0000 7feaefe00640 1 --2- 172.21.15.19:0/1993215598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaf006db10 0x7feaf015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.074+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 >> 172.21.15.19:0/1993215598 conn(0x7feaf0072f40 msgr2=0x7feaf015e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:52.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.074+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 shutdown_connections 2024-04-25T11:37:52.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.074+0000 7feaefe00640 1 -- 172.21.15.19:0/1993215598 wait complete. 2024-04-25T11:37:52.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.378+0000 7ff3e3000640 1 Processor -- start 2024-04-25T11:37:52.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.378+0000 7ff3e3000640 1 -- start start 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3e3000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff3dc0b4830 0x7ff3dc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3e3000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff3dc0b2dc0 0x7ff3dc0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3e3000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3dc06d870 con 0x7ff3dc0d4570 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3e3000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff3dc0b4c40 con 0x7ff3dc0b4830 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3e3000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff3dc06e460 con 0x7ff3dc0b2dc0 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3db400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff3dc0b4830 0x7ff3dc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3dbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff3dc0d4570 0x7ff3dc0d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47346/0 (socket says 172.21.15.19:47346) 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3dbe00640 1 -- 172.21.15.19:0/2366846404 learned_addr learned my addr 172.21.15.19:0/2366846404 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:52.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.381+0000 7ff3db400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff3dc0b4830 0x7ff3dc06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48998/0 (socket says 172.21.15.19:48998) 2024-04-25T11:37:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.382+0000 7ff3db400640 1 -- 172.21.15.19:0/2366846404 >> v2:172.21.15.169:3300/0 conn(0x7ff3dc0b2dc0 msgr2=0x7ff3dc0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.382+0000 7ff3db400640 1 --2- 172.21.15.19:0/2366846404 >> v2:172.21.15.169:3300/0 conn(0x7ff3dc0b2dc0 0x7ff3dc0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.382+0000 7ff3db400640 1 -- 172.21.15.19:0/2366846404 >> v1:172.21.15.19:6789/0 conn(0x7ff3dc0d4570 legacy=0x7ff3dc0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.382+0000 7ff3db400640 1 -- 172.21.15.19:0/2366846404 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc06db10 con 0x7ff3dc0b4830 2024-04-25T11:37:52.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.382+0000 7ff3daa00640 1 -- 172.21.15.19:0/2366846404 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937702473 0 0) 0x7ff3dc06d870 con 0x7ff3dc0d4570 2024-04-25T11:37:52.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.383+0000 7ff3db400640 1 --2- 172.21.15.19:0/2366846404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc06e8f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc06e5e0 tx=0x7ff3cc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8595f4da59d92e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:52.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.383+0000 7ff3daa00640 1 -- 172.21.15.19:0/2366846404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc031600 con 0x7ff3dc0b4830 2024-04-25T11:37:52.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3daa00640 1 -- 172.21.15.19:0/2366846404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3cc0317a0 con 0x7ff3dc0b4830 2024-04-25T11:37:52.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3daa00640 1 -- 172.21.15.19:0/2366846404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc031b70 con 0x7ff3dc0b4830 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3e3000640 1 -- 172.21.15.19:0/2366846404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 msgr2=0x7ff3dc06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3e3000640 1 --2- 172.21.15.19:0/2366846404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc06e8f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc06e5e0 tx=0x7ff3cc02a5d0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3e3000640 1 -- 172.21.15.19:0/2366846404 shutdown_connections 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3e3000640 1 --2- 172.21.15.19:0/2366846404 >> v2:172.21.15.169:3300/0 conn(0x7ff3dc0b2dc0 0x7ff3dc0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3e3000640 1 --2- 172.21.15.19:0/2366846404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc06e8f0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3e3000640 1 -- 172.21.15.19:0/2366846404 >> 172.21.15.19:0/2366846404 conn(0x7ff3dc0d3a50 msgr2=0x7ff3dc0d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.384+0000 7ff3e3000640 1 -- 172.21.15.19:0/2366846404 shutdown_connections 2024-04-25T11:37:52.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.385+0000 7ff3e3000640 1 -- 172.21.15.19:0/2366846404 wait complete. 2024-04-25T11:37:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.387+0000 7ff3e3000640 1 Processor -- start 2024-04-25T11:37:52.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.387+0000 7ff3e3000640 1 -- start start 2024-04-25T11:37:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3e3000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3dc0b2dc0 0x7ff3dc0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3e3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3e3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3dc0d4570 0x7ff3dc1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3e3000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc0d0fb0 con 0x7ff3dc0b4830 2024-04-25T11:37:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3e3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc0d0bc0 con 0x7ff3dc0d4570 2024-04-25T11:37:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3e3000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff3dc0b3710 con 0x7ff3dc0b2dc0 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc1630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49002/0 (socket says 172.21.15.19:49002) 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3db400640 1 -- 172.21.15.19:0/3036026097 learned_addr learned my addr 172.21.15.19:0/3036026097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.389+0000 7ff3dbe00640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3dc0b2dc0 0x7ff3dc0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3db400640 1 -- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3dc0b2dc0 msgr2=0x7ff3dc0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3db400640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3dc0b2dc0 0x7ff3dc0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3db400640 1 -- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3dc0d4570 msgr2=0x7ff3dc1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3e1000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3dc0d4570 0x7ff3dc1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3db400640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3dc0d4570 0x7ff3dc1578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3db400640 1 -- 172.21.15.19:0/3036026097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc0c84d0 con 0x7ff3dc0b4830 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3dbe00640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3dc0b2dc0 0x7ff3dc0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3e1000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3dc0d4570 0x7ff3dc1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3db400640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc1630c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009e50 tx=0x7ff3cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:52.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3d3e00640 1 -- 172.21.15.19:0/3036026097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc040070 con 0x7ff3dc0b4830 2024-04-25T11:37:52.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3d3e00640 1 -- 172.21.15.19:0/3036026097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3cc030be0 con 0x7ff3dc0b4830 2024-04-25T11:37:52.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3dc0a0980 con 0x7ff3dc0b4830 2024-04-25T11:37:52.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.390+0000 7ff3d3e00640 1 -- 172.21.15.19:0/3036026097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc009450 con 0x7ff3dc0b4830 2024-04-25T11:37:52.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.391+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3dc0a0f40 con 0x7ff3dc0b4830 2024-04-25T11:37:52.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.392+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3dc0b6640 con 0x7ff3dc0b4830 2024-04-25T11:37:52.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.393+0000 7ff3d3e00640 1 -- 172.21.15.19:0/3036026097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3cc039040 con 0x7ff3dc0b4830 2024-04-25T11:37:52.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.394+0000 7ff3d3e00640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3b80744b0 0x7ff3b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:52.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.394+0000 7ff3d3e00640 1 -- 172.21.15.19:0/3036026097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6913+0+0 (secure 0 0 0) 0x7ff3cc0b6190 con 0x7ff3dc0b4830 2024-04-25T11:37:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.395+0000 7ff3dbe00640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3b80744b0 0x7ff3b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:52.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.396+0000 7ff3dbe00640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3b80744b0 0x7ff3b80769c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400a850 tx=0x7ff3c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.399+0000 7ff3d3e00640 1 -- 172.21.15.19:0/3036026097 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3dc0b6640 con 0x7ff3dc0b4830 2024-04-25T11:37:52.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.642+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3dc050740 con 0x7ff3dc0b4830 2024-04-25T11:37:52.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.643+0000 7ff3d3e00640 1 -- 172.21.15.19:0/3036026097 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) ==== 95+0+5187 (secure 0 0 0) 0x7ff3cc0803c0 con 0x7ff3dc0b4830 2024-04-25T11:37:52.632 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:52.632 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":183,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:52.077192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9039":{"gid":9039,"name":"a","rank":0,"incarnation":183,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/3471202301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3471202301},{"type":"v1","addr":"172.21.15.19:6835","nonce":3471202301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:52.632 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 183 2024-04-25T11:37:52.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.645+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3b80744b0 msgr2=0x7ff3b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.645+0000 7ff3e3000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3b80744b0 0x7ff3b80769c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400a850 tx=0x7ff3c4008040 comp rx=0 tx=0).stop 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.645+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 msgr2=0x7ff3dc1630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.645+0000 7ff3e3000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc1630c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009e50 tx=0x7ff3cc004a40 comp rx=0 tx=0).stop 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 shutdown_connections 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3b80744b0 0x7ff3b80769c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3dc0d4570 0x7ff3dc1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3dc0b4830 0x7ff3dc1630c0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 --2- 172.21.15.19:0/3036026097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3dc0b2dc0 0x7ff3dc0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 >> 172.21.15.19:0/3036026097 conn(0x7ff3dc0d3a50 msgr2=0x7ff3dc0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 shutdown_connections 2024-04-25T11:37:52.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:52.646+0000 7ff3e3000640 1 -- 172.21.15.19:0/3036026097 wait complete. 2024-04-25T11:37:52.659 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:37:55.113 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:55.125+0000 7f0c95a00640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-04-25T11:37:55.113 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:37:55.125+0000 7f0c95a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-25T11:37:55.114 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:55.126+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:37:55.115 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:37:55.126+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:37:55.133 INFO:tasks.ceph.mds.a.smithi019.stderr: -19> 2024-04-25T11:37:55.125+0000 7f0c95a00640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-04-25T11:37:55.133 INFO:tasks.ceph.mds.a.smithi019.stderr: -17> 2024-04-25T11:37:55.125+0000 7f0c95a00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-25T11:37:55.277 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:37:57.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.968+0000 7fe9dec00640 1 Processor -- start 2024-04-25T11:37:57.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.968+0000 7fe9dec00640 1 -- start start 2024-04-25T11:37:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.970+0000 7fe9dec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.970+0000 7fe9dec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9d8162610 0x7fe9d8144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.970+0000 7fe9dec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d806fc70 con 0x7fe9d8161af0 2024-04-25T11:37:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.970+0000 7fe9dec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d806ff50 con 0x7fe9d8162610 2024-04-25T11:37:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.970+0000 7fe9dec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d8070230 con 0x7fe9d8144350 2024-04-25T11:37:57.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.971+0000 7fe9dd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:57.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.971+0000 7fe9de200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9d8162610 0x7fe9d8144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:57.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.971+0000 7fe9dd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:60370/0 (socket says 172.21.15.19:60370) 2024-04-25T11:37:57.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.971+0000 7fe9dd800640 1 -- 172.21.15.19:0/1152793514 learned_addr learned my addr 172.21.15.19:0/1152793514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:57.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.971+0000 7fe9dce00640 1 --1- 172.21.15.19:0/1152793514 >> v1:172.21.15.169:6789/0 conn(0x7fe9d8161af0 0x7fe9d8161f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:32828/0 (socket says 172.21.15.19:32828) 2024-04-25T11:37:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.972+0000 7fe9de200640 1 -- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 msgr2=0x7fe9d80d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.972+0000 7fe9de200640 1 --2- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.972+0000 7fe9de200640 1 -- 172.21.15.19:0/1152793514 >> v1:172.21.15.169:6789/0 conn(0x7fe9d8161af0 legacy=0x7fe9d8161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.972+0000 7fe9de200640 1 -- 172.21.15.19:0/1152793514 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d8070510 con 0x7fe9d8162610 2024-04-25T11:37:57.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.972+0000 7fe9dd800640 1 --2- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:37:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.974+0000 7fe9de200640 1 --2- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 0x7fe9d8144e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00097a0 tx=0x7fe9d002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=926202a877b35fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.974+0000 7fe9d7e00640 1 -- 172.21.15.19:0/1152793514 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0037070 con 0x7fe9d8162610 2024-04-25T11:37:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9d7e00640 1 -- 172.21.15.19:0/1152793514 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d0032440 con 0x7fe9d8162610 2024-04-25T11:37:57.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9d7e00640 1 -- 172.21.15.19:0/1152793514 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0032810 con 0x7fe9d8162610 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9dec00640 1 -- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 msgr2=0x7fe9d8144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9dec00640 1 --2- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 0x7fe9d8144e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00097a0 tx=0x7fe9d002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9dec00640 1 -- 172.21.15.19:0/1152793514 shutdown_connections 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9dec00640 1 --2- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 0x7fe9d8144e60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9dec00640 1 --2- 172.21.15.19:0/1152793514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.975+0000 7fe9dec00640 1 -- 172.21.15.19:0/1152793514 >> 172.21.15.19:0/1152793514 conn(0x7fe9d80d1720 msgr2=0x7fe9d80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.976+0000 7fe9dec00640 1 -- 172.21.15.19:0/1152793514 shutdown_connections 2024-04-25T11:37:57.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.976+0000 7fe9dec00640 1 -- 172.21.15.19:0/1152793514 wait complete. 2024-04-25T11:37:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.979+0000 7fe9dec00640 1 Processor -- start 2024-04-25T11:37:57.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.979+0000 7fe9dec00640 1 -- start start 2024-04-25T11:37:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d8161af0 0x7fe9d815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:57.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 0x7fe9d8145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d8145520 con 0x7fe9d8161af0 2024-04-25T11:37:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d81453a0 con 0x7fe9d8162610 2024-04-25T11:37:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d81456a0 con 0x7fe9d8144350 2024-04-25T11:37:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d8161af0 0x7fe9d815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d8161af0 0x7fe9d815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42896/0 (socket says 172.21.15.19:42896) 2024-04-25T11:37:57.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.981+0000 7fe9dce00640 1 -- 172.21.15.19:0/2274408323 learned_addr learned my addr 172.21.15.19:0/2274408323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9dd800640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9de200640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 0x7fe9d8145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9dce00640 1 -- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 msgr2=0x7fe9d80b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9dce00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9dce00640 1 -- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 msgr2=0x7fe9d8145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9dce00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 0x7fe9d8145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9dce00640 1 -- 172.21.15.19:0/2274408323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d815eb20 con 0x7fe9d8161af0 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9dce00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d8161af0 0x7fe9d815e2f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c700 tx=0x7fe9c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.982+0000 7fe9d5600640 1 -- 172.21.15.19:0/2274408323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4018070 con 0x7fe9d8161af0 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.983+0000 7fe9d5600640 1 -- 172.21.15.19:0/2274408323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c4004d80 con 0x7fe9d8161af0 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.983+0000 7fe9d5600640 1 -- 172.21.15.19:0/2274408323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400a880 con 0x7fe9d8161af0 2024-04-25T11:37:57.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.983+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d815ee00 con 0x7fe9d8161af0 2024-04-25T11:37:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.983+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d80aa3d0 con 0x7fe9d8161af0 2024-04-25T11:37:57.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.984+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d815f320 con 0x7fe9d8161af0 2024-04-25T11:37:57.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.986+0000 7fe9d5600640 1 -- 172.21.15.19:0/2274408323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe9c401b070 con 0x7fe9d8161af0 2024-04-25T11:37:57.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.987+0000 7fe9d5600640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9b80744b0 0x7fe9b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.988+0000 7fe9d5600640 1 -- 172.21.15.19:0/2274408323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7fe9c4094c20 con 0x7fe9d8161af0 2024-04-25T11:37:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.988+0000 7fe9dd800640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9b80744b0 0x7fe9b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:57.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.990+0000 7fe9dd800640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9b80744b0 0x7fe9b80769c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8147d40 tx=0x7fe9c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:57.993+0000 7fe9d5600640 1 -- 172.21.15.19:0/2274408323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9d815f320 con 0x7fe9d8161af0 2024-04-25T11:37:58.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.237+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9d815e8b0 con 0x7fe9d8161af0 2024-04-25T11:37:58.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.238+0000 7fe9d5600640 1 -- 172.21.15.19:0/2274408323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 188 v188) ==== 95+0+5178 (secure 0 0 0) 0x7fe9c405ee00 con 0x7fe9d8161af0 2024-04-25T11:37:58.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:58.227 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":188,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9108,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2271566376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2271566376},{"type":"v1","addr":"172.21.15.19:6835","nonce":2271566376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":188}],"filesystems":[{"mdsmap":{"epoch":187,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:55.128296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 188 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9b80744b0 msgr2=0x7fe9b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9b80744b0 0x7fe9b80769c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8147d40 tx=0x7fe9c8008040 comp rx=0 tx=0).stop 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d8161af0 msgr2=0x7fe9d815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d8161af0 0x7fe9d815e2f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c700 tx=0x7fe9c400cbc0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 shutdown_connections 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9b80744b0 0x7fe9b80769c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9d8162610 0x7fe9d8145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d8161af0 0x7fe9d815e2f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 --2- 172.21.15.19:0/2274408323 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9d8144350 0x7fe9d80b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 >> 172.21.15.19:0/2274408323 conn(0x7fe9d80d1720 msgr2=0x7fe9d813da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 shutdown_connections 2024-04-25T11:37:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.241+0000 7fe9dec00640 1 -- 172.21.15.19:0/2274408323 wait complete. 2024-04-25T11:37:58.255 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:37:58.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:58.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.534+0000 7f61cb200640 1 Processor -- start 2024-04-25T11:37:58.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.534+0000 7f61cb200640 1 -- start start 2024-04-25T11:37:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.536+0000 7f61cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc06c5c0 0x7f61cc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:58.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.536+0000 7f61cb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61cc144370 0x7f61cc0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:58.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.536+0000 7f61cb200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61cc0b50e0 con 0x7f61cc0b4cd0 2024-04-25T11:37:58.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.536+0000 7f61cb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f61cc0d1230 con 0x7f61cc144370 2024-04-25T11:37:58.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.536+0000 7f61cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc0d06b0 con 0x7f61cc06c5c0 2024-04-25T11:37:58.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61cc144370 0x7f61cc0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f61cc144370 0x7f61cc0b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42902/0 (socket says 172.21.15.19:42902) 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9400640 1 -- 172.21.15.19:0/3477296784 learned_addr learned my addr 172.21.15.19:0/3477296784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9e00640 1 --2- 172.21.15.19:0/3477296784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc06c5c0 0x7f61cc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9400640 1 -- 172.21.15.19:0/3477296784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc06c5c0 msgr2=0x7f61cc0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9400640 1 --2- 172.21.15.19:0/3477296784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc06c5c0 0x7f61cc0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9400640 1 -- 172.21.15.19:0/3477296784 >> v1:172.21.15.169:6789/0 conn(0x7f61cc0b4cd0 legacy=0x7f61cc161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9400640 1 -- 172.21.15.19:0/3477296784 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc0d0990 con 0x7f61cc144370 2024-04-25T11:37:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.537+0000 7f61c9e00640 1 --2- 172.21.15.19:0/3477296784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc06c5c0 0x7f61cc0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.538+0000 7f61c9400640 1 --2- 172.21.15.19:0/3477296784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc144370 0x7f61cc0b4790 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009e50 tx=0x7f61b802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63aedd314bf243a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.539+0000 7f61c8a00640 1 -- 172.21.15.19:0/3477296784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8036070 con 0x7f61cc144370 2024-04-25T11:37:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.539+0000 7f61c8a00640 1 -- 172.21.15.19:0/3477296784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8031600 con 0x7f61cc144370 2024-04-25T11:37:58.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.539+0000 7f61c8a00640 1 -- 172.21.15.19:0/3477296784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b80319d0 con 0x7f61cc144370 2024-04-25T11:37:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 -- 172.21.15.19:0/3477296784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc144370 msgr2=0x7f61cc0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 --2- 172.21.15.19:0/3477296784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc144370 0x7f61cc0b4790 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009e50 tx=0x7f61b802a6c0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 -- 172.21.15.19:0/3477296784 shutdown_connections 2024-04-25T11:37:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 --2- 172.21.15.19:0/3477296784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc144370 0x7f61cc0b4790 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 --2- 172.21.15.19:0/3477296784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc06c5c0 0x7f61cc0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 -- 172.21.15.19:0/3477296784 >> 172.21.15.19:0/3477296784 conn(0x7f61cc0d1720 msgr2=0x7f61cc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 -- 172.21.15.19:0/3477296784 shutdown_connections 2024-04-25T11:37:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.540+0000 7f61cb200640 1 -- 172.21.15.19:0/3477296784 wait complete. 2024-04-25T11:37:58.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.542+0000 7f61cb200640 1 Processor -- start 2024-04-25T11:37:58.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.542+0000 7f61cb200640 1 -- start start 2024-04-25T11:37:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61cb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc06c5c0 0x7f61cc13c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61cb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc0b4cd0 0x7f61cc159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc144370 0x7f61cc143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61cb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0d02d0 con 0x7f61cc06c5c0 2024-04-25T11:37:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61cb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc0d0f50 con 0x7f61cc0b4cd0 2024-04-25T11:37:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f61cc0d0450 con 0x7f61cc144370 2024-04-25T11:37:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc06c5c0 0x7f61cc13c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc06c5c0 0x7f61cc13c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42906/0 (socket says 172.21.15.19:42906) 2024-04-25T11:37:58.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.544+0000 7f61c9e00640 1 -- 172.21.15.19:0/1235354777 learned_addr learned my addr 172.21.15.19:0/1235354777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c9400640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc0b4cd0 0x7f61cc159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61ca800640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc144370 0x7f61cc143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c9e00640 1 -- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc144370 msgr2=0x7f61cc143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c9e00640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc144370 0x7f61cc143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c9e00640 1 -- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc0b4cd0 msgr2=0x7f61cc159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c9e00640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc0b4cd0 0x7f61cc159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c9e00640 1 -- 172.21.15.19:0/1235354777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc159650 con 0x7f61cc06c5c0 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c9e00640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc06c5c0 0x7f61cc13c2f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c820 tx=0x7f61bc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c2000640 1 -- 172.21.15.19:0/1235354777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc015070 con 0x7f61cc06c5c0 2024-04-25T11:37:58.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c2000640 1 -- 172.21.15.19:0/1235354777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61bc007a30 con 0x7f61cc06c5c0 2024-04-25T11:37:58.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61c2000640 1 -- 172.21.15.19:0/1235354777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc007de0 con 0x7f61cc06c5c0 2024-04-25T11:37:58.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.545+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc161090 con 0x7f61cc06c5c0 2024-04-25T11:37:58.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.546+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc161650 con 0x7f61cc06c5c0 2024-04-25T11:37:58.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.547+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc050740 con 0x7f61cc06c5c0 2024-04-25T11:37:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.548+0000 7f61c2000640 1 -- 172.21.15.19:0/1235354777 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61bc018070 con 0x7f61cc06c5c0 2024-04-25T11:37:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.549+0000 7f61c2000640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:58.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.549+0000 7f61c2000640 1 -- 172.21.15.19:0/1235354777 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7f61bc095630 con 0x7f61cc06c5c0 2024-04-25T11:37:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.550+0000 7f61c9400640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.551+0000 7f61c9400640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f61b8033000 tx=0x7f61b8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.554+0000 7f61c2000640 1 -- 172.21.15.19:0/1235354777 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61bc05f8b0 con 0x7f61cc06c5c0 2024-04-25T11:37:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.801+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61cc0b5920 con 0x7f61cc06c5c0 2024-04-25T11:37:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.802+0000 7f61c2000640 1 -- 172.21.15.19:0/1235354777 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 188 v188) ==== 95+0+5178 (secure 0 0 0) 0x7f61bc05f1b0 con 0x7f61cc06c5c0 2024-04-25T11:37:58.791 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:58.791 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":188,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9108,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2271566376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2271566376},{"type":"v1","addr":"172.21.15.19:6835","nonce":2271566376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":188}],"filesystems":[{"mdsmap":{"epoch":187,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:55.128296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 188 2024-04-25T11:37:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 msgr2=0x7f61ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f61b8033000 tx=0x7f61b8009000 comp rx=0 tx=0).stop 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc06c5c0 msgr2=0x7f61cc13c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc06c5c0 0x7f61cc13c2f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c820 tx=0x7f61bc00cce0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 shutdown_connections 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f61ac0744b0 0x7f61ac0769c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61cc0b4cd0 0x7f61cc159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61cc144370 0x7f61cc143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 --2- 172.21.15.19:0/1235354777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f61cc06c5c0 0x7f61cc13c2f0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 >> 172.21.15.19:0/1235354777 conn(0x7f61cc0d1720 msgr2=0x7f61cc06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.805+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 shutdown_connections 2024-04-25T11:37:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:58.806+0000 7f61cb200640 1 -- 172.21.15.19:0/1235354777 wait complete. 2024-04-25T11:37:58.820 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:37:58.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:37:59.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.114+0000 7f323f200640 1 Processor -- start 2024-04-25T11:37:59.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.114+0000 7f323f200640 1 -- start start 2024-04-25T11:37:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.116+0000 7f323f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f324006c5c0 0x7f32400cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.116+0000 7f323f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3240144370 0x7f32400b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.116+0000 7f323f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32400b4cd0 0x7f3240161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.116+0000 7f323f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f32400b50e0 con 0x7f32400b4cd0 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.116+0000 7f323f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f32400d1230 con 0x7f324006c5c0 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.116+0000 7f323f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32400d06b0 con 0x7f3240144370 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32400b4cd0 0x7f3240161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32400b4cd0 0x7f3240161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42916/0 (socket says 172.21.15.19:42916) 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 -- 172.21.15.19:0/3584707254 learned_addr learned my addr 172.21.15.19:0/3584707254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323de00640 1 --2- 172.21.15.19:0/3584707254 >> v2:172.21.15.169:3300/0 conn(0x7f324006c5c0 0x7f32400cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f3237e00640 1 --2- 172.21.15.19:0/3584707254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3240144370 0x7f32400b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 -- 172.21.15.19:0/3584707254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3240144370 msgr2=0x7f32400b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 --2- 172.21.15.19:0/3584707254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3240144370 0x7f32400b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 -- 172.21.15.19:0/3584707254 >> v2:172.21.15.169:3300/0 conn(0x7f324006c5c0 msgr2=0x7f32400cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 --2- 172.21.15.19:0/3584707254 >> v2:172.21.15.169:3300/0 conn(0x7f324006c5c0 0x7f32400cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323e800640 1 -- 172.21.15.19:0/3584707254 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32400d0990 con 0x7f32400b4cd0 2024-04-25T11:37:59.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.117+0000 7f323de00640 1 --2- 172.21.15.19:0/3584707254 >> v2:172.21.15.169:3300/0 conn(0x7f324006c5c0 0x7f32400cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:59.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.119+0000 7f323e800640 1 --2- 172.21.15.19:0/3584707254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4cd0 0x7f3240161a10 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f32400726b0 tx=0x7f323802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4ab8177dcd9e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:59.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.119+0000 7f323d400640 1 -- 172.21.15.19:0/3584707254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238036070 con 0x7f32400b4cd0 2024-04-25T11:37:59.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.119+0000 7f323d400640 1 -- 172.21.15.19:0/3584707254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3238031600 con 0x7f32400b4cd0 2024-04-25T11:37:59.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.119+0000 7f323d400640 1 -- 172.21.15.19:0/3584707254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32380319d0 con 0x7f32400b4cd0 2024-04-25T11:37:59.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 -- 172.21.15.19:0/3584707254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4cd0 msgr2=0x7f3240161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 --2- 172.21.15.19:0/3584707254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4cd0 0x7f3240161a10 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f32400726b0 tx=0x7f323802a6a0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 -- 172.21.15.19:0/3584707254 shutdown_connections 2024-04-25T11:37:59.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 --2- 172.21.15.19:0/3584707254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32400b4cd0 0x7f3240161a10 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 --2- 172.21.15.19:0/3584707254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3240144370 0x7f32400b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 --2- 172.21.15.19:0/3584707254 >> v2:172.21.15.169:3300/0 conn(0x7f324006c5c0 0x7f32400cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 -- 172.21.15.19:0/3584707254 >> 172.21.15.19:0/3584707254 conn(0x7f32400d1720 msgr2=0x7f32400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 -- 172.21.15.19:0/3584707254 shutdown_connections 2024-04-25T11:37:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.120+0000 7f323f200640 1 -- 172.21.15.19:0/3584707254 wait complete. 2024-04-25T11:37:59.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.123+0000 7f323f200640 1 Processor -- start 2024-04-25T11:37:59.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.123+0000 7f323f200640 1 -- start start 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324006c5c0 0x7f32400cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400b4cd0 0x7f32401621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3240144370 0x7f32400b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32400d02d0 con 0x7f324006c5c0 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f32400d0f50 con 0x7f3240144370 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f32400d0450 con 0x7f32400b4cd0 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324006c5c0 0x7f32400cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324006c5c0 0x7f32400cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42920/0 (socket says 172.21.15.19:42920) 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.125+0000 7f323de00640 1 -- 172.21.15.19:0/1167925439 learned_addr learned my addr 172.21.15.19:0/1167925439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:59.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f323de00640 1 -- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400b4cd0 msgr2=0x7f32401621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f323de00640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400b4cd0 0x7f32401621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f323de00640 1 -- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3240144370 msgr2=0x7f32400b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f323de00640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3240144370 0x7f32400b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f323de00640 1 -- 172.21.15.19:0/1167925439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324015f910 con 0x7f324006c5c0 2024-04-25T11:37:59.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f323de00640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324006c5c0 0x7f32400cab10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f323000c820 tx=0x7f323000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:59.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f3236000640 1 -- 172.21.15.19:0/1167925439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230015070 con 0x7f324006c5c0 2024-04-25T11:37:59.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f3236000640 1 -- 172.21.15.19:0/1167925439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3230007a30 con 0x7f324006c5c0 2024-04-25T11:37:59.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324013ed30 con 0x7f324006c5c0 2024-04-25T11:37:59.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.126+0000 7f3236000640 1 -- 172.21.15.19:0/1167925439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230007de0 con 0x7f324006c5c0 2024-04-25T11:37:59.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.127+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324013f2f0 con 0x7f324006c5c0 2024-04-25T11:37:59.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.128+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324006d400 con 0x7f324006c5c0 2024-04-25T11:37:59.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.130+0000 7f3236000640 1 -- 172.21.15.19:0/1167925439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3230018070 con 0x7f324006c5c0 2024-04-25T11:37:59.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.130+0000 7f3236000640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f321c0744b0 0x7f321c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.131+0000 7f3236000640 1 -- 172.21.15.19:0/1167925439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7f3230095710 con 0x7f324006c5c0 2024-04-25T11:37:59.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.131+0000 7f3237e00640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f321c0744b0 0x7f321c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.132+0000 7f3237e00640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f321c0744b0 0x7f321c0769c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32400709e0 tx=0x7f3228008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:59.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.136+0000 7f3236000640 1 -- 172.21.15.19:0/1167925439 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f323005f8f0 con 0x7f324006c5c0 2024-04-25T11:37:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.379+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f324015fae0 con 0x7f324006c5c0 2024-04-25T11:37:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.381+0000 7f3236000640 1 -- 172.21.15.19:0/1167925439 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 188 v188) ==== 95+0+5178 (secure 0 0 0) 0x7f323005f1f0 con 0x7f324006c5c0 2024-04-25T11:37:59.369 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:37:59.369 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":188,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9108,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2271566376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2271566376},{"type":"v1","addr":"172.21.15.19:6835","nonce":2271566376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":188}],"filesystems":[{"mdsmap":{"epoch":187,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:37:55.128296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:37:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 188 2024-04-25T11:37:59.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f321c0744b0 msgr2=0x7f321c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f321c0744b0 0x7f321c0769c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32400709e0 tx=0x7f3228008040 comp rx=0 tx=0).stop 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324006c5c0 msgr2=0x7f32400cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324006c5c0 0x7f32400cab10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f323000c820 tx=0x7f323000cce0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 shutdown_connections 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f321c0744b0 0x7f321c0769c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3240144370 0x7f32400b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.383+0000 7f323f200640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32400b4cd0 0x7f32401621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.384+0000 7f323f200640 1 --2- 172.21.15.19:0/1167925439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f324006c5c0 0x7f32400cab10 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.384+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 >> 172.21.15.19:0/1167925439 conn(0x7f32400d1720 msgr2=0x7f324015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.384+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 shutdown_connections 2024-04-25T11:37:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.384+0000 7f323f200640 1 -- 172.21.15.19:0/1167925439 wait complete. 2024-04-25T11:37:59.400 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2024-04-25T11:37:59.400 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2024-04-25T11:37:59.400 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:37:59.400 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:37:59.451 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:37:59.451 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:37:59.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:37:59.451 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:37:59.545 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:37:59.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:37:59.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:37:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.837+0000 7f3e50e00640 1 Processor -- start 2024-04-25T11:37:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.837+0000 7f3e50e00640 1 -- start start 2024-04-25T11:37:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.840+0000 7f3e50e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e4c06c5c0 0x7f3e4c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.840+0000 7f3e50e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c0b4ea0 0x7f3e4c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.840+0000 7f3e50e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e4c0d1230 con 0x7f3e4c144370 2024-04-25T11:37:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.840+0000 7f3e50e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3e4c0d06b0 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.840+0000 7f3e50e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4c0d0990 con 0x7f3e4c0b4ea0 2024-04-25T11:37:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.841+0000 7f3e4be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c0b4ea0 0x7f3e4c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.841+0000 7f3e4b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e4c06c5c0 0x7f3e4c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.841+0000 7f3e4aa00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f3e4c144370 0x7f3e4c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:32856/0 (socket says 172.21.15.19:32856) 2024-04-25T11:37:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.841+0000 7f3e4aa00640 1 -- 172.21.15.19:0/1650039739 learned_addr learned my addr 172.21.15.19:0/1650039739 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:37:59.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.842+0000 7f3e4b400640 1 -- 172.21.15.19:0/1650039739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c0b4ea0 msgr2=0x7f3e4c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:37:59.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.842+0000 7f3e4b400640 1 --2- 172.21.15.19:0/1650039739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c0b4ea0 0x7f3e4c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.842+0000 7f3e4b400640 1 -- 172.21.15.19:0/1650039739 >> v1:172.21.15.169:6789/0 conn(0x7f3e4c144370 legacy=0x7f3e4c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.842+0000 7f3e4b400640 1 -- 172.21.15.19:0/1650039739 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c0d0c70 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.843+0000 7f3e4b400640 1 --2- 172.21.15.19:0/1650039739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c0cfa70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0097a0 tx=0x7f3e3c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1232730793726afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:59.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.843+0000 7f3e4a000640 1 -- 172.21.15.19:0/1650039739 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c037070 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.843+0000 7f3e4a000640 1 -- 172.21.15.19:0/1650039739 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3c032440 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.843+0000 7f3e4a000640 1 -- 172.21.15.19:0/1650039739 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c032810 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.844+0000 7f3e50e00640 1 -- 172.21.15.19:0/1650039739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c06c5c0 msgr2=0x7f3e4c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.844+0000 7f3e50e00640 1 --2- 172.21.15.19:0/1650039739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c0cfa70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0097a0 tx=0x7f3e3c02a180 comp rx=0 tx=0).stop 2024-04-25T11:37:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.844+0000 7f3e50e00640 1 -- 172.21.15.19:0/1650039739 shutdown_connections 2024-04-25T11:37:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.844+0000 7f3e50e00640 1 --2- 172.21.15.19:0/1650039739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c0b4ea0 0x7f3e4c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.844+0000 7f3e50e00640 1 --2- 172.21.15.19:0/1650039739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c0cfa70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.844+0000 7f3e50e00640 1 -- 172.21.15.19:0/1650039739 >> 172.21.15.19:0/1650039739 conn(0x7f3e4c0d1720 msgr2=0x7f3e4c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.844+0000 7f3e50e00640 1 -- 172.21.15.19:0/1650039739 shutdown_connections 2024-04-25T11:37:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.845+0000 7f3e50e00640 1 -- 172.21.15.19:0/1650039739 wait complete. 2024-04-25T11:37:59.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.848+0000 7f3e50e00640 1 Processor -- start 2024-04-25T11:37:59.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.848+0000 7f3e50e00640 1 -- start start 2024-04-25T11:37:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e50e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e50e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c144370 0x7f3e4c143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e50e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c15f910 0x7f3e4c13f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e50e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c0d0450 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e50e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c0d02d0 con 0x7f3e4c144370 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e50e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4c147760 con 0x7f3e4c15f910 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42922/0 (socket says 172.21.15.19:42922) 2024-04-25T11:37:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.850+0000 7f3e4b400640 1 -- 172.21.15.19:0/2240317280 learned_addr learned my addr 172.21.15.19:0/2240317280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:37:59.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.851+0000 7f3e4aa00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c144370 0x7f3e4c143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.851+0000 7f3e4be00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c15f910 0x7f3e4c13f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.851+0000 7f3e4b400640 1 -- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c15f910 msgr2=0x7f3e4c13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.851+0000 7f3e4b400640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c15f910 0x7f3e4c13f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.851+0000 7f3e4b400640 1 -- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c144370 msgr2=0x7f3e4c143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.851+0000 7f3e4b400640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c144370 0x7f3e4c143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.851+0000 7f3e4b400640 1 -- 172.21.15.19:0/2240317280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c0cd2b0 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.852+0000 7f3e4b400640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c142fe0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c02a770 tx=0x7f3e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.852+0000 7f3e43400640 1 -- 172.21.15.19:0/2240317280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c037050 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.852+0000 7f3e43400640 1 -- 172.21.15.19:0/2240317280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3c043070 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.852+0000 7f3e43400640 1 -- 172.21.15.19:0/2240317280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c03cb30 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.852+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c0cd590 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.852+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c13ee30 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.853+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c050740 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.855+0000 7f3e43400640 1 -- 172.21.15.19:0/2240317280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e3c03ccd0 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.855+0000 7f3e43400640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e2c0744b0 0x7f3e2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.856+0000 7f3e43400640 1 -- 172.21.15.19:0/2240317280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7f3e3c0b64e0 con 0x7f3e4c06c5c0 2024-04-25T11:37:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.856+0000 7f3e4aa00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e2c0744b0 0x7f3e2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.857+0000 7f3e4aa00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e2c0744b0 0x7f3e2c0769c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c1f98c0 tx=0x7f3e38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:37:59.860+0000 7f3e43400640 1 -- 172.21.15.19:0/2240317280 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e3c080760 con 0x7f3e4c06c5c0 2024-04-25T11:38:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.106+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3e4c140d10 con 0x7f3e4c06c5c0 2024-04-25T11:38:00.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.164+0000 7f3e43400640 1 -- 172.21.15.19:0/2240317280 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v189) ==== 161+0+0 (secure 0 0 0) 0x7f3e3c080060 con 0x7f3e4c06c5c0 2024-04-25T11:38:00.153 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:00.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e2c0744b0 msgr2=0x7f3e2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:00.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e2c0744b0 0x7f3e2c0769c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c1f98c0 tx=0x7f3e38009040 comp rx=0 tx=0).stop 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e4c06c5c0 msgr2=0x7f3e4c142fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c142fe0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c02a770 tx=0x7f3e3c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 shutdown_connections 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3e2c0744b0 0x7f3e2c0769c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e4c144370 0x7f3e4c143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e4c15f910 0x7f3e4c13f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 --2- 172.21.15.19:0/2240317280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e4c06c5c0 0x7f3e4c142fe0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 >> 172.21.15.19:0/2240317280 conn(0x7f3e4c0d1720 msgr2=0x7f3e4c15c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 shutdown_connections 2024-04-25T11:38:00.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.168+0000 7f3e50e00640 1 -- 172.21.15.19:0/2240317280 wait complete. 2024-04-25T11:38:00.186 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:00.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.476+0000 7f250a800640 1 Processor -- start 2024-04-25T11:38:00.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.476+0000 7f250a800640 1 -- start start 2024-04-25T11:38:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.478+0000 7f250a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2504144370 0x7f25040cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.478+0000 7f250a800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25040d1230 con 0x7f25040b4ea0 2024-04-25T11:38:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.478+0000 7f250a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25040d06b0 con 0x7f250406c5c0 2024-04-25T11:38:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.478+0000 7f250a800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f25040d0990 con 0x7f2504144370 2024-04-25T11:38:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.479+0000 7f2508a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f25040b4ea0 0x7f250415e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57864/0 (socket says 172.21.15.19:57864) 2024-04-25T11:38:00.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.479+0000 7f2508a00640 1 -- 172.21.15.19:0/3415993724 learned_addr learned my addr 172.21.15.19:0/3415993724 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.479+0000 7f2509e00640 1 --2- 172.21.15.19:0/3415993724 >> v2:172.21.15.19:3300/0 conn(0x7f2504144370 0x7f25040cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.479+0000 7f2503e00640 1 -- 172.21.15.19:0/3415993724 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399718073 0 0) 0x7f25040d1230 con 0x7f25040b4ea0 2024-04-25T11:38:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.479+0000 7f2503e00640 1 -- 172.21.15.19:0/3415993724 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24ec003520 con 0x7f25040b4ea0 2024-04-25T11:38:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.480+0000 7f2509e00640 1 -- 172.21.15.19:0/3415993724 >> v1:172.21.15.169:6789/0 conn(0x7f250406c5c0 legacy=0x7f25040b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.480+0000 7f2509e00640 1 -- 172.21.15.19:0/3415993724 >> v1:172.21.15.19:6789/0 conn(0x7f25040b4ea0 legacy=0x7f250415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.480+0000 7f2503e00640 1 -- 172.21.15.19:0/3415993724 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 523210603 0 0) 0x7f24ec003520 con 0x7f25040b4ea0 2024-04-25T11:38:00.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.480+0000 7f2509e00640 1 -- 172.21.15.19:0/3415993724 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040d0c70 con 0x7f2504144370 2024-04-25T11:38:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.481+0000 7f2509e00640 1 --2- 172.21.15.19:0/3415993724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25040cfa70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009d90 tx=0x7f24fc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119664be8b06ee42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.481+0000 7f2503e00640 1 -- 172.21.15.19:0/3415993724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0001670 con 0x7f2504144370 2024-04-25T11:38:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.481+0000 7f2503e00640 1 -- 172.21.15.19:0/3415993724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24fc007540 con 0x7f2504144370 2024-04-25T11:38:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.481+0000 7f2503e00640 1 -- 172.21.15.19:0/3415993724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc007910 con 0x7f2504144370 2024-04-25T11:38:00.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.482+0000 7f250a800640 1 -- 172.21.15.19:0/3415993724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 msgr2=0x7f25040cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:00.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.482+0000 7f250a800640 1 --2- 172.21.15.19:0/3415993724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25040cfa70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009d90 tx=0x7f24fc02a680 comp rx=0 tx=0).stop 2024-04-25T11:38:00.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.482+0000 7f250a800640 1 -- 172.21.15.19:0/3415993724 shutdown_connections 2024-04-25T11:38:00.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.482+0000 7f250a800640 1 --2- 172.21.15.19:0/3415993724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25040cfa70 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.482+0000 7f250a800640 1 -- 172.21.15.19:0/3415993724 >> 172.21.15.19:0/3415993724 conn(0x7f25040d1720 msgr2=0x7f25040d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:00.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.482+0000 7f250a800640 1 -- 172.21.15.19:0/3415993724 shutdown_connections 2024-04-25T11:38:00.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.482+0000 7f250a800640 1 -- 172.21.15.19:0/3415993724 wait complete. 2024-04-25T11:38:00.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.485+0000 7f250a800640 1 Processor -- start 2024-04-25T11:38:00.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.485+0000 7f250a800640 1 -- start start 2024-04-25T11:38:00.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.486+0000 7f250a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250406c5c0 0x7f250415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.486+0000 7f250a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25041f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.486+0000 7f250a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2504139970 0x7f2504144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.486+0000 7f250a800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25040d0450 con 0x7f2504144370 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.486+0000 7f250a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f25040d02d0 con 0x7f250406c5c0 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.486+0000 7f250a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2504147760 con 0x7f2504139970 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25041f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25041f9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42936/0 (socket says 172.21.15.19:42936) 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 -- 172.21.15.19:0/846943570 learned_addr learned my addr 172.21.15.19:0/846943570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2509400640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250406c5c0 0x7f250415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2509e00640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2504139970 0x7f2504144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 -- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2504139970 msgr2=0x7f2504144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2504139970 0x7f2504144be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 -- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250406c5c0 msgr2=0x7f250415d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250406c5c0 0x7f250415d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 -- 172.21.15.19:0/846943570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25041f9310 con 0x7f2504144370 2024-04-25T11:38:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.487+0000 7f2508a00640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25041f9ad0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c580 tx=0x7f24f000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.488+0000 7f2501600640 1 -- 172.21.15.19:0/846943570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0017070 con 0x7f2504144370 2024-04-25T11:38:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.488+0000 7f2501600640 1 -- 172.21.15.19:0/846943570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f0005ce0 con 0x7f2504144370 2024-04-25T11:38:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.488+0000 7f2501600640 1 -- 172.21.15.19:0/846943570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0007330 con 0x7f2504144370 2024-04-25T11:38:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.488+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250415da50 con 0x7f2504144370 2024-04-25T11:38:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.488+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2504160780 con 0x7f2504144370 2024-04-25T11:38:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.489+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2504050740 con 0x7f2504144370 2024-04-25T11:38:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.490+0000 7f2501600640 1 -- 172.21.15.19:0/846943570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f24f001a070 con 0x7f2504144370 2024-04-25T11:38:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.491+0000 7f2501600640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.492+0000 7f2501600640 1 -- 172.21.15.19:0/846943570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7f24f0094c90 con 0x7f2504144370 2024-04-25T11:38:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.492+0000 7f2509400640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.493+0000 7f2509400640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f24f8000f60 tx=0x7f24f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:00.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.496+0000 7f2501600640 1 -- 172.21.15.19:0/846943570 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24f005ef10 con 0x7f2504144370 2024-04-25T11:38:00.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:00.775+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f250406ce40 con 0x7f2504144370 2024-04-25T11:38:01.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.190+0000 7f2501600640 1 -- 172.21.15.19:0/846943570 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v190) ==== 101+0+0 (secure 0 0 0) 0x7f24f00650e0 con 0x7f2504144370 2024-04-25T11:38:01.179 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:38:01.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 msgr2=0x7f24e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f24f8000f60 tx=0x7f24f8009040 comp rx=0 tx=0).stop 2024-04-25T11:38:01.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 msgr2=0x7f25041f9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25041f9ad0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c580 tx=0x7f24f000ca40 comp rx=0 tx=0).stop 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 shutdown_connections 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f24e40744b0 0x7f24e40769c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2504139970 0x7f2504144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2504144370 0x7f25041f9ad0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 --2- 172.21.15.19:0/846943570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f250406c5c0 0x7f250415d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 >> 172.21.15.19:0/846943570 conn(0x7f25040d1720 msgr2=0x7f25040d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 shutdown_connections 2024-04-25T11:38:01.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.194+0000 7f250a800640 1 -- 172.21.15.19:0/846943570 wait complete. 2024-04-25T11:38:01.214 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:01.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.412+0000 7f38e14edf40 1 Processor -- start 2024-04-25T11:38:01.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.412+0000 7f38e14edf40 1 -- start start 2024-04-25T11:38:01.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.413+0000 7f38e14edf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967195430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.413+0000 7f38e14edf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967224d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.413+0000 7f38e14edf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 0x55f96717ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.413+0000 7f38e14edf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f9670f8d20 con 0x55f967227960 2024-04-25T11:38:01.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.413+0000 7f38e14edf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f967198680 con 0x55f96719a6d0 2024-04-25T11:38:01.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.413+0000 7f38e14edf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f9672b60e0 con 0x55f967192330 2024-04-25T11:38:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967224d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967224d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42960/0 (socket says 172.21.15.19:42960) 2024-04-25T11:38:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 -- 172.21.15.19:0/3338686040 learned_addr learned my addr 172.21.15.19:0/3338686040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38da600640 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38db000640 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 0x55f96717ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 -- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 msgr2=0x55f96717ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 0x55f96717ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 -- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 msgr2=0x55f967195430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967195430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38d9c00640 1 -- 172.21.15.19:0/3338686040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96719a3e0 con 0x55f967227960 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.414+0000 7f38da600640 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967195430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.415+0000 7f38d9c00640 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967224d90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f38c8001590 tx=0x7f38c8001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7370d2d01ab9550 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.415+0000 7f38d3e00640 1 -- 172.21.15.19:0/3338686040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8016070 con 0x55f967227960 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.415+0000 7f38d3e00640 1 -- 172.21.15.19:0/3338686040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c800f730 con 0x55f967227960 2024-04-25T11:38:01.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.415+0000 7f38d3e00640 1 -- 172.21.15.19:0/3338686040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800fb20 con 0x55f967227960 2024-04-25T11:38:01.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 -- 172.21.15.19:0/3338686040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 msgr2=0x55f967224d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967224d90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f38c8001590 tx=0x7f38c8001a50 comp rx=0 tx=0).stop 2024-04-25T11:38:01.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 -- 172.21.15.19:0/3338686040 shutdown_connections 2024-04-25T11:38:01.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 0x55f96717ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967224d90 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 --2- 172.21.15.19:0/3338686040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967195430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 -- 172.21.15.19:0/3338686040 >> 172.21.15.19:0/3338686040 conn(0x55f967074ac0 msgr2=0x55f967191580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:01.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 -- 172.21.15.19:0/3338686040 shutdown_connections 2024-04-25T11:38:01.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.416+0000 7f38e14edf40 1 -- 172.21.15.19:0/3338686040 wait complete. 2024-04-25T11:38:01.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.417+0000 7f38e14edf40 1 Processor -- start 2024-04-25T11:38:01.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.417+0000 7f38e14edf40 1 -- start start 2024-04-25T11:38:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38e14edf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 0x55f96721c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38e14edf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967223330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38e14edf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967223870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38e14edf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f967199f80 con 0x55f967227960 2024-04-25T11:38:01.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38e14edf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f967197350 con 0x55f96719a6d0 2024-04-25T11:38:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38e14edf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f967193060 con 0x55f967192330 2024-04-25T11:38:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38db000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967223870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38db000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967223870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42968/0 (socket says 172.21.15.19:42968) 2024-04-25T11:38:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 learned_addr learned my addr 172.21.15.19:0/1397155118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38da600640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 0x55f96721c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.419+0000 7f38d9c00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967223330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 msgr2=0x55f96721c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38db000640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f967192330 0x55f96721c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 msgr2=0x55f967223330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38db000640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f96719a6d0 0x55f967223330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f967224000 con 0x55f967227960 2024-04-25T11:38:01.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38db000640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967223870 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f38d4006b60 tx=0x7f38d4005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38d2a00640 1 -- 172.21.15.19:0/1397155118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4017070 con 0x55f967227960 2024-04-25T11:38:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38d2a00640 1 -- 172.21.15.19:0/1397155118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d4012600 con 0x55f967227960 2024-04-25T11:38:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38d2a00640 1 -- 172.21.15.19:0/1397155118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4006e60 con 0x55f967227960 2024-04-25T11:38:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f967221f10 con 0x55f967227960 2024-04-25T11:38:01.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.420+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9672242a0 con 0x55f967227960 2024-04-25T11:38:01.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.423+0000 7f38d2a00640 1 -- 172.21.15.19:0/1397155118 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f38d401a070 con 0x55f967227960 2024-04-25T11:38:01.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.424+0000 7f38d2a00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38bc072e90 0x7f38bc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.424+0000 7f38d2a00640 1 -- 172.21.15.19:0/1397155118 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7f38d40936e0 con 0x55f967227960 2024-04-25T11:38:01.412 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:01.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.425+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f96717fa70 0x55f96721dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.425+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cc1a0 con 0x55f96717fa70 2024-04-25T11:38:01.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.425+0000 7f38da600640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38bc072e90 0x7f38bc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.425+0000 7f38d9c00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f96717fa70 0x55f96721dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.427+0000 7f38d9c00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f96717fa70 0x55f96721dc60 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.427+0000 7f38da600640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38bc072e90 0x7f38bc0753a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f38cc057000 tx=0x7f38cc052000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.430+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v107'443 uv417 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.430+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672cc540 con 0x55f96717fa70 2024-04-25T11:38:01.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.434+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v107'444 uv444 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.434+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cc8e0 con 0x55f96717fa70 2024-04-25T11:38:01.425 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:01.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.437+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v107'445 uv445 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.437+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cdc90 con 0x55f96717fa70 2024-04-25T11:38:01.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.441+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'446 uv446 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.441+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f96721e320 con 0x55f96717fa70 2024-04-25T11:38:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.444+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.444+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672ccc80 con 0x55f96717fa70 2024-04-25T11:38:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.447+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.447+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd070 con 0x55f96717fa70 2024-04-25T11:38:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.450+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v107'449 uv423 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.450+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672cd070 con 0x55f96717fa70 2024-04-25T11:38:01.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.454+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v107'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.454+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.445 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:01.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.457+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v107'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.457+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.461+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.461+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.464+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.464+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.467+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.467+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.471+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v107'455 uv430 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.471+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.475+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'456 uv456 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.475+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.467 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.479+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v107'457 uv457 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.479+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.482+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'458 uv458 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.482+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.486+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'459 uv459 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.486+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd020 con 0x55f96717fa70 2024-04-25T11:38:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.490+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'460 uv460 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.490+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd4a0 con 0x55f96717fa70 2024-04-25T11:38:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.493+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'461 uv461 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.494+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd450 con 0x55f96717fa70 2024-04-25T11:38:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.498+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v107'462 uv436 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.498+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672cd3e0 con 0x55f96717fa70 2024-04-25T11:38:01.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.502+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v107'463 uv463 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.502+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd3e0 con 0x55f96717fa70 2024-04-25T11:38:01.494 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:01.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.506+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v107'464 uv464 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.506+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd3e0 con 0x55f96717fa70 2024-04-25T11:38:01.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.510+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'465 uv465 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.510+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd3e0 con 0x55f96717fa70 2024-04-25T11:38:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.514+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'466 uv466 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.515+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f96717fa70 2024-04-25T11:38:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.519+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'467 uv467 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.519+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f96717fa70 2024-04-25T11:38:01.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.523+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v107'468 uv442 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.523+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f96717fa70 2024-04-25T11:38:01.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.527+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v107'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.527+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f96717fa70 2024-04-25T11:38:01.519 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:01.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.531+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v107'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.531+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f96717fa70 2024-04-25T11:38:01.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.535+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.535+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f96717fa70 2024-04-25T11:38:01.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.539+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.539+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f96717fa70 2024-04-25T11:38:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.544+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.544+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f9672ce030 0x55f9672d0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.544+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672cd2e0 con 0x55f9672ce030 2024-04-25T11:38:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.545+0000 7f38da600640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f9672ce030 0x55f9672d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.546+0000 7f38da600640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f9672ce030 0x55f9672d0450 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.550+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'330 uv312 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.550+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672d0990 con 0x55f9672ce030 2024-04-25T11:38:01.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.555+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'331 uv331 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.555+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d0d30 con 0x55f9672ce030 2024-04-25T11:38:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.559+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'332 uv332 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.547 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.559+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d10d0 con 0x55f9672ce030 2024-04-25T11:38:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.563+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v107'333 uv333 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.563+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1470 con 0x55f9672ce030 2024-04-25T11:38:01.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.567+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'334 uv334 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.567+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1810 con 0x55f9672ce030 2024-04-25T11:38:01.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.572+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v107'335 uv317 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.572+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672d1bb0 con 0x55f9672ce030 2024-04-25T11:38:01.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.576+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v107'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.576+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.568 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:01.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.580+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v107'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.580+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.585+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'338 uv338 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.585+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.589+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.589+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.593+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v107'340 uv322 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.593+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.598+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v107'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.598+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.602+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v107'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.590 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.602+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d6640 con 0x55f9672ce030 2024-04-25T11:38:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.608+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v107'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.608+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d6640 con 0x55f9672ce030 2024-04-25T11:38:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.612+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.612+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.616+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v107'345 uv328 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.616+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.621+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v107'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.621+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.613 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.625+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v107'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.625+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d2f60 con 0x55f9672ce030 2024-04-25T11:38:01.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.629+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.629+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d2f60 con 0x55f9672ce030 2024-04-25T11:38:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.633+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.633+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d1f50 con 0x55f9672ce030 2024-04-25T11:38:01.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.638+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672ce030 2024-04-25T11:38:01.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.638+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f9672d2460 0x55f9672d4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.638+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d4dc0 con 0x55f9672d2460 2024-04-25T11:38:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.639+0000 7f38d9c00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f9672d2460 0x55f9672d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.641+0000 7f38d9c00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f9672d2460 0x55f9672d4880 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.645+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.645+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672d4dc0 con 0x55f9672d2460 2024-04-25T11:38:01.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.649+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v107'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.649+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d4e40 con 0x55f9672d2460 2024-04-25T11:38:01.641 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.653+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v107'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.653+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d6240 con 0x55f9672d2460 2024-04-25T11:38:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.657+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.657+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d6240 con 0x55f9672d2460 2024-04-25T11:38:01.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.661+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.661+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d5230 con 0x55f9672d2460 2024-04-25T11:38:01.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.665+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.665+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d5230 con 0x55f96717fa70 2024-04-25T11:38:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.670+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'70 uv69 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.670+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672d5230 con 0x55f96717fa70 2024-04-25T11:38:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.674+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.674+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d5230 con 0x55f96717fa70 2024-04-25T11:38:01.666 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:01.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.678+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'72 uv72 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.678+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d5230 con 0x55f96717fa70 2024-04-25T11:38:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.682+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'73 uv73 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.682+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f9672d5740 0x55f9672d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.682+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d81c0 con 0x55f9672d5740 2024-04-25T11:38:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.684+0000 7f38db000640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f9672d5740 0x55f9672d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.684+0000 7f38db000640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f9672d5740 0x55f9672d7b60 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.688+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.688+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672d8640 con 0x55f9672d5740 2024-04-25T11:38:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.692+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v107'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.692+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d8b20 con 0x55f9672d5740 2024-04-25T11:38:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.696+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v107'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.684 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.696+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672da3d0 con 0x55f9672d5740 2024-04-25T11:38:01.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.700+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.700+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672da770 con 0x55f9672d5740 2024-04-25T11:38:01.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.704+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.704+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672d93c0 con 0x55f9672d5740 2024-04-25T11:38:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.708+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.709+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f9672d9cc0 0x55f9672dcb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.709+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672dd060 con 0x55f9672d9cc0 2024-04-25T11:38:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.709+0000 7f38da600640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f9672d9cc0 0x55f9672dcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.711+0000 7f38da600640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f9672d9cc0 0x55f9672dcb20 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.715+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v107'160 uv154 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.715+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672dd060 con 0x55f9672d9cc0 2024-04-25T11:38:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.719+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v107'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.719+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672dd060 con 0x55f9672d9cc0 2024-04-25T11:38:01.711 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.723+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v107'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.723+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672de4b0 con 0x55f9672d9cc0 2024-04-25T11:38:01.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.727+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.727+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672de4b0 con 0x55f9672d9cc0 2024-04-25T11:38:01.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.731+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.731+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672dd4a0 con 0x55f9672d9cc0 2024-04-25T11:38:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.735+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.735+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672dd4a0 con 0x55f9672d9cc0 2024-04-25T11:38:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.739+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v107'166 uv159 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.739+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672dd4a0 con 0x55f9672d9cc0 2024-04-25T11:38:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.743+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v107'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.744+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672dd4a0 con 0x55f9672d9cc0 2024-04-25T11:38:01.735 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.747+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v107'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.747+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672dd4a0 con 0x55f9672d9cc0 2024-04-25T11:38:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.751+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.752+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672dd4a0 con 0x55f9672d9cc0 2024-04-25T11:38:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.756+0000 7f38da600640 1 -- 172.21.15.19:0/1397155118 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38cc066070 con 0x55f9672d9cc0 2024-04-25T11:38:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.756+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f9672dd9b0 0x55f9672dfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.756+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0550 con 0x55f9672dd9b0 2024-04-25T11:38:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.757+0000 7f38d9c00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f9672dd9b0 0x55f9672dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.759+0000 7f38d9c00640 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f9672dd9b0 0x55f9672dfe30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.762+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v107'71 uv70 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.762+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e0550 con 0x55f9672dd9b0 2024-04-25T11:38:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.766+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v107'72 uv72 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.766+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672dd9b0 2024-04-25T11:38:01.758 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:01.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.770+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v107'73 uv73 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.770+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672dd9b0 2024-04-25T11:38:01.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.775+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'74 uv74 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.775+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672dd9b0 2024-04-25T11:38:01.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.778+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.779+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f96717fa70 2024-04-25T11:38:01.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.783+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.783+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f96717fa70 2024-04-25T11:38:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.787+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v107'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.787+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f96717fa70 2024-04-25T11:38:01.779 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.791+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v107'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.791+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e19b0 con 0x55f96717fa70 2024-04-25T11:38:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.795+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.795+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e19b0 con 0x55f96717fa70 2024-04-25T11:38:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.799+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.799+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f96717fa70 2024-04-25T11:38:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.803+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f96717fa70 2024-04-25T11:38:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.804+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.809+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v107'142 uv136 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.809+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.813+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v107'143 uv143 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.813+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.805 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:01.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.817+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v107'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.817+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.822+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.822+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.826+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.826+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.830+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v107'147 uv141 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.830+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.835+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v107'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.835+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.826 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:01.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.838+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v107'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.838+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e09a0 con 0x55f9672d2460 2024-04-25T11:38:01.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.843+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v107'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.843+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e3750 con 0x55f9672d2460 2024-04-25T11:38:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.848+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v107'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672d2460 2024-04-25T11:38:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.848+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e3710 con 0x55f9672d5740 2024-04-25T11:38:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.852+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v107'66 uv64 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.852+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672d5740 2024-04-25T11:38:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.856+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v107'67 uv67 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.856+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672d5740 2024-04-25T11:38:01.847 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.859+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v107'68 uv68 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.859+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672d5740 2024-04-25T11:38:01.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.864+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v107'69 uv69 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.864+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672d5740 2024-04-25T11:38:01.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.867+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v107'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.867+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672d5740 2024-04-25T11:38:01.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.872+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v107'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.872+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672d5740 2024-04-25T11:38:01.864 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:01.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.876+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v107'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.876+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e1da0 con 0x55f9672d5740 2024-04-25T11:38:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.881+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.881+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e1da0 con 0x55f9672d5740 2024-04-25T11:38:01.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.885+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v107'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.885+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672d5740 2024-04-25T11:38:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.890+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38d4019030 con 0x55f9672d5740 2024-04-25T11:38:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.890+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d90 con 0x55f9672dd9b0 2024-04-25T11:38:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.894+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v107'74 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.894+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.898+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v107'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.898+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.902+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v107'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.890 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:01.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.902+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.907+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v107'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.907+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.911+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v107'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.912+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.916+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v107'182 uv172 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.916+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.920+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v107'183 uv183 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.920+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.912 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:01.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.924+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v107'184 uv184 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.924+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.928+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v107'185 uv185 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.928+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.932+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v107'186 uv186 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.932+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.924 INFO:teuthology.orchestra.run.smithi019.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:01.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.936+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v107'187 uv187 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.936+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.940+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v107'188 uv188 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.941+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.944+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v107'189 uv189 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.945+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e0d40 con 0x55f9672dd9b0 2024-04-25T11:38:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.949+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [omap-set-header] v107'190 uv190 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.949+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55f9672e1600 con 0x55f9672dd9b0 2024-04-25T11:38:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38d9c00640 1 -- 172.21.15.19:0/1397155118 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v107'191 uv191 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f38c800f070 con 0x55f9672dd9b0 2024-04-25T11:38:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f9672ce030 msgr2=0x55f9672d0450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f9672ce030 0x55f9672d0450 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f9672d9cc0 msgr2=0x55f9672dcb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f9672d9cc0 0x55f9672dcb20 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f9672dd9b0 msgr2=0x55f9672dfe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f9672dd9b0 0x55f9672dfe30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f9672d2460 msgr2=0x55f9672d4880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f9672d2460 0x55f9672d4880 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f96717fa70 msgr2=0x55f96721dc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f96717fa70 0x55f96721dc60 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f9672d5740 msgr2=0x55f9672d7b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f9672d5740 0x55f9672d7b60 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 reap_dead start 2024-04-25T11:38:01.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38db000640 1 -- 172.21.15.19:0/1397155118 reap_dead start 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38bc072e90 msgr2=0x7f38bc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38bc072e90 0x7f38bc0753a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f38cc057000 tx=0x7f38cc052000 comp rx=0 tx=0).stop 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 msgr2=0x55f967223870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.954+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967223870 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f38d4006b60 tx=0x7f38d4005700 comp rx=0 tx=0).stop 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.955+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 shutdown_connections 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.955+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38bc072e90 0x7f38bc0753a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.955+0000 7f38e14edf40 1 --2- 172.21.15.19:0/1397155118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f967227960 0x55f967223870 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.955+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 >> 172.21.15.19:0/1397155118 conn(0x55f967074ac0 msgr2=0x55f9672b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.955+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 shutdown_connections 2024-04-25T11:38:01.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:01.955+0000 7f38e14edf40 1 -- 172.21.15.19:0/1397155118 wait complete. 2024-04-25T11:38:01.976 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2024-04-25T11:38:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.223+0000 7fd897e49f40 1 Processor -- start 2024-04-25T11:38:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.223+0000 7fd897e49f40 1 -- start start 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd897e49f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae78cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd897e49f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae6c05f0 0x5593ae70cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd897e49f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae799dd0 0x5593ae70d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd897e49f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5593ae708e70 con 0x5593ae6c01e0 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd897e49f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5593ae666d20 con 0x5593ae799dd0 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd897e49f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5593ae709110 con 0x5593ae6c05f0 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd891000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae78cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd891000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae78cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42984/0 (socket says 172.21.15.19:42984) 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.225+0000 7fd891000640 1 -- 172.21.15.19:0/3641532201 learned_addr learned my addr 172.21.15.19:0/3641532201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd890600640 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae6c05f0 0x5593ae70cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd891000640 1 -- 172.21.15.19:0/3641532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae6c05f0 msgr2=0x5593ae70cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:02.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd891000640 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae6c05f0 0x5593ae70cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd891000640 1 -- 172.21.15.19:0/3641532201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae799dd0 msgr2=0x5593ae70d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:02.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd891000640 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae799dd0 0x5593ae70d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd891000640 1 -- 172.21.15.19:0/3641532201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ae78d8f0 con 0x5593ae6c01e0 2024-04-25T11:38:02.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd891000640 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae78cfb0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd88000a430 tx=0x7fd88000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f59e15c8e7bc32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:02.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd88f200640 1 -- 172.21.15.19:0/3641532201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88000b8c0 con 0x5593ae6c01e0 2024-04-25T11:38:02.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd88f200640 1 -- 172.21.15.19:0/3641532201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd88000f030 con 0x5593ae6c01e0 2024-04-25T11:38:02.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.226+0000 7fd88f200640 1 -- 172.21.15.19:0/3641532201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88000bcc0 con 0x5593ae6c01e0 2024-04-25T11:38:02.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 -- 172.21.15.19:0/3641532201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 msgr2=0x5593ae78cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae78cfb0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd88000a430 tx=0x7fd88000a8f0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 -- 172.21.15.19:0/3641532201 shutdown_connections 2024-04-25T11:38:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae799dd0 0x5593ae70d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae6c05f0 0x5593ae70cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 --2- 172.21.15.19:0/3641532201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae78cfb0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 -- 172.21.15.19:0/3641532201 >> 172.21.15.19:0/3641532201 conn(0x5593ae5e2ac0 msgr2=0x5593ae6f23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 -- 172.21.15.19:0/3641532201 shutdown_connections 2024-04-25T11:38:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.227+0000 7fd897e49f40 1 -- 172.21.15.19:0/3641532201 wait complete. 2024-04-25T11:38:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.228+0000 7fd897e49f40 1 Processor -- start 2024-04-25T11:38:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.229+0000 7fd897e49f40 1 -- start start 2024-04-25T11:38:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd897e49f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae795120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd897e49f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae6c05f0 0x5593ae7069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd897e49f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae799dd0 0x5593ae6f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd897e49f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5593ae796610 con 0x5593ae6c01e0 2024-04-25T11:38:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd897e49f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5593ae70a3e0 con 0x5593ae6c05f0 2024-04-25T11:38:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd897e49f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5593ae7946d0 con 0x5593ae799dd0 2024-04-25T11:38:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd891000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae795120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd891000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae795120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43000/0 (socket says 172.21.15.19:43000) 2024-04-25T11:38:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.230+0000 7fd891000640 1 -- 172.21.15.19:0/2094550164 learned_addr learned my addr 172.21.15.19:0/2094550164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891a00640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae799dd0 0x5593ae6f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891000640 1 -- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae799dd0 msgr2=0x5593ae6f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd890600640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae6c05f0 0x5593ae7069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891000640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae799dd0 0x5593ae6f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891000640 1 -- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae6c05f0 msgr2=0x5593ae7069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891000640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5593ae6c05f0 0x5593ae7069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891000640 1 -- 172.21.15.19:0/2094550164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593ae6f1fd0 con 0x5593ae6c01e0 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891a00640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5593ae799dd0 0x5593ae6f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.231+0000 7fd891000640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae795120 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x5593ae70bbd0 tx=0x7fd880002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.232+0000 7fd88de00640 1 -- 172.21.15.19:0/2094550164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880024070 con 0x5593ae6c01e0 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.232+0000 7fd88de00640 1 -- 172.21.15.19:0/2094550164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd880014420 con 0x5593ae6c01e0 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.232+0000 7fd88de00640 1 -- 172.21.15.19:0/2094550164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880014810 con 0x5593ae6c01e0 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.232+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5593ae794920 con 0x5593ae6c01e0 2024-04-25T11:38:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.232+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593ae6bfb60 con 0x5593ae6c01e0 2024-04-25T11:38:02.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.234+0000 7fd88de00640 1 -- 172.21.15.19:0/2094550164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd880027070 con 0x5593ae6c01e0 2024-04-25T11:38:02.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.235+0000 7fd88de00640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd870072e90 0x7fd8700753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.235+0000 7fd88de00640 1 -- 172.21.15.19:0/2094550164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7fd8800952e0 con 0x5593ae6c01e0 2024-04-25T11:38:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.236+0000 7fd897e49f40 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5593ae791110 0x5593ae791520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.236+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x5593ae795660 con 0x5593ae791110 2024-04-25T11:38:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.236+0000 7fd890600640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd870072e90 0x7fd8700753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.236+0000 7fd891a00640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5593ae791110 0x5593ae791520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.237+0000 7fd891a00640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5593ae791110 0x5593ae791520 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:02.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.238+0000 7fd890600640 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd870072e90 0x7fd8700753a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd87c005400 tx=0x7fd87c005310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:02.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.240+0000 7fd891a00640 1 -- 172.21.15.19:0/2094550164 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v107'351 uv351 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd88805b4e0 con 0x5593ae791110 2024-04-25T11:38:02.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5593ae791110 msgr2=0x5593ae791520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5593ae791110 0x5593ae791520 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd870072e90 msgr2=0x7fd8700753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd870072e90 0x7fd8700753a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd87c005400 tx=0x7fd87c005310 comp rx=0 tx=0).stop 2024-04-25T11:38:02.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 msgr2=0x5593ae795120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 --2- 172.21.15.19:0/2094550164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593ae6c01e0 0x5593ae795120 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x5593ae70bbd0 tx=0x7fd880002f70 comp rx=0 tx=0).stop 2024-04-25T11:38:02.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd891a00640 1 -- 172.21.15.19:0/2094550164 reap_dead start 2024-04-25T11:38:02.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 shutdown_connections 2024-04-25T11:38:02.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 >> 172.21.15.19:0/2094550164 conn(0x5593ae5e2ac0 msgr2=0x5593ae78cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:02.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 shutdown_connections 2024-04-25T11:38:02.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.241+0000 7fd897e49f40 1 -- 172.21.15.19:0/2094550164 wait complete. 2024-04-25T11:38:02.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.527+0000 7f7062e00640 1 Processor -- start 2024-04-25T11:38:02.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.527+0000 7f7062e00640 1 -- start start 2024-04-25T11:38:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.529+0000 7f7062e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f705c06dce0 0x7f705c16b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.529+0000 7f7062e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705c06f440 con 0x7f705c160390 2024-04-25T11:38:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.529+0000 7f7062e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705c06f720 con 0x7f705c0af710 2024-04-25T11:38:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.529+0000 7f7062e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f705c0afb20 con 0x7f705c06dce0 2024-04-25T11:38:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.530+0000 7f7053400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f705c160390 0x7f705c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57874/0 (socket says 172.21.15.19:57874) 2024-04-25T11:38:02.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.530+0000 7f7053400640 1 -- 172.21.15.19:0/1725233915 learned_addr learned my addr 172.21.15.19:0/1725233915 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.530+0000 7f7060e00640 1 --2- 172.21.15.19:0/1725233915 >> v2:172.21.15.169:3300/0 conn(0x7f705c06dce0 0x7f705c16b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.530+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417739298 0 0) 0x7f705c06f440 con 0x7f705c160390 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.530+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f703c003520 con 0x7f705c160390 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1965913462 0 0) 0x7f703c003520 con 0x7f705c160390 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 >> v2:172.21.15.169:3300/0 conn(0x7f705c06dce0 msgr2=0x7f705c16b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 --2- 172.21.15.19:0/1725233915 >> v2:172.21.15.169:3300/0 conn(0x7f705c06dce0 0x7f705c16b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 >> v1:172.21.15.169:6789/0 conn(0x7f705c0af710 legacy=0x7f705c164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c06fa00 con 0x7f705c160390 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7044001670 con 0x7f705c160390 2024-04-25T11:38:02.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 >> v1:172.21.15.19:6789/0 conn(0x7f705c160390 legacy=0x7f705c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703c0039b0 0x7f703c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703c004800 0x7f703c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c005250 0x7f703c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f705c0afb20 con 0x7f703c0039b0 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f703c00c1e0 con 0x7f703c004800 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f703c00c3e0 con 0x7f703c005250 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.531+0000 7f705be00640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703c0039b0 0x7f703c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f7053400640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703c004800 0x7f703c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f7060e00640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c005250 0x7f703c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f705be00640 1 -- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c005250 msgr2=0x7f703c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f705be00640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c005250 0x7f703c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f705be00640 1 -- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703c004800 msgr2=0x7f703c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f705be00640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f703c004800 0x7f703c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f705be00640 1 -- 172.21.15.19:0/1725233915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c003520 con 0x7f703c0039b0 2024-04-25T11:38:02.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.532+0000 7f7060e00640 1 -- 172.21.15.19:0/1725233915 reap_dead start 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.533+0000 7f705be00640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703c0039b0 0x7f703c003e20 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f704c001750 tx=0x7f703c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bba768eee92d706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.533+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c001370 con 0x7f703c0039b0 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.533+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c0052d0 con 0x7f703c0039b0 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.533+0000 7f705b400640 1 -- 172.21.15.19:0/1725233915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c02abc0 con 0x7f703c0039b0 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.534+0000 7f7062e00640 1 -- 172.21.15.19:0/1725233915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703c0039b0 msgr2=0x7f703c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.534+0000 7f7062e00640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703c0039b0 0x7f703c003e20 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f704c001750 tx=0x7f703c005830 comp rx=0 tx=0).stop 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.534+0000 7f7062e00640 1 -- 172.21.15.19:0/1725233915 shutdown_connections 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.534+0000 7f7062e00640 1 --2- 172.21.15.19:0/1725233915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f703c0039b0 0x7f703c003e20 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.534+0000 7f7062e00640 1 -- 172.21.15.19:0/1725233915 >> 172.21.15.19:0/1725233915 conn(0x7f705c073090 msgr2=0x7f705c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.534+0000 7f7062e00640 1 -- 172.21.15.19:0/1725233915 shutdown_connections 2024-04-25T11:38:02.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.534+0000 7f7062e00640 1 -- 172.21.15.19:0/1725233915 wait complete. 2024-04-25T11:38:02.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.536+0000 7f7062e00640 1 Processor -- start 2024-04-25T11:38:02.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.537+0000 7f7062e00640 1 -- start start 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7062e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c0039b0 0x7f705c149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7062e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c0d0480 0x7f705c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7062e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c1f9d50 0x7f705c1fa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7062e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f705c06f0d0 con 0x7f705c0d0480 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7062e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f705c06fda0 con 0x7f705c1f9d50 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7062e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f705c0b1110 con 0x7f703c0039b0 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7053400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c0d0480 0x7f705c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7053400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c0d0480 0x7f705c0d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43008/0 (socket says 172.21.15.19:43008) 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.538+0000 7f7053400640 1 -- 172.21.15.19:0/917720604 learned_addr learned my addr 172.21.15.19:0/917720604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f705be00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c0039b0 0x7f705c149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7060e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c1f9d50 0x7f705c1fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7053400640 1 -- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c0039b0 msgr2=0x7f705c149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7053400640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c0039b0 0x7f705c149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7053400640 1 -- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c1f9d50 msgr2=0x7f705c1fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7053400640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c1f9d50 0x7f705c1fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7053400640 1 -- 172.21.15.19:0/917720604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c0d0dd0 con 0x7f705c0d0480 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7060e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c1f9d50 0x7f705c1fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.539+0000 7f7053400640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c0d0480 0x7f705c0d0890 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f704400e730 tx=0x7f704400eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.540+0000 7f7058c00640 1 -- 172.21.15.19:0/917720604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704400c6c0 con 0x7f705c0d0480 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.540+0000 7f7058c00640 1 -- 172.21.15.19:0/917720604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7044018070 con 0x7f705c0d0480 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.540+0000 7f7058c00640 1 -- 172.21.15.19:0/917720604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704400caa0 con 0x7f705c0d0480 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.540+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c149db0 con 0x7f705c0d0480 2024-04-25T11:38:02.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.540+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c14a370 con 0x7f705c0d0480 2024-04-25T11:38:02.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.541+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c050740 con 0x7f705c0d0480 2024-04-25T11:38:02.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.542+0000 7f7058c00640 1 -- 172.21.15.19:0/917720604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f704400cc40 con 0x7f705c0d0480 2024-04-25T11:38:02.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.543+0000 7f7058c00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70380744b0 0x7f70380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:02.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.543+0000 7f7058c00640 1 -- 172.21.15.19:0/917720604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6999+0+0 (secure 0 0 0) 0x7f7044095600 con 0x7f705c0d0480 2024-04-25T11:38:02.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.544+0000 7f705be00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70380744b0 0x7f70380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:02.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.546+0000 7f705be00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70380744b0 0x7f70380769c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f704c032000 tx=0x7f704c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:02.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.548+0000 7f7058c00640 1 -- 172.21.15.19:0/917720604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f705c0d0dd0 con 0x7f705c0d0480 2024-04-25T11:38:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:02.799+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f705c0b5d40 con 0x7f705c0d0480 2024-04-25T11:38:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.197+0000 7f7058c00640 1 -- 172.21.15.19:0/917720604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v191) ==== 162+0+0 (secure 0 0 0) 0x7f704405f7e0 con 0x7f705c0d0480 2024-04-25T11:38:03.185 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.200+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70380744b0 msgr2=0x7f70380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.200+0000 7f7062e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70380744b0 0x7f70380769c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f704c032000 tx=0x7f704c02e000 comp rx=0 tx=0).stop 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.200+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c0d0480 msgr2=0x7f705c0d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.200+0000 7f7062e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c0d0480 0x7f705c0d0890 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f704400e730 tx=0x7f704400eb30 comp rx=0 tx=0).stop 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.200+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 shutdown_connections 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.200+0000 7f7062e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70380744b0 0x7f70380769c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.201+0000 7f7062e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f705c1f9d50 0x7f705c1fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.201+0000 7f7062e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f705c0d0480 0x7f705c0d0890 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.201+0000 7f7062e00640 1 --2- 172.21.15.19:0/917720604 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f703c0039b0 0x7f705c149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.201+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 >> 172.21.15.19:0/917720604 conn(0x7f705c073090 msgr2=0x7f705c0b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:03.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.201+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 shutdown_connections 2024-04-25T11:38:03.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.201+0000 7f7062e00640 1 -- 172.21.15.19:0/917720604 wait complete. 2024-04-25T11:38:03.208 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:03.220+0000 7facf6200640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-04-25T11:38:03.208 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:03.220+0000 7facf6200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:38:03.209 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:03.221+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:03.209 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:03.221+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:38:03.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:03.224 INFO:tasks.ceph.mds.a.smithi019.stderr: -16> 2024-04-25T11:38:03.220+0000 7facf6200640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-04-25T11:38:03.224 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:38:03.220+0000 7facf6200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:38:03.353 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:03.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.502+0000 7f1999400640 1 Processor -- start 2024-04-25T11:38:03.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.503+0000 7f1999400640 1 -- start start 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.505+0000 7f1999400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f199406c5c0 0x7f19940cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.505+0000 7f1999400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f19940b4ea0 0x7f199415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.505+0000 7f1999400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19940d1230 con 0x7f1994144370 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.505+0000 7f1999400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f19940d06b0 con 0x7f199406c5c0 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.505+0000 7f1999400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f19940d0990 con 0x7f19940b4ea0 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1993e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f199406c5c0 0x7f19940cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f198be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1994144370 0x7f19940b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57906/0 (socket says 172.21.15.19:57906) 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f198be00640 1 -- 172.21.15.19:0/1513724763 learned_addr learned my addr 172.21.15.19:0/1513724763 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:03.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1993e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f199406c5c0 0x7f19940cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43036/0 (socket says 172.21.15.19:43036) 2024-04-25T11:38:03.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1998a00640 1 --2- 172.21.15.19:0/1513724763 >> v2:172.21.15.169:3300/0 conn(0x7f19940b4ea0 0x7f199415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1993e00640 1 -- 172.21.15.19:0/1513724763 >> v2:172.21.15.169:3300/0 conn(0x7f19940b4ea0 msgr2=0x7f199415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:03.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1993e00640 1 --2- 172.21.15.19:0/1513724763 >> v2:172.21.15.169:3300/0 conn(0x7f19940b4ea0 0x7f199415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1993e00640 1 -- 172.21.15.19:0/1513724763 >> v1:172.21.15.19:6789/0 conn(0x7f1994144370 legacy=0x7f19940b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1993e00640 1 -- 172.21.15.19:0/1513724763 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19940d0c70 con 0x7f199406c5c0 2024-04-25T11:38:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.506+0000 7f1993400640 1 -- 172.21.15.19:0/1513724763 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706091208 0 0) 0x7f19940d1230 con 0x7f1994144370 2024-04-25T11:38:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.507+0000 7f1993e00640 1 --2- 172.21.15.19:0/1513724763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f199406c5c0 0x7f19940cfa70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f19940d26c0 tx=0x7f198402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e029055a499c5215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.508+0000 7f1993400640 1 -- 172.21.15.19:0/1513724763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984037070 con 0x7f199406c5c0 2024-04-25T11:38:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.508+0000 7f1993400640 1 -- 172.21.15.19:0/1513724763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1984032440 con 0x7f199406c5c0 2024-04-25T11:38:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.508+0000 7f1993400640 1 -- 172.21.15.19:0/1513724763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984032810 con 0x7f199406c5c0 2024-04-25T11:38:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 -- 172.21.15.19:0/1513724763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f199406c5c0 msgr2=0x7f19940cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 --2- 172.21.15.19:0/1513724763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f199406c5c0 0x7f19940cfa70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f19940d26c0 tx=0x7f198402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 -- 172.21.15.19:0/1513724763 shutdown_connections 2024-04-25T11:38:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 --2- 172.21.15.19:0/1513724763 >> v2:172.21.15.169:3300/0 conn(0x7f19940b4ea0 0x7f199415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 --2- 172.21.15.19:0/1513724763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f199406c5c0 0x7f19940cfa70 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 -- 172.21.15.19:0/1513724763 >> 172.21.15.19:0/1513724763 conn(0x7f19940d1720 msgr2=0x7f19940d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 -- 172.21.15.19:0/1513724763 shutdown_connections 2024-04-25T11:38:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.509+0000 7f1999400640 1 -- 172.21.15.19:0/1513724763 wait complete. 2024-04-25T11:38:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.511+0000 7f1999400640 1 Processor -- start 2024-04-25T11:38:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.512+0000 7f1999400640 1 -- start start 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.513+0000 7f1999400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199406c5c0 0x7f1994140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.513+0000 7f1999400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994144370 0x7f19940cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.513+0000 7f1999400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f199415adb0 0x7f199415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.513+0000 7f1999400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19940d0450 con 0x7f1994144370 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.513+0000 7f1999400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19940d02d0 con 0x7f199406c5c0 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.513+0000 7f1999400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1994147760 con 0x7f199415adb0 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.513+0000 7f198be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994144370 0x7f19940cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994144370 0x7f19940cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43042/0 (socket says 172.21.15.19:43042) 2024-04-25T11:38:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 -- 172.21.15.19:0/1514324067 learned_addr learned my addr 172.21.15.19:0/1514324067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f1993e00640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199406c5c0 0x7f1994140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 -- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f199415adb0 msgr2=0x7f199415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f199415adb0 0x7f199415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 -- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199406c5c0 msgr2=0x7f1994140cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199406c5c0 0x7f1994140cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 -- 172.21.15.19:0/1514324067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19940cc6d0 con 0x7f1994144370 2024-04-25T11:38:03.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.514+0000 7f198be00640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994144370 0x7f19940cab10 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c460 tx=0x7f197c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:03.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.515+0000 7f1990c00640 1 -- 172.21.15.19:0/1514324067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c018070 con 0x7f1994144370 2024-04-25T11:38:03.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.515+0000 7f1990c00640 1 -- 172.21.15.19:0/1514324067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c007500 con 0x7f1994144370 2024-04-25T11:38:03.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.515+0000 7f1990c00640 1 -- 172.21.15.19:0/1514324067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c004a20 con 0x7f1994144370 2024-04-25T11:38:03.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.515+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199415b700 con 0x7f1994144370 2024-04-25T11:38:03.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.515+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19940cc940 con 0x7f1994144370 2024-04-25T11:38:03.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.516+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19941f6a30 con 0x7f1994144370 2024-04-25T11:38:03.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.517+0000 7f1990c00640 1 -- 172.21.15.19:0/1514324067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f197c004bc0 con 0x7f1994144370 2024-04-25T11:38:03.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.518+0000 7f1990c00640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19700744e0 0x7f19700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:03.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.518+0000 7f1990c00640 1 -- 172.21.15.19:0/1514324067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7f197c094a90 con 0x7f1994144370 2024-04-25T11:38:03.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.519+0000 7f1993e00640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19700744e0 0x7f19700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:03.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.521+0000 7f1993e00640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19700744e0 0x7f19700769f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1984034000 tx=0x7f198402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:03.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.524+0000 7f1990c00640 1 -- 172.21.15.19:0/1514324067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f199415b700 con 0x7f1994144370 2024-04-25T11:38:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.767+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1994050740 con 0x7f1994144370 2024-04-25T11:38:03.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.769+0000 7f1990c00640 1 -- 172.21.15.19:0/1514324067 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) ==== 95+0+4352 (secure 0 0 0) 0x7f197c05ecb0 con 0x7f1994144370 2024-04-25T11:38:03.757 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:03.757 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":193,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:03.223065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 193 2024-04-25T11:38:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.771+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19700744e0 msgr2=0x7f19700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.771+0000 7f1999400640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19700744e0 0x7f19700769f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1984034000 tx=0x7f198402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994144370 msgr2=0x7f19940cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994144370 0x7f19940cab10 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c460 tx=0x7f197c00c920 comp rx=0 tx=0).stop 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 shutdown_connections 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19700744e0 0x7f19700769f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f199415adb0 0x7f199415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1994144370 0x7f19940cab10 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 --2- 172.21.15.19:0/1514324067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f199406c5c0 0x7f1994140cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 >> 172.21.15.19:0/1514324067 conn(0x7f19940d1720 msgr2=0x7f199415bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 shutdown_connections 2024-04-25T11:38:03.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:03.772+0000 7f1999400640 1 -- 172.21.15.19:0/1514324067 wait complete. 2024-04-25T11:38:03.789 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:03.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.085+0000 7fb398a00640 1 Processor -- start 2024-04-25T11:38:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.085+0000 7fb398a00640 1 -- start start 2024-04-25T11:38:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb398a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3940cf700 0x7fb39406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb398a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb394071180 con 0x7fb394168720 2024-04-25T11:38:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb398a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3940700e0 con 0x7fb3940cf2f0 2024-04-25T11:38:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb398a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb394070cf0 con 0x7fb3940cf700 2024-04-25T11:38:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb393e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3940cf700 0x7fb39406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb393e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3940cf700 0x7fb39406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57754/0 (socket says 172.21.15.19:57754) 2024-04-25T11:38:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb393e00640 1 -- 172.21.15.19:0/3146776330 learned_addr learned my addr 172.21.15.19:0/3146776330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.088+0000 7fb392a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb394168720 0x7fb39406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48460/0 (socket says 172.21.15.19:48460) 2024-04-25T11:38:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.089+0000 7fb393e00640 1 -- 172.21.15.19:0/3146776330 >> v1:172.21.15.169:6789/0 conn(0x7fb3940cf2f0 legacy=0x7fb394168010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.089+0000 7fb393e00640 1 -- 172.21.15.19:0/3146776330 >> v1:172.21.15.19:6789/0 conn(0x7fb394168720 legacy=0x7fb39406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.089+0000 7fb393e00640 1 -- 172.21.15.19:0/3146776330 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3940703c0 con 0x7fb3940cf700 2024-04-25T11:38:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.090+0000 7fb393e00640 1 --2- 172.21.15.19:0/3146776330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3940cf700 0x7fb39406ccd0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009870 tx=0x7fb38c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a45b58b444189c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.090+0000 7fb392000640 1 -- 172.21.15.19:0/3146776330 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c037070 con 0x7fb3940cf700 2024-04-25T11:38:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.090+0000 7fb392000640 1 -- 172.21.15.19:0/3146776330 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c02ae50 con 0x7fb3940cf700 2024-04-25T11:38:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.090+0000 7fb392000640 1 -- 172.21.15.19:0/3146776330 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c032860 con 0x7fb3940cf700 2024-04-25T11:38:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.091+0000 7fb398a00640 1 -- 172.21.15.19:0/3146776330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3940cf700 msgr2=0x7fb39406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.091+0000 7fb398a00640 1 --2- 172.21.15.19:0/3146776330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3940cf700 0x7fb39406ccd0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009870 tx=0x7fb38c02a300 comp rx=0 tx=0).stop 2024-04-25T11:38:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.091+0000 7fb398a00640 1 -- 172.21.15.19:0/3146776330 shutdown_connections 2024-04-25T11:38:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.091+0000 7fb398a00640 1 --2- 172.21.15.19:0/3146776330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3940cf700 0x7fb39406ccd0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.091+0000 7fb398a00640 1 -- 172.21.15.19:0/3146776330 >> 172.21.15.19:0/3146776330 conn(0x7fb3940d1720 msgr2=0x7fb3940d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.091+0000 7fb398a00640 1 -- 172.21.15.19:0/3146776330 shutdown_connections 2024-04-25T11:38:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.091+0000 7fb398a00640 1 -- 172.21.15.19:0/3146776330 wait complete. 2024-04-25T11:38:04.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.094+0000 7fb398a00640 1 Processor -- start 2024-04-25T11:38:04.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.094+0000 7fb398a00640 1 -- start start 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.095+0000 7fb398a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3940cf2f0 0x7fb394147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.095+0000 7fb398a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3940cf700 0x7fb39415fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.095+0000 7fb398a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3941631a0 0x7fb39415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.095+0000 7fb398a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb394168d00 con 0x7fb3941631a0 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.095+0000 7fb398a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb394070980 con 0x7fb3940cf2f0 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.095+0000 7fb398a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb394168e80 con 0x7fb3940cf700 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3941631a0 0x7fb39415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3941631a0 0x7fb39415ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57762/0 (socket says 172.21.15.19:57762) 2024-04-25T11:38:04.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 -- 172.21.15.19:0/941753705 learned_addr learned my addr 172.21.15.19:0/941753705 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:04.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393400640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3940cf2f0 0x7fb394147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb392a00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3940cf700 0x7fb39415fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 -- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3940cf700 msgr2=0x7fb39415fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3940cf700 0x7fb39415fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 -- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3940cf2f0 msgr2=0x7fb394147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3940cf2f0 0x7fb394147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393e00640 1 -- 172.21.15.19:0/941753705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3941638d0 con 0x7fb3941631a0 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.096+0000 7fb393400640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3940cf2f0 0x7fb394147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.097+0000 7fb393e00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3941631a0 0x7fb39415ff40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb38c02a340 tx=0x7fb38c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.097+0000 7fb38b400640 1 -- 172.21.15.19:0/941753705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c041070 con 0x7fb3941631a0 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.097+0000 7fb38b400640 1 -- 172.21.15.19:0/941753705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb38c03c890 con 0x7fb3941631a0 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.097+0000 7fb38b400640 1 -- 172.21.15.19:0/941753705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c037040 con 0x7fb3941631a0 2024-04-25T11:38:04.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.097+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb394163bb0 con 0x7fb3941631a0 2024-04-25T11:38:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.097+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb394164170 con 0x7fb3941631a0 2024-04-25T11:38:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.098+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb394050740 con 0x7fb3941631a0 2024-04-25T11:38:04.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.100+0000 7fb38b400640 1 -- 172.21.15.19:0/941753705 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb38c044070 con 0x7fb3941631a0 2024-04-25T11:38:04.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.101+0000 7fb38b400640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3740744b0 0x7fb3740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.101+0000 7fb38b400640 1 -- 172.21.15.19:0/941753705 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7fb38c0b5ff0 con 0x7fb3941631a0 2024-04-25T11:38:04.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.101+0000 7fb393400640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3740744b0 0x7fb3740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.103+0000 7fb393400640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3740744b0 0x7fb3740769c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb3940d11e0 tx=0x7fb384009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.106+0000 7fb38b400640 1 -- 172.21.15.19:0/941753705 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb38c080270 con 0x7fb3941631a0 2024-04-25T11:38:04.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.349+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39406fb60 con 0x7fb3941631a0 2024-04-25T11:38:04.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.350+0000 7fb38b400640 1 -- 172.21.15.19:0/941753705 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) ==== 95+0+5178 (secure 0 0 0) 0x7fb38c07fb70 con 0x7fb3941631a0 2024-04-25T11:38:04.339 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:04.339 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":194,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2065862570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2065862570},{"type":"v1","addr":"172.21.15.19:6835","nonce":2065862570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:03.223065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:04.339 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 194 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.352+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3740744b0 msgr2=0x7fb3740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.352+0000 7fb398a00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3740744b0 0x7fb3740769c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb3940d11e0 tx=0x7fb384009040 comp rx=0 tx=0).stop 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3941631a0 msgr2=0x7fb39415ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3941631a0 0x7fb39415ff40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb38c02a340 tx=0x7fb38c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 shutdown_connections 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3740744b0 0x7fb3740769c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3941631a0 0x7fb39415ff40 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3940cf700 0x7fb39415fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 --2- 172.21.15.19:0/941753705 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3940cf2f0 0x7fb394147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 >> 172.21.15.19:0/941753705 conn(0x7fb3940d1720 msgr2=0x7fb3940b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 shutdown_connections 2024-04-25T11:38:04.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.353+0000 7fb398a00640 1 -- 172.21.15.19:0/941753705 wait complete. 2024-04-25T11:38:04.365 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:04.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.653+0000 7f3b01400640 1 Processor -- start 2024-04-25T11:38:04.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.653+0000 7f3b01400640 1 -- start start 2024-04-25T11:38:04.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3b01400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3b01400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3afc144370 0x7f3afc0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3b01400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3afc0b50e0 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3b01400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3afc0d1230 con 0x7f3afc144370 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3b01400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3afc0d06b0 con 0x7f3afc06c5c0 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3af3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3afc144370 0x7f3afc0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3af3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3afc144370 0x7f3afc0b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57764/0 (socket says 172.21.15.19:57764) 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.656+0000 7f3af3400640 1 -- 172.21.15.19:0/1252252089 learned_addr learned my addr 172.21.15.19:0/1252252089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:04.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.657+0000 7f3afbe00640 1 --2- 172.21.15.19:0/1252252089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.657+0000 7f3af3400640 1 -- 172.21.15.19:0/1252252089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 msgr2=0x7f3afc0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.657+0000 7f3af3400640 1 --2- 172.21.15.19:0/1252252089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.657+0000 7f3af3400640 1 -- 172.21.15.19:0/1252252089 >> v1:172.21.15.169:6789/0 conn(0x7f3afc0b4cd0 legacy=0x7f3afc161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.657+0000 7f3af3400640 1 -- 172.21.15.19:0/1252252089 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc0d0990 con 0x7f3afc144370 2024-04-25T11:38:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.658+0000 7f3af3400640 1 --2- 172.21.15.19:0/1252252089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc144370 0x7f3afc0b4790 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009870 tx=0x7f3ae402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c0083aa3b99d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.659+0000 7f3afb400640 1 -- 172.21.15.19:0/1252252089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4037070 con 0x7f3afc144370 2024-04-25T11:38:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.659+0000 7f3afb400640 1 -- 172.21.15.19:0/1252252089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae4032440 con 0x7f3afc144370 2024-04-25T11:38:04.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.659+0000 7f3afb400640 1 -- 172.21.15.19:0/1252252089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4032810 con 0x7f3afc144370 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 -- 172.21.15.19:0/1252252089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc144370 msgr2=0x7f3afc0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 --2- 172.21.15.19:0/1252252089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc144370 0x7f3afc0b4790 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009870 tx=0x7f3ae402a270 comp rx=0 tx=0).stop 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 -- 172.21.15.19:0/1252252089 shutdown_connections 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 --2- 172.21.15.19:0/1252252089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc144370 0x7f3afc0b4790 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 --2- 172.21.15.19:0/1252252089 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 -- 172.21.15.19:0/1252252089 >> 172.21.15.19:0/1252252089 conn(0x7f3afc0d1720 msgr2=0x7f3afc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 -- 172.21.15.19:0/1252252089 shutdown_connections 2024-04-25T11:38:04.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.660+0000 7f3b01400640 1 -- 172.21.15.19:0/1252252089 wait complete. 2024-04-25T11:38:04.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.663+0000 7f3b01400640 1 Processor -- start 2024-04-25T11:38:04.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.663+0000 7f3b01400640 1 -- start start 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.664+0000 7f3b01400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3b01400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc0b4cd0 0x7f3afc1e7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3b01400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc144370 0x7f3afc1e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3b01400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc0d02d0 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3b01400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc0d0f50 con 0x7f3afc144370 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3b01400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3afc0d0450 con 0x7f3afc06c5c0 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc0b4cd0 0x7f3afc1e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc0b4cd0 0x7f3afc1e7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57772/0 (socket says 172.21.15.19:57772) 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 -- 172.21.15.19:0/3579936385 learned_addr learned my addr 172.21.15.19:0/3579936385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3afbe00640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 -- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 msgr2=0x7f3afc15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc15c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 -- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc144370 msgr2=0x7f3afc1e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3b00a00640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc144370 0x7f3afc1e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc144370 0x7f3afc1e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3af3400640 1 -- 172.21.15.19:0/3579936385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc161220 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.665+0000 7f3b00a00640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc144370 0x7f3afc1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:04.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.666+0000 7f3af3400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc0b4cd0 0x7f3afc1e7850 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3ae402a2b0 tx=0x7f3ae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.666+0000 7f3af8c00640 1 -- 172.21.15.19:0/3579936385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4041020 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.666+0000 7f3af8c00640 1 -- 172.21.15.19:0/3579936385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae40306e0 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.666+0000 7f3af8c00640 1 -- 172.21.15.19:0/3579936385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4037040 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.666+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc161500 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.666+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc161ac0 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.668+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3afc160f70 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.669+0000 7f3af8c00640 1 -- 172.21.15.19:0/3579936385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ae4044020 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.670+0000 7f3af8c00640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ad80744b0 0x7f3ad80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:04.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.671+0000 7f3af8c00640 1 -- 172.21.15.19:0/3579936385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7f3ae40b6160 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.671+0000 7f3afbe00640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ad80744b0 0x7f3ad80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:04.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.673+0000 7f3afbe00640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ad80744b0 0x7f3ad80769c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3aec006fd0 tx=0x7f3aec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:04.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.676+0000 7f3af8c00640 1 -- 172.21.15.19:0/3579936385 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ae404b030 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.919+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3afc050740 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.921+0000 7f3af8c00640 1 -- 172.21.15.19:0/3579936385 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) ==== 95+0+5178 (secure 0 0 0) 0x7f3ae40802f0 con 0x7f3afc0b4cd0 2024-04-25T11:38:04.909 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:04.909 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":194,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9168,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2065862570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2065862570},{"type":"v1","addr":"172.21.15.19:6835","nonce":2065862570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:03.223065+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 194 2024-04-25T11:38:04.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.923+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ad80744b0 msgr2=0x7f3ad80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.923+0000 7f3b01400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ad80744b0 0x7f3ad80769c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3aec006fd0 tx=0x7f3aec008040 comp rx=0 tx=0).stop 2024-04-25T11:38:04.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.923+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc0b4cd0 msgr2=0x7f3afc1e7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:04.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.923+0000 7f3b01400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc0b4cd0 0x7f3afc1e7850 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3ae402a2b0 tx=0x7f3ae40046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 shutdown_connections 2024-04-25T11:38:04.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3ad80744b0 0x7f3ad80769c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3afc144370 0x7f3afc1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3afc0b4cd0 0x7f3afc1e7850 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 --2- 172.21.15.19:0/3579936385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3afc06c5c0 0x7f3afc15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 >> 172.21.15.19:0/3579936385 conn(0x7f3afc0d1720 msgr2=0x7f3afc146790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 shutdown_connections 2024-04-25T11:38:04.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:04.924+0000 7f3b01400640 1 -- 172.21.15.19:0/3579936385 wait complete. 2024-04-25T11:38:04.936 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2024-04-25T11:38:04.936 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2024-04-25T11:38:04.936 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:04.937 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:04.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:38:04.986 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:04.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:04.986 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:05.079 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:38:05.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:05.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:05.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.369+0000 7fdb46e00640 1 Processor -- start 2024-04-25T11:38:05.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.369+0000 7fdb46e00640 1 -- start start 2024-04-25T11:38:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb46e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb400b19e0 0x7fdb4006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:05.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb46e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdb40071780 0x7fdb40071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb46e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb400720d0 con 0x7fdb40161550 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb46e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdb400b65c0 con 0x7fdb400b19e0 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb46e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdb400d1740 con 0x7fdb40071780 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb45a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb400b19e0 0x7fdb4006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb45000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdb40161550 0x7fdb4015ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48464/0 (socket says 172.21.15.19:48464) 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb45a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb400b19e0 0x7fdb4006d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57774/0 (socket says 172.21.15.19:57774) 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.372+0000 7fdb45000640 1 -- 172.21.15.19:0/3504999703 learned_addr learned my addr 172.21.15.19:0/3504999703 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.373+0000 7fdb46400640 1 --2- 172.21.15.19:0/3504999703 >> v2:172.21.15.169:3300/0 conn(0x7fdb40071780 0x7fdb40071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.373+0000 7fdb45a00640 1 -- 172.21.15.19:0/3504999703 >> v2:172.21.15.169:3300/0 conn(0x7fdb40071780 msgr2=0x7fdb40071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.373+0000 7fdb45a00640 1 --2- 172.21.15.19:0/3504999703 >> v2:172.21.15.169:3300/0 conn(0x7fdb40071780 0x7fdb40071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.373+0000 7fdb45a00640 1 -- 172.21.15.19:0/3504999703 >> v1:172.21.15.19:6789/0 conn(0x7fdb40161550 legacy=0x7fdb4015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.373+0000 7fdb45a00640 1 -- 172.21.15.19:0/3504999703 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb40161960 con 0x7fdb400b19e0 2024-04-25T11:38:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.373+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3504999703 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575621304 0 0) 0x7fdb2c001110 con 0x7fdb40161550 2024-04-25T11:38:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.374+0000 7fdb45a00640 1 --2- 172.21.15.19:0/3504999703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb400b19e0 0x7fdb4006d6d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fdb400b2150 tx=0x7fdb3402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e036f7b970aafe99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.374+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3504999703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34037070 con 0x7fdb400b19e0 2024-04-25T11:38:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.374+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3504999703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb34032440 con 0x7fdb400b19e0 2024-04-25T11:38:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.374+0000 7fdb3fe00640 1 -- 172.21.15.19:0/3504999703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3402ae60 con 0x7fdb400b19e0 2024-04-25T11:38:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 -- 172.21.15.19:0/3504999703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb400b19e0 msgr2=0x7fdb4006d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3504999703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb400b19e0 0x7fdb4006d6d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fdb400b2150 tx=0x7fdb3402a110 comp rx=0 tx=0).stop 2024-04-25T11:38:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 -- 172.21.15.19:0/3504999703 shutdown_connections 2024-04-25T11:38:05.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3504999703 >> v2:172.21.15.169:3300/0 conn(0x7fdb40071780 0x7fdb40071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:05.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3504999703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb400b19e0 0x7fdb4006d6d0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:05.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 -- 172.21.15.19:0/3504999703 >> 172.21.15.19:0/3504999703 conn(0x7fdb40070270 msgr2=0x7fdb40070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:05.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 -- 172.21.15.19:0/3504999703 shutdown_connections 2024-04-25T11:38:05.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.375+0000 7fdb46e00640 1 -- 172.21.15.19:0/3504999703 wait complete. 2024-04-25T11:38:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.378+0000 7fdb46e00640 1 Processor -- start 2024-04-25T11:38:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.378+0000 7fdb46e00640 1 -- start start 2024-04-25T11:38:05.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40071780 0x7fdb4014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:05.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b19e0 0x7fdb40153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40161550 0x7fdb40155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40072550 con 0x7fdb40161550 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb400d1cf0 con 0x7fdb400b19e0 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb400d1fd0 con 0x7fdb40071780 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40161550 0x7fdb40155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40161550 0x7fdb40155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57782/0 (socket says 172.21.15.19:57782) 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb46400640 1 -- 172.21.15.19:0/3938416716 learned_addr learned my addr 172.21.15.19:0/3938416716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.380+0000 7fdb45a00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40071780 0x7fdb4014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46400640 1 -- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40071780 msgr2=0x7fdb4014fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46400640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40071780 0x7fdb4014fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46400640 1 -- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b19e0 msgr2=0x7fdb40153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46400640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b19e0 0x7fdb40153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:05.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46400640 1 -- 172.21.15.19:0/3938416716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb40154190 con 0x7fdb40161550 2024-04-25T11:38:05.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46400640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40161550 0x7fdb40155d30 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fdb38007e90 tx=0x7fdb3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:05.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb3d600640 1 -- 172.21.15.19:0/3938416716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38014420 con 0x7fdb40161550 2024-04-25T11:38:05.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb3d600640 1 -- 172.21.15.19:0/3938416716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb38019070 con 0x7fdb40161550 2024-04-25T11:38:05.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4015a9c0 con 0x7fdb40161550 2024-04-25T11:38:05.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb3d600640 1 -- 172.21.15.19:0/3938416716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38014800 con 0x7fdb40161550 2024-04-25T11:38:05.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.381+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4015af80 con 0x7fdb40161550 2024-04-25T11:38:05.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.382+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb40050740 con 0x7fdb40161550 2024-04-25T11:38:05.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.384+0000 7fdb3d600640 1 -- 172.21.15.19:0/3938416716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdb380149a0 con 0x7fdb40161550 2024-04-25T11:38:05.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.385+0000 7fdb3d600640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb20074500 0x7fdb20076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:05.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.385+0000 7fdb3d600640 1 -- 172.21.15.19:0/3938416716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7fdb38096060 con 0x7fdb40161550 2024-04-25T11:38:05.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.386+0000 7fdb45a00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb20074500 0x7fdb20076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:05.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.387+0000 7fdb45a00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb20074500 0x7fdb20076a10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb34034000 tx=0x7fdb3402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:05.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.390+0000 7fdb3d600640 1 -- 172.21.15.19:0/3938416716 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb40154190 con 0x7fdb40161550 2024-04-25T11:38:05.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:05.602 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:05.638+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdb4006e490 con 0x7fdb40161550 2024-04-25T11:38:06.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.246+0000 7fdb3d600640 1 -- 172.21.15.19:0/3938416716 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v195) ==== 161+0+0 (secure 0 0 0) 0x7fdb38060270 con 0x7fdb40161550 2024-04-25T11:38:06.235 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb20074500 msgr2=0x7fdb20076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb20074500 0x7fdb20076a10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb34034000 tx=0x7fdb3402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40161550 msgr2=0x7fdb40155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40161550 0x7fdb40155d30 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fdb38007e90 tx=0x7fdb3800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 shutdown_connections 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb20074500 0x7fdb20076a10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb400b19e0 0x7fdb40153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb40161550 0x7fdb40155d30 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 --2- 172.21.15.19:0/3938416716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb40071780 0x7fdb4014fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 >> 172.21.15.19:0/3938416716 conn(0x7fdb40070270 msgr2=0x7fdb4006e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.250+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 shutdown_connections 2024-04-25T11:38:06.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.251+0000 7fdb46e00640 1 -- 172.21.15.19:0/3938416716 wait complete. 2024-04-25T11:38:06.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:06.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.563+0000 7fa823200640 1 Processor -- start 2024-04-25T11:38:06.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.563+0000 7fa823200640 1 -- start start 2024-04-25T11:38:06.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.565+0000 7fa823200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 0x7fa81c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.565+0000 7fa823200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81c0d1230 con 0x7fa81c144370 2024-04-25T11:38:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.565+0000 7fa823200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa81c0d06b0 con 0x7fa81c0b4ea0 2024-04-25T11:38:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.565+0000 7fa823200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c0d0990 con 0x7fa81c06c5c0 2024-04-25T11:38:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.566+0000 7fa821400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa81c144370 0x7fa81c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48476/0 (socket says 172.21.15.19:48476) 2024-04-25T11:38:06.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.566+0000 7fa821400640 1 -- 172.21.15.19:0/1510753748 learned_addr learned my addr 172.21.15.19:0/1510753748 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:06.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.566+0000 7fa821e00640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 0x7fa81c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.566+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980489927 0 0) 0x7fa81c0d1230 con 0x7fa81c144370 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.566+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa804003520 con 0x7fa81c144370 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242734417 0 0) 0x7fa81c0d06b0 con 0x7fa81c0b4ea0 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa81c0d1230 con 0x7fa81c0b4ea0 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3783806741 0 0) 0x7fa804003520 con 0x7fa81c144370 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 msgr2=0x7fa81c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 0x7fa81c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 >> v1:172.21.15.169:6789/0 conn(0x7fa81c0b4ea0 legacy=0x7fa81c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c0d0c70 con 0x7fa81c144370 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa80c001670 con 0x7fa81c144370 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 >> v1:172.21.15.19:6789/0 conn(0x7fa81c144370 legacy=0x7fa81c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804003b70 0x7fa804003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 0x7fa804004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa804004d40 0x7fa804005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c0d0990 con 0x7fa804004d40 2024-04-25T11:38:06.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa80400ff60 con 0x7fa804003b70 2024-04-25T11:38:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa804010130 con 0x7fa81c06c5c0 2024-04-25T11:38:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.567+0000 7fa822800640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa804004d40 0x7fa804005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.568+0000 7fa821e00640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804003b70 0x7fa804003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.568+0000 7fa821400640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 0x7fa804004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.568+0000 7fa822800640 1 -- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 msgr2=0x7fa804004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.568+0000 7fa822800640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa81c06c5c0 0x7fa804004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.568+0000 7fa822800640 1 -- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804003b70 msgr2=0x7fa804003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.568+0000 7fa822800640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa804003b70 0x7fa804003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.568+0000 7fa822800640 1 -- 172.21.15.19:0/1510753748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa804003520 con 0x7fa804004d40 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.569+0000 7fa822800640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa804004d40 0x7fa804005190 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa818001fa0 tx=0x7fa804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfacad3ae492a991 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.569+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818001bf0 con 0x7fa804004d40 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.569+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818003b70 con 0x7fa804004d40 2024-04-25T11:38:06.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.570+0000 7fa820a00640 1 -- 172.21.15.19:0/1510753748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180050d0 con 0x7fa804004d40 2024-04-25T11:38:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.570+0000 7fa823200640 1 -- 172.21.15.19:0/1510753748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa804004d40 msgr2=0x7fa804005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.570+0000 7fa823200640 1 --2- 172.21.15.19:0/1510753748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa804004d40 0x7fa804005190 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa818001fa0 tx=0x7fa804005830 comp rx=0 tx=0).stop 2024-04-25T11:38:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.570+0000 7fa822800640 1 -- 172.21.15.19:0/1510753748 reap_dead start 2024-04-25T11:38:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.571+0000 7fa823200640 1 -- 172.21.15.19:0/1510753748 shutdown_connections 2024-04-25T11:38:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.571+0000 7fa823200640 1 -- 172.21.15.19:0/1510753748 >> 172.21.15.19:0/1510753748 conn(0x7fa81c0d1720 msgr2=0x7fa81c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.571+0000 7fa823200640 1 -- 172.21.15.19:0/1510753748 shutdown_connections 2024-04-25T11:38:06.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.571+0000 7fa823200640 1 -- 172.21.15.19:0/1510753748 wait complete. 2024-04-25T11:38:06.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.573+0000 7fa823200640 1 Processor -- start 2024-04-25T11:38:06.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.573+0000 7fa823200640 1 -- start start 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa823200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa804004d40 0x7fa81c162050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa823200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c137110 0x7fa81c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa823200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c15d050 0x7fa81c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa823200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c0d0450 con 0x7fa81c137110 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa823200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c0d02d0 con 0x7fa81c15d050 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa823200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c147760 con 0x7fa804004d40 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa821400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c137110 0x7fa81c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa821400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c137110 0x7fa81c15cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57786/0 (socket says 172.21.15.19:57786) 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa821400640 1 -- 172.21.15.19:0/7910030 learned_addr learned my addr 172.21.15.19:0/7910030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:06.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa821e00640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa804004d40 0x7fa81c162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.575+0000 7fa822800640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c15d050 0x7fa81c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa821400640 1 -- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa804004d40 msgr2=0x7fa81c162050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa821400640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa804004d40 0x7fa81c162050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa821400640 1 -- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c15d050 msgr2=0x7fa81c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa821400640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c15d050 0x7fa81c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa821400640 1 -- 172.21.15.19:0/7910030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c15d460 con 0x7fa81c137110 2024-04-25T11:38:06.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa821400640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c137110 0x7fa81c15cb10 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa80c007290 tx=0x7fa80c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa816000640 1 -- 172.21.15.19:0/7910030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c001640 con 0x7fa81c137110 2024-04-25T11:38:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa816000640 1 -- 172.21.15.19:0/7910030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80c018070 con 0x7fa81c137110 2024-04-25T11:38:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa816000640 1 -- 172.21.15.19:0/7910030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00fc20 con 0x7fa81c137110 2024-04-25T11:38:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.576+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c1603c0 con 0x7fa81c137110 2024-04-25T11:38:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.577+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c160660 con 0x7fa81c137110 2024-04-25T11:38:06.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.578+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c050740 con 0x7fa81c137110 2024-04-25T11:38:06.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.579+0000 7fa816000640 1 -- 172.21.15.19:0/7910030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa80c01c410 con 0x7fa81c137110 2024-04-25T11:38:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.580+0000 7fa816000640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8000744e0 0x7fa8000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.580+0000 7fa816000640 1 -- 172.21.15.19:0/7910030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7fa80c0953e0 con 0x7fa81c137110 2024-04-25T11:38:06.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.581+0000 7fa821e00640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8000744e0 0x7fa8000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.582+0000 7fa821e00640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8000744e0 0x7fa8000769f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa810009b00 tx=0x7fa810008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:06.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.585+0000 7fa816000640 1 -- 172.21.15.19:0/7910030 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa81c15d460 con 0x7fa81c137110 2024-04-25T11:38:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:06.868+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa81c162830 con 0x7fa81c137110 2024-04-25T11:38:07.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.258+0000 7fa816000640 1 -- 172.21.15.19:0/7910030 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v196) ==== 101+0+0 (secure 0 0 0) 0x7fa80c05f570 con 0x7fa81c137110 2024-04-25T11:38:07.247 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:38:07.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.261+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8000744e0 msgr2=0x7fa8000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:07.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.261+0000 7fa823200640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8000744e0 0x7fa8000769f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa810009b00 tx=0x7fa810008040 comp rx=0 tx=0).stop 2024-04-25T11:38:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.261+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c137110 msgr2=0x7fa81c15cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.261+0000 7fa823200640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c137110 0x7fa81c15cb10 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fa80c007290 tx=0x7fa80c007750 comp rx=0 tx=0).stop 2024-04-25T11:38:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 shutdown_connections 2024-04-25T11:38:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8000744e0 0x7fa8000769f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa81c15d050 0x7fa81c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa81c137110 0x7fa81c15cb10 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 --2- 172.21.15.19:0/7910030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa804004d40 0x7fa81c162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 >> 172.21.15.19:0/7910030 conn(0x7fa81c0d1720 msgr2=0x7fa81c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:07.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 shutdown_connections 2024-04-25T11:38:07.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.262+0000 7fa823200640 1 -- 172.21.15.19:0/7910030 wait complete. 2024-04-25T11:38:07.285 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:07.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.480+0000 7f3d303dbf40 1 Processor -- start 2024-04-25T11:38:07.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.480+0000 7f3d303dbf40 1 -- start start 2024-04-25T11:38:07.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d303dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d10003c40 0x564068d56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d303dbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564068dfd1c0 0x564068dfd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d303dbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068df3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d303dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564068cccd20 con 0x7f3d10003c40 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d303dbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564068cd0d60 con 0x564068df33f0 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d303dbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564068d6f3f0 con 0x564068dfd1c0 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d29600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d10003c40 0x564068d56d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d29600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d10003c40 0x564068d56d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57814/0 (socket says 172.21.15.19:57814) 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.482+0000 7f3d29600640 1 -- 172.21.15.19:0/2624261797 learned_addr learned my addr 172.21.15.19:0/2624261797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d2a000640 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068df3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d28c00640 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564068dfd1c0 0x564068dfd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d29600640 1 -- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564068dfd1c0 msgr2=0x564068dfd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d29600640 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564068dfd1c0 0x564068dfd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d29600640 1 -- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 msgr2=0x564068df3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d29600640 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068df3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d29600640 1 -- 172.21.15.19:0/2624261797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564068d26ea0 con 0x7f3d10003c40 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d2a000640 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068df3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d29600640 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d10003c40 0x564068d56d20 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d6c0 tx=0x7f3d1c00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7962c4f8a18dab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d23400640 1 -- 172.21.15.19:0/2624261797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c013070 con 0x7f3d10003c40 2024-04-25T11:38:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d23400640 1 -- 172.21.15.19:0/2624261797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1c00b8d0 con 0x7f3d10003c40 2024-04-25T11:38:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.483+0000 7f3d23400640 1 -- 172.21.15.19:0/2624261797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00bcc0 con 0x7f3d10003c40 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 -- 172.21.15.19:0/2624261797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d10003c40 msgr2=0x564068d56d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d10003c40 0x564068d56d20 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d6c0 tx=0x7f3d1c00db80 comp rx=0 tx=0).stop 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 -- 172.21.15.19:0/2624261797 shutdown_connections 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068df3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564068dfd1c0 0x564068dfd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/2624261797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d10003c40 0x564068d56d20 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 -- 172.21.15.19:0/2624261797 >> 172.21.15.19:0/2624261797 conn(0x564068c48ac0 msgr2=0x564068d56000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 -- 172.21.15.19:0/2624261797 shutdown_connections 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.484+0000 7f3d303dbf40 1 -- 172.21.15.19:0/2624261797 wait complete. 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.485+0000 7f3d303dbf40 1 Processor -- start 2024-04-25T11:38:07.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.486+0000 7f3d303dbf40 1 -- start start 2024-04-25T11:38:07.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d303dbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068dde680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d303dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564068dfd1c0 0x564068dddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d303dbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d10003c40 0x564068d55fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d303dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564068df6ec0 con 0x564068dfd1c0 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d303dbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564068d58490 con 0x564068df33f0 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d303dbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564068ddef80 con 0x7f3d10003c40 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d28c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564068dfd1c0 0x564068dddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d28c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564068dfd1c0 0x564068dddcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57828/0 (socket says 172.21.15.19:57828) 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.487+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 learned_addr learned my addr 172.21.15.19:0/814939806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:07.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.488+0000 7f3d29600640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068dde680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.488+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d10003c40 msgr2=0x564068d55fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:07.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.488+0000 7f3d28c00640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d10003c40 0x564068d55fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.488+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 msgr2=0x564068dde680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:07.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.488+0000 7f3d28c00640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564068df33f0 0x564068dde680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:07.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.488+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564068df7850 con 0x564068dfd1c0 2024-04-25T11:38:07.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.488+0000 7f3d28c00640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564068dfd1c0 0x564068dddcb0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f3d14002ed0 tx=0x7f3d1400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.489+0000 7f3d22000640 1 -- 172.21.15.19:0/814939806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14010640 con 0x564068dfd1c0 2024-04-25T11:38:07.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.489+0000 7f3d22000640 1 -- 172.21.15.19:0/814939806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d140107e0 con 0x564068dfd1c0 2024-04-25T11:38:07.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.489+0000 7f3d22000640 1 -- 172.21.15.19:0/814939806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14010bb0 con 0x564068dfd1c0 2024-04-25T11:38:07.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.489+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564068df7b30 con 0x564068dfd1c0 2024-04-25T11:38:07.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.489+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564068df2ce0 con 0x564068dfd1c0 2024-04-25T11:38:07.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.491+0000 7f3d22000640 1 -- 172.21.15.19:0/814939806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3d1400e040 con 0x564068dfd1c0 2024-04-25T11:38:07.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.492+0000 7f3d22000640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d08072e90 0x7f3d080753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.492+0000 7f3d22000640 1 -- 172.21.15.19:0/814939806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7f3d140945a0 con 0x564068dfd1c0 2024-04-25T11:38:07.481 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.492+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564068d6b360 0x564068d6b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.492+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068e9f8c0 con 0x564068d6b360 2024-04-25T11:38:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.493+0000 7f3d29600640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d08072e90 0x7f3d080753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.493+0000 7f3d2a000640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564068d6b360 0x564068d6b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.494+0000 7f3d29600640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d08072e90 0x7f3d080753a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d6c0 tx=0x7f3d1c01a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.495+0000 7f3d2a000640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564068d6b360 0x564068d6b770 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.498+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v108'474 uv448 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.498+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068e9fc60 con 0x564068d6b360 2024-04-25T11:38:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.501+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v108'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.501+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0000 con 0x564068d6b360 2024-04-25T11:38:07.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.504+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v108'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.492 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:07.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.504+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea13b0 con 0x564068d6b360 2024-04-25T11:38:07.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.508+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.508+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea1750 con 0x564068d6b360 2024-04-25T11:38:07.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.511+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.511+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.514+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.514+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.518+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v108'480 uv454 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.518+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.522+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v108'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.522+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.526+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v108'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.514 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:07.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.526+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.530+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.530+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.535+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'484 uv484 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.535+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.538+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'485 uv485 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.538+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.543+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v108'486 uv461 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.543+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.547+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'487 uv487 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.547+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.552+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v108'488 uv488 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.540 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:07.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.552+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea1af0 con 0x564068d6b360 2024-04-25T11:38:07.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.556+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'489 uv489 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.556+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea1af0 con 0x564068d6b360 2024-04-25T11:38:07.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.561+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'490 uv490 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.561+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea03a0 con 0x564068d6b360 2024-04-25T11:38:07.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.565+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'491 uv491 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.566+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.570+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'492 uv492 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.570+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.575+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v108'493 uv467 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.575+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.580+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v108'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.580+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.584+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v108'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.573 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.584+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.588+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.588+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.593+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.593+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.597+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.598+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.602+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v108'499 uv473 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.602+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.606+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v108'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.607+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.600 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:07.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.612+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v108'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.612+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.616+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.616+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.621+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.621+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068d6b360 2024-04-25T11:38:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.626+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.626+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564068ea1af0 0x564068ea3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.626+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea0660 con 0x564068ea1af0 2024-04-25T11:38:07.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.627+0000 7f3d29600640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564068ea1af0 0x564068ea3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.628+0000 7f3d29600640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564068ea1af0 0x564068ea3f10 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.632+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'352 uv351 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.633+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea0ff0 con 0x564068ea1af0 2024-04-25T11:38:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.637+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'353 uv353 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.637+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea4450 con 0x564068ea1af0 2024-04-25T11:38:07.629 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.641+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'354 uv354 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.641+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea47f0 con 0x564068ea1af0 2024-04-25T11:38:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.645+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v108'355 uv355 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.646+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea4b90 con 0x564068ea1af0 2024-04-25T11:38:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.649+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'356 uv356 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.649+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea4f30 con 0x564068ea1af0 2024-04-25T11:38:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.653+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v108'357 uv339 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.653+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.657+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v108'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.658+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.650 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.662+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v108'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.662+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.666+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.666+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.670+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.670+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.674+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v108'362 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.674+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.679+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v108'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.679+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.671 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.683+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v108'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.683+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea99c0 con 0x564068ea1af0 2024-04-25T11:38:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.688+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v108'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.688+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea99c0 con 0x564068ea1af0 2024-04-25T11:38:07.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.692+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'366 uv366 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.692+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.696+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v108'367 uv350 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.696+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.700+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v108'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.700+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.704+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v108'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.704+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea62e0 con 0x564068ea1af0 2024-04-25T11:38:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.708+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.708+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea62e0 con 0x564068ea1af0 2024-04-25T11:38:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.712+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.712+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea52d0 con 0x564068ea1af0 2024-04-25T11:38:07.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.716+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ea1af0 2024-04-25T11:38:07.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.716+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564068ea57e0 0x564068ea7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.717+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea8140 con 0x564068ea57e0 2024-04-25T11:38:07.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.718+0000 7f3d28c00640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564068ea57e0 0x564068ea7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.720+0000 7f3d28c00640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564068ea57e0 0x564068ea7c00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.723+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.723+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea84e0 con 0x564068ea57e0 2024-04-25T11:38:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.728+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v108'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.728+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea88c0 con 0x564068ea57e0 2024-04-25T11:38:07.720 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.732+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v108'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.732+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eaa170 con 0x564068ea57e0 2024-04-25T11:38:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.736+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.736+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eaa510 con 0x564068ea57e0 2024-04-25T11:38:07.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.740+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.741+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea9160 con 0x564068ea57e0 2024-04-25T11:38:07.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.744+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.745+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea9570 con 0x564068d6b360 2024-04-25T11:38:07.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.750+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'74 uv73 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.750+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ea9570 con 0x564068d6b360 2024-04-25T11:38:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.753+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'75 uv75 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.753+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea9570 con 0x564068d6b360 2024-04-25T11:38:07.746 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:07.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.758+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'76 uv76 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.758+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ea9570 con 0x564068d6b360 2024-04-25T11:38:07.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.762+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'77 uv77 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.762+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564068eaa8b0 0x564068eaccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.762+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ead210 con 0x564068eaa8b0 2024-04-25T11:38:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.763+0000 7f3d2a000640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564068eaa8b0 0x564068eaccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.765+0000 7f3d2a000640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564068eaa8b0 0x564068eaccd0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.769+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.769+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068ead210 con 0x564068eaa8b0 2024-04-25T11:38:07.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.773+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v108'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.773+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ead210 con 0x564068eaa8b0 2024-04-25T11:38:07.765 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:07.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.777+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v108'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.777+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eae220 con 0x564068eaa8b0 2024-04-25T11:38:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.781+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.781+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eae220 con 0x564068eaa8b0 2024-04-25T11:38:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.785+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.785+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068ead210 con 0x564068eaa8b0 2024-04-25T11:38:07.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.790+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.790+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564068ead890 0x564068eafcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.790+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb0280 con 0x564068ead890 2024-04-25T11:38:07.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.791+0000 7f3d29600640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564068ead890 0x564068eafcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.792+0000 7f3d29600640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564068ead890 0x564068eafcb0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.796+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v108'171 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.796+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb0340 con 0x564068ead890 2024-04-25T11:38:07.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.800+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v108'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.800+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb0760 con 0x564068ead890 2024-04-25T11:38:07.792 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:07.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.804+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v108'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.804+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb1770 con 0x564068ead890 2024-04-25T11:38:07.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.808+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.808+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb1770 con 0x564068ead890 2024-04-25T11:38:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.813+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.813+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb0760 con 0x564068ead890 2024-04-25T11:38:07.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.817+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.817+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb0760 con 0x564068ead890 2024-04-25T11:38:07.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.821+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v108'177 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.821+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb0760 con 0x564068ead890 2024-04-25T11:38:07.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.826+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v108'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.826+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb0760 con 0x564068ead890 2024-04-25T11:38:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.830+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v108'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.818 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.830+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb0760 con 0x564068ead890 2024-04-25T11:38:07.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.834+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.834+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb0760 con 0x564068ead890 2024-04-25T11:38:07.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.838+0000 7f3d29600640 1 -- 172.21.15.19:0/814939806 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c015030 con 0x564068ead890 2024-04-25T11:38:07.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.838+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564068eb0de0 0x564068eb3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:07.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.838+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb38b0 con 0x564068eb0de0 2024-04-25T11:38:07.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.839+0000 7f3d28c00640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564068eb0de0 0x564068eb3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.840+0000 7f3d28c00640 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564068eb0de0 0x564068eb3200 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.844+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v108'76 uv75 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.844+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb3c80 con 0x564068eb0de0 2024-04-25T11:38:07.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.848+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v108'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.848+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068eb0de0 2024-04-25T11:38:07.840 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:07.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.852+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v108'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.852+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068eb0de0 2024-04-25T11:38:07.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.856+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'79 uv79 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.856+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068eb0de0 2024-04-25T11:38:07.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.860+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'80 uv80 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.861+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068d6b360 2024-04-25T11:38:07.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.865+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.865+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068d6b360 2024-04-25T11:38:07.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.869+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v108'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.869+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068d6b360 2024-04-25T11:38:07.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.873+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v108'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.862 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:07.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.874+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb50b0 con 0x564068d6b360 2024-04-25T11:38:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.878+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.878+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb50b0 con 0x564068d6b360 2024-04-25T11:38:07.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.882+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.883+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068d6b360 2024-04-25T11:38:07.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.887+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068d6b360 2024-04-25T11:38:07.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.887+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.891+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v108'152 uv146 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.891+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.896+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v108'153 uv153 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.896+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.888 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:07.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.900+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v108'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.900+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.905+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.905+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.910+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.910+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.914+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v108'157 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.914+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.919+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v108'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.919+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.910 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:07.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.923+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v108'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.923+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb40a0 con 0x564068ea57e0 2024-04-25T11:38:07.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.927+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.928+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb6e50 con 0x564068ea57e0 2024-04-25T11:38:07.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.932+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068ea57e0 2024-04-25T11:38:07.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.932+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb6e10 con 0x564068eaa8b0 2024-04-25T11:38:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.936+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v108'70 uv69 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.937+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb6e10 con 0x564068eaa8b0 2024-04-25T11:38:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.942+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v108'71 uv71 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.942+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb6e10 con 0x564068eaa8b0 2024-04-25T11:38:07.933 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.945+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v108'72 uv72 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.945+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb6e10 con 0x564068eaa8b0 2024-04-25T11:38:07.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.950+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v108'73 uv73 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.950+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eaa8b0 2024-04-25T11:38:07.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.954+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v108'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.954+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eaa8b0 2024-04-25T11:38:07.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.958+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v108'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.958+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eaa8b0 2024-04-25T11:38:07.951 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.962+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v108'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.963+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eaa8b0 2024-04-25T11:38:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.967+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v108'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.967+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb5430 con 0x564068eaa8b0 2024-04-25T11:38:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.971+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v108'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.971+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eaa8b0 2024-04-25T11:38:07.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.975+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v108'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d2404e570 con 0x564068eaa8b0 2024-04-25T11:38:07.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.976+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eb0de0 2024-04-25T11:38:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.980+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v108'79 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.980+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eb0de0 2024-04-25T11:38:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.984+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v108'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.984+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb4420 con 0x564068eb0de0 2024-04-25T11:38:07.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.988+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v108'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.976 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:07.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.988+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb47e0 con 0x564068eb0de0 2024-04-25T11:38:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.992+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v108'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.992+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.996+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v108'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:07.996+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.000+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v108'192 uv186 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.000+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:07.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.005+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v108'193 uv193 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.005+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:07.997 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:07.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.009+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v108'194 uv194 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:07.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.009+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:08.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.014+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v108'195 uv195 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.014+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.019+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v108'196 uv196 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.019+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.023+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v108'197 uv191 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.023+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:08.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.027+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v108'198 uv198 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.027+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:08.019 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:08.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.030+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v108'199 uv199 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.031+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb54c0 con 0x564068eb0de0 2024-04-25T11:38:08.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.035+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v108'200 uv200 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.035+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb54c0 con 0x564068eb0de0 2024-04-25T11:38:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.039+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v108'201 uv201 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.039+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb44b0 con 0x564068eb0de0 2024-04-25T11:38:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.042+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v108'202 uv202 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.043+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x564068eb4c10 con 0x564068eb0de0 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d28c00640 1 -- 172.21.15.19:0/814939806 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v108'203 uv203 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3d14062120 con 0x564068eb0de0 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564068ea1af0 msgr2=0x564068ea3f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564068ea1af0 0x564068ea3f10 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564068ead890 msgr2=0x564068eafcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564068ead890 0x564068eafcb0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564068eb0de0 msgr2=0x564068eb3200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564068eb0de0 0x564068eb3200 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564068ea57e0 msgr2=0x564068ea7c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564068ea57e0 0x564068ea7c00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564068d6b360 msgr2=0x564068d6b770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d2a000640 1 -- 172.21.15.19:0/814939806 reap_dead start 2024-04-25T11:38:08.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564068d6b360 0x564068d6b770 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564068eaa8b0 msgr2=0x564068eaccd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.047+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564068eaa8b0 0x564068eaccd0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d08072e90 msgr2=0x7f3d080753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d08072e90 0x7f3d080753a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d6c0 tx=0x7f3d1c01a040 comp rx=0 tx=0).stop 2024-04-25T11:38:08.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564068dfd1c0 msgr2=0x564068dddcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564068dfd1c0 0x564068dddcb0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f3d14002ed0 tx=0x7f3d1400b610 comp rx=0 tx=0).stop 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 shutdown_connections 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564068eaa8b0 0x564068eaccd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564068d6b360 0x564068d6b770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d08072e90 0x7f3d080753a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 --2- 172.21.15.19:0/814939806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564068dfd1c0 0x564068dddcb0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 >> 172.21.15.19:0/814939806 conn(0x564068c48ac0 msgr2=0x564068d57d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 shutdown_connections 2024-04-25T11:38:08.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.048+0000 7f3d303dbf40 1 -- 172.21.15.19:0/814939806 wait complete. 2024-04-25T11:38:08.071 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2024-04-25T11:38:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.317+0000 7f8398fd0f40 1 Processor -- start 2024-04-25T11:38:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.317+0000 7f8398fd0f40 1 -- start start 2024-04-25T11:38:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8398fd0f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8398fd0f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c564ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8398fd0f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 0x5570c564bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8398fd0f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570c548dd20 con 0x5570c54e86e0 2024-04-25T11:38:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8398fd0f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5570c54e8af0 con 0x5570c55bb6d0 2024-04-25T11:38:08.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8398fd0f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5570c55bd680 con 0x5570c55bc960 2024-04-25T11:38:08.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8392200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8392200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55be940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57834/0 (socket says 172.21.15.19:57834) 2024-04-25T11:38:08.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.319+0000 7f8392200640 1 -- 172.21.15.19:0/2116465763 learned_addr learned my addr 172.21.15.19:0/2116465763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8391800640 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c564ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8392c00640 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 0x5570c564bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8392200640 1 -- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 msgr2=0x5570c564bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8392200640 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 0x5570c564bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8392200640 1 -- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 msgr2=0x5570c564ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8392200640 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c564ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8392200640 1 -- 172.21.15.19:0/2116465763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c55bdba0 con 0x5570c54e86e0 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f8392200640 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55be940 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8384001c80 tx=0x7f838400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4102bbbe30ddc834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f838be00640 1 -- 172.21.15.19:0/2116465763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838400bcc0 con 0x5570c54e86e0 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f838be00640 1 -- 172.21.15.19:0/2116465763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838400f400 con 0x5570c54e86e0 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.320+0000 7f838be00640 1 -- 172.21.15.19:0/2116465763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838400be60 con 0x5570c54e86e0 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 -- 172.21.15.19:0/2116465763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 msgr2=0x5570c55be940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55be940 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f8384001c80 tx=0x7f838400b5b0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 -- 172.21.15.19:0/2116465763 shutdown_connections 2024-04-25T11:38:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c564ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 0x5570c564bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/2116465763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55be940 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 -- 172.21.15.19:0/2116465763 >> 172.21.15.19:0/2116465763 conn(0x5570c5409ac0 msgr2=0x5570c552ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 -- 172.21.15.19:0/2116465763 shutdown_connections 2024-04-25T11:38:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.321+0000 7f8398fd0f40 1 -- 172.21.15.19:0/2116465763 wait complete. 2024-04-25T11:38:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.322+0000 7f8398fd0f40 1 Processor -- start 2024-04-25T11:38:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.323+0000 7f8398fd0f40 1 -- start start 2024-04-25T11:38:08.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8398fd0f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8398fd0f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c55af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8398fd0f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 0x5570c55b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8398fd0f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570c55bde60 con 0x5570c54e86e0 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8398fd0f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5570c55be140 con 0x5570c55bb6d0 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8398fd0f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5570c55b5a40 con 0x5570c55bc960 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8392200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8392200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57846/0 (socket says 172.21.15.19:57846) 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.324+0000 7f8392200640 1 -- 172.21.15.19:0/3172122363 learned_addr learned my addr 172.21.15.19:0/3172122363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8392c00640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 0x5570c55b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8391800640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c55af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8392200640 1 -- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 msgr2=0x5570c55b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8392200640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5570c55bc960 0x5570c55b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8392200640 1 -- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 msgr2=0x5570c55af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8392200640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c55af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8392200640 1 -- 172.21.15.19:0/3172122363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570c55af8c0 con 0x5570c54e86e0 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8391800640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5570c55bb6d0 0x5570c55af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8392200640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55b5110 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f838400bab0 tx=0x7f8384000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f838aa00640 1 -- 172.21.15.19:0/3172122363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838401c440 con 0x5570c54e86e0 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f838aa00640 1 -- 172.21.15.19:0/3172122363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8384021020 con 0x5570c54e86e0 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f838aa00640 1 -- 172.21.15.19:0/3172122363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838401c620 con 0x5570c54e86e0 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.325+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570c55b0210 con 0x5570c54e86e0 2024-04-25T11:38:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.326+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570c55afc50 con 0x5570c54e86e0 2024-04-25T11:38:08.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.328+0000 7f838aa00640 1 -- 172.21.15.19:0/3172122363 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8384017070 con 0x5570c54e86e0 2024-04-25T11:38:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.329+0000 7f838aa00640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8370072e90 0x7f83700753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.329+0000 7f838aa00640 1 -- 172.21.15.19:0/3172122363 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7f8384095d80 con 0x5570c54e86e0 2024-04-25T11:38:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.329+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5570c55bd010 0x5570c55119b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.329+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5570c5511ef0 con 0x5570c55bd010 2024-04-25T11:38:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.329+0000 7f8391800640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8370072e90 0x7f83700753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.330+0000 7f8392c00640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5570c55bd010 0x5570c55119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.330+0000 7f8392c00640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5570c55bd010 0x5570c55119b0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.331+0000 7f8391800640 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8370072e90 0x7f83700753a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f837c057000 tx=0x7f837c056040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8392c00640 1 -- 172.21.15.19:0/3172122363 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v108'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f838c011040 con 0x5570c55bd010 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5570c55bd010 msgr2=0x5570c55119b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5570c55bd010 0x5570c55119b0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8370072e90 msgr2=0x7f83700753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8370072e90 0x7f83700753a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f837c057000 tx=0x7f837c056040 comp rx=0 tx=0).stop 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 msgr2=0x5570c55b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 --2- 172.21.15.19:0/3172122363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570c54e86e0 0x5570c55b5110 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f838400bab0 tx=0x7f8384000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8392c00640 1 -- 172.21.15.19:0/3172122363 reap_dead start 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 shutdown_connections 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 >> 172.21.15.19:0/3172122363 conn(0x5570c5409ac0 msgr2=0x5570c552b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 shutdown_connections 2024-04-25T11:38:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.334+0000 7f8398fd0f40 1 -- 172.21.15.19:0/3172122363 wait complete. 2024-04-25T11:38:08.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.636+0000 7fe18c600640 1 Processor -- start 2024-04-25T11:38:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.636+0000 7fe18c600640 1 -- start start 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.638+0000 7fe18c600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe184160390 0x7fe18406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.638+0000 7fe18c600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe18406db10 0x7fe184070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe18c600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe18406df20 con 0x7fe1840af710 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe18c600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe1840afb20 con 0x7fe18406db10 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe18c600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe18406f440 con 0x7fe184160390 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe189c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe1840af710 0x7fe184164a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48500/0 (socket says 172.21.15.19:48500) 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe189c00640 1 -- 172.21.15.19:0/4196031585 learned_addr learned my addr 172.21.15.19:0/4196031585 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe18a600640 1 --2- 172.21.15.19:0/4196031585 >> v2:172.21.15.19:3300/0 conn(0x7fe18406db10 0x7fe184070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe189200640 1 --2- 172.21.15.19:0/4196031585 >> v2:172.21.15.169:3300/0 conn(0x7fe184160390 0x7fe18406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.639+0000 7fe183e00640 1 -- 172.21.15.19:0/4196031585 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2392477090 0 0) 0x7fe178001110 con 0x7fe1840af710 2024-04-25T11:38:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.640+0000 7fe183e00640 1 -- 172.21.15.19:0/4196031585 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe16c003520 con 0x7fe1840af710 2024-04-25T11:38:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.640+0000 7fe18a600640 1 -- 172.21.15.19:0/4196031585 >> v2:172.21.15.169:3300/0 conn(0x7fe184160390 msgr2=0x7fe18406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.640+0000 7fe18a600640 1 --2- 172.21.15.19:0/4196031585 >> v2:172.21.15.169:3300/0 conn(0x7fe184160390 0x7fe18406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.640+0000 7fe183e00640 1 -- 172.21.15.19:0/4196031585 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3881150198 0 0) 0x7fe16c003520 con 0x7fe1840af710 2024-04-25T11:38:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.640+0000 7fe18a600640 1 -- 172.21.15.19:0/4196031585 >> v1:172.21.15.19:6789/0 conn(0x7fe1840af710 legacy=0x7fe184164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.640+0000 7fe18a600640 1 -- 172.21.15.19:0/4196031585 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18406f720 con 0x7fe18406db10 2024-04-25T11:38:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.641+0000 7fe18a600640 1 --2- 172.21.15.19:0/4196031585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18406db10 0x7fe184070270 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fe1840afca0 tx=0x7fe17c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a84cc537f1b608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.641+0000 7fe183e00640 1 -- 172.21.15.19:0/4196031585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178001840 con 0x7fe18406db10 2024-04-25T11:38:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.642+0000 7fe183e00640 1 -- 172.21.15.19:0/4196031585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe17c007540 con 0x7fe18406db10 2024-04-25T11:38:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.642+0000 7fe183e00640 1 -- 172.21.15.19:0/4196031585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0078f0 con 0x7fe18406db10 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.642+0000 7fe18c600640 1 -- 172.21.15.19:0/4196031585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18406db10 msgr2=0x7fe184070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.642+0000 7fe18c600640 1 --2- 172.21.15.19:0/4196031585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18406db10 0x7fe184070270 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fe1840afca0 tx=0x7fe17c02a560 comp rx=0 tx=0).stop 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.643+0000 7fe18c600640 1 -- 172.21.15.19:0/4196031585 shutdown_connections 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.643+0000 7fe18c600640 1 --2- 172.21.15.19:0/4196031585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe18406db10 0x7fe184070270 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.643+0000 7fe18c600640 1 --2- 172.21.15.19:0/4196031585 >> v2:172.21.15.169:3300/0 conn(0x7fe184160390 0x7fe18406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.643+0000 7fe18c600640 1 -- 172.21.15.19:0/4196031585 >> 172.21.15.19:0/4196031585 conn(0x7fe184073090 msgr2=0x7fe1840734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.643+0000 7fe18c600640 1 -- 172.21.15.19:0/4196031585 shutdown_connections 2024-04-25T11:38:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.643+0000 7fe18c600640 1 -- 172.21.15.19:0/4196031585 wait complete. 2024-04-25T11:38:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.645+0000 7fe18c600640 1 Processor -- start 2024-04-25T11:38:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.645+0000 7fe18c600640 1 -- start start 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18c600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18406db10 0x7fe18415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18c600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1840af710 0x7fe18414e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18c600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe184160390 0x7fe184161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18c600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe18406f0d0 con 0x7fe184160390 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18c600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe18406ffc0 con 0x7fe18406db10 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18c600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1840b1110 con 0x7fe1840af710 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe184160390 0x7fe184161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe184160390 0x7fe184161880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57874/0 (socket says 172.21.15.19:57874) 2024-04-25T11:38:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.647+0000 7fe18a600640 1 -- 172.21.15.19:0/3130268771 learned_addr learned my addr 172.21.15.19:0/3130268771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe189c00640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18406db10 0x7fe18415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe189200640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1840af710 0x7fe18414e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe18a600640 1 -- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1840af710 msgr2=0x7fe18414e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe18a600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1840af710 0x7fe18414e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe18a600640 1 -- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18406db10 msgr2=0x7fe18415ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe18a600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18406db10 0x7fe18415ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe18a600640 1 -- 172.21.15.19:0/3130268771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18415f370 con 0x7fe184160390 2024-04-25T11:38:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.648+0000 7fe18a600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe184160390 0x7fe184161880 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fe1840d2970 tx=0x7fe17c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.649+0000 7fe181600640 1 -- 172.21.15.19:0/3130268771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c03f070 con 0x7fe184160390 2024-04-25T11:38:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.649+0000 7fe181600640 1 -- 172.21.15.19:0/3130268771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe17c031c70 con 0x7fe184160390 2024-04-25T11:38:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.649+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18415af30 con 0x7fe184160390 2024-04-25T11:38:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.649+0000 7fe181600640 1 -- 172.21.15.19:0/3130268771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c009450 con 0x7fe184160390 2024-04-25T11:38:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.649+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18415e630 con 0x7fe184160390 2024-04-25T11:38:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.650+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18415b450 con 0x7fe184160390 2024-04-25T11:38:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.651+0000 7fe181600640 1 -- 172.21.15.19:0/3130268771 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe17c003bd0 con 0x7fe184160390 2024-04-25T11:38:08.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.653+0000 7fe181600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1640744e0 0x7fe1640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:08.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.653+0000 7fe181600640 1 -- 172.21.15.19:0/3130268771 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7085+0+0 (secure 0 0 0) 0x7fe17c0b6460 con 0x7fe184160390 2024-04-25T11:38:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.654+0000 7fe189c00640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1640744e0 0x7fe1640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:08.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.655+0000 7fe189c00640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1640744e0 0x7fe1640769f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe178000f60 tx=0x7fe178009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.658+0000 7fe181600640 1 -- 172.21.15.19:0/3130268771 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe18415f370 con 0x7fe184160390 2024-04-25T11:38:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:08.911+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe184050740 con 0x7fe184160390 2024-04-25T11:38:09.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.135+0000 7fe181600640 1 -- 172.21.15.19:0/3130268771 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v197) ==== 162+0+0 (secure 0 0 0) 0x7fe17c031e20 con 0x7fe184160390 2024-04-25T11:38:09.123 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.138+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1640744e0 msgr2=0x7fe1640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.138+0000 7fe18c600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1640744e0 0x7fe1640769f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe178000f60 tx=0x7fe178009040 comp rx=0 tx=0).stop 2024-04-25T11:38:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.138+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe184160390 msgr2=0x7fe184161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.138+0000 7fe18c600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe184160390 0x7fe184161880 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fe1840d2970 tx=0x7fe17c004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 shutdown_connections 2024-04-25T11:38:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe1640744e0 0x7fe1640769f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe184160390 0x7fe184161880 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1840af710 0x7fe18414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 --2- 172.21.15.19:0/3130268771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe18406db10 0x7fe18415ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 >> 172.21.15.19:0/3130268771 conn(0x7fe184073090 msgr2=0x7fe1840b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 shutdown_connections 2024-04-25T11:38:09.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.139+0000 7fe18c600640 1 -- 172.21.15.19:0/3130268771 wait complete. 2024-04-25T11:38:09.136 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:09.148+0000 7ff003a00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-04-25T11:38:09.137 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:09.149+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:09.137 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:09.149+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:38:09.147 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:38:09.148+0000 7ff003a00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-04-25T11:38:09.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:09.277 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:09.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.448+0000 7fa8cd800640 1 Processor -- start 2024-04-25T11:38:09.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.448+0000 7fa8cd800640 1 -- start start 2024-04-25T11:38:09.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.450+0000 7fa8cd800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8c8160d60 0x7fa8c8161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.450+0000 7fa8cd800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c80b7940 con 0x7fa8c80d50e0 2024-04-25T11:38:09.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.450+0000 7fa8cd800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c8161280 con 0x7fa8c80d2cb0 2024-04-25T11:38:09.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.450+0000 7fa8cd800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c80d4600 con 0x7fa8c8160d60 2024-04-25T11:38:09.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.450+0000 7fa8c7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa8c80d50e0 0x7fa8c80d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48508/0 (socket says 172.21.15.19:48508) 2024-04-25T11:38:09.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.450+0000 7fa8c7e00640 1 -- 172.21.15.19:0/568721206 learned_addr learned my addr 172.21.15.19:0/568721206 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733268211 0 0) 0x7fa8b8001110 con 0x7fa8c80d50e0 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8ac003520 con 0x7fa8c80d50e0 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8cce00640 1 --2- 172.21.15.19:0/568721206 >> v2:172.21.15.169:3300/0 conn(0x7fa8c8160d60 0x7fa8c8161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 639688986 0 0) 0x7fa8ac003520 con 0x7fa8c80d50e0 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 >> v2:172.21.15.169:3300/0 conn(0x7fa8c8160d60 msgr2=0x7fa8c8161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 --2- 172.21.15.19:0/568721206 >> v2:172.21.15.169:3300/0 conn(0x7fa8c8160d60 0x7fa8c8161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 >> v1:172.21.15.169:6789/0 conn(0x7fa8c80d2cb0 legacy=0x7fa8c8160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c80d48e0 con 0x7fa8c80d50e0 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa8b8001840 con 0x7fa8c80d50e0 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 >> v1:172.21.15.19:6789/0 conn(0x7fa8c80d50e0 legacy=0x7fa8c80d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0039b0 0x7fa8ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8ac0051c0 0x7fa8ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80d4600 con 0x7fa8ac004800 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac00c160 con 0x7fa8ac0039b0 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.451+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ac00c360 con 0x7fa8ac0051c0 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8c7400640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8c7e00640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0039b0 0x7fa8ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8cce00640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8ac0051c0 0x7fa8ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8c7400640 1 -- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8ac0051c0 msgr2=0x7fa8ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8c7400640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8ac0051c0 0x7fa8ac00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8c7400640 1 -- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0039b0 msgr2=0x7fa8ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8c7400640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8ac0039b0 0x7fa8ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8c7400640 1 -- 172.21.15.19:0/568721206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ac003520 con 0x7fa8ac004800 2024-04-25T11:38:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.452+0000 7fa8cce00640 1 -- 172.21.15.19:0/568721206 reap_dead start 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.453+0000 7fa8c7400640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80d5b90 tx=0x7fa8ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37093f5ea41aaa37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.454+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4001370 con 0x7fa8ac004800 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.454+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b40052d0 con 0x7fa8ac004800 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.454+0000 7fa8c6a00640 1 -- 172.21.15.19:0/568721206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b402ab70 con 0x7fa8ac004800 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.454+0000 7fa8cd800640 1 -- 172.21.15.19:0/568721206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 msgr2=0x7fa8ac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.454+0000 7fa8cd800640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80d5b90 tx=0x7fa8ac005830 comp rx=0 tx=0).stop 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.455+0000 7fa8cd800640 1 -- 172.21.15.19:0/568721206 shutdown_connections 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.455+0000 7fa8cd800640 1 --2- 172.21.15.19:0/568721206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c10 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.455+0000 7fa8cd800640 1 -- 172.21.15.19:0/568721206 >> 172.21.15.19:0/568721206 conn(0x7fa8c806c5c0 msgr2=0x7fa8c806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.455+0000 7fa8cd800640 1 -- 172.21.15.19:0/568721206 shutdown_connections 2024-04-25T11:38:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.455+0000 7fa8cd800640 1 -- 172.21.15.19:0/568721206 wait complete. 2024-04-25T11:38:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.457+0000 7fa8cd800640 1 Processor -- start 2024-04-25T11:38:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.457+0000 7fa8cd800640 1 -- start start 2024-04-25T11:38:09.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8cd800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8c8157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8cd800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c8155750 0x7fa8c8156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8cd800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c80c85c0 0x7fa8c81469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8cd800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c80d54f0 con 0x7fa8ac004800 2024-04-25T11:38:09.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8cd800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8161560 con 0x7fa8c8155750 2024-04-25T11:38:09.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8cd800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c81475c0 con 0x7fa8c80c85c0 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8c7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8c8157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8c7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8c8157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57922/0 (socket says 172.21.15.19:57922) 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.459+0000 7fa8c7e00640 1 -- 172.21.15.19:0/736957424 learned_addr learned my addr 172.21.15.19:0/736957424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8c7e00640 1 -- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c80c85c0 msgr2=0x7fa8c81469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8c7e00640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c80c85c0 0x7fa8c81469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8c7e00640 1 -- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c8155750 msgr2=0x7fa8c8156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8c7400640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c8155750 0x7fa8c8156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8c7e00640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c8155750 0x7fa8c8156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8c7e00640 1 -- 172.21.15.19:0/736957424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c81574d0 con 0x7fa8ac004800 2024-04-25T11:38:09.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8c7e00640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8c8157b90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e700 tx=0x7fa8b800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.460+0000 7fa8bfe00640 1 -- 172.21.15.19:0/736957424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800c5f0 con 0x7fa8ac004800 2024-04-25T11:38:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.461+0000 7fa8bfe00640 1 -- 172.21.15.19:0/736957424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b8018070 con 0x7fa8ac004800 2024-04-25T11:38:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.461+0000 7fa8bfe00640 1 -- 172.21.15.19:0/736957424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800c9f0 con 0x7fa8ac004800 2024-04-25T11:38:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.461+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c8155ef0 con 0x7fa8ac004800 2024-04-25T11:38:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.461+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81564b0 con 0x7fa8ac004800 2024-04-25T11:38:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.462+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c8050740 con 0x7fa8ac004800 2024-04-25T11:38:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.463+0000 7fa8bfe00640 1 -- 172.21.15.19:0/736957424 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa8b800cb90 con 0x7fa8ac004800 2024-04-25T11:38:09.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.464+0000 7fa8bfe00640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8a80744b0 0x7fa8a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:09.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.464+0000 7fa8bfe00640 1 -- 172.21.15.19:0/736957424 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7fa8b8095560 con 0x7fa8ac004800 2024-04-25T11:38:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.465+0000 7fa8c7400640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8a80744b0 0x7fa8a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:09.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.466+0000 7fa8c7400640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8a80744b0 0x7fa8a80769c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4032000 tx=0x7fa8b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:09.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.469+0000 7fa8bfe00640 1 -- 172.21.15.19:0/736957424 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8c80b7940 con 0x7fa8ac004800 2024-04-25T11:38:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.715+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8c80b8aa0 con 0x7fa8ac004800 2024-04-25T11:38:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.717+0000 7fa8bfe00640 1 -- 172.21.15.19:0/736957424 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+4352 (secure 0 0 0) 0x7fa8b805f720 con 0x7fa8ac004800 2024-04-25T11:38:09.705 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:09.705 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":198,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:09.150495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:09.705 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 198 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.719+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8a80744b0 msgr2=0x7fa8a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.719+0000 7fa8cd800640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8a80744b0 0x7fa8a80769c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4032000 tx=0x7fa8b402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.719+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 msgr2=0x7fa8c8157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.719+0000 7fa8cd800640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8c8157b90 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e700 tx=0x7fa8b800eb00 comp rx=0 tx=0).stop 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 shutdown_connections 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8a80744b0 0x7fa8a80769c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c8155750 0x7fa8c8156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c80c85c0 0x7fa8c81469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 --2- 172.21.15.19:0/736957424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8ac004800 0x7fa8c8157b90 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:09.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 >> 172.21.15.19:0/736957424 conn(0x7fa8c806c5c0 msgr2=0x7fa8c815fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:09.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 shutdown_connections 2024-04-25T11:38:09.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:09.720+0000 7fa8cd800640 1 -- 172.21.15.19:0/736957424 wait complete. 2024-04-25T11:38:09.736 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:09.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.027+0000 7f1735400640 1 Processor -- start 2024-04-25T11:38:10.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.027+0000 7f1735400640 1 -- start start 2024-04-25T11:38:10.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.029+0000 7f1735400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17300b4ea0 0x7f17300cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.029+0000 7f1735400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1730144370 0x7f173015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.029+0000 7f1735400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17300d1230 con 0x7f173006c5c0 2024-04-25T11:38:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.029+0000 7f1735400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f17300d06b0 con 0x7f1730144370 2024-04-25T11:38:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.029+0000 7f1735400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f17300d0990 con 0x7f17300b4ea0 2024-04-25T11:38:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f173006c5c0 0x7f17300b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48516/0 (socket says 172.21.15.19:48516) 2024-04-25T11:38:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172fe00640 1 -- 172.21.15.19:0/948273631 learned_addr learned my addr 172.21.15.19:0/948273631 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f1734a00640 1 --2- 172.21.15.19:0/948273631 >> v2:172.21.15.169:3300/0 conn(0x7f1730144370 0x7f173015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172f400640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17300b4ea0 0x7f17300cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767083037 0 0) 0x7f17300d1230 con 0x7f173006c5c0 2024-04-25T11:38:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1714003520 con 0x7f173006c5c0 2024-04-25T11:38:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1010947288 0 0) 0x7f1714003520 con 0x7f173006c5c0 2024-04-25T11:38:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17300b4ea0 msgr2=0x7f17300cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.030+0000 7f172ea00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17300b4ea0 0x7f17300cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 >> v2:172.21.15.169:3300/0 conn(0x7f1730144370 msgr2=0x7f173015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 --2- 172.21.15.19:0/948273631 >> v2:172.21.15.169:3300/0 conn(0x7f1730144370 0x7f173015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17300d0c70 con 0x7f173006c5c0 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1720001670 con 0x7f173006c5c0 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 >> v1:172.21.15.19:6789/0 conn(0x7f173006c5c0 legacy=0x7f17300b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17140039b0 0x7f1714003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1714004800 0x7f1714004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1714005150 0x7f171400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17300d06b0 con 0x7f1714005150 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17300d0990 con 0x7f17140039b0 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f171400bf50 con 0x7f1714004800 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f1734a00640 1 --2- 172.21.15.19:0/948273631 >> v2:172.21.15.169:3300/0 conn(0x7f1730144370 0x7f173015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f1734a00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1714005150 0x7f171400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.031+0000 7f172f400640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1714004800 0x7f1714004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.032+0000 7f172fe00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17140039b0 0x7f1714003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.032+0000 7f1734a00640 1 -- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1714004800 msgr2=0x7f1714004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.032+0000 7f1734a00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1714004800 0x7f1714004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.032+0000 7f1734a00640 1 -- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17140039b0 msgr2=0x7f1714003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.032+0000 7f1734a00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17140039b0 0x7f1714003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.032+0000 7f1734a00640 1 -- 172.21.15.19:0/948273631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1714003520 con 0x7f1714005150 2024-04-25T11:38:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.032+0000 7f172fe00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17140039b0 0x7f1714003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.033+0000 7f1734a00640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1714005150 0x7f171400b9c0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f1728007410 tx=0x7f1714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bdac364df671d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.033+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728009c70 con 0x7f1714005150 2024-04-25T11:38:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.033+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172802c6c0 con 0x7f1714005150 2024-04-25T11:38:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.033+0000 7f172ea00640 1 -- 172.21.15.19:0/948273631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172802ca70 con 0x7f1714005150 2024-04-25T11:38:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.034+0000 7f1735400640 1 -- 172.21.15.19:0/948273631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1714005150 msgr2=0x7f171400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.034+0000 7f1735400640 1 --2- 172.21.15.19:0/948273631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1714005150 0x7f171400b9c0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f1728007410 tx=0x7f1714005830 comp rx=0 tx=0).stop 2024-04-25T11:38:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.034+0000 7f1734a00640 1 -- 172.21.15.19:0/948273631 reap_dead start 2024-04-25T11:38:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.034+0000 7f1735400640 1 -- 172.21.15.19:0/948273631 shutdown_connections 2024-04-25T11:38:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.034+0000 7f1735400640 1 -- 172.21.15.19:0/948273631 >> 172.21.15.19:0/948273631 conn(0x7f17300d1720 msgr2=0x7f17300d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.034+0000 7f1735400640 1 -- 172.21.15.19:0/948273631 shutdown_connections 2024-04-25T11:38:10.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.034+0000 7f1735400640 1 -- 172.21.15.19:0/948273631 wait complete. 2024-04-25T11:38:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.037+0000 7f1735400640 1 Processor -- start 2024-04-25T11:38:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.037+0000 7f1735400640 1 -- start start 2024-04-25T11:38:10.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.038+0000 7f1735400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1730160220 0x7f173015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.038+0000 7f1735400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17301358a0 0x7f1730140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.038+0000 7f1735400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173015e670 0x7f173015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.038+0000 7f1735400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17300d0450 con 0x7f173015e670 2024-04-25T11:38:10.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.038+0000 7f1735400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17300d02d0 con 0x7f17301358a0 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.038+0000 7f1735400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1730147760 con 0x7f1730160220 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173015e670 0x7f173015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173015e670 0x7f173015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57952/0 (socket says 172.21.15.19:57952) 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 -- 172.21.15.19:0/2510731603 learned_addr learned my addr 172.21.15.19:0/2510731603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f172fe00640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1730160220 0x7f173015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f172f400640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17301358a0 0x7f1730140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 -- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1730160220 msgr2=0x7f173015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1730160220 0x7f173015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 -- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17301358a0 msgr2=0x7f1730140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17301358a0 0x7f1730140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.039+0000 7f1734a00640 1 -- 172.21.15.19:0/2510731603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173015efc0 con 0x7f173015e670 2024-04-25T11:38:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.040+0000 7f1734a00640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173015e670 0x7f173015ea80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1728007410 tx=0x7f1728004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.040+0000 7f1727e00640 1 -- 172.21.15.19:0/2510731603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728009c70 con 0x7f173015e670 2024-04-25T11:38:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.040+0000 7f1727e00640 1 -- 172.21.15.19:0/2510731603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1728046070 con 0x7f173015e670 2024-04-25T11:38:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.040+0000 7f1727e00640 1 -- 172.21.15.19:0/2510731603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728034b20 con 0x7f173015e670 2024-04-25T11:38:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.040+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1730160720 con 0x7f173015e670 2024-04-25T11:38:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.040+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17300cbf30 con 0x7f173015e670 2024-04-25T11:38:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.041+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1730050740 con 0x7f173015e670 2024-04-25T11:38:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.043+0000 7f1727e00640 1 -- 172.21.15.19:0/2510731603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1728034cc0 con 0x7f173015e670 2024-04-25T11:38:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.044+0000 7f1727e00640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17100744b0 0x7f17100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.044+0000 7f1727e00640 1 -- 172.21.15.19:0/2510731603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7f17280b6960 con 0x7f173015e670 2024-04-25T11:38:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.044+0000 7f172fe00640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17100744b0 0x7f17100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.046+0000 7f172fe00640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17100744b0 0x7f17100769c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f17300cd9b0 tx=0x7f1720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.048+0000 7f1727e00640 1 -- 172.21.15.19:0/2510731603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1728080a90 con 0x7f173015e670 2024-04-25T11:38:10.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.292+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17300cdf70 con 0x7f173015e670 2024-04-25T11:38:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.296+0000 7f1727e00640 1 -- 172.21.15.19:0/2510731603 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) ==== 95+0+4352 (secure 0 0 0) 0x7f1728080390 con 0x7f173015e670 2024-04-25T11:38:10.284 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:10.284 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":198,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:09.150495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 198 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.298+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17100744b0 msgr2=0x7f17100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.298+0000 7f1735400640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17100744b0 0x7f17100769c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f17300cd9b0 tx=0x7f1720009040 comp rx=0 tx=0).stop 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.298+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173015e670 msgr2=0x7f173015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.298+0000 7f1735400640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173015e670 0x7f173015ea80 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f1728007410 tx=0x7f1728004ab0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 shutdown_connections 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17100744b0 0x7f17100769c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17301358a0 0x7f1730140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f173015e670 0x7f173015ea80 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 --2- 172.21.15.19:0/2510731603 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1730160220 0x7f173015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 >> 172.21.15.19:0/2510731603 conn(0x7f17300d1720 msgr2=0x7f173006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 shutdown_connections 2024-04-25T11:38:10.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.299+0000 7f1735400640 1 -- 172.21.15.19:0/2510731603 wait complete. 2024-04-25T11:38:10.314 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:10.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:10.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.606+0000 7fa884e00640 1 Processor -- start 2024-04-25T11:38:10.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.606+0000 7fa884e00640 1 -- start start 2024-04-25T11:38:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.608+0000 7fa884e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8800cf9e0 0x7fa880161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.608+0000 7fa884e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8800b0800 0x7fa880070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.608+0000 7fa884e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8801615d0 con 0x7fa8800707d0 2024-04-25T11:38:10.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.608+0000 7fa884e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa880070be0 con 0x7fa8800b0800 2024-04-25T11:38:10.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.608+0000 7fa884e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa8800cffc0 con 0x7fa8800cf9e0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.609+0000 7fa87fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa8800707d0 0x7fa8801569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48522/0 (socket says 172.21.15.19:48522) 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.609+0000 7fa87fe00640 1 -- 172.21.15.19:0/2087672770 learned_addr learned my addr 172.21.15.19:0/2087672770 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.609+0000 7fa87f400640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8800cf9e0 0x7fa880161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.609+0000 7fa87ea00640 1 --2- 172.21.15.19:0/2087672770 >> v2:172.21.15.169:3300/0 conn(0x7fa8800b0800 0x7fa880070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.609+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749555413 0 0) 0x7fa878001490 con 0x7fa8800707d0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.609+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa864003520 con 0x7fa8800707d0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3421338178 0 0) 0x7fa864003520 con 0x7fa8800707d0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8800cf9e0 msgr2=0x7fa880161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8800cf9e0 0x7fa880161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 >> v2:172.21.15.169:3300/0 conn(0x7fa8800b0800 msgr2=0x7fa880070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 --2- 172.21.15.19:0/2087672770 >> v2:172.21.15.169:3300/0 conn(0x7fa8800b0800 0x7fa880070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8800d02a0 con 0x7fa8800707d0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa878001bc0 con 0x7fa8800707d0 2024-04-25T11:38:10.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 >> v1:172.21.15.19:6789/0 conn(0x7fa8800707d0 legacy=0x7fa8801569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8800cf9e0 0x7fa8640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa864003f30 0x7fa864004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa864004d40 0x7fa864005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa880070be0 con 0x7fa864003f30 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800cffc0 con 0x7fa864004d40 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa86400bac0 con 0x7fa8800cf9e0 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.610+0000 7fa87ea00640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa864003f30 0x7fa864004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.611+0000 7fa87ea00640 1 -- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8800cf9e0 msgr2=0x7fa8640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.611+0000 7fa87ea00640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8800cf9e0 0x7fa8640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.611+0000 7fa87ea00640 1 -- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa864004d40 msgr2=0x7fa864005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.611+0000 7fa87ea00640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa864004d40 0x7fa864005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.611+0000 7fa87ea00640 1 -- 172.21.15.19:0/2087672770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa864003520 con 0x7fa864003f30 2024-04-25T11:38:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.612+0000 7fa87ea00640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa864003f30 0x7fa864004800 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009a90 tx=0x7fa864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e131bbff0b978dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.612+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c036070 con 0x7fa864003f30 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.612+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c031400 con 0x7fa864003f30 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.612+0000 7fa87e000640 1 -- 172.21.15.19:0/2087672770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c009690 con 0x7fa864003f30 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.613+0000 7fa884e00640 1 -- 172.21.15.19:0/2087672770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa864003f30 msgr2=0x7fa864004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.613+0000 7fa884e00640 1 --2- 172.21.15.19:0/2087672770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa864003f30 0x7fa864004800 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009a90 tx=0x7fa864005830 comp rx=0 tx=0).stop 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.613+0000 7fa87fe00640 1 -- 172.21.15.19:0/2087672770 reap_dead start 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.613+0000 7fa884e00640 1 -- 172.21.15.19:0/2087672770 shutdown_connections 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.613+0000 7fa884e00640 1 -- 172.21.15.19:0/2087672770 >> 172.21.15.19:0/2087672770 conn(0x7fa8800d1720 msgr2=0x7fa8800d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.613+0000 7fa884e00640 1 -- 172.21.15.19:0/2087672770 shutdown_connections 2024-04-25T11:38:10.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.613+0000 7fa884e00640 1 -- 172.21.15.19:0/2087672770 wait complete. 2024-04-25T11:38:10.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.615+0000 7fa884e00640 1 Processor -- start 2024-04-25T11:38:10.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.615+0000 7fa884e00640 1 -- start start 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa884e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa864003f30 0x7fa880143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa884e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880152b30 0x7fa880160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa884e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880152f40 0x7fa8801609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa884e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800b0c10 con 0x7fa880152b30 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa884e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800d0860 con 0x7fa880152f40 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa884e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa8800b0d90 con 0x7fa864003f30 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa87ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880152b30 0x7fa880160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa87ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880152b30 0x7fa880160480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57980/0 (socket says 172.21.15.19:57980) 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.617+0000 7fa87ea00640 1 -- 172.21.15.19:0/1078974935 learned_addr learned my addr 172.21.15.19:0/1078974935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa87f400640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa864003f30 0x7fa880143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa87ea00640 1 -- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa864003f30 msgr2=0x7fa880143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa87ea00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa864003f30 0x7fa880143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa87ea00640 1 -- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880152f40 msgr2=0x7fa8801609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa87ea00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880152f40 0x7fa8801609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa87ea00640 1 -- 172.21.15.19:0/1078974935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8801444f0 con 0x7fa880152b30 2024-04-25T11:38:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa87ea00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880152b30 0x7fa880160480 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034040 tx=0x7fa86c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.618+0000 7fa877400640 1 -- 172.21.15.19:0/1078974935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c040020 con 0x7fa880152b30 2024-04-25T11:38:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.619+0000 7fa877400640 1 -- 172.21.15.19:0/1078974935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c003c60 con 0x7fa880152b30 2024-04-25T11:38:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.619+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8801536a0 con 0x7fa880152b30 2024-04-25T11:38:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.619+0000 7fa877400640 1 -- 172.21.15.19:0/1078974935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c036040 con 0x7fa880152b30 2024-04-25T11:38:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.619+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa880153c60 con 0x7fa880152b30 2024-04-25T11:38:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.620+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa880153410 con 0x7fa880152b30 2024-04-25T11:38:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.621+0000 7fa877400640 1 -- 172.21.15.19:0/1078974935 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa86c043020 con 0x7fa880152b30 2024-04-25T11:38:10.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.622+0000 7fa877400640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8600744e0 0x7fa8600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:10.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.622+0000 7fa877400640 1 -- 172.21.15.19:0/1078974935 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7fa86c0b5b00 con 0x7fa880152b30 2024-04-25T11:38:10.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.623+0000 7fa87f400640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8600744e0 0x7fa8600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:10.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.624+0000 7fa87f400640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8600744e0 0x7fa8600769f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa870009160 tx=0x7fa870008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:10.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.627+0000 7fa877400640 1 -- 172.21.15.19:0/1078974935 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa86c04a070 con 0x7fa880152b30 2024-04-25T11:38:10.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.868+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa880050740 con 0x7fa880152b30 2024-04-25T11:38:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.870+0000 7fa877400640 1 -- 172.21.15.19:0/1078974935 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 199 v199) ==== 95+0+5175 (secure 0 0 0) 0x7fa86c0044e0 con 0x7fa880152b30 2024-04-25T11:38:10.858 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:10.858 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":199,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/896318924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":896318924},{"type":"v1","addr":"172.21.15.19:6835","nonce":896318924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:09.150495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:10.858 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 199 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.872+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8600744e0 msgr2=0x7fa8600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.872+0000 7fa884e00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8600744e0 0x7fa8600769f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa870009160 tx=0x7fa870008040 comp rx=0 tx=0).stop 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.872+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880152b30 msgr2=0x7fa880160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.872+0000 7fa884e00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880152b30 0x7fa880160480 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034040 tx=0x7fa86c004730 comp rx=0 tx=0).stop 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 shutdown_connections 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa8600744e0 0x7fa8600769f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880152f40 0x7fa8801609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa880152b30 0x7fa880160480 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 --2- 172.21.15.19:0/1078974935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa864003f30 0x7fa880143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 >> 172.21.15.19:0/1078974935 conn(0x7fa8800d1720 msgr2=0x7fa8800cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 shutdown_connections 2024-04-25T11:38:10.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:10.873+0000 7fa884e00640 1 -- 172.21.15.19:0/1078974935 wait complete. 2024-04-25T11:38:10.887 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2024-04-25T11:38:10.887 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2024-04-25T11:38:10.887 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:10.887 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:10.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:38:10.936 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:10.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:10.936 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:11.030 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:38:11.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:11.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.333+0000 7efd48000640 1 Processor -- start 2024-04-25T11:38:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.333+0000 7efd48000640 1 -- start start 2024-04-25T11:38:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.335+0000 7efd48000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd400b63a0 0x7efd401647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.335+0000 7efd48000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd4006c7d0 con 0x7efd40160360 2024-04-25T11:38:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.335+0000 7efd48000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd4006cab0 con 0x7efd40070680 2024-04-25T11:38:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.335+0000 7efd48000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd4006d100 con 0x7efd400b63a0 2024-04-25T11:38:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd44c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd40160360 0x7efd400b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48528/0 (socket says 172.21.15.19:48528) 2024-04-25T11:38:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd44c00640 1 -- 172.21.15.19:0/1376704812 learned_addr learned my addr 172.21.15.19:0/1376704812 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 310284535 0 0) 0x7efd4006c7d0 con 0x7efd40160360 2024-04-25T11:38:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd46000640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd400b63a0 0x7efd401647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd28003520 con 0x7efd40160360 2024-04-25T11:38:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 993417501 0 0) 0x7efd28003520 con 0x7efd40160360 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd400b63a0 msgr2=0x7efd401647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd400b63a0 0x7efd401647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 >> v1:172.21.15.169:6789/0 conn(0x7efd40070680 legacy=0x7efd400af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4006cd90 con 0x7efd40160360 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.336+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efd300019f0 con 0x7efd40160360 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 >> v1:172.21.15.19:6789/0 conn(0x7efd40160360 legacy=0x7efd400b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd3fe00640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd280039e0 0x7efd28003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd3fe00640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd28004800 0x7efd28004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd3fe00640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd28005270 0x7efd2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd4006d100 con 0x7efd280039e0 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd2800c130 con 0x7efd28005270 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd2800c330 con 0x7efd28004800 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd45600640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd280039e0 0x7efd28003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd44c00640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd28004800 0x7efd28004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.337+0000 7efd46000640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd28005270 0x7efd2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.338+0000 7efd45600640 1 -- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd28004800 msgr2=0x7efd28004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.338+0000 7efd45600640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd28004800 0x7efd28004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.338+0000 7efd45600640 1 -- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd28005270 msgr2=0x7efd2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:11.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.338+0000 7efd45600640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd28005270 0x7efd2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.338+0000 7efd45600640 1 -- 172.21.15.19:0/1376704812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd28003520 con 0x7efd280039e0 2024-04-25T11:38:11.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.339+0000 7efd45600640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd280039e0 0x7efd28003e50 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7efd34001720 tx=0x7efd28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20406291c50d96f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:11.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.339+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34001370 con 0x7efd280039e0 2024-04-25T11:38:11.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.339+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd340052d0 con 0x7efd280039e0 2024-04-25T11:38:11.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.339+0000 7efd3fe00640 1 -- 172.21.15.19:0/1376704812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3402abc0 con 0x7efd280039e0 2024-04-25T11:38:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.340+0000 7efd48000640 1 -- 172.21.15.19:0/1376704812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd280039e0 msgr2=0x7efd28003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.340+0000 7efd48000640 1 --2- 172.21.15.19:0/1376704812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd280039e0 0x7efd28003e50 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7efd34001720 tx=0x7efd28005830 comp rx=0 tx=0).stop 2024-04-25T11:38:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.340+0000 7efd46000640 1 -- 172.21.15.19:0/1376704812 reap_dead start 2024-04-25T11:38:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.340+0000 7efd48000640 1 -- 172.21.15.19:0/1376704812 shutdown_connections 2024-04-25T11:38:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.340+0000 7efd48000640 1 -- 172.21.15.19:0/1376704812 >> 172.21.15.19:0/1376704812 conn(0x7efd400d1ae0 msgr2=0x7efd400d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.340+0000 7efd48000640 1 -- 172.21.15.19:0/1376704812 shutdown_connections 2024-04-25T11:38:11.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.340+0000 7efd48000640 1 -- 172.21.15.19:0/1376704812 wait complete. 2024-04-25T11:38:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.342+0000 7efd48000640 1 Processor -- start 2024-04-25T11:38:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.343+0000 7efd48000640 1 -- start start 2024-04-25T11:38:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.344+0000 7efd48000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd280039e0 0x7efd4014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.344+0000 7efd48000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd40155500 0x7efd4015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.344+0000 7efd48000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd4015bfa0 0x7efd4015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.344+0000 7efd48000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd400afd30 con 0x7efd4015bfa0 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.344+0000 7efd48000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd400afbb0 con 0x7efd40155500 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.344+0000 7efd48000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd400b63a0 con 0x7efd280039e0 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.344+0000 7efd46000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd4015bfa0 0x7efd4015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd4015bfa0 0x7efd4015cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57998/0 (socket says 172.21.15.19:57998) 2024-04-25T11:38:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 -- 172.21.15.19:0/988353879 learned_addr learned my addr 172.21.15.19:0/988353879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd45600640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd280039e0 0x7efd4014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 -- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd280039e0 msgr2=0x7efd4014b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd280039e0 0x7efd4014b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd44c00640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd40155500 0x7efd4015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 -- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd40155500 msgr2=0x7efd4015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd40155500 0x7efd4015ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 -- 172.21.15.19:0/988353879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4015f730 con 0x7efd4015bfa0 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.345+0000 7efd46000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd4015bfa0 0x7efd4015cfa0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efd40163d40 tx=0x7efd3800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.346+0000 7efd3d600640 1 -- 172.21.15.19:0/988353879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38004440 con 0x7efd4015bfa0 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.346+0000 7efd3d600640 1 -- 172.21.15.19:0/988353879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd38019070 con 0x7efd4015bfa0 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.346+0000 7efd3d600640 1 -- 172.21.15.19:0/988353879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38003850 con 0x7efd4015bfa0 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.346+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd401f7420 con 0x7efd4015bfa0 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.346+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd401f79e0 con 0x7efd4015bfa0 2024-04-25T11:38:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.347+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd40050740 con 0x7efd4015bfa0 2024-04-25T11:38:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.348+0000 7efd3d600640 1 -- 172.21.15.19:0/988353879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd38012070 con 0x7efd4015bfa0 2024-04-25T11:38:11.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.350+0000 7efd3d600640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd240744e0 0x7efd240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:11.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.350+0000 7efd3d600640 1 -- 172.21.15.19:0/988353879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7efd38095190 con 0x7efd4015bfa0 2024-04-25T11:38:11.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.351+0000 7efd45600640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd240744e0 0x7efd240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.352+0000 7efd45600640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd240744e0 0x7efd240769f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efd34032000 tx=0x7efd3402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:11.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.354+0000 7efd3d600640 1 -- 172.21.15.19:0/988353879 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd3805f2c0 con 0x7efd4015bfa0 2024-04-25T11:38:11.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:11.595+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efd4006e100 con 0x7efd4015bfa0 2024-04-25T11:38:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.309+0000 7efd3d600640 1 -- 172.21.15.19:0/988353879 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v200) ==== 161+0+0 (secure 0 0 0) 0x7efd380045e0 con 0x7efd4015bfa0 2024-04-25T11:38:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:12.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.312+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd240744e0 msgr2=0x7efd240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.312+0000 7efd48000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd240744e0 0x7efd240769f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7efd34032000 tx=0x7efd3402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.312+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd4015bfa0 msgr2=0x7efd4015cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.312+0000 7efd48000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd4015bfa0 0x7efd4015cfa0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efd40163d40 tx=0x7efd3800f920 comp rx=0 tx=0).stop 2024-04-25T11:38:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 shutdown_connections 2024-04-25T11:38:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd240744e0 0x7efd240769f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd40155500 0x7efd4015ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd4015bfa0 0x7efd4015cfa0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 --2- 172.21.15.19:0/988353879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd280039e0 0x7efd4014b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 >> 172.21.15.19:0/988353879 conn(0x7efd400d1ae0 msgr2=0x7efd40163d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 shutdown_connections 2024-04-25T11:38:12.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.313+0000 7efd48000640 1 -- 172.21.15.19:0/988353879 wait complete. 2024-04-25T11:38:12.335 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:12.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.630+0000 7f34c5800640 1 Processor -- start 2024-04-25T11:38:12.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.630+0000 7f34c5800640 1 -- start start 2024-04-25T11:38:12.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.632+0000 7f34c5800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f34c006c5c0 0x7f34c00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.632+0000 7f34c5800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c00d1230 con 0x7f34c00b4ea0 2024-04-25T11:38:12.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.632+0000 7f34c5800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c00d06b0 con 0x7f34c0144370 2024-04-25T11:38:12.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.632+0000 7f34c5800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f34c00d0990 con 0x7f34c006c5c0 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.633+0000 7f34c4e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f34c00b4ea0 0x7f34c015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48538/0 (socket says 172.21.15.19:48538) 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.633+0000 7f34c4e00640 1 -- 172.21.15.19:0/1797616125 learned_addr learned my addr 172.21.15.19:0/1797616125 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.633+0000 7f34bfe00640 1 --2- 172.21.15.19:0/1797616125 >> v2:172.21.15.169:3300/0 conn(0x7f34c006c5c0 0x7f34c00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.633+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354134426 0 0) 0x7f34c00d1230 con 0x7f34c00b4ea0 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.633+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34a4003520 con 0x7f34c00b4ea0 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2331876812 0 0) 0x7f34a4003520 con 0x7f34c00b4ea0 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 >> v2:172.21.15.169:3300/0 conn(0x7f34c006c5c0 msgr2=0x7f34c00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 --2- 172.21.15.19:0/1797616125 >> v2:172.21.15.169:3300/0 conn(0x7f34c006c5c0 0x7f34c00cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 >> v1:172.21.15.169:6789/0 conn(0x7f34c0144370 legacy=0x7f34c00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c00d0c70 con 0x7f34c00b4ea0 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f34b80019f0 con 0x7f34c00b4ea0 2024-04-25T11:38:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 >> v1:172.21.15.19:6789/0 conn(0x7f34c00b4ea0 legacy=0x7f34c015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40039b0 0x7f34a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a4004800 0x7f34a4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34c00d0990 con 0x7f34a40051c0 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34a400c160 con 0x7f34a4004800 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34a400c360 con 0x7f34a40039b0 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34c4e00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34bfe00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40039b0 0x7f34a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.634+0000 7f34c4e00640 1 -- 172.21.15.19:0/1797616125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40039b0 msgr2=0x7f34a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.635+0000 7f34c4e00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40039b0 0x7f34a4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.635+0000 7f34c4e00640 1 -- 172.21.15.19:0/1797616125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a4004800 msgr2=0x7f34a4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.635+0000 7f34c4e00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a4004800 0x7f34a4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.635+0000 7f34c4e00640 1 -- 172.21.15.19:0/1797616125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a4003520 con 0x7f34a40051c0 2024-04-25T11:38:12.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.635+0000 7f34c4e00640 1 -- 172.21.15.19:0/1797616125 reap_dead start 2024-04-25T11:38:12.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.635+0000 7f34c4e00640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f34b800a860 tx=0x7f34a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270183d40ce64297 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:12.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.636+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b800a480 con 0x7f34a40051c0 2024-04-25T11:38:12.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.636+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b802f070 con 0x7f34a40051c0 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.636+0000 7f34bea00640 1 -- 172.21.15.19:0/1797616125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b802bbe0 con 0x7f34a40051c0 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.636+0000 7f34c5800640 1 -- 172.21.15.19:0/1797616125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a40051c0 msgr2=0x7f34a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.636+0000 7f34c5800640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f34b800a860 tx=0x7f34a4005830 comp rx=0 tx=0).stop 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.637+0000 7f34c5800640 1 -- 172.21.15.19:0/1797616125 shutdown_connections 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.637+0000 7f34c5800640 1 --2- 172.21.15.19:0/1797616125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a40051c0 0x7f34a400b9c0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.637+0000 7f34c5800640 1 -- 172.21.15.19:0/1797616125 >> 172.21.15.19:0/1797616125 conn(0x7f34c00d1720 msgr2=0x7f34c00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.637+0000 7f34c5800640 1 -- 172.21.15.19:0/1797616125 shutdown_connections 2024-04-25T11:38:12.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.637+0000 7f34c5800640 1 -- 172.21.15.19:0/1797616125 wait complete. 2024-04-25T11:38:12.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.639+0000 7f34c5800640 1 Processor -- start 2024-04-25T11:38:12.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.639+0000 7f34c5800640 1 -- start start 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.640+0000 7f34c5800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40051c0 0x7f34c0159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.640+0000 7f34c5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c0160960 0x7f34c0132bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34c5800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c0133100 0x7f34c01336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34c5800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34c00d0450 con 0x7f34c0160960 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34c5800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34c00d02d0 con 0x7f34c0133100 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34c5800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34c0147760 con 0x7f34a40051c0 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c0160960 0x7f34c0132bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c0160960 0x7f34c0132bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58008/0 (socket says 172.21.15.19:58008) 2024-04-25T11:38:12.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 -- 172.21.15.19:0/2151678931 learned_addr learned my addr 172.21.15.19:0/2151678931 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:12.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 -- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40051c0 msgr2=0x7f34c0159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bfe00640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40051c0 0x7f34c0159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40051c0 0x7f34c0159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 -- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c0133100 msgr2=0x7f34c01336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c0133100 0x7f34c01336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.641+0000 7f34bf400640 1 -- 172.21.15.19:0/2151678931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c01610d0 con 0x7f34c0160960 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.642+0000 7f34bf400640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c0160960 0x7f34c0132bc0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c500 tx=0x7f34ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.642+0000 7f34b7e00640 1 -- 172.21.15.19:0/2151678931 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0052d0 con 0x7f34c0160960 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.642+0000 7f34b7e00640 1 -- 172.21.15.19:0/2151678931 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ac005470 con 0x7f34c0160960 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.642+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c01613b0 con 0x7f34c0160960 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.642+0000 7f34b7e00640 1 -- 172.21.15.19:0/2151678931 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac012850 con 0x7f34c0160960 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.642+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c0161970 con 0x7f34c0160960 2024-04-25T11:38:12.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.643+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c0160e40 con 0x7f34c0160960 2024-04-25T11:38:12.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.644+0000 7f34b7e00640 1 -- 172.21.15.19:0/2151678931 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34ac0129f0 con 0x7f34c0160960 2024-04-25T11:38:12.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.645+0000 7f34b7e00640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34a00744b0 0x7f34a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:12.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.645+0000 7f34b7e00640 1 -- 172.21.15.19:0/2151678931 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7f34ac095510 con 0x7f34c0160960 2024-04-25T11:38:12.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.646+0000 7f34bfe00640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34a00744b0 0x7f34a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:12.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.647+0000 7f34bfe00640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34a00744b0 0x7f34a00769c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009910 tx=0x7f34b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:12.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.650+0000 7f34b7e00640 1 -- 172.21.15.19:0/2151678931 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34ac05f760 con 0x7f34c0160960 2024-04-25T11:38:12.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:12.939+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f34c006ccb0 con 0x7f34c0160960 2024-04-25T11:38:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.333+0000 7f34b7e00640 1 -- 172.21.15.19:0/2151678931 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v201) ==== 101+0+0 (secure 0 0 0) 0x7f34ac05f060 con 0x7f34c0160960 2024-04-25T11:38:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:38:13.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34a00744b0 msgr2=0x7f34a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34a00744b0 0x7f34a00769c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009910 tx=0x7f34b0008040 comp rx=0 tx=0).stop 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c0160960 msgr2=0x7f34c0132bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c0160960 0x7f34c0132bc0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c500 tx=0x7f34ac00c9c0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 shutdown_connections 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f34a00744b0 0x7f34a00769c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c0133100 0x7f34c01336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34c0160960 0x7f34c0132bc0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 --2- 172.21.15.19:0/2151678931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a40051c0 0x7f34c0159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 >> 172.21.15.19:0/2151678931 conn(0x7f34c00d1720 msgr2=0x7f34c015e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.337+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 shutdown_connections 2024-04-25T11:38:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.338+0000 7f34c5800640 1 -- 172.21.15.19:0/2151678931 wait complete. 2024-04-25T11:38:13.362 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:13.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.560+0000 7fac2de72f40 1 Processor -- start 2024-04-25T11:38:13.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.560+0000 7fac2de72f40 1 -- start start 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.561+0000 7fac2de72f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c81159f1e0 0x55c811674590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.561+0000 7fac2de72f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f5f0 0x55c81159e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.561+0000 7fac2de72f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c811678fa0 0x55c8115d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.561+0000 7fac2de72f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c811545d20 con 0x55c81159f5f0 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.561+0000 7fac2de72f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c811549d60 con 0x55c811678fa0 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.561+0000 7fac2de72f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c8115e83f0 con 0x55c81159f1e0 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f5f0 0x55c81159e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f5f0 0x55c81159e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58014/0 (socket says 172.21.15.19:58014) 2024-04-25T11:38:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 -- 172.21.15.19:0/2322030556 learned_addr learned my addr 172.21.15.19:0/2322030556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac27000640 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c81159f1e0 0x55c811674590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac27a00640 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c811678fa0 0x55c8115d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 -- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c81159f1e0 msgr2=0x55c811674590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c81159f1e0 0x55c811674590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 -- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c811678fa0 msgr2=0x55c8115d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c811678fa0 0x55c8115d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.562+0000 7fac26600640 1 -- 172.21.15.19:0/2322030556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8115ea1f0 con 0x55c81159f5f0 2024-04-25T11:38:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.563+0000 7fac26600640 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f5f0 0x55c81159e180 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fac10001df0 tx=0x7fac1000b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea30075ec5ab3f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.563+0000 7fac25200640 1 -- 172.21.15.19:0/2322030556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1000be30 con 0x55c81159f5f0 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.563+0000 7fac25200640 1 -- 172.21.15.19:0/2322030556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac10006070 con 0x55c81159f5f0 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.563+0000 7fac25200640 1 -- 172.21.15.19:0/2322030556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1000e7e0 con 0x55c81159f5f0 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 -- 172.21.15.19:0/2322030556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f5f0 msgr2=0x55c81159e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f5f0 0x55c81159e180 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fac10001df0 tx=0x7fac1000b600 comp rx=0 tx=0).stop 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 -- 172.21.15.19:0/2322030556 shutdown_connections 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c811678fa0 0x55c8115d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f5f0 0x55c81159e180 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2322030556 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c81159f1e0 0x55c811674590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 -- 172.21.15.19:0/2322030556 >> 172.21.15.19:0/2322030556 conn(0x55c8114c1ac0 msgr2=0x55c8115ec720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 -- 172.21.15.19:0/2322030556 shutdown_connections 2024-04-25T11:38:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.564+0000 7fac2de72f40 1 -- 172.21.15.19:0/2322030556 wait complete. 2024-04-25T11:38:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.565+0000 7fac2de72f40 1 Processor -- start 2024-04-25T11:38:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.565+0000 7fac2de72f40 1 -- start start 2024-04-25T11:38:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac2de72f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f1e0 0x55c811673910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac2de72f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c81159f5f0 0x55c81159e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac2de72f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c811678fa0 0x55c81166b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac2de72f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c811675860 con 0x55c81159f1e0 2024-04-25T11:38:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac2de72f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c8115d1280 con 0x55c81159f5f0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac2de72f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c81166f5e0 con 0x55c811678fa0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f1e0 0x55c811673910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f1e0 0x55c811673910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58024/0 (socket says 172.21.15.19:58024) 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 learned_addr learned my addr 172.21.15.19:0/2365310660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27a00640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c811678fa0 0x55c81166b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c811678fa0 msgr2=0x55c81166b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c811678fa0 0x55c81166b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c81159f5f0 msgr2=0x55c81159e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c81159f5f0 0x55c81159e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.567+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81166c620 con 0x55c81159f1e0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.568+0000 7fac27000640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f1e0 0x55c811673910 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fac18000f80 tx=0x7fac18003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.568+0000 7fac1f400640 1 -- 172.21.15.19:0/2365310660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18068070 con 0x55c81159f1e0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.568+0000 7fac1f400640 1 -- 172.21.15.19:0/2365310660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18059450 con 0x55c81159f1e0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.568+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c81166bcc0 con 0x55c81159f1e0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.568+0000 7fac1f400640 1 -- 172.21.15.19:0/2365310660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18054040 con 0x55c81159f1e0 2024-04-25T11:38:13.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.568+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8116727f0 con 0x55c81159f1e0 2024-04-25T11:38:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.570+0000 7fac1f400640 1 -- 172.21.15.19:0/2365310660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac1806b070 con 0x55c81159f1e0 2024-04-25T11:38:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.571+0000 7fac1f400640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac04072ec0 0x7fac040753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.572+0000 7fac1f400640 1 -- 172.21.15.19:0/2365310660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7fac180d7dc0 con 0x55c81159f1e0 2024-04-25T11:38:13.560 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.572+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c81166e650 0x55c81166ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.572+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117077a0 con 0x55c81166e650 2024-04-25T11:38:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.572+0000 7fac26600640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac04072ec0 0x7fac040753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.573+0000 7fac27a00640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c81166e650 0x55c81166ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.574+0000 7fac26600640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac04072ec0 0x7fac040753d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fac10001df0 tx=0x7fac10006000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.574+0000 7fac27a00640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c81166e650 0x55c81166ea60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.577+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v109'505 uv479 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.577+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811718960 con 0x55c81166e650 2024-04-25T11:38:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.581+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v109'506 uv506 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.581+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811718d00 con 0x55c81166e650 2024-04-25T11:38:13.571 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.584+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v109'507 uv507 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.584+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171a0b0 con 0x55c81166e650 2024-04-25T11:38:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.586+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'508 uv508 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.586+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171a450 con 0x55c81166e650 2024-04-25T11:38:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.589+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'509 uv509 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.589+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117190a0 con 0x55c81166e650 2024-04-25T11:38:13.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.593+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.593+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719490 con 0x55c81166e650 2024-04-25T11:38:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.596+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v109'511 uv485 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.597+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811719880 con 0x55c81166e650 2024-04-25T11:38:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.600+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v109'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.600+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719880 con 0x55c81166e650 2024-04-25T11:38:13.591 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.603+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v109'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.603+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.607+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.607+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.610+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.611+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.614+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.614+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.618+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v109'517 uv492 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.618+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.621+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'518 uv518 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.622+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.613 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:13.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.625+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v109'519 uv519 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.625+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.628+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'520 uv520 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.629+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.633+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'521 uv521 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.633+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.636+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'522 uv522 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.637+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.641+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'523 uv523 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.641+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.645+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v109'524 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.645+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.649+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v109'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.649+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.641 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:13.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.653+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v109'526 uv526 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.653+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.657+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'527 uv527 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.657+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.662+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'528 uv528 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.662+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.666+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'529 uv529 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.666+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.670+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v109'530 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.670+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.674+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v109'531 uv531 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.674+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.666 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.678+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v109'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.678+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.682+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.682+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.686+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.686+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c81166e650 2024-04-25T11:38:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.691+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.691+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c811719bf0 0x55c81171cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.691+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811719830 con 0x55c811719bf0 2024-04-25T11:38:13.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.692+0000 7fac27000640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c811719bf0 0x55c81171cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.693+0000 7fac27000640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c811719bf0 0x55c81171cd10 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.698+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'374 uv356 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.698+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81171d250 con 0x55c811719bf0 2024-04-25T11:38:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.703+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'375 uv375 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.703+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171d5f0 con 0x55c811719bf0 2024-04-25T11:38:13.695 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.707+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'376 uv376 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.707+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171d990 con 0x55c811719bf0 2024-04-25T11:38:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.711+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v109'377 uv377 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.711+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171dd30 con 0x55c811719bf0 2024-04-25T11:38:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.716+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'378 uv378 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.716+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171e0d0 con 0x55c811719bf0 2024-04-25T11:38:13.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.720+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v109'379 uv373 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.721+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81171e470 con 0x55c811719bf0 2024-04-25T11:38:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.725+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v109'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.725+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171e470 con 0x55c811719bf0 2024-04-25T11:38:13.717 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.729+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v109'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.729+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.734+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.734+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.739+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.739+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.743+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v109'384 uv366 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.743+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.748+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v109'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.748+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.740 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.752+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v109'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.752+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.758+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v109'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.758+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.762+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.762+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.766+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v109'389 uv372 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.767+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.771+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v109'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.771+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.763 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:13.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.775+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v109'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.775+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.779+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.779+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.784+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.784+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8115cf0e0 con 0x55c811719bf0 2024-04-25T11:38:13.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.788+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811719bf0 2024-04-25T11:38:13.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.788+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c81171ee30 0x55c811721250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.788+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811721790 con 0x55c81171ee30 2024-04-25T11:38:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.791+0000 7fac26600640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c81171ee30 0x55c811721250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.791+0000 7fac26600640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c81171ee30 0x55c811721250 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.795+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.796+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811721b30 con 0x55c81171ee30 2024-04-25T11:38:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.800+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v109'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.800+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811721f50 con 0x55c81171ee30 2024-04-25T11:38:13.793 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:13.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.805+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v109'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.805+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81171e550 con 0x55c81171ee30 2024-04-25T11:38:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.809+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.809+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117238c0 con 0x55c81171ee30 2024-04-25T11:38:13.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.813+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.813+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117227f0 con 0x55c81171ee30 2024-04-25T11:38:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.818+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.818+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811722c00 con 0x55c81166e650 2024-04-25T11:38:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.823+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'78 uv77 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.823+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811722c00 con 0x55c81166e650 2024-04-25T11:38:13.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.827+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'79 uv79 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.827+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811722c00 con 0x55c81166e650 2024-04-25T11:38:13.819 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:13.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.831+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'80 uv80 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.831+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811722c20 con 0x55c81166e650 2024-04-25T11:38:13.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.836+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'81 uv81 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.836+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c811723c60 0x55c811726080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.836+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117265c0 con 0x55c811723c60 2024-04-25T11:38:13.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.836+0000 7fac27a00640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c811723c60 0x55c811726080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.838+0000 7fac27a00640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c811723c60 0x55c811726080 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.842+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:13.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.842+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c8117265c0 con 0x55c811723c60 2024-04-25T11:38:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.846+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v109'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.846+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117265c0 con 0x55c811723c60 2024-04-25T11:38:13.838 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:13.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.850+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v109'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:13.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.850+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117275d0 con 0x55c811723c60 2024-04-25T11:38:13.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.854+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:13.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.854+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117275d0 con 0x55c811723c60 2024-04-25T11:38:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.858+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:13.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.859+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117265c0 con 0x55c811723c60 2024-04-25T11:38:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.862+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.863+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c811726c90 0x55c8117270c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.863+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117299c0 con 0x55c811726c90 2024-04-25T11:38:13.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.863+0000 7fac27000640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c811726c90 0x55c8117270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.865+0000 7fac27000640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c811726c90 0x55c8117270c0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.869+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v109'182 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.869+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c8117299c0 con 0x55c811726c90 2024-04-25T11:38:13.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.873+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v109'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.873+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811729db0 con 0x55c811726c90 2024-04-25T11:38:13.865 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.878+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v109'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.878+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172adc0 con 0x55c811726c90 2024-04-25T11:38:13.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.881+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.881+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172adc0 con 0x55c811726c90 2024-04-25T11:38:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.886+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.886+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811729db0 con 0x55c811726c90 2024-04-25T11:38:13.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.889+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.890+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811729db0 con 0x55c811726c90 2024-04-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.893+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v109'188 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.894+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c811729db0 con 0x55c811726c90 2024-04-25T11:38:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.898+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v109'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.898+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811729db0 con 0x55c811726c90 2024-04-25T11:38:13.890 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:13.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.902+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v109'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.902+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811729db0 con 0x55c811726c90 2024-04-25T11:38:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.907+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.907+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811729db0 con 0x55c811726c90 2024-04-25T11:38:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.911+0000 7fac27000640 1 -- 172.21.15.19:0/2365310660 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac1806a030 con 0x55c811726c90 2024-04-25T11:38:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.911+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c81172a2c0 0x55c81172c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.911+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172ce70 con 0x55c81172a2c0 2024-04-25T11:38:13.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.912+0000 7fac26600640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c81172a2c0 0x55c81172c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:13.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.914+0000 7fac26600640 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c81172a2c0 0x55c81172c720 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.918+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v109'81 uv80 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.918+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81172d300 con 0x55c81172a2c0 2024-04-25T11:38:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.923+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v109'82 uv82 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:13.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.923+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81172a2c0 2024-04-25T11:38:13.914 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:13.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.926+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v109'83 uv83 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.926+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81172a2c0 2024-04-25T11:38:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.930+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v109'84 uv84 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.931+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81172a2c0 2024-04-25T11:38:13.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.935+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v109'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:13.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.935+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81166e650 2024-04-25T11:38:13.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.940+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.940+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81166e650 2024-04-25T11:38:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.944+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v109'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.945+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81166e650 2024-04-25T11:38:13.937 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.949+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v109'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.949+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172e730 con 0x55c81166e650 2024-04-25T11:38:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.953+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.953+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172e730 con 0x55c81166e650 2024-04-25T11:38:13.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.958+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.958+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81166e650 2024-04-25T11:38:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.962+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c81166e650 2024-04-25T11:38:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.962+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.967+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v109'162 uv156 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.967+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.971+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v109'163 uv163 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.971+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.963 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:13.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.975+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v109'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.975+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.980+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v109'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.980+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.985+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v109'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.985+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.989+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v109'167 uv161 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.989+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.994+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v109'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.994+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.986 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.998+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v109'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:13.998+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172d720 con 0x55c81171ee30 2024-04-25T11:38:13.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.003+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v109'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.003+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c811722430 con 0x55c81171ee30 2024-04-25T11:38:13.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.008+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v109'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81171ee30 2024-04-25T11:38:13.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.008+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223f0 con 0x55c811723c60 2024-04-25T11:38:14.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.012+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v109'74 uv73 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.012+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c8117223f0 con 0x55c811723c60 2024-04-25T11:38:14.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.016+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v109'75 uv75 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.016+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223f0 con 0x55c811723c60 2024-04-25T11:38:14.008 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.021+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v109'76 uv76 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.021+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223f0 con 0x55c811723c60 2024-04-25T11:38:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.024+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v109'77 uv77 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.024+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223f0 con 0x55c811723c60 2024-04-25T11:38:14.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.028+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v109'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.028+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c8117223f0 con 0x55c811723c60 2024-04-25T11:38:14.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.032+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v109'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.033+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223d0 con 0x55c811723c60 2024-04-25T11:38:14.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.037+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v109'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.025 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:14.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.037+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223d0 con 0x55c811723c60 2024-04-25T11:38:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.041+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v109'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.041+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223d0 con 0x55c811723c60 2024-04-25T11:38:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.045+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v109'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.045+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223d0 con 0x55c811723c60 2024-04-25T11:38:14.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.049+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac2000b4d0 con 0x55c811723c60 2024-04-25T11:38:14.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.049+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223d0 con 0x55c81172a2c0 2024-04-25T11:38:14.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.053+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v109'84 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.053+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c8117223d0 con 0x55c81172a2c0 2024-04-25T11:38:14.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.058+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v109'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.058+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c8117223d0 con 0x55c81172a2c0 2024-04-25T11:38:14.050 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.062+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v109'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.062+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.066+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v109'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.066+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.070+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v109'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.070+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.075+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v109'204 uv196 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.075+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.079+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v109'205 uv205 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.079+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.084+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v109'206 uv206 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.072 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:14.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.084+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.088+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v109'207 uv207 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.088+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.093+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v109'208 uv208 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.093+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.097+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v109'209 uv203 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.097+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.102+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v109'210 uv210 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.102+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.093 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:14.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.105+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v109'211 uv211 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.105+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172ee40 con 0x55c81172a2c0 2024-04-25T11:38:14.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.110+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v109'212 uv212 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.110+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172ee40 con 0x55c81172a2c0 2024-04-25T11:38:14.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.114+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v109'213 uv213 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.114+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172de30 con 0x55c81172a2c0 2024-04-25T11:38:14.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.118+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v109'214 uv214 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.118+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55c81172e0e0 con 0x55c81172a2c0 2024-04-25T11:38:14.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.122+0000 7fac26600640 1 -- 172.21.15.19:0/2365310660 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v109'215 uv215 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fac10025040 con 0x55c81172a2c0 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c811719bf0 msgr2=0x55c81171cd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55c811719bf0 0x55c81171cd10 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c811726c90 msgr2=0x55c8117270c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55c811726c90 0x55c8117270c0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c81172a2c0 msgr2=0x55c81172c720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55c81172a2c0 0x55c81172c720 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c81171ee30 msgr2=0x55c811721250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55c81171ee30 0x55c811721250 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c81166e650 msgr2=0x55c81166ea60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c81166e650 0x55c81166ea60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac27a00640 1 -- 172.21.15.19:0/2365310660 reap_dead start 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c811723c60 msgr2=0x55c811726080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c811723c60 0x55c811726080 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.123+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac04072ec0 msgr2=0x7fac040753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac04072ec0 0x7fac040753d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fac10001df0 tx=0x7fac10006000 comp rx=0 tx=0).stop 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f1e0 msgr2=0x55c811673910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f1e0 0x55c811673910 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fac18000f80 tx=0x7fac18003330 comp rx=0 tx=0).stop 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 shutdown_connections 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55c811723c60 0x55c811726080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c81166e650 0x55c81166ea60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac04072ec0 0x7fac040753d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 --2- 172.21.15.19:0/2365310660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c81159f1e0 0x55c811673910 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 >> 172.21.15.19:0/2365310660 conn(0x55c8114c1ac0 msgr2=0x55c811673ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 shutdown_connections 2024-04-25T11:38:14.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.124+0000 7fac2de72f40 1 -- 172.21.15.19:0/2365310660 wait complete. 2024-04-25T11:38:14.149 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2024-04-25T11:38:14.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.398+0000 7f7d2a232f40 1 Processor -- start 2024-04-25T11:38:14.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.398+0000 7f7d2a232f40 1 -- start start 2024-04-25T11:38:14.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.399+0000 7f7d2a232f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d080030f0 0x5577862f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.399+0000 7f7d2a232f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577862f5360 0x5577862ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.399+0000 7f7d2a232f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x5577862fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.399+0000 7f7d2a232f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577861ced20 con 0x5577862fdf60 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.399+0000 7f7d2a232f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5577861d2d60 con 0x5577862f5360 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.399+0000 7f7d2a232f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x557786271110 con 0x7f7d080030f0 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.399+0000 7f7d23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x5577862fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x5577862fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52926/0 (socket says 172.21.15.19:52926) 2024-04-25T11:38:14.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 -- 172.21.15.19:0/1265973516 learned_addr learned my addr 172.21.15.19:0/1265973516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23400640 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d080030f0 0x5577862f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d22a00640 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577862f5360 0x5577862ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 -- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d080030f0 msgr2=0x5577862f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d080030f0 0x5577862f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 -- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577862f5360 msgr2=0x5577862ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577862f5360 0x5577862ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 -- 172.21.15.19:0/1265973516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557786272500 con 0x5577862fdf60 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.400+0000 7f7d23e00640 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x5577862fcf80 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x55778622e830 tx=0x7f7d1c00c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda928d9a2d0d0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d21600640 1 -- 172.21.15.19:0/1265973516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c010470 con 0x5577862fdf60 2024-04-25T11:38:14.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d21600640 1 -- 172.21.15.19:0/1265973516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1c010630 con 0x5577862fdf60 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d21600640 1 -- 172.21.15.19:0/1265973516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c010a60 con 0x5577862fdf60 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 -- 172.21.15.19:0/1265973516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 msgr2=0x5577862fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x5577862fcf80 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x55778622e830 tx=0x7f7d1c00c620 comp rx=0 tx=0).stop 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 -- 172.21.15.19:0/1265973516 shutdown_connections 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577862f5360 0x5577862ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x5577862fcf80 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/1265973516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d080030f0 0x5577862f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 -- 172.21.15.19:0/1265973516 >> 172.21.15.19:0/1265973516 conn(0x55778614aac0 msgr2=0x55778625a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.401+0000 7f7d2a232f40 1 -- 172.21.15.19:0/1265973516 shutdown_connections 2024-04-25T11:38:14.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.402+0000 7f7d2a232f40 1 -- 172.21.15.19:0/1265973516 wait complete. 2024-04-25T11:38:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.403+0000 7f7d2a232f40 1 Processor -- start 2024-04-25T11:38:14.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.403+0000 7f7d2a232f40 1 -- start start 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.404+0000 7f7d2a232f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577862f5360 0x557786399660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.404+0000 7f7d2a232f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x557786399ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.404+0000 7f7d2a232f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d080030f0 0x5577862f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.404+0000 7f7d2a232f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577862f4fd0 con 0x5577862fdf60 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.404+0000 7f7d2a232f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5577862fa2d0 con 0x7f7d080030f0 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.404+0000 7f7d2a232f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5577862f3dd0 con 0x5577862f5360 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.404+0000 7f7d22a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x557786399ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d22a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x557786399ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52940/0 (socket says 172.21.15.19:52940) 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d22a00640 1 -- 172.21.15.19:0/339993472 learned_addr learned my addr 172.21.15.19:0/339993472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d23400640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577862f5360 0x557786399660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d23e00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d080030f0 0x5577862f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d22a00640 1 -- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577862f5360 msgr2=0x557786399660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d22a00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577862f5360 0x557786399660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d22a00640 1 -- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d080030f0 msgr2=0x5577862f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d22a00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d080030f0 0x5577862f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d22a00640 1 -- 172.21.15.19:0/339993472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577862743d0 con 0x5577862fdf60 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d23400640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577862f5360 0x557786399660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:14.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.405+0000 7f7d23e00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d080030f0 0x5577862f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:14.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.406+0000 7f7d22a00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x557786399ba0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f7d100462e0 tx=0x7f7d10003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:14.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.406+0000 7f7d1be00640 1 -- 172.21.15.19:0/339993472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10066030 con 0x5577862fdf60 2024-04-25T11:38:14.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.406+0000 7f7d1be00640 1 -- 172.21.15.19:0/339993472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1000a100 con 0x5577862fdf60 2024-04-25T11:38:14.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.406+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55778626a3d0 con 0x5577862fdf60 2024-04-25T11:38:14.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.406+0000 7f7d1be00640 1 -- 172.21.15.19:0/339993472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000a2f0 con 0x5577862fdf60 2024-04-25T11:38:14.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.406+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778639a0e0 con 0x5577862fdf60 2024-04-25T11:38:14.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.408+0000 7f7d1be00640 1 -- 172.21.15.19:0/339993472 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d10069070 con 0x5577862fdf60 2024-04-25T11:38:14.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.409+0000 7f7d1be00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d04072e90 0x7f7d040753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.409+0000 7f7d1be00640 1 -- 172.21.15.19:0/339993472 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7f7d100d7db0 con 0x5577862fdf60 2024-04-25T11:38:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.410+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577862f4320 0x5577862f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.410+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x55778626be30 con 0x5577862f4320 2024-04-25T11:38:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.410+0000 7f7d23400640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d04072e90 0x7f7d040753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.410+0000 7f7d23e00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577862f4320 0x5577862f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.411+0000 7f7d23e00640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577862f4320 0x5577862f99e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:14.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.411+0000 7f7d23400640 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d04072e90 0x7f7d040753a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7d14001d90 tx=0x7f7d14002210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:14.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d23e00640 1 -- 172.21.15.19:0/339993472 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v109'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7d1c010860 con 0x5577862f4320 2024-04-25T11:38:14.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577862f4320 msgr2=0x5577862f99e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577862f4320 0x5577862f99e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d04072e90 msgr2=0x7f7d040753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d04072e90 0x7f7d040753a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7d14001d90 tx=0x7f7d14002210 comp rx=0 tx=0).stop 2024-04-25T11:38:14.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 msgr2=0x557786399ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 --2- 172.21.15.19:0/339993472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577862fdf60 0x557786399ba0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f7d100462e0 tx=0x7f7d10003020 comp rx=0 tx=0).stop 2024-04-25T11:38:14.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d23e00640 1 -- 172.21.15.19:0/339993472 reap_dead start 2024-04-25T11:38:14.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 shutdown_connections 2024-04-25T11:38:14.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 >> 172.21.15.19:0/339993472 conn(0x55778614aac0 msgr2=0x5577862fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:14.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 shutdown_connections 2024-04-25T11:38:14.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.415+0000 7f7d2a232f40 1 -- 172.21.15.19:0/339993472 wait complete. 2024-04-25T11:38:14.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:14.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.728+0000 7f2cb7a00640 1 Processor -- start 2024-04-25T11:38:14.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.728+0000 7f2cb7a00640 1 -- start start 2024-04-25T11:38:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.731+0000 7f2cb7a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2cb00cd9a0 0x7f2cb015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.731+0000 7f2cb7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb006dce0 0x7f2cb015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.731+0000 7f2cb7a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb00cdea0 con 0x7f2cb00b0af0 2024-04-25T11:38:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.731+0000 7f2cb7a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb00ceab0 con 0x7f2cb00cd9a0 2024-04-25T11:38:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.731+0000 7f2cb7a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00ce180 con 0x7f2cb006dce0 2024-04-25T11:38:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.731+0000 7f2cafe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2cb00b0af0 0x7f2cb006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57316/0 (socket says 172.21.15.19:57316) 2024-04-25T11:38:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.731+0000 7f2cafe00640 1 -- 172.21.15.19:0/3534284100 learned_addr learned my addr 172.21.15.19:0/3534284100 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2cb5000640 1 --2- 172.21.15.19:0/3534284100 >> v2:172.21.15.169:3300/0 conn(0x7f2cb00cd9a0 0x7f2cb015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2cb5a00640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb006dce0 0x7f2cb015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59240575 0 0) 0x7f2cb00cdea0 con 0x7f2cb00b0af0 2024-04-25T11:38:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c94003520 con 0x7f2cb00b0af0 2024-04-25T11:38:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1447713485 0 0) 0x7f2c94003520 con 0x7f2cb00b0af0 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb006dce0 msgr2=0x7f2cb015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb006dce0 0x7f2cb015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 >> v2:172.21.15.169:3300/0 conn(0x7f2cb00cd9a0 msgr2=0x7f2cb015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 --2- 172.21.15.19:0/3534284100 >> v2:172.21.15.169:3300/0 conn(0x7f2cb00cd9a0 0x7f2cb015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb00ce460 con 0x7f2cb00b0af0 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2c98001670 con 0x7f2cb00b0af0 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 >> v1:172.21.15.19:6789/0 conn(0x7f2cb00b0af0 legacy=0x7f2cb006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.732+0000 7f2caf400640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c940039b0 0x7f2c94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2caf400640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c94004800 0x7f2c94004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2caf400640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c94005150 0x7f2c9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00ceab0 con 0x7f2c94004800 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00ce180 con 0x7f2c940039b0 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9400bf00 con 0x7f2c94005150 2024-04-25T11:38:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2cafe00640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c94004800 0x7f2c94004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2cb5000640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c940039b0 0x7f2c94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2cb5a00640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c94005150 0x7f2c9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2cafe00640 1 -- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c94005150 msgr2=0x7f2c9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2cafe00640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2c94005150 0x7f2c9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2cafe00640 1 -- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c940039b0 msgr2=0x7f2c94003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.733+0000 7f2cafe00640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c940039b0 0x7f2c94003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.734+0000 7f2cafe00640 1 -- 172.21.15.19:0/3534284100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c94003520 con 0x7f2c94004800 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.735+0000 7f2cafe00640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c94004800 0x7f2c94004c10 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c160 tx=0x7f2c94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407d345635fa6b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.735+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98007c80 con 0x7f2c94004800 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.735+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c98005ad0 con 0x7f2c94004800 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.735+0000 7f2caf400640 1 -- 172.21.15.19:0/3534284100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98007070 con 0x7f2c94004800 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.736+0000 7f2cb7a00640 1 -- 172.21.15.19:0/3534284100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c94004800 msgr2=0x7f2c94004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.736+0000 7f2cb7a00640 1 --2- 172.21.15.19:0/3534284100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c94004800 0x7f2c94004c10 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c160 tx=0x7f2c94005830 comp rx=0 tx=0).stop 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.736+0000 7f2cb5a00640 1 -- 172.21.15.19:0/3534284100 reap_dead start 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.736+0000 7f2cb7a00640 1 -- 172.21.15.19:0/3534284100 shutdown_connections 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.736+0000 7f2cb7a00640 1 -- 172.21.15.19:0/3534284100 >> 172.21.15.19:0/3534284100 conn(0x7f2cb0073090 msgr2=0x7f2cb00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.736+0000 7f2cb7a00640 1 -- 172.21.15.19:0/3534284100 shutdown_connections 2024-04-25T11:38:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.736+0000 7f2cb7a00640 1 -- 172.21.15.19:0/3534284100 wait complete. 2024-04-25T11:38:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.738+0000 7f2cb7a00640 1 Processor -- start 2024-04-25T11:38:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.739+0000 7f2cb7a00640 1 -- start start 2024-04-25T11:38:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb7a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c94004800 0x7f2cb013f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb7a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb0154900 0x7f2cb0145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb7a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cb0154d10 0x7f2cb01462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb7a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00cef40 con 0x7f2cb0154d10 2024-04-25T11:38:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb7a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00b0f00 con 0x7f2c94004800 2024-04-25T11:38:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb7a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00cf0c0 con 0x7f2cb0154900 2024-04-25T11:38:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cb0154d10 0x7f2cb01462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cb0154d10 0x7f2cb01462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52968/0 (socket says 172.21.15.19:52968) 2024-04-25T11:38:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.740+0000 7f2cb5a00640 1 -- 172.21.15.19:0/798415661 learned_addr learned my addr 172.21.15.19:0/798415661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5000640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c94004800 0x7f2cb013f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cafe00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb0154900 0x7f2cb0145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5a00640 1 -- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb0154900 msgr2=0x7f2cb0145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb0154900 0x7f2cb0145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5a00640 1 -- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c94004800 msgr2=0x7f2cb013f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c94004800 0x7f2cb013f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5a00640 1 -- 172.21.15.19:0/798415661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb013f650 con 0x7f2cb0154d10 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5000640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c94004800 0x7f2cb013f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.741+0000 7f2cb5a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cb0154d10 0x7f2cb01462c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00cddd0 tx=0x7f2ca80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.742+0000 7f2cacc00640 1 -- 172.21.15.19:0/798415661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8011e40 con 0x7f2cb0154d10 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.742+0000 7f2cacc00640 1 -- 172.21.15.19:0/798415661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca801b070 con 0x7f2cb0154d10 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.742+0000 7f2cacc00640 1 -- 172.21.15.19:0/798415661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8003990 con 0x7f2cb0154d10 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.742+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb0140e00 con 0x7f2cb0154d10 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.742+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb0143c20 con 0x7f2cb0154d10 2024-04-25T11:38:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.743+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb00740d0 con 0x7f2cb0154d10 2024-04-25T11:38:14.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.744+0000 7f2cacc00640 1 -- 172.21.15.19:0/798415661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ca8005d10 con 0x7f2cb0154d10 2024-04-25T11:38:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.745+0000 7f2cacc00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2c8c0744b0 0x7f2c8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.746+0000 7f2cacc00640 1 -- 172.21.15.19:0/798415661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7171+0+0 (secure 0 0 0) 0x7f2ca8095bf0 con 0x7f2cb0154d10 2024-04-25T11:38:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.746+0000 7f2cb5000640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2c8c0744b0 0x7f2c8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:14.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.747+0000 7f2cb5000640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2c8c0744b0 0x7f2c8c0769c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009a00 tx=0x7f2ca0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:14.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.751+0000 7f2cacc00640 1 -- 172.21.15.19:0/798415661 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2cb013f650 con 0x7f2cb0154d10 2024-04-25T11:38:14.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:14.996+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2cb0140b90 con 0x7f2cb0154d10 2024-04-25T11:38:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.350+0000 7f2cacc00640 1 -- 172.21.15.19:0/798415661 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v202) ==== 162+0+0 (secure 0 0 0) 0x7f2ca805fdb0 con 0x7f2cb0154d10 2024-04-25T11:38:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2c8c0744b0 msgr2=0x7f2c8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2c8c0744b0 0x7f2c8c0769c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009a00 tx=0x7f2ca0008040 comp rx=0 tx=0).stop 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cb0154d10 msgr2=0x7f2cb01462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cb0154d10 0x7f2cb01462c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00cddd0 tx=0x7f2ca80115a0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 shutdown_connections 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2c8c0744b0 0x7f2c8c0769c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cb0154d10 0x7f2cb01462c0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2cb0154900 0x7f2cb0145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 --2- 172.21.15.19:0/798415661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c94004800 0x7f2cb013f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.354+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 >> 172.21.15.19:0/798415661 conn(0x7f2cb0073090 msgr2=0x7f2cb006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.355+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 shutdown_connections 2024-04-25T11:38:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.355+0000 7f2cb7a00640 1 -- 172.21.15.19:0/798415661 wait complete. 2024-04-25T11:38:15.365 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:15.377+0000 7f55ae000640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-04-25T11:38:15.365 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:15.378+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:15.366 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:15.378+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:38:15.374 INFO:tasks.ceph.mds.a.smithi019.stderr: -16> 2024-04-25T11:38:15.377+0000 7f55ae000640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-04-25T11:38:15.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:15.506 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.653+0000 7f8ae9000640 1 Processor -- start 2024-04-25T11:38:15.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.655+0000 7f8ae9000640 1 -- start start 2024-04-25T11:38:15.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae9000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ae415c800 0x7f8ae406f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ae40cfa00 0x7f8ae4167180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae9000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae4071230 con 0x7f8ae4070e20 2024-04-25T11:38:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae9000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae406f6c0 con 0x7f8ae40cfa00 2024-04-25T11:38:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae9000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae406f8f0 con 0x7f8ae415c800 2024-04-25T11:38:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ae40cfa00 0x7f8ae4167180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ae40cfa00 0x7f8ae4167180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52990/0 (socket says 172.21.15.19:52990) 2024-04-25T11:38:15.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae3e00640 1 -- 172.21.15.19:0/2566976292 learned_addr learned my addr 172.21.15.19:0/2566976292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:15.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae3e00640 1 -- 172.21.15.19:0/2566976292 >> v2:172.21.15.169:3300/0 conn(0x7f8ae415c800 msgr2=0x7f8ae406f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:15.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae3e00640 1 --2- 172.21.15.19:0/2566976292 >> v2:172.21.15.169:3300/0 conn(0x7f8ae415c800 0x7f8ae406f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae3e00640 1 -- 172.21.15.19:0/2566976292 >> v1:172.21.15.169:6789/0 conn(0x7f8ae4070e20 legacy=0x7f8ae40cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:15.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.657+0000 7f8ae3e00640 1 -- 172.21.15.19:0/2566976292 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae406fbd0 con 0x7f8ae40cfa00 2024-04-25T11:38:15.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.658+0000 7f8ae3e00640 1 --2- 172.21.15.19:0/2566976292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae40cfa00 0x7f8ae4167180 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40d2790 tx=0x7f8adc02c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d01f721e408353c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:15.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.659+0000 7f8ae2000640 1 -- 172.21.15.19:0/2566976292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00b2d0 con 0x7f8ae40cfa00 2024-04-25T11:38:15.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.659+0000 7f8ae2000640 1 -- 172.21.15.19:0/2566976292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8adc00b470 con 0x7f8ae40cfa00 2024-04-25T11:38:15.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.659+0000 7f8ae2000640 1 -- 172.21.15.19:0/2566976292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc009820 con 0x7f8ae40cfa00 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.659+0000 7f8ae9000640 1 -- 172.21.15.19:0/2566976292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae40cfa00 msgr2=0x7f8ae4167180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.659+0000 7f8ae9000640 1 --2- 172.21.15.19:0/2566976292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae40cfa00 0x7f8ae4167180 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40d2790 tx=0x7f8adc02c8f0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.660+0000 7f8ae9000640 1 -- 172.21.15.19:0/2566976292 shutdown_connections 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.660+0000 7f8ae9000640 1 --2- 172.21.15.19:0/2566976292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae40cfa00 0x7f8ae4167180 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.660+0000 7f8ae9000640 1 --2- 172.21.15.19:0/2566976292 >> v2:172.21.15.169:3300/0 conn(0x7f8ae415c800 0x7f8ae406f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.660+0000 7f8ae9000640 1 -- 172.21.15.19:0/2566976292 >> 172.21.15.19:0/2566976292 conn(0x7f8ae40d1720 msgr2=0x7f8ae40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.660+0000 7f8ae9000640 1 -- 172.21.15.19:0/2566976292 shutdown_connections 2024-04-25T11:38:15.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.660+0000 7f8ae9000640 1 -- 172.21.15.19:0/2566976292 wait complete. 2024-04-25T11:38:15.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.664+0000 7f8ae9000640 1 Processor -- start 2024-04-25T11:38:15.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.664+0000 7f8ae9000640 1 -- start start 2024-04-25T11:38:15.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae4070e20 0x7f8ae414fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae40cfa00 0x7f8ae41e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae415c800 0x7f8ae40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae9000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae415cc10 con 0x7f8ae4070e20 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4070190 con 0x7f8ae40cfa00 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae406d5d0 con 0x7f8ae415c800 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae4070e20 0x7f8ae414fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae4070e20 0x7f8ae414fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52994/0 (socket says 172.21.15.19:52994) 2024-04-25T11:38:15.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 -- 172.21.15.19:0/1920783337 learned_addr learned my addr 172.21.15.19:0/1920783337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 -- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae415c800 msgr2=0x7f8ae40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae415c800 0x7f8ae40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 -- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae40cfa00 msgr2=0x7f8ae41e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae2a00640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae40cfa00 0x7f8ae41e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae40cfa00 0x7f8ae41e9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae3400640 1 -- 172.21.15.19:0/1920783337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae415cf30 con 0x7f8ae4070e20 2024-04-25T11:38:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.666+0000 7f8ae2a00640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae40cfa00 0x7f8ae41e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.667+0000 7f8ae3400640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae4070e20 0x7f8ae414fd50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400b3e0 tx=0x7f8ad400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.667+0000 7f8adb400640 1 -- 172.21.15.19:0/1920783337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4015020 con 0x7f8ae4070e20 2024-04-25T11:38:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.667+0000 7f8adb400640 1 -- 172.21.15.19:0/1920783337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad4011850 con 0x7f8ae4070e20 2024-04-25T11:38:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.667+0000 7f8adb400640 1 -- 172.21.15.19:0/1920783337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4011c40 con 0x7f8ae4070e20 2024-04-25T11:38:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.667+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae415d210 con 0x7f8ae4070e20 2024-04-25T11:38:15.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.667+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae41e50c0 con 0x7f8ae4070e20 2024-04-25T11:38:15.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.669+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae4050740 con 0x7f8ae4070e20 2024-04-25T11:38:15.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.670+0000 7f8adb400640 1 -- 172.21.15.19:0/1920783337 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8ad4018020 con 0x7f8ae4070e20 2024-04-25T11:38:15.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.672+0000 7f8adb400640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac4074500 0x7f8ac4076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:15.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.672+0000 7f8adb400640 1 -- 172.21.15.19:0/1920783337 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f8ad40957f0 con 0x7f8ae4070e20 2024-04-25T11:38:15.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.673+0000 7f8ae2a00640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac4074500 0x7f8ac4076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:15.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.673+0000 7f8ae2a00640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac4074500 0x7f8ac4076a10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8ae41e7380 tx=0x7f8ad0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:15.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.676+0000 7f8adb400640 1 -- 172.21.15.19:0/1920783337 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ad401f020 con 0x7f8ae4070e20 2024-04-25T11:38:15.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.923+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ae40738f0 con 0x7f8ae4070e20 2024-04-25T11:38:15.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.924+0000 7f8adb400640 1 -- 172.21.15.19:0/1920783337 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) ==== 95+0+4352 (secure 0 0 0) 0x7f8ad405f950 con 0x7f8ae4070e20 2024-04-25T11:38:15.913 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:15.913 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":204,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:15.379389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:15.913 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 204 2024-04-25T11:38:15.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac4074500 msgr2=0x7f8ac4076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac4074500 0x7f8ac4076a10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8ae41e7380 tx=0x7f8ad0008040 comp rx=0 tx=0).stop 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae4070e20 msgr2=0x7f8ae414fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae4070e20 0x7f8ae414fd50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400b3e0 tx=0x7f8ad400b8a0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 shutdown_connections 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac4074500 0x7f8ac4076a10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae40cfa00 0x7f8ae41e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae415c800 0x7f8ae40afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 --2- 172.21.15.19:0/1920783337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae4070e20 0x7f8ae414fd50 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 >> 172.21.15.19:0/1920783337 conn(0x7f8ae40d1720 msgr2=0x7f8ae40b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 shutdown_connections 2024-04-25T11:38:15.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:15.927+0000 7f8ae9000640 1 -- 172.21.15.19:0/1920783337 wait complete. 2024-04-25T11:38:15.943 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:15.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.239+0000 7f4a73200640 1 Processor -- start 2024-04-25T11:38:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.239+0000 7f4a73200640 1 -- start start 2024-04-25T11:38:16.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.241+0000 7f4a73200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4a740cf2f0 0x7f4a740b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.241+0000 7f4a73200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a740b74d0 con 0x7f4a7415a810 2024-04-25T11:38:16.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.241+0000 7f4a73200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a740b77b0 con 0x7f4a741608d0 2024-04-25T11:38:16.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.241+0000 7f4a73200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7406f870 con 0x7f4a740cf2f0 2024-04-25T11:38:16.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.242+0000 7f4a71400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4a7415a810 0x7f4a740b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57324/0 (socket says 172.21.15.19:57324) 2024-04-25T11:38:16.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.242+0000 7f4a71400640 1 -- 172.21.15.19:0/817579687 learned_addr learned my addr 172.21.15.19:0/817579687 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:16.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.242+0000 7f4a71e00640 1 --2- 172.21.15.19:0/817579687 >> v2:172.21.15.169:3300/0 conn(0x7f4a740cf2f0 0x7f4a740b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.242+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792405955 0 0) 0x7f4a740b74d0 con 0x7f4a7415a810 2024-04-25T11:38:16.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a58003520 con 0x7f4a7415a810 2024-04-25T11:38:16.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 695303122 0 0) 0x7f4a58003520 con 0x7f4a7415a810 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 >> v2:172.21.15.169:3300/0 conn(0x7f4a740cf2f0 msgr2=0x7f4a740b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 --2- 172.21.15.19:0/817579687 >> v2:172.21.15.169:3300/0 conn(0x7f4a740cf2f0 0x7f4a740b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 >> v1:172.21.15.169:6789/0 conn(0x7f4a741608d0 legacy=0x7f4a74160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7406f220 con 0x7f4a7415a810 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4a5c0019f0 con 0x7f4a7415a810 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 >> v1:172.21.15.19:6789/0 conn(0x7f4a7415a810 legacy=0x7f4a740b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a58003a00 0x7f4a58003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a58004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a58005220 0x7f4a5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7406f870 con 0x7f4a58004800 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5800c1b0 con 0x7f4a58005220 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.243+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5800c3b0 con 0x7f4a58003a00 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a71400640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a58004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a71400640 1 -- 172.21.15.19:0/817579687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a58003a00 msgr2=0x7f4a58003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a71e00640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a58003a00 0x7f4a58003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a71400640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a58003a00 0x7f4a58003e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a71400640 1 -- 172.21.15.19:0/817579687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a58005220 msgr2=0x7f4a5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a71400640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a58005220 0x7f4a5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a71400640 1 -- 172.21.15.19:0/817579687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a58003520 con 0x7f4a58004800 2024-04-25T11:38:16.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.244+0000 7f4a72800640 1 -- 172.21.15.19:0/817579687 reap_dead start 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.245+0000 7f4a71400640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a58004c10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00a770 tx=0x7f4a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3939f7cd34c8b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.245+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00a390 con 0x7f4a58004800 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.245+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5c02f070 con 0x7f4a58004800 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.245+0000 7f4a70a00640 1 -- 172.21.15.19:0/817579687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c02bc20 con 0x7f4a58004800 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.246+0000 7f4a73200640 1 -- 172.21.15.19:0/817579687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 msgr2=0x7f4a58004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.246+0000 7f4a73200640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a58004c10 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00a770 tx=0x7f4a58005830 comp rx=0 tx=0).stop 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.246+0000 7f4a73200640 1 -- 172.21.15.19:0/817579687 shutdown_connections 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.246+0000 7f4a73200640 1 --2- 172.21.15.19:0/817579687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a58004c10 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.246+0000 7f4a73200640 1 -- 172.21.15.19:0/817579687 >> 172.21.15.19:0/817579687 conn(0x7f4a740d1720 msgr2=0x7f4a740d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.246+0000 7f4a73200640 1 -- 172.21.15.19:0/817579687 shutdown_connections 2024-04-25T11:38:16.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.246+0000 7f4a73200640 1 -- 172.21.15.19:0/817579687 wait complete. 2024-04-25T11:38:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.249+0000 7f4a73200640 1 Processor -- start 2024-04-25T11:38:16.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.249+0000 7f4a73200640 1 -- start start 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a73200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a741f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a73200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a741e66f0 0x7f4a740aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a73200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74146fa0 0x7f4a7415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a73200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a740b6e10 con 0x7f4a58004800 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a73200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7415ac20 con 0x7f4a74146fa0 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a73200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4a740b6f90 con 0x7f4a741e66f0 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a741f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a741f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53012/0 (socket says 172.21.15.19:53012) 2024-04-25T11:38:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 -- 172.21.15.19:0/4279677716 learned_addr learned my addr 172.21.15.19:0/4279677716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a72800640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74146fa0 0x7f4a7415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71400640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a741e66f0 0x7f4a740aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 -- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a741e66f0 msgr2=0x7f4a740aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a741e66f0 0x7f4a740aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 -- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74146fa0 msgr2=0x7f4a7415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74146fa0 0x7f4a7415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.251+0000 7f4a71e00640 1 -- 172.21.15.19:0/4279677716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a741fa6a0 con 0x7f4a58004800 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.252+0000 7f4a72800640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74146fa0 0x7f4a7415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.252+0000 7f4a71e00640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a741f9790 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4a64010790 tx=0x7f4a64010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.252+0000 7f4a6a000640 1 -- 172.21.15.19:0/4279677716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400f040 con 0x7f4a58004800 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.252+0000 7f4a6a000640 1 -- 172.21.15.19:0/4279677716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6400eb60 con 0x7f4a58004800 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.252+0000 7f4a6a000640 1 -- 172.21.15.19:0/4279677716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64010ea0 con 0x7f4a58004800 2024-04-25T11:38:16.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.252+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a741fa0e0 con 0x7f4a58004800 2024-04-25T11:38:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.252+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a740ae6e0 con 0x7f4a58004800 2024-04-25T11:38:16.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.254+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a74050740 con 0x7f4a58004800 2024-04-25T11:38:16.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.256+0000 7f4a6a000640 1 -- 172.21.15.19:0/4279677716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a6401c400 con 0x7f4a58004800 2024-04-25T11:38:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.257+0000 7f4a6a000640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a500744b0 0x7f4a500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.257+0000 7f4a6a000640 1 -- 172.21.15.19:0/4279677716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f4a64095f40 con 0x7f4a58004800 2024-04-25T11:38:16.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.257+0000 7f4a71400640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a500744b0 0x7f4a500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.259+0000 7f4a71400640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a500744b0 0x7f4a500769c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c033000 tx=0x7f4a5c038040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.262+0000 7f4a6a000640 1 -- 172.21.15.19:0/4279677716 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a64018030 con 0x7f4a58004800 2024-04-25T11:38:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.510+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a7406c770 con 0x7f4a58004800 2024-04-25T11:38:16.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.511+0000 7f4a6a000640 1 -- 172.21.15.19:0/4279677716 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) ==== 95+0+5175 (secure 0 0 0) 0x7f4a64060020 con 0x7f4a58004800 2024-04-25T11:38:16.499 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:16.499 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":205,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/614168622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":614168622},{"type":"v1","addr":"172.21.15.19:6835","nonce":614168622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":205}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:15.379389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 205 2024-04-25T11:38:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a500744b0 msgr2=0x7f4a500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a500744b0 0x7f4a500769c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c033000 tx=0x7f4a5c038040 comp rx=0 tx=0).stop 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 msgr2=0x7f4a741f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a741f9790 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4a64010790 tx=0x7f4a64010c50 comp rx=0 tx=0).stop 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 shutdown_connections 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a500744b0 0x7f4a500769c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a74146fa0 0x7f4a7415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a741e66f0 0x7f4a740aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 --2- 172.21.15.19:0/4279677716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a58004800 0x7f4a741f9790 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 >> 172.21.15.19:0/4279677716 conn(0x7f4a740d1720 msgr2=0x7f4a740d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 shutdown_connections 2024-04-25T11:38:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.513+0000 7f4a73200640 1 -- 172.21.15.19:0/4279677716 wait complete. 2024-04-25T11:38:16.529 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:16.529 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:16.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.815+0000 7f1201e00640 1 Processor -- start 2024-04-25T11:38:16.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.815+0000 7f1201e00640 1 -- start start 2024-04-25T11:38:16.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.817+0000 7f1201e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f11fc0d0b70 0x7f11fc0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.817+0000 7f1201e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11fc0d14c0 con 0x7f11fc0b9180 2024-04-25T11:38:16.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.817+0000 7f1201e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11fc160780 con 0x7f11fc160370 2024-04-25T11:38:16.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.817+0000 7f1201e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f11fc164840 con 0x7f11fc0d0b70 2024-04-25T11:38:16.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.817+0000 7f1201400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f11fc0b9180 0x7f11fc0b9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57326/0 (socket says 172.21.15.19:57326) 2024-04-25T11:38:16.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.817+0000 7f1201400640 1 -- 172.21.15.19:0/3152787183 learned_addr learned my addr 172.21.15.19:0/3152787183 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fbe00640 1 --2- 172.21.15.19:0/3152787183 >> v2:172.21.15.169:3300/0 conn(0x7f11fc0d0b70 0x7f11fc0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862288451 0 0) 0x7f11f4001490 con 0x7f11fc0b9180 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11e0003520 con 0x7f11fc0b9180 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 373202513 0 0) 0x7f11e0003520 con 0x7f11fc0b9180 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 >> v2:172.21.15.169:3300/0 conn(0x7f11fc0d0b70 msgr2=0x7f11fc0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 --2- 172.21.15.19:0/3152787183 >> v2:172.21.15.169:3300/0 conn(0x7f11fc0d0b70 0x7f11fc0d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 >> v1:172.21.15.169:6789/0 conn(0x7f11fc160370 legacy=0x7f11fc0d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc0d3420 con 0x7f11fc0b9180 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f11f4001bc0 con 0x7f11fc0b9180 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 >> v1:172.21.15.19:6789/0 conn(0x7f11fc0b9180 legacy=0x7f11fc0b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e00039b0 0x7f11e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11e0004800 0x7f11e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00051c0 0x7f11e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.818+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc164840 con 0x7f11e00039b0 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11e000c160 con 0x7f11e0004800 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11e000c360 con 0x7f11e00051c0 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f1200a00640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e00039b0 0x7f11e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f1200a00640 1 -- 172.21.15.19:0/3152787183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00051c0 msgr2=0x7f11e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f1200a00640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00051c0 0x7f11e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f1200a00640 1 -- 172.21.15.19:0/3152787183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11e0004800 msgr2=0x7f11e0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f1200a00640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11e0004800 0x7f11e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f1200a00640 1 -- 172.21.15.19:0/3152787183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e0003520 con 0x7f11e00039b0 2024-04-25T11:38:16.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.819+0000 7f1201400640 1 -- 172.21.15.19:0/3152787183 reap_dead start 2024-04-25T11:38:16.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.820+0000 7f1200a00640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e00039b0 0x7f11e0003e20 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f11ec001750 tx=0x7f11e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869d6cf978490c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec001370 con 0x7f11e00039b0 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec0052d0 con 0x7f11e00039b0 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f11fb400640 1 -- 172.21.15.19:0/3152787183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec02ab00 con 0x7f11e00039b0 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f1201e00640 1 -- 172.21.15.19:0/3152787183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e00039b0 msgr2=0x7f11e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f1201e00640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e00039b0 0x7f11e0003e20 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f11ec001750 tx=0x7f11e0005830 comp rx=0 tx=0).stop 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f1201e00640 1 -- 172.21.15.19:0/3152787183 shutdown_connections 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f1201e00640 1 --2- 172.21.15.19:0/3152787183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11e00039b0 0x7f11e0003e20 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.821+0000 7f1201e00640 1 -- 172.21.15.19:0/3152787183 >> 172.21.15.19:0/3152787183 conn(0x7f11fc06dfe0 msgr2=0x7f11fc06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.822+0000 7f1201e00640 1 -- 172.21.15.19:0/3152787183 shutdown_connections 2024-04-25T11:38:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.822+0000 7f1201e00640 1 -- 172.21.15.19:0/3152787183 wait complete. 2024-04-25T11:38:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.824+0000 7f1201e00640 1 Processor -- start 2024-04-25T11:38:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.824+0000 7f1201e00640 1 -- start start 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00039b0 0x7f11fc15f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc1f75b0 0x7f11fc15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc15d6d0 0x7f11fc1ec090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1644d0 con 0x7f11fc15d6d0 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1641f0 con 0x7f11fc1f75b0 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc164cd0 con 0x7f11e00039b0 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc15d6d0 0x7f11fc1ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc15d6d0 0x7f11fc1ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53032/0 (socket says 172.21.15.19:53032) 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201400640 1 -- 172.21.15.19:0/2005425311 learned_addr learned my addr 172.21.15.19:0/2005425311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f11fbe00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc1f75b0 0x7f11fc15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1200a00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00039b0 0x7f11fc15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201400640 1 -- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00039b0 msgr2=0x7f11fc15f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201400640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00039b0 0x7f11fc15f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201400640 1 -- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc1f75b0 msgr2=0x7f11fc15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.826+0000 7f1201400640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc1f75b0 0x7f11fc15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.827+0000 7f1201400640 1 -- 172.21.15.19:0/2005425311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc15cde0 con 0x7f11fc15d6d0 2024-04-25T11:38:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.827+0000 7f1201400640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc15d6d0 0x7f11fc1ec090 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f11fc08e2f0 tx=0x7f11f400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.827+0000 7f11f8c00640 1 -- 172.21.15.19:0/2005425311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f400a610 con 0x7f11fc15d6d0 2024-04-25T11:38:16.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.827+0000 7f11f8c00640 1 -- 172.21.15.19:0/2005425311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f400a7b0 con 0x7f11fc15d6d0 2024-04-25T11:38:16.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.827+0000 7f11f8c00640 1 -- 172.21.15.19:0/2005425311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f400aba0 con 0x7f11fc15d6d0 2024-04-25T11:38:16.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.827+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc15d3a0 con 0x7f11fc15d6d0 2024-04-25T11:38:16.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.827+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc1eb890 con 0x7f11fc15d6d0 2024-04-25T11:38:16.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.828+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11fc06e920 con 0x7f11fc15d6d0 2024-04-25T11:38:16.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.830+0000 7f11f8c00640 1 -- 172.21.15.19:0/2005425311 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f11f400ad40 con 0x7f11fc15d6d0 2024-04-25T11:38:16.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.831+0000 7f11f8c00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11d80744e0 0x7f11d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.831+0000 7f11f8c00640 1 -- 172.21.15.19:0/2005425311 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f11f4094c50 con 0x7f11fc15d6d0 2024-04-25T11:38:16.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.832+0000 7f1200a00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11d80744e0 0x7f11d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.833+0000 7f1200a00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11d80744e0 0x7f11d80769f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f11ec032000 tx=0x7f11ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:16.836+0000 7f11f8c00640 1 -- 172.21.15.19:0/2005425311 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11fc06e920 con 0x7f11fc15d6d0 2024-04-25T11:38:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.081+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11fc15fc60 con 0x7f11fc15d6d0 2024-04-25T11:38:17.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.083+0000 7f11f8c00640 1 -- 172.21.15.19:0/2005425311 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) ==== 95+0+5175 (secure 0 0 0) 0x7f11fc15fc60 con 0x7f11fc15d6d0 2024-04-25T11:38:17.070 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:17.070 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":205,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9300,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/614168622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":614168622},{"type":"v1","addr":"172.21.15.19:6835","nonce":614168622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":205}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:15.379389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:17.070 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 205 2024-04-25T11:38:17.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11d80744e0 msgr2=0x7f11d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11d80744e0 0x7f11d80769f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f11ec032000 tx=0x7f11ec02e000 comp rx=0 tx=0).stop 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc15d6d0 msgr2=0x7f11fc1ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc15d6d0 0x7f11fc1ec090 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f11fc08e2f0 tx=0x7f11f400ccd0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 shutdown_connections 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f11d80744e0 0x7f11d80769f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11fc1f75b0 0x7f11fc15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11fc15d6d0 0x7f11fc1ec090 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.085+0000 7f1201e00640 1 --2- 172.21.15.19:0/2005425311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11e00039b0 0x7f11fc15f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.086+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 >> 172.21.15.19:0/2005425311 conn(0x7f11fc06dfe0 msgr2=0x7f11fc0ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.086+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 shutdown_connections 2024-04-25T11:38:17.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.086+0000 7f1201e00640 1 -- 172.21.15.19:0/2005425311 wait complete. 2024-04-25T11:38:17.097 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2024-04-25T11:38:17.098 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2024-04-25T11:38:17.098 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:17.098 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:17.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:38:17.149 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:17.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:17.149 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:17.201 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:38:17.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:17.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:17.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.507+0000 7fba71200640 1 Processor -- start 2024-04-25T11:38:17.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.507+0000 7fba71200640 1 -- start start 2024-04-25T11:38:17.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.509+0000 7fba71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.509+0000 7fba71200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fba6c0d4a50 0x7fba6c0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.510+0000 7fba71200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba6c0d1850 con 0x7fba6c06c5c0 2024-04-25T11:38:17.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.510+0000 7fba71200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fba6c160de0 con 0x7fba6c0d4a50 2024-04-25T11:38:17.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.510+0000 7fba71200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba6c1610c0 con 0x7fba6c0d0e00 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.510+0000 7fba6a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fba6c0d4a50 0x7fba6c0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.510+0000 7fba69e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.510+0000 7fba61400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fba6c06c5c0 0x7fba6c06c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:56470/0 (socket says 172.21.15.19:56470) 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.511+0000 7fba6a800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fba6c0d4a50 0x7fba6c0d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:54356/0 (socket says 172.21.15.19:54356) 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.511+0000 7fba6a800640 1 -- 172.21.15.19:0/2545390997 learned_addr learned my addr 172.21.15.19:0/2545390997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.511+0000 7fba6a800640 1 -- 172.21.15.19:0/2545390997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 msgr2=0x7fba6c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.511+0000 7fba6a800640 1 --2- 172.21.15.19:0/2545390997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.511+0000 7fba6a800640 1 -- 172.21.15.19:0/2545390997 >> v1:172.21.15.169:6789/0 conn(0x7fba6c06c5c0 legacy=0x7fba6c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.511+0000 7fba6a800640 1 -- 172.21.15.19:0/2545390997 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6c1613a0 con 0x7fba6c0d4a50 2024-04-25T11:38:17.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.513+0000 7fba6a800640 1 --2- 172.21.15.19:0/2545390997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c0d4e60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fba64009870 tx=0x7fba6402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9629c7b6c982fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.513+0000 7fba69400640 1 -- 172.21.15.19:0/2545390997 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64037070 con 0x7fba6c0d4a50 2024-04-25T11:38:17.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.513+0000 7fba69400640 1 -- 172.21.15.19:0/2545390997 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba64032440 con 0x7fba6c0d4a50 2024-04-25T11:38:17.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.513+0000 7fba69400640 1 -- 172.21.15.19:0/2545390997 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64032810 con 0x7fba6c0d4a50 2024-04-25T11:38:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 -- 172.21.15.19:0/2545390997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 msgr2=0x7fba6c0d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 --2- 172.21.15.19:0/2545390997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c0d4e60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fba64009870 tx=0x7fba6402a250 comp rx=0 tx=0).stop 2024-04-25T11:38:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 -- 172.21.15.19:0/2545390997 shutdown_connections 2024-04-25T11:38:17.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 --2- 172.21.15.19:0/2545390997 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c0d4e60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 --2- 172.21.15.19:0/2545390997 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 -- 172.21.15.19:0/2545390997 >> 172.21.15.19:0/2545390997 conn(0x7fba6c0d5d90 msgr2=0x7fba6c0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 -- 172.21.15.19:0/2545390997 shutdown_connections 2024-04-25T11:38:17.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.514+0000 7fba71200640 1 -- 172.21.15.19:0/2545390997 wait complete. 2024-04-25T11:38:17.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.517+0000 7fba71200640 1 Processor -- start 2024-04-25T11:38:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.517+0000 7fba71200640 1 -- start start 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.518+0000 7fba71200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba6c06c5c0 0x7fba6c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.518+0000 7fba71200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c1fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.518+0000 7fba71200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c1fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.518+0000 7fba71200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c164e00 con 0x7fba6c06c5c0 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.518+0000 7fba71200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba6c0b5320 con 0x7fba6c0d4a50 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.518+0000 7fba71200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba6c164f80 con 0x7fba6c0d0e00 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.518+0000 7fba69e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba6c06c5c0 0x7fba6c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba69e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba6c06c5c0 0x7fba6c15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53046/0 (socket says 172.21.15.19:53046) 2024-04-25T11:38:17.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba69e00640 1 -- 172.21.15.19:0/315155665 learned_addr learned my addr 172.21.15.19:0/315155665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:17.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba6a800640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c1fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba61400640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c1fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba69e00640 1 -- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 msgr2=0x7fba6c1fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba69e00640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c1fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba69e00640 1 -- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 msgr2=0x7fba6c1fa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba69e00640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c1fa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba69e00640 1 -- 172.21.15.19:0/315155665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6c15bf90 con 0x7fba6c06c5c0 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba61400640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c1fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.519+0000 7fba6a800640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c1fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.520+0000 7fba69e00640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba6c06c5c0 0x7fba6c15b3f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009d10 tx=0x7fba5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.520+0000 7fba62a00640 1 -- 172.21.15.19:0/315155665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c018020 con 0x7fba6c06c5c0 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.520+0000 7fba62a00640 1 -- 172.21.15.19:0/315155665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c014480 con 0x7fba6c06c5c0 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.520+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba6c15c270 con 0x7fba6c06c5c0 2024-04-25T11:38:17.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.520+0000 7fba62a00640 1 -- 172.21.15.19:0/315155665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c014830 con 0x7fba6c06c5c0 2024-04-25T11:38:17.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.520+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba6c157f60 con 0x7fba6c06c5c0 2024-04-25T11:38:17.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.521+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba6c068c00 con 0x7fba6c06c5c0 2024-04-25T11:38:17.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.522+0000 7fba62a00640 1 -- 172.21.15.19:0/315155665 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fba5c005d10 con 0x7fba6c06c5c0 2024-04-25T11:38:17.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.523+0000 7fba62a00640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fba480744e0 0x7fba480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.523+0000 7fba62a00640 1 -- 172.21.15.19:0/315155665 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7fba5c0963d0 con 0x7fba6c06c5c0 2024-04-25T11:38:17.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.524+0000 7fba61400640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fba480744e0 0x7fba480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.525+0000 7fba61400640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fba480744e0 0x7fba480769f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0d4040 tx=0x7fba54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.528+0000 7fba62a00640 1 -- 172.21.15.19:0/315155665 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba6c068c00 con 0x7fba6c06c5c0 2024-04-25T11:38:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:17.777+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fba6c050740 con 0x7fba6c06c5c0 2024-04-25T11:38:18.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.401+0000 7fba62a00640 1 -- 172.21.15.19:0/315155665 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v206) ==== 161+0+0 (secure 0 0 0) 0x7fba5c022070 con 0x7fba6c06c5c0 2024-04-25T11:38:18.389 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fba480744e0 msgr2=0x7fba480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fba480744e0 0x7fba480769f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0d4040 tx=0x7fba54009040 comp rx=0 tx=0).stop 2024-04-25T11:38:18.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba6c06c5c0 msgr2=0x7fba6c15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba6c06c5c0 0x7fba6c15b3f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009d10 tx=0x7fba5c00c490 comp rx=0 tx=0).stop 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 shutdown_connections 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fba480744e0 0x7fba480769f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fba6c0d4a50 0x7fba6c1fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba6c0d0e00 0x7fba6c1fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 --2- 172.21.15.19:0/315155665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba6c06c5c0 0x7fba6c15b3f0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 >> 172.21.15.19:0/315155665 conn(0x7fba6c0d5d90 msgr2=0x7fba6c15ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 shutdown_connections 2024-04-25T11:38:18.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.405+0000 7fba71200640 1 -- 172.21.15.19:0/315155665 wait complete. 2024-04-25T11:38:18.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:18.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.713+0000 7f5f5ce00640 1 Processor -- start 2024-04-25T11:38:18.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.713+0000 7f5f5ce00640 1 -- start start 2024-04-25T11:38:18.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.715+0000 7f5f5ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5f580d0bd0 0x7f5f580d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.716+0000 7f5f5ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f580d1520 con 0x7f5f58160aa0 2024-04-25T11:38:18.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.716+0000 7f5f5ce00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f5806c7d0 con 0x7f5f580b6430 2024-04-25T11:38:18.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.716+0000 7f5f5ce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5f5806df80 con 0x7f5f580d0bd0 2024-04-25T11:38:18.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.716+0000 7f5f4ea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5f58160aa0 0x7f5f58160eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57334/0 (socket says 172.21.15.19:57334) 2024-04-25T11:38:18.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.716+0000 7f5f4ea00640 1 -- 172.21.15.19:0/346504026 learned_addr learned my addr 172.21.15.19:0/346504026 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:18.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.716+0000 7f5f57e00640 1 --2- 172.21.15.19:0/346504026 >> v2:172.21.15.19:3300/0 conn(0x7f5f580d0bd0 0x7f5f580d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.716+0000 7f5f56a00640 1 -- 172.21.15.19:0/346504026 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020738745 0 0) 0x7f5f580d1520 con 0x7f5f58160aa0 2024-04-25T11:38:18.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.717+0000 7f5f56a00640 1 -- 172.21.15.19:0/346504026 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f3c005730 con 0x7f5f58160aa0 2024-04-25T11:38:18.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.717+0000 7f5f57e00640 1 -- 172.21.15.19:0/346504026 >> v1:172.21.15.169:6789/0 conn(0x7f5f580b6430 legacy=0x7f5f5806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.717+0000 7f5f57e00640 1 -- 172.21.15.19:0/346504026 >> v1:172.21.15.19:6789/0 conn(0x7f5f58160aa0 legacy=0x7f5f58160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.717+0000 7f5f57e00640 1 -- 172.21.15.19:0/346504026 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5806cab0 con 0x7f5f580d0bd0 2024-04-25T11:38:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.718+0000 7f5f57e00640 1 --2- 172.21.15.19:0/346504026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580d0fe0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5f500097a0 tx=0x7f5f5002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69ad474a37c6362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.718+0000 7f5f56a00640 1 -- 172.21.15.19:0/346504026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50037070 con 0x7f5f580d0bd0 2024-04-25T11:38:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.718+0000 7f5f56a00640 1 -- 172.21.15.19:0/346504026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f50032440 con 0x7f5f580d0bd0 2024-04-25T11:38:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.718+0000 7f5f56a00640 1 -- 172.21.15.19:0/346504026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5002ae60 con 0x7f5f580d0bd0 2024-04-25T11:38:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.719+0000 7f5f5ce00640 1 -- 172.21.15.19:0/346504026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 msgr2=0x7f5f580d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.719+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/346504026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580d0fe0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5f500097a0 tx=0x7f5f5002a110 comp rx=0 tx=0).stop 2024-04-25T11:38:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.719+0000 7f5f5ce00640 1 -- 172.21.15.19:0/346504026 shutdown_connections 2024-04-25T11:38:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.719+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/346504026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580d0fe0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.719+0000 7f5f5ce00640 1 -- 172.21.15.19:0/346504026 >> 172.21.15.19:0/346504026 conn(0x7f5f580d1720 msgr2=0x7f5f580d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.719+0000 7f5f5ce00640 1 -- 172.21.15.19:0/346504026 shutdown_connections 2024-04-25T11:38:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.719+0000 7f5f5ce00640 1 -- 172.21.15.19:0/346504026 wait complete. 2024-04-25T11:38:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.721+0000 7f5f5ce00640 1 Processor -- start 2024-04-25T11:38:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.722+0000 7f5f5ce00640 1 -- start start 2024-04-25T11:38:18.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.723+0000 7f5f5ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f580b6430 0x7f5f581f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.723+0000 7f5f5ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.723+0000 7f5f5ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58160aa0 0x7f5f58143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.723+0000 7f5f5ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f580b6840 con 0x7f5f580d0bd0 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.723+0000 7f5f5ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5806d070 con 0x7f5f58160aa0 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.723+0000 7f5f5ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5f580b4f00 con 0x7f5f580b6430 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.723+0000 7f5f4ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f4ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53058/0 (socket says 172.21.15.19:53058) 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f4ea00640 1 -- 172.21.15.19:0/1328605158 learned_addr learned my addr 172.21.15.19:0/1328605158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f57e00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58160aa0 0x7f5f58143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f57400640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f580b6430 0x7f5f581f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f4ea00640 1 -- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f580b6430 msgr2=0x7f5f581f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f4ea00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f580b6430 0x7f5f581f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f4ea00640 1 -- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58160aa0 msgr2=0x7f5f58143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f4ea00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58160aa0 0x7f5f58143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.724+0000 7f5f4ea00640 1 -- 172.21.15.19:0/1328605158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f58159020 con 0x7f5f580d0bd0 2024-04-25T11:38:18.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.725+0000 7f5f4ea00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580c7330 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000cbc0 tx=0x7f5f4000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.725+0000 7f5f4fe00640 1 -- 172.21.15.19:0/1328605158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40017070 con 0x7f5f580d0bd0 2024-04-25T11:38:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.725+0000 7f5f4fe00640 1 -- 172.21.15.19:0/1328605158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f4000ad40 con 0x7f5f580d0bd0 2024-04-25T11:38:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.725+0000 7f5f4fe00640 1 -- 172.21.15.19:0/1328605158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40012620 con 0x7f5f580d0bd0 2024-04-25T11:38:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.725+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f58158990 con 0x7f5f580d0bd0 2024-04-25T11:38:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.725+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f581592c0 con 0x7f5f580d0bd0 2024-04-25T11:38:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.726+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f580c7c10 con 0x7f5f580d0bd0 2024-04-25T11:38:18.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.728+0000 7f5f4fe00640 1 -- 172.21.15.19:0/1328605158 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f4001a070 con 0x7f5f580d0bd0 2024-04-25T11:38:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.729+0000 7f5f4fe00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f340744e0 0x7f5f340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.729+0000 7f5f4fe00640 1 -- 172.21.15.19:0/1328605158 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f5f40095670 con 0x7f5f580d0bd0 2024-04-25T11:38:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.729+0000 7f5f57400640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f340744e0 0x7f5f340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.731+0000 7f5f57400640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f340744e0 0x7f5f340769f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5f48007fd0 tx=0x7f5f48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:18.733+0000 7f5f4fe00640 1 -- 172.21.15.19:0/1328605158 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f4005f7f0 con 0x7f5f580d0bd0 2024-04-25T11:38:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.021+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5f5806e100 con 0x7f5f580d0bd0 2024-04-25T11:38:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.423+0000 7f5f4fe00640 1 -- 172.21.15.19:0/1328605158 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v207) ==== 101+0+0 (secure 0 0 0) 0x7f5f4005f0f0 con 0x7f5f580d0bd0 2024-04-25T11:38:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:38:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f340744e0 msgr2=0x7f5f340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f340744e0 0x7f5f340769f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5f48007fd0 tx=0x7f5f48009040 comp rx=0 tx=0).stop 2024-04-25T11:38:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 msgr2=0x7f5f580c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580c7330 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000cbc0 tx=0x7f5f4000a430 comp rx=0 tx=0).stop 2024-04-25T11:38:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 shutdown_connections 2024-04-25T11:38:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5f340744e0 0x7f5f340769f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f58160aa0 0x7f5f58143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f580d0bd0 0x7f5f580c7330 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 --2- 172.21.15.19:0/1328605158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f580b6430 0x7f5f581f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 >> 172.21.15.19:0/1328605158 conn(0x7f5f580d1720 msgr2=0x7f5f580d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.427+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 shutdown_connections 2024-04-25T11:38:19.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.428+0000 7f5f5ce00640 1 -- 172.21.15.19:0/1328605158 wait complete. 2024-04-25T11:38:19.444 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.632+0000 7f984a31cf40 1 Processor -- start 2024-04-25T11:38:19.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.632+0000 7f984a31cf40 1 -- start start 2024-04-25T11:38:19.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f984a31cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98280030f0 0x55dcee521390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f984a31cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee4d4930 0x55dcee5a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f984a31cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcee5aa060 0x55dcee5aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f984a31cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcee51de70 con 0x55dcee5aa060 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f984a31cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dcee47bd20 con 0x7f98280030f0 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f984a31cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55dcee51e110 con 0x55dcee4d4930 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcee5aa060 0x55dcee5aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcee5aa060 0x55dcee5aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53090/0 (socket says 172.21.15.19:53090) 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 -- 172.21.15.19:0/1488824865 learned_addr learned my addr 172.21.15.19:0/1488824865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9842000640 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee4d4930 0x55dcee5a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9842a00640 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98280030f0 0x55dcee521390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 -- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee4d4930 msgr2=0x55dcee5a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee4d4930 0x55dcee5a8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 -- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98280030f0 msgr2=0x55dcee521390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98280030f0 0x55dcee521390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9843400640 1 -- 172.21.15.19:0/1488824865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcee5aa9b0 con 0x55dcee5aa060 2024-04-25T11:38:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.634+0000 7f9842000640 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee4d4930 0x55dcee5a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.635+0000 7f9843400640 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcee5aa060 0x55dcee5aa470 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x55dcee51c940 tx=0x7f983c00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7db8940fcdd7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.635+0000 7f9840c00640 1 -- 172.21.15.19:0/1488824865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00e040 con 0x55dcee5aa060 2024-04-25T11:38:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.635+0000 7f9840c00640 1 -- 172.21.15.19:0/1488824865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983c018070 con 0x55dcee5aa060 2024-04-25T11:38:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.635+0000 7f9840c00640 1 -- 172.21.15.19:0/1488824865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c012850 con 0x55dcee5aa060 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 -- 172.21.15.19:0/1488824865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcee5aa060 msgr2=0x55dcee5aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcee5aa060 0x55dcee5aa470 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x55dcee51c940 tx=0x7f983c00d8d0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 -- 172.21.15.19:0/1488824865 shutdown_connections 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dcee5aa060 0x55dcee5aa470 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee4d4930 0x55dcee5a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 --2- 172.21.15.19:0/1488824865 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98280030f0 0x55dcee521390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 -- 172.21.15.19:0/1488824865 >> 172.21.15.19:0/1488824865 conn(0x55dcee3f7ac0 msgr2=0x55dcee506ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 -- 172.21.15.19:0/1488824865 shutdown_connections 2024-04-25T11:38:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.636+0000 7f984a31cf40 1 -- 172.21.15.19:0/1488824865 wait complete. 2024-04-25T11:38:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.637+0000 7f984a31cf40 1 Processor -- start 2024-04-25T11:38:19.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.637+0000 7f984a31cf40 1 -- start start 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f984a31cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcee5aa060 0x55dcee5aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f984a31cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98280030f0 0x55dcee506110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f984a31cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee506650 0x55dcee58fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f984a31cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dcee5a9780 con 0x7f98280030f0 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f984a31cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dcee5a91c0 con 0x55dcee5aa060 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f984a31cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55dcee5a94a0 con 0x55dcee506650 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f9842000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98280030f0 0x55dcee506110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f9842000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98280030f0 0x55dcee506110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53094/0 (socket says 172.21.15.19:53094) 2024-04-25T11:38:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.639+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 learned_addr learned my addr 172.21.15.19:0/2266017888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9843400640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee506650 0x55dcee58fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9842a00640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcee5aa060 0x55dcee5aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee506650 msgr2=0x55dcee58fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9842000640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dcee506650 0x55dcee58fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcee5aa060 msgr2=0x55dcee5aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9842000640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dcee5aa060 0x55dcee5aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcee506e80 con 0x7f98280030f0 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f9842000640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98280030f0 0x55dcee506110 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9830046180 tx=0x7f9830003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.640+0000 7f983b400640 1 -- 172.21.15.19:0/2266017888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830068070 con 0x7f98280030f0 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.641+0000 7f983b400640 1 -- 172.21.15.19:0/2266017888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983004fa80 con 0x7f98280030f0 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.641+0000 7f983b400640 1 -- 172.21.15.19:0/2266017888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983004fe50 con 0x7f98280030f0 2024-04-25T11:38:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.641+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dcee590750 con 0x7f98280030f0 2024-04-25T11:38:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.641+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcee590d10 con 0x7f98280030f0 2024-04-25T11:38:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.643+0000 7f983b400640 1 -- 172.21.15.19:0/2266017888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f983006b070 con 0x7f98280030f0 2024-04-25T11:38:19.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.644+0000 7f983b400640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9824073060 0x7f9824075570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.644+0000 7f983b400640 1 -- 172.21.15.19:0/2266017888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f98300d77e0 con 0x7f98280030f0 2024-04-25T11:38:19.631 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:19.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.644+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55dcee591a20 0x55dcee591e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.644+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee5ab380 con 0x55dcee591a20 2024-04-25T11:38:19.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.645+0000 7f9842a00640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9824073060 0x7f9824075570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.645+0000 7f9843400640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55dcee591a20 0x55dcee591e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.646+0000 7f9843400640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55dcee591a20 0x55dcee591e30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.646+0000 7f9842a00640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9824073060 0x7f9824075570 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9834000c00 tx=0x7f983400f8a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.649+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v110'536 uv510 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.649+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee64f060 con 0x55dcee591a20 2024-04-25T11:38:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.652+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v110'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.652+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64f400 con 0x55dcee591a20 2024-04-25T11:38:19.642 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:19.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.655+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v110'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.655+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6507b0 con 0x55dcee591a20 2024-04-25T11:38:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.658+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.658+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee650b50 con 0x55dcee591a20 2024-04-25T11:38:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.661+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.661+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64f7a0 con 0x55dcee591a20 2024-04-25T11:38:19.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.665+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.665+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.669+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v110'542 uv516 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.669+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.673+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v110'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.673+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.677+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v110'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.677+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.681+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.681+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.685+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.685+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.689+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.689+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.694+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v110'548 uv523 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.694+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.697+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'549 uv549 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.697+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.688 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:19.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.701+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v110'550 uv550 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.701+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.705+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'551 uv551 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.705+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.709+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'552 uv552 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.709+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fb60 con 0x55dcee591a20 2024-04-25T11:38:19.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.712+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'553 uv553 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.712+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64ff20 con 0x55dcee591a20 2024-04-25T11:38:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.717+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'554 uv554 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.717+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64ff20 con 0x55dcee591a20 2024-04-25T11:38:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.721+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v110'555 uv529 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.721+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee64ff20 con 0x55dcee591a20 2024-04-25T11:38:19.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.725+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v110'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.725+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64ff20 con 0x55dcee591a20 2024-04-25T11:38:19.716 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:19.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.729+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v110'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.730+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64ff20 con 0x55dcee591a20 2024-04-25T11:38:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.734+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.734+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.738+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.738+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.742+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.743+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.747+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v110'561 uv535 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.747+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.751+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v110'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.752+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.755+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v110'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.743 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.756+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.760+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.760+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.764+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.765+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee591a20 2024-04-25T11:38:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.769+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.769+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55dcee650ef0 0x55dcee653310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.769+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee64fe20 con 0x55dcee650ef0 2024-04-25T11:38:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.770+0000 7f9842a00640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55dcee650ef0 0x55dcee653310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.771+0000 7f9842a00640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55dcee650ef0 0x55dcee653310 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.776+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'396 uv378 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.776+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee653850 con 0x55dcee650ef0 2024-04-25T11:38:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.780+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'397 uv397 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.780+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee653bf0 con 0x55dcee650ef0 2024-04-25T11:38:19.771 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.784+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'398 uv398 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.784+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee653f90 con 0x55dcee650ef0 2024-04-25T11:38:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.789+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v110'399 uv399 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.789+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654330 con 0x55dcee650ef0 2024-04-25T11:38:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.793+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'400 uv400 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.793+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6546d0 con 0x55dcee650ef0 2024-04-25T11:38:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.797+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v110'401 uv383 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.797+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.801+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v110'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.801+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.793 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.806+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v110'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.806+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.810+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.811+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.815+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.815+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.819+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v110'406 uv395 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.819+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.823+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v110'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.824+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.814 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.827+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v110'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.828+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659160 con 0x55dcee650ef0 2024-04-25T11:38:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.832+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v110'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.833+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659160 con 0x55dcee650ef0 2024-04-25T11:38:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.837+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.837+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.841+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v110'411 uv394 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.841+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.845+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v110'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.845+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.836 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.849+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v110'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.850+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee655a80 con 0x55dcee650ef0 2024-04-25T11:38:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.854+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.854+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee655a80 con 0x55dcee650ef0 2024-04-25T11:38:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.858+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.858+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee654a70 con 0x55dcee650ef0 2024-04-25T11:38:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.862+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee650ef0 2024-04-25T11:38:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.862+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55dcee6550f0 0x55dcee657510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.862+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee657b20 con 0x55dcee6550f0 2024-04-25T11:38:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.863+0000 7f9842000640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55dcee6550f0 0x55dcee657510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.865+0000 7f9842000640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55dcee6550f0 0x55dcee657510 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.868+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.869+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee658010 con 0x55dcee6550f0 2024-04-25T11:38:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.873+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v110'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:19.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.873+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6584f0 con 0x55dcee6550f0 2024-04-25T11:38:19.864 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.877+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v110'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.877+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659da0 con 0x55dcee6550f0 2024-04-25T11:38:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.881+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.881+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee65a140 con 0x55dcee6550f0 2024-04-25T11:38:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.885+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:19.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.885+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee658d90 con 0x55dcee6550f0 2024-04-25T11:38:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.889+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:19.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.890+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659180 con 0x55dcee591a20 2024-04-25T11:38:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.894+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'82 uv81 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.894+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee659180 con 0x55dcee591a20 2024-04-25T11:38:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.898+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'83 uv83 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.899+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659180 con 0x55dcee591a20 2024-04-25T11:38:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.902+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'84 uv84 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.890 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:19.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.903+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659180 con 0x55dcee591a20 2024-04-25T11:38:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.906+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'85 uv85 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.907+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55dcee65a4e0 0x55dcee65c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.907+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659180 con 0x55dcee65a4e0 2024-04-25T11:38:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.907+0000 7f9843400640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55dcee65a4e0 0x55dcee65c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.909+0000 7f9843400640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55dcee65a4e0 0x55dcee65c900 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.913+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.913+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee659180 con 0x55dcee65a4e0 2024-04-25T11:38:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.917+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v110'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:19.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.917+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee659180 con 0x55dcee65a4e0 2024-04-25T11:38:19.908 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.921+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v110'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.921+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee65dea0 con 0x55dcee65a4e0 2024-04-25T11:38:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.926+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:19.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.926+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee65dea0 con 0x55dcee65a4e0 2024-04-25T11:38:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.930+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.930+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee65ce90 con 0x55dcee65a4e0 2024-04-25T11:38:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.934+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.934+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55dcee65d3a0 0x55dcee65f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.934+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee65fdd0 con 0x55dcee65d3a0 2024-04-25T11:38:19.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.935+0000 7f9842a00640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55dcee65d3a0 0x55dcee65f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.937+0000 7f9842a00640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55dcee65d3a0 0x55dcee65f7c0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.941+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v110'193 uv187 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.941+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.945+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v110'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.945+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.949+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v110'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.936 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.949+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6611b0 con 0x55dcee65d3a0 2024-04-25T11:38:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.953+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.953+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6611b0 con 0x55dcee65d3a0 2024-04-25T11:38:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.957+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.958+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.961+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.962+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.965+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v110'199 uv192 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.965+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.970+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v110'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.970+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.974+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v110'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.961 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:19.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.974+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.978+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.978+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6601a0 con 0x55dcee65d3a0 2024-04-25T11:38:19.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.983+0000 7f9842a00640 1 -- 172.21.15.19:0/2266017888 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9834009040 con 0x55dcee65d3a0 2024-04-25T11:38:19.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.983+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dcee660ac0 0x55dcee660ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.983+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6635c0 con 0x55dcee660ac0 2024-04-25T11:38:19.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.984+0000 7f9842000640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dcee660ac0 0x55dcee660ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.985+0000 7f9842000640 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dcee660ac0 0x55dcee660ed0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.989+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v110'86 uv85 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:19.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.989+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee6635c0 con 0x55dcee660ac0 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.993+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v110'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:19.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.993+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663600 con 0x55dcee660ac0 2024-04-25T11:38:19.985 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.998+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v110'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:19.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:19.998+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6639f0 con 0x55dcee660ac0 2024-04-25T11:38:19.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.002+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'89 uv89 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:19.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.002+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6639f0 con 0x55dcee660ac0 2024-04-25T11:38:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.006+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'90 uv90 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.006+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6636d0 con 0x55dcee591a20 2024-04-25T11:38:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.011+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.011+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee6636d0 con 0x55dcee591a20 2024-04-25T11:38:20.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.015+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v110'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:20.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.015+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6636d0 con 0x55dcee591a20 2024-04-25T11:38:20.006 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.019+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v110'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.019+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee664b20 con 0x55dcee591a20 2024-04-25T11:38:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.023+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.023+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee664830 con 0x55dcee591a20 2024-04-25T11:38:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.027+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:20.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.027+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663820 con 0x55dcee591a20 2024-04-25T11:38:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.032+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee591a20 2024-04-25T11:38:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.032+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6637d0 con 0x55dcee6550f0 2024-04-25T11:38:20.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.037+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v110'172 uv166 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.037+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee6637d0 con 0x55dcee6550f0 2024-04-25T11:38:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.041+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v110'173 uv173 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.041+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6637d0 con 0x55dcee6550f0 2024-04-25T11:38:20.032 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.045+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v110'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.046+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663bc0 con 0x55dcee6550f0 2024-04-25T11:38:20.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.050+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v110'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.050+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663bc0 con 0x55dcee6550f0 2024-04-25T11:38:20.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.054+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v110'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.054+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663870 con 0x55dcee6550f0 2024-04-25T11:38:20.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.058+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v110'177 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.058+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee663870 con 0x55dcee6550f0 2024-04-25T11:38:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.062+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v110'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.062+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663870 con 0x55dcee6550f0 2024-04-25T11:38:20.053 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.066+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v110'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.066+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663870 con 0x55dcee6550f0 2024-04-25T11:38:20.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.070+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v110'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.070+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee6550f0 2024-04-25T11:38:20.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.075+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v110'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee6550f0 2024-04-25T11:38:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.075+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.080+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v110'78 uv77 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.080+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.084+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v110'79 uv79 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.084+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.075 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:20.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.088+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v110'80 uv80 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.088+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.092+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v110'81 uv81 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.093+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.097+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v110'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.097+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.101+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v110'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.101+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.092 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:20.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.105+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v110'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.106+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.110+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.110+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.114+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.114+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee65a4e0 2024-04-25T11:38:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.119+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983c0127b0 con 0x55dcee65a4e0 2024-04-25T11:38:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.119+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663c80 con 0x55dcee660ac0 2024-04-25T11:38:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.123+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v110'89 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.123+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.127+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v110'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.127+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.118 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:20.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.131+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v110'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.131+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.135+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v110'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.135+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.139+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v110'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.139+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.144+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v110'216 uv208 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.144+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.148+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v110'217 uv217 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.148+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.140 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.153+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v110'218 uv218 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.153+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.157+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v110'219 uv219 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.158+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.162+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v110'220 uv220 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.162+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.166+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v110'221 uv215 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.166+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.171+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v110'222 uv222 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.171+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.162 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.175+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v110'223 uv223 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.175+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.179+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v110'224 uv224 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.179+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.183+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v110'225 uv225 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.184+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee663b70 con 0x55dcee660ac0 2024-04-25T11:38:20.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.188+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v110'226 uv226 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.188+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55dcee6644f0 con 0x55dcee660ac0 2024-04-25T11:38:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.192+0000 7f9842000640 1 -- 172.21.15.19:0/2266017888 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v110'227 uv227 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f983006a030 con 0x55dcee660ac0 2024-04-25T11:38:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55dcee650ef0 msgr2=0x55dcee653310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55dcee650ef0 0x55dcee653310 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55dcee65d3a0 msgr2=0x55dcee65f7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55dcee65d3a0 0x55dcee65f7c0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dcee660ac0 msgr2=0x55dcee660ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55dcee660ac0 0x55dcee660ed0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55dcee6550f0 msgr2=0x55dcee657510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55dcee6550f0 0x55dcee657510 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55dcee591a20 msgr2=0x55dcee591e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f9843400640 1 -- 172.21.15.19:0/2266017888 reap_dead start 2024-04-25T11:38:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55dcee591a20 0x55dcee591e30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55dcee65a4e0 msgr2=0x55dcee65c900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55dcee65a4e0 0x55dcee65c900 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9824073060 msgr2=0x7f9824075570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9824073060 0x7f9824075570 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9834000c00 tx=0x7f983400f8a0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98280030f0 msgr2=0x55dcee506110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.193+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98280030f0 0x55dcee506110 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9830046180 tx=0x7f9830003020 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 shutdown_connections 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55dcee65a4e0 0x55dcee65c900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55dcee591a20 0x55dcee591e30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9824073060 0x7f9824075570 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 --2- 172.21.15.19:0/2266017888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98280030f0 0x55dcee506110 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 >> 172.21.15.19:0/2266017888 conn(0x55dcee3f7ac0 msgr2=0x55dcee5a8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:20.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 shutdown_connections 2024-04-25T11:38:20.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.194+0000 7f984a31cf40 1 -- 172.21.15.19:0/2266017888 wait complete. 2024-04-25T11:38:20.200 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2024-04-25T11:38:20.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.447+0000 7fd6d4121f40 1 Processor -- start 2024-04-25T11:38:20.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.447+0000 7fd6d4121f40 1 -- start start 2024-04-25T11:38:20.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6d4121f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 0x55d292ec81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6d4121f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6d4121f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292e95180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6d4121f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d292edee70 con 0x55d292f6ffa0 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6d4121f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d292e3cd20 con 0x55d292e965f0 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6d4121f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d292edf3f0 con 0x55d292e961e0 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6cdc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292e95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6cdc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292e95180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53114/0 (socket says 172.21.15.19:53114) 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6cdc00640 1 -- 172.21.15.19:0/3628112731 learned_addr learned my addr 172.21.15.19:0/3628112731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6cd200640 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 0x55d292ec81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6cc800640 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6cdc00640 1 -- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 msgr2=0x55d292ec81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:20.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.449+0000 7fd6cdc00640 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 0x55d292ec81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cdc00640 1 -- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 msgr2=0x55d292ec72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:20.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cdc00640 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cdc00640 1 -- 172.21.15.19:0/3628112731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d292f69670 con 0x55d292f6ffa0 2024-04-25T11:38:20.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cc800640 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cdc00640 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292e95180 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400a4e0 tx=0x7fd6c400a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e46abae8a49a54d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cb400640 1 -- 172.21.15.19:0/3628112731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4015070 con 0x55d292f6ffa0 2024-04-25T11:38:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cb400640 1 -- 172.21.15.19:0/3628112731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c400e710 con 0x55d292f6ffa0 2024-04-25T11:38:20.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.450+0000 7fd6cb400640 1 -- 172.21.15.19:0/3628112731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400eae0 con 0x55d292f6ffa0 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 -- 172.21.15.19:0/3628112731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 msgr2=0x55d292e95180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292e95180 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400a4e0 tx=0x7fd6c400a9a0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 -- 172.21.15.19:0/3628112731 shutdown_connections 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292e95180 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/3628112731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 0x55d292ec81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 -- 172.21.15.19:0/3628112731 >> 172.21.15.19:0/3628112731 conn(0x55d292db8ac0 msgr2=0x55d292ee3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 -- 172.21.15.19:0/3628112731 shutdown_connections 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.451+0000 7fd6d4121f40 1 -- 172.21.15.19:0/3628112731 wait complete. 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.452+0000 7fd6d4121f40 1 Processor -- start 2024-04-25T11:38:20.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.452+0000 7fd6d4121f40 1 -- start start 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6d4121f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 0x55d292f5cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6d4121f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6d4121f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292ec82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6d4121f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d292f6cbf0 con 0x55d292f6ffa0 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6d4121f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d292f62730 con 0x55d292e965f0 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6d4121f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d292f61770 con 0x55d292e961e0 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6cdc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292ec82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6cdc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292ec82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53130/0 (socket says 172.21.15.19:53130) 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6cdc00640 1 -- 172.21.15.19:0/2147253981 learned_addr learned my addr 172.21.15.19:0/2147253981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.454+0000 7fd6cd200640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 0x55d292f5cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6cc800640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6cdc00640 1 -- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 msgr2=0x55d292f5cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6cdc00640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d292e961e0 0x55d292f5cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6cdc00640 1 -- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 msgr2=0x55d292ec7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6cdc00640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d292e965f0 0x55d292ec7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6cdc00640 1 -- 172.21.15.19:0/2147253981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d292f66300 con 0x55d292f6ffa0 2024-04-25T11:38:20.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6cdc00640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292ec82f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400a9e0 tx=0x7fd6c4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:20.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6ca000640 1 -- 172.21.15.19:0/2147253981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4020070 con 0x55d292f6ffa0 2024-04-25T11:38:20.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6ca000640 1 -- 172.21.15.19:0/2147253981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c40107b0 con 0x55d292f6ffa0 2024-04-25T11:38:20.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6ca000640 1 -- 172.21.15.19:0/2147253981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4015040 con 0x55d292f6ffa0 2024-04-25T11:38:20.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.455+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d292ee1790 con 0x55d292f6ffa0 2024-04-25T11:38:20.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.456+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d292f665a0 con 0x55d292f6ffa0 2024-04-25T11:38:20.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.458+0000 7fd6ca000640 1 -- 172.21.15.19:0/2147253981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6c4024400 con 0x55d292f6ffa0 2024-04-25T11:38:20.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.459+0000 7fd6ca000640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6ac072e90 0x7fd6ac0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.459+0000 7fd6ca000640 1 -- 172.21.15.19:0/2147253981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7fd6c4095280 con 0x55d292f6ffa0 2024-04-25T11:38:20.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.460+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d292f68f50 0x55d292f5fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.460+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55d292f60290 con 0x55d292f68f50 2024-04-25T11:38:20.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.460+0000 7fd6cc800640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d292f68f50 0x55d292f5fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.460+0000 7fd6cd200640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6ac072e90 0x7fd6ac0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.461+0000 7fd6cc800640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d292f68f50 0x55d292f5fd50 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:20.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.462+0000 7fd6cd200640 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6ac072e90 0x7fd6ac0753a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0057000 tx=0x7fd6c0052000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:20.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.464+0000 7fd6cc800640 1 -- 172.21.15.19:0/2147253981 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v110'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd6b800b430 con 0x55d292f68f50 2024-04-25T11:38:20.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.464+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d292f68f50 msgr2=0x55d292f5fd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.464+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d292f68f50 0x55d292f5fd50 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6ac072e90 msgr2=0x7fd6ac0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6ac072e90 0x7fd6ac0753a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0057000 tx=0x7fd6c0052000 comp rx=0 tx=0).stop 2024-04-25T11:38:20.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 msgr2=0x55d292ec82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 --2- 172.21.15.19:0/2147253981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d292f6ffa0 0x55d292ec82f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400a9e0 tx=0x7fd6c4003330 comp rx=0 tx=0).stop 2024-04-25T11:38:20.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6cdc00640 1 -- 172.21.15.19:0/2147253981 reap_dead start 2024-04-25T11:38:20.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 shutdown_connections 2024-04-25T11:38:20.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 >> 172.21.15.19:0/2147253981 conn(0x55d292db8ac0 msgr2=0x55d292f61db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:20.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 shutdown_connections 2024-04-25T11:38:20.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.465+0000 7fd6d4121f40 1 -- 172.21.15.19:0/2147253981 wait complete. 2024-04-25T11:38:20.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:20.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.764+0000 7febfa000640 1 Processor -- start 2024-04-25T11:38:20.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.765+0000 7febfa000640 1 -- start start 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.766+0000 7febfa000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febf40cd9a0 0x7febf415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.766+0000 7febfa000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf406dce0 0x7febf415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.766+0000 7febfa000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf40cdea0 con 0x7febf40b0af0 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.766+0000 7febfa000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7febf40ceab0 con 0x7febf40cd9a0 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.767+0000 7febfa000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7febf40ce180 con 0x7febf406dce0 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.767+0000 7febf3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febf40cd9a0 0x7febf415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.767+0000 7febf3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febf40cd9a0 0x7febf415f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53136/0 (socket says 172.21.15.19:53136) 2024-04-25T11:38:20.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.767+0000 7febf3400640 1 -- 172.21.15.19:0/461250948 learned_addr learned my addr 172.21.15.19:0/461250948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:20.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.767+0000 7febf3400640 1 -- 172.21.15.19:0/461250948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf406dce0 msgr2=0x7febf415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:20.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.767+0000 7febf3e00640 1 --2- 172.21.15.19:0/461250948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf406dce0 0x7febf415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.768+0000 7febf3400640 1 --2- 172.21.15.19:0/461250948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf406dce0 0x7febf415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.767+0000 7febf2a00640 1 -- 172.21.15.19:0/461250948 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535474578 0 0) 0x7febf40cdea0 con 0x7febf40b0af0 2024-04-25T11:38:20.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.768+0000 7febf3400640 1 -- 172.21.15.19:0/461250948 >> v1:172.21.15.19:6789/0 conn(0x7febf40b0af0 legacy=0x7febf406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.768+0000 7febf3400640 1 -- 172.21.15.19:0/461250948 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf40ce460 con 0x7febf40cd9a0 2024-04-25T11:38:20.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.768+0000 7febf3e00640 1 --2- 172.21.15.19:0/461250948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf406dce0 0x7febf415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:20.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.769+0000 7febf3400640 1 --2- 172.21.15.19:0/461250948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf415f360 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7febe4009870 tx=0x7febe402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e79ffe7bd3811fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:20.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.769+0000 7febf2a00640 1 -- 172.21.15.19:0/461250948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4037070 con 0x7febf40cd9a0 2024-04-25T11:38:20.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.769+0000 7febf2a00640 1 -- 172.21.15.19:0/461250948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4032440 con 0x7febf40cd9a0 2024-04-25T11:38:20.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.769+0000 7febf2a00640 1 -- 172.21.15.19:0/461250948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4032810 con 0x7febf40cd9a0 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 -- 172.21.15.19:0/461250948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 msgr2=0x7febf415f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 --2- 172.21.15.19:0/461250948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf415f360 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7febe4009870 tx=0x7febe402a250 comp rx=0 tx=0).stop 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 -- 172.21.15.19:0/461250948 shutdown_connections 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 --2- 172.21.15.19:0/461250948 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf406dce0 0x7febf415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 --2- 172.21.15.19:0/461250948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf415f360 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 -- 172.21.15.19:0/461250948 >> 172.21.15.19:0/461250948 conn(0x7febf4073090 msgr2=0x7febf40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 -- 172.21.15.19:0/461250948 shutdown_connections 2024-04-25T11:38:20.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.770+0000 7febfa000640 1 -- 172.21.15.19:0/461250948 wait complete. 2024-04-25T11:38:20.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.773+0000 7febfa000640 1 Processor -- start 2024-04-25T11:38:20.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.773+0000 7febfa000640 1 -- start start 2024-04-25T11:38:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.774+0000 7febfa000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febf40b0af0 0x7febf413ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.774+0000 7febfa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf41fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.774+0000 7febfa000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf41fa970 0x7febf40b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.774+0000 7febfa000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febf40cef40 con 0x7febf40cd9a0 2024-04-25T11:38:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.774+0000 7febfa000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febf40b0f00 con 0x7febf40b0af0 2024-04-25T11:38:20.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.774+0000 7febfa000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7febf40cf0c0 con 0x7febf41fa970 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf41fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf41fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53146/0 (socket says 172.21.15.19:53146) 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 -- 172.21.15.19:0/2411840387 learned_addr learned my addr 172.21.15.19:0/2411840387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febf3e00640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf41fa970 0x7febf40b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febf3400640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febf40b0af0 0x7febf413ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 -- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf41fa970 msgr2=0x7febf40b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf41fa970 0x7febf40b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 -- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febf40b0af0 msgr2=0x7febf413ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febf40b0af0 0x7febf413ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 -- 172.21.15.19:0/2411840387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf4140910 con 0x7febf40cd9a0 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febf3e00640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf41fa970 0x7febf40b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febf3400640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febf40b0af0 0x7febf413ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:20.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.775+0000 7febeaa00640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf41fb360 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c3c0 tx=0x7febdc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.776+0000 7febebe00640 1 -- 172.21.15.19:0/2411840387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc017070 con 0x7febf40cd9a0 2024-04-25T11:38:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.776+0000 7febebe00640 1 -- 172.21.15.19:0/2411840387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febdc0035f0 con 0x7febf40cd9a0 2024-04-25T11:38:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.776+0000 7febebe00640 1 -- 172.21.15.19:0/2411840387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc0039e0 con 0x7febf40cd9a0 2024-04-25T11:38:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.776+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf4140bf0 con 0x7febf40cd9a0 2024-04-25T11:38:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.776+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf41411b0 con 0x7febf40cd9a0 2024-04-25T11:38:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.777+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf4050740 con 0x7febf40cd9a0 2024-04-25T11:38:20.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.779+0000 7febebe00640 1 -- 172.21.15.19:0/2411840387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febdc01a070 con 0x7febf40cd9a0 2024-04-25T11:38:20.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.780+0000 7febebe00640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd00744b0 0x7febd00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:20.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.780+0000 7febebe00640 1 -- 172.21.15.19:0/2411840387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7febdc094ca0 con 0x7febf40cd9a0 2024-04-25T11:38:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.780+0000 7febf3400640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd00744b0 0x7febd00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:20.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.782+0000 7febf3400640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd00744b0 0x7febd00769c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7febe4034000 tx=0x7febe402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:20.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:20.785+0000 7febebe00640 1 -- 172.21.15.19:0/2411840387 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febdc05ed80 con 0x7febf40cd9a0 2024-04-25T11:38:21.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.029+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7febf4143fd0 con 0x7febf40cd9a0 2024-04-25T11:38:21.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.444+0000 7febebe00640 1 -- 172.21.15.19:0/2411840387 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v208) ==== 162+0+0 (secure 0 0 0) 0x7febdc05e680 con 0x7febf40cd9a0 2024-04-25T11:38:21.431 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:21.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd00744b0 msgr2=0x7febd00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd00744b0 0x7febd00769c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7febe4034000 tx=0x7febe402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 msgr2=0x7febf41fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf41fb360 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c3c0 tx=0x7febdc00c880 comp rx=0 tx=0).stop 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 shutdown_connections 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febd00744b0 0x7febd00769c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febf41fa970 0x7febf40b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febf40cd9a0 0x7febf41fb360 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 --2- 172.21.15.19:0/2411840387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febf40b0af0 0x7febf413ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.447+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 >> 172.21.15.19:0/2411840387 conn(0x7febf4073090 msgr2=0x7febf406d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.448+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 shutdown_connections 2024-04-25T11:38:21.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.448+0000 7febfa000640 1 -- 172.21.15.19:0/2411840387 wait complete. 2024-04-25T11:38:21.447 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:21.460+0000 7fdfc5600640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-04-25T11:38:21.448 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:21.460+0000 7fdfc5600640 -1 mds.0.209 unhandled write error (22) Invalid argument, force readonly... 2024-04-25T11:38:21.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.746+0000 7ff049600640 1 Processor -- start 2024-04-25T11:38:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.746+0000 7ff049600640 1 -- start start 2024-04-25T11:38:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.749+0000 7ff049600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff0440cf9e0 0x7ff044161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.749+0000 7ff049600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440b0800 0x7ff044070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.749+0000 7ff049600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0441615d0 con 0x7ff0440707d0 2024-04-25T11:38:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.749+0000 7ff049600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff044070be0 con 0x7ff0440cf9e0 2024-04-25T11:38:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.749+0000 7ff049600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff0440cffc0 con 0x7ff0440b0800 2024-04-25T11:38:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff03be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440b0800 0x7ff044070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff03be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440b0800 0x7ff044070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:42936/0 (socket says 172.21.15.19:42936) 2024-04-25T11:38:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff03be00640 1 -- 172.21.15.19:0/2378020520 learned_addr learned my addr 172.21.15.19:0/2378020520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff043e00640 1 --2- 172.21.15.19:0/2378020520 >> v2:172.21.15.169:3300/0 conn(0x7ff0440cf9e0 0x7ff044161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff03be00640 1 -- 172.21.15.19:0/2378020520 >> v2:172.21.15.169:3300/0 conn(0x7ff0440cf9e0 msgr2=0x7ff044161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff03be00640 1 --2- 172.21.15.19:0/2378020520 >> v2:172.21.15.169:3300/0 conn(0x7ff0440cf9e0 0x7ff044161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff03be00640 1 -- 172.21.15.19:0/2378020520 >> v1:172.21.15.169:6789/0 conn(0x7ff0440707d0 legacy=0x7ff0441569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.750+0000 7ff03be00640 1 -- 172.21.15.19:0/2378020520 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0440d02a0 con 0x7ff0440b0800 2024-04-25T11:38:21.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.752+0000 7ff03be00640 1 --2- 172.21.15.19:0/2378020520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440b0800 0x7ff044070290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009870 tx=0x7ff02c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d92a09a94921dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:21.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.752+0000 7ff043400640 1 -- 172.21.15.19:0/2378020520 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c037070 con 0x7ff0440b0800 2024-04-25T11:38:21.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.752+0000 7ff043400640 1 -- 172.21.15.19:0/2378020520 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c032440 con 0x7ff0440b0800 2024-04-25T11:38:21.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.752+0000 7ff043400640 1 -- 172.21.15.19:0/2378020520 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c032810 con 0x7ff0440b0800 2024-04-25T11:38:21.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 -- 172.21.15.19:0/2378020520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440b0800 msgr2=0x7ff044070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 --2- 172.21.15.19:0/2378020520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440b0800 0x7ff044070290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009870 tx=0x7ff02c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 -- 172.21.15.19:0/2378020520 shutdown_connections 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 --2- 172.21.15.19:0/2378020520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440b0800 0x7ff044070290 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 --2- 172.21.15.19:0/2378020520 >> v2:172.21.15.169:3300/0 conn(0x7ff0440cf9e0 0x7ff044161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 -- 172.21.15.19:0/2378020520 >> 172.21.15.19:0/2378020520 conn(0x7ff0440d1720 msgr2=0x7ff0440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 -- 172.21.15.19:0/2378020520 shutdown_connections 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.753+0000 7ff049600640 1 -- 172.21.15.19:0/2378020520 wait complete. 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.756+0000 7ff049600640 1 Processor -- start 2024-04-25T11:38:21.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.756+0000 7ff049600640 1 -- start start 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.757+0000 7ff049600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440707d0 0x7ff044143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.757+0000 7ff049600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0440b0800 0x7ff044147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.757+0000 7ff049600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0440cf9e0 0x7ff0440b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.757+0000 7ff049600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0440b0c10 con 0x7ff0440b0800 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.757+0000 7ff049600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0440d0860 con 0x7ff0440cf9e0 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.757+0000 7ff049600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff0440b0d90 con 0x7ff0440707d0 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0440b0800 0x7ff044147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0440b0800 0x7ff044147130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53148/0 (socket says 172.21.15.19:53148) 2024-04-25T11:38:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 -- 172.21.15.19:0/3484347951 learned_addr learned my addr 172.21.15.19:0/3484347951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:21.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff043e00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440707d0 0x7ff044143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff048c00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0440cf9e0 0x7ff0440b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 -- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440707d0 msgr2=0x7ff044143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440707d0 0x7ff044143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 -- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0440cf9e0 msgr2=0x7ff0440b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0440cf9e0 0x7ff0440b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff03be00640 1 -- 172.21.15.19:0/3484347951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff044147670 con 0x7ff0440b0800 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.758+0000 7ff048c00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0440cf9e0 0x7ff0440b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.759+0000 7ff03be00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0440b0800 0x7ff044147130 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff02c000c00 tx=0x7ff02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.759+0000 7ff040c00640 1 -- 172.21.15.19:0/3484347951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c037040 con 0x7ff0440b0800 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.759+0000 7ff040c00640 1 -- 172.21.15.19:0/3484347951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02c042070 con 0x7ff0440b0800 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.759+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0441ea020 con 0x7ff0440b0800 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.759+0000 7ff040c00640 1 -- 172.21.15.19:0/3484347951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c030880 con 0x7ff0440b0800 2024-04-25T11:38:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.759+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0441ea5e0 con 0x7ff0440b0800 2024-04-25T11:38:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.760+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0441e9d90 con 0x7ff0440b0800 2024-04-25T11:38:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.762+0000 7ff040c00640 1 -- 172.21.15.19:0/3484347951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff02c02ad10 con 0x7ff0440b0800 2024-04-25T11:38:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.762+0000 7ff040c00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0200744b0 0x7ff0200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.763+0000 7ff040c00640 1 -- 172.21.15.19:0/3484347951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7ff02c0b5db0 con 0x7ff0440b0800 2024-04-25T11:38:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.763+0000 7ff043e00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0200744b0 0x7ff0200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.765+0000 7ff043e00640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0200744b0 0x7ff0200769c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff0440aea20 tx=0x7ff034008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:21.767+0000 7ff040c00640 1 -- 172.21.15.19:0/3484347951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff02c0ba070 con 0x7ff0440b0800 2024-04-25T11:38:22.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.013+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff044050740 con 0x7ff0440b0800 2024-04-25T11:38:22.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.015+0000 7ff040c00640 1 -- 172.21.15.19:0/3484347951 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) ==== 95+0+5184 (secure 0 0 0) 0x7ff044147670 con 0x7ff0440b0800 2024-04-25T11:38:22.002 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:22.002 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":209,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":209,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:21.450550+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9300":{"gid":9300,"name":"a","rank":0,"incarnation":209,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/614168622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":614168622},{"type":"v1","addr":"172.21.15.19:6835","nonce":614168622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:22.003 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 209 2024-04-25T11:38:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.017+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0200744b0 msgr2=0x7ff0200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.017+0000 7ff049600640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0200744b0 0x7ff0200769c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff0440aea20 tx=0x7ff034008040 comp rx=0 tx=0).stop 2024-04-25T11:38:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.017+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0440b0800 msgr2=0x7ff044147130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.017+0000 7ff049600640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0440b0800 0x7ff044147130 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff02c000c00 tx=0x7ff02c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 shutdown_connections 2024-04-25T11:38:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0200744b0 0x7ff0200769c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0440cf9e0 0x7ff0440b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0440b0800 0x7ff044147130 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 --2- 172.21.15.19:0/3484347951 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0440707d0 0x7ff044143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 >> 172.21.15.19:0/3484347951 conn(0x7ff0440d1720 msgr2=0x7ff0440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 shutdown_connections 2024-04-25T11:38:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.018+0000 7ff049600640 1 -- 172.21.15.19:0/3484347951 wait complete. 2024-04-25T11:38:22.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:22.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.728+0000 7f458f200640 1 Processor -- start 2024-04-25T11:38:22.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.728+0000 7f458f200640 1 -- start start 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.730+0000 7f458f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45900cf2f0 0x7f45900cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.730+0000 7f458f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4590168010 0x7f4590168420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.730+0000 7f458f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45901649d0 0x7f4590164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.730+0000 7f458f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4590070cf0 con 0x7f45900cf2f0 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.730+0000 7f458f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4590168960 con 0x7f4590168010 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4590071180 con 0x7f45901649d0 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45900cf2f0 0x7f45900cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45900cf2f0 0x7f45900cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53154/0 (socket says 172.21.15.19:53154) 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 -- 172.21.15.19:0/1463413014 learned_addr learned my addr 172.21.15.19:0/1463413014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:22.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 -- 172.21.15.19:0/1463413014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45901649d0 msgr2=0x7f4590164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 --2- 172.21.15.19:0/1463413014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45901649d0 0x7f4590164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 -- 172.21.15.19:0/1463413014 >> v2:172.21.15.169:3300/0 conn(0x7f4590168010 msgr2=0x7f4590168420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 --2- 172.21.15.19:0/1463413014 >> v2:172.21.15.169:3300/0 conn(0x7f4590168010 0x7f4590168420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.731+0000 7f458de00640 1 -- 172.21.15.19:0/1463413014 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45900700e0 con 0x7f45900cf2f0 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.732+0000 7f458de00640 1 --2- 172.21.15.19:0/1463413014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45900cf2f0 0x7f45900cf700 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f45900cfed0 tx=0x7f457802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e161b32e2515308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.733+0000 7f458ca00640 1 -- 172.21.15.19:0/1463413014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578037070 con 0x7f45900cf2f0 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.733+0000 7f458ca00640 1 -- 172.21.15.19:0/1463413014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4578032440 con 0x7f45900cf2f0 2024-04-25T11:38:22.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.733+0000 7f458ca00640 1 -- 172.21.15.19:0/1463413014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578032810 con 0x7f45900cf2f0 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.733+0000 7f458f200640 1 -- 172.21.15.19:0/1463413014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45900cf2f0 msgr2=0x7f45900cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.733+0000 7f458f200640 1 --2- 172.21.15.19:0/1463413014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45900cf2f0 0x7f45900cf700 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f45900cfed0 tx=0x7f457802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.734+0000 7f458f200640 1 -- 172.21.15.19:0/1463413014 shutdown_connections 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.734+0000 7f458f200640 1 --2- 172.21.15.19:0/1463413014 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45901649d0 0x7f4590164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.734+0000 7f458f200640 1 --2- 172.21.15.19:0/1463413014 >> v2:172.21.15.169:3300/0 conn(0x7f4590168010 0x7f4590168420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.734+0000 7f458f200640 1 --2- 172.21.15.19:0/1463413014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45900cf2f0 0x7f45900cf700 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.734+0000 7f458f200640 1 -- 172.21.15.19:0/1463413014 >> 172.21.15.19:0/1463413014 conn(0x7f45900d1720 msgr2=0x7f45900d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.734+0000 7f458f200640 1 -- 172.21.15.19:0/1463413014 shutdown_connections 2024-04-25T11:38:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.734+0000 7f458f200640 1 -- 172.21.15.19:0/1463413014 wait complete. 2024-04-25T11:38:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.736+0000 7f458f200640 1 Processor -- start 2024-04-25T11:38:22.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.736+0000 7f458f200640 1 -- start start 2024-04-25T11:38:22.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45900cf2f0 0x7f4590161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45901649d0 0x7f459014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590168010 0x7f45900b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4590070980 con 0x7f45901649d0 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45900706a0 con 0x7f4590168010 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4590168d00 con 0x7f45900cf2f0 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45901649d0 0x7f459014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45901649d0 0x7f459014d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53160/0 (socket says 172.21.15.19:53160) 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458d400640 1 -- 172.21.15.19:0/841430157 learned_addr learned my addr 172.21.15.19:0/841430157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.738+0000 7f458de00640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45900cf2f0 0x7f4590161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458d400640 1 -- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45900cf2f0 msgr2=0x7f4590161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458d400640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45900cf2f0 0x7f4590161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458d400640 1 -- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590168010 msgr2=0x7f45900b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458e800640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590168010 0x7f45900b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458d400640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590168010 0x7f45900b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458d400640 1 -- 172.21.15.19:0/841430157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4590161c60 con 0x7f45901649d0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458e800640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590168010 0x7f45900b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458d400640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45901649d0 0x7f459014d2d0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f458000ba80 tx=0x7f458000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f4586000640 1 -- 172.21.15.19:0/841430157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580019070 con 0x7f45901649d0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f4586000640 1 -- 172.21.15.19:0/841430157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458000cc20 con 0x7f45901649d0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f4586000640 1 -- 172.21.15.19:0/841430157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580014440 con 0x7f45901649d0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45901ee4e0 con 0x7f45901649d0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.739+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45901eeaa0 con 0x7f45901649d0 2024-04-25T11:38:22.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.741+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4590050740 con 0x7f45901649d0 2024-04-25T11:38:22.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.742+0000 7f4586000640 1 -- 172.21.15.19:0/841430157 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f458001c070 con 0x7f45901649d0 2024-04-25T11:38:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.743+0000 7f4586000640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f456c0744b0 0x7f456c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:22.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.743+0000 7f4586000640 1 -- 172.21.15.19:0/841430157 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f4580095590 con 0x7f45901649d0 2024-04-25T11:38:22.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.744+0000 7f458de00640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f456c0744b0 0x7f456c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:22.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.745+0000 7f458de00640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f456c0744b0 0x7f456c0769c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4578034000 tx=0x7f457802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:22.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.748+0000 7f4586000640 1 -- 172.21.15.19:0/841430157 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f458005f020 con 0x7f45901649d0 2024-04-25T11:38:22.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.997+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f459006fb60 con 0x7f45901649d0 2024-04-25T11:38:22.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:22.999+0000 7f4586000640 1 -- 172.21.15.19:0/841430157 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 210 v210) ==== 95+0+5187 (secure 0 0 0) 0x7f4580063320 con 0x7f45901649d0 2024-04-25T11:38:22.986 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:22.986 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":210,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":210,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:21.467485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9300":{"gid":9300,"name":"a","rank":0,"incarnation":209,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.19:6835/614168622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":614168622},{"type":"v1","addr":"172.21.15.19:6835","nonce":614168622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:22.986 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 210 2024-04-25T11:38:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f456c0744b0 msgr2=0x7f456c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f456c0744b0 0x7f456c0769c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4578034000 tx=0x7f457802e000 comp rx=0 tx=0).stop 2024-04-25T11:38:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45901649d0 msgr2=0x7f459014d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:22.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45901649d0 0x7f459014d2d0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f458000ba80 tx=0x7f458000bf40 comp rx=0 tx=0).stop 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 shutdown_connections 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f456c0744b0 0x7f456c0769c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590168010 0x7f45900b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45901649d0 0x7f459014d2d0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 --2- 172.21.15.19:0/841430157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45900cf2f0 0x7f4590161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 >> 172.21.15.19:0/841430157 conn(0x7f45900d1720 msgr2=0x7f4590167020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 shutdown_connections 2024-04-25T11:38:22.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:23.002+0000 7f458f200640 1 -- 172.21.15.19:0/841430157 wait complete. 2024-04-25T11:38:23.015 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 210, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:38:21.467485+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9300': {'gid': 9300, 'name': 'a', 'rank': 0, 'incarnation': 209, 'state': 'up:reconnect', 'state_seq': 3, 'addr': '172.21.15.19:6835/614168622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 614168622}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 614168622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:38:23.015 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9300:up:reconnect 2024-04-25T11:38:23.015 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:38:26.390 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:26.403+0000 7fdfc9200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:38:28.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:28.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.327+0000 7fb4f7800640 1 Processor -- start 2024-04-25T11:38:28.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.328+0000 7fb4f7800640 1 -- start start 2024-04-25T11:38:28.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00cf5e0 0x7fb4f00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f7800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4f006c5c0 0x7fb4f006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f7800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f00cff30 con 0x7fb4f01443c0 2024-04-25T11:38:28.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f7800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f00708d0 con 0x7fb4f006c5c0 2024-04-25T11:38:28.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f015f160 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f4e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb4f01443c0 0x7fb4f006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42050/0 (socket says 172.21.15.19:42050) 2024-04-25T11:38:28.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f5800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4f006c5c0 0x7fb4f006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.330+0000 7fb4f4e00640 1 -- 172.21.15.19:0/570564195 learned_addr learned my addr 172.21.15.19:0/570564195 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:28.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.331+0000 7fb4efe00640 1 --2- 172.21.15.19:0/570564195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00cf5e0 0x7fb4f00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.331+0000 7fb4f5800640 1 -- 172.21.15.19:0/570564195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00cf5e0 msgr2=0x7fb4f00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:28.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.331+0000 7fb4f5800640 1 --2- 172.21.15.19:0/570564195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00cf5e0 0x7fb4f00cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.331+0000 7fb4f5800640 1 -- 172.21.15.19:0/570564195 >> v1:172.21.15.19:6789/0 conn(0x7fb4f01443c0 legacy=0x7fb4f006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.331+0000 7fb4f5800640 1 -- 172.21.15.19:0/570564195 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f015f5f0 con 0x7fb4f006c5c0 2024-04-25T11:38:28.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.331+0000 7fb4efe00640 1 --2- 172.21.15.19:0/570564195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00cf5e0 0x7fb4f00cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:28.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.332+0000 7fb4f5800640 1 --2- 172.21.15.19:0/570564195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f006c5c0 0x7fb4f006c9d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80097a0 tx=0x7fb4e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418f2ded37e4d75a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.333+0000 7fb4ef400640 1 -- 172.21.15.19:0/570564195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037070 con 0x7fb4f006c5c0 2024-04-25T11:38:28.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.333+0000 7fb4ef400640 1 -- 172.21.15.19:0/570564195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e8032440 con 0x7fb4f006c5c0 2024-04-25T11:38:28.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.333+0000 7fb4ef400640 1 -- 172.21.15.19:0/570564195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8032810 con 0x7fb4f006c5c0 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.333+0000 7fb4f7800640 1 -- 172.21.15.19:0/570564195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f006c5c0 msgr2=0x7fb4f006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.333+0000 7fb4f7800640 1 --2- 172.21.15.19:0/570564195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f006c5c0 0x7fb4f006c9d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80097a0 tx=0x7fb4e802a180 comp rx=0 tx=0).stop 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.334+0000 7fb4f7800640 1 -- 172.21.15.19:0/570564195 shutdown_connections 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.334+0000 7fb4f7800640 1 --2- 172.21.15.19:0/570564195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f006c5c0 0x7fb4f006c9d0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.334+0000 7fb4f7800640 1 --2- 172.21.15.19:0/570564195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f00cf5e0 0x7fb4f00cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.334+0000 7fb4f7800640 1 -- 172.21.15.19:0/570564195 >> 172.21.15.19:0/570564195 conn(0x7fb4f00d1ae0 msgr2=0x7fb4f00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.334+0000 7fb4f7800640 1 -- 172.21.15.19:0/570564195 shutdown_connections 2024-04-25T11:38:28.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.334+0000 7fb4f7800640 1 -- 172.21.15.19:0/570564195 wait complete. 2024-04-25T11:38:28.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.336+0000 7fb4f7800640 1 Processor -- start 2024-04-25T11:38:28.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.336+0000 7fb4f7800640 1 -- start start 2024-04-25T11:38:28.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4f7800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f006c5c0 0x7fb4f01f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4f7800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f00cf5e0 0x7fb4f0160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4f7800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f01443c0 0x7fb4f0161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4f7800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f015edf0 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4f7800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f015eb10 con 0x7fb4f006c5c0 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4f7800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f01447d0 con 0x7fb4f01443c0 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f00cf5e0 0x7fb4f0160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f00cf5e0 0x7fb4f0160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36242/0 (socket says 172.21.15.19:36242) 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.338+0000 7fb4efe00640 1 -- 172.21.15.19:0/1278776844 learned_addr learned my addr 172.21.15.19:0/1278776844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4f5800640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f01443c0 0x7fb4f0161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4efe00640 1 -- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f01443c0 msgr2=0x7fb4f0161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4efe00640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f01443c0 0x7fb4f0161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4efe00640 1 -- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f006c5c0 msgr2=0x7fb4f01f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4efe00640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f006c5c0 0x7fb4f01f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4efe00640 1 -- 172.21.15.19:0/1278776844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f01f7590 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4efe00640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f00cf5e0 0x7fb4f0160b50 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800ce10 tx=0x7fb4d800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4ecc00640 1 -- 172.21.15.19:0/1278776844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800f680 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4ecc00640 1 -- 172.21.15.19:0/1278776844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d800f820 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f01e8c10 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.339+0000 7fb4ecc00640 1 -- 172.21.15.19:0/1278776844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800fc10 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.340+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f01f7d00 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.341+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f0143fb0 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.342+0000 7fb4ecc00640 1 -- 172.21.15.19:0/1278776844 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb4d80213f0 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.343+0000 7fb4ecc00640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4cc0744e0 0x7fb4cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.343+0000 7fb4ecc00640 1 -- 172.21.15.19:0/1278776844 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7fb4d8096680 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.344+0000 7fb4f4e00640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4cc0744e0 0x7fb4cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.345+0000 7fb4f4e00640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4cc0744e0 0x7fb4cc0769f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0000f60 tx=0x7fb4e0007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.348+0000 7fb4ecc00640 1 -- 172.21.15.19:0/1278776844 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4f0143fb0 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.603+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4f00d04b0 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.605+0000 7fb4ecc00640 1 -- 172.21.15.19:0/1278776844 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 213 v213) ==== 95+0+5191 (secure 0 0 0) 0x7fb4f00d04b0 con 0x7fb4f00cf5e0 2024-04-25T11:38:28.592 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:28.592 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":213,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":213,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:27.477999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9300":{"gid":9300,"name":"a","rank":0,"incarnation":209,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6835/614168622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":614168622},{"type":"v1","addr":"172.21.15.19:6835","nonce":614168622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9300,"qdb_cluster":[9300]},"id":6}]} 2024-04-25T11:38:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 213 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.607+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4cc0744e0 msgr2=0x7fb4cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.607+0000 7fb4f7800640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4cc0744e0 0x7fb4cc0769f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0000f60 tx=0x7fb4e0007a80 comp rx=0 tx=0).stop 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.607+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f00cf5e0 msgr2=0x7fb4f0160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.607+0000 7fb4f7800640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f00cf5e0 0x7fb4f0160b50 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800ce10 tx=0x7fb4d800a650 comp rx=0 tx=0).stop 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.607+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 shutdown_connections 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.607+0000 7fb4f7800640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4cc0744e0 0x7fb4cc0769f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.608+0000 7fb4f7800640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4f01443c0 0x7fb4f0161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.608+0000 7fb4f7800640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4f00cf5e0 0x7fb4f0160b50 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.608+0000 7fb4f7800640 1 --2- 172.21.15.19:0/1278776844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4f006c5c0 0x7fb4f01f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.608+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 >> 172.21.15.19:0/1278776844 conn(0x7fb4f00d1ae0 msgr2=0x7fb4f015c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:28.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.608+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 shutdown_connections 2024-04-25T11:38:28.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:28.608+0000 7fb4f7800640 1 -- 172.21.15.19:0/1278776844 wait complete. 2024-04-25T11:38:29.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.336+0000 7f2ecc000640 1 Processor -- start 2024-04-25T11:38:29.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.336+0000 7f2ecc000640 1 -- start start 2024-04-25T11:38:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.338+0000 7f2ecc000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ec414ad60 0x7f2ec4070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.338+0000 7f2ecc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 0x7f2ec40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.339+0000 7f2ecc000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec40b6790 con 0x7f2ec41681d0 2024-04-25T11:38:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.339+0000 7f2ecc000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec40b4af0 con 0x7f2ec414ad60 2024-04-25T11:38:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.339+0000 7f2ecc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec40b4dd0 con 0x7f2ec40b5e40 2024-04-25T11:38:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.339+0000 7f2eca000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2ec41681d0 0x7f2ec41685e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42066/0 (socket says 172.21.15.19:42066) 2024-04-25T11:38:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.339+0000 7f2eca000640 1 -- 172.21.15.19:0/2347252912 learned_addr learned my addr 172.21.15.19:0/2347252912 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:29.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.339+0000 7f2ec8c00640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 0x7f2ec40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec9600640 1 --2- 172.21.15.19:0/2347252912 >> v2:172.21.15.169:3300/0 conn(0x7f2ec414ad60 0x7f2ec4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759394776 0 0) 0x7f2ebc001490 con 0x7f2ec41681d0 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2eac003520 con 0x7f2ec41681d0 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4175488147 0 0) 0x7f2eac003520 con 0x7f2ec41681d0 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 msgr2=0x7f2ec40b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 0x7f2ec40b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 >> v2:172.21.15.169:3300/0 conn(0x7f2ec414ad60 msgr2=0x7f2ec4070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 --2- 172.21.15.19:0/2347252912 >> v2:172.21.15.169:3300/0 conn(0x7f2ec414ad60 0x7f2ec4070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec40b50b0 con 0x7f2ec41681d0 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2ebc001bc0 con 0x7f2ec41681d0 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec9600640 1 --2- 172.21.15.19:0/2347252912 >> v2:172.21.15.169:3300/0 conn(0x7f2ec414ad60 0x7f2ec4070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 >> v1:172.21.15.19:6789/0 conn(0x7f2ec41681d0 legacy=0x7f2ec41685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2eac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eac004800 0x7f2eac004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 0x7f2eac0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec40b4af0 con 0x7f2eac0039b0 2024-04-25T11:38:29.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec40b4dd0 con 0x7f2eac004800 2024-04-25T11:38:29.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.340+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2eac00bba0 con 0x7f2ec40b5e40 2024-04-25T11:38:29.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.341+0000 7f2ec9600640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2eac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.341+0000 7f2eca000640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 0x7f2eac0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.341+0000 7f2ec9600640 1 -- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 msgr2=0x7f2eac0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.341+0000 7f2ec9600640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec40b5e40 0x7f2eac0052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.341+0000 7f2ec9600640 1 -- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eac004800 msgr2=0x7f2eac004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.341+0000 7f2ec9600640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2eac004800 0x7f2eac004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.341+0000 7f2ec9600640 1 -- 172.21.15.19:0/2347252912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac003520 con 0x7f2eac0039b0 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.342+0000 7f2ec9600640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2eac003e20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40b1590 tx=0x7f2eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e169cf864de4280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.342+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8037070 con 0x7f2eac0039b0 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.342+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb802ad00 con 0x7f2eac0039b0 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.342+0000 7f2ec3e00640 1 -- 172.21.15.19:0/2347252912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800f040 con 0x7f2eac0039b0 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.343+0000 7f2ecc000640 1 -- 172.21.15.19:0/2347252912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 msgr2=0x7f2eac003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.343+0000 7f2ecc000640 1 --2- 172.21.15.19:0/2347252912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2eac003e20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40b1590 tx=0x7f2eac005830 comp rx=0 tx=0).stop 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.343+0000 7f2eca000640 1 -- 172.21.15.19:0/2347252912 reap_dead start 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.343+0000 7f2ecc000640 1 -- 172.21.15.19:0/2347252912 shutdown_connections 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.343+0000 7f2ecc000640 1 -- 172.21.15.19:0/2347252912 >> 172.21.15.19:0/2347252912 conn(0x7f2ec40d1ae0 msgr2=0x7f2ec40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.343+0000 7f2ecc000640 1 -- 172.21.15.19:0/2347252912 shutdown_connections 2024-04-25T11:38:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.343+0000 7f2ecc000640 1 -- 172.21.15.19:0/2347252912 wait complete. 2024-04-25T11:38:29.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.346+0000 7f2ecc000640 1 Processor -- start 2024-04-25T11:38:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.346+0000 7f2ecc000640 1 -- start start 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ecc000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2ec4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ecc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec41480d0 0x7f2ec40c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ecc000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ec41484e0 0x7f2ec40c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ecc000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec4070550 con 0x7f2eac0039b0 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ecc000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec40b5670 con 0x7f2ec41484e0 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ecc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec40706d0 con 0x7f2ec41480d0 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2ec4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2ec4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36254/0 (socket says 172.21.15.19:36254) 2024-04-25T11:38:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 -- 172.21.15.19:0/3660469396 learned_addr learned my addr 172.21.15.19:0/3660469396 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:29.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 -- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec41480d0 msgr2=0x7f2ec40c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec41480d0 0x7f2ec40c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 -- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ec41484e0 msgr2=0x7f2ec40c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ec41484e0 0x7f2ec40c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.348+0000 7f2ec9600640 1 -- 172.21.15.19:0/3660469396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec40c75b0 con 0x7f2eac0039b0 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.349+0000 7f2ec9600640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2ec4157a00 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2eb802a7f0 tx=0x7f2eb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.349+0000 7f2ec1600640 1 -- 172.21.15.19:0/3660469396 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8040020 con 0x7f2eac0039b0 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.349+0000 7f2ec1600640 1 -- 172.21.15.19:0/3660469396 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb8038040 con 0x7f2eac0039b0 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.349+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec4146950 con 0x7f2eac0039b0 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.349+0000 7f2ec1600640 1 -- 172.21.15.19:0/3660469396 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8037070 con 0x7f2eac0039b0 2024-04-25T11:38:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.349+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec40c7850 con 0x7f2eac0039b0 2024-04-25T11:38:29.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.351+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec4050740 con 0x7f2eac0039b0 2024-04-25T11:38:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.352+0000 7f2ec1600640 1 -- 172.21.15.19:0/3660469396 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2eb8043020 con 0x7f2eac0039b0 2024-04-25T11:38:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.353+0000 7f2ec1600640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ea40744e0 0x7f2ea40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.353+0000 7f2ec1600640 1 -- 172.21.15.19:0/3660469396 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f2eb80b59a0 con 0x7f2eac0039b0 2024-04-25T11:38:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.354+0000 7f2ec8c00640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ea40744e0 0x7f2ea40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.355+0000 7f2ec8c00640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ea40744e0 0x7f2ea40769f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006f00 tx=0x7f2eb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.358+0000 7f2ec1600640 1 -- 172.21.15.19:0/3660469396 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2eb804a030 con 0x7f2eac0039b0 2024-04-25T11:38:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.608+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec4146e70 con 0x7f2eac0039b0 2024-04-25T11:38:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.609+0000 7f2ec1600640 1 -- 172.21.15.19:0/3660469396 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 213 v213) ==== 95+0+5191 (secure 0 0 0) 0x7f2eb807fa80 con 0x7f2eac0039b0 2024-04-25T11:38:29.596 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:29.597 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":213,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":213,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:27.477999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9300":{"gid":9300,"name":"a","rank":0,"incarnation":209,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6835/614168622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":614168622},{"type":"v1","addr":"172.21.15.19:6835","nonce":614168622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9300,"qdb_cluster":[9300]},"id":6}]} 2024-04-25T11:38:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 213 2024-04-25T11:38:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ea40744e0 msgr2=0x7f2ea40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ea40744e0 0x7f2ea40769f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006f00 tx=0x7f2eb0008040 comp rx=0 tx=0).stop 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 msgr2=0x7f2ec4157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2ec4157a00 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2eb802a7f0 tx=0x7f2eb8004730 comp rx=0 tx=0).stop 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 shutdown_connections 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ea40744e0 0x7f2ea40769f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ec41484e0 0x7f2ec40c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ec41480d0 0x7f2ec40c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 --2- 172.21.15.19:0/3660469396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eac0039b0 0x7f2ec4157a00 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 >> 172.21.15.19:0/3660469396 conn(0x7f2ec40d1ae0 msgr2=0x7f2ec4161d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 shutdown_connections 2024-04-25T11:38:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.612+0000 7f2ecc000640 1 -- 172.21.15.19:0/3660469396 wait complete. 2024-04-25T11:38:29.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 213, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:38:27.477999+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9300': {'gid': 9300, 'name': 'a', 'rank': 0, 'incarnation': 209, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.19:6835/614168622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 614168622}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 614168622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9300, 'qdb_cluster': [9300]} 2024-04-25T11:38:29.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:38:29.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:38:29.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.912+0000 7f30ed800640 1 Processor -- start 2024-04-25T11:38:29.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.912+0000 7f30ed800640 1 -- start start 2024-04-25T11:38:29.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.915+0000 7f30ed800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80b63a0 0x7f30e81647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.915+0000 7f30ed800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e806c7d0 con 0x7f30e8160360 2024-04-25T11:38:29.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.915+0000 7f30ed800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e806cab0 con 0x7f30e8070680 2024-04-25T11:38:29.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.915+0000 7f30ed800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30e806d100 con 0x7f30e80b63a0 2024-04-25T11:38:29.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.915+0000 7f30de000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f30e8160360 0x7f30e80b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42078/0 (socket says 172.21.15.19:42078) 2024-04-25T11:38:29.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.915+0000 7f30de000640 1 -- 172.21.15.19:0/3515077587 learned_addr learned my addr 172.21.15.19:0/3515077587 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3998095073 0 0) 0x7f30e806c7d0 con 0x7f30e8160360 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e7400640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80b63a0 0x7f30e81647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30cc003520 con 0x7f30e8160360 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2357343726 0 0) 0x7f30cc003520 con 0x7f30e8160360 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80b63a0 msgr2=0x7f30e81647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80b63a0 0x7f30e81647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 >> v1:172.21.15.169:6789/0 conn(0x7f30e8070680 legacy=0x7f30e80af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e806cd90 con 0x7f30e8160360 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f30d00019f0 con 0x7f30e8160360 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 >> v1:172.21.15.19:6789/0 conn(0x7f30e8160360 legacy=0x7f30e80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc0039e0 0x7f30cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.916+0000 7f30e6000640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30cc004800 0x7f30cc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6000640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc005270 0x7f30cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30e806d100 con 0x7f30cc0039e0 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc00c110 con 0x7f30cc005270 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30cc00c310 con 0x7f30cc004800 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc0039e0 0x7f30cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30de000640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30cc004800 0x7f30cc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e7400640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc005270 0x7f30cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6a00640 1 -- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30cc004800 msgr2=0x7f30cc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30cc004800 0x7f30cc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6a00640 1 -- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc005270 msgr2=0x7f30cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc005270 0x7f30cc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.917+0000 7f30e6a00640 1 -- 172.21.15.19:0/3515077587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc003520 con 0x7f30cc0039e0 2024-04-25T11:38:29.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.918+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc0039e0 0x7f30cc003e50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f30d8001720 tx=0x7f30cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d577281fb15771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.919+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8001370 con 0x7f30cc0039e0 2024-04-25T11:38:29.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.919+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d80052d0 con 0x7f30cc0039e0 2024-04-25T11:38:29.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.919+0000 7f30e6000640 1 -- 172.21.15.19:0/3515077587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d802abc0 con 0x7f30cc0039e0 2024-04-25T11:38:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.919+0000 7f30ed800640 1 -- 172.21.15.19:0/3515077587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc0039e0 msgr2=0x7f30cc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.919+0000 7f30ed800640 1 --2- 172.21.15.19:0/3515077587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30cc0039e0 0x7f30cc003e50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f30d8001720 tx=0x7f30cc005830 comp rx=0 tx=0).stop 2024-04-25T11:38:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.919+0000 7f30e7400640 1 -- 172.21.15.19:0/3515077587 reap_dead start 2024-04-25T11:38:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.920+0000 7f30ed800640 1 -- 172.21.15.19:0/3515077587 shutdown_connections 2024-04-25T11:38:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.920+0000 7f30ed800640 1 -- 172.21.15.19:0/3515077587 >> 172.21.15.19:0/3515077587 conn(0x7f30e80d1ae0 msgr2=0x7f30e80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.920+0000 7f30ed800640 1 -- 172.21.15.19:0/3515077587 shutdown_connections 2024-04-25T11:38:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.920+0000 7f30ed800640 1 -- 172.21.15.19:0/3515077587 wait complete. 2024-04-25T11:38:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.922+0000 7f30ed800640 1 Processor -- start 2024-04-25T11:38:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.922+0000 7f30ed800640 1 -- start start 2024-04-25T11:38:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.924+0000 7f30ed800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc0039e0 0x7f30e814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.924+0000 7f30ed800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80c7390 0x7f30e8161af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.924+0000 7f30ed800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e81f98f0 0x7f30e81f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.924+0000 7f30ed800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30e80afd30 con 0x7f30e81f98f0 2024-04-25T11:38:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.924+0000 7f30ed800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30e80afbb0 con 0x7f30cc0039e0 2024-04-25T11:38:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.924+0000 7f30ed800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30e80b63a0 con 0x7f30e80c7390 2024-04-25T11:38:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.924+0000 7f30e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e81f98f0 0x7f30e81f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e81f98f0 0x7f30e81f9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36284/0 (socket says 172.21.15.19:36284) 2024-04-25T11:38:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 -- 172.21.15.19:0/3004026773 learned_addr learned my addr 172.21.15.19:0/3004026773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30de000640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80c7390 0x7f30e8161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc0039e0 0x7f30e814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 -- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80c7390 msgr2=0x7f30e8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80c7390 0x7f30e8161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 -- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc0039e0 msgr2=0x7f30e814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc0039e0 0x7f30e814acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 -- 172.21.15.19:0/3004026773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e81fa240 con 0x7f30e81f98f0 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30de000640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80c7390 0x7f30e8161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.925+0000 7f30e7400640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e81f98f0 0x7f30e81f9d00 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f30e0006fd0 tx=0x7f30e000e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.926+0000 7f30df400640 1 -- 172.21.15.19:0/3004026773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000f040 con 0x7f30e81f98f0 2024-04-25T11:38:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.926+0000 7f30df400640 1 -- 172.21.15.19:0/3004026773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e0019070 con 0x7f30e81f98f0 2024-04-25T11:38:29.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.926+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e8160a50 con 0x7f30e81f98f0 2024-04-25T11:38:29.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.926+0000 7f30df400640 1 -- 172.21.15.19:0/3004026773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0013850 con 0x7f30e81f98f0 2024-04-25T11:38:29.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.926+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e8161010 con 0x7f30e81f98f0 2024-04-25T11:38:29.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.928+0000 7f30df400640 1 -- 172.21.15.19:0/3004026773 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f30e00139f0 con 0x7f30e81f98f0 2024-04-25T11:38:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.930+0000 7f30df400640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744b0 0x7f30c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.930+0000 7f30df400640 1 -- 172.21.15.19:0/3004026773 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f30e00951c0 con 0x7f30e81f98f0 2024-04-25T11:38:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.931+0000 7f30f479f740 1 Processor -- start 2024-04-25T11:38:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.931+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744b0 0x7f30c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.931+0000 7f30f479f740 1 -- start start 2024-04-25T11:38:29.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.932+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744b0 0x7f30c40769c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f30d8032000 tx=0x7f30d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30f479f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b6b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b85d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda577d590 0x55dda5b862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30f479f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dda5aa4600 con 0x55dda579e270 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dda5a41100 con 0x55dda57753b0 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55dda5aa05c0 con 0x55dda577d590 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b6b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b6b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36288/0 (socket says 172.21.15.19:36288) 2024-04-25T11:38:29.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30e6a00640 1 -- 172.21.15.19:0/2616372164 learned_addr learned my addr 172.21.15.19:0/2616372164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30e6a00640 1 -- 172.21.15.19:0/2616372164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda577d590 msgr2=0x55dda5b862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30e6a00640 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda577d590 0x55dda5b862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30e6a00640 1 -- 172.21.15.19:0/2616372164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 msgr2=0x55dda5b85d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30de000640 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b85d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.933+0000 7f30e6a00640 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b85d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.934+0000 7f30e6a00640 1 -- 172.21.15.19:0/2616372164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda57b62d0 con 0x55dda579e270 2024-04-25T11:38:29.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.934+0000 7f30e6a00640 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b6b040 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f30d8001810 tx=0x7f30d802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc91ae1e2e3910ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.934+0000 7f30d7400640 1 -- 172.21.15.19:0/2616372164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d804d070 con 0x55dda579e270 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.934+0000 7f30d7400640 1 -- 172.21.15.19:0/2616372164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d80052f0 con 0x55dda579e270 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.934+0000 7f30d7400640 1 -- 172.21.15.19:0/2616372164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80069d0 con 0x55dda579e270 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 -- 172.21.15.19:0/2616372164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 msgr2=0x55dda5b6b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b6b040 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f30d8001810 tx=0x7f30d802c000 comp rx=0 tx=0).stop 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 -- 172.21.15.19:0/2616372164 shutdown_connections 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b85d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda577d590 0x55dda5b862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 --2- 172.21.15.19:0/2616372164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b6b040 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 -- 172.21.15.19:0/2616372164 >> 172.21.15.19:0/2616372164 conn(0x55dda57923c0 msgr2=0x55dda5b406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 -- 172.21.15.19:0/2616372164 shutdown_connections 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.935+0000 7f30f479f740 1 -- 172.21.15.19:0/2616372164 wait complete. 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.936+0000 7f30f479f740 1 -- start start 2024-04-25T11:38:29.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.937+0000 7f30f479f740 1 Processor -- start 2024-04-25T11:38:29.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b7f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30f479f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b6d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 0x55dda5b6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30f479f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dda5a2bf70 con 0x55dda577d590 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dda5b7eee0 con 0x55dda57753b0 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55dda5b80ae0 con 0x55dda579e270 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30f479f740 10 client.0 _ll_register_callbacks cb 0x55dda5a9a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.938+0000 7f30de000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b6d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b6d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36294/0 (socket says 172.21.15.19:36294) 2024-04-25T11:38:29.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 -- 172.21.15.19:0/3638954745 learned_addr learned my addr 172.21.15.19:0/3638954745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b7f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30e7400640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 0x55dda5b6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 -- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 msgr2=0x55dda5b6d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 0x55dda5b6d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 -- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 msgr2=0x55dda5b7f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b7f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 -- 172.21.15.19:0/3638954745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda58806b0 con 0x55dda577d590 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b7f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30de000640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b6d390 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f30d0007300 tx=0x7f30d00077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76127514934116c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.939+0000 7f30cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30cbe00640 1 -- 172.21.15.19:0/3638954745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0018020 con 0x55dda577d590 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30cbe00640 1 -- 172.21.15.19:0/3638954745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f30d00109c0 con 0x55dda577d590 2024-04-25T11:38:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d80052f0 con 0x55dda577d590 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30cbe00640 1 -- 172.21.15.19:0/3638954745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=213) ==== 24+0+0 (secure 0 0 0) 0x7f30d0010bc0 con 0x55dda577d590 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30f479f740 10 client.9393 fetch_fsmap learned FSMap version 213 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=213,monmap=2+}) -- 0x55dda5b82a20 con 0x55dda577d590 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30cbe00640 1 -- 172.21.15.19:0/3638954745 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d80052f0 con 0x55dda577d590 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30cbe00640 1 -- 172.21.15.19:0/3638954745 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 213) ==== 3494+0+0 (secure 0 0 0) 0x7f30d0014400 con 0x55dda577d590 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30f479f740 10 client.9393 fetch_fsmap finished waiting for FSMap version 213 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.940+0000 7f30f479f740 10 client.9393 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.941+0000 7f30f479f740 20 client.9393 populate_metadata read hostname 'smithi019' 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.941+0000 7f30f479f740 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda575ce40 0x55dda5b98e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.941+0000 7f30f479f740 4 client.9393 mds_command: new command op to 9300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.941+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 --> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dda5ab2210 con 0x55dda575ce40 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.941+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda575ce40 0x55dda5b98e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.941+0000 7f30e6a00640 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda575ce40 0x55dda5b98e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe13eec424f5690f server_cookie=937eb297f805cac6 in_seq=0 out_seq=0 2024-04-25T11:38:29.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.941+0000 7f30cbe00640 10 client.9393 ms_handle_connect on v2:172.21.15.19:6834/614168622 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.942+0000 7f30cbe00640 1 -- 172.21.15.19:0/3638954745 <== mds.0 v2:172.21.15.19:6834/614168622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dda5ab2210 con 0x55dda575ce40 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.942+0000 7f30cbe00640 10 client.9393 handle_command_reply: tid=0 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.942+0000 7f30f479f740 1 client.9393 shutdown 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 msgr2=0x55dda5b6d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b6d390 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f30d0007300 tx=0x7f30d00077c0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 shutdown_connections 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda575ce40 0x55dda5b98e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 0x55dda5b6d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b6d390 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 --2- 172.21.15.19:0/3638954745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5b7f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30cbe00640 0 client.9393 ms_handle_reset on v2:172.21.15.19:6834/614168622 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 >> 172.21.15.19:0/3638954745 conn(0x55dda57923c0 msgr2=0x55dda5b406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 shutdown_connections 2024-04-25T11:38:29.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 1 -- 172.21.15.19:0/3638954745 wait complete. 2024-04-25T11:38:29.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.943+0000 7f30f479f740 20 client.9393 trim_cache size 0 max 16384 2024-04-25T11:38:29.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.964+0000 7f30f479f740 1 Processor -- start 2024-04-25T11:38:29.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.964+0000 7f30f479f740 1 -- start start 2024-04-25T11:38:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.965+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda57753b0 0x55dda5b9f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.965+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda577d590 0x55dda5b97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.965+0000 7f30f479f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b97cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.965+0000 7f30f479f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30e8202f50 con 0x55dda579e270 2024-04-25T11:38:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.965+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30e814e270 con 0x55dda577d590 2024-04-25T11:38:29.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.965+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30e8202900 con 0x55dda57753b0 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b97cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b97cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36302/0 (socket says 172.21.15.19:36302) 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 -- 172.21.15.19:0/1082390605 learned_addr learned my addr 172.21.15.19:0/1082390605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 -- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda57753b0 msgr2=0x55dda5b9f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e6a00640 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda57753b0 0x55dda5b9f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30de000640 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda577d590 0x55dda5b97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda57753b0 0x55dda5b9f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 -- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda577d590 msgr2=0x55dda5b97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda577d590 0x55dda5b97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e7400640 1 -- 172.21.15.19:0/1082390605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda5b82580 con 0x55dda579e270 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30e6a00640 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda57753b0 0x55dda5b9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.966+0000 7f30de000640 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda577d590 0x55dda5b97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.967+0000 7f30e7400640 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b97cc0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f30e000edb0 tx=0x7f30e005ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dc5cc35d2c8fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.967+0000 7f30d6a00640 1 -- 172.21.15.19:0/1082390605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e005f2a0 con 0x55dda579e270 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.967+0000 7f30d6a00640 1 -- 172.21.15.19:0/1082390605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e005eba0 con 0x55dda579e270 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.967+0000 7f30d6a00640 1 -- 172.21.15.19:0/1082390605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e005e000 con 0x55dda579e270 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 -- 172.21.15.19:0/1082390605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 msgr2=0x55dda5b97cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b97cc0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f30e000edb0 tx=0x7f30e005ea80 comp rx=0 tx=0).stop 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 -- 172.21.15.19:0/1082390605 shutdown_connections 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda577d590 0x55dda5b97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda579e270 0x55dda5b97cc0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 --2- 172.21.15.19:0/1082390605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda57753b0 0x55dda5b9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 -- 172.21.15.19:0/1082390605 >> 172.21.15.19:0/1082390605 conn(0x55dda57923c0 msgr2=0x55dda5b7d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 -- 172.21.15.19:0/1082390605 shutdown_connections 2024-04-25T11:38:29.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.968+0000 7f30f479f740 1 -- 172.21.15.19:0/1082390605 wait complete. 2024-04-25T11:38:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.969+0000 7f30f479f740 1 -- start start 2024-04-25T11:38:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.969+0000 7f30f479f740 1 Processor -- start 2024-04-25T11:38:29.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.970+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5c03e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.970+0000 7f30f479f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b3a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.970+0000 7f30f479f740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 0x55dda5b7b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30f479f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dda5b7e070 con 0x55dda577d590 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55dda5b80810 con 0x55dda57753b0 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30f479f740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55dda5b7e6c0 con 0x55dda579e270 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30f479f740 10 client.0 _ll_register_callbacks cb 0x7f30d00012f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b3a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b3a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36306/0 (socket says 172.21.15.19:36306) 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 -- 172.21.15.19:0/494206981 learned_addr learned my addr 172.21.15.19:0/494206981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30e6a00640 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5c03e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 -- 172.21.15.19:0/494206981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 msgr2=0x55dda5b7b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 0x55dda5b7b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 -- 172.21.15.19:0/494206981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 msgr2=0x55dda5c03e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5c03e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.971+0000 7f30de000640 1 -- 172.21.15.19:0/494206981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda5b40ab0 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30de000640 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b3a9a0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x55dda5b6f190 tx=0x7f30d0009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3f095b4c039a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30cbe00640 1 -- 172.21.15.19:0/494206981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0018070 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30cbe00640 1 -- 172.21.15.19:0/494206981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f30d00109c0 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e005eba0 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30cbe00640 1 -- 172.21.15.19:0/494206981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=213) ==== 24+0+0 (secure 0 0 0) 0x7f30d0014bb0 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30f479f740 10 client.9399 fetch_fsmap learned FSMap version 213 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=213,monmap=2+}) -- 0x55dda5c044d0 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30cbe00640 1 -- 172.21.15.19:0/494206981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e005eba0 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.972+0000 7f30cbe00640 1 -- 172.21.15.19:0/494206981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 213) ==== 3494+0+0 (secure 0 0 0) 0x7f30d002e430 con 0x55dda577d590 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30f479f740 10 client.9399 fetch_fsmap finished waiting for FSMap version 213 2024-04-25T11:38:29.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30f479f740 10 client.9399 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30f479f740 20 client.9399 populate_metadata read hostname 'smithi019' 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30f479f740 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda5b6c200 0x55dda5b6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30f479f740 4 client.9399 mds_command: new command op to 9300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 --> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] -- command(tid 0: {"prefix": "status"}) -- 0x55dda5aac080 con 0x55dda5b6c200 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30e6a00640 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda5b6c200 0x55dda5b6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30e6a00640 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda5b6c200 0x55dda5b6c610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b6abe880c76bb2 server_cookie=d3010c0e077c0ef9 in_seq=0 out_seq=0 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.973+0000 7f30cbe00640 10 client.9399 ms_handle_connect on v2:172.21.15.19:6834/614168622 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30cbe00640 1 -- 172.21.15.19:0/494206981 <== mds.0 v2:172.21.15.19:6834/614168622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55dda5aac080 con 0x55dda5b6c200 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30cbe00640 10 client.9399 handle_command_reply: tid=0 2024-04-25T11:38:29.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 client.9399 shutdown 2024-04-25T11:38:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 msgr2=0x55dda5b3a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b3a9a0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x55dda5b6f190 tx=0x7f30d0009050 comp rx=0 tx=0).stop 2024-04-25T11:38:29.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 shutdown_connections 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:6834/614168622,v1:172.21.15.19:6835/614168622] conn(0x55dda5b6c200 0x55dda5b6c610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55dda579e270 0x55dda5b7b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dda577d590 0x55dda5b3a9a0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 --2- 172.21.15.19:0/494206981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55dda57753b0 0x55dda5c03e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 >> 172.21.15.19:0/494206981 conn(0x55dda57923c0 msgr2=0x55dda5ba9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30cbe00640 0 client.9399 ms_handle_reset on v2:172.21.15.19:6834/614168622 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 shutdown_connections 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 1 -- 172.21.15.19:0/494206981 wait complete. 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.974+0000 7f30f479f740 20 client.9399 trim_cache size 0 max 16384 2024-04-25T11:38:29.965 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 9300, 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 8.5201030109999998, 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 213, 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 110, 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 110, 2024-04-25T11:38:29.966 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 14.448782205000001 2024-04-25T11:38:29.967 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744b0 msgr2=0x7f30c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744b0 0x7f30c40769c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f30d8032000 tx=0x7f30d802e000 comp rx=0 tx=0).stop 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e81f98f0 msgr2=0x7f30e81f9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e81f98f0 0x7f30e81f9d00 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f30e0006fd0 tx=0x7f30e000e8b0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 shutdown_connections 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744b0 0x7f30c40769c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e81f98f0 0x7f30e81f9d00 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e80c7390 0x7f30e8161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 --2- 172.21.15.19:0/3004026773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30cc0039e0 0x7f30e814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 >> 172.21.15.19:0/3004026773 conn(0x7f30e80d1ae0 msgr2=0x7f30e80b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 shutdown_connections 2024-04-25T11:38:29.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:29.975+0000 7f30ed800640 1 -- 172.21.15.19:0/3004026773 wait complete. 2024-04-25T11:38:29.987 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:38:29.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:30.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.274+0000 7f496f200640 1 Processor -- start 2024-04-25T11:38:30.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.274+0000 7f496f200640 1 -- start start 2024-04-25T11:38:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49700b6e10 0x7f497015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49701608d0 0x7f4970160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4970161220 con 0x7f49700cf2f0 2024-04-25T11:38:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f497006f870 con 0x7f49701608d0 2024-04-25T11:38:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f49700b7220 con 0x7f49700b6e10 2024-04-25T11:38:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49701608d0 0x7f4970160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f49700cf2f0 0x7f49700b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42080/0 (socket says 172.21.15.19:42080) 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.277+0000 7f496de00640 1 -- 172.21.15.19:0/545132271 learned_addr learned my addr 172.21.15.19:0/545132271 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.278+0000 7f4967e00640 1 --2- 172.21.15.19:0/545132271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49700b6e10 0x7f497015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.278+0000 7f496d400640 1 -- 172.21.15.19:0/545132271 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004706680 0 0) 0x7f4960001110 con 0x7f49700cf2f0 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.278+0000 7f496d400640 1 -- 172.21.15.19:0/545132271 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4954003520 con 0x7f49700cf2f0 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.278+0000 7f496e800640 1 -- 172.21.15.19:0/545132271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49700b6e10 msgr2=0x7f497015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.278+0000 7f496e800640 1 --2- 172.21.15.19:0/545132271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49700b6e10 0x7f497015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.278+0000 7f496e800640 1 -- 172.21.15.19:0/545132271 >> v1:172.21.15.19:6789/0 conn(0x7f49700cf2f0 legacy=0x7f49700b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.278+0000 7f496e800640 1 -- 172.21.15.19:0/545132271 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49700b74d0 con 0x7f49701608d0 2024-04-25T11:38:30.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.279+0000 7f496e800640 1 --2- 172.21.15.19:0/545132271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49701608d0 0x7f4970160ce0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4968009d90 tx=0x7f496802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530af7b69def187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.279+0000 7f496d400640 1 -- 172.21.15.19:0/545132271 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968007740 con 0x7f49701608d0 2024-04-25T11:38:30.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.279+0000 7f496d400640 1 -- 172.21.15.19:0/545132271 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49680078e0 con 0x7f49701608d0 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.279+0000 7f496d400640 1 -- 172.21.15.19:0/545132271 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968007c90 con 0x7f49701608d0 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.280+0000 7f496f200640 1 -- 172.21.15.19:0/545132271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49701608d0 msgr2=0x7f4970160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.280+0000 7f496f200640 1 --2- 172.21.15.19:0/545132271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49701608d0 0x7f4970160ce0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f4968009d90 tx=0x7f496802a660 comp rx=0 tx=0).stop 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.280+0000 7f496f200640 1 -- 172.21.15.19:0/545132271 shutdown_connections 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.280+0000 7f496f200640 1 --2- 172.21.15.19:0/545132271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49701608d0 0x7f4970160ce0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.280+0000 7f496f200640 1 --2- 172.21.15.19:0/545132271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49700b6e10 0x7f497015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.280+0000 7f496f200640 1 -- 172.21.15.19:0/545132271 >> 172.21.15.19:0/545132271 conn(0x7f49700d1720 msgr2=0x7f49700d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.280+0000 7f496f200640 1 -- 172.21.15.19:0/545132271 shutdown_connections 2024-04-25T11:38:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.281+0000 7f496f200640 1 -- 172.21.15.19:0/545132271 wait complete. 2024-04-25T11:38:30.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.283+0000 7f496f200640 1 Processor -- start 2024-04-25T11:38:30.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.283+0000 7f496f200640 1 -- start start 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.284+0000 7f496f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49700b6e10 0x7f4970157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.284+0000 7f496f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49700cf2f0 0x7f497014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.284+0000 7f496f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49701608d0 0x7f49700cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.284+0000 7f496f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f497006f500 con 0x7f49700cf2f0 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.284+0000 7f496f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f497006f220 con 0x7f49700b6e10 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.284+0000 7f496f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f497006fd00 con 0x7f49701608d0 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49700cf2f0 0x7f497014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49700cf2f0 0x7f497014a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36316/0 (socket says 172.21.15.19:36316) 2024-04-25T11:38:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 -- 172.21.15.19:0/3406232581 learned_addr learned my addr 172.21.15.19:0/3406232581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 -- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49701608d0 msgr2=0x7f49700cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f496e800640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49701608d0 0x7f49700cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49701608d0 0x7f49700cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 -- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49700b6e10 msgr2=0x7f4970157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49700b6e10 0x7f4970157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.285+0000 7f4967e00640 1 -- 172.21.15.19:0/3406232581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4970157e40 con 0x7f49700cf2f0 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.286+0000 7f4967e00640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49700cf2f0 0x7f497014a120 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f495800d620 tx=0x7f495800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.286+0000 7f4966000640 1 -- 172.21.15.19:0/3406232581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800b620 con 0x7f49700cf2f0 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.286+0000 7f4966000640 1 -- 172.21.15.19:0/3406232581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4958019070 con 0x7f49700cf2f0 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.286+0000 7f4966000640 1 -- 172.21.15.19:0/3406232581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800ba00 con 0x7f49700cf2f0 2024-04-25T11:38:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.286+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4970156580 con 0x7f49700cf2f0 2024-04-25T11:38:30.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.286+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4970156b40 con 0x7f49700cf2f0 2024-04-25T11:38:30.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.287+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49701562f0 con 0x7f49700cf2f0 2024-04-25T11:38:30.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.288+0000 7f4966000640 1 -- 172.21.15.19:0/3406232581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f495800bba0 con 0x7f49700cf2f0 2024-04-25T11:38:30.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.290+0000 7f4966000640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f494c0744b0 0x7f494c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.290+0000 7f4966000640 1 -- 172.21.15.19:0/3406232581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f4958095d80 con 0x7f49700cf2f0 2024-04-25T11:38:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.291+0000 7f496de00640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f494c0744b0 0x7f494c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.292+0000 7f496de00640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f494c0744b0 0x7f494c0769c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4960000f60 tx=0x7f4960001110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.295+0000 7f4966000640 1 -- 172.21.15.19:0/3406232581 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f495805fe60 con 0x7f49700cf2f0 2024-04-25T11:38:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.540+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4970050740 con 0x7f49700cf2f0 2024-04-25T11:38:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.542+0000 7f4966000640 1 -- 172.21.15.19:0/3406232581 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 213 v213) ==== 95+0+5191 (secure 0 0 0) 0x7f495805f760 con 0x7f49700cf2f0 2024-04-25T11:38:30.529 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:30.529 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":213,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":213,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:27.477999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9300":{"gid":9300,"name":"a","rank":0,"incarnation":209,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6835/614168622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":614168622},{"type":"v1","addr":"172.21.15.19:6835","nonce":614168622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9300,"qdb_cluster":[9300]},"id":6}]} 2024-04-25T11:38:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 213 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f494c0744b0 msgr2=0x7f494c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f494c0744b0 0x7f494c0769c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4960000f60 tx=0x7f4960001110 comp rx=0 tx=0).stop 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49700cf2f0 msgr2=0x7f497014a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49700cf2f0 0x7f497014a120 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f495800d620 tx=0x7f495800dae0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 shutdown_connections 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f494c0744b0 0x7f494c0769c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f49701608d0 0x7f49700cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49700cf2f0 0x7f497014a120 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 --2- 172.21.15.19:0/3406232581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49700b6e10 0x7f4970157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 >> 172.21.15.19:0/3406232581 conn(0x7f49700d1720 msgr2=0x7f49700d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 shutdown_connections 2024-04-25T11:38:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.545+0000 7f496f200640 1 -- 172.21.15.19:0/3406232581 wait complete. 2024-04-25T11:38:30.560 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2024-04-25T11:38:30.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:38:30.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.853+0000 7f461b200640 1 Processor -- start 2024-04-25T11:38:30.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.854+0000 7f461b200640 1 -- start start 2024-04-25T11:38:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.856+0000 7f461b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 0x7f461c168e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.856+0000 7f461b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f461c141210 con 0x7f461c140e00 2024-04-25T11:38:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.856+0000 7f461b200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f461c141510 con 0x7f461c06d5d0 2024-04-25T11:38:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.856+0000 7f461b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f461c141b40 con 0x7f461c142fe0 2024-04-25T11:38:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.856+0000 7f4619e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f461c140e00 0x7f461c143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42086/0 (socket says 172.21.15.19:42086) 2024-04-25T11:38:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.856+0000 7f4619e00640 1 -- 172.21.15.19:0/798777085 learned_addr learned my addr 172.21.15.19:0/798777085 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f461a800640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 0x7f461c168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300683184 0 0) 0x7f461c141210 con 0x7f461c140e00 2024-04-25T11:38:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4600003520 con 0x7f461c140e00 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3476967245 0 0) 0x7f4600003520 con 0x7f461c140e00 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 msgr2=0x7f461c168e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f4618a00640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 0x7f461c168e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 >> v1:172.21.15.169:6789/0 conn(0x7f461c06d5d0 legacy=0x7f461c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.857+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c1440f0 con 0x7f461c140e00 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f460c001670 con 0x7f461c140e00 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 >> v1:172.21.15.19:6789/0 conn(0x7f461c140e00 legacy=0x7f461c143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 0x7f4600003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004080 0x7f4600004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4600004db0 0x7f4600005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f461c141b40 con 0x7f4600004db0 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4600010080 con 0x7f4600004080 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4600010280 con 0x7f461c142fe0 2024-04-25T11:38:30.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f461a800640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4600004db0 0x7f4600005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4619e00640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 0x7f4600003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f4619400640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004080 0x7f4600004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f461a800640 1 -- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 msgr2=0x7f4600003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.858+0000 7f461a800640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c142fe0 0x7f4600003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.859+0000 7f461a800640 1 -- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004080 msgr2=0x7f4600004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.859+0000 7f461a800640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004080 0x7f4600004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.859+0000 7f461a800640 1 -- 172.21.15.19:0/798777085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4600003520 con 0x7f4600004db0 2024-04-25T11:38:30.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.859+0000 7f4619400640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004080 0x7f4600004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.860+0000 7f461a800640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4600004db0 0x7f4600005280 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4614007360 tx=0x7f4600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907e1e3e4df11953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.860+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614034020 con 0x7f4600004db0 2024-04-25T11:38:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.860+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4614030680 con 0x7f4600004db0 2024-04-25T11:38:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.860+0000 7f4618a00640 1 -- 172.21.15.19:0/798777085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614009be0 con 0x7f4600004db0 2024-04-25T11:38:30.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.861+0000 7f461b200640 1 -- 172.21.15.19:0/798777085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4600004db0 msgr2=0x7f4600005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.861+0000 7f461b200640 1 --2- 172.21.15.19:0/798777085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4600004db0 0x7f4600005280 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4614007360 tx=0x7f4600005830 comp rx=0 tx=0).stop 2024-04-25T11:38:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.861+0000 7f461a800640 1 -- 172.21.15.19:0/798777085 reap_dead start 2024-04-25T11:38:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.861+0000 7f461b200640 1 -- 172.21.15.19:0/798777085 shutdown_connections 2024-04-25T11:38:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.861+0000 7f461b200640 1 -- 172.21.15.19:0/798777085 >> 172.21.15.19:0/798777085 conn(0x7f461c072ee0 msgr2=0x7f461c0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.861+0000 7f461b200640 1 -- 172.21.15.19:0/798777085 shutdown_connections 2024-04-25T11:38:30.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.861+0000 7f461b200640 1 -- 172.21.15.19:0/798777085 wait complete. 2024-04-25T11:38:30.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.864+0000 7f461b200640 1 Processor -- start 2024-04-25T11:38:30.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.864+0000 7f461b200640 1 -- start start 2024-04-25T11:38:30.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.866+0000 7f461b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004db0 0x7f461c1450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.866+0000 7f461b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461c1578b0 0x7f461c13c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.866+0000 7f461b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c146ac0 0x7f461c146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.866+0000 7f461b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f461c0d1490 con 0x7f461c1578b0 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.866+0000 7f461b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f461c1417d0 con 0x7f4600004db0 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.866+0000 7f461b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f461c143c60 con 0x7f461c146ac0 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461c1578b0 0x7f461c13c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461c1578b0 0x7f461c13c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36326/0 (socket says 172.21.15.19:36326) 2024-04-25T11:38:30.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 -- 172.21.15.19:0/4118506678 learned_addr learned my addr 172.21.15.19:0/4118506678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:30.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619e00640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004db0 0x7f461c1450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f461a800640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c146ac0 0x7f461c146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 -- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c146ac0 msgr2=0x7f461c146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c146ac0 0x7f461c146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 -- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004db0 msgr2=0x7f461c1450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004db0 0x7f461c1450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f4619400640 1 -- 172.21.15.19:0/4118506678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c147410 con 0x7f461c1578b0 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.867+0000 7f461a800640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c146ac0 0x7f461c146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.868+0000 7f4619400640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461c1578b0 0x7f461c13c440 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f4608000f60 tx=0x7f460800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.868+0000 7f4612000640 1 -- 172.21.15.19:0/4118506678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608005480 con 0x7f461c1578b0 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.868+0000 7f4612000640 1 -- 172.21.15.19:0/4118506678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4608017070 con 0x7f461c1578b0 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.868+0000 7f4612000640 1 -- 172.21.15.19:0/4118506678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608003810 con 0x7f461c1578b0 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.868+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461c1f20e0 con 0x7f461c1578b0 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.868+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461c145d30 con 0x7f461c1578b0 2024-04-25T11:38:30.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.870+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461c050740 con 0x7f461c1578b0 2024-04-25T11:38:30.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.871+0000 7f4612000640 1 -- 172.21.15.19:0/4118506678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46080039b0 con 0x7f461c1578b0 2024-04-25T11:38:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.873+0000 7f4612000640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45fc0744b0 0x7f45fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.873+0000 7f4612000640 1 -- 172.21.15.19:0/4118506678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f4608095710 con 0x7f461c1578b0 2024-04-25T11:38:30.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.874+0000 7f4619e00640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45fc0744b0 0x7f45fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.875+0000 7f4619e00640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45fc0744b0 0x7f45fc0769c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f460c001050 tx=0x7f460c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:30.878+0000 7f4612000640 1 -- 172.21.15.19:0/4118506678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f461c147410 con 0x7f461c1578b0 2024-04-25T11:38:31.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.122+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f461c0b7710 con 0x7f461c1578b0 2024-04-25T11:38:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.123+0000 7f4612000640 1 -- 172.21.15.19:0/4118506678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v213) ==== 53+0+73 (secure 0 0 0) 0x7f460805f880 con 0x7f461c1578b0 2024-04-25T11:38:31.111 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:38:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45fc0744b0 msgr2=0x7f45fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45fc0744b0 0x7f45fc0769c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f460c001050 tx=0x7f460c009040 comp rx=0 tx=0).stop 2024-04-25T11:38:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461c1578b0 msgr2=0x7f461c13c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461c1578b0 0x7f461c13c440 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f4608000f60 tx=0x7f460800d9f0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 shutdown_connections 2024-04-25T11:38:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45fc0744b0 0x7f45fc0769c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f461c146ac0 0x7f461c146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f461c1578b0 0x7f461c13c440 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 --2- 172.21.15.19:0/4118506678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4600004db0 0x7f461c1450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 >> 172.21.15.19:0/4118506678 conn(0x7f461c072ee0 msgr2=0x7f461c06fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 shutdown_connections 2024-04-25T11:38:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31.126+0000 7f461b200640 1 -- 172.21.15.19:0/4118506678 wait complete. 2024-04-25T11:38:31.140 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:38:31.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:38:31.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:38:31.140 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:38:31.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:38:31.141 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:38:31.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:38:31.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:38:31.141 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:38:31.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:31.141 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:38:31.193 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:38:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:38:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:38:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:38:31.193 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41962sec preferred_lft 41962sec 2024-04-25T11:38:31.194 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:38:31.195 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:38:31.196 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:38:31.196 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:38:31.196 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:38:31.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:31.197 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:38:31.276 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:38:31.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:31.280 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:38:31.334 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:38:31.334 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:38:31.334 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:38:31.334 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:38:31.335 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:38:31.335 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:38:31.335 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:38:31.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:38:31.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:38:31.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:38:31.336 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:38:31.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:38:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:38:31.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:31.557 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:38:31.608 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:38:31.608 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:38:31.609 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:38:31.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:31.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:38:31.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:38:31.694 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:38:31.752 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:38:31.752 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:38:31.752 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:38:31.807 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:38:31.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:31.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:38:32.019 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:38:32.019 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:38:32.063 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:38:32.063 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:38:32.063 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:38:32.063 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:38:32.063 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:38:32.065 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:38:32.065 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:38:32.065 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:38:32.065 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:38:32.065 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:38:32.070 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:38:32.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:32.070 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:38:32.144 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:38:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:38:32.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:38:32.261 INFO:teuthology.orchestra.run.smithi019.stdout:client7084 2024-04-25T11:38:32.262 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:32.262 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:32.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:32.315 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:38:32.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:32.322 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2024-04-25T11:38:32.372 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:38:32.373 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:38:32.373 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:32.373 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:38:32.373 INFO:teuthology.orchestra.run.smithi019.stdout:sixmegs 2024-04-25T11:38:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:mkdir: cannot create directory ‘foo’: Read-only file system 2024-04-25T11:38:32.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:32.410 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory ‘foo’: Read-only file system 2024-04-25T11:38:32.410 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2024-04-25T11:38:32.410 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:32.410 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:32.507 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:32.508 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:32.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.859+0000 7f0d28800640 1 Processor -- start 2024-04-25T11:38:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.859+0000 7f0d28800640 1 -- start start 2024-04-25T11:38:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.861+0000 7f0d28800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.861+0000 7f0d28800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d2006c5c0 0x7f0d2006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.861+0000 7f0d28800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d200cff30 con 0x7f0d201443c0 2024-04-25T11:38:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.861+0000 7f0d28800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0d200708d0 con 0x7f0d2006c5c0 2024-04-25T11:38:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.861+0000 7f0d28800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2015f160 con 0x7f0d200cf5e0 2024-04-25T11:38:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.862+0000 7f0d26800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d2006c5c0 0x7f0d2006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.862+0000 7f0d25400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.862+0000 7f0d25400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:34024/0 (socket says 172.21.15.19:34024) 2024-04-25T11:38:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.862+0000 7f0d26800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0d2006c5c0 0x7f0d2006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:47106/0 (socket says 172.21.15.19:47106) 2024-04-25T11:38:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.862+0000 7f0d25400640 1 -- 172.21.15.19:0/3997342578 learned_addr learned my addr 172.21.15.19:0/3997342578 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.862+0000 7f0d25e00640 1 --1- 172.21.15.19:0/3997342578 >> v1:172.21.15.169:6789/0 conn(0x7f0d201443c0 0x7f0d2006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:47542/0 (socket says 172.21.15.19:47542) 2024-04-25T11:38:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.863+0000 7f0d25400640 1 -- 172.21.15.19:0/3997342578 >> v2:172.21.15.169:3300/0 conn(0x7f0d2006c5c0 msgr2=0x7f0d2006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.863+0000 7f0d25400640 1 --2- 172.21.15.19:0/3997342578 >> v2:172.21.15.169:3300/0 conn(0x7f0d2006c5c0 0x7f0d2006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.863+0000 7f0d25400640 1 -- 172.21.15.19:0/3997342578 >> v1:172.21.15.169:6789/0 conn(0x7f0d201443c0 legacy=0x7f0d2006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.863+0000 7f0d25400640 1 -- 172.21.15.19:0/3997342578 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2015f5f0 con 0x7f0d200cf5e0 2024-04-25T11:38:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.864+0000 7f0d25400640 1 --2- 172.21.15.19:0/3997342578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200cf9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009870 tx=0x7f0d1002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a87a95c16e9dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.865+0000 7f0d24a00640 1 -- 172.21.15.19:0/3997342578 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10037070 con 0x7f0d200cf5e0 2024-04-25T11:38:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.865+0000 7f0d24a00640 1 -- 172.21.15.19:0/3997342578 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d10032440 con 0x7f0d200cf5e0 2024-04-25T11:38:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.865+0000 7f0d24a00640 1 -- 172.21.15.19:0/3997342578 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10032810 con 0x7f0d200cf5e0 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 -- 172.21.15.19:0/3997342578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 msgr2=0x7f0d200cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 --2- 172.21.15.19:0/3997342578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200cf9f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009870 tx=0x7f0d1002a250 comp rx=0 tx=0).stop 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 -- 172.21.15.19:0/3997342578 shutdown_connections 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 --2- 172.21.15.19:0/3997342578 >> v2:172.21.15.169:3300/0 conn(0x7f0d2006c5c0 0x7f0d2006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 --2- 172.21.15.19:0/3997342578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200cf9f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 -- 172.21.15.19:0/3997342578 >> 172.21.15.19:0/3997342578 conn(0x7f0d200d1ae0 msgr2=0x7f0d200d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 -- 172.21.15.19:0/3997342578 shutdown_connections 2024-04-25T11:38:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.866+0000 7f0d28800640 1 -- 172.21.15.19:0/3997342578 wait complete. 2024-04-25T11:38:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.868+0000 7f0d28800640 1 Processor -- start 2024-04-25T11:38:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.869+0000 7f0d28800640 1 -- start start 2024-04-25T11:38:32.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.870+0000 7f0d28800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006c5c0 0x7f0d2013fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.870+0000 7f0d28800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.870+0000 7f0d28800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d201443c0 0x7f0d2015d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.870+0000 7f0d28800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2015edf0 con 0x7f0d2006c5c0 2024-04-25T11:38:32.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.870+0000 7f0d28800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2015eb10 con 0x7f0d201443c0 2024-04-25T11:38:32.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.870+0000 7f0d28800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0d201447d0 con 0x7f0d200cf5e0 2024-04-25T11:38:32.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006c5c0 0x7f0d2013fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006c5c0 0x7f0d2013fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36338/0 (socket says 172.21.15.19:36338) 2024-04-25T11:38:32.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 -- 172.21.15.19:0/2281495430 learned_addr learned my addr 172.21.15.19:0/2281495430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:32.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25400640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d26800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d201443c0 0x7f0d2015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 -- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 msgr2=0x7f0d200b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 -- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d201443c0 msgr2=0x7f0d2015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d201443c0 0x7f0d2015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d25e00640 1 -- 172.21.15.19:0/2281495430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d201f28b0 con 0x7f0d2006c5c0 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.871+0000 7f0d26800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d201443c0 0x7f0d2015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.872+0000 7f0d25e00640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006c5c0 0x7f0d2013fc50 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c6e0 tx=0x7f0d1800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.872+0000 7f0d16000640 1 -- 172.21.15.19:0/2281495430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18018070 con 0x7f0d2006c5c0 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.872+0000 7f0d16000640 1 -- 172.21.15.19:0/2281495430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18004d60 con 0x7f0d2006c5c0 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.872+0000 7f0d16000640 1 -- 172.21.15.19:0/2281495430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800a880 con 0x7f0d2006c5c0 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.872+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d201f3780 con 0x7f0d2006c5c0 2024-04-25T11:38:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.872+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201f2e50 con 0x7f0d2006c5c0 2024-04-25T11:38:32.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.873+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d20162b00 con 0x7f0d2006c5c0 2024-04-25T11:38:32.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.875+0000 7f0d16000640 1 -- 172.21.15.19:0/2281495430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0d1800aa20 con 0x7f0d2006c5c0 2024-04-25T11:38:32.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.875+0000 7f0d16000640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d040744e0 0x7f0d040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:32.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.876+0000 7f0d16000640 1 -- 172.21.15.19:0/2281495430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f0d18094d60 con 0x7f0d2006c5c0 2024-04-25T11:38:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.876+0000 7f0d25400640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d040744e0 0x7f0d040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.878+0000 7f0d25400640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d040744e0 0x7f0d040769f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:32.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:32.880+0000 7f0d16000640 1 -- 172.21.15.19:0/2281495430 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d20162b00 con 0x7f0d2006c5c0 2024-04-25T11:38:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.087+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d20050740 con 0x7f0d2006c5c0 2024-04-25T11:38:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.091+0000 7f0d16000640 1 -- 172.21.15.19:0/2281495430 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+18998 (secure 0 0 0) 0x7f0d200cff30 con 0x7f0d2006c5c0 2024-04-25T11:38:33.078 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:33.078 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":110,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:38:15.379212+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d040744e0 msgr2=0x7f0d040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d040744e0 0x7f0d040769f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d1002e000 comp rx=0 tx=0).stop 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006c5c0 msgr2=0x7f0d2013fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006c5c0 0x7f0d2013fc50 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c6e0 tx=0x7f0d1800cba0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 shutdown_connections 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0d040744e0 0x7f0d040769f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d201443c0 0x7f0d2015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d200cf5e0 0x7f0d200b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.093+0000 7f0d28800640 1 --2- 172.21.15.19:0/2281495430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006c5c0 0x7f0d2013fc50 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.094+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 >> 172.21.15.19:0/2281495430 conn(0x7f0d200d1ae0 msgr2=0x7f0d20161160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.094+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 shutdown_connections 2024-04-25T11:38:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.094+0000 7f0d28800640 1 -- 172.21.15.19:0/2281495430 wait complete. 2024-04-25T11:38:33.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:38:33.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:38:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.413+0000 7f720dc00640 1 Processor -- start 2024-04-25T11:38:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.413+0000 7f720dc00640 1 -- start start 2024-04-25T11:38:33.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.415+0000 7f720dc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f720806dce0 0x7f720815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.415+0000 7f720dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815ec90 0x7f720815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.415+0000 7f720dc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7208070000 con 0x7f7208144340 2024-04-25T11:38:33.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.415+0000 7f720dc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7208144750 con 0x7f720806dce0 2024-04-25T11:38:33.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.415+0000 7f720dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f720806f240 con 0x7f720815ec90 2024-04-25T11:38:33.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.415+0000 7f7207e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7208144340 0x7f720806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42100/0 (socket says 172.21.15.19:42100) 2024-04-25T11:38:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.415+0000 7f7207e00640 1 -- 172.21.15.19:0/4104872880 learned_addr learned my addr 172.21.15.19:0/4104872880 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f720d200640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815ec90 0x7f720815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7207400640 1 --2- 172.21.15.19:0/4104872880 >> v2:172.21.15.169:3300/0 conn(0x7f720806dce0 0x7f720815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999495632 0 0) 0x7f7208070000 con 0x7f7208144340 2024-04-25T11:38:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71ec003520 con 0x7f7208144340 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1195732685 0 0) 0x7f71ec003520 con 0x7f7208144340 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815ec90 msgr2=0x7f720815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815ec90 0x7f720815f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 >> v2:172.21.15.169:3300/0 conn(0x7f720806dce0 msgr2=0x7f720815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 --2- 172.21.15.19:0/4104872880 >> v2:172.21.15.169:3300/0 conn(0x7f720806dce0 0x7f720815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720806f520 con 0x7f7208144340 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f71f8001670 con 0x7f7208144340 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 >> v1:172.21.15.19:6789/0 conn(0x7f7208144340 legacy=0x7f720806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f71ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71ec004800 0x7f71ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815ec90 0x7f71ec005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7208144750 con 0x7f71ec0039b0 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f720806f240 con 0x7f71ec004800 2024-04-25T11:38:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.416+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec00b9f0 con 0x7f720815ec90 2024-04-25T11:38:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.417+0000 7f7207e00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f71ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.417+0000 7f7207e00640 1 -- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815ec90 msgr2=0x7f71ec005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.417+0000 7f7207400640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71ec004800 0x7f71ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.417+0000 7f7207e00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f720815ec90 0x7f71ec005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.417+0000 7f7207e00640 1 -- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71ec004800 msgr2=0x7f71ec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.417+0000 7f7207e00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71ec004800 0x7f71ec004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.417+0000 7f7207e00640 1 -- 172.21.15.19:0/4104872880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec003520 con 0x7f71ec0039b0 2024-04-25T11:38:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.418+0000 7f7207e00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f71ec003e20 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c160 tx=0x7f71ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee36086b81f1380e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.419+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007c80 con 0x7f71ec0039b0 2024-04-25T11:38:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.419+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8005ad0 con 0x7f71ec0039b0 2024-04-25T11:38:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.419+0000 7f7206a00640 1 -- 172.21.15.19:0/4104872880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007070 con 0x7f71ec0039b0 2024-04-25T11:38:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.419+0000 7f720dc00640 1 -- 172.21.15.19:0/4104872880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 msgr2=0x7f71ec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.419+0000 7f720dc00640 1 --2- 172.21.15.19:0/4104872880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f71ec003e20 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c160 tx=0x7f71ec005830 comp rx=0 tx=0).stop 2024-04-25T11:38:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.419+0000 7f720d200640 1 -- 172.21.15.19:0/4104872880 reap_dead start 2024-04-25T11:38:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.420+0000 7f720dc00640 1 -- 172.21.15.19:0/4104872880 shutdown_connections 2024-04-25T11:38:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.420+0000 7f720dc00640 1 -- 172.21.15.19:0/4104872880 >> 172.21.15.19:0/4104872880 conn(0x7f7208072f40 msgr2=0x7f7208073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.420+0000 7f720dc00640 1 -- 172.21.15.19:0/4104872880 shutdown_connections 2024-04-25T11:38:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.420+0000 7f720dc00640 1 -- 172.21.15.19:0/4104872880 wait complete. 2024-04-25T11:38:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.422+0000 7f720dc00640 1 Processor -- start 2024-04-25T11:38:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.423+0000 7f720dc00640 1 -- start start 2024-04-25T11:38:33.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.424+0000 7f720dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f72080ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.424+0000 7f720dc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7208158b70 0x7f720815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.424+0000 7f720dc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720815dac0 0x7f720815fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.424+0000 7f720dc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f720815f0a0 con 0x7f71ec0039b0 2024-04-25T11:38:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.424+0000 7f720dc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f720806fae0 con 0x7f720815dac0 2024-04-25T11:38:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.424+0000 7f720dc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f720815f220 con 0x7f7208158b70 2024-04-25T11:38:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.424+0000 7f7207e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f72080ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f72080ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36360/0 (socket says 172.21.15.19:36360) 2024-04-25T11:38:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 -- 172.21.15.19:0/4122180871 learned_addr learned my addr 172.21.15.19:0/4122180871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207400640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7208158b70 0x7f720815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f720d200640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720815dac0 0x7f720815fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 -- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7208158b70 msgr2=0x7f720815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7208158b70 0x7f720815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 -- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720815dac0 msgr2=0x7f720815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720815dac0 0x7f720815fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 -- 172.21.15.19:0/4122180871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720815ded0 con 0x7f71ec0039b0 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207400640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7208158b70 0x7f720815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.425+0000 7f7207e00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f72080ae640 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c160 tx=0x7f720806f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.426+0000 7f71ffe00640 1 -- 172.21.15.19:0/4122180871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8001630 con 0x7f71ec0039b0 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.426+0000 7f71ffe00640 1 -- 172.21.15.19:0/4122180871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8042070 con 0x7f71ec0039b0 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.426+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72080b0630 con 0x7f71ec0039b0 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.426+0000 7f71ffe00640 1 -- 172.21.15.19:0/4122180871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80072b0 con 0x7f71ec0039b0 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.426+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72080b0bf0 con 0x7f71ec0039b0 2024-04-25T11:38:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.427+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7208076c60 con 0x7f71ec0039b0 2024-04-25T11:38:33.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.429+0000 7f71ffe00640 1 -- 172.21.15.19:0/4122180871 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71f8007450 con 0x7f71ec0039b0 2024-04-25T11:38:33.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.430+0000 7f71ffe00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71e80744b0 0x7f71e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.431+0000 7f71ffe00640 1 -- 172.21.15.19:0/4122180871 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f71f80b58d0 con 0x7f71ec0039b0 2024-04-25T11:38:33.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.431+0000 7f7207400640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71e80744b0 0x7f71e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.432+0000 7f7207400640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71e80744b0 0x7f71e80769c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009160 tx=0x7f71f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:33.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.436+0000 7f71ffe00640 1 -- 172.21.15.19:0/4122180871 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f720815ded0 con 0x7f71ec0039b0 2024-04-25T11:38:33.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.644+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72080b7920 con 0x7f71ec0039b0 2024-04-25T11:38:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.645+0000 7f71ffe00640 1 -- 172.21.15.19:0/4122180871 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f71f8005aa0 con 0x7f71ec0039b0 2024-04-25T11:38:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:38:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.648+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71e80744b0 msgr2=0x7f71e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.648+0000 7f720dc00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71e80744b0 0x7f71e80769c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009160 tx=0x7f71f4008040 comp rx=0 tx=0).stop 2024-04-25T11:38:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.648+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 msgr2=0x7f72080ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.648+0000 7f720dc00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f72080ae640 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c160 tx=0x7f720806f6f0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 shutdown_connections 2024-04-25T11:38:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71e80744b0 0x7f71e80769c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f720815dac0 0x7f720815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7208158b70 0x7f720815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 --2- 172.21.15.19:0/4122180871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71ec0039b0 0x7f72080ae640 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 >> 172.21.15.19:0/4122180871 conn(0x7f7208072f40 msgr2=0x7f7208143430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 shutdown_connections 2024-04-25T11:38:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.649+0000 7f720dc00640 1 -- 172.21.15.19:0/4122180871 wait complete. 2024-04-25T11:38:33.666 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:38:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.968+0000 7fbd1ca00640 1 Processor -- start 2024-04-25T11:38:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.968+0000 7fbd1ca00640 1 -- start start 2024-04-25T11:38:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.970+0000 7fbd1ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbd181460a0 0x7fbd180ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.970+0000 7fbd1ca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd18070190 con 0x7fbd18145710 2024-04-25T11:38:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.971+0000 7fbd1ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd18070470 con 0x7fbd180ce610 2024-04-25T11:38:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.971+0000 7fbd1ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbd18143c20 con 0x7fbd181460a0 2024-04-25T11:38:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.971+0000 7fbd17200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbd18145710 0x7fbd18166a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60126/0 (socket says 172.21.15.19:60126) 2024-04-25T11:38:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.971+0000 7fbd17200640 1 -- 172.21.15.19:0/2867850919 learned_addr learned my addr 172.21.15.19:0/2867850919 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.971+0000 7fbd16800640 1 --2- 172.21.15.19:0/2867850919 >> v2:172.21.15.169:3300/0 conn(0x7fbd181460a0 0x7fbd180ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.971+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931538186 0 0) 0x7fbd18070190 con 0x7fbd18145710 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.971+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcfc003520 con 0x7fbd18145710 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3107394552 0 0) 0x7fbcfc003520 con 0x7fbd18145710 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 >> v2:172.21.15.169:3300/0 conn(0x7fbd181460a0 msgr2=0x7fbd180ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 --2- 172.21.15.19:0/2867850919 >> v2:172.21.15.169:3300/0 conn(0x7fbd181460a0 0x7fbd180ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 >> v1:172.21.15.169:6789/0 conn(0x7fbd180ce610 legacy=0x7fbd180cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd18143f00 con 0x7fbd18145710 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fbd100019f0 con 0x7fbd18145710 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 >> v1:172.21.15.19:6789/0 conn(0x7fbd18145710 legacy=0x7fbd18166a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcfc003a00 0x7fbcfc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcfc004800 0x7fbcfc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbcfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd18143c20 con 0x7fbcfc005270 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc00c190 con 0x7fbcfc003a00 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc00e5a0 con 0x7fbcfc004800 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.972+0000 7fbd17200640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbcfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.973+0000 7fbd16800640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcfc003a00 0x7fbcfc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.973+0000 7fbd0fe00640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcfc004800 0x7fbcfc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.973+0000 7fbd17200640 1 -- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcfc004800 msgr2=0x7fbcfc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.973+0000 7fbd17200640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbcfc004800 0x7fbcfc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.973+0000 7fbd17200640 1 -- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcfc003a00 msgr2=0x7fbcfc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.973+0000 7fbd17200640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbcfc003a00 0x7fbcfc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.973+0000 7fbd17200640 1 -- 172.21.15.19:0/2867850919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc003520 con 0x7fbcfc005270 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.974+0000 7fbd17200640 1 -- 172.21.15.19:0/2867850919 reap_dead start 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.974+0000 7fbd17200640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbcfc00b9c0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000a790 tx=0x7fbcfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ee4940fa612fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.974+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000a3b0 con 0x7fbcfc005270 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.974+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1002f070 con 0x7fbcfc005270 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.975+0000 7fbd15e00640 1 -- 172.21.15.19:0/2867850919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1002bc20 con 0x7fbcfc005270 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.975+0000 7fbd1ca00640 1 -- 172.21.15.19:0/2867850919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 msgr2=0x7fbcfc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.975+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbcfc00b9c0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000a790 tx=0x7fbcfc005830 comp rx=0 tx=0).stop 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.975+0000 7fbd1ca00640 1 -- 172.21.15.19:0/2867850919 shutdown_connections 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.975+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/2867850919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbcfc00b9c0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.975+0000 7fbd1ca00640 1 -- 172.21.15.19:0/2867850919 >> 172.21.15.19:0/2867850919 conn(0x7fbd18072f40 msgr2=0x7fbd18073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.975+0000 7fbd1ca00640 1 -- 172.21.15.19:0/2867850919 shutdown_connections 2024-04-25T11:38:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.976+0000 7fbd1ca00640 1 -- 172.21.15.19:0/2867850919 wait complete. 2024-04-25T11:38:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.978+0000 7fbd1ca00640 1 Processor -- start 2024-04-25T11:38:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.978+0000 7fbd1ca00640 1 -- start start 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd1ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbd1815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd1ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd180c6ed0 0x7fbd181f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd1ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd180c72e0 0x7fbd181f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd1ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180ce250 con 0x7fbcfc005270 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd1ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180ce0d0 con 0x7fbd180c72e0 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd1ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd180ce3d0 con 0x7fbd180c6ed0 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbd1815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbd1815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33018/0 (socket says 172.21.15.19:33018) 2024-04-25T11:38:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 -- 172.21.15.19:0/3642583285 learned_addr learned my addr 172.21.15.19:0/3642583285 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 -- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd180c6ed0 msgr2=0x7fbd181f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd180c6ed0 0x7fbd181f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 -- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd180c72e0 msgr2=0x7fbd181f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd180c72e0 0x7fbd181f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.980+0000 7fbd16800640 1 -- 172.21.15.19:0/3642583285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd181e54d0 con 0x7fbcfc005270 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.981+0000 7fbd16800640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbd1815c380 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fbd08011570 tx=0x7fbd08011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.981+0000 7fbd0ea00640 1 -- 172.21.15.19:0/3642583285 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08015930 con 0x7fbcfc005270 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.981+0000 7fbd0ea00640 1 -- 172.21.15.19:0/3642583285 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd08015ad0 con 0x7fbcfc005270 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.981+0000 7fbd0ea00640 1 -- 172.21.15.19:0/3642583285 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0800f460 con 0x7fbcfc005270 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.981+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd181e57b0 con 0x7fbcfc005270 2024-04-25T11:38:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.981+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181f2260 con 0x7fbcfc005270 2024-04-25T11:38:33.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.983+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd18076c60 con 0x7fbcfc005270 2024-04-25T11:38:33.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.984+0000 7fbd0ea00640 1 -- 172.21.15.19:0/3642583285 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd0800f600 con 0x7fbcfc005270 2024-04-25T11:38:33.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.985+0000 7fbd0ea00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbcf4074420 0x7fbcf4076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:33.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.985+0000 7fbd0ea00640 1 -- 172.21.15.19:0/3642583285 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7fbd08096230 con 0x7fbcfc005270 2024-04-25T11:38:33.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.986+0000 7fbd0fe00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbcf4074420 0x7fbcf4076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:33.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.987+0000 7fbd0fe00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbcf4074420 0x7fbcf4076930 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007fd0 tx=0x7fbd00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:33.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:33.990+0000 7fbd0ea00640 1 -- 172.21.15.19:0/3642583285 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd08019030 con 0x7fbcfc005270 2024-04-25T11:38:34.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.208+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd1806d750 con 0x7fbcfc005270 2024-04-25T11:38:34.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.209+0000 7fbd0ea00640 1 -- 172.21.15.19:0/3642583285 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7fbd08060310 con 0x7fbcfc005270 2024-04-25T11:38:34.197 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:38:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbcf4074420 msgr2=0x7fbcf4076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbcf4074420 0x7fbcf4076930 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007fd0 tx=0x7fbd00009040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 msgr2=0x7fbd1815c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbd1815c380 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fbd08011570 tx=0x7fbd08011a30 comp rx=0 tx=0).stop 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 shutdown_connections 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbcf4074420 0x7fbcf4076930 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd180c72e0 0x7fbd181f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd180c6ed0 0x7fbd181f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 --2- 172.21.15.19:0/3642583285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcfc005270 0x7fbd1815c380 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 >> 172.21.15.19:0/3642583285 conn(0x7fbd18072f40 msgr2=0x7fbd18142910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.212+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 shutdown_connections 2024-04-25T11:38:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.213+0000 7fbd1ca00640 1 -- 172.21.15.19:0/3642583285 wait complete. 2024-04-25T11:38:34.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:38:34.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.535+0000 7ff815800640 1 Processor -- start 2024-04-25T11:38:34.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.535+0000 7ff815800640 1 -- start start 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff815800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff810070d20 0x7ff810161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff815800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff81006f930 0x7ff81006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff815800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8101623f0 con 0x7ff81006fd40 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff815800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff8101626d0 con 0x7ff81006f930 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff815800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff81006f130 con 0x7ff810070d20 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff80f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff81006f930 0x7ff81006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff814e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff81006fd40 0x7ff8100ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60138/0 (socket says 172.21.15.19:60138) 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff80f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff81006f930 0x7ff81006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33028/0 (socket says 172.21.15.19:33028) 2024-04-25T11:38:34.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff814e00640 1 -- 172.21.15.19:0/1195493785 learned_addr learned my addr 172.21.15.19:0/1195493785 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:34.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.538+0000 7ff80fe00640 1 --2- 172.21.15.19:0/1195493785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff810070d20 0x7ff810161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.539+0000 7ff80f400640 1 -- 172.21.15.19:0/1195493785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff810070d20 msgr2=0x7ff810161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:34.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.539+0000 7ff80f400640 1 --2- 172.21.15.19:0/1195493785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff810070d20 0x7ff810161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.539+0000 7ff80f400640 1 -- 172.21.15.19:0/1195493785 >> v1:172.21.15.19:6789/0 conn(0x7ff81006fd40 legacy=0x7ff8100ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.539+0000 7ff80ea00640 1 -- 172.21.15.19:0/1195493785 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3586778096 0 0) 0x7ff8101623f0 con 0x7ff81006fd40 2024-04-25T11:38:34.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.539+0000 7ff80f400640 1 -- 172.21.15.19:0/1195493785 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81006f410 con 0x7ff81006f930 2024-04-25T11:38:34.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.540+0000 7ff80f400640 1 --2- 172.21.15.19:0/1195493785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81006f930 0x7ff81006d5d0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80097a0 tx=0x7ff7f802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31144f1d945bacab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.540+0000 7ff80ea00640 1 -- 172.21.15.19:0/1195493785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8037070 con 0x7ff81006f930 2024-04-25T11:38:34.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.540+0000 7ff80ea00640 1 -- 172.21.15.19:0/1195493785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f802ae50 con 0x7ff81006f930 2024-04-25T11:38:34.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.540+0000 7ff80ea00640 1 -- 172.21.15.19:0/1195493785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8032660 con 0x7ff81006f930 2024-04-25T11:38:34.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 -- 172.21.15.19:0/1195493785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81006f930 msgr2=0x7ff81006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 --2- 172.21.15.19:0/1195493785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81006f930 0x7ff81006d5d0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80097a0 tx=0x7ff7f802a100 comp rx=0 tx=0).stop 2024-04-25T11:38:34.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 -- 172.21.15.19:0/1195493785 shutdown_connections 2024-04-25T11:38:34.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 --2- 172.21.15.19:0/1195493785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff81006f930 0x7ff81006d5d0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 --2- 172.21.15.19:0/1195493785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff810070d20 0x7ff810161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 -- 172.21.15.19:0/1195493785 >> 172.21.15.19:0/1195493785 conn(0x7ff810073160 msgr2=0x7ff810073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 -- 172.21.15.19:0/1195493785 shutdown_connections 2024-04-25T11:38:34.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.541+0000 7ff815800640 1 -- 172.21.15.19:0/1195493785 wait complete. 2024-04-25T11:38:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.544+0000 7ff815800640 1 Processor -- start 2024-04-25T11:38:34.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.544+0000 7ff815800640 1 -- start start 2024-04-25T11:38:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.545+0000 7ff815800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff81006f930 0x7ff8101521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.545+0000 7ff815800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81006fd40 0x7ff810153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.545+0000 7ff815800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810070d20 0x7ff8101e9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.545+0000 7ff815800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff81015e2f0 con 0x7ff810070d20 2024-04-25T11:38:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.545+0000 7ff815800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff810162070 con 0x7ff81006f930 2024-04-25T11:38:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.545+0000 7ff815800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff81015e470 con 0x7ff81006fd40 2024-04-25T11:38:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810070d20 0x7ff8101e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810070d20 0x7ff8101e9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33040/0 (socket says 172.21.15.19:33040) 2024-04-25T11:38:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 -- 172.21.15.19:0/3858248325 learned_addr learned my addr 172.21.15.19:0/3858248325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff80f400640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81006fd40 0x7ff810153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff80fe00640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff81006f930 0x7ff8101521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 -- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81006fd40 msgr2=0x7ff810153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81006fd40 0x7ff810153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 -- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff81006f930 msgr2=0x7ff8101521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff81006f930 0x7ff8101521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 -- 172.21.15.19:0/3858248325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8101f4970 con 0x7ff810070d20 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.546+0000 7ff814e00640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810070d20 0x7ff8101e9390 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c6d0 tx=0x7ff80800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.547+0000 7ff807e00640 1 -- 172.21.15.19:0/3858248325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808018070 con 0x7ff810070d20 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.547+0000 7ff807e00640 1 -- 172.21.15.19:0/3858248325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff808004d60 con 0x7ff810070d20 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.547+0000 7ff807e00640 1 -- 172.21.15.19:0/3858248325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80800a880 con 0x7ff810070d20 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.547+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8101f4c50 con 0x7ff810070d20 2024-04-25T11:38:34.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.547+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8101e8e70 con 0x7ff810070d20 2024-04-25T11:38:34.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.548+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff810076c60 con 0x7ff810070d20 2024-04-25T11:38:34.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.549+0000 7ff807e00640 1 -- 172.21.15.19:0/3858248325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff80801b070 con 0x7ff810070d20 2024-04-25T11:38:34.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.550+0000 7ff807e00640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7ec0744e0 0x7ff7ec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:34.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.550+0000 7ff807e00640 1 -- 172.21.15.19:0/3858248325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7ff808094fb0 con 0x7ff810070d20 2024-04-25T11:38:34.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.551+0000 7ff80fe00640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7ec0744e0 0x7ff7ec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:34.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.552+0000 7ff80fe00640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7ec0744e0 0x7ff7ec0769f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff800006fd0 tx=0x7ff800008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:34.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.555+0000 7ff807e00640 1 -- 172.21.15.19:0/3858248325 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff80805f130 con 0x7ff810070d20 2024-04-25T11:38:34.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.765+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff8101f4e20 con 0x7ff810070d20 2024-04-25T11:38:34.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.766+0000 7ff807e00640 1 -- 172.21.15.19:0/3858248325 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7ff8080671b0 con 0x7ff810070d20 2024-04-25T11:38:34.753 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7ec0744e0 msgr2=0x7ff7ec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7ec0744e0 0x7ff7ec0769f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff800006fd0 tx=0x7ff800008040 comp rx=0 tx=0).stop 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810070d20 msgr2=0x7ff8101e9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810070d20 0x7ff8101e9390 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c6d0 tx=0x7ff80800cb90 comp rx=0 tx=0).stop 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 shutdown_connections 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7ec0744e0 0x7ff7ec0769f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff810070d20 0x7ff8101e9390 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff81006fd40 0x7ff810153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 --2- 172.21.15.19:0/3858248325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff81006f930 0x7ff8101521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.769+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 >> 172.21.15.19:0/3858248325 conn(0x7ff810073160 msgr2=0x7ff8100b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.770+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 shutdown_connections 2024-04-25T11:38:34.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:34.770+0000 7ff815800640 1 -- 172.21.15.19:0/3858248325 wait complete. 2024-04-25T11:38:34.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:38:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.075+0000 7f9772000640 1 Processor -- start 2024-04-25T11:38:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.075+0000 7f9772000640 1 -- start start 2024-04-25T11:38:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.077+0000 7f9772000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f976c1612f0 0x7f976c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.077+0000 7f9772000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f976c06f3e0 con 0x7f976c1434d0 2024-04-25T11:38:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.077+0000 7f9772000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f976c06f6c0 con 0x7f976c0d0ed0 2024-04-25T11:38:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.077+0000 7f9772000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f976c1438e0 con 0x7f976c1612f0 2024-04-25T11:38:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.077+0000 7f976b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f976c1434d0 0x7f976c070480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60152/0 (socket says 172.21.15.19:60152) 2024-04-25T11:38:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.077+0000 7f976b400640 1 -- 172.21.15.19:0/2318374830 learned_addr learned my addr 172.21.15.19:0/2318374830 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492199211 0 0) 0x7f976c06f3e0 con 0x7f976c1434d0 2024-04-25T11:38:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976be00640 1 --2- 172.21.15.19:0/2318374830 >> v2:172.21.15.169:3300/0 conn(0x7f976c1612f0 0x7f976c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9750005730 con 0x7f976c1434d0 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3133035380 0 0) 0x7f9750005730 con 0x7f976c1434d0 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 >> v2:172.21.15.169:3300/0 conn(0x7f976c1612f0 msgr2=0x7f976c0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 --2- 172.21.15.19:0/2318374830 >> v2:172.21.15.169:3300/0 conn(0x7f976c1612f0 0x7f976c0d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 >> v1:172.21.15.169:6789/0 conn(0x7f976c0d0ed0 legacy=0x7f976c160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c06f9a0 con 0x7f976c1434d0 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f975c001670 con 0x7f976c1434d0 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 >> v1:172.21.15.19:6789/0 conn(0x7f976c1434d0 legacy=0x7f976c070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9750005bf0 0x7f9750006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f9750006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9750007480 0x7f975000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f976c1438e0 con 0x7f9750006a10 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f975000e3a0 con 0x7f9750005bf0 2024-04-25T11:38:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.078+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f975000e5a0 con 0x7f9750007480 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f9762a00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f9750006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f976b400640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9750005bf0 0x7f9750006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f976be00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9750007480 0x7f975000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f9762a00640 1 -- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9750007480 msgr2=0x7f975000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f9762a00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9750007480 0x7f975000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f9762a00640 1 -- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9750005bf0 msgr2=0x7f9750006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f9762a00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9750005bf0 0x7f9750006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f9762a00640 1 -- 172.21.15.19:0/2318374830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9750005730 con 0x7f9750006a10 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f976be00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9750007480 0x7f975000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f976b400640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9750005bf0 0x7f9750006060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.079+0000 7f976be00640 1 -- 172.21.15.19:0/2318374830 reap_dead start 2024-04-25T11:38:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.080+0000 7f9762a00640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f9750006e20 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f9754001720 tx=0x7f9750007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfb09072031e043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754001370 con 0x7f9750006a10 2024-04-25T11:38:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97540052d0 con 0x7f9750006a10 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f976aa00640 1 -- 172.21.15.19:0/2318374830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975402ab70 con 0x7f9750006a10 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f9772000640 1 -- 172.21.15.19:0/2318374830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 msgr2=0x7f9750006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f9772000640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f9750006e20 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f9754001720 tx=0x7f9750007a40 comp rx=0 tx=0).stop 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f9772000640 1 -- 172.21.15.19:0/2318374830 shutdown_connections 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f9772000640 1 --2- 172.21.15.19:0/2318374830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f9750006e20 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.081+0000 7f9772000640 1 -- 172.21.15.19:0/2318374830 >> 172.21.15.19:0/2318374830 conn(0x7f976c0d1ae0 msgr2=0x7f976c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.082+0000 7f9772000640 1 -- 172.21.15.19:0/2318374830 shutdown_connections 2024-04-25T11:38:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.082+0000 7f9772000640 1 -- 172.21.15.19:0/2318374830 wait complete. 2024-04-25T11:38:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.084+0000 7f9772000640 1 Processor -- start 2024-04-25T11:38:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.084+0000 7f9772000640 1 -- start start 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9772000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f976c15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9772000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f976c0aa2c0 0x7f976c0aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9772000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f976c141270 0x7f976c141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9772000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f976c0d12e0 con 0x7f9750006a10 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9772000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f976c06f090 con 0x7f976c0aa2c0 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9772000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f976c0d1460 con 0x7f976c141270 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f976c15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f976c15c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33054/0 (socket says 172.21.15.19:33054) 2024-04-25T11:38:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 -- 172.21.15.19:0/266694751 learned_addr learned my addr 172.21.15.19:0/266694751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9762a00640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f976c0aa2c0 0x7f976c0aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976be00640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f976c141270 0x7f976c141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 -- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f976c141270 msgr2=0x7f976c141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f976c141270 0x7f976c141680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 -- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f976c0aa2c0 msgr2=0x7f976c0aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f976c0aa2c0 0x7f976c0aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f976b400640 1 -- 172.21.15.19:0/266694751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c141bc0 con 0x7f9750006a10 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.086+0000 7f9762a00640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f976c0aa2c0 0x7f976c0aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.087+0000 7f976b400640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f976c15c230 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d770 tx=0x7f975c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.087+0000 7f9763e00640 1 -- 172.21.15.19:0/266694751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c019070 con 0x7f9750006a10 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.087+0000 7f9763e00640 1 -- 172.21.15.19:0/266694751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975c0074e0 con 0x7f9750006a10 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.087+0000 7f9763e00640 1 -- 172.21.15.19:0/266694751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c004a00 con 0x7f9750006a10 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.087+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c146600 con 0x7f9750006a10 2024-04-25T11:38:35.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.087+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c146bc0 con 0x7f9750006a10 2024-04-25T11:38:35.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.088+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976c050740 con 0x7f9750006a10 2024-04-25T11:38:35.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.090+0000 7f9763e00640 1 -- 172.21.15.19:0/266694751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f975c01c070 con 0x7f9750006a10 2024-04-25T11:38:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.091+0000 7f9763e00640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97480744b0 0x7f97480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.091+0000 7f9763e00640 1 -- 172.21.15.19:0/266694751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7f975c095270 con 0x7f9750006a10 2024-04-25T11:38:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.092+0000 7f9762a00640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97480744b0 0x7f97480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.093+0000 7f9762a00640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97480744b0 0x7f97480769c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9754032000 tx=0x7f975402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.096+0000 7f9763e00640 1 -- 172.21.15.19:0/266694751 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f976c141bc0 con 0x7f9750006a10 2024-04-25T11:38:35.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.315+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f976c1409b0 con 0x7f9750006a10 2024-04-25T11:38:35.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.316+0000 7f9763e00640 1 -- 172.21.15.19:0/266694751 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 36 entries v110) ==== 81+0+2182 (secure 0 0 0) 0x7f975c00de20 con 0x7f9750006a10 2024-04-25T11:38:35.303 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:38:35.304 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:38:35.305 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:38:35.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:listed 36 entries 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97480744b0 msgr2=0x7f97480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97480744b0 0x7f97480769c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9754032000 tx=0x7f975402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 msgr2=0x7f976c15c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f976c15c230 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d770 tx=0x7f975c00dc30 comp rx=0 tx=0).stop 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 shutdown_connections 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97480744b0 0x7f97480769c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f976c0aa2c0 0x7f976c0aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f976c141270 0x7f976c141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 --2- 172.21.15.19:0/266694751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9750006a10 0x7f976c15c230 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 >> 172.21.15.19:0/266694751 conn(0x7f976c0d1ae0 msgr2=0x7f976c070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 shutdown_connections 2024-04-25T11:38:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.319+0000 7f9772000640 1 -- 172.21.15.19:0/266694751 wait complete. 2024-04-25T11:38:35.333 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:38:35.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:35.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:38:35.437 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:35.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:35.437 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:35.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.776+0000 7fc533800640 1 Processor -- start 2024-04-25T11:38:35.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.777+0000 7fc533800640 1 -- start start 2024-04-25T11:38:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.780+0000 7fc533800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc52c0b63a0 0x7fc52c1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.780+0000 7fc533800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c06c7d0 con 0x7fc52c070680 2024-04-25T11:38:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.780+0000 7fc533800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c06cab0 con 0x7fc52c160360 2024-04-25T11:38:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.780+0000 7fc533800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc52c06d100 con 0x7fc52c0b63a0 2024-04-25T11:38:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.780+0000 7fc530e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc52c070680 0x7fc52c0af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60154/0 (socket says 172.21.15.19:60154) 2024-04-25T11:38:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.780+0000 7fc530e00640 1 -- 172.21.15.19:0/3319606972 learned_addr learned my addr 172.21.15.19:0/3319606972 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:35.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.780+0000 7fc531800640 1 --2- 172.21.15.19:0/3319606972 >> v2:172.21.15.19:3300/0 conn(0x7fc52c0b63a0 0x7fc52c1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.781+0000 7fc52b400640 1 -- 172.21.15.19:0/3319606972 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824487916 0 0) 0x7fc52c06c7d0 con 0x7fc52c070680 2024-04-25T11:38:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.781+0000 7fc52b400640 1 -- 172.21.15.19:0/3319606972 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc510003520 con 0x7fc52c070680 2024-04-25T11:38:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.781+0000 7fc531800640 1 -- 172.21.15.19:0/3319606972 >> v1:172.21.15.169:6789/0 conn(0x7fc52c160360 legacy=0x7fc52c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.781+0000 7fc531800640 1 -- 172.21.15.19:0/3319606972 >> v1:172.21.15.19:6789/0 conn(0x7fc52c070680 legacy=0x7fc52c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.781+0000 7fc531800640 1 -- 172.21.15.19:0/3319606972 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c06cd90 con 0x7fc52c0b63a0 2024-04-25T11:38:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.782+0000 7fc531800640 1 --2- 172.21.15.19:0/3319606972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c0b63a0 0x7fc52c1647f0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc52c06d280 tx=0x7fc52402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f26228204b0296c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.782+0000 7fc52b400640 1 -- 172.21.15.19:0/3319606972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524007740 con 0x7fc52c0b63a0 2024-04-25T11:38:35.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.782+0000 7fc52b400640 1 -- 172.21.15.19:0/3319606972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5240078e0 con 0x7fc52c0b63a0 2024-04-25T11:38:35.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.782+0000 7fc52b400640 1 -- 172.21.15.19:0/3319606972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524007cb0 con 0x7fc52c0b63a0 2024-04-25T11:38:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.783+0000 7fc533800640 1 -- 172.21.15.19:0/3319606972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c0b63a0 msgr2=0x7fc52c1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.783+0000 7fc533800640 1 --2- 172.21.15.19:0/3319606972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c0b63a0 0x7fc52c1647f0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc52c06d280 tx=0x7fc52402a660 comp rx=0 tx=0).stop 2024-04-25T11:38:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.783+0000 7fc533800640 1 -- 172.21.15.19:0/3319606972 shutdown_connections 2024-04-25T11:38:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.783+0000 7fc533800640 1 --2- 172.21.15.19:0/3319606972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c0b63a0 0x7fc52c1647f0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.783+0000 7fc533800640 1 -- 172.21.15.19:0/3319606972 >> 172.21.15.19:0/3319606972 conn(0x7fc52c0d1ae0 msgr2=0x7fc52c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.783+0000 7fc533800640 1 -- 172.21.15.19:0/3319606972 shutdown_connections 2024-04-25T11:38:35.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.783+0000 7fc533800640 1 -- 172.21.15.19:0/3319606972 wait complete. 2024-04-25T11:38:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.786+0000 7fc533800640 1 Processor -- start 2024-04-25T11:38:35.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.786+0000 7fc533800640 1 -- start start 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.787+0000 7fc533800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc52c070680 0x7fc52c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.787+0000 7fc533800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c160360 0x7fc52c154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.787+0000 7fc533800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52c150c50 0x7fc52c151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc533800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0afd30 con 0x7fc52c160360 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc533800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0afbb0 con 0x7fc52c150c50 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc533800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c06d590 con 0x7fc52c070680 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c160360 0x7fc52c154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c160360 0x7fc52c154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33066/0 (socket says 172.21.15.19:33066) 2024-04-25T11:38:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 -- 172.21.15.19:0/2373628091 learned_addr learned my addr 172.21.15.19:0/2373628091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:35.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc530e00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc52c070680 0x7fc52c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc531800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52c150c50 0x7fc52c151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 -- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc52c070680 msgr2=0x7fc52c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc52c070680 0x7fc52c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 -- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52c150c50 msgr2=0x7fc52c151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52c150c50 0x7fc52c151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc52be00640 1 -- 172.21.15.19:0/2373628091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c1509e0 con 0x7fc52c160360 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc531800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52c150c50 0x7fc52c151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.788+0000 7fc530e00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc52c070680 0x7fc52c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.789+0000 7fc52be00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c160360 0x7fc52c154e10 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c3c0 tx=0x7fc51400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.789+0000 7fc528c00640 1 -- 172.21.15.19:0/2373628091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5140052d0 con 0x7fc52c160360 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.789+0000 7fc528c00640 1 -- 172.21.15.19:0/2373628091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514005490 con 0x7fc52c160360 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.789+0000 7fc528c00640 1 -- 172.21.15.19:0/2373628091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514007260 con 0x7fc52c160360 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.789+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c1515a0 con 0x7fc52c160360 2024-04-25T11:38:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.789+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c155a20 con 0x7fc52c160360 2024-04-25T11:38:35.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.790+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52c070ae0 con 0x7fc52c160360 2024-04-25T11:38:35.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.792+0000 7fc528c00640 1 -- 172.21.15.19:0/2373628091 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc514007400 con 0x7fc52c160360 2024-04-25T11:38:35.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.793+0000 7fc528c00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc5080744b0 0x7fc5080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:35.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.793+0000 7fc528c00640 1 -- 172.21.15.19:0/2373628091 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7257+0+0 (secure 0 0 0) 0x7fc514094c70 con 0x7fc52c160360 2024-04-25T11:38:35.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.793+0000 7fc530e00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc5080744b0 0x7fc5080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:35.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.795+0000 7fc530e00640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc5080744b0 0x7fc5080769c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0011e0 tx=0x7fc51c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:35.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:35.797+0000 7fc528c00640 1 -- 172.21.15.19:0/2373628091 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc52c070ae0 con 0x7fc52c160360 2024-04-25T11:38:35.867 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:35.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:38:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.046+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc52c050740 con 0x7fc52c160360 2024-04-25T11:38:36.034 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:36.047+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:36.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.051+0000 7fc528c00640 1 -- 172.21.15.19:0/2373628091 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v214) ==== 161+0+0 (secure 0 0 0) 0x7fc52c1515a0 con 0x7fc52c160360 2024-04-25T11:38:36.038 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc5080744b0 msgr2=0x7fc5080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc5080744b0 0x7fc5080769c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0011e0 tx=0x7fc51c009040 comp rx=0 tx=0).stop 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c160360 msgr2=0x7fc52c154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c160360 0x7fc52c154e10 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c3c0 tx=0x7fc51400c7c0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 shutdown_connections 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc5080744b0 0x7fc5080769c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc52c150c50 0x7fc52c151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c160360 0x7fc52c154e10 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 --2- 172.21.15.19:0/2373628091 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc52c070680 0x7fc52c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.054+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 >> 172.21.15.19:0/2373628091 conn(0x7fc52c0d1ae0 msgr2=0x7fc52c0b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.055+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 shutdown_connections 2024-04-25T11:38:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.055+0000 7fc533800640 1 -- 172.21.15.19:0/2373628091 wait complete. 2024-04-25T11:38:36.049 INFO:tasks.ceph.mds.a.smithi019.stderr: -1967> 2024-04-25T11:38:21.460+0000 7fdfc5600640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-04-25T11:38:36.049 INFO:tasks.ceph.mds.a.smithi019.stderr: -1963> 2024-04-25T11:38:21.460+0000 7fdfc5600640 -1 mds.0.209 unhandled write error (22) Invalid argument, force readonly... 2024-04-25T11:38:36.054 INFO:tasks.ceph.mds.a.smithi019.stderr: -835> 2024-04-25T11:38:26.403+0000 7fdfc9200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:38:36.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:36.193 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.336+0000 7fcd37000640 1 Processor -- start 2024-04-25T11:38:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.336+0000 7fcd37000640 1 -- start start 2024-04-25T11:38:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.338+0000 7fcd37000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 0x7fcd300d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.338+0000 7fcd37000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcd3006dcf0 0x7fcd300d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.338+0000 7fcd37000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd300b3c10 con 0x7fcd300d04a0 2024-04-25T11:38:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.338+0000 7fcd37000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcd300b9520 con 0x7fcd3006dcf0 2024-04-25T11:38:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.338+0000 7fcd37000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd300b4820 con 0x7fcd3006d8e0 2024-04-25T11:38:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.338+0000 7fcd35000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcd300d04a0 0x7fcd300d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60164/0 (socket says 172.21.15.19:60164) 2024-04-25T11:38:36.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.338+0000 7fcd35000640 1 -- 172.21.15.19:0/3462878466 learned_addr learned my addr 172.21.15.19:0/3462878466 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:36.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832260668 0 0) 0x7fcd300b3c10 con 0x7fcd300d04a0 2024-04-25T11:38:36.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd14003520 con 0x7fcd300d04a0 2024-04-25T11:38:36.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd27400640 1 --2- 172.21.15.19:0/3462878466 >> v2:172.21.15.169:3300/0 conn(0x7fcd3006dcf0 0x7fcd300d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2fe00640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 0x7fcd300d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2875741111 0 0) 0x7fcd14003520 con 0x7fcd300d04a0 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 msgr2=0x7fcd300d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 0x7fcd300d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 >> v2:172.21.15.169:3300/0 conn(0x7fcd3006dcf0 msgr2=0x7fcd300d3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 --2- 172.21.15.19:0/3462878466 >> v2:172.21.15.169:3300/0 conn(0x7fcd3006dcf0 0x7fcd300d3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd300b3ef0 con 0x7fcd300d04a0 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fcd28001670 con 0x7fcd300d04a0 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 >> v1:172.21.15.19:6789/0 conn(0x7fcd300d04a0 legacy=0x7fcd300d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.339+0000 7fcd2f400640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd140039b0 0x7fcd14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd2f400640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 0x7fcd14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd2f400640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14004d40 0x7fcd14005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd300b9520 con 0x7fcd14004d40 2024-04-25T11:38:36.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd300b4820 con 0x7fcd140039b0 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1400bc00 con 0x7fcd3006d8e0 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd35000640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14004d40 0x7fcd14005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd27400640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 0x7fcd14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd2fe00640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd140039b0 0x7fcd14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd35000640 1 -- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 msgr2=0x7fcd14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd35000640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd3006d8e0 0x7fcd14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd35000640 1 -- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd140039b0 msgr2=0x7fcd14003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd35000640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd140039b0 0x7fcd14003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.340+0000 7fcd35000640 1 -- 172.21.15.19:0/3462878466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd14003520 con 0x7fcd14004d40 2024-04-25T11:38:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.341+0000 7fcd35000640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14004d40 0x7fcd14005150 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c190 tx=0x7fcd14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef065a65581bd928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.342+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28007c60 con 0x7fcd14004d40 2024-04-25T11:38:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.342+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd28005af0 con 0x7fcd14004d40 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.342+0000 7fcd2f400640 1 -- 172.21.15.19:0/3462878466 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28007070 con 0x7fcd14004d40 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.342+0000 7fcd37000640 1 -- 172.21.15.19:0/3462878466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14004d40 msgr2=0x7fcd14005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.342+0000 7fcd37000640 1 --2- 172.21.15.19:0/3462878466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14004d40 0x7fcd14005150 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c190 tx=0x7fcd14005830 comp rx=0 tx=0).stop 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.343+0000 7fcd35000640 1 -- 172.21.15.19:0/3462878466 reap_dead start 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.343+0000 7fcd37000640 1 -- 172.21.15.19:0/3462878466 shutdown_connections 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.343+0000 7fcd37000640 1 -- 172.21.15.19:0/3462878466 >> 172.21.15.19:0/3462878466 conn(0x7fcd3008d110 msgr2=0x7fcd3008d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.343+0000 7fcd37000640 1 -- 172.21.15.19:0/3462878466 shutdown_connections 2024-04-25T11:38:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.343+0000 7fcd37000640 1 -- 172.21.15.19:0/3462878466 wait complete. 2024-04-25T11:38:36.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.345+0000 7fcd37000640 1 Processor -- start 2024-04-25T11:38:36.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.345+0000 7fcd37000640 1 -- start start 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd37000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14004d40 0x7fcd300cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd37000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3009f6d0 0x7fcd3009fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd37000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd301f9a00 0x7fcd301f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd37000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd300b38e0 con 0x7fcd3009f6d0 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd37000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd300b44b0 con 0x7fcd14004d40 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd37000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd300d0fc0 con 0x7fcd301f9a00 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd27400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3009f6d0 0x7fcd3009fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd27400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3009f6d0 0x7fcd3009fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33106/0 (socket says 172.21.15.19:33106) 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.347+0000 7fcd27400640 1 -- 172.21.15.19:0/3677595535 learned_addr learned my addr 172.21.15.19:0/3677595535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd35000640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd301f9a00 0x7fcd301f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd2fe00640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14004d40 0x7fcd300cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd27400640 1 -- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd301f9a00 msgr2=0x7fcd301f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd27400640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd301f9a00 0x7fcd301f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd27400640 1 -- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14004d40 msgr2=0x7fcd300cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:36.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd27400640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14004d40 0x7fcd300cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:36.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd27400640 1 -- 172.21.15.19:0/3677595535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd300a0460 con 0x7fcd3009f6d0 2024-04-25T11:38:36.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd27400640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3009f6d0 0x7fcd3009fae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800f5a0 tx=0x7fcd1800fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd2cc00640 1 -- 172.21.15.19:0/3677595535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18019020 con 0x7fcd3009f6d0 2024-04-25T11:38:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd2cc00640 1 -- 172.21.15.19:0/3677595535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd18014870 con 0x7fcd3009f6d0 2024-04-25T11:38:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.348+0000 7fcd2cc00640 1 -- 172.21.15.19:0/3677595535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18014c60 con 0x7fcd3009f6d0 2024-04-25T11:38:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.349+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd301fa350 con 0x7fcd3009f6d0 2024-04-25T11:38:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.349+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3014dc10 con 0x7fcd3009f6d0 2024-04-25T11:38:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.350+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd30050740 con 0x7fcd3009f6d0 2024-04-25T11:38:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.351+0000 7fcd2cc00640 1 -- 172.21.15.19:0/3677595535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd1801c020 con 0x7fcd3009f6d0 2024-04-25T11:38:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.353+0000 7fcd2cc00640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd0c0744e0 0x7fcd0c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.353+0000 7fcd2cc00640 1 -- 172.21.15.19:0/3677595535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7343+0+0 (secure 0 0 0) 0x7fcd18096170 con 0x7fcd3009f6d0 2024-04-25T11:38:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.353+0000 7fcd2fe00640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd0c0744e0 0x7fcd0c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.355+0000 7fcd2fe00640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd0c0744e0 0x7fcd0c0769f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcd300744c0 tx=0x7fcd2000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.358+0000 7fcd2cc00640 1 -- 172.21.15.19:0/3677595535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd18023020 con 0x7fcd3009f6d0 2024-04-25T11:38:36.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:36.644+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fcd300d45c0 con 0x7fcd3009f6d0 2024-04-25T11:38:37.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.055+0000 7fcd2cc00640 1 -- 172.21.15.19:0/3677595535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v215) ==== 107+0+0 (secure 0 0 0) 0x7fcd301fa350 con 0x7fcd3009f6d0 2024-04-25T11:38:37.043 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:38:37.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd0c0744e0 msgr2=0x7fcd0c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd0c0744e0 0x7fcd0c0769f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcd300744c0 tx=0x7fcd2000b040 comp rx=0 tx=0).stop 2024-04-25T11:38:37.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3009f6d0 msgr2=0x7fcd3009fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3009f6d0 0x7fcd3009fae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800f5a0 tx=0x7fcd1800fa60 comp rx=0 tx=0).stop 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 shutdown_connections 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd0c0744e0 0x7fcd0c0769f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd301f9a00 0x7fcd301f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.060+0000 7fcd37000640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3009f6d0 0x7fcd3009fae0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.061+0000 7fcd37000640 1 --2- 172.21.15.19:0/3677595535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14004d40 0x7fcd300cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.061+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 >> 172.21.15.19:0/3677595535 conn(0x7fcd3008d110 msgr2=0x7fcd30071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.061+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 shutdown_connections 2024-04-25T11:38:37.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.061+0000 7fcd37000640 1 -- 172.21.15.19:0/3677595535 wait complete. 2024-04-25T11:38:37.085 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.281+0000 7fa2f700bf40 1 Processor -- start 2024-04-25T11:38:37.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.281+0000 7fa2f700bf40 1 -- start start 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.282+0000 7fa2f700bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e88851e0 0x55e9e8951e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.282+0000 7fa2f700bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88855f0 0x55e9e895bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.282+0000 7fa2f700bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e895edd0 0x55e9e895c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.282+0000 7fa2f700bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9e88cde70 con 0x55e9e88855f0 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.282+0000 7fa2f700bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e9e882bd20 con 0x55e9e88851e0 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.282+0000 7fa2f700bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e9e88ce3f0 con 0x55e9e895edd0 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88855f0 0x55e9e895bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88855f0 0x55e9e895bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33112/0 (socket says 172.21.15.19:33112) 2024-04-25T11:38:37.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 -- 172.21.15.19:0/2583214176 learned_addr learned my addr 172.21.15.19:0/2583214176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2ef400640 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e88851e0 0x55e9e8951e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2efe00640 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e895edd0 0x55e9e895c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 -- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e895edd0 msgr2=0x55e9e895c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e895edd0 0x55e9e895c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 -- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e88851e0 msgr2=0x55e9e8951e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e88851e0 0x55e9e8951e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2eea00640 1 -- 172.21.15.19:0/2583214176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e89557a0 con 0x55e9e88855f0 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.283+0000 7fa2efe00640 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e895edd0 0x55e9e895c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.284+0000 7fa2eea00640 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88855f0 0x55e9e895bfd0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009780 tx=0x7fa2d8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90eb83c26882fdeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.284+0000 7fa2ed600640 1 -- 172.21.15.19:0/2583214176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8008870 con 0x55e9e88855f0 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.284+0000 7fa2ed600640 1 -- 172.21.15.19:0/2583214176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d8018070 con 0x55e9e88855f0 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.284+0000 7fa2ed600640 1 -- 172.21.15.19:0/2583214176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8008c30 con 0x55e9e88855f0 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 -- 172.21.15.19:0/2583214176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88855f0 msgr2=0x55e9e895bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88855f0 0x55e9e895bfd0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009780 tx=0x7fa2d8005a00 comp rx=0 tx=0).stop 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 -- 172.21.15.19:0/2583214176 shutdown_connections 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e895edd0 0x55e9e895c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88855f0 0x55e9e895bfd0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/2583214176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e88851e0 0x55e9e8951e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 -- 172.21.15.19:0/2583214176 >> 172.21.15.19:0/2583214176 conn(0x55e9e87a7ac0 msgr2=0x55e9e88b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:37.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 -- 172.21.15.19:0/2583214176 shutdown_connections 2024-04-25T11:38:37.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.285+0000 7fa2f700bf40 1 -- 172.21.15.19:0/2583214176 wait complete. 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.286+0000 7fa2f700bf40 1 Processor -- start 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.286+0000 7fa2f700bf40 1 -- start start 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2f700bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88851e0 0x55e9e8950810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2f700bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e88855f0 0x55e9e89539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2f700bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e895edd0 0x55e9e8953ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2f700bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e9e8884260 con 0x55e9e88851e0 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2f700bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e9e895b5e0 con 0x55e9e895edd0 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2f700bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e9e895b840 con 0x55e9e88855f0 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88851e0 0x55e9e8950810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88851e0 0x55e9e8950810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33114/0 (socket says 172.21.15.19:33114) 2024-04-25T11:38:37.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 learned_addr learned my addr 172.21.15.19:0/4002481561 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2efe00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e895edd0 0x55e9e8953ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2eea00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e88855f0 0x55e9e89539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e88855f0 msgr2=0x55e9e89539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.288+0000 7fa2ef400640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e9e88855f0 0x55e9e89539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e895edd0 msgr2=0x55e9e8953ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2ef400640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e895edd0 0x55e9e8953ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e89ed820 con 0x55e9e88851e0 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2efe00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e9e895edd0 0x55e9e8953ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:37.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2ef400640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88851e0 0x55e9e8950810 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0054040 tx=0x7fa2e0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2e7e00640 1 -- 172.21.15.19:0/4002481561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0068070 con 0x55e9e88851e0 2024-04-25T11:38:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2e7e00640 1 -- 172.21.15.19:0/4002481561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e00514b0 con 0x55e9e88851e0 2024-04-25T11:38:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e9e89ea660 con 0x55e9e88851e0 2024-04-25T11:38:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2e7e00640 1 -- 172.21.15.19:0/4002481561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0055040 con 0x55e9e88851e0 2024-04-25T11:38:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.289+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e89eac20 con 0x55e9e88851e0 2024-04-25T11:38:37.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.292+0000 7fa2e7e00640 1 -- 172.21.15.19:0/4002481561 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa2e006b070 con 0x55e9e88851e0 2024-04-25T11:38:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.293+0000 7fa2e7e00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2cc072ec0 0x7fa2cc0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.293+0000 7fa2e7e00640 1 -- 172.21.15.19:0/4002481561 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7343+0+0 (secure 0 0 0) 0x7fa2e00d7aa0 con 0x55e9e88851e0 2024-04-25T11:38:37.280 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.293+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e9e89ec720 0x55e9e89ecb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.293+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e88b7540 con 0x55e9e89ec720 2024-04-25T11:38:37.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.293+0000 7fa2eea00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2cc072ec0 0x7fa2cc0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.294+0000 7fa2efe00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e9e89ec720 0x55e9e89ecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.295+0000 7fa2eea00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2cc072ec0 0x7fa2cc0753d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009780 tx=0x7fa2d8012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.295+0000 7fa2efe00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e9e89ec720 0x55e9e89ecb30 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.299+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v111'567 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.299+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8951660 con 0x55e9e89ec720 2024-04-25T11:38:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.303+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v111'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.303+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ed070 con 0x55e9e89ec720 2024-04-25T11:38:37.293 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.306+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v111'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.306+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89fff00 con 0x55e9e89ec720 2024-04-25T11:38:37.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.310+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.310+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a002a0 con 0x55e9e89ec720 2024-04-25T11:38:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.313+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'571 uv571 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.313+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89feef0 con 0x55e9e89ec720 2024-04-25T11:38:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.316+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'572 uv572 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.316+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff2e0 con 0x55e9e89ec720 2024-04-25T11:38:37.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.320+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v111'573 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.320+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e89ff6d0 con 0x55e9e89ec720 2024-04-25T11:38:37.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.324+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v111'574 uv574 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.324+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff6d0 con 0x55e9e89ec720 2024-04-25T11:38:37.315 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:37.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.328+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v111'575 uv575 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.328+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.332+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.332+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.336+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.336+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.340+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.340+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.344+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v111'579 uv554 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.344+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.348+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'580 uv580 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.348+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.339 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:37.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.353+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v111'581 uv581 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.353+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.357+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'582 uv582 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.357+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.361+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'583 uv583 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.361+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff680 con 0x55e9e89ec720 2024-04-25T11:38:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.365+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'584 uv584 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.365+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffad0 con 0x55e9e89ec720 2024-04-25T11:38:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.369+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'585 uv585 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.370+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.374+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v111'586 uv560 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.374+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.378+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v111'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.378+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.370 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.383+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v111'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.383+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.387+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.387+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.392+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.392+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.398+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.398+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.403+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v111'592 uv566 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.403+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.407+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v111'593 uv593 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.408+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.399 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.412+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v111'594 uv594 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.412+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ffa60 con 0x55e9e89ec720 2024-04-25T11:38:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.417+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'595 uv595 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.417+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff940 con 0x55e9e89ec720 2024-04-25T11:38:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.422+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'596 uv596 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.422+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff940 con 0x55e9e89ec720 2024-04-25T11:38:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.426+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'597 uv597 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.426+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e9e8a00b50 0x55e9e8a02f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.426+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e89ff940 con 0x55e9e8a00b50 2024-04-25T11:38:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.427+0000 7fa2ef400640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e9e8a00b50 0x55e9e8a02f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.428+0000 7fa2ef400640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e9e8a00b50 0x55e9e8a02f70 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.433+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'421 uv400 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.433+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a034b0 con 0x55e9e8a00b50 2024-04-25T11:38:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.438+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'422 uv422 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.438+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a03850 con 0x55e9e8a00b50 2024-04-25T11:38:37.430 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.443+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'423 uv423 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.443+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a03bf0 con 0x55e9e8a00b50 2024-04-25T11:38:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.448+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v111'424 uv424 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.448+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a03f90 con 0x55e9e8a00b50 2024-04-25T11:38:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.452+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'425 uv425 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.452+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a04330 con 0x55e9e8a00b50 2024-04-25T11:38:37.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.457+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v111'426 uv419 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.457+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a046d0 con 0x55e9e8a00b50 2024-04-25T11:38:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.462+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v111'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.462+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a046d0 con 0x55e9e8a00b50 2024-04-25T11:38:37.453 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.466+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v111'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.467+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.471+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.471+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.476+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.476+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.481+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v111'431 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.481+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.485+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v111'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.485+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.476 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.489+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v111'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.490+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.494+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v111'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.495+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.499+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.499+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.503+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v111'436 uv416 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.503+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.507+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v111'437 uv437 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.507+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.498 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.511+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v111'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.511+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.515+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.515+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8883b90 con 0x55e9e8a00b50 2024-04-25T11:38:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.519+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.520+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a04b80 con 0x55e9e8a00b50 2024-04-25T11:38:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.524+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a00b50 2024-04-25T11:38:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.524+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e9e8a05450 0x55e9e8a07bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.524+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a04b80 con 0x55e9e8a05450 2024-04-25T11:38:37.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.525+0000 7fa2eea00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e9e8a05450 0x55e9e8a07bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.527+0000 7fa2eea00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e9e8a05450 0x55e9e8a07bc0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.531+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.531+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a08100 con 0x55e9e8a05450 2024-04-25T11:38:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.535+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v111'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.535+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a084a0 con 0x55e9e8a05450 2024-04-25T11:38:37.526 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:37.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.539+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v111'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.539+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a04790 con 0x55e9e8a05450 2024-04-25T11:38:37.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.543+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.543+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a09da0 con 0x55e9e8a05450 2024-04-25T11:38:37.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.548+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.548+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a08cf0 con 0x55e9e8a05450 2024-04-25T11:38:37.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.552+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.552+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a090e0 con 0x55e9e89ec720 2024-04-25T11:38:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.556+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'86 uv85 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.556+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a090e0 con 0x55e9e89ec720 2024-04-25T11:38:37.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.561+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'87 uv87 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.561+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a090e0 con 0x55e9e89ec720 2024-04-25T11:38:37.552 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.565+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.565+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a09100 con 0x55e9e89ec720 2024-04-25T11:38:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.569+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.569+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e9e8a0a190 0x55e9e8a0c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.569+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a0caf0 con 0x55e9e8a0a190 2024-04-25T11:38:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.570+0000 7fa2efe00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e9e8a0a190 0x55e9e8a0c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.571+0000 7fa2efe00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e9e8a0a190 0x55e9e8a0c5b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.575+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.575+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a0caf0 con 0x55e9e8a0a190 2024-04-25T11:38:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.580+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v111'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.580+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a0caf0 con 0x55e9e8a0a190 2024-04-25T11:38:37.571 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.584+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v111'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.584+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a0db00 con 0x55e9e8a0a190 2024-04-25T11:38:37.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.588+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.588+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a0db00 con 0x55e9e8a0a190 2024-04-25T11:38:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.592+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.592+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a0caf0 con 0x55e9e8a0a190 2024-04-25T11:38:37.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.597+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.597+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e9e8a0d3f0 0x55e9e8a0d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.597+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a0ff00 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.598+0000 7fa2ef400640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e9e8a0d3f0 0x55e9e8a0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.599+0000 7fa2ef400640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e9e8a0d3f0 0x55e9e8a0d800 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.603+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v111'205 uv198 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.603+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a0ff00 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.608+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v111'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.608+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a0ff80 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.599 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:37.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.612+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v111'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.612+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a11380 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.617+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.617+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a11380 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.621+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.621+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a10370 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.625+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.625+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a10370 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.629+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v111'211 uv204 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.629+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a10370 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.633+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v111'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.633+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a10370 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.624 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:37.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.637+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v111'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.637+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a10370 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.642+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.642+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a10370 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.646+0000 7fa2ef400640 1 -- 172.21.15.19:0/4002481561 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e006a030 con 0x55e9e8a0d3f0 2024-04-25T11:38:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.646+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e9e8a10880 0x55e9e8a12ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.647+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a133f0 con 0x55e9e8a10880 2024-04-25T11:38:37.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.647+0000 7fa2eea00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e9e8a10880 0x55e9e8a12ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:37.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.649+0000 7fa2eea00640 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e9e8a10880 0x55e9e8a12ca0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:37.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.653+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v111'91 uv90 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.653+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a137c0 con 0x55e9e8a10880 2024-04-25T11:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.657+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v111'92 uv92 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.657+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a10880 2024-04-25T11:38:37.648 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.661+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v111'93 uv93 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.661+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a10880 2024-04-25T11:38:37.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.665+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.665+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a10880 2024-04-25T11:38:37.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.669+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.669+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e89ec720 2024-04-25T11:38:37.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.674+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.674+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e89ec720 2024-04-25T11:38:37.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.678+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v111'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.678+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e89ec720 2024-04-25T11:38:37.669 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.682+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v111'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.682+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14bf0 con 0x55e9e89ec720 2024-04-25T11:38:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.686+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.686+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14bf0 con 0x55e9e89ec720 2024-04-25T11:38:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.691+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.691+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e89ec720 2024-04-25T11:38:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.695+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e89ec720 2024-04-25T11:38:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.695+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.699+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v111'182 uv176 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.699+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.704+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v111'183 uv183 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.704+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:37.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.708+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v111'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.708+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.712+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.712+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.717+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.717+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.721+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v111'187 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.721+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.725+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v111'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.725+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.729+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v111'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.716 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.729+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a13be0 con 0x55e9e8a05450 2024-04-25T11:38:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.734+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v111'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.734+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a169f0 con 0x55e9e8a05450 2024-04-25T11:38:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.738+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v111'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a05450 2024-04-25T11:38:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.739+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.743+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v111'83 uv81 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.743+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.747+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v111'84 uv84 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.747+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.738 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.751+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v111'85 uv85 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.752+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.756+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v111'86 uv86 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.756+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.760+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v111'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.761+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.765+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v111'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.765+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.756 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:37.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.769+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v111'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.769+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.774+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v111'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.774+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.779+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v111'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.779+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a0a190 2024-04-25T11:38:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.784+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v111'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2e800d3a0 con 0x55e9e8a0a190 2024-04-25T11:38:37.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.784+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.788+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v111'94 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.788+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.792+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v111'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.792+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.796+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v111'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.784 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:37.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.797+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.802+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v111'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.802+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.806+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v111'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.806+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.810+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v111'229 uv220 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.810+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.814+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v111'230 uv230 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.814+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.805 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.818+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v111'231 uv231 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.818+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.823+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v111'232 uv232 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.823+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.827+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v111'233 uv233 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.827+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.831+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v111'234 uv227 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.831+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.836+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v111'235 uv235 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.836+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.827 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.840+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v111'236 uv236 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.840+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.844+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v111'237 uv237 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.844+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.848+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v111'238 uv238 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.848+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14090 con 0x55e9e8a10880 2024-04-25T11:38:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.853+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v111'239 uv239 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.853+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x55e9e8a14930 con 0x55e9e8a10880 2024-04-25T11:38:37.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2eea00640 1 -- 172.21.15.19:0/4002481561 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v111'240 uv240 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa2d8008a80 con 0x55e9e8a10880 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e9e8a00b50 msgr2=0x55e9e8a02f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55e9e8a00b50 0x55e9e8a02f70 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e9e8a0d3f0 msgr2=0x55e9e8a0d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55e9e8a0d3f0 0x55e9e8a0d800 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e9e8a10880 msgr2=0x55e9e8a12ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55e9e8a10880 0x55e9e8a12ca0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e9e8a05450 msgr2=0x55e9e8a07bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 reap_dead start 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.857+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55e9e8a05450 0x55e9e8a07bc0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e9e89ec720 msgr2=0x55e9e89ecb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55e9e89ec720 0x55e9e89ecb30 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e9e8a0a190 msgr2=0x55e9e8a0c5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55e9e8a0a190 0x55e9e8a0c5b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2cc072ec0 msgr2=0x7fa2cc0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2cc072ec0 0x7fa2cc0753d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009780 tx=0x7fa2d8012000 comp rx=0 tx=0).stop 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88851e0 msgr2=0x55e9e8950810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 --2- 172.21.15.19:0/4002481561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e9e88851e0 0x55e9e8950810 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0054040 tx=0x7fa2e0003330 comp rx=0 tx=0).stop 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2efe00640 1 -- 172.21.15.19:0/4002481561 reap_dead start 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 shutdown_connections 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.858+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 >> 172.21.15.19:0/4002481561 conn(0x55e9e87a7ac0 msgr2=0x55e9e895c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.859+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 shutdown_connections 2024-04-25T11:38:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:37.859+0000 7fa2f700bf40 1 -- 172.21.15.19:0/4002481561 wait complete. 2024-04-25T11:38:37.872 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2024-04-25T11:38:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.118+0000 7f4d069dbf40 1 Processor -- start 2024-04-25T11:38:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.118+0000 7f4d069dbf40 1 -- start start 2024-04-25T11:38:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d069dbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818071e0 0x563a818d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d069dbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818075f0 0x563a818dce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d069dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818e0dd0 0x563a818dd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d069dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a817add20 con 0x563a818e0dd0 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d069dbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563a817b1d60 con 0x563a818075f0 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d069dbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563a81850110 con 0x563a818071e0 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d00600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818e0dd0 0x563a818dd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d00600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818e0dd0 0x563a818dd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33128/0 (socket says 172.21.15.19:33128) 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.119+0000 7f4d00600640 1 -- 172.21.15.19:0/370781669 learned_addr learned my addr 172.21.15.19:0/370781669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4cff200640 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818075f0 0x563a818dce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4d00600640 1 -- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818071e0 msgr2=0x563a818d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4cffc00640 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818071e0 0x563a818d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4d00600640 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818071e0 0x563a818d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4d00600640 1 -- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818075f0 msgr2=0x563a818dce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4d00600640 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818075f0 0x563a818dce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4d00600640 1 -- 172.21.15.19:0/370781669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a81852b90 con 0x563a818e0dd0 2024-04-25T11:38:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4cff200640 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818075f0 0x563a818dce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4d00600640 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818e0dd0 0x563a818dd380 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006ec0 tx=0x7f4cf800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155c38be5db23a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.120+0000 7f4cfde00640 1 -- 172.21.15.19:0/370781669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8012070 con 0x563a818e0dd0 2024-04-25T11:38:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4cfde00640 1 -- 172.21.15.19:0/370781669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf800e690 con 0x563a818e0dd0 2024-04-25T11:38:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4cfde00640 1 -- 172.21.15.19:0/370781669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800eac0 con 0x563a818e0dd0 2024-04-25T11:38:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 -- 172.21.15.19:0/370781669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818e0dd0 msgr2=0x563a818dd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818e0dd0 0x563a818dd380 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006ec0 tx=0x7f4cf800b6c0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 -- 172.21.15.19:0/370781669 shutdown_connections 2024-04-25T11:38:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818075f0 0x563a818dce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818e0dd0 0x563a818dd380 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/370781669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818071e0 0x563a818d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 -- 172.21.15.19:0/370781669 >> 172.21.15.19:0/370781669 conn(0x563a81729ac0 msgr2=0x563a818390a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.121+0000 7f4d069dbf40 1 -- 172.21.15.19:0/370781669 shutdown_connections 2024-04-25T11:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.122+0000 7f4d069dbf40 1 -- 172.21.15.19:0/370781669 wait complete. 2024-04-25T11:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.123+0000 7f4d069dbf40 1 Processor -- start 2024-04-25T11:38:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.123+0000 7f4d069dbf40 1 -- start start 2024-04-25T11:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.124+0000 7f4d069dbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818071e0 0x563a818d45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.124+0000 7f4d069dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818075f0 0x563a818d8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.124+0000 7f4d069dbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818e0dd0 0x563a81837000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.124+0000 7f4d069dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a81852e70 con 0x563a818075f0 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.124+0000 7f4d069dbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563a818060f0 con 0x563a818071e0 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.124+0000 7f4d069dbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563a818063d0 con 0x563a818e0dd0 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.124+0000 7f4cff200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818075f0 0x563a818d8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818075f0 0x563a818d8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33142/0 (socket says 172.21.15.19:33142) 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 -- 172.21.15.19:0/4173497266 learned_addr learned my addr 172.21.15.19:0/4173497266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4d00600640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818e0dd0 0x563a81837000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cffc00640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818071e0 0x563a818d45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 -- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818e0dd0 msgr2=0x563a81837000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563a818e0dd0 0x563a81837000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 -- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818071e0 msgr2=0x563a818d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818071e0 0x563a818d45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 -- 172.21.15.19:0/4173497266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a818ca850 con 0x563a818075f0 2024-04-25T11:38:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cffc00640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563a818071e0 0x563a818d45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.125+0000 7f4cff200640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818075f0 0x563a818d8b50 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8045ef0 tx=0x7f4ce8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.126+0000 7f4cfca00640 1 -- 172.21.15.19:0/4173497266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8054aa0 con 0x563a818075f0 2024-04-25T11:38:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.126+0000 7f4cfca00640 1 -- 172.21.15.19:0/4173497266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce8054c40 con 0x563a818075f0 2024-04-25T11:38:38.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.126+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a818db7b0 con 0x563a818075f0 2024-04-25T11:38:38.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.126+0000 7f4cfca00640 1 -- 172.21.15.19:0/4173497266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce805f420 con 0x563a818075f0 2024-04-25T11:38:38.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.126+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a81837de0 con 0x563a818075f0 2024-04-25T11:38:38.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.128+0000 7f4cfca00640 1 -- 172.21.15.19:0/4173497266 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ce805c040 con 0x563a818075f0 2024-04-25T11:38:38.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.129+0000 7f4cfca00640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cdc072e90 0x7f4cdc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.130+0000 7f4cfca00640 1 -- 172.21.15.19:0/4173497266 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7343+0+0 (secure 0 0 0) 0x7f4ce80d8880 con 0x563a818075f0 2024-04-25T11:38:38.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.130+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563a81977fb0 0x563a819783c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.130+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x563a818cb1c0 con 0x563a81977fb0 2024-04-25T11:38:38.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.130+0000 7f4cffc00640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cdc072e90 0x7f4cdc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.130+0000 7f4d00600640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563a81977fb0 0x563a819783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.131+0000 7f4d00600640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563a81977fb0 0x563a819783c0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:38.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.131+0000 7f4cffc00640 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cdc072e90 0x7f4cdc0753a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0000f80 tx=0x7f4cf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.134+0000 7f4d00600640 1 -- 172.21.15.19:0/4173497266 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v111'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4cf8014030 con 0x563a81977fb0 2024-04-25T11:38:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.134+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563a81977fb0 msgr2=0x563a819783c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x563a81977fb0 0x563a819783c0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cdc072e90 msgr2=0x7f4cdc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4cdc072e90 0x7f4cdc0753a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0000f80 tx=0x7f4cf0008040 comp rx=0 tx=0).stop 2024-04-25T11:38:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818075f0 msgr2=0x563a818d8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 --2- 172.21.15.19:0/4173497266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a818075f0 0x563a818d8b50 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8045ef0 tx=0x7f4ce8002fb0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d00600640 1 -- 172.21.15.19:0/4173497266 reap_dead start 2024-04-25T11:38:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 shutdown_connections 2024-04-25T11:38:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 >> 172.21.15.19:0/4173497266 conn(0x563a81729ac0 msgr2=0x563a81852290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 shutdown_connections 2024-04-25T11:38:38.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.135+0000 7f4d069dbf40 1 -- 172.21.15.19:0/4173497266 wait complete. 2024-04-25T11:38:38.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:38.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.432+0000 7fe767200640 1 Processor -- start 2024-04-25T11:38:38.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.432+0000 7fe767200640 1 -- start start 2024-04-25T11:38:38.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.434+0000 7fe767200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7680ce520 0x7fe76815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe767200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7680ce930 0x7fe76815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe767200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7680cda60 con 0x7fe76806d5d0 2024-04-25T11:38:38.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe767200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe76815dfe0 con 0x7fe7680ce520 2024-04-25T11:38:38.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe767200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe76815e2c0 con 0x7fe7680ce930 2024-04-25T11:38:38.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe765e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7680ce520 0x7fe76815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe765e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7680ce520 0x7fe76815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33148/0 (socket says 172.21.15.19:33148) 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe765e00640 1 -- 172.21.15.19:0/4201819601 learned_addr learned my addr 172.21.15.19:0/4201819601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.435+0000 7fe765400640 1 --2- 172.21.15.19:0/4201819601 >> v2:172.21.15.169:3300/0 conn(0x7fe7680ce930 0x7fe76815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.436+0000 7fe765e00640 1 -- 172.21.15.19:0/4201819601 >> v2:172.21.15.169:3300/0 conn(0x7fe7680ce930 msgr2=0x7fe76815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.436+0000 7fe765e00640 1 --2- 172.21.15.19:0/4201819601 >> v2:172.21.15.169:3300/0 conn(0x7fe7680ce930 0x7fe76815d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.436+0000 7fe765e00640 1 -- 172.21.15.19:0/4201819601 >> v1:172.21.15.19:6789/0 conn(0x7fe76806d5d0 legacy=0x7fe76806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.436+0000 7fe765e00640 1 -- 172.21.15.19:0/4201819601 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680cdd20 con 0x7fe7680ce520 2024-04-25T11:38:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.436+0000 7fe764a00640 1 -- 172.21.15.19:0/4201819601 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653755873 0 0) 0x7fe7680cda60 con 0x7fe76806d5d0 2024-04-25T11:38:38.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.437+0000 7fe765e00640 1 --2- 172.21.15.19:0/4201819601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce520 0x7fe76815d4b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe7680b5150 tx=0x7fe75802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ae90d0229521f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:38.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.437+0000 7fe764a00640 1 -- 172.21.15.19:0/4201819601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758037070 con 0x7fe7680ce520 2024-04-25T11:38:38.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe764a00640 1 -- 172.21.15.19:0/4201819601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe758032440 con 0x7fe7680ce520 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe764a00640 1 -- 172.21.15.19:0/4201819601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758032810 con 0x7fe7680ce520 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe767200640 1 -- 172.21.15.19:0/4201819601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce520 msgr2=0x7fe76815d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe767200640 1 --2- 172.21.15.19:0/4201819601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce520 0x7fe76815d4b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe7680b5150 tx=0x7fe75802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe767200640 1 -- 172.21.15.19:0/4201819601 shutdown_connections 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe767200640 1 --2- 172.21.15.19:0/4201819601 >> v2:172.21.15.169:3300/0 conn(0x7fe7680ce930 0x7fe76815d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe767200640 1 --2- 172.21.15.19:0/4201819601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce520 0x7fe76815d4b0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe767200640 1 -- 172.21.15.19:0/4201819601 >> 172.21.15.19:0/4201819601 conn(0x7fe768072f40 msgr2=0x7fe768073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.438+0000 7fe767200640 1 -- 172.21.15.19:0/4201819601 shutdown_connections 2024-04-25T11:38:38.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.439+0000 7fe767200640 1 -- 172.21.15.19:0/4201819601 wait complete. 2024-04-25T11:38:38.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.441+0000 7fe767200640 1 Processor -- start 2024-04-25T11:38:38.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.441+0000 7fe767200640 1 -- start start 2024-04-25T11:38:38.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.442+0000 7fe767200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76806d5d0 0x7fe76806fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.442+0000 7fe767200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680ce520 0x7fe7680b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe767200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce930 0x7fe7680b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe767200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7680b73d0 con 0x7fe7680ce930 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe767200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe768160d30 con 0x7fe7680ce520 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe767200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe7680b7550 con 0x7fe76806d5d0 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce930 0x7fe7680b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce930 0x7fe7680b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33156/0 (socket says 172.21.15.19:33156) 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 -- 172.21.15.19:0/4160888275 learned_addr learned my addr 172.21.15.19:0/4160888275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe765400640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680ce520 0x7fe7680b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 -- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76806d5d0 msgr2=0x7fe76806fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe765e00640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76806d5d0 0x7fe76806fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76806d5d0 0x7fe76806fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 -- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680ce520 msgr2=0x7fe7680b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680ce520 0x7fe7680b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe766800640 1 -- 172.21.15.19:0/4160888275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7681574e0 con 0x7fe7680ce930 2024-04-25T11:38:38.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.443+0000 7fe765400640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680ce520 0x7fe7680b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.444+0000 7fe765e00640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76806d5d0 0x7fe76806fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.444+0000 7fe766800640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce930 0x7fe7680b26f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c780 tx=0x7fe76000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.444+0000 7fe75e000640 1 -- 172.21.15.19:0/4160888275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760003b70 con 0x7fe7680ce930 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.444+0000 7fe75e000640 1 -- 172.21.15.19:0/4160888275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe760003d10 con 0x7fe7680ce930 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.444+0000 7fe75e000640 1 -- 172.21.15.19:0/4160888275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760005270 con 0x7fe7680ce930 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.444+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe768070720 con 0x7fe7680ce930 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.444+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe768157780 con 0x7fe7680ce930 2024-04-25T11:38:38.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.445+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe768076c60 con 0x7fe7680ce930 2024-04-25T11:38:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.447+0000 7fe75e000640 1 -- 172.21.15.19:0/4160888275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe760005410 con 0x7fe7680ce930 2024-04-25T11:38:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.448+0000 7fe75e000640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7480744b0 0x7fe7480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.448+0000 7fe75e000640 1 -- 172.21.15.19:0/4160888275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7343+0+0 (secure 0 0 0) 0x7fe760095790 con 0x7fe7680ce930 2024-04-25T11:38:38.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.448+0000 7fe765e00640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7480744b0 0x7fe7480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:38.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.450+0000 7fe765e00640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7480744b0 0x7fe7480769c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe758034000 tx=0x7fe75802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:38.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.453+0000 7fe75e000640 1 -- 172.21.15.19:0/4160888275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe76005f8b0 con 0x7fe7680ce930 2024-04-25T11:38:38.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:38.699+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe7680b5480 con 0x7fe7680ce930 2024-04-25T11:38:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.074+0000 7fe75e000640 1 -- 172.21.15.19:0/4160888275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v216) ==== 162+0+0 (secure 0 0 0) 0x7fe76005f1b0 con 0x7fe7680ce930 2024-04-25T11:38:39.061 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.077+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7480744b0 msgr2=0x7fe7480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.077+0000 7fe767200640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7480744b0 0x7fe7480769c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe758034000 tx=0x7fe75802e000 comp rx=0 tx=0).stop 2024-04-25T11:38:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.077+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce930 msgr2=0x7fe7680b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.077+0000 7fe767200640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce930 0x7fe7680b26f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c780 tx=0x7fe76000cb80 comp rx=0 tx=0).stop 2024-04-25T11:38:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 shutdown_connections 2024-04-25T11:38:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe7480744b0 0x7fe7480769c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7680ce520 0x7fe7680b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7680ce930 0x7fe7680b26f0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 --2- 172.21.15.19:0/4160888275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe76806d5d0 0x7fe76806fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 >> 172.21.15.19:0/4160888275 conn(0x7fe768072f40 msgr2=0x7fe76806d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 shutdown_connections 2024-04-25T11:38:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.078+0000 7fe767200640 1 -- 172.21.15.19:0/4160888275 wait complete. 2024-04-25T11:38:39.079 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:39.092+0000 7f1523600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input [buffer:3] 2024-04-25T11:38:39.080 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:39.093+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:39.080 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:39.093+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:38:39.089 INFO:tasks.ceph.mds.a.smithi019.stderr: -23> 2024-04-25T11:38:39.092+0000 7f1523600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 101: Malformed input [buffer:3] 2024-04-25T11:38:39.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:39.222 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:39.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.380+0000 7f40b0a00640 1 Processor -- start 2024-04-25T11:38:39.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.381+0000 7f40b0a00640 1 -- start start 2024-04-25T11:38:39.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.383+0000 7f40b0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac0b4ea0 0x7f40ac0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.383+0000 7f40b0a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40ac0d1230 con 0x7f40ac06c5c0 2024-04-25T11:38:39.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.383+0000 7f40b0a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40ac0d06b0 con 0x7f40ac144370 2024-04-25T11:38:39.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.383+0000 7f40b0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac0d0990 con 0x7f40ac0b4ea0 2024-04-25T11:38:39.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.383+0000 7f40aa800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f40ac06c5c0 0x7f40ac0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60190/0 (socket says 172.21.15.19:60190) 2024-04-25T11:38:39.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.383+0000 7f40aa800640 1 -- 172.21.15.19:0/1890789880 learned_addr learned my addr 172.21.15.19:0/1890789880 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:39.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4005269856 0 0) 0x7f40ac0d1230 con 0x7f40ac06c5c0 2024-04-25T11:38:39.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4090003520 con 0x7f40ac06c5c0 2024-04-25T11:38:39.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9e00640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac0b4ea0 0x7f40ac0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2987142856 0 0) 0x7f4090003520 con 0x7f40ac06c5c0 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac0b4ea0 msgr2=0x7f40ac0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac0b4ea0 0x7f40ac0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 >> v1:172.21.15.169:6789/0 conn(0x7f40ac144370 legacy=0x7f40ac15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac0d0c70 con 0x7f40ac06c5c0 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f409c001670 con 0x7f40ac06c5c0 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 >> v1:172.21.15.19:6789/0 conn(0x7f40ac06c5c0 legacy=0x7f40ac0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40900039e0 0x7f4090003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4090004800 0x7f4090004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f409000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac0d0990 con 0x7f4090005150 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f409000bfb0 con 0x7f4090004800 2024-04-25T11:38:39.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.384+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f409000c1b0 con 0x7f40900039e0 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40ab200640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f409000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40aa800640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40900039e0 0x7f4090003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40ab200640 1 -- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40900039e0 msgr2=0x7f4090003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40a9e00640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4090004800 0x7f4090004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40ab200640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40900039e0 0x7f4090003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40ab200640 1 -- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4090004800 msgr2=0x7f4090004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40ab200640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4090004800 0x7f4090004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.385+0000 7f40ab200640 1 -- 172.21.15.19:0/1890789880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4090003520 con 0x7f4090005150 2024-04-25T11:38:39.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.386+0000 7f40ab200640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f409000b9c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f40a4001e50 tx=0x7f4090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd29e35e5d8c472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.386+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4001a70 con 0x7f4090005150 2024-04-25T11:38:39.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a4003970 con 0x7f4090005150 2024-04-25T11:38:39.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40a9400640 1 -- 172.21.15.19:0/1890789880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4003d60 con 0x7f4090005150 2024-04-25T11:38:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40b0a00640 1 -- 172.21.15.19:0/1890789880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 msgr2=0x7f409000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40b0a00640 1 --2- 172.21.15.19:0/1890789880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f409000b9c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f40a4001e50 tx=0x7f4090005830 comp rx=0 tx=0).stop 2024-04-25T11:38:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40ab200640 1 -- 172.21.15.19:0/1890789880 reap_dead start 2024-04-25T11:38:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40b0a00640 1 -- 172.21.15.19:0/1890789880 shutdown_connections 2024-04-25T11:38:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40b0a00640 1 -- 172.21.15.19:0/1890789880 >> 172.21.15.19:0/1890789880 conn(0x7f40ac0d1720 msgr2=0x7f40ac0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40b0a00640 1 -- 172.21.15.19:0/1890789880 shutdown_connections 2024-04-25T11:38:39.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.387+0000 7f40b0a00640 1 -- 172.21.15.19:0/1890789880 wait complete. 2024-04-25T11:38:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.390+0000 7f40b0a00640 1 Processor -- start 2024-04-25T11:38:39.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.390+0000 7f40b0a00640 1 -- start start 2024-04-25T11:38:39.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40b0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f40ac13aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40b0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac15b840 0x7f40ac1f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40b0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac1609d0 0x7f40ac13f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40b0a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac0d0450 con 0x7f4090005150 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40b0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac0d02d0 con 0x7f40ac1609d0 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40b0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac147760 con 0x7f40ac15b840 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40aa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f40ac13aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40aa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f40ac13aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33188/0 (socket says 172.21.15.19:33188) 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40aa800640 1 -- 172.21.15.19:0/1985251981 learned_addr learned my addr 172.21.15.19:0/1985251981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40a9e00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac15b840 0x7f40ac1f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40aa800640 1 -- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac15b840 msgr2=0x7f40ac1f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40aa800640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac15b840 0x7f40ac1f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40aa800640 1 -- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac1609d0 msgr2=0x7f40ac13f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40ab200640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac1609d0 0x7f40ac13f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.392+0000 7f40aa800640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac1609d0 0x7f40ac13f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.393+0000 7f40aa800640 1 -- 172.21.15.19:0/1985251981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac13e0c0 con 0x7f4090005150 2024-04-25T11:38:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.393+0000 7f40aa800640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f40ac13aaf0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f409c00d7f0 tx=0x7f409c00dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.393+0000 7f40a2a00640 1 -- 172.21.15.19:0/1985251981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c019070 con 0x7f4090005150 2024-04-25T11:38:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.393+0000 7f40a2a00640 1 -- 172.21.15.19:0/1985251981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f409c0074e0 con 0x7f4090005150 2024-04-25T11:38:39.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.393+0000 7f40a2a00640 1 -- 172.21.15.19:0/1985251981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c004a00 con 0x7f4090005150 2024-04-25T11:38:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.393+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac13e3a0 con 0x7f4090005150 2024-04-25T11:38:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.393+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac1f50a0 con 0x7f4090005150 2024-04-25T11:38:39.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.395+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ac1f4170 con 0x7f4090005150 2024-04-25T11:38:39.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.396+0000 7f40a2a00640 1 -- 172.21.15.19:0/1985251981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f409c01c070 con 0x7f4090005150 2024-04-25T11:38:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.397+0000 7f40a2a00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f408c0744b0 0x7f408c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.397+0000 7f40a2a00640 1 -- 172.21.15.19:0/1985251981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7f409c095470 con 0x7f4090005150 2024-04-25T11:38:39.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.398+0000 7f40a9e00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f408c0744b0 0x7f408c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.399+0000 7f40a9e00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f408c0744b0 0x7f408c0769c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0cda00 tx=0x7f4098008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.402+0000 7f40a2a00640 1 -- 172.21.15.19:0/1985251981 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f409c05f530 con 0x7f4090005150 2024-04-25T11:38:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.648+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40ac050740 con 0x7f4090005150 2024-04-25T11:38:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.649+0000 7f40a2a00640 1 -- 172.21.15.19:0/1985251981 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) ==== 95+0+4352 (secure 0 0 0) 0x7f409c05ee30 con 0x7f4090005150 2024-04-25T11:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:39.637 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":218,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:39.094244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 218 2024-04-25T11:38:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f408c0744b0 msgr2=0x7f408c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f408c0744b0 0x7f408c0769c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0cda00 tx=0x7f4098008040 comp rx=0 tx=0).stop 2024-04-25T11:38:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 msgr2=0x7f40ac13aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f40ac13aaf0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f409c00d7f0 tx=0x7f409c00dcb0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 shutdown_connections 2024-04-25T11:38:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f408c0744b0 0x7f408c0769c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40ac1609d0 0x7f40ac13f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40ac15b840 0x7f40ac1f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 --2- 172.21.15.19:0/1985251981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4090005150 0x7f40ac13aaf0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 >> 172.21.15.19:0/1985251981 conn(0x7f40ac0d1720 msgr2=0x7f40ac0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.652+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 shutdown_connections 2024-04-25T11:38:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.653+0000 7f40b0a00640 1 -- 172.21.15.19:0/1985251981 wait complete. 2024-04-25T11:38:39.665 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:39.665 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.950+0000 7f76a9600640 1 Processor -- start 2024-04-25T11:38:39.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.950+0000 7f76a9600640 1 -- start start 2024-04-25T11:38:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.953+0000 7f76a9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.953+0000 7f76a9600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f76a40b4ea0 0x7f76a415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.953+0000 7f76a9600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a40d1230 con 0x7f76a4144370 2024-04-25T11:38:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.953+0000 7f76a9600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f76a40d06b0 con 0x7f76a40b4ea0 2024-04-25T11:38:39.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.953+0000 7f76a9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76a40d0990 con 0x7f76a406c5c0 2024-04-25T11:38:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.954+0000 7f76a3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.954+0000 7f76a3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a40cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:58034/0 (socket says 172.21.15.19:58034) 2024-04-25T11:38:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.954+0000 7f76a3e00640 1 -- 172.21.15.19:0/3508831144 learned_addr learned my addr 172.21.15.19:0/3508831144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:39.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.954+0000 7f76a8c00640 1 --2- 172.21.15.19:0/3508831144 >> v2:172.21.15.169:3300/0 conn(0x7f76a40b4ea0 0x7f76a415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.955+0000 7f76a3e00640 1 -- 172.21.15.19:0/3508831144 >> v2:172.21.15.169:3300/0 conn(0x7f76a40b4ea0 msgr2=0x7f76a415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.955+0000 7f76a3e00640 1 --2- 172.21.15.19:0/3508831144 >> v2:172.21.15.169:3300/0 conn(0x7f76a40b4ea0 0x7f76a415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.955+0000 7f76a3e00640 1 -- 172.21.15.19:0/3508831144 >> v1:172.21.15.169:6789/0 conn(0x7f76a4144370 legacy=0x7f76a40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.955+0000 7f76a3e00640 1 -- 172.21.15.19:0/3508831144 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a40d0c70 con 0x7f76a406c5c0 2024-04-25T11:38:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.956+0000 7f76a3e00640 1 --2- 172.21.15.19:0/3508831144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a40cfa70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7694009870 tx=0x7f769402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c474c4918a5aa090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.957+0000 7f76a2a00640 1 -- 172.21.15.19:0/3508831144 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694037070 con 0x7f76a406c5c0 2024-04-25T11:38:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.957+0000 7f76a2a00640 1 -- 172.21.15.19:0/3508831144 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7694032440 con 0x7f76a406c5c0 2024-04-25T11:38:39.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.957+0000 7f76a2a00640 1 -- 172.21.15.19:0/3508831144 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694032810 con 0x7f76a406c5c0 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.957+0000 7f76a9600640 1 -- 172.21.15.19:0/3508831144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 msgr2=0x7f76a40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.957+0000 7f76a9600640 1 --2- 172.21.15.19:0/3508831144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a40cfa70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7694009870 tx=0x7f769402a270 comp rx=0 tx=0).stop 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.958+0000 7f76a9600640 1 -- 172.21.15.19:0/3508831144 shutdown_connections 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.958+0000 7f76a9600640 1 --2- 172.21.15.19:0/3508831144 >> v2:172.21.15.169:3300/0 conn(0x7f76a40b4ea0 0x7f76a415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.958+0000 7f76a9600640 1 --2- 172.21.15.19:0/3508831144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a40cfa70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.958+0000 7f76a9600640 1 -- 172.21.15.19:0/3508831144 >> 172.21.15.19:0/3508831144 conn(0x7f76a40d1720 msgr2=0x7f76a40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.958+0000 7f76a9600640 1 -- 172.21.15.19:0/3508831144 shutdown_connections 2024-04-25T11:38:39.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.958+0000 7f76a9600640 1 -- 172.21.15.19:0/3508831144 wait complete. 2024-04-25T11:38:39.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.961+0000 7f76a9600640 1 Processor -- start 2024-04-25T11:38:39.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.961+0000 7f76a9600640 1 -- start start 2024-04-25T11:38:39.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.962+0000 7f76a9600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a4141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.962+0000 7f76a9600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a4144370 0x7f76a40ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.962+0000 7f76a9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a41e8ec0 0x7f76a41e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.962+0000 7f76a9600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76a40d0450 con 0x7f76a41e8ec0 2024-04-25T11:38:39.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.962+0000 7f76a9600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76a40d02d0 con 0x7f76a4144370 2024-04-25T11:38:39.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.962+0000 7f76a9600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76a4147760 con 0x7f76a406c5c0 2024-04-25T11:38:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a41e8ec0 0x7f76a41e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a41e8ec0 0x7f76a41e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33192/0 (socket says 172.21.15.19:33192) 2024-04-25T11:38:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 -- 172.21.15.19:0/3880479228 learned_addr learned my addr 172.21.15.19:0/3880479228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a3e00640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a4141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a3400640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a4144370 0x7f76a40ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 -- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 msgr2=0x7f76a4141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a4141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 -- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a4144370 msgr2=0x7f76a40ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a4144370 0x7f76a40ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.963+0000 7f76a8c00640 1 -- 172.21.15.19:0/3880479228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a41f9d50 con 0x7f76a41e8ec0 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.964+0000 7f76a8c00640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a41e8ec0 0x7f76a41e92d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f769c009cb0 tx=0x7f769c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.964+0000 7f769be00640 1 -- 172.21.15.19:0/3880479228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c018020 con 0x7f76a41e8ec0 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.964+0000 7f769be00640 1 -- 172.21.15.19:0/3880479228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f769c014480 con 0x7f76a41e8ec0 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.964+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a41fa030 con 0x7f76a41e8ec0 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.964+0000 7f769be00640 1 -- 172.21.15.19:0/3880479228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c014870 con 0x7f76a41e8ec0 2024-04-25T11:38:39.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.964+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a41fa5f0 con 0x7f76a41e8ec0 2024-04-25T11:38:39.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.965+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a415fe80 con 0x7f76a41e8ec0 2024-04-25T11:38:39.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.967+0000 7f769be00640 1 -- 172.21.15.19:0/3880479228 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f769c005d10 con 0x7f76a41e8ec0 2024-04-25T11:38:39.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.968+0000 7f769be00640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76840744b0 0x7f76840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:39.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.968+0000 7f769be00640 1 -- 172.21.15.19:0/3880479228 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7f769c0965a0 con 0x7f76a41e8ec0 2024-04-25T11:38:39.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.969+0000 7f76a3e00640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76840744b0 0x7f76840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:39.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.970+0000 7f76a3e00640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76840744b0 0x7f76840769c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7694034000 tx=0x7f769402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:39.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:39.973+0000 7f769be00640 1 -- 172.21.15.19:0/3880479228 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f769c022030 con 0x7f76a41e8ec0 2024-04-25T11:38:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.222+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76a4050740 con 0x7f76a41e8ec0 2024-04-25T11:38:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.223+0000 7f769be00640 1 -- 172.21.15.19:0/3880479228 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) ==== 95+0+5178 (secure 0 0 0) 0x7f769c010040 con 0x7f76a41e8ec0 2024-04-25T11:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:40.210 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":219,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9501,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1160647715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1160647715},{"type":"v1","addr":"172.21.15.19:6835","nonce":1160647715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":219}],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:39.094244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 219 2024-04-25T11:38:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76840744b0 msgr2=0x7f76840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76840744b0 0x7f76840769c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7694034000 tx=0x7f769402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a41e8ec0 msgr2=0x7f76a41e92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a41e8ec0 0x7f76a41e92d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f769c009cb0 tx=0x7f769c00c490 comp rx=0 tx=0).stop 2024-04-25T11:38:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 shutdown_connections 2024-04-25T11:38:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76840744b0 0x7f76840769c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a4144370 0x7f76a40ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a41e8ec0 0x7f76a41e92d0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 --2- 172.21.15.19:0/3880479228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76a406c5c0 0x7f76a4141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 >> 172.21.15.19:0/3880479228 conn(0x7f76a40d1720 msgr2=0x7f76a40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 shutdown_connections 2024-04-25T11:38:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.226+0000 7f76a9600640 1 -- 172.21.15.19:0/3880479228 wait complete. 2024-04-25T11:38:40.243 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:40.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:40.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.542+0000 7f631be00640 1 Processor -- start 2024-04-25T11:38:40.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.542+0000 7f631be00640 1 -- start start 2024-04-25T11:38:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.545+0000 7f631be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0b4ea0 0x7f631c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.545+0000 7f631be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c0d1230 con 0x7f631c06c5c0 2024-04-25T11:38:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.545+0000 7f631be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c0d06b0 con 0x7f631c144370 2024-04-25T11:38:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.545+0000 7f631be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f631c0d0990 con 0x7f631c0b4ea0 2024-04-25T11:38:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.545+0000 7f631aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f631c06c5c0 0x7f631c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60202/0 (socket says 172.21.15.19:60202) 2024-04-25T11:38:40.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.545+0000 7f631aa00640 1 -- 172.21.15.19:0/657607888 learned_addr learned my addr 172.21.15.19:0/657607888 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:40.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f6312000640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0b4ea0 0x7f631c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.545+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2634818174 0 0) 0x7f631c0d1230 con 0x7f631c06c5c0 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6300003520 con 0x7f631c06c5c0 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3640411850 0 0) 0x7f6300003520 con 0x7f631c06c5c0 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0b4ea0 msgr2=0x7f631c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0b4ea0 0x7f631c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 >> v1:172.21.15.169:6789/0 conn(0x7f631c144370 legacy=0x7f631c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c0d0c70 con 0x7f631c06c5c0 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f630c001670 con 0x7f631c06c5c0 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 >> v1:172.21.15.19:6789/0 conn(0x7f631c06c5c0 legacy=0x7f631c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63000039e0 0x7f6300003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300004800 0x7f6300004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300005270 0x7f630000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0d0990 con 0x7f6300005270 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f630000c1b0 con 0x7f63000039e0 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f630000c3b0 con 0x7f6300004800 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.546+0000 7f6312000640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0b4ea0 0x7f631c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631b400640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300005270 0x7f630000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631aa00640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63000039e0 0x7f6300003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631b400640 1 -- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300004800 msgr2=0x7f6300004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631b400640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6300004800 0x7f6300004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631b400640 1 -- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63000039e0 msgr2=0x7f6300003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631b400640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63000039e0 0x7f6300003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631b400640 1 -- 172.21.15.19:0/657607888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6300003520 con 0x7f6300005270 2024-04-25T11:38:40.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.547+0000 7f631aa00640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63000039e0 0x7f6300003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.548+0000 7f631b400640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300005270 0x7f630000b9c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6314001e50 tx=0x7f6300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a7d0493a2cca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.548+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314001a70 con 0x7f6300005270 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.548+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6314003970 con 0x7f6300005270 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.548+0000 7f631a000640 1 -- 172.21.15.19:0/657607888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314003d60 con 0x7f6300005270 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.549+0000 7f631be00640 1 -- 172.21.15.19:0/657607888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300005270 msgr2=0x7f630000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.549+0000 7f631be00640 1 --2- 172.21.15.19:0/657607888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300005270 0x7f630000b9c0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6314001e50 tx=0x7f6300005830 comp rx=0 tx=0).stop 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.549+0000 7f631b400640 1 -- 172.21.15.19:0/657607888 reap_dead start 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.549+0000 7f631be00640 1 -- 172.21.15.19:0/657607888 shutdown_connections 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.549+0000 7f631be00640 1 -- 172.21.15.19:0/657607888 >> 172.21.15.19:0/657607888 conn(0x7f631c0d1720 msgr2=0x7f631c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.549+0000 7f631be00640 1 -- 172.21.15.19:0/657607888 shutdown_connections 2024-04-25T11:38:40.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.550+0000 7f631be00640 1 -- 172.21.15.19:0/657607888 wait complete. 2024-04-25T11:38:40.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.552+0000 7f631be00640 1 Processor -- start 2024-04-25T11:38:40.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.552+0000 7f631be00640 1 -- start start 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300005270 0x7f631c161d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f631c15d6e0 0x7f631c13f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0cc4c0 0x7f631c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0d0450 con 0x7f631c15d6e0 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0d02d0 con 0x7f6300005270 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f631c147760 con 0x7f631c0cc4c0 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f6312000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f631c15d6e0 0x7f631c13f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f6312000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f631c15d6e0 0x7f631c13f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33214/0 (socket says 172.21.15.19:33214) 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f6312000640 1 -- 172.21.15.19:0/711826119 learned_addr learned my addr 172.21.15.19:0/711826119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631b400640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0cc4c0 0x7f631c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.554+0000 7f631aa00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300005270 0x7f631c161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6312000640 1 -- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0cc4c0 msgr2=0x7f631c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6312000640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0cc4c0 0x7f631c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6312000640 1 -- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300005270 msgr2=0x7f631c161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6312000640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300005270 0x7f631c161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6312000640 1 -- 172.21.15.19:0/711826119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c13f6d0 con 0x7f631c15d6e0 2024-04-25T11:38:40.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6312000640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f631c15d6e0 0x7f631c13f190 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f630400e560 tx=0x7f630400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:40.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6313400640 1 -- 172.21.15.19:0/711826119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400f040 con 0x7f631c15d6e0 2024-04-25T11:38:40.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6313400640 1 -- 172.21.15.19:0/711826119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6304004250 con 0x7f631c15d6e0 2024-04-25T11:38:40.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c0cce10 con 0x7f631c15d6e0 2024-04-25T11:38:40.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f6313400640 1 -- 172.21.15.19:0/711826119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304003610 con 0x7f631c15d6e0 2024-04-25T11:38:40.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.555+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c160320 con 0x7f631c15d6e0 2024-04-25T11:38:40.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.557+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c1622c0 con 0x7f631c15d6e0 2024-04-25T11:38:40.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.558+0000 7f6313400640 1 -- 172.21.15.19:0/711826119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f63040037b0 con 0x7f631c15d6e0 2024-04-25T11:38:40.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.559+0000 7f6313400640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62f80744b0 0x7f62f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:40.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.560+0000 7f6313400640 1 -- 172.21.15.19:0/711826119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7f63040954b0 con 0x7f631c15d6e0 2024-04-25T11:38:40.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.560+0000 7f631aa00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62f80744b0 0x7f62f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:40.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.562+0000 7f631aa00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62f80744b0 0x7f62f80769c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f630c001eb0 tx=0x7f630c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:40.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.564+0000 7f6313400640 1 -- 172.21.15.19:0/711826119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f631c0cce10 con 0x7f631c15d6e0 2024-04-25T11:38:40.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.810+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f631c050740 con 0x7f631c15d6e0 2024-04-25T11:38:40.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.812+0000 7f6313400640 1 -- 172.21.15.19:0/711826119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) ==== 95+0+5178 (secure 0 0 0) 0x7f631c13f6d0 con 0x7f631c15d6e0 2024-04-25T11:38:40.799 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:40.799 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":219,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9501,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1160647715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1160647715},{"type":"v1","addr":"172.21.15.19:6835","nonce":1160647715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":219}],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:39.094244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:40.799 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 219 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.814+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62f80744b0 msgr2=0x7f62f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.814+0000 7f631be00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62f80744b0 0x7f62f80769c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f630c001eb0 tx=0x7f630c009040 comp rx=0 tx=0).stop 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.814+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f631c15d6e0 msgr2=0x7f631c13f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.814+0000 7f631be00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f631c15d6e0 0x7f631c13f190 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f630400e560 tx=0x7f630400ea20 comp rx=0 tx=0).stop 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 shutdown_connections 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62f80744b0 0x7f62f80769c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c0cc4c0 0x7f631c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f631c15d6e0 0x7f631c13f190 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 --2- 172.21.15.19:0/711826119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6300005270 0x7f631c161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 >> 172.21.15.19:0/711826119 conn(0x7f631c0d1720 msgr2=0x7f631c06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 shutdown_connections 2024-04-25T11:38:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:40.815+0000 7f631be00640 1 -- 172.21.15.19:0/711826119 wait complete. 2024-04-25T11:38:40.828 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2024-04-25T11:38:40.828 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2024-04-25T11:38:40.829 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:40.829 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:40.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:38:40.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:40.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:40.881 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:40.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:40.931 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:38:40.931 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.213+0000 7f77dee00640 1 Processor -- start 2024-04-25T11:38:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.213+0000 7f77dee00640 1 -- start start 2024-04-25T11:38:41.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.216+0000 7f77dee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77d8144500 0x7f77d8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:41.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.216+0000 7f77dee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdb90 0x7f77d8161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:41.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.216+0000 7f77dee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d806f5f0 con 0x7f77d80cdfa0 2024-04-25T11:38:41.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.216+0000 7f77dee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f77d8070200 con 0x7f77d8144500 2024-04-25T11:38:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.216+0000 7f77dee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f77d806f8d0 con 0x7f77d80cdb90 2024-04-25T11:38:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.216+0000 7f77d7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77d8144500 0x7f77d8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77d7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f77d8144500 0x7f77d8070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33228/0 (socket says 172.21.15.19:33228) 2024-04-25T11:38:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77d7e00640 1 -- 172.21.15.19:0/2256325943 learned_addr learned my addr 172.21.15.19:0/2256325943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77dce00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f77d80cdfa0 0x7f77d80ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60212/0 (socket says 172.21.15.19:60212) 2024-04-25T11:38:41.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77cfe00640 1 --2- 172.21.15.19:0/2256325943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdb90 0x7f77d8161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:41.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77d7e00640 1 -- 172.21.15.19:0/2256325943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdb90 msgr2=0x7f77d8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:41.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77d7e00640 1 --2- 172.21.15.19:0/2256325943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdb90 0x7f77d8161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:41.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77d7e00640 1 -- 172.21.15.19:0/2256325943 >> v1:172.21.15.19:6789/0 conn(0x7f77d80cdfa0 legacy=0x7f77d80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:41.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77d7e00640 1 -- 172.21.15.19:0/2256325943 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d806fbb0 con 0x7f77d8144500 2024-04-25T11:38:41.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.217+0000 7f77d7400640 1 -- 172.21.15.19:0/2256325943 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646460830 0 0) 0x7f77d806f5f0 con 0x7f77d80cdfa0 2024-04-25T11:38:41.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.218+0000 7f77d7e00640 1 --2- 172.21.15.19:0/2256325943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d8070690 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009870 tx=0x7f77c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270cf4eb6b13f9ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:41.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.219+0000 7f77d7400640 1 -- 172.21.15.19:0/2256325943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8037070 con 0x7f77d8144500 2024-04-25T11:38:41.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.219+0000 7f77d7400640 1 -- 172.21.15.19:0/2256325943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8032440 con 0x7f77d8144500 2024-04-25T11:38:41.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.219+0000 7f77d7400640 1 -- 172.21.15.19:0/2256325943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8032810 con 0x7f77d8144500 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.219+0000 7f77dee00640 1 -- 172.21.15.19:0/2256325943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 msgr2=0x7f77d8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.219+0000 7f77dee00640 1 --2- 172.21.15.19:0/2256325943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d8070690 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009870 tx=0x7f77c802a270 comp rx=0 tx=0).stop 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.220+0000 7f77dee00640 1 -- 172.21.15.19:0/2256325943 shutdown_connections 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.220+0000 7f77dee00640 1 --2- 172.21.15.19:0/2256325943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdb90 0x7f77d8161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.220+0000 7f77dee00640 1 --2- 172.21.15.19:0/2256325943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d8070690 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.220+0000 7f77dee00640 1 -- 172.21.15.19:0/2256325943 >> 172.21.15.19:0/2256325943 conn(0x7f77d8073050 msgr2=0x7f77d8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.220+0000 7f77dee00640 1 -- 172.21.15.19:0/2256325943 shutdown_connections 2024-04-25T11:38:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.220+0000 7f77dee00640 1 -- 172.21.15.19:0/2256325943 wait complete. 2024-04-25T11:38:41.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.222+0000 7f77dee00640 1 Processor -- start 2024-04-25T11:38:41.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.222+0000 7f77dee00640 1 -- start start 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d80cdb90 0x7f77d81e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdfa0 0x7f77d8155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d813efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77d806f2e0 con 0x7f77d8144500 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f77d806f160 con 0x7f77d80cdb90 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f77d8162140 con 0x7f77d80cdfa0 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d813efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d813efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33230/0 (socket says 172.21.15.19:33230) 2024-04-25T11:38:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77dce00640 1 -- 172.21.15.19:0/4017321747 learned_addr learned my addr 172.21.15.19:0/4017321747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77cfe00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdfa0 0x7f77d8155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.224+0000 7f77d7e00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d80cdb90 0x7f77d81e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77dce00640 1 -- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdfa0 msgr2=0x7f77d8155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77dce00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdfa0 0x7f77d8155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77dce00640 1 -- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d80cdb90 msgr2=0x7f77d81e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77dce00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d80cdb90 0x7f77d81e90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77dce00640 1 -- 172.21.15.19:0/4017321747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d81e8020 con 0x7f77d8144500 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77dce00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d813efe0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c760 tx=0x7f77d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77d4c00640 1 -- 172.21.15.19:0/4017321747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0018070 con 0x7f77d8144500 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77d4c00640 1 -- 172.21.15.19:0/4017321747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d000a640 con 0x7f77d8144500 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77d4c00640 1 -- 172.21.15.19:0/4017321747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d000aa30 con 0x7f77d8144500 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.225+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d81e8300 con 0x7f77d8144500 2024-04-25T11:38:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.226+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d81e88c0 con 0x7f77d8144500 2024-04-25T11:38:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.227+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d8050740 con 0x7f77d8144500 2024-04-25T11:38:41.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.228+0000 7f77d4c00640 1 -- 172.21.15.19:0/4017321747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f77d001b070 con 0x7f77d8144500 2024-04-25T11:38:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.229+0000 7f77d4c00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77b40744b0 0x7f77b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.229+0000 7f77d4c00640 1 -- 172.21.15.19:0/4017321747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7f77d0094e00 con 0x7f77d8144500 2024-04-25T11:38:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.230+0000 7f77d7e00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77b40744b0 0x7f77b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:41.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.231+0000 7f77d7e00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77b40744b0 0x7f77b40769c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f77c8034000 tx=0x7f77c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.234+0000 7f77d4c00640 1 -- 172.21.15.19:0/4017321747 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77d005ef50 con 0x7f77d8144500 2024-04-25T11:38:41.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:41.473+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f77d813ad40 con 0x7f77d8144500 2024-04-25T11:38:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.118+0000 7f77d4c00640 1 -- 172.21.15.19:0/4017321747 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v220) ==== 161+0+0 (secure 0 0 0) 0x7f77d005e850 con 0x7f77d8144500 2024-04-25T11:38:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.123+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77b40744b0 msgr2=0x7f77b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.123+0000 7f77dee00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77b40744b0 0x7f77b40769c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f77c8034000 tx=0x7f77c802e000 comp rx=0 tx=0).stop 2024-04-25T11:38:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.123+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 msgr2=0x7f77d813efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.123+0000 7f77dee00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d813efe0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c760 tx=0x7f77d000cc20 comp rx=0 tx=0).stop 2024-04-25T11:38:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 shutdown_connections 2024-04-25T11:38:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77b40744b0 0x7f77b40769c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d8144500 0x7f77d813efe0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77d80cdfa0 0x7f77d8155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 --2- 172.21.15.19:0/4017321747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77d80cdb90 0x7f77d81e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 >> 172.21.15.19:0/4017321747 conn(0x7f77d8073050 msgr2=0x7f77d815fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 shutdown_connections 2024-04-25T11:38:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.124+0000 7f77dee00640 1 -- 172.21.15.19:0/4017321747 wait complete. 2024-04-25T11:38:42.140 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:42.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.440+0000 7f9559a00640 1 Processor -- start 2024-04-25T11:38:42.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.440+0000 7f9559a00640 1 -- start start 2024-04-25T11:38:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.442+0000 7f9559a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f95540b4ff0 0x7f95541611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.442+0000 7f9559a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f95540d1f70 0x7f95540d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.442+0000 7f9559a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9554161720 con 0x7f955408cd60 2024-04-25T11:38:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.442+0000 7f9559a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f955406e9d0 con 0x7f95540d1f70 2024-04-25T11:38:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.442+0000 7f9559a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f955406ddc0 con 0x7f95540b4ff0 2024-04-25T11:38:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.443+0000 7f954b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f955408cd60 0x7f955408d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:60220/0 (socket says 172.21.15.19:60220) 2024-04-25T11:38:42.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.443+0000 7f954b400640 1 -- 172.21.15.19:0/1624114969 learned_addr learned my addr 172.21.15.19:0/1624114969 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.443+0000 7f9559000640 1 --2- 172.21.15.19:0/1624114969 >> v2:172.21.15.19:3300/0 conn(0x7f95540d1f70 0x7f95540d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.443+0000 7f9553e00640 1 --2- 172.21.15.19:0/1624114969 >> v2:172.21.15.169:3300/0 conn(0x7f95540b4ff0 0x7f95541611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.443+0000 7f9553400640 1 -- 172.21.15.19:0/1624114969 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052721479 0 0) 0x7f953c001110 con 0x7f955408cd60 2024-04-25T11:38:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.443+0000 7f9553400640 1 -- 172.21.15.19:0/1624114969 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9538003520 con 0x7f955408cd60 2024-04-25T11:38:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.444+0000 7f9559000640 1 -- 172.21.15.19:0/1624114969 >> v2:172.21.15.169:3300/0 conn(0x7f95540b4ff0 msgr2=0x7f95541611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.444+0000 7f9559000640 1 --2- 172.21.15.19:0/1624114969 >> v2:172.21.15.169:3300/0 conn(0x7f95540b4ff0 0x7f95541611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.444+0000 7f9559000640 1 -- 172.21.15.19:0/1624114969 >> v1:172.21.15.19:6789/0 conn(0x7f955408cd60 legacy=0x7f955408d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.444+0000 7f9553400640 1 -- 172.21.15.19:0/1624114969 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1604374595 0 0) 0x7f9538003520 con 0x7f955408cd60 2024-04-25T11:38:42.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.444+0000 7f9559000640 1 -- 172.21.15.19:0/1624114969 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955406e0a0 con 0x7f95540d1f70 2024-04-25T11:38:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.445+0000 7f9559000640 1 --2- 172.21.15.19:0/1624114969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95540d1f70 0x7f95540d2380 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9554074480 tx=0x7f954c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9734d304bd42f517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.445+0000 7f9553400640 1 -- 172.21.15.19:0/1624114969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c037070 con 0x7f95540d1f70 2024-04-25T11:38:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.445+0000 7f9553400640 1 -- 172.21.15.19:0/1624114969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c02ab30 con 0x7f95540d1f70 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.445+0000 7f9553400640 1 -- 172.21.15.19:0/1624114969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c001840 con 0x7f95540d1f70 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 -- 172.21.15.19:0/1624114969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95540d1f70 msgr2=0x7f95540d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 --2- 172.21.15.19:0/1624114969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95540d1f70 0x7f95540d2380 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9554074480 tx=0x7f954c02a0a0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 -- 172.21.15.19:0/1624114969 shutdown_connections 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 --2- 172.21.15.19:0/1624114969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95540d1f70 0x7f95540d2380 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 --2- 172.21.15.19:0/1624114969 >> v2:172.21.15.169:3300/0 conn(0x7f95540b4ff0 0x7f95541611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 -- 172.21.15.19:0/1624114969 >> 172.21.15.19:0/1624114969 conn(0x7f95540b5460 msgr2=0x7f95540b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 -- 172.21.15.19:0/1624114969 shutdown_connections 2024-04-25T11:38:42.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.446+0000 7f9559a00640 1 -- 172.21.15.19:0/1624114969 wait complete. 2024-04-25T11:38:42.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.449+0000 7f9559a00640 1 Processor -- start 2024-04-25T11:38:42.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.449+0000 7f9559a00640 1 -- start start 2024-04-25T11:38:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.450+0000 7f9559a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955408cd60 0x7f955414b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9559a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95540b4ff0 0x7f955415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9559a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95540d1f70 0x7f9554159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9559a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95540b9180 con 0x7f955408cd60 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9559a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f955406e660 con 0x7f95540d1f70 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9559a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f955406da20 con 0x7f95540b4ff0 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955408cd60 0x7f955414b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955408cd60 0x7f955414b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33250/0 (socket says 172.21.15.19:33250) 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 -- 172.21.15.19:0/4078907826 learned_addr learned my addr 172.21.15.19:0/4078907826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 -- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95540b4ff0 msgr2=0x7f955415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95540b4ff0 0x7f955415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 -- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95540d1f70 msgr2=0x7f9554159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95540d1f70 0x7f9554159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.451+0000 7f9553e00640 1 -- 172.21.15.19:0/4078907826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9554148b00 con 0x7f955408cd60 2024-04-25T11:38:42.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.452+0000 7f9553e00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955408cd60 0x7f955414b050 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f954400efc0 tx=0x7f954400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:42.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.452+0000 7f9550c00640 1 -- 172.21.15.19:0/4078907826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544018020 con 0x7f955408cd60 2024-04-25T11:38:42.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.452+0000 7f9550c00640 1 -- 172.21.15.19:0/4078907826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9544014480 con 0x7f955408cd60 2024-04-25T11:38:42.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.452+0000 7f9550c00640 1 -- 172.21.15.19:0/4078907826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544014850 con 0x7f955408cd60 2024-04-25T11:38:42.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.452+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955415bf20 con 0x7f955408cd60 2024-04-25T11:38:42.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.452+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9554148d70 con 0x7f955408cd60 2024-04-25T11:38:42.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.454+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9554050740 con 0x7f955408cd60 2024-04-25T11:38:42.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.455+0000 7f9550c00640 1 -- 172.21.15.19:0/4078907826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f954401b020 con 0x7f955408cd60 2024-04-25T11:38:42.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.457+0000 7f9550c00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95300744b0 0x7f95300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.457+0000 7f9550c00640 1 -- 172.21.15.19:0/4078907826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7f9544096810 con 0x7f955408cd60 2024-04-25T11:38:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.457+0000 7f954b400640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95300744b0 0x7f95300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.459+0000 7f954b400640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95300744b0 0x7f95300769c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f953c005880 tx=0x7f953c0016a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:42.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.462+0000 7f9550c00640 1 -- 172.21.15.19:0/4078907826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9544022020 con 0x7f955408cd60 2024-04-25T11:38:42.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:42.736+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f95540b6370 con 0x7f955408cd60 2024-04-25T11:38:43.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.141+0000 7f9550c00640 1 -- 172.21.15.19:0/4078907826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v221) ==== 101+0+0 (secure 0 0 0) 0x7f9544010040 con 0x7f955408cd60 2024-04-25T11:38:43.128 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:38:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95300744b0 msgr2=0x7f95300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95300744b0 0x7f95300769c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f953c005880 tx=0x7f953c0016a0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955408cd60 msgr2=0x7f955414b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955408cd60 0x7f955414b050 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f954400efc0 tx=0x7f954400c490 comp rx=0 tx=0).stop 2024-04-25T11:38:43.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 shutdown_connections 2024-04-25T11:38:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95300744b0 0x7f95300769c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95540d1f70 0x7f9554159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95540b4ff0 0x7f955415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 --2- 172.21.15.19:0/4078907826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955408cd60 0x7f955414b050 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 >> 172.21.15.19:0/4078907826 conn(0x7f95540b5460 msgr2=0x7f95540d2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 shutdown_connections 2024-04-25T11:38:43.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.144+0000 7f9559a00640 1 -- 172.21.15.19:0/4078907826 wait complete. 2024-04-25T11:38:43.160 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.359+0000 7f641a090f40 1 Processor -- start 2024-04-25T11:38:43.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.359+0000 7f641a090f40 1 -- start start 2024-04-25T11:38:43.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f641a090f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa71c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f641a090f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 0x562bfa677660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f641a090f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562bfa677ba0 0x562bfa7af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f641a090f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562bfa5f2d20 con 0x7f63f80022d0 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f641a090f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562bfa5f6d60 con 0x562bfa71eb10 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f641a090f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562bfa696cf0 con 0x562bfa677ba0 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa71c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa71c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33270/0 (socket says 172.21.15.19:33270) 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 -- 172.21.15.19:0/3463123893 learned_addr learned my addr 172.21.15.19:0/3463123893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 -- 172.21.15.19:0/3463123893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562bfa677ba0 msgr2=0x562bfa7af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 --2- 172.21.15.19:0/3463123893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562bfa677ba0 0x562bfa7af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 -- 172.21.15.19:0/3463123893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 msgr2=0x562bfa677660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 --2- 172.21.15.19:0/3463123893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 0x562bfa677660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.361+0000 7f6413200640 1 -- 172.21.15.19:0/3463123893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfa71c7c0 con 0x7f63f80022d0 2024-04-25T11:38:43.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.362+0000 7f6413200640 1 --2- 172.21.15.19:0/3463123893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa71c280 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f6404000c00 tx=0x7f6404051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82210f528867f65f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.362+0000 7f6411e00640 1 -- 172.21.15.19:0/3463123893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404055050 con 0x7f63f80022d0 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.362+0000 7f6411e00640 1 -- 172.21.15.19:0/3463123893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64040519a0 con 0x7f63f80022d0 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.362+0000 7f6411e00640 1 -- 172.21.15.19:0/3463123893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404051d70 con 0x7f63f80022d0 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 -- 172.21.15.19:0/3463123893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 msgr2=0x562bfa71c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 --2- 172.21.15.19:0/3463123893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa71c280 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f6404000c00 tx=0x7f6404051000 comp rx=0 tx=0).stop 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 -- 172.21.15.19:0/3463123893 shutdown_connections 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 --2- 172.21.15.19:0/3463123893 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 0x562bfa677660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 --2- 172.21.15.19:0/3463123893 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562bfa677ba0 0x562bfa7af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 --2- 172.21.15.19:0/3463123893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa71c280 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 -- 172.21.15.19:0/3463123893 >> 172.21.15.19:0/3463123893 conn(0x562bfa56eac0 msgr2=0x562bfa71d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:43.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 -- 172.21.15.19:0/3463123893 shutdown_connections 2024-04-25T11:38:43.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.363+0000 7f641a090f40 1 -- 172.21.15.19:0/3463123893 wait complete. 2024-04-25T11:38:43.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.364+0000 7f641a090f40 1 Processor -- start 2024-04-25T11:38:43.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.364+0000 7f641a090f40 1 -- start start 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f641a090f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562bfa677ba0 0x562bfa64e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f641a090f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 0x562bfa64e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f641a090f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa706170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f641a090f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562bfa71e0c0 con 0x7f63f80022d0 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f641a090f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562bfa677fb0 con 0x562bfa71eb10 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f641a090f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x562bfa70ec60 con 0x562bfa677ba0 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f6413c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa706170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f6413c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa706170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33274/0 (socket says 172.21.15.19:33274) 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 learned_addr learned my addr 172.21.15.19:0/1990159430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562bfa677ba0 msgr2=0x562bfa64e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f6413c00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x562bfa677ba0 0x562bfa64e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 msgr2=0x562bfa64e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.366+0000 7f640be00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 0x562bfa64e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f6413c00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x562bfa71eb10 0x562bfa64e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfa71d9a0 con 0x7f63f80022d0 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f6413c00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa706170 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x562bfa650190 tx=0x7f640c00b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f6410a00640 1 -- 172.21.15.19:0/1990159430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0062d0 con 0x7f63f80022d0 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f6410a00640 1 -- 172.21.15.19:0/1990159430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c012450 con 0x7f63f80022d0 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f6410a00640 1 -- 172.21.15.19:0/1990159430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c012840 con 0x7f63f80022d0 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562bfa697630 con 0x7f63f80022d0 2024-04-25T11:38:43.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.367+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfa710290 con 0x7f63f80022d0 2024-04-25T11:38:43.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.370+0000 7f6410a00640 1 -- 172.21.15.19:0/1990159430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f640c003450 con 0x7f63f80022d0 2024-04-25T11:38:43.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.370+0000 7f6410a00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63f0073030 0x7f63f0075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.371+0000 7f6410a00640 1 -- 172.21.15.19:0/1990159430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7f640c093c50 con 0x7f63f80022d0 2024-04-25T11:38:43.358 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:43.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.371+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562bfa71ef90 0x562bfa70ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.371+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa70e410 con 0x562bfa71ef90 2024-04-25T11:38:43.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.371+0000 7f6413200640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63f0073030 0x7f63f0075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.371+0000 7f640be00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562bfa71ef90 0x562bfa70ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.373+0000 7f6413200640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63f0073030 0x7f63f0075540 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6404045bd0 tx=0x7f640400c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.373+0000 7f640be00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562bfa71ef90 0x562bfa70ded0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.376+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v112'598 uv572 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.376+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7c6da0 con 0x562bfa71ef90 2024-04-25T11:38:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.380+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v112'599 uv599 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.380+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7140 con 0x562bfa71ef90 2024-04-25T11:38:43.370 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:43.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.384+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v112'600 uv600 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.384+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c84f0 con 0x562bfa71ef90 2024-04-25T11:38:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.387+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.387+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c8890 con 0x562bfa71ef90 2024-04-25T11:38:43.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.391+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.391+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c74e0 con 0x562bfa71ef90 2024-04-25T11:38:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.394+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.394+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7880 con 0x562bfa71ef90 2024-04-25T11:38:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.399+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v112'604 uv578 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.399+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.402+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v112'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.402+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.406+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v112'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.393 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:43.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.406+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.410+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.411+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.414+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.414+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.418+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.418+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.422+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v112'610 uv585 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.422+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.426+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'611 uv611 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.426+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.417 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:43.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.430+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v112'612 uv612 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.430+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.434+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'613 uv613 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.434+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.438+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'614 uv614 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.438+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.442+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'615 uv615 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.442+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.446+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'616 uv616 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.446+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.451+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v112'617 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.451+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.455+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v112'618 uv618 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.455+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.446 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.459+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v112'619 uv619 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.459+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.463+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'620 uv620 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.463+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.467+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'621 uv621 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.468+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.472+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.472+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.476+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v112'623 uv597 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.476+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.481+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v112'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.481+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.485+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v112'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.472 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:43.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.485+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.489+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.490+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.494+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.494+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c7c20 con 0x562bfa71ef90 2024-04-25T11:38:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.498+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.499+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562bfa7c8c30 0x562bfa7cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.499+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cb590 con 0x562bfa7c8c30 2024-04-25T11:38:43.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.499+0000 7f6413200640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562bfa7c8c30 0x562bfa7cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.501+0000 7f6413200640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562bfa7c8c30 0x562bfa7cb050 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.505+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'442 uv425 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.506+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7cb930 con 0x562bfa7c8c30 2024-04-25T11:38:43.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.511+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'443 uv443 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.511+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cbcd0 con 0x562bfa7c8c30 2024-04-25T11:38:43.502 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.515+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'444 uv444 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.515+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cc070 con 0x562bfa7c8c30 2024-04-25T11:38:43.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.520+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v112'445 uv445 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.520+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cc410 con 0x562bfa7c8c30 2024-04-25T11:38:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.524+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'446 uv446 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.524+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7c8130 con 0x562bfa7c8c30 2024-04-25T11:38:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.529+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v112'447 uv430 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.529+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7cc7b0 con 0x562bfa7c8c30 2024-04-25T11:38:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.533+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v112'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.534+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.524 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:43.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.537+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v112'449 uv449 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.538+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.542+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.542+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.547+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.547+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.552+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v112'452 uv435 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.552+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.556+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v112'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.556+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.547 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.560+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v112'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.561+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d1240 con 0x562bfa7c8c30 2024-04-25T11:38:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.566+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v112'455 uv455 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.566+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d1240 con 0x562bfa7c8c30 2024-04-25T11:38:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.570+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'456 uv456 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.571+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.575+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v112'457 uv441 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.575+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.580+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v112'458 uv458 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.580+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.571 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.584+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v112'459 uv459 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.585+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cdb60 con 0x562bfa7c8c30 2024-04-25T11:38:43.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.589+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'460 uv460 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.589+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cdb60 con 0x562bfa7c8c30 2024-04-25T11:38:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.593+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'461 uv461 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.594+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ccb50 con 0x562bfa7c8c30 2024-04-25T11:38:43.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.598+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'462 uv462 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7c8c30 2024-04-25T11:38:43.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.599+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562bfa7cd060 0x562bfa7cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.599+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa7cd060 2024-04-25T11:38:43.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.600+0000 7f640be00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562bfa7cd060 0x562bfa7cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.602+0000 7f640be00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562bfa7cd060 0x562bfa7cf480 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.606+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.606+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa7cd060 2024-04-25T11:38:43.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.610+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v112'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.610+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa7cd060 2024-04-25T11:38:43.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.614+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v112'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.602 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.614+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d09d0 con 0x562bfa7cd060 2024-04-25T11:38:43.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.618+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.619+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d09d0 con 0x562bfa7cd060 2024-04-25T11:38:43.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.623+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.623+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa7cd060 2024-04-25T11:38:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.627+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.627+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa71ef90 2024-04-25T11:38:43.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.631+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'90 uv89 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.631+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa71ef90 2024-04-25T11:38:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.635+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.636+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa71ef90 2024-04-25T11:38:43.626 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.640+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'92 uv92 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.640+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7cf9c0 con 0x562bfa71ef90 2024-04-25T11:38:43.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.644+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'93 uv93 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.644+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562bfa7cfed0 0x562bfa7d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.644+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d2830 con 0x562bfa7cfed0 2024-04-25T11:38:43.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.645+0000 7f6413c00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562bfa7cfed0 0x562bfa7d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.646+0000 7f6413c00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562bfa7cfed0 0x562bfa7d22f0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.650+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.651+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7d2d20 con 0x562bfa7cfed0 2024-04-25T11:38:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.655+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v112'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.655+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d3200 con 0x562bfa7cfed0 2024-04-25T11:38:43.646 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:43.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.659+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v112'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.659+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d4ab0 con 0x562bfa7cfed0 2024-04-25T11:38:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.664+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.664+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d4e50 con 0x562bfa7cfed0 2024-04-25T11:38:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.668+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.668+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d3aa0 con 0x562bfa7cfed0 2024-04-25T11:38:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.672+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.672+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562bfa7d43a0 0x562bfa7d7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.673+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d7740 con 0x562bfa7d43a0 2024-04-25T11:38:43.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.673+0000 7f6413200640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562bfa7d43a0 0x562bfa7d7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.675+0000 7f6413200640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562bfa7d43a0 0x562bfa7d7200 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.679+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v112'216 uv210 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.679+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7d7740 con 0x562bfa7d43a0 2024-04-25T11:38:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.683+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v112'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.683+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d77c0 con 0x562bfa7d43a0 2024-04-25T11:38:43.675 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.688+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v112'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.688+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d8bc0 con 0x562bfa7d43a0 2024-04-25T11:38:43.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.692+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.692+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d8bc0 con 0x562bfa7d43a0 2024-04-25T11:38:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.696+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.696+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d7bb0 con 0x562bfa7d43a0 2024-04-25T11:38:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.701+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.701+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d7bb0 con 0x562bfa7d43a0 2024-04-25T11:38:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.705+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v112'222 uv215 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.705+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7d7bb0 con 0x562bfa7d43a0 2024-04-25T11:38:43.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.710+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v112'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.710+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d7bb0 con 0x562bfa7d43a0 2024-04-25T11:38:43.701 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:43.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.714+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v112'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.714+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d7bb0 con 0x562bfa7d43a0 2024-04-25T11:38:43.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.719+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.719+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7d7bb0 con 0x562bfa7d43a0 2024-04-25T11:38:43.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.723+0000 7f6413200640 1 -- 172.21.15.19:0/1990159430 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6404057030 con 0x562bfa7d43a0 2024-04-25T11:38:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.724+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562bfa7d80c0 0x562bfa7da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.724+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dab40 con 0x562bfa7d80c0 2024-04-25T11:38:43.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.725+0000 7f640be00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562bfa7d80c0 0x562bfa7da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:43.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.726+0000 7f640be00640 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562bfa7d80c0 0x562bfa7da4e0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.730+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v112'97 uv96 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.730+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7dab40 con 0x562bfa7d80c0 2024-04-25T11:38:43.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.734+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v112'98 uv98 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.734+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7d80c0 2024-04-25T11:38:43.725 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:43.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.738+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v112'99 uv99 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.739+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7d80c0 2024-04-25T11:38:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.743+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'100 uv100 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.743+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7d80c0 2024-04-25T11:38:43.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.746+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'101 uv101 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.747+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa71ef90 2024-04-25T11:38:43.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.751+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.751+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa71ef90 2024-04-25T11:38:43.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.756+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v112'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.756+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa71ef90 2024-04-25T11:38:43.747 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:43.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.760+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v112'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.760+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dbfa0 con 0x562bfa71ef90 2024-04-25T11:38:43.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.765+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.765+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dbfa0 con 0x562bfa71ef90 2024-04-25T11:38:43.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.769+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.769+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa71ef90 2024-04-25T11:38:43.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.774+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa71ef90 2024-04-25T11:38:43.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.774+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.779+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v112'192 uv186 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.779+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.783+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v112'193 uv193 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.783+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.774 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:43.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.787+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v112'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.788+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.792+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.792+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.797+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.797+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.801+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v112'197 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.801+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.805+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v112'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.805+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.810+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v112'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.797 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.810+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7daf90 con 0x562bfa7cd060 2024-04-25T11:38:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.814+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.814+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7ddce0 con 0x562bfa7cd060 2024-04-25T11:38:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.819+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7cd060 2024-04-25T11:38:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.819+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db2e0 con 0x562bfa7cfed0 2024-04-25T11:38:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.823+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v112'87 uv86 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.823+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7cfed0 2024-04-25T11:38:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.828+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v112'88 uv88 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.828+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7cfed0 2024-04-25T11:38:43.820 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.833+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v112'89 uv89 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.833+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7cfed0 2024-04-25T11:38:43.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.837+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v112'90 uv90 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.837+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7cfed0 2024-04-25T11:38:43.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.841+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v112'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.842+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7cfed0 2024-04-25T11:38:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.846+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v112'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.846+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7cfed0 2024-04-25T11:38:43.837 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.850+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v112'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.850+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dc690 con 0x562bfa7cfed0 2024-04-25T11:38:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.855+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v112'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.855+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dc690 con 0x562bfa7cfed0 2024-04-25T11:38:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.859+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v112'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.859+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7cfed0 2024-04-25T11:38:43.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.863+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v112'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f640c00f070 con 0x562bfa7cfed0 2024-04-25T11:38:43.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.864+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.868+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v112'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.868+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.872+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v112'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.872+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.863 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:43.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.876+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v112'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.876+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.881+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v112'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.881+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.885+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v112'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.885+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.889+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v112'241 uv233 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.890+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.894+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v112'242 uv242 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.894+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.885 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:43.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.898+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v112'243 uv243 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.898+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.903+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v112'244 uv244 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.903+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.907+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v112'245 uv245 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.907+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.911+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v112'246 uv240 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.911+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.916+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v112'247 uv247 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.917+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.921+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v112'248 uv248 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.908 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:43.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.921+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dc690 con 0x562bfa7d80c0 2024-04-25T11:38:43.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.925+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v112'249 uv249 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.925+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dc690 con 0x562bfa7d80c0 2024-04-25T11:38:43.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.928+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v112'250 uv250 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.929+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7db680 con 0x562bfa7d80c0 2024-04-25T11:38:43.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.933+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v112'251 uv251 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.933+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x562bfa7dbef0 con 0x562bfa7d80c0 2024-04-25T11:38:43.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.937+0000 7f640be00640 1 -- 172.21.15.19:0/1990159430 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v112'252 uv252 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63fc00d420 con 0x562bfa7d80c0 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.937+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562bfa7c8c30 msgr2=0x562bfa7cb050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.937+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562bfa7c8c30 0x562bfa7cb050 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.937+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562bfa7d43a0 msgr2=0x562bfa7d7200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.937+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562bfa7d43a0 0x562bfa7d7200 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.937+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562bfa7d80c0 msgr2=0x562bfa7da4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.937+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562bfa7d80c0 0x562bfa7da4e0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562bfa7cd060 msgr2=0x562bfa7cf480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x562bfa7cd060 0x562bfa7cf480 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 reap_dead start 2024-04-25T11:38:43.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562bfa71ef90 msgr2=0x562bfa70ded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x562bfa71ef90 0x562bfa70ded0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f6413c00640 1 -- 172.21.15.19:0/1990159430 reap_dead start 2024-04-25T11:38:43.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562bfa7cfed0 msgr2=0x562bfa7d22f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562bfa7cfed0 0x562bfa7d22f0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63f0073030 msgr2=0x7f63f0075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63f0073030 0x7f63f0075540 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6404045bd0 tx=0x7f640400c040 comp rx=0 tx=0).stop 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 msgr2=0x562bfa706170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.938+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa706170 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x562bfa650190 tx=0x7f640c00b570 comp rx=0 tx=0).stop 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.939+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 shutdown_connections 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.939+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x562bfa7cfed0 0x562bfa7d22f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.939+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f63f0073030 0x7f63f0075540 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.939+0000 7f641a090f40 1 --2- 172.21.15.19:0/1990159430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63f80022d0 0x562bfa706170 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.939+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 >> 172.21.15.19:0/1990159430 conn(0x562bfa56eac0 msgr2=0x562bfa676c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.939+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 shutdown_connections 2024-04-25T11:38:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:43.939+0000 7f641a090f40 1 -- 172.21.15.19:0/1990159430 wait complete. 2024-04-25T11:38:43.955 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T11:38:44.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.155+0000 7f857b4edf40 1 Processor -- start 2024-04-25T11:38:44.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.155+0000 7f857b4edf40 1 -- start start 2024-04-25T11:38:44.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.156+0000 7f857b4edf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.156+0000 7f857b4edf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 0x55a8113e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.156+0000 7f857b4edf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.156+0000 7f857b4edf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8112bcd20 con 0x55a8113edee0 2024-04-25T11:38:44.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.156+0000 7f857b4edf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a8112c0d60 con 0x55a8113e5260 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.156+0000 7f857b4edf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a81135f3f0 con 0x7f8558003630 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113ee2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36376/0 (socket says 172.21.15.19:36376) 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 -- 172.21.15.19:0/609818546 learned_addr learned my addr 172.21.15.19:0/609818546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8574600640 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8573c00640 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 0x55a8113e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 -- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 msgr2=0x55a8113deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113deca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 -- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 msgr2=0x55a8113e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 0x55a8113e5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.157+0000 7f8575000640 1 -- 172.21.15.19:0/609818546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8113e5bb0 con 0x55a8113edee0 2024-04-25T11:38:44.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.158+0000 7f8575000640 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113ee2f0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f856c046810 tx=0x7f856c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652b232bd1791d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:44.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.158+0000 7f8572800640 1 -- 172.21.15.19:0/609818546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c0019f0 con 0x55a8113edee0 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.158+0000 7f8572800640 1 -- 172.21.15.19:0/609818546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f856c006070 con 0x55a8113edee0 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.158+0000 7f8572800640 1 -- 172.21.15.19:0/609818546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c04d970 con 0x55a8113edee0 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 -- 172.21.15.19:0/609818546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 msgr2=0x55a8113ee2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113ee2f0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f856c046810 tx=0x7f856c008030 comp rx=0 tx=0).stop 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 -- 172.21.15.19:0/609818546 shutdown_connections 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 0x55a8113e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113ee2f0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 --2- 172.21.15.19:0/609818546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 -- 172.21.15.19:0/609818546 >> 172.21.15.19:0/609818546 conn(0x55a811238ac0 msgr2=0x55a8113638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 -- 172.21.15.19:0/609818546 shutdown_connections 2024-04-25T11:38:44.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.159+0000 7f857b4edf40 1 -- 172.21.15.19:0/609818546 wait complete. 2024-04-25T11:38:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.160+0000 7f857b4edf40 1 Processor -- start 2024-04-25T11:38:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.160+0000 7f857b4edf40 1 -- start start 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.161+0000 7f857b4edf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 0x55a8113d5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.161+0000 7f857b4edf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.161+0000 7f857b4edf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.161+0000 7f857b4edf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a811361220 con 0x55a8113edee0 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.161+0000 7f857b4edf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a811348300 con 0x55a8113e5260 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.161+0000 7f857b4edf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a81135d220 con 0x7f8558003630 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113153f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36382/0 (socket says 172.21.15.19:36382) 2024-04-25T11:38:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 -- 172.21.15.19:0/804841378 learned_addr learned my addr 172.21.15.19:0/804841378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8575000640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8574600640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 0x55a8113d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 -- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 msgr2=0x55a8113ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 -- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 msgr2=0x55a8113d5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a8113e5260 0x55a8113d5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8573c00640 1 -- 172.21.15.19:0/804841378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8113dd410 con 0x55a8113edee0 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.162+0000 7f8575000640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8558003630 0x55a8113ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.163+0000 7f8573c00640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113153f0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f856000ee00 tx=0x7f856000c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.163+0000 7f8571400640 1 -- 172.21.15.19:0/804841378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560015070 con 0x55a8113edee0 2024-04-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.163+0000 7f8571400640 1 -- 172.21.15.19:0/804841378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85600113f0 con 0x55a8113edee0 2024-04-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.163+0000 7f8571400640 1 -- 172.21.15.19:0/804841378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600117c0 con 0x55a8113edee0 2024-04-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.163+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a8113dd6f0 con 0x55a8113edee0 2024-04-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.163+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8113ddcb0 con 0x55a8113edee0 2024-04-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.165+0000 7f8571400640 1 -- 172.21.15.19:0/804841378 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8560018070 con 0x55a8113edee0 2024-04-25T11:38:44.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.166+0000 7f8571400640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8554072ec0 0x7f85540753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.166+0000 7f8571400640 1 -- 172.21.15.19:0/804841378 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7f8560094360 con 0x55a8113edee0 2024-04-25T11:38:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.167+0000 7f857b4edf40 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a811361d50 0x55a811362160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.167+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55a811315930 con 0x55a811361d50 2024-04-25T11:38:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.167+0000 7f8574600640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8554072ec0 0x7f85540753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.168+0000 7f8575000640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a811361d50 0x55a811362160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.169+0000 7f8574600640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8554072ec0 0x7f85540753d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8564005fd0 tx=0x7f856400c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:44.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.169+0000 7f8575000640 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a811361d50 0x55a811362160 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f8575000640 1 -- 172.21.15.19:0/804841378 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v112'202 uv202 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f856c0017b0 con 0x55a811361d50 2024-04-25T11:38:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a811361d50 msgr2=0x55a811362160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a811361d50 0x55a811362160 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8554072ec0 msgr2=0x7f85540753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8554072ec0 0x7f85540753d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8564005fd0 tx=0x7f856400c040 comp rx=0 tx=0).stop 2024-04-25T11:38:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 msgr2=0x55a8113153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 --2- 172.21.15.19:0/804841378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8113edee0 0x55a8113153f0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f856000ee00 tx=0x7f856000c5c0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f8575000640 1 -- 172.21.15.19:0/804841378 reap_dead start 2024-04-25T11:38:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 shutdown_connections 2024-04-25T11:38:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 >> 172.21.15.19:0/804841378 conn(0x55a811238ac0 msgr2=0x55a811344720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 shutdown_connections 2024-04-25T11:38:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.174+0000 7f857b4edf40 1 -- 172.21.15.19:0/804841378 wait complete. 2024-04-25T11:38:44.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.481+0000 7ffa07800640 1 Processor -- start 2024-04-25T11:38:44.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.481+0000 7ffa07800640 1 -- start start 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.483+0000 7ffa07800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffa0006dce0 0x7ffa000cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.483+0000 7ffa07800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa0015adf0 0x7ffa00160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.483+0000 7ffa07800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa00071110 con 0x7ffa00070630 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.483+0000 7ffa07800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ffa000713f0 con 0x7ffa0006dce0 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.483+0000 7ffa07800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa000701a0 con 0x7ffa0015adf0 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ff9f7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffa0006dce0 0x7ffa000cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ffa04e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ffa00070630 0x7ffa0006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:57768/0 (socket says 172.21.15.19:57768) 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ffa04e00640 1 -- 172.21.15.19:0/3245965423 learned_addr learned my addr 172.21.15.19:0/3245965423 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:44.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ffa05800640 1 --2- 172.21.15.19:0/3245965423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa0015adf0 0x7ffa00160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ff9f7e00640 1 -- 172.21.15.19:0/3245965423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa0015adf0 msgr2=0x7ffa00160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ff9f7e00640 1 --2- 172.21.15.19:0/3245965423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa0015adf0 0x7ffa00160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ff9f7e00640 1 -- 172.21.15.19:0/3245965423 >> v1:172.21.15.169:6789/0 conn(0x7ffa00070630 legacy=0x7ffa0006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.484+0000 7ff9f7e00640 1 -- 172.21.15.19:0/3245965423 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0006fb50 con 0x7ffa0006dce0 2024-04-25T11:38:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.486+0000 7ff9f7e00640 1 --2- 172.21.15.19:0/3245965423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0006dce0 0x7ffa000cec50 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80097a0 tx=0x7ff9e802c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f676e7a996ee8271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.486+0000 7ff9ffe00640 1 -- 172.21.15.19:0/3245965423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8039070 con 0x7ffa0006dce0 2024-04-25T11:38:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.486+0000 7ff9ffe00640 1 -- 172.21.15.19:0/3245965423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8034640 con 0x7ffa0006dce0 2024-04-25T11:38:44.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.486+0000 7ff9ffe00640 1 -- 172.21.15.19:0/3245965423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e802ce60 con 0x7ffa0006dce0 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 -- 172.21.15.19:0/3245965423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0006dce0 msgr2=0x7ffa000cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 --2- 172.21.15.19:0/3245965423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0006dce0 0x7ffa000cec50 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80097a0 tx=0x7ff9e802c310 comp rx=0 tx=0).stop 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 -- 172.21.15.19:0/3245965423 shutdown_connections 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 --2- 172.21.15.19:0/3245965423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa0015adf0 0x7ffa00160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 --2- 172.21.15.19:0/3245965423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0006dce0 0x7ffa000cec50 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 -- 172.21.15.19:0/3245965423 >> 172.21.15.19:0/3245965423 conn(0x7ffa00073090 msgr2=0x7ffa000734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 -- 172.21.15.19:0/3245965423 shutdown_connections 2024-04-25T11:38:44.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.487+0000 7ffa07800640 1 -- 172.21.15.19:0/3245965423 wait complete. 2024-04-25T11:38:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.489+0000 7ffa07800640 1 Processor -- start 2024-04-25T11:38:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.489+0000 7ffa07800640 1 -- start start 2024-04-25T11:38:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ffa07800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa00070630 0x7ffa00152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ffa07800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0015adf0 0x7ffa0014da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ffa07800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa000b5980 0x7ffa000b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ffa07800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa000b0e80 con 0x7ffa0015adf0 2024-04-25T11:38:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ffa07800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa000b0d00 con 0x7ffa000b5980 2024-04-25T11:38:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ffa07800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa00070c80 con 0x7ffa00070630 2024-04-25T11:38:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ff9f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0015adf0 0x7ffa0014da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ff9f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0015adf0 0x7ffa0014da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36394/0 (socket says 172.21.15.19:36394) 2024-04-25T11:38:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.491+0000 7ff9f7e00640 1 -- 172.21.15.19:0/4226663950 learned_addr learned my addr 172.21.15.19:0/4226663950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ffa04e00640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa00070630 0x7ffa00152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9f7e00640 1 -- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa00070630 msgr2=0x7ffa00152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9f7e00640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa00070630 0x7ffa00152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9f7e00640 1 -- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa000b5980 msgr2=0x7ffa000b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9f7e00640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa000b5980 0x7ffa000b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9f7e00640 1 -- 172.21.15.19:0/4226663950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa000b62d0 con 0x7ffa0015adf0 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9f7e00640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0015adf0 0x7ffa0014da60 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80098c0 tx=0x7ff9e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9fd600640 1 -- 172.21.15.19:0/4226663950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8043020 con 0x7ffa0015adf0 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9fd600640 1 -- 172.21.15.19:0/4226663950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8032980 con 0x7ffa0015adf0 2024-04-25T11:38:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ff9fd600640 1 -- 172.21.15.19:0/4226663950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8039070 con 0x7ffa0015adf0 2024-04-25T11:38:44.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.492+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa001504f0 con 0x7ffa0015adf0 2024-04-25T11:38:44.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.493+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa0014f730 con 0x7ffa0015adf0 2024-04-25T11:38:44.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.494+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa00050740 con 0x7ffa0015adf0 2024-04-25T11:38:44.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.495+0000 7ff9fd600640 1 -- 172.21.15.19:0/4226663950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9e803f440 con 0x7ffa0015adf0 2024-04-25T11:38:44.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.496+0000 7ff9fd600640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9dc0744b0 0x7ff9dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:44.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.496+0000 7ff9fd600640 1 -- 172.21.15.19:0/4226663950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7429+0+0 (secure 0 0 0) 0x7ff9e80b5ee0 con 0x7ffa0015adf0 2024-04-25T11:38:44.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.497+0000 7ffa04e00640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9dc0744b0 0x7ff9dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.498+0000 7ffa04e00640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9dc0744b0 0x7ff9dc0769c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0000f60 tx=0x7ff9f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.501+0000 7ff9fd600640 1 -- 172.21.15.19:0/4226663950 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9e804d060 con 0x7ffa0015adf0 2024-04-25T11:38:44.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:44.754+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ffa000b67f0 con 0x7ffa0015adf0 2024-04-25T11:38:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.147+0000 7ff9fd600640 1 -- 172.21.15.19:0/4226663950 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v222) ==== 162+0+0 (secure 0 0 0) 0x7ffa000b62d0 con 0x7ffa0015adf0 2024-04-25T11:38:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.151+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9dc0744b0 msgr2=0x7ff9dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.151+0000 7ffa07800640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9dc0744b0 0x7ff9dc0769c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0000f60 tx=0x7ff9f0009040 comp rx=0 tx=0).stop 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.151+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0015adf0 msgr2=0x7ffa0014da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.151+0000 7ffa07800640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0015adf0 0x7ffa0014da60 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80098c0 tx=0x7ff9e80046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 shutdown_connections 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9dc0744b0 0x7ff9dc0769c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa000b5980 0x7ffa000b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa0015adf0 0x7ffa0014da60 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 --2- 172.21.15.19:0/4226663950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa00070630 0x7ffa00152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 >> 172.21.15.19:0/4226663950 conn(0x7ffa00073090 msgr2=0x7ffa0006d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:45.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 shutdown_connections 2024-04-25T11:38:45.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.152+0000 7ffa07800640 1 -- 172.21.15.19:0/4226663950 wait complete. 2024-04-25T11:38:45.157 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:45.170+0000 7ff1d8600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input [buffer:3] 2024-04-25T11:38:45.158 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:45.171+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:45.158 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:45.171+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:38:45.169 INFO:tasks.ceph.mds.a.smithi019.stderr: -19> 2024-04-25T11:38:45.170+0000 7ff1d8600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 2 < 101: Malformed input [buffer:3] 2024-04-25T11:38:45.177 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:45.303 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.457+0000 7f7e65c00640 1 Processor -- start 2024-04-25T11:38:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.457+0000 7f7e65c00640 1 -- start start 2024-04-25T11:38:45.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.459+0000 7f7e65c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e600b7c50 0x7f7e600b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.459+0000 7f7e65c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e60161650 con 0x7f7e600d44a0 2024-04-25T11:38:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.459+0000 7f7e65c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e60161370 con 0x7f7e60164860 2024-04-25T11:38:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.459+0000 7f7e65c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7e601643d0 con 0x7f7e600b7c50 2024-04-25T11:38:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e65200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7e600d44a0 0x7f7e600d48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56782/0 (socket says 172.21.15.19:56782) 2024-04-25T11:38:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e65200640 1 -- 172.21.15.19:0/3521038530 learned_addr learned my addr 172.21.15.19:0/3521038530 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e5f400640 1 --2- 172.21.15.19:0/3521038530 >> v2:172.21.15.19:3300/0 conn(0x7f7e600b7c50 0x7f7e600b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e5ea00640 1 -- 172.21.15.19:0/3521038530 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322699297 0 0) 0x7f7e60161650 con 0x7f7e600d44a0 2024-04-25T11:38:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e5ea00640 1 -- 172.21.15.19:0/3521038530 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e40003520 con 0x7f7e600d44a0 2024-04-25T11:38:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e5f400640 1 -- 172.21.15.19:0/3521038530 >> v1:172.21.15.169:6789/0 conn(0x7f7e60164860 legacy=0x7f7e600b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e5f400640 1 -- 172.21.15.19:0/3521038530 >> v1:172.21.15.19:6789/0 conn(0x7f7e600d44a0 legacy=0x7f7e600d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.460+0000 7f7e5f400640 1 -- 172.21.15.19:0/3521038530 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e60163d80 con 0x7f7e600b7c50 2024-04-25T11:38:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.462+0000 7f7e5f400640 1 --2- 172.21.15.19:0/3521038530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600b7c50 0x7f7e600b4d30 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009d90 tx=0x7f7e4802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f63f1ad37a79e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.462+0000 7f7e5ea00640 1 -- 172.21.15.19:0/3521038530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48007740 con 0x7f7e600b7c50 2024-04-25T11:38:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.462+0000 7f7e5ea00640 1 -- 172.21.15.19:0/3521038530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e480078e0 con 0x7f7e600b7c50 2024-04-25T11:38:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.462+0000 7f7e5ea00640 1 -- 172.21.15.19:0/3521038530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48007cb0 con 0x7f7e600b7c50 2024-04-25T11:38:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.463+0000 7f7e65c00640 1 -- 172.21.15.19:0/3521038530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600b7c50 msgr2=0x7f7e600b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.463+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3521038530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600b7c50 0x7f7e600b4d30 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009d90 tx=0x7f7e4802a6f0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.463+0000 7f7e65c00640 1 -- 172.21.15.19:0/3521038530 shutdown_connections 2024-04-25T11:38:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.463+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3521038530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600b7c50 0x7f7e600b4d30 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.463+0000 7f7e65c00640 1 -- 172.21.15.19:0/3521038530 >> 172.21.15.19:0/3521038530 conn(0x7f7e600d2720 msgr2=0x7f7e600d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.463+0000 7f7e65c00640 1 -- 172.21.15.19:0/3521038530 shutdown_connections 2024-04-25T11:38:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.463+0000 7f7e65c00640 1 -- 172.21.15.19:0/3521038530 wait complete. 2024-04-25T11:38:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.465+0000 7f7e65c00640 1 Processor -- start 2024-04-25T11:38:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.465+0000 7f7e65c00640 1 -- start start 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e65c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600d44a0 0x7f7e601eaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e65c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e60164860 0x7f7e600ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e65c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e600cedc0 0x7f7e6014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e65c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e600d4fc0 con 0x7f7e600d44a0 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e65c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e600b5270 con 0x7f7e60164860 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e65c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e600d5140 con 0x7f7e600cedc0 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600d44a0 0x7f7e601eaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600d44a0 0x7f7e601eaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36426/0 (socket says 172.21.15.19:36426) 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.467+0000 7f7e5fe00640 1 -- 172.21.15.19:0/3457701278 learned_addr learned my addr 172.21.15.19:0/3457701278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e5f400640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e60164860 0x7f7e600ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e65200640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e600cedc0 0x7f7e6014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e5fe00640 1 -- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e600cedc0 msgr2=0x7f7e6014cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e5fe00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e600cedc0 0x7f7e6014cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e5fe00640 1 -- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e60164860 msgr2=0x7f7e600ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e5fe00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e60164860 0x7f7e600ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e5fe00640 1 -- 172.21.15.19:0/3457701278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e601eb4e0 con 0x7f7e600d44a0 2024-04-25T11:38:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e5fe00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600d44a0 0x7f7e601eaa60 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c3c0 tx=0x7f7e5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.468+0000 7f7e57e00640 1 -- 172.21.15.19:0/3457701278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50017070 con 0x7f7e600d44a0 2024-04-25T11:38:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.469+0000 7f7e57e00640 1 -- 172.21.15.19:0/3457701278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e500052d0 con 0x7f7e600d44a0 2024-04-25T11:38:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.469+0000 7f7e57e00640 1 -- 172.21.15.19:0/3457701278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50003830 con 0x7f7e600d44a0 2024-04-25T11:38:45.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.469+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e601eb7c0 con 0x7f7e600d44a0 2024-04-25T11:38:45.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.469+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e601ebd80 con 0x7f7e600d44a0 2024-04-25T11:38:45.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.470+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e60050740 con 0x7f7e600d44a0 2024-04-25T11:38:45.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.471+0000 7f7e57e00640 1 -- 172.21.15.19:0/3457701278 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e5001a070 con 0x7f7e600d44a0 2024-04-25T11:38:45.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.472+0000 7f7e57e00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:45.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.472+0000 7f7e57e00640 1 -- 172.21.15.19:0/3457701278 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7f7e50094c30 con 0x7f7e600d44a0 2024-04-25T11:38:45.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.473+0000 7f7e5f400640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:45.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.474+0000 7f7e5f400640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e48034000 tx=0x7f7e48033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:45.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.477+0000 7f7e57e00640 1 -- 172.21.15.19:0/3457701278 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e5005ed30 con 0x7f7e600d44a0 2024-04-25T11:38:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.727+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e600d3f30 con 0x7f7e600d44a0 2024-04-25T11:38:45.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.728+0000 7f7e57e00640 1 -- 172.21.15.19:0/3457701278 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 224 v224) ==== 95+0+4352 (secure 0 0 0) 0x7f7e5005e630 con 0x7f7e600d44a0 2024-04-25T11:38:45.715 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:45.716 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":224,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":224,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:45.171956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:45.716 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 224 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.731+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e3c0744e0 msgr2=0x7f7e3c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.731+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e48034000 tx=0x7f7e48033040 comp rx=0 tx=0).stop 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.731+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600d44a0 msgr2=0x7f7e601eaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.731+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600d44a0 0x7f7e601eaa60 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c3c0 tx=0x7f7e5000c880 comp rx=0 tx=0).stop 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 shutdown_connections 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e60164860 0x7f7e600ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e600cedc0 0x7f7e6014cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 --2- 172.21.15.19:0/3457701278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e600d44a0 0x7f7e601eaa60 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 >> 172.21.15.19:0/3457701278 conn(0x7f7e600d2720 msgr2=0x7f7e60160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 shutdown_connections 2024-04-25T11:38:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:45.732+0000 7f7e65c00640 1 -- 172.21.15.19:0/3457701278 wait complete. 2024-04-25T11:38:45.745 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:45.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:46.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.036+0000 7fca8ee00640 1 Processor -- start 2024-04-25T11:38:46.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.036+0000 7fca8ee00640 1 -- start start 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8ee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca880b90e0 0x7fca880b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8808cd60 0x7fca8808d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca8808d6b0 con 0x7fca8806e680 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8ee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fca880d1d70 con 0x7fca880b90e0 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fca880ce6d0 con 0x7fca8808cd60 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca880b90e0 0x7fca880b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca880b90e0 0x7fca880b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36440/0 (socket says 172.21.15.19:36440) 2024-04-25T11:38:46.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8d000640 1 -- 172.21.15.19:0/4002749224 learned_addr learned my addr 172.21.15.19:0/4002749224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8e400640 1 --2- 172.21.15.19:0/4002749224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8808cd60 0x7fca8808d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.039+0000 7fca8d000640 1 -- 172.21.15.19:0/4002749224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8808cd60 msgr2=0x7fca8808d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:46.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.040+0000 7fca8d000640 1 --2- 172.21.15.19:0/4002749224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8808cd60 0x7fca8808d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.040+0000 7fca8d000640 1 -- 172.21.15.19:0/4002749224 >> v1:172.21.15.169:6789/0 conn(0x7fca8806e680 legacy=0x7fca880cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.040+0000 7fca8d000640 1 -- 172.21.15.19:0/4002749224 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca880d1fa0 con 0x7fca880b90e0 2024-04-25T11:38:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.040+0000 7fca8e400640 1 --2- 172.21.15.19:0/4002749224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8808cd60 0x7fca8808d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.041+0000 7fca8d000640 1 --2- 172.21.15.19:0/4002749224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca880b90e0 0x7fca880b4e80 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fca8806d4a0 tx=0x7fca7402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ecc3e00aaef416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.041+0000 7fca87e00640 1 -- 172.21.15.19:0/4002749224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74037070 con 0x7fca880b90e0 2024-04-25T11:38:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.041+0000 7fca87e00640 1 -- 172.21.15.19:0/4002749224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74032440 con 0x7fca880b90e0 2024-04-25T11:38:46.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.041+0000 7fca87e00640 1 -- 172.21.15.19:0/4002749224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74032810 con 0x7fca880b90e0 2024-04-25T11:38:46.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 -- 172.21.15.19:0/4002749224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca880b90e0 msgr2=0x7fca880b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 --2- 172.21.15.19:0/4002749224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca880b90e0 0x7fca880b4e80 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fca8806d4a0 tx=0x7fca7402a160 comp rx=0 tx=0).stop 2024-04-25T11:38:46.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 -- 172.21.15.19:0/4002749224 shutdown_connections 2024-04-25T11:38:46.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 --2- 172.21.15.19:0/4002749224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8808cd60 0x7fca8808d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 --2- 172.21.15.19:0/4002749224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca880b90e0 0x7fca880b4e80 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 -- 172.21.15.19:0/4002749224 >> 172.21.15.19:0/4002749224 conn(0x7fca880b5460 msgr2=0x7fca880b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:46.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 -- 172.21.15.19:0/4002749224 shutdown_connections 2024-04-25T11:38:46.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.042+0000 7fca8ee00640 1 -- 172.21.15.19:0/4002749224 wait complete. 2024-04-25T11:38:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.044+0000 7fca8ee00640 1 Processor -- start 2024-04-25T11:38:46.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.045+0000 7fca8ee00640 1 -- start start 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.046+0000 7fca8ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8806e680 0x7fca88153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.046+0000 7fca8ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8808cd60 0x7fca880c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.046+0000 7fca8ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca880b90e0 0x7fca8815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.046+0000 7fca8ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca880d2560 con 0x7fca8808cd60 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.046+0000 7fca8ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca880b94f0 con 0x7fca880b90e0 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.046+0000 7fca8ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fca880d2840 con 0x7fca8806e680 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8808cd60 0x7fca880c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8808cd60 0x7fca880c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36452/0 (socket says 172.21.15.19:36452) 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 -- 172.21.15.19:0/2242011460 learned_addr learned my addr 172.21.15.19:0/2242011460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:46.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8da00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8806e680 0x7fca88153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 -- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8806e680 msgr2=0x7fca88153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8806e680 0x7fca88153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 -- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca880b90e0 msgr2=0x7fca8815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca880b90e0 0x7fca8815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 -- 172.21.15.19:0/2242011460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca880c8fb0 con 0x7fca8808cd60 2024-04-25T11:38:46.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.047+0000 7fca8d000640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8808cd60 0x7fca880c84b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fca74000c00 tx=0x7fca740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:46.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.048+0000 7fca85600640 1 -- 172.21.15.19:0/2242011460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74041020 con 0x7fca8808cd60 2024-04-25T11:38:46.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.048+0000 7fca85600640 1 -- 172.21.15.19:0/2242011460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74030970 con 0x7fca8808cd60 2024-04-25T11:38:46.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.048+0000 7fca85600640 1 -- 172.21.15.19:0/2242011460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74037040 con 0x7fca8808cd60 2024-04-25T11:38:46.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.048+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca880c9290 con 0x7fca8808cd60 2024-04-25T11:38:46.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.048+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca88160a60 con 0x7fca8808cd60 2024-04-25T11:38:46.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.049+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca88050740 con 0x7fca8808cd60 2024-04-25T11:38:46.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.050+0000 7fca85600640 1 -- 172.21.15.19:0/2242011460 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fca74044020 con 0x7fca8808cd60 2024-04-25T11:38:46.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.051+0000 7fca85600640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca680744b0 0x7fca680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.052+0000 7fca85600640 1 -- 172.21.15.19:0/2242011460 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7fca740b64b0 con 0x7fca8808cd60 2024-04-25T11:38:46.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.054+0000 7fca8da00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca680744b0 0x7fca680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.056+0000 7fca85600640 1 -- 172.21.15.19:0/2242011460 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca7404b030 con 0x7fca8808cd60 2024-04-25T11:38:46.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.056+0000 7fca8da00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca680744b0 0x7fca680769c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fca88074b10 tx=0x7fca7c0072d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.302+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca8808d890 con 0x7fca8808cd60 2024-04-25T11:38:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.304+0000 7fca85600640 1 -- 172.21.15.19:0/2242011460 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 225 v225) ==== 95+0+5175 (secure 0 0 0) 0x7fca8808d6b0 con 0x7fca8808cd60 2024-04-25T11:38:46.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:46.291 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":225,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9564,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/611091976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":611091976},{"type":"v1","addr":"172.21.15.19:6835","nonce":611091976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":225}],"filesystems":[{"mdsmap":{"epoch":224,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:45.171956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 225 2024-04-25T11:38:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.306+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca680744b0 msgr2=0x7fca680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.306+0000 7fca8ee00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca680744b0 0x7fca680769c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fca88074b10 tx=0x7fca7c0072d0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.306+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8808cd60 msgr2=0x7fca880c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8808cd60 0x7fca880c84b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fca74000c00 tx=0x7fca740046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 shutdown_connections 2024-04-25T11:38:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca680744b0 0x7fca680769c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca880b90e0 0x7fca8815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca8808cd60 0x7fca880c84b0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 --2- 172.21.15.19:0/2242011460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca8806e680 0x7fca88153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 >> 172.21.15.19:0/2242011460 conn(0x7fca880b5460 msgr2=0x7fca880b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 shutdown_connections 2024-04-25T11:38:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.307+0000 7fca8ee00640 1 -- 172.21.15.19:0/2242011460 wait complete. 2024-04-25T11:38:46.319 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:46.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.616+0000 7fafeca00640 1 Processor -- start 2024-04-25T11:38:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.617+0000 7fafeca00640 1 -- start start 2024-04-25T11:38:46.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.619+0000 7fafeca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fafe8167000 0x7fafe8169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.619+0000 7fafeca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe815f5f0 con 0x7fafe80cf5e0 2024-04-25T11:38:46.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.619+0000 7fafeca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe815e830 con 0x7fafe81443c0 2024-04-25T11:38:46.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.619+0000 7fafeca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fafe80708d0 con 0x7fafe8167000 2024-04-25T11:38:46.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.619+0000 7fafe6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fafe8167000 0x7fafe8169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.619+0000 7fafe6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fafe8167000 0x7fafe8169420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36466/0 (socket says 172.21.15.19:36466) 2024-04-25T11:38:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.619+0000 7fafe6800640 1 -- 172.21.15.19:0/273881590 learned_addr learned my addr 172.21.15.19:0/273881590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.620+0000 7fafe6800640 1 -- 172.21.15.19:0/273881590 >> v1:172.21.15.169:6789/0 conn(0x7fafe81443c0 legacy=0x7fafe806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.620+0000 7fafe6800640 1 -- 172.21.15.19:0/273881590 >> v1:172.21.15.19:6789/0 conn(0x7fafe80cf5e0 legacy=0x7fafe80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.620+0000 7fafe6800640 1 -- 172.21.15.19:0/273881590 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe815eb10 con 0x7fafe8167000 2024-04-25T11:38:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.621+0000 7fafe6800640 1 --2- 172.21.15.19:0/273881590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe8169420 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009e50 tx=0x7fafe002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f314d8476eb6e6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.621+0000 7fafe5400640 1 -- 172.21.15.19:0/273881590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0036070 con 0x7fafe8167000 2024-04-25T11:38:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.621+0000 7fafe5400640 1 -- 172.21.15.19:0/273881590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe0031600 con 0x7fafe8167000 2024-04-25T11:38:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.621+0000 7fafe5400640 1 -- 172.21.15.19:0/273881590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe00319d0 con 0x7fafe8167000 2024-04-25T11:38:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.622+0000 7fafeca00640 1 -- 172.21.15.19:0/273881590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 msgr2=0x7fafe8169420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.622+0000 7fafeca00640 1 --2- 172.21.15.19:0/273881590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe8169420 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009e50 tx=0x7fafe002a760 comp rx=0 tx=0).stop 2024-04-25T11:38:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.622+0000 7fafeca00640 1 -- 172.21.15.19:0/273881590 shutdown_connections 2024-04-25T11:38:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.622+0000 7fafeca00640 1 --2- 172.21.15.19:0/273881590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe8169420 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.622+0000 7fafeca00640 1 -- 172.21.15.19:0/273881590 >> 172.21.15.19:0/273881590 conn(0x7fafe80d1ae0 msgr2=0x7fafe80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.622+0000 7fafeca00640 1 -- 172.21.15.19:0/273881590 shutdown_connections 2024-04-25T11:38:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.622+0000 7fafeca00640 1 -- 172.21.15.19:0/273881590 wait complete. 2024-04-25T11:38:46.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.625+0000 7fafeca00640 1 Processor -- start 2024-04-25T11:38:46.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.625+0000 7fafeca00640 1 -- start start 2024-04-25T11:38:46.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.626+0000 7fafeca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafe80cf5e0 0x7fafe8163540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.626+0000 7fafeca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafe81443c0 0x7fafe8143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.626+0000 7fafeca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe81f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.626+0000 7fafeca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafe806d5d0 con 0x7fafe8167000 2024-04-25T11:38:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.626+0000 7fafeca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fafe815f160 con 0x7fafe81443c0 2024-04-25T11:38:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.626+0000 7fafeca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fafe81447d0 con 0x7fafe80cf5e0 2024-04-25T11:38:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe81f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe81f9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36472/0 (socket says 172.21.15.19:36472) 2024-04-25T11:38:46.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 -- 172.21.15.19:0/643456262 learned_addr learned my addr 172.21.15.19:0/643456262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 -- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafe80cf5e0 msgr2=0x7fafe8163540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafe80cf5e0 0x7fafe8163540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 -- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafe81443c0 msgr2=0x7fafe8143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafe81443c0 0x7fafe8143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 -- 172.21.15.19:0/643456262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe8163e70 con 0x7fafe8167000 2024-04-25T11:38:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.627+0000 7fafe6800640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe81f9f30 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009e50 tx=0x7fafe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:46.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.628+0000 7fafde000640 1 -- 172.21.15.19:0/643456262 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0041020 con 0x7fafe8167000 2024-04-25T11:38:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.628+0000 7fafde000640 1 -- 172.21.15.19:0/643456262 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe0007780 con 0x7fafe8167000 2024-04-25T11:38:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.628+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe8164150 con 0x7fafe8167000 2024-04-25T11:38:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.628+0000 7fafde000640 1 -- 172.21.15.19:0/643456262 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0036040 con 0x7fafe8167000 2024-04-25T11:38:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.628+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe80c68e0 con 0x7fafe8167000 2024-04-25T11:38:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.629+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe8050740 con 0x7fafe8167000 2024-04-25T11:38:46.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.630+0000 7fafde000640 1 -- 172.21.15.19:0/643456262 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fafe0044020 con 0x7fafe8167000 2024-04-25T11:38:46.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.631+0000 7fafde000640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafc40744e0 0x7fafc40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:46.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.632+0000 7fafde000640 1 -- 172.21.15.19:0/643456262 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7fafe00b66e0 con 0x7fafe8167000 2024-04-25T11:38:46.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.632+0000 7fafe5e00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafc40744e0 0x7fafc40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:46.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.633+0000 7fafe5e00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafc40744e0 0x7fafc40769f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fafd8006fd0 tx=0x7fafd8006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:46.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.636+0000 7fafde000640 1 -- 172.21.15.19:0/643456262 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafe004b030 con 0x7fafe8167000 2024-04-25T11:38:46.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.884+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafe806f950 con 0x7fafe8167000 2024-04-25T11:38:46.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.885+0000 7fafde000640 1 -- 172.21.15.19:0/643456262 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 225 v225) ==== 95+0+5175 (secure 0 0 0) 0x7fafe00807c0 con 0x7fafe8167000 2024-04-25T11:38:46.872 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:46.872 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":225,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9564,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/611091976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":611091976},{"type":"v1","addr":"172.21.15.19:6835","nonce":611091976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":225}],"filesystems":[{"mdsmap":{"epoch":224,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:45.171956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:46.872 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 225 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafc40744e0 msgr2=0x7fafc40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafc40744e0 0x7fafc40769f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fafd8006fd0 tx=0x7fafd8006bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 msgr2=0x7fafe81f9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe81f9f30 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009e50 tx=0x7fafe0004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 shutdown_connections 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fafc40744e0 0x7fafc40769f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafe81443c0 0x7fafe8143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafe8167000 0x7fafe81f9f30 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 --2- 172.21.15.19:0/643456262 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafe80cf5e0 0x7fafe8163540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:46.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 >> 172.21.15.19:0/643456262 conn(0x7fafe80d1ae0 msgr2=0x7fafe80b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:46.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 shutdown_connections 2024-04-25T11:38:46.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:46.888+0000 7fafeca00640 1 -- 172.21.15.19:0/643456262 wait complete. 2024-04-25T11:38:46.901 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2024-04-25T11:38:46.901 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2024-04-25T11:38:46.902 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:46.902 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:46.952 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:38:46.952 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:46.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:46.953 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:47.005 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:38:47.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:47.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.310+0000 7f8b4be00640 1 Processor -- start 2024-04-25T11:38:47.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.310+0000 7f8b4be00640 1 -- start start 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.312+0000 7f8b4be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b4c06c7e0 0x7f8b4c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.312+0000 7f8b4be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4c06d130 con 0x7f8b4c06f520 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b4be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4c0cf740 con 0x7f8b4c0aef50 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b4be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4c0cfa20 con 0x7f8b4c06c7e0 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b43e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b4c06c7e0 0x7f8b4c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b43e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b4c06c7e0 0x7f8b4c06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36476/0 (socket says 172.21.15.19:36476) 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b43e00640 1 -- 172.21.15.19:0/678573906 learned_addr learned my addr 172.21.15.19:0/678573906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:47.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b4b400640 1 --1- 172.21.15.19:0/678573906 >> v1:172.21.15.19:6789/0 conn(0x7f8b4c06f520 0x7f8b4c06f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56802/0 (socket says 172.21.15.19:56802) 2024-04-25T11:38:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b43e00640 1 -- 172.21.15.19:0/678573906 >> v1:172.21.15.169:6789/0 conn(0x7f8b4c0aef50 legacy=0x7f8b4c070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b43e00640 1 -- 172.21.15.19:0/678573906 >> v1:172.21.15.19:6789/0 conn(0x7f8b4c06f520 legacy=0x7f8b4c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.313+0000 7f8b43e00640 1 -- 172.21.15.19:0/678573906 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c0cfd00 con 0x7f8b4c06c7e0 2024-04-25T11:38:47.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.315+0000 7f8b43e00640 1 --2- 172.21.15.19:0/678573906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c06cbf0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009870 tx=0x7f8b3402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08734e6f72ae629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.315+0000 7f8b4a000640 1 -- 172.21.15.19:0/678573906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34037070 con 0x7f8b4c06c7e0 2024-04-25T11:38:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.315+0000 7f8b4a000640 1 -- 172.21.15.19:0/678573906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b34032440 con 0x7f8b4c06c7e0 2024-04-25T11:38:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.315+0000 7f8b4a000640 1 -- 172.21.15.19:0/678573906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34032810 con 0x7f8b4c06c7e0 2024-04-25T11:38:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.316+0000 7f8b4be00640 1 -- 172.21.15.19:0/678573906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06c7e0 msgr2=0x7f8b4c06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:47.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.316+0000 7f8b4be00640 1 --2- 172.21.15.19:0/678573906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c06cbf0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009870 tx=0x7f8b3402a2a0 comp rx=0 tx=0).stop 2024-04-25T11:38:47.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.316+0000 7f8b4be00640 1 -- 172.21.15.19:0/678573906 shutdown_connections 2024-04-25T11:38:47.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.316+0000 7f8b4be00640 1 --2- 172.21.15.19:0/678573906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c06cbf0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:47.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.316+0000 7f8b4be00640 1 -- 172.21.15.19:0/678573906 >> 172.21.15.19:0/678573906 conn(0x7f8b4c0d1d50 msgr2=0x7f8b4c072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:47.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.316+0000 7f8b4be00640 1 -- 172.21.15.19:0/678573906 shutdown_connections 2024-04-25T11:38:47.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.316+0000 7f8b4be00640 1 -- 172.21.15.19:0/678573906 wait complete. 2024-04-25T11:38:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.318+0000 7f8b4be00640 1 Processor -- start 2024-04-25T11:38:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.319+0000 7f8b4be00640 1 -- start start 2024-04-25T11:38:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.320+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c1ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.320+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06f520 0x7f8b4c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.320+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4c0aef50 0x7f8b4c161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.320+0000 7f8b4be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c0af360 con 0x7f8b4c06f520 2024-04-25T11:38:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.320+0000 7f8b4be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c070e90 con 0x7f8b4c06c7e0 2024-04-25T11:38:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.320+0000 7f8b4be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4c0af4e0 con 0x7f8b4c0aef50 2024-04-25T11:38:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.320+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06f520 0x7f8b4c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06f520 0x7f8b4c1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36488/0 (socket says 172.21.15.19:36488) 2024-04-25T11:38:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 -- 172.21.15.19:0/3329102905 learned_addr learned my addr 172.21.15.19:0/3329102905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b4aa00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c1ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 -- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4c0aef50 msgr2=0x7f8b4c161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4c0aef50 0x7f8b4c161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 -- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4c06c7e0 msgr2=0x7f8b4c1ea9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c1ea9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 -- 172.21.15.19:0/3329102905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c1eb220 con 0x7f8b4c06f520 2024-04-25T11:38:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b4aa00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c1ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.321+0000 7f8b43e00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06f520 0x7f8b4c1522a0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009870 tx=0x7f8b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.322+0000 7f8b42a00640 1 -- 172.21.15.19:0/3329102905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34041070 con 0x7f8b4c06f520 2024-04-25T11:38:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.322+0000 7f8b42a00640 1 -- 172.21.15.19:0/3329102905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3403c870 con 0x7f8b4c06f520 2024-04-25T11:38:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.322+0000 7f8b42a00640 1 -- 172.21.15.19:0/3329102905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34037040 con 0x7f8b4c06f520 2024-04-25T11:38:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.322+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c1eb500 con 0x7f8b4c06f520 2024-04-25T11:38:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.322+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c0b6700 con 0x7f8b4c06f520 2024-04-25T11:38:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.323+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4c1eaf90 con 0x7f8b4c06f520 2024-04-25T11:38:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.324+0000 7f8b42a00640 1 -- 172.21.15.19:0/3329102905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8b34044070 con 0x7f8b4c06f520 2024-04-25T11:38:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.326+0000 7f8b42a00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b280744b0 0x7f8b280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.326+0000 7f8b42a00640 1 -- 172.21.15.19:0/3329102905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7f8b340b64e0 con 0x7f8b4c06f520 2024-04-25T11:38:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.326+0000 7f8b4aa00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b280744b0 0x7f8b280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.328+0000 7f8b4aa00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b280744b0 0x7f8b280769c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0d1450 tx=0x7f8b3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.331+0000 7f8b42a00640 1 -- 172.21.15.19:0/3329102905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b34080550 con 0x7f8b4c06f520 2024-04-25T11:38:47.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:47.577+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8b4c050740 con 0x7f8b4c06f520 2024-04-25T11:38:48.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.196+0000 7f8b42a00640 1 -- 172.21.15.19:0/3329102905 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v226) ==== 161+0+0 (secure 0 0 0) 0x7f8b3407fe50 con 0x7f8b4c06f520 2024-04-25T11:38:48.183 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b280744b0 msgr2=0x7f8b280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b280744b0 0x7f8b280769c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0d1450 tx=0x7f8b3c008040 comp rx=0 tx=0).stop 2024-04-25T11:38:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06f520 msgr2=0x7f8b4c1522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06f520 0x7f8b4c1522a0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009870 tx=0x7f8b340046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 shutdown_connections 2024-04-25T11:38:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8b280744b0 0x7f8b280769c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8b4c0aef50 0x7f8b4c161bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b4c06f520 0x7f8b4c1522a0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 --2- 172.21.15.19:0/3329102905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8b4c06c7e0 0x7f8b4c1ea9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 >> 172.21.15.19:0/3329102905 conn(0x7f8b4c0d1d50 msgr2=0x7f8b4c0d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 shutdown_connections 2024-04-25T11:38:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.200+0000 7f8b4be00640 1 -- 172.21.15.19:0/3329102905 wait complete. 2024-04-25T11:38:48.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.517+0000 7fda39400640 1 Processor -- start 2024-04-25T11:38:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.518+0000 7fda39400640 1 -- start start 2024-04-25T11:38:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.520+0000 7fda39400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda340cf2f0 0x7fda340b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.520+0000 7fda39400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda341608d0 0x7fda34160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.520+0000 7fda39400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda34161220 con 0x7fda3415a810 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.520+0000 7fda39400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fda3406f870 con 0x7fda341608d0 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.520+0000 7fda39400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fda340b74d0 con 0x7fda340cf2f0 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.521+0000 7fda38a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda341608d0 0x7fda34160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.521+0000 7fda38a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda341608d0 0x7fda34160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36494/0 (socket says 172.21.15.19:36494) 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.521+0000 7fda38a00640 1 -- 172.21.15.19:0/1764917852 learned_addr learned my addr 172.21.15.19:0/1764917852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.521+0000 7fda38a00640 1 -- 172.21.15.19:0/1764917852 >> v2:172.21.15.169:3300/0 conn(0x7fda340cf2f0 msgr2=0x7fda340b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.521+0000 7fda38a00640 1 --2- 172.21.15.19:0/1764917852 >> v2:172.21.15.169:3300/0 conn(0x7fda340cf2f0 0x7fda340b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.521+0000 7fda38a00640 1 -- 172.21.15.19:0/1764917852 >> v1:172.21.15.169:6789/0 conn(0x7fda3415a810 legacy=0x7fda340b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.521+0000 7fda38a00640 1 -- 172.21.15.19:0/1764917852 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda340b77b0 con 0x7fda341608d0 2024-04-25T11:38:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.522+0000 7fda38a00640 1 --2- 172.21.15.19:0/1764917852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34160ce0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fda340d26d0 tx=0x7fda2c02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3effa7bb0cb57e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.523+0000 7fda32a00640 1 -- 172.21.15.19:0/1764917852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c033610 con 0x7fda341608d0 2024-04-25T11:38:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.523+0000 7fda32a00640 1 -- 172.21.15.19:0/1764917852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c0337b0 con 0x7fda341608d0 2024-04-25T11:38:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.523+0000 7fda32a00640 1 -- 172.21.15.19:0/1764917852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c033b80 con 0x7fda341608d0 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 -- 172.21.15.19:0/1764917852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 msgr2=0x7fda34160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 --2- 172.21.15.19:0/1764917852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34160ce0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fda340d26d0 tx=0x7fda2c02c900 comp rx=0 tx=0).stop 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 -- 172.21.15.19:0/1764917852 shutdown_connections 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 --2- 172.21.15.19:0/1764917852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34160ce0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 --2- 172.21.15.19:0/1764917852 >> v2:172.21.15.169:3300/0 conn(0x7fda340cf2f0 0x7fda340b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 -- 172.21.15.19:0/1764917852 >> 172.21.15.19:0/1764917852 conn(0x7fda340d1720 msgr2=0x7fda340d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 -- 172.21.15.19:0/1764917852 shutdown_connections 2024-04-25T11:38:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.524+0000 7fda39400640 1 -- 172.21.15.19:0/1764917852 wait complete. 2024-04-25T11:38:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.526+0000 7fda39400640 1 Processor -- start 2024-04-25T11:38:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.527+0000 7fda39400640 1 -- start start 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.528+0000 7fda39400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda340cf2f0 0x7fda340af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.528+0000 7fda39400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3415a810 0x7fda341f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.528+0000 7fda39400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.528+0000 7fda39400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda3415ac20 con 0x7fda341608d0 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.528+0000 7fda39400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda3406f500 con 0x7fda3415a810 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.528+0000 7fda39400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda3406fd00 con 0x7fda340cf2f0 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.528+0000 7fda38a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36508/0 (socket says 172.21.15.19:36508) 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 -- 172.21.15.19:0/3071329746 learned_addr learned my addr 172.21.15.19:0/3071329746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda33e00640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda340cf2f0 0x7fda340af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda33400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3415a810 0x7fda341f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 -- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda340cf2f0 msgr2=0x7fda340af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda340cf2f0 0x7fda340af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 -- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3415a810 msgr2=0x7fda341f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3415a810 0x7fda341f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 -- 172.21.15.19:0/3071329746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda341f7070 con 0x7fda341608d0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda33400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3415a810 0x7fda341f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.529+0000 7fda38a00640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34147210 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0093e0 tx=0x7fda2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.530+0000 7fda2be00640 1 -- 172.21.15.19:0/3071329746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c040070 con 0x7fda341608d0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.530+0000 7fda2be00640 1 -- 172.21.15.19:0/3071329746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2c03c830 con 0x7fda341608d0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.530+0000 7fda2be00640 1 -- 172.21.15.19:0/3071329746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c03cc20 con 0x7fda341608d0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.530+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda341f7350 con 0x7fda341608d0 2024-04-25T11:38:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.530+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda340aeb30 con 0x7fda341608d0 2024-04-25T11:38:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.531+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda341f6de0 con 0x7fda341608d0 2024-04-25T11:38:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.533+0000 7fda2be00640 1 -- 172.21.15.19:0/3071329746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda2c043070 con 0x7fda341608d0 2024-04-25T11:38:48.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.534+0000 7fda2be00640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda100744b0 0x7fda100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:48.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.534+0000 7fda2be00640 1 -- 172.21.15.19:0/3071329746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7fda2c0b6600 con 0x7fda341608d0 2024-04-25T11:38:48.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.534+0000 7fda33e00640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda100744b0 0x7fda100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.536+0000 7fda33e00640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda100744b0 0x7fda100769c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fda2400a770 tx=0x7fda24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.539+0000 7fda2be00640 1 -- 172.21.15.19:0/3071329746 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda34161220 con 0x7fda341608d0 2024-04-25T11:38:48.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:48.821+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fda340ae640 con 0x7fda341608d0 2024-04-25T11:38:49.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.219+0000 7fda2be00640 1 -- 172.21.15.19:0/3071329746 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v227) ==== 101+0+0 (secure 0 0 0) 0x7fda2c0805e0 con 0x7fda341608d0 2024-04-25T11:38:49.207 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.224+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda100744b0 msgr2=0x7fda100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.224+0000 7fda39400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda100744b0 0x7fda100769c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fda2400a770 tx=0x7fda24008040 comp rx=0 tx=0).stop 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.224+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 msgr2=0x7fda34147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.224+0000 7fda39400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34147210 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0093e0 tx=0x7fda2c004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 shutdown_connections 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda100744b0 0x7fda100769c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda3415a810 0x7fda341f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda341608d0 0x7fda34147210 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 --2- 172.21.15.19:0/3071329746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda340cf2f0 0x7fda340af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 >> 172.21.15.19:0/3071329746 conn(0x7fda340d1720 msgr2=0x7fda340b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 shutdown_connections 2024-04-25T11:38:49.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.225+0000 7fda39400640 1 -- 172.21.15.19:0/3071329746 wait complete. 2024-04-25T11:38:49.239 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.437+0000 7f53ea71df40 1 Processor -- start 2024-04-25T11:38:49.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.437+0000 7f53ea71df40 1 -- start start 2024-04-25T11:38:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53ea71df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481174ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53ea71df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810dc190 0x5624810daf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53ea71df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810f61d0 0x5624810db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53ea71df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5624810f6cf0 con 0x5624810dc190 2024-04-25T11:38:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53ea71df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x562481052d20 con 0x5624810af600 2024-04-25T11:38:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53ea71df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5624810f5d40 con 0x5624810f61d0 2024-04-25T11:38:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53da400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810dc190 0x5624810daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53da400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810dc190 0x5624810daf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36530/0 (socket says 172.21.15.19:36530) 2024-04-25T11:38:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53da400640 1 -- 172.21.15.19:0/2833760895 learned_addr learned my addr 172.21.15.19:0/2833760895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.439+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481174ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53e3800640 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810f61d0 0x5624810db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53da400640 1 -- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810f61d0 msgr2=0x5624810db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53da400640 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810f61d0 0x5624810db470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53da400640 1 -- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 msgr2=0x562481174ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53da400640 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481174ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53da400640 1 -- 172.21.15.19:0/2833760895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624810f6810 con 0x5624810dc190 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53da400640 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810dc190 0x5624810daf30 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00a3f0 tx=0x7f53cc00a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5abe682faa87268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53e1a00640 1 -- 172.21.15.19:0/2833760895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc010070 con 0x5624810dc190 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53e1a00640 1 -- 172.21.15.19:0/2833760895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53cc00c8f0 con 0x5624810dc190 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.440+0000 7f53e1a00640 1 -- 172.21.15.19:0/2833760895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00cce0 con 0x5624810dc190 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 -- 172.21.15.19:0/2833760895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810dc190 msgr2=0x5624810daf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810dc190 0x5624810daf30 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00a3f0 tx=0x7f53cc00a8b0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 -- 172.21.15.19:0/2833760895 shutdown_connections 2024-04-25T11:38:49.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810f61d0 0x5624810db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810dc190 0x5624810daf30 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2833760895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481174ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 -- 172.21.15.19:0/2833760895 >> 172.21.15.19:0/2833760895 conn(0x562480fceac0 msgr2=0x5624810d8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 -- 172.21.15.19:0/2833760895 shutdown_connections 2024-04-25T11:38:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.441+0000 7f53ea71df40 1 -- 172.21.15.19:0/2833760895 wait complete. 2024-04-25T11:38:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.443+0000 7f53ea71df40 1 Processor -- start 2024-04-25T11:38:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.443+0000 7f53ea71df40 1 -- start start 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.444+0000 7f53ea71df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481183260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.444+0000 7f53ea71df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810dc190 0x56248117f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.444+0000 7f53ea71df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810f61d0 0x56248117f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53ea71df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562481175420 con 0x5624810f61d0 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53ea71df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56248117ee10 con 0x5624810af600 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53ea71df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5624810ac0a0 con 0x5624810dc190 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810f61d0 0x56248117f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810f61d0 0x56248117f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36544/0 (socket says 172.21.15.19:36544) 2024-04-25T11:38:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 learned_addr learned my addr 172.21.15.19:0/2244803624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53da400640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810dc190 0x56248117f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481183260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810dc190 msgr2=0x56248117f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5624810dc190 0x56248117f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 msgr2=0x562481183260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481183260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624810f11c0 con 0x5624810f61d0 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.445+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5624810af600 0x562481183260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.446+0000 7f53e3800640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810f61d0 0x56248117f800 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f53dc025dc0 tx=0x7f53dc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.446+0000 7f53dbe00640 1 -- 172.21.15.19:0/2244803624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc06a070 con 0x5624810f61d0 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.446+0000 7f53dbe00640 1 -- 172.21.15.19:0/2244803624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc052b30 con 0x5624810f61d0 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.446+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56248117cc60 con 0x5624810f61d0 2024-04-25T11:38:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.446+0000 7f53dbe00640 1 -- 172.21.15.19:0/2244803624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc05b450 con 0x5624810f61d0 2024-04-25T11:38:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.446+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624810f09c0 con 0x5624810f61d0 2024-04-25T11:38:49.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.449+0000 7f53dbe00640 1 -- 172.21.15.19:0/2244803624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f53dc06d070 con 0x5624810f61d0 2024-04-25T11:38:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.450+0000 7f53dbe00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53c0072ec0 0x7f53c00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.450+0000 7f53dbe00640 1 -- 172.21.15.19:0/2244803624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7f53dc0d8540 con 0x5624810f61d0 2024-04-25T11:38:49.437 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.450+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5624810dbca0 0x562481172030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.450+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481172570 con 0x5624810dbca0 2024-04-25T11:38:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.450+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53c0072ec0 0x7f53c00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.451+0000 7f53da400640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5624810dbca0 0x562481172030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.452+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53c0072ec0 0x7f53c00753d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f53d400d540 tx=0x7f53d4007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.452+0000 7f53da400640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5624810dbca0 0x562481172030 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.456+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v113'629 uv603 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.456+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5624812258f0 con 0x5624810dbca0 2024-04-25T11:38:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.459+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v113'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.459+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481225c90 con 0x5624810dbca0 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.461+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v113'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.448 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.462+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227040 con 0x5624810dbca0 2024-04-25T11:38:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.465+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.465+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812273e0 con 0x5624810dbca0 2024-04-25T11:38:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.467+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.468+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481226030 con 0x5624810dbca0 2024-04-25T11:38:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.471+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.471+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481226420 con 0x5624810dbca0 2024-04-25T11:38:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.474+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v113'635 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.474+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.478+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v113'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.478+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.468 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.481+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v113'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.481+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.484+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'638 uv638 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.484+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.488+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'639 uv639 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.488+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.491+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'640 uv640 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.492+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.496+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v113'641 uv616 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.496+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.500+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'642 uv642 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.500+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.491 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.504+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v113'643 uv643 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.504+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812267c0 con 0x5624810dbca0 2024-04-25T11:38:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.508+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'644 uv644 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.508+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481226c40 con 0x5624810dbca0 2024-04-25T11:38:49.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.512+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'645 uv645 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.512+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481226c40 con 0x5624810dbca0 2024-04-25T11:38:49.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.516+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'646 uv646 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.516+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.520+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'647 uv647 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.520+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.525+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v113'648 uv622 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.525+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.529+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v113'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.529+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.532+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v113'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.520 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.532+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481228790 con 0x5624810dbca0 2024-04-25T11:38:49.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.537+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.537+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481228790 con 0x5624810dbca0 2024-04-25T11:38:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.540+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.540+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.545+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.545+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.549+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v113'654 uv628 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.549+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.553+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v113'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.554+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.544 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:49.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.557+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v113'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.558+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481228790 con 0x5624810dbca0 2024-04-25T11:38:49.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.561+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.562+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481228790 con 0x5624810dbca0 2024-04-25T11:38:49.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.566+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.566+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481227780 con 0x5624810dbca0 2024-04-25T11:38:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.569+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'659 uv659 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.570+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562481227c90 0x56248122a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.570+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122a5f0 con 0x562481227c90 2024-04-25T11:38:49.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.570+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562481227c90 0x56248122a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.572+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562481227c90 0x56248122a0b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.576+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'463 uv446 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.577+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248122a990 con 0x562481227c90 2024-04-25T11:38:49.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.581+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'464 uv464 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.581+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122ad30 con 0x562481227c90 2024-04-25T11:38:49.572 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:49.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.585+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'465 uv465 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.586+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122b0d0 con 0x562481227c90 2024-04-25T11:38:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.591+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v113'466 uv466 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.591+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122b470 con 0x562481227c90 2024-04-25T11:38:49.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.595+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'467 uv467 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.595+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122b810 con 0x562481227c90 2024-04-25T11:38:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.599+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v113'468 uv451 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.599+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248122bbb0 con 0x562481227c90 2024-04-25T11:38:49.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.603+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v113'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.603+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.595 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.608+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v113'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.608+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.612+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.612+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.616+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.616+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.621+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v113'473 uv456 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.621+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.626+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v113'474 uv474 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.626+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.617 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:49.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.630+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v113'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.630+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481230640 con 0x562481227c90 2024-04-25T11:38:49.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.635+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v113'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.635+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481230640 con 0x562481227c90 2024-04-25T11:38:49.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.640+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.640+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.644+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v113'478 uv462 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.644+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.649+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v113'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.649+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.640 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:49.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.653+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v113'480 uv480 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.653+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122cf60 con 0x562481227c90 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.657+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.657+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122cf60 con 0x562481227c90 2024-04-25T11:38:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.661+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.662+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122bf50 con 0x562481227c90 2024-04-25T11:38:49.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.666+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481227c90 2024-04-25T11:38:49.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.666+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56248122c5d0 0x56248122e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.666+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122ef30 con 0x56248122c5d0 2024-04-25T11:38:49.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.667+0000 7f53da400640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56248122c5d0 0x56248122e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.669+0000 7f53da400640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56248122c5d0 0x56248122e9f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.673+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.673+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248122ef30 con 0x56248122c5d0 2024-04-25T11:38:49.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.678+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v113'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.678+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122f320 con 0x56248122c5d0 2024-04-25T11:38:49.669 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:49.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.682+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v113'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.682+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481230330 con 0x56248122c5d0 2024-04-25T11:38:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.686+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.686+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481230330 con 0x56248122c5d0 2024-04-25T11:38:49.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.690+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.691+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122f320 con 0x56248122c5d0 2024-04-25T11:38:49.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.695+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.695+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122f320 con 0x5624810dbca0 2024-04-25T11:38:49.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.699+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'94 uv93 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.700+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248122f320 con 0x5624810dbca0 2024-04-25T11:38:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.704+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'95 uv95 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.704+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122f320 con 0x5624810dbca0 2024-04-25T11:38:49.694 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:49.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.708+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'96 uv96 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.708+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248122f320 con 0x5624810dbca0 2024-04-25T11:38:49.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.713+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'97 uv97 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.713+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56248122f9a0 0x562481231dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.713+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481232340 con 0x56248122f9a0 2024-04-25T11:38:49.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.714+0000 7f53e3800640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56248122f9a0 0x562481231dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.715+0000 7f53e3800640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56248122f9a0 0x562481231dc0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.719+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.719+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5624812327c0 con 0x56248122f9a0 2024-04-25T11:38:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.723+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v113'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.723+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481232c70 con 0x56248122f9a0 2024-04-25T11:38:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.727+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v113'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.714 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:49.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.727+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481234520 con 0x56248122f9a0 2024-04-25T11:38:49.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.732+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.732+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812348c0 con 0x56248122f9a0 2024-04-25T11:38:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.736+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.736+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481233510 con 0x56248122f9a0 2024-04-25T11:38:49.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.740+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.741+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562481233e30 0x562481236c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.741+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x5624812371b0 con 0x562481233e30 2024-04-25T11:38:49.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.742+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562481233e30 0x562481236c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.743+0000 7f53e2e00640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562481233e30 0x562481236c70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.747+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v113'227 uv221 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.747+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x5624812371b0 con 0x562481233e30 2024-04-25T11:38:49.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.752+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v113'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.752+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481237230 con 0x562481233e30 2024-04-25T11:38:49.743 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:49.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.756+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v113'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.756+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481238630 con 0x562481233e30 2024-04-25T11:38:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.760+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.761+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481238630 con 0x562481233e30 2024-04-25T11:38:49.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.765+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.765+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481237620 con 0x562481233e30 2024-04-25T11:38:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.769+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.769+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481237620 con 0x562481233e30 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.774+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v113'233 uv226 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.774+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x562481237620 con 0x562481233e30 2024-04-25T11:38:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.778+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v113'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.778+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481237620 con 0x562481233e30 2024-04-25T11:38:49.769 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:49.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.782+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v113'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.783+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481237620 con 0x562481233e30 2024-04-25T11:38:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.787+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.787+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562481237620 con 0x562481233e30 2024-04-25T11:38:49.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.792+0000 7f53e2e00640 1 -- 172.21.15.19:0/2244803624 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53d401c040 con 0x562481233e30 2024-04-25T11:38:49.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.792+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562481237b30 0x562481239f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:49.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.792+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123a6a0 con 0x562481237b30 2024-04-25T11:38:49.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.793+0000 7f53da400640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562481237b30 0x562481239f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.795+0000 7f53da400640 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562481237b30 0x562481239f90 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.799+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v113'102 uv101 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.799+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123a6a0 con 0x562481237b30 2024-04-25T11:38:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.803+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v113'103 uv103 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.803+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x562481237b30 2024-04-25T11:38:49.794 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.807+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v113'104 uv104 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.807+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x562481237b30 2024-04-25T11:38:49.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.811+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.812+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x562481237b30 2024-04-25T11:38:49.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.815+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.816+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x5624810dbca0 2024-04-25T11:38:49.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.820+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.820+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123aaf0 con 0x5624810dbca0 2024-04-25T11:38:49.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.824+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v113'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.824+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x5624810dbca0 2024-04-25T11:38:49.816 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:49.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.829+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v113'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.829+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123bb00 con 0x5624810dbca0 2024-04-25T11:38:49.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.833+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.834+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123bb00 con 0x5624810dbca0 2024-04-25T11:38:49.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.838+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.838+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x5624810dbca0 2024-04-25T11:38:49.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.843+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x5624810dbca0 2024-04-25T11:38:49.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.843+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.848+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v113'203 uv202 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.848+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.853+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v113'204 uv204 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.853+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.845 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:49.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.858+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v113'205 uv205 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.858+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.863+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'206 uv206 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.863+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.867+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'207 uv207 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.867+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.871+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v113'208 uv201 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.871+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.876+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v113'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.876+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.867 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:49.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.880+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v113'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.881+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123aaf0 con 0x56248122c5d0 2024-04-25T11:38:49.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.884+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.885+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123d900 con 0x56248122c5d0 2024-04-25T11:38:49.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.890+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x56248122c5d0 2024-04-25T11:38:49.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.890+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123afa0 con 0x56248122f9a0 2024-04-25T11:38:49.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.894+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v113'91 uv90 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.894+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123b340 con 0x56248122f9a0 2024-04-25T11:38:49.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.899+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'92 uv92 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.899+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x56248122f9a0 2024-04-25T11:38:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.903+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v113'93 uv93 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.890 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.903+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x56248122f9a0 2024-04-25T11:38:49.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.907+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'94 uv94 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.907+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x56248122f9a0 2024-04-25T11:38:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.911+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v113'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.912+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123b340 con 0x56248122f9a0 2024-04-25T11:38:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.916+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v113'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.916+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x56248122f9a0 2024-04-25T11:38:49.907 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:49.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.920+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v113'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.920+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123c350 con 0x56248122f9a0 2024-04-25T11:38:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.924+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.924+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123c350 con 0x56248122f9a0 2024-04-25T11:38:49.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.929+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.929+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x56248122f9a0 2024-04-25T11:38:49.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.933+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53dc06c030 con 0x56248122f9a0 2024-04-25T11:38:49.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.933+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.937+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v113'104 uv103 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.937+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.942+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v113'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.942+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.933 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.946+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v113'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.946+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.950+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.950+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.953+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.954+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.957+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v113'253 uv245 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.958+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.961+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v113'254 uv254 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.961+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.953 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:49.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.966+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v113'255 uv255 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.966+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.970+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v113'256 uv256 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.970+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.974+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'257 uv257 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.974+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.978+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v113'258 uv252 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.978+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.983+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v113'259 uv259 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.983+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.974 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:49.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.987+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v113'260 uv260 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.987+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123c350 con 0x562481237b30 2024-04-25T11:38:49.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.990+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'261 uv261 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.991+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123c350 con 0x562481237b30 2024-04-25T11:38:49.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.994+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'262 uv262 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.995+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123b340 con 0x562481237b30 2024-04-25T11:38:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.999+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'263 uv263 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:49.999+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56248123bbb0 con 0x562481237b30 2024-04-25T11:38:49.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.003+0000 7f53da400640 1 -- 172.21.15.19:0/2244803624 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'264 uv264 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f53cc012030 con 0x562481237b30 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.003+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562481227c90 msgr2=0x56248122a0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.003+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x562481227c90 0x56248122a0b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.003+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562481233e30 msgr2=0x562481236c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.003+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x562481233e30 0x562481236c70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.003+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562481237b30 msgr2=0x562481239f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.003+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x562481237b30 0x562481239f90 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56248122c5d0 msgr2=0x56248122e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56248122c5d0 0x56248122e9f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 reap_dead start 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5624810dbca0 msgr2=0x562481172030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5624810dbca0 0x562481172030 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53e3800640 1 -- 172.21.15.19:0/2244803624 reap_dead start 2024-04-25T11:38:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56248122f9a0 msgr2=0x562481231dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56248122f9a0 0x562481231dc0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53c0072ec0 msgr2=0x7f53c00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53c0072ec0 0x7f53c00753d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f53d400d540 tx=0x7f53d4007040 comp rx=0 tx=0).stop 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810f61d0 msgr2=0x56248117f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.004+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810f61d0 0x56248117f800 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f53dc025dc0 tx=0x7f53dc002fb0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.005+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 shutdown_connections 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.005+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56248122f9a0 0x562481231dc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.005+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53c0072ec0 0x7f53c00753d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.005+0000 7f53ea71df40 1 --2- 172.21.15.19:0/2244803624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5624810f61d0 0x56248117f800 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.005+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 >> 172.21.15.19:0/2244803624 conn(0x562480fceac0 msgr2=0x562481178350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.005+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 shutdown_connections 2024-04-25T11:38:49.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.005+0000 7f53ea71df40 1 -- 172.21.15.19:0/2244803624 wait complete. 2024-04-25T11:38:50.021 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2024-04-25T11:38:50.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.271+0000 7fb7e4022f40 1 Processor -- start 2024-04-25T11:38:50.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.271+0000 7fb7e4022f40 1 -- start start 2024-04-25T11:38:50.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.272+0000 7fb7e4022f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54d16c0 0x55adf56335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.272+0000 7fb7e4022f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54fd180 0x55adf55a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.272+0000 7fb7e4022f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf54f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.272+0000 7fb7e4022f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55adf5516ce0 con 0x55adf54fd180 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.272+0000 7fb7e4022f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55adf5474d20 con 0x55adf54fe180 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.272+0000 7fb7e4022f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55adf55172a0 con 0x55adf54d16c0 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54fd180 0x55adf55a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54fd180 0x55adf55a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36560/0 (socket says 172.21.15.19:36560) 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 -- 172.21.15.19:0/801260332 learned_addr learned my addr 172.21.15.19:0/801260332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dd200640 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54d16c0 0x55adf56335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 -- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54d16c0 msgr2=0x55adf56335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54d16c0 0x55adf56335f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 -- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 msgr2=0x55adf54f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:50.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7ddc00640 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf54f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf54f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7dc800640 1 -- 172.21.15.19:0/801260332 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf54d1390 con 0x55adf54fd180 2024-04-25T11:38:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.273+0000 7fb7ddc00640 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf54f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.274+0000 7fb7dc800640 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54fd180 0x55adf55a0f80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006e90 tx=0x7fb7c800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13301ab129c6381 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.274+0000 7fb7db400640 1 -- 172.21.15.19:0/801260332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8012070 con 0x55adf54fd180 2024-04-25T11:38:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.274+0000 7fb7db400640 1 -- 172.21.15.19:0/801260332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c800e690 con 0x55adf54fd180 2024-04-25T11:38:50.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.274+0000 7fb7db400640 1 -- 172.21.15.19:0/801260332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800eac0 con 0x55adf54fd180 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 -- 172.21.15.19:0/801260332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54fd180 msgr2=0x55adf55a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54fd180 0x55adf55a0f80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006e90 tx=0x7fb7c800b6c0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 -- 172.21.15.19:0/801260332 shutdown_connections 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf54f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54fd180 0x55adf55a0f80 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/801260332 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54d16c0 0x55adf56335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 -- 172.21.15.19:0/801260332 >> 172.21.15.19:0/801260332 conn(0x55adf53f0ac0 msgr2=0x55adf5513940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 -- 172.21.15.19:0/801260332 shutdown_connections 2024-04-25T11:38:50.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.275+0000 7fb7e4022f40 1 -- 172.21.15.19:0/801260332 wait complete. 2024-04-25T11:38:50.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.276+0000 7fb7e4022f40 1 Processor -- start 2024-04-25T11:38:50.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.276+0000 7fb7e4022f40 1 -- start start 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7e4022f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54d16c0 0x55adf558eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7e4022f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54fd180 0x55adf55a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7e4022f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf55a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7e4022f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55adf55140f0 con 0x55adf54d16c0 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7e4022f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55adf55143d0 con 0x55adf54fe180 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7e4022f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55adf55a34d0 con 0x55adf54fd180 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54d16c0 0x55adf558eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54d16c0 0x55adf558eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36566/0 (socket says 172.21.15.19:36566) 2024-04-25T11:38:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 -- 172.21.15.19:0/2858288088 learned_addr learned my addr 172.21.15.19:0/2858288088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dc800640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54fd180 0x55adf55a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7ddc00640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf55a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 -- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54fd180 msgr2=0x55adf55a71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55adf54fd180 0x55adf55a71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 -- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 msgr2=0x55adf55a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55adf54fe180 0x55adf55a7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.278+0000 7fb7dd200640 1 -- 172.21.15.19:0/2858288088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf558eff0 con 0x55adf54d16c0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.279+0000 7fb7dd200640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54d16c0 0x55adf558eab0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0045e50 tx=0x7fb7d0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.279+0000 7fb7da000640 1 -- 172.21.15.19:0/2858288088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0065070 con 0x55adf54d16c0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.279+0000 7fb7da000640 1 -- 172.21.15.19:0/2858288088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d0054c90 con 0x55adf54d16c0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.279+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55adf54cfe80 con 0x55adf54d16c0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.279+0000 7fb7da000640 1 -- 172.21.15.19:0/2858288088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0059040 con 0x55adf54d16c0 2024-04-25T11:38:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.279+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adf54d0440 con 0x55adf54d16c0 2024-04-25T11:38:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.282+0000 7fb7da000640 1 -- 172.21.15.19:0/2858288088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7d0068070 con 0x55adf54d16c0 2024-04-25T11:38:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.283+0000 7fb7da000640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7bc072e90 0x7fb7bc0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.283+0000 7fb7da000640 1 -- 172.21.15.19:0/2858288088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7fb7d00d8330 con 0x55adf54d16c0 2024-04-25T11:38:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.283+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55adf559dee0 0x55adf559d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.283+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55adf54f9c40 con 0x55adf559dee0 2024-04-25T11:38:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.283+0000 7fb7dc800640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7bc072e90 0x7fb7bc0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.283+0000 7fb7ddc00640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55adf559dee0 0x55adf559d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.284+0000 7fb7ddc00640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55adf559dee0 0x55adf559d340 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.285+0000 7fb7dc800640 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7bc072e90 0x7fb7bc0753a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006e90 tx=0x7fb7c8019040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7ddc00640 1 -- 172.21.15.19:0/2858288088 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v113'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb7d400c580 con 0x55adf559dee0 2024-04-25T11:38:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55adf559dee0 msgr2=0x55adf559d340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55adf559dee0 0x55adf559d340 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7bc072e90 msgr2=0x7fb7bc0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7bc072e90 0x7fb7bc0753a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006e90 tx=0x7fb7c8019040 comp rx=0 tx=0).stop 2024-04-25T11:38:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54d16c0 msgr2=0x55adf558eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 --2- 172.21.15.19:0/2858288088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adf54d16c0 0x55adf558eab0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0045e50 tx=0x7fb7d0002fb0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7ddc00640 1 -- 172.21.15.19:0/2858288088 reap_dead start 2024-04-25T11:38:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 shutdown_connections 2024-04-25T11:38:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 >> 172.21.15.19:0/2858288088 conn(0x55adf53f0ac0 msgr2=0x55adf5633230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 shutdown_connections 2024-04-25T11:38:50.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.286+0000 7fb7e4022f40 1 -- 172.21.15.19:0/2858288088 wait complete. 2024-04-25T11:38:50.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.591+0000 7fc656c00640 1 Processor -- start 2024-04-25T11:38:50.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.591+0000 7fc656c00640 1 -- start start 2024-04-25T11:38:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.593+0000 7fc656c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc65015b710 0x7fc6500d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.593+0000 7fc656c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc650168670 con 0x7fc65015bb20 2024-04-25T11:38:50.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.593+0000 7fc656c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc650168950 con 0x7fc65006dce0 2024-04-25T11:38:50.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.593+0000 7fc656c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc650169280 con 0x7fc65015b710 2024-04-25T11:38:50.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.594+0000 7fc64fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc65015b710 0x7fc6500d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.594+0000 7fc64fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc65015b710 0x7fc6500d0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36570/0 (socket says 172.21.15.19:36570) 2024-04-25T11:38:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.594+0000 7fc64fe00640 1 -- 172.21.15.19:0/475091757 learned_addr learned my addr 172.21.15.19:0/475091757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.594+0000 7fc64f400640 1 --1- 172.21.15.19:0/475091757 >> v1:172.21.15.19:6789/0 conn(0x7fc65015bb20 0x7fc65006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56814/0 (socket says 172.21.15.19:56814) 2024-04-25T11:38:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.594+0000 7fc64fe00640 1 -- 172.21.15.19:0/475091757 >> v1:172.21.15.169:6789/0 conn(0x7fc65006dce0 legacy=0x7fc650165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.594+0000 7fc64fe00640 1 -- 172.21.15.19:0/475091757 >> v1:172.21.15.19:6789/0 conn(0x7fc65015bb20 legacy=0x7fc65006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.595+0000 7fc64fe00640 1 -- 172.21.15.19:0/475091757 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc650168c30 con 0x7fc65015b710 2024-04-25T11:38:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.595+0000 7fc64ea00640 1 -- 172.21.15.19:0/475091757 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 164606452 0 0) 0x7fc650168670 con 0x7fc65015bb20 2024-04-25T11:38:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.596+0000 7fc64fe00640 1 --2- 172.21.15.19:0/475091757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015b710 0x7fc6500d0f20 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d10a03dc8cf61a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.596+0000 7fc64ea00640 1 -- 172.21.15.19:0/475091757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640037070 con 0x7fc65015b710 2024-04-25T11:38:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.596+0000 7fc64ea00640 1 -- 172.21.15.19:0/475091757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc640032440 con 0x7fc65015b710 2024-04-25T11:38:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.596+0000 7fc64ea00640 1 -- 172.21.15.19:0/475091757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640032810 con 0x7fc65015b710 2024-04-25T11:38:50.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.597+0000 7fc656c00640 1 -- 172.21.15.19:0/475091757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015b710 msgr2=0x7fc6500d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.597+0000 7fc656c00640 1 --2- 172.21.15.19:0/475091757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015b710 0x7fc6500d0f20 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002a2a0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.597+0000 7fc656c00640 1 -- 172.21.15.19:0/475091757 shutdown_connections 2024-04-25T11:38:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.597+0000 7fc656c00640 1 --2- 172.21.15.19:0/475091757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015b710 0x7fc6500d0f20 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.597+0000 7fc656c00640 1 -- 172.21.15.19:0/475091757 >> 172.21.15.19:0/475091757 conn(0x7fc650073090 msgr2=0x7fc6500734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.597+0000 7fc656c00640 1 -- 172.21.15.19:0/475091757 shutdown_connections 2024-04-25T11:38:50.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.597+0000 7fc656c00640 1 -- 172.21.15.19:0/475091757 wait complete. 2024-04-25T11:38:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.599+0000 7fc656c00640 1 Processor -- start 2024-04-25T11:38:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.600+0000 7fc656c00640 1 -- start start 2024-04-25T11:38:50.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc656c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65015b710 0x7fc6501fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc656c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015bb20 0x7fc6501528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc656c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6500b2580 0x7fc6500b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc656c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500b57c0 con 0x7fc65015bb20 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc656c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500b6180 con 0x7fc65015b710 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc656c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500d1460 con 0x7fc6500b2580 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc64f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015bb20 0x7fc6501528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc64f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015bb20 0x7fc6501528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36576/0 (socket says 172.21.15.19:36576) 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.601+0000 7fc64f400640 1 -- 172.21.15.19:0/2882272726 learned_addr learned my addr 172.21.15.19:0/2882272726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:50.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc64f400640 1 -- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6500b2580 msgr2=0x7fc6500b2990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:38:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc64f400640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6500b2580 0x7fc6500b2990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc64f400640 1 -- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65015b710 msgr2=0x7fc6501fd700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:38:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc64f400640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65015b710 0x7fc6501fd700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc64f400640 1 -- 172.21.15.19:0/2882272726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6501600b0 con 0x7fc65015bb20 2024-04-25T11:38:50.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc64f400640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015bb20 0x7fc6501528e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c460 tx=0x7fc63c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc647e00640 1 -- 172.21.15.19:0/2882272726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c018070 con 0x7fc65015bb20 2024-04-25T11:38:50.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc647e00640 1 -- 172.21.15.19:0/2882272726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c007500 con 0x7fc65015bb20 2024-04-25T11:38:50.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6500b2ed0 con 0x7fc65015bb20 2024-04-25T11:38:50.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.602+0000 7fc647e00640 1 -- 172.21.15.19:0/2882272726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c004a20 con 0x7fc65015bb20 2024-04-25T11:38:50.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.603+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc650161ec0 con 0x7fc65015bb20 2024-04-25T11:38:50.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.604+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc650050740 con 0x7fc65015bb20 2024-04-25T11:38:50.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.605+0000 7fc647e00640 1 -- 172.21.15.19:0/2882272726 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc63c01b070 con 0x7fc65015bb20 2024-04-25T11:38:50.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.606+0000 7fc647e00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc630074500 0x7fc630076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:50.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.606+0000 7fc647e00640 1 -- 172.21.15.19:0/2882272726 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7515+0+0 (secure 0 0 0) 0x7fc63c094d20 con 0x7fc65015bb20 2024-04-25T11:38:50.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.607+0000 7fc64fe00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc630074500 0x7fc630076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:50.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.609+0000 7fc64fe00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc630074500 0x7fc630076a10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc640034000 tx=0x7fc64002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.612+0000 7fc647e00640 1 -- 172.21.15.19:0/2882272726 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc6500b2ed0 con 0x7fc65015bb20 2024-04-25T11:38:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:50.857+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc6500b5d40 con 0x7fc65015bb20 2024-04-25T11:38:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.228+0000 7fc647e00640 1 -- 172.21.15.19:0/2882272726 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v228) ==== 162+0+0 (secure 0 0 0) 0x7fc63c05ed80 con 0x7fc65015bb20 2024-04-25T11:38:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.232+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc630074500 msgr2=0x7fc630076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc630074500 0x7fc630076a10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc640034000 tx=0x7fc64002e000 comp rx=0 tx=0).stop 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015bb20 msgr2=0x7fc6501528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015bb20 0x7fc6501528e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c460 tx=0x7fc63c00c920 comp rx=0 tx=0).stop 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 shutdown_connections 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc630074500 0x7fc630076a10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6500b2580 0x7fc6500b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc65015bb20 0x7fc6501528e0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 --2- 172.21.15.19:0/2882272726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc65015b710 0x7fc6501fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 >> 172.21.15.19:0/2882272726 conn(0x7fc650073090 msgr2=0x7fc65015ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 shutdown_connections 2024-04-25T11:38:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.233+0000 7fc656c00640 1 -- 172.21.15.19:0/2882272726 wait complete. 2024-04-25T11:38:51.234 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:51.247+0000 7f9ee3000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-25T11:38:51.235 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:51.248+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:51.235 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:51.248+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:38:51.246 INFO:tasks.ceph.mds.a.smithi019.stderr: -24> 2024-04-25T11:38:51.247+0000 7f9ee3000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-25T11:38:51.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:51.375 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.550+0000 7f4aae400640 1 Processor -- start 2024-04-25T11:38:51.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.550+0000 7f4aae400640 1 -- start start 2024-04-25T11:38:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.553+0000 7f4aae400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 0x7f4aa815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.553+0000 7f4aae400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa81611e0 con 0x7f4aa806d560 2024-04-25T11:38:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.553+0000 7f4aae400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aa8160420 con 0x7f4aa815a2f0 2024-04-25T11:38:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.553+0000 7f4aae400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa8160d50 con 0x7f4aa80b6a00 2024-04-25T11:38:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.553+0000 7f4aa7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4aa806d560 0x7f4aa80b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56822/0 (socket says 172.21.15.19:56822) 2024-04-25T11:38:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.553+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1778981264 learned_addr learned my addr 172.21.15.19:0/1778981264 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.553+0000 7f4aa7400640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 0x7f4aa815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464435243 0 0) 0x7f4aa81611e0 con 0x7f4aa806d560 2024-04-25T11:38:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a8c003440 con 0x7f4aa806d560 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3363533009 0 0) 0x7f4a8c003440 con 0x7f4aa806d560 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 msgr2=0x7f4aa815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 0x7f4aa815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 >> v1:172.21.15.169:6789/0 conn(0x7f4aa815a2f0 legacy=0x7f4aa806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa8160700 con 0x7f4aa806d560 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f4aa00019f0 con 0x7f4aa806d560 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 >> v1:172.21.15.19:6789/0 conn(0x7f4aa806d560 legacy=0x7f4aa80b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 0x7f4a8c003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a8c004050 0x7f4a8c00a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c00b040 0x7f4a8c00d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.554+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa8160d50 con 0x7f4a8c00b040 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c00dd60 con 0x7f4a8c004050 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c00df60 con 0x7f4aa80b6a00 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c00b040 0x7f4a8c00d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa7400640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 0x7f4a8c003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa6a00640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a8c004050 0x7f4a8c00a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 msgr2=0x7f4a8c003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa80b6a00 0x7f4a8c003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a8c004050 msgr2=0x7f4a8c00a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a8c004050 0x7f4a8c00a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.555+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1778981264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c003440 con 0x7f4a8c00b040 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.557+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c00b040 0x7f4a8c00d520 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4aa806d9a0 tx=0x7f4a8c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e2dd26b2bf4ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.557+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000a460 con 0x7f4a8c00b040 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.557+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa002f070 con 0x7f4a8c00b040 2024-04-25T11:38:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.557+0000 7f4aa6000640 1 -- 172.21.15.19:0/1778981264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa002bc90 con 0x7f4a8c00b040 2024-04-25T11:38:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.558+0000 7f4aae400640 1 -- 172.21.15.19:0/1778981264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c00b040 msgr2=0x7f4a8c00d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.558+0000 7f4aae400640 1 --2- 172.21.15.19:0/1778981264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c00b040 0x7f4a8c00d520 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f4aa806d9a0 tx=0x7f4a8c003730 comp rx=0 tx=0).stop 2024-04-25T11:38:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.558+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1778981264 reap_dead start 2024-04-25T11:38:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.558+0000 7f4aae400640 1 -- 172.21.15.19:0/1778981264 shutdown_connections 2024-04-25T11:38:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.558+0000 7f4aae400640 1 -- 172.21.15.19:0/1778981264 >> 172.21.15.19:0/1778981264 conn(0x7f4aa80d1ae0 msgr2=0x7f4aa80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.558+0000 7f4aae400640 1 -- 172.21.15.19:0/1778981264 shutdown_connections 2024-04-25T11:38:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.558+0000 7f4aae400640 1 -- 172.21.15.19:0/1778981264 wait complete. 2024-04-25T11:38:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.561+0000 7f4aae400640 1 Processor -- start 2024-04-25T11:38:51.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.561+0000 7f4aae400640 1 -- start start 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.562+0000 7f4aae400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80cbe70 0x7f4aa81fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.562+0000 7f4aae400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa8157c10 0x7f4aa8158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.562+0000 7f4aae400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa81f82c0 0x7f4aa81f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.562+0000 7f4aae400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa80ce580 con 0x7f4aa80cbe70 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.562+0000 7f4aae400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa815a700 con 0x7f4aa8157c10 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.562+0000 7f4aae400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa80ce700 con 0x7f4aa81f82c0 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80cbe70 0x7f4aa81fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80cbe70 0x7f4aa81fabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36610/0 (socket says 172.21.15.19:36610) 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 -- 172.21.15.19:0/2647171888 learned_addr learned my addr 172.21.15.19:0/2647171888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 -- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa81f82c0 msgr2=0x7f4aa81f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa81f82c0 0x7f4aa81f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 -- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa8157c10 msgr2=0x7f4aa8158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa8157c10 0x7f4aa8158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 -- 172.21.15.19:0/2647171888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa80cc280 con 0x7f4aa80cbe70 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.563+0000 7f4aa7400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80cbe70 0x7f4aa81fabf0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4a980115d0 tx=0x7f4a98011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.564+0000 7f4a9f400640 1 -- 172.21.15.19:0/2647171888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98015870 con 0x7f4aa80cbe70 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.564+0000 7f4a9f400640 1 -- 172.21.15.19:0/2647171888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a98015a10 con 0x7f4aa80cbe70 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.564+0000 7f4a9f400640 1 -- 172.21.15.19:0/2647171888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98015e00 con 0x7f4aa80cbe70 2024-04-25T11:38:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.564+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa8158560 con 0x7f4aa80cbe70 2024-04-25T11:38:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.564+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa80cc500 con 0x7f4aa80cbe70 2024-04-25T11:38:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.565+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa80b33e0 con 0x7f4aa80cbe70 2024-04-25T11:38:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.566+0000 7f4a9f400640 1 -- 172.21.15.19:0/2647171888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a9800f480 con 0x7f4aa80cbe70 2024-04-25T11:38:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.568+0000 7f4a9f400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a880744b0 0x7f4a880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.568+0000 7f4a9f400640 1 -- 172.21.15.19:0/2647171888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7f4a980962b0 con 0x7f4aa80cbe70 2024-04-25T11:38:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.569+0000 7f4aa6a00640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a880744b0 0x7f4a880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.570+0000 7f4aa6a00640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a880744b0 0x7f4a880769c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007fd0 tx=0x7f4a94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.573+0000 7f4a9f400640 1 -- 172.21.15.19:0/2647171888 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4aa8158560 con 0x7f4aa80cbe70 2024-04-25T11:38:51.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.819+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aa80b5820 con 0x7f4aa80cbe70 2024-04-25T11:38:51.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.820+0000 7f4a9f400640 1 -- 172.21.15.19:0/2647171888 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 230 v230) ==== 95+0+4352 (secure 0 0 0) 0x7f4aa80cc280 con 0x7f4aa80cbe70 2024-04-25T11:38:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:51.807 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":230,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":230,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:51.249224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:51.807 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 230 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a880744b0 msgr2=0x7f4a880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a880744b0 0x7f4a880769c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007fd0 tx=0x7f4a94009040 comp rx=0 tx=0).stop 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80cbe70 msgr2=0x7f4aa81fabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80cbe70 0x7f4aa81fabf0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f4a980115d0 tx=0x7f4a98011a90 comp rx=0 tx=0).stop 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 shutdown_connections 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a880744b0 0x7f4a880769c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa8157c10 0x7f4aa8158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa81f82c0 0x7f4aa81f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 --2- 172.21.15.19:0/2647171888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa80cbe70 0x7f4aa81fabf0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.822+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 >> 172.21.15.19:0/2647171888 conn(0x7f4aa80d1ae0 msgr2=0x7f4aa80cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.823+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 shutdown_connections 2024-04-25T11:38:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:51.823+0000 7f4aae400640 1 -- 172.21.15.19:0/2647171888 wait complete. 2024-04-25T11:38:51.837 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:51.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.137+0000 7f9acfe00640 1 Processor -- start 2024-04-25T11:38:52.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.137+0000 7f9acfe00640 1 -- start start 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.139+0000 7f9acfe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9ad00d50e0 0x7f9ad00d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.139+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad00d2ae0 0x7f9ad0160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.139+0000 7f9acfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ad0160b90 0x7f9ad0161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.139+0000 7f9acfe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad00b7940 con 0x7f9ad0160b90 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.139+0000 7f9acfe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad00d2ef0 con 0x7f9ad00d50e0 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad0161280 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ad0160b90 0x7f9ad0161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9ad0160b90 0x7f9ad0161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36624/0 (socket says 172.21.15.19:36624) 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 -- 172.21.15.19:0/1443129651 learned_addr learned my addr 172.21.15.19:0/1443129651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9ac6000640 1 --2- 172.21.15.19:0/1443129651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad00d2ae0 0x7f9ad0160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acea00640 1 --2- 172.21.15.19:0/1443129651 >> v2:172.21.15.169:3300/0 conn(0x7f9ad00d50e0 0x7f9ad00d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 -- 172.21.15.19:0/1443129651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad00d2ae0 msgr2=0x7f9ad0160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:52.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 --2- 172.21.15.19:0/1443129651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad00d2ae0 0x7f9ad0160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 -- 172.21.15.19:0/1443129651 >> v2:172.21.15.169:3300/0 conn(0x7f9ad00d50e0 msgr2=0x7f9ad00d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 --2- 172.21.15.19:0/1443129651 >> v2:172.21.15.169:3300/0 conn(0x7f9ad00d50e0 0x7f9ad00d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.140+0000 7f9acf400640 1 -- 172.21.15.19:0/1443129651 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad00d4600 con 0x7f9ad0160b90 2024-04-25T11:38:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.142+0000 7f9acf400640 1 --2- 172.21.15.19:0/1443129651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0160b90 0x7f9ad0161810 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00d6cc0 tx=0x7f9ac802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e33489e390d87a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.142+0000 7f9ace000640 1 -- 172.21.15.19:0/1443129651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8037070 con 0x7f9ad0160b90 2024-04-25T11:38:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.142+0000 7f9ace000640 1 -- 172.21.15.19:0/1443129651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac8032440 con 0x7f9ad0160b90 2024-04-25T11:38:52.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.142+0000 7f9ace000640 1 -- 172.21.15.19:0/1443129651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8032810 con 0x7f9ad0160b90 2024-04-25T11:38:52.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 -- 172.21.15.19:0/1443129651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0160b90 msgr2=0x7f9ad0161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 --2- 172.21.15.19:0/1443129651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0160b90 0x7f9ad0161810 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00d6cc0 tx=0x7f9ac802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 -- 172.21.15.19:0/1443129651 shutdown_connections 2024-04-25T11:38:52.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 --2- 172.21.15.19:0/1443129651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0160b90 0x7f9ad0161810 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 --2- 172.21.15.19:0/1443129651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad00d2ae0 0x7f9ad0160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 --2- 172.21.15.19:0/1443129651 >> v2:172.21.15.169:3300/0 conn(0x7f9ad00d50e0 0x7f9ad00d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 -- 172.21.15.19:0/1443129651 >> 172.21.15.19:0/1443129651 conn(0x7f9ad006c5c0 msgr2=0x7f9ad006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 -- 172.21.15.19:0/1443129651 shutdown_connections 2024-04-25T11:38:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.143+0000 7f9acfe00640 1 -- 172.21.15.19:0/1443129651 wait complete. 2024-04-25T11:38:52.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.146+0000 7f9acfe00640 1 Processor -- start 2024-04-25T11:38:52.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.146+0000 7f9acfe00640 1 -- start start 2024-04-25T11:38:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.147+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad00d2ae0 0x7f9ad0156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.147+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00d50e0 0x7f9ad00d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.147+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0160b90 0x7f9ad0143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.147+0000 7f9acfe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0160fa0 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.147+0000 7f9acfe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00d4bc0 con 0x7f9ad00d50e0 2024-04-25T11:38:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.147+0000 7f9acfe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad0161560 con 0x7f9ad0160b90 2024-04-25T11:38:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad00d2ae0 0x7f9ad0156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad00d2ae0 0x7f9ad0156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36632/0 (socket says 172.21.15.19:36632) 2024-04-25T11:38:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 -- 172.21.15.19:0/2962301067 learned_addr learned my addr 172.21.15.19:0/2962301067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acf400640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0160b90 0x7f9ad0143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 -- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0160b90 msgr2=0x7f9ad0143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0160b90 0x7f9ad0143bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9ac6000640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00d50e0 0x7f9ad00d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 -- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00d50e0 msgr2=0x7f9ad00d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00d50e0 0x7f9ad00d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 -- 172.21.15.19:0/2962301067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad015a6d0 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.148+0000 7f9acea00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad00d2ae0 0x7f9ad0156410 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0007e40 tx=0x7f9ac000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.149+0000 7f9ac7400640 1 -- 172.21.15.19:0/2962301067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0014620 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.149+0000 7f9ac7400640 1 -- 172.21.15.19:0/2962301067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac0019070 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.149+0000 7f9ac7400640 1 -- 172.21.15.19:0/2962301067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0014a00 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.149+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad015a3f0 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.149+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad0155c10 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.150+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad0050740 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.151+0000 7f9ac7400640 1 -- 172.21.15.19:0/2962301067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ac0014ba0 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.153+0000 7f9ac7400640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9aac0744b0 0x7f9aac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.153+0000 7f9ac7400640 1 -- 172.21.15.19:0/2962301067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7f9ac0096270 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.153+0000 7f9ac6000640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9aac0744b0 0x7f9aac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.155+0000 7f9ac6000640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9aac0744b0 0x7f9aac0769c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8006fd0 tx=0x7f9ab8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.158+0000 7f9ac7400640 1 -- 172.21.15.19:0/2962301067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ac0060280 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.396+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ad00b8aa0 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.397+0000 7f9ac7400640 1 -- 172.21.15.19:0/2962301067 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 231 v231) ==== 95+0+5178 (secure 0 0 0) 0x7f9ac005fb80 con 0x7f9ad00d2ae0 2024-04-25T11:38:52.384 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:52.385 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":231,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1758836153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1758836153},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758836153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":231}],"filesystems":[{"mdsmap":{"epoch":230,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:51.249224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 231 2024-04-25T11:38:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9aac0744b0 msgr2=0x7f9aac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9aac0744b0 0x7f9aac0769c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8006fd0 tx=0x7f9ab8008040 comp rx=0 tx=0).stop 2024-04-25T11:38:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad00d2ae0 msgr2=0x7f9ad0156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad00d2ae0 0x7f9ad0156410 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0007e40 tx=0x7f9ac000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 shutdown_connections 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9aac0744b0 0x7f9aac0769c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ad00d50e0 0x7f9ad00d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ad0160b90 0x7f9ad0143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 --2- 172.21.15.19:0/2962301067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad00d2ae0 0x7f9ad0156410 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 >> 172.21.15.19:0/2962301067 conn(0x7f9ad006c5c0 msgr2=0x7f9ad01535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 shutdown_connections 2024-04-25T11:38:52.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.400+0000 7f9acfe00640 1 -- 172.21.15.19:0/2962301067 wait complete. 2024-04-25T11:38:52.415 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:52.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.707+0000 7fc369e00640 1 Processor -- start 2024-04-25T11:38:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.707+0000 7fc369e00640 1 -- start start 2024-04-25T11:38:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.709+0000 7fc369e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3640b4ff0 0x7fc3641611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.709+0000 7fc369e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc36408d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.709+0000 7fc369e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc364161720 con 0x7fc364163d20 2024-04-25T11:38:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc369e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc36406e9d0 con 0x7fc3640b4ff0 2024-04-25T11:38:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc369e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc36408d6b0 con 0x7fc36408cd60 2024-04-25T11:38:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc368a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3640b4ff0 0x7fc3641611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc369400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc364163d20 0x7fc364164130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56832/0 (socket says 172.21.15.19:56832) 2024-04-25T11:38:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc369400640 1 -- 172.21.15.19:0/4149376217 learned_addr learned my addr 172.21.15.19:0/4149376217 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc363e00640 1 --2- 172.21.15.19:0/4149376217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc36408d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc368a00640 1 -- 172.21.15.19:0/4149376217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 msgr2=0x7fc36408d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc368a00640 1 --2- 172.21.15.19:0/4149376217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc36408d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc368a00640 1 -- 172.21.15.19:0/4149376217 >> v1:172.21.15.19:6789/0 conn(0x7fc364163d20 legacy=0x7fc364164130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc368a00640 1 -- 172.21.15.19:0/4149376217 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36406ddc0 con 0x7fc3640b4ff0 2024-04-25T11:38:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.710+0000 7fc363400640 1 -- 172.21.15.19:0/4149376217 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827506202 0 0) 0x7fc35c001490 con 0x7fc364163d20 2024-04-25T11:38:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.712+0000 7fc368a00640 1 --2- 172.21.15.19:0/4149376217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640b4ff0 0x7fc3641611e0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fc354009790 tx=0x7fc35402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f7157f5d6fe35f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.712+0000 7fc363400640 1 -- 172.21.15.19:0/4149376217 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354037070 con 0x7fc3640b4ff0 2024-04-25T11:38:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.712+0000 7fc363400640 1 -- 172.21.15.19:0/4149376217 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35402ae50 con 0x7fc3640b4ff0 2024-04-25T11:38:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.712+0000 7fc363400640 1 -- 172.21.15.19:0/4149376217 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354032660 con 0x7fc3640b4ff0 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 -- 172.21.15.19:0/4149376217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640b4ff0 msgr2=0x7fc3641611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 --2- 172.21.15.19:0/4149376217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640b4ff0 0x7fc3641611e0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fc354009790 tx=0x7fc35402a100 comp rx=0 tx=0).stop 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 -- 172.21.15.19:0/4149376217 shutdown_connections 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 --2- 172.21.15.19:0/4149376217 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc36408d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 --2- 172.21.15.19:0/4149376217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3640b4ff0 0x7fc3641611e0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 -- 172.21.15.19:0/4149376217 >> 172.21.15.19:0/4149376217 conn(0x7fc3640b5460 msgr2=0x7fc3640b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 -- 172.21.15.19:0/4149376217 shutdown_connections 2024-04-25T11:38:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.713+0000 7fc369e00640 1 -- 172.21.15.19:0/4149376217 wait complete. 2024-04-25T11:38:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.715+0000 7fc369e00640 1 Processor -- start 2024-04-25T11:38:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.716+0000 7fc369e00640 1 -- start start 2024-04-25T11:38:52.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.717+0000 7fc369e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc364152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.717+0000 7fc369e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640b4ff0 0x7fc3640b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.717+0000 7fc369e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc364163d20 0x7fc3640b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.717+0000 7fc369e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc36406e660 con 0x7fc364163d20 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.717+0000 7fc369e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc36406e380 con 0x7fc3640b4ff0 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.717+0000 7fc369e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3640b9180 con 0x7fc36408cd60 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.717+0000 7fc369400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc364163d20 0x7fc3640b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc364163d20 0x7fc3640b7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36652/0 (socket says 172.21.15.19:36652) 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 -- 172.21.15.19:0/2288049035 learned_addr learned my addr 172.21.15.19:0/2288049035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc368a00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc364152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc363e00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640b4ff0 0x7fc3640b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 -- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 msgr2=0x7fc364152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc364152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 -- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640b4ff0 msgr2=0x7fc3640b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640b4ff0 0x7fc3640b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 -- 172.21.15.19:0/2288049035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36409f650 con 0x7fc364163d20 2024-04-25T11:38:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.718+0000 7fc369400640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc364163d20 0x7fc3640b7b20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c7b0 tx=0x7fc35c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.719+0000 7fc360c00640 1 -- 172.21.15.19:0/2288049035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c018070 con 0x7fc364163d20 2024-04-25T11:38:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.719+0000 7fc360c00640 1 -- 172.21.15.19:0/2288049035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35c00a640 con 0x7fc364163d20 2024-04-25T11:38:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.719+0000 7fc360c00640 1 -- 172.21.15.19:0/2288049035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00ce60 con 0x7fc364163d20 2024-04-25T11:38:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.719+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36409f930 con 0x7fc364163d20 2024-04-25T11:38:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.719+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36409fef0 con 0x7fc364163d20 2024-04-25T11:38:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.720+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc364050740 con 0x7fc364163d20 2024-04-25T11:38:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.721+0000 7fc360c00640 1 -- 172.21.15.19:0/2288049035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc35c01b070 con 0x7fc364163d20 2024-04-25T11:38:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.722+0000 7fc360c00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3400744b0 0x7fc3400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.722+0000 7fc360c00640 1 -- 172.21.15.19:0/2288049035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7fc35c094ef0 con 0x7fc364163d20 2024-04-25T11:38:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.723+0000 7fc368a00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3400744b0 0x7fc3400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.724+0000 7fc368a00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3400744b0 0x7fc3400769c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc354034000 tx=0x7fc35402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.727+0000 7fc360c00640 1 -- 172.21.15.19:0/2288049035 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc364161720 con 0x7fc364163d20 2024-04-25T11:38:52.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.971+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3640a15a0 con 0x7fc364163d20 2024-04-25T11:38:52.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.972+0000 7fc360c00640 1 -- 172.21.15.19:0/2288049035 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 231 v231) ==== 95+0+5178 (secure 0 0 0) 0x7fc35c05ef00 con 0x7fc364163d20 2024-04-25T11:38:52.960 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:52.960 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":231,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1758836153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1758836153},{"type":"v1","addr":"172.21.15.19:6835","nonce":1758836153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":231}],"filesystems":[{"mdsmap":{"epoch":230,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:51.249224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 231 2024-04-25T11:38:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3400744b0 msgr2=0x7fc3400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3400744b0 0x7fc3400769c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc354034000 tx=0x7fc35402e000 comp rx=0 tx=0).stop 2024-04-25T11:38:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc364163d20 msgr2=0x7fc3640b7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc364163d20 0x7fc3640b7b20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c7b0 tx=0x7fc35c00cc70 comp rx=0 tx=0).stop 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 shutdown_connections 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3400744b0 0x7fc3400769c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3640b4ff0 0x7fc3640b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc364163d20 0x7fc3640b7b20 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 --2- 172.21.15.19:0/2288049035 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc36408cd60 0x7fc364152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 >> 172.21.15.19:0/2288049035 conn(0x7fc3640b5460 msgr2=0x7fc3640d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 shutdown_connections 2024-04-25T11:38:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:52.975+0000 7fc369e00640 1 -- 172.21.15.19:0/2288049035 wait complete. 2024-04-25T11:38:52.987 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2024-04-25T11:38:52.987 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2024-04-25T11:38:52.988 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:52.988 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:53.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:38:53.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:53.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:53.041 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:53.094 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:38:53.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:53.095 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.392+0000 7feac7200640 1 Processor -- start 2024-04-25T11:38:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.392+0000 7feac7200640 1 -- start start 2024-04-25T11:38:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.395+0000 7feac7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feac80d2cb0 0x7feac8160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.395+0000 7feac7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac8160b90 0x7feac8161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.395+0000 7feac7200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac80b7940 con 0x7feac80d50e0 2024-04-25T11:38:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.395+0000 7feac7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7feac8161280 con 0x7feac80d2cb0 2024-04-25T11:38:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.395+0000 7feac7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feac80d4600 con 0x7feac8160b90 2024-04-25T11:38:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.395+0000 7feac5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feac80d50e0 0x7feac80d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56842/0 (socket says 172.21.15.19:56842) 2024-04-25T11:38:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.395+0000 7feac5e00640 1 -- 172.21.15.19:0/3142482358 learned_addr learned my addr 172.21.15.19:0/3142482358 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955982905 0 0) 0x7feab8001490 con 0x7feac80d50e0 2024-04-25T11:38:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaac003520 con 0x7feac80d50e0 2024-04-25T11:38:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feabd400640 1 --2- 172.21.15.19:0/3142482358 >> v2:172.21.15.169:3300/0 conn(0x7feac80d2cb0 0x7feac8160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac6800640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac8160b90 0x7feac8161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2551573198 0 0) 0x7feaac003520 con 0x7feac80d50e0 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac8160b90 msgr2=0x7feac8161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac8160b90 0x7feac8161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 >> v2:172.21.15.169:3300/0 conn(0x7feac80d2cb0 msgr2=0x7feac8160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 --2- 172.21.15.19:0/3142482358 >> v2:172.21.15.169:3300/0 conn(0x7feac80d2cb0 0x7feac8160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac80d48e0 con 0x7feac80d50e0 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7feab8001bc0 con 0x7feac80d50e0 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 >> v1:172.21.15.19:6789/0 conn(0x7feac80d50e0 legacy=0x7feac80d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac0039b0 0x7feaac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaac004800 0x7feaac004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac8160b90 0x7feaac0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feac8161280 con 0x7feaac004800 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feac80d4600 con 0x7feaac0039b0 2024-04-25T11:38:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feaac00bb50 con 0x7feac8160b90 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.396+0000 7feabd400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaac004800 0x7feaac004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.397+0000 7feac5e00640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac0039b0 0x7feaac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.397+0000 7feabd400640 1 -- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac8160b90 msgr2=0x7feaac0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.397+0000 7feabd400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feac8160b90 0x7feaac0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.397+0000 7feabd400640 1 -- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac0039b0 msgr2=0x7feaac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.397+0000 7feabd400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac0039b0 0x7feaac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.397+0000 7feabd400640 1 -- 172.21.15.19:0/3142482358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac003520 con 0x7feaac004800 2024-04-25T11:38:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.397+0000 7feac5e00640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feaac0039b0 0x7feaac003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.398+0000 7feabd400640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaac004800 0x7feaac004c10 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7feac80d5b90 tx=0x7feaac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20723eafd7d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.398+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0030070 con 0x7feaac004800 2024-04-25T11:38:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.398+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab002adb0 con 0x7feaac004800 2024-04-25T11:38:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.398+0000 7feac5400640 1 -- 172.21.15.19:0/3142482358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab000b610 con 0x7feaac004800 2024-04-25T11:38:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.399+0000 7feac7200640 1 -- 172.21.15.19:0/3142482358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaac004800 msgr2=0x7feaac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.399+0000 7feac7200640 1 --2- 172.21.15.19:0/3142482358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feaac004800 0x7feaac004c10 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7feac80d5b90 tx=0x7feaac005830 comp rx=0 tx=0).stop 2024-04-25T11:38:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.399+0000 7feac6800640 1 -- 172.21.15.19:0/3142482358 reap_dead start 2024-04-25T11:38:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.399+0000 7feac7200640 1 -- 172.21.15.19:0/3142482358 shutdown_connections 2024-04-25T11:38:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.399+0000 7feac7200640 1 -- 172.21.15.19:0/3142482358 >> 172.21.15.19:0/3142482358 conn(0x7feac806c5c0 msgr2=0x7feac806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.399+0000 7feac7200640 1 -- 172.21.15.19:0/3142482358 shutdown_connections 2024-04-25T11:38:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.399+0000 7feac7200640 1 -- 172.21.15.19:0/3142482358 wait complete. 2024-04-25T11:38:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.401+0000 7feac7200640 1 Processor -- start 2024-04-25T11:38:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.402+0000 7feac7200640 1 -- start start 2024-04-25T11:38:53.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaac004800 0x7feac814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feac8150810 0x7feac81f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac814b0c0 0x7feac81511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feac8161560 con 0x7feac814b0c0 2024-04-25T11:38:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feac8160fa0 con 0x7feac8150810 2024-04-25T11:38:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feac80d54f0 con 0x7feaac004800 2024-04-25T11:38:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac814b0c0 0x7feac81511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac814b0c0 0x7feac81511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36664/0 (socket says 172.21.15.19:36664) 2024-04-25T11:38:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.403+0000 7feac6800640 1 -- 172.21.15.19:0/589952248 learned_addr learned my addr 172.21.15.19:0/589952248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feabd400640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feac8150810 0x7feac81f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feac5e00640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaac004800 0x7feac814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feac6800640 1 -- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaac004800 msgr2=0x7feac814ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feac6800640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaac004800 0x7feac814ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feac6800640 1 -- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feac8150810 msgr2=0x7feac81f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feac6800640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feac8150810 0x7feac81f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feac6800640 1 -- 172.21.15.19:0/589952248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac814f770 con 0x7feac814b0c0 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feabd400640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feac8150810 0x7feac81f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.404+0000 7feac6800640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac814b0c0 0x7feac81511f0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7feac80b55d0 tx=0x7feac000d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.405+0000 7feabea00640 1 -- 172.21.15.19:0/589952248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac001a070 con 0x7feac814b0c0 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.405+0000 7feabea00640 1 -- 172.21.15.19:0/589952248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac0004030 con 0x7feac814b0c0 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.405+0000 7feabea00640 1 -- 172.21.15.19:0/589952248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0004440 con 0x7feac814b0c0 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.405+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac814fa50 con 0x7feac814b0c0 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.405+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac8150010 con 0x7feac814b0c0 2024-04-25T11:38:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.406+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac814f4e0 con 0x7feac814b0c0 2024-04-25T11:38:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.407+0000 7feabea00640 1 -- 172.21.15.19:0/589952248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feac001d070 con 0x7feac814b0c0 2024-04-25T11:38:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.408+0000 7feabea00640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa40744e0 0x7feaa40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:53.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.408+0000 7feabea00640 1 -- 172.21.15.19:0/589952248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7feac0094e40 con 0x7feac814b0c0 2024-04-25T11:38:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.409+0000 7feac5e00640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa40744e0 0x7feaa40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:53.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.410+0000 7feac5e00640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa40744e0 0x7feaa40769f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7feab8002230 tx=0x7feab8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:53.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.413+0000 7feabea00640 1 -- 172.21.15.19:0/589952248 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feac005ee90 con 0x7feac814b0c0 2024-04-25T11:38:53.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:53.648+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feac80b6910 con 0x7feac814b0c0 2024-04-25T11:38:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.284+0000 7feabea00640 1 -- 172.21.15.19:0/589952248 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v232) ==== 161+0+0 (secure 0 0 0) 0x7feac005e790 con 0x7feac814b0c0 2024-04-25T11:38:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa40744e0 msgr2=0x7feaa40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa40744e0 0x7feaa40769f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7feab8002230 tx=0x7feab8009040 comp rx=0 tx=0).stop 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac814b0c0 msgr2=0x7feac81511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac814b0c0 0x7feac81511f0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7feac80b55d0 tx=0x7feac000d820 comp rx=0 tx=0).stop 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 shutdown_connections 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa40744e0 0x7feaa40769f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feac8150810 0x7feac81f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feac814b0c0 0x7feac81511f0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 --2- 172.21.15.19:0/589952248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feaac004800 0x7feac814ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 >> 172.21.15.19:0/589952248 conn(0x7feac806c5c0 msgr2=0x7feac80b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 shutdown_connections 2024-04-25T11:38:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.288+0000 7feac7200640 1 -- 172.21.15.19:0/589952248 wait complete. 2024-04-25T11:38:54.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:38:54.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.603+0000 7f3bda800640 1 Processor -- start 2024-04-25T11:38:54.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.603+0000 7f3bda800640 1 -- start start 2024-04-25T11:38:54.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.605+0000 7f3bda800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 0x7f3bd40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.605+0000 7f3bda800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3bd415a810 0x7f3bd40b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.605+0000 7f3bda800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bd40b4e90 con 0x7f3bd41608d0 2024-04-25T11:38:54.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.605+0000 7f3bda800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd406f870 con 0x7f3bd415a810 2024-04-25T11:38:54.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.605+0000 7f3bda800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd40b74d0 con 0x7f3bd40cf2f0 2024-04-25T11:38:54.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.606+0000 7f3bd9e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3bd41608d0 0x7f3bd4160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47806/0 (socket says 172.21.15.19:47806) 2024-04-25T11:38:54.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.606+0000 7f3bd9e00640 1 -- 172.21.15.19:0/2208774414 learned_addr learned my addr 172.21.15.19:0/2208774414 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:54.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.606+0000 7f3bd8a00640 1 --2- 172.21.15.19:0/2208774414 >> v2:172.21.15.169:3300/0 conn(0x7f3bd415a810 0x7f3bd40b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.606+0000 7f3bd9400640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 0x7f3bd40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326025850 0 0) 0x7f3bd40b4e90 con 0x7f3bd41608d0 2024-04-25T11:38:54.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bbc003520 con 0x7f3bd41608d0 2024-04-25T11:38:54.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3415149971 0 0) 0x7f3bbc003520 con 0x7f3bd41608d0 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 msgr2=0x7f3bd40b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 0x7f3bd40b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 >> v2:172.21.15.169:3300/0 conn(0x7f3bd415a810 msgr2=0x7f3bd40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 --2- 172.21.15.19:0/2208774414 >> v2:172.21.15.169:3300/0 conn(0x7f3bd415a810 0x7f3bd40b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd40b77b0 con 0x7f3bd41608d0 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f3bcc0019f0 con 0x7f3bd41608d0 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 >> v1:172.21.15.19:6789/0 conn(0x7f3bd41608d0 legacy=0x7f3bd4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 0x7f3bbc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc003f30 0x7f3bbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc004d40 0x7f3bbc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd406f870 con 0x7f3bbc004d40 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40b74d0 con 0x7f3bbc003f30 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.607+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3bbc00dc20 con 0x7f3bd40cf2f0 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc004d40 0x7f3bbc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd8a00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc003f30 0x7f3bbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9400640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 0x7f3bbc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9e00640 1 -- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 msgr2=0x7f3bbc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 0x7f3bbc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9e00640 1 -- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc003f30 msgr2=0x7f3bbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bbc003f30 0x7f3bbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9e00640 1 -- 172.21.15.19:0/2208774414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc003520 con 0x7f3bbc004d40 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.608+0000 7f3bd9400640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd40cf2f0 0x7f3bbc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.609+0000 7f3bd9e00640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc004d40 0x7f3bbc005150 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0013d0 tx=0x7f3bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ee3a8dd6d76158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.609+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00a4c0 con 0x7f3bbc004d40 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bcc02f070 con 0x7f3bbc004d40 2024-04-25T11:38:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bd3e00640 1 -- 172.21.15.19:0/2208774414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc02bc70 con 0x7f3bbc004d40 2024-04-25T11:38:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bda800640 1 -- 172.21.15.19:0/2208774414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc004d40 msgr2=0x7f3bbc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bda800640 1 --2- 172.21.15.19:0/2208774414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bbc004d40 0x7f3bbc005150 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0013d0 tx=0x7f3bbc005830 comp rx=0 tx=0).stop 2024-04-25T11:38:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bd9e00640 1 -- 172.21.15.19:0/2208774414 reap_dead start 2024-04-25T11:38:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bda800640 1 -- 172.21.15.19:0/2208774414 shutdown_connections 2024-04-25T11:38:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bda800640 1 -- 172.21.15.19:0/2208774414 >> 172.21.15.19:0/2208774414 conn(0x7f3bd40d1720 msgr2=0x7f3bd40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bda800640 1 -- 172.21.15.19:0/2208774414 shutdown_connections 2024-04-25T11:38:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.610+0000 7f3bda800640 1 -- 172.21.15.19:0/2208774414 wait complete. 2024-04-25T11:38:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.613+0000 7f3bda800640 1 Processor -- start 2024-04-25T11:38:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.613+0000 7f3bda800640 1 -- start start 2024-04-25T11:38:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.614+0000 7f3bda800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd4158d20 0x7f3bd4160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.614+0000 7f3bda800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41f9f90 0x7f3bd41fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.614+0000 7f3bda800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd414a8f0 0x7f3bd414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.614+0000 7f3bda800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd415ac20 con 0x7f3bd414a8f0 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.614+0000 7f3bda800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd406f500 con 0x7f3bd41f9f90 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.614+0000 7f3bda800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd40b6e10 con 0x7f3bd4158d20 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd414a8f0 0x7f3bd414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd414a8f0 0x7f3bd414ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33300/0 (socket says 172.21.15.19:33300) 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 -- 172.21.15.19:0/457379184 learned_addr learned my addr 172.21.15.19:0/457379184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd8a00640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41f9f90 0x7f3bd41fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9400640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd4158d20 0x7f3bd4160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 -- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd4158d20 msgr2=0x7f3bd4160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd4158d20 0x7f3bd4160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 -- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41f9f90 msgr2=0x7f3bd41fa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41f9f90 0x7f3bd41fa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.615+0000 7f3bd9e00640 1 -- 172.21.15.19:0/457379184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd41fa8e0 con 0x7f3bd414a8f0 2024-04-25T11:38:54.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.616+0000 7f3bd9e00640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd414a8f0 0x7f3bd414ad00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0013d0 tx=0x7f3bd406f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.616+0000 7f3bd1600640 1 -- 172.21.15.19:0/457379184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00a4c0 con 0x7f3bd414a8f0 2024-04-25T11:38:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.616+0000 7f3bd1600640 1 -- 172.21.15.19:0/457379184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bcc034470 con 0x7f3bd414a8f0 2024-04-25T11:38:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.616+0000 7f3bd1600640 1 -- 172.21.15.19:0/457379184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc002030 con 0x7f3bd414a8f0 2024-04-25T11:38:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.616+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd41540c0 con 0x7f3bd414a8f0 2024-04-25T11:38:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.616+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd4154680 con 0x7f3bd414a8f0 2024-04-25T11:38:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.618+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd4050740 con 0x7f3bd414a8f0 2024-04-25T11:38:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.619+0000 7f3bd1600640 1 -- 172.21.15.19:0/457379184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3bcc034860 con 0x7f3bd414a8f0 2024-04-25T11:38:54.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.621+0000 7f3bd1600640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bb80744b0 0x7f3bb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:54.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.621+0000 7f3bd1600640 1 -- 172.21.15.19:0/457379184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7f3bcc0b5af0 con 0x7f3bd414a8f0 2024-04-25T11:38:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.622+0000 7f3bd9400640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bb80744b0 0x7f3bb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.623+0000 7f3bd9400640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bb80744b0 0x7f3bb80769c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4158550 tx=0x7f3bc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.626+0000 7f3bd1600640 1 -- 172.21.15.19:0/457379184 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3bd41fa8e0 con 0x7f3bd414a8f0 2024-04-25T11:38:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:54.896+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3bd40b5680 con 0x7f3bd414a8f0 2024-04-25T11:38:55.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.293+0000 7f3bd1600640 1 -- 172.21.15.19:0/457379184 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v233) ==== 101+0+0 (secure 0 0 0) 0x7f3bcc07fb90 con 0x7f3bd414a8f0 2024-04-25T11:38:55.281 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:38:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bb80744b0 msgr2=0x7f3bb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bb80744b0 0x7f3bb80769c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4158550 tx=0x7f3bc4008040 comp rx=0 tx=0).stop 2024-04-25T11:38:55.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd414a8f0 msgr2=0x7f3bd414ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd414a8f0 0x7f3bd414ad00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0013d0 tx=0x7f3bd406f7e0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 shutdown_connections 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3bb80744b0 0x7f3bb80769c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41f9f90 0x7f3bd41fa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd414a8f0 0x7f3bd414ad00 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 --2- 172.21.15.19:0/457379184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3bd4158d20 0x7f3bd4160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 >> 172.21.15.19:0/457379184 conn(0x7f3bd40d1720 msgr2=0x7f3bd415ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.297+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 shutdown_connections 2024-04-25T11:38:55.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.298+0000 7f3bda800640 1 -- 172.21.15.19:0/457379184 wait complete. 2024-04-25T11:38:55.313 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:38:55.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.512+0000 7feac823bf40 1 Processor -- start 2024-04-25T11:38:55.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.513+0000 7feac823bf40 1 -- start start 2024-04-25T11:38:55.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac823bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc81e0 0x5577ccd94b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac823bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577cccc85f0 0x5577cccc6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac823bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577ccda1fa0 0x5577cccc74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac823bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577ccc6ed20 con 0x5577cccc81e0 2024-04-25T11:38:55.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac823bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5577ccc72d60 con 0x5577ccda1fa0 2024-04-25T11:38:55.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac823bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5577ccd113f0 con 0x5577cccc85f0 2024-04-25T11:38:55.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc81e0 0x5577ccd94b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc81e0 0x5577ccd94b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33318/0 (socket says 172.21.15.19:33318) 2024-04-25T11:38:55.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.514+0000 7feac1400640 1 -- 172.21.15.19:0/3027757962 learned_addr learned my addr 172.21.15.19:0/3027757962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1e00640 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577ccda1fa0 0x5577cccc74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1400640 1 -- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577cccc85f0 msgr2=0x5577cccc6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1400640 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577cccc85f0 0x5577cccc6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1400640 1 -- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577ccda1fa0 msgr2=0x5577cccc74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1400640 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577ccda1fa0 0x5577cccc74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1400640 1 -- 172.21.15.19:0/3027757962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577ccd9aa30 con 0x5577cccc81e0 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1e00640 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577ccda1fa0 0x5577cccc74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.515+0000 7feac1400640 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc81e0 0x5577ccd94b40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7feab4025a60 tx=0x7feab4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8dd6e329f85252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feabbe00640 1 -- 172.21.15.19:0/3027757962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4055040 con 0x5577cccc81e0 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feabbe00640 1 -- 172.21.15.19:0/3027757962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab4051a00 con 0x5577cccc81e0 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feabbe00640 1 -- 172.21.15.19:0/3027757962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4051db0 con 0x5577cccc81e0 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 -- 172.21.15.19:0/3027757962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc81e0 msgr2=0x5577ccd94b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc81e0 0x5577ccd94b40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7feab4025a60 tx=0x7feab4051000 comp rx=0 tx=0).stop 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 -- 172.21.15.19:0/3027757962 shutdown_connections 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577ccda1fa0 0x5577cccc74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577cccc85f0 0x5577cccc6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 --2- 172.21.15.19:0/3027757962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc81e0 0x5577ccd94b40 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 -- 172.21.15.19:0/3027757962 >> 172.21.15.19:0/3027757962 conn(0x5577ccbeaac0 msgr2=0x5577ccd156d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 -- 172.21.15.19:0/3027757962 shutdown_connections 2024-04-25T11:38:55.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.516+0000 7feac823bf40 1 -- 172.21.15.19:0/3027757962 wait complete. 2024-04-25T11:38:55.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.518+0000 7feac823bf40 1 Processor -- start 2024-04-25T11:38:55.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.518+0000 7feac823bf40 1 -- start start 2024-04-25T11:38:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.519+0000 7feac823bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577cccc81e0 0x5577cccf85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.519+0000 7feac823bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc85f0 0x5577ccd90450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feac823bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577ccda1fa0 0x5577ccd90990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feac823bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577ccd16110 con 0x5577cccc85f0 2024-04-25T11:38:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feac823bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5577ccd9b7b0 con 0x5577cccc81e0 2024-04-25T11:38:55.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feac823bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5577cce3cfa0 con 0x5577ccda1fa0 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc85f0 0x5577ccd90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc85f0 0x5577ccd90450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33324/0 (socket says 172.21.15.19:33324) 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 learned_addr learned my addr 172.21.15.19:0/3521373385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577cccc81e0 0x5577cccf85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577ccda1fa0 msgr2=0x5577ccd90990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577ccda1fa0 0x5577ccd90990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577cccc81e0 msgr2=0x5577cccf85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577cccc81e0 0x5577cccf85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cccf6310 con 0x5577cccc85f0 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.520+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577cccc81e0 0x5577cccf85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:55.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.521+0000 7feab8a00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc85f0 0x5577ccd90450 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7feaac00bad0 tx=0x7feaac00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.521+0000 7feabaa00640 1 -- 172.21.15.19:0/3521373385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac009e40 con 0x5577cccc85f0 2024-04-25T11:38:55.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.521+0000 7feabaa00640 1 -- 172.21.15.19:0/3521373385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac00e470 con 0x5577cccc85f0 2024-04-25T11:38:55.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.521+0000 7feabaa00640 1 -- 172.21.15.19:0/3521373385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac00e880 con 0x5577cccc85f0 2024-04-25T11:38:55.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.521+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577ccd92bd0 con 0x5577cccc85f0 2024-04-25T11:38:55.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.521+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577ccd93190 con 0x5577cccc85f0 2024-04-25T11:38:55.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.523+0000 7feabaa00640 1 -- 172.21.15.19:0/3521373385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feaac00ea20 con 0x5577cccc85f0 2024-04-25T11:38:55.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.524+0000 7feabaa00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa0072ec0 0x7feaa00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.524+0000 7feabaa00640 1 -- 172.21.15.19:0/3521373385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7feaac0946e0 con 0x5577cccc85f0 2024-04-25T11:38:55.511 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:38:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.525+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577cce3a290 0x5577cce3a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.525+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cccf8b30 con 0x5577cce3a290 2024-04-25T11:38:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.525+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa0072ec0 0x7feaa00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.525+0000 7feac1e00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577cce3a290 0x5577cce3a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.526+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa0072ec0 0x7feaa00753d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7feab400a4a0 tx=0x7feab4052000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.527+0000 7feac1e00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577cce3a290 0x5577cce3a6a0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.530+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v114'660 uv634 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.530+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce3abe0 con 0x5577cce3a290 2024-04-25T11:38:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.534+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v114'661 uv661 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.534+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce461f0 con 0x5577cce3a290 2024-04-25T11:38:55.524 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:38:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.537+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v114'662 uv662 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.537+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce475a0 con 0x5577cce3a290 2024-04-25T11:38:55.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.541+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'663 uv663 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.541+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce47940 con 0x5577cce3a290 2024-04-25T11:38:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.545+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.545+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46590 con 0x5577cce3a290 2024-04-25T11:38:55.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.549+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.549+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.552+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v114'666 uv640 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.553+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.556+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v114'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.556+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.545 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:38:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.559+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v114'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.559+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.562+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.562+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.566+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.566+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.570+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.570+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.574+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v114'672 uv647 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.574+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.578+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'673 uv673 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.579+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.569 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:38:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.582+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v114'674 uv674 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.582+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.586+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'675 uv675 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.586+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.590+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'676 uv676 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.590+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46930 con 0x5577cce3a290 2024-04-25T11:38:55.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.595+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'677 uv677 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.595+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46d10 con 0x5577cce3a290 2024-04-25T11:38:55.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.599+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'678 uv678 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.600+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46d10 con 0x5577cce3a290 2024-04-25T11:38:55.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.604+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v114'679 uv653 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.604+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce46d10 con 0x5577cce3a290 2024-04-25T11:38:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.609+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v114'680 uv680 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.609+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46d10 con 0x5577cce3a290 2024-04-25T11:38:55.600 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:38:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.613+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v114'681 uv681 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.613+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46d10 con 0x5577cce3a290 2024-04-25T11:38:55.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.618+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'682 uv682 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.618+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce47110 con 0x5577cce3a290 2024-04-25T11:38:55.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.622+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'683 uv683 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.622+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce47110 con 0x5577cce3a290 2024-04-25T11:38:55.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.627+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'684 uv684 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.627+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce47110 con 0x5577cce3a290 2024-04-25T11:38:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.632+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v114'685 uv659 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.632+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce47110 con 0x5577cce3a290 2024-04-25T11:38:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.636+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v114'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.636+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce47110 con 0x5577cce3a290 2024-04-25T11:38:55.628 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:38:55.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.641+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v114'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.641+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce47110 con 0x5577cce3a290 2024-04-25T11:38:55.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.645+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.645+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46bf0 con 0x5577cce3a290 2024-04-25T11:38:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.650+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.650+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46bf0 con 0x5577cce3a290 2024-04-25T11:38:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.655+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.655+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577cce47ce0 0x5577cce4a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.655+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce46bf0 con 0x5577cce47ce0 2024-04-25T11:38:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.656+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577cce47ce0 0x5577cce4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.658+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577cce47ce0 0x5577cce4a100 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.662+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'484 uv467 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.663+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce4a640 con 0x5577cce47ce0 2024-04-25T11:38:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.667+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'485 uv485 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.667+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4a9e0 con 0x5577cce47ce0 2024-04-25T11:38:55.661 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:38:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.672+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'486 uv486 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.672+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4ad80 con 0x5577cce47ce0 2024-04-25T11:38:55.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.680+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v114'487 uv487 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.680+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b120 con 0x5577cce47ce0 2024-04-25T11:38:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.685+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'488 uv488 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.685+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b4c0 con 0x5577cce47ce0 2024-04-25T11:38:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.689+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v114'489 uv472 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.689+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.694+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v114'490 uv490 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.694+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.685 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:38:55.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.698+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v114'491 uv491 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.698+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.703+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'492 uv492 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.703+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.708+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'493 uv493 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.708+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.712+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v114'494 uv477 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.712+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.717+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v114'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.717+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.708 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:38:55.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.721+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v114'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.721+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4ff50 con 0x5577cce47ce0 2024-04-25T11:38:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.727+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v114'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.727+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4ff50 con 0x5577cce47ce0 2024-04-25T11:38:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.731+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.731+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.735+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v114'499 uv483 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.735+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.740+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v114'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.740+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.731 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:38:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.744+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v114'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.744+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4c890 con 0x5577cce47ce0 2024-04-25T11:38:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.749+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.749+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4c890 con 0x5577cce47ce0 2024-04-25T11:38:55.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.753+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.753+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4b860 con 0x5577cce47ce0 2024-04-25T11:38:55.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.758+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce47ce0 2024-04-25T11:38:55.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.758+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577cce4bd70 0x5577cce4c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.758+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4eca0 con 0x5577cce4bd70 2024-04-25T11:38:55.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.759+0000 7feab8a00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577cce4bd70 0x5577cce4c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.760+0000 7feab8a00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577cce4bd70 0x5577cce4c180 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.765+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.765+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce4f040 con 0x5577cce4bd70 2024-04-25T11:38:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.770+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v114'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.770+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4f520 con 0x5577cce4bd70 2024-04-25T11:38:55.761 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:38:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.774+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v114'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.774+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce50dd0 con 0x5577cce4bd70 2024-04-25T11:38:55.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.778+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.778+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce51170 con 0x5577cce4bd70 2024-04-25T11:38:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.783+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.783+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce4fdc0 con 0x5577cce4bd70 2024-04-25T11:38:55.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.788+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.788+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce501d0 con 0x5577cce3a290 2024-04-25T11:38:55.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.792+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'98 uv97 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.792+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce501d0 con 0x5577cce3a290 2024-04-25T11:38:55.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.796+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'99 uv99 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.796+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce501d0 con 0x5577cce3a290 2024-04-25T11:38:55.786 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:38:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.800+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'100 uv100 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.800+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce501d0 con 0x5577cce3a290 2024-04-25T11:38:55.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.804+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'101 uv101 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.804+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577cce51510 0x5577cce53930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.804+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce501d0 con 0x5577cce51510 2024-04-25T11:38:55.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.805+0000 7feac1e00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577cce51510 0x5577cce53930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.807+0000 7feac1e00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577cce51510 0x5577cce53930 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.811+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.811+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce501d0 con 0x5577cce51510 2024-04-25T11:38:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.816+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v114'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.816+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce501d0 con 0x5577cce51510 2024-04-25T11:38:55.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.820+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v114'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.808 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:38:55.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.821+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce54e80 con 0x5577cce51510 2024-04-25T11:38:55.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.825+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.825+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce54e80 con 0x5577cce51510 2024-04-25T11:38:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.829+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.829+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce53e70 con 0x5577cce51510 2024-04-25T11:38:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.834+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.834+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577cce54380 0x5577cce567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.834+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce56dd0 con 0x5577cce54380 2024-04-25T11:38:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.835+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577cce54380 0x5577cce567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.837+0000 7feac1400640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577cce54380 0x5577cce567a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.841+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v114'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.841+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce56e90 con 0x5577cce54380 2024-04-25T11:38:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.845+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v114'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.845+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce572b0 con 0x5577cce54380 2024-04-25T11:38:55.836 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:38:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.849+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v114'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.850+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce582c0 con 0x5577cce54380 2024-04-25T11:38:55.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.854+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.854+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce582c0 con 0x5577cce54380 2024-04-25T11:38:55.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.858+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.858+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce572b0 con 0x5577cce54380 2024-04-25T11:38:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.862+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'243 uv243 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.862+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce572b0 con 0x5577cce54380 2024-04-25T11:38:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.866+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v114'244 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.866+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce572b0 con 0x5577cce54380 2024-04-25T11:38:55.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.870+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v114'245 uv245 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.870+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce572b0 con 0x5577cce54380 2024-04-25T11:38:55.861 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:38:55.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.874+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v114'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.874+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce572b0 con 0x5577cce54380 2024-04-25T11:38:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.879+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.879+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce572b0 con 0x5577cce54380 2024-04-25T11:38:55.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.883+0000 7feac1400640 1 -- 172.21.15.19:0/3521373385 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feab4057030 con 0x5577cce54380 2024-04-25T11:38:55.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.883+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577cce57930 0x5577cce59d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:55.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.883+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5a410 con 0x5577cce57930 2024-04-25T11:38:55.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.884+0000 7feab8a00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577cce57930 0x5577cce59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.885+0000 7feab8a00640 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577cce57930 0x5577cce59d50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:55.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.889+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v114'107 uv106 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:55.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.889+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5a7b0 con 0x5577cce57930 2024-04-25T11:38:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.893+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v114'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.893+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce57930 2024-04-25T11:38:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.897+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v114'109 uv109 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:55.885 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:38:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.898+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce57930 2024-04-25T11:38:55.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.902+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'110 uv110 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:55.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.902+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce57930 2024-04-25T11:38:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.906+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'111 uv111 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.907+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce3a290 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.911+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.911+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce3a290 2024-04-25T11:38:55.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.916+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v114'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.916+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce3a290 2024-04-25T11:38:55.907 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:38:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.920+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v114'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.920+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5bbe0 con 0x5577cce3a290 2024-04-25T11:38:55.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.924+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.925+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5bbe0 con 0x5577cce3a290 2024-04-25T11:38:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.928+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.929+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce3a290 2024-04-25T11:38:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.933+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce3a290 2024-04-25T11:38:55.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.933+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce4bd70 2024-04-25T11:38:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.937+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v114'213 uv207 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.938+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce4bd70 2024-04-25T11:38:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.942+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v114'214 uv214 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.942+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5abd0 con 0x5577cce4bd70 2024-04-25T11:38:55.933 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:38:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.946+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v114'215 uv215 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.946+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce4bd70 2024-04-25T11:38:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.951+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'216 uv216 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.951+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce4bd70 2024-04-25T11:38:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.956+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'217 uv217 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.956+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce4bd70 2024-04-25T11:38:55.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.960+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v114'218 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.960+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce4bd70 2024-04-25T11:38:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.964+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v114'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.964+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce4bd70 2024-04-25T11:38:55.955 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:38:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.969+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v114'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.969+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce4bd70 2024-04-25T11:38:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.972+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.973+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce4bd70 2024-04-25T11:38:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.977+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce4bd70 2024-04-25T11:38:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.977+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce51510 2024-04-25T11:38:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.981+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v114'95 uv94 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.981+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce51510 2024-04-25T11:38:55.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.985+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'96 uv96 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.985+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5ac70 con 0x5577cce51510 2024-04-25T11:38:55.975 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:38:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.988+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v114'97 uv97 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.989+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce51510 2024-04-25T11:38:55.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.993+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'98 uv98 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.993+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce51510 2024-04-25T11:38:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.997+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v114'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:55.997+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce51510 2024-04-25T11:38:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.001+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v114'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.001+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce51510 2024-04-25T11:38:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:38:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.004+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v114'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.005+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce51510 2024-04-25T11:38:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.008+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.008+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce51510 2024-04-25T11:38:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.013+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:56.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.013+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce51510 2024-04-25T11:38:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.016+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feabc0093d0 con 0x5577cce51510 2024-04-25T11:38:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.016+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.020+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v114'110 uv109 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.020+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.024+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v114'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.025+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.029+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v114'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.016 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:38:56.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.029+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.034+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.034+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.038+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.038+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.042+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v114'265 uv257 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.042+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.046+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v114'266 uv266 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.046+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.051+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v114'267 uv267 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.038 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:38:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.051+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.055+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v114'268 uv268 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.056+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.060+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v114'269 uv269 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.060+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.064+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v114'270 uv264 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.064+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.068+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v114'271 uv271 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.069+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.072+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v114'272 uv272 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.060 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:38:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.073+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.076+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v114'273 uv273 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.077+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.081+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v114'274 uv274 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.081+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b120 con 0x5577cce57930 2024-04-25T11:38:56.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.085+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v114'275 uv275 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.085+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5577cce5b9b0 con 0x5577cce57930 2024-04-25T11:38:56.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feab8a00640 1 -- 172.21.15.19:0/3521373385 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v114'276 uv276 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feaac05a310 con 0x5577cce57930 2024-04-25T11:38:56.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577cce47ce0 msgr2=0x5577cce4a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577cce47ce0 0x5577cce4a100 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577cce54380 msgr2=0x5577cce567a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577cce54380 0x5577cce567a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577cce57930 msgr2=0x5577cce59d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577cce57930 0x5577cce59d50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.090+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577cce4bd70 msgr2=0x5577cce4c180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577cce4bd70 0x5577cce4c180 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577cce3a290 msgr2=0x5577cce3a6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577cce3a290 0x5577cce3a6a0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac1e00640 1 -- 172.21.15.19:0/3521373385 reap_dead start 2024-04-25T11:38:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577cce51510 msgr2=0x5577cce53930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577cce51510 0x5577cce53930 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa0072ec0 msgr2=0x7feaa00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa0072ec0 0x7feaa00753d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7feab400a4a0 tx=0x7feab4052000 comp rx=0 tx=0).stop 2024-04-25T11:38:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc85f0 msgr2=0x5577ccd90450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.091+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc85f0 0x5577ccd90450 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7feaac00bad0 tx=0x7feaac00bf90 comp rx=0 tx=0).stop 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 shutdown_connections 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577cce51510 0x5577cce53930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577cce3a290 0x5577cce3a6a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feaa0072ec0 0x7feaa00753d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 --2- 172.21.15.19:0/3521373385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577cccc85f0 0x5577ccd90450 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 >> 172.21.15.19:0/3521373385 conn(0x5577ccbeaac0 msgr2=0x5577ccd9e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 shutdown_connections 2024-04-25T11:38:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.092+0000 7feac823bf40 1 -- 172.21.15.19:0/3521373385 wait complete. 2024-04-25T11:38:56.110 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2024-04-25T11:38:56.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.318+0000 7f0e9f80cf40 1 Processor -- start 2024-04-25T11:38:56.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.318+0000 7f0e9f80cf40 1 -- start start 2024-04-25T11:38:56.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.319+0000 7f0e9f80cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e7c002870 0x55f2eff77190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.319+0000 7f0e9f80cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2eff820d0 0x55f2efeef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.319+0000 7f0e9f80cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff824e0 0x55f2efeeff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.319+0000 7f0e9f80cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2efe51d20 con 0x55f2eff824e0 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.319+0000 7f0e9f80cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f2efeacaf0 con 0x7f0e7c002870 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.319+0000 7f0e9f80cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f2eff7d640 con 0x55f2eff820d0 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff824e0 0x55f2efeeff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff824e0 0x55f2efeeff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33340/0 (socket says 172.21.15.19:33340) 2024-04-25T11:38:56.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 -- 172.21.15.19:0/804045623 learned_addr learned my addr 172.21.15.19:0/804045623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:56.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 -- 172.21.15.19:0/804045623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2eff820d0 msgr2=0x55f2efeef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:56.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 --2- 172.21.15.19:0/804045623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2eff820d0 0x55f2efeef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 -- 172.21.15.19:0/804045623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e7c002870 msgr2=0x55f2eff77190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 --2- 172.21.15.19:0/804045623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e7c002870 0x55f2eff77190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 -- 172.21.15.19:0/804045623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2eff776d0 con 0x55f2eff824e0 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.320+0000 7f0e98600640 1 --2- 172.21.15.19:0/804045623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff824e0 0x55f2efeeff20 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x55f2efeae6d0 tx=0x7f0e90006850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f566040c27472c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e95e00640 1 -- 172.21.15.19:0/804045623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90003650 con 0x55f2eff824e0 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e95e00640 1 -- 172.21.15.19:0/804045623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e90003830 con 0x55f2eff824e0 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e95e00640 1 -- 172.21.15.19:0/804045623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90003c20 con 0x55f2eff824e0 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/804045623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff824e0 msgr2=0x55f2efeeff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/804045623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff824e0 0x55f2efeeff20 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x55f2efeae6d0 tx=0x7f0e90006850 comp rx=0 tx=0).stop 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/804045623 shutdown_connections 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/804045623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff824e0 0x55f2efeeff20 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/804045623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2eff820d0 0x55f2efeef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/804045623 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e7c002870 0x55f2eff77190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/804045623 >> 172.21.15.19:0/804045623 conn(0x55f2efdcdac0 msgr2=0x55f2eff811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:56.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/804045623 shutdown_connections 2024-04-25T11:38:56.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.321+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/804045623 wait complete. 2024-04-25T11:38:56.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.323+0000 7f0e9f80cf40 1 Processor -- start 2024-04-25T11:38:56.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.323+0000 7f0e9f80cf40 1 -- start start 2024-04-25T11:38:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e9f80cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff820d0 0x55f2eff73840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e9f80cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2eff824e0 0x55f2eff7cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e9f80cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e7c002870 0x55f2eff7de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e9f80cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2eff7e9b0 con 0x55f2eff820d0 2024-04-25T11:38:56.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e9f80cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f2eff81a90 con 0x55f2eff824e0 2024-04-25T11:38:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e9f80cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f2eff81d60 con 0x7f0e7c002870 2024-04-25T11:38:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e97c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff820d0 0x55f2eff73840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e97c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff820d0 0x55f2eff73840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33356/0 (socket says 172.21.15.19:33356) 2024-04-25T11:38:56.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.324+0000 7f0e97c00640 1 -- 172.21.15.19:0/1058256946 learned_addr learned my addr 172.21.15.19:0/1058256946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.325+0000 7f0e97c00640 1 -- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e7c002870 msgr2=0x55f2eff7de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.325+0000 7f0e97c00640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e7c002870 0x55f2eff7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.325+0000 7f0e97c00640 1 -- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2eff824e0 msgr2=0x55f2eff7cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.325+0000 7f0e97c00640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2eff824e0 0x55f2eff7cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.325+0000 7f0e97c00640 1 -- 172.21.15.19:0/1058256946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f000cf20 con 0x55f2eff820d0 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.325+0000 7f0e97c00640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff820d0 0x55f2eff73840 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f0e88002ff0 tx=0x7f0e880084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.325+0000 7f0e94a00640 1 -- 172.21.15.19:0/1058256946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88019070 con 0x55f2eff820d0 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.326+0000 7f0e94a00640 1 -- 172.21.15.19:0/1058256946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e880143f0 con 0x55f2eff820d0 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.326+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f2f000d200 con 0x55f2eff820d0 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.326+0000 7f0e94a00640 1 -- 172.21.15.19:0/1058256946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e880147e0 con 0x55f2eff820d0 2024-04-25T11:38:56.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.326+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f000d7c0 con 0x55f2eff820d0 2024-04-25T11:38:56.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.328+0000 7f0e94a00640 1 -- 172.21.15.19:0/1058256946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e8801c070 con 0x55f2eff820d0 2024-04-25T11:38:56.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.329+0000 7f0e94a00640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e74072ec0 0x7f0e740753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.329+0000 7f0e94a00640 1 -- 172.21.15.19:0/1058256946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7f0e88093b90 con 0x55f2eff820d0 2024-04-25T11:38:56.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.329+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f2efea9f90 0x55f2efeaa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.330+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x55f2efeaa8e0 con 0x55f2efea9f90 2024-04-25T11:38:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.330+0000 7f0e97200640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e74072ec0 0x7f0e740753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.330+0000 7f0e98600640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f2efea9f90 0x55f2efeaa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.330+0000 7f0e98600640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f2efea9f90 0x55f2efeaa3a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.331+0000 7f0e97200640 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e74072ec0 0x7f0e740753d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003070 tx=0x7f0e8004f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e98600640 1 -- 172.21.15.19:0/1058256946 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v114'277 uv277 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0e9000f030 con 0x55f2efea9f90 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f2efea9f90 msgr2=0x55f2efeaa3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f2efea9f90 0x55f2efeaa3a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e74072ec0 msgr2=0x7f0e740753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0e74072ec0 0x7f0e740753d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003070 tx=0x7f0e8004f000 comp rx=0 tx=0).stop 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff820d0 msgr2=0x55f2eff73840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 --2- 172.21.15.19:0/1058256946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2eff820d0 0x55f2eff73840 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f0e88002ff0 tx=0x7f0e880084c0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e98600640 1 -- 172.21.15.19:0/1058256946 reap_dead start 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 shutdown_connections 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.334+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 >> 172.21.15.19:0/1058256946 conn(0x55f2efdcdac0 msgr2=0x55f2eff7fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.335+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 shutdown_connections 2024-04-25T11:38:56.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.335+0000 7f0e9f80cf40 1 -- 172.21.15.19:0/1058256946 wait complete. 2024-04-25T11:38:56.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:38:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.626+0000 7f9710c00640 1 Processor -- start 2024-04-25T11:38:56.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.626+0000 7f9710c00640 1 -- start start 2024-04-25T11:38:56.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f9710c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f970c144340 0x7f970c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f9710c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f970c15ee60 0x7f970c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f9710c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970c06df20 con 0x7f970c06db10 2024-04-25T11:38:56.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f9710c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f970c15f270 con 0x7f970c15ee60 2024-04-25T11:38:56.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f9710c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f970c144750 con 0x7f970c144340 2024-04-25T11:38:56.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f970be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f970c15ee60 0x7f970c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f9702a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f970c06db10 0x7f970c15e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47828/0 (socket says 172.21.15.19:47828) 2024-04-25T11:38:56.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f9702a00640 1 -- 172.21.15.19:0/108268763 learned_addr learned my addr 172.21.15.19:0/108268763 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.629+0000 7f970b400640 1 --2- 172.21.15.19:0/108268763 >> v2:172.21.15.169:3300/0 conn(0x7f970c144340 0x7f970c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.630+0000 7f970be00640 1 -- 172.21.15.19:0/108268763 >> v2:172.21.15.169:3300/0 conn(0x7f970c144340 msgr2=0x7f970c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.630+0000 7f970be00640 1 --2- 172.21.15.19:0/108268763 >> v2:172.21.15.169:3300/0 conn(0x7f970c144340 0x7f970c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.630+0000 7f970be00640 1 -- 172.21.15.19:0/108268763 >> v1:172.21.15.19:6789/0 conn(0x7f970c06db10 legacy=0x7f970c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.630+0000 7f970be00640 1 -- 172.21.15.19:0/108268763 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c070000 con 0x7f970c15ee60 2024-04-25T11:38:56.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.630+0000 7f970b400640 1 --2- 172.21.15.19:0/108268763 >> v2:172.21.15.169:3300/0 conn(0x7f970c144340 0x7f970c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.631+0000 7f970be00640 1 --2- 172.21.15.19:0/108268763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c15f4a0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f970c0b6e30 tx=0x7f970402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1874e42ee28e5d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.631+0000 7f970aa00640 1 -- 172.21.15.19:0/108268763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704007740 con 0x7f970c15ee60 2024-04-25T11:38:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.631+0000 7f970aa00640 1 -- 172.21.15.19:0/108268763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97040078e0 con 0x7f970c15ee60 2024-04-25T11:38:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.631+0000 7f970aa00640 1 -- 172.21.15.19:0/108268763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704007cb0 con 0x7f970c15ee60 2024-04-25T11:38:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 -- 172.21.15.19:0/108268763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 msgr2=0x7f970c15f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 --2- 172.21.15.19:0/108268763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c15f4a0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f970c0b6e30 tx=0x7f970402a680 comp rx=0 tx=0).stop 2024-04-25T11:38:56.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 -- 172.21.15.19:0/108268763 shutdown_connections 2024-04-25T11:38:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 --2- 172.21.15.19:0/108268763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c15f4a0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 --2- 172.21.15.19:0/108268763 >> v2:172.21.15.169:3300/0 conn(0x7f970c144340 0x7f970c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 -- 172.21.15.19:0/108268763 >> 172.21.15.19:0/108268763 conn(0x7f970c072f40 msgr2=0x7f970c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 -- 172.21.15.19:0/108268763 shutdown_connections 2024-04-25T11:38:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.632+0000 7f9710c00640 1 -- 172.21.15.19:0/108268763 wait complete. 2024-04-25T11:38:56.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.635+0000 7f9710c00640 1 Processor -- start 2024-04-25T11:38:56.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.635+0000 7f9710c00640 1 -- start start 2024-04-25T11:38:56.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.636+0000 7f9710c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f970c06db10 0x7f970c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.636+0000 7f9710c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f970c144340 0x7f970c1607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.636+0000 7f9710c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c1614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.636+0000 7f9710c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f970c06f800 con 0x7f970c15ee60 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.636+0000 7f9710c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f970c06f520 con 0x7f970c144340 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.636+0000 7f9710c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f970c06fae0 con 0x7f970c06db10 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c1614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c1614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33366/0 (socket says 172.21.15.19:33366) 2024-04-25T11:38:56.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 -- 172.21.15.19:0/931729832 learned_addr learned my addr 172.21.15.19:0/931729832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:56.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 -- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f970c06db10 msgr2=0x7f970c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f970c06db10 0x7f970c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 -- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f970c144340 msgr2=0x7f970c1607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f970c144340 0x7f970c1607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.637+0000 7f970be00640 1 -- 172.21.15.19:0/931729832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c161a00 con 0x7f970c15ee60 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.638+0000 7f970be00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c1614c0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f970402ac70 tx=0x7f9704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.638+0000 7f9703e00640 1 -- 172.21.15.19:0/931729832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970403f070 con 0x7f970c15ee60 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.638+0000 7f9703e00640 1 -- 172.21.15.19:0/931729832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9704031c60 con 0x7f970c15ee60 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.638+0000 7f9703e00640 1 -- 172.21.15.19:0/931729832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704043650 con 0x7f970c15ee60 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.638+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c1f6b90 con 0x7f970c15ee60 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.638+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c1f7150 con 0x7f970c15ee60 2024-04-25T11:38:56.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.640+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c076c60 con 0x7f970c15ee60 2024-04-25T11:38:56.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.641+0000 7f9703e00640 1 -- 172.21.15.19:0/931729832 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f97040437f0 con 0x7f970c15ee60 2024-04-25T11:38:56.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.642+0000 7f9703e00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96e8074500 0x7f96e8076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.642+0000 7f9703e00640 1 -- 172.21.15.19:0/931729832 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7601+0+0 (secure 0 0 0) 0x7f97040b6e40 con 0x7f970c15ee60 2024-04-25T11:38:56.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.643+0000 7f970b400640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96e8074500 0x7f96e8076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.644+0000 7f970b400640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96e8074500 0x7f96e8076a10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006e80 tx=0x7f96fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.647+0000 7f9703e00640 1 -- 172.21.15.19:0/931729832 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f970c161a00 con 0x7f970c15ee60 2024-04-25T11:38:56.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:56.899+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f970c0b7920 con 0x7f970c15ee60 2024-04-25T11:38:57.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.314+0000 7f9703e00640 1 -- 172.21.15.19:0/931729832 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v234) ==== 162+0+0 (secure 0 0 0) 0x7f9704080ed0 con 0x7f970c15ee60 2024-04-25T11:38:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:38:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96e8074500 msgr2=0x7f96e8076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96e8074500 0x7f96e8076a10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f96fc006e80 tx=0x7f96fc008040 comp rx=0 tx=0).stop 2024-04-25T11:38:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 msgr2=0x7f970c1614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c1614c0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f970402ac70 tx=0x7f9704004a40 comp rx=0 tx=0).stop 2024-04-25T11:38:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 shutdown_connections 2024-04-25T11:38:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96e8074500 0x7f96e8076a10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f970c144340 0x7f970c1607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f970c15ee60 0x7f970c1614c0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 --2- 172.21.15.19:0/931729832 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f970c06db10 0x7f970c15f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 >> 172.21.15.19:0/931729832 conn(0x7f970c072f40 msgr2=0x7f970c15dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 shutdown_connections 2024-04-25T11:38:57.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.318+0000 7f9710c00640 1 -- 172.21.15.19:0/931729832 wait complete. 2024-04-25T11:38:57.328 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:57.341+0000 7f58d6200640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-04-25T11:38:57.328 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:38:57.341+0000 7f58d6200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:38:57.329 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:57.342+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:57.329 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:38:57.342+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:38:57.346 INFO:tasks.ceph.mds.a.smithi019.stderr: -23> 2024-04-25T11:38:57.341+0000 7f58d6200640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-04-25T11:38:57.346 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:38:57.341+0000 7f58d6200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:38:57.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:57.489 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:38:57.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.630+0000 7f23db000640 1 Processor -- start 2024-04-25T11:38:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.631+0000 7f23db000640 1 -- start start 2024-04-25T11:38:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.633+0000 7f23db000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f23d40b4ea0 0x7f23d40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.633+0000 7f23db000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d40d1230 con 0x7f23d406c5c0 2024-04-25T11:38:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.633+0000 7f23db000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d40d06b0 con 0x7f23d4144370 2024-04-25T11:38:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.633+0000 7f23db000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f23d40d0990 con 0x7f23d40b4ea0 2024-04-25T11:38:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.633+0000 7f23d9c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f23d406c5c0 0x7f23d40b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47836/0 (socket says 172.21.15.19:47836) 2024-04-25T11:38:57.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.633+0000 7f23d9c00640 1 -- 172.21.15.19:0/3938805093 learned_addr learned my addr 172.21.15.19:0/3938805093 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244299497 0 0) 0x7f23d40d1230 con 0x7f23d406c5c0 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23bc003520 con 0x7f23d406c5c0 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d9200640 1 --2- 172.21.15.19:0/3938805093 >> v2:172.21.15.169:3300/0 conn(0x7f23d40b4ea0 0x7f23d40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2862181943 0 0) 0x7f23bc003520 con 0x7f23d406c5c0 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 >> v2:172.21.15.169:3300/0 conn(0x7f23d40b4ea0 msgr2=0x7f23d40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 --2- 172.21.15.19:0/3938805093 >> v2:172.21.15.169:3300/0 conn(0x7f23d40b4ea0 0x7f23d40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 >> v1:172.21.15.169:6789/0 conn(0x7f23d4144370 legacy=0x7f23d415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d40d0c70 con 0x7f23d406c5c0 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f23c8001670 con 0x7f23d406c5c0 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 >> v1:172.21.15.19:6789/0 conn(0x7f23d406c5c0 legacy=0x7f23d40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23bc003a00 0x7f23bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23bc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23bc005350 0x7f23bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40d0990 con 0x7f23bc004800 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc00c250 con 0x7f23bc005350 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.634+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f23bc00c450 con 0x7f23bc003a00 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23d9200640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23bc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23da600640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23bc005350 0x7f23bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23d9200640 1 -- 172.21.15.19:0/3938805093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23bc003a00 msgr2=0x7f23bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23d9200640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23bc003a00 0x7f23bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23d9200640 1 -- 172.21.15.19:0/3938805093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23bc005350 msgr2=0x7f23bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23d9200640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23bc005350 0x7f23bc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23d9200640 1 -- 172.21.15.19:0/3938805093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc003520 con 0x7f23bc004800 2024-04-25T11:38:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.635+0000 7f23da600640 1 -- 172.21.15.19:0/3938805093 reap_dead start 2024-04-25T11:38:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.636+0000 7f23d9200640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23bc004c10 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f23c000ba30 tx=0x7f23bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece01a182625980c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.636+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0030070 con 0x7f23bc004800 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.636+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c002ade0 con 0x7f23bc004800 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.636+0000 7f23d3e00640 1 -- 172.21.15.19:0/3938805093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c000b600 con 0x7f23bc004800 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.637+0000 7f23db000640 1 -- 172.21.15.19:0/3938805093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 msgr2=0x7f23bc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.637+0000 7f23db000640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23bc004c10 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f23c000ba30 tx=0x7f23bc005830 comp rx=0 tx=0).stop 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.637+0000 7f23db000640 1 -- 172.21.15.19:0/3938805093 shutdown_connections 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.637+0000 7f23db000640 1 --2- 172.21.15.19:0/3938805093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23bc004c10 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.637+0000 7f23db000640 1 -- 172.21.15.19:0/3938805093 >> 172.21.15.19:0/3938805093 conn(0x7f23d40d1720 msgr2=0x7f23d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.637+0000 7f23db000640 1 -- 172.21.15.19:0/3938805093 shutdown_connections 2024-04-25T11:38:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.637+0000 7f23db000640 1 -- 172.21.15.19:0/3938805093 wait complete. 2024-04-25T11:38:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.640+0000 7f23db000640 1 Processor -- start 2024-04-25T11:38:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.640+0000 7f23db000640 1 -- start start 2024-04-25T11:38:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.641+0000 7f23db000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23d40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.641+0000 7f23db000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23d415b720 0x7f23d4141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.641+0000 7f23db000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23d41420e0 0x7f23d4134900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.641+0000 7f23db000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40d0450 con 0x7f23bc004800 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.641+0000 7f23db000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40d02d0 con 0x7f23d41420e0 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.641+0000 7f23db000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f23d4147760 con 0x7f23d415b720 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23d40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23d40cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33396/0 (socket says 172.21.15.19:33396) 2024-04-25T11:38:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 -- 172.21.15.19:0/2250657943 learned_addr learned my addr 172.21.15.19:0/2250657943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 -- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23d415b720 msgr2=0x7f23d4141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9200640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23d415b720 0x7f23d4141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23d415b720 0x7f23d4141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23da600640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23d41420e0 0x7f23d4134900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 -- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23d41420e0 msgr2=0x7f23d4134900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23d41420e0 0x7f23d4134900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23d9c00640 1 -- 172.21.15.19:0/2250657943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d41424f0 con 0x7f23bc004800 2024-04-25T11:38:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.642+0000 7f23da600640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23d41420e0 0x7f23d4134900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.643+0000 7f23d9c00640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23d40cb8b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c760 tx=0x7f23c800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.643+0000 7f23d1600640 1 -- 172.21.15.19:0/2250657943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8018070 con 0x7f23bc004800 2024-04-25T11:38:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.643+0000 7f23d1600640 1 -- 172.21.15.19:0/2250657943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c80072f0 con 0x7f23bc004800 2024-04-25T11:38:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.643+0000 7f23d1600640 1 -- 172.21.15.19:0/2250657943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c80047e0 con 0x7f23bc004800 2024-04-25T11:38:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.643+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d4162390 con 0x7f23bc004800 2024-04-25T11:38:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.643+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d4132a10 con 0x7f23bc004800 2024-04-25T11:38:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.645+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d4073c70 con 0x7f23bc004800 2024-04-25T11:38:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.646+0000 7f23d1600640 1 -- 172.21.15.19:0/2250657943 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f23c8004980 con 0x7f23bc004800 2024-04-25T11:38:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.647+0000 7f23d1600640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23b40744e0 0x7f23b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.648+0000 7f23d1600640 1 -- 172.21.15.19:0/2250657943 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7f23c8095080 con 0x7f23bc004800 2024-04-25T11:38:57.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.648+0000 7f23d9200640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23b40744e0 0x7f23b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:57.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.650+0000 7f23d9200640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23b40744e0 0x7f23b40769f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f23c000ba00 tx=0x7f23c0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:57.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.653+0000 7f23d1600640 1 -- 172.21.15.19:0/2250657943 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23d4073c70 con 0x7f23bc004800 2024-04-25T11:38:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.901+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23d41628b0 con 0x7f23bc004800 2024-04-25T11:38:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.902+0000 7f23d1600640 1 -- 172.21.15.19:0/2250657943 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 236 v236) ==== 95+0+4352 (secure 0 0 0) 0x7f23d41424f0 con 0x7f23bc004800 2024-04-25T11:38:57.890 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:57.890 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":236,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":236,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:57.343081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:57.890 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 236 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.905+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23b40744e0 msgr2=0x7f23b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.905+0000 7f23db000640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23b40744e0 0x7f23b40769f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f23c000ba00 tx=0x7f23c0030000 comp rx=0 tx=0).stop 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.905+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 msgr2=0x7f23d40cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.905+0000 7f23db000640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23d40cb8b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c760 tx=0x7f23c800cc20 comp rx=0 tx=0).stop 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 shutdown_connections 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23b40744e0 0x7f23b40769f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23d41420e0 0x7f23d4134900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23d415b720 0x7f23d4141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 --2- 172.21.15.19:0/2250657943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23bc004800 0x7f23d40cb8b0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 >> 172.21.15.19:0/2250657943 conn(0x7f23d40d1720 msgr2=0x7f23d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 shutdown_connections 2024-04-25T11:38:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:57.906+0000 7f23db000640 1 -- 172.21.15.19:0/2250657943 wait complete. 2024-04-25T11:38:57.919 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:57.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.217+0000 7fa237e00640 1 Processor -- start 2024-04-25T11:38:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.217+0000 7fa237e00640 1 -- start start 2024-04-25T11:38:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.219+0000 7fa237e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa230160360 0x7fa2301647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.219+0000 7fa237e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2300b5c90 0x7fa2300b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.219+0000 7fa237e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2300b65e0 con 0x7fa230070680 2024-04-25T11:38:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.219+0000 7fa237e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa23006d100 con 0x7fa230160360 2024-04-25T11:38:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.219+0000 7fa237e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa23006c7d0 con 0x7fa2300b5c90 2024-04-25T11:38:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.219+0000 7fa234a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa230160360 0x7fa2301647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa234a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa230160360 0x7fa2301647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33408/0 (socket says 172.21.15.19:33408) 2024-04-25T11:38:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa234a00640 1 -- 172.21.15.19:0/3190284606 learned_addr learned my addr 172.21.15.19:0/3190284606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa235e00640 1 --2- 172.21.15.19:0/3190284606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2300b5c90 0x7fa2300b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa234a00640 1 -- 172.21.15.19:0/3190284606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2300b5c90 msgr2=0x7fa2300b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa234a00640 1 --2- 172.21.15.19:0/3190284606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2300b5c90 0x7fa2300b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa234a00640 1 -- 172.21.15.19:0/3190284606 >> v1:172.21.15.169:6789/0 conn(0x7fa230070680 legacy=0x7fa2300af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa234a00640 1 -- 172.21.15.19:0/3190284606 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23006cab0 con 0x7fa230160360 2024-04-25T11:38:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.220+0000 7fa235e00640 1 --2- 172.21.15.19:0/3190284606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2300b5c90 0x7fa2300b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.221+0000 7fa234a00640 1 --2- 172.21.15.19:0/3190284606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa230160360 0x7fa2301647f0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009870 tx=0x7fa21c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39617609f593fb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.221+0000 7fa22fe00640 1 -- 172.21.15.19:0/3190284606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c037070 con 0x7fa230160360 2024-04-25T11:38:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.222+0000 7fa22fe00640 1 -- 172.21.15.19:0/3190284606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c02ae10 con 0x7fa230160360 2024-04-25T11:38:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.222+0000 7fa22fe00640 1 -- 172.21.15.19:0/3190284606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c032660 con 0x7fa230160360 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.222+0000 7fa237e00640 1 -- 172.21.15.19:0/3190284606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa230160360 msgr2=0x7fa2301647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.222+0000 7fa237e00640 1 --2- 172.21.15.19:0/3190284606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa230160360 0x7fa2301647f0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009870 tx=0x7fa21c02a100 comp rx=0 tx=0).stop 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.223+0000 7fa237e00640 1 -- 172.21.15.19:0/3190284606 shutdown_connections 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.223+0000 7fa237e00640 1 --2- 172.21.15.19:0/3190284606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2300b5c90 0x7fa2300b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.223+0000 7fa237e00640 1 --2- 172.21.15.19:0/3190284606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa230160360 0x7fa2301647f0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.223+0000 7fa237e00640 1 -- 172.21.15.19:0/3190284606 >> 172.21.15.19:0/3190284606 conn(0x7fa2300d1ae0 msgr2=0x7fa2300d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.223+0000 7fa237e00640 1 -- 172.21.15.19:0/3190284606 shutdown_connections 2024-04-25T11:38:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.223+0000 7fa237e00640 1 -- 172.21.15.19:0/3190284606 wait complete. 2024-04-25T11:38:58.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.225+0000 7fa237e00640 1 Processor -- start 2024-04-25T11:38:58.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.225+0000 7fa237e00640 1 -- start start 2024-04-25T11:38:58.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa237e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa230070680 0x7fa230150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa237e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2300b5c90 0x7fa23015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa237e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa230160360 0x7fa2300c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa237e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2300afbb0 con 0x7fa2300b5c90 2024-04-25T11:38:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa237e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa23006fcc0 con 0x7fa230070680 2024-04-25T11:38:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa237e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa2300afd30 con 0x7fa230160360 2024-04-25T11:38:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa234a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2300b5c90 0x7fa23015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa234a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2300b5c90 0x7fa23015dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33410/0 (socket says 172.21.15.19:33410) 2024-04-25T11:38:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa234a00640 1 -- 172.21.15.19:0/641724148 learned_addr learned my addr 172.21.15.19:0/641724148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa235400640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa230070680 0x7fa230150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.227+0000 7fa235e00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa230160360 0x7fa2300c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.228+0000 7fa234a00640 1 -- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa230160360 msgr2=0x7fa2300c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.228+0000 7fa234a00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa230160360 0x7fa2300c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.228+0000 7fa234a00640 1 -- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa230070680 msgr2=0x7fa230150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.228+0000 7fa234a00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa230070680 0x7fa230150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.228+0000 7fa234a00640 1 -- 172.21.15.19:0/641724148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2300c6b10 con 0x7fa2300b5c90 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.228+0000 7fa235400640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa230070680 0x7fa230150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.228+0000 7fa234a00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2300b5c90 0x7fa23015dd00 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fa21c000c00 tx=0x7fa21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.229+0000 7fa22d600640 1 -- 172.21.15.19:0/641724148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c041020 con 0x7fa2300b5c90 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.229+0000 7fa22d600640 1 -- 172.21.15.19:0/641724148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c03c670 con 0x7fa2300b5c90 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.229+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2301509c0 con 0x7fa2300b5c90 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.229+0000 7fa22d600640 1 -- 172.21.15.19:0/641724148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c037040 con 0x7fa2300b5c90 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.229+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23015eb30 con 0x7fa2300b5c90 2024-04-25T11:38:58.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.230+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa230050740 con 0x7fa2300b5c90 2024-04-25T11:38:58.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.232+0000 7fa22d600640 1 -- 172.21.15.19:0/641724148 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa21c044020 con 0x7fa2300b5c90 2024-04-25T11:38:58.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.233+0000 7fa22d600640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2100744e0 0x7fa2100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.233+0000 7fa22d600640 1 -- 172.21.15.19:0/641724148 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7fa21c0b6100 con 0x7fa2300b5c90 2024-04-25T11:38:58.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.234+0000 7fa235400640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2100744e0 0x7fa2100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.235+0000 7fa235400640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2100744e0 0x7fa2100769f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa220006fd0 tx=0x7fa220008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:58.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.239+0000 7fa22d600640 1 -- 172.21.15.19:0/641724148 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa21c04b070 con 0x7fa2300b5c90 2024-04-25T11:38:58.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.492+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2300b0390 con 0x7fa2300b5c90 2024-04-25T11:38:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.494+0000 7fa22d600640 1 -- 172.21.15.19:0/641724148 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) ==== 95+0+5178 (secure 0 0 0) 0x7fa2300b65e0 con 0x7fa2300b5c90 2024-04-25T11:38:58.481 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:58.481 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":237,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1735507451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1735507451},{"type":"v1","addr":"172.21.15.19:6835","nonce":1735507451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":237}],"filesystems":[{"mdsmap":{"epoch":236,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:57.343081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:58.482 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 237 2024-04-25T11:38:58.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.496+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2100744e0 msgr2=0x7fa2100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.496+0000 7fa237e00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2100744e0 0x7fa2100769f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa220006fd0 tx=0x7fa220008040 comp rx=0 tx=0).stop 2024-04-25T11:38:58.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2300b5c90 msgr2=0x7fa23015dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2300b5c90 0x7fa23015dd00 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fa21c000c00 tx=0x7fa21c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 shutdown_connections 2024-04-25T11:38:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2100744e0 0x7fa2100769f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa230160360 0x7fa2300c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2300b5c90 0x7fa23015dd00 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 --2- 172.21.15.19:0/641724148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa230070680 0x7fa230150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 >> 172.21.15.19:0/641724148 conn(0x7fa2300d1ae0 msgr2=0x7fa2300b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 shutdown_connections 2024-04-25T11:38:58.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.497+0000 7fa237e00640 1 -- 172.21.15.19:0/641724148 wait complete. 2024-04-25T11:38:58.515 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:38:58.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.807+0000 7f79f3e00640 1 Processor -- start 2024-04-25T11:38:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.808+0000 7f79f3e00640 1 -- start start 2024-04-25T11:38:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.809+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 0x7f79f4161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.809+0000 7f79f3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79f40b0800 0x7f79f4070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.809+0000 7f79f3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f41615d0 con 0x7f79f40707d0 2024-04-25T11:38:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.809+0000 7f79f3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f79f4070be0 con 0x7f79f40b0800 2024-04-25T11:38:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79f3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79f40cffc0 con 0x7f79f40cf9e0 2024-04-25T11:38:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79ea000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f79f40b0800 0x7f79f4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79f3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f79f40707d0 0x7f79f41569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47840/0 (socket says 172.21.15.19:47840) 2024-04-25T11:38:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79f3400640 1 -- 172.21.15.19:0/981571161 learned_addr learned my addr 172.21.15.19:0/981571161 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:38:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79f2a00640 1 --2- 172.21.15.19:0/981571161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 0x7f79f4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79ea000640 1 -- 172.21.15.19:0/981571161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 msgr2=0x7f79f4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79ea000640 1 --2- 172.21.15.19:0/981571161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 0x7f79f4161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79ea000640 1 -- 172.21.15.19:0/981571161 >> v1:172.21.15.19:6789/0 conn(0x7f79f40707d0 legacy=0x7f79f41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79ea000640 1 -- 172.21.15.19:0/981571161 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f40d02a0 con 0x7f79f40b0800 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.810+0000 7f79f2000640 1 -- 172.21.15.19:0/981571161 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3085244208 0 0) 0x7f79ec001490 con 0x7f79f40707d0 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.812+0000 7f79ea000640 1 --2- 172.21.15.19:0/981571161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40b0800 0x7f79f4070290 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0097a0 tx=0x7f79dc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b949cab25228106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.812+0000 7f79f2000640 1 -- 172.21.15.19:0/981571161 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc037070 con 0x7f79f40b0800 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.812+0000 7f79f2000640 1 -- 172.21.15.19:0/981571161 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79dc02ae50 con 0x7f79f40b0800 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.812+0000 7f79f2000640 1 -- 172.21.15.19:0/981571161 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc032660 con 0x7f79f40b0800 2024-04-25T11:38:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 -- 172.21.15.19:0/981571161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40b0800 msgr2=0x7f79f4070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 --2- 172.21.15.19:0/981571161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40b0800 0x7f79f4070290 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0097a0 tx=0x7f79dc02a100 comp rx=0 tx=0).stop 2024-04-25T11:38:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 -- 172.21.15.19:0/981571161 shutdown_connections 2024-04-25T11:38:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 --2- 172.21.15.19:0/981571161 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40b0800 0x7f79f4070290 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 --2- 172.21.15.19:0/981571161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 0x7f79f4161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 -- 172.21.15.19:0/981571161 >> 172.21.15.19:0/981571161 conn(0x7f79f40d1720 msgr2=0x7f79f40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 -- 172.21.15.19:0/981571161 shutdown_connections 2024-04-25T11:38:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.813+0000 7f79f3e00640 1 -- 172.21.15.19:0/981571161 wait complete. 2024-04-25T11:38:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.815+0000 7f79f3e00640 1 Processor -- start 2024-04-25T11:38:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.815+0000 7f79f3e00640 1 -- start start 2024-04-25T11:38:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40707d0 0x7f79f4147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f40b0800 0x7f79f41441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 0x7f79f4157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79f40b0c10 con 0x7f79f40707d0 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f79f40d0860 con 0x7f79f40b0800 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f79f40b0d90 con 0x7f79f40cf9e0 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40707d0 0x7f79f4147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40707d0 0x7f79f4147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33428/0 (socket says 172.21.15.19:33428) 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.817+0000 7f79f2a00640 1 -- 172.21.15.19:0/3455628706 learned_addr learned my addr 172.21.15.19:0/3455628706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79f2a00640 1 -- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 msgr2=0x7f79f4157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79f2a00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 0x7f79f4157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79f2a00640 1 -- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f40b0800 msgr2=0x7f79f41441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79f2a00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f40b0800 0x7f79f41441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79f2a00640 1 -- 172.21.15.19:0/3455628706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f4147ba0 con 0x7f79f40707d0 2024-04-25T11:38:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79f2a00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40707d0 0x7f79f4147660 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f79e400d5d0 tx=0x7f79e400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79eb400640 1 -- 172.21.15.19:0/3455628706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4004240 con 0x7f79f40707d0 2024-04-25T11:38:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79eb400640 1 -- 172.21.15.19:0/3455628706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e40043e0 con 0x7f79f40707d0 2024-04-25T11:38:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f41f98f0 con 0x7f79f40707d0 2024-04-25T11:38:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.818+0000 7f79eb400640 1 -- 172.21.15.19:0/3455628706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4003830 con 0x7f79f40707d0 2024-04-25T11:38:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.819+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f41fa070 con 0x7f79f40707d0 2024-04-25T11:38:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.820+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f4050740 con 0x7f79f40707d0 2024-04-25T11:38:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.821+0000 7f79eb400640 1 -- 172.21.15.19:0/3455628706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79e40039d0 con 0x7f79f40707d0 2024-04-25T11:38:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.823+0000 7f79eb400640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d00744b0 0x7f79d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.823+0000 7f79eb400640 1 -- 172.21.15.19:0/3455628706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7f79e4095d70 con 0x7f79f40707d0 2024-04-25T11:38:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.823+0000 7f79ea000640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d00744b0 0x7f79d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.825+0000 7f79ea000640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d00744b0 0x7f79d00769c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f79dc034000 tx=0x7f79dc02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:58.828+0000 7f79eb400640 1 -- 172.21.15.19:0/3455628706 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79f4147ba0 con 0x7f79f40707d0 2024-04-25T11:38:59.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.079+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79f4144c10 con 0x7f79f40707d0 2024-04-25T11:38:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.081+0000 7f79eb400640 1 -- 172.21.15.19:0/3455628706 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) ==== 95+0+5178 (secure 0 0 0) 0x7f79e405fca0 con 0x7f79f40707d0 2024-04-25T11:38:59.068 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:38:59.069 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":237,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1735507451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1735507451},{"type":"v1","addr":"172.21.15.19:6835","nonce":1735507451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":237}],"filesystems":[{"mdsmap":{"epoch":236,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:38:57.343081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:38:59.069 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 237 2024-04-25T11:38:59.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d00744b0 msgr2=0x7f79d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d00744b0 0x7f79d00769c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f79dc034000 tx=0x7f79dc02c000 comp rx=0 tx=0).stop 2024-04-25T11:38:59.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40707d0 msgr2=0x7f79f4147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40707d0 0x7f79f4147660 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f79e400d5d0 tx=0x7f79e400da90 comp rx=0 tx=0).stop 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 shutdown_connections 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f79d00744b0 0x7f79d00769c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79f40b0800 0x7f79f41441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79f40cf9e0 0x7f79f4157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 --2- 172.21.15.19:0/3455628706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79f40707d0 0x7f79f4147660 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 >> 172.21.15.19:0/3455628706 conn(0x7f79f40d1720 msgr2=0x7f79f4162200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.084+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 shutdown_connections 2024-04-25T11:38:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.085+0000 7f79f3e00640 1 -- 172.21.15.19:0/3455628706 wait complete. 2024-04-25T11:38:59.100 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2024-04-25T11:38:59.100 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2024-04-25T11:38:59.100 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:38:59.100 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:59.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:38:59.153 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:38:59.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:59.153 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:59.208 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:38:59.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:38:59.209 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:59.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.513+0000 7ff17b200640 1 Processor -- start 2024-04-25T11:38:59.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.513+0000 7ff17b200640 1 -- start start 2024-04-25T11:38:59.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff17b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff17c06c5c0 0x7ff17c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff17b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff17c144370 0x7ff17c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff17b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff17c0b50e0 con 0x7ff17c0b4cd0 2024-04-25T11:38:59.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff17b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff17c0d1230 con 0x7ff17c06c5c0 2024-04-25T11:38:59.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff17b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff17c0d06b0 con 0x7ff17c144370 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff179e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff17c06c5c0 0x7ff17c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff179e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff17c06c5c0 0x7ff17c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33440/0 (socket says 172.21.15.19:33440) 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff17a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff17c0b4cd0 0x7ff17c161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47850/0 (socket says 172.21.15.19:47850) 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.516+0000 7ff179e00640 1 -- 172.21.15.19:0/994973658 learned_addr learned my addr 172.21.15.19:0/994973658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.517+0000 7ff179400640 1 --2- 172.21.15.19:0/994973658 >> v2:172.21.15.169:3300/0 conn(0x7ff17c144370 0x7ff17c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.517+0000 7ff179e00640 1 -- 172.21.15.19:0/994973658 >> v2:172.21.15.169:3300/0 conn(0x7ff17c144370 msgr2=0x7ff17c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.517+0000 7ff179e00640 1 --2- 172.21.15.19:0/994973658 >> v2:172.21.15.169:3300/0 conn(0x7ff17c144370 0x7ff17c0b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.517+0000 7ff179e00640 1 -- 172.21.15.19:0/994973658 >> v1:172.21.15.19:6789/0 conn(0x7ff17c0b4cd0 legacy=0x7ff17c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.517+0000 7ff179e00640 1 -- 172.21.15.19:0/994973658 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c0d0990 con 0x7ff17c06c5c0 2024-04-25T11:38:59.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.518+0000 7ff179e00640 1 --2- 172.21.15.19:0/994973658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c0cfa70 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0b5550 tx=0x7ff16c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95ffa6c12e66225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.518+0000 7ff178a00640 1 -- 172.21.15.19:0/994973658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c037070 con 0x7ff17c06c5c0 2024-04-25T11:38:59.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.518+0000 7ff178a00640 1 -- 172.21.15.19:0/994973658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c032440 con 0x7ff17c06c5c0 2024-04-25T11:38:59.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.518+0000 7ff178a00640 1 -- 172.21.15.19:0/994973658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c032810 con 0x7ff17c06c5c0 2024-04-25T11:38:59.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 -- 172.21.15.19:0/994973658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 msgr2=0x7ff17c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 --2- 172.21.15.19:0/994973658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c0cfa70 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0b5550 tx=0x7ff16c02a270 comp rx=0 tx=0).stop 2024-04-25T11:38:59.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 -- 172.21.15.19:0/994973658 shutdown_connections 2024-04-25T11:38:59.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 --2- 172.21.15.19:0/994973658 >> v2:172.21.15.169:3300/0 conn(0x7ff17c144370 0x7ff17c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 --2- 172.21.15.19:0/994973658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c0cfa70 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 -- 172.21.15.19:0/994973658 >> 172.21.15.19:0/994973658 conn(0x7ff17c0d1720 msgr2=0x7ff17c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 -- 172.21.15.19:0/994973658 shutdown_connections 2024-04-25T11:38:59.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.519+0000 7ff17b200640 1 -- 172.21.15.19:0/994973658 wait complete. 2024-04-25T11:38:59.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.522+0000 7ff17b200640 1 Processor -- start 2024-04-25T11:38:59.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.522+0000 7ff17b200640 1 -- start start 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.523+0000 7ff17b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.523+0000 7ff17b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0b4cd0 0x7ff17c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.523+0000 7ff17b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c144370 0x7ff17c1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.523+0000 7ff17b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c0d02d0 con 0x7ff17c06c5c0 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.523+0000 7ff17b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c0d0f50 con 0x7ff17c0b4cd0 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.523+0000 7ff17b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff17c0d0450 con 0x7ff17c144370 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c15d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33452/0 (socket says 172.21.15.19:33452) 2024-04-25T11:38:59.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 -- 172.21.15.19:0/3544709934 learned_addr learned my addr 172.21.15.19:0/3544709934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:38:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179400640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0b4cd0 0x7ff17c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 -- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c144370 msgr2=0x7ff17c1426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c144370 0x7ff17c1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 -- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0b4cd0 msgr2=0x7ff17c0b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0b4cd0 0x7ff17c0b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179e00640 1 -- 172.21.15.19:0/3544709934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c15fcf0 con 0x7ff17c06c5c0 2024-04-25T11:38:59.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.524+0000 7ff179400640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0b4cd0 0x7ff17c0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.525+0000 7ff179e00640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c15d1f0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000c00 tx=0x7ff16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.525+0000 7ff172000640 1 -- 172.21.15.19:0/3544709934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c041020 con 0x7ff17c06c5c0 2024-04-25T11:38:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.525+0000 7ff172000640 1 -- 172.21.15.19:0/3544709934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c0306c0 con 0x7ff17c06c5c0 2024-04-25T11:38:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.525+0000 7ff172000640 1 -- 172.21.15.19:0/3544709934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c037040 con 0x7ff17c06c5c0 2024-04-25T11:38:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.525+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c15ffd0 con 0x7ff17c06c5c0 2024-04-25T11:38:59.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.525+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c160590 con 0x7ff17c06c5c0 2024-04-25T11:38:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.526+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c160970 con 0x7ff17c06c5c0 2024-04-25T11:38:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.528+0000 7ff172000640 1 -- 172.21.15.19:0/3544709934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff16c044020 con 0x7ff17c06c5c0 2024-04-25T11:38:59.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.529+0000 7ff172000640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff15c0744e0 0x7ff15c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:59.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.529+0000 7ff172000640 1 -- 172.21.15.19:0/3544709934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7ff16c0b61f0 con 0x7ff17c06c5c0 2024-04-25T11:38:59.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.530+0000 7ff179400640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff15c0744e0 0x7ff15c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:59.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.531+0000 7ff179400640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff15c0744e0 0x7ff15c0769f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff168006fd0 tx=0x7ff168008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:59.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.535+0000 7ff172000640 1 -- 172.21.15.19:0/3544709934 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff16c04b030 con 0x7ff17c06c5c0 2024-04-25T11:38:59.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:38:59.783+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff17c06e100 con 0x7ff17c06c5c0 2024-04-25T11:39:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.366+0000 7ff172000640 1 -- 172.21.15.19:0/3544709934 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v238) ==== 161+0+0 (secure 0 0 0) 0x7ff16c080240 con 0x7ff17c06c5c0 2024-04-25T11:39:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:00.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff15c0744e0 msgr2=0x7ff15c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff15c0744e0 0x7ff15c0769f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff168006fd0 tx=0x7ff168008040 comp rx=0 tx=0).stop 2024-04-25T11:39:00.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 msgr2=0x7ff17c15d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c15d1f0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7ff16c000c00 tx=0x7ff16c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 shutdown_connections 2024-04-25T11:39:00.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff15c0744e0 0x7ff15c0769f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff17c0b4cd0 0x7ff17c0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff17c144370 0x7ff17c1426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 --2- 172.21.15.19:0/3544709934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff17c06c5c0 0x7ff17c15d1f0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 >> 172.21.15.19:0/3544709934 conn(0x7ff17c0d1720 msgr2=0x7ff17c0b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 shutdown_connections 2024-04-25T11:39:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.371+0000 7ff17b200640 1 -- 172.21.15.19:0/3544709934 wait complete. 2024-04-25T11:39:00.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:00.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.682+0000 7fbd03e00640 1 Processor -- start 2024-04-25T11:39:00.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.682+0000 7fbd03e00640 1 -- start start 2024-04-25T11:39:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbd03e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd040cf2f0 0x7fbd040b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbd03e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbd041608d0 0x7fbd04160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbd03e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd04161220 con 0x7fbd0415a810 2024-04-25T11:39:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbd03e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbd0406f870 con 0x7fbd040cf2f0 2024-04-25T11:39:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbd03e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbd040b74d0 con 0x7fbd041608d0 2024-04-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbcfbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbd0415a810 0x7fbd040b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47856/0 (socket says 172.21.15.19:47856) 2024-04-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbcfbe00640 1 -- 172.21.15.19:0/1419103867 learned_addr learned my addr 172.21.15.19:0/1419103867 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.685+0000 7fbd02a00640 1 --2- 172.21.15.19:0/1419103867 >> v2:172.21.15.19:3300/0 conn(0x7fbd040cf2f0 0x7fbd040b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.686+0000 7fbd03400640 1 --2- 172.21.15.19:0/1419103867 >> v2:172.21.15.169:3300/0 conn(0x7fbd041608d0 0x7fbd04160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.686+0000 7fbd02000640 1 -- 172.21.15.19:0/1419103867 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 992183391 0 0) 0x7fbcec001110 con 0x7fbd0415a810 2024-04-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.686+0000 7fbd02000640 1 -- 172.21.15.19:0/1419103867 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce8005730 con 0x7fbd0415a810 2024-04-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.686+0000 7fbd02a00640 1 -- 172.21.15.19:0/1419103867 >> v2:172.21.15.169:3300/0 conn(0x7fbd041608d0 msgr2=0x7fbd04160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.686+0000 7fbd02a00640 1 --2- 172.21.15.19:0/1419103867 >> v2:172.21.15.169:3300/0 conn(0x7fbd041608d0 0x7fbd04160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.686+0000 7fbd02a00640 1 -- 172.21.15.19:0/1419103867 >> v1:172.21.15.19:6789/0 conn(0x7fbd0415a810 legacy=0x7fbd040b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.686+0000 7fbd02a00640 1 -- 172.21.15.19:0/1419103867 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd040b77b0 con 0x7fbd040cf2f0 2024-04-25T11:39:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.687+0000 7fbd02a00640 1 --2- 172.21.15.19:0/1419103867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd040cf2f0 0x7fbd040b6e10 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fbd040d27a0 tx=0x7fbcf402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a3d1d349ba4bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.687+0000 7fbd02000640 1 -- 172.21.15.19:0/1419103867 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4037070 con 0x7fbd040cf2f0 2024-04-25T11:39:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.687+0000 7fbd02000640 1 -- 172.21.15.19:0/1419103867 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf4032440 con 0x7fbd040cf2f0 2024-04-25T11:39:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.687+0000 7fbd02000640 1 -- 172.21.15.19:0/1419103867 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4032810 con 0x7fbd040cf2f0 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 -- 172.21.15.19:0/1419103867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd040cf2f0 msgr2=0x7fbd040b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 --2- 172.21.15.19:0/1419103867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd040cf2f0 0x7fbd040b6e10 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fbd040d27a0 tx=0x7fbcf402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 -- 172.21.15.19:0/1419103867 shutdown_connections 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 --2- 172.21.15.19:0/1419103867 >> v2:172.21.15.169:3300/0 conn(0x7fbd041608d0 0x7fbd04160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 --2- 172.21.15.19:0/1419103867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd040cf2f0 0x7fbd040b6e10 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 -- 172.21.15.19:0/1419103867 >> 172.21.15.19:0/1419103867 conn(0x7fbd040d1720 msgr2=0x7fbd040d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 -- 172.21.15.19:0/1419103867 shutdown_connections 2024-04-25T11:39:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.688+0000 7fbd03e00640 1 -- 172.21.15.19:0/1419103867 wait complete. 2024-04-25T11:39:00.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.691+0000 7fbd03e00640 1 Processor -- start 2024-04-25T11:39:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.691+0000 7fbd03e00640 1 -- start start 2024-04-25T11:39:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.692+0000 7fbd03e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd040cf2f0 0x7fbd041555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.692+0000 7fbd03e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0415a810 0x7fbd04156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.692+0000 7fbd03e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd041608d0 0x7fbd04158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.692+0000 7fbd03e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0415ac20 con 0x7fbd0415a810 2024-04-25T11:39:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.692+0000 7fbd03e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0406f500 con 0x7fbd041608d0 2024-04-25T11:39:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.692+0000 7fbd03e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0406fd00 con 0x7fbd040cf2f0 2024-04-25T11:39:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0415a810 0x7fbd04156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0415a810 0x7fbd04156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33458/0 (socket says 172.21.15.19:33458) 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 -- 172.21.15.19:0/2066044214 learned_addr learned my addr 172.21.15.19:0/2066044214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbd03400640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd041608d0 0x7fbd04158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbd02a00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd040cf2f0 0x7fbd041555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 -- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd040cf2f0 msgr2=0x7fbd041555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd040cf2f0 0x7fbd041555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 -- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd041608d0 msgr2=0x7fbd04158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd041608d0 0x7fbd04158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbcfbe00640 1 -- 172.21.15.19:0/2066044214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd04157c40 con 0x7fbd0415a810 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbd02a00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd040cf2f0 0x7fbd041555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.693+0000 7fbd03400640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd041608d0 0x7fbd04158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.694+0000 7fbcfbe00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0415a810 0x7fbd04156d60 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c3a0 tx=0x7fbcec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.694+0000 7fbcfaa00640 1 -- 172.21.15.19:0/2066044214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec017070 con 0x7fbd0415a810 2024-04-25T11:39:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.694+0000 7fbcfaa00640 1 -- 172.21.15.19:0/2066044214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcec005ce0 con 0x7fbd0415a810 2024-04-25T11:39:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.694+0000 7fbcfaa00640 1 -- 172.21.15.19:0/2066044214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec007310 con 0x7fbd0415a810 2024-04-25T11:39:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.694+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd04157f20 con 0x7fbd0415a810 2024-04-25T11:39:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.694+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd041f6670 con 0x7fbd0415a810 2024-04-25T11:39:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.695+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd04050740 con 0x7fbd0415a810 2024-04-25T11:39:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.697+0000 7fbcfaa00640 1 -- 172.21.15.19:0/2066044214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbcec01a070 con 0x7fbd0415a810 2024-04-25T11:39:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.698+0000 7fbcfaa00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbce00744b0 0x7fbce00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.698+0000 7fbcfaa00640 1 -- 172.21.15.19:0/2066044214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7fbcec094df0 con 0x7fbd0415a810 2024-04-25T11:39:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.699+0000 7fbd02a00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbce00744b0 0x7fbce00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:00.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.700+0000 7fbd02a00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbce00744b0 0x7fbce00769c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4034000 tx=0x7fbcf402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:00.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.703+0000 7fbcfaa00640 1 -- 172.21.15.19:0/2066044214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd04161220 con 0x7fbd0415a810 2024-04-25T11:39:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:00.984+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fbd04157990 con 0x7fbd0415a810 2024-04-25T11:39:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.390+0000 7fbcfaa00640 1 -- 172.21.15.19:0/2066044214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v239) ==== 101+0+0 (secure 0 0 0) 0x7fbcec05edb0 con 0x7fbd0415a810 2024-04-25T11:39:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbce00744b0 msgr2=0x7fbce00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbce00744b0 0x7fbce00769c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4034000 tx=0x7fbcf402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0415a810 msgr2=0x7fbd04156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0415a810 0x7fbd04156d60 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c3a0 tx=0x7fbcec00c860 comp rx=0 tx=0).stop 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 shutdown_connections 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbce00744b0 0x7fbce00769c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd041608d0 0x7fbd04158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0415a810 0x7fbd04156d60 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 --2- 172.21.15.19:0/2066044214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd040cf2f0 0x7fbd041555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.395+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 >> 172.21.15.19:0/2066044214 conn(0x7fbd040d1720 msgr2=0x7fbd040b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.396+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 shutdown_connections 2024-04-25T11:39:01.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.396+0000 7fbd03e00640 1 -- 172.21.15.19:0/2066044214 wait complete. 2024-04-25T11:39:01.411 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.607+0000 7f828d5dbf40 1 Processor -- start 2024-04-25T11:39:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.607+0000 7f828d5dbf40 1 -- start start 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f828d5dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a1d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f828d5dbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a100560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f828d5dbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f828d5dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56401a14b3f0 con 0x56401a1021e0 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f828d5dbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56401a14ae70 con 0x56401a1dbdd0 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f828d5dbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56401a14b6d0 con 0x56401a1025f0 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f8286800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a1d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f8286800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a1d5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33474/0 (socket says 172.21.15.19:33474) 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.608+0000 7f8286800640 1 -- 172.21.15.19:0/1387319485 learned_addr learned my addr 172.21.15.19:0/1387319485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8287200640 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8285e00640 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8286800640 1 -- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 msgr2=0x56401a100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8286800640 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a100560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8286800640 1 -- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 msgr2=0x56401a100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8286800640 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8286800640 1 -- 172.21.15.19:0/1387319485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401a02bb30 con 0x56401a1021e0 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8287200640 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:01.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.609+0000 7f8286800640 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a1d5b40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f827800b620 tx=0x7f827800bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d22483716debd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f8284a00640 1 -- 172.21.15.19:0/1387319485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278014020 con 0x56401a1021e0 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f8284a00640 1 -- 172.21.15.19:0/1387319485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8278010820 con 0x56401a1021e0 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f8284a00640 1 -- 172.21.15.19:0/1387319485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278010bd0 con 0x56401a1021e0 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f828d5dbf40 1 -- 172.21.15.19:0/1387319485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 msgr2=0x56401a1d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a1d5b40 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f827800b620 tx=0x7f827800bae0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f828d5dbf40 1 -- 172.21.15.19:0/1387319485 shutdown_connections 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/1387319485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a1d5b40 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.610+0000 7f828d5dbf40 1 -- 172.21.15.19:0/1387319485 >> 172.21.15.19:0/1387319485 conn(0x56401a024ac0 msgr2=0x56401a101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.611+0000 7f828d5dbf40 1 -- 172.21.15.19:0/1387319485 shutdown_connections 2024-04-25T11:39:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.611+0000 7f828d5dbf40 1 -- 172.21.15.19:0/1387319485 wait complete. 2024-04-25T11:39:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.612+0000 7f828d5dbf40 1 Processor -- start 2024-04-25T11:39:01.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.612+0000 7f828d5dbf40 1 -- start start 2024-04-25T11:39:01.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.613+0000 7f828d5dbf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a14c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.613+0000 7f828d5dbf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a268100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.613+0000 7f828d5dbf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a268640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.613+0000 7f828d5dbf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56401a146b80 con 0x56401a1021e0 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.613+0000 7f828d5dbf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56401a133df0 con 0x56401a1dbdd0 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.613+0000 7f828d5dbf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56401a2692f0 con 0x56401a1025f0 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a14c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a14c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33482/0 (socket says 172.21.15.19:33482) 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 learned_addr learned my addr 172.21.15.19:0/3592409345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 msgr2=0x56401a268100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a268100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8287200640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a268640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a268100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 msgr2=0x56401a268640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a268640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401a1d49d0 con 0x56401a1021e0 2024-04-25T11:39:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8287200640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56401a1dbdd0 0x56401a268640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.614+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56401a1025f0 0x56401a268100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.615+0000 7f8286800640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a14c440 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8278013040 tx=0x7f82780033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.615+0000 7f827f400640 1 -- 172.21.15.19:0/3592409345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278029070 con 0x56401a1021e0 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.615+0000 7f827f400640 1 -- 172.21.15.19:0/3592409345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827801a860 con 0x56401a1021e0 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.615+0000 7f827f400640 1 -- 172.21.15.19:0/3592409345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278014040 con 0x56401a1021e0 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.615+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56401a1d61f0 con 0x56401a1021e0 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.615+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401a132e10 con 0x56401a1021e0 2024-04-25T11:39:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.617+0000 7f827f400640 1 -- 172.21.15.19:0/3592409345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f827802c070 con 0x56401a1021e0 2024-04-25T11:39:01.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.618+0000 7f827f400640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8268072ec0 0x7f82680753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.618+0000 7f827f400640 1 -- 172.21.15.19:0/3592409345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7f8278095da0 con 0x56401a1021e0 2024-04-25T11:39:01.606 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:01.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.619+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56401a1d4c40 0x56401a273130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.619+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a14c980 con 0x56401a1d4c40 2024-04-25T11:39:01.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.619+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8268072ec0 0x7f82680753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.619+0000 7f8287200640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56401a1d4c40 0x56401a273130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.621+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8268072ec0 0x7f82680753d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8274057000 tx=0x7f8274052000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.621+0000 7f8287200640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56401a1d4c40 0x56401a273130 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.625+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v115'691 uv665 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.625+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a1324e0 con 0x56401a1d4c40 2024-04-25T11:39:01.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.628+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v115'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.628+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a268b80 con 0x56401a1d4c40 2024-04-25T11:39:01.618 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.631+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v115'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.631+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a268f50 con 0x56401a1d4c40 2024-04-25T11:39:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.634+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.634+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27cf00 con 0x56401a1d4c40 2024-04-25T11:39:01.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.637+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.637+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27bef0 con 0x56401a1d4c40 2024-04-25T11:39:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.641+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.641+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c290 con 0x56401a1d4c40 2024-04-25T11:39:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.645+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v115'697 uv671 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.645+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.648+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v115'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.649+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.640 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.653+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v115'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.653+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.657+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.657+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.661+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'701 uv701 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.661+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.665+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'702 uv702 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.666+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.670+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v115'703 uv678 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.670+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.675+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'704 uv704 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.675+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.666 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:01.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.679+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v115'705 uv705 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.679+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.684+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'706 uv706 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.684+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27cab0 con 0x56401a1d4c40 2024-04-25T11:39:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.689+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'707 uv707 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.689+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27cab0 con 0x56401a1d4c40 2024-04-25T11:39:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.693+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'708 uv708 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.693+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.698+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'709 uv709 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.698+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.702+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v115'710 uv684 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.703+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.707+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v115'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.707+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.698 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.711+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v115'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.711+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.716+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.716+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.721+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.721+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.725+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.726+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.731+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v115'716 uv690 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.731+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.735+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v115'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.735+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.727 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.740+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v115'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.740+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.744+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.744+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.749+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.749+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a27c630 con 0x56401a1d4c40 2024-04-25T11:39:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.754+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.754+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56401a27d7b0 0x56401a27fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.754+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a280110 con 0x56401a27d7b0 2024-04-25T11:39:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.755+0000 7f8286800640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56401a27d7b0 0x56401a27fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.756+0000 7f8286800640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56401a27d7b0 0x56401a27fbd0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.761+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'505 uv488 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.761+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a2804b0 con 0x56401a27d7b0 2024-04-25T11:39:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.766+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'506 uv506 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.766+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a280850 con 0x56401a27d7b0 2024-04-25T11:39:01.757 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.770+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'507 uv507 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.770+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a280bf0 con 0x56401a27d7b0 2024-04-25T11:39:01.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.775+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v115'508 uv508 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.775+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a280f90 con 0x56401a27d7b0 2024-04-25T11:39:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.779+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'509 uv509 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.779+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a281330 con 0x56401a27d7b0 2024-04-25T11:39:01.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.784+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v115'510 uv493 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.784+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.788+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v115'511 uv511 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.788+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.779 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.792+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v115'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.792+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.797+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.797+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.801+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.801+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.806+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v115'515 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.806+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.810+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v115'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.810+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.801 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:01.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.814+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v115'517 uv517 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.815+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285dc0 con 0x56401a27d7b0 2024-04-25T11:39:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.820+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v115'518 uv518 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.820+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285dc0 con 0x56401a27d7b0 2024-04-25T11:39:01.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.824+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'519 uv519 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.825+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.829+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v115'520 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.829+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.833+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v115'521 uv521 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.834+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.824 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:01.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.837+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v115'522 uv522 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.838+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2826e0 con 0x56401a27d7b0 2024-04-25T11:39:01.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.841+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'523 uv523 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.842+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2826e0 con 0x56401a27d7b0 2024-04-25T11:39:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.845+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'524 uv524 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.845+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2816d0 con 0x56401a27d7b0 2024-04-25T11:39:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.850+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a27d7b0 2024-04-25T11:39:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.850+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56401a281d50 0x56401a284170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.850+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2847c0 con 0x56401a281d50 2024-04-25T11:39:01.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.851+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56401a281d50 0x56401a284170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.853+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56401a281d50 0x56401a284170 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.857+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.858+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a284c90 con 0x56401a281d50 2024-04-25T11:39:01.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.863+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v115'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:01.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.863+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285170 con 0x56401a281d50 2024-04-25T11:39:01.854 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:01.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.867+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v115'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:01.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.867+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a286a20 con 0x56401a281d50 2024-04-25T11:39:01.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.872+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:01.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.872+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a286dc0 con 0x56401a281d50 2024-04-25T11:39:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.876+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:01.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.876+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285a10 con 0x56401a281d50 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.881+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:01.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.881+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285e00 con 0x56401a1d4c40 2024-04-25T11:39:01.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.885+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'102 uv101 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.885+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a285e00 con 0x56401a1d4c40 2024-04-25T11:39:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.890+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'103 uv103 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.890+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285e00 con 0x56401a1d4c40 2024-04-25T11:39:01.881 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.894+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'104 uv104 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.894+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285e00 con 0x56401a1d4c40 2024-04-25T11:39:01.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.898+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'105 uv105 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.898+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56401a287160 0x56401a289580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.898+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285e00 con 0x56401a287160 2024-04-25T11:39:01.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.899+0000 7f8287200640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56401a287160 0x56401a289580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.902+0000 7f8287200640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56401a287160 0x56401a289580 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.906+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:01.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.906+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a285e00 con 0x56401a287160 2024-04-25T11:39:01.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.910+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v115'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:01.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.910+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a285e00 con 0x56401a287160 2024-04-25T11:39:01.901 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.914+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v115'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:01.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.914+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28aad0 con 0x56401a287160 2024-04-25T11:39:01.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.919+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:01.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.919+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28aad0 con 0x56401a287160 2024-04-25T11:39:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.922+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:01.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.923+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a289ac0 con 0x56401a287160 2024-04-25T11:39:01.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.927+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:01.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.927+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56401a289fd0 0x56401a28c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.927+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28caa0 con 0x56401a289fd0 2024-04-25T11:39:01.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.928+0000 7f8286800640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56401a289fd0 0x56401a28c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.929+0000 7f8286800640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56401a289fd0 0x56401a28c3f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.933+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v115'249 uv243 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.933+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a28cb60 con 0x56401a289fd0 2024-04-25T11:39:01.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.938+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v115'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.938+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28cf80 con 0x56401a289fd0 2024-04-25T11:39:01.929 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.942+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v115'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.942+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28df90 con 0x56401a289fd0 2024-04-25T11:39:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.946+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.947+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28df90 con 0x56401a289fd0 2024-04-25T11:39:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.951+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.951+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28cf80 con 0x56401a289fd0 2024-04-25T11:39:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.955+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.955+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28cf80 con 0x56401a289fd0 2024-04-25T11:39:01.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.959+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v115'255 uv248 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.959+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a28cf80 con 0x56401a289fd0 2024-04-25T11:39:01.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.963+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v115'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.963+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28cf80 con 0x56401a289fd0 2024-04-25T11:39:01.955 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:01.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.968+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v115'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.968+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28cf80 con 0x56401a289fd0 2024-04-25T11:39:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.972+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.972+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a28cf80 con 0x56401a289fd0 2024-04-25T11:39:01.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.976+0000 7f8286800640 1 -- 172.21.15.19:0/3592409345 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f827802b030 con 0x56401a289fd0 2024-04-25T11:39:01.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.976+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56401a28d490 0x56401a28f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:01.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.976+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290010 con 0x56401a28d490 2024-04-25T11:39:01.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.977+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56401a28d490 0x56401a28f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:01.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.979+0000 7f8285e00640 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56401a28d490 0x56401a28f930 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:01.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.982+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v115'112 uv111 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:01.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.982+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a2903e0 con 0x56401a28d490 2024-04-25T11:39:01.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.987+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v115'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:01.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.987+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290800 con 0x56401a28d490 2024-04-25T11:39:01.978 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:01.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.991+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v115'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:01.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.991+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290800 con 0x56401a28d490 2024-04-25T11:39:01.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.995+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:01.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.995+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290800 con 0x56401a28d490 2024-04-25T11:39:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.999+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:01.999+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290800 con 0x56401a1d4c40 2024-04-25T11:39:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.003+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.003+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a290800 con 0x56401a1d4c40 2024-04-25T11:39:01.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.008+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v115'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:01.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.008+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290800 con 0x56401a1d4c40 2024-04-25T11:39:02.001 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:02.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.012+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v115'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:02.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.012+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a291810 con 0x56401a1d4c40 2024-04-25T11:39:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.016+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.017+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a291810 con 0x56401a1d4c40 2024-04-25T11:39:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.021+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.021+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290800 con 0x56401a1d4c40 2024-04-25T11:39:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.025+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a1d4c40 2024-04-25T11:39:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.025+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.029+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v115'223 uv217 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.029+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.035+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v115'224 uv224 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.035+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.039+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v115'225 uv225 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.026 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:02.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.039+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.044+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'226 uv226 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.044+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.048+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'227 uv227 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.048+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.052+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v115'228 uv222 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.052+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.057+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v115'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.057+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.061+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v115'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.048 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.061+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a2855e0 con 0x56401a281d50 2024-04-25T11:39:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.065+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.065+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a293610 con 0x56401a281d50 2024-04-25T11:39:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.070+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a281d50 2024-04-25T11:39:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.070+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.074+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v115'99 uv98 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.074+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.079+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'100 uv100 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.079+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.069 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.083+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v115'101 uv101 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.083+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.086+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'102 uv102 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.086+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.091+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v115'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.091+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.095+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v115'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.095+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.086 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.099+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v115'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.099+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a291cc0 con 0x56401a287160 2024-04-25T11:39:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.103+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.103+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a291cc0 con 0x56401a287160 2024-04-25T11:39:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.108+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.108+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a287160 2024-04-25T11:39:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.113+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8280007610 con 0x56401a287160 2024-04-25T11:39:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.113+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.117+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v115'115 uv114 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.117+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.121+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v115'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.122+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.112 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.126+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v115'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.126+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.130+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.130+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.135+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.135+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.139+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v115'278 uv277 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.139+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.144+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v115'279 uv279 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.144+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.148+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v115'280 uv280 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.135 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:02.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.148+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.152+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v115'281 uv281 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.153+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.157+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'282 uv282 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.157+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.162+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v115'283 uv276 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.162+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.167+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v115'284 uv284 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.167+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.171+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v115'285 uv285 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.158 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:02.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.171+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a291cc0 con 0x56401a28d490 2024-04-25T11:39:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.175+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v115'286 uv286 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.176+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a291cc0 con 0x56401a28d490 2024-04-25T11:39:02.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.180+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v115'287 uv287 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.181+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a290cb0 con 0x56401a28d490 2024-04-25T11:39:02.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.185+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v115'288 uv288 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.185+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x56401a291210 con 0x56401a28d490 2024-04-25T11:39:02.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f8285e00640 1 -- 172.21.15.19:0/3592409345 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v115'289 uv289 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8274053ac0 con 0x56401a28d490 2024-04-25T11:39:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56401a27d7b0 msgr2=0x56401a27fbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x56401a27d7b0 0x56401a27fbd0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56401a289fd0 msgr2=0x56401a28c3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x56401a289fd0 0x56401a28c3f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56401a28d490 msgr2=0x56401a28f930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56401a28d490 0x56401a28f930 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56401a281d50 msgr2=0x56401a284170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x56401a281d50 0x56401a284170 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56401a1d4c40 msgr2=0x56401a273130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x56401a1d4c40 0x56401a273130 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56401a287160 msgr2=0x56401a289580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x56401a287160 0x56401a289580 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 reap_dead start 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 reap_dead start 2024-04-25T11:39:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.190+0000 7f8287200640 1 -- 172.21.15.19:0/3592409345 reap_dead start 2024-04-25T11:39:02.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8268072ec0 msgr2=0x7f82680753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8268072ec0 0x7f82680753d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8274057000 tx=0x7f8274052000 comp rx=0 tx=0).stop 2024-04-25T11:39:02.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 msgr2=0x56401a14c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a14c440 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8278013040 tx=0x7f82780033a0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 shutdown_connections 2024-04-25T11:39:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8268072ec0 0x7f82680753d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 --2- 172.21.15.19:0/3592409345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56401a1021e0 0x56401a14c440 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 >> 172.21.15.19:0/3592409345 conn(0x56401a024ac0 msgr2=0x56401a1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.191+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 shutdown_connections 2024-04-25T11:39:02.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.192+0000 7f828d5dbf40 1 -- 172.21.15.19:0/3592409345 wait complete. 2024-04-25T11:39:02.204 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2024-04-25T11:39:02.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.457+0000 7f7f86f97f40 1 Processor -- start 2024-04-25T11:39:02.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.457+0000 7f7f86f97f40 1 -- start start 2024-04-25T11:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f86f97f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220371e0 0x56512210d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f86f97f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651220375f0 0x56512210a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f86f97f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565122110fa0 0x56512219c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f86f97f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565121fddd20 con 0x5651220371e0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f86f97f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x565121fe1d60 con 0x5651220375f0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f86f97f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x565122080110 con 0x565122110fa0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f80200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220371e0 0x56512210d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f80200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220371e0 0x56512210d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33498/0 (socket says 172.21.15.19:33498) 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.458+0000 7f7f80200640 1 -- 172.21.15.19:0/3487323176 learned_addr learned my addr 172.21.15.19:0/3487323176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f7f800640 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651220375f0 0x56512210a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f80200640 1 -- 172.21.15.19:0/3487323176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565122110fa0 msgr2=0x56512219c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f80200640 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565122110fa0 0x56512219c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f80200640 1 -- 172.21.15.19:0/3487323176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651220375f0 msgr2=0x56512210a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f80200640 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651220375f0 0x56512210a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f80200640 1 -- 172.21.15.19:0/3487323176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56512207dea0 con 0x5651220371e0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f80200640 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220371e0 0x56512210d300 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000b240 tx=0x7f7f7000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2706628f0c4855aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.459+0000 7f7f7e400640 1 -- 172.21.15.19:0/3487323176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70008070 con 0x5651220371e0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f7e400640 1 -- 172.21.15.19:0/3487323176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7000d500 con 0x5651220371e0 2024-04-25T11:39:02.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f7e400640 1 -- 172.21.15.19:0/3487323176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000d910 con 0x5651220371e0 2024-04-25T11:39:02.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 -- 172.21.15.19:0/3487323176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220371e0 msgr2=0x56512210d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220371e0 0x56512210d300 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000b240 tx=0x7f7f7000b700 comp rx=0 tx=0).stop 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 -- 172.21.15.19:0/3487323176 shutdown_connections 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651220375f0 0x56512210a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x565122110fa0 0x56512219c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/3487323176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220371e0 0x56512210d300 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 -- 172.21.15.19:0/3487323176 >> 172.21.15.19:0/3487323176 conn(0x565121f59ac0 msgr2=0x565122069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 -- 172.21.15.19:0/3487323176 shutdown_connections 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.460+0000 7f7f86f97f40 1 -- 172.21.15.19:0/3487323176 wait complete. 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.462+0000 7f7f86f97f40 1 Processor -- start 2024-04-25T11:39:02.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.462+0000 7f7f86f97f40 1 -- start start 2024-04-25T11:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.463+0000 7f7f86f97f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651220371e0 0x56512210d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.463+0000 7f7f86f97f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220375f0 0x56512207a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.463+0000 7f7f86f97f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565122110fa0 0x5651221082d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.463+0000 7f7f86f97f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56512210b2d0 con 0x5651220375f0 2024-04-25T11:39:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.463+0000 7f7f86f97f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56512210b5b0 con 0x565122110fa0 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.463+0000 7f7f86f97f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56512207d4c0 con 0x5651220371e0 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220375f0 0x56512207a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220375f0 0x56512207a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33508/0 (socket says 172.21.15.19:33508) 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 -- 172.21.15.19:0/4101696801 learned_addr learned my addr 172.21.15.19:0/4101696801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f80c00640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565122110fa0 0x5651221082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f80200640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651220371e0 0x56512210d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 -- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651220371e0 msgr2=0x56512210d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651220371e0 0x56512210d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 -- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565122110fa0 msgr2=0x5651221082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x565122110fa0 0x5651221082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 -- 172.21.15.19:0/4101696801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56512207d220 con 0x5651220375f0 2024-04-25T11:39:02.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.464+0000 7f7f7f800640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220375f0 0x56512207a730 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00df70 tx=0x7f7f6c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.465+0000 7f7f7d000640 1 -- 172.21.15.19:0/4101696801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0030d0 con 0x5651220375f0 2024-04-25T11:39:02.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.465+0000 7f7f7d000640 1 -- 172.21.15.19:0/4101696801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6c016070 con 0x5651220375f0 2024-04-25T11:39:02.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.465+0000 7f7f7d000640 1 -- 172.21.15.19:0/4101696801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c001590 con 0x5651220375f0 2024-04-25T11:39:02.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.465+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565122079320 con 0x5651220375f0 2024-04-25T11:39:02.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.465+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565122107290 con 0x5651220375f0 2024-04-25T11:39:02.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.467+0000 7f7f7d000640 1 -- 172.21.15.19:0/4101696801 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f6c001730 con 0x5651220375f0 2024-04-25T11:39:02.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.468+0000 7f7f7d000640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f60072ec0 0x7f7f600753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.468+0000 7f7f7d000640 1 -- 172.21.15.19:0/4101696801 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7f7f6c094440 con 0x5651220375f0 2024-04-25T11:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.469+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x565122107500 0x56512207cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.469+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x565122084e10 con 0x565122107500 2024-04-25T11:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.469+0000 7f7f80200640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f60072ec0 0x7f7f600753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.469+0000 7f7f80c00640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x565122107500 0x56512207cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.470+0000 7f7f80c00640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x565122107500 0x56512207cb30 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.470+0000 7f7f80200640 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f60072ec0 0x7f7f600753d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000b240 tx=0x7f7f70009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f80c00640 1 -- 172.21.15.19:0/4101696801 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v115'526 uv526 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7f7804e040 con 0x565122107500 2024-04-25T11:39:02.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x565122107500 msgr2=0x56512207cb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x565122107500 0x56512207cb30 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f60072ec0 msgr2=0x7f7f600753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7f60072ec0 0x7f7f600753d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000b240 tx=0x7f7f70009000 comp rx=0 tx=0).stop 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220375f0 msgr2=0x56512207a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 --2- 172.21.15.19:0/4101696801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651220375f0 0x56512207a730 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00df70 tx=0x7f7f6c00b5b0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f80c00640 1 -- 172.21.15.19:0/4101696801 reap_dead start 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 shutdown_connections 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 >> 172.21.15.19:0/4101696801 conn(0x565121f59ac0 msgr2=0x56512210bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 shutdown_connections 2024-04-25T11:39:02.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.473+0000 7f7f86f97f40 1 -- 172.21.15.19:0/4101696801 wait complete. 2024-04-25T11:39:02.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.772+0000 7f955ea00640 1 Processor -- start 2024-04-25T11:39:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.772+0000 7f955ea00640 1 -- start start 2024-04-25T11:39:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9558162210 0x7f955806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f955806cb70 0x7f955815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955806d3a0 con 0x7f95581441d0 2024-04-25T11:39:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f955806d680 con 0x7f9558162210 2024-04-25T11:39:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f955806d960 con 0x7f955806cb70 2024-04-25T11:39:02.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9558162210 0x7f955806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9558162210 0x7f955806c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33514/0 (socket says 172.21.15.19:33514) 2024-04-25T11:39:02.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955cc00640 1 -- 172.21.15.19:0/2895646158 learned_addr learned my addr 172.21.15.19:0/2895646158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.775+0000 7f955e000640 1 --2- 172.21.15.19:0/2895646158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f955806cb70 0x7f955815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.776+0000 7f955cc00640 1 -- 172.21.15.19:0/2895646158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f955806cb70 msgr2=0x7f955815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.776+0000 7f955cc00640 1 --2- 172.21.15.19:0/2895646158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f955806cb70 0x7f955815fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.776+0000 7f955cc00640 1 -- 172.21.15.19:0/2895646158 >> v1:172.21.15.169:6789/0 conn(0x7f95581441d0 legacy=0x7f9558161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.776+0000 7f955cc00640 1 -- 172.21.15.19:0/2895646158 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955806dc40 con 0x7f9558162210 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.776+0000 7f955e000640 1 --2- 172.21.15.19:0/2895646158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f955806cb70 0x7f955815fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.777+0000 7f955cc00640 1 --2- 172.21.15.19:0/2895646158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9558162210 0x7f955806c630 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f954402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e1386c32fa3d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.777+0000 7f9557e00640 1 -- 172.21.15.19:0/2895646158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544037070 con 0x7f9558162210 2024-04-25T11:39:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.777+0000 7f9557e00640 1 -- 172.21.15.19:0/2895646158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9544032440 con 0x7f9558162210 2024-04-25T11:39:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.777+0000 7f9557e00640 1 -- 172.21.15.19:0/2895646158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544032810 con 0x7f9558162210 2024-04-25T11:39:02.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 -- 172.21.15.19:0/2895646158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9558162210 msgr2=0x7f955806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 --2- 172.21.15.19:0/2895646158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9558162210 0x7f955806c630 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f954402a180 comp rx=0 tx=0).stop 2024-04-25T11:39:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 -- 172.21.15.19:0/2895646158 shutdown_connections 2024-04-25T11:39:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 --2- 172.21.15.19:0/2895646158 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f955806cb70 0x7f955815fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 --2- 172.21.15.19:0/2895646158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9558162210 0x7f955806c630 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 -- 172.21.15.19:0/2895646158 >> 172.21.15.19:0/2895646158 conn(0x7f9558072f40 msgr2=0x7f9558073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 -- 172.21.15.19:0/2895646158 shutdown_connections 2024-04-25T11:39:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.778+0000 7f955ea00640 1 -- 172.21.15.19:0/2895646158 wait complete. 2024-04-25T11:39:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.781+0000 7f955ea00640 1 Processor -- start 2024-04-25T11:39:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.781+0000 7f955ea00640 1 -- start start 2024-04-25T11:39:02.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.782+0000 7f955ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955806cb70 0x7f95581558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.782+0000 7f955ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95581441d0 0x7f9558159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.782+0000 7f955ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9558159e90 0x7f955815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.782+0000 7f955ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f955806d100 con 0x7f955806cb70 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.782+0000 7f955ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f955806cf80 con 0x7f95581441d0 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.782+0000 7f955ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f955815ff80 con 0x7f9558159e90 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955806cb70 0x7f95581558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955806cb70 0x7f95581558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33526/0 (socket says 172.21.15.19:33526) 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 -- 172.21.15.19:0/1336479730 learned_addr learned my addr 172.21.15.19:0/1336479730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955cc00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95581441d0 0x7f9558159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955e000640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9558159e90 0x7f955815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 -- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9558159e90 msgr2=0x7f955815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9558159e90 0x7f955815a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 -- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95581441d0 msgr2=0x7f9558159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95581441d0 0x7f9558159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 -- 172.21.15.19:0/1336479730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955815aa00 con 0x7f955806cb70 2024-04-25T11:39:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.783+0000 7f955d600640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955806cb70 0x7f95581558c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b420 tx=0x7f954c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.784+0000 7f9555600640 1 -- 172.21.15.19:0/1336479730 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c011830 con 0x7f955806cb70 2024-04-25T11:39:02.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.784+0000 7f9555600640 1 -- 172.21.15.19:0/1336479730 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c016070 con 0x7f955806cb70 2024-04-25T11:39:02.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.784+0000 7f9555600640 1 -- 172.21.15.19:0/1336479730 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c011c10 con 0x7f955806cb70 2024-04-25T11:39:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.784+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955815ace0 con 0x7f955806cb70 2024-04-25T11:39:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.784+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955815b2a0 con 0x7f955806cb70 2024-04-25T11:39:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.785+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9558076c60 con 0x7f955806cb70 2024-04-25T11:39:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.787+0000 7f9555600640 1 -- 172.21.15.19:0/1336479730 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f954c01b460 con 0x7f955806cb70 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.788+0000 7f9555600640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95380744b0 0x7f95380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.788+0000 7f9555600640 1 -- 172.21.15.19:0/1336479730 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7687+0+0 (secure 0 0 0) 0x7f954c095550 con 0x7f955806cb70 2024-04-25T11:39:02.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.789+0000 7f955cc00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95380744b0 0x7f95380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:02.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.790+0000 7f955cc00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95380744b0 0x7f95380769c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9544034000 tx=0x7f954402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:02.793+0000 7f9555600640 1 -- 172.21.15.19:0/1336479730 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f954c020020 con 0x7f955806cb70 2024-04-25T11:39:03.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.040+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f95580707e0 con 0x7f955806cb70 2024-04-25T11:39:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.396+0000 7f9555600640 1 -- 172.21.15.19:0/1336479730 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v240) ==== 162+0+0 (secure 0 0 0) 0x7f954c05f510 con 0x7f955806cb70 2024-04-25T11:39:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95380744b0 msgr2=0x7f95380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95380744b0 0x7f95380769c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9544034000 tx=0x7f954402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955806cb70 msgr2=0x7f95581558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955806cb70 0x7f95581558c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b420 tx=0x7f954c00b8e0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 shutdown_connections 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95380744b0 0x7f95380769c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95581441d0 0x7f9558159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9558159e90 0x7f955815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 --2- 172.21.15.19:0/1336479730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f955806cb70 0x7f95581558c0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.400+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 >> 172.21.15.19:0/1336479730 conn(0x7f9558072f40 msgr2=0x7f9558141da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.401+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 shutdown_connections 2024-04-25T11:39:03.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.401+0000 7f955ea00640 1 -- 172.21.15.19:0/1336479730 wait complete. 2024-04-25T11:39:03.410 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:03.423+0000 7f1097a00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-25T11:39:03.410 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:03.423+0000 7f1097a00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:39:03.411 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:03.424+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:03.411 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:03.424+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:39:03.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:03.431 INFO:tasks.ceph.mds.a.smithi019.stderr: -22> 2024-04-25T11:39:03.423+0000 7f1097a00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-04-25T11:39:03.431 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:39:03.423+0000 7f1097a00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:39:03.572 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:03.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.707+0000 7f8afec00640 1 Processor -- start 2024-04-25T11:39:03.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.707+0000 7f8afec00640 1 -- start start 2024-04-25T11:39:03.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.709+0000 7f8afec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8af80d33d0 0x7f8af80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.709+0000 7f8afec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8af80d45e0 0x7f8af80d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.709+0000 7f8afec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af80d2330 con 0x7f8af806dc70 2024-04-25T11:39:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.709+0000 7f8afec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8af80d2610 con 0x7f8af80d33d0 2024-04-25T11:39:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.709+0000 7f8afec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8af80d28f0 con 0x7f8af80d45e0 2024-04-25T11:39:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8af80d33d0 0x7f8af80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8af80d33d0 0x7f8af80b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33560/0 (socket says 172.21.15.19:33560) 2024-04-25T11:39:03.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7e00640 1 -- 172.21.15.19:0/1400395065 learned_addr learned my addr 172.21.15.19:0/1400395065 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7400640 1 --2- 172.21.15.19:0/1400395065 >> v2:172.21.15.169:3300/0 conn(0x7f8af80d45e0 0x7f8af80d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7e00640 1 -- 172.21.15.19:0/1400395065 >> v2:172.21.15.169:3300/0 conn(0x7f8af80d45e0 msgr2=0x7f8af80d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7e00640 1 --2- 172.21.15.19:0/1400395065 >> v2:172.21.15.169:3300/0 conn(0x7f8af80d45e0 0x7f8af80d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7e00640 1 -- 172.21.15.19:0/1400395065 >> v1:172.21.15.169:6789/0 conn(0x7f8af806dc70 legacy=0x7f8af8164110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.710+0000 7f8af7e00640 1 -- 172.21.15.19:0/1400395065 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af80d2bd0 con 0x7f8af80d33d0 2024-04-25T11:39:03.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.711+0000 7f8af7e00640 1 --2- 172.21.15.19:0/1400395065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d33d0 0x7f8af80b4fb0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f8af8070da0 tx=0x7f8ae802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758c78d4dc71ad64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.712+0000 7f8af6a00640 1 -- 172.21.15.19:0/1400395065 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8036070 con 0x7f8af80d33d0 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.712+0000 7f8af6a00640 1 -- 172.21.15.19:0/1400395065 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae8031600 con 0x7f8af80d33d0 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.712+0000 7f8af6a00640 1 -- 172.21.15.19:0/1400395065 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80319d0 con 0x7f8af80d33d0 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 -- 172.21.15.19:0/1400395065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d33d0 msgr2=0x7f8af80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 --2- 172.21.15.19:0/1400395065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d33d0 0x7f8af80b4fb0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f8af8070da0 tx=0x7f8ae802a730 comp rx=0 tx=0).stop 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 -- 172.21.15.19:0/1400395065 shutdown_connections 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 --2- 172.21.15.19:0/1400395065 >> v2:172.21.15.169:3300/0 conn(0x7f8af80d45e0 0x7f8af80d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 --2- 172.21.15.19:0/1400395065 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d33d0 0x7f8af80b4fb0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 -- 172.21.15.19:0/1400395065 >> 172.21.15.19:0/1400395065 conn(0x7f8af80d3ac0 msgr2=0x7f8af80d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 -- 172.21.15.19:0/1400395065 shutdown_connections 2024-04-25T11:39:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.713+0000 7f8afec00640 1 -- 172.21.15.19:0/1400395065 wait complete. 2024-04-25T11:39:03.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.715+0000 7f8afec00640 1 Processor -- start 2024-04-25T11:39:03.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.715+0000 7f8afec00640 1 -- start start 2024-04-25T11:39:03.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af806dc70 0x7f8af809f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af80d33d0 0x7f8af80ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d45e0 0x7f8af81f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8af80d2040 con 0x7f8af80d45e0 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8af80d1ec0 con 0x7f8af80d33d0 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8af80d0a10 con 0x7f8af806dc70 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d45e0 0x7f8af81f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d45e0 0x7f8af81f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33572/0 (socket says 172.21.15.19:33572) 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.717+0000 7f8afcc00640 1 -- 172.21.15.19:0/1146872618 learned_addr learned my addr 172.21.15.19:0/1146872618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8af7400640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af80d33d0 0x7f8af80ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afcc00640 1 -- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af806dc70 msgr2=0x7f8af809f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afcc00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af806dc70 0x7f8af809f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afcc00640 1 -- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af80d33d0 msgr2=0x7f8af80ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afcc00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af80d33d0 0x7f8af80ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afcc00640 1 -- 172.21.15.19:0/1146872618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af81fa760 con 0x7f8af80d45e0 2024-04-25T11:39:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8af7400640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af80d33d0 0x7f8af80ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afcc00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d45e0 0x7f8af81f9b80 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b410 tx=0x7f8af000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8aefe00640 1 -- 172.21.15.19:0/1146872618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0015030 con 0x7f8af80d45e0 2024-04-25T11:39:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8aefe00640 1 -- 172.21.15.19:0/1146872618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af0004020 con 0x7f8af80d45e0 2024-04-25T11:39:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8aefe00640 1 -- 172.21.15.19:0/1146872618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0004410 con 0x7f8af80d45e0 2024-04-25T11:39:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af80b2770 con 0x7f8af80d45e0 2024-04-25T11:39:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.718+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af80cd520 con 0x7f8af80d45e0 2024-04-25T11:39:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.719+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af81fa4d0 con 0x7f8af80d45e0 2024-04-25T11:39:03.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.721+0000 7f8aefe00640 1 -- 172.21.15.19:0/1146872618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8af0018070 con 0x7f8af80d45e0 2024-04-25T11:39:03.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.722+0000 7f8aefe00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ad80744b0 0x7f8ad80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:03.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.722+0000 7f8aefe00640 1 -- 172.21.15.19:0/1146872618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7f8af0095610 con 0x7f8af80d45e0 2024-04-25T11:39:03.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.723+0000 7f8af7e00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ad80744b0 0x7f8ad80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:03.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.724+0000 7f8af7e00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ad80744b0 0x7f8ad80769c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8033000 tx=0x7f8ae802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:03.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.727+0000 7f8aefe00640 1 -- 172.21.15.19:0/1146872618 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8af005f4f0 con 0x7f8af80d45e0 2024-04-25T11:39:03.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.968+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8af80b2560 con 0x7f8af80d45e0 2024-04-25T11:39:03.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.969+0000 7f8aefe00640 1 -- 172.21.15.19:0/1146872618 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 242 v242) ==== 95+0+4352 (secure 0 0 0) 0x7f8af005edf0 con 0x7f8af80d45e0 2024-04-25T11:39:03.956 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:03.956 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":242,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":242,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:03.424882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:03.956 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 242 2024-04-25T11:39:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.971+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ad80744b0 msgr2=0x7f8ad80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.971+0000 7f8afec00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ad80744b0 0x7f8ad80769c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8033000 tx=0x7f8ae802e000 comp rx=0 tx=0).stop 2024-04-25T11:39:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.971+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d45e0 msgr2=0x7f8af81f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:03.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.971+0000 7f8afec00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d45e0 0x7f8af81f9b80 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b410 tx=0x7f8af000b8d0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 shutdown_connections 2024-04-25T11:39:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ad80744b0 0x7f8ad80769c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af80d33d0 0x7f8af80ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8af80d45e0 0x7f8af81f9b80 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 --2- 172.21.15.19:0/1146872618 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8af806dc70 0x7f8af809f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 >> 172.21.15.19:0/1146872618 conn(0x7f8af80d3ac0 msgr2=0x7f8af80b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 shutdown_connections 2024-04-25T11:39:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:03.972+0000 7f8afec00640 1 -- 172.21.15.19:0/1146872618 wait complete. 2024-04-25T11:39:03.984 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:03.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.274+0000 7fa621400640 1 Processor -- start 2024-04-25T11:39:04.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.274+0000 7fa621400640 1 -- start start 2024-04-25T11:39:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa621400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa61c0b4ea0 0x7fa61c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa621400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa61c144370 0x7fa61c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa621400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa61c0d1230 con 0x7fa61c06c5c0 2024-04-25T11:39:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa621400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa61c0d06b0 con 0x7fa61c0b4ea0 2024-04-25T11:39:04.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa621400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa61c0d0990 con 0x7fa61c144370 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa61be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa61c06c5c0 0x7fa61c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40828/0 (socket says 172.21.15.19:40828) 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa61be00640 1 -- 172.21.15.19:0/2688533779 learned_addr learned my addr 172.21.15.19:0/2688533779 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.276+0000 7fa61b400640 1 --2- 172.21.15.19:0/2688533779 >> v2:172.21.15.19:3300/0 conn(0x7fa61c0b4ea0 0x7fa61c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.277+0000 7fa620a00640 1 --2- 172.21.15.19:0/2688533779 >> v2:172.21.15.169:3300/0 conn(0x7fa61c144370 0x7fa61c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.277+0000 7fa61aa00640 1 -- 172.21.15.19:0/2688533779 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13132745 0 0) 0x7fa61c0d1230 con 0x7fa61c06c5c0 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.277+0000 7fa61b400640 1 -- 172.21.15.19:0/2688533779 >> v2:172.21.15.169:3300/0 conn(0x7fa61c144370 msgr2=0x7fa61c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.277+0000 7fa61b400640 1 --2- 172.21.15.19:0/2688533779 >> v2:172.21.15.169:3300/0 conn(0x7fa61c144370 0x7fa61c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.277+0000 7fa61b400640 1 -- 172.21.15.19:0/2688533779 >> v1:172.21.15.19:6789/0 conn(0x7fa61c06c5c0 legacy=0x7fa61c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.277+0000 7fa61b400640 1 -- 172.21.15.19:0/2688533779 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c0d0c70 con 0x7fa61c0b4ea0 2024-04-25T11:39:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.278+0000 7fa61b400640 1 --2- 172.21.15.19:0/2688533779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c0b4ea0 0x7fa61c0cfa70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0d26c0 tx=0x7fa60802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4ffddbd40859db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.278+0000 7fa61aa00640 1 -- 172.21.15.19:0/2688533779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608037070 con 0x7fa61c0b4ea0 2024-04-25T11:39:04.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.278+0000 7fa61aa00640 1 -- 172.21.15.19:0/2688533779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa608032440 con 0x7fa61c0b4ea0 2024-04-25T11:39:04.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.278+0000 7fa61aa00640 1 -- 172.21.15.19:0/2688533779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608032810 con 0x7fa61c0b4ea0 2024-04-25T11:39:04.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.279+0000 7fa621400640 1 -- 172.21.15.19:0/2688533779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c0b4ea0 msgr2=0x7fa61c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.279+0000 7fa621400640 1 --2- 172.21.15.19:0/2688533779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c0b4ea0 0x7fa61c0cfa70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0d26c0 tx=0x7fa60802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.279+0000 7fa621400640 1 -- 172.21.15.19:0/2688533779 shutdown_connections 2024-04-25T11:39:04.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.279+0000 7fa621400640 1 --2- 172.21.15.19:0/2688533779 >> v2:172.21.15.169:3300/0 conn(0x7fa61c144370 0x7fa61c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.279+0000 7fa621400640 1 --2- 172.21.15.19:0/2688533779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c0b4ea0 0x7fa61c0cfa70 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.279+0000 7fa621400640 1 -- 172.21.15.19:0/2688533779 >> 172.21.15.19:0/2688533779 conn(0x7fa61c0d1720 msgr2=0x7fa61c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.279+0000 7fa621400640 1 -- 172.21.15.19:0/2688533779 shutdown_connections 2024-04-25T11:39:04.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.280+0000 7fa621400640 1 -- 172.21.15.19:0/2688533779 wait complete. 2024-04-25T11:39:04.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.282+0000 7fa621400640 1 Processor -- start 2024-04-25T11:39:04.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.282+0000 7fa621400640 1 -- start start 2024-04-25T11:39:04.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.283+0000 7fa621400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61c06c5c0 0x7fa61c13f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa621400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c144370 0x7fa61c1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa621400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61c0cb200 0x7fa61c1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa621400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c0d0450 con 0x7fa61c144370 2024-04-25T11:39:04.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa621400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c0d02d0 con 0x7fa61c0cb200 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa621400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa61c147760 con 0x7fa61c06c5c0 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c144370 0x7fa61c1f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c144370 0x7fa61c1f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40928/0 (socket says 172.21.15.19:40928) 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 -- 172.21.15.19:0/4008750 learned_addr learned my addr 172.21.15.19:0/4008750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa620a00640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61c0cb200 0x7fa61c1f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61be00640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61c06c5c0 0x7fa61c13f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 -- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61c06c5c0 msgr2=0x7fa61c13f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61c06c5c0 0x7fa61c13f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 -- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61c0cb200 msgr2=0x7fa61c1f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61c0cb200 0x7fa61c1f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61b400640 1 -- 172.21.15.19:0/4008750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c15f030 con 0x7fa61c144370 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa620a00640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61c0cb200 0x7fa61c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:04.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.284+0000 7fa61be00640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61c06c5c0 0x7fa61c13f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.285+0000 7fa61b400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c144370 0x7fa61c1f5d40 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fa608035040 tx=0x7fa6080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.285+0000 7fa613e00640 1 -- 172.21.15.19:0/4008750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608041070 con 0x7fa61c144370 2024-04-25T11:39:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.285+0000 7fa613e00640 1 -- 172.21.15.19:0/4008750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa608004250 con 0x7fa61c144370 2024-04-25T11:39:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.285+0000 7fa613e00640 1 -- 172.21.15.19:0/4008750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608037040 con 0x7fa61c144370 2024-04-25T11:39:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.285+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c161b10 con 0x7fa61c144370 2024-04-25T11:39:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.285+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1620d0 con 0x7fa61c144370 2024-04-25T11:39:04.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.287+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c050740 con 0x7fa61c144370 2024-04-25T11:39:04.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.288+0000 7fa613e00640 1 -- 172.21.15.19:0/4008750 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa608030770 con 0x7fa61c144370 2024-04-25T11:39:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.289+0000 7fa613e00640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5fc0744e0 0x7fa5fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.289+0000 7fa613e00640 1 -- 172.21.15.19:0/4008750 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7fa6080b6220 con 0x7fa61c144370 2024-04-25T11:39:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.290+0000 7fa61be00640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5fc0744e0 0x7fa5fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.291+0000 7fa61be00640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5fc0744e0 0x7fa5fc0769f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa60c000f60 tx=0x7fa60c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.294+0000 7fa613e00640 1 -- 172.21.15.19:0/4008750 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa608080190 con 0x7fa61c144370 2024-04-25T11:39:04.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.541+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa61c1e8420 con 0x7fa61c144370 2024-04-25T11:39:04.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.542+0000 7fa613e00640 1 -- 172.21.15.19:0/4008750 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 243 v243) ==== 95+0+5178 (secure 0 0 0) 0x7fa608088180 con 0x7fa61c144370 2024-04-25T11:39:04.530 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:04.530 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":243,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1200575562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1200575562},{"type":"v1","addr":"172.21.15.19:6835","nonce":1200575562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":243}],"filesystems":[{"mdsmap":{"epoch":242,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:03.424882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 243 2024-04-25T11:39:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5fc0744e0 msgr2=0x7fa5fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5fc0744e0 0x7fa5fc0769f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa60c000f60 tx=0x7fa60c009040 comp rx=0 tx=0).stop 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c144370 msgr2=0x7fa61c1f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c144370 0x7fa61c1f5d40 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fa608035040 tx=0x7fa6080046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 shutdown_connections 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5fc0744e0 0x7fa5fc0769f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa61c0cb200 0x7fa61c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa61c144370 0x7fa61c1f5d40 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 --2- 172.21.15.19:0/4008750 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa61c06c5c0 0x7fa61c13f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.545+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 >> 172.21.15.19:0/4008750 conn(0x7fa61c0d1720 msgr2=0x7fa61c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.546+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 shutdown_connections 2024-04-25T11:39:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.546+0000 7fa621400640 1 -- 172.21.15.19:0/4008750 wait complete. 2024-04-25T11:39:04.559 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:04.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.854+0000 7f5a6b200640 1 Processor -- start 2024-04-25T11:39:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.854+0000 7f5a6b200640 1 -- start start 2024-04-25T11:39:04.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.857+0000 7f5a6b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a6c0b6e10 0x7f5a6c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.857+0000 7f5a6b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5a6c1608d0 0x7f5a6c160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.857+0000 7f5a6b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6c161220 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.858+0000 7f5a6b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6c06f870 con 0x7f5a6c1608d0 2024-04-25T11:39:04.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.858+0000 7f5a6b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6c0b7220 con 0x7f5a6c0b6e10 2024-04-25T11:39:04.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.858+0000 7f5a6a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5a6c1608d0 0x7f5a6c160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.858+0000 7f5a69e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5a6c0cf2f0 0x7f5a6c0b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40840/0 (socket says 172.21.15.19:40840) 2024-04-25T11:39:04.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.858+0000 7f5a69e00640 1 -- 172.21.15.19:0/270045811 learned_addr learned my addr 172.21.15.19:0/270045811 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.858+0000 7f5a69400640 1 --2- 172.21.15.19:0/270045811 >> v2:172.21.15.169:3300/0 conn(0x7f5a6c0b6e10 0x7f5a6c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.859+0000 7f5a6a800640 1 -- 172.21.15.19:0/270045811 >> v2:172.21.15.169:3300/0 conn(0x7f5a6c0b6e10 msgr2=0x7f5a6c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.859+0000 7f5a6a800640 1 --2- 172.21.15.19:0/270045811 >> v2:172.21.15.169:3300/0 conn(0x7f5a6c0b6e10 0x7f5a6c15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.859+0000 7f5a6a800640 1 -- 172.21.15.19:0/270045811 >> v1:172.21.15.19:6789/0 conn(0x7f5a6c0cf2f0 legacy=0x7f5a6c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.859+0000 7f5a6a800640 1 -- 172.21.15.19:0/270045811 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c0b74d0 con 0x7f5a6c1608d0 2024-04-25T11:39:04.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.860+0000 7f5a6a800640 1 --2- 172.21.15.19:0/270045811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c1608d0 0x7f5a6c160ce0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c070dd0 tx=0x7f5a6402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f92bf61b86d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.860+0000 7f5a68a00640 1 -- 172.21.15.19:0/270045811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64007740 con 0x7f5a6c1608d0 2024-04-25T11:39:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.860+0000 7f5a68a00640 1 -- 172.21.15.19:0/270045811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a640078e0 con 0x7f5a6c1608d0 2024-04-25T11:39:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.860+0000 7f5a68a00640 1 -- 172.21.15.19:0/270045811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64007cb0 con 0x7f5a6c1608d0 2024-04-25T11:39:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 -- 172.21.15.19:0/270045811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c1608d0 msgr2=0x7f5a6c160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 --2- 172.21.15.19:0/270045811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c1608d0 0x7f5a6c160ce0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c070dd0 tx=0x7f5a6402a680 comp rx=0 tx=0).stop 2024-04-25T11:39:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 -- 172.21.15.19:0/270045811 shutdown_connections 2024-04-25T11:39:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 --2- 172.21.15.19:0/270045811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c1608d0 0x7f5a6c160ce0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 --2- 172.21.15.19:0/270045811 >> v2:172.21.15.169:3300/0 conn(0x7f5a6c0b6e10 0x7f5a6c15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 -- 172.21.15.19:0/270045811 >> 172.21.15.19:0/270045811 conn(0x7f5a6c0d1720 msgr2=0x7f5a6c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 -- 172.21.15.19:0/270045811 shutdown_connections 2024-04-25T11:39:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.861+0000 7f5a6b200640 1 -- 172.21.15.19:0/270045811 wait complete. 2024-04-25T11:39:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.864+0000 7f5a6b200640 1 Processor -- start 2024-04-25T11:39:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.864+0000 7f5a6b200640 1 -- start start 2024-04-25T11:39:04.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.865+0000 7f5a6b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a6c0b6e10 0x7f5a6c147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.865+0000 7f5a6b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c0cf2f0 0x7f5a6c072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.865+0000 7f5a6b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a6c1608d0 0x7f5a6c0734e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a6b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c06f500 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a6b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c06f220 con 0x7f5a6c1608d0 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a6b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c06fd00 con 0x7f5a6c0b6e10 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c0cf2f0 0x7f5a6c072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c0cf2f0 0x7f5a6c072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40936/0 (socket says 172.21.15.19:40936) 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 -- 172.21.15.19:0/1109955714 learned_addr learned my addr 172.21.15.19:0/1109955714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69e00640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a6c0b6e10 0x7f5a6c147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a6a800640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a6c1608d0 0x7f5a6c0734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 -- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a6c0b6e10 msgr2=0x7f5a6c147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a6c0b6e10 0x7f5a6c147180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 -- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a6c1608d0 msgr2=0x7f5a6c0734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a6c1608d0 0x7f5a6c0734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69400640 1 -- 172.21.15.19:0/1109955714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c14b300 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.866+0000 7f5a69e00640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a6c0b6e10 0x7f5a6c147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.867+0000 7f5a69400640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c0cf2f0 0x7f5a6c072fa0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800d620 tx=0x7f5a5800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.867+0000 7f5a62000640 1 -- 172.21.15.19:0/1109955714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5800b620 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.867+0000 7f5a62000640 1 -- 172.21.15.19:0/1109955714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a58019070 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.867+0000 7f5a62000640 1 -- 172.21.15.19:0/1109955714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5800ba00 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.867+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c14b800 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.867+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c1489e0 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.868+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c050740 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.870+0000 7f5a62000640 1 -- 172.21.15.19:0/1109955714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a5800bba0 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.871+0000 7f5a62000640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a4c0744b0 0x7f5a4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.871+0000 7f5a62000640 1 -- 172.21.15.19:0/1109955714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7f5a58095df0 con 0x7f5a6c0cf2f0 2024-04-25T11:39:04.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.871+0000 7f5a69e00640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a4c0744b0 0x7f5a4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:04.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.873+0000 7f5a69e00640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a4c0744b0 0x7f5a4c0769c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c000f60 tx=0x7f5a5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:04.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:04.876+0000 7f5a62000640 1 -- 172.21.15.19:0/1109955714 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a5805fdf0 con 0x7f5a6c0cf2f0 2024-04-25T11:39:05.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.120+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a6c14b590 con 0x7f5a6c0cf2f0 2024-04-25T11:39:05.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.121+0000 7f5a62000640 1 -- 172.21.15.19:0/1109955714 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 243 v243) ==== 95+0+5178 (secure 0 0 0) 0x7f5a5805f6f0 con 0x7f5a6c0cf2f0 2024-04-25T11:39:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:05.108 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":243,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1200575562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1200575562},{"type":"v1","addr":"172.21.15.19:6835","nonce":1200575562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":243}],"filesystems":[{"mdsmap":{"epoch":242,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:03.424882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:05.108 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 243 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a4c0744b0 msgr2=0x7f5a4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a4c0744b0 0x7f5a4c0769c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c000f60 tx=0x7f5a5c009040 comp rx=0 tx=0).stop 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c0cf2f0 msgr2=0x7f5a6c072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c0cf2f0 0x7f5a6c072fa0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800d620 tx=0x7f5a5800dae0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 shutdown_connections 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5a4c0744b0 0x7f5a4c0769c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a6c1608d0 0x7f5a6c0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c0cf2f0 0x7f5a6c072fa0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 --2- 172.21.15.19:0/1109955714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a6c0b6e10 0x7f5a6c147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.123+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 >> 172.21.15.19:0/1109955714 conn(0x7f5a6c0d1720 msgr2=0x7f5a6c0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.124+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 shutdown_connections 2024-04-25T11:39:05.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.124+0000 7f5a6b200640 1 -- 172.21.15.19:0/1109955714 wait complete. 2024-04-25T11:39:05.138 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2024-04-25T11:39:05.138 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2024-04-25T11:39:05.138 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:05.138 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:05.191 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:39:05.191 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:05.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:05.191 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:05.286 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:39:05.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:39:05.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.598+0000 7f7e16a00640 1 Processor -- start 2024-04-25T11:39:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.599+0000 7f7e16a00640 1 -- start start 2024-04-25T11:39:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.601+0000 7f7e16a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e100b4ea0 0x7f7e100cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.601+0000 7f7e16a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e100d1230 con 0x7f7e1006c5c0 2024-04-25T11:39:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.601+0000 7f7e16a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e100d06b0 con 0x7f7e10144370 2024-04-25T11:39:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.601+0000 7f7e16a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7e100d0990 con 0x7f7e100b4ea0 2024-04-25T11:39:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.601+0000 7f7e15600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7e1006c5c0 0x7f7e100b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40852/0 (socket says 172.21.15.19:40852) 2024-04-25T11:39:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.602+0000 7f7e15600640 1 -- 172.21.15.19:0/2244449053 learned_addr learned my addr 172.21.15.19:0/2244449053 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.602+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667297788 0 0) 0x7f7e100d1230 con 0x7f7e1006c5c0 2024-04-25T11:39:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.602+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7df8003520 con 0x7f7e1006c5c0 2024-04-25T11:39:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.602+0000 7f7e14c00640 1 --2- 172.21.15.19:0/2244449053 >> v2:172.21.15.169:3300/0 conn(0x7f7e100b4ea0 0x7f7e100cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1160802233 0 0) 0x7f7df8003520 con 0x7f7e1006c5c0 2024-04-25T11:39:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 >> v2:172.21.15.169:3300/0 conn(0x7f7e100b4ea0 msgr2=0x7f7e100cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 --2- 172.21.15.19:0/2244449053 >> v2:172.21.15.169:3300/0 conn(0x7f7e100b4ea0 0x7f7e100cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 >> v1:172.21.15.169:6789/0 conn(0x7f7e10144370 legacy=0x7f7e1015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e100d0c70 con 0x7f7e1006c5c0 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7e040019f0 con 0x7f7e1006c5c0 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 >> v1:172.21.15.19:6789/0 conn(0x7f7e1006c5c0 legacy=0x7f7e100b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df80039b0 0x7f7df8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df8004800 0x7f7df8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df80051c0 0x7f7df800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100d0990 con 0x7f7df80039b0 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7df800c160 con 0x7f7df8004800 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7df800c360 con 0x7f7df80051c0 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.603+0000 7f7e15600640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df80039b0 0x7f7df8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e14c00640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df8004800 0x7f7df8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e15600640 1 -- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df80051c0 msgr2=0x7f7df800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e16000640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df80051c0 0x7f7df800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e15600640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df80051c0 0x7f7df800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e15600640 1 -- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df8004800 msgr2=0x7f7df8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:05.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e15600640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df8004800 0x7f7df8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e15600640 1 -- 172.21.15.19:0/2244449053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df8003520 con 0x7f7df80039b0 2024-04-25T11:39:05.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e14c00640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df8004800 0x7f7df8004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:05.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e16000640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7df80051c0 0x7f7df800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:05.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.604+0000 7f7e16000640 1 -- 172.21.15.19:0/2244449053 reap_dead start 2024-04-25T11:39:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.605+0000 7f7e15600640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df80039b0 0x7f7df8003e20 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400a860 tx=0x7f7df8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f8595fb9864ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.605+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0400a480 con 0x7f7df80039b0 2024-04-25T11:39:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.605+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0402f070 con 0x7f7df80039b0 2024-04-25T11:39:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.605+0000 7f7e0fe00640 1 -- 172.21.15.19:0/2244449053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0402bc60 con 0x7f7df80039b0 2024-04-25T11:39:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.606+0000 7f7e16a00640 1 -- 172.21.15.19:0/2244449053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df80039b0 msgr2=0x7f7df8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.606+0000 7f7e16a00640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df80039b0 0x7f7df8003e20 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400a860 tx=0x7f7df8005830 comp rx=0 tx=0).stop 2024-04-25T11:39:05.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.606+0000 7f7e16a00640 1 -- 172.21.15.19:0/2244449053 shutdown_connections 2024-04-25T11:39:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.606+0000 7f7e16a00640 1 --2- 172.21.15.19:0/2244449053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7df80039b0 0x7f7df8003e20 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.606+0000 7f7e16a00640 1 -- 172.21.15.19:0/2244449053 >> 172.21.15.19:0/2244449053 conn(0x7f7e100d1720 msgr2=0x7f7e100d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.606+0000 7f7e16a00640 1 -- 172.21.15.19:0/2244449053 shutdown_connections 2024-04-25T11:39:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.606+0000 7f7e16a00640 1 -- 172.21.15.19:0/2244449053 wait complete. 2024-04-25T11:39:05.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.609+0000 7f7e16a00640 1 Processor -- start 2024-04-25T11:39:05.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.609+0000 7f7e16a00640 1 -- start start 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.610+0000 7f7e16a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df80039b0 0x7f7e100cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.610+0000 7f7e16a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e1015d750 0x7f7e1015cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.610+0000 7f7e16a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e101e8a90 0x7f7e101e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.610+0000 7f7e16a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100d0450 con 0x7f7e1015d750 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.610+0000 7f7e16a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100d02d0 con 0x7f7df80039b0 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.610+0000 7f7e16a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e10147760 con 0x7f7e101e8a90 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.611+0000 7f7e14c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e1015d750 0x7f7e1015cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.611+0000 7f7e14c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e1015d750 0x7f7e1015cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40960/0 (socket says 172.21.15.19:40960) 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.611+0000 7f7e14c00640 1 -- 172.21.15.19:0/1047243636 learned_addr learned my addr 172.21.15.19:0/1047243636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.611+0000 7f7e16000640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e101e8a90 0x7f7e101e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.611+0000 7f7e15600640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df80039b0 0x7f7e100cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.611+0000 7f7e14c00640 1 -- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e101e8a90 msgr2=0x7f7e101e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.611+0000 7f7e14c00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e101e8a90 0x7f7e101e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e14c00640 1 -- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df80039b0 msgr2=0x7f7e100cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e14c00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df80039b0 0x7f7e100cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e14c00640 1 -- 172.21.15.19:0/1047243636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e101e79f0 con 0x7f7e1015d750 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e14c00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e1015d750 0x7f7e1015cf10 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc010a00 tx=0x7f7dfc010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e0d600640 1 -- 172.21.15.19:0/1047243636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00f040 con 0x7f7e1015d750 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e0d600640 1 -- 172.21.15.19:0/1047243636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dfc00ec20 con 0x7f7e1015d750 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e101e7cd0 con 0x7f7e1015d750 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.612+0000 7f7e0d600640 1 -- 172.21.15.19:0/1047243636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc01c400 con 0x7f7e1015d750 2024-04-25T11:39:05.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.613+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e101e8290 con 0x7f7e1015d750 2024-04-25T11:39:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.614+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e100d1bd0 con 0x7f7e1015d750 2024-04-25T11:39:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.615+0000 7f7e0d600640 1 -- 172.21.15.19:0/1047243636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7dfc01c5a0 con 0x7f7e1015d750 2024-04-25T11:39:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.616+0000 7f7e0d600640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7df00744e0 0x7f7df00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.616+0000 7f7e0d600640 1 -- 172.21.15.19:0/1047243636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7f7dfc096330 con 0x7f7e1015d750 2024-04-25T11:39:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.617+0000 7f7e15600640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7df00744e0 0x7f7df00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:05.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.618+0000 7f7e15600640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7df00744e0 0x7f7df00769f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7e04033000 tx=0x7f7e04009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.623+0000 7f7e0d600640 1 -- 172.21.15.19:0/1047243636 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7dfc018030 con 0x7f7e1015d750 2024-04-25T11:39:05.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:05.872+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7e100cfdf0 con 0x7f7e1015d750 2024-04-25T11:39:06.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:06.013 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:06.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.457+0000 7f7e0d600640 1 -- 172.21.15.19:0/1047243636 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v244) ==== 161+0+0 (secure 0 0 0) 0x7f7dfc060210 con 0x7f7e1015d750 2024-04-25T11:39:06.445 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:06.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7df00744e0 msgr2=0x7f7df00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:06.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7df00744e0 0x7f7df00769f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7e04033000 tx=0x7f7e04009030 comp rx=0 tx=0).stop 2024-04-25T11:39:06.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e1015d750 msgr2=0x7f7e1015cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:06.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e1015d750 0x7f7e1015cf10 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc010a00 tx=0x7f7dfc010ec0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 shutdown_connections 2024-04-25T11:39:06.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7df00744e0 0x7f7df00769f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e101e8a90 0x7f7e101e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e1015d750 0x7f7e1015cf10 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 --2- 172.21.15.19:0/1047243636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7df80039b0 0x7f7e100cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 >> 172.21.15.19:0/1047243636 conn(0x7f7e100d1720 msgr2=0x7f7e100b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:06.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.462+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 shutdown_connections 2024-04-25T11:39:06.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.463+0000 7f7e16a00640 1 -- 172.21.15.19:0/1047243636 wait complete. 2024-04-25T11:39:06.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.773+0000 7ff9d9800640 1 Processor -- start 2024-04-25T11:39:06.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.773+0000 7ff9d9800640 1 -- start start 2024-04-25T11:39:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.776+0000 7ff9d9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d40d0e00 0x7ff9d40d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:06.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.776+0000 7ff9d9800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9d406c5c0 0x7ff9d406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.776+0000 7ff9d9800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9d406cf10 con 0x7ff9d40b7810 2024-04-25T11:39:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.776+0000 7ff9d9800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d40d5a60 con 0x7ff9d406c5c0 2024-04-25T11:39:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.776+0000 7ff9d9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d40d1850 con 0x7ff9d40d0e00 2024-04-25T11:39:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.776+0000 7ff9ca000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9d406c5c0 0x7ff9d406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.777+0000 7ff9ca000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9d406c5c0 0x7ff9d406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40962/0 (socket says 172.21.15.19:40962) 2024-04-25T11:39:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.777+0000 7ff9ca000640 1 -- 172.21.15.19:0/3923533856 learned_addr learned my addr 172.21.15.19:0/3923533856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:06.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.777+0000 7ff9d2a00640 1 --2- 172.21.15.19:0/3923533856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d40d0e00 0x7ff9d40d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:06.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.777+0000 7ff9ca000640 1 -- 172.21.15.19:0/3923533856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d40d0e00 msgr2=0x7ff9d40d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:06.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.777+0000 7ff9ca000640 1 --2- 172.21.15.19:0/3923533856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d40d0e00 0x7ff9d40d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.777+0000 7ff9ca000640 1 -- 172.21.15.19:0/3923533856 >> v1:172.21.15.169:6789/0 conn(0x7ff9d40b7810 legacy=0x7ff9d40b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:06.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.777+0000 7ff9ca000640 1 -- 172.21.15.19:0/3923533856 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d4160de0 con 0x7ff9d406c5c0 2024-04-25T11:39:06.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.779+0000 7ff9ca000640 1 --2- 172.21.15.19:0/3923533856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d406c5c0 0x7ff9d406c9d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009870 tx=0x7ff9bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af982a8b1fc42e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:06.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.779+0000 7ff9d2000640 1 -- 172.21.15.19:0/3923533856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc037070 con 0x7ff9d406c5c0 2024-04-25T11:39:06.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.779+0000 7ff9d2000640 1 -- 172.21.15.19:0/3923533856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc032440 con 0x7ff9d406c5c0 2024-04-25T11:39:06.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.779+0000 7ff9d2000640 1 -- 172.21.15.19:0/3923533856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc032810 con 0x7ff9d406c5c0 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 -- 172.21.15.19:0/3923533856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d406c5c0 msgr2=0x7ff9d406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 --2- 172.21.15.19:0/3923533856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d406c5c0 0x7ff9d406c9d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009870 tx=0x7ff9bc02a270 comp rx=0 tx=0).stop 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 -- 172.21.15.19:0/3923533856 shutdown_connections 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 --2- 172.21.15.19:0/3923533856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d406c5c0 0x7ff9d406c9d0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 --2- 172.21.15.19:0/3923533856 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d40d0e00 0x7ff9d40d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 -- 172.21.15.19:0/3923533856 >> 172.21.15.19:0/3923533856 conn(0x7ff9d40d5d90 msgr2=0x7ff9d40d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 -- 172.21.15.19:0/3923533856 shutdown_connections 2024-04-25T11:39:06.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.780+0000 7ff9d9800640 1 -- 172.21.15.19:0/3923533856 wait complete. 2024-04-25T11:39:06.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.782+0000 7ff9d9800640 1 Processor -- start 2024-04-25T11:39:06.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.783+0000 7ff9d9800640 1 -- start start 2024-04-25T11:39:06.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.784+0000 7ff9d9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d406c5c0 0x7ff9d40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:06.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.784+0000 7ff9d9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d40b7810 0x7ff9d41f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:06.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.784+0000 7ff9d9800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d40d0e00 0x7ff9d41f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:06.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.784+0000 7ff9d9800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4160ad0 con 0x7ff9d40b7810 2024-04-25T11:39:06.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.784+0000 7ff9d9800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d41613a0 con 0x7ff9d40d0e00 2024-04-25T11:39:06.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.784+0000 7ff9d9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d40b5320 con 0x7ff9d406c5c0 2024-04-25T11:39:06.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.784+0000 7ff9ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d40b7810 0x7ff9d41f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:06.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d40b7810 0x7ff9d41f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40964/0 (socket says 172.21.15.19:40964) 2024-04-25T11:39:06.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 -- 172.21.15.19:0/2398232105 learned_addr learned my addr 172.21.15.19:0/2398232105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:06.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9d2a00640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d406c5c0 0x7ff9d40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:06.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 -- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d406c5c0 msgr2=0x7ff9d40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:06.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d406c5c0 0x7ff9d40ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9d3400640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d40d0e00 0x7ff9d41f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:06.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 -- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d40d0e00 msgr2=0x7ff9d41f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:06.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d40d0e00 0x7ff9d41f9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 -- 172.21.15.19:0/2398232105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d40cf1a0 con 0x7ff9d40b7810 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.785+0000 7ff9ca000640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d40b7810 0x7ff9d41f3e30 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc000c00 tx=0x7ff9bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.786+0000 7ff9cb400640 1 -- 172.21.15.19:0/2398232105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc041020 con 0x7ff9d40b7810 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.786+0000 7ff9cb400640 1 -- 172.21.15.19:0/2398232105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc0306c0 con 0x7ff9d40b7810 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.786+0000 7ff9cb400640 1 -- 172.21.15.19:0/2398232105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc037040 con 0x7ff9d40b7810 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.786+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d40cf480 con 0x7ff9d40b7810 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.786+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d41ea360 con 0x7ff9d40b7810 2024-04-25T11:39:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.787+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d40cef10 con 0x7ff9d40b7810 2024-04-25T11:39:06.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.789+0000 7ff9cb400640 1 -- 172.21.15.19:0/2398232105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9bc044020 con 0x7ff9d40b7810 2024-04-25T11:39:06.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.790+0000 7ff9cb400640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9b00744b0 0x7ff9b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:06.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.790+0000 7ff9cb400640 1 -- 172.21.15.19:0/2398232105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7ff9bc0b66e0 con 0x7ff9d40b7810 2024-04-25T11:39:06.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.790+0000 7ff9d2a00640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9b00744b0 0x7ff9b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:06.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.792+0000 7ff9d2a00640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9b00744b0 0x7ff9b00769c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009e90 tx=0x7ff9c400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:06.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:06.793+0000 7ff9cb400640 1 -- 172.21.15.19:0/2398232105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9bc04b030 con 0x7ff9d40b7810 2024-04-25T11:39:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.077+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff9d4050740 con 0x7ff9d40b7810 2024-04-25T11:39:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.481+0000 7ff9cb400640 1 -- 172.21.15.19:0/2398232105 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v245) ==== 101+0+0 (secure 0 0 0) 0x7ff9bc0806d0 con 0x7ff9d40b7810 2024-04-25T11:39:07.468 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:39:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9b00744b0 msgr2=0x7ff9b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9b00744b0 0x7ff9b00769c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009e90 tx=0x7ff9c400b040 comp rx=0 tx=0).stop 2024-04-25T11:39:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d40b7810 msgr2=0x7ff9d41f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d40b7810 0x7ff9d41f3e30 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc000c00 tx=0x7ff9bc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 shutdown_connections 2024-04-25T11:39:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9b00744b0 0x7ff9b00769c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d40d0e00 0x7ff9d41f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d40b7810 0x7ff9d41f3e30 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 --2- 172.21.15.19:0/2398232105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d406c5c0 0x7ff9d40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 >> 172.21.15.19:0/2398232105 conn(0x7ff9d40d5d90 msgr2=0x7ff9d4163960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.485+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 shutdown_connections 2024-04-25T11:39:07.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.486+0000 7ff9d9800640 1 -- 172.21.15.19:0/2398232105 wait complete. 2024-04-25T11:39:07.502 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.701+0000 7f9c3125cf40 1 Processor -- start 2024-04-25T11:39:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.701+0000 7f9c3125cf40 1 -- start start 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c3125cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 0x55d3f1178340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c3125cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f1182150 0x55d3f10dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c3125cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f10dc770 0x55d3f10dd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c3125cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d3f1052d20 con 0x55d3f1182150 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c3125cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d3f1056d60 con 0x7f9c100030f0 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c3125cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d3f10f5110 con 0x55d3f10dc770 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c29a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f1182150 0x55d3f10dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c29a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f1182150 0x55d3f10dc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40986/0 (socket says 172.21.15.19:40986) 2024-04-25T11:39:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c29a00640 1 -- 172.21.15.19:0/2439044746 learned_addr learned my addr 172.21.15.19:0/2439044746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c2ae00640 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f10dc770 0x55d3f10dd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c29a00640 1 -- 172.21.15.19:0/2439044746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f10dc770 msgr2=0x55d3f10dd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.703+0000 7f9c29a00640 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f10dc770 0x55d3f10dd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.704+0000 7f9c29a00640 1 -- 172.21.15.19:0/2439044746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 msgr2=0x55d3f1178340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.704+0000 7f9c29a00640 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 0x55d3f1178340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.704+0000 7f9c29a00640 1 -- 172.21.15.19:0/2439044746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f10dcb80 con 0x55d3f1182150 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.704+0000 7f9c29a00640 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f1182150 0x55d3f10dc230 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9c14025830 tx=0x7f9c1404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42423b268b6b1731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.704+0000 7f9c23e00640 1 -- 172.21.15.19:0/2439044746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14063070 con 0x55d3f1182150 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.704+0000 7f9c23e00640 1 -- 172.21.15.19:0/2439044746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c140549e0 con 0x55d3f1182150 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.704+0000 7f9c23e00640 1 -- 172.21.15.19:0/2439044746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14059040 con 0x55d3f1182150 2024-04-25T11:39:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 -- 172.21.15.19:0/2439044746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f1182150 msgr2=0x55d3f10dc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f1182150 0x55d3f10dc230 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f9c14025830 tx=0x7f9c1404e000 comp rx=0 tx=0).stop 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 -- 172.21.15.19:0/2439044746 shutdown_connections 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f10dc770 0x55d3f10dd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f1182150 0x55d3f10dc230 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/2439044746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 0x55d3f1178340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 -- 172.21.15.19:0/2439044746 >> 172.21.15.19:0/2439044746 conn(0x55d3f0fceac0 msgr2=0x55d3f10da730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 -- 172.21.15.19:0/2439044746 shutdown_connections 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.705+0000 7f9c3125cf40 1 -- 172.21.15.19:0/2439044746 wait complete. 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.706+0000 7f9c3125cf40 1 Processor -- start 2024-04-25T11:39:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.706+0000 7f9c3125cf40 1 -- start start 2024-04-25T11:39:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c3125cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f10dc770 0x55d3f11771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c3125cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f1182150 0x55d3f11776f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c3125cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 0x55d3f12115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c3125cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d3f10f91f0 con 0x55d3f10dc770 2024-04-25T11:39:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c3125cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d3f117b0e0 con 0x7f9c100030f0 2024-04-25T11:39:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c3125cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d3f116e9c0 con 0x55d3f1182150 2024-04-25T11:39:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c2a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f10dc770 0x55d3f11771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c2a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f10dc770 0x55d3f11771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40988/0 (socket says 172.21.15.19:40988) 2024-04-25T11:39:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.708+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 learned_addr learned my addr 172.21.15.19:0/1274273340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f1182150 msgr2=0x55d3f11776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c2a400640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d3f1182150 0x55d3f11776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c2ae00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 0x55d3f12115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 msgr2=0x55d3f12115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c2a400640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c100030f0 0x55d3f12115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f117de00 con 0x55d3f10dc770 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c2a400640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f10dc770 0x55d3f11771b0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d470 tx=0x7f9c1c00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.709+0000 7f9c22a00640 1 -- 172.21.15.19:0/1274273340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c018070 con 0x55d3f10dc770 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.710+0000 7f9c22a00640 1 -- 172.21.15.19:0/1274273340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1c00f510 con 0x55d3f10dc770 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.710+0000 7f9c22a00640 1 -- 172.21.15.19:0/1274273340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00f8e0 con 0x55d3f10dc770 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.710+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3f1213860 con 0x55d3f10dc770 2024-04-25T11:39:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.710+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f1214340 con 0x55d3f10dc770 2024-04-25T11:39:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.712+0000 7f9c22a00640 1 -- 172.21.15.19:0/1274273340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c1c01b070 con 0x55d3f10dc770 2024-04-25T11:39:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.713+0000 7f9c22a00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c08072ec0 0x7f9c080753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.713+0000 7f9c22a00640 1 -- 172.21.15.19:0/1274273340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7f9c1c093e90 con 0x55d3f10dc770 2024-04-25T11:39:07.700 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.714+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d3f11825d0 0x55d3f117bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.714+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f117c690 con 0x55d3f11825d0 2024-04-25T11:39:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.714+0000 7f9c29a00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c08072ec0 0x7f9c080753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.714+0000 7f9c2ae00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d3f11825d0 0x55d3f117bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.715+0000 7f9c29a00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c08072ec0 0x7f9c080753d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9c14025830 tx=0x7f9c14001730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:07.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.716+0000 7f9c2ae00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d3f11825d0 0x55d3f117bb00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:07.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.720+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v116'722 uv696 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.720+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f117ca60 con 0x55d3f11825d0 2024-04-25T11:39:07.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.722+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v116'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.722+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f117ce30 con 0x55d3f11825d0 2024-04-25T11:39:07.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.725+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v116'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.712 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:07.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.726+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f117d220 con 0x55d3f11825d0 2024-04-25T11:39:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.729+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.729+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1228890 con 0x55d3f11825d0 2024-04-25T11:39:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.732+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'726 uv726 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.732+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227880 con 0x55d3f11825d0 2024-04-25T11:39:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.736+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'727 uv727 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.736+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227c20 con 0x55d3f11825d0 2024-04-25T11:39:07.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.740+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v116'728 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.740+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.744+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v116'729 uv729 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.744+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.734 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:07.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.748+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v116'730 uv730 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.748+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.752+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'731 uv731 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.752+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.755+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.755+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.760+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.760+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.764+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v116'734 uv709 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.765+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.769+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'735 uv735 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.769+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.773+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v116'736 uv736 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.760 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:07.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.773+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.778+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'737 uv737 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.778+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.782+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'738 uv738 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.782+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.787+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'739 uv739 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.787+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.792+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'740 uv740 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.792+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.797+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v116'741 uv715 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.797+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.801+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v116'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.801+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.793 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:07.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.806+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v116'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.806+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.810+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.810+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.815+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.815+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.819+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.819+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.824+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v116'747 uv721 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.824+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.829+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v116'748 uv748 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.829+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.833+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v116'749 uv749 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.821 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:07.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.834+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.838+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'750 uv750 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.838+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.843+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'751 uv751 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.843+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f11825d0 2024-04-25T11:39:07.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.847+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'752 uv752 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.847+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55d3f1228c30 0x55d3f122b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.847+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1227fc0 con 0x55d3f1228c30 2024-04-25T11:39:07.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.848+0000 7f9c2a400640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55d3f1228c30 0x55d3f122b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.850+0000 7f9c2a400640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55d3f1228c30 0x55d3f122b050 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:07.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.854+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'527 uv526 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.854+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f122b590 con 0x55d3f1228c30 2024-04-25T11:39:07.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.858+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'528 uv528 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.859+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122b930 con 0x55d3f1228c30 2024-04-25T11:39:07.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.862+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'529 uv529 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.850 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:07.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.863+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122bcd0 con 0x55d3f1228c30 2024-04-25T11:39:07.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.867+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v116'530 uv530 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.867+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122c070 con 0x55d3f1228c30 2024-04-25T11:39:07.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.872+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'531 uv531 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.872+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122c410 con 0x55d3f1228c30 2024-04-25T11:39:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.876+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v116'532 uv514 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.877+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f122c7b0 con 0x55d3f1228c30 2024-04-25T11:39:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.881+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v116'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.881+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.872 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:07.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.885+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v116'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.886+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.890+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.890+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.894+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'536 uv536 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.894+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.899+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v116'537 uv519 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.899+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.903+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v116'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.903+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.894 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:07.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.908+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v116'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.908+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1231240 con 0x55d3f1228c30 2024-04-25T11:39:07.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.913+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v116'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.913+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1231240 con 0x55d3f1228c30 2024-04-25T11:39:07.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.917+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.918+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.922+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v116'542 uv525 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.922+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.926+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v116'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.926+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.917 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:07.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.931+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v116'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.931+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122db60 con 0x55d3f1228c30 2024-04-25T11:39:07.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.935+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.936+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122db60 con 0x55d3f1228c30 2024-04-25T11:39:07.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.939+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.940+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122cb50 con 0x55d3f1228c30 2024-04-25T11:39:07.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.943+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f1228c30 2024-04-25T11:39:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.944+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d3f122d060 0x55d3f122f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.944+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f122fa10 con 0x55d3f122d060 2024-04-25T11:39:07.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.945+0000 7f9c29a00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d3f122d060 0x55d3f122f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.946+0000 7f9c29a00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d3f122d060 0x55d3f122f480 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:07.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.950+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:07.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.950+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f122feb0 con 0x55d3f122d060 2024-04-25T11:39:07.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.955+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v116'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:07.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.955+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1230390 con 0x55d3f122d060 2024-04-25T11:39:07.946 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:07.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.959+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v116'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:07.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.959+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1231c40 con 0x55d3f122d060 2024-04-25T11:39:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.963+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.964+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1231fe0 con 0x55d3f122d060 2024-04-25T11:39:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.968+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.968+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1230c30 con 0x55d3f122d060 2024-04-25T11:39:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.972+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:07.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.972+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1230fd0 con 0x55d3f11825d0 2024-04-25T11:39:07.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.977+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'106 uv105 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.977+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1230fd0 con 0x55d3f11825d0 2024-04-25T11:39:07.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.980+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'107 uv107 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.981+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1230fd0 con 0x55d3f11825d0 2024-04-25T11:39:07.971 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.984+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'108 uv108 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.985+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1230fd0 con 0x55d3f11825d0 2024-04-25T11:39:07.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.989+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'109 uv109 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:07.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.989+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55d3f1232380 0x55d3f12347a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:07.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.989+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1234ce0 con 0x55d3f1232380 2024-04-25T11:39:07.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.990+0000 7f9c2ae00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55d3f1232380 0x55d3f12347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.992+0000 7f9c2ae00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55d3f1232380 0x55d3f12347a0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.996+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:07.996+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1234ce0 con 0x55d3f1232380 2024-04-25T11:39:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.000+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v116'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.000+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1234ce0 con 0x55d3f1232380 2024-04-25T11:39:07.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.005+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v116'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:07.992 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:07.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.005+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1235cf0 con 0x55d3f1232380 2024-04-25T11:39:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.009+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.010+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1235cf0 con 0x55d3f1232380 2024-04-25T11:39:08.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.013+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.014+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1234ce0 con 0x55d3f1232380 2024-04-25T11:39:08.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.018+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.018+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d3f12351f0 0x55d3f1237610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.018+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1237cc0 con 0x55d3f12351f0 2024-04-25T11:39:08.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.019+0000 7f9c2a400640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d3f12351f0 0x55d3f1237610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.020+0000 7f9c2a400640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d3f12351f0 0x55d3f1237610 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.024+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v116'260 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.024+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1237d80 con 0x55d3f12351f0 2024-04-25T11:39:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.029+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v116'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.029+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1238170 con 0x55d3f12351f0 2024-04-25T11:39:08.020 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:08.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.033+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v116'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.033+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1239180 con 0x55d3f12351f0 2024-04-25T11:39:08.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.037+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'263 uv263 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.037+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1239180 con 0x55d3f12351f0 2024-04-25T11:39:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.040+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.041+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1238170 con 0x55d3f12351f0 2024-04-25T11:39:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.045+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.045+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1238170 con 0x55d3f12351f0 2024-04-25T11:39:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.049+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v116'266 uv259 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.049+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f1238170 con 0x55d3f12351f0 2024-04-25T11:39:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.053+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v116'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.053+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1238170 con 0x55d3f12351f0 2024-04-25T11:39:08.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.057+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v116'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.044 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:08.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.057+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1238170 con 0x55d3f12351f0 2024-04-25T11:39:08.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.061+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.062+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f1238170 con 0x55d3f12351f0 2024-04-25T11:39:08.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.065+0000 7f9c2a400640 1 -- 172.21.15.19:0/1274273340 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1c01a030 con 0x55d3f12351f0 2024-04-25T11:39:08.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.066+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55d3f1238680 0x55d3f123aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.066+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b230 con 0x55d3f1238680 2024-04-25T11:39:08.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.067+0000 7f9c29a00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55d3f1238680 0x55d3f123aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.068+0000 7f9c29a00640 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55d3f1238680 0x55d3f123aae0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.072+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v116'117 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.072+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123b230 con 0x55d3f1238680 2024-04-25T11:39:08.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.076+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v116'118 uv118 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.076+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f1238680 2024-04-25T11:39:08.067 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:08.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.080+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v116'119 uv119 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.080+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f1238680 2024-04-25T11:39:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.085+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'120 uv120 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.085+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f1238680 2024-04-25T11:39:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.089+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'121 uv121 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.089+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f11825d0 2024-04-25T11:39:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.093+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.093+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f11825d0 2024-04-25T11:39:08.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.097+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v116'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:08.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.097+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f11825d0 2024-04-25T11:39:08.088 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:08.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.101+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v116'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:08.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.102+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123c690 con 0x55d3f11825d0 2024-04-25T11:39:08.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.105+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:08.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.106+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123c690 con 0x55d3f11825d0 2024-04-25T11:39:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.110+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.110+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f11825d0 2024-04-25T11:39:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.114+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f11825d0 2024-04-25T11:39:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.114+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.119+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v116'233 uv227 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.119+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.123+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v116'234 uv234 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.123+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.114 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.127+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v116'235 uv235 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.127+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.132+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'236 uv236 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.132+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.136+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'237 uv237 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.136+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.140+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v116'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.140+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.144+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v116'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.144+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.135 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.149+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v116'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.149+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123b680 con 0x55d3f122d060 2024-04-25T11:39:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.153+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.153+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123e490 con 0x55d3f122d060 2024-04-25T11:39:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.158+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f122d060 2024-04-25T11:39:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.158+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.162+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v116'103 uv102 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.162+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.166+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'104 uv104 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.167+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.157 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:08.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.171+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v116'105 uv105 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.171+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.175+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'106 uv106 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.175+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.180+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v116'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.180+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.185+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v116'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.185+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.177 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.190+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v116'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.190+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123cb40 con 0x55d3f1232380 2024-04-25T11:39:08.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.194+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.194+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123cb40 con 0x55d3f1232380 2024-04-25T11:39:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.199+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.199+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1232380 2024-04-25T11:39:08.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.203+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c240094f0 con 0x55d3f1232380 2024-04-25T11:39:08.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.203+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.207+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v116'120 uv119 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.207+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.212+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v116'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.212+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.216+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v116'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.203 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:08.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.216+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.220+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.221+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.224+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.224+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.228+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v116'290 uv282 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.229+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.233+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v116'291 uv291 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.233+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.224 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:08.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.237+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v116'292 uv292 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.238+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.242+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v116'293 uv293 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.242+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.246+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'294 uv294 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.246+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.251+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v116'295 uv289 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.251+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.255+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v116'296 uv296 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.255+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.246 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:08.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.259+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v116'297 uv297 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.259+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123cb40 con 0x55d3f1238680 2024-04-25T11:39:08.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.263+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'298 uv298 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.263+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123cb40 con 0x55d3f1238680 2024-04-25T11:39:08.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.268+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'299 uv299 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.268+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123bb30 con 0x55d3f1238680 2024-04-25T11:39:08.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.272+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'300 uv300 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.272+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55d3f123c090 con 0x55d3f1238680 2024-04-25T11:39:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.276+0000 7f9c29a00640 1 -- 172.21.15.19:0/1274273340 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'301 uv301 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c1405c040 con 0x55d3f1238680 2024-04-25T11:39:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55d3f1228c30 msgr2=0x55d3f122b050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55d3f1228c30 0x55d3f122b050 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d3f12351f0 msgr2=0x55d3f1237610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55d3f12351f0 0x55d3f1237610 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55d3f1238680 msgr2=0x55d3f123aae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55d3f1238680 0x55d3f123aae0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d3f122d060 msgr2=0x55d3f122f480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d3f122d060 0x55d3f122f480 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 reap_dead start 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d3f11825d0 msgr2=0x55d3f117bb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d3f11825d0 0x55d3f117bb00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c2ae00640 1 -- 172.21.15.19:0/1274273340 reap_dead start 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55d3f1232380 msgr2=0x55d3f12347a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55d3f1232380 0x55d3f12347a0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c08072ec0 msgr2=0x7f9c080753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c08072ec0 0x7f9c080753d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9c14025830 tx=0x7f9c14001730 comp rx=0 tx=0).stop 2024-04-25T11:39:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f10dc770 msgr2=0x55d3f11771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.277+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f10dc770 0x55d3f11771b0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d470 tx=0x7f9c1c00d930 comp rx=0 tx=0).stop 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.278+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 shutdown_connections 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.278+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55d3f1232380 0x55d3f12347a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.278+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c08072ec0 0x7f9c080753d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.278+0000 7f9c3125cf40 1 --2- 172.21.15.19:0/1274273340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d3f10dc770 0x55d3f11771b0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.278+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 >> 172.21.15.19:0/1274273340 conn(0x55d3f0fceac0 msgr2=0x55d3f1178340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.278+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 shutdown_connections 2024-04-25T11:39:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.278+0000 7f9c3125cf40 1 -- 172.21.15.19:0/1274273340 wait complete. 2024-04-25T11:39:08.295 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2024-04-25T11:39:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.544+0000 7f6f5371bf40 1 Processor -- start 2024-04-25T11:39:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.544+0000 7f6f5371bf40 1 -- start start 2024-04-25T11:39:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.545+0000 7f6f5371bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 0x55acf25135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.545+0000 7f6f5371bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf24405f0 0x55acf248d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.545+0000 7f6f5371bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf2519fa0 0x55acf248d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.545+0000 7f6f5371bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55acf2488e70 con 0x55acf24405f0 2024-04-25T11:39:08.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.545+0000 7f6f5371bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55acf23e6d20 con 0x55acf24401e0 2024-04-25T11:39:08.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.545+0000 7f6f5371bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55acf24893f0 con 0x55acf2519fa0 2024-04-25T11:39:08.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.545+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf24405f0 0x55acf248d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf24405f0 0x55acf248d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41014/0 (socket says 172.21.15.19:41014) 2024-04-25T11:39:08.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 -- 172.21.15.19:0/1941622442 learned_addr learned my addr 172.21.15.19:0/1941622442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4c800640 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf2519fa0 0x55acf248d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 -- 172.21.15.19:0/1941622442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf2519fa0 msgr2=0x55acf248d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf2519fa0 0x55acf248d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 -- 172.21.15.19:0/1941622442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 msgr2=0x55acf25135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 0x55acf25135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 -- 172.21.15.19:0/1941622442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf243ef70 con 0x55acf24405f0 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.546+0000 7f6f4b400640 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf24405f0 0x55acf248d060 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800b240 tx=0x7f6f3800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3ba1a40cf70763 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f4a000640 1 -- 172.21.15.19:0/1941622442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38008070 con 0x55acf24405f0 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f4a000640 1 -- 172.21.15.19:0/1941622442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f3800d500 con 0x55acf24405f0 2024-04-25T11:39:08.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f4a000640 1 -- 172.21.15.19:0/1941622442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800d910 con 0x55acf24405f0 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1941622442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf24405f0 msgr2=0x55acf248d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf24405f0 0x55acf248d060 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800b240 tx=0x7f6f3800b700 comp rx=0 tx=0).stop 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1941622442 shutdown_connections 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf2519fa0 0x55acf248d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf24405f0 0x55acf248d060 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1941622442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 0x55acf25135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1941622442 >> 172.21.15.19:0/1941622442 conn(0x55acf2362ac0 msgr2=0x55acf243f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1941622442 shutdown_connections 2024-04-25T11:39:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.547+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1941622442 wait complete. 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.549+0000 7f6f5371bf40 1 Processor -- start 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.549+0000 7f6f5371bf40 1 -- start start 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f5371bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 0x55acf25a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f5371bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf24405f0 0x55acf248cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f5371bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf2519fa0 0x55acf248d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f5371bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55acf2514600 con 0x55acf2519fa0 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f5371bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55acf2516360 con 0x55acf24401e0 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f5371bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55acf2516640 con 0x55acf24405f0 2024-04-25T11:39:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f4c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf2519fa0 0x55acf248d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f4c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf2519fa0 0x55acf248d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41028/0 (socket says 172.21.15.19:41028) 2024-04-25T11:39:08.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.550+0000 7f6f4c800640 1 -- 172.21.15.19:0/1983218442 learned_addr learned my addr 172.21.15.19:0/1983218442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:08.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4b400640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf24405f0 0x55acf248cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4be00640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 0x55acf25a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4c800640 1 -- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf24405f0 msgr2=0x55acf248cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4c800640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf24405f0 0x55acf248cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4c800640 1 -- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 msgr2=0x55acf25a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4c800640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 0x55acf25a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4c800640 1 -- 172.21.15.19:0/1983218442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf2487030 con 0x55acf2519fa0 2024-04-25T11:39:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4b400640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55acf24405f0 0x55acf248cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4be00640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55acf24401e0 0x55acf25a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.551+0000 7f6f4c800640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf2519fa0 0x55acf248d300 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f6f44025c70 tx=0x7f6f44002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.552+0000 7f6f48c00640 1 -- 172.21.15.19:0/1983218442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44054b70 con 0x55acf2519fa0 2024-04-25T11:39:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.552+0000 7f6f48c00640 1 -- 172.21.15.19:0/1983218442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f44054d10 con 0x55acf2519fa0 2024-04-25T11:39:08.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.552+0000 7f6f48c00640 1 -- 172.21.15.19:0/1983218442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4405e650 con 0x55acf2519fa0 2024-04-25T11:39:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.552+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55acf246ff70 con 0x55acf2519fa0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.552+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf2470530 con 0x55acf2519fa0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.554+0000 7f6f48c00640 1 -- 172.21.15.19:0/1983218442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6f4405e7f0 con 0x55acf2519fa0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.555+0000 7f6f48c00640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f2c072e90 0x7f6f2c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.555+0000 7f6f48c00640 1 -- 172.21.15.19:0/1983218442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7f6f440d8770 con 0x55acf2519fa0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.556+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55acf2485fa0 0x55acf24863b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.556+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55acf2512d40 con 0x55acf2485fa0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.556+0000 7f6f4b400640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55acf2485fa0 0x55acf24863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.556+0000 7f6f4be00640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f2c072e90 0x7f6f2c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.557+0000 7f6f4b400640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55acf2485fa0 0x55acf24863b0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.557+0000 7f6f4be00640 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f2c072e90 0x7f6f2c0753a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c005700 tx=0x7f6f3c007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f4b400640 1 -- 172.21.15.19:0/1983218442 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v116'548 uv548 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6f3800d730 con 0x55acf2485fa0 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55acf2485fa0 msgr2=0x55acf24863b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55acf2485fa0 0x55acf24863b0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f2c072e90 msgr2=0x7f6f2c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f2c072e90 0x7f6f2c0753a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c005700 tx=0x7f6f3c007040 comp rx=0 tx=0).stop 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf2519fa0 msgr2=0x55acf248d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 --2- 172.21.15.19:0/1983218442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55acf2519fa0 0x55acf248d300 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f6f44025c70 tx=0x7f6f44002fb0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f4c800640 1 -- 172.21.15.19:0/1983218442 reap_dead start 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 shutdown_connections 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 >> 172.21.15.19:0/1983218442 conn(0x55acf2362ac0 msgr2=0x55acf243f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.560+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 shutdown_connections 2024-04-25T11:39:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.561+0000 7f6f5371bf40 1 -- 172.21.15.19:0/1983218442 wait complete. 2024-04-25T11:39:08.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.851+0000 7f45d7000640 1 Processor -- start 2024-04-25T11:39:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.851+0000 7f45d7000640 1 -- start start 2024-04-25T11:39:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d7000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45d0144340 0x7f45d006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d7000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d7000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d006df20 con 0x7f45d015ec90 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d7000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f45d0144750 con 0x7f45d0144340 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d7000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45d0070000 con 0x7f45d006db10 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45d0144340 0x7f45d006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45d0144340 0x7f45d006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41040/0 (socket says 172.21.15.19:41040) 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.854+0000 7f45d5c00640 1 -- 172.21.15.19:0/1291092251 learned_addr learned my addr 172.21.15.19:0/1291092251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:08.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.855+0000 7f45d5200640 1 --2- 172.21.15.19:0/1291092251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.855+0000 7f45d5c00640 1 -- 172.21.15.19:0/1291092251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 msgr2=0x7f45d015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.855+0000 7f45d5c00640 1 --2- 172.21.15.19:0/1291092251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.855+0000 7f45d5c00640 1 -- 172.21.15.19:0/1291092251 >> v1:172.21.15.169:6789/0 conn(0x7f45d015ec90 legacy=0x7f45d0161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.855+0000 7f45d5c00640 1 -- 172.21.15.19:0/1291092251 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d006f240 con 0x7f45d0144340 2024-04-25T11:39:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.856+0000 7f45d5c00640 1 --2- 172.21.15.19:0/1291092251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d0144340 0x7f45d006d5d0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009870 tx=0x7f45c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d9a73f2f835631 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.856+0000 7f45cfe00640 1 -- 172.21.15.19:0/1291092251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4037070 con 0x7f45d0144340 2024-04-25T11:39:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.856+0000 7f45cfe00640 1 -- 172.21.15.19:0/1291092251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4032440 con 0x7f45d0144340 2024-04-25T11:39:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.856+0000 7f45cfe00640 1 -- 172.21.15.19:0/1291092251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4032810 con 0x7f45d0144340 2024-04-25T11:39:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.857+0000 7f45d7000640 1 -- 172.21.15.19:0/1291092251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d0144340 msgr2=0x7f45d006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.857+0000 7f45d7000640 1 --2- 172.21.15.19:0/1291092251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d0144340 0x7f45d006d5d0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009870 tx=0x7f45c402a210 comp rx=0 tx=0).stop 2024-04-25T11:39:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.857+0000 7f45d7000640 1 -- 172.21.15.19:0/1291092251 shutdown_connections 2024-04-25T11:39:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.857+0000 7f45d7000640 1 --2- 172.21.15.19:0/1291092251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.857+0000 7f45d7000640 1 --2- 172.21.15.19:0/1291092251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d0144340 0x7f45d006d5d0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.857+0000 7f45d7000640 1 -- 172.21.15.19:0/1291092251 >> 172.21.15.19:0/1291092251 conn(0x7f45d0072f40 msgr2=0x7f45d0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.858+0000 7f45d7000640 1 -- 172.21.15.19:0/1291092251 shutdown_connections 2024-04-25T11:39:08.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.858+0000 7f45d7000640 1 -- 172.21.15.19:0/1291092251 wait complete. 2024-04-25T11:39:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.861+0000 7f45d7000640 1 Processor -- start 2024-04-25T11:39:08.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.861+0000 7f45d7000640 1 -- start start 2024-04-25T11:39:08.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d7000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d7000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d015ec90 0x7f45d00b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d7000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d01475a0 0x7f45d00b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d7000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45d006fae0 con 0x7f45d01475a0 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d7000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45d006f800 con 0x7f45d015ec90 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d7000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45d015f0a0 con 0x7f45d006db10 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d01475a0 0x7f45d00b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d01475a0 0x7f45d00b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41046/0 (socket says 172.21.15.19:41046) 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d6600640 1 -- 172.21.15.19:0/3745882548 learned_addr learned my addr 172.21.15.19:0/3745882548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.863+0000 7f45d5c00640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d5200640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d015ec90 0x7f45d00b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d6600640 1 -- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 msgr2=0x7f45d015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d6600640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d6600640 1 -- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d015ec90 msgr2=0x7f45d00b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d6600640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d015ec90 0x7f45d00b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d6600640 1 -- 172.21.15.19:0/3745882548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d00af860 con 0x7f45d01475a0 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d5200640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d015ec90 0x7f45d00b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.864+0000 7f45d6600640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d01475a0 0x7f45d00b0c50 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f45c800b9a0 tx=0x7f45c800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.865+0000 7f45cd600640 1 -- 172.21.15.19:0/3745882548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8019070 con 0x7f45d01475a0 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.865+0000 7f45cd600640 1 -- 172.21.15.19:0/3745882548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c800cba0 con 0x7f45d01475a0 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.865+0000 7f45cd600640 1 -- 172.21.15.19:0/3745882548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8014440 con 0x7f45d01475a0 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.865+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d00afb40 con 0x7f45d01475a0 2024-04-25T11:39:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.865+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d00b4c50 con 0x7f45d01475a0 2024-04-25T11:39:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.866+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d0076c60 con 0x7f45d01475a0 2024-04-25T11:39:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.867+0000 7f45cd600640 1 -- 172.21.15.19:0/3745882548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f45c801c070 con 0x7f45d01475a0 2024-04-25T11:39:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.869+0000 7f45cd600640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45b00744e0 0x7f45b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.869+0000 7f45cd600640 1 -- 172.21.15.19:0/3745882548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7773+0+0 (secure 0 0 0) 0x7f45c8095690 con 0x7f45d01475a0 2024-04-25T11:39:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.869+0000 7f45d5c00640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45b00744e0 0x7f45b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.871+0000 7f45d5c00640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45b00744e0 0x7f45b00769f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45c4034000 tx=0x7f45c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:08.874+0000 7f45cd600640 1 -- 172.21.15.19:0/3745882548 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45c805ef20 con 0x7f45d01475a0 2024-04-25T11:39:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.123+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f45d0142f70 con 0x7f45d01475a0 2024-04-25T11:39:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.158+0000 7f45cd600640 1 -- 172.21.15.19:0/3745882548 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v246) ==== 162+0+0 (secure 0 0 0) 0x7f45c8063220 con 0x7f45d01475a0 2024-04-25T11:39:09.146 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.161+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45b00744e0 msgr2=0x7f45b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.161+0000 7f45d7000640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45b00744e0 0x7f45b00769f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45c4034000 tx=0x7f45c402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d01475a0 msgr2=0x7f45d00b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d01475a0 0x7f45d00b0c50 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f45c800b9a0 tx=0x7f45c800be60 comp rx=0 tx=0).stop 2024-04-25T11:39:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 shutdown_connections 2024-04-25T11:39:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45b00744e0 0x7f45b00769f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45d015ec90 0x7f45d00b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45d01475a0 0x7f45d00b0c50 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 --2- 172.21.15.19:0/3745882548 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45d006db10 0x7f45d015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 >> 172.21.15.19:0/3745882548 conn(0x7f45d0072f40 msgr2=0x7f45d015c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 shutdown_connections 2024-04-25T11:39:09.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.162+0000 7f45d7000640 1 -- 172.21.15.19:0/3745882548 wait complete. 2024-04-25T11:39:09.156 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:09.170+0000 7fefde000640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:39:09.157 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:09.170+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:09.157 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:09.170+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:39:09.170 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:39:09.170+0000 7fefde000640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:39:09.177 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:09.294 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.450+0000 7fe6e8a00640 1 Processor -- start 2024-04-25T11:39:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.451+0000 7fe6e8a00640 1 -- start start 2024-04-25T11:39:09.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.453+0000 7fe6e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 0x7fe6e40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.453+0000 7fe6e8a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e40d1230 con 0x7fe6e4144370 2024-04-25T11:39:09.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.453+0000 7fe6e8a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e40d06b0 con 0x7fe6e40b4ea0 2024-04-25T11:39:09.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.453+0000 7fe6e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e40d0990 con 0x7fe6e406c5c0 2024-04-25T11:39:09.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.453+0000 7fe6e2a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe6e4144370 0x7fe6e40b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40884/0 (socket says 172.21.15.19:40884) 2024-04-25T11:39:09.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.453+0000 7fe6e2a00640 1 -- 172.21.15.19:0/3122841072 learned_addr learned my addr 172.21.15.19:0/3122841072 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e3400640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 0x7fe6e40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:09.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591427108 0 0) 0x7fe6e40d1230 con 0x7fe6e4144370 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6c8003520 con 0x7fe6e4144370 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 731763887 0 0) 0x7fe6c8003520 con 0x7fe6e4144370 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 msgr2=0x7fe6e40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 0x7fe6e40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 >> v1:172.21.15.169:6789/0 conn(0x7fe6e40b4ea0 legacy=0x7fe6e415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e40d0c70 con 0x7fe6e4144370 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe6d40019f0 con 0x7fe6e4144370 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 >> v1:172.21.15.19:6789/0 conn(0x7fe6e4144370 legacy=0x7fe6e40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 0x7fe6c80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c8003f60 0x7fe6c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c8004d40 0x7fe6c8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40d0990 con 0x7fe6c8004d40 2024-04-25T11:39:09.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c800bb50 con 0x7fe6c8003f60 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.454+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c800bd50 con 0x7fe6e406c5c0 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3e00640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c8004d40 0x7fe6c8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3400640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 0x7fe6c80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3e00640 1 -- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 msgr2=0x7fe6c80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e2a00640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c8003f60 0x7fe6c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3e00640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 0x7fe6c80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3e00640 1 -- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c8003f60 msgr2=0x7fe6c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3e00640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6c8003f60 0x7fe6c8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3e00640 1 -- 172.21.15.19:0/3122841072 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c8003520 con 0x7fe6c8004d40 2024-04-25T11:39:09.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.455+0000 7fe6e3400640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6e406c5c0 0x7fe6c80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.456+0000 7fe6e3e00640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c8004d40 0x7fe6c8005150 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc001720 tx=0x7fe6c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6bca4a53affd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.456+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc001370 con 0x7fe6c8004d40 2024-04-25T11:39:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.456+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc0052d0 con 0x7fe6c8004d40 2024-04-25T11:39:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.456+0000 7fe6e2000640 1 -- 172.21.15.19:0/3122841072 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc02abc0 con 0x7fe6c8004d40 2024-04-25T11:39:09.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.457+0000 7fe6e8a00640 1 -- 172.21.15.19:0/3122841072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c8004d40 msgr2=0x7fe6c8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.457+0000 7fe6e8a00640 1 --2- 172.21.15.19:0/3122841072 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6c8004d40 0x7fe6c8005150 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc001720 tx=0x7fe6c8005830 comp rx=0 tx=0).stop 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.457+0000 7fe6e3e00640 1 -- 172.21.15.19:0/3122841072 reap_dead start 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.457+0000 7fe6e8a00640 1 -- 172.21.15.19:0/3122841072 shutdown_connections 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.457+0000 7fe6e8a00640 1 -- 172.21.15.19:0/3122841072 >> 172.21.15.19:0/3122841072 conn(0x7fe6e40d1720 msgr2=0x7fe6e40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.457+0000 7fe6e8a00640 1 -- 172.21.15.19:0/3122841072 shutdown_connections 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.457+0000 7fe6e8a00640 1 -- 172.21.15.19:0/3122841072 wait complete. 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.460+0000 7fe6e8a00640 1 Processor -- start 2024-04-25T11:39:09.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.460+0000 7fe6e8a00640 1 -- start start 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.461+0000 7fe6e8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c8004d40 0x7fe6e4143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.461+0000 7fe6e8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6e4155d30 0x7fe6e4153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.461+0000 7fe6e8a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4160f90 0x7fe6e415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.461+0000 7fe6e8a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40d0450 con 0x7fe6e4155d30 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.461+0000 7fe6e8a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40d02d0 con 0x7fe6e4160f90 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.461+0000 7fe6e8a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e4147760 con 0x7fe6c8004d40 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6e4155d30 0x7fe6e4153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6e4155d30 0x7fe6e4153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41096/0 (socket says 172.21.15.19:41096) 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 -- 172.21.15.19:0/2326166535 learned_addr learned my addr 172.21.15.19:0/2326166535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e3400640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c8004d40 0x7fe6e4143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:09.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 -- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c8004d40 msgr2=0x7fe6e4143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c8004d40 0x7fe6e4143140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 -- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4160f90 msgr2=0x7fe6e415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4160f90 0x7fe6e415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 -- 172.21.15.19:0/2326166535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e415fef0 con 0x7fe6e4155d30 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.462+0000 7fe6e2a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6e4155d30 0x7fe6e4153110 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4002140 tx=0x7fe6d400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.463+0000 7fe6db400640 1 -- 172.21.15.19:0/2326166535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4019070 con 0x7fe6e4155d30 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.463+0000 7fe6db400640 1 -- 172.21.15.19:0/2326166535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d4004d60 con 0x7fe6e4155d30 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.463+0000 7fe6db400640 1 -- 172.21.15.19:0/2326166535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d40072d0 con 0x7fe6e4155d30 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.463+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e41601d0 con 0x7fe6e4155d30 2024-04-25T11:39:09.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.463+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e4160790 con 0x7fe6e4155d30 2024-04-25T11:39:09.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.464+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e4143680 con 0x7fe6e4155d30 2024-04-25T11:39:09.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.465+0000 7fe6db400640 1 -- 172.21.15.19:0/2326166535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe6d401c070 con 0x7fe6e4155d30 2024-04-25T11:39:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.466+0000 7fe6db400640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6c00744b0 0x7fe6c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.466+0000 7fe6db400640 1 -- 172.21.15.19:0/2326166535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7fe6d4095ca0 con 0x7fe6e4155d30 2024-04-25T11:39:09.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.467+0000 7fe6e3400640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6c00744b0 0x7fe6c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:09.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.468+0000 7fe6e3400640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6c00744b0 0x7fe6c00769c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0099c0 tx=0x7fe6cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:09.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.471+0000 7fe6db400640 1 -- 172.21.15.19:0/2326166535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6e4143680 con 0x7fe6e4155d30 2024-04-25T11:39:09.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.713+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6e4050740 con 0x7fe6e4155d30 2024-04-25T11:39:09.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.714+0000 7fe6db400640 1 -- 172.21.15.19:0/2326166535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 247 v247) ==== 95+0+4352 (secure 0 0 0) 0x7fe6d405fb20 con 0x7fe6e4155d30 2024-04-25T11:39:09.701 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:09.701 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":247,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":247,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:09.171149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:09.701 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 247 2024-04-25T11:39:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6c00744b0 msgr2=0x7fe6c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6c00744b0 0x7fe6c00769c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0099c0 tx=0x7fe6cc008040 comp rx=0 tx=0).stop 2024-04-25T11:39:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6e4155d30 msgr2=0x7fe6e4153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6e4155d30 0x7fe6e4153110 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4002140 tx=0x7fe6d400b4a0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 shutdown_connections 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6c00744b0 0x7fe6c00769c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6e4160f90 0x7fe6e415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6e4155d30 0x7fe6e4153110 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 --2- 172.21.15.19:0/2326166535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6c8004d40 0x7fe6e4143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 >> 172.21.15.19:0/2326166535 conn(0x7fe6e40d1720 msgr2=0x7fe6e4143910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 shutdown_connections 2024-04-25T11:39:09.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:09.716+0000 7fe6e8a00640 1 -- 172.21.15.19:0/2326166535 wait complete. 2024-04-25T11:39:09.729 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:09.729 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:10.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.019+0000 7f45fc000640 1 Processor -- start 2024-04-25T11:39:10.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.020+0000 7f45fc000640 1 -- start start 2024-04-25T11:39:10.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.021+0000 7f45fc000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45f41443c0 0x7f45f406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.021+0000 7f45fc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f40cfcf0 0x7f45f406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.021+0000 7f45fc000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f415f5f0 con 0x7f45f4070370 2024-04-25T11:39:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.021+0000 7f45fc000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f45f40708d0 con 0x7f45f41443c0 2024-04-25T11:39:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.021+0000 7f45fc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45f415f160 con 0x7f45f40cfcf0 2024-04-25T11:39:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45f41443c0 0x7f45f406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45f41443c0 0x7f45f406fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41102/0 (socket says 172.21.15.19:41102) 2024-04-25T11:39:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f9600640 1 -- 172.21.15.19:0/780907978 learned_addr learned my addr 172.21.15.19:0/780907978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45fa000640 1 --2- 172.21.15.19:0/780907978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f40cfcf0 0x7f45f406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f9600640 1 -- 172.21.15.19:0/780907978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f40cfcf0 msgr2=0x7f45f406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f9600640 1 --2- 172.21.15.19:0/780907978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f40cfcf0 0x7f45f406c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f9600640 1 -- 172.21.15.19:0/780907978 >> v1:172.21.15.19:6789/0 conn(0x7f45f4070370 legacy=0x7f45f40cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f3e00640 1 -- 172.21.15.19:0/780907978 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649468059 0 0) 0x7f45f415f5f0 con 0x7f45f4070370 2024-04-25T11:39:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45f9600640 1 -- 172.21.15.19:0/780907978 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f415e830 con 0x7f45f41443c0 2024-04-25T11:39:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.022+0000 7f45fa000640 1 --2- 172.21.15.19:0/780907978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f40cfcf0 0x7f45f406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:10.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.024+0000 7f45f9600640 1 --2- 172.21.15.19:0/780907978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f406fe30 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009870 tx=0x7f45e802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2369bfdf5da958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.024+0000 7f45f3e00640 1 -- 172.21.15.19:0/780907978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8037070 con 0x7f45f41443c0 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.024+0000 7f45f3e00640 1 -- 172.21.15.19:0/780907978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e8032440 con 0x7f45f41443c0 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.024+0000 7f45f3e00640 1 -- 172.21.15.19:0/780907978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8032810 con 0x7f45f41443c0 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 -- 172.21.15.19:0/780907978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 msgr2=0x7f45f406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 --2- 172.21.15.19:0/780907978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f406fe30 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009870 tx=0x7f45e802a250 comp rx=0 tx=0).stop 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 -- 172.21.15.19:0/780907978 shutdown_connections 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 --2- 172.21.15.19:0/780907978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f40cfcf0 0x7f45f406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 --2- 172.21.15.19:0/780907978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f406fe30 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 -- 172.21.15.19:0/780907978 >> 172.21.15.19:0/780907978 conn(0x7f45f40d1ae0 msgr2=0x7f45f40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 -- 172.21.15.19:0/780907978 shutdown_connections 2024-04-25T11:39:10.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.025+0000 7f45fc000640 1 -- 172.21.15.19:0/780907978 wait complete. 2024-04-25T11:39:10.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.027+0000 7f45fc000640 1 Processor -- start 2024-04-25T11:39:10.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.027+0000 7f45fc000640 1 -- start start 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45fc000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f4070370 0x7f45f40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45fc000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f41f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45fc000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f41f6ba0 0x7f45f40afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45fc000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45f41447d0 con 0x7f45f41443c0 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45fc000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f45f415edf0 con 0x7f45f41f6ba0 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45fc000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45f4162b00 con 0x7f45f4070370 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45f8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f41f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45f8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f41f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41114/0 (socket says 172.21.15.19:41114) 2024-04-25T11:39:10.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.029+0000 7f45f8c00640 1 -- 172.21.15.19:0/1166803694 learned_addr learned my addr 172.21.15.19:0/1166803694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f9600640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f4070370 0x7f45f40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45fa000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f41f6ba0 0x7f45f40afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f8c00640 1 -- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f4070370 msgr2=0x7f45f40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f8c00640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f4070370 0x7f45f40c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f8c00640 1 -- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f41f6ba0 msgr2=0x7f45f40afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f8c00640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f41f6ba0 0x7f45f40afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f8c00640 1 -- 172.21.15.19:0/1166803694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f41f7180 con 0x7f45f41443c0 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45fa000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f41f6ba0 0x7f45f40afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f8c00640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f41f6660 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c440 tx=0x7f45e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f1600640 1 -- 172.21.15.19:0/1166803694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0017070 con 0x7f45f41443c0 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f1600640 1 -- 172.21.15.19:0/1166803694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e00035f0 con 0x7f45f41443c0 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45f1600640 1 -- 172.21.15.19:0/1166803694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e00039e0 con 0x7f45f41443c0 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f41f7460 con 0x7f45f41443c0 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.030+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f41f8fc0 con 0x7f45f41443c0 2024-04-25T11:39:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.032+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f4050740 con 0x7f45f41443c0 2024-04-25T11:39:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.033+0000 7f45f1600640 1 -- 172.21.15.19:0/1166803694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f45e001a070 con 0x7f45f41443c0 2024-04-25T11:39:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.034+0000 7f45f1600640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45d40744b0 0x7f45d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.034+0000 7f45f1600640 1 -- 172.21.15.19:0/1166803694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7f45e0094de0 con 0x7f45f41443c0 2024-04-25T11:39:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.035+0000 7f45f9600640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45d40744b0 0x7f45d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.036+0000 7f45f9600640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45d40744b0 0x7f45d40769c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f45e8034000 tx=0x7f45e802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:10.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.039+0000 7f45f1600640 1 -- 172.21.15.19:0/1166803694 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45e005ee40 con 0x7f45f41443c0 2024-04-25T11:39:10.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.293+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45f406f950 con 0x7f45f41443c0 2024-04-25T11:39:10.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.294+0000 7f45f1600640 1 -- 172.21.15.19:0/1166803694 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 248 v248) ==== 95+0+5178 (secure 0 0 0) 0x7f45e0092e20 con 0x7f45f41443c0 2024-04-25T11:39:10.281 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:10.281 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":248,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9852,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3000889873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3000889873},{"type":"v1","addr":"172.21.15.19:6835","nonce":3000889873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":248}],"filesystems":[{"mdsmap":{"epoch":247,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:09.171149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:10.281 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 248 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45d40744b0 msgr2=0x7f45d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45d40744b0 0x7f45d40769c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f45e8034000 tx=0x7f45e802e000 comp rx=0 tx=0).stop 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 msgr2=0x7f45f41f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f41f6660 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c440 tx=0x7f45e000c900 comp rx=0 tx=0).stop 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 shutdown_connections 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45d40744b0 0x7f45d40769c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45f41f6ba0 0x7f45f40afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45f41443c0 0x7f45f41f6660 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 --2- 172.21.15.19:0/1166803694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f45f4070370 0x7f45f40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 >> 172.21.15.19:0/1166803694 conn(0x7f45f40d1ae0 msgr2=0x7f45f40b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:10.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 shutdown_connections 2024-04-25T11:39:10.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.297+0000 7f45fc000640 1 -- 172.21.15.19:0/1166803694 wait complete. 2024-04-25T11:39:10.311 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:10.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.614+0000 7fac38000640 1 Processor -- start 2024-04-25T11:39:10.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.614+0000 7fac38000640 1 -- start start 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac38000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fac301443c0 0x7fac3006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac38000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fac30070370 0x7fac300cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac38000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac300cfb20 0x7fac300ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac38000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fac300708d0 con 0x7fac301443c0 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac38000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fac3015f160 con 0x7fac30070370 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac38000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac300cff30 con 0x7fac300cfb20 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac35600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fac301443c0 0x7fac3006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac35600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fac301443c0 0x7fac3006fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41128/0 (socket says 172.21.15.19:41128) 2024-04-25T11:39:10.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac35600640 1 -- 172.21.15.19:0/433961710 learned_addr learned my addr 172.21.15.19:0/433961710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac34c00640 1 --2- 172.21.15.19:0/433961710 >> v2:172.21.15.169:3300/0 conn(0x7fac30070370 0x7fac300cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.617+0000 7fac36000640 1 --2- 172.21.15.19:0/433961710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac300cfb20 0x7fac300ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.618+0000 7fac35600640 1 -- 172.21.15.19:0/433961710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac300cfb20 msgr2=0x7fac300ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.618+0000 7fac35600640 1 --2- 172.21.15.19:0/433961710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac300cfb20 0x7fac300ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.618+0000 7fac35600640 1 -- 172.21.15.19:0/433961710 >> v2:172.21.15.169:3300/0 conn(0x7fac30070370 msgr2=0x7fac300cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:10.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.618+0000 7fac35600640 1 --2- 172.21.15.19:0/433961710 >> v2:172.21.15.169:3300/0 conn(0x7fac30070370 0x7fac300cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.618+0000 7fac35600640 1 -- 172.21.15.19:0/433961710 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3015f5f0 con 0x7fac301443c0 2024-04-25T11:39:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.618+0000 7fac36000640 1 --2- 172.21.15.19:0/433961710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac300cfb20 0x7fac300ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.619+0000 7fac35600640 1 --2- 172.21.15.19:0/433961710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac301443c0 0x7fac3006fe30 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fac3006f920 tx=0x7fac2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b76fc2453a104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.619+0000 7fac2fe00640 1 -- 172.21.15.19:0/433961710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24037070 con 0x7fac301443c0 2024-04-25T11:39:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.619+0000 7fac2fe00640 1 -- 172.21.15.19:0/433961710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac24032440 con 0x7fac301443c0 2024-04-25T11:39:10.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.619+0000 7fac2fe00640 1 -- 172.21.15.19:0/433961710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24032810 con 0x7fac301443c0 2024-04-25T11:39:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 -- 172.21.15.19:0/433961710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac301443c0 msgr2=0x7fac3006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 --2- 172.21.15.19:0/433961710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac301443c0 0x7fac3006fe30 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fac3006f920 tx=0x7fac2402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 -- 172.21.15.19:0/433961710 shutdown_connections 2024-04-25T11:39:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 --2- 172.21.15.19:0/433961710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac300cfb20 0x7fac300ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 --2- 172.21.15.19:0/433961710 >> v2:172.21.15.169:3300/0 conn(0x7fac30070370 0x7fac300cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 --2- 172.21.15.19:0/433961710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac301443c0 0x7fac3006fe30 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 -- 172.21.15.19:0/433961710 >> 172.21.15.19:0/433961710 conn(0x7fac300d1ae0 msgr2=0x7fac300d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 -- 172.21.15.19:0/433961710 shutdown_connections 2024-04-25T11:39:10.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.620+0000 7fac38000640 1 -- 172.21.15.19:0/433961710 wait complete. 2024-04-25T11:39:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.623+0000 7fac38000640 1 Processor -- start 2024-04-25T11:39:10.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.623+0000 7fac38000640 1 -- start start 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.624+0000 7fac38000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac30070370 0x7fac30159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.624+0000 7fac38000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac300cfb20 0x7fac300b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.624+0000 7fac38000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac301443c0 0x7fac30158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.624+0000 7fac38000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac3015edf0 con 0x7fac300cfb20 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.624+0000 7fac38000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fac3015eb10 con 0x7fac30070370 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.624+0000 7fac38000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fac301447d0 con 0x7fac301443c0 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac300cfb20 0x7fac300b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac300cfb20 0x7fac300b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41142/0 (socket says 172.21.15.19:41142) 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 -- 172.21.15.19:0/1942204990 learned_addr learned my addr 172.21.15.19:0/1942204990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 -- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac301443c0 msgr2=0x7fac30158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac35600640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac30070370 0x7fac30159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac36000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac301443c0 0x7fac30158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac301443c0 0x7fac30158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 -- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac30070370 msgr2=0x7fac30159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac30070370 0x7fac30159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 -- 172.21.15.19:0/1942204990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac301e8440 con 0x7fac300cfb20 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac36000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac301443c0 0x7fac30158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac35600640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac30070370 0x7fac30159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.625+0000 7fac34c00640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac300cfb20 0x7fac300b5ca0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fac2000d5c0 tx=0x7fac2000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.626+0000 7fac2d600640 1 -- 172.21.15.19:0/1942204990 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20004240 con 0x7fac300cfb20 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.626+0000 7fac2d600640 1 -- 172.21.15.19:0/1942204990 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac20019070 con 0x7fac300cfb20 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.626+0000 7fac2d600640 1 -- 172.21.15.19:0/1942204990 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20003610 con 0x7fac300cfb20 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.626+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac301e8720 con 0x7fac300cfb20 2024-04-25T11:39:10.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.626+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3015a6c0 con 0x7fac300cfb20 2024-04-25T11:39:10.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.627+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3015a0a0 con 0x7fac300cfb20 2024-04-25T11:39:10.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.629+0000 7fac2d600640 1 -- 172.21.15.19:0/1942204990 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac200037b0 con 0x7fac300cfb20 2024-04-25T11:39:10.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.630+0000 7fac2d600640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744b0 0x7fac140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:10.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.630+0000 7fac2d600640 1 -- 172.21.15.19:0/1942204990 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7fac20095d90 con 0x7fac300cfb20 2024-04-25T11:39:10.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.631+0000 7fac35600640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744b0 0x7fac140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:10.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.632+0000 7fac35600640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744b0 0x7fac140769c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fac24034000 tx=0x7fac2402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:10.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.634+0000 7fac2d600640 1 -- 172.21.15.19:0/1942204990 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac2005fc10 con 0x7fac300cfb20 2024-04-25T11:39:10.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.874+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac30050740 con 0x7fac300cfb20 2024-04-25T11:39:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.875+0000 7fac2d600640 1 -- 172.21.15.19:0/1942204990 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 248 v248) ==== 95+0+5178 (secure 0 0 0) 0x7fac2005f510 con 0x7fac300cfb20 2024-04-25T11:39:10.863 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:10.863 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":248,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9852,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3000889873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3000889873},{"type":"v1","addr":"172.21.15.19:6835","nonce":3000889873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":248}],"filesystems":[{"mdsmap":{"epoch":247,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:09.171149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:10.863 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 248 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744b0 msgr2=0x7fac140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744b0 0x7fac140769c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fac24034000 tx=0x7fac2402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac300cfb20 msgr2=0x7fac300b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac300cfb20 0x7fac300b5ca0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fac2000d5c0 tx=0x7fac2000da80 comp rx=0 tx=0).stop 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 shutdown_connections 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac140744b0 0x7fac140769c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac301443c0 0x7fac30158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac300cfb20 0x7fac300b5ca0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 --2- 172.21.15.19:0/1942204990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac30070370 0x7fac30159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 >> 172.21.15.19:0/1942204990 conn(0x7fac300d1ae0 msgr2=0x7fac3006d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 shutdown_connections 2024-04-25T11:39:10.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:10.878+0000 7fac38000640 1 -- 172.21.15.19:0/1942204990 wait complete. 2024-04-25T11:39:10.893 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2024-04-25T11:39:10.893 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2024-04-25T11:39:10.893 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:10.893 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:10.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:39:10.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:10.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:10.945 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:39:11.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:39:11.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.340+0000 7ff15ac00640 1 Processor -- start 2024-04-25T11:39:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.341+0000 7ff15ac00640 1 -- start start 2024-04-25T11:39:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.343+0000 7ff15ac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff15406c5c0 0x7ff1540cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.343+0000 7ff15ac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff154144370 0x7ff1540b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.343+0000 7ff15ac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1540b50e0 con 0x7ff1540b4cd0 2024-04-25T11:39:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.343+0000 7ff15ac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff1540d1230 con 0x7ff154144370 2024-04-25T11:39:11.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.343+0000 7ff15ac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff1540d06b0 con 0x7ff15406c5c0 2024-04-25T11:39:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.343+0000 7ff158e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff154144370 0x7ff1540b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff158e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff154144370 0x7ff1540b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41148/0 (socket says 172.21.15.19:41148) 2024-04-25T11:39:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff158e00640 1 -- 172.21.15.19:0/235329362 learned_addr learned my addr 172.21.15.19:0/235329362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff159800640 1 --2- 172.21.15.19:0/235329362 >> v2:172.21.15.169:3300/0 conn(0x7ff15406c5c0 0x7ff1540cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff158e00640 1 -- 172.21.15.19:0/235329362 >> v2:172.21.15.169:3300/0 conn(0x7ff15406c5c0 msgr2=0x7ff1540cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff158e00640 1 --2- 172.21.15.19:0/235329362 >> v2:172.21.15.169:3300/0 conn(0x7ff15406c5c0 0x7ff1540cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff158e00640 1 -- 172.21.15.19:0/235329362 >> v1:172.21.15.169:6789/0 conn(0x7ff1540b4cd0 legacy=0x7ff154161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff158e00640 1 -- 172.21.15.19:0/235329362 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1540d0990 con 0x7ff154144370 2024-04-25T11:39:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.344+0000 7ff159800640 1 --2- 172.21.15.19:0/235329362 >> v2:172.21.15.169:3300/0 conn(0x7ff15406c5c0 0x7ff1540cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.345+0000 7ff158e00640 1 --2- 172.21.15.19:0/235329362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff154144370 0x7ff1540b4790 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7ff1540726b0 tx=0x7ff14002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca68ca1621ca068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.346+0000 7ff153e00640 1 -- 172.21.15.19:0/235329362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140037070 con 0x7ff154144370 2024-04-25T11:39:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.346+0000 7ff153e00640 1 -- 172.21.15.19:0/235329362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140032440 con 0x7ff154144370 2024-04-25T11:39:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.346+0000 7ff153e00640 1 -- 172.21.15.19:0/235329362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140032810 con 0x7ff154144370 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 -- 172.21.15.19:0/235329362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff154144370 msgr2=0x7ff1540b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 --2- 172.21.15.19:0/235329362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff154144370 0x7ff1540b4790 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7ff1540726b0 tx=0x7ff14002a270 comp rx=0 tx=0).stop 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 -- 172.21.15.19:0/235329362 shutdown_connections 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 --2- 172.21.15.19:0/235329362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff154144370 0x7ff1540b4790 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 --2- 172.21.15.19:0/235329362 >> v2:172.21.15.169:3300/0 conn(0x7ff15406c5c0 0x7ff1540cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 -- 172.21.15.19:0/235329362 >> 172.21.15.19:0/235329362 conn(0x7ff1540d1720 msgr2=0x7ff1540d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 -- 172.21.15.19:0/235329362 shutdown_connections 2024-04-25T11:39:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.347+0000 7ff15ac00640 1 -- 172.21.15.19:0/235329362 wait complete. 2024-04-25T11:39:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.349+0000 7ff15ac00640 1 Processor -- start 2024-04-25T11:39:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.350+0000 7ff15ac00640 1 -- start start 2024-04-25T11:39:11.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff15ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff15406c5c0 0x7ff1540c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:11.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff15ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1540b4cd0 0x7ff1540cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:11.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff15ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff154144370 0x7ff1540cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:11.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff15ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1540d02d0 con 0x7ff1540b4cd0 2024-04-25T11:39:11.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff15ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1540d0f50 con 0x7ff154144370 2024-04-25T11:39:11.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff15ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1540d0450 con 0x7ff15406c5c0 2024-04-25T11:39:11.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff158e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1540b4cd0 0x7ff1540cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.351+0000 7ff158e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1540b4cd0 0x7ff1540cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41160/0 (socket says 172.21.15.19:41160) 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff158e00640 1 -- 172.21.15.19:0/3474488961 learned_addr learned my addr 172.21.15.19:0/3474488961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff159800640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff15406c5c0 0x7ff1540c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff15a200640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff154144370 0x7ff1540cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff158e00640 1 -- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff15406c5c0 msgr2=0x7ff1540c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff158e00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff15406c5c0 0x7ff1540c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff158e00640 1 -- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff154144370 msgr2=0x7ff1540cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff158e00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff154144370 0x7ff1540cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff158e00640 1 -- 172.21.15.19:0/3474488961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1541413c0 con 0x7ff1540b4cd0 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff15a200640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff154144370 0x7ff1540cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.352+0000 7ff158e00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1540b4cd0 0x7ff1540cc470 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff14002a2b0 tx=0x7ff1400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:11.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.353+0000 7ff151600640 1 -- 172.21.15.19:0/3474488961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140041020 con 0x7ff1540b4cd0 2024-04-25T11:39:11.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.353+0000 7ff151600640 1 -- 172.21.15.19:0/3474488961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400306e0 con 0x7ff1540b4cd0 2024-04-25T11:39:11.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.353+0000 7ff151600640 1 -- 172.21.15.19:0/3474488961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140037040 con 0x7ff1540b4cd0 2024-04-25T11:39:11.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.353+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1540c7780 con 0x7ff1540b4cd0 2024-04-25T11:39:11.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.353+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1541621f0 con 0x7ff1540b4cd0 2024-04-25T11:39:11.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.354+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff154050740 con 0x7ff1540b4cd0 2024-04-25T11:39:11.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.355+0000 7ff151600640 1 -- 172.21.15.19:0/3474488961 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff140044020 con 0x7ff1540b4cd0 2024-04-25T11:39:11.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.356+0000 7ff151600640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1340744b0 0x7ff1340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:11.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.356+0000 7ff151600640 1 -- 172.21.15.19:0/3474488961 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7ff1400b63b0 con 0x7ff1540b4cd0 2024-04-25T11:39:11.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.357+0000 7ff159800640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1340744b0 0x7ff1340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.358+0000 7ff159800640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1340744b0 0x7ff1340769c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff148006fd0 tx=0x7ff148008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:11.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.361+0000 7ff151600640 1 -- 172.21.15.19:0/3474488961 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff14004b030 con 0x7ff1540b4cd0 2024-04-25T11:39:11.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:11.607+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff1540c79b0 con 0x7ff1540b4cd0 2024-04-25T11:39:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.517+0000 7ff151600640 1 -- 172.21.15.19:0/3474488961 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v249) ==== 161+0+0 (secure 0 0 0) 0x7ff1400802c0 con 0x7ff1540b4cd0 2024-04-25T11:39:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1340744b0 msgr2=0x7ff1340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1340744b0 0x7ff1340769c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff148006fd0 tx=0x7ff148008040 comp rx=0 tx=0).stop 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1540b4cd0 msgr2=0x7ff1540cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1540b4cd0 0x7ff1540cc470 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff14002a2b0 tx=0x7ff1400046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 shutdown_connections 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1340744b0 0x7ff1340769c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff154144370 0x7ff1540cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1540b4cd0 0x7ff1540cc470 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 --2- 172.21.15.19:0/3474488961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff15406c5c0 0x7ff1540c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.521+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 >> 172.21.15.19:0/3474488961 conn(0x7ff1540d1720 msgr2=0x7ff15415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.522+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 shutdown_connections 2024-04-25T11:39:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.522+0000 7ff15ac00640 1 -- 172.21.15.19:0/3474488961 wait complete. 2024-04-25T11:39:12.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.834+0000 7fa781200640 1 Processor -- start 2024-04-25T11:39:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.834+0000 7fa781200640 1 -- start start 2024-04-25T11:39:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.836+0000 7fa781200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa77c1443c0 0x7fa77c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.836+0000 7fa781200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c070370 0x7fa77c0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.836+0000 7fa781200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77c0cff30 con 0x7fa77c0cfb20 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.836+0000 7fa781200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa77c0708d0 con 0x7fa77c1443c0 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.836+0000 7fa781200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c15f160 con 0x7fa77c070370 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa779e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa77c1443c0 0x7fa77c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa779e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa77c1443c0 0x7fa77c06fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41174/0 (socket says 172.21.15.19:41174) 2024-04-25T11:39:12.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa779e00640 1 -- 172.21.15.19:0/3704908126 learned_addr learned my addr 172.21.15.19:0/3704908126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa773e00640 1 --2- 172.21.15.19:0/3704908126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c070370 0x7fa77c0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa779e00640 1 -- 172.21.15.19:0/3704908126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c070370 msgr2=0x7fa77c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa779e00640 1 --2- 172.21.15.19:0/3704908126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c070370 0x7fa77c0cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa779e00640 1 -- 172.21.15.19:0/3704908126 >> v1:172.21.15.169:6789/0 conn(0x7fa77c0cfb20 legacy=0x7fa77c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa779e00640 1 -- 172.21.15.19:0/3704908126 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c15f5f0 con 0x7fa77c1443c0 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.837+0000 7fa773e00640 1 --2- 172.21.15.19:0/3704908126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c070370 0x7fa77c0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:12.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.838+0000 7fa779e00640 1 --2- 172.21.15.19:0/3704908126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c1443c0 0x7fa77c06fe30 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009870 tx=0x7fa76c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eeaef508861bce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.839+0000 7fa779400640 1 -- 172.21.15.19:0/3704908126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c037070 con 0x7fa77c1443c0 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.839+0000 7fa779400640 1 -- 172.21.15.19:0/3704908126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c032440 con 0x7fa77c1443c0 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.839+0000 7fa779400640 1 -- 172.21.15.19:0/3704908126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c032810 con 0x7fa77c1443c0 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.839+0000 7fa781200640 1 -- 172.21.15.19:0/3704908126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c1443c0 msgr2=0x7fa77c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.839+0000 7fa781200640 1 --2- 172.21.15.19:0/3704908126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c1443c0 0x7fa77c06fe30 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009870 tx=0x7fa76c02a210 comp rx=0 tx=0).stop 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.840+0000 7fa781200640 1 -- 172.21.15.19:0/3704908126 shutdown_connections 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.840+0000 7fa781200640 1 --2- 172.21.15.19:0/3704908126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c070370 0x7fa77c0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.840+0000 7fa781200640 1 --2- 172.21.15.19:0/3704908126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c1443c0 0x7fa77c06fe30 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.840+0000 7fa781200640 1 -- 172.21.15.19:0/3704908126 >> 172.21.15.19:0/3704908126 conn(0x7fa77c0d1ae0 msgr2=0x7fa77c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.840+0000 7fa781200640 1 -- 172.21.15.19:0/3704908126 shutdown_connections 2024-04-25T11:39:12.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.840+0000 7fa781200640 1 -- 172.21.15.19:0/3704908126 wait complete. 2024-04-25T11:39:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.842+0000 7fa781200640 1 Processor -- start 2024-04-25T11:39:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.842+0000 7fa781200640 1 -- start start 2024-04-25T11:39:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa781200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa77c070370 0x7fa77c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa781200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c0cfb20 0x7fa77c15b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa781200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c1443c0 0x7fa77c1f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa781200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c15edf0 con 0x7fa77c0cfb20 2024-04-25T11:39:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa781200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c15eb10 con 0x7fa77c070370 2024-04-25T11:39:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa781200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa77c06d5d0 con 0x7fa77c1443c0 2024-04-25T11:39:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa773e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c0cfb20 0x7fa77c15b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa773e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c0cfb20 0x7fa77c15b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41190/0 (socket says 172.21.15.19:41190) 2024-04-25T11:39:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa773e00640 1 -- 172.21.15.19:0/4112746711 learned_addr learned my addr 172.21.15.19:0/4112746711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa77a800640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c1443c0 0x7fa77c1f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.844+0000 7fa773e00640 1 -- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c1443c0 msgr2=0x7fa77c1f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa779e00640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa77c070370 0x7fa77c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa773e00640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c1443c0 0x7fa77c1f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa773e00640 1 -- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa77c070370 msgr2=0x7fa77c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa773e00640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa77c070370 0x7fa77c0b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa773e00640 1 -- 172.21.15.19:0/4112746711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c13c1b0 con 0x7fa77c0cfb20 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa77a800640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c1443c0 0x7fa77c1f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa773e00640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c0cfb20 0x7fa77c15b290 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c7f0 tx=0x7fa76400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa772000640 1 -- 172.21.15.19:0/4112746711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764015070 con 0x7fa77c0cfb20 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa772000640 1 -- 172.21.15.19:0/4112746711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa764007820 con 0x7fa77c0cfb20 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa772000640 1 -- 172.21.15.19:0/4112746711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764007bf0 con 0x7fa77c0cfb20 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.845+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c13d1b0 con 0x7fa77c0cfb20 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.846+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c13d770 con 0x7fa77c0cfb20 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.847+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c050740 con 0x7fa77c0cfb20 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.848+0000 7fa772000640 1 -- 172.21.15.19:0/4112746711 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa764018070 con 0x7fa77c0cfb20 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.849+0000 7fa772000640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7580744e0 0x7fa7580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.849+0000 7fa772000640 1 -- 172.21.15.19:0/4112746711 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7fa764095c50 con 0x7fa77c0cfb20 2024-04-25T11:39:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.850+0000 7fa779e00640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7580744e0 0x7fa7580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.851+0000 7fa779e00640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7580744e0 0x7fa7580769f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa76c034000 tx=0x7fa76c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:12.854+0000 7fa772000640 1 -- 172.21.15.19:0/4112746711 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa76405fad0 con 0x7fa77c0cfb20 2024-04-25T11:39:13.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.136+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa77c13cf40 con 0x7fa77c0cfb20 2024-04-25T11:39:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.531+0000 7fa772000640 1 -- 172.21.15.19:0/4112746711 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v250) ==== 101+0+0 (secure 0 0 0) 0x7fa76405f3d0 con 0x7fa77c0cfb20 2024-04-25T11:39:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:39:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7580744e0 msgr2=0x7fa7580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7580744e0 0x7fa7580769f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa76c034000 tx=0x7fa76c02e000 comp rx=0 tx=0).stop 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c0cfb20 msgr2=0x7fa77c15b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c0cfb20 0x7fa77c15b290 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c7f0 tx=0x7fa76400ccb0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 shutdown_connections 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa7580744e0 0x7fa7580769f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa77c1443c0 0x7fa77c1f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa77c0cfb20 0x7fa77c15b290 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 --2- 172.21.15.19:0/4112746711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa77c070370 0x7fa77c0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 >> 172.21.15.19:0/4112746711 conn(0x7fa77c0d1ae0 msgr2=0x7fa77c0737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 shutdown_connections 2024-04-25T11:39:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.535+0000 7fa781200640 1 -- 172.21.15.19:0/4112746711 wait complete. 2024-04-25T11:39:13.559 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.761+0000 7f98bbdd2f40 1 Processor -- start 2024-04-25T11:39:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.761+0000 7f98bbdd2f40 1 -- start start 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98bbdd2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b51b51e0 0x5577b5283bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98bbdd2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b55f0 0x5577b5202250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98bbdd2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b528edd0 0x5577b52829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98bbdd2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577b51fde70 con 0x5577b528edd0 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98bbdd2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5577b515bd20 con 0x5577b51b55f0 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98bbdd2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5577b51fe110 con 0x5577b51b51e0 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98b5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b528edd0 0x5577b52829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98b5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b528edd0 0x5577b52829b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41192/0 (socket says 172.21.15.19:41192) 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.763+0000 7f98b5a00640 1 -- 172.21.15.19:0/3237995839 learned_addr learned my addr 172.21.15.19:0/3237995839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b4600640 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b55f0 0x5577b5202250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b5000640 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b51b51e0 0x5577b5283bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b5a00640 1 -- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b51b51e0 msgr2=0x5577b5283bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b5a00640 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b51b51e0 0x5577b5283bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b5a00640 1 -- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b55f0 msgr2=0x5577b5202250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b5a00640 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b55f0 0x5577b5202250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b5a00640 1 -- 172.21.15.19:0/3237995839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b528bd80 con 0x5577b528edd0 2024-04-25T11:39:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.764+0000 7f98b5a00640 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b528edd0 0x5577b52829b0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009760 tx=0x7f98ac005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfa87e6f3f9e085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98b3200640 1 -- 172.21.15.19:0/3237995839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac019070 con 0x5577b528edd0 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98b3200640 1 -- 172.21.15.19:0/3237995839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac008890 con 0x5577b528edd0 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98b3200640 1 -- 172.21.15.19:0/3237995839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac008c80 con 0x5577b528edd0 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/3237995839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b528edd0 msgr2=0x5577b52829b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b528edd0 0x5577b52829b0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009760 tx=0x7f98ac005a00 comp rx=0 tx=0).stop 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/3237995839 shutdown_connections 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b55f0 0x5577b5202250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b528edd0 0x5577b52829b0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/3237995839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b51b51e0 0x5577b5283bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/3237995839 >> 172.21.15.19:0/3237995839 conn(0x5577b50d7ac0 msgr2=0x5577b51e74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/3237995839 shutdown_connections 2024-04-25T11:39:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.765+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/3237995839 wait complete. 2024-04-25T11:39:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.767+0000 7f98bbdd2f40 1 Processor -- start 2024-04-25T11:39:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.767+0000 7f98bbdd2f40 1 -- start start 2024-04-25T11:39:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.768+0000 7f98bbdd2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b51e0 0x5577b5284ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.768+0000 7f98bbdd2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b51b55f0 0x5577b5281fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.768+0000 7f98bbdd2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b528edd0 0x5577b5271c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.768+0000 7f98bbdd2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5577b5201dc0 con 0x5577b51b55f0 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.768+0000 7f98bbdd2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5577b5201000 con 0x5577b51b51e0 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.768+0000 7f98bbdd2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5577b52876a0 con 0x5577b528edd0 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b51b55f0 0x5577b5281fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b51b55f0 0x5577b5281fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41206/0 (socket says 172.21.15.19:41206) 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 learned_addr learned my addr 172.21.15.19:0/223524673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b5000640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b51e0 0x5577b5284ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b5a00640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b528edd0 0x5577b5271c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b528edd0 msgr2=0x5577b5271c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b528edd0 0x5577b5271c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b51e0 msgr2=0x5577b5284ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5577b51b51e0 0x5577b5284ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b5280f00 con 0x5577b51b55f0 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.769+0000 7f98b5a00640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5577b528edd0 0x5577b5271c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.770+0000 7f98b4600640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b51b55f0 0x5577b5281fa0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f98a0025ca0 tx=0x7f98a0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.770+0000 7f98b1e00640 1 -- 172.21.15.19:0/223524673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0063070 con 0x5577b51b55f0 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.770+0000 7f98b1e00640 1 -- 172.21.15.19:0/223524673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a0054b70 con 0x5577b51b55f0 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.770+0000 7f98b1e00640 1 -- 172.21.15.19:0/223524673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0068460 con 0x5577b51b55f0 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.770+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577b5285550 con 0x5577b51b55f0 2024-04-25T11:39:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.770+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b52814c0 con 0x5577b51b55f0 2024-04-25T11:39:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.773+0000 7f98b1e00640 1 -- 172.21.15.19:0/223524673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98a005c040 con 0x5577b51b55f0 2024-04-25T11:39:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.774+0000 7f98b1e00640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9894072ec0 0x7f98940753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.774+0000 7f98b1e00640 1 -- 172.21.15.19:0/223524673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7f98a00d8cd0 con 0x5577b51b55f0 2024-04-25T11:39:13.761 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.774+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577b5286fb0 0x5577b528b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.774+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b52848e0 con 0x5577b5286fb0 2024-04-25T11:39:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.775+0000 7f98b5000640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9894072ec0 0x7f98940753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.775+0000 7f98b5a00640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577b5286fb0 0x5577b528b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.776+0000 7f98b5000640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9894072ec0 0x7f98940753d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e350 tx=0x7f98a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.776+0000 7f98b5a00640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577b5286fb0 0x5577b528b660 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.780+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v117'753 uv727 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.780+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b532ec70 con 0x5577b5286fb0 2024-04-25T11:39:13.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.783+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v117'754 uv754 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.783+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532f010 con 0x5577b5286fb0 2024-04-25T11:39:13.774 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:13.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.787+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v117'755 uv755 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.787+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53303c0 con 0x5577b5286fb0 2024-04-25T11:39:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.791+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'756 uv756 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.791+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330760 con 0x5577b5286fb0 2024-04-25T11:39:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.794+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.794+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532f3b0 con 0x5577b5286fb0 2024-04-25T11:39:13.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.798+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.798+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532f7c0 con 0x5577b5286fb0 2024-04-25T11:39:13.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.803+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v117'759 uv733 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.803+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.807+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v117'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.807+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.798 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:13.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.811+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v117'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.811+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.815+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.815+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.819+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.820+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.824+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.824+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.829+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v117'765 uv740 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.829+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.834+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'766 uv766 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.834+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.825 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.838+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v117'767 uv767 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.838+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.842+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'768 uv768 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.843+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.847+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'769 uv769 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.847+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b532fb60 con 0x5577b5286fb0 2024-04-25T11:39:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.851+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'770 uv770 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.852+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.856+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'771 uv771 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.856+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.861+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v117'772 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.861+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.865+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v117'773 uv773 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.866+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.857 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:13.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.870+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v117'774 uv774 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.870+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5331b10 con 0x5577b5286fb0 2024-04-25T11:39:13.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.874+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'775 uv775 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.874+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5331b10 con 0x5577b5286fb0 2024-04-25T11:39:13.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.878+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'776 uv776 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.878+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.883+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'777 uv777 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.883+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.888+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v117'778 uv752 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.888+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.892+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v117'779 uv779 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.892+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.884 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:13.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.897+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v117'780 uv780 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.897+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5331b10 con 0x5577b5286fb0 2024-04-25T11:39:13.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.902+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'781 uv781 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.902+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5331b10 con 0x5577b5286fb0 2024-04-25T11:39:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.906+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.906+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5330b00 con 0x5577b5286fb0 2024-04-25T11:39:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.911+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.911+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577b5331010 0x5577b5333430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.912+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5333970 con 0x5577b5331010 2024-04-25T11:39:13.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.912+0000 7f98b5000640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577b5331010 0x5577b5333430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.914+0000 7f98b5000640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577b5331010 0x5577b5333430 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.919+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'549 uv531 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.919+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5333d10 con 0x5577b5331010 2024-04-25T11:39:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.923+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'550 uv550 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.924+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53340b0 con 0x5577b5331010 2024-04-25T11:39:13.914 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.928+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'551 uv551 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.928+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5334450 con 0x5577b5331010 2024-04-25T11:39:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.932+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v117'552 uv552 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.932+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53347f0 con 0x5577b5331010 2024-04-25T11:39:13.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.936+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'553 uv553 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.936+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5334b90 con 0x5577b5331010 2024-04-25T11:39:13.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.941+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v117'554 uv548 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.941+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5334f30 con 0x5577b5331010 2024-04-25T11:39:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.945+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v117'555 uv555 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.945+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.936 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.949+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v117'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.950+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.954+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.954+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.959+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.959+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.963+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v117'559 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.964+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.968+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v117'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.968+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.959 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:13.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.972+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v117'561 uv561 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.972+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53399c0 con 0x5577b5331010 2024-04-25T11:39:13.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.977+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v117'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.978+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53399c0 con 0x5577b5331010 2024-04-25T11:39:13.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.982+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.982+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.986+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v117'564 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.986+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.991+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v117'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.991+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.982 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.995+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v117'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.995+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53362e0 con 0x5577b5331010 2024-04-25T11:39:13.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:13.999+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'567 uv567 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.000+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53362e0 con 0x5577b5331010 2024-04-25T11:39:13.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.004+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.004+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53352d0 con 0x5577b5331010 2024-04-25T11:39:13.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.009+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b5331010 2024-04-25T11:39:13.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.009+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577b53357e0 0x5577b5337c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:13.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.009+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338140 con 0x5577b53357e0 2024-04-25T11:39:13.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.010+0000 7f98b4600640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577b53357e0 0x5577b5337c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:13.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.012+0000 7f98b4600640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577b53357e0 0x5577b5337c00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.016+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.016+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b53384e0 con 0x5577b53357e0 2024-04-25T11:39:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.021+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v117'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.021+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338880 con 0x5577b53357e0 2024-04-25T11:39:14.011 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.025+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v117'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.025+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b533a0e0 con 0x5577b53357e0 2024-04-25T11:39:14.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.029+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.029+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b533a480 con 0x5577b53357e0 2024-04-25T11:39:14.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.034+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.034+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53390d0 con 0x5577b53357e0 2024-04-25T11:39:14.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.038+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.038+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53394c0 con 0x5577b5286fb0 2024-04-25T11:39:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.043+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'110 uv109 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.043+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b53394c0 con 0x5577b5286fb0 2024-04-25T11:39:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.047+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.047+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53394c0 con 0x5577b5286fb0 2024-04-25T11:39:14.038 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:14.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.051+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.051+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53394c0 con 0x5577b5286fb0 2024-04-25T11:39:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.056+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.056+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577b533a820 0x5577b533cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.056+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53394c0 con 0x5577b533a820 2024-04-25T11:39:14.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.057+0000 7f98b5a00640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577b533a820 0x5577b533cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.059+0000 7f98b5a00640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577b533a820 0x5577b533cc40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.063+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.063+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b53394c0 con 0x5577b533a820 2024-04-25T11:39:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.067+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v117'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.067+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53394c0 con 0x5577b533a820 2024-04-25T11:39:14.058 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:14.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.071+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v117'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.072+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b533e190 con 0x5577b533a820 2024-04-25T11:39:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.076+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.076+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b533e190 con 0x5577b533a820 2024-04-25T11:39:14.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.080+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.080+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b533d180 con 0x5577b533a820 2024-04-25T11:39:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.084+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.084+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577b533d690 0x5577b533fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.085+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53400c0 con 0x5577b533d690 2024-04-25T11:39:14.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.085+0000 7f98b5000640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577b533d690 0x5577b533fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.087+0000 7f98b5000640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577b533d690 0x5577b533fab0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.091+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v117'271 uv265 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.091+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.096+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v117'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.096+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.086 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.100+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v117'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.100+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53414a0 con 0x5577b533d690 2024-04-25T11:39:14.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.104+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.104+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53414a0 con 0x5577b533d690 2024-04-25T11:39:14.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.108+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.108+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.112+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.112+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.116+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v117'277 uv270 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.116+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.121+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v117'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.121+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.112 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:14.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.125+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v117'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.125+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.130+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.130+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5340490 con 0x5577b533d690 2024-04-25T11:39:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.134+0000 7f98b5000640 1 -- 172.21.15.19:0/223524673 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a40146d0 con 0x5577b533d690 2024-04-25T11:39:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.134+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577b5340b10 0x5577b5342f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.134+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343670 con 0x5577b5340b10 2024-04-25T11:39:14.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.135+0000 7f98b4600640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577b5340b10 0x5577b5342f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.136+0000 7f98b4600640 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577b5340b10 0x5577b5342f50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.140+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v117'122 uv121 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.140+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5343a40 con 0x5577b5340b10 2024-04-25T11:39:14.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.144+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v117'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.144+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b5340b10 2024-04-25T11:39:14.135 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:14.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.148+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v117'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.148+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b5340b10 2024-04-25T11:39:14.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.153+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.153+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b5340b10 2024-04-25T11:39:14.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.157+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.157+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b5286fb0 2024-04-25T11:39:14.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.161+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.161+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b5286fb0 2024-04-25T11:39:14.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.165+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v117'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.165+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b5286fb0 2024-04-25T11:39:14.156 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:14.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.169+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v117'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.169+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5344e40 con 0x5577b5286fb0 2024-04-25T11:39:14.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.174+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.174+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5344e40 con 0x5577b5286fb0 2024-04-25T11:39:14.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.177+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.178+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b5286fb0 2024-04-25T11:39:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.181+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b5286fb0 2024-04-25T11:39:14.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.182+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b53357e0 2024-04-25T11:39:14.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.186+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v117'243 uv237 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.186+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b53357e0 2024-04-25T11:39:14.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.190+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v117'244 uv244 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.190+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5343e30 con 0x5577b53357e0 2024-04-25T11:39:14.181 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.194+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v117'245 uv245 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.194+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5344240 con 0x5577b53357e0 2024-04-25T11:39:14.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.198+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'246 uv246 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.198+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5344240 con 0x5577b53357e0 2024-04-25T11:39:14.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.203+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'247 uv247 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.203+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5344240 con 0x5577b53357e0 2024-04-25T11:39:14.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.207+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v117'248 uv242 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.207+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5344240 con 0x5577b53357e0 2024-04-25T11:39:14.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.211+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v117'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.211+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5344240 con 0x5577b53357e0 2024-04-25T11:39:14.202 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:14.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.216+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v117'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.216+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5344240 con 0x5577b53357e0 2024-04-25T11:39:14.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.220+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.220+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338d10 con 0x5577b53357e0 2024-04-25T11:39:14.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.225+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b53357e0 2024-04-25T11:39:14.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.225+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338cd0 con 0x5577b533a820 2024-04-25T11:39:14.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.229+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v117'107 uv106 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.229+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5338cd0 con 0x5577b533a820 2024-04-25T11:39:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.233+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'108 uv108 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.234+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338cd0 con 0x5577b533a820 2024-04-25T11:39:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.238+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v117'109 uv109 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.225 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.238+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338cd0 con 0x5577b533a820 2024-04-25T11:39:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.243+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'110 uv110 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.243+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338cb0 con 0x5577b533a820 2024-04-25T11:39:14.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.248+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v117'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.248+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5338cb0 con 0x5577b533a820 2024-04-25T11:39:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.252+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v117'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.252+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338cb0 con 0x5577b533a820 2024-04-25T11:39:14.243 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.257+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v117'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.257+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b533a820 2024-04-25T11:39:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.261+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.261+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b533a820 2024-04-25T11:39:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.266+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.266+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b533a820 2024-04-25T11:39:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.270+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98ac008a40 con 0x5577b533a820 2024-04-25T11:39:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.271+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b5340b10 2024-04-25T11:39:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.275+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v117'125 uv124 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.275+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b5340b10 2024-04-25T11:39:14.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.278+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v117'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.279+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b5340b10 2024-04-25T11:39:14.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.283+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v117'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.270 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:14.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.283+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b5340b10 2024-04-25T11:39:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.287+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.288+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c90 con 0x5577b5340b10 2024-04-25T11:39:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.291+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.291+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.296+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v117'302 uv294 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.296+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.300+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v117'303 uv303 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.300+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.305+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v117'304 uv304 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.292 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.305+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.310+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v117'305 uv305 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.310+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.314+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'306 uv306 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.314+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.319+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v117'307 uv301 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.319+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.323+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v117'308 uv308 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.323+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.314 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:14.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.327+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v117'309 uv309 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.327+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c70 con 0x5577b5340b10 2024-04-25T11:39:14.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.331+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'310 uv310 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.331+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c40 con 0x5577b5340b10 2024-04-25T11:39:14.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.336+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'311 uv311 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.336+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b5338c40 con 0x5577b5340b10 2024-04-25T11:39:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.340+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'312 uv312 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.340+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5577b53448f0 con 0x5577b5340b10 2024-04-25T11:39:14.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98b4600640 1 -- 172.21.15.19:0/223524673 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'313 uv313 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f98a009e800 con 0x5577b5340b10 2024-04-25T11:39:14.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577b5331010 msgr2=0x5577b5333430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5577b5331010 0x5577b5333430 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577b533d690 msgr2=0x5577b533fab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5577b533d690 0x5577b533fab0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577b5340b10 msgr2=0x5577b5342f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5577b5340b10 0x5577b5342f50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577b53357e0 msgr2=0x5577b5337c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5577b53357e0 0x5577b5337c00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577b5286fb0 msgr2=0x5577b528b660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5577b5286fb0 0x5577b528b660 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577b533a820 msgr2=0x5577b533cc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5577b533a820 0x5577b533cc40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 reap_dead start 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 reap_dead start 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98b5a00640 1 -- 172.21.15.19:0/223524673 reap_dead start 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9894072ec0 msgr2=0x7f98940753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.345+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9894072ec0 0x7f98940753d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f98a400e350 tx=0x7f98a4008040 comp rx=0 tx=0).stop 2024-04-25T11:39:14.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.346+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b51b55f0 msgr2=0x5577b5281fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.346+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b51b55f0 0x5577b5281fa0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f98a0025ca0 tx=0x7f98a0002fb0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.346+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 shutdown_connections 2024-04-25T11:39:14.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.346+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9894072ec0 0x7f98940753d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.346+0000 7f98bbdd2f40 1 --2- 172.21.15.19:0/223524673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5577b51b55f0 0x5577b5281fa0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.346+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 >> 172.21.15.19:0/223524673 conn(0x5577b50d7ac0 msgr2=0x5577b52829b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.346+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 shutdown_connections 2024-04-25T11:39:14.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.347+0000 7f98bbdd2f40 1 -- 172.21.15.19:0/223524673 wait complete. 2024-04-25T11:39:14.356 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2024-04-25T11:39:14.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.609+0000 7f206073af40 1 Processor -- start 2024-04-25T11:39:14.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.609+0000 7f206073af40 1 -- start start 2024-04-25T11:39:14.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.610+0000 7f206073af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2040003c40 0x5587dd2ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.610+0000 7f206073af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 0x5587dd38f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.610+0000 7f206073af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd388600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.610+0000 7f206073af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5587dd25fd20 con 0x5587dd2e7730 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.610+0000 7f206073af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5587dd263d60 con 0x7f2040003c40 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.610+0000 7f206073af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5587dd3023f0 con 0x5587dd3007d0 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd388600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd388600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52894/0 (socket says 172.21.15.19:52894) 2024-04-25T11:39:14.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 -- 172.21.15.19:0/4288894119 learned_addr learned my addr 172.21.15.19:0/4288894119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:14.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 -- 172.21.15.19:0/4288894119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 msgr2=0x5587dd38f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f2058e00640 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 0x5587dd38f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 0x5587dd38f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 -- 172.21.15.19:0/4288894119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2040003c40 msgr2=0x5587dd2ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2040003c40 0x5587dd2ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.611+0000 7f205a200640 1 -- 172.21.15.19:0/4288894119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587dd389770 con 0x5587dd2e7730 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.612+0000 7f205a200640 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd388600 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f2054046240 tx=0x7f2054052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957900f1a0783f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.612+0000 7f2053400640 1 -- 172.21.15.19:0/4288894119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054068070 con 0x5587dd2e7730 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.612+0000 7f2053400640 1 -- 172.21.15.19:0/4288894119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205404e7b0 con 0x5587dd2e7730 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.612+0000 7f2053400640 1 -- 172.21.15.19:0/4288894119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205404eba0 con 0x5587dd2e7730 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 -- 172.21.15.19:0/4288894119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 msgr2=0x5587dd388600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd388600 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f2054046240 tx=0x7f2054052000 comp rx=0 tx=0).stop 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 -- 172.21.15.19:0/4288894119 shutdown_connections 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd388600 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 0x5587dd38f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 --2- 172.21.15.19:0/4288894119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2040003c40 0x5587dd2ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 -- 172.21.15.19:0/4288894119 >> 172.21.15.19:0/4288894119 conn(0x5587dd1dbac0 msgr2=0x5587dd38db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 -- 172.21.15.19:0/4288894119 shutdown_connections 2024-04-25T11:39:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.613+0000 7f206073af40 1 -- 172.21.15.19:0/4288894119 wait complete. 2024-04-25T11:39:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.614+0000 7f206073af40 1 Processor -- start 2024-04-25T11:39:14.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.614+0000 7f206073af40 1 -- start start 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.615+0000 7f206073af40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd38ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.615+0000 7f206073af40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 0x5587dd3904a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.615+0000 7f206073af40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2040003c40 0x5587dd38b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.615+0000 7f206073af40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5587dd38e290 con 0x5587dd2e7730 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.615+0000 7f206073af40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5587dd38fde0 con 0x7f2040003c40 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f206073af40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5587dd389000 con 0x5587dd3007d0 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd38ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd38ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52900/0 (socket says 172.21.15.19:52900) 2024-04-25T11:39:14.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 -- 172.21.15.19:0/720952441 learned_addr learned my addr 172.21.15.19:0/720952441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2058e00640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 0x5587dd3904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 -- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 msgr2=0x5587dd3904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587dd3007d0 0x5587dd3904a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 -- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2040003c40 msgr2=0x5587dd38b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2040003c40 0x5587dd38b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.616+0000 7f2059800640 1 -- 172.21.15.19:0/720952441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587dd42aad0 con 0x5587dd2e7730 2024-04-25T11:39:14.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.617+0000 7f2059800640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd38ff60 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f204c001c60 tx=0x7f204c00a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.617+0000 7f2052000640 1 -- 172.21.15.19:0/720952441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c012070 con 0x5587dd2e7730 2024-04-25T11:39:14.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.617+0000 7f2052000640 1 -- 172.21.15.19:0/720952441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c00e680 con 0x5587dd2e7730 2024-04-25T11:39:14.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.617+0000 7f2052000640 1 -- 172.21.15.19:0/720952441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c00ea70 con 0x5587dd2e7730 2024-04-25T11:39:14.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.617+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5587dd38cd80 con 0x5587dd2e7730 2024-04-25T11:39:14.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.617+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587dd38e860 con 0x5587dd2e7730 2024-04-25T11:39:14.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.619+0000 7f2052000640 1 -- 172.21.15.19:0/720952441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f204c009070 con 0x5587dd2e7730 2024-04-25T11:39:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.620+0000 7f2052000640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f203c072e90 0x7f203c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.620+0000 7f2052000640 1 -- 172.21.15.19:0/720952441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7f204c094b40 con 0x5587dd2e7730 2024-04-25T11:39:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.621+0000 7f206073af40 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5587dd38bb90 0x5587dd38bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.621+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:1 12.14 12:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5587dd2e9000 con 0x5587dd38bb90 2024-04-25T11:39:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.621+0000 7f2058e00640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f203c072e90 0x7f203c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.621+0000 7f205a200640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5587dd38bb90 0x5587dd38bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.622+0000 7f205a200640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5587dd38bb90 0x5587dd38bfa0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.622+0000 7f2058e00640 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f203c072e90 0x7f203c0753a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x5587dd38a4a0 tx=0x7f20480024f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f205a200640 1 -- 172.21.15.19:0/720952441 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v117'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f205406b040 con 0x5587dd38bb90 2024-04-25T11:39:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5587dd38bb90 msgr2=0x5587dd38bfa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5587dd38bb90 0x5587dd38bfa0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f203c072e90 msgr2=0x7f203c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f203c072e90 0x7f203c0753a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x5587dd38a4a0 tx=0x7f20480024f0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 msgr2=0x5587dd38ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 --2- 172.21.15.19:0/720952441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587dd2e7730 0x5587dd38ff60 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f204c001c60 tx=0x7f204c00a6b0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f205a200640 1 -- 172.21.15.19:0/720952441 reap_dead start 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 shutdown_connections 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.625+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 >> 172.21.15.19:0/720952441 conn(0x5587dd1dbac0 msgr2=0x5587dd2eb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.626+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 shutdown_connections 2024-04-25T11:39:14.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.626+0000 7f206073af40 1 -- 172.21.15.19:0/720952441 wait complete. 2024-04-25T11:39:14.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.908+0000 7f1ec5a00640 1 Processor -- start 2024-04-25T11:39:14.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.908+0000 7f1ec5a00640 1 -- start start 2024-04-25T11:39:14.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.910+0000 7f1ec5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec0164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.910+0000 7f1ec5a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec0164fc0 con 0x7f1ec006dce0 2024-04-25T11:39:14.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.910+0000 7f1ec5a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec006f440 con 0x7f1ec0160390 2024-04-25T11:39:14.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.910+0000 7f1ec5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00afb20 con 0x7f1ec00af710 2024-04-25T11:39:14.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.911+0000 7f1ebf400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ec006dce0 0x7f1ec0070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46136/0 (socket says 172.21.15.19:46136) 2024-04-25T11:39:14.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.911+0000 7f1ebf400640 1 -- 172.21.15.19:0/2578857348 learned_addr learned my addr 172.21.15.19:0/2578857348 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:14.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.911+0000 7f1ebea00640 1 --2- 172.21.15.19:0/2578857348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec0164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.911+0000 7f1ebe000640 1 -- 172.21.15.19:0/2578857348 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1773490972 0 0) 0x7f1ec0164fc0 con 0x7f1ec006dce0 2024-04-25T11:39:14.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.911+0000 7f1ebe000640 1 -- 172.21.15.19:0/2578857348 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ea4003520 con 0x7f1ec006dce0 2024-04-25T11:39:14.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.912+0000 7f1ebea00640 1 -- 172.21.15.19:0/2578857348 >> v1:172.21.15.169:6789/0 conn(0x7f1ec0160390 legacy=0x7f1ec006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.912+0000 7f1ebea00640 1 -- 172.21.15.19:0/2578857348 >> v1:172.21.15.19:6789/0 conn(0x7f1ec006dce0 legacy=0x7f1ec0070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.912+0000 7f1ebe000640 1 -- 172.21.15.19:0/2578857348 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 492130897 0 0) 0x7f1ea4003520 con 0x7f1ec006dce0 2024-04-25T11:39:14.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.912+0000 7f1ebea00640 1 -- 172.21.15.19:0/2578857348 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec006f720 con 0x7f1ec00af710 2024-04-25T11:39:14.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.913+0000 7f1ebea00640 1 --2- 172.21.15.19:0/2578857348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec0164a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00afca0 tx=0x7f1eb002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734d60576cac5e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.914+0000 7f1ebe000640 1 -- 172.21.15.19:0/2578857348 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0037070 con 0x7f1ec00af710 2024-04-25T11:39:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.914+0000 7f1ebe000640 1 -- 172.21.15.19:0/2578857348 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb002ab90 con 0x7f1ec00af710 2024-04-25T11:39:14.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.914+0000 7f1ebe000640 1 -- 172.21.15.19:0/2578857348 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8001a10 con 0x7f1ec00af710 2024-04-25T11:39:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.915+0000 7f1ec5a00640 1 -- 172.21.15.19:0/2578857348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 msgr2=0x7f1ec0164a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.915+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/2578857348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec0164a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00afca0 tx=0x7f1eb002a100 comp rx=0 tx=0).stop 2024-04-25T11:39:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.915+0000 7f1ec5a00640 1 -- 172.21.15.19:0/2578857348 shutdown_connections 2024-04-25T11:39:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.915+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/2578857348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec0164a80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.915+0000 7f1ec5a00640 1 -- 172.21.15.19:0/2578857348 >> 172.21.15.19:0/2578857348 conn(0x7f1ec0073090 msgr2=0x7f1ec00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.915+0000 7f1ec5a00640 1 -- 172.21.15.19:0/2578857348 shutdown_connections 2024-04-25T11:39:14.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.915+0000 7f1ec5a00640 1 -- 172.21.15.19:0/2578857348 wait complete. 2024-04-25T11:39:14.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.917+0000 7f1ec5a00640 1 Processor -- start 2024-04-25T11:39:14.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.918+0000 7f1ec5a00640 1 -- start start 2024-04-25T11:39:14.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1ec5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1ec5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec0160390 0x7f1ec01f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1ec5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec015baa0 0x7f1ec01f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1ec5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec006f0d0 con 0x7f1ec0160390 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1ec5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec006ffc0 con 0x7f1ec015baa0 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1ec5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00b1110 con 0x7f1ec00af710 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1eb7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec0160390 0x7f1ec01f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.919+0000 7f1eb7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec0160390 0x7f1ec01f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52906/0 (socket says 172.21.15.19:52906) 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1eb7e00640 1 -- 172.21.15.19:0/4287427685 learned_addr learned my addr 172.21.15.19:0/4287427685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1ebea00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1ebf400640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec015baa0 0x7f1ec01f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1eb7e00640 1 -- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 msgr2=0x7f1ec006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1eb7e00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec006d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1eb7e00640 1 -- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec015baa0 msgr2=0x7f1ec01f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:14.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1eb7e00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec015baa0 0x7f1ec01f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1eb7e00640 1 -- 172.21.15.19:0/4287427685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec015aa00 con 0x7f1ec0160390 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.920+0000 7f1eb7e00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec0160390 0x7f1ec01f63b0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c3f0 tx=0x7f1ea800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.921+0000 7f1eb6a00640 1 -- 172.21.15.19:0/4287427685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8017070 con 0x7f1ec0160390 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.921+0000 7f1eb6a00640 1 -- 172.21.15.19:0/4287427685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea80052d0 con 0x7f1ec0160390 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.921+0000 7f1eb6a00640 1 -- 172.21.15.19:0/4287427685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8003830 con 0x7f1ec0160390 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.921+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec015ace0 con 0x7f1ec0160390 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.921+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec015b2a0 con 0x7f1ec0160390 2024-04-25T11:39:14.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.922+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec015a770 con 0x7f1ec0160390 2024-04-25T11:39:14.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.923+0000 7f1eb6a00640 1 -- 172.21.15.19:0/4287427685 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ea801a070 con 0x7f1ec0160390 2024-04-25T11:39:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.924+0000 7f1eb6a00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e9c0744b0 0x7f1e9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:14.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.924+0000 7f1eb6a00640 1 -- 172.21.15.19:0/4287427685 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7859+0+0 (secure 0 0 0) 0x7f1ea8094cc0 con 0x7f1ec0160390 2024-04-25T11:39:14.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.925+0000 7f1ebea00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e9c0744b0 0x7f1e9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:14.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.926+0000 7f1ebea00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e9c0744b0 0x7f1e9c0769c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0034000 tx=0x7f1eb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:14.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:14.930+0000 7f1eb6a00640 1 -- 172.21.15.19:0/4287427685 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ea809b080 con 0x7f1ec0160390 2024-04-25T11:39:15.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.183+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1ec0050740 con 0x7f1ec0160390 2024-04-25T11:39:15.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.547+0000 7f1eb6a00640 1 -- 172.21.15.19:0/4287427685 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v251) ==== 162+0+0 (secure 0 0 0) 0x7f1ea805ec60 con 0x7f1ec0160390 2024-04-25T11:39:15.536 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:15.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e9c0744b0 msgr2=0x7f1e9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e9c0744b0 0x7f1e9c0769c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0034000 tx=0x7f1eb002e000 comp rx=0 tx=0).stop 2024-04-25T11:39:15.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec0160390 msgr2=0x7f1ec01f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec0160390 0x7f1ec01f63b0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800c3f0 tx=0x7f1ea800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 shutdown_connections 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e9c0744b0 0x7f1e9c0769c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ec015baa0 0x7f1ec01f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ec0160390 0x7f1ec01f63b0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 --2- 172.21.15.19:0/4287427685 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ec00af710 0x7f1ec006d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 >> 172.21.15.19:0/4287427685 conn(0x7f1ec0073090 msgr2=0x7f1ec00b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 shutdown_connections 2024-04-25T11:39:15.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.551+0000 7f1ec5a00640 1 -- 172.21.15.19:0/4287427685 wait complete. 2024-04-25T11:39:15.561 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:15.573+0000 7f6d4fe00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:39:15.562 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:15.574+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:15.562 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:15.574+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:39:15.573 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:39:15.573+0000 7f6d4fe00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-04-25T11:39:15.582 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:15.705 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.850+0000 7f7e3f400640 1 Processor -- start 2024-04-25T11:39:15.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.850+0000 7f7e3f400640 1 -- start start 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e381400a0 0x7f7e380b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38142730 0x7f7e38142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7e38070270 0x7f7e38070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3f400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7e38070bc0 con 0x7f7e381400a0 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3f400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7e381431d0 con 0x7f7e38070270 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e381434b0 con 0x7f7e38142730 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e381400a0 0x7f7e380b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e381400a0 0x7f7e380b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52938/0 (socket says 172.21.15.19:52938) 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.853+0000 7f7e3ca00640 1 -- 172.21.15.19:0/4108077709 learned_addr learned my addr 172.21.15.19:0/4108077709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.854+0000 7f7e3d400640 1 --2- 172.21.15.19:0/4108077709 >> v2:172.21.15.169:3300/0 conn(0x7f7e38070270 0x7f7e38070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.854+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/4108077709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38142730 0x7f7e38142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.854+0000 7f7e3ca00640 1 -- 172.21.15.19:0/4108077709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38142730 msgr2=0x7f7e38142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.854+0000 7f7e3ca00640 1 --2- 172.21.15.19:0/4108077709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38142730 0x7f7e38142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.854+0000 7f7e3ca00640 1 -- 172.21.15.19:0/4108077709 >> v2:172.21.15.169:3300/0 conn(0x7f7e38070270 msgr2=0x7f7e38070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.854+0000 7f7e3ca00640 1 --2- 172.21.15.19:0/4108077709 >> v2:172.21.15.169:3300/0 conn(0x7f7e38070270 0x7f7e38070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.854+0000 7f7e3ca00640 1 -- 172.21.15.19:0/4108077709 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e380b6090 con 0x7f7e381400a0 2024-04-25T11:39:15.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.855+0000 7f7e3ca00640 1 --2- 172.21.15.19:0/4108077709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e380b6890 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f7e3815d650 tx=0x7f7e2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df300d6f1ac76628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e37e00640 1 -- 172.21.15.19:0/4108077709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28037070 con 0x7f7e381400a0 2024-04-25T11:39:15.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e37e00640 1 -- 172.21.15.19:0/4108077709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e28032440 con 0x7f7e381400a0 2024-04-25T11:39:15.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e37e00640 1 -- 172.21.15.19:0/4108077709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28032810 con 0x7f7e381400a0 2024-04-25T11:39:15.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e3f400640 1 -- 172.21.15.19:0/4108077709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 msgr2=0x7f7e380b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4108077709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e380b6890 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f7e3815d650 tx=0x7f7e2802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e3f400640 1 -- 172.21.15.19:0/4108077709 shutdown_connections 2024-04-25T11:39:15.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4108077709 >> v2:172.21.15.169:3300/0 conn(0x7f7e38070270 0x7f7e38070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4108077709 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38142730 0x7f7e38142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4108077709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e380b6890 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.856+0000 7f7e3f400640 1 -- 172.21.15.19:0/4108077709 >> 172.21.15.19:0/4108077709 conn(0x7f7e38073050 msgr2=0x7f7e38073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.857+0000 7f7e3f400640 1 -- 172.21.15.19:0/4108077709 shutdown_connections 2024-04-25T11:39:15.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.857+0000 7f7e3f400640 1 -- 172.21.15.19:0/4108077709 wait complete. 2024-04-25T11:39:15.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.859+0000 7f7e3f400640 1 Processor -- start 2024-04-25T11:39:15.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.859+0000 7f7e3f400640 1 -- start start 2024-04-25T11:39:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3f400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38070270 0x7f7e3813e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e38157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3f400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38142730 0x7f7e38159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3f400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380cddb0 con 0x7f7e381400a0 2024-04-25T11:39:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3f400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7e380b6dd0 con 0x7f7e38142730 2024-04-25T11:39:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3f400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7e380b49a0 con 0x7f7e38070270 2024-04-25T11:39:15.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e38157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e38157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52952/0 (socket says 172.21.15.19:52952) 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e2fe00640 1 -- 172.21.15.19:0/4056076205 learned_addr learned my addr 172.21.15.19:0/4056076205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3d400640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38142730 0x7f7e38159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e3ca00640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38070270 0x7f7e3813e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e2fe00640 1 -- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38070270 msgr2=0x7f7e3813e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38070270 0x7f7e3813e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e2fe00640 1 -- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38142730 msgr2=0x7f7e38159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.861+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38142730 0x7f7e38159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.862+0000 7f7e2fe00640 1 -- 172.21.15.19:0/4056076205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e381581c0 con 0x7f7e381400a0 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.862+0000 7f7e2fe00640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e38157c80 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000d5f0 tx=0x7f7e2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.862+0000 7f7e35600640 1 -- 172.21.15.19:0/4056076205 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200125f0 con 0x7f7e381400a0 2024-04-25T11:39:15.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.862+0000 7f7e35600640 1 -- 172.21.15.19:0/4056076205 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e20019070 con 0x7f7e381400a0 2024-04-25T11:39:15.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.862+0000 7f7e35600640 1 -- 172.21.15.19:0/4056076205 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200129d0 con 0x7f7e381400a0 2024-04-25T11:39:15.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.862+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3815fd00 con 0x7f7e381400a0 2024-04-25T11:39:15.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.862+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3815ffe0 con 0x7f7e381400a0 2024-04-25T11:39:15.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.863+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e38142e70 con 0x7f7e381400a0 2024-04-25T11:39:15.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.865+0000 7f7e35600640 1 -- 172.21.15.19:0/4056076205 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e20012b70 con 0x7f7e381400a0 2024-04-25T11:39:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.866+0000 7f7e35600640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e140744e0 0x7f7e140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.866+0000 7f7e35600640 1 -- 172.21.15.19:0/4056076205 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f7e20095f50 con 0x7f7e381400a0 2024-04-25T11:39:15.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.867+0000 7f7e3ca00640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e140744e0 0x7f7e140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:15.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.868+0000 7f7e3ca00640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e140744e0 0x7f7e140769f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7e28034000 tx=0x7f7e2802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:15.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:15.871+0000 7f7e35600640 1 -- 172.21.15.19:0/4056076205 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e38142e70 con 0x7f7e381400a0 2024-04-25T11:39:16.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.120+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e38158440 con 0x7f7e381400a0 2024-04-25T11:39:16.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.121+0000 7f7e35600640 1 -- 172.21.15.19:0/4056076205 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 253 v253) ==== 95+0+4352 (secure 0 0 0) 0x7f7e2005fd80 con 0x7f7e381400a0 2024-04-25T11:39:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:16.110 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":253,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:15.575835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:16.110 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 253 2024-04-25T11:39:16.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e140744e0 msgr2=0x7f7e140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e140744e0 0x7f7e140769f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7e28034000 tx=0x7f7e2802e000 comp rx=0 tx=0).stop 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 msgr2=0x7f7e38157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e38157c80 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000d5f0 tx=0x7f7e2000dab0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 shutdown_connections 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e140744e0 0x7f7e140769f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38142730 0x7f7e38159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e381400a0 0x7f7e38157c80 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 --2- 172.21.15.19:0/4056076205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e38070270 0x7f7e3813e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.124+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 >> 172.21.15.19:0/4056076205 conn(0x7f7e38073050 msgr2=0x7f7e380b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.125+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 shutdown_connections 2024-04-25T11:39:16.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.125+0000 7f7e3f400640 1 -- 172.21.15.19:0/4056076205 wait complete. 2024-04-25T11:39:16.139 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:16.139 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:16.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.430+0000 7ff262800640 1 Processor -- start 2024-04-25T11:39:16.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.430+0000 7ff262800640 1 -- start start 2024-04-25T11:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff262800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff25c0cc5c0 0x7ff25c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff262800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff262800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff25c0b7a20 con 0x7ff25c06db50 2024-04-25T11:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff262800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff25c0b1e20 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff262800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c0b2100 con 0x7ff25c0b4100 2024-04-25T11:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff260a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff25c06db50 0x7ff25c0b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46154/0 (socket says 172.21.15.19:46154) 2024-04-25T11:39:16.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff260a00640 1 -- 172.21.15.19:0/971081985 learned_addr learned my addr 172.21.15.19:0/971081985 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:16.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.433+0000 7ff261400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff25c0cc5c0 0x7ff25c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.434+0000 7ff261e00640 1 --2- 172.21.15.19:0/971081985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.434+0000 7ff25be00640 1 -- 172.21.15.19:0/971081985 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804336657 0 0) 0x7ff25c0b7a20 con 0x7ff25c06db50 2024-04-25T11:39:16.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.434+0000 7ff25be00640 1 -- 172.21.15.19:0/971081985 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff244005730 con 0x7ff25c06db50 2024-04-25T11:39:16.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.434+0000 7ff261400640 1 -- 172.21.15.19:0/971081985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 msgr2=0x7ff25c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:16.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.434+0000 7ff261400640 1 --2- 172.21.15.19:0/971081985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.434+0000 7ff261400640 1 -- 172.21.15.19:0/971081985 >> v1:172.21.15.19:6789/0 conn(0x7ff25c06db50 legacy=0x7ff25c0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.434+0000 7ff261400640 1 -- 172.21.15.19:0/971081985 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c0b75f0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.436+0000 7ff261400640 1 --2- 172.21.15.19:0/971081985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c0ce3f0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7ff250009d90 tx=0x7ff25002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419260ce5bdaaf67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.436+0000 7ff25be00640 1 -- 172.21.15.19:0/971081985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007740 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.436+0000 7ff25be00640 1 -- 172.21.15.19:0/971081985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2500078e0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.436+0000 7ff25be00640 1 -- 172.21.15.19:0/971081985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007c90 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 -- 172.21.15.19:0/971081985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 msgr2=0x7ff25c0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 --2- 172.21.15.19:0/971081985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c0ce3f0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7ff250009d90 tx=0x7ff25002a660 comp rx=0 tx=0).stop 2024-04-25T11:39:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 -- 172.21.15.19:0/971081985 shutdown_connections 2024-04-25T11:39:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 --2- 172.21.15.19:0/971081985 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c06c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 --2- 172.21.15.19:0/971081985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c0ce3f0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 -- 172.21.15.19:0/971081985 >> 172.21.15.19:0/971081985 conn(0x7ff25c09b7d0 msgr2=0x7ff25c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 -- 172.21.15.19:0/971081985 shutdown_connections 2024-04-25T11:39:16.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.437+0000 7ff262800640 1 -- 172.21.15.19:0/971081985 wait complete. 2024-04-25T11:39:16.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.440+0000 7ff262800640 1 Processor -- start 2024-04-25T11:39:16.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.440+0000 7ff262800640 1 -- start start 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.441+0000 7ff262800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c06db50 0x7ff25c0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff262800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff262800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff262800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0b77c0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff262800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0b2860 con 0x7ff25c06db50 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff262800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c06cb00 con 0x7ff25c0b4100 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52958/0 (socket says 172.21.15.19:52958) 2024-04-25T11:39:16.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 -- 172.21.15.19:0/1379772808 learned_addr learned my addr 172.21.15.19:0/1379772808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff260a00640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261400640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c06db50 0x7ff25c0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 -- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 msgr2=0x7ff25c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 -- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c06db50 msgr2=0x7ff25c0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c06db50 0x7ff25c0b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff261e00640 1 -- 172.21.15.19:0/1379772808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c09d330 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.442+0000 7ff260a00640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.443+0000 7ff261e00640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c09e5d0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7ff25400d5d0 tx=0x7ff25400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.443+0000 7ff259600640 1 -- 172.21.15.19:0/1379772808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540125f0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.443+0000 7ff259600640 1 -- 172.21.15.19:0/1379772808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff254019070 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.443+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c09d610 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.443+0000 7ff259600640 1 -- 172.21.15.19:0/1379772808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540129f0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.443+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c09dbd0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.445+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25c0b36d0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.446+0000 7ff259600640 1 -- 172.21.15.19:0/1379772808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff254012b90 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.447+0000 7ff259600640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff23c0744e0 0x7ff23c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.447+0000 7ff259600640 1 -- 172.21.15.19:0/1379772808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7ff254095e10 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.448+0000 7ff261400640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff23c0744e0 0x7ff23c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.449+0000 7ff261400640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff23c0744e0 0x7ff23c0769f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff250034000 tx=0x7ff25002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.452+0000 7ff259600640 1 -- 172.21.15.19:0/1379772808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff25c0b36d0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.687+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff25c09d0c0 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.688+0000 7ff259600640 1 -- 172.21.15.19:0/1379772808 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 254 v254) ==== 95+0+5178 (secure 0 0 0) 0x7ff25405fd80 con 0x7ff25c0cc5c0 2024-04-25T11:39:16.676 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:16.676 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":254,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3564168877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3564168877},{"type":"v1","addr":"172.21.15.19:6835","nonce":3564168877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:15.575835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:16.676 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 254 2024-04-25T11:39:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff23c0744e0 msgr2=0x7ff23c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff23c0744e0 0x7ff23c0769f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff250034000 tx=0x7ff25002f000 comp rx=0 tx=0).stop 2024-04-25T11:39:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 msgr2=0x7ff25c09e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c09e5d0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7ff25400d5d0 tx=0x7ff25400da90 comp rx=0 tx=0).stop 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 shutdown_connections 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff23c0744e0 0x7ff23c0769f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff25c0cc5c0 0x7ff25c09e5d0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff25c0b4100 0x7ff25c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 --2- 172.21.15.19:0/1379772808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff25c06db50 0x7ff25c0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 >> 172.21.15.19:0/1379772808 conn(0x7ff25c09b7d0 msgr2=0x7ff25c0702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.690+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 shutdown_connections 2024-04-25T11:39:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.691+0000 7ff262800640 1 -- 172.21.15.19:0/1379772808 wait complete. 2024-04-25T11:39:16.707 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:16.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.983+0000 7f2410e00640 1 Processor -- start 2024-04-25T11:39:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.983+0000 7f2410e00640 1 -- start start 2024-04-25T11:39:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.985+0000 7f2410e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f240c1443c0 0x7f240c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.985+0000 7f2410e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c070370 0x7f240c0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.986+0000 7f2410e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f240c0cff30 con 0x7f240c0cfb20 2024-04-25T11:39:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.986+0000 7f2410e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f240c0708d0 con 0x7f240c1443c0 2024-04-25T11:39:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.986+0000 7f2410e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f240c15f160 con 0x7f240c070370 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.986+0000 7f2409e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f240c1443c0 0x7f240c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.986+0000 7f2409e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f240c1443c0 0x7f240c06fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52964/0 (socket says 172.21.15.19:52964) 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.986+0000 7f2409e00640 1 -- 172.21.15.19:0/3285974824 learned_addr learned my addr 172.21.15.19:0/3285974824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.986+0000 7f2403e00640 1 --2- 172.21.15.19:0/3285974824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c070370 0x7f240c0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.987+0000 7f2409e00640 1 -- 172.21.15.19:0/3285974824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c070370 msgr2=0x7f240c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.987+0000 7f2409e00640 1 --2- 172.21.15.19:0/3285974824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c070370 0x7f240c0cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.987+0000 7f2409e00640 1 -- 172.21.15.19:0/3285974824 >> v1:172.21.15.169:6789/0 conn(0x7f240c0cfb20 legacy=0x7f240c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.987+0000 7f2409e00640 1 -- 172.21.15.19:0/3285974824 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c15f5f0 con 0x7f240c1443c0 2024-04-25T11:39:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.988+0000 7f2409e00640 1 --2- 172.21.15.19:0/3285974824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c06fe30 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009870 tx=0x7f23fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95f75b3eed6f3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.989+0000 7f2409400640 1 -- 172.21.15.19:0/3285974824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc037070 con 0x7f240c1443c0 2024-04-25T11:39:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.989+0000 7f2409400640 1 -- 172.21.15.19:0/3285974824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23fc032440 con 0x7f240c1443c0 2024-04-25T11:39:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.989+0000 7f2409400640 1 -- 172.21.15.19:0/3285974824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc032810 con 0x7f240c1443c0 2024-04-25T11:39:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 -- 172.21.15.19:0/3285974824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 msgr2=0x7f240c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 --2- 172.21.15.19:0/3285974824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c06fe30 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009870 tx=0x7f23fc02a270 comp rx=0 tx=0).stop 2024-04-25T11:39:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 -- 172.21.15.19:0/3285974824 shutdown_connections 2024-04-25T11:39:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 --2- 172.21.15.19:0/3285974824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c070370 0x7f240c0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 --2- 172.21.15.19:0/3285974824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c06fe30 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 -- 172.21.15.19:0/3285974824 >> 172.21.15.19:0/3285974824 conn(0x7f240c0d1ae0 msgr2=0x7f240c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 -- 172.21.15.19:0/3285974824 shutdown_connections 2024-04-25T11:39:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.990+0000 7f2410e00640 1 -- 172.21.15.19:0/3285974824 wait complete. 2024-04-25T11:39:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.993+0000 7f2410e00640 1 Processor -- start 2024-04-25T11:39:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.994+0000 7f2410e00640 1 -- start start 2024-04-25T11:39:16.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f2410e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f240c070370 0x7f240c0b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f2410e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c0cfb20 0x7f240c0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f2410e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f2410e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f240c15edf0 con 0x7f240c1443c0 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f2410e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f240c15eb10 con 0x7f240c070370 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f2410e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f240c06d5d0 con 0x7f240c0cfb20 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f240a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f240a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c13de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52974/0 (socket says 172.21.15.19:52974) 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f240a800640 1 -- 172.21.15.19:0/1220726406 learned_addr learned my addr 172.21.15.19:0/1220726406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.996+0000 7f2409e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f240c070370 0x7f240c0b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f2403e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c0cfb20 0x7f240c0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f240a800640 1 -- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c0cfb20 msgr2=0x7f240c0b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f240a800640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c0cfb20 0x7f240c0b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f240a800640 1 -- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f240c070370 msgr2=0x7f240c0b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:16.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f240a800640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f240c070370 0x7f240c0b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:16.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f240a800640 1 -- 172.21.15.19:0/1220726406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c13a290 con 0x7f240c1443c0 2024-04-25T11:39:16.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f2409e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f240c070370 0x7f240c0b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.997+0000 7f240a800640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c13de30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f240400c6b0 tx=0x7f240400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.998+0000 7f2402000640 1 -- 172.21.15.19:0/1220726406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404003b70 con 0x7f240c1443c0 2024-04-25T11:39:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.998+0000 7f2402000640 1 -- 172.21.15.19:0/1220726406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2404003d30 con 0x7f240c1443c0 2024-04-25T11:39:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.998+0000 7f2402000640 1 -- 172.21.15.19:0/1220726406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404005310 con 0x7f240c1443c0 2024-04-25T11:39:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.998+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c0b5cc0 con 0x7f240c1443c0 2024-04-25T11:39:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.998+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c0b6280 con 0x7f240c1443c0 2024-04-25T11:39:16.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:16.999+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c13a000 con 0x7f240c1443c0 2024-04-25T11:39:16.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.001+0000 7f2402000640 1 -- 172.21.15.19:0/1220726406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f24040054b0 con 0x7f240c1443c0 2024-04-25T11:39:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.002+0000 7f2402000640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23e80744b0 0x7f23e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.002+0000 7f2402000640 1 -- 172.21.15.19:0/1220726406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f2404095950 con 0x7f240c1443c0 2024-04-25T11:39:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.002+0000 7f2409e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23e80744b0 0x7f23e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.003+0000 7f2409e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23e80744b0 0x7f23e80769c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f23fc034000 tx=0x7f23fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:16.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.007+0000 7f2402000640 1 -- 172.21.15.19:0/1220726406 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f240c0cff30 con 0x7f240c1443c0 2024-04-25T11:39:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.253+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f240c074310 con 0x7f240c1443c0 2024-04-25T11:39:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.254+0000 7f2402000640 1 -- 172.21.15.19:0/1220726406 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 254 v254) ==== 95+0+5178 (secure 0 0 0) 0x7f240c074310 con 0x7f240c1443c0 2024-04-25T11:39:17.242 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:17.242 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":254,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9912,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3564168877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3564168877},{"type":"v1","addr":"172.21.15.19:6835","nonce":3564168877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:15.575835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 254 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23e80744b0 msgr2=0x7f23e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23e80744b0 0x7f23e80769c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f23fc034000 tx=0x7f23fc02e000 comp rx=0 tx=0).stop 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 msgr2=0x7f240c13de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c13de30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f240400c6b0 tx=0x7f240400cb70 comp rx=0 tx=0).stop 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 shutdown_connections 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f23e80744b0 0x7f23e80769c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f240c1443c0 0x7f240c13de30 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f240c0cfb20 0x7f240c0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 --2- 172.21.15.19:0/1220726406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f240c070370 0x7f240c0b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 >> 172.21.15.19:0/1220726406 conn(0x7f240c0d1ae0 msgr2=0x7f240c140280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 shutdown_connections 2024-04-25T11:39:17.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.257+0000 7f2410e00640 1 -- 172.21.15.19:0/1220726406 wait complete. 2024-04-25T11:39:17.273 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2024-04-25T11:39:17.273 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2024-04-25T11:39:17.273 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:17.273 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:17.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:39:17.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:17.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:17.328 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:39:17.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:39:17.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.688+0000 7fcbfac00640 1 Processor -- start 2024-04-25T11:39:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.688+0000 7fcbfac00640 1 -- start start 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbfac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbf40b0800 0x7fcbf4070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbfac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbfac00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf4070be0 con 0x7fcbf40cf9e0 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbfac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf40cffc0 con 0x7fcbf40b0800 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbfac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf40d02a0 con 0x7fcbf40707d0 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbf8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbf40b0800 0x7fcbf4070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbf8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbf40b0800 0x7fcbf4070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52986/0 (socket says 172.21.15.19:52986) 2024-04-25T11:39:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.691+0000 7fcbf8e00640 1 -- 172.21.15.19:0/2368039062 learned_addr learned my addr 172.21.15.19:0/2368039062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.692+0000 7fcbf8e00640 1 -- 172.21.15.19:0/2368039062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 msgr2=0x7fcbf41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.692+0000 7fcbfa200640 1 --2- 172.21.15.19:0/2368039062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.692+0000 7fcbf8e00640 1 --2- 172.21.15.19:0/2368039062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.692+0000 7fcbf8e00640 1 -- 172.21.15.19:0/2368039062 >> v1:172.21.15.169:6789/0 conn(0x7fcbf40cf9e0 legacy=0x7fcbf4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.692+0000 7fcbf8e00640 1 -- 172.21.15.19:0/2368039062 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf40d0580 con 0x7fcbf40b0800 2024-04-25T11:39:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.693+0000 7fcbf8e00640 1 --2- 172.21.15.19:0/2368039062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf4070290 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009870 tx=0x7fcbe002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463571c5a4ac4234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.693+0000 7fcbf3e00640 1 -- 172.21.15.19:0/2368039062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0037070 con 0x7fcbf40b0800 2024-04-25T11:39:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.693+0000 7fcbf3e00640 1 -- 172.21.15.19:0/2368039062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe0032440 con 0x7fcbf40b0800 2024-04-25T11:39:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.693+0000 7fcbf3e00640 1 -- 172.21.15.19:0/2368039062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0032810 con 0x7fcbf40b0800 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 -- 172.21.15.19:0/2368039062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 msgr2=0x7fcbf4070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 --2- 172.21.15.19:0/2368039062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf4070290 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009870 tx=0x7fcbe002a180 comp rx=0 tx=0).stop 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 -- 172.21.15.19:0/2368039062 shutdown_connections 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 --2- 172.21.15.19:0/2368039062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 --2- 172.21.15.19:0/2368039062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf4070290 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 -- 172.21.15.19:0/2368039062 >> 172.21.15.19:0/2368039062 conn(0x7fcbf40d1720 msgr2=0x7fcbf40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 -- 172.21.15.19:0/2368039062 shutdown_connections 2024-04-25T11:39:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.694+0000 7fcbfac00640 1 -- 172.21.15.19:0/2368039062 wait complete. 2024-04-25T11:39:17.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.697+0000 7fcbfac00640 1 Processor -- start 2024-04-25T11:39:17.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.697+0000 7fcbfac00640 1 -- start start 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.698+0000 7fcbfac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.698+0000 7fcbfac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf41572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.698+0000 7fcbfac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbf40cf9e0 0x7fcbf4146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.698+0000 7fcbfac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf40b0d90 con 0x7fcbf40b0800 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.698+0000 7fcbfac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf40b0c10 con 0x7fcbf40cf9e0 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.698+0000 7fcbfac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf4156ef0 con 0x7fcbf40707d0 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf41572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf41572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52998/0 (socket says 172.21.15.19:52998) 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 -- 172.21.15.19:0/1888287380 learned_addr learned my addr 172.21.15.19:0/1888287380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf9800640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 -- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 msgr2=0x7fcbf41479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41479c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbfa200640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbf40cf9e0 0x7fcbf4146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 -- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbf40cf9e0 msgr2=0x7fcbf4146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:17.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbf40cf9e0 0x7fcbf4146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:17.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.699+0000 7fcbf8e00640 1 -- 172.21.15.19:0/1888287380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf4147f00 con 0x7fcbf40b0800 2024-04-25T11:39:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.700+0000 7fcbf8e00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf41572b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009870 tx=0x7fcbe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.700+0000 7fcbf1600640 1 -- 172.21.15.19:0/1888287380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0041070 con 0x7fcbf40b0800 2024-04-25T11:39:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.700+0000 7fcbf1600640 1 -- 172.21.15.19:0/1888287380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe003c870 con 0x7fcbf40b0800 2024-04-25T11:39:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.700+0000 7fcbf1600640 1 -- 172.21.15.19:0/1888287380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0037040 con 0x7fcbf40b0800 2024-04-25T11:39:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.700+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf415a220 con 0x7fcbf40b0800 2024-04-25T11:39:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.700+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf415a7e0 con 0x7fcbf40b0800 2024-04-25T11:39:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.701+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf4159f90 con 0x7fcbf40b0800 2024-04-25T11:39:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.702+0000 7fcbf1600640 1 -- 172.21.15.19:0/1888287380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcbe0044070 con 0x7fcbf40b0800 2024-04-25T11:39:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.703+0000 7fcbf1600640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcbd40744b0 0x7fcbd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.704+0000 7fcbf1600640 1 -- 172.21.15.19:0/1888287380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fcbe00b6760 con 0x7fcbf40b0800 2024-04-25T11:39:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.704+0000 7fcbf9800640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcbd40744b0 0x7fcbd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:17.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.706+0000 7fcbf9800640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcbd40744b0 0x7fcbd40769c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40aea20 tx=0x7fcbe8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:17.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.708+0000 7fcbf1600640 1 -- 172.21.15.19:0/1888287380 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbf4070be0 con 0x7fcbf40b0800 2024-04-25T11:39:17.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:17.953+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcbf4050740 con 0x7fcbf40b0800 2024-04-25T11:39:18.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.608+0000 7fcbf1600640 1 -- 172.21.15.19:0/1888287380 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v255) ==== 161+0+0 (secure 0 0 0) 0x7fcbe0080620 con 0x7fcbf40b0800 2024-04-25T11:39:18.597 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcbd40744b0 msgr2=0x7fcbd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcbd40744b0 0x7fcbd40769c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40aea20 tx=0x7fcbe8008040 comp rx=0 tx=0).stop 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 msgr2=0x7fcbf41572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf41572b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009870 tx=0x7fcbe00046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 shutdown_connections 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcbd40744b0 0x7fcbd40769c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbf40cf9e0 0x7fcbf4146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf40b0800 0x7fcbf41572b0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 --2- 172.21.15.19:0/1888287380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbf40707d0 0x7fcbf41479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 >> 172.21.15.19:0/1888287380 conn(0x7fcbf40d1720 msgr2=0x7fcbf40b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 shutdown_connections 2024-04-25T11:39:18.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.612+0000 7fcbfac00640 1 -- 172.21.15.19:0/1888287380 wait complete. 2024-04-25T11:39:18.633 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:18.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.928+0000 7fdeb3200640 1 Processor -- start 2024-04-25T11:39:18.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.928+0000 7fdeb3200640 1 -- start start 2024-04-25T11:39:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.931+0000 7fdeb3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdeb4144370 0x7fdeb40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.931+0000 7fdeb3200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb40d1230 con 0x7fdeb40b4ea0 2024-04-25T11:39:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.931+0000 7fdeb3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb40d06b0 con 0x7fdeb406c5c0 2024-04-25T11:39:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.931+0000 7fdeb3200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb40d0990 con 0x7fdeb4144370 2024-04-25T11:39:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.931+0000 7fdeb2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdeb4144370 0x7fdeb40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.931+0000 7fdeb2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdeb4144370 0x7fdeb40cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53000/0 (socket says 172.21.15.19:53000) 2024-04-25T11:39:18.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.931+0000 7fdeb2800640 1 -- 172.21.15.19:0/3140136727 learned_addr learned my addr 172.21.15.19:0/3140136727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.932+0000 7fdeb2800640 1 -- 172.21.15.19:0/3140136727 >> v1:172.21.15.169:6789/0 conn(0x7fdeb406c5c0 legacy=0x7fdeb40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.932+0000 7fdeb2800640 1 -- 172.21.15.19:0/3140136727 >> v1:172.21.15.19:6789/0 conn(0x7fdeb40b4ea0 legacy=0x7fdeb415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.932+0000 7fdeb2800640 1 -- 172.21.15.19:0/3140136727 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb40d0c70 con 0x7fdeb4144370 2024-04-25T11:39:18.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.933+0000 7fdeb2800640 1 --2- 172.21.15.19:0/3140136727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4144370 0x7fdeb40cfa70 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009e50 tx=0x7fdeac02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccc2d178e1c2662 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.933+0000 7fdeb0a00640 1 -- 172.21.15.19:0/3140136727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac036070 con 0x7fdeb4144370 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.933+0000 7fdeb0a00640 1 -- 172.21.15.19:0/3140136727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac031600 con 0x7fdeb4144370 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.933+0000 7fdeb0a00640 1 -- 172.21.15.19:0/3140136727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0319d0 con 0x7fdeb4144370 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.934+0000 7fdeb3200640 1 -- 172.21.15.19:0/3140136727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4144370 msgr2=0x7fdeb40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.934+0000 7fdeb3200640 1 --2- 172.21.15.19:0/3140136727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4144370 0x7fdeb40cfa70 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009e50 tx=0x7fdeac02a760 comp rx=0 tx=0).stop 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.934+0000 7fdeb3200640 1 -- 172.21.15.19:0/3140136727 shutdown_connections 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.934+0000 7fdeb3200640 1 --2- 172.21.15.19:0/3140136727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb4144370 0x7fdeb40cfa70 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.934+0000 7fdeb3200640 1 -- 172.21.15.19:0/3140136727 >> 172.21.15.19:0/3140136727 conn(0x7fdeb40d1720 msgr2=0x7fdeb40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.934+0000 7fdeb3200640 1 -- 172.21.15.19:0/3140136727 shutdown_connections 2024-04-25T11:39:18.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.934+0000 7fdeb3200640 1 -- 172.21.15.19:0/3140136727 wait complete. 2024-04-25T11:39:18.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.937+0000 7fdeb3200640 1 Processor -- start 2024-04-25T11:39:18.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.937+0000 7fdeb3200640 1 -- start start 2024-04-25T11:39:18.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.938+0000 7fdeb3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb406c5c0 0x7fdeb4140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.938+0000 7fdeb3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb4144370 0x7fdeb41606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.938+0000 7fdeb3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb4160c10 0x7fdeb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.938+0000 7fdeb3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb40d0450 con 0x7fdeb406c5c0 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.938+0000 7fdeb3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb40d02d0 con 0x7fdeb4160c10 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.938+0000 7fdeb3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb4147760 con 0x7fdeb4144370 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb406c5c0 0x7fdeb4140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb406c5c0 0x7fdeb4140a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53016/0 (socket says 172.21.15.19:53016) 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 -- 172.21.15.19:0/1163891686 learned_addr learned my addr 172.21.15.19:0/1163891686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1400640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb4144370 0x7fdeb41606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb2800640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb4160c10 0x7fdeb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 -- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb4144370 msgr2=0x7fdeb41606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb4144370 0x7fdeb41606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 -- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb4160c10 msgr2=0x7fdeb406d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb4160c10 0x7fdeb406d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb1e00640 1 -- 172.21.15.19:0/1163891686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb4145c70 con 0x7fdeb406c5c0 2024-04-25T11:39:18.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.939+0000 7fdeb2800640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb4160c10 0x7fdeb406d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.940+0000 7fdeb1e00640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb406c5c0 0x7fdeb4140a10 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fdea400b3f0 tx=0x7fdea400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.940+0000 7fdeaa000640 1 -- 172.21.15.19:0/1163891686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4015020 con 0x7fdeb406c5c0 2024-04-25T11:39:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.940+0000 7fdeaa000640 1 -- 172.21.15.19:0/1163891686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea4011830 con 0x7fdeb406c5c0 2024-04-25T11:39:18.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.940+0000 7fdeaa000640 1 -- 172.21.15.19:0/1163891686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011c20 con 0x7fdeb406c5c0 2024-04-25T11:39:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.940+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb4145f50 con 0x7fdeb406c5c0 2024-04-25T11:39:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.940+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb4146510 con 0x7fdeb406c5c0 2024-04-25T11:39:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.941+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb4050740 con 0x7fdeb406c5c0 2024-04-25T11:39:18.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.943+0000 7fdeaa000640 1 -- 172.21.15.19:0/1163891686 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdea4018020 con 0x7fdeb406c5c0 2024-04-25T11:39:18.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.944+0000 7fdeaa000640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde940744e0 0x7fde940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.944+0000 7fdeaa000640 1 -- 172.21.15.19:0/1163891686 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fdea4095770 con 0x7fdeb406c5c0 2024-04-25T11:39:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.944+0000 7fdeb1400640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde940744e0 0x7fde940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:18.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.946+0000 7fdeb1400640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde940744e0 0x7fde940769f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40b0940 tx=0x7fdea0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:18.949+0000 7fdeaa000640 1 -- 172.21.15.19:0/1163891686 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdea401f030 con 0x7fdeb406c5c0 2024-04-25T11:39:19.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.235+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fdeb40cd770 con 0x7fdeb406c5c0 2024-04-25T11:39:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.613+0000 7fdeaa000640 1 -- 172.21.15.19:0/1163891686 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v256) ==== 101+0+0 (secure 0 0 0) 0x7fdea405f630 con 0x7fdeb406c5c0 2024-04-25T11:39:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:39:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde940744e0 msgr2=0x7fde940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde940744e0 0x7fde940769f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40b0940 tx=0x7fdea0009040 comp rx=0 tx=0).stop 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb406c5c0 msgr2=0x7fdeb4140a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb406c5c0 0x7fdeb4140a10 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fdea400b3f0 tx=0x7fdea400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 shutdown_connections 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fde940744e0 0x7fde940769f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdeb4160c10 0x7fdeb406d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdeb4144370 0x7fdeb41606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 --2- 172.21.15.19:0/1163891686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdeb406c5c0 0x7fdeb4140a10 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 >> 172.21.15.19:0/1163891686 conn(0x7fdeb40d1720 msgr2=0x7fdeb40b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.616+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 shutdown_connections 2024-04-25T11:39:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.617+0000 7fdeb3200640 1 -- 172.21.15.19:0/1163891686 wait complete. 2024-04-25T11:39:19.638 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.829+0000 7fc459b66f40 1 Processor -- start 2024-04-25T11:39:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.829+0000 7fc459b66f40 1 -- start start 2024-04-25T11:39:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc459b66f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad82e0600 0x564ad83a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc459b66f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad830d190 0x564ad83aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc459b66f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad8327710 0x564ad83af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc459b66f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ad82de8b0 con 0x564ad82e0600 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc459b66f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ad8283d20 con 0x564ad8327710 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc459b66f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564ad8326d40 con 0x564ad830d190 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc452c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad82e0600 0x564ad83a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc452c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad82e0600 0x564ad83a62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53044/0 (socket says 172.21.15.19:53044) 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.831+0000 7fc452c00640 1 -- 172.21.15.19:0/172095393 learned_addr learned my addr 172.21.15.19:0/172095393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc452c00640 1 -- 172.21.15.19:0/172095393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad830d190 msgr2=0x564ad83aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc452c00640 1 --2- 172.21.15.19:0/172095393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad830d190 0x564ad83aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc452c00640 1 -- 172.21.15.19:0/172095393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad8327710 msgr2=0x564ad83af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc452c00640 1 --2- 172.21.15.19:0/172095393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad8327710 0x564ad83af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc452c00640 1 -- 172.21.15.19:0/172095393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad83b0000 con 0x564ad82e0600 2024-04-25T11:39:19.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc452c00640 1 --2- 172.21.15.19:0/172095393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad82e0600 0x564ad83a62c0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fc44400a510 tx=0x7fc444005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7740905d028b1473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc450e00640 1 -- 172.21.15.19:0/172095393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44400f7b0 con 0x564ad82e0600 2024-04-25T11:39:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc450e00640 1 -- 172.21.15.19:0/172095393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc444015070 con 0x564ad82e0600 2024-04-25T11:39:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.832+0000 7fc450e00640 1 -- 172.21.15.19:0/172095393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44400fb90 con 0x564ad82e0600 2024-04-25T11:39:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 -- 172.21.15.19:0/172095393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad82e0600 msgr2=0x564ad83a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 --2- 172.21.15.19:0/172095393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad82e0600 0x564ad83a62c0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fc44400a510 tx=0x7fc444005f90 comp rx=0 tx=0).stop 2024-04-25T11:39:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 -- 172.21.15.19:0/172095393 shutdown_connections 2024-04-25T11:39:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 --2- 172.21.15.19:0/172095393 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad8327710 0x564ad83af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 --2- 172.21.15.19:0/172095393 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad830d190 0x564ad83aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 --2- 172.21.15.19:0/172095393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad82e0600 0x564ad83a62c0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 -- 172.21.15.19:0/172095393 >> 172.21.15.19:0/172095393 conn(0x564ad81ffac0 msgr2=0x564ad83ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 -- 172.21.15.19:0/172095393 shutdown_connections 2024-04-25T11:39:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.833+0000 7fc459b66f40 1 -- 172.21.15.19:0/172095393 wait complete. 2024-04-25T11:39:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.834+0000 7fc459b66f40 1 Processor -- start 2024-04-25T11:39:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.834+0000 7fc459b66f40 1 -- start start 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc459b66f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad82e0600 0x564ad83a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc459b66f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad830d190 0x564ad83b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc459b66f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad8327710 0x564ad83b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc459b66f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ad82dfbd0 con 0x564ad8327710 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc459b66f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564ad83a6800 con 0x564ad830d190 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc459b66f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564ad83a3620 con 0x564ad82e0600 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc453600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad8327710 0x564ad83b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc453600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad8327710 0x564ad83b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53050/0 (socket says 172.21.15.19:53050) 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.836+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 learned_addr learned my addr 172.21.15.19:0/1939968311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc452c00640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad82e0600 0x564ad83a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad82e0600 msgr2=0x564ad83a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc452200640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad830d190 0x564ad83b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:19.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc453600640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564ad82e0600 0x564ad83a30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad830d190 msgr2=0x564ad83b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc453600640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564ad830d190 0x564ad83b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad83a39b0 con 0x564ad8327710 2024-04-25T11:39:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc453600640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad8327710 0x564ad83b1d60 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00adf0 tx=0x7fc44c0086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc44b400640 1 -- 172.21.15.19:0/1939968311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c014670 con 0x564ad8327710 2024-04-25T11:39:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc44b400640 1 -- 172.21.15.19:0/1939968311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c00f070 con 0x564ad8327710 2024-04-25T11:39:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc44b400640 1 -- 172.21.15.19:0/1939968311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c014a70 con 0x564ad8327710 2024-04-25T11:39:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.837+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ad83a3c90 con 0x564ad8327710 2024-04-25T11:39:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.838+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad83a4250 con 0x564ad8327710 2024-04-25T11:39:19.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.840+0000 7fc44b400640 1 -- 172.21.15.19:0/1939968311 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc44c00b400 con 0x564ad8327710 2024-04-25T11:39:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.841+0000 7fc44b400640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc434072ec0 0x7fc4340753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.841+0000 7fc44b400640 1 -- 172.21.15.19:0/1939968311 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fc44c0949d0 con 0x564ad8327710 2024-04-25T11:39:19.829 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.841+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564ad844a3b0 0x564ad83a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.841+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad83b43f0 con 0x564ad844a3b0 2024-04-25T11:39:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.842+0000 7fc452c00640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc434072ec0 0x7fc4340753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.842+0000 7fc452200640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564ad844a3b0 0x564ad83a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.843+0000 7fc452c00640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc434072ec0 0x7fc4340753d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc444012040 tx=0x7fc44400e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.844+0000 7fc452200640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564ad844a3b0 0x564ad83a4f30 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.847+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v118'784 uv758 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.847+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad83a5470 con 0x564ad844a3b0 2024-04-25T11:39:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.851+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v118'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.851+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84569a0 con 0x564ad844a3b0 2024-04-25T11:39:19.843 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.856+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v118'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.856+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457d50 con 0x564ad844a3b0 2024-04-25T11:39:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.859+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.859+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84580f0 con 0x564ad844a3b0 2024-04-25T11:39:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.862+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.862+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8456d40 con 0x564ad844a3b0 2024-04-25T11:39:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.866+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.866+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84570e0 con 0x564ad844a3b0 2024-04-25T11:39:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.870+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v118'790 uv764 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.870+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad84574d0 con 0x564ad844a3b0 2024-04-25T11:39:19.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.874+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v118'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.874+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84574d0 con 0x564ad844a3b0 2024-04-25T11:39:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.877+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v118'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.865 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:19.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.877+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.882+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.882+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.886+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.886+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.890+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.890+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.894+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v118'796 uv771 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.894+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.899+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'797 uv797 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.899+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.904+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v118'798 uv798 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.892 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.904+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.908+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'799 uv799 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.908+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.912+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'800 uv800 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.912+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8457480 con 0x564ad844a3b0 2024-04-25T11:39:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.917+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'801 uv801 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.917+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.921+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'802 uv802 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.922+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.927+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v118'803 uv777 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.927+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.931+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v118'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.932+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.936+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v118'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.924 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.936+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84594a0 con 0x564ad844a3b0 2024-04-25T11:39:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.941+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.941+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84594a0 con 0x564ad844a3b0 2024-04-25T11:39:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.946+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.946+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.951+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.951+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.956+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v118'809 uv783 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.956+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.961+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v118'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.961+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.965+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v118'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.954 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:19.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.966+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84594a0 con 0x564ad844a3b0 2024-04-25T11:39:19.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.970+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.970+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84594a0 con 0x564ad844a3b0 2024-04-25T11:39:19.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.975+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.975+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8458490 con 0x564ad844a3b0 2024-04-25T11:39:19.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.979+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:19.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.980+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564ad84589a0 0x564ad845adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.980+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845b300 con 0x564ad84589a0 2024-04-25T11:39:19.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.981+0000 7fc452c00640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564ad84589a0 0x564ad845adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:19.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.982+0000 7fc452c00640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564ad84589a0 0x564ad845adc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:19.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.987+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'571 uv553 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:19.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.987+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad845b6a0 con 0x564ad84589a0 2024-04-25T11:39:19.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.991+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'572 uv572 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:19.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.991+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845ba40 con 0x564ad84589a0 2024-04-25T11:39:19.983 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:19.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.995+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'573 uv573 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:19.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:19.995+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845bde0 con 0x564ad84589a0 2024-04-25T11:39:19.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.000+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v118'574 uv574 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:19.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.000+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c180 con 0x564ad84589a0 2024-04-25T11:39:19.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.004+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'575 uv575 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:19.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.004+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c520 con 0x564ad84589a0 2024-04-25T11:39:19.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.009+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v118'576 uv558 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.009+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.013+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v118'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.013+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.018+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v118'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.006 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.018+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.022+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'579 uv579 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.023+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.027+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'580 uv580 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.027+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.031+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v118'581 uv570 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.031+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.036+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v118'582 uv582 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.036+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.040+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v118'583 uv583 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.029 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:20.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.041+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8460fb0 con 0x564ad84589a0 2024-04-25T11:39:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.046+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v118'584 uv584 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.046+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8460fb0 con 0x564ad84589a0 2024-04-25T11:39:20.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.050+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'585 uv585 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.051+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.055+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v118'586 uv569 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.055+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.059+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v118'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.059+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.063+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v118'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.051 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.063+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845d8d0 con 0x564ad84589a0 2024-04-25T11:39:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.068+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.068+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845d8d0 con 0x564ad84589a0 2024-04-25T11:39:20.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.072+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.072+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845c8c0 con 0x564ad84589a0 2024-04-25T11:39:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.076+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84589a0 2024-04-25T11:39:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.077+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564ad845cdd0 0x564ad845f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.077+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845f810 con 0x564ad845cdd0 2024-04-25T11:39:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.077+0000 7fc452200640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564ad845cdd0 0x564ad845f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.079+0000 7fc452200640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564ad845cdd0 0x564ad845f1f0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.083+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.084+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad845fbe0 con 0x564ad845cdd0 2024-04-25T11:39:20.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.087+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v118'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.087+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845fbe0 con 0x564ad845cdd0 2024-04-25T11:39:20.079 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.091+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v118'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.091+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8460bf0 con 0x564ad845cdd0 2024-04-25T11:39:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.096+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.096+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8460bf0 con 0x564ad845cdd0 2024-04-25T11:39:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.100+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.100+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845fbe0 con 0x564ad845cdd0 2024-04-25T11:39:20.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.104+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.105+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845fbe0 con 0x564ad844a3b0 2024-04-25T11:39:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.109+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'114 uv113 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.109+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad845fbe0 con 0x564ad844a3b0 2024-04-25T11:39:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.113+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'115 uv115 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.113+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845fbe0 con 0x564ad844a3b0 2024-04-25T11:39:20.105 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.117+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'116 uv116 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.117+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad845fbb0 con 0x564ad844a3b0 2024-04-25T11:39:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.122+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'117 uv117 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.122+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564ad84605b0 0x564ad84629d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.122+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8462f10 con 0x564ad84605b0 2024-04-25T11:39:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.122+0000 7fc453600640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564ad84605b0 0x564ad84629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.124+0000 7fc453600640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564ad84605b0 0x564ad84629d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.128+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.128+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad84632b0 con 0x564ad84605b0 2024-04-25T11:39:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.133+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v118'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.133+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463710 con 0x564ad84605b0 2024-04-25T11:39:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.137+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v118'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.125 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.137+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8464fc0 con 0x564ad84605b0 2024-04-25T11:39:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.141+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.141+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8465360 con 0x564ad84605b0 2024-04-25T11:39:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.146+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.146+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463fb0 con 0x564ad84605b0 2024-04-25T11:39:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.150+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.151+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564ad84648b0 0x564ad8467710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.151+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8467c50 con 0x564ad84648b0 2024-04-25T11:39:20.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.151+0000 7fc452c00640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564ad84648b0 0x564ad8467710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.153+0000 7fc452c00640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564ad84648b0 0x564ad8467710 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.157+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v118'282 uv276 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.157+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8467c50 con 0x564ad84648b0 2024-04-25T11:39:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.162+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v118'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.162+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8467cd0 con 0x564ad84648b0 2024-04-25T11:39:20.154 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.166+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v118'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.166+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84690d0 con 0x564ad84648b0 2024-04-25T11:39:20.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.170+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.170+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84690d0 con 0x564ad84648b0 2024-04-25T11:39:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.174+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.175+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84680c0 con 0x564ad84648b0 2024-04-25T11:39:20.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.179+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.179+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84680c0 con 0x564ad84648b0 2024-04-25T11:39:20.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.183+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v118'288 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.183+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad84680c0 con 0x564ad84648b0 2024-04-25T11:39:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.188+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v118'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.188+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84680c0 con 0x564ad84648b0 2024-04-25T11:39:20.180 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.193+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v118'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.193+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84680c0 con 0x564ad84648b0 2024-04-25T11:39:20.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.197+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.197+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad84680c0 con 0x564ad84648b0 2024-04-25T11:39:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.202+0000 7fc452c00640 1 -- 172.21.15.19:0/1939968311 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44400e070 con 0x564ad84648b0 2024-04-25T11:39:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.202+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564ad84685d0 0x564ad846a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.202+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b0a0 con 0x564ad84685d0 2024-04-25T11:39:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.203+0000 7fc452200640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564ad84685d0 0x564ad846a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.204+0000 7fc452200640 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564ad84685d0 0x564ad846a9f0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.208+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v118'127 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.208+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad846b0a0 con 0x564ad84685d0 2024-04-25T11:39:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.213+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v118'128 uv128 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.213+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad84685d0 2024-04-25T11:39:20.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.217+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v118'129 uv129 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.205 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:20.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.217+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad84685d0 2024-04-25T11:39:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.221+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'130 uv130 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.221+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad84685d0 2024-04-25T11:39:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.226+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'131 uv131 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.226+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad844a3b0 2024-04-25T11:39:20.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.230+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.230+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad844a3b0 2024-04-25T11:39:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.234+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v118'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.235+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad844a3b0 2024-04-25T11:39:20.227 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:20.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.239+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v118'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.239+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846c4d0 con 0x564ad844a3b0 2024-04-25T11:39:20.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.243+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.243+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846c4d0 con 0x564ad844a3b0 2024-04-25T11:39:20.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.247+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.248+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad844a3b0 2024-04-25T11:39:20.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.252+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad844a3b0 2024-04-25T11:39:20.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.252+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad845cdd0 2024-04-25T11:39:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.256+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v118'253 uv247 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.256+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad845cdd0 2024-04-25T11:39:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.261+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v118'254 uv254 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.262+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b4c0 con 0x564ad845cdd0 2024-04-25T11:39:20.253 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.266+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v118'255 uv255 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.266+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b880 con 0x564ad845cdd0 2024-04-25T11:39:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.270+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'256 uv256 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.270+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b880 con 0x564ad845cdd0 2024-04-25T11:39:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.275+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'257 uv257 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.275+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b880 con 0x564ad845cdd0 2024-04-25T11:39:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.280+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v118'258 uv252 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.280+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad846b880 con 0x564ad845cdd0 2024-04-25T11:39:20.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.284+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v118'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.284+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b560 con 0x564ad845cdd0 2024-04-25T11:39:20.275 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:20.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.287+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v118'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.287+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846b560 con 0x564ad845cdd0 2024-04-25T11:39:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.292+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.292+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463be0 con 0x564ad845cdd0 2024-04-25T11:39:20.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.297+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad845cdd0 2024-04-25T11:39:20.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.297+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463ba0 con 0x564ad84605b0 2024-04-25T11:39:20.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.301+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v118'111 uv110 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.301+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8463ba0 con 0x564ad84605b0 2024-04-25T11:39:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.305+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'112 uv112 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.305+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463ba0 con 0x564ad84605b0 2024-04-25T11:39:20.297 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.310+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v118'113 uv113 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.310+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463ba0 con 0x564ad84605b0 2024-04-25T11:39:20.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.314+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'114 uv114 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.314+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463ba0 con 0x564ad84605b0 2024-04-25T11:39:20.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.319+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v118'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.319+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8463b80 con 0x564ad84605b0 2024-04-25T11:39:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.323+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v118'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.323+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b80 con 0x564ad84605b0 2024-04-25T11:39:20.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.327+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v118'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.315 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:20.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.327+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b80 con 0x564ad84605b0 2024-04-25T11:39:20.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.331+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.331+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b60 con 0x564ad84605b0 2024-04-25T11:39:20.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.335+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.336+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b60 con 0x564ad84605b0 2024-04-25T11:39:20.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.340+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc44c05a4b0 con 0x564ad84605b0 2024-04-25T11:39:20.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.340+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b60 con 0x564ad84685d0 2024-04-25T11:39:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.344+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v118'130 uv129 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.344+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8463b60 con 0x564ad84685d0 2024-04-25T11:39:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.348+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v118'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.348+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b60 con 0x564ad84685d0 2024-04-25T11:39:20.340 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.352+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v118'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.352+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b60 con 0x564ad84685d0 2024-04-25T11:39:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.357+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.357+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b60 con 0x564ad84685d0 2024-04-25T11:39:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.361+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.361+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.365+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v118'314 uv306 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.365+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.370+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v118'315 uv315 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.370+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.362 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.374+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v118'316 uv316 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.375+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.379+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v118'317 uv317 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.379+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.384+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'318 uv318 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.384+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.388+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v118'319 uv313 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.388+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.392+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v118'320 uv320 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.393+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.385 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.397+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v118'321 uv321 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.397+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.401+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'322 uv322 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.401+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.405+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'323 uv323 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.406+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad8463b40 con 0x564ad84685d0 2024-04-25T11:39:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.410+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'324 uv324 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.410+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564ad846c030 con 0x564ad84685d0 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.414+0000 7fc452200640 1 -- 172.21.15.19:0/1939968311 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'325 uv325 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc4400527f0 con 0x564ad84685d0 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.414+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564ad84589a0 msgr2=0x564ad845adc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.414+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x564ad84589a0 0x564ad845adc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564ad84648b0 msgr2=0x564ad8467710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x564ad84648b0 0x564ad8467710 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564ad84685d0 msgr2=0x564ad846a9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564ad84685d0 0x564ad846a9f0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564ad845cdd0 msgr2=0x564ad845f1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x564ad845cdd0 0x564ad845f1f0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 reap_dead start 2024-04-25T11:39:20.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564ad844a3b0 msgr2=0x564ad83a4f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x564ad844a3b0 0x564ad83a4f30 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564ad84605b0 msgr2=0x564ad84629d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc453600640 1 -- 172.21.15.19:0/1939968311 reap_dead start 2024-04-25T11:39:20.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564ad84605b0 0x564ad84629d0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc434072ec0 msgr2=0x7fc4340753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc434072ec0 0x7fc4340753d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc444012040 tx=0x7fc44400e000 comp rx=0 tx=0).stop 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad8327710 msgr2=0x564ad83b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.415+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad8327710 0x564ad83b1d60 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00adf0 tx=0x7fc44c0086e0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.416+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 shutdown_connections 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.416+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x564ad84605b0 0x564ad84629d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.416+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc434072ec0 0x7fc4340753d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.416+0000 7fc459b66f40 1 --2- 172.21.15.19:0/1939968311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ad8327710 0x564ad83b1d60 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.416+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 >> 172.21.15.19:0/1939968311 conn(0x564ad81ffac0 msgr2=0x564ad83ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.416+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 shutdown_connections 2024-04-25T11:39:20.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.416+0000 7fc459b66f40 1 -- 172.21.15.19:0/1939968311 wait complete. 2024-04-25T11:39:20.426 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2024-04-25T11:39:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.673+0000 7ff53df97f40 1 Processor -- start 2024-04-25T11:39:20.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.673+0000 7ff53df97f40 1 -- start start 2024-04-25T11:39:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.674+0000 7ff53df97f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee11e0 0x560bd7fadd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.674+0000 7ff53df97f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7ee15f0 0x560bd7fb9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.674+0000 7ff53df97f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7fbadd0 0x560bd7fb9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.674+0000 7ff53df97f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560bd7f2a3f0 con 0x560bd7fbadd0 2024-04-25T11:39:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.674+0000 7ff53df97f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560bd7f29e70 con 0x560bd7ee15f0 2024-04-25T11:39:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.674+0000 7ff53df97f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560bd7f2a6d0 con 0x560bd7ee11e0 2024-04-25T11:39:20.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.674+0000 7ff537c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7fbadd0 0x560bd7fb9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7fbadd0 0x560bd7fb9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53066/0 (socket says 172.21.15.19:53066) 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 -- 172.21.15.19:0/3448924137 learned_addr learned my addr 172.21.15.19:0/3448924137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 -- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee11e0 msgr2=0x560bd7fadd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537200640 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee11e0 0x560bd7fadd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff536800640 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7ee15f0 0x560bd7fb9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee11e0 0x560bd7fadd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 -- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7ee15f0 msgr2=0x560bd7fb9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7ee15f0 0x560bd7fb9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 -- 172.21.15.19:0/3448924137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd7e0ab30 con 0x560bd7fbadd0 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff536800640 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7ee15f0 0x560bd7fb9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:20.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.675+0000 7ff537c00640 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7fbadd0 0x560bd7fb9640 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ff530025c20 tx=0x7ff53004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740c5355da4c210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.676+0000 7ff535400640 1 -- 172.21.15.19:0/3448924137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530059040 con 0x560bd7fbadd0 2024-04-25T11:39:20.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.676+0000 7ff535400640 1 -- 172.21.15.19:0/3448924137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5300547f0 con 0x560bd7fbadd0 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.676+0000 7ff535400640 1 -- 172.21.15.19:0/3448924137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530054bc0 con 0x560bd7fbadd0 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.676+0000 7ff53df97f40 1 -- 172.21.15.19:0/3448924137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7fbadd0 msgr2=0x560bd7fb9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.676+0000 7ff53df97f40 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7fbadd0 0x560bd7fb9640 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ff530025c20 tx=0x7ff53004e000 comp rx=0 tx=0).stop 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.677+0000 7ff53df97f40 1 -- 172.21.15.19:0/3448924137 shutdown_connections 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.677+0000 7ff53df97f40 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7ee15f0 0x560bd7fb9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.677+0000 7ff53df97f40 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7fbadd0 0x560bd7fb9640 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.677+0000 7ff53df97f40 1 --2- 172.21.15.19:0/3448924137 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee11e0 0x560bd7fadd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.677+0000 7ff53df97f40 1 -- 172.21.15.19:0/3448924137 >> 172.21.15.19:0/3448924137 conn(0x560bd7e03ac0 msgr2=0x560bd7f13360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.677+0000 7ff53df97f40 1 -- 172.21.15.19:0/3448924137 shutdown_connections 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.677+0000 7ff53df97f40 1 -- 172.21.15.19:0/3448924137 wait complete. 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.678+0000 7ff53df97f40 1 Processor -- start 2024-04-25T11:39:20.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.678+0000 7ff53df97f40 1 -- start start 2024-04-25T11:39:20.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.679+0000 7ff53df97f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7ee11e0 0x560bd8049330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.679+0000 7ff53df97f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee15f0 0x560bd7faaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.679+0000 7ff53df97f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7fbadd0 0x560bd7fab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.679+0000 7ff53df97f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560bd7edfa00 con 0x560bd7ee11e0 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.679+0000 7ff53df97f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560bd7ee0870 con 0x560bd7fbadd0 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.679+0000 7ff53df97f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560bd7f13a60 con 0x560bd7ee15f0 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7ee11e0 0x560bd8049330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7ee11e0 0x560bd8049330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53072/0 (socket says 172.21.15.19:53072) 2024-04-25T11:39:20.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 -- 172.21.15.19:0/1684205250 learned_addr learned my addr 172.21.15.19:0/1684205250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537c00640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7fbadd0 0x560bd7fab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff536800640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee15f0 0x560bd7faaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 -- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee15f0 msgr2=0x560bd7faaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560bd7ee15f0 0x560bd7faaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 -- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7fbadd0 msgr2=0x560bd7fab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7fbadd0 0x560bd7fab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537200640 1 -- 172.21.15.19:0/1684205250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd8049870 con 0x560bd7ee11e0 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.680+0000 7ff537c00640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560bd7fbadd0 0x560bd7fab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.681+0000 7ff537200640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7ee11e0 0x560bd8049330 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7ff528008250 tx=0x7ff528008710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.681+0000 7ff52fe00640 1 -- 172.21.15.19:0/1684205250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528016020 con 0x560bd7ee11e0 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.681+0000 7ff52fe00640 1 -- 172.21.15.19:0/1684205250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff528012610 con 0x560bd7ee11e0 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.681+0000 7ff52fe00640 1 -- 172.21.15.19:0/1684205250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52800e040 con 0x560bd7ee11e0 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.681+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560bd7fba0a0 con 0x560bd7ee11e0 2024-04-25T11:39:20.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.681+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd7fb4210 con 0x560bd7ee11e0 2024-04-25T11:39:20.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.684+0000 7ff52fe00640 1 -- 172.21.15.19:0/1684205250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff528019020 con 0x560bd7ee11e0 2024-04-25T11:39:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.684+0000 7ff52fe00640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff514072ec0 0x7ff5140753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.685+0000 7ff52fe00640 1 -- 172.21.15.19:0/1684205250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7ff528094480 con 0x560bd7ee11e0 2024-04-25T11:39:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.685+0000 7ff53df97f40 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560bd7f12a20 0x560bd7f12e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.685+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:1 12.15 12:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x560bd7f13370 con 0x560bd7f12a20 2024-04-25T11:39:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.685+0000 7ff537c00640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560bd7f12a20 0x560bd7f12e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.685+0000 7ff536800640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff514072ec0 0x7ff5140753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.686+0000 7ff537c00640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560bd7f12a20 0x560bd7f12e30 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.686+0000 7ff536800640 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff514072ec0 0x7ff5140753d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x560bd7ede3f0 tx=0x7ff520005820 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff537c00640 1 -- 172.21.15.19:0/1684205250 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v118'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff530064070 con 0x560bd7f12a20 2024-04-25T11:39:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560bd7f12a20 msgr2=0x560bd7f12e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560bd7f12a20 0x560bd7f12e30 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff514072ec0 msgr2=0x7ff5140753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff514072ec0 0x7ff5140753d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x560bd7ede3f0 tx=0x7ff520005820 comp rx=0 tx=0).stop 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7ee11e0 msgr2=0x560bd8049330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 --2- 172.21.15.19:0/1684205250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd7ee11e0 0x560bd8049330 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7ff528008250 tx=0x7ff528008710 comp rx=0 tx=0).stop 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff537c00640 1 -- 172.21.15.19:0/1684205250 reap_dead start 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 shutdown_connections 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 >> 172.21.15.19:0/1684205250 conn(0x560bd7e03ac0 msgr2=0x560bd7f2b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 shutdown_connections 2024-04-25T11:39:20.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.689+0000 7ff53df97f40 1 -- 172.21.15.19:0/1684205250 wait complete. 2024-04-25T11:39:20.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:20.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.983+0000 7fd6f0a00640 1 Processor -- start 2024-04-25T11:39:20.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.983+0000 7fd6f0a00640 1 -- start start 2024-04-25T11:39:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.985+0000 7fd6f0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec143330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.985+0000 7fd6f0a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd6ec147610 0x7fd6ec148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.985+0000 7fd6f0a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ec0b5c20 con 0x7fd6ec1495b0 2024-04-25T11:39:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.985+0000 7fd6f0a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ec06c5c0 con 0x7fd6ec147610 2024-04-25T11:39:20.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.985+0000 7fd6f0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec0b5f00 con 0x7fd6ec0b5740 2024-04-25T11:39:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.986+0000 7fd6ea800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.986+0000 7fd6ea800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec143330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:45844/0 (socket says 172.21.15.19:45844) 2024-04-25T11:39:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.986+0000 7fd6ea800640 1 -- 172.21.15.19:0/3297970293 learned_addr learned my addr 172.21.15.19:0/3297970293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:20.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.986+0000 7fd6eb200640 1 --2- 172.21.15.19:0/3297970293 >> v2:172.21.15.169:3300/0 conn(0x7fd6ec147610 0x7fd6ec148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.987+0000 7fd6ea800640 1 -- 172.21.15.19:0/3297970293 >> v2:172.21.15.169:3300/0 conn(0x7fd6ec147610 msgr2=0x7fd6ec148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.987+0000 7fd6ea800640 1 --2- 172.21.15.19:0/3297970293 >> v2:172.21.15.169:3300/0 conn(0x7fd6ec147610 0x7fd6ec148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.987+0000 7fd6ea800640 1 -- 172.21.15.19:0/3297970293 >> v1:172.21.15.169:6789/0 conn(0x7fd6ec1495b0 legacy=0x7fd6ec146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.987+0000 7fd6ea800640 1 -- 172.21.15.19:0/3297970293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec0b61e0 con 0x7fd6ec0b5740 2024-04-25T11:39:20.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.988+0000 7fd6ea800640 1 --2- 172.21.15.19:0/3297970293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec143330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009870 tx=0x7fd6d402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8bc2ef1b26e32a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.989+0000 7fd6e9400640 1 -- 172.21.15.19:0/3297970293 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4037070 con 0x7fd6ec0b5740 2024-04-25T11:39:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.989+0000 7fd6e9400640 1 -- 172.21.15.19:0/3297970293 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d4032440 con 0x7fd6ec0b5740 2024-04-25T11:39:20.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.989+0000 7fd6e9400640 1 -- 172.21.15.19:0/3297970293 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4032810 con 0x7fd6ec0b5740 2024-04-25T11:39:20.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.990+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3297970293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 msgr2=0x7fd6ec143330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.990+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3297970293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec143330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009870 tx=0x7fd6d402a270 comp rx=0 tx=0).stop 2024-04-25T11:39:20.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.990+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3297970293 shutdown_connections 2024-04-25T11:39:20.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.990+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3297970293 >> v2:172.21.15.169:3300/0 conn(0x7fd6ec147610 0x7fd6ec148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.990+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3297970293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec143330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.990+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3297970293 >> 172.21.15.19:0/3297970293 conn(0x7fd6ec0b83a0 msgr2=0x7fd6ec070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:20.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.990+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3297970293 shutdown_connections 2024-04-25T11:39:20.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.991+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3297970293 wait complete. 2024-04-25T11:39:20.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.993+0000 7fd6f0a00640 1 Processor -- start 2024-04-25T11:39:20.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.993+0000 7fd6f0a00640 1 -- start start 2024-04-25T11:39:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.994+0000 7fd6f0a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.994+0000 7fd6f0a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6ec1495b0 0x7fd6ec130c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.994+0000 7fd6f0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6ec131140 0x7fd6ec131550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.994+0000 7fd6f0a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec149c50 con 0x7fd6ec131140 2024-04-25T11:39:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.994+0000 7fd6f0a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec0b67a0 con 0x7fd6ec1495b0 2024-04-25T11:39:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.994+0000 7fd6f0a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec149dd0 con 0x7fd6ec0b5740 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6ec131140 0x7fd6ec131550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6ec131140 0x7fd6ec131550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53086/0 (socket says 172.21.15.19:53086) 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 -- 172.21.15.19:0/3264721885 learned_addr learned my addr 172.21.15.19:0/3264721885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6ea800640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6e9e00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6ec1495b0 0x7fd6ec130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 -- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 msgr2=0x7fd6ec160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 -- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6ec1495b0 msgr2=0x7fd6ec130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6ec1495b0 0x7fd6ec130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.995+0000 7fd6eb200640 1 -- 172.21.15.19:0/3264721885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec131a90 con 0x7fd6ec131140 2024-04-25T11:39:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.996+0000 7fd6eb200640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6ec131140 0x7fd6ec131550 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400e7c0 tx=0x7fd6e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.996+0000 7fd6e2a00640 1 -- 172.21.15.19:0/3264721885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4017070 con 0x7fd6ec131140 2024-04-25T11:39:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.996+0000 7fd6e2a00640 1 -- 172.21.15.19:0/3264721885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e400c730 con 0x7fd6ec131140 2024-04-25T11:39:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.996+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ec0ce7d0 con 0x7fd6ec131140 2024-04-25T11:39:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.996+0000 7fd6e2a00640 1 -- 172.21.15.19:0/3264721885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400cb20 con 0x7fd6ec131140 2024-04-25T11:39:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.996+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ec130400 con 0x7fd6ec131140 2024-04-25T11:39:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.998+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ec162790 con 0x7fd6ec131140 2024-04-25T11:39:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:20.999+0000 7fd6e2a00640 1 -- 172.21.15.19:0/3264721885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6e401a070 con 0x7fd6ec131140 2024-04-25T11:39:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.000+0000 7fd6e2a00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6c80744e0 0x7fd6c80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.000+0000 7fd6e2a00640 1 -- 172.21.15.19:0/3264721885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fd6e4095940 con 0x7fd6ec131140 2024-04-25T11:39:20.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.001+0000 7fd6ea800640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6c80744e0 0x7fd6c80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.002+0000 7fd6ea800640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6c80744e0 0x7fd6c80769f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4034000 tx=0x7fd6d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.005+0000 7fd6e2a00640 1 -- 172.21.15.19:0/3264721885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6e405f810 con 0x7fd6ec131140 2024-04-25T11:39:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.260+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd6ec0ce560 con 0x7fd6ec131140 2024-04-25T11:39:21.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.638+0000 7fd6e2a00640 1 -- 172.21.15.19:0/3264721885 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v257) ==== 162+0+0 (secure 0 0 0) 0x7fd6e405f110 con 0x7fd6ec131140 2024-04-25T11:39:21.627 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:21.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.641+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6c80744e0 msgr2=0x7fd6c80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:21.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.641+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6c80744e0 0x7fd6c80769f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4034000 tx=0x7fd6d402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:21.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.641+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6ec131140 msgr2=0x7fd6ec131550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:21.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.641+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6ec131140 0x7fd6ec131550 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400e7c0 tx=0x7fd6e400ec80 comp rx=0 tx=0).stop 2024-04-25T11:39:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 shutdown_connections 2024-04-25T11:39:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd6c80744e0 0x7fd6c80769f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6ec1495b0 0x7fd6ec130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6ec131140 0x7fd6ec131550 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 --2- 172.21.15.19:0/3264721885 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6ec0b5740 0x7fd6ec160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 >> 172.21.15.19:0/3264721885 conn(0x7fd6ec0b83a0 msgr2=0x7fd6ec0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 shutdown_connections 2024-04-25T11:39:21.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:21.642+0000 7fd6f0a00640 1 -- 172.21.15.19:0/3264721885 wait complete. 2024-04-25T11:39:22.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.376+0000 7fdd76600640 1 Processor -- start 2024-04-25T11:39:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.376+0000 7fdd76600640 1 -- start start 2024-04-25T11:39:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.379+0000 7fdd76600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdd70167000 0x7fdd70169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.379+0000 7fdd76600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd7015f5f0 con 0x7fdd700cf5e0 2024-04-25T11:39:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.379+0000 7fdd76600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd7015e830 con 0x7fdd701443c0 2024-04-25T11:39:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.379+0000 7fdd76600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fdd700708d0 con 0x7fdd70167000 2024-04-25T11:39:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.379+0000 7fdd66a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdd700cf5e0 0x7fdd700cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:46168/0 (socket says 172.21.15.19:46168) 2024-04-25T11:39:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.379+0000 7fdd66a00640 1 -- 172.21.15.19:0/4293111231 learned_addr learned my addr 172.21.15.19:0/4293111231 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.379+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447731960 0 0) 0x7fdd7015f5f0 con 0x7fdd700cf5e0 2024-04-25T11:39:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd54003520 con 0x7fdd700cf5e0 2024-04-25T11:39:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6fe00640 1 --2- 172.21.15.19:0/4293111231 >> v2:172.21.15.169:3300/0 conn(0x7fdd70167000 0x7fdd70169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2939220796 0 0) 0x7fdd54003520 con 0x7fdd700cf5e0 2024-04-25T11:39:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 >> v2:172.21.15.169:3300/0 conn(0x7fdd70167000 msgr2=0x7fdd70169420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 --2- 172.21.15.19:0/4293111231 >> v2:172.21.15.169:3300/0 conn(0x7fdd70167000 0x7fdd70169420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 >> v1:172.21.15.169:6789/0 conn(0x7fdd701443c0 legacy=0x7fdd7006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7015eb10 con 0x7fdd700cf5e0 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fdd58001670 con 0x7fdd700cf5e0 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6fe00640 1 --2- 172.21.15.19:0/4293111231 >> v2:172.21.15.169:3300/0 conn(0x7fdd70167000 0x7fdd70169420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 >> v1:172.21.15.19:6789/0 conn(0x7fdd700cf5e0 legacy=0x7fdd700cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd54003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd54004800 0x7fdd54004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd54005270 0x7fdd5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd700708d0 con 0x7fdd540039b0 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5400c200 con 0x7fdd54005270 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.380+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5400c400 con 0x7fdd54004800 2024-04-25T11:39:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6f400640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd54003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6fe00640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd54005270 0x7fdd5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd66a00640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd54004800 0x7fdd54004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6f400640 1 -- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd54004800 msgr2=0x7fdd54004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6f400640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd54004800 0x7fdd54004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6f400640 1 -- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd54005270 msgr2=0x7fdd5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6f400640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd54005270 0x7fdd5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6f400640 1 -- 172.21.15.19:0/4293111231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd54003520 con 0x7fdd540039b0 2024-04-25T11:39:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.381+0000 7fdd6fe00640 1 -- 172.21.15.19:0/4293111231 reap_dead start 2024-04-25T11:39:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.382+0000 7fdd6f400640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd54003e20 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fdd60001720 tx=0x7fdd54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e03064c3919580d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.383+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60001370 con 0x7fdd540039b0 2024-04-25T11:39:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.383+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd600052d0 con 0x7fdd540039b0 2024-04-25T11:39:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.383+0000 7fdd6ea00640 1 -- 172.21.15.19:0/4293111231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6002ab90 con 0x7fdd540039b0 2024-04-25T11:39:22.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.383+0000 7fdd76600640 1 -- 172.21.15.19:0/4293111231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 msgr2=0x7fdd54003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.383+0000 7fdd76600640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd54003e20 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fdd60001720 tx=0x7fdd54005830 comp rx=0 tx=0).stop 2024-04-25T11:39:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.383+0000 7fdd76600640 1 -- 172.21.15.19:0/4293111231 shutdown_connections 2024-04-25T11:39:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.384+0000 7fdd76600640 1 --2- 172.21.15.19:0/4293111231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd54003e20 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.384+0000 7fdd76600640 1 -- 172.21.15.19:0/4293111231 >> 172.21.15.19:0/4293111231 conn(0x7fdd700d1ae0 msgr2=0x7fdd700d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.384+0000 7fdd76600640 1 -- 172.21.15.19:0/4293111231 shutdown_connections 2024-04-25T11:39:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.384+0000 7fdd76600640 1 -- 172.21.15.19:0/4293111231 wait complete. 2024-04-25T11:39:22.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.386+0000 7fdd76600640 1 Processor -- start 2024-04-25T11:39:22.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.387+0000 7fdd76600640 1 -- start start 2024-04-25T11:39:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.388+0000 7fdd76600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd7015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.388+0000 7fdd76600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7015cf60 0x7fdd70157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.388+0000 7fdd76600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd70157660 0x7fdd701459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.388+0000 7fdd76600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701447d0 con 0x7fdd540039b0 2024-04-25T11:39:22.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.388+0000 7fdd76600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7015f160 con 0x7fdd70157660 2024-04-25T11:39:22.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.388+0000 7fdd76600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdd70162b00 con 0x7fdd7015cf60 2024-04-25T11:39:22.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd7015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd7015ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53100/0 (socket says 172.21.15.19:53100) 2024-04-25T11:39:22.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 -- 172.21.15.19:0/3928146099 learned_addr learned my addr 172.21.15.19:0/3928146099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd66a00640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7015cf60 0x7fdd70157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6fe00640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd70157660 0x7fdd701459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 -- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7015cf60 msgr2=0x7fdd70157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7015cf60 0x7fdd70157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 -- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd70157660 msgr2=0x7fdd701459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd70157660 0x7fdd701459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6f400640 1 -- 172.21.15.19:0/3928146099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7015d370 con 0x7fdd540039b0 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.389+0000 7fdd6fe00640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd70157660 0x7fdd701459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.390+0000 7fdd6f400640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd7015ca20 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fdd60005040 tx=0x7fdd60001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.390+0000 7fdd67e00640 1 -- 172.21.15.19:0/3928146099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6003f020 con 0x7fdd540039b0 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.390+0000 7fdd67e00640 1 -- 172.21.15.19:0/3928146099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd60005480 con 0x7fdd540039b0 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.390+0000 7fdd67e00640 1 -- 172.21.15.19:0/3928146099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60037040 con 0x7fdd540039b0 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.390+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd700b4bd0 con 0x7fdd540039b0 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.390+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd700b4e90 con 0x7fdd540039b0 2024-04-25T11:39:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.391+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd70050740 con 0x7fdd540039b0 2024-04-25T11:39:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.393+0000 7fdd67e00640 1 -- 172.21.15.19:0/3928146099 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd60042020 con 0x7fdd540039b0 2024-04-25T11:39:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.394+0000 7fdd67e00640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd4c0744b0 0x7fdd4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:22.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.394+0000 7fdd67e00640 1 -- 172.21.15.19:0/3928146099 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fdd600b5750 con 0x7fdd540039b0 2024-04-25T11:39:22.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.394+0000 7fdd66a00640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd4c0744b0 0x7fdd4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.396+0000 7fdd66a00640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd4c0744b0 0x7fdd4c0769c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdd58001200 tx=0x7fdd58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:22.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.399+0000 7fdd67e00640 1 -- 172.21.15.19:0/3928146099 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd60049030 con 0x7fdd540039b0 2024-04-25T11:39:22.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.653+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd7013b850 con 0x7fdd540039b0 2024-04-25T11:39:22.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.655+0000 7fdd67e00640 1 -- 172.21.15.19:0/3928146099 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 258 v258) ==== 95+0+5187 (secure 0 0 0) 0x7fdd7015d370 con 0x7fdd540039b0 2024-04-25T11:39:22.643 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:22.643 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":258,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:21.647338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9912":{"gid":9912,"name":"a","rank":0,"incarnation":258,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/3564168877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3564168877},{"type":"v1","addr":"172.21.15.19:6835","nonce":3564168877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:22.643 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 258 2024-04-25T11:39:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd4c0744b0 msgr2=0x7fdd4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd4c0744b0 0x7fdd4c0769c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdd58001200 tx=0x7fdd58009040 comp rx=0 tx=0).stop 2024-04-25T11:39:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 msgr2=0x7fdd7015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd7015ca20 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fdd60005040 tx=0x7fdd60001120 comp rx=0 tx=0).stop 2024-04-25T11:39:22.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 shutdown_connections 2024-04-25T11:39:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdd4c0744b0 0x7fdd4c0769c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd70157660 0x7fdd701459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdd7015cf60 0x7fdd70157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 --2- 172.21.15.19:0/3928146099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd540039b0 0x7fdd7015ca20 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 >> 172.21.15.19:0/3928146099 conn(0x7fdd700d1ae0 msgr2=0x7fdd7013bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.657+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 shutdown_connections 2024-04-25T11:39:22.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:22.658+0000 7fdd76600640 1 -- 172.21.15.19:0/3928146099 wait complete. 2024-04-25T11:39:22.674 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 258, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:39:21.647338+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9912}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9912': {'gid': 9912, 'name': 'a', 'rank': 0, 'incarnation': 258, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/3564168877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 3564168877}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3564168877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:39:22.674 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9912:up:replay 2024-04-25T11:39:22.674 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:39:26.586 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:26.598+0000 7f3a21600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:39:28.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:28.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.383+0000 7fd1b8e00640 1 Processor -- start 2024-04-25T11:39:28.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.383+0000 7fd1b8e00640 1 -- start start 2024-04-25T11:39:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.385+0000 7fd1b8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406c5c0 0x7fd1b40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.385+0000 7fd1b8e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd1b4144370 0x7fd1b40b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.385+0000 7fd1b8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1b40b4cd0 0x7fd1b4161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.385+0000 7fd1b8e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b40b50e0 con 0x7fd1b40b4cd0 2024-04-25T11:39:28.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.385+0000 7fd1b8e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b40d1230 con 0x7fd1b4144370 2024-04-25T11:39:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.385+0000 7fd1b8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b40d06b0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1b40b4cd0 0x7fd1b4161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1b40b4cd0 0x7fd1b4161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52144/0 (socket says 172.21.15.19:52144) 2024-04-25T11:39:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 -- 172.21.15.19:0/3475945500 learned_addr learned my addr 172.21.15.19:0/3475945500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:28.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3400640 1 --2- 172.21.15.19:0/3475945500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406c5c0 0x7fd1b40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b2a00640 1 --2- 172.21.15.19:0/3475945500 >> v2:172.21.15.169:3300/0 conn(0x7fd1b4144370 0x7fd1b40b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 -- 172.21.15.19:0/3475945500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406c5c0 msgr2=0x7fd1b40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 --2- 172.21.15.19:0/3475945500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406c5c0 0x7fd1b40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 -- 172.21.15.19:0/3475945500 >> v2:172.21.15.169:3300/0 conn(0x7fd1b4144370 msgr2=0x7fd1b40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 --2- 172.21.15.19:0/3475945500 >> v2:172.21.15.169:3300/0 conn(0x7fd1b4144370 0x7fd1b40b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.386+0000 7fd1b3e00640 1 -- 172.21.15.19:0/3475945500 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b40d0990 con 0x7fd1b40b4cd0 2024-04-25T11:39:28.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.387+0000 7fd1b3e00640 1 --2- 172.21.15.19:0/3475945500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4161a10 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40d2770 tx=0x7fd1ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c40d29f5686ec6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.388+0000 7fd1b2000640 1 -- 172.21.15.19:0/3475945500 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac037070 con 0x7fd1b40b4cd0 2024-04-25T11:39:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.388+0000 7fd1b2000640 1 -- 172.21.15.19:0/3475945500 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac032440 con 0x7fd1b40b4cd0 2024-04-25T11:39:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.388+0000 7fd1b2000640 1 -- 172.21.15.19:0/3475945500 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac032810 con 0x7fd1b40b4cd0 2024-04-25T11:39:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3475945500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b40b4cd0 msgr2=0x7fd1b4161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3475945500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4161a10 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40d2770 tx=0x7fd1ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3475945500 shutdown_connections 2024-04-25T11:39:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3475945500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4161a10 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3475945500 >> v2:172.21.15.169:3300/0 conn(0x7fd1b4144370 0x7fd1b40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3475945500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406c5c0 0x7fd1b40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3475945500 >> 172.21.15.19:0/3475945500 conn(0x7fd1b40d1720 msgr2=0x7fd1b40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3475945500 shutdown_connections 2024-04-25T11:39:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.389+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3475945500 wait complete. 2024-04-25T11:39:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.391+0000 7fd1b8e00640 1 Processor -- start 2024-04-25T11:39:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.392+0000 7fd1b8e00640 1 -- start start 2024-04-25T11:39:28.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406c5c0 0x7fd1b415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144370 0x7fd1b4139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b8e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b40d02d0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b8e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b40d0f50 con 0x7fd1b40b4cd0 2024-04-25T11:39:28.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b40d0450 con 0x7fd1b4144370 2024-04-25T11:39:28.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406c5c0 0x7fd1b415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406c5c0 0x7fd1b415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52152/0 (socket says 172.21.15.19:52152) 2024-04-25T11:39:28.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.393+0000 7fd1b3400640 1 -- 172.21.15.19:0/3038760868 learned_addr learned my addr 172.21.15.19:0/3038760868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:28.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b2a00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b3e00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144370 0x7fd1b4139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b3400640 1 -- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144370 msgr2=0x7fd1b4139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b3400640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144370 0x7fd1b4139170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b3400640 1 -- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b40b4cd0 msgr2=0x7fd1b4138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b3400640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b3400640 1 -- 172.21.15.19:0/3038760868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b4145790 con 0x7fd1b406c5c0 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b2a00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.394+0000 7fd1b3400640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406c5c0 0x7fd1b415c6a0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4007e40 tx=0x7fd1a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.395+0000 7fd1ab400640 1 -- 172.21.15.19:0/3038760868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4014620 con 0x7fd1b406c5c0 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.395+0000 7fd1ab400640 1 -- 172.21.15.19:0/3038760868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a4019070 con 0x7fd1b406c5c0 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.395+0000 7fd1ab400640 1 -- 172.21.15.19:0/3038760868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4014a00 con 0x7fd1b406c5c0 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.395+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b4144ef0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.395+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41454b0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.396+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b4050740 con 0x7fd1b406c5c0 2024-04-25T11:39:28.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.398+0000 7fd1ab400640 1 -- 172.21.15.19:0/3038760868 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd1a4014ba0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.399+0000 7fd1ab400640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1900744b0 0x7fd1900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.399+0000 7fd1ab400640 1 -- 172.21.15.19:0/3038760868 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fd1a4097190 con 0x7fd1b406c5c0 2024-04-25T11:39:28.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.399+0000 7fd1b2a00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1900744b0 0x7fd1900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.401+0000 7fd1b2a00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1900744b0 0x7fd1900769c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd19c006fd0 tx=0x7fd19c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.403+0000 7fd1ab400640 1 -- 172.21.15.19:0/3038760868 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1a40611d0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.650+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1b415cbe0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.652+0000 7fd1ab400640 1 -- 172.21.15.19:0/3038760868 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) ==== 95+0+5194 (secure 0 0 0) 0x7fd1a4060ad0 con 0x7fd1b406c5c0 2024-04-25T11:39:28.640 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:28.640 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":261,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:24.670319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9912},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9912":{"gid":9912,"name":"a","rank":0,"incarnation":258,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6835/3564168877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3564168877},{"type":"v1","addr":"172.21.15.19:6835","nonce":3564168877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9912,"qdb_cluster":[9912]},"id":6}]} 2024-04-25T11:39:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 261 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.654+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1900744b0 msgr2=0x7fd1900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.654+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1900744b0 0x7fd1900769c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd19c006fd0 tx=0x7fd19c008040 comp rx=0 tx=0).stop 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.654+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406c5c0 msgr2=0x7fd1b415c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.654+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406c5c0 0x7fd1b415c6a0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4007e40 tx=0x7fd1a400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 shutdown_connections 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1900744b0 0x7fd1900769c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b40b4cd0 0x7fd1b4138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144370 0x7fd1b4139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 --2- 172.21.15.19:0/3038760868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406c5c0 0x7fd1b415c6a0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 >> 172.21.15.19:0/3038760868 conn(0x7fd1b40d1720 msgr2=0x7fd1b4168400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 shutdown_connections 2024-04-25T11:39:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.655+0000 7fd1b8e00640 1 -- 172.21.15.19:0/3038760868 wait complete. 2024-04-25T11:39:28.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 261, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:39:24.670319+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9912}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9912': {'gid': 9912, 'name': 'a', 'rank': 0, 'incarnation': 258, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6835/3564168877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 3564168877}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3564168877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9912, 'qdb_cluster': [9912]} 2024-04-25T11:39:28.670 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:39:28.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:39:28.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.969+0000 7f1a45400640 1 Processor -- start 2024-04-25T11:39:28.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.969+0000 7f1a45400640 1 -- start start 2024-04-25T11:39:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.972+0000 7f1a45400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1a400b4ea0 0x7f1a400cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.972+0000 7f1a45400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a40144370 0x7f1a4015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.972+0000 7f1a45400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a400d1230 con 0x7f1a4006c5c0 2024-04-25T11:39:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.972+0000 7f1a45400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1a400d06b0 con 0x7f1a400b4ea0 2024-04-25T11:39:28.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.972+0000 7f1a45400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a400d0990 con 0x7f1a40144370 2024-04-25T11:39:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.973+0000 7f1a44a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a40144370 0x7f1a4015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.973+0000 7f1a3fe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f1a4006c5c0 0x7f1a400b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:38916/0 (socket says 172.21.15.19:38916) 2024-04-25T11:39:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.973+0000 7f1a3fe00640 1 -- 172.21.15.19:0/3259082022 learned_addr learned my addr 172.21.15.19:0/3259082022 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:28.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.973+0000 7f1a37e00640 1 --2- 172.21.15.19:0/3259082022 >> v2:172.21.15.169:3300/0 conn(0x7f1a400b4ea0 0x7f1a400cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.974+0000 7f1a44a00640 1 -- 172.21.15.19:0/3259082022 >> v2:172.21.15.169:3300/0 conn(0x7f1a400b4ea0 msgr2=0x7f1a400cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:28.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.974+0000 7f1a44a00640 1 --2- 172.21.15.19:0/3259082022 >> v2:172.21.15.169:3300/0 conn(0x7f1a400b4ea0 0x7f1a400cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.974+0000 7f1a44a00640 1 -- 172.21.15.19:0/3259082022 >> v1:172.21.15.169:6789/0 conn(0x7f1a4006c5c0 legacy=0x7f1a400b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.974+0000 7f1a44a00640 1 -- 172.21.15.19:0/3259082022 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a400d0c70 con 0x7f1a40144370 2024-04-25T11:39:28.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.974+0000 7f1a37e00640 1 --2- 172.21.15.19:0/3259082022 >> v2:172.21.15.169:3300/0 conn(0x7f1a400b4ea0 0x7f1a400cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:39:28.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.975+0000 7f1a44a00640 1 --2- 172.21.15.19:0/3259082022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a40144370 0x7f1a4015e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009d90 tx=0x7f1a3802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46fdf4df79b87686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.976+0000 7f1a3f400640 1 -- 172.21.15.19:0/3259082022 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38007740 con 0x7f1a40144370 2024-04-25T11:39:28.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.976+0000 7f1a3f400640 1 -- 172.21.15.19:0/3259082022 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a380078e0 con 0x7f1a40144370 2024-04-25T11:39:28.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.976+0000 7f1a3f400640 1 -- 172.21.15.19:0/3259082022 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38007cb0 con 0x7f1a40144370 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.976+0000 7f1a45400640 1 -- 172.21.15.19:0/3259082022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a40144370 msgr2=0x7f1a4015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.976+0000 7f1a45400640 1 --2- 172.21.15.19:0/3259082022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a40144370 0x7f1a4015e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009d90 tx=0x7f1a3802a660 comp rx=0 tx=0).stop 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.977+0000 7f1a45400640 1 -- 172.21.15.19:0/3259082022 shutdown_connections 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.977+0000 7f1a45400640 1 --2- 172.21.15.19:0/3259082022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a40144370 0x7f1a4015e770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.977+0000 7f1a45400640 1 --2- 172.21.15.19:0/3259082022 >> v2:172.21.15.169:3300/0 conn(0x7f1a400b4ea0 0x7f1a400cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.977+0000 7f1a45400640 1 -- 172.21.15.19:0/3259082022 >> 172.21.15.19:0/3259082022 conn(0x7f1a400d1720 msgr2=0x7f1a400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.977+0000 7f1a45400640 1 -- 172.21.15.19:0/3259082022 shutdown_connections 2024-04-25T11:39:28.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.977+0000 7f1a45400640 1 -- 172.21.15.19:0/3259082022 wait complete. 2024-04-25T11:39:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.979+0000 7f1a45400640 1 Processor -- start 2024-04-25T11:39:28.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.979+0000 7f1a45400640 1 -- start start 2024-04-25T11:39:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a45400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4006c5c0 0x7f1a401624e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a45400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40144370 0x7f1a40143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a45400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400ae640 0x7f1a4013fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a45400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a400d0450 con 0x7f1a40144370 2024-04-25T11:39:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a45400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a400d02d0 con 0x7f1a4006c5c0 2024-04-25T11:39:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a45400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a40147760 con 0x7f1a400ae640 2024-04-25T11:39:28.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40144370 0x7f1a40143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40144370 0x7f1a40143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52158/0 (socket says 172.21.15.19:52158) 2024-04-25T11:39:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a37e00640 1 -- 172.21.15.19:0/1631482922 learned_addr learned my addr 172.21.15.19:0/1631482922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4006c5c0 0x7f1a401624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.981+0000 7f1a44a00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400ae640 0x7f1a4013fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.982+0000 7f1a37e00640 1 -- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400ae640 msgr2=0x7f1a4013fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.982+0000 7f1a37e00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400ae640 0x7f1a4013fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.982+0000 7f1a37e00640 1 -- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4006c5c0 msgr2=0x7f1a401624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.982+0000 7f1a37e00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4006c5c0 0x7f1a401624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.982+0000 7f1a37e00640 1 -- 172.21.15.19:0/1631482922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a400b0ba0 con 0x7f1a40144370 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.982+0000 7f1a37e00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40144370 0x7f1a40143180 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e990 tx=0x7f1a2800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.983+0000 7f1a3cc00640 1 -- 172.21.15.19:0/1631482922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28017070 con 0x7f1a40144370 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.983+0000 7f1a3cc00640 1 -- 172.21.15.19:0/1631482922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2800c9e0 con 0x7f1a40144370 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.983+0000 7f1a3cc00640 1 -- 172.21.15.19:0/1631482922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800cdb0 con 0x7f1a40144370 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.983+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a400aeb40 con 0x7f1a40144370 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.983+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a40070bc0 con 0x7f1a40144370 2024-04-25T11:39:28.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.985+0000 7f1a3cc00640 1 -- 172.21.15.19:0/1631482922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a2801a070 con 0x7f1a40144370 2024-04-25T11:39:28.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.986+0000 7f1a3cc00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744b0 0x7f1a1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.987+0000 7f1a3cc00640 1 -- 172.21.15.19:0/1631482922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f1a28095870 con 0x7f1a40144370 2024-04-25T11:39:28.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.987+0000 7f1a4da1d740 1 Processor -- start 2024-04-25T11:39:28.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.987+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744b0 0x7f1a1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.987+0000 7f1a4da1d740 1 -- start start 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744b0 0x7f1a1c0769c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a30007fd0 tx=0x7f1a30007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994798270 0x55c994b65210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b7ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994777590 0x55c994b80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c994a9e600 con 0x55c994798270 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c994a3b100 con 0x55c99476f3b0 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c994a9a5c0 con 0x55c994777590 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994798270 0x55c994b65210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994798270 0x55c994b65210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52172/0 (socket says 172.21.15.19:52172) 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 -- 172.21.15.19:0/2779090242 learned_addr learned my addr 172.21.15.19:0/2779090242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a44a00640 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994777590 0x55c994b80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a37e00640 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b7ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 -- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994777590 msgr2=0x55c994b80480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994777590 0x55c994b80480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 -- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 msgr2=0x55c994b7ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b7ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.989+0000 7f1a3fe00640 1 -- 172.21.15.19:0/2779090242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9947b02d0 con 0x55c994798270 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.990+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994798270 0x55c994b65210 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f1a300017b0 tx=0x7f1a30013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96aed5a185ddcb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.990+0000 7f1a2fe00640 1 -- 172.21.15.19:0/2779090242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30026070 con 0x55c994798270 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.990+0000 7f1a2fe00640 1 -- 172.21.15.19:0/2779090242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a30001d30 con 0x55c994798270 2024-04-25T11:39:28.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.990+0000 7f1a2fe00640 1 -- 172.21.15.19:0/2779090242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30013d80 con 0x55c994798270 2024-04-25T11:39:28.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2779090242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994798270 msgr2=0x55c994b65210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994798270 0x55c994b65210 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f1a300017b0 tx=0x7f1a30013000 comp rx=0 tx=0).stop 2024-04-25T11:39:28.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2779090242 shutdown_connections 2024-04-25T11:39:28.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b7ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994777590 0x55c994b80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2779090242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994798270 0x55c994b65210 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2779090242 >> 172.21.15.19:0/2779090242 conn(0x55c99478c3c0 msgr2=0x55c994b3a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:28.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2779090242 shutdown_connections 2024-04-25T11:39:28.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.991+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2779090242 wait complete. 2024-04-25T11:39:28.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.992+0000 7f1a4da1d740 1 -- start start 2024-04-25T11:39:28.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.993+0000 7f1a4da1d740 1 Processor -- start 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.994+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c99476f3b0 0x55c994b67e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.994+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c994777590 0x55c994b53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.994+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b7db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.994+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c994a25f70 con 0x55c99476f3b0 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.994+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c994b7cca0 con 0x55c994777590 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.994+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c994b68410 con 0x55c994798270 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.994+0000 7f1a4da1d740 10 client.0 _ll_register_callbacks cb 0x55c994a94480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c99476f3b0 0x55c994b67e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c99476f3b0 0x55c994b67e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52184/0 (socket says 172.21.15.19:52184) 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 -- 172.21.15.19:0/1488338410 learned_addr learned my addr 172.21.15.19:0/1488338410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:28.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a44a00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b7db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a37e00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c994777590 0x55c994b53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 -- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 msgr2=0x55c994b7db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b7db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 -- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c994777590 msgr2=0x55c994b53960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c994777590 0x55c994b53960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 -- 172.21.15.19:0/1488338410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99487a6b0 con 0x55c99476f3b0 2024-04-25T11:39:28.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a37e00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c994777590 0x55c994b53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c99476f3b0 0x55c994b67e00 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f1a30004c90 tx=0x7f1a30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961bb6339241ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.995+0000 7f1a2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.996+0000 7f1a2cc00640 1 -- 172.21.15.19:0/1488338410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30026040 con 0x55c99476f3b0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.996+0000 7f1a2cc00640 1 -- 172.21.15.19:0/1488338410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f1a30022070 con 0x55c99476f3b0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.996+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a30001d30 con 0x55c99476f3b0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.996+0000 7f1a2cc00640 1 -- 172.21.15.19:0/1488338410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7f1a30022270 con 0x55c99476f3b0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.996+0000 7f1a4da1d740 10 client.9990 fetch_fsmap learned FSMap version 261 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.996+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=261,monmap=2+}) -- 0x55c994b7e070 con 0x55c99476f3b0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.996+0000 7f1a2cc00640 1 -- 172.21.15.19:0/1488338410 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a30001d30 con 0x55c99476f3b0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a2cc00640 1 -- 172.21.15.19:0/1488338410 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 261) ==== 3494+0+0 (secure 0 0 0) 0x7f1a30022540 con 0x55c99476f3b0 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a4da1d740 10 client.9990 fetch_fsmap finished waiting for FSMap version 261 2024-04-25T11:39:28.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a4da1d740 10 client.9990 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:39:28.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a4da1d740 20 client.9990 populate_metadata read hostname 'smithi019' 2024-04-25T11:39:28.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994756e40 0x55c994b7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:28.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a4da1d740 4 client.9990 mds_command: new command op to 9912 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:39:28.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 --> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c994aac210 con 0x55c994756e40 2024-04-25T11:39:28.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.997+0000 7f1a37e00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994756e40 0x55c994b7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:28.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.998+0000 7f1a37e00640 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994756e40 0x55c994b7bbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a216cf8a86d476f server_cookie=16ffe05895c47c53 in_seq=0 out_seq=0 2024-04-25T11:39:28.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.998+0000 7f1a2cc00640 10 client.9990 ms_handle_connect on v2:172.21.15.19:6834/3564168877 2024-04-25T11:39:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.998+0000 7f1a2cc00640 1 -- 172.21.15.19:0/1488338410 <== mds.0 v2:172.21.15.19:6834/3564168877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c994aac210 con 0x55c994756e40 2024-04-25T11:39:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.998+0000 7f1a2cc00640 10 client.9990 handle_command_reply: tid=0 2024-04-25T11:39:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 client.9990 shutdown 2024-04-25T11:39:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c99476f3b0 msgr2=0x55c994b67e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c99476f3b0 0x55c994b67e00 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f1a30004c90 tx=0x7f1a30000bc0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 shutdown_connections 2024-04-25T11:39:28.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994756e40 0x55c994b7bbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c994777590 0x55c994b53960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b7db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/1488338410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c99476f3b0 0x55c994b67e00 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a2cc00640 0 client.9990 ms_handle_reset on v2:172.21.15.19:6834/3564168877 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 >> 172.21.15.19:0/1488338410 conn(0x55c99478c3c0 msgr2=0x55c994b3a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 shutdown_connections 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 1 -- 172.21.15.19:0/1488338410 wait complete. 2024-04-25T11:39:28.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:28.999+0000 7f1a4da1d740 20 client.9990 trim_cache size 0 max 16384 2024-04-25T11:39:29.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.020+0000 7f1a4da1d740 1 Processor -- start 2024-04-25T11:39:29.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.020+0000 7f1a4da1d740 1 -- start start 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994ba3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994ba2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a40203110 con 0x55c994777590 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a40070030 con 0x55c99476f3b0 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a40202af0 con 0x55c994798270 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994ba2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994ba2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52198/0 (socket says 172.21.15.19:52198) 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a37e00640 1 -- 172.21.15.19:0/2673228355 learned_addr learned my addr 172.21.15.19:0/2673228355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:29.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a44a00640 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.022+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994ba3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a37e00640 1 -- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 msgr2=0x55c994b617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:29.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a37e00640 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a37e00640 1 -- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 msgr2=0x55c994ba3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:29.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a37e00640 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994ba3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a37e00640 1 -- 172.21.15.19:0/2673228355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c994b7ed30 con 0x55c994777590 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a44a00640 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a37e00640 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994ba2ca0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f1a2805ee70 tx=0x7f1a2800e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7b4ff9a993d640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a2f400640 1 -- 172.21.15.19:0/2673228355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28021040 con 0x55c994777590 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a2f400640 1 -- 172.21.15.19:0/2673228355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a28078030 con 0x55c994777590 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.023+0000 7f1a2f400640 1 -- 172.21.15.19:0/2673228355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28067e90 con 0x55c994777590 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2673228355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 msgr2=0x55c994ba2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994ba2ca0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f1a2805ee70 tx=0x7f1a2800e8e0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2673228355 shutdown_connections 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994ba2ca0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/2673228355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994ba3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2673228355 >> 172.21.15.19:0/2673228355 conn(0x55c99478c3c0 msgr2=0x55c994b7a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2673228355 shutdown_connections 2024-04-25T11:39:29.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.024+0000 7f1a4da1d740 1 -- 172.21.15.19:0/2673228355 wait complete. 2024-04-25T11:39:29.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.025+0000 7f1a4da1d740 1 -- start start 2024-04-25T11:39:29.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.025+0000 7f1a4da1d740 1 Processor -- start 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b82080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994c04e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a4da1d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b38a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c994b7fad0 con 0x55c994777590 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c994b7e360 con 0x55c99476f3b0 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a4da1d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c994b80090 con 0x55c994798270 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a4da1d740 10 client.0 _ll_register_callbacks cb 0x7f1a30024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994c04e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994c04e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52214/0 (socket says 172.21.15.19:52214) 2024-04-25T11:39:29.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.027+0000 7f1a37e00640 1 -- 172.21.15.19:0/362957214 learned_addr learned my addr 172.21.15.19:0/362957214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b82080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a44a00640 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b38a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a37e00640 1 -- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 msgr2=0x55c994b38a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a37e00640 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b38a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a37e00640 1 -- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 msgr2=0x55c994b82080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a37e00640 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b82080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a37e00640 1 -- 172.21.15.19:0/362957214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c994b3ae30 con 0x55c994777590 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a37e00640 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994c04e40 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c520 tx=0x7f1a28062040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e4e93d7f883d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a2cc00640 1 -- 172.21.15.19:0/362957214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28021040 con 0x55c994777590 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a2cc00640 1 -- 172.21.15.19:0/362957214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f1a2805b400 con 0x55c994777590 2024-04-25T11:39:29.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a28078030 con 0x55c994777590 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.028+0000 7f1a2cc00640 1 -- 172.21.15.19:0/362957214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=261) ==== 24+0+0 (secure 0 0 0) 0x7f1a28067e90 con 0x55c994777590 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 10 client.9996 fetch_fsmap learned FSMap version 261 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=261,monmap=2+}) -- 0x55c994b53170 con 0x55c994777590 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a2cc00640 1 -- 172.21.15.19:0/362957214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a28078030 con 0x55c994777590 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a2cc00640 1 -- 172.21.15.19:0/362957214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 261) ==== 3494+0+0 (secure 0 0 0) 0x7f1a28024060 con 0x55c994777590 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 10 client.9996 fetch_fsmap finished waiting for FSMap version 261 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 10 client.9996 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 20 client.9996 populate_metadata read hostname 'smithi019' 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994b7a7e0 0x55c994b663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 4 client.9996 mds_command: new command op to 9912 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 --> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] -- command(tid 0: {"prefix": "status"}) -- 0x55c994aa6080 con 0x55c994b7a7e0 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.029+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994b7a7e0 0x55c994b663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.030+0000 7f1a3fe00640 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994b7a7e0 0x55c994b663d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f5f44433191b90 server_cookie=78996a4addbedea1 in_seq=0 out_seq=0 2024-04-25T11:39:29.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.030+0000 7f1a2cc00640 10 client.9996 ms_handle_connect on v2:172.21.15.19:6834/3564168877 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.030+0000 7f1a2cc00640 1 -- 172.21.15.19:0/362957214 <== mds.0 v2:172.21.15.19:6834/3564168877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55c994aa6080 con 0x55c994b7a7e0 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.030+0000 7f1a2cc00640 10 client.9996 handle_command_reply: tid=0 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.030+0000 7f1a4da1d740 1 client.9996 shutdown 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 msgr2=0x55c994c04e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994c04e40 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c520 tx=0x7f1a28062040 comp rx=0 tx=0).stop 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 shutdown_connections 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:6834/3564168877,v1:172.21.15.19:6835/3564168877] conn(0x55c994b7a7e0 0x55c994b663d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c994798270 0x55c994b38a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c994777590 0x55c994c04e40 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 --2- 172.21.15.19:0/362957214 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c99476f3b0 0x55c994b82080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a2cc00640 0 client.9996 ms_handle_reset on v2:172.21.15.19:6834/3564168877 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 >> 172.21.15.19:0/362957214 conn(0x55c99478c3c0 msgr2=0x55c994b3c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 shutdown_connections 2024-04-25T11:39:29.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 1 -- 172.21.15.19:0/362957214 wait complete. 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.031+0000 7f1a4da1d740 20 client.9996 trim_cache size 0 max 16384 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 9912, 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3776373770000001, 2024-04-25T11:39:29.022 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 261, 2024-04-25T11:39:29.023 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 118, 2024-04-25T11:39:29.023 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 118, 2024-04-25T11:39:29.023 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.308470926 2024-04-25T11:39:29.023 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744b0 msgr2=0x7f1a1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744b0 0x7f1a1c0769c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a30007fd0 tx=0x7f1a30007e80 comp rx=0 tx=0).stop 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40144370 msgr2=0x7f1a40143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40144370 0x7f1a40143180 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e990 tx=0x7f1a2800ee50 comp rx=0 tx=0).stop 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 shutdown_connections 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a1c0744b0 0x7f1a1c0769c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a400ae640 0x7f1a4013fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a40144370 0x7f1a40143180 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 --2- 172.21.15.19:0/1631482922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a4006c5c0 0x7f1a401624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 >> 172.21.15.19:0/1631482922 conn(0x7f1a400d1720 msgr2=0x7f1a4006ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 shutdown_connections 2024-04-25T11:39:29.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.032+0000 7f1a45400640 1 -- 172.21.15.19:0/1631482922 wait complete. 2024-04-25T11:39:29.047 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:39:29.048 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2024-04-25T11:39:29.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:39:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.338+0000 7f9249a00640 1 Processor -- start 2024-04-25T11:39:29.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.339+0000 7f9249a00640 1 -- start start 2024-04-25T11:39:29.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.341+0000 7f9249a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f924413fe10 0x7f924406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.341+0000 7f9249a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924406df20 con 0x7f9244142e40 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.341+0000 7f9249a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f924415d5e0 con 0x7f924406db10 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.341+0000 7f9249a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9244143250 con 0x7f924413fe10 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9249000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9244142e40 0x7f92440d0030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59576/0 (socket says 172.21.15.19:59576) 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9249000640 1 -- 172.21.15.19:0/2737730256 learned_addr learned my addr 172.21.15.19:0/2737730256 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9243e00640 1 --2- 172.21.15.19:0/2737730256 >> v2:172.21.15.19:3300/0 conn(0x7f924413fe10 0x7f924406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9242a00640 1 -- 172.21.15.19:0/2737730256 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860793894 0 0) 0x7f923c001110 con 0x7f9244142e40 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9242a00640 1 -- 172.21.15.19:0/2737730256 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9228003520 con 0x7f9244142e40 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9243e00640 1 -- 172.21.15.19:0/2737730256 >> v1:172.21.15.169:6789/0 conn(0x7f924406db10 legacy=0x7f9244142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9243e00640 1 -- 172.21.15.19:0/2737730256 >> v1:172.21.15.19:6789/0 conn(0x7f9244142e40 legacy=0x7f92440d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.342+0000 7f9243e00640 1 -- 172.21.15.19:0/2737730256 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924415d8c0 con 0x7f924413fe10 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.344+0000 7f9243e00640 1 --2- 172.21.15.19:0/2737730256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f924406d5d0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d90 tx=0x7f923402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5421a073873794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.344+0000 7f9242a00640 1 -- 172.21.15.19:0/2737730256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234007740 con 0x7f924413fe10 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.344+0000 7f9242a00640 1 -- 172.21.15.19:0/2737730256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92340078e0 con 0x7f924413fe10 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.344+0000 7f9242a00640 1 -- 172.21.15.19:0/2737730256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234007cb0 con 0x7f924413fe10 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.345+0000 7f9249a00640 1 -- 172.21.15.19:0/2737730256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 msgr2=0x7f924406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.345+0000 7f9249a00640 1 --2- 172.21.15.19:0/2737730256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f924406d5d0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d90 tx=0x7f923402a660 comp rx=0 tx=0).stop 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.345+0000 7f9249a00640 1 -- 172.21.15.19:0/2737730256 shutdown_connections 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.345+0000 7f9249a00640 1 --2- 172.21.15.19:0/2737730256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f924406d5d0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.345+0000 7f9249a00640 1 -- 172.21.15.19:0/2737730256 >> 172.21.15.19:0/2737730256 conn(0x7f9244072ee0 msgr2=0x7f92440732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.345+0000 7f9249a00640 1 -- 172.21.15.19:0/2737730256 shutdown_connections 2024-04-25T11:39:29.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.345+0000 7f9249a00640 1 -- 172.21.15.19:0/2737730256 wait complete. 2024-04-25T11:39:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.347+0000 7f9249a00640 1 Processor -- start 2024-04-25T11:39:29.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.347+0000 7f9249a00640 1 -- start start 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9249a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924406db10 0x7f9244140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9249a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f92441f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9249a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9244142e40 0x7f9244158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9249a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f924415d2d0 con 0x7f924413fe10 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9249a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f924415de80 con 0x7f9244142e40 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9249a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f92440d0740 con 0x7f924406db10 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9243400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f92441f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9243400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f92441f76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52226/0 (socket says 172.21.15.19:52226) 2024-04-25T11:39:29.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.349+0000 7f9243400640 1 -- 172.21.15.19:0/3938250876 learned_addr learned my addr 172.21.15.19:0/3938250876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243e00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924406db10 0x7f9244140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243400640 1 -- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924406db10 msgr2=0x7f9244140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243400640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924406db10 0x7f9244140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243400640 1 -- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9244142e40 msgr2=0x7f9244158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243400640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9244142e40 0x7f9244158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243400640 1 -- 172.21.15.19:0/3938250876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9244159070 con 0x7f924413fe10 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243e00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924406db10 0x7f9244140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f9243400640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f92441f76d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c3c0 tx=0x7f922c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f923be00640 1 -- 172.21.15.19:0/3938250876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c0052d0 con 0x7f924413fe10 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.350+0000 7f923be00640 1 -- 172.21.15.19:0/3938250876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f922c005490 con 0x7f924413fe10 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.351+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9244144250 con 0x7f924413fe10 2024-04-25T11:39:29.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.351+0000 7f923be00640 1 -- 172.21.15.19:0/3938250876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c007240 con 0x7f924413fe10 2024-04-25T11:39:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.351+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9244159360 con 0x7f924413fe10 2024-04-25T11:39:29.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.352+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244050740 con 0x7f924413fe10 2024-04-25T11:39:29.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.353+0000 7f923be00640 1 -- 172.21.15.19:0/3938250876 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f922c0073e0 con 0x7f924413fe10 2024-04-25T11:39:29.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.354+0000 7f923be00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92200744e0 0x7f92200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:29.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.354+0000 7f923be00640 1 -- 172.21.15.19:0/3938250876 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f922c095050 con 0x7f924413fe10 2024-04-25T11:39:29.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.355+0000 7f9243e00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92200744e0 0x7f92200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:29.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.356+0000 7f9243e00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92200744e0 0x7f92200769f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9234034000 tx=0x7f9234033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:29.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.360+0000 7f923be00640 1 -- 172.21.15.19:0/3938250876 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f922c05ee80 con 0x7f924413fe10 2024-04-25T11:39:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.611+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f92440b5350 con 0x7f924413fe10 2024-04-25T11:39:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.612+0000 7f923be00640 1 -- 172.21.15.19:0/3938250876 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v261) ==== 53+0+73 (secure 0 0 0) 0x7f922c05e780 con 0x7f924413fe10 2024-04-25T11:39:29.600 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92200744e0 msgr2=0x7f92200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92200744e0 0x7f92200769f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9234034000 tx=0x7f9234033040 comp rx=0 tx=0).stop 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 msgr2=0x7f92441f76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f92441f76d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c3c0 tx=0x7f922c00c880 comp rx=0 tx=0).stop 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 shutdown_connections 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f92200744e0 0x7f92200769f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9244142e40 0x7f9244158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f924413fe10 0x7f92441f76d0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 --2- 172.21.15.19:0/3938250876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f924406db10 0x7f9244140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 >> 172.21.15.19:0/3938250876 conn(0x7f9244072ee0 msgr2=0x7f924413f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 shutdown_connections 2024-04-25T11:39:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29.615+0000 7f9249a00640 1 -- 172.21.15.19:0/3938250876 wait complete. 2024-04-25T11:39:29.626 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:39:29.626 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:39:29.627 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:39:29.627 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:39:29.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:29.627 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:39:29.627 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:39:29.627 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:39:29.627 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:39:29.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:29.627 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:39:29.679 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41904sec preferred_lft 41904sec 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:39:29.680 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:39:29.681 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:39:29.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:29.682 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:39:29.760 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:39:29.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:29.764 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:39:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:39:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:39:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:39:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:39:29.820 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:39:29.821 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:39:29.821 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:39:29.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:39:29.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:39:29.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:39:29.821 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:39:29.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:39:30.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:30.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:30.040 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:39:30.092 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:39:30.092 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:39:30.093 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:39:30.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:30.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:39:30.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:39:30.185 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:30.242 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:39:30.242 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:39:30.242 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:30.294 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:39:30.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:30.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:39:30.512 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:39:30.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:39:30.556 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:39:30.556 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:39:30.556 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:39:30.556 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:39:30.556 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:39:30.558 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:39:30.558 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:39:30.558 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:39:30.558 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:39:30.558 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:39:30.563 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:39:30.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:30.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:39:30.633 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:39:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:39:30.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:39:30.740 INFO:teuthology.orchestra.run.smithi019.stdout:client6590 2024-04-25T11:39:30.740 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:30.740 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:30.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:30.793 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:39:30.799 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:39:30.854 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:39:30.854 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:39:30.854 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:30.854 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:39:30.854 INFO:teuthology.orchestra.run.smithi019.stdout:sixmegs 2024-04-25T11:39:30.855 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2024-04-25T11:39:30.855 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2024-04-25T11:39:30.855 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:30.855 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:30.911 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:30.911 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:30.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:39:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.248+0000 7f457d800640 1 Processor -- start 2024-04-25T11:39:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.248+0000 7f457d800640 1 -- start start 2024-04-25T11:39:31.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f457d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45780b1990 0x7f457806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f457d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f457806cb00 0x7f45780af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f457d800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457806cf10 con 0x7f4578160700 2024-04-25T11:39:31.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f457d800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f45780b7a90 con 0x7f45780b1990 2024-04-25T11:39:31.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f457d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f457806d1a0 con 0x7f457806cb00 2024-04-25T11:39:31.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f4576000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f45780b1990 0x7f457806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f4576a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4578160700 0x7f45780b1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59588/0 (socket says 172.21.15.19:59588) 2024-04-25T11:39:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.251+0000 7f4576a00640 1 -- 172.21.15.19:0/2479804270 learned_addr learned my addr 172.21.15.19:0/2479804270 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.252+0000 7f4577400640 1 --2- 172.21.15.19:0/2479804270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f457806cb00 0x7f45780af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.252+0000 7f4576000640 1 -- 172.21.15.19:0/2479804270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f457806cb00 msgr2=0x7f45780af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.252+0000 7f4576000640 1 --2- 172.21.15.19:0/2479804270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f457806cb00 0x7f45780af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.252+0000 7f4576000640 1 -- 172.21.15.19:0/2479804270 >> v1:172.21.15.19:6789/0 conn(0x7f4578160700 legacy=0x7f45780b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.252+0000 7f4576000640 1 -- 172.21.15.19:0/2479804270 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457806d480 con 0x7f45780b1990 2024-04-25T11:39:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.253+0000 7f4576000640 1 --2- 172.21.15.19:0/2479804270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45780b1990 0x7f457806c5c0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f45600097a0 tx=0x7f456002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f9f82512407303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:31.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.253+0000 7f4575600640 1 -- 172.21.15.19:0/2479804270 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560037070 con 0x7f45780b1990 2024-04-25T11:39:31.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.253+0000 7f4575600640 1 -- 172.21.15.19:0/2479804270 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4560032440 con 0x7f45780b1990 2024-04-25T11:39:31.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.253+0000 7f4575600640 1 -- 172.21.15.19:0/2479804270 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560032810 con 0x7f45780b1990 2024-04-25T11:39:31.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.254+0000 7f457d800640 1 -- 172.21.15.19:0/2479804270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45780b1990 msgr2=0x7f457806c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.254+0000 7f457d800640 1 --2- 172.21.15.19:0/2479804270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45780b1990 0x7f457806c5c0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f45600097a0 tx=0x7f456002a180 comp rx=0 tx=0).stop 2024-04-25T11:39:31.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.254+0000 7f457d800640 1 -- 172.21.15.19:0/2479804270 shutdown_connections 2024-04-25T11:39:31.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.254+0000 7f457d800640 1 --2- 172.21.15.19:0/2479804270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f457806cb00 0x7f45780af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.254+0000 7f457d800640 1 --2- 172.21.15.19:0/2479804270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45780b1990 0x7f457806c5c0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.254+0000 7f457d800640 1 -- 172.21.15.19:0/2479804270 >> 172.21.15.19:0/2479804270 conn(0x7f457809b7d0 msgr2=0x7f45780b7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:31.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.254+0000 7f457d800640 1 -- 172.21.15.19:0/2479804270 shutdown_connections 2024-04-25T11:39:31.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.255+0000 7f457d800640 1 -- 172.21.15.19:0/2479804270 wait complete. 2024-04-25T11:39:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.257+0000 7f457d800640 1 Processor -- start 2024-04-25T11:39:31.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.257+0000 7f457d800640 1 -- start start 2024-04-25T11:39:31.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f457d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f457806cb00 0x7f45780c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f457d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4578160700 0x7f45780c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f457d800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45780d18d0 0x7f45780d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f457d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f457806dd20 con 0x7f457806cb00 2024-04-25T11:39:31.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f457d800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f457806da40 con 0x7f45780d18d0 2024-04-25T11:39:31.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f457d800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f45780af690 con 0x7f4578160700 2024-04-25T11:39:31.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f4576a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f457806cb00 0x7f45780c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.259+0000 7f4576a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f457806cb00 0x7f45780c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52250/0 (socket says 172.21.15.19:52250) 2024-04-25T11:39:31.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576a00640 1 -- 172.21.15.19:0/316009111 learned_addr learned my addr 172.21.15.19:0/316009111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4577400640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45780d18d0 0x7f45780d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576000640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4578160700 0x7f45780c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576a00640 1 -- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4578160700 msgr2=0x7f45780c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576a00640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4578160700 0x7f45780c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576a00640 1 -- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45780d18d0 msgr2=0x7f45780d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576a00640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45780d18d0 0x7f45780d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576a00640 1 -- 172.21.15.19:0/316009111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45780d0830 con 0x7f457806cb00 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.260+0000 7f4576a00640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f457806cb00 0x7f45780c9250 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f456800cad0 tx=0x7f456800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:31.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.261+0000 7f456ea00640 1 -- 172.21.15.19:0/316009111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568017070 con 0x7f457806cb00 2024-04-25T11:39:31.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.261+0000 7f456ea00640 1 -- 172.21.15.19:0/316009111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456800ab70 con 0x7f457806cb00 2024-04-25T11:39:31.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.261+0000 7f456ea00640 1 -- 172.21.15.19:0/316009111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568012400 con 0x7f457806cb00 2024-04-25T11:39:31.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.261+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45780d0b10 con 0x7f457806cb00 2024-04-25T11:39:31.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.261+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45780d10d0 con 0x7f457806cb00 2024-04-25T11:39:31.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.262+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4578050740 con 0x7f457806cb00 2024-04-25T11:39:31.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.263+0000 7f456ea00640 1 -- 172.21.15.19:0/316009111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f456801a070 con 0x7f457806cb00 2024-04-25T11:39:31.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.264+0000 7f456ea00640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45540744e0 0x7f45540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.264+0000 7f456ea00640 1 -- 172.21.15.19:0/316009111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f4568095860 con 0x7f457806cb00 2024-04-25T11:39:31.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.265+0000 7f4576000640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45540744e0 0x7f45540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.266+0000 7f4576000640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45540744e0 0x7f45540769f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4560034000 tx=0x7f456002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:31.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.269+0000 7f456ea00640 1 -- 172.21.15.19:0/316009111 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f456805f690 con 0x7f457806cb00 2024-04-25T11:39:31.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.476+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45780b4a60 con 0x7f457806cb00 2024-04-25T11:39:31.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.479+0000 7f456ea00640 1 -- 172.21.15.19:0/316009111 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+20034 (secure 0 0 0) 0x7f456805ef90 con 0x7f457806cb00 2024-04-25T11:39:31.468 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:31.468 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":118,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:39:15.575643+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:39:31.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45540744e0 msgr2=0x7f45540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45540744e0 0x7f45540769f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4560034000 tx=0x7f456002e000 comp rx=0 tx=0).stop 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f457806cb00 msgr2=0x7f45780c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f457806cb00 0x7f45780c9250 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f456800cad0 tx=0x7f456800cf90 comp rx=0 tx=0).stop 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 shutdown_connections 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45540744e0 0x7f45540769f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f45780d18d0 0x7f45780d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4578160700 0x7f45780c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 --2- 172.21.15.19:0/316009111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f457806cb00 0x7f45780c9250 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.482+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 >> 172.21.15.19:0/316009111 conn(0x7f457809b7d0 msgr2=0x7f45780ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.483+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 shutdown_connections 2024-04-25T11:39:31.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.483+0000 7f457d800640 1 -- 172.21.15.19:0/316009111 wait complete. 2024-04-25T11:39:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:39:31.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:39:31.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.802+0000 7f1be3600640 1 Processor -- start 2024-04-25T11:39:31.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.802+0000 7f1be3600640 1 -- start start 2024-04-25T11:39:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1be3600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bdc06dce0 0x7f1bdc0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1be3600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc15adf0 0x7f1bdc160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1be3600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bdc071110 con 0x7f1bdc070630 2024-04-25T11:39:31.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1be3600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1bdc0713f0 con 0x7f1bdc06dce0 2024-04-25T11:39:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1be3600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1bdc0701a0 con 0x7f1bdc15adf0 2024-04-25T11:39:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1bdbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bdc06dce0 0x7f1bdc0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1bdbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1bdc06dce0 0x7f1bdc0cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52256/0 (socket says 172.21.15.19:52256) 2024-04-25T11:39:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.805+0000 7f1bdbe00640 1 -- 172.21.15.19:0/1747157703 learned_addr learned my addr 172.21.15.19:0/1747157703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.806+0000 7f1be1600640 1 --2- 172.21.15.19:0/1747157703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc15adf0 0x7f1bdc160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.806+0000 7f1bdbe00640 1 -- 172.21.15.19:0/1747157703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc15adf0 msgr2=0x7f1bdc160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.806+0000 7f1bdbe00640 1 --2- 172.21.15.19:0/1747157703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc15adf0 0x7f1bdc160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.806+0000 7f1bdbe00640 1 -- 172.21.15.19:0/1747157703 >> v1:172.21.15.169:6789/0 conn(0x7f1bdc070630 legacy=0x7f1bdc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.806+0000 7f1bdbe00640 1 -- 172.21.15.19:0/1747157703 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc06fb50 con 0x7f1bdc06dce0 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.807+0000 7f1bdbe00640 1 --2- 172.21.15.19:0/1747157703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc06dce0 0x7f1bdc0cec50 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a87b9aedf55be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.807+0000 7f1bdb400640 1 -- 172.21.15.19:0/1747157703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037070 con 0x7f1bdc06dce0 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.807+0000 7f1bdb400640 1 -- 172.21.15.19:0/1747157703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8032440 con 0x7f1bdc06dce0 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.808+0000 7f1bdb400640 1 -- 172.21.15.19:0/1747157703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8032810 con 0x7f1bdc06dce0 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.808+0000 7f1be3600640 1 -- 172.21.15.19:0/1747157703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc06dce0 msgr2=0x7f1bdc0cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.808+0000 7f1be3600640 1 --2- 172.21.15.19:0/1747157703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc06dce0 0x7f1bdc0cec50 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.808+0000 7f1be3600640 1 -- 172.21.15.19:0/1747157703 shutdown_connections 2024-04-25T11:39:31.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.808+0000 7f1be3600640 1 --2- 172.21.15.19:0/1747157703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc15adf0 0x7f1bdc160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.808+0000 7f1be3600640 1 --2- 172.21.15.19:0/1747157703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc06dce0 0x7f1bdc0cec50 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.808+0000 7f1be3600640 1 -- 172.21.15.19:0/1747157703 >> 172.21.15.19:0/1747157703 conn(0x7f1bdc073090 msgr2=0x7f1bdc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.809+0000 7f1be3600640 1 -- 172.21.15.19:0/1747157703 shutdown_connections 2024-04-25T11:39:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.809+0000 7f1be3600640 1 -- 172.21.15.19:0/1747157703 wait complete. 2024-04-25T11:39:31.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.811+0000 7f1be3600640 1 Processor -- start 2024-04-25T11:39:31.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.811+0000 7f1be3600640 1 -- start start 2024-04-25T11:39:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be3600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc070630 0x7f1bdc14e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be3600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc15adf0 0x7f1bdc0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be3600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc1597c0 0x7f1bdc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be3600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc0b0e80 con 0x7f1bdc15adf0 2024-04-25T11:39:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be3600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc0b0d00 con 0x7f1bdc070630 2024-04-25T11:39:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be3600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1bdc070c80 con 0x7f1bdc1597c0 2024-04-25T11:39:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc15adf0 0x7f1bdc0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc15adf0 0x7f1bdc0b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52270/0 (socket says 172.21.15.19:52270) 2024-04-25T11:39:31.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1bdbe00640 1 -- 172.21.15.19:0/3938556321 learned_addr learned my addr 172.21.15.19:0/3938556321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be0c00640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc070630 0x7f1bdc14e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.813+0000 7f1be1600640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc1597c0 0x7f1bdc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bdbe00640 1 -- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc1597c0 msgr2=0x7f1bdc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bdbe00640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc1597c0 0x7f1bdc15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bdbe00640 1 -- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc070630 msgr2=0x7f1bdc14e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:31.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bdbe00640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc070630 0x7f1bdc14e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:31.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bdbe00640 1 -- 172.21.15.19:0/3938556321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc0b1650 con 0x7f1bdc15adf0 2024-04-25T11:39:31.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bdbe00640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc15adf0 0x7f1bdc0b26f0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8000c00 tx=0x7f1bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bd8c00640 1 -- 172.21.15.19:0/3938556321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8041020 con 0x7f1bdc15adf0 2024-04-25T11:39:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bd8c00640 1 -- 172.21.15.19:0/3938556321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc80306c0 con 0x7f1bdc15adf0 2024-04-25T11:39:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.814+0000 7f1bd8c00640 1 -- 172.21.15.19:0/3938556321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037040 con 0x7f1bdc15adf0 2024-04-25T11:39:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.815+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc0b1930 con 0x7f1bdc15adf0 2024-04-25T11:39:31.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.815+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc0b1ef0 con 0x7f1bdc15adf0 2024-04-25T11:39:31.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.816+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc050740 con 0x7f1bdc15adf0 2024-04-25T11:39:31.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.817+0000 7f1bd8c00640 1 -- 172.21.15.19:0/3938556321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1bc8044020 con 0x7f1bdc15adf0 2024-04-25T11:39:31.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.818+0000 7f1bd8c00640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1bbc0744b0 0x7f1bbc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:31.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.819+0000 7f1bd8c00640 1 -- 172.21.15.19:0/3938556321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f1bc80b62e0 con 0x7f1bdc15adf0 2024-04-25T11:39:31.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.819+0000 7f1be0c00640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1bbc0744b0 0x7f1bbc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:31.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.821+0000 7f1be0c00640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1bbc0744b0 0x7f1bbc0769c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007fd0 tx=0x7f1bd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:31.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:31.823+0000 7f1bd8c00640 1 -- 172.21.15.19:0/3938556321 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bc804b030 con 0x7f1bdc15adf0 2024-04-25T11:39:32.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.025+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1bdc0b13e0 con 0x7f1bdc15adf0 2024-04-25T11:39:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.026+0000 7f1bd8c00640 1 -- 172.21.15.19:0/3938556321 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f1bc8080230 con 0x7f1bdc15adf0 2024-04-25T11:39:32.014 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:39:32.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1bbc0744b0 msgr2=0x7f1bbc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1bbc0744b0 0x7f1bbc0769c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007fd0 tx=0x7f1bd0009040 comp rx=0 tx=0).stop 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc15adf0 msgr2=0x7f1bdc0b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc15adf0 0x7f1bdc0b26f0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8000c00 tx=0x7f1bc80046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 shutdown_connections 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1bbc0744b0 0x7f1bbc0769c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1bdc1597c0 0x7f1bdc15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bdc15adf0 0x7f1bdc0b26f0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 --2- 172.21.15.19:0/3938556321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bdc070630 0x7f1bdc14e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 >> 172.21.15.19:0/3938556321 conn(0x7f1bdc073090 msgr2=0x7f1bdc0b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.029+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 shutdown_connections 2024-04-25T11:39:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.030+0000 7f1be3600640 1 -- 172.21.15.19:0/3938556321 wait complete. 2024-04-25T11:39:32.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:39:32.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.335+0000 7fcd19000640 1 Processor -- start 2024-04-25T11:39:32.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.335+0000 7fcd19000640 1 -- start start 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd19000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd1413fcb0 0x7fcd140701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd19000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd14143000 0x7fcd140d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd19000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd14070b20 con 0x7fcd14070710 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd19000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1415e430 con 0x7fcd1413fcb0 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd19000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd140ce630 con 0x7fcd14143000 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd13400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd1413fcb0 0x7fcd140701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd0aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcd14070710 0x7fcd141428f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59590/0 (socket says 172.21.15.19:59590) 2024-04-25T11:39:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.338+0000 7fcd0aa00640 1 -- 172.21.15.19:0/2786458153 learned_addr learned my addr 172.21.15.19:0/2786458153 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.339+0000 7fcd13e00640 1 --2- 172.21.15.19:0/2786458153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd14143000 0x7fcd140d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.339+0000 7fcd13400640 1 -- 172.21.15.19:0/2786458153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd14143000 msgr2=0x7fcd140d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.339+0000 7fcd13400640 1 --2- 172.21.15.19:0/2786458153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd14143000 0x7fcd140d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.339+0000 7fcd13400640 1 -- 172.21.15.19:0/2786458153 >> v1:172.21.15.19:6789/0 conn(0x7fcd14070710 legacy=0x7fcd141428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.339+0000 7fcd13400640 1 -- 172.21.15.19:0/2786458153 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd140cda60 con 0x7fcd1413fcb0 2024-04-25T11:39:32.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.339+0000 7fcd12a00640 1 -- 172.21.15.19:0/2786458153 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4086079336 0 0) 0x7fccfc001490 con 0x7fcd14070710 2024-04-25T11:39:32.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.340+0000 7fcd13400640 1 --2- 172.21.15.19:0/2786458153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1413fcb0 0x7fcd140701d0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd0402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4c34e7991a5fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.340+0000 7fcd12a00640 1 -- 172.21.15.19:0/2786458153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x7fcd1413fcb0 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.340+0000 7fcd12a00640 1 -- 172.21.15.19:0/2786458153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd04032440 con 0x7fcd1413fcb0 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.340+0000 7fcd12a00640 1 -- 172.21.15.19:0/2786458153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04032810 con 0x7fcd1413fcb0 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 -- 172.21.15.19:0/2786458153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1413fcb0 msgr2=0x7fcd140701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 --2- 172.21.15.19:0/2786458153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1413fcb0 0x7fcd140701d0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd0402a180 comp rx=0 tx=0).stop 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 -- 172.21.15.19:0/2786458153 shutdown_connections 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 --2- 172.21.15.19:0/2786458153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd14143000 0x7fcd140d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 --2- 172.21.15.19:0/2786458153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1413fcb0 0x7fcd140701d0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 -- 172.21.15.19:0/2786458153 >> 172.21.15.19:0/2786458153 conn(0x7fcd14072f40 msgr2=0x7fcd14073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 -- 172.21.15.19:0/2786458153 shutdown_connections 2024-04-25T11:39:32.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.341+0000 7fcd19000640 1 -- 172.21.15.19:0/2786458153 wait complete. 2024-04-25T11:39:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.344+0000 7fcd19000640 1 Processor -- start 2024-04-25T11:39:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.344+0000 7fcd19000640 1 -- start start 2024-04-25T11:39:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.345+0000 7fcd19000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14070710 0x7fcd141555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.345+0000 7fcd19000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1413fcb0 0x7fcd141f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.345+0000 7fcd19000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14143000 0x7fcd141371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.345+0000 7fcd19000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd140ce2c0 con 0x7fcd14143000 2024-04-25T11:39:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.345+0000 7fcd19000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd140cdfe0 con 0x7fcd14070710 2024-04-25T11:39:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.345+0000 7fcd19000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd140ceac0 con 0x7fcd1413fcb0 2024-04-25T11:39:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14143000 0x7fcd141371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14143000 0x7fcd141371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52288/0 (socket says 172.21.15.19:52288) 2024-04-25T11:39:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 -- 172.21.15.19:0/2892482203 learned_addr learned my addr 172.21.15.19:0/2892482203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13400640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14070710 0x7fcd141555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd0aa00640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1413fcb0 0x7fcd141f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 -- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1413fcb0 msgr2=0x7fcd141f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1413fcb0 0x7fcd141f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 -- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14070710 msgr2=0x7fcd141555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14070710 0x7fcd141555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd13e00640 1 -- 172.21.15.19:0/2892482203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd14156160 con 0x7fcd14143000 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.346+0000 7fcd0aa00640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1413fcb0 0x7fcd141f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.347+0000 7fcd13e00640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14143000 0x7fcd141371c0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c5c0 tx=0x7fcd0c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.347+0000 7fcd0be00640 1 -- 172.21.15.19:0/2892482203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c015070 con 0x7fcd14143000 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.347+0000 7fcd0be00640 1 -- 172.21.15.19:0/2892482203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c004220 con 0x7fcd14143000 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.347+0000 7fcd0be00640 1 -- 172.21.15.19:0/2892482203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c003610 con 0x7fcd14143000 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.347+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd14156440 con 0x7fcd14143000 2024-04-25T11:39:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.347+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd14156a00 con 0x7fcd14143000 2024-04-25T11:39:32.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.348+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd14155ed0 con 0x7fcd14143000 2024-04-25T11:39:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.350+0000 7fcd0be00640 1 -- 172.21.15.19:0/2892482203 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd0c0037b0 con 0x7fcd14143000 2024-04-25T11:39:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.351+0000 7fcd0be00640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccf00744b0 0x7fccf00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.351+0000 7fcd0be00640 1 -- 172.21.15.19:0/2892482203 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fcd0c095830 con 0x7fcd14143000 2024-04-25T11:39:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.351+0000 7fcd13400640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccf00744b0 0x7fccf00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.353+0000 7fcd13400640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccf00744b0 0x7fccf00769c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcd04034000 tx=0x7fcd0402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:32.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.356+0000 7fcd0be00640 1 -- 172.21.15.19:0/2892482203 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd0c05f7c0 con 0x7fcd14143000 2024-04-25T11:39:32.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.561+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd1406d9c0 con 0x7fcd14143000 2024-04-25T11:39:32.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.562+0000 7fcd0be00640 1 -- 172.21.15.19:0/2892482203 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7fcd0c05f0c0 con 0x7fcd14143000 2024-04-25T11:39:32.551 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:39:32.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccf00744b0 msgr2=0x7fccf00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccf00744b0 0x7fccf00769c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcd04034000 tx=0x7fcd0402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:32.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14143000 msgr2=0x7fcd141371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14143000 0x7fcd141371c0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c5c0 tx=0x7fcd0c00ca80 comp rx=0 tx=0).stop 2024-04-25T11:39:32.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 shutdown_connections 2024-04-25T11:39:32.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccf00744b0 0x7fccf00769c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd14143000 0x7fcd141371c0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd1413fcb0 0x7fcd141f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 --2- 172.21.15.19:0/2892482203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd14070710 0x7fcd141555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 >> 172.21.15.19:0/2892482203 conn(0x7fcd14072f40 msgr2=0x7fcd1406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:32.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 shutdown_connections 2024-04-25T11:39:32.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.565+0000 7fcd19000640 1 -- 172.21.15.19:0/2892482203 wait complete. 2024-04-25T11:39:32.579 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:39:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.872+0000 7ff34b200640 1 Processor -- start 2024-04-25T11:39:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.872+0000 7ff34b200640 1 -- start start 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.874+0000 7ff34b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c070d20 0x7ff34c161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.874+0000 7ff34b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff34c06f930 0x7ff34c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff34b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c1623f0 con 0x7ff34c06fd40 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff34b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff34c1626d0 con 0x7ff34c06f930 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff34b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c06f130 con 0x7ff34c070d20 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff34a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff34c06fd40 0x7ff34c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59602/0 (socket says 172.21.15.19:59602) 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff34a800640 1 -- 172.21.15.19:0/107739573 learned_addr learned my addr 172.21.15.19:0/107739573 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:32.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff343e00640 1 --2- 172.21.15.19:0/107739573 >> v2:172.21.15.19:3300/0 conn(0x7ff34c06f930 0x7ff34c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff349e00640 1 --2- 172.21.15.19:0/107739573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c070d20 0x7ff34c161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.875+0000 7ff349400640 1 -- 172.21.15.19:0/107739573 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914342373 0 0) 0x7ff34c1623f0 con 0x7ff34c06fd40 2024-04-25T11:39:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.876+0000 7ff349400640 1 -- 172.21.15.19:0/107739573 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff330003520 con 0x7ff34c06fd40 2024-04-25T11:39:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.876+0000 7ff343e00640 1 -- 172.21.15.19:0/107739573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c070d20 msgr2=0x7ff34c161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.876+0000 7ff343e00640 1 --2- 172.21.15.19:0/107739573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c070d20 0x7ff34c161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.876+0000 7ff343e00640 1 -- 172.21.15.19:0/107739573 >> v1:172.21.15.19:6789/0 conn(0x7ff34c06fd40 legacy=0x7ff34c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.876+0000 7ff343e00640 1 -- 172.21.15.19:0/107739573 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c06f410 con 0x7ff34c06f930 2024-04-25T11:39:32.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.877+0000 7ff343e00640 1 --2- 172.21.15.19:0/107739573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c06d5d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097a0 tx=0x7ff33402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf8099b58a01603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:32.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.877+0000 7ff349400640 1 -- 172.21.15.19:0/107739573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334037070 con 0x7ff34c06f930 2024-04-25T11:39:32.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.877+0000 7ff349400640 1 -- 172.21.15.19:0/107739573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33402ae50 con 0x7ff34c06f930 2024-04-25T11:39:32.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.877+0000 7ff349400640 1 -- 172.21.15.19:0/107739573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334032660 con 0x7ff34c06f930 2024-04-25T11:39:32.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 -- 172.21.15.19:0/107739573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 msgr2=0x7ff34c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 --2- 172.21.15.19:0/107739573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c06d5d0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ff3340097a0 tx=0x7ff33402a100 comp rx=0 tx=0).stop 2024-04-25T11:39:32.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 -- 172.21.15.19:0/107739573 shutdown_connections 2024-04-25T11:39:32.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 --2- 172.21.15.19:0/107739573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c06d5d0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 --2- 172.21.15.19:0/107739573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c070d20 0x7ff34c161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 -- 172.21.15.19:0/107739573 >> 172.21.15.19:0/107739573 conn(0x7ff34c073160 msgr2=0x7ff34c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:32.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 -- 172.21.15.19:0/107739573 shutdown_connections 2024-04-25T11:39:32.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.878+0000 7ff34b200640 1 -- 172.21.15.19:0/107739573 wait complete. 2024-04-25T11:39:32.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.881+0000 7ff34b200640 1 Processor -- start 2024-04-25T11:39:32.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.881+0000 7ff34b200640 1 -- start start 2024-04-25T11:39:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.882+0000 7ff34b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.882+0000 7ff34b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c06fd40 0x7ff34c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.882+0000 7ff34b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c070d20 0x7ff34c0b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.882+0000 7ff34b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c15e2f0 con 0x7ff34c06f930 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.882+0000 7ff34b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c162070 con 0x7ff34c070d20 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.882+0000 7ff34b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c15e470 con 0x7ff34c06fd40 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52318/0 (socket says 172.21.15.19:52318) 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 -- 172.21.15.19:0/555360025 learned_addr learned my addr 172.21.15.19:0/555360025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff34a800640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c070d20 0x7ff34c0b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff343e00640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c06fd40 0x7ff34c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 -- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c06fd40 msgr2=0x7ff34c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c06fd40 0x7ff34c0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 -- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c070d20 msgr2=0x7ff34c0b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c070d20 0x7ff34c0b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff349e00640 1 -- 172.21.15.19:0/555360025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0cbaa0 con 0x7ff34c06f930 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.883+0000 7ff34a800640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c070d20 0x7ff34c0b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.884+0000 7ff349e00640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c148fa0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d600 tx=0x7ff33c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.884+0000 7ff342000640 1 -- 172.21.15.19:0/555360025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c004240 con 0x7ff34c06f930 2024-04-25T11:39:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.884+0000 7ff342000640 1 -- 172.21.15.19:0/555360025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33c0043e0 con 0x7ff34c06f930 2024-04-25T11:39:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.884+0000 7ff342000640 1 -- 172.21.15.19:0/555360025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c003830 con 0x7ff34c06f930 2024-04-25T11:39:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.884+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c0caee0 con 0x7ff34c06f930 2024-04-25T11:39:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.884+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c0b6310 con 0x7ff34c06f930 2024-04-25T11:39:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.885+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c0cacd0 con 0x7ff34c06f930 2024-04-25T11:39:32.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.886+0000 7ff342000640 1 -- 172.21.15.19:0/555360025 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff33c0039d0 con 0x7ff34c06f930 2024-04-25T11:39:32.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.887+0000 7ff342000640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3280744b0 0x7ff3280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:32.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.888+0000 7ff342000640 1 -- 172.21.15.19:0/555360025 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7ff33c095e00 con 0x7ff34c06f930 2024-04-25T11:39:32.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.888+0000 7ff343e00640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3280744b0 0x7ff3280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.889+0000 7ff343e00640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3280744b0 0x7ff3280769c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:32.892+0000 7ff342000640 1 -- 172.21.15.19:0/555360025 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff33c05fcc0 con 0x7ff34c06f930 2024-04-25T11:39:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.098+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff34c076c60 con 0x7ff34c06f930 2024-04-25T11:39:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.099+0000 7ff342000640 1 -- 172.21.15.19:0/555360025 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7ff33c05f5c0 con 0x7ff34c06f930 2024-04-25T11:39:33.087 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3280744b0 msgr2=0x7ff3280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3280744b0 0x7ff3280769c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 msgr2=0x7ff34c148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c148fa0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d600 tx=0x7ff33c00dac0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 shutdown_connections 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3280744b0 0x7ff3280769c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34c070d20 0x7ff34c0b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff34c06fd40 0x7ff34c0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 --2- 172.21.15.19:0/555360025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34c06f930 0x7ff34c148fa0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 >> 172.21.15.19:0/555360025 conn(0x7ff34c073160 msgr2=0x7ff34c15c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 shutdown_connections 2024-04-25T11:39:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.102+0000 7ff34b200640 1 -- 172.21.15.19:0/555360025 wait complete. 2024-04-25T11:39:33.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:39:33.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.404+0000 7fbad2400640 1 Processor -- start 2024-04-25T11:39:33.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.404+0000 7fbad2400640 1 -- start start 2024-04-25T11:39:33.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.407+0000 7fbad2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbacc0b4ea0 0x7fbacc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.407+0000 7fbad2400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc0d1230 con 0x7fbacc144370 2024-04-25T11:39:33.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.407+0000 7fbad2400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc0d06b0 con 0x7fbacc06c5c0 2024-04-25T11:39:33.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.407+0000 7fbad2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc0d0990 con 0x7fbacc0b4ea0 2024-04-25T11:39:33.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.407+0000 7fbad1a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbacc144370 0x7fbacc15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59612/0 (socket says 172.21.15.19:59612) 2024-04-25T11:39:33.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.407+0000 7fbad1a00640 1 -- 172.21.15.19:0/2912112972 learned_addr learned my addr 172.21.15.19:0/2912112972 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.407+0000 7fbacbe00640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbacc0b4ea0 0x7fbacc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114986476 0 0) 0x7fbacc0d1230 con 0x7fbacc144370 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbab4003520 con 0x7fbacc144370 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3820940111 0 0) 0x7fbab4003520 con 0x7fbacc144370 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbacc0b4ea0 msgr2=0x7fbacc0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbacc0b4ea0 0x7fbacc0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 >> v1:172.21.15.169:6789/0 conn(0x7fbacc06c5c0 legacy=0x7fbacc0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc0d0c70 con 0x7fbacc144370 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fbac40019f0 con 0x7fbacc144370 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 >> v1:172.21.15.19:6789/0 conn(0x7fbacc144370 legacy=0x7fbacc15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab40039e0 0x7fbab4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbab4004800 0x7fbab4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab4005270 0x7fbab400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0d0990 con 0x7fbab40039e0 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbab400bf50 con 0x7fbab4004800 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbab400c150 con 0x7fbab4005270 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.408+0000 7fbad1000640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab40039e0 0x7fbab4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.409+0000 7fbacbe00640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbab4004800 0x7fbab4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.409+0000 7fbad1000640 1 -- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab4005270 msgr2=0x7fbab400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.409+0000 7fbad1000640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab4005270 0x7fbab400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.409+0000 7fbad1000640 1 -- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbab4004800 msgr2=0x7fbab4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.409+0000 7fbad1000640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbab4004800 0x7fbab4004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.409+0000 7fbad1000640 1 -- 172.21.15.19:0/2912112972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab4003520 con 0x7fbab40039e0 2024-04-25T11:39:33.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.410+0000 7fbad1000640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab40039e0 0x7fbab4003e50 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fbac0001750 tx=0x7fbab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddca6b4031687623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:33.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.410+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0001370 con 0x7fbab40039e0 2024-04-25T11:39:33.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.410+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac00052d0 con 0x7fbab40039e0 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.410+0000 7fbacb400640 1 -- 172.21.15.19:0/2912112972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac002abe0 con 0x7fbab40039e0 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.411+0000 7fbad2400640 1 -- 172.21.15.19:0/2912112972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab40039e0 msgr2=0x7fbab4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.411+0000 7fbad2400640 1 --2- 172.21.15.19:0/2912112972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab40039e0 0x7fbab4003e50 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fbac0001750 tx=0x7fbab4005830 comp rx=0 tx=0).stop 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.411+0000 7fbad1a00640 1 -- 172.21.15.19:0/2912112972 reap_dead start 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.411+0000 7fbad2400640 1 -- 172.21.15.19:0/2912112972 shutdown_connections 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.411+0000 7fbad2400640 1 -- 172.21.15.19:0/2912112972 >> 172.21.15.19:0/2912112972 conn(0x7fbacc0d1720 msgr2=0x7fbacc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.411+0000 7fbad2400640 1 -- 172.21.15.19:0/2912112972 shutdown_connections 2024-04-25T11:39:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.411+0000 7fbad2400640 1 -- 172.21.15.19:0/2912112972 wait complete. 2024-04-25T11:39:33.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.413+0000 7fbad2400640 1 Processor -- start 2024-04-25T11:39:33.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.414+0000 7fbad2400640 1 -- start start 2024-04-25T11:39:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab40039e0 0x7fbacc13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad2400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbacc13ffe0 0x7fbacc1303b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1308f0 0x7fbacc15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad2400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0d0450 con 0x7fbacc1308f0 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad2400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0d02d0 con 0x7fbacc13ffe0 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc147760 con 0x7fbab40039e0 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1308f0 0x7fbacc15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1308f0 0x7fbacc15f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52336/0 (socket says 172.21.15.19:52336) 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.415+0000 7fbad1a00640 1 -- 172.21.15.19:0/305621164 learned_addr learned my addr 172.21.15.19:0/305621164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbacbe00640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbacc13ffe0 0x7fbacc1303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad1000640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab40039e0 0x7fbacc13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad1a00640 1 -- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab40039e0 msgr2=0x7fbacc13faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad1a00640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab40039e0 0x7fbacc13faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad1a00640 1 -- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbacc13ffe0 msgr2=0x7fbacc1303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad1a00640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbacc13ffe0 0x7fbacc1303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad1a00640 1 -- 172.21.15.19:0/305621164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc1403f0 con 0x7fbacc1308f0 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad1a00640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1308f0 0x7fbacc15f910 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b7a50 tx=0x7fbac400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbac8c00640 1 -- 172.21.15.19:0/305621164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac400a5b0 con 0x7fbacc1308f0 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbac8c00640 1 -- 172.21.15.19:0/305621164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac400a770 con 0x7fbacc1308f0 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbac8c00640 1 -- 172.21.15.19:0/305621164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac400ab60 con 0x7fbacc1308f0 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.416+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc12e830 con 0x7fbacc1308f0 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.417+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc1317b0 con 0x7fbacc1308f0 2024-04-25T11:39:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.418+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc12ea00 con 0x7fbacc1308f0 2024-04-25T11:39:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.419+0000 7fbac8c00640 1 -- 172.21.15.19:0/305621164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbac400ad00 con 0x7fbacc1308f0 2024-04-25T11:39:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.420+0000 7fbac8c00640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbab00744b0 0x7fbab00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.420+0000 7fbac8c00640 1 -- 172.21.15.19:0/305621164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7fbac40950b0 con 0x7fbacc1308f0 2024-04-25T11:39:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.421+0000 7fbad1000640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbab00744b0 0x7fbab00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:33.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.422+0000 7fbad1000640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbab00744b0 0x7fbab00769c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbac0032000 tx=0x7fbac002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.425+0000 7fbac8c00640 1 -- 172.21.15.19:0/305621164 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbac401a030 con 0x7fbacc1308f0 2024-04-25T11:39:33.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.643+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbacc131240 con 0x7fbacc1308f0 2024-04-25T11:39:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.645+0000 7fbac8c00640 1 -- 172.21.15.19:0/305621164 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 52 entries v118) ==== 81+0+3154 (secure 0 0 0) 0x7fbacc1403f0 con 0x7fbacc1308f0 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:39:33.633 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:39:33.634 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:39:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:39:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:39:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:listed 52 entries 2024-04-25T11:39:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbab00744b0 msgr2=0x7fbab00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbab00744b0 0x7fbab00769c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbac0032000 tx=0x7fbac002e000 comp rx=0 tx=0).stop 2024-04-25T11:39:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1308f0 msgr2=0x7fbacc15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1308f0 0x7fbacc15f910 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b7a50 tx=0x7fbac400ccd0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 shutdown_connections 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbab00744b0 0x7fbab00769c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbacc13ffe0 0x7fbacc1303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbacc1308f0 0x7fbacc15f910 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 --2- 172.21.15.19:0/305621164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbab40039e0 0x7fbacc13faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.647+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 >> 172.21.15.19:0/305621164 conn(0x7fbacc0d1720 msgr2=0x7fbacc15d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.648+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 shutdown_connections 2024-04-25T11:39:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:33.648+0000 7fbad2400640 1 -- 172.21.15.19:0/305621164 wait complete. 2024-04-25T11:39:33.665 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:39:33.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:33.665 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:39:33.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:33.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:33.778 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:33.830 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.120+0000 7f1286800640 1 Processor -- start 2024-04-25T11:39:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.121+0000 7f1286800640 1 -- start start 2024-04-25T11:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.123+0000 7f1286800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800b4ea0 0x7f12800cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.123+0000 7f1286800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12800d1230 con 0x7f1280144370 2024-04-25T11:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.123+0000 7f1286800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12800d06b0 con 0x7f128006c5c0 2024-04-25T11:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.123+0000 7f1286800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12800d0990 con 0x7f12800b4ea0 2024-04-25T11:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.123+0000 7f1285e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1280144370 0x7f128015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58002/0 (socket says 172.21.15.19:58002) 2024-04-25T11:39:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.123+0000 7f1285e00640 1 -- 172.21.15.19:0/2864969177 learned_addr learned my addr 172.21.15.19:0/2864969177 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.123+0000 7f1284a00640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800b4ea0 0x7f12800cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939253815 0 0) 0x7f12800d1230 con 0x7f1280144370 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1268003520 con 0x7f1280144370 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 723636742 0 0) 0x7f1268003520 con 0x7f1280144370 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800b4ea0 msgr2=0x7f12800cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800b4ea0 0x7f12800cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 >> v1:172.21.15.169:6789/0 conn(0x7f128006c5c0 legacy=0x7f12800b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12800d0c70 con 0x7f1280144370 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f12780019f0 con 0x7f1280144370 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 >> v1:172.21.15.19:6789/0 conn(0x7f1280144370 legacy=0x7f128015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12680039e0 0x7f1268003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1268004800 0x7f1268004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1268005150 0x7f126800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12800d0990 con 0x7f12680039e0 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f126800bf00 con 0x7f1268005150 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f126800c100 con 0x7f1268004800 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.124+0000 7f1285400640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12680039e0 0x7f1268003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.125+0000 7f1285400640 1 -- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1268004800 msgr2=0x7f1268004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.125+0000 7f1285400640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1268004800 0x7f1268004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.125+0000 7f1285400640 1 -- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1268005150 msgr2=0x7f126800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.125+0000 7f1285400640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1268005150 0x7f126800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.125+0000 7f1285400640 1 -- 172.21.15.19:0/2864969177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1268003520 con 0x7f12680039e0 2024-04-25T11:39:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.126+0000 7f1285400640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12680039e0 0x7f1268003e50 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f1274001750 tx=0x7f1268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c22a2e8fb7ae97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.126+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274001370 con 0x7f12680039e0 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.126+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12740052d0 con 0x7f12680039e0 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.126+0000 7f127fe00640 1 -- 172.21.15.19:0/2864969177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127402abe0 con 0x7f12680039e0 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.127+0000 7f1286800640 1 -- 172.21.15.19:0/2864969177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12680039e0 msgr2=0x7f1268003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.127+0000 7f1286800640 1 --2- 172.21.15.19:0/2864969177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12680039e0 0x7f1268003e50 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f1274001750 tx=0x7f1268005830 comp rx=0 tx=0).stop 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.127+0000 7f1285e00640 1 -- 172.21.15.19:0/2864969177 reap_dead start 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.127+0000 7f1286800640 1 -- 172.21.15.19:0/2864969177 shutdown_connections 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.127+0000 7f1286800640 1 -- 172.21.15.19:0/2864969177 >> 172.21.15.19:0/2864969177 conn(0x7f12800d1720 msgr2=0x7f12800d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.127+0000 7f1286800640 1 -- 172.21.15.19:0/2864969177 shutdown_connections 2024-04-25T11:39:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.127+0000 7f1286800640 1 -- 172.21.15.19:0/2864969177 wait complete. 2024-04-25T11:39:34.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.129+0000 7f1286800640 1 Processor -- start 2024-04-25T11:39:34.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.129+0000 7f1286800640 1 -- start start 2024-04-25T11:39:34.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1286800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12680039e0 0x7f12801594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1286800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12801460d0 0x7f12800700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1286800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800705f0 0x7f12801617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1286800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12800d0450 con 0x7f12801460d0 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1286800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12800d02d0 con 0x7f12680039e0 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1286800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1280147760 con 0x7f12800705f0 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1284a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12801460d0 0x7f12800700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1284a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12801460d0 0x7f12800700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60288/0 (socket says 172.21.15.19:60288) 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1284a00640 1 -- 172.21.15.19:0/1619773122 learned_addr learned my addr 172.21.15.19:0/1619773122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:34.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1285400640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12680039e0 0x7f12801594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1284a00640 1 -- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800705f0 msgr2=0x7f12801617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1284a00640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800705f0 0x7f12801617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.131+0000 7f1284a00640 1 -- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12680039e0 msgr2=0x7f12801594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f1284a00640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12680039e0 0x7f12801594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f1284a00640 1 -- 172.21.15.19:0/1619773122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1280070a00 con 0x7f12801460d0 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f1285400640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12680039e0 0x7f12801594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f1284a00640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12801460d0 0x7f12800700b0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f126c00e7b0 tx=0x7f126c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f127d600640 1 -- 172.21.15.19:0/1619773122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c017070 con 0x7f12801460d0 2024-04-25T11:39:34.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f127d600640 1 -- 172.21.15.19:0/1619773122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f126c00c940 con 0x7f12801460d0 2024-04-25T11:39:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f127d600640 1 -- 172.21.15.19:0/1619773122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c00cd30 con 0x7f12801460d0 2024-04-25T11:39:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1280159a20 con 0x7f12801460d0 2024-04-25T11:39:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.132+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1280144d40 con 0x7f12801460d0 2024-04-25T11:39:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.133+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1280050740 con 0x7f12801460d0 2024-04-25T11:39:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.135+0000 7f127d600640 1 -- 172.21.15.19:0/1619773122 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f126c01a070 con 0x7f12801460d0 2024-04-25T11:39:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.136+0000 7f127d600640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744b0 0x7f12600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.136+0000 7f127d600640 1 -- 172.21.15.19:0/1619773122 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7945+0+0 (secure 0 0 0) 0x7f126c095960 con 0x7f12801460d0 2024-04-25T11:39:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.136+0000 7f1285400640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744b0 0x7f12600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.138+0000 7f1285400640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744b0 0x7f12600769c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1274001750 tx=0x7f127402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:34.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.140+0000 7f127d600640 1 -- 172.21.15.19:0/1619773122 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1280070a00 con 0x7f12801460d0 2024-04-25T11:39:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.384+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1280146d50 con 0x7f12801460d0 2024-04-25T11:39:34.372 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:34.385+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:34.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.389+0000 7f127d600640 1 -- 172.21.15.19:0/1619773122 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v262) ==== 161+0+0 (secure 0 0 0) 0x7f126c05f830 con 0x7f12801460d0 2024-04-25T11:39:34.377 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744b0 msgr2=0x7f12600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744b0 0x7f12600769c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1274001750 tx=0x7f127402e000 comp rx=0 tx=0).stop 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12801460d0 msgr2=0x7f12800700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12801460d0 0x7f12800700b0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f126c00e7b0 tx=0x7f126c00ec70 comp rx=0 tx=0).stop 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 shutdown_connections 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744b0 0x7f12600769c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12800705f0 0x7f12801617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12801460d0 0x7f12800700b0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 --2- 172.21.15.19:0/1619773122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12680039e0 0x7f12801594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 >> 172.21.15.19:0/1619773122 conn(0x7f12800d1720 msgr2=0x7f128015f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.393+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 shutdown_connections 2024-04-25T11:39:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.394+0000 7f1286800640 1 -- 172.21.15.19:0/1619773122 wait complete. 2024-04-25T11:39:34.387 INFO:tasks.ceph.mds.a.smithi019.stderr: -767> 2024-04-25T11:39:26.598+0000 7f3a21600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:39:34.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:34.527 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.710+0000 7f72b3e00640 1 Processor -- start 2024-04-25T11:39:34.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.710+0000 7f72b3e00640 1 -- start start 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.712+0000 7f72b3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f72b40b4ea0 0x7f72b40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.712+0000 7f72b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72b4144370 0x7f72b415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.712+0000 7f72b3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b40d1230 con 0x7f72b406c5c0 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f72b40d06b0 con 0x7f72b4144370 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f72b40d0990 con 0x7f72b40b4ea0 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72b4144370 0x7f72b415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f72b4144370 0x7f72b415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60312/0 (socket says 172.21.15.19:60312) 2024-04-25T11:39:34.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3400640 1 -- 172.21.15.19:0/650303571 learned_addr learned my addr 172.21.15.19:0/650303571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3400640 1 -- 172.21.15.19:0/650303571 >> v2:172.21.15.169:3300/0 conn(0x7f72b40b4ea0 msgr2=0x7f72b40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3400640 1 --2- 172.21.15.19:0/650303571 >> v2:172.21.15.169:3300/0 conn(0x7f72b40b4ea0 0x7f72b40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3400640 1 -- 172.21.15.19:0/650303571 >> v1:172.21.15.169:6789/0 conn(0x7f72b406c5c0 legacy=0x7f72b40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.713+0000 7f72b3400640 1 -- 172.21.15.19:0/650303571 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b40d0c70 con 0x7f72b4144370 2024-04-25T11:39:34.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.715+0000 7f72b3400640 1 --2- 172.21.15.19:0/650303571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415e770 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b5550 tx=0x7f72ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeae8e90516be7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.715+0000 7f72b1600640 1 -- 172.21.15.19:0/650303571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac036070 con 0x7f72b4144370 2024-04-25T11:39:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.715+0000 7f72b1600640 1 -- 172.21.15.19:0/650303571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac031600 con 0x7f72b4144370 2024-04-25T11:39:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.715+0000 7f72b1600640 1 -- 172.21.15.19:0/650303571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0319d0 con 0x7f72b4144370 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 -- 172.21.15.19:0/650303571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 msgr2=0x7f72b415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 --2- 172.21.15.19:0/650303571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415e770 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b5550 tx=0x7f72ac02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 -- 172.21.15.19:0/650303571 shutdown_connections 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 --2- 172.21.15.19:0/650303571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415e770 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 --2- 172.21.15.19:0/650303571 >> v2:172.21.15.169:3300/0 conn(0x7f72b40b4ea0 0x7f72b40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 -- 172.21.15.19:0/650303571 >> 172.21.15.19:0/650303571 conn(0x7f72b40d1720 msgr2=0x7f72b40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 -- 172.21.15.19:0/650303571 shutdown_connections 2024-04-25T11:39:34.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.716+0000 7f72b3e00640 1 -- 172.21.15.19:0/650303571 wait complete. 2024-04-25T11:39:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.719+0000 7f72b3e00640 1 Processor -- start 2024-04-25T11:39:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.719+0000 7f72b3e00640 1 -- start start 2024-04-25T11:39:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.720+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b406c5c0 0x7f72b4146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.720+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.720+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4142a40 0x7f72b4142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72b40d0450 con 0x7f72b4144370 2024-04-25T11:39:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72b40d02d0 con 0x7f72b406c5c0 2024-04-25T11:39:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72b4147760 con 0x7f72b4142a40 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60328/0 (socket says 172.21.15.19:60328) 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b2000640 1 -- 172.21.15.19:0/1672514000 learned_addr learned my addr 172.21.15.19:0/1672514000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b2a00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b406c5c0 0x7f72b4146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b3400640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4142a40 0x7f72b4142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b2000640 1 -- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4142a40 msgr2=0x7f72b4142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.721+0000 7f72b2000640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4142a40 0x7f72b4142e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72b2000640 1 -- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b406c5c0 msgr2=0x7f72b4146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72b2000640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b406c5c0 0x7f72b4146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72b2000640 1 -- 172.21.15.19:0/1672514000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b4143390 con 0x7f72b4144370 2024-04-25T11:39:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72b3400640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4142a40 0x7f72b4142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72b2000640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415d510 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f72a000b440 tx=0x7f72a000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72ab400640 1 -- 172.21.15.19:0/1672514000 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0015020 con 0x7f72b4144370 2024-04-25T11:39:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72ab400640 1 -- 172.21.15.19:0/1672514000 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a0011890 con 0x7f72b4144370 2024-04-25T11:39:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72ab400640 1 -- 172.21.15.19:0/1672514000 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0011c80 con 0x7f72b4144370 2024-04-25T11:39:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b41456c0 con 0x7f72b4144370 2024-04-25T11:39:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.722+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b4144e20 con 0x7f72b4144370 2024-04-25T11:39:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.723+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b4076cb0 con 0x7f72b4144370 2024-04-25T11:39:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.725+0000 7f72ab400640 1 -- 172.21.15.19:0/1672514000 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f72a0018020 con 0x7f72b4144370 2024-04-25T11:39:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.726+0000 7f72ab400640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72940744e0 0x7f72940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.726+0000 7f72ab400640 1 -- 172.21.15.19:0/1672514000 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8031+0+0 (secure 0 0 0) 0x7f72a0095bb0 con 0x7f72b4144370 2024-04-25T11:39:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.727+0000 7f72b2a00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72940744e0 0x7f72940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:34.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.728+0000 7f72b2a00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72940744e0 0x7f72940769f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f72a4006fd0 tx=0x7f72a4009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:34.731+0000 7f72ab400640 1 -- 172.21.15.19:0/1672514000 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72b4076cb0 con 0x7f72b4144370 2024-04-25T11:39:35.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.013+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f72b4068f10 con 0x7f72b4144370 2024-04-25T11:39:35.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.393+0000 7f72ab400640 1 -- 172.21.15.19:0/1672514000 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v263) ==== 107+0+0 (secure 0 0 0) 0x7f72b4068f10 con 0x7f72b4144370 2024-04-25T11:39:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:39:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72940744e0 msgr2=0x7f72940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72940744e0 0x7f72940769f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f72a4006fd0 tx=0x7f72a4009060 comp rx=0 tx=0).stop 2024-04-25T11:39:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 msgr2=0x7f72b415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415d510 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f72a000b440 tx=0x7f72a000b900 comp rx=0 tx=0).stop 2024-04-25T11:39:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 shutdown_connections 2024-04-25T11:39:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72940744e0 0x7f72940769f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72b4142a40 0x7f72b4142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b4144370 0x7f72b415d510 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 --2- 172.21.15.19:0/1672514000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f72b406c5c0 0x7f72b4146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 >> 172.21.15.19:0/1672514000 conn(0x7f72b40d1720 msgr2=0x7f72b406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:35.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 shutdown_connections 2024-04-25T11:39:35.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.397+0000 7f72b3e00640 1 -- 172.21.15.19:0/1672514000 wait complete. 2024-04-25T11:39:35.417 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.603+0000 7f4aee217f40 1 Processor -- start 2024-04-25T11:39:35.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.603+0000 7f4aee217f40 1 -- start start 2024-04-25T11:39:35.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4aee217f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4acc0022a0 0x55f848033170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4aee217f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8480336b0 0x55f847f87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4aee217f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f847f881d0 0x55f847f885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4aee217f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f847f01d20 con 0x55f8480336b0 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4aee217f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f847f05d60 con 0x55f847f881d0 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4aee217f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f847fa43f0 con 0x7f4acc0022a0 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8480336b0 0x55f847f87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8480336b0 0x55f847f87c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60346/0 (socket says 172.21.15.19:60346) 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae6a00640 1 -- 172.21.15.19:0/864460220 learned_addr learned my addr 172.21.15.19:0/864460220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae6a00640 1 -- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4acc0022a0 msgr2=0x55f848033170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae7400640 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4acc0022a0 0x55f848033170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae7e00640 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f847f881d0 0x55f847f885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4acc0022a0 0x55f848033170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.605+0000 7f4ae6a00640 1 -- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f847f881d0 msgr2=0x55f847f885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:35.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.606+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f847f881d0 0x55f847f885e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.606+0000 7f4ae6a00640 1 -- 172.21.15.19:0/864460220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f847fa7990 con 0x55f8480336b0 2024-04-25T11:39:35.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.606+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8480336b0 0x55f847f87c90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006800 tx=0x7f4ad4006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa98648b4b82ed5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.606+0000 7f4ae5600640 1 -- 172.21.15.19:0/864460220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40062d0 con 0x55f8480336b0 2024-04-25T11:39:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.606+0000 7f4ae5600640 1 -- 172.21.15.19:0/864460220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad400b6c0 con 0x55f8480336b0 2024-04-25T11:39:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.606+0000 7f4ae5600640 1 -- 172.21.15.19:0/864460220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400bab0 con 0x55f8480336b0 2024-04-25T11:39:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 -- 172.21.15.19:0/864460220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8480336b0 msgr2=0x55f847f87c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8480336b0 0x55f847f87c90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006800 tx=0x7f4ad4006cc0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 -- 172.21.15.19:0/864460220 shutdown_connections 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f847f881d0 0x55f847f885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f8480336b0 0x55f847f87c90 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 --2- 172.21.15.19:0/864460220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4acc0022a0 0x55f848033170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 -- 172.21.15.19:0/864460220 >> 172.21.15.19:0/864460220 conn(0x55f847e7dac0 msgr2=0x55f847f8d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 -- 172.21.15.19:0/864460220 shutdown_connections 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.607+0000 7f4aee217f40 1 -- 172.21.15.19:0/864460220 wait complete. 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.608+0000 7f4aee217f40 1 Processor -- start 2024-04-25T11:39:35.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.608+0000 7f4aee217f40 1 -- start start 2024-04-25T11:39:35.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4aee217f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f847f881d0 0x55f847f5d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4aee217f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8480336b0 0x55f847fa63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4aee217f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc0022a0 0x55f848027f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4aee217f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f847f88b20 con 0x7f4acc0022a0 2024-04-25T11:39:35.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4aee217f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f847fa7bb0 con 0x55f8480336b0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4aee217f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f847f5e080 con 0x55f847f881d0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4ae7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc0022a0 0x55f848027f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4ae7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc0022a0 0x55f848027f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60356/0 (socket says 172.21.15.19:60356) 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 learned_addr learned my addr 172.21.15.19:0/649149922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f847f881d0 0x55f847f5d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.610+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8480336b0 0x55f847fa63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f847f881d0 msgr2=0x55f847f5d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4ae7e00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f847f881d0 0x55f847f5d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8480336b0 msgr2=0x55f847fa63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4ae7e00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8480336b0 0x55f847fa63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f848033210 con 0x7f4acc0022a0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f847f881d0 0x55f847f5d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4ae7e00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc0022a0 0x55f848027f30 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x55f847fa29b0 tx=0x7f4ae000a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4adfe00640 1 -- 172.21.15.19:0/649149922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0014020 con 0x7f4acc0022a0 2024-04-25T11:39:35.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4adfe00640 1 -- 172.21.15.19:0/649149922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae000cca0 con 0x7f4acc0022a0 2024-04-25T11:39:35.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4adfe00640 1 -- 172.21.15.19:0/649149922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00106b0 con 0x7f4acc0022a0 2024-04-25T11:39:35.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.611+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f847f88710 con 0x7f4acc0022a0 2024-04-25T11:39:35.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.612+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f847f5ddc0 con 0x7f4acc0022a0 2024-04-25T11:39:35.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.614+0000 7f4adfe00640 1 -- 172.21.15.19:0/649149922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ae0017020 con 0x7f4acc0022a0 2024-04-25T11:39:35.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.615+0000 7f4adfe00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ac8072e90 0x7f4ac80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.615+0000 7f4adfe00640 1 -- 172.21.15.19:0/649149922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8031+0+0 (secure 0 0 0) 0x7f4ae0094440 con 0x7f4acc0022a0 2024-04-25T11:39:35.603 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:35.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.615+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f84802c4a0 0x55f8480d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.615+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f847fa6920 con 0x55f84802c4a0 2024-04-25T11:39:35.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.616+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ac8072e90 0x7f4ac80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.616+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f84802c4a0 0x55f8480d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.617+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ac8072e90 0x7f4ac80753a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8056000 tx=0x7f4ad8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.618+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f84802c4a0 0x55f8480d99e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.621+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v119'815 uv789 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.621+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f84802c8b0 con 0x55f84802c4a0 2024-04-25T11:39:35.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.625+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v119'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.625+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480d9f20 con 0x55f84802c4a0 2024-04-25T11:39:35.616 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.629+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v119'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.629+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480db2d0 con 0x55f84802c4a0 2024-04-25T11:39:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.633+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.633+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480db670 con 0x55f84802c4a0 2024-04-25T11:39:35.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.637+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.637+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da2c0 con 0x55f84802c4a0 2024-04-25T11:39:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.641+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'820 uv820 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.641+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da6b0 con 0x55f84802c4a0 2024-04-25T11:39:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.645+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v119'821 uv795 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.645+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480da6b0 con 0x55f84802c4a0 2024-04-25T11:39:35.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.649+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v119'822 uv822 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.649+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.641 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.654+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v119'823 uv823 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.654+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.658+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'824 uv824 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.658+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.661+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'825 uv825 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.662+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.666+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'826 uv826 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.666+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.671+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v119'827 uv802 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.671+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.675+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'828 uv828 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.675+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.667 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.680+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v119'829 uv829 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.680+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.684+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'830 uv830 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.684+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.689+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'831 uv831 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.689+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.693+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'832 uv832 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.693+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.698+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'833 uv833 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.698+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.703+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v119'834 uv808 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.703+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.708+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v119'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.708+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.700 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.712+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v119'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.712+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.717+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.717+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.721+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.722+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.726+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.726+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.732+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v119'840 uv814 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.732+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.736+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v119'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.736+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.728 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:35.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.741+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v119'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.741+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.745+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.745+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.750+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.750+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f84802c4a0 2024-04-25T11:39:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.754+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'845 uv845 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.754+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f8480dba10 0x55f8480dde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.754+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480da660 con 0x55f8480dba10 2024-04-25T11:39:35.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.755+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f8480dba10 0x55f8480dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.757+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f8480dba10 0x55f8480dde30 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.761+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'596 uv575 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.762+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480de370 con 0x55f8480dba10 2024-04-25T11:39:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.766+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'597 uv597 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.766+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480de710 con 0x55f8480dba10 2024-04-25T11:39:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.770+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'598 uv598 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.758 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.770+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480deab0 con 0x55f8480dba10 2024-04-25T11:39:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.776+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v119'599 uv599 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.776+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480dee50 con 0x55f8480dba10 2024-04-25T11:39:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.780+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'600 uv600 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.780+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df1f0 con 0x55f8480dba10 2024-04-25T11:39:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.784+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v119'601 uv594 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.784+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480df590 con 0x55f8480dba10 2024-04-25T11:39:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.789+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v119'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.789+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.781 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:35.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.793+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v119'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.794+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.798+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'604 uv604 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.799+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.803+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.803+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.808+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v119'606 uv595 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.809+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.813+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v119'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.813+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.805 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:35.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.817+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v119'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.818+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e4020 con 0x55f8480dba10 2024-04-25T11:39:35.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.823+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v119'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.823+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e4020 con 0x55f8480dba10 2024-04-25T11:39:35.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.828+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'610 uv610 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.828+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.832+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v119'611 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.833+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.837+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v119'612 uv612 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.837+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.841+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v119'613 uv613 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.829 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:35.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.842+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e0940 con 0x55f8480dba10 2024-04-25T11:39:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.846+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'614 uv614 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.846+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e0940 con 0x55f8480dba10 2024-04-25T11:39:35.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.850+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'615 uv615 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.850+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480df930 con 0x55f8480dba10 2024-04-25T11:39:35.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.855+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'616 uv616 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480dba10 2024-04-25T11:39:35.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.855+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f8480dfe40 0x55f8480e2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.856+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e27a0 con 0x55f8480dfe40 2024-04-25T11:39:35.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.857+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f8480dfe40 0x55f8480e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.858+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f8480dfe40 0x55f8480e2260 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.863+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f8480dfe40 2024-04-25T11:39:35.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.863+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480e27a0 con 0x55f8480dfe40 2024-04-25T11:39:35.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.868+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v119'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f8480dfe40 2024-04-25T11:39:35.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.868+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e2820 con 0x55f8480dfe40 2024-04-25T11:39:35.860 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.873+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v119'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f8480dfe40 2024-04-25T11:39:35.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.873+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e3c50 con 0x55f8480dfe40 2024-04-25T11:39:35.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.877+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f8480dfe40 2024-04-25T11:39:35.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.877+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e3c50 con 0x55f8480dfe40 2024-04-25T11:39:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.882+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f8480dfe40 2024-04-25T11:39:35.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.882+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e2c40 con 0x55f8480dfe40 2024-04-25T11:39:35.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.886+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f8480dfe40 2024-04-25T11:39:35.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.886+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e2c40 con 0x55f84802c4a0 2024-04-25T11:39:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.891+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'120 uv119 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.891+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480e2c40 con 0x55f84802c4a0 2024-04-25T11:39:35.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.895+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'121 uv121 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.895+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e2c40 con 0x55f84802c4a0 2024-04-25T11:39:35.886 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:35.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.899+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'122 uv122 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.899+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e2c40 con 0x55f84802c4a0 2024-04-25T11:39:35.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.903+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'123 uv123 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400f030 con 0x55f84802c4a0 2024-04-25T11:39:35.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.903+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f8480e32c0 0x55f8480e56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.903+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e5c20 con 0x55f8480e32c0 2024-04-25T11:39:35.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.905+0000 7f4ae7e00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f8480e32c0 0x55f8480e56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.906+0000 7f4ae7e00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f8480e32c0 0x55f8480e56e0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.910+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.910+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480e6040 con 0x55f8480e32c0 2024-04-25T11:39:35.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.915+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v119'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:35.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.915+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e6520 con 0x55f8480e32c0 2024-04-25T11:39:35.907 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:35.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.920+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v119'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:35.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.920+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e7dd0 con 0x55f8480e32c0 2024-04-25T11:39:35.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.924+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:35.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.924+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e8170 con 0x55f8480e32c0 2024-04-25T11:39:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.928+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.928+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480e6dc0 con 0x55f8480e32c0 2024-04-25T11:39:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.933+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.933+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f8480e76e0 0x55f8480ea520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.933+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eaa60 con 0x55f8480e76e0 2024-04-25T11:39:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.934+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f8480e76e0 0x55f8480ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.936+0000 7f4ae7400640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f8480e76e0 0x55f8480ea520 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.940+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v119'294 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.940+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480eaa60 con 0x55f8480e76e0 2024-04-25T11:39:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.944+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v119'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.944+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eaae0 con 0x55f8480e76e0 2024-04-25T11:39:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.948+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v119'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.936 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:35.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.948+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ebf10 con 0x55f8480e76e0 2024-04-25T11:39:35.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.952+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.952+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ebf10 con 0x55f8480e76e0 2024-04-25T11:39:35.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.956+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.956+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eaf00 con 0x55f8480e76e0 2024-04-25T11:39:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.960+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.960+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eaf00 con 0x55f8480e76e0 2024-04-25T11:39:35.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.964+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v119'300 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.964+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480eaf00 con 0x55f8480e76e0 2024-04-25T11:39:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.968+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v119'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.968+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eaf00 con 0x55f8480e76e0 2024-04-25T11:39:35.960 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.972+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v119'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.972+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eaf00 con 0x55f8480e76e0 2024-04-25T11:39:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.977+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.977+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eaf00 con 0x55f8480e76e0 2024-04-25T11:39:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.981+0000 7f4ae7400640 1 -- 172.21.15.19:0/649149922 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad8056030 con 0x55f8480e76e0 2024-04-25T11:39:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.981+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f8480eb410 0x55f8480ed830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.981+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480edec0 con 0x55f8480eb410 2024-04-25T11:39:35.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.982+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f8480eb410 0x55f8480ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:35.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.983+0000 7f4ae6a00640 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f8480eb410 0x55f8480ed830 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.987+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v119'132 uv131 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.987+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480edec0 con 0x55f8480eb410 2024-04-25T11:39:35.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.991+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v119'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:35.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.991+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480eb410 2024-04-25T11:39:35.983 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.995+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v119'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:35.995+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480eb410 2024-04-25T11:39:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.000+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.000+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480eb410 2024-04-25T11:39:35.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.004+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:35.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.004+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f84802c4a0 2024-04-25T11:39:35.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.009+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f84802c4a0 2024-04-25T11:39:35.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.009+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f84802c4a0 2024-04-25T11:39:36.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.012+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v119'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f84802c4a0 2024-04-25T11:39:36.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.013+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f84802c4a0 2024-04-25T11:39:36.004 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:36.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.016+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v119'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f84802c4a0 2024-04-25T11:39:36.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.017+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ef320 con 0x55f84802c4a0 2024-04-25T11:39:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.020+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f84802c4a0 2024-04-25T11:39:36.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.021+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ef320 con 0x55f84802c4a0 2024-04-25T11:39:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.025+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f84802c4a0 2024-04-25T11:39:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.025+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f84802c4a0 2024-04-25T11:39:36.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.029+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f84802c4a0 2024-04-25T11:39:36.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.029+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.033+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v119'263 uv257 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.033+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.038+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v119'264 uv264 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.038+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.041+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v119'265 uv265 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.030 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.042+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.046+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'266 uv266 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.046+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.051+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'267 uv267 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.051+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.054+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v119'268 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.054+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.059+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v119'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.059+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.051 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.063+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v119'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.063+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee310 con 0x55f8480dfe40 2024-04-25T11:39:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.067+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.067+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480f10c0 con 0x55f8480dfe40 2024-04-25T11:39:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.073+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480dfe40 2024-04-25T11:39:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.073+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480f1080 con 0x55f8480e32c0 2024-04-25T11:39:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.076+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v119'116 uv114 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.076+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.080+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'117 uv117 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.080+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.072 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.084+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v119'118 uv118 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.084+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.088+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'119 uv119 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.089+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.093+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v119'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.093+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.098+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v119'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.098+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.102+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v119'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.102+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.107+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.107+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.111+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.111+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480e32c0 2024-04-25T11:39:36.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.116+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ae0020070 con 0x55f8480e32c0 2024-04-25T11:39:36.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.116+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee700 con 0x55f8480eb410 2024-04-25T11:39:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.121+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v119'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.121+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.125+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v119'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.125+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.117 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:36.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.129+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v119'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.129+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.133+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.133+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.137+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.138+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.142+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v119'327 uv318 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.142+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.147+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v119'328 uv328 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.147+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.139 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.151+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v119'329 uv329 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.152+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee680 con 0x55f8480eb410 2024-04-25T11:39:36.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.156+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v119'330 uv330 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.156+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee660 con 0x55f8480eb410 2024-04-25T11:39:36.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.160+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'331 uv331 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.160+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee630 con 0x55f8480eb410 2024-04-25T11:39:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.165+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v119'332 uv325 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.165+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e119) -- 0x55f8480ee630 con 0x55f8480eb410 2024-04-25T11:39:36.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.169+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v119'333 uv333 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.169+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee630 con 0x55f8480eb410 2024-04-25T11:39:36.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:36.161 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:36.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.173+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v119'334 uv334 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.173+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ef640 con 0x55f8480eb410 2024-04-25T11:39:36.168 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:39:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.177+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'335 uv335 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.177+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ef640 con 0x55f8480eb410 2024-04-25T11:39:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.182+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'336 uv336 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.182+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480ee630 con 0x55f8480eb410 2024-04-25T11:39:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.186+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'337 uv337 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.186+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55f8480eeed0 con 0x55f8480eb410 2024-04-25T11:39:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4ae6a00640 1 -- 172.21.15.19:0/649149922 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'338 uv338 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad402b040 con 0x55f8480eb410 2024-04-25T11:39:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f8480dba10 msgr2=0x55f8480dde30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f8480dba10 0x55f8480dde30 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f8480e76e0 msgr2=0x55f8480ea520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f8480e76e0 0x55f8480ea520 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f8480eb410 msgr2=0x55f8480ed830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f8480eb410 0x55f8480ed830 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 reap_dead start 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f8480dfe40 msgr2=0x55f8480e2260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f8480dfe40 0x55f8480e2260 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f84802c4a0 msgr2=0x55f8480d99e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f84802c4a0 0x55f8480d99e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f8480e32c0 msgr2=0x55f8480e56e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.190+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f8480e32c0 0x55f8480e56e0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ac8072e90 msgr2=0x7f4ac80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ac8072e90 0x7f4ac80753a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8056000 tx=0x7f4ad8009000 comp rx=0 tx=0).stop 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc0022a0 msgr2=0x55f848027f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 --2- 172.21.15.19:0/649149922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4acc0022a0 0x55f848027f30 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x55f847fa29b0 tx=0x7f4ae000a5a0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4ae7e00640 1 -- 172.21.15.19:0/649149922 reap_dead start 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 shutdown_connections 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 >> 172.21.15.19:0/649149922 conn(0x55f847e7dac0 msgr2=0x55f847f8d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 shutdown_connections 2024-04-25T11:39:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.191+0000 7f4aee217f40 1 -- 172.21.15.19:0/649149922 wait complete. 2024-04-25T11:39:36.196 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2024-04-25T11:39:36.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.435+0000 7f7130213f40 1 Processor -- start 2024-04-25T11:39:36.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.435+0000 7f7130213f40 1 -- start start 2024-04-25T11:39:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7130213f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b2544870 0x5561b260d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7130213f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b261b340 0x5561b253fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7130213f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b2540200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7130213f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561b258a2b0 con 0x5561b261bc90 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7130213f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5561b2589fd0 con 0x5561b2544870 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7130213f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5561b258a590 con 0x5561b261b340 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b2540200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b2540200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60376/0 (socket says 172.21.15.19:60376) 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 -- 172.21.15.19:0/2687996864 learned_addr learned my addr 172.21.15.19:0/2687996864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129400640 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b2544870 0x5561b260d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7128a00640 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b261b340 0x5561b253fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 -- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b261b340 msgr2=0x5561b253fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b261b340 0x5561b253fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 -- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b2544870 msgr2=0x5561b260d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b2544870 0x5561b260d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129e00640 1 -- 172.21.15.19:0/2687996864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b2469b30 con 0x5561b261bc90 2024-04-25T11:39:36.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.437+0000 7f7129400640 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b2544870 0x5561b260d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:36.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.438+0000 7f7129e00640 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b2540200 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f7124025b20 tx=0x7f7124050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42580fbe3db54e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.438+0000 7f7123400640 1 -- 172.21.15.19:0/2687996864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124054040 con 0x5561b261bc90 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.438+0000 7f7123400640 1 -- 172.21.15.19:0/2687996864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124050940 con 0x5561b261bc90 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.438+0000 7f7123400640 1 -- 172.21.15.19:0/2687996864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124050cf0 con 0x5561b261bc90 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 -- 172.21.15.19:0/2687996864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 msgr2=0x5561b2540200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b2540200 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f7124025b20 tx=0x7f7124050000 comp rx=0 tx=0).stop 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 -- 172.21.15.19:0/2687996864 shutdown_connections 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b2540200 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b261b340 0x5561b253fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 --2- 172.21.15.19:0/2687996864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b2544870 0x5561b260d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 -- 172.21.15.19:0/2687996864 >> 172.21.15.19:0/2687996864 conn(0x5561b2462ac0 msgr2=0x5561b256c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 -- 172.21.15.19:0/2687996864 shutdown_connections 2024-04-25T11:39:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.439+0000 7f7130213f40 1 -- 172.21.15.19:0/2687996864 wait complete. 2024-04-25T11:39:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.440+0000 7f7130213f40 1 Processor -- start 2024-04-25T11:39:36.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.440+0000 7f7130213f40 1 -- start start 2024-04-25T11:39:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7130213f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b2544870 0x5561b2540ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7130213f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b261b340 0x5561b260a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7130213f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b260ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7130213f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5561b2540740 con 0x5561b261bc90 2024-04-25T11:39:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7130213f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5561b260b960 con 0x5561b261b340 2024-04-25T11:39:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7130213f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5561b26066b0 con 0x5561b2544870 2024-04-25T11:39:36.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7129e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b260ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7129e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b260ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60378/0 (socket says 172.21.15.19:60378) 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7129e00640 1 -- 172.21.15.19:0/365717263 learned_addr learned my addr 172.21.15.19:0/365717263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7128a00640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b261b340 0x5561b260a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.442+0000 7f7129400640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b2544870 0x5561b2540ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7129e00640 1 -- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b2544870 msgr2=0x5561b2540ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7129e00640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5561b2544870 0x5561b2540ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7129e00640 1 -- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b261b340 msgr2=0x5561b260a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7129e00640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b261b340 0x5561b260a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7129e00640 1 -- 172.21.15.19:0/365717263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b2606900 con 0x5561b261bc90 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7128a00640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5561b261b340 0x5561b260a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7129e00640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b260ae00 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f7124025b20 tx=0x7f7124003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7122000640 1 -- 172.21.15.19:0/365717263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124054070 con 0x5561b261bc90 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7122000640 1 -- 172.21.15.19:0/365717263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124065840 con 0x5561b261bc90 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7122000640 1 -- 172.21.15.19:0/365717263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124065c10 con 0x5561b261bc90 2024-04-25T11:39:36.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5561b2604070 con 0x5561b261bc90 2024-04-25T11:39:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.443+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561b2604630 con 0x5561b261bc90 2024-04-25T11:39:36.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.446+0000 7f7122000640 1 -- 172.21.15.19:0/365717263 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7124057070 con 0x5561b261bc90 2024-04-25T11:39:36.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.447+0000 7f7122000640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7108072e90 0x7f71080753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.447+0000 7f7122000640 1 -- 172.21.15.19:0/365717263 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8031+0+0 (secure 0 0 0) 0x7f71240d9510 con 0x5561b261bc90 2024-04-25T11:39:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.447+0000 7f7130213f40 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561b25411e0 0x5561b2601880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.447+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.d 12:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x5561b26023f0 con 0x5561b25411e0 2024-04-25T11:39:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.447+0000 7f7128a00640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561b25411e0 0x5561b2601880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.447+0000 7f7129400640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7108072e90 0x7f71080753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.448+0000 7f7128a00640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561b25411e0 0x5561b2601880 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:36.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.449+0000 7f7129400640 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7108072e90 0x7f71080753a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x5561b253f8e0 tx=0x7f711c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7128a00640 1 -- 172.21.15.19:0/365717263 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v119'137 uv137 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f711400a5f0 con 0x5561b25411e0 2024-04-25T11:39:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561b25411e0 msgr2=0x5561b2601880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5561b25411e0 0x5561b2601880 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7108072e90 msgr2=0x7f71080753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7108072e90 0x7f71080753a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x5561b253f8e0 tx=0x7f711c009040 comp rx=0 tx=0).stop 2024-04-25T11:39:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 msgr2=0x5561b260ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 --2- 172.21.15.19:0/365717263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5561b261bc90 0x5561b260ae00 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f7124025b20 tx=0x7f7124003330 comp rx=0 tx=0).stop 2024-04-25T11:39:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7129e00640 1 -- 172.21.15.19:0/365717263 reap_dead start 2024-04-25T11:39:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 shutdown_connections 2024-04-25T11:39:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 >> 172.21.15.19:0/365717263 conn(0x5561b2462ac0 msgr2=0x5561b256f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.451+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 shutdown_connections 2024-04-25T11:39:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.452+0000 7f7130213f40 1 -- 172.21.15.19:0/365717263 wait complete. 2024-04-25T11:39:36.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:36.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.739+0000 7f221a600640 1 Processor -- start 2024-04-25T11:39:36.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.739+0000 7f221a600640 1 -- start start 2024-04-25T11:39:36.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.741+0000 7f221a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 0x7f221415f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.741+0000 7f221a600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2214164ad0 con 0x7f22140b4f70 2024-04-25T11:39:36.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.741+0000 7f221a600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22140b5380 con 0x7f221406db30 2024-04-25T11:39:36.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.741+0000 7f221a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2214164510 con 0x7f22140d5020 2024-04-25T11:39:36.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.741+0000 7f2219200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f22140b4f70 0x7f2214163c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58028/0 (socket says 172.21.15.19:58028) 2024-04-25T11:39:36.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.741+0000 7f2219200640 1 -- 172.21.15.19:0/3330528489 learned_addr learned my addr 172.21.15.19:0/3330528489 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:36.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2219c00640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 0x7f221415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288509227 0 0) 0x7f2208001110 con 0x7f22140b4f70 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21fc003520 con 0x7f22140b4f70 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2473302916 0 0) 0x7f21fc003520 con 0x7f22140b4f70 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 msgr2=0x7f221415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 0x7f221415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 >> v1:172.21.15.169:6789/0 conn(0x7f221406db30 legacy=0x7f221406df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22141647f0 con 0x7f22140b4f70 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2208001840 con 0x7f22140b4f70 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 >> v1:172.21.15.19:6789/0 conn(0x7f22140b4f70 legacy=0x7f2214163c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.742+0000 7f2213400640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 0x7f21fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213400640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc003f60 0x7f21fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213400640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc004d40 0x7f21fc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2214164510 con 0x7f21fc003f60 2024-04-25T11:39:36.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc00bc00 con 0x7f21fc004d40 2024-04-25T11:39:36.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc00be00 con 0x7f22140d5020 2024-04-25T11:39:36.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213e00640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc003f60 0x7f21fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2219200640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 0x7f21fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2219c00640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc004d40 0x7f21fc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213e00640 1 -- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 msgr2=0x7f21fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:36.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213e00640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22140d5020 0x7f21fc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213e00640 1 -- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc004d40 msgr2=0x7f21fc005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213e00640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21fc004d40 0x7f21fc005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.743+0000 7f2213e00640 1 -- 172.21.15.19:0/3330528489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc003520 con 0x7f21fc003f60 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.744+0000 7f2213e00640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc003f60 0x7f21fc004800 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f2204001720 tx=0x7f21fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebecc9705db18ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.745+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204001370 con 0x7f21fc003f60 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.745+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22040052d0 con 0x7f21fc003f60 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.745+0000 7f2213400640 1 -- 172.21.15.19:0/3330528489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220402abc0 con 0x7f21fc003f60 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.745+0000 7f221a600640 1 -- 172.21.15.19:0/3330528489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc003f60 msgr2=0x7f21fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.745+0000 7f221a600640 1 --2- 172.21.15.19:0/3330528489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21fc003f60 0x7f21fc004800 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f2204001720 tx=0x7f21fc005830 comp rx=0 tx=0).stop 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.745+0000 7f2219c00640 1 -- 172.21.15.19:0/3330528489 reap_dead start 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.746+0000 7f221a600640 1 -- 172.21.15.19:0/3330528489 shutdown_connections 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.746+0000 7f221a600640 1 -- 172.21.15.19:0/3330528489 >> 172.21.15.19:0/3330528489 conn(0x7f22140d6140 msgr2=0x7f22140b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.746+0000 7f221a600640 1 -- 172.21.15.19:0/3330528489 shutdown_connections 2024-04-25T11:39:36.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.746+0000 7f221a600640 1 -- 172.21.15.19:0/3330528489 wait complete. 2024-04-25T11:39:36.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.748+0000 7f221a600640 1 Processor -- start 2024-04-25T11:39:36.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.748+0000 7f221a600640 1 -- start start 2024-04-25T11:39:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f221a600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc003f60 0x7f221414be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f221a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2214158ea0 0x7f22141eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f221a600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22141eb410 0x7f22141eb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f221a600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22141612c0 con 0x7f2214158ea0 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f221a600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f22140d3af0 con 0x7f22141eb410 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f221a600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2214161440 con 0x7f21fc003f60 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f2213e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2214158ea0 0x7f22141eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f2213e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2214158ea0 0x7f22141eaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60396/0 (socket says 172.21.15.19:60396) 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.750+0000 7f2213e00640 1 -- 172.21.15.19:0/2451399296 learned_addr learned my addr 172.21.15.19:0/2451399296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2219200640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc003f60 0x7f221414be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2219c00640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22141eb410 0x7f22141eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2213e00640 1 -- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc003f60 msgr2=0x7f221414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2213e00640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc003f60 0x7f221414be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2213e00640 1 -- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22141eb410 msgr2=0x7f22141eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2213e00640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22141eb410 0x7f22141eb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2213e00640 1 -- 172.21.15.19:0/2451399296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22141ea570 con 0x7f2214158ea0 2024-04-25T11:39:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2219c00640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22141eb410 0x7f22141eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.751+0000 7f2213e00640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2214158ea0 0x7f22141eaed0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f2204001840 tx=0x7f2204001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.752+0000 7f2210c00640 1 -- 172.21.15.19:0/2451399296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204037040 con 0x7f2214158ea0 2024-04-25T11:39:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.752+0000 7f2210c00640 1 -- 172.21.15.19:0/2451399296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204005480 con 0x7f2214158ea0 2024-04-25T11:39:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.752+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22141ea850 con 0x7f2214158ea0 2024-04-25T11:39:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.752+0000 7f2210c00640 1 -- 172.21.15.19:0/2451399296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204007a30 con 0x7f2214158ea0 2024-04-25T11:39:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.752+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22140b9270 con 0x7f2214158ea0 2024-04-25T11:39:36.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.753+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221415ca00 con 0x7f2214158ea0 2024-04-25T11:39:36.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.755+0000 7f2210c00640 1 -- 172.21.15.19:0/2451399296 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2204044020 con 0x7f2214158ea0 2024-04-25T11:39:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.756+0000 7f2210c00640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21f80744b0 0x7f21f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.756+0000 7f2210c00640 1 -- 172.21.15.19:0/2451399296 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8031+0+0 (secure 0 0 0) 0x7f22040b6380 con 0x7f2214158ea0 2024-04-25T11:39:36.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.756+0000 7f2219200640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21f80744b0 0x7f21f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:36.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.758+0000 7f2219200640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21f80744b0 0x7f21f80769c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f22080019b0 tx=0x7f2208009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:36.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:36.759+0000 7f2210c00640 1 -- 172.21.15.19:0/2451399296 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f220404f3e0 con 0x7f2214158ea0 2024-04-25T11:39:36.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.003+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2214076c60 con 0x7f2214158ea0 2024-04-25T11:39:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.873+0000 7f2210c00640 1 -- 172.21.15.19:0/2451399296 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v264) ==== 162+0+0 (secure 0 0 0) 0x7f2204080270 con 0x7f2214158ea0 2024-04-25T11:39:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21f80744b0 msgr2=0x7f21f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21f80744b0 0x7f21f80769c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f22080019b0 tx=0x7f2208009040 comp rx=0 tx=0).stop 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2214158ea0 msgr2=0x7f22141eaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2214158ea0 0x7f22141eaed0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f2204001840 tx=0x7f2204001120 comp rx=0 tx=0).stop 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 shutdown_connections 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21f80744b0 0x7f21f80769c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22141eb410 0x7f22141eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2214158ea0 0x7f22141eaed0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 --2- 172.21.15.19:0/2451399296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21fc003f60 0x7f221414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 >> 172.21.15.19:0/2451399296 conn(0x7f22140d6140 msgr2=0x7f221415f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 shutdown_connections 2024-04-25T11:39:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:37.878+0000 7f221a600640 1 -- 172.21.15.19:0/2451399296 wait complete. 2024-04-25T11:39:37.881 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:37.893+0000 7f9865e00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-04-25T11:39:37.882 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:37.895+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:37.882 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:37.895+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:39:37.894 INFO:tasks.ceph.mds.a.smithi019.stderr: -17> 2024-04-25T11:39:37.893+0000 7f9865e00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-04-25T11:39:37.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:38.023 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:38.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.180+0000 7fb59da00640 1 Processor -- start 2024-04-25T11:39:38.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.180+0000 7fb59da00640 1 -- start start 2024-04-25T11:39:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.183+0000 7fb59da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 0x7fb59806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.183+0000 7fb59da00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59806c7d0 con 0x7fb598160aa0 2024-04-25T11:39:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.183+0000 7fb59da00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59806cab0 con 0x7fb5980d0bd0 2024-04-25T11:39:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.183+0000 7fb59da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb59806cd90 con 0x7fb5980b6430 2024-04-25T11:39:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.183+0000 7fb58f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb598160aa0 0x7fb598160eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58042/0 (socket says 172.21.15.19:58042) 2024-04-25T11:39:38.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.183+0000 7fb58f400640 1 -- 172.21.15.19:0/3570988474 learned_addr learned my addr 172.21.15.19:0/3570988474 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597e00640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 0x7fb59806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697212258 0 0) 0x7fb59806c7d0 con 0x7fb598160aa0 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb57c003520 con 0x7fb598160aa0 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 114117925 0 0) 0x7fb57c003520 con 0x7fb598160aa0 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 msgr2=0x7fb59806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 0x7fb59806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 >> v1:172.21.15.169:6789/0 conn(0x7fb5980d0bd0 legacy=0x7fb5980d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59806d070 con 0x7fb598160aa0 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb580001670 con 0x7fb598160aa0 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 >> v1:172.21.15.19:6789/0 conn(0x7fb598160aa0 legacy=0x7fb598160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 0x7fb57c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57c003f60 0x7fb57c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57c004d40 0x7fb57c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb59806cd90 con 0x7fb57c004d40 2024-04-25T11:39:38.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c00bc80 con 0x7fb57c003f60 2024-04-25T11:39:38.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.184+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c00e090 con 0x7fb5980b6430 2024-04-25T11:39:38.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb59d000640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57c004d40 0x7fb57c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb58f400640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57c003f60 0x7fb57c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb597e00640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 0x7fb57c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb59d000640 1 -- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 msgr2=0x7fb57c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:38.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb59d000640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980b6430 0x7fb57c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb59d000640 1 -- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57c003f60 msgr2=0x7fb57c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:38.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb59d000640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57c003f60 0x7fb57c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.185+0000 7fb59d000640 1 -- 172.21.15.19:0/3570988474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c003520 con 0x7fb57c004d40 2024-04-25T11:39:38.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.186+0000 7fb58f400640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb57c003f60 0x7fb57c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:38.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.187+0000 7fb59d000640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57c004d40 0x7fb57c0051f0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fb5980d16f0 tx=0x7fb57c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231c9149449b016a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.187+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590001a70 con 0x7fb57c004d40 2024-04-25T11:39:38.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.187+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59002ae10 con 0x7fb57c004d40 2024-04-25T11:39:38.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.187+0000 7fb597400640 1 -- 172.21.15.19:0/3570988474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590003bb0 con 0x7fb57c004d40 2024-04-25T11:39:38.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.188+0000 7fb59da00640 1 -- 172.21.15.19:0/3570988474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57c004d40 msgr2=0x7fb57c0051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.188+0000 7fb59da00640 1 --2- 172.21.15.19:0/3570988474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb57c004d40 0x7fb57c0051f0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fb5980d16f0 tx=0x7fb57c005830 comp rx=0 tx=0).stop 2024-04-25T11:39:38.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.188+0000 7fb59d000640 1 -- 172.21.15.19:0/3570988474 reap_dead start 2024-04-25T11:39:38.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.188+0000 7fb59da00640 1 -- 172.21.15.19:0/3570988474 shutdown_connections 2024-04-25T11:39:38.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.188+0000 7fb59da00640 1 -- 172.21.15.19:0/3570988474 >> 172.21.15.19:0/3570988474 conn(0x7fb5980d1720 msgr2=0x7fb5980d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:38.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.188+0000 7fb59da00640 1 -- 172.21.15.19:0/3570988474 shutdown_connections 2024-04-25T11:39:38.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.188+0000 7fb59da00640 1 -- 172.21.15.19:0/3570988474 wait complete. 2024-04-25T11:39:38.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.191+0000 7fb59da00640 1 Processor -- start 2024-04-25T11:39:38.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.191+0000 7fb59da00640 1 -- start start 2024-04-25T11:39:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.192+0000 7fb59da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980af6b0 0x7fb5980b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.192+0000 7fb59da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5981f9760 0x7fb5981f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.192+0000 7fb59da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5981f6170 0x7fb5981f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.192+0000 7fb59da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb59806d9f0 con 0x7fb5981f6170 2024-04-25T11:39:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.192+0000 7fb59da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb59806d870 con 0x7fb5981f9760 2024-04-25T11:39:38.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.192+0000 7fb59da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb59806db70 con 0x7fb5980af6b0 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb59d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5981f6170 0x7fb5981f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb59d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5981f6170 0x7fb5981f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60422/0 (socket says 172.21.15.19:60422) 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb59d000640 1 -- 172.21.15.19:0/283831444 learned_addr learned my addr 172.21.15.19:0/283831444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb58f400640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5981f9760 0x7fb5981f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb597e00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980af6b0 0x7fb5980b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb59d000640 1 -- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980af6b0 msgr2=0x7fb5980b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb59d000640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980af6b0 0x7fb5980b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb59d000640 1 -- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5981f9760 msgr2=0x7fb5981f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.193+0000 7fb59d000640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5981f9760 0x7fb5981f9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.194+0000 7fb59d000640 1 -- 172.21.15.19:0/283831444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5981fa0b0 con 0x7fb5981f6170 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.194+0000 7fb59d000640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5981f6170 0x7fb5981f6580 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fb59002a8c0 tx=0x7fb5900014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.194+0000 7fb594c00640 1 -- 172.21.15.19:0/283831444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590040070 con 0x7fb5981f6170 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.194+0000 7fb594c00640 1 -- 172.21.15.19:0/283831444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5900050d0 con 0x7fb5981f6170 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.194+0000 7fb594c00640 1 -- 172.21.15.19:0/283831444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590037040 con 0x7fb5981f6170 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.194+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5980b0900 con 0x7fb5981f6170 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.195+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb598150470 con 0x7fb5981f6170 2024-04-25T11:39:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.196+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5980d0750 con 0x7fb5981f6170 2024-04-25T11:39:38.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.197+0000 7fb594c00640 1 -- 172.21.15.19:0/283831444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb590043070 con 0x7fb5981f6170 2024-04-25T11:39:38.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.198+0000 7fb594c00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5740744b0 0x7fb5740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.198+0000 7fb594c00640 1 -- 172.21.15.19:0/283831444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7fb5900b4af0 con 0x7fb5981f6170 2024-04-25T11:39:38.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.199+0000 7fb597e00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5740744b0 0x7fb5740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.200+0000 7fb597e00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5740744b0 0x7fb5740769c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb5981513c0 tx=0x7fb588008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.205+0000 7fb594c00640 1 -- 172.21.15.19:0/283831444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5981fa0b0 con 0x7fb5981f6170 2024-04-25T11:39:38.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.457+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5980d0470 con 0x7fb5981f6170 2024-04-25T11:39:38.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.459+0000 7fb594c00640 1 -- 172.21.15.19:0/283831444 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 266 v266) ==== 95+0+4352 (secure 0 0 0) 0x7fb59007e2b0 con 0x7fb5981f6170 2024-04-25T11:39:38.447 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:38.447 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":266,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":266,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:37.896405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 266 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5740744b0 msgr2=0x7fb5740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5740744b0 0x7fb5740769c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb5981513c0 tx=0x7fb588008040 comp rx=0 tx=0).stop 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5981f6170 msgr2=0x7fb5981f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5981f6170 0x7fb5981f6580 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fb59002a8c0 tx=0x7fb5900014a0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 shutdown_connections 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb5740744b0 0x7fb5740769c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5981f9760 0x7fb5981f9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5981f6170 0x7fb5981f6580 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 --2- 172.21.15.19:0/283831444 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5980af6b0 0x7fb5980b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.462+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 >> 172.21.15.19:0/283831444 conn(0x7fb5980d1720 msgr2=0x7fb598160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:38.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.463+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 shutdown_connections 2024-04-25T11:39:38.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.463+0000 7fb59da00640 1 -- 172.21.15.19:0/283831444 wait complete. 2024-04-25T11:39:38.478 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:38.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.766+0000 7f66b2800640 1 Processor -- start 2024-04-25T11:39:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.766+0000 7f66b2800640 1 -- start start 2024-04-25T11:39:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66ac06c5c0 0x7f66ac0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66ac144370 0x7f66ac0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b2800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66ac0b50e0 con 0x7f66ac0b4cd0 2024-04-25T11:39:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f66ac0d1230 con 0x7f66ac144370 2024-04-25T11:39:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f66ac0d06b0 con 0x7f66ac06c5c0 2024-04-25T11:39:38.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b0a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66ac144370 0x7f66ac0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b0a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66ac144370 0x7f66ac0b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60424/0 (socket says 172.21.15.19:60424) 2024-04-25T11:39:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.769+0000 7f66b0a00640 1 -- 172.21.15.19:0/3300558367 learned_addr learned my addr 172.21.15.19:0/3300558367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.770+0000 7f66b0a00640 1 -- 172.21.15.19:0/3300558367 >> v2:172.21.15.169:3300/0 conn(0x7f66ac06c5c0 msgr2=0x7f66ac0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:38.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.770+0000 7f66b1400640 1 --2- 172.21.15.19:0/3300558367 >> v2:172.21.15.169:3300/0 conn(0x7f66ac06c5c0 0x7f66ac0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.770+0000 7f66b0a00640 1 --2- 172.21.15.19:0/3300558367 >> v2:172.21.15.169:3300/0 conn(0x7f66ac06c5c0 0x7f66ac0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.770+0000 7f66b0a00640 1 -- 172.21.15.19:0/3300558367 >> v1:172.21.15.169:6789/0 conn(0x7f66ac0b4cd0 legacy=0x7f66ac161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.770+0000 7f66b0a00640 1 -- 172.21.15.19:0/3300558367 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac0d0990 con 0x7f66ac144370 2024-04-25T11:39:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.770+0000 7f66b1400640 1 --2- 172.21.15.19:0/3300558367 >> v2:172.21.15.169:3300/0 conn(0x7f66ac06c5c0 0x7f66ac0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.771+0000 7f66b0a00640 1 --2- 172.21.15.19:0/3300558367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac144370 0x7f66ac0b4790 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0726b0 tx=0x7f669802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5e961bc62a8474 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.772+0000 7f66abe00640 1 -- 172.21.15.19:0/3300558367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698037070 con 0x7f66ac144370 2024-04-25T11:39:38.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.772+0000 7f66abe00640 1 -- 172.21.15.19:0/3300558367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6698032440 con 0x7f66ac144370 2024-04-25T11:39:38.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.772+0000 7f66abe00640 1 -- 172.21.15.19:0/3300558367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698032810 con 0x7f66ac144370 2024-04-25T11:39:38.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.772+0000 7f66b2800640 1 -- 172.21.15.19:0/3300558367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac144370 msgr2=0x7f66ac0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.772+0000 7f66b2800640 1 --2- 172.21.15.19:0/3300558367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac144370 0x7f66ac0b4790 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0726b0 tx=0x7f669802a230 comp rx=0 tx=0).stop 2024-04-25T11:39:38.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.773+0000 7f66b2800640 1 -- 172.21.15.19:0/3300558367 shutdown_connections 2024-04-25T11:39:38.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.773+0000 7f66b2800640 1 --2- 172.21.15.19:0/3300558367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac144370 0x7f66ac0b4790 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.773+0000 7f66b2800640 1 --2- 172.21.15.19:0/3300558367 >> v2:172.21.15.169:3300/0 conn(0x7f66ac06c5c0 0x7f66ac0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.773+0000 7f66b2800640 1 -- 172.21.15.19:0/3300558367 >> 172.21.15.19:0/3300558367 conn(0x7f66ac0d1720 msgr2=0x7f66ac0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:38.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.773+0000 7f66b2800640 1 -- 172.21.15.19:0/3300558367 shutdown_connections 2024-04-25T11:39:38.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.773+0000 7f66b2800640 1 -- 172.21.15.19:0/3300558367 wait complete. 2024-04-25T11:39:38.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.775+0000 7f66b2800640 1 Processor -- start 2024-04-25T11:39:38.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.775+0000 7f66b2800640 1 -- start start 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac06c5c0 0x7f66ac160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66ac0b4cd0 0x7f66ac13cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac144370 0x7f66ac13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac0d02d0 con 0x7f66ac06c5c0 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac0d0f50 con 0x7f66ac144370 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66ac0d0450 con 0x7f66ac0b4cd0 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac06c5c0 0x7f66ac160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac06c5c0 0x7f66ac160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60438/0 (socket says 172.21.15.19:60438) 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.777+0000 7f66b1400640 1 -- 172.21.15.19:0/3392884049 learned_addr learned my addr 172.21.15.19:0/3392884049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b0a00640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66ac0b4cd0 0x7f66ac13cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1e00640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac144370 0x7f66ac13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1400640 1 -- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66ac0b4cd0 msgr2=0x7f66ac13cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1400640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66ac0b4cd0 0x7f66ac13cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1400640 1 -- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac144370 msgr2=0x7f66ac13d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:38.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1400640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac144370 0x7f66ac13d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:38.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1400640 1 -- 172.21.15.19:0/3392884049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac161650 con 0x7f66ac06c5c0 2024-04-25T11:39:38.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1e00640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac144370 0x7f66ac13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:38.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.778+0000 7f66b1400640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac06c5c0 0x7f66ac160830 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c820 tx=0x7f66a000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.779+0000 7f66a9600640 1 -- 172.21.15.19:0/3392884049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0015070 con 0x7f66ac06c5c0 2024-04-25T11:39:38.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.779+0000 7f66a9600640 1 -- 172.21.15.19:0/3392884049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0007a30 con 0x7f66ac06c5c0 2024-04-25T11:39:38.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.779+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ac161930 con 0x7f66ac06c5c0 2024-04-25T11:39:38.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.779+0000 7f66a9600640 1 -- 172.21.15.19:0/3392884049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0007de0 con 0x7f66ac06c5c0 2024-04-25T11:39:38.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.779+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ac161ef0 con 0x7f66ac06c5c0 2024-04-25T11:39:38.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.780+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ac050740 con 0x7f66ac06c5c0 2024-04-25T11:39:38.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.782+0000 7f66a9600640 1 -- 172.21.15.19:0/3392884049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66a0018070 con 0x7f66ac06c5c0 2024-04-25T11:39:38.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.782+0000 7f66a9600640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f668c0744e0 0x7f668c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:38.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.783+0000 7f66a9600640 1 -- 172.21.15.19:0/3392884049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7f66a0095b10 con 0x7f66ac06c5c0 2024-04-25T11:39:38.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.783+0000 7f66b0a00640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f668c0744e0 0x7f668c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:38.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.785+0000 7f66b0a00640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f668c0744e0 0x7f668c0769f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6698034000 tx=0x7f669802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:38.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:38.787+0000 7f66a9600640 1 -- 172.21.15.19:0/3392884049 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66a005f920 con 0x7f66ac06c5c0 2024-04-25T11:39:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.041+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66ac159c90 con 0x7f66ac06c5c0 2024-04-25T11:39:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.043+0000 7f66a9600640 1 -- 172.21.15.19:0/3392884049 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) ==== 95+0+5179 (secure 0 0 0) 0x7f66a005f220 con 0x7f66ac06c5c0 2024-04-25T11:39:39.031 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:39.031 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":267,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1755072450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1755072450},{"type":"v1","addr":"172.21.15.19:6835","nonce":1755072450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":267}],"filesystems":[{"mdsmap":{"epoch":266,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:37.896405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 267 2024-04-25T11:39:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f668c0744e0 msgr2=0x7f668c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f668c0744e0 0x7f668c0769f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6698034000 tx=0x7f669802e000 comp rx=0 tx=0).stop 2024-04-25T11:39:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac06c5c0 msgr2=0x7f66ac160830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac06c5c0 0x7f66ac160830 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c820 tx=0x7f66a000cce0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 shutdown_connections 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f668c0744e0 0x7f668c0769f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66ac144370 0x7f66ac13d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66ac0b4cd0 0x7f66ac13cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 --2- 172.21.15.19:0/3392884049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66ac06c5c0 0x7f66ac160830 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.045+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 >> 172.21.15.19:0/3392884049 conn(0x7f66ac0d1720 msgr2=0x7f66ac15c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.046+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 shutdown_connections 2024-04-25T11:39:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.046+0000 7f66b2800640 1 -- 172.21.15.19:0/3392884049 wait complete. 2024-04-25T11:39:39.063 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:39.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:39.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.357+0000 7fd84de00640 1 Processor -- start 2024-04-25T11:39:39.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.358+0000 7fd84de00640 1 -- start start 2024-04-25T11:39:39.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.360+0000 7fd84de00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd8481600a0 0x7fd8480b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:39.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.360+0000 7fd84de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8481604b0 0x7fd8480b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:39.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.360+0000 7fd84de00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd84806fd10 con 0x7fd8480b4eb0 2024-04-25T11:39:39.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.360+0000 7fd84de00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd8480af430 con 0x7fd8481600a0 2024-04-25T11:39:39.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.360+0000 7fd84de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd84806fff0 con 0x7fd8481604b0 2024-04-25T11:39:39.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.361+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8481604b0 0x7fd8480b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:39.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.361+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8481604b0 0x7fd8480b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:42080/0 (socket says 172.21.15.19:42080) 2024-04-25T11:39:39.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.361+0000 7fd83fe00640 1 -- 172.21.15.19:0/1971833316 learned_addr learned my addr 172.21.15.19:0/1971833316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:39.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.361+0000 7fd846a00640 1 --2- 172.21.15.19:0/1971833316 >> v2:172.21.15.169:3300/0 conn(0x7fd8481600a0 0x7fd8480b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.362+0000 7fd83fe00640 1 -- 172.21.15.19:0/1971833316 >> v2:172.21.15.169:3300/0 conn(0x7fd8481600a0 msgr2=0x7fd8480b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.362+0000 7fd83fe00640 1 --2- 172.21.15.19:0/1971833316 >> v2:172.21.15.169:3300/0 conn(0x7fd8481600a0 0x7fd8480b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.362+0000 7fd83fe00640 1 -- 172.21.15.19:0/1971833316 >> v1:172.21.15.169:6789/0 conn(0x7fd8480b4eb0 legacy=0x7fd84806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.362+0000 7fd83fe00640 1 -- 172.21.15.19:0/1971833316 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8480af660 con 0x7fd8481604b0 2024-04-25T11:39:39.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.362+0000 7fd846a00640 1 --2- 172.21.15.19:0/1971833316 >> v2:172.21.15.169:3300/0 conn(0x7fd8481600a0 0x7fd8480b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:39:39.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.363+0000 7fd83fe00640 1 --2- 172.21.15.19:0/1971833316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8481604b0 0x7fd8480b0140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd830009870 tx=0x7fd83002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c02233790a5efb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:39.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.364+0000 7fd846000640 1 -- 172.21.15.19:0/1971833316 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830037070 con 0x7fd8481604b0 2024-04-25T11:39:39.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.364+0000 7fd846000640 1 -- 172.21.15.19:0/1971833316 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd830032440 con 0x7fd8481604b0 2024-04-25T11:39:39.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.364+0000 7fd846000640 1 -- 172.21.15.19:0/1971833316 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830032810 con 0x7fd8481604b0 2024-04-25T11:39:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 -- 172.21.15.19:0/1971833316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8481604b0 msgr2=0x7fd8480b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 --2- 172.21.15.19:0/1971833316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8481604b0 0x7fd8480b0140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd830009870 tx=0x7fd83002a270 comp rx=0 tx=0).stop 2024-04-25T11:39:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 -- 172.21.15.19:0/1971833316 shutdown_connections 2024-04-25T11:39:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 --2- 172.21.15.19:0/1971833316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8481604b0 0x7fd8480b0140 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 --2- 172.21.15.19:0/1971833316 >> v2:172.21.15.169:3300/0 conn(0x7fd8481600a0 0x7fd8480b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 -- 172.21.15.19:0/1971833316 >> 172.21.15.19:0/1971833316 conn(0x7fd8480d1ae0 msgr2=0x7fd8480d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:39.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 -- 172.21.15.19:0/1971833316 shutdown_connections 2024-04-25T11:39:39.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.365+0000 7fd84de00640 1 -- 172.21.15.19:0/1971833316 wait complete. 2024-04-25T11:39:39.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.368+0000 7fd84de00640 1 Processor -- start 2024-04-25T11:39:39.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.368+0000 7fd84de00640 1 -- start start 2024-04-25T11:39:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.369+0000 7fd84de00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8480b4eb0 0x7fd848146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.369+0000 7fd84de00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8481600a0 0x7fd8480cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.369+0000 7fd84de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8481495e0 0x7fd8481499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.369+0000 7fd84de00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd84806ccd0 con 0x7fd8481495e0 2024-04-25T11:39:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.369+0000 7fd84de00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480afc20 con 0x7fd8481600a0 2024-04-25T11:39:39.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.369+0000 7fd84de00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd84806ce50 con 0x7fd8480b4eb0 2024-04-25T11:39:39.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8481495e0 0x7fd8481499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:39.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8481495e0 0x7fd8481499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60444/0 (socket says 172.21.15.19:60444) 2024-04-25T11:39:39.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 -- 172.21.15.19:0/3735380783 learned_addr learned my addr 172.21.15.19:0/3735380783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:39.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd846a00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8480b4eb0 0x7fd848146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:39.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd83fe00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8481600a0 0x7fd8480cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:39.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 -- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8480b4eb0 msgr2=0x7fd848146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8480b4eb0 0x7fd848146ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 -- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8481600a0 msgr2=0x7fd8480cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8481600a0 0x7fd8480cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd847400640 1 -- 172.21.15.19:0/3735380783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd848149f30 con 0x7fd8481495e0 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.370+0000 7fd83fe00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8481600a0 0x7fd8480cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.371+0000 7fd847400640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8481495e0 0x7fd8481499f0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c6e0 tx=0x7fd84000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.371+0000 7fd83f400640 1 -- 172.21.15.19:0/3735380783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840003b70 con 0x7fd8481495e0 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.371+0000 7fd83f400640 1 -- 172.21.15.19:0/3735380783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd840003d30 con 0x7fd8481495e0 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.371+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8481624d0 con 0x7fd8481495e0 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.371+0000 7fd83f400640 1 -- 172.21.15.19:0/3735380783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840005310 con 0x7fd8481495e0 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.371+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd848162c20 con 0x7fd8481495e0 2024-04-25T11:39:39.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.373+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8480b19b0 con 0x7fd8481495e0 2024-04-25T11:39:39.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.374+0000 7fd83f400640 1 -- 172.21.15.19:0/3735380783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd8400054b0 con 0x7fd8481495e0 2024-04-25T11:39:39.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.376+0000 7fd83f400640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8240744b0 0x7fd8240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:39.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.376+0000 7fd83f400640 1 -- 172.21.15.19:0/3735380783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7fd840095ac0 con 0x7fd8481495e0 2024-04-25T11:39:39.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.376+0000 7fd846a00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8240744b0 0x7fd8240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:39.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.378+0000 7fd846a00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8240744b0 0x7fd8240769c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd838009a30 tx=0x7fd838008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:39.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.381+0000 7fd83f400640 1 -- 172.21.15.19:0/3735380783 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8480b19b0 con 0x7fd8481495e0 2024-04-25T11:39:39.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.640+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd848050740 con 0x7fd8481495e0 2024-04-25T11:39:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.642+0000 7fd83f400640 1 -- 172.21.15.19:0/3735380783 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) ==== 95+0+5179 (secure 0 0 0) 0x7fd848149f30 con 0x7fd8481495e0 2024-04-25T11:39:39.630 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:39.631 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":267,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10095,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1755072450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1755072450},{"type":"v1","addr":"172.21.15.19:6835","nonce":1755072450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":267}],"filesystems":[{"mdsmap":{"epoch":266,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:37.896405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 267 2024-04-25T11:39:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8240744b0 msgr2=0x7fd8240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8240744b0 0x7fd8240769c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd838009a30 tx=0x7fd838008040 comp rx=0 tx=0).stop 2024-04-25T11:39:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8481495e0 msgr2=0x7fd8481499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8481495e0 0x7fd8481499f0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c6e0 tx=0x7fd84000cba0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 shutdown_connections 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd8240744b0 0x7fd8240769c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8481600a0 0x7fd8480cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8481495e0 0x7fd8481499f0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 --2- 172.21.15.19:0/3735380783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd8480b4eb0 0x7fd848146ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.644+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 >> 172.21.15.19:0/3735380783 conn(0x7fd8480d1ae0 msgr2=0x7fd8480b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.645+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 shutdown_connections 2024-04-25T11:39:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:39.645+0000 7fd84de00640 1 -- 172.21.15.19:0/3735380783 wait complete. 2024-04-25T11:39:39.658 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2024-04-25T11:39:39.658 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2024-04-25T11:39:39.659 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:39.659 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:39.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:39:39.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:39.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:39.710 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:39.756 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:39:39.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:39:39.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.103+0000 7f4911600640 1 Processor -- start 2024-04-25T11:39:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.103+0000 7f4911600640 1 -- start start 2024-04-25T11:39:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.105+0000 7f4911600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c0b4ea0 0x7f490c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.105+0000 7f4911600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f490c144370 0x7f490c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.105+0000 7f4911600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f490c0d1230 con 0x7f490c06c5c0 2024-04-25T11:39:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.105+0000 7f4911600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f490c0d06b0 con 0x7f490c144370 2024-04-25T11:39:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.105+0000 7f4911600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f490c0d0990 con 0x7f490c0b4ea0 2024-04-25T11:39:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.106+0000 7f4903400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c0b4ea0 0x7f490c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.106+0000 7f4903400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c0b4ea0 0x7f490c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:42106/0 (socket says 172.21.15.19:42106) 2024-04-25T11:39:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.106+0000 7f4903400640 1 -- 172.21.15.19:0/140054877 learned_addr learned my addr 172.21.15.19:0/140054877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.106+0000 7f4910c00640 1 --2- 172.21.15.19:0/140054877 >> v2:172.21.15.169:3300/0 conn(0x7f490c144370 0x7f490c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.107+0000 7f4903400640 1 -- 172.21.15.19:0/140054877 >> v2:172.21.15.169:3300/0 conn(0x7f490c144370 msgr2=0x7f490c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.107+0000 7f4903400640 1 --2- 172.21.15.19:0/140054877 >> v2:172.21.15.169:3300/0 conn(0x7f490c144370 0x7f490c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.107+0000 7f4903400640 1 -- 172.21.15.19:0/140054877 >> v1:172.21.15.169:6789/0 conn(0x7f490c06c5c0 legacy=0x7f490c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.107+0000 7f4903400640 1 -- 172.21.15.19:0/140054877 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c0d0c70 con 0x7f490c0b4ea0 2024-04-25T11:39:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.108+0000 7f4903400640 1 --2- 172.21.15.19:0/140054877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c0b4ea0 0x7f490c0cfa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009870 tx=0x7f48f402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18f93ecec361ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.109+0000 7f490b400640 1 -- 172.21.15.19:0/140054877 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4037070 con 0x7f490c0b4ea0 2024-04-25T11:39:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.109+0000 7f490b400640 1 -- 172.21.15.19:0/140054877 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f4032440 con 0x7f490c0b4ea0 2024-04-25T11:39:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.109+0000 7f490b400640 1 -- 172.21.15.19:0/140054877 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4032810 con 0x7f490c0b4ea0 2024-04-25T11:39:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 -- 172.21.15.19:0/140054877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c0b4ea0 msgr2=0x7f490c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 --2- 172.21.15.19:0/140054877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c0b4ea0 0x7f490c0cfa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009870 tx=0x7f48f402a270 comp rx=0 tx=0).stop 2024-04-25T11:39:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 -- 172.21.15.19:0/140054877 shutdown_connections 2024-04-25T11:39:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 --2- 172.21.15.19:0/140054877 >> v2:172.21.15.169:3300/0 conn(0x7f490c144370 0x7f490c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 --2- 172.21.15.19:0/140054877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c0b4ea0 0x7f490c0cfa70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 -- 172.21.15.19:0/140054877 >> 172.21.15.19:0/140054877 conn(0x7f490c0d1720 msgr2=0x7f490c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 -- 172.21.15.19:0/140054877 shutdown_connections 2024-04-25T11:39:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.110+0000 7f4911600640 1 -- 172.21.15.19:0/140054877 wait complete. 2024-04-25T11:39:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.112+0000 7f4911600640 1 Processor -- start 2024-04-25T11:39:40.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.112+0000 7f4911600640 1 -- start start 2024-04-25T11:39:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4911600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f490c06c5c0 0x7f490c136120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4911600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c144370 0x7f490c070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4911600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c140200 0x7f490c140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4911600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f490c0d0450 con 0x7f490c144370 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4911600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f490c0d02d0 con 0x7f490c06c5c0 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4911600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f490c147760 con 0x7f490c140200 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4903400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c144370 0x7f490c070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4903400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c144370 0x7f490c070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60460/0 (socket says 172.21.15.19:60460) 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f4903400640 1 -- 172.21.15.19:0/4138069602 learned_addr learned my addr 172.21.15.19:0/4138069602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.114+0000 7f490be00640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f490c06c5c0 0x7f490c136120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4910c00640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c140200 0x7f490c140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4903400640 1 -- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c140200 msgr2=0x7f490c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4903400640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c140200 0x7f490c140610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4903400640 1 -- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f490c06c5c0 msgr2=0x7f490c136120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4903400640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f490c06c5c0 0x7f490c136120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4903400640 1 -- 172.21.15.19:0/4138069602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c140b50 con 0x7f490c144370 2024-04-25T11:39:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4903400640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c144370 0x7f490c070c00 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f48f4000c00 tx=0x7f48f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4908c00640 1 -- 172.21.15.19:0/4138069602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4037040 con 0x7f490c144370 2024-04-25T11:39:40.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4908c00640 1 -- 172.21.15.19:0/4138069602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f4042070 con 0x7f490c144370 2024-04-25T11:39:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4908c00640 1 -- 172.21.15.19:0/4138069602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4030880 con 0x7f490c144370 2024-04-25T11:39:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c06f720 con 0x7f490c144370 2024-04-25T11:39:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.115+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c06fce0 con 0x7f490c144370 2024-04-25T11:39:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.116+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c050740 con 0x7f490c144370 2024-04-25T11:39:40.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.118+0000 7f4908c00640 1 -- 172.21.15.19:0/4138069602 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48f4030a20 con 0x7f490c144370 2024-04-25T11:39:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.119+0000 7f4908c00640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f48e80744b0 0x7f48e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.119+0000 7f4908c00640 1 -- 172.21.15.19:0/4138069602 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7f48f40b64a0 con 0x7f490c144370 2024-04-25T11:39:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.119+0000 7f490be00640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f48e80744b0 0x7f48e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.121+0000 7f490be00640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f48e80744b0 0x7f48e80769c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007fd0 tx=0x7f48fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.124+0000 7f4908c00640 1 -- 172.21.15.19:0/4138069602 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f490c140b50 con 0x7f490c144370 2024-04-25T11:39:40.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.361+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f490c0b5e20 con 0x7f490c144370 2024-04-25T11:39:40.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.922+0000 7f4908c00640 1 -- 172.21.15.19:0/4138069602 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v268) ==== 161+0+0 (secure 0 0 0) 0x7f48f4080340 con 0x7f490c144370 2024-04-25T11:39:40.911 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f48e80744b0 msgr2=0x7f48e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f48e80744b0 0x7f48e80769c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007fd0 tx=0x7f48fc009040 comp rx=0 tx=0).stop 2024-04-25T11:39:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c144370 msgr2=0x7f490c070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c144370 0x7f490c070c00 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f48f4000c00 tx=0x7f48f40046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 shutdown_connections 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f48e80744b0 0x7f48e80769c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f490c140200 0x7f490c140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f490c144370 0x7f490c070c00 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 --2- 172.21.15.19:0/4138069602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f490c06c5c0 0x7f490c136120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 >> 172.21.15.19:0/4138069602 conn(0x7f490c0d1720 msgr2=0x7f490c15c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.926+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 shutdown_connections 2024-04-25T11:39:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:40.927+0000 7f4911600640 1 -- 172.21.15.19:0/4138069602 wait complete. 2024-04-25T11:39:40.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:41.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.247+0000 7f20fd600640 1 Processor -- start 2024-04-25T11:39:41.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.247+0000 7f20fd600640 1 -- start start 2024-04-25T11:39:41.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.250+0000 7f20fd600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20f80d1070 0x7f20f8162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:41.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.250+0000 7f20fd600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f806fc70 con 0x7f20f8144e60 2024-04-25T11:39:41.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.250+0000 7f20fd600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f806ff50 con 0x7f20f8144350 2024-04-25T11:39:41.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.250+0000 7f20fd600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f20f8070230 con 0x7f20f80d1070 2024-04-25T11:39:41.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.250+0000 7f20efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20f80d1070 0x7f20f8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:41.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.250+0000 7f20efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20f80d1070 0x7f20f8162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60464/0 (socket says 172.21.15.19:60464) 2024-04-25T11:39:41.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.250+0000 7f20efe00640 1 -- 172.21.15.19:0/2262212232 learned_addr learned my addr 172.21.15.19:0/2262212232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:41.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.251+0000 7f20efe00640 1 -- 172.21.15.19:0/2262212232 >> v1:172.21.15.169:6789/0 conn(0x7f20f8144350 legacy=0x7f20f8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:41.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.251+0000 7f20efe00640 1 -- 172.21.15.19:0/2262212232 >> v1:172.21.15.19:6789/0 conn(0x7f20f8144e60 legacy=0x7f20f8145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:41.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.251+0000 7f20f7400640 1 -- 172.21.15.19:0/2262212232 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778240919 0 0) 0x7f20f806fc70 con 0x7f20f8144e60 2024-04-25T11:39:41.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.251+0000 7f20efe00640 1 -- 172.21.15.19:0/2262212232 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f8070510 con 0x7f20f80d1070 2024-04-25T11:39:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.252+0000 7f20efe00640 1 --2- 172.21.15.19:0/2262212232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8162200 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009870 tx=0x7f20e002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9c55f3792e2c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.252+0000 7f20f7400640 1 -- 172.21.15.19:0/2262212232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0037070 con 0x7f20f80d1070 2024-04-25T11:39:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20f7400640 1 -- 172.21.15.19:0/2262212232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e0032440 con 0x7f20f80d1070 2024-04-25T11:39:41.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20f7400640 1 -- 172.21.15.19:0/2262212232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0032810 con 0x7f20f80d1070 2024-04-25T11:39:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20fd600640 1 -- 172.21.15.19:0/2262212232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 msgr2=0x7f20f8162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20fd600640 1 --2- 172.21.15.19:0/2262212232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8162200 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009870 tx=0x7f20e002a2a0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20fd600640 1 -- 172.21.15.19:0/2262212232 shutdown_connections 2024-04-25T11:39:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20fd600640 1 --2- 172.21.15.19:0/2262212232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8162200 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20fd600640 1 -- 172.21.15.19:0/2262212232 >> 172.21.15.19:0/2262212232 conn(0x7f20f80d1720 msgr2=0x7f20f80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:41.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20fd600640 1 -- 172.21.15.19:0/2262212232 shutdown_connections 2024-04-25T11:39:41.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.253+0000 7f20fd600640 1 -- 172.21.15.19:0/2262212232 wait complete. 2024-04-25T11:39:41.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.256+0000 7f20fd600640 1 Processor -- start 2024-04-25T11:39:41.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.256+0000 7f20fd600640 1 -- start start 2024-04-25T11:39:41.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.257+0000 7f20fd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.257+0000 7f20fd600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f8144350 0x7f20f815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.257+0000 7f20fd600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f8144e60 0x7f20f8160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.257+0000 7f20fd600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8162740 con 0x7f20f80d1070 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.257+0000 7f20fd600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8147910 con 0x7f20f8144350 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.257+0000 7f20fd600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20f81628c0 con 0x7f20f8144e60 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8132750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60474/0 (socket says 172.21.15.19:60474) 2024-04-25T11:39:41.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 -- 172.21.15.19:0/2332282807 learned_addr learned my addr 172.21.15.19:0/2332282807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:41.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20fcc00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f8144e60 0x7f20f8160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 -- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f8144e60 msgr2=0x7f20f8160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f8144e60 0x7f20f8160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 -- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f8144350 msgr2=0x7f20f815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20efe00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f8144350 0x7f20f815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f8144350 0x7f20f815d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.258+0000 7f20f7e00640 1 -- 172.21.15.19:0/2332282807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f8132c90 con 0x7f20f80d1070 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.259+0000 7f20f7e00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8132750 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b420 tx=0x7f20e800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.259+0000 7f20f4c00640 1 -- 172.21.15.19:0/2332282807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011830 con 0x7f20f80d1070 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.259+0000 7f20f4c00640 1 -- 172.21.15.19:0/2332282807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e8016070 con 0x7f20f80d1070 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.259+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f81e8e40 con 0x7f20f80d1070 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.259+0000 7f20f4c00640 1 -- 172.21.15.19:0/2332282807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011c10 con 0x7f20f80d1070 2024-04-25T11:39:41.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.259+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81e9400 con 0x7f20f80d1070 2024-04-25T11:39:41.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.260+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f8050740 con 0x7f20f80d1070 2024-04-25T11:39:41.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.262+0000 7f20f4c00640 1 -- 172.21.15.19:0/2332282807 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f20e801b460 con 0x7f20f80d1070 2024-04-25T11:39:41.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.263+0000 7f20f4c00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20d40744e0 0x7f20d40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:41.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.263+0000 7f20f4c00640 1 -- 172.21.15.19:0/2332282807 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7f20e8095660 con 0x7f20f80d1070 2024-04-25T11:39:41.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.264+0000 7f20efe00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20d40744e0 0x7f20d40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:41.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.266+0000 7f20efe00640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20d40744e0 0x7f20d40769f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f20e0034000 tx=0x7f20e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:41.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.268+0000 7f20f4c00640 1 -- 172.21.15.19:0/2332282807 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20e8020020 con 0x7f20f80d1070 2024-04-25T11:39:41.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.549+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f20f812fd00 con 0x7f20f80d1070 2024-04-25T11:39:41.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.940+0000 7f20f4c00640 1 -- 172.21.15.19:0/2332282807 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v269) ==== 101+0+0 (secure 0 0 0) 0x7f20e805f470 con 0x7f20f80d1070 2024-04-25T11:39:41.928 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:39:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20d40744e0 msgr2=0x7f20d40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20d40744e0 0x7f20d40769f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f20e0034000 tx=0x7f20e002e000 comp rx=0 tx=0).stop 2024-04-25T11:39:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 msgr2=0x7f20f8132750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8132750 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b420 tx=0x7f20e800b8e0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 shutdown_connections 2024-04-25T11:39:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20d40744e0 0x7f20d40769f0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20f8144350 0x7f20f815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20f8144e60 0x7f20f8160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 --2- 172.21.15.19:0/2332282807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20f80d1070 0x7f20f8132750 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 >> 172.21.15.19:0/2332282807 conn(0x7f20f80d1720 msgr2=0x7f20f8142050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 shutdown_connections 2024-04-25T11:39:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:41.944+0000 7f20fd600640 1 -- 172.21.15.19:0/2332282807 wait complete. 2024-04-25T11:39:41.961 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:42.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.156+0000 7fa0d805ef40 1 Processor -- start 2024-04-25T11:39:42.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.156+0000 7fa0d805ef40 1 -- start start 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d805ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 0x561db58a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d805ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db58535f0 0x561db58977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d805ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db592cdd0 0x561db5883950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d805ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561db589be70 con 0x561db58535f0 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d805ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561db57f9d20 con 0x561db58531e0 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d805ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561db589c110 con 0x561db592cdd0 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db58535f0 0x561db58977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db58535f0 0x561db58977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60492/0 (socket says 172.21.15.19:60492) 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.157+0000 7fa0d0800640 1 -- 172.21.15.19:0/1397038405 learned_addr learned my addr 172.21.15.19:0/1397038405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.158+0000 7fa0d1c00640 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db592cdd0 0x561db5883950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.158+0000 7fa0d0800640 1 -- 172.21.15.19:0/1397038405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db592cdd0 msgr2=0x561db5883950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:42.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.158+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db592cdd0 0x561db5883950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.158+0000 7fa0d0800640 1 -- 172.21.15.19:0/1397038405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 msgr2=0x561db58a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:42.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.158+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 0x561db58a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.158+0000 7fa0d0800640 1 -- 172.21.15.19:0/1397038405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db58a0a10 con 0x561db58535f0 2024-04-25T11:39:42.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.158+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db58535f0 0x561db58977c0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009600 tx=0x7fa0bc009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3629e699191dae15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0cf400640 1 -- 172.21.15.19:0/1397038405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc017070 con 0x561db58535f0 2024-04-25T11:39:42.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0cf400640 1 -- 172.21.15.19:0/1397038405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0066b0 con 0x561db58535f0 2024-04-25T11:39:42.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0cf400640 1 -- 172.21.15.19:0/1397038405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc006a60 con 0x561db58535f0 2024-04-25T11:39:42.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1397038405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db58535f0 msgr2=0x561db58977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db58535f0 0x561db58977c0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009600 tx=0x7fa0bc009ac0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1397038405 shutdown_connections 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db592cdd0 0x561db5883950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db58535f0 0x561db58977c0 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1397038405 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 0x561db58a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1397038405 >> 172.21.15.19:0/1397038405 conn(0x561db5775ac0 msgr2=0x561db589fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1397038405 shutdown_connections 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.159+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1397038405 wait complete. 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.160+0000 7fa0d805ef40 1 Processor -- start 2024-04-25T11:39:42.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.161+0000 7fa0d805ef40 1 -- start start 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d805ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 0x561db59b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d805ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db58535f0 0x561db592b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d805ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db592cdd0 0x561db5922380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d805ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561db5885880 con 0x561db592cdd0 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d805ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561db58831c0 con 0x561db58531e0 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d805ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561db58854d0 con 0x561db58535f0 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db592cdd0 0x561db5922380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db592cdd0 0x561db5922380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60500/0 (socket says 172.21.15.19:60500) 2024-04-25T11:39:42.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.162+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 learned_addr learned my addr 172.21.15.19:0/1567907031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db58535f0 0x561db592b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 0x561db59b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db58535f0 msgr2=0x561db592b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1c00640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db58535f0 0x561db592b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 msgr2=0x561db59b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1c00640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 0x561db59b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db5899560 con 0x561db592cdd0 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561db58531e0 0x561db59b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561db58535f0 0x561db592b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.163+0000 7fa0d1c00640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db592cdd0 0x561db5922380 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800a5a0 tx=0x7fa0c8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.164+0000 7fa0ce000640 1 -- 172.21.15.19:0/1567907031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c806b070 con 0x561db592cdd0 2024-04-25T11:39:42.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.164+0000 7fa0ce000640 1 -- 172.21.15.19:0/1567907031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c8053910 con 0x561db592cdd0 2024-04-25T11:39:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.164+0000 7fa0ce000640 1 -- 172.21.15.19:0/1567907031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8053d00 con 0x561db592cdd0 2024-04-25T11:39:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.164+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561db5852800 con 0x561db592cdd0 2024-04-25T11:39:42.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.164+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db5922d40 con 0x561db592cdd0 2024-04-25T11:39:42.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.166+0000 7fa0ce000640 1 -- 172.21.15.19:0/1567907031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa0c805c420 con 0x561db592cdd0 2024-04-25T11:39:42.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.167+0000 7fa0ce000640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0b0072e90 0x7fa0b00753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.167+0000 7fa0ce000640 1 -- 172.21.15.19:0/1567907031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7fa0c80d8520 con 0x561db592cdd0 2024-04-25T11:39:42.155 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.168+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561db5925e50 0x561db59c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.168+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db5881b20 con 0x561db5925e50 2024-04-25T11:39:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.168+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0b0072e90 0x7fa0b00753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.168+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561db5925e50 0x561db59c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.170+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561db5925e50 0x561db59c4fd0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.170+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0b0072e90 0x7fa0b00753a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400b4a0 tx=0x7fa0c4007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.173+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v120'846 uv820 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.173+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db592b830 con 0x561db5925e50 2024-04-25T11:39:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.176+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v120'847 uv847 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.176+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59228c0 con 0x561db5925e50 2024-04-25T11:39:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.180+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v120'848 uv848 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.168 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.180+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59ce8f0 con 0x561db5925e50 2024-04-25T11:39:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.184+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'849 uv849 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.184+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cec90 con 0x561db5925e50 2024-04-25T11:39:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.188+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'850 uv850 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.188+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cd8e0 con 0x561db5925e50 2024-04-25T11:39:42.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.192+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.192+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdcd0 con 0x561db5925e50 2024-04-25T11:39:42.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.197+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v120'852 uv826 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.197+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59cdcd0 con 0x561db5925e50 2024-04-25T11:39:42.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.200+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v120'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.200+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.191 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.203+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v120'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.204+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.208+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.208+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.211+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.211+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.215+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.215+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.220+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v120'858 uv833 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.220+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.224+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'859 uv859 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.224+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.215 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.228+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v120'860 uv860 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.228+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.232+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'861 uv861 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.232+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.236+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'862 uv862 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.236+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdc80 con 0x561db5925e50 2024-04-25T11:39:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.240+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'863 uv863 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.240+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59ce040 con 0x561db5925e50 2024-04-25T11:39:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.244+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'864 uv864 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.245+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59ce4b0 con 0x561db5925e50 2024-04-25T11:39:42.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.249+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v120'865 uv839 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.249+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59ce4b0 con 0x561db5925e50 2024-04-25T11:39:42.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.253+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v120'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.254+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59ce440 con 0x561db5925e50 2024-04-25T11:39:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.257+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v120'867 uv867 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.258+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59ce440 con 0x561db5925e50 2024-04-25T11:39:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.262+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'868 uv868 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.262+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.266+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'869 uv869 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.266+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.271+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'870 uv870 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.271+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.275+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v120'871 uv845 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.275+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.279+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v120'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.280+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.284+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v120'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.272 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.284+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.288+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.289+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.293+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.293+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db5925e50 2024-04-25T11:39:42.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.297+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.297+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561db59cf030 0x561db59d1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.297+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59cdf40 con 0x561db59cf030 2024-04-25T11:39:42.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.298+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561db59cf030 0x561db59d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.300+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561db59cf030 0x561db59d1450 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.304+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'617 uv600 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.304+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d1990 con 0x561db59cf030 2024-04-25T11:39:42.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.309+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'618 uv618 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.309+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d1d30 con 0x561db59cf030 2024-04-25T11:39:42.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.313+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'619 uv619 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.301 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:42.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.314+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d20d0 con 0x561db59cf030 2024-04-25T11:39:42.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.318+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v120'620 uv620 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.318+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2470 con 0x561db59cf030 2024-04-25T11:39:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.322+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'621 uv621 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.322+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2810 con 0x561db59cf030 2024-04-25T11:39:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.327+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v120'622 uv605 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.328+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.332+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v120'623 uv623 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.332+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.336+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v120'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.324 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:42.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.336+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.341+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.341+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.345+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.345+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.350+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v120'627 uv610 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.350+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.354+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v120'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.354+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.358+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v120'629 uv629 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.346 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:42.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.358+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d72a0 con 0x561db59cf030 2024-04-25T11:39:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.364+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v120'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.364+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d72a0 con 0x561db59cf030 2024-04-25T11:39:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.368+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.368+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.373+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v120'632 uv616 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.373+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.377+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v120'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.377+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.369 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.381+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v120'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.381+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d3bc0 con 0x561db59cf030 2024-04-25T11:39:42.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.385+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'635 uv635 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.385+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d3bc0 con 0x561db59cf030 2024-04-25T11:39:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.389+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.389+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d2bb0 con 0x561db59cf030 2024-04-25T11:39:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.394+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59cf030 2024-04-25T11:39:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.394+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561db59d30c0 0x561db59d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.394+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d5af0 con 0x561db59d30c0 2024-04-25T11:39:42.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.395+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561db59d30c0 0x561db59d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.397+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561db59d30c0 0x561db59d54e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.401+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.401+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d5ec0 con 0x561db59d30c0 2024-04-25T11:39:42.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.405+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v120'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.406+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d5ec0 con 0x561db59d30c0 2024-04-25T11:39:42.397 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:42.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.409+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v120'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.410+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d6ed0 con 0x561db59d30c0 2024-04-25T11:39:42.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.414+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.414+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d6ed0 con 0x561db59d30c0 2024-04-25T11:39:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.418+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.418+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d5ec0 con 0x561db59d30c0 2024-04-25T11:39:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.422+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.422+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d5ec0 con 0x561db5925e50 2024-04-25T11:39:42.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.427+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'124 uv123 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.427+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d5ec0 con 0x561db5925e50 2024-04-25T11:39:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.431+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'125 uv125 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.431+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d5ec0 con 0x561db5925e50 2024-04-25T11:39:42.422 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.435+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'126 uv126 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.435+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d5ec0 con 0x561db5925e50 2024-04-25T11:39:42.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.439+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'127 uv127 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.439+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561db59d6810 0x561db59d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.439+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9170 con 0x561db59d6810 2024-04-25T11:39:42.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.440+0000 7fa0d1c00640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561db59d6810 0x561db59d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.441+0000 7fa0d1c00640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561db59d6810 0x561db59d8c30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.445+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.445+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d9510 con 0x561db59d6810 2024-04-25T11:39:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.450+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v120'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.450+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d99c0 con 0x561db59d6810 2024-04-25T11:39:42.441 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:42.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.454+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v120'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.454+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59db270 con 0x561db59d6810 2024-04-25T11:39:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.458+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.458+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59db610 con 0x561db59d6810 2024-04-25T11:39:42.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.462+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.462+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59da260 con 0x561db59d6810 2024-04-25T11:39:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.466+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.466+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561db59dab30 0x561db59dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.466+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59ddf00 con 0x561db59dab30 2024-04-25T11:39:42.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.467+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561db59dab30 0x561db59dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.469+0000 7fa0d1200640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561db59dab30 0x561db59dd9c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.473+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v120'305 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.473+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59ddf00 con 0x561db59dab30 2024-04-25T11:39:42.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.477+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v120'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.478+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59ddf80 con 0x561db59dab30 2024-04-25T11:39:42.469 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:42.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.481+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v120'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.482+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59df380 con 0x561db59dab30 2024-04-25T11:39:42.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.485+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.485+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59df380 con 0x561db59dab30 2024-04-25T11:39:42.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.489+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.489+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59de370 con 0x561db59dab30 2024-04-25T11:39:42.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.494+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.494+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59de370 con 0x561db59dab30 2024-04-25T11:39:42.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.498+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v120'311 uv304 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.498+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59de370 con 0x561db59dab30 2024-04-25T11:39:42.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.502+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v120'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.502+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59de370 con 0x561db59dab30 2024-04-25T11:39:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.506+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v120'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.494 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:42.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.507+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59de370 con 0x561db59dab30 2024-04-25T11:39:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.511+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.511+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59de370 con 0x561db59dab30 2024-04-25T11:39:42.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.515+0000 7fa0d1200640 1 -- 172.21.15.19:0/1567907031 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c4010890 con 0x561db59dab30 2024-04-25T11:39:42.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.515+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561db59de880 0x561db59e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.515+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e12e0 con 0x561db59de880 2024-04-25T11:39:42.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.516+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561db59de880 0x561db59e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.518+0000 7fa0d0800640 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561db59de880 0x561db59e0ca0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.522+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v120'138 uv137 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.522+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59e12e0 con 0x561db59de880 2024-04-25T11:39:42.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.526+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v120'139 uv139 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.526+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db59de880 2024-04-25T11:39:42.518 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:42.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.530+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v120'140 uv140 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.530+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db59de880 2024-04-25T11:39:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.534+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'141 uv141 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.535+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db59de880 2024-04-25T11:39:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.539+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'142 uv142 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.539+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db5925e50 2024-04-25T11:39:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.543+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.543+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db5925e50 2024-04-25T11:39:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.547+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v120'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.547+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db5925e50 2024-04-25T11:39:42.539 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.551+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v120'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.551+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2740 con 0x561db5925e50 2024-04-25T11:39:42.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.556+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.556+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2740 con 0x561db5925e50 2024-04-25T11:39:42.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.560+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.560+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db5925e50 2024-04-25T11:39:42.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.564+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db5925e50 2024-04-25T11:39:42.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.564+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db59d30c0 2024-04-25T11:39:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.568+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v120'273 uv267 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.568+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db59d30c0 2024-04-25T11:39:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.574+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v120'274 uv274 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.574+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1730 con 0x561db59d30c0 2024-04-25T11:39:42.565 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.577+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v120'275 uv275 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.577+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1af0 con 0x561db59d30c0 2024-04-25T11:39:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.582+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'276 uv276 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.582+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1af0 con 0x561db59d30c0 2024-04-25T11:39:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.586+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'277 uv277 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.586+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1af0 con 0x561db59d30c0 2024-04-25T11:39:42.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.590+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v120'278 uv272 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.590+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59e1af0 con 0x561db59d30c0 2024-04-25T11:39:42.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.593+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v120'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.594+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1af0 con 0x561db59d30c0 2024-04-25T11:39:42.585 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:42.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.597+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v120'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.597+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1af0 con 0x561db59d30c0 2024-04-25T11:39:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.602+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.602+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e4900 con 0x561db59d30c0 2024-04-25T11:39:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.607+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59d30c0 2024-04-25T11:39:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.607+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e1fa0 con 0x561db59d6810 2024-04-25T11:39:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.611+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v120'120 uv119 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.611+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59d6810 2024-04-25T11:39:42.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.615+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'121 uv121 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.615+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59d6810 2024-04-25T11:39:42.606 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:42.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.618+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v120'122 uv122 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.619+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59d6810 2024-04-25T11:39:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.624+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'123 uv123 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.624+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59d6810 2024-04-25T11:39:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.627+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v120'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.627+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59d6810 2024-04-25T11:39:42.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.631+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v120'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.632+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59d6810 2024-04-25T11:39:42.623 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.636+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v120'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.636+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e3350 con 0x561db59d6810 2024-04-25T11:39:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.640+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.640+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e3350 con 0x561db59d6810 2024-04-25T11:39:42.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.644+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.644+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59d6810 2024-04-25T11:39:42.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.648+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0c8051070 con 0x561db59d6810 2024-04-25T11:39:42.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.649+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59de880 2024-04-25T11:39:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.652+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v120'140 uv139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.653+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59de880 2024-04-25T11:39:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.657+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v120'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.657+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59de880 2024-04-25T11:39:42.649 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:42.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.661+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v120'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.661+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59e2340 con 0x561db59de880 2024-04-25T11:39:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.665+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.666+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e40 con 0x561db59de880 2024-04-25T11:39:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.669+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.669+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.674+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v120'339 uv331 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.674+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.679+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v120'340 uv340 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.679+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.671 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.683+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v120'341 uv341 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.683+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.688+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v120'342 uv342 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.688+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.692+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'343 uv343 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.693+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.697+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v120'344 uv338 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.697+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.701+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v120'345 uv345 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.701+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.694 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.706+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v120'346 uv346 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.706+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.710+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'347 uv347 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.710+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.715+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'348 uv348 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.715+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.719+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'349 uv349 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.719+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x561db59d9e10 con 0x561db59de880 2024-04-25T11:39:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d0800640 1 -- 172.21.15.19:0/1567907031 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'350 uv350 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa0bc01a040 con 0x561db59de880 2024-04-25T11:39:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561db59cf030 msgr2=0x561db59d1450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561db59cf030 0x561db59d1450 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561db59dab30 msgr2=0x561db59dd9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561db59dab30 0x561db59dd9c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561db59de880 msgr2=0x561db59e0ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561db59de880 0x561db59e0ca0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561db59d30c0 msgr2=0x561db59d54e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561db59d30c0 0x561db59d54e0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561db5925e50 msgr2=0x561db59c4fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561db5925e50 0x561db59c4fd0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561db59d6810 msgr2=0x561db59d8c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561db59d6810 0x561db59d8c30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.723+0000 7fa0d1c00640 1 -- 172.21.15.19:0/1567907031 reap_dead start 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.724+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0b0072e90 msgr2=0x7fa0b00753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.724+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0b0072e90 0x7fa0b00753a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400b4a0 tx=0x7fa0c4007040 comp rx=0 tx=0).stop 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.724+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db592cdd0 msgr2=0x561db5922380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.724+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db592cdd0 0x561db5922380 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800a5a0 tx=0x7fa0c8002fb0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.724+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 shutdown_connections 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.724+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561db59d6810 0x561db59d8c30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.725+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa0b0072e90 0x7fa0b00753a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.725+0000 7fa0d805ef40 1 --2- 172.21.15.19:0/1567907031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561db592cdd0 0x561db5922380 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.725+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 >> 172.21.15.19:0/1567907031 conn(0x561db5775ac0 msgr2=0x561db592be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.725+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 shutdown_connections 2024-04-25T11:39:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.725+0000 7fa0d805ef40 1 -- 172.21.15.19:0/1567907031 wait complete. 2024-04-25T11:39:42.743 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2024-04-25T11:39:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.992+0000 7f4a91877f40 1 Processor -- start 2024-04-25T11:39:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.992+0000 7f4a91877f40 1 -- start start 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a91877f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a700030f0 0x557d6c0dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a91877f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1ab680 0x557d6c0dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a91877f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6c1a93b0 0x557d6c1a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a91877f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d6c084d20 con 0x557d6c1a93b0 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a91877f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557d6c088d60 con 0x7f4a700030f0 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a91877f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x557d6c1273f0 con 0x557d6c1ab680 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6c1a93b0 0x557d6c1a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6c1a93b0 0x557d6c1a97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60522/0 (socket says 172.21.15.19:60522) 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 -- 172.21.15.19:0/2052057348 learned_addr learned my addr 172.21.15.19:0/2052057348 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8aa00640 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a700030f0 0x557d6c0dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8a000640 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1ab680 0x557d6c0dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 -- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1ab680 msgr2=0x557d6c0dc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1ab680 0x557d6c0dc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 -- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a700030f0 msgr2=0x557d6c0dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a700030f0 0x557d6c0dd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8b400640 1 -- 172.21.15.19:0/2052057348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6c1b4a00 con 0x557d6c1a93b0 2024-04-25T11:39:42.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.994+0000 7f4a8a000640 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1ab680 0x557d6c0dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.995+0000 7f4a8b400640 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6c1a93b0 0x557d6c1a97c0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x557d6c127fb0 tx=0x7f4a84006850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cceb3a3f9ad0b78a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.995+0000 7f4a88c00640 1 -- 172.21.15.19:0/2052057348 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84003650 con 0x557d6c1a93b0 2024-04-25T11:39:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.995+0000 7f4a88c00640 1 -- 172.21.15.19:0/2052057348 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a840037f0 con 0x557d6c1a93b0 2024-04-25T11:39:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.995+0000 7f4a88c00640 1 -- 172.21.15.19:0/2052057348 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84003be0 con 0x557d6c1a93b0 2024-04-25T11:39:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 -- 172.21.15.19:0/2052057348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6c1a93b0 msgr2=0x557d6c1a97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6c1a93b0 0x557d6c1a97c0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x557d6c127fb0 tx=0x7f4a84006850 comp rx=0 tx=0).stop 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 -- 172.21.15.19:0/2052057348 shutdown_connections 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d6c1a93b0 0x557d6c1a97c0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1ab680 0x557d6c0dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 --2- 172.21.15.19:0/2052057348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a700030f0 0x557d6c0dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 -- 172.21.15.19:0/2052057348 >> 172.21.15.19:0/2052057348 conn(0x557d6c000ac0 msgr2=0x557d6c0dd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 -- 172.21.15.19:0/2052057348 shutdown_connections 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.996+0000 7f4a91877f40 1 -- 172.21.15.19:0/2052057348 wait complete. 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.997+0000 7f4a91877f40 1 Processor -- start 2024-04-25T11:39:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.997+0000 7f4a91877f40 1 -- start start 2024-04-25T11:39:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a91877f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1a93b0 0x557d6c1a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a91877f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d6c1ab680 0x557d6c24b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a91877f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a700030f0 0x557d6c1a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a91877f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d6c1108d0 con 0x7f4a700030f0 2024-04-25T11:39:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a91877f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557d6c24c120 con 0x557d6c1ab680 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a91877f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x557d6c1a4fd0 con 0x557d6c1a93b0 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a700030f0 0x557d6c1a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a700030f0 0x557d6c1a62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60536/0 (socket says 172.21.15.19:60536) 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 -- 172.21.15.19:0/615397692 learned_addr learned my addr 172.21.15.19:0/615397692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8a000640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d6c1ab680 0x557d6c24b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8aa00640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1a93b0 0x557d6c1a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 -- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1a93b0 msgr2=0x557d6c1a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557d6c1a93b0 0x557d6c1a3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 -- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d6c1ab680 msgr2=0x557d6c24b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d6c1ab680 0x557d6c24b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:42.999+0000 7f4a8b400640 1 -- 172.21.15.19:0/615397692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6c24bc70 con 0x7f4a700030f0 2024-04-25T11:39:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.000+0000 7f4a8b400640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a700030f0 0x557d6c1a62c0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f4a84000f80 tx=0x7f4a84001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.000+0000 7f4a83400640 1 -- 172.21.15.19:0/615397692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84025070 con 0x7f4a700030f0 2024-04-25T11:39:42.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.000+0000 7f4a83400640 1 -- 172.21.15.19:0/615397692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a840128c0 con 0x7f4a700030f0 2024-04-25T11:39:42.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.000+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557d6c1a5220 con 0x7f4a700030f0 2024-04-25T11:39:42.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.000+0000 7f4a83400640 1 -- 172.21.15.19:0/615397692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84012cb0 con 0x7f4a700030f0 2024-04-25T11:39:42.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.000+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6c1a57e0 con 0x7f4a700030f0 2024-04-25T11:39:42.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.003+0000 7f4a83400640 1 -- 172.21.15.19:0/615397692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a84028070 con 0x7f4a700030f0 2024-04-25T11:39:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.003+0000 7f4a83400640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a68072e90 0x7f4a680753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.004+0000 7f4a83400640 1 -- 172.21.15.19:0/615397692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7f4a84096200 con 0x7f4a700030f0 2024-04-25T11:39:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.004+0000 7f4a91877f40 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x557d6c1b6100 0x557d6c1b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.004+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x557d6c1a80b0 con 0x557d6c1b6100 2024-04-25T11:39:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.004+0000 7f4a8aa00640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a68072e90 0x7f4a680753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.005+0000 7f4a8a000640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x557d6c1b6100 0x557d6c1b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.005+0000 7f4a8aa00640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a68072e90 0x7f4a680753a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c000c00 tx=0x7f4a7c002590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.006+0000 7f4a8a000640 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x557d6c1b6100 0x557d6c1b6510 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a8a000640 1 -- 172.21.15.19:0/615397692 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v120'283 uv283 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4a7400a140 con 0x557d6c1b6100 2024-04-25T11:39:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x557d6c1b6100 msgr2=0x557d6c1b6510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x557d6c1b6100 0x557d6c1b6510 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a68072e90 msgr2=0x7f4a680753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a68072e90 0x7f4a680753a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c000c00 tx=0x7f4a7c002590 comp rx=0 tx=0).stop 2024-04-25T11:39:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a700030f0 msgr2=0x557d6c1a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 --2- 172.21.15.19:0/615397692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a700030f0 0x557d6c1a62c0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f4a84000f80 tx=0x7f4a84001dc0 comp rx=0 tx=0).stop 2024-04-25T11:39:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a8b400640 1 -- 172.21.15.19:0/615397692 reap_dead start 2024-04-25T11:39:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 shutdown_connections 2024-04-25T11:39:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 >> 172.21.15.19:0/615397692 conn(0x557d6c000ac0 msgr2=0x557d6c1ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 shutdown_connections 2024-04-25T11:39:42.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.010+0000 7f4a91877f40 1 -- 172.21.15.19:0/615397692 wait complete. 2024-04-25T11:39:43.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.298+0000 7f78c3200640 1 Processor -- start 2024-04-25T11:39:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.298+0000 7f78c3200640 1 -- start start 2024-04-25T11:39:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c40d0fb0 0x7f78c40cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78c4141d20 0x7f78c4142130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c3200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c4142670 con 0x7f78c40cf870 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f78c40cfc80 con 0x7f78c4141d20 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40cff10 con 0x7f78c40d0fb0 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78c4141d20 0x7f78c4142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78bbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c40d0fb0 0x7f78c40cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78c4141d20 0x7f78c4142130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:60604/0 (socket says 172.21.15.19:60604) 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c1e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f78c40cf870 0x7f78c415d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:32976/0 (socket says 172.21.15.19:32976) 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78c1e00640 1 -- 172.21.15.19:0/2501869013 learned_addr learned my addr 172.21.15.19:0/2501869013 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.301+0000 7f78bbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c40d0fb0 0x7f78c40cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:42168/0 (socket says 172.21.15.19:42168) 2024-04-25T11:39:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.302+0000 7f78c2800640 1 -- 172.21.15.19:0/2501869013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c40d0fb0 msgr2=0x7f78c40cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.302+0000 7f78c2800640 1 --2- 172.21.15.19:0/2501869013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c40d0fb0 0x7f78c40cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.302+0000 7f78c2800640 1 -- 172.21.15.19:0/2501869013 >> v1:172.21.15.169:6789/0 conn(0x7f78c40cf870 legacy=0x7f78c415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.302+0000 7f78c2800640 1 -- 172.21.15.19:0/2501869013 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c40d01f0 con 0x7f78c4141d20 2024-04-25T11:39:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.304+0000 7f78c2800640 1 --2- 172.21.15.19:0/2501869013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c4141d20 0x7f78c4142130 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009d90 tx=0x7f78bc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec38e8f567f928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.304+0000 7f78c1400640 1 -- 172.21.15.19:0/2501869013 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc007740 con 0x7f78c4141d20 2024-04-25T11:39:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c1400640 1 -- 172.21.15.19:0/2501869013 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78bc0078e0 con 0x7f78c4141d20 2024-04-25T11:39:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c1400640 1 -- 172.21.15.19:0/2501869013 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc007cb0 con 0x7f78c4141d20 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c3200640 1 -- 172.21.15.19:0/2501869013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c4141d20 msgr2=0x7f78c4142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c3200640 1 --2- 172.21.15.19:0/2501869013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c4141d20 0x7f78c4142130 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009d90 tx=0x7f78bc02a660 comp rx=0 tx=0).stop 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c3200640 1 -- 172.21.15.19:0/2501869013 shutdown_connections 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c3200640 1 --2- 172.21.15.19:0/2501869013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c4141d20 0x7f78c4142130 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c3200640 1 --2- 172.21.15.19:0/2501869013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c40d0fb0 0x7f78c40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.305+0000 7f78c3200640 1 -- 172.21.15.19:0/2501869013 >> 172.21.15.19:0/2501869013 conn(0x7f78c4072f40 msgr2=0x7f78c4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.306+0000 7f78c3200640 1 -- 172.21.15.19:0/2501869013 shutdown_connections 2024-04-25T11:39:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.306+0000 7f78c3200640 1 -- 172.21.15.19:0/2501869013 wait complete. 2024-04-25T11:39:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.308+0000 7f78c3200640 1 Processor -- start 2024-04-25T11:39:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.308+0000 7f78c3200640 1 -- start start 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c40cf870 0x7f78c40cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c40d0fb0 0x7f78c413ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c4141d20 0x7f78c41fa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40d13c0 con 0x7f78c40cf870 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40d0a90 con 0x7f78c40d0fb0 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40ce2a0 con 0x7f78c4141d20 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c40cf870 0x7f78c40cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c40cf870 0x7f78c40cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60550/0 (socket says 172.21.15.19:60550) 2024-04-25T11:39:43.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78c1e00640 1 -- 172.21.15.19:0/169995380 learned_addr learned my addr 172.21.15.19:0/169995380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.310+0000 7f78bbe00640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c40d0fb0 0x7f78c413ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78c2800640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c4141d20 0x7f78c41fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78c1e00640 1 -- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c4141d20 msgr2=0x7f78c41fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78c1e00640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c4141d20 0x7f78c41fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78c1e00640 1 -- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c40d0fb0 msgr2=0x7f78c413ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78c1e00640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c40d0fb0 0x7f78c413ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78c1e00640 1 -- 172.21.15.19:0/169995380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c4156790 con 0x7f78c40cf870 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78c1e00640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c40cf870 0x7f78c40cd270 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c440 tx=0x7f78b400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.311+0000 7f78ba000640 1 -- 172.21.15.19:0/169995380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4017070 con 0x7f78c40cf870 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.312+0000 7f78ba000640 1 -- 172.21.15.19:0/169995380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4005ce0 con 0x7f78c40cf870 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.312+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c4156a90 con 0x7f78c40cf870 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.312+0000 7f78ba000640 1 -- 172.21.15.19:0/169995380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40072f0 con 0x7f78c40cf870 2024-04-25T11:39:43.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.312+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c4154ef0 con 0x7f78c40cf870 2024-04-25T11:39:43.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.313+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c4073810 con 0x7f78c40cf870 2024-04-25T11:39:43.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.314+0000 7f78ba000640 1 -- 172.21.15.19:0/169995380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f78b401a070 con 0x7f78c40cf870 2024-04-25T11:39:43.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.315+0000 7f78ba000640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f78a00744b0 0x7f78a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:43.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.316+0000 7f78ba000640 1 -- 172.21.15.19:0/169995380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8117+0+0 (secure 0 0 0) 0x7f78b4094f20 con 0x7f78c40cf870 2024-04-25T11:39:43.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.316+0000 7f78bbe00640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f78a00744b0 0x7f78a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:43.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.317+0000 7f78bbe00640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f78a00744b0 0x7f78a00769c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f78c40ccae0 tx=0x7f78ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:43.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.320+0000 7f78ba000640 1 -- 172.21.15.19:0/169995380 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78c4073810 con 0x7f78c40cf870 2024-04-25T11:39:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.565+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f78c4076c60 con 0x7f78c40cf870 2024-04-25T11:39:43.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.947+0000 7f78ba000640 1 -- 172.21.15.19:0/169995380 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v270) ==== 162+0+0 (secure 0 0 0) 0x7f78c4142670 con 0x7f78c40cf870 2024-04-25T11:39:43.935 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f78a00744b0 msgr2=0x7f78a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f78a00744b0 0x7f78a00769c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f78c40ccae0 tx=0x7f78ac008040 comp rx=0 tx=0).stop 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c40cf870 msgr2=0x7f78c40cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c40cf870 0x7f78c40cd270 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c440 tx=0x7f78b400c900 comp rx=0 tx=0).stop 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 shutdown_connections 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f78a00744b0 0x7f78a00769c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78c40d0fb0 0x7f78c413ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78c4141d20 0x7f78c41fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 --2- 172.21.15.19:0/169995380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78c40cf870 0x7f78c40cd270 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 >> 172.21.15.19:0/169995380 conn(0x7f78c4072f40 msgr2=0x7f78c415f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:43.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 shutdown_connections 2024-04-25T11:39:43.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:43.951+0000 7f78c3200640 1 -- 172.21.15.19:0/169995380 wait complete. 2024-04-25T11:39:43.956 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:43.968+0000 7f31f6400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-04-25T11:39:43.957 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:43.969+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:43.957 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:43.969+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:39:43.971 INFO:tasks.ceph.mds.a.smithi019.stderr: -18> 2024-04-25T11:39:43.968+0000 7f31f6400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-04-25T11:39:43.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:44.106 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:44.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.247+0000 7f0acd000640 1 Processor -- start 2024-04-25T11:39:44.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.248+0000 7f0acd000640 1 -- start start 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0acd000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ac806c5c0 0x7f0ac80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0acd000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0ac80b4ea0 0x7f0ac815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0acd000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac80d1230 con 0x7f0ac8144370 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0acd000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac80d06b0 con 0x7f0ac806c5c0 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0acd000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac80d0990 con 0x7f0ac80b4ea0 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0ac7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ac806c5c0 0x7f0ac80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0ac7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0ac806c5c0 0x7f0ac80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37568/0 (socket says 172.21.15.19:37568) 2024-04-25T11:39:44.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.250+0000 7f0ac7400640 1 -- 172.21.15.19:0/536893816 learned_addr learned my addr 172.21.15.19:0/536893816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:44.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.251+0000 7f0ac7e00640 1 --2- 172.21.15.19:0/536893816 >> v2:172.21.15.169:3300/0 conn(0x7f0ac80b4ea0 0x7f0ac815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.251+0000 7f0ac7400640 1 -- 172.21.15.19:0/536893816 >> v2:172.21.15.169:3300/0 conn(0x7f0ac80b4ea0 msgr2=0x7f0ac815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:44.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.251+0000 7f0ac7400640 1 --2- 172.21.15.19:0/536893816 >> v2:172.21.15.169:3300/0 conn(0x7f0ac80b4ea0 0x7f0ac815e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.251+0000 7f0ac7400640 1 -- 172.21.15.19:0/536893816 >> v1:172.21.15.169:6789/0 conn(0x7f0ac8144370 legacy=0x7f0ac80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:44.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.251+0000 7f0ac7400640 1 -- 172.21.15.19:0/536893816 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac80d0c70 con 0x7f0ac806c5c0 2024-04-25T11:39:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.252+0000 7f0ac7400640 1 --2- 172.21.15.19:0/536893816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac806c5c0 0x7f0ac80cfa70 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80d26c0 tx=0x7f0ab802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94409bec6a797b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.252+0000 7f0ac6a00640 1 -- 172.21.15.19:0/536893816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8037070 con 0x7f0ac806c5c0 2024-04-25T11:39:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.252+0000 7f0ac6a00640 1 -- 172.21.15.19:0/536893816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab8032440 con 0x7f0ac806c5c0 2024-04-25T11:39:44.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.252+0000 7f0ac6a00640 1 -- 172.21.15.19:0/536893816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8032810 con 0x7f0ac806c5c0 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 -- 172.21.15.19:0/536893816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac806c5c0 msgr2=0x7f0ac80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 --2- 172.21.15.19:0/536893816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac806c5c0 0x7f0ac80cfa70 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80d26c0 tx=0x7f0ab802a230 comp rx=0 tx=0).stop 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 -- 172.21.15.19:0/536893816 shutdown_connections 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 --2- 172.21.15.19:0/536893816 >> v2:172.21.15.169:3300/0 conn(0x7f0ac80b4ea0 0x7f0ac815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 --2- 172.21.15.19:0/536893816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac806c5c0 0x7f0ac80cfa70 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 -- 172.21.15.19:0/536893816 >> 172.21.15.19:0/536893816 conn(0x7f0ac80d1720 msgr2=0x7f0ac80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 -- 172.21.15.19:0/536893816 shutdown_connections 2024-04-25T11:39:44.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.253+0000 7f0acd000640 1 -- 172.21.15.19:0/536893816 wait complete. 2024-04-25T11:39:44.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.256+0000 7f0acd000640 1 Processor -- start 2024-04-25T11:39:44.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.256+0000 7f0acd000640 1 -- start start 2024-04-25T11:39:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.257+0000 7f0acd000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac806c5c0 0x7f0ac8139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.257+0000 7f0acd000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac8144370 0x7f0ac80ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.257+0000 7f0acd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac8145a30 0x7f0ac8145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.257+0000 7f0acd000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac80d0450 con 0x7f0ac8145a30 2024-04-25T11:39:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.257+0000 7f0acd000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac80d02d0 con 0x7f0ac806c5c0 2024-04-25T11:39:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.257+0000 7f0acd000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac8147760 con 0x7f0ac8144370 2024-04-25T11:39:44.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac8145a30 0x7f0ac8145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac8145a30 0x7f0ac8145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37580/0 (socket says 172.21.15.19:37580) 2024-04-25T11:39:44.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 -- 172.21.15.19:0/97011017 learned_addr learned my addr 172.21.15.19:0/97011017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7400640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac806c5c0 0x7f0ac8139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0abea00640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac8144370 0x7f0ac80ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 -- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac8144370 msgr2=0x7f0ac80ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac8144370 0x7f0ac80ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 -- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac806c5c0 msgr2=0x7f0ac8139560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac806c5c0 0x7f0ac8139560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.258+0000 7f0ac7e00640 1 -- 172.21.15.19:0/97011017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac8146380 con 0x7f0ac8145a30 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.259+0000 7f0ac7e00640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac8145a30 0x7f0ac8145e40 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000cc60 tx=0x7f0ac000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:44.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.259+0000 7f0abfe00640 1 -- 172.21.15.19:0/97011017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0003990 con 0x7f0ac8145a30 2024-04-25T11:39:44.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.259+0000 7f0abfe00640 1 -- 172.21.15.19:0/97011017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac0003b30 con 0x7f0ac8145a30 2024-04-25T11:39:44.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.259+0000 7f0abfe00640 1 -- 172.21.15.19:0/97011017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0004020 con 0x7f0ac8145a30 2024-04-25T11:39:44.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.259+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac80aeb80 con 0x7f0ac8145a30 2024-04-25T11:39:44.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.259+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac80af370 con 0x7f0ac8145a30 2024-04-25T11:39:44.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.260+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac80aee00 con 0x7f0ac8145a30 2024-04-25T11:39:44.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.262+0000 7f0abfe00640 1 -- 172.21.15.19:0/97011017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0ac00041c0 con 0x7f0ac8145a30 2024-04-25T11:39:44.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.263+0000 7f0abfe00640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0aa40744b0 0x7f0aa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.263+0000 7f0abfe00640 1 -- 172.21.15.19:0/97011017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7f0ac0096120 con 0x7f0ac8145a30 2024-04-25T11:39:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.263+0000 7f0ac7400640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0aa40744b0 0x7f0aa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.264+0000 7f0ac7400640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0aa40744b0 0x7f0aa40769c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8034000 tx=0x7f0ab802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.268+0000 7f0abfe00640 1 -- 172.21.15.19:0/97011017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ac8146380 con 0x7f0ac8145a30 2024-04-25T11:39:44.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.517+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ac8050740 con 0x7f0ac8145a30 2024-04-25T11:39:44.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.518+0000 7f0abfe00640 1 -- 172.21.15.19:0/97011017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) ==== 95+0+4352 (secure 0 0 0) 0x7f0ac0014030 con 0x7f0ac8145a30 2024-04-25T11:39:44.506 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:44.506 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":272,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":272,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:43.970829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:44.506 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 272 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0aa40744b0 msgr2=0x7f0aa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0aa40744b0 0x7f0aa40769c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8034000 tx=0x7f0ab802e000 comp rx=0 tx=0).stop 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac8145a30 msgr2=0x7f0ac8145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac8145a30 0x7f0ac8145e40 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000cc60 tx=0x7f0ac000a5c0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 shutdown_connections 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0aa40744b0 0x7f0aa40769c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0ac8145a30 0x7f0ac8145e40 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ac8144370 0x7f0ac80ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 --2- 172.21.15.19:0/97011017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ac806c5c0 0x7f0ac8139560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 >> 172.21.15.19:0/97011017 conn(0x7f0ac80d1720 msgr2=0x7f0ac806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:44.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 shutdown_connections 2024-04-25T11:39:44.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.521+0000 7f0acd000640 1 -- 172.21.15.19:0/97011017 wait complete. 2024-04-25T11:39:44.538 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:44.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:44.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.837+0000 7ffb22c00640 1 Processor -- start 2024-04-25T11:39:44.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.838+0000 7ffb22c00640 1 -- start start 2024-04-25T11:39:44.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.840+0000 7ffb22c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffb1c15c800 0x7ffb1c06f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.840+0000 7ffb22c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb1c071230 con 0x7ffb1c070e20 2024-04-25T11:39:44.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.840+0000 7ffb22c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb1c06f8f0 con 0x7ffb1c0cfa00 2024-04-25T11:39:44.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.840+0000 7ffb22c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ffb1c06f6c0 con 0x7ffb1c15c800 2024-04-25T11:39:44.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.840+0000 7ffb20e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffb1c15c800 0x7ffb1c06f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.841+0000 7ffb21800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffb1c070e20 0x7ffb1c0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52230/0 (socket says 172.21.15.19:52230) 2024-04-25T11:39:44.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.841+0000 7ffb21800640 1 -- 172.21.15.19:0/2954434978 learned_addr learned my addr 172.21.15.19:0/2954434978 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:44.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.841+0000 7ffb20e00640 1 -- 172.21.15.19:0/2954434978 >> v1:172.21.15.169:6789/0 conn(0x7ffb1c0cfa00 legacy=0x7ffb1c167180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:44.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.841+0000 7ffb20e00640 1 -- 172.21.15.19:0/2954434978 >> v1:172.21.15.19:6789/0 conn(0x7ffb1c070e20 legacy=0x7ffb1c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.841+0000 7ffb20e00640 1 -- 172.21.15.19:0/2954434978 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c06fbd0 con 0x7ffb1c15c800 2024-04-25T11:39:44.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.841+0000 7ffb1be00640 1 -- 172.21.15.19:0/2954434978 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807029038 0 0) 0x7ffb1c071230 con 0x7ffb1c070e20 2024-04-25T11:39:44.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.842+0000 7ffb20e00640 1 --2- 172.21.15.19:0/2954434978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c15c800 0x7ffb1c06f180 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7ffb080097a0 tx=0x7ffb0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623f46d6c78bea86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:44.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.843+0000 7ffb1be00640 1 -- 172.21.15.19:0/2954434978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08037070 con 0x7ffb1c15c800 2024-04-25T11:39:44.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.843+0000 7ffb1be00640 1 -- 172.21.15.19:0/2954434978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb08032440 con 0x7ffb1c15c800 2024-04-25T11:39:44.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.843+0000 7ffb1be00640 1 -- 172.21.15.19:0/2954434978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08032810 con 0x7ffb1c15c800 2024-04-25T11:39:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.843+0000 7ffb22c00640 1 -- 172.21.15.19:0/2954434978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c15c800 msgr2=0x7ffb1c06f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.843+0000 7ffb22c00640 1 --2- 172.21.15.19:0/2954434978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c15c800 0x7ffb1c06f180 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7ffb080097a0 tx=0x7ffb0802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.844+0000 7ffb22c00640 1 -- 172.21.15.19:0/2954434978 shutdown_connections 2024-04-25T11:39:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.844+0000 7ffb22c00640 1 --2- 172.21.15.19:0/2954434978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c15c800 0x7ffb1c06f180 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.844+0000 7ffb22c00640 1 -- 172.21.15.19:0/2954434978 >> 172.21.15.19:0/2954434978 conn(0x7ffb1c0d1720 msgr2=0x7ffb1c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.844+0000 7ffb22c00640 1 -- 172.21.15.19:0/2954434978 shutdown_connections 2024-04-25T11:39:44.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.844+0000 7ffb22c00640 1 -- 172.21.15.19:0/2954434978 wait complete. 2024-04-25T11:39:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.846+0000 7ffb22c00640 1 Processor -- start 2024-04-25T11:39:44.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.846+0000 7ffb22c00640 1 -- start start 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb22c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c070e20 0x7ffb1c14f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb22c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb1c0cfa00 0x7ffb1c15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb22c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb1c15c800 0x7ffb1c0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb22c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c15cc10 con 0x7ffb1c070e20 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb22c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c070190 con 0x7ffb1c15c800 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb22c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffb1c06d5d0 con 0x7ffb1c0cfa00 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb21800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c070e20 0x7ffb1c14f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb21800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c070e20 0x7ffb1c14f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37590/0 (socket says 172.21.15.19:37590) 2024-04-25T11:39:44.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.848+0000 7ffb21800640 1 -- 172.21.15.19:0/3030364275 learned_addr learned my addr 172.21.15.19:0/3030364275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb20e00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb1c0cfa00 0x7ffb1c15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb21800640 1 -- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb1c0cfa00 msgr2=0x7ffb1c15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb22200640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb1c15c800 0x7ffb1c0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb21800640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb1c0cfa00 0x7ffb1c15dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb21800640 1 -- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb1c15c800 msgr2=0x7ffb1c0b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb21800640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb1c15c800 0x7ffb1c0b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb21800640 1 -- 172.21.15.19:0/3030364275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c1f9c90 con 0x7ffb1c070e20 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb20e00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb1c0cfa00 0x7ffb1c15dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb22200640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb1c15c800 0x7ffb1c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb21800640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c070e20 0x7ffb1c14f5c0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c460 tx=0x7ffb1000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb19600640 1 -- 172.21.15.19:0/3030364275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10018070 con 0x7ffb1c070e20 2024-04-25T11:39:44.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb19600640 1 -- 172.21.15.19:0/3030364275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb10007500 con 0x7ffb1c070e20 2024-04-25T11:39:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c1f9f70 con 0x7ffb1c070e20 2024-04-25T11:39:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb19600640 1 -- 172.21.15.19:0/3030364275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10004a20 con 0x7ffb1c070e20 2024-04-25T11:39:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.849+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c14da30 con 0x7ffb1c070e20 2024-04-25T11:39:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.851+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c050740 con 0x7ffb1c070e20 2024-04-25T11:39:44.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.852+0000 7ffb19600640 1 -- 172.21.15.19:0/3030364275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffb1001b070 con 0x7ffb1c070e20 2024-04-25T11:39:44.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.853+0000 7ffb19600640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffafc0744b0 0x7ffafc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:44.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.853+0000 7ffb19600640 1 -- 172.21.15.19:0/3030364275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7ffb10094f20 con 0x7ffb1c070e20 2024-04-25T11:39:44.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.854+0000 7ffb20e00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffafc0744b0 0x7ffafc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:44.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.855+0000 7ffb20e00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffafc0744b0 0x7ffafc0769c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ffb08034000 tx=0x7ffb0802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:44.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:44.857+0000 7ffb19600640 1 -- 172.21.15.19:0/3030364275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb1005ecd0 con 0x7ffb1c070e20 2024-04-25T11:39:45.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.104+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb1c0cce90 con 0x7ffb1c070e20 2024-04-25T11:39:45.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.106+0000 7ffb19600640 1 -- 172.21.15.19:0/3030364275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 273 v273) ==== 95+0+5176 (secure 0 0 0) 0x7ffb1005e5d0 con 0x7ffb1c070e20 2024-04-25T11:39:45.094 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:45.094 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":273,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/571302363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":571302363},{"type":"v1","addr":"172.21.15.19:6835","nonce":571302363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":272,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:43.970829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:45.094 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 273 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffafc0744b0 msgr2=0x7ffafc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffafc0744b0 0x7ffafc0769c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ffb08034000 tx=0x7ffb0802e000 comp rx=0 tx=0).stop 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c070e20 msgr2=0x7ffb1c14f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c070e20 0x7ffb1c14f5c0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c460 tx=0x7ffb1000c920 comp rx=0 tx=0).stop 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 shutdown_connections 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffafc0744b0 0x7ffafc0769c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb1c15c800 0x7ffb1c0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb1c0cfa00 0x7ffb1c15dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 --2- 172.21.15.19:0/3030364275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb1c070e20 0x7ffb1c14f5c0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 >> 172.21.15.19:0/3030364275 conn(0x7ffb1c0d1720 msgr2=0x7ffb1c0d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 shutdown_connections 2024-04-25T11:39:45.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.109+0000 7ffb22c00640 1 -- 172.21.15.19:0/3030364275 wait complete. 2024-04-25T11:39:45.126 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:45.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.410+0000 7fe667200640 1 Processor -- start 2024-04-25T11:39:45.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.410+0000 7fe667200640 1 -- start start 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe667200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe66806e680 0x7fe6680b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe667200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6680b4e80 0x7fe6680cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe667200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe66808cd60 0x7fe66808d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe667200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe6680d1d70 con 0x7fe66808cd60 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe667200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe6680b5290 con 0x7fe66806e680 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe667200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe66808d6b0 con 0x7fe6680b4e80 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe666800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe66808cd60 0x7fe66808d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe666800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe66808cd60 0x7fe66808d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37598/0 (socket says 172.21.15.19:37598) 2024-04-25T11:39:45.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe666800640 1 -- 172.21.15.19:0/2843462056 learned_addr learned my addr 172.21.15.19:0/2843462056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.412+0000 7fe665e00640 1 --2- 172.21.15.19:0/2843462056 >> v2:172.21.15.169:3300/0 conn(0x7fe66806e680 0x7fe6680b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.413+0000 7fe665400640 1 --2- 172.21.15.19:0/2843462056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6680b4e80 0x7fe6680cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.413+0000 7fe666800640 1 -- 172.21.15.19:0/2843462056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6680b4e80 msgr2=0x7fe6680cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.413+0000 7fe666800640 1 --2- 172.21.15.19:0/2843462056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6680b4e80 0x7fe6680cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.413+0000 7fe666800640 1 -- 172.21.15.19:0/2843462056 >> v2:172.21.15.169:3300/0 conn(0x7fe66806e680 msgr2=0x7fe6680b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:45.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.413+0000 7fe666800640 1 --2- 172.21.15.19:0/2843462056 >> v2:172.21.15.169:3300/0 conn(0x7fe66806e680 0x7fe6680b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.413+0000 7fe666800640 1 -- 172.21.15.19:0/2843462056 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6680d1fa0 con 0x7fe66808cd60 2024-04-25T11:39:45.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.414+0000 7fe666800640 1 --2- 172.21.15.19:0/2843462056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe66808cd60 0x7fe66808d170 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fe668074310 tx=0x7fe66002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396a6f420fb63dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:45.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.414+0000 7fe664a00640 1 -- 172.21.15.19:0/2843462056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660036070 con 0x7fe66808cd60 2024-04-25T11:39:45.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.414+0000 7fe664a00640 1 -- 172.21.15.19:0/2843462056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe660031600 con 0x7fe66808cd60 2024-04-25T11:39:45.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.414+0000 7fe664a00640 1 -- 172.21.15.19:0/2843462056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6600319d0 con 0x7fe66808cd60 2024-04-25T11:39:45.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 -- 172.21.15.19:0/2843462056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe66808cd60 msgr2=0x7fe66808d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 --2- 172.21.15.19:0/2843462056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe66808cd60 0x7fe66808d170 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fe668074310 tx=0x7fe66002a610 comp rx=0 tx=0).stop 2024-04-25T11:39:45.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 -- 172.21.15.19:0/2843462056 shutdown_connections 2024-04-25T11:39:45.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 --2- 172.21.15.19:0/2843462056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe66808cd60 0x7fe66808d170 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 --2- 172.21.15.19:0/2843462056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe6680b4e80 0x7fe6680cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 --2- 172.21.15.19:0/2843462056 >> v2:172.21.15.169:3300/0 conn(0x7fe66806e680 0x7fe6680b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 -- 172.21.15.19:0/2843462056 >> 172.21.15.19:0/2843462056 conn(0x7fe6680b5460 msgr2=0x7fe6680b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:45.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 -- 172.21.15.19:0/2843462056 shutdown_connections 2024-04-25T11:39:45.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.415+0000 7fe667200640 1 -- 172.21.15.19:0/2843462056 wait complete. 2024-04-25T11:39:45.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.418+0000 7fe667200640 1 Processor -- start 2024-04-25T11:39:45.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.418+0000 7fe667200640 1 -- start start 2024-04-25T11:39:45.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.419+0000 7fe667200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe66806e680 0x7fe6680c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.419+0000 7fe667200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe66808cd60 0x7fe668161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.419+0000 7fe667200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680b4e80 0x7fe668160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.419+0000 7fe667200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6680d2840 con 0x7fe6680b4e80 2024-04-25T11:39:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.419+0000 7fe667200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe6680d2560 con 0x7fe66808cd60 2024-04-25T11:39:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.419+0000 7fe667200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe66806ec40 con 0x7fe66806e680 2024-04-25T11:39:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680b4e80 0x7fe668160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680b4e80 0x7fe668160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37610/0 (socket says 172.21.15.19:37610) 2024-04-25T11:39:45.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 -- 172.21.15.19:0/3091516785 learned_addr learned my addr 172.21.15.19:0/3091516785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe665e00640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe66806e680 0x7fe6680c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe665400640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe66808cd60 0x7fe668161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 -- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe66806e680 msgr2=0x7fe6680c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe66806e680 0x7fe6680c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 -- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe66808cd60 msgr2=0x7fe668161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe66808cd60 0x7fe668161ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe666800640 1 -- 172.21.15.19:0/3091516785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6680c8d50 con 0x7fe6680b4e80 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.420+0000 7fe665400640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe66808cd60 0x7fe668161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.421+0000 7fe666800640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680b4e80 0x7fe668160970 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fe660009f40 tx=0x7fe660004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.421+0000 7fe65e000640 1 -- 172.21.15.19:0/3091516785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66003f020 con 0x7fe6680b4e80 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.421+0000 7fe65e000640 1 -- 172.21.15.19:0/3091516785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe660037040 con 0x7fe6680b4e80 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.421+0000 7fe65e000640 1 -- 172.21.15.19:0/3091516785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660036070 con 0x7fe6680b4e80 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.421+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6680c9030 con 0x7fe6680b4e80 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.421+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6680c95f0 con 0x7fe6680b4e80 2024-04-25T11:39:45.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.422+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe668050740 con 0x7fe6680b4e80 2024-04-25T11:39:45.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.424+0000 7fe65e000640 1 -- 172.21.15.19:0/3091516785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe660042020 con 0x7fe6680b4e80 2024-04-25T11:39:45.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.425+0000 7fe65e000640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6480744b0 0x7fe6480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.425+0000 7fe65e000640 1 -- 172.21.15.19:0/3091516785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7fe6600b6570 con 0x7fe6680b4e80 2024-04-25T11:39:45.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.426+0000 7fe665e00640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6480744b0 0x7fe6480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.427+0000 7fe665e00640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6480744b0 0x7fe6480769c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe668074b10 tx=0x7fe658008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:45.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.430+0000 7fe65e000640 1 -- 172.21.15.19:0/3091516785 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe660049020 con 0x7fe6680b4e80 2024-04-25T11:39:45.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.677+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6680c8ac0 con 0x7fe6680b4e80 2024-04-25T11:39:45.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.678+0000 7fe65e000640 1 -- 172.21.15.19:0/3091516785 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 273 v273) ==== 95+0+5176 (secure 0 0 0) 0x7fe660080290 con 0x7fe6680b4e80 2024-04-25T11:39:45.667 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:45.667 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":273,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10149,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/571302363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":571302363},{"type":"v1","addr":"172.21.15.19:6835","nonce":571302363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":273}],"filesystems":[{"mdsmap":{"epoch":272,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:43.970829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:45.667 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 273 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6480744b0 msgr2=0x7fe6480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6480744b0 0x7fe6480769c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe668074b10 tx=0x7fe658008040 comp rx=0 tx=0).stop 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680b4e80 msgr2=0x7fe668160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680b4e80 0x7fe668160970 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fe660009f40 tx=0x7fe660004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 shutdown_connections 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe6480744b0 0x7fe6480769c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe66808cd60 0x7fe668161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe6680b4e80 0x7fe668160970 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 --2- 172.21.15.19:0/3091516785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe66806e680 0x7fe6680c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.681+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 >> 172.21.15.19:0/3091516785 conn(0x7fe6680b5460 msgr2=0x7fe6680d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:45.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.682+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 shutdown_connections 2024-04-25T11:39:45.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:45.682+0000 7fe667200640 1 -- 172.21.15.19:0/3091516785 wait complete. 2024-04-25T11:39:45.698 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2024-04-25T11:39:45.698 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2024-04-25T11:39:45.698 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:45.698 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:45.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:39:45.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:45.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:45.756 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:39:45.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:39:45.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:46.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.114+0000 7f6254a00640 1 Processor -- start 2024-04-25T11:39:46.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.114+0000 7f6254a00640 1 -- start start 2024-04-25T11:39:46.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.116+0000 7f6254a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f62500b7c50 0x7f62500b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:46.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.116+0000 7f6254a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f62500d44a0 0x7f62500d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:46.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.116+0000 7f6254a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6250161650 con 0x7f6250164860 2024-04-25T11:39:46.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f6254a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f62501643d0 con 0x7f62500b7c50 2024-04-25T11:39:46.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f6254a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f62500b5270 con 0x7f62500d44a0 2024-04-25T11:39:46.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f62500b7c50 0x7f62500b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:46.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f62500b7c50 0x7f62500b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37616/0 (socket says 172.21.15.19:37616) 2024-04-25T11:39:46.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624de00640 1 -- 172.21.15.19:0/181188602 learned_addr learned my addr 172.21.15.19:0/181188602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624f200640 1 --2- 172.21.15.19:0/181188602 >> v2:172.21.15.169:3300/0 conn(0x7f62500d44a0 0x7f62500d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624de00640 1 -- 172.21.15.19:0/181188602 >> v2:172.21.15.169:3300/0 conn(0x7f62500d44a0 msgr2=0x7f62500d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624de00640 1 --2- 172.21.15.19:0/181188602 >> v2:172.21.15.169:3300/0 conn(0x7f62500d44a0 0x7f62500d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624de00640 1 -- 172.21.15.19:0/181188602 >> v1:172.21.15.19:6789/0 conn(0x7f6250164860 legacy=0x7f62500b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624de00640 1 -- 172.21.15.19:0/181188602 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6250161370 con 0x7f62500b7c50 2024-04-25T11:39:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.117+0000 7f624f200640 1 --2- 172.21.15.19:0/181188602 >> v2:172.21.15.169:3300/0 conn(0x7f62500d44a0 0x7f62500d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:46.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.118+0000 7f624de00640 1 --2- 172.21.15.19:0/181188602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62500b7c50 0x7f62500b4d30 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f6250164340 tx=0x7f623c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bae4e4912735054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.119+0000 7f624d400640 1 -- 172.21.15.19:0/181188602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c037070 con 0x7f62500b7c50 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.119+0000 7f624d400640 1 -- 172.21.15.19:0/181188602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623c032440 con 0x7f62500b7c50 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.119+0000 7f624d400640 1 -- 172.21.15.19:0/181188602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c032810 con 0x7f62500b7c50 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.119+0000 7f6254a00640 1 -- 172.21.15.19:0/181188602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62500b7c50 msgr2=0x7f62500b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.119+0000 7f6254a00640 1 --2- 172.21.15.19:0/181188602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62500b7c50 0x7f62500b4d30 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f6250164340 tx=0x7f623c02a270 comp rx=0 tx=0).stop 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.120+0000 7f6254a00640 1 -- 172.21.15.19:0/181188602 shutdown_connections 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.120+0000 7f6254a00640 1 --2- 172.21.15.19:0/181188602 >> v2:172.21.15.169:3300/0 conn(0x7f62500d44a0 0x7f62500d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.120+0000 7f6254a00640 1 --2- 172.21.15.19:0/181188602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62500b7c50 0x7f62500b4d30 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.120+0000 7f6254a00640 1 -- 172.21.15.19:0/181188602 >> 172.21.15.19:0/181188602 conn(0x7f62500d2720 msgr2=0x7f62500d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.120+0000 7f6254a00640 1 -- 172.21.15.19:0/181188602 shutdown_connections 2024-04-25T11:39:46.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.120+0000 7f6254a00640 1 -- 172.21.15.19:0/181188602 wait complete. 2024-04-25T11:39:46.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.122+0000 7f6254a00640 1 Processor -- start 2024-04-25T11:39:46.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.122+0000 7f6254a00640 1 -- start start 2024-04-25T11:39:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f6254a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62500d44a0 0x7f625015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f6254a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6250164860 0x7f625014f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f6254a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625015a8e0 0x7f625015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f6254a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62500d4df0 con 0x7f6250164860 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f6254a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6250164060 con 0x7f62500d44a0 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f6254a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f62500d4f70 con 0x7f625015a8e0 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f624de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6250164860 0x7f625014f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f624de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6250164860 0x7f625014f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37628/0 (socket says 172.21.15.19:37628) 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f624de00640 1 -- 172.21.15.19:0/3036119119 learned_addr learned my addr 172.21.15.19:0/3036119119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.124+0000 7f624e800640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62500d44a0 0x7f625015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624f200640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625015a8e0 0x7f625015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624de00640 1 -- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625015a8e0 msgr2=0x7f625015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624de00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625015a8e0 0x7f625015acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624de00640 1 -- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62500d44a0 msgr2=0x7f625015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624de00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62500d44a0 0x7f625015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624de00640 1 -- 172.21.15.19:0/3036119119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625015b650 con 0x7f6250164860 2024-04-25T11:39:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624f200640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625015a8e0 0x7f625015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f624de00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6250164860 0x7f625014f060 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f623c02a2b0 tx=0x7f623c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f6246a00640 1 -- 172.21.15.19:0/3036119119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c037040 con 0x7f6250164860 2024-04-25T11:39:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f6246a00640 1 -- 172.21.15.19:0/3036119119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623c042070 con 0x7f6250164860 2024-04-25T11:39:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f6246a00640 1 -- 172.21.15.19:0/3036119119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c030bb0 con 0x7f6250164860 2024-04-25T11:39:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.125+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625015b930 con 0x7f6250164860 2024-04-25T11:39:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.126+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625015bef0 con 0x7f6250164860 2024-04-25T11:39:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.127+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f625015b3c0 con 0x7f6250164860 2024-04-25T11:39:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.128+0000 7f6246a00640 1 -- 172.21.15.19:0/3036119119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f623c047460 con 0x7f6250164860 2024-04-25T11:39:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.129+0000 7f6246a00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.129+0000 7f6246a00640 1 -- 172.21.15.19:0/3036119119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7f623c0b6910 con 0x7f6250164860 2024-04-25T11:39:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.130+0000 7f624e800640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.131+0000 7f624e800640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6240007fd0 tx=0x7f6240009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.134+0000 7f6246a00640 1 -- 172.21.15.19:0/3036119119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f623c0806c0 con 0x7f6250164860 2024-04-25T11:39:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.381+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6250050740 con 0x7f6250164860 2024-04-25T11:39:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.992+0000 7f6246a00640 1 -- 172.21.15.19:0/3036119119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v274) ==== 161+0+0 (secure 0 0 0) 0x7f623c07ffc0 con 0x7f6250164860 2024-04-25T11:39:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:46.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 msgr2=0x7f62300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:46.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6240007fd0 tx=0x7f6240009040 comp rx=0 tx=0).stop 2024-04-25T11:39:46.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6250164860 msgr2=0x7f625014f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:46.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6250164860 0x7f625014f060 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f623c02a2b0 tx=0x7f623c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 shutdown_connections 2024-04-25T11:39:46.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f625015a8e0 0x7f625015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6250164860 0x7f625014f060 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 --2- 172.21.15.19:0/3036119119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62500d44a0 0x7f625015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 >> 172.21.15.19:0/3036119119 conn(0x7f62500d2720 msgr2=0x7f62500684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.998+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 shutdown_connections 2024-04-25T11:39:46.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:46.999+0000 7f6254a00640 1 -- 172.21.15.19:0/3036119119 wait complete. 2024-04-25T11:39:47.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.320+0000 7f443f200640 1 Processor -- start 2024-04-25T11:39:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.320+0000 7f443f200640 1 -- start start 2024-04-25T11:39:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.322+0000 7f443f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4438153820 0x7f44380b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.322+0000 7f443f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44380cee00 0x7f44380d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.323+0000 7f443f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44380b5380 con 0x7f443815d960 2024-04-25T11:39:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.323+0000 7f443f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f443806dce0 con 0x7f4438153820 2024-04-25T11:39:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.323+0000 7f443f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f44380b5660 con 0x7f44380cee00 2024-04-25T11:39:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.323+0000 7f4437e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f443815d960 0x7f443806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52250/0 (socket says 172.21.15.19:52250) 2024-04-25T11:39:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.323+0000 7f4437e00640 1 -- 172.21.15.19:0/1867015723 learned_addr learned my addr 172.21.15.19:0/1867015723 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.323+0000 7f4437400640 1 --2- 172.21.15.19:0/1867015723 >> v2:172.21.15.19:3300/0 conn(0x7f4438153820 0x7f44380b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.323+0000 7f443d200640 1 --2- 172.21.15.19:0/1867015723 >> v2:172.21.15.169:3300/0 conn(0x7f44380cee00 0x7f44380d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.324+0000 7f4436a00640 1 -- 172.21.15.19:0/1867015723 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756974974 0 0) 0x7f44380b5380 con 0x7f443815d960 2024-04-25T11:39:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.324+0000 7f4436a00640 1 -- 172.21.15.19:0/1867015723 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f441c003520 con 0x7f443815d960 2024-04-25T11:39:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.324+0000 7f4437400640 1 -- 172.21.15.19:0/1867015723 >> v2:172.21.15.169:3300/0 conn(0x7f44380cee00 msgr2=0x7f44380d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.324+0000 7f4437400640 1 --2- 172.21.15.19:0/1867015723 >> v2:172.21.15.169:3300/0 conn(0x7f44380cee00 0x7f44380d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.324+0000 7f4437400640 1 -- 172.21.15.19:0/1867015723 >> v1:172.21.15.19:6789/0 conn(0x7f443815d960 legacy=0x7f443806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.324+0000 7f4437400640 1 -- 172.21.15.19:0/1867015723 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44380b5940 con 0x7f4438153820 2024-04-25T11:39:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.325+0000 7f4437400640 1 --2- 172.21.15.19:0/1867015723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f44380b6140 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f44380b6680 tx=0x7f442402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee263ba0e79465f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.325+0000 7f4436a00640 1 -- 172.21.15.19:0/1867015723 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424037070 con 0x7f4438153820 2024-04-25T11:39:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.325+0000 7f4436a00640 1 -- 172.21.15.19:0/1867015723 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4424032440 con 0x7f4438153820 2024-04-25T11:39:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.325+0000 7f4436a00640 1 -- 172.21.15.19:0/1867015723 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442402ae60 con 0x7f4438153820 2024-04-25T11:39:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 -- 172.21.15.19:0/1867015723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 msgr2=0x7f44380b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 --2- 172.21.15.19:0/1867015723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f44380b6140 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f44380b6680 tx=0x7f442402a110 comp rx=0 tx=0).stop 2024-04-25T11:39:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 -- 172.21.15.19:0/1867015723 shutdown_connections 2024-04-25T11:39:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 --2- 172.21.15.19:0/1867015723 >> v2:172.21.15.169:3300/0 conn(0x7f44380cee00 0x7f44380d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 --2- 172.21.15.19:0/1867015723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f44380b6140 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 -- 172.21.15.19:0/1867015723 >> 172.21.15.19:0/1867015723 conn(0x7f4438073050 msgr2=0x7f4438073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 -- 172.21.15.19:0/1867015723 shutdown_connections 2024-04-25T11:39:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.326+0000 7f443f200640 1 -- 172.21.15.19:0/1867015723 wait complete. 2024-04-25T11:39:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.329+0000 7f443f200640 1 Processor -- start 2024-04-25T11:39:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.329+0000 7f443f200640 1 -- start start 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.330+0000 7f443f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44380cee00 0x7f4438149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.330+0000 7f443f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f4438144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f443f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f443815d960 0x7f44380b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f443f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44380cf390 con 0x7f4438153820 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f443f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f44380cf210 con 0x7f44380cee00 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f443f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f44380cf510 con 0x7f443815d960 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f4438144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f4438144f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37650/0 (socket says 172.21.15.19:37650) 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 -- 172.21.15.19:0/1355858157 learned_addr learned my addr 172.21.15.19:0/1355858157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437e00640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44380cee00 0x7f4438149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f443d200640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f443815d960 0x7f44380b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 -- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f443815d960 msgr2=0x7f44380b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f443815d960 0x7f44380b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 -- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44380cee00 msgr2=0x7f4438149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44380cee00 0x7f4438149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.331+0000 7f4437400640 1 -- 172.21.15.19:0/1355858157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44381461d0 con 0x7f4438153820 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.332+0000 7f4437400640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f4438144f50 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f4424009890 tx=0x7f44240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.332+0000 7f442fe00640 1 -- 172.21.15.19:0/1355858157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424037040 con 0x7f4438153820 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.332+0000 7f442fe00640 1 -- 172.21.15.19:0/1355858157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4424042070 con 0x7f4438153820 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.332+0000 7f442fe00640 1 -- 172.21.15.19:0/1355858157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424030960 con 0x7f4438153820 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.332+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44381464b0 con 0x7f4438153820 2024-04-25T11:39:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.333+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4438146a70 con 0x7f4438153820 2024-04-25T11:39:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.334+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438145f40 con 0x7f4438153820 2024-04-25T11:39:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.335+0000 7f442fe00640 1 -- 172.21.15.19:0/1355858157 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4424030b00 con 0x7f4438153820 2024-04-25T11:39:47.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.336+0000 7f442fe00640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44180744b0 0x7f44180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:47.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.336+0000 7f442fe00640 1 -- 172.21.15.19:0/1355858157 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7f44240b65d0 con 0x7f4438153820 2024-04-25T11:39:47.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.337+0000 7f4437e00640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44180744b0 0x7f44180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:47.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.338+0000 7f4437e00640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44180744b0 0x7f44180769c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4428005880 tx=0x7f4428009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.341+0000 7f442fe00640 1 -- 172.21.15.19:0/1355858157 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4424080350 con 0x7f4438153820 2024-04-25T11:39:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.624+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f4438050740 con 0x7f4438153820 2024-04-25T11:39:47.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:47.997+0000 7f442fe00640 1 -- 172.21.15.19:0/1355858157 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v275) ==== 101+0+0 (secure 0 0 0) 0x7f442407fc50 con 0x7f4438153820 2024-04-25T11:39:47.985 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:39:47.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.000+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44180744b0 msgr2=0x7f44180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:47.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.000+0000 7f443f200640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44180744b0 0x7f44180769c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4428005880 tx=0x7f4428009040 comp rx=0 tx=0).stop 2024-04-25T11:39:47.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.000+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 msgr2=0x7f4438144f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:47.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.000+0000 7f443f200640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f4438144f50 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f4424009890 tx=0x7f44240046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 shutdown_connections 2024-04-25T11:39:47.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44180744b0 0x7f44180769c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f443815d960 0x7f44380b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4438153820 0x7f4438144f50 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 --2- 172.21.15.19:0/1355858157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44380cee00 0x7f4438149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:47.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 >> 172.21.15.19:0/1355858157 conn(0x7f4438073050 msgr2=0x7f443815f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:47.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 shutdown_connections 2024-04-25T11:39:47.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.001+0000 7f443f200640 1 -- 172.21.15.19:0/1355858157 wait complete. 2024-04-25T11:39:48.022 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:48.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.223+0000 7f686c422f40 1 Processor -- start 2024-04-25T11:39:48.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.224+0000 7f686c422f40 1 -- start start 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f686c422f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c886c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f686c422f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c7fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f686c422f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 0x55eb4c7fc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f686c422f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb4c771d20 con 0x55eb4c7cb1e0 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f686c422f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb4c775d60 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f686c422f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb4c8143f0 con 0x55eb4c8a4fa0 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f6865600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c886c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f6865600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c886c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37670/0 (socket says 172.21.15.19:37670) 2024-04-25T11:39:48.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.225+0000 7f6865600640 1 -- 172.21.15.19:0/42481689 learned_addr learned my addr 172.21.15.19:0/42481689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f6866000640 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 0x55eb4c7fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f685fe00640 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c7fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f6865600640 1 -- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 msgr2=0x55eb4c7fc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f6865600640 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 0x55eb4c7fc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f6865600640 1 -- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb5f0 msgr2=0x55eb4c7fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f6865600640 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c7fc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f6865600640 1 -- 172.21.15.19:0/42481689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4c8871c0 con 0x55eb4c7cb1e0 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.226+0000 7f6865600640 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c886c80 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f685800a520 tx=0x7f685804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a84033717cf65ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f685f400640 1 -- 172.21.15.19:0/42481689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858065070 con 0x55eb4c7cb1e0 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f685f400640 1 -- 172.21.15.19:0/42481689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685805a040 con 0x55eb4c7cb1e0 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f685f400640 1 -- 172.21.15.19:0/42481689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858059070 con 0x55eb4c7cb1e0 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 -- 172.21.15.19:0/42481689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb1e0 msgr2=0x55eb4c886c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c886c80 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f685800a520 tx=0x7f685804e000 comp rx=0 tx=0).stop 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 -- 172.21.15.19:0/42481689 shutdown_connections 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c7fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 0x55eb4c7fc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 --2- 172.21.15.19:0/42481689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c886c80 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 -- 172.21.15.19:0/42481689 >> 172.21.15.19:0/42481689 conn(0x55eb4c6edac0 msgr2=0x55eb4c7c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 -- 172.21.15.19:0/42481689 shutdown_connections 2024-04-25T11:39:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.227+0000 7f686c422f40 1 -- 172.21.15.19:0/42481689 wait complete. 2024-04-25T11:39:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.229+0000 7f686c422f40 1 Processor -- start 2024-04-25T11:39:48.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.229+0000 7f686c422f40 1 -- start start 2024-04-25T11:39:48.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f686c422f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c885c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f686c422f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c93bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f686c422f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 0x55eb4c93c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f686c422f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb4c898040 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f686c422f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55eb4c9326b0 con 0x55eb4c7cb1e0 2024-04-25T11:39:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f686c422f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55eb4c7f98d0 con 0x55eb4c8a4fa0 2024-04-25T11:39:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c93bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c93bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37686/0 (socket says 172.21.15.19:37686) 2024-04-25T11:39:48.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 learned_addr learned my addr 172.21.15.19:0/3060801476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f6866000640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 0x55eb4c93c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f6865600640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c885c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 msgr2=0x55eb4c93c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55eb4c8a4fa0 0x55eb4c93c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb1e0 msgr2=0x55eb4c885c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55eb4c7cb1e0 0x55eb4c885c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.231+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4c886e20 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.232+0000 7f685fe00640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c93bd80 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f6850001e90 tx=0x7f685000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.232+0000 7f685e000640 1 -- 172.21.15.19:0/3060801476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850014020 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.232+0000 7f685e000640 1 -- 172.21.15.19:0/3060801476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6850010410 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.232+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55eb4c7ca4f0 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.232+0000 7f685e000640 1 -- 172.21.15.19:0/3060801476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850008030 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.232+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4c9323f0 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.234+0000 7f685e000640 1 -- 172.21.15.19:0/3060801476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6850017020 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.235+0000 7f685e000640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6844072e90 0x7f68440753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.235+0000 7f685e000640 1 -- 172.21.15.19:0/3060801476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7f6850094870 con 0x55eb4c7cb5f0 2024-04-25T11:39:48.223 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.236+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55eb4c8996e0 0x55eb4c931410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.236+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c93c800 con 0x55eb4c8996e0 2024-04-25T11:39:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.236+0000 7f6865600640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6844072e90 0x7f68440753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.236+0000 7f6866000640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55eb4c8996e0 0x55eb4c931410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.238+0000 7f6865600640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6844072e90 0x7f68440753a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6858000c00 tx=0x7f6858051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.238+0000 7f6866000640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55eb4c8996e0 0x55eb4c931410 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.241+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v121'877 uv851 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.241+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c931950 con 0x55eb4c8996e0 2024-04-25T11:39:48.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.245+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v121'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.245+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c931d20 con 0x55eb4c8996e0 2024-04-25T11:39:48.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.249+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v121'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.237 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:48.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.249+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945e00 con 0x55eb4c8996e0 2024-04-25T11:39:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.252+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.252+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9461a0 con 0x55eb4c8996e0 2024-04-25T11:39:48.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.256+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.256+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c944df0 con 0x55eb4c8996e0 2024-04-25T11:39:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.259+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.259+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945190 con 0x55eb4c8996e0 2024-04-25T11:39:48.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.265+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v121'883 uv857 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.265+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.267+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v121'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.267+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.259 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.271+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v121'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.271+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.275+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.275+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.279+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.279+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.283+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'888 uv888 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.284+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.288+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v121'889 uv864 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.289+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.293+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'890 uv890 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.293+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.298+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v121'891 uv891 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.286 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:48.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.298+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.302+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'892 uv892 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.302+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.307+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'893 uv893 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.307+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945530 con 0x55eb4c8996e0 2024-04-25T11:39:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.312+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'894 uv894 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.312+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c945960 con 0x55eb4c8996e0 2024-04-25T11:39:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.317+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'895 uv895 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.317+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.322+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v121'896 uv870 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.322+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.326+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v121'897 uv897 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.327+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.331+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v121'898 uv898 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.319 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:48.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.332+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.336+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'899 uv899 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.336+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.341+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'900 uv900 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.341+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.345+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'901 uv901 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.346+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.350+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v121'902 uv876 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.351+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.355+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v121'903 uv903 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.356+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.348 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:48.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.360+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v121'904 uv904 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.360+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.365+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'905 uv905 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.365+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.369+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'906 uv906 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.369+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c8996e0 2024-04-25T11:39:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.374+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'907 uv907 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.374+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55eb4c946a50 0x55eb4c948e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.374+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9457f0 con 0x55eb4c946a50 2024-04-25T11:39:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.375+0000 7f6865600640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55eb4c946a50 0x55eb4c948e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.376+0000 7f6865600640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55eb4c946a50 0x55eb4c948e70 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.381+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'638 uv621 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.381+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c9493b0 con 0x55eb4c946a50 2024-04-25T11:39:48.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.385+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'639 uv639 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.385+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c949750 con 0x55eb4c946a50 2024-04-25T11:39:48.377 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:48.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.390+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'640 uv640 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.390+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c949af0 con 0x55eb4c946a50 2024-04-25T11:39:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.395+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v121'641 uv641 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.395+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c949e90 con 0x55eb4c946a50 2024-04-25T11:39:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.400+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'642 uv642 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.400+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a230 con 0x55eb4c946a50 2024-04-25T11:39:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.404+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v121'643 uv626 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.404+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.409+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v121'644 uv644 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.409+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.401 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.414+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v121'645 uv645 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.414+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.418+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'646 uv646 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.419+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.423+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'647 uv647 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.423+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.427+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v121'648 uv631 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.427+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.432+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v121'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.432+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.436+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v121'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.424 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:48.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.436+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94ecc0 con 0x55eb4c946a50 2024-04-25T11:39:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.442+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v121'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.442+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94ecc0 con 0x55eb4c946a50 2024-04-25T11:39:48.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.446+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.447+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.451+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v121'653 uv637 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.451+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.456+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v121'654 uv654 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.456+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.448 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.460+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v121'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.460+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94b5e0 con 0x55eb4c946a50 2024-04-25T11:39:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.464+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.464+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94b5e0 con 0x55eb4c946a50 2024-04-25T11:39:48.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.469+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.469+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94a5d0 con 0x55eb4c946a50 2024-04-25T11:39:48.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.473+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c946a50 2024-04-25T11:39:48.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.473+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55eb4c94aae0 0x55eb4c94cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.474+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94d570 con 0x55eb4c94aae0 2024-04-25T11:39:48.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.475+0000 7f685fe00640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55eb4c94aae0 0x55eb4c94cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.477+0000 7f685fe00640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55eb4c94aae0 0x55eb4c94cf80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.480+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.480+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c94da30 con 0x55eb4c94aae0 2024-04-25T11:39:48.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.485+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v121'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.485+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94df10 con 0x55eb4c94aae0 2024-04-25T11:39:48.477 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.489+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v121'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.490+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94f7c0 con 0x55eb4c94aae0 2024-04-25T11:39:48.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.494+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.494+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94fb60 con 0x55eb4c94aae0 2024-04-25T11:39:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.498+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.498+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e7b0 con 0x55eb4c94aae0 2024-04-25T11:39:48.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.503+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.503+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e7b0 con 0x55eb4c8996e0 2024-04-25T11:39:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.508+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'128 uv127 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.508+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c94e7b0 con 0x55eb4c8996e0 2024-04-25T11:39:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.513+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.513+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e7b0 con 0x55eb4c8996e0 2024-04-25T11:39:48.504 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.517+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.517+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e7b0 con 0x55eb4c8996e0 2024-04-25T11:39:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.521+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.521+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55eb4c94f0d0 0x55eb4c951f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.521+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c952450 con 0x55eb4c94f0d0 2024-04-25T11:39:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.521+0000 7f6866000640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55eb4c94f0d0 0x55eb4c951f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.523+0000 7f6866000640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55eb4c94f0d0 0x55eb4c951f10 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.527+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.528+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c952450 con 0x55eb4c94f0d0 2024-04-25T11:39:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.531+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v121'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.532+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c952450 con 0x55eb4c94f0d0 2024-04-25T11:39:48.523 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.535+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v121'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.536+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c953460 con 0x55eb4c94f0d0 2024-04-25T11:39:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.539+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.540+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c953460 con 0x55eb4c94f0d0 2024-04-25T11:39:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.544+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.544+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c952450 con 0x55eb4c94f0d0 2024-04-25T11:39:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.548+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.548+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55eb4c952960 0x55eb4c954d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.548+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9554a0 con 0x55eb4c952960 2024-04-25T11:39:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.549+0000 7f6865600640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55eb4c952960 0x55eb4c954d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.550+0000 7f6865600640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55eb4c952960 0x55eb4c954d80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.555+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v121'316 uv310 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.555+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.560+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v121'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.560+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.551 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:48.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.563+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v121'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.564+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9568a0 con 0x55eb4c952960 2024-04-25T11:39:48.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.568+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.568+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9568a0 con 0x55eb4c952960 2024-04-25T11:39:48.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.572+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.572+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.577+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.577+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.581+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v121'322 uv315 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.581+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.585+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v121'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.585+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.578 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.590+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v121'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.590+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.594+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.595+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c955890 con 0x55eb4c952960 2024-04-25T11:39:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.599+0000 7f6865600640 1 -- 172.21.15.19:0/3060801476 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6858054d30 con 0x55eb4c952960 2024-04-25T11:39:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.599+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55eb4c955f10 0x55eb4c958330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.599+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958a60 con 0x55eb4c955f10 2024-04-25T11:39:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.601+0000 7f685fe00640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55eb4c955f10 0x55eb4c958330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.602+0000 7f685fe00640 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55eb4c955f10 0x55eb4c958330 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.605+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v121'143 uv142 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.606+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c958e30 con 0x55eb4c955f10 2024-04-25T11:39:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.610+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v121'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.610+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958e30 con 0x55eb4c955f10 2024-04-25T11:39:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.614+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v121'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.602 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.614+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958e30 con 0x55eb4c955f10 2024-04-25T11:39:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.619+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.619+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958e30 con 0x55eb4c955f10 2024-04-25T11:39:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.623+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'147 uv147 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.623+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958e30 con 0x55eb4c8996e0 2024-04-25T11:39:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.627+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.628+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c958e30 con 0x55eb4c8996e0 2024-04-25T11:39:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.632+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v121'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.632+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958e30 con 0x55eb4c8996e0 2024-04-25T11:39:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:48.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.637+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v121'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.637+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c95a230 con 0x55eb4c8996e0 2024-04-25T11:39:48.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.641+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.641+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c95a230 con 0x55eb4c8996e0 2024-04-25T11:39:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.646+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.646+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958fc0 con 0x55eb4c8996e0 2024-04-25T11:39:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.651+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c8996e0 2024-04-25T11:39:48.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.651+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958fc0 con 0x55eb4c94aae0 2024-04-25T11:39:48.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.656+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v121'284 uv283 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.656+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c958fc0 con 0x55eb4c94aae0 2024-04-25T11:39:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.661+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v121'285 uv285 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.661+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958fc0 con 0x55eb4c94aae0 2024-04-25T11:39:48.652 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.665+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v121'286 uv286 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.666+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e3a0 con 0x55eb4c94aae0 2024-04-25T11:39:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.670+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'287 uv287 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.670+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e3a0 con 0x55eb4c94aae0 2024-04-25T11:39:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.675+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'288 uv288 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.675+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e3a0 con 0x55eb4c94aae0 2024-04-25T11:39:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.679+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v121'289 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.679+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c94e3a0 con 0x55eb4c94aae0 2024-04-25T11:39:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.684+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v121'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.684+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e3a0 con 0x55eb4c94aae0 2024-04-25T11:39:48.675 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:48.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.688+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v121'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.688+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c94e3a0 con 0x55eb4c94aae0 2024-04-25T11:39:48.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.693+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.693+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c95be10 con 0x55eb4c94aae0 2024-04-25T11:39:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.697+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c94aae0 2024-04-25T11:39:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.697+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958fc0 con 0x55eb4c94f0d0 2024-04-25T11:39:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.702+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v121'124 uv123 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.702+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c958f40 con 0x55eb4c94f0d0 2024-04-25T11:39:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.706+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'125 uv125 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.706+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958f40 con 0x55eb4c94f0d0 2024-04-25T11:39:48.697 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.710+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v121'126 uv126 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.710+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c958f40 con 0x55eb4c94f0d0 2024-04-25T11:39:48.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.715+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'127 uv127 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.715+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c94f0d0 2024-04-25T11:39:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.719+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v121'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.719+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c94f0d0 2024-04-25T11:39:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.723+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v121'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.723+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c94f0d0 2024-04-25T11:39:48.713 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:48.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.726+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v121'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.726+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c95a400 con 0x55eb4c94f0d0 2024-04-25T11:39:48.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.730+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.730+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c95a400 con 0x55eb4c94f0d0 2024-04-25T11:39:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.734+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.734+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c94f0d0 2024-04-25T11:39:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.738+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f686000d590 con 0x55eb4c94f0d0 2024-04-25T11:39:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.738+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.742+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v121'145 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.742+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.746+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v121'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.746+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.738 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:48.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.750+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v121'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.750+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.755+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.755+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.759+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.759+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.764+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v121'351 uv343 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.764+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.769+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v121'352 uv352 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.769+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.761 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:48.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.773+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v121'353 uv353 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.773+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.778+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v121'354 uv354 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.778+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.782+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'355 uv355 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.782+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.787+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v121'356 uv350 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.787+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.792+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v121'357 uv357 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.792+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.783 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.796+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v121'358 uv358 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.796+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.800+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'359 uv359 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.800+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c95a400 con 0x55eb4c955f10 2024-04-25T11:39:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.804+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'360 uv360 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.804+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c9593f0 con 0x55eb4c955f10 2024-04-25T11:39:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.808+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'361 uv361 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.808+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55eb4c959950 con 0x55eb4c955f10 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f685fe00640 1 -- 172.21.15.19:0/3060801476 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'362 uv362 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6850020070 con 0x55eb4c955f10 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55eb4c946a50 msgr2=0x55eb4c948e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55eb4c946a50 0x55eb4c948e70 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55eb4c952960 msgr2=0x55eb4c954d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55eb4c952960 0x55eb4c954d80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55eb4c955f10 msgr2=0x55eb4c958330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55eb4c955f10 0x55eb4c958330 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55eb4c94aae0 msgr2=0x55eb4c94cf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55eb4c94aae0 0x55eb4c94cf80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 reap_dead start 2024-04-25T11:39:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55eb4c8996e0 msgr2=0x55eb4c931410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55eb4c8996e0 0x55eb4c931410 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55eb4c94f0d0 msgr2=0x55eb4c951f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55eb4c94f0d0 0x55eb4c951f10 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6844072e90 msgr2=0x7f68440753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6844072e90 0x7f68440753a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6858000c00 tx=0x7f6858051000 comp rx=0 tx=0).stop 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb5f0 msgr2=0x55eb4c93bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.813+0000 7f686c422f40 1 --2- 172.21.15.19:0/3060801476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4c7cb5f0 0x55eb4c93bd80 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f6850001e90 tx=0x7f685000df90 comp rx=0 tx=0).stop 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.814+0000 7f6866000640 1 -- 172.21.15.19:0/3060801476 reap_dead start 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.814+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 shutdown_connections 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.814+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 >> 172.21.15.19:0/3060801476 conn(0x55eb4c6edac0 msgr2=0x55eb4c7c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.814+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 shutdown_connections 2024-04-25T11:39:48.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:48.814+0000 7f686c422f40 1 -- 172.21.15.19:0/3060801476 wait complete. 2024-04-25T11:39:48.833 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2024-04-25T11:39:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.079+0000 7f290cd28f40 1 Processor -- start 2024-04-25T11:39:49.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.080+0000 7f290cd28f40 1 -- start start 2024-04-25T11:39:49.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f290cd28f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f290cd28f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56135c8e8a30 0x56135c9879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f290cd28f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c902ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f290cd28f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56135c85dd20 con 0x56135c8e8a30 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f290cd28f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56135c861d60 con 0x56135c987ef0 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f290cd28f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56135c9003f0 con 0x7f28ec0022f0 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56135c8e8a30 0x56135c9879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56135c8e8a30 0x56135c9879b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37706/0 (socket says 172.21.15.19:37706) 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 -- 172.21.15.19:0/82804787 learned_addr learned my addr 172.21.15.19:0/82804787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2905e00640 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c902ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2905400640 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 -- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28ec0022f0 msgr2=0x56135c987130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 -- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 msgr2=0x56135c902ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:49.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c902ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 -- 172.21.15.19:0/82804787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135c98e660 con 0x56135c8e8a30 2024-04-25T11:39:49.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2905400640 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:49.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2905e00640 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c902ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:49.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f2904a00640 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56135c8e8a30 0x56135c9879b0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c6e0 tx=0x7f28f000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbe4ec38f5172df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:49.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f28ff400640 1 -- 172.21.15.19:0/82804787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0012070 con 0x56135c8e8a30 2024-04-25T11:39:49.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f28ff400640 1 -- 172.21.15.19:0/82804787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28f000a860 con 0x56135c8e8a30 2024-04-25T11:39:49.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.082+0000 7f28ff400640 1 -- 172.21.15.19:0/82804787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f000ac10 con 0x56135c8e8a30 2024-04-25T11:39:49.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 -- 172.21.15.19:0/82804787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56135c8e8a30 msgr2=0x56135c9879b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56135c8e8a30 0x56135c9879b0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c6e0 tx=0x7f28f000cba0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 -- 172.21.15.19:0/82804787 shutdown_connections 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c902ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56135c8e8a30 0x56135c9879b0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 --2- 172.21.15.19:0/82804787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 -- 172.21.15.19:0/82804787 >> 172.21.15.19:0/82804787 conn(0x56135c7d9ac0 msgr2=0x56135c8e74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 -- 172.21.15.19:0/82804787 shutdown_connections 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.083+0000 7f290cd28f40 1 -- 172.21.15.19:0/82804787 wait complete. 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.084+0000 7f290cd28f40 1 Processor -- start 2024-04-25T11:39:49.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.086+0000 7f290cd28f40 1 -- start start 2024-04-25T11:39:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f290cd28f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56135c8e8a30 0x56135c97a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f290cd28f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c97b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f290cd28f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f290cd28f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56135c8fd8b0 con 0x7f28ec0022f0 2024-04-25T11:39:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f290cd28f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56135c903d00 con 0x56135c987ef0 2024-04-25T11:39:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f290cd28f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x56135c903700 con 0x56135c8e8a30 2024-04-25T11:39:49.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec0022f0 0x56135c987130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37710/0 (socket says 172.21.15.19:37710) 2024-04-25T11:39:49.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 -- 172.21.15.19:0/390738070 learned_addr learned my addr 172.21.15.19:0/390738070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2904a00640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c97b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905400640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56135c8e8a30 0x56135c97a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 -- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56135c8e8a30 msgr2=0x56135c97a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56135c8e8a30 0x56135c97a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 -- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 msgr2=0x56135c97b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c97b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 -- 172.21.15.19:0/390738070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135c97bc70 con 0x7f28ec0022f0 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2904a00640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56135c987ef0 0x56135c97b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905400640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56135c8e8a30 0x56135c97a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f2905e00640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec0022f0 0x56135c987130 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x56135c8fe9b0 tx=0x7f290000d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.087+0000 7f28fe000640 1 -- 172.21.15.19:0/390738070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900017070 con 0x7f28ec0022f0 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.088+0000 7f28fe000640 1 -- 172.21.15.19:0/390738070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29000032d0 con 0x7f28ec0022f0 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.088+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56135c97c180 con 0x7f28ec0022f0 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.088+0000 7f28fe000640 1 -- 172.21.15.19:0/390738070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900001850 con 0x7f28ec0022f0 2024-04-25T11:39:49.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.088+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135c97c740 con 0x7f28ec0022f0 2024-04-25T11:39:49.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.090+0000 7f28fe000640 1 -- 172.21.15.19:0/390738070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f290001a070 con 0x7f28ec0022f0 2024-04-25T11:39:49.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.092+0000 7f28fe000640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28e4072e90 0x7f28e40753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.092+0000 7f28fe000640 1 -- 172.21.15.19:0/390738070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7f2900093cf0 con 0x7f28ec0022f0 2024-04-25T11:39:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.092+0000 7f290cd28f40 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56135c8fc480 0x56135c8fc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.092+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x56135c9023c0 con 0x56135c8fc480 2024-04-25T11:39:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.092+0000 7f2905400640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28e4072e90 0x7f28e40753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.092+0000 7f2904a00640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56135c8fc480 0x56135c8fc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.092+0000 7f2904a00640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56135c8fc480 0x56135c8fc890 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:49.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.093+0000 7f2905400640 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28e4072e90 0x7f28e40753a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009810 tx=0x7f28f805a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:49.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f2904a00640 1 -- 172.21.15.19:0/390738070 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v121'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f28f000aa10 con 0x56135c8fc480 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56135c8fc480 msgr2=0x56135c8fc890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x56135c8fc480 0x56135c8fc890 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28e4072e90 msgr2=0x7f28e40753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f28e4072e90 0x7f28e40753a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009810 tx=0x7f28f805a040 comp rx=0 tx=0).stop 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec0022f0 msgr2=0x56135c987130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 --2- 172.21.15.19:0/390738070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28ec0022f0 0x56135c987130 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x56135c8fe9b0 tx=0x7f290000d590 comp rx=0 tx=0).stop 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f2905e00640 1 -- 172.21.15.19:0/390738070 reap_dead start 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 shutdown_connections 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.094+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 >> 172.21.15.19:0/390738070 conn(0x56135c7d9ac0 msgr2=0x56135c8e8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.095+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 shutdown_connections 2024-04-25T11:39:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.095+0000 7f290cd28f40 1 -- 172.21.15.19:0/390738070 wait complete. 2024-04-25T11:39:49.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.390+0000 7f4ecac00640 1 Processor -- start 2024-04-25T11:39:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.390+0000 7f4ecac00640 1 -- start start 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.392+0000 7f4ecac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.392+0000 7f4ecac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4ec406dce0 0x7f4ec4160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.392+0000 7f4ecac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec4071110 con 0x7f4ec415adf0 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.393+0000 7f4ecac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec40713f0 con 0x7f4ec406dce0 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.393+0000 7f4ecac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40701a0 con 0x7f4ec4070630 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.393+0000 7f4ec3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4ec415adf0 0x7f4ec406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52280/0 (socket says 172.21.15.19:52280) 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.393+0000 7f4ec3400640 1 -- 172.21.15.19:0/528241304 learned_addr learned my addr 172.21.15.19:0/528241304 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.393+0000 7f4ec8c00640 1 --2- 172.21.15.19:0/528241304 >> v2:172.21.15.19:3300/0 conn(0x7f4ec406dce0 0x7f4ec4160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.393+0000 7f4ec3e00640 1 --2- 172.21.15.19:0/528241304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.393+0000 7f4ec2a00640 1 -- 172.21.15.19:0/528241304 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3475062833 0 0) 0x7f4ec4071110 con 0x7f4ec415adf0 2024-04-25T11:39:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.394+0000 7f4ec2a00640 1 -- 172.21.15.19:0/528241304 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ea8003520 con 0x7f4ec415adf0 2024-04-25T11:39:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.394+0000 7f4ec8c00640 1 -- 172.21.15.19:0/528241304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 msgr2=0x7f4ec40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.394+0000 7f4ec8c00640 1 --2- 172.21.15.19:0/528241304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec40cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.394+0000 7f4ec8c00640 1 -- 172.21.15.19:0/528241304 >> v1:172.21.15.19:6789/0 conn(0x7f4ec415adf0 legacy=0x7f4ec406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.394+0000 7f4ec8c00640 1 -- 172.21.15.19:0/528241304 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec406fb50 con 0x7f4ec406dce0 2024-04-25T11:39:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.395+0000 7f4ec8c00640 1 --2- 172.21.15.19:0/528241304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec406dce0 0x7f4ec4160d80 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009d90 tx=0x7f4ebc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39b4347be52c4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.395+0000 7f4ec2a00640 1 -- 172.21.15.19:0/528241304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc007740 con 0x7f4ec406dce0 2024-04-25T11:39:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.395+0000 7f4ec2a00640 1 -- 172.21.15.19:0/528241304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc0078e0 con 0x7f4ec406dce0 2024-04-25T11:39:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.395+0000 7f4ec2a00640 1 -- 172.21.15.19:0/528241304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc007cb0 con 0x7f4ec406dce0 2024-04-25T11:39:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 -- 172.21.15.19:0/528241304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec406dce0 msgr2=0x7f4ec4160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 --2- 172.21.15.19:0/528241304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec406dce0 0x7f4ec4160d80 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009d90 tx=0x7f4ebc02a5f0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 -- 172.21.15.19:0/528241304 shutdown_connections 2024-04-25T11:39:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 --2- 172.21.15.19:0/528241304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec406dce0 0x7f4ec4160d80 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 --2- 172.21.15.19:0/528241304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec40cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 -- 172.21.15.19:0/528241304 >> 172.21.15.19:0/528241304 conn(0x7f4ec4073090 msgr2=0x7f4ec40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 -- 172.21.15.19:0/528241304 shutdown_connections 2024-04-25T11:39:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.396+0000 7f4ecac00640 1 -- 172.21.15.19:0/528241304 wait complete. 2024-04-25T11:39:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.399+0000 7f4ecac00640 1 Processor -- start 2024-04-25T11:39:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.399+0000 7f4ecac00640 1 -- start start 2024-04-25T11:39:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.400+0000 7f4ecac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec4152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.400+0000 7f4ecac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec415adf0 0x7f4ec414afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.400+0000 7f4ecac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec41531b0 0x7f4ec414b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.400+0000 7f4ecac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40b0e80 con 0x7f4ec415adf0 2024-04-25T11:39:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.400+0000 7f4ecac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40b0d00 con 0x7f4ec41531b0 2024-04-25T11:39:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.400+0000 7f4ecac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec4070c80 con 0x7f4ec4070630 2024-04-25T11:39:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec415adf0 0x7f4ec414afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec415adf0 0x7f4ec414afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37738/0 (socket says 172.21.15.19:37738) 2024-04-25T11:39:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 -- 172.21.15.19:0/2516877811 learned_addr learned my addr 172.21.15.19:0/2516877811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec8c00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec41531b0 0x7f4ec414b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3e00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec4152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 -- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 msgr2=0x7f4ec4152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec4152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 -- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec41531b0 msgr2=0x7f4ec414b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec41531b0 0x7f4ec414b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3400640 1 -- 172.21.15.19:0/2516877811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec41539e0 con 0x7f4ec415adf0 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec3e00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec4152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.401+0000 7f4ec8c00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec41531b0 0x7f4ec414b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.402+0000 7f4ec3400640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec415adf0 0x7f4ec414afc0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c390 tx=0x7f4eb0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.402+0000 7f4ebbe00640 1 -- 172.21.15.19:0/2516877811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0017070 con 0x7f4ec415adf0 2024-04-25T11:39:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.402+0000 7f4ebbe00640 1 -- 172.21.15.19:0/2516877811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb0005ce0 con 0x7f4ec415adf0 2024-04-25T11:39:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.402+0000 7f4ebbe00640 1 -- 172.21.15.19:0/2516877811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00072f0 con 0x7f4ec415adf0 2024-04-25T11:39:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.402+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec4153cc0 con 0x7f4ec415adf0 2024-04-25T11:39:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.402+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec4154280 con 0x7f4ec415adf0 2024-04-25T11:39:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.403+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec4153750 con 0x7f4ec415adf0 2024-04-25T11:39:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.405+0000 7f4ebbe00640 1 -- 172.21.15.19:0/2516877811 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4eb001a070 con 0x7f4ec415adf0 2024-04-25T11:39:49.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.406+0000 7f4ebbe00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ea40744b0 0x7f4ea40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:49.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.406+0000 7f4ebbe00640 1 -- 172.21.15.19:0/2516877811 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8203+0+0 (secure 0 0 0) 0x7f4eb0094f40 con 0x7f4ec415adf0 2024-04-25T11:39:49.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.406+0000 7f4ec3e00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ea40744b0 0x7f4ea40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.408+0000 7f4ec3e00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ea40744b0 0x7f4ea40769c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40cfd00 tx=0x7f4eb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:49.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.411+0000 7f4ebbe00640 1 -- 172.21.15.19:0/2516877811 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4eb005ec60 con 0x7f4ec415adf0 2024-04-25T11:39:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:49.655+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4ec4050740 con 0x7f4ec415adf0 2024-04-25T11:39:50.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.024+0000 7f4ebbe00640 1 -- 172.21.15.19:0/2516877811 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v276) ==== 162+0+0 (secure 0 0 0) 0x7f4eb005e560 con 0x7f4ec415adf0 2024-04-25T11:39:50.011 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:50.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.028+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ea40744b0 msgr2=0x7f4ea40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.028+0000 7f4ecac00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ea40744b0 0x7f4ea40769c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4ec40cfd00 tx=0x7f4eb4008040 comp rx=0 tx=0).stop 2024-04-25T11:39:50.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.028+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec415adf0 msgr2=0x7f4ec414afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.028+0000 7f4ecac00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec415adf0 0x7f4ec414afc0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c390 tx=0x7f4eb0001e70 comp rx=0 tx=0).stop 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 shutdown_connections 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4ea40744b0 0x7f4ea40769c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ec41531b0 0x7f4ec414b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ec415adf0 0x7f4ec414afc0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 --2- 172.21.15.19:0/2516877811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ec4070630 0x7f4ec4152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 >> 172.21.15.19:0/2516877811 conn(0x7f4ec4073090 msgr2=0x7f4ec40cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 shutdown_connections 2024-04-25T11:39:50.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.029+0000 7f4ecac00640 1 -- 172.21.15.19:0/2516877811 wait complete. 2024-04-25T11:39:50.030 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:50.043+0000 7f0f05a00640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-25T11:39:50.031 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:50.044+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:50.031 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:50.044+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:39:50.042 INFO:tasks.ceph.mds.a.smithi019.stderr: -25> 2024-04-25T11:39:50.043+0000 7f0f05a00640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-04-25T11:39:50.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:50.175 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.326+0000 7f7341000640 1 Processor -- start 2024-04-25T11:39:50.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.326+0000 7f7341000640 1 -- start start 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f7341000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f733c0b75f0 0x7f733c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f7341000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f7341000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733c0cfb40 con 0x7f733c0cf270 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f7341000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f733c0cf680 con 0x7f733c0b75f0 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f7341000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f733c06c6f0 con 0x7f733c1605c0 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f733b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f733c0b75f0 0x7f733c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f733b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f733c0b75f0 0x7f733c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37768/0 (socket says 172.21.15.19:37768) 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.329+0000 7f733b400640 1 -- 172.21.15.19:0/2815504256 learned_addr learned my addr 172.21.15.19:0/2815504256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:50.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.330+0000 7f733aa00640 1 --2- 172.21.15.19:0/2815504256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.330+0000 7f733b400640 1 -- 172.21.15.19:0/2815504256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 msgr2=0x7f733c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.330+0000 7f733b400640 1 --2- 172.21.15.19:0/2815504256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.330+0000 7f733b400640 1 -- 172.21.15.19:0/2815504256 >> v1:172.21.15.169:6789/0 conn(0x7f733c0cf270 legacy=0x7f733c169000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.330+0000 7f733b400640 1 -- 172.21.15.19:0/2815504256 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c0764e0 con 0x7f733c0b75f0 2024-04-25T11:39:50.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.330+0000 7f733aa00640 1 --2- 172.21.15.19:0/2815504256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:50.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.331+0000 7f733b400640 1 --2- 172.21.15.19:0/2815504256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c0ce7f0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f732c009e50 tx=0x7f732c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54d014b2fef7e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:50.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.331+0000 7f733a000640 1 -- 172.21.15.19:0/2815504256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c036070 con 0x7f733c0b75f0 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.331+0000 7f733a000640 1 -- 172.21.15.19:0/2815504256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c031600 con 0x7f733c0b75f0 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.331+0000 7f733a000640 1 -- 172.21.15.19:0/2815504256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0319d0 con 0x7f733c0b75f0 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 -- 172.21.15.19:0/2815504256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 msgr2=0x7f733c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 --2- 172.21.15.19:0/2815504256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c0ce7f0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f732c009e50 tx=0x7f732c02a6c0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 -- 172.21.15.19:0/2815504256 shutdown_connections 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 --2- 172.21.15.19:0/2815504256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 --2- 172.21.15.19:0/2815504256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c0ce7f0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 -- 172.21.15.19:0/2815504256 >> 172.21.15.19:0/2815504256 conn(0x7f733c09b7d0 msgr2=0x7f733c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:50.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 -- 172.21.15.19:0/2815504256 shutdown_connections 2024-04-25T11:39:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.332+0000 7f7341000640 1 -- 172.21.15.19:0/2815504256 wait complete. 2024-04-25T11:39:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.335+0000 7f7341000640 1 Processor -- start 2024-04-25T11:39:50.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.335+0000 7f7341000640 1 -- start start 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.336+0000 7f7341000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c1472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.336+0000 7f7341000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f733c0cf270 0x7f733c1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.336+0000 7f7341000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.336+0000 7f7341000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f733c076200 con 0x7f733c0b75f0 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.336+0000 7f7341000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f733c075f20 con 0x7f733c0cf270 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.336+0000 7f7341000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f733c06cb80 con 0x7f733c1605c0 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c1472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c1472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37784/0 (socket says 172.21.15.19:37784) 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 -- 172.21.15.19:0/348509720 learned_addr learned my addr 172.21.15.19:0/348509720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:50.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 -- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 msgr2=0x7f733c149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 -- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f733c0cf270 msgr2=0x7f733c1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733aa00640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f733c0cf270 0x7f733c1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f733c0cf270 0x7f733c1609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733b400640 1 -- 172.21.15.19:0/348509720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c15e220 con 0x7f733c0b75f0 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.337+0000 7f733aa00640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f733c0cf270 0x7f733c1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.338+0000 7f733b400640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c1472d0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f732c009e50 tx=0x7f732c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.338+0000 7f7333400640 1 -- 172.21.15.19:0/348509720 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c041020 con 0x7f733c0b75f0 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.338+0000 7f7333400640 1 -- 172.21.15.19:0/348509720 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732c007820 con 0x7f733c0b75f0 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.338+0000 7f7333400640 1 -- 172.21.15.19:0/348509720 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c036040 con 0x7f733c0b75f0 2024-04-25T11:39:50.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.338+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c15e500 con 0x7f733c0b75f0 2024-04-25T11:39:50.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.338+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c15eac0 con 0x7f733c0b75f0 2024-04-25T11:39:50.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.339+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c15ed00 con 0x7f733c0b75f0 2024-04-25T11:39:50.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.340+0000 7f7333400640 1 -- 172.21.15.19:0/348509720 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f732c044020 con 0x7f733c0b75f0 2024-04-25T11:39:50.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.341+0000 7f7333400640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73180744e0 0x7f73180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.341+0000 7f7333400640 1 -- 172.21.15.19:0/348509720 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7f732c0b6b00 con 0x7f733c0b75f0 2024-04-25T11:39:50.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.342+0000 7f733aa00640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73180744e0 0x7f73180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.343+0000 7f733aa00640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73180744e0 0x7f73180769f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7324006fd0 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:50.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.346+0000 7f7333400640 1 -- 172.21.15.19:0/348509720 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f733c15ed00 con 0x7f733c0b75f0 2024-04-25T11:39:50.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.594+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f733c050740 con 0x7f733c0b75f0 2024-04-25T11:39:50.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.595+0000 7f7333400640 1 -- 172.21.15.19:0/348509720 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 278 v278) ==== 95+0+4352 (secure 0 0 0) 0x7f732c04b030 con 0x7f733c0b75f0 2024-04-25T11:39:50.583 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:50.583 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":278,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":278,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:50.045229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:50.583 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 278 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.598+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73180744e0 msgr2=0x7f73180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.598+0000 7f7341000640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73180744e0 0x7f73180769f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7324006fd0 tx=0x7f7324008040 comp rx=0 tx=0).stop 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.598+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 msgr2=0x7f733c1472d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.598+0000 7f7341000640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c1472d0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f732c009e50 tx=0x7f732c004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 shutdown_connections 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f73180744e0 0x7f73180769f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f733c0cf270 0x7f733c1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f733c1605c0 0x7f733c149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 --2- 172.21.15.19:0/348509720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f733c0b75f0 0x7f733c1472d0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 >> 172.21.15.19:0/348509720 conn(0x7f733c09b7d0 msgr2=0x7f733c070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:50.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 shutdown_connections 2024-04-25T11:39:50.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.599+0000 7f7341000640 1 -- 172.21.15.19:0/348509720 wait complete. 2024-04-25T11:39:50.610 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:50.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.912+0000 7f7771000640 1 Processor -- start 2024-04-25T11:39:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.912+0000 7f7771000640 1 -- start start 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f7771000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 0x7f776c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f7771000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f776c06f180 0x7f776c06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f7771000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776c06d7a0 con 0x7f776c06d390 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f7771000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f776c071230 con 0x7f776c06f180 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f7771000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f776c06f8f0 con 0x7f776c070e20 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f776aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f776c06f180 0x7f776c06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f776aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f776c06f180 0x7f776c06ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37786/0 (socket says 172.21.15.19:37786) 2024-04-25T11:39:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.915+0000 7f776aa00640 1 -- 172.21.15.19:0/3794832666 learned_addr learned my addr 172.21.15.19:0/3794832666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:50.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.916+0000 7f776b400640 1 --2- 172.21.15.19:0/3794832666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 0x7f776c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.916+0000 7f776aa00640 1 -- 172.21.15.19:0/3794832666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 msgr2=0x7f776c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:50.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.916+0000 7f776aa00640 1 --2- 172.21.15.19:0/3794832666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 0x7f776c15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.916+0000 7f776aa00640 1 -- 172.21.15.19:0/3794832666 >> v1:172.21.15.169:6789/0 conn(0x7f776c06d390 legacy=0x7f776c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:50.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.916+0000 7f776aa00640 1 -- 172.21.15.19:0/3794832666 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c06fbd0 con 0x7f776c06f180 2024-04-25T11:39:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.917+0000 7f776aa00640 1 --2- 172.21.15.19:0/3794832666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c06ce50 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f775c009e50 tx=0x7f775c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66344ce94e7b276e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.917+0000 7f776a000640 1 -- 172.21.15.19:0/3794832666 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c036070 con 0x7f776c06f180 2024-04-25T11:39:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.917+0000 7f776a000640 1 -- 172.21.15.19:0/3794832666 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775c031600 con 0x7f776c06f180 2024-04-25T11:39:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.917+0000 7f776a000640 1 -- 172.21.15.19:0/3794832666 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0319d0 con 0x7f776c06f180 2024-04-25T11:39:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 -- 172.21.15.19:0/3794832666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 msgr2=0x7f776c06ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 --2- 172.21.15.19:0/3794832666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c06ce50 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f775c009e50 tx=0x7f775c02a6c0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 -- 172.21.15.19:0/3794832666 shutdown_connections 2024-04-25T11:39:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 --2- 172.21.15.19:0/3794832666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c06ce50 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 --2- 172.21.15.19:0/3794832666 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 0x7f776c15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 -- 172.21.15.19:0/3794832666 >> 172.21.15.19:0/3794832666 conn(0x7f776c0d1720 msgr2=0x7f776c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 -- 172.21.15.19:0/3794832666 shutdown_connections 2024-04-25T11:39:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.918+0000 7f7771000640 1 -- 172.21.15.19:0/3794832666 wait complete. 2024-04-25T11:39:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.920+0000 7f7771000640 1 Processor -- start 2024-04-25T11:39:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.921+0000 7f7771000640 1 -- start start 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.922+0000 7f7771000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f776c06d390 0x7f776c153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.922+0000 7f7771000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.922+0000 7f7771000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 0x7f776c14bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.922+0000 7f7771000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0b6e30 con 0x7f776c06f180 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.922+0000 7f7771000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f776c070190 con 0x7f776c06d390 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.922+0000 7f7771000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f776c06f590 con 0x7f776c070e20 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.922+0000 7f776aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37790/0 (socket says 172.21.15.19:37790) 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 -- 172.21.15.19:0/1627698700 learned_addr learned my addr 172.21.15.19:0/1627698700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776b400640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f776c06d390 0x7f776c153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 -- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 msgr2=0x7f776c14bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 0x7f776c14bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 -- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f776c06d390 msgr2=0x7f776c153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f776c06d390 0x7f776c153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 -- 172.21.15.19:0/1627698700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c149910 con 0x7f776c06f180 2024-04-25T11:39:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.923+0000 7f776aa00640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c147ee0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f775c034040 tx=0x7f775c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.924+0000 7f7763400640 1 -- 172.21.15.19:0/1627698700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c036050 con 0x7f776c06f180 2024-04-25T11:39:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.924+0000 7f7763400640 1 -- 172.21.15.19:0/1627698700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775c042070 con 0x7f776c06f180 2024-04-25T11:39:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.924+0000 7f7763400640 1 -- 172.21.15.19:0/1627698700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c009510 con 0x7f776c06f180 2024-04-25T11:39:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.924+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c149bf0 con 0x7f776c06f180 2024-04-25T11:39:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.924+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c14a1b0 con 0x7f776c06f180 2024-04-25T11:39:50.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.925+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c149680 con 0x7f776c06f180 2024-04-25T11:39:50.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.926+0000 7f7763400640 1 -- 172.21.15.19:0/1627698700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f775c007c50 con 0x7f776c06f180 2024-04-25T11:39:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.927+0000 7f7763400640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77480744e0 0x7f77480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.928+0000 7f7763400640 1 -- 172.21.15.19:0/1627698700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7f775c0b6c90 con 0x7f776c06f180 2024-04-25T11:39:50.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.928+0000 7f776b400640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77480744e0 0x7f77480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:50.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.930+0000 7f776b400640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77480744e0 0x7f77480769f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f775400b7c0 tx=0x7f7754008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:50.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:50.932+0000 7f7763400640 1 -- 172.21.15.19:0/1627698700 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f776c06d7a0 con 0x7f776c06f180 2024-04-25T11:39:51.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.179+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f776c14a590 con 0x7f776c06f180 2024-04-25T11:39:51.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.180+0000 7f7763400640 1 -- 172.21.15.19:0/1627698700 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5179 (secure 0 0 0) 0x7f776c14a590 con 0x7f776c06f180 2024-04-25T11:39:51.167 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:51.167 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":279,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2406308121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2406308121},{"type":"v1","addr":"172.21.15.19:6835","nonce":2406308121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":279}],"filesystems":[{"mdsmap":{"epoch":278,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:50.045229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:51.168 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 279 2024-04-25T11:39:51.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77480744e0 msgr2=0x7f77480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77480744e0 0x7f77480769f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f775400b7c0 tx=0x7f7754008040 comp rx=0 tx=0).stop 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 msgr2=0x7f776c147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c147ee0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f775c034040 tx=0x7f775c004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 shutdown_connections 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77480744e0 0x7f77480769f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f776c070e20 0x7f776c14bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f776c06f180 0x7f776c147ee0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 --2- 172.21.15.19:0/1627698700 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f776c06d390 0x7f776c153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.183+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 >> 172.21.15.19:0/1627698700 conn(0x7f776c0d1720 msgr2=0x7f776c15b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.184+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 shutdown_connections 2024-04-25T11:39:51.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.184+0000 7f7771000640 1 -- 172.21.15.19:0/1627698700 wait complete. 2024-04-25T11:39:51.198 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:51.198 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:51.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.485+0000 7faca7e00640 1 Processor -- start 2024-04-25T11:39:51.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.486+0000 7faca7e00640 1 -- start start 2024-04-25T11:39:51.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.488+0000 7faca7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca80b4ea0 0x7faca80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:51.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.488+0000 7faca7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faca8144370 0x7faca815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:51.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.488+0000 7faca7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca80d1230 con 0x7faca806c5c0 2024-04-25T11:39:51.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.488+0000 7faca7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faca80d06b0 con 0x7faca8144370 2024-04-25T11:39:51.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.488+0000 7faca7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faca80d0990 con 0x7faca80b4ea0 2024-04-25T11:39:51.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7faca806c5c0 0x7faca80b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52300/0 (socket says 172.21.15.19:52300) 2024-04-25T11:39:51.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca6a00640 1 -- 172.21.15.19:0/2134850732 learned_addr learned my addr 172.21.15.19:0/2134850732 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:51.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca7400640 1 --2- 172.21.15.19:0/2134850732 >> v2:172.21.15.19:3300/0 conn(0x7faca8144370 0x7faca815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:51.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7fac9fe00640 1 --2- 172.21.15.19:0/2134850732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca80b4ea0 0x7faca80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca6000640 1 -- 172.21.15.19:0/2134850732 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3642740828 0 0) 0x7faca80d1230 con 0x7faca806c5c0 2024-04-25T11:39:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca7400640 1 -- 172.21.15.19:0/2134850732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca80b4ea0 msgr2=0x7faca80cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca7400640 1 --2- 172.21.15.19:0/2134850732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca80b4ea0 0x7faca80cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca7400640 1 -- 172.21.15.19:0/2134850732 >> v1:172.21.15.19:6789/0 conn(0x7faca806c5c0 legacy=0x7faca80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.489+0000 7faca7400640 1 -- 172.21.15.19:0/2134850732 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca80d0c70 con 0x7faca8144370 2024-04-25T11:39:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.490+0000 7faca7400640 1 --2- 172.21.15.19:0/2134850732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca815e770 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7faca0009d90 tx=0x7faca002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a377bb480574afae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:51.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.491+0000 7faca6000640 1 -- 172.21.15.19:0/2134850732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0007740 con 0x7faca8144370 2024-04-25T11:39:51.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.491+0000 7faca6000640 1 -- 172.21.15.19:0/2134850732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca00078e0 con 0x7faca8144370 2024-04-25T11:39:51.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.491+0000 7faca6000640 1 -- 172.21.15.19:0/2134850732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0007cb0 con 0x7faca8144370 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.491+0000 7faca7e00640 1 -- 172.21.15.19:0/2134850732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 msgr2=0x7faca815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.491+0000 7faca7e00640 1 --2- 172.21.15.19:0/2134850732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca815e770 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7faca0009d90 tx=0x7faca002a660 comp rx=0 tx=0).stop 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.492+0000 7faca7e00640 1 -- 172.21.15.19:0/2134850732 shutdown_connections 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.492+0000 7faca7e00640 1 --2- 172.21.15.19:0/2134850732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca815e770 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.492+0000 7faca7e00640 1 --2- 172.21.15.19:0/2134850732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca80b4ea0 0x7faca80cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.492+0000 7faca7e00640 1 -- 172.21.15.19:0/2134850732 >> 172.21.15.19:0/2134850732 conn(0x7faca80d1720 msgr2=0x7faca80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.492+0000 7faca7e00640 1 -- 172.21.15.19:0/2134850732 shutdown_connections 2024-04-25T11:39:51.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.492+0000 7faca7e00640 1 -- 172.21.15.19:0/2134850732 wait complete. 2024-04-25T11:39:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.494+0000 7faca7e00640 1 Processor -- start 2024-04-25T11:39:51.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.494+0000 7faca7e00640 1 -- start start 2024-04-25T11:39:51.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7faca7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faca806c5c0 0x7faca8146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7faca7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca813f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7faca7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca813ff30 0x7faca8070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7faca7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faca80d0450 con 0x7faca8144370 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7faca7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faca80d02d0 con 0x7faca806c5c0 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7faca7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faca8147760 con 0x7faca813ff30 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7fac9fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca813f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7fac9fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca813f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37808/0 (socket says 172.21.15.19:37808) 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7fac9fe00640 1 -- 172.21.15.19:0/1936869788 learned_addr learned my addr 172.21.15.19:0/1936869788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.496+0000 7faca6a00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faca806c5c0 0x7faca8146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9fe00640 1 -- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca813ff30 msgr2=0x7faca8070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7faca7400640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca813ff30 0x7faca8070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9fe00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca813ff30 0x7faca8070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9fe00640 1 -- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faca806c5c0 msgr2=0x7faca8146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9fe00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faca806c5c0 0x7faca8146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9fe00640 1 -- 172.21.15.19:0/1936869788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca806f560 con 0x7faca8144370 2024-04-25T11:39:51.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7faca6a00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faca806c5c0 0x7faca8146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:51.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9fe00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca813f9f0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fac9000d5f0 tx=0x7fac9000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:51.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9ea00640 1 -- 172.21.15.19:0/1936869788 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900125f0 con 0x7faca8144370 2024-04-25T11:39:51.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9ea00640 1 -- 172.21.15.19:0/1936869788 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac90019070 con 0x7faca8144370 2024-04-25T11:39:51.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca806f840 con 0x7faca8144370 2024-04-25T11:39:51.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7fac9ea00640 1 -- 172.21.15.19:0/1936869788 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900129d0 con 0x7faca8144370 2024-04-25T11:39:51.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.497+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca806fe00 con 0x7faca8144370 2024-04-25T11:39:51.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.499+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca8050740 con 0x7faca8144370 2024-04-25T11:39:51.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.500+0000 7fac9ea00640 1 -- 172.21.15.19:0/1936869788 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac90012b70 con 0x7faca8144370 2024-04-25T11:39:51.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.501+0000 7fac9ea00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac840744e0 0x7fac840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:51.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.501+0000 7fac9ea00640 1 -- 172.21.15.19:0/1936869788 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7fac90095f90 con 0x7faca8144370 2024-04-25T11:39:51.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.501+0000 7faca6a00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac840744e0 0x7fac840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:51.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.503+0000 7faca6a00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac840744e0 0x7fac840769f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fac98000f60 tx=0x7fac98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.506+0000 7fac9ea00640 1 -- 172.21.15.19:0/1936869788 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac9005fd40 con 0x7faca8144370 2024-04-25T11:39:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.752+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faca806d7f0 con 0x7faca8144370 2024-04-25T11:39:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.754+0000 7fac9ea00640 1 -- 172.21.15.19:0/1936869788 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5179 (secure 0 0 0) 0x7fac9005f640 con 0x7faca8144370 2024-04-25T11:39:51.741 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:51.741 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":279,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2406308121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2406308121},{"type":"v1","addr":"172.21.15.19:6835","nonce":2406308121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":279}],"filesystems":[{"mdsmap":{"epoch":278,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:50.045229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:51.741 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 279 2024-04-25T11:39:51.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac840744e0 msgr2=0x7fac840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac840744e0 0x7fac840769f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fac98000f60 tx=0x7fac98009040 comp rx=0 tx=0).stop 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 msgr2=0x7faca813f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca813f9f0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fac9000d5f0 tx=0x7fac9000dab0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 shutdown_connections 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fac840744e0 0x7fac840769f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faca813ff30 0x7faca8070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faca8144370 0x7faca813f9f0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 --2- 172.21.15.19:0/1936869788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faca806c5c0 0x7faca8146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.756+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 >> 172.21.15.19:0/1936869788 conn(0x7faca80d1720 msgr2=0x7faca80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.757+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 shutdown_connections 2024-04-25T11:39:51.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:51.757+0000 7faca7e00640 1 -- 172.21.15.19:0/1936869788 wait complete. 2024-04-25T11:39:51.769 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2024-04-25T11:39:51.769 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2024-04-25T11:39:51.770 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:51.770 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:51.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:39:51.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:51.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:51.824 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:51.876 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:39:51.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:39:51.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.169+0000 7f713ee00640 1 Processor -- start 2024-04-25T11:39:52.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.169+0000 7f713ee00640 1 -- start start 2024-04-25T11:39:52.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713ee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f71380d1070 0x7f7138162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:52.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713ee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713806fc70 con 0x7f7138144e60 2024-04-25T11:39:52.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713ee00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713806ff50 con 0x7f7138144350 2024-04-25T11:39:52.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713ee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7138070230 con 0x7f71380d1070 2024-04-25T11:39:52.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f71380d1070 0x7f7138162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:52.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f71380d1070 0x7f7138162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37814/0 (socket says 172.21.15.19:37814) 2024-04-25T11:39:52.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713d000640 1 -- 172.21.15.19:0/1438975478 learned_addr learned my addr 172.21.15.19:0/1438975478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:52.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.171+0000 7f713e400640 1 --1- 172.21.15.19:0/1438975478 >> v1:172.21.15.19:6789/0 conn(0x7f7138144e60 0x7f7138145270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52304/0 (socket says 172.21.15.19:52304) 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.172+0000 7f713d000640 1 -- 172.21.15.19:0/1438975478 >> v1:172.21.15.169:6789/0 conn(0x7f7138144350 legacy=0x7f7138161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.172+0000 7f713d000640 1 -- 172.21.15.19:0/1438975478 >> v1:172.21.15.19:6789/0 conn(0x7f7138144e60 legacy=0x7f7138145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.172+0000 7f713d000640 1 -- 172.21.15.19:0/1438975478 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7138070510 con 0x7f71380d1070 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.172+0000 7f7137e00640 1 -- 172.21.15.19:0/1438975478 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 534019574 0 0) 0x7f713806fc70 con 0x7f7138144e60 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.173+0000 7f713d000640 1 --2- 172.21.15.19:0/1438975478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138162200 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f7124009870 tx=0x7f712402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9d7ad5d962be98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.173+0000 7f7137e00640 1 -- 172.21.15.19:0/1438975478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124037070 con 0x7f71380d1070 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.173+0000 7f7137e00640 1 -- 172.21.15.19:0/1438975478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124032440 con 0x7f71380d1070 2024-04-25T11:39:52.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.173+0000 7f7137e00640 1 -- 172.21.15.19:0/1438975478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124032810 con 0x7f71380d1070 2024-04-25T11:39:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.174+0000 7f713ee00640 1 -- 172.21.15.19:0/1438975478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 msgr2=0x7f7138162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.174+0000 7f713ee00640 1 --2- 172.21.15.19:0/1438975478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138162200 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f7124009870 tx=0x7f712402a2a0 comp rx=0 tx=0).stop 2024-04-25T11:39:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.174+0000 7f713ee00640 1 -- 172.21.15.19:0/1438975478 shutdown_connections 2024-04-25T11:39:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.174+0000 7f713ee00640 1 --2- 172.21.15.19:0/1438975478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138162200 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.174+0000 7f713ee00640 1 -- 172.21.15.19:0/1438975478 >> 172.21.15.19:0/1438975478 conn(0x7f71380d1720 msgr2=0x7f71380d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.174+0000 7f713ee00640 1 -- 172.21.15.19:0/1438975478 shutdown_connections 2024-04-25T11:39:52.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.174+0000 7f713ee00640 1 -- 172.21.15.19:0/1438975478 wait complete. 2024-04-25T11:39:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.176+0000 7f713ee00640 1 Processor -- start 2024-04-25T11:39:52.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.177+0000 7f713ee00640 1 -- start start 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713ee00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7138144350 0x7f713815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713ee00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7138144e60 0x7f7138160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7138162740 con 0x7f71380d1070 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713ee00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7138147910 con 0x7f7138144e60 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713ee00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71381628c0 con 0x7f7138144350 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:52.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138132750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37822/0 (socket says 172.21.15.19:37822) 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.178+0000 7f713da00640 1 -- 172.21.15.19:0/2376120728 learned_addr learned my addr 172.21.15.19:0/2376120728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713da00640 1 -- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7138144350 msgr2=0x7f713815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713da00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7138144350 0x7f713815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713da00640 1 -- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7138144e60 msgr2=0x7f7138160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713da00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7138144e60 0x7f7138160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713da00640 1 -- 172.21.15.19:0/2376120728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7138132c90 con 0x7f71380d1070 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713da00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138132750 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f712c00b420 tx=0x7f712c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f7135600640 1 -- 172.21.15.19:0/2376120728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c011830 con 0x7f71380d1070 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f7135600640 1 -- 172.21.15.19:0/2376120728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712c016070 con 0x7f71380d1070 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f7135600640 1 -- 172.21.15.19:0/2376120728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c011c10 con 0x7f71380d1070 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71381e8e40 con 0x7f71380d1070 2024-04-25T11:39:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.179+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71381e9400 con 0x7f71380d1070 2024-04-25T11:39:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.181+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7138050740 con 0x7f71380d1070 2024-04-25T11:39:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.182+0000 7f7135600640 1 -- 172.21.15.19:0/2376120728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f712c01b460 con 0x7f71380d1070 2024-04-25T11:39:52.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.183+0000 7f7135600640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71180744e0 0x7f71180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:52.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.183+0000 7f7135600640 1 -- 172.21.15.19:0/2376120728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7f712c095900 con 0x7f71380d1070 2024-04-25T11:39:52.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.184+0000 7f713d000640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71180744e0 0x7f71180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:52.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.185+0000 7f713d000640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71180744e0 0x7f71180769f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7124034000 tx=0x7f712402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.188+0000 7f7135600640 1 -- 172.21.15.19:0/2376120728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f712c05f650 con 0x7f71380d1070 2024-04-25T11:39:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:52.422+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f713812fd00 con 0x7f71380d1070 2024-04-25T11:39:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.067+0000 7f7135600640 1 -- 172.21.15.19:0/2376120728 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v280) ==== 161+0+0 (secure 0 0 0) 0x7f712c05ef50 con 0x7f71380d1070 2024-04-25T11:39:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71180744e0 msgr2=0x7f71180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71180744e0 0x7f71180769f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7124034000 tx=0x7f712402c000 comp rx=0 tx=0).stop 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 msgr2=0x7f7138132750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138132750 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f712c00b420 tx=0x7f712c00b8e0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 shutdown_connections 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71180744e0 0x7f71180769f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7138144e60 0x7f7138160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7138144350 0x7f713815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.071+0000 7f713ee00640 1 --2- 172.21.15.19:0/2376120728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71380d1070 0x7f7138132750 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.072+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 >> 172.21.15.19:0/2376120728 conn(0x7f71380d1720 msgr2=0x7f7138142050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.072+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 shutdown_connections 2024-04-25T11:39:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.072+0000 7f713ee00640 1 -- 172.21.15.19:0/2376120728 wait complete. 2024-04-25T11:39:53.091 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:53.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.387+0000 7fedbae00640 1 Processor -- start 2024-04-25T11:39:53.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.387+0000 7fedbae00640 1 -- start start 2024-04-25T11:39:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.389+0000 7fedbae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fedb406c5c0 0x7fedb40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.389+0000 7fedbae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 0x7fedb40b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.389+0000 7fedbae00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb40b50e0 con 0x7fedb40b4cd0 2024-04-25T11:39:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.389+0000 7fedbae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fedb40d1230 con 0x7fedb406c5c0 2024-04-25T11:39:53.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.389+0000 7fedbae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fedb40d06b0 con 0x7fedb4144370 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.390+0000 7fedba400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fedb40b4cd0 0x7fedb4161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52306/0 (socket says 172.21.15.19:52306) 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.390+0000 7fedba400640 1 -- 172.21.15.19:0/3198778726 learned_addr learned my addr 172.21.15.19:0/3198778726 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.390+0000 7fedb9000640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 0x7fedb40b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.390+0000 7fedb9a00640 1 --2- 172.21.15.19:0/3198778726 >> v2:172.21.15.169:3300/0 conn(0x7fedb406c5c0 0x7fedb40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.390+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511137069 0 0) 0x7fedac001490 con 0x7fedb40b4cd0 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed9c003520 con 0x7fedb40b4cd0 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1538152625 0 0) 0x7fed9c003520 con 0x7fedb40b4cd0 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 msgr2=0x7fedb40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:53.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 0x7fedb40b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 >> v2:172.21.15.169:3300/0 conn(0x7fedb406c5c0 msgr2=0x7fedb40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 --2- 172.21.15.19:0/3198778726 >> v2:172.21.15.169:3300/0 conn(0x7fedb406c5c0 0x7fedb40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb40d0990 con 0x7fedb40b4cd0 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fedac001bc0 con 0x7fedb40b4cd0 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 >> v1:172.21.15.19:6789/0 conn(0x7fedb40b4cd0 legacy=0x7fedb4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c0039b0 0x7fed9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c004800 0x7fed9c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 0x7fed9c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40d1230 con 0x7fed9c0039b0 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40d06b0 con 0x7fed9c004800 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c00b9f0 con 0x7fedb4144370 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.391+0000 7fedb9a00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c0039b0 0x7fed9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.392+0000 7fedba400640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 0x7fed9c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.392+0000 7fedb9a00640 1 -- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 msgr2=0x7fed9c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.392+0000 7fedb9a00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4144370 0x7fed9c005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.392+0000 7fedb9a00640 1 -- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c004800 msgr2=0x7fed9c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.392+0000 7fedb9a00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c004800 0x7fed9c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.392+0000 7fedb9a00640 1 -- 172.21.15.19:0/3198778726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c003520 con 0x7fed9c0039b0 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.393+0000 7fedb9a00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c0039b0 0x7fed9c003e20 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7feda8009ac0 tx=0x7fed9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1ac184a6c57a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.394+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8036070 con 0x7fed9c0039b0 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.394+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda8031400 con 0x7fed9c0039b0 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.394+0000 7fedb3e00640 1 -- 172.21.15.19:0/3198778726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8009690 con 0x7fed9c0039b0 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.394+0000 7fedbae00640 1 -- 172.21.15.19:0/3198778726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c0039b0 msgr2=0x7fed9c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.394+0000 7fedbae00640 1 --2- 172.21.15.19:0/3198778726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed9c0039b0 0x7fed9c003e20 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7feda8009ac0 tx=0x7fed9c005830 comp rx=0 tx=0).stop 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.394+0000 7fedba400640 1 -- 172.21.15.19:0/3198778726 reap_dead start 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.395+0000 7fedbae00640 1 -- 172.21.15.19:0/3198778726 shutdown_connections 2024-04-25T11:39:53.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.395+0000 7fedbae00640 1 -- 172.21.15.19:0/3198778726 >> 172.21.15.19:0/3198778726 conn(0x7fedb40d1720 msgr2=0x7fedb40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.395+0000 7fedbae00640 1 -- 172.21.15.19:0/3198778726 shutdown_connections 2024-04-25T11:39:53.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.395+0000 7fedbae00640 1 -- 172.21.15.19:0/3198778726 wait complete. 2024-04-25T11:39:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.397+0000 7fedbae00640 1 Processor -- start 2024-04-25T11:39:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.397+0000 7fedbae00640 1 -- start start 2024-04-25T11:39:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedbae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c0039b0 0x7fedb4138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedbae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb40c77c0 0x7fedb4160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedbae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4160ea0 0x7fedb40c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedbae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40d02d0 con 0x7fedb40c77c0 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedbae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40d0f50 con 0x7fed9c0039b0 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedbae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fedb40d0450 con 0x7fedb4160ea0 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedb9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb40c77c0 0x7fedb4160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedb9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb40c77c0 0x7fedb4160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37842/0 (socket says 172.21.15.19:37842) 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedb9000640 1 -- 172.21.15.19:0/2326580747 learned_addr learned my addr 172.21.15.19:0/2326580747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedb9a00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c0039b0 0x7fedb4138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.399+0000 7fedba400640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4160ea0 0x7fedb40c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb9000640 1 -- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4160ea0 msgr2=0x7fedb40c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb9000640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4160ea0 0x7fedb40c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb9000640 1 -- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c0039b0 msgr2=0x7fedb4138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb9000640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c0039b0 0x7fedb4138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb9000640 1 -- 172.21.15.19:0/2326580747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb41612b0 con 0x7fedb40c77c0 2024-04-25T11:39:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb9000640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb40c77c0 0x7fedb4160960 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7feda000e7a0 tx=0x7feda000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb1600640 1 -- 172.21.15.19:0/2326580747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0017070 con 0x7fedb40c77c0 2024-04-25T11:39:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb1600640 1 -- 172.21.15.19:0/2326580747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda000c940 con 0x7fedb40c77c0 2024-04-25T11:39:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.400+0000 7fedb1600640 1 -- 172.21.15.19:0/2326580747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda000cd10 con 0x7fedb40c77c0 2024-04-25T11:39:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.401+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb41f7990 con 0x7fedb40c77c0 2024-04-25T11:39:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.401+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb41f7c70 con 0x7fedb40c77c0 2024-04-25T11:39:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.402+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb4050740 con 0x7fedb40c77c0 2024-04-25T11:39:53.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.403+0000 7fedb1600640 1 -- 172.21.15.19:0/2326580747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feda001a070 con 0x7fedb40c77c0 2024-04-25T11:39:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.404+0000 7fedb1600640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed940744e0 0x7fed940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:53.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.404+0000 7fedb1600640 1 -- 172.21.15.19:0/2326580747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7feda0095ba0 con 0x7fedb40c77c0 2024-04-25T11:39:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.405+0000 7fedb9a00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed940744e0 0x7fed940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.406+0000 7fedb9a00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed940744e0 0x7fed940769f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7feda8033000 tx=0x7feda8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:53.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.409+0000 7fedb1600640 1 -- 172.21.15.19:0/2326580747 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedb41612b0 con 0x7fedb40c77c0 2024-04-25T11:39:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:53.695+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fedb40d1eb0 con 0x7fedb40c77c0 2024-04-25T11:39:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.071+0000 7fedb1600640 1 -- 172.21.15.19:0/2326580747 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v281) ==== 101+0+0 (secure 0 0 0) 0x7feda005f860 con 0x7fedb40c77c0 2024-04-25T11:39:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:39:54.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed940744e0 msgr2=0x7fed940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed940744e0 0x7fed940769f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7feda8033000 tx=0x7feda8008030 comp rx=0 tx=0).stop 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb40c77c0 msgr2=0x7fedb4160960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb40c77c0 0x7fedb4160960 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7feda000e7a0 tx=0x7feda000ec60 comp rx=0 tx=0).stop 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 shutdown_connections 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed940744e0 0x7fed940769f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fedb4160ea0 0x7fedb40c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fedb40c77c0 0x7fedb4160960 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 --2- 172.21.15.19:0/2326580747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c0039b0 0x7fedb4138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.075+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 >> 172.21.15.19:0/2326580747 conn(0x7fedb40d1720 msgr2=0x7fedb4146830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.076+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 shutdown_connections 2024-04-25T11:39:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.076+0000 7fedbae00640 1 -- 172.21.15.19:0/2326580747 wait complete. 2024-04-25T11:39:54.100 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:39:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.303+0000 7f1a16b01f40 1 Processor -- start 2024-04-25T11:39:54.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.303+0000 7f1a16b01f40 1 -- start start 2024-04-25T11:39:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.304+0000 7f1a16b01f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfde665f0 0x55ddfdf32ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.304+0000 7f1a16b01f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf3b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.304+0000 7f1a16b01f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 0x55ddfdf3b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.304+0000 7f1a16b01f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ddfdeaee70 con 0x55ddfdf3ffa0 2024-04-25T11:39:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.304+0000 7f1a16b01f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ddfde0cd20 con 0x55ddfde665f0 2024-04-25T11:39:54.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.304+0000 7f1a16b01f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ddfdeaf3f0 con 0x55ddfdf3c290 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf3b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf3b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47728/0 (socket says 172.21.15.19:47728) 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 -- 172.21.15.19:0/1931103265 learned_addr learned my addr 172.21.15.19:0/1931103265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfde665f0 0x55ddfdf32ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a10600640 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 0x55ddfdf3b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 -- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 msgr2=0x55ddfdf3b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 0x55ddfdf3b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 -- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfde665f0 msgr2=0x55ddfdf32ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfde665f0 0x55ddfdf32ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.305+0000 7f1a0f200640 1 -- 172.21.15.19:0/1931103265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddfde988d0 con 0x55ddfdf3ffa0 2024-04-25T11:39:54.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.306+0000 7f1a0f200640 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf3b4a0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006f50 tx=0x7f19fc00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2d77f6ff89661c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.306+0000 7f1a0de00640 1 -- 172.21.15.19:0/1931103265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc012070 con 0x55ddfdf3ffa0 2024-04-25T11:39:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.306+0000 7f1a0de00640 1 -- 172.21.15.19:0/1931103265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc00e670 con 0x55ddfdf3ffa0 2024-04-25T11:39:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.306+0000 7f1a0de00640 1 -- 172.21.15.19:0/1931103265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00ea60 con 0x55ddfdf3ffa0 2024-04-25T11:39:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 -- 172.21.15.19:0/1931103265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfdf3ffa0 msgr2=0x55ddfdf3b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf3b4a0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f19fc006f50 tx=0x7f19fc00b780 comp rx=0 tx=0).stop 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 -- 172.21.15.19:0/1931103265 shutdown_connections 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 0x55ddfdf3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf3b4a0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/1931103265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfde665f0 0x55ddfdf32ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 -- 172.21.15.19:0/1931103265 >> 172.21.15.19:0/1931103265 conn(0x55ddfdd88ac0 msgr2=0x55ddfdf3ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 -- 172.21.15.19:0/1931103265 shutdown_connections 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.307+0000 7f1a16b01f40 1 -- 172.21.15.19:0/1931103265 wait complete. 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.308+0000 7f1a16b01f40 1 Processor -- start 2024-04-25T11:39:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.308+0000 7f1a16b01f40 1 -- start start 2024-04-25T11:39:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a16b01f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfde665f0 0x55ddfde657a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a16b01f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 0x55ddfdf28f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a16b01f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf29470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a16b01f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ddfdeb2550 con 0x55ddfde665f0 2024-04-25T11:39:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a16b01f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ddfde98b70 con 0x55ddfdf3ffa0 2024-04-25T11:39:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a16b01f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ddfde978b0 con 0x55ddfdf3c290 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfde665f0 0x55ddfde657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfde665f0 0x55ddfde657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47736/0 (socket says 172.21.15.19:47736) 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 learned_addr learned my addr 172.21.15.19:0/447908616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 msgr2=0x55ddfdf28f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ddfdf3c290 0x55ddfdf28f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfdf3ffa0 msgr2=0x55ddfdf29470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a10600640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf29470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ddfdf3ffa0 0x55ddfdf29470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.310+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddfdf2a350 con 0x55ddfde665f0 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.311+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfde665f0 0x55ddfde657a0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f1a00045ee0 tx=0x7f1a000033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.311+0000 7f1a0ca00640 1 -- 172.21.15.19:0/447908616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0006a020 con 0x55ddfde665f0 2024-04-25T11:39:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.311+0000 7f1a0ca00640 1 -- 172.21.15.19:0/447908616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a000539b0 con 0x55ddfde665f0 2024-04-25T11:39:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.311+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ddfdf33c60 con 0x55ddfde665f0 2024-04-25T11:39:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.311+0000 7f1a0ca00640 1 -- 172.21.15.19:0/447908616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00053d80 con 0x55ddfde665f0 2024-04-25T11:39:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.311+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddfdf34220 con 0x55ddfde665f0 2024-04-25T11:39:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.314+0000 7f1a0ca00640 1 -- 172.21.15.19:0/447908616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a0006d070 con 0x55ddfde665f0 2024-04-25T11:39:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.314+0000 7f1a0ca00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19f0072ec0 0x7f19f00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.315+0000 7f1a0ca00640 1 -- 172.21.15.19:0/447908616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7f1a000d7d80 con 0x55ddfde665f0 2024-04-25T11:39:54.302 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:39:54.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.315+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ddfde65ce0 0x55ddfdf32fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.315+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdf38050 con 0x55ddfde65ce0 2024-04-25T11:39:54.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.315+0000 7f1a0f200640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19f0072ec0 0x7f19f00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.316+0000 7f1a10600640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ddfde65ce0 0x55ddfdf32fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.317+0000 7f1a0f200640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19f0072ec0 0x7f19f00753d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55ddfdf3c8c0 tx=0x7f19fc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.317+0000 7f1a10600640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ddfde65ce0 0x55ddfdf32fc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.321+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v122'908 uv882 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.321+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdf33500 con 0x55ddfde65ce0 2024-04-25T11:39:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.324+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v122'909 uv909 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.324+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfdffb0 con 0x55ddfde65ce0 2024-04-25T11:39:54.314 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:39:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.328+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v122'910 uv910 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.328+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe1360 con 0x55ddfde65ce0 2024-04-25T11:39:54.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.331+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'911 uv911 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.331+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe1700 con 0x55ddfde65ce0 2024-04-25T11:39:54.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.335+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'912 uv912 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.335+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0350 con 0x55ddfde65ce0 2024-04-25T11:39:54.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.338+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'913 uv913 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.339+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0740 con 0x55ddfde65ce0 2024-04-25T11:39:54.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.343+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v122'914 uv888 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.343+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe0b30 con 0x55ddfde65ce0 2024-04-25T11:39:54.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.346+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v122'915 uv915 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.347+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0b30 con 0x55ddfde65ce0 2024-04-25T11:39:54.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.350+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v122'916 uv916 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.337 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:39:54.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.350+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.354+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'917 uv917 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.354+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.358+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'918 uv918 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.358+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.362+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'919 uv919 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.362+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.367+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v122'920 uv895 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.367+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.372+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'921 uv921 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.372+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.376+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v122'922 uv922 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.363 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:39:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.376+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.381+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'923 uv923 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.381+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.385+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'924 uv924 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.385+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.390+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'925 uv925 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.390+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.395+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'926 uv926 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.395+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.399+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v122'927 uv901 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.400+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.404+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v122'928 uv928 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.404+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.396 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:39:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.409+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v122'929 uv929 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.409+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.414+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'930 uv930 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.414+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.418+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'931 uv931 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.419+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.423+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'932 uv932 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.424+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.428+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v122'933 uv907 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.429+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.433+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v122'934 uv934 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.433+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.425 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:39:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.438+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v122'935 uv935 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.438+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.444+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'936 uv936 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.444+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.449+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'937 uv937 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.450+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe0ae0 con 0x55ddfde65ce0 2024-04-25T11:39:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.454+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'938 uv938 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.455+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ddfdfe0ef0 0x55ddfdfe3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.455+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe4500 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.456+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ddfdfe0ef0 0x55ddfdfe3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.458+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ddfdfe0ef0 0x55ddfdfe3fc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.463+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'659 uv642 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.464+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe48a0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.468+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'660 uv660 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.468+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe4c40 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.473+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'661 uv661 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.460 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:39:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.473+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe4fe0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.478+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v122'662 uv662 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.478+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe53d0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.482+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'663 uv663 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.482+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5770 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.487+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v122'664 uv647 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.487+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe5b10 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.492+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v122'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.492+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.496+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v122'666 uv666 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.483 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:39:54.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.496+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.500+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.500+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.504+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.505+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.509+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v122'669 uv652 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.509+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.514+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v122'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.514+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.504 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:39:54.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.518+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v122'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.518+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfea5a0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.523+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v122'672 uv672 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.524+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfea5a0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.528+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'673 uv673 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.528+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.533+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v122'674 uv658 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.533+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.537+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v122'675 uv675 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.537+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.528 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:39:54.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.542+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v122'676 uv676 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.542+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.546+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'677 uv677 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.546+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe6ec0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.550+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'678 uv678 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.550+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe5eb0 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.555+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'679 uv679 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfe0ef0 2024-04-25T11:39:54.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.555+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ddfdfe6530 0x55ddfdfe6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.555+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe9320 con 0x55ddfdfe6530 2024-04-25T11:39:54.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.556+0000 7f1a0f200640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ddfdfe6530 0x55ddfdfe6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.558+0000 7f1a0f200640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ddfdfe6530 0x55ddfdfe6980 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.562+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.562+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfe96c0 con 0x55ddfdfe6530 2024-04-25T11:39:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.567+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v122'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.567+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfe9aa0 con 0x55ddfdfe6530 2024-04-25T11:39:54.558 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:39:54.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.571+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v122'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.571+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdf29bd0 con 0x55ddfdfe6530 2024-04-25T11:39:54.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.575+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.575+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfeb430 con 0x55ddfdfe6530 2024-04-25T11:39:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.579+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.579+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfea340 con 0x55ddfdfe6530 2024-04-25T11:39:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.584+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.584+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfea760 con 0x55ddfde65ce0 2024-04-25T11:39:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.589+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'132 uv131 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.589+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfea760 con 0x55ddfde65ce0 2024-04-25T11:39:54.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.593+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.593+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfea730 con 0x55ddfde65ce0 2024-04-25T11:39:54.585 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:39:54.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.598+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'134 uv134 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.598+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfea790 con 0x55ddfde65ce0 2024-04-25T11:39:54.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.603+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'135 uv135 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.603+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ddfdfeb820 0x55ddfdfedc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.603+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfee180 con 0x55ddfdfeb820 2024-04-25T11:39:54.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.604+0000 7f1a10600640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ddfdfeb820 0x55ddfdfedc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.606+0000 7f1a10600640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ddfdfeb820 0x55ddfdfedc40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.610+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.611+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdfee180 con 0x55ddfdfeb820 2024-04-25T11:39:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.615+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v122'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.615+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfee180 con 0x55ddfdfeb820 2024-04-25T11:39:54.606 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:39:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.619+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v122'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.620+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfef190 con 0x55ddfdfeb820 2024-04-25T11:39:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.624+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.624+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfef190 con 0x55ddfdfeb820 2024-04-25T11:39:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.628+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.628+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdfee180 con 0x55ddfdfeb820 2024-04-25T11:39:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.632+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.633+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ddfdfeea80 0x55ddfdff0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.633+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff13e0 con 0x55ddfdfeea80 2024-04-25T11:39:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.633+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ddfdfeea80 0x55ddfdff0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.635+0000 7f1a0fc00640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ddfdfeea80 0x55ddfdff0ea0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.640+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v122'327 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.640+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff1460 con 0x55ddfdfeea80 2024-04-25T11:39:54.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.645+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v122'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.645+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff1850 con 0x55ddfdfeea80 2024-04-25T11:39:54.635 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:39:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.649+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v122'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.649+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff2860 con 0x55ddfdfeea80 2024-04-25T11:39:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.653+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.653+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff2860 con 0x55ddfdfeea80 2024-04-25T11:39:54.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.657+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.657+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff1850 con 0x55ddfdfeea80 2024-04-25T11:39:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.661+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.661+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff1850 con 0x55ddfdfeea80 2024-04-25T11:39:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.665+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v122'333 uv326 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.665+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff1850 con 0x55ddfdfeea80 2024-04-25T11:39:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.670+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v122'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.670+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff1850 con 0x55ddfdfeea80 2024-04-25T11:39:54.661 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:39:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.674+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v122'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.674+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff1850 con 0x55ddfdfeea80 2024-04-25T11:39:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.678+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.678+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff1850 con 0x55ddfdfeea80 2024-04-25T11:39:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.682+0000 7f1a0fc00640 1 -- 172.21.15.19:0/447908616 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a0006c030 con 0x55ddfdfeea80 2024-04-25T11:39:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.683+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ddfdff1ed0 0x55ddfdff42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.683+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff48c0 con 0x55ddfdff1ed0 2024-04-25T11:39:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.683+0000 7f1a0f200640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ddfdff1ed0 0x55ddfdff42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:54.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.685+0000 7f1a0f200640 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ddfdff1ed0 0x55ddfdff42f0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.689+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v122'148 uv147 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.689+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff4c90 con 0x55ddfdff1ed0 2024-04-25T11:39:54.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.693+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v122'149 uv149 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.693+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5190 con 0x55ddfdff1ed0 2024-04-25T11:39:54.684 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:39:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.697+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v122'150 uv150 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.697+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5580 con 0x55ddfdff1ed0 2024-04-25T11:39:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.702+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'151 uv151 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.702+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5580 con 0x55ddfdff1ed0 2024-04-25T11:39:54.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.706+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'152 uv152 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.706+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff53b0 con 0x55ddfde65ce0 2024-04-25T11:39:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.710+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.711+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff53b0 con 0x55ddfde65ce0 2024-04-25T11:39:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.715+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v122'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.715+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff53b0 con 0x55ddfde65ce0 2024-04-25T11:39:54.706 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:39:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.720+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v122'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.720+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff6420 con 0x55ddfde65ce0 2024-04-25T11:39:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.724+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.724+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff63e0 con 0x55ddfde65ce0 2024-04-25T11:39:54.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.728+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.728+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff63e0 con 0x55ddfde65ce0 2024-04-25T11:39:54.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.733+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfde65ce0 2024-04-25T11:39:54.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.733+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff63e0 con 0x55ddfdfe6530 2024-04-25T11:39:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.737+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v122'294 uv288 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.737+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff63e0 con 0x55ddfdfe6530 2024-04-25T11:39:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.742+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v122'295 uv295 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.742+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff63e0 con 0x55ddfdfe6530 2024-04-25T11:39:54.733 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:39:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.746+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v122'296 uv296 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.746+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5310 con 0x55ddfdfe6530 2024-04-25T11:39:54.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.751+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'297 uv297 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.751+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff6480 con 0x55ddfdfe6530 2024-04-25T11:39:54.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.756+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'298 uv298 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.756+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff56d0 con 0x55ddfdfe6530 2024-04-25T11:39:54.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.760+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v122'299 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.760+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff53a0 con 0x55ddfdfe6530 2024-04-25T11:39:54.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.765+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v122'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.765+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5350 con 0x55ddfdfe6530 2024-04-25T11:39:54.756 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:39:54.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.769+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v122'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.769+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5310 con 0x55ddfdfe6530 2024-04-25T11:39:54.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.773+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.774+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff57f0 con 0x55ddfdfe6530 2024-04-25T11:39:54.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.779+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdfe6530 2024-04-25T11:39:54.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.779+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.783+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v122'128 uv127 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.783+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.787+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'129 uv129 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.787+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.778 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:39:54.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.791+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v122'130 uv130 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.792+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.795+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'131 uv131 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.796+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.799+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v122'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.800+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.803+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v122'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.804+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.795 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:39:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.808+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v122'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.808+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.812+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.812+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.816+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.816+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdfeb820 2024-04-25T11:39:54.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.821+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a08009790 con 0x55ddfdfeb820 2024-04-25T11:39:54.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.821+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdff1ed0 2024-04-25T11:39:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.825+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v122'151 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.825+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdff1ed0 2024-04-25T11:39:54.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.830+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v122'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.830+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5690 con 0x55ddfdff1ed0 2024-04-25T11:39:54.821 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:39:54.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.834+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v122'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.834+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5690 con 0x55ddfdff1ed0 2024-04-25T11:39:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.838+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.839+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5690 con 0x55ddfdff1ed0 2024-04-25T11:39:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.843+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.843+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5690 con 0x55ddfdff1ed0 2024-04-25T11:39:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.847+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v122'363 uv355 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.847+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff5690 con 0x55ddfdff1ed0 2024-04-25T11:39:54.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.852+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v122'364 uv364 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.852+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5670 con 0x55ddfdff1ed0 2024-04-25T11:39:54.843 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:39:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.857+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v122'365 uv365 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.857+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.861+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v122'366 uv366 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.861+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.865+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'367 uv367 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.865+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.870+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v122'368 uv362 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.870+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.874+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v122'369 uv369 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.874+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.878+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v122'370 uv370 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.865 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:39:54.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.878+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.882+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'371 uv371 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.882+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.886+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'372 uv372 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.886+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5640 con 0x55ddfdff1ed0 2024-04-25T11:39:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.890+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'373 uv373 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.890+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55ddfdff5eb0 con 0x55ddfdff1ed0 2024-04-25T11:39:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a0f200640 1 -- 172.21.15.19:0/447908616 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'374 uv374 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f19fc020030 con 0x55ddfdff1ed0 2024-04-25T11:39:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ddfdfe0ef0 msgr2=0x55ddfdfe3fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ddfdfe0ef0 0x55ddfdfe3fc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ddfdfeea80 msgr2=0x55ddfdff0ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ddfdfeea80 0x55ddfdff0ea0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ddfdff1ed0 msgr2=0x55ddfdff42f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ddfdff1ed0 0x55ddfdff42f0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ddfdfe6530 msgr2=0x55ddfdfe6980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ddfdfe6530 0x55ddfdfe6980 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ddfde65ce0 msgr2=0x55ddfdf32fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ddfde65ce0 0x55ddfdf32fc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ddfdfeb820 msgr2=0x55ddfdfedc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ddfdfeb820 0x55ddfdfedc40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 reap_dead start 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.895+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 reap_dead start 2024-04-25T11:39:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a10600640 1 -- 172.21.15.19:0/447908616 reap_dead start 2024-04-25T11:39:54.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19f0072ec0 msgr2=0x7f19f00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19f0072ec0 0x7f19f00753d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55ddfdf3c8c0 tx=0x7f19fc009000 comp rx=0 tx=0).stop 2024-04-25T11:39:54.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfde665f0 msgr2=0x55ddfde657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:54.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfde665f0 0x55ddfde657a0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f1a00045ee0 tx=0x7f1a000033a0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 shutdown_connections 2024-04-25T11:39:54.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f19f0072ec0 0x7f19f00753d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 --2- 172.21.15.19:0/447908616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ddfde665f0 0x55ddfde657a0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:54.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.896+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 >> 172.21.15.19:0/447908616 conn(0x55ddfdd88ac0 msgr2=0x55ddfdf3ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:54.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.897+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 shutdown_connections 2024-04-25T11:39:54.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:54.897+0000 7f1a16b01f40 1 -- 172.21.15.19:0/447908616 wait complete. 2024-04-25T11:39:54.910 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2024-04-25T11:39:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.156+0000 7fa790a0bf40 1 Processor -- start 2024-04-25T11:39:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.156+0000 7fa790a0bf40 1 -- start start 2024-04-25T11:39:55.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa790a0bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4db0b91e0 0x55a4db183440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa790a0bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db0b95f0 0x55a4db18c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa790a0bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db192dd0 0x55a4db0e9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa790a0bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4db1023f0 con 0x55a4db192dd0 2024-04-25T11:39:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa790a0bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a4db101e70 con 0x55a4db0b91e0 2024-04-25T11:39:55.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa790a0bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a4db1026d0 con 0x55a4db0b95f0 2024-04-25T11:39:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa789800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db192dd0 0x55a4db0e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa789800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db192dd0 0x55a4db0e9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47756/0 (socket says 172.21.15.19:47756) 2024-04-25T11:39:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.158+0000 7fa789800640 1 -- 172.21.15.19:0/351937408 learned_addr learned my addr 172.21.15.19:0/351937408 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa789800640 1 -- 172.21.15.19:0/351937408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db0b95f0 msgr2=0x55a4db18c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa789800640 1 --2- 172.21.15.19:0/351937408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db0b95f0 0x55a4db18c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa789800640 1 -- 172.21.15.19:0/351937408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4db0b91e0 msgr2=0x55a4db183440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa789800640 1 --2- 172.21.15.19:0/351937408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4db0b91e0 0x55a4db183440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa789800640 1 -- 172.21.15.19:0/351937408 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4dafe2b30 con 0x55a4db192dd0 2024-04-25T11:39:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa789800640 1 --2- 172.21.15.19:0/351937408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db192dd0 0x55a4db0e9000 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fa780009490 tx=0x7fa780009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a49a8766abc063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa787000640 1 -- 172.21.15.19:0/351937408 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780006440 con 0x55a4db192dd0 2024-04-25T11:39:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa787000640 1 -- 172.21.15.19:0/351937408 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7800065e0 con 0x55a4db192dd0 2024-04-25T11:39:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.159+0000 7fa787000640 1 -- 172.21.15.19:0/351937408 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780006a10 con 0x55a4db192dd0 2024-04-25T11:39:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 -- 172.21.15.19:0/351937408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db192dd0 msgr2=0x55a4db0e9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/351937408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db192dd0 0x55a4db0e9000 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fa780009490 tx=0x7fa780009890 comp rx=0 tx=0).stop 2024-04-25T11:39:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 -- 172.21.15.19:0/351937408 shutdown_connections 2024-04-25T11:39:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/351937408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db192dd0 0x55a4db0e9000 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/351937408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db0b95f0 0x55a4db18c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/351937408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4db0b91e0 0x55a4db183440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 -- 172.21.15.19:0/351937408 >> 172.21.15.19:0/351937408 conn(0x55a4dafdbac0 msgr2=0x55a4db0e7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 -- 172.21.15.19:0/351937408 shutdown_connections 2024-04-25T11:39:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.160+0000 7fa790a0bf40 1 -- 172.21.15.19:0/351937408 wait complete. 2024-04-25T11:39:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.161+0000 7fa790a0bf40 1 Processor -- start 2024-04-25T11:39:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.162+0000 7fa790a0bf40 1 -- start start 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa790a0bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db0b91e0 0x55a4db183a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa790a0bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4db0b95f0 0x55a4db183f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa790a0bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db192dd0 0x55a4db180990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa790a0bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4db1043e0 con 0x55a4db0b91e0 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa790a0bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a4db18f3b0 con 0x55a4db0b95f0 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa790a0bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a4db227a40 con 0x55a4db192dd0 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa788e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db0b91e0 0x55a4db183a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa788e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db0b91e0 0x55a4db183a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47768/0 (socket says 172.21.15.19:47768) 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.163+0000 7fa788e00640 1 -- 172.21.15.19:0/2261545568 learned_addr learned my addr 172.21.15.19:0/2261545568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa789800640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db192dd0 0x55a4db180990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa788e00640 1 -- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db192dd0 msgr2=0x55a4db180990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa788e00640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4db192dd0 0x55a4db180990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa788e00640 1 -- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4db0b95f0 msgr2=0x55a4db183f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa788e00640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4db0b95f0 0x55a4db183f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa788e00640 1 -- 172.21.15.19:0/2261545568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4db226920 con 0x55a4db0b91e0 2024-04-25T11:39:55.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa788e00640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db0b91e0 0x55a4db183a10 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fa77800a190 tx=0x7fa778045fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:55.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.164+0000 7fa785c00640 1 -- 172.21.15.19:0/2261545568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778055040 con 0x55a4db0b91e0 2024-04-25T11:39:55.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.165+0000 7fa785c00640 1 -- 172.21.15.19:0/2261545568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77805b430 con 0x55a4db0b91e0 2024-04-25T11:39:55.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.165+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4db226c00 con 0x55a4db0b91e0 2024-04-25T11:39:55.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.165+0000 7fa785c00640 1 -- 172.21.15.19:0/2261545568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77805b840 con 0x55a4db0b91e0 2024-04-25T11:39:55.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.165+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4db2271c0 con 0x55a4db0b91e0 2024-04-25T11:39:55.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.167+0000 7fa785c00640 1 -- 172.21.15.19:0/2261545568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa778058020 con 0x55a4db0b91e0 2024-04-25T11:39:55.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.168+0000 7fa785c00640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa768072e90 0x7fa7680753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.168+0000 7fa785c00640 1 -- 172.21.15.19:0/2261545568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7fa7780d8950 con 0x55a4db0b91e0 2024-04-25T11:39:55.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.169+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a4db177fb0 0x55a4db1813e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.169+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x55a4db185c00 con 0x55a4db177fb0 2024-04-25T11:39:55.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.169+0000 7fa788400640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa768072e90 0x7fa7680753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.169+0000 7fa789800640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a4db177fb0 0x55a4db1813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.169+0000 7fa789800640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a4db177fb0 0x55a4db1813e0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:55.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.170+0000 7fa788400640 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa768072e90 0x7fa7680753a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55a4db178520 tx=0x7fa774005590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa789800640 1 -- 172.21.15.19:0/2261545568 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v122'375 uv375 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa780006790 con 0x55a4db177fb0 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a4db177fb0 msgr2=0x55a4db1813e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a4db177fb0 0x55a4db1813e0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa768072e90 msgr2=0x7fa7680753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa768072e90 0x7fa7680753a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55a4db178520 tx=0x7fa774005590 comp rx=0 tx=0).stop 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db0b91e0 msgr2=0x55a4db183a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 --2- 172.21.15.19:0/2261545568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4db0b91e0 0x55a4db183a10 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fa77800a190 tx=0x7fa778045fb0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa789800640 1 -- 172.21.15.19:0/2261545568 reap_dead start 2024-04-25T11:39:55.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 shutdown_connections 2024-04-25T11:39:55.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 >> 172.21.15.19:0/2261545568 conn(0x55a4dafdbac0 msgr2=0x55a4db0e7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:55.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 shutdown_connections 2024-04-25T11:39:55.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.173+0000 7fa790a0bf40 1 -- 172.21.15.19:0/2261545568 wait complete. 2024-04-25T11:39:55.172 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:39:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.470+0000 7f75b5a00640 1 Processor -- start 2024-04-25T11:39:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.471+0000 7f75b5a00640 1 -- start start 2024-04-25T11:39:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.473+0000 7f75b5a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75b00b6900 0x7f75b00b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.473+0000 7f75b5a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b00ce670 con 0x7f75b00d20c0 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.473+0000 7f75b5a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b008d9a0 con 0x7f75b00b4e70 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.473+0000 7f75b5a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f75b008d770 con 0x7f75b00b6900 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.473+0000 7f75aea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75b00b6900 0x7f75b00b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.474+0000 7f75aea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75b00b6900 0x7f75b00b8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47782/0 (socket says 172.21.15.19:47782) 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.474+0000 7f75aea00640 1 -- 172.21.15.19:0/1894002557 learned_addr learned my addr 172.21.15.19:0/1894002557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.474+0000 7f75aea00640 1 -- 172.21.15.19:0/1894002557 >> v1:172.21.15.169:6789/0 conn(0x7f75b00b4e70 legacy=0x7f75b00d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.474+0000 7f75aea00640 1 -- 172.21.15.19:0/1894002557 >> v1:172.21.15.19:6789/0 conn(0x7f75b00d20c0 legacy=0x7f75b0161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.474+0000 7f75aea00640 1 -- 172.21.15.19:0/1894002557 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b008dc60 con 0x7f75b00b6900 2024-04-25T11:39:55.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.475+0000 7f75aea00640 1 --2- 172.21.15.19:0/1894002557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b6900 0x7f75b00b8ab0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c8bebd6fa2c65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:55.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.475+0000 7f75ad600640 1 -- 172.21.15.19:0/1894002557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0037070 con 0x7f75b00b6900 2024-04-25T11:39:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.476+0000 7f75ad600640 1 -- 172.21.15.19:0/1894002557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a0032440 con 0x7f75b00b6900 2024-04-25T11:39:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.476+0000 7f75ad600640 1 -- 172.21.15.19:0/1894002557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0032810 con 0x7f75b00b6900 2024-04-25T11:39:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.476+0000 7f75b5a00640 1 -- 172.21.15.19:0/1894002557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b6900 msgr2=0x7f75b00b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.476+0000 7f75b5a00640 1 --2- 172.21.15.19:0/1894002557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b6900 0x7f75b00b8ab0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a002a2a0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.477+0000 7f75b5a00640 1 -- 172.21.15.19:0/1894002557 shutdown_connections 2024-04-25T11:39:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.477+0000 7f75b5a00640 1 --2- 172.21.15.19:0/1894002557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b6900 0x7f75b00b8ab0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.477+0000 7f75b5a00640 1 -- 172.21.15.19:0/1894002557 >> 172.21.15.19:0/1894002557 conn(0x7f75b00d0530 msgr2=0x7f75b00d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.477+0000 7f75b5a00640 1 -- 172.21.15.19:0/1894002557 shutdown_connections 2024-04-25T11:39:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.477+0000 7f75b5a00640 1 -- 172.21.15.19:0/1894002557 wait complete. 2024-04-25T11:39:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.479+0000 7f75b5a00640 1 Processor -- start 2024-04-25T11:39:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.479+0000 7f75b5a00640 1 -- start start 2024-04-25T11:39:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75b5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b4e70 0x7f75b01fa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75b5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75b00b6900 0x7f75b006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75b5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75b00d20c0 0x7f75b00ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75b5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00b5280 con 0x7f75b00b4e70 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75b5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00ce980 con 0x7f75b00d20c0 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75b5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f75b00b5400 con 0x7f75b00b6900 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75aea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b4e70 0x7f75b01fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75aea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b4e70 0x7f75b01fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47788/0 (socket says 172.21.15.19:47788) 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.481+0000 7f75aea00640 1 -- 172.21.15.19:0/4265164429 learned_addr learned my addr 172.21.15.19:0/4265164429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75ae000640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75b00b6900 0x7f75b006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75aea00640 1 -- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75b00b6900 msgr2=0x7f75b006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75aea00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75b00b6900 0x7f75b006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75aea00640 1 -- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75b00d20c0 msgr2=0x7f75b00ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75aea00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75b00d20c0 0x7f75b00ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75aea00640 1 -- 172.21.15.19:0/4265164429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b006e6a0 con 0x7f75b00b4e70 2024-04-25T11:39:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75ae000640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75b00b6900 0x7f75b006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75aea00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b4e70 0x7f75b01fa760 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75a6a00640 1 -- 172.21.15.19:0/4265164429 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0041070 con 0x7f75b00b4e70 2024-04-25T11:39:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75a6a00640 1 -- 172.21.15.19:0/4265164429 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a003c870 con 0x7f75b00b4e70 2024-04-25T11:39:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.482+0000 7f75a6a00640 1 -- 172.21.15.19:0/4265164429 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0037040 con 0x7f75b00b4e70 2024-04-25T11:39:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.483+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b006e980 con 0x7f75b00b4e70 2024-04-25T11:39:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.483+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b006ef40 con 0x7f75b00b4e70 2024-04-25T11:39:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.484+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b0050740 con 0x7f75b00b4e70 2024-04-25T11:39:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.485+0000 7f75a6a00640 1 -- 172.21.15.19:0/4265164429 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f75a0044070 con 0x7f75b00b4e70 2024-04-25T11:39:55.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.486+0000 7f75a6a00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f758c0744b0 0x7f758c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:55.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.486+0000 7f75a6a00640 1 -- 172.21.15.19:0/4265164429 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8289+0+0 (secure 0 0 0) 0x7f75a00b6990 con 0x7f75b00b4e70 2024-04-25T11:39:55.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.487+0000 7f75ae000640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f758c0744b0 0x7f758c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.488+0000 7f75ae000640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f758c0744b0 0x7f758c0769c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f75b006f410 tx=0x7f75980073c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.492+0000 7f75a6a00640 1 -- 172.21.15.19:0/4265164429 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75a0080650 con 0x7f75b00b4e70 2024-04-25T11:39:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:55.738+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f75b00687a0 con 0x7f75b00b4e70 2024-04-25T11:39:56.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.112+0000 7f75a6a00640 1 -- 172.21.15.19:0/4265164429 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v282) ==== 162+0+0 (secure 0 0 0) 0x7f75a007ff50 con 0x7f75b00b4e70 2024-04-25T11:39:56.099 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:39:56.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f758c0744b0 msgr2=0x7f758c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f758c0744b0 0x7f758c0769c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f75b006f410 tx=0x7f75980073c0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b4e70 msgr2=0x7f75b01fa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b4e70 0x7f75b01fa760 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a00046c0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 shutdown_connections 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f758c0744b0 0x7f758c0769c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75b00d20c0 0x7f75b00ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75b00b6900 0x7f75b006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 --2- 172.21.15.19:0/4265164429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00b4e70 0x7f75b01fa760 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.115+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 >> 172.21.15.19:0/4265164429 conn(0x7f75b00d0530 msgr2=0x7f75b00728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.116+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 shutdown_connections 2024-04-25T11:39:56.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.116+0000 7f75b5a00640 1 -- 172.21.15.19:0/4265164429 wait complete. 2024-04-25T11:39:56.124 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:56.137+0000 7f453a600640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-25T11:39:56.124 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:39:56.137+0000 7f453a600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:39:56.124 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:56.138+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:39:56.125 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:56.138+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:39:56.138 INFO:tasks.ceph.mds.a.smithi019.stderr: -16> 2024-04-25T11:39:56.137+0000 7f453a600640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-04-25T11:39:56.138 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:39:56.137+0000 7f453a600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-04-25T11:39:56.140 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:56.273 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:39:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.409+0000 7febec800640 1 Processor -- start 2024-04-25T11:39:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.409+0000 7febec800640 1 -- start start 2024-04-25T11:39:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febec800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7febe40cf5e0 0x7febe40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febec800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febe406c5c0 0x7febe406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febec800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe40cff30 con 0x7febe41443c0 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febec800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7febe40708d0 con 0x7febe406c5c0 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febec800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7febe415f160 con 0x7febe40cf5e0 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febea800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febe406c5c0 0x7febe406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febea800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7febe406c5c0 0x7febe406c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47810/0 (socket says 172.21.15.19:47810) 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febea800640 1 -- 172.21.15.19:0/1562033841 learned_addr learned my addr 172.21.15.19:0/1562033841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.412+0000 7febe9400640 1 --2- 172.21.15.19:0/1562033841 >> v2:172.21.15.169:3300/0 conn(0x7febe40cf5e0 0x7febe40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.413+0000 7febea800640 1 -- 172.21.15.19:0/1562033841 >> v2:172.21.15.169:3300/0 conn(0x7febe40cf5e0 msgr2=0x7febe40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.413+0000 7febea800640 1 --2- 172.21.15.19:0/1562033841 >> v2:172.21.15.169:3300/0 conn(0x7febe40cf5e0 0x7febe40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.413+0000 7febea800640 1 -- 172.21.15.19:0/1562033841 >> v1:172.21.15.19:6789/0 conn(0x7febe41443c0 legacy=0x7febe406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.413+0000 7febea800640 1 -- 172.21.15.19:0/1562033841 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe415f5f0 con 0x7febe406c5c0 2024-04-25T11:39:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.413+0000 7febe8a00640 1 -- 172.21.15.19:0/1562033841 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979935599 0 0) 0x7febd8001110 con 0x7febe41443c0 2024-04-25T11:39:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.414+0000 7febea800640 1 --2- 172.21.15.19:0/1562033841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe406c5c0 0x7febe406c9d0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7febe406f8f0 tx=0x7febe002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990d8573a9355812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.414+0000 7febe8a00640 1 -- 172.21.15.19:0/1562033841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0036070 con 0x7febe406c5c0 2024-04-25T11:39:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.414+0000 7febe8a00640 1 -- 172.21.15.19:0/1562033841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe0031600 con 0x7febe406c5c0 2024-04-25T11:39:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.414+0000 7febe8a00640 1 -- 172.21.15.19:0/1562033841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00319d0 con 0x7febe406c5c0 2024-04-25T11:39:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 -- 172.21.15.19:0/1562033841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe406c5c0 msgr2=0x7febe406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 --2- 172.21.15.19:0/1562033841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe406c5c0 0x7febe406c9d0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7febe406f8f0 tx=0x7febe002a730 comp rx=0 tx=0).stop 2024-04-25T11:39:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 -- 172.21.15.19:0/1562033841 shutdown_connections 2024-04-25T11:39:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 --2- 172.21.15.19:0/1562033841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe406c5c0 0x7febe406c9d0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 --2- 172.21.15.19:0/1562033841 >> v2:172.21.15.169:3300/0 conn(0x7febe40cf5e0 0x7febe40cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 -- 172.21.15.19:0/1562033841 >> 172.21.15.19:0/1562033841 conn(0x7febe40d1ae0 msgr2=0x7febe40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 -- 172.21.15.19:0/1562033841 shutdown_connections 2024-04-25T11:39:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.415+0000 7febec800640 1 -- 172.21.15.19:0/1562033841 wait complete. 2024-04-25T11:39:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.417+0000 7febec800640 1 Processor -- start 2024-04-25T11:39:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.418+0000 7febec800640 1 -- start start 2024-04-25T11:39:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febec800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febe406c5c0 0x7febe41f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febec800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe40cf5e0 0x7febe41fa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febec800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe41443c0 0x7febe40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febec800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febe415edf0 con 0x7febe40cf5e0 2024-04-25T11:39:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febec800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7febe415eb10 con 0x7febe41443c0 2024-04-25T11:39:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febec800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7febe41447d0 con 0x7febe406c5c0 2024-04-25T11:39:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febe9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe40cf5e0 0x7febe41fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febe9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe40cf5e0 0x7febe41fa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47814/0 (socket says 172.21.15.19:47814) 2024-04-25T11:39:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.419+0000 7febe9400640 1 -- 172.21.15.19:0/318590185 learned_addr learned my addr 172.21.15.19:0/318590185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febea800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe41443c0 0x7febe40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febe9400640 1 -- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febe406c5c0 msgr2=0x7febe41f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febe9e00640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febe406c5c0 0x7febe41f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febe9400640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febe406c5c0 0x7febe41f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febe9400640 1 -- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe41443c0 msgr2=0x7febe40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febe9400640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe41443c0 0x7febe40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febe9400640 1 -- 172.21.15.19:0/318590185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe415d5b0 con 0x7febe40cf5e0 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febea800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe41443c0 0x7febe40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.420+0000 7febe9400640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe40cf5e0 0x7febe41fa660 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7febd400d5c0 tx=0x7febd400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.421+0000 7febde000640 1 -- 172.21.15.19:0/318590185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40125f0 con 0x7febe40cf5e0 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.421+0000 7febde000640 1 -- 172.21.15.19:0/318590185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd4019070 con 0x7febe40cf5e0 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.421+0000 7febde000640 1 -- 172.21.15.19:0/318590185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40129d0 con 0x7febe40cf5e0 2024-04-25T11:39:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.421+0000 7febec800640 1 -- 172.21.15.19:0/318590185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe415ca30 con 0x7febe40cf5e0 2024-04-25T11:39:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.421+0000 7febec800640 1 -- 172.21.15.19:0/318590185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe415cff0 con 0x7febe40cf5e0 2024-04-25T11:39:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.422+0000 7febec800640 1 -- 172.21.15.19:0/318590185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe415c7a0 con 0x7febe40cf5e0 2024-04-25T11:39:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.423+0000 7febde000640 1 -- 172.21.15.19:0/318590185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febd4012b70 con 0x7febe40cf5e0 2024-04-25T11:39:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.424+0000 7febde000640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febc80744b0 0x7febc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.424+0000 7febde000640 1 -- 172.21.15.19:0/318590185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7febd4096120 con 0x7febe40cf5e0 2024-04-25T11:39:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.425+0000 7febe9e00640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febc80744b0 0x7febc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.426+0000 7febe9e00640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febc80744b0 0x7febc80769c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7febd8000f60 tx=0x7febd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.429+0000 7febde000640 1 -- 172.21.15.19:0/318590185 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febd405fdf0 con 0x7febe40cf5e0 2024-04-25T11:39:56.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.674+0000 7febec800640 1 -- 172.21.15.19:0/318590185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febe4050740 con 0x7febe40cf5e0 2024-04-25T11:39:56.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.675+0000 7febde000640 1 -- 172.21.15.19:0/318590185 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 284 v284) ==== 95+0+4352 (secure 0 0 0) 0x7febd4066e20 con 0x7febe40cf5e0 2024-04-25T11:39:56.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:56.663 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":284,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":284,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:56.138866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:56.663 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 284 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 -- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febc80744b0 msgr2=0x7febc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febc80744b0 0x7febc80769c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7febd8000f60 tx=0x7febd8009040 comp rx=0 tx=0).stop 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 -- 172.21.15.19:0/318590185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe40cf5e0 msgr2=0x7febe41fa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe40cf5e0 0x7febe41fa660 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7febd400d5c0 tx=0x7febd400da80 comp rx=0 tx=0).stop 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 -- 172.21.15.19:0/318590185 shutdown_connections 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7febc80744b0 0x7febc80769c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febe41443c0 0x7febe40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe40cf5e0 0x7febe41fa660 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 --2- 172.21.15.19:0/318590185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7febe406c5c0 0x7febe41f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.678+0000 7febec800640 1 -- 172.21.15.19:0/318590185 >> 172.21.15.19:0/318590185 conn(0x7febe40d1ae0 msgr2=0x7febe40ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:56.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.679+0000 7febec800640 1 -- 172.21.15.19:0/318590185 shutdown_connections 2024-04-25T11:39:56.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.679+0000 7febec800640 1 -- 172.21.15.19:0/318590185 wait complete. 2024-04-25T11:39:56.692 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:56.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.991+0000 7f98f3600640 1 Processor -- start 2024-04-25T11:39:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.992+0000 7f98f3600640 1 -- start start 2024-04-25T11:39:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.994+0000 7f98f3600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98ec06ccd0 0x7f98ec0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.994+0000 7f98f3600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98ec0d4a50 0x7f98ec0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.994+0000 7f98f3600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98ec0d1850 con 0x7f98ec0d0e00 2024-04-25T11:39:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.994+0000 7f98f3600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f98ec160de0 con 0x7f98ec0d4a50 2024-04-25T11:39:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.994+0000 7f98f3600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f98ec1610c0 con 0x7f98ec06ccd0 2024-04-25T11:39:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.994+0000 7f98f1600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98ec0d4a50 0x7f98ec0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.995+0000 7f98f1600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98ec0d4a50 0x7f98ec0d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47824/0 (socket says 172.21.15.19:47824) 2024-04-25T11:39:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.995+0000 7f98f1600640 1 -- 172.21.15.19:0/42223123 learned_addr learned my addr 172.21.15.19:0/42223123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.995+0000 7f98ebe00640 1 --2- 172.21.15.19:0/42223123 >> v2:172.21.15.169:3300/0 conn(0x7f98ec06ccd0 0x7f98ec0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.995+0000 7f98f1600640 1 -- 172.21.15.19:0/42223123 >> v2:172.21.15.169:3300/0 conn(0x7f98ec06ccd0 msgr2=0x7f98ec0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.995+0000 7f98f1600640 1 --2- 172.21.15.19:0/42223123 >> v2:172.21.15.169:3300/0 conn(0x7f98ec06ccd0 0x7f98ec0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.995+0000 7f98f1600640 1 -- 172.21.15.19:0/42223123 >> v1:172.21.15.169:6789/0 conn(0x7f98ec0d0e00 legacy=0x7f98ec06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.995+0000 7f98f1600640 1 -- 172.21.15.19:0/42223123 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec1613a0 con 0x7f98ec0d4a50 2024-04-25T11:39:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.996+0000 7f98f1600640 1 --2- 172.21.15.19:0/42223123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0d4a50 0x7f98ec0d4e60 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f98ec08f170 tx=0x7f98e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c24fdf970f6de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.997+0000 7f98eb400640 1 -- 172.21.15.19:0/42223123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4036070 con 0x7f98ec0d4a50 2024-04-25T11:39:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.997+0000 7f98eb400640 1 -- 172.21.15.19:0/42223123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e4031600 con 0x7f98ec0d4a50 2024-04-25T11:39:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.997+0000 7f98eb400640 1 -- 172.21.15.19:0/42223123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e40319d0 con 0x7f98ec0d4a50 2024-04-25T11:39:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.997+0000 7f98f3600640 1 -- 172.21.15.19:0/42223123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0d4a50 msgr2=0x7f98ec0d4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.997+0000 7f98f3600640 1 --2- 172.21.15.19:0/42223123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0d4a50 0x7f98ec0d4e60 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f98ec08f170 tx=0x7f98e402a660 comp rx=0 tx=0).stop 2024-04-25T11:39:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.998+0000 7f98f3600640 1 -- 172.21.15.19:0/42223123 shutdown_connections 2024-04-25T11:39:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.998+0000 7f98f3600640 1 --2- 172.21.15.19:0/42223123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0d4a50 0x7f98ec0d4e60 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.998+0000 7f98f3600640 1 --2- 172.21.15.19:0/42223123 >> v2:172.21.15.169:3300/0 conn(0x7f98ec06ccd0 0x7f98ec0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.998+0000 7f98f3600640 1 -- 172.21.15.19:0/42223123 >> 172.21.15.19:0/42223123 conn(0x7f98ec0d5d90 msgr2=0x7f98ec0d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.998+0000 7f98f3600640 1 -- 172.21.15.19:0/42223123 shutdown_connections 2024-04-25T11:39:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:56.998+0000 7f98f3600640 1 -- 172.21.15.19:0/42223123 wait complete. 2024-04-25T11:39:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.000+0000 7f98f3600640 1 Processor -- start 2024-04-25T11:39:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.001+0000 7f98f3600640 1 -- start start 2024-04-25T11:39:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.002+0000 7f98f3600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.002+0000 7f98f3600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98ec0d4a50 0x7f98ec0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.002+0000 7f98f3600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0b6110 0x7f98ec0b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.002+0000 7f98f3600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec164e00 con 0x7f98ec0b6110 2024-04-25T11:39:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.002+0000 7f98f3600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec0b5320 con 0x7f98ec0d0e00 2024-04-25T11:39:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.002+0000 7f98f3600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec164f80 con 0x7f98ec0d4a50 2024-04-25T11:39:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0b6110 0x7f98ec0b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0b6110 0x7f98ec0b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47830/0 (socket says 172.21.15.19:47830) 2024-04-25T11:39:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 -- 172.21.15.19:0/2034278789 learned_addr learned my addr 172.21.15.19:0/2034278789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98ebe00640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98ec0d4a50 0x7f98ec0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f0c00640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 -- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98ec0d4a50 msgr2=0x7f98ec0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98ec0d4a50 0x7f98ec0b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 -- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98ec0d0e00 msgr2=0x7f98ec0c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.003+0000 7f98f1600640 1 -- 172.21.15.19:0/2034278789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec0b6a60 con 0x7f98ec0b6110 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.004+0000 7f98f1600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0b6110 0x7f98ec0b6520 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009f40 tx=0x7f98e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.004+0000 7f98e8c00640 1 -- 172.21.15.19:0/2034278789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4036050 con 0x7f98ec0b6110 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.004+0000 7f98e8c00640 1 -- 172.21.15.19:0/2034278789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e4042070 con 0x7f98ec0b6110 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.004+0000 7f98e8c00640 1 -- 172.21.15.19:0/2034278789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4009510 con 0x7f98ec0b6110 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.004+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec0c9c20 con 0x7f98ec0b6110 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.004+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec0ca380 con 0x7f98ec0b6110 2024-04-25T11:39:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.006+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ec050740 con 0x7f98ec0b6110 2024-04-25T11:39:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.007+0000 7f98e8c00640 1 -- 172.21.15.19:0/2034278789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98e4007bd0 con 0x7f98ec0b6110 2024-04-25T11:39:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.008+0000 7f98e8c00640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d00744e0 0x7f98d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.008+0000 7f98e8c00640 1 -- 172.21.15.19:0/2034278789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f98e40b6bc0 con 0x7f98ec0b6110 2024-04-25T11:39:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.009+0000 7f98f0c00640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d00744e0 0x7f98d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:56.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.010+0000 7f98f0c00640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d00744e0 0x7f98d00769f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f98e0006fd0 tx=0x7f98e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:57.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.013+0000 7f98e8c00640 1 -- 172.21.15.19:0/2034278789 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98e40808c0 con 0x7f98ec0b6110 2024-04-25T11:39:57.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.262+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98ec0d4320 con 0x7f98ec0b6110 2024-04-25T11:39:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.263+0000 7f98e8c00640 1 -- 172.21.15.19:0/2034278789 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 285 v285) ==== 95+0+5179 (secure 0 0 0) 0x7f98e40801c0 con 0x7f98ec0b6110 2024-04-25T11:39:57.251 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:57.251 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":285,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2891720740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2891720740},{"type":"v1","addr":"172.21.15.19:6835","nonce":2891720740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":285}],"filesystems":[{"mdsmap":{"epoch":284,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:56.138866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:57.251 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 285 2024-04-25T11:39:57.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.266+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d00744e0 msgr2=0x7f98d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.266+0000 7f98f3600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d00744e0 0x7f98d00769f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f98e0006fd0 tx=0x7f98e0008040 comp rx=0 tx=0).stop 2024-04-25T11:39:57.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.266+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0b6110 msgr2=0x7f98ec0b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.266+0000 7f98f3600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0b6110 0x7f98ec0b6520 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009f40 tx=0x7f98e4004a40 comp rx=0 tx=0).stop 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 shutdown_connections 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d00744e0 0x7f98d00769f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98ec0b6110 0x7f98ec0b6520 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98ec0d4a50 0x7f98ec0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 --2- 172.21.15.19:0/2034278789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98ec0d0e00 0x7f98ec0c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 >> 172.21.15.19:0/2034278789 conn(0x7f98ec0d5d90 msgr2=0x7f98ec0bb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 shutdown_connections 2024-04-25T11:39:57.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.267+0000 7f98f3600640 1 -- 172.21.15.19:0/2034278789 wait complete. 2024-04-25T11:39:57.282 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:39:57.282 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:39:57.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.562+0000 7f9825400640 1 Processor -- start 2024-04-25T11:39:57.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.563+0000 7f9825400640 1 -- start start 2024-04-25T11:39:57.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.565+0000 7f9825400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 0x7f982006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.565+0000 7f9825400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9820070370 0x7f98200cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.565+0000 7f9825400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98200cff30 con 0x7f98200cfb20 2024-04-25T11:39:57.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.565+0000 7f9825400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f98200708d0 con 0x7f9820070370 2024-04-25T11:39:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.565+0000 7f9825400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f982015f160 con 0x7f98201443c0 2024-04-25T11:39:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.566+0000 7f981f200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f98200cfb20 0x7f98200ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56490/0 (socket says 172.21.15.19:56490) 2024-04-25T11:39:57.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.566+0000 7f981f200640 1 -- 172.21.15.19:0/465100585 learned_addr learned my addr 172.21.15.19:0/465100585 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.566+0000 7f981e800640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 0x7f982006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.566+0000 7f981de00640 1 --2- 172.21.15.19:0/465100585 >> v2:172.21.15.169:3300/0 conn(0x7f9820070370 0x7f98200cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.566+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813879861 0 0) 0x7f9818001110 con 0x7f98200cfb20 2024-04-25T11:39:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.566+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9804003520 con 0x7f98200cfb20 2024-04-25T11:39:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 491151507 0 0) 0x7f9804003520 con 0x7f98200cfb20 2024-04-25T11:39:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 msgr2=0x7f982006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:57.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 0x7f982006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 >> v2:172.21.15.169:3300/0 conn(0x7f9820070370 msgr2=0x7f98200cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 --2- 172.21.15.19:0/465100585 >> v2:172.21.15.169:3300/0 conn(0x7f9820070370 0x7f98200cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982015f5f0 con 0x7f98200cfb20 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9818001840 con 0x7f98200cfb20 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 >> v1:172.21.15.19:6789/0 conn(0x7f98200cfb20 legacy=0x7f98200ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 0x7f98040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9804003f80 0x7f9804004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9804004d40 0x7f9804005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98200708d0 con 0x7f9804004d40 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f982015f160 con 0x7f9804003f80 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f980400ba70 con 0x7f98201443c0 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.567+0000 7f981f200640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9804004d40 0x7f9804005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981e800640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 0x7f98040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981de00640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9804003f80 0x7f9804004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981f200640 1 -- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 msgr2=0x7f98040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:57.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981f200640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98201443c0 0x7f98040039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981f200640 1 -- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9804003f80 msgr2=0x7f9804004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:57.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981f200640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9804003f80 0x7f9804004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981f200640 1 -- 172.21.15.19:0/465100585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9804003520 con 0x7f9804004d40 2024-04-25T11:39:57.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.568+0000 7f981de00640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9804003f80 0x7f9804004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:57.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.569+0000 7f981f200640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9804004d40 0x7f9804005150 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f981800c2f0 tx=0x7f9804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df35be0505e296c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:57.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.570+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818035070 con 0x7f9804004d40 2024-04-25T11:39:57.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.570+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98180058d0 con 0x7f9804004d40 2024-04-25T11:39:57.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.570+0000 7f981d400640 1 -- 172.21.15.19:0/465100585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818007e60 con 0x7f9804004d40 2024-04-25T11:39:57.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.570+0000 7f9825400640 1 -- 172.21.15.19:0/465100585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9804004d40 msgr2=0x7f9804005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.570+0000 7f9825400640 1 --2- 172.21.15.19:0/465100585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9804004d40 0x7f9804005150 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f981800c2f0 tx=0x7f9804005830 comp rx=0 tx=0).stop 2024-04-25T11:39:57.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.570+0000 7f981f200640 1 -- 172.21.15.19:0/465100585 reap_dead start 2024-04-25T11:39:57.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.571+0000 7f9825400640 1 -- 172.21.15.19:0/465100585 shutdown_connections 2024-04-25T11:39:57.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.571+0000 7f9825400640 1 -- 172.21.15.19:0/465100585 >> 172.21.15.19:0/465100585 conn(0x7f98200d1ae0 msgr2=0x7f98200d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:57.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.571+0000 7f9825400640 1 -- 172.21.15.19:0/465100585 shutdown_connections 2024-04-25T11:39:57.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.571+0000 7f9825400640 1 -- 172.21.15.19:0/465100585 wait complete. 2024-04-25T11:39:57.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.573+0000 7f9825400640 1 Processor -- start 2024-04-25T11:39:57.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.573+0000 7f9825400640 1 -- start start 2024-04-25T11:39:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f9825400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98200b2a20 0x7f98201f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f9825400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98200b2e30 0x7f98201f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f9825400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98201f72b0 0x7f98201f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f9825400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f982015edf0 con 0x7f98200b2e30 2024-04-25T11:39:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f9825400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f982015eb10 con 0x7f98201f72b0 2024-04-25T11:39:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f9825400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f982006d5d0 con 0x7f98200b2a20 2024-04-25T11:39:57.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f981de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98200b2e30 0x7f98201f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f981de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98200b2e30 0x7f98201f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47838/0 (socket says 172.21.15.19:47838) 2024-04-25T11:39:57.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f981de00640 1 -- 172.21.15.19:0/3444556962 learned_addr learned my addr 172.21.15.19:0/3444556962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:57.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.575+0000 7f981e800640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98200b2a20 0x7f98201f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.576+0000 7f981f200640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98201f72b0 0x7f98201f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.576+0000 7f981de00640 1 -- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98200b2a20 msgr2=0x7f98201f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.576+0000 7f981de00640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98200b2a20 0x7f98201f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.576+0000 7f981de00640 1 -- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98201f72b0 msgr2=0x7f98201f9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.576+0000 7f981de00640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98201f72b0 0x7f98201f9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.576+0000 7f981de00640 1 -- 172.21.15.19:0/3444556962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98201f76c0 con 0x7f98200b2e30 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.576+0000 7f981de00640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98200b2e30 0x7f98201f6d70 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f9808007e70 tx=0x7f98080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.577+0000 7f9816a00640 1 -- 172.21.15.19:0/3444556962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980801a020 con 0x7f98200b2e30 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.577+0000 7f9816a00640 1 -- 172.21.15.19:0/3444556962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808016470 con 0x7f98200b2e30 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.577+0000 7f9816a00640 1 -- 172.21.15.19:0/3444556962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808016840 con 0x7f98200b2e30 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.577+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98201e9ab0 con 0x7f98200b2e30 2024-04-25T11:39:57.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.577+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98201ea070 con 0x7f98200b2e30 2024-04-25T11:39:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.578+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9820050740 con 0x7f98200b2e30 2024-04-25T11:39:57.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.579+0000 7f9816a00640 1 -- 172.21.15.19:0/3444556962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9808005d10 con 0x7f98200b2e30 2024-04-25T11:39:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.581+0000 7f9816a00640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97fc0744b0 0x7f97fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.581+0000 7f9816a00640 1 -- 172.21.15.19:0/3444556962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f9808096a80 con 0x7f98200b2e30 2024-04-25T11:39:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.582+0000 7f981e800640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97fc0744b0 0x7f97fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.583+0000 7f981e800640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97fc0744b0 0x7f97fc0769c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9820144180 tx=0x7f9810009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.586+0000 7f9816a00640 1 -- 172.21.15.19:0/3444556962 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9808098e20 con 0x7f98200b2e30 2024-04-25T11:39:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.833+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f982006f950 con 0x7f98200b2e30 2024-04-25T11:39:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.834+0000 7f9816a00640 1 -- 172.21.15.19:0/3444556962 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 285 v285) ==== 95+0+5179 (secure 0 0 0) 0x7f9808024030 con 0x7f98200b2e30 2024-04-25T11:39:57.821 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:39:57.821 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":285,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10296,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2891720740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2891720740},{"type":"v1","addr":"172.21.15.19:6835","nonce":2891720740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":285}],"filesystems":[{"mdsmap":{"epoch":284,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:39:56.138866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:39:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 285 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97fc0744b0 msgr2=0x7f97fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97fc0744b0 0x7f97fc0769c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9820144180 tx=0x7f9810009190 comp rx=0 tx=0).stop 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98200b2e30 msgr2=0x7f98201f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98200b2e30 0x7f98201f6d70 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f9808007e70 tx=0x7f98080115a0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 shutdown_connections 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f97fc0744b0 0x7f97fc0769c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98201f72b0 0x7f98201f9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98200b2e30 0x7f98201f6d70 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 --2- 172.21.15.19:0/3444556962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98200b2a20 0x7f98201f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.836+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 >> 172.21.15.19:0/3444556962 conn(0x7f98200d1ae0 msgr2=0x7f982015cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.837+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 shutdown_connections 2024-04-25T11:39:57.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:57.837+0000 7f9825400640 1 -- 172.21.15.19:0/3444556962 wait complete. 2024-04-25T11:39:57.851 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2024-04-25T11:39:57.851 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2024-04-25T11:39:57.851 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:39:57.851 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:39:57.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:39:57.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:39:57.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:39:57.904 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:39:57.955 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:39:57.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:39:57.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:39:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.257+0000 7fe3d3200640 1 Processor -- start 2024-04-25T11:39:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.257+0000 7fe3d3200640 1 -- start start 2024-04-25T11:39:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe3d40b4ea0 0x7fe3d40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 0x7fe3d415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d3200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d40d1230 con 0x7fe3d406c5c0 2024-04-25T11:39:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d40d06b0 con 0x7fe3d40b4ea0 2024-04-25T11:39:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d40d0990 con 0x7fe3d4144370 2024-04-25T11:39:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d1e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe3d406c5c0 0x7fe3d40b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56494/0 (socket says 172.21.15.19:56494) 2024-04-25T11:39:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d1e00640 1 -- 172.21.15.19:0/167115927 learned_addr learned my addr 172.21.15.19:0/167115927 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.260+0000 7fe3d2800640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 0x7fe3d415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d1400640 1 --2- 172.21.15.19:0/167115927 >> v2:172.21.15.169:3300/0 conn(0x7fe3d40b4ea0 0x7fe3d40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854541950 0 0) 0x7fe3d40d1230 con 0x7fe3d406c5c0 2024-04-25T11:39:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3b8003520 con 0x7fe3d406c5c0 2024-04-25T11:39:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3631608056 0 0) 0x7fe3b8003520 con 0x7fe3d406c5c0 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 msgr2=0x7fe3d415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 0x7fe3d415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 >> v2:172.21.15.169:3300/0 conn(0x7fe3d40b4ea0 msgr2=0x7fe3d40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 --2- 172.21.15.19:0/167115927 >> v2:172.21.15.169:3300/0 conn(0x7fe3d40b4ea0 0x7fe3d40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d40d0c70 con 0x7fe3d406c5c0 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe3c4001670 con 0x7fe3d406c5c0 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 >> v1:172.21.15.19:6789/0 conn(0x7fe3d406c5c0 legacy=0x7fe3d40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b80039b0 0x7fe3b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 0x7fe3b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.261+0000 7fe3d0a00640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3b8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40d06b0 con 0x7fe3b8004d40 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40d0990 con 0x7fe3b80039b0 2024-04-25T11:39:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b800b9c0 con 0x7fe3d4144370 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d2800640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3b8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d1400640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 0x7fe3b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d2800640 1 -- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 msgr2=0x7fe3b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d1e00640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b80039b0 0x7fe3b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d2800640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 0x7fe3b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d2800640 1 -- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b80039b0 msgr2=0x7fe3b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d2800640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b80039b0 0x7fe3b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d2800640 1 -- 172.21.15.19:0/167115927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b8003520 con 0x7fe3b8004d40 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d1400640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4144370 0x7fe3b8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.262+0000 7fe3d1e00640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3b80039b0 0x7fe3b8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:39:58.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.263+0000 7fe3d2800640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3b8005150 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009fc0 tx=0x7fe3b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b85b3e8bfb74ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:58.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.264+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc034020 con 0x7fe3b8004d40 2024-04-25T11:39:58.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.264+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3cc0305f0 con 0x7fe3b8004d40 2024-04-25T11:39:58.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.264+0000 7fe3d0a00640 1 -- 172.21.15.19:0/167115927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc009c10 con 0x7fe3b8004d40 2024-04-25T11:39:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.265+0000 7fe3d3200640 1 -- 172.21.15.19:0/167115927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 msgr2=0x7fe3b8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.265+0000 7fe3d3200640 1 --2- 172.21.15.19:0/167115927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3b8005150 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009fc0 tx=0x7fe3b8005830 comp rx=0 tx=0).stop 2024-04-25T11:39:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.265+0000 7fe3d2800640 1 -- 172.21.15.19:0/167115927 reap_dead start 2024-04-25T11:39:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.265+0000 7fe3d3200640 1 -- 172.21.15.19:0/167115927 shutdown_connections 2024-04-25T11:39:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.265+0000 7fe3d3200640 1 -- 172.21.15.19:0/167115927 >> 172.21.15.19:0/167115927 conn(0x7fe3d40d1720 msgr2=0x7fe3d40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.265+0000 7fe3d3200640 1 -- 172.21.15.19:0/167115927 shutdown_connections 2024-04-25T11:39:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.265+0000 7fe3d3200640 1 -- 172.21.15.19:0/167115927 wait complete. 2024-04-25T11:39:58.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.267+0000 7fe3d3200640 1 Processor -- start 2024-04-25T11:39:58.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.268+0000 7fe3d3200640 1 -- start start 2024-04-25T11:39:58.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.269+0000 7fe3d3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3d415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.269+0000 7fe3d3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d4160e20 0x7fe3d40cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.269+0000 7fe3d3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4135ad0 0x7fe3d40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.269+0000 7fe3d3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40d0450 con 0x7fe3b8004d40 2024-04-25T11:39:58.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.269+0000 7fe3d3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40d02d0 con 0x7fe3d4160e20 2024-04-25T11:39:58.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.269+0000 7fe3d3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d4147760 con 0x7fe3d4135ad0 2024-04-25T11:39:58.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3d415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3d415c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47852/0 (socket says 172.21.15.19:47852) 2024-04-25T11:39:58.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 -- 172.21.15.19:0/89952654 learned_addr learned my addr 172.21.15.19:0/89952654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d2800640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4135ad0 0x7fe3d40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1400640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d4160e20 0x7fe3d40cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 -- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4135ad0 msgr2=0x7fe3d40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4135ad0 0x7fe3d40cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 -- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d4160e20 msgr2=0x7fe3d40cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d4160e20 0x7fe3d40cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.270+0000 7fe3d1e00640 1 -- 172.21.15.19:0/89952654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d415f910 con 0x7fe3b8004d40 2024-04-25T11:39:58.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.271+0000 7fe3d1e00640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3d415c4a0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400d910 tx=0x7fe3c400ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:58.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.271+0000 7fe3ca000640 1 -- 172.21.15.19:0/89952654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4019070 con 0x7fe3b8004d40 2024-04-25T11:39:58.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.271+0000 7fe3ca000640 1 -- 172.21.15.19:0/89952654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c40074e0 con 0x7fe3b8004d40 2024-04-25T11:39:58.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.271+0000 7fe3ca000640 1 -- 172.21.15.19:0/89952654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40049e0 con 0x7fe3b8004d40 2024-04-25T11:39:58.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.271+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d415c9e0 con 0x7fe3b8004d40 2024-04-25T11:39:58.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.271+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d4160060 con 0x7fe3b8004d40 2024-04-25T11:39:58.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.272+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d40d14e0 con 0x7fe3b8004d40 2024-04-25T11:39:58.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.273+0000 7fe3ca000640 1 -- 172.21.15.19:0/89952654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe3c401c070 con 0x7fe3b8004d40 2024-04-25T11:39:58.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.274+0000 7fe3ca000640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3b00744b0 0x7fe3b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:58.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.275+0000 7fe3ca000640 1 -- 172.21.15.19:0/89952654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7fe3c4095990 con 0x7fe3b8004d40 2024-04-25T11:39:58.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.275+0000 7fe3d1400640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3b00744b0 0x7fe3b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:58.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.277+0000 7fe3d1400640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3b00744b0 0x7fe3b00769c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4161830 tx=0x7fe3bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:58.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.279+0000 7fe3ca000640 1 -- 172.21.15.19:0/89952654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3c405f600 con 0x7fe3b8004d40 2024-04-25T11:39:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:58.526+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe3d406f160 con 0x7fe3b8004d40 2024-04-25T11:39:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.164+0000 7fe3ca000640 1 -- 172.21.15.19:0/89952654 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v286) ==== 161+0+0 (secure 0 0 0) 0x7fe3c405ef00 con 0x7fe3b8004d40 2024-04-25T11:39:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3b00744b0 msgr2=0x7fe3b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3b00744b0 0x7fe3b00769c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4161830 tx=0x7fe3bc008040 comp rx=0 tx=0).stop 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 msgr2=0x7fe3d415c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3d415c4a0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400d910 tx=0x7fe3c400ddd0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 shutdown_connections 2024-04-25T11:39:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3b00744b0 0x7fe3b00769c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3d4160e20 0x7fe3d40cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3d4135ad0 0x7fe3d40cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 --2- 172.21.15.19:0/89952654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b8004d40 0x7fe3d415c4a0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.168+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 >> 172.21.15.19:0/89952654 conn(0x7fe3d40d1720 msgr2=0x7fe3d415d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.169+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 shutdown_connections 2024-04-25T11:39:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.169+0000 7fe3d3200640 1 -- 172.21.15.19:0/89952654 wait complete. 2024-04-25T11:39:59.208 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:39:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.497+0000 7fa4e5200640 1 Processor -- start 2024-04-25T11:39:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.497+0000 7fa4e5200640 1 -- start start 2024-04-25T11:39:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.499+0000 7fa4e5200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa4e00cf2f0 0x7fa4e00cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.499+0000 7fa4e5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa4e006c5c0 0x7fa4e006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.499+0000 7fa4e5200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4e0071180 con 0x7fa4e0168010 2024-04-25T11:39:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.499+0000 7fa4e5200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e0070cf0 con 0x7fa4e00cf2f0 2024-04-25T11:39:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.499+0000 7fa4e5200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e00700e0 con 0x7fa4e006c5c0 2024-04-25T11:39:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4df400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa4e00cf2f0 0x7fa4e00cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4dea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa4e0168010 0x7fa4e0168420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56510/0 (socket says 172.21.15.19:56510) 2024-04-25T11:39:59.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4dea00640 1 -- 172.21.15.19:0/1849838115 learned_addr learned my addr 172.21.15.19:0/1849838115 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:39:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4dfe00640 1 --2- 172.21.15.19:0/1849838115 >> v2:172.21.15.169:3300/0 conn(0x7fa4e006c5c0 0x7fa4e006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4df400640 1 -- 172.21.15.19:0/1849838115 >> v2:172.21.15.169:3300/0 conn(0x7fa4e006c5c0 msgr2=0x7fa4e006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:39:59.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4df400640 1 --2- 172.21.15.19:0/1849838115 >> v2:172.21.15.169:3300/0 conn(0x7fa4e006c5c0 0x7fa4e006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4df400640 1 -- 172.21.15.19:0/1849838115 >> v1:172.21.15.19:6789/0 conn(0x7fa4e0168010 legacy=0x7fa4e0168420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.500+0000 7fa4df400640 1 -- 172.21.15.19:0/1849838115 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e00703c0 con 0x7fa4e00cf2f0 2024-04-25T11:39:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.501+0000 7fa4df400640 1 --2- 172.21.15.19:0/1849838115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e00cf2f0 0x7fa4e00cf700 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00b1390 tx=0x7fa4d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28db3003f8f0245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.502+0000 7fa4de000640 1 -- 172.21.15.19:0/1849838115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0037070 con 0x7fa4e00cf2f0 2024-04-25T11:39:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.502+0000 7fa4de000640 1 -- 172.21.15.19:0/1849838115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d0032440 con 0x7fa4e00cf2f0 2024-04-25T11:39:59.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.502+0000 7fa4de000640 1 -- 172.21.15.19:0/1849838115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0032810 con 0x7fa4e00cf2f0 2024-04-25T11:39:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.502+0000 7fa4e5200640 1 -- 172.21.15.19:0/1849838115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e00cf2f0 msgr2=0x7fa4e00cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.502+0000 7fa4e5200640 1 --2- 172.21.15.19:0/1849838115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e00cf2f0 0x7fa4e00cf700 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00b1390 tx=0x7fa4d002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.503+0000 7fa4e5200640 1 -- 172.21.15.19:0/1849838115 shutdown_connections 2024-04-25T11:39:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.503+0000 7fa4e5200640 1 --2- 172.21.15.19:0/1849838115 >> v2:172.21.15.169:3300/0 conn(0x7fa4e006c5c0 0x7fa4e006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.503+0000 7fa4e5200640 1 --2- 172.21.15.19:0/1849838115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e00cf2f0 0x7fa4e00cf700 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.503+0000 7fa4e5200640 1 -- 172.21.15.19:0/1849838115 >> 172.21.15.19:0/1849838115 conn(0x7fa4e00d1720 msgr2=0x7fa4e00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.503+0000 7fa4e5200640 1 -- 172.21.15.19:0/1849838115 shutdown_connections 2024-04-25T11:39:59.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.503+0000 7fa4e5200640 1 -- 172.21.15.19:0/1849838115 wait complete. 2024-04-25T11:39:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.506+0000 7fa4e5200640 1 Processor -- start 2024-04-25T11:39:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.506+0000 7fa4e5200640 1 -- start start 2024-04-25T11:39:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4e5200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4e006c5c0 0x7fa4e0160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4e5200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4e00cf2f0 0x7fa4e01fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4e5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e0168010 0x7fa4e01fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4e5200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0168d00 con 0x7fa4e0168010 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4e5200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0070980 con 0x7fa4e006c5c0 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4e5200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e0168e80 con 0x7fa4e00cf2f0 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e0168010 0x7fa4e01fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e0168010 0x7fa4e01fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47866/0 (socket says 172.21.15.19:47866) 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.508+0000 7fa4dfe00640 1 -- 172.21.15.19:0/724077172 learned_addr learned my addr 172.21.15.19:0/724077172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4dfe00640 1 -- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4e00cf2f0 msgr2=0x7fa4e01fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4df400640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4e006c5c0 0x7fa4e0160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4dea00640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4e00cf2f0 0x7fa4e01fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4dfe00640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4e00cf2f0 0x7fa4e01fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4dfe00640 1 -- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4e006c5c0 msgr2=0x7fa4e0160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4dfe00640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4e006c5c0 0x7fa4e0160cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4dfe00640 1 -- 172.21.15.19:0/724077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e01fbe90 con 0x7fa4e0168010 2024-04-25T11:39:59.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4dfe00640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e0168010 0x7fa4e01fd430 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800cc00 tx=0x7fa4d800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4d7400640 1 -- 172.21.15.19:0/724077172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800ce10 con 0x7fa4e0168010 2024-04-25T11:39:59.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4d7400640 1 -- 172.21.15.19:0/724077172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d8003990 con 0x7fa4e0168010 2024-04-25T11:39:59.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4d7400640 1 -- 172.21.15.19:0/724077172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8003d60 con 0x7fa4e0168010 2024-04-25T11:39:59.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.509+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e015e930 con 0x7fa4e0168010 2024-04-25T11:39:59.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.510+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e01fc410 con 0x7fa4e0168010 2024-04-25T11:39:59.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.511+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e01edea0 con 0x7fa4e0168010 2024-04-25T11:39:59.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.512+0000 7fa4d7400640 1 -- 172.21.15.19:0/724077172 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa4d8003f00 con 0x7fa4e0168010 2024-04-25T11:39:59.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.513+0000 7fa4d7400640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:59.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.513+0000 7fa4d7400640 1 -- 172.21.15.19:0/724077172 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7fa4d8096150 con 0x7fa4e0168010 2024-04-25T11:39:59.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.514+0000 7fa4df400640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:59.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.515+0000 7fa4df400640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0034000 tx=0x7fa4d002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:59.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.518+0000 7fa4d7400640 1 -- 172.21.15.19:0/724077172 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4d8014030 con 0x7fa4e0168010 2024-04-25T11:39:59.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:39:59.806+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa4e0050740 con 0x7fa4e0168010 2024-04-25T11:39:59.986 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:39:59.999+0000 7fb164400640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline; 1 mds daemon damaged 2024-04-25T11:40:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.179+0000 7fa4d7400640 1 -- 172.21.15.19:0/724077172 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v287) ==== 101+0+0 (secure 0 0 0) 0x7fa4d805fe50 con 0x7fa4e0168010 2024-04-25T11:40:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:40:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa4bc0744b0 msgr2=0x7fa4bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0034000 tx=0x7fa4d002e000 comp rx=0 tx=0).stop 2024-04-25T11:40:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e0168010 msgr2=0x7fa4e01fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e0168010 0x7fa4e01fd430 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800cc00 tx=0x7fa4d800a5c0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 shutdown_connections 2024-04-25T11:40:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4e0168010 0x7fa4e01fd430 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4e00cf2f0 0x7fa4e01fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 --2- 172.21.15.19:0/724077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4e006c5c0 0x7fa4e0160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 >> 172.21.15.19:0/724077172 conn(0x7fa4e00d1720 msgr2=0x7fa4e006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 shutdown_connections 2024-04-25T11:40:00.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.183+0000 7fa4e5200640 1 -- 172.21.15.19:0/724077172 wait complete. 2024-04-25T11:40:00.199 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:40:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.398+0000 7fe358733f40 1 Processor -- start 2024-04-25T11:40:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.398+0000 7fe358733f40 1 -- start start 2024-04-25T11:40:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.399+0000 7fe358733f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd7e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.399+0000 7fe358733f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbcb55f0 0x5646dbd8a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.399+0000 7fe358733f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbd8edd0 0x5646dbd8b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.399+0000 7fe358733f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646dbcfde70 con 0x5646dbcb51e0 2024-04-25T11:40:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.399+0000 7fe358733f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646dbc5bd20 con 0x5646dbcb55f0 2024-04-25T11:40:00.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.399+0000 7fe358733f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5646dbcfe110 con 0x5646dbd8edd0 2024-04-25T11:40:00.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd7e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd7e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47898/0 (socket says 172.21.15.19:47898) 2024-04-25T11:40:00.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 -- 172.21.15.19:0/3288151286 learned_addr learned my addr 172.21.15.19:0/3288151286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 -- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbd8edd0 msgr2=0x5646dbd8b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe350e00640 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbcb55f0 0x5646dbd8a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe352200640 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbd8edd0 0x5646dbd8b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbd8edd0 0x5646dbd8b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 -- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbcb55f0 msgr2=0x5646dbd8a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbcb55f0 0x5646dbd8a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 -- 172.21.15.19:0/3288151286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dbd02cf0 con 0x5646dbcb51e0 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe352200640 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbd8edd0 0x5646dbd8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.400+0000 7fe351800640 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd7e1a0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fe34400a510 tx=0x7fe344005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a91b9f10046d20a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.401+0000 7fe34b400640 1 -- 172.21.15.19:0/3288151286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34400f7b0 con 0x5646dbcb51e0 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.401+0000 7fe34b400640 1 -- 172.21.15.19:0/3288151286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe344015070 con 0x5646dbcb51e0 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.401+0000 7fe34b400640 1 -- 172.21.15.19:0/3288151286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34400fb90 con 0x5646dbcb51e0 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.401+0000 7fe358733f40 1 -- 172.21.15.19:0/3288151286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 msgr2=0x5646dbd7e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.401+0000 7fe358733f40 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd7e1a0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fe34400a510 tx=0x7fe344005f90 comp rx=0 tx=0).stop 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.402+0000 7fe358733f40 1 -- 172.21.15.19:0/3288151286 shutdown_connections 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.402+0000 7fe358733f40 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbcb55f0 0x5646dbd8a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.402+0000 7fe358733f40 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbd8edd0 0x5646dbd8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.402+0000 7fe358733f40 1 --2- 172.21.15.19:0/3288151286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd7e1a0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.402+0000 7fe358733f40 1 -- 172.21.15.19:0/3288151286 >> 172.21.15.19:0/3288151286 conn(0x5646dbbd7ac0 msgr2=0x5646dbd8dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.402+0000 7fe358733f40 1 -- 172.21.15.19:0/3288151286 shutdown_connections 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.402+0000 7fe358733f40 1 -- 172.21.15.19:0/3288151286 wait complete. 2024-04-25T11:40:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.403+0000 7fe358733f40 1 Processor -- start 2024-04-25T11:40:00.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.403+0000 7fe358733f40 1 -- start start 2024-04-25T11:40:00.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.404+0000 7fe358733f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.404+0000 7fe358733f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbcb55f0 0x5646dbd78370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.404+0000 7fe358733f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbd8edd0 0x5646dbd788b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.404+0000 7fe358733f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646dbd84070 con 0x5646dbcb51e0 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.404+0000 7fe358733f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5646dbd795a0 con 0x5646dbd8edd0 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.404+0000 7fe358733f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5646dbd8a590 con 0x5646dbcb55f0 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd01fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47904/0 (socket says 172.21.15.19:47904) 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 learned_addr learned my addr 172.21.15.19:0/656203016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe350e00640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbcb55f0 0x5646dbd78370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe352200640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbd8edd0 0x5646dbd788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbcb55f0 msgr2=0x5646dbd78370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5646dbcb55f0 0x5646dbd78370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbd8edd0 msgr2=0x5646dbd788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5646dbd8edd0 0x5646dbd788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.405+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dbd8ad00 con 0x5646dbcb51e0 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.406+0000 7fe351800640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd01fb0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fe3440104c0 tx=0x7fe3440032b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.406+0000 7fe34a000640 1 -- 172.21.15.19:0/656203016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344010ab0 con 0x5646dbcb51e0 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.406+0000 7fe34a000640 1 -- 172.21.15.19:0/656203016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe344020030 con 0x5646dbcb51e0 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.406+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646dbd8aa20 con 0x5646dbcb51e0 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.406+0000 7fe34a000640 1 -- 172.21.15.19:0/656203016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344025420 con 0x5646dbcb51e0 2024-04-25T11:40:00.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.406+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646dbcfbaf0 con 0x5646dbcb51e0 2024-04-25T11:40:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.409+0000 7fe34a000640 1 -- 172.21.15.19:0/656203016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe34401c420 con 0x5646dbcb51e0 2024-04-25T11:40:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.410+0000 7fe34a000640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe330072ec0 0x7fe3300753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.410+0000 7fe34a000640 1 -- 172.21.15.19:0/656203016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7fe344096ae0 con 0x5646dbcb51e0 2024-04-25T11:40:00.397 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:40:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.410+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646dbd798d0 0x5646dbd88740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.410+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd024f0 con 0x5646dbd798d0 2024-04-25T11:40:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.410+0000 7fe350e00640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe330072ec0 0x7fe3300753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.411+0000 7fe352200640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646dbd798d0 0x5646dbd88740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.412+0000 7fe350e00640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe330072ec0 0x7fe3300753d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe33c057000 tx=0x7fe33c052000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.412+0000 7fe352200640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646dbd798d0 0x5646dbd88740 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.416+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v123'939 uv913 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.416+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbd78df0 con 0x5646dbd798d0 2024-04-25T11:40:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.420+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v123'940 uv940 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.420+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd791c0 con 0x5646dbd798d0 2024-04-25T11:40:00.409 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:40:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.423+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v123'941 uv941 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.423+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe30500 con 0x5646dbd798d0 2024-04-25T11:40:00.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.427+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'942 uv942 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.427+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe308a0 con 0x5646dbd798d0 2024-04-25T11:40:00.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.430+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'943 uv943 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.430+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2f4f0 con 0x5646dbd798d0 2024-04-25T11:40:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.434+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'944 uv944 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.434+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2f8e0 con 0x5646dbd798d0 2024-04-25T11:40:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.437+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v123'945 uv919 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.438+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe2fcd0 con 0x5646dbd798d0 2024-04-25T11:40:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.441+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v123'946 uv946 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.441+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fcd0 con 0x5646dbd798d0 2024-04-25T11:40:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.444+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v123'947 uv947 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.431 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:40:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.445+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.448+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'948 uv948 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.448+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.452+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'949 uv949 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.452+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.456+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'950 uv950 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.457+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.461+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v123'951 uv926 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.461+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.465+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'952 uv952 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.465+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.456 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:40:00.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.469+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v123'953 uv953 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.469+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.473+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'954 uv954 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.473+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.478+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'955 uv955 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.478+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe2fc80 con 0x5646dbd798d0 2024-04-25T11:40:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.482+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'956 uv956 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.482+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe30070 con 0x5646dbd798d0 2024-04-25T11:40:00.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.487+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'957 uv957 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.487+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe30070 con 0x5646dbd798d0 2024-04-25T11:40:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.491+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v123'958 uv932 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.491+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe30070 con 0x5646dbd798d0 2024-04-25T11:40:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.496+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v123'959 uv959 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.496+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe30070 con 0x5646dbd798d0 2024-04-25T11:40:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.501+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v123'960 uv960 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.488 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:40:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.501+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.505+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'961 uv961 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.505+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.510+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'962 uv962 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.510+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.514+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'963 uv963 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.514+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.520+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v123'964 uv938 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.520+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.524+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v123'965 uv965 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.525+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.515 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:40:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.529+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v123'966 uv966 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.529+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.533+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'967 uv967 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.533+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.538+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'968 uv968 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.538+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbd798d0 2024-04-25T11:40:00.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.543+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'969 uv969 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.543+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646dbe31150 0x5646dbe33570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.543+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbd88c80 con 0x5646dbe31150 2024-04-25T11:40:00.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.544+0000 7fe351800640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646dbe31150 0x5646dbe33570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.545+0000 7fe351800640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646dbe31150 0x5646dbe33570 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.550+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'680 uv663 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.550+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe33ab0 con 0x5646dbe31150 2024-04-25T11:40:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.555+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'681 uv681 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.555+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe33e50 con 0x5646dbe31150 2024-04-25T11:40:00.546 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:40:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.559+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'682 uv682 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.560+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe341f0 con 0x5646dbe31150 2024-04-25T11:40:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.564+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v123'683 uv683 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.564+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34590 con 0x5646dbe31150 2024-04-25T11:40:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.568+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'684 uv684 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.569+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34930 con 0x5646dbe31150 2024-04-25T11:40:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.573+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v123'685 uv668 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.573+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.577+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v123'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.577+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.568 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:40:00.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.581+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v123'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.582+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.586+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.586+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.591+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.591+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.595+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v123'690 uv673 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.595+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.599+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v123'691 uv691 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.599+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.590 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:40:00.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.603+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v123'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.604+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe393c0 con 0x5646dbe31150 2024-04-25T11:40:00.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.609+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v123'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.610+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe393c0 con 0x5646dbe31150 2024-04-25T11:40:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.614+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.614+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.618+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v123'695 uv679 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.619+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.623+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v123'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.623+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.614 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:40:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.628+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v123'697 uv697 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.628+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe35ce0 con 0x5646dbe31150 2024-04-25T11:40:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.632+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.632+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe35ce0 con 0x5646dbe31150 2024-04-25T11:40:00.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.636+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.636+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe34cd0 con 0x5646dbe31150 2024-04-25T11:40:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.641+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe31150 2024-04-25T11:40:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.641+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646dbe35350 0x5646dbe37770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.642+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe37d00 con 0x5646dbe35350 2024-04-25T11:40:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.643+0000 7fe350e00640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646dbe35350 0x5646dbe37770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.644+0000 7fe350e00640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646dbe35350 0x5646dbe37770 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.649+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.649+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe381a0 con 0x5646dbe35350 2024-04-25T11:40:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.653+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v123'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.653+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe38680 con 0x5646dbe35350 2024-04-25T11:40:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.657+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v123'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.644 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:40:00.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.658+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe39f30 con 0x5646dbe35350 2024-04-25T11:40:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.662+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.662+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe3a2d0 con 0x5646dbe35350 2024-04-25T11:40:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.667+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.667+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe38f20 con 0x5646dbe35350 2024-04-25T11:40:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.671+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.671+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe39310 con 0x5646dbd798d0 2024-04-25T11:40:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.676+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'136 uv135 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.676+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe39310 con 0x5646dbd798d0 2024-04-25T11:40:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.680+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'137 uv137 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.680+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe39310 con 0x5646dbd798d0 2024-04-25T11:40:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.684+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'138 uv138 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.671 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:40:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.684+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe39310 con 0x5646dbd798d0 2024-04-25T11:40:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.689+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'139 uv139 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.689+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646dbe3a670 0x5646dbe3ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.689+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe39310 con 0x5646dbe3a670 2024-04-25T11:40:00.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.690+0000 7fe352200640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646dbe3a670 0x5646dbe3ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.692+0000 7fe352200640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646dbe3a670 0x5646dbe3ca90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.697+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.697+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe39310 con 0x5646dbe3a670 2024-04-25T11:40:00.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.701+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v123'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.702+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe39310 con 0x5646dbe3a670 2024-04-25T11:40:00.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.705+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v123'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.692 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:40:00.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.706+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe3dfe0 con 0x5646dbe3a670 2024-04-25T11:40:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.710+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.710+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe3dfe0 con 0x5646dbe3a670 2024-04-25T11:40:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.714+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.715+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe3cfd0 con 0x5646dbe3a670 2024-04-25T11:40:00.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.719+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.719+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646dbe3d4e0 0x5646dbe3f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.719+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe3ff60 con 0x5646dbe3d4e0 2024-04-25T11:40:00.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.720+0000 7fe351800640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646dbe3d4e0 0x5646dbe3f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.721+0000 7fe351800640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646dbe3d4e0 0x5646dbe3f900 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.726+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v123'338 uv332 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.726+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe40020 con 0x5646dbe3d4e0 2024-04-25T11:40:00.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.730+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v123'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.730+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe40440 con 0x5646dbe3d4e0 2024-04-25T11:40:00.721 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:40:00.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.734+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v123'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.734+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe41450 con 0x5646dbe3d4e0 2024-04-25T11:40:00.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.739+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.739+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe41450 con 0x5646dbe3d4e0 2024-04-25T11:40:00.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.743+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.743+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe40440 con 0x5646dbe3d4e0 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.747+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.748+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe40440 con 0x5646dbe3d4e0 2024-04-25T11:40:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.752+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v123'344 uv337 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.752+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe40440 con 0x5646dbe3d4e0 2024-04-25T11:40:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.756+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v123'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.757+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe40440 con 0x5646dbe3d4e0 2024-04-25T11:40:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.760+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v123'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.747 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:40:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.761+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe40440 con 0x5646dbe3d4e0 2024-04-25T11:40:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.765+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.765+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe40440 con 0x5646dbe3d4e0 2024-04-25T11:40:00.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.769+0000 7fe351800640 1 -- 172.21.15.19:0/656203016 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34402c070 con 0x5646dbe3d4e0 2024-04-25T11:40:00.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.769+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646dbe40ac0 0x5646dbe42ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:00.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.770+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe434f0 con 0x5646dbe40ac0 2024-04-25T11:40:00.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.770+0000 7fe350e00640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646dbe40ac0 0x5646dbe42ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:00.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.772+0000 7fe350e00640 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646dbe40ac0 0x5646dbe42ee0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:00.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.776+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v123'153 uv152 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.776+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe438c0 con 0x5646dbe40ac0 2024-04-25T11:40:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.781+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v123'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.781+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbe40ac0 2024-04-25T11:40:00.772 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:40:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.785+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v123'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.785+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbe40ac0 2024-04-25T11:40:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.790+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.790+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbe40ac0 2024-04-25T11:40:00.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.794+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'157 uv157 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.794+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbd798d0 2024-04-25T11:40:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.798+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.799+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbd798d0 2024-04-25T11:40:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.803+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v123'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.803+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbd798d0 2024-04-25T11:40:00.795 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:40:00.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.808+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v123'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.808+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44cc0 con 0x5646dbd798d0 2024-04-25T11:40:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.812+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.812+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44cc0 con 0x5646dbd798d0 2024-04-25T11:40:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.816+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.816+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbd798d0 2024-04-25T11:40:00.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.821+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbd798d0 2024-04-25T11:40:00.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.821+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbe35350 2024-04-25T11:40:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.825+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v123'304 uv298 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.825+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbe35350 2024-04-25T11:40:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.830+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v123'305 uv305 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.830+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbe35350 2024-04-25T11:40:00.820 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:40:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.834+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v123'306 uv306 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.834+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe38b10 con 0x5646dbe35350 2024-04-25T11:40:00.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.838+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'307 uv307 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.838+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe38b10 con 0x5646dbe35350 2024-04-25T11:40:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.843+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'308 uv308 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.843+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe38b10 con 0x5646dbe35350 2024-04-25T11:40:00.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.847+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v123'309 uv303 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.847+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe38b10 con 0x5646dbe35350 2024-04-25T11:40:00.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.852+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v123'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.852+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe38b10 con 0x5646dbe35350 2024-04-25T11:40:00.844 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:40:00.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.857+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v123'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.857+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe38b10 con 0x5646dbe35350 2024-04-25T11:40:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.861+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.861+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe46b00 con 0x5646dbe35350 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.866+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe35350 2024-04-25T11:40:00.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.866+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe43cb0 con 0x5646dbe3a670 2024-04-25T11:40:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.870+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v123'132 uv131 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.870+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe3a670 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.874+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'133 uv133 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.875+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe3a670 2024-04-25T11:40:00.865 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:40:00.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.879+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v123'134 uv134 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.879+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe3a670 2024-04-25T11:40:00.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.883+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'135 uv135 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.883+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe3a670 2024-04-25T11:40:00.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.888+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v123'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.888+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe3a670 2024-04-25T11:40:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.891+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v123'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.891+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe3a670 2024-04-25T11:40:00.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.896+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v123'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.883 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:40:00.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.896+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe45170 con 0x5646dbe3a670 2024-04-25T11:40:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.900+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.900+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe45170 con 0x5646dbe3a670 2024-04-25T11:40:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.904+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.904+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe3a670 2024-04-25T11:40:00.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.908+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe34c00b5b0 con 0x5646dbe3a670 2024-04-25T11:40:00.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.908+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.912+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v123'156 uv155 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.912+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.916+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v123'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.916+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.921+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v123'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.907 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:40:00.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.921+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.925+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.925+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.929+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.929+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.934+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v123'376 uv375 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.934+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.938+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v123'377 uv377 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.938+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.929 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:40:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.942+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v123'378 uv378 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.942+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.947+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v123'379 uv379 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.947+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.951+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'380 uv380 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.951+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.955+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v123'381 uv374 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.955+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.959+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v123'382 uv382 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.959+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.950 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:40:00.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.963+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v123'383 uv383 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.964+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe45170 con 0x5646dbe40ac0 2024-04-25T11:40:00.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.968+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'384 uv384 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.968+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe45170 con 0x5646dbe40ac0 2024-04-25T11:40:00.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.971+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'385 uv385 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.972+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe44160 con 0x5646dbe40ac0 2024-04-25T11:40:00.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.975+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'386 uv386 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.976+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5646dbe446c0 con 0x5646dbe40ac0 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe350e00640 1 -- 172.21.15.19:0/656203016 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'387 uv387 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fe33c051070 con 0x5646dbe40ac0 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646dbe31150 msgr2=0x5646dbe33570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x5646dbe31150 0x5646dbe33570 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646dbe3d4e0 msgr2=0x5646dbe3f900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x5646dbe3d4e0 0x5646dbe3f900 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646dbe40ac0 msgr2=0x5646dbe42ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5646dbe40ac0 0x5646dbe42ee0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646dbe35350 msgr2=0x5646dbe37770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x5646dbe35350 0x5646dbe37770 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 reap_dead start 2024-04-25T11:40:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646dbd798d0 msgr2=0x5646dbd88740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x5646dbd798d0 0x5646dbd88740 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe352200640 1 -- 172.21.15.19:0/656203016 reap_dead start 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646dbe3a670 msgr2=0x5646dbe3ca90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.980+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646dbe3a670 0x5646dbe3ca90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe330072ec0 msgr2=0x7fe3300753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe330072ec0 0x7fe3300753d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe33c057000 tx=0x7fe33c052000 comp rx=0 tx=0).stop 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 msgr2=0x5646dbd01fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd01fb0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fe3440104c0 tx=0x7fe3440032b0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 shutdown_connections 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x5646dbe3a670 0x5646dbe3ca90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe330072ec0 0x7fe3300753d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 --2- 172.21.15.19:0/656203016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646dbcb51e0 0x5646dbd01fb0 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.981+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 >> 172.21.15.19:0/656203016 conn(0x5646dbbd7ac0 msgr2=0x5646dbd83620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.982+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 shutdown_connections 2024-04-25T11:40:00.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:00.982+0000 7fe358733f40 1 -- 172.21.15.19:0/656203016 wait complete. 2024-04-25T11:40:00.998 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:40:01.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.251+0000 7f992bf78f40 1 Processor -- start 2024-04-25T11:40:01.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.252+0000 7f992bf78f40 1 -- start start 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.253+0000 7f992bf78f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04210 0x55f208fd0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.253+0000 7f992bf78f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04620 0x55f208f02fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.253+0000 7f992bf78f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208f03500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.253+0000 7f992bf78f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f208f4cea0 con 0x55f208fddfd0 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.253+0000 7f992bf78f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f208eaad20 con 0x55f208f04210 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.253+0000 7f992bf78f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f208f4d420 con 0x55f208f04620 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.253+0000 7f9925a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208f03500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208f03500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47914/0 (socket says 172.21.15.19:47914) 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 -- 172.21.15.19:0/3379642144 learned_addr learned my addr 172.21.15.19:0/3379642144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925000640 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04210 0x55f208fd0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9924600640 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04620 0x55f208f02fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 -- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04620 msgr2=0x55f208f02fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:01.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04620 0x55f208f02fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 -- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04210 msgr2=0x55f208fd0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04210 0x55f208fd0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 -- 172.21.15.19:0/3379642144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f208fd0f90 con 0x55f208fddfd0 2024-04-25T11:40:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925000640 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04210 0x55f208fd0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9924600640 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04620 0x55f208f02fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:01.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.254+0000 7f9925a00640 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208f03500 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f991c00a560 tx=0x7f991c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951cb952cd256f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f9923200640 1 -- 172.21.15.19:0/3379642144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c019070 con 0x55f208fddfd0 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f9923200640 1 -- 172.21.15.19:0/3379642144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991c014470 con 0x55f208fddfd0 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f9923200640 1 -- 172.21.15.19:0/3379642144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c014860 con 0x55f208fddfd0 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f992bf78f40 1 -- 172.21.15.19:0/3379642144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 msgr2=0x55f208f03500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f992bf78f40 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208f03500 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f991c00a560 tx=0x7f991c00a650 comp rx=0 tx=0).stop 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f992bf78f40 1 -- 172.21.15.19:0/3379642144 shutdown_connections 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f992bf78f40 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208f03500 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f992bf78f40 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04620 0x55f208f02fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f992bf78f40 1 --2- 172.21.15.19:0/3379642144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04210 0x55f208fd0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.255+0000 7f992bf78f40 1 -- 172.21.15.19:0/3379642144 >> 172.21.15.19:0/3379642144 conn(0x55f208e26ac0 msgr2=0x55f208f51bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.256+0000 7f992bf78f40 1 -- 172.21.15.19:0/3379642144 shutdown_connections 2024-04-25T11:40:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.256+0000 7f992bf78f40 1 -- 172.21.15.19:0/3379642144 wait complete. 2024-04-25T11:40:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.257+0000 7f992bf78f40 1 Processor -- start 2024-04-25T11:40:01.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.257+0000 7f992bf78f40 1 -- start start 2024-04-25T11:40:01.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.258+0000 7f992bf78f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04210 0x55f208fcbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.258+0000 7f992bf78f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04620 0x55f208fced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.258+0000 7f992bf78f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208fcf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f992bf78f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f208f36a90 con 0x55f208fddfd0 2024-04-25T11:40:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f992bf78f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f208f51960 con 0x55f208f04620 2024-04-25T11:40:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f992bf78f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f208fd2b80 con 0x55f208f04210 2024-04-25T11:40:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208fcf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208fcf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47928/0 (socket says 172.21.15.19:47928) 2024-04-25T11:40:01.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 -- 172.21.15.19:0/115468083 learned_addr learned my addr 172.21.15.19:0/115468083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:01.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925000640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04210 0x55f208fcbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9924600640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04620 0x55f208fced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 -- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04210 msgr2=0x55f208fcbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f208f04210 0x55f208fcbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 -- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04620 msgr2=0x55f208fced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f208f04620 0x55f208fced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.259+0000 7f9925a00640 1 -- 172.21.15.19:0/115468083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f208f03840 con 0x55f208fddfd0 2024-04-25T11:40:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.260+0000 7f9925a00640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208fcf2d0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f991c005700 tx=0x7f991c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.260+0000 7f9921e00640 1 -- 172.21.15.19:0/115468083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c019050 con 0x55f208fddfd0 2024-04-25T11:40:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.260+0000 7f9921e00640 1 -- 172.21.15.19:0/115468083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991c025070 con 0x55f208fddfd0 2024-04-25T11:40:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.260+0000 7f9921e00640 1 -- 172.21.15.19:0/115468083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c020890 con 0x55f208fddfd0 2024-04-25T11:40:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.260+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f208f03560 con 0x55f208fddfd0 2024-04-25T11:40:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.260+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f208fd8f30 con 0x55f208fddfd0 2024-04-25T11:40:01.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.262+0000 7f9921e00640 1 -- 172.21.15.19:0/115468083 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f991c020a30 con 0x55f208fddfd0 2024-04-25T11:40:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.263+0000 7f9921e00640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9904072ec0 0x7f99040753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.263+0000 7f9921e00640 1 -- 172.21.15.19:0/115468083 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f991c095dc0 con 0x55f208fddfd0 2024-04-25T11:40:01.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.264+0000 7f992bf78f40 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f208fcc200 0x55f20907dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.264+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55f208fd24a0 con 0x55f208fcc200 2024-04-25T11:40:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.264+0000 7f9925000640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9904072ec0 0x7f99040753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.265+0000 7f9924600640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f208fcc200 0x55f20907dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.266+0000 7f9925000640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9904072ec0 0x7f99040753d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9914056000 tx=0x7f9914050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.266+0000 7f9924600640 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f208fcc200 0x55f20907dd20 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f9924600640 1 -- 172.21.15.19:0/115468083 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v123'970 uv970 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f99100017f0 con 0x55f208fcc200 2024-04-25T11:40:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f208fcc200 msgr2=0x55f20907dd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f992bf78f40 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f208fcc200 0x55f20907dd20 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9904072ec0 msgr2=0x7f99040753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f992bf78f40 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9904072ec0 0x7f99040753d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f9914056000 tx=0x7f9914050000 comp rx=0 tx=0).stop 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 msgr2=0x55f208fcf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f992bf78f40 1 --2- 172.21.15.19:0/115468083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f208fddfd0 0x55f208fcf2d0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f991c005700 tx=0x7f991c003020 comp rx=0 tx=0).stop 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.269+0000 7f9925a00640 1 -- 172.21.15.19:0/115468083 reap_dead start 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.270+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 shutdown_connections 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.270+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 >> 172.21.15.19:0/115468083 conn(0x55f208e26ac0 msgr2=0x55f208f34030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.270+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 shutdown_connections 2024-04-25T11:40:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.270+0000 7f992bf78f40 1 -- 172.21.15.19:0/115468083 wait complete. 2024-04-25T11:40:01.268 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:40:01.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.558+0000 7faaea600640 1 Processor -- start 2024-04-25T11:40:01.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.558+0000 7faaea600640 1 -- start start 2024-04-25T11:40:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.560+0000 7faaea600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faae4070630 0x7faae40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.560+0000 7faaea600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.560+0000 7faaea600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faae406df20 con 0x7faae406db10 2024-04-25T11:40:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.560+0000 7faaea600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faae40713f0 con 0x7faae4070630 2024-04-25T11:40:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.560+0000 7faaea600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faae40701a0 con 0x7faae415adf0 2024-04-25T11:40:01.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faae4070630 0x7faae40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faae4070630 0x7faae40cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47930/0 (socket says 172.21.15.19:47930) 2024-04-25T11:40:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae3400640 1 -- 172.21.15.19:0/646596329 learned_addr learned my addr 172.21.15.19:0/646596329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae2a00640 1 --2- 172.21.15.19:0/646596329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae3400640 1 -- 172.21.15.19:0/646596329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 msgr2=0x7faae406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae3400640 1 --2- 172.21.15.19:0/646596329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae3400640 1 -- 172.21.15.19:0/646596329 >> v1:172.21.15.169:6789/0 conn(0x7faae406db10 legacy=0x7faae40b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:01.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.561+0000 7faae3400640 1 -- 172.21.15.19:0/646596329 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae4071110 con 0x7faae4070630 2024-04-25T11:40:01.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.562+0000 7faae3400640 1 --2- 172.21.15.19:0/646596329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40cec50 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81bb7d0a7168394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.563+0000 7faae2000640 1 -- 172.21.15.19:0/646596329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4037070 con 0x7faae4070630 2024-04-25T11:40:01.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.563+0000 7faae2000640 1 -- 172.21.15.19:0/646596329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad4032440 con 0x7faae4070630 2024-04-25T11:40:01.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.563+0000 7faae2000640 1 -- 172.21.15.19:0/646596329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4032810 con 0x7faae4070630 2024-04-25T11:40:01.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 -- 172.21.15.19:0/646596329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 msgr2=0x7faae40cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 --2- 172.21.15.19:0/646596329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40cec50 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad402a180 comp rx=0 tx=0).stop 2024-04-25T11:40:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 -- 172.21.15.19:0/646596329 shutdown_connections 2024-04-25T11:40:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 --2- 172.21.15.19:0/646596329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 --2- 172.21.15.19:0/646596329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40cec50 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 -- 172.21.15.19:0/646596329 >> 172.21.15.19:0/646596329 conn(0x7faae4073090 msgr2=0x7faae40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 -- 172.21.15.19:0/646596329 shutdown_connections 2024-04-25T11:40:01.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.564+0000 7faaea600640 1 -- 172.21.15.19:0/646596329 wait complete. 2024-04-25T11:40:01.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.566+0000 7faaea600640 1 Processor -- start 2024-04-25T11:40:01.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.566+0000 7faaea600640 1 -- start start 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faaea600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae406db10 0x7faae414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faaea600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faaea600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae40af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faaea600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faae40b0d00 con 0x7faae4070630 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faaea600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faae406fe30 con 0x7faae406db10 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faaea600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faae40b0e80 con 0x7faae415adf0 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faae2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faae2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47934/0 (socket says 172.21.15.19:47934) 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.568+0000 7faae2a00640 1 -- 172.21.15.19:0/1463083876 learned_addr learned my addr 172.21.15.19:0/1463083876 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae3400640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae406db10 0x7faae414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae2a00640 1 -- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 msgr2=0x7faae40af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae3e00640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae40af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae2a00640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae40af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae2a00640 1 -- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae406db10 msgr2=0x7faae414b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae2a00640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae406db10 0x7faae414b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae2a00640 1 -- 172.21.15.19:0/1463083876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae41f6d30 con 0x7faae4070630 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae3400640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae406db10 0x7faae414b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae3e00640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae40af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:01.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.569+0000 7faae2a00640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40af140 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7faad000d5f0 tx=0x7faad000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.570+0000 7faadb400640 1 -- 172.21.15.19:0/1463083876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad00125f0 con 0x7faae4070630 2024-04-25T11:40:01.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.570+0000 7faadb400640 1 -- 172.21.15.19:0/1463083876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad0019070 con 0x7faae4070630 2024-04-25T11:40:01.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.570+0000 7faadb400640 1 -- 172.21.15.19:0/1463083876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad00129d0 con 0x7faae4070630 2024-04-25T11:40:01.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.570+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae41f7010 con 0x7faae4070630 2024-04-25T11:40:01.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.570+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae41f75d0 con 0x7faae4070630 2024-04-25T11:40:01.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.571+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae4050740 con 0x7faae4070630 2024-04-25T11:40:01.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.572+0000 7faadb400640 1 -- 172.21.15.19:0/1463083876 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faad0012b70 con 0x7faae4070630 2024-04-25T11:40:01.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.573+0000 7faadb400640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faac40744b0 0x7faac40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:01.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.573+0000 7faadb400640 1 -- 172.21.15.19:0/1463083876 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7faad0096230 con 0x7faae4070630 2024-04-25T11:40:01.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.574+0000 7faae3400640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faac40744b0 0x7faac40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:01.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.575+0000 7faae3400640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faac40744b0 0x7faac40769c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7faad4034000 tx=0x7faad402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:01.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.578+0000 7faadb400640 1 -- 172.21.15.19:0/1463083876 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faad005fea0 con 0x7faae4070630 2024-04-25T11:40:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:01.825+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7faae40b67f0 con 0x7faae4070630 2024-04-25T11:40:02.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.187+0000 7faadb400640 1 -- 172.21.15.19:0/1463083876 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v288) ==== 162+0+0 (secure 0 0 0) 0x7faad005f7a0 con 0x7faae4070630 2024-04-25T11:40:02.174 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:40:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faac40744b0 msgr2=0x7faac40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faac40744b0 0x7faac40769c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7faad4034000 tx=0x7faad402e000 comp rx=0 tx=0).stop 2024-04-25T11:40:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 msgr2=0x7faae40af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40af140 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7faad000d5f0 tx=0x7faad000dab0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 shutdown_connections 2024-04-25T11:40:02.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faac40744b0 0x7faac40769c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faae415adf0 0x7faae40af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faae4070630 0x7faae40af140 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 --2- 172.21.15.19:0/1463083876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faae406db10 0x7faae414b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.190+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 >> 172.21.15.19:0/1463083876 conn(0x7faae4073090 msgr2=0x7faae40b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.191+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 shutdown_connections 2024-04-25T11:40:02.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.191+0000 7faaea600640 1 -- 172.21.15.19:0/1463083876 wait complete. 2024-04-25T11:40:02.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.920+0000 7fc839a00640 1 Processor -- start 2024-04-25T11:40:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.920+0000 7fc839a00640 1 -- start start 2024-04-25T11:40:02.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.922+0000 7fc839a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8341443c0 0x7fc83406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.922+0000 7fc839a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc834070370 0x7fc8340cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.922+0000 7fc839a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8340cff30 con 0x7fc8340cfb20 2024-04-25T11:40:02.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.922+0000 7fc839a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc8340708d0 con 0x7fc8341443c0 2024-04-25T11:40:02.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.922+0000 7fc839a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc83415f160 con 0x7fc834070370 2024-04-25T11:40:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.923+0000 7fc832000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc834070370 0x7fc8340cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.923+0000 7fc832a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8341443c0 0x7fc83406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.923+0000 7fc832000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc834070370 0x7fc8340cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52008/0 (socket says 172.21.15.19:52008) 2024-04-25T11:40:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.923+0000 7fc832000640 1 -- 172.21.15.19:0/2133716546 learned_addr learned my addr 172.21.15.19:0/2133716546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.924+0000 7fc831600640 1 -- 172.21.15.19:0/2133716546 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997335412 0 0) 0x7fc82c001110 con 0x7fc8340cfb20 2024-04-25T11:40:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.924+0000 7fc831600640 1 -- 172.21.15.19:0/2133716546 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc818003520 con 0x7fc8340cfb20 2024-04-25T11:40:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.924+0000 7fc832a00640 1 -- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc834070370 msgr2=0x7fc8340cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.924+0000 7fc832a00640 1 --2- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc834070370 0x7fc8340cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.924+0000 7fc832a00640 1 -- 172.21.15.19:0/2133716546 >> v1:172.21.15.169:6789/0 conn(0x7fc8340cfb20 legacy=0x7fc8340ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.924+0000 7fc832a00640 1 -- 172.21.15.19:0/2133716546 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83415f5f0 con 0x7fc8341443c0 2024-04-25T11:40:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.924+0000 7fc832000640 1 --2- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc834070370 0x7fc8340cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:40:02.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.925+0000 7fc832a00640 1 --2- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8341443c0 0x7fc83406fe30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc82400f370 tx=0x7fc82402ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6923ea57454df63d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:02.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.926+0000 7fc831600640 1 -- 172.21.15.19:0/2133716546 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824036070 con 0x7fc8341443c0 2024-04-25T11:40:02.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.926+0000 7fc831600640 1 -- 172.21.15.19:0/2133716546 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82400d4e0 con 0x7fc8341443c0 2024-04-25T11:40:02.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.926+0000 7fc831600640 1 -- 172.21.15.19:0/2133716546 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400b960 con 0x7fc8341443c0 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 -- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8341443c0 msgr2=0x7fc83406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 --2- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8341443c0 0x7fc83406fe30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc82400f370 tx=0x7fc82402ea70 comp rx=0 tx=0).stop 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 -- 172.21.15.19:0/2133716546 shutdown_connections 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 --2- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc834070370 0x7fc8340cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 --2- 172.21.15.19:0/2133716546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8341443c0 0x7fc83406fe30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 -- 172.21.15.19:0/2133716546 >> 172.21.15.19:0/2133716546 conn(0x7fc8340d1ae0 msgr2=0x7fc8340d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 -- 172.21.15.19:0/2133716546 shutdown_connections 2024-04-25T11:40:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.927+0000 7fc839a00640 1 -- 172.21.15.19:0/2133716546 wait complete. 2024-04-25T11:40:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.929+0000 7fc839a00640 1 Processor -- start 2024-04-25T11:40:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.930+0000 7fc839a00640 1 -- start start 2024-04-25T11:40:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc839a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc834070370 0x7fc83413d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc839a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8340cfb20 0x7fc83415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc839a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8341443c0 0x7fc8341f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc839a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc83415edf0 con 0x7fc8340cfb20 2024-04-25T11:40:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc839a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc83415eb10 con 0x7fc834070370 2024-04-25T11:40:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc839a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc83406d5d0 con 0x7fc8341443c0 2024-04-25T11:40:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc832000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8340cfb20 0x7fc83415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc832000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8340cfb20 0x7fc83415b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47948/0 (socket says 172.21.15.19:47948) 2024-04-25T11:40:02.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.931+0000 7fc832000640 1 -- 172.21.15.19:0/1363070001 learned_addr learned my addr 172.21.15.19:0/1363070001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc833400640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8341443c0 0x7fc8341f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc832a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc834070370 0x7fc83413d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc832000640 1 -- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8341443c0 msgr2=0x7fc8341f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc832000640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8341443c0 0x7fc8341f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc832000640 1 -- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc834070370 msgr2=0x7fc83413d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc832000640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc834070370 0x7fc83413d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc832000640 1 -- 172.21.15.19:0/1363070001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8340b4af0 con 0x7fc8340cfb20 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.932+0000 7fc832000640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8340cfb20 0x7fc83415b220 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fc82000e980 tx=0x7fc82000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.933+0000 7fc82aa00640 1 -- 172.21.15.19:0/1363070001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820017070 con 0x7fc8340cfb20 2024-04-25T11:40:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.933+0000 7fc82aa00640 1 -- 172.21.15.19:0/1363070001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82000ca00 con 0x7fc8340cfb20 2024-04-25T11:40:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.933+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8340b4d90 con 0x7fc8340cfb20 2024-04-25T11:40:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.933+0000 7fc82aa00640 1 -- 172.21.15.19:0/1363070001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82000cdd0 con 0x7fc8340cfb20 2024-04-25T11:40:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.933+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc834141b90 con 0x7fc8340cfb20 2024-04-25T11:40:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.934+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc834050740 con 0x7fc8340cfb20 2024-04-25T11:40:02.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.935+0000 7fc82aa00640 1 -- 172.21.15.19:0/1363070001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc82001a070 con 0x7fc8340cfb20 2024-04-25T11:40:02.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.936+0000 7fc82aa00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8140744e0 0x7fc8140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:02.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.936+0000 7fc82aa00640 1 -- 172.21.15.19:0/1363070001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7fc820095cd0 con 0x7fc8340cfb20 2024-04-25T11:40:02.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.937+0000 7fc832a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8140744e0 0x7fc8140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:02.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.938+0000 7fc832a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8140744e0 0x7fc8140769f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc824038000 tx=0x7fc82400d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:02.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:02.941+0000 7fc82aa00640 1 -- 172.21.15.19:0/1363070001 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc82005f940 con 0x7fc8340cfb20 2024-04-25T11:40:03.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.188+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc83406f950 con 0x7fc8340cfb20 2024-04-25T11:40:03.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.189+0000 7fc82aa00640 1 -- 172.21.15.19:0/1363070001 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) ==== 95+0+5190 (secure 0 0 0) 0x7fc82005f240 con 0x7fc8340cfb20 2024-04-25T11:40:03.176 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:03.176 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":289,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":289,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:02.193327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10296":{"gid":10296,"name":"a","rank":0,"incarnation":289,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/2891720740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2891720740},{"type":"v1","addr":"172.21.15.19:6835","nonce":2891720740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:40:03.177 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 289 2024-04-25T11:40:03.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8140744e0 msgr2=0x7fc8140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:03.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8140744e0 0x7fc8140769f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc824038000 tx=0x7fc82400d000 comp rx=0 tx=0).stop 2024-04-25T11:40:03.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8340cfb20 msgr2=0x7fc83415b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:03.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8340cfb20 0x7fc83415b220 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fc82000e980 tx=0x7fc82000ee40 comp rx=0 tx=0).stop 2024-04-25T11:40:03.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 shutdown_connections 2024-04-25T11:40:03.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8140744e0 0x7fc8140769f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8341443c0 0x7fc8341f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8340cfb20 0x7fc83415b220 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 --2- 172.21.15.19:0/1363070001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc834070370 0x7fc83413d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:03.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.192+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 >> 172.21.15.19:0/1363070001 conn(0x7fc8340d1ae0 msgr2=0x7fc83415d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:03.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.193+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 shutdown_connections 2024-04-25T11:40:03.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:03.193+0000 7fc839a00640 1 -- 172.21.15.19:0/1363070001 wait complete. 2024-04-25T11:40:03.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 289, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:40:02.193327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10296': {'gid': 10296, 'name': 'a', 'rank': 0, 'incarnation': 289, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/2891720740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2891720740}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2891720740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:40:03.209 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10296:up:replay 2024-04-25T11:40:03.209 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:40:06.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:06.299 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:07.151 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:40:07.164+0000 7f4a5fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:40:08.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.917+0000 7f1855400640 1 Processor -- start 2024-04-25T11:40:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.917+0000 7f1855400640 1 -- start start 2024-04-25T11:40:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.919+0000 7f1855400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f18500cf830 0x7f185015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.919+0000 7f1855400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185015ecb0 con 0x7f18500d1540 2024-04-25T11:40:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.919+0000 7f1855400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f185015ef90 con 0x7f18500ce6c0 2024-04-25T11:40:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.919+0000 7f1855400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f185015f270 con 0x7f18500cf830 2024-04-25T11:40:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.920+0000 7f184fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f18500cf830 0x7f185015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.920+0000 7f1854a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f18500d1540 0x7f18500d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56384/0 (socket says 172.21.15.19:56384) 2024-04-25T11:40:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.920+0000 7f1854a00640 1 -- 172.21.15.19:0/2311857657 learned_addr learned my addr 172.21.15.19:0/2311857657 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.920+0000 7f184fe00640 1 -- 172.21.15.19:0/2311857657 >> v1:172.21.15.169:6789/0 conn(0x7f18500ce6c0 legacy=0x7f18500b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.920+0000 7f184fe00640 1 -- 172.21.15.19:0/2311857657 >> v1:172.21.15.19:6789/0 conn(0x7f18500d1540 legacy=0x7f18500d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.920+0000 7f184fe00640 1 -- 172.21.15.19:0/2311857657 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185015f550 con 0x7f18500cf830 2024-04-25T11:40:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.920+0000 7f184ea00640 1 -- 172.21.15.19:0/2311857657 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198865682 0 0) 0x7f185015ecb0 con 0x7f18500d1540 2024-04-25T11:40:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.922+0000 7f184fe00640 1 --2- 172.21.15.19:0/2311857657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500cf830 0x7f185015e6d0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f1840009790 tx=0x7f184002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f15cd2344472a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.922+0000 7f184ea00640 1 -- 172.21.15.19:0/2311857657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037070 con 0x7f18500cf830 2024-04-25T11:40:08.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.922+0000 7f184ea00640 1 -- 172.21.15.19:0/2311857657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1840032440 con 0x7f18500cf830 2024-04-25T11:40:08.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.922+0000 7f184ea00640 1 -- 172.21.15.19:0/2311857657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840032810 con 0x7f18500cf830 2024-04-25T11:40:08.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.923+0000 7f1855400640 1 -- 172.21.15.19:0/2311857657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500cf830 msgr2=0x7f185015e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.923+0000 7f1855400640 1 --2- 172.21.15.19:0/2311857657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500cf830 0x7f185015e6d0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f1840009790 tx=0x7f184002a210 comp rx=0 tx=0).stop 2024-04-25T11:40:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.923+0000 7f1855400640 1 -- 172.21.15.19:0/2311857657 shutdown_connections 2024-04-25T11:40:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.923+0000 7f1855400640 1 --2- 172.21.15.19:0/2311857657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500cf830 0x7f185015e6d0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.923+0000 7f1855400640 1 -- 172.21.15.19:0/2311857657 >> 172.21.15.19:0/2311857657 conn(0x7f18500b5460 msgr2=0x7f18500b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.923+0000 7f1855400640 1 -- 172.21.15.19:0/2311857657 shutdown_connections 2024-04-25T11:40:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.923+0000 7f1855400640 1 -- 172.21.15.19:0/2311857657 wait complete. 2024-04-25T11:40:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.925+0000 7f1855400640 1 Processor -- start 2024-04-25T11:40:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.926+0000 7f1855400640 1 -- start start 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.927+0000 7f1855400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18500ce6c0 0x7f18500b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.927+0000 7f1855400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18500cf830 0x7f185015de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.927+0000 7f1855400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500d1540 0x7f1850160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.927+0000 7f1855400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f185008d280 con 0x7f18500d1540 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.927+0000 7f1855400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f18500cead0 con 0x7f18500ce6c0 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.927+0000 7f1855400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f185008d400 con 0x7f18500cf830 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500d1540 0x7f1850160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500d1540 0x7f1850160770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41582/0 (socket says 172.21.15.19:41582) 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 -- 172.21.15.19:0/3344623062 learned_addr learned my addr 172.21.15.19:0/3344623062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f184fe00640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18500ce6c0 0x7f18500b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f184f400640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18500cf830 0x7f185015de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 -- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18500cf830 msgr2=0x7f185015de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18500cf830 0x7f185015de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 -- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18500ce6c0 msgr2=0x7f18500b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18500ce6c0 0x7f18500b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.928+0000 7f1854a00640 1 -- 172.21.15.19:0/3344623062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18500b3740 con 0x7f18500d1540 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.929+0000 7f1854a00640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500d1540 0x7f1850160770 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f184800c410 tx=0x7f184800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.929+0000 7f1847e00640 1 -- 172.21.15.19:0/3344623062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480074e0 con 0x7f18500d1540 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.929+0000 7f1847e00640 1 -- 172.21.15.19:0/3344623062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18480047e0 con 0x7f18500d1540 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.929+0000 7f1847e00640 1 -- 172.21.15.19:0/3344623062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848004bd0 con 0x7f18500d1540 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.929+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185015e360 con 0x7f18500d1540 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.929+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18500b3180 con 0x7f18500d1540 2024-04-25T11:40:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.930+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850050740 con 0x7f18500d1540 2024-04-25T11:40:08.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.931+0000 7f1847e00640 1 -- 172.21.15.19:0/3344623062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1848011070 con 0x7f18500d1540 2024-04-25T11:40:08.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.932+0000 7f1847e00640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f182c0744e0 0x7f182c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:08.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.933+0000 7f1847e00640 1 -- 172.21.15.19:0/3344623062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f1848095850 con 0x7f18500d1540 2024-04-25T11:40:08.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.933+0000 7f184fe00640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f182c0744e0 0x7f182c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:08.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.934+0000 7f184fe00640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f182c0744e0 0x7f182c0769f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1840034000 tx=0x7f184002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:08.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:08.938+0000 7f1847e00640 1 -- 172.21.15.19:0/3344623062 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f184805f550 con 0x7f18500d1540 2024-04-25T11:40:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.180+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18500b6370 con 0x7f18500d1540 2024-04-25T11:40:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.181+0000 7f1847e00640 1 -- 172.21.15.19:0/3344623062 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 292 v292) ==== 95+0+5199 (secure 0 0 0) 0x7f184805ee50 con 0x7f18500d1540 2024-04-25T11:40:09.168 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:09.168 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":292,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":292,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:05.235512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10296":{"gid":10296,"name":"a","rank":0,"incarnation":289,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6835/2891720740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2891720740},{"type":"v1","addr":"172.21.15.19:6835","nonce":2891720740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10296,"qdb_cluster":[10296]},"id":6}]} 2024-04-25T11:40:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 292 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f182c0744e0 msgr2=0x7f182c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f182c0744e0 0x7f182c0769f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1840034000 tx=0x7f184002e000 comp rx=0 tx=0).stop 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500d1540 msgr2=0x7f1850160770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500d1540 0x7f1850160770 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f184800c410 tx=0x7f184800c8d0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 shutdown_connections 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f182c0744e0 0x7f182c0769f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18500d1540 0x7f1850160770 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f18500cf830 0x7f185015de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 --2- 172.21.15.19:0/3344623062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f18500ce6c0 0x7f18500b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 >> 172.21.15.19:0/3344623062 conn(0x7f18500b5460 msgr2=0x7f18500b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.184+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 shutdown_connections 2024-04-25T11:40:09.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.185+0000 7f1855400640 1 -- 172.21.15.19:0/3344623062 wait complete. 2024-04-25T11:40:09.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 292, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:40:05.235512+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10296': {'gid': 10296, 'name': 'a', 'rank': 0, 'incarnation': 289, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6835/2891720740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2891720740}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2891720740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10296, 'qdb_cluster': [10296]} 2024-04-25T11:40:09.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:40:09.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:40:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.506+0000 7f1a98a00640 1 Processor -- start 2024-04-25T11:40:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.506+0000 7f1a98a00640 1 -- start start 2024-04-25T11:40:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a98a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1a941443c0 0x7f1a9406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a98a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a94070370 0x7f1a940cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a940ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a98a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1a940708d0 con 0x7f1a94070370 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a98a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1a9415f160 con 0x7f1a941443c0 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a98a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a940cff30 con 0x7f1a940cfb20 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a89400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a94070370 0x7f1a940cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a89400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1a94070370 0x7f1a940cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41588/0 (socket says 172.21.15.19:41588) 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a89400640 1 -- 172.21.15.19:0/2866057349 learned_addr learned my addr 172.21.15.19:0/2866057349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a91e00640 1 --2- 172.21.15.19:0/2866057349 >> v2:172.21.15.169:3300/0 conn(0x7f1a941443c0 0x7f1a9406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.508+0000 7f1a92800640 1 --2- 172.21.15.19:0/2866057349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a940ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.509+0000 7f1a89400640 1 -- 172.21.15.19:0/2866057349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 msgr2=0x7f1a940ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.509+0000 7f1a89400640 1 --2- 172.21.15.19:0/2866057349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a940ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.509+0000 7f1a89400640 1 -- 172.21.15.19:0/2866057349 >> v2:172.21.15.169:3300/0 conn(0x7f1a941443c0 msgr2=0x7f1a9406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.509+0000 7f1a89400640 1 --2- 172.21.15.19:0/2866057349 >> v2:172.21.15.169:3300/0 conn(0x7f1a941443c0 0x7f1a9406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.509+0000 7f1a89400640 1 -- 172.21.15.19:0/2866057349 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9415f5f0 con 0x7f1a94070370 2024-04-25T11:40:09.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.510+0000 7f1a89400640 1 --2- 172.21.15.19:0/2866057349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a94070370 0x7f1a940cf5e0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f1a94072820 tx=0x7f1a7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e1fab14a9e159e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.510+0000 7f1a91400640 1 -- 172.21.15.19:0/2866057349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c037070 con 0x7f1a94070370 2024-04-25T11:40:09.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.510+0000 7f1a91400640 1 -- 172.21.15.19:0/2866057349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7c032440 con 0x7f1a94070370 2024-04-25T11:40:09.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.510+0000 7f1a91400640 1 -- 172.21.15.19:0/2866057349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c032810 con 0x7f1a94070370 2024-04-25T11:40:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 -- 172.21.15.19:0/2866057349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a94070370 msgr2=0x7f1a940cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2866057349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a94070370 0x7f1a940cf5e0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f1a94072820 tx=0x7f1a7c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 -- 172.21.15.19:0/2866057349 shutdown_connections 2024-04-25T11:40:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2866057349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a94070370 0x7f1a940cf5e0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2866057349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a940ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2866057349 >> v2:172.21.15.169:3300/0 conn(0x7f1a941443c0 0x7f1a9406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 -- 172.21.15.19:0/2866057349 >> 172.21.15.19:0/2866057349 conn(0x7f1a940d1ae0 msgr2=0x7f1a940d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 -- 172.21.15.19:0/2866057349 shutdown_connections 2024-04-25T11:40:09.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.511+0000 7f1a98a00640 1 -- 172.21.15.19:0/2866057349 wait complete. 2024-04-25T11:40:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.514+0000 7f1a98a00640 1 Processor -- start 2024-04-25T11:40:09.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.514+0000 7f1a98a00640 1 -- start start 2024-04-25T11:40:09.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.515+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a94070370 0x7f1a94143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.515+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a9415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.515+0000 7f1a98a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941443c0 0x7f1a940b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.515+0000 7f1a98a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9415edf0 con 0x7f1a941443c0 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.515+0000 7f1a98a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9415eb10 con 0x7f1a94070370 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.515+0000 7f1a98a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a941447d0 con 0x7f1a940cfb20 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941443c0 0x7f1a940b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941443c0 0x7f1a940b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41596/0 (socket says 172.21.15.19:41596) 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 -- 172.21.15.19:0/2859053509 learned_addr learned my addr 172.21.15.19:0/2859053509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a91e00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a94070370 0x7f1a94143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a89400640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a9415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 -- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 msgr2=0x7f1a9415ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a9415ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 -- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a94070370 msgr2=0x7f1a94143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a94070370 0x7f1a94143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 -- 172.21.15.19:0/2859053509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a94154e50 con 0x7f1a941443c0 2024-04-25T11:40:09.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.516+0000 7f1a92800640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941443c0 0x7f1a940b5ac0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00d530 tx=0x7f1a8c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.517+0000 7f1a8aa00640 1 -- 172.21.15.19:0/2859053509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c018020 con 0x7f1a941443c0 2024-04-25T11:40:09.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.517+0000 7f1a8aa00640 1 -- 172.21.15.19:0/2859053509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c014620 con 0x7f1a941443c0 2024-04-25T11:40:09.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.517+0000 7f1a8aa00640 1 -- 172.21.15.19:0/2859053509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c014a10 con 0x7f1a941443c0 2024-04-25T11:40:09.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.517+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a94155130 con 0x7f1a941443c0 2024-04-25T11:40:09.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.517+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a941556f0 con 0x7f1a941443c0 2024-04-25T11:40:09.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.521+0000 7f1a8aa00640 1 -- 172.21.15.19:0/2859053509 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a8c01b020 con 0x7f1a941443c0 2024-04-25T11:40:09.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.522+0000 7f1a8aa00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a700744e0 0x7f1a700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.522+0000 7f1a8aa00640 1 -- 172.21.15.19:0/2859053509 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f1a8c0960d0 con 0x7f1a941443c0 2024-04-25T11:40:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.522+0000 7f1a9fa33740 1 Processor -- start 2024-04-25T11:40:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.522+0000 7f1a9fa33740 1 -- start start 2024-04-25T11:40:09.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.522+0000 7f1a91e00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a700744e0 0x7f1a700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a91e00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a700744e0 0x7f1a700769f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1a940d1860 tx=0x7f1a84003990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059aa270 0x55ba05d77040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d91d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba05cb0600 con 0x55ba05989590 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba05c4d100 con 0x55ba059aa270 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba05cac5c0 con 0x55ba059813b0 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d922b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41598/0 (socket says 172.21.15.19:41598) 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a92800640 1 -- 172.21.15.19:0/2579067617 learned_addr learned my addr 172.21.15.19:0/2579067617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a91e00640 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059aa270 0x55ba05d77040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a89400640 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d91d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a92800640 1 -- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 msgr2=0x55ba05d91d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a92800640 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d91d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a92800640 1 -- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059aa270 msgr2=0x55ba05d77040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.524+0000 7f1a92800640 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059aa270 0x55ba05d77040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.525+0000 7f1a92800640 1 -- 172.21.15.19:0/2579067617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba059c22d0 con 0x55ba05989590 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.525+0000 7f1a92800640 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d922b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c05b720 tx=0x7f1a8c05eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc4b46eef34c979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.525+0000 7f1a82600640 1 -- 172.21.15.19:0/2579067617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c05f6a0 con 0x55ba05989590 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.525+0000 7f1a82600640 1 -- 172.21.15.19:0/2579067617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c063a50 con 0x55ba05989590 2024-04-25T11:40:09.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.525+0000 7f1a82600640 1 -- 172.21.15.19:0/2579067617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c014dd0 con 0x55ba05989590 2024-04-25T11:40:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 -- 172.21.15.19:0/2579067617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 msgr2=0x55ba05d922b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d922b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c05b720 tx=0x7f1a8c05eed0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 -- 172.21.15.19:0/2579067617 shutdown_connections 2024-04-25T11:40:09.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d922b0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d91d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/2579067617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059aa270 0x55ba05d77040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 -- 172.21.15.19:0/2579067617 >> 172.21.15.19:0/2579067617 conn(0x55ba0599e3c0 msgr2=0x55ba05d4c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 -- 172.21.15.19:0/2579067617 shutdown_connections 2024-04-25T11:40:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.526+0000 7f1a9fa33740 1 -- 172.21.15.19:0/2579067617 wait complete. 2024-04-25T11:40:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.527+0000 7f1a9fa33740 1 -- start start 2024-04-25T11:40:09.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.528+0000 7f1a9fa33740 1 Processor -- start 2024-04-25T11:40:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.529+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d8cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.529+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba05989590 0x55ba05d8df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.529+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05d8e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.529+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba05c37f70 con 0x55ba059aa270 2024-04-25T11:40:09.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.529+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba05d90420 con 0x55ba05989590 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.529+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba05d8b8b0 con 0x55ba059813b0 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.529+0000 7f1a9fa33740 10 client.0 _ll_register_callbacks cb 0x55ba05ca6480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05d8e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05d8e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41614/0 (socket says 172.21.15.19:41614) 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 -- 172.21.15.19:0/322140448 learned_addr learned my addr 172.21.15.19:0/322140448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a89400640 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba05989590 0x55ba05d8df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a91e00640 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d8cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 -- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 msgr2=0x55ba05d8cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d8cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 -- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba05989590 msgr2=0x55ba05d8df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba05989590 0x55ba05d8df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 -- 172.21.15.19:0/322140448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba05a8c6b0 con 0x55ba059aa270 2024-04-25T11:40:09.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a92800640 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05d8e450 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c05f3a0 tx=0x7f1a8c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a44ee56ddffba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.530+0000 7f1a76a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a76a00640 1 -- 172.21.15.19:0/322140448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c07c060 con 0x55ba059aa270 2024-04-25T11:40:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a76a00640 1 -- 172.21.15.19:0/322140448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f1a8c014db0 con 0x55ba059aa270 2024-04-25T11:40:09.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8c063a50 con 0x55ba059aa270 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a76a00640 1 -- 172.21.15.19:0/322140448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=292) ==== 24+0+0 (secure 0 0 0) 0x7f1a8c05f6a0 con 0x55ba059aa270 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a9fa33740 10 client.10383 fetch_fsmap learned FSMap version 292 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=292,monmap=2+}) -- 0x55ba05d8d220 con 0x55ba059aa270 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a76a00640 1 -- 172.21.15.19:0/322140448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c063a50 con 0x55ba059aa270 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a76a00640 1 -- 172.21.15.19:0/322140448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 292) ==== 3494+0+0 (secure 0 0 0) 0x7f1a8c08cce0 con 0x55ba059aa270 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a9fa33740 10 client.10383 fetch_fsmap finished waiting for FSMap version 292 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a9fa33740 10 client.10383 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:09.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.531+0000 7f1a9fa33740 20 client.10383 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.532+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05968e40 0x55ba05da49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.532+0000 7f1a9fa33740 4 client.10383 mds_command: new command op to 10296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.532+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 --> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba05cbe210 con 0x55ba05968e40 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.532+0000 7f1a91e00640 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05968e40 0x55ba05da49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.532+0000 7f1a91e00640 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05968e40 0x55ba05da49d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d59b56343c16d8e server_cookie=c5ab4b98cb363ed4 in_seq=0 out_seq=0 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.532+0000 7f1a76a00640 10 client.10383 ms_handle_connect on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.533+0000 7f1a76a00640 1 -- 172.21.15.19:0/322140448 <== mds.0 v2:172.21.15.19:6834/2891720740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba05cbe210 con 0x55ba05968e40 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.533+0000 7f1a76a00640 10 client.10383 handle_command_reply: tid=0 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.533+0000 7f1a9fa33740 1 client.10383 shutdown 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 msgr2=0x55ba05d8e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05d8e450 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c05f3a0 tx=0x7f1a8c012000 comp rx=0 tx=0).stop 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 shutdown_connections 2024-04-25T11:40:09.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05968e40 0x55ba05da49d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba05989590 0x55ba05d8df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05d8e450 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/322140448 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059813b0 0x55ba05d8cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 >> 172.21.15.19:0/322140448 conn(0x55ba0599e3c0 msgr2=0x55ba05d73d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a76a00640 0 client.10383 ms_handle_reset on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 shutdown_connections 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 1 -- 172.21.15.19:0/322140448 wait complete. 2024-04-25T11:40:09.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.534+0000 7f1a9fa33740 20 client.10383 trim_cache size 0 max 16384 2024-04-25T11:40:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.555+0000 7f1a9fa33740 1 Processor -- start 2024-04-25T11:40:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.555+0000 7f1a9fa33740 1 -- start start 2024-04-25T11:40:09.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05d79a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba05989590 0x55ba05d79f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05db4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94203950 con 0x55ba059aa270 2024-04-25T11:40:09.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1a94202ce0 con 0x55ba059813b0 2024-04-25T11:40:09.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1a94203300 con 0x55ba05989590 2024-04-25T11:40:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05db4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a92800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05db4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41628/0 (socket says 172.21.15.19:41628) 2024-04-25T11:40:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a92800640 1 -- 172.21.15.19:0/3680475791 learned_addr learned my addr 172.21.15.19:0/3680475791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a91e00640 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05d79a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.557+0000 7f1a89400640 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba05989590 0x55ba05d79f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.558+0000 7f1a92800640 1 -- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba05989590 msgr2=0x55ba05d79f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.558+0000 7f1a92800640 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba05989590 0x55ba05d79f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.558+0000 7f1a92800640 1 -- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 msgr2=0x55ba05d79a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.558+0000 7f1a92800640 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05d79a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.558+0000 7f1a92800640 1 -- 172.21.15.19:0/3680475791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba05d725e0 con 0x55ba059aa270 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.558+0000 7f1a91e00640 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05d79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.558+0000 7f1a92800640 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05db4d30 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c05b720 tx=0x7f1a8c08e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7a3007ade65720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.559+0000 7f1a81c00640 1 -- 172.21.15.19:0/3680475791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c07c070 con 0x55ba059aa270 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.559+0000 7f1a81c00640 1 -- 172.21.15.19:0/3680475791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c063a50 con 0x55ba059aa270 2024-04-25T11:40:09.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.559+0000 7f1a81c00640 1 -- 172.21.15.19:0/3680475791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c014df0 con 0x55ba059aa270 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.559+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3680475791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 msgr2=0x55ba05db4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.559+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05db4d30 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c05b720 tx=0x7f1a8c08e040 comp rx=0 tx=0).stop 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.560+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3680475791 shutdown_connections 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.560+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba059aa270 0x55ba05db4d30 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.560+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba05989590 0x55ba05d79f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.560+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3680475791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05d79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.560+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3680475791 >> 172.21.15.19:0/3680475791 conn(0x55ba0599e3c0 msgr2=0x55ba05d48a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.560+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3680475791 shutdown_connections 2024-04-25T11:40:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.560+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3680475791 wait complete. 2024-04-25T11:40:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.561+0000 7f1a9fa33740 1 -- start start 2024-04-25T11:40:09.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.561+0000 7f1a9fa33740 1 Processor -- start 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.562+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05df83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.562+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a9fa33740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059aa270 0x55ba05d73ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba05d901f0 con 0x55ba05989590 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba05c37f70 con 0x55ba059813b0 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a9fa33740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba05d8a050 con 0x55ba059aa270 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a9fa33740 10 client.0 _ll_register_callbacks cb 0x7f1a8c05eff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:09.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d73590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41644/0 (socket says 172.21.15.19:41644) 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 -- 172.21.15.19:0/3961532230 learned_addr learned my addr 172.21.15.19:0/3961532230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a91e00640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05df83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a92800640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059aa270 0x55ba05d73ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 -- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059aa270 msgr2=0x55ba05d73ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059aa270 0x55ba05d73ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 -- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 msgr2=0x55ba05df83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05df83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a89400640 1 -- 172.21.15.19:0/3961532230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba05d8e320 con 0x55ba05989590 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.563+0000 7f1a92800640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059aa270 0x55ba05d73ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a89400640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d73590 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c000c00 tx=0x7f1a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de377ea621dd440f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a76a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:09.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a76a00640 1 -- 172.21.15.19:0/3961532230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c042070 con 0x55ba05989590 2024-04-25T11:40:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a76a00640 1 -- 172.21.15.19:0/3961532230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f1a7c032d80 con 0x55ba05989590 2024-04-25T11:40:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8c063a50 con 0x55ba05989590 2024-04-25T11:40:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a76a00640 1 -- 172.21.15.19:0/3961532230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=292) ==== 24+0+0 (secure 0 0 0) 0x7f1a7c037040 con 0x55ba05989590 2024-04-25T11:40:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a9fa33740 10 client.10389 fetch_fsmap learned FSMap version 292 2024-04-25T11:40:09.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.564+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=292,monmap=2+}) -- 0x55ba05db5110 con 0x55ba05989590 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a76a00640 1 -- 172.21.15.19:0/3961532230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c063a50 con 0x55ba05989590 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a76a00640 1 -- 172.21.15.19:0/3961532230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 292) ==== 3494+0+0 (secure 0 0 0) 0x7f1a7c03c410 con 0x55ba05989590 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a9fa33740 10 client.10389 fetch_fsmap finished waiting for FSMap version 292 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a9fa33740 10 client.10389 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a9fa33740 20 client.10389 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05db4090 0x55ba05db44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a9fa33740 4 client.10389 mds_command: new command op to 10296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 --> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] -- command(tid 0: {"prefix": "status"}) -- 0x55ba05d8fc80 con 0x55ba05db4090 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.565+0000 7f1a91e00640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05db4090 0x55ba05db44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.566+0000 7f1a91e00640 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05db4090 0x55ba05db44a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8fe55963f78482 server_cookie=57f389098b86da0f in_seq=0 out_seq=0 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.566+0000 7f1a76a00640 10 client.10389 ms_handle_connect on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.566+0000 7f1a76a00640 1 -- 172.21.15.19:0/3961532230 <== mds.0 v2:172.21.15.19:6834/2891720740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55ba05d8fc80 con 0x55ba05db4090 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.566+0000 7f1a76a00640 10 client.10389 handle_command_reply: tid=0 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.566+0000 7f1a9fa33740 1 client.10389 shutdown 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.566+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 msgr2=0x55ba05d73590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.566+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d73590 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c000c00 tx=0x7f1a7c004730 comp rx=0 tx=0).stop 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 shutdown_connections 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x55ba05db4090 0x55ba05db44a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba059aa270 0x55ba05d73ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba05989590 0x55ba05d73590 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 --2- 172.21.15.19:0/3961532230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba059813b0 0x55ba05df83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 >> 172.21.15.19:0/3961532230 conn(0x55ba0599e3c0 msgr2=0x55ba05d8ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a76a00640 0 client.10389 ms_handle_reset on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:09.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 shutdown_connections 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 1 -- 172.21.15.19:0/3961532230 wait complete. 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.567+0000 7f1a9fa33740 20 client.10389 trim_cache size 0 max 16384 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 10296, 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:40:09.557 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3660861899999999, 2024-04-25T11:40:09.558 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 292, 2024-04-25T11:40:09.558 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 123, 2024-04-25T11:40:09.558 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 123, 2024-04-25T11:40:09.558 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.275567111999999 2024-04-25T11:40:09.558 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:40:09.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a700744e0 msgr2=0x7f1a700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a700744e0 0x7f1a700769f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1a940d1860 tx=0x7f1a84003990 comp rx=0 tx=0).stop 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941443c0 msgr2=0x7f1a940b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941443c0 0x7f1a940b5ac0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00d530 tx=0x7f1a8c00d9f0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 shutdown_connections 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1a700744e0 0x7f1a700769f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a941443c0 0x7f1a940b5ac0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a940cfb20 0x7f1a9415ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 --2- 172.21.15.19:0/2859053509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a94070370 0x7f1a94143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 >> 172.21.15.19:0/2859053509 conn(0x7f1a940d1ae0 msgr2=0x7f1a94153000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 shutdown_connections 2024-04-25T11:40:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.568+0000 7f1a98a00640 1 -- 172.21.15.19:0/2859053509 wait complete. 2024-04-25T11:40:09.579 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:40:09.579 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2024-04-25T11:40:09.580 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:40:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.870+0000 7f34a4a00640 1 Processor -- start 2024-04-25T11:40:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.871+0000 7f34a4a00640 1 -- start start 2024-04-25T11:40:09.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f34a4a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34a006d5d0 0x7f34a006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f34a4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a0141fd0 0x7f34a01423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f34a4a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a006df20 con 0x7f34a0140e00 2024-04-25T11:40:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f34a4a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f34a0141b40 con 0x7f34a006d5d0 2024-04-25T11:40:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f34a4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34a0141210 con 0x7f34a0141fd0 2024-04-25T11:40:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f3497e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34a006d5d0 0x7f34a006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f349f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f34a0140e00 0x7f34a0143550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56400/0 (socket says 172.21.15.19:56400) 2024-04-25T11:40:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f349f400640 1 -- 172.21.15.19:0/1698588705 learned_addr learned my addr 172.21.15.19:0/1698588705 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:09.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.873+0000 7f349fe00640 1 --2- 172.21.15.19:0/1698588705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a0141fd0 0x7f34a01423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.874+0000 7f3497e00640 1 -- 172.21.15.19:0/1698588705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a0141fd0 msgr2=0x7f34a01423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:09.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.874+0000 7f3497e00640 1 --2- 172.21.15.19:0/1698588705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a0141fd0 0x7f34a01423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.874+0000 7f3497e00640 1 -- 172.21.15.19:0/1698588705 >> v1:172.21.15.19:6789/0 conn(0x7f34a0140e00 legacy=0x7f34a0143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.874+0000 7f3497e00640 1 -- 172.21.15.19:0/1698588705 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a0141510 con 0x7f34a006d5d0 2024-04-25T11:40:09.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.875+0000 7f3497e00640 1 --2- 172.21.15.19:0/1698588705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a006d5d0 0x7f34a006d9e0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f34880097a0 tx=0x7f348802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255c7fb6a217f7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.875+0000 7f349ea00640 1 -- 172.21.15.19:0/1698588705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488037070 con 0x7f34a006d5d0 2024-04-25T11:40:09.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.875+0000 7f349ea00640 1 -- 172.21.15.19:0/1698588705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3488032440 con 0x7f34a006d5d0 2024-04-25T11:40:09.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.875+0000 7f349ea00640 1 -- 172.21.15.19:0/1698588705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488032810 con 0x7f34a006d5d0 2024-04-25T11:40:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 -- 172.21.15.19:0/1698588705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a006d5d0 msgr2=0x7f34a006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 --2- 172.21.15.19:0/1698588705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a006d5d0 0x7f34a006d9e0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f34880097a0 tx=0x7f348802a180 comp rx=0 tx=0).stop 2024-04-25T11:40:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 -- 172.21.15.19:0/1698588705 shutdown_connections 2024-04-25T11:40:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 --2- 172.21.15.19:0/1698588705 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a0141fd0 0x7f34a01423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 --2- 172.21.15.19:0/1698588705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a006d5d0 0x7f34a006d9e0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 -- 172.21.15.19:0/1698588705 >> 172.21.15.19:0/1698588705 conn(0x7f34a0072ee0 msgr2=0x7f34a00732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:09.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 -- 172.21.15.19:0/1698588705 shutdown_connections 2024-04-25T11:40:09.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.876+0000 7f34a4a00640 1 -- 172.21.15.19:0/1698588705 wait complete. 2024-04-25T11:40:09.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.879+0000 7f34a4a00640 1 Processor -- start 2024-04-25T11:40:09.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.879+0000 7f34a4a00640 1 -- start start 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.880+0000 7f34a4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a006d5d0 0x7f34a01450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f34a4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a0140e00 0x7f34a01391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f34a4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a0141fd0 0x7f34a01ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f34a4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01417d0 con 0x7f34a0141fd0 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f34a4a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01443d0 con 0x7f34a0140e00 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f34a4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f34a00d1490 con 0x7f34a006d5d0 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a0141fd0 0x7f34a01ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a0141fd0 0x7f34a01ebf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41666/0 (socket says 172.21.15.19:41666) 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349fe00640 1 -- 172.21.15.19:0/4017631613 learned_addr learned my addr 172.21.15.19:0/4017631613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349f400640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a006d5d0 0x7f34a01450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f3497e00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a0140e00 0x7f34a01391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349fe00640 1 -- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a006d5d0 msgr2=0x7f34a01450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349fe00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a006d5d0 0x7f34a01450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349fe00640 1 -- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a0140e00 msgr2=0x7f34a01391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.881+0000 7f349fe00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a0140e00 0x7f34a01391f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f349fe00640 1 -- 172.21.15.19:0/4017631613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a0145fa0 con 0x7f34a0141fd0 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f3497e00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a0140e00 0x7f34a01391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f349fe00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a0141fd0 0x7f34a01ebf80 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f3498007e60 tx=0x7f349800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f3497400640 1 -- 172.21.15.19:0/4017631613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498014420 con 0x7f34a0141fd0 2024-04-25T11:40:09.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f3497400640 1 -- 172.21.15.19:0/4017631613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3498019070 con 0x7f34a0141fd0 2024-04-25T11:40:09.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f3497400640 1 -- 172.21.15.19:0/4017631613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498014820 con 0x7f34a0141fd0 2024-04-25T11:40:09.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a0146280 con 0x7f34a0141fd0 2024-04-25T11:40:09.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.882+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a0156f60 con 0x7f34a0141fd0 2024-04-25T11:40:09.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.884+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a01457c0 con 0x7f34a0141fd0 2024-04-25T11:40:09.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.885+0000 7f3497400640 1 -- 172.21.15.19:0/4017631613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f34980149c0 con 0x7f34a0141fd0 2024-04-25T11:40:09.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.886+0000 7f3497400640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f347c0744b0 0x7f347c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.886+0000 7f3497400640 1 -- 172.21.15.19:0/4017631613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f3498096580 con 0x7f34a0141fd0 2024-04-25T11:40:09.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.887+0000 7f349f400640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f347c0744b0 0x7f347c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:09.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.888+0000 7f349f400640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f347c0744b0 0x7f347c0769c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3490000f60 tx=0x7f3490009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:09.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:09.891+0000 7f3497400640 1 -- 172.21.15.19:0/4017631613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f349805fc00 con 0x7f34a0141fd0 2024-04-25T11:40:10.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.138+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f34a0050740 con 0x7f34a0141fd0 2024-04-25T11:40:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.139+0000 7f3497400640 1 -- 172.21.15.19:0/4017631613 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v292) ==== 53+0+73 (secure 0 0 0) 0x7f3498063f00 con 0x7f34a0141fd0 2024-04-25T11:40:10.125 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.141+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f347c0744b0 msgr2=0x7f347c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.141+0000 7f34a4a00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f347c0744b0 0x7f347c0769c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3490000f60 tx=0x7f3490009040 comp rx=0 tx=0).stop 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.141+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a0141fd0 msgr2=0x7f34a01ebf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.141+0000 7f34a4a00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a0141fd0 0x7f34a01ebf80 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f3498007e60 tx=0x7f349800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 shutdown_connections 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f347c0744b0 0x7f347c0769c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34a0140e00 0x7f34a01391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34a0141fd0 0x7f34a01ebf80 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 --2- 172.21.15.19:0/4017631613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34a006d5d0 0x7f34a01450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 >> 172.21.15.19:0/4017631613 conn(0x7f34a0072ee0 msgr2=0x7f34a006f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 shutdown_connections 2024-04-25T11:40:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10.142+0000 7f34a4a00640 1 -- 172.21.15.19:0/4017631613 wait complete. 2024-04-25T11:40:10.153 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:40:10.248 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:40:10.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:10.249 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:40:10.306 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:40:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:40:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:40:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:10.307 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:40:10.309 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41863sec preferred_lft 41863sec 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:10.310 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:40:10.311 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:10.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:10.312 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:40:10.393 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:40:10.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:10.397 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:40:10.446 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:40:10.447 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:40:10.447 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:40:10.447 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:40:10.447 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:40:10.448 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:40:10.448 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:40:10.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:40:10.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:40:10.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:40:10.448 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:40:10.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:40:10.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:40:10.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:10.668 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:40:10.722 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:40:10.722 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:40:10.722 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:40:10.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:10.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:40:10.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:40:10.812 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:10.869 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:40:10.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:40:10.869 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:10.920 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:40:10.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:10.920 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:40:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:40:11.129 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:40:11.171 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:40:11.171 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:40:11.171 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:40:11.171 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:40:11.171 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:40:11.173 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:40:11.173 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:40:11.173 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:40:11.173 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:40:11.174 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:40:11.179 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:40:11.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:11.179 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:11.259 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:40:11.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:40:11.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:40:11.387 INFO:teuthology.orchestra.run.smithi019.stdout:client10401 2024-04-25T11:40:11.387 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:40:11.388 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:40:11.440 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:40:11.440 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:40:11.440 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T11:40:11.441 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:40:11.446 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:40:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:11 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-25T11:40:17.547 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-25T11:40:17.548 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-04-25T11:40:17.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:17.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.855+0000 7f76dd400640 1 Processor -- start 2024-04-25T11:40:17.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.855+0000 7f76dd400640 1 -- start start 2024-04-25T11:40:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76dd400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f76d80d2030 0x7f76d80b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76dd400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d80b3ef0 con 0x7f76d8164aa0 2024-04-25T11:40:17.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76dd400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d8163fc0 con 0x7f76d80cfb90 2024-04-25T11:40:17.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76dd400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f76d8164610 con 0x7f76d80d2030 2024-04-25T11:40:17.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76d7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f76d8164aa0 0x7f76d80d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40646/0 (socket says 172.21.15.19:40646) 2024-04-25T11:40:17.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76d7e00640 1 -- 172.21.15.19:0/3058179907 learned_addr learned my addr 172.21.15.19:0/3058179907 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76dca00640 1 --2- 172.21.15.19:0/3058179907 >> v2:172.21.15.169:3300/0 conn(0x7f76d80d2030 0x7f76d80b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.858+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554820519 0 0) 0x7f76d80b3ef0 con 0x7f76d8164aa0 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76bc003520 con 0x7f76d8164aa0 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3650648104 0 0) 0x7f76bc003520 con 0x7f76d8164aa0 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 >> v2:172.21.15.169:3300/0 conn(0x7f76d80d2030 msgr2=0x7f76d80b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 --2- 172.21.15.19:0/3058179907 >> v2:172.21.15.169:3300/0 conn(0x7f76d80d2030 0x7f76d80b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 >> v1:172.21.15.169:6789/0 conn(0x7f76d80cfb90 legacy=0x7f76d80b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d81642a0 con 0x7f76d8164aa0 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f76c8001a10 con 0x7f76d8164aa0 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 >> v1:172.21.15.19:6789/0 conn(0x7f76d8164aa0 legacy=0x7f76d80d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc0039b0 0x7f76bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76bc004800 0x7f76bc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76bc0051c0 0x7f76bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8164610 con 0x7f76bc004800 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc00c160 con 0x7f76bc0051c0 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.859+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76bc00c360 con 0x7f76bc0039b0 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76cf400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76bc004800 0x7f76bc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76d7e00640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc0039b0 0x7f76bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76cf400640 1 -- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc0039b0 msgr2=0x7f76bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76cf400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc0039b0 0x7f76bc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76cf400640 1 -- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76bc0051c0 msgr2=0x7f76bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76cf400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76bc0051c0 0x7f76bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76cf400640 1 -- 172.21.15.19:0/3058179907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc003520 con 0x7f76bc004800 2024-04-25T11:40:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76dca00640 1 -- 172.21.15.19:0/3058179907 reap_dead start 2024-04-25T11:40:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.860+0000 7f76d7e00640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc0039b0 0x7f76bc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.861+0000 7f76cf400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76bc004800 0x7f76bc004c10 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f76c0001750 tx=0x7f76bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798599277dff40d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.861+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0001370 con 0x7f76bc004800 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.861+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c00052d0 con 0x7f76bc004800 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.861+0000 7f76d7400640 1 -- 172.21.15.19:0/3058179907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c002abc0 con 0x7f76bc004800 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.862+0000 7f76dd400640 1 -- 172.21.15.19:0/3058179907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76bc004800 msgr2=0x7f76bc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.862+0000 7f76dd400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76bc004800 0x7f76bc004c10 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f76c0001750 tx=0x7f76bc005830 comp rx=0 tx=0).stop 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.862+0000 7f76dd400640 1 -- 172.21.15.19:0/3058179907 shutdown_connections 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.862+0000 7f76dd400640 1 --2- 172.21.15.19:0/3058179907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76bc004800 0x7f76bc004c10 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.862+0000 7f76dd400640 1 -- 172.21.15.19:0/3058179907 >> 172.21.15.19:0/3058179907 conn(0x7f76d809b7d0 msgr2=0x7f76d809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.862+0000 7f76dd400640 1 -- 172.21.15.19:0/3058179907 shutdown_connections 2024-04-25T11:40:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.862+0000 7f76dd400640 1 -- 172.21.15.19:0/3058179907 wait complete. 2024-04-25T11:40:17.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.865+0000 7f76dd400640 1 Processor -- start 2024-04-25T11:40:17.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.865+0000 7f76dd400640 1 -- start start 2024-04-25T11:40:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.866+0000 7f76dd400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc004800 0x7f76d80b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.866+0000 7f76dd400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76d8154d80 0x7f76d8161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d8161840 0x7f76d806d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dd400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76d80b3a90 con 0x7f76d8161840 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dd400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76d80b3910 con 0x7f76d8154d80 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dd400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f76d80b3c10 con 0x7f76bc004800 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d8161840 0x7f76d806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d8161840 0x7f76d806d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41240/0 (socket says 172.21.15.19:41240) 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 -- 172.21.15.19:0/3537878830 learned_addr learned my addr 172.21.15.19:0/3537878830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76d7e00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc004800 0x7f76d80b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 -- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc004800 msgr2=0x7f76d80b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc004800 0x7f76d80b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 -- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76d8154d80 msgr2=0x7f76d8161300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76cf400640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76d8154d80 0x7f76d8161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76d8154d80 0x7f76d8161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76dca00640 1 -- 172.21.15.19:0/3537878830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d80b0ea0 con 0x7f76d8161840 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.867+0000 7f76d7e00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc004800 0x7f76d80b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:17.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.868+0000 7f76dca00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d8161840 0x7f76d806d610 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f76d80d1620 tx=0x7f76d000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:17.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.868+0000 7f76d4c00640 1 -- 172.21.15.19:0/3537878830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0004030 con 0x7f76d8161840 2024-04-25T11:40:17.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.868+0000 7f76d4c00640 1 -- 172.21.15.19:0/3537878830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d0019070 con 0x7f76d8161840 2024-04-25T11:40:17.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.868+0000 7f76d4c00640 1 -- 172.21.15.19:0/3537878830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0004410 con 0x7f76d8161840 2024-04-25T11:40:17.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.868+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d80b1180 con 0x7f76d8161840 2024-04-25T11:40:17.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.868+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d80b1740 con 0x7f76d8161840 2024-04-25T11:40:17.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.869+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d8050740 con 0x7f76d8161840 2024-04-25T11:40:17.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.871+0000 7f76d4c00640 1 -- 172.21.15.19:0/3537878830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f76d0003610 con 0x7f76d8161840 2024-04-25T11:40:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.872+0000 7f76d4c00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76b40744b0 0x7f76b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.872+0000 7f76d4c00640 1 -- 172.21.15.19:0/3537878830 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f76d0095cb0 con 0x7f76d8161840 2024-04-25T11:40:17.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.873+0000 7f76d7e00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76b40744b0 0x7f76b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:17.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.874+0000 7f76d7e00640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76b40744b0 0x7f76b40769c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001b80 tx=0x7f76c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:17.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:17.878+0000 7f76d4c00640 1 -- 172.21.15.19:0/3537878830 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76d005f9b0 con 0x7f76d8161840 2024-04-25T11:40:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.125+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76d80cf460 con 0x7f76d8161840 2024-04-25T11:40:18.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.127+0000 7f76d4c00640 1 -- 172.21.15.19:0/3537878830 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 292 v292) ==== 95+0+5199 (secure 0 0 0) 0x7f76d005f2b0 con 0x7f76d8161840 2024-04-25T11:40:18.113 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:18.114 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":292,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":292,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:05.235512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10296":{"gid":10296,"name":"a","rank":0,"incarnation":289,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6835/2891720740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2891720740},{"type":"v1","addr":"172.21.15.19:6835","nonce":2891720740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10296,"qdb_cluster":[10296]},"id":6}]} 2024-04-25T11:40:18.114 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 292 2024-04-25T11:40:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76b40744b0 msgr2=0x7f76b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76b40744b0 0x7f76b40769c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f76c8001b80 tx=0x7f76c8009040 comp rx=0 tx=0).stop 2024-04-25T11:40:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d8161840 msgr2=0x7f76d806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d8161840 0x7f76d806d610 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f76d80d1620 tx=0x7f76d000ec00 comp rx=0 tx=0).stop 2024-04-25T11:40:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 shutdown_connections 2024-04-25T11:40:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f76b40744b0 0x7f76b40769c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76d8154d80 0x7f76d8161300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76d8161840 0x7f76d806d610 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 --2- 172.21.15.19:0/3537878830 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f76bc004800 0x7f76d80b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 >> 172.21.15.19:0/3537878830 conn(0x7f76d809b7d0 msgr2=0x7f76d80cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.129+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 shutdown_connections 2024-04-25T11:40:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.130+0000 7f76dd400640 1 -- 172.21.15.19:0/3537878830 wait complete. 2024-04-25T11:40:18.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T11:40:18.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.445+0000 7f208b000640 1 Processor -- start 2024-04-25T11:40:18.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.445+0000 7f208b000640 1 -- start start 2024-04-25T11:40:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.447+0000 7f208b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f208406e270 0x7f208416ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.447+0000 7f208b000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20840b53c0 con 0x7f2084160440 2024-04-25T11:40:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.447+0000 7f208b000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20840d37e0 con 0x7f20840d56c0 2024-04-25T11:40:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.447+0000 7f208b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20840d43f0 con 0x7f208406e270 2024-04-25T11:40:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.448+0000 7f2089200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2084160440 0x7f208406db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40652/0 (socket says 172.21.15.19:40652) 2024-04-25T11:40:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.448+0000 7f2089200640 1 -- 172.21.15.19:0/475607897 learned_addr learned my addr 172.21.15.19:0/475607897 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.448+0000 7f208a600640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f208406e270 0x7f208416ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.448+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479259134 0 0) 0x7f20840b53c0 con 0x7f2084160440 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.448+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f206c003520 con 0x7f2084160440 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1119854342 0 0) 0x7f206c003520 con 0x7f2084160440 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f208406e270 msgr2=0x7f208416ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f208406e270 0x7f208416ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 >> v1:172.21.15.169:6789/0 conn(0x7f20840d56c0 legacy=0x7f20840b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20840d3ac0 con 0x7f2084160440 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2070001670 con 0x7f2084160440 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 >> v1:172.21.15.19:6789/0 conn(0x7f2084160440 legacy=0x7f208406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f206c0039e0 0x7f206c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f206c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f206c005270 0x7f206c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20840d43f0 con 0x7f206c004800 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f206c00bf50 con 0x7f206c0039e0 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f206c00c150 con 0x7f206c005270 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.449+0000 7f2089200640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f206c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.450+0000 7f208a600640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f206c005270 0x7f206c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.450+0000 7f2089c00640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f206c0039e0 0x7f206c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.450+0000 7f2089200640 1 -- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f206c005270 msgr2=0x7f206c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.450+0000 7f2089200640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f206c005270 0x7f206c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.450+0000 7f2089200640 1 -- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f206c0039e0 msgr2=0x7f206c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.450+0000 7f2089200640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f206c0039e0 0x7f206c003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.450+0000 7f2089200640 1 -- 172.21.15.19:0/475607897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c003520 con 0x7f206c004800 2024-04-25T11:40:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.451+0000 7f2089200640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f206c004c10 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f207000c190 tx=0x7f206c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b637bffcb35ebe3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.451+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070007c60 con 0x7f206c004800 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.452+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2070005af0 con 0x7f206c004800 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.452+0000 7f2083e00640 1 -- 172.21.15.19:0/475607897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070007070 con 0x7f206c004800 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.452+0000 7f208b000640 1 -- 172.21.15.19:0/475607897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 msgr2=0x7f206c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.452+0000 7f208b000640 1 --2- 172.21.15.19:0/475607897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f206c004c10 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f207000c190 tx=0x7f206c005830 comp rx=0 tx=0).stop 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.452+0000 7f208a600640 1 -- 172.21.15.19:0/475607897 reap_dead start 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.453+0000 7f208b000640 1 -- 172.21.15.19:0/475607897 shutdown_connections 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.453+0000 7f208b000640 1 -- 172.21.15.19:0/475607897 >> 172.21.15.19:0/475607897 conn(0x7f20840d2c20 msgr2=0x7f20840d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.453+0000 7f208b000640 1 -- 172.21.15.19:0/475607897 shutdown_connections 2024-04-25T11:40:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.453+0000 7f208b000640 1 -- 172.21.15.19:0/475607897 wait complete. 2024-04-25T11:40:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.455+0000 7f208b000640 1 Processor -- start 2024-04-25T11:40:18.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.455+0000 7f208b000640 1 -- start start 2024-04-25T11:40:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f208b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f2084159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f208b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f208415c760 0x7f208415cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f208b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20841f6060 0x7f20841f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f208b000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20840ba270 con 0x7f206c004800 2024-04-25T11:40:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f208b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f20840d4080 con 0x7f208415c760 2024-04-25T11:40:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f208b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f20840d4880 con 0x7f20841f6060 2024-04-25T11:40:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f2089c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f2084159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f2089c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f2084159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41248/0 (socket says 172.21.15.19:41248) 2024-04-25T11:40:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.457+0000 7f2089c00640 1 -- 172.21.15.19:0/1589087509 learned_addr learned my addr 172.21.15.19:0/1589087509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.458+0000 7f2089c00640 1 -- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20841f6060 msgr2=0x7f20841f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.458+0000 7f2089c00640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20841f6060 0x7f20841f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.458+0000 7f2089c00640 1 -- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f208415c760 msgr2=0x7f208415cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.458+0000 7f2089c00640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f208415c760 0x7f208415cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.458+0000 7f2089c00640 1 -- 172.21.15.19:0/1589087509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208415d0b0 con 0x7f206c004800 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.458+0000 7f2089c00640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f2084159dd0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f207800d5d0 tx=0x7f207800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.459+0000 7f2081600640 1 -- 172.21.15.19:0/1589087509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078003610 con 0x7f206c004800 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.459+0000 7f2081600640 1 -- 172.21.15.19:0/1589087509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2078017070 con 0x7f206c004800 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.459+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208415b7e0 con 0x7f206c004800 2024-04-25T11:40:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.459+0000 7f2081600640 1 -- 172.21.15.19:0/1589087509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20780039b0 con 0x7f206c004800 2024-04-25T11:40:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.459+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208415aa20 con 0x7f206c004800 2024-04-25T11:40:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.461+0000 7f2081600640 1 -- 172.21.15.19:0/1589087509 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2078003b50 con 0x7f206c004800 2024-04-25T11:40:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.462+0000 7f2081600640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20640744e0 0x7f20640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.463+0000 7f2081600640 1 -- 172.21.15.19:0/1589087509 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f20780959e0 con 0x7f206c004800 2024-04-25T11:40:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.463+0000 7f2089200640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20640744e0 0x7f20640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.463+0000 7f2093584740 1 Processor -- start 2024-04-25T11:40:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.463+0000 7f2093584740 1 -- start start 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2089200640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20640744e0 0x7f20640769f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f207000c160 tx=0x7f2070007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2093584740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x55880005f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x55880005c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x5588000353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2093584740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5587fff98bc0 con 0x5587ffc92350 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5587fff34ed0 con 0x5587ffc69490 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5587fff94b80 con 0x5587ffc71670 2024-04-25T11:40:18.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2089c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x55880005f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2089c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x55880005f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41252/0 (socket says 172.21.15.19:41252) 2024-04-25T11:40:18.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.465+0000 7f2089c00640 1 -- 172.21.15.19:0/213216592 learned_addr learned my addr 172.21.15.19:0/213216592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.466+0000 7f2089c00640 1 -- 172.21.15.19:0/213216592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 msgr2=0x5588000353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.466+0000 7f2089c00640 1 --2- 172.21.15.19:0/213216592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x5588000353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.466+0000 7f2089c00640 1 -- 172.21.15.19:0/213216592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 msgr2=0x55880005c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.466+0000 7f2089c00640 1 --2- 172.21.15.19:0/213216592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x55880005c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.466+0000 7f2089c00640 1 -- 172.21.15.19:0/213216592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ffd767c0 con 0x5587ffc92350 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.466+0000 7f2089c00640 1 --2- 172.21.15.19:0/213216592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x55880005f760 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2078008fd0 tx=0x7f207805ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a801fe1995bf54b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.467+0000 7f2076000640 1 -- 172.21.15.19:0/213216592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078091070 con 0x5587ffc92350 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.467+0000 7f2076000640 1 -- 172.21.15.19:0/213216592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207805f0a0 con 0x5587ffc92350 2024-04-25T11:40:18.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.467+0000 7f2076000640 1 -- 172.21.15.19:0/213216592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078063450 con 0x5587ffc92350 2024-04-25T11:40:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 -- 172.21.15.19:0/213216592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 msgr2=0x55880005f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 --2- 172.21.15.19:0/213216592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x55880005f760 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2078008fd0 tx=0x7f207805ef90 comp rx=0 tx=0).stop 2024-04-25T11:40:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 -- 172.21.15.19:0/213216592 shutdown_connections 2024-04-25T11:40:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 --2- 172.21.15.19:0/213216592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x55880005c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 --2- 172.21.15.19:0/213216592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x5588000353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 --2- 172.21.15.19:0/213216592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x55880005f760 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 -- 172.21.15.19:0/213216592 >> 172.21.15.19:0/213216592 conn(0x5587ffc864a0 msgr2=0x558800034b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 -- 172.21.15.19:0/213216592 shutdown_connections 2024-04-25T11:40:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.468+0000 7f2093584740 1 -- 172.21.15.19:0/213216592 wait complete. 2024-04-25T11:40:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.469+0000 7f2093584740 1 -- start start 2024-04-25T11:40:18.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.469+0000 7f2093584740 1 Processor -- start 2024-04-25T11:40:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x55880008a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x558800062b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2093584740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800059ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2093584740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5587fff1f9f0 con 0x5587ffc92350 2024-04-25T11:40:18.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55880008a960 con 0x5587ffc69490 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x558800063230 con 0x5587ffc71670 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2093584740 10 client.0 _ll_register_callbacks cb 0x5587fff1df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f208a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800059ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f208a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800059ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41256/0 (socket says 172.21.15.19:41256) 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f208a600640 1 -- 172.21.15.19:0/2495064651 learned_addr learned my addr 172.21.15.19:0/2495064651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2089200640 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x558800062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.471+0000 7f2089c00640 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x55880008a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f208a600640 1 -- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 msgr2=0x558800062b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f208a600640 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x558800062b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f208a600640 1 -- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 msgr2=0x55880008a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f208a600640 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x55880008a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f208a600640 1 -- 172.21.15.19:0/2495064651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ffdd6f50 con 0x5587ffc92350 2024-04-25T11:40:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f208a600640 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800059ff0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f206c004780 tx=0x7f207c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf14af2ed1b57bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f206aa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f206aa00640 1 -- 172.21.15.19:0/2495064651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c003ba0 con 0x5587ffc92350 2024-04-25T11:40:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f206aa00640 1 -- 172.21.15.19:0/2495064651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f207c012040 con 0x5587ffc92350 2024-04-25T11:40:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.472+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207805f0a0 con 0x5587ffc92350 2024-04-25T11:40:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f206aa00640 1 -- 172.21.15.19:0/2495064651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=292) ==== 24+0+0 (secure 0 0 0) 0x7f207c003d40 con 0x5587ffc92350 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 10 client.10419 fetch_fsmap learned FSMap version 292 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=292,monmap=2+}) -- 0x5587ffc6a0b0 con 0x5587ffc92350 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f206aa00640 1 -- 172.21.15.19:0/2495064651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207805f0a0 con 0x5587ffc92350 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f206aa00640 1 -- 172.21.15.19:0/2495064651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 292) ==== 3494+0+0 (secure 0 0 0) 0x7f207c011860 con 0x5587ffc92350 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 10 client.10419 fetch_fsmap finished waiting for FSMap version 292 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 10 client.10419 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 20 client.10419 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5587ffc51130 0x55880008db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 4 client.10419 mds_command: new command op to 10296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 --> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587fffa67d0 con 0x5587ffc51130 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.473+0000 7f2089c00640 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5587ffc51130 0x55880008db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.474+0000 7f2089c00640 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5587ffc51130 0x55880008db20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47bc1d15ff7b5ea server_cookie=e35f15d3e1ab44ff in_seq=0 out_seq=0 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.474+0000 7f206aa00640 10 client.10419 ms_handle_connect on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:18.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f206aa00640 1 -- 172.21.15.19:0/2495064651 <== mds.0 v2:172.21.15.19:6834/2891720740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587fffa67d0 con 0x5587ffc51130 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f206aa00640 10 client.10419 handle_command_reply: tid=0 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 client.10419 shutdown 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 msgr2=0x558800059ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800059ff0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f206c004780 tx=0x7f207c007f10 comp rx=0 tx=0).stop 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 shutdown_connections 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5587ffc51130 0x55880008db20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800059ff0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x558800062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 --2- 172.21.15.19:0/2495064651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x55880008a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f206aa00640 0 client.10419 ms_handle_reset on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 >> 172.21.15.19:0/2495064651 conn(0x5587ffc864a0 msgr2=0x558800033f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 shutdown_connections 2024-04-25T11:40:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.475+0000 7f2093584740 1 -- 172.21.15.19:0/2495064651 wait complete. 2024-04-25T11:40:18.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.476+0000 7f2093584740 20 client.10419 trim_cache size 0 max 16384 2024-04-25T11:40:18.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.491+0000 7f2093584740 1 Processor -- start 2024-04-25T11:40:18.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.491+0000 7f2093584740 1 -- start start 2024-04-25T11:40:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.492+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x5588000560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.492+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x55880002e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.492+0000 7f2093584740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800035330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.492+0000 7f2093584740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20842035d0 con 0x5587ffc92350 2024-04-25T11:40:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.492+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2084202960 con 0x5587ffc69490 2024-04-25T11:40:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.492+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2084202fb0 con 0x5587ffc71670 2024-04-25T11:40:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800035330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800035330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41262/0 (socket says 172.21.15.19:41262) 2024-04-25T11:40:18.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 -- 172.21.15.19:0/391300096 learned_addr learned my addr 172.21.15.19:0/391300096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 -- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 msgr2=0x55880002e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f2089200640 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x55880002e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f2089c00640 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x5588000560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x55880002e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 -- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 msgr2=0x5588000560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x5588000560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.493+0000 7f208a600640 1 -- 172.21.15.19:0/391300096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880005f130 con 0x5587ffc92350 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.494+0000 7f208a600640 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800035330 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f207c000f80 tx=0x7f207c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb60ae594355530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.494+0000 7f2075600640 1 -- 172.21.15.19:0/391300096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c011860 con 0x5587ffc92350 2024-04-25T11:40:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.494+0000 7f2075600640 1 -- 172.21.15.19:0/391300096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c01b070 con 0x5587ffc92350 2024-04-25T11:40:18.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.494+0000 7f2075600640 1 -- 172.21.15.19:0/391300096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c003fb0 con 0x5587ffc92350 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 -- 172.21.15.19:0/391300096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 msgr2=0x558800035330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800035330 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f207c000f80 tx=0x7f207c016000 comp rx=0 tx=0).stop 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 -- 172.21.15.19:0/391300096 shutdown_connections 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc92350 0x558800035330 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc71670 0x55880002e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 --2- 172.21.15.19:0/391300096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x5588000560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 -- 172.21.15.19:0/391300096 >> 172.21.15.19:0/391300096 conn(0x5587ffc864a0 msgr2=0x558800076e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 -- 172.21.15.19:0/391300096 shutdown_connections 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.495+0000 7f2093584740 1 -- 172.21.15.19:0/391300096 wait complete. 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.496+0000 7f2093584740 1 -- start start 2024-04-25T11:40:18.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.496+0000 7f2093584740 1 Processor -- start 2024-04-25T11:40:18.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x558800035330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2093584740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc71670 0x558800035870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2093584740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc92350 0x558800075c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2093584740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5588000613e0 con 0x5587ffc71670 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558800060f90 con 0x5587ffc69490 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2093584740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55880007bac0 con 0x5587ffc92350 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2093584740 10 client.0 _ll_register_callbacks cb 0x7f207c004ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc71670 0x558800035870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc71670 0x558800035870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41266/0 (socket says 172.21.15.19:41266) 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 -- 172.21.15.19:0/3295427321 learned_addr learned my addr 172.21.15.19:0/3295427321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:18.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089c00640 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x558800035330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 -- 172.21.15.19:0/3295427321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc92350 msgr2=0x558800075c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc92350 0x558800075c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 -- 172.21.15.19:0/3295427321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 msgr2=0x558800035330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x558800035330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.498+0000 7f2089200640 1 -- 172.21.15.19:0/3295427321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588000fb0e0 con 0x5587ffc71670 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f2089200640 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc71670 0x558800035870 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f207000c2b0 tx=0x7f207002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18faced89058ef5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f206aa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f206aa00640 1 -- 172.21.15.19:0/3295427321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070051060 con 0x5587ffc71670 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f206aa00640 1 -- 172.21.15.19:0/3295427321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f2070007540 con 0x5587ffc71670 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207c01b070 con 0x5587ffc71670 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f206aa00640 1 -- 172.21.15.19:0/3295427321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=292) ==== 24+0+0 (secure 0 0 0) 0x7f2070044070 con 0x5587ffc71670 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f2093584740 10 client.10425 fetch_fsmap learned FSMap version 292 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.499+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=292,monmap=2+}) -- 0x558800076c20 con 0x5587ffc71670 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f206aa00640 1 -- 172.21.15.19:0/3295427321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c01b070 con 0x5587ffc71670 2024-04-25T11:40:18.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f206aa00640 1 -- 172.21.15.19:0/3295427321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 292) ==== 3494+0+0 (secure 0 0 0) 0x7f207004a030 con 0x5587ffc71670 2024-04-25T11:40:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f2093584740 10 client.10425 fetch_fsmap finished waiting for FSMap version 292 2024-04-25T11:40:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f2093584740 10 client.10425 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f2093584740 20 client.10425 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f2093584740 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5588000364e0 0x558800088920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f2093584740 4 client.10425 mds_command: new command op to 10296 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T11:40:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 --> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5587fffa0640 con 0x5588000364e0 2024-04-25T11:40:18.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.500+0000 7f2089c00640 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5588000364e0 0x558800088920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2089c00640 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5588000364e0 0x558800088920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01d9ba72f38c8d3 server_cookie=833e64eeddb49e7 in_seq=0 out_seq=0 2024-04-25T11:40:18.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f206aa00640 10 client.10425 ms_handle_connect on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:18.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f206aa00640 1 -- 172.21.15.19:0/3295427321 <== mds.0 v2:172.21.15.19:6834/2891720740 1 ==== command_reply(tid 0: 0 ) ==== 8+0+214 (crc 0 0 0) 0x5587fffa0640 con 0x5588000364e0 2024-04-25T11:40:18.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f206aa00640 10 client.10425 handle_command_reply: tid=0 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 client.10425 shutdown 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc71670 msgr2=0x558800035870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc71670 0x558800035870 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f207000c2b0 tx=0x7f207002c000 comp rx=0 tx=0).stop 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 shutdown_connections 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:6834/2891720740,v1:172.21.15.19:6835/2891720740] conn(0x5588000364e0 0x558800088920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5587ffc92350 0x558800075c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587ffc71670 0x558800035870 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 --2- 172.21.15.19:0/3295427321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5587ffc69490 0x558800035330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f206aa00640 0 client.10425 ms_handle_reset on v2:172.21.15.19:6834/2891720740 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 >> 172.21.15.19:0/3295427321 conn(0x5587ffc864a0 msgr2=0x558800059e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.501+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 shutdown_connections 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f2093584740 1 -- 172.21.15.19:0/3295427321 wait complete. 2024-04-25T11:40:18.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f2093584740 20 client.10425 trim_cache size 0 max 16384 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "dentry", 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 365683610, 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627776, 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "frag": "*", 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "sixmegs", 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_id": "head", 2024-04-25T11:40:18.494 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/subdir/sixmegs" 2024-04-25T11:40:18.495 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:40:18.495 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:40:18.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20640744e0 msgr2=0x7f20640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20640744e0 0x7f20640769f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f207000c160 tx=0x7f2070007000 comp rx=0 tx=0).stop 2024-04-25T11:40:18.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 msgr2=0x7f2084159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f2084159dd0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f207800d5d0 tx=0x7f207800da90 comp rx=0 tx=0).stop 2024-04-25T11:40:18.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 shutdown_connections 2024-04-25T11:40:18.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f20640744e0 0x7f20640769f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f208415c760 0x7f208415cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20841f6060 0x7f20841f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 --2- 172.21.15.19:0/1589087509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f206c004800 0x7f2084159dd0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.502+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 >> 172.21.15.19:0/1589087509 conn(0x7f20840d2c20 msgr2=0x7f20841560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.503+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 shutdown_connections 2024-04-25T11:40:18.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.503+0000 7f208b000640 1 -- 172.21.15.19:0/1589087509 wait complete. 2024-04-25T11:40:18.516 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2024-04-25T11:40:18.516 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:40:18.516 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:40:18.563 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:40:18.564 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:40:18.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:40:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.951+0000 7fb7e3e00640 1 Processor -- start 2024-04-25T11:40:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.952+0000 7fb7e3e00640 1 -- start start 2024-04-25T11:40:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.954+0000 7fb7e3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb7e406c5c0 0x7fb7e40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.954+0000 7fb7e3e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7e4144370 0x7fb7e40b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.954+0000 7fb7e3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e40b50e0 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.954+0000 7fb7e3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e40d1230 con 0x7fb7e406c5c0 2024-04-25T11:40:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.954+0000 7fb7e3e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e40d06b0 con 0x7fb7e4144370 2024-04-25T11:40:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.954+0000 7fb7e2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb7e406c5c0 0x7fb7e40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb7e406c5c0 0x7fb7e40cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41268/0 (socket says 172.21.15.19:41268) 2024-04-25T11:40:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2a00640 1 -- 172.21.15.19:0/2422420238 learned_addr learned my addr 172.21.15.19:0/2422420238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2a00640 1 -- 172.21.15.19:0/2422420238 >> v2:172.21.15.169:3300/0 conn(0x7fb7e4144370 msgr2=0x7fb7e40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2000640 1 --2- 172.21.15.19:0/2422420238 >> v2:172.21.15.169:3300/0 conn(0x7fb7e4144370 0x7fb7e40b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2a00640 1 --2- 172.21.15.19:0/2422420238 >> v2:172.21.15.169:3300/0 conn(0x7fb7e4144370 0x7fb7e40b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2a00640 1 -- 172.21.15.19:0/2422420238 >> v1:172.21.15.169:6789/0 conn(0x7fb7e40b4cd0 legacy=0x7fb7e4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2a00640 1 -- 172.21.15.19:0/2422420238 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e40d0990 con 0x7fb7e406c5c0 2024-04-25T11:40:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.955+0000 7fb7e2000640 1 --2- 172.21.15.19:0/2422420238 >> v2:172.21.15.169:3300/0 conn(0x7fb7e4144370 0x7fb7e40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.956+0000 7fb7e2a00640 1 --2- 172.21.15.19:0/2422420238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e406c5c0 0x7fb7e40cfa70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b5550 tx=0x7fb7d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce829cbc880ef47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.957+0000 7fb7e1600640 1 -- 172.21.15.19:0/2422420238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4037070 con 0x7fb7e406c5c0 2024-04-25T11:40:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.957+0000 7fb7e1600640 1 -- 172.21.15.19:0/2422420238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d4032440 con 0x7fb7e406c5c0 2024-04-25T11:40:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.957+0000 7fb7e1600640 1 -- 172.21.15.19:0/2422420238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4032810 con 0x7fb7e406c5c0 2024-04-25T11:40:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 -- 172.21.15.19:0/2422420238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e406c5c0 msgr2=0x7fb7e40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/2422420238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e406c5c0 0x7fb7e40cfa70 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b5550 tx=0x7fb7d402a270 comp rx=0 tx=0).stop 2024-04-25T11:40:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 -- 172.21.15.19:0/2422420238 shutdown_connections 2024-04-25T11:40:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/2422420238 >> v2:172.21.15.169:3300/0 conn(0x7fb7e4144370 0x7fb7e40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/2422420238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e406c5c0 0x7fb7e40cfa70 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 -- 172.21.15.19:0/2422420238 >> 172.21.15.19:0/2422420238 conn(0x7fb7e40d1720 msgr2=0x7fb7e40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 -- 172.21.15.19:0/2422420238 shutdown_connections 2024-04-25T11:40:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.958+0000 7fb7e3e00640 1 -- 172.21.15.19:0/2422420238 wait complete. 2024-04-25T11:40:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.960+0000 7fb7e3e00640 1 Processor -- start 2024-04-25T11:40:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.961+0000 7fb7e3e00640 1 -- start start 2024-04-25T11:40:18.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7e406c5c0 0x7fb7e4161740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e40b4cd0 0x7fb7e4142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7e4144370 0x7fb7e4143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e40d02d0 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e40d0f50 con 0x7fb7e4144370 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e40d0450 con 0x7fb7e406c5c0 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e40b4cd0 0x7fb7e4142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.962+0000 7fb7e2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e40b4cd0 0x7fb7e4142ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41282/0 (socket says 172.21.15.19:41282) 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2000640 1 -- 172.21.15.19:0/341327726 learned_addr learned my addr 172.21.15.19:0/341327726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2a00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7e406c5c0 0x7fb7e4161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e3400640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7e4144370 0x7fb7e4143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2000640 1 -- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7e406c5c0 msgr2=0x7fb7e4161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2000640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7e406c5c0 0x7fb7e4161740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2000640 1 -- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7e4144370 msgr2=0x7fb7e4143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2000640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7e4144370 0x7fb7e4143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2000640 1 -- 172.21.15.19:0/341327726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e40ae7e0 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2a00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7e406c5c0 0x7fb7e4161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.963+0000 7fb7e2000640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e40b4cd0 0x7fb7e4142ce0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000c7f0 tx=0x7fb7d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.964+0000 7fb7db400640 1 -- 172.21.15.19:0/341327726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0015070 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.964+0000 7fb7db400640 1 -- 172.21.15.19:0/341327726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d0007820 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.964+0000 7fb7db400640 1 -- 172.21.15.19:0/341327726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0007bd0 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.964+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e41620b0 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.964+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e4162670 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.965+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e4161e20 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.966+0000 7fb7db400640 1 -- 172.21.15.19:0/341327726 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7d0018070 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.967+0000 7fb7db400640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7c40744b0 0x7fb7c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.967+0000 7fb7db400640 1 -- 172.21.15.19:0/341327726 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7fb7d0095cd0 con 0x7fb7e40b4cd0 2024-04-25T11:40:18.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.968+0000 7fb7e2a00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7c40744b0 0x7fb7c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:18.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.969+0000 7fb7e2a00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7c40744b0 0x7fb7c40769c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4034000 tx=0x7fb7d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:18.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:18.972+0000 7fb7db400640 1 -- 172.21.15.19:0/341327726 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7d005f940 con 0x7fb7e40b4cd0 2024-04-25T11:40:19.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.178+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7e40b0000 con 0x7fb7e40b4cd0 2024-04-25T11:40:19.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.180+0000 7fb7db400640 1 -- 172.21.15.19:0/341327726 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+20682 (secure 0 0 0) 0x7fb7d005f240 con 0x7fb7e40b4cd0 2024-04-25T11:40:19.168 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:19.168 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:39:56.138618+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:40:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7c40744b0 msgr2=0x7fb7c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7c40744b0 0x7fb7c40769c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4034000 tx=0x7fb7d402e000 comp rx=0 tx=0).stop 2024-04-25T11:40:19.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e40b4cd0 msgr2=0x7fb7e4142ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e40b4cd0 0x7fb7e4142ce0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000c7f0 tx=0x7fb7d000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 shutdown_connections 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7c40744b0 0x7fb7c40769c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7e4144370 0x7fb7e4143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7e40b4cd0 0x7fb7e4142ce0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 --2- 172.21.15.19:0/341327726 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7e406c5c0 0x7fb7e4161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.183+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 >> 172.21.15.19:0/341327726 conn(0x7fb7e40d1720 msgr2=0x7fb7e4146040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.184+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 shutdown_connections 2024-04-25T11:40:19.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.184+0000 7fb7e3e00640 1 -- 172.21.15.19:0/341327726 wait complete. 2024-04-25T11:40:19.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:40:19.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:40:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.504+0000 7f6c47c00640 1 Processor -- start 2024-04-25T11:40:19.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.504+0000 7f6c47c00640 1 -- start start 2024-04-25T11:40:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.506+0000 7f6c47c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c400cd9a0 0x7f6c4015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.506+0000 7f6c47c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6c4006dce0 0x7f6c4015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.506+0000 7f6c47c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c400cdea0 con 0x7f6c400b0af0 2024-04-25T11:40:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.506+0000 7f6c47c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6c400ceab0 con 0x7f6c4006dce0 2024-04-25T11:40:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.506+0000 7f6c47c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6c400ce180 con 0x7f6c400cd9a0 2024-04-25T11:40:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.507+0000 7f6c45200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c400cd9a0 0x7f6c4015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.507+0000 7f6c45200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c400cd9a0 0x7f6c4015f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:45264/0 (socket says 172.21.15.19:45264) 2024-04-25T11:40:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.507+0000 7f6c45200640 1 -- 172.21.15.19:0/2222724215 learned_addr learned my addr 172.21.15.19:0/2222724215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.507+0000 7f6c45c00640 1 --2- 172.21.15.19:0/2222724215 >> v2:172.21.15.169:3300/0 conn(0x7f6c4006dce0 0x7f6c4015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.508+0000 7f6c45200640 1 -- 172.21.15.19:0/2222724215 >> v2:172.21.15.169:3300/0 conn(0x7f6c4006dce0 msgr2=0x7f6c4015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.508+0000 7f6c45200640 1 --2- 172.21.15.19:0/2222724215 >> v2:172.21.15.169:3300/0 conn(0x7f6c4006dce0 0x7f6c4015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.508+0000 7f6c45200640 1 -- 172.21.15.19:0/2222724215 >> v1:172.21.15.169:6789/0 conn(0x7f6c400b0af0 legacy=0x7f6c4006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.508+0000 7f6c45200640 1 -- 172.21.15.19:0/2222724215 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c400ce460 con 0x7f6c400cd9a0 2024-04-25T11:40:19.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.508+0000 7f6c45c00640 1 --2- 172.21.15.19:0/2222724215 >> v2:172.21.15.169:3300/0 conn(0x7f6c4006dce0 0x7f6c4015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:40:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.509+0000 7f6c45200640 1 --2- 172.21.15.19:0/2222724215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c400cd9a0 0x7f6c4015f360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009870 tx=0x7f6c2802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aabd1f2f21a2cce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.510+0000 7f6c3f400640 1 -- 172.21.15.19:0/2222724215 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28037070 con 0x7f6c400cd9a0 2024-04-25T11:40:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.510+0000 7f6c3f400640 1 -- 172.21.15.19:0/2222724215 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c28032440 con 0x7f6c400cd9a0 2024-04-25T11:40:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.510+0000 7f6c3f400640 1 -- 172.21.15.19:0/2222724215 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28032810 con 0x7f6c400cd9a0 2024-04-25T11:40:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.510+0000 7f6c47c00640 1 -- 172.21.15.19:0/2222724215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c400cd9a0 msgr2=0x7f6c4015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.510+0000 7f6c47c00640 1 --2- 172.21.15.19:0/2222724215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c400cd9a0 0x7f6c4015f360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009870 tx=0x7f6c2802a250 comp rx=0 tx=0).stop 2024-04-25T11:40:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.511+0000 7f6c47c00640 1 -- 172.21.15.19:0/2222724215 shutdown_connections 2024-04-25T11:40:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.511+0000 7f6c47c00640 1 --2- 172.21.15.19:0/2222724215 >> v2:172.21.15.169:3300/0 conn(0x7f6c4006dce0 0x7f6c4015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.511+0000 7f6c47c00640 1 --2- 172.21.15.19:0/2222724215 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c400cd9a0 0x7f6c4015f360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.511+0000 7f6c47c00640 1 -- 172.21.15.19:0/2222724215 >> 172.21.15.19:0/2222724215 conn(0x7f6c40073090 msgr2=0x7f6c400734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.511+0000 7f6c47c00640 1 -- 172.21.15.19:0/2222724215 shutdown_connections 2024-04-25T11:40:19.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.511+0000 7f6c47c00640 1 -- 172.21.15.19:0/2222724215 wait complete. 2024-04-25T11:40:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.513+0000 7f6c47c00640 1 Processor -- start 2024-04-25T11:40:19.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.513+0000 7f6c47c00640 1 -- start start 2024-04-25T11:40:19.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c47c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c400b0af0 0x7f6c40137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c47c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c400cd9a0 0x7f6c40145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c47c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c40146340 0x7f6c401e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c47c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400cef40 con 0x7f6c400cd9a0 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c47c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400b0f00 con 0x7f6c400b0af0 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c47c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6c400cf0c0 con 0x7f6c40146340 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c400cd9a0 0x7f6c40145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c400cd9a0 0x7f6c40145e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41286/0 (socket says 172.21.15.19:41286) 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.515+0000 7f6c3fe00640 1 -- 172.21.15.19:0/1772343605 learned_addr learned my addr 172.21.15.19:0/1772343605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c45200640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c400b0af0 0x7f6c40137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c45c00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c40146340 0x7f6c401e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c3fe00640 1 -- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c40146340 msgr2=0x7f6c401e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c3fe00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c40146340 0x7f6c401e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c3fe00640 1 -- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c400b0af0 msgr2=0x7f6c40137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c3fe00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c400b0af0 0x7f6c40137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c3fe00640 1 -- 172.21.15.19:0/1772343605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c401e7e20 con 0x7f6c400cd9a0 2024-04-25T11:40:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.516+0000 7f6c3fe00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c400cd9a0 0x7f6c40145e00 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c6b0 tx=0x7f6c3000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:19.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.517+0000 7f6c3cc00640 1 -- 172.21.15.19:0/1772343605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30016020 con 0x7f6c400cd9a0 2024-04-25T11:40:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.517+0000 7f6c3cc00640 1 -- 172.21.15.19:0/1772343605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c30006d50 con 0x7f6c400cd9a0 2024-04-25T11:40:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.517+0000 7f6c3cc00640 1 -- 172.21.15.19:0/1772343605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30003bb0 con 0x7f6c400cd9a0 2024-04-25T11:40:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.517+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c40146750 con 0x7f6c400cd9a0 2024-04-25T11:40:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.517+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c401e8090 con 0x7f6c400cd9a0 2024-04-25T11:40:19.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.518+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c40050740 con 0x7f6c400cd9a0 2024-04-25T11:40:19.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.519+0000 7f6c3cc00640 1 -- 172.21.15.19:0/1772343605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6c300050d0 con 0x7f6c400cd9a0 2024-04-25T11:40:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.520+0000 7f6c3cc00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6c1c0744b0 0x7f6c1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:19.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.521+0000 7f6c3cc00640 1 -- 172.21.15.19:0/1772343605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f6c30095900 con 0x7f6c400cd9a0 2024-04-25T11:40:19.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.521+0000 7f6c45200640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6c1c0744b0 0x7f6c1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:19.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.523+0000 7f6c45200640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6c1c0744b0 0x7f6c1c0769c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6c28034000 tx=0x7f6c2802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:19.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.526+0000 7f6c3cc00640 1 -- 172.21.15.19:0/1772343605 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c30020020 con 0x7f6c400cd9a0 2024-04-25T11:40:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.733+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c40147e00 con 0x7f6c400cd9a0 2024-04-25T11:40:19.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.734+0000 7f6c3cc00640 1 -- 172.21.15.19:0/1772343605 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7f6c40146750 con 0x7f6c400cd9a0 2024-04-25T11:40:19.722 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:40:19.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6c1c0744b0 msgr2=0x7f6c1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6c1c0744b0 0x7f6c1c0769c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6c28034000 tx=0x7f6c2802e000 comp rx=0 tx=0).stop 2024-04-25T11:40:19.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c400cd9a0 msgr2=0x7f6c40145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:19.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c400cd9a0 0x7f6c40145e00 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000c6b0 tx=0x7f6c3000cb70 comp rx=0 tx=0).stop 2024-04-25T11:40:19.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 shutdown_connections 2024-04-25T11:40:19.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6c1c0744b0 0x7f6c1c0769c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c40146340 0x7f6c401e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c400cd9a0 0x7f6c40145e00 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 --2- 172.21.15.19:0/1772343605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c400b0af0 0x7f6c40137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 >> 172.21.15.19:0/1772343605 conn(0x7f6c40073090 msgr2=0x7f6c400cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 shutdown_connections 2024-04-25T11:40:19.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:19.737+0000 7f6c47c00640 1 -- 172.21.15.19:0/1772343605 wait complete. 2024-04-25T11:40:19.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:40:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.062+0000 7f2659400640 1 Processor -- start 2024-04-25T11:40:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.062+0000 7f2659400640 1 -- start start 2024-04-25T11:40:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2659400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f26540cd9a0 0x7f265415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2659400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540b0af0 0x7f265406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2659400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f265406db10 0x7f2654161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2659400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f26540ceab0 con 0x7f265406db10 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2659400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f265406df20 con 0x7f26540cd9a0 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2659400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26540cdea0 con 0x7f26540b0af0 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2653200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f265406db10 0x7f2654161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2653200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f265406db10 0x7f2654161ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41296/0 (socket says 172.21.15.19:41296) 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2653200640 1 -- 172.21.15.19:0/886495898 learned_addr learned my addr 172.21.15.19:0/886495898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.065+0000 7f2652800640 1 --2- 172.21.15.19:0/886495898 >> v2:172.21.15.169:3300/0 conn(0x7f26540cd9a0 0x7f265415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.066+0000 7f2649e00640 1 --2- 172.21.15.19:0/886495898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540b0af0 0x7f265406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.066+0000 7f2653200640 1 -- 172.21.15.19:0/886495898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540b0af0 msgr2=0x7f265406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.066+0000 7f2653200640 1 --2- 172.21.15.19:0/886495898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540b0af0 0x7f265406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.066+0000 7f2653200640 1 -- 172.21.15.19:0/886495898 >> v2:172.21.15.169:3300/0 conn(0x7f26540cd9a0 msgr2=0x7f265415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:20.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.066+0000 7f2653200640 1 --2- 172.21.15.19:0/886495898 >> v2:172.21.15.169:3300/0 conn(0x7f26540cd9a0 0x7f265415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.066+0000 7f2653200640 1 -- 172.21.15.19:0/886495898 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26540ce180 con 0x7f265406db10 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.067+0000 7f2653200640 1 --2- 172.21.15.19:0/886495898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f265406db10 0x7f2654161ed0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2654072860 tx=0x7f264c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60466557da36bcca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.067+0000 7f2651e00640 1 -- 172.21.15.19:0/886495898 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c037070 con 0x7f265406db10 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.067+0000 7f2651e00640 1 -- 172.21.15.19:0/886495898 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c032440 con 0x7f265406db10 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.067+0000 7f2651e00640 1 -- 172.21.15.19:0/886495898 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c032810 con 0x7f265406db10 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 -- 172.21.15.19:0/886495898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f265406db10 msgr2=0x7f2654161ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 --2- 172.21.15.19:0/886495898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f265406db10 0x7f2654161ed0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2654072860 tx=0x7f264c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 -- 172.21.15.19:0/886495898 shutdown_connections 2024-04-25T11:40:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 --2- 172.21.15.19:0/886495898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f265406db10 0x7f2654161ed0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 --2- 172.21.15.19:0/886495898 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540b0af0 0x7f265406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 --2- 172.21.15.19:0/886495898 >> v2:172.21.15.169:3300/0 conn(0x7f26540cd9a0 0x7f265415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 -- 172.21.15.19:0/886495898 >> 172.21.15.19:0/886495898 conn(0x7f2654073090 msgr2=0x7f26540734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 -- 172.21.15.19:0/886495898 shutdown_connections 2024-04-25T11:40:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.068+0000 7f2659400640 1 -- 172.21.15.19:0/886495898 wait complete. 2024-04-25T11:40:20.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.071+0000 7f2659400640 1 Processor -- start 2024-04-25T11:40:20.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.071+0000 7f2659400640 1 -- start start 2024-04-25T11:40:20.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.072+0000 7f2659400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265406db10 0x7f2654160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.072+0000 7f2659400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26540b0af0 0x7f26540afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.072+0000 7f2659400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540cd9a0 0x7f265413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.072+0000 7f2659400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26540b0f00 con 0x7f26540b0af0 2024-04-25T11:40:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.072+0000 7f2659400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26540ce740 con 0x7f265406db10 2024-04-25T11:40:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.072+0000 7f2659400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f26540cef40 con 0x7f26540cd9a0 2024-04-25T11:40:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26540b0af0 0x7f26540afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26540b0af0 0x7f26540afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41304/0 (socket says 172.21.15.19:41304) 2024-04-25T11:40:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 -- 172.21.15.19:0/1835752133 learned_addr learned my addr 172.21.15.19:0/1835752133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2652800640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265406db10 0x7f2654160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2653200640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540cd9a0 0x7f265413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 -- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540cd9a0 msgr2=0x7f265413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540cd9a0 0x7f265413ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 -- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265406db10 msgr2=0x7f2654160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265406db10 0x7f2654160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.073+0000 7f2649e00640 1 -- 172.21.15.19:0/1835752133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2654142b20 con 0x7f26540b0af0 2024-04-25T11:40:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.074+0000 7f2649e00640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26540b0af0 0x7f26540afe40 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d620 tx=0x7f263c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.074+0000 7f264b400640 1 -- 172.21.15.19:0/1835752133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c00b620 con 0x7f26540b0af0 2024-04-25T11:40:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.074+0000 7f264b400640 1 -- 172.21.15.19:0/1835752133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c019070 con 0x7f26540b0af0 2024-04-25T11:40:20.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.074+0000 7f264b400640 1 -- 172.21.15.19:0/1835752133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c00ba00 con 0x7f26540b0af0 2024-04-25T11:40:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.074+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265415fb20 con 0x7f26540b0af0 2024-04-25T11:40:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.074+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2654142e00 con 0x7f26540b0af0 2024-04-25T11:40:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.075+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2654050740 con 0x7f26540b0af0 2024-04-25T11:40:20.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.077+0000 7f264b400640 1 -- 172.21.15.19:0/1835752133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f263c00bba0 con 0x7f26540b0af0 2024-04-25T11:40:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.078+0000 7f264b400640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f26300744e0 0x7f26300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.078+0000 7f264b400640 1 -- 172.21.15.19:0/1835752133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f263c096170 con 0x7f26540b0af0 2024-04-25T11:40:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.078+0000 7f2652800640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f26300744e0 0x7f26300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.080+0000 7f2652800640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f26300744e0 0x7f26300769f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2644009940 tx=0x7f2644003cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:20.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.083+0000 7f264b400640 1 -- 172.21.15.19:0/1835752133 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f263c05fde0 con 0x7f26540b0af0 2024-04-25T11:40:20.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.286+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f26541428b0 con 0x7f26540b0af0 2024-04-25T11:40:20.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.287+0000 7f264b400640 1 -- 172.21.15.19:0/1835752133 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) ==== 104+0+11 (secure 0 0 0) 0x7f263c05f6e0 con 0x7f26540b0af0 2024-04-25T11:40:20.274 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:40:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f26300744e0 msgr2=0x7f26300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f26300744e0 0x7f26300769f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2644009940 tx=0x7f2644003cd0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26540b0af0 msgr2=0x7f26540afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26540b0af0 0x7f26540afe40 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d620 tx=0x7f263c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 shutdown_connections 2024-04-25T11:40:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f26300744e0 0x7f26300769f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f26540cd9a0 0x7f265413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26540b0af0 0x7f26540afe40 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 --2- 172.21.15.19:0/1835752133 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265406db10 0x7f2654160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 >> 172.21.15.19:0/1835752133 conn(0x7f2654073090 msgr2=0x7f2654070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 shutdown_connections 2024-04-25T11:40:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.290+0000 7f2659400640 1 -- 172.21.15.19:0/1835752133 wait complete. 2024-04-25T11:40:20.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:40:20.313 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:40:20.320+0000 7fb164400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T11:40:20.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.604+0000 7f59d0e00640 1 Processor -- start 2024-04-25T11:40:20.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.604+0000 7f59d0e00640 1 -- start start 2024-04-25T11:40:20.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59d0e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f59cc06dce0 0x7f59cc15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59d0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 0x7f59cc15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59d0e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59cc0cdea0 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59d0e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f59cc0ceab0 con 0x7f59cc06dce0 2024-04-25T11:40:20.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59d0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc0ce180 con 0x7f59cc0b0af0 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59c9e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f59cc0cd9a0 0x7f59cc06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40662/0 (socket says 172.21.15.19:40662) 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59c9e00640 1 -- 172.21.15.19:0/1538748700 learned_addr learned my addr 172.21.15.19:0/1538748700 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.607+0000 7f59c9400640 1 --2- 172.21.15.19:0/1538748700 >> v2:172.21.15.19:3300/0 conn(0x7f59cc06dce0 0x7f59cc15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.608+0000 7f59ca800640 1 --2- 172.21.15.19:0/1538748700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 0x7f59cc15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.608+0000 7f59c8a00640 1 -- 172.21.15.19:0/1538748700 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954337839 0 0) 0x7f59cc0cdea0 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.608+0000 7f59c8a00640 1 -- 172.21.15.19:0/1538748700 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59b0003520 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.608+0000 7f59c9400640 1 -- 172.21.15.19:0/1538748700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 msgr2=0x7f59cc15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.608+0000 7f59c9400640 1 --2- 172.21.15.19:0/1538748700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 0x7f59cc15faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.608+0000 7f59c9400640 1 -- 172.21.15.19:0/1538748700 >> v1:172.21.15.19:6789/0 conn(0x7f59cc0cd9a0 legacy=0x7f59cc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.608+0000 7f59c9400640 1 -- 172.21.15.19:0/1538748700 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc0ce460 con 0x7f59cc06dce0 2024-04-25T11:40:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.609+0000 7f59c9400640 1 --2- 172.21.15.19:0/1538748700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc06dce0 0x7f59cc15f360 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f59b80097a0 tx=0x7f59b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2603293a1b03430e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.609+0000 7f59c8a00640 1 -- 172.21.15.19:0/1538748700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8037070 con 0x7f59cc06dce0 2024-04-25T11:40:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.609+0000 7f59c8a00640 1 -- 172.21.15.19:0/1538748700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8032440 con 0x7f59cc06dce0 2024-04-25T11:40:20.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.609+0000 7f59c8a00640 1 -- 172.21.15.19:0/1538748700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8032810 con 0x7f59cc06dce0 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 -- 172.21.15.19:0/1538748700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc06dce0 msgr2=0x7f59cc15f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1538748700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc06dce0 0x7f59cc15f360 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f59b80097a0 tx=0x7f59b802a180 comp rx=0 tx=0).stop 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 -- 172.21.15.19:0/1538748700 shutdown_connections 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1538748700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 0x7f59cc15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1538748700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc06dce0 0x7f59cc15f360 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 -- 172.21.15.19:0/1538748700 >> 172.21.15.19:0/1538748700 conn(0x7f59cc073090 msgr2=0x7f59cc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 -- 172.21.15.19:0/1538748700 shutdown_connections 2024-04-25T11:40:20.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.610+0000 7f59d0e00640 1 -- 172.21.15.19:0/1538748700 wait complete. 2024-04-25T11:40:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.612+0000 7f59d0e00640 1 Processor -- start 2024-04-25T11:40:20.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.613+0000 7f59d0e00640 1 -- start start 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59d0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 0x7f59cc160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59d0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc0cd9a0 0x7f59cc1f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59d0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59cc15d3b0 0x7f59cc15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59d0e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0cef40 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59d0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0b0f00 con 0x7f59cc15d3b0 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59d0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc0cf0c0 con 0x7f59cc0b0af0 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc0cd9a0 0x7f59cc1f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc0cd9a0 0x7f59cc1f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41328/0 (socket says 172.21.15.19:41328) 2024-04-25T11:40:20.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.614+0000 7f59c9400640 1 -- 172.21.15.19:0/1518392478 learned_addr learned my addr 172.21.15.19:0/1518392478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.615+0000 7f59c9400640 1 -- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 msgr2=0x7f59cc160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.615+0000 7f59c9400640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 0x7f59cc160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.615+0000 7f59c9400640 1 -- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59cc15d3b0 msgr2=0x7f59cc15d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.615+0000 7f59c9400640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59cc15d3b0 0x7f59cc15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.615+0000 7f59c9400640 1 -- 172.21.15.19:0/1518392478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc15dd00 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.615+0000 7f59c9400640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc0cd9a0 0x7f59cc1f5c90 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098c0 tx=0x7f59b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.615+0000 7f59c2000640 1 -- 172.21.15.19:0/1518392478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8037040 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.616+0000 7f59c2000640 1 -- 172.21.15.19:0/1518392478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8030500 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.616+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc157f10 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.616+0000 7f59c2000640 1 -- 172.21.15.19:0/1518392478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b80308d0 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.616+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1584d0 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.617+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc050740 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.618+0000 7f59c2000640 1 -- 172.21.15.19:0/1518392478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f59b8044020 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.619+0000 7f59c2000640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59ac0744e0 0x7f59ac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.619+0000 7f59c2000640 1 -- 172.21.15.19:0/1518392478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f59b80b5f00 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.620+0000 7f59c9e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59ac0744e0 0x7f59ac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:20.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.621+0000 7f59c9e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59ac0744e0 0x7f59ac0769f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f59bc000f60 tx=0x7f59bc0078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:20.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.624+0000 7f59c2000640 1 -- 172.21.15.19:0/1518392478 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59b807fc00 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.825+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f59cc157ca0 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.826+0000 7f59c2000640 1 -- 172.21.15.19:0/1518392478 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) ==== 100+0+11 (secure 0 0 0) 0x7f59b807f500 con 0x7f59cc0cd9a0 2024-04-25T11:40:20.813 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.828+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59ac0744e0 msgr2=0x7f59ac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.828+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59ac0744e0 0x7f59ac0769f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f59bc000f60 tx=0x7f59bc0078f0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.828+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc0cd9a0 msgr2=0x7f59cc1f5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.828+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc0cd9a0 0x7f59cc1f5c90 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098c0 tx=0x7f59b80046c0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 shutdown_connections 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59ac0744e0 0x7f59ac0769f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59cc15d3b0 0x7f59cc15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59cc0cd9a0 0x7f59cc1f5c90 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 --2- 172.21.15.19:0/1518392478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f59cc0b0af0 0x7f59cc160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 >> 172.21.15.19:0/1518392478 conn(0x7f59cc073090 msgr2=0x7f59cc0b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 shutdown_connections 2024-04-25T11:40:20.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:20.829+0000 7f59d0e00640 1 -- 172.21.15.19:0/1518392478 wait complete. 2024-04-25T11:40:20.840 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:40:21.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.126+0000 7f9c2fe00640 1 Processor -- start 2024-04-25T11:40:21.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.126+0000 7f9c2fe00640 1 -- start start 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9c280cf5e0 0x7f9c280cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c2806c5c0 0x7f9c2806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2fe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c280cff30 con 0x7f9c281443c0 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9c280708d0 con 0x7f9c2806c5c0 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9c2815f160 con 0x7f9c280cf5e0 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c2806c5c0 0x7f9c2806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9c2806c5c0 0x7f9c2806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41342/0 (socket says 172.21.15.19:41342) 2024-04-25T11:40:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.128+0000 7f9c2de00640 1 -- 172.21.15.19:0/1782664235 learned_addr learned my addr 172.21.15.19:0/1782664235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.129+0000 7f9c2ca00640 1 --2- 172.21.15.19:0/1782664235 >> v2:172.21.15.169:3300/0 conn(0x7f9c280cf5e0 0x7f9c280cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.129+0000 7f9c2de00640 1 -- 172.21.15.19:0/1782664235 >> v2:172.21.15.169:3300/0 conn(0x7f9c280cf5e0 msgr2=0x7f9c280cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.129+0000 7f9c2de00640 1 --2- 172.21.15.19:0/1782664235 >> v2:172.21.15.169:3300/0 conn(0x7f9c280cf5e0 0x7f9c280cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.129+0000 7f9c2de00640 1 -- 172.21.15.19:0/1782664235 >> v1:172.21.15.169:6789/0 conn(0x7f9c281443c0 legacy=0x7f9c2806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.129+0000 7f9c2de00640 1 -- 172.21.15.19:0/1782664235 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2815f5f0 con 0x7f9c2806c5c0 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.129+0000 7f9c2ca00640 1 --2- 172.21.15.19:0/1782664235 >> v2:172.21.15.169:3300/0 conn(0x7f9c280cf5e0 0x7f9c280cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.130+0000 7f9c2de00640 1 --2- 172.21.15.19:0/1782664235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c2806c9d0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f9c2806f8f0 tx=0x7f9c2002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2380c35b2730b105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.130+0000 7f9c27e00640 1 -- 172.21.15.19:0/1782664235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20036070 con 0x7f9c2806c5c0 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.130+0000 7f9c27e00640 1 -- 172.21.15.19:0/1782664235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c20031600 con 0x7f9c2806c5c0 2024-04-25T11:40:21.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.130+0000 7f9c27e00640 1 -- 172.21.15.19:0/1782664235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c200319d0 con 0x7f9c2806c5c0 2024-04-25T11:40:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 -- 172.21.15.19:0/1782664235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 msgr2=0x7f9c2806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/1782664235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c2806c9d0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f9c2806f8f0 tx=0x7f9c2002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 -- 172.21.15.19:0/1782664235 shutdown_connections 2024-04-25T11:40:21.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/1782664235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c2806c9d0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/1782664235 >> v2:172.21.15.169:3300/0 conn(0x7f9c280cf5e0 0x7f9c280cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 -- 172.21.15.19:0/1782664235 >> 172.21.15.19:0/1782664235 conn(0x7f9c280d1ae0 msgr2=0x7f9c280d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:21.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 -- 172.21.15.19:0/1782664235 shutdown_connections 2024-04-25T11:40:21.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.131+0000 7f9c2fe00640 1 -- 172.21.15.19:0/1782664235 wait complete. 2024-04-25T11:40:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.133+0000 7f9c2fe00640 1 Processor -- start 2024-04-25T11:40:21.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.133+0000 7f9c2fe00640 1 -- start start 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c28158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c280cf5e0 0x7f9c280b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c281443c0 0x7f9c281437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2815edf0 con 0x7f9c2806c5c0 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2815eb10 con 0x7f9c280cf5e0 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9c281447d0 con 0x7f9c281443c0 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c28158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c28158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41358/0 (socket says 172.21.15.19:41358) 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.135+0000 7f9c2d400640 1 -- 172.21.15.19:0/255777105 learned_addr learned my addr 172.21.15.19:0/255777105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2ca00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c280cf5e0 0x7f9c280b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2d400640 1 -- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c281443c0 msgr2=0x7f9c281437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2d400640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c281443c0 0x7f9c281437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2d400640 1 -- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c280cf5e0 msgr2=0x7f9c280b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2d400640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c280cf5e0 0x7f9c280b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2d400640 1 -- 172.21.15.19:0/255777105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c28158b20 con 0x7f9c2806c5c0 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2d400640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c28158360 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b3c0 tx=0x7f9c1c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c25600640 1 -- 172.21.15.19:0/255777105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c011830 con 0x7f9c2806c5c0 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c25600640 1 -- 172.21.15.19:0/255777105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1c016070 con 0x7f9c2806c5c0 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c28158e00 con 0x7f9c2806c5c0 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c25600640 1 -- 172.21.15.19:0/255777105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c011c50 con 0x7f9c2806c5c0 2024-04-25T11:40:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.136+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281593c0 con 0x7f9c2806c5c0 2024-04-25T11:40:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.137+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c28050740 con 0x7f9c2806c5c0 2024-04-25T11:40:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.139+0000 7f9c25600640 1 -- 172.21.15.19:0/255777105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c1c01b460 con 0x7f9c2806c5c0 2024-04-25T11:40:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.140+0000 7f9c25600640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c080744b0 0x7f9c080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.140+0000 7f9c25600640 1 -- 172.21.15.19:0/255777105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f9c1c0957a0 con 0x7f9c2806c5c0 2024-04-25T11:40:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.140+0000 7f9c2ca00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c080744b0 0x7f9c080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.142+0000 7f9c2ca00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c080744b0 0x7f9c080769c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9c28150dc0 tx=0x7f9c14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.144+0000 7f9c25600640 1 -- 172.21.15.19:0/255777105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c1c020020 con 0x7f9c2806c5c0 2024-04-25T11:40:21.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.353+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c2806f950 con 0x7f9c2806c5c0 2024-04-25T11:40:21.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.354+0000 7f9c25600640 1 -- 172.21.15.19:0/255777105 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 62 entries v123) ==== 81+0+3762 (secure 0 0 0) 0x7f9c1c05f4a0 con 0x7f9c2806c5c0 2024-04-25T11:40:21.341 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:40:21.341 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:40:21.341 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:40:21.341 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:40:21.341 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:40:21.341 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:40:21.342 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:40:21.343 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:40:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:40:21.345 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:40:21.346 INFO:teuthology.orchestra.run.smithi019.stderr:listed 62 entries 2024-04-25T11:40:21.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c080744b0 msgr2=0x7f9c080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c080744b0 0x7f9c080769c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9c28150dc0 tx=0x7f9c14008040 comp rx=0 tx=0).stop 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 msgr2=0x7f9c28158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c28158360 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00b3c0 tx=0x7f9c1c00b880 comp rx=0 tx=0).stop 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 shutdown_connections 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9c080744b0 0x7f9c080769c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c280cf5e0 0x7f9c280b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c281443c0 0x7f9c281437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 --2- 172.21.15.19:0/255777105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c2806c5c0 0x7f9c28158360 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.357+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 >> 172.21.15.19:0/255777105 conn(0x7f9c280d1ae0 msgr2=0x7f9c280b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.358+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 shutdown_connections 2024-04-25T11:40:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.358+0000 7f9c2fe00640 1 -- 172.21.15.19:0/255777105 wait complete. 2024-04-25T11:40:21.370 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:40:21.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:21.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:40:21.459 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:40:21.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:21.459 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:40:21.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:40:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.792+0000 7f8903200640 1 Processor -- start 2024-04-25T11:40:21.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.793+0000 7f8903200640 1 -- start start 2024-04-25T11:40:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f8903200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89040b6e10 0x7f890415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f8903200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89041608d0 0x7f8904160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f8903200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8904161220 con 0x7f89040cf2f0 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f8903200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f890406f870 con 0x7f89040b6e10 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f8903200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f89040b7220 con 0x7f89041608d0 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f88fbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89040b6e10 0x7f890415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f88fbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89040b6e10 0x7f890415a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41364/0 (socket says 172.21.15.19:41364) 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.795+0000 7f88fbe00640 1 -- 172.21.15.19:0/90333288 learned_addr learned my addr 172.21.15.19:0/90333288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.796+0000 7f8902800640 1 --2- 172.21.15.19:0/90333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89041608d0 0x7f8904160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.796+0000 7f88fbe00640 1 -- 172.21.15.19:0/90333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89041608d0 msgr2=0x7f8904160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.796+0000 7f88fbe00640 1 --2- 172.21.15.19:0/90333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89041608d0 0x7f8904160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.796+0000 7f88fbe00640 1 -- 172.21.15.19:0/90333288 >> v1:172.21.15.169:6789/0 conn(0x7f89040cf2f0 legacy=0x7f89040b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.796+0000 7f88fbe00640 1 -- 172.21.15.19:0/90333288 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89040b74d0 con 0x7f89040b6e10 2024-04-25T11:40:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.797+0000 7f88fbe00640 1 --2- 172.21.15.19:0/90333288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890415a810 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f8904070dd0 tx=0x7f88ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71078ecb8a432582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.797+0000 7f8901400640 1 -- 172.21.15.19:0/90333288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec037070 con 0x7f89040b6e10 2024-04-25T11:40:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.797+0000 7f8901400640 1 -- 172.21.15.19:0/90333288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec032440 con 0x7f89040b6e10 2024-04-25T11:40:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.797+0000 7f8901400640 1 -- 172.21.15.19:0/90333288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec032810 con 0x7f89040b6e10 2024-04-25T11:40:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 -- 172.21.15.19:0/90333288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 msgr2=0x7f890415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 --2- 172.21.15.19:0/90333288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890415a810 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f8904070dd0 tx=0x7f88ec02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 -- 172.21.15.19:0/90333288 shutdown_connections 2024-04-25T11:40:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 --2- 172.21.15.19:0/90333288 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89041608d0 0x7f8904160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 --2- 172.21.15.19:0/90333288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890415a810 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 -- 172.21.15.19:0/90333288 >> 172.21.15.19:0/90333288 conn(0x7f89040d1720 msgr2=0x7f89040d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 -- 172.21.15.19:0/90333288 shutdown_connections 2024-04-25T11:40:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.798+0000 7f8903200640 1 -- 172.21.15.19:0/90333288 wait complete. 2024-04-25T11:40:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.801+0000 7f8903200640 1 Processor -- start 2024-04-25T11:40:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.801+0000 7f8903200640 1 -- start start 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.802+0000 7f8903200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890414a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.802+0000 7f8903200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89040cf2f0 0x7f890415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.802+0000 7f8903200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89041608d0 0x7f89040d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.802+0000 7f8903200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f890406f500 con 0x7f89040b6e10 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.802+0000 7f8903200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f890406f220 con 0x7f89041608d0 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.802+0000 7f8903200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f890406fd00 con 0x7f89040cf2f0 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890414a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890414a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41372/0 (socket says 172.21.15.19:41372) 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 -- 172.21.15.19:0/2923385696 learned_addr learned my addr 172.21.15.19:0/2923385696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f88fbe00640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89040cf2f0 0x7f890415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8902800640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89041608d0 0x7f89040d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 -- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89040cf2f0 msgr2=0x7f890415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89040cf2f0 0x7f890415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 -- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89041608d0 msgr2=0x7f89040d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89041608d0 0x7f89040d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8901e00640 1 -- 172.21.15.19:0/2923385696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890414aa10 con 0x7f89040b6e10 2024-04-25T11:40:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.803+0000 7f8902800640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89041608d0 0x7f89040d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.804+0000 7f8901e00640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890414a4d0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b420 tx=0x7f88f400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.804+0000 7f88fa000640 1 -- 172.21.15.19:0/2923385696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4011830 con 0x7f89040b6e10 2024-04-25T11:40:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.804+0000 7f88fa000640 1 -- 172.21.15.19:0/2923385696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f4016070 con 0x7f89040b6e10 2024-04-25T11:40:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.804+0000 7f88fa000640 1 -- 172.21.15.19:0/2923385696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4011c10 con 0x7f89040b6e10 2024-04-25T11:40:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.804+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890415cd50 con 0x7f89040b6e10 2024-04-25T11:40:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.804+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890415aaf0 con 0x7f89040b6e10 2024-04-25T11:40:21.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.805+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8904050740 con 0x7f89040b6e10 2024-04-25T11:40:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.807+0000 7f88fa000640 1 -- 172.21.15.19:0/2923385696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88f401b460 con 0x7f89040b6e10 2024-04-25T11:40:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.808+0000 7f88fa000640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88e00744b0 0x7f88e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.808+0000 7f88fa000640 1 -- 172.21.15.19:0/2923385696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8375+0+0 (secure 0 0 0) 0x7f88f4095800 con 0x7f89040b6e10 2024-04-25T11:40:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.809+0000 7f88fbe00640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88e00744b0 0x7f88e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:21.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.810+0000 7f88fbe00640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88e00744b0 0x7f88e00769c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f88ec034000 tx=0x7f88ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:21.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:21.813+0000 7f88fa000640 1 -- 172.21.15.19:0/2923385696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88f4020020 con 0x7f89040b6e10 2024-04-25T11:40:22.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.058+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f890406c770 con 0x7f89040b6e10 2024-04-25T11:40:22.047 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:40:22.060+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:40:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.064+0000 7f88fa000640 1 -- 172.21.15.19:0/2923385696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v294) ==== 161+0+0 (secure 0 0 0) 0x7f88f405f470 con 0x7f89040b6e10 2024-04-25T11:40:22.051 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:40:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88e00744b0 msgr2=0x7f88e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88e00744b0 0x7f88e00769c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f88ec034000 tx=0x7f88ec02e000 comp rx=0 tx=0).stop 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 msgr2=0x7f890414a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890414a4d0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b420 tx=0x7f88f400b8e0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 shutdown_connections 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f88e00744b0 0x7f88e00769c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89041608d0 0x7f89040d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89040cf2f0 0x7f890415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 --2- 172.21.15.19:0/2923385696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89040b6e10 0x7f890414a4d0 secure :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f88f400b420 tx=0x7f88f400b8e0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.067+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 >> 172.21.15.19:0/2923385696 conn(0x7f89040d1720 msgr2=0x7f89040cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.068+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 shutdown_connections 2024-04-25T11:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.068+0000 7f8903200640 1 -- 172.21.15.19:0/2923385696 wait complete. 2024-04-25T11:40:22.072 INFO:tasks.ceph.mds.a.smithi019.stderr: -1090> 2024-04-25T11:40:07.164+0000 7f4a5fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:40:22.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:40:22.213 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:40:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.373+0000 7f7273e00640 1 Processor -- start 2024-04-25T11:40:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.374+0000 7f7273e00640 1 -- start start 2024-04-25T11:40:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.376+0000 7f7273e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 0x7f72740cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.376+0000 7f7273e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72740d1230 con 0x7f727406c5c0 2024-04-25T11:40:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.376+0000 7f7273e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72740d06b0 con 0x7f72740b4ea0 2024-04-25T11:40:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.376+0000 7f7273e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f72740d0990 con 0x7f7274144370 2024-04-25T11:40:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.377+0000 7f7272a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f727406c5c0 0x7f72740b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:40678/0 (socket says 172.21.15.19:40678) 2024-04-25T11:40:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.377+0000 7f7272a00640 1 -- 172.21.15.19:0/1666838575 learned_addr learned my addr 172.21.15.19:0/1666838575 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.377+0000 7f7273400640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 0x7f72740cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.377+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604092697 0 0) 0x7f72740d1230 con 0x7f727406c5c0 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.377+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7258003520 con 0x7f727406c5c0 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3886449914 0 0) 0x7f7258003520 con 0x7f727406c5c0 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 msgr2=0x7f72740cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 0x7f72740cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 >> v1:172.21.15.169:6789/0 conn(0x7f72740b4ea0 legacy=0x7f727415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72740d0c70 con 0x7f727406c5c0 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7264001670 con 0x7f727406c5c0 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 >> v1:172.21.15.19:6789/0 conn(0x7f727406c5c0 legacy=0x7f72740b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:22.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 0x7f72580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258004060 0x7f7258004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7258004d90 0x7f7258005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72740d0990 con 0x7f7258004d90 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f725800bc50 con 0x7f7258004060 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f725800be50 con 0x7f7274144370 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7273400640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7258004d90 0x7f7258005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.378+0000 7f7272a00640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 0x7f72580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.379+0000 7f726a000640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258004060 0x7f7258004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.379+0000 7f7273400640 1 -- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 msgr2=0x7f72580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.379+0000 7f7273400640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 0x7f72580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.379+0000 7f7273400640 1 -- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258004060 msgr2=0x7f7258004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.379+0000 7f7273400640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7258004060 0x7f7258004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.379+0000 7f7273400640 1 -- 172.21.15.19:0/1666838575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7258003520 con 0x7f7258004d90 2024-04-25T11:40:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.379+0000 7f7272a00640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7274144370 0x7f72580039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.380+0000 7f7273400640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7258004d90 0x7f7258005260 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f726c007330 tx=0x7f7258005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734e9caf8ee8d7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.380+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c009be0 con 0x7f7258004d90 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.380+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726c036600 con 0x7f7258004d90 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.380+0000 7f7272000640 1 -- 172.21.15.19:0/1666838575 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c0369d0 con 0x7f7258004d90 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.381+0000 7f7273e00640 1 -- 172.21.15.19:0/1666838575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7258004d90 msgr2=0x7f7258005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.381+0000 7f7273e00640 1 --2- 172.21.15.19:0/1666838575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7258004d90 0x7f7258005260 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f726c007330 tx=0x7f7258005830 comp rx=0 tx=0).stop 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.381+0000 7f7273400640 1 -- 172.21.15.19:0/1666838575 reap_dead start 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.381+0000 7f7273e00640 1 -- 172.21.15.19:0/1666838575 shutdown_connections 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.381+0000 7f7273e00640 1 -- 172.21.15.19:0/1666838575 >> 172.21.15.19:0/1666838575 conn(0x7f72740d1720 msgr2=0x7f72740d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.381+0000 7f7273e00640 1 -- 172.21.15.19:0/1666838575 shutdown_connections 2024-04-25T11:40:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.381+0000 7f7273e00640 1 -- 172.21.15.19:0/1666838575 wait complete. 2024-04-25T11:40:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.383+0000 7f7273e00640 1 Processor -- start 2024-04-25T11:40:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.384+0000 7f7273e00640 1 -- start start 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.385+0000 7f7273e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f727415c820 0x7f72740cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.385+0000 7f7273e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72740cd6c0 0x7f727415f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.385+0000 7f7273e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f727415fe50 0x7f7274160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.385+0000 7f7273e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72740d0450 con 0x7f727415c820 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.385+0000 7f7273e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f72740d02d0 con 0x7f727415fe50 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.385+0000 7f7273e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7274147760 con 0x7f72740cd6c0 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f727415c820 0x7f72740cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f727415c820 0x7f72740cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41430/0 (socket says 172.21.15.19:41430) 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 -- 172.21.15.19:0/2525237254 learned_addr learned my addr 172.21.15.19:0/2525237254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 -- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72740cd6c0 msgr2=0x7f727415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72740cd6c0 0x7f727415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 -- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f727415fe50 msgr2=0x7f7274160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7273400640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f727415fe50 0x7f7274160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f727415fe50 0x7f7274160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:22.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 -- 172.21.15.19:0/2525237254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7274160260 con 0x7f727415c820 2024-04-25T11:40:22.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.386+0000 7f7272a00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f727415c820 0x7f72740cd180 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f726400db10 tx=0x7f726400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:22.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.387+0000 7f726b400640 1 -- 172.21.15.19:0/2525237254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264019070 con 0x7f727415c820 2024-04-25T11:40:22.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.387+0000 7f726b400640 1 -- 172.21.15.19:0/2525237254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7264004cb0 con 0x7f727415c820 2024-04-25T11:40:22.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.387+0000 7f726b400640 1 -- 172.21.15.19:0/2525237254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264004e60 con 0x7f727415c820 2024-04-25T11:40:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.387+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72740cc660 con 0x7f727415c820 2024-04-25T11:40:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.387+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72740cbb80 con 0x7f727415c820 2024-04-25T11:40:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.388+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7274050740 con 0x7f727415c820 2024-04-25T11:40:22.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.390+0000 7f726b400640 1 -- 172.21.15.19:0/2525237254 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f72640072d0 con 0x7f727415c820 2024-04-25T11:40:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.391+0000 7f726b400640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72500744e0 0x7f72500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.392+0000 7f726b400640 1 -- 172.21.15.19:0/2525237254 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8461+0+0 (secure 0 0 0) 0x7f72640957f0 con 0x7f727415c820 2024-04-25T11:40:22.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.392+0000 7f726a000640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72500744e0 0x7f72500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:22.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.393+0000 7f726a000640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72500744e0 0x7f72500769f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f725c001640 tx=0x7f725c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:22.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.397+0000 7f726b400640 1 -- 172.21.15.19:0/2525237254 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7274160260 con 0x7f727415c820 2024-04-25T11:40:22.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:22.679+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f7274146fb0 con 0x7f727415c820 2024-04-25T11:40:23.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.068+0000 7f726b400640 1 -- 172.21.15.19:0/2525237254 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v295) ==== 107+0+0 (secure 0 0 0) 0x7f726405f4a0 con 0x7f727415c820 2024-04-25T11:40:23.055 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72500744e0 msgr2=0x7f72500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72500744e0 0x7f72500769f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f725c001640 tx=0x7f725c009040 comp rx=0 tx=0).stop 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f727415c820 msgr2=0x7f72740cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f727415c820 0x7f72740cd180 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f726400db10 tx=0x7f726400df10 comp rx=0 tx=0).stop 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 shutdown_connections 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72500744e0 0x7f72500769f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f727415fe50 0x7f7274160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72740cd6c0 0x7f727415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 --2- 172.21.15.19:0/2525237254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f727415c820 0x7f72740cd180 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.072+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 >> 172.21.15.19:0/2525237254 conn(0x7f72740d1720 msgr2=0x7f727415d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.073+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 shutdown_connections 2024-04-25T11:40:23.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.073+0000 7f7273e00640 1 -- 172.21.15.19:0/2525237254 wait complete. 2024-04-25T11:40:23.095 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:40:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.291+0000 7feedd3c2f40 1 Processor -- start 2024-04-25T11:40:23.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.291+0000 7feedd3c2f40 1 -- start start 2024-04-25T11:40:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feedd3c2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a463790 0x55ad5a4e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feedd3c2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 0x55ad5a4e9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feedd3c2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a4e9610 0x55ad5a4ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feedd3c2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad5a41bf90 con 0x55ad5a463790 2024-04-25T11:40:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feedd3c2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ad5a3c3d20 con 0x55ad5a4e9610 2024-04-25T11:40:23.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feedd3c2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ad5a466390 con 0x55ad5a465af0 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a463790 0x55ad5a4e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a463790 0x55ad5a4e8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41456/0 (socket says 172.21.15.19:41456) 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed6600640 1 -- 172.21.15.19:0/2878424739 learned_addr learned my addr 172.21.15.19:0/2878424739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed7000640 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a4e9610 0x55ad5a4ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed5c00640 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 0x55ad5a4e9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed6600640 1 -- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 msgr2=0x55ad5a4e9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed6600640 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 0x55ad5a4e9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed6600640 1 -- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a4e9610 msgr2=0x55ad5a4ea480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.293+0000 7feed6600640 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a4e9610 0x55ad5a4ea480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.294+0000 7feed6600640 1 -- 172.21.15.19:0/2878424739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5a4f14e0 con 0x55ad5a463790 2024-04-25T11:40:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.294+0000 7feed6600640 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a463790 0x55ad5a4e8e50 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7feec8045fb0 tx=0x7feec8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc575df3a66d9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.294+0000 7feecfe00640 1 -- 172.21.15.19:0/2878424739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec806a070 con 0x55ad5a463790 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.294+0000 7feecfe00640 1 -- 172.21.15.19:0/2878424739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8052a70 con 0x55ad5a463790 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.294+0000 7feecfe00640 1 -- 172.21.15.19:0/2878424739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8052e60 con 0x55ad5a463790 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2878424739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a463790 msgr2=0x55ad5a4e8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a463790 0x55ad5a4e8e50 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7feec8045fb0 tx=0x7feec8051000 comp rx=0 tx=0).stop 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2878424739 shutdown_connections 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a4e9610 0x55ad5a4ea480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 0x55ad5a4e9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2878424739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a463790 0x55ad5a4e8e50 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2878424739 >> 172.21.15.19:0/2878424739 conn(0x55ad5a33fac0 msgr2=0x55ad5a4f36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2878424739 shutdown_connections 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.295+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2878424739 wait complete. 2024-04-25T11:40:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.296+0000 7feedd3c2f40 1 Processor -- start 2024-04-25T11:40:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.296+0000 7feedd3c2f40 1 -- start start 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feedd3c2f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a463790 0x55ad5a4f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feedd3c2f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 0x55ad5a4e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feedd3c2f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a4e9610 0x55ad5a41ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feedd3c2f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ad5a4f1950 con 0x55ad5a4e9610 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feedd3c2f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ad5a4469b0 con 0x55ad5a463790 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feedd3c2f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ad5a4f8770 con 0x55ad5a465af0 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a4e9610 0x55ad5a41ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a4e9610 0x55ad5a41ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41458/0 (socket says 172.21.15.19:41458) 2024-04-25T11:40:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 learned_addr learned my addr 172.21.15.19:0/2130667975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a463790 0x55ad5a4f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 msgr2=0x55ad5a4e4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed5c00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 0x55ad5a4e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ad5a465af0 0x55ad5a4e4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a463790 msgr2=0x55ad5a4f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a463790 0x55ad5a4f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad5a4e9b40 con 0x55ad5a4e9610 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.298+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ad5a463790 0x55ad5a4f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.299+0000 7feed7000640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a4e9610 0x55ad5a41ce30 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7feed000b3d0 tx=0x7feed000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.299+0000 7feecea00640 1 -- 172.21.15.19:0/2130667975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0017020 con 0x55ad5a4e9610 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.299+0000 7feecea00640 1 -- 172.21.15.19:0/2130667975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed0012660 con 0x55ad5a4e9610 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.299+0000 7feecea00640 1 -- 172.21.15.19:0/2130667975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0012a50 con 0x55ad5a4e9610 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.299+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ad5a4ea040 con 0x55ad5a4e9610 2024-04-25T11:40:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.299+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad5a4ea600 con 0x55ad5a4e9610 2024-04-25T11:40:23.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.301+0000 7feecea00640 1 -- 172.21.15.19:0/2130667975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feed001a020 con 0x55ad5a4e9610 2024-04-25T11:40:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.302+0000 7feecea00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feeb4072e90 0x7feeb40753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.302+0000 7feecea00640 1 -- 172.21.15.19:0/2130667975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8461+0+0 (secure 0 0 0) 0x7feed0094ff0 con 0x55ad5a4e9610 2024-04-25T11:40:23.289 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:40:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.303+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ad5a41a1b0 0x55ad5a41a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.303+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59ada0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.303+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feeb4072e90 0x7feeb40753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.303+0000 7feed5c00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ad5a41a1b0 0x55ad5a41a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.305+0000 7feed5c00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ad5a41a1b0 0x55ad5a41a5c0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.305+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feeb4072e90 0x7feeb40753a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7feec8067000 tx=0x7feec8053000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.308+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v124'971 uv944 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.309+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a59b140 con 0x55ad5a41a1b0 2024-04-25T11:40:23.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.312+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v124'972 uv972 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.312+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59b4e0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.302 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:40:23.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.315+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v124'973 uv973 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.315+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c890 con 0x55ad5a41a1b0 2024-04-25T11:40:23.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.318+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'974 uv974 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.318+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a4f8380 con 0x55ad5a41a1b0 2024-04-25T11:40:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.322+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'975 uv975 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.322+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59b880 con 0x55ad5a41a1b0 2024-04-25T11:40:23.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.325+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'976 uv976 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.326+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59bc70 con 0x55ad5a41a1b0 2024-04-25T11:40:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.329+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v124'977 uv950 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.329+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a59c060 con 0x55ad5a41a1b0 2024-04-25T11:40:23.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.334+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v124'978 uv978 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.334+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c060 con 0x55ad5a41a1b0 2024-04-25T11:40:23.324 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:40:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.338+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v124'979 uv979 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.338+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.342+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'980 uv980 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.342+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.346+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'981 uv981 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.346+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.350+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'982 uv982 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.350+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.355+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v124'983 uv970 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.355+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.359+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'984 uv984 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.359+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.350 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:40:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.363+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v124'985 uv985 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.364+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.367+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'986 uv986 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.367+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.370+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'987 uv987 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.370+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c010 con 0x55ad5a41a1b0 2024-04-25T11:40:23.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.374+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'988 uv988 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.374+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c3d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.378+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'989 uv989 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.378+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c3d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.382+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v124'990 uv963 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.383+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a59c3d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.386+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v124'991 uv991 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.386+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c3d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.391+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v124'992 uv992 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.378 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:40:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.391+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c3d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.395+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'993 uv993 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.395+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.399+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'994 uv994 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.399+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.404+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'995 uv995 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.404+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.408+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v124'996 uv969 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.408+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.413+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v124'997 uv997 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.413+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.404 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:40:23.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.417+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v124'998 uv998 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.417+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.422+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'999 uv999 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.422+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.426+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'1000 uv1000 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.426+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.431+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'1001 uv1001 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.431+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ad5a59d140 0x55ad5a59f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.431+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59c2d0 con 0x55ad5a59d140 2024-04-25T11:40:23.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.432+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ad5a59d140 0x55ad5a59f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.433+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ad5a59d140 0x55ad5a59f560 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.438+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'705 uv684 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.438+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a59faa0 con 0x55ad5a59d140 2024-04-25T11:40:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.443+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'706 uv706 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.443+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a59fe40 con 0x55ad5a59d140 2024-04-25T11:40:23.433 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:40:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.447+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'707 uv707 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.447+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a01e0 con 0x55ad5a59d140 2024-04-25T11:40:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.451+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v124'708 uv708 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.452+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a0580 con 0x55ad5a59d140 2024-04-25T11:40:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.456+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'709 uv709 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.456+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a0920 con 0x55ad5a59d140 2024-04-25T11:40:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.460+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v124'710 uv703 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.461+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5a0cc0 con 0x55ad5a59d140 2024-04-25T11:40:23.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.465+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v124'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.465+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.456 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:40:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.469+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v124'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.469+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.474+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.474+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.478+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.479+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.483+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v124'715 uv704 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.483+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.488+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v124'716 uv716 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.488+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.479 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:40:23.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.492+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v124'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.493+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a5750 con 0x55ad5a59d140 2024-04-25T11:40:23.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.498+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v124'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.498+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a5750 con 0x55ad5a59d140 2024-04-25T11:40:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.502+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.502+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.507+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v124'720 uv700 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.507+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.511+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v124'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.511+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.502 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:40:23.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.515+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v124'722 uv722 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.515+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a2070 con 0x55ad5a59d140 2024-04-25T11:40:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.520+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.520+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a2070 con 0x55ad5a59d140 2024-04-25T11:40:23.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.525+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.525+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a1060 con 0x55ad5a59d140 2024-04-25T11:40:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.529+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a59d140 2024-04-25T11:40:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.529+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ad5a5a1570 0x55ad5a5a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.529+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a3ff0 con 0x55ad5a5a1570 2024-04-25T11:40:23.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.530+0000 7feed5c00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ad5a5a1570 0x55ad5a5a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.531+0000 7feed5c00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ad5a5a1570 0x55ad5a5a39b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.536+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a5a1570 2024-04-25T11:40:23.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.536+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5a40b0 con 0x55ad5a5a1570 2024-04-25T11:40:23.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.540+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v124'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a5a1570 2024-04-25T11:40:23.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.540+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a44d0 con 0x55ad5a5a1570 2024-04-25T11:40:23.531 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:40:23.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.544+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v124'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a5a1570 2024-04-25T11:40:23.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.544+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a54e0 con 0x55ad5a5a1570 2024-04-25T11:40:23.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.548+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a5a1570 2024-04-25T11:40:23.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.548+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a54e0 con 0x55ad5a5a1570 2024-04-25T11:40:23.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.553+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a5a1570 2024-04-25T11:40:23.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.553+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a44d0 con 0x55ad5a5a1570 2024-04-25T11:40:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.557+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a5a1570 2024-04-25T11:40:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.557+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a44d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.561+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'142 uv141 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.561+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5a44d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.566+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'143 uv143 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.566+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a44d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.569+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'144 uv144 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.556 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:40:23.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.570+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a44d0 con 0x55ad5a41a1b0 2024-04-25T11:40:23.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.574+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'145 uv145 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7feec000a440 con 0x55ad5a41a1b0 2024-04-25T11:40:23.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.574+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ad5a5a4e10 0x55ad5a5a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.574+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a7770 con 0x55ad5a5a4e10 2024-04-25T11:40:23.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.575+0000 7feed7000640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ad5a5a4e10 0x55ad5a5a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.577+0000 7feed7000640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ad5a5a4e10 0x55ad5a5a7230 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.580+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.581+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5a7b10 con 0x55ad5a5a4e10 2024-04-25T11:40:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.585+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v124'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.585+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a7fc0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.576 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:40:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.589+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v124'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.590+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a9870 con 0x55ad5a5a4e10 2024-04-25T11:40:23.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.593+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.594+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a9c10 con 0x55ad5a5a4e10 2024-04-25T11:40:23.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.598+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.598+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5a8860 con 0x55ad5a5a4e10 2024-04-25T11:40:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.602+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.603+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ad5a5a9160 0x55ad5a5abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.603+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ac500 con 0x55ad5a5a9160 2024-04-25T11:40:23.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.603+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ad5a5a9160 0x55ad5a5abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.605+0000 7feed6600640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ad5a5a9160 0x55ad5a5abfc0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.609+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v124'349 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.609+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5ac500 con 0x55ad5a5a9160 2024-04-25T11:40:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.614+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v124'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.614+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ac500 con 0x55ad5a5a9160 2024-04-25T11:40:23.605 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:40:23.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.617+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v124'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.618+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ad930 con 0x55ad5a5a9160 2024-04-25T11:40:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.621+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.621+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ad930 con 0x55ad5a5a9160 2024-04-25T11:40:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.626+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.626+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ac920 con 0x55ad5a5a9160 2024-04-25T11:40:23.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.630+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.630+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ac920 con 0x55ad5a5a9160 2024-04-25T11:40:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.634+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v124'355 uv348 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.634+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5ac920 con 0x55ad5a5a9160 2024-04-25T11:40:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.638+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v124'356 uv356 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.638+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ac920 con 0x55ad5a5a9160 2024-04-25T11:40:23.629 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:40:23.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.642+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v124'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.642+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ac920 con 0x55ad5a5a9160 2024-04-25T11:40:23.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.646+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.647+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5ac920 con 0x55ad5a5a9160 2024-04-25T11:40:23.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.651+0000 7feed6600640 1 -- 172.21.15.19:0/2130667975 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec8073440 con 0x55ad5a5a9160 2024-04-25T11:40:23.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.651+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ad5a5ace30 0x55ad5a5af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:23.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.651+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5af8e0 con 0x55ad5a5ace30 2024-04-25T11:40:23.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.652+0000 7feed5c00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ad5a5ace30 0x55ad5a5af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:23.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.653+0000 7feed5c00640 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ad5a5ace30 0x55ad5a5af250 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:23.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.657+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v124'158 uv157 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.657+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5af8e0 con 0x55ad5a5ace30 2024-04-25T11:40:23.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.661+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v124'159 uv159 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.661+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5ace30 2024-04-25T11:40:23.652 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:40:23.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.665+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v124'160 uv160 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.665+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5ace30 2024-04-25T11:40:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.670+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'161 uv161 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.670+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5ace30 2024-04-25T11:40:23.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.674+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'162 uv162 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.674+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a41a1b0 2024-04-25T11:40:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.679+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a41a1b0 2024-04-25T11:40:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.679+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a41a1b0 2024-04-25T11:40:23.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.683+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v124'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a41a1b0 2024-04-25T11:40:23.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.683+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a41a1b0 2024-04-25T11:40:23.673 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:40:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.686+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v124'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a41a1b0 2024-04-25T11:40:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.687+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b0d40 con 0x55ad5a41a1b0 2024-04-25T11:40:23.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.691+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a41a1b0 2024-04-25T11:40:23.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.691+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b0d40 con 0x55ad5a41a1b0 2024-04-25T11:40:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.695+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a41a1b0 2024-04-25T11:40:23.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.695+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a41a1b0 2024-04-25T11:40:23.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.699+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a41a1b0 2024-04-25T11:40:23.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.699+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.704+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v124'314 uv308 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.704+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.708+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v124'315 uv315 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.708+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.700 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:40:23.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.713+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v124'316 uv316 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.713+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.717+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'317 uv317 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.718+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.722+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'318 uv318 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.722+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.726+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v124'319 uv313 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.726+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.731+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v124'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.731+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.722 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:40:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.735+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v124'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.735+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5afd30 con 0x55ad5a5a1570 2024-04-25T11:40:23.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.739+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.739+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b2ac0 con 0x55ad5a5a1570 2024-04-25T11:40:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.744+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5a1570 2024-04-25T11:40:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.744+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b0120 con 0x55ad5a5a4e10 2024-04-25T11:40:23.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.749+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v124'137 uv135 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.749+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.752+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'138 uv138 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.752+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.743 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:40:23.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.757+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v124'139 uv139 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.757+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.761+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'140 uv140 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.761+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.765+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v124'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.765+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.770+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v124'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.770+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.761 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:40:23.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.775+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v124'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.775+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b14d0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.779+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.780+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b14d0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.784+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.784+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5a4e10 2024-04-25T11:40:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.788+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feed0023070 con 0x55ad5a5a4e10 2024-04-25T11:40:23.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.788+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.792+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v124'161 uv160 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.792+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.796+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v124'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.796+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.786 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:40:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.800+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v124'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.800+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.805+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.805+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.809+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.809+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.813+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v124'389 uv380 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.813+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.818+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v124'390 uv390 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.818+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.809 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:40:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.822+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v124'391 uv391 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.822+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.826+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v124'392 uv392 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.826+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.831+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'393 uv393 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.831+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.835+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v124'394 uv387 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.835+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.839+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v124'395 uv395 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.839+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.830 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:40:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.843+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v124'396 uv396 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.844+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b14d0 con 0x55ad5a5ace30 2024-04-25T11:40:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.848+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'397 uv397 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.848+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b14d0 con 0x55ad5a5ace30 2024-04-25T11:40:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.852+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'398 uv398 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.852+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b04c0 con 0x55ad5a5ace30 2024-04-25T11:40:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.856+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'399 uv399 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.856+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55ad5a5b0c20 con 0x55ad5a5ace30 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feed5c00640 1 -- 172.21.15.19:0/2130667975 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'400 uv400 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7feec0023040 con 0x55ad5a5ace30 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ad5a59d140 msgr2=0x55ad5a59f560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55ad5a59d140 0x55ad5a59f560 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ad5a5a9160 msgr2=0x55ad5a5abfc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55ad5a5a9160 0x55ad5a5abfc0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ad5a5ace30 msgr2=0x55ad5a5af250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55ad5a5ace30 0x55ad5a5af250 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ad5a5a1570 msgr2=0x55ad5a5a39b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55ad5a5a1570 0x55ad5a5a39b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ad5a41a1b0 msgr2=0x55ad5a41a5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55ad5a41a1b0 0x55ad5a41a5c0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ad5a5a4e10 msgr2=0x55ad5a5a7230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55ad5a5a4e10 0x55ad5a5a7230 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 reap_dead start 2024-04-25T11:40:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feed7000640 1 -- 172.21.15.19:0/2130667975 reap_dead start 2024-04-25T11:40:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feeb4072e90 msgr2=0x7feeb40753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feeb4072e90 0x7feeb40753a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7feec8067000 tx=0x7feec8053000 comp rx=0 tx=0).stop 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a4e9610 msgr2=0x55ad5a41ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.861+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a4e9610 0x55ad5a41ce30 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7feed000b3d0 tx=0x7feed000b890 comp rx=0 tx=0).stop 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.862+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 shutdown_connections 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.862+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7feeb4072e90 0x7feeb40753a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.862+0000 7feedd3c2f40 1 --2- 172.21.15.19:0/2130667975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ad5a4e9610 0x55ad5a41ce30 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.862+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 >> 172.21.15.19:0/2130667975 conn(0x55ad5a33fac0 msgr2=0x55ad5a4e8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.862+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 shutdown_connections 2024-04-25T11:40:23.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:23.862+0000 7feedd3c2f40 1 -- 172.21.15.19:0/2130667975 wait complete. 2024-04-25T11:40:23.880 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:40:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.131+0000 7f91d952ff40 1 Processor -- start 2024-04-25T11:40:24.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.131+0000 7f91d952ff40 1 -- start start 2024-04-25T11:40:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.132+0000 7f91d952ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfe5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.132+0000 7f91d952ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 0x55cd8df60010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d952ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 0x55cd8df60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d952ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd8df5dea0 con 0x55cd8df15210 2024-04-25T11:40:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d952ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cd8debbd20 con 0x55cd8dfeefd0 2024-04-25T11:40:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d952ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55cd8df5e420 con 0x55cd8df15620 2024-04-25T11:40:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfe5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfe5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51934/0 (socket says 172.21.15.19:51934) 2024-04-25T11:40:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d2600640 1 -- 172.21.15.19:0/2066683228 learned_addr learned my addr 172.21.15.19:0/2066683228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:24.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d3000640 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 0x55cd8df60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d2600640 1 -- 172.21.15.19:0/2066683228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 msgr2=0x55cd8df60010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:24.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.133+0000 7f91d2600640 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 0x55cd8df60010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.134+0000 7f91d2600640 1 -- 172.21.15.19:0/2066683228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 msgr2=0x55cd8df60550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:24.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.134+0000 7f91d2600640 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 0x55cd8df60550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.134+0000 7f91d2600640 1 -- 172.21.15.19:0/2066683228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8df47440 con 0x55cd8df15210 2024-04-25T11:40:24.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.134+0000 7f91d2600640 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfe5240 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c4b0 tx=0x7f91c400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e1ba4029909b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.134+0000 7f91cbe00640 1 -- 172.21.15.19:0/2066683228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40116b0 con 0x55cd8df15210 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.134+0000 7f91cbe00640 1 -- 172.21.15.19:0/2066683228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c4017070 con 0x55cd8df15210 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.134+0000 7f91cbe00640 1 -- 172.21.15.19:0/2066683228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4011a90 con 0x55cd8df15210 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 -- 172.21.15.19:0/2066683228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 msgr2=0x55cd8dfe5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfe5240 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c4b0 tx=0x7f91c400c970 comp rx=0 tx=0).stop 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 -- 172.21.15.19:0/2066683228 shutdown_connections 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 0x55cd8df60550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 0x55cd8df60010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 --2- 172.21.15.19:0/2066683228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfe5240 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 -- 172.21.15.19:0/2066683228 >> 172.21.15.19:0/2066683228 conn(0x55cd8de37ac0 msgr2=0x55cd8df616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 -- 172.21.15.19:0/2066683228 shutdown_connections 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.135+0000 7f91d952ff40 1 -- 172.21.15.19:0/2066683228 wait complete. 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.136+0000 7f91d952ff40 1 Processor -- start 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.136+0000 7f91d952ff40 1 -- start start 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d952ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfc9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d952ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 0x55cd8df61df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d952ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 0x55cd8df62330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d952ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd8df47720 con 0x55cd8df15210 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d952ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55cd8dfeb720 con 0x55cd8dfeefd0 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d952ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55cd8dfe17e0 con 0x55cd8df15620 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfc9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfc9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51938/0 (socket says 172.21.15.19:51938) 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.138+0000 7f91d2600640 1 -- 172.21.15.19:0/121016661 learned_addr learned my addr 172.21.15.19:0/121016661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d1c00640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 0x55cd8df61df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d3000640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 0x55cd8df62330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d2600640 1 -- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 msgr2=0x55cd8df61df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d2600640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55cd8df15620 0x55cd8df61df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d2600640 1 -- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 msgr2=0x55cd8df62330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d2600640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55cd8dfeefd0 0x55cd8df62330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d2600640 1 -- 172.21.15.19:0/121016661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8dfca410 con 0x55cd8df15210 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.139+0000 7f91d2600640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfc9ed0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f91c4014040 tx=0x7f91c40033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.140+0000 7f91caa00640 1 -- 172.21.15.19:0/121016661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4021070 con 0x55cd8df15210 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.140+0000 7f91caa00640 1 -- 172.21.15.19:0/121016661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c40126a0 con 0x55cd8df15210 2024-04-25T11:40:24.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.140+0000 7f91caa00640 1 -- 172.21.15.19:0/121016661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4012a90 con 0x55cd8df15210 2024-04-25T11:40:24.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.140+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cd8dfe36f0 con 0x55cd8df15210 2024-04-25T11:40:24.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.140+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8dfe3cb0 con 0x55cd8df15210 2024-04-25T11:40:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.142+0000 7f91caa00640 1 -- 172.21.15.19:0/121016661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f91c402c060 con 0x55cd8df15210 2024-04-25T11:40:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.143+0000 7f91caa00640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f91b4072e90 0x7f91b40753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.143+0000 7f91caa00640 1 -- 172.21.15.19:0/121016661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8461+0+0 (secure 0 0 0) 0x7f91c4095ff0 con 0x55cd8df15210 2024-04-25T11:40:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.143+0000 7f91d952ff40 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cd8dfe09b0 0x55cd8dfe0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.143+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55cd8dfebb70 con 0x55cd8dfe09b0 2024-04-25T11:40:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.144+0000 7f91d1c00640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f91b4072e90 0x7f91b40753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.144+0000 7f91d3000640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cd8dfe09b0 0x55cd8dfe0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.145+0000 7f91d1c00640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f91b4072e90 0x7f91b40753a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x55cd8e07d450 tx=0x7f91c00012d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:24.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.145+0000 7f91d3000640 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cd8dfe09b0 0x55cd8dfe0dc0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.149+0000 7f91d3000640 1 -- 172.21.15.19:0/121016661 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v124'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f91cc05c040 con 0x55cd8dfe09b0 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.149+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cd8dfe09b0 msgr2=0x55cd8dfe0dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.149+0000 7f91d952ff40 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55cd8dfe09b0 0x55cd8dfe0dc0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f91b4072e90 msgr2=0x7f91b40753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f91b4072e90 0x7f91b40753a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x55cd8e07d450 tx=0x7f91c00012d0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 msgr2=0x55cd8dfc9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 --2- 172.21.15.19:0/121016661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8df15210 0x55cd8dfc9ed0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f91c4014040 tx=0x7f91c40033a0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d3000640 1 -- 172.21.15.19:0/121016661 reap_dead start 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 shutdown_connections 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 >> 172.21.15.19:0/121016661 conn(0x55cd8de37ac0 msgr2=0x55cd8dfe8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 shutdown_connections 2024-04-25T11:40:24.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.150+0000 7f91d952ff40 1 -- 172.21.15.19:0/121016661 wait complete. 2024-04-25T11:40:24.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:40:24.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.441+0000 7f2aa1a00640 1 Processor -- start 2024-04-25T11:40:24.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.441+0000 7f2aa1a00640 1 -- start start 2024-04-25T11:40:24.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.443+0000 7f2aa1a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a9c0b7c90 0x7f2a9c0bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.443+0000 7f2aa1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0b4db0 0x7f2a9c0b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.443+0000 7f2aa1a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9c0d4900 con 0x7f2a9c0d4250 2024-04-25T11:40:24.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.443+0000 7f2aa1a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9c0b9730 con 0x7f2a9c0b7c90 2024-04-25T11:40:24.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.443+0000 7f2aa1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c0d4be0 con 0x7f2a9c0b4db0 2024-04-25T11:40:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a9c0b7c90 0x7f2a9c0bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2a9c0d4250 0x7f2a9c0b7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55644/0 (socket says 172.21.15.19:55644) 2024-04-25T11:40:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9be00640 1 -- 172.21.15.19:0/219052040 learned_addr learned my addr 172.21.15.19:0/219052040 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2aa1000640 1 --2- 172.21.15.19:0/219052040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0b4db0 0x7f2a9c0b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9b400640 1 -- 172.21.15.19:0/219052040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0b4db0 msgr2=0x7f2a9c0b91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9b400640 1 --2- 172.21.15.19:0/219052040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0b4db0 0x7f2a9c0b91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9b400640 1 -- 172.21.15.19:0/219052040 >> v1:172.21.15.19:6789/0 conn(0x7f2a9c0d4250 legacy=0x7f2a9c0b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9b400640 1 -- 172.21.15.19:0/219052040 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c0d4ec0 con 0x7f2a9c0b7c90 2024-04-25T11:40:24.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.444+0000 7f2a9aa00640 1 -- 172.21.15.19:0/219052040 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4000593039 0 0) 0x7f2a9c0d4900 con 0x7f2a9c0d4250 2024-04-25T11:40:24.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.446+0000 7f2a9b400640 1 --2- 172.21.15.19:0/219052040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c0b7c90 0x7f2a9c0bb2e0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009d90 tx=0x7f2a8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7e4c2313799136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:24.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.446+0000 7f2a9aa00640 1 -- 172.21.15.19:0/219052040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84007740 con 0x7f2a9c0b7c90 2024-04-25T11:40:24.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.446+0000 7f2a9aa00640 1 -- 172.21.15.19:0/219052040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a840078e0 con 0x7f2a9c0b7c90 2024-04-25T11:40:24.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.446+0000 7f2a9aa00640 1 -- 172.21.15.19:0/219052040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84007cb0 con 0x7f2a9c0b7c90 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 -- 172.21.15.19:0/219052040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c0b7c90 msgr2=0x7f2a9c0bb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/219052040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c0b7c90 0x7f2a9c0bb2e0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009d90 tx=0x7f2a8402a660 comp rx=0 tx=0).stop 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 -- 172.21.15.19:0/219052040 shutdown_connections 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/219052040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0b4db0 0x7f2a9c0b91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/219052040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c0b7c90 0x7f2a9c0bb2e0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 -- 172.21.15.19:0/219052040 >> 172.21.15.19:0/219052040 conn(0x7f2a9c0d2760 msgr2=0x7f2a9c0d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 -- 172.21.15.19:0/219052040 shutdown_connections 2024-04-25T11:40:24.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.447+0000 7f2aa1a00640 1 -- 172.21.15.19:0/219052040 wait complete. 2024-04-25T11:40:24.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.450+0000 7f2aa1a00640 1 Processor -- start 2024-04-25T11:40:24.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.450+0000 7f2aa1a00640 1 -- start start 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.451+0000 7f2aa1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c0b4db0 0x7f2a9c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.451+0000 7f2aa1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0d4250 0x7f2a9c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c157f20 0x7f2a9c0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c0d1700 con 0x7f2a9c157f20 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c0d5480 con 0x7f2a9c0b4db0 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c163ff0 con 0x7f2a9c0d4250 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c157f20 0x7f2a9c0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c157f20 0x7f2a9c0cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51960/0 (socket says 172.21.15.19:51960) 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 -- 172.21.15.19:0/3710953483 learned_addr learned my addr 172.21.15.19:0/3710953483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2a9be00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c0b4db0 0x7f2a9c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2a9b400640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0d4250 0x7f2a9c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 -- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0d4250 msgr2=0x7f2a9c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0d4250 0x7f2a9c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 -- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c0b4db0 msgr2=0x7f2a9c157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c0b4db0 0x7f2a9c157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:24.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.452+0000 7f2aa1000640 1 -- 172.21.15.19:0/3710953483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c0ccd20 con 0x7f2a9c157f20 2024-04-25T11:40:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.453+0000 7f2aa1000640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c157f20 0x7f2a9c0cddc0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400d5d0 tx=0x7f2a9400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.453+0000 7f2a93e00640 1 -- 172.21.15.19:0/3710953483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940125f0 con 0x7f2a9c157f20 2024-04-25T11:40:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.453+0000 7f2a93e00640 1 -- 172.21.15.19:0/3710953483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a94019070 con 0x7f2a9c157f20 2024-04-25T11:40:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.453+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c0cd000 con 0x7f2a9c157f20 2024-04-25T11:40:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.453+0000 7f2a93e00640 1 -- 172.21.15.19:0/3710953483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940129b0 con 0x7f2a9c157f20 2024-04-25T11:40:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.453+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c0cd5c0 con 0x7f2a9c157f20 2024-04-25T11:40:24.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.454+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c076c60 con 0x7f2a9c157f20 2024-04-25T11:40:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.456+0000 7f2a93e00640 1 -- 172.21.15.19:0/3710953483 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a94012b50 con 0x7f2a9c157f20 2024-04-25T11:40:24.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.457+0000 7f2a93e00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a780744b0 0x7f2a780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:24.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.457+0000 7f2a93e00640 1 -- 172.21.15.19:0/3710953483 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8461+0+0 (secure 0 0 0) 0x7f2a940961a0 con 0x7f2a9c157f20 2024-04-25T11:40:24.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.458+0000 7f2a9be00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a780744b0 0x7f2a780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:24.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.459+0000 7f2a9be00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a780744b0 0x7f2a780769c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000f60 tx=0x7f2a8c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:24.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.463+0000 7f2a93e00640 1 -- 172.21.15.19:0/3710953483 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a9405fdd0 con 0x7f2a9c157f20 2024-04-25T11:40:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:24.708+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f2a9c0cbf60 con 0x7f2a9c157f20 2024-04-25T11:40:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.373+0000 7f2a93e00640 1 -- 172.21.15.19:0/3710953483 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v296) ==== 162+0+0 (secure 0 0 0) 0x7f2a9405f6d0 con 0x7f2a9c157f20 2024-04-25T11:40:25.360 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:40:25.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.376+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a780744b0 msgr2=0x7f2a780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:25.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.376+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a780744b0 0x7f2a780769c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000f60 tx=0x7f2a8c007c00 comp rx=0 tx=0).stop 2024-04-25T11:40:25.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.376+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c157f20 msgr2=0x7f2a9c0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:25.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.376+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c157f20 0x7f2a9c0cddc0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400d5d0 tx=0x7f2a9400da90 comp rx=0 tx=0).stop 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 shutdown_connections 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a780744b0 0x7f2a780769c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a9c157f20 0x7f2a9c0cddc0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a9c0d4250 0x7f2a9c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 --2- 172.21.15.19:0/3710953483 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c0b4db0 0x7f2a9c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 >> 172.21.15.19:0/3710953483 conn(0x7f2a9c0d2760 msgr2=0x7f2a9c0d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 shutdown_connections 2024-04-25T11:40:25.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.377+0000 7f2aa1a00640 1 -- 172.21.15.19:0/3710953483 wait complete. 2024-04-25T11:40:25.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.677+0000 7f3f6aa00640 1 Processor -- start 2024-04-25T11:40:25.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.677+0000 7f3f6aa00640 1 -- start start 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f6aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3f6406c5c0 0x7f3f640cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f6aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f64144370 0x7f3f640b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f6aa00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f640b50e0 con 0x7f3f640b4cd0 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f6aa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3f640d1230 con 0x7f3f64144370 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f6aa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f3f640d06b0 con 0x7f3f6406c5c0 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f68c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f64144370 0x7f3f640b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f68c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f64144370 0x7f3f640b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51976/0 (socket says 172.21.15.19:51976) 2024-04-25T11:40:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.680+0000 7f3f68c00640 1 -- 172.21.15.19:0/4138177496 learned_addr learned my addr 172.21.15.19:0/4138177496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.681+0000 7f3f69600640 1 --2- 172.21.15.19:0/4138177496 >> v2:172.21.15.169:3300/0 conn(0x7f3f6406c5c0 0x7f3f640cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.681+0000 7f3f68c00640 1 -- 172.21.15.19:0/4138177496 >> v2:172.21.15.169:3300/0 conn(0x7f3f6406c5c0 msgr2=0x7f3f640cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.681+0000 7f3f68c00640 1 --2- 172.21.15.19:0/4138177496 >> v2:172.21.15.169:3300/0 conn(0x7f3f6406c5c0 0x7f3f640cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.681+0000 7f3f68c00640 1 -- 172.21.15.19:0/4138177496 >> v1:172.21.15.169:6789/0 conn(0x7f3f640b4cd0 legacy=0x7f3f64161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:25.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.681+0000 7f3f68c00640 1 -- 172.21.15.19:0/4138177496 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f640d0990 con 0x7f3f64144370 2024-04-25T11:40:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.682+0000 7f3f68c00640 1 --2- 172.21.15.19:0/4138177496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f64144370 0x7f3f640b4790 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f3f640726b0 tx=0x7f3f5002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d09821ba2550411 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.683+0000 7f3f63e00640 1 -- 172.21.15.19:0/4138177496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50037070 con 0x7f3f64144370 2024-04-25T11:40:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.683+0000 7f3f63e00640 1 -- 172.21.15.19:0/4138177496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f50032440 con 0x7f3f64144370 2024-04-25T11:40:25.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.683+0000 7f3f63e00640 1 -- 172.21.15.19:0/4138177496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50032810 con 0x7f3f64144370 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 -- 172.21.15.19:0/4138177496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f64144370 msgr2=0x7f3f640b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/4138177496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f64144370 0x7f3f640b4790 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f3f640726b0 tx=0x7f3f5002a230 comp rx=0 tx=0).stop 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 -- 172.21.15.19:0/4138177496 shutdown_connections 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/4138177496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f64144370 0x7f3f640b4790 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/4138177496 >> v2:172.21.15.169:3300/0 conn(0x7f3f6406c5c0 0x7f3f640cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 -- 172.21.15.19:0/4138177496 >> 172.21.15.19:0/4138177496 conn(0x7f3f640d1720 msgr2=0x7f3f640d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 -- 172.21.15.19:0/4138177496 shutdown_connections 2024-04-25T11:40:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.684+0000 7f3f6aa00640 1 -- 172.21.15.19:0/4138177496 wait complete. 2024-04-25T11:40:25.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.687+0000 7f3f6aa00640 1 Processor -- start 2024-04-25T11:40:25.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.687+0000 7f3f6aa00640 1 -- start start 2024-04-25T11:40:25.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.689+0000 7f3f6aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6406c5c0 0x7f3f6413fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:25.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.689+0000 7f3f6aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f640b4cd0 0x7f3f641603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:25.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.689+0000 7f3f6aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f64144370 0x7f3f640c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:25.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.689+0000 7f3f6aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f640d02d0 con 0x7f3f6406c5c0 2024-04-25T11:40:25.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.689+0000 7f3f6aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3f640d0f50 con 0x7f3f64144370 2024-04-25T11:40:25.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.689+0000 7f3f6aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3f640d0450 con 0x7f3f640b4cd0 2024-04-25T11:40:25.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6406c5c0 0x7f3f6413fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:25.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6406c5c0 0x7f3f6413fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51982/0 (socket says 172.21.15.19:51982) 2024-04-25T11:40:25.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 -- 172.21.15.19:0/904249429 learned_addr learned my addr 172.21.15.19:0/904249429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 -- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f640b4cd0 msgr2=0x7f3f641603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f640b4cd0 0x7f3f641603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f6a000640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f64144370 0x7f3f640c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 -- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f64144370 msgr2=0x7f3f640c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f64144370 0x7f3f640c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.690+0000 7f3f69600640 1 -- 172.21.15.19:0/904249429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f64160e10 con 0x7f3f6406c5c0 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.691+0000 7f3f69600640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6406c5c0 0x7f3f6413fc50 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800d5f0 tx=0x7f3f5800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:25.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.691+0000 7f3f61600640 1 -- 172.21.15.19:0/904249429 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f580125f0 con 0x7f3f6406c5c0 2024-04-25T11:40:25.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.691+0000 7f3f61600640 1 -- 172.21.15.19:0/904249429 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f58019070 con 0x7f3f6406c5c0 2024-04-25T11:40:25.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.691+0000 7f3f61600640 1 -- 172.21.15.19:0/904249429 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f580129d0 con 0x7f3f6406c5c0 2024-04-25T11:40:25.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.691+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f641f2390 con 0x7f3f6406c5c0 2024-04-25T11:40:25.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.691+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f640cd900 con 0x7f3f6406c5c0 2024-04-25T11:40:25.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.692+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f64050740 con 0x7f3f6406c5c0 2024-04-25T11:40:25.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.694+0000 7f3f61600640 1 -- 172.21.15.19:0/904249429 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3f58012b70 con 0x7f3f6406c5c0 2024-04-25T11:40:25.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.695+0000 7f3f61600640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3f440744b0 0x7f3f440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:25.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.695+0000 7f3f61600640 1 -- 172.21.15.19:0/904249429 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8461+0+0 (secure 0 0 0) 0x7f3f580961e0 con 0x7f3f6406c5c0 2024-04-25T11:40:25.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.696+0000 7f3f68c00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3f440744b0 0x7f3f440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:25.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.697+0000 7f3f68c00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3f440744b0 0x7f3f440769c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3f50034000 tx=0x7f3f5002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:25.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.700+0000 7f3f61600640 1 -- 172.21.15.19:0/904249429 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f5805fe00 con 0x7f3f6406c5c0 2024-04-25T11:40:25.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.945+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f64147010 con 0x7f3f6406c5c0 2024-04-25T11:40:25.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.947+0000 7f3f61600640 1 -- 172.21.15.19:0/904249429 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) ==== 95+0+5187 (secure 0 0 0) 0x7f3f5805f700 con 0x7f3f6406c5c0 2024-04-25T11:40:25.934 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:25.934 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":297,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":297,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:25.377489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"a","rank":0,"incarnation":297,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/698241108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":698241108},{"type":"v1","addr":"172.21.15.19:6835","nonce":698241108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:40:25.934 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 297 2024-04-25T11:40:25.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3f440744b0 msgr2=0x7f3f440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:25.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3f440744b0 0x7f3f440769c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3f50034000 tx=0x7f3f5002c000 comp rx=0 tx=0).stop 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6406c5c0 msgr2=0x7f3f6413fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6406c5c0 0x7f3f6413fc50 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800d5f0 tx=0x7f3f5800dab0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 shutdown_connections 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3f440744b0 0x7f3f440769c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f64144370 0x7f3f640c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f640b4cd0 0x7f3f641603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 --2- 172.21.15.19:0/904249429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6406c5c0 0x7f3f6413fc50 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 >> 172.21.15.19:0/904249429 conn(0x7f3f640d1720 msgr2=0x7f3f64072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 shutdown_connections 2024-04-25T11:40:25.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:25.950+0000 7f3f6aa00640 1 -- 172.21.15.19:0/904249429 wait complete. 2024-04-25T11:40:25.966 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:40:28.404 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:40:28.417+0000 7f6c8bc00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-04-25T11:40:28.406 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:40:28.419+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:40:28.406 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:40:28.419+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:40:28.424 INFO:tasks.ceph.mds.a.smithi019.stderr: -17> 2024-04-25T11:40:28.417+0000 7f6c8bc00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-04-25T11:40:28.556 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:40:30.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.271+0000 7fed44a00640 1 Processor -- start 2024-04-25T11:40:31.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.271+0000 7fed44a00640 1 -- start start 2024-04-25T11:40:31.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed44a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed400cf2f0 0x7fed400cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed44a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed40168010 0x7fed40168420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed44a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed40168960 con 0x7fed401649d0 2024-04-25T11:40:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed44a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fed40070cf0 con 0x7fed400cf2f0 2024-04-25T11:40:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed44a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed40071180 con 0x7fed40168010 2024-04-25T11:40:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed3e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed400cf2f0 0x7fed400cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed3e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed400cf2f0 0x7fed400cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52018/0 (socket says 172.21.15.19:52018) 2024-04-25T11:40:31.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed3e800640 1 -- 172.21.15.19:0/2194289435 learned_addr learned my addr 172.21.15.19:0/2194289435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:31.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.274+0000 7fed3de00640 1 --2- 172.21.15.19:0/2194289435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed40168010 0x7fed40168420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.275+0000 7fed3e800640 1 -- 172.21.15.19:0/2194289435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed40168010 msgr2=0x7fed40168420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:31.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.275+0000 7fed3e800640 1 --2- 172.21.15.19:0/2194289435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed40168010 0x7fed40168420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.275+0000 7fed3e800640 1 -- 172.21.15.19:0/2194289435 >> v1:172.21.15.169:6789/0 conn(0x7fed401649d0 legacy=0x7fed40164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.275+0000 7fed3e800640 1 -- 172.21.15.19:0/2194289435 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed400700e0 con 0x7fed400cf2f0 2024-04-25T11:40:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.276+0000 7fed3e800640 1 --2- 172.21.15.19:0/2194289435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed400cf2f0 0x7fed400cf700 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fed30009870 tx=0x7fed3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b757b136df0b2827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.276+0000 7fed3d400640 1 -- 172.21.15.19:0/2194289435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30037070 con 0x7fed400cf2f0 2024-04-25T11:40:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.276+0000 7fed3d400640 1 -- 172.21.15.19:0/2194289435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed30032440 con 0x7fed400cf2f0 2024-04-25T11:40:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed3d400640 1 -- 172.21.15.19:0/2194289435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30032810 con 0x7fed400cf2f0 2024-04-25T11:40:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 -- 172.21.15.19:0/2194289435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed400cf2f0 msgr2=0x7fed400cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 --2- 172.21.15.19:0/2194289435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed400cf2f0 0x7fed400cf700 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fed30009870 tx=0x7fed3002a270 comp rx=0 tx=0).stop 2024-04-25T11:40:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 -- 172.21.15.19:0/2194289435 shutdown_connections 2024-04-25T11:40:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 --2- 172.21.15.19:0/2194289435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed40168010 0x7fed40168420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 --2- 172.21.15.19:0/2194289435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed400cf2f0 0x7fed400cf700 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 -- 172.21.15.19:0/2194289435 >> 172.21.15.19:0/2194289435 conn(0x7fed400d1720 msgr2=0x7fed400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 -- 172.21.15.19:0/2194289435 shutdown_connections 2024-04-25T11:40:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.277+0000 7fed44a00640 1 -- 172.21.15.19:0/2194289435 wait complete. 2024-04-25T11:40:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.280+0000 7fed44a00640 1 Processor -- start 2024-04-25T11:40:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.280+0000 7fed44a00640 1 -- start start 2024-04-25T11:40:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.281+0000 7fed44a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed400cf2f0 0x7fed4015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.281+0000 7fed44a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed401649d0 0x7fed401fe060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed44a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed40168010 0x7fed401fe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed44a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed40070980 con 0x7fed40168010 2024-04-25T11:40:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed44a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed400706a0 con 0x7fed401649d0 2024-04-25T11:40:31.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed44a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fed4006d5d0 con 0x7fed400cf2f0 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed40168010 0x7fed401fe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed40168010 0x7fed401fe5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52028/0 (socket says 172.21.15.19:52028) 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3f200640 1 -- 172.21.15.19:0/452418717 learned_addr learned my addr 172.21.15.19:0/452418717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3e800640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed400cf2f0 0x7fed4015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3de00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed401649d0 0x7fed401fe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3f200640 1 -- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed400cf2f0 msgr2=0x7fed4015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3f200640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed400cf2f0 0x7fed4015ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.282+0000 7fed3f200640 1 -- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed401649d0 msgr2=0x7fed401fe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed3f200640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed401649d0 0x7fed401fe060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed3f200640 1 -- 172.21.15.19:0/452418717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4015f7d0 con 0x7fed40168010 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed3de00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed401649d0 0x7fed401fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed3f200640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed40168010 0x7fed401fe5a0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fed3800c3f0 tx=0x7fed3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed36a00640 1 -- 172.21.15.19:0/452418717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38017070 con 0x7fed40168010 2024-04-25T11:40:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed36a00640 1 -- 172.21.15.19:0/452418717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed380052d0 con 0x7fed40168010 2024-04-25T11:40:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4015fab0 con 0x7fed40168010 2024-04-25T11:40:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed36a00640 1 -- 172.21.15.19:0/452418717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38003830 con 0x7fed40168010 2024-04-25T11:40:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.283+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed40160070 con 0x7fed40168010 2024-04-25T11:40:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.285+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed40050740 con 0x7fed40168010 2024-04-25T11:40:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.286+0000 7fed36a00640 1 -- 172.21.15.19:0/452418717 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed3801a070 con 0x7fed40168010 2024-04-25T11:40:31.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.287+0000 7fed36a00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed1c0744e0 0x7fed1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.287+0000 7fed36a00640 1 -- 172.21.15.19:0/452418717 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7fed38095060 con 0x7fed40168010 2024-04-25T11:40:31.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.288+0000 7fed3e800640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed1c0744e0 0x7fed1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.289+0000 7fed3e800640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed1c0744e0 0x7fed1c0769f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fed30034000 tx=0x7fed3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.292+0000 7fed36a00640 1 -- 172.21.15.19:0/452418717 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed40168960 con 0x7fed40168010 2024-04-25T11:40:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.538+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed4015f520 con 0x7fed40168010 2024-04-25T11:40:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.539+0000 7fed36a00640 1 -- 172.21.15.19:0/452418717 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 302 v302) ==== 95+0+5176 (secure 0 0 0) 0x7fed3805ec20 con 0x7fed40168010 2024-04-25T11:40:31.526 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:31.526 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":302,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/258414220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":258414220},{"type":"v1","addr":"172.21.15.19:6835","nonce":258414220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":302}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:28.419890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:40:31.527 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 302 2024-04-25T11:40:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed1c0744e0 msgr2=0x7fed1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed1c0744e0 0x7fed1c0769f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fed30034000 tx=0x7fed3002e000 comp rx=0 tx=0).stop 2024-04-25T11:40:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed40168010 msgr2=0x7fed401fe5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed40168010 0x7fed401fe5a0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fed3800c3f0 tx=0x7fed3800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 shutdown_connections 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fed1c0744e0 0x7fed1c0769f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed401649d0 0x7fed401fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed40168010 0x7fed401fe5a0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 --2- 172.21.15.19:0/452418717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed400cf2f0 0x7fed4015ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.542+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 >> 172.21.15.19:0/452418717 conn(0x7fed400d1720 msgr2=0x7fed400b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.543+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 shutdown_connections 2024-04-25T11:40:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.543+0000 7fed44a00640 1 -- 172.21.15.19:0/452418717 wait complete. 2024-04-25T11:40:31.555 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:40:31.555 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.855+0000 7fea0a800640 1 Processor -- start 2024-04-25T11:40:31.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.856+0000 7fea0a800640 1 -- start start 2024-04-25T11:40:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.858+0000 7fea0a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fea0406d560 0x7fea0415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.858+0000 7fea0a800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea041611e0 con 0x7fea0415a2f0 2024-04-25T11:40:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.858+0000 7fea0a800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea04160420 con 0x7fea040b6a00 2024-04-25T11:40:31.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.858+0000 7fea0a800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fea04160d50 con 0x7fea0406d560 2024-04-25T11:40:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.859+0000 7fea02a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fea0406d560 0x7fea0415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.859+0000 7fea03e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fea0415a2f0 0x7fea040b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55656/0 (socket says 172.21.15.19:55656) 2024-04-25T11:40:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.859+0000 7fea03e00640 1 -- 172.21.15.19:0/2400120005 learned_addr learned my addr 172.21.15.19:0/2400120005 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.859+0000 7fea02a00640 1 -- 172.21.15.19:0/2400120005 >> v1:172.21.15.169:6789/0 conn(0x7fea040b6a00 legacy=0x7fea0406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.859+0000 7fea02a00640 1 -- 172.21.15.19:0/2400120005 >> v1:172.21.15.19:6789/0 conn(0x7fea0415a2f0 legacy=0x7fea040b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.859+0000 7fea02a00640 1 -- 172.21.15.19:0/2400120005 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea04160700 con 0x7fea0406d560 2024-04-25T11:40:31.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.859+0000 7fea02000640 1 -- 172.21.15.19:0/2400120005 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4163195771 0 0) 0x7fea041611e0 con 0x7fea0415a2f0 2024-04-25T11:40:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.861+0000 7fea02a00640 1 --2- 172.21.15.19:0/2400120005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea0406d560 0x7fea0415d6c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00a940 tx=0x7fe9ec02b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6600260a8f840f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.861+0000 7fea02000640 1 -- 172.21.15.19:0/2400120005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00b040 con 0x7fea0406d560 2024-04-25T11:40:31.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.861+0000 7fea02000640 1 -- 172.21.15.19:0/2400120005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ec030510 con 0x7fea0406d560 2024-04-25T11:40:31.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.861+0000 7fea02000640 1 -- 172.21.15.19:0/2400120005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0308e0 con 0x7fea0406d560 2024-04-25T11:40:31.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.862+0000 7fea0a800640 1 -- 172.21.15.19:0/2400120005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea0406d560 msgr2=0x7fea0415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.862+0000 7fea0a800640 1 --2- 172.21.15.19:0/2400120005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea0406d560 0x7fea0415d6c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00a940 tx=0x7fe9ec02b5e0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.862+0000 7fea0a800640 1 -- 172.21.15.19:0/2400120005 shutdown_connections 2024-04-25T11:40:31.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.862+0000 7fea0a800640 1 --2- 172.21.15.19:0/2400120005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea0406d560 0x7fea0415d6c0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.862+0000 7fea0a800640 1 -- 172.21.15.19:0/2400120005 >> 172.21.15.19:0/2400120005 conn(0x7fea040d1ae0 msgr2=0x7fea040d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:31.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.862+0000 7fea0a800640 1 -- 172.21.15.19:0/2400120005 shutdown_connections 2024-04-25T11:40:31.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.862+0000 7fea0a800640 1 -- 172.21.15.19:0/2400120005 wait complete. 2024-04-25T11:40:31.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.864+0000 7fea0a800640 1 Processor -- start 2024-04-25T11:40:31.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.864+0000 7fea0a800640 1 -- start start 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea0a800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0406d560 0x7fea040cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea0a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea040b6a00 0x7fea04150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea0a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea0415a2f0 0x7fea04073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea0a800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea040ce580 con 0x7fea040b6a00 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea0a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fea0415a700 con 0x7fea0415a2f0 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea0a800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fea040ce700 con 0x7fea0406d560 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea02a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea040b6a00 0x7fea04150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea02a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea040b6a00 0x7fea04150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52046/0 (socket says 172.21.15.19:52046) 2024-04-25T11:40:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.866+0000 7fea02a00640 1 -- 172.21.15.19:0/324566023 learned_addr learned my addr 172.21.15.19:0/324566023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea03e00640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea0415a2f0 0x7fea04073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea02a00640 1 -- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0406d560 msgr2=0x7fea040cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea03400640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0406d560 0x7fea040cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea02a00640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0406d560 0x7fea040cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea02a00640 1 -- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea0415a2f0 msgr2=0x7fea04073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea02a00640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea0415a2f0 0x7fea04073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea02a00640 1 -- 172.21.15.19:0/324566023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0415fbf0 con 0x7fea040b6a00 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea03e00640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea0415a2f0 0x7fea04073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.867+0000 7fea02a00640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea040b6a00 0x7fea04150030 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00aa60 tx=0x7fe9ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.868+0000 7fe9fb400640 1 -- 172.21.15.19:0/324566023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec03f020 con 0x7fea040b6a00 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.868+0000 7fe9fb400640 1 -- 172.21.15.19:0/324566023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ec0328b0 con 0x7fea040b6a00 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.868+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea041521e0 con 0x7fea040b6a00 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.868+0000 7fe9fb400640 1 -- 172.21.15.19:0/324566023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec032ca0 con 0x7fea040b6a00 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.868+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea04153d40 con 0x7fea040b6a00 2024-04-25T11:40:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.869+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0415f960 con 0x7fea040b6a00 2024-04-25T11:40:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.870+0000 7fe9fb400640 1 -- 172.21.15.19:0/324566023 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe9ec042020 con 0x7fea040b6a00 2024-04-25T11:40:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.871+0000 7fe9fb400640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9e00744e0 0x7fe9e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.871+0000 7fe9fb400640 1 -- 172.21.15.19:0/324566023 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7fe9ec0b64c0 con 0x7fea040b6a00 2024-04-25T11:40:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.872+0000 7fea03400640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9e00744e0 0x7fe9e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.873+0000 7fea03400640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9e00744e0 0x7fe9e00769f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000f60 tx=0x7fe9f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:31.877+0000 7fe9fb400640 1 -- 172.21.15.19:0/324566023 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe9ec049030 con 0x7fea040b6a00 2024-04-25T11:40:32.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.124+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea040705a0 con 0x7fea040b6a00 2024-04-25T11:40:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.125+0000 7fe9fb400640 1 -- 172.21.15.19:0/324566023 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 302 v302) ==== 95+0+5176 (secure 0 0 0) 0x7fe9ec00a030 con 0x7fea040b6a00 2024-04-25T11:40:32.112 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:32.112 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":302,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/258414220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":258414220},{"type":"v1","addr":"172.21.15.19:6835","nonce":258414220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":302}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:28.419890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:40:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 302 2024-04-25T11:40:32.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.128+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9e00744e0 msgr2=0x7fe9e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.128+0000 7fea0a800640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9e00744e0 0x7fe9e00769f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000f60 tx=0x7fe9f4009040 comp rx=0 tx=0).stop 2024-04-25T11:40:32.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.128+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea040b6a00 msgr2=0x7fea04150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.128+0000 7fea0a800640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea040b6a00 0x7fea04150030 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00aa60 tx=0x7fe9ec004730 comp rx=0 tx=0).stop 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 shutdown_connections 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9e00744e0 0x7fe9e00769f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fea0415a2f0 0x7fea04073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea040b6a00 0x7fea04150030 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 --2- 172.21.15.19:0/324566023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fea0406d560 0x7fea040cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 >> 172.21.15.19:0/324566023 conn(0x7fea040d1ae0 msgr2=0x7fea040d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 shutdown_connections 2024-04-25T11:40:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.129+0000 7fea0a800640 1 -- 172.21.15.19:0/324566023 wait complete. 2024-04-25T11:40:32.141 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:40:32.141 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.417+0000 7fb930c00640 1 Processor -- start 2024-04-25T11:40:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.417+0000 7fb930c00640 1 -- start start 2024-04-25T11:40:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.419+0000 7fb930c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 0x7fb92c0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.419+0000 7fb930c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb92c1608d0 0x7fb92c160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.419+0000 7fb930c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb92c161220 con 0x7fb92c15a810 2024-04-25T11:40:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.419+0000 7fb930c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb92c06f870 con 0x7fb92c1608d0 2024-04-25T11:40:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.419+0000 7fb930c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c0b74d0 con 0x7fb92c0cf2f0 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.420+0000 7fb92aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb92c15a810 0x7fb92c0b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55662/0 (socket says 172.21.15.19:55662) 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.420+0000 7fb92aa00640 1 -- 172.21.15.19:0/1437132932 learned_addr learned my addr 172.21.15.19:0/1437132932 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.420+0000 7fb92be00640 1 --2- 172.21.15.19:0/1437132932 >> v2:172.21.15.169:3300/0 conn(0x7fb92c1608d0 0x7fb92c160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.420+0000 7fb92b400640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 0x7fb92c0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.420+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847704107 0 0) 0x7fb918001490 con 0x7fb92c15a810 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.420+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb910005730 con 0x7fb92c15a810 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 172442407 0 0) 0x7fb910005730 con 0x7fb92c15a810 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 msgr2=0x7fb92c0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 0x7fb92c0b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 >> v2:172.21.15.169:3300/0 conn(0x7fb92c1608d0 msgr2=0x7fb92c160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 --2- 172.21.15.19:0/1437132932 >> v2:172.21.15.169:3300/0 conn(0x7fb92c1608d0 0x7fb92c160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c0b77b0 con 0x7fb92c15a810 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb918001bc0 con 0x7fb92c15a810 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 >> v1:172.21.15.19:6789/0 conn(0x7fb92c15a810 legacy=0x7fb92c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 0x7fb910005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb910006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb910006f50 0x7fb910007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c06f870 con 0x7fb910006160 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c0b74d0 con 0x7fb910006f50 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb91000dc50 con 0x7fb92c0cf2f0 2024-04-25T11:40:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.421+0000 7fb92aa00640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb910006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.422+0000 7fb92be00640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb910006f50 0x7fb910007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.422+0000 7fb92b400640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 0x7fb910005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.422+0000 7fb92aa00640 1 -- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 msgr2=0x7fb910005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.422+0000 7fb92aa00640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c0cf2f0 0x7fb910005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.422+0000 7fb92aa00640 1 -- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb910006f50 msgr2=0x7fb910007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.422+0000 7fb92aa00640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb910006f50 0x7fb910007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.422+0000 7fb92aa00640 1 -- 172.21.15.19:0/1437132932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb910005730 con 0x7fb910006160 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.423+0000 7fb92aa00640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb910006a10 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fb91800aa50 tx=0x7fb910007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f7f2f24f02015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.423+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91800a670 con 0x7fb910006160 2024-04-25T11:40:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.423+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91802f070 con 0x7fb910006160 2024-04-25T11:40:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.423+0000 7fb92a000640 1 -- 172.21.15.19:0/1437132932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91802be40 con 0x7fb910006160 2024-04-25T11:40:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.424+0000 7fb930c00640 1 -- 172.21.15.19:0/1437132932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 msgr2=0x7fb910006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.424+0000 7fb930c00640 1 --2- 172.21.15.19:0/1437132932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb910006a10 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fb91800aa50 tx=0x7fb910007a40 comp rx=0 tx=0).stop 2024-04-25T11:40:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.424+0000 7fb92be00640 1 -- 172.21.15.19:0/1437132932 reap_dead start 2024-04-25T11:40:32.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.424+0000 7fb930c00640 1 -- 172.21.15.19:0/1437132932 shutdown_connections 2024-04-25T11:40:32.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.424+0000 7fb930c00640 1 -- 172.21.15.19:0/1437132932 >> 172.21.15.19:0/1437132932 conn(0x7fb92c0d1720 msgr2=0x7fb92c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:32.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.424+0000 7fb930c00640 1 -- 172.21.15.19:0/1437132932 shutdown_connections 2024-04-25T11:40:32.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.424+0000 7fb930c00640 1 -- 172.21.15.19:0/1437132932 wait complete. 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.427+0000 7fb930c00640 1 Processor -- start 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.427+0000 7fb930c00640 1 -- start start 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.428+0000 7fb930c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb92c152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.428+0000 7fb930c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c15c7e0 0x7fb92c1e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.428+0000 7fb930c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c1e8050 0x7fb92c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.428+0000 7fb930c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c15ac20 con 0x7fb910006160 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.428+0000 7fb930c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c06f500 con 0x7fb92c15c7e0 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.428+0000 7fb930c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c0b6e10 con 0x7fb92c1e8050 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb92c152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb92c152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52064/0 (socket says 172.21.15.19:52064) 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 -- 172.21.15.19:0/3837280443 learned_addr learned my addr 172.21.15.19:0/3837280443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92aa00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c15c7e0 0x7fb92c1e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92be00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c1e8050 0x7fb92c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 -- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c1e8050 msgr2=0x7fb92c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c1e8050 0x7fb92c1585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 -- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c15c7e0 msgr2=0x7fb92c1e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c15c7e0 0x7fb92c1e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 -- 172.21.15.19:0/3837280443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c1e8460 con 0x7fb910006160 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92aa00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c15c7e0 0x7fb92c1e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.429+0000 7fb92b400640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb92c152470 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fb91c011630 tx=0x7fb91c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.430+0000 7fb923400640 1 -- 172.21.15.19:0/3837280443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00f920 con 0x7fb910006160 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.430+0000 7fb923400640 1 -- 172.21.15.19:0/3837280443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c00fac0 con 0x7fb910006160 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.430+0000 7fb923400640 1 -- 172.21.15.19:0/3837280443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00fe70 con 0x7fb910006160 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.430+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c1542d0 con 0x7fb910006160 2024-04-25T11:40:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.430+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c154890 con 0x7fb910006160 2024-04-25T11:40:32.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.431+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92c050740 con 0x7fb910006160 2024-04-25T11:40:32.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.433+0000 7fb923400640 1 -- 172.21.15.19:0/3837280443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb91c015460 con 0x7fb910006160 2024-04-25T11:40:32.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.434+0000 7fb923400640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb90c0744b0 0x7fb90c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:32.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.434+0000 7fb923400640 1 -- 172.21.15.19:0/3837280443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7fb91c096380 con 0x7fb910006160 2024-04-25T11:40:32.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.434+0000 7fb92aa00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb90c0744b0 0x7fb90c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.436+0000 7fb92aa00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb90c0744b0 0x7fb90c0769c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb91800aa50 tx=0x7fb918007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:32.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.438+0000 7fb923400640 1 -- 172.21.15.19:0/3837280443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb92c1e8460 con 0x7fb910006160 2024-04-25T11:40:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.687+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb92c06c770 con 0x7fb910006160 2024-04-25T11:40:32.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.688+0000 7fb923400640 1 -- 172.21.15.19:0/3837280443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 302 v302) ==== 95+0+5176 (secure 0 0 0) 0x7fb91c019030 con 0x7fb910006160 2024-04-25T11:40:32.675 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:32.675 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":302,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/258414220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":258414220},{"type":"v1","addr":"172.21.15.19:6835","nonce":258414220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":302}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:28.419890+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:40:32.676 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 302 2024-04-25T11:40:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb90c0744b0 msgr2=0x7fb90c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb90c0744b0 0x7fb90c0769c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb91800aa50 tx=0x7fb918007000 comp rx=0 tx=0).stop 2024-04-25T11:40:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 msgr2=0x7fb92c152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb92c152470 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fb91c011630 tx=0x7fb91c011af0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 shutdown_connections 2024-04-25T11:40:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb90c0744b0 0x7fb90c0769c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb92c15c7e0 0x7fb92c1e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb92c1e8050 0x7fb92c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 --2- 172.21.15.19:0/3837280443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb910006160 0x7fb92c152470 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.691+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 >> 172.21.15.19:0/3837280443 conn(0x7fb92c0d1720 msgr2=0x7fb92c073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:32.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.692+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 shutdown_connections 2024-04-25T11:40:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:32.692+0000 7fb930c00640 1 -- 172.21.15.19:0/3837280443 wait complete. 2024-04-25T11:40:32.705 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2024-04-25T11:40:32.705 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2024-04-25T11:40:32.705 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:40:32.705 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:40:32.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:40:32.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:40:32.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:32.758 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:40:32.811 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:40:32.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:40:32.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:40:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.106+0000 7fd1ffe00640 1 Processor -- start 2024-04-25T11:40:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.107+0000 7fd1ffe00640 1 -- start start 2024-04-25T11:40:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.109+0000 7fd1ffe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd20006c5c0 0x7fd2000cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.109+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2000b4ea0 0x7fd20015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.109+0000 7fd1ffe00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2000d1230 con 0x7fd200144370 2024-04-25T11:40:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.109+0000 7fd1ffe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd2000d06b0 con 0x7fd20006c5c0 2024-04-25T11:40:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.109+0000 7fd1ffe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd2000d0990 con 0x7fd2000b4ea0 2024-04-25T11:40:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1fea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd20006c5c0 0x7fd2000cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1fea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd20006c5c0 0x7fd2000cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52076/0 (socket says 172.21.15.19:52076) 2024-04-25T11:40:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1fea00640 1 -- 172.21.15.19:0/586717606 learned_addr learned my addr 172.21.15.19:0/586717606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1fea00640 1 -- 172.21.15.19:0/586717606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2000b4ea0 msgr2=0x7fd20015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1ff400640 1 --2- 172.21.15.19:0/586717606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2000b4ea0 0x7fd20015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1fea00640 1 --2- 172.21.15.19:0/586717606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2000b4ea0 0x7fd20015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1fea00640 1 -- 172.21.15.19:0/586717606 >> v1:172.21.15.169:6789/0 conn(0x7fd200144370 legacy=0x7fd2000b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1fea00640 1 -- 172.21.15.19:0/586717606 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2000d0c70 con 0x7fd20006c5c0 2024-04-25T11:40:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.110+0000 7fd1ff400640 1 --2- 172.21.15.19:0/586717606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2000b4ea0 0x7fd20015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.111+0000 7fd1fea00640 1 --2- 172.21.15.19:0/586717606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20006c5c0 0x7fd2000cfa70 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009e50 tx=0x7fd1f002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cab85a48f26153 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.112+0000 7fd1fe000640 1 -- 172.21.15.19:0/586717606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0036070 con 0x7fd20006c5c0 2024-04-25T11:40:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.112+0000 7fd1fe000640 1 -- 172.21.15.19:0/586717606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f0031600 con 0x7fd20006c5c0 2024-04-25T11:40:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.112+0000 7fd1fe000640 1 -- 172.21.15.19:0/586717606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00319d0 con 0x7fd20006c5c0 2024-04-25T11:40:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 -- 172.21.15.19:0/586717606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20006c5c0 msgr2=0x7fd2000cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/586717606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20006c5c0 0x7fd2000cfa70 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009e50 tx=0x7fd1f002a6c0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 -- 172.21.15.19:0/586717606 shutdown_connections 2024-04-25T11:40:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/586717606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2000b4ea0 0x7fd20015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/586717606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20006c5c0 0x7fd2000cfa70 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 -- 172.21.15.19:0/586717606 >> 172.21.15.19:0/586717606 conn(0x7fd2000d1720 msgr2=0x7fd2000d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 -- 172.21.15.19:0/586717606 shutdown_connections 2024-04-25T11:40:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.113+0000 7fd1ffe00640 1 -- 172.21.15.19:0/586717606 wait complete. 2024-04-25T11:40:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.115+0000 7fd1ffe00640 1 Processor -- start 2024-04-25T11:40:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.115+0000 7fd1ffe00640 1 -- start start 2024-04-25T11:40:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd20006c5c0 0x7fd200140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd200144370 0x7fd2000ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd200162690 0x7fd2001425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ffe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2000d0450 con 0x7fd200162690 2024-04-25T11:40:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ffe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2000d02d0 con 0x7fd200144370 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ffe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd200147760 con 0x7fd20006c5c0 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd200162690 0x7fd2001425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd200162690 0x7fd2001425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52080/0 (socket says 172.21.15.19:52080) 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1ff400640 1 -- 172.21.15.19:0/2449535933 learned_addr learned my addr 172.21.15.19:0/2449535933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.117+0000 7fd1fea00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd20006c5c0 0x7fd200140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1f6000640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd200144370 0x7fd2000ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1ff400640 1 -- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd20006c5c0 msgr2=0x7fd200140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1ff400640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd20006c5c0 0x7fd200140010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1ff400640 1 -- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd200144370 msgr2=0x7fd2000ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1ff400640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd200144370 0x7fd2000ab400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1ff400640 1 -- 172.21.15.19:0/2449535933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd200140550 con 0x7fd200162690 2024-04-25T11:40:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1ff400640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd200162690 0x7fd2001425c0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c7c0 tx=0x7fd1f800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.118+0000 7fd1f7e00640 1 -- 172.21.15.19:0/2449535933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8015020 con 0x7fd200162690 2024-04-25T11:40:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.119+0000 7fd1f7e00640 1 -- 172.21.15.19:0/2449535933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f8004230 con 0x7fd200162690 2024-04-25T11:40:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.119+0000 7fd1f7e00640 1 -- 172.21.15.19:0/2449535933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8007620 con 0x7fd200162690 2024-04-25T11:40:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.119+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2001615f0 con 0x7fd200162690 2024-04-25T11:40:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.119+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd200161bb0 con 0x7fd200162690 2024-04-25T11:40:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.120+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd200050740 con 0x7fd200162690 2024-04-25T11:40:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.121+0000 7fd1f7e00640 1 -- 172.21.15.19:0/2449535933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd1f8018020 con 0x7fd200162690 2024-04-25T11:40:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.122+0000 7fd1f7e00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1dc0744e0 0x7fd1dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.122+0000 7fd1f7e00640 1 -- 172.21.15.19:0/2449535933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7fd1f8095e30 con 0x7fd200162690 2024-04-25T11:40:33.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.123+0000 7fd1fea00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1dc0744e0 0x7fd1dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.124+0000 7fd1fea00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1dc0744e0 0x7fd1dc0769f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0033000 tx=0x7fd1f0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.127+0000 7fd1f7e00640 1 -- 172.21.15.19:0/2449535933 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1f801f020 con 0x7fd200162690 2024-04-25T11:40:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.373+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd20015c350 con 0x7fd200162690 2024-04-25T11:40:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.455+0000 7fd1f7e00640 1 -- 172.21.15.19:0/2449535933 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v303) ==== 161+0+0 (secure 0 0 0) 0x7fd1f805fab0 con 0x7fd200162690 2024-04-25T11:40:33.442 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1dc0744e0 msgr2=0x7fd1dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1dc0744e0 0x7fd1dc0769f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0033000 tx=0x7fd1f0009000 comp rx=0 tx=0).stop 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd200162690 msgr2=0x7fd2001425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd200162690 0x7fd2001425c0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c7c0 tx=0x7fd1f800cc80 comp rx=0 tx=0).stop 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 shutdown_connections 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1dc0744e0 0x7fd1dc0769f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd200144370 0x7fd2000ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd200162690 0x7fd2001425c0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 --2- 172.21.15.19:0/2449535933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd20006c5c0 0x7fd200140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 >> 172.21.15.19:0/2449535933 conn(0x7fd2000d1720 msgr2=0x7fd20015cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:33.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 shutdown_connections 2024-04-25T11:40:33.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.459+0000 7fd1ffe00640 1 -- 172.21.15.19:0/2449535933 wait complete. 2024-04-25T11:40:33.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:40:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.760+0000 7f6ed1a00640 1 Processor -- start 2024-04-25T11:40:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.761+0000 7f6ed1a00640 1 -- start start 2024-04-25T11:40:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.763+0000 7f6ed1a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ecc160fe0 0x7f6ecc15a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.763+0000 7f6ed1a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ecc0b7220 con 0x7f6ecc0cf2f0 2024-04-25T11:40:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.763+0000 7f6ed1a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ecc0b74d0 con 0x7f6ecc0b6e10 2024-04-25T11:40:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.763+0000 7f6ed1a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6ecc06f870 con 0x7f6ecc160fe0 2024-04-25T11:40:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.763+0000 7f6ecbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6ecc0cf2f0 0x7f6ecc0b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55672/0 (socket says 172.21.15.19:55672) 2024-04-25T11:40:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.763+0000 7f6ecbe00640 1 -- 172.21.15.19:0/3605230696 learned_addr learned my addr 172.21.15.19:0/3605230696 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.763+0000 7f6ed1000640 1 --2- 172.21.15.19:0/3605230696 >> v2:172.21.15.19:3300/0 conn(0x7f6ecc160fe0 0x7f6ecc15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.764+0000 7f6ed1000640 1 -- 172.21.15.19:0/3605230696 >> v1:172.21.15.169:6789/0 conn(0x7f6ecc0b6e10 legacy=0x7f6ecc1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.764+0000 7f6ed1000640 1 -- 172.21.15.19:0/3605230696 >> v1:172.21.15.19:6789/0 conn(0x7f6ecc0cf2f0 legacy=0x7f6ecc0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.764+0000 7f6ed1000640 1 -- 172.21.15.19:0/3605230696 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc0b77b0 con 0x7f6ecc160fe0 2024-04-25T11:40:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.765+0000 7f6ed1000640 1 --2- 172.21.15.19:0/3605230696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc160fe0 0x7f6ecc15a810 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40097a0 tx=0x7f6ec402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4f8637dead2e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.765+0000 7f6ecb400640 1 -- 172.21.15.19:0/3605230696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4037070 con 0x7f6ecc160fe0 2024-04-25T11:40:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.765+0000 7f6ecb400640 1 -- 172.21.15.19:0/3605230696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4032440 con 0x7f6ecc160fe0 2024-04-25T11:40:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.766+0000 7f6ecb400640 1 -- 172.21.15.19:0/3605230696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4032810 con 0x7f6ecc160fe0 2024-04-25T11:40:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.766+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3605230696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc160fe0 msgr2=0x7f6ecc15a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.766+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3605230696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc160fe0 0x7f6ecc15a810 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40097a0 tx=0x7f6ec402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.766+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3605230696 shutdown_connections 2024-04-25T11:40:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.766+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3605230696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc160fe0 0x7f6ecc15a810 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.766+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3605230696 >> 172.21.15.19:0/3605230696 conn(0x7f6ecc0d1720 msgr2=0x7f6ecc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.767+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3605230696 shutdown_connections 2024-04-25T11:40:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.767+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3605230696 wait complete. 2024-04-25T11:40:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.769+0000 7f6ed1a00640 1 Processor -- start 2024-04-25T11:40:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.769+0000 7f6ed1a00640 1 -- start start 2024-04-25T11:40:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0b6e10 0x7f6ecc1f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0cf2f0 0x7f6ecc1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1e78c0 0x7f6ecc1e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc06fd00 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc06f500 con 0x7f6ecc0cf2f0 2024-04-25T11:40:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ecc06fe80 con 0x7f6ecc0b6e10 2024-04-25T11:40:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1e78c0 0x7f6ecc1e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1e78c0 0x7f6ecc1e7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52092/0 (socket says 172.21.15.19:52092) 2024-04-25T11:40:33.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 -- 172.21.15.19:0/3326062920 learned_addr learned my addr 172.21.15.19:0/3326062920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 -- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0b6e10 msgr2=0x7f6ecc1f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0b6e10 0x7f6ecc1f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 -- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0cf2f0 msgr2=0x7f6ecc1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0cf2f0 0x7f6ecc1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:33.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.771+0000 7f6ed1000640 1 -- 172.21.15.19:0/3326062920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1fa310 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.772+0000 7f6ed1000640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1e78c0 0x7f6ecc1e7e50 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40097a0 tx=0x7f6ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.772+0000 7f6ec8c00640 1 -- 172.21.15.19:0/3326062920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4037040 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.772+0000 7f6ec8c00640 1 -- 172.21.15.19:0/3326062920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4042070 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.772+0000 7f6ec8c00640 1 -- 172.21.15.19:0/3326062920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4030b30 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.772+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1e6820 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.772+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1e6de0 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.774+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc050740 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.775+0000 7f6ec8c00640 1 -- 172.21.15.19:0/3326062920 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ec4030cd0 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.776+0000 7f6ec8c00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6ea80744b0 0x7f6ea80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.776+0000 7f6ec8c00640 1 -- 172.21.15.19:0/3326062920 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f6ec40b68f0 con 0x7f6ecc1e78c0 2024-04-25T11:40:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.777+0000 7f6ecbe00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6ea80744b0 0x7f6ea80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.778+0000 7f6ecbe00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6ea80744b0 0x7f6ea80769c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0012e0 tx=0x7f6ebc001590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:33.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:33.781+0000 7f6ec8c00640 1 -- 172.21.15.19:0/3326062920 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ec404c020 con 0x7f6ecc1e78c0 2024-04-25T11:40:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.060+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f6ecc06c770 con 0x7f6ecc1e78c0 2024-04-25T11:40:34.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.473+0000 7f6ec8c00640 1 -- 172.21.15.19:0/3326062920 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v304) ==== 101+0+0 (secure 0 0 0) 0x7f6ec40804b0 con 0x7f6ecc1e78c0 2024-04-25T11:40:34.461 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6ea80744b0 msgr2=0x7f6ea80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6ea80744b0 0x7f6ea80769c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0012e0 tx=0x7f6ebc001590 comp rx=0 tx=0).stop 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1e78c0 msgr2=0x7f6ecc1e7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1e78c0 0x7f6ecc1e7e50 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40097a0 tx=0x7f6ec40046c0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 shutdown_connections 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6ea80744b0 0x7f6ea80769c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ecc0cf2f0 0x7f6ecc1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ecc1e78c0 0x7f6ecc1e7e50 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 --2- 172.21.15.19:0/3326062920 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ecc0b6e10 0x7f6ecc1f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.477+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 >> 172.21.15.19:0/3326062920 conn(0x7f6ecc0d1720 msgr2=0x7f6ecc0d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.478+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 shutdown_connections 2024-04-25T11:40:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.478+0000 7f6ed1a00640 1 -- 172.21.15.19:0/3326062920 wait complete. 2024-04-25T11:40:34.491 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:40:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.685+0000 7f0b6382ef40 1 Processor -- start 2024-04-25T11:40:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.685+0000 7f0b6382ef40 1 -- start start 2024-04-25T11:40:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b6382ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245df71e0 0x55f245ec3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b6382ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b6382ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245ed0dd0 0x55f245e42520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b6382ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f245e403f0 con 0x55f245df75f0 2024-04-25T11:40:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b6382ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f245e3fe70 con 0x55f245df71e0 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b6382ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f245e406d0 con 0x55f245ed0dd0 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39614/0 (socket says 172.21.15.19:39614) 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5c000640 1 -- 172.21.15.19:0/891949129 learned_addr learned my addr 172.21.15.19:0/891949129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5d400640 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245ed0dd0 0x55f245e42520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245df71e0 0x55f245ec3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5c000640 1 -- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245ed0dd0 msgr2=0x55f245e42520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5c000640 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245ed0dd0 0x55f245e42520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5c000640 1 -- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245df71e0 msgr2=0x55f245ec3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.687+0000 7f0b5c000640 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245df71e0 0x55f245ec3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.688+0000 7f0b5c000640 1 -- 172.21.15.19:0/891949129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f245d20b30 con 0x55f245df75f0 2024-04-25T11:40:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.688+0000 7f0b5c000640 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41fe0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f0b48055040 tx=0x7f0b48051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f261eba4abe715cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.688+0000 7f0b5ac00640 1 -- 172.21.15.19:0/891949129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48053880 con 0x55f245df75f0 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.688+0000 7f0b5ac00640 1 -- 172.21.15.19:0/891949129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b48053a40 con 0x55f245df75f0 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.688+0000 7f0b5ac00640 1 -- 172.21.15.19:0/891949129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48053e30 con 0x55f245df75f0 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 -- 172.21.15.19:0/891949129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 msgr2=0x55f245e41fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41fe0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f0b48055040 tx=0x7f0b48051000 comp rx=0 tx=0).stop 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 -- 172.21.15.19:0/891949129 shutdown_connections 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245ed0dd0 0x55f245e42520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41fe0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/891949129 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245df71e0 0x55f245ec3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 -- 172.21.15.19:0/891949129 >> 172.21.15.19:0/891949129 conn(0x55f245d19ac0 msgr2=0x55f245e291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 -- 172.21.15.19:0/891949129 shutdown_connections 2024-04-25T11:40:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.689+0000 7f0b6382ef40 1 -- 172.21.15.19:0/891949129 wait complete. 2024-04-25T11:40:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.690+0000 7f0b6382ef40 1 Processor -- start 2024-04-25T11:40:34.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.690+0000 7f0b6382ef40 1 -- start start 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b6382ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245df71e0 0x55f245ec1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b6382ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b6382ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245ed0dd0 0x55f245e41790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b6382ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f245ecb0d0 con 0x55f245df75f0 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b6382ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f245ecd240 con 0x55f245ed0dd0 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b6382ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f245f659f0 con 0x55f245df71e0 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39618/0 (socket says 172.21.15.19:39618) 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 learned_addr learned my addr 172.21.15.19:0/3624878773 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:34.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245df71e0 0x55f245ec1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245df71e0 msgr2=0x55f245ec1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f245df71e0 0x55f245ec1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245ed0dd0 msgr2=0x55f245e41790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f245ed0dd0 0x55f245e41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.692+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f245ecb500 con 0x55f245df75f0 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.693+0000 7f0b5c000640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41250 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f0b48045f50 tx=0x7f0b48002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.693+0000 7f0b59800640 1 -- 172.21.15.19:0/3624878773 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b480674f0 con 0x55f245df75f0 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.693+0000 7f0b59800640 1 -- 172.21.15.19:0/3624878773 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b48074070 con 0x55f245df75f0 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.693+0000 7f0b59800640 1 -- 172.21.15.19:0/3624878773 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b480678b0 con 0x55f245df75f0 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.693+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f245df5c50 con 0x55f245df75f0 2024-04-25T11:40:34.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.694+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f245f65730 con 0x55f245df75f0 2024-04-25T11:40:34.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.696+0000 7f0b59800640 1 -- 172.21.15.19:0/3624878773 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0b48067a50 con 0x55f245df75f0 2024-04-25T11:40:34.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.696+0000 7f0b59800640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b3c072ec0 0x7f0b3c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.697+0000 7f0b59800640 1 -- 172.21.15.19:0/3624878773 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f0b480dafa0 con 0x55f245df75f0 2024-04-25T11:40:34.683 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:40:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.697+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f245f646b0 0x55f245f64ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.697+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f65000 con 0x55f245f646b0 2024-04-25T11:40:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.697+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b3c072ec0 0x7f0b3c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.698+0000 7f0b5d400640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f245f646b0 0x55f245f64ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.699+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b3c072ec0 0x7f0b3c0753d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c005fd0 tx=0x7f0b4c007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.699+0000 7f0b5d400640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f245f646b0 0x55f245f64ac0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.703+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v125'1002 uv976 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.703+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f70770 con 0x55f245f646b0 2024-04-25T11:40:34.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.706+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v125'1003 uv1003 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.706+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f70b10 con 0x55f245f646b0 2024-04-25T11:40:34.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.710+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v125'1004 uv1004 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.696 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:40:34.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.710+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f71ec0 con 0x55f245f646b0 2024-04-25T11:40:34.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.713+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'1005 uv1005 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.713+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f72260 con 0x55f245f646b0 2024-04-25T11:40:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.717+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'1006 uv1006 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.717+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f70eb0 con 0x55f245f646b0 2024-04-25T11:40:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.720+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'1007 uv1007 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.720+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f71250 con 0x55f245f646b0 2024-04-25T11:40:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.724+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v125'1008 uv982 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.724+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.728+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v125'1009 uv1009 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.728+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.718 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:40:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.732+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v125'1010 uv1010 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.732+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.735+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'1011 uv1011 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.736+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.739+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'1012 uv1012 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.740+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.743+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'1013 uv1013 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.743+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.748+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v125'1014 uv989 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.748+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.752+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'1015 uv1015 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.752+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.756+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v125'1016 uv1016 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.743 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:40:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.756+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.760+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'1017 uv1017 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.761+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.765+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'1018 uv1018 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.765+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f715f0 con 0x55f245f646b0 2024-04-25T11:40:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.769+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'1019 uv1019 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.769+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f719d0 con 0x55f245f646b0 2024-04-25T11:40:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.774+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'1020 uv1020 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.774+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.778+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v125'1021 uv995 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.778+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.783+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v125'1022 uv1022 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.783+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.787+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v125'1023 uv1023 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.774 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:40:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.787+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.791+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'1024 uv1024 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.791+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.796+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'1025 uv1025 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.796+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.801+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'1026 uv1026 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.801+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.806+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v125'1027 uv1001 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.806+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.810+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v125'1028 uv1028 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.810+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.801 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:40:34.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.815+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v125'1029 uv1029 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.815+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.819+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'1030 uv1030 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.819+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.823+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'1031 uv1031 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.823+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f646b0 2024-04-25T11:40:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.828+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'1032 uv1032 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.828+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f245f72b10 0x55f245f74f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.828+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f718b0 con 0x55f245f72b10 2024-04-25T11:40:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.829+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f245f72b10 0x55f245f74f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.831+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f245f72b10 0x55f245f74f30 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.835+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'727 uv709 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.836+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f75470 con 0x55f245f72b10 2024-04-25T11:40:34.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.840+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'728 uv728 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.840+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f75810 con 0x55f245f72b10 2024-04-25T11:40:34.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.844+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'729 uv729 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.831 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:40:34.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.844+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f75bb0 con 0x55f245f72b10 2024-04-25T11:40:34.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.849+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v125'730 uv730 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.849+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f75f50 con 0x55f245f72b10 2024-04-25T11:40:34.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.853+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'731 uv731 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.853+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f762f0 con 0x55f245f72b10 2024-04-25T11:40:34.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.858+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v125'732 uv714 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.858+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f76690 con 0x55f245f72b10 2024-04-25T11:40:34.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.862+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v125'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.863+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.867+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v125'734 uv734 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.854 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:40:34.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.867+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.871+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'735 uv735 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.872+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.876+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'736 uv736 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.877+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.881+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v125'737 uv719 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.882+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.886+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v125'738 uv738 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.886+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.890+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v125'739 uv739 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.877 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:40:34.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.890+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7b120 con 0x55f245f72b10 2024-04-25T11:40:34.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.896+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v125'740 uv740 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.896+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7b120 con 0x55f245f72b10 2024-04-25T11:40:34.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.900+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'741 uv741 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.900+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.904+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v125'742 uv725 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.904+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.909+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v125'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.909+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.913+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v125'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.900 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:40:34.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.913+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f77a40 con 0x55f245f72b10 2024-04-25T11:40:34.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.917+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.918+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f77a40 con 0x55f245f72b10 2024-04-25T11:40:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.922+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.922+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f76a30 con 0x55f245f72b10 2024-04-25T11:40:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.926+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'747 uv747 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f72b10 2024-04-25T11:40:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.926+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f245f770b0 0x55f245f794d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.926+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f79a10 con 0x55f245f770b0 2024-04-25T11:40:34.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.927+0000 7f0b5c000640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f245f770b0 0x55f245f794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.929+0000 7f0b5c000640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f245f770b0 0x55f245f794d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.932+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.933+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f79e70 con 0x55f245f770b0 2024-04-25T11:40:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.936+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v125'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.937+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7a350 con 0x55f245f770b0 2024-04-25T11:40:34.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.941+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v125'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:34.928 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:40:34.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.941+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7bc00 con 0x55f245f770b0 2024-04-25T11:40:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.946+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.946+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7bfa0 con 0x55f245f770b0 2024-04-25T11:40:34.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.950+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:34.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.950+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7abf0 con 0x55f245f770b0 2024-04-25T11:40:34.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.955+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:34.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.955+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7afe0 con 0x55f245f646b0 2024-04-25T11:40:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.959+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'146 uv145 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.959+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f7afe0 con 0x55f245f646b0 2024-04-25T11:40:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.963+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'147 uv147 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.963+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7afe0 con 0x55f245f646b0 2024-04-25T11:40:34.953 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:40:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.967+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'148 uv148 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.967+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7afe0 con 0x55f245f646b0 2024-04-25T11:40:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.971+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'149 uv149 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.971+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f245f7c340 0x55f245f7e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.972+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7afe0 con 0x55f245f7c340 2024-04-25T11:40:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.973+0000 7f0b5d400640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f245f7c340 0x55f245f7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.974+0000 7f0b5d400640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f245f7c340 0x55f245f7e760 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.979+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:34.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.979+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f7afe0 con 0x55f245f7c340 2024-04-25T11:40:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.982+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v125'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.983+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7afe0 con 0x55f245f7c340 2024-04-25T11:40:34.973 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:40:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.986+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v125'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:34.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.986+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7fcb0 con 0x55f245f7c340 2024-04-25T11:40:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.990+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:34.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.990+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7fcb0 con 0x55f245f7c340 2024-04-25T11:40:34.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.994+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:34.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.995+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7eca0 con 0x55f245f7c340 2024-04-25T11:40:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.998+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.999+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f245f7f1b0 0x55f245f815d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.999+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f81c80 con 0x55f245f7f1b0 2024-04-25T11:40:34.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:34.999+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f245f7f1b0 0x55f245f815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:34.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.002+0000 7f0b5ca00640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f245f7f1b0 0x55f245f815d0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.006+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v125'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.007+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:34.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.011+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v125'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:34.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.011+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:35.002 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:40:35.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.015+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v125'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.015+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f83060 con 0x55f245f7f1b0 2024-04-25T11:40:35.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.019+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.019+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f83060 con 0x55f245f7f1b0 2024-04-25T11:40:35.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.024+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.024+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.028+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.028+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.032+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v125'366 uv359 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.032+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.037+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v125'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.037+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:35.027 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:40:35.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.041+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v125'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.041+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.045+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.046+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f82050 con 0x55f245f7f1b0 2024-04-25T11:40:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.050+0000 7f0b5ca00640 1 -- 172.21.15.19:0/3624878773 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b4c0137d0 con 0x55f245f7f1b0 2024-04-25T11:40:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.050+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f245f82920 0x55f245f84d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.050+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85280 con 0x55f245f82920 2024-04-25T11:40:35.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.051+0000 7f0b5c000640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f245f82920 0x55f245f84d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.053+0000 7f0b5c000640 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f245f82920 0x55f245f84d40 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.057+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v125'163 uv162 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.057+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85650 con 0x55f245f82920 2024-04-25T11:40:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.061+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v125'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.061+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f82920 2024-04-25T11:40:35.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.065+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v125'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.052 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:40:35.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.065+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f82920 2024-04-25T11:40:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.069+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.069+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f82920 2024-04-25T11:40:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.073+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'167 uv167 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.074+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f646b0 2024-04-25T11:40:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.078+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:35.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.078+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f646b0 2024-04-25T11:40:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.082+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v125'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.082+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f646b0 2024-04-25T11:40:35.073 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:40:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.087+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v125'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.087+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f86a50 con 0x55f245f646b0 2024-04-25T11:40:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.091+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.091+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f86a50 con 0x55f245f646b0 2024-04-25T11:40:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.095+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.095+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f646b0 2024-04-25T11:40:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.100+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f646b0 2024-04-25T11:40:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.100+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f770b0 2024-04-25T11:40:35.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.105+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v125'325 uv318 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.105+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f770b0 2024-04-25T11:40:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.109+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v125'326 uv326 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.109+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85a40 con 0x55f245f770b0 2024-04-25T11:40:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.113+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v125'327 uv327 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.100 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:40:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.113+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7a7e0 con 0x55f245f770b0 2024-04-25T11:40:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.118+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'328 uv328 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.118+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7a7e0 con 0x55f245f770b0 2024-04-25T11:40:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.122+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'329 uv329 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.122+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7a7e0 con 0x55f245f770b0 2024-04-25T11:40:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.126+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v125'330 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.126+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f7a7e0 con 0x55f245f770b0 2024-04-25T11:40:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.130+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v125'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.130+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7a7e0 con 0x55f245f770b0 2024-04-25T11:40:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.134+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v125'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.121 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:40:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.134+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f7a7e0 con 0x55f245f770b0 2024-04-25T11:40:35.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.139+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'333 uv333 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.139+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f88850 con 0x55f245f770b0 2024-04-25T11:40:35.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.143+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f770b0 2024-04-25T11:40:35.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.144+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.148+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v125'142 uv140 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.148+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.152+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'143 uv143 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.152+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.142 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:40:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.156+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v125'144 uv144 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.156+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.160+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'145 uv145 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.160+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.165+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v125'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.165+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.169+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v125'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.169+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.160 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:40:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.173+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v125'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.174+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f86f00 con 0x55f245f7c340 2024-04-25T11:40:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.178+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.178+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f86f00 con 0x55f245f7c340 2024-04-25T11:40:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.182+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.182+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f7c340 2024-04-25T11:40:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.186+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b54007470 con 0x55f245f7c340 2024-04-25T11:40:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.186+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.190+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v125'166 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.190+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.194+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v125'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.194+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.198+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v125'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.198+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.202+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.203+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.206+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.206+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.211+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v125'402 uv393 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.211+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.215+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v125'403 uv403 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.215+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.206 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:40:35.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.219+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v125'404 uv404 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.219+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.224+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v125'405 uv405 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.224+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.228+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'406 uv406 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.229+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.233+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v125'407 uv400 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.233+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.237+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v125'408 uv408 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.237+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.227 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:40:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.241+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v125'409 uv409 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.241+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f86f00 con 0x55f245f82920 2024-04-25T11:40:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.245+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'410 uv410 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.246+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f86f00 con 0x55f245f82920 2024-04-25T11:40:35.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.250+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'411 uv411 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.250+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f85ef0 con 0x55f245f82920 2024-04-25T11:40:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.254+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'412 uv412 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.254+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x55f245f86450 con 0x55f245f82920 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.258+0000 7f0b5c000640 1 -- 172.21.15.19:0/3624878773 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'413 uv413 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0b48051070 con 0x55f245f82920 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f245f72b10 msgr2=0x55f245f74f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55f245f72b10 0x55f245f74f30 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f245f7f1b0 msgr2=0x55f245f815d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55f245f7f1b0 0x55f245f815d0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f245f82920 msgr2=0x55f245f84d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55f245f82920 0x55f245f84d40 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f245f770b0 msgr2=0x55f245f794d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55f245f770b0 0x55f245f794d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 reap_dead start 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f245f646b0 msgr2=0x55f245f64ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55f245f646b0 0x55f245f64ac0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f245f7c340 msgr2=0x55f245f7e760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b5d400640 1 -- 172.21.15.19:0/3624878773 reap_dead start 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f245f7c340 0x55f245f7e760 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b3c072ec0 msgr2=0x7f0b3c0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b3c072ec0 0x7f0b3c0753d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c005fd0 tx=0x7f0b4c007040 comp rx=0 tx=0).stop 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 msgr2=0x55f245e41250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.259+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41250 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f0b48045f50 tx=0x7f0b48002fb0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.260+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 shutdown_connections 2024-04-25T11:40:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.260+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f245f7c340 0x55f245f7e760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.260+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f0b3c072ec0 0x7f0b3c0753d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.260+0000 7f0b6382ef40 1 --2- 172.21.15.19:0/3624878773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f245df75f0 0x55f245e41250 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.260+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 >> 172.21.15.19:0/3624878773 conn(0x55f245d19ac0 msgr2=0x55f245ec9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.260+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 shutdown_connections 2024-04-25T11:40:35.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.260+0000 7f0b6382ef40 1 -- 172.21.15.19:0/3624878773 wait complete. 2024-04-25T11:40:35.281 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:40:35.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.500+0000 7f94c23cdf40 1 Processor -- start 2024-04-25T11:40:35.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.500+0000 7f94c23cdf40 1 -- start start 2024-04-25T11:40:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94c23cdf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a00030f0 0x5579b387cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94c23cdf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579b37df150 0x5579b37df5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94c23cdf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37dfb20 0x5579b37acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94c23cdf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5579b3754d20 con 0x5579b37df150 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94c23cdf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5579b3758d60 con 0x5579b37dfb20 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94c23cdf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5579b37f7140 con 0x7f94a00030f0 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579b37df150 0x5579b37df5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579b37df150 0x5579b37df5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39628/0 (socket says 172.21.15.19:39628) 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 -- 172.21.15.19:0/2839071169 learned_addr learned my addr 172.21.15.19:0/2839071169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bb600640 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a00030f0 0x5579b387cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bc000640 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37dfb20 0x5579b37acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 -- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a00030f0 msgr2=0x5579b387cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a00030f0 0x5579b387cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 -- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37dfb20 msgr2=0x5579b37acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37dfb20 0x5579b37acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bac00640 1 -- 172.21.15.19:0/2839071169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b37f9be0 con 0x5579b37df150 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.502+0000 7f94bc000640 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37dfb20 0x5579b37acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.503+0000 7f94bac00640 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579b37df150 0x5579b37df5e0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f94a8026420 tx=0x7f94a8050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4696a29d7cc2e908 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.503+0000 7f94b9800640 1 -- 172.21.15.19:0/2839071169 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8001630 con 0x5579b37df150 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.503+0000 7f94b9800640 1 -- 172.21.15.19:0/2839071169 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a8066070 con 0x5579b37df150 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.503+0000 7f94b9800640 1 -- 172.21.15.19:0/2839071169 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8061650 con 0x5579b37df150 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2839071169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579b37df150 msgr2=0x5579b37df5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579b37df150 0x5579b37df5e0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f94a8026420 tx=0x7f94a8050000 comp rx=0 tx=0).stop 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2839071169 shutdown_connections 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37dfb20 0x5579b37acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579b37df150 0x5579b37df5e0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2839071169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94a00030f0 0x5579b387cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2839071169 >> 172.21.15.19:0/2839071169 conn(0x5579b36d0ac0 msgr2=0x5579b37e0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2839071169 shutdown_connections 2024-04-25T11:40:35.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.504+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2839071169 wait complete. 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.505+0000 7f94c23cdf40 1 Processor -- start 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.505+0000 7f94c23cdf40 1 -- start start 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94c23cdf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37df150 0x5579b3875900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94c23cdf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579b37dfb20 0x5579b37ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94c23cdf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a00030f0 0x5579b37acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94c23cdf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5579b387d420 con 0x7f94a00030f0 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94c23cdf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5579b38805b0 con 0x5579b37df150 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94c23cdf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5579b391bb00 con 0x5579b37dfb20 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94bc000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a00030f0 0x5579b37acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94bc000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a00030f0 0x5579b37acad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39638/0 (socket says 172.21.15.19:39638) 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94bc000640 1 -- 172.21.15.19:0/2260859613 learned_addr learned my addr 172.21.15.19:0/2260859613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94bac00640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579b37dfb20 0x5579b37ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.507+0000 7f94bb600640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37df150 0x5579b3875900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94bc000640 1 -- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579b37dfb20 msgr2=0x5579b37ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94bc000640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5579b37dfb20 0x5579b37ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94bc000640 1 -- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37df150 msgr2=0x5579b3875900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94bc000640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5579b37df150 0x5579b3875900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94bc000640 1 -- 172.21.15.19:0/2260859613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b3880350 con 0x7f94a00030f0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94bc000640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a00030f0 0x5579b37acad0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x5579b37f7fe0 tx=0x7f94b4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94b3e00640 1 -- 172.21.15.19:0/2260859613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4007dd0 con 0x7f94a00030f0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94b3e00640 1 -- 172.21.15.19:0/2260859613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b4016410 con 0x7f94a00030f0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94b3e00640 1 -- 172.21.15.19:0/2260859613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4016800 con 0x7f94a00030f0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.508+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5579b37f37e0 con 0x7f94a00030f0 2024-04-25T11:40:35.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.509+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b37f3da0 con 0x7f94a00030f0 2024-04-25T11:40:35.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.511+0000 7f94b3e00640 1 -- 172.21.15.19:0/2260859613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f94b4014070 con 0x7f94a00030f0 2024-04-25T11:40:35.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.512+0000 7f94b3e00640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f949c072ec0 0x7f949c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.512+0000 7f94b3e00640 1 -- 172.21.15.19:0/2260859613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f94b4093f80 con 0x7f94a00030f0 2024-04-25T11:40:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.512+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5579b387f7e0 0x5579b387fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.512+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5579b3927d10 con 0x5579b387f7e0 2024-04-25T11:40:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.513+0000 7f94bb600640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f949c072ec0 0x7f949c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.513+0000 7f94bac00640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5579b387f7e0 0x5579b387fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.513+0000 7f94bac00640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5579b387f7e0 0x5579b387fbf0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.514+0000 7f94bb600640 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f949c072ec0 0x7f949c0753d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f94ac006df0 tx=0x7f94ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.516+0000 7f94bac00640 1 -- 172.21.15.19:0/2260859613 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v125'414 uv414 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f94a80014b0 con 0x5579b387f7e0 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5579b387f7e0 msgr2=0x5579b387fbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x5579b387f7e0 0x5579b387fbf0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f949c072ec0 msgr2=0x7f949c0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f949c072ec0 0x7f949c0753d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f94ac006df0 tx=0x7f94ac008040 comp rx=0 tx=0).stop 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a00030f0 msgr2=0x5579b37acad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 --2- 172.21.15.19:0/2260859613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a00030f0 0x5579b37acad0 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x5579b37f7fe0 tx=0x7f94b4009e50 comp rx=0 tx=0).stop 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94bc000640 1 -- 172.21.15.19:0/2260859613 reap_dead start 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 shutdown_connections 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 >> 172.21.15.19:0/2260859613 conn(0x5579b36d0ac0 msgr2=0x5579b37f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 shutdown_connections 2024-04-25T11:40:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.517+0000 7f94c23cdf40 1 -- 172.21.15.19:0/2260859613 wait complete. 2024-04-25T11:40:35.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:40:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.808+0000 7f43e8e00640 1 Processor -- start 2024-04-25T11:40:35.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.808+0000 7f43e8e00640 1 -- start start 2024-04-25T11:40:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e8e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f43e4144340 0x7f43e406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f43e415ee60 0x7f43e415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e8e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43e406df20 con 0x7f43e406db10 2024-04-25T11:40:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e8e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f43e415f270 con 0x7f43e415ee60 2024-04-25T11:40:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e8e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f43e4144750 con 0x7f43e4144340 2024-04-25T11:40:35.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f43e415ee60 0x7f43e415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f43e415ee60 0x7f43e415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39646/0 (socket says 172.21.15.19:39646) 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.811+0000 7f43e3e00640 1 -- 172.21.15.19:0/2350888115 learned_addr learned my addr 172.21.15.19:0/2350888115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.812+0000 7f43e3e00640 1 -- 172.21.15.19:0/2350888115 >> v2:172.21.15.169:3300/0 conn(0x7f43e4144340 msgr2=0x7f43e406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.812+0000 7f43e3e00640 1 --2- 172.21.15.19:0/2350888115 >> v2:172.21.15.169:3300/0 conn(0x7f43e4144340 0x7f43e406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.812+0000 7f43e3e00640 1 -- 172.21.15.19:0/2350888115 >> v1:172.21.15.169:6789/0 conn(0x7f43e406db10 legacy=0x7f43e415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.812+0000 7f43e3e00640 1 -- 172.21.15.19:0/2350888115 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e4070000 con 0x7f43e415ee60 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.813+0000 7f43e3e00640 1 --2- 172.21.15.19:0/2350888115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e415ee60 0x7f43e415f4a0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f43e40b6e30 tx=0x7f43dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be3d0180358c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.813+0000 7f43e2000640 1 -- 172.21.15.19:0/2350888115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc036070 con 0x7f43e415ee60 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.813+0000 7f43e2000640 1 -- 172.21.15.19:0/2350888115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43dc031600 con 0x7f43e415ee60 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.813+0000 7f43e2000640 1 -- 172.21.15.19:0/2350888115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0319d0 con 0x7f43e415ee60 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 -- 172.21.15.19:0/2350888115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e415ee60 msgr2=0x7f43e415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2350888115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e415ee60 0x7f43e415f4a0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f43e40b6e30 tx=0x7f43dc02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 -- 172.21.15.19:0/2350888115 shutdown_connections 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2350888115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e415ee60 0x7f43e415f4a0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2350888115 >> v2:172.21.15.169:3300/0 conn(0x7f43e4144340 0x7f43e406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 -- 172.21.15.19:0/2350888115 >> 172.21.15.19:0/2350888115 conn(0x7f43e4072f40 msgr2=0x7f43e4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 -- 172.21.15.19:0/2350888115 shutdown_connections 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.814+0000 7f43e8e00640 1 -- 172.21.15.19:0/2350888115 wait complete. 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.817+0000 7f43e8e00640 1 Processor -- start 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.817+0000 7f43e8e00640 1 -- start start 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e406db10 0x7f43e4158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43e4144340 0x7f43e415d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43e415ee60 0x7f43e415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e8e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43e406f800 con 0x7f43e406db10 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e8e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f43e406f520 con 0x7f43e415ee60 2024-04-25T11:40:35.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f43e406fae0 con 0x7f43e4144340 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e406db10 0x7f43e4158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e406db10 0x7f43e4158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39648/0 (socket says 172.21.15.19:39648) 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 -- 172.21.15.19:0/2721700988 learned_addr learned my addr 172.21.15.19:0/2721700988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 -- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43e4144340 msgr2=0x7f43e415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43e4144340 0x7f43e415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 -- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43e415ee60 msgr2=0x7f43e415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43e415ee60 0x7f43e415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.819+0000 7f43e3400640 1 -- 172.21.15.19:0/2721700988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e40b6790 con 0x7f43e406db10 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.820+0000 7f43e3400640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e406db10 0x7f43e4158d30 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f43d400b440 tx=0x7f43d400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.820+0000 7f43db400640 1 -- 172.21.15.19:0/2721700988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4015020 con 0x7f43e406db10 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.820+0000 7f43db400640 1 -- 172.21.15.19:0/2721700988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d4011830 con 0x7f43e406db10 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.820+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e415b8f0 con 0x7f43e406db10 2024-04-25T11:40:35.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.820+0000 7f43db400640 1 -- 172.21.15.19:0/2721700988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4011c20 con 0x7f43e406db10 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.820+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e4149e30 con 0x7f43e406db10 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.821+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e4076c60 con 0x7f43e406db10 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.823+0000 7f43db400640 1 -- 172.21.15.19:0/2721700988 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f43d4018020 con 0x7f43e406db10 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.824+0000 7f43db400640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43c4074500 0x7f43c4076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.824+0000 7f43db400640 1 -- 172.21.15.19:0/2721700988 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f43d4095c50 con 0x7f43e406db10 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.824+0000 7f43e2a00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43c4074500 0x7f43c4076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.826+0000 7f43e2a00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43c4074500 0x7f43c4076a10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f43e40cfa70 tx=0x7f43d0006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:35.829+0000 7f43db400640 1 -- 172.21.15.19:0/2721700988 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43d401f020 con 0x7f43e406db10 2024-04-25T11:40:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.076+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f43e4140a70 con 0x7f43e406db10 2024-04-25T11:40:36.425 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:36.433 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:40:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.479+0000 7f43db400640 1 -- 172.21.15.19:0/2721700988 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v305) ==== 162+0+0 (secure 0 0 0) 0x7f43d405f890 con 0x7f43e406db10 2024-04-25T11:40:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:40:36.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.483+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43c4074500 msgr2=0x7f43c4076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:36.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.483+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43c4074500 0x7f43c4076a10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f43e40cfa70 tx=0x7f43d0006ba0 comp rx=0 tx=0).stop 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.483+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e406db10 msgr2=0x7f43e4158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.483+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e406db10 0x7f43e4158d30 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f43d400b440 tx=0x7f43d400b900 comp rx=0 tx=0).stop 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 shutdown_connections 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43c4074500 0x7f43c4076a10 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43e415ee60 0x7f43e415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43e4144340 0x7f43e415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 --2- 172.21.15.19:0/2721700988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43e406db10 0x7f43e4158d30 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 >> 172.21.15.19:0/2721700988 conn(0x7f43e4072f40 msgr2=0x7f43e40b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 shutdown_connections 2024-04-25T11:40:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:36.484+0000 7f43e8e00640 1 -- 172.21.15.19:0/2721700988 wait complete. 2024-04-25T11:40:36.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.219+0000 7f6f57200640 1 Processor -- start 2024-04-25T11:40:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.219+0000 7f6f57200640 1 -- start start 2024-04-25T11:40:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.221+0000 7f6f57200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6f58144350 0x7f6f580d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.221+0000 7f6f57200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6f58162610 0x7f6f58144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.222+0000 7f6f57200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f5806fc70 con 0x7f6f58161af0 2024-04-25T11:40:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.222+0000 7f6f57200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6f5806ff50 con 0x7f6f58162610 2024-04-25T11:40:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.222+0000 7f6f57200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6f58070230 con 0x7f6f58144350 2024-04-25T11:40:37.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.222+0000 7f6f56800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6f58162610 0x7f6f58144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:37.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.222+0000 7f6f56800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6f58162610 0x7f6f58144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39658/0 (socket says 172.21.15.19:39658) 2024-04-25T11:40:37.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.222+0000 7f6f56800640 1 -- 172.21.15.19:0/376946978 learned_addr learned my addr 172.21.15.19:0/376946978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:37.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.223+0000 7f6f56800640 1 -- 172.21.15.19:0/376946978 >> v2:172.21.15.169:3300/0 conn(0x7f6f58144350 msgr2=0x7f6f580d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:37.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.223+0000 7f6f55e00640 1 --2- 172.21.15.19:0/376946978 >> v2:172.21.15.169:3300/0 conn(0x7f6f58144350 0x7f6f580d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:37.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.223+0000 7f6f56800640 1 --2- 172.21.15.19:0/376946978 >> v2:172.21.15.169:3300/0 conn(0x7f6f58144350 0x7f6f580d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.223+0000 7f6f56800640 1 -- 172.21.15.19:0/376946978 >> v1:172.21.15.169:6789/0 conn(0x7f6f58161af0 legacy=0x7f6f58161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:37.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.223+0000 7f6f56800640 1 -- 172.21.15.19:0/376946978 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f58070510 con 0x7f6f58162610 2024-04-25T11:40:37.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.223+0000 7f6f55e00640 1 --2- 172.21.15.19:0/376946978 >> v2:172.21.15.169:3300/0 conn(0x7f6f58144350 0x7f6f580d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:37.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.224+0000 7f6f56800640 1 --2- 172.21.15.19:0/376946978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58162610 0x7f6f58144e60 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f6f580b6aa0 tx=0x7f6f5002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6459f10a08597a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:37.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.224+0000 7f6f55400640 1 -- 172.21.15.19:0/376946978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f50036070 con 0x7f6f58162610 2024-04-25T11:40:37.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.224+0000 7f6f55400640 1 -- 172.21.15.19:0/376946978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f50031600 con 0x7f6f58162610 2024-04-25T11:40:37.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.224+0000 7f6f55400640 1 -- 172.21.15.19:0/376946978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f500319d0 con 0x7f6f58162610 2024-04-25T11:40:37.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.225+0000 7f6f57200640 1 -- 172.21.15.19:0/376946978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58162610 msgr2=0x7f6f58144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:37.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.225+0000 7f6f57200640 1 --2- 172.21.15.19:0/376946978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58162610 0x7f6f58144e60 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f6f580b6aa0 tx=0x7f6f5002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.225+0000 7f6f57200640 1 -- 172.21.15.19:0/376946978 shutdown_connections 2024-04-25T11:40:37.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.225+0000 7f6f57200640 1 --2- 172.21.15.19:0/376946978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58162610 0x7f6f58144e60 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.225+0000 7f6f57200640 1 --2- 172.21.15.19:0/376946978 >> v2:172.21.15.169:3300/0 conn(0x7f6f58144350 0x7f6f580d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.226+0000 7f6f57200640 1 -- 172.21.15.19:0/376946978 >> 172.21.15.19:0/376946978 conn(0x7f6f580d1720 msgr2=0x7f6f580d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.226+0000 7f6f57200640 1 -- 172.21.15.19:0/376946978 shutdown_connections 2024-04-25T11:40:37.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.226+0000 7f6f57200640 1 -- 172.21.15.19:0/376946978 wait complete. 2024-04-25T11:40:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.229+0000 7f6f57200640 1 Processor -- start 2024-04-25T11:40:37.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.229+0000 7f6f57200640 1 -- start start 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.231+0000 7f6f57200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f58144350 0x7f6f5815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.231+0000 7f6f57200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58161af0 0x7f6f580b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.231+0000 7f6f57200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58162610 0x7f6f58160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.231+0000 7f6f57200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f58145520 con 0x7f6f58161af0 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.231+0000 7f6f57200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6f581453a0 con 0x7f6f58162610 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.231+0000 7f6f57200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6f581456a0 con 0x7f6f58144350 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58161af0 0x7f6f580b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58161af0 0x7f6f580b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39672/0 (socket says 172.21.15.19:39672) 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 -- 172.21.15.19:0/4212689054 learned_addr learned my addr 172.21.15.19:0/4212689054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f55e00640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f58144350 0x7f6f5815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:37.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f56800640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58162610 0x7f6f58160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 -- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f58144350 msgr2=0x7f6f5815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f58144350 0x7f6f5815d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 -- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58162610 msgr2=0x7f6f58160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58162610 0x7f6f58160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.232+0000 7f6f4d400640 1 -- 172.21.15.19:0/4212689054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f580b4ef0 con 0x7f6f58161af0 2024-04-25T11:40:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.233+0000 7f6f4d400640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58161af0 0x7f6f580b4510 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000b420 tx=0x7f6f4000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.233+0000 7f6f4ea00640 1 -- 172.21.15.19:0/4212689054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40011830 con 0x7f6f58161af0 2024-04-25T11:40:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.233+0000 7f6f4ea00640 1 -- 172.21.15.19:0/4212689054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f40016070 con 0x7f6f58161af0 2024-04-25T11:40:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.233+0000 7f6f4ea00640 1 -- 172.21.15.19:0/4212689054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40011c30 con 0x7f6f58161af0 2024-04-25T11:40:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.233+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f580b51d0 con 0x7f6f58161af0 2024-04-25T11:40:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.233+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f580b5790 con 0x7f6f58161af0 2024-04-25T11:40:37.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.235+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f58050740 con 0x7f6f58161af0 2024-04-25T11:40:37.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.236+0000 7f6f4ea00640 1 -- 172.21.15.19:0/4212689054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6f4001b460 con 0x7f6f58161af0 2024-04-25T11:40:37.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.237+0000 7f6f4ea00640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f340744b0 0x7f6f340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:37.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.237+0000 7f6f4ea00640 1 -- 172.21.15.19:0/4212689054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f6f40095b00 con 0x7f6f58161af0 2024-04-25T11:40:37.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.238+0000 7f6f55e00640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f340744b0 0x7f6f340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:37.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.239+0000 7f6f55e00640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f340744b0 0x7f6f340769c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6f58147d40 tx=0x7f6f48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:37.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.243+0000 7f6f4ea00640 1 -- 172.21.15.19:0/4212689054 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f4005f6c0 con 0x7f6f58161af0 2024-04-25T11:40:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.492+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f580d04d0 con 0x7f6f58161af0 2024-04-25T11:40:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.493+0000 7f6f4ea00640 1 -- 172.21.15.19:0/4212689054 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 306 v306) ==== 95+0+5187 (secure 0 0 0) 0x7f6f40095e20 con 0x7f6f58161af0 2024-04-25T11:40:37.480 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:37.480 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":306,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":306,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:36.484515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10515":{"gid":10515,"name":"a","rank":0,"incarnation":306,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/258414220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":258414220},{"type":"v1","addr":"172.21.15.19:6835","nonce":258414220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:40:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 306 2024-04-25T11:40:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f340744b0 msgr2=0x7f6f340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f340744b0 0x7f6f340769c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6f58147d40 tx=0x7f6f48008040 comp rx=0 tx=0).stop 2024-04-25T11:40:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58161af0 msgr2=0x7f6f580b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58161af0 0x7f6f580b4510 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000b420 tx=0x7f6f4000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 shutdown_connections 2024-04-25T11:40:37.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6f340744b0 0x7f6f340769c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f58162610 0x7f6f58160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f58161af0 0x7f6f580b4510 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 --2- 172.21.15.19:0/4212689054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6f58144350 0x7f6f5815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:37.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.496+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 >> 172.21.15.19:0/4212689054 conn(0x7f6f580d1720 msgr2=0x7f6f580d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:37.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.497+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 shutdown_connections 2024-04-25T11:40:37.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:37.497+0000 7f6f57200640 1 -- 172.21.15.19:0/4212689054 wait complete. 2024-04-25T11:40:37.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 306, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:40:36.484515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10515}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10515': {'gid': 10515, 'name': 'a', 'rank': 0, 'incarnation': 306, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/258414220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 258414220}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 258414220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:40:37.514 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10515:up:replay 2024-04-25T11:40:37.514 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:40:42.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:43.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.218+0000 7fb3eda00640 1 Processor -- start 2024-04-25T11:40:43.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.218+0000 7fb3eda00640 1 -- start start 2024-04-25T11:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.221+0000 7fb3eda00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e806c5c0 0x7fb3e80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.221+0000 7fb3eda00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3e80b4ea0 0x7fb3e815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.221+0000 7fb3eda00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e80d1230 con 0x7fb3e8144370 2024-04-25T11:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.221+0000 7fb3eda00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e80d06b0 con 0x7fb3e80b4ea0 2024-04-25T11:40:43.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.221+0000 7fb3eda00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e80d0990 con 0x7fb3e806c5c0 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.221+0000 7fb3ed000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3e80b4ea0 0x7fb3e815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3ed000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3e80b4ea0 0x7fb3e815e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39684/0 (socket says 172.21.15.19:39684) 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3ed000640 1 -- 172.21.15.19:0/2622683785 learned_addr learned my addr 172.21.15.19:0/2622683785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3e7e00640 1 --2- 172.21.15.19:0/2622683785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e806c5c0 0x7fb3e80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3ed000640 1 -- 172.21.15.19:0/2622683785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e806c5c0 msgr2=0x7fb3e80cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3ed000640 1 --2- 172.21.15.19:0/2622683785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e806c5c0 0x7fb3e80cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3ed000640 1 -- 172.21.15.19:0/2622683785 >> v1:172.21.15.169:6789/0 conn(0x7fb3e8144370 legacy=0x7fb3e80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3ed000640 1 -- 172.21.15.19:0/2622683785 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e80d0c70 con 0x7fb3e80b4ea0 2024-04-25T11:40:43.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.222+0000 7fb3e7e00640 1 --2- 172.21.15.19:0/2622683785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e806c5c0 0x7fb3e80cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:40:43.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.223+0000 7fb3ed000640 1 --2- 172.21.15.19:0/2622683785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e80b4ea0 0x7fb3e815e770 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009870 tx=0x7fb3e002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28a080abe07bf1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.223+0000 7fb3e7400640 1 -- 172.21.15.19:0/2622683785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0037070 con 0x7fb3e80b4ea0 2024-04-25T11:40:43.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.224+0000 7fb3e7400640 1 -- 172.21.15.19:0/2622683785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e0032440 con 0x7fb3e80b4ea0 2024-04-25T11:40:43.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.224+0000 7fb3e7400640 1 -- 172.21.15.19:0/2622683785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0032810 con 0x7fb3e80b4ea0 2024-04-25T11:40:43.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.224+0000 7fb3eda00640 1 -- 172.21.15.19:0/2622683785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e80b4ea0 msgr2=0x7fb3e815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.224+0000 7fb3eda00640 1 --2- 172.21.15.19:0/2622683785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e80b4ea0 0x7fb3e815e770 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009870 tx=0x7fb3e002a250 comp rx=0 tx=0).stop 2024-04-25T11:40:43.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.225+0000 7fb3eda00640 1 -- 172.21.15.19:0/2622683785 shutdown_connections 2024-04-25T11:40:43.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.225+0000 7fb3eda00640 1 --2- 172.21.15.19:0/2622683785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e80b4ea0 0x7fb3e815e770 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.225+0000 7fb3eda00640 1 --2- 172.21.15.19:0/2622683785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e806c5c0 0x7fb3e80cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.225+0000 7fb3eda00640 1 -- 172.21.15.19:0/2622683785 >> 172.21.15.19:0/2622683785 conn(0x7fb3e80d1720 msgr2=0x7fb3e80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.225+0000 7fb3eda00640 1 -- 172.21.15.19:0/2622683785 shutdown_connections 2024-04-25T11:40:43.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.225+0000 7fb3eda00640 1 -- 172.21.15.19:0/2622683785 wait complete. 2024-04-25T11:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.227+0000 7fb3eda00640 1 Processor -- start 2024-04-25T11:40:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.227+0000 7fb3eda00640 1 -- start start 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3eda00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e806c5c0 0x7fb3e813e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3eda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e8144370 0x7fb3e80c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3eda00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e8140270 0x7fb3e8140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3eda00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e80d0450 con 0x7fb3e8144370 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3eda00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e80d02d0 con 0x7fb3e806c5c0 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3eda00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e8147760 con 0x7fb3e8140270 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3df400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e8144370 0x7fb3e80c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3df400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e8144370 0x7fb3e80c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39686/0 (socket says 172.21.15.19:39686) 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3df400640 1 -- 172.21.15.19:0/672914833 learned_addr learned my addr 172.21.15.19:0/672914833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.229+0000 7fb3e7e00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e806c5c0 0x7fb3e813e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3ed000640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e8140270 0x7fb3e8140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3df400640 1 -- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e8140270 msgr2=0x7fb3e8140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3df400640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e8140270 0x7fb3e8140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3df400640 1 -- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e806c5c0 msgr2=0x7fb3e813e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3df400640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e806c5c0 0x7fb3e813e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3df400640 1 -- 172.21.15.19:0/672914833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e815cd40 con 0x7fb3e8144370 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3df400640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e8144370 0x7fb3e80c81e0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3c0 tx=0x7fb3d000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3e4c00640 1 -- 172.21.15.19:0/672914833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00052d0 con 0x7fb3e8144370 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3e4c00640 1 -- 172.21.15.19:0/672914833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d0005490 con 0x7fb3e8144370 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e8140bc0 con 0x7fb3e8144370 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3e4c00640 1 -- 172.21.15.19:0/672914833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0007260 con 0x7fb3e8144370 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.230+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e80c6b80 con 0x7fb3e8144370 2024-04-25T11:40:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.232+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e8050740 con 0x7fb3e8144370 2024-04-25T11:40:43.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.233+0000 7fb3e4c00640 1 -- 172.21.15.19:0/672914833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb3d0007400 con 0x7fb3e8144370 2024-04-25T11:40:43.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.234+0000 7fb3e4c00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3c40744b0 0x7fb3c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.234+0000 7fb3e4c00640 1 -- 172.21.15.19:0/672914833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7fb3d00952b0 con 0x7fb3e8144370 2024-04-25T11:40:43.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.235+0000 7fb3e7e00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3c40744b0 0x7fb3c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.236+0000 7fb3e7e00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3c40744b0 0x7fb3c40769c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8003fd0 tx=0x7fb3d8003ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.239+0000 7fb3e4c00640 1 -- 172.21.15.19:0/672914833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3e8140bc0 con 0x7fb3e8144370 2024-04-25T11:40:43.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.480+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3e80cdba0 con 0x7fb3e8144370 2024-04-25T11:40:43.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.481+0000 7fb3e4c00640 1 -- 172.21.15.19:0/672914833 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 309 v309) ==== 95+0+5196 (secure 0 0 0) 0x7fb3e815cd40 con 0x7fb3e8144370 2024-04-25T11:40:43.468 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:43.468 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":309,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":309,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:39.511249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10515":{"gid":10515,"name":"a","rank":0,"incarnation":306,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6835/258414220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":258414220},{"type":"v1","addr":"172.21.15.19:6835","nonce":258414220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10515,"qdb_cluster":[10515]},"id":6}]} 2024-04-25T11:40:43.469 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 309 2024-04-25T11:40:43.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.483+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3c40744b0 msgr2=0x7fb3c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.483+0000 7fb3eda00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3c40744b0 0x7fb3c40769c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8003fd0 tx=0x7fb3d8003ce0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.483+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e8144370 msgr2=0x7fb3e80c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.483+0000 7fb3eda00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e8144370 0x7fb3e80c81e0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3c0 tx=0x7fb3d000c7c0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 shutdown_connections 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb3c40744b0 0x7fb3c40769c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e8140270 0x7fb3e8140680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3e8144370 0x7fb3e80c81e0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 --2- 172.21.15.19:0/672914833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e806c5c0 0x7fb3e813e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 >> 172.21.15.19:0/672914833 conn(0x7fb3e80d1720 msgr2=0x7fb3e815eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 shutdown_connections 2024-04-25T11:40:43.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.484+0000 7fb3eda00640 1 -- 172.21.15.19:0/672914833 wait complete. 2024-04-25T11:40:43.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 309, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:40:39.511249+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10515}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10515': {'gid': 10515, 'name': 'a', 'rank': 0, 'incarnation': 306, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6835/258414220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 258414220}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 258414220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10515, 'qdb_cluster': [10515]} 2024-04-25T11:40:43.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:40:43.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:40:43.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.795+0000 7f3ac7e00640 1 Processor -- start 2024-04-25T11:40:43.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.795+0000 7f3ac7e00640 1 -- start start 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ac806c5c0 0x7f3ac80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac80b4ea0 0x7f3ac815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ac80d1230 con 0x7f3ac8144370 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3ac80d06b0 con 0x7f3ac806c5c0 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac80d0990 con 0x7f3ac80b4ea0 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ac806c5c0 0x7f3ac80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ac806c5c0 0x7f3ac80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45986/0 (socket says 172.21.15.19:45986) 2024-04-25T11:40:43.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.798+0000 7f3ac6a00640 1 -- 172.21.15.19:0/4057750623 learned_addr learned my addr 172.21.15.19:0/4057750623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.799+0000 7f3ac6a00640 1 -- 172.21.15.19:0/4057750623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac80b4ea0 msgr2=0x7f3ac815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.799+0000 7f3ac7400640 1 --2- 172.21.15.19:0/4057750623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac80b4ea0 0x7f3ac815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.799+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/4057750623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac80b4ea0 0x7f3ac815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.799+0000 7f3ac6a00640 1 -- 172.21.15.19:0/4057750623 >> v1:172.21.15.169:6789/0 conn(0x7f3ac8144370 legacy=0x7f3ac80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.799+0000 7f3ac6a00640 1 -- 172.21.15.19:0/4057750623 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac80d0c70 con 0x7f3ac806c5c0 2024-04-25T11:40:43.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.799+0000 7f3ac7400640 1 --2- 172.21.15.19:0/4057750623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac80b4ea0 0x7f3ac815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.800+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/4057750623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac806c5c0 0x7f3ac80cfa70 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009870 tx=0x7f3ab802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa816932a27770b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac6000640 1 -- 172.21.15.19:0/4057750623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8037070 con 0x7f3ac806c5c0 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac6000640 1 -- 172.21.15.19:0/4057750623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab8032440 con 0x7f3ac806c5c0 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac6000640 1 -- 172.21.15.19:0/4057750623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8032810 con 0x7f3ac806c5c0 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac7e00640 1 -- 172.21.15.19:0/4057750623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac806c5c0 msgr2=0x7f3ac80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/4057750623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac806c5c0 0x7f3ac80cfa70 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009870 tx=0x7f3ab802a250 comp rx=0 tx=0).stop 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac7e00640 1 -- 172.21.15.19:0/4057750623 shutdown_connections 2024-04-25T11:40:43.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/4057750623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac80b4ea0 0x7f3ac815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/4057750623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac806c5c0 0x7f3ac80cfa70 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac7e00640 1 -- 172.21.15.19:0/4057750623 >> 172.21.15.19:0/4057750623 conn(0x7f3ac80d1720 msgr2=0x7f3ac80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.801+0000 7f3ac7e00640 1 -- 172.21.15.19:0/4057750623 shutdown_connections 2024-04-25T11:40:43.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.802+0000 7f3ac7e00640 1 -- 172.21.15.19:0/4057750623 wait complete. 2024-04-25T11:40:43.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.804+0000 7f3ac7e00640 1 Processor -- start 2024-04-25T11:40:43.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.804+0000 7f3ac7e00640 1 -- start start 2024-04-25T11:40:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac806c5c0 0x7f3ac815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac8144370 0x7f3ac8143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ac815fb70 0x7f3ac8136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac80d0450 con 0x7f3ac8144370 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac80d02d0 con 0x7f3ac815fb70 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac8147760 con 0x7f3ac806c5c0 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac8144370 0x7f3ac8143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac8144370 0x7f3ac8143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46000/0 (socket says 172.21.15.19:46000) 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3abfe00640 1 -- 172.21.15.19:0/887169 learned_addr learned my addr 172.21.15.19:0/887169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac7400640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ac815fb70 0x7f3ac8136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.806+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac806c5c0 0x7f3ac815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abfe00640 1 -- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac806c5c0 msgr2=0x7f3ac815c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abfe00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac806c5c0 0x7f3ac815c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abfe00640 1 -- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ac815fb70 msgr2=0x7f3ac8136ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abfe00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ac815fb70 0x7f3ac8136ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abfe00640 1 -- 172.21.15.19:0/887169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac8142a00 con 0x7f3ac8144370 2024-04-25T11:40:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac806c5c0 0x7f3ac815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abfe00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac8144370 0x7f3ac8143aa0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c3c0 tx=0x7f3ab000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abf400640 1 -- 172.21.15.19:0/887169 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00052d0 con 0x7f3ac8144370 2024-04-25T11:40:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abf400640 1 -- 172.21.15.19:0/887169 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab0005490 con 0x7f3ac8144370 2024-04-25T11:40:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3abf400640 1 -- 172.21.15.19:0/887169 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0007260 con 0x7f3ac8144370 2024-04-25T11:40:43.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.807+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac8142ce0 con 0x7f3ac8144370 2024-04-25T11:40:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.808+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac81432a0 con 0x7f3ac8144370 2024-04-25T11:40:43.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.810+0000 7f3abf400640 1 -- 172.21.15.19:0/887169 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ab0007400 con 0x7f3ac8144370 2024-04-25T11:40:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.812+0000 7f3abf400640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3aa40744e0 0x7f3aa40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.812+0000 7f3abf400640 1 -- 172.21.15.19:0/887169 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f3ab0095280 con 0x7f3ac8144370 2024-04-25T11:40:43.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.812+0000 7f3ad4f88740 1 Processor -- start 2024-04-25T11:40:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.812+0000 7f3ad4f88740 1 -- start start 2024-04-25T11:40:43.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.812+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3aa40744e0 0x7f3aa40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.813+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9ee3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3aa40744e0 0x7f3aa40769f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8034000 tx=0x7f3ab802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9efdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9efe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5649b9e1c600 con 0x5649b9af5590 2024-04-25T11:40:43.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649b9db9100 con 0x5649b9aed3b0 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5649b9e185c0 con 0x5649b9b16270 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9efe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9efe480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46014/0 (socket says 172.21.15.19:46014) 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 -- 172.21.15.19:0/2999177461 learned_addr learned my addr 172.21.15.19:0/2999177461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3abfe00640 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9efdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9ee3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 -- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 msgr2=0x5649b9ee3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9ee3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 -- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 msgr2=0x5649b9efdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9efdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac7400640 1 -- 172.21.15.19:0/2999177461 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b9b2e2d0 con 0x5649b9af5590 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.814+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9ee3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:43.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.815+0000 7f3ac7400640 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9efe480 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007fb0 tx=0x7f3ac000e810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e350925ef034661e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.815+0000 7f3ab7400640 1 -- 172.21.15.19:0/2999177461 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000f6b0 con 0x5649b9af5590 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.815+0000 7f3ab7400640 1 -- 172.21.15.19:0/2999177461 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac000f850 con 0x5649b9af5590 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.815+0000 7f3ab7400640 1 -- 172.21.15.19:0/2999177461 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000fc40 con 0x5649b9af5590 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2999177461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 msgr2=0x5649b9efe480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9efe480 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007fb0 tx=0x7f3ac000e810 comp rx=0 tx=0).stop 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2999177461 shutdown_connections 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9efdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9efe480 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2999177461 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9ee3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2999177461 >> 172.21.15.19:0/2999177461 conn(0x5649b9b0a3c0 msgr2=0x5649b9eb8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2999177461 shutdown_connections 2024-04-25T11:40:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.816+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2999177461 wait complete. 2024-04-25T11:40:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.817+0000 7f3ad4f88740 1 -- start start 2024-04-25T11:40:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.817+0000 7f3ad4f88740 1 Processor -- start 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9aed3b0 0x5649b9edea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9af5590 0x5649b9efb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9efcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5649b9da3f70 con 0x5649b9aed3b0 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5649b9f0e5a0 con 0x5649b9af5590 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5649b9efbd00 con 0x5649b9b16270 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ad4f88740 10 client.0 _ll_register_callbacks cb 0x5649b9e12480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ac6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9aed3b0 0x5649b9edea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ac6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9aed3b0 0x5649b9edea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46030/0 (socket says 172.21.15.19:46030) 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.819+0000 7f3ac6a00640 1 -- 172.21.15.19:0/3065163888 learned_addr learned my addr 172.21.15.19:0/3065163888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3abfe00640 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9af5590 0x5649b9efb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3ac6a00640 1 -- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 msgr2=0x5649b9efcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3ac7400640 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9efcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9efcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3ac6a00640 1 -- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9af5590 msgr2=0x5649b9efb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9af5590 0x5649b9efb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3ac6a00640 1 -- 172.21.15.19:0/3065163888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b9bf86b0 con 0x5649b9aed3b0 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9aed3b0 0x5649b9edea10 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009990 tx=0x7f3ab802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b433ded4a70d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3aabe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3aabe00640 1 -- 172.21.15.19:0/3065163888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8037040 con 0x5649b9aed3b0 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.820+0000 7f3aabe00640 1 -- 172.21.15.19:0/3065163888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f3ab803d9d0 con 0x5649b9aed3b0 2024-04-25T11:40:43.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac000f850 con 0x5649b9aed3b0 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3aabe00640 1 -- 172.21.15.19:0/3065163888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=309) ==== 24+0+0 (secure 0 0 0) 0x7f3ab8056050 con 0x5649b9aed3b0 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 10 client.10599 fetch_fsmap learned FSMap version 309 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=309,monmap=2+}) -- 0x5649b9ef9fa0 con 0x5649b9aed3b0 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3aabe00640 1 -- 172.21.15.19:0/3065163888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac000f850 con 0x5649b9aed3b0 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3aabe00640 1 -- 172.21.15.19:0/3065163888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 309) ==== 3494+0+0 (secure 0 0 0) 0x7f3ab803a030 con 0x5649b9aed3b0 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 10 client.10599 fetch_fsmap finished waiting for FSMap version 309 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 10 client.10599 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 20 client.10599 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9ad4e40 0x5649b9f10e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 4 client.10599 mds_command: new command op to 10515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.821+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 --> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649b9e2a210 con 0x5649b9ad4e40 2024-04-25T11:40:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.822+0000 7f3abfe00640 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9ad4e40 0x5649b9f10e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.822+0000 7f3abfe00640 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9ad4e40 0x5649b9f10e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74cdc6b6902c944f server_cookie=4fabb576e57fb1b8 in_seq=0 out_seq=0 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.822+0000 7f3aabe00640 10 client.10599 ms_handle_connect on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3aabe00640 1 -- 172.21.15.19:0/3065163888 <== mds.0 v2:172.21.15.19:6834/258414220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5649b9e2a210 con 0x5649b9ad4e40 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3aabe00640 10 client.10599 handle_command_reply: tid=0 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 client.10599 shutdown 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9aed3b0 msgr2=0x5649b9edea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9aed3b0 0x5649b9edea10 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009990 tx=0x7f3ab802c070 comp rx=0 tx=0).stop 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 shutdown_connections 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9ad4e40 0x5649b9f10e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9af5590 0x5649b9efb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9b16270 0x5649b9efcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/3065163888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9aed3b0 0x5649b9edea10 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3aabe00640 0 client.10599 ms_handle_reset on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.823+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 >> 172.21.15.19:0/3065163888 conn(0x5649b9b0a3c0 msgr2=0x5649b9eb8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.824+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 shutdown_connections 2024-04-25T11:40:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.824+0000 7f3ad4f88740 1 -- 172.21.15.19:0/3065163888 wait complete. 2024-04-25T11:40:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.824+0000 7f3ad4f88740 20 client.10599 trim_cache size 0 max 16384 2024-04-25T11:40:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.845+0000 7f3ad4f88740 1 Processor -- start 2024-04-25T11:40:43.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.845+0000 7f3ad4f88740 1 -- start start 2024-04-25T11:40:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.846+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9aed3b0 0x5649b9ee6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.846+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9eb8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.846+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9b16270 0x5649b9eb94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.846+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8204750 con 0x5649b9af5590 2024-04-25T11:40:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.846+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac82037b0 con 0x5649b9b16270 2024-04-25T11:40:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.846+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac8202b40 con 0x5649b9aed3b0 2024-04-25T11:40:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9eb8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9eb8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46042/0 (socket says 172.21.15.19:46042) 2024-04-25T11:40:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 -- 172.21.15.19:0/2374165320 learned_addr learned my addr 172.21.15.19:0/2374165320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 -- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9aed3b0 msgr2=0x5649b9ee6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9aed3b0 0x5649b9ee6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3ac7400640 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9b16270 0x5649b9eb94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9aed3b0 0x5649b9ee6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 -- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9b16270 msgr2=0x5649b9eb94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9b16270 0x5649b9eb94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 -- 172.21.15.19:0/2374165320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b9eb68c0 con 0x5649b9af5590 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.847+0000 7f3abfe00640 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9eb8fb0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0067070 tx=0x7f3ab0000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e60f477653c673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.848+0000 7f3ab6a00640 1 -- 172.21.15.19:0/2374165320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab005e740 con 0x5649b9af5590 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.848+0000 7f3ab6a00640 1 -- 172.21.15.19:0/2374165320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab0077020 con 0x5649b9af5590 2024-04-25T11:40:43.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.848+0000 7f3ab6a00640 1 -- 172.21.15.19:0/2374165320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0062af0 con 0x5649b9af5590 2024-04-25T11:40:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.848+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2374165320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 msgr2=0x5649b9eb8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.848+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9eb8fb0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0067070 tx=0x7f3ab0000f70 comp rx=0 tx=0).stop 2024-04-25T11:40:43.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.849+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2374165320 shutdown_connections 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.849+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9b16270 0x5649b9eb94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.849+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9af5590 0x5649b9eb8fb0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.849+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2374165320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9aed3b0 0x5649b9ee6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.849+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2374165320 >> 172.21.15.19:0/2374165320 conn(0x5649b9b0a3c0 msgr2=0x5649b9ee3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.849+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2374165320 shutdown_connections 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.849+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2374165320 wait complete. 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.850+0000 7f3ad4f88740 1 -- start start 2024-04-25T11:40:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.850+0000 7f3ad4f88740 1 Processor -- start 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.851+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9f0cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.851+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9af5590 0x5649b9f7f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.851+0000 7f3ad4f88740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9b16270 0x5649b9eb8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.851+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5649b9efb900 con 0x5649b9b16270 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.851+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8203160 con 0x5649b9aed3b0 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ad4f88740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5649b9ef9100 con 0x5649b9af5590 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ad4f88740 10 client.0 _ll_register_callbacks cb 0x7f3ab8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9b16270 0x5649b9eb8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9b16270 0x5649b9eb8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46058/0 (socket says 172.21.15.19:46058) 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 -- 172.21.15.19:0/2686965094 learned_addr learned my addr 172.21.15.19:0/2686965094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9f0cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3abfe00640 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9af5590 0x5649b9f7f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 -- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9af5590 msgr2=0x5649b9f7f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9af5590 0x5649b9f7f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 -- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 msgr2=0x5649b9f0cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9f0cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.852+0000 7f3ac7400640 1 -- 172.21.15.19:0/2686965094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b9ede3e0 con 0x5649b9b16270 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3ac7400640 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9b16270 0x5649b9eb8fb0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ed10 tx=0x7f3ac0011070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0dc705d4d76c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3aabe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3aabe00640 1 -- 172.21.15.19:0/2686965094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0003da0 con 0x5649b9b16270 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3aabe00640 1 -- 172.21.15.19:0/2686965094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f3ac0004040 con 0x5649b9b16270 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab0077020 con 0x5649b9b16270 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3aabe00640 1 -- 172.21.15.19:0/2686965094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=309) ==== 24+0+0 (secure 0 0 0) 0x7f3ac0031050 con 0x5649b9b16270 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3ad4f88740 10 client.10605 fetch_fsmap learned FSMap version 309 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.853+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=309,monmap=2+}) -- 0x5649b9ed7490 con 0x5649b9b16270 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3aabe00640 1 -- 172.21.15.19:0/2686965094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab0077020 con 0x5649b9b16270 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3aabe00640 1 -- 172.21.15.19:0/2686965094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 309) ==== 3494+0+0 (secure 0 0 0) 0x7f3ac000c070 con 0x5649b9b16270 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3ad4f88740 10 client.10605 fetch_fsmap finished waiting for FSMap version 309 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3ad4f88740 10 client.10605 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3ad4f88740 20 client.10605 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9f64980 0x5649b9eb6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3ad4f88740 4 client.10605 mds_command: new command op to 10515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 --> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] -- command(tid 0: {"prefix": "status"}) -- 0x5649b9e24080 con 0x5649b9f64980 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.854+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9f64980 0x5649b9eb6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ac6a00640 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9f64980 0x5649b9eb6b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad344437d03026d server_cookie=1ca6be369b897ff1 in_seq=0 out_seq=0 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3aabe00640 10 client.10605 ms_handle_connect on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3aabe00640 1 -- 172.21.15.19:0/2686965094 <== mds.0 v2:172.21.15.19:6834/258414220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x5649b9e24080 con 0x5649b9f64980 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3aabe00640 10 client.10605 handle_command_reply: tid=0 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 client.10605 shutdown 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9b16270 msgr2=0x5649b9eb8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9b16270 0x5649b9eb8fb0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ed10 tx=0x7f3ac0011070 comp rx=0 tx=0).stop 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 shutdown_connections 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x5649b9f64980 0x5649b9eb6b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5649b9b16270 0x5649b9eb8fb0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5649b9af5590 0x5649b9f7f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3aabe00640 0 client.10605 ms_handle_reset on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 --2- 172.21.15.19:0/2686965094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5649b9aed3b0 0x5649b9f0cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.855+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 >> 172.21.15.19:0/2686965094 conn(0x5649b9b0a3c0 msgr2=0x5649b9ee6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.856+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 shutdown_connections 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.856+0000 7f3ad4f88740 1 -- 172.21.15.19:0/2686965094 wait complete. 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.856+0000 7f3ad4f88740 20 client.10605 trim_cache size 0 max 16384 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 10515, 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:40:43.845 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:40:43.846 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:40:43.846 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3652316659999997, 2024-04-25T11:40:43.846 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 309, 2024-04-25T11:40:43.846 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 125, 2024-04-25T11:40:43.846 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 125, 2024-04-25T11:40:43.846 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 15.281347308000001 2024-04-25T11:40:43.846 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.856+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3aa40744e0 msgr2=0x7f3aa40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.856+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3aa40744e0 0x7f3aa40769f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8034000 tx=0x7f3ab802e000 comp rx=0 tx=0).stop 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.856+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac8144370 msgr2=0x7f3ac8143aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.856+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac8144370 0x7f3ac8143aa0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c3c0 tx=0x7f3ab000c880 comp rx=0 tx=0).stop 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 shutdown_connections 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3aa40744e0 0x7f3aa40769f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ac815fb70 0x7f3ac8136ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ac8144370 0x7f3ac8143aa0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 --2- 172.21.15.19:0/887169 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ac806c5c0 0x7f3ac815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 >> 172.21.15.19:0/887169 conn(0x7f3ac80d1720 msgr2=0x7f3ac80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 shutdown_connections 2024-04-25T11:40:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:43.857+0000 7f3ac7e00640 1 -- 172.21.15.19:0/887169 wait complete. 2024-04-25T11:40:43.869 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:40:43.869 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2024-04-25T11:40:43.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:40:44.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.159+0000 7f40e6200640 1 Processor -- start 2024-04-25T11:40:44.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.159+0000 7f40e6200640 1 -- start start 2024-04-25T11:40:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.161+0000 7f40e6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40e00cf5e0 0x7f40e00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.161+0000 7f40e6200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e00cff30 con 0x7f40e01443c0 2024-04-25T11:40:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.161+0000 7f40e6200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e015f5f0 con 0x7f40e006c5c0 2024-04-25T11:40:44.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.161+0000 7f40e6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40e00708d0 con 0x7f40e00cf5e0 2024-04-25T11:40:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.162+0000 7f40df400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f40e01443c0 0x7f40e006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37838/0 (socket says 172.21.15.19:37838) 2024-04-25T11:40:44.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.162+0000 7f40df400640 1 -- 172.21.15.19:0/4117002414 learned_addr learned my addr 172.21.15.19:0/4117002414 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.162+0000 7f40dea00640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40e00cf5e0 0x7f40e00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.162+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981831651 0 0) 0x7f40d0001110 con 0x7f40e01443c0 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40c4003520 con 0x7f40e01443c0 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3304816783 0 0) 0x7f40c4003520 con 0x7f40e01443c0 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40e00cf5e0 msgr2=0x7f40e00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40e00cf5e0 0x7f40e00cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 >> v1:172.21.15.169:6789/0 conn(0x7f40e006c5c0 legacy=0x7f40e006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e015e830 con 0x7f40e01443c0 2024-04-25T11:40:44.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f40d0001840 con 0x7f40e01443c0 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 >> v1:172.21.15.19:6789/0 conn(0x7f40e01443c0 legacy=0x7f40e006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40039e0 0x7f40c4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c4004800 0x7f40c4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40e00cf5e0 0x7f40c40052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40e00708d0 con 0x7f40c40039e0 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40c400bb50 con 0x7f40c4004800 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40c400bd50 con 0x7f40e00cf5e0 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.163+0000 7f40df400640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40039e0 0x7f40c4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.164+0000 7f40dea00640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c4004800 0x7f40c4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.164+0000 7f40df400640 1 -- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40e00cf5e0 msgr2=0x7f40c40052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.164+0000 7f40df400640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40e00cf5e0 0x7f40c40052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.164+0000 7f40df400640 1 -- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c4004800 msgr2=0x7f40c4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.164+0000 7f40df400640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c4004800 0x7f40c4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.164+0000 7f40df400640 1 -- 172.21.15.19:0/4117002414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c4003520 con 0x7f40c40039e0 2024-04-25T11:40:44.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.164+0000 7f40dea00640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40c4004800 0x7f40c4004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.165+0000 7f40df400640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40039e0 0x7f40c4003e50 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c2f0 tx=0x7f40c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32765a4026658d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.165+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0036070 con 0x7f40c40039e0 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.165+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d002d070 con 0x7f40c40039e0 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.165+0000 7f40de000640 1 -- 172.21.15.19:0/4117002414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0007e00 con 0x7f40c40039e0 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.166+0000 7f40e6200640 1 -- 172.21.15.19:0/4117002414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40039e0 msgr2=0x7f40c4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.166+0000 7f40e6200640 1 --2- 172.21.15.19:0/4117002414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40c40039e0 0x7f40c4003e50 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c2f0 tx=0x7f40c4005830 comp rx=0 tx=0).stop 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.166+0000 7f40dfe00640 1 -- 172.21.15.19:0/4117002414 reap_dead start 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.166+0000 7f40e6200640 1 -- 172.21.15.19:0/4117002414 shutdown_connections 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.166+0000 7f40e6200640 1 -- 172.21.15.19:0/4117002414 >> 172.21.15.19:0/4117002414 conn(0x7f40e00d1ae0 msgr2=0x7f40e00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.166+0000 7f40e6200640 1 -- 172.21.15.19:0/4117002414 shutdown_connections 2024-04-25T11:40:44.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.166+0000 7f40e6200640 1 -- 172.21.15.19:0/4117002414 wait complete. 2024-04-25T11:40:44.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.169+0000 7f40e6200640 1 Processor -- start 2024-04-25T11:40:44.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.169+0000 7f40e6200640 1 -- start start 2024-04-25T11:40:44.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.170+0000 7f40e6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c40039e0 0x7f40e0159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.170+0000 7f40e6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40e0133b70 0x7f40e01f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.170+0000 7f40e6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40e01f8810 0x7f40e00b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.170+0000 7f40e6200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40e015edf0 con 0x7f40e0133b70 2024-04-25T11:40:44.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.170+0000 7f40e6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f40e015f160 con 0x7f40e01f8810 2024-04-25T11:40:44.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.170+0000 7f40e6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f40e01447d0 con 0x7f40c40039e0 2024-04-25T11:40:44.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.171+0000 7f40dea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40e0133b70 0x7f40e01f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.171+0000 7f40dfe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40e01f8810 0x7f40e00b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.171+0000 7f40dea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40e0133b70 0x7f40e01f82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46064/0 (socket says 172.21.15.19:46064) 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.171+0000 7f40dea00640 1 -- 172.21.15.19:0/2617701654 learned_addr learned my addr 172.21.15.19:0/2617701654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.171+0000 7f40df400640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c40039e0 0x7f40e0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40dea00640 1 -- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c40039e0 msgr2=0x7f40e0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40dea00640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c40039e0 0x7f40e0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40dea00640 1 -- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40e01f8810 msgr2=0x7f40e00b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40dea00640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40e01f8810 0x7f40e00b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40dea00640 1 -- 172.21.15.19:0/2617701654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e0159b60 con 0x7f40e0133b70 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40dea00640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40e0133b70 0x7f40e01f82d0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0108b0 tx=0x7f40cc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:44.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40d7400640 1 -- 172.21.15.19:0/2617701654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00f040 con 0x7f40e0133b70 2024-04-25T11:40:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40d7400640 1 -- 172.21.15.19:0/2617701654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40cc00eb60 con 0x7f40e0133b70 2024-04-25T11:40:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40d7400640 1 -- 172.21.15.19:0/2617701654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc01c400 con 0x7f40e0133b70 2024-04-25T11:40:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e0157810 con 0x7f40e0133b70 2024-04-25T11:40:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.172+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e0157dd0 con 0x7f40e0133b70 2024-04-25T11:40:44.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.174+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e01575a0 con 0x7f40e0133b70 2024-04-25T11:40:44.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.175+0000 7f40d7400640 1 -- 172.21.15.19:0/2617701654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f40cc01c5a0 con 0x7f40e0133b70 2024-04-25T11:40:44.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.176+0000 7f40d7400640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f40c00744e0 0x7f40c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:44.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.176+0000 7f40d7400640 1 -- 172.21.15.19:0/2617701654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f40cc0962a0 con 0x7f40e0133b70 2024-04-25T11:40:44.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.177+0000 7f40df400640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f40c00744e0 0x7f40c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:44.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.178+0000 7f40df400640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f40c00744e0 0x7f40c00769f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f40d0033000 tx=0x7f40d0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:44.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.182+0000 7f40d7400640 1 -- 172.21.15.19:0/2617701654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f40cc018030 con 0x7f40e0133b70 2024-04-25T11:40:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.426+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f40e006d0e0 con 0x7f40e0133b70 2024-04-25T11:40:44.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.426+0000 7f40d7400640 1 -- 172.21.15.19:0/2617701654 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v309) ==== 53+0+73 (secure 0 0 0) 0x7f40e006d0e0 con 0x7f40e0133b70 2024-04-25T11:40:44.413 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f40c00744e0 msgr2=0x7f40c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f40c00744e0 0x7f40c00769f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f40d0033000 tx=0x7f40d0009030 comp rx=0 tx=0).stop 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40e0133b70 msgr2=0x7f40e01f82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40e0133b70 0x7f40e01f82d0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0108b0 tx=0x7f40cc010d70 comp rx=0 tx=0).stop 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 shutdown_connections 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f40c00744e0 0x7f40c00769f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40e01f8810 0x7f40e00b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40e0133b70 0x7f40e01f82d0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 --2- 172.21.15.19:0/2617701654 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40c40039e0 0x7f40e0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:44.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 >> 172.21.15.19:0/2617701654 conn(0x7f40e00d1ae0 msgr2=0x7f40e006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:44.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 shutdown_connections 2024-04-25T11:40:44.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44.429+0000 7f40e6200640 1 -- 172.21.15.19:0/2617701654 wait complete. 2024-04-25T11:40:44.432 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:40:44.446+0000 7fb959800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:40:44.445 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:40:44.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:44.446 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:40:44.499 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:40:44.499 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:40:44.499 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:40:44.499 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:44.499 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:40:44.499 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:44.499 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41829sec preferred_lft 41829sec 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:44.500 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:40:44.501 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:40:44.502 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:40:44.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:44.503 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:40:44.581 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:40:44.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:44.586 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:40:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:40:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:40:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:40:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:40:44.639 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:40:44.640 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:40:44.640 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:40:44.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:40:44.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:40:44.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:40:44.640 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:40:44.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:40:44.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:40:44.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:44.864 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:40:44.913 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:40:44.913 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:40:44.913 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:40:44.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:44.913 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:40:44.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:40:44.990 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:45.035 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:40:45.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:40:45.035 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:45.133 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:40:45.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:45.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:40:45.348 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:40:45.348 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:40:45.391 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:40:45.391 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:40:45.392 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:40:45.392 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:40:45.392 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:40:45.393 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:40:45.394 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:40:45.394 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:40:45.394 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:40:45.394 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:40:45.399 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:40:45.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:45.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:40:45.471 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:40:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:40:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:40:45.598 INFO:teuthology.orchestra.run.smithi019.stdout:client6791 2024-04-25T11:40:45.598 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:40:45.598 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2024-04-25T11:40:45.654 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-04-25T11:40:45.655 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-04-25T11:40:45.655 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:40:45.655 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T11:40:45.655 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-04-25T11:40:45.655 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T11:40:45.655 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-04-25T11:40:45.655 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:40:45.662 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:40:45.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:45 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-04-25T11:40:46.842 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:40:46.854+0000 7fb164400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T11:40:51.763 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-04-25T11:40:51.763 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-04-25T11:40:51.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.075+0000 7f15a9c00640 1 Processor -- start 2024-04-25T11:40:52.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.075+0000 7f15a9c00640 1 -- start start 2024-04-25T11:40:52.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.077+0000 7f15a9c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15a40cf9e0 0x7f15a4161090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.078+0000 7f15a9c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a41615d0 con 0x7f15a40b0800 2024-04-25T11:40:52.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.078+0000 7f15a9c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a40cffc0 con 0x7f15a40709a0 2024-04-25T11:40:52.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.078+0000 7f15a9c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f15a40d02a0 con 0x7f15a40cf9e0 2024-04-25T11:40:52.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.078+0000 7f159b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f15a40b0800 0x7f15a4070290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37846/0 (socket says 172.21.15.19:37846) 2024-04-25T11:40:52.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.078+0000 7f159b400640 1 -- 172.21.15.19:0/1817072739 learned_addr learned my addr 172.21.15.19:0/1817072739 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.078+0000 7f15a3e00640 1 --2- 172.21.15.19:0/1817072739 >> v2:172.21.15.169:3300/0 conn(0x7f15a40cf9e0 0x7f15a4161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2660721151 0 0) 0x7f158c001110 con 0x7f15a40b0800 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1588003520 con 0x7f15a40b0800 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 492029618 0 0) 0x7f1588003520 con 0x7f15a40b0800 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 >> v2:172.21.15.169:3300/0 conn(0x7f15a40cf9e0 msgr2=0x7f15a4161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 --2- 172.21.15.19:0/1817072739 >> v2:172.21.15.169:3300/0 conn(0x7f15a40cf9e0 0x7f15a4161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 >> v1:172.21.15.169:6789/0 conn(0x7f15a40709a0 legacy=0x7f15a41569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a40d0580 con 0x7f15a40b0800 2024-04-25T11:40:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f158c001840 con 0x7f15a40b0800 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 >> v1:172.21.15.19:6789/0 conn(0x7f15a40b0800 legacy=0x7f15a4070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15880039b0 0x7f1588003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588004800 0x7f1588004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588005250 0x7f158800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15a40d02a0 con 0x7f1588004800 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f158800c1e0 con 0x7f15880039b0 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.079+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f158800c3e0 con 0x7f1588005250 2024-04-25T11:40:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f159b400640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588004800 0x7f1588004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f15a3e00640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15880039b0 0x7f1588003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f15a9200640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588005250 0x7f158800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f159b400640 1 -- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588005250 msgr2=0x7f158800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f159b400640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588005250 0x7f158800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f159b400640 1 -- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15880039b0 msgr2=0x7f1588003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f159b400640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15880039b0 0x7f1588003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f159b400640 1 -- 172.21.15.19:0/1817072739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1588003520 con 0x7f1588004800 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.080+0000 7f15a9200640 1 -- 172.21.15.19:0/1817072739 reap_dead start 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.081+0000 7f159b400640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588004800 0x7f1588004c10 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c4c0 tx=0x7f1588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9c5f10b5da9175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.081+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c00c0e0 con 0x7f1588004800 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.081+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158c034070 con 0x7f1588004800 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.081+0000 7f15a3400640 1 -- 172.21.15.19:0/1817072739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c005d70 con 0x7f1588004800 2024-04-25T11:40:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.082+0000 7f15a9c00640 1 -- 172.21.15.19:0/1817072739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588004800 msgr2=0x7f1588004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.082+0000 7f15a9c00640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588004800 0x7f1588004c10 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c4c0 tx=0x7f1588005830 comp rx=0 tx=0).stop 2024-04-25T11:40:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.082+0000 7f15a9c00640 1 -- 172.21.15.19:0/1817072739 shutdown_connections 2024-04-25T11:40:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.082+0000 7f15a9c00640 1 --2- 172.21.15.19:0/1817072739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1588004800 0x7f1588004c10 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.082+0000 7f15a9c00640 1 -- 172.21.15.19:0/1817072739 >> 172.21.15.19:0/1817072739 conn(0x7f15a40d1720 msgr2=0x7f15a40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.082+0000 7f15a9c00640 1 -- 172.21.15.19:0/1817072739 shutdown_connections 2024-04-25T11:40:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.082+0000 7f15a9c00640 1 -- 172.21.15.19:0/1817072739 wait complete. 2024-04-25T11:40:52.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.085+0000 7f15a9c00640 1 Processor -- start 2024-04-25T11:40:52.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.085+0000 7f15a9c00640 1 -- start start 2024-04-25T11:40:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a9c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588004800 0x7f15a4156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a4147930 0x7f15a4147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a9c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15a4148280 0x7f15a40b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a9c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15a40b0d90 con 0x7f15a4147930 2024-04-25T11:40:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a9c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15a40b0c10 con 0x7f15a4148280 2024-04-25T11:40:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a9c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15a4161090 con 0x7f1588004800 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f159b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a4147930 0x7f15a4147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f159b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a4147930 0x7f15a4147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46084/0 (socket says 172.21.15.19:46084) 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f159b400640 1 -- 172.21.15.19:0/2345584669 learned_addr learned my addr 172.21.15.19:0/2345584669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a3e00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588004800 0x7f15a4156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f15a9200640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15a4148280 0x7f15a40b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f159b400640 1 -- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588004800 msgr2=0x7f15a4156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.087+0000 7f159b400640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588004800 0x7f15a4156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f159b400640 1 -- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15a4148280 msgr2=0x7f15a40b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f159b400640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15a4148280 0x7f15a40b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f159b400640 1 -- 172.21.15.19:0/2345584669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a40b7aa0 con 0x7f15a4147930 2024-04-25T11:40:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f159b400640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a4147930 0x7f15a4147d40 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f158c031040 tx=0x7f15a40d0b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f15a0c00640 1 -- 172.21.15.19:0/2345584669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c007270 con 0x7f15a4147930 2024-04-25T11:40:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f15a0c00640 1 -- 172.21.15.19:0/2345584669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158c03f030 con 0x7f15a4147930 2024-04-25T11:40:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f15a0c00640 1 -- 172.21.15.19:0/2345584669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c007610 con 0x7f15a4147930 2024-04-25T11:40:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a4157450 con 0x7f15a4147930 2024-04-25T11:40:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.088+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a40b4e90 con 0x7f15a4147930 2024-04-25T11:40:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.090+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a4050740 con 0x7f15a4147930 2024-04-25T11:40:52.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.091+0000 7f15a0c00640 1 -- 172.21.15.19:0/2345584669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f158c02f460 con 0x7f15a4147930 2024-04-25T11:40:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.092+0000 7f15a0c00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15800744b0 0x7f15800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.092+0000 7f15a0c00640 1 -- 172.21.15.19:0/2345584669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f158c0b6750 con 0x7f15a4147930 2024-04-25T11:40:52.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.093+0000 7f15a3e00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15800744b0 0x7f15800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.094+0000 7f15a3e00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15800744b0 0x7f15800769c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f15a406bc20 tx=0x7f1594008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.097+0000 7f15a0c00640 1 -- 172.21.15.19:0/2345584669 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f158c049030 con 0x7f15a4147930 2024-04-25T11:40:52.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.344+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15a41571e0 con 0x7f15a4147930 2024-04-25T11:40:52.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.345+0000 7f15a0c00640 1 -- 172.21.15.19:0/2345584669 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5196 (secure 0 0 0) 0x7f15a41615d0 con 0x7f15a4147930 2024-04-25T11:40:52.333 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:52.333 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":310,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:46.512768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10515":{"gid":10515,"name":"a","rank":0,"incarnation":306,"state":"up:active","state_seq":7,"addr":"172.21.15.19:6835/258414220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":258414220},{"type":"v1","addr":"172.21.15.19:6835","nonce":258414220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10515,"qdb_cluster":[10515]},"id":6}]} 2024-04-25T11:40:52.333 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 310 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15800744b0 msgr2=0x7f15800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15800744b0 0x7f15800769c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f15a406bc20 tx=0x7f1594008040 comp rx=0 tx=0).stop 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a4147930 msgr2=0x7f15a4147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a4147930 0x7f15a4147d40 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f158c031040 tx=0x7f15a40d0b40 comp rx=0 tx=0).stop 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 shutdown_connections 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15800744b0 0x7f15800769c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15a4148280 0x7f15a40b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a4147930 0x7f15a4147d40 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 --2- 172.21.15.19:0/2345584669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1588004800 0x7f15a4156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 >> 172.21.15.19:0/2345584669 conn(0x7f15a40d1720 msgr2=0x7f15a4072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 shutdown_connections 2024-04-25T11:40:52.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.348+0000 7f15a9c00640 1 -- 172.21.15.19:0/2345584669 wait complete. 2024-04-25T11:40:52.362 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T11:40:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.665+0000 7f2da5000640 1 Processor -- start 2024-04-25T11:40:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.665+0000 7f2da5000640 1 -- start start 2024-04-25T11:40:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.667+0000 7f2da5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 0x7f2da015e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.667+0000 7f2da5000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2da006c5c0 0x7f2da006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2da5000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2da01609b0 con 0x7f2da0163fa0 2024-04-25T11:40:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2da5000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2da006cf10 con 0x7f2da006c5c0 2024-04-25T11:40:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2da5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2da0164cd0 con 0x7f2da0161710 2024-04-25T11:40:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2d9e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2da0163fa0 0x7f2da01643b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37858/0 (socket says 172.21.15.19:37858) 2024-04-25T11:40:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2d9e800640 1 -- 172.21.15.19:0/233799354 learned_addr learned my addr 172.21.15.19:0/233799354 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2d9de00640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 0x7f2da015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2d9d400640 1 --2- 172.21.15.19:0/233799354 >> v2:172.21.15.169:3300/0 conn(0x7f2da006c5c0 0x7f2da006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.668+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008173692 0 0) 0x7f2d98001490 con 0x7f2da0163fa0 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d84003520 con 0x7f2da0163fa0 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3062836968 0 0) 0x7f2d84003520 con 0x7f2da0163fa0 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 msgr2=0x7f2da015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 0x7f2da015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 >> v2:172.21.15.169:3300/0 conn(0x7f2da006c5c0 msgr2=0x7f2da006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 --2- 172.21.15.19:0/233799354 >> v2:172.21.15.169:3300/0 conn(0x7f2da006c5c0 0x7f2da006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da0160c30 con 0x7f2da0163fa0 2024-04-25T11:40:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2d98001bc0 con 0x7f2da0163fa0 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 >> v1:172.21.15.19:6789/0 conn(0x7f2da0163fa0 legacy=0x7f2da01643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 0x7f2d840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d84003f80 0x7f2d84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d84004d40 0x7f2d84005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2da006cf10 con 0x7f2d84004d40 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0164cd0 con 0x7f2d84003f80 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8400bb20 con 0x7f2da0161710 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.669+0000 7f2d9e800640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d84004d40 0x7f2d84005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.670+0000 7f2d9de00640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 0x7f2d840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.670+0000 7f2d9d400640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d84003f80 0x7f2d84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.670+0000 7f2d9e800640 1 -- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 msgr2=0x7f2d840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.670+0000 7f2d9e800640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da0161710 0x7f2d840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.670+0000 7f2d9e800640 1 -- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d84003f80 msgr2=0x7f2d84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.670+0000 7f2d9e800640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d84003f80 0x7f2d84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.670+0000 7f2d9e800640 1 -- 172.21.15.19:0/233799354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d84003520 con 0x7f2d84004d40 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.671+0000 7f2d9e800640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d84004d40 0x7f2d84005150 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f2d980013d0 tx=0x7f2d84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766a420b9a84bb88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.671+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800a5d0 con 0x7f2d84004d40 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.671+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9802f070 con 0x7f2d84004d40 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.671+0000 7f2d9ca00640 1 -- 172.21.15.19:0/233799354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9802bd80 con 0x7f2d84004d40 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.672+0000 7f2da5000640 1 -- 172.21.15.19:0/233799354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d84004d40 msgr2=0x7f2d84005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.672+0000 7f2da5000640 1 --2- 172.21.15.19:0/233799354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d84004d40 0x7f2d84005150 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f2d980013d0 tx=0x7f2d84005830 comp rx=0 tx=0).stop 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.672+0000 7f2d9e800640 1 -- 172.21.15.19:0/233799354 reap_dead start 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.672+0000 7f2da5000640 1 -- 172.21.15.19:0/233799354 shutdown_connections 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.672+0000 7f2da5000640 1 -- 172.21.15.19:0/233799354 >> 172.21.15.19:0/233799354 conn(0x7f2da00d5d90 msgr2=0x7f2da00d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.672+0000 7f2da5000640 1 -- 172.21.15.19:0/233799354 shutdown_connections 2024-04-25T11:40:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.672+0000 7f2da5000640 1 -- 172.21.15.19:0/233799354 wait complete. 2024-04-25T11:40:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.675+0000 7f2da5000640 1 Processor -- start 2024-04-25T11:40:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.675+0000 7f2da5000640 1 -- start start 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.676+0000 7f2da5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00c8950 0x7f2da01e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.676+0000 7f2da5000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00bb0f0 0x7f2da00bb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2da5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da01f9a70 0x7f2da01f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2da5000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2da015e080 con 0x7f2da01f9a70 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2da5000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01611f0 con 0x7f2da00bb0f0 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2da5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2da015e200 con 0x7f2da00c8950 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da01f9a70 0x7f2da01f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da01f9a70 0x7f2da01f9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46106/0 (socket says 172.21.15.19:46106) 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 -- 172.21.15.19:0/3499724307 learned_addr learned my addr 172.21.15.19:0/3499724307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9de00640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00c8950 0x7f2da01e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9d400640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00bb0f0 0x7f2da00bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 -- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00c8950 msgr2=0x7f2da01e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00c8950 0x7f2da01e9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 -- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00bb0f0 msgr2=0x7f2da00bb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00bb0f0 0x7f2da00bb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9e800640 1 -- 172.21.15.19:0/3499724307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da00bba40 con 0x7f2da01f9a70 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.677+0000 7f2d9d400640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00bb0f0 0x7f2da00bb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.678+0000 7f2d9e800640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da01f9a70 0x7f2da01f9e80 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f2d980013d0 tx=0x7f2d98001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.678+0000 7f2d96000640 1 -- 172.21.15.19:0/3499724307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98045070 con 0x7f2da01f9a70 2024-04-25T11:40:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.678+0000 7f2d96000640 1 -- 172.21.15.19:0/3499724307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d98034670 con 0x7f2da01f9a70 2024-04-25T11:40:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.678+0000 7f2d96000640 1 -- 172.21.15.19:0/3499724307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800a5d0 con 0x7f2da01f9a70 2024-04-25T11:40:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.678+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da00cd1b0 con 0x7f2da01f9a70 2024-04-25T11:40:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.678+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da00cd770 con 0x7f2da01f9a70 2024-04-25T11:40:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.682+0000 7f2d96000640 1 -- 172.21.15.19:0/3499724307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d98048070 con 0x7f2da01f9a70 2024-04-25T11:40:52.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.683+0000 7f2d96000640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d800744b0 0x7f2d800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.683+0000 7f2d96000640 1 -- 172.21.15.19:0/3499724307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f2d980b6070 con 0x7f2da01f9a70 2024-04-25T11:40:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.683+0000 7f2dac172740 1 Processor -- start 2024-04-25T11:40:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.684+0000 7f2dac172740 1 -- start start 2024-04-25T11:40:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.684+0000 7f2d9de00640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d800744b0 0x7f2d800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7090ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2dac172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70524670 0x55ba708e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2dac172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba7084bbc0 con 0x55ba70524670 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba707e7ed0 con 0x55ba7051c490 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba70847b80 con 0x55ba70545350 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2d9de00640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d800744b0 0x7f2d800769c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2da01f99d0 tx=0x7f2d90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70524670 0x55ba708e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70524670 0x55ba708e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46114/0 (socket says 172.21.15.19:46114) 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.685+0000 7f2d9e800640 1 -- 172.21.15.19:0/4118441979 learned_addr learned my addr 172.21.15.19:0/4118441979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9de00640 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9d400640 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7090ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9e800640 1 -- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70545350 msgr2=0x55ba70912580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9e800640 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9e800640 1 -- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 msgr2=0x55ba7090ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9e800640 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7090ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9e800640 1 -- 172.21.15.19:0/4118441979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba706297c0 con 0x55ba70524670 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9d400640 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7090ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d9e800640 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70524670 0x55ba708e81e0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f2d9807f5e0 tx=0x7f2d9807b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f166e2276d7df03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d8aa00640 1 -- 172.21.15.19:0/4118441979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98001490 con 0x55ba70524670 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d8aa00640 1 -- 172.21.15.19:0/4118441979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9802f020 con 0x55ba70524670 2024-04-25T11:40:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.686+0000 7f2d8aa00640 1 -- 172.21.15.19:0/4118441979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98084d90 con 0x55ba70524670 2024-04-25T11:40:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 -- 172.21.15.19:0/4118441979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70524670 msgr2=0x55ba708e81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70524670 0x55ba708e81e0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f2d9807f5e0 tx=0x7f2d9807b640 comp rx=0 tx=0).stop 2024-04-25T11:40:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 -- 172.21.15.19:0/4118441979 shutdown_connections 2024-04-25T11:40:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7090ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70524670 0x55ba708e81e0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 --2- 172.21.15.19:0/4118441979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 -- 172.21.15.19:0/4118441979 >> 172.21.15.19:0/4118441979 conn(0x55ba705394a0 msgr2=0x55ba708e7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 -- 172.21.15.19:0/4118441979 shutdown_connections 2024-04-25T11:40:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.687+0000 7f2dac172740 1 -- 172.21.15.19:0/4118441979 wait complete. 2024-04-25T11:40:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.688+0000 7f2dac172740 1 -- start start 2024-04-25T11:40:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.689+0000 7f2dac172740 1 Processor -- start 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.690+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7092b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.690+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba7093f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.690+0000 7f2dac172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.690+0000 7f2dac172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba707e7ed0 con 0x55ba70545350 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.690+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba707d29f0 con 0x55ba7051c490 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.690+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba7084bbc0 con 0x55ba70524670 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.690+0000 7f2dac172740 10 client.0 _ll_register_callbacks cb 0x55ba707d0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46124/0 (socket says 172.21.15.19:46124) 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 -- 172.21.15.19:0/2040567609 learned_addr learned my addr 172.21.15.19:0/2040567609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9de00640 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7092b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 -- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 msgr2=0x55ba7093f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba7093f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 -- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 msgr2=0x55ba7092b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7092b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 -- 172.21.15.19:0/2040567609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba70689f50 con 0x55ba70545350 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9de00640 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7092b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d9e800640 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093fd90 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f2d9807f5e0 tx=0x7f2d98002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f06b9f0091ce0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.691+0000 7f2d7f400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2d7f400640 1 -- 172.21.15.19:0/2040567609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9809d060 con 0x55ba70545350 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2d7f400640 1 -- 172.21.15.19:0/2040567609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f2d9807e8b0 con 0x55ba70545350 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9802f020 con 0x55ba70545350 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2d7f400640 1 -- 172.21.15.19:0/2040567609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=310) ==== 24+0+0 (secure 0 0 0) 0x7f2d98001490 con 0x55ba70545350 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 10 client.10629 fetch_fsmap learned FSMap version 310 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=310,monmap=2+}) -- 0x55ba7051d0b0 con 0x55ba70545350 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2d7f400640 1 -- 172.21.15.19:0/2040567609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9802f020 con 0x55ba70545350 2024-04-25T11:40:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2d7f400640 1 -- 172.21.15.19:0/2040567609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 310) ==== 3494+0+0 (secure 0 0 0) 0x7f2d9807afc0 con 0x55ba70545350 2024-04-25T11:40:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 10 client.10629 fetch_fsmap finished waiting for FSMap version 310 2024-04-25T11:40:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 10 client.10629 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 20 client.10629 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba70504130 0x55ba709142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 4 client.10629 mds_command: new command op to 10515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:40:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.692+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 --> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba708597d0 con 0x55ba70504130 2024-04-25T11:40:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.693+0000 7f2d9de00640 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba70504130 0x55ba709142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.693+0000 7f2d9de00640 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba70504130 0x55ba709142d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cdc02706cd8a546 server_cookie=fee06abb8380a446 in_seq=0 out_seq=0 2024-04-25T11:40:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.693+0000 7f2d7f400640 10 client.10629 ms_handle_connect on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.694+0000 7f2d7f400640 1 -- 172.21.15.19:0/2040567609 <== mds.0 v2:172.21.15.19:6834/258414220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba708597d0 con 0x55ba70504130 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.694+0000 7f2d7f400640 10 client.10629 handle_command_reply: tid=0 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.694+0000 7f2dac172740 1 client.10629 shutdown 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.694+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 msgr2=0x55ba7093fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.694+0000 7f2dac172740 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093fd90 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f2d9807f5e0 tx=0x7f2d98002140 comp rx=0 tx=0).stop 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 shutdown_connections 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba70504130 0x55ba709142d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093fd90 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba7093f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 --2- 172.21.15.19:0/2040567609 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba7051c490 0x55ba7092b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 >> 172.21.15.19:0/2040567609 conn(0x55ba705394a0 msgr2=0x55ba70912580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2d7f400640 0 client.10629 ms_handle_reset on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 shutdown_connections 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 1 -- 172.21.15.19:0/2040567609 wait complete. 2024-04-25T11:40:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.695+0000 7f2dac172740 20 client.10629 trim_cache size 0 max 16384 2024-04-25T11:40:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.710+0000 7f2dac172740 1 Processor -- start 2024-04-25T11:40:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.711+0000 7f2dac172740 1 -- start start 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba7051c490 0x55ba70912580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70524670 0x55ba70912ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2dac172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2dac172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2da02044f0 con 0x55ba70545350 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0203850 con 0x55ba70524670 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2da0203ea0 con 0x55ba7051c490 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46126/0 (socket says 172.21.15.19:46126) 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.712+0000 7f2d9e800640 1 -- 172.21.15.19:0/2893809473 learned_addr learned my addr 172.21.15.19:0/2893809473 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9d400640 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70524670 0x55ba70912ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9de00640 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba7051c490 0x55ba70912580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9e800640 1 -- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba7051c490 msgr2=0x55ba70912580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9e800640 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba7051c490 0x55ba70912580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9e800640 1 -- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70524670 msgr2=0x55ba70912ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9e800640 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70524670 0x55ba70912ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9e800640 1 -- 172.21.15.19:0/2893809473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba708e3bd0 con 0x55ba70545350 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9de00640 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba7051c490 0x55ba70912580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d9e800640 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093f6f0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f2d9807c8c0 tx=0x7f2d98096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5dc4fa42e73af52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d8a000640 1 -- 172.21.15.19:0/2893809473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9809d040 con 0x55ba70545350 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d8a000640 1 -- 172.21.15.19:0/2893809473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9802f040 con 0x55ba70545350 2024-04-25T11:40:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.713+0000 7f2d8a000640 1 -- 172.21.15.19:0/2893809473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98088420 con 0x55ba70545350 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 -- 172.21.15.19:0/2893809473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 msgr2=0x55ba7093f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093f6f0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f2d9807c8c0 tx=0x7f2d98096040 comp rx=0 tx=0).stop 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 -- 172.21.15.19:0/2893809473 shutdown_connections 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70524670 0x55ba70912ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba70545350 0x55ba7093f6f0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 --2- 172.21.15.19:0/2893809473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba7051c490 0x55ba70912580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 -- 172.21.15.19:0/2893809473 >> 172.21.15.19:0/2893809473 conn(0x55ba705394a0 msgr2=0x55ba70924030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 -- 172.21.15.19:0/2893809473 shutdown_connections 2024-04-25T11:40:52.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.714+0000 7f2dac172740 1 -- 172.21.15.19:0/2893809473 wait complete. 2024-04-25T11:40:52.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.715+0000 7f2dac172740 1 -- start start 2024-04-25T11:40:52.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.716+0000 7f2dac172740 1 Processor -- start 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2dac172740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba7051c490 0x55ba7092f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba70929e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2dac172740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2dac172740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba707e7ed0 con 0x55ba7051c490 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55ba707d29f0 con 0x55ba70545350 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2dac172740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55ba7093c310 con 0x55ba70524670 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2dac172740 10 client.0 _ll_register_callbacks cb 0x7f2d98097070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.717+0000 7f2d9de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba7051c490 0x55ba7092f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba7051c490 0x55ba7092f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46128/0 (socket says 172.21.15.19:46128) 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 -- 172.21.15.19:0/1944644718 learned_addr learned my addr 172.21.15.19:0/1944644718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 -- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 msgr2=0x55ba70929e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9d400640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba70929e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9e800640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba70929e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 -- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70545350 msgr2=0x55ba70912580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 -- 172.21.15.19:0/1944644718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba7092a730 con 0x55ba7051c490 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9d400640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba70929e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9e800640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d9de00640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba7051c490 0x55ba7092f0d0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x55ba7090d310 tx=0x7f2d90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570cda9c3d96a6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.718+0000 7f2d7f400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:40:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2d7f400640 1 -- 172.21.15.19:0/1944644718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9002c250 con 0x55ba7051c490 2024-04-25T11:40:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2d7f400640 1 -- 172.21.15.19:0/1944644718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f2d9002c3f0 con 0x55ba7051c490 2024-04-25T11:40:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9802f040 con 0x55ba7051c490 2024-04-25T11:40:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2d7f400640 1 -- 172.21.15.19:0/1944644718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=310) ==== 24+0+0 (secure 0 0 0) 0x7f2d9003b050 con 0x55ba7051c490 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2dac172740 10 client.10635 fetch_fsmap learned FSMap version 310 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=310,monmap=2+}) -- 0x55ba709504f0 con 0x55ba7051c490 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2d7f400640 1 -- 172.21.15.19:0/1944644718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9802f040 con 0x55ba7051c490 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2d7f400640 1 -- 172.21.15.19:0/1944644718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 310) ==== 3494+0+0 (secure 0 0 0) 0x7f2d9002cce0 con 0x55ba7051c490 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.719+0000 7f2dac172740 10 client.10635 fetch_fsmap finished waiting for FSMap version 310 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.720+0000 7f2dac172740 10 client.10635 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.720+0000 7f2dac172740 20 client.10635 populate_metadata read hostname 'smithi019' 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.720+0000 7f2dac172740 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba7094ed00 0x55ba7094f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.720+0000 7f2dac172740 4 client.10635 mds_command: new command op to 10515 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.720+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 --> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55ba70853640 con 0x55ba7094ed00 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.720+0000 7f2d9d400640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba7094ed00 0x55ba7094f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2d9d400640 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba7094ed00 0x55ba7094f110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1abef3ed74dfbf server_cookie=7053b708cede3f11 in_seq=0 out_seq=0 2024-04-25T11:40:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2d7f400640 10 client.10635 ms_handle_connect on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2d7f400640 1 -- 172.21.15.19:0/1944644718 <== mds.0 v2:172.21.15.19:6834/258414220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+194 (crc 0 0 0) 0x55ba70853640 con 0x55ba7094ed00 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2d7f400640 10 client.10635 handle_command_reply: tid=0 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 client.10635 shutdown 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba7051c490 msgr2=0x55ba7092f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba7051c490 0x55ba7092f0d0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x55ba7090d310 tx=0x7f2d90004730 comp rx=0 tx=0).stop 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 shutdown_connections 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:6834/258414220,v1:172.21.15.19:6835/258414220] conn(0x55ba7094ed00 0x55ba7094f110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ba70545350 0x55ba70912580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ba70524670 0x55ba70929e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 --2- 172.21.15.19:0/1944644718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba7051c490 0x55ba7092f0d0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2d7f400640 0 client.10635 ms_handle_reset on v2:172.21.15.19:6834/258414220 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 >> 172.21.15.19:0/1944644718 conn(0x55ba705394a0 msgr2=0x55ba708e1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 shutdown_connections 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.721+0000 7f2dac172740 1 -- 172.21.15.19:0/1944644718 wait complete. 2024-04-25T11:40:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2dac172740 20 client.10635 trim_cache size 0 max 16384 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "dentry", 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3654978527, 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1, 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: "frag": "*", 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "subdir", 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_id": "head", 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/subdir" 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:40:52.712 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:40:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d800744b0 msgr2=0x7f2d800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d800744b0 0x7f2d800769c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2da01f99d0 tx=0x7f2d90008040 comp rx=0 tx=0).stop 2024-04-25T11:40:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da01f9a70 msgr2=0x7f2da01f9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da01f9a70 0x7f2da01f9e80 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f2d980013d0 tx=0x7f2d98001360 comp rx=0 tx=0).stop 2024-04-25T11:40:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 shutdown_connections 2024-04-25T11:40:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d800744b0 0x7f2d800769c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00bb0f0 0x7f2da00bb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2da01f9a70 0x7f2da01f9e80 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 --2- 172.21.15.19:0/3499724307 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00c8950 0x7f2da01e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 >> 172.21.15.19:0/3499724307 conn(0x7f2da00d5d90 msgr2=0x7f2da00d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.722+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 shutdown_connections 2024-04-25T11:40:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:52.723+0000 7f2da5000640 1 -- 172.21.15.19:0/3499724307 wait complete. 2024-04-25T11:40:52.737 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2024-04-25T11:40:52.738 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:40:52.738 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:40:52.789 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:40:52.790 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:40:52.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:40:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.183+0000 7f4b8d200640 1 Processor -- start 2024-04-25T11:40:53.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.183+0000 7f4b8d200640 1 -- start start 2024-04-25T11:40:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.185+0000 7f4b8d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b880cf2f0 0x7f4b880b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.185+0000 7f4b8d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b881608d0 0x7f4b88160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.185+0000 7f4b8d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b88161220 con 0x7f4b8815a810 2024-04-25T11:40:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.185+0000 7f4b8d200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8806f870 con 0x7f4b881608d0 2024-04-25T11:40:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.185+0000 7f4b8d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4b880b74d0 con 0x7f4b880cf2f0 2024-04-25T11:40:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.186+0000 7f4b87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b881608d0 0x7f4b88160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.186+0000 7f4b87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b881608d0 0x7f4b88160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46140/0 (socket says 172.21.15.19:46140) 2024-04-25T11:40:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.186+0000 7f4b87e00640 1 -- 172.21.15.19:0/2464110910 learned_addr learned my addr 172.21.15.19:0/2464110910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.186+0000 7f4b87400640 1 --2- 172.21.15.19:0/2464110910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b880cf2f0 0x7f4b880b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.187+0000 7f4b87e00640 1 -- 172.21.15.19:0/2464110910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b880cf2f0 msgr2=0x7f4b880b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.187+0000 7f4b87e00640 1 --2- 172.21.15.19:0/2464110910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b880cf2f0 0x7f4b880b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.187+0000 7f4b87e00640 1 -- 172.21.15.19:0/2464110910 >> v1:172.21.15.169:6789/0 conn(0x7f4b8815a810 legacy=0x7f4b880b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.187+0000 7f4b87e00640 1 -- 172.21.15.19:0/2464110910 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b880b77b0 con 0x7f4b881608d0 2024-04-25T11:40:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.188+0000 7f4b87e00640 1 --2- 172.21.15.19:0/2464110910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b881608d0 0x7f4b88160ce0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f4b8806f9f0 tx=0x7f4b8002c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e6cdd8be104e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.188+0000 7f4b86000640 1 -- 172.21.15.19:0/2464110910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80009600 con 0x7f4b881608d0 2024-04-25T11:40:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.188+0000 7f4b86000640 1 -- 172.21.15.19:0/2464110910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b80037070 con 0x7f4b881608d0 2024-04-25T11:40:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.188+0000 7f4b86000640 1 -- 172.21.15.19:0/2464110910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800099c0 con 0x7f4b881608d0 2024-04-25T11:40:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.189+0000 7f4b8d200640 1 -- 172.21.15.19:0/2464110910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b881608d0 msgr2=0x7f4b88160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.189+0000 7f4b8d200640 1 --2- 172.21.15.19:0/2464110910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b881608d0 0x7f4b88160ce0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f4b8806f9f0 tx=0x7f4b8002c910 comp rx=0 tx=0).stop 2024-04-25T11:40:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.189+0000 7f4b8d200640 1 -- 172.21.15.19:0/2464110910 shutdown_connections 2024-04-25T11:40:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.189+0000 7f4b8d200640 1 --2- 172.21.15.19:0/2464110910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b881608d0 0x7f4b88160ce0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.189+0000 7f4b8d200640 1 --2- 172.21.15.19:0/2464110910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b880cf2f0 0x7f4b880b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.189+0000 7f4b8d200640 1 -- 172.21.15.19:0/2464110910 >> 172.21.15.19:0/2464110910 conn(0x7f4b880d1720 msgr2=0x7f4b880d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.189+0000 7f4b8d200640 1 -- 172.21.15.19:0/2464110910 shutdown_connections 2024-04-25T11:40:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.190+0000 7f4b8d200640 1 -- 172.21.15.19:0/2464110910 wait complete. 2024-04-25T11:40:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.193+0000 7f4b8d200640 1 Processor -- start 2024-04-25T11:40:53.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.193+0000 7f4b8d200640 1 -- start start 2024-04-25T11:40:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b8d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b880cf2f0 0x7f4b880ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b8d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8815a810 0x7f4b880b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b8d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b881608d0 0x7f4b88149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b8d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8815ac20 con 0x7f4b880cf2f0 2024-04-25T11:40:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b8d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8806f500 con 0x7f4b8815a810 2024-04-25T11:40:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b8d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8806fd00 con 0x7f4b881608d0 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b87400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b880cf2f0 0x7f4b880ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b87400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b880cf2f0 0x7f4b880ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46150/0 (socket says 172.21.15.19:46150) 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b87400640 1 -- 172.21.15.19:0/3435081848 learned_addr learned my addr 172.21.15.19:0/3435081848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.195+0000 7f4b86a00640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8815a810 0x7f4b880b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b87e00640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b881608d0 0x7f4b88149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b87400640 1 -- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b881608d0 msgr2=0x7f4b88149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b87400640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b881608d0 0x7f4b88149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b87400640 1 -- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8815a810 msgr2=0x7f4b880b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b87400640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8815a810 0x7f4b880b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b87400640 1 -- 172.21.15.19:0/3435081848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b881fa6e0 con 0x7f4b880cf2f0 2024-04-25T11:40:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b87400640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b880cf2f0 0x7f4b880ae640 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800e870 tx=0x7f4b7800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b7f400640 1 -- 172.21.15.19:0/3435081848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78017070 con 0x7f4b880cf2f0 2024-04-25T11:40:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b7f400640 1 -- 172.21.15.19:0/3435081848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7800c940 con 0x7f4b880cf2f0 2024-04-25T11:40:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.196+0000 7f4b7f400640 1 -- 172.21.15.19:0/3435081848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800cd30 con 0x7f4b880cf2f0 2024-04-25T11:40:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.197+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b88151670 con 0x7f4b880cf2f0 2024-04-25T11:40:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.197+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881f9c00 con 0x7f4b880cf2f0 2024-04-25T11:40:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.198+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88050740 con 0x7f4b880cf2f0 2024-04-25T11:40:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.199+0000 7f4b7f400640 1 -- 172.21.15.19:0/3435081848 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b7801a070 con 0x7f4b880cf2f0 2024-04-25T11:40:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.200+0000 7f4b7f400640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b640744b0 0x7f4b640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.200+0000 7f4b7f400640 1 -- 172.21.15.19:0/3435081848 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f4b78095bb0 con 0x7f4b880cf2f0 2024-04-25T11:40:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.201+0000 7f4b86a00640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b640744b0 0x7f4b640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.202+0000 7f4b86a00640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b640744b0 0x7f4b640769c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4b70007fd0 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:53.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.205+0000 7f4b7f400640 1 -- 172.21.15.19:0/3435081848 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b7805f8f0 con 0x7f4b880cf2f0 2024-04-25T11:40:53.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.411+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b8806c770 con 0x7f4b880cf2f0 2024-04-25T11:40:53.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.415+0000 7f4b7f400640 1 -- 172.21.15.19:0/3435081848 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+20940 (secure 0 0 0) 0x7f4b7805f1f0 con 0x7f4b880cf2f0 2024-04-25T11:40:53.403 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:40:53.403 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":125,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:40:28.419570+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6834/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:40:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b640744b0 msgr2=0x7f4b640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b640744b0 0x7f4b640769c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4b70007fd0 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2024-04-25T11:40:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b880cf2f0 msgr2=0x7f4b880ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b880cf2f0 0x7f4b880ae640 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800e870 tx=0x7f4b7800ed30 comp rx=0 tx=0).stop 2024-04-25T11:40:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 shutdown_connections 2024-04-25T11:40:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4b640744b0 0x7f4b640769c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b8815a810 0x7f4b880b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b881608d0 0x7f4b88149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 --2- 172.21.15.19:0/3435081848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b880cf2f0 0x7f4b880ae640 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 >> 172.21.15.19:0/3435081848 conn(0x7f4b880d1720 msgr2=0x7f4b880cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 shutdown_connections 2024-04-25T11:40:53.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.418+0000 7f4b8d200640 1 -- 172.21.15.19:0/3435081848 wait complete. 2024-04-25T11:40:53.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:40:53.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:40:53.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.739+0000 7fd6c4a00640 1 Processor -- start 2024-04-25T11:40:53.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.739+0000 7fd6c4a00640 1 -- start start 2024-04-25T11:40:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.742+0000 7fd6c4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 0x7fd6c015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.742+0000 7fd6c4a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c00cda60 con 0x7fd6c00ce930 2024-04-25T11:40:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.742+0000 7fd6c4a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c015dfe0 con 0x7fd6c006dce0 2024-04-25T11:40:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.742+0000 7fd6c4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c015e2c0 con 0x7fd6c00ce520 2024-04-25T11:40:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.742+0000 7fd6b7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd6c00ce930 0x7fd6c006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37866/0 (socket says 172.21.15.19:37866) 2024-04-25T11:40:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.742+0000 7fd6b7e00640 1 -- 172.21.15.19:0/750268022 learned_addr learned my addr 172.21.15.19:0/750268022 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bf400640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 0x7fd6c015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437531233 0 0) 0x7fd6c00cda60 con 0x7fd6c00ce930 2024-04-25T11:40:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a4005730 con 0x7fd6c00ce930 2024-04-25T11:40:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 39165333 0 0) 0x7fd6a4005730 con 0x7fd6c00ce930 2024-04-25T11:40:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 msgr2=0x7fd6c015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 0x7fd6c015d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 >> v1:172.21.15.169:6789/0 conn(0x7fd6c006dce0 legacy=0x7fd6c00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c00cdd20 con 0x7fd6c00ce930 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fd6a8001670 con 0x7fd6c00ce930 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 >> v1:172.21.15.19:6789/0 conn(0x7fd6c00ce930 legacy=0x7fd6c006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.743+0000 7fd6bea00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6a4005bf0 0x7fd6a4006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bea00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 0x7fd6a4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bea00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a4006f50 0x7fd6a40073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c015e2c0 con 0x7fd6a4005bf0 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a400de00 con 0x7fd6a4006f50 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a400e000 con 0x7fd6c00ce520 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bf400640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6a4005bf0 0x7fd6a4006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6b7e00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 0x7fd6a4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bfe00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a4006f50 0x7fd6a40073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bf400640 1 -- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 msgr2=0x7fd6a4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bf400640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 0x7fd6a4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bf400640 1 -- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a4006f50 msgr2=0x7fd6a40073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bf400640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a4006f50 0x7fd6a40073c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bf400640 1 -- 172.21.15.19:0/750268022 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a4005730 con 0x7fd6a4005bf0 2024-04-25T11:40:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6b7e00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6c00ce520 0x7fd6a4006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.744+0000 7fd6bfe00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6a4006f50 0x7fd6a40073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.746+0000 7fd6bf400640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6a4005bf0 0x7fd6a4006060 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009a90 tx=0x7fd6a4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c05b43f0503ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.746+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0036070 con 0x7fd6a4005bf0 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.746+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0031400 con 0x7fd6a4005bf0 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.746+0000 7fd6bea00640 1 -- 172.21.15.19:0/750268022 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0009690 con 0x7fd6a4005bf0 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.747+0000 7fd6c4a00640 1 -- 172.21.15.19:0/750268022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6a4005bf0 msgr2=0x7fd6a4006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.747+0000 7fd6c4a00640 1 --2- 172.21.15.19:0/750268022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6a4005bf0 0x7fd6a4006060 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009a90 tx=0x7fd6a4007a40 comp rx=0 tx=0).stop 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.747+0000 7fd6bfe00640 1 -- 172.21.15.19:0/750268022 reap_dead start 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.747+0000 7fd6c4a00640 1 -- 172.21.15.19:0/750268022 shutdown_connections 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.747+0000 7fd6c4a00640 1 -- 172.21.15.19:0/750268022 >> 172.21.15.19:0/750268022 conn(0x7fd6c0072f40 msgr2=0x7fd6c0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.747+0000 7fd6c4a00640 1 -- 172.21.15.19:0/750268022 shutdown_connections 2024-04-25T11:40:53.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.747+0000 7fd6c4a00640 1 -- 172.21.15.19:0/750268022 wait complete. 2024-04-25T11:40:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.749+0000 7fd6c4a00640 1 Processor -- start 2024-04-25T11:40:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.750+0000 7fd6c4a00640 1 -- start start 2024-04-25T11:40:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.751+0000 7fd6c4a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6a4005bf0 0x7fd6c0147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.751+0000 7fd6c4a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6c0070a40 0x7fd6c00cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.751+0000 7fd6c4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6c00cc180 0x7fd6c0149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.751+0000 7fd6c4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00b73d0 con 0x7fd6c00cc180 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.751+0000 7fd6c4a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0160d30 con 0x7fd6c0070a40 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.751+0000 7fd6c4a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c00b7550 con 0x7fd6a4005bf0 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6c00cc180 0x7fd6c0149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6c00cc180 0x7fd6c0149a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46164/0 (socket says 172.21.15.19:46164) 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 -- 172.21.15.19:0/2591375009 learned_addr learned my addr 172.21.15.19:0/2591375009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6b7e00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6c0070a40 0x7fd6c00cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bf400640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6a4005bf0 0x7fd6c0147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 -- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6a4005bf0 msgr2=0x7fd6c0147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6a4005bf0 0x7fd6c0147250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 -- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6c0070a40 msgr2=0x7fd6c00cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6c0070a40 0x7fd6c00cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 -- 172.21.15.19:0/2591375009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c00cc590 con 0x7fd6c00cc180 2024-04-25T11:40:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.752+0000 7fd6bfe00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6c00cc180 0x7fd6c0149a70 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00b4f50 tx=0x7fd6b800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.753+0000 7fd6b7400640 1 -- 172.21.15.19:0/2591375009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8018070 con 0x7fd6c00cc180 2024-04-25T11:40:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.753+0000 7fd6b7400640 1 -- 172.21.15.19:0/2591375009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b8003da0 con 0x7fd6c00cc180 2024-04-25T11:40:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.753+0000 7fd6b7400640 1 -- 172.21.15.19:0/2591375009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8006d90 con 0x7fd6c00cc180 2024-04-25T11:40:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.753+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c006f240 con 0x7fd6c00cc180 2024-04-25T11:40:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.753+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c006f7e0 con 0x7fd6c00cc180 2024-04-25T11:40:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.754+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c0076c60 con 0x7fd6c00cc180 2024-04-25T11:40:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.755+0000 7fd6b7400640 1 -- 172.21.15.19:0/2591375009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6b8005070 con 0x7fd6c00cc180 2024-04-25T11:40:53.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.756+0000 7fd6b7400640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd69c0744b0 0x7fd69c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:53.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.756+0000 7fd6b7400640 1 -- 172.21.15.19:0/2591375009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7fd6b8094e70 con 0x7fd6c00cc180 2024-04-25T11:40:53.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.757+0000 7fd6bf400640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd69c0744b0 0x7fd69c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.758+0000 7fd6bf400640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd69c0744b0 0x7fd69c0769c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0033000 tx=0x7fd6b0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.761+0000 7fd6b7400640 1 -- 172.21.15.19:0/2591375009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6c00cc590 con 0x7fd6c00cc180 2024-04-25T11:40:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.970+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6c015b010 con 0x7fd6c00cc180 2024-04-25T11:40:53.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.971+0000 7fd6b7400640 1 -- 172.21.15.19:0/2591375009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7fd6b805e3e0 con 0x7fd6c00cc180 2024-04-25T11:40:53.959 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd69c0744b0 msgr2=0x7fd69c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd69c0744b0 0x7fd69c0769c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0033000 tx=0x7fd6b0008030 comp rx=0 tx=0).stop 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6c00cc180 msgr2=0x7fd6c0149a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6c00cc180 0x7fd6c0149a70 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00b4f50 tx=0x7fd6b800d9d0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 shutdown_connections 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd69c0744b0 0x7fd69c0769c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6c0070a40 0x7fd6c00cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6c00cc180 0x7fd6c0149a70 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 --2- 172.21.15.19:0/2591375009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6a4005bf0 0x7fd6c0147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 >> 172.21.15.19:0/2591375009 conn(0x7fd6c0072f40 msgr2=0x7fd6c015b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 shutdown_connections 2024-04-25T11:40:53.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:53.974+0000 7fd6c4a00640 1 -- 172.21.15.19:0/2591375009 wait complete. 2024-04-25T11:40:53.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:40:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.285+0000 7f0516800640 1 Processor -- start 2024-04-25T11:40:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.285+0000 7f0516800640 1 -- start start 2024-04-25T11:40:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.288+0000 7f0516800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f051006dce0 0x7f05101606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.288+0000 7f0516800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05100b0090 con 0x7f051015de90 2024-04-25T11:40:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.288+0000 7f0516800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0510160c80 con 0x7f0510163c90 2024-04-25T11:40:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.288+0000 7f0516800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0510160f60 con 0x7f051006dce0 2024-04-25T11:40:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.288+0000 7f0515e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f051006dce0 0x7f05101606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.288+0000 7f0515400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f051015de90 0x7f051006f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49592/0 (socket says 172.21.15.19:49592) 2024-04-25T11:40:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.288+0000 7f0515400640 1 -- 172.21.15.19:0/3826035285 learned_addr learned my addr 172.21.15.19:0/3826035285 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.289+0000 7f050fe00640 1 -- 172.21.15.19:0/3826035285 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3353563253 0 0) 0x7f05100b0090 con 0x7f051015de90 2024-04-25T11:40:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.289+0000 7f050fe00640 1 -- 172.21.15.19:0/3826035285 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04f8003520 con 0x7f051015de90 2024-04-25T11:40:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.289+0000 7f0515e00640 1 -- 172.21.15.19:0/3826035285 >> v1:172.21.15.169:6789/0 conn(0x7f0510163c90 legacy=0x7f051006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.289+0000 7f0515e00640 1 -- 172.21.15.19:0/3826035285 >> v1:172.21.15.19:6789/0 conn(0x7f051015de90 legacy=0x7f051006f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.289+0000 7f0515e00640 1 -- 172.21.15.19:0/3826035285 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0510161240 con 0x7f051006dce0 2024-04-25T11:40:54.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.291+0000 7f0515e00640 1 --2- 172.21.15.19:0/3826035285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051006dce0 0x7f05101606a0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f0508009d90 tx=0x7f050802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6e12c0776e22b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:54.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.291+0000 7f050fe00640 1 -- 172.21.15.19:0/3826035285 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508007740 con 0x7f051006dce0 2024-04-25T11:40:54.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.291+0000 7f050fe00640 1 -- 172.21.15.19:0/3826035285 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05080078e0 con 0x7f051006dce0 2024-04-25T11:40:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.291+0000 7f050fe00640 1 -- 172.21.15.19:0/3826035285 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0508007cb0 con 0x7f051006dce0 2024-04-25T11:40:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.292+0000 7f0516800640 1 -- 172.21.15.19:0/3826035285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051006dce0 msgr2=0x7f05101606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.292+0000 7f0516800640 1 --2- 172.21.15.19:0/3826035285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051006dce0 0x7f05101606a0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f0508009d90 tx=0x7f050802a660 comp rx=0 tx=0).stop 2024-04-25T11:40:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.292+0000 7f0516800640 1 -- 172.21.15.19:0/3826035285 shutdown_connections 2024-04-25T11:40:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.292+0000 7f0516800640 1 --2- 172.21.15.19:0/3826035285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051006dce0 0x7f05101606a0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.292+0000 7f0516800640 1 -- 172.21.15.19:0/3826035285 >> 172.21.15.19:0/3826035285 conn(0x7f0510072f40 msgr2=0x7f0510073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.292+0000 7f0516800640 1 -- 172.21.15.19:0/3826035285 shutdown_connections 2024-04-25T11:40:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.292+0000 7f0516800640 1 -- 172.21.15.19:0/3826035285 wait complete. 2024-04-25T11:40:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.295+0000 7f0516800640 1 Processor -- start 2024-04-25T11:40:54.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.295+0000 7f0516800640 1 -- start start 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.296+0000 7f0516800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051015de90 0x7f05100d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.296+0000 7f0516800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0510163c90 0x7f051014d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.296+0000 7f0516800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05101f9d90 0x7f0510162890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.296+0000 7f0516800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0510070eb0 con 0x7f051015de90 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.296+0000 7f0516800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0510070d30 con 0x7f05101f9d90 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.296+0000 7f0516800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f05100b0570 con 0x7f0510163c90 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051015de90 0x7f05100d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051015de90 0x7f05100d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48200/0 (socket says 172.21.15.19:48200) 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 -- 172.21.15.19:0/2067497451 learned_addr learned my addr 172.21.15.19:0/2067497451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:54.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0514a00640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0510163c90 0x7f051014d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515e00640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05101f9d90 0x7f0510162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 -- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0510163c90 msgr2=0x7f051014d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0510163c90 0x7f051014d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 -- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05101f9d90 msgr2=0x7f0510162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05101f9d90 0x7f0510162890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0515400640 1 -- 172.21.15.19:0/2067497451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051014af80 con 0x7f051015de90 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.297+0000 7f0514a00640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0510163c90 0x7f051014d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:54.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.298+0000 7f0515400640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051015de90 0x7f05100d0ab0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f050400c430 tx=0x7f050400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.298+0000 7f050d600640 1 -- 172.21.15.19:0/2067497451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504017070 con 0x7f051015de90 2024-04-25T11:40:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.298+0000 7f050d600640 1 -- 172.21.15.19:0/2067497451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0504005ce0 con 0x7f051015de90 2024-04-25T11:40:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.298+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051014dfb0 con 0x7f051015de90 2024-04-25T11:40:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.298+0000 7f050d600640 1 -- 172.21.15.19:0/2067497451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040072f0 con 0x7f051015de90 2024-04-25T11:40:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.298+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101f9590 con 0x7f051015de90 2024-04-25T11:40:54.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.299+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0510076c60 con 0x7f051015de90 2024-04-25T11:40:54.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.300+0000 7f050d600640 1 -- 172.21.15.19:0/2067497451 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f050401a070 con 0x7f051015de90 2024-04-25T11:40:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.301+0000 7f050d600640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04f00744b0 0x7f04f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.302+0000 7f050d600640 1 -- 172.21.15.19:0/2067497451 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f0504095210 con 0x7f051015de90 2024-04-25T11:40:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.302+0000 7f0514a00640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04f00744b0 0x7f04f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.303+0000 7f0514a00640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04f00744b0 0x7f04f00769c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0510150d60 tx=0x7f04fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.307+0000 7f050d600640 1 -- 172.21.15.19:0/2067497451 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f051014af80 con 0x7f051015de90 2024-04-25T11:40:54.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.511+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0510071130 con 0x7f051015de90 2024-04-25T11:40:54.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.512+0000 7f050d600640 1 -- 172.21.15.19:0/2067497451 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) ==== 104+0+11 (secure 0 0 0) 0x7f050405ee90 con 0x7f051015de90 2024-04-25T11:40:54.500 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04f00744b0 msgr2=0x7f04f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04f00744b0 0x7f04f00769c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0510150d60 tx=0x7f04fc009040 comp rx=0 tx=0).stop 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051015de90 msgr2=0x7f05100d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051015de90 0x7f05100d0ab0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f050400c430 tx=0x7f050400c8f0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 shutdown_connections 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04f00744b0 0x7f04f00769c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05101f9d90 0x7f0510162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0510163c90 0x7f051014d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 --2- 172.21.15.19:0/2067497451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f051015de90 0x7f05100d0ab0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 >> 172.21.15.19:0/2067497451 conn(0x7f0510072f40 msgr2=0x7f051006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 shutdown_connections 2024-04-25T11:40:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.515+0000 7f0516800640 1 -- 172.21.15.19:0/2067497451 wait complete. 2024-04-25T11:40:54.531 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:40:54.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.832+0000 7fc00a000640 1 Processor -- start 2024-04-25T11:40:54.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.833+0000 7fc00a000640 1 -- start start 2024-04-25T11:40:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.835+0000 7fc00a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0040af710 0x7fc004164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.835+0000 7fc00a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc004160390 0x7fc00406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.835+0000 7fc00a000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00406df20 con 0x7fc00406db10 2024-04-25T11:40:54.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.835+0000 7fc00a000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc0040afb20 con 0x7fc0040af710 2024-04-25T11:40:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.835+0000 7fc00a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc004164fc0 con 0x7fc004160390 2024-04-25T11:40:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc003400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0040af710 0x7fc004164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc003400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0040af710 0x7fc004164a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48208/0 (socket says 172.21.15.19:48208) 2024-04-25T11:40:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc003400640 1 -- 172.21.15.19:0/2756362705 learned_addr learned my addr 172.21.15.19:0/2756362705 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:54.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc002a00640 1 --2- 172.21.15.19:0/2756362705 >> v2:172.21.15.169:3300/0 conn(0x7fc004160390 0x7fc00406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc003400640 1 -- 172.21.15.19:0/2756362705 >> v2:172.21.15.169:3300/0 conn(0x7fc004160390 msgr2=0x7fc00406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc003400640 1 --2- 172.21.15.19:0/2756362705 >> v2:172.21.15.169:3300/0 conn(0x7fc004160390 0x7fc00406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc003400640 1 -- 172.21.15.19:0/2756362705 >> v1:172.21.15.169:6789/0 conn(0x7fc00406db10 legacy=0x7fc0040b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc003400640 1 -- 172.21.15.19:0/2756362705 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00406f440 con 0x7fc0040af710 2024-04-25T11:40:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.836+0000 7fc002a00640 1 --2- 172.21.15.19:0/2756362705 >> v2:172.21.15.169:3300/0 conn(0x7fc004160390 0x7fc00406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:54.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.837+0000 7fc003400640 1 --2- 172.21.15.19:0/2756362705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0040af710 0x7fc004164a80 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fc0040b5d10 tx=0x7fbff402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec0fa5523e59232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:54.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.838+0000 7fc002000640 1 -- 172.21.15.19:0/2756362705 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4037070 con 0x7fc0040af710 2024-04-25T11:40:54.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.838+0000 7fc002000640 1 -- 172.21.15.19:0/2756362705 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff4032440 con 0x7fc0040af710 2024-04-25T11:40:54.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.838+0000 7fc002000640 1 -- 172.21.15.19:0/2756362705 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4032810 con 0x7fc0040af710 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 -- 172.21.15.19:0/2756362705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0040af710 msgr2=0x7fc004164a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 --2- 172.21.15.19:0/2756362705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0040af710 0x7fc004164a80 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fc0040b5d10 tx=0x7fbff402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 -- 172.21.15.19:0/2756362705 shutdown_connections 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 --2- 172.21.15.19:0/2756362705 >> v2:172.21.15.169:3300/0 conn(0x7fc004160390 0x7fc00406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 --2- 172.21.15.19:0/2756362705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0040af710 0x7fc004164a80 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 -- 172.21.15.19:0/2756362705 >> 172.21.15.19:0/2756362705 conn(0x7fc004073090 msgr2=0x7fc0040734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 -- 172.21.15.19:0/2756362705 shutdown_connections 2024-04-25T11:40:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.839+0000 7fc00a000640 1 -- 172.21.15.19:0/2756362705 wait complete. 2024-04-25T11:40:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.842+0000 7fc00a000640 1 Processor -- start 2024-04-25T11:40:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.842+0000 7fc00a000640 1 -- start start 2024-04-25T11:40:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.843+0000 7fc00a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00406db10 0x7fc00414fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.843+0000 7fc00a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0040af710 0x7fc00415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.843+0000 7fc00a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc004160390 0x7fc0040b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.843+0000 7fc00a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc00406ffc0 con 0x7fc004160390 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.843+0000 7fc00a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc00406fce0 con 0x7fc00406db10 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.843+0000 7fc00a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc00406f0d0 con 0x7fc0040af710 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc004160390 0x7fc0040b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc004160390 0x7fc0040b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48222/0 (socket says 172.21.15.19:48222) 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 -- 172.21.15.19:0/615657105 learned_addr learned my addr 172.21.15.19:0/615657105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003400640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00406db10 0x7fc00414fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc002a00640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0040af710 0x7fc00415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 -- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0040af710 msgr2=0x7fc00415c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0040af710 0x7fc00415c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 -- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00406db10 msgr2=0x7fc00414fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00406db10 0x7fc00414fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc003e00640 1 -- 172.21.15.19:0/615657105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc004153e50 con 0x7fc004160390 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.844+0000 7fc002a00640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0040af710 0x7fc00415c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:54.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.845+0000 7fc003e00640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc004160390 0x7fc0040b4990 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b970 tx=0x7fbffc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.845+0000 7fbffb400640 1 -- 172.21.15.19:0/615657105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc019070 con 0x7fc004160390 2024-04-25T11:40:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.845+0000 7fbffb400640 1 -- 172.21.15.19:0/615657105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc00cb40 con 0x7fc004160390 2024-04-25T11:40:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.845+0000 7fbffb400640 1 -- 172.21.15.19:0/615657105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc014440 con 0x7fc004160390 2024-04-25T11:40:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.845+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00415a630 con 0x7fc004160390 2024-04-25T11:40:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.845+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00415cea0 con 0x7fc004160390 2024-04-25T11:40:54.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.846+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc004050740 con 0x7fc004160390 2024-04-25T11:40:54.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.848+0000 7fbffb400640 1 -- 172.21.15.19:0/615657105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbffc01c070 con 0x7fc004160390 2024-04-25T11:40:54.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.849+0000 7fbffb400640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfe40744e0 0x7fbfe40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:54.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.849+0000 7fbffb400640 1 -- 172.21.15.19:0/615657105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7fbffc095ac0 con 0x7fc004160390 2024-04-25T11:40:54.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.850+0000 7fc003400640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfe40744e0 0x7fbfe40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:54.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.851+0000 7fc003400640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfe40744e0 0x7fbfe40769f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbff4034000 tx=0x7fbff402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:54.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:54.854+0000 7fbffb400640 1 -- 172.21.15.19:0/615657105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbffc05f6e0 con 0x7fc004160390 2024-04-25T11:40:55.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.062+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0040b5d40 con 0x7fc004160390 2024-04-25T11:40:55.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.063+0000 7fbffb400640 1 -- 172.21.15.19:0/615657105 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) ==== 100+0+11 (secure 0 0 0) 0x7fbffc05efe0 con 0x7fc004160390 2024-04-25T11:40:55.051 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:40:55.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.065+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfe40744e0 msgr2=0x7fbfe40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.065+0000 7fc00a000640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfe40744e0 0x7fbfe40769f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbff4034000 tx=0x7fbff402e000 comp rx=0 tx=0).stop 2024-04-25T11:40:55.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.065+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc004160390 msgr2=0x7fc0040b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.065+0000 7fc00a000640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc004160390 0x7fc0040b4990 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b970 tx=0x7fbffc00be30 comp rx=0 tx=0).stop 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 shutdown_connections 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbfe40744e0 0x7fbfe40769f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc004160390 0x7fc0040b4990 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0040af710 0x7fc00415c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 --2- 172.21.15.19:0/615657105 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc00406db10 0x7fc00414fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 >> 172.21.15.19:0/615657105 conn(0x7fc004073090 msgr2=0x7fc0041589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 shutdown_connections 2024-04-25T11:40:55.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.066+0000 7fc00a000640 1 -- 172.21.15.19:0/615657105 wait complete. 2024-04-25T11:40:55.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:40:55.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.375+0000 7ff9e0800640 1 Processor -- start 2024-04-25T11:40:55.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.376+0000 7ff9e0800640 1 -- start start 2024-04-25T11:40:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.378+0000 7ff9e0800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9d8144500 0x7ff9d8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.378+0000 7ff9e0800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff9d8162310 0x7ff9d80ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.378+0000 7ff9e0800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9d806f5f0 con 0x7ff9d80cdb90 2024-04-25T11:40:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.378+0000 7ff9e0800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d8070200 con 0x7ff9d8144500 2024-04-25T11:40:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.378+0000 7ff9e0800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d806f8d0 con 0x7ff9d8162310 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dde00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9d8144500 0x7ff9d8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dd400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff9d80cdb90 0x7ff9d8161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49602/0 (socket says 172.21.15.19:49602) 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dd400640 1 -- 172.21.15.19:0/1848137103 learned_addr learned my addr 172.21.15.19:0/1848137103 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9de800640 1 --2- 172.21.15.19:0/1848137103 >> v2:172.21.15.169:3300/0 conn(0x7ff9d8162310 0x7ff9d80ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dde00640 1 -- 172.21.15.19:0/1848137103 >> v2:172.21.15.169:3300/0 conn(0x7ff9d8162310 msgr2=0x7ff9d80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dde00640 1 --2- 172.21.15.19:0/1848137103 >> v2:172.21.15.169:3300/0 conn(0x7ff9d8162310 0x7ff9d80ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dde00640 1 -- 172.21.15.19:0/1848137103 >> v1:172.21.15.19:6789/0 conn(0x7ff9d80cdb90 legacy=0x7ff9d8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dde00640 1 -- 172.21.15.19:0/1848137103 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d806fbb0 con 0x7ff9d8144500 2024-04-25T11:40:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.379+0000 7ff9dca00640 1 -- 172.21.15.19:0/1848137103 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181889540 0 0) 0x7ff9d806f5f0 con 0x7ff9d80cdb90 2024-04-25T11:40:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.381+0000 7ff9dde00640 1 --2- 172.21.15.19:0/1848137103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d8070690 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80ce840 tx=0x7ff9cc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160fa78ce343ee0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.381+0000 7ff9dca00640 1 -- 172.21.15.19:0/1848137103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc037070 con 0x7ff9d8144500 2024-04-25T11:40:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.381+0000 7ff9dca00640 1 -- 172.21.15.19:0/1848137103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9cc032440 con 0x7ff9d8144500 2024-04-25T11:40:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.381+0000 7ff9dca00640 1 -- 172.21.15.19:0/1848137103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc02ae60 con 0x7ff9d8144500 2024-04-25T11:40:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 -- 172.21.15.19:0/1848137103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 msgr2=0x7ff9d8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 --2- 172.21.15.19:0/1848137103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d8070690 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80ce840 tx=0x7ff9cc02a110 comp rx=0 tx=0).stop 2024-04-25T11:40:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 -- 172.21.15.19:0/1848137103 shutdown_connections 2024-04-25T11:40:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 --2- 172.21.15.19:0/1848137103 >> v2:172.21.15.169:3300/0 conn(0x7ff9d8162310 0x7ff9d80ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 --2- 172.21.15.19:0/1848137103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d8070690 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 -- 172.21.15.19:0/1848137103 >> 172.21.15.19:0/1848137103 conn(0x7ff9d8073050 msgr2=0x7ff9d8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 -- 172.21.15.19:0/1848137103 shutdown_connections 2024-04-25T11:40:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.382+0000 7ff9e0800640 1 -- 172.21.15.19:0/1848137103 wait complete. 2024-04-25T11:40:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.385+0000 7ff9e0800640 1 Processor -- start 2024-04-25T11:40:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.385+0000 7ff9e0800640 1 -- start start 2024-04-25T11:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.386+0000 7ff9e0800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d80cdb90 0x7ff9d81419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.386+0000 7ff9e0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d80b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.386+0000 7ff9e0800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d80b1b70 0x7ff9d80b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.386+0000 7ff9e0800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d806f2e0 con 0x7ff9d8144500 2024-04-25T11:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.386+0000 7ff9e0800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d806f160 con 0x7ff9d80b1b70 2024-04-25T11:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.386+0000 7ff9e0800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d80cdfa0 con 0x7ff9d80cdb90 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d80b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d80b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48228/0 (socket says 172.21.15.19:48228) 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 -- 172.21.15.19:0/3568472860 learned_addr learned my addr 172.21.15.19:0/3568472860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9de800640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d80b1b70 0x7ff9d80b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dde00640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d80cdb90 0x7ff9d81419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 -- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d80cdb90 msgr2=0x7ff9d81419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d80cdb90 0x7ff9d81419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 -- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d80b1b70 msgr2=0x7ff9d80b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d80b1b70 0x7ff9d80b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dd400640 1 -- 172.21.15.19:0/3568472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d80b4aa0 con 0x7ff9d8144500 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.387+0000 7ff9dde00640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d80cdb90 0x7ff9d81419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.388+0000 7ff9dd400640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d80b1630 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c440 tx=0x7ff9c800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.388+0000 7ff9d2000640 1 -- 172.21.15.19:0/3568472860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8018070 con 0x7ff9d8144500 2024-04-25T11:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.388+0000 7ff9d2000640 1 -- 172.21.15.19:0/3568472860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c80074e0 con 0x7ff9d8144500 2024-04-25T11:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.388+0000 7ff9d2000640 1 -- 172.21.15.19:0/3568472860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8004a20 con 0x7ff9d8144500 2024-04-25T11:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.388+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d80af430 con 0x7ff9d8144500 2024-04-25T11:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.388+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d80b62e0 con 0x7ff9d8144500 2024-04-25T11:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.389+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d80d03c0 con 0x7ff9d8144500 2024-04-25T11:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.391+0000 7ff9d2000640 1 -- 172.21.15.19:0/3568472860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9c801b070 con 0x7ff9d8144500 2024-04-25T11:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.392+0000 7ff9d2000640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc0744b0 0x7ff9bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.392+0000 7ff9d2000640 1 -- 172.21.15.19:0/3568472860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7ff9c8094fd0 con 0x7ff9d8144500 2024-04-25T11:40:55.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.393+0000 7ff9dde00640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc0744b0 0x7ff9bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:55.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.394+0000 7ff9dde00640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc0744b0 0x7ff9bc0769c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc034000 tx=0x7ff9cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.396+0000 7ff9d2000640 1 -- 172.21.15.19:0/3568472860 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9d80d03c0 con 0x7ff9d8144500 2024-04-25T11:40:55.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.624+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff9d8050740 con 0x7ff9d8144500 2024-04-25T11:40:55.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.626+0000 7ff9d2000640 1 -- 172.21.15.19:0/3568472860 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 66 entries v125) ==== 81+0+4004 (secure 0 0 0) 0x7ff9c805ec20 con 0x7ff9d8144500 2024-04-25T11:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/698241108 2024-04-26T11:40:28.419530+0000 2024-04-25T11:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/698241108 2024-04-26T11:40:28.419530+0000 2024-04-25T11:40:55.613 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2891720740 2024-04-26T11:40:22.060358+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2891720740 2024-04-26T11:40:22.060358+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:40:55.614 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:40:55.615 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:40:55.616 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:40:55.617 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:40:55.618 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:40:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:listed 66 entries 2024-04-25T11:40:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc0744b0 msgr2=0x7ff9bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc0744b0 0x7ff9bc0769c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc034000 tx=0x7ff9cc02e000 comp rx=0 tx=0).stop 2024-04-25T11:40:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 msgr2=0x7ff9d80b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d80b1630 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800c440 tx=0x7ff9c800c900 comp rx=0 tx=0).stop 2024-04-25T11:40:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 shutdown_connections 2024-04-25T11:40:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9bc0744b0 0x7ff9bc0769c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9d80b1b70 0x7ff9d80b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d8144500 0x7ff9d80b1630 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 --2- 172.21.15.19:0/3568472860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9d80cdb90 0x7ff9d81419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 >> 172.21.15.19:0/3568472860 conn(0x7ff9d8073050 msgr2=0x7ff9d8143270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.628+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 shutdown_connections 2024-04-25T11:40:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:55.629+0000 7ff9e0800640 1 -- 172.21.15.19:0/3568472860 wait complete. 2024-04-25T11:40:55.642 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:40:55.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:55.642 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:40:55.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:40:55.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:40:55.733 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:40:55.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:40:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.076+0000 7f4c90e00640 1 Processor -- start 2024-04-25T11:40:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.076+0000 7f4c90e00640 1 -- start start 2024-04-25T11:40:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c90e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4c8c06e740 0x7f4c8c15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c90e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c8c163fc0 0x7f4c8c1643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c90e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c8c164c80 con 0x7f4c8c1616a0 2024-04-25T11:40:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c90e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4c8c160980 con 0x7f4c8c06e740 2024-04-25T11:40:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c90e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4c8c160bc0 con 0x7f4c8c163fc0 2024-04-25T11:40:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c81400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4c8c06e740 0x7f4c8c15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c89e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4c8c1616a0 0x7f4c8c06e030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49612/0 (socket says 172.21.15.19:49612) 2024-04-25T11:40:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.078+0000 7f4c89e00640 1 -- 172.21.15.19:0/147914579 learned_addr learned my addr 172.21.15.19:0/147914579 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.079+0000 7f4c8a800640 1 --2- 172.21.15.19:0/147914579 >> v2:172.21.15.169:3300/0 conn(0x7f4c8c163fc0 0x7f4c8c1643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.079+0000 7f4c81400640 1 -- 172.21.15.19:0/147914579 >> v2:172.21.15.169:3300/0 conn(0x7f4c8c163fc0 msgr2=0x7f4c8c1643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.079+0000 7f4c81400640 1 --2- 172.21.15.19:0/147914579 >> v2:172.21.15.169:3300/0 conn(0x7f4c8c163fc0 0x7f4c8c1643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.079+0000 7f4c81400640 1 -- 172.21.15.19:0/147914579 >> v1:172.21.15.19:6789/0 conn(0x7f4c8c1616a0 legacy=0x7f4c8c06e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.079+0000 7f4c89400640 1 -- 172.21.15.19:0/147914579 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2434140109 0 0) 0x7f4c8c164c80 con 0x7f4c8c1616a0 2024-04-25T11:40:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.079+0000 7f4c81400640 1 -- 172.21.15.19:0/147914579 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c160ea0 con 0x7f4c8c06e740 2024-04-25T11:40:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.079+0000 7f4c8a800640 1 --2- 172.21.15.19:0/147914579 >> v2:172.21.15.169:3300/0 conn(0x7f4c8c163fc0 0x7f4c8c1643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.080+0000 7f4c81400640 1 --2- 172.21.15.19:0/147914579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c06e740 0x7f4c8c15ded0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0d23f0 tx=0x7f4c7402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d9837a472b20dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.080+0000 7f4c89400640 1 -- 172.21.15.19:0/147914579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74037070 con 0x7f4c8c06e740 2024-04-25T11:40:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.080+0000 7f4c89400640 1 -- 172.21.15.19:0/147914579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c74032440 con 0x7f4c8c06e740 2024-04-25T11:40:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.080+0000 7f4c89400640 1 -- 172.21.15.19:0/147914579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7402ae60 con 0x7f4c8c06e740 2024-04-25T11:40:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 -- 172.21.15.19:0/147914579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c06e740 msgr2=0x7f4c8c15ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 --2- 172.21.15.19:0/147914579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c06e740 0x7f4c8c15ded0 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0d23f0 tx=0x7f4c7402a110 comp rx=0 tx=0).stop 2024-04-25T11:40:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 -- 172.21.15.19:0/147914579 shutdown_connections 2024-04-25T11:40:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 --2- 172.21.15.19:0/147914579 >> v2:172.21.15.169:3300/0 conn(0x7f4c8c163fc0 0x7f4c8c1643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 --2- 172.21.15.19:0/147914579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c06e740 0x7f4c8c15ded0 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 -- 172.21.15.19:0/147914579 >> 172.21.15.19:0/147914579 conn(0x7f4c8c0d27c0 msgr2=0x7f4c8c0d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 -- 172.21.15.19:0/147914579 shutdown_connections 2024-04-25T11:40:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.081+0000 7f4c90e00640 1 -- 172.21.15.19:0/147914579 wait complete. 2024-04-25T11:40:56.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.084+0000 7f4c90e00640 1 Processor -- start 2024-04-25T11:40:56.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.084+0000 7f4c90e00640 1 -- start start 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.085+0000 7f4c90e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c8c1616a0 0x7f4c8c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.085+0000 7f4c90e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c8c163fc0 0x7f4c8c0c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.085+0000 7f4c90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c14ec20 0x7f4c8c14f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.085+0000 7f4c90e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c0b77c0 con 0x7f4c8c14ec20 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.085+0000 7f4c90e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c164910 con 0x7f4c8c163fc0 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.085+0000 7f4c90e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c0b7940 con 0x7f4c8c1616a0 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c14ec20 0x7f4c8c14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c14ec20 0x7f4c8c14f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48250/0 (socket says 172.21.15.19:48250) 2024-04-25T11:40:56.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 -- 172.21.15.19:0/788196029 learned_addr learned my addr 172.21.15.19:0/788196029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c89e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c8c1616a0 0x7f4c8c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 -- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c8c1616a0 msgr2=0x7f4c8c14c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c8c1616a0 0x7f4c8c14c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 -- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c8c163fc0 msgr2=0x7f4c8c0c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c8c163fc0 0x7f4c8c0c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c8a800640 1 -- 172.21.15.19:0/788196029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c14f570 con 0x7f4c8c14ec20 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.086+0000 7f4c89e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c8c1616a0 0x7f4c8c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.087+0000 7f4c8a800640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c14ec20 0x7f4c8c14f030 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400cd80 tx=0x7f4c8400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.087+0000 7f4c82a00640 1 -- 172.21.15.19:0/788196029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400f680 con 0x7f4c8c14ec20 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.087+0000 7f4c82a00640 1 -- 172.21.15.19:0/788196029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8400f840 con 0x7f4c8c14ec20 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.087+0000 7f4c82a00640 1 -- 172.21.15.19:0/788196029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400fc30 con 0x7f4c8c14ec20 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.087+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c0ac6b0 con 0x7f4c8c14ec20 2024-04-25T11:40:56.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.087+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c14cd10 con 0x7f4c8c14ec20 2024-04-25T11:40:56.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.088+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c050740 con 0x7f4c8c14ec20 2024-04-25T11:40:56.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.089+0000 7f4c82a00640 1 -- 172.21.15.19:0/788196029 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4c840213f0 con 0x7f4c8c14ec20 2024-04-25T11:40:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.091+0000 7f4c82a00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c680744b0 0x7f4c680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.091+0000 7f4c82a00640 1 -- 172.21.15.19:0/788196029 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8547+0+0 (secure 0 0 0) 0x7f4c84096bd0 con 0x7f4c8c14ec20 2024-04-25T11:40:56.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.092+0000 7f4c89e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c680744b0 0x7f4c680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.093+0000 7f4c89e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c680744b0 0x7f4c680769c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c000f60 tx=0x7f4c7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.096+0000 7f4c82a00640 1 -- 172.21.15.19:0/788196029 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c8c14f570 con 0x7f4c8c14ec20 2024-04-25T11:40:56.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.343+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4c8c0d3980 con 0x7f4c8c14ec20 2024-04-25T11:40:56.332 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:40:56.344+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:40:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.348+0000 7f4c82a00640 1 -- 172.21.15.19:0/788196029 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v311) ==== 161+0+0 (secure 0 0 0) 0x7f4c84014030 con 0x7f4c8c14ec20 2024-04-25T11:40:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:40:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.352+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c680744b0 msgr2=0x7f4c680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.352+0000 7f4c90e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c680744b0 0x7f4c680769c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c000f60 tx=0x7f4c7c009040 comp rx=0 tx=0).stop 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.352+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c14ec20 msgr2=0x7f4c8c14f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.352+0000 7f4c90e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c14ec20 0x7f4c8c14f030 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400cd80 tx=0x7f4c8400a650 comp rx=0 tx=0).stop 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.353+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 shutdown_connections 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.353+0000 7f4c90e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4c680744b0 0x7f4c680769c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.353+0000 7f4c90e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c8c163fc0 0x7f4c8c0c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.353+0000 7f4c90e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c8c14ec20 0x7f4c8c14f030 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.353+0000 7f4c90e00640 1 --2- 172.21.15.19:0/788196029 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c8c1616a0 0x7f4c8c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.353+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 >> 172.21.15.19:0/788196029 conn(0x7f4c8c0d27c0 msgr2=0x7f4c8c0bad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:56.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.354+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 shutdown_connections 2024-04-25T11:40:56.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.354+0000 7f4c90e00640 1 -- 172.21.15.19:0/788196029 wait complete. 2024-04-25T11:40:56.354 INFO:tasks.ceph.mds.a.smithi019.stderr: -811> 2024-04-25T11:40:44.446+0000 7fb959800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:40:56.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:40:56.490 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:40:56.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.656+0000 7f07eb600640 1 Processor -- start 2024-04-25T11:40:56.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.656+0000 7f07eb600640 1 -- start start 2024-04-25T11:40:56.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.658+0000 7f07eb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d0e00 0x7f07e40d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.658+0000 7f07eb600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07e406c5c0 0x7f07e406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.658+0000 7f07eb600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e406cf10 con 0x7f07e40b7810 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.659+0000 7f07eb600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f07e40d5a60 con 0x7f07e406c5c0 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.659+0000 7f07eb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07e40d1850 con 0x7f07e40d0e00 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.659+0000 7f07e3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07e406c5c0 0x7f07e406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.659+0000 7f07e9600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f07e40b7810 0x7f07e40b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49630/0 (socket says 172.21.15.19:49630) 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.659+0000 7f07e9600640 1 -- 172.21.15.19:0/285304816 learned_addr learned my addr 172.21.15.19:0/285304816 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.659+0000 7f07e8c00640 1 --2- 172.21.15.19:0/285304816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d0e00 0x7f07e40d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.660+0000 7f07e3e00640 1 -- 172.21.15.19:0/285304816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d0e00 msgr2=0x7f07e40d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:56.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.660+0000 7f07e3e00640 1 --2- 172.21.15.19:0/285304816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d0e00 0x7f07e40d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.660+0000 7f07e3e00640 1 -- 172.21.15.19:0/285304816 >> v1:172.21.15.19:6789/0 conn(0x7f07e40b7810 legacy=0x7f07e40b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.660+0000 7f07e3e00640 1 -- 172.21.15.19:0/285304816 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e4160de0 con 0x7f07e406c5c0 2024-04-25T11:40:56.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.661+0000 7f07e3e00640 1 --2- 172.21.15.19:0/285304816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e406c5c0 0x7f07e406c9d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0097a0 tx=0x7f07cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9479d669c3f29c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:56.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.661+0000 7f07e3400640 1 -- 172.21.15.19:0/285304816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc037070 con 0x7f07e406c5c0 2024-04-25T11:40:56.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.661+0000 7f07e3400640 1 -- 172.21.15.19:0/285304816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc032440 con 0x7f07e406c5c0 2024-04-25T11:40:56.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.661+0000 7f07e3400640 1 -- 172.21.15.19:0/285304816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc032810 con 0x7f07e406c5c0 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 -- 172.21.15.19:0/285304816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e406c5c0 msgr2=0x7f07e406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 --2- 172.21.15.19:0/285304816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e406c5c0 0x7f07e406c9d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0097a0 tx=0x7f07cc02a180 comp rx=0 tx=0).stop 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 -- 172.21.15.19:0/285304816 shutdown_connections 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 --2- 172.21.15.19:0/285304816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e406c5c0 0x7f07e406c9d0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 --2- 172.21.15.19:0/285304816 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40d0e00 0x7f07e40d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 -- 172.21.15.19:0/285304816 >> 172.21.15.19:0/285304816 conn(0x7f07e40d5d90 msgr2=0x7f07e40d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 -- 172.21.15.19:0/285304816 shutdown_connections 2024-04-25T11:40:56.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.662+0000 7f07eb600640 1 -- 172.21.15.19:0/285304816 wait complete. 2024-04-25T11:40:56.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.665+0000 7f07eb600640 1 Processor -- start 2024-04-25T11:40:56.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.666+0000 7f07eb600640 1 -- start start 2024-04-25T11:40:56.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07eb600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e406c5c0 0x7f07e415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07eb600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40b7810 0x7f07e40ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07eb600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d0e00 0x7f07e41f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07eb600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4160ad0 con 0x7f07e40d0e00 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07eb600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07e41613a0 con 0x7f07e406c5c0 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07eb600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f07e40b5320 con 0x7f07e40b7810 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07e9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d0e00 0x7f07e41f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07e9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d0e00 0x7f07e41f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48310/0 (socket says 172.21.15.19:48310) 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07e9600640 1 -- 172.21.15.19:0/1534877803 learned_addr learned my addr 172.21.15.19:0/1534877803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07e8c00640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e406c5c0 0x7f07e415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.668+0000 7f07e3e00640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40b7810 0x7f07e40ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e9600640 1 -- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40b7810 msgr2=0x7f07e40ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e9600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40b7810 0x7f07e40ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e9600640 1 -- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e406c5c0 msgr2=0x7f07e415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e9600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e406c5c0 0x7f07e415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e9600640 1 -- 172.21.15.19:0/1534877803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e40b40d0 con 0x7f07e40d0e00 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e9600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d0e00 0x7f07e41f70d0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00cad0 tx=0x7f07dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e0c00640 1 -- 172.21.15.19:0/1534877803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc017070 con 0x7f07e40d0e00 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e0c00640 1 -- 172.21.15.19:0/1534877803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc00ac30 con 0x7f07e40d0e00 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e40b43b0 con 0x7f07e40d0e00 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.669+0000 7f07e0c00640 1 -- 172.21.15.19:0/1534877803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc012400 con 0x7f07e40d0e00 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.670+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e41f6bb0 con 0x7f07e40d0e00 2024-04-25T11:40:56.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.671+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e4050740 con 0x7f07e40d0e00 2024-04-25T11:40:56.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.672+0000 7f07e0c00640 1 -- 172.21.15.19:0/1534877803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f07dc01a070 con 0x7f07e40d0e00 2024-04-25T11:40:56.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.674+0000 7f07e0c00640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:56.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.674+0000 7f07e0c00640 1 -- 172.21.15.19:0/1534877803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f07dc0959d0 con 0x7f07e40d0e00 2024-04-25T11:40:56.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.674+0000 7f07e8c00640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:56.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.676+0000 7f07e8c00640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f07d4006fd0 tx=0x7f07d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:56.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.679+0000 7f07e0c00640 1 -- 172.21.15.19:0/1534877803 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07e406cf10 con 0x7f07e40d0e00 2024-04-25T11:40:56.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:56.968+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f07e40d4320 con 0x7f07e40d0e00 2024-04-25T11:40:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.352+0000 7f07e0c00640 1 -- 172.21.15.19:0/1534877803 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v312) ==== 107+0+0 (secure 0 0 0) 0x7f07dc05eef0 con 0x7f07e40d0e00 2024-04-25T11:40:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 msgr2=0x7f07c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f07d4006fd0 tx=0x7f07d4008040 comp rx=0 tx=0).stop 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d0e00 msgr2=0x7f07e41f70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d0e00 0x7f07e41f70d0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00cad0 tx=0x7f07dc00cf90 comp rx=0 tx=0).stop 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 shutdown_connections 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f07c00744e0 0x7f07c00769f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e40d0e00 0x7f07e41f70d0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f07e40b7810 0x7f07e40ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 --2- 172.21.15.19:0/1534877803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e406c5c0 0x7f07e415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 >> 172.21.15.19:0/1534877803 conn(0x7f07e40d5d90 msgr2=0x7f07e415c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 shutdown_connections 2024-04-25T11:40:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.356+0000 7f07eb600640 1 -- 172.21.15.19:0/1534877803 wait complete. 2024-04-25T11:40:57.372 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:40:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.577+0000 7f45c7e17f40 1 Processor -- start 2024-04-25T11:40:57.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.577+0000 7f45c7e17f40 1 -- start start 2024-04-25T11:40:57.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c7e17f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6aebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c7e17f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6a4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c7e17f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 0x561ff6a4ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c7e17f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561ff6a683f0 con 0x561ff6a1f1e0 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c7e17f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561ff6a67e70 con 0x561ff6af8dd0 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c7e17f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561ff6a686d0 con 0x561ff6a1f5f0 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6aebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6aebe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48330/0 (socket says 172.21.15.19:48330) 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 -- 172.21.15.19:0/4292554496 learned_addr learned my addr 172.21.15.19:0/4292554496 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:57.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45bbe00640 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6a4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 -- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 msgr2=0x561ff6a4fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6a4fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 -- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 msgr2=0x561ff6a4ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 0x561ff6a4ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45c1000640 1 -- 172.21.15.19:0/4292554496 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff6948b30 con 0x561ff6a1f1e0 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.579+0000 7f45bbe00640 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6a4fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.580+0000 7f45c1000640 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6aebe90 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f45b4025d20 tx=0x7f45b4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c88be35591577a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.580+0000 7f45baa00640 1 -- 172.21.15.19:0/4292554496 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4053880 con 0x561ff6a1f1e0 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.580+0000 7f45baa00640 1 -- 172.21.15.19:0/4292554496 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b4053a20 con 0x561ff6a1f1e0 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.580+0000 7f45baa00640 1 -- 172.21.15.19:0/4292554496 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4053dd0 con 0x561ff6a1f1e0 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 -- 172.21.15.19:0/4292554496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 msgr2=0x561ff6aebe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6aebe90 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f45b4025d20 tx=0x7f45b4051000 comp rx=0 tx=0).stop 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 -- 172.21.15.19:0/4292554496 shutdown_connections 2024-04-25T11:40:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 0x561ff6a4ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6a4fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/4292554496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6aebe90 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 -- 172.21.15.19:0/4292554496 >> 172.21.15.19:0/4292554496 conn(0x561ff6941ac0 msgr2=0x561ff6af4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 -- 172.21.15.19:0/4292554496 shutdown_connections 2024-04-25T11:40:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.581+0000 7f45c7e17f40 1 -- 172.21.15.19:0/4292554496 wait complete. 2024-04-25T11:40:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.582+0000 7f45c7e17f40 1 Processor -- start 2024-04-25T11:40:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.582+0000 7f45c7e17f40 1 -- start start 2024-04-25T11:40:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c7e17f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6a4f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c7e17f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6b86440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c7e17f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 0x561ff6b86980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c7e17f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561ff6af3680 con 0x561ff6a1f1e0 2024-04-25T11:40:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c7e17f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x561ff6af38d0 con 0x561ff6af8dd0 2024-04-25T11:40:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c7e17f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x561ff6b87550 con 0x561ff6a1f5f0 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6a4f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6a4f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48338/0 (socket says 172.21.15.19:48338) 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 learned_addr learned my addr 172.21.15.19:0/309318060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45bbe00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6b86440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.584+0000 7f45c1a00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 0x561ff6b86980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 msgr2=0x561ff6b86440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45c1000640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561ff6a1f5f0 0x561ff6b86440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 msgr2=0x561ff6b86980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45c1000640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561ff6af8dd0 0x561ff6b86980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff6ae9470 con 0x561ff6a1f1e0 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45c1000640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6a4f860 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005570 tx=0x7f45b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45b9600640 1 -- 172.21.15.19:0/309318060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4053cd0 con 0x561ff6a1f1e0 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45b9600640 1 -- 172.21.15.19:0/309318060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b4067430 con 0x561ff6a1f1e0 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561ff6af5590 con 0x561ff6a1f1e0 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.585+0000 7f45b9600640 1 -- 172.21.15.19:0/309318060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40677e0 con 0x561ff6a1f1e0 2024-04-25T11:40:57.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.586+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff6aec600 con 0x561ff6a1f1e0 2024-04-25T11:40:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.588+0000 7f45b9600640 1 -- 172.21.15.19:0/309318060 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f45b4067980 con 0x561ff6a1f1e0 2024-04-25T11:40:57.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.589+0000 7f45b9600640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45a0072e90 0x7f45a00753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.589+0000 7f45b9600640 1 -- 172.21.15.19:0/309318060 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f45b40db260 con 0x561ff6a1f1e0 2024-04-25T11:40:57.577 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:40:57.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.589+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561ff6ae9770 0x561ff6b901f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.589+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6a4fda0 con 0x561ff6ae9770 2024-04-25T11:40:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.590+0000 7f45bbe00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45a0072e90 0x7f45a00753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.590+0000 7f45c1a00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561ff6ae9770 0x561ff6b901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.591+0000 7f45bbe00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45a0072e90 0x7f45a00753a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x561ff6af5cb0 tx=0x7f45ac005b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.592+0000 7f45c1a00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561ff6ae9770 0x561ff6b901f0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.596+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v126'1033 uv1007 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.596+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b86ec0 con 0x561ff6ae9770 2024-04-25T11:40:57.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.599+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v126'1034 uv1034 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.599+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b98f40 con 0x561ff6ae9770 2024-04-25T11:40:57.590 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:40:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.602+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v126'1035 uv1035 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.603+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9a2f0 con 0x561ff6ae9770 2024-04-25T11:40:57.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.607+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'1036 uv1036 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.607+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9a690 con 0x561ff6ae9770 2024-04-25T11:40:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.611+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'1037 uv1037 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.611+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b992e0 con 0x561ff6ae9770 2024-04-25T11:40:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.615+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'1038 uv1038 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.615+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99680 con 0x561ff6ae9770 2024-04-25T11:40:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.620+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v126'1039 uv1013 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.620+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b99a70 con 0x561ff6ae9770 2024-04-25T11:40:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.624+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v126'1040 uv1040 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.624+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a70 con 0x561ff6ae9770 2024-04-25T11:40:57.615 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:40:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.628+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v126'1041 uv1041 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.628+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.632+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'1042 uv1042 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.632+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.636+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'1043 uv1043 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.636+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.641+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'1044 uv1044 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.641+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.646+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v126'1045 uv1020 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.646+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.650+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'1046 uv1046 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.651+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.643 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:40:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.655+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v126'1047 uv1047 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.655+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.660+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'1048 uv1048 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.660+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.664+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'1049 uv1049 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.664+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.669+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'1050 uv1050 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.669+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.674+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'1051 uv1051 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.674+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.679+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v126'1052 uv1026 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.679+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.683+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v126'1053 uv1053 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.683+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.676 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:40:57.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.688+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v126'1054 uv1054 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.688+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.693+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'1055 uv1055 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.693+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.697+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'1056 uv1056 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.698+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.702+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'1057 uv1057 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.703+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.708+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v126'1058 uv1032 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.708+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.713+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v126'1059 uv1059 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.713+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.706 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:40:57.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.718+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v126'1060 uv1060 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.718+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.723+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'1061 uv1061 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.724+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.728+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'1062 uv1062 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.728+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6ae9770 2024-04-25T11:40:57.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.733+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'1063 uv1063 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.733+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561ff6b99e00 0x561ff6b9cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.734+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b99a20 con 0x561ff6b99e00 2024-04-25T11:40:57.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.734+0000 7f45c1000640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561ff6b99e00 0x561ff6b9cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.736+0000 7f45c1000640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561ff6b99e00 0x561ff6b9cf50 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.741+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'752 uv731 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.741+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b9d490 con 0x561ff6b99e00 2024-04-25T11:40:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.745+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'753 uv753 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.745+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9d830 con 0x561ff6b99e00 2024-04-25T11:40:57.736 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:40:57.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.749+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'754 uv754 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.749+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9dbd0 con 0x561ff6b99e00 2024-04-25T11:40:57.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.753+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v126'755 uv755 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.753+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9df70 con 0x561ff6b99e00 2024-04-25T11:40:57.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.757+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'756 uv756 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.757+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9e310 con 0x561ff6b99e00 2024-04-25T11:40:57.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.761+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v126'757 uv750 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.761+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b9e6b0 con 0x561ff6b99e00 2024-04-25T11:40:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.765+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v126'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.765+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.756 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:40:57.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.769+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v126'759 uv759 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.769+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.774+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.774+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.778+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.778+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.783+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v126'762 uv751 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.783+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.787+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v126'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.788+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.779 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:40:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.792+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v126'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.792+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba3140 con 0x561ff6b99e00 2024-04-25T11:40:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.797+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v126'765 uv765 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.797+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba3140 con 0x561ff6b99e00 2024-04-25T11:40:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.802+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'766 uv766 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.802+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.806+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v126'767 uv747 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.806+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.811+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v126'768 uv768 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.811+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.803 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:40:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.815+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v126'769 uv769 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.815+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9fa60 con 0x561ff6b99e00 2024-04-25T11:40:57.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.819+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'770 uv770 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.819+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9fa60 con 0x561ff6b99e00 2024-04-25T11:40:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.824+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'771 uv771 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.824+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6b9ea50 con 0x561ff6b99e00 2024-04-25T11:40:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.828+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'772 uv772 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6b99e00 2024-04-25T11:40:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.828+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561ff6b9ef60 0x561ff6ba1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.828+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba1900 con 0x561ff6b9ef60 2024-04-25T11:40:57.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.829+0000 7f45bbe00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561ff6b9ef60 0x561ff6ba1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.831+0000 7f45bbe00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561ff6b9ef60 0x561ff6ba1380 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.835+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:57.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.835+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6ba2150 con 0x561ff6b9ef60 2024-04-25T11:40:57.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.840+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v126'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:57.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.840+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba24f0 con 0x561ff6b9ef60 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.844+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v126'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.844+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba38a0 con 0x561ff6b9ef60 2024-04-25T11:40:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.848+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.848+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba3d00 con 0x561ff6b9ef60 2024-04-25T11:40:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.852+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.853+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba2890 con 0x561ff6b9ef60 2024-04-25T11:40:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.856+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.857+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba2cc0 con 0x561ff6ae9770 2024-04-25T11:40:57.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.861+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'151 uv150 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.861+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6ba2c80 con 0x561ff6ae9770 2024-04-25T11:40:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.865+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'152 uv152 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.865+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba2c80 con 0x561ff6ae9770 2024-04-25T11:40:57.857 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:40:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.869+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.870+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba2c80 con 0x561ff6ae9770 2024-04-25T11:40:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.874+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.874+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561ff6ba40a0 0x561ff6ba64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.874+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba6a00 con 0x561ff6ba40a0 2024-04-25T11:40:57.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.875+0000 7f45c1a00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561ff6ba40a0 0x561ff6ba64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.877+0000 7f45c1a00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561ff6ba40a0 0x561ff6ba64c0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.881+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:57.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.881+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6ba6a00 con 0x561ff6ba40a0 2024-04-25T11:40:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.885+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v126'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.886+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba6a00 con 0x561ff6ba40a0 2024-04-25T11:40:57.877 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:40:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.890+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v126'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.890+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba7a10 con 0x561ff6ba40a0 2024-04-25T11:40:57.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.894+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:57.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.894+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba7a10 con 0x561ff6ba40a0 2024-04-25T11:40:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.898+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.898+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba6a00 con 0x561ff6ba40a0 2024-04-25T11:40:57.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.902+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.903+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561ff6ba7080 0x561ff6ba94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.903+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba9c00 con 0x561ff6ba7080 2024-04-25T11:40:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.904+0000 7f45c1000640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561ff6ba7080 0x561ff6ba94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.905+0000 7f45c1000640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561ff6ba7080 0x561ff6ba94e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.909+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v126'371 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.910+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.914+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v126'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.914+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.906 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:40:57.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.918+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v126'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.918+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6baafe0 con 0x561ff6ba7080 2024-04-25T11:40:57.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.922+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.923+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6baafe0 con 0x561ff6ba7080 2024-04-25T11:40:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.927+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.927+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.931+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.931+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.935+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v126'377 uv370 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.936+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.940+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v126'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.940+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.944+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v126'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.932 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:40:57.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.944+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.949+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.949+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6ba9fd0 con 0x561ff6ba7080 2024-04-25T11:40:57.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.954+0000 7f45c1000640 1 -- 172.21.15.19:0/309318060 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45b4051070 con 0x561ff6ba7080 2024-04-25T11:40:57.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.954+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561ff6baa8d0 0x561ff6baccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.954+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6bad2c0 con 0x561ff6baa8d0 2024-04-25T11:40:57.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.955+0000 7f45bbe00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561ff6baa8d0 0x561ff6baccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.956+0000 7f45bbe00640 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561ff6baa8d0 0x561ff6baccf0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.961+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v126'168 uv167 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.961+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6bad690 con 0x561ff6baa8d0 2024-04-25T11:40:57.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.966+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v126'169 uv169 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:57.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.966+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6baa8d0 2024-04-25T11:40:57.957 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:40:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.970+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v126'170 uv170 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:57.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.970+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6baa8d0 2024-04-25T11:40:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.974+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'171 uv171 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.974+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6baa8d0 2024-04-25T11:40:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.978+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'172 uv172 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.979+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6ae9770 2024-04-25T11:40:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.983+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.984+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6ae9770 2024-04-25T11:40:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.988+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v126'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.989+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6ae9770 2024-04-25T11:40:57.980 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:40:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.993+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v126'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.993+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6baeac0 con 0x561ff6ae9770 2024-04-25T11:40:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.997+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:57.998+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6baeac0 con 0x561ff6ae9770 2024-04-25T11:40:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.002+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.002+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6ae9770 2024-04-25T11:40:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.007+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ae9770 2024-04-25T11:40:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.007+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.011+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v126'335 uv329 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.011+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.015+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v126'336 uv336 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.016+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.007 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:40:58.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.020+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v126'337 uv337 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.020+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.025+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'338 uv338 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.025+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.028+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'339 uv339 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.029+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.033+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v126'340 uv334 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.033+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.037+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v126'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.038+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.029 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:40:58.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.042+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v126'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.042+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badab0 con 0x561ff6b9ef60 2024-04-25T11:40:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.046+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.046+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6bb08c0 con 0x561ff6b9ef60 2024-04-25T11:40:58.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.052+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6b9ef60 2024-04-25T11:40:58.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.052+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.056+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v126'147 uv145 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.056+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.060+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'148 uv148 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.060+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.051 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:40:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.064+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v126'149 uv149 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.064+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.068+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'150 uv150 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.068+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.072+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v126'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.072+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.076+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v126'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.076+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.080+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v126'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.068 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:40:58.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.080+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.084+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.084+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.088+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.088+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6ba40a0 2024-04-25T11:40:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.092+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45bc00c510 con 0x561ff6ba40a0 2024-04-25T11:40:58.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.093+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.097+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v126'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.097+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.101+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v126'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.101+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.092 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:40:58.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.104+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v126'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.105+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.109+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.109+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.113+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.114+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.118+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v126'416 uv406 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.118+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.123+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v126'417 uv417 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.123+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.115 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:40:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.127+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v126'418 uv418 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.127+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.132+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v126'419 uv419 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.132+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.136+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'420 uv420 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.136+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.141+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v126'421 uv414 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.141+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.145+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v126'422 uv422 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.145+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.137 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:40:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.150+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v126'423 uv423 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.150+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.154+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'424 uv424 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.154+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.159+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'425 uv425 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.159+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6badf60 con 0x561ff6baa8d0 2024-04-25T11:40:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.163+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'426 uv426 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.163+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x561ff6bae800 con 0x561ff6baa8d0 2024-04-25T11:40:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45bbe00640 1 -- 172.21.15.19:0/309318060 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'427 uv427 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f45ac009920 con 0x561ff6baa8d0 2024-04-25T11:40:58.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561ff6b99e00 msgr2=0x561ff6b9cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x561ff6b99e00 0x561ff6b9cf50 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561ff6ba7080 msgr2=0x561ff6ba94e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x561ff6ba7080 0x561ff6ba94e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561ff6baa8d0 msgr2=0x561ff6baccf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x561ff6baa8d0 0x561ff6baccf0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561ff6b9ef60 msgr2=0x561ff6ba1380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x561ff6b9ef60 0x561ff6ba1380 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561ff6ae9770 msgr2=0x561ff6b901f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c1a00640 1 -- 172.21.15.19:0/309318060 reap_dead start 2024-04-25T11:40:58.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561ff6ae9770 0x561ff6b901f0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561ff6ba40a0 msgr2=0x561ff6ba64c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561ff6ba40a0 0x561ff6ba64c0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.168+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45a0072e90 msgr2=0x7f45a00753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45a0072e90 0x7f45a00753a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x561ff6af5cb0 tx=0x7f45ac005b90 comp rx=0 tx=0).stop 2024-04-25T11:40:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 msgr2=0x561ff6a4f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6a4f860 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005570 tx=0x7f45b4002fb0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 shutdown_connections 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x561ff6ba40a0 0x561ff6ba64c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x561ff6ae9770 0x561ff6b901f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45a0072e90 0x7f45a00753a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 --2- 172.21.15.19:0/309318060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6a1f1e0 0x561ff6a4f860 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 >> 172.21.15.19:0/309318060 conn(0x561ff6941ac0 msgr2=0x561ff6af4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.169+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 shutdown_connections 2024-04-25T11:40:58.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.170+0000 7f45c7e17f40 1 -- 172.21.15.19:0/309318060 wait complete. 2024-04-25T11:40:58.186 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:40:58.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.436+0000 7f6115a0cf40 1 Processor -- start 2024-04-25T11:40:58.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.436+0000 7f6115a0cf40 1 -- start start 2024-04-25T11:40:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f6115a0cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f40022f0 0x55d5629df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f6115a0cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562a7e0e0 0x55d5629deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f6115a0cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d562b16a60 0x55d562b16e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f6115a0cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d562954d20 con 0x55d562a7e0e0 2024-04-25T11:40:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f6115a0cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d562958d60 con 0x7f60f40022f0 2024-04-25T11:40:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f6115a0cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d5629f73f0 con 0x55d562b16a60 2024-04-25T11:40:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f610d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562a7e0e0 0x55d5629deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f610d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562a7e0e0 0x55d5629deed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48354/0 (socket says 172.21.15.19:48354) 2024-04-25T11:40:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f610d400640 1 -- 172.21.15.19:0/3809462058 learned_addr learned my addr 172.21.15.19:0/3809462058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f610e800640 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d562b16a60 0x55d562b16e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.438+0000 7f610de00640 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f40022f0 0x55d5629df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f610d400640 1 -- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d562b16a60 msgr2=0x55d562b16e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f610d400640 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d562b16a60 0x55d562b16e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f610d400640 1 -- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f40022f0 msgr2=0x55d5629df6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f610d400640 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f40022f0 0x55d5629df6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f610d400640 1 -- 172.21.15.19:0/3809462058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d562a7e7f0 con 0x55d562a7e0e0 2024-04-25T11:40:58.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f610e800640 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d562b16a60 0x55d562b16e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f610d400640 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562a7e0e0 0x55d5629deed0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b560 tx=0x7f60f800ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a16a812c28ebb51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f6107e00640 1 -- 172.21.15.19:0/3809462058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8008070 con 0x55d562a7e0e0 2024-04-25T11:40:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f6107e00640 1 -- 172.21.15.19:0/3809462058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f800d7c0 con 0x55d562a7e0e0 2024-04-25T11:40:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.439+0000 7f6107e00640 1 -- 172.21.15.19:0/3809462058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f800dbf0 con 0x55d562a7e0e0 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 -- 172.21.15.19:0/3809462058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562a7e0e0 msgr2=0x55d5629deed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562a7e0e0 0x55d5629deed0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b560 tx=0x7f60f800ba20 comp rx=0 tx=0).stop 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 -- 172.21.15.19:0/3809462058 shutdown_connections 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d562b16a60 0x55d562b16e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562a7e0e0 0x55d5629deed0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/3809462058 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60f40022f0 0x55d5629df6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 -- 172.21.15.19:0/3809462058 >> 172.21.15.19:0/3809462058 conn(0x55d5628d0ac0 msgr2=0x55d5629e0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 -- 172.21.15.19:0/3809462058 shutdown_connections 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.440+0000 7f6115a0cf40 1 -- 172.21.15.19:0/3809462058 wait complete. 2024-04-25T11:40:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.442+0000 7f6115a0cf40 1 Processor -- start 2024-04-25T11:40:58.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.442+0000 7f6115a0cf40 1 -- start start 2024-04-25T11:40:58.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f6115a0cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d562a7e0e0 0x55d562a75380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f6115a0cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562b16a60 0x55d5629deed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f6115a0cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f40022f0 0x55d5629df410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f6115a0cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d562a865c0 con 0x55d562b16a60 2024-04-25T11:40:58.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f6115a0cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d562a86810 con 0x55d562a7e0e0 2024-04-25T11:40:58.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f6115a0cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d562b14550 con 0x7f60f40022f0 2024-04-25T11:40:58.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f610d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562b16a60 0x55d5629deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f610d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562b16a60 0x55d5629deed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48368/0 (socket says 172.21.15.19:48368) 2024-04-25T11:40:58.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f610d400640 1 -- 172.21.15.19:0/887309787 learned_addr learned my addr 172.21.15.19:0/887309787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.444+0000 7f610de00640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d562a7e0e0 0x55d562a75380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610e800640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f40022f0 0x55d5629df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610d400640 1 -- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f40022f0 msgr2=0x55d5629df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610d400640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f40022f0 0x55d5629df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610d400640 1 -- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d562a7e0e0 msgr2=0x55d562a75380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610d400640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d562a7e0e0 0x55d562a75380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610d400640 1 -- 172.21.15.19:0/887309787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d562a758c0 con 0x55d562b16a60 2024-04-25T11:40:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610e800640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60f40022f0 0x55d5629df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f610d400640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562b16a60 0x55d5629deed0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f60f8010000 tx=0x7f60f8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f6106a00640 1 -- 172.21.15.19:0/887309787 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f800fd70 con 0x55d562b16a60 2024-04-25T11:40:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f6106a00640 1 -- 172.21.15.19:0/887309787 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f8019460 con 0x55d562b16a60 2024-04-25T11:40:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f6106a00640 1 -- 172.21.15.19:0/887309787 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8019850 con 0x55d562b16a60 2024-04-25T11:40:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.445+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d562a79a40 con 0x55d562b16a60 2024-04-25T11:40:58.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.446+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5629f4350 con 0x55d562b16a60 2024-04-25T11:40:58.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.448+0000 7f6106a00640 1 -- 172.21.15.19:0/887309787 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f60f80199f0 con 0x55d562b16a60 2024-04-25T11:40:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.449+0000 7f6106a00640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60ec072f10 0x7f60ec075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.450+0000 7f6106a00640 1 -- 172.21.15.19:0/887309787 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f60f8097350 con 0x55d562b16a60 2024-04-25T11:40:58.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.450+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d562b13a70 0x55d562b28430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.450+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55d562b13e80 con 0x55d562b13a70 2024-04-25T11:40:58.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.450+0000 7f610de00640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60ec072f10 0x7f60ec075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.451+0000 7f610e800640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d562b13a70 0x55d562b28430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.452+0000 7f610de00640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60ec072f10 0x7f60ec075420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6100058000 tx=0x7f6100053000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:58.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.452+0000 7f610e800640 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d562b13a70 0x55d562b28430 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f610e800640 1 -- 172.21.15.19:0/887309787 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v126'1064 uv1064 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f610800f3a0 con 0x55d562b13a70 2024-04-25T11:40:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d562b13a70 msgr2=0x55d562b28430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55d562b13a70 0x55d562b28430 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60ec072f10 msgr2=0x7f60ec075420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f60ec072f10 0x7f60ec075420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6100058000 tx=0x7f6100053000 comp rx=0 tx=0).stop 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562b16a60 msgr2=0x55d5629deed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 --2- 172.21.15.19:0/887309787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d562b16a60 0x55d5629deed0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f60f8010000 tx=0x7f60f8000ed0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f610e800640 1 -- 172.21.15.19:0/887309787 reap_dead start 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 shutdown_connections 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 >> 172.21.15.19:0/887309787 conn(0x55d5628d0ac0 msgr2=0x55d562a86c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:58.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.458+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 shutdown_connections 2024-04-25T11:40:58.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.459+0000 7f6115a0cf40 1 -- 172.21.15.19:0/887309787 wait complete. 2024-04-25T11:40:58.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:40:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.755+0000 7f4543000640 1 Processor -- start 2024-04-25T11:40:58.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.755+0000 7f4543000640 1 -- start start 2024-04-25T11:40:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.758+0000 7f4543000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f453c160390 0x7f453c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.758+0000 7f4543000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.758+0000 7f4543000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f453c06df20 con 0x7f453c0af710 2024-04-25T11:40:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.758+0000 7f4543000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f453c0afb20 con 0x7f453c160390 2024-04-25T11:40:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.758+0000 7f4543000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f453c06f440 con 0x7f453c06db10 2024-04-25T11:40:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.759+0000 7f4541000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.759+0000 7f453be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f453c0af710 0x7f453c164a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:46724/0 (socket says 172.21.15.19:46724) 2024-04-25T11:40:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.759+0000 7f453be00640 1 -- 172.21.15.19:0/2234335095 learned_addr learned my addr 172.21.15.19:0/2234335095 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:40:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.759+0000 7f4533e00640 1 --2- 172.21.15.19:0/2234335095 >> v2:172.21.15.169:3300/0 conn(0x7f453c160390 0x7f453c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.760+0000 7f453b400640 1 -- 172.21.15.19:0/2234335095 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981545422 0 0) 0x7f452c001110 con 0x7f453c0af710 2024-04-25T11:40:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.760+0000 7f453b400640 1 -- 172.21.15.19:0/2234335095 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4520003520 con 0x7f453c0af710 2024-04-25T11:40:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.760+0000 7f4541000640 1 -- 172.21.15.19:0/2234335095 >> v2:172.21.15.169:3300/0 conn(0x7f453c160390 msgr2=0x7f453c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.760+0000 7f4541000640 1 --2- 172.21.15.19:0/2234335095 >> v2:172.21.15.169:3300/0 conn(0x7f453c160390 0x7f453c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.760+0000 7f4541000640 1 -- 172.21.15.19:0/2234335095 >> v1:172.21.15.169:6789/0 conn(0x7f453c0af710 legacy=0x7f453c164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.760+0000 7f4541000640 1 -- 172.21.15.19:0/2234335095 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c06f720 con 0x7f453c06db10 2024-04-25T11:40:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.761+0000 7f4541000640 1 --2- 172.21.15.19:0/2234335095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c070270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4534009d90 tx=0x7f453402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=525ad7734a83bcf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.762+0000 7f453b400640 1 -- 172.21.15.19:0/2234335095 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534007740 con 0x7f453c06db10 2024-04-25T11:40:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.762+0000 7f453b400640 1 -- 172.21.15.19:0/2234335095 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45340078e0 con 0x7f453c06db10 2024-04-25T11:40:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.762+0000 7f453b400640 1 -- 172.21.15.19:0/2234335095 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534007cb0 con 0x7f453c06db10 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.763+0000 7f4543000640 1 -- 172.21.15.19:0/2234335095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 msgr2=0x7f453c070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.763+0000 7f4543000640 1 --2- 172.21.15.19:0/2234335095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c070270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4534009d90 tx=0x7f453402a660 comp rx=0 tx=0).stop 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.763+0000 7f4543000640 1 -- 172.21.15.19:0/2234335095 shutdown_connections 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.764+0000 7f4543000640 1 --2- 172.21.15.19:0/2234335095 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c070270 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.764+0000 7f4543000640 1 --2- 172.21.15.19:0/2234335095 >> v2:172.21.15.169:3300/0 conn(0x7f453c160390 0x7f453c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.764+0000 7f4543000640 1 -- 172.21.15.19:0/2234335095 >> 172.21.15.19:0/2234335095 conn(0x7f453c073090 msgr2=0x7f453c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.764+0000 7f4543000640 1 -- 172.21.15.19:0/2234335095 shutdown_connections 2024-04-25T11:40:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.764+0000 7f4543000640 1 -- 172.21.15.19:0/2234335095 wait complete. 2024-04-25T11:40:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.767+0000 7f4543000640 1 Processor -- start 2024-04-25T11:40:58.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.767+0000 7f4543000640 1 -- start start 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4543000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4543000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c0af710 0x7f453c1441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4543000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f453c160390 0x7f453c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4543000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f453c06ffc0 con 0x7f453c0af710 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4543000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f453c06fd20 con 0x7f453c160390 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4543000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f453c06f0d0 con 0x7f453c06db10 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4533e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c0af710 0x7f453c1441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4533e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c0af710 0x7f453c1441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48382/0 (socket says 172.21.15.19:48382) 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.769+0000 7f4533e00640 1 -- 172.21.15.19:0/1799450714 learned_addr learned my addr 172.21.15.19:0/1799450714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4541000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f453c160390 0x7f453c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f453be00640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4533e00640 1 -- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 msgr2=0x7f453c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4533e00640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c15c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4533e00640 1 -- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f453c160390 msgr2=0x7f453c157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4533e00640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f453c160390 0x7f453c157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4533e00640 1 -- 172.21.15.19:0/1799450714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c1fa550 con 0x7f453c0af710 2024-04-25T11:40:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4541000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f453c160390 0x7f453c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.770+0000 7f4533e00640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c0af710 0x7f453c1441f0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f452400e870 tx=0x7f452400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.771+0000 7f4538c00640 1 -- 172.21.15.19:0/1799450714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524017070 con 0x7f453c0af710 2024-04-25T11:40:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.771+0000 7f4538c00640 1 -- 172.21.15.19:0/1799450714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452400c940 con 0x7f453c0af710 2024-04-25T11:40:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.771+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c1fa830 con 0x7f453c0af710 2024-04-25T11:40:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.771+0000 7f4538c00640 1 -- 172.21.15.19:0/1799450714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452400cd30 con 0x7f453c0af710 2024-04-25T11:40:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.771+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c1fadf0 con 0x7f453c0af710 2024-04-25T11:40:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.772+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453c050740 con 0x7f453c0af710 2024-04-25T11:40:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.773+0000 7f4538c00640 1 -- 172.21.15.19:0/1799450714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f452401a070 con 0x7f453c0af710 2024-04-25T11:40:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.775+0000 7f4538c00640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45180744b0 0x7f45180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.775+0000 7f4538c00640 1 -- 172.21.15.19:0/1799450714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f4524096080 con 0x7f453c0af710 2024-04-25T11:40:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.776+0000 7f453be00640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45180744b0 0x7f45180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.777+0000 7f453be00640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45180744b0 0x7f45180769c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f452c0012b0 tx=0x7f452c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:58.780+0000 7f4538c00640 1 -- 172.21.15.19:0/1799450714 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f452405fc80 con 0x7f453c0af710 2024-04-25T11:40:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.025+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f453c0b5d40 con 0x7f453c0af710 2024-04-25T11:40:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.190+0000 7f4538c00640 1 -- 172.21.15.19:0/1799450714 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v313) ==== 162+0+0 (secure 0 0 0) 0x7f452405f580 con 0x7f453c0af710 2024-04-25T11:40:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.193+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45180744b0 msgr2=0x7f45180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.193+0000 7f4543000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45180744b0 0x7f45180769c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f452c0012b0 tx=0x7f452c009040 comp rx=0 tx=0).stop 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c0af710 msgr2=0x7f453c1441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c0af710 0x7f453c1441f0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f452400e870 tx=0x7f452400ed30 comp rx=0 tx=0).stop 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 shutdown_connections 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f45180744b0 0x7f45180769c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f453c160390 0x7f453c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f453c0af710 0x7f453c1441f0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 --2- 172.21.15.19:0/1799450714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f453c06db10 0x7f453c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 >> 172.21.15.19:0/1799450714 conn(0x7f453c073090 msgr2=0x7f453c0cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 shutdown_connections 2024-04-25T11:40:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.194+0000 7f4543000640 1 -- 172.21.15.19:0/1799450714 wait complete. 2024-04-25T11:40:59.614 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:40:59.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.909+0000 7f8f82800640 1 Processor -- start 2024-04-25T11:40:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.909+0000 7f8f82800640 1 -- start start 2024-04-25T11:40:59.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.911+0000 7f8f82800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:59.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.911+0000 7f8f82800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f7c08cd60 0x7f8f7c08d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:59.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.911+0000 7f8f82800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c0b5290 con 0x7f8f7c0b4e80 2024-04-25T11:40:59.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.911+0000 7f8f82800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7c0d1d70 con 0x7f8f7c08cd60 2024-04-25T11:40:59.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.911+0000 7f8f82800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c0ce6d0 con 0x7f8f7c06e680 2024-04-25T11:40:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f7c08cd60 0x7f8f7c08d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8f7c08cd60 0x7f8f7c08d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48394/0 (socket says 172.21.15.19:48394) 2024-04-25T11:40:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81e00640 1 -- 172.21.15.19:0/3585145962 learned_addr learned my addr 172.21.15.19:0/3585145962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81400640 1 --2- 172.21.15.19:0/3585145962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81e00640 1 -- 172.21.15.19:0/3585145962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 msgr2=0x7f8f7c0b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:59.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81e00640 1 --2- 172.21.15.19:0/3585145962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81e00640 1 -- 172.21.15.19:0/3585145962 >> v1:172.21.15.169:6789/0 conn(0x7f8f7c0b4e80 legacy=0x7f8f7c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:40:59.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.912+0000 7f8f81e00640 1 -- 172.21.15.19:0/3585145962 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c08d6b0 con 0x7f8f7c08cd60 2024-04-25T11:40:59.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.914+0000 7f8f81e00640 1 --2- 172.21.15.19:0/3585145962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c08d170 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009e50 tx=0x7f8f7402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4830ced1d81b972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:59.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.914+0000 7f8f7be00640 1 -- 172.21.15.19:0/3585145962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74036070 con 0x7f8f7c08cd60 2024-04-25T11:40:59.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.914+0000 7f8f7be00640 1 -- 172.21.15.19:0/3585145962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f74031600 con 0x7f8f7c08cd60 2024-04-25T11:40:59.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.914+0000 7f8f7be00640 1 -- 172.21.15.19:0/3585145962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f740319d0 con 0x7f8f7c08cd60 2024-04-25T11:40:59.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 -- 172.21.15.19:0/3585145962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 msgr2=0x7f8f7c08d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 --2- 172.21.15.19:0/3585145962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c08d170 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009e50 tx=0x7f8f7402a630 comp rx=0 tx=0).stop 2024-04-25T11:40:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 -- 172.21.15.19:0/3585145962 shutdown_connections 2024-04-25T11:40:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 --2- 172.21.15.19:0/3585145962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c08d170 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 --2- 172.21.15.19:0/3585145962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 -- 172.21.15.19:0/3585145962 >> 172.21.15.19:0/3585145962 conn(0x7f8f7c0b5460 msgr2=0x7f8f7c0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 -- 172.21.15.19:0/3585145962 shutdown_connections 2024-04-25T11:40:59.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.915+0000 7f8f82800640 1 -- 172.21.15.19:0/3585145962 wait complete. 2024-04-25T11:40:59.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.917+0000 7f8f82800640 1 Processor -- start 2024-04-25T11:40:59.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.918+0000 7f8f82800640 1 -- start start 2024-04-25T11:40:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f82800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f82800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c1f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f82800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c0b4e80 0x7f8f7c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f82800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d2560 con 0x7f8f7c08cd60 2024-04-25T11:40:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f82800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d2280 con 0x7f8f7c0b4e80 2024-04-25T11:40:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f82800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d2840 con 0x7f8f7c06e680 2024-04-25T11:40:59.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f80a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c1f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f80a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c1f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48408/0 (socket says 172.21.15.19:48408) 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.919+0000 7f8f80a00640 1 -- 172.21.15.19:0/2501309825 learned_addr learned my addr 172.21.15.19:0/2501309825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f81400640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f81e00640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c0b4e80 0x7f8f7c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f80a00640 1 -- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 msgr2=0x7f8f7c0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f80a00640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f80a00640 1 -- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c0b4e80 msgr2=0x7f8f7c1f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f80a00640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c0b4e80 0x7f8f7c1f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f80a00640 1 -- 172.21.15.19:0/2501309825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c162470 con 0x7f8f7c08cd60 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f81e00640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c0b4e80 0x7f8f7c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.920+0000 7f8f80a00640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c1f7930 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000b420 tx=0x7f8f7000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.921+0000 7f8f79600640 1 -- 172.21.15.19:0/2501309825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70011830 con 0x7f8f7c08cd60 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.921+0000 7f8f79600640 1 -- 172.21.15.19:0/2501309825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f70016070 con 0x7f8f7c08cd60 2024-04-25T11:40:59.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.921+0000 7f8f79600640 1 -- 172.21.15.19:0/2501309825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70011c10 con 0x7f8f7c08cd60 2024-04-25T11:40:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.921+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c162750 con 0x7f8f7c08cd60 2024-04-25T11:40:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.921+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c163230 con 0x7f8f7c08cd60 2024-04-25T11:40:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.922+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c15cfa0 con 0x7f8f7c08cd60 2024-04-25T11:40:59.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.923+0000 7f8f79600640 1 -- 172.21.15.19:0/2501309825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f7001b460 con 0x7f8f7c08cd60 2024-04-25T11:40:59.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.924+0000 7f8f79600640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f600744e0 0x7f8f600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:59.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.924+0000 7f8f79600640 1 -- 172.21.15.19:0/2501309825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f8f70095870 con 0x7f8f7c08cd60 2024-04-25T11:40:59.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.925+0000 7f8f81400640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f600744e0 0x7f8f600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:59.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.926+0000 7f8f81400640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f600744e0 0x7f8f600769f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c074b10 tx=0x7f8f6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:40:59.929+0000 7f8f79600640 1 -- 172.21.15.19:0/2501309825 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f70020020 con 0x7f8f7c08cd60 2024-04-25T11:41:00.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.174+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f7c0b4b50 con 0x7f8f7c08cd60 2024-04-25T11:41:00.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.175+0000 7f8f79600640 1 -- 172.21.15.19:0/2501309825 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 313 v313) ==== 95+0+5190 (secure 0 0 0) 0x7f8f7c0b4b50 con 0x7f8f7c08cd60 2024-04-25T11:41:00.163 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:00.163 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":313,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":313,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:40:59.187869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10689},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10689":{"gid":10689,"name":"a","rank":0,"incarnation":313,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/2088909328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2088909328},{"type":"v1","addr":"172.21.15.19:6835","nonce":2088909328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 313 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f600744e0 msgr2=0x7f8f600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f600744e0 0x7f8f600769f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c074b10 tx=0x7f8f6c008040 comp rx=0 tx=0).stop 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 msgr2=0x7f8f7c1f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c1f7930 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000b420 tx=0x7f8f7000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 shutdown_connections 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8f600744e0 0x7f8f600769f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f7c0b4e80 0x7f8f7c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c08cd60 0x7f8f7c1f7930 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 --2- 172.21.15.19:0/2501309825 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f7c06e680 0x7f8f7c0b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 >> 172.21.15.19:0/2501309825 conn(0x7f8f7c0b5460 msgr2=0x7f8f7c0711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 shutdown_connections 2024-04-25T11:41:00.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:00.178+0000 7f8f82800640 1 -- 172.21.15.19:0/2501309825 wait complete. 2024-04-25T11:41:00.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 313, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:40:59.187869+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10689}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10689': {'gid': 10689, 'name': 'a', 'rank': 0, 'incarnation': 313, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/2088909328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2088909328}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2088909328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:41:00.193 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10689:up:replay 2024-04-25T11:41:00.193 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:41:03.355 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:03.368+0000 7fe3d5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:41:05.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.903+0000 7ffa19a00640 1 Processor -- start 2024-04-25T11:41:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.903+0000 7ffa19a00640 1 -- start start 2024-04-25T11:41:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.906+0000 7ffa19a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffa140b4ea0 0x7ffa140cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.906+0000 7ffa19a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 0x7ffa1415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.906+0000 7ffa19a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa140d1230 con 0x7ffa1406c5c0 2024-04-25T11:41:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.906+0000 7ffa19a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ffa140d06b0 con 0x7ffa140b4ea0 2024-04-25T11:41:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.906+0000 7ffa19a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa140d0990 con 0x7ffa14144370 2024-04-25T11:41:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.906+0000 7ffa13e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffa1406c5c0 0x7ffa140b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56740/0 (socket says 172.21.15.19:56740) 2024-04-25T11:41:05.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.906+0000 7ffa13e00640 1 -- 172.21.15.19:0/1837269216 learned_addr learned my addr 172.21.15.19:0/1837269216 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:05.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa19000640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 0x7ffa1415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa13400640 1 --2- 172.21.15.19:0/1837269216 >> v2:172.21.15.169:3300/0 conn(0x7ffa140b4ea0 0x7ffa140cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694551195 0 0) 0x7ffa140d1230 con 0x7ffa1406c5c0 2024-04-25T11:41:05.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9f8003520 con 0x7ffa1406c5c0 2024-04-25T11:41:05.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2941571500 0 0) 0x7ff9f8003520 con 0x7ffa1406c5c0 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 msgr2=0x7ffa1415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 0x7ffa1415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 >> v2:172.21.15.169:3300/0 conn(0x7ffa140b4ea0 msgr2=0x7ffa140cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 --2- 172.21.15.19:0/1837269216 >> v2:172.21.15.169:3300/0 conn(0x7ffa140b4ea0 0x7ffa140cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa140d0c70 con 0x7ffa1406c5c0 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ffa04001670 con 0x7ffa1406c5c0 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.907+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 >> v1:172.21.15.19:6789/0 conn(0x7ffa1406c5c0 legacy=0x7ffa140b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa12a00640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 0x7ff9f80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa12a00640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f8003f80 0x7ff9f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa12a00640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f8004d40 0x7ff9f8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140d06b0 con 0x7ff9f8004d40 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140d0990 con 0x7ff9f8003f80 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f800ba70 con 0x7ffa14144370 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa19000640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f8004d40 0x7ff9f8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa13e00640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 0x7ff9f80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa13400640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f8003f80 0x7ff9f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa19000640 1 -- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 msgr2=0x7ff9f80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:05.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa19000640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffa14144370 0x7ff9f80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa19000640 1 -- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f8003f80 msgr2=0x7ff9f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:05.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa19000640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff9f8003f80 0x7ff9f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.908+0000 7ffa19000640 1 -- 172.21.15.19:0/1837269216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8003520 con 0x7ff9f8004d40 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.910+0000 7ffa19000640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f8004d40 0x7ff9f8005150 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009fc0 tx=0x7ff9f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc599e92ee2c23a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.910+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c034020 con 0x7ff9f8004d40 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.910+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c0305f0 con 0x7ff9f8004d40 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.910+0000 7ffa12a00640 1 -- 172.21.15.19:0/1837269216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c009c10 con 0x7ff9f8004d40 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.911+0000 7ffa19a00640 1 -- 172.21.15.19:0/1837269216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f8004d40 msgr2=0x7ff9f8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.911+0000 7ffa19a00640 1 --2- 172.21.15.19:0/1837269216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9f8004d40 0x7ff9f8005150 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009fc0 tx=0x7ff9f8005830 comp rx=0 tx=0).stop 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.911+0000 7ffa19000640 1 -- 172.21.15.19:0/1837269216 reap_dead start 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.911+0000 7ffa19a00640 1 -- 172.21.15.19:0/1837269216 shutdown_connections 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.911+0000 7ffa19a00640 1 -- 172.21.15.19:0/1837269216 >> 172.21.15.19:0/1837269216 conn(0x7ffa140d1720 msgr2=0x7ffa140d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.911+0000 7ffa19a00640 1 -- 172.21.15.19:0/1837269216 shutdown_connections 2024-04-25T11:41:05.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.911+0000 7ffa19a00640 1 -- 172.21.15.19:0/1837269216 wait complete. 2024-04-25T11:41:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.913+0000 7ffa19a00640 1 Processor -- start 2024-04-25T11:41:05.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.913+0000 7ffa19a00640 1 -- start start 2024-04-25T11:41:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8004d40 0x7ffa141388d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa14138e10 0x7ffa141f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa141fa010 0x7ffa141fa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140d0450 con 0x7ffa141fa010 2024-04-25T11:41:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffa140d02d0 con 0x7ffa14138e10 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ffa14147760 con 0x7ff9f8004d40 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa141fa010 0x7ffa141fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa141fa010 0x7ffa141fa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38102/0 (socket says 172.21.15.19:38102) 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.915+0000 7ffa19000640 1 -- 172.21.15.19:0/1154980710 learned_addr learned my addr 172.21.15.19:0/1154980710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa13e00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8004d40 0x7ffa141388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa13400640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa14138e10 0x7ffa141f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa19000640 1 -- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8004d40 msgr2=0x7ffa141388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa19000640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8004d40 0x7ffa141388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa19000640 1 -- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa14138e10 msgr2=0x7ffa141f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa19000640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa14138e10 0x7ffa141f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa19000640 1 -- 172.21.15.19:0/1154980710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa14139220 con 0x7ffa141fa010 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa13400640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa14138e10 0x7ffa141f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa19000640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa141fa010 0x7ffa141fa420 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c033040 tx=0x7ffa0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa0be00640 1 -- 172.21.15.19:0/1154980710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c040070 con 0x7ffa141fa010 2024-04-25T11:41:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa0be00640 1 -- 172.21.15.19:0/1154980710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c003990 con 0x7ffa141fa010 2024-04-25T11:41:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa0be00640 1 -- 172.21.15.19:0/1154980710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c034040 con 0x7ffa141fa010 2024-04-25T11:41:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.916+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa140cacb0 con 0x7ffa141fa010 2024-04-25T11:41:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.917+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa140cb270 con 0x7ffa141fa010 2024-04-25T11:41:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.918+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa14160c00 con 0x7ffa141fa010 2024-04-25T11:41:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.919+0000 7ffa0be00640 1 -- 172.21.15.19:0/1154980710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffa0c037020 con 0x7ffa141fa010 2024-04-25T11:41:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.921+0000 7ffa0be00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9f00744b0 0x7ff9f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.921+0000 7ffa0be00640 1 -- 172.21.15.19:0/1154980710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7ffa0c0b5fe0 con 0x7ffa141fa010 2024-04-25T11:41:05.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.921+0000 7ffa13e00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9f00744b0 0x7ff9f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.923+0000 7ffa13e00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9f00744b0 0x7ff9f00769c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ffa04001eb0 tx=0x7ffa04000f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:05.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:05.926+0000 7ffa0be00640 1 -- 172.21.15.19:0/1154980710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa0c07fc70 con 0x7ffa141fa010 2024-04-25T11:41:06.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.177+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa14161250 con 0x7ffa141fa010 2024-04-25T11:41:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.178+0000 7ffa0be00640 1 -- 172.21.15.19:0/1154980710 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 316 v316) ==== 95+0+5199 (secure 0 0 0) 0x7ffa0c07f570 con 0x7ffa141fa010 2024-04-25T11:41:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:06.166 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":316,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":316,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:02.208011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10689},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10689":{"gid":10689,"name":"a","rank":0,"incarnation":313,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/2088909328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2088909328},{"type":"v1","addr":"172.21.15.19:6835","nonce":2088909328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10689,"qdb_cluster":[10689]},"id":6}]} 2024-04-25T11:41:06.166 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 316 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9f00744b0 msgr2=0x7ff9f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9f00744b0 0x7ff9f00769c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ffa04001eb0 tx=0x7ffa04000f50 comp rx=0 tx=0).stop 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa141fa010 msgr2=0x7ffa141fa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa141fa010 0x7ffa141fa420 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c033040 tx=0x7ffa0c009750 comp rx=0 tx=0).stop 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 shutdown_connections 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff9f00744b0 0x7ff9f00769c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffa14138e10 0x7ffa141f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa141fa010 0x7ffa141fa420 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 --2- 172.21.15.19:0/1154980710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff9f8004d40 0x7ffa141388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 >> 172.21.15.19:0/1154980710 conn(0x7ffa140d1720 msgr2=0x7ffa1406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 shutdown_connections 2024-04-25T11:41:06.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.181+0000 7ffa19a00640 1 -- 172.21.15.19:0/1154980710 wait complete. 2024-04-25T11:41:06.196 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 316, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:41:02.208011+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10689}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10689': {'gid': 10689, 'name': 'a', 'rank': 0, 'incarnation': 313, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/2088909328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2088909328}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2088909328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10689, 'qdb_cluster': [10689]} 2024-04-25T11:41:06.196 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:41:06.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:41:06.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.495+0000 7f870be00640 1 Processor -- start 2024-04-25T11:41:06.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.495+0000 7f870be00640 1 -- start start 2024-04-25T11:41:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.497+0000 7f870be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f870c0b4ea0 0x7f870c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.497+0000 7f870be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870c0d1230 con 0x7f870c144370 2024-04-25T11:41:06.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.497+0000 7f870be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870c0d06b0 con 0x7f870c06c5c0 2024-04-25T11:41:06.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.497+0000 7f870be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f870c0d0990 con 0x7f870c0b4ea0 2024-04-25T11:41:06.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f870c144370 0x7f870c15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56748/0 (socket says 172.21.15.19:56748) 2024-04-25T11:41:06.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870b400640 1 -- 172.21.15.19:0/392443587 learned_addr learned my addr 172.21.15.19:0/392443587 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:06.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f8703e00640 1 --2- 172.21.15.19:0/392443587 >> v2:172.21.15.169:3300/0 conn(0x7f870c0b4ea0 0x7f870c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1429752769 0 0) 0x7f870c0d1230 con 0x7f870c144370 2024-04-25T11:41:06.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86f0003520 con 0x7f870c144370 2024-04-25T11:41:06.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1272282326 0 0) 0x7f86f0003520 con 0x7f870c144370 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 >> v2:172.21.15.169:3300/0 conn(0x7f870c0b4ea0 msgr2=0x7f870c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 --2- 172.21.15.19:0/392443587 >> v2:172.21.15.169:3300/0 conn(0x7f870c0b4ea0 0x7f870c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 >> v1:172.21.15.169:6789/0 conn(0x7f870c06c5c0 legacy=0x7f870c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c0d0c70 con 0x7f870c144370 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.498+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f87040019f0 con 0x7f870c144370 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 >> v1:172.21.15.19:6789/0 conn(0x7f870c144370 legacy=0x7f870c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870a000640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f00039b0 0x7f86f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870a000640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f0004800 0x7f86f0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870a000640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f00051c0 0x7f86f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f870c0d0990 con 0x7f86f00039b0 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f86f000c160 con 0x7f86f00051c0 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f86f000c360 con 0x7f86f0004800 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870aa00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f00039b0 0x7f86f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f870b400640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f00051c0 0x7f86f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.499+0000 7f8703e00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f0004800 0x7f86f0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f870aa00640 1 -- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f0004800 msgr2=0x7f86f0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f870aa00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f0004800 0x7f86f0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f870aa00640 1 -- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f00051c0 msgr2=0x7f86f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f870aa00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f00051c0 0x7f86f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f870aa00640 1 -- 172.21.15.19:0/392443587 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f0003520 con 0x7f86f00039b0 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f8703e00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f0004800 0x7f86f0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f870b400640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86f00051c0 0x7f86f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:06.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.500+0000 7f870b400640 1 -- 172.21.15.19:0/392443587 reap_dead start 2024-04-25T11:41:06.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.501+0000 7f870aa00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f00039b0 0x7f86f0003e20 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f86fc001750 tx=0x7f86f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dea676cf8b00307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.501+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc001370 con 0x7f86f00039b0 2024-04-25T11:41:06.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.501+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86fc0052d0 con 0x7f86f00039b0 2024-04-25T11:41:06.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.501+0000 7f870a000640 1 -- 172.21.15.19:0/392443587 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc02ab00 con 0x7f86f00039b0 2024-04-25T11:41:06.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.502+0000 7f870be00640 1 -- 172.21.15.19:0/392443587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f00039b0 msgr2=0x7f86f0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.502+0000 7f870be00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f00039b0 0x7f86f0003e20 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f86fc001750 tx=0x7f86f0005830 comp rx=0 tx=0).stop 2024-04-25T11:41:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.502+0000 7f870be00640 1 -- 172.21.15.19:0/392443587 shutdown_connections 2024-04-25T11:41:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.502+0000 7f870be00640 1 --2- 172.21.15.19:0/392443587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86f00039b0 0x7f86f0003e20 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.502+0000 7f870be00640 1 -- 172.21.15.19:0/392443587 >> 172.21.15.19:0/392443587 conn(0x7f870c0d1720 msgr2=0x7f870c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.502+0000 7f870be00640 1 -- 172.21.15.19:0/392443587 shutdown_connections 2024-04-25T11:41:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.502+0000 7f870be00640 1 -- 172.21.15.19:0/392443587 wait complete. 2024-04-25T11:41:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.504+0000 7f870be00640 1 Processor -- start 2024-04-25T11:41:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.504+0000 7f870be00640 1 -- start start 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f870be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f00039b0 0x7f870c141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f870be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f870c14de00 0x7f870c14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f870be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f870c14d650 0x7f870c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f870be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f870c0d0450 con 0x7f870c14de00 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f870be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f870c0d02d0 con 0x7f870c14d650 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f870be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f870c147760 con 0x7f86f00039b0 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f8703e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f870c14de00 0x7f870c14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f8703e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f870c14de00 0x7f870c14d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38118/0 (socket says 172.21.15.19:38118) 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f8703e00640 1 -- 172.21.15.19:0/3193950100 learned_addr learned my addr 172.21.15.19:0/3193950100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.506+0000 7f870aa00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f00039b0 0x7f870c141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f870b400640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f870c14d650 0x7f870c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8703e00640 1 -- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f00039b0 msgr2=0x7f870c141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8703e00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f00039b0 0x7f870c141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8703e00640 1 -- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f870c14d650 msgr2=0x7f870c15f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8703e00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f870c14d650 0x7f870c15f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8703e00640 1 -- 172.21.15.19:0/3193950100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c14da60 con 0x7f870c14de00 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8703e00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f870c14de00 0x7f870c14d110 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f86f40107f0 tx=0x7f86f4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8702a00640 1 -- 172.21.15.19:0/3193950100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f400f040 con 0x7f870c14de00 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8702a00640 1 -- 172.21.15.19:0/3193950100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f400eb60 con 0x7f870c14de00 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c142260 con 0x7f870c14de00 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f8702a00640 1 -- 172.21.15.19:0/3193950100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f401c400 con 0x7f870c14de00 2024-04-25T11:41:06.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.507+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1427f0 con 0x7f870c14de00 2024-04-25T11:41:06.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.511+0000 7f8702a00640 1 -- 172.21.15.19:0/3193950100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f86f401c5a0 con 0x7f870c14de00 2024-04-25T11:41:06.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.512+0000 7f8702a00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86e80744e0 0x7f86e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.512+0000 7f8702a00640 1 -- 172.21.15.19:0/3193950100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f86f4096440 con 0x7f870c14de00 2024-04-25T11:41:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.512+0000 7f8718a10740 1 Processor -- start 2024-04-25T11:41:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.513+0000 7f8718a10740 1 -- start start 2024-04-25T11:41:06.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.513+0000 7f870aa00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86e80744e0 0x7f86e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a973ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f8718a10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9740480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f8718a10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e3a965e600 con 0x55e3a9337590 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e3a95fb100 con 0x55e3a9358270 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e3a965a5c0 con 0x55e3a932f3b0 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f870b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9740480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f870b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9740480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38126/0 (socket says 172.21.15.19:38126) 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.514+0000 7f870b400640 1 -- 172.21.15.19:0/825776023 learned_addr learned my addr 172.21.15.19:0/825776023 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870aa00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86e80744e0 0x7f86e80769f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f86fc032000 tx=0x7f86fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870aa00640 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f8703e00640 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a973ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870b400640 1 -- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 msgr2=0x55e3a973ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870b400640 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a973ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870b400640 1 -- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 msgr2=0x55e3a9725210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870b400640 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870b400640 1 -- 172.21.15.19:0/825776023 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a93702d0 con 0x55e3a9337590 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f8703e00640 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a973ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f870b400640 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9740480 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f870c0b7a50 tx=0x7f8704001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9279b72aa33847 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f86fb400640 1 -- 172.21.15.19:0/825776023 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704019070 con 0x55e3a9337590 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f86fb400640 1 -- 172.21.15.19:0/825776023 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87040079b0 con 0x55e3a9337590 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.515+0000 7f86fb400640 1 -- 172.21.15.19:0/825776023 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870400d070 con 0x55e3a9337590 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 -- 172.21.15.19:0/825776023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 msgr2=0x55e3a9740480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9740480 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f870c0b7a50 tx=0x7f8704001b20 comp rx=0 tx=0).stop 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 -- 172.21.15.19:0/825776023 shutdown_connections 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9740480 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a973ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 --2- 172.21.15.19:0/825776023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 -- 172.21.15.19:0/825776023 >> 172.21.15.19:0/825776023 conn(0x55e3a934c3c0 msgr2=0x55e3a96fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 -- 172.21.15.19:0/825776023 shutdown_connections 2024-04-25T11:41:06.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.516+0000 7f8718a10740 1 -- 172.21.15.19:0/825776023 wait complete. 2024-04-25T11:41:06.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.517+0000 7f8718a10740 1 -- start start 2024-04-25T11:41:06.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.518+0000 7f8718a10740 1 Processor -- start 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a932f3b0 0x55e3a9740b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8718a10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a973da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a973dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8718a10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e3a95e5f70 con 0x55e3a9337590 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e3a973b790 con 0x55e3a932f3b0 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e3a9716980 con 0x55e3a9358270 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8718a10740 10 client.0 _ll_register_callbacks cb 0x55e3a9654480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.519+0000 7f8703e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a973da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a973da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38138/0 (socket says 172.21.15.19:38138) 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 -- 172.21.15.19:0/2928727349 learned_addr learned my addr 172.21.15.19:0/2928727349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f870aa00640 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a932f3b0 0x55e3a9740b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 -- 172.21.15.19:0/2928727349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 msgr2=0x55e3a973dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a973dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 -- 172.21.15.19:0/2928727349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a932f3b0 msgr2=0x55e3a9740b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a932f3b0 0x55e3a9740b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 -- 172.21.15.19:0/2928727349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a943a6b0 con 0x55e3a9337590 2024-04-25T11:41:06.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8703e00640 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a973da70 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f86f405cbb0 tx=0x7f86f405f290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c3abe1418581a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f86efe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:41:06.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f86efe00640 1 -- 172.21.15.19:0/2928727349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f405eda0 con 0x55e3a9337590 2024-04-25T11:41:06.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f86efe00640 1 -- 172.21.15.19:0/2928727349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f86f405b2b0 con 0x55e3a9337590 2024-04-25T11:41:06.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.520+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87040079b0 con 0x55e3a9337590 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f86efe00640 1 -- 172.21.15.19:0/2928727349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=316) ==== 24+0+0 (secure 0 0 0) 0x7f86f405f450 con 0x55e3a9337590 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 10 client.10743 fetch_fsmap learned FSMap version 316 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=316,monmap=2+}) -- 0x55e3a973b4d0 con 0x55e3a9337590 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f86efe00640 1 -- 172.21.15.19:0/2928727349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87040079b0 con 0x55e3a9337590 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f86efe00640 1 -- 172.21.15.19:0/2928727349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 316) ==== 3494+0+0 (secure 0 0 0) 0x7f86f405b4b0 con 0x55e3a9337590 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 10 client.10743 fetch_fsmap finished waiting for FSMap version 316 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 10 client.10743 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:41:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 20 client.10743 populate_metadata read hostname 'smithi019' 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a9316e40 0x55e3a97528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 4 client.10743 mds_command: new command op to 10689 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 --> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3a966c210 con 0x55e3a9316e40 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.521+0000 7f870aa00640 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a9316e40 0x55e3a97528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.522+0000 7f86efe00640 10 client.10743 ms_handle_connect on v2:172.21.15.19:6834/2088909328 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.522+0000 7f870aa00640 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a9316e40 0x55e3a97528f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9468dcfd0f67bbe8 server_cookie=8af7946e5c87cd88 in_seq=0 out_seq=0 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f86efe00640 1 -- 172.21.15.19:0/2928727349 <== mds.0 v2:172.21.15.19:6834/2088909328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3a966c210 con 0x55e3a9316e40 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f86efe00640 10 client.10743 handle_command_reply: tid=0 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 client.10743 shutdown 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 msgr2=0x55e3a973da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a973da70 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f86f405cbb0 tx=0x7f86f405f290 comp rx=0 tx=0).stop 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 shutdown_connections 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a9316e40 0x55e3a97528f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a973dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a973da70 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 --2- 172.21.15.19:0/2928727349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a932f3b0 0x55e3a9740b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f86efe00640 0 client.10743 ms_handle_reset on v2:172.21.15.19:6834/2088909328 2024-04-25T11:41:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.523+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 >> 172.21.15.19:0/2928727349 conn(0x55e3a934c3c0 msgr2=0x55e3a96fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.524+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 shutdown_connections 2024-04-25T11:41:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.524+0000 7f8718a10740 1 -- 172.21.15.19:0/2928727349 wait complete. 2024-04-25T11:41:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.524+0000 7f8718a10740 20 client.10743 trim_cache size 0 max 16384 2024-04-25T11:41:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.545+0000 7f8718a10740 1 Processor -- start 2024-04-25T11:41:06.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.545+0000 7f8718a10740 1 -- start start 2024-04-25T11:41:06.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.546+0000 7f8718a10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a932f3b0 0x55e3a9758bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.546+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9337590 0x55e3a9714fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.546+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.546+0000 7f8718a10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ff700 con 0x55e3a932f3b0 2024-04-25T11:41:06.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.546+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1fea90 con 0x55e3a9337590 2024-04-25T11:41:06.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.546+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f870c1ff0b0 con 0x55e3a9358270 2024-04-25T11:41:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.547+0000 7f870aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a932f3b0 0x55e3a9758bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.547+0000 7f8703e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9337590 0x55e3a9714fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.547+0000 7f8703e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9337590 0x55e3a9714fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:56646/0 (socket says 172.21.15.19:56646) 2024-04-25T11:41:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.547+0000 7f8703e00640 1 -- 172.21.15.19:0/4071058978 learned_addr learned my addr 172.21.15.19:0/4071058978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:06.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.547+0000 7f870b400640 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f8703e00640 1 -- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 msgr2=0x55e3a9725210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f8703e00640 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f8703e00640 1 -- 172.21.15.19:0/4071058978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a932f3b0 msgr2=0x55e3a9758bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f8703e00640 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a932f3b0 0x55e3a9758bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f8703e00640 1 -- 172.21.15.19:0/4071058978 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a96f88c0 con 0x55e3a9337590 2024-04-25T11:41:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f870aa00640 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a932f3b0 0x55e3a9758bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:06.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f870b400640 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.548+0000 7f8703e00640 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9337590 0x55e3a9714fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86f405bf30 tx=0x7f86f4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a337b9a4b3ab49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.549+0000 7f86faa00640 1 -- 172.21.15.19:0/4071058978 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f405f0b0 con 0x55e3a9337590 2024-04-25T11:41:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.549+0000 7f86faa00640 1 -- 172.21.15.19:0/4071058978 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f405eda0 con 0x55e3a9337590 2024-04-25T11:41:06.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.549+0000 7f86faa00640 1 -- 172.21.15.19:0/4071058978 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f405b4b0 con 0x55e3a9337590 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.549+0000 7f8718a10740 1 -- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9337590 msgr2=0x55e3a9714fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.549+0000 7f8718a10740 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9337590 0x55e3a9714fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86f405bf30 tx=0x7f86f4016000 comp rx=0 tx=0).stop 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.550+0000 7f8718a10740 1 -- 172.21.15.19:0/4071058978 shutdown_connections 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.550+0000 7f8718a10740 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9337590 0x55e3a9714fc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.550+0000 7f8718a10740 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a9358270 0x55e3a9725210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.550+0000 7f8718a10740 1 --2- 172.21.15.19:0/4071058978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a932f3b0 0x55e3a9758bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.550+0000 7f8718a10740 1 -- 172.21.15.19:0/4071058978 >> 172.21.15.19:0/4071058978 conn(0x55e3a934c3c0 msgr2=0x55e3a97418d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.550+0000 7f8718a10740 1 -- 172.21.15.19:0/4071058978 shutdown_connections 2024-04-25T11:41:06.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.550+0000 7f8718a10740 1 -- 172.21.15.19:0/4071058978 wait complete. 2024-04-25T11:41:06.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.551+0000 7f8718a10740 1 -- start start 2024-04-25T11:41:06.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.551+0000 7f8718a10740 1 Processor -- start 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.552+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a9741830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.552+0000 7f8718a10740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9741d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.552+0000 7f8718a10740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 0x55e3a97bcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.552+0000 7f8718a10740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e3a97286d0 con 0x55e3a9337590 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.552+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e3a9715f90 con 0x55e3a9358270 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.552+0000 7f8718a10740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e3a9728d20 con 0x55e3a932f3b0 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8718a10740 10 client.0 _ll_register_callbacks cb 0x7f86f4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9741d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9741d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38154/0 (socket says 172.21.15.19:38154) 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 -- 172.21.15.19:0/1410869987 learned_addr learned my addr 172.21.15.19:0/1410869987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:06.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 -- 172.21.15.19:0/1410869987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 msgr2=0x55e3a9741830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a9741830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 -- 172.21.15.19:0/1410869987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 msgr2=0x55e3a97bcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 0x55e3a97bcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 -- 172.21.15.19:0/1410869987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a9721c00 con 0x55e3a9337590 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f8703e00640 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9741d70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f86f405f640 tx=0x7f86f4093c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3f2ce4b23da0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.553+0000 7f86efe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f86efe00640 1 -- 172.21.15.19:0/1410869987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f405f0b0 con 0x55e3a9337590 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f86efe00640 1 -- 172.21.15.19:0/1410869987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f86f405e610 con 0x55e3a9337590 2024-04-25T11:41:06.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f405eda0 con 0x55e3a9337590 2024-04-25T11:41:06.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f86efe00640 1 -- 172.21.15.19:0/1410869987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=316) ==== 24+0+0 (secure 0 0 0) 0x7f86f4063cf0 con 0x55e3a9337590 2024-04-25T11:41:06.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 10 client.10749 fetch_fsmap learned FSMap version 316 2024-04-25T11:41:06.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=316,monmap=2+}) -- 0x55e3a97bd9e0 con 0x55e3a9337590 2024-04-25T11:41:06.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f86efe00640 1 -- 172.21.15.19:0/1410869987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f405eda0 con 0x55e3a9337590 2024-04-25T11:41:06.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f86efe00640 1 -- 172.21.15.19:0/1410869987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 316) ==== 3494+0+0 (secure 0 0 0) 0x7f86f405e810 con 0x55e3a9337590 2024-04-25T11:41:06.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 10 client.10749 fetch_fsmap finished waiting for FSMap version 316 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 10 client.10749 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 20 client.10749 populate_metadata read hostname 'smithi019' 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a97bd4b0 0x55e3a97619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 4 client.10749 mds_command: new command op to 10689 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.554+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 --> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] -- command(tid 0: {"prefix": "status"}) -- 0x55e3a9666080 con 0x55e3a97bd4b0 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.555+0000 7f870aa00640 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a97bd4b0 0x55e3a97619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.555+0000 7f86efe00640 10 client.10749 ms_handle_connect on v2:172.21.15.19:6834/2088909328 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.555+0000 7f870aa00640 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a97bd4b0 0x55e3a97619f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cdec77f587d4f37 server_cookie=c8b260bccb13ca57 in_seq=0 out_seq=0 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.555+0000 7f86efe00640 1 -- 172.21.15.19:0/1410869987 <== mds.0 v2:172.21.15.19:6834/2088909328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55e3a9666080 con 0x55e3a97bd4b0 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.555+0000 7f86efe00640 10 client.10749 handle_command_reply: tid=0 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.555+0000 7f8718a10740 1 client.10749 shutdown 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 msgr2=0x55e3a9741d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9741d70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f86f405f640 tx=0x7f86f4093c20 comp rx=0 tx=0).stop 2024-04-25T11:41:06.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 shutdown_connections 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:6834/2088909328,v1:172.21.15.19:6835/2088909328] conn(0x55e3a97bd4b0 0x55e3a97619f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e3a9358270 0x55e3a97bcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e3a9337590 0x55e3a9741d70 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 --2- 172.21.15.19:0/1410869987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e3a932f3b0 0x55e3a9741830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f86efe00640 0 client.10749 ms_handle_reset on v2:172.21.15.19:6834/2088909328 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 >> 172.21.15.19:0/1410869987 conn(0x55e3a934c3c0 msgr2=0x55e3a97255d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 shutdown_connections 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 1 -- 172.21.15.19:0/1410869987 wait complete. 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.556+0000 7f8718a10740 20 client.10749 trim_cache size 0 max 16384 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:41:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 10689, 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3637989839999998, 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 316, 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 126, 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 126, 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 10.048468456 2024-04-25T11:41:06.547 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86e80744e0 msgr2=0x7f86e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86e80744e0 0x7f86e80769f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f86fc032000 tx=0x7f86fc02e000 comp rx=0 tx=0).stop 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f870c14de00 msgr2=0x7f870c14d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f870c14de00 0x7f870c14d110 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f86f40107f0 tx=0x7f86f4010cb0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 shutdown_connections 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86e80744e0 0x7f86e80769f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f870c14d650 0x7f870c15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f870c14de00 0x7f870c14d110 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 --2- 172.21.15.19:0/3193950100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86f00039b0 0x7f870c141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 >> 172.21.15.19:0/3193950100 conn(0x7f870c0d1720 msgr2=0x7f870c0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 shutdown_connections 2024-04-25T11:41:06.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.557+0000 7f870be00640 1 -- 172.21.15.19:0/3193950100 wait complete. 2024-04-25T11:41:06.569 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:41:06.569 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2024-04-25T11:41:06.570 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:41:06.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:41:06.622 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:41:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.838+0000 7fc0aec00640 1 Processor -- start 2024-04-25T11:41:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.838+0000 7fc0aec00640 1 -- start start 2024-04-25T11:41:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.840+0000 7fc0aec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0a813fe10 0x7fc0a806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0aec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a806df20 con 0x7fc0a806db10 2024-04-25T11:41:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0aec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a815d5e0 con 0x7fc0a8142e40 2024-04-25T11:41:06.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0aec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a8143250 con 0x7fc0a813fe10 2024-04-25T11:41:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0ad800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0a813fe10 0x7fc0a806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0ace00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc0a806db10 0x7fc0a8142730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56756/0 (socket says 172.21.15.19:56756) 2024-04-25T11:41:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0ace00640 1 -- 172.21.15.19:0/414581450 learned_addr learned my addr 172.21.15.19:0/414581450 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0ad800640 1 -- 172.21.15.19:0/414581450 >> v1:172.21.15.169:6789/0 conn(0x7fc0a8142e40 legacy=0x7fc0a80d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0ad800640 1 -- 172.21.15.19:0/414581450 >> v1:172.21.15.19:6789/0 conn(0x7fc0a806db10 legacy=0x7fc0a8142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.841+0000 7fc0a7e00640 1 -- 172.21.15.19:0/414581450 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054615665 0 0) 0x7fc09c001110 con 0x7fc0a806db10 2024-04-25T11:41:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.842+0000 7fc0ad800640 1 -- 172.21.15.19:0/414581450 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a815d8c0 con 0x7fc0a813fe10 2024-04-25T11:41:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.843+0000 7fc0ad800640 1 --2- 172.21.15.19:0/414581450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a813fe10 0x7fc0a806d5d0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fc0980097a0 tx=0x7fc09802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eaad5d648d59f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.843+0000 7fc0a7e00640 1 -- 172.21.15.19:0/414581450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098037070 con 0x7fc0a813fe10 2024-04-25T11:41:06.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.843+0000 7fc0a7e00640 1 -- 172.21.15.19:0/414581450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09802ae50 con 0x7fc0a813fe10 2024-04-25T11:41:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.843+0000 7fc0a7e00640 1 -- 172.21.15.19:0/414581450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098032660 con 0x7fc0a813fe10 2024-04-25T11:41:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.844+0000 7fc0aec00640 1 -- 172.21.15.19:0/414581450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a813fe10 msgr2=0x7fc0a806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.844+0000 7fc0aec00640 1 --2- 172.21.15.19:0/414581450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a813fe10 0x7fc0a806d5d0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fc0980097a0 tx=0x7fc09802a140 comp rx=0 tx=0).stop 2024-04-25T11:41:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.844+0000 7fc0aec00640 1 -- 172.21.15.19:0/414581450 shutdown_connections 2024-04-25T11:41:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.844+0000 7fc0aec00640 1 --2- 172.21.15.19:0/414581450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a813fe10 0x7fc0a806d5d0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.844+0000 7fc0aec00640 1 -- 172.21.15.19:0/414581450 >> 172.21.15.19:0/414581450 conn(0x7fc0a8072ee0 msgr2=0x7fc0a80732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.844+0000 7fc0aec00640 1 -- 172.21.15.19:0/414581450 shutdown_connections 2024-04-25T11:41:06.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.844+0000 7fc0aec00640 1 -- 172.21.15.19:0/414581450 wait complete. 2024-04-25T11:41:06.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.846+0000 7fc0aec00640 1 Processor -- start 2024-04-25T11:41:06.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.846+0000 7fc0aec00640 1 -- start start 2024-04-25T11:41:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0aec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0a806db10 0x7fc0a815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0aec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a813fe10 0x7fc0a80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0aec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a8142e40 0x7fc0a81f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0aec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a815d2d0 con 0x7fc0a8142e40 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0aec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a815de80 con 0x7fc0a813fe10 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0aec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a80d0740 con 0x7fc0a806db10 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0ae200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a8142e40 0x7fc0a81f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0ae200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a8142e40 0x7fc0a81f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38180/0 (socket says 172.21.15.19:38180) 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.848+0000 7fc0ae200640 1 -- 172.21.15.19:0/3419059009 learned_addr learned my addr 172.21.15.19:0/3419059009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ace00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a813fe10 0x7fc0a80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ad800640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0a806db10 0x7fc0a815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ae200640 1 -- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0a806db10 msgr2=0x7fc0a815bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ae200640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0a806db10 0x7fc0a815bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ae200640 1 -- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a813fe10 msgr2=0x7fc0a80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ae200640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a813fe10 0x7fc0a80ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ae200640 1 -- 172.21.15.19:0/3419059009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a8143b10 con 0x7fc0a8142e40 2024-04-25T11:41:06.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ace00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a813fe10 0x7fc0a80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ad800640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0a806db10 0x7fc0a815bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0ae200640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a8142e40 0x7fc0a81f9c40 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000b9a0 tx=0x7fc0a000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0a5600640 1 -- 172.21.15.19:0/3419059009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0019070 con 0x7fc0a8142e40 2024-04-25T11:41:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0a5600640 1 -- 172.21.15.19:0/3419059009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a000cba0 con 0x7fc0a8142e40 2024-04-25T11:41:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.849+0000 7fc0a5600640 1 -- 172.21.15.19:0/3419059009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0014440 con 0x7fc0a8142e40 2024-04-25T11:41:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.850+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a8143df0 con 0x7fc0a8142e40 2024-04-25T11:41:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.850+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81443b0 con 0x7fc0a8142e40 2024-04-25T11:41:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.851+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a8050740 con 0x7fc0a8142e40 2024-04-25T11:41:06.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.852+0000 7fc0a5600640 1 -- 172.21.15.19:0/3419059009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc0a001c070 con 0x7fc0a8142e40 2024-04-25T11:41:06.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.853+0000 7fc0a5600640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc08c0744e0 0x7fc08c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:06.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.853+0000 7fc0a5600640 1 -- 172.21.15.19:0/3419059009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7fc0a0095a30 con 0x7fc0a8142e40 2024-04-25T11:41:06.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.854+0000 7fc0ad800640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc08c0744e0 0x7fc08c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:06.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.855+0000 7fc0ad800640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc08c0744e0 0x7fc08c0769f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc098034000 tx=0x7fc09802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:06.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:06.858+0000 7fc0a5600640 1 -- 172.21.15.19:0/3419059009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0a005f5a0 con 0x7fc0a8142e40 2024-04-25T11:41:07.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.106+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc0a8143860 con 0x7fc0a8142e40 2024-04-25T11:41:07.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.107+0000 7fc0a5600640 1 -- 172.21.15.19:0/3419059009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v316) ==== 53+0+73 (secure 0 0 0) 0x7fc0a005eea0 con 0x7fc0a8142e40 2024-04-25T11:41:07.095 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:41:07.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc08c0744e0 msgr2=0x7fc08c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc08c0744e0 0x7fc08c0769f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc098034000 tx=0x7fc09802e000 comp rx=0 tx=0).stop 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a8142e40 msgr2=0x7fc0a81f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a8142e40 0x7fc0a81f9c40 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000b9a0 tx=0x7fc0a000be60 comp rx=0 tx=0).stop 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 shutdown_connections 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc08c0744e0 0x7fc08c0769f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a813fe10 0x7fc0a80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0a8142e40 0x7fc0a81f9c40 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 --2- 172.21.15.19:0/3419059009 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0a806db10 0x7fc0a815bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 >> 172.21.15.19:0/3419059009 conn(0x7fc0a8072ee0 msgr2=0x7fc0a80d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 shutdown_connections 2024-04-25T11:41:07.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07.110+0000 7fc0aec00640 1 -- 172.21.15.19:0/3419059009 wait complete. 2024-04-25T11:41:07.124 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:41:07.124 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:41:07.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:41:07.125 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:41:07.125 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:07.125 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:41:07.125 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:41:07.125 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:41:07.125 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:41:07.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:07.125 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:41:07.180 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:41:07.180 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:41:07.181 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41806sec preferred_lft 41806sec 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:07.182 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:41:07.183 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:07.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:07.184 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:41:07.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:07.270 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:41:07.326 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:41:07.326 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:41:07.327 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:41:07.327 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:41:07.327 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:41:07.327 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:41:07.327 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:41:07.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:41:07.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:41:07.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:41:07.327 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:41:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:41:07.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:41:07.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:07.550 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:41:07.597 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:41:07.597 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:41:07.597 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:41:07.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:07.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:41:07.681 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:41:07.681 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:07.737 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:41:07.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:41:07.737 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:07.795 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:41:07.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:07.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:41:08.013 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:41:08.013 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:41:08.059 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:41:08.060 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:41:08.060 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:41:08.060 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:41:08.060 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:41:08.062 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:41:08.062 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:41:08.062 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:41:08.062 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:41:08.062 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:41:08.066 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:41:08.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:08.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:08.142 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:41:08.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:41:08.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:41:08.265 INFO:teuthology.orchestra.run.smithi019.stdout:client10761 2024-04-25T11:41:08.266 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:08.266 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:08.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:08.316 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:41:08.321 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:41:08.375 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:08.388+0000 7fe3d0e00640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:41:08.376 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:41:08.376 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:41:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:08.376 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:41:08.377 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2024-04-25T11:41:08.377 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2024-04-25T11:41:08.377 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:08.377 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:08.428 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:08.428 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:08.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:41:08.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.832+0000 7f3fcbe00640 1 Processor -- start 2024-04-25T11:41:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.832+0000 7f3fcbe00640 1 -- start start 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.834+0000 7f3fcbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fcc06c5c0 0x7f3fcc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.834+0000 7f3fcbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc0b4ea0 0x7f3fcc15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.834+0000 7f3fcbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fcc0d1230 con 0x7f3fcc144370 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.834+0000 7f3fcbe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3fcc0d06b0 con 0x7f3fcc06c5c0 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.834+0000 7f3fcbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3fcc0d0990 con 0x7f3fcc0b4ea0 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcaa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fcc06c5c0 0x7f3fcc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcaa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fcc06c5c0 0x7f3fcc0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38188/0 (socket says 172.21.15.19:38188) 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcaa00640 1 -- 172.21.15.19:0/3926227968 learned_addr learned my addr 172.21.15.19:0/3926227968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcb400640 1 --2- 172.21.15.19:0/3926227968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc0b4ea0 0x7f3fcc15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcaa00640 1 -- 172.21.15.19:0/3926227968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc0b4ea0 msgr2=0x7f3fcc15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcaa00640 1 --2- 172.21.15.19:0/3926227968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc0b4ea0 0x7f3fcc15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcaa00640 1 -- 172.21.15.19:0/3926227968 >> v1:172.21.15.19:6789/0 conn(0x7f3fcc144370 legacy=0x7f3fcc0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.835+0000 7f3fcaa00640 1 -- 172.21.15.19:0/3926227968 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc0d0c70 con 0x7f3fcc06c5c0 2024-04-25T11:41:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.836+0000 7f3fcaa00640 1 --2- 172.21.15.19:0/3926227968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc06c5c0 0x7f3fcc0cfa70 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009870 tx=0x7f3fbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f159a866a90e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.837+0000 7f3fca000640 1 -- 172.21.15.19:0/3926227968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc037070 con 0x7f3fcc06c5c0 2024-04-25T11:41:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.837+0000 7f3fca000640 1 -- 172.21.15.19:0/3926227968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fbc032440 con 0x7f3fcc06c5c0 2024-04-25T11:41:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.837+0000 7f3fca000640 1 -- 172.21.15.19:0/3926227968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc032810 con 0x7f3fcc06c5c0 2024-04-25T11:41:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 -- 172.21.15.19:0/3926227968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc06c5c0 msgr2=0x7f3fcc0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/3926227968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc06c5c0 0x7f3fcc0cfa70 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009870 tx=0x7f3fbc02a270 comp rx=0 tx=0).stop 2024-04-25T11:41:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 -- 172.21.15.19:0/3926227968 shutdown_connections 2024-04-25T11:41:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/3926227968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc0b4ea0 0x7f3fcc15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/3926227968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc06c5c0 0x7f3fcc0cfa70 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 -- 172.21.15.19:0/3926227968 >> 172.21.15.19:0/3926227968 conn(0x7f3fcc0d1720 msgr2=0x7f3fcc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 -- 172.21.15.19:0/3926227968 shutdown_connections 2024-04-25T11:41:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.838+0000 7f3fcbe00640 1 -- 172.21.15.19:0/3926227968 wait complete. 2024-04-25T11:41:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.840+0000 7f3fcbe00640 1 Processor -- start 2024-04-25T11:41:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.841+0000 7f3fcbe00640 1 -- start start 2024-04-25T11:41:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.842+0000 7f3fcbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc06c5c0 0x7f3fcc0afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.842+0000 7f3fcbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fcc144370 0x7f3fcc139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.842+0000 7f3fcbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc136550 0x7f3fcc1f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.842+0000 7f3fcbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc0d0450 con 0x7f3fcc136550 2024-04-25T11:41:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.842+0000 7f3fcbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc0d02d0 con 0x7f3fcc144370 2024-04-25T11:41:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.842+0000 7f3fcbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3fcc147760 con 0x7f3fcc06c5c0 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.842+0000 7f3fcb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc136550 0x7f3fcc1f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc136550 0x7f3fcc1f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38200/0 (socket says 172.21.15.19:38200) 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 -- 172.21.15.19:0/1561109016 learned_addr learned my addr 172.21.15.19:0/1561109016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcaa00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc06c5c0 0x7f3fcc0afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 -- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc06c5c0 msgr2=0x7f3fcc0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc06c5c0 0x7f3fcc0afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 -- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fcc144370 msgr2=0x7f3fcc139480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fcc144370 0x7f3fcc139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 -- 172.21.15.19:0/1561109016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc135ab0 con 0x7f3fcc136550 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcaa00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc06c5c0 0x7f3fcc0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fcb400640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc136550 0x7f3fcc1f5ed0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4007e90 tx=0x7f3fc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.843+0000 7f3fc2a00640 1 -- 172.21.15.19:0/1561109016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4014420 con 0x7f3fcc136550 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.844+0000 7f3fc2a00640 1 -- 172.21.15.19:0/1561109016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc4019070 con 0x7f3fcc136550 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.844+0000 7f3fc2a00640 1 -- 172.21.15.19:0/1561109016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4014800 con 0x7f3fcc136550 2024-04-25T11:41:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.844+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc0b02d0 con 0x7f3fcc136550 2024-04-25T11:41:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.844+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc134470 con 0x7f3fcc136550 2024-04-25T11:41:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.845+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc050740 con 0x7f3fcc136550 2024-04-25T11:41:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.847+0000 7f3fc2a00640 1 -- 172.21.15.19:0/1561109016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fc40149a0 con 0x7f3fcc136550 2024-04-25T11:41:08.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.848+0000 7f3fc2a00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fa80744b0 0x7f3fa80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:08.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.848+0000 7f3fc2a00640 1 -- 172.21.15.19:0/1561109016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f3fc4096660 con 0x7f3fcc136550 2024-04-25T11:41:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.849+0000 7f3fcaa00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fa80744b0 0x7f3fa80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:08.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.850+0000 7f3fcaa00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fa80744b0 0x7f3fa80769c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:08.853+0000 7f3fc2a00640 1 -- 172.21.15.19:0/1561109016 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fc405fb80 con 0x7f3fcc136550 2024-04-25T11:41:09.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.062+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fcc0b0060 con 0x7f3fcc136550 2024-04-25T11:41:09.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.064+0000 7f3fc2a00640 1 -- 172.21.15.19:0/1561109016 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+21068 (secure 0 0 0) 0x7f3fc4063e80 con 0x7f3fcc136550 2024-04-25T11:41:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:09.053 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":126,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:40:56.345968+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6834/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.067+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fa80744b0 msgr2=0x7f3fa80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.067+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fa80744b0 0x7f3fa80769c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).stop 2024-04-25T11:41:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc136550 msgr2=0x7f3fcc1f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc136550 0x7f3fcc1f5ed0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4007e90 tx=0x7f3fc400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 shutdown_connections 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fa80744b0 0x7f3fa80769c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fcc144370 0x7f3fcc139480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fcc136550 0x7f3fcc1f5ed0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 --2- 172.21.15.19:0/1561109016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fcc06c5c0 0x7f3fcc0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 >> 172.21.15.19:0/1561109016 conn(0x7f3fcc0d1720 msgr2=0x7f3fcc06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 shutdown_connections 2024-04-25T11:41:09.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.068+0000 7f3fcbe00640 1 -- 172.21.15.19:0/1561109016 wait complete. 2024-04-25T11:41:09.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:41:09.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:41:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.369+0000 7f227f000640 1 Processor -- start 2024-04-25T11:41:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.369+0000 7f227f000640 1 -- start start 2024-04-25T11:41:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.371+0000 7f227f000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22780d11f0 0x7f22780d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.371+0000 7f227f000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22780d1830 con 0x7f22780d0de0 2024-04-25T11:41:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.371+0000 7f227f000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2278160dc0 con 0x7f227806ccd0 2024-04-25T11:41:09.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.371+0000 7f227f000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f22780d1600 con 0x7f22780d11f0 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.372+0000 7f2277e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f22780d0de0 0x7f227806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56778/0 (socket says 172.21.15.19:56778) 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.372+0000 7f2277e00640 1 -- 172.21.15.19:0/1992161469 learned_addr learned my addr 172.21.15.19:0/1992161469 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.372+0000 7f227d000640 1 --2- 172.21.15.19:0/1992161469 >> v2:172.21.15.169:3300/0 conn(0x7f22780d11f0 0x7f22780d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955884625 0 0) 0x7f22780d1830 con 0x7f22780d0de0 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f225c003520 con 0x7f22780d0de0 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3800535288 0 0) 0x7f225c003520 con 0x7f22780d0de0 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 >> v2:172.21.15.169:3300/0 conn(0x7f22780d11f0 msgr2=0x7f22780d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 --2- 172.21.15.19:0/1992161469 >> v2:172.21.15.169:3300/0 conn(0x7f22780d11f0 0x7f22780d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 >> v1:172.21.15.169:6789/0 conn(0x7f227806ccd0 legacy=0x7f22780d4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22781610a0 con 0x7f22780d0de0 2024-04-25T11:41:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2268001670 con 0x7f22780d0de0 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f227d000640 1 --2- 172.21.15.19:0/1992161469 >> v2:172.21.15.169:3300/0 conn(0x7f22780d11f0 0x7f22780d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 >> v1:172.21.15.19:6789/0 conn(0x7f22780d0de0 legacy=0x7f227806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f225c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c004800 0x7f225c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.373+0000 7f2276a00640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c005250 0x7f225c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f225c00c1b0 con 0x7f225c0039b0 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f225c00c360 con 0x7f225c005250 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f225c00c560 con 0x7f225c004800 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2277e00640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f225c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2277400640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c004800 0x7f225c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f227d000640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c005250 0x7f225c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2277e00640 1 -- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c004800 msgr2=0x7f225c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2277e00640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f225c004800 0x7f225c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2277e00640 1 -- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c005250 msgr2=0x7f225c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2277e00640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f225c005250 0x7f225c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f227d000640 1 -- 172.21.15.19:0/1992161469 reap_dead start 2024-04-25T11:41:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.374+0000 7f2277e00640 1 -- 172.21.15.19:0/1992161469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c003520 con 0x7f225c0039b0 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.376+0000 7f2277e00640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f225c003e20 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f226800c240 tx=0x7f225c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5af6bfd48fe0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.376+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268007cf0 con 0x7f225c0039b0 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.376+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2268005af0 con 0x7f225c0039b0 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.376+0000 7f2276a00640 1 -- 172.21.15.19:0/1992161469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268007070 con 0x7f225c0039b0 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.377+0000 7f227f000640 1 -- 172.21.15.19:0/1992161469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 msgr2=0x7f225c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.377+0000 7f227f000640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f225c003e20 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f226800c240 tx=0x7f225c005830 comp rx=0 tx=0).stop 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.377+0000 7f227f000640 1 -- 172.21.15.19:0/1992161469 shutdown_connections 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.377+0000 7f227f000640 1 --2- 172.21.15.19:0/1992161469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f225c003e20 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.377+0000 7f227f000640 1 -- 172.21.15.19:0/1992161469 >> 172.21.15.19:0/1992161469 conn(0x7f22780d5d70 msgr2=0x7f22780d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.377+0000 7f227f000640 1 -- 172.21.15.19:0/1992161469 shutdown_connections 2024-04-25T11:41:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.377+0000 7f227f000640 1 -- 172.21.15.19:0/1992161469 wait complete. 2024-04-25T11:41:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.379+0000 7f227f000640 1 Processor -- start 2024-04-25T11:41:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.380+0000 7f227f000640 1 -- start start 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.381+0000 7f227f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f22780cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.381+0000 7f227f000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227815b730 0x7f227815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.381+0000 7f227f000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22781f9ca0 0x7f22781fa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.381+0000 7f227f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22780b5300 con 0x7f225c0039b0 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.381+0000 7f227f000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2278160ab0 con 0x7f227815b730 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.381+0000 7f227f000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2278164de0 con 0x7f22781f9ca0 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f22780cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f22780cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38214/0 (socket says 172.21.15.19:38214) 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 -- 172.21.15.19:0/771029191 learned_addr learned my addr 172.21.15.19:0/771029191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f227d000640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22781f9ca0 0x7f22781fa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277400640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227815b730 0x7f227815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 -- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22781f9ca0 msgr2=0x7f22781fa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22781f9ca0 0x7f22781fa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 -- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227815b730 msgr2=0x7f227815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227815b730 0x7f227815b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277e00640 1 -- 172.21.15.19:0/771029191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22781644f0 con 0x7f225c0039b0 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.382+0000 7f2277400640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227815b730 0x7f227815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.383+0000 7f2277e00640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f22780cf810 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f226800c240 tx=0x7f2278161270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.383+0000 7f226fe00640 1 -- 172.21.15.19:0/771029191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268037040 con 0x7f225c0039b0 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.383+0000 7f226fe00640 1 -- 172.21.15.19:0/771029191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2268041070 con 0x7f225c0039b0 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.383+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22781faab0 con 0x7f225c0039b0 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.383+0000 7f226fe00640 1 -- 172.21.15.19:0/771029191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268001670 con 0x7f225c0039b0 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.383+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2278164a70 con 0x7f225c0039b0 2024-04-25T11:41:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.385+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2278050740 con 0x7f225c0039b0 2024-04-25T11:41:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.386+0000 7f226fe00640 1 -- 172.21.15.19:0/771029191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f22680049e0 con 0x7f225c0039b0 2024-04-25T11:41:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.387+0000 7f226fe00640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22580744b0 0x7f22580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.387+0000 7f226fe00640 1 -- 172.21.15.19:0/771029191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f22680b5cb0 con 0x7f225c0039b0 2024-04-25T11:41:09.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.388+0000 7f2277400640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22580744b0 0x7f22580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.389+0000 7f2277400640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22580744b0 0x7f22580769c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f2264009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:09.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.392+0000 7f226fe00640 1 -- 172.21.15.19:0/771029191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f226807f940 con 0x7f225c0039b0 2024-04-25T11:41:09.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.598+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f22780b5b50 con 0x7f225c0039b0 2024-04-25T11:41:09.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.599+0000 7f226fe00640 1 -- 172.21.15.19:0/771029191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f226807f240 con 0x7f225c0039b0 2024-04-25T11:41:09.587 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:41:09.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22580744b0 msgr2=0x7f22580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22580744b0 0x7f22580769c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f2264009040 comp rx=0 tx=0).stop 2024-04-25T11:41:09.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 msgr2=0x7f22780cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f22780cf810 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f226800c240 tx=0x7f2278161270 comp rx=0 tx=0).stop 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 shutdown_connections 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f22580744b0 0x7f22580769c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f227815b730 0x7f227815b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22781f9ca0 0x7f22781fa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 --2- 172.21.15.19:0/771029191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f225c0039b0 0x7f22780cf810 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.602+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 >> 172.21.15.19:0/771029191 conn(0x7f22780d5d70 msgr2=0x7f22780b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.603+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 shutdown_connections 2024-04-25T11:41:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.603+0000 7f227f000640 1 -- 172.21.15.19:0/771029191 wait complete. 2024-04-25T11:41:09.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:41:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.909+0000 7f98ff800640 1 Processor -- start 2024-04-25T11:41:09.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.910+0000 7f98ff800640 1 -- start start 2024-04-25T11:41:09.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.912+0000 7f98ff800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98f8158850 0x7f98f8070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.912+0000 7f98ff800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98f806d5d0 0x7f98f806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.912+0000 7f98ff800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98f806df20 con 0x7f98f815e780 2024-04-25T11:41:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.913+0000 7f98ff800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f98f8158c60 con 0x7f98f806d5d0 2024-04-25T11:41:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.913+0000 7f98ff800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f98f815ef90 con 0x7f98f8158850 2024-04-25T11:41:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.913+0000 7f98fd800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98f806d5d0 0x7f98f806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.913+0000 7f98efe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f98f815e780 0x7f98f815f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56794/0 (socket says 172.21.15.19:56794) 2024-04-25T11:41:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.913+0000 7f98efe00640 1 -- 172.21.15.19:0/241178012 learned_addr learned my addr 172.21.15.19:0/241178012 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:09.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.913+0000 7f98fce00640 1 --2- 172.21.15.19:0/241178012 >> v2:172.21.15.169:3300/0 conn(0x7f98f8158850 0x7f98f8070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.914+0000 7f98fd800640 1 -- 172.21.15.19:0/241178012 >> v2:172.21.15.169:3300/0 conn(0x7f98f8158850 msgr2=0x7f98f8070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:09.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.914+0000 7f98fd800640 1 --2- 172.21.15.19:0/241178012 >> v2:172.21.15.169:3300/0 conn(0x7f98f8158850 0x7f98f8070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.914+0000 7f98fd800640 1 -- 172.21.15.19:0/241178012 >> v1:172.21.15.19:6789/0 conn(0x7f98f815e780 legacy=0x7f98f815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.914+0000 7f98fd800640 1 -- 172.21.15.19:0/241178012 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f815f270 con 0x7f98f806d5d0 2024-04-25T11:41:09.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.915+0000 7f98fd800640 1 --2- 172.21.15.19:0/241178012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f806d5d0 0x7f98f806d9e0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b6ca0 tx=0x7f98f002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5863aa9b0d1e0311 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.915+0000 7f98f7e00640 1 -- 172.21.15.19:0/241178012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0007740 con 0x7f98f806d5d0 2024-04-25T11:41:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.915+0000 7f98f7e00640 1 -- 172.21.15.19:0/241178012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f00078e0 con 0x7f98f806d5d0 2024-04-25T11:41:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.915+0000 7f98f7e00640 1 -- 172.21.15.19:0/241178012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0007cb0 con 0x7f98f806d5d0 2024-04-25T11:41:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 -- 172.21.15.19:0/241178012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f806d5d0 msgr2=0x7f98f806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 --2- 172.21.15.19:0/241178012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f806d5d0 0x7f98f806d9e0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b6ca0 tx=0x7f98f002a680 comp rx=0 tx=0).stop 2024-04-25T11:41:09.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 -- 172.21.15.19:0/241178012 shutdown_connections 2024-04-25T11:41:09.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 --2- 172.21.15.19:0/241178012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f806d5d0 0x7f98f806d9e0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 --2- 172.21.15.19:0/241178012 >> v2:172.21.15.169:3300/0 conn(0x7f98f8158850 0x7f98f8070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 -- 172.21.15.19:0/241178012 >> 172.21.15.19:0/241178012 conn(0x7f98f80730b0 msgr2=0x7f98f80734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:09.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 -- 172.21.15.19:0/241178012 shutdown_connections 2024-04-25T11:41:09.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.916+0000 7f98ff800640 1 -- 172.21.15.19:0/241178012 wait complete. 2024-04-25T11:41:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.918+0000 7f98ff800640 1 Processor -- start 2024-04-25T11:41:09.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.919+0000 7f98ff800640 1 -- start start 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.920+0000 7f98ff800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f806d5d0 0x7f98f8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.920+0000 7f98ff800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f8158850 0x7f98f8140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.920+0000 7f98ff800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98f815e780 0x7f98f815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.920+0000 7f98ff800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8160010 con 0x7f98f8158850 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.920+0000 7f98ff800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98f806f040 con 0x7f98f815e780 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.920+0000 7f98ff800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f98f815eb90 con 0x7f98f806d5d0 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.920+0000 7f98efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f8158850 0x7f98f8140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f8158850 0x7f98f8140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38224/0 (socket says 172.21.15.19:38224) 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 -- 172.21.15.19:0/4192762000 learned_addr learned my addr 172.21.15.19:0/4192762000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98fce00640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f806d5d0 0x7f98f8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 -- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f806d5d0 msgr2=0x7f98f8157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f806d5d0 0x7f98f8157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 -- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98f815e780 msgr2=0x7f98f815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:09.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98f815e780 0x7f98f815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:09.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 -- 172.21.15.19:0/4192762000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f8141f50 con 0x7f98f8158850 2024-04-25T11:41:09.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98fce00640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f806d5d0 0x7f98f8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:09.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.921+0000 7f98efe00640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f8158850 0x7f98f8140180 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c3f0 tx=0x7f98e000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.922+0000 7f98f5600640 1 -- 172.21.15.19:0/4192762000 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0017070 con 0x7f98f8158850 2024-04-25T11:41:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.922+0000 7f98f5600640 1 -- 172.21.15.19:0/4192762000 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e0005ce0 con 0x7f98f8158850 2024-04-25T11:41:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.922+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f8140eb0 con 0x7f98f8158850 2024-04-25T11:41:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.922+0000 7f98f5600640 1 -- 172.21.15.19:0/4192762000 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00072f0 con 0x7f98f8158850 2024-04-25T11:41:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.922+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f8141470 con 0x7f98f8158850 2024-04-25T11:41:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.923+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f80761c0 con 0x7f98f8158850 2024-04-25T11:41:09.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.924+0000 7f98f5600640 1 -- 172.21.15.19:0/4192762000 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98e001a070 con 0x7f98f8158850 2024-04-25T11:41:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.926+0000 7f98f5600640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d40744b0 0x7f98d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:09.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.926+0000 7f98f5600640 1 -- 172.21.15.19:0/4192762000 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f98e00951b0 con 0x7f98f8158850 2024-04-25T11:41:09.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.926+0000 7f98fce00640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d40744b0 0x7f98d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:09.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.928+0000 7f98fce00640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d40744b0 0x7f98d40769c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006fd0 tx=0x7f98e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:09.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:09.931+0000 7f98f5600640 1 -- 172.21.15.19:0/4192762000 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98f806df20 con 0x7f98f8158850 2024-04-25T11:41:10.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.136+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98f8146180 con 0x7f98f8158850 2024-04-25T11:41:10.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.137+0000 7f98f5600640 1 -- 172.21.15.19:0/4192762000 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) ==== 104+0+11 (secure 0 0 0) 0x7f98e005ed20 con 0x7f98f8158850 2024-04-25T11:41:10.125 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:41:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d40744b0 msgr2=0x7f98d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d40744b0 0x7f98d40769c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006fd0 tx=0x7f98e8008040 comp rx=0 tx=0).stop 2024-04-25T11:41:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f8158850 msgr2=0x7f98f8140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f8158850 0x7f98f8140180 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f98e000c3f0 tx=0x7f98e000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 shutdown_connections 2024-04-25T11:41:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98d40744b0 0x7f98d40769c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98f815e780 0x7f98f815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98f8158850 0x7f98f8140180 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 --2- 172.21.15.19:0/4192762000 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98f806d5d0 0x7f98f8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 >> 172.21.15.19:0/4192762000 conn(0x7f98f80730b0 msgr2=0x7f98f8147760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.140+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 shutdown_connections 2024-04-25T11:41:10.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.141+0000 7f98ff800640 1 -- 172.21.15.19:0/4192762000 wait complete. 2024-04-25T11:41:10.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:41:10.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.458+0000 7f7eaa000640 1 Processor -- start 2024-04-25T11:41:10.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.458+0000 7f7eaa000640 1 -- start start 2024-04-25T11:41:10.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.460+0000 7f7eaa000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7ea406f930 0x7f7ea406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.460+0000 7f7eaa000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ea406fd40 0x7f7ea406db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.461+0000 7f7eaa000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea41623f0 con 0x7f7ea4070d20 2024-04-25T11:41:10.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.461+0000 7f7eaa000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea41626d0 con 0x7f7ea406fd40 2024-04-25T11:41:10.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.461+0000 7f7eaa000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea406f130 con 0x7f7ea406f930 2024-04-25T11:41:10.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.461+0000 7f7ea9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ea406fd40 0x7f7ea406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.461+0000 7f7ea9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ea406fd40 0x7f7ea406db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38228/0 (socket says 172.21.15.19:38228) 2024-04-25T11:41:10.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.461+0000 7f7ea9600640 1 -- 172.21.15.19:0/1406697510 learned_addr learned my addr 172.21.15.19:0/1406697510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:10.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.461+0000 7f7ea3e00640 1 --2- 172.21.15.19:0/1406697510 >> v2:172.21.15.169:3300/0 conn(0x7f7ea406f930 0x7f7ea406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.462+0000 7f7ea9600640 1 -- 172.21.15.19:0/1406697510 >> v2:172.21.15.169:3300/0 conn(0x7f7ea406f930 msgr2=0x7f7ea406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:10.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.462+0000 7f7ea9600640 1 --2- 172.21.15.19:0/1406697510 >> v2:172.21.15.169:3300/0 conn(0x7f7ea406f930 0x7f7ea406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.462+0000 7f7ea9600640 1 -- 172.21.15.19:0/1406697510 >> v1:172.21.15.19:6789/0 conn(0x7f7ea4070d20 legacy=0x7f7ea4161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.462+0000 7f7ea9600640 1 -- 172.21.15.19:0/1406697510 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea406f410 con 0x7f7ea406fd40 2024-04-25T11:41:10.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.463+0000 7f7ea9600640 1 --2- 172.21.15.19:0/1406697510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406fd40 0x7f7ea406db10 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4161a90 tx=0x7f7e9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3baab7259febe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:10.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.464+0000 7f7ea3400640 1 -- 172.21.15.19:0/1406697510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c036070 con 0x7f7ea406fd40 2024-04-25T11:41:10.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.464+0000 7f7ea3400640 1 -- 172.21.15.19:0/1406697510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c031600 con 0x7f7ea406fd40 2024-04-25T11:41:10.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.464+0000 7f7ea3400640 1 -- 172.21.15.19:0/1406697510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0319d0 con 0x7f7ea406fd40 2024-04-25T11:41:10.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 -- 172.21.15.19:0/1406697510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406fd40 msgr2=0x7f7ea406db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1406697510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406fd40 0x7f7ea406db10 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4161a90 tx=0x7f7e9c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 -- 172.21.15.19:0/1406697510 shutdown_connections 2024-04-25T11:41:10.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1406697510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406fd40 0x7f7ea406db10 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1406697510 >> v2:172.21.15.169:3300/0 conn(0x7f7ea406f930 0x7f7ea406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 -- 172.21.15.19:0/1406697510 >> 172.21.15.19:0/1406697510 conn(0x7f7ea4073160 msgr2=0x7f7ea4073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:10.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 -- 172.21.15.19:0/1406697510 shutdown_connections 2024-04-25T11:41:10.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.465+0000 7f7eaa000640 1 -- 172.21.15.19:0/1406697510 wait complete. 2024-04-25T11:41:10.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.468+0000 7f7eaa000640 1 Processor -- start 2024-04-25T11:41:10.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.468+0000 7f7eaa000640 1 -- start start 2024-04-25T11:41:10.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7eaa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406f930 0x7f7ea41546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7eaa000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406fd40 0x7f7ea4160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7eaa000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea4070d20 0x7f7ea41609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7eaa000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea415e470 con 0x7f7ea406f930 2024-04-25T11:41:10.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7eaa000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea415e2f0 con 0x7f7ea4070d20 2024-04-25T11:41:10.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7eaa000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40ce7e0 con 0x7f7ea406fd40 2024-04-25T11:41:10.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406f930 0x7f7ea41546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406f930 0x7f7ea41546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38234/0 (socket says 172.21.15.19:38234) 2024-04-25T11:41:10.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 -- 172.21.15.19:0/1494705460 learned_addr learned my addr 172.21.15.19:0/1494705460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:10.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 -- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406fd40 msgr2=0x7f7ea4160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406fd40 0x7f7ea4160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 -- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea4070d20 msgr2=0x7f7ea41609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea9600640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea4070d20 0x7f7ea41609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea4070d20 0x7f7ea41609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.471+0000 7f7ea8c00640 1 -- 172.21.15.19:0/1494705460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea41fa990 con 0x7f7ea406f930 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.472+0000 7f7ea8c00640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406f930 0x7f7ea41546c0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c440 tx=0x7f7e9800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.472+0000 7f7ea0c00640 1 -- 172.21.15.19:0/1494705460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980052d0 con 0x7f7ea406f930 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.472+0000 7f7ea0c00640 1 -- 172.21.15.19:0/1494705460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98005470 con 0x7f7ea406f930 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.472+0000 7f7ea0c00640 1 -- 172.21.15.19:0/1494705460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98007260 con 0x7f7ea406f930 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.472+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41553d0 con 0x7f7ea406f930 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.472+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea4155990 con 0x7f7ea406f930 2024-04-25T11:41:10.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.474+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea4076c60 con 0x7f7ea406f930 2024-04-25T11:41:10.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.475+0000 7f7ea0c00640 1 -- 172.21.15.19:0/1494705460 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e98007400 con 0x7f7ea406f930 2024-04-25T11:41:10.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.476+0000 7f7ea0c00640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e840744e0 0x7f7e840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.476+0000 7f7ea0c00640 1 -- 172.21.15.19:0/1494705460 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f7e980951e0 con 0x7f7ea406f930 2024-04-25T11:41:10.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.477+0000 7f7ea3e00640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e840744e0 0x7f7e840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.478+0000 7f7ea3e00640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e840744e0 0x7f7e840769f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006fd0 tx=0x7f7e90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:10.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.481+0000 7f7ea0c00640 1 -- 172.21.15.19:0/1494705460 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e9805ede0 con 0x7f7ea406f930 2024-04-25T11:41:10.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.690+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ea40ce360 con 0x7f7ea406f930 2024-04-25T11:41:10.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.691+0000 7f7ea0c00640 1 -- 172.21.15.19:0/1494705460 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) ==== 100+0+11 (secure 0 0 0) 0x7f7e9805e6e0 con 0x7f7ea406f930 2024-04-25T11:41:10.679 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:41:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.694+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e840744e0 msgr2=0x7f7e840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.694+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e840744e0 0x7f7e840769f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006fd0 tx=0x7f7e90008040 comp rx=0 tx=0).stop 2024-04-25T11:41:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.694+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406f930 msgr2=0x7f7ea41546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.694+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406f930 0x7f7ea41546c0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c440 tx=0x7f7e9800c900 comp rx=0 tx=0).stop 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.694+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 shutdown_connections 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.694+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e840744e0 0x7f7e840769f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.694+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea4070d20 0x7f7ea41609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.695+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406fd40 0x7f7ea4160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.695+0000 7f7eaa000640 1 --2- 172.21.15.19:0/1494705460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea406f930 0x7f7ea41546c0 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.695+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 >> 172.21.15.19:0/1494705460 conn(0x7f7ea4073160 msgr2=0x7f7ea40cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.695+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 shutdown_connections 2024-04-25T11:41:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:10.695+0000 7f7eaa000640 1 -- 172.21.15.19:0/1494705460 wait complete. 2024-04-25T11:41:10.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:41:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.009+0000 7f5a11400640 1 Processor -- start 2024-04-25T11:41:10.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.009+0000 7f5a11400640 1 -- start start 2024-04-25T11:41:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.011+0000 7f5a11400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c0b4ea0 0x7f5a0c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.011+0000 7f5a11400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a0c144370 0x7f5a0c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.011+0000 7f5a11400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a0c0d1230 con 0x7f5a0c06c5c0 2024-04-25T11:41:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.011+0000 7f5a11400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5a0c0d06b0 con 0x7f5a0c144370 2024-04-25T11:41:10.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.011+0000 7f5a11400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c0d0990 con 0x7f5a0c0b4ea0 2024-04-25T11:41:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.012+0000 7f5a10a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a0c144370 0x7f5a0c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.012+0000 7f5a0be00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f5a0c06c5c0 0x7f5a0c0b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:60982/0 (socket says 172.21.15.19:60982) 2024-04-25T11:41:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.012+0000 7f5a0b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c0b4ea0 0x7f5a0c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.012+0000 7f5a0be00640 1 -- 172.21.15.19:0/3014301472 learned_addr learned my addr 172.21.15.19:0/3014301472 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.013+0000 7f5a10a00640 1 -- 172.21.15.19:0/3014301472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c0b4ea0 msgr2=0x7f5a0c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:11.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.013+0000 7f5a10a00640 1 --2- 172.21.15.19:0/3014301472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c0b4ea0 0x7f5a0c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.013+0000 7f5a10a00640 1 -- 172.21.15.19:0/3014301472 >> v1:172.21.15.169:6789/0 conn(0x7f5a0c06c5c0 legacy=0x7f5a0c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.013+0000 7f5a10a00640 1 -- 172.21.15.19:0/3014301472 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c0d0c70 con 0x7f5a0c144370 2024-04-25T11:41:11.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.014+0000 7f5a10a00640 1 --2- 172.21.15.19:0/3014301472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c144370 0x7f5a0c15e770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5a040097a0 tx=0x7f5a0402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81ef655d104a509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:11.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.015+0000 7f5a0aa00640 1 -- 172.21.15.19:0/3014301472 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04037070 con 0x7f5a0c144370 2024-04-25T11:41:11.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.015+0000 7f5a0aa00640 1 -- 172.21.15.19:0/3014301472 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a04032440 con 0x7f5a0c144370 2024-04-25T11:41:11.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.015+0000 7f5a0aa00640 1 -- 172.21.15.19:0/3014301472 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04032810 con 0x7f5a0c144370 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 -- 172.21.15.19:0/3014301472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c144370 msgr2=0x7f5a0c15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 --2- 172.21.15.19:0/3014301472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c144370 0x7f5a0c15e770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5a040097a0 tx=0x7f5a0402a180 comp rx=0 tx=0).stop 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 -- 172.21.15.19:0/3014301472 shutdown_connections 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 --2- 172.21.15.19:0/3014301472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c144370 0x7f5a0c15e770 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 --2- 172.21.15.19:0/3014301472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c0b4ea0 0x7f5a0c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 -- 172.21.15.19:0/3014301472 >> 172.21.15.19:0/3014301472 conn(0x7f5a0c0d1720 msgr2=0x7f5a0c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 -- 172.21.15.19:0/3014301472 shutdown_connections 2024-04-25T11:41:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.016+0000 7f5a11400640 1 -- 172.21.15.19:0/3014301472 wait complete. 2024-04-25T11:41:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.019+0000 7f5a11400640 1 Processor -- start 2024-04-25T11:41:11.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.019+0000 7f5a11400640 1 -- start start 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.020+0000 7f5a11400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a0c06c5c0 0x7f5a0c161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a11400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c144370 0x7f5a0c162330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a11400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c15f910 0x7f5a0c1354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a11400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0d0450 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a11400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0d02d0 con 0x7f5a0c15f910 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a11400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c147760 con 0x7f5a0c144370 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a0be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a0c06c5c0 0x7f5a0c161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a0be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a0c06c5c0 0x7f5a0c161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38246/0 (socket says 172.21.15.19:38246) 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a0be00640 1 -- 172.21.15.19:0/3600700790 learned_addr learned my addr 172.21.15.19:0/3600700790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:11.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a10a00640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c15f910 0x7f5a0c1354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a0b400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c144370 0x7f5a0c162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.021+0000 7f5a0be00640 1 -- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c144370 msgr2=0x7f5a0c162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a0be00640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c144370 0x7f5a0c162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a0be00640 1 -- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c15f910 msgr2=0x7f5a0c1354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a0be00640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c15f910 0x7f5a0c1354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a0be00640 1 -- 172.21.15.19:0/3600700790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c138940 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a0be00640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a0c06c5c0 0x7f5a0c161260 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f59f400c3c0 tx=0x7f59f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a03e00640 1 -- 172.21.15.19:0/3600700790 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4017070 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a03e00640 1 -- 172.21.15.19:0/3600700790 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f4005ce0 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a03e00640 1 -- 172.21.15.19:0/3600700790 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f40072f0 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c138c20 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.022+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c1fabe0 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.024+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c0cf630 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.025+0000 7f5a03e00640 1 -- 172.21.15.19:0/3600700790 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f59f401a070 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.026+0000 7f5a03e00640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59e80744b0 0x7f59e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.026+0000 7f5a03e00640 1 -- 172.21.15.19:0/3600700790 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f59f4095220 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.027+0000 7f5a0b400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59e80744b0 0x7f59e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.028+0000 7f5a0b400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59e80744b0 0x7f59e80769c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0d1ba0 tx=0x7f59fc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:11.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.031+0000 7f5a03e00640 1 -- 172.21.15.19:0/3600700790 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a0c0cf630 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.249+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a0c138df0 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.251+0000 7f5a03e00640 1 -- 172.21.15.19:0/3600700790 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 68 entries v126) ==== 81+0+4124 (secure 0 0 0) 0x7f59f405ed90 con 0x7f5a0c06c5c0 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/698241108 2024-04-26T11:40:28.419530+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/698241108 2024-04-26T11:40:28.419530+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2891720740 2024-04-26T11:40:22.060358+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2891720740 2024-04-26T11:40:22.060358+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:41:11.239 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:41:11.240 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/258414220 2024-04-26T11:40:56.345928+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:41:11.241 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/258414220 2024-04-26T11:40:56.345928+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:41:11.242 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:41:11.243 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:41:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:listed 68 entries 2024-04-25T11:41:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59e80744b0 msgr2=0x7f59e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59e80744b0 0x7f59e80769c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0d1ba0 tx=0x7f59fc00b040 comp rx=0 tx=0).stop 2024-04-25T11:41:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a0c06c5c0 msgr2=0x7f5a0c161260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a0c06c5c0 0x7f5a0c161260 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f59f400c3c0 tx=0x7f59f400c880 comp rx=0 tx=0).stop 2024-04-25T11:41:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 shutdown_connections 2024-04-25T11:41:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59e80744b0 0x7f59e80769c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a0c15f910 0x7f5a0c1354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a0c144370 0x7f5a0c162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 --2- 172.21.15.19:0/3600700790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a0c06c5c0 0x7f5a0c161260 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.253+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 >> 172.21.15.19:0/3600700790 conn(0x7f5a0c0d1720 msgr2=0x7f5a0c0b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.254+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 shutdown_connections 2024-04-25T11:41:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.254+0000 7f5a11400640 1 -- 172.21.15.19:0/3600700790 wait complete. 2024-04-25T11:41:11.272 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:41:11.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:11.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:41:11.374 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:41:11.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:11.374 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:41:11.425 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:41:11.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.713+0000 7f7d7ea00640 1 Processor -- start 2024-04-25T11:41:11.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.713+0000 7f7d7ea00640 1 -- start start 2024-04-25T11:41:11.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.715+0000 7f7d7ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7d78144350 0x7f7d780d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78162610 0x7f7d78144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7ea00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7806fc70 con 0x7f7d78161af0 2024-04-25T11:41:11.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7806ff50 con 0x7f7d78144350 2024-04-25T11:41:11.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d78070230 con 0x7f7d78162610 2024-04-25T11:41:11.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7d78144350 0x7f7d780d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7d78144350 0x7f7d780d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38252/0 (socket says 172.21.15.19:38252) 2024-04-25T11:41:11.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7d600640 1 -- 172.21.15.19:0/1081529879 learned_addr learned my addr 172.21.15.19:0/1081529879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:11.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.716+0000 7f7d7e000640 1 --2- 172.21.15.19:0/1081529879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78162610 0x7f7d78144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.717+0000 7f7d7d600640 1 -- 172.21.15.19:0/1081529879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78162610 msgr2=0x7f7d78144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:11.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.717+0000 7f7d7d600640 1 --2- 172.21.15.19:0/1081529879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78162610 0x7f7d78144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.717+0000 7f7d7d600640 1 -- 172.21.15.19:0/1081529879 >> v1:172.21.15.169:6789/0 conn(0x7f7d78161af0 legacy=0x7f7d78161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:11.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.717+0000 7f7d7d600640 1 -- 172.21.15.19:0/1081529879 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d78070510 con 0x7f7d78144350 2024-04-25T11:41:11.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.718+0000 7f7d7d600640 1 --2- 172.21.15.19:0/1081529879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78144350 0x7f7d780d1070 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f7d780b6aa0 tx=0x7f7d6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967c09a2f8cb9805 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:11.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.718+0000 7f7d77e00640 1 -- 172.21.15.19:0/1081529879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c037070 con 0x7f7d78144350 2024-04-25T11:41:11.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.718+0000 7f7d77e00640 1 -- 172.21.15.19:0/1081529879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6c032440 con 0x7f7d78144350 2024-04-25T11:41:11.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.718+0000 7f7d77e00640 1 -- 172.21.15.19:0/1081529879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c032810 con 0x7f7d78144350 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 -- 172.21.15.19:0/1081529879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78144350 msgr2=0x7f7d780d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/1081529879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78144350 0x7f7d780d1070 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f7d780b6aa0 tx=0x7f7d6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 -- 172.21.15.19:0/1081529879 shutdown_connections 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/1081529879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78162610 0x7f7d78144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/1081529879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78144350 0x7f7d780d1070 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 -- 172.21.15.19:0/1081529879 >> 172.21.15.19:0/1081529879 conn(0x7f7d780d1720 msgr2=0x7f7d780d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 -- 172.21.15.19:0/1081529879 shutdown_connections 2024-04-25T11:41:11.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.719+0000 7f7d7ea00640 1 -- 172.21.15.19:0/1081529879 wait complete. 2024-04-25T11:41:11.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.722+0000 7f7d7ea00640 1 Processor -- start 2024-04-25T11:41:11.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.723+0000 7f7d7ea00640 1 -- start start 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78144350 0x7f7d78160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78161af0 0x7f7d781e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78162610 0x7f7d780af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78145520 con 0x7f7d78162610 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781453a0 con 0x7f7d78144350 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7d781456a0 con 0x7f7d78161af0 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78162610 0x7f7d780af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78162610 0x7f7d780af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38256/0 (socket says 172.21.15.19:38256) 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7e000640 1 -- 172.21.15.19:0/850646301 learned_addr learned my addr 172.21.15.19:0/850646301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7d600640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78144350 0x7f7d78160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.725+0000 7f7d7cc00640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78161af0 0x7f7d781e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.726+0000 7f7d7e000640 1 -- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78161af0 msgr2=0x7f7d781e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.726+0000 7f7d7e000640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78161af0 0x7f7d781e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.726+0000 7f7d7e000640 1 -- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78144350 msgr2=0x7f7d78160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.726+0000 7f7d7e000640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78144350 0x7f7d78160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.726+0000 7f7d7e000640 1 -- 172.21.15.19:0/850646301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7813e210 con 0x7f7d78162610 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.726+0000 7f7d7d600640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78144350 0x7f7d78160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.726+0000 7f7d7e000640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78162610 0x7f7d780af690 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000cc80 tx=0x7f7d7000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.727+0000 7f7d75600640 1 -- 172.21.15.19:0/850646301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7000ce30 con 0x7f7d78162610 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.727+0000 7f7d75600640 1 -- 172.21.15.19:0/850646301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d70003990 con 0x7f7d78162610 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.727+0000 7f7d75600640 1 -- 172.21.15.19:0/850646301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70003d60 con 0x7f7d78162610 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.727+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d781e7a80 con 0x7f7d78162610 2024-04-25T11:41:11.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.727+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d781e8040 con 0x7f7d78162610 2024-04-25T11:41:11.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.728+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d780d06f0 con 0x7f7d78162610 2024-04-25T11:41:11.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.730+0000 7f7d75600640 1 -- 172.21.15.19:0/850646301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d70003f00 con 0x7f7d78162610 2024-04-25T11:41:11.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.731+0000 7f7d75600640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d580744e0 0x7f7d580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:11.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.731+0000 7f7d75600640 1 -- 172.21.15.19:0/850646301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8633+0+0 (secure 0 0 0) 0x7f7d70096130 con 0x7f7d78162610 2024-04-25T11:41:11.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.731+0000 7f7d7d600640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d580744e0 0x7f7d580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:11.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.733+0000 7f7d7d600640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d580744e0 0x7f7d580769f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c034000 tx=0x7f7d6c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:11.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.736+0000 7f7d75600640 1 -- 172.21.15.19:0/850646301 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d780d06f0 con 0x7f7d78162610 2024-04-25T11:41:11.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.984+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7d78050740 con 0x7f7d78162610 2024-04-25T11:41:11.973 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:11.986+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:41:11.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.990+0000 7f7d75600640 1 -- 172.21.15.19:0/850646301 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v317) ==== 161+0+0 (secure 0 0 0) 0x7f7d70014030 con 0x7f7d78162610 2024-04-25T11:41:11.977 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:41:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.994+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d580744e0 msgr2=0x7f7d580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.994+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d580744e0 0x7f7d580769f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c034000 tx=0x7f7d6c02e000 comp rx=0 tx=0).stop 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.994+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78162610 msgr2=0x7f7d780af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.994+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78162610 0x7f7d780af690 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000cc80 tx=0x7f7d7000a5c0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 shutdown_connections 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7d580744e0 0x7f7d580769f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d78162610 0x7f7d780af690 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d78161af0 0x7f7d781e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 --2- 172.21.15.19:0/850646301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78144350 0x7f7d78160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 >> 172.21.15.19:0/850646301 conn(0x7f7d780d1720 msgr2=0x7f7d78161f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 shutdown_connections 2024-04-25T11:41:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:11.995+0000 7f7d7ea00640 1 -- 172.21.15.19:0/850646301 wait complete. 2024-04-25T11:41:11.998 INFO:tasks.ceph.mds.a.smithi019.stderr: -765> 2024-04-25T11:41:03.368+0000 7fe3d5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:41:12.001 INFO:tasks.ceph.mds.a.smithi019.stderr: -226> 2024-04-25T11:41:08.388+0000 7fe3d0e00640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:41:12.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:41:12.143 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:41:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.293+0000 7ff1cea00640 1 Processor -- start 2024-04-25T11:41:12.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.293+0000 7ff1cea00640 1 -- start start 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.295+0000 7ff1cea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1c806c5c0 0x7ff1c80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.295+0000 7ff1cea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c8144370 0x7ff1c80b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.295+0000 7ff1cea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c80b50e0 con 0x7ff1c80b4cd0 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c80d1230 con 0x7ff1c806c5c0 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c80d06b0 con 0x7ff1c8144370 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cd600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1c806c5c0 0x7ff1c80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cd600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1c806c5c0 0x7ff1c80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38274/0 (socket says 172.21.15.19:38274) 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cd600640 1 -- 172.21.15.19:0/978443821 learned_addr learned my addr 172.21.15.19:0/978443821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1ce000640 1 --1- 172.21.15.19:0/978443821 >> v1:172.21.15.19:6789/0 conn(0x7ff1c80b4cd0 0x7ff1c8161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:56828/0 (socket says 172.21.15.19:56828) 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1ccc00640 1 --2- 172.21.15.19:0/978443821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c8144370 0x7ff1c80b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cd600640 1 -- 172.21.15.19:0/978443821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c8144370 msgr2=0x7ff1c80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cd600640 1 --2- 172.21.15.19:0/978443821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c8144370 0x7ff1c80b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cd600640 1 -- 172.21.15.19:0/978443821 >> v1:172.21.15.19:6789/0 conn(0x7ff1c80b4cd0 legacy=0x7ff1c8161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.296+0000 7ff1cd600640 1 -- 172.21.15.19:0/978443821 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c80d0990 con 0x7ff1c806c5c0 2024-04-25T11:41:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.298+0000 7ff1cd600640 1 --2- 172.21.15.19:0/978443821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c80cfa70 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ec03f4d4408c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:12.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.298+0000 7ff1c7e00640 1 -- 172.21.15.19:0/978443821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc037070 con 0x7ff1c806c5c0 2024-04-25T11:41:12.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.298+0000 7ff1c7e00640 1 -- 172.21.15.19:0/978443821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc032440 con 0x7ff1c806c5c0 2024-04-25T11:41:12.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.298+0000 7ff1c7e00640 1 -- 172.21.15.19:0/978443821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc032810 con 0x7ff1c806c5c0 2024-04-25T11:41:12.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 -- 172.21.15.19:0/978443821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 msgr2=0x7ff1c80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 --2- 172.21.15.19:0/978443821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c80cfa70 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc02a250 comp rx=0 tx=0).stop 2024-04-25T11:41:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 -- 172.21.15.19:0/978443821 shutdown_connections 2024-04-25T11:41:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 --2- 172.21.15.19:0/978443821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c8144370 0x7ff1c80b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 --2- 172.21.15.19:0/978443821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c80cfa70 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 -- 172.21.15.19:0/978443821 >> 172.21.15.19:0/978443821 conn(0x7ff1c80d1720 msgr2=0x7ff1c80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 -- 172.21.15.19:0/978443821 shutdown_connections 2024-04-25T11:41:12.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.299+0000 7ff1cea00640 1 -- 172.21.15.19:0/978443821 wait complete. 2024-04-25T11:41:12.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.301+0000 7ff1cea00640 1 Processor -- start 2024-04-25T11:41:12.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.301+0000 7ff1cea00640 1 -- start start 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c81622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c80b4cd0 0x7ff1c815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c8144370 0x7ff1c8160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c80d02d0 con 0x7ff1c806c5c0 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c80d0f50 con 0x7ff1c8144370 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c80d0450 con 0x7ff1c80b4cd0 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c81622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c81622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38286/0 (socket says 172.21.15.19:38286) 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.303+0000 7ff1cd600640 1 -- 172.21.15.19:0/3851673479 learned_addr learned my addr 172.21.15.19:0/3851673479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1ccc00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c80b4cd0 0x7ff1c815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1cd600640 1 -- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c80b4cd0 msgr2=0x7ff1c815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1cd600640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c80b4cd0 0x7ff1c815fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1cd600640 1 -- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c8144370 msgr2=0x7ff1c8160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1ce000640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c8144370 0x7ff1c8160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1cd600640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c8144370 0x7ff1c8160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1cd600640 1 -- 172.21.15.19:0/3851673479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c8133e90 con 0x7ff1c806c5c0 2024-04-25T11:41:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.304+0000 7ff1cd600640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c81622f0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc02a290 tx=0x7ff1bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.305+0000 7ff1c5600640 1 -- 172.21.15.19:0/3851673479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc037040 con 0x7ff1c806c5c0 2024-04-25T11:41:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.305+0000 7ff1c5600640 1 -- 172.21.15.19:0/3851673479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc042070 con 0x7ff1c806c5c0 2024-04-25T11:41:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.305+0000 7ff1c5600640 1 -- 172.21.15.19:0/3851673479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc030bd0 con 0x7ff1c806c5c0 2024-04-25T11:41:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.305+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c8134170 con 0x7ff1c806c5c0 2024-04-25T11:41:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.305+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c8134730 con 0x7ff1c806c5c0 2024-04-25T11:41:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.306+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c8050740 con 0x7ff1c806c5c0 2024-04-25T11:41:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.307+0000 7ff1c5600640 1 -- 172.21.15.19:0/3851673479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff1bc047460 con 0x7ff1c806c5c0 2024-04-25T11:41:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.309+0000 7ff1c5600640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1a80744e0 0x7ff1a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.309+0000 7ff1c5600640 1 -- 172.21.15.19:0/3851673479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8719+0+0 (secure 0 0 0) 0x7ff1bc0b6b60 con 0x7ff1c806c5c0 2024-04-25T11:41:12.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.310+0000 7ff1ccc00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1a80744e0 0x7ff1a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.311+0000 7ff1ccc00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1a80744e0 0x7ff1a80769f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006fd0 tx=0x7ff1b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:12.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.314+0000 7ff1c5600640 1 -- 172.21.15.19:0/3851673479 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1bc080030 con 0x7ff1c806c5c0 2024-04-25T11:41:12.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.598+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff1c80cdde0 con 0x7ff1c806c5c0 2024-04-25T11:41:12.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.995+0000 7ff1c5600640 1 -- 172.21.15.19:0/3851673479 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v318) ==== 107+0+0 (secure 0 0 0) 0x7ff1bc084330 con 0x7ff1c806c5c0 2024-04-25T11:41:12.983 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1a80744e0 msgr2=0x7ff1a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1a80744e0 0x7ff1a80769f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006fd0 tx=0x7ff1b4008040 comp rx=0 tx=0).stop 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 msgr2=0x7ff1c81622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c81622f0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc02a290 tx=0x7ff1bc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 shutdown_connections 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1a80744e0 0x7ff1a80769f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1c8144370 0x7ff1c8160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1c80b4cd0 0x7ff1c815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 --2- 172.21.15.19:0/3851673479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1c806c5c0 0x7ff1c81622f0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 >> 172.21.15.19:0/3851673479 conn(0x7ff1c80d1720 msgr2=0x7ff1c815c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.998+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 shutdown_connections 2024-04-25T11:41:12.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:12.999+0000 7ff1cea00640 1 -- 172.21.15.19:0/3851673479 wait complete. 2024-04-25T11:41:13.019 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:41:13.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.221+0000 7f29f860ff40 1 Processor -- start 2024-04-25T11:41:13.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.222+0000 7f29f860ff40 1 -- start start 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.223+0000 7f29f860ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bcd1e0 0x558ae1ca36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.223+0000 7f29f860ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1bcd5f0 0x558ae1c99c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.223+0000 7f29f860ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1ca6dd0 0x558ae1ca5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.223+0000 7f29f860ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ae1c163f0 con 0x558ae1bcd1e0 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.223+0000 7f29f860ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558ae1c15e70 con 0x558ae1ca6dd0 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.223+0000 7f29f860ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x558ae1c166d0 con 0x558ae1bcd5f0 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bcd1e0 0x558ae1ca36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bcd1e0 0x558ae1ca36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38300/0 (socket says 172.21.15.19:38300) 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 -- 172.21.15.19:0/2627226786 learned_addr learned my addr 172.21.15.19:0/2627226786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 -- 172.21.15.19:0/2627226786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1bcd5f0 msgr2=0x558ae1c99c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:13.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2627226786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1bcd5f0 0x558ae1c99c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 -- 172.21.15.19:0/2627226786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1ca6dd0 msgr2=0x558ae1ca5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:13.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2627226786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1ca6dd0 0x558ae1ca5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 -- 172.21.15.19:0/2627226786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae1af6b30 con 0x558ae1bcd1e0 2024-04-25T11:41:13.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.224+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2627226786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bcd1e0 0x558ae1ca36c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f29e000c490 tx=0x7f29e000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa4abbcb77c1b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29eec00640 1 -- 172.21.15.19:0/2627226786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0010590 con 0x558ae1bcd1e0 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29eec00640 1 -- 172.21.15.19:0/2627226786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e0016070 con 0x558ae1bcd1e0 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29eec00640 1 -- 172.21.15.19:0/2627226786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0010970 con 0x558ae1bcd1e0 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29f860ff40 1 -- 172.21.15.19:0/2627226786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bcd1e0 msgr2=0x558ae1ca36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2627226786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bcd1e0 0x558ae1ca36c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f29e000c490 tx=0x7f29e000c950 comp rx=0 tx=0).stop 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29f860ff40 1 -- 172.21.15.19:0/2627226786 shutdown_connections 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2627226786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1ca6dd0 0x558ae1ca5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2627226786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1bcd5f0 0x558ae1c99c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2627226786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bcd1e0 0x558ae1ca36c0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.225+0000 7f29f860ff40 1 -- 172.21.15.19:0/2627226786 >> 172.21.15.19:0/2627226786 conn(0x558ae1aefac0 msgr2=0x558ae1bff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.226+0000 7f29f860ff40 1 -- 172.21.15.19:0/2627226786 shutdown_connections 2024-04-25T11:41:13.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.226+0000 7f29f860ff40 1 -- 172.21.15.19:0/2627226786 wait complete. 2024-04-25T11:41:13.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.227+0000 7f29f860ff40 1 Processor -- start 2024-04-25T11:41:13.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.227+0000 7f29f860ff40 1 -- start start 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.228+0000 7f29f860ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1bcd5f0 0x558ae1c97f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.228+0000 7f29f860ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1ca6dd0 0x558ae1c9ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.228+0000 7f29f860ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bccbe0 0x558ae1c9d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.228+0000 7f29f860ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558ae1ca3510 con 0x558ae1bccbe0 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.228+0000 7f29f860ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x558ae1c0f670 con 0x558ae1bcd5f0 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.228+0000 7f29f860ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x558ae1c9bfe0 con 0x558ae1ca6dd0 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bccbe0 0x558ae1c9d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bccbe0 0x558ae1c9d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38308/0 (socket says 172.21.15.19:38308) 2024-04-25T11:41:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 learned_addr learned my addr 172.21.15.19:0/2249892312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1ca6dd0 0x558ae1c9ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1bcd5f0 0x558ae1c97f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1ca6dd0 msgr2=0x558ae1c9ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1ca6dd0 0x558ae1c9ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1bcd5f0 msgr2=0x558ae1c97f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558ae1bcd5f0 0x558ae1c97f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae1c17680 con 0x558ae1bccbe0 2024-04-25T11:41:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558ae1ca6dd0 0x558ae1c9ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.229+0000 7f29f1400640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bccbe0 0x558ae1c9d390 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f29e8025c20 tx=0x7f29e8054530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.230+0000 7f29ed800640 1 -- 172.21.15.19:0/2249892312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8064020 con 0x558ae1bccbe0 2024-04-25T11:41:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.230+0000 7f29ed800640 1 -- 172.21.15.19:0/2249892312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e805a040 con 0x558ae1bccbe0 2024-04-25T11:41:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.230+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558ae1c98920 con 0x558ae1bccbe0 2024-04-25T11:41:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.230+0000 7f29ed800640 1 -- 172.21.15.19:0/2249892312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8059070 con 0x558ae1bccbe0 2024-04-25T11:41:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.230+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae1c9bd20 con 0x558ae1bccbe0 2024-04-25T11:41:13.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.232+0000 7f29ed800640 1 -- 172.21.15.19:0/2249892312 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f29e8067020 con 0x558ae1bccbe0 2024-04-25T11:41:13.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.233+0000 7f29ed800640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29cc072ec0 0x7f29cc0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.234+0000 7f29ed800640 1 -- 172.21.15.19:0/2249892312 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8719+0+0 (secure 0 0 0) 0x7f29e80d8520 con 0x558ae1bccbe0 2024-04-25T11:41:13.221 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:41:13.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.234+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x558ae1bcc6e0 0x558ae1d463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.234+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d46900 con 0x558ae1bcc6e0 2024-04-25T11:41:13.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.234+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29cc072ec0 0x7f29cc0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.235+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x558ae1bcc6e0 0x558ae1d463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.236+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29cc072ec0 0x7f29cc0753d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x558ae1ca1b10 tx=0x7f29e0010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.236+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x558ae1bcc6e0 0x558ae1d463c0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.240+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v127'1065 uv1038 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.240+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d46ca0 con 0x558ae1bcc6e0 2024-04-25T11:41:13.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.244+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v127'1066 uv1066 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.244+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.247+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v127'1067 uv1067 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.235 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:41:13.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.248+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d483f0 con 0x558ae1bcc6e0 2024-04-25T11:41:13.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.251+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'1068 uv1068 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.251+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48790 con 0x558ae1bcc6e0 2024-04-25T11:41:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.255+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'1069 uv1069 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.255+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d473e0 con 0x558ae1bcc6e0 2024-04-25T11:41:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.259+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'1070 uv1070 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.259+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d477d0 con 0x558ae1bcc6e0 2024-04-25T11:41:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.264+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v127'1071 uv1044 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.264+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d47bc0 con 0x558ae1bcc6e0 2024-04-25T11:41:13.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.268+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v127'1072 uv1072 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.268+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47bc0 con 0x558ae1bcc6e0 2024-04-25T11:41:13.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.272+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v127'1073 uv1073 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.259 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:41:13.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.272+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.277+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'1074 uv1074 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.277+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.281+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'1075 uv1075 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.281+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.286+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'1076 uv1076 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.286+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.291+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v127'1077 uv1064 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.291+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.296+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'1078 uv1078 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.296+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.300+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v127'1079 uv1079 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.288 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:41:13.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.301+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.305+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'1080 uv1080 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.305+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.310+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'1081 uv1081 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.310+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47b70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.314+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'1082 uv1082 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.314+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.319+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'1083 uv1083 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.319+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.324+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v127'1084 uv1057 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.324+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.329+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v127'1085 uv1085 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.329+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.321 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:41:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.334+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v127'1086 uv1086 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.334+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d49b40 con 0x558ae1bcc6e0 2024-04-25T11:41:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.339+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'1087 uv1087 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.339+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d49b40 con 0x558ae1bcc6e0 2024-04-25T11:41:13.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.343+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'1088 uv1088 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.343+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.347+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'1089 uv1089 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.348+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.352+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v127'1090 uv1063 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.353+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.357+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v127'1091 uv1091 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.357+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.349 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:41:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.362+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v127'1092 uv1092 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.362+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d49b40 con 0x558ae1bcc6e0 2024-04-25T11:41:13.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.366+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'1093 uv1093 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.366+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d49b40 con 0x558ae1bcc6e0 2024-04-25T11:41:13.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.370+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'1094 uv1094 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.371+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d48b30 con 0x558ae1bcc6e0 2024-04-25T11:41:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.375+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'1095 uv1095 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.375+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558ae1d49040 0x558ae1d4b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.376+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4b9a0 con 0x558ae1d49040 2024-04-25T11:41:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.376+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558ae1d49040 0x558ae1d4b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.379+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558ae1d49040 0x558ae1d4b460 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.384+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'777 uv756 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.384+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d4bd40 con 0x558ae1d49040 2024-04-25T11:41:13.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.389+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'778 uv778 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.389+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4c0e0 con 0x558ae1d49040 2024-04-25T11:41:13.380 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:41:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.393+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'779 uv779 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.393+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4c480 con 0x558ae1d49040 2024-04-25T11:41:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.398+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v127'780 uv780 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.398+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4c820 con 0x558ae1d49040 2024-04-25T11:41:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.402+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'781 uv781 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.402+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4cbc0 con 0x558ae1d49040 2024-04-25T11:41:13.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.407+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v127'782 uv775 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.407+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d4cf60 con 0x558ae1d49040 2024-04-25T11:41:13.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.412+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v127'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.412+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.404 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:41:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.417+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v127'784 uv784 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.417+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.421+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.421+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.425+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.426+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.430+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v127'787 uv776 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.430+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.435+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v127'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.435+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.427 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:41:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.440+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v127'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.440+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d519f0 con 0x558ae1d49040 2024-04-25T11:41:13.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.446+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v127'790 uv790 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.446+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d519f0 con 0x558ae1d49040 2024-04-25T11:41:13.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.451+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.451+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.456+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v127'792 uv772 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.456+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.460+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v127'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.461+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.453 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:41:13.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.465+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v127'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.465+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4e310 con 0x558ae1d49040 2024-04-25T11:41:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.470+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.470+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4e310 con 0x558ae1d49040 2024-04-25T11:41:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.474+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'796 uv796 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.475+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d4d300 con 0x558ae1d49040 2024-04-25T11:41:13.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.479+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'797 uv797 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d49040 2024-04-25T11:41:13.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.479+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x558ae1d4d810 0x558ae1d4fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.479+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d50170 con 0x558ae1d4d810 2024-04-25T11:41:13.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.480+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x558ae1d4d810 0x558ae1d4fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.482+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x558ae1d4d810 0x558ae1d4fc30 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.486+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.486+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d50170 con 0x558ae1d4d810 2024-04-25T11:41:13.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.490+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v127'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.491+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d50560 con 0x558ae1d4d810 2024-04-25T11:41:13.482 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:41:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.495+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v127'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.495+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d51570 con 0x558ae1d4d810 2024-04-25T11:41:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.499+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.499+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d51570 con 0x558ae1d4d810 2024-04-25T11:41:13.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.503+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.504+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d50560 con 0x558ae1d4d810 2024-04-25T11:41:13.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.508+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.508+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d50560 con 0x558ae1bcc6e0 2024-04-25T11:41:13.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.512+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'157 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.512+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d50560 con 0x558ae1bcc6e0 2024-04-25T11:41:13.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.517+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'158 uv158 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.517+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d50560 con 0x558ae1bcc6e0 2024-04-25T11:41:13.508 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:41:13.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.521+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'159 uv159 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.521+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d50560 con 0x558ae1bcc6e0 2024-04-25T11:41:13.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.525+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'160 uv160 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.525+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x558ae1d50eb0 0x558ae1d532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.526+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d53810 con 0x558ae1d50eb0 2024-04-25T11:41:13.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.526+0000 7f29f1400640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x558ae1d50eb0 0x558ae1d532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.528+0000 7f29f1400640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x558ae1d50eb0 0x558ae1d532d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.532+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.532+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d53bb0 con 0x558ae1d50eb0 2024-04-25T11:41:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.536+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v127'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.536+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d53fd0 con 0x558ae1d50eb0 2024-04-25T11:41:13.527 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:41:13.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.540+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v127'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.540+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d47f30 con 0x558ae1d50eb0 2024-04-25T11:41:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.544+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.544+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d55960 con 0x558ae1d50eb0 2024-04-25T11:41:13.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.548+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.548+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d54870 con 0x558ae1d50eb0 2024-04-25T11:41:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.552+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.553+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558ae1d55330 0x558ae1d57d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.553+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d58330 con 0x558ae1d55330 2024-04-25T11:41:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.553+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558ae1d55330 0x558ae1d57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.555+0000 7f29f0a00640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558ae1d55330 0x558ae1d57d60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.560+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v127'382 uv376 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.560+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.564+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v127'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.564+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.568+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v127'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.556 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:41:13.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.568+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d59710 con 0x558ae1d55330 2024-04-25T11:41:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.572+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.572+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d59710 con 0x558ae1d55330 2024-04-25T11:41:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.576+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.577+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.581+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.581+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.585+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v127'388 uv381 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.585+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.589+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v127'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.590+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.593+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v127'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.581 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:41:13.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.594+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.599+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.599+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d58700 con 0x558ae1d55330 2024-04-25T11:41:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.603+0000 7f29f0a00640 1 -- 172.21.15.19:0/2249892312 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e00109d0 con 0x558ae1d55330 2024-04-25T11:41:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.603+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x558ae1d58d80 0x558ae1d5b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.603+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5b910 con 0x558ae1d58d80 2024-04-25T11:41:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.604+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x558ae1d58d80 0x558ae1d5b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:13.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.605+0000 7f29f0000640 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x558ae1d58d80 0x558ae1d5b1c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.610+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v127'173 uv172 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.610+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5b910 con 0x558ae1d58d80 2024-04-25T11:41:13.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.614+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v127'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.614+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d58d80 2024-04-25T11:41:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.618+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v127'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.605 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:41:13.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.618+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d58d80 2024-04-25T11:41:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.622+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.622+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d58d80 2024-04-25T11:41:13.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.626+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'177 uv177 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.626+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1bcc6e0 2024-04-25T11:41:13.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.631+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.631+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1bcc6e0 2024-04-25T11:41:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.635+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v127'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.635+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1bcc6e0 2024-04-25T11:41:13.626 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:41:13.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.639+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v127'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.639+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5cd70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.644+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.644+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5cd70 con 0x558ae1bcc6e0 2024-04-25T11:41:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.648+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.648+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1bcc6e0 2024-04-25T11:41:13.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.652+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1bcc6e0 2024-04-25T11:41:13.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.653+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.658+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v127'345 uv339 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.658+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.662+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v127'346 uv346 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.662+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.654 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:41:13.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.667+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v127'347 uv347 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.667+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.672+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'348 uv348 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.672+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.677+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'349 uv349 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.677+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.681+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v127'350 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.681+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.685+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v127'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.685+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bd60 con 0x558ae1d4d810 2024-04-25T11:41:13.677 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:41:13.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.690+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v127'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.690+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5bde0 con 0x558ae1d4d810 2024-04-25T11:41:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.694+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.694+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5f060 con 0x558ae1d4d810 2024-04-25T11:41:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.700+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d4d810 2024-04-25T11:41:13.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.700+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5c700 con 0x558ae1d50eb0 2024-04-25T11:41:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.705+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v127'152 uv150 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.705+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5caf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.709+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'153 uv153 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.709+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d50eb0 2024-04-25T11:41:13.700 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:41:13.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.713+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v127'154 uv154 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.713+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d50eb0 2024-04-25T11:41:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.718+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'155 uv155 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.718+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d50eb0 2024-04-25T11:41:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.723+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v127'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.723+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d50eb0 2024-04-25T11:41:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.726+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v127'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.727+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d50eb0 2024-04-25T11:41:13.718 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:41:13.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.731+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v127'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.731+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5dea0 con 0x558ae1d50eb0 2024-04-25T11:41:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.735+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.735+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d50eb0 2024-04-25T11:41:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.740+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.740+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d50eb0 2024-04-25T11:41:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.744+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29e809dcf0 con 0x558ae1d50eb0 2024-04-25T11:41:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.745+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.748+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v127'176 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.749+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.753+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v127'177 uv177 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.753+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.756+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v127'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.744 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:41:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.757+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.761+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.761+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.765+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.765+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.769+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v127'429 uv420 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.769+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.774+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v127'430 uv430 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.774+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.766 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:41:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.778+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v127'431 uv431 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.778+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.783+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v127'432 uv432 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.783+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.787+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'433 uv433 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.787+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.791+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v127'434 uv427 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.791+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.795+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v127'435 uv435 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.795+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.787 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:41:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.799+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v127'436 uv436 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.799+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.803+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'437 uv437 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.803+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.807+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'438 uv438 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.808+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5ce90 con 0x558ae1d58d80 2024-04-25T11:41:13.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.812+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'439 uv439 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.812+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x558ae1d5d700 con 0x558ae1d58d80 2024-04-25T11:41:13.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.816+0000 7f29f0000640 1 -- 172.21.15.19:0/2249892312 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'440 uv440 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f29d800b040 con 0x558ae1d58d80 2024-04-25T11:41:13.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.816+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558ae1d49040 msgr2=0x558ae1d4b460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.816+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x558ae1d49040 0x558ae1d4b460 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.816+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558ae1d55330 msgr2=0x558ae1d57d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.816+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558ae1d55330 0x558ae1d57d60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.816+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x558ae1d58d80 msgr2=0x558ae1d5b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.816+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x558ae1d58d80 0x558ae1d5b1c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x558ae1d4d810 msgr2=0x558ae1d4fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x558ae1d4d810 0x558ae1d4fc30 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 reap_dead start 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x558ae1bcc6e0 msgr2=0x558ae1d463c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x558ae1bcc6e0 0x558ae1d463c0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f1400640 1 -- 172.21.15.19:0/2249892312 reap_dead start 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x558ae1d50eb0 msgr2=0x558ae1d532d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x558ae1d50eb0 0x558ae1d532d0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29cc072ec0 msgr2=0x7f29cc0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29cc072ec0 0x7f29cc0753d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x558ae1ca1b10 tx=0x7f29e0010000 comp rx=0 tx=0).stop 2024-04-25T11:41:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bccbe0 msgr2=0x558ae1c9d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bccbe0 0x558ae1c9d390 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f29e8025c20 tx=0x7f29e8054530 comp rx=0 tx=0).stop 2024-04-25T11:41:13.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.817+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 shutdown_connections 2024-04-25T11:41:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.818+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x558ae1d50eb0 0x558ae1d532d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.818+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29cc072ec0 0x7f29cc0753d0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.818+0000 7f29f860ff40 1 --2- 172.21.15.19:0/2249892312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558ae1bccbe0 0x558ae1c9d390 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.818+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 >> 172.21.15.19:0/2249892312 conn(0x558ae1aefac0 msgr2=0x558ae1ca0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.818+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 shutdown_connections 2024-04-25T11:41:13.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:13.818+0000 7f29f860ff40 1 -- 172.21.15.19:0/2249892312 wait complete. 2024-04-25T11:41:13.843 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:41:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.093+0000 7f94756d1f40 1 Processor -- start 2024-04-25T11:41:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.093+0000 7f94756d1f40 1 -- start start 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f94756d1f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848081e0 0x55c8848ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f94756d1f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848085f0 0x55c884966ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f94756d1f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 0x55c884967430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f94756d1f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c884850e70 con 0x55c8848081e0 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f94756d1f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c8847aed20 con 0x55c8848085f0 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f94756d1f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c884851110 con 0x55c8848e1dd0 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f946e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848081e0 0x55c8848ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f946e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848081e0 0x55c8848ddcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48968/0 (socket says 172.21.15.19:48968) 2024-04-25T11:41:14.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.095+0000 7f946e800640 1 -- 172.21.15.19:0/4123784394 learned_addr learned my addr 172.21.15.19:0/4123784394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946e800640 1 -- 172.21.15.19:0/4123784394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 msgr2=0x55c884967430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946e800640 1 --2- 172.21.15.19:0/4123784394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 0x55c884967430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946e800640 1 -- 172.21.15.19:0/4123784394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848085f0 msgr2=0x55c884966ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946e800640 1 --2- 172.21.15.19:0/4123784394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848085f0 0x55c884966ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946e800640 1 -- 172.21.15.19:0/4123784394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c884855a40 con 0x55c8848081e0 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946e800640 1 --2- 172.21.15.19:0/4123784394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848081e0 0x55c8848ddcd0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f946000b2c0 tx=0x7f946000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d42c0075d953bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946ca00640 1 -- 172.21.15.19:0/4123784394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460008070 con 0x55c8848081e0 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946ca00640 1 -- 172.21.15.19:0/4123784394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946000d500 con 0x55c8848081e0 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.096+0000 7f946ca00640 1 -- 172.21.15.19:0/4123784394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946000d8d0 con 0x55c8848081e0 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 -- 172.21.15.19:0/4123784394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848081e0 msgr2=0x55c8848ddcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 --2- 172.21.15.19:0/4123784394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848081e0 0x55c8848ddcd0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f946000b2c0 tx=0x7f946000b780 comp rx=0 tx=0).stop 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 -- 172.21.15.19:0/4123784394 shutdown_connections 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 --2- 172.21.15.19:0/4123784394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848085f0 0x55c884966ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 --2- 172.21.15.19:0/4123784394 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 0x55c884967430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 --2- 172.21.15.19:0/4123784394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848081e0 0x55c8848ddcd0 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 -- 172.21.15.19:0/4123784394 >> 172.21.15.19:0/4123784394 conn(0x55c88472aac0 msgr2=0x55c884854dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 -- 172.21.15.19:0/4123784394 shutdown_connections 2024-04-25T11:41:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.097+0000 7f94756d1f40 1 -- 172.21.15.19:0/4123784394 wait complete. 2024-04-25T11:41:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.098+0000 7f94756d1f40 1 Processor -- start 2024-04-25T11:41:14.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.098+0000 7f94756d1f40 1 -- start start 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f94756d1f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848081e0 0x55c8848ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f94756d1f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848085f0 0x55c884806de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f94756d1f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 0x55c884854dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f94756d1f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c884855cb0 con 0x55c8848085f0 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f94756d1f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c884853fb0 con 0x55c8848081e0 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f94756d1f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c884976ea0 con 0x55c8848e1dd0 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f946de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848085f0 0x55c884806de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f946de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848085f0 0x55c884806de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48980/0 (socket says 172.21.15.19:48980) 2024-04-25T11:41:14.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.100+0000 7f946de00640 1 -- 172.21.15.19:0/242118771 learned_addr learned my addr 172.21.15.19:0/242118771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946f200640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 0x55c884854dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946de00640 1 -- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 msgr2=0x55c884854dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946e800640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848081e0 0x55c8848ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946de00640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c8848e1dd0 0x55c884854dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946de00640 1 -- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848081e0 msgr2=0x55c8848ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946de00640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c8848081e0 0x55c8848ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946de00640 1 -- 172.21.15.19:0/242118771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8848dc9b0 con 0x55c8848085f0 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f946de00640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848085f0 0x55c884806de0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f945c009dd0 tx=0x7f945c006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f9467400640 1 -- 172.21.15.19:0/242118771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c017070 con 0x55c8848085f0 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f9467400640 1 -- 172.21.15.19:0/242118771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945c012410 con 0x55c8848085f0 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8848dceb0 con 0x55c8848085f0 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.101+0000 7f9467400640 1 -- 172.21.15.19:0/242118771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c006e60 con 0x55c8848085f0 2024-04-25T11:41:14.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.102+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8848dd470 con 0x55c8848085f0 2024-04-25T11:41:14.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.104+0000 7f9467400640 1 -- 172.21.15.19:0/242118771 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f945c01a070 con 0x55c8848085f0 2024-04-25T11:41:14.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.105+0000 7f9467400640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9450072e90 0x7f94500753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.105+0000 7f9467400640 1 -- 172.21.15.19:0/242118771 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8719+0+0 (secure 0 0 0) 0x7f945c093ba0 con 0x55c8848085f0 2024-04-25T11:41:14.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.106+0000 7f94756d1f40 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c884976070 0x55c884976480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.106+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55c884981af0 con 0x55c884976070 2024-04-25T11:41:14.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.106+0000 7f946e800640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9450072e90 0x7f94500753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.106+0000 7f946f200640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c884976070 0x55c884976480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.108+0000 7f946e800640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9450072e90 0x7f94500753a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f946000b2c0 tx=0x7f9460008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.108+0000 7f946f200640 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c884976070 0x55c884976480 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.111+0000 7f946f200640 1 -- 172.21.15.19:0/242118771 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v127'1096 uv1096 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9468057030 con 0x55c884976070 2024-04-25T11:41:14.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c884976070 msgr2=0x55c884976480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55c884976070 0x55c884976480 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9450072e90 msgr2=0x7f94500753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9450072e90 0x7f94500753a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f946000b2c0 tx=0x7f9460008000 comp rx=0 tx=0).stop 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848085f0 msgr2=0x55c884806de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 --2- 172.21.15.19:0/242118771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c8848085f0 0x55c884806de0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f945c009dd0 tx=0x7f945c006750 comp rx=0 tx=0).stop 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f946f200640 1 -- 172.21.15.19:0/242118771 reap_dead start 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 shutdown_connections 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 >> 172.21.15.19:0/242118771 conn(0x55c88472aac0 msgr2=0x55c884807700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 shutdown_connections 2024-04-25T11:41:14.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.112+0000 7f94756d1f40 1 -- 172.21.15.19:0/242118771 wait complete. 2024-04-25T11:41:14.114 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:41:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.426+0000 7fad57200640 1 Processor -- start 2024-04-25T11:41:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.427+0000 7fad57200640 1 -- start start 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.428+0000 7fad57200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad581601e0 0x7fad580b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.428+0000 7fad57200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad580b8150 0x7fad580b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.428+0000 7fad57200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad580b4e90 con 0x7fad580b96c0 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.428+0000 7fad57200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fad580d3e10 con 0x7fad580b8150 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.428+0000 7fad57200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fad580d40f0 con 0x7fad581601e0 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad55400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad580b8150 0x7fad580b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad55400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fad580b8150 0x7fad580b9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48984/0 (socket says 172.21.15.19:48984) 2024-04-25T11:41:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad55400640 1 -- 172.21.15.19:0/4002790539 learned_addr learned my addr 172.21.15.19:0/4002790539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad56800640 1 --1- 172.21.15.19:0/4002790539 >> v1:172.21.15.19:6789/0 conn(0x7fad580b96c0 0x7fad580d28f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59024/0 (socket says 172.21.15.19:59024) 2024-04-25T11:41:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad55e00640 1 --2- 172.21.15.19:0/4002790539 >> v2:172.21.15.169:3300/0 conn(0x7fad581601e0 0x7fad580b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad55400640 1 -- 172.21.15.19:0/4002790539 >> v2:172.21.15.169:3300/0 conn(0x7fad581601e0 msgr2=0x7fad580b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad55400640 1 --2- 172.21.15.19:0/4002790539 >> v2:172.21.15.169:3300/0 conn(0x7fad581601e0 0x7fad580b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad55400640 1 -- 172.21.15.19:0/4002790539 >> v1:172.21.15.19:6789/0 conn(0x7fad580b96c0 legacy=0x7fad580d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.430+0000 7fad55400640 1 -- 172.21.15.19:0/4002790539 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad580d43d0 con 0x7fad580b8150 2024-04-25T11:41:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.429+0000 7fad54a00640 1 -- 172.21.15.19:0/4002790539 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345957115 0 0) 0x7fad580b4e90 con 0x7fad580b96c0 2024-04-25T11:41:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.431+0000 7fad55400640 1 --2- 172.21.15.19:0/4002790539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad580b9180 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fad580d6b60 tx=0x7fad4402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200a7a6e9faed55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.431+0000 7fad54a00640 1 -- 172.21.15.19:0/4002790539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44037070 con 0x7fad580b8150 2024-04-25T11:41:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.431+0000 7fad54a00640 1 -- 172.21.15.19:0/4002790539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad44032440 con 0x7fad580b8150 2024-04-25T11:41:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.431+0000 7fad54a00640 1 -- 172.21.15.19:0/4002790539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44032810 con 0x7fad580b8150 2024-04-25T11:41:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 -- 172.21.15.19:0/4002790539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 msgr2=0x7fad580b9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 --2- 172.21.15.19:0/4002790539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad580b9180 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fad580d6b60 tx=0x7fad4402a1f0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 -- 172.21.15.19:0/4002790539 shutdown_connections 2024-04-25T11:41:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 --2- 172.21.15.19:0/4002790539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad580b9180 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 --2- 172.21.15.19:0/4002790539 >> v2:172.21.15.169:3300/0 conn(0x7fad581601e0 0x7fad580b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 -- 172.21.15.19:0/4002790539 >> 172.21.15.19:0/4002790539 conn(0x7fad5806dfe0 msgr2=0x7fad5806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 -- 172.21.15.19:0/4002790539 shutdown_connections 2024-04-25T11:41:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.432+0000 7fad57200640 1 -- 172.21.15.19:0/4002790539 wait complete. 2024-04-25T11:41:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.434+0000 7fad57200640 1 Processor -- start 2024-04-25T11:41:14.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.434+0000 7fad57200640 1 -- start start 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad57200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad58162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad57200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad580b96c0 0x7fad580d5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad57200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad581601e0 0x7fad580d6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad57200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad580d3000 con 0x7fad580b8150 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad57200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad580b4ba0 con 0x7fad580b96c0 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad57200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad580d3180 con 0x7fad581601e0 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad55e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad58162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad55e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad58162bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48996/0 (socket says 172.21.15.19:48996) 2024-04-25T11:41:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.436+0000 7fad55e00640 1 -- 172.21.15.19:0/1727212701 learned_addr learned my addr 172.21.15.19:0/1727212701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55400640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad580b96c0 0x7fad580d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad56800640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad581601e0 0x7fad580d6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55e00640 1 -- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad581601e0 msgr2=0x7fad580d6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55e00640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad581601e0 0x7fad580d6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55e00640 1 -- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad580b96c0 msgr2=0x7fad580d5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55e00640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad580b96c0 0x7fad580d5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55e00640 1 -- 172.21.15.19:0/1727212701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad580cb2e0 con 0x7fad580b8150 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55400640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad580b96c0 0x7fad580d5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.437+0000 7fad55e00640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad58162bc0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fad4800d620 tx=0x7fad4800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.438+0000 7fad4e000640 1 -- 172.21.15.19:0/1727212701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4800b620 con 0x7fad580b8150 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.438+0000 7fad4e000640 1 -- 172.21.15.19:0/1727212701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48019070 con 0x7fad580b8150 2024-04-25T11:41:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.438+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad580cb5c0 con 0x7fad580b8150 2024-04-25T11:41:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.438+0000 7fad4e000640 1 -- 172.21.15.19:0/1727212701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4800ba00 con 0x7fad580b8150 2024-04-25T11:41:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.438+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad580cbb80 con 0x7fad580b8150 2024-04-25T11:41:14.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.439+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad580cbdc0 con 0x7fad580b8150 2024-04-25T11:41:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.441+0000 7fad4e000640 1 -- 172.21.15.19:0/1727212701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad4800bba0 con 0x7fad580b8150 2024-04-25T11:41:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.442+0000 7fad4e000640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad380744b0 0x7fad380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.442+0000 7fad4e000640 1 -- 172.21.15.19:0/1727212701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8719+0+0 (secure 0 0 0) 0x7fad48096360 con 0x7fad580b8150 2024-04-25T11:41:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.443+0000 7fad55400640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad380744b0 0x7fad380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.444+0000 7fad55400640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad380744b0 0x7fad380769c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fad44034000 tx=0x7fad4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.448+0000 7fad4e000640 1 -- 172.21.15.19:0/1727212701 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad580cbdc0 con 0x7fad580b8150 2024-04-25T11:41:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:14.696+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fad58076c60 con 0x7fad580b8150 2024-04-25T11:41:15.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.314+0000 7fad4e000640 1 -- 172.21.15.19:0/1727212701 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v319) ==== 162+0+0 (secure 0 0 0) 0x7fad4805fe80 con 0x7fad580b8150 2024-04-25T11:41:15.302 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:41:15.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad380744b0 msgr2=0x7fad380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad380744b0 0x7fad380769c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fad44034000 tx=0x7fad4402e000 comp rx=0 tx=0).stop 2024-04-25T11:41:15.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 msgr2=0x7fad58162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad58162bc0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fad4800d620 tx=0x7fad4800dae0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 shutdown_connections 2024-04-25T11:41:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad380744b0 0x7fad380769c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad580b96c0 0x7fad580d5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad581601e0 0x7fad580d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 --2- 172.21.15.19:0/1727212701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad580b8150 0x7fad58162bc0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.318+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 >> 172.21.15.19:0/1727212701 conn(0x7fad5806dfe0 msgr2=0x7fad580ba040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.319+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 shutdown_connections 2024-04-25T11:41:15.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.319+0000 7fad57200640 1 -- 172.21.15.19:0/1727212701 wait complete. 2024-04-25T11:41:15.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:15.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.629+0000 7f3955a00640 1 Processor -- start 2024-04-25T11:41:15.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.629+0000 7f3955a00640 1 -- start start 2024-04-25T11:41:15.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f3955a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f395006c5c0 0x7f39500cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:15.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f3955a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3950144370 0x7f39500b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:15.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f3955a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f39500b4cd0 0x7f3950161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:15.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f3955a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f39500b50e0 con 0x7f395006c5c0 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f3955a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f39500d1230 con 0x7f39500b4cd0 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f3955a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f39500d06b0 con 0x7f3950144370 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f394fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f395006c5c0 0x7f39500cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f394fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f395006c5c0 0x7f39500cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48998/0 (socket says 172.21.15.19:48998) 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.632+0000 7f394fe00640 1 -- 172.21.15.19:0/3853210510 learned_addr learned my addr 172.21.15.19:0/3853210510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.633+0000 7f394fe00640 1 -- 172.21.15.19:0/3853210510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3950144370 msgr2=0x7f39500b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.633+0000 7f394fe00640 1 --2- 172.21.15.19:0/3853210510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3950144370 0x7f39500b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.633+0000 7f394fe00640 1 -- 172.21.15.19:0/3853210510 >> v2:172.21.15.169:3300/0 conn(0x7f39500b4cd0 msgr2=0x7f3950161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:15.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.633+0000 7f394fe00640 1 --2- 172.21.15.19:0/3853210510 >> v2:172.21.15.169:3300/0 conn(0x7f39500b4cd0 0x7f3950161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.633+0000 7f394fe00640 1 -- 172.21.15.19:0/3853210510 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39500d0990 con 0x7f395006c5c0 2024-04-25T11:41:15.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.634+0000 7f394fe00640 1 --2- 172.21.15.19:0/3853210510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f395006c5c0 0x7f39500cfa70 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f39500b6cd0 tx=0x7f394002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414209af280ca31f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:15.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.634+0000 7f394ea00640 1 -- 172.21.15.19:0/3853210510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940036070 con 0x7f395006c5c0 2024-04-25T11:41:15.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.634+0000 7f394ea00640 1 -- 172.21.15.19:0/3853210510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3940031600 con 0x7f395006c5c0 2024-04-25T11:41:15.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.634+0000 7f394ea00640 1 -- 172.21.15.19:0/3853210510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39400319d0 con 0x7f395006c5c0 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 -- 172.21.15.19:0/3853210510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f395006c5c0 msgr2=0x7f39500cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 --2- 172.21.15.19:0/3853210510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f395006c5c0 0x7f39500cfa70 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f39500b6cd0 tx=0x7f394002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 -- 172.21.15.19:0/3853210510 shutdown_connections 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 --2- 172.21.15.19:0/3853210510 >> v2:172.21.15.169:3300/0 conn(0x7f39500b4cd0 0x7f3950161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 --2- 172.21.15.19:0/3853210510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3950144370 0x7f39500b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 --2- 172.21.15.19:0/3853210510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f395006c5c0 0x7f39500cfa70 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 -- 172.21.15.19:0/3853210510 >> 172.21.15.19:0/3853210510 conn(0x7f39500d1720 msgr2=0x7f39500d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 -- 172.21.15.19:0/3853210510 shutdown_connections 2024-04-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.635+0000 7f3955a00640 1 -- 172.21.15.19:0/3853210510 wait complete. 2024-04-25T11:41:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.638+0000 7f3955a00640 1 Processor -- start 2024-04-25T11:41:15.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.638+0000 7f3955a00640 1 -- start start 2024-04-25T11:41:15.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f395006c5c0 0x7f39501f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39500b4cd0 0x7f3950142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3950144370 0x7f3950142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f39500d02d0 con 0x7f3950144370 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f39500d0f50 con 0x7f39500b4cd0 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f39500d0450 con 0x7f395006c5c0 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3950144370 0x7f3950142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3950144370 0x7f3950142ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49014/0 (socket says 172.21.15.19:49014) 2024-04-25T11:41:15.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.640+0000 7f3955000640 1 -- 172.21.15.19:0/3349555810 learned_addr learned my addr 172.21.15.19:0/3349555810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f394fe00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f395006c5c0 0x7f39501f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f394f400640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39500b4cd0 0x7f3950142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:15.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3955000640 1 -- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f395006c5c0 msgr2=0x7f39501f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3955000640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f395006c5c0 0x7f39501f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3955000640 1 -- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39500b4cd0 msgr2=0x7f3950142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3955000640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39500b4cd0 0x7f3950142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3955000640 1 -- 172.21.15.19:0/3349555810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3950144b20 con 0x7f3950144370 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3955000640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3950144370 0x7f3950142ec0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f394800b410 tx=0x7f394800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3947e00640 1 -- 172.21.15.19:0/3349555810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948015020 con 0x7f3950144370 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.641+0000 7f3947e00640 1 -- 172.21.15.19:0/3349555810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3948011850 con 0x7f3950144370 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.642+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3950147dc0 con 0x7f3950144370 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.642+0000 7f3947e00640 1 -- 172.21.15.19:0/3349555810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948011c40 con 0x7f3950144370 2024-04-25T11:41:15.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.642+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3950148380 con 0x7f3950144370 2024-04-25T11:41:15.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.643+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3950144cf0 con 0x7f3950144370 2024-04-25T11:41:15.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.644+0000 7f3947e00640 1 -- 172.21.15.19:0/3349555810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3948018020 con 0x7f3950144370 2024-04-25T11:41:15.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.645+0000 7f3947e00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f39300744b0 0x7f39300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:15.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.645+0000 7f3947e00640 1 -- 172.21.15.19:0/3349555810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8719+0+0 (secure 0 0 0) 0x7f3948095cd0 con 0x7f3950144370 2024-04-25T11:41:15.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.646+0000 7f394fe00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f39300744b0 0x7f39300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:15.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.647+0000 7f394fe00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f39300744b0 0x7f39300769c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3940033000 tx=0x7f3940009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:15.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.650+0000 7f3947e00640 1 -- 172.21.15.19:0/3349555810 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f394801f020 con 0x7f3950144370 2024-04-25T11:41:15.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.895+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3950143460 con 0x7f3950144370 2024-04-25T11:41:15.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.896+0000 7f3947e00640 1 -- 172.21.15.19:0/3349555810 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 320 v320) ==== 95+0+5187 (secure 0 0 0) 0x7f394805f880 con 0x7f3950144370 2024-04-25T11:41:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:15.884 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":320,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":320,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:15.321587+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10812":{"gid":10812,"name":"a","rank":0,"incarnation":320,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/295899246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":295899246},{"type":"v1","addr":"172.21.15.19:6835","nonce":295899246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:15.885 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 320 2024-04-25T11:41:15.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f39300744b0 msgr2=0x7f39300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f39300744b0 0x7f39300769c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3940033000 tx=0x7f3940009000 comp rx=0 tx=0).stop 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3950144370 msgr2=0x7f3950142ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3950144370 0x7f3950142ec0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f394800b410 tx=0x7f394800b8d0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 shutdown_connections 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f39300744b0 0x7f39300769c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39500b4cd0 0x7f3950142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3950144370 0x7f3950142ec0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 --2- 172.21.15.19:0/3349555810 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f395006c5c0 0x7f39501f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 >> 172.21.15.19:0/3349555810 conn(0x7f39500d1720 msgr2=0x7f395015daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 shutdown_connections 2024-04-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:15.899+0000 7f3955a00640 1 -- 172.21.15.19:0/3349555810 wait complete. 2024-04-25T11:41:15.913 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:41:18.352 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:18.365+0000 7f0edc800640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:41:18.353 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:18.365+0000 7f0edc800640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-25T11:41:18.353 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:18.366+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:41:18.353 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:18.366+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:41:18.371 INFO:tasks.ceph.mds.a.smithi019.stderr: -17> 2024-04-25T11:41:18.365+0000 7f0edc800640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:41:18.371 INFO:tasks.ceph.mds.a.smithi019.stderr: -14> 2024-04-25T11:41:18.365+0000 7f0edc800640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-04-25T11:41:18.509 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:41:20.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.213+0000 7fc3dd400640 1 Processor -- start 2024-04-25T11:41:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.213+0000 7fc3dd400640 1 -- start start 2024-04-25T11:41:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.215+0000 7fc3dd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3d80b0a90 0x7fc3d815d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.215+0000 7fc3dd400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3d806de90 con 0x7fc3d815d120 2024-04-25T11:41:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.215+0000 7fc3dd400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3d80b2cf0 con 0x7fc3d8075e80 2024-04-25T11:41:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.215+0000 7fc3dd400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc3d80b3340 con 0x7fc3d80b0a90 2024-04-25T11:41:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.216+0000 7fc3dca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3d80b0a90 0x7fc3d815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.216+0000 7fc3d7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc3d815d120 0x7fc3d80b0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59038/0 (socket says 172.21.15.19:59038) 2024-04-25T11:41:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.216+0000 7fc3dca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3d80b0a90 0x7fc3d815d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49032/0 (socket says 172.21.15.19:49032) 2024-04-25T11:41:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.216+0000 7fc3d7400640 1 -- 172.21.15.19:0/3050039702 learned_addr learned my addr 172.21.15.19:0/3050039702 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.217+0000 7fc3dca00640 1 -- 172.21.15.19:0/3050039702 >> v1:172.21.15.169:6789/0 conn(0x7fc3d8075e80 legacy=0x7fc3d806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.217+0000 7fc3dca00640 1 -- 172.21.15.19:0/3050039702 >> v1:172.21.15.19:6789/0 conn(0x7fc3d815d120 legacy=0x7fc3d80b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.217+0000 7fc3dca00640 1 -- 172.21.15.19:0/3050039702 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d80b2fd0 con 0x7fc3d80b0a90 2024-04-25T11:41:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.218+0000 7fc3dca00640 1 --2- 172.21.15.19:0/3050039702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d80b0a90 0x7fc3d815d530 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009dc0 tx=0x7fc3d002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb386942fc2f8a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:21.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.218+0000 7fc3d6a00640 1 -- 172.21.15.19:0/3050039702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0007740 con 0x7fc3d80b0a90 2024-04-25T11:41:21.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.218+0000 7fc3d6a00640 1 -- 172.21.15.19:0/3050039702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d00078e0 con 0x7fc3d80b0a90 2024-04-25T11:41:21.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.218+0000 7fc3d6a00640 1 -- 172.21.15.19:0/3050039702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0007cb0 con 0x7fc3d80b0a90 2024-04-25T11:41:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.219+0000 7fc3dd400640 1 -- 172.21.15.19:0/3050039702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d80b0a90 msgr2=0x7fc3d815d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.219+0000 7fc3dd400640 1 --2- 172.21.15.19:0/3050039702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d80b0a90 0x7fc3d815d530 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009dc0 tx=0x7fc3d002a730 comp rx=0 tx=0).stop 2024-04-25T11:41:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.219+0000 7fc3dd400640 1 -- 172.21.15.19:0/3050039702 shutdown_connections 2024-04-25T11:41:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.219+0000 7fc3dd400640 1 --2- 172.21.15.19:0/3050039702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d80b0a90 0x7fc3d815d530 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.219+0000 7fc3dd400640 1 -- 172.21.15.19:0/3050039702 >> 172.21.15.19:0/3050039702 conn(0x7fc3d8073950 msgr2=0x7fc3d80b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.219+0000 7fc3dd400640 1 -- 172.21.15.19:0/3050039702 shutdown_connections 2024-04-25T11:41:21.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.219+0000 7fc3dd400640 1 -- 172.21.15.19:0/3050039702 wait complete. 2024-04-25T11:41:21.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.221+0000 7fc3dd400640 1 Processor -- start 2024-04-25T11:41:21.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.221+0000 7fc3dd400640 1 -- start start 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dd400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3d8075e80 0x7fc3d8152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dd400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3d80b0a90 0x7fc3d814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d815d120 0x7fc3d80d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dd400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d806c740 con 0x7fc3d815d120 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dd400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d806c5c0 con 0x7fc3d80b0a90 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dd400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d806c8c0 con 0x7fc3d8075e80 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d815d120 0x7fc3d80d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d815d120 0x7fc3d80d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49034/0 (socket says 172.21.15.19:49034) 2024-04-25T11:41:21.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.223+0000 7fc3dca00640 1 -- 172.21.15.19:0/2234386525 learned_addr learned my addr 172.21.15.19:0/2234386525 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:21.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3d7e00640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3d8075e80 0x7fc3d8152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3d7400640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3d80b0a90 0x7fc3d814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3dca00640 1 -- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3d8075e80 msgr2=0x7fc3d8152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3dca00640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3d8075e80 0x7fc3d8152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3dca00640 1 -- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3d80b0a90 msgr2=0x7fc3d814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3dca00640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3d80b0a90 0x7fc3d814bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3dca00640 1 -- 172.21.15.19:0/2234386525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d809ddc0 con 0x7fc3d815d120 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.224+0000 7fc3dca00640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d815d120 0x7fc3d80d0970 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009ee0 tx=0x7fc3d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.225+0000 7fc3cfe00640 1 -- 172.21.15.19:0/2234386525 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0040070 con 0x7fc3d815d120 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.225+0000 7fc3cfe00640 1 -- 172.21.15.19:0/2234386525 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d0031bc0 con 0x7fc3d815d120 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.225+0000 7fc3cfe00640 1 -- 172.21.15.19:0/2234386525 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0045460 con 0x7fc3d815d120 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.225+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d809e0a0 con 0x7fc3d815d120 2024-04-25T11:41:21.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.225+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d80d0450 con 0x7fc3d815d120 2024-04-25T11:41:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.226+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d8050740 con 0x7fc3d815d120 2024-04-25T11:41:21.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.227+0000 7fc3cfe00640 1 -- 172.21.15.19:0/2234386525 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc3d0039040 con 0x7fc3d815d120 2024-04-25T11:41:21.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.228+0000 7fc3cfe00640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3b80744b0 0x7fc3b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.228+0000 7fc3cfe00640 1 -- 172.21.15.19:0/2234386525 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7fc3d00b78a0 con 0x7fc3d815d120 2024-04-25T11:41:21.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.229+0000 7fc3d7e00640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3b80744b0 0x7fc3b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.230+0000 7fc3d7e00640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3b80744b0 0x7fc3b80769c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8007fd0 tx=0x7fc3c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:21.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.233+0000 7fc3cfe00640 1 -- 172.21.15.19:0/2234386525 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3d00813f0 con 0x7fc3d815d120 2024-04-25T11:41:21.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.484+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3d80b2320 con 0x7fc3d815d120 2024-04-25T11:41:21.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.485+0000 7fc3cfe00640 1 -- 172.21.15.19:0/2234386525 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 325 v325) ==== 95+0+5179 (secure 0 0 0) 0x7fc3d0080cf0 con 0x7fc3d815d120 2024-04-25T11:41:21.472 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:21.472 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":325,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3563649842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3563649842},{"type":"v1","addr":"172.21.15.19:6835","nonce":3563649842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":325}],"filesystems":[{"mdsmap":{"epoch":324,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:18.367783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:21.472 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 325 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3b80744b0 msgr2=0x7fc3b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3b80744b0 0x7fc3b80769c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8007fd0 tx=0x7fc3c8009040 comp rx=0 tx=0).stop 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d815d120 msgr2=0x7fc3d80d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d815d120 0x7fc3d80d0970 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009ee0 tx=0x7fc3d0004a40 comp rx=0 tx=0).stop 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 shutdown_connections 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc3b80744b0 0x7fc3b80769c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3d80b0a90 0x7fc3d814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3d815d120 0x7fc3d80d0970 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 --2- 172.21.15.19:0/2234386525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc3d8075e80 0x7fc3d8152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 >> 172.21.15.19:0/2234386525 conn(0x7fc3d8073950 msgr2=0x7fc3d80b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 shutdown_connections 2024-04-25T11:41:21.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.488+0000 7fc3dd400640 1 -- 172.21.15.19:0/2234386525 wait complete. 2024-04-25T11:41:21.500 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:41:21.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.790+0000 7f96b6a00640 1 Processor -- start 2024-04-25T11:41:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.791+0000 7f96b6a00640 1 -- start start 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.793+0000 7f96b6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b00b4ea0 0x7f96b00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.793+0000 7f96b6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96b0144370 0x7f96b015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.793+0000 7f96b6a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b00d1230 con 0x7f96b006c5c0 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.793+0000 7f96b6a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f96b00d06b0 con 0x7f96b0144370 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.793+0000 7f96b6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96b00d0990 con 0x7f96b00b4ea0 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.793+0000 7f96b6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96b0144370 0x7f96b015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.794+0000 7f96b6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96b0144370 0x7f96b015e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49044/0 (socket says 172.21.15.19:49044) 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.794+0000 7f96b6000640 1 -- 172.21.15.19:0/99620521 learned_addr learned my addr 172.21.15.19:0/99620521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.794+0000 7f96b4c00640 1 --2- 172.21.15.19:0/99620521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b00b4ea0 0x7f96b00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.794+0000 7f96b6000640 1 -- 172.21.15.19:0/99620521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b00b4ea0 msgr2=0x7f96b00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.794+0000 7f96b6000640 1 --2- 172.21.15.19:0/99620521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b00b4ea0 0x7f96b00cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.794+0000 7f96b6000640 1 -- 172.21.15.19:0/99620521 >> v1:172.21.15.169:6789/0 conn(0x7f96b006c5c0 legacy=0x7f96b00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.794+0000 7f96b6000640 1 -- 172.21.15.19:0/99620521 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b00d0c70 con 0x7f96b0144370 2024-04-25T11:41:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.795+0000 7f96b6000640 1 --2- 172.21.15.19:0/99620521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b0144370 0x7f96b015e770 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009e50 tx=0x7f96a802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f954b125e92578df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96afe00640 1 -- 172.21.15.19:0/99620521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8036070 con 0x7f96b0144370 2024-04-25T11:41:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96afe00640 1 -- 172.21.15.19:0/99620521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a8031600 con 0x7f96b0144370 2024-04-25T11:41:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96afe00640 1 -- 172.21.15.19:0/99620521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80319d0 con 0x7f96b0144370 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96b6a00640 1 -- 172.21.15.19:0/99620521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b0144370 msgr2=0x7f96b015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96b6a00640 1 --2- 172.21.15.19:0/99620521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b0144370 0x7f96b015e770 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009e50 tx=0x7f96a802a730 comp rx=0 tx=0).stop 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96b6a00640 1 -- 172.21.15.19:0/99620521 shutdown_connections 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96b6a00640 1 --2- 172.21.15.19:0/99620521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b0144370 0x7f96b015e770 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96b6a00640 1 --2- 172.21.15.19:0/99620521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b00b4ea0 0x7f96b00cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.796+0000 7f96b6a00640 1 -- 172.21.15.19:0/99620521 >> 172.21.15.19:0/99620521 conn(0x7f96b00d1720 msgr2=0x7f96b00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.797+0000 7f96b6a00640 1 -- 172.21.15.19:0/99620521 shutdown_connections 2024-04-25T11:41:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.797+0000 7f96b6a00640 1 -- 172.21.15.19:0/99620521 wait complete. 2024-04-25T11:41:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.799+0000 7f96b6a00640 1 Processor -- start 2024-04-25T11:41:21.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.799+0000 7f96b6a00640 1 -- start start 2024-04-25T11:41:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b006c5c0 0x7f96b006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b6a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b0144370 0x7f96b01609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b013fc50 0x7f96b00b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b6a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96b00d0450 con 0x7f96b006c5c0 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f96b00d02d0 con 0x7f96b013fc50 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b6a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f96b0147760 con 0x7f96b0144370 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b006c5c0 0x7f96b006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b006c5c0 0x7f96b006fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49058/0 (socket says 172.21.15.19:49058) 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b5600640 1 -- 172.21.15.19:0/2912484828 learned_addr learned my addr 172.21.15.19:0/2912484828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b4c00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b0144370 0x7f96b01609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.801+0000 7f96b6000640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b013fc50 0x7f96b00b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b5600640 1 -- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b0144370 msgr2=0x7f96b01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b5600640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b0144370 0x7f96b01609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b5600640 1 -- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b013fc50 msgr2=0x7f96b00b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b5600640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b013fc50 0x7f96b00b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b5600640 1 -- 172.21.15.19:0/2912484828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b0070b40 con 0x7f96b006c5c0 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b5600640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b006c5c0 0x7f96b006fa20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c3c0 tx=0x7f96a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96ad600640 1 -- 172.21.15.19:0/2912484828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a40052d0 con 0x7f96b006c5c0 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96ad600640 1 -- 172.21.15.19:0/2912484828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4005490 con 0x7f96b006c5c0 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b0070e20 con 0x7f96b006c5c0 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96ad600640 1 -- 172.21.15.19:0/2912484828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4007260 con 0x7f96b006c5c0 2024-04-25T11:41:21.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.802+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b006f220 con 0x7f96b006c5c0 2024-04-25T11:41:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.804+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b006d150 con 0x7f96b006c5c0 2024-04-25T11:41:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.805+0000 7f96ad600640 1 -- 172.21.15.19:0/2912484828 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f96a4007400 con 0x7f96b006c5c0 2024-04-25T11:41:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.807+0000 7f96ad600640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96900744e0 0x7f96900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:21.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.807+0000 7f96ad600640 1 -- 172.21.15.19:0/2912484828 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f96a40952d0 con 0x7f96b006c5c0 2024-04-25T11:41:21.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.807+0000 7f96b4c00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96900744e0 0x7f96900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:21.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.809+0000 7f96b4c00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96900744e0 0x7f96900769f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f969c006fd0 tx=0x7f969c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:21.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:21.812+0000 7f96ad600640 1 -- 172.21.15.19:0/2912484828 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96b006d150 con 0x7f96b006c5c0 2024-04-25T11:41:22.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.058+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96b0073d10 con 0x7f96b006c5c0 2024-04-25T11:41:22.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.059+0000 7f96ad600640 1 -- 172.21.15.19:0/2912484828 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 325 v325) ==== 95+0+5179 (secure 0 0 0) 0x7f96b0073d10 con 0x7f96b006c5c0 2024-04-25T11:41:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:22.046 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":325,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3563649842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3563649842},{"type":"v1","addr":"172.21.15.19:6835","nonce":3563649842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":325}],"filesystems":[{"mdsmap":{"epoch":324,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:18.367783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:22.046 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 325 2024-04-25T11:41:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.062+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96900744e0 msgr2=0x7f96900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.062+0000 7f96b6a00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96900744e0 0x7f96900769f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f969c006fd0 tx=0x7f969c008040 comp rx=0 tx=0).stop 2024-04-25T11:41:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.062+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b006c5c0 msgr2=0x7f96b006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.062+0000 7f96b6a00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b006c5c0 0x7f96b006fa20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c3c0 tx=0x7f96a400c880 comp rx=0 tx=0).stop 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 shutdown_connections 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f96900744e0 0x7f96900769f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b013fc50 0x7f96b00b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b0144370 0x7f96b01609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 --2- 172.21.15.19:0/2912484828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b006c5c0 0x7f96b006fa20 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 >> 172.21.15.19:0/2912484828 conn(0x7f96b00d1720 msgr2=0x7f96b00b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 shutdown_connections 2024-04-25T11:41:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.063+0000 7f96b6a00640 1 -- 172.21.15.19:0/2912484828 wait complete. 2024-04-25T11:41:22.076 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:41:22.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.371+0000 7ff348e00640 1 Processor -- start 2024-04-25T11:41:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.371+0000 7ff348e00640 1 -- start start 2024-04-25T11:41:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.373+0000 7ff348e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff34406e680 0x7ff3440b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.373+0000 7ff348e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3440b5290 con 0x7ff34408cd60 2024-04-25T11:41:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.373+0000 7ff348e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3440d1fa0 con 0x7ff3440b4e80 2024-04-25T11:41:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.373+0000 7ff348e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff3440d1d70 con 0x7ff34406e680 2024-04-25T11:41:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.374+0000 7ff343400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff34406e680 0x7ff3440b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.374+0000 7ff343e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff34408cd60 0x7ff34408d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59048/0 (socket says 172.21.15.19:59048) 2024-04-25T11:41:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.374+0000 7ff343e00640 1 -- 172.21.15.19:0/2533010554 learned_addr learned my addr 172.21.15.19:0/2533010554 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.374+0000 7ff343400640 1 -- 172.21.15.19:0/2533010554 >> v1:172.21.15.169:6789/0 conn(0x7ff3440b4e80 legacy=0x7ff3440cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.374+0000 7ff343400640 1 -- 172.21.15.19:0/2533010554 >> v1:172.21.15.19:6789/0 conn(0x7ff34408cd60 legacy=0x7ff34408d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.374+0000 7ff342000640 1 -- 172.21.15.19:0/2533010554 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667236138 0 0) 0x7ff3440b5290 con 0x7ff34408cd60 2024-04-25T11:41:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.374+0000 7ff343400640 1 -- 172.21.15.19:0/2533010554 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3440d2280 con 0x7ff34406e680 2024-04-25T11:41:22.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.376+0000 7ff343400640 1 --2- 172.21.15.19:0/2533010554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34406e680 0x7ff3440b90e0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7ff334009d80 tx=0x7ff33402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd6061f25014a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:22.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.376+0000 7ff342000640 1 -- 172.21.15.19:0/2533010554 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334007740 con 0x7ff34406e680 2024-04-25T11:41:22.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.376+0000 7ff342000640 1 -- 172.21.15.19:0/2533010554 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3340078e0 con 0x7ff34406e680 2024-04-25T11:41:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.376+0000 7ff342000640 1 -- 172.21.15.19:0/2533010554 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334007cb0 con 0x7ff34406e680 2024-04-25T11:41:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.377+0000 7ff348e00640 1 -- 172.21.15.19:0/2533010554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34406e680 msgr2=0x7ff3440b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.377+0000 7ff348e00640 1 --2- 172.21.15.19:0/2533010554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34406e680 0x7ff3440b90e0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7ff334009d80 tx=0x7ff33402a660 comp rx=0 tx=0).stop 2024-04-25T11:41:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.377+0000 7ff348e00640 1 -- 172.21.15.19:0/2533010554 shutdown_connections 2024-04-25T11:41:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.377+0000 7ff348e00640 1 --2- 172.21.15.19:0/2533010554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34406e680 0x7ff3440b90e0 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.377+0000 7ff348e00640 1 -- 172.21.15.19:0/2533010554 >> 172.21.15.19:0/2533010554 conn(0x7ff3440b5460 msgr2=0x7ff3440b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.377+0000 7ff348e00640 1 -- 172.21.15.19:0/2533010554 shutdown_connections 2024-04-25T11:41:22.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.377+0000 7ff348e00640 1 -- 172.21.15.19:0/2533010554 wait complete. 2024-04-25T11:41:22.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.379+0000 7ff348e00640 1 Processor -- start 2024-04-25T11:41:22.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.379+0000 7ff348e00640 1 -- start start 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff348e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34406e680 0x7ff34415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff348e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34408cd60 0x7ff3440c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff348e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3440b4e80 0x7ff3440c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff348e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3440d2840 con 0x7ff34408cd60 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff348e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff3440ce6d0 con 0x7ff34406e680 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff348e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff34406ec40 con 0x7ff3440b4e80 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34408cd60 0x7ff3440c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34408cd60 0x7ff3440c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49082/0 (socket says 172.21.15.19:49082) 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 -- 172.21.15.19:0/1099025541 learned_addr learned my addr 172.21.15.19:0/1099025541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff343e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3440b4e80 0x7ff3440c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff343400640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34406e680 0x7ff34415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 -- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3440b4e80 msgr2=0x7ff3440c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3440b4e80 0x7ff3440c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 -- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34406e680 msgr2=0x7ff34415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34406e680 0x7ff34415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff342a00640 1 -- 172.21.15.19:0/1099025541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3440c8370 con 0x7ff34408cd60 2024-04-25T11:41:22.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.381+0000 7ff343e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3440b4e80 0x7ff3440c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.382+0000 7ff342a00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34408cd60 0x7ff3440c75c0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c3f0 tx=0x7ff33000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.382+0000 7ff33b400640 1 -- 172.21.15.19:0/1099025541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330017070 con 0x7ff34408cd60 2024-04-25T11:41:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.382+0000 7ff33b400640 1 -- 172.21.15.19:0/1099025541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3300052d0 con 0x7ff34408cd60 2024-04-25T11:41:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.382+0000 7ff33b400640 1 -- 172.21.15.19:0/1099025541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330003830 con 0x7ff34408cd60 2024-04-25T11:41:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.382+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3440c8650 con 0x7ff34408cd60 2024-04-25T11:41:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.383+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3440c8c10 con 0x7ff34408cd60 2024-04-25T11:41:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.384+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3440c80e0 con 0x7ff34408cd60 2024-04-25T11:41:22.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.385+0000 7ff33b400640 1 -- 172.21.15.19:0/1099025541 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff33001a070 con 0x7ff34408cd60 2024-04-25T11:41:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.386+0000 7ff33b400640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3240744b0 0x7ff3240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.387+0000 7ff33b400640 1 -- 172.21.15.19:0/1099025541 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7ff330095120 con 0x7ff34408cd60 2024-04-25T11:41:22.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.387+0000 7ff343400640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3240744b0 0x7ff3240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.389+0000 7ff343400640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3240744b0 0x7ff3240769c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:22.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.393+0000 7ff33b400640 1 -- 172.21.15.19:0/1099025541 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff33005ec70 con 0x7ff34408cd60 2024-04-25T11:41:22.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.633+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff344050740 con 0x7ff34408cd60 2024-04-25T11:41:22.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.634+0000 7ff33b400640 1 -- 172.21.15.19:0/1099025541 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 325 v325) ==== 95+0+5179 (secure 0 0 0) 0x7ff33005e570 con 0x7ff34408cd60 2024-04-25T11:41:22.621 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:22.621 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":325,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":10863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3563649842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3563649842},{"type":"v1","addr":"172.21.15.19:6835","nonce":3563649842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":325}],"filesystems":[{"mdsmap":{"epoch":324,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:18.367783+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:22.621 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 325 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3240744b0 msgr2=0x7ff3240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3240744b0 0x7ff3240769c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff334034000 tx=0x7ff33402f000 comp rx=0 tx=0).stop 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34408cd60 msgr2=0x7ff3440c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34408cd60 0x7ff3440c75c0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c3f0 tx=0x7ff33000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 shutdown_connections 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff3240744b0 0x7ff3240769c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3440b4e80 0x7ff3440c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff34408cd60 0x7ff3440c75c0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 --2- 172.21.15.19:0/1099025541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff34406e680 0x7ff34415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 >> 172.21.15.19:0/1099025541 conn(0x7ff3440b5460 msgr2=0x7ff3440b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:22.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.636+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 shutdown_connections 2024-04-25T11:41:22.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:22.637+0000 7ff348e00640 1 -- 172.21.15.19:0/1099025541 wait complete. 2024-04-25T11:41:22.651 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2024-04-25T11:41:22.651 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2024-04-25T11:41:22.652 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:22.652 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:22.701 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:41:22.701 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:41:22.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:22.702 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:41:22.798 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:41:22.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:22.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:41:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.102+0000 7f1e2c400640 1 Processor -- start 2024-04-25T11:41:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.102+0000 7f1e2c400640 1 -- start start 2024-04-25T11:41:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.104+0000 7f1e2c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 0x7f1e240cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.104+0000 7f1e2c400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e240cff30 con 0x7f1e2406c5c0 2024-04-25T11:41:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.104+0000 7f1e2c400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e2415f5f0 con 0x7f1e241443c0 2024-04-25T11:41:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.104+0000 7f1e2c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1e240708d0 con 0x7f1e240cf5e0 2024-04-25T11:41:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.105+0000 7f1e2a400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1e2406c5c0 0x7f1e2406c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59060/0 (socket says 172.21.15.19:59060) 2024-04-25T11:41:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.105+0000 7f1e2a400640 1 -- 172.21.15.19:0/4163805471 learned_addr learned my addr 172.21.15.19:0/4163805471 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.105+0000 7f1e29000640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 0x7f1e240cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.105+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609357442 0 0) 0x7f1e1c001490 con 0x7f1e2406c5c0 2024-04-25T11:41:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e0c003520 con 0x7f1e2406c5c0 2024-04-25T11:41:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1416365531 0 0) 0x7f1e0c003520 con 0x7f1e2406c5c0 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 msgr2=0x7f1e240cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 0x7f1e240cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 >> v1:172.21.15.169:6789/0 conn(0x7f1e241443c0 legacy=0x7f1e2406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2415e830 con 0x7f1e2406c5c0 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1e1c001bc0 con 0x7f1e2406c5c0 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 >> v1:172.21.15.19:6789/0 conn(0x7f1e2406c5c0 legacy=0x7f1e2406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e0c0039e0 0x7f1e0c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 0x7f1e0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0c004d40 0x7f1e0c0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e240708d0 con 0x7f1e0c004d40 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c00bb50 con 0x7f1e0c0039e0 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.106+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c00bd50 con 0x7f1e240cf5e0 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e2a400640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0c004d40 0x7f1e0c0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e29000640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 0x7f1e0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e29a00640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e0c0039e0 0x7f1e0c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e2a400640 1 -- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 msgr2=0x7f1e0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:23.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e2a400640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e240cf5e0 0x7f1e0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e2a400640 1 -- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e0c0039e0 msgr2=0x7f1e0c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:23.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e2a400640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e0c0039e0 0x7f1e0c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.107+0000 7f1e2a400640 1 -- 172.21.15.19:0/4163805471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c003520 con 0x7f1e0c004d40 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.108+0000 7f1e2a400640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0c004d40 0x7f1e0c0051b0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00a920 tx=0x7f1e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2b134685515575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.108+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00a570 con 0x7f1e0c004d40 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c02f070 con 0x7f1e0c004d40 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e23e00640 1 -- 172.21.15.19:0/4163805471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c02bd90 con 0x7f1e0c004d40 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e2c400640 1 -- 172.21.15.19:0/4163805471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0c004d40 msgr2=0x7f1e0c0051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e2c400640 1 --2- 172.21.15.19:0/4163805471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0c004d40 0x7f1e0c0051b0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00a920 tx=0x7f1e0c005830 comp rx=0 tx=0).stop 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e2a400640 1 -- 172.21.15.19:0/4163805471 reap_dead start 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e2c400640 1 -- 172.21.15.19:0/4163805471 shutdown_connections 2024-04-25T11:41:23.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e2c400640 1 -- 172.21.15.19:0/4163805471 >> 172.21.15.19:0/4163805471 conn(0x7f1e240d1ae0 msgr2=0x7f1e240d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:23.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.109+0000 7f1e2c400640 1 -- 172.21.15.19:0/4163805471 shutdown_connections 2024-04-25T11:41:23.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.110+0000 7f1e2c400640 1 -- 172.21.15.19:0/4163805471 wait complete. 2024-04-25T11:41:23.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.112+0000 7f1e2c400640 1 Processor -- start 2024-04-25T11:41:23.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.112+0000 7f1e2c400640 1 -- start start 2024-04-25T11:41:23.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.113+0000 7f1e2c400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e0c004d40 0x7f1e2414cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.113+0000 7f1e2c400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e2413f950 0x7f1e2413c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e2c400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24149f90 0x7f1e2413dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e2c400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2415edf0 con 0x7f1e2413f950 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e2c400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2415f160 con 0x7f1e24149f90 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e2c400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1e241447d0 con 0x7f1e0c004d40 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e2413f950 0x7f1e2413c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e2413f950 0x7f1e2413c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49100/0 (socket says 172.21.15.19:49100) 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 -- 172.21.15.19:0/3065649962 learned_addr learned my addr 172.21.15.19:0/3065649962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 -- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e0c004d40 msgr2=0x7f1e2414cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e0c004d40 0x7f1e2414cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 -- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24149f90 msgr2=0x7f1e2413dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24149f90 0x7f1e2413dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.114+0000 7f1e29000640 1 -- 172.21.15.19:0/3065649962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2413ccb0 con 0x7f1e2413f950 2024-04-25T11:41:23.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.115+0000 7f1e29000640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e2413f950 0x7f1e2413c340 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f1e100115d0 tx=0x7f1e10011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:23.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.115+0000 7f1e21600640 1 -- 172.21.15.19:0/3065649962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10015870 con 0x7f1e2413f950 2024-04-25T11:41:23.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.115+0000 7f1e21600640 1 -- 172.21.15.19:0/3065649962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e10015a10 con 0x7f1e2413f950 2024-04-25T11:41:23.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.115+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2413cf90 con 0x7f1e2413f950 2024-04-25T11:41:23.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.115+0000 7f1e21600640 1 -- 172.21.15.19:0/3065649962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10015e00 con 0x7f1e2413f950 2024-04-25T11:41:23.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.115+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2413d550 con 0x7f1e2413f950 2024-04-25T11:41:23.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.116+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e24050740 con 0x7f1e2413f950 2024-04-25T11:41:23.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.118+0000 7f1e21600640 1 -- 172.21.15.19:0/3065649962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e1001d400 con 0x7f1e2413f950 2024-04-25T11:41:23.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.119+0000 7f1e21600640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e040744b0 0x7f1e040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.119+0000 7f1e21600640 1 -- 172.21.15.19:0/3065649962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f1e10096650 con 0x7f1e2413f950 2024-04-25T11:41:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.120+0000 7f1e29a00640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e040744b0 0x7f1e040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.121+0000 7f1e29a00640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e040744b0 0x7f1e040769c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1e18007fd0 tx=0x7f1e18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:23.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.124+0000 7f1e21600640 1 -- 172.21.15.19:0/3065649962 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e10019030 con 0x7f1e2413f950 2024-04-25T11:41:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.376+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1e2415ccd0 con 0x7f1e2413f950 2024-04-25T11:41:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.397+0000 7f1e21600640 1 -- 172.21.15.19:0/3065649962 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v326) ==== 161+0+0 (secure 0 0 0) 0x7f1e10060200 con 0x7f1e2413f950 2024-04-25T11:41:23.383 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:41:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.399+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e040744b0 msgr2=0x7f1e040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.399+0000 7f1e2c400640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e040744b0 0x7f1e040769c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1e18007fd0 tx=0x7f1e18009040 comp rx=0 tx=0).stop 2024-04-25T11:41:23.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.399+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e2413f950 msgr2=0x7f1e2413c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.399+0000 7f1e2c400640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e2413f950 0x7f1e2413c340 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f1e100115d0 tx=0x7f1e10011a90 comp rx=0 tx=0).stop 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 shutdown_connections 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1e040744b0 0x7f1e040769c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1e24149f90 0x7f1e2413dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e2413f950 0x7f1e2413c340 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 --2- 172.21.15.19:0/3065649962 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1e0c004d40 0x7f1e2414cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 >> 172.21.15.19:0/3065649962 conn(0x7f1e240d1ae0 msgr2=0x7f1e2415da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 shutdown_connections 2024-04-25T11:41:23.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.400+0000 7f1e2c400640 1 -- 172.21.15.19:0/3065649962 wait complete. 2024-04-25T11:41:23.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:41:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.716+0000 7f14cb400640 1 Processor -- start 2024-04-25T11:41:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.716+0000 7f14cb400640 1 -- start start 2024-04-25T11:41:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.718+0000 7f14cb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c41434d0 0x7f14c4070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.718+0000 7f14cb400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14c40d0ed0 0x7f14c40d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.718+0000 7f14cb400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c40709c0 con 0x7f14c4160be0 2024-04-25T11:41:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.718+0000 7f14cb400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f14c41438e0 con 0x7f14c40d0ed0 2024-04-25T11:41:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.718+0000 7f14cb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14c406f3e0 con 0x7f14c41434d0 2024-04-25T11:41:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.719+0000 7f14c8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c41434d0 0x7f14c4070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.719+0000 7f14c9400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f14c4160be0 0x7f14c4160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:59868/0 (socket says 172.21.15.19:59868) 2024-04-25T11:41:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.719+0000 7f14c8a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c41434d0 0x7f14c4070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:45680/0 (socket says 172.21.15.19:45680) 2024-04-25T11:41:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.719+0000 7f14c9400640 1 -- 172.21.15.19:0/456193507 learned_addr learned my addr 172.21.15.19:0/456193507 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.719+0000 7f14c3e00640 1 --2- 172.21.15.19:0/456193507 >> v2:172.21.15.169:3300/0 conn(0x7f14c40d0ed0 0x7f14c40d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.720+0000 7f14c8a00640 1 -- 172.21.15.19:0/456193507 >> v2:172.21.15.169:3300/0 conn(0x7f14c40d0ed0 msgr2=0x7f14c40d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.720+0000 7f14c8a00640 1 --2- 172.21.15.19:0/456193507 >> v2:172.21.15.169:3300/0 conn(0x7f14c40d0ed0 0x7f14c40d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.720+0000 7f14c8a00640 1 -- 172.21.15.19:0/456193507 >> v1:172.21.15.169:6789/0 conn(0x7f14c4160be0 legacy=0x7f14c4160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.720+0000 7f14c8a00640 1 -- 172.21.15.19:0/456193507 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c406f6c0 con 0x7f14c41434d0 2024-04-25T11:41:23.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.721+0000 7f14c8a00640 1 --2- 172.21.15.19:0/456193507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c41434d0 0x7f14c4070480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14b400ba40 tx=0x7f14b402c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b4c1b85d642c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:23.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.722+0000 7f14c3400640 1 -- 172.21.15.19:0/456193507 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4032750 con 0x7f14c41434d0 2024-04-25T11:41:23.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.722+0000 7f14c3400640 1 -- 172.21.15.19:0/456193507 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b4030050 con 0x7f14c41434d0 2024-04-25T11:41:23.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.722+0000 7f14c3400640 1 -- 172.21.15.19:0/456193507 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4032b10 con 0x7f14c41434d0 2024-04-25T11:41:23.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 -- 172.21.15.19:0/456193507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c41434d0 msgr2=0x7f14c4070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 --2- 172.21.15.19:0/456193507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c41434d0 0x7f14c4070480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14b400ba40 tx=0x7f14b402c850 comp rx=0 tx=0).stop 2024-04-25T11:41:23.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 -- 172.21.15.19:0/456193507 shutdown_connections 2024-04-25T11:41:23.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 --2- 172.21.15.19:0/456193507 >> v2:172.21.15.169:3300/0 conn(0x7f14c40d0ed0 0x7f14c40d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 --2- 172.21.15.19:0/456193507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c41434d0 0x7f14c4070480 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 -- 172.21.15.19:0/456193507 >> 172.21.15.19:0/456193507 conn(0x7f14c40d1ae0 msgr2=0x7f14c40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:23.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 -- 172.21.15.19:0/456193507 shutdown_connections 2024-04-25T11:41:23.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.723+0000 7f14cb400640 1 -- 172.21.15.19:0/456193507 wait complete. 2024-04-25T11:41:23.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.725+0000 7f14cb400640 1 Processor -- start 2024-04-25T11:41:23.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.725+0000 7f14cb400640 1 -- start start 2024-04-25T11:41:23.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14cb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c40d0ed0 0x7f14c415de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14cb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c41434d0 0x7f14c4135dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14c4160be0 0x7f14c415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14cb400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14c406f090 con 0x7f14c4160be0 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14cb400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f14c406fc80 con 0x7f14c41434d0 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14cb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f14c40b51a0 con 0x7f14c40d0ed0 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14c4160be0 0x7f14c415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14c4160be0 0x7f14c415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49112/0 (socket says 172.21.15.19:49112) 2024-04-25T11:41:23.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.727+0000 7f14c9400640 1 -- 172.21.15.19:0/825116066 learned_addr learned my addr 172.21.15.19:0/825116066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c3e00640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c41434d0 0x7f14c4135dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c8a00640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c40d0ed0 0x7f14c415de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c9400640 1 -- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c40d0ed0 msgr2=0x7f14c415de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c9400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c40d0ed0 0x7f14c415de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c9400640 1 -- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c41434d0 msgr2=0x7f14c4135dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c9400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c41434d0 0x7f14c4135dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c9400640 1 -- 172.21.15.19:0/825116066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c415ef60 con 0x7f14c4160be0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c9400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14c4160be0 0x7f14c415a910 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00ca80 tx=0x7f14bc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.728+0000 7f14c0c00640 1 -- 172.21.15.19:0/825116066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc017070 con 0x7f14c4160be0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.729+0000 7f14c0c00640 1 -- 172.21.15.19:0/825116066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc00abd0 con 0x7f14c4160be0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.729+0000 7f14c0c00640 1 -- 172.21.15.19:0/825116066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc012400 con 0x7f14c4160be0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.729+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c415f240 con 0x7f14c4160be0 2024-04-25T11:41:23.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.729+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c4136410 con 0x7f14c4160be0 2024-04-25T11:41:23.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.730+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c41e6f50 con 0x7f14c4160be0 2024-04-25T11:41:23.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.731+0000 7f14c0c00640 1 -- 172.21.15.19:0/825116066 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14bc01a070 con 0x7f14c4160be0 2024-04-25T11:41:23.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.732+0000 7f14c0c00640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14a00744e0 0x7f14a00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:23.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.733+0000 7f14c0c00640 1 -- 172.21.15.19:0/825116066 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f14bc095ad0 con 0x7f14c4160be0 2024-04-25T11:41:23.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.733+0000 7f14c8a00640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14a00744e0 0x7f14a00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:23.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.735+0000 7f14c8a00640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14a00744e0 0x7f14a00769f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f14b4037000 tx=0x7f14b4032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:23.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:23.737+0000 7f14c0c00640 1 -- 172.21.15.19:0/825116066 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14bc05ef50 con 0x7f14c4160be0 2024-04-25T11:41:24.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.013+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f14c40b5760 con 0x7f14c4160be0 2024-04-25T11:41:24.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.400+0000 7f14c0c00640 1 -- 172.21.15.19:0/825116066 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v327) ==== 101+0+0 (secure 0 0 0) 0x7f14bc063250 con 0x7f14c4160be0 2024-04-25T11:41:24.387 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:41:24.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14a00744e0 msgr2=0x7f14a00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14a00744e0 0x7f14a00769f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f14b4037000 tx=0x7f14b4032000 comp rx=0 tx=0).stop 2024-04-25T11:41:24.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14c4160be0 msgr2=0x7f14c415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14c4160be0 0x7f14c415a910 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00ca80 tx=0x7f14bc00cf40 comp rx=0 tx=0).stop 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 shutdown_connections 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14a00744e0 0x7f14a00769f0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14c41434d0 0x7f14c4135dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14c4160be0 0x7f14c415a910 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 --2- 172.21.15.19:0/825116066 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14c40d0ed0 0x7f14c415de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.405+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 >> 172.21.15.19:0/825116066 conn(0x7f14c40d1ae0 msgr2=0x7f14c40b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.406+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 shutdown_connections 2024-04-25T11:41:24.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.406+0000 7f14cb400640 1 -- 172.21.15.19:0/825116066 wait complete. 2024-04-25T11:41:24.423 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:41:24.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.624+0000 7f309bd0df40 1 Processor -- start 2024-04-25T11:41:24.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.624+0000 7f309bd0df40 1 -- start start 2024-04-25T11:41:24.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.625+0000 7f309bd0df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a21e0 0x556f358a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.625+0000 7f309bd0df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a25f0 0x556f35972360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.625+0000 7f309bd0df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f35972bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.625+0000 7f309bd0df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f358eae70 con 0x556f358a25f0 2024-04-25T11:41:24.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.625+0000 7f309bd0df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556f35848d20 con 0x556f358a21e0 2024-04-25T11:41:24.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.625+0000 7f309bd0df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556f358eb3f0 con 0x556f3597bfa0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a25f0 0x556f35972360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a25f0 0x556f35972360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57642/0 (socket says 172.21.15.19:57642) 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 -- 172.21.15.19:0/2700448390 learned_addr learned my addr 172.21.15.19:0/2700448390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3095800640 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f35972bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 -- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 msgr2=0x556f35972bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f35972bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094e00640 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a21e0 0x556f358a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 -- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a21e0 msgr2=0x556f358a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a21e0 0x556f358a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3094400640 1 -- 172.21.15.19:0/2700448390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f359751c0 con 0x556f358a25f0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.626+0000 7f3095800640 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f35972bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.627+0000 7f3094400640 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a25f0 0x556f35972360 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f308000b270 tx=0x7f308000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17749ec6f4f6143c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.627+0000 7f3093000640 1 -- 172.21.15.19:0/2700448390 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080008070 con 0x556f358a25f0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.627+0000 7f3093000640 1 -- 172.21.15.19:0/2700448390 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308000d500 con 0x556f358a25f0 2024-04-25T11:41:24.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.627+0000 7f3093000640 1 -- 172.21.15.19:0/2700448390 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308000d8b0 con 0x556f358a25f0 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 -- 172.21.15.19:0/2700448390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a25f0 msgr2=0x556f35972360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a25f0 0x556f35972360 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f308000b270 tx=0x7f308000b730 comp rx=0 tx=0).stop 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 -- 172.21.15.19:0/2700448390 shutdown_connections 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f35972bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a25f0 0x556f35972360 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 --2- 172.21.15.19:0/2700448390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a21e0 0x556f358a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 -- 172.21.15.19:0/2700448390 >> 172.21.15.19:0/2700448390 conn(0x556f357c4ac0 msgr2=0x556f358d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 -- 172.21.15.19:0/2700448390 shutdown_connections 2024-04-25T11:41:24.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.628+0000 7f309bd0df40 1 -- 172.21.15.19:0/2700448390 wait complete. 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.629+0000 7f309bd0df40 1 Processor -- start 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.629+0000 7f309bd0df40 1 -- start start 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.630+0000 7f309bd0df40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a21e0 0x556f35973280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.630+0000 7f309bd0df40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a25f0 0x556f358a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.630+0000 7f309bd0df40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f359637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.630+0000 7f309bd0df40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556f358d4a30 con 0x556f358a21e0 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.630+0000 7f309bd0df40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556f359754a0 con 0x556f358a25f0 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.630+0000 7f309bd0df40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556f35970800 con 0x556f3597bfa0 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a21e0 0x556f35973280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a21e0 0x556f35973280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57652/0 (socket says 172.21.15.19:57652) 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 learned_addr learned my addr 172.21.15.19:0/3237914624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3095800640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f359637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a25f0 0x556f358a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 msgr2=0x556f359637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f359637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a25f0 msgr2=0x556f358a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a25f0 0x556f358a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f359737c0 con 0x556f358a21e0 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3095800640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556f3597bfa0 0x556f359637a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.631+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556f358a25f0 0x556f358a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.632+0000 7f3094e00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a21e0 0x556f35973280 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f308400a4a0 tx=0x7f308400a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.632+0000 7f3091c00640 1 -- 172.21.15.19:0/3237914624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084015020 con 0x556f358a21e0 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.632+0000 7f3091c00640 1 -- 172.21.15.19:0/3237914624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3084011870 con 0x556f358a21e0 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.632+0000 7f3091c00640 1 -- 172.21.15.19:0/3237914624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084011c40 con 0x556f358a21e0 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.632+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556f35972bf0 con 0x556f358a21e0 2024-04-25T11:41:24.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.632+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3596e430 con 0x556f358a21e0 2024-04-25T11:41:24.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.634+0000 7f3091c00640 1 -- 172.21.15.19:0/3237914624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3084018020 con 0x556f358a21e0 2024-04-25T11:41:24.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.635+0000 7f3091c00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3074072ec0 0x7f30740753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.635+0000 7f3091c00640 1 -- 172.21.15.19:0/3237914624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f3084094ff0 con 0x556f358a21e0 2024-04-25T11:41:24.622 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:41:24.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.636+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x556f358a0c90 0x556f3596fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.636+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f3596eed0 con 0x556f358a0c90 2024-04-25T11:41:24.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.636+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3074072ec0 0x7f30740753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.636+0000 7f3095800640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x556f358a0c90 0x556f3596fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.637+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3074072ec0 0x7f30740753d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f308000b270 tx=0x7f3080008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.638+0000 7f3095800640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x556f358a0c90 0x556f3596fdc0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.641+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v128'1097 uv1070 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.641+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a1b760 con 0x556f358a0c90 2024-04-25T11:41:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.645+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v128'1098 uv1098 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.645+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1bb00 con 0x556f358a0c90 2024-04-25T11:41:24.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.648+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v128'1099 uv1099 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.635 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:41:24.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.648+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1ceb0 con 0x556f358a0c90 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.652+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'1100 uv1100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.652+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1d250 con 0x556f358a0c90 2024-04-25T11:41:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.655+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'1101 uv1101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.655+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1bea0 con 0x556f358a0c90 2024-04-25T11:41:24.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.658+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'1102 uv1102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.658+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c290 con 0x556f358a0c90 2024-04-25T11:41:24.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.663+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v128'1103 uv1076 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.663+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a1c680 con 0x556f358a0c90 2024-04-25T11:41:24.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.666+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v128'1104 uv1104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.666+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c680 con 0x556f358a0c90 2024-04-25T11:41:24.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.669+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v128'1105 uv1105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.656 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:41:24.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.670+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.673+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'1106 uv1106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.673+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.677+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'1107 uv1107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.677+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.681+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'1108 uv1108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.681+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.685+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v128'1109 uv1083 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.685+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.689+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'1110 uv1110 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.690+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.693+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v128'1111 uv1111 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.680 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:41:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.693+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.697+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'1112 uv1112 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.697+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.701+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'1113 uv1113 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.701+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1c630 con 0x556f358a0c90 2024-04-25T11:41:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.705+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'1114 uv1114 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.706+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1ca30 con 0x556f358a0c90 2024-04-25T11:41:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.709+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'1115 uv1115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.709+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1ca30 con 0x556f358a0c90 2024-04-25T11:41:24.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.714+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v128'1116 uv1096 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.714+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a1ca30 con 0x556f358a0c90 2024-04-25T11:41:24.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.719+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v128'1117 uv1117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.719+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1ca30 con 0x556f358a0c90 2024-04-25T11:41:24.710 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:41:24.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.723+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v128'1118 uv1118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.723+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a1ca30 con 0x556f358a0c90 2024-04-25T11:41:24.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.727+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'1119 uv1119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.727+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.731+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'1120 uv1120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.732+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.736+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'1121 uv1121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.737+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.741+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v128'1122 uv1095 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.742+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.746+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v128'1123 uv1123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.746+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.751+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v128'1124 uv1124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.737 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:41:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.751+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.755+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'1125 uv1125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.756+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.760+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'1126 uv1126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.760+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f358a0c90 2024-04-25T11:41:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.765+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'1127 uv1127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.765+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556f35a1db00 0x556f35a1ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.765+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35963ce0 con 0x556f35a1db00 2024-04-25T11:41:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.766+0000 7f3094e00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556f35a1db00 0x556f35a1ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.767+0000 7f3094e00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556f35a1db00 0x556f35a1ff20 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.772+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'799 uv781 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.772+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a20460 con 0x556f35a1db00 2024-04-25T11:41:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.777+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'800 uv800 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.777+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a20800 con 0x556f35a1db00 2024-04-25T11:41:24.768 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:41:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.781+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'801 uv801 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.782+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a20ba0 con 0x556f35a1db00 2024-04-25T11:41:24.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.786+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v128'802 uv802 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.786+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a20f40 con 0x556f35a1db00 2024-04-25T11:41:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.791+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'803 uv803 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.791+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a212e0 con 0x556f35a1db00 2024-04-25T11:41:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.796+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v128'804 uv786 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.796+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a21680 con 0x556f35a1db00 2024-04-25T11:41:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.800+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v128'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.801+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.791 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:41:24.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.805+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v128'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.805+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.810+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.810+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.815+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.815+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.820+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v128'809 uv791 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.820+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.824+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v128'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.824+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.828+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v128'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.815 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:41:24.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.829+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a26110 con 0x556f35a1db00 2024-04-25T11:41:24.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.834+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v128'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.834+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a26110 con 0x556f35a1db00 2024-04-25T11:41:24.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.839+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.839+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.843+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v128'814 uv797 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.843+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.848+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v128'815 uv815 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.848+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.839 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:41:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.853+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v128'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.853+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a22a30 con 0x556f35a1db00 2024-04-25T11:41:24.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.857+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.857+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a22a30 con 0x556f35a1db00 2024-04-25T11:41:24.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.862+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.862+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a21a20 con 0x556f35a1db00 2024-04-25T11:41:24.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.867+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a1db00 2024-04-25T11:41:24.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.867+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x556f35a21f30 0x556f35a24350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.867+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a24890 con 0x556f35a21f30 2024-04-25T11:41:24.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.868+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x556f35a21f30 0x556f35a24350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.870+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x556f35a21f30 0x556f35a24350 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.874+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:24.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.874+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a24c30 con 0x556f35a21f30 2024-04-25T11:41:24.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.878+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v128'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:24.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.879+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a25050 con 0x556f35a21f30 2024-04-25T11:41:24.869 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:41:24.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.882+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v128'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:24.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.883+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a26900 con 0x556f35a21f30 2024-04-25T11:41:24.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.887+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:24.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.888+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a26ca0 con 0x556f35a21f30 2024-04-25T11:41:24.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.891+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:24.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.892+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a258f0 con 0x556f35a21f30 2024-04-25T11:41:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.896+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.896+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a25ce0 con 0x556f358a0c90 2024-04-25T11:41:24.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.900+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'161 uv160 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.900+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a25ce0 con 0x556f358a0c90 2024-04-25T11:41:24.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.904+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'162 uv162 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.905+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a25ce0 con 0x556f358a0c90 2024-04-25T11:41:24.896 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:41:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.909+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'163 uv163 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.910+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a25ce0 con 0x556f358a0c90 2024-04-25T11:41:24.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.914+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'164 uv164 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:24.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.914+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x556f35a25ce0 0x556f35a29050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.914+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a29590 con 0x556f35a25ce0 2024-04-25T11:41:24.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.915+0000 7f3095800640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x556f35a25ce0 0x556f35a29050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.916+0000 7f3095800640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x556f35a25ce0 0x556f35a29050 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.921+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:24.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.921+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a29590 con 0x556f35a25ce0 2024-04-25T11:41:24.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.926+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v128'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:24.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.926+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a29590 con 0x556f35a25ce0 2024-04-25T11:41:24.917 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:41:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.930+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v128'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:24.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.931+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2a9c0 con 0x556f35a25ce0 2024-04-25T11:41:24.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.935+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:24.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.935+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2a9c0 con 0x556f35a25ce0 2024-04-25T11:41:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.940+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:24.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.940+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a299b0 con 0x556f35a25ce0 2024-04-25T11:41:24.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.945+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.945+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x556f35a29ec0 0x556f35a2c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.945+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2c950 con 0x556f35a29ec0 2024-04-25T11:41:24.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.946+0000 7f3094e00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x556f35a29ec0 0x556f35a2c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.947+0000 7f3094e00640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x556f35a29ec0 0x556f35a2c2e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.952+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v128'393 uv387 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.952+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a2c9d0 con 0x556f35a29ec0 2024-04-25T11:41:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.956+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v128'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.957+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2cdf0 con 0x556f35a29ec0 2024-04-25T11:41:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.961+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v128'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.947 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:41:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.961+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2de00 con 0x556f35a29ec0 2024-04-25T11:41:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.965+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'396 uv396 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.965+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2de00 con 0x556f35a29ec0 2024-04-25T11:41:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.970+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'397 uv397 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.970+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2cdf0 con 0x556f35a29ec0 2024-04-25T11:41:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.974+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'398 uv398 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.974+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2cdf0 con 0x556f35a29ec0 2024-04-25T11:41:24.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.979+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v128'399 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.979+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a2cdf0 con 0x556f35a29ec0 2024-04-25T11:41:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.984+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v128'400 uv400 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.984+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2cdf0 con 0x556f35a29ec0 2024-04-25T11:41:24.974 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:41:24.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.988+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v128'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.988+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2cdf0 con 0x556f35a29ec0 2024-04-25T11:41:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.993+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.993+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2cdf0 con 0x556f35a29ec0 2024-04-25T11:41:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.997+0000 7f3094e00640 1 -- 172.21.15.19:0/3237914624 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3084021070 con 0x556f35a29ec0 2024-04-25T11:41:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.998+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556f35a2d470 0x556f35a2f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.998+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a2ff40 con 0x556f35a2d470 2024-04-25T11:41:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:24.998+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556f35a2d470 0x556f35a2f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.000+0000 7f3094400640 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556f35a2d470 0x556f35a2f890 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.004+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v128'178 uv177 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:24.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.004+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a30310 con 0x556f35a2d470 2024-04-25T11:41:24.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.008+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v128'179 uv179 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:24.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.008+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a2d470 2024-04-25T11:41:24.999 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:41:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.012+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v128'180 uv180 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:24.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.012+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a2d470 2024-04-25T11:41:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.017+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'181 uv181 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.017+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a2d470 2024-04-25T11:41:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.021+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'182 uv182 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.021+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f358a0c90 2024-04-25T11:41:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.026+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.026+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f358a0c90 2024-04-25T11:41:25.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.031+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v128'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:25.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.031+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f358a0c90 2024-04-25T11:41:25.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.036+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v128'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:25.023 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:41:25.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.036+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a31740 con 0x556f358a0c90 2024-04-25T11:41:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.040+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:25.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.040+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a31740 con 0x556f358a0c90 2024-04-25T11:41:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.045+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:25.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.045+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f358a0c90 2024-04-25T11:41:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.050+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f358a0c90 2024-04-25T11:41:25.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.050+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.055+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v128'355 uv349 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.055+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.060+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v128'356 uv356 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.060+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.050 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:41:25.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.064+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v128'357 uv357 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.064+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.068+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'358 uv358 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.068+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.072+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'359 uv359 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.072+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.077+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v128'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.077+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a30af0 con 0x556f35a21f30 2024-04-25T11:41:25.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.081+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v128'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.081+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.071 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:41:25.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.085+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v128'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.085+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a30730 con 0x556f35a21f30 2024-04-25T11:41:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.089+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.090+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a21f30 2024-04-25T11:41:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.094+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a21f30 2024-04-25T11:41:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.094+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.098+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v128'157 uv155 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.098+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.103+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'158 uv158 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.103+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.094 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:41:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.107+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v128'159 uv159 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.107+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.111+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'160 uv160 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.111+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.115+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v128'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.116+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.120+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v128'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.120+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.111 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:41:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.124+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v128'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.124+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.128+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.128+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.133+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.133+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a25ce0 2024-04-25T11:41:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.138+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308c050050 con 0x556f35a25ce0 2024-04-25T11:41:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.138+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.142+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v128'181 uv180 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.142+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.147+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v128'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.147+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.137 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:41:25.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.151+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v128'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.151+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.155+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.156+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.160+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.160+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.165+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v128'442 uv433 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.165+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.169+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v128'443 uv443 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.169+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.173+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v128'444 uv444 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.160 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:41:25.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.174+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.178+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v128'445 uv445 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.178+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.183+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'446 uv446 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.183+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.187+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v128'447 uv440 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.187+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.192+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v128'448 uv448 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.192+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.182 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:41:25.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.196+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v128'449 uv449 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.196+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.201+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'450 uv450 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.201+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.205+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'451 uv451 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.205+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a254f0 con 0x556f35a2d470 2024-04-25T11:41:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.209+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'452 uv452 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.209+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x556f35a31320 con 0x556f35a2d470 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f3094400640 1 -- 172.21.15.19:0/3237914624 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'453 uv453 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f308000dbb0 con 0x556f35a2d470 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556f35a1db00 msgr2=0x556f35a1ff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x556f35a1db00 0x556f35a1ff20 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x556f35a29ec0 msgr2=0x556f35a2c2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x556f35a29ec0 0x556f35a2c2e0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556f35a2d470 msgr2=0x556f35a2f890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x556f35a2d470 0x556f35a2f890 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x556f35a21f30 msgr2=0x556f35a24350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x556f35a21f30 0x556f35a24350 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 reap_dead start 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x556f358a0c90 msgr2=0x556f3596fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x556f358a0c90 0x556f3596fdc0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x556f35a25ce0 msgr2=0x556f35a29050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f3095800640 1 -- 172.21.15.19:0/3237914624 reap_dead start 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.214+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x556f35a25ce0 0x556f35a29050 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3074072ec0 msgr2=0x7f30740753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3074072ec0 0x7f30740753d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f308000b270 tx=0x7f3080008000 comp rx=0 tx=0).stop 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a21e0 msgr2=0x556f35973280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a21e0 0x556f35973280 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f308400a4a0 tx=0x7f308400a960 comp rx=0 tx=0).stop 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 shutdown_connections 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x556f35a25ce0 0x556f35a29050 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3074072ec0 0x7f30740753d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 --2- 172.21.15.19:0/3237914624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556f358a21e0 0x556f35973280 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.215+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 >> 172.21.15.19:0/3237914624 conn(0x556f357c4ac0 msgr2=0x556f358d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.216+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 shutdown_connections 2024-04-25T11:41:25.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.216+0000 7f309bd0df40 1 -- 172.21.15.19:0/3237914624 wait complete. 2024-04-25T11:41:25.237 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:41:25.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.481+0000 7f2a45a5ef40 1 Processor -- start 2024-04-25T11:41:25.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.481+0000 7f2a45a5ef40 1 -- start start 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a45a5ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984f07a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a45a5ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efe20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a45a5ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 0x55984efe1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a45a5ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55984ef57d20 con 0x55984f080520 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a45a5ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55984ef5bd60 con 0x55984efe1000 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a45a5ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55984effa3f0 con 0x7f2a240030f0 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efe20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efe20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57680/0 (socket says 172.21.15.19:57680) 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 -- 172.21.15.19:0/1485109400 learned_addr learned my addr 172.21.15.19:0/1485109400 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3ec00640 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984f07a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 -- 172.21.15.19:0/1485109400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 msgr2=0x55984f07a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:25.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984f07a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 -- 172.21.15.19:0/1485109400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 msgr2=0x55984efe1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 0x55984efe1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.483+0000 7f2a3e200640 1 -- 172.21.15.19:0/1485109400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984efdf930 con 0x55984f080520 2024-04-25T11:41:25.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.484+0000 7f2a3e200640 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efe20f0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0469a0 tx=0x7f2a2c04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6880662fe99ddc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:25.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.484+0000 7f2a3ce00640 1 -- 172.21.15.19:0/1485109400 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c04db00 con 0x55984f080520 2024-04-25T11:41:25.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.484+0000 7f2a3ce00640 1 -- 172.21.15.19:0/1485109400 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2c066070 con 0x55984f080520 2024-04-25T11:41:25.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.484+0000 7f2a3ce00640 1 -- 172.21.15.19:0/1485109400 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c061450 con 0x55984f080520 2024-04-25T11:41:25.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/1485109400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 msgr2=0x55984efe20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efe20f0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0469a0 tx=0x7f2a2c04f000 comp rx=0 tx=0).stop 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/1485109400 shutdown_connections 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 0x55984efe1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efe20f0 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/1485109400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984f07a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/1485109400 >> 172.21.15.19:0/1485109400 conn(0x55984eed3ac0 msgr2=0x55984efe32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/1485109400 shutdown_connections 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.485+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/1485109400 wait complete. 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.486+0000 7f2a45a5ef40 1 Processor -- start 2024-04-25T11:41:25.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.486+0000 7f2a45a5ef40 1 -- start start 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a45a5ef40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 0x55984f076b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a45a5ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efb0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a45a5ef40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984efb09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a45a5ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55984f087fd0 con 0x55984f080520 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a45a5ef40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55984effd610 con 0x55984efe1000 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a45a5ef40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55984effd8f0 con 0x7f2a240030f0 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a3e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efb0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a3e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efb0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57686/0 (socket says 172.21.15.19:57686) 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a3e200640 1 -- 172.21.15.19:0/366493513 learned_addr learned my addr 172.21.15.19:0/366493513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a3f600640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984efb09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.488+0000 7f2a3ec00640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 0x55984f076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3e200640 1 -- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 msgr2=0x55984efb09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3e200640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984efb09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3e200640 1 -- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 msgr2=0x55984f076b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3e200640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 0x55984f076b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3e200640 1 -- 172.21.15.19:0/366493513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984f07ada0 con 0x55984f080520 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3f600640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2a240030f0 0x55984efb09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3ec00640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55984efe1000 0x55984f076b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a3e200640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efb0470 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c063040 tx=0x7f2a2c0018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a37400640 1 -- 172.21.15.19:0/366493513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c061650 con 0x55984f080520 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a37400640 1 -- 172.21.15.19:0/366493513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2c0617f0 con 0x55984f080520 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55984f077080 con 0x55984f080520 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a37400640 1 -- 172.21.15.19:0/366493513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c061be0 con 0x55984f080520 2024-04-25T11:41:25.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.489+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984f0817d0 con 0x55984f080520 2024-04-25T11:41:25.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.492+0000 7f2a37400640 1 -- 172.21.15.19:0/366493513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a2c05e050 con 0x55984f080520 2024-04-25T11:41:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.493+0000 7f2a37400640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a20072e90 0x7f2a200753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.493+0000 7f2a37400640 1 -- 172.21.15.19:0/366493513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f2a2c0db3a0 con 0x55984f080520 2024-04-25T11:41:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.493+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55984f11f4c0 0x55984f11f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.493+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x55984eff7f70 con 0x55984f11f4c0 2024-04-25T11:41:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.494+0000 7f2a3ec00640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a20072e90 0x7f2a200753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.494+0000 7f2a3f600640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55984f11f4c0 0x55984f11f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.495+0000 7f2a3ec00640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a20072e90 0x7f2a200753a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x55984efe0ef0 tx=0x7f2a30007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:25.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.495+0000 7f2a3f600640 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55984f11f4c0 0x55984f11f8d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:25.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a3f600640 1 -- 172.21.15.19:0/366493513 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v128'165 uv165 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2a3800e330 con 0x55984f11f4c0 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55984f11f4c0 msgr2=0x55984f11f8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55984f11f4c0 0x55984f11f8d0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a20072e90 msgr2=0x7f2a200753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2a20072e90 0x7f2a200753a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x55984efe0ef0 tx=0x7f2a30007040 comp rx=0 tx=0).stop 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 msgr2=0x55984efb0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 --2- 172.21.15.19:0/366493513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55984f080520 0x55984efb0470 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c063040 tx=0x7f2a2c0018b0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a3f600640 1 -- 172.21.15.19:0/366493513 reap_dead start 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 shutdown_connections 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 >> 172.21.15.19:0/366493513 conn(0x55984eed3ac0 msgr2=0x55984f0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 shutdown_connections 2024-04-25T11:41:25.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.500+0000 7f2a45a5ef40 1 -- 172.21.15.19:0/366493513 wait complete. 2024-04-25T11:41:25.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:41:25.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.794+0000 7fb46da00640 1 Processor -- start 2024-04-25T11:41:25.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.794+0000 7fb46da00640 1 -- start start 2024-04-25T11:41:25.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.797+0000 7fb46da00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb468144340 0x7fb46806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.797+0000 7fb46da00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb46806df20 con 0x7fb46815ee60 2024-04-25T11:41:25.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.797+0000 7fb46da00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb468070000 con 0x7fb46806db10 2024-04-25T11:41:25.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.797+0000 7fb46da00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb46815f270 con 0x7fb468144340 2024-04-25T11:41:25.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.797+0000 7fb46d000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb46815ee60 0x7fb46815f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36904/0 (socket says 172.21.15.19:36904) 2024-04-25T11:41:25.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.797+0000 7fb46d000640 1 -- 172.21.15.19:0/2044888892 learned_addr learned my addr 172.21.15.19:0/2044888892 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:25.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.798+0000 7fb467e00640 1 --2- 172.21.15.19:0/2044888892 >> v2:172.21.15.169:3300/0 conn(0x7fb468144340 0x7fb46806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.798+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3900624108 0 0) 0x7fb460001490 con 0x7fb46815ee60 2024-04-25T11:41:25.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.798+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb44c003520 con 0x7fb46815ee60 2024-04-25T11:41:25.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.798+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2201563126 0 0) 0x7fb44c003520 con 0x7fb46815ee60 2024-04-25T11:41:25.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.798+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 >> v2:172.21.15.169:3300/0 conn(0x7fb468144340 msgr2=0x7fb46806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.798+0000 7fb466a00640 1 --2- 172.21.15.19:0/2044888892 >> v2:172.21.15.169:3300/0 conn(0x7fb468144340 0x7fb46806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.798+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 >> v1:172.21.15.169:6789/0 conn(0x7fb46806db10 legacy=0x7fb46815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46806f240 con 0x7fb46815ee60 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb460001bc0 con 0x7fb46815ee60 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 >> v1:172.21.15.19:6789/0 conn(0x7fb46815ee60 legacy=0x7fb46815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb44c0039b0 0x7fb44c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb44c004800 0x7fb44c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb44c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c00c0c0 con 0x7fb44c0051a0 2024-04-25T11:41:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c00c270 con 0x7fb44c004800 2024-04-25T11:41:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c010880 con 0x7fb44c0039b0 2024-04-25T11:41:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb46d000640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb44c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.799+0000 7fb467e00640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb44c0039b0 0x7fb44c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.800+0000 7fb467400640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb44c004800 0x7fb44c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.800+0000 7fb46d000640 1 -- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb44c0039b0 msgr2=0x7fb44c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.800+0000 7fb46d000640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb44c0039b0 0x7fb44c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.800+0000 7fb46d000640 1 -- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb44c004800 msgr2=0x7fb44c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.800+0000 7fb46d000640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb44c004800 0x7fb44c004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.800+0000 7fb46d000640 1 -- 172.21.15.19:0/2044888892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46806f240 con 0x7fb44c0051a0 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.801+0000 7fb46d000640 1 -- 172.21.15.19:0/2044888892 reap_dead start 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.801+0000 7fb46d000640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb44c00b9c0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fb46000ab30 tx=0x7fb44c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb39caaf31822c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.801+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000a730 con 0x7fb44c0051a0 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.801+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb46002f070 con 0x7fb44c0051a0 2024-04-25T11:41:25.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.801+0000 7fb466a00640 1 -- 172.21.15.19:0/2044888892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460034470 con 0x7fb44c0051a0 2024-04-25T11:41:25.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.802+0000 7fb46da00640 1 -- 172.21.15.19:0/2044888892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 msgr2=0x7fb44c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.802+0000 7fb46da00640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb44c00b9c0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7fb46000ab30 tx=0x7fb44c005830 comp rx=0 tx=0).stop 2024-04-25T11:41:25.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.802+0000 7fb46da00640 1 -- 172.21.15.19:0/2044888892 shutdown_connections 2024-04-25T11:41:25.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.802+0000 7fb46da00640 1 --2- 172.21.15.19:0/2044888892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb44c00b9c0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.802+0000 7fb46da00640 1 -- 172.21.15.19:0/2044888892 >> 172.21.15.19:0/2044888892 conn(0x7fb468072f40 msgr2=0x7fb468073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:25.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.802+0000 7fb46da00640 1 -- 172.21.15.19:0/2044888892 shutdown_connections 2024-04-25T11:41:25.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.802+0000 7fb46da00640 1 -- 172.21.15.19:0/2044888892 wait complete. 2024-04-25T11:41:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.804+0000 7fb46da00640 1 Processor -- start 2024-04-25T11:41:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.805+0000 7fb46da00640 1 -- start start 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb46da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb4681460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb46da00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4681605a0 0x7fb4680b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb46da00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4680b0b80 0x7fb4681448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb46da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb46806f800 con 0x7fb44c0051a0 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb46da00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb468144750 con 0x7fb4681605a0 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb46da00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb46806fae0 con 0x7fb4680b0b80 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb467e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb4681460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb467e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb4681460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57706/0 (socket says 172.21.15.19:57706) 2024-04-25T11:41:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.806+0000 7fb467e00640 1 -- 172.21.15.19:0/3299933510 learned_addr learned my addr 172.21.15.19:0/3299933510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb46d000640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4680b0b80 0x7fb4681448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb467400640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4681605a0 0x7fb4680b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb467e00640 1 -- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4680b0b80 msgr2=0x7fb4681448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb467e00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4680b0b80 0x7fb4681448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb467e00640 1 -- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4681605a0 msgr2=0x7fb4680b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb467e00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4681605a0 0x7fb4680b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb467e00640 1 -- 172.21.15.19:0/3299933510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46815f540 con 0x7fb44c0051a0 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb467e00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb4681460e0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fb458011570 tx=0x7fb458011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.807+0000 7fb45fe00640 1 -- 172.21.15.19:0/3299933510 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458015930 con 0x7fb44c0051a0 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.808+0000 7fb45fe00640 1 -- 172.21.15.19:0/3299933510 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb458015ad0 con 0x7fb44c0051a0 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.808+0000 7fb45fe00640 1 -- 172.21.15.19:0/3299933510 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45801d400 con 0x7fb44c0051a0 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.808+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681609b0 con 0x7fb44c0051a0 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.808+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46815fac0 con 0x7fb44c0051a0 2024-04-25T11:41:25.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.809+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb468076c60 con 0x7fb44c0051a0 2024-04-25T11:41:25.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.810+0000 7fb45fe00640 1 -- 172.21.15.19:0/3299933510 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb45801d5a0 con 0x7fb44c0051a0 2024-04-25T11:41:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.811+0000 7fb45fe00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4480744e0 0x7fb4480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.811+0000 7fb45fe00640 1 -- 172.21.15.19:0/3299933510 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7fb458096760 con 0x7fb44c0051a0 2024-04-25T11:41:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.812+0000 7fb467400640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4480744e0 0x7fb4480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:25.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.814+0000 7fb467400640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4480744e0 0x7fb4480769f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb4680b5160 tx=0x7fb454009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:25.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:25.817+0000 7fb45fe00640 1 -- 172.21.15.19:0/3299933510 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb458019030 con 0x7fb44c0051a0 2024-04-25T11:41:26.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.061+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb4680b7110 con 0x7fb44c0051a0 2024-04-25T11:41:26.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.436+0000 7fb45fe00640 1 -- 172.21.15.19:0/3299933510 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v328) ==== 162+0+0 (secure 0 0 0) 0x7fb458060220 con 0x7fb44c0051a0 2024-04-25T11:41:26.423 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:41:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.439+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4480744e0 msgr2=0x7fb4480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:26.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.439+0000 7fb46da00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4480744e0 0x7fb4480769f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb4680b5160 tx=0x7fb454009040 comp rx=0 tx=0).stop 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.439+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 msgr2=0x7fb4681460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.439+0000 7fb46da00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb4681460e0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fb458011570 tx=0x7fb458011a30 comp rx=0 tx=0).stop 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 shutdown_connections 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4480744e0 0x7fb4480769f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4681605a0 0x7fb4680b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4680b0b80 0x7fb4681448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 --2- 172.21.15.19:0/3299933510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb44c0051a0 0x7fb4681460e0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 >> 172.21.15.19:0/3299933510 conn(0x7fb468072f40 msgr2=0x7fb46815c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 shutdown_connections 2024-04-25T11:41:26.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:26.440+0000 7fb46da00640 1 -- 172.21.15.19:0/3299933510 wait complete. 2024-04-25T11:41:26.441 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:26.454+0000 7f5993200640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-04-25T11:41:26.860 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.167+0000 7f12c3e00640 1 Processor -- start 2024-04-25T11:41:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.168+0000 7f12c3e00640 1 -- start start 2024-04-25T11:41:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.170+0000 7f12c3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12c40b1c70 0x7f12c4167a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.170+0000 7f12c3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c40d1d30 con 0x7f12c406db00 2024-04-25T11:41:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.170+0000 7f12c3e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c40760d0 con 0x7f12c40b4690 2024-04-25T11:41:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.170+0000 7f12c3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f12c40ce760 con 0x7f12c40b1c70 2024-04-25T11:41:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.171+0000 7f12c3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12c40b1c70 0x7f12c4167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.171+0000 7f12c3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12c40b1c70 0x7f12c4167a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57708/0 (socket says 172.21.15.19:57708) 2024-04-25T11:41:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.171+0000 7f12c3400640 1 -- 172.21.15.19:0/1741640244 learned_addr learned my addr 172.21.15.19:0/1741640244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.171+0000 7f12c2a00640 1 --1- 172.21.15.19:0/1741640244 >> v1:172.21.15.19:6789/0 conn(0x7f12c406db00 0x7f12c40b0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36908/0 (socket says 172.21.15.19:36908) 2024-04-25T11:41:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.171+0000 7f12c3400640 1 -- 172.21.15.19:0/1741640244 >> v1:172.21.15.169:6789/0 conn(0x7f12c40b4690 legacy=0x7f12c40b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.171+0000 7f12c3400640 1 -- 172.21.15.19:0/1741640244 >> v1:172.21.15.19:6789/0 conn(0x7f12c406db00 legacy=0x7f12c40b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.171+0000 7f12c3400640 1 -- 172.21.15.19:0/1741640244 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c40763b0 con 0x7f12c40b1c70 2024-04-25T11:41:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.173+0000 7f12c3400640 1 --2- 172.21.15.19:0/1741640244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c40b1c70 0x7f12c4167a30 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00d150 tx=0x7f12bc02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18caf15b842bf208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.173+0000 7f12c2000640 1 -- 172.21.15.19:0/1741640244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00b2e0 con 0x7f12c40b1c70 2024-04-25T11:41:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.173+0000 7f12c2000640 1 -- 172.21.15.19:0/1741640244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12bc009600 con 0x7f12c40b1c70 2024-04-25T11:41:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.173+0000 7f12c2000640 1 -- 172.21.15.19:0/1741640244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00b480 con 0x7f12c40b1c70 2024-04-25T11:41:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.174+0000 7f12c3e00640 1 -- 172.21.15.19:0/1741640244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c40b1c70 msgr2=0x7f12c4167a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.174+0000 7f12c3e00640 1 --2- 172.21.15.19:0/1741640244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c40b1c70 0x7f12c4167a30 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00d150 tx=0x7f12bc02c8a0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.174+0000 7f12c3e00640 1 -- 172.21.15.19:0/1741640244 shutdown_connections 2024-04-25T11:41:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.174+0000 7f12c3e00640 1 --2- 172.21.15.19:0/1741640244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c40b1c70 0x7f12c4167a30 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.174+0000 7f12c3e00640 1 -- 172.21.15.19:0/1741640244 >> 172.21.15.19:0/1741640244 conn(0x7f12c4073920 msgr2=0x7f12c40b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.174+0000 7f12c3e00640 1 -- 172.21.15.19:0/1741640244 shutdown_connections 2024-04-25T11:41:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.174+0000 7f12c3e00640 1 -- 172.21.15.19:0/1741640244 wait complete. 2024-04-25T11:41:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.176+0000 7f12c3e00640 1 Processor -- start 2024-04-25T11:41:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.176+0000 7f12c3e00640 1 -- start start 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c406db00 0x7f12c41e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12c40b1c70 0x7f12c4153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c40b4690 0x7f12c41e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12c40ce2d0 con 0x7f12c406db00 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c3e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f12c4075e80 con 0x7f12c40b4690 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c3e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f12c406c5c0 con 0x7f12c40b1c70 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c406db00 0x7f12c41e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c406db00 0x7f12c41e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57720/0 (socket says 172.21.15.19:57720) 2024-04-25T11:41:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.178+0000 7f12c2a00640 1 -- 172.21.15.19:0/3062782060 learned_addr learned my addr 172.21.15.19:0/3062782060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12ba000640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12c40b1c70 0x7f12c4153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12c2a00640 1 -- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12c40b1c70 msgr2=0x7f12c4153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12c2a00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12c40b1c70 0x7f12c4153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12c2a00640 1 -- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c40b4690 msgr2=0x7f12c41e7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12c2a00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c40b4690 0x7f12c41e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12c2a00640 1 -- 172.21.15.19:0/3062782060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c40cb380 con 0x7f12c406db00 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12c2a00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c406db00 0x7f12c41e6590 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f12b400c3c0 tx=0x7f12b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12bb400640 1 -- 172.21.15.19:0/3062782060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b40074e0 con 0x7f12c406db00 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12bb400640 1 -- 172.21.15.19:0/3062782060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b4007680 con 0x7f12c406db00 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12bb400640 1 -- 172.21.15.19:0/3062782060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4004a20 con 0x7f12c406db00 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.179+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c40cb660 con 0x7f12c406db00 2024-04-25T11:41:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.180+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c40cbc20 con 0x7f12c406db00 2024-04-25T11:41:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.181+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c4068cd0 con 0x7f12c406db00 2024-04-25T11:41:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.182+0000 7f12bb400640 1 -- 172.21.15.19:0/3062782060 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f12b401a020 con 0x7f12c406db00 2024-04-25T11:41:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.183+0000 7f12bb400640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a00744e0 0x7f12a00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.183+0000 7f12bb400640 1 -- 172.21.15.19:0/3062782060 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f12b4095680 con 0x7f12c406db00 2024-04-25T11:41:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.184+0000 7f12ba000640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a00744e0 0x7f12a00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.185+0000 7f12ba000640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a00744e0 0x7f12a00769f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f12c41f9d90 tx=0x7f12ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:27.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.188+0000 7f12bb400640 1 -- 172.21.15.19:0/3062782060 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f12c4068cd0 con 0x7f12c406db00 2024-04-25T11:41:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.432+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12c4050740 con 0x7f12c406db00 2024-04-25T11:41:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.433+0000 7f12bb400640 1 -- 172.21.15.19:0/3062782060 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 329 v329) ==== 95+0+5190 (secure 0 0 0) 0x7f12b405f1a0 con 0x7f12c406db00 2024-04-25T11:41:27.420 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:27.420 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":329,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":329,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:26.441958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10863":{"gid":10863,"name":"a","rank":0,"incarnation":329,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/3563649842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3563649842},{"type":"v1","addr":"172.21.15.19:6835","nonce":3563649842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 329 2024-04-25T11:41:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.436+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a00744e0 msgr2=0x7f12a00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.436+0000 7f12c3e00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a00744e0 0x7f12a00769f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f12c41f9d90 tx=0x7f12ac009040 comp rx=0 tx=0).stop 2024-04-25T11:41:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.436+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c406db00 msgr2=0x7f12c41e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.436+0000 7f12c3e00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c406db00 0x7f12c41e6590 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f12b400c3c0 tx=0x7f12b400c880 comp rx=0 tx=0).stop 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 shutdown_connections 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12a00744e0 0x7f12a00769f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12c40b4690 0x7f12c41e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12c40b1c70 0x7f12c4153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 --2- 172.21.15.19:0/3062782060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12c406db00 0x7f12c41e6590 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 >> 172.21.15.19:0/3062782060 conn(0x7f12c4073920 msgr2=0x7f12c40b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 shutdown_connections 2024-04-25T11:41:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:27.437+0000 7f12c3e00640 1 -- 172.21.15.19:0/3062782060 wait complete. 2024-04-25T11:41:27.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 329, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:41:26.441958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10863': {'gid': 10863, 'name': 'a', 'rank': 0, 'incarnation': 329, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/3563649842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 3563649842}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3563649842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:41:27.450 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10863:up:replay 2024-04-25T11:41:27.450 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:41:32.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.164+0000 7f0570e00640 1 Processor -- start 2024-04-25T11:41:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.164+0000 7f0570e00640 1 -- start start 2024-04-25T11:41:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.167+0000 7f0570e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 0x7f056c16b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.167+0000 7f0570e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056c06ddc0 con 0x7f056c0b4ff0 2024-04-25T11:41:33.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.167+0000 7f0570e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056c06e0a0 con 0x7f056c08cd60 2024-04-25T11:41:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.167+0000 7f0570e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f056c06e9d0 con 0x7f056c0d2f80 2024-04-25T11:41:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.167+0000 7f056b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f056c0b4ff0 0x7f056c1611e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36912/0 (socket says 172.21.15.19:36912) 2024-04-25T11:41:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.167+0000 7f056b400640 1 -- 172.21.15.19:0/1402357038 learned_addr learned my addr 172.21.15.19:0/1402357038 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.167+0000 7f056be00640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 0x7f056c16b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832244776 0 0) 0x7f056c06ddc0 con 0x7f056c0b4ff0 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0550003520 con 0x7f056c0b4ff0 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 561390373 0 0) 0x7f0550003520 con 0x7f056c0b4ff0 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 msgr2=0x7f056c16b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 0x7f056c16b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 >> v1:172.21.15.169:6789/0 conn(0x7f056c08cd60 legacy=0x7f056c08d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c06e380 con 0x7f056c0b4ff0 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f055c0017f0 con 0x7f056c0b4ff0 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 >> v1:172.21.15.19:6789/0 conn(0x7f056c0b4ff0 legacy=0x7f056c1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05500039e0 0x7f0550003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0550004800 0x7f0550004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 0x7f05500052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f055000bb50 con 0x7f0550004800 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f055000bd00 con 0x7f05500039e0 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.168+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f055000bf00 con 0x7f056c0d2f80 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056aa00640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0550004800 0x7f0550004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056b400640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05500039e0 0x7f0550003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056be00640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 0x7f05500052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056aa00640 1 -- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 msgr2=0x7f05500052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056aa00640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0d2f80 0x7f05500052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056aa00640 1 -- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05500039e0 msgr2=0x7f0550003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056aa00640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05500039e0 0x7f0550003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.169+0000 7f056aa00640 1 -- 172.21.15.19:0/1402357038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0550003520 con 0x7f0550004800 2024-04-25T11:41:33.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.170+0000 7f056aa00640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0550004800 0x7f0550004c10 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f0554001750 tx=0x7f0550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da97fb2c7579d113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.170+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554001370 con 0x7f0550004800 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05540052d0 con 0x7f0550004800 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f056a000640 1 -- 172.21.15.19:0/1402357038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055402abe0 con 0x7f0550004800 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f0570e00640 1 -- 172.21.15.19:0/1402357038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0550004800 msgr2=0x7f0550004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f0570e00640 1 --2- 172.21.15.19:0/1402357038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0550004800 0x7f0550004c10 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f0554001750 tx=0x7f0550005830 comp rx=0 tx=0).stop 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f056be00640 1 -- 172.21.15.19:0/1402357038 reap_dead start 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f0570e00640 1 -- 172.21.15.19:0/1402357038 shutdown_connections 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f0570e00640 1 -- 172.21.15.19:0/1402357038 >> 172.21.15.19:0/1402357038 conn(0x7f056c0b5460 msgr2=0x7f056c0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f0570e00640 1 -- 172.21.15.19:0/1402357038 shutdown_connections 2024-04-25T11:41:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.171+0000 7f0570e00640 1 -- 172.21.15.19:0/1402357038 wait complete. 2024-04-25T11:41:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.174+0000 7f0570e00640 1 Processor -- start 2024-04-25T11:41:33.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.174+0000 7f0570e00640 1 -- start start 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.175+0000 7f0570e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0550004800 0x7f056c14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.175+0000 7f0570e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f056c15c6b0 0x7f056c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f0570e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c157e50 0x7f056c1fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f0570e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f056c06da20 con 0x7f056c15c6b0 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f0570e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f056c0b9180 con 0x7f056c157e50 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f0570e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f056c06dba0 con 0x7f0550004800 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f056c15c6b0 0x7f056c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f056c15c6b0 0x7f056c157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57732/0 (socket says 172.21.15.19:57732) 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 -- 172.21.15.19:0/4288160899 learned_addr learned my addr 172.21.15.19:0/4288160899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056b400640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0550004800 0x7f056c14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056be00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c157e50 0x7f056c1fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 -- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0550004800 msgr2=0x7f056c14f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0550004800 0x7f056c14f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 -- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c157e50 msgr2=0x7f056c1fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c157e50 0x7f056c1fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.176+0000 7f056aa00640 1 -- 172.21.15.19:0/4288160899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c158260 con 0x7f056c15c6b0 2024-04-25T11:41:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.177+0000 7f056be00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c157e50 0x7f056c1fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.177+0000 7f056aa00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f056c15c6b0 0x7f056c157910 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f0554001870 tx=0x7f0554001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.177+0000 7f0563400640 1 -- 172.21.15.19:0/4288160899 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554037040 con 0x7f056c15c6b0 2024-04-25T11:41:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.177+0000 7f0563400640 1 -- 172.21.15.19:0/4288160899 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554000ed0 con 0x7f056c15c6b0 2024-04-25T11:41:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.177+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c15d090 con 0x7f056c15c6b0 2024-04-25T11:41:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.177+0000 7f0563400640 1 -- 172.21.15.19:0/4288160899 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554007a30 con 0x7f056c15c6b0 2024-04-25T11:41:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.177+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c15d650 con 0x7f056c15c6b0 2024-04-25T11:41:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.178+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056c0d1290 con 0x7f056c15c6b0 2024-04-25T11:41:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.180+0000 7f0563400640 1 -- 172.21.15.19:0/4288160899 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0554044020 con 0x7f056c15c6b0 2024-04-25T11:41:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.181+0000 7f0563400640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05480744e0 0x7f05480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.181+0000 7f0563400640 1 -- 172.21.15.19:0/4288160899 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f05540b5980 con 0x7f056c15c6b0 2024-04-25T11:41:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.181+0000 7f056b400640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05480744e0 0x7f05480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.183+0000 7f056b400640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05480744e0 0x7f05480769f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f055c001eb0 tx=0x7f055c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.186+0000 7f0563400640 1 -- 172.21.15.19:0/4288160899 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f056c0d1290 con 0x7f056c15c6b0 2024-04-25T11:41:33.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.430+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f056c15ce20 con 0x7f056c15c6b0 2024-04-25T11:41:33.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.432+0000 7f0563400640 1 -- 172.21.15.19:0/4288160899 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 332 v332) ==== 95+0+5199 (secure 0 0 0) 0x7f055404b020 con 0x7f056c15c6b0 2024-04-25T11:41:33.418 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:33.419 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":332,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":332,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:29.474074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10863":{"gid":10863,"name":"a","rank":0,"incarnation":329,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6835/3563649842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3563649842},{"type":"v1","addr":"172.21.15.19:6835","nonce":3563649842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10863,"qdb_cluster":[10863]},"id":6}]} 2024-04-25T11:41:33.419 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 332 2024-04-25T11:41:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.434+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05480744e0 msgr2=0x7f05480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.434+0000 7f0570e00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05480744e0 0x7f05480769f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f055c001eb0 tx=0x7f055c009040 comp rx=0 tx=0).stop 2024-04-25T11:41:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.434+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f056c15c6b0 msgr2=0x7f056c157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.434+0000 7f0570e00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f056c15c6b0 0x7f056c157910 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f0554001870 tx=0x7f0554001120 comp rx=0 tx=0).stop 2024-04-25T11:41:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 shutdown_connections 2024-04-25T11:41:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f05480744e0 0x7f05480769f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c157e50 0x7f056c1fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f056c15c6b0 0x7f056c157910 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 --2- 172.21.15.19:0/4288160899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0550004800 0x7f056c14f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 >> 172.21.15.19:0/4288160899 conn(0x7f056c0b5460 msgr2=0x7f056c0b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 shutdown_connections 2024-04-25T11:41:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.435+0000 7f0570e00640 1 -- 172.21.15.19:0/4288160899 wait complete. 2024-04-25T11:41:33.452 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 332, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:35:13.963880+0000', 'modified': '2024-04-25T11:41:29.474074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10863': {'gid': 10863, 'name': 'a', 'rank': 0, 'incarnation': 329, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6835/3563649842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 3563649842}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3563649842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10863, 'qdb_cluster': [10863]} 2024-04-25T11:41:33.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:41:33.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:41:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.747+0000 7f6dd6200640 1 Processor -- start 2024-04-25T11:41:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.747+0000 7f6dd6200640 1 -- start start 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dd6200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6dd00afba0 0x7f6dd015c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dd6200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6dd00b3790 0x7f6dd00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dd6200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd00b4520 con 0x7f6dd00b1970 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dd6200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd006cad0 con 0x7f6dd00afba0 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dd6200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd00b4800 con 0x7f6dd00b3790 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dd4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6dd00afba0 0x7f6dd015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dc7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6dd00b1970 0x7f6dd00b1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36914/0 (socket says 172.21.15.19:36914) 2024-04-25T11:41:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.750+0000 7f6dc7e00640 1 -- 172.21.15.19:0/3115320393 learned_addr learned my addr 172.21.15.19:0/3115320393 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.751+0000 7f6dd5800640 1 --2- 172.21.15.19:0/3115320393 >> v2:172.21.15.169:3300/0 conn(0x7f6dd00b3790 0x7f6dd00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.751+0000 7f6dd4e00640 1 -- 172.21.15.19:0/3115320393 >> v2:172.21.15.169:3300/0 conn(0x7f6dd00b3790 msgr2=0x7f6dd00b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.751+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/3115320393 >> v2:172.21.15.169:3300/0 conn(0x7f6dd00b3790 0x7f6dd00b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.751+0000 7f6dd4e00640 1 -- 172.21.15.19:0/3115320393 >> v1:172.21.15.19:6789/0 conn(0x7f6dd00b1970 legacy=0x7f6dd00b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.751+0000 7f6dd4e00640 1 -- 172.21.15.19:0/3115320393 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd006c760 con 0x7f6dd00afba0 2024-04-25T11:41:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.751+0000 7f6dcfe00640 1 -- 172.21.15.19:0/3115320393 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1989220779 0 0) 0x7f6dd00b4520 con 0x7f6dd00b1970 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.752+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/3115320393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd015c630 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b34b0 tx=0x7f6dc002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ac1377f3b66aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.752+0000 7f6dcfe00640 1 -- 172.21.15.19:0/3115320393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0037070 con 0x7f6dd00afba0 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.752+0000 7f6dcfe00640 1 -- 172.21.15.19:0/3115320393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc002add0 con 0x7f6dd00afba0 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.752+0000 7f6dcfe00640 1 -- 172.21.15.19:0/3115320393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0032660 con 0x7f6dd00afba0 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 -- 172.21.15.19:0/3115320393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 msgr2=0x7f6dd015c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 --2- 172.21.15.19:0/3115320393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd015c630 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b34b0 tx=0x7f6dc002a080 comp rx=0 tx=0).stop 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 -- 172.21.15.19:0/3115320393 shutdown_connections 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 --2- 172.21.15.19:0/3115320393 >> v2:172.21.15.169:3300/0 conn(0x7f6dd00b3790 0x7f6dd00b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 --2- 172.21.15.19:0/3115320393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd015c630 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 -- 172.21.15.19:0/3115320393 >> 172.21.15.19:0/3115320393 conn(0x7f6dd0073920 msgr2=0x7f6dd00b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 -- 172.21.15.19:0/3115320393 shutdown_connections 2024-04-25T11:41:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.753+0000 7f6dd6200640 1 -- 172.21.15.19:0/3115320393 wait complete. 2024-04-25T11:41:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.756+0000 7f6dd6200640 1 Processor -- start 2024-04-25T11:41:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.756+0000 7f6dd6200640 1 -- start start 2024-04-25T11:41:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.757+0000 7f6dd6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd014d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.757+0000 7f6dd6200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00b1970 0x7f6dd01f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.757+0000 7f6dd6200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00b3790 0x7f6dd00d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd6200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0075e80 con 0x7f6dd00afba0 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd6200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0076000 con 0x7f6dd00b1970 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd6200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd0076180 con 0x7f6dd00b3790 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd014d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd014d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57752/0 (socket says 172.21.15.19:57752) 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 -- 172.21.15.19:0/386027073 learned_addr learned my addr 172.21.15.19:0/386027073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00b1970 0x7f6dd01f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 -- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00b3790 msgr2=0x7f6dd00d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00b3790 0x7f6dd00d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 -- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00b1970 msgr2=0x7f6dd01f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00b1970 0x7f6dd01f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dd4e00640 1 -- 172.21.15.19:0/386027073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd00d0fc0 con 0x7f6dd00afba0 2024-04-25T11:41:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.758+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00b1970 0x7f6dd01f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.759+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd014d2f0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0000c00 tx=0x7f6dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.759+0000 7f6dcd600640 1 -- 172.21.15.19:0/386027073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0037050 con 0x7f6dd00afba0 2024-04-25T11:41:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.759+0000 7f6dcd600640 1 -- 172.21.15.19:0/386027073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0038040 con 0x7f6dd00afba0 2024-04-25T11:41:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.759+0000 7f6dcd600640 1 -- 172.21.15.19:0/386027073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0004270 con 0x7f6dd00afba0 2024-04-25T11:41:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.759+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd0151980 con 0x7f6dd00afba0 2024-04-25T11:41:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.759+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd0151f40 con 0x7f6dd00afba0 2024-04-25T11:41:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.763+0000 7f6dcd600640 1 -- 172.21.15.19:0/386027073 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6dc0003610 con 0x7f6dd00afba0 2024-04-25T11:41:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.764+0000 7f6dcd600640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6dac0744b0 0x7f6dac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.764+0000 7f6dcd600640 1 -- 172.21.15.19:0/386027073 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f6dc00b65b0 con 0x7f6dd00afba0 2024-04-25T11:41:33.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.764+0000 7f6dde79a740 1 Processor -- start 2024-04-25T11:41:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.764+0000 7f6dde79a740 1 -- start start 2024-04-25T11:41:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.764+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6dac0744b0 0x7f6dac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379bb270 0x560b37d849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37da4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b3799a590 0x560b37da46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dde79a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560b37cc1600 con 0x560b379bb270 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560b37c5e100 con 0x560b379923b0 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560b37cbd5c0 con 0x560b3799a590 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6dac0744b0 0x7f6dac0769c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b2090 tx=0x7f6db8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dd4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379bb270 0x560b37d849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dd4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379bb270 0x560b37d849e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57764/0 (socket says 172.21.15.19:57764) 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.766+0000 7f6dd4e00640 1 -- 172.21.15.19:0/4140102717 learned_addr learned my addr 172.21.15.19:0/4140102717 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37da4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dd4e00640 1 -- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b3799a590 msgr2=0x560b37da46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b3799a590 0x560b37da46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dd4e00640 1 -- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 msgr2=0x560b37da4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37da4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dd4e00640 1 -- 172.21.15.19:0/4140102717 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b379d32d0 con 0x560b379bb270 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37da4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379bb270 0x560b37d849e0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f6dc007bff0 tx=0x7f6dc007f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99798bba8fa53c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dc5600640 1 -- 172.21.15.19:0/4140102717 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc007f510 con 0x560b379bb270 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dc5600640 1 -- 172.21.15.19:0/4140102717 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0080100 con 0x560b379bb270 2024-04-25T11:41:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.767+0000 7f6dc5600640 1 -- 172.21.15.19:0/4140102717 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc007fa00 con 0x560b379bb270 2024-04-25T11:41:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 -- 172.21.15.19:0/4140102717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379bb270 msgr2=0x560b37d849e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379bb270 0x560b37d849e0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f6dc007bff0 tx=0x7f6dc007f900 comp rx=0 tx=0).stop 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 -- 172.21.15.19:0/4140102717 shutdown_connections 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37da4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b3799a590 0x560b37da46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 --2- 172.21.15.19:0/4140102717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379bb270 0x560b37d849e0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 -- 172.21.15.19:0/4140102717 >> 172.21.15.19:0/4140102717 conn(0x560b379af3c0 msgr2=0x560b37d5d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 -- 172.21.15.19:0/4140102717 shutdown_connections 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.768+0000 7f6dde79a740 1 -- 172.21.15.19:0/4140102717 wait complete. 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.769+0000 7f6dde79a740 1 -- start start 2024-04-25T11:41:33.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.770+0000 7f6dde79a740 1 Processor -- start 2024-04-25T11:41:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.771+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379923b0 0x560b37d86310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.771+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b3799a590 0x560b37da8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.771+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37da90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.771+0000 7f6dde79a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560b37cc1600 con 0x560b379923b0 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.771+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560b37c48f70 con 0x560b3799a590 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.771+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560b37da7ae0 con 0x560b379bb270 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.771+0000 7f6dde79a740 10 client.0 _ll_register_callbacks cb 0x560b37cb7480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379923b0 0x560b37d86310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379923b0 0x560b37d86310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57774/0 (socket says 172.21.15.19:57774) 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 -- 172.21.15.19:0/140096767 learned_addr learned my addr 172.21.15.19:0/140096767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b3799a590 0x560b37da8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd5800640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37da90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 -- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 msgr2=0x560b37da90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37da90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 -- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b3799a590 msgr2=0x560b37da8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b3799a590 0x560b37da8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd4e00640 1 -- 172.21.15.19:0/140096767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b37a9d6b0 con 0x560b379923b0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dd5800640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37da90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.772+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b3799a590 0x560b37da8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379923b0 0x560b37d86310 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f6dc007bff0 tx=0x7f6dc0048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9803c7025ff035b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dbe000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dbe000640 1 -- 172.21.15.19:0/140096767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc007f510 con 0x560b379923b0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dbe000640 1 -- 172.21.15.19:0/140096767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f6dc0088340 con 0x560b379923b0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc0080100 con 0x560b379923b0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dbe000640 1 -- 172.21.15.19:0/140096767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=332) ==== 24+0+0 (secure 0 0 0) 0x7f6dc00a1120 con 0x560b379923b0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dde79a740 10 client.10953 fetch_fsmap learned FSMap version 332 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=332,monmap=2+}) -- 0x560b37d87830 con 0x560b379923b0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dbe000640 1 -- 172.21.15.19:0/140096767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0080100 con 0x560b379923b0 2024-04-25T11:41:33.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dbe000640 1 -- 172.21.15.19:0/140096767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 332) ==== 3494+0+0 (secure 0 0 0) 0x7f6dc0086070 con 0x560b379923b0 2024-04-25T11:41:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.773+0000 7f6dde79a740 10 client.10953 fetch_fsmap finished waiting for FSMap version 332 2024-04-25T11:41:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.774+0000 7f6dde79a740 10 client.10953 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:41:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.774+0000 7f6dde79a740 20 client.10953 populate_metadata read hostname 'smithi019' 2024-04-25T11:41:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.774+0000 7f6dde79a740 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37979e40 0x560b37db68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.774+0000 7f6dde79a740 4 client.10953 mds_command: new command op to 10863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:41:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.774+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 --> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b37ccf210 con 0x560b37979e40 2024-04-25T11:41:33.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.774+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37979e40 0x560b37db68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.775+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37979e40 0x560b37db68f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cd2b9e7ea0d3d58 server_cookie=8484b7c63869c4b5 in_seq=0 out_seq=0 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.775+0000 7f6dbe000640 10 client.10953 ms_handle_connect on v2:172.21.15.19:6834/3563649842 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dbe000640 1 -- 172.21.15.19:0/140096767 <== mds.0 v2:172.21.15.19:6834/3563649842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560b37ccf210 con 0x560b37979e40 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dbe000640 10 client.10953 handle_command_reply: tid=0 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 client.10953 shutdown 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379923b0 msgr2=0x560b37d86310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379923b0 0x560b37d86310 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f6dc007bff0 tx=0x7f6dc0048000 comp rx=0 tx=0).stop 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 shutdown_connections 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37979e40 0x560b37db68f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b3799a590 0x560b37da8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37da90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 --2- 172.21.15.19:0/140096767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b379923b0 0x560b37d86310 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 >> 172.21.15.19:0/140096767 conn(0x560b379af3c0 msgr2=0x560b37d84010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dbe000640 0 client.10953 ms_handle_reset on v2:172.21.15.19:6834/3563649842 2024-04-25T11:41:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 shutdown_connections 2024-04-25T11:41:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.776+0000 7f6dde79a740 1 -- 172.21.15.19:0/140096767 wait complete. 2024-04-25T11:41:33.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.777+0000 7f6dde79a740 20 client.10953 trim_cache size 0 max 16384 2024-04-25T11:41:33.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.797+0000 7f6dde79a740 1 Processor -- start 2024-04-25T11:41:33.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.798+0000 7f6dde79a740 1 -- start start 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379923b0 0x560b37d880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379bb270 0x560b37e1b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dde79a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd015c0a0 con 0x560b3799a590 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd015b400 con 0x560b379bb270 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd015ba50 con 0x560b379923b0 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d88ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57416/0 (socket says 172.21.15.19:57416) 2024-04-25T11:41:33.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.799+0000 7f6dc7e00640 1 -- 172.21.15.19:0/3433809751 learned_addr learned my addr 172.21.15.19:0/3433809751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:33.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379923b0 0x560b37d880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dd5800640 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379bb270 0x560b37e1b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dc7e00640 1 -- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379923b0 msgr2=0x560b37d880a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379923b0 0x560b37d880a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dc7e00640 1 -- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379bb270 msgr2=0x560b37e1b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379bb270 0x560b37e1b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dc7e00640 1 -- 172.21.15.19:0/3433809751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b37dcad00 con 0x560b3799a590 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.800+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d88ad0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b4f30 tx=0x7f6db8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c6b81367a699db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.801+0000 7f6dc4c00640 1 -- 172.21.15.19:0/3433809751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8005c80 con 0x560b3799a590 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.801+0000 7f6dc4c00640 1 -- 172.21.15.19:0/3433809751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8028020 con 0x560b3799a590 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.801+0000 7f6dc4c00640 1 -- 172.21.15.19:0/3433809751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800de90 con 0x560b3799a590 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.801+0000 7f6dde79a740 1 -- 172.21.15.19:0/3433809751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 msgr2=0x560b37d88ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.801+0000 7f6dde79a740 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d88ad0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b4f30 tx=0x7f6db8016040 comp rx=0 tx=0).stop 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.802+0000 7f6dde79a740 1 -- 172.21.15.19:0/3433809751 shutdown_connections 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.802+0000 7f6dde79a740 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379bb270 0x560b37e1b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.802+0000 7f6dde79a740 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d88ad0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.802+0000 7f6dde79a740 1 --2- 172.21.15.19:0/3433809751 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379923b0 0x560b37d880a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.802+0000 7f6dde79a740 1 -- 172.21.15.19:0/3433809751 >> 172.21.15.19:0/3433809751 conn(0x560b379af3c0 msgr2=0x560b37da8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.802+0000 7f6dde79a740 1 -- 172.21.15.19:0/3433809751 shutdown_connections 2024-04-25T11:41:33.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.802+0000 7f6dde79a740 1 -- 172.21.15.19:0/3433809751 wait complete. 2024-04-25T11:41:33.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.803+0000 7f6dde79a740 1 -- start start 2024-04-25T11:41:33.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.803+0000 7f6dde79a740 1 Processor -- start 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.804+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37d57e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d81ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dde79a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37dba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dde79a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560b37cc1600 con 0x560b3799a590 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560b37c48f70 con 0x560b379923b0 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dde79a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560b37d868a0 con 0x560b379bb270 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dde79a740 10 client.0 _ll_register_callbacks cb 0x7f6dc0098070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d81ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d81ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57426/0 (socket says 172.21.15.19:57426) 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 -- 172.21.15.19:0/2659401953 learned_addr learned my addr 172.21.15.19:0/2659401953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37d57e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dd5800640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37dba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 -- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 msgr2=0x560b37dba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37dba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 -- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 msgr2=0x560b37d57e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37d57e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dc7e00640 1 -- 172.21.15.19:0/2659401953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b37d880a0 con 0x560b3799a590 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.805+0000 7f6dd5800640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37dba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dc7e00640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d81ca0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009010 tx=0x7f6db8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f67fb2da99de00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dbe000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dbe000640 1 -- 172.21.15.19:0/2659401953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8027070 con 0x560b3799a590 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dbe000640 1 -- 172.21.15.19:0/2659401953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f6db8023420 con 0x560b3799a590 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db8028020 con 0x560b3799a590 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dbe000640 1 -- 172.21.15.19:0/2659401953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=332) ==== 24+0+0 (secure 0 0 0) 0x7f6db800de70 con 0x560b3799a590 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dde79a740 10 client.10959 fetch_fsmap learned FSMap version 332 2024-04-25T11:41:33.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.806+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=332,monmap=2+}) -- 0x560b37d57680 con 0x560b3799a590 2024-04-25T11:41:33.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dbe000640 1 -- 172.21.15.19:0/2659401953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8028020 con 0x560b3799a590 2024-04-25T11:41:33.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dbe000640 1 -- 172.21.15.19:0/2659401953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 332) ==== 3494+0+0 (secure 0 0 0) 0x7f6db8023620 con 0x560b3799a590 2024-04-25T11:41:33.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dde79a740 10 client.10959 fetch_fsmap finished waiting for FSMap version 332 2024-04-25T11:41:33.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dde79a740 10 client.10959 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dde79a740 20 client.10959 populate_metadata read hostname 'smithi019' 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dde79a740 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37dca740 0x560b37dc9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dde79a740 4 client.10959 mds_command: new command op to 10863 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 --> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] -- command(tid 0: {"prefix": "status"}) -- 0x560b37da78f0 con 0x560b37dca740 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.807+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37dca740 0x560b37dc9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dd4e00640 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37dca740 0x560b37dc9350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d12b407f98881c9 server_cookie=95275e5242a7dfe3 in_seq=0 out_seq=0 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dbe000640 10 client.10959 ms_handle_connect on v2:172.21.15.19:6834/3563649842 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dbe000640 1 -- 172.21.15.19:0/2659401953 <== mds.0 v2:172.21.15.19:6834/3563649842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x560b37da78f0 con 0x560b37dca740 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dbe000640 10 client.10959 handle_command_reply: tid=0 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 client.10959 shutdown 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 msgr2=0x560b37d81ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d81ca0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009010 tx=0x7f6db8000ed0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 shutdown_connections 2024-04-25T11:41:33.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:6834/3563649842,v1:172.21.15.19:6835/3563649842] conn(0x560b37dca740 0x560b37dc9350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560b379bb270 0x560b37dba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560b3799a590 0x560b37d81ca0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 --2- 172.21.15.19:0/2659401953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560b379923b0 0x560b37d57e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dbe000640 0 client.10959 ms_handle_reset on v2:172.21.15.19:6834/3563649842 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 >> 172.21.15.19:0/2659401953 conn(0x560b379af3c0 msgr2=0x560b37e1a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.808+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 shutdown_connections 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dde79a740 1 -- 172.21.15.19:0/2659401953 wait complete. 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dde79a740 20 client.10959 trim_cache size 0 max 16384 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:41:33.798 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 10863, 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3620282059999997, 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 332, 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 128, 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 128, 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 15.281146232999999 2024-04-25T11:41:33.799 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:41:33.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6dac0744b0 msgr2=0x7f6dac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6dac0744b0 0x7f6dac0769c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00b2090 tx=0x7f6db8009040 comp rx=0 tx=0).stop 2024-04-25T11:41:33.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 msgr2=0x7f6dd014d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd014d2f0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0000c00 tx=0x7f6dc00046c0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 shutdown_connections 2024-04-25T11:41:33.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6dac0744b0 0x7f6dac0769c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6dd00b1970 0x7f6dd01f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6dd00b3790 0x7f6dd00d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 --2- 172.21.15.19:0/386027073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dd00afba0 0x7f6dd014d2f0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.809+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 >> 172.21.15.19:0/386027073 conn(0x7f6dd0073920 msgr2=0x7f6dd006f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.810+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 shutdown_connections 2024-04-25T11:41:33.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:33.810+0000 7f6dd6200640 1 -- 172.21.15.19:0/386027073 wait complete. 2024-04-25T11:41:33.820 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:41:33.820 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2024-04-25T11:41:33.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:41:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.115+0000 7fec4aa00640 1 Processor -- start 2024-04-25T11:41:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.115+0000 7fec4aa00640 1 -- start start 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec4aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec440d0e00 0x7fec440d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec4aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec4406c5c0 0x7fec4406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec4aa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4406cf10 con 0x7fec440b7810 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec4aa00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fec440d5a60 con 0x7fec440d0e00 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec4aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec440d1850 con 0x7fec4406c5c0 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec48a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fec440b7810 0x7fec440b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49290/0 (socket says 172.21.15.19:49290) 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec48a00640 1 -- 172.21.15.19:0/43242069 learned_addr learned my addr 172.21.15.19:0/43242069 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.118+0000 7fec3b400640 1 --2- 172.21.15.19:0/43242069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec4406c5c0 0x7fec4406c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:34.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec43e00640 1 --2- 172.21.15.19:0/43242069 >> v2:172.21.15.169:3300/0 conn(0x7fec440d0e00 0x7fec440d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:34.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec43400640 1 -- 172.21.15.19:0/43242069 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3611439912 0 0) 0x7fec3c001110 con 0x7fec440b7810 2024-04-25T11:41:34.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec43400640 1 -- 172.21.15.19:0/43242069 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec28003520 con 0x7fec440b7810 2024-04-25T11:41:34.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec3b400640 1 -- 172.21.15.19:0/43242069 >> v2:172.21.15.169:3300/0 conn(0x7fec440d0e00 msgr2=0x7fec440d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:34.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec3b400640 1 --2- 172.21.15.19:0/43242069 >> v2:172.21.15.169:3300/0 conn(0x7fec440d0e00 0x7fec440d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec43400640 1 -- 172.21.15.19:0/43242069 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2882005848 0 0) 0x7fec28003520 con 0x7fec440b7810 2024-04-25T11:41:34.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec3b400640 1 -- 172.21.15.19:0/43242069 >> v1:172.21.15.19:6789/0 conn(0x7fec440b7810 legacy=0x7fec440b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.119+0000 7fec3b400640 1 -- 172.21.15.19:0/43242069 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec44160de0 con 0x7fec4406c5c0 2024-04-25T11:41:34.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.121+0000 7fec3b400640 1 --2- 172.21.15.19:0/43242069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec4406c5c0 0x7fec4406c9d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0097a0 tx=0x7fec2c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e52cdfa437194e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:34.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.121+0000 7fec43400640 1 -- 172.21.15.19:0/43242069 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c037070 con 0x7fec4406c5c0 2024-04-25T11:41:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.121+0000 7fec43400640 1 -- 172.21.15.19:0/43242069 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2c02aba0 con 0x7fec4406c5c0 2024-04-25T11:41:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.121+0000 7fec43400640 1 -- 172.21.15.19:0/43242069 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c001840 con 0x7fec4406c5c0 2024-04-25T11:41:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 -- 172.21.15.19:0/43242069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec4406c5c0 msgr2=0x7fec4406c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 --2- 172.21.15.19:0/43242069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec4406c5c0 0x7fec4406c9d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0097a0 tx=0x7fec2c02a110 comp rx=0 tx=0).stop 2024-04-25T11:41:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 -- 172.21.15.19:0/43242069 shutdown_connections 2024-04-25T11:41:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 --2- 172.21.15.19:0/43242069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec4406c5c0 0x7fec4406c9d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 --2- 172.21.15.19:0/43242069 >> v2:172.21.15.169:3300/0 conn(0x7fec440d0e00 0x7fec440d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 -- 172.21.15.19:0/43242069 >> 172.21.15.19:0/43242069 conn(0x7fec440d5d90 msgr2=0x7fec440d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 -- 172.21.15.19:0/43242069 shutdown_connections 2024-04-25T11:41:34.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.122+0000 7fec4aa00640 1 -- 172.21.15.19:0/43242069 wait complete. 2024-04-25T11:41:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.125+0000 7fec4aa00640 1 Processor -- start 2024-04-25T11:41:34.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.125+0000 7fec4aa00640 1 -- start start 2024-04-25T11:41:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.126+0000 7fec4aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4406c5c0 0x7fec4414f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.126+0000 7fec4aa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec440b7810 0x7fec440ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.126+0000 7fec4aa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec440d0e00 0x7fec440cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.126+0000 7fec4aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec44160ad0 con 0x7fec4406c5c0 2024-04-25T11:41:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.126+0000 7fec4aa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fec441613a0 con 0x7fec440b7810 2024-04-25T11:41:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.126+0000 7fec4aa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fec440b5320 con 0x7fec440d0e00 2024-04-25T11:41:34.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4406c5c0 0x7fec4414f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4406c5c0 0x7fec4414f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57430/0 (socket says 172.21.15.19:57430) 2024-04-25T11:41:34.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 -- 172.21.15.19:0/2030430610 learned_addr learned my addr 172.21.15.19:0/2030430610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec3b400640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec440b7810 0x7fec440ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 -- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec440d0e00 msgr2=0x7fec440cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec440d0e00 0x7fec440cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 -- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec440b7810 msgr2=0x7fec440ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec440b7810 0x7fec440ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec43e00640 1 -- 172.21.15.19:0/2030430610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4415acb0 con 0x7fec4406c5c0 2024-04-25T11:41:34.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.127+0000 7fec3b400640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec440b7810 0x7fec440ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.128+0000 7fec43e00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4406c5c0 0x7fec4414f130 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fec34009d10 tx=0x7fec3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.128+0000 7fec40c00640 1 -- 172.21.15.19:0/2030430610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34018020 con 0x7fec4406c5c0 2024-04-25T11:41:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.128+0000 7fec40c00640 1 -- 172.21.15.19:0/2030430610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34014480 con 0x7fec4406c5c0 2024-04-25T11:41:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.128+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec440cd940 con 0x7fec4406c5c0 2024-04-25T11:41:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.128+0000 7fec40c00640 1 -- 172.21.15.19:0/2030430610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34014830 con 0x7fec4406c5c0 2024-04-25T11:41:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.128+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec440cdf00 con 0x7fec4406c5c0 2024-04-25T11:41:34.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.129+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec44050740 con 0x7fec4406c5c0 2024-04-25T11:41:34.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.130+0000 7fec40c00640 1 -- 172.21.15.19:0/2030430610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec34005d10 con 0x7fec4406c5c0 2024-04-25T11:41:34.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.132+0000 7fec40c00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec200744b0 0x7fec200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:34.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.132+0000 7fec40c00640 1 -- 172.21.15.19:0/2030430610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7fec34096a00 con 0x7fec4406c5c0 2024-04-25T11:41:34.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.133+0000 7fec3b400640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec200744b0 0x7fec200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:34.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.134+0000 7fec3b400640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec200744b0 0x7fec200769c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fec2c034000 tx=0x7fec2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:34.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.137+0000 7fec40c00640 1 -- 172.21.15.19:0/2030430610 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec34022070 con 0x7fec4406c5c0 2024-04-25T11:41:34.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.385+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fec440d4320 con 0x7fec4406c5c0 2024-04-25T11:41:34.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.386+0000 7fec40c00640 1 -- 172.21.15.19:0/2030430610 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v332) ==== 53+0+73 (secure 0 0 0) 0x7fec34010040 con 0x7fec4406c5c0 2024-04-25T11:41:34.373 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec200744b0 msgr2=0x7fec200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec200744b0 0x7fec200769c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fec2c034000 tx=0x7fec2c02e000 comp rx=0 tx=0).stop 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4406c5c0 msgr2=0x7fec4414f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4406c5c0 0x7fec4414f130 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fec34009d10 tx=0x7fec3400c490 comp rx=0 tx=0).stop 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 shutdown_connections 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec200744b0 0x7fec200769c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec440b7810 0x7fec440ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec440d0e00 0x7fec440cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 --2- 172.21.15.19:0/2030430610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4406c5c0 0x7fec4414f130 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 >> 172.21.15.19:0/2030430610 conn(0x7fec440d5d90 msgr2=0x7fec440b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 shutdown_connections 2024-04-25T11:41:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34.389+0000 7fec4aa00640 1 -- 172.21.15.19:0/2030430610 wait complete. 2024-04-25T11:41:34.383 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:34.396+0000 7f5998200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:41:34.401 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:41:34.402 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:41:34.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:34.402 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:41:34.451 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:41:34.451 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:41:34.451 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41779sec preferred_lft 41779sec 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:41:34.452 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:41:34.453 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:41:34.454 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:41:34.454 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:41:34.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:34.455 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:41:34.532 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:41:34.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:34.537 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:41:34.593 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:41:34.594 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:41:34.594 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:41:34.594 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:41:34.594 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:41:34.594 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:41:34.594 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:41:34.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:41:34.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:41:34.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:41:34.594 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:41:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:41:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:41:34.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:34.822 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:41:34.875 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:41:34.876 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:41:34.876 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:41:34.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:34.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:41:34.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:41:34.962 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:35.018 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:41:35.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:41:35.019 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:35.069 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:41:35.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:35.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:41:35.274 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:41:35.274 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:41:35.321 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:41:35.321 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:41:35.321 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:41:35.321 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:41:35.321 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:41:35.323 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:41:35.323 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:41:35.324 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:41:35.324 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:41:35.324 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:41:35.328 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:41:35.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:35.329 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:41:35.407 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:41:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:41:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:41:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:client10968 2024-04-25T11:41:35.533 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:35.533 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:35.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:35.587 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-04-25T11:41:35.594 INFO:teuthology.orchestra.run:waiting for 20 2024-04-25T11:41:35.645 INFO:teuthology.orchestra.run.smithi019.stdout:.: 2024-04-25T11:41:35.645 INFO:teuthology.orchestra.run.smithi019.stdout:subdir 2024-04-25T11:41:35.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:35.645 INFO:teuthology.orchestra.run.smithi019.stdout:./subdir: 2024-04-25T11:41:35.645 INFO:teuthology.orchestra.run.smithi019.stdout:sixmegs 2024-04-25T11:41:35.646 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2024-04-25T11:41:35.646 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2024-04-25T11:41:35.647 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:35.647 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:35.703 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:35.703 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:35.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:41:36.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.057+0000 7fa07a000640 1 Processor -- start 2024-04-25T11:41:36.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.057+0000 7fa07a000640 1 -- start start 2024-04-25T11:41:36.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa07a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0740d50e0 0x7fa0740d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa07a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0740d2ae0 0x7fa074160650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa07a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa074160b90 0x7fa074161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa07a000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa0740b7940 con 0x7fa0740d2ae0 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa07a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa0740d2ef0 con 0x7fa0740d50e0 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa07a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa074161280 con 0x7fa074160b90 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa06be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0740d2ae0 0x7fa074160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa06be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0740d2ae0 0x7fa074160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57440/0 (socket says 172.21.15.19:57440) 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.059+0000 7fa06be00640 1 -- 172.21.15.19:0/213479582 learned_addr learned my addr 172.21.15.19:0/213479582 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa079600640 1 --2- 172.21.15.19:0/213479582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa074160b90 0x7fa074161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa06be00640 1 -- 172.21.15.19:0/213479582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa074160b90 msgr2=0x7fa074161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa078c00640 1 --2- 172.21.15.19:0/213479582 >> v2:172.21.15.169:3300/0 conn(0x7fa0740d50e0 0x7fa0740d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa06be00640 1 --2- 172.21.15.19:0/213479582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa074160b90 0x7fa074161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa06be00640 1 -- 172.21.15.19:0/213479582 >> v2:172.21.15.169:3300/0 conn(0x7fa0740d50e0 msgr2=0x7fa0740d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa06be00640 1 --2- 172.21.15.19:0/213479582 >> v2:172.21.15.169:3300/0 conn(0x7fa0740d50e0 0x7fa0740d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa06be00640 1 -- 172.21.15.19:0/213479582 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740d4600 con 0x7fa0740d2ae0 2024-04-25T11:41:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.060+0000 7fa079600640 1 --2- 172.21.15.19:0/213479582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa074160b90 0x7fa074161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.061+0000 7fa06be00640 1 --2- 172.21.15.19:0/213479582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0740d2ae0 0x7fa074160650 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fa0740d6cc0 tx=0x7fa05c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e9ae08899c58bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.061+0000 7fa073e00640 1 -- 172.21.15.19:0/213479582 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c037070 con 0x7fa0740d2ae0 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.061+0000 7fa073e00640 1 -- 172.21.15.19:0/213479582 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05c032440 con 0x7fa0740d2ae0 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.061+0000 7fa073e00640 1 -- 172.21.15.19:0/213479582 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c032810 con 0x7fa0740d2ae0 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 -- 172.21.15.19:0/213479582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0740d2ae0 msgr2=0x7fa074160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 --2- 172.21.15.19:0/213479582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0740d2ae0 0x7fa074160650 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fa0740d6cc0 tx=0x7fa05c02a1d0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 -- 172.21.15.19:0/213479582 shutdown_connections 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 --2- 172.21.15.19:0/213479582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0740d2ae0 0x7fa074160650 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 --2- 172.21.15.19:0/213479582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa074160b90 0x7fa074161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 --2- 172.21.15.19:0/213479582 >> v2:172.21.15.169:3300/0 conn(0x7fa0740d50e0 0x7fa0740d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 -- 172.21.15.19:0/213479582 >> 172.21.15.19:0/213479582 conn(0x7fa07406c5c0 msgr2=0x7fa07406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 -- 172.21.15.19:0/213479582 shutdown_connections 2024-04-25T11:41:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.062+0000 7fa07a000640 1 -- 172.21.15.19:0/213479582 wait complete. 2024-04-25T11:41:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.065+0000 7fa07a000640 1 Processor -- start 2024-04-25T11:41:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.065+0000 7fa07a000640 1 -- start start 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.066+0000 7fa07a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0740d2ae0 0x7fa0740c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.066+0000 7fa07a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0740d50e0 0x7fa074158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.066+0000 7fa07a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa074160b90 0x7fa074147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.066+0000 7fa07a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa074160fa0 con 0x7fa074160b90 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.066+0000 7fa07a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740d4bc0 con 0x7fa0740d50e0 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.066+0000 7fa07a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa074161560 con 0x7fa0740d2ae0 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa074160b90 0x7fa074147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa074160b90 0x7fa074147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57454/0 (socket says 172.21.15.19:57454) 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 -- 172.21.15.19:0/3925515017 learned_addr learned my addr 172.21.15.19:0/3925515017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:36.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa078c00640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0740d2ae0 0x7fa0740c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa06be00640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0740d50e0 0x7fa074158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 -- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0740d2ae0 msgr2=0x7fa0740c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0740d2ae0 0x7fa0740c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 -- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0740d50e0 msgr2=0x7fa074158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0740d50e0 0x7fa074158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.067+0000 7fa079600640 1 -- 172.21.15.19:0/3925515017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740c8070 con 0x7fa074160b90 2024-04-25T11:41:36.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.068+0000 7fa079600640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa074160b90 0x7fa074147d10 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00c7f0 tx=0x7fa06c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.068+0000 7fa071600640 1 -- 172.21.15.19:0/3925515017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c015020 con 0x7fa074160b90 2024-04-25T11:41:36.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.068+0000 7fa071600640 1 -- 172.21.15.19:0/3925515017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa06c004230 con 0x7fa074160b90 2024-04-25T11:41:36.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.068+0000 7fa071600640 1 -- 172.21.15.19:0/3925515017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06c007620 con 0x7fa074160b90 2024-04-25T11:41:36.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.068+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0741597a0 con 0x7fa074160b90 2024-04-25T11:41:36.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.068+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa074159d60 con 0x7fa074160b90 2024-04-25T11:41:36.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.069+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa074159510 con 0x7fa074160b90 2024-04-25T11:41:36.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.071+0000 7fa071600640 1 -- 172.21.15.19:0/3925515017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa06c018020 con 0x7fa074160b90 2024-04-25T11:41:36.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.072+0000 7fa071600640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa050074500 0x7fa050076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.072+0000 7fa071600640 1 -- 172.21.15.19:0/3925515017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7fa06c096130 con 0x7fa074160b90 2024-04-25T11:41:36.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.072+0000 7fa078c00640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa050074500 0x7fa050076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.074+0000 7fa078c00640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa050074500 0x7fa050076a10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa0640070b0 tx=0x7fa064007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.077+0000 7fa071600640 1 -- 172.21.15.19:0/3925515017 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa06c01f020 con 0x7fa074160b90 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.281+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa074050740 con 0x7fa074160b90 2024-04-25T11:41:36.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.283+0000 7fa071600640 1 -- 172.21.15.19:0/3925515017 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+21326 (secure 0 0 0) 0x7fa06c05fd30 con 0x7fa074160b90 2024-04-25T11:41:36.270 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:36.270 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":128,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:41:18.367381+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6834/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.286+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa050074500 msgr2=0x7fa050076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.286+0000 7fa07a000640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa050074500 0x7fa050076a10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa0640070b0 tx=0x7fa064007040 comp rx=0 tx=0).stop 2024-04-25T11:41:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.286+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa074160b90 msgr2=0x7fa074147d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.286+0000 7fa07a000640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa074160b90 0x7fa074147d10 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00c7f0 tx=0x7fa06c00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 shutdown_connections 2024-04-25T11:41:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa050074500 0x7fa050076a10 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0740d50e0 0x7fa074158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa074160b90 0x7fa074147d10 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 --2- 172.21.15.19:0/3925515017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0740d2ae0 0x7fa0740c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 >> 172.21.15.19:0/3925515017 conn(0x7fa07406c5c0 msgr2=0x7fa0740b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:36.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 shutdown_connections 2024-04-25T11:41:36.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.287+0000 7fa07a000640 1 -- 172.21.15.19:0/3925515017 wait complete. 2024-04-25T11:41:36.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:41:36.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:41:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.603+0000 7ff18b000640 1 Processor -- start 2024-04-25T11:41:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.603+0000 7ff18b000640 1 -- start start 2024-04-25T11:41:36.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff18b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff18406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff18b000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff18406db10 0x7ff18415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff18b000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff18406df20 con 0x7ff18415ec90 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff18b000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff184144750 con 0x7ff18406db10 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff18b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff184070000 con 0x7ff184144340 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff189200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff18406db10 0x7ff18415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff189200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff18406db10 0x7ff18415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57456/0 (socket says 172.21.15.19:57456) 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff189200640 1 -- 172.21.15.19:0/3631224670 learned_addr learned my addr 172.21.15.19:0/3631224670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:36.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.605+0000 7ff18a600640 1 --1- 172.21.15.19:0/3631224670 >> v1:172.21.15.19:6789/0 conn(0x7ff18415ec90 0x7ff184161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49292/0 (socket says 172.21.15.19:49292) 2024-04-25T11:41:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.606+0000 7ff189c00640 1 --2- 172.21.15.19:0/3631224670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff18406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.606+0000 7ff189200640 1 -- 172.21.15.19:0/3631224670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 msgr2=0x7ff18406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.606+0000 7ff189200640 1 --2- 172.21.15.19:0/3631224670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff18406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.606+0000 7ff189200640 1 -- 172.21.15.19:0/3631224670 >> v1:172.21.15.19:6789/0 conn(0x7ff18415ec90 legacy=0x7ff184161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.606+0000 7ff189200640 1 -- 172.21.15.19:0/3631224670 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18406f240 con 0x7ff18406db10 2024-04-25T11:41:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.606+0000 7ff183e00640 1 -- 172.21.15.19:0/3631224670 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2903194095 0 0) 0x7ff17c001490 con 0x7ff18415ec90 2024-04-25T11:41:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.607+0000 7ff189200640 1 --2- 172.21.15.19:0/3631224670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18406db10 0x7ff18415e750 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7ff170009870 tx=0x7ff17002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170241de4e5c3f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.607+0000 7ff183e00640 1 -- 172.21.15.19:0/3631224670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170037070 con 0x7ff18406db10 2024-04-25T11:41:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.607+0000 7ff183e00640 1 -- 172.21.15.19:0/3631224670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff170032440 con 0x7ff18406db10 2024-04-25T11:41:36.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.607+0000 7ff183e00640 1 -- 172.21.15.19:0/3631224670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170032810 con 0x7ff18406db10 2024-04-25T11:41:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.608+0000 7ff18b000640 1 -- 172.21.15.19:0/3631224670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18406db10 msgr2=0x7ff18415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.608+0000 7ff18b000640 1 --2- 172.21.15.19:0/3631224670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18406db10 0x7ff18415e750 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7ff170009870 tx=0x7ff17002a250 comp rx=0 tx=0).stop 2024-04-25T11:41:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.608+0000 7ff18b000640 1 -- 172.21.15.19:0/3631224670 shutdown_connections 2024-04-25T11:41:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.608+0000 7ff18b000640 1 --2- 172.21.15.19:0/3631224670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18406db10 0x7ff18415e750 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.608+0000 7ff18b000640 1 --2- 172.21.15.19:0/3631224670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff18406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.608+0000 7ff18b000640 1 -- 172.21.15.19:0/3631224670 >> 172.21.15.19:0/3631224670 conn(0x7ff184072f40 msgr2=0x7ff184073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:36.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.609+0000 7ff18b000640 1 -- 172.21.15.19:0/3631224670 shutdown_connections 2024-04-25T11:41:36.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.609+0000 7ff18b000640 1 -- 172.21.15.19:0/3631224670 wait complete. 2024-04-25T11:41:36.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.611+0000 7ff18b000640 1 Processor -- start 2024-04-25T11:41:36.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.611+0000 7ff18b000640 1 -- start start 2024-04-25T11:41:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18b000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18406db10 0x7ff1841f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18b000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff1841f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18415ec90 0x7ff184161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18b000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff18406fae0 con 0x7ff18415ec90 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18b000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff18406f800 con 0x7ff18406db10 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18b000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff18415f0a0 con 0x7ff184144340 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18415ec90 0x7ff184161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18415ec90 0x7ff184161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57472/0 (socket says 172.21.15.19:57472) 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.613+0000 7ff18a600640 1 -- 172.21.15.19:0/3635820024 learned_addr learned my addr 172.21.15.19:0/3635820024 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff189200640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff1841f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff189c00640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18406db10 0x7ff1841f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff18a600640 1 -- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 msgr2=0x7ff1841f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff18a600640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff1841f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff18a600640 1 -- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18406db10 msgr2=0x7ff1841f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff18a600640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18406db10 0x7ff1841f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff18a600640 1 -- 172.21.15.19:0/3635820024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18415f4a0 con 0x7ff18415ec90 2024-04-25T11:41:36.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff18a600640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18415ec90 0x7ff184161640 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00c730 tx=0x7ff17c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.614+0000 7ff181600640 1 -- 172.21.15.19:0/3635820024 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c018070 con 0x7ff18415ec90 2024-04-25T11:41:36.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.615+0000 7ff181600640 1 -- 172.21.15.19:0/3635820024 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17c00ce40 con 0x7ff18415ec90 2024-04-25T11:41:36.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.615+0000 7ff181600640 1 -- 172.21.15.19:0/3635820024 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00a860 con 0x7ff18415ec90 2024-04-25T11:41:36.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.615+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1841fa6d0 con 0x7ff18415ec90 2024-04-25T11:41:36.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.615+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1841fac90 con 0x7ff18415ec90 2024-04-25T11:41:36.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.616+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff184076c60 con 0x7ff18415ec90 2024-04-25T11:41:36.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.617+0000 7ff181600640 1 -- 172.21.15.19:0/3635820024 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff17c01b070 con 0x7ff18415ec90 2024-04-25T11:41:36.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.618+0000 7ff181600640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1640744b0 0x7ff1640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.618+0000 7ff181600640 1 -- 172.21.15.19:0/3635820024 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7ff17c095340 con 0x7ff18415ec90 2024-04-25T11:41:36.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.619+0000 7ff189c00640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1640744b0 0x7ff1640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.620+0000 7ff189c00640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1640744b0 0x7ff1640769c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff178006fd0 tx=0x7ff178008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.623+0000 7ff181600640 1 -- 172.21.15.19:0/3635820024 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff18406df20 con 0x7ff18415ec90 2024-04-25T11:41:36.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.828+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1841fa460 con 0x7ff18415ec90 2024-04-25T11:41:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.829+0000 7ff181600640 1 -- 172.21.15.19:0/3635820024 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7ff17c05ef20 con 0x7ff18415ec90 2024-04-25T11:41:36.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:41:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1640744b0 msgr2=0x7ff1640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1640744b0 0x7ff1640769c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff178006fd0 tx=0x7ff178008040 comp rx=0 tx=0).stop 2024-04-25T11:41:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18415ec90 msgr2=0x7ff184161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18415ec90 0x7ff184161640 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00c730 tx=0x7ff17c00cbf0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 shutdown_connections 2024-04-25T11:41:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1640744b0 0x7ff1640769c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff18415ec90 0x7ff184161640 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff184144340 0x7ff1841f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 --2- 172.21.15.19:0/3635820024 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff18406db10 0x7ff1841f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 >> 172.21.15.19:0/3635820024 conn(0x7ff184072f40 msgr2=0x7ff18415ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 shutdown_connections 2024-04-25T11:41:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:36.832+0000 7ff18b000640 1 -- 172.21.15.19:0/3635820024 wait complete. 2024-04-25T11:41:36.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:41:36.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:41:36.906 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:41:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.141+0000 7f961be00640 1 Processor -- start 2024-04-25T11:41:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.141+0000 7f961be00640 1 -- start start 2024-04-25T11:41:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.143+0000 7f961be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f961c14aac0 0x7f961c1655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c0b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f961c165ae0 con 0x7f961c06d5d0 2024-04-25T11:41:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f961c14aed0 con 0x7f961c14aac0 2024-04-25T11:41:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f961c06f460 con 0x7f961c0b50b0 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f961c14aac0 0x7f961c1655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f961c14aac0 0x7f961c1655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57482/0 (socket says 172.21.15.19:57482) 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961aa00640 1 -- 172.21.15.19:0/1471985542 learned_addr learned my addr 172.21.15.19:0/1471985542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.144+0000 7f961b400640 1 --2- 172.21.15.19:0/1471985542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c0b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.145+0000 7f961aa00640 1 -- 172.21.15.19:0/1471985542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 msgr2=0x7f961c0b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.145+0000 7f961aa00640 1 --2- 172.21.15.19:0/1471985542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c0b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.145+0000 7f961aa00640 1 -- 172.21.15.19:0/1471985542 >> v1:172.21.15.169:6789/0 conn(0x7f961c06d5d0 legacy=0x7f961c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.145+0000 7f961aa00640 1 -- 172.21.15.19:0/1471985542 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c06f740 con 0x7f961c14aac0 2024-04-25T11:41:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.146+0000 7f961aa00640 1 --2- 172.21.15.19:0/1471985542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c14aac0 0x7f961c1655a0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f960c009870 tx=0x7f960c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7956804456dcc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.147+0000 7f9619600640 1 -- 172.21.15.19:0/1471985542 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c037070 con 0x7f961c14aac0 2024-04-25T11:41:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.147+0000 7f9619600640 1 -- 172.21.15.19:0/1471985542 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c032440 con 0x7f961c14aac0 2024-04-25T11:41:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.147+0000 7f9619600640 1 -- 172.21.15.19:0/1471985542 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c032810 con 0x7f961c14aac0 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.147+0000 7f961be00640 1 -- 172.21.15.19:0/1471985542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c14aac0 msgr2=0x7f961c1655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.147+0000 7f961be00640 1 --2- 172.21.15.19:0/1471985542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c14aac0 0x7f961c1655a0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f960c009870 tx=0x7f960c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.148+0000 7f961be00640 1 -- 172.21.15.19:0/1471985542 shutdown_connections 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.148+0000 7f961be00640 1 --2- 172.21.15.19:0/1471985542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c0b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.148+0000 7f961be00640 1 --2- 172.21.15.19:0/1471985542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c14aac0 0x7f961c1655a0 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.148+0000 7f961be00640 1 -- 172.21.15.19:0/1471985542 >> 172.21.15.19:0/1471985542 conn(0x7f961c072f40 msgr2=0x7f961c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.148+0000 7f961be00640 1 -- 172.21.15.19:0/1471985542 shutdown_connections 2024-04-25T11:41:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.148+0000 7f961be00640 1 -- 172.21.15.19:0/1471985542 wait complete. 2024-04-25T11:41:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.150+0000 7f961be00640 1 Processor -- start 2024-04-25T11:41:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.150+0000 7f961be00640 1 -- start start 2024-04-25T11:41:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c06d5d0 0x7f961c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f961c14aac0 0x7f961c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f961c06ffe0 con 0x7f961c06d5d0 2024-04-25T11:41:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f961c06fd00 con 0x7f961c14aac0 2024-04-25T11:41:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f961c0b4a90 con 0x7f961c0b50b0 2024-04-25T11:41:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c06d5d0 0x7f961c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c06d5d0 0x7f961c15cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57488/0 (socket says 172.21.15.19:57488) 2024-04-25T11:41:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.152+0000 7f961aa00640 1 -- 172.21.15.19:0/420281203 learned_addr learned my addr 172.21.15.19:0/420281203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961a000640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961b400640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f961c14aac0 0x7f961c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961aa00640 1 -- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 msgr2=0x7f961c15dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961aa00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c15dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961aa00640 1 -- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f961c14aac0 msgr2=0x7f961c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961aa00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f961c14aac0 0x7f961c15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961aa00640 1 -- 172.21.15.19:0/420281203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c15d1b0 con 0x7f961c06d5d0 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.153+0000 7f961aa00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c06d5d0 0x7f961c15cc70 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f960c009870 tx=0x7f960c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.154+0000 7f9613400640 1 -- 172.21.15.19:0/420281203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c041020 con 0x7f961c06d5d0 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.154+0000 7f9613400640 1 -- 172.21.15.19:0/420281203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c0306c0 con 0x7f961c06d5d0 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.154+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c166bb0 con 0x7f961c06d5d0 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.154+0000 7f9613400640 1 -- 172.21.15.19:0/420281203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c037040 con 0x7f961c06d5d0 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.154+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c167170 con 0x7f961c06d5d0 2024-04-25T11:41:37.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.155+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c076c60 con 0x7f961c06d5d0 2024-04-25T11:41:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.156+0000 7f9613400640 1 -- 172.21.15.19:0/420281203 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f960c044020 con 0x7f961c06d5d0 2024-04-25T11:41:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.157+0000 7f9613400640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95f80744e0 0x7f95f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.158+0000 7f9613400640 1 -- 172.21.15.19:0/420281203 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f960c0b66b0 con 0x7f961c06d5d0 2024-04-25T11:41:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.158+0000 7f961a000640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95f80744e0 0x7f95f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.159+0000 7f961a000640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95f80744e0 0x7f95f80769f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b1550 tx=0x7f9604009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.162+0000 7f9613400640 1 -- 172.21.15.19:0/420281203 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f960c04b030 con 0x7f961c06d5d0 2024-04-25T11:41:37.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.366+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f961c0cf1a0 con 0x7f961c06d5d0 2024-04-25T11:41:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.367+0000 7f9613400640 1 -- 172.21.15.19:0/420281203 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7f961c15d1b0 con 0x7f961c06d5d0 2024-04-25T11:41:37.353 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:41:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95f80744e0 msgr2=0x7f95f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95f80744e0 0x7f95f80769f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b1550 tx=0x7f9604009040 comp rx=0 tx=0).stop 2024-04-25T11:41:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c06d5d0 msgr2=0x7f961c15cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c06d5d0 0x7f961c15cc70 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f960c009870 tx=0x7f960c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 shutdown_connections 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f95f80744e0 0x7f95f80769f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f961c14aac0 0x7f961c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f961c0b50b0 0x7f961c15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 --2- 172.21.15.19:0/420281203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f961c06d5d0 0x7f961c15cc70 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 >> 172.21.15.19:0/420281203 conn(0x7f961c072f40 msgr2=0x7f961c0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 shutdown_connections 2024-04-25T11:41:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.370+0000 7f961be00640 1 -- 172.21.15.19:0/420281203 wait complete. 2024-04-25T11:41:37.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:41:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.686+0000 7fecc1000640 1 Processor -- start 2024-04-25T11:41:37.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.686+0000 7fecc1000640 1 -- start start 2024-04-25T11:41:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.688+0000 7fecc1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fecbc070d20 0x7fecbc161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.688+0000 7fecc1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fecbc06f930 0x7fecbc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.688+0000 7fecc1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecbc1623f0 con 0x7fecbc06fd40 2024-04-25T11:41:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.688+0000 7fecc1000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fecbc1626d0 con 0x7fecbc070d20 2024-04-25T11:41:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.688+0000 7fecc1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fecbc06f130 con 0x7fecbc06f930 2024-04-25T11:41:37.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fecbc070d20 0x7fecbc161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fecbc070d20 0x7fecbc161b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57500/0 (socket says 172.21.15.19:57500) 2024-04-25T11:41:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbb400640 1 -- 172.21.15.19:0/1308089716 learned_addr learned my addr 172.21.15.19:0/1308089716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbaa00640 1 --2- 172.21.15.19:0/1308089716 >> v2:172.21.15.169:3300/0 conn(0x7fecbc06f930 0x7fecbc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbb400640 1 -- 172.21.15.19:0/1308089716 >> v2:172.21.15.169:3300/0 conn(0x7fecbc06f930 msgr2=0x7fecbc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbb400640 1 --2- 172.21.15.19:0/1308089716 >> v2:172.21.15.169:3300/0 conn(0x7fecbc06f930 0x7fecbc06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbb400640 1 -- 172.21.15.19:0/1308089716 >> v1:172.21.15.169:6789/0 conn(0x7fecbc06fd40 legacy=0x7fecbc0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:37.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbb400640 1 -- 172.21.15.19:0/1308089716 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc06f410 con 0x7fecbc070d20 2024-04-25T11:41:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.689+0000 7fecbaa00640 1 --2- 172.21.15.19:0/1308089716 >> v2:172.21.15.169:3300/0 conn(0x7fecbc06f930 0x7fecbc06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.690+0000 7fecbb400640 1 --2- 172.21.15.19:0/1308089716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc070d20 0x7fecbc161b30 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fecbc161a90 tx=0x7fecac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3147e8560adfa09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.691+0000 7fecba000640 1 -- 172.21.15.19:0/1308089716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac037070 con 0x7fecbc070d20 2024-04-25T11:41:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.691+0000 7fecba000640 1 -- 172.21.15.19:0/1308089716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecac032440 con 0x7fecbc070d20 2024-04-25T11:41:37.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.691+0000 7fecba000640 1 -- 172.21.15.19:0/1308089716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac032810 con 0x7fecbc070d20 2024-04-25T11:41:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 -- 172.21.15.19:0/1308089716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc070d20 msgr2=0x7fecbc161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 --2- 172.21.15.19:0/1308089716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc070d20 0x7fecbc161b30 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fecbc161a90 tx=0x7fecac02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 -- 172.21.15.19:0/1308089716 shutdown_connections 2024-04-25T11:41:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 --2- 172.21.15.19:0/1308089716 >> v2:172.21.15.169:3300/0 conn(0x7fecbc06f930 0x7fecbc06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 --2- 172.21.15.19:0/1308089716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc070d20 0x7fecbc161b30 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 -- 172.21.15.19:0/1308089716 >> 172.21.15.19:0/1308089716 conn(0x7fecbc073160 msgr2=0x7fecbc073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 -- 172.21.15.19:0/1308089716 shutdown_connections 2024-04-25T11:41:37.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.692+0000 7fecc1000640 1 -- 172.21.15.19:0/1308089716 wait complete. 2024-04-25T11:41:37.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.694+0000 7fecc1000640 1 Processor -- start 2024-04-25T11:41:37.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.695+0000 7fecc1000640 1 -- start start 2024-04-25T11:41:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.696+0000 7fecc1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc06f930 0x7fecbc1f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.696+0000 7fecc1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecbc06fd40 0x7fecbc0cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.696+0000 7fecc1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecbc070d20 0x7fecbc0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.696+0000 7fecc1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc15e2f0 con 0x7fecbc06f930 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.696+0000 7fecc1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc162070 con 0x7fecbc070d20 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.696+0000 7fecc1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fecbc15e470 con 0x7fecbc06fd40 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.696+0000 7fecbb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc06f930 0x7fecbc1f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc06f930 0x7fecbc1f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57512/0 (socket says 172.21.15.19:57512) 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 -- 172.21.15.19:0/2327841335 learned_addr learned my addr 172.21.15.19:0/2327841335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 -- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecbc06fd40 msgr2=0x7fecbc0cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbaa00640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecbc06fd40 0x7fecbc0cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbbe00640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecbc070d20 0x7fecbc0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecbc06fd40 0x7fecbc0cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 -- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecbc070d20 msgr2=0x7fecbc0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecbc070d20 0x7fecbc0d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 -- 172.21.15.19:0/2327841335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc1f84a0 con 0x7fecbc06f930 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbaa00640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecbc06fd40 0x7fecbc0cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbbe00640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecbc070d20 0x7fecbc0d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.697+0000 7fecbb400640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc06f930 0x7fecbc1f7f60 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fecac000c00 tx=0x7fecac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.698+0000 7fecb3400640 1 -- 172.21.15.19:0/2327841335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac041020 con 0x7fecbc06f930 2024-04-25T11:41:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.698+0000 7fecb3400640 1 -- 172.21.15.19:0/2327841335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecac030970 con 0x7fecbc06f930 2024-04-25T11:41:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.698+0000 7fecb3400640 1 -- 172.21.15.19:0/2327841335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac037040 con 0x7fecbc06f930 2024-04-25T11:41:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.698+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc1538f0 con 0x7fecbc06f930 2024-04-25T11:41:37.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.698+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc153eb0 con 0x7fecbc06f930 2024-04-25T11:41:37.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.699+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecbc15f8b0 con 0x7fecbc06f930 2024-04-25T11:41:37.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.700+0000 7fecb3400640 1 -- 172.21.15.19:0/2327841335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fecac044020 con 0x7fecbc06f930 2024-04-25T11:41:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.701+0000 7fecb3400640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec9c0744b0 0x7fec9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.702+0000 7fecb3400640 1 -- 172.21.15.19:0/2327841335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7fecac0b6970 con 0x7fecbc06f930 2024-04-25T11:41:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.702+0000 7fecbaa00640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec9c0744b0 0x7fec9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:37.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.704+0000 7fecbaa00640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec9c0744b0 0x7fec9c0769c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feca8006fd0 tx=0x7feca8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:37.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.707+0000 7fecb3400640 1 -- 172.21.15.19:0/2327841335 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fecbc15f8b0 con 0x7fecbc06f930 2024-04-25T11:41:37.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.914+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fecbc076c60 con 0x7fecbc06f930 2024-04-25T11:41:37.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.915+0000 7fecb3400640 1 -- 172.21.15.19:0/2327841335 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7fecac04b030 con 0x7fecbc06f930 2024-04-25T11:41:37.902 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:41:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec9c0744b0 msgr2=0x7fec9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec9c0744b0 0x7fec9c0769c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feca8006fd0 tx=0x7feca8008040 comp rx=0 tx=0).stop 2024-04-25T11:41:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc06f930 msgr2=0x7fecbc1f7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc06f930 0x7fecbc1f7f60 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fecac000c00 tx=0x7fecac0046c0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 shutdown_connections 2024-04-25T11:41:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fec9c0744b0 0x7fec9c0769c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fecbc070d20 0x7fecbc0d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fecbc06fd40 0x7fecbc0cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 --2- 172.21.15.19:0/2327841335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecbc06f930 0x7fecbc1f7f60 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.918+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 >> 172.21.15.19:0/2327841335 conn(0x7fecbc073160 msgr2=0x7fecbc0af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.919+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 shutdown_connections 2024-04-25T11:41:37.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:37.919+0000 7fecc1000640 1 -- 172.21.15.19:0/2327841335 wait complete. 2024-04-25T11:41:37.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:41:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.237+0000 7fa574c00640 1 Processor -- start 2024-04-25T11:41:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.237+0000 7fa574c00640 1 -- start start 2024-04-25T11:41:38.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.239+0000 7fa574c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa57006ccd0 0x7fa5700d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.239+0000 7fa574c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5700d1850 con 0x7fa5700d4a50 2024-04-25T11:41:38.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.239+0000 7fa574c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa570160de0 con 0x7fa5700d0e00 2024-04-25T11:41:38.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.239+0000 7fa574c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa5701610c0 con 0x7fa57006ccd0 2024-04-25T11:41:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.239+0000 7fa56e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa5700d4a50 0x7fa5700d4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49304/0 (socket says 172.21.15.19:49304) 2024-04-25T11:41:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.239+0000 7fa56e800640 1 -- 172.21.15.19:0/1870062878 learned_addr learned my addr 172.21.15.19:0/1870062878 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56d400640 1 --2- 172.21.15.19:0/1870062878 >> v2:172.21.15.169:3300/0 conn(0x7fa57006ccd0 0x7fa5700d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771656965 0 0) 0x7fa5700d1850 con 0x7fa5700d4a50 2024-04-25T11:41:38.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa554003520 con 0x7fa5700d4a50 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1822096245 0 0) 0x7fa554003520 con 0x7fa5700d4a50 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 >> v2:172.21.15.169:3300/0 conn(0x7fa57006ccd0 msgr2=0x7fa5700d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56ca00640 1 --2- 172.21.15.19:0/1870062878 >> v2:172.21.15.169:3300/0 conn(0x7fa57006ccd0 0x7fa5700d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 >> v1:172.21.15.169:6789/0 conn(0x7fa5700d0e00 legacy=0x7fa57006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.240+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5701613a0 con 0x7fa5700d4a50 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa5680019f0 con 0x7fa5700d4a50 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 >> v1:172.21.15.19:6789/0 conn(0x7fa5700d4a50 legacy=0x7fa5700d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5540039b0 0x7fa554003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa554004800 0x7fa554004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554005250 0x7fa55400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5701610c0 con 0x7fa554004800 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa55400c1e0 con 0x7fa554005250 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa55400c3e0 con 0x7fa5540039b0 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56d400640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa554004800 0x7fa554004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56e800640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554005250 0x7fa55400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56d400640 1 -- 172.21.15.19:0/1870062878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5540039b0 msgr2=0x7fa554003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56d400640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa5540039b0 0x7fa554003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56d400640 1 -- 172.21.15.19:0/1870062878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554005250 msgr2=0x7fa55400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56d400640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554005250 0x7fa55400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56e800640 1 -- 172.21.15.19:0/1870062878 reap_dead start 2024-04-25T11:41:38.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.241+0000 7fa56d400640 1 -- 172.21.15.19:0/1870062878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa554003520 con 0x7fa554004800 2024-04-25T11:41:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.243+0000 7fa56d400640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa554004800 0x7fa554004c10 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009ac0 tx=0x7fa554005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851ebc28352d2fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.243+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c036070 con 0x7fa554004800 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.243+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c031400 con 0x7fa554004800 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.243+0000 7fa56ca00640 1 -- 172.21.15.19:0/1870062878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c009690 con 0x7fa554004800 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.243+0000 7fa574c00640 1 -- 172.21.15.19:0/1870062878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa554004800 msgr2=0x7fa554004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.243+0000 7fa574c00640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa554004800 0x7fa554004c10 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009ac0 tx=0x7fa554005830 comp rx=0 tx=0).stop 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.244+0000 7fa574c00640 1 -- 172.21.15.19:0/1870062878 shutdown_connections 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.244+0000 7fa574c00640 1 --2- 172.21.15.19:0/1870062878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa554004800 0x7fa554004c10 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.244+0000 7fa574c00640 1 -- 172.21.15.19:0/1870062878 >> 172.21.15.19:0/1870062878 conn(0x7fa5700d5d90 msgr2=0x7fa5700d61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.244+0000 7fa574c00640 1 -- 172.21.15.19:0/1870062878 shutdown_connections 2024-04-25T11:41:38.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.244+0000 7fa574c00640 1 -- 172.21.15.19:0/1870062878 wait complete. 2024-04-25T11:41:38.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.246+0000 7fa574c00640 1 Processor -- start 2024-04-25T11:41:38.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.246+0000 7fa574c00640 1 -- start start 2024-04-25T11:41:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa574c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa554004800 0x7fa5700ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa574c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57014ae50 0x7fa57014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa574c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5700b6de0 0x7fa5700b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa574c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa570164e00 con 0x7fa57014ae50 2024-04-25T11:41:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa574c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa5700b5320 con 0x7fa5700b6de0 2024-04-25T11:41:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa574c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa570164f80 con 0x7fa554004800 2024-04-25T11:41:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa56d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57014ae50 0x7fa57014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa56d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57014ae50 0x7fa57014b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57528/0 (socket says 172.21.15.19:57528) 2024-04-25T11:41:38.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa56d400640 1 -- 172.21.15.19:0/1884817639 learned_addr learned my addr 172.21.15.19:0/1884817639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.248+0000 7fa56de00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa554004800 0x7fa5700ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56d400640 1 -- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa554004800 msgr2=0x7fa5700ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56d400640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa554004800 0x7fa5700ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56e800640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5700b6de0 0x7fa5700b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56d400640 1 -- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5700b6de0 msgr2=0x7fa5700b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56d400640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5700b6de0 0x7fa5700b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56d400640 1 -- 172.21.15.19:0/1884817639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57014b7a0 con 0x7fa57014ae50 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56de00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa554004800 0x7fa5700ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56e800640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5700b6de0 0x7fa5700b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa56d400640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57014ae50 0x7fa57014b260 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fa55c034040 tx=0x7fa55c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa566000640 1 -- 172.21.15.19:0/1884817639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c03f020 con 0x7fa57014ae50 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa566000640 1 -- 172.21.15.19:0/1884817639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c037040 con 0x7fa57014ae50 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa566000640 1 -- 172.21.15.19:0/1884817639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c036070 con 0x7fa57014ae50 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.249+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5700cc3a0 con 0x7fa57014ae50 2024-04-25T11:41:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.250+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5700b6300 con 0x7fa57014ae50 2024-04-25T11:41:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.251+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5700d4320 con 0x7fa57014ae50 2024-04-25T11:41:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.252+0000 7fa566000640 1 -- 172.21.15.19:0/1884817639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa55c042020 con 0x7fa57014ae50 2024-04-25T11:41:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.253+0000 7fa566000640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5500744b0 0x7fa5500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.253+0000 7fa566000640 1 -- 172.21.15.19:0/1884817639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7fa55c0b60c0 con 0x7fa57014ae50 2024-04-25T11:41:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.254+0000 7fa56de00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5500744b0 0x7fa5500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.256+0000 7fa56de00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5500744b0 0x7fa5500769c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa57006bc20 tx=0x7fa560009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.258+0000 7fa566000640 1 -- 172.21.15.19:0/1884817639 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa55c049030 con 0x7fa57014ae50 2024-04-25T11:41:38.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.483+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa570148750 con 0x7fa57014ae50 2024-04-25T11:41:38.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.485+0000 7fa566000640 1 -- 172.21.15.19:0/1884817639 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 72 entries v128) ==== 81+0+4366 (secure 0 0 0) 0x7fa55c07fc40 con 0x7fa57014ae50 2024-04-25T11:41:38.471 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/295899246 2024-04-26T11:41:18.367356+0000 2024-04-25T11:41:38.471 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/295899246 2024-04-26T11:41:18.367356+0000 2024-04-25T11:41:38.471 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2088909328 2024-04-26T11:41:11.987047+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/698241108 2024-04-26T11:40:28.419530+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/698241108 2024-04-26T11:40:28.419530+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2891720740 2024-04-26T11:40:22.060358+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2891720740 2024-04-26T11:40:22.060358+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/527442501 2024-04-26T11:37:46.017825+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2088909328 2024-04-26T11:41:11.987047+0000 2024-04-25T11:41:38.472 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3000889873 2024-04-26T11:39:15.575600+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/571302363 2024-04-26T11:39:50.044945+0000 2024-04-25T11:41:38.473 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2855003271 2024-04-26T11:36:56.884963+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3307408746 2024-04-26T11:37:23.802616+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/390026662 2024-04-26T11:37:15.698053+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/258414220 2024-04-26T11:40:56.345928+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2716758396 2024-04-26T11:35:28.536900+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1755072450 2024-04-26T11:39:43.970561+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1160647715 2024-04-26T11:38:45.171692+0000 2024-04-25T11:41:38.474 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1743540779 2024-04-26T11:36:01.298382+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2103355299 2024-04-26T11:37:42.702999+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/2406308121 2024-04-26T11:39:56.138589+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/258414220 2024-04-26T11:40:56.345928+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/46202637 2024-04-26T11:35:54.966496+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2065862570 2024-04-26T11:38:09.150250+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/845688948 2024-04-26T11:37:06.603392+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3471202301 2024-04-26T11:37:55.127992+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/432323448 2024-04-26T11:36:41.348188+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:41:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3516161211 2024-04-26T11:36:10.407078+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3555498549 2024-04-26T11:36:16.444892+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2645472884 2024-04-26T11:36:22.535971+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/2271566376 2024-04-26T11:38:03.222860+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/614168622 2024-04-26T11:38:36.047498+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/896318924 2024-04-26T11:38:15.379181+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3710093903 2024-04-26T11:38:39.094007+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3430409163 2024-04-26T11:37:00.526677+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/611091976 2024-04-26T11:38:51.248979+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1758836153 2024-04-26T11:38:57.342691+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1735507451 2024-04-26T11:39:03.424646+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1200575562 2024-04-26T11:39:09.170950+0000 2024-04-25T11:41:38.476 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/3564168877 2024-04-26T11:39:34.386529+0000 2024-04-25T11:41:38.477 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:41:38.477 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/4272759579 2024-04-26T11:39:37.896090+0000 2024-04-25T11:41:38.477 INFO:teuthology.orchestra.run.smithi019.stderr:listed 72 entries 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5500744b0 msgr2=0x7fa5500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5500744b0 0x7fa5500769c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa57006bc20 tx=0x7fa560009040 comp rx=0 tx=0).stop 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57014ae50 msgr2=0x7fa57014b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57014ae50 0x7fa57014b260 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fa55c034040 tx=0x7fa55c004730 comp rx=0 tx=0).stop 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 shutdown_connections 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa5500744b0 0x7fa5500769c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5700b6de0 0x7fa5700b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa57014ae50 0x7fa57014b260 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 --2- 172.21.15.19:0/1884817639 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa554004800 0x7fa5700ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 >> 172.21.15.19:0/1884817639 conn(0x7fa5700d5d90 msgr2=0x7fa570162f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.488+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 shutdown_connections 2024-04-25T11:41:38.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.489+0000 7fa574c00640 1 -- 172.21.15.19:0/1884817639 wait complete. 2024-04-25T11:41:38.500 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:41:38.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:38.501 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:41:38.610 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:41:38.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:38.610 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:41:38.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:41:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.956+0000 7f5751c00640 1 Processor -- start 2024-04-25T11:41:38.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.956+0000 7f5751c00640 1 -- start start 2024-04-25T11:41:38.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.959+0000 7f5751c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f574c1434d0 0x7f574c070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.959+0000 7f5751c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574c0709c0 con 0x7f574c1612f0 2024-04-25T11:41:38.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.959+0000 7f5751c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574c06f3e0 con 0x7f574c0d0ed0 2024-04-25T11:41:38.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.959+0000 7f5751c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f574c1438e0 con 0x7f574c1434d0 2024-04-25T11:41:38.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.959+0000 7f574b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f574c1612f0 0x7f574c0b51a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49314/0 (socket says 172.21.15.19:49314) 2024-04-25T11:41:38.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.959+0000 7f574b400640 1 -- 172.21.15.19:0/3376357177 learned_addr learned my addr 172.21.15.19:0/3376357177 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:38.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f574aa00640 1 --2- 172.21.15.19:0/3376357177 >> v2:172.21.15.169:3300/0 conn(0x7f574c1434d0 0x7f574c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303223760 0 0) 0x7f574c0709c0 con 0x7f574c1612f0 2024-04-25T11:41:38.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5730003520 con 0x7f574c1612f0 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1306175725 0 0) 0x7f5730003520 con 0x7f574c1612f0 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 >> v2:172.21.15.169:3300/0 conn(0x7f574c1434d0 msgr2=0x7f574c070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 --2- 172.21.15.19:0/3376357177 >> v2:172.21.15.169:3300/0 conn(0x7f574c1434d0 0x7f574c070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 >> v1:172.21.15.169:6789/0 conn(0x7f574c0d0ed0 legacy=0x7f574c160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c06f6c0 con 0x7f574c1612f0 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f5744001670 con 0x7f574c1612f0 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 >> v1:172.21.15.19:6789/0 conn(0x7f574c1612f0 legacy=0x7f574c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57300039b0 0x7f5730003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5730004800 0x7f5730004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f573000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f574c1438e0 con 0x7f57300051c0 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f573000e370 con 0x7f5730004800 2024-04-25T11:41:38.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.960+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f573000e570 con 0x7f57300039b0 2024-04-25T11:41:38.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.961+0000 7f574b400640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f573000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.961+0000 7f574b400640 1 -- 172.21.15.19:0/3376357177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57300039b0 msgr2=0x7f5730003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:38.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.961+0000 7f574b400640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57300039b0 0x7f5730003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.961+0000 7f574b400640 1 -- 172.21.15.19:0/3376357177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5730004800 msgr2=0x7f5730004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:38.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.961+0000 7f574b400640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5730004800 0x7f5730004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.961+0000 7f574b400640 1 -- 172.21.15.19:0/3376357177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5730003520 con 0x7f57300051c0 2024-04-25T11:41:38.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.962+0000 7f574b400640 1 -- 172.21.15.19:0/3376357177 reap_dead start 2024-04-25T11:41:38.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.962+0000 7f574b400640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f573000dbd0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f574400c240 tx=0x7f5730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a8592ad64cd54c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.962+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744007cf0 con 0x7f57300051c0 2024-04-25T11:41:38.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.962+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5744005af0 con 0x7f57300051c0 2024-04-25T11:41:38.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.962+0000 7f5749600640 1 -- 172.21.15.19:0/3376357177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744007070 con 0x7f57300051c0 2024-04-25T11:41:38.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.963+0000 7f5751c00640 1 -- 172.21.15.19:0/3376357177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 msgr2=0x7f573000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.963+0000 7f5751c00640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f573000dbd0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f574400c240 tx=0x7f5730005830 comp rx=0 tx=0).stop 2024-04-25T11:41:38.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.963+0000 7f5751c00640 1 -- 172.21.15.19:0/3376357177 shutdown_connections 2024-04-25T11:41:38.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.963+0000 7f5751c00640 1 --2- 172.21.15.19:0/3376357177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f573000dbd0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.963+0000 7f5751c00640 1 -- 172.21.15.19:0/3376357177 >> 172.21.15.19:0/3376357177 conn(0x7f574c0d1ae0 msgr2=0x7f574c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:38.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.963+0000 7f5751c00640 1 -- 172.21.15.19:0/3376357177 shutdown_connections 2024-04-25T11:41:38.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.963+0000 7f5751c00640 1 -- 172.21.15.19:0/3376357177 wait complete. 2024-04-25T11:41:38.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.966+0000 7f5751c00640 1 Processor -- start 2024-04-25T11:41:38.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.966+0000 7f5751c00640 1 -- start start 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.967+0000 7f5751c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f574c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f5751c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c1598d0 0x7f574c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f5751c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c15b8f0 0x7f574c15bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f5751c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f574c06f090 con 0x7f57300051c0 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f5751c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f574c06fc80 con 0x7f574c15b8f0 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f5751c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f574c070480 con 0x7f574c1598d0 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f574c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f574c1466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57538/0 (socket says 172.21.15.19:57538) 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 -- 172.21.15.19:0/1133345786 learned_addr learned my addr 172.21.15.19:0/1133345786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 -- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c1598d0 msgr2=0x7f574c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c1598d0 0x7f574c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 -- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c15b8f0 msgr2=0x7f574c15bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c15b8f0 0x7f574c15bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.968+0000 7f574aa00640 1 -- 172.21.15.19:0/1133345786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c15c240 con 0x7f57300051c0 2024-04-25T11:41:38.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.969+0000 7f574aa00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f574c1466f0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f573c00ef40 tx=0x7f573c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.969+0000 7f5742a00640 1 -- 172.21.15.19:0/1133345786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c018020 con 0x7f57300051c0 2024-04-25T11:41:38.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.969+0000 7f5742a00640 1 -- 172.21.15.19:0/1133345786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c014400 con 0x7f57300051c0 2024-04-25T11:41:38.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.969+0000 7f5742a00640 1 -- 172.21.15.19:0/1133345786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c0147f0 con 0x7f57300051c0 2024-04-25T11:41:38.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.969+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c146c30 con 0x7f57300051c0 2024-04-25T11:41:38.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.969+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c15a8c0 con 0x7f57300051c0 2024-04-25T11:41:38.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.970+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c0d0cf0 con 0x7f57300051c0 2024-04-25T11:41:38.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.972+0000 7f5742a00640 1 -- 172.21.15.19:0/1133345786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f573c005d10 con 0x7f57300051c0 2024-04-25T11:41:38.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.973+0000 7f5742a00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f57280744b0 0x7f57280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.973+0000 7f5742a00640 1 -- 172.21.15.19:0/1133345786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8805+0+0 (secure 0 0 0) 0x7f573c096530 con 0x7f57300051c0 2024-04-25T11:41:38.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.974+0000 7f574a000640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f57280744b0 0x7f57280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.975+0000 7f574a000640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f57280744b0 0x7f57280769c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f574c15e550 tx=0x7f57340079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:38.979+0000 7f5742a00640 1 -- 172.21.15.19:0/1133345786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f574c0d0cf0 con 0x7f57300051c0 2024-04-25T11:41:39.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.224+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f574c050740 con 0x7f57300051c0 2024-04-25T11:41:39.212 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:39.226+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:41:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.230+0000 7f5742a00640 1 -- 172.21.15.19:0/1133345786 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v333) ==== 161+0+0 (secure 0 0 0) 0x7f573c022070 con 0x7f57300051c0 2024-04-25T11:41:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:41:39.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.233+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f57280744b0 msgr2=0x7f57280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.233+0000 7f5751c00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f57280744b0 0x7f57280769c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f574c15e550 tx=0x7f57340079b0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.233+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 msgr2=0x7f574c1466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.233+0000 7f5751c00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f574c1466f0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f573c00ef40 tx=0x7f573c00b480 comp rx=0 tx=0).stop 2024-04-25T11:41:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.234+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 shutdown_connections 2024-04-25T11:41:39.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.234+0000 7f5751c00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f57280744b0 0x7f57280769c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.234+0000 7f5751c00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f574c15b8f0 0x7f574c15bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.234+0000 7f5751c00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f574c1598d0 0x7f574c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.234+0000 7f5751c00640 1 --2- 172.21.15.19:0/1133345786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57300051c0 0x7f574c1466f0 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.234+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 >> 172.21.15.19:0/1133345786 conn(0x7f574c0d1ae0 msgr2=0x7f574c0b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.234+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 shutdown_connections 2024-04-25T11:41:39.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.235+0000 7f5751c00640 1 -- 172.21.15.19:0/1133345786 wait complete. 2024-04-25T11:41:39.223 INFO:tasks.ceph.mds.a.smithi019.stderr: -1906> 2024-04-25T11:41:26.454+0000 7f5993200640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-04-25T11:41:39.227 INFO:tasks.ceph.mds.a.smithi019.stderr: -646> 2024-04-25T11:41:34.396+0000 7f5998200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:41:39.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:41:39.357 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:41:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.543+0000 7f5494c00640 1 Processor -- start 2024-04-25T11:41:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.543+0000 7f5494c00640 1 -- start start 2024-04-25T11:41:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.546+0000 7f5494c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549006d900 0x7f54900ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.546+0000 7f5494c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f54900cc5c0 0x7f54900ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.546+0000 7f5494c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54900760b0 con 0x7f549015e2b0 2024-04-25T11:41:39.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.546+0000 7f5494c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f54900b7720 con 0x7f54900cc5c0 2024-04-25T11:41:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.546+0000 7f5494c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f54900b7a00 con 0x7f549006d900 2024-04-25T11:41:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.547+0000 7f548f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f549015e2b0 0x7f549006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49322/0 (socket says 172.21.15.19:49322) 2024-04-25T11:41:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.547+0000 7f548f400640 1 -- 172.21.15.19:0/4278670786 learned_addr learned my addr 172.21.15.19:0/4278670786 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.547+0000 7f5486a00640 1 --2- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549006d900 0x7f54900ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.547+0000 7f548fe00640 1 --2- 172.21.15.19:0/4278670786 >> v2:172.21.15.169:3300/0 conn(0x7f54900cc5c0 0x7f54900ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.548+0000 7f548fe00640 1 -- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549006d900 msgr2=0x7f54900ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.548+0000 7f548fe00640 1 --2- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549006d900 0x7f54900ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.548+0000 7f548fe00640 1 -- 172.21.15.19:0/4278670786 >> v1:172.21.15.19:6789/0 conn(0x7f549015e2b0 legacy=0x7f549006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.548+0000 7f548fe00640 1 -- 172.21.15.19:0/4278670786 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54900b2ec0 con 0x7f54900cc5c0 2024-04-25T11:41:39.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.548+0000 7f5486a00640 1 --2- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549006d900 0x7f54900ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:41:39.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.549+0000 7f548fe00640 1 --2- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54900cc5c0 0x7f54900ce9f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5488009d90 tx=0x7f548802a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147e079eea54e7a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.557+0000 7f548ea00640 1 -- 172.21.15.19:0/4278670786 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488007740 con 0x7f54900cc5c0 2024-04-25T11:41:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.557+0000 7f548ea00640 1 -- 172.21.15.19:0/4278670786 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54880078e0 con 0x7f54900cc5c0 2024-04-25T11:41:39.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.557+0000 7f548ea00640 1 -- 172.21.15.19:0/4278670786 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488007cb0 con 0x7f54900cc5c0 2024-04-25T11:41:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 -- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54900cc5c0 msgr2=0x7f54900ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 --2- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54900cc5c0 0x7f54900ce9f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5488009d90 tx=0x7f548802a530 comp rx=0 tx=0).stop 2024-04-25T11:41:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 -- 172.21.15.19:0/4278670786 shutdown_connections 2024-04-25T11:41:39.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 --2- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54900cc5c0 0x7f54900ce9f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 --2- 172.21.15.19:0/4278670786 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f549006d900 0x7f54900ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 -- 172.21.15.19:0/4278670786 >> 172.21.15.19:0/4278670786 conn(0x7f549009b7d0 msgr2=0x7f54900b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:39.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 -- 172.21.15.19:0/4278670786 shutdown_connections 2024-04-25T11:41:39.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.558+0000 7f5494c00640 1 -- 172.21.15.19:0/4278670786 wait complete. 2024-04-25T11:41:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.560+0000 7f5494c00640 1 Processor -- start 2024-04-25T11:41:39.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.561+0000 7f5494c00640 1 -- start start 2024-04-25T11:41:39.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.562+0000 7f5494c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54900cc5c0 0x7f5490150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:39.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.562+0000 7f5494c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549015e2b0 0x7f549015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:39.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.562+0000 7f5494c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54901601f0 0x7f54900cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:39.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.562+0000 7f5494c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54900b14d0 con 0x7f549015e2b0 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.562+0000 7f5494c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f54900b2b70 con 0x7f54901601f0 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.562+0000 7f5494c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f54900b1650 con 0x7f54900cc5c0 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.562+0000 7f5486a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549015e2b0 0x7f549015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549015e2b0 0x7f549015fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57574/0 (socket says 172.21.15.19:57574) 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 -- 172.21.15.19:0/3720130828 learned_addr learned my addr 172.21.15.19:0/3720130828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f548f400640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54900cc5c0 0x7f5490150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f548fe00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54901601f0 0x7f54900cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 -- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54900cc5c0 msgr2=0x7f5490150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54900cc5c0 0x7f5490150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 -- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54901601f0 msgr2=0x7f54900cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54901601f0 0x7f54900cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:39.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 -- 172.21.15.19:0/3720130828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5490160cf0 con 0x7f549015e2b0 2024-04-25T11:41:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.563+0000 7f5486a00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549015e2b0 0x7f549015fcb0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f547800e960 tx=0x7f547800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.564+0000 7f5487e00640 1 -- 172.21.15.19:0/3720130828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478017070 con 0x7f549015e2b0 2024-04-25T11:41:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.564+0000 7f5487e00640 1 -- 172.21.15.19:0/3720130828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547800ca00 con 0x7f549015e2b0 2024-04-25T11:41:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.564+0000 7f5487e00640 1 -- 172.21.15.19:0/3720130828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547800cdd0 con 0x7f549015e2b0 2024-04-25T11:41:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.564+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5490160a10 con 0x7f549015e2b0 2024-04-25T11:41:39.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.564+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54900cfec0 con 0x7f549015e2b0 2024-04-25T11:41:39.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.565+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5490160780 con 0x7f549015e2b0 2024-04-25T11:41:39.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.566+0000 7f5487e00640 1 -- 172.21.15.19:0/3720130828 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f547801a070 con 0x7f549015e2b0 2024-04-25T11:41:39.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.567+0000 7f5487e00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f546c0744b0 0x7f546c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:39.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.567+0000 7f5487e00640 1 -- 172.21.15.19:0/3720130828 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8891+0+0 (secure 0 0 0) 0x7f5478095dc0 con 0x7f549015e2b0 2024-04-25T11:41:39.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.568+0000 7f548f400640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f546c0744b0 0x7f546c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:39.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.570+0000 7f548f400640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f546c0744b0 0x7f546c0769c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5480000f60 tx=0x7f5480009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.573+0000 7f5487e00640 1 -- 172.21.15.19:0/3720130828 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f547805f830 con 0x7f549015e2b0 2024-04-25T11:41:39.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:39.853+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5490050740 con 0x7f549015e2b0 2024-04-25T11:41:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.234+0000 7f5487e00640 1 -- 172.21.15.19:0/3720130828 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v334) ==== 107+0+0 (secure 0 0 0) 0x7f547805f130 con 0x7f549015e2b0 2024-04-25T11:41:40.221 INFO:teuthology.orchestra.run.smithi019.stderr:nothing to do: rank is not damaged 2024-04-25T11:41:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.237+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f546c0744b0 msgr2=0x7f546c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.237+0000 7f5494c00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f546c0744b0 0x7f546c0769c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5480000f60 tx=0x7f5480009040 comp rx=0 tx=0).stop 2024-04-25T11:41:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549015e2b0 msgr2=0x7f549015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:40.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549015e2b0 0x7f549015fcb0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f547800e960 tx=0x7f547800ee20 comp rx=0 tx=0).stop 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 shutdown_connections 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f546c0744b0 0x7f546c0769c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54901601f0 0x7f54900cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549015e2b0 0x7f549015fcb0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 --2- 172.21.15.19:0/3720130828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f54900cc5c0 0x7f5490150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 >> 172.21.15.19:0/3720130828 conn(0x7f549009b7d0 msgr2=0x7f549015af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 shutdown_connections 2024-04-25T11:41:40.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.238+0000 7f5494c00640 1 -- 172.21.15.19:0/3720130828 wait complete. 2024-04-25T11:41:40.255 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:41:40.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.455+0000 7ffaa55cdf40 1 Processor -- start 2024-04-25T11:41:40.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.455+0000 7ffaa55cdf40 1 -- start start 2024-04-25T11:41:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffaa55cdf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5b731e0 0x560fd5c496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffaa55cdf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b735f0 0x560fd5c41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffaa55cdf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5c4cdd0 0x560fd5c3fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffaa55cdf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560fd5bbbe70 con 0x560fd5b731e0 2024-04-25T11:41:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffaa55cdf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560fd5b19d20 con 0x560fd5c4cdd0 2024-04-25T11:41:40.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffaa55cdf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560fd5bbc3f0 con 0x560fd5b735f0 2024-04-25T11:41:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5b731e0 0x560fd5c496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5b731e0 0x560fd5c496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57594/0 (socket says 172.21.15.19:57594) 2024-04-25T11:41:40.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 -- 172.21.15.19:0/2193131046 learned_addr learned my addr 172.21.15.19:0/2193131046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b735f0 0x560fd5c41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 -- 172.21.15.19:0/2193131046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b735f0 msgr2=0x560fd5c41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b735f0 0x560fd5c41c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 -- 172.21.15.19:0/2193131046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5c4cdd0 msgr2=0x560fd5c3fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5c4cdd0 0x560fd5c3fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.457+0000 7ffa9e800640 1 -- 172.21.15.19:0/2193131046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd5c42190 con 0x560fd5b731e0 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.458+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5b731e0 0x560fd5c496c0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b5d0 tx=0x7ffa9000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6420b58fb06187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.458+0000 7ffa9d400640 1 -- 172.21.15.19:0/2193131046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90014030 con 0x560fd5b731e0 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.458+0000 7ffa9d400640 1 -- 172.21.15.19:0/2193131046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa90010620 con 0x560fd5b731e0 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.458+0000 7ffa9d400640 1 -- 172.21.15.19:0/2193131046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa900109d0 con 0x560fd5b731e0 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2193131046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5b731e0 msgr2=0x560fd5c496c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5b731e0 0x560fd5c496c0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b5d0 tx=0x7ffa9000ba90 comp rx=0 tx=0).stop 2024-04-25T11:41:40.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2193131046 shutdown_connections 2024-04-25T11:41:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5c4cdd0 0x560fd5c3fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b735f0 0x560fd5c41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2193131046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5b731e0 0x560fd5c496c0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2193131046 >> 172.21.15.19:0/2193131046 conn(0x560fd5a95ac0 msgr2=0x560fd5c49fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2193131046 shutdown_connections 2024-04-25T11:41:40.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.459+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2193131046 wait complete. 2024-04-25T11:41:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.460+0000 7ffaa55cdf40 1 Processor -- start 2024-04-25T11:41:40.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.460+0000 7ffaa55cdf40 1 -- start start 2024-04-25T11:41:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffaa55cdf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b731e0 0x560fd5c2dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffaa55cdf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5b735f0 0x560fd5cd4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffaa55cdf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5c4cdd0 0x560fd5cd5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffaa55cdf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560fd5c49c00 con 0x560fd5c4cdd0 2024-04-25T11:41:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffaa55cdf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x560fd5b72ac0 con 0x560fd5b735f0 2024-04-25T11:41:40.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffaa55cdf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x560fd5b72fc0 con 0x560fd5b731e0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5c4cdd0 0x560fd5cd5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5c4cdd0 0x560fd5cd5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57610/0 (socket says 172.21.15.19:57610) 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 learned_addr learned my addr 172.21.15.19:0/2399919490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b731e0 0x560fd5c2dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5b735f0 0x560fd5cd4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b731e0 msgr2=0x560fd5c2dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560fd5b731e0 0x560fd5c2dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5b735f0 msgr2=0x560fd5cd4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5b735f0 0x560fd5cd4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.462+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd5c2e190 con 0x560fd5c4cdd0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.463+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560fd5b735f0 0x560fd5cd4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.463+0000 7ffa9f200640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5c4cdd0 0x560fd5cd5910 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800aa50 tx=0x7ffa9800af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.463+0000 7ffa97e00640 1 -- 172.21.15.19:0/2399919490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98008df0 con 0x560fd5c4cdd0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.463+0000 7ffa97e00640 1 -- 172.21.15.19:0/2399919490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9800f070 con 0x560fd5c4cdd0 2024-04-25T11:41:40.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.463+0000 7ffa97e00640 1 -- 172.21.15.19:0/2399919490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980146b0 con 0x560fd5c4cdd0 2024-04-25T11:41:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.463+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560fd5cd5270 con 0x560fd5c4cdd0 2024-04-25T11:41:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.463+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd5cd3f70 con 0x560fd5c4cdd0 2024-04-25T11:41:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.466+0000 7ffa97e00640 1 -- 172.21.15.19:0/2399919490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffa9801e3f0 con 0x560fd5c4cdd0 2024-04-25T11:41:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.466+0000 7ffa97e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa7c072e90 0x7ffa7c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.467+0000 7ffa97e00640 1 -- 172.21.15.19:0/2399919490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8891+0+0 (secure 0 0 0) 0x7ffa980944f0 con 0x560fd5c4cdd0 2024-04-25T11:41:40.454 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:41:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.467+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560fd5c3f0f0 0x560fd5b71ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.467+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2e930 con 0x560fd5c3f0f0 2024-04-25T11:41:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.467+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa7c072e90 0x7ffa7c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.468+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560fd5c3f0f0 0x560fd5b71ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.469+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa7c072e90 0x7ffa7c0753a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000bad0 tx=0x7ffa90010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.469+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560fd5c3f0f0 0x560fd5b71ef0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.473+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v129'1128 uv1102 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.474+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cece00 con 0x560fd5c3f0f0 2024-04-25T11:41:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.477+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v129'1129 uv1129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.477+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5ced1a0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.481+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v129'1130 uv1130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.467 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:41:40.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.481+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cee550 con 0x560fd5c3f0f0 2024-04-25T11:41:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.485+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v129'1131 uv1131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.485+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cee8f0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.488+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v129'1132 uv1132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.488+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5ced540 con 0x560fd5c3f0f0 2024-04-25T11:41:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.492+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v129'1133 uv1133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.492+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5ced950 con 0x560fd5c3f0f0 2024-04-25T11:41:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.497+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v129'1134 uv1108 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.497+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cedd40 con 0x560fd5c3f0f0 2024-04-25T11:41:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.500+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v129'1135 uv1135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.500+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedd40 con 0x560fd5c3f0f0 2024-04-25T11:41:40.490 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:41:40.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.504+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v129'1136 uv1136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.504+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.508+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v129'1137 uv1137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.509+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.512+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v129'1138 uv1138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.512+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.516+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v129'1139 uv1139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.517+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.522+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v129'1140 uv1115 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.522+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.526+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v129'1141 uv1141 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.526+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.531+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v129'1142 uv1142 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.517 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:41:40.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.531+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.535+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v129'1143 uv1143 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.535+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.540+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v129'1144 uv1144 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.541+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cedcf0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.545+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v129'1145 uv1145 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.545+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.550+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v129'1146 uv1146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.550+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.556+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v129'1147 uv1121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.556+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.561+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v129'1148 uv1148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.561+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.566+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v129'1149 uv1149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.552 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:41:40.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.566+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.571+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v129'1150 uv1150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.571+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.575+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v129'1151 uv1151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.576+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.580+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v129'1152 uv1152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.581+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.586+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v129'1153 uv1127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.586+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.591+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v129'1154 uv1154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.591+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.596+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v129'1155 uv1155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.583 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:41:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.596+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.601+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v129'1156 uv1156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.601+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.606+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v129'1157 uv1157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.606+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.611+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v129'1158 uv1158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.611+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560fd5cef1a0 0x560fd5cf15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.611+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5c2ecd0 con 0x560fd5cef1a0 2024-04-25T11:41:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.612+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560fd5cef1a0 0x560fd5cf15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.614+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560fd5cef1a0 0x560fd5cf15c0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.619+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v129'824 uv803 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.619+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf1b00 con 0x560fd5cef1a0 2024-04-25T11:41:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.624+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v129'825 uv825 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.624+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf1ea0 con 0x560fd5cef1a0 2024-04-25T11:41:40.614 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:41:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.628+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v129'826 uv826 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.628+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2240 con 0x560fd5cef1a0 2024-04-25T11:41:40.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.633+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v129'827 uv827 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.633+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf25e0 con 0x560fd5cef1a0 2024-04-25T11:41:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.638+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v129'828 uv828 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.638+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.643+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v129'829 uv822 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.643+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.648+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v129'830 uv830 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.648+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.653+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v129'831 uv831 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.639 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:41:40.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.653+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.658+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v129'832 uv832 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.659+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.663+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v129'833 uv833 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.663+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.669+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v129'834 uv823 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.669+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.674+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v129'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.674+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.665 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:41:40.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.679+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v129'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.679+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf7070 con 0x560fd5cef1a0 2024-04-25T11:41:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.684+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v129'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.684+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf7070 con 0x560fd5cef1a0 2024-04-25T11:41:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.688+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v129'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.689+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.694+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v129'839 uv819 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.694+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.699+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v129'840 uv840 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.699+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.689 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:41:40.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.703+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v129'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.703+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf3990 con 0x560fd5cef1a0 2024-04-25T11:41:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.708+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v129'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.708+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf3990 con 0x560fd5cef1a0 2024-04-25T11:41:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.713+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v129'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.713+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf2980 con 0x560fd5cef1a0 2024-04-25T11:41:40.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.717+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v129'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cef1a0 2024-04-25T11:41:40.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.717+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x560fd5cf2e90 0x560fd5cf52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.717+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf57f0 con 0x560fd5cf2e90 2024-04-25T11:41:40.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.718+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x560fd5cf2e90 0x560fd5cf52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.720+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x560fd5cf2e90 0x560fd5cf52b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.724+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.725+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf57f0 con 0x560fd5cf2e90 2024-04-25T11:41:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.728+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v129'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.728+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf57f0 con 0x560fd5cf2e90 2024-04-25T11:41:40.719 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:41:40.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.733+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v129'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.733+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf6880 con 0x560fd5cf2e90 2024-04-25T11:41:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.737+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.737+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf6840 con 0x560fd5cf2e90 2024-04-25T11:41:40.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.740+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.741+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf6840 con 0x560fd5cf2e90 2024-04-25T11:41:40.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.745+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.745+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf6820 con 0x560fd5c3f0f0 2024-04-25T11:41:40.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.750+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v129'168 uv167 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.750+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf6820 con 0x560fd5c3f0f0 2024-04-25T11:41:40.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.754+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v129'169 uv169 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.754+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf58c0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.745 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:41:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.758+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v129'170 uv170 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.759+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf58c0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.763+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v129'171 uv171 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.763+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x560fd5cf61e0 0x560fd5cf65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.763+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf8c90 con 0x560fd5cf61e0 2024-04-25T11:41:40.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.764+0000 7ffa9f200640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x560fd5cf61e0 0x560fd5cf65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.766+0000 7ffa9f200640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x560fd5cf61e0 0x560fd5cf65f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.770+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.770+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf9030 con 0x560fd5cf61e0 2024-04-25T11:41:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.774+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v129'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.774+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf9450 con 0x560fd5cf61e0 2024-04-25T11:41:40.764 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:41:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.777+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v129'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.778+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfad00 con 0x560fd5cf61e0 2024-04-25T11:41:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.782+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.782+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfb0a0 con 0x560fd5cf61e0 2024-04-25T11:41:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.787+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.787+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf9cf0 con 0x560fd5cf61e0 2024-04-25T11:41:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.791+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.791+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560fd5cfa5f0 0x560fd5cfd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.791+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfd990 con 0x560fd5cfa5f0 2024-04-25T11:41:40.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.792+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560fd5cfa5f0 0x560fd5cfd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.794+0000 7ffa9e800640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560fd5cfa5f0 0x560fd5cfd450 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.799+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v129'404 uv398 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.799+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cfd990 con 0x560fd5cfa5f0 2024-04-25T11:41:40.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.803+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v129'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.803+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfda10 con 0x560fd5cfa5f0 2024-04-25T11:41:40.794 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:41:40.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.807+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v129'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.807+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfee10 con 0x560fd5cfa5f0 2024-04-25T11:41:40.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.812+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v129'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.812+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfee10 con 0x560fd5cfa5f0 2024-04-25T11:41:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.816+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v129'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.816+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfde00 con 0x560fd5cfa5f0 2024-04-25T11:41:40.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.820+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v129'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.820+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfde00 con 0x560fd5cfa5f0 2024-04-25T11:41:40.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.824+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v129'410 uv403 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.824+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cfde00 con 0x560fd5cfa5f0 2024-04-25T11:41:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.829+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v129'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.829+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfde00 con 0x560fd5cfa5f0 2024-04-25T11:41:40.819 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:41:40.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.833+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v129'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.833+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfde00 con 0x560fd5cfa5f0 2024-04-25T11:41:40.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.838+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v129'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.838+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cfde00 con 0x560fd5cfa5f0 2024-04-25T11:41:40.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.842+0000 7ffa9e800640 1 -- 172.21.15.19:0/2399919490 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v129'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa90028430 con 0x560fd5cfa5f0 2024-04-25T11:41:40.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.842+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x560fd5cfe310 0x560fd5d00730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:40.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.842+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d00e30 con 0x560fd5cfe310 2024-04-25T11:41:40.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.843+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x560fd5cfe310 0x560fd5d00730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:40.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.845+0000 7ffa95e00640 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x560fd5cfe310 0x560fd5d00730 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:40.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.849+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v129'183 uv182 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.850+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d00e30 con 0x560fd5cfe310 2024-04-25T11:41:40.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.854+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v129'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.854+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5cfe310 2024-04-25T11:41:40.844 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:41:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.858+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v129'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.858+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5cfe310 2024-04-25T11:41:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.863+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v129'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.863+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5cfe310 2024-04-25T11:41:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.867+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v129'187 uv187 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.867+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.871+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.872+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.876+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v129'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.876+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.880+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v129'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.867 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:41:40.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.880+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d022b0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.884+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.884+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d022b0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.888+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.888+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5c3f0f0 2024-04-25T11:41:40.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.893+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5c3f0f0 2024-04-25T11:41:40.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.893+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5cf2e90 2024-04-25T11:41:40.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.898+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v129'365 uv359 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.898+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5cf2e90 2024-04-25T11:41:40.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.902+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v129'366 uv366 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.902+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d012a0 con 0x560fd5cf2e90 2024-04-25T11:41:40.894 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:41:40.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.907+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v129'367 uv367 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.907+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf9900 con 0x560fd5cf2e90 2024-04-25T11:41:40.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.911+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v129'368 uv368 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.912+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf9900 con 0x560fd5cf2e90 2024-04-25T11:41:40.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.916+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v129'369 uv369 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.916+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf9900 con 0x560fd5cf2e90 2024-04-25T11:41:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.920+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v129'370 uv364 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.920+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5cf9900 con 0x560fd5cf2e90 2024-04-25T11:41:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.925+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v129'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.925+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf98e0 con 0x560fd5cf2e90 2024-04-25T11:41:40.916 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:41:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.929+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v129'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.930+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5cf98e0 con 0x560fd5cf2e90 2024-04-25T11:41:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.934+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v129'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.934+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d040b0 con 0x560fd5cf2e90 2024-04-25T11:41:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.939+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v129'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cf2e90 2024-04-25T11:41:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.939+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.943+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v129'162 uv160 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.943+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.947+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v129'163 uv163 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.947+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.952+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v129'164 uv164 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.938 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:41:40.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.952+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.956+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v129'165 uv165 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.956+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.960+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v129'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.961+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.965+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v129'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.965+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.955 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:41:40.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.969+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v129'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.969+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.973+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v129'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.974+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d02760 con 0x560fd5cf61e0 2024-04-25T11:41:40.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.977+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v129'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.977+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cf61e0 2024-04-25T11:41:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.981+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v129'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa98059c10 con 0x560fd5cf61e0 2024-04-25T11:41:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.982+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:40.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.986+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v129'186 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.986+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:40.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.991+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v129'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.991+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:40.981 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:41:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.995+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v129'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.995+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.999+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v129'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:40.999+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.003+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v129'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.003+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.008+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v129'455 uv446 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.008+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:40.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.013+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v129'456 uv456 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:40.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.013+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.017+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v129'457 uv457 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:41:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.017+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:41.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.021+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v129'458 uv458 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.021+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:41.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.026+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v129'459 uv459 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.026+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:41.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.030+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v129'460 uv453 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.030+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.034+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v129'461 uv461 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.034+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.038+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v129'462 uv462 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.025 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:41:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.039+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d02760 con 0x560fd5cfe310 2024-04-25T11:41:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.043+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v129'463 uv463 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.043+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d02760 con 0x560fd5cfe310 2024-04-25T11:41:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.047+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v129'464 uv464 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.047+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01750 con 0x560fd5cfe310 2024-04-25T11:41:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.051+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v129'465 uv465 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.051+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x560fd5d01ff0 con 0x560fd5cfe310 2024-04-25T11:41:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffa95e00640 1 -- 172.21.15.19:0/2399919490 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v129'466 uv466 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ffa88051070 con 0x560fd5cfe310 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560fd5cef1a0 msgr2=0x560fd5cf15c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x560fd5cef1a0 0x560fd5cf15c0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560fd5cfa5f0 msgr2=0x560fd5cfd450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x560fd5cfa5f0 0x560fd5cfd450 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x560fd5cfe310 msgr2=0x560fd5d00730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x560fd5cfe310 0x560fd5d00730 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x560fd5cf2e90 msgr2=0x560fd5cf52b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x560fd5cf2e90 0x560fd5cf52b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 reap_dead start 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560fd5c3f0f0 msgr2=0x560fd5b71ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x560fd5c3f0f0 0x560fd5b71ef0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffa9f200640 1 -- 172.21.15.19:0/2399919490 reap_dead start 2024-04-25T11:41:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x560fd5cf61e0 msgr2=0x560fd5cf65f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x560fd5cf61e0 0x560fd5cf65f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa7c072e90 msgr2=0x7ffa7c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa7c072e90 0x7ffa7c0753a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000bad0 tx=0x7ffa90010000 comp rx=0 tx=0).stop 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5c4cdd0 msgr2=0x560fd5cd5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.056+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5c4cdd0 0x560fd5cd5910 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800aa50 tx=0x7ffa9800af10 comp rx=0 tx=0).stop 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.057+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 shutdown_connections 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.057+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x560fd5cf61e0 0x560fd5cf65f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.057+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ffa7c072e90 0x7ffa7c0753a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.057+0000 7ffaa55cdf40 1 --2- 172.21.15.19:0/2399919490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560fd5c4cdd0 0x560fd5cd5910 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.057+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 >> 172.21.15.19:0/2399919490 conn(0x560fd5a95ac0 msgr2=0x560fd5c427f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.057+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 shutdown_connections 2024-04-25T11:41:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.057+0000 7ffaa55cdf40 1 -- 172.21.15.19:0/2399919490 wait complete. 2024-04-25T11:41:41.074 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:41:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.323+0000 7fcee7eedf40 1 Processor -- start 2024-04-25T11:41:41.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.324+0000 7fcee7eedf40 1 -- start start 2024-04-25T11:41:41.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee7eedf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee7eedf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5e6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee7eedf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 0x55d2a5e6d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee7eedf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2a5eb6e70 con 0x55d2a5e6e1e0 2024-04-25T11:41:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee7eedf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d2a5e14d20 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee7eedf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d2a5eb7110 con 0x55d2a5f47dd0 2024-04-25T11:41:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57620/0 (socket says 172.21.15.19:57620) 2024-04-25T11:41:41.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.325+0000 7fcee1000640 1 -- 172.21.15.19:0/708605954 learned_addr learned my addr 172.21.15.19:0/708605954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcee1000640 1 -- 172.21.15.19:0/708605954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 msgr2=0x55d2a5e6d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:41.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcee1000640 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 0x55d2a5e6d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcee1000640 1 -- 172.21.15.19:0/708605954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e5f0 msgr2=0x55d2a5e6cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcee0600640 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5e6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcee1000640 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5e6cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcee1000640 1 -- 172.21.15.19:0/708605954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a5f3e5e0 con 0x55d2a5e6e1e0 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcee1000640 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3e0a0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fced000b270 tx=0x7fced000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6df1b71878bcc14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcedf200640 1 -- 172.21.15.19:0/708605954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0008070 con 0x55d2a5e6e1e0 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcedf200640 1 -- 172.21.15.19:0/708605954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced000d500 con 0x55d2a5e6e1e0 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.326+0000 7fcedf200640 1 -- 172.21.15.19:0/708605954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced000d8b0 con 0x55d2a5e6e1e0 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 -- 172.21.15.19:0/708605954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e1e0 msgr2=0x55d2a5f3e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3e0a0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fced000b270 tx=0x7fced000b730 comp rx=0 tx=0).stop 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 -- 172.21.15.19:0/708605954 shutdown_connections 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5e6cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 0x55d2a5e6d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/708605954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3e0a0 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 -- 172.21.15.19:0/708605954 >> 172.21.15.19:0/708605954 conn(0x55d2a5d90ac0 msgr2=0x55d2a5e9d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 -- 172.21.15.19:0/708605954 shutdown_connections 2024-04-25T11:41:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.327+0000 7fcee7eedf40 1 -- 172.21.15.19:0/708605954 wait complete. 2024-04-25T11:41:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.328+0000 7fcee7eedf40 1 Processor -- start 2024-04-25T11:41:41.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.328+0000 7fcee7eedf40 1 -- start start 2024-04-25T11:41:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee7eedf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee7eedf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5f3b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee7eedf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 0x55d2a5f3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee7eedf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d2a5f44650 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee7eedf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d2a5f44300 con 0x55d2a5e6e1e0 2024-04-25T11:41:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee7eedf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55d2a5f3c700 con 0x55d2a5f47dd0 2024-04-25T11:41:41.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee0600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5f3b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee0600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5f3b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57630/0 (socket says 172.21.15.19:57630) 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee0600640 1 -- 172.21.15.19:0/1439411583 learned_addr learned my addr 172.21.15.19:0/1439411583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.330+0000 7fcee1000640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcee1a00640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 0x55d2a5f3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcee0600640 1 -- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 msgr2=0x55d2a5f3bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcee0600640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55d2a5f47dd0 0x55d2a5f3bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcee0600640 1 -- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e1e0 msgr2=0x55d2a5f3f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcee0600640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d2a5e6e1e0 0x55d2a5f3f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcee0600640 1 -- 172.21.15.19:0/1439411583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a5f3e3f0 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcee0600640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5f3b5f0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007e30 tx=0x7fcecc006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.331+0000 7fcedde00640 1 -- 172.21.15.19:0/1439411583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc017070 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.332+0000 7fcedde00640 1 -- 172.21.15.19:0/1439411583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc010710 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.332+0000 7fcedde00640 1 -- 172.21.15.19:0/1439411583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc006e60 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.332+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a5f3a590 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.332+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a5fd6360 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.334+0000 7fcedde00640 1 -- 172.21.15.19:0/1439411583 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcecc01a070 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.335+0000 7fcedde00640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcec0072ec0 0x7fcec00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.335+0000 7fcedde00640 1 -- 172.21.15.19:0/1439411583 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8891+0+0 (secure 0 0 0) 0x7fcecc093e80 con 0x55d2a5e6e5f0 2024-04-25T11:41:41.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.336+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d2a5f3fef0 0x55d2a5fd5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.336+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55d2a5f3c070 con 0x55d2a5f3fef0 2024-04-25T11:41:41.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.336+0000 7fcee1000640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcec0072ec0 0x7fcec00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.336+0000 7fcee1a00640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d2a5f3fef0 0x55d2a5fd5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.337+0000 7fcee1000640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcec0072ec0 0x7fcec00753d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fced000b270 tx=0x7fced0008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.338+0000 7fcee1a00640 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d2a5f3fef0 0x55d2a5fd5b70 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.341+0000 7fcee1a00640 1 -- 172.21.15.19:0/1439411583 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v129'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fced8057030 con 0x55d2a5f3fef0 2024-04-25T11:41:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d2a5f3fef0 msgr2=0x55d2a5fd5b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55d2a5f3fef0 0x55d2a5fd5b70 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcec0072ec0 msgr2=0x7fcec00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcec0072ec0 0x7fcec00753d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fced000b270 tx=0x7fced0008000 comp rx=0 tx=0).stop 2024-04-25T11:41:41.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e5f0 msgr2=0x55d2a5f3b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 --2- 172.21.15.19:0/1439411583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d2a5e6e5f0 0x55d2a5f3b5f0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007e30 tx=0x7fcecc006750 comp rx=0 tx=0).stop 2024-04-25T11:41:41.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee1a00640 1 -- 172.21.15.19:0/1439411583 reap_dead start 2024-04-25T11:41:41.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 shutdown_connections 2024-04-25T11:41:41.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 >> 172.21.15.19:0/1439411583 conn(0x55d2a5d90ac0 msgr2=0x55d2a5eba080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 shutdown_connections 2024-04-25T11:41:41.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.342+0000 7fcee7eedf40 1 -- 172.21.15.19:0/1439411583 wait complete. 2024-04-25T11:41:41.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:41:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.652+0000 7f98e0e00640 1 Processor -- start 2024-04-25T11:41:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.652+0000 7f98e0e00640 1 -- start start 2024-04-25T11:41:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.654+0000 7f98e0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06dcb0 0x7f98dc15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.654+0000 7f98e0e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98dc06d760 0x7f98dc075e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98e0e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98dc0b7d00 con 0x7f98dc0af050 2024-04-25T11:41:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98e0e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f98dc0b77d0 con 0x7f98dc06d760 2024-04-25T11:41:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98e0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f98dc161260 con 0x7f98dc06dcb0 2024-04-25T11:41:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98dc06d760 0x7f98dc075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98dc06d760 0x7f98dc075e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57638/0 (socket says 172.21.15.19:57638) 2024-04-25T11:41:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9400640 1 -- 172.21.15.19:0/4113437131 learned_addr learned my addr 172.21.15.19:0/4113437131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:41.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9e00640 1 --2- 172.21.15.19:0/4113437131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06dcb0 0x7f98dc15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9400640 1 -- 172.21.15.19:0/4113437131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06dcb0 msgr2=0x7f98dc15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9400640 1 --2- 172.21.15.19:0/4113437131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06dcb0 0x7f98dc15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9400640 1 -- 172.21.15.19:0/4113437131 >> v1:172.21.15.19:6789/0 conn(0x7f98dc0af050 legacy=0x7f98dc0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9400640 1 -- 172.21.15.19:0/4113437131 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc0b7a20 con 0x7f98dc06d760 2024-04-25T11:41:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.655+0000 7f98d9e00640 1 --2- 172.21.15.19:0/4113437131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06dcb0 0x7f98dc15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:41.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.657+0000 7f98d9400640 1 --2- 172.21.15.19:0/4113437131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc06d760 0x7f98dc075e80 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009870 tx=0x7f98c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a092d02d8b140e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.657+0000 7f98d8a00640 1 -- 172.21.15.19:0/4113437131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4037070 con 0x7f98dc06d760 2024-04-25T11:41:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.657+0000 7f98d8a00640 1 -- 172.21.15.19:0/4113437131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c4032440 con 0x7f98dc06d760 2024-04-25T11:41:41.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.657+0000 7f98d8a00640 1 -- 172.21.15.19:0/4113437131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4032810 con 0x7f98dc06d760 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 -- 172.21.15.19:0/4113437131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc06d760 msgr2=0x7f98dc075e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 --2- 172.21.15.19:0/4113437131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc06d760 0x7f98dc075e80 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009870 tx=0x7f98c402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 -- 172.21.15.19:0/4113437131 shutdown_connections 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 --2- 172.21.15.19:0/4113437131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc06d760 0x7f98dc075e80 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 --2- 172.21.15.19:0/4113437131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06dcb0 0x7f98dc15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 -- 172.21.15.19:0/4113437131 >> 172.21.15.19:0/4113437131 conn(0x7f98dc09b7d0 msgr2=0x7f98dc0b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 -- 172.21.15.19:0/4113437131 shutdown_connections 2024-04-25T11:41:41.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.658+0000 7f98e0e00640 1 -- 172.21.15.19:0/4113437131 wait complete. 2024-04-25T11:41:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.660+0000 7f98e0e00640 1 Processor -- start 2024-04-25T11:41:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.660+0000 7f98e0e00640 1 -- start start 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98e0e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06d760 0x7f98dc15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98e0e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98dc06dcb0 0x7f98dc0d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98e0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc0af050 0x7f98dc0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98e0e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc0763c0 con 0x7f98dc0af050 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98e0e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc160ef0 con 0x7f98dc06dcb0 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98e0e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f98dc076540 con 0x7f98dc06d760 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98da800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc0af050 0x7f98dc0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98da800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc0af050 0x7f98dc0d2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57646/0 (socket says 172.21.15.19:57646) 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.662+0000 7f98da800640 1 -- 172.21.15.19:0/3454405584 learned_addr learned my addr 172.21.15.19:0/3454405584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98d9e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06d760 0x7f98dc15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98d9400640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98dc06dcb0 0x7f98dc0d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98da800640 1 -- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06d760 msgr2=0x7f98dc15bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98da800640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06d760 0x7f98dc15bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98da800640 1 -- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98dc06dcb0 msgr2=0x7f98dc0d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98da800640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98dc06dcb0 0x7f98dc0d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98da800640 1 -- 172.21.15.19:0/3454405584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc1f7e20 con 0x7f98dc0af050 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.663+0000 7f98da800640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc0af050 0x7f98dc0d2800 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c6b0 tx=0x7f98d400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.664+0000 7f98d2000640 1 -- 172.21.15.19:0/3454405584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4003b70 con 0x7f98dc0af050 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.664+0000 7f98d2000640 1 -- 172.21.15.19:0/3454405584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d4003d30 con 0x7f98dc0af050 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.664+0000 7f98d2000640 1 -- 172.21.15.19:0/3454405584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4005310 con 0x7f98dc0af050 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.664+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc1f8100 con 0x7f98dc0af050 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.664+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc14fd70 con 0x7f98dc0af050 2024-04-25T11:41:41.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.665+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98dc16b150 con 0x7f98dc0af050 2024-04-25T11:41:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.666+0000 7f98d2000640 1 -- 172.21.15.19:0/3454405584 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98d40054b0 con 0x7f98dc0af050 2024-04-25T11:41:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.668+0000 7f98d2000640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98b80744e0 0x7f98b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.668+0000 7f98d2000640 1 -- 172.21.15.19:0/3454405584 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8891+0+0 (secure 0 0 0) 0x7f98d4095cd0 con 0x7f98dc0af050 2024-04-25T11:41:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.669+0000 7f98d9e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98b80744e0 0x7f98b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.670+0000 7f98d9e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98b80744e0 0x7f98b80769f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0c5940 tx=0x7f98cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.673+0000 7f98d2000640 1 -- 172.21.15.19:0/3454405584 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98d405f830 con 0x7f98dc0af050 2024-04-25T11:41:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:41.919+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f98dc050740 con 0x7f98dc0af050 2024-04-25T11:41:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.577+0000 7f98d2000640 1 -- 172.21.15.19:0/3454405584 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v335) ==== 162+0+0 (secure 0 0 0) 0x7f98d405f130 con 0x7f98dc0af050 2024-04-25T11:41:42.564 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:41:42.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98b80744e0 msgr2=0x7f98b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98b80744e0 0x7f98b80769f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0c5940 tx=0x7f98cc008040 comp rx=0 tx=0).stop 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc0af050 msgr2=0x7f98dc0d2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc0af050 0x7f98dc0d2800 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c6b0 tx=0x7f98d400cb70 comp rx=0 tx=0).stop 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 shutdown_connections 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f98b80744e0 0x7f98b80769f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98dc06dcb0 0x7f98dc0d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98dc0af050 0x7f98dc0d2800 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 --2- 172.21.15.19:0/3454405584 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98dc06d760 0x7f98dc15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 >> 172.21.15.19:0/3454405584 conn(0x7f98dc09b7d0 msgr2=0x7f98dc0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 shutdown_connections 2024-04-25T11:41:42.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.581+0000 7f98e0e00640 1 -- 172.21.15.19:0/3454405584 wait complete. 2024-04-25T11:41:42.605 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:42.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.886+0000 7f9a91a00640 1 Processor -- start 2024-04-25T11:41:42.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.886+0000 7f9a91a00640 1 -- start start 2024-04-25T11:41:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.888+0000 7f9a91a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9a8c06c5c0 0x7f9a8c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.888+0000 7f9a91a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8c0d1230 con 0x7f9a8c0b4ea0 2024-04-25T11:41:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.888+0000 7f9a91a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8c0d06b0 con 0x7f9a8c144370 2024-04-25T11:41:42.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.888+0000 7f9a91a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9a8c0d0990 con 0x7f9a8c06c5c0 2024-04-25T11:41:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.888+0000 7f9a91000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9a8c0b4ea0 0x7f9a8c15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49346/0 (socket says 172.21.15.19:49346) 2024-04-25T11:41:42.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a91000640 1 -- 172.21.15.19:0/3608665002 learned_addr learned my addr 172.21.15.19:0/3608665002 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8be00640 1 --2- 172.21.15.19:0/3608665002 >> v2:172.21.15.169:3300/0 conn(0x7f9a8c06c5c0 0x7f9a8c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2305916992 0 0) 0x7f9a8c0d1230 con 0x7f9a8c0b4ea0 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a70003520 con 0x7f9a8c0b4ea0 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1706527562 0 0) 0x7f9a70003520 con 0x7f9a8c0b4ea0 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 >> v2:172.21.15.169:3300/0 conn(0x7f9a8c06c5c0 msgr2=0x7f9a8c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8aa00640 1 --2- 172.21.15.19:0/3608665002 >> v2:172.21.15.169:3300/0 conn(0x7f9a8c06c5c0 0x7f9a8c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 >> v1:172.21.15.169:6789/0 conn(0x7f9a8c144370 legacy=0x7f9a8c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.889+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c0d0c70 con 0x7f9a8c0b4ea0 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9a840019f0 con 0x7f9a8c0b4ea0 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 >> v1:172.21.15.19:6789/0 conn(0x7f9a8c0b4ea0 legacy=0x7f9a8c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a70003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a70004800 0x7f9a70004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a70005270 0x7f9a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c0d0990 con 0x7f9a70003a00 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7000c190 con 0x7f9a70004800 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7000c390 con 0x7f9a70005270 2024-04-25T11:41:42.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8be00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a70003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8be00640 1 -- 172.21.15.19:0/3608665002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a70005270 msgr2=0x7f9a7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8be00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a70005270 0x7f9a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8be00640 1 -- 172.21.15.19:0/3608665002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a70004800 msgr2=0x7f9a70004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.890+0000 7f9a8be00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a70004800 0x7f9a70004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.891+0000 7f9a8be00640 1 -- 172.21.15.19:0/3608665002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a70003520 con 0x7f9a70003a00 2024-04-25T11:41:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.891+0000 7f9a91000640 1 -- 172.21.15.19:0/3608665002 reap_dead start 2024-04-25T11:41:42.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.892+0000 7f9a8be00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a70003e70 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009ac0 tx=0x7f9a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba41035d1b9a5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.892+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c036070 con 0x7f9a70003a00 2024-04-25T11:41:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.892+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c031400 con 0x7f9a70003a00 2024-04-25T11:41:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.892+0000 7f9a8aa00640 1 -- 172.21.15.19:0/3608665002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c009690 con 0x7f9a70003a00 2024-04-25T11:41:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.893+0000 7f9a91a00640 1 -- 172.21.15.19:0/3608665002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 msgr2=0x7f9a70003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.893+0000 7f9a91a00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a70003e70 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009ac0 tx=0x7f9a70005830 comp rx=0 tx=0).stop 2024-04-25T11:41:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.893+0000 7f9a91a00640 1 -- 172.21.15.19:0/3608665002 shutdown_connections 2024-04-25T11:41:42.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.893+0000 7f9a91a00640 1 --2- 172.21.15.19:0/3608665002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a70003e70 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.893+0000 7f9a91a00640 1 -- 172.21.15.19:0/3608665002 >> 172.21.15.19:0/3608665002 conn(0x7f9a8c0d1720 msgr2=0x7f9a8c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.893+0000 7f9a91a00640 1 -- 172.21.15.19:0/3608665002 shutdown_connections 2024-04-25T11:41:42.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.893+0000 7f9a91a00640 1 -- 172.21.15.19:0/3608665002 wait complete. 2024-04-25T11:41:42.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.895+0000 7f9a91a00640 1 Processor -- start 2024-04-25T11:41:42.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.895+0000 7f9a91a00640 1 -- start start 2024-04-25T11:41:42.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a91a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a8c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a91a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a8c160a50 0x7f9a8c146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a91a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a8c1430b0 0x7f9a8c15e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a91a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c0d0450 con 0x7f9a70003a00 2024-04-25T11:41:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a91a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c0d02d0 con 0x7f9a8c160a50 2024-04-25T11:41:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a91a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8c147760 con 0x7f9a8c1430b0 2024-04-25T11:41:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a8be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a8c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a8be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a8c1616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57668/0 (socket says 172.21.15.19:57668) 2024-04-25T11:41:42.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.897+0000 7f9a8be00640 1 -- 172.21.15.19:0/4191191688 learned_addr learned my addr 172.21.15.19:0/4191191688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8b400640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a8c160a50 0x7f9a8c146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8be00640 1 -- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a8c1430b0 msgr2=0x7f9a8c15e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a91000640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a8c1430b0 0x7f9a8c15e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8be00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a8c1430b0 0x7f9a8c15e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8be00640 1 -- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a8c160a50 msgr2=0x7f9a8c146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8be00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a8c160a50 0x7f9a8c146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8be00640 1 -- 172.21.15.19:0/4191191688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c146c60 con 0x7f9a70003a00 2024-04-25T11:41:42.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8b400640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a8c160a50 0x7f9a8c146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a91000640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a8c1430b0 0x7f9a8c15e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a8be00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a8c1616b0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009be0 tx=0x7f9a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.898+0000 7f9a83e00640 1 -- 172.21.15.19:0/4191191688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c03f020 con 0x7f9a70003a00 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.899+0000 7f9a83e00640 1 -- 172.21.15.19:0/4191191688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c037040 con 0x7f9a70003a00 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.899+0000 7f9a83e00640 1 -- 172.21.15.19:0/4191191688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c036070 con 0x7f9a70003a00 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.899+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c15f9b0 con 0x7f9a70003a00 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.899+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c15ff70 con 0x7f9a70003a00 2024-04-25T11:41:42.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.900+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c050740 con 0x7f9a70003a00 2024-04-25T11:41:42.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.901+0000 7f9a83e00640 1 -- 172.21.15.19:0/4191191688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9a7c042020 con 0x7f9a70003a00 2024-04-25T11:41:42.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.902+0000 7f9a83e00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a68074450 0x7f9a68076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.902+0000 7f9a83e00640 1 -- 172.21.15.19:0/4191191688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8891+0+0 (secure 0 0 0) 0x7f9a7c0b60a0 con 0x7f9a70003a00 2024-04-25T11:41:42.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.903+0000 7f9a8b400640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a68074450 0x7f9a68076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.904+0000 7f9a8b400640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a68074450 0x7f9a68076960 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c15f550 tx=0x7f9a74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:42.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:42.907+0000 7f9a83e00640 1 -- 172.21.15.19:0/4191191688 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a7c049030 con 0x7f9a70003a00 2024-04-25T11:41:43.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.154+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a8c0cfe80 con 0x7f9a70003a00 2024-04-25T11:41:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.155+0000 7f9a83e00640 1 -- 172.21.15.19:0/4191191688 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 336 v336) ==== 95+0+5190 (secure 0 0 0) 0x7f9a7c07fbd0 con 0x7f9a70003a00 2024-04-25T11:41:43.142 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:43.142 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":336,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":336,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:42.581805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11013":{"gid":11013,"name":"a","rank":0,"incarnation":336,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/1787178525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1787178525},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787178525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:43.142 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 336 2024-04-25T11:41:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a68074450 msgr2=0x7f9a68076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a68074450 0x7f9a68076960 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c15f550 tx=0x7f9a74009040 comp rx=0 tx=0).stop 2024-04-25T11:41:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 msgr2=0x7f9a8c1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a8c1616b0 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009be0 tx=0x7f9a7c004730 comp rx=0 tx=0).stop 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 shutdown_connections 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9a68074450 0x7f9a68076960 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9a8c160a50 0x7f9a8c146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9a8c1430b0 0x7f9a8c15e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 --2- 172.21.15.19:0/4191191688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70003a00 0x7f9a8c1616b0 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 >> 172.21.15.19:0/4191191688 conn(0x7f9a8c0d1720 msgr2=0x7f9a8c06f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 shutdown_connections 2024-04-25T11:41:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:43.159+0000 7f9a91a00640 1 -- 172.21.15.19:0/4191191688 wait complete. 2024-04-25T11:41:43.172 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:41:45.624 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:45.638+0000 7f8346600640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:41:45.627 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:45.641+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:41:45.627 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:45.641+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:41:45.648 INFO:tasks.ceph.mds.a.smithi019.stderr: -22> 2024-04-25T11:41:45.638+0000 7f8346600640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:41:45.784 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:41:48.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:48.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.481+0000 7efd92e00640 1 Processor -- start 2024-04-25T11:41:48.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.482+0000 7efd92e00640 1 -- start start 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.484+0000 7efd92e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd8c06c5c0 0x7efd8c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.484+0000 7efd92e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd8c144370 0x7efd8c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.484+0000 7efd92e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8c0b50e0 con 0x7efd8c0b4cd0 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.484+0000 7efd92e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efd8c0d1230 con 0x7efd8c144370 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.484+0000 7efd92e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7efd8c0d06b0 con 0x7efd8c06c5c0 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.484+0000 7efd91000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd8c144370 0x7efd8c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.485+0000 7efd91000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efd8c144370 0x7efd8c0b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47408/0 (socket says 172.21.15.19:47408) 2024-04-25T11:41:48.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.485+0000 7efd91000640 1 -- 172.21.15.19:0/75733880 learned_addr learned my addr 172.21.15.19:0/75733880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:48.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.485+0000 7efd91a00640 1 --2- 172.21.15.19:0/75733880 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06c5c0 0x7efd8c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.485+0000 7efd91000640 1 -- 172.21.15.19:0/75733880 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06c5c0 msgr2=0x7efd8c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:48.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.485+0000 7efd91000640 1 --2- 172.21.15.19:0/75733880 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06c5c0 0x7efd8c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.485+0000 7efd91000640 1 -- 172.21.15.19:0/75733880 >> v1:172.21.15.169:6789/0 conn(0x7efd8c0b4cd0 legacy=0x7efd8c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:48.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.485+0000 7efd91000640 1 -- 172.21.15.19:0/75733880 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c0d0990 con 0x7efd8c144370 2024-04-25T11:41:48.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.486+0000 7efd91000640 1 --2- 172.21.15.19:0/75733880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c144370 0x7efd8c0b4790 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0726b0 tx=0x7efd7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc84f53bc80617d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:48.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.486+0000 7efd8be00640 1 -- 172.21.15.19:0/75733880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78037070 con 0x7efd8c144370 2024-04-25T11:41:48.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd8be00640 1 -- 172.21.15.19:0/75733880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd78032440 con 0x7efd8c144370 2024-04-25T11:41:48.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd8be00640 1 -- 172.21.15.19:0/75733880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78032810 con 0x7efd8c144370 2024-04-25T11:41:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd92e00640 1 -- 172.21.15.19:0/75733880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c144370 msgr2=0x7efd8c0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd92e00640 1 --2- 172.21.15.19:0/75733880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c144370 0x7efd8c0b4790 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0726b0 tx=0x7efd7802a230 comp rx=0 tx=0).stop 2024-04-25T11:41:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd92e00640 1 -- 172.21.15.19:0/75733880 shutdown_connections 2024-04-25T11:41:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd92e00640 1 --2- 172.21.15.19:0/75733880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c144370 0x7efd8c0b4790 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd92e00640 1 --2- 172.21.15.19:0/75733880 >> v2:172.21.15.169:3300/0 conn(0x7efd8c06c5c0 0x7efd8c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.487+0000 7efd92e00640 1 -- 172.21.15.19:0/75733880 >> 172.21.15.19:0/75733880 conn(0x7efd8c0d1720 msgr2=0x7efd8c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:48.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.488+0000 7efd92e00640 1 -- 172.21.15.19:0/75733880 shutdown_connections 2024-04-25T11:41:48.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.488+0000 7efd92e00640 1 -- 172.21.15.19:0/75733880 wait complete. 2024-04-25T11:41:48.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.490+0000 7efd92e00640 1 Processor -- start 2024-04-25T11:41:48.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.490+0000 7efd92e00640 1 -- start start 2024-04-25T11:41:48.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd92e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c06c5c0 0x7efd8c139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd92e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0b4cd0 0x7efd8c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd92e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c144370 0x7efd8c1f9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd92e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0d02d0 con 0x7efd8c06c5c0 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd92e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0d0f50 con 0x7efd8c0b4cd0 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd92e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c0d0450 con 0x7efd8c144370 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd91a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c06c5c0 0x7efd8c139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd91a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c06c5c0 0x7efd8c139030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47420/0 (socket says 172.21.15.19:47420) 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd91a00640 1 -- 172.21.15.19:0/2191406936 learned_addr learned my addr 172.21.15.19:0/2191406936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd91000640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0b4cd0 0x7efd8c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd91a00640 1 -- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c144370 msgr2=0x7efd8c1f9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.492+0000 7efd92400640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c144370 0x7efd8c1f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd91a00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c144370 0x7efd8c1f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd91a00640 1 -- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0b4cd0 msgr2=0x7efd8c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd91a00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0b4cd0 0x7efd8c06d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd91a00640 1 -- 172.21.15.19:0/2191406936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c139570 con 0x7efd8c06c5c0 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd91000640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0b4cd0 0x7efd8c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd91a00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c06c5c0 0x7efd8c139030 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c820 tx=0x7efd8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd89600640 1 -- 172.21.15.19:0/2191406936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80015070 con 0x7efd8c06c5c0 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd89600640 1 -- 172.21.15.19:0/2191406936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd80004020 con 0x7efd8c06c5c0 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd89600640 1 -- 172.21.15.19:0/2191406936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800043f0 con 0x7efd8c06c5c0 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1621f0 con 0x7efd8c06c5c0 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.493+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c06f2b0 con 0x7efd8c06c5c0 2024-04-25T11:41:48.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.494+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c050740 con 0x7efd8c06c5c0 2024-04-25T11:41:48.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.496+0000 7efd89600640 1 -- 172.21.15.19:0/2191406936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd800077d0 con 0x7efd8c06c5c0 2024-04-25T11:41:48.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.497+0000 7efd89600640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd6c0744b0 0x7efd6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.497+0000 7efd89600640 1 -- 172.21.15.19:0/2191406936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7efd80095c80 con 0x7efd8c06c5c0 2024-04-25T11:41:48.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.497+0000 7efd91000640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd6c0744b0 0x7efd6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.499+0000 7efd91000640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd6c0744b0 0x7efd6c0769c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7efd78034000 tx=0x7efd7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:48.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.502+0000 7efd89600640 1 -- 172.21.15.19:0/2191406936 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd8005f780 con 0x7efd8c06c5c0 2024-04-25T11:41:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.747+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd8c141540 con 0x7efd8c06c5c0 2024-04-25T11:41:48.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.748+0000 7efd89600640 1 -- 172.21.15.19:0/2191406936 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) ==== 95+0+5176 (secure 0 0 0) 0x7efd8005f080 con 0x7efd8c06c5c0 2024-04-25T11:41:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:48.735 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":341,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11055,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/608506291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":608506291},{"type":"v1","addr":"172.21.15.19:6835","nonce":608506291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":341}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:45.641807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:48.735 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 341 2024-04-25T11:41:48.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.751+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd6c0744b0 msgr2=0x7efd6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.751+0000 7efd92e00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd6c0744b0 0x7efd6c0769c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7efd78034000 tx=0x7efd7802e000 comp rx=0 tx=0).stop 2024-04-25T11:41:48.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.751+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c06c5c0 msgr2=0x7efd8c139030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.751+0000 7efd92e00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c06c5c0 0x7efd8c139030 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c820 tx=0x7efd8000cce0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 shutdown_connections 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efd6c0744b0 0x7efd6c0769c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd8c0b4cd0 0x7efd8c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd8c144370 0x7efd8c1f9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 --2- 172.21.15.19:0/2191406936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd8c06c5c0 0x7efd8c139030 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 >> 172.21.15.19:0/2191406936 conn(0x7efd8c0d1720 msgr2=0x7efd8c146790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 shutdown_connections 2024-04-25T11:41:48.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:48.752+0000 7efd92e00640 1 -- 172.21.15.19:0/2191406936 wait complete. 2024-04-25T11:41:48.766 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:41:48.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:49.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.058+0000 7fca64600640 1 Processor -- start 2024-04-25T11:41:49.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.058+0000 7fca64600640 1 -- start start 2024-04-25T11:41:49.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.060+0000 7fca64600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca5c1443c0 0x7fca5c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.060+0000 7fca64600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fca5c070370 0x7fca5c0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca64600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c0cfb20 0x7fca5c0ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca64600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fca5c0708d0 con 0x7fca5c1443c0 2024-04-25T11:41:49.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca64600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fca5c15f160 con 0x7fca5c070370 2024-04-25T11:41:49.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca64600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fca5c0cff30 con 0x7fca5c0cfb20 2024-04-25T11:41:49.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca61c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca5c1443c0 0x7fca5c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca61c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca5c1443c0 0x7fca5c06fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47434/0 (socket says 172.21.15.19:47434) 2024-04-25T11:41:49.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca61c00640 1 -- 172.21.15.19:0/3371953127 learned_addr learned my addr 172.21.15.19:0/3371953127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:49.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca62600640 1 --2- 172.21.15.19:0/3371953127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c0cfb20 0x7fca5c0ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.061+0000 7fca61200640 1 --2- 172.21.15.19:0/3371953127 >> v2:172.21.15.169:3300/0 conn(0x7fca5c070370 0x7fca5c0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.062+0000 7fca61c00640 1 -- 172.21.15.19:0/3371953127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c0cfb20 msgr2=0x7fca5c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:49.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.062+0000 7fca61c00640 1 --2- 172.21.15.19:0/3371953127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c0cfb20 0x7fca5c0ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.062+0000 7fca61c00640 1 -- 172.21.15.19:0/3371953127 >> v2:172.21.15.169:3300/0 conn(0x7fca5c070370 msgr2=0x7fca5c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:49.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.062+0000 7fca61c00640 1 --2- 172.21.15.19:0/3371953127 >> v2:172.21.15.169:3300/0 conn(0x7fca5c070370 0x7fca5c0cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.062+0000 7fca61c00640 1 -- 172.21.15.19:0/3371953127 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c15f5f0 con 0x7fca5c1443c0 2024-04-25T11:41:49.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.063+0000 7fca61c00640 1 --2- 172.21.15.19:0/3371953127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c1443c0 0x7fca5c06fe30 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fca5c06f920 tx=0x7fca5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170c82cb908c4a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.063+0000 7fca5be00640 1 -- 172.21.15.19:0/3371953127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50037070 con 0x7fca5c1443c0 2024-04-25T11:41:49.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.063+0000 7fca5be00640 1 -- 172.21.15.19:0/3371953127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca50032440 con 0x7fca5c1443c0 2024-04-25T11:41:49.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.063+0000 7fca5be00640 1 -- 172.21.15.19:0/3371953127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50032810 con 0x7fca5c1443c0 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 -- 172.21.15.19:0/3371953127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c1443c0 msgr2=0x7fca5c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 --2- 172.21.15.19:0/3371953127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c1443c0 0x7fca5c06fe30 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fca5c06f920 tx=0x7fca5002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 -- 172.21.15.19:0/3371953127 shutdown_connections 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 --2- 172.21.15.19:0/3371953127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c0cfb20 0x7fca5c0ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 --2- 172.21.15.19:0/3371953127 >> v2:172.21.15.169:3300/0 conn(0x7fca5c070370 0x7fca5c0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 --2- 172.21.15.19:0/3371953127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c1443c0 0x7fca5c06fe30 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 -- 172.21.15.19:0/3371953127 >> 172.21.15.19:0/3371953127 conn(0x7fca5c0d1ae0 msgr2=0x7fca5c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.064+0000 7fca64600640 1 -- 172.21.15.19:0/3371953127 shutdown_connections 2024-04-25T11:41:49.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.065+0000 7fca64600640 1 -- 172.21.15.19:0/3371953127 wait complete. 2024-04-25T11:41:49.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.067+0000 7fca64600640 1 Processor -- start 2024-04-25T11:41:49.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.067+0000 7fca64600640 1 -- start start 2024-04-25T11:41:49.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.069+0000 7fca64600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c070370 0x7fca5c140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.069+0000 7fca64600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c0cfb20 0x7fca5c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.069+0000 7fca64600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5c1443c0 0x7fca5c0b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.069+0000 7fca64600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c15edf0 con 0x7fca5c0cfb20 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.069+0000 7fca64600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c15eb10 con 0x7fca5c1443c0 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.069+0000 7fca64600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fca5c1447d0 con 0x7fca5c070370 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c0cfb20 0x7fca5c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c0cfb20 0x7fca5c1587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47448/0 (socket says 172.21.15.19:47448) 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 -- 172.21.15.19:0/4002264501 learned_addr learned my addr 172.21.15.19:0/4002264501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca62600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5c1443c0 0x7fca5c0b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61c00640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c070370 0x7fca5c140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 -- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c070370 msgr2=0x7fca5c140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c070370 0x7fca5c140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 -- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5c1443c0 msgr2=0x7fca5c0b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5c1443c0 0x7fca5c0b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61200640 1 -- 172.21.15.19:0/4002264501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c1430a0 con 0x7fca5c0cfb20 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.070+0000 7fca61c00640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c070370 0x7fca5c140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.071+0000 7fca61200640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c0cfb20 0x7fca5c1587f0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fca4800d5c0 tx=0x7fca4800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.071+0000 7fca59600640 1 -- 172.21.15.19:0/4002264501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48004240 con 0x7fca5c0cfb20 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.071+0000 7fca59600640 1 -- 172.21.15.19:0/4002264501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca48019070 con 0x7fca5c0cfb20 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.071+0000 7fca59600640 1 -- 172.21.15.19:0/4002264501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48003610 con 0x7fca5c0cfb20 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.071+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c15ab90 con 0x7fca5c0cfb20 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.071+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c15b150 con 0x7fca5c0cfb20 2024-04-25T11:41:49.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.072+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c050740 con 0x7fca5c0cfb20 2024-04-25T11:41:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.075+0000 7fca59600640 1 -- 172.21.15.19:0/4002264501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fca480037b0 con 0x7fca5c0cfb20 2024-04-25T11:41:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.077+0000 7fca59600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca3c074500 0x7fca3c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.077+0000 7fca59600640 1 -- 172.21.15.19:0/4002264501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7fca48096220 con 0x7fca5c0cfb20 2024-04-25T11:41:49.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.078+0000 7fca61c00640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca3c074500 0x7fca3c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.079+0000 7fca61c00640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca3c074500 0x7fca3c076a10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fca50034000 tx=0x7fca5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.082+0000 7fca59600640 1 -- 172.21.15.19:0/4002264501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca4805fcb0 con 0x7fca5c0cfb20 2024-04-25T11:41:49.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.329+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca5c06f950 con 0x7fca5c0cfb20 2024-04-25T11:41:49.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.330+0000 7fca59600640 1 -- 172.21.15.19:0/4002264501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) ==== 95+0+5176 (secure 0 0 0) 0x7fca4805f5b0 con 0x7fca5c0cfb20 2024-04-25T11:41:49.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:49.317 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":341,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11055,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/608506291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":608506291},{"type":"v1","addr":"172.21.15.19:6835","nonce":608506291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":341}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:45.641807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:49.317 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 341 2024-04-25T11:41:49.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca3c074500 msgr2=0x7fca3c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca3c074500 0x7fca3c076a10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fca50034000 tx=0x7fca5002e000 comp rx=0 tx=0).stop 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c0cfb20 msgr2=0x7fca5c1587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c0cfb20 0x7fca5c1587f0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fca4800d5c0 tx=0x7fca4800da80 comp rx=0 tx=0).stop 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 shutdown_connections 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fca3c074500 0x7fca3c076a10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca5c1443c0 0x7fca5c0b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca5c0cfb20 0x7fca5c1587f0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 --2- 172.21.15.19:0/4002264501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fca5c070370 0x7fca5c140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 >> 172.21.15.19:0/4002264501 conn(0x7fca5c0d1ae0 msgr2=0x7fca5c06d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 shutdown_connections 2024-04-25T11:41:49.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.333+0000 7fca64600640 1 -- 172.21.15.19:0/4002264501 wait complete. 2024-04-25T11:41:49.347 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:41:49.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:49.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.642+0000 7fd24f200640 1 Processor -- start 2024-04-25T11:41:49.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.642+0000 7fd24f200640 1 -- start start 2024-04-25T11:41:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.645+0000 7fd24f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd2500b4ea0 0x7fd2500cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.645+0000 7fd24f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd250144370 0x7fd25015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.645+0000 7fd24f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2500d1230 con 0x7fd25006c5c0 2024-04-25T11:41:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.645+0000 7fd24f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500d06b0 con 0x7fd250144370 2024-04-25T11:41:49.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.645+0000 7fd24f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd2500d0990 con 0x7fd2500b4ea0 2024-04-25T11:41:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd250144370 0x7fd25015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd25006c5c0 0x7fd2500b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50382/0 (socket says 172.21.15.19:50382) 2024-04-25T11:41:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24de00640 1 -- 172.21.15.19:0/967616131 learned_addr learned my addr 172.21.15.19:0/967616131 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:49.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24d400640 1 --2- 172.21.15.19:0/967616131 >> v2:172.21.15.169:3300/0 conn(0x7fd2500b4ea0 0x7fd2500cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24e800640 1 -- 172.21.15.19:0/967616131 >> v2:172.21.15.169:3300/0 conn(0x7fd2500b4ea0 msgr2=0x7fd2500cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24e800640 1 --2- 172.21.15.19:0/967616131 >> v2:172.21.15.169:3300/0 conn(0x7fd2500b4ea0 0x7fd2500cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24e800640 1 -- 172.21.15.19:0/967616131 >> v1:172.21.15.19:6789/0 conn(0x7fd25006c5c0 legacy=0x7fd2500b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.646+0000 7fd24e800640 1 -- 172.21.15.19:0/967616131 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2500d0c70 con 0x7fd250144370 2024-04-25T11:41:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.648+0000 7fd24e800640 1 --2- 172.21.15.19:0/967616131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd25015e770 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fd2500b5550 tx=0x7fd24802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863173dd98982bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.648+0000 7fd24ca00640 1 -- 172.21.15.19:0/967616131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248007740 con 0x7fd250144370 2024-04-25T11:41:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.648+0000 7fd24ca00640 1 -- 172.21.15.19:0/967616131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2480078e0 con 0x7fd250144370 2024-04-25T11:41:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.648+0000 7fd24ca00640 1 -- 172.21.15.19:0/967616131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248007cb0 con 0x7fd250144370 2024-04-25T11:41:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 -- 172.21.15.19:0/967616131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 msgr2=0x7fd25015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 --2- 172.21.15.19:0/967616131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd25015e770 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fd2500b5550 tx=0x7fd24802a680 comp rx=0 tx=0).stop 2024-04-25T11:41:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 -- 172.21.15.19:0/967616131 shutdown_connections 2024-04-25T11:41:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 --2- 172.21.15.19:0/967616131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd25015e770 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 --2- 172.21.15.19:0/967616131 >> v2:172.21.15.169:3300/0 conn(0x7fd2500b4ea0 0x7fd2500cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 -- 172.21.15.19:0/967616131 >> 172.21.15.19:0/967616131 conn(0x7fd2500d1720 msgr2=0x7fd2500d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 -- 172.21.15.19:0/967616131 shutdown_connections 2024-04-25T11:41:49.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.649+0000 7fd24f200640 1 -- 172.21.15.19:0/967616131 wait complete. 2024-04-25T11:41:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.651+0000 7fd24f200640 1 Processor -- start 2024-04-25T11:41:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.651+0000 7fd24f200640 1 -- start start 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd25006c5c0 0x7fd2501f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd250162430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2500cb2e0 0x7fd2500cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500d0450 con 0x7fd250144370 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500d02d0 con 0x7fd2500cb2e0 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd250147760 con 0x7fd25006c5c0 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd250162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd250162430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47478/0 (socket says 172.21.15.19:47478) 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.653+0000 7fd24d400640 1 -- 172.21.15.19:0/3966170303 learned_addr learned my addr 172.21.15.19:0/3966170303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24e800640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2500cb2e0 0x7fd2500cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24de00640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd25006c5c0 0x7fd2501f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24d400640 1 -- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd25006c5c0 msgr2=0x7fd2501f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24d400640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd25006c5c0 0x7fd2501f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24d400640 1 -- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2500cb2e0 msgr2=0x7fd2500cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24d400640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2500cb2e0 0x7fd2500cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24d400640 1 -- 172.21.15.19:0/3966170303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2501f9b60 con 0x7fd250144370 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24de00640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd25006c5c0 0x7fd2501f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.654+0000 7fd24d400640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd250162430 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e870 tx=0x7fd23c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.655+0000 7fd246000640 1 -- 172.21.15.19:0/3966170303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c017070 con 0x7fd250144370 2024-04-25T11:41:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.655+0000 7fd246000640 1 -- 172.21.15.19:0/3966170303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c00c940 con 0x7fd250144370 2024-04-25T11:41:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.655+0000 7fd246000640 1 -- 172.21.15.19:0/3966170303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00cd30 con 0x7fd250144370 2024-04-25T11:41:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.655+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2501f9e40 con 0x7fd250144370 2024-04-25T11:41:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.655+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2500cbca0 con 0x7fd250144370 2024-04-25T11:41:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.656+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2500cba80 con 0x7fd250144370 2024-04-25T11:41:49.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.657+0000 7fd246000640 1 -- 172.21.15.19:0/3966170303 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd23c01a070 con 0x7fd250144370 2024-04-25T11:41:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.658+0000 7fd246000640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd230074500 0x7fd230076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.658+0000 7fd246000640 1 -- 172.21.15.19:0/3966170303 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7fd23c096190 con 0x7fd250144370 2024-04-25T11:41:49.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.659+0000 7fd24de00640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd230074500 0x7fd230076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.660+0000 7fd24de00640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd230074500 0x7fd230076a10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd240000f60 tx=0x7fd240009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.663+0000 7fd246000640 1 -- 172.21.15.19:0/3966170303 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd23c05fcb0 con 0x7fd250144370 2024-04-25T11:41:49.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.899+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd25015cc30 con 0x7fd250144370 2024-04-25T11:41:49.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.901+0000 7fd246000640 1 -- 172.21.15.19:0/3966170303 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) ==== 95+0+5176 (secure 0 0 0) 0x7fd23c05f5b0 con 0x7fd250144370 2024-04-25T11:41:49.888 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:49.888 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":341,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11055,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/608506291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":608506291},{"type":"v1","addr":"172.21.15.19:6835","nonce":608506291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":341}],"filesystems":[{"mdsmap":{"epoch":340,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:45.641807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:49.888 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 341 2024-04-25T11:41:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd230074500 msgr2=0x7fd230076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd230074500 0x7fd230076a10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd240000f60 tx=0x7fd240009040 comp rx=0 tx=0).stop 2024-04-25T11:41:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 msgr2=0x7fd250162430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd250162430 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e870 tx=0x7fd23c00ed30 comp rx=0 tx=0).stop 2024-04-25T11:41:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 shutdown_connections 2024-04-25T11:41:49.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd230074500 0x7fd230076a10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2500cb2e0 0x7fd2500cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.903+0000 7fd24f200640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd250144370 0x7fd250162430 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.904+0000 7fd24f200640 1 --2- 172.21.15.19:0/3966170303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd25006c5c0 0x7fd2501f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.904+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 >> 172.21.15.19:0/3966170303 conn(0x7fd2500d1720 msgr2=0x7fd2500d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:49.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.904+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 shutdown_connections 2024-04-25T11:41:49.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:49.904+0000 7fd24f200640 1 -- 172.21.15.19:0/3966170303 wait complete. 2024-04-25T11:41:49.914 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2024-04-25T11:41:49.914 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2024-04-25T11:41:49.914 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:49.914 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:49.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:41:49.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:41:49.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:49.965 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:41:50.059 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:41:50.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:50.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:41:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.365+0000 7f86bb400640 1 Processor -- start 2024-04-25T11:41:50.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.365+0000 7f86bb400640 1 -- start start 2024-04-25T11:41:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.368+0000 7f86bb400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86b4167000 0x7f86b4169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.368+0000 7f86bb400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b415f5f0 con 0x7f86b40cf5e0 2024-04-25T11:41:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.368+0000 7f86bb400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86b415e830 con 0x7f86b41443c0 2024-04-25T11:41:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.368+0000 7f86bb400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f86b40708d0 con 0x7f86b4167000 2024-04-25T11:41:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.368+0000 7f86b3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f86b40cf5e0 0x7f86b40cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50398/0 (socket says 172.21.15.19:50398) 2024-04-25T11:41:50.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.368+0000 7f86b3e00640 1 -- 172.21.15.19:0/3178605731 learned_addr learned my addr 172.21.15.19:0/3178605731 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.368+0000 7f86b9400640 1 --2- 172.21.15.19:0/3178605731 >> v2:172.21.15.169:3300/0 conn(0x7f86b4167000 0x7f86b4169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550299005 0 0) 0x7f86b415f5f0 con 0x7f86b40cf5e0 2024-04-25T11:41:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8698003520 con 0x7f86b40cf5e0 2024-04-25T11:41:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3627825316 0 0) 0x7f8698003520 con 0x7f86b40cf5e0 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 >> v2:172.21.15.169:3300/0 conn(0x7f86b4167000 msgr2=0x7f86b4169420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 --2- 172.21.15.19:0/3178605731 >> v2:172.21.15.169:3300/0 conn(0x7f86b4167000 0x7f86b4169420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 >> v1:172.21.15.169:6789/0 conn(0x7f86b41443c0 legacy=0x7f86b406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b415eb10 con 0x7f86b40cf5e0 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f86a0001670 con 0x7f86b40cf5e0 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 >> v1:172.21.15.19:6789/0 conn(0x7f86b40cf5e0 legacy=0x7f86b40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86980039b0 0x7f8698003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8698004800 0x7f8698004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8698005270 0x7f869800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86b40708d0 con 0x7f8698005270 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f869800c200 con 0x7f86980039b0 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.369+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f869800c400 con 0x7f8698004800 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b9400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8698005270 0x7f869800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b8a00640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86980039b0 0x7f8698003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b3e00640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8698004800 0x7f8698004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b9400640 1 -- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8698004800 msgr2=0x7f8698004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b9400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8698004800 0x7f8698004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b9400640 1 -- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86980039b0 msgr2=0x7f8698003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b9400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86980039b0 0x7f8698003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b9400640 1 -- 172.21.15.19:0/3178605731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8698003520 con 0x7f8698005270 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.370+0000 7f86b3e00640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8698004800 0x7f8698004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.371+0000 7f86b9400640 1 -- 172.21.15.19:0/3178605731 reap_dead start 2024-04-25T11:41:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.371+0000 7f86b9400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8698005270 0x7f869800b9c0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00ba00 tx=0x7f8698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3251e0da71e4cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.371+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac030070 con 0x7f8698005270 2024-04-25T11:41:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.371+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac02ade0 con 0x7f8698005270 2024-04-25T11:41:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.371+0000 7f86b3400640 1 -- 172.21.15.19:0/3178605731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00b600 con 0x7f8698005270 2024-04-25T11:41:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.372+0000 7f86bb400640 1 -- 172.21.15.19:0/3178605731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8698005270 msgr2=0x7f869800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.372+0000 7f86bb400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8698005270 0x7f869800b9c0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00ba00 tx=0x7f8698005830 comp rx=0 tx=0).stop 2024-04-25T11:41:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.372+0000 7f86bb400640 1 -- 172.21.15.19:0/3178605731 shutdown_connections 2024-04-25T11:41:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.372+0000 7f86bb400640 1 --2- 172.21.15.19:0/3178605731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8698005270 0x7f869800b9c0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.372+0000 7f86bb400640 1 -- 172.21.15.19:0/3178605731 >> 172.21.15.19:0/3178605731 conn(0x7f86b40d1ae0 msgr2=0x7f86b40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.372+0000 7f86bb400640 1 -- 172.21.15.19:0/3178605731 shutdown_connections 2024-04-25T11:41:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.372+0000 7f86bb400640 1 -- 172.21.15.19:0/3178605731 wait complete. 2024-04-25T11:41:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.375+0000 7f86bb400640 1 Processor -- start 2024-04-25T11:41:50.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.375+0000 7f86bb400640 1 -- start start 2024-04-25T11:41:50.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.376+0000 7f86bb400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698005270 0x7f86b40b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.376+0000 7f86bb400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b41435d0 0x7f86b415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b415c9a0 0x7f86b40b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86bb400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86b41447d0 con 0x7f86b415c9a0 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86bb400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f86b415f160 con 0x7f8698005270 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86bb400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f86b4162b00 con 0x7f86b41435d0 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b415c9a0 0x7f86b40b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b415c9a0 0x7f86b40b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47492/0 (socket says 172.21.15.19:47492) 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 -- 172.21.15.19:0/1576579096 learned_addr learned my addr 172.21.15.19:0/1576579096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:50.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b3e00640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b41435d0 0x7f86b415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b8a00640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698005270 0x7f86b40b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 -- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b41435d0 msgr2=0x7f86b415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b41435d0 0x7f86b415c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 -- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698005270 msgr2=0x7f86b40b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698005270 0x7f86b40b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b9400640 1 -- 172.21.15.19:0/1576579096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b415cdb0 con 0x7f86b415c9a0 2024-04-25T11:41:50.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.377+0000 7f86b8a00640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698005270 0x7f86b40b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.378+0000 7f86b9400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b415c9a0 0x7f86b40b4a70 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f86ac02a8d0 tx=0x7f86ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.378+0000 7f86b0c00640 1 -- 172.21.15.19:0/1576579096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac02abe0 con 0x7f86b415c9a0 2024-04-25T11:41:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.378+0000 7f86b0c00640 1 -- 172.21.15.19:0/1576579096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac004020 con 0x7f86b415c9a0 2024-04-25T11:41:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.378+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b40af750 con 0x7f86b415c9a0 2024-04-25T11:41:50.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.378+0000 7f86b0c00640 1 -- 172.21.15.19:0/1576579096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac004410 con 0x7f86b415c9a0 2024-04-25T11:41:50.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.378+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b40afd10 con 0x7f86b415c9a0 2024-04-25T11:41:50.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.379+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b40af4c0 con 0x7f86b415c9a0 2024-04-25T11:41:50.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.381+0000 7f86b0c00640 1 -- 172.21.15.19:0/1576579096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f86ac003610 con 0x7f86b415c9a0 2024-04-25T11:41:50.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.382+0000 7f86b0c00640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86940744e0 0x7f86940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.382+0000 7f86b0c00640 1 -- 172.21.15.19:0/1576579096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7f86ac0b6100 con 0x7f86b415c9a0 2024-04-25T11:41:50.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.383+0000 7f86b8a00640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86940744e0 0x7f86940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.384+0000 7f86b8a00640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86940744e0 0x7f86940769f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f86b40d1860 tx=0x7f86a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.385+0000 7f86b0c00640 1 -- 172.21.15.19:0/1576579096 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86ac04c020 con 0x7f86b415c9a0 2024-04-25T11:41:50.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:50.630+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f86b4050740 con 0x7f86b415c9a0 2024-04-25T11:41:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.228+0000 7f86b0c00640 1 -- 172.21.15.19:0/1576579096 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v342) ==== 161+0+0 (secure 0 0 0) 0x7f86b415cdb0 con 0x7f86b415c9a0 2024-04-25T11:41:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86940744e0 msgr2=0x7f86940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86940744e0 0x7f86940769f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f86b40d1860 tx=0x7f86a4009040 comp rx=0 tx=0).stop 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b415c9a0 msgr2=0x7f86b40b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b415c9a0 0x7f86b40b4a70 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f86ac02a8d0 tx=0x7f86ac0046c0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 shutdown_connections 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f86940744e0 0x7f86940769f0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86b415c9a0 0x7f86b40b4a70 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b41435d0 0x7f86b415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 --2- 172.21.15.19:0/1576579096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8698005270 0x7f86b40b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 >> 172.21.15.19:0/1576579096 conn(0x7f86b40d1ae0 msgr2=0x7f86b40684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 shutdown_connections 2024-04-25T11:41:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.232+0000 7f86bb400640 1 -- 172.21.15.19:0/1576579096 wait complete. 2024-04-25T11:41:51.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:41:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.549+0000 7efc81e00640 1 Processor -- start 2024-04-25T11:41:51.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.550+0000 7efc81e00640 1 -- start start 2024-04-25T11:41:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.552+0000 7efc81e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c0b4ea0 0x7efc7c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.552+0000 7efc81e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7c0d1230 con 0x7efc7c144370 2024-04-25T11:41:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.552+0000 7efc81e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7c0d06b0 con 0x7efc7c06c5c0 2024-04-25T11:41:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.552+0000 7efc81e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c0d0990 con 0x7efc7c0b4ea0 2024-04-25T11:41:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc81400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efc7c144370 0x7efc7c15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50414/0 (socket says 172.21.15.19:50414) 2024-04-25T11:41:51.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc81400640 1 -- 172.21.15.19:0/938848887 learned_addr learned my addr 172.21.15.19:0/938848887 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7be00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c0b4ea0 0x7efc7c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719606992 0 0) 0x7efc7c0d1230 con 0x7efc7c144370 2024-04-25T11:41:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc64003520 con 0x7efc7c144370 2024-04-25T11:41:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3162575922 0 0) 0x7efc64003520 con 0x7efc7c144370 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c0b4ea0 msgr2=0x7efc7c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c0b4ea0 0x7efc7c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 >> v1:172.21.15.169:6789/0 conn(0x7efc7c06c5c0 legacy=0x7efc7c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c0d0c70 con 0x7efc7c144370 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.553+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7efc740019f0 con 0x7efc7c144370 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 >> v1:172.21.15.19:6789/0 conn(0x7efc7c144370 legacy=0x7efc7c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7b400640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640039e0 0x7efc64003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7b400640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64004800 0x7efc64004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7b400640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc64005270 0x7efc6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c0d0990 con 0x7efc64004800 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efc6400bfb0 con 0x7efc640039e0 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc6400c1b0 con 0x7efc64005270 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7be00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64004800 0x7efc64004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc81400640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc64005270 0x7efc6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7be00640 1 -- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc64005270 msgr2=0x7efc6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc80a00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640039e0 0x7efc64003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7be00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc64005270 0x7efc6400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7be00640 1 -- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640039e0 msgr2=0x7efc64003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.554+0000 7efc7be00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640039e0 0x7efc64003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.555+0000 7efc7be00640 1 -- 172.21.15.19:0/938848887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64003520 con 0x7efc64004800 2024-04-25T11:41:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.555+0000 7efc80a00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640039e0 0x7efc64003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.556+0000 7efc7be00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64004800 0x7efc64004c10 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7efc6800ba30 tx=0x7efc64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e5ac3b4f895796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.556+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68030070 con 0x7efc64004800 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.556+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6802ae00 con 0x7efc64004800 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.556+0000 7efc7b400640 1 -- 172.21.15.19:0/938848887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6800b600 con 0x7efc64004800 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.557+0000 7efc81e00640 1 -- 172.21.15.19:0/938848887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64004800 msgr2=0x7efc64004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.557+0000 7efc81e00640 1 --2- 172.21.15.19:0/938848887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc64004800 0x7efc64004c10 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7efc6800ba30 tx=0x7efc64005830 comp rx=0 tx=0).stop 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.557+0000 7efc81400640 1 -- 172.21.15.19:0/938848887 reap_dead start 2024-04-25T11:41:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.557+0000 7efc81e00640 1 -- 172.21.15.19:0/938848887 shutdown_connections 2024-04-25T11:41:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.557+0000 7efc81e00640 1 -- 172.21.15.19:0/938848887 >> 172.21.15.19:0/938848887 conn(0x7efc7c0d1720 msgr2=0x7efc7c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.557+0000 7efc81e00640 1 -- 172.21.15.19:0/938848887 shutdown_connections 2024-04-25T11:41:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.557+0000 7efc81e00640 1 -- 172.21.15.19:0/938848887 wait complete. 2024-04-25T11:41:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.559+0000 7efc81e00640 1 Processor -- start 2024-04-25T11:41:51.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.559+0000 7efc81e00640 1 -- start start 2024-04-25T11:41:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc81e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64004800 0x7efc7c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc81e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c13de50 0x7efc7c06ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc81e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1f1a80 0x7efc7c070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc81e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c0d0450 con 0x7efc7c13de50 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc81e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c0d02d0 con 0x7efc64004800 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc81e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efc7c147760 con 0x7efc7c1f1a80 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c13de50 0x7efc7c06ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c13de50 0x7efc7c06ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47518/0 (socket says 172.21.15.19:47518) 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.561+0000 7efc7be00640 1 -- 172.21.15.19:0/2096065872 learned_addr learned my addr 172.21.15.19:0/2096065872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc81400640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1f1a80 0x7efc7c070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc80a00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64004800 0x7efc7c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc7be00640 1 -- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1f1a80 msgr2=0x7efc7c070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc7be00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1f1a80 0x7efc7c070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc7be00640 1 -- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64004800 msgr2=0x7efc7c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc7be00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64004800 0x7efc7c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc7be00640 1 -- 172.21.15.19:0/2096065872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c1f2000 con 0x7efc7c13de50 2024-04-25T11:41:51.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.562+0000 7efc7be00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c13de50 0x7efc7c06ff40 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7efc6802a8f0 tx=0x7efc68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.563+0000 7efc78c00640 1 -- 172.21.15.19:0/2096065872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68042030 con 0x7efc7c13de50 2024-04-25T11:41:51.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.563+0000 7efc78c00640 1 -- 172.21.15.19:0/2096065872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6802ac00 con 0x7efc7c13de50 2024-04-25T11:41:51.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.563+0000 7efc78c00640 1 -- 172.21.15.19:0/2096065872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6800b600 con 0x7efc7c13de50 2024-04-25T11:41:51.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.563+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c1f22e0 con 0x7efc7c13de50 2024-04-25T11:41:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.563+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c1f28a0 con 0x7efc7c13de50 2024-04-25T11:41:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.564+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7c050740 con 0x7efc7c13de50 2024-04-25T11:41:51.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.565+0000 7efc78c00640 1 -- 172.21.15.19:0/2096065872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc68045070 con 0x7efc7c13de50 2024-04-25T11:41:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.567+0000 7efc78c00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc5c0744b0 0x7efc5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.567+0000 7efc78c00640 1 -- 172.21.15.19:0/2096065872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7efc680b67a0 con 0x7efc7c13de50 2024-04-25T11:41:51.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.567+0000 7efc80a00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc5c0744b0 0x7efc5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.569+0000 7efc80a00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc5c0744b0 0x7efc5c0769c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7efc70001750 tx=0x7efc70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.572+0000 7efc78c00640 1 -- 172.21.15.19:0/2096065872 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc680801b0 con 0x7efc7c13de50 2024-04-25T11:41:51.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:51.851+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7efc7c13e840 con 0x7efc7c13de50 2024-04-25T11:41:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.232+0000 7efc78c00640 1 -- 172.21.15.19:0/2096065872 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v343) ==== 101+0+0 (secure 0 0 0) 0x7efc6807fab0 con 0x7efc7c13de50 2024-04-25T11:41:52.219 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:41:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.235+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc5c0744b0 msgr2=0x7efc5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.235+0000 7efc81e00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc5c0744b0 0x7efc5c0769c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7efc70001750 tx=0x7efc70009040 comp rx=0 tx=0).stop 2024-04-25T11:41:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.235+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c13de50 msgr2=0x7efc7c06ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.235+0000 7efc81e00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c13de50 0x7efc7c06ff40 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7efc6802a8f0 tx=0x7efc68004730 comp rx=0 tx=0).stop 2024-04-25T11:41:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 shutdown_connections 2024-04-25T11:41:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc5c0744b0 0x7efc5c0769c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc7c1f1a80 0x7efc7c070480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc7c13de50 0x7efc7c06ff40 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 --2- 172.21.15.19:0/2096065872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc64004800 0x7efc7c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 >> 172.21.15.19:0/2096065872 conn(0x7efc7c0d1720 msgr2=0x7efc7c15a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 shutdown_connections 2024-04-25T11:41:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.236+0000 7efc81e00640 1 -- 172.21.15.19:0/2096065872 wait complete. 2024-04-25T11:41:52.249 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:41:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.441+0000 7f6546340f40 1 Processor -- start 2024-04-25T11:41:52.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.441+0000 7f6546340f40 1 -- start start 2024-04-25T11:41:52.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f6546340f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e5961e0 0x55699e66c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f6546340f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5965f0 0x55699e662660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f6546340f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e66ffa0 0x55699e667ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f6546340f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55699e5dee70 con 0x55699e5965f0 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f6546340f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55699e53cd20 con 0x55699e66ffa0 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f6546340f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55699e5df3f0 con 0x55699e5961e0 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f653ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5965f0 0x55699e662660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f653ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5965f0 0x55699e662660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47538/0 (socket says 172.21.15.19:47538) 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f653ea00640 1 -- 172.21.15.19:0/984221857 learned_addr learned my addr 172.21.15.19:0/984221857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f653f400640 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e5961e0 0x55699e66c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.443+0000 7f653fe00640 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e66ffa0 0x55699e667ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653ea00640 1 -- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e5961e0 msgr2=0x55699e66c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653ea00640 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e5961e0 0x55699e66c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653ea00640 1 -- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e66ffa0 msgr2=0x55699e667ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:52.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653ea00640 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e66ffa0 0x55699e667ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653ea00640 1 -- 172.21.15.19:0/984221857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699e5948d0 con 0x55699e5965f0 2024-04-25T11:41:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653fe00640 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e66ffa0 0x55699e667ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653f400640 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e5961e0 0x55699e66c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653ea00640 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5965f0 0x55699e662660 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f652800a430 tx=0x7f652800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442632fa3f21f6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653d600640 1 -- 172.21.15.19:0/984221857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652800b920 con 0x55699e5965f0 2024-04-25T11:41:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653d600640 1 -- 172.21.15.19:0/984221857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f652800f030 con 0x55699e5965f0 2024-04-25T11:41:52.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.444+0000 7f653d600640 1 -- 172.21.15.19:0/984221857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652800bd20 con 0x55699e5965f0 2024-04-25T11:41:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 -- 172.21.15.19:0/984221857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5965f0 msgr2=0x55699e662660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5965f0 0x55699e662660 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f652800a430 tx=0x7f652800a8f0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 -- 172.21.15.19:0/984221857 shutdown_connections 2024-04-25T11:41:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e66ffa0 0x55699e667ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5965f0 0x55699e662660 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 --2- 172.21.15.19:0/984221857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e5961e0 0x55699e66c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 -- 172.21.15.19:0/984221857 >> 172.21.15.19:0/984221857 conn(0x55699e4b8ac0 msgr2=0x55699e5c80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 -- 172.21.15.19:0/984221857 shutdown_connections 2024-04-25T11:41:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.445+0000 7f6546340f40 1 -- 172.21.15.19:0/984221857 wait complete. 2024-04-25T11:41:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.446+0000 7f6546340f40 1 Processor -- start 2024-04-25T11:41:52.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.447+0000 7f6546340f40 1 -- start start 2024-04-25T11:41:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f6546340f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5961e0 0x55699e665b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f6546340f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e5965f0 0x55699e5e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f6546340f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e66ffa0 0x55699e5e3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f6546340f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55699e594f30 con 0x55699e5961e0 2024-04-25T11:41:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f6546340f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55699e595210 con 0x55699e5965f0 2024-04-25T11:41:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f6546340f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55699e7054e0 con 0x55699e66ffa0 2024-04-25T11:41:52.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f653f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5961e0 0x55699e665b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f653f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5961e0 0x55699e665b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47548/0 (socket says 172.21.15.19:47548) 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.448+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 learned_addr learned my addr 172.21.15.19:0/2782760308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e5965f0 0x55699e5e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e66ffa0 msgr2=0x55699e5e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653fe00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e66ffa0 0x55699e5e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653f400640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55699e66ffa0 0x55699e5e3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e5965f0 msgr2=0x55699e5e2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653f400640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e5965f0 0x55699e5e2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699e5c4d30 con 0x55699e5961e0 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55699e5965f0 0x55699e5e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.449+0000 7f653f400640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5961e0 0x55699e665b30 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6530007d50 tx=0x7f6530006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.450+0000 7f6537e00640 1 -- 172.21.15.19:0/2782760308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530010710 con 0x55699e5961e0 2024-04-25T11:41:52.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.450+0000 7f6537e00640 1 -- 172.21.15.19:0/2782760308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65300108b0 con 0x55699e5961e0 2024-04-25T11:41:52.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.450+0000 7f6537e00640 1 -- 172.21.15.19:0/2782760308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530010ca0 con 0x55699e5961e0 2024-04-25T11:41:52.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.450+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55699e6611f0 con 0x55699e5961e0 2024-04-25T11:41:52.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.450+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699e6617b0 con 0x55699e5961e0 2024-04-25T11:41:52.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.452+0000 7f6537e00640 1 -- 172.21.15.19:0/2782760308 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6530012400 con 0x55699e5961e0 2024-04-25T11:41:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.453+0000 7f6537e00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f651c072e90 0x7f651c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.453+0000 7f6537e00640 1 -- 172.21.15.19:0/2782760308 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7f6530094330 con 0x55699e5961e0 2024-04-25T11:41:52.440 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:41:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.454+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55699e661fb0 0x55699e704b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.454+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e70f780 con 0x55699e661fb0 2024-04-25T11:41:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.454+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f651c072e90 0x7f651c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.454+0000 7f653fe00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55699e661fb0 0x55699e704b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.455+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f651c072e90 0x7f651c0753a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f652800a430 tx=0x7f6528016040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.455+0000 7f653fe00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55699e661fb0 0x55699e704b40 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.460+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v130'1159 uv1133 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.460+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e70fb20 con 0x55699e661fb0 2024-04-25T11:41:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.464+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v130'1160 uv1160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.464+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e70fec0 con 0x55699e661fb0 2024-04-25T11:41:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.468+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v130'1161 uv1161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.454 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:41:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.468+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e711270 con 0x55699e661fb0 2024-04-25T11:41:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.471+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v130'1162 uv1162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.471+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e705080 con 0x55699e661fb0 2024-04-25T11:41:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.474+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v130'1163 uv1163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.475+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710260 con 0x55699e661fb0 2024-04-25T11:41:52.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.478+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v130'1164 uv1164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.478+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710650 con 0x55699e661fb0 2024-04-25T11:41:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.482+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v130'1165 uv1139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.483+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.486+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v130'1166 uv1166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.487+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.490+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v130'1167 uv1167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.477 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:41:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.491+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.495+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v130'1168 uv1168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.495+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.498+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v130'1169 uv1169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.498+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.503+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v130'1170 uv1170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.503+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.508+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v130'1171 uv1146 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.508+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.512+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v130'1172 uv1172 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.512+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.503 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:41:52.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.517+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v130'1173 uv1173 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.517+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.521+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v130'1174 uv1174 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.521+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.525+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v130'1175 uv1175 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.526+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7109f0 con 0x55699e661fb0 2024-04-25T11:41:52.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.530+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v130'1176 uv1176 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.530+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e65a090 con 0x55699e661fb0 2024-04-25T11:41:52.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.535+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v130'1177 uv1177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.535+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e65a090 con 0x55699e661fb0 2024-04-25T11:41:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.540+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v130'1178 uv1152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.540+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e65a090 con 0x55699e661fb0 2024-04-25T11:41:52.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.544+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v130'1179 uv1179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.544+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e65a090 con 0x55699e661fb0 2024-04-25T11:41:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.549+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v130'1180 uv1180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.536 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:41:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.549+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e65a090 con 0x55699e661fb0 2024-04-25T11:41:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.553+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v130'1181 uv1181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.554+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.558+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v130'1182 uv1182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.558+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.563+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v130'1183 uv1183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.563+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.568+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v130'1184 uv1158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.568+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.573+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v130'1185 uv1185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.573+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.578+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v130'1186 uv1186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.564 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:41:52.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.578+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.583+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v130'1187 uv1187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.583+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.587+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v130'1188 uv1188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.587+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e661fb0 2024-04-25T11:41:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.592+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v130'1189 uv1189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.592+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55699e711b20 0x55699e713f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.592+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e710cb0 con 0x55699e711b20 2024-04-25T11:41:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.593+0000 7f653f400640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55699e711b20 0x55699e713f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.594+0000 7f653f400640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55699e711b20 0x55699e713f40 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.599+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v130'846 uv828 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.599+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e714480 con 0x55699e711b20 2024-04-25T11:41:52.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.604+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v130'847 uv847 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.604+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e714820 con 0x55699e711b20 2024-04-25T11:41:52.595 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:41:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.608+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v130'848 uv848 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.609+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e714bc0 con 0x55699e711b20 2024-04-25T11:41:52.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.613+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v130'849 uv849 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.613+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e714f60 con 0x55699e711b20 2024-04-25T11:41:52.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.617+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v130'850 uv850 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.618+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715300 con 0x55699e711b20 2024-04-25T11:41:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.623+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v130'851 uv833 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.623+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e7156a0 con 0x55699e711b20 2024-04-25T11:41:52.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.627+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v130'852 uv852 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.627+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.618 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:41:52.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.631+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v130'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.631+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.636+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v130'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.636+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.641+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v130'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.641+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.646+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v130'856 uv838 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.646+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.650+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v130'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.650+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.654+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v130'858 uv858 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.641 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:41:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.655+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71a130 con 0x55699e711b20 2024-04-25T11:41:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.660+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v130'859 uv859 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.661+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71a130 con 0x55699e711b20 2024-04-25T11:41:52.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.665+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v130'860 uv860 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.665+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.670+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v130'861 uv844 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.670+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.675+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v130'862 uv862 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.675+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715a40 con 0x55699e711b20 2024-04-25T11:41:52.665 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:41:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.679+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v130'863 uv863 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.679+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e716a50 con 0x55699e711b20 2024-04-25T11:41:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.683+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v130'864 uv864 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.683+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e716a50 con 0x55699e711b20 2024-04-25T11:41:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.687+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v130'865 uv865 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.687+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e715ef0 con 0x55699e711b20 2024-04-25T11:41:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.692+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v130'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e711b20 2024-04-25T11:41:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.692+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55699e716400 0x55699e718820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.692+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e718d60 con 0x55699e716400 2024-04-25T11:41:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.693+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55699e716400 0x55699e718820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.695+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55699e716400 0x55699e718820 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.699+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.699+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e719100 con 0x55699e716400 2024-04-25T11:41:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.704+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v130'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.704+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719560 con 0x55699e716400 2024-04-25T11:41:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.707+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v130'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.694 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:41:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.708+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71ae10 con 0x55699e716400 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.711+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.712+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71b1b0 con 0x55699e716400 2024-04-25T11:41:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.716+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.716+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719e00 con 0x55699e716400 2024-04-25T11:41:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.720+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.720+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71a210 con 0x55699e661fb0 2024-04-25T11:41:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.725+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v130'172 uv171 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.725+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e71a210 con 0x55699e661fb0 2024-04-25T11:41:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.729+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v130'173 uv173 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.729+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71a210 con 0x55699e661fb0 2024-04-25T11:41:52.719 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:41:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.732+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v130'174 uv174 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.733+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71a210 con 0x55699e661fb0 2024-04-25T11:41:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.737+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v130'175 uv175 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6538051940 con 0x55699e661fb0 2024-04-25T11:41:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.737+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55699e71b550 0x55699e71d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.737+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71a210 con 0x55699e71b550 2024-04-25T11:41:52.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.738+0000 7f653fe00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55699e71b550 0x55699e71d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.740+0000 7f653fe00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55699e71b550 0x55699e71d970 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.744+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.745+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e71a210 con 0x55699e71b550 2024-04-25T11:41:52.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.749+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v130'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.749+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71a210 con 0x55699e71b550 2024-04-25T11:41:52.740 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:41:52.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.754+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v130'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.754+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71ef10 con 0x55699e71b550 2024-04-25T11:41:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.758+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.758+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71ef10 con 0x55699e71b550 2024-04-25T11:41:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.763+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.763+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e71df00 con 0x55699e71b550 2024-04-25T11:41:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.767+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.767+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55699e71e410 0x55699e720830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.768+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e720dc0 con 0x55699e71e410 2024-04-25T11:41:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.768+0000 7f653f400640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55699e71e410 0x55699e720830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.770+0000 7f653f400640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55699e71e410 0x55699e720830 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.774+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v130'415 uv409 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.774+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.778+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v130'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.778+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.768 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:41:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.782+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v130'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.782+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7221a0 con 0x55699e71e410 2024-04-25T11:41:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.786+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v130'418 uv418 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.786+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7221a0 con 0x55699e71e410 2024-04-25T11:41:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.791+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v130'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.791+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.795+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v130'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.795+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.799+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v130'421 uv414 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.799+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.804+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v130'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.804+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.794 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:41:52.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.808+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v130'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.808+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.813+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v130'424 uv424 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.813+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e721190 con 0x55699e71e410 2024-04-25T11:41:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.817+0000 7f653f400640 1 -- 172.21.15.19:0/2782760308 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v130'425 uv425 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6530010070 con 0x55699e71e410 2024-04-25T11:41:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.818+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55699e721860 0x55699e723ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.818+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724370 con 0x55699e721860 2024-04-25T11:41:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.818+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55699e721860 0x55699e723ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.820+0000 7f653ea00640 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55699e721860 0x55699e723ca0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.824+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v130'188 uv187 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.824+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e724740 con 0x55699e721860 2024-04-25T11:41:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.829+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v130'189 uv189 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.829+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e721860 2024-04-25T11:41:52.819 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:41:52.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.832+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v130'190 uv190 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.833+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e721860 2024-04-25T11:41:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.837+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v130'191 uv191 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.837+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e721860 2024-04-25T11:41:52.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.841+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v130'192 uv192 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.841+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e661fb0 2024-04-25T11:41:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.846+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e661fb0 2024-04-25T11:41:52.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.846+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e661fb0 2024-04-25T11:41:52.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.850+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v130'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e661fb0 2024-04-25T11:41:52.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.850+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e661fb0 2024-04-25T11:41:52.841 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:41:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.855+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v130'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e661fb0 2024-04-25T11:41:52.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.855+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e725b70 con 0x55699e661fb0 2024-04-25T11:41:52.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.859+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e661fb0 2024-04-25T11:41:52.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.859+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e725b70 con 0x55699e661fb0 2024-04-25T11:41:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.864+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e661fb0 2024-04-25T11:41:52.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.864+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e661fb0 2024-04-25T11:41:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.868+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e661fb0 2024-04-25T11:41:52.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.868+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e716400 2024-04-25T11:41:52.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.873+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v130'376 uv369 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.873+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e716400 2024-04-25T11:41:52.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.877+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v130'377 uv377 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.877+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724b60 con 0x55699e716400 2024-04-25T11:41:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.882+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v130'378 uv378 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.868 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:41:52.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.882+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724f20 con 0x55699e716400 2024-04-25T11:41:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.887+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v130'379 uv379 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.887+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724f20 con 0x55699e716400 2024-04-25T11:41:52.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.891+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v130'380 uv380 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.891+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724f20 con 0x55699e716400 2024-04-25T11:41:52.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.896+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v130'381 uv375 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.896+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e724f20 con 0x55699e716400 2024-04-25T11:41:52.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.900+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v130'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.900+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724f20 con 0x55699e716400 2024-04-25T11:41:52.891 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:41:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.904+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v130'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.905+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724f20 con 0x55699e716400 2024-04-25T11:41:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.909+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v130'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.909+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719a20 con 0x55699e716400 2024-04-25T11:41:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.914+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v130'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e716400 2024-04-25T11:41:52.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.914+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199e0 con 0x55699e71b550 2024-04-25T11:41:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.918+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v130'167 uv165 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.918+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e7199e0 con 0x55699e71b550 2024-04-25T11:41:52.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.923+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v130'168 uv168 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.923+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199e0 con 0x55699e71b550 2024-04-25T11:41:52.914 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:41:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.928+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v130'169 uv169 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.928+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199e0 con 0x55699e71b550 2024-04-25T11:41:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.932+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v130'170 uv170 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.932+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199e0 con 0x55699e71b550 2024-04-25T11:41:52.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.937+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v130'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.937+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e7199c0 con 0x55699e71b550 2024-04-25T11:41:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.941+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v130'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.941+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199c0 con 0x55699e71b550 2024-04-25T11:41:52.931 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:41:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.945+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v130'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.945+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199c0 con 0x55699e71b550 2024-04-25T11:41:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.949+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v130'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.949+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199c0 con 0x55699e71b550 2024-04-25T11:41:52.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.954+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v130'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.954+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199c0 con 0x55699e71b550 2024-04-25T11:41:52.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.958+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v130'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6538051d70 con 0x55699e71b550 2024-04-25T11:41:52.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.959+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199a0 con 0x55699e721860 2024-04-25T11:41:52.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.962+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v130'191 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.962+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e7199a0 con 0x55699e721860 2024-04-25T11:41:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.967+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v130'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.967+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199a0 con 0x55699e721860 2024-04-25T11:41:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.971+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v130'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.957 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:41:52.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.971+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e7199a0 con 0x55699e721860 2024-04-25T11:41:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.975+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v130'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.975+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719980 con 0x55699e721860 2024-04-25T11:41:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.979+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v130'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.979+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719980 con 0x55699e721860 2024-04-25T11:41:52.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.984+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v130'468 uv459 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.985+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e719980 con 0x55699e721860 2024-04-25T11:41:52.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.989+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v130'469 uv469 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.989+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719980 con 0x55699e721860 2024-04-25T11:41:52.979 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:41:52.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.993+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v130'470 uv470 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.993+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719980 con 0x55699e721860 2024-04-25T11:41:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.998+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v130'471 uv471 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:52.998+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719980 con 0x55699e721860 2024-04-25T11:41:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.002+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v130'472 uv472 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.003+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719980 con 0x55699e721860 2024-04-25T11:41:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.007+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v130'473 uv466 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.007+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e130) -- 0x55699e719950 con 0x55699e721860 2024-04-25T11:41:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.011+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v130'474 uv474 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.011+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e719950 con 0x55699e721860 2024-04-25T11:41:53.002 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:41:53.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.016+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v130'475 uv475 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:53.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.016+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e725f30 con 0x55699e721860 2024-04-25T11:41:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.020+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v130'476 uv476 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:53.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.020+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e725f30 con 0x55699e721860 2024-04-25T11:41:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.024+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v130'477 uv477 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:53.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.025+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e724f20 con 0x55699e721860 2024-04-25T11:41:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.029+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v130'478 uv478 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:53.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.029+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55699e725680 con 0x55699e721860 2024-04-25T11:41:53.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.033+0000 7f653ea00640 1 -- 172.21.15.19:0/2782760308 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v130'479 uv479 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f652800cd70 con 0x55699e721860 2024-04-25T11:41:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.033+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55699e711b20 msgr2=0x55699e713f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.033+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55699e711b20 0x55699e713f40 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.033+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55699e71e410 msgr2=0x55699e720830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.033+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55699e71e410 0x55699e720830 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.033+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55699e721860 msgr2=0x55699e723ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.033+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55699e721860 0x55699e723ca0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55699e716400 msgr2=0x55699e718820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55699e716400 0x55699e718820 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55699e661fb0 msgr2=0x55699e704b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55699e661fb0 0x55699e704b40 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55699e71b550 msgr2=0x55699e71d970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55699e71b550 0x55699e71d970 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 reap_dead start 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 reap_dead start 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f653fe00640 1 -- 172.21.15.19:0/2782760308 reap_dead start 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f651c072e90 msgr2=0x7f651c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f651c072e90 0x7f651c0753a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f652800a430 tx=0x7f6528016040 comp rx=0 tx=0).stop 2024-04-25T11:41:53.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5961e0 msgr2=0x55699e665b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5961e0 0x55699e665b30 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6530007d50 tx=0x7f6530006750 comp rx=0 tx=0).stop 2024-04-25T11:41:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 shutdown_connections 2024-04-25T11:41:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f651c072e90 0x7f651c0753a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 --2- 172.21.15.19:0/2782760308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55699e5961e0 0x55699e665b30 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.034+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 >> 172.21.15.19:0/2782760308 conn(0x55699e4b8ac0 msgr2=0x55699e5e19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.035+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 shutdown_connections 2024-04-25T11:41:53.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.035+0000 7f6546340f40 1 -- 172.21.15.19:0/2782760308 wait complete. 2024-04-25T11:41:53.057 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:41:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.303+0000 7fbaca085f40 1 Processor -- start 2024-04-25T11:41:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.303+0000 7fbaca085f40 1 -- start start 2024-04-25T11:41:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbaca085f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8a4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbaca085f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 0x55f2ce97ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbaca085f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce97b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbaca085f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce8ede70 con 0x55f2ce8a51e0 2024-04-25T11:41:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbaca085f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce84bd20 con 0x55f2ce8a55f0 2024-04-25T11:41:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbaca085f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f2ce8ee110 con 0x55f2ce97edd0 2024-04-25T11:41:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbac3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbac3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8a4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47564/0 (socket says 172.21.15.19:47564) 2024-04-25T11:41:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.305+0000 7fbac3200640 1 -- 172.21.15.19:0/1765877333 learned_addr learned my addr 172.21.15.19:0/1765877333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3c00640 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce97b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac2800640 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 0x55f2ce97ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3200640 1 -- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 msgr2=0x55f2ce97b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3200640 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce97b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3200640 1 -- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 msgr2=0x55f2ce97ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3200640 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 0x55f2ce97ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3200640 1 -- 172.21.15.19:0/1765877333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ce97c7c0 con 0x55f2ce8a51e0 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3c00640 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce97b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac2800640 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 0x55f2ce97ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac3200640 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8a4180 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fbab40016b0 tx=0x7fbab4001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb040f5da30d5eca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac1400640 1 -- 172.21.15.19:0/1765877333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab40062e0 con 0x55f2ce8a51e0 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac1400640 1 -- 172.21.15.19:0/1765877333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab4003650 con 0x55f2ce8a51e0 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.306+0000 7fbac1400640 1 -- 172.21.15.19:0/1765877333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4003a20 con 0x55f2ce8a51e0 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 -- 172.21.15.19:0/1765877333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 msgr2=0x55f2ce8a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8a4180 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fbab40016b0 tx=0x7fbab4001b70 comp rx=0 tx=0).stop 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 -- 172.21.15.19:0/1765877333 shutdown_connections 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 0x55f2ce97ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce97b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 --2- 172.21.15.19:0/1765877333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8a4180 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 -- 172.21.15.19:0/1765877333 >> 172.21.15.19:0/1765877333 conn(0x55f2ce7c7ac0 msgr2=0x55f2ce8d73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 -- 172.21.15.19:0/1765877333 shutdown_connections 2024-04-25T11:41:53.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.307+0000 7fbaca085f40 1 -- 172.21.15.19:0/1765877333 wait complete. 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.308+0000 7fbaca085f40 1 Processor -- start 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.309+0000 7fbaca085f40 1 -- start start 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbaca085f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbaca085f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 0x55f2ce9722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbaca085f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce9679a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbaca085f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce971850 con 0x55f2ce8a51e0 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbaca085f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55f2ce97caa0 con 0x55f2ce8a55f0 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbaca085f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55f2ce971aa0 con 0x55f2ce97edd0 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbac3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbac3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8d73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47574/0 (socket says 172.21.15.19:47574) 2024-04-25T11:41:53.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.310+0000 7fbac3200640 1 -- 172.21.15.19:0/3115439564 learned_addr learned my addr 172.21.15.19:0/3115439564 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:53.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbac3c00640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce9679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbac3200640 1 -- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 msgr2=0x55f2ce9679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbac3200640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f2ce97edd0 0x55f2ce9679a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbac3200640 1 -- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 msgr2=0x55f2ce9722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbac3200640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f2ce8a55f0 0x55f2ce9722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbac3200640 1 -- 172.21.15.19:0/3115439564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ce9714d0 con 0x55f2ce8a51e0 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbac3200640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8d73c0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fbab4002070 tx=0x7fbab4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.311+0000 7fbabbe00640 1 -- 172.21.15.19:0/3115439564 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4025070 con 0x55f2ce8a51e0 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.312+0000 7fbabbe00640 1 -- 172.21.15.19:0/3115439564 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab4019040 con 0x55f2ce8a51e0 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.312+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f2ce96f860 con 0x55f2ce8a51e0 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.312+0000 7fbabbe00640 1 -- 172.21.15.19:0/3115439564 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4013a30 con 0x55f2ce8a51e0 2024-04-25T11:41:53.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.312+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ce968400 con 0x55f2ce8a51e0 2024-04-25T11:41:53.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.314+0000 7fbabbe00640 1 -- 172.21.15.19:0/3115439564 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbab4010070 con 0x55f2ce8a51e0 2024-04-25T11:41:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.315+0000 7fbabbe00640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbaa4072ec0 0x7fbaa40753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.315+0000 7fbabbe00640 1 -- 172.21.15.19:0/3115439564 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7fbab4095f90 con 0x55f2ce8a51e0 2024-04-25T11:41:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.315+0000 7fbaca085f40 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f2ce96e590 0x55f2ce96e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.315+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:1 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55f2cea1ed70 con 0x55f2ce96e590 2024-04-25T11:41:53.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.316+0000 7fbac2800640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbaa4072ec0 0x7fbaa40753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.316+0000 7fbac3c00640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f2ce96e590 0x55f2ce96e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.317+0000 7fbac3c00640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f2ce96e590 0x55f2ce96e9a0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:53.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.318+0000 7fbac2800640 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbaa4072ec0 0x7fbaa40753d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005770 tx=0x7fbab0005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:53.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.321+0000 7fbac3c00640 1 -- 172.21.15.19:0/3115439564 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v130'171 uv171 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fbabc054970 con 0x55f2ce96e590 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f2ce96e590 msgr2=0x55f2ce96e9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55f2ce96e590 0x55f2ce96e9a0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbaa4072ec0 msgr2=0x7fbaa40753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbaa4072ec0 0x7fbaa40753d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005770 tx=0x7fbab0005860 comp rx=0 tx=0).stop 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 msgr2=0x55f2ce8d73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 --2- 172.21.15.19:0/3115439564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f2ce8a51e0 0x55f2ce8d73c0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fbab4002070 tx=0x7fbab4001250 comp rx=0 tx=0).stop 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbac3c00640 1 -- 172.21.15.19:0/3115439564 reap_dead start 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 shutdown_connections 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 >> 172.21.15.19:0/3115439564 conn(0x55f2ce7c7ac0 msgr2=0x55f2ce8a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 shutdown_connections 2024-04-25T11:41:53.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.322+0000 7fbaca085f40 1 -- 172.21.15.19:0/3115439564 wait complete. 2024-04-25T11:41:53.320 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:41:53.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.618+0000 7f03b8e00640 1 Processor -- start 2024-04-25T11:41:53.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.618+0000 7f03b8e00640 1 -- start start 2024-04-25T11:41:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.620+0000 7f03b8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03b4144340 0x7f03b406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.620+0000 7f03b8e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f03b415ee60 0x7f03b415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.621+0000 7f03b8e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b406df20 con 0x7f03b406db10 2024-04-25T11:41:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.621+0000 7f03b8e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f03b415f270 con 0x7f03b4144340 2024-04-25T11:41:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.621+0000 7f03b8e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f03b4144750 con 0x7f03b415ee60 2024-04-25T11:41:53.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.621+0000 7f03b3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03b4144340 0x7f03b406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.621+0000 7f03b3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03b4144340 0x7f03b406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47582/0 (socket says 172.21.15.19:47582) 2024-04-25T11:41:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.621+0000 7f03b3400640 1 -- 172.21.15.19:0/2854825685 learned_addr learned my addr 172.21.15.19:0/2854825685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.621+0000 7f03b3e00640 1 --2- 172.21.15.19:0/2854825685 >> v2:172.21.15.169:3300/0 conn(0x7f03b415ee60 0x7f03b415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.622+0000 7f03b3400640 1 -- 172.21.15.19:0/2854825685 >> v2:172.21.15.169:3300/0 conn(0x7f03b415ee60 msgr2=0x7f03b415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:53.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.622+0000 7f03b3400640 1 --2- 172.21.15.19:0/2854825685 >> v2:172.21.15.169:3300/0 conn(0x7f03b415ee60 0x7f03b415f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.622+0000 7f03b3400640 1 -- 172.21.15.19:0/2854825685 >> v1:172.21.15.169:6789/0 conn(0x7f03b406db10 legacy=0x7f03b415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.622+0000 7f03b3400640 1 -- 172.21.15.19:0/2854825685 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b4070000 con 0x7f03b4144340 2024-04-25T11:41:53.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.623+0000 7f03b3400640 1 --2- 172.21.15.19:0/2854825685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4144340 0x7f03b406d5d0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f03b40d1790 tx=0x7f03a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eeaf0c7e921bc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.624+0000 7f03b2a00640 1 -- 172.21.15.19:0/2854825685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4037070 con 0x7f03b4144340 2024-04-25T11:41:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.624+0000 7f03b2a00640 1 -- 172.21.15.19:0/2854825685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a4032440 con 0x7f03b4144340 2024-04-25T11:41:53.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.624+0000 7f03b2a00640 1 -- 172.21.15.19:0/2854825685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4032810 con 0x7f03b4144340 2024-04-25T11:41:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.624+0000 7f03b8e00640 1 -- 172.21.15.19:0/2854825685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4144340 msgr2=0x7f03b406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.624+0000 7f03b8e00640 1 --2- 172.21.15.19:0/2854825685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4144340 0x7f03b406d5d0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f03b40d1790 tx=0x7f03a402a230 comp rx=0 tx=0).stop 2024-04-25T11:41:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.625+0000 7f03b8e00640 1 -- 172.21.15.19:0/2854825685 shutdown_connections 2024-04-25T11:41:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.625+0000 7f03b8e00640 1 --2- 172.21.15.19:0/2854825685 >> v2:172.21.15.169:3300/0 conn(0x7f03b415ee60 0x7f03b415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.625+0000 7f03b8e00640 1 --2- 172.21.15.19:0/2854825685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b4144340 0x7f03b406d5d0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.625+0000 7f03b8e00640 1 -- 172.21.15.19:0/2854825685 >> 172.21.15.19:0/2854825685 conn(0x7f03b4072f40 msgr2=0x7f03b4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:53.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.625+0000 7f03b8e00640 1 -- 172.21.15.19:0/2854825685 shutdown_connections 2024-04-25T11:41:53.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.625+0000 7f03b8e00640 1 -- 172.21.15.19:0/2854825685 wait complete. 2024-04-25T11:41:53.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.628+0000 7f03b8e00640 1 Processor -- start 2024-04-25T11:41:53.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.628+0000 7f03b8e00640 1 -- start start 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.630+0000 7f03b8e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03b406db10 0x7f03b4161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.630+0000 7f03b8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b415ee60 0x7f03b4160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.630+0000 7f03b8e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b40af220 0x7f03b4142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.630+0000 7f03b8e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03b406f800 con 0x7f03b415ee60 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.630+0000 7f03b8e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f03b406f520 con 0x7f03b40af220 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.630+0000 7f03b8e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f03b406fae0 con 0x7f03b406db10 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b415ee60 0x7f03b4160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b415ee60 0x7f03b4160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47592/0 (socket says 172.21.15.19:47592) 2024-04-25T11:41:53.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 -- 172.21.15.19:0/110504867 learned_addr learned my addr 172.21.15.19:0/110504867 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:53.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03b3400640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03b406db10 0x7f03b4161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03b3e00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b40af220 0x7f03b4142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 -- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03b406db10 msgr2=0x7f03b4161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03b406db10 0x7f03b4161260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 -- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b40af220 msgr2=0x7f03b4142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:53.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b40af220 0x7f03b4142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.631+0000 7f03abe00640 1 -- 172.21.15.19:0/110504867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b4160660 con 0x7f03b415ee60 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.632+0000 7f03abe00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b415ee60 0x7f03b4160120 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b3f0 tx=0x7f039c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.632+0000 7f03ab400640 1 -- 172.21.15.19:0/110504867 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c015020 con 0x7f03b415ee60 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.632+0000 7f03ab400640 1 -- 172.21.15.19:0/110504867 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039c011830 con 0x7f03b415ee60 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.632+0000 7f03ab400640 1 -- 172.21.15.19:0/110504867 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c011c20 con 0x7f03b415ee60 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.632+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b4161ce0 con 0x7f03b415ee60 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.632+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b4161fc0 con 0x7f03b415ee60 2024-04-25T11:41:53.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.633+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b4076c60 con 0x7f03b415ee60 2024-04-25T11:41:53.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.635+0000 7f03ab400640 1 -- 172.21.15.19:0/110504867 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f039c018020 con 0x7f03b415ee60 2024-04-25T11:41:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.636+0000 7f03ab400640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03900744e0 0x7f03900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:53.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.636+0000 7f03ab400640 1 -- 172.21.15.19:0/110504867 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8977+0+0 (secure 0 0 0) 0x7f039c095bc0 con 0x7f03b415ee60 2024-04-25T11:41:53.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.636+0000 7f03b3400640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03900744e0 0x7f03900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:53.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.638+0000 7f03b3400640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03900744e0 0x7f03900769f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f03a4034000 tx=0x7f03a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:53.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.641+0000 7f03ab400640 1 -- 172.21.15.19:0/110504867 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f039c01f030 con 0x7f03b415ee60 2024-04-25T11:41:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:53.892+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f03b4161a30 con 0x7f03b415ee60 2024-04-25T11:41:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.215+0000 7f03ab400640 1 -- 172.21.15.19:0/110504867 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v344) ==== 162+0+0 (secure 0 0 0) 0x7f039c05f5d0 con 0x7f03b415ee60 2024-04-25T11:41:54.202 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.218+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03900744e0 msgr2=0x7f03900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.218+0000 7f03b8e00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03900744e0 0x7f03900769f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f03a4034000 tx=0x7f03a402e000 comp rx=0 tx=0).stop 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.218+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b415ee60 msgr2=0x7f03b4160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.218+0000 7f03b8e00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b415ee60 0x7f03b4160120 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b3f0 tx=0x7f039c00b8b0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 shutdown_connections 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03900744e0 0x7f03900769f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03b40af220 0x7f03b4142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b415ee60 0x7f03b4160120 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 --2- 172.21.15.19:0/110504867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f03b406db10 0x7f03b4161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 >> 172.21.15.19:0/110504867 conn(0x7f03b4072f40 msgr2=0x7f03b40b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:54.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 shutdown_connections 2024-04-25T11:41:54.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.219+0000 7f03b8e00640 1 -- 172.21.15.19:0/110504867 wait complete. 2024-04-25T11:41:54.212 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:41:54.226+0000 7fda19600640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input [buffer:3] 2024-04-25T11:41:54.213 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:54.227+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:41:54.213 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:41:54.227+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:41:54.224 INFO:tasks.ceph.mds.a.smithi019.stderr: -18> 2024-04-25T11:41:54.226+0000 7fda19600640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: virtual void SessionMapStore::decode_header(ceph::bufferlist&) no longer understand old encoding version 1 < 101: Malformed input [buffer:3] 2024-04-25T11:41:54.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:54.349 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:41:54.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.515+0000 7f597b400640 1 Processor -- start 2024-04-25T11:41:54.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.515+0000 7f597b400640 1 -- start start 2024-04-25T11:41:54.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.517+0000 7f597b400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f59741443c0 0x7f597406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.517+0000 7f597b400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597415f5f0 con 0x7f59740cfcf0 2024-04-25T11:41:54.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.517+0000 7f597b400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597415e830 con 0x7f5974070370 2024-04-25T11:41:54.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.517+0000 7f597b400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f59740708d0 con 0x7f59741443c0 2024-04-25T11:41:54.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.518+0000 7f5979400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f59740cfcf0 0x7f597406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52224/0 (socket says 172.21.15.19:52224) 2024-04-25T11:41:54.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.518+0000 7f5979400640 1 -- 172.21.15.19:0/3065080766 learned_addr learned my addr 172.21.15.19:0/3065080766 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.518+0000 7f5978a00640 1 --2- 172.21.15.19:0/3065080766 >> v2:172.21.15.169:3300/0 conn(0x7f59741443c0 0x7f597406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.518+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2113705858 0 0) 0x7f597415f5f0 con 0x7f59740cfcf0 2024-04-25T11:41:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.518+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f595c003520 con 0x7f59740cfcf0 2024-04-25T11:41:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 810702956 0 0) 0x7f595c003520 con 0x7f59740cfcf0 2024-04-25T11:41:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 >> v2:172.21.15.169:3300/0 conn(0x7f59741443c0 msgr2=0x7f597406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:54.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 --2- 172.21.15.19:0/3065080766 >> v2:172.21.15.169:3300/0 conn(0x7f59741443c0 0x7f597406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 >> v1:172.21.15.169:6789/0 conn(0x7f5974070370 legacy=0x7f59740cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597415eb10 con 0x7f59740cfcf0 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f596c0019f0 con 0x7f59740cfcf0 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 >> v1:172.21.15.19:6789/0 conn(0x7f59740cfcf0 legacy=0x7f597406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f595c003a00 0x7f595c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c004800 0x7f595c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f595c005220 0x7f595c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59740708d0 con 0x7f595c003a00 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f595c00c1b0 con 0x7f595c005220 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f595c00c3b0 con 0x7f595c004800 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.519+0000 7f5978a00640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f595c003a00 0x7f595c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.520+0000 7f5978a00640 1 -- 172.21.15.19:0/3065080766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c004800 msgr2=0x7f595c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:54.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.520+0000 7f5978a00640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c004800 0x7f595c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.520+0000 7f5978a00640 1 -- 172.21.15.19:0/3065080766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f595c005220 msgr2=0x7f595c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:54.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.520+0000 7f5978a00640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f595c005220 0x7f595c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.520+0000 7f5978a00640 1 -- 172.21.15.19:0/3065080766 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c003520 con 0x7f595c003a00 2024-04-25T11:41:54.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.520+0000 7f5979400640 1 -- 172.21.15.19:0/3065080766 reap_dead start 2024-04-25T11:41:54.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.521+0000 7f5978a00640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f595c003a00 0x7f595c003e90 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f5968009a90 tx=0x7f595c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ca01b20343edf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:54.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.521+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968036070 con 0x7f595c003a00 2024-04-25T11:41:54.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.521+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5968031400 con 0x7f595c003a00 2024-04-25T11:41:54.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.521+0000 7f5973400640 1 -- 172.21.15.19:0/3065080766 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968009690 con 0x7f595c003a00 2024-04-25T11:41:54.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.522+0000 7f597b400640 1 -- 172.21.15.19:0/3065080766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f595c003a00 msgr2=0x7f595c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.522+0000 7f597b400640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f595c003a00 0x7f595c003e90 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f5968009a90 tx=0x7f595c005830 comp rx=0 tx=0).stop 2024-04-25T11:41:54.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.522+0000 7f597b400640 1 -- 172.21.15.19:0/3065080766 shutdown_connections 2024-04-25T11:41:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.522+0000 7f597b400640 1 --2- 172.21.15.19:0/3065080766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f595c003a00 0x7f595c003e90 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.522+0000 7f597b400640 1 -- 172.21.15.19:0/3065080766 >> 172.21.15.19:0/3065080766 conn(0x7f59740d1ae0 msgr2=0x7f59740d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.522+0000 7f597b400640 1 -- 172.21.15.19:0/3065080766 shutdown_connections 2024-04-25T11:41:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.522+0000 7f597b400640 1 -- 172.21.15.19:0/3065080766 wait complete. 2024-04-25T11:41:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.524+0000 7f597b400640 1 Processor -- start 2024-04-25T11:41:54.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.524+0000 7f597b400640 1 -- start start 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f597b400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c003a00 0x7f597415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f597b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5974143420 0x7f59741f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f597b400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59740b5ac0 0x7f59740b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f597b400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59741447d0 con 0x7f5974143420 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f597b400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f597415f160 con 0x7f59740b5ac0 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f597b400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5974162b00 con 0x7f595c003a00 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f5973e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5974143420 0x7f59741f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f5973e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5974143420 0x7f59741f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48004/0 (socket says 172.21.15.19:48004) 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.526+0000 7f5973e00640 1 -- 172.21.15.19:0/1556513237 learned_addr learned my addr 172.21.15.19:0/1556513237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5978a00640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c003a00 0x7f597415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5979400640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59740b5ac0 0x7f59740b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5973e00640 1 -- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c003a00 msgr2=0x7f597415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5973e00640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c003a00 0x7f597415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5973e00640 1 -- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59740b5ac0 msgr2=0x7f59740b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5973e00640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59740b5ac0 0x7f59740b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5973e00640 1 -- 172.21.15.19:0/1556513237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59741f6720 con 0x7f5974143420 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.527+0000 7f5973e00640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5974143420 0x7f59741f61e0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f596000c3c0 tx=0x7f596000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.528+0000 7f5970c00640 1 -- 172.21.15.19:0/1556513237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59600052d0 con 0x7f5974143420 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.528+0000 7f5970c00640 1 -- 172.21.15.19:0/1556513237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5960005490 con 0x7f5974143420 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.528+0000 7f5970c00640 1 -- 172.21.15.19:0/1556513237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960007260 con 0x7f5974143420 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.528+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59740b6410 con 0x7f5974143420 2024-04-25T11:41:54.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.528+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741f6ed0 con 0x7f5974143420 2024-04-25T11:41:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.529+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59741f6960 con 0x7f5974143420 2024-04-25T11:41:54.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.530+0000 7f5970c00640 1 -- 172.21.15.19:0/1556513237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5960007400 con 0x7f5974143420 2024-04-25T11:41:54.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.532+0000 7f5970c00640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.532+0000 7f5970c00640 1 -- 172.21.15.19:0/1556513237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7f5960095390 con 0x7f5974143420 2024-04-25T11:41:54.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.533+0000 7f5978a00640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.534+0000 7f5978a00640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5968033000 tx=0x7f5968008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:54.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.537+0000 7f5970c00640 1 -- 172.21.15.19:0/1556513237 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59740b6410 con 0x7f5974143420 2024-04-25T11:41:54.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.783+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5974050740 con 0x7f5974143420 2024-04-25T11:41:54.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.785+0000 7f5970c00640 1 -- 172.21.15.19:0/1556513237 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 345 v345) ==== 95+0+4352 (secure 0 0 0) 0x7f596005ee70 con 0x7f5974143420 2024-04-25T11:41:54.771 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:54.772 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":345,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:54.227642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:54.772 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 345 2024-04-25T11:41:54.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.787+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 msgr2=0x7f59540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.787+0000 7f597b400640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5968033000 tx=0x7f5968008030 comp rx=0 tx=0).stop 2024-04-25T11:41:54.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.787+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5974143420 msgr2=0x7f59741f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.787+0000 7f597b400640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5974143420 0x7f59741f61e0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f596000c3c0 tx=0x7f596000c880 comp rx=0 tx=0).stop 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 shutdown_connections 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f59540744b0 0x7f59540769c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59740b5ac0 0x7f59740b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5974143420 0x7f59741f61e0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 --2- 172.21.15.19:0/1556513237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f595c003a00 0x7f597415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 >> 172.21.15.19:0/1556513237 conn(0x7f59740d1ae0 msgr2=0x7f59740ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 shutdown_connections 2024-04-25T11:41:54.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:54.788+0000 7f597b400640 1 -- 172.21.15.19:0/1556513237 wait complete. 2024-04-25T11:41:54.804 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:41:54.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:55.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.089+0000 7f52c3800640 1 Processor -- start 2024-04-25T11:41:55.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.090+0000 7f52c3800640 1 -- start start 2024-04-25T11:41:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.093+0000 7f52c3800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52bc160360 0x7f52bc1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.093+0000 7f52c3800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52bc06c7d0 con 0x7f52bc070680 2024-04-25T11:41:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.093+0000 7f52c3800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52bc06cab0 con 0x7f52bc0b5c90 2024-04-25T11:41:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.093+0000 7f52c3800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f52bc06d100 con 0x7f52bc160360 2024-04-25T11:41:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.093+0000 7f52bbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52bc160360 0x7f52bc1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.093+0000 7f52c0e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f52bc070680 0x7f52bc0af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52232/0 (socket says 172.21.15.19:52232) 2024-04-25T11:41:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.093+0000 7f52c0e00640 1 -- 172.21.15.19:0/178655701 learned_addr learned my addr 172.21.15.19:0/178655701 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.094+0000 7f52bbe00640 1 -- 172.21.15.19:0/178655701 >> v1:172.21.15.169:6789/0 conn(0x7f52bc0b5c90 legacy=0x7f52bc0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.094+0000 7f52bbe00640 1 -- 172.21.15.19:0/178655701 >> v1:172.21.15.19:6789/0 conn(0x7f52bc070680 legacy=0x7f52bc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.094+0000 7f52bbe00640 1 -- 172.21.15.19:0/178655701 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc06cd90 con 0x7f52bc160360 2024-04-25T11:41:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.094+0000 7f52bb400640 1 -- 172.21.15.19:0/178655701 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515770616 0 0) 0x7f52bc06c7d0 con 0x7f52bc070680 2024-04-25T11:41:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.095+0000 7f52bbe00640 1 --2- 172.21.15.19:0/178655701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc1647f0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f52bc06d280 tx=0x7f52a802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79523d3ea71eb5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.095+0000 7f52bb400640 1 -- 172.21.15.19:0/178655701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8037070 con 0x7f52bc160360 2024-04-25T11:41:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.095+0000 7f52bb400640 1 -- 172.21.15.19:0/178655701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a802ae40 con 0x7f52bc160360 2024-04-25T11:41:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.095+0000 7f52bb400640 1 -- 172.21.15.19:0/178655701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8032660 con 0x7f52bc160360 2024-04-25T11:41:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.096+0000 7f52c3800640 1 -- 172.21.15.19:0/178655701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 msgr2=0x7f52bc1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.096+0000 7f52c3800640 1 --2- 172.21.15.19:0/178655701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc1647f0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f52bc06d280 tx=0x7f52a802a130 comp rx=0 tx=0).stop 2024-04-25T11:41:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.096+0000 7f52c3800640 1 -- 172.21.15.19:0/178655701 shutdown_connections 2024-04-25T11:41:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.096+0000 7f52c3800640 1 --2- 172.21.15.19:0/178655701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc1647f0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.096+0000 7f52c3800640 1 -- 172.21.15.19:0/178655701 >> 172.21.15.19:0/178655701 conn(0x7f52bc0d1ae0 msgr2=0x7f52bc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:55.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.096+0000 7f52c3800640 1 -- 172.21.15.19:0/178655701 shutdown_connections 2024-04-25T11:41:55.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.096+0000 7f52c3800640 1 -- 172.21.15.19:0/178655701 wait complete. 2024-04-25T11:41:55.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.098+0000 7f52c3800640 1 Processor -- start 2024-04-25T11:41:55.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.099+0000 7f52c3800640 1 -- start start 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.100+0000 7f52c3800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc070680 0x7f52bc1fa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.100+0000 7f52c3800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0b5c90 0x7f52bc147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.100+0000 7f52c3800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.100+0000 7f52c3800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc0afd30 con 0x7f52bc160360 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.100+0000 7f52c3800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc0afbb0 con 0x7f52bc0b5c90 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.100+0000 7f52c3800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc06d590 con 0x7f52bc070680 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc15bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48024/0 (socket says 172.21.15.19:48024) 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 -- 172.21.15.19:0/3167107568 learned_addr learned my addr 172.21.15.19:0/3167107568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52bbe00640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0b5c90 0x7f52bc147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c0e00640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc070680 0x7f52bc1fa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 -- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc070680 msgr2=0x7f52bc1fa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc070680 0x7f52bc1fa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 -- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0b5c90 msgr2=0x7f52bc147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0b5c90 0x7f52bc147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.101+0000 7f52c1800640 1 -- 172.21.15.19:0/3167107568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc148420 con 0x7f52bc160360 2024-04-25T11:41:55.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.102+0000 7f52c1800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc15bfa0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f52b400ba10 tx=0x7f52b400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:55.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.102+0000 7f52b8c00640 1 -- 172.21.15.19:0/3167107568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4019070 con 0x7f52bc160360 2024-04-25T11:41:55.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.102+0000 7f52b8c00640 1 -- 172.21.15.19:0/3167107568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b400cb40 con 0x7f52bc160360 2024-04-25T11:41:55.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.102+0000 7f52b8c00640 1 -- 172.21.15.19:0/3167107568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4014440 con 0x7f52bc160360 2024-04-25T11:41:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.102+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc148700 con 0x7f52bc160360 2024-04-25T11:41:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.102+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1f9eb0 con 0x7f52bc160360 2024-04-25T11:41:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.104+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52bc148170 con 0x7f52bc160360 2024-04-25T11:41:55.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.105+0000 7f52b8c00640 1 -- 172.21.15.19:0/3167107568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f52b401c070 con 0x7f52bc160360 2024-04-25T11:41:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.106+0000 7f52b8c00640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f529c0744b0 0x7f529c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.106+0000 7f52b8c00640 1 -- 172.21.15.19:0/3167107568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7f52b4095bc0 con 0x7f52bc160360 2024-04-25T11:41:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.106+0000 7f52c0e00640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f529c0744b0 0x7f529c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.108+0000 7f52c0e00640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f529c0744b0 0x7f529c0769c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f52b0000f60 tx=0x7f52b0007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:55.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.111+0000 7f52b8c00640 1 -- 172.21.15.19:0/3167107568 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52b405f5e0 con 0x7f52bc160360 2024-04-25T11:41:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.356+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52bc050740 con 0x7f52bc160360 2024-04-25T11:41:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.357+0000 7f52b8c00640 1 -- 172.21.15.19:0/3167107568 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 346 v346) ==== 95+0+5176 (secure 0 0 0) 0x7f52b405eee0 con 0x7f52bc160360 2024-04-25T11:41:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:55.344 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":346,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11121,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/478011367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":478011367},{"type":"v1","addr":"172.21.15.19:6835","nonce":478011367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:54.227642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 346 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.360+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f529c0744b0 msgr2=0x7f529c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.360+0000 7f52c3800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f529c0744b0 0x7f529c0769c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f52b0000f60 tx=0x7f52b0007c00 comp rx=0 tx=0).stop 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.360+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 msgr2=0x7f52bc15bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.360+0000 7f52c3800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc15bfa0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f52b400ba10 tx=0x7f52b400bed0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 shutdown_connections 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f529c0744b0 0x7f529c0769c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52bc0b5c90 0x7f52bc147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52bc160360 0x7f52bc15bfa0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 --2- 172.21.15.19:0/3167107568 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52bc070680 0x7f52bc1fa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 >> 172.21.15.19:0/3167107568 conn(0x7f52bc0d1ae0 msgr2=0x7f52bc0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:55.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 shutdown_connections 2024-04-25T11:41:55.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.361+0000 7f52c3800640 1 -- 172.21.15.19:0/3167107568 wait complete. 2024-04-25T11:41:55.373 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:41:55.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.672+0000 7f1289000640 1 Processor -- start 2024-04-25T11:41:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.672+0000 7f1289000640 1 -- start start 2024-04-25T11:41:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.674+0000 7f1289000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12840b4ff0 0x7f12841611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.674+0000 7f1289000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12840d1f70 0x7f12840d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.674+0000 7f1289000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1284161720 con 0x7f128408cd60 2024-04-25T11:41:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.674+0000 7f1289000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f128406e9d0 con 0x7f12840b4ff0 2024-04-25T11:41:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1289000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f128406ddc0 con 0x7f12840d1f70 2024-04-25T11:41:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12840b4ff0 0x7f12841611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12840b4ff0 0x7f12841611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48032/0 (socket says 172.21.15.19:48032) 2024-04-25T11:41:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283400640 1 -- 172.21.15.19:0/1157590871 learned_addr learned my addr 172.21.15.19:0/1157590871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283e00640 1 --2- 172.21.15.19:0/1157590871 >> v2:172.21.15.169:3300/0 conn(0x7f12840d1f70 0x7f12840d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283400640 1 -- 172.21.15.19:0/1157590871 >> v2:172.21.15.169:3300/0 conn(0x7f12840d1f70 msgr2=0x7f12840d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283400640 1 --2- 172.21.15.19:0/1157590871 >> v2:172.21.15.169:3300/0 conn(0x7f12840d1f70 0x7f12840d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283400640 1 -- 172.21.15.19:0/1157590871 >> v1:172.21.15.19:6789/0 conn(0x7f128408cd60 legacy=0x7f128408d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283400640 1 -- 172.21.15.19:0/1157590871 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128406e0a0 con 0x7f12840b4ff0 2024-04-25T11:41:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.675+0000 7f1283e00640 1 --2- 172.21.15.19:0/1157590871 >> v2:172.21.15.169:3300/0 conn(0x7f12840d1f70 0x7f12840d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.677+0000 7f1283400640 1 --2- 172.21.15.19:0/1157590871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f12841611e0 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f12840b8ec0 tx=0x7f127402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dc23b399ea952a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.677+0000 7f1282a00640 1 -- 172.21.15.19:0/1157590871 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274004030 con 0x7f12840b4ff0 2024-04-25T11:41:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.677+0000 7f1282a00640 1 -- 172.21.15.19:0/1157590871 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12740041d0 con 0x7f12840b4ff0 2024-04-25T11:41:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.677+0000 7f1282a00640 1 -- 172.21.15.19:0/1157590871 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274003610 con 0x7f12840b4ff0 2024-04-25T11:41:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 -- 172.21.15.19:0/1157590871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 msgr2=0x7f12841611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 --2- 172.21.15.19:0/1157590871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f12841611e0 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f12840b8ec0 tx=0x7f127402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 -- 172.21.15.19:0/1157590871 shutdown_connections 2024-04-25T11:41:55.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 --2- 172.21.15.19:0/1157590871 >> v2:172.21.15.169:3300/0 conn(0x7f12840d1f70 0x7f12840d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 --2- 172.21.15.19:0/1157590871 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f12841611e0 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 -- 172.21.15.19:0/1157590871 >> 172.21.15.19:0/1157590871 conn(0x7f12840b5460 msgr2=0x7f12840b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:55.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 -- 172.21.15.19:0/1157590871 shutdown_connections 2024-04-25T11:41:55.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.678+0000 7f1289000640 1 -- 172.21.15.19:0/1157590871 wait complete. 2024-04-25T11:41:55.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.680+0000 7f1289000640 1 Processor -- start 2024-04-25T11:41:55.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.681+0000 7f1289000640 1 -- start start 2024-04-25T11:41:55.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f1289000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128408cd60 0x7f128414b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f1289000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f1284157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f1289000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12840d1f70 0x7f1284161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f1289000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12840b9180 con 0x7f12840b4ff0 2024-04-25T11:41:55.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f1289000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f128406e660 con 0x7f128408cd60 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f1289000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f128406da20 con 0x7f12840d1f70 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f127be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f1284157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f127be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f1284157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48044/0 (socket says 172.21.15.19:48044) 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.682+0000 7f127be00640 1 -- 172.21.15.19:0/3737320772 learned_addr learned my addr 172.21.15.19:0/3737320772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f1283400640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128408cd60 0x7f128414b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127be00640 1 -- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12840d1f70 msgr2=0x7f1284161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127be00640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12840d1f70 0x7f1284161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127be00640 1 -- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128408cd60 msgr2=0x7f128414b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127be00640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128408cd60 0x7f128414b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127be00640 1 -- 172.21.15.19:0/3737320772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1284164230 con 0x7f12840b4ff0 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f1283400640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128408cd60 0x7f128414b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127be00640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f1284157db0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f126c00c3c0 tx=0x7f126c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127b400640 1 -- 172.21.15.19:0/3737320772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c0052d0 con 0x7f12840b4ff0 2024-04-25T11:41:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127b400640 1 -- 172.21.15.19:0/3737320772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f126c005490 con 0x7f12840b4ff0 2024-04-25T11:41:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.683+0000 7f127b400640 1 -- 172.21.15.19:0/3737320772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c007260 con 0x7f12840b4ff0 2024-04-25T11:41:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.684+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1284164510 con 0x7f12840b4ff0 2024-04-25T11:41:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.684+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1284164ad0 con 0x7f12840b4ff0 2024-04-25T11:41:55.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.685+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1284163fa0 con 0x7f12840b4ff0 2024-04-25T11:41:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.686+0000 7f127b400640 1 -- 172.21.15.19:0/3737320772 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f126c007400 con 0x7f12840b4ff0 2024-04-25T11:41:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.687+0000 7f127b400640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744e0 0x7f12600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.687+0000 7f127b400640 1 -- 172.21.15.19:0/3737320772 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7f126c095300 con 0x7f12840b4ff0 2024-04-25T11:41:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.688+0000 7f1283400640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744e0 0x7f12600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.689+0000 7f1283400640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744e0 0x7f12600769f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1274033000 tx=0x7f127402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.692+0000 7f127b400640 1 -- 172.21.15.19:0/3737320772 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1284161720 con 0x7f12840b4ff0 2024-04-25T11:41:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.938+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1284050740 con 0x7f12840b4ff0 2024-04-25T11:41:55.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.939+0000 7f127b400640 1 -- 172.21.15.19:0/3737320772 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 346 v346) ==== 95+0+5176 (secure 0 0 0) 0x7f126c05ee40 con 0x7f12840b4ff0 2024-04-25T11:41:55.926 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:41:55.926 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":346,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11121,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/478011367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":478011367},{"type":"v1","addr":"172.21.15.19:6835","nonce":478011367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":346}],"filesystems":[{"mdsmap":{"epoch":345,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:41:54.227642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:41:55.926 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 346 2024-04-25T11:41:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744e0 msgr2=0x7f12600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744e0 0x7f12600769f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1274033000 tx=0x7f127402e000 comp rx=0 tx=0).stop 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 msgr2=0x7f1284157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f1284157db0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f126c00c3c0 tx=0x7f126c00c880 comp rx=0 tx=0).stop 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 shutdown_connections 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f12600744e0 0x7f12600769f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12840d1f70 0x7f1284161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12840b4ff0 0x7f1284157db0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 --2- 172.21.15.19:0/3737320772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f128408cd60 0x7f128414b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 >> 172.21.15.19:0/3737320772 conn(0x7f12840b5460 msgr2=0x7f12840b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 shutdown_connections 2024-04-25T11:41:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:55.942+0000 7f1289000640 1 -- 172.21.15.19:0/3737320772 wait complete. 2024-04-25T11:41:55.957 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2024-04-25T11:41:55.957 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2024-04-25T11:41:55.957 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:41:55.958 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:41:56.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:41:56.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:41:56.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:41:56.012 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:41:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T11:41:56.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:56.070 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:41:56.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.373+0000 7fe9b1400640 1 Processor -- start 2024-04-25T11:41:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.373+0000 7fe9b1400640 1 -- start start 2024-04-25T11:41:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b1400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9ac15a810 0x7fe9ac0b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9ac1608d0 0x7fe9ac160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b1400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ac06f870 con 0x7fe9ac1608d0 2024-04-25T11:41:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b1400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ac161220 con 0x7fe9ac15a810 2024-04-25T11:41:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac0b4e90 con 0x7fe9ac0cf2f0 2024-04-25T11:41:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b0a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9ac1608d0 0x7fe9ac160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b0a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9ac1608d0 0x7fe9ac160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48060/0 (socket says 172.21.15.19:48060) 2024-04-25T11:41:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b0a00640 1 -- 172.21.15.19:0/2507607252 learned_addr learned my addr 172.21.15.19:0/2507607252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9a3e00640 1 --2- 172.21.15.19:0/2507607252 >> v2:172.21.15.169:3300/0 conn(0x7fe9ac15a810 0x7fe9ac0b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9b0a00640 1 -- 172.21.15.19:0/2507607252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 msgr2=0x7fe9ac0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.376+0000 7fe9abe00640 1 --2- 172.21.15.19:0/2507607252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.377+0000 7fe9b0a00640 1 --2- 172.21.15.19:0/2507607252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac0b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.377+0000 7fe9b0a00640 1 -- 172.21.15.19:0/2507607252 >> v2:172.21.15.169:3300/0 conn(0x7fe9ac15a810 msgr2=0x7fe9ac0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.377+0000 7fe9b0a00640 1 --2- 172.21.15.19:0/2507607252 >> v2:172.21.15.169:3300/0 conn(0x7fe9ac15a810 0x7fe9ac0b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.377+0000 7fe9b0a00640 1 -- 172.21.15.19:0/2507607252 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac0b74d0 con 0x7fe9ac1608d0 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.377+0000 7fe9a3e00640 1 --2- 172.21.15.19:0/2507607252 >> v2:172.21.15.169:3300/0 conn(0x7fe9ac15a810 0x7fe9ac0b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.377+0000 7fe9abe00640 1 --2- 172.21.15.19:0/2507607252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac0b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.378+0000 7fe9b0a00640 1 --2- 172.21.15.19:0/2507607252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac1608d0 0x7fe9ac160ce0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac06e010 tx=0x7fe9a402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c3dab3d987bb65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.378+0000 7fe9ab400640 1 -- 172.21.15.19:0/2507607252 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4036070 con 0x7fe9ac1608d0 2024-04-25T11:41:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.378+0000 7fe9ab400640 1 -- 172.21.15.19:0/2507607252 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a4031600 con 0x7fe9ac1608d0 2024-04-25T11:41:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.378+0000 7fe9ab400640 1 -- 172.21.15.19:0/2507607252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40319d0 con 0x7fe9ac1608d0 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.379+0000 7fe9b1400640 1 -- 172.21.15.19:0/2507607252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac1608d0 msgr2=0x7fe9ac160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.379+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2507607252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac1608d0 0x7fe9ac160ce0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac06e010 tx=0x7fe9a402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.379+0000 7fe9b1400640 1 -- 172.21.15.19:0/2507607252 shutdown_connections 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.379+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2507607252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac1608d0 0x7fe9ac160ce0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.379+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2507607252 >> v2:172.21.15.169:3300/0 conn(0x7fe9ac15a810 0x7fe9ac0b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.379+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2507607252 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac0b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.379+0000 7fe9b1400640 1 -- 172.21.15.19:0/2507607252 >> 172.21.15.19:0/2507607252 conn(0x7fe9ac0d1720 msgr2=0x7fe9ac0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.380+0000 7fe9b1400640 1 -- 172.21.15.19:0/2507607252 shutdown_connections 2024-04-25T11:41:56.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.380+0000 7fe9b1400640 1 -- 172.21.15.19:0/2507607252 wait complete. 2024-04-25T11:41:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.382+0000 7fe9b1400640 1 Processor -- start 2024-04-25T11:41:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.382+0000 7fe9b1400640 1 -- start start 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9b1400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac15a810 0x7fe9ac156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9b1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9ac1608d0 0x7fe9ac1f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9b1400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac06f500 con 0x7fe9ac15a810 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9b1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac06f220 con 0x7fe9ac1608d0 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9b1400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac15ac20 con 0x7fe9ac0cf2f0 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac15a810 0x7fe9ac156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac15a810 0x7fe9ac156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48062/0 (socket says 172.21.15.19:48062) 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.384+0000 7fe9a3e00640 1 -- 172.21.15.19:0/2308225459 learned_addr learned my addr 172.21.15.19:0/2308225459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9abe00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:56.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9b0a00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9ac1608d0 0x7fe9ac1f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a3e00640 1 -- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 msgr2=0x7fe9ac1575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a3e00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac1575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a3e00640 1 -- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9ac1608d0 msgr2=0x7fe9ac1f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a3e00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9ac1608d0 0x7fe9ac1f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:56.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a3e00640 1 -- 172.21.15.19:0/2308225459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac160700 con 0x7fe9ac15a810 2024-04-25T11:41:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a3e00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac15a810 0x7fe9ac156180 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fe99400c820 tx=0x7fe99400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a8c00640 1 -- 172.21.15.19:0/2308225459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994015070 con 0x7fe9ac15a810 2024-04-25T11:41:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a8c00640 1 -- 172.21.15.19:0/2308225459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9940079d0 con 0x7fe9ac15a810 2024-04-25T11:41:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9a8c00640 1 -- 172.21.15.19:0/2308225459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994007d80 con 0x7fe9ac15a810 2024-04-25T11:41:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.385+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac155f10 con 0x7fe9ac15a810 2024-04-25T11:41:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.386+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac155950 con 0x7fe9ac15a810 2024-04-25T11:41:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.387+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac050740 con 0x7fe9ac15a810 2024-04-25T11:41:56.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.388+0000 7fe9a8c00640 1 -- 172.21.15.19:0/2308225459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe994018070 con 0x7fe9ac15a810 2024-04-25T11:41:56.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.389+0000 7fe9a8c00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9880744e0 0x7fe9880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.389+0000 7fe9a8c00640 1 -- 172.21.15.19:0/2308225459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7fe994095e30 con 0x7fe9ac15a810 2024-04-25T11:41:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.390+0000 7fe9abe00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9880744e0 0x7fe9880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:56.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.391+0000 7fe9abe00640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9880744e0 0x7fe9880769f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00a950 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:56.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.397+0000 7fe9a8c00640 1 -- 172.21.15.19:0/2308225459 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe99405f7f0 con 0x7fe9ac15a810 2024-04-25T11:41:56.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:56.645+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe9ac06c770 con 0x7fe9ac15a810 2024-04-25T11:41:57.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.289+0000 7fe9a8c00640 1 -- 172.21.15.19:0/2308225459 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v347) ==== 161+0+0 (secure 0 0 0) 0x7fe99405f0f0 con 0x7fe9ac15a810 2024-04-25T11:41:57.276 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:41:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9880744e0 msgr2=0x7fe9880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9880744e0 0x7fe9880769f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00a950 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac15a810 msgr2=0x7fe9ac156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac15a810 0x7fe9ac156180 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fe99400c820 tx=0x7fe99400cce0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 shutdown_connections 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe9880744e0 0x7fe9880769f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9ac1608d0 0x7fe9ac1f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9ac15a810 0x7fe9ac156180 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 --2- 172.21.15.19:0/2308225459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9ac0cf2f0 0x7fe9ac1575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 >> 172.21.15.19:0/2308225459 conn(0x7fe9ac0d1720 msgr2=0x7fe9ac0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.293+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 shutdown_connections 2024-04-25T11:41:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.294+0000 7fe9b1400640 1 -- 172.21.15.19:0/2308225459 wait complete. 2024-04-25T11:41:57.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-04-25T11:41:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.613+0000 7f386f200640 1 Processor -- start 2024-04-25T11:41:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.613+0000 7f386f200640 1 -- start start 2024-04-25T11:41:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.615+0000 7f386f200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3870160fe0 0x7f387015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.615+0000 7f386f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38700b7220 con 0x7f38700b6e10 2024-04-25T11:41:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.615+0000 7f386f200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38700b74d0 con 0x7f38700cf2f0 2024-04-25T11:41:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.615+0000 7f386f200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f387006f870 con 0x7f3870160fe0 2024-04-25T11:41:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.615+0000 7f386d400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f38700b6e10 0x7f38701608d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52258/0 (socket says 172.21.15.19:52258) 2024-04-25T11:41:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.615+0000 7f386d400640 1 -- 172.21.15.19:0/2448780293 learned_addr learned my addr 172.21.15.19:0/2448780293 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945725850 0 0) 0x7f38700b7220 con 0x7f38700b6e10 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3854003520 con 0x7f38700b6e10 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386e800640 1 --2- 172.21.15.19:0/2448780293 >> v2:172.21.15.169:3300/0 conn(0x7f3870160fe0 0x7f387015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3961854556 0 0) 0x7f3854003520 con 0x7f38700b6e10 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 >> v2:172.21.15.169:3300/0 conn(0x7f3870160fe0 msgr2=0x7f387015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 --2- 172.21.15.19:0/2448780293 >> v2:172.21.15.169:3300/0 conn(0x7f3870160fe0 0x7f387015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 >> v1:172.21.15.169:6789/0 conn(0x7f38700cf2f0 legacy=0x7f38700b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38700b77b0 con 0x7f38700b6e10 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f3858001670 con 0x7f38700b6e10 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 >> v1:172.21.15.19:6789/0 conn(0x7f38700b6e10 legacy=0x7f38701608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38540039b0 0x7f3854003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.616+0000 7f386ca00640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3854004800 0x7f3854004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386ca00640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3854005270 0x7f385400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f387006f870 con 0x7f38540039b0 2024-04-25T11:41:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f385400c200 con 0x7f3854004800 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f385400c400 con 0x7f3854005270 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386de00640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38540039b0 0x7f3854003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386d400640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3854004800 0x7f3854004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386de00640 1 -- 172.21.15.19:0/2448780293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3854005270 msgr2=0x7f385400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386de00640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3854005270 0x7f385400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386de00640 1 -- 172.21.15.19:0/2448780293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3854004800 msgr2=0x7f3854004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386de00640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3854004800 0x7f3854004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386de00640 1 -- 172.21.15.19:0/2448780293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3854003520 con 0x7f38540039b0 2024-04-25T11:41:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.617+0000 7f386e800640 1 -- 172.21.15.19:0/2448780293 reap_dead start 2024-04-25T11:41:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.618+0000 7f386de00640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38540039b0 0x7f3854003e20 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f3860001e50 tx=0x7f3854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c1252b9ddcbb5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.619+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860001a70 con 0x7f38540039b0 2024-04-25T11:41:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.619+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386002ae40 con 0x7f38540039b0 2024-04-25T11:41:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.619+0000 7f386ca00640 1 -- 172.21.15.19:0/2448780293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860003b90 con 0x7f38540039b0 2024-04-25T11:41:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.619+0000 7f386f200640 1 -- 172.21.15.19:0/2448780293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38540039b0 msgr2=0x7f3854003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.619+0000 7f386f200640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38540039b0 0x7f3854003e20 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f3860001e50 tx=0x7f3854005830 comp rx=0 tx=0).stop 2024-04-25T11:41:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.620+0000 7f386f200640 1 -- 172.21.15.19:0/2448780293 shutdown_connections 2024-04-25T11:41:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.620+0000 7f386f200640 1 --2- 172.21.15.19:0/2448780293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38540039b0 0x7f3854003e20 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.620+0000 7f386f200640 1 -- 172.21.15.19:0/2448780293 >> 172.21.15.19:0/2448780293 conn(0x7f38700d1720 msgr2=0x7f38700d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.620+0000 7f386f200640 1 -- 172.21.15.19:0/2448780293 shutdown_connections 2024-04-25T11:41:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.620+0000 7f386f200640 1 -- 172.21.15.19:0/2448780293 wait complete. 2024-04-25T11:41:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.622+0000 7f386f200640 1 Processor -- start 2024-04-25T11:41:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.622+0000 7f386f200640 1 -- start start 2024-04-25T11:41:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386f200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38540039b0 0x7f3870159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387015c220 0x7f387015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386f200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387015fdd0 0x7f38701601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f387015a810 con 0x7f387015c220 2024-04-25T11:41:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386f200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f387006f500 con 0x7f387015fdd0 2024-04-25T11:41:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386f200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f387015a990 con 0x7f38540039b0 2024-04-25T11:41:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387015c220 0x7f387015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387015c220 0x7f387015c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48076/0 (socket says 172.21.15.19:48076) 2024-04-25T11:41:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.624+0000 7f386d400640 1 -- 172.21.15.19:0/22112431 learned_addr learned my addr 172.21.15.19:0/22112431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386e800640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387015fdd0 0x7f38701601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386de00640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38540039b0 0x7f3870159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386d400640 1 -- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38540039b0 msgr2=0x7f3870159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386d400640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38540039b0 0x7f3870159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386d400640 1 -- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387015fdd0 msgr2=0x7f38701601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386d400640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387015fdd0 0x7f38701601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386d400640 1 -- 172.21.15.19:0/22112431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3870160720 con 0x7f387015c220 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386d400640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387015c220 0x7f387015c630 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f385800e8f0 tx=0x7f385800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f3866000640 1 -- 172.21.15.19:0/22112431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385800c8d0 con 0x7f387015c220 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f3866000640 1 -- 172.21.15.19:0/22112431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3858018070 con 0x7f387015c220 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f3866000640 1 -- 172.21.15.19:0/22112431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385800ccb0 con 0x7f387015c220 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.625+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387015adc0 con 0x7f387015c220 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.626+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387015b340 con 0x7f387015c220 2024-04-25T11:41:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.627+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3870050740 con 0x7f387015c220 2024-04-25T11:41:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.628+0000 7f3866000640 1 -- 172.21.15.19:0/22112431 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f385801d460 con 0x7f387015c220 2024-04-25T11:41:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.630+0000 7f3866000640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f384c0744b0 0x7f384c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.630+0000 7f3866000640 1 -- 172.21.15.19:0/22112431 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7f3858095ee0 con 0x7f387015c220 2024-04-25T11:41:57.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.630+0000 7f386de00640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f384c0744b0 0x7f384c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.632+0000 7f386de00640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f384c0744b0 0x7f384c0769c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3860032000 tx=0x7f386002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:57.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.635+0000 7f3866000640 1 -- 172.21.15.19:0/22112431 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3870160720 con 0x7f387015c220 2024-04-25T11:41:57.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:57.911+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f387015cd20 con 0x7f387015c220 2024-04-25T11:41:58.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.303+0000 7f3866000640 1 -- 172.21.15.19:0/22112431 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v348) ==== 101+0+0 (secure 0 0 0) 0x7f385805f8a0 con 0x7f387015c220 2024-04-25T11:41:58.290 INFO:teuthology.orchestra.run.smithi019.stderr:repaired: restoring rank 6:0 2024-04-25T11:41:58.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f384c0744b0 msgr2=0x7f384c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:58.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f384c0744b0 0x7f384c0769c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3860032000 tx=0x7f386002e000 comp rx=0 tx=0).stop 2024-04-25T11:41:58.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387015c220 msgr2=0x7f387015c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387015c220 0x7f387015c630 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f385800e8f0 tx=0x7f385800edb0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 shutdown_connections 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f384c0744b0 0x7f384c0769c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387015fdd0 0x7f38701601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387015c220 0x7f387015c630 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 --2- 172.21.15.19:0/22112431 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38540039b0 0x7f3870159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 >> 172.21.15.19:0/22112431 conn(0x7f38700d1720 msgr2=0x7f3870145a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 shutdown_connections 2024-04-25T11:41:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.307+0000 7f386f200640 1 -- 172.21.15.19:0/22112431 wait complete. 2024-04-25T11:41:58.322 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-04-25T11:41:58.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.519+0000 7fb55ed16f40 1 Processor -- start 2024-04-25T11:41:58.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.519+0000 7fb55ed16f40 1 -- start start 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb55ed16f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 0x55a537ea4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb55ed16f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537dd6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb55ed16f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 0x55a537dd74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb55ed16f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a537e20e70 con 0x55a537dd85f0 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb55ed16f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a537d7ed20 con 0x55a537dd81e0 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb55ed16f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a537e213f0 con 0x55a537eb1fa0 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb557400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537dd6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb557400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537dd6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48102/0 (socket says 172.21.15.19:48102) 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb557400640 1 -- 172.21.15.19:0/1250846984 learned_addr learned my addr 172.21.15.19:0/1250846984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.521+0000 7fb557e00640 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 0x55a537ea4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb558800640 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 0x55a537dd74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb557400640 1 -- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 msgr2=0x55a537dd74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb557400640 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 0x55a537dd74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb557400640 1 -- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 msgr2=0x55a537ea4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb557400640 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 0x55a537ea4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb557400640 1 -- 172.21.15.19:0/1250846984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a537e0a9a0 con 0x55a537dd85f0 2024-04-25T11:41:58.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb557400640 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537dd6f90 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fb54000b240 tx=0x7fb54000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42abb068a0e7c2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb556000640 1 -- 172.21.15.19:0/1250846984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540008070 con 0x55a537dd85f0 2024-04-25T11:41:58.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb556000640 1 -- 172.21.15.19:0/1250846984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54000d500 con 0x55a537dd85f0 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.522+0000 7fb556000640 1 -- 172.21.15.19:0/1250846984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54000d8b0 con 0x55a537dd85f0 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1250846984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 msgr2=0x55a537dd6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537dd6f90 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fb54000b240 tx=0x7fb54000b700 comp rx=0 tx=0).stop 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1250846984 shutdown_connections 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 0x55a537dd74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537dd6f90 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1250846984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 0x55a537ea4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1250846984 >> 172.21.15.19:0/1250846984 conn(0x55a537cfaac0 msgr2=0x55a537eae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1250846984 shutdown_connections 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.523+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1250846984 wait complete. 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.524+0000 7fb55ed16f40 1 Processor -- start 2024-04-25T11:41:58.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.524+0000 7fb55ed16f40 1 -- start start 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb55ed16f40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 0x55a537e9c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb55ed16f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537ea8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb55ed16f40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 0x55a537ea8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb55ed16f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a537ea49a0 con 0x55a537dd85f0 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb55ed16f40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a537ea4c10 con 0x55a537dd81e0 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb55ed16f40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a537ea0900 con 0x55a537eb1fa0 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb557400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537ea8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb557400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537ea8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48104/0 (socket says 172.21.15.19:48104) 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.526+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 learned_addr learned my addr 172.21.15.19:0/1778918854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:58.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb558800640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 0x55a537ea8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 msgr2=0x55a537ea8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb557e00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 0x55a537e9c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb557400640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a537eb1fa0 0x55a537ea8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 msgr2=0x55a537e9c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb557400640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a537dd81e0 0x55a537e9c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a537dd67b0 con 0x55a537dd85f0 2024-04-25T11:41:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb557400640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537ea8100 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fb540000c00 tx=0x7fb540000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb554c00640 1 -- 172.21.15.19:0/1778918854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540020070 con 0x55a537dd85f0 2024-04-25T11:41:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb554c00640 1 -- 172.21.15.19:0/1778918854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54000de50 con 0x55a537dd85f0 2024-04-25T11:41:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb554c00640 1 -- 172.21.15.19:0/1778918854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54001cab0 con 0x55a537dd85f0 2024-04-25T11:41:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a537dd6a90 con 0x55a537dd85f0 2024-04-25T11:41:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.527+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a537dd7050 con 0x55a537dd85f0 2024-04-25T11:41:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.530+0000 7fb554c00640 1 -- 172.21.15.19:0/1778918854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb540023070 con 0x55a537dd85f0 2024-04-25T11:41:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.531+0000 7fb554c00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb534072e90 0x7fb5340753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.531+0000 7fb554c00640 1 -- 172.21.15.19:0/1778918854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7fb540096d30 con 0x55a537dd85f0 2024-04-25T11:41:58.517 INFO:teuthology.orchestra.run.smithi019.stdout:Importing pool 2024-04-25T11:41:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.531+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55a537ea8640 0x55a537ea37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.531+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:1 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537ea3d30 con 0x55a537ea8640 2024-04-25T11:41:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.531+0000 7fb557e00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb534072e90 0x7fb5340753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.532+0000 7fb558800640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55a537ea8640 0x55a537ea37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.533+0000 7fb557e00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb534072e90 0x7fb5340753a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb548056000 tx=0x7fb548007030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.533+0000 7fb558800640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55a537ea8640 0x55a537ea37f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.537+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(1 601.00000000 [create] v131'1190 uv1164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.537+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:2 12.10 12:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f45010 con 0x55a537ea8640 2024-04-25T11:41:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.540+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(2 601.00000000 [delete] v131'1191 uv1191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.540+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:3 12.10 12:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f453e0 con 0x55a537ea8640 2024-04-25T11:41:58.529 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-04-25T11:41:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.543+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(3 601.00000000 [create] v131'1192 uv1192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.543+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 12.10 12:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f457b0 con 0x55a537ea8640 2024-04-25T11:41:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.546+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v131'1193 uv1193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.546+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:5 12.10 12:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537ea9450 con 0x55a537ea8640 2024-04-25T11:41:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.549+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v131'1194 uv1194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.549+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 12.10 12:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f52780 con 0x55a537ea8640 2024-04-25T11:41:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.553+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v131'1195 uv1195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.553+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f52b20 con 0x55a537ea8640 2024-04-25T11:41:58.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.556+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 7 ==== osd_op_reply(7 602.00000000 [create] v131'1196 uv1170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.556+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:8 12.10 12:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f52ec0 con 0x55a537ea8640 2024-04-25T11:41:58.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.560+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 8 ==== osd_op_reply(8 602.00000000 [delete] v131'1197 uv1197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.560+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:9 12.10 12:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f52ec0 con 0x55a537ea8640 2024-04-25T11:41:58.550 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-04-25T11:41:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.563+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 9 ==== osd_op_reply(9 602.00000000 [create] v131'1198 uv1198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.564+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:10 12.10 12:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53ed0 con 0x55a537ea8640 2024-04-25T11:41:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.567+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v131'1199 uv1199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.567+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:11 12.10 12:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53ed0 con 0x55a537ea8640 2024-04-25T11:41:58.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.570+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v131'1200 uv1200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.570+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 12.10 12:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f52ec0 con 0x55a537ea8640 2024-04-25T11:41:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.574+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v131'1201 uv1201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.574+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:13 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f52ec0 con 0x55a537ea8640 2024-04-25T11:41:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.579+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 13 ==== osd_op_reply(13 10000000000.00000000 [create] v131'1202 uv1177 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.579+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:14 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f52ec0 con 0x55a537ea8640 2024-04-25T11:41:58.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.583+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v131'1203 uv1203 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.583+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:15 12.10 12:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f52ec0 con 0x55a537ea8640 2024-04-25T11:41:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.587+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 15 ==== osd_op_reply(15 10000000000.00000000 [create] v131'1204 uv1204 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.573 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-04-25T11:41:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.587+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:16 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53ed0 con 0x55a537ea8640 2024-04-25T11:41:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.591+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v131'1205 uv1205 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.591+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:17 12.10 12:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53ed0 con 0x55a537ea8640 2024-04-25T11:41:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.595+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v131'1206 uv1206 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.596+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:18 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f52ec0 con 0x55a537ea8640 2024-04-25T11:41:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.600+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v131'1207 uv1207 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.600+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:19 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.605+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v131'1208 uv1208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.605+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:20 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.609+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 20 ==== osd_op_reply(20 600.00000000 [create] v131'1209 uv1183 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.609+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:21 12.10 12:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.614+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 21 ==== osd_op_reply(21 600.00000000 [delete] v131'1210 uv1210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.614+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:22 12.10 12:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.604 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-04-25T11:41:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.618+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 22 ==== osd_op_reply(22 600.00000000 [create] v131'1211 uv1211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.618+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:23 12.10 12:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f54640 con 0x55a537ea8640 2024-04-25T11:41:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.623+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v131'1212 uv1212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.623+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:24 12.10 12:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f54640 con 0x55a537ea8640 2024-04-25T11:41:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.627+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v131'1213 uv1213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.627+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:25 12.10 12:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.632+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v131'1214 uv1214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.632+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:26 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.637+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 26 ==== osd_op_reply(26 603.00000000 [create] v131'1215 uv1189 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.637+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:27 12.10 12:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.642+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 27 ==== osd_op_reply(27 603.00000000 [delete] v131'1216 uv1216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.642+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:28 12.10 12:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.632 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-04-25T11:41:58.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.646+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 28 ==== osd_op_reply(28 603.00000000 [create] v131'1217 uv1217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.646+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:29 12.10 12:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f54640 con 0x55a537ea8640 2024-04-25T11:41:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.650+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v131'1218 uv1218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.651+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:30 12.10 12:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f54640 con 0x55a537ea8640 2024-04-25T11:41:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.655+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v131'1219 uv1219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.655+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:31 12.10 12:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f53630 con 0x55a537ea8640 2024-04-25T11:41:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.660+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v131'1220 uv1220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.660+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a537f53b40 0x55a537f55f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.660+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:32 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f564a0 con 0x55a537f53b40 2024-04-25T11:41:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.661+0000 7fb557e00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a537f53b40 0x55a537f55f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.662+0000 7fb557e00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a537f53b40 0x55a537f55f60 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.668+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== osd_op_reply(32 1.00000000.inode [create] v131'867 uv850 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.668+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:33 12.14 12:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f56840 con 0x55a537f53b40 2024-04-25T11:41:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.672+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== osd_op_reply(33 1.00000000.inode [delete] v131'868 uv868 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.673+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:34 12.14 12:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f56be0 con 0x55a537f53b40 2024-04-25T11:41:58.663 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-04-25T11:41:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.677+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 3 ==== osd_op_reply(34 1.00000000.inode [create] v131'869 uv869 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.677+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:35 12.14 12:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f56f80 con 0x55a537f53b40 2024-04-25T11:41:58.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.681+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v131'870 uv870 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.682+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:36 12.14 12:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57320 con 0x55a537f53b40 2024-04-25T11:41:58.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.686+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v131'871 uv871 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.686+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:37 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f576c0 con 0x55a537f53b40 2024-04-25T11:41:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.691+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 6 ==== osd_op_reply(37 200.00000000 [create] v131'872 uv855 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.691+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:38 12.14 12:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f57a60 con 0x55a537f53b40 2024-04-25T11:41:58.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.695+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 7 ==== osd_op_reply(38 200.00000000 [delete] v131'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.696+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:39 12.14 12:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.686 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-04-25T11:41:58.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.700+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 8 ==== osd_op_reply(39 200.00000000 [create] v131'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.700+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:40 12.14 12:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.705+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v131'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.705+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:41 12.14 12:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.710+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v131'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.710+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:42 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.715+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 11 ==== osd_op_reply(42 200.00000001 [create] v131'877 uv860 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.715+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:43 12.14 12:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.720+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 12 ==== osd_op_reply(43 200.00000001 [delete] v131'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.720+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:44 12.14 12:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.711 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-04-25T11:41:58.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.725+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 13 ==== osd_op_reply(44 200.00000001 [create] v131'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.725+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:45 12.14 12:2e2fa760:::200.00000001:head [write 0~18117 in=18117b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5c4f0 con 0x55a537f53b40 2024-04-25T11:41:58.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.730+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 14 ==== osd_op_reply(45 200.00000001 [write 0~18117] v131'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.730+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:46 12.14 12:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5c4f0 con 0x55a537f53b40 2024-04-25T11:41:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.735+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v131'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.735+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:47 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.740+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 16 ==== osd_op_reply(47 606.00000000 [create] v131'882 uv866 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.740+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:48 12.14 12:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.744+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 17 ==== osd_op_reply(48 606.00000000 [delete] v131'883 uv883 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.745+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:49 12.14 12:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.735 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-04-25T11:41:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.749+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 18 ==== osd_op_reply(49 606.00000000 [create] v131'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.749+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:50 12.14 12:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f58e10 con 0x55a537f53b40 2024-04-25T11:41:58.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.753+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v131'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.753+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:51 12.14 12:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f58e10 con 0x55a537f53b40 2024-04-25T11:41:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.758+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v131'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.758+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- osd_op(unknown.0.0:52 12.14 12:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f57e00 con 0x55a537f53b40 2024-04-25T11:41:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.762+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.0 v2:172.21.15.19:6808/1318554626 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v131'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f53b40 2024-04-25T11:41:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.763+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a537f58310 0x55a537f5a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.763+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:53 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5ac70 con 0x55a537f58310 2024-04-25T11:41:58.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.764+0000 7fb557400640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a537f58310 0x55a537f5a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.765+0000 7fb557400640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a537f58310 0x55a537f5a730 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.769+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(53 607.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.769+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:54 12.1c 12:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f5b010 con 0x55a537f58310 2024-04-25T11:41:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.774+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== osd_op_reply(54 607.00000000 [delete] v131'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.774+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:55 12.1c 12:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5b3b0 con 0x55a537f58310 2024-04-25T11:41:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-04-25T11:41:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.778+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 3 ==== osd_op_reply(55 607.00000000 [create] v131'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.778+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:56 12.1c 12:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5c760 con 0x55a537f58310 2024-04-25T11:41:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.782+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.782+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:57 12.1c 12:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5cbe0 con 0x55a537f58310 2024-04-25T11:41:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.786+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.787+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:58 12.1c 12:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5b750 con 0x55a537f58310 2024-04-25T11:41:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.791+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.791+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:59 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5bb40 con 0x55a537ea8640 2024-04-25T11:41:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.796+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 32 ==== osd_op_reply(59 mds0_openfiles.0 [create] v131'176 uv175 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.796+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:60 12.6 12:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f5bb40 con 0x55a537ea8640 2024-04-25T11:41:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.800+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 33 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v131'177 uv177 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.800+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:61 12.6 12:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5bb40 con 0x55a537ea8640 2024-04-25T11:41:58.790 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-04-25T11:41:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.804+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 34 ==== osd_op_reply(61 mds0_openfiles.0 [create] v131'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.804+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:62 12.6 12:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5bb40 con 0x55a537ea8640 2024-04-25T11:41:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.808+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 35 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v131'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.808+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55a537f5d030 0x55a537f5f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.808+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:63 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5f990 con 0x55a537f5d030 2024-04-25T11:41:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.809+0000 7fb558800640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55a537f5d030 0x55a537f5f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.811+0000 7fb558800640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55a537f5d030 0x55a537f5f450 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.815+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(63 608.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.815+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:64 12.e 12:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f5f990 con 0x55a537f5d030 2024-04-25T11:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.820+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== osd_op_reply(64 608.00000000 [delete] v131'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.820+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:65 12.e 12:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5f990 con 0x55a537f5d030 2024-04-25T11:41:58.810 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-04-25T11:41:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.824+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 3 ==== osd_op_reply(65 608.00000000 [create] v131'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.824+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:66 12.e 12:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f609e0 con 0x55a537f5d030 2024-04-25T11:41:58.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.829+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 4 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.829+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:67 12.e 12:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f609a0 con 0x55a537f5d030 2024-04-25T11:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.833+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 5 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.834+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:68 12.e 12:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f5f990 con 0x55a537f5d030 2024-04-25T11:41:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.838+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 6 ==== osd_op_reply(68 608.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.838+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55a537f60060 0x55a537f60490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.838+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:69 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f62e00 con 0x55a537f60060 2024-04-25T11:41:58.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.839+0000 7fb557e00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55a537f60060 0x55a537f60490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.841+0000 7fb557e00640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55a537f60060 0x55a537f60490 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.845+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(69 604.00000000 [create] v131'426 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.845+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:70 12.15 12:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f62e00 con 0x55a537f60060 2024-04-25T11:41:58.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.849+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(70 604.00000000 [delete] v131'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.850+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:71 12.15 12:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f62e00 con 0x55a537f60060 2024-04-25T11:41:58.840 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-04-25T11:41:58.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.854+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(71 604.00000000 [create] v131'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.854+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:72 12.15 12:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f64200 con 0x55a537f60060 2024-04-25T11:41:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.859+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v131'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.859+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:73 12.15 12:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f64200 con 0x55a537f60060 2024-04-25T11:41:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.863+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 5 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v131'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.863+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:74 12.15 12:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f631f0 con 0x55a537f60060 2024-04-25T11:41:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.867+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 6 ==== osd_op_reply(74 604.00000000 [omap-set-header] v131'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.867+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:75 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f631f0 con 0x55a537f60060 2024-04-25T11:41:58.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.871+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 7 ==== osd_op_reply(75 500.00000000 [create] v131'432 uv425 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.871+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:76 12.15 12:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f631f0 con 0x55a537f60060 2024-04-25T11:41:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.876+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 8 ==== osd_op_reply(76 500.00000000 [delete] v131'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.876+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:77 12.15 12:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f631f0 con 0x55a537f60060 2024-04-25T11:41:58.866 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-04-25T11:41:58.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.880+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 9 ==== osd_op_reply(77 500.00000000 [create] v131'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.880+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:78 12.15 12:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f631f0 con 0x55a537f60060 2024-04-25T11:41:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.884+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 10 ==== osd_op_reply(78 500.00000000 [write 0~90] v131'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.885+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:79 12.15 12:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f631f0 con 0x55a537f60060 2024-04-25T11:41:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.889+0000 7fb557e00640 1 -- 172.21.15.19:0/1778918854 <== osd.1 v2:172.21.15.19:6800/856233193 11 ==== osd_op_reply(79 500.00000000 [omap-set-header] v131'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb548063070 con 0x55a537f60060 2024-04-25T11:41:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.889+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a537f63700 0x55a537f65b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.889+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:80 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f661e0 con 0x55a537f63700 2024-04-25T11:41:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.890+0000 7fb557400640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a537f63700 0x55a537f65b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:58.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.891+0000 7fb557400640 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a537f63700 0x55a537f65b20 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.896+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(80 mds_snaptable [create] v131'193 uv192 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:58.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.896+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:81 12.d 12:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f665b0 con 0x55a537f63700 2024-04-25T11:41:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.900+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== osd_op_reply(81 mds_snaptable [delete] v131'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:58.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.901+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:82 12.d 12:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f63700 2024-04-25T11:41:58.891 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-04-25T11:41:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.905+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 3 ==== osd_op_reply(82 mds_snaptable [create] v131'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.905+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:83 12.d 12:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f63700 2024-04-25T11:41:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.910+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 4 ==== osd_op_reply(83 mds_snaptable [write 0~70] v131'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.910+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:84 12.d 12:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f63700 2024-04-25T11:41:58.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.914+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 5 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v131'197 uv197 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:58.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.914+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:85 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537ea8640 2024-04-25T11:41:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.918+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 36 ==== osd_op_reply(85 605.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.918+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:86 12.1d 12:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537ea8640 2024-04-25T11:41:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.923+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 37 ==== osd_op_reply(86 605.00000000 [delete] v131'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.923+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:87 12.1d 12:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537ea8640 2024-04-25T11:41:58.914 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-04-25T11:41:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.927+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 38 ==== osd_op_reply(87 605.00000000 [create] v131'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.927+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:88 12.1d 12:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f679e0 con 0x55a537ea8640 2024-04-25T11:41:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.931+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 39 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.931+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f679e0 con 0x55a537ea8640 2024-04-25T11:41:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.936+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 40 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.936+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:90 12.1d 12:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537ea8640 2024-04-25T11:41:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.940+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.6 v2:172.21.15.169:6824/2510135033 41 ==== osd_op_reply(90 605.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537ea8640 2024-04-25T11:41:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.940+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:91 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.945+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 7 ==== osd_op_reply(91 mds0_inotable [create] v131'386 uv380 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.945+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:92 12.13 12:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.949+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 8 ==== osd_op_reply(92 mds0_inotable [delete] v131'387 uv387 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.949+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:93 12.13 12:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.940 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-04-25T11:41:58.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.954+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 9 ==== osd_op_reply(93 mds0_inotable [create] v131'388 uv388 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.954+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:94 12.13 12:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.958+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v131'389 uv389 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.958+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:95 12.13 12:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.963+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v131'390 uv390 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.963+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:96 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.967+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 12 ==== osd_op_reply(96 100.00000000 [create] v131'391 uv385 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.967+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:97 12.13 12:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.972+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 13 ==== osd_op_reply(97 100.00000000 [delete] v131'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.972+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:98 12.13 12:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.962 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-04-25T11:41:58.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.976+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 14 ==== osd_op_reply(98 100.00000000 [create] v131'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.976+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:99 12.13 12:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f669d0 con 0x55a537f58310 2024-04-25T11:41:58.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.980+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v131'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.980+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:100 12.13 12:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f697e0 con 0x55a537f58310 2024-04-25T11:41:58.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.985+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.5 v2:172.21.15.169:6808/2942667610 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v131'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f58310 2024-04-25T11:41:58.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.985+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:101 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:58.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.989+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 7 ==== osd_op_reply(101 mds0_sessionmap [create] v131'172 uv171 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.989+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:102 12.b 12:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:58.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.993+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 8 ==== osd_op_reply(102 mds0_sessionmap [delete] v131'173 uv173 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.993+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:103 12.b 12:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:58.983 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-04-25T11:41:58.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.997+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 9 ==== osd_op_reply(103 mds0_sessionmap [create] v131'174 uv174 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:58.997+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:104 12.b 12:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:58.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.001+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 10 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v131'175 uv175 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.001+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:105 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.005+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 11 ==== osd_op_reply(105 609.00000000 [create] v131'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.006+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:106 12.7 12:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.010+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 12 ==== osd_op_reply(106 609.00000000 [delete] v131'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.010+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:107 12.7 12:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.014+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 13 ==== osd_op_reply(107 609.00000000 [create] v131'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:59.000 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-04-25T11:41:59.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.014+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:108 12.7 12:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f67e90 con 0x55a537f5d030 2024-04-25T11:41:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.018+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 14 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v131'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.018+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:109 12.7 12:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f67e90 con 0x55a537f5d030 2024-04-25T11:41:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.022+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 15 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v131'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.022+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:110 12.7 12:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f5d030 2024-04-25T11:41:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.027+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 <== osd.7 v2:172.21.15.169:6800/2355270624 16 ==== osd_op_reply(110 609.00000000 [omap-set-header] v131'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb5500075a0 con 0x55a537f5d030 2024-04-25T11:41:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.027+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:111 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.031+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 6 ==== osd_op_reply(111 400.00000000 [create] v131'196 uv195 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.031+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:112 12.f 12:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.036+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 7 ==== osd_op_reply(112 400.00000000 [delete] v131'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.036+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:113 12.f 12:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.040+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 8 ==== osd_op_reply(113 400.00000000 [create] v131'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.026 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-04-25T11:41:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.040+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:114 12.f 12:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.044+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 9 ==== osd_op_reply(114 400.00000000 [write 0~22] v131'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.045+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:115 12.f 12:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.049+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 10 ==== osd_op_reply(115 400.00000000 [omap-set-header] v131'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.049+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:116 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.053+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 11 ==== osd_op_reply(116 100.00000000.inode [create] v131'480 uv472 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.053+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:117 12.1f 12:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.058+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 12 ==== osd_op_reply(117 100.00000000.inode [delete] v131'481 uv481 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.058+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:118 12.1f 12:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.048 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-04-25T11:41:59.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.062+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 13 ==== osd_op_reply(118 100.00000000.inode [create] v131'482 uv482 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.062+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:119 12.1f 12:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.066+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 14 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v131'483 uv483 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.066+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:120 12.1f 12:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.071+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 15 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v131'484 uv484 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.071+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:121 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.075+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 16 ==== osd_op_reply(121 1.00000000 [create] v131'485 uv479 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.075+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:122 12.1f 12:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.079+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 17 ==== osd_op_reply(122 1.00000000 [delete] v131'486 uv486 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.079+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:123 12.1f 12:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.069 INFO:teuthology.orchestra.run.smithi019.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-04-25T11:41:59.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.083+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 18 ==== osd_op_reply(123 1.00000000 [create] v131'487 uv487 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.083+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:124 12.1f 12:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.087+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 19 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v131'488 uv488 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.087+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:125 12.1f 12:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.091+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 20 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v131'489 uv489 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.091+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:126 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f66e80 con 0x55a537f63700 2024-04-25T11:41:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.095+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 21 ==== osd_op_reply(126 1.00000000 [omap-set-header] v131'490 uv490 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.096+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:127 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55a537f67740 con 0x55a537f63700 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.099+0000 7fb557400640 1 -- 172.21.15.19:0/1778918854 <== osd.3 v2:172.21.15.19:6824/2286957460 22 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v131'491 uv491 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb540022030 con 0x55a537f63700 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a537f53b40 msgr2=0x55a537f55f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x55a537f53b40 0x55a537f55f60 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55a537f60060 msgr2=0x55a537f60490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x55a537f60060 0x55a537f60490 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a537f63700 msgr2=0x55a537f65b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55a537f63700 0x55a537f65b20 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a537f58310 msgr2=0x55a537f5a730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55a537f58310 0x55a537f5a730 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb558800640 1 -- 172.21.15.19:0/1778918854 reap_dead start 2024-04-25T11:41:59.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55a537ea8640 msgr2=0x55a537ea37f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55a537ea8640 0x55a537ea37f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55a537f5d030 msgr2=0x55a537f5f450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55a537f5d030 0x55a537f5f450 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb534072e90 msgr2=0x7fb5340753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb534072e90 0x7fb5340753a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb548056000 tx=0x7fb548007030 comp rx=0 tx=0).stop 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 msgr2=0x55a537ea8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.100+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537ea8100 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fb540000c00 tx=0x7fb540000ed0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 shutdown_connections 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55a537f5d030 0x55a537f5f450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55a537ea8640 0x55a537ea37f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb534072e90 0x7fb5340753a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 --2- 172.21.15.19:0/1778918854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a537dd85f0 0x55a537ea8100 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 >> 172.21.15.19:0/1778918854 conn(0x55a537cfaac0 msgr2=0x55a537eac240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 shutdown_connections 2024-04-25T11:41:59.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.101+0000 7fb55ed16f40 1 -- 172.21.15.19:0/1778918854 wait complete. 2024-04-25T11:41:59.118 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T11:41:59.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.369+0000 7fb0a0b0ff40 1 Processor -- start 2024-04-25T11:41:59.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.369+0000 7fb0a0b0ff40 1 -- start start 2024-04-25T11:41:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb0a0b0ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7331e0 0x564a5b800950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb0a0b0ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b7335f0 0x564a5b731560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb0a0b0ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b80cfa0 0x564a5b731aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb0a0b0ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a5b77be70 con 0x564a5b80cfa0 2024-04-25T11:41:59.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb0a0b0ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564a5b6d9d20 con 0x564a5b7331e0 2024-04-25T11:41:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb0a0b0ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564a5b77c3f0 con 0x564a5b7335f0 2024-04-25T11:41:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b80cfa0 0x564a5b731aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b80cfa0 0x564a5b731aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48112/0 (socket says 172.21.15.19:48112) 2024-04-25T11:41:59.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 -- 172.21.15.19:0/3960126532 learned_addr learned my addr 172.21.15.19:0/3960126532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb099200640 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b7335f0 0x564a5b731560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 -- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b7335f0 msgr2=0x564a5b731560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb099c00640 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7331e0 0x564a5b800950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b7335f0 0x564a5b731560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 -- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7331e0 msgr2=0x564a5b800950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7331e0 0x564a5b800950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.371+0000 7fb09a600640 1 -- 172.21.15.19:0/3960126532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a5b809c60 con 0x564a5b80cfa0 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.372+0000 7fb099200640 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b7335f0 0x564a5b731560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.372+0000 7fb09a600640 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b80cfa0 0x564a5b731aa0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fb094025ca0 tx=0x7fb094051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1034f017c65f168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.372+0000 7fb093400640 1 -- 172.21.15.19:0/3960126532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094062070 con 0x564a5b80cfa0 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.372+0000 7fb093400640 1 -- 172.21.15.19:0/3960126532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb094054970 con 0x564a5b80cfa0 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.372+0000 7fb093400640 1 -- 172.21.15.19:0/3960126532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094054d20 con 0x564a5b80cfa0 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/3960126532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b80cfa0 msgr2=0x564a5b731aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b80cfa0 0x564a5b731aa0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7fb094025ca0 tx=0x7fb094051000 comp rx=0 tx=0).stop 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/3960126532 shutdown_connections 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b80cfa0 0x564a5b731aa0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b7335f0 0x564a5b731560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/3960126532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7331e0 0x564a5b800950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/3960126532 >> 172.21.15.19:0/3960126532 conn(0x564a5b655ac0 msgr2=0x564a5b807d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/3960126532 shutdown_connections 2024-04-25T11:41:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.373+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/3960126532 wait complete. 2024-04-25T11:41:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.374+0000 7fb0a0b0ff40 1 Processor -- start 2024-04-25T11:41:59.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.374+0000 7fb0a0b0ff40 1 -- start start 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb0a0b0ff40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b7331e0 0x564a5b7f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb0a0b0ff40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7335f0 0x564a5b89b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb0a0b0ff40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b80cfa0 0x564a5b77fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb0a0b0ff40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a5b8037b0 con 0x564a5b7331e0 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb0a0b0ff40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x564a5b809980 con 0x564a5b7335f0 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb0a0b0ff40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x564a5b7f92a0 con 0x564a5b80cfa0 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb099c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b7331e0 0x564a5b7f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb099c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b7331e0 0x564a5b7f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48114/0 (socket says 172.21.15.19:48114) 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.376+0000 7fb099c00640 1 -- 172.21.15.19:0/268876552 learned_addr learned my addr 172.21.15.19:0/268876552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb099200640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7335f0 0x564a5b89b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb09a600640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b80cfa0 0x564a5b77fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb099c00640 1 -- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b80cfa0 msgr2=0x564a5b77fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb099c00640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x564a5b80cfa0 0x564a5b77fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb099c00640 1 -- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7335f0 msgr2=0x564a5b89b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb099c00640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x564a5b7335f0 0x564a5b89b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb099c00640 1 -- 172.21.15.19:0/268876552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a5b7f8610 con 0x564a5b7331e0 2024-04-25T11:41:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.377+0000 7fb099c00640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b7331e0 0x564a5b7f80d0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00be10 tx=0x7fb08c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.378+0000 7fb092000640 1 -- 172.21.15.19:0/268876552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c019070 con 0x564a5b7331e0 2024-04-25T11:41:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.378+0000 7fb092000640 1 -- 172.21.15.19:0/268876552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb08c007b50 con 0x564a5b7331e0 2024-04-25T11:41:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.378+0000 7fb092000640 1 -- 172.21.15.19:0/268876552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c014410 con 0x564a5b7331e0 2024-04-25T11:41:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.378+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564a5b7f8b20 con 0x564a5b7331e0 2024-04-25T11:41:59.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.378+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a5b8997b0 con 0x564a5b7331e0 2024-04-25T11:41:59.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.380+0000 7fb092000640 1 -- 172.21.15.19:0/268876552 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb08c01c070 con 0x564a5b7331e0 2024-04-25T11:41:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.381+0000 7fb092000640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb07c072e90 0x7fb07c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.381+0000 7fb092000640 1 -- 172.21.15.19:0/268876552 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7fb08c093d10 con 0x564a5b7331e0 2024-04-25T11:41:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.382+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564a5b7ff8d0 0x564a5b7ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.382+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:1 12.1f 12:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x564a5b89b830 con 0x564a5b7ff8d0 2024-04-25T11:41:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.382+0000 7fb099200640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb07c072e90 0x7fb07c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.382+0000 7fb09a600640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564a5b7ff8d0 0x564a5b7ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.383+0000 7fb09a600640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564a5b7ff8d0 0x564a5b7ffce0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:59.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.383+0000 7fb099200640 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb07c072e90 0x7fb07c0753a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x564a5b77a3a0 tx=0x7fb088008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb09a600640 1 -- 172.21.15.19:0/268876552 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v131'492 uv492 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb094064030 con 0x564a5b7ff8d0 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564a5b7ff8d0 msgr2=0x564a5b7ffce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x564a5b7ff8d0 0x564a5b7ffce0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb07c072e90 msgr2=0x7fb07c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb07c072e90 0x7fb07c0753a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x564a5b77a3a0 tx=0x7fb088008040 comp rx=0 tx=0).stop 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b7331e0 msgr2=0x564a5b7f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 --2- 172.21.15.19:0/268876552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a5b7331e0 0x564a5b7f80d0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00be10 tx=0x7fb08c00bf00 comp rx=0 tx=0).stop 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb09a600640 1 -- 172.21.15.19:0/268876552 reap_dead start 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 shutdown_connections 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 >> 172.21.15.19:0/268876552 conn(0x564a5b655ac0 msgr2=0x564a5b803060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 shutdown_connections 2024-04-25T11:41:59.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.386+0000 7fb0a0b0ff40 1 -- 172.21.15.19:0/268876552 wait complete. 2024-04-25T11:41:59.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:41:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.674+0000 7efcad000640 1 Processor -- start 2024-04-25T11:41:59.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.674+0000 7efcad000640 1 -- start start 2024-04-25T11:41:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efcad000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efca8070630 0x7efca80cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efcad000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca8071110 con 0x7efca815adf0 2024-04-25T11:41:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efcad000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca806fb50 con 0x7efca806dce0 2024-04-25T11:41:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efcad000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efca80713f0 con 0x7efca8070630 2024-04-25T11:41:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efca5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efca8070630 0x7efca80cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efca5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efca8070630 0x7efca80cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48116/0 (socket says 172.21.15.19:48116) 2024-04-25T11:41:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efc9d400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efca815adf0 0x7efca806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52284/0 (socket says 172.21.15.19:52284) 2024-04-25T11:41:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.677+0000 7efca5e00640 1 -- 172.21.15.19:0/234438196 learned_addr learned my addr 172.21.15.19:0/234438196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.678+0000 7efca5e00640 1 -- 172.21.15.19:0/234438196 >> v1:172.21.15.169:6789/0 conn(0x7efca806dce0 legacy=0x7efca8160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:41:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.678+0000 7efca5e00640 1 -- 172.21.15.19:0/234438196 >> v1:172.21.15.19:6789/0 conn(0x7efca815adf0 legacy=0x7efca806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.678+0000 7efca5e00640 1 -- 172.21.15.19:0/234438196 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca806fe30 con 0x7efca8070630 2024-04-25T11:41:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.678+0000 7efca5400640 1 -- 172.21.15.19:0/234438196 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358170041 0 0) 0x7efca8071110 con 0x7efca815adf0 2024-04-25T11:41:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.679+0000 7efca5e00640 1 --2- 172.21.15.19:0/234438196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca8070630 0x7efca80cec50 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7efc98009870 tx=0x7efc9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef6b433b1ae10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.679+0000 7efca5400640 1 -- 172.21.15.19:0/234438196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98037070 con 0x7efca8070630 2024-04-25T11:41:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.679+0000 7efca5400640 1 -- 172.21.15.19:0/234438196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc98032440 con 0x7efca8070630 2024-04-25T11:41:59.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.679+0000 7efca5400640 1 -- 172.21.15.19:0/234438196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98032810 con 0x7efca8070630 2024-04-25T11:41:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.680+0000 7efcad000640 1 -- 172.21.15.19:0/234438196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca8070630 msgr2=0x7efca80cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.680+0000 7efcad000640 1 --2- 172.21.15.19:0/234438196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca8070630 0x7efca80cec50 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7efc98009870 tx=0x7efc9802a210 comp rx=0 tx=0).stop 2024-04-25T11:41:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.680+0000 7efcad000640 1 -- 172.21.15.19:0/234438196 shutdown_connections 2024-04-25T11:41:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.680+0000 7efcad000640 1 --2- 172.21.15.19:0/234438196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca8070630 0x7efca80cec50 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.680+0000 7efcad000640 1 -- 172.21.15.19:0/234438196 >> 172.21.15.19:0/234438196 conn(0x7efca8073090 msgr2=0x7efca80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.680+0000 7efcad000640 1 -- 172.21.15.19:0/234438196 shutdown_connections 2024-04-25T11:41:59.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.680+0000 7efcad000640 1 -- 172.21.15.19:0/234438196 wait complete. 2024-04-25T11:41:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.683+0000 7efcad000640 1 Processor -- start 2024-04-25T11:41:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.683+0000 7efcad000640 1 -- start start 2024-04-25T11:41:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.684+0000 7efcad000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca815adf0 0x7efca814b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.684+0000 7efcad000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca8159ab0 0x7efca815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.684+0000 7efcad000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca815f9b0 0x7efca81f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.684+0000 7efcad000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efca8070c80 con 0x7efca815f9b0 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.684+0000 7efcad000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efca80701a0 con 0x7efca815adf0 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.684+0000 7efcad000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efca8070e00 con 0x7efca8159ab0 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca815f9b0 0x7efca81f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca815f9b0 0x7efca81f9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48118/0 (socket says 172.21.15.19:48118) 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 -- 172.21.15.19:0/2388413503 learned_addr learned my addr 172.21.15.19:0/2388413503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efc9d400640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca8159ab0 0x7efca815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca5e00640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca815adf0 0x7efca814b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 -- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca8159ab0 msgr2=0x7efca815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca8159ab0 0x7efca815f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 -- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca815adf0 msgr2=0x7efca814b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca815adf0 0x7efca814b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca6800640 1 -- 172.21.15.19:0/2388413503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca81576a0 con 0x7efca815f9b0 2024-04-25T11:41:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.685+0000 7efca5e00640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca815adf0 0x7efca814b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:59.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.686+0000 7efca6800640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca815f9b0 0x7efca81f9990 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7efca000b410 tx=0x7efca000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:59.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.686+0000 7efc9ea00640 1 -- 172.21.15.19:0/2388413503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0015030 con 0x7efca815f9b0 2024-04-25T11:41:59.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.686+0000 7efc9ea00640 1 -- 172.21.15.19:0/2388413503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca0004020 con 0x7efca815f9b0 2024-04-25T11:41:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.686+0000 7efc9ea00640 1 -- 172.21.15.19:0/2388413503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0004430 con 0x7efca815f9b0 2024-04-25T11:41:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.686+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca8157980 con 0x7efca815f9b0 2024-04-25T11:41:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.686+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca8157f40 con 0x7efca815f9b0 2024-04-25T11:41:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.687+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca80cfe50 con 0x7efca815f9b0 2024-04-25T11:41:59.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.689+0000 7efc9ea00640 1 -- 172.21.15.19:0/2388413503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efca0018070 con 0x7efca815f9b0 2024-04-25T11:41:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.690+0000 7efc9ea00640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc840744b0 0x7efc840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.690+0000 7efc9ea00640 1 -- 172.21.15.19:0/2388413503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7efca0095aa0 con 0x7efca815f9b0 2024-04-25T11:41:59.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.690+0000 7efca5e00640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc840744b0 0x7efc840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:59.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.692+0000 7efca5e00640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc840744b0 0x7efc840769c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efc98034000 tx=0x7efc9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:59.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.695+0000 7efc9ea00640 1 -- 172.21.15.19:0/2388413503 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efca80cfe50 con 0x7efca815f9b0 2024-04-25T11:41:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:41:59.939+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7efca81606c0 con 0x7efca815f9b0 2024-04-25T11:42:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.310+0000 7efc9ea00640 1 -- 172.21.15.19:0/2388413503 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v349) ==== 162+0+0 (secure 0 0 0) 0x7efca005f460 con 0x7efca815f9b0 2024-04-25T11:42:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:42:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc840744b0 msgr2=0x7efc840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc840744b0 0x7efc840769c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efc98034000 tx=0x7efc9802e000 comp rx=0 tx=0).stop 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca815f9b0 msgr2=0x7efca81f9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca815f9b0 0x7efca81f9990 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7efca000b410 tx=0x7efca000b8d0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 shutdown_connections 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efc840744b0 0x7efc840769c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efca815f9b0 0x7efca81f9990 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efca8159ab0 0x7efca815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 --2- 172.21.15.19:0/2388413503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efca815adf0 0x7efca814b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 >> 172.21.15.19:0/2388413503 conn(0x7efca8073090 msgr2=0x7efca80ae640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 shutdown_connections 2024-04-25T11:42:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.315+0000 7efcad000640 1 -- 172.21.15.19:0/2388413503 wait complete. 2024-04-25T11:42:00.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:00.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.618+0000 7f7807a00640 1 Processor -- start 2024-04-25T11:42:00.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.619+0000 7f7807a00640 1 -- start start 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.621+0000 7f7807a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7800160360 0x7f78001647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.621+0000 7f7807a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78000b5c90 0x7f78000b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.621+0000 7f7807a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78000b65e0 con 0x7f7800070680 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.621+0000 7f7807a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f780006d100 con 0x7f78000b5c90 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.621+0000 7f7807a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f780006c7d0 con 0x7f7800160360 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f7805a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78000b5c90 0x7f78000b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f7805a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78000b5c90 0x7f78000b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48134/0 (socket says 172.21.15.19:48134) 2024-04-25T11:42:00.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f7805a00640 1 -- 172.21.15.19:0/2933232267 learned_addr learned my addr 172.21.15.19:0/2933232267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f77ffe00640 1 --2- 172.21.15.19:0/2933232267 >> v2:172.21.15.169:3300/0 conn(0x7f7800160360 0x7f78001647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f7805a00640 1 -- 172.21.15.19:0/2933232267 >> v2:172.21.15.169:3300/0 conn(0x7f7800160360 msgr2=0x7f78001647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f7805a00640 1 --2- 172.21.15.19:0/2933232267 >> v2:172.21.15.169:3300/0 conn(0x7f7800160360 0x7f78001647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f7805a00640 1 -- 172.21.15.19:0/2933232267 >> v1:172.21.15.169:6789/0 conn(0x7f7800070680 legacy=0x7f78000af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f7805a00640 1 -- 172.21.15.19:0/2933232267 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780006cab0 con 0x7f78000b5c90 2024-04-25T11:42:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.622+0000 7f77ffe00640 1 --2- 172.21.15.19:0/2933232267 >> v2:172.21.15.169:3300/0 conn(0x7f7800160360 0x7f78001647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.623+0000 7f7805a00640 1 --2- 172.21.15.19:0/2933232267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78000b5c90 0x7f78000b60a0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f78000b46e0 tx=0x7f77f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8828819a9609dd5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.624+0000 7f77ff400640 1 -- 172.21.15.19:0/2933232267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8037070 con 0x7f78000b5c90 2024-04-25T11:42:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.624+0000 7f77ff400640 1 -- 172.21.15.19:0/2933232267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f8032440 con 0x7f78000b5c90 2024-04-25T11:42:00.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.624+0000 7f77ff400640 1 -- 172.21.15.19:0/2933232267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8032810 con 0x7f78000b5c90 2024-04-25T11:42:00.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 -- 172.21.15.19:0/2933232267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78000b5c90 msgr2=0x7f78000b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 --2- 172.21.15.19:0/2933232267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78000b5c90 0x7f78000b60a0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f78000b46e0 tx=0x7f77f802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 -- 172.21.15.19:0/2933232267 shutdown_connections 2024-04-25T11:42:00.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 --2- 172.21.15.19:0/2933232267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78000b5c90 0x7f78000b60a0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 --2- 172.21.15.19:0/2933232267 >> v2:172.21.15.169:3300/0 conn(0x7f7800160360 0x7f78001647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 -- 172.21.15.19:0/2933232267 >> 172.21.15.19:0/2933232267 conn(0x7f78000d1ae0 msgr2=0x7f78000d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 -- 172.21.15.19:0/2933232267 shutdown_connections 2024-04-25T11:42:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.625+0000 7f7807a00640 1 -- 172.21.15.19:0/2933232267 wait complete. 2024-04-25T11:42:00.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.627+0000 7f7807a00640 1 Processor -- start 2024-04-25T11:42:00.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.627+0000 7f7807a00640 1 -- start start 2024-04-25T11:42:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7807a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7800070680 0x7f78000d0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7807a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000b5c90 0x7f780015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7807a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7800160360 0x7f780014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7807a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78000afbb0 con 0x7f7800070680 2024-04-25T11:42:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7807a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f780006fcc0 con 0x7f7800160360 2024-04-25T11:42:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7807a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f78000afd30 con 0x7f78000b5c90 2024-04-25T11:42:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7805000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7800070680 0x7f78000d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7805000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7800070680 0x7f78000d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48136/0 (socket says 172.21.15.19:48136) 2024-04-25T11:42:00.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.629+0000 7f7805000640 1 -- 172.21.15.19:0/3437967185 learned_addr learned my addr 172.21.15.19:0/3437967185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7800160360 0x7f780014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805000640 1 -- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000b5c90 msgr2=0x7f780015e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f77ffe00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000b5c90 0x7f780015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805000640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000b5c90 0x7f780015e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805000640 1 -- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7800160360 msgr2=0x7f780014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805000640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7800160360 0x7f780014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805000640 1 -- 172.21.15.19:0/3437967185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78000cf0f0 con 0x7f7800070680 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7800160360 0x7f780014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f77ffe00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000b5c90 0x7f780015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7805000640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7800070680 0x7f78000d0790 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b3f0 tx=0x7f77f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f77fcc00640 1 -- 172.21.15.19:0/3437967185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0015020 con 0x7f7800070680 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f77fcc00640 1 -- 172.21.15.19:0/3437967185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f0011830 con 0x7f7800070680 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f77fcc00640 1 -- 172.21.15.19:0/3437967185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011c20 con 0x7f7800070680 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.630+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78000cf3d0 con 0x7f7800070680 2024-04-25T11:42:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.631+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78000cf990 con 0x7f7800070680 2024-04-25T11:42:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.632+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7800050740 con 0x7f7800070680 2024-04-25T11:42:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.633+0000 7f77fcc00640 1 -- 172.21.15.19:0/3437967185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f77f0018020 con 0x7f7800070680 2024-04-25T11:42:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.634+0000 7f77fcc00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77e00744b0 0x7f77e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.634+0000 7f77fcc00640 1 -- 172.21.15.19:0/3437967185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9063+0+0 (secure 0 0 0) 0x7f77f0095c70 con 0x7f7800070680 2024-04-25T11:42:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.635+0000 7f77ffe00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77e00744b0 0x7f77e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.636+0000 7f77ffe00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77e00744b0 0x7f77e00769c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f78000b2210 tx=0x7f77ec0074b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.639+0000 7f77fcc00640 1 -- 172.21.15.19:0/3437967185 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77f001f030 con 0x7f7800070680 2024-04-25T11:42:00.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.885+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7800073260 con 0x7f7800070680 2024-04-25T11:42:00.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.886+0000 7f77fcc00640 1 -- 172.21.15.19:0/3437967185 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 350 v350) ==== 95+0+5187 (secure 0 0 0) 0x7f77f005f630 con 0x7f7800070680 2024-04-25T11:42:00.873 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:00.873 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":350,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":350,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:00.315838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":11121},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11121":{"gid":11121,"name":"a","rank":0,"incarnation":350,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/478011367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":478011367},{"type":"v1","addr":"172.21.15.19:6835","nonce":478011367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:00.873 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 350 2024-04-25T11:42:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.889+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77e00744b0 msgr2=0x7f77e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.889+0000 7f7807a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77e00744b0 0x7f77e00769c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f78000b2210 tx=0x7f77ec0074b0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.889+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7800070680 msgr2=0x7f78000d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.889+0000 7f7807a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7800070680 0x7f78000d0790 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b3f0 tx=0x7f77f000b8b0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 shutdown_connections 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f77e00744b0 0x7f77e00769c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7800160360 0x7f780014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78000b5c90 0x7f780015e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 --2- 172.21.15.19:0/3437967185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7800070680 0x7f78000d0790 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 >> 172.21.15.19:0/3437967185 conn(0x7f78000d1ae0 msgr2=0x7f78000736b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 shutdown_connections 2024-04-25T11:42:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:00.890+0000 7f7807a00640 1 -- 172.21.15.19:0/3437967185 wait complete. 2024-04-25T11:42:00.901 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:42:03.354 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:42:03.368+0000 7f8b3c000640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:42:03.354 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-04-25T11:42:03.368+0000 7f8b3c000640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-25T11:42:03.358 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:42:03.372+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:42:03.358 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:42:03.372+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-04-25T11:42:03.374 INFO:tasks.ceph.mds.a.smithi019.stderr: -23> 2024-04-25T11:42:03.368+0000 7f8b3c000640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 3 < 101: Malformed input [buffer:3] 2024-04-25T11:42:03.374 INFO:tasks.ceph.mds.a.smithi019.stderr: -20> 2024-04-25T11:42:03.368+0000 7f8b3c000640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-04-25T11:42:03.507 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:42:05.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:06.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.216+0000 7f141be00640 1 Processor -- start 2024-04-25T11:42:06.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.216+0000 7f141be00640 1 -- start start 2024-04-25T11:42:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.218+0000 7f141be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f141c070e20 0x7f141c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.218+0000 7f141be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c06f180 0x7f141c0cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.219+0000 7f141be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f141c071230 con 0x7f141c0cf830 2024-04-25T11:42:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.219+0000 7f141be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f141c06f8f0 con 0x7f141c070e20 2024-04-25T11:42:06.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.219+0000 7f141be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f141c06fbd0 con 0x7f141c06f180 2024-04-25T11:42:06.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.219+0000 7f141aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f141c070e20 0x7f141c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141b400640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f141c0cf830 0x7f141c06c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:58812/0 (socket says 172.21.15.19:58812) 2024-04-25T11:42:06.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.219+0000 7f141a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c06f180 0x7f141c0cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141aa00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f141c070e20 0x7f141c15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:38560/0 (socket says 172.21.15.19:38560) 2024-04-25T11:42:06.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141b400640 1 -- 172.21.15.19:0/118342522 learned_addr learned my addr 172.21.15.19:0/118342522 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:06.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141aa00640 1 -- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c06f180 msgr2=0x7f141c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141aa00640 1 --2- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c06f180 0x7f141c0cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141aa00640 1 -- 172.21.15.19:0/118342522 >> v1:172.21.15.169:6789/0 conn(0x7f141c0cf830 legacy=0x7f141c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141aa00640 1 -- 172.21.15.19:0/118342522 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c06feb0 con 0x7f141c070e20 2024-04-25T11:42:06.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.220+0000 7f141a000640 1 --2- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c06f180 0x7f141c0cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:06.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.222+0000 7f141aa00640 1 --2- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c070e20 0x7f141c15c800 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f140c009d90 tx=0x7f140c02c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9fea097efcaef80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:06.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.222+0000 7f1419600640 1 -- 172.21.15.19:0/118342522 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c032770 con 0x7f141c070e20 2024-04-25T11:42:06.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.222+0000 7f1419600640 1 -- 172.21.15.19:0/118342522 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f140c038070 con 0x7f141c070e20 2024-04-25T11:42:06.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.222+0000 7f1419600640 1 -- 172.21.15.19:0/118342522 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c032b30 con 0x7f141c070e20 2024-04-25T11:42:06.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 -- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c070e20 msgr2=0x7f141c15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 --2- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c070e20 0x7f141c15c800 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f140c009d90 tx=0x7f140c02c870 comp rx=0 tx=0).stop 2024-04-25T11:42:06.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 -- 172.21.15.19:0/118342522 shutdown_connections 2024-04-25T11:42:06.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 --2- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c06f180 0x7f141c0cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 --2- 172.21.15.19:0/118342522 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c070e20 0x7f141c15c800 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 -- 172.21.15.19:0/118342522 >> 172.21.15.19:0/118342522 conn(0x7f141c0d1720 msgr2=0x7f141c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:06.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 -- 172.21.15.19:0/118342522 shutdown_connections 2024-04-25T11:42:06.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.223+0000 7f141be00640 1 -- 172.21.15.19:0/118342522 wait complete. 2024-04-25T11:42:06.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.226+0000 7f141be00640 1 Processor -- start 2024-04-25T11:42:06.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.226+0000 7f141be00640 1 -- start start 2024-04-25T11:42:06.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.227+0000 7f141be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c06f180 0x7f141c1501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.227+0000 7f141be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c070e20 0x7f141c15ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.227+0000 7f141be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c0cf830 0x7f141c0aac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.227+0000 7f141be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f141c0cfc40 con 0x7f141c0cf830 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.227+0000 7f141be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f141c06f590 con 0x7f141c06f180 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.227+0000 7f141be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f141c0cfdc0 con 0x7f141c070e20 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c0cf830 0x7f141c0aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c0cf830 0x7f141c0aac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38870/0 (socket says 172.21.15.19:38870) 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 -- 172.21.15.19:0/3526841655 learned_addr learned my addr 172.21.15.19:0/3526841655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141aa00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c06f180 0x7f141c1501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141a000640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c070e20 0x7f141c15ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 -- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c070e20 msgr2=0x7f141c15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c070e20 0x7f141c15ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 -- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c06f180 msgr2=0x7f141c1501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c06f180 0x7f141c1501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.228+0000 7f141b400640 1 -- 172.21.15.19:0/3526841655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c1e5f50 con 0x7f141c0cf830 2024-04-25T11:42:06.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.229+0000 7f141b400640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c0cf830 0x7f141c0aac10 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f141400c3f0 tx=0x7f141400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.229+0000 7f1412a00640 1 -- 172.21.15.19:0/3526841655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414017070 con 0x7f141c0cf830 2024-04-25T11:42:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.229+0000 7f1412a00640 1 -- 172.21.15.19:0/3526841655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1414005ce0 con 0x7f141c0cf830 2024-04-25T11:42:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.229+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c1e6230 con 0x7f141c0cf830 2024-04-25T11:42:06.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.229+0000 7f1412a00640 1 -- 172.21.15.19:0/3526841655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140072f0 con 0x7f141c0cf830 2024-04-25T11:42:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.229+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c1e4790 con 0x7f141c0cf830 2024-04-25T11:42:06.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.231+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141c050740 con 0x7f141c0cf830 2024-04-25T11:42:06.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.232+0000 7f1412a00640 1 -- 172.21.15.19:0/3526841655 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f141401a070 con 0x7f141c0cf830 2024-04-25T11:42:06.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.233+0000 7f1412a00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13fc0744b0 0x7f13fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.233+0000 7f1412a00640 1 -- 172.21.15.19:0/3526841655 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f1414095350 con 0x7f141c0cf830 2024-04-25T11:42:06.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.234+0000 7f141aa00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13fc0744b0 0x7f13fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.235+0000 7f141aa00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13fc0744b0 0x7f13fc0769c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f140c037000 tx=0x7f140c032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:06.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.239+0000 7f1412a00640 1 -- 172.21.15.19:0/3526841655 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f141405ecc0 con 0x7f141c0cf830 2024-04-25T11:42:06.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.493+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f141c1e6400 con 0x7f141c0cf830 2024-04-25T11:42:06.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.494+0000 7f1412a00640 1 -- 172.21.15.19:0/3526841655 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) ==== 95+0+5179 (secure 0 0 0) 0x7f141405e5c0 con 0x7f141c0cf830 2024-04-25T11:42:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:06.480 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":355,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:03.372367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:06.481 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 355 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13fc0744b0 msgr2=0x7f13fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13fc0744b0 0x7f13fc0769c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f140c037000 tx=0x7f140c032000 comp rx=0 tx=0).stop 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c0cf830 msgr2=0x7f141c0aac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c0cf830 0x7f141c0aac10 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f141400c3f0 tx=0x7f141400c8b0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 shutdown_connections 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13fc0744b0 0x7f13fc0769c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f141c0cf830 0x7f141c0aac10 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f141c070e20 0x7f141c15ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 --2- 172.21.15.19:0/3526841655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f141c06f180 0x7f141c1501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 >> 172.21.15.19:0/3526841655 conn(0x7f141c0d1720 msgr2=0x7f141c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 shutdown_connections 2024-04-25T11:42:06.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.497+0000 7f141be00640 1 -- 172.21.15.19:0/3526841655 wait complete. 2024-04-25T11:42:06.511 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:42:06.512 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:06.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.812+0000 7f2ad6000640 1 Processor -- start 2024-04-25T11:42:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.812+0000 7f2ad6000640 1 -- start start 2024-04-25T11:42:06.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2ad6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ad01616a0 0x7f2ad015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2ad6000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ad0163fc0 0x7f2ad01643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2ad6000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad0164c80 con 0x7f2ad006e030 2024-04-25T11:42:06.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2ad6000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad0160980 con 0x7f2ad01616a0 2024-04-25T11:42:06.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2ad6000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad0160bc0 con 0x7f2ad0163fc0 2024-04-25T11:42:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2acf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ad01616a0 0x7f2ad015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2acea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2ad006e030 0x7f2ad006e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36902/0 (socket says 172.21.15.19:36902) 2024-04-25T11:42:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.815+0000 7f2acea00640 1 -- 172.21.15.19:0/1571299010 learned_addr learned my addr 172.21.15.19:0/1571299010 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.816+0000 7f2acfe00640 1 --2- 172.21.15.19:0/1571299010 >> v2:172.21.15.169:3300/0 conn(0x7f2ad0163fc0 0x7f2ad01643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.816+0000 7f2acf400640 1 -- 172.21.15.19:0/1571299010 >> v2:172.21.15.169:3300/0 conn(0x7f2ad0163fc0 msgr2=0x7f2ad01643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.816+0000 7f2acf400640 1 --2- 172.21.15.19:0/1571299010 >> v2:172.21.15.169:3300/0 conn(0x7f2ad0163fc0 0x7f2ad01643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.816+0000 7f2acf400640 1 -- 172.21.15.19:0/1571299010 >> v1:172.21.15.19:6789/0 conn(0x7f2ad006e030 legacy=0x7f2ad006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.816+0000 7f2ace000640 1 -- 172.21.15.19:0/1571299010 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556836660 0 0) 0x7f2ad0164c80 con 0x7f2ad006e030 2024-04-25T11:42:06.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.816+0000 7f2acf400640 1 -- 172.21.15.19:0/1571299010 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad0160ea0 con 0x7f2ad01616a0 2024-04-25T11:42:06.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.817+0000 7f2acf400640 1 --2- 172.21.15.19:0/1571299010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad01616a0 0x7f2ad015ded0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00d23f0 tx=0x7f2ab802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cd6fce4b995849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:06.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.818+0000 7f2ace000640 1 -- 172.21.15.19:0/1571299010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8037070 con 0x7f2ad01616a0 2024-04-25T11:42:06.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.818+0000 7f2ace000640 1 -- 172.21.15.19:0/1571299010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab8032440 con 0x7f2ad01616a0 2024-04-25T11:42:06.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.818+0000 7f2ace000640 1 -- 172.21.15.19:0/1571299010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab802ae60 con 0x7f2ad01616a0 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.818+0000 7f2ad6000640 1 -- 172.21.15.19:0/1571299010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad01616a0 msgr2=0x7f2ad015ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.818+0000 7f2ad6000640 1 --2- 172.21.15.19:0/1571299010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad01616a0 0x7f2ad015ded0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00d23f0 tx=0x7f2ab802a110 comp rx=0 tx=0).stop 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.819+0000 7f2ad6000640 1 -- 172.21.15.19:0/1571299010 shutdown_connections 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.819+0000 7f2ad6000640 1 --2- 172.21.15.19:0/1571299010 >> v2:172.21.15.169:3300/0 conn(0x7f2ad0163fc0 0x7f2ad01643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.819+0000 7f2ad6000640 1 --2- 172.21.15.19:0/1571299010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad01616a0 0x7f2ad015ded0 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.819+0000 7f2ad6000640 1 -- 172.21.15.19:0/1571299010 >> 172.21.15.19:0/1571299010 conn(0x7f2ad00d27c0 msgr2=0x7f2ad00d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.819+0000 7f2ad6000640 1 -- 172.21.15.19:0/1571299010 shutdown_connections 2024-04-25T11:42:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.819+0000 7f2ad6000640 1 -- 172.21.15.19:0/1571299010 wait complete. 2024-04-25T11:42:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.821+0000 7f2ad6000640 1 Processor -- start 2024-04-25T11:42:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.821+0000 7f2ad6000640 1 -- start start 2024-04-25T11:42:06.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.823+0000 7f2ad6000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ad006e030 0x7f2ad015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.823+0000 7f2ad6000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad01616a0 0x7f2ad00cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.823+0000 7f2ad6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad0163fc0 0x7f2ad0155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.823+0000 7f2ad6000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad00b77c0 con 0x7f2ad0163fc0 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.823+0000 7f2ad6000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0164910 con 0x7f2ad01616a0 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.823+0000 7f2ad6000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad00b7940 con 0x7f2ad006e030 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.823+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad0163fc0 0x7f2ad0155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad0163fc0 0x7f2ad0155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38896/0 (socket says 172.21.15.19:38896) 2024-04-25T11:42:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 -- 172.21.15.19:0/867858691 learned_addr learned my addr 172.21.15.19:0/867858691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:06.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acea00640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad01616a0 0x7f2ad00cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acf400640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ad006e030 0x7f2ad015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 -- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ad006e030 msgr2=0x7f2ad015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ad006e030 0x7f2ad015a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 -- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad01616a0 msgr2=0x7f2ad00cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad01616a0 0x7f2ad00cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 -- 172.21.15.19:0/867858691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad015aa70 con 0x7f2ad0163fc0 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.824+0000 7f2acfe00640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad0163fc0 0x7f2ad0155d40 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800d5a0 tx=0x7f2ac800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.825+0000 7f2ac7400640 1 -- 172.21.15.19:0/867858691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80125f0 con 0x7f2ad0163fc0 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.825+0000 7f2ac7400640 1 -- 172.21.15.19:0/867858691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac8019070 con 0x7f2ad0163fc0 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.825+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad00cd780 con 0x7f2ad0163fc0 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.825+0000 7f2ac7400640 1 -- 172.21.15.19:0/867858691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80129b0 con 0x7f2ad0163fc0 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.825+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad00cf2e0 con 0x7f2ad0163fc0 2024-04-25T11:42:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.826+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad015a7e0 con 0x7f2ad0163fc0 2024-04-25T11:42:06.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.827+0000 7f2ac7400640 1 -- 172.21.15.19:0/867858691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ac8012b50 con 0x7f2ad0163fc0 2024-04-25T11:42:06.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.828+0000 7f2ac7400640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2aac0744b0 0x7f2aac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:06.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.828+0000 7f2ac7400640 1 -- 172.21.15.19:0/867858691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f2ac8096370 con 0x7f2ad0163fc0 2024-04-25T11:42:06.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.829+0000 7f2acf400640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2aac0744b0 0x7f2aac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.830+0000 7f2acf400640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2aac0744b0 0x7f2aac0769c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8034000 tx=0x7f2ab802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:06.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:06.833+0000 7f2ac7400640 1 -- 172.21.15.19:0/867858691 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ac805fd50 con 0x7f2ad0163fc0 2024-04-25T11:42:07.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:42:07.044 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:42:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.083+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ad00cf540 con 0x7f2ad0163fc0 2024-04-25T11:42:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.084+0000 7f2ac7400640 1 -- 172.21.15.19:0/867858691 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) ==== 95+0+5179 (secure 0 0 0) 0x7f2ac805f650 con 0x7f2ad0163fc0 2024-04-25T11:42:07.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:07.071 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":355,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:03.372367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:07.071 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 355 2024-04-25T11:42:07.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2aac0744b0 msgr2=0x7f2aac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2aac0744b0 0x7f2aac0769c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8034000 tx=0x7f2ab802e000 comp rx=0 tx=0).stop 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad0163fc0 msgr2=0x7f2ad0155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad0163fc0 0x7f2ad0155d40 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800d5a0 tx=0x7f2ac800da60 comp rx=0 tx=0).stop 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 shutdown_connections 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2aac0744b0 0x7f2aac0769c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ad01616a0 0x7f2ad00cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ad0163fc0 0x7f2ad0155d40 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 --2- 172.21.15.19:0/867858691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ad006e030 0x7f2ad015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 >> 172.21.15.19:0/867858691 conn(0x7f2ad00d27c0 msgr2=0x7f2ad0163310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:07.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 shutdown_connections 2024-04-25T11:42:07.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.087+0000 7f2ad6000640 1 -- 172.21.15.19:0/867858691 wait complete. 2024-04-25T11:42:07.104 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:42:07.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:07.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.442+0000 7fa229a00640 1 Processor -- start 2024-04-25T11:42:07.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.442+0000 7fa229a00640 1 -- start start 2024-04-25T11:42:07.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa22406c5c0 0x7fa2240cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:07.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2240b4ea0 0x7fa22415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:07.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2240d1230 con 0x7fa224144370 2024-04-25T11:42:07.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa2240d06b0 con 0x7fa2240b4ea0 2024-04-25T11:42:07.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa2240d0990 con 0x7fa22406c5c0 2024-04-25T11:42:07.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2240b4ea0 0x7fa22415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:07.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2240b4ea0 0x7fa22415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38908/0 (socket says 172.21.15.19:38908) 2024-04-25T11:42:07.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.445+0000 7fa229000640 1 -- 172.21.15.19:0/4265452489 learned_addr learned my addr 172.21.15.19:0/4265452489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:07.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.446+0000 7fa223e00640 1 --2- 172.21.15.19:0/4265452489 >> v2:172.21.15.169:3300/0 conn(0x7fa22406c5c0 0x7fa2240cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:07.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.446+0000 7fa229000640 1 -- 172.21.15.19:0/4265452489 >> v2:172.21.15.169:3300/0 conn(0x7fa22406c5c0 msgr2=0x7fa2240cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:07.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.446+0000 7fa229000640 1 --2- 172.21.15.19:0/4265452489 >> v2:172.21.15.169:3300/0 conn(0x7fa22406c5c0 0x7fa2240cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.446+0000 7fa229000640 1 -- 172.21.15.19:0/4265452489 >> v1:172.21.15.169:6789/0 conn(0x7fa224144370 legacy=0x7fa2240b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.446+0000 7fa229000640 1 -- 172.21.15.19:0/4265452489 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2240d0c70 con 0x7fa2240b4ea0 2024-04-25T11:42:07.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.447+0000 7fa229000640 1 --2- 172.21.15.19:0/4265452489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2240b4ea0 0x7fa22415e770 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fa2240726b0 tx=0x7fa21c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489cf4474e60429a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.448+0000 7fa222a00640 1 -- 172.21.15.19:0/4265452489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c036070 con 0x7fa2240b4ea0 2024-04-25T11:42:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.448+0000 7fa222a00640 1 -- 172.21.15.19:0/4265452489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c031600 con 0x7fa2240b4ea0 2024-04-25T11:42:07.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.448+0000 7fa222a00640 1 -- 172.21.15.19:0/4265452489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0319d0 con 0x7fa2240b4ea0 2024-04-25T11:42:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 -- 172.21.15.19:0/4265452489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2240b4ea0 msgr2=0x7fa22415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 --2- 172.21.15.19:0/4265452489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2240b4ea0 0x7fa22415e770 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fa2240726b0 tx=0x7fa21c02a730 comp rx=0 tx=0).stop 2024-04-25T11:42:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 -- 172.21.15.19:0/4265452489 shutdown_connections 2024-04-25T11:42:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 --2- 172.21.15.19:0/4265452489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2240b4ea0 0x7fa22415e770 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 --2- 172.21.15.19:0/4265452489 >> v2:172.21.15.169:3300/0 conn(0x7fa22406c5c0 0x7fa2240cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 -- 172.21.15.19:0/4265452489 >> 172.21.15.19:0/4265452489 conn(0x7fa2240d1720 msgr2=0x7fa2240d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:07.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 -- 172.21.15.19:0/4265452489 shutdown_connections 2024-04-25T11:42:07.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.449+0000 7fa229a00640 1 -- 172.21.15.19:0/4265452489 wait complete. 2024-04-25T11:42:07.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.451+0000 7fa229a00640 1 Processor -- start 2024-04-25T11:42:07.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.452+0000 7fa229a00640 1 -- start start 2024-04-25T11:42:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa229a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa22406c5c0 0x7fa2241613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa229a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224144370 0x7fa224143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa229a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa224146060 0x7fa224146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa229a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2240d0450 con 0x7fa22406c5c0 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa229a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa2240d02d0 con 0x7fa224144370 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa229a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa224147760 con 0x7fa224146060 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa223e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa22406c5c0 0x7fa2241613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa223e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa22406c5c0 0x7fa2241613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38918/0 (socket says 172.21.15.19:38918) 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.453+0000 7fa223e00640 1 -- 172.21.15.19:0/3750198312 learned_addr learned my addr 172.21.15.19:0/3750198312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.454+0000 7fa223400640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224144370 0x7fa224143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.454+0000 7fa223e00640 1 -- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa224146060 msgr2=0x7fa224146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.454+0000 7fa223e00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa224146060 0x7fa224146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.454+0000 7fa223e00640 1 -- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224144370 msgr2=0x7fa224143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.454+0000 7fa223e00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224144370 0x7fa224143160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.454+0000 7fa223e00640 1 -- 172.21.15.19:0/3750198312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2241469b0 con 0x7fa22406c5c0 2024-04-25T11:42:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.454+0000 7fa223e00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa22406c5c0 0x7fa2241613c0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d5f0 tx=0x7fa21400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.455+0000 7fa21be00640 1 -- 172.21.15.19:0/3750198312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140125f0 con 0x7fa22406c5c0 2024-04-25T11:42:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.455+0000 7fa21be00640 1 -- 172.21.15.19:0/3750198312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa214019070 con 0x7fa22406c5c0 2024-04-25T11:42:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.455+0000 7fa21be00640 1 -- 172.21.15.19:0/3750198312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140129d0 con 0x7fa22406c5c0 2024-04-25T11:42:07.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.455+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa224161e60 con 0x7fa22406c5c0 2024-04-25T11:42:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.455+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa224162420 con 0x7fa22406c5c0 2024-04-25T11:42:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.456+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa224050740 con 0x7fa22406c5c0 2024-04-25T11:42:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.457+0000 7fa21be00640 1 -- 172.21.15.19:0/3750198312 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa214012b70 con 0x7fa22406c5c0 2024-04-25T11:42:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.459+0000 7fa21be00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2040744e0 0x7fa2040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.459+0000 7fa21be00640 1 -- 172.21.15.19:0/3750198312 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7fa214096430 con 0x7fa22406c5c0 2024-04-25T11:42:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.459+0000 7fa223400640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2040744e0 0x7fa2040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.461+0000 7fa223400640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2040744e0 0x7fa2040769f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa210007fd0 tx=0x7fa210009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:07.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.463+0000 7fa21be00640 1 -- 172.21.15.19:0/3750198312 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2241469b0 con 0x7fa22406c5c0 2024-04-25T11:42:07.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.710+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa22415ee60 con 0x7fa22406c5c0 2024-04-25T11:42:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.711+0000 7fa21be00640 1 -- 172.21.15.19:0/3750198312 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) ==== 95+0+5179 (secure 0 0 0) 0x7fa21405fda0 con 0x7fa22406c5c0 2024-04-25T11:42:07.697 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:07.697 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":355,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:03.372367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:07.698 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 355 2024-04-25T11:42:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.713+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2040744e0 msgr2=0x7fa2040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2040744e0 0x7fa2040769f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa210007fd0 tx=0x7fa210009040 comp rx=0 tx=0).stop 2024-04-25T11:42:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa22406c5c0 msgr2=0x7fa2241613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa22406c5c0 0x7fa2241613c0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fa21400d5f0 tx=0x7fa21400dab0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 shutdown_connections 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa2040744e0 0x7fa2040769f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa224144370 0x7fa224143160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa224146060 0x7fa224146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 --2- 172.21.15.19:0/3750198312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa22406c5c0 0x7fa2241613c0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 >> 172.21.15.19:0/3750198312 conn(0x7fa2240d1720 msgr2=0x7fa2240b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 shutdown_connections 2024-04-25T11:42:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:07.714+0000 7fa229a00640 1 -- 172.21.15.19:0/3750198312 wait complete. 2024-04-25T11:42:07.726 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2024-04-25T11:42:07.726 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2024-04-25T11:42:07.727 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:42:07.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:42:07.843 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:42:07.951 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:42:07.951 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:42:07.951 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:07.951 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:08.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:42:08.012 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:42:08.012 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:42:08.012 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:08.069 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:42:08.069 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:42:08.069 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:08.069 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:08.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:42:08.127 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:42:08.127 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:42:08.127 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:08.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:42:08.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:08.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.489+0000 7fc64a200640 1 Processor -- start 2024-04-25T11:42:08.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.489+0000 7fc64a200640 1 -- start start 2024-04-25T11:42:08.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.491+0000 7fc64a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64406c5c0 0x7fc6440cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:08.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc64a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc644144370 0x7fc6440b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:08.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc64a200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6440b50e0 con 0x7fc6440b4cd0 2024-04-25T11:42:08.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc64a200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc6440d1230 con 0x7fc64406c5c0 2024-04-25T11:42:08.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc64a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc6440d06b0 con 0x7fc644144370 2024-04-25T11:42:08.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc648e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64406c5c0 0x7fc6440cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:08.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc649800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc6440b4cd0 0x7fc644161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36908/0 (socket says 172.21.15.19:36908) 2024-04-25T11:42:08.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc648e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc64406c5c0 0x7fc6440cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38928/0 (socket says 172.21.15.19:38928) 2024-04-25T11:42:08.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc649800640 1 -- 172.21.15.19:0/2316171009 learned_addr learned my addr 172.21.15.19:0/2316171009 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:08.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.492+0000 7fc643e00640 1 --2- 172.21.15.19:0/2316171009 >> v2:172.21.15.169:3300/0 conn(0x7fc644144370 0x7fc6440b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.493+0000 7fc648e00640 1 -- 172.21.15.19:0/2316171009 >> v2:172.21.15.169:3300/0 conn(0x7fc644144370 msgr2=0x7fc6440b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.493+0000 7fc648e00640 1 --2- 172.21.15.19:0/2316171009 >> v2:172.21.15.169:3300/0 conn(0x7fc644144370 0x7fc6440b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.493+0000 7fc648e00640 1 -- 172.21.15.19:0/2316171009 >> v1:172.21.15.19:6789/0 conn(0x7fc6440b4cd0 legacy=0x7fc644161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.493+0000 7fc648e00640 1 -- 172.21.15.19:0/2316171009 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6440d0990 con 0x7fc64406c5c0 2024-04-25T11:42:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.493+0000 7fc643400640 1 -- 172.21.15.19:0/2316171009 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522613200 0 0) 0x7fc63c001490 con 0x7fc6440b4cd0 2024-04-25T11:42:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.494+0000 7fc648e00640 1 --2- 172.21.15.19:0/2316171009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64406c5c0 0x7fc6440cfa70 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc6440b5550 tx=0x7fc63802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f402acfde76386d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:08.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.494+0000 7fc643400640 1 -- 172.21.15.19:0/2316171009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638037070 con 0x7fc64406c5c0 2024-04-25T11:42:08.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.494+0000 7fc643400640 1 -- 172.21.15.19:0/2316171009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc638032440 con 0x7fc64406c5c0 2024-04-25T11:42:08.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.494+0000 7fc643400640 1 -- 172.21.15.19:0/2316171009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638032810 con 0x7fc64406c5c0 2024-04-25T11:42:08.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 -- 172.21.15.19:0/2316171009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64406c5c0 msgr2=0x7fc6440cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:08.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 --2- 172.21.15.19:0/2316171009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64406c5c0 0x7fc6440cfa70 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc6440b5550 tx=0x7fc63802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 -- 172.21.15.19:0/2316171009 shutdown_connections 2024-04-25T11:42:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 --2- 172.21.15.19:0/2316171009 >> v2:172.21.15.169:3300/0 conn(0x7fc644144370 0x7fc6440b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 --2- 172.21.15.19:0/2316171009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc64406c5c0 0x7fc6440cfa70 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 -- 172.21.15.19:0/2316171009 >> 172.21.15.19:0/2316171009 conn(0x7fc6440d1720 msgr2=0x7fc6440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 -- 172.21.15.19:0/2316171009 shutdown_connections 2024-04-25T11:42:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.495+0000 7fc64a200640 1 -- 172.21.15.19:0/2316171009 wait complete. 2024-04-25T11:42:08.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.498+0000 7fc64a200640 1 Processor -- start 2024-04-25T11:42:08.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.498+0000 7fc64a200640 1 -- start start 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.499+0000 7fc64a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc64406c5c0 0x7fc644141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.499+0000 7fc64a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6440b4cd0 0x7fc64415f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc64a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc644144370 0x7fc644162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc64a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440d02d0 con 0x7fc644144370 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc64a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc6440d0f50 con 0x7fc64406c5c0 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc64a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc6440d0450 con 0x7fc6440b4cd0 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc644144370 0x7fc644162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc644144370 0x7fc644162340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38938/0 (socket says 172.21.15.19:38938) 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 -- 172.21.15.19:0/6505940 learned_addr learned my addr 172.21.15.19:0/6505940 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc648e00640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc64406c5c0 0x7fc644141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc643e00640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6440b4cd0 0x7fc64415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 -- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6440b4cd0 msgr2=0x7fc64415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:08.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6440b4cd0 0x7fc64415f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 -- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc64406c5c0 msgr2=0x7fc644141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:08.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc64406c5c0 0x7fc644141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc649800640 1 -- 172.21.15.19:0/6505940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64415fe50 con 0x7fc644144370 2024-04-25T11:42:08.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.500+0000 7fc648e00640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc64406c5c0 0x7fc644141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:08.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.501+0000 7fc649800640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc644144370 0x7fc644162340 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c730 tx=0x7fc63c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:08.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.501+0000 7fc640c00640 1 -- 172.21.15.19:0/6505940 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c018070 con 0x7fc644144370 2024-04-25T11:42:08.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.501+0000 7fc640c00640 1 -- 172.21.15.19:0/6505940 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c00ce40 con 0x7fc644144370 2024-04-25T11:42:08.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.501+0000 7fc640c00640 1 -- 172.21.15.19:0/6505940 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00a880 con 0x7fc644144370 2024-04-25T11:42:08.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.501+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc644162880 con 0x7fc644144370 2024-04-25T11:42:08.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.501+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6440cbbe0 con 0x7fc644144370 2024-04-25T11:42:08.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.502+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6440cb670 con 0x7fc644144370 2024-04-25T11:42:08.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.504+0000 7fc640c00640 1 -- 172.21.15.19:0/6505940 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc63c00aa20 con 0x7fc644144370 2024-04-25T11:42:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.505+0000 7fc640c00640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc6240744e0 0x7fc6240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.505+0000 7fc640c00640 1 -- 172.21.15.19:0/6505940 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7fc63c095610 con 0x7fc644144370 2024-04-25T11:42:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.505+0000 7fc648e00640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc6240744e0 0x7fc6240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:08.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.507+0000 7fc648e00640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc6240744e0 0x7fc6240769f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc638034000 tx=0x7fc63802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:08.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.509+0000 7fc640c00640 1 -- 172.21.15.19:0/6505940 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc6440b50e0 con 0x7fc644144370 2024-04-25T11:42:08.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.753+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc644050740 con 0x7fc644144370 2024-04-25T11:42:08.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.755+0000 7fc640c00640 1 -- 172.21.15.19:0/6505940 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) ==== 95+0+5179 (secure 0 0 0) 0x7fc63c05efc0 con 0x7fc644144370 2024-04-25T11:42:08.741 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:08.741 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":355,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:03.372367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:08.742 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 355 2024-04-25T11:42:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.757+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc6240744e0 msgr2=0x7fc6240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.757+0000 7fc64a200640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc6240744e0 0x7fc6240769f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc638034000 tx=0x7fc63802e000 comp rx=0 tx=0).stop 2024-04-25T11:42:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.757+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc644144370 msgr2=0x7fc644162340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.757+0000 7fc64a200640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc644144370 0x7fc644162340 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00c730 tx=0x7fc63c00cbf0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 shutdown_connections 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc6240744e0 0x7fc6240769f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc644144370 0x7fc644162340 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6440b4cd0 0x7fc64415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 --2- 172.21.15.19:0/6505940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc64406c5c0 0x7fc644141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 >> 172.21.15.19:0/6505940 conn(0x7fc6440d1720 msgr2=0x7fc6440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 shutdown_connections 2024-04-25T11:42:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:08.758+0000 7fc64a200640 1 -- 172.21.15.19:0/6505940 wait complete. 2024-04-25T11:42:08.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:09.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.067+0000 7f726be00640 1 Processor -- start 2024-04-25T11:42:09.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.067+0000 7f726be00640 1 -- start start 2024-04-25T11:42:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.069+0000 7f726be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f726c0b90e0 0x7f726c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.069+0000 7f726be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f726c0d1fa0 con 0x7f726c06e680 2024-04-25T11:42:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.069+0000 7f726be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f726c0d2280 con 0x7f726c08cd60 2024-04-25T11:42:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.069+0000 7f726be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f726c0d1d70 con 0x7f726c0b90e0 2024-04-25T11:42:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.069+0000 7f726aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f726c06e680 0x7f726c0cdfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36916/0 (socket says 172.21.15.19:36916) 2024-04-25T11:42:09.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.069+0000 7f726aa00640 1 -- 172.21.15.19:0/514796851 learned_addr learned my addr 172.21.15.19:0/514796851 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f726a000640 1 --2- 172.21.15.19:0/514796851 >> v2:172.21.15.169:3300/0 conn(0x7f726c0b90e0 0x7f726c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487904339 0 0) 0x7f726c0d1fa0 con 0x7f726c06e680 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7250003520 con 0x7f726c06e680 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 363652679 0 0) 0x7f7250003520 con 0x7f726c06e680 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 >> v2:172.21.15.169:3300/0 conn(0x7f726c0b90e0 msgr2=0x7f726c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 --2- 172.21.15.19:0/514796851 >> v2:172.21.15.169:3300/0 conn(0x7f726c0b90e0 0x7f726c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 >> v1:172.21.15.169:6789/0 conn(0x7f726c08cd60 legacy=0x7f726c08d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c0d2560 con 0x7f726c06e680 2024-04-25T11:42:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f72540017f0 con 0x7f726c06e680 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 >> v1:172.21.15.19:6789/0 conn(0x7f726c06e680 legacy=0x7f726c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250003a00 0x7f7250003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7250004800 0x7f7250004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.070+0000 7f7269600640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72500052a0 0x7f725000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f725000c200 con 0x7f7250004800 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f725000c3b0 con 0x7f7250003a00 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f725000c5b0 con 0x7f72500052a0 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726a000640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7250004800 0x7f7250004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726aa00640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250003a00 0x7f7250003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726a000640 1 -- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72500052a0 msgr2=0x7f725000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726b400640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72500052a0 0x7f725000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726a000640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72500052a0 0x7f725000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726a000640 1 -- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250003a00 msgr2=0x7f7250003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726a000640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250003a00 0x7f7250003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726a000640 1 -- 172.21.15.19:0/514796851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7250003520 con 0x7f7250004800 2024-04-25T11:42:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726aa00640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250003a00 0x7f7250003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:09.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726b400640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f72500052a0 0x7f725000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:09.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.071+0000 7f726b400640 1 -- 172.21.15.19:0/514796851 reap_dead start 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.072+0000 7f726a000640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7250004800 0x7f7250004c10 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f726c074780 tx=0x7f7250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2609d70e87e54002 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.073+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c036070 con 0x7f7250004800 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.073+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725c031400 con 0x7f7250004800 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.073+0000 7f7269600640 1 -- 172.21.15.19:0/514796851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c009690 con 0x7f7250004800 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.073+0000 7f726be00640 1 -- 172.21.15.19:0/514796851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7250004800 msgr2=0x7f7250004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.073+0000 7f726be00640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7250004800 0x7f7250004c10 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f726c074780 tx=0x7f7250005830 comp rx=0 tx=0).stop 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.074+0000 7f726be00640 1 -- 172.21.15.19:0/514796851 shutdown_connections 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.074+0000 7f726be00640 1 --2- 172.21.15.19:0/514796851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7250004800 0x7f7250004c10 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.074+0000 7f726be00640 1 -- 172.21.15.19:0/514796851 >> 172.21.15.19:0/514796851 conn(0x7f726c0b5460 msgr2=0x7f726c0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.074+0000 7f726be00640 1 -- 172.21.15.19:0/514796851 shutdown_connections 2024-04-25T11:42:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.074+0000 7f726be00640 1 -- 172.21.15.19:0/514796851 wait complete. 2024-04-25T11:42:09.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.076+0000 7f726be00640 1 Processor -- start 2024-04-25T11:42:09.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.076+0000 7f726be00640 1 -- start start 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250004800 0x7f726c0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c1630e0 0x7f726c15ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f726c1634f0 0x7f726c15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f726c06ec40 con 0x7f726c1634f0 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f726c0b94f0 con 0x7f7250004800 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f726c08dab0 con 0x7f726c1630e0 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f726c1634f0 0x7f726c15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f726c1634f0 0x7f726c15ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38960/0 (socket says 172.21.15.19:38960) 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726b400640 1 -- 172.21.15.19:0/1951612300 learned_addr learned my addr 172.21.15.19:0/1951612300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726a000640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c1630e0 0x7f726c15ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.078+0000 7f726aa00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250004800 0x7f726c0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f726b400640 1 -- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c1630e0 msgr2=0x7f726c15ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f726b400640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c1630e0 0x7f726c15ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f726b400640 1 -- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250004800 msgr2=0x7f726c0cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f726b400640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250004800 0x7f726c0cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f726b400640 1 -- 172.21.15.19:0/1951612300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c1fa0c0 con 0x7f726c1634f0 2024-04-25T11:42:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f726b400640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f726c1634f0 0x7f726c15ef40 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f726c164f80 tx=0x7f726400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f7262a00640 1 -- 172.21.15.19:0/1951612300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264018070 con 0x7f726c1634f0 2024-04-25T11:42:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f7262a00640 1 -- 172.21.15.19:0/1951612300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7264003b70 con 0x7f726c1634f0 2024-04-25T11:42:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726c1fa3a0 con 0x7f726c1634f0 2024-04-25T11:42:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.079+0000 7f7262a00640 1 -- 172.21.15.19:0/1951612300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264005070 con 0x7f726c1634f0 2024-04-25T11:42:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.080+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726c1fa960 con 0x7f726c1634f0 2024-04-25T11:42:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.081+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726c050740 con 0x7f726c1634f0 2024-04-25T11:42:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.082+0000 7f7262a00640 1 -- 172.21.15.19:0/1951612300 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f726401b070 con 0x7f726c1634f0 2024-04-25T11:42:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.083+0000 7f7262a00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72480744b0 0x7f72480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.083+0000 7f7262a00640 1 -- 172.21.15.19:0/1951612300 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f72640956f0 con 0x7f726c1634f0 2024-04-25T11:42:09.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.084+0000 7f726aa00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72480744b0 0x7f72480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.085+0000 7f726aa00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72480744b0 0x7f72480769c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7254001eb0 tx=0x7f7254009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:09.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.088+0000 7f7262a00640 1 -- 172.21.15.19:0/1951612300 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f726405f060 con 0x7f726c1634f0 2024-04-25T11:42:09.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.292+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f726c068cc0 con 0x7f726c1634f0 2024-04-25T11:42:09.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.296+0000 7f7262a00640 1 -- 172.21.15.19:0/1951612300 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+21842 (secure 0 0 0) 0x7f72640670e0 con 0x7f726c1634f0 2024-04-25T11:42:09.283 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:09.283 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":132,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:03.371686+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6834/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6835/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6834/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6834/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:09.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72480744b0 msgr2=0x7f72480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72480744b0 0x7f72480769c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7254001eb0 tx=0x7f7254009040 comp rx=0 tx=0).stop 2024-04-25T11:42:09.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f726c1634f0 msgr2=0x7f726c15ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f726c1634f0 0x7f726c15ef40 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f726c164f80 tx=0x7f726400dc00 comp rx=0 tx=0).stop 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 shutdown_connections 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f72480744b0 0x7f72480769c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f726c1634f0 0x7f726c15ef40 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f726c1630e0 0x7f726c15ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 --2- 172.21.15.19:0/1951612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7250004800 0x7f726c0cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 >> 172.21.15.19:0/1951612300 conn(0x7f726c0b5460 msgr2=0x7f726c06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 shutdown_connections 2024-04-25T11:42:09.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.299+0000 7f726be00640 1 -- 172.21.15.19:0/1951612300 wait complete. 2024-04-25T11:42:09.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:35:12.150487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:35:12.936871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:42:09.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:42:09.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.615+0000 7f6323000640 1 Processor -- start 2024-04-25T11:42:09.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.615+0000 7f6323000640 1 -- start start 2024-04-25T11:42:09.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.617+0000 7f6323000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f631c0ce520 0x7f631c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.617+0000 7f6323000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c0cda60 con 0x7f631c0ce930 2024-04-25T11:42:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.617+0000 7f6323000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c15dfe0 con 0x7f631c06dce0 2024-04-25T11:42:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.617+0000 7f6323000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f631c15e2c0 con 0x7f631c0ce520 2024-04-25T11:42:09.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.618+0000 7f6321200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f631c0ce930 0x7f631c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36926/0 (socket says 172.21.15.19:36926) 2024-04-25T11:42:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.618+0000 7f6321200640 1 -- 172.21.15.19:0/3440829481 learned_addr learned my addr 172.21.15.19:0/3440829481 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.618+0000 7f6321c00640 1 --2- 172.21.15.19:0/3440829481 >> v2:172.21.15.169:3300/0 conn(0x7f631c0ce520 0x7f631c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.618+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364624203 0 0) 0x7f631c0cda60 con 0x7f631c0ce930 2024-04-25T11:42:09.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.618+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6304003520 con 0x7f631c0ce930 2024-04-25T11:42:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2322514338 0 0) 0x7f6304003520 con 0x7f631c0ce930 2024-04-25T11:42:09.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 >> v2:172.21.15.169:3300/0 conn(0x7f631c0ce520 msgr2=0x7f631c15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 --2- 172.21.15.19:0/3440829481 >> v2:172.21.15.169:3300/0 conn(0x7f631c0ce520 0x7f631c15d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 >> v1:172.21.15.169:6789/0 conn(0x7f631c06dce0 legacy=0x7f631c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c0cdd20 con 0x7f631c0ce930 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f6308001670 con 0x7f631c0ce930 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 >> v1:172.21.15.19:6789/0 conn(0x7f631c0ce930 legacy=0x7f631c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040039b0 0x7f6304003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f6304004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6304005250 0x7f630400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f631c15e2c0 con 0x7f6304004800 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f630400c1e0 con 0x7f63040039b0 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f630400c3e0 con 0x7f6304005250 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.619+0000 7f6321200640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f6304004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6321c00640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040039b0 0x7f6304003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6322600640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6304005250 0x7f630400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6321200640 1 -- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6304005250 msgr2=0x7f630400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:09.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6321200640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6304005250 0x7f630400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6321200640 1 -- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040039b0 msgr2=0x7f6304003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:09.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6321200640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63040039b0 0x7f6304003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6321200640 1 -- 172.21.15.19:0/3440829481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6304003520 con 0x7f6304004800 2024-04-25T11:42:09.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.620+0000 7f6322600640 1 -- 172.21.15.19:0/3440829481 reap_dead start 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.621+0000 7f6321200640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f6304004c10 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f630800c240 tx=0x7f6304005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1264c770e4f04b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.621+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308007d10 con 0x7f6304004800 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.621+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6308005ad0 con 0x7f6304004800 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.621+0000 7f631be00640 1 -- 172.21.15.19:0/3440829481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308007070 con 0x7f6304004800 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.622+0000 7f6323000640 1 -- 172.21.15.19:0/3440829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 msgr2=0x7f6304004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.622+0000 7f6323000640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f6304004c10 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f630800c240 tx=0x7f6304005830 comp rx=0 tx=0).stop 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.622+0000 7f6323000640 1 -- 172.21.15.19:0/3440829481 shutdown_connections 2024-04-25T11:42:09.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.622+0000 7f6323000640 1 --2- 172.21.15.19:0/3440829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f6304004c10 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.622+0000 7f6323000640 1 -- 172.21.15.19:0/3440829481 >> 172.21.15.19:0/3440829481 conn(0x7f631c072f40 msgr2=0x7f631c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:09.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.622+0000 7f6323000640 1 -- 172.21.15.19:0/3440829481 shutdown_connections 2024-04-25T11:42:09.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.622+0000 7f6323000640 1 -- 172.21.15.19:0/3440829481 wait complete. 2024-04-25T11:42:09.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.625+0000 7f6323000640 1 Processor -- start 2024-04-25T11:42:09.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.625+0000 7f6323000640 1 -- start start 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.626+0000 7f6323000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f631c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.626+0000 7f6323000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f631c0cc760 0x7f631c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.626+0000 7f6323000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c15fd80 0x7f631c160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.626+0000 7f6323000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0b73d0 con 0x7f6304004800 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.626+0000 7f6323000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f631c160d30 con 0x7f631c0cc760 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.626+0000 7f6323000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f631c0b7550 con 0x7f631c15fd80 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f631c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f631c14f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38978/0 (socket says 172.21.15.19:38978) 2024-04-25T11:42:09.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 -- 172.21.15.19:0/2882691846 learned_addr learned my addr 172.21.15.19:0/2882691846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6322600640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c15fd80 0x7f631c160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321200640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f631c0cc760 0x7f631c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 -- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c15fd80 msgr2=0x7f631c160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c15fd80 0x7f631c160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 -- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f631c0cc760 msgr2=0x7f631c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f631c0cc760 0x7f631c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321c00640 1 -- 172.21.15.19:0/2882691846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c1606d0 con 0x7f6304004800 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6322600640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c15fd80 0x7f631c160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.627+0000 7f6321200640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f631c0cc760 0x7f631c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.628+0000 7f6321c00640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f631c14f4a0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f63100115f0 tx=0x7f6310011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.628+0000 7f6319600640 1 -- 172.21.15.19:0/2882691846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631000f8f0 con 0x7f6304004800 2024-04-25T11:42:09.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.628+0000 7f6319600640 1 -- 172.21.15.19:0/2882691846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f631000fa90 con 0x7f6304004800 2024-04-25T11:42:09.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.628+0000 7f6319600640 1 -- 172.21.15.19:0/2882691846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631000fe60 con 0x7f6304004800 2024-04-25T11:42:09.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.628+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c15a7d0 con 0x7f6304004800 2024-04-25T11:42:09.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.628+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c0cbf60 con 0x7f6304004800 2024-04-25T11:42:09.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.629+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c076c60 con 0x7f6304004800 2024-04-25T11:42:09.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.631+0000 7f6319600640 1 -- 172.21.15.19:0/2882691846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6310015460 con 0x7f6304004800 2024-04-25T11:42:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.632+0000 7f6319600640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62fc0744e0 0x7f62fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.632+0000 7f6319600640 1 -- 172.21.15.19:0/2882691846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f63100969a0 con 0x7f6304004800 2024-04-25T11:42:09.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.632+0000 7f6321200640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62fc0744e0 0x7f62fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:09.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.634+0000 7f6321200640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62fc0744e0 0x7f62fc0769f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6308033000 tx=0x7f630802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:09.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.637+0000 7f6319600640 1 -- 172.21.15.19:0/2882691846 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f631c1606d0 con 0x7f6304004800 2024-04-25T11:42:09.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.841+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f631c1550f0 con 0x7f6304004800 2024-04-25T11:42:09.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.842+0000 7f6319600640 1 -- 172.21.15.19:0/2882691846 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) ==== 93+0+10 (secure 0 0 0) 0x7f6310019030 con 0x7f6304004800 2024-04-25T11:42:09.828 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:42:09.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62fc0744e0 msgr2=0x7f62fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62fc0744e0 0x7f62fc0769f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6308033000 tx=0x7f630802e000 comp rx=0 tx=0).stop 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 msgr2=0x7f631c14f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f631c14f4a0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f63100115f0 tx=0x7f6310011ab0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 shutdown_connections 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f62fc0744e0 0x7f62fc0769f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f631c0cc760 0x7f631c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f631c15fd80 0x7f631c160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 --2- 172.21.15.19:0/2882691846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6304004800 0x7f631c14f4a0 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 >> 172.21.15.19:0/2882691846 conn(0x7f631c072f40 msgr2=0x7f631c06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 shutdown_connections 2024-04-25T11:42:09.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:09.845+0000 7f6323000640 1 -- 172.21.15.19:0/2882691846 wait complete. 2024-04-25T11:42:09.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:42:10.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.160+0000 7f2fdf200640 1 Processor -- start 2024-04-25T11:42:10.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.160+0000 7f2fdf200640 1 -- start start 2024-04-25T11:42:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.162+0000 7f2fdf200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2fd80b4c60 0x7f2fd80d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.162+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.162+0000 7f2fdf200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd80b40b0 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.162+0000 7f2fdf200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd8160a70 con 0x7f2fd80b4c60 2024-04-25T11:42:10.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.162+0000 7f2fdf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80b4390 con 0x7f2fd8164d70 2024-04-25T11:42:10.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.163+0000 7f2fdd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.163+0000 7f2fd7400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2fd80b4c60 0x7f2fd80d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.163+0000 7f2fd7e00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f2fd80b3bd0 0x7f2fd80b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:58848/0 (socket says 172.21.15.19:58848) 2024-04-25T11:42:10.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.163+0000 7f2fdd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd80cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:57268/0 (socket says 172.21.15.19:57268) 2024-04-25T11:42:10.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.163+0000 7f2fdd200640 1 -- 172.21.15.19:0/583180337 learned_addr learned my addr 172.21.15.19:0/583180337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.164+0000 7f2fd7400640 1 -- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 msgr2=0x7f2fd80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.164+0000 7f2fd7400640 1 --2- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd80cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.164+0000 7f2fd7400640 1 -- 172.21.15.19:0/583180337 >> v1:172.21.15.169:6789/0 conn(0x7f2fd80b3bd0 legacy=0x7f2fd80b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.164+0000 7f2fd7400640 1 -- 172.21.15.19:0/583180337 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd8164290 con 0x7f2fd80b4c60 2024-04-25T11:42:10.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.164+0000 7f2fdd200640 1 --2- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:10.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.165+0000 7f2fd7400640 1 --2- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd80b4c60 0x7f2fd80d2420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00097a0 tx=0x7f2fc002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f34707322b3a7621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.166+0000 7f2fd6a00640 1 -- 172.21.15.19:0/583180337 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0037070 con 0x7f2fd80b4c60 2024-04-25T11:42:10.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.166+0000 7f2fd6a00640 1 -- 172.21.15.19:0/583180337 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc0032440 con 0x7f2fd80b4c60 2024-04-25T11:42:10.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.166+0000 7f2fd6a00640 1 -- 172.21.15.19:0/583180337 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0032810 con 0x7f2fd80b4c60 2024-04-25T11:42:10.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 -- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd80b4c60 msgr2=0x7f2fd80d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 --2- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd80b4c60 0x7f2fd80d2420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00097a0 tx=0x7f2fc002a180 comp rx=0 tx=0).stop 2024-04-25T11:42:10.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 -- 172.21.15.19:0/583180337 shutdown_connections 2024-04-25T11:42:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 --2- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 --2- 172.21.15.19:0/583180337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd80b4c60 0x7f2fd80d2420 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 -- 172.21.15.19:0/583180337 >> 172.21.15.19:0/583180337 conn(0x7f2fd809b7d0 msgr2=0x7f2fd809bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 -- 172.21.15.19:0/583180337 shutdown_connections 2024-04-25T11:42:10.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.167+0000 7f2fdf200640 1 -- 172.21.15.19:0/583180337 wait complete. 2024-04-25T11:42:10.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.170+0000 7f2fdf200640 1 Processor -- start 2024-04-25T11:42:10.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.170+0000 7f2fdf200640 1 -- start start 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.171+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fd80b3bd0 0x7f2fd814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.171+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd814e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.171+0000 7f2fdf200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd814e590 0x7f2fd814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.171+0000 7f2fdf200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd80cedb0 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.171+0000 7f2fdf200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8164850 con 0x7f2fd814e590 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.171+0000 7f2fdf200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80cd070 con 0x7f2fd8164d70 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fd80b3bd0 0x7f2fd814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fd80b3bd0 0x7f2fd814cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38980/0 (socket says 172.21.15.19:38980) 2024-04-25T11:42:10.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 -- 172.21.15.19:0/2464142581 learned_addr learned my addr 172.21.15.19:0/2464142581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7400640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd814e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fdd200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd814e590 0x7f2fd814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 -- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 msgr2=0x7f2fd814e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd814e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 -- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd814e590 msgr2=0x7f2fd814e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd814e590 0x7f2fd814e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fd7e00640 1 -- 172.21.15.19:0/2464142581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd814eee0 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.172+0000 7f2fdd200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd814e590 0x7f2fd814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.173+0000 7f2fd7e00640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fd80b3bd0 0x7f2fd814cba0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c3c0 tx=0x7f2fc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.173+0000 7f2fcfe00640 1 -- 172.21.15.19:0/2464142581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8017070 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.173+0000 7f2fcfe00640 1 -- 172.21.15.19:0/2464142581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc8005c70 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.173+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd814d0e0 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.173+0000 7f2fcfe00640 1 -- 172.21.15.19:0/2464142581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80072f0 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.173+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd814d570 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.174+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd80d33e0 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.176+0000 7f2fcfe00640 1 -- 172.21.15.19:0/2464142581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fc801a070 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.177+0000 7f2fcfe00640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2fb40744e0 0x7f2fb40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.177+0000 7f2fcfe00640 1 -- 172.21.15.19:0/2464142581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f2fc8095350 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.178+0000 7f2fd7400640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2fb40744e0 0x7f2fb40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.179+0000 7f2fd7400640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2fb40744e0 0x7f2fb40769f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0034000 tx=0x7f2fc002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.183+0000 7f2fcfe00640 1 -- 172.21.15.19:0/2464142581 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fd80d33e0 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.386+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2fd8050740 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.388+0000 7f2fcfe00640 1 -- 172.21.15.19:0/2464142581 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) ==== 104+0+11 (secure 0 0 0) 0x7f2fc8005e20 con 0x7f2fd80b3bd0 2024-04-25T11:42:10.374 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:42:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.390+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2fb40744e0 msgr2=0x7f2fb40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.390+0000 7f2fdf200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2fb40744e0 0x7f2fb40769f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0034000 tx=0x7f2fc002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.390+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fd80b3bd0 msgr2=0x7f2fd814cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.390+0000 7f2fdf200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fd80b3bd0 0x7f2fd814cba0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c3c0 tx=0x7f2fc800c880 comp rx=0 tx=0).stop 2024-04-25T11:42:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 shutdown_connections 2024-04-25T11:42:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2fb40744e0 0x7f2fb40769f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fd814e590 0x7f2fd814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2fd8164d70 0x7f2fd814e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 --2- 172.21.15.19:0/2464142581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fd80b3bd0 0x7f2fd814cba0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 >> 172.21.15.19:0/2464142581 conn(0x7f2fd809b7d0 msgr2=0x7f2fd80b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 shutdown_connections 2024-04-25T11:42:10.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.391+0000 7f2fdf200640 1 -- 172.21.15.19:0/2464142581 wait complete. 2024-04-25T11:42:10.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:42:10.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.705+0000 7f478be00640 1 Processor -- start 2024-04-25T11:42:10.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.705+0000 7f478be00640 1 -- start start 2024-04-25T11:42:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f478c0b06c0 0x7f478c0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c070a00 0x7f478c0aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f478c070240 con 0x7f478c06d5d0 2024-04-25T11:42:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f478c070570 con 0x7f478c0b06c0 2024-04-25T11:42:10.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f478c0b0ad0 con 0x7f478c070a00 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f478c0b06c0 0x7f478c0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f478c0b06c0 0x7f478c0d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38990/0 (socket says 172.21.15.19:38990) 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478aa00640 1 -- 172.21.15.19:0/230018829 learned_addr learned my addr 172.21.15.19:0/230018829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.708+0000 7f478b400640 1 --1- 172.21.15.19:0/230018829 >> v1:172.21.15.19:6789/0 conn(0x7f478c06d5d0 0x7f478c06d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36936/0 (socket says 172.21.15.19:36936) 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.709+0000 7f4783e00640 1 --2- 172.21.15.19:0/230018829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c070a00 0x7f478c0aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.709+0000 7f478aa00640 1 -- 172.21.15.19:0/230018829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c070a00 msgr2=0x7f478c0aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.709+0000 7f478aa00640 1 --2- 172.21.15.19:0/230018829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c070a00 0x7f478c0aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.709+0000 7f478aa00640 1 -- 172.21.15.19:0/230018829 >> v1:172.21.15.19:6789/0 conn(0x7f478c06d5d0 legacy=0x7f478c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.709+0000 7f478aa00640 1 -- 172.21.15.19:0/230018829 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c0b0d80 con 0x7f478c0b06c0 2024-04-25T11:42:10.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.709+0000 7f4783e00640 1 --2- 172.21.15.19:0/230018829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c070a00 0x7f478c0aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.710+0000 7f478aa00640 1 --2- 172.21.15.19:0/230018829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c0b06c0 0x7f478c0d0a30 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f477c00bb10 tx=0x7f477c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501a6a842c25aeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.710+0000 7f478a000640 1 -- 172.21.15.19:0/230018829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c036020 con 0x7f478c0b06c0 2024-04-25T11:42:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.710+0000 7f478a000640 1 -- 172.21.15.19:0/230018829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c032820 con 0x7f478c0b06c0 2024-04-25T11:42:10.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.710+0000 7f478a000640 1 -- 172.21.15.19:0/230018829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c032bf0 con 0x7f478c0b06c0 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 -- 172.21.15.19:0/230018829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c0b06c0 msgr2=0x7f478c0d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 --2- 172.21.15.19:0/230018829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c0b06c0 0x7f478c0d0a30 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f477c00bb10 tx=0x7f477c02c920 comp rx=0 tx=0).stop 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 -- 172.21.15.19:0/230018829 shutdown_connections 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 --2- 172.21.15.19:0/230018829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c070a00 0x7f478c0aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 --2- 172.21.15.19:0/230018829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c0b06c0 0x7f478c0d0a30 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 -- 172.21.15.19:0/230018829 >> 172.21.15.19:0/230018829 conn(0x7f478c073160 msgr2=0x7f478c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 -- 172.21.15.19:0/230018829 shutdown_connections 2024-04-25T11:42:10.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.711+0000 7f478be00640 1 -- 172.21.15.19:0/230018829 wait complete. 2024-04-25T11:42:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.714+0000 7f478be00640 1 Processor -- start 2024-04-25T11:42:10.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.714+0000 7f478be00640 1 -- start start 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.715+0000 7f478be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f478c06d5d0 0x7f478c15ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.715+0000 7f478be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c070a00 0x7f478c14a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.715+0000 7f478be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c0b06c0 0x7f478c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.715+0000 7f478be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0af100 con 0x7f478c070a00 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.715+0000 7f478be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0b1340 con 0x7f478c06d5d0 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.715+0000 7f478be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f478c0af280 con 0x7f478c0b06c0 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c070a00 0x7f478c14a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c070a00 0x7f478c14a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38998/0 (socket says 172.21.15.19:38998) 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 -- 172.21.15.19:0/3477726606 learned_addr learned my addr 172.21.15.19:0/3477726606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f478aa00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f478c06d5d0 0x7f478c15ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f478b400640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c0b06c0 0x7f478c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 -- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c0b06c0 msgr2=0x7f478c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c0b06c0 0x7f478c1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 -- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f478c06d5d0 msgr2=0x7f478c15ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f478c06d5d0 0x7f478c15ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.716+0000 7f4783e00640 1 -- 172.21.15.19:0/3477726606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c15e0a0 con 0x7f478c070a00 2024-04-25T11:42:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.717+0000 7f4783e00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c070a00 0x7f478c14a140 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f477400c7f0 tx=0x7f477400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.717+0000 7f4782a00640 1 -- 172.21.15.19:0/3477726606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774015070 con 0x7f478c070a00 2024-04-25T11:42:10.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.717+0000 7f4782a00640 1 -- 172.21.15.19:0/3477726606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47740077c0 con 0x7f478c070a00 2024-04-25T11:42:10.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.717+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478c14f780 con 0x7f478c070a00 2024-04-25T11:42:10.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.717+0000 7f4782a00640 1 -- 172.21.15.19:0/3477726606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774007b70 con 0x7f478c070a00 2024-04-25T11:42:10.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.717+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478c15e340 con 0x7f478c070a00 2024-04-25T11:42:10.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.718+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f478c153d70 con 0x7f478c070a00 2024-04-25T11:42:10.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.720+0000 7f4782a00640 1 -- 172.21.15.19:0/3477726606 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4774018070 con 0x7f478c070a00 2024-04-25T11:42:10.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.721+0000 7f4782a00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47680744b0 0x7f47680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.722+0000 7f4782a00640 1 -- 172.21.15.19:0/3477726606 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f4774096100 con 0x7f478c070a00 2024-04-25T11:42:10.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.722+0000 7f478aa00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47680744b0 0x7f47680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.723+0000 7f478aa00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47680744b0 0x7f47680769c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f477c037000 tx=0x7f477c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.726+0000 7f4782a00640 1 -- 172.21.15.19:0/3477726606 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f477405fa70 con 0x7f478c070a00 2024-04-25T11:42:10.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.933+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f478c06a390 con 0x7f478c070a00 2024-04-25T11:42:10.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.934+0000 7f4782a00640 1 -- 172.21.15.19:0/3477726606 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) ==== 100+0+11 (secure 0 0 0) 0x7f478c06a390 con 0x7f478c070a00 2024-04-25T11:42:10.920 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:42:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.937+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47680744b0 msgr2=0x7f47680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.937+0000 7f478be00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47680744b0 0x7f47680769c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f477c037000 tx=0x7f477c008040 comp rx=0 tx=0).stop 2024-04-25T11:42:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.937+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c070a00 msgr2=0x7f478c14a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.937+0000 7f478be00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c070a00 0x7f478c14a140 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f477400c7f0 tx=0x7f477400ccb0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 shutdown_connections 2024-04-25T11:42:10.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f47680744b0 0x7f47680769c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f478c0b06c0 0x7f478c1554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f478c070a00 0x7f478c14a140 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 --2- 172.21.15.19:0/3477726606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f478c06d5d0 0x7f478c15ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 >> 172.21.15.19:0/3477726606 conn(0x7f478c073160 msgr2=0x7f478c06d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 shutdown_connections 2024-04-25T11:42:10.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:10.938+0000 7f478be00640 1 -- 172.21.15.19:0/3477726606 wait complete. 2024-04-25T11:42:10.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.250+0000 7fcf1fe00640 1 Processor -- start 2024-04-25T11:42:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.250+0000 7fcf1fe00640 1 -- start start 2024-04-25T11:42:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf1fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2006c5c0 0x7fcf200cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf1fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf200b4ea0 0x7fcf2015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf1fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf200d1230 con 0x7fcf20144370 2024-04-25T11:42:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf1fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcf200d06b0 con 0x7fcf200b4ea0 2024-04-25T11:42:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf1fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf200d0990 con 0x7fcf2006c5c0 2024-04-25T11:42:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf1f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf200b4ea0 0x7fcf2015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf16000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcf20144370 0x7fcf200b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36950/0 (socket says 172.21.15.19:36950) 2024-04-25T11:42:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.253+0000 7fcf16000640 1 -- 172.21.15.19:0/2258602627 learned_addr learned my addr 172.21.15.19:0/2258602627 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.254+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2258602627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2006c5c0 0x7fcf200cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.254+0000 7fcf1f400640 1 -- 172.21.15.19:0/2258602627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2006c5c0 msgr2=0x7fcf200cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.254+0000 7fcf1f400640 1 --2- 172.21.15.19:0/2258602627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2006c5c0 0x7fcf200cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.254+0000 7fcf1f400640 1 -- 172.21.15.19:0/2258602627 >> v1:172.21.15.19:6789/0 conn(0x7fcf20144370 legacy=0x7fcf200b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.254+0000 7fcf1f400640 1 -- 172.21.15.19:0/2258602627 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf200d0c70 con 0x7fcf200b4ea0 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.254+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2258602627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2006c5c0 0x7fcf200cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.255+0000 7fcf1f400640 1 --2- 172.21.15.19:0/2258602627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf200b4ea0 0x7fcf2015e770 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009d90 tx=0x7fcf1802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fca257201998165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.255+0000 7fcf1e000640 1 -- 172.21.15.19:0/2258602627 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18007740 con 0x7fcf200b4ea0 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.255+0000 7fcf1e000640 1 -- 172.21.15.19:0/2258602627 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf180078e0 con 0x7fcf200b4ea0 2024-04-25T11:42:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.255+0000 7fcf1e000640 1 -- 172.21.15.19:0/2258602627 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18007cb0 con 0x7fcf200b4ea0 2024-04-25T11:42:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2258602627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf200b4ea0 msgr2=0x7fcf2015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2258602627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf200b4ea0 0x7fcf2015e770 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009d90 tx=0x7fcf1802a660 comp rx=0 tx=0).stop 2024-04-25T11:42:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2258602627 shutdown_connections 2024-04-25T11:42:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2258602627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf200b4ea0 0x7fcf2015e770 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2258602627 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf2006c5c0 0x7fcf200cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2258602627 >> 172.21.15.19:0/2258602627 conn(0x7fcf200d1720 msgr2=0x7fcf200d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2258602627 shutdown_connections 2024-04-25T11:42:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.256+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2258602627 wait complete. 2024-04-25T11:42:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.259+0000 7fcf1fe00640 1 Processor -- start 2024-04-25T11:42:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.259+0000 7fcf1fe00640 1 -- start start 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.260+0000 7fcf1fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2006c5c0 0x7fcf20136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.260+0000 7fcf1fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf20144370 0x7fcf20146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.260+0000 7fcf1fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf201601a0 0x7fcf201605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.260+0000 7fcf1fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf200d0450 con 0x7fcf201601a0 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.260+0000 7fcf1fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf200d02d0 con 0x7fcf2006c5c0 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.260+0000 7fcf1fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf20147760 con 0x7fcf20144370 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf201601a0 0x7fcf201605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf201601a0 0x7fcf201605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39020/0 (socket says 172.21.15.19:39020) 2024-04-25T11:42:11.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 -- 172.21.15.19:0/2520443178 learned_addr learned my addr 172.21.15.19:0/2520443178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:11.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2006c5c0 0x7fcf20136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 -- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf20144370 msgr2=0x7fcf20146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf20144370 0x7fcf20146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 -- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2006c5c0 msgr2=0x7fcf20136bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2006c5c0 0x7fcf20136bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1f400640 1 -- 172.21.15.19:0/2520443178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf20144d60 con 0x7fcf201601a0 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.261+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2006c5c0 0x7fcf20136bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.262+0000 7fcf1f400640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf201601a0 0x7fcf201605b0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009eb0 tx=0x7fcf18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.262+0000 7fcf17400640 1 -- 172.21.15.19:0/2520443178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1803f070 con 0x7fcf201601a0 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.262+0000 7fcf17400640 1 -- 172.21.15.19:0/2520443178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf18031c00 con 0x7fcf201601a0 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.262+0000 7fcf17400640 1 -- 172.21.15.19:0/2520443178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18044460 con 0x7fcf201601a0 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.262+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf201451d0 con 0x7fcf201601a0 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.262+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf20145790 con 0x7fcf201601a0 2024-04-25T11:42:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.263+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf200b61c0 con 0x7fcf201601a0 2024-04-25T11:42:11.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.265+0000 7fcf17400640 1 -- 172.21.15.19:0/2520443178 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf18044600 con 0x7fcf201601a0 2024-04-25T11:42:11.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.266+0000 7fcf17400640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcefc0744e0 0x7fcefc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.266+0000 7fcf17400640 1 -- 172.21.15.19:0/2520443178 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7fcf180b7920 con 0x7fcf201601a0 2024-04-25T11:42:11.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.267+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcefc0744e0 0x7fcefc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.268+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcefc0744e0 0x7fcefc0769f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fcf10006fd0 tx=0x7fcf10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.271+0000 7fcf17400640 1 -- 172.21.15.19:0/2520443178 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf200b61c0 con 0x7fcf201601a0 2024-04-25T11:42:11.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.518+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf20144f30 con 0x7fcf201601a0 2024-04-25T11:42:11.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.519+0000 7fcf17400640 1 -- 172.21.15.19:0/2520443178 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) ==== 95+0+5179 (secure 0 0 0) 0x7fcf18081290 con 0x7fcf201601a0 2024-04-25T11:42:11.507 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:11.507 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":355,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:03.372367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:11.507 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 355 2024-04-25T11:42:11.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcefc0744e0 msgr2=0x7fcefc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcefc0744e0 0x7fcefc0769f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fcf10006fd0 tx=0x7fcf10008040 comp rx=0 tx=0).stop 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf201601a0 msgr2=0x7fcf201605b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf201601a0 0x7fcf201605b0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009eb0 tx=0x7fcf18004a40 comp rx=0 tx=0).stop 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 shutdown_connections 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcefc0744e0 0x7fcefc0769f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf201601a0 0x7fcf201605b0 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf20144370 0x7fcf20146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 --2- 172.21.15.19:0/2520443178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf2006c5c0 0x7fcf20136bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 >> 172.21.15.19:0/2520443178 conn(0x7fcf200d1720 msgr2=0x7fcf200d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 shutdown_connections 2024-04-25T11:42:11.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.522+0000 7fcf1fe00640 1 -- 172.21.15.19:0/2520443178 wait complete. 2024-04-25T11:42:11.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.821+0000 7feead000640 1 Processor -- start 2024-04-25T11:42:11.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.821+0000 7feead000640 1 -- start start 2024-04-25T11:42:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.823+0000 7feead000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feea80b4ea0 0x7feea80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.823+0000 7feead000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea80d1230 con 0x7feea8144370 2024-04-25T11:42:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.823+0000 7feead000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea80d06b0 con 0x7feea806c5c0 2024-04-25T11:42:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.823+0000 7feead000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7feea80d0990 con 0x7feea80b4ea0 2024-04-25T11:42:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.824+0000 7feea6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feea80b4ea0 0x7feea80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.824+0000 7feea7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feea8144370 0x7feea815e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36962/0 (socket says 172.21.15.19:36962) 2024-04-25T11:42:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.824+0000 7feea6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feea80b4ea0 0x7feea80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39024/0 (socket says 172.21.15.19:39024) 2024-04-25T11:42:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.824+0000 7feea7e00640 1 -- 172.21.15.19:0/307313924 learned_addr learned my addr 172.21.15.19:0/307313924 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.824+0000 7feea6a00640 1 -- 172.21.15.19:0/307313924 >> v1:172.21.15.169:6789/0 conn(0x7feea806c5c0 legacy=0x7feea80b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.824+0000 7feea6a00640 1 -- 172.21.15.19:0/307313924 >> v1:172.21.15.19:6789/0 conn(0x7feea8144370 legacy=0x7feea815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.824+0000 7feea6a00640 1 -- 172.21.15.19:0/307313924 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea80d0c70 con 0x7feea80b4ea0 2024-04-25T11:42:11.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.825+0000 7feea6a00640 1 --2- 172.21.15.19:0/307313924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea80b4ea0 0x7feea80cfa70 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fee940097a0 tx=0x7fee9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b4e9e551fc7111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feea6000640 1 -- 172.21.15.19:0/307313924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94037070 con 0x7feea80b4ea0 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feea6000640 1 -- 172.21.15.19:0/307313924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee94032440 con 0x7feea80b4ea0 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feea6000640 1 -- 172.21.15.19:0/307313924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94032810 con 0x7feea80b4ea0 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feead000640 1 -- 172.21.15.19:0/307313924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea80b4ea0 msgr2=0x7feea80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feead000640 1 --2- 172.21.15.19:0/307313924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea80b4ea0 0x7feea80cfa70 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fee940097a0 tx=0x7fee9402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feead000640 1 -- 172.21.15.19:0/307313924 shutdown_connections 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feead000640 1 --2- 172.21.15.19:0/307313924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea80b4ea0 0x7feea80cfa70 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feead000640 1 -- 172.21.15.19:0/307313924 >> 172.21.15.19:0/307313924 conn(0x7feea80d1720 msgr2=0x7feea80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.826+0000 7feead000640 1 -- 172.21.15.19:0/307313924 shutdown_connections 2024-04-25T11:42:11.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.827+0000 7feead000640 1 -- 172.21.15.19:0/307313924 wait complete. 2024-04-25T11:42:11.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.829+0000 7feead000640 1 Processor -- start 2024-04-25T11:42:11.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.829+0000 7feead000640 1 -- start start 2024-04-25T11:42:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.830+0000 7feead000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea806c5c0 0x7feea80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.830+0000 7feead000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea8144370 0x7feea8137590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.830+0000 7feead000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea81f5c90 0x7feea81457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.830+0000 7feead000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feea80d0450 con 0x7feea81f5c90 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.830+0000 7feead000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feea80d02d0 con 0x7feea8144370 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.830+0000 7feead000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7feea8147760 con 0x7feea806c5c0 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea81f5c90 0x7feea81457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea81f5c90 0x7feea81457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39040/0 (socket says 172.21.15.19:39040) 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 -- 172.21.15.19:0/3811796557 learned_addr learned my addr 172.21.15.19:0/3811796557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea6a00640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea8144370 0x7feea8137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7400640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea806c5c0 0x7feea80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 -- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea806c5c0 msgr2=0x7feea80cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea806c5c0 0x7feea80cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 -- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea8144370 msgr2=0x7feea8137590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea8144370 0x7feea8137590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea7e00640 1 -- 172.21.15.19:0/3811796557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea81f62e0 con 0x7feea81f5c90 2024-04-25T11:42:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.831+0000 7feea6a00640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea8144370 0x7feea8137590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.832+0000 7feea7e00640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea81f5c90 0x7feea81457d0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7feea000c3a0 tx=0x7feea000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.832+0000 7fee9f400640 1 -- 172.21.15.19:0/3811796557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0017070 con 0x7feea81f5c90 2024-04-25T11:42:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.832+0000 7fee9f400640 1 -- 172.21.15.19:0/3811796557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea0005ce0 con 0x7feea81f5c90 2024-04-25T11:42:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.832+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea81f65c0 con 0x7feea81f5c90 2024-04-25T11:42:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.832+0000 7fee9f400640 1 -- 172.21.15.19:0/3811796557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea00072d0 con 0x7feea81f5c90 2024-04-25T11:42:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.832+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea81f6b80 con 0x7feea81f5c90 2024-04-25T11:42:11.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.834+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea8050740 con 0x7feea81f5c90 2024-04-25T11:42:11.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.835+0000 7fee9f400640 1 -- 172.21.15.19:0/3811796557 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feea001a070 con 0x7feea81f5c90 2024-04-25T11:42:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.836+0000 7fee9f400640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee880744e0 0x7fee880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.836+0000 7fee9f400640 1 -- 172.21.15.19:0/3811796557 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7feea0095170 con 0x7feea81f5c90 2024-04-25T11:42:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.836+0000 7feea7400640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee880744e0 0x7fee880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.837+0000 7feea7400640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee880744e0 0x7fee880769f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fee98007bf0 tx=0x7fee98007240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:11.840+0000 7fee9f400640 1 -- 172.21.15.19:0/3811796557 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feea005e4f0 con 0x7feea81f5c90 2024-04-25T11:42:12.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.042+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feea81612c0 con 0x7feea81f5c90 2024-04-25T11:42:12.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.045+0000 7fee9f400640 1 -- 172.21.15.19:0/3811796557 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+21842 (secure 0 0 0) 0x7feea00627f0 con 0x7feea81f5c90 2024-04-25T11:42:12.031 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:12.032 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":132,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:03.371686+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6834/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6835/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6834/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6834/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:12.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee880744e0 msgr2=0x7fee880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee880744e0 0x7fee880769f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fee98007bf0 tx=0x7fee98007240 comp rx=0 tx=0).stop 2024-04-25T11:42:12.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea81f5c90 msgr2=0x7feea81457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea81f5c90 0x7feea81457d0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7feea000c3a0 tx=0x7feea000c860 comp rx=0 tx=0).stop 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 shutdown_connections 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fee880744e0 0x7fee880769f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feea8144370 0x7feea8137590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea81f5c90 0x7feea81457d0 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 --2- 172.21.15.19:0/3811796557 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feea806c5c0 0x7feea80cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 >> 172.21.15.19:0/3811796557 conn(0x7feea80d1720 msgr2=0x7feea806ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 shutdown_connections 2024-04-25T11:42:12.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.048+0000 7feead000640 1 -- 172.21.15.19:0/3811796557 wait complete. 2024-04-25T11:42:12.063 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:42:12.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.364+0000 7fdf83200640 1 Processor -- start 2024-04-25T11:42:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.364+0000 7fdf83200640 1 -- start start 2024-04-25T11:42:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.366+0000 7fdf83200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf8406c5c0 0x7fdf840cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.366+0000 7fdf83200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf840b4ea0 0x7fdf8415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.366+0000 7fdf83200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf840d1230 con 0x7fdf84144370 2024-04-25T11:42:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.366+0000 7fdf83200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdf840d06b0 con 0x7fdf840b4ea0 2024-04-25T11:42:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.366+0000 7fdf83200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdf840d0990 con 0x7fdf8406c5c0 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf82800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf840b4ea0 0x7fdf8415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf82800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf840b4ea0 0x7fdf8415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39048/0 (socket says 172.21.15.19:39048) 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf82800640 1 -- 172.21.15.19:0/3687332535 learned_addr learned my addr 172.21.15.19:0/3687332535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf81e00640 1 --2- 172.21.15.19:0/3687332535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf8406c5c0 0x7fdf840cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf82800640 1 -- 172.21.15.19:0/3687332535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf8406c5c0 msgr2=0x7fdf840cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf82800640 1 --2- 172.21.15.19:0/3687332535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf8406c5c0 0x7fdf840cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf82800640 1 -- 172.21.15.19:0/3687332535 >> v1:172.21.15.169:6789/0 conn(0x7fdf84144370 legacy=0x7fdf840b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.367+0000 7fdf82800640 1 -- 172.21.15.19:0/3687332535 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf840d0c70 con 0x7fdf840b4ea0 2024-04-25T11:42:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.368+0000 7fdf82800640 1 --2- 172.21.15.19:0/3687332535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf840b4ea0 0x7fdf8415e770 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009e50 tx=0x7fdf7c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a5179e70d1a878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.369+0000 7fdf80a00640 1 -- 172.21.15.19:0/3687332535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c036070 con 0x7fdf840b4ea0 2024-04-25T11:42:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.369+0000 7fdf80a00640 1 -- 172.21.15.19:0/3687332535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c031600 con 0x7fdf840b4ea0 2024-04-25T11:42:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.369+0000 7fdf80a00640 1 -- 172.21.15.19:0/3687332535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0319d0 con 0x7fdf840b4ea0 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.369+0000 7fdf83200640 1 -- 172.21.15.19:0/3687332535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf840b4ea0 msgr2=0x7fdf8415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.369+0000 7fdf83200640 1 --2- 172.21.15.19:0/3687332535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf840b4ea0 0x7fdf8415e770 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009e50 tx=0x7fdf7c02a6c0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.370+0000 7fdf83200640 1 -- 172.21.15.19:0/3687332535 shutdown_connections 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.370+0000 7fdf83200640 1 --2- 172.21.15.19:0/3687332535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf840b4ea0 0x7fdf8415e770 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.370+0000 7fdf83200640 1 --2- 172.21.15.19:0/3687332535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf8406c5c0 0x7fdf840cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.370+0000 7fdf83200640 1 -- 172.21.15.19:0/3687332535 >> 172.21.15.19:0/3687332535 conn(0x7fdf840d1720 msgr2=0x7fdf840d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.370+0000 7fdf83200640 1 -- 172.21.15.19:0/3687332535 shutdown_connections 2024-04-25T11:42:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.370+0000 7fdf83200640 1 -- 172.21.15.19:0/3687332535 wait complete. 2024-04-25T11:42:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.372+0000 7fdf83200640 1 Processor -- start 2024-04-25T11:42:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.372+0000 7fdf83200640 1 -- start start 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf83200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8406c5c0 0x7fdf84140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf83200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf84144370 0x7fdf840cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf83200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf84162070 0x7fdf840cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf83200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf840d0450 con 0x7fdf84162070 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf83200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdf840d02d0 con 0x7fdf8406c5c0 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf83200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdf84147760 con 0x7fdf84144370 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf82800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf84162070 0x7fdf840cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf82800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf84162070 0x7fdf840cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39054/0 (socket says 172.21.15.19:39054) 2024-04-25T11:42:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.374+0000 7fdf82800640 1 -- 172.21.15.19:0/3945453794 learned_addr learned my addr 172.21.15.19:0/3945453794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf81e00640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8406c5c0 0x7fdf84140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf81400640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf84144370 0x7fdf840cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf82800640 1 -- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf84144370 msgr2=0x7fdf840cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf82800640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf84144370 0x7fdf840cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf82800640 1 -- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8406c5c0 msgr2=0x7fdf84140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf82800640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8406c5c0 0x7fdf84140450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf82800640 1 -- 172.21.15.19:0/3945453794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf840cc2d0 con 0x7fdf84162070 2024-04-25T11:42:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf81400640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf84144370 0x7fdf840cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.375+0000 7fdf82800640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf84162070 0x7fdf840cbb00 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009e50 tx=0x7fdf7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.376+0000 7fdf7a000640 1 -- 172.21.15.19:0/3945453794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c036050 con 0x7fdf84162070 2024-04-25T11:42:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.376+0000 7fdf7a000640 1 -- 172.21.15.19:0/3945453794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c042070 con 0x7fdf84162070 2024-04-25T11:42:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.376+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf84160fd0 con 0x7fdf84162070 2024-04-25T11:42:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.376+0000 7fdf7a000640 1 -- 172.21.15.19:0/3945453794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c009510 con 0x7fdf84162070 2024-04-25T11:42:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.376+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf84161590 con 0x7fdf84162070 2024-04-25T11:42:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.377+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf84143f60 con 0x7fdf84162070 2024-04-25T11:42:12.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.379+0000 7fdf7a000640 1 -- 172.21.15.19:0/3945453794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf7c007bd0 con 0x7fdf84162070 2024-04-25T11:42:12.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.380+0000 7fdf7a000640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf600744b0 0x7fdf600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.380+0000 7fdf7a000640 1 -- 172.21.15.19:0/3945453794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7fdf7c0b6ea0 con 0x7fdf84162070 2024-04-25T11:42:12.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.380+0000 7fdf81e00640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf600744b0 0x7fdf600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.382+0000 7fdf81e00640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf600744b0 0x7fdf600769c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fdf74006fd0 tx=0x7fdf74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.385+0000 7fdf7a000640 1 -- 172.21.15.19:0/3945453794 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf84143f60 con 0x7fdf84162070 2024-04-25T11:42:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.636+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf840cfc10 con 0x7fdf84162070 2024-04-25T11:42:12.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.637+0000 7fdf7a000640 1 -- 172.21.15.19:0/3945453794 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) ==== 95+0+5179 (secure 0 0 0) 0x7fdf7c0808a0 con 0x7fdf84162070 2024-04-25T11:42:12.624 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:12.624 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":355,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:03.372367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:12.624 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 355 2024-04-25T11:42:12.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf600744b0 msgr2=0x7fdf600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf600744b0 0x7fdf600769c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fdf74006fd0 tx=0x7fdf74008040 comp rx=0 tx=0).stop 2024-04-25T11:42:12.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf84162070 msgr2=0x7fdf840cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf84162070 0x7fdf840cbb00 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009e50 tx=0x7fdf7c004a40 comp rx=0 tx=0).stop 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 shutdown_connections 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdf600744b0 0x7fdf600769c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf84162070 0x7fdf840cbb00 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdf84144370 0x7fdf840cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 --2- 172.21.15.19:0/3945453794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdf8406c5c0 0x7fdf84140450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 >> 172.21.15.19:0/3945453794 conn(0x7fdf840d1720 msgr2=0x7fdf840b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 shutdown_connections 2024-04-25T11:42:12.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.641+0000 7fdf83200640 1 -- 172.21.15.19:0/3945453794 wait complete. 2024-04-25T11:42:12.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.958+0000 7f2551c00640 1 Processor -- start 2024-04-25T11:42:12.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.959+0000 7f2551c00640 1 -- start start 2024-04-25T11:42:12.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.961+0000 7f2551c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f254c0b4ea0 0x7f254c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.961+0000 7f2551c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254c0d1230 con 0x7f254c144370 2024-04-25T11:42:12.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.961+0000 7f2551c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254c0d06b0 con 0x7f254c06c5c0 2024-04-25T11:42:12.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.961+0000 7f2551c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f254c0d0990 con 0x7f254c0b4ea0 2024-04-25T11:42:12.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.961+0000 7f2551200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f254c144370 0x7f254c15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36974/0 (socket says 172.21.15.19:36974) 2024-04-25T11:42:12.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.961+0000 7f2551200640 1 -- 172.21.15.19:0/13427421 learned_addr learned my addr 172.21.15.19:0/13427421 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:12.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254b400640 1 --2- 172.21.15.19:0/13427421 >> v2:172.21.15.169:3300/0 conn(0x7f254c0b4ea0 0x7f254c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3514837949 0 0) 0x7f254c0d1230 con 0x7f254c144370 2024-04-25T11:42:12.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2530003520 con 0x7f254c144370 2024-04-25T11:42:12.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2940312944 0 0) 0x7f2530003520 con 0x7f254c144370 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 >> v2:172.21.15.169:3300/0 conn(0x7f254c0b4ea0 msgr2=0x7f254c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 --2- 172.21.15.19:0/13427421 >> v2:172.21.15.169:3300/0 conn(0x7f254c0b4ea0 0x7f254c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 >> v1:172.21.15.169:6789/0 conn(0x7f254c06c5c0 legacy=0x7f254c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c0d0c70 con 0x7f254c144370 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f25440019f0 con 0x7f254c144370 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 >> v1:172.21.15.19:6789/0 conn(0x7f254c144370 legacy=0x7f254c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25300039b0 0x7f2530003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530004800 0x7f2530004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25300051c0 0x7f253000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f254c0d0990 con 0x7f2530004800 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f253000c160 con 0x7f25300051c0 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.962+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f253000c360 con 0x7f25300039b0 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254b400640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530004800 0x7f2530004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254b400640 1 -- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25300039b0 msgr2=0x7f2530003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254be00640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25300039b0 0x7f2530003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f2551200640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25300051c0 0x7f253000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254b400640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25300039b0 0x7f2530003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254b400640 1 -- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25300051c0 msgr2=0x7f253000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254b400640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25300051c0 0x7f253000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254b400640 1 -- 172.21.15.19:0/13427421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2530003520 con 0x7f2530004800 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f2551200640 1 -- 172.21.15.19:0/13427421 reap_dead start 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.963+0000 7f254be00640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25300039b0 0x7f2530003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:12.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.964+0000 7f254b400640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530004800 0x7f2530004c10 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f2538009ac0 tx=0x7f2530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154fb6d373996b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:12.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538036070 con 0x7f2530004800 2024-04-25T11:42:12.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538031400 con 0x7f2530004800 2024-04-25T11:42:12.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f254aa00640 1 -- 172.21.15.19:0/13427421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538009690 con 0x7f2530004800 2024-04-25T11:42:12.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f2551c00640 1 -- 172.21.15.19:0/13427421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530004800 msgr2=0x7f2530004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f2551c00640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530004800 0x7f2530004c10 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f2538009ac0 tx=0x7f2530005830 comp rx=0 tx=0).stop 2024-04-25T11:42:12.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f2551c00640 1 -- 172.21.15.19:0/13427421 shutdown_connections 2024-04-25T11:42:12.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f2551c00640 1 --2- 172.21.15.19:0/13427421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2530004800 0x7f2530004c10 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.965+0000 7f2551c00640 1 -- 172.21.15.19:0/13427421 >> 172.21.15.19:0/13427421 conn(0x7f254c0d1720 msgr2=0x7f254c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:12.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.966+0000 7f2551c00640 1 -- 172.21.15.19:0/13427421 shutdown_connections 2024-04-25T11:42:12.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.966+0000 7f2551c00640 1 -- 172.21.15.19:0/13427421 wait complete. 2024-04-25T11:42:12.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.968+0000 7f2551c00640 1 Processor -- start 2024-04-25T11:42:12.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.968+0000 7f2551c00640 1 -- start start 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2530004800 0x7f254c0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254c15b6b0 0x7f254c161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f254c161c90 0x7f254c1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f254c0d0450 con 0x7f254c161c90 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f254c0d02d0 con 0x7f2530004800 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f254c147760 con 0x7f254c15b6b0 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f254c161c90 0x7f254c1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f254c161c90 0x7f254c1620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39076/0 (socket says 172.21.15.19:39076) 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f2551200640 1 -- 172.21.15.19:0/2705276886 learned_addr learned my addr 172.21.15.19:0/2705276886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f254be00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2530004800 0x7f254c0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.970+0000 7f254b400640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254c15b6b0 0x7f254c161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551200640 1 -- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254c15b6b0 msgr2=0x7f254c161750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551200640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254c15b6b0 0x7f254c161750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551200640 1 -- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2530004800 msgr2=0x7f254c0cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:12.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551200640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2530004800 0x7f254c0cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:12.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551200640 1 -- 172.21.15.19:0/2705276886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c1625e0 con 0x7f254c161c90 2024-04-25T11:42:12.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f254be00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2530004800 0x7f254c0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551200640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f254c161c90 0x7f254c1620a0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f2544001580 tx=0x7f254400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2543e00640 1 -- 172.21.15.19:0/2705276886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544004d60 con 0x7f254c161c90 2024-04-25T11:42:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2543e00640 1 -- 172.21.15.19:0/2705276886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f254400bb20 con 0x7f254c161c90 2024-04-25T11:42:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c161270 con 0x7f254c161c90 2024-04-25T11:42:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2543e00640 1 -- 172.21.15.19:0/2705276886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544007110 con 0x7f254c161c90 2024-04-25T11:42:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.971+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c0cbb40 con 0x7f254c161c90 2024-04-25T11:42:12.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.973+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254c0cd3c0 con 0x7f254c161c90 2024-04-25T11:42:12.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.974+0000 7f2543e00640 1 -- 172.21.15.19:0/2705276886 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f25440072b0 con 0x7f254c161c90 2024-04-25T11:42:12.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.975+0000 7f2543e00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f252c0744b0 0x7f252c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:12.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.975+0000 7f2543e00640 1 -- 172.21.15.19:0/2705276886 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f2544095e90 con 0x7f254c161c90 2024-04-25T11:42:12.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.976+0000 7f254be00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f252c0744b0 0x7f252c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:12.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.977+0000 7f254be00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f252c0744b0 0x7f252c0769c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f253c000c00 tx=0x7f253c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:12.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:12.980+0000 7f2543e00640 1 -- 172.21.15.19:0/2705276886 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f254405f920 con 0x7f254c161c90 2024-04-25T11:42:13.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.227+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f254c1627b0 con 0x7f254c161c90 2024-04-25T11:42:13.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.229+0000 7f2543e00640 1 -- 172.21.15.19:0/2705276886 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 355 v355) ==== 95+0+5179 (secure 0 0 0) 0x7f2544097020 con 0x7f254c161c90 2024-04-25T11:42:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:13.216 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":355,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[{"mdsmap":{"epoch":354,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:35:13.963880+0000","modified":"2024-04-25T11:42:03.372367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-25T11:42:13.216 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 355 2024-04-25T11:42:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f252c0744b0 msgr2=0x7f252c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f252c0744b0 0x7f252c0769c0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f253c000c00 tx=0x7f253c009040 comp rx=0 tx=0).stop 2024-04-25T11:42:13.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f254c161c90 msgr2=0x7f254c1620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f254c161c90 0x7f254c1620a0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f2544001580 tx=0x7f254400b4a0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 shutdown_connections 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f252c0744b0 0x7f252c0769c0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f254c161c90 0x7f254c1620a0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f254c15b6b0 0x7f254c161750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 --2- 172.21.15.19:0/2705276886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2530004800 0x7f254c0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 >> 172.21.15.19:0/2705276886 conn(0x7f254c0d1720 msgr2=0x7f254c15eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 shutdown_connections 2024-04-25T11:42:13.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.232+0000 7f2551c00640 1 -- 172.21.15.19:0/2705276886 wait complete. 2024-04-25T11:42:13.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:13.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.542+0000 7f70ed000640 1 Processor -- start 2024-04-25T11:42:13.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.542+0000 7f70ed000640 1 -- start start 2024-04-25T11:42:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.544+0000 7f70ed000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f70e80b6e10 0x7f70e815a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.544+0000 7f70ed000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 0x7f70e8160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.544+0000 7f70ed000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e8161220 con 0x7f70e80cf2f0 2024-04-25T11:42:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.544+0000 7f70ed000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f70e806f870 con 0x7f70e80b6e10 2024-04-25T11:42:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.544+0000 7f70ed000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70e80b7220 con 0x7f70e81608d0 2024-04-25T11:42:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.545+0000 7f70e7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f70e80cf2f0 0x7f70e80b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36986/0 (socket says 172.21.15.19:36986) 2024-04-25T11:42:13.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.545+0000 7f70e7400640 1 -- 172.21.15.19:0/1685084337 learned_addr learned my addr 172.21.15.19:0/1685084337 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.545+0000 7f70e6a00640 1 --2- 172.21.15.19:0/1685084337 >> v2:172.21.15.169:3300/0 conn(0x7f70e80b6e10 0x7f70e815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.545+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073719106 0 0) 0x7f70d8001110 con 0x7f70e80cf2f0 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.545+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70cc003520 con 0x7f70e80cf2f0 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.545+0000 7f70e7e00640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 0x7f70e8160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1226222983 0 0) 0x7f70cc003520 con 0x7f70e80cf2f0 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 msgr2=0x7f70e8160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 0x7f70e8160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 >> v2:172.21.15.169:3300/0 conn(0x7f70e80b6e10 msgr2=0x7f70e815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 --2- 172.21.15.19:0/1685084337 >> v2:172.21.15.169:3300/0 conn(0x7f70e80b6e10 0x7f70e815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e80b74d0 con 0x7f70e80cf2f0 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f70d8001860 con 0x7f70e80cf2f0 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 >> v1:172.21.15.19:6789/0 conn(0x7f70e80cf2f0 legacy=0x7f70e80b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70cc0039b0 0x7f70cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 0x7f70cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70cc004d40 0x7f70cc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70e806f870 con 0x7f70cc004d40 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70e80b7220 con 0x7f70cc0039b0 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc00dc80 con 0x7f70e81608d0 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.546+0000 7f70e7e00640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70cc004d40 0x7f70cc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e6a00640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 0x7f70cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e7400640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70cc0039b0 0x7f70cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e7e00640 1 -- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 msgr2=0x7f70cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e7e00640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70e81608d0 0x7f70cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e7e00640 1 -- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70cc0039b0 msgr2=0x7f70cc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e7e00640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70cc0039b0 0x7f70cc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e7e00640 1 -- 172.21.15.19:0/1685084337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc003520 con 0x7f70cc004d40 2024-04-25T11:42:13.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.547+0000 7f70e7400640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70cc0039b0 0x7f70cc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:13.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.548+0000 7f70e7e00640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70cc004d40 0x7f70cc005150 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f70e80d27c0 tx=0x7f70cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5122cd88c33dde6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:13.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.548+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0033070 con 0x7f70cc004d40 2024-04-25T11:42:13.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.548+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e002f630 con 0x7f70cc004d40 2024-04-25T11:42:13.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.548+0000 7f70e6000640 1 -- 172.21.15.19:0/1685084337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0009be0 con 0x7f70cc004d40 2024-04-25T11:42:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.549+0000 7f70ed000640 1 -- 172.21.15.19:0/1685084337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70cc004d40 msgr2=0x7f70cc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.549+0000 7f70ed000640 1 --2- 172.21.15.19:0/1685084337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70cc004d40 0x7f70cc005150 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f70e80d27c0 tx=0x7f70cc005830 comp rx=0 tx=0).stop 2024-04-25T11:42:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.549+0000 7f70e7e00640 1 -- 172.21.15.19:0/1685084337 reap_dead start 2024-04-25T11:42:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.549+0000 7f70ed000640 1 -- 172.21.15.19:0/1685084337 shutdown_connections 2024-04-25T11:42:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.549+0000 7f70ed000640 1 -- 172.21.15.19:0/1685084337 >> 172.21.15.19:0/1685084337 conn(0x7f70e80d1720 msgr2=0x7f70e80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.549+0000 7f70ed000640 1 -- 172.21.15.19:0/1685084337 shutdown_connections 2024-04-25T11:42:13.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.549+0000 7f70ed000640 1 -- 172.21.15.19:0/1685084337 wait complete. 2024-04-25T11:42:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.552+0000 7f70ed000640 1 Processor -- start 2024-04-25T11:42:13.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.552+0000 7f70ed000640 1 -- start start 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.553+0000 7f70ed000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70cc004d40 0x7f70e815eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.553+0000 7f70ed000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e81518c0 0x7f70e8149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70ed000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8158c00 0x7f70e814a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70ed000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70e806f500 con 0x7f70e81518c0 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70ed000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70e806f220 con 0x7f70e8158c00 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70ed000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f70e8160ce0 con 0x7f70cc004d40 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e81518c0 0x7f70e8149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e81518c0 0x7f70e8149cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39092/0 (socket says 172.21.15.19:39092) 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 -- 172.21.15.19:0/1731332331 learned_addr learned my addr 172.21.15.19:0/1731332331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e7400640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70cc004d40 0x7f70e815eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e7e00640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8158c00 0x7f70e814a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 -- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70cc004d40 msgr2=0x7f70e815eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70cc004d40 0x7f70e815eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 -- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8158c00 msgr2=0x7f70e814a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8158c00 0x7f70e814a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.554+0000 7f70e6a00640 1 -- 172.21.15.19:0/1731332331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e815f380 con 0x7f70e81518c0 2024-04-25T11:42:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.555+0000 7f70e6a00640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e81518c0 0x7f70e8149cf0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f70d4011700 tx=0x7f70d4011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.555+0000 7f70df400640 1 -- 172.21.15.19:0/1731332331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d400f9c0 con 0x7f70e81518c0 2024-04-25T11:42:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.555+0000 7f70df400640 1 -- 172.21.15.19:0/1731332331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d400fb60 con 0x7f70e81518c0 2024-04-25T11:42:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.555+0000 7f70df400640 1 -- 172.21.15.19:0/1731332331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4015460 con 0x7f70e81518c0 2024-04-25T11:42:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.555+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e8158120 con 0x7f70e81518c0 2024-04-25T11:42:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.555+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e815f660 con 0x7f70e81518c0 2024-04-25T11:42:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.557+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e815f0f0 con 0x7f70e81518c0 2024-04-25T11:42:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.558+0000 7f70df400640 1 -- 172.21.15.19:0/1731332331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f70d4015600 con 0x7f70e81518c0 2024-04-25T11:42:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.559+0000 7f70df400640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70c80744e0 0x7f70c80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.559+0000 7f70df400640 1 -- 172.21.15.19:0/1731332331 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f70d40969a0 con 0x7f70e81518c0 2024-04-25T11:42:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.560+0000 7f70e7400640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70c80744e0 0x7f70c80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.561+0000 7f70e7400640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70c80744e0 0x7f70c80769f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f70d8001eb0 tx=0x7f70d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.564+0000 7f70df400640 1 -- 172.21.15.19:0/1731332331 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70d4019030 con 0x7f70e81518c0 2024-04-25T11:42:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.763+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70e8050740 con 0x7f70e81518c0 2024-04-25T11:42:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.766+0000 7f70df400640 1 -- 172.21.15.19:0/1731332331 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+21842 (secure 0 0 0) 0x7f70d40603a0 con 0x7f70e81518c0 2024-04-25T11:42:13.753 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:13.753 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":132,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:03.371686+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:35:12.150487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T11:35:12.936871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6834/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6835/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6834/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6834/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70c80744e0 msgr2=0x7f70c80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70c80744e0 0x7f70c80769f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f70d8001eb0 tx=0x7f70d8009040 comp rx=0 tx=0).stop 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e81518c0 msgr2=0x7f70e8149cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e81518c0 0x7f70e8149cf0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f70d4011700 tx=0x7f70d4011bc0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 shutdown_connections 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70c80744e0 0x7f70c80769f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70e8158c00 0x7f70e814a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70e81518c0 0x7f70e8149cf0 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 --2- 172.21.15.19:0/1731332331 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70cc004d40 0x7f70e815eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 >> 172.21.15.19:0/1731332331 conn(0x7f70e80d1720 msgr2=0x7f70e80cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 shutdown_connections 2024-04-25T11:42:13.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:13.769+0000 7f70ed000640 1 -- 172.21.15.19:0/1731332331 wait complete. 2024-04-25T11:42:13.781 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:42:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.064+0000 7f1517200640 1 Processor -- start 2024-04-25T11:42:14.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.064+0000 7f1517200640 1 -- start start 2024-04-25T11:42:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.066+0000 7f1517200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1510160fe0 0x7f151015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.066+0000 7f1517200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15100b7220 con 0x7f15100cf2f0 2024-04-25T11:42:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.066+0000 7f1517200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15100b74d0 con 0x7f15100b6e10 2024-04-25T11:42:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.066+0000 7f1517200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f151006f870 con 0x7f1510160fe0 2024-04-25T11:42:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1515e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f15100cf2f0 0x7f15100b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50490/0 (socket says 172.21.15.19:50490) 2024-04-25T11:42:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1515e00640 1 -- 172.21.15.19:0/839957817 learned_addr learned my addr 172.21.15.19:0/839957817 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:14.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1516800640 1 --2- 172.21.15.19:0/839957817 >> v2:172.21.15.19:3300/0 conn(0x7f1510160fe0 0x7f151015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1514a00640 1 -- 172.21.15.19:0/839957817 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839836543 0 0) 0x7f15100b7220 con 0x7f15100cf2f0 2024-04-25T11:42:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1514a00640 1 -- 172.21.15.19:0/839957817 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14f8003520 con 0x7f15100cf2f0 2024-04-25T11:42:14.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1516800640 1 -- 172.21.15.19:0/839957817 >> v1:172.21.15.169:6789/0 conn(0x7f15100b6e10 legacy=0x7f15101608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:14.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1516800640 1 -- 172.21.15.19:0/839957817 >> v1:172.21.15.19:6789/0 conn(0x7f15100cf2f0 legacy=0x7f15100b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.067+0000 7f1516800640 1 -- 172.21.15.19:0/839957817 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15100b77b0 con 0x7f1510160fe0 2024-04-25T11:42:14.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.069+0000 7f1516800640 1 --2- 172.21.15.19:0/839957817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510160fe0 0x7f151015a810 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f150c009d90 tx=0x7f150c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd7923a5c27a667 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:14.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.069+0000 7f1514a00640 1 -- 172.21.15.19:0/839957817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c007740 con 0x7f1510160fe0 2024-04-25T11:42:14.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.069+0000 7f1514a00640 1 -- 172.21.15.19:0/839957817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c0078e0 con 0x7f1510160fe0 2024-04-25T11:42:14.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.069+0000 7f1514a00640 1 -- 172.21.15.19:0/839957817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c007c90 con 0x7f1510160fe0 2024-04-25T11:42:14.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.070+0000 7f1517200640 1 -- 172.21.15.19:0/839957817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510160fe0 msgr2=0x7f151015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.070+0000 7f1517200640 1 --2- 172.21.15.19:0/839957817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510160fe0 0x7f151015a810 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f150c009d90 tx=0x7f150c02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.070+0000 7f1517200640 1 -- 172.21.15.19:0/839957817 shutdown_connections 2024-04-25T11:42:14.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.070+0000 7f1517200640 1 --2- 172.21.15.19:0/839957817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1510160fe0 0x7f151015a810 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.070+0000 7f1517200640 1 -- 172.21.15.19:0/839957817 >> 172.21.15.19:0/839957817 conn(0x7f15100d1720 msgr2=0x7f15100d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:14.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.070+0000 7f1517200640 1 -- 172.21.15.19:0/839957817 shutdown_connections 2024-04-25T11:42:14.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.070+0000 7f1517200640 1 -- 172.21.15.19:0/839957817 wait complete. 2024-04-25T11:42:14.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.072+0000 7f1517200640 1 Processor -- start 2024-04-25T11:42:14.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.072+0000 7f1517200640 1 -- start start 2024-04-25T11:42:14.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1517200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100b6e10 0x7f151013b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1517200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15100cf2f0 0x7f15101358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1517200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15101595f0 0x7f151013a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1517200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f151006fd00 con 0x7f15100cf2f0 2024-04-25T11:42:14.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1517200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f151006f500 con 0x7f15101595f0 2024-04-25T11:42:14.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1517200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f151006fe80 con 0x7f15100b6e10 2024-04-25T11:42:14.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1515400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15100cf2f0 0x7f15101358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1515400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15100cf2f0 0x7f15101358a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50684/0 (socket says 172.21.15.19:50684) 2024-04-25T11:42:14.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1515400640 1 -- 172.21.15.19:0/930528539 learned_addr learned my addr 172.21.15.19:0/930528539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1516800640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15101595f0 0x7f151013a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.074+0000 7f1515e00640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100b6e10 0x7f151013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f1515400640 1 -- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100b6e10 msgr2=0x7f151013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f1515400640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100b6e10 0x7f151013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f1515400640 1 -- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15101595f0 msgr2=0x7f151013a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f1515400640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15101595f0 0x7f151013a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f1515400640 1 -- 172.21.15.19:0/930528539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1510135de0 con 0x7f15100cf2f0 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f1515400640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15100cf2f0 0x7f15101358a0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00c430 tx=0x7f14fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f150a000640 1 -- 172.21.15.19:0/930528539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc017070 con 0x7f15100cf2f0 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f150a000640 1 -- 172.21.15.19:0/930528539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14fc0052d0 con 0x7f15100cf2f0 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f150a000640 1 -- 172.21.15.19:0/930528539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc003830 con 0x7f15100cf2f0 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.075+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15101393b0 con 0x7f15100cf2f0 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.076+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1510139970 con 0x7f15100cf2f0 2024-04-25T11:42:14.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.077+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1510050740 con 0x7f15100cf2f0 2024-04-25T11:42:14.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.078+0000 7f150a000640 1 -- 172.21.15.19:0/930528539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14fc01a070 con 0x7f15100cf2f0 2024-04-25T11:42:14.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.079+0000 7f150a000640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14f00744b0 0x7f14f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.079+0000 7f150a000640 1 -- 172.21.15.19:0/930528539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f14fc0953c0 con 0x7f15100cf2f0 2024-04-25T11:42:14.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.080+0000 7f1515e00640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14f00744b0 0x7f14f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.081+0000 7f1515e00640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14f00744b0 0x7f14f00769c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f15040011e0 tx=0x7f1504009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.084+0000 7f150a000640 1 -- 172.21.15.19:0/930528539 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14fc05ed30 con 0x7f15100cf2f0 2024-04-25T11:42:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.341+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f151006c770 con 0x7f15100cf2f0 2024-04-25T11:42:14.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.462+0000 7f150a000640 1 -- 172.21.15.19:0/930528539 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v356) ==== 161+0+0 (secure 0 0 0) 0x7f14fc05e630 con 0x7f15100cf2f0 2024-04-25T11:42:14.448 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:42:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14f00744b0 msgr2=0x7f14f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14f00744b0 0x7f14f00769c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f15040011e0 tx=0x7f1504009040 comp rx=0 tx=0).stop 2024-04-25T11:42:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15100cf2f0 msgr2=0x7f15101358a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15100cf2f0 0x7f15101358a0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00c430 tx=0x7f14fc00c8f0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 shutdown_connections 2024-04-25T11:42:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f14f00744b0 0x7f14f00769c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15101595f0 0x7f151013a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15100cf2f0 0x7f15101358a0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 --2- 172.21.15.19:0/930528539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15100b6e10 0x7f151013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.465+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 >> 172.21.15.19:0/930528539 conn(0x7f15100d1720 msgr2=0x7f1510073b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.466+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 shutdown_connections 2024-04-25T11:42:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.466+0000 7f1517200640 1 -- 172.21.15.19:0/930528539 wait complete. 2024-04-25T11:42:14.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:42:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.773+0000 7f666ce00640 1 Processor -- start 2024-04-25T11:42:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.773+0000 7f666ce00640 1 -- start start 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.775+0000 7f666ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f66680d1340 0x7f6668070260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.775+0000 7f666ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66680707a0 0x7f6668140490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.775+0000 7f666ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6668070bb0 con 0x7f666813ff60 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.775+0000 7f666ce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f666815e340 con 0x7f66680d1340 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.775+0000 7f666ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f66680d02a0 con 0x7f66680707a0 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.776+0000 7f6667400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f666813ff60 0x7f666815dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50496/0 (socket says 172.21.15.19:50496) 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.776+0000 7f6667400640 1 -- 172.21.15.19:0/2651012088 learned_addr learned my addr 172.21.15.19:0/2651012088 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.776+0000 7f665fe00640 1 --2- 172.21.15.19:0/2651012088 >> v2:172.21.15.19:3300/0 conn(0x7f66680d1340 0x7f6668070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.776+0000 7f6667e00640 1 --2- 172.21.15.19:0/2651012088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66680707a0 0x7f6668140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.776+0000 7f6666a00640 1 -- 172.21.15.19:0/2651012088 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170000811 0 0) 0x7f6658001110 con 0x7f666813ff60 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.776+0000 7f6666a00640 1 -- 172.21.15.19:0/2651012088 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f664c003520 con 0x7f666813ff60 2024-04-25T11:42:14.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.777+0000 7f665fe00640 1 -- 172.21.15.19:0/2651012088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66680707a0 msgr2=0x7f6668140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.777+0000 7f665fe00640 1 --2- 172.21.15.19:0/2651012088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66680707a0 0x7f6668140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.777+0000 7f665fe00640 1 -- 172.21.15.19:0/2651012088 >> v1:172.21.15.19:6789/0 conn(0x7f666813ff60 legacy=0x7f666815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.777+0000 7f665fe00640 1 -- 172.21.15.19:0/2651012088 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66680d0580 con 0x7f66680d1340 2024-04-25T11:42:14.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.778+0000 7f665fe00640 1 --2- 172.21.15.19:0/2651012088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f6668070260 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f66500097a0 tx=0x7f665002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58a2bee8e6f7c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f6666a00640 1 -- 172.21.15.19:0/2651012088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650037070 con 0x7f66680d1340 2024-04-25T11:42:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f6666a00640 1 -- 172.21.15.19:0/2651012088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665002ae40 con 0x7f66680d1340 2024-04-25T11:42:14.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f6666a00640 1 -- 172.21.15.19:0/2651012088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650032660 con 0x7f66680d1340 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f666ce00640 1 -- 172.21.15.19:0/2651012088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 msgr2=0x7f6668070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f666ce00640 1 --2- 172.21.15.19:0/2651012088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f6668070260 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f66500097a0 tx=0x7f665002a0f0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f666ce00640 1 -- 172.21.15.19:0/2651012088 shutdown_connections 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f666ce00640 1 --2- 172.21.15.19:0/2651012088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66680707a0 0x7f6668140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.779+0000 7f666ce00640 1 --2- 172.21.15.19:0/2651012088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f6668070260 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.780+0000 7f666ce00640 1 -- 172.21.15.19:0/2651012088 >> 172.21.15.19:0/2651012088 conn(0x7f66680b4260 msgr2=0x7f6668072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.780+0000 7f666ce00640 1 -- 172.21.15.19:0/2651012088 shutdown_connections 2024-04-25T11:42:14.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.780+0000 7f666ce00640 1 -- 172.21.15.19:0/2651012088 wait complete. 2024-04-25T11:42:14.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.782+0000 7f666ce00640 1 Processor -- start 2024-04-25T11:42:14.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.782+0000 7f666ce00640 1 -- start start 2024-04-25T11:42:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f666ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f66681484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f666ce00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f666813ff60 0x7f66681f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f666ce00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6668160e10 0x7f66680b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f666ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66680d0e20 con 0x7f66680d1340 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f666ce00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f66680d0b40 con 0x7f666813ff60 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f666ce00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f666815d600 con 0x7f6668160e10 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f66681484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f66681484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50710/0 (socket says 172.21.15.19:50710) 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 -- 172.21.15.19:0/2453186687 learned_addr learned my addr 172.21.15.19:0/2453186687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667e00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6668160e10 0x7f66680b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f665fe00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f666813ff60 0x7f66681f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 -- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6668160e10 msgr2=0x7f66680b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6668160e10 0x7f66680b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 -- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f666813ff60 msgr2=0x7f66681f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f666813ff60 0x7f66681f5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.784+0000 7f6667400640 1 -- 172.21.15.19:0/2453186687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66680b5fe0 con 0x7f66680d1340 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.785+0000 7f6667400640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f66681484e0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f665800c700 tx=0x7f665800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:14.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.785+0000 7f665f400640 1 -- 172.21.15.19:0/2453186687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658017070 con 0x7f66680d1340 2024-04-25T11:42:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.785+0000 7f665f400640 1 -- 172.21.15.19:0/2453186687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6658005a70 con 0x7f66680d1340 2024-04-25T11:42:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.785+0000 7f665f400640 1 -- 172.21.15.19:0/2453186687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658005e60 con 0x7f66680d1340 2024-04-25T11:42:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.785+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66680b62c0 con 0x7f66680d1340 2024-04-25T11:42:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.785+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66680b6880 con 0x7f66680d1340 2024-04-25T11:42:14.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.787+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66680b5d40 con 0x7f66680d1340 2024-04-25T11:42:14.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.788+0000 7f665f400640 1 -- 172.21.15.19:0/2453186687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f665801a070 con 0x7f66680d1340 2024-04-25T11:42:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.789+0000 7f665f400640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66440744e0 0x7f66440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.790+0000 7f665f400640 1 -- 172.21.15.19:0/2453186687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f6658095250 con 0x7f66680d1340 2024-04-25T11:42:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.790+0000 7f665fe00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66440744e0 0x7f66440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:14.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.791+0000 7f665fe00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66440744e0 0x7f66440769f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6650034000 tx=0x7f665002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:14.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:14.796+0000 7f665f400640 1 -- 172.21.15.19:0/2453186687 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6668070bb0 con 0x7f66680d1340 2024-04-25T11:42:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.046+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6668076c60 con 0x7f66680d1340 2024-04-25T11:42:15.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.484+0000 7f665f400640 1 -- 172.21.15.19:0/2453186687 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v357) ==== 104+0+0 (secure 0 0 0) 0x7f665805ec20 con 0x7f66680d1340 2024-04-25T11:42:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66440744e0 msgr2=0x7f66440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66440744e0 0x7f66440769f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6650034000 tx=0x7f665002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 msgr2=0x7f66681484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f66681484e0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f665800c700 tx=0x7f665800cbc0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 shutdown_connections 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f66440744e0 0x7f66440769f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f666813ff60 0x7f66681f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6668160e10 0x7f66680b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 --2- 172.21.15.19:0/2453186687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66680d1340 0x7f66681484e0 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 >> 172.21.15.19:0/2453186687 conn(0x7f66680707a0 msgr2=0x7f6668142580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 shutdown_connections 2024-04-25T11:42:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.488+0000 7f666ce00640 1 -- 172.21.15.19:0/2453186687 wait complete. 2024-04-25T11:42:15.502 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:42:15.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.807+0000 7f02a5a00640 1 Processor -- start 2024-04-25T11:42:15.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.807+0000 7f02a5a00640 1 -- start start 2024-04-25T11:42:15.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.809+0000 7f02a5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a00d1280 0x7f02a00d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.809+0000 7f02a5a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a01616d0 con 0x7f02a00cc690 2024-04-25T11:42:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.809+0000 7f02a5a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a00cfdd0 con 0x7f02a0161180 2024-04-25T11:42:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.809+0000 7f02a5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02a00d00b0 con 0x7f02a00d1280 2024-04-25T11:42:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.809+0000 7f029f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f02a00cc690 0x7f02a006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50512/0 (socket says 172.21.15.19:50512) 2024-04-25T11:42:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.809+0000 7f029f400640 1 -- 172.21.15.19:0/2968402530 learned_addr learned my addr 172.21.15.19:0/2968402530 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.810+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741944653 0 0) 0x7f02a01616d0 con 0x7f02a00cc690 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.810+0000 7f02a5000640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a00d1280 0x7f02a00d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.810+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0284003520 con 0x7f02a00cc690 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.810+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 310665342 0 0) 0x7f0284003520 con 0x7f02a00cc690 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.810+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a00d1280 msgr2=0x7f02a00d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.810+0000 7f029ea00640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a00d1280 0x7f02a00d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.810+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 >> v1:172.21.15.169:6789/0 conn(0x7f02a0161180 legacy=0x7f02a00cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a00d0390 con 0x7f02a00cc690 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f028c001670 con 0x7f02a00cc690 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 >> v1:172.21.15.19:6789/0 conn(0x7f02a00cc690 legacy=0x7f02a006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02840039e0 0x7f0284003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0284004800 0x7f0284004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a00d1280 0x7f028400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00d00b0 con 0x7f0284004800 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02840054e0 con 0x7f02840039e0 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f028400bf00 con 0x7f02a00d1280 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029f400640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0284004800 0x7f0284004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.811+0000 7f029fe00640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02840039e0 0x7f0284003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.812+0000 7f029f400640 1 -- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a00d1280 msgr2=0x7f028400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.812+0000 7f029f400640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02a00d1280 0x7f028400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.812+0000 7f029f400640 1 -- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02840039e0 msgr2=0x7f0284003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:15.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.812+0000 7f029f400640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02840039e0 0x7f0284003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.812+0000 7f029f400640 1 -- 172.21.15.19:0/2968402530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0284003520 con 0x7f0284004800 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.813+0000 7f029f400640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0284004800 0x7f0284004c10 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f028c00c190 tx=0x7f0284005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e42ab5ebcf83bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.813+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c007c60 con 0x7f0284004800 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.813+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c005af0 con 0x7f0284004800 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.813+0000 7f029ea00640 1 -- 172.21.15.19:0/2968402530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c007070 con 0x7f0284004800 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.814+0000 7f02a5a00640 1 -- 172.21.15.19:0/2968402530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0284004800 msgr2=0x7f0284004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.814+0000 7f02a5a00640 1 --2- 172.21.15.19:0/2968402530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0284004800 0x7f0284004c10 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f028c00c190 tx=0x7f0284005830 comp rx=0 tx=0).stop 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.814+0000 7f02a5000640 1 -- 172.21.15.19:0/2968402530 reap_dead start 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.814+0000 7f02a5a00640 1 -- 172.21.15.19:0/2968402530 shutdown_connections 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.814+0000 7f02a5a00640 1 -- 172.21.15.19:0/2968402530 >> 172.21.15.19:0/2968402530 conn(0x7f02a00ce3b0 msgr2=0x7f02a00ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.814+0000 7f02a5a00640 1 -- 172.21.15.19:0/2968402530 shutdown_connections 2024-04-25T11:42:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.814+0000 7f02a5a00640 1 -- 172.21.15.19:0/2968402530 wait complete. 2024-04-25T11:42:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.816+0000 7f02a5a00640 1 Processor -- start 2024-04-25T11:42:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.816+0000 7f02a5a00640 1 -- start start 2024-04-25T11:42:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f02a5a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0284004800 0x7f02a01f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f02a5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a00c6190 0x7f02a00c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f02a5a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a01f9840 0x7f02a01f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f02a5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00cfa80 con 0x7f02a00c6190 2024-04-25T11:42:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f02a5a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00cf900 con 0x7f02a01f9840 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f02a5a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f02a00d0b90 con 0x7f0284004800 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f029f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a00c6190 0x7f02a00c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f029f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a00c6190 0x7f02a00c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50724/0 (socket says 172.21.15.19:50724) 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f029f400640 1 -- 172.21.15.19:0/1757470165 learned_addr learned my addr 172.21.15.19:0/1757470165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.818+0000 7f029fe00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0284004800 0x7f02a01f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f02a5000640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a01f9840 0x7f02a01f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f029f400640 1 -- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0284004800 msgr2=0x7f02a01f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f029f400640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0284004800 0x7f02a01f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f029f400640 1 -- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a01f9840 msgr2=0x7f02a01f9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f029f400640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a01f9840 0x7f02a01f9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f029f400640 1 -- 172.21.15.19:0/1757470165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a01fa190 con 0x7f02a00c6190 2024-04-25T11:42:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f029f400640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a00c6190 0x7f02a00c65a0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f028c02ad60 tx=0x7f028c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f0297e00640 1 -- 172.21.15.19:0/1757470165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c0015b0 con 0x7f02a00c6190 2024-04-25T11:42:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f0297e00640 1 -- 172.21.15.19:0/1757470165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c042070 con 0x7f02a00c6190 2024-04-25T11:42:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.819+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a01522d0 con 0x7f02a00c6190 2024-04-25T11:42:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.820+0000 7f0297e00640 1 -- 172.21.15.19:0/1757470165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c0072b0 con 0x7f02a00c6190 2024-04-25T11:42:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.820+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a0152890 con 0x7f02a00c6190 2024-04-25T11:42:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.821+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a00761c0 con 0x7f02a00c6190 2024-04-25T11:42:15.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.822+0000 7f0297e00640 1 -- 172.21.15.19:0/1757470165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f028c007450 con 0x7f02a00c6190 2024-04-25T11:42:15.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.823+0000 7f0297e00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02800744e0 0x7f02800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.823+0000 7f0297e00640 1 -- 172.21.15.19:0/1757470165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9149+0+0 (secure 0 0 0) 0x7f028c0b60b0 con 0x7f02a00c6190 2024-04-25T11:42:15.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.824+0000 7f029fe00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02800744e0 0x7f02800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.825+0000 7f029fe00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02800744e0 0x7f02800769f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f02a0051770 tx=0x7f0290009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:15.828+0000 7f0297e00640 1 -- 172.21.15.19:0/1757470165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02a01fa190 con 0x7f02a00c6190 2024-04-25T11:42:16.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.037+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f02a00c6c80 con 0x7f02a00c6190 2024-04-25T11:42:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.501+0000 7f0297e00640 1 -- 172.21.15.19:0/1757470165 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) ==== 181+0+0 (secure 0 0 0) 0x7f028c07fab0 con 0x7f02a00c6190 2024-04-25T11:42:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:42:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.507+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02800744e0 msgr2=0x7f02800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.507+0000 7f02a5a00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02800744e0 0x7f02800769f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f02a0051770 tx=0x7f0290009040 comp rx=0 tx=0).stop 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.507+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a00c6190 msgr2=0x7f02a00c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.507+0000 7f02a5a00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a00c6190 0x7f02a00c65a0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f028c02ad60 tx=0x7f028c001970 comp rx=0 tx=0).stop 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.508+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 shutdown_connections 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.508+0000 7f02a5a00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f02800744e0 0x7f02800769f0 secure :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f02a0051770 tx=0x7f0290009040 comp rx=0 tx=0).stop 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.508+0000 7f02a5a00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02a01f9840 0x7f02a01f9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.508+0000 7f02a5a00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02a00c6190 0x7f02a00c65a0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.508+0000 7f02a5a00640 1 --2- 172.21.15.19:0/1757470165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0284004800 0x7f02a01f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.508+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 >> 172.21.15.19:0/1757470165 conn(0x7f02a00ce3b0 msgr2=0x7f02a006fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.509+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 shutdown_connections 2024-04-25T11:42:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.509+0000 7f02a5a00640 1 -- 172.21.15.19:0/1757470165 wait complete. 2024-04-25T11:42:16.531 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:42:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.820+0000 7f8c72800640 1 Processor -- start 2024-04-25T11:42:16.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.821+0000 7f8c72800640 1 -- start start 2024-04-25T11:42:16.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.823+0000 7f8c72800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c0cf870 0x7f8c6c0d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.823+0000 7f8c72800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c6c0cdd60 0x7f8c6c141d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.823+0000 7f8c72800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c6c0cfc80 con 0x7f8c6c15d620 2024-04-25T11:42:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.823+0000 7f8c72800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8c6c06dce0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.823+0000 7f8c72800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6c0cff10 con 0x7f8c6c0cf870 2024-04-25T11:42:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.823+0000 7f8c71e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c6c0cdd60 0x7f8c6c141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.823+0000 7f8c70a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8c6c15d620 0x7f8c6c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50518/0 (socket says 172.21.15.19:50518) 2024-04-25T11:42:16.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.824+0000 7f8c70a00640 1 -- 172.21.15.19:0/4061032880 learned_addr learned my addr 172.21.15.19:0/4061032880 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.824+0000 7f8c71400640 1 --2- 172.21.15.19:0/4061032880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c0cf870 0x7f8c6c0d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.824+0000 7f8c71e00640 1 -- 172.21.15.19:0/4061032880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c0cf870 msgr2=0x7f8c6c0d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.824+0000 7f8c71e00640 1 --2- 172.21.15.19:0/4061032880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c0cf870 0x7f8c6c0d0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.824+0000 7f8c71e00640 1 -- 172.21.15.19:0/4061032880 >> v1:172.21.15.19:6789/0 conn(0x7f8c6c15d620 legacy=0x7f8c6c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.824+0000 7f8c71e00640 1 -- 172.21.15.19:0/4061032880 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c0d01f0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.824+0000 7f8c6be00640 1 -- 172.21.15.19:0/4061032880 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486296269 0 0) 0x7f8c6c0cfc80 con 0x7f8c6c15d620 2024-04-25T11:42:16.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.826+0000 7f8c71e00640 1 --2- 172.21.15.19:0/4061032880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c141d20 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009d90 tx=0x7f8c6402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9559e3862e2374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:16.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.827+0000 7f8c6be00640 1 -- 172.21.15.19:0/4061032880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64007740 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.827+0000 7f8c6be00640 1 -- 172.21.15.19:0/4061032880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c640078e0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.827+0000 7f8c6be00640 1 -- 172.21.15.19:0/4061032880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64007cb0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.827+0000 7f8c72800640 1 -- 172.21.15.19:0/4061032880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 msgr2=0x7f8c6c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.827+0000 7f8c72800640 1 --2- 172.21.15.19:0/4061032880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c141d20 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009d90 tx=0x7f8c6402a660 comp rx=0 tx=0).stop 2024-04-25T11:42:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.828+0000 7f8c72800640 1 -- 172.21.15.19:0/4061032880 shutdown_connections 2024-04-25T11:42:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.828+0000 7f8c72800640 1 --2- 172.21.15.19:0/4061032880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c141d20 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.828+0000 7f8c72800640 1 --2- 172.21.15.19:0/4061032880 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c0cf870 0x7f8c6c0d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.828+0000 7f8c72800640 1 -- 172.21.15.19:0/4061032880 >> 172.21.15.19:0/4061032880 conn(0x7f8c6c072f40 msgr2=0x7f8c6c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.828+0000 7f8c72800640 1 -- 172.21.15.19:0/4061032880 shutdown_connections 2024-04-25T11:42:16.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.828+0000 7f8c72800640 1 -- 172.21.15.19:0/4061032880 wait complete. 2024-04-25T11:42:16.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.831+0000 7f8c72800640 1 Processor -- start 2024-04-25T11:42:16.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.831+0000 7f8c72800640 1 -- start start 2024-04-25T11:42:16.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.833+0000 7f8c72800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c1f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.833+0000 7f8c72800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c6c0cf870 0x7f8c6c1f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.833+0000 7f8c72800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c15d620 0x7f8c6c0b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.833+0000 7f8c72800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c0d0a90 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.833+0000 7f8c72800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c0d07b0 con 0x7f8c6c0cf870 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.833+0000 7f8c72800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6c15da30 con 0x7f8c6c15d620 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.833+0000 7f8c71400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c1f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c1f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50750/0 (socket says 172.21.15.19:50750) 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 -- 172.21.15.19:0/3277224883 learned_addr learned my addr 172.21.15.19:0/3277224883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c70a00640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c6c0cf870 0x7f8c6c1f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 -- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c15d620 msgr2=0x7f8c6c0b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c15d620 0x7f8c6c0b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 -- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c6c0cf870 msgr2=0x7f8c6c1f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c6c0cf870 0x7f8c6c1f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:16.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 -- 172.21.15.19:0/3277224883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c0c75c0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c70a00640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c6c0cf870 0x7f8c6c1f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.834+0000 7f8c71400640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c1f70d0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d5d0 tx=0x7f8c6000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:16.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.835+0000 7f8c69600640 1 -- 172.21.15.19:0/3277224883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60004240 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.835+0000 7f8c69600640 1 -- 172.21.15.19:0/3277224883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c600043e0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.835+0000 7f8c69600640 1 -- 172.21.15.19:0/3277224883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60003830 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.835+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6c153ca0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.835+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6c154260 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.836+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6c076c60 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.838+0000 7f8c69600640 1 -- 172.21.15.19:0/3277224883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8c600039d0 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.839+0000 7f8c69600640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c4c0744b0 0x7f8c4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:16.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.839+0000 7f8c69600640 1 -- 172.21.15.19:0/3277224883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 8711+0+0 (secure 0 0 0) 0x7f8c60096110 con 0x7f8c6c0cdd60 2024-04-25T11:42:16.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.840+0000 7f8c70a00640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c4c0744b0 0x7f8c4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:16.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.841+0000 7f8c70a00640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c4c0744b0 0x7f8c4c0769c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000f60 tx=0x7f8c58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:16.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:16.844+0000 7f8c69600640 1 -- 172.21.15.19:0/3277224883 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c6005fcc0 con 0x7f8c6c0cdd60 2024-04-25T11:42:17.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.049+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8c6c0b4c80 con 0x7f8c6c0cdd60 2024-04-25T11:42:17.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.514+0000 7f8c69600640 1 -- 172.21.15.19:0/3277224883 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) ==== 169+0+0 (secure 0 0 0) 0x7f8c6005f5c0 con 0x7f8c6c0cdd60 2024-04-25T11:42:17.500 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:42:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.517+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c4c0744b0 msgr2=0x7f8c4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.517+0000 7f8c72800640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c4c0744b0 0x7f8c4c0769c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000f60 tx=0x7f8c58009040 comp rx=0 tx=0).stop 2024-04-25T11:42:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.517+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 msgr2=0x7f8c6c1f70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.517+0000 7f8c72800640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c1f70d0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d5d0 tx=0x7f8c6000da90 comp rx=0 tx=0).stop 2024-04-25T11:42:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.517+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 shutdown_connections 2024-04-25T11:42:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.518+0000 7f8c72800640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8c4c0744b0 0x7f8c4c0769c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.518+0000 7f8c72800640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c6c0cf870 0x7f8c6c1f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.518+0000 7f8c72800640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c6c15d620 0x7f8c6c0b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.518+0000 7f8c72800640 1 --2- 172.21.15.19:0/3277224883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c6c0cdd60 0x7f8c6c1f70d0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.518+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 >> 172.21.15.19:0/3277224883 conn(0x7f8c6c072f40 msgr2=0x7f8c6c13f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.518+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 shutdown_connections 2024-04-25T11:42:17.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.518+0000 7f8c72800640 1 -- 172.21.15.19:0/3277224883 wait complete. 2024-04-25T11:42:17.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:42:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.845+0000 7f71e9400640 1 Processor -- start 2024-04-25T11:42:17.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.845+0000 7f71e9400640 1 -- start start 2024-04-25T11:42:17.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e9400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71e4155350 0x7f71e415f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e4161ce0 0x7f71e406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e9400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e4162650 con 0x7f71e41620f0 2024-04-25T11:42:17.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e9400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f71e4162c10 con 0x7f71e4155350 2024-04-25T11:42:17.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71e4162930 con 0x7f71e4161ce0 2024-04-25T11:42:17.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71e4155350 0x7f71e415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f71e4155350 0x7f71e415f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:40880/0 (socket says 172.21.15.19:40880) 2024-04-25T11:42:17.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.849+0000 7f71e2800640 1 -- 172.21.15.19:0/739259933 learned_addr learned my addr 172.21.15.19:0/739259933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.850+0000 7f71e1e00640 1 --2- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e4161ce0 0x7f71e406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.850+0000 7f71e2800640 1 -- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e4161ce0 msgr2=0x7f71e406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.850+0000 7f71e2800640 1 --2- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e4161ce0 0x7f71e406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.851+0000 7f71e2800640 1 -- 172.21.15.19:0/739259933 >> v1:172.21.15.169:6789/0 conn(0x7f71e41620f0 legacy=0x7f71e415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.851+0000 7f71e2800640 1 -- 172.21.15.19:0/739259933 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e415e890 con 0x7f71e4155350 2024-04-25T11:42:17.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.851+0000 7f71e1e00640 1 --2- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e4161ce0 0x7f71e406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:17.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.852+0000 7f71e2800640 1 --2- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e415f090 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009870 tx=0x7f71d402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b84b67ec5bfcda18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.853+0000 7f71e1400640 1 -- 172.21.15.19:0/739259933 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4037070 con 0x7f71e4155350 2024-04-25T11:42:17.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.853+0000 7f71e1400640 1 -- 172.21.15.19:0/739259933 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d4032440 con 0x7f71e4155350 2024-04-25T11:42:17.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.853+0000 7f71e1400640 1 -- 172.21.15.19:0/739259933 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4032810 con 0x7f71e4155350 2024-04-25T11:42:17.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 -- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 msgr2=0x7f71e415f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 --2- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e415f090 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009870 tx=0x7f71d402a210 comp rx=0 tx=0).stop 2024-04-25T11:42:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 -- 172.21.15.19:0/739259933 shutdown_connections 2024-04-25T11:42:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 --2- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e4161ce0 0x7f71e406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 --2- 172.21.15.19:0/739259933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e415f090 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 -- 172.21.15.19:0/739259933 >> 172.21.15.19:0/739259933 conn(0x7f71e4073120 msgr2=0x7f71e4073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 -- 172.21.15.19:0/739259933 shutdown_connections 2024-04-25T11:42:17.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.854+0000 7f71e9400640 1 -- 172.21.15.19:0/739259933 wait complete. 2024-04-25T11:42:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.858+0000 7f71e9400640 1 Processor -- start 2024-04-25T11:42:17.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.858+0000 7f71e9400640 1 -- start start 2024-04-25T11:42:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.860+0000 7f71e9400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e41562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.860+0000 7f71e9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71e4161ce0 0x7f71e4156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.860+0000 7f71e9400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e41620f0 0x7f71e4157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.860+0000 7f71e9400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71e40707b0 con 0x7f71e4161ce0 2024-04-25T11:42:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.860+0000 7f71e9400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71e415f5d0 con 0x7f71e4155350 2024-04-25T11:42:17.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.860+0000 7f71e9400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f71e4070930 con 0x7f71e41620f0 2024-04-25T11:42:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71e4161ce0 0x7f71e4156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71e4161ce0 0x7f71e4156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50760/0 (socket says 172.21.15.19:50760) 2024-04-25T11:42:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 -- 172.21.15.19:0/3090250824 learned_addr learned my addr 172.21.15.19:0/3090250824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e2800640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e41562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e3200640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e41620f0 0x7f71e4157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 -- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e41620f0 msgr2=0x7f71e4157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e41620f0 0x7f71e4157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 -- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 msgr2=0x7f71e41562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e41562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e1e00640 1 -- 172.21.15.19:0/3090250824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e4156d60 con 0x7f71e4161ce0 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.861+0000 7f71e2800640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e41562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.862+0000 7f71e1e00640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71e4161ce0 0x7f71e4156820 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c820 tx=0x7f71cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.862+0000 7f71daa00640 1 -- 172.21.15.19:0/3090250824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc015070 con 0x7f71e4161ce0 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.862+0000 7f71daa00640 1 -- 172.21.15.19:0/3090250824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71cc007a30 con 0x7f71e4161ce0 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.862+0000 7f71daa00640 1 -- 172.21.15.19:0/3090250824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc007de0 con 0x7f71e4161ce0 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.862+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e4157510 con 0x7f71e4161ce0 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.863+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e414c540 con 0x7f71e4161ce0 2024-04-25T11:42:17.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.864+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e40761c0 con 0x7f71e4161ce0 2024-04-25T11:42:17.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.865+0000 7f71daa00640 1 -- 172.21.15.19:0/3090250824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71cc018070 con 0x7f71e4161ce0 2024-04-25T11:42:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.866+0000 7f71daa00640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71c00744b0 0x7f71c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.866+0000 7f71daa00640 1 -- 172.21.15.19:0/3090250824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8297+0+0 (secure 0 0 0) 0x7f71cc0959c0 con 0x7f71e4161ce0 2024-04-25T11:42:17.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.867+0000 7f71e2800640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71c00744b0 0x7f71c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.868+0000 7f71e2800640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71c00744b0 0x7f71c00769c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f71d4034000 tx=0x7f71d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:17.871+0000 7f71daa00640 1 -- 172.21.15.19:0/3090250824 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71cc097e20 con 0x7f71e4161ce0 2024-04-25T11:42:18.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.078+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f71e4050740 con 0x7f71e4161ce0 2024-04-25T11:42:18.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.085+0000 7f71daa00640 1 -- 172.21.15.19:0/3090250824 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f71cc05f1b0 con 0x7f71e4161ce0 2024-04-25T11:42:18.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.088+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71c00744b0 msgr2=0x7f71c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:18.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.088+0000 7f71e9400640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71c00744b0 0x7f71c00769c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f71d4034000 tx=0x7f71d402e000 comp rx=0 tx=0).stop 2024-04-25T11:42:18.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.088+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71e4161ce0 msgr2=0x7f71e4156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.088+0000 7f71e9400640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71e4161ce0 0x7f71e4156820 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00c820 tx=0x7f71cc00cce0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 shutdown_connections 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f71c00744b0 0x7f71c00769c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f71e41620f0 0x7f71e4157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71e4161ce0 0x7f71e4156820 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 --2- 172.21.15.19:0/3090250824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71e4155350 0x7f71e41562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 >> 172.21.15.19:0/3090250824 conn(0x7f71e4073120 msgr2=0x7f71e40b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 shutdown_connections 2024-04-25T11:42:18.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.089+0000 7f71e9400640 1 -- 172.21.15.19:0/3090250824 wait complete. 2024-04-25T11:42:18.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-04-25T11:42:18.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.419+0000 7fe310800640 1 Processor -- start 2024-04-25T11:42:18.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.419+0000 7fe310800640 1 -- start start 2024-04-25T11:42:18.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.421+0000 7fe310800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe308167000 0x7fe308169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:18.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.421+0000 7fe310800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe30815f5f0 con 0x7fe3081443c0 2024-04-25T11:42:18.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.421+0000 7fe310800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe30815e830 con 0x7fe3080cf5e0 2024-04-25T11:42:18.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.421+0000 7fe310800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe3080708d0 con 0x7fe308167000 2024-04-25T11:42:18.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.421+0000 7fe30e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe308167000 0x7fe308169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:18.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.421+0000 7fe30de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe3081443c0 0x7fe30806fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50522/0 (socket says 172.21.15.19:50522) 2024-04-25T11:42:18.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.421+0000 7fe30de00640 1 -- 172.21.15.19:0/639759876 learned_addr learned my addr 172.21.15.19:0/639759876 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:18.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.422+0000 7fe30e800640 1 -- 172.21.15.19:0/639759876 >> v1:172.21.15.169:6789/0 conn(0x7fe3080cf5e0 legacy=0x7fe3080cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:18.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.422+0000 7fe30e800640 1 -- 172.21.15.19:0/639759876 >> v1:172.21.15.19:6789/0 conn(0x7fe3081443c0 legacy=0x7fe30806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:18.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.422+0000 7fe30e800640 1 -- 172.21.15.19:0/639759876 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30815eb10 con 0x7fe308167000 2024-04-25T11:42:18.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.422+0000 7fe30ca00640 1 -- 172.21.15.19:0/639759876 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 881421655 0 0) 0x7fe30815f5f0 con 0x7fe3081443c0 2024-04-25T11:42:18.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.423+0000 7fe30e800640 1 --2- 172.21.15.19:0/639759876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308167000 0x7fe308169420 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fe304009d90 tx=0x7fe30402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488d8ab266e4c01e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:18.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.423+0000 7fe30ca00640 1 -- 172.21.15.19:0/639759876 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304007740 con 0x7fe308167000 2024-04-25T11:42:18.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.423+0000 7fe30ca00640 1 -- 172.21.15.19:0/639759876 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3040078e0 con 0x7fe308167000 2024-04-25T11:42:18.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.423+0000 7fe30ca00640 1 -- 172.21.15.19:0/639759876 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304007cb0 con 0x7fe308167000 2024-04-25T11:42:18.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.424+0000 7fe310800640 1 -- 172.21.15.19:0/639759876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308167000 msgr2=0x7fe308169420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:18.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.424+0000 7fe310800640 1 --2- 172.21.15.19:0/639759876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308167000 0x7fe308169420 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fe304009d90 tx=0x7fe30402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.424+0000 7fe310800640 1 -- 172.21.15.19:0/639759876 shutdown_connections 2024-04-25T11:42:18.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.424+0000 7fe310800640 1 --2- 172.21.15.19:0/639759876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe308167000 0x7fe308169420 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.424+0000 7fe310800640 1 -- 172.21.15.19:0/639759876 >> 172.21.15.19:0/639759876 conn(0x7fe3080d1ae0 msgr2=0x7fe3080d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:18.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.424+0000 7fe310800640 1 -- 172.21.15.19:0/639759876 shutdown_connections 2024-04-25T11:42:18.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.424+0000 7fe310800640 1 -- 172.21.15.19:0/639759876 wait complete. 2024-04-25T11:42:18.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.426+0000 7fe310800640 1 Processor -- start 2024-04-25T11:42:18.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.427+0000 7fe310800640 1 -- start start 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.428+0000 7fe310800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080cf5e0 0x7fe3080c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.428+0000 7fe310800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3081443c0 0x7fe3081517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.428+0000 7fe310800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe308167000 0x7fe30814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.428+0000 7fe310800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe30806d5d0 con 0x7fe3080cf5e0 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.428+0000 7fe310800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe30815f160 con 0x7fe3081443c0 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.428+0000 7fe310800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3081447d0 con 0x7fe308167000 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.428+0000 7fe30de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080cf5e0 0x7fe3080c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080cf5e0 0x7fe3080c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50774/0 (socket says 172.21.15.19:50774) 2024-04-25T11:42:18.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 -- 172.21.15.19:0/1929201974 learned_addr learned my addr 172.21.15.19:0/1929201974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:18.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 -- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe308167000 msgr2=0x7fe30814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:18.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe308167000 0x7fe30814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 -- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3081443c0 msgr2=0x7fe3081517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3081443c0 0x7fe3081517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 -- 172.21.15.19:0/1929201974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3080c6e20 con 0x7fe3080cf5e0 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.429+0000 7fe30de00640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080cf5e0 0x7fe3080c68e0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c410 tx=0x7fe30000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.430+0000 7fe2fe000640 1 -- 172.21.15.19:0/1929201974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3000074e0 con 0x7fe3080cf5e0 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.430+0000 7fe2fe000640 1 -- 172.21.15.19:0/1929201974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3000047e0 con 0x7fe3080cf5e0 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.430+0000 7fe2fe000640 1 -- 172.21.15.19:0/1929201974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300004bd0 con 0x7fe3080cf5e0 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.430+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3080c7330 con 0x7fe3080cf5e0 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.430+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3080c78f0 con 0x7fe3080cf5e0 2024-04-25T11:42:18.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.431+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3080c8010 con 0x7fe3080cf5e0 2024-04-25T11:42:18.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.432+0000 7fe2fe000640 1 -- 172.21.15.19:0/1929201974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe300011070 con 0x7fe3080cf5e0 2024-04-25T11:42:18.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.433+0000 7fe2fe000640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2ec0744e0 0x7fe2ec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:18.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.434+0000 7fe2fe000640 1 -- 172.21.15.19:0/1929201974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8297+0+0 (secure 0 0 0) 0x7fe3000958d0 con 0x7fe3080cf5e0 2024-04-25T11:42:18.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.434+0000 7fe30d400640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2ec0744e0 0x7fe2ec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:18.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.436+0000 7fe30d400640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2ec0744e0 0x7fe2ec0769f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006fd0 tx=0x7fe2f8006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:18.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.438+0000 7fe2fe000640 1 -- 172.21.15.19:0/1929201974 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe30005f620 con 0x7fe3080cf5e0 2024-04-25T11:42:18.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:18.724+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7fe308050740 con 0x7fe3080cf5e0 2024-04-25T11:42:19.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.527+0000 7fe2fe000640 1 -- 172.21.15.19:0/1929201974 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v785) ==== 135+0+0 (secure 0 0 0) 0x7fe30005ef20 con 0x7fe3080cf5e0 2024-04-25T11:42:19.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.532+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2ec0744e0 msgr2=0x7fe2ec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.532+0000 7fe310800640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2ec0744e0 0x7fe2ec0769f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006fd0 tx=0x7fe2f8006bc0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.532+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080cf5e0 msgr2=0x7fe3080c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.532+0000 7fe310800640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080cf5e0 0x7fe3080c68e0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fe30000c410 tx=0x7fe30000c8d0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 shutdown_connections 2024-04-25T11:42:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2ec0744e0 0x7fe2ec0769f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3081443c0 0x7fe3081517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe308167000 0x7fe30814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 --2- 172.21.15.19:0/1929201974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3080cf5e0 0x7fe3080c68e0 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 >> 172.21.15.19:0/1929201974 conn(0x7fe3080d1ae0 msgr2=0x7fe3080b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 shutdown_connections 2024-04-25T11:42:19.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.533+0000 7fe310800640 1 -- 172.21.15.19:0/1929201974 wait complete. 2024-04-25T11:42:19.551 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2024-04-25T11:42:19.551 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2024-04-25T11:42:19.551 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-04-25T11:42:19.552 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.850+0000 7f2d5e400640 1 Processor -- start 2024-04-25T11:42:19.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.850+0000 7f2d5e400640 1 -- start start 2024-04-25T11:42:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.852+0000 7f2d5e400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2d581400a0 0x7f2d580b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.852+0000 7f2d5e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d581431d0 con 0x7f2d58142730 2024-04-25T11:42:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.852+0000 7f2d5e400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d581434b0 con 0x7f2d58070980 2024-04-25T11:42:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.852+0000 7f2d5e400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2d580b6090 con 0x7f2d581400a0 2024-04-25T11:42:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.853+0000 7f2d56a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2d58142730 0x7f2d58070270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50532/0 (socket says 172.21.15.19:50532) 2024-04-25T11:42:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.853+0000 7f2d56a00640 1 -- 172.21.15.19:0/640943499 learned_addr learned my addr 172.21.15.19:0/640943499 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.853+0000 7f2d57400640 1 --2- 172.21.15.19:0/640943499 >> v2:172.21.15.169:3300/0 conn(0x7f2d581400a0 0x7f2d580b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.853+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 328849548 0 0) 0x7f2d581431d0 con 0x7f2d58142730 2024-04-25T11:42:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.853+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d3c005730 con 0x7f2d58142730 2024-04-25T11:42:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 325518336 0 0) 0x7f2d3c005730 con 0x7f2d58142730 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 >> v2:172.21.15.169:3300/0 conn(0x7f2d581400a0 msgr2=0x7f2d580b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 --2- 172.21.15.19:0/640943499 >> v2:172.21.15.169:3300/0 conn(0x7f2d581400a0 0x7f2d580b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 >> v1:172.21.15.169:6789/0 conn(0x7f2d58070980 legacy=0x7f2d580b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d580b6370 con 0x7f2d58142730 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2d44001670 con 0x7f2d58142730 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 >> v1:172.21.15.19:6789/0 conn(0x7f2d58142730 legacy=0x7f2d58070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c005c10 0x7f2d3c006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d3c006a10 0x7f2d3c006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3c007480 0x7f2d3c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d580b6090 con 0x7f2d3c006a10 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c00e3a0 con 0x7f2d3c007480 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c00e5a0 con 0x7f2d3c005c10 2024-04-25T11:42:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.854+0000 7f2d56a00640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d3c006a10 0x7f2d3c006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d57400640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c005c10 0x7f2d3c006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d56a00640 1 -- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c005c10 msgr2=0x7f2d3c006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d56a00640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c005c10 0x7f2d3c006080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d56a00640 1 -- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3c007480 msgr2=0x7f2d3c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d56a00640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d3c007480 0x7f2d3c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d56a00640 1 -- 172.21.15.19:0/640943499 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c005730 con 0x7f2d3c006a10 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d57e00640 1 -- 172.21.15.19:0/640943499 reap_dead start 2024-04-25T11:42:19.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.855+0000 7f2d57400640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c005c10 0x7f2d3c006080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.856+0000 7f2d56a00640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d3c006a10 0x7f2d3c006e20 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c110 tx=0x7f2d3c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc09a91a0c56a1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.856+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44007c40 con 0x7f2d3c006a10 2024-04-25T11:42:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.856+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d44005af0 con 0x7f2d3c006a10 2024-04-25T11:42:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.856+0000 7f2d56000640 1 -- 172.21.15.19:0/640943499 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44007070 con 0x7f2d3c006a10 2024-04-25T11:42:19.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.857+0000 7f2d5e400640 1 -- 172.21.15.19:0/640943499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d3c006a10 msgr2=0x7f2d3c006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.857+0000 7f2d5e400640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d3c006a10 0x7f2d3c006e20 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c110 tx=0x7f2d3c007a40 comp rx=0 tx=0).stop 2024-04-25T11:42:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.857+0000 7f2d5e400640 1 -- 172.21.15.19:0/640943499 shutdown_connections 2024-04-25T11:42:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.857+0000 7f2d5e400640 1 --2- 172.21.15.19:0/640943499 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d3c006a10 0x7f2d3c006e20 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.857+0000 7f2d5e400640 1 -- 172.21.15.19:0/640943499 >> 172.21.15.19:0/640943499 conn(0x7f2d58073050 msgr2=0x7f2d58073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.857+0000 7f2d5e400640 1 -- 172.21.15.19:0/640943499 shutdown_connections 2024-04-25T11:42:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.857+0000 7f2d5e400640 1 -- 172.21.15.19:0/640943499 wait complete. 2024-04-25T11:42:19.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.860+0000 7f2d5e400640 1 Processor -- start 2024-04-25T11:42:19.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.860+0000 7f2d5e400640 1 -- start start 2024-04-25T11:42:19.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.861+0000 7f2d5e400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c006a10 0x7f2d58160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.861+0000 7f2d5e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d58138060 0x7f2d581417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.861+0000 7f2d5e400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d58141d30 0x7f2d58134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.861+0000 7f2d5e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d58142cc0 con 0x7f2d58138060 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.861+0000 7f2d5e400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2d58142b40 con 0x7f2d58141d30 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.861+0000 7f2d5e400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2d58142e40 con 0x7f2d3c006a10 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d58138060 0x7f2d581417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d58138060 0x7f2d581417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50782/0 (socket says 172.21.15.19:50782) 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 -- 172.21.15.19:0/1057061136 learned_addr learned my addr 172.21.15.19:0/1057061136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d57400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c006a10 0x7f2d58160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d57e00640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d58141d30 0x7f2d58134a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 -- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c006a10 msgr2=0x7f2d58160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:19.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c006a10 0x7f2d58160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 -- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d58141d30 msgr2=0x7f2d58134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d58141d30 0x7f2d58134a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 -- 172.21.15.19:0/1057061136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d58142140 con 0x7f2d58138060 2024-04-25T11:42:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d57400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c006a10 0x7f2d58160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:19.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.862+0000 7f2d56a00640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d58138060 0x7f2d581417f0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f2d4402ad60 tx=0x7f2d58143680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.863+0000 7f2d4f400640 1 -- 172.21.15.19:0/1057061136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44037040 con 0x7f2d58138060 2024-04-25T11:42:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.863+0000 7f2d4f400640 1 -- 172.21.15.19:0/1057061136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d44041070 con 0x7f2d58138060 2024-04-25T11:42:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.863+0000 7f2d4f400640 1 -- 172.21.15.19:0/1057061136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44001670 con 0x7f2d58138060 2024-04-25T11:42:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.863+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d58138f00 con 0x7f2d58138060 2024-04-25T11:42:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.863+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d58139650 con 0x7f2d58138060 2024-04-25T11:42:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.864+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d58050740 con 0x7f2d58138060 2024-04-25T11:42:19.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.865+0000 7f2d4f400640 1 -- 172.21.15.19:0/1057061136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d44004a20 con 0x7f2d58138060 2024-04-25T11:42:19.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.866+0000 7f2d4f400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d380744e0 0x7f2d380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.867+0000 7f2d4f400640 1 -- 172.21.15.19:0/1057061136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8297+0+0 (secure 0 0 0) 0x7f2d440b5c10 con 0x7f2d58138060 2024-04-25T11:42:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.867+0000 7f2d57400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d380744e0 0x7f2d380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.869+0000 7f2d57400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d380744e0 0x7f2d380769f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2d5806bc20 tx=0x7f2d48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:19.872+0000 7f2d4f400640 1 -- 172.21.15.19:0/1057061136 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d58142140 con 0x7f2d58138060 2024-04-25T11:42:20.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.073+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d581390d0 con 0x7f2d58138060 2024-04-25T11:42:20.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.075+0000 7f2d4f400640 1 -- 172.21.15.19:0/1057061136 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+18062 (secure 0 0 0) 0x7f2d4407f8e0 con 0x7f2d58138060 2024-04-25T11:42:20.063 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:20.063 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":134,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:17.502861+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6834/478011367":"2024-04-26T11:42:03.371590+0000","172.21.15.19:6835/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/608506291":"2024-04-26T11:41:54.227362+0000","172.21.15.19:6834/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/295899246":"2024-04-26T11:41:18.367356+0000","172.21.15.19:6834/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/698241108":"2024-04-26T11:40:28.419530+0000","172.21.15.19:6835/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6834/2891720740":"2024-04-26T11:40:22.060358+0000","172.21.15.19:6835/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6835/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6835/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/1787178525":"2024-04-26T11:41:45.641501+0000","172.21.15.19:6835/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6835/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6835/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/527442501":"2024-04-26T11:37:46.017825+0000","172.21.15.19:6835/2088909328":"2024-04-26T11:41:11.987047+0000","172.21.15.19:6834/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6835/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6835/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6834/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/3000889873":"2024-04-26T11:39:15.575600+0000","172.21.15.19:6835/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6834/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6834/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/571302363":"2024-04-26T11:39:50.044945+0000","172.21.15.19:6834/2855003271":"2024-04-26T11:36:56.884963+0000","172.21.15.19:6834/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6835/3307408746":"2024-04-26T11:37:23.802616+0000","172.21.15.19:6834/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6835/390026662":"2024-04-26T11:37:15.698053+0000","172.21.15.19:6834/3563649842":"2024-04-26T11:41:39.225887+0000","172.21.15.19:6835/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6834/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/2716758396":"2024-04-26T11:35:28.536900+0000","172.21.15.19:6834/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/1755072450":"2024-04-26T11:39:43.970561+0000","172.21.15.19:6834/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/1160647715":"2024-04-26T11:38:45.171692+0000","172.21.15.19:6834/1743540779":"2024-04-26T11:36:01.298382+0000","172.21.15.19:6834/2103355299":"2024-04-26T11:37:42.702999+0000","172.21.15.19:6835/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6834/2406308121":"2024-04-26T11:39:56.138589+0000","172.21.15.19:6834/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/258414220":"2024-04-26T11:40:56.345928+0000","172.21.15.19:6835/46202637":"2024-04-26T11:35:54.966496+0000","172.21.15.19:6835/2065862570":"2024-04-26T11:38:09.150250+0000","172.21.15.19:6835/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6835/845688948":"2024-04-26T11:37:06.603392+0000","172.21.15.19:6834/3471202301":"2024-04-26T11:37:55.127992+0000","172.21.15.19:6834/432323448":"2024-04-26T11:36:41.348188+0000","172.21.15.19:6835/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6834/3516161211":"2024-04-26T11:36:10.407078+0000","172.21.15.19:6835/3555498549":"2024-04-26T11:36:16.444892+0000","172.21.15.19:6835/2645472884":"2024-04-26T11:36:22.535971+0000","172.21.15.19:6835/2271566376":"2024-04-26T11:38:03.222860+0000","172.21.15.19:6835/614168622":"2024-04-26T11:38:36.047498+0000","172.21.15.19:6834/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6835/896318924":"2024-04-26T11:38:15.379181+0000","172.21.15.19:6834/3710093903":"2024-04-26T11:38:39.094007+0000","172.21.15.19:6835/3430409163":"2024-04-26T11:37:00.526677+0000","172.21.15.19:6834/611091976":"2024-04-26T11:38:51.248979+0000","172.21.15.19:6834/1758836153":"2024-04-26T11:38:57.342691+0000","172.21.15.19:6835/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1735507451":"2024-04-26T11:39:03.424646+0000","172.21.15.19:6834/1200575562":"2024-04-26T11:39:09.170950+0000","172.21.15.19:6834/3564168877":"2024-04-26T11:39:34.386529+0000","172.21.15.19:6834/4272759579":"2024-04-26T11:39:37.896090+0000","172.21.15.19:6835/4272759579":"2024-04-26T11:39:37.896090+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d380744e0 msgr2=0x7f2d380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d380744e0 0x7f2d380769f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2d5806bc20 tx=0x7f2d48008040 comp rx=0 tx=0).stop 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d58138060 msgr2=0x7f2d581417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d58138060 0x7f2d581417f0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f2d4402ad60 tx=0x7f2d58143680 comp rx=0 tx=0).stop 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 shutdown_connections 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2d380744e0 0x7f2d380769f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d58141d30 0x7f2d58134a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d58138060 0x7f2d581417f0 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 --2- 172.21.15.19:0/1057061136 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d3c006a10 0x7f2d58160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.078+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 >> 172.21.15.19:0/1057061136 conn(0x7f2d58073050 msgr2=0x7f2d58073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:20.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.079+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 shutdown_connections 2024-04-25T11:42:20.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.079+0000 7f2d5e400640 1 -- 172.21.15.19:0/1057061136 wait complete. 2024-04-25T11:42:20.091 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:42:20.091 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:42:20.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.387+0000 7f30eb000640 1 Processor -- start 2024-04-25T11:42:20.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.387+0000 7f30eb000640 1 -- start start 2024-04-25T11:42:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.389+0000 7f30eb000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30e40ce520 0x7f30e415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.389+0000 7f30eb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e406dce0 0x7f30e415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.389+0000 7f30eb000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e40cda60 con 0x7f30e40ce930 2024-04-25T11:42:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.389+0000 7f30eb000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f30e415dfe0 con 0x7f30e40ce520 2024-04-25T11:42:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.389+0000 7f30eb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30e415e2c0 con 0x7f30e406dce0 2024-04-25T11:42:20.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.389+0000 7f30e9200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f30e40ce930 0x7f30e406d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50538/0 (socket says 172.21.15.19:50538) 2024-04-25T11:42:20.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.389+0000 7f30e9200640 1 -- 172.21.15.19:0/3213717050 learned_addr learned my addr 172.21.15.19:0/3213717050 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:20.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30ea600640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e406dce0 0x7f30e415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e9c00640 1 --2- 172.21.15.19:0/3213717050 >> v2:172.21.15.169:3300/0 conn(0x7f30e40ce520 0x7f30e415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695549955 0 0) 0x7f30e40cda60 con 0x7f30e40ce930 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30c8003520 con 0x7f30e40ce930 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2269595441 0 0) 0x7f30c8003520 con 0x7f30e40ce930 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e406dce0 msgr2=0x7f30e415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e406dce0 0x7f30e415d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 >> v2:172.21.15.169:3300/0 conn(0x7f30e40ce520 msgr2=0x7f30e415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 --2- 172.21.15.19:0/3213717050 >> v2:172.21.15.169:3300/0 conn(0x7f30e40ce520 0x7f30e415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e40cdd20 con 0x7f30e40ce930 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f30d0001690 con 0x7f30e40ce930 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 >> v1:172.21.15.19:6789/0 conn(0x7f30e40ce930 legacy=0x7f30e406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30c80039b0 0x7f30c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c8004800 0x7f30c8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30c8005150 0x7f30c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30e415dfe0 con 0x7f30c80039b0 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30e415e2c0 con 0x7f30c8004800 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.390+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30c800bf00 con 0x7f30c8005150 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.391+0000 7f30e9c00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30c80039b0 0x7f30c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.391+0000 7f30e9c00640 1 -- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30c8005150 msgr2=0x7f30c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.391+0000 7f30e9c00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30c8005150 0x7f30c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.391+0000 7f30e9c00640 1 -- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c8004800 msgr2=0x7f30c8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.391+0000 7f30e9c00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c8004800 0x7f30c8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.391+0000 7f30e9c00640 1 -- 172.21.15.19:0/3213717050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c8003520 con 0x7f30c80039b0 2024-04-25T11:42:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.392+0000 7f30e9c00640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30c80039b0 0x7f30c8003e20 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009a90 tx=0x7f30c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db2baa40b2d87c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.392+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4036070 con 0x7f30c80039b0 2024-04-25T11:42:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.393+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d4031400 con 0x7f30c80039b0 2024-04-25T11:42:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.393+0000 7f30e3e00640 1 -- 172.21.15.19:0/3213717050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4009690 con 0x7f30c80039b0 2024-04-25T11:42:20.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.393+0000 7f30eb000640 1 -- 172.21.15.19:0/3213717050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30c80039b0 msgr2=0x7f30c8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.393+0000 7f30eb000640 1 --2- 172.21.15.19:0/3213717050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30c80039b0 0x7f30c8003e20 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009a90 tx=0x7f30c8005830 comp rx=0 tx=0).stop 2024-04-25T11:42:20.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.393+0000 7f30ea600640 1 -- 172.21.15.19:0/3213717050 reap_dead start 2024-04-25T11:42:20.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.393+0000 7f30eb000640 1 -- 172.21.15.19:0/3213717050 shutdown_connections 2024-04-25T11:42:20.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.393+0000 7f30eb000640 1 -- 172.21.15.19:0/3213717050 >> 172.21.15.19:0/3213717050 conn(0x7f30e4072f40 msgr2=0x7f30e4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:20.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.394+0000 7f30eb000640 1 -- 172.21.15.19:0/3213717050 shutdown_connections 2024-04-25T11:42:20.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.394+0000 7f30eb000640 1 -- 172.21.15.19:0/3213717050 wait complete. 2024-04-25T11:42:20.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.396+0000 7f30eb000640 1 Processor -- start 2024-04-25T11:42:20.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.396+0000 7f30eb000640 1 -- start start 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30eb000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c80039b0 0x7f30e415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30eb000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e414fdb0 0x7f30e41f9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30eb000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e41f9a40 0x7f30e4157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30eb000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30e40b73d0 con 0x7f30e41f9a40 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30eb000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30e4160d30 con 0x7f30c80039b0 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30eb000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30e40b7550 con 0x7f30e414fdb0 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e41f9a40 0x7f30e4157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e41f9a40 0x7f30e4157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50812/0 (socket says 172.21.15.19:50812) 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 -- 172.21.15.19:0/765111232 learned_addr learned my addr 172.21.15.19:0/765111232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30e9200640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e414fdb0 0x7f30e41f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30e9c00640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c80039b0 0x7f30e415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 -- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e414fdb0 msgr2=0x7f30e41f9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e414fdb0 0x7f30e41f9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 -- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c80039b0 msgr2=0x7f30e415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c80039b0 0x7f30e415a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30ea600640 1 -- 172.21.15.19:0/765111232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e41f9e50 con 0x7f30e41f9a40 2024-04-25T11:42:20.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.398+0000 7f30e9200640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e414fdb0 0x7f30e41f9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.399+0000 7f30ea600640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e41f9a40 0x7f30e4157a70 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004780 tx=0x7f30dc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.399+0000 7f30e1600640 1 -- 172.21.15.19:0/765111232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00bce0 con 0x7f30e41f9a40 2024-04-25T11:42:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.399+0000 7f30e1600640 1 -- 172.21.15.19:0/765111232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30dc01a070 con 0x7f30e41f9a40 2024-04-25T11:42:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.399+0000 7f30e1600640 1 -- 172.21.15.19:0/765111232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc014640 con 0x7f30e41f9a40 2024-04-25T11:42:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.399+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e415a740 con 0x7f30e41f9a40 2024-04-25T11:42:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.399+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e40cbb30 con 0x7f30e41f9a40 2024-04-25T11:42:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.400+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e40cf3e0 con 0x7f30e41f9a40 2024-04-25T11:42:20.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.402+0000 7f30e1600640 1 -- 172.21.15.19:0/765111232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f30dc005d10 con 0x7f30e41f9a40 2024-04-25T11:42:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.403+0000 7f30e1600640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744e0 0x7f30c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.403+0000 7f30e1600640 1 -- 172.21.15.19:0/765111232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8297+0+0 (secure 0 0 0) 0x7f30dc096060 con 0x7f30e41f9a40 2024-04-25T11:42:20.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.403+0000 7f30e9c00640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744e0 0x7f30c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.405+0000 7f30e9c00640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744e0 0x7f30c40769f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f30d4033000 tx=0x7f30d4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:20.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.409+0000 7f30e1600640 1 -- 172.21.15.19:0/765111232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30e40cf3e0 con 0x7f30e41f9a40 2024-04-25T11:42:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.619+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30e4076c60 con 0x7f30e41f9a40 2024-04-25T11:42:20.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.620+0000 7f30e1600640 1 -- 172.21.15.19:0/765111232 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7f30dc024020 con 0x7f30e41f9a40 2024-04-25T11:42:20.607 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:42:20.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744e0 msgr2=0x7f30c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744e0 0x7f30c40769f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f30d4033000 tx=0x7f30d4008030 comp rx=0 tx=0).stop 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e41f9a40 msgr2=0x7f30e4157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e41f9a40 0x7f30e4157a70 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f30c8004780 tx=0x7f30dc00b480 comp rx=0 tx=0).stop 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 shutdown_connections 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30c40744e0 0x7f30c40769f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30e41f9a40 0x7f30e4157a70 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30e414fdb0 0x7f30e41f9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 --2- 172.21.15.19:0/765111232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30c80039b0 0x7f30e415a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 >> 172.21.15.19:0/765111232 conn(0x7f30e4072f40 msgr2=0x7f30e406d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 shutdown_connections 2024-04-25T11:42:20.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.623+0000 7f30eb000640 1 -- 172.21.15.19:0/765111232 wait complete. 2024-04-25T11:42:20.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-04-25T11:42:20.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.938+0000 7f9afcc00640 1 Processor -- start 2024-04-25T11:42:20.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.938+0000 7f9afcc00640 1 -- start start 2024-04-25T11:42:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.940+0000 7f9afcc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9af8144500 0x7f9af8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.940+0000 7f9afcc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af806f5f0 con 0x7f9af80cdb90 2024-04-25T11:42:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.940+0000 7f9afcc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af806f8d0 con 0x7f9af8162310 2024-04-25T11:42:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.940+0000 7f9afcc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9af8070200 con 0x7f9af8144500 2024-04-25T11:42:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.940+0000 7f9af5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9af8144500 0x7f9af8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.941+0000 7f9af5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9af8144500 0x7f9af8070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50828/0 (socket says 172.21.15.19:50828) 2024-04-25T11:42:20.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.941+0000 7f9af5e00640 1 -- 172.21.15.19:0/792160943 learned_addr learned my addr 172.21.15.19:0/792160943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:20.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.941+0000 7f9af5e00640 1 -- 172.21.15.19:0/792160943 >> v1:172.21.15.169:6789/0 conn(0x7f9af8162310 legacy=0x7f9af80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:20.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.941+0000 7f9af5e00640 1 -- 172.21.15.19:0/792160943 >> v1:172.21.15.19:6789/0 conn(0x7f9af80cdb90 legacy=0x7f9af8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.941+0000 7f9af5e00640 1 -- 172.21.15.19:0/792160943 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af806fbb0 con 0x7f9af8144500 2024-04-25T11:42:20.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.941+0000 7f9af4a00640 1 -- 172.21.15.19:0/792160943 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867935013 0 0) 0x7f9af806f5f0 con 0x7f9af80cdb90 2024-04-25T11:42:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.942+0000 7f9af5e00640 1 --2- 172.21.15.19:0/792160943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af8144500 0x7f9af8070690 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05e661f898f09d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.943+0000 7f9af4a00640 1 -- 172.21.15.19:0/792160943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8037070 con 0x7f9af8144500 2024-04-25T11:42:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.943+0000 7f9af4a00640 1 -- 172.21.15.19:0/792160943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae8032440 con 0x7f9af8144500 2024-04-25T11:42:20.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.943+0000 7f9af4a00640 1 -- 172.21.15.19:0/792160943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8032810 con 0x7f9af8144500 2024-04-25T11:42:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.943+0000 7f9afcc00640 1 -- 172.21.15.19:0/792160943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af8144500 msgr2=0x7f9af8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.943+0000 7f9afcc00640 1 --2- 172.21.15.19:0/792160943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af8144500 0x7f9af8070690 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae802a210 comp rx=0 tx=0).stop 2024-04-25T11:42:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.944+0000 7f9afcc00640 1 -- 172.21.15.19:0/792160943 shutdown_connections 2024-04-25T11:42:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.944+0000 7f9afcc00640 1 --2- 172.21.15.19:0/792160943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af8144500 0x7f9af8070690 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.944+0000 7f9afcc00640 1 -- 172.21.15.19:0/792160943 >> 172.21.15.19:0/792160943 conn(0x7f9af8073050 msgr2=0x7f9af8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.944+0000 7f9afcc00640 1 -- 172.21.15.19:0/792160943 shutdown_connections 2024-04-25T11:42:20.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.944+0000 7f9afcc00640 1 -- 172.21.15.19:0/792160943 wait complete. 2024-04-25T11:42:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.946+0000 7f9afcc00640 1 Processor -- start 2024-04-25T11:42:20.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.946+0000 7f9afcc00640 1 -- start start 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9afcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af80cdb90 0x7f9af81f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9afcc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9af8144500 0x7f9af80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9afcc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9af80cc9b0 0x7f9af814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9afcc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9af806f2e0 con 0x7f9af80cdb90 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9afcc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9af806f160 con 0x7f9af8144500 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9afcc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9af80cdfa0 con 0x7f9af80cc9b0 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af80cdb90 0x7f9af81f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af80cdb90 0x7f9af81f9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50844/0 (socket says 172.21.15.19:50844) 2024-04-25T11:42:20.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 -- 172.21.15.19:0/2569293019 learned_addr learned my addr 172.21.15.19:0/2569293019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 -- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9af80cc9b0 msgr2=0x7f9af814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9af80cc9b0 0x7f9af814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 -- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9af8144500 msgr2=0x7f9af80cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9af8144500 0x7f9af80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.948+0000 7f9af5e00640 1 -- 172.21.15.19:0/2569293019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af80ccdc0 con 0x7f9af80cdb90 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.949+0000 7f9af5e00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af80cdb90 0x7f9af81f9b40 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.949+0000 7f9aee000640 1 -- 172.21.15.19:0/2569293019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8037040 con 0x7f9af80cdb90 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.949+0000 7f9aee000640 1 -- 172.21.15.19:0/2569293019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae8004250 con 0x7f9af80cdb90 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.949+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af81fa4a0 con 0x7f9af80cdb90 2024-04-25T11:42:20.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.949+0000 7f9aee000640 1 -- 172.21.15.19:0/2569293019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae803c670 con 0x7f9af80cdb90 2024-04-25T11:42:20.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.949+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81faa60 con 0x7f9af80cdb90 2024-04-25T11:42:20.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.951+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af8050740 con 0x7f9af80cdb90 2024-04-25T11:42:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.952+0000 7f9aee000640 1 -- 172.21.15.19:0/2569293019 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ae803a070 con 0x7f9af80cdb90 2024-04-25T11:42:20.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.953+0000 7f9aee000640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9ad40744b0 0x7f9ad40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:20.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.953+0000 7f9aee000640 1 -- 172.21.15.19:0/2569293019 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8297+0+0 (secure 0 0 0) 0x7f9ae80b6650 con 0x7f9af80cdb90 2024-04-25T11:42:20.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.954+0000 7f9af5400640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9ad40744b0 0x7f9ad40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:20.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.955+0000 7f9af5400640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9ad40744b0 0x7f9ad40769c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3200 tx=0x7f9ae0007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:20.957+0000 7f9aee000640 1 -- 172.21.15.19:0/2569293019 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9af80ccdc0 con 0x7f9af80cdb90 2024-04-25T11:42:21.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.244+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f9af806dae0 con 0x7f9af80cdb90 2024-04-25T11:42:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.542+0000 7f9aee000640 1 -- 172.21.15.19:0/2569293019 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v787) ==== 138+0+0 (secure 0 0 0) 0x7f9ae8041020 con 0x7f9af80cdb90 2024-04-25T11:42:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.546+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9ad40744b0 msgr2=0x7f9ad40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.546+0000 7f9afcc00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9ad40744b0 0x7f9ad40769c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3200 tx=0x7f9ae0007cb0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.546+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af80cdb90 msgr2=0x7f9af81f9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.546+0000 7f9afcc00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af80cdb90 0x7f9af81f9b40 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae80046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 shutdown_connections 2024-04-25T11:42:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9ad40744b0 0x7f9ad40769c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9af8144500 0x7f9af80cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9af80cc9b0 0x7f9af814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 --2- 172.21.15.19:0/2569293019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9af80cdb90 0x7f9af81f9b40 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 >> 172.21.15.19:0/2569293019 conn(0x7f9af8073050 msgr2=0x7f9af80ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 shutdown_connections 2024-04-25T11:42:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.547+0000 7f9afcc00640 1 -- 172.21.15.19:0/2569293019 wait complete. 2024-04-25T11:42:21.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:42:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.871+0000 7f2e54400640 1 Processor -- start 2024-04-25T11:42:21.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.871+0000 7f2e54400640 1 -- start start 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.873+0000 7f2e54400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e4c0cd9a0 0x7f2e4c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.873+0000 7f2e54400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06dce0 0x7f2e4c15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.873+0000 7f2e54400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4c0cdea0 con 0x7f2e4c0b0af0 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.873+0000 7f2e54400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4c0ceab0 con 0x7f2e4c0cd9a0 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.874+0000 7f2e54400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4c0ce180 con 0x7f2e4c06dce0 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.874+0000 7f2e51a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e4c0cd9a0 0x7f2e4c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.874+0000 7f2e51000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2e4c0b0af0 0x7f2e4c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50544/0 (socket says 172.21.15.19:50544) 2024-04-25T11:42:21.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.874+0000 7f2e51000640 1 -- 172.21.15.19:0/2883182531 learned_addr learned my addr 172.21.15.19:0/2883182531 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.874+0000 7f2e52400640 1 --2- 172.21.15.19:0/2883182531 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06dce0 0x7f2e4c15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.875+0000 7f2e51a00640 1 -- 172.21.15.19:0/2883182531 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06dce0 msgr2=0x7f2e4c15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.875+0000 7f2e51a00640 1 --2- 172.21.15.19:0/2883182531 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06dce0 0x7f2e4c15faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.875+0000 7f2e51a00640 1 -- 172.21.15.19:0/2883182531 >> v1:172.21.15.19:6789/0 conn(0x7f2e4c0b0af0 legacy=0x7f2e4c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.875+0000 7f2e51a00640 1 -- 172.21.15.19:0/2883182531 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c0ce460 con 0x7f2e4c0cd9a0 2024-04-25T11:42:21.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.875+0000 7f2e4be00640 1 -- 172.21.15.19:0/2883182531 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245655001 0 0) 0x7f2e4c0cdea0 con 0x7f2e4c0b0af0 2024-04-25T11:42:21.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.876+0000 7f2e51a00640 1 --2- 172.21.15.19:0/2883182531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c15f360 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c148020 tx=0x7f2e4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c795d0095ad269f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.877+0000 7f2e4be00640 1 -- 172.21.15.19:0/2883182531 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40037070 con 0x7f2e4c0cd9a0 2024-04-25T11:42:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.877+0000 7f2e4be00640 1 -- 172.21.15.19:0/2883182531 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e40032440 con 0x7f2e4c0cd9a0 2024-04-25T11:42:21.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.877+0000 7f2e4be00640 1 -- 172.21.15.19:0/2883182531 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40032810 con 0x7f2e4c0cd9a0 2024-04-25T11:42:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.877+0000 7f2e54400640 1 -- 172.21.15.19:0/2883182531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c0cd9a0 msgr2=0x7f2e4c15f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.878+0000 7f2e54400640 1 --2- 172.21.15.19:0/2883182531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c15f360 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c148020 tx=0x7f2e4002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.878+0000 7f2e54400640 1 -- 172.21.15.19:0/2883182531 shutdown_connections 2024-04-25T11:42:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.878+0000 7f2e54400640 1 --2- 172.21.15.19:0/2883182531 >> v2:172.21.15.169:3300/0 conn(0x7f2e4c06dce0 0x7f2e4c15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.878+0000 7f2e54400640 1 --2- 172.21.15.19:0/2883182531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c15f360 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.878+0000 7f2e54400640 1 -- 172.21.15.19:0/2883182531 >> 172.21.15.19:0/2883182531 conn(0x7f2e4c073090 msgr2=0x7f2e4c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.878+0000 7f2e54400640 1 -- 172.21.15.19:0/2883182531 shutdown_connections 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.878+0000 7f2e54400640 1 -- 172.21.15.19:0/2883182531 wait complete. 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.880+0000 7f2e54400640 1 Processor -- start 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.880+0000 7f2e54400640 1 -- start start 2024-04-25T11:42:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e54400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c0b0af0 0x7f2e4c1e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e54400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e54400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c143050 0x7f2e4c143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e54400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0cef40 con 0x7f2e4c143050 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e54400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0b0f00 con 0x7f2e4c0cd9a0 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e54400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c0cf0c0 con 0x7f2e4c0b0af0 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e52400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c143050 0x7f2e4c143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e52400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c143050 0x7f2e4c143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50866/0 (socket says 172.21.15.19:50866) 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.882+0000 7f2e52400640 1 -- 172.21.15.19:0/2563367953 learned_addr learned my addr 172.21.15.19:0/2563367953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e51a00640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c0b0af0 0x7f2e4c1e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e51000640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e52400640 1 -- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c0b0af0 msgr2=0x7f2e4c1e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e52400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c0b0af0 0x7f2e4c1e8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e52400640 1 -- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c0cd9a0 msgr2=0x7f2e4c157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e52400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e52400640 1 -- 172.21.15.19:0/2563367953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c13f560 con 0x7f2e4c143050 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e51a00640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c0b0af0 0x7f2e4c1e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e51000640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e52400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c143050 0x7f2e4c143460 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400efc0 tx=0x7f2e4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e49600640 1 -- 172.21.15.19:0/2563367953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44018020 con 0x7f2e4c143050 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e49600640 1 -- 172.21.15.19:0/2563367953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e44014480 con 0x7f2e4c143050 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e49600640 1 -- 172.21.15.19:0/2563367953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44014870 con 0x7f2e4c143050 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.883+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c141e90 con 0x7f2e4c143050 2024-04-25T11:42:21.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.884+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c13f1f0 con 0x7f2e4c143050 2024-04-25T11:42:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.885+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c050740 con 0x7f2e4c143050 2024-04-25T11:42:21.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.886+0000 7f2e49600640 1 -- 172.21.15.19:0/2563367953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2e44005d10 con 0x7f2e4c143050 2024-04-25T11:42:21.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.887+0000 7f2e49600640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e2c0744b0 0x7f2e2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.887+0000 7f2e49600640 1 -- 172.21.15.19:0/2563367953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8297+0+0 (secure 0 0 0) 0x7f2e44096ab0 con 0x7f2e4c143050 2024-04-25T11:42:21.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.888+0000 7f2e51a00640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e2c0744b0 0x7f2e2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.889+0000 7f2e51a00640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e2c0744b0 0x7f2e2c0769c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2e40034000 tx=0x7f2e4002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:21.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:21.892+0000 7f2e49600640 1 -- 172.21.15.19:0/2563367953 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e44022030 con 0x7f2e4c143050 2024-04-25T11:42:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.098+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2e4c142d30 con 0x7f2e4c143050 2024-04-25T11:42:22.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.105+0000 7f2e49600640 1 -- 172.21.15.19:0/2563367953 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f2e4c142d30 con 0x7f2e4c143050 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e2c0744b0 msgr2=0x7f2e2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e2c0744b0 0x7f2e2c0769c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2e40034000 tx=0x7f2e4002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c143050 msgr2=0x7f2e4c143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c143050 0x7f2e4c143460 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400efc0 tx=0x7f2e4400c490 comp rx=0 tx=0).stop 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 shutdown_connections 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e2c0744b0 0x7f2e2c0769c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e4c0cd9a0 0x7f2e4c157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.108+0000 7f2e54400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e4c143050 0x7f2e4c143460 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.109+0000 7f2e54400640 1 --2- 172.21.15.19:0/2563367953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2e4c0b0af0 0x7f2e4c1e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.109+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 >> 172.21.15.19:0/2563367953 conn(0x7f2e4c073090 msgr2=0x7f2e4c06d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:22.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.109+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 shutdown_connections 2024-04-25T11:42:22.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.109+0000 7f2e54400640 1 -- 172.21.15.19:0/2563367953 wait complete. 2024-04-25T11:42:22.123 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:42:22.123 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:42:22.236 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:42:22.352 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:42:22.352 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:22.352 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:22.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:42:22.415 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:42:22.415 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:22.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:42:22.473 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:22.473 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:22.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:42:22.531 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:42:22.531 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:22.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:42:22.586 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:42:22.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:22.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.891+0000 7f9e26200640 1 Processor -- start 2024-04-25T11:42:22.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.891+0000 7f9e26200640 1 -- start start 2024-04-25T11:42:22.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e26200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9e200d4570 0x7f9e200b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e26200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e200d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e26200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9e200d09f0 0x7f9e200b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e26200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2006e460 con 0x7f9e200d09f0 2024-04-25T11:42:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e26200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2006d870 con 0x7f9e200d4570 2024-04-25T11:42:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e26200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2006db10 con 0x7f9e2006e8f0 2024-04-25T11:42:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e1fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9e200d09f0 0x7f9e200b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e1fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9e200d09f0 0x7f9e200b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50874/0 (socket says 172.21.15.19:50874) 2024-04-25T11:42:22.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.894+0000 7f9e1fe00640 1 -- 172.21.15.19:0/2236850901 learned_addr learned my addr 172.21.15.19:0/2236850901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.895+0000 7f9e1f400640 1 --2- 172.21.15.19:0/2236850901 >> v2:172.21.15.169:3300/0 conn(0x7f9e200d4570 0x7f9e200b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.895+0000 7f9e1ea00640 1 --2- 172.21.15.19:0/2236850901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e200d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.895+0000 7f9e1fe00640 1 -- 172.21.15.19:0/2236850901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 msgr2=0x7f9e200d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.895+0000 7f9e1fe00640 1 --2- 172.21.15.19:0/2236850901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e200d04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.895+0000 7f9e1fe00640 1 -- 172.21.15.19:0/2236850901 >> v2:172.21.15.169:3300/0 conn(0x7f9e200d4570 msgr2=0x7f9e200b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.895+0000 7f9e1fe00640 1 --2- 172.21.15.19:0/2236850901 >> v2:172.21.15.169:3300/0 conn(0x7f9e200d4570 0x7f9e200b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.895+0000 7f9e1fe00640 1 -- 172.21.15.19:0/2236850901 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2006e0f0 con 0x7f9e200d09f0 2024-04-25T11:42:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.896+0000 7f9e1fe00640 1 --2- 172.21.15.19:0/2236850901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d09f0 0x7f9e200b2dc0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f9e2008e040 tx=0x7f9e1802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f240a151193edf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.897+0000 7f9e1e000640 1 -- 172.21.15.19:0/2236850901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18037070 con 0x7f9e200d09f0 2024-04-25T11:42:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.897+0000 7f9e1e000640 1 -- 172.21.15.19:0/2236850901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e18032440 con 0x7f9e200d09f0 2024-04-25T11:42:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.897+0000 7f9e1e000640 1 -- 172.21.15.19:0/2236850901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18032810 con 0x7f9e200d09f0 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.897+0000 7f9e26200640 1 -- 172.21.15.19:0/2236850901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d09f0 msgr2=0x7f9e200b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.897+0000 7f9e26200640 1 --2- 172.21.15.19:0/2236850901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d09f0 0x7f9e200b2dc0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f9e2008e040 tx=0x7f9e1802a140 comp rx=0 tx=0).stop 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.897+0000 7f9e26200640 1 -- 172.21.15.19:0/2236850901 shutdown_connections 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.898+0000 7f9e26200640 1 --2- 172.21.15.19:0/2236850901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d09f0 0x7f9e200b2dc0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.898+0000 7f9e26200640 1 --2- 172.21.15.19:0/2236850901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e200d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.898+0000 7f9e26200640 1 --2- 172.21.15.19:0/2236850901 >> v2:172.21.15.169:3300/0 conn(0x7f9e200d4570 0x7f9e200b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.898+0000 7f9e26200640 1 -- 172.21.15.19:0/2236850901 >> 172.21.15.19:0/2236850901 conn(0x7f9e200d3a50 msgr2=0x7f9e200d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.898+0000 7f9e26200640 1 -- 172.21.15.19:0/2236850901 shutdown_connections 2024-04-25T11:42:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.898+0000 7f9e26200640 1 -- 172.21.15.19:0/2236850901 wait complete. 2024-04-25T11:42:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.900+0000 7f9e26200640 1 Processor -- start 2024-04-25T11:42:22.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.900+0000 7f9e26200640 1 -- start start 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e26200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e201f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e26200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e200d09f0 0x7f9e201f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e26200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d4570 0x7f9e200b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e26200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e200b3300 con 0x7f9e200d4570 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e26200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2006ed00 con 0x7f9e200d09f0 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e26200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9e200b3480 con 0x7f9e2006e8f0 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e1fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d4570 0x7f9e200b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e1fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d4570 0x7f9e200b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50884/0 (socket says 172.21.15.19:50884) 2024-04-25T11:42:22.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e1fe00640 1 -- 172.21.15.19:0/373406320 learned_addr learned my addr 172.21.15.19:0/373406320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e1f400640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e201f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.902+0000 7f9e1ea00640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e200d09f0 0x7f9e201f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e1fe00640 1 -- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 msgr2=0x7f9e201f95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e1fe00640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e201f95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e1fe00640 1 -- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e200d09f0 msgr2=0x7f9e201f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e1fe00640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e200d09f0 0x7f9e201f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e1fe00640 1 -- 172.21.15.19:0/373406320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e201f7510 con 0x7f9e200d4570 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e1f400640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e201f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:22.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e1fe00640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d4570 0x7f9e200b78f0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f9e1802a180 tx=0x7f9e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e17400640 1 -- 172.21.15.19:0/373406320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18037040 con 0x7f9e200d4570 2024-04-25T11:42:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e17400640 1 -- 172.21.15.19:0/373406320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e18042070 con 0x7f9e200d4570 2024-04-25T11:42:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e17400640 1 -- 172.21.15.19:0/373406320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18030900 con 0x7f9e200d4570 2024-04-25T11:42:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.903+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2009dcc0 con 0x7f9e200d4570 2024-04-25T11:42:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.904+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2009e280 con 0x7f9e200d4570 2024-04-25T11:42:22.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.905+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2009da30 con 0x7f9e200d4570 2024-04-25T11:42:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.906+0000 7f9e17400640 1 -- 172.21.15.19:0/373406320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9e18030aa0 con 0x7f9e200d4570 2024-04-25T11:42:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.907+0000 7f9e17400640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e000744e0 0x7f9e000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.907+0000 7f9e17400640 1 -- 172.21.15.19:0/373406320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8297+0+0 (secure 0 0 0) 0x7f9e180b6680 con 0x7f9e200d4570 2024-04-25T11:42:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.908+0000 7f9e1f400640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e000744e0 0x7f9e000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.909+0000 7f9e1f400640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e000744e0 0x7f9e000769f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009ad0 tx=0x7f9e100039d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:22.912+0000 7f9e17400640 1 -- 172.21.15.19:0/373406320 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e18080340 con 0x7f9e200d4570 2024-04-25T11:42:23.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.157+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e20050740 con 0x7f9e200d4570 2024-04-25T11:42:23.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.159+0000 7f9e17400640 1 -- 172.21.15.19:0/373406320 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 357 v357) ==== 95+0+3878 (secure 0 0 0) 0x7f9e1807fc40 con 0x7f9e200d4570 2024-04-25T11:42:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:23.147 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":357,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5621,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/4181530484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":4181530484},{"type":"v1","addr":"172.21.15.169:6837","nonce":4181530484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/1772097713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":1772097713},{"type":"v1","addr":"172.21.15.169:6835","nonce":1772097713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/2952831873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2952831873},{"type":"v1","addr":"172.21.15.19:6833","nonce":2952831873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":11202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2689099865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2689099865},{"type":"v1","addr":"172.21.15.19:6835","nonce":2689099865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":355}],"filesystems":[]} 2024-04-25T11:42:23.147 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 357 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e000744e0 msgr2=0x7f9e000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e000744e0 0x7f9e000769f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009ad0 tx=0x7f9e100039d0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d4570 msgr2=0x7f9e200b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d4570 0x7f9e200b78f0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f9e1802a180 tx=0x7f9e180046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 shutdown_connections 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9e000744e0 0x7f9e000769f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9e200d09f0 0x7f9e201f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e200d4570 0x7f9e200b78f0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 --2- 172.21.15.19:0/373406320 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9e2006e8f0 0x7f9e201f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 >> 172.21.15.19:0/373406320 conn(0x7f9e200d3a50 msgr2=0x7f9e200cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 shutdown_connections 2024-04-25T11:42:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.162+0000 7f9e26200640 1 -- 172.21.15.19:0/373406320 wait complete. 2024-04-25T11:42:23.179 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:42:23.179 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:42:23.179 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:42:23.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:42:23.179 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:42:23.180 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:42:23.180 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:42:23.180 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:42:23.180 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:42:23.180 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:42:23.180 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:42:23.180 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:42:23.181 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:42:23.181 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:42:23.181 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:42:23.181 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:42:23.211 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:42:23.211 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:42:23.212 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:42:23.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:42:23.224 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:42:23.224 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:42:23.230 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:42:23.230 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:42:23.271 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:42:23.271 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:23.277 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:42:23.277 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:23.277 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:42:23.278 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:23.283 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:42:23.283 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:23.283 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:42:23.498 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:42:23.498 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-04-25T11:42:23.507 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:42:23.516 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2024-04-25T11:42:23.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.553+0000 7f8084a00640 1 Processor -- start 2024-04-25T11:42:23.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.553+0000 7f8084a00640 1 -- start start 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.555+0000 7f8084a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f80800cf2f0 0x7f80800b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.555+0000 7f8084a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f808015a810 0x7f80800b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.555+0000 7f8084a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80800b4e90 con 0x7f80801608d0 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.555+0000 7f8084a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f808006f870 con 0x7f808015a810 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.555+0000 7f8084a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f80800b74d0 con 0x7f80800cf2f0 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f80800cf2f0 0x7f80800b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807f400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f80800cf2f0 0x7f80800b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:41032/0 (socket says 172.21.15.19:41032) 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807f400640 1 -- 172.21.15.19:0/3702604787 learned_addr learned my addr 172.21.15.19:0/3702604787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807ea00640 1 --2- 172.21.15.19:0/3702604787 >> v2:172.21.15.19:3300/0 conn(0x7f808015a810 0x7f80800b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:23.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807f400640 1 -- 172.21.15.19:0/3702604787 >> v2:172.21.15.19:3300/0 conn(0x7f808015a810 msgr2=0x7f80800b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807f400640 1 --2- 172.21.15.19:0/3702604787 >> v2:172.21.15.19:3300/0 conn(0x7f808015a810 0x7f80800b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807f400640 1 -- 172.21.15.19:0/3702604787 >> v1:172.21.15.169:6789/0 conn(0x7f80801608d0 legacy=0x7f8080160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.556+0000 7f807f400640 1 -- 172.21.15.19:0/3702604787 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80800b77b0 con 0x7f80800cf2f0 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.558+0000 7f807f400640 1 --2- 172.21.15.19:0/3702604787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 0x7f80800b6e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f80800d26d0 tx=0x7f807002c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f93a2a52cea2392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.558+0000 7f807e000640 1 -- 172.21.15.19:0/3702604787 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070036020 con 0x7f80800cf2f0 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.558+0000 7f807e000640 1 -- 172.21.15.19:0/3702604787 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8070032830 con 0x7f80800cf2f0 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.558+0000 7f807e000640 1 -- 172.21.15.19:0/3702604787 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070032c00 con 0x7f80800cf2f0 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 -- 172.21.15.19:0/3702604787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 msgr2=0x7f80800b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 --2- 172.21.15.19:0/3702604787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 0x7f80800b6e10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f80800d26d0 tx=0x7f807002c930 comp rx=0 tx=0).stop 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 -- 172.21.15.19:0/3702604787 shutdown_connections 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 --2- 172.21.15.19:0/3702604787 >> v2:172.21.15.19:3300/0 conn(0x7f808015a810 0x7f80800b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 --2- 172.21.15.19:0/3702604787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 0x7f80800b6e10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 -- 172.21.15.19:0/3702604787 >> 172.21.15.19:0/3702604787 conn(0x7f80800d1720 msgr2=0x7f80800d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 -- 172.21.15.19:0/3702604787 shutdown_connections 2024-04-25T11:42:23.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.559+0000 7f8084a00640 1 -- 172.21.15.19:0/3702604787 wait complete. 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.562+0000 7f8084a00640 1 Processor -- start 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.562+0000 7f8084a00640 1 -- start start 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.563+0000 7f8084a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 0x7f8080159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.563+0000 7f8084a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f808015a810 0x7f80801f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.563+0000 7f8084a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80801608d0 0x7f808015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.563+0000 7f8084a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f808015ac20 con 0x7f808015a810 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.563+0000 7f8084a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f808006f500 con 0x7f80800cf2f0 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.563+0000 7f8084a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f808006fd00 con 0x7f80801608d0 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f808015a810 0x7f80801f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f808015a810 0x7f80801f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50936/0 (socket says 172.21.15.19:50936) 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 -- 172.21.15.19:0/3715895821 learned_addr learned my addr 172.21.15.19:0/3715895821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807f400640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 0x7f8080159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:23.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807fe00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80801608d0 0x7f808015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 -- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80801608d0 msgr2=0x7f808015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80801608d0 0x7f808015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 -- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 msgr2=0x7f8080159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 0x7f8080159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807ea00640 1 -- 172.21.15.19:0/3715895821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80801f6a70 con 0x7f808015a810 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.564+0000 7f807fe00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80801608d0 0x7f808015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.565+0000 7f807ea00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f808015a810 0x7f80801f6530 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f806c00e870 tx=0x7f806c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.568+0000 7f8077400640 1 -- 172.21.15.19:0/3715895821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c017070 con 0x7f808015a810 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.568+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801f6f80 con 0x7f808015a810 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.568+0000 7f8077400640 1 -- 172.21.15.19:0/3715895821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806c00c8e0 con 0x7f808015a810 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.568+0000 7f8077400640 1 -- 172.21.15.19:0/3715895821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c00ccd0 con 0x7f808015a810 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.568+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80800cbed0 con 0x7f808015a810 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.569+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80801f6cd0 con 0x7f808015a810 2024-04-25T11:42:23.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.572+0000 7f8077400640 1 -- 172.21.15.19:0/3715895821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f806c01a070 con 0x7f808015a810 2024-04-25T11:42:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.573+0000 7f8077400640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80600744b0 0x7f80600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.573+0000 7f8077400640 1 -- 172.21.15.19:0/3715895821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 8297+0+0 (secure 0 0 0) 0x7f806c095eb0 con 0x7f808015a810 2024-04-25T11:42:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.573+0000 7f807f400640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80600744b0 0x7f80600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.574+0000 7f807f400640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80600744b0 0x7f80600769c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8070037000 tx=0x7f8070032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:23.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.579+0000 7f8077400640 1 -- 172.21.15.19:0/3715895821 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f806c05f520 con 0x7f808015a810 2024-04-25T11:42:23.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:23.793+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8080050740 con 0x7f808015a810 2024-04-25T11:42:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.582+0000 7f8077400640 1 -- 172.21.15.19:0/3715895821 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7f806c063820 con 0x7f808015a810 2024-04-25T11:42:24.570 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-04-25T11:42:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80600744b0 msgr2=0x7f80600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80600744b0 0x7f80600769c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8070037000 tx=0x7f8070032000 comp rx=0 tx=0).stop 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f808015a810 msgr2=0x7f80801f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f808015a810 0x7f80801f6530 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f806c00e870 tx=0x7f806c00ed30 comp rx=0 tx=0).stop 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 shutdown_connections 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f80600744b0 0x7f80600769c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f80801608d0 0x7f808015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f808015a810 0x7f80801f6530 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 --2- 172.21.15.19:0/3715895821 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f80800cf2f0 0x7f8080159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 >> 172.21.15.19:0/3715895821 conn(0x7f80800d1720 msgr2=0x7f80800b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 shutdown_connections 2024-04-25T11:42:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.587+0000 7f8084a00640 1 -- 172.21.15.19:0/3715895821 wait complete. 2024-04-25T11:42:24.602 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:42:24.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.892+0000 7f6b57e00640 1 Processor -- start 2024-04-25T11:42:24.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.892+0000 7f6b57e00640 1 -- start start 2024-04-25T11:42:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.894+0000 7f6b57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b58162160 0x7f6b58140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.894+0000 7f6b57e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b58144010 0x7f6b5806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.894+0000 7f6b57e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b580b5980 con 0x7f6b580b5570 2024-04-25T11:42:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.894+0000 7f6b57e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6b580b5c10 con 0x7f6b58144010 2024-04-25T11:42:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.894+0000 7f6b57e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b580b5ef0 con 0x7f6b58162160 2024-04-25T11:42:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.895+0000 7f6b4e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b58162160 0x7f6b58140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:24.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.895+0000 7f6b57400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b58144010 0x7f6b5806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:24.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.895+0000 7f6b57400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6b58144010 0x7f6b5806c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:60450/0 (socket says 172.21.15.19:60450) 2024-04-25T11:42:24.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.895+0000 7f6b57400640 1 -- 172.21.15.19:0/4006389947 learned_addr learned my addr 172.21.15.19:0/4006389947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:24.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.895+0000 7f6b4e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b58162160 0x7f6b58140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:56976/0 (socket says 172.21.15.19:56976) 2024-04-25T11:42:24.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.896+0000 7f6b57400640 1 -- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b58162160 msgr2=0x7f6b58140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:24.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.896+0000 7f6b57400640 1 --2- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b58162160 0x7f6b58140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.896+0000 7f6b57400640 1 -- 172.21.15.19:0/4006389947 >> v1:172.21.15.169:6789/0 conn(0x7f6b580b5570 legacy=0x7f6b58161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:24.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.896+0000 7f6b57400640 1 -- 172.21.15.19:0/4006389947 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b580b61d0 con 0x7f6b58144010 2024-04-25T11:42:24.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.896+0000 7f6b4e000640 1 --2- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b58162160 0x7f6b58140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:24.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.897+0000 7f6b57400640 1 --2- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b58144010 0x7f6b5806c8a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009e50 tx=0x7f6b5002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dee8ba6b86b33a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:24.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.898+0000 7f6b56000640 1 -- 172.21.15.19:0/4006389947 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50036070 con 0x7f6b58144010 2024-04-25T11:42:24.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.898+0000 7f6b56000640 1 -- 172.21.15.19:0/4006389947 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b50031600 con 0x7f6b58144010 2024-04-25T11:42:24.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.898+0000 7f6b56000640 1 -- 172.21.15.19:0/4006389947 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500319d0 con 0x7f6b58144010 2024-04-25T11:42:24.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 -- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b58144010 msgr2=0x7f6b5806c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:24.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 --2- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b58144010 0x7f6b5806c8a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009e50 tx=0x7f6b5002a710 comp rx=0 tx=0).stop 2024-04-25T11:42:24.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 -- 172.21.15.19:0/4006389947 shutdown_connections 2024-04-25T11:42:24.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 --2- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b58144010 0x7f6b5806c8a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 --2- 172.21.15.19:0/4006389947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b58162160 0x7f6b58140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 -- 172.21.15.19:0/4006389947 >> 172.21.15.19:0/4006389947 conn(0x7f6b580d1b40 msgr2=0x7f6b580d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:24.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 -- 172.21.15.19:0/4006389947 shutdown_connections 2024-04-25T11:42:24.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.899+0000 7f6b57e00640 1 -- 172.21.15.19:0/4006389947 wait complete. 2024-04-25T11:42:24.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.901+0000 7f6b57e00640 1 Processor -- start 2024-04-25T11:42:24.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.902+0000 7f6b57e00640 1 -- start start 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b57e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b580b5570 0x7f6b580cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b57e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b58144010 0x7f6b5813f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b57e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b581423c0 0x7f6b581427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b57e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b58144420 con 0x7f6b58144010 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b57e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b580b6790 con 0x7f6b580b5570 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b57e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5806cde0 con 0x7f6b581423c0 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b4e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b58144010 0x7f6b5813f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b4e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b58144010 0x7f6b5813f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40560/0 (socket says 172.21.15.19:40560) 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.903+0000 7f6b4e000640 1 -- 172.21.15.19:0/1496870229 learned_addr learned my addr 172.21.15.19:0/1496870229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:24.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b57400640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b581423c0 0x7f6b581427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b56a00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b580b5570 0x7f6b580cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b4e000640 1 -- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b581423c0 msgr2=0x7f6b581427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b4e000640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b581423c0 0x7f6b581427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b4e000640 1 -- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b580b5570 msgr2=0x7f6b580cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b4e000640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b580b5570 0x7f6b580cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b4e000640 1 -- 172.21.15.19:0/1496870229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b58142d10 con 0x7f6b58144010 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b56a00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b580b5570 0x7f6b580cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b57400640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b581423c0 0x7f6b581427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.904+0000 7f6b4e000640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b58144010 0x7f6b5813f070 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000e990 tx=0x7f6b4000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.905+0000 7f6b4f400640 1 -- 172.21.15.19:0/1496870229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40017070 con 0x7f6b58144010 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.905+0000 7f6b4f400640 1 -- 172.21.15.19:0/1496870229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4000c9e0 con 0x7f6b58144010 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.905+0000 7f6b4f400640 1 -- 172.21.15.19:0/1496870229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000cdb0 con 0x7f6b58144010 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.905+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b581447f0 con 0x7f6b58144010 2024-04-25T11:42:24.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.905+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b58144db0 con 0x7f6b58144010 2024-04-25T11:42:24.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.906+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b58050740 con 0x7f6b58144010 2024-04-25T11:42:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.908+0000 7f6b4f400640 1 -- 172.21.15.19:0/1496870229 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b4001a070 con 0x7f6b58144010 2024-04-25T11:42:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.909+0000 7f6b4f400640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b340744e0 0x7f6b340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.909+0000 7f6b4f400640 1 -- 172.21.15.19:0/1496870229 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4857+0+0 (secure 0 0 0) 0x7f6b40095180 con 0x7f6b58144010 2024-04-25T11:42:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.909+0000 7f6b56a00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b340744e0 0x7f6b340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.910+0000 7f6b56a00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b340744e0 0x7f6b340769f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6b480015e0 tx=0x7f6b48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:24.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:24.914+0000 7f6b4f400640 1 -- 172.21.15.19:0/1496870229 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b58142d10 con 0x7f6b58144010 2024-04-25T11:42:25.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.189+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6b580cf960 con 0x7f6b58144010 2024-04-25T11:42:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.190+0000 7f6b4f400640 1 -- 172.21.15.19:0/1496870229 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f6b4005fc60 con 0x7f6b58144010 2024-04-25T11:42:25.178 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:25.178 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:42:25.178 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-04-25T11:42:25.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.178 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-04-25T11:42:25.178 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpm1NZGLhAA6zgjJBviKyJqbSTdhaL7lQ==", 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmJu+jGhAAYzZtETok9dw2y7PJ8/9FHw==", 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.179 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmVA9ZBhAAxLBPreFJTQNpzCgRZN9Zcw==", 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.180 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDPPSpmGXW4KxAAbq0iy1RSr+U8Ejz3I4I+eA==", 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:42:25.181 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUPSpmeiEmHRAAqIYZ3fWkBEBzsjWc0Ju6rQ==", 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDXPSpmiJ2tLxAAj6c2xnNw2a8KdI/KHmEAHw==", 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbPSpmdiyGFBAAM+GonPojMcZ98Z25uZgS2Q==", 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.183 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDePSpm/8sCJRAAoCV1dVq2+vvIxgID6xbpjw==", 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDlPSpmQDc9ABAA0ot0xNGmOV7RMSqcAtWdJw==", 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDoPSpmuOrHEBAATA1qssd3Jg9bIBD3Hcrmrw==", 2024-04-25T11:42:25.185 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDrPSpmfV2cIRAAJf6cIBEl43vs9qM+vg6qaQ==", 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.186 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDuPSpmL6TyMBAAT78eFujedxPiReDJI1PADA==", 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.187 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w==", 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmnDU5HhAAhEopNLMoPg1O09rzC4aU+Q==", 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.188 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmCXYJEBAAjmusu3J+EkoSFFCDQrJheg==", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:42:25.189 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDQPSpmFtERLBAA6R9IIQouoZvvBemL55pe+Q==", 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDNPSpmHT+MHRAAPZNLFcl/fxDgWnMeN3nAHw==", 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-04-25T11:42:25.191 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmwTkuHhAAqjWIXfqDJySN6JqeDchGmA==", 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDOPSpmLeg1DxAArVgKG5MU4d8NN/X9njXQNA==", 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-04-25T11:42:25.192 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.193 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:25.193 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:42:25.193 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b340744e0 msgr2=0x7f6b340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b340744e0 0x7f6b340769f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f6b480015e0 tx=0x7f6b48009040 comp rx=0 tx=0).stop 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b58144010 msgr2=0x7f6b5813f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b58144010 0x7f6b5813f070 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000e990 tx=0x7f6b4000ee50 comp rx=0 tx=0).stop 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 shutdown_connections 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6b340744e0 0x7f6b340769f0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6b581423c0 0x7f6b581427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b58144010 0x7f6b5813f070 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 --2- 172.21.15.19:0/1496870229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b580b5570 0x7f6b580cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 >> 172.21.15.19:0/1496870229 conn(0x7f6b580d1b40 msgr2=0x7f6b580684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 shutdown_connections 2024-04-25T11:42:25.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.193+0000 7f6b57e00640 1 -- 172.21.15.19:0/1496870229 wait complete. 2024-04-25T11:42:25.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:25.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.517+0000 7f2af4400640 1 Processor -- start 2024-04-25T11:42:25.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.517+0000 7f2af4400640 1 -- start start 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af4400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2aec1443c0 0x7f2aec06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af4400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2aec070370 0x7f2aec0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af4400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0cff30 con 0x7f2aec0cfb20 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af4400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec0708d0 con 0x7f2aec070370 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af4400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec15f160 con 0x7f2aec1443c0 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2aec070370 0x7f2aec0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2aec070370 0x7f2aec0cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40562/0 (socket says 172.21.15.19:40562) 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af1000640 1 -- 172.21.15.19:0/320382845 learned_addr learned my addr 172.21.15.19:0/320382845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.520+0000 7f2af1a00640 1 --2- 172.21.15.19:0/320382845 >> v2:172.21.15.169:3300/0 conn(0x7f2aec1443c0 0x7f2aec06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.521+0000 7f2af1000640 1 -- 172.21.15.19:0/320382845 >> v2:172.21.15.169:3300/0 conn(0x7f2aec1443c0 msgr2=0x7f2aec06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:25.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.521+0000 7f2af1000640 1 --2- 172.21.15.19:0/320382845 >> v2:172.21.15.169:3300/0 conn(0x7f2aec1443c0 0x7f2aec06fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.521+0000 7f2af1000640 1 -- 172.21.15.19:0/320382845 >> v1:172.21.15.169:6789/0 conn(0x7f2aec0cfb20 legacy=0x7f2aec0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:25.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.521+0000 7f2af1000640 1 -- 172.21.15.19:0/320382845 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec15f5f0 con 0x7f2aec070370 2024-04-25T11:42:25.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.522+0000 7f2af1000640 1 --2- 172.21.15.19:0/320382845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec070370 0x7f2aec0cf5e0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f2aec072820 tx=0x7f2ad802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89374788f3cc2f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:25.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.522+0000 7f2aebe00640 1 -- 172.21.15.19:0/320382845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8037070 con 0x7f2aec070370 2024-04-25T11:42:25.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.522+0000 7f2aebe00640 1 -- 172.21.15.19:0/320382845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8032440 con 0x7f2aec070370 2024-04-25T11:42:25.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.522+0000 7f2aebe00640 1 -- 172.21.15.19:0/320382845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8032810 con 0x7f2aec070370 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.523+0000 7f2af4400640 1 -- 172.21.15.19:0/320382845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec070370 msgr2=0x7f2aec0cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.523+0000 7f2af4400640 1 --2- 172.21.15.19:0/320382845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec070370 0x7f2aec0cf5e0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f2aec072820 tx=0x7f2ad802a230 comp rx=0 tx=0).stop 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.523+0000 7f2af4400640 1 -- 172.21.15.19:0/320382845 shutdown_connections 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.523+0000 7f2af4400640 1 --2- 172.21.15.19:0/320382845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec070370 0x7f2aec0cf5e0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.523+0000 7f2af4400640 1 --2- 172.21.15.19:0/320382845 >> v2:172.21.15.169:3300/0 conn(0x7f2aec1443c0 0x7f2aec06fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.523+0000 7f2af4400640 1 -- 172.21.15.19:0/320382845 >> 172.21.15.19:0/320382845 conn(0x7f2aec0d1ae0 msgr2=0x7f2aec0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.523+0000 7f2af4400640 1 -- 172.21.15.19:0/320382845 shutdown_connections 2024-04-25T11:42:25.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.524+0000 7f2af4400640 1 -- 172.21.15.19:0/320382845 wait complete. 2024-04-25T11:42:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.526+0000 7f2af4400640 1 Processor -- start 2024-04-25T11:42:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.526+0000 7f2af4400640 1 -- start start 2024-04-25T11:42:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.527+0000 7f2af4400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec070370 0x7f2aec1f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.527+0000 7f2af4400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec0cfb20 0x7f2aec0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af4400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec1443c0 0x7f2aec1f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af4400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec15edf0 con 0x7f2aec1443c0 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af4400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec15eb10 con 0x7f2aec070370 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af4400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec06d5d0 con 0x7f2aec0cfb20 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec1443c0 0x7f2aec1f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec1443c0 0x7f2aec1f9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40568/0 (socket says 172.21.15.19:40568) 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 -- 172.21.15.19:0/3291553714 learned_addr learned my addr 172.21.15.19:0/3291553714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af1000640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec0cfb20 0x7f2aec0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af1a00640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec070370 0x7f2aec1f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 -- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec0cfb20 msgr2=0x7f2aec0b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec0cfb20 0x7f2aec0b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 -- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec070370 msgr2=0x7f2aec1f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec070370 0x7f2aec1f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.528+0000 7f2af2400640 1 -- 172.21.15.19:0/3291553714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec1faa00 con 0x7f2aec1443c0 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.529+0000 7f2af2400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec1443c0 0x7f2aec1f9df0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400b9a0 tx=0x7f2ae400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.529+0000 7f2ae9600640 1 -- 172.21.15.19:0/3291553714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4019070 con 0x7f2aec1443c0 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.529+0000 7f2ae9600640 1 -- 172.21.15.19:0/3291553714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae400cba0 con 0x7f2aec1443c0 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.529+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec1face0 con 0x7f2aec1443c0 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.529+0000 7f2ae9600640 1 -- 172.21.15.19:0/3291553714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4014440 con 0x7f2aec1443c0 2024-04-25T11:42:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.529+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1f6bf0 con 0x7f2aec1443c0 2024-04-25T11:42:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.531+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec1fa770 con 0x7f2aec1443c0 2024-04-25T11:42:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.532+0000 7f2ae9600640 1 -- 172.21.15.19:0/3291553714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ae401c070 con 0x7f2aec1443c0 2024-04-25T11:42:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.533+0000 7f2ae9600640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2acc0744e0 0x7f2acc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.533+0000 7f2ae9600640 1 -- 172.21.15.19:0/3291553714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4857+0+0 (secure 0 0 0) 0x7f2ae4094b50 con 0x7f2aec1443c0 2024-04-25T11:42:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.534+0000 7f2af1a00640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2acc0744e0 0x7f2acc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.535+0000 7f2af1a00640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2acc0744e0 0x7f2acc0769f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0006fd0 tx=0x7f2ae0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.537+0000 7f2ae9600640 1 -- 172.21.15.19:0/3291553714 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ae405f620 con 0x7f2aec1443c0 2024-04-25T11:42:25.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.745+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2aec050740 con 0x7f2aec1443c0 2024-04-25T11:42:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.746+0000 7f2ae9600640 1 -- 172.21.15.19:0/3291553714 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+12893 (secure 0 0 0) 0x7f2ae405ef20 con 0x7f2aec1443c0 2024-04-25T11:42:25.734 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:25.734 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":139,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:24.580605+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2acc0744e0 msgr2=0x7f2acc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2acc0744e0 0x7f2acc0769f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0006fd0 tx=0x7f2ae0008040 comp rx=0 tx=0).stop 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec1443c0 msgr2=0x7f2aec1f9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec1443c0 0x7f2aec1f9df0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400b9a0 tx=0x7f2ae400be60 comp rx=0 tx=0).stop 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 shutdown_connections 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2acc0744e0 0x7f2acc0769f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2aec1443c0 0x7f2aec1f9df0 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2aec0cfb20 0x7f2aec0b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 --2- 172.21.15.19:0/3291553714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2aec070370 0x7f2aec1f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.749+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 >> 172.21.15.19:0/3291553714 conn(0x7f2aec0d1ae0 msgr2=0x7f2aec15ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.750+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 shutdown_connections 2024-04-25T11:42:25.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:25.750+0000 7f2af4400640 1 -- 172.21.15.19:0/3291553714 wait complete. 2024-04-25T11:42:25.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:42:25.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:42:26.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.066+0000 7fa935e00640 1 Processor -- start 2024-04-25T11:42:26.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.066+0000 7fa935e00640 1 -- start start 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.069+0000 7fa935e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa93006dce0 0x7fa93015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.069+0000 7fa935e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa9300b0af0 0x7fa93015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.069+0000 7fa935e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9300cdea0 con 0x7fa9300cd9a0 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.069+0000 7fa935e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa9300ceab0 con 0x7fa9300b0af0 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.069+0000 7fa935e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9300ce180 con 0x7fa93006dce0 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.069+0000 7fa92ea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa9300cd9a0 0x7fa93006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51624/0 (socket says 172.21.15.19:51624) 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.069+0000 7fa92ea00640 1 -- 172.21.15.19:0/1953681613 learned_addr learned my addr 172.21.15.19:0/1953681613 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92e000640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa93006dce0 0x7fa93015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 54241564 0 0) 0x7fa9300cdea0 con 0x7fa9300cd9a0 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92f400640 1 --2- 172.21.15.19:0/1953681613 >> v2:172.21.15.169:3300/0 conn(0x7fa9300b0af0 0x7fa93015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa914003520 con 0x7fa9300cd9a0 2024-04-25T11:42:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2476942261 0 0) 0x7fa914003520 con 0x7fa9300cd9a0 2024-04-25T11:42:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa93006dce0 msgr2=0x7fa93015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa93006dce0 0x7fa93015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 >> v2:172.21.15.169:3300/0 conn(0x7fa9300b0af0 msgr2=0x7fa93015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 --2- 172.21.15.19:0/1953681613 >> v2:172.21.15.169:3300/0 conn(0x7fa9300b0af0 0x7fa93015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9300ce460 con 0x7fa9300cd9a0 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fa920001670 con 0x7fa9300cd9a0 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92f400640 1 --2- 172.21.15.19:0/1953681613 >> v2:172.21.15.169:3300/0 conn(0x7fa9300b0af0 0x7fa93015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 >> v1:172.21.15.19:6789/0 conn(0x7fa9300cd9a0 legacy=0x7fa93006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.070+0000 7fa92d600640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9140039b0 0x7fa914003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92d600640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914004800 0x7fa914004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92d600640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa914005150 0x7fa91400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9300ceab0 con 0x7fa914005150 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9300ce180 con 0x7fa9140039b0 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa91400bf00 con 0x7fa914004800 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92f400640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa914005150 0x7fa91400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92e000640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914004800 0x7fa914004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92ea00640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9140039b0 0x7fa914003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92f400640 1 -- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914004800 msgr2=0x7fa914004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92f400640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914004800 0x7fa914004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92f400640 1 -- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9140039b0 msgr2=0x7fa914003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92f400640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9140039b0 0x7fa914003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92f400640 1 -- 172.21.15.19:0/1953681613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa914003520 con 0x7fa914005150 2024-04-25T11:42:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.071+0000 7fa92e000640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914004800 0x7fa914004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.072+0000 7fa92f400640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa914005150 0x7fa91400b9c0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fa930071580 tx=0x7fa914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0d75e6553abe2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.073+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928009c80 con 0x7fa914005150 2024-04-25T11:42:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.073+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92802c620 con 0x7fa914005150 2024-04-25T11:42:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.073+0000 7fa92d600640 1 -- 172.21.15.19:0/1953681613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92802c9d0 con 0x7fa914005150 2024-04-25T11:42:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.073+0000 7fa935e00640 1 -- 172.21.15.19:0/1953681613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa914005150 msgr2=0x7fa91400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.073+0000 7fa935e00640 1 --2- 172.21.15.19:0/1953681613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa914005150 0x7fa91400b9c0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fa930071580 tx=0x7fa914005830 comp rx=0 tx=0).stop 2024-04-25T11:42:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.074+0000 7fa92f400640 1 -- 172.21.15.19:0/1953681613 reap_dead start 2024-04-25T11:42:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.074+0000 7fa935e00640 1 -- 172.21.15.19:0/1953681613 shutdown_connections 2024-04-25T11:42:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.074+0000 7fa935e00640 1 -- 172.21.15.19:0/1953681613 >> 172.21.15.19:0/1953681613 conn(0x7fa930073090 msgr2=0x7fa9300734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.074+0000 7fa935e00640 1 -- 172.21.15.19:0/1953681613 shutdown_connections 2024-04-25T11:42:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.074+0000 7fa935e00640 1 -- 172.21.15.19:0/1953681613 wait complete. 2024-04-25T11:42:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.076+0000 7fa935e00640 1 Processor -- start 2024-04-25T11:42:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.076+0000 7fa935e00640 1 -- start start 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa935e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914005150 0x7fa93013b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa935e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa93013bc10 0x7fa93013d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa935e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa93013ddc0 0x7fa9300b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa935e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9300cef40 con 0x7fa93013bc10 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa935e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa9300b0f00 con 0x7fa93013ddc0 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa935e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa9300cf0c0 con 0x7fa914005150 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa92e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa93013bc10 0x7fa93013d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa92e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa93013bc10 0x7fa93013d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40590/0 (socket says 172.21.15.19:40590) 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.078+0000 7fa92e000640 1 -- 172.21.15.19:0/169442283 learned_addr learned my addr 172.21.15.19:0/169442283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92ea00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914005150 0x7fa93013b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92f400640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa93013ddc0 0x7fa9300b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92e000640 1 -- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914005150 msgr2=0x7fa93013b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92e000640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914005150 0x7fa93013b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92e000640 1 -- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa93013ddc0 msgr2=0x7fa9300b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92e000640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa93013ddc0 0x7fa9300b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92e000640 1 -- 172.21.15.19:0/169442283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9300b5830 con 0x7fa93013bc10 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.079+0000 7fa92e000640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa93013bc10 0x7fa93013d880 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0108e0 tx=0x7fa91c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.080+0000 7fa927400640 1 -- 172.21.15.19:0/169442283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00f040 con 0x7fa93013bc10 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.080+0000 7fa927400640 1 -- 172.21.15.19:0/169442283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c00ebc0 con 0x7fa93013bc10 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.080+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93013f970 con 0x7fa93013bc10 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.080+0000 7fa927400640 1 -- 172.21.15.19:0/169442283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c01c400 con 0x7fa93013bc10 2024-04-25T11:42:26.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.080+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93013ff30 con 0x7fa93013bc10 2024-04-25T11:42:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.081+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa930050740 con 0x7fa93013bc10 2024-04-25T11:42:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.082+0000 7fa927400640 1 -- 172.21.15.19:0/169442283 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa91c01c5a0 con 0x7fa93013bc10 2024-04-25T11:42:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.083+0000 7fa927400640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9100744e0 0x7fa9100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.084+0000 7fa927400640 1 -- 172.21.15.19:0/169442283 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4857+0+0 (secure 0 0 0) 0x7fa91c095630 con 0x7fa93013bc10 2024-04-25T11:42:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.084+0000 7fa92ea00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9100744e0 0x7fa9100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.086+0000 7fa92ea00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9100744e0 0x7fa9100769f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa920005a30 tx=0x7fa920009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.089+0000 7fa927400640 1 -- 172.21.15.19:0/169442283 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa91c018030 con 0x7fa93013bc10 2024-04-25T11:42:26.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.294+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa93006c760 con 0x7fa93013bc10 2024-04-25T11:42:26.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.295+0000 7fa927400640 1 -- 172.21.15.19:0/169442283 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) ==== 93+0+10 (secure 0 0 0) 0x7fa91c060100 con 0x7fa93013bc10 2024-04-25T11:42:26.283 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:42:26.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9100744e0 msgr2=0x7fa9100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9100744e0 0x7fa9100769f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa920005a30 tx=0x7fa920009040 comp rx=0 tx=0).stop 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa93013bc10 msgr2=0x7fa93013d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa93013bc10 0x7fa93013d880 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0108e0 tx=0x7fa91c010da0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 shutdown_connections 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa9100744e0 0x7fa9100769f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa93013ddc0 0x7fa9300b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa93013bc10 0x7fa93013d880 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 --2- 172.21.15.19:0/169442283 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa914005150 0x7fa93013b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 >> 172.21.15.19:0/169442283 conn(0x7fa930073090 msgr2=0x7fa93015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 shutdown_connections 2024-04-25T11:42:26.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.298+0000 7fa935e00640 1 -- 172.21.15.19:0/169442283 wait complete. 2024-04-25T11:42:26.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:42:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.605+0000 7efbfec00640 1 Processor -- start 2024-04-25T11:42:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.606+0000 7efbfec00640 1 -- start start 2024-04-25T11:42:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbfec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efbf8144500 0x7efbf8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbfec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8162310 0x7efbf80ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbfec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbf806f5f0 con 0x7efbf80cdb90 2024-04-25T11:42:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbfec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efbf8070200 con 0x7efbf8144500 2024-04-25T11:42:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbfec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efbf806f8d0 con 0x7efbf8162310 2024-04-25T11:42:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbf7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efbf8144500 0x7efbf8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbf7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efbf8144500 0x7efbf8070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40598/0 (socket says 172.21.15.19:40598) 2024-04-25T11:42:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbf7e00640 1 -- 172.21.15.19:0/2659246884 learned_addr learned my addr 172.21.15.19:0/2659246884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:26.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.608+0000 7efbef400640 1 --1- 172.21.15.19:0/2659246884 >> v1:172.21.15.19:6789/0 conn(0x7efbf80cdb90 0x7efbf8161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51640/0 (socket says 172.21.15.19:51640) 2024-04-25T11:42:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.609+0000 7efbf7e00640 1 -- 172.21.15.19:0/2659246884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8162310 msgr2=0x7efbf80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.609+0000 7efbfcc00640 1 --2- 172.21.15.19:0/2659246884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8162310 0x7efbf80ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.609+0000 7efbf7e00640 1 --2- 172.21.15.19:0/2659246884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8162310 0x7efbf80ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.609+0000 7efbf7e00640 1 -- 172.21.15.19:0/2659246884 >> v1:172.21.15.19:6789/0 conn(0x7efbf80cdb90 legacy=0x7efbf8161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.609+0000 7efbf7e00640 1 -- 172.21.15.19:0/2659246884 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf806fbb0 con 0x7efbf8144500 2024-04-25T11:42:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.609+0000 7efbfcc00640 1 --2- 172.21.15.19:0/2659246884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8162310 0x7efbf80ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:26.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.609+0000 7efbf7400640 1 -- 172.21.15.19:0/2659246884 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597396400 0 0) 0x7efbf806f5f0 con 0x7efbf80cdb90 2024-04-25T11:42:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.610+0000 7efbf7e00640 1 --2- 172.21.15.19:0/2659246884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf8144500 0x7efbf8070690 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009e50 tx=0x7efbe802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f25c63df55dbb95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.610+0000 7efbf7400640 1 -- 172.21.15.19:0/2659246884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8036070 con 0x7efbf8144500 2024-04-25T11:42:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.610+0000 7efbf7400640 1 -- 172.21.15.19:0/2659246884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe8031600 con 0x7efbf8144500 2024-04-25T11:42:26.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.610+0000 7efbf7400640 1 -- 172.21.15.19:0/2659246884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe80319d0 con 0x7efbf8144500 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 -- 172.21.15.19:0/2659246884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf8144500 msgr2=0x7efbf8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 --2- 172.21.15.19:0/2659246884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf8144500 0x7efbf8070690 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009e50 tx=0x7efbe802a710 comp rx=0 tx=0).stop 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 -- 172.21.15.19:0/2659246884 shutdown_connections 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 --2- 172.21.15.19:0/2659246884 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf8162310 0x7efbf80ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 --2- 172.21.15.19:0/2659246884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf8144500 0x7efbf8070690 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 -- 172.21.15.19:0/2659246884 >> 172.21.15.19:0/2659246884 conn(0x7efbf8073050 msgr2=0x7efbf8073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 -- 172.21.15.19:0/2659246884 shutdown_connections 2024-04-25T11:42:26.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.611+0000 7efbfec00640 1 -- 172.21.15.19:0/2659246884 wait complete. 2024-04-25T11:42:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.614+0000 7efbfec00640 1 Processor -- start 2024-04-25T11:42:26.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.614+0000 7efbfec00640 1 -- start start 2024-04-25T11:42:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.615+0000 7efbfec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf80cdb90 0x7efbf81577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.615+0000 7efbfec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf8144500 0x7efbf8157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbfec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf80b5820 0x7efbf8158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbfec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efbf806f2e0 con 0x7efbf80cdb90 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbfec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efbf806f160 con 0x7efbf8144500 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbfec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7efbf80cdfa0 con 0x7efbf80b5820 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf80cdb90 0x7efbf81577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf80cdb90 0x7efbf81577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40600/0 (socket says 172.21.15.19:40600) 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 -- 172.21.15.19:0/2725800157 learned_addr learned my addr 172.21.15.19:0/2725800157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 -- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf80b5820 msgr2=0x7efbf8158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf80b5820 0x7efbf8158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 -- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf8144500 msgr2=0x7efbf8157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf8144500 0x7efbf8157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.616+0000 7efbf7e00640 1 -- 172.21.15.19:0/2725800157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf8153940 con 0x7efbf80cdb90 2024-04-25T11:42:26.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.617+0000 7efbf7e00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf80cdb90 0x7efbf81577a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7efbe8034040 tx=0x7efbe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.617+0000 7efbf4c00640 1 -- 172.21.15.19:0/2725800157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8036050 con 0x7efbf80cdb90 2024-04-25T11:42:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.617+0000 7efbf4c00640 1 -- 172.21.15.19:0/2725800157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe8042070 con 0x7efbf80cdb90 2024-04-25T11:42:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.617+0000 7efbf4c00640 1 -- 172.21.15.19:0/2725800157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8009510 con 0x7efbf80cdb90 2024-04-25T11:42:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.617+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf80b6000 con 0x7efbf80cdb90 2024-04-25T11:42:26.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.617+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf80cb0f0 con 0x7efbf80cdb90 2024-04-25T11:42:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.618+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf815fb90 con 0x7efbf80cdb90 2024-04-25T11:42:26.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.619+0000 7efbf4c00640 1 -- 172.21.15.19:0/2725800157 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efbe8007c50 con 0x7efbf80cdb90 2024-04-25T11:42:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.620+0000 7efbf4c00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbd40744b0 0x7efbd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.620+0000 7efbf4c00640 1 -- 172.21.15.19:0/2725800157 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4857+0+0 (secure 0 0 0) 0x7efbe80b5e20 con 0x7efbf80cdb90 2024-04-25T11:42:26.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.621+0000 7efbef400640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbd40744b0 0x7efbd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.622+0000 7efbef400640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbd40744b0 0x7efbd40769c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efbe0007fd0 tx=0x7efbe0007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.625+0000 7efbf4c00640 1 -- 172.21.15.19:0/2725800157 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efbf815fb90 con 0x7efbf80cdb90 2024-04-25T11:42:26.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.828+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efbf8153b10 con 0x7efbf80cdb90 2024-04-25T11:42:26.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.830+0000 7efbf4c00640 1 -- 172.21.15.19:0/2725800157 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) ==== 84+0+65 (secure 0 0 0) 0x7efbe8080330 con 0x7efbf80cdb90 2024-04-25T11:42:26.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:26.818 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:42:26.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:26.819 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T11:42:26.819 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T11:42:26.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:26.819 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:42:26.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbd40744b0 msgr2=0x7efbd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbd40744b0 0x7efbd40769c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efbe0007fd0 tx=0x7efbe0007960 comp rx=0 tx=0).stop 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf80cdb90 msgr2=0x7efbf81577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf80cdb90 0x7efbf81577a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7efbe8034040 tx=0x7efbe8004a40 comp rx=0 tx=0).stop 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 shutdown_connections 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7efbd40744b0 0x7efbd40769c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf8144500 0x7efbf8157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efbf80b5820 0x7efbf8158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 --2- 172.21.15.19:0/2725800157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbf80cdb90 0x7efbf81577a0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 >> 172.21.15.19:0/2725800157 conn(0x7efbf8073050 msgr2=0x7efbf81609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 shutdown_connections 2024-04-25T11:42:26.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:26.833+0000 7efbfec00640 1 -- 172.21.15.19:0/2725800157 wait complete. 2024-04-25T11:42:26.849 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:42:26.849 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:42:27.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.151+0000 7f7037200640 1 Processor -- start 2024-04-25T11:42:27.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.151+0000 7f7037200640 1 -- start start 2024-04-25T11:42:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.153+0000 7f7037200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f70300d1f50 0x7f70300d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.153+0000 7f7037200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70300d28a0 0x7f70300cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.153+0000 7f7037200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7030074840 con 0x7f703006e470 2024-04-25T11:42:27.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.153+0000 7f7037200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f703006ef90 con 0x7f70300d1f50 2024-04-25T11:42:27.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.153+0000 7f7037200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f703009d010 con 0x7f70300d28a0 2024-04-25T11:42:27.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.153+0000 7f7035e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f703006e470 0x7f703006e880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51642/0 (socket says 172.21.15.19:51642) 2024-04-25T11:42:27.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.153+0000 7f7035e00640 1 -- 172.21.15.19:0/3499726021 learned_addr learned my addr 172.21.15.19:0/3499726021 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7036800640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70300d28a0 0x7f70300cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7035400640 1 --2- 172.21.15.19:0/3499726021 >> v2:172.21.15.169:3300/0 conn(0x7f70300d1f50 0x7f70300d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959216088 0 0) 0x7f7030074840 con 0x7f703006e470 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f701c003520 con 0x7f703006e470 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 705637619 0 0) 0x7f701c003520 con 0x7f703006e470 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70300d28a0 msgr2=0x7f70300cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70300d28a0 0x7f70300cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 >> v2:172.21.15.169:3300/0 conn(0x7f70300d1f50 msgr2=0x7f70300d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 --2- 172.21.15.19:0/3499726021 >> v2:172.21.15.169:3300/0 conn(0x7f70300d1f50 0x7f70300d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70300b9110 con 0x7f703006e470 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7028001b90 con 0x7f703006e470 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 >> v1:172.21.15.19:6789/0 conn(0x7f703006e470 legacy=0x7f703006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f701c0039b0 0x7f701c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c004800 0x7f701c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70300d28a0 0x7f701c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f703009d010 con 0x7f701c0039b0 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f701c00b9c0 con 0x7f701c004800 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.154+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f701c00bb70 con 0x7f70300d28a0 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035e00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f701c0039b0 0x7f701c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035400640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c004800 0x7f701c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035e00640 1 -- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70300d28a0 msgr2=0x7f701c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035e00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70300d28a0 0x7f701c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035e00640 1 -- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c004800 msgr2=0x7f701c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035e00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c004800 0x7f701c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035e00640 1 -- 172.21.15.19:0/3499726021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c003520 con 0x7f701c0039b0 2024-04-25T11:42:27.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.155+0000 7f7035400640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c004800 0x7f701c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.156+0000 7f7035e00640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f701c0039b0 0x7f701c003e20 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f7030074a10 tx=0x7f701c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5cfc64de0afeb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.157+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028032020 con 0x7f701c0039b0 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.157+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7028005d40 con 0x7f701c0039b0 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.157+0000 7f7034a00640 1 -- 172.21.15.19:0/3499726021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702800c100 con 0x7f701c0039b0 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.157+0000 7f7037200640 1 -- 172.21.15.19:0/3499726021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f701c0039b0 msgr2=0x7f701c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.157+0000 7f7037200640 1 --2- 172.21.15.19:0/3499726021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f701c0039b0 0x7f701c003e20 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f7030074a10 tx=0x7f701c005830 comp rx=0 tx=0).stop 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.157+0000 7f7036800640 1 -- 172.21.15.19:0/3499726021 reap_dead start 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.158+0000 7f7037200640 1 -- 172.21.15.19:0/3499726021 shutdown_connections 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.158+0000 7f7037200640 1 -- 172.21.15.19:0/3499726021 >> 172.21.15.19:0/3499726021 conn(0x7f70300b53f0 msgr2=0x7f70300b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.158+0000 7f7037200640 1 -- 172.21.15.19:0/3499726021 shutdown_connections 2024-04-25T11:42:27.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.158+0000 7f7037200640 1 -- 172.21.15.19:0/3499726021 wait complete. 2024-04-25T11:42:27.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.160+0000 7f7037200640 1 Processor -- start 2024-04-25T11:42:27.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.161+0000 7f7037200640 1 -- start start 2024-04-25T11:42:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.162+0000 7f7037200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0039b0 0x7f70301e1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.162+0000 7f7037200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70301d4e30 0x7f70300caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.162+0000 7f7037200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70301e1d70 0x7f70300caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.162+0000 7f7037200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70300d3100 con 0x7f70301e1d70 2024-04-25T11:42:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.162+0000 7f7037200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f70300b93d0 con 0x7f701c0039b0 2024-04-25T11:42:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.162+0000 7f7037200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7030074210 con 0x7f70301d4e30 2024-04-25T11:42:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70301e1d70 0x7f70300caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70301e1d70 0x7f70300caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40620/0 (socket says 172.21.15.19:40620) 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 -- 172.21.15.19:0/497202413 learned_addr learned my addr 172.21.15.19:0/497202413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7035e00640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0039b0 0x7f70301e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7035400640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70301d4e30 0x7f70300caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 -- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70301d4e30 msgr2=0x7f70300caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70301d4e30 0x7f70300caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 -- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0039b0 msgr2=0x7f70301e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0039b0 0x7f70301e1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 -- 172.21.15.19:0/497202413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70301e24b0 con 0x7f70301e1d70 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7035400640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70301d4e30 0x7f70300caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.163+0000 7f7036800640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70301e1d70 0x7f70300caf80 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f70300b5320 tx=0x7f702c00e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.164+0000 7f7026000640 1 -- 172.21.15.19:0/497202413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c013680 con 0x7f70301e1d70 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.164+0000 7f7026000640 1 -- 172.21.15.19:0/497202413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c01a070 con 0x7f70301e1d70 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.164+0000 7f7026000640 1 -- 172.21.15.19:0/497202413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c013a60 con 0x7f70301e1d70 2024-04-25T11:42:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.164+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70301e2790 con 0x7f70301e1d70 2024-04-25T11:42:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.164+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70301e2d50 con 0x7f70301e1d70 2024-04-25T11:42:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.165+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70301e2220 con 0x7f70301e1d70 2024-04-25T11:42:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.166+0000 7f7026000640 1 -- 172.21.15.19:0/497202413 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f702c013c00 con 0x7f70301e1d70 2024-04-25T11:42:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.167+0000 7f7026000640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744e0 0x7f70140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.167+0000 7f7026000640 1 -- 172.21.15.19:0/497202413 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4857+0+0 (secure 0 0 0) 0x7f702c094b60 con 0x7f70301e1d70 2024-04-25T11:42:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.168+0000 7f7035e00640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744e0 0x7f70140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.170+0000 7f7035e00640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744e0 0x7f70140769f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7028033000 tx=0x7f702802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.172+0000 7f7026000640 1 -- 172.21.15.19:0/497202413 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f702c024020 con 0x7f70301e1d70 2024-04-25T11:42:27.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.374+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7030076c60 con 0x7f70301e1d70 2024-04-25T11:42:27.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.607+0000 7f7026000640 1 -- 172.21.15.19:0/497202413 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) ==== 138+0+0 (secure 0 0 0) 0x7f702c05f630 con 0x7f70301e1d70 2024-04-25T11:42:27.594 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744e0 msgr2=0x7f70140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744e0 0x7f70140769f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7028033000 tx=0x7f702802e000 comp rx=0 tx=0).stop 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70301e1d70 msgr2=0x7f70300caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70301e1d70 0x7f70300caf80 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f70300b5320 tx=0x7f702c00e9f0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 shutdown_connections 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f70140744e0 0x7f70140769f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70301e1d70 0x7f70300caf80 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70301d4e30 0x7f70300caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 --2- 172.21.15.19:0/497202413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f701c0039b0 0x7f70301e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.611+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 >> 172.21.15.19:0/497202413 conn(0x7f70300b53f0 msgr2=0x7f70300b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.612+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 shutdown_connections 2024-04-25T11:42:27.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.612+0000 7f7037200640 1 -- 172.21.15.19:0/497202413 wait complete. 2024-04-25T11:42:27.635 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:42:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.936+0000 7fcbc2800640 1 Processor -- start 2024-04-25T11:42:27.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.936+0000 7fcbc2800640 1 -- start start 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.938+0000 7fcbc2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbbc144340 0x7fcbbc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.938+0000 7fcbc2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc06db10 0x7fcbbc15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.938+0000 7fcbc2800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcbbc15ec90 0x7fcbbc161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.938+0000 7fcbc2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc144750 con 0x7fcbbc144340 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.938+0000 7fcbc2800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc06df20 con 0x7fcbbc15ec90 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc070000 con 0x7fcbbc06db10 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbbc144340 0x7fcbbc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbbc144340 0x7fcbbc06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40624/0 (socket says 172.21.15.19:40624) 2024-04-25T11:42:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 -- 172.21.15.19:0/3332895632 learned_addr learned my addr 172.21.15.19:0/3332895632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:27.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc0a00640 1 --2- 172.21.15.19:0/3332895632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc06db10 0x7fcbbc15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1e00640 1 --2- 172.21.15.19:0/3332895632 >> v2:172.21.15.169:3300/0 conn(0x7fcbbc15ec90 0x7fcbbc161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 -- 172.21.15.19:0/3332895632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc06db10 msgr2=0x7fcbbc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 --2- 172.21.15.19:0/3332895632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc06db10 0x7fcbbc15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 -- 172.21.15.19:0/3332895632 >> v2:172.21.15.169:3300/0 conn(0x7fcbbc15ec90 msgr2=0x7fcbbc161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 --2- 172.21.15.19:0/3332895632 >> v2:172.21.15.169:3300/0 conn(0x7fcbbc15ec90 0x7fcbbc161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.939+0000 7fcbc1400640 1 -- 172.21.15.19:0/3332895632 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc06f240 con 0x7fcbbc144340 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.940+0000 7fcbc1e00640 1 --2- 172.21.15.19:0/3332895632 >> v2:172.21.15.169:3300/0 conn(0x7fcbbc15ec90 0x7fcbbc161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:27.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.941+0000 7fcbc1400640 1 --2- 172.21.15.19:0/3332895632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc144340 0x7fcbbc06d5d0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc147c30 tx=0x7fcbb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956723a4f710217d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.941+0000 7fcbbbe00640 1 -- 172.21.15.19:0/3332895632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0037070 con 0x7fcbbc144340 2024-04-25T11:42:27.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.941+0000 7fcbbbe00640 1 -- 172.21.15.19:0/3332895632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb0032440 con 0x7fcbbc144340 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.941+0000 7fcbbbe00640 1 -- 172.21.15.19:0/3332895632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0032810 con 0x7fcbbc144340 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 -- 172.21.15.19:0/3332895632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc144340 msgr2=0x7fcbbc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 --2- 172.21.15.19:0/3332895632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc144340 0x7fcbbc06d5d0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc147c30 tx=0x7fcbb002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 -- 172.21.15.19:0/3332895632 shutdown_connections 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 --2- 172.21.15.19:0/3332895632 >> v2:172.21.15.169:3300/0 conn(0x7fcbbc15ec90 0x7fcbbc161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 --2- 172.21.15.19:0/3332895632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc06db10 0x7fcbbc15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 --2- 172.21.15.19:0/3332895632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc144340 0x7fcbbc06d5d0 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 -- 172.21.15.19:0/3332895632 >> 172.21.15.19:0/3332895632 conn(0x7fcbbc072f40 msgr2=0x7fcbbc073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 -- 172.21.15.19:0/3332895632 shutdown_connections 2024-04-25T11:42:27.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.942+0000 7fcbc2800640 1 -- 172.21.15.19:0/3332895632 wait complete. 2024-04-25T11:42:27.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.944+0000 7fcbc2800640 1 Processor -- start 2024-04-25T11:42:27.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.945+0000 7fcbc2800640 1 -- start start 2024-04-25T11:42:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.946+0000 7fcbc2800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbbc06db10 0x7fcbbc1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.946+0000 7fcbc2800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc144340 0x7fcbbc161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.946+0000 7fcbc2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc15ec90 0x7fcbbc15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.946+0000 7fcbc2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc06fae0 con 0x7fcbbc15ec90 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.946+0000 7fcbc2800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc06f800 con 0x7fcbbc06db10 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.946+0000 7fcbc2800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc15f0a0 con 0x7fcbbc144340 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc15ec90 0x7fcbbc15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc15ec90 0x7fcbbc15d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40636/0 (socket says 172.21.15.19:40636) 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 -- 172.21.15.19:0/1273378198 learned_addr learned my addr 172.21.15.19:0/1273378198 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1400640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbbc06db10 0x7fcbbc1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc0a00640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc144340 0x7fcbbc161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 -- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc144340 msgr2=0x7fcbbc161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc144340 0x7fcbbc161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 -- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbbc06db10 msgr2=0x7fcbbc1f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbbc06db10 0x7fcbbc1f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 -- 172.21.15.19:0/1273378198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc144970 con 0x7fcbbc15ec90 2024-04-25T11:42:27.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1400640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbbc06db10 0x7fcbbc1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:27.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.947+0000 7fcbc1e00640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc15ec90 0x7fcbbc15d780 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400cd80 tx=0x7fcbb400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.948+0000 7fcbb9600640 1 -- 172.21.15.19:0/1273378198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400f680 con 0x7fcbbc15ec90 2024-04-25T11:42:27.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.948+0000 7fcbb9600640 1 -- 172.21.15.19:0/1273378198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb400f840 con 0x7fcbbc15ec90 2024-04-25T11:42:27.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.948+0000 7fcbb9600640 1 -- 172.21.15.19:0/1273378198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400fc30 con 0x7fcbbc15ec90 2024-04-25T11:42:27.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.948+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc1623f0 con 0x7fcbbc15ec90 2024-04-25T11:42:27.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.948+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc1f70d0 con 0x7fcbbc15ec90 2024-04-25T11:42:27.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.949+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc076c60 con 0x7fcbbc15ec90 2024-04-25T11:42:27.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.951+0000 7fcbb9600640 1 -- 172.21.15.19:0/1273378198 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcbb40213f0 con 0x7fcbbc15ec90 2024-04-25T11:42:27.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.952+0000 7fcbb9600640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.952+0000 7fcbb9600640 1 -- 172.21.15.19:0/1273378198 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5227+0+0 (secure 0 0 0) 0x7fcbb4096090 con 0x7fcbbc15ec90 2024-04-25T11:42:27.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.953+0000 7fcbc1400640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.954+0000 7fcbc1400640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0034000 tx=0x7fcbb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:27.956+0000 7fcbb9600640 1 -- 172.21.15.19:0/1273378198 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbbc1623f0 con 0x7fcbbc15ec90 2024-04-25T11:42:28.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.164+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcbbc146bc0 con 0x7fcbbc15ec90 2024-04-25T11:42:28.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.615+0000 7fcbb9600640 1 -- 172.21.15.19:0/1273378198 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) ==== 170+0+0 (secure 0 0 0) 0x7fcbb4014030 con 0x7fcbbc15ec90 2024-04-25T11:42:28.603 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T11:42:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.618+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb9c0744b0 msgr2=0x7fcb9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.618+0000 7fcbc2800640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0034000 tx=0x7fcbb002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:28.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc15ec90 msgr2=0x7fcbbc15d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc15ec90 0x7fcbbc15d780 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400cd80 tx=0x7fcbb400a650 comp rx=0 tx=0).stop 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 shutdown_connections 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb9c0744b0 0x7fcb9c0769c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbbc15ec90 0x7fcbbc15d780 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbbc144340 0x7fcbbc161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 --2- 172.21.15.19:0/1273378198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbbc06db10 0x7fcbbc1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 >> 172.21.15.19:0/1273378198 conn(0x7fcbbc072f40 msgr2=0x7fcbbc0b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 shutdown_connections 2024-04-25T11:42:28.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.619+0000 7fcbc2800640 1 -- 172.21.15.19:0/1273378198 wait complete. 2024-04-25T11:42:28.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:42:28.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.923+0000 7f4483e00640 1 Processor -- start 2024-04-25T11:42:28.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.923+0000 7f4483e00640 1 -- start start 2024-04-25T11:42:28.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.925+0000 7f4483e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4484144340 0x7f448406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.925+0000 7f4483e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f448415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.925+0000 7f4483e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f448415ec90 0x7f4484161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.925+0000 7f4483e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4484144750 con 0x7f4484144340 2024-04-25T11:42:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.925+0000 7f4483e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f448406df20 con 0x7f448415ec90 2024-04-25T11:42:28.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.925+0000 7f4483e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4484070000 con 0x7f448406db10 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4484144340 0x7f448406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4484144340 0x7f448406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40648/0 (socket says 172.21.15.19:40648) 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 -- 172.21.15.19:0/3421500074 learned_addr learned my addr 172.21.15.19:0/3421500074 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482000640 1 --2- 172.21.15.19:0/3421500074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f448415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4483400640 1 --2- 172.21.15.19:0/3421500074 >> v2:172.21.15.169:3300/0 conn(0x7f448415ec90 0x7f4484161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 -- 172.21.15.19:0/3421500074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 msgr2=0x7f448415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 --2- 172.21.15.19:0/3421500074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f448415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 -- 172.21.15.19:0/3421500074 >> v2:172.21.15.169:3300/0 conn(0x7f448415ec90 msgr2=0x7f4484161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 --2- 172.21.15.19:0/3421500074 >> v2:172.21.15.169:3300/0 conn(0x7f448415ec90 0x7f4484161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.926+0000 7f4482a00640 1 -- 172.21.15.19:0/3421500074 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448406f240 con 0x7f4484144340 2024-04-25T11:42:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.928+0000 7f4482a00640 1 --2- 172.21.15.19:0/3421500074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448406d5d0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f4484147c30 tx=0x7f447402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec97b436054909d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.928+0000 7f4481600640 1 -- 172.21.15.19:0/3421500074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474037070 con 0x7f4484144340 2024-04-25T11:42:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.928+0000 7f4481600640 1 -- 172.21.15.19:0/3421500074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4474032440 con 0x7f4484144340 2024-04-25T11:42:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.928+0000 7f4481600640 1 -- 172.21.15.19:0/3421500074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474032810 con 0x7f4484144340 2024-04-25T11:42:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 -- 172.21.15.19:0/3421500074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 msgr2=0x7f448406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 --2- 172.21.15.19:0/3421500074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448406d5d0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f4484147c30 tx=0x7f447402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 -- 172.21.15.19:0/3421500074 shutdown_connections 2024-04-25T11:42:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 --2- 172.21.15.19:0/3421500074 >> v2:172.21.15.169:3300/0 conn(0x7f448415ec90 0x7f4484161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 --2- 172.21.15.19:0/3421500074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f448415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 --2- 172.21.15.19:0/3421500074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448406d5d0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 -- 172.21.15.19:0/3421500074 >> 172.21.15.19:0/3421500074 conn(0x7f4484072f40 msgr2=0x7f4484073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 -- 172.21.15.19:0/3421500074 shutdown_connections 2024-04-25T11:42:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.929+0000 7f4483e00640 1 -- 172.21.15.19:0/3421500074 wait complete. 2024-04-25T11:42:28.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.931+0000 7f4483e00640 1 Processor -- start 2024-04-25T11:42:28.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.932+0000 7f4483e00640 1 -- start start 2024-04-25T11:42:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.933+0000 7f4483e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f44841448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.933+0000 7f4483e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.933+0000 7f4483e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448415ec90 0x7f4484161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.933+0000 7f4483e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f448406fae0 con 0x7f4484144340 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.933+0000 7f4483e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f448406f800 con 0x7f448415ec90 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.933+0000 7f4483e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f448415f0a0 con 0x7f448406db10 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.933+0000 7f4482000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40650/0 (socket says 172.21.15.19:40650) 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 -- 172.21.15.19:0/3945583966 learned_addr learned my addr 172.21.15.19:0/3945583966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482a00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f44841448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4483400640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448415ec90 0x7f4484161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 -- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 msgr2=0x7f44841448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f44841448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 -- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448415ec90 msgr2=0x7f4484161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448415ec90 0x7f4484161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 -- 172.21.15.19:0/3945583966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4484147770 con 0x7f4484144340 2024-04-25T11:42:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4483400640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448415ec90 0x7f4484161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.934+0000 7f4482000640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448415b150 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f447000d5c0 tx=0x7f447000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.935+0000 7f447aa00640 1 -- 172.21.15.19:0/3945583966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700125f0 con 0x7f4484144340 2024-04-25T11:42:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.935+0000 7f447aa00640 1 -- 172.21.15.19:0/3945583966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4470019070 con 0x7f4484144340 2024-04-25T11:42:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.935+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4484160370 con 0x7f4484144340 2024-04-25T11:42:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.935+0000 7f447aa00640 1 -- 172.21.15.19:0/3945583966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700129d0 con 0x7f4484144340 2024-04-25T11:42:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.935+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4484160930 con 0x7f4484144340 2024-04-25T11:42:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.936+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4484076c60 con 0x7f4484144340 2024-04-25T11:42:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.938+0000 7f447aa00640 1 -- 172.21.15.19:0/3945583966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4470012b70 con 0x7f4484144340 2024-04-25T11:42:28.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.939+0000 7f447aa00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44640744b0 0x7f44640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.939+0000 7f447aa00640 1 -- 172.21.15.19:0/3945583966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5609+0+0 (secure 0 0 0) 0x7f4470095670 con 0x7f4484144340 2024-04-25T11:42:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.940+0000 7f4482a00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44640744b0 0x7f44640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.941+0000 7f4482a00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44640744b0 0x7f44640769c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4474034000 tx=0x7f447402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:28.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:28.945+0000 7f447aa00640 1 -- 172.21.15.19:0/3945583966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f447005fdc0 con 0x7f4484144340 2024-04-25T11:42:29.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.190+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4484147500 con 0x7f4484144340 2024-04-25T11:42:29.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.675+0000 7f447aa00640 1 -- 172.21.15.19:0/3945583966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v360) ==== 174+0+0 (secure 0 0 0) 0x7f447005f6c0 con 0x7f4484144340 2024-04-25T11:42:29.665 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T11:42:29.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.681+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44640744b0 msgr2=0x7f44640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:29.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.681+0000 7f4483e00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44640744b0 0x7f44640769c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4474034000 tx=0x7f447402e000 comp rx=0 tx=0).stop 2024-04-25T11:42:29.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.681+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 msgr2=0x7f448415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:29.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.681+0000 7f4483e00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448415b150 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f447000d5c0 tx=0x7f447000da80 comp rx=0 tx=0).stop 2024-04-25T11:42:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.684+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 shutdown_connections 2024-04-25T11:42:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.684+0000 7f4483e00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f44640744b0 0x7f44640769c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.684+0000 7f4483e00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f448415ec90 0x7f4484161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.684+0000 7f4483e00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4484144340 0x7f448415b150 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.684+0000 7f4483e00640 1 --2- 172.21.15.19:0/3945583966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f448406db10 0x7f44841448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.684+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 >> 172.21.15.19:0/3945583966 conn(0x7f4484072f40 msgr2=0x7f44841411e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.686+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 shutdown_connections 2024-04-25T11:42:29.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.686+0000 7f4483e00640 1 -- 172.21.15.19:0/3945583966 wait complete. 2024-04-25T11:42:29.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.979+0000 7fb4adc00640 1 Processor -- start 2024-04-25T11:42:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.979+0000 7fb4adc00640 1 -- start start 2024-04-25T11:42:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.981+0000 7fb4adc00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4a8070e20 0x7fb4a815c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.981+0000 7fb4adc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.981+0000 7fb4adc00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a806d7a0 con 0x7fb4a806d390 2024-04-25T11:42:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.982+0000 7fb4adc00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a8071230 con 0x7fb4a8070e20 2024-04-25T11:42:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.982+0000 7fb4adc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a806f8f0 con 0x7fb4a806f180 2024-04-25T11:42:29.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.982+0000 7fb4a7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4a8070e20 0x7fb4a815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:29.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.982+0000 7fb4a7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:29.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.982+0000 7fb4a7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4a8070e20 0x7fb4a815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:60596/0 (socket says 172.21.15.19:60596) 2024-04-25T11:42:29.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.982+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1588799495 learned_addr learned my addr 172.21.15.19:0/1588799495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:29.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.983+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1588799495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 msgr2=0x7fb4a806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:29.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.983+0000 7fb4a7e00640 1 --2- 172.21.15.19:0/1588799495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a806ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.983+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1588799495 >> v1:172.21.15.169:6789/0 conn(0x7fb4a806d390 legacy=0x7fb4a80cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:29.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.983+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1588799495 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a806fbd0 con 0x7fb4a8070e20 2024-04-25T11:42:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.985+0000 7fb4a7e00640 1 --2- 172.21.15.19:0/1588799495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a815c800 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb49800af60 tx=0x7fb49802c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00e114c36ee3975 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.985+0000 7fb4a6a00640 1 -- 172.21.15.19:0/1588799495 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498008500 con 0x7fb4a8070e20 2024-04-25T11:42:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.985+0000 7fb4a6a00640 1 -- 172.21.15.19:0/1588799495 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb498037070 con 0x7fb4a8070e20 2024-04-25T11:42:29.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.985+0000 7fb4a6a00640 1 -- 172.21.15.19:0/1588799495 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980088c0 con 0x7fb4a8070e20 2024-04-25T11:42:29.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 -- 172.21.15.19:0/1588799495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 msgr2=0x7fb4a815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:29.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1588799495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a815c800 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb49800af60 tx=0x7fb49802c930 comp rx=0 tx=0).stop 2024-04-25T11:42:29.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 -- 172.21.15.19:0/1588799495 shutdown_connections 2024-04-25T11:42:29.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1588799495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a806ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1588799495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a815c800 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 -- 172.21.15.19:0/1588799495 >> 172.21.15.19:0/1588799495 conn(0x7fb4a80d1720 msgr2=0x7fb4a80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:29.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 -- 172.21.15.19:0/1588799495 shutdown_connections 2024-04-25T11:42:29.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.986+0000 7fb4adc00640 1 -- 172.21.15.19:0/1588799495 wait complete. 2024-04-25T11:42:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.989+0000 7fb4adc00640 1 Processor -- start 2024-04-25T11:42:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.989+0000 7fb4adc00640 1 -- start start 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.990+0000 7fb4adc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4a806d390 0x7fb4a8141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.990+0000 7fb4adc00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a8154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.990+0000 7fb4adc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a81e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.990+0000 7fb4adc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a80b6e30 con 0x7fb4a806d390 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.990+0000 7fb4adc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8070190 con 0x7fb4a8070e20 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.990+0000 7fb4adc00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a806f590 con 0x7fb4a806f180 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4a806d390 0x7fb4a8141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4a806d390 0x7fb4a8141120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40656/0 (socket says 172.21.15.19:40656) 2024-04-25T11:42:29.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1598386207 learned_addr learned my addr 172.21.15.19:0/1598386207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7400640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a8154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4ad200640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a81e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 msgr2=0x7fb4a8154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a8154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 msgr2=0x7fb4a81e5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a81e5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7e00640 1 -- 172.21.15.19:0/1598386207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a8141660 con 0x7fb4a806d390 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4a7400640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a8154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.991+0000 7fb4ad200640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a81e5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.992+0000 7fb4a7e00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4a806d390 0x7fb4a8141120 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7fb498034040 tx=0x7fb498004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.992+0000 7fb49fe00640 1 -- 172.21.15.19:0/1598386207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498041070 con 0x7fb4a806d390 2024-04-25T11:42:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.992+0000 7fb49fe00640 1 -- 172.21.15.19:0/1598386207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb498008c40 con 0x7fb4a806d390 2024-04-25T11:42:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.992+0000 7fb49fe00640 1 -- 172.21.15.19:0/1598386207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49803b420 con 0x7fb4a806d390 2024-04-25T11:42:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.992+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a81555c0 con 0x7fb4a806d390 2024-04-25T11:42:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.992+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a8155b80 con 0x7fb4a806d390 2024-04-25T11:42:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.994+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a8050740 con 0x7fb4a806d390 2024-04-25T11:42:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.995+0000 7fb49fe00640 1 -- 172.21.15.19:0/1598386207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb498005d10 con 0x7fb4a806d390 2024-04-25T11:42:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.995+0000 7fb49fe00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4880744b0 0x7fb4880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.996+0000 7fb49fe00640 1 -- 172.21.15.19:0/1598386207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7fb4980b5750 con 0x7fb4a806d390 2024-04-25T11:42:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.996+0000 7fb4a7400640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4880744b0 0x7fb4880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:29.998+0000 7fb4a7400640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4880744b0 0x7fb4880769c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80b1850 tx=0x7fb494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.001+0000 7fb49fe00640 1 -- 172.21.15.19:0/1598386207 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb49804b030 con 0x7fb4a806d390 2024-04-25T11:42:30.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.204+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4a80732a0 con 0x7fb4a806d390 2024-04-25T11:42:30.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.206+0000 7fb49fe00640 1 -- 172.21.15.19:0/1598386207 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16632 (secure 0 0 0) 0x7fb49807ff60 con 0x7fb4a806d390 2024-04-25T11:42:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:30.194 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":142,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:29.193781+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4880744b0 msgr2=0x7fb4880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4880744b0 0x7fb4880769c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80b1850 tx=0x7fb494008040 comp rx=0 tx=0).stop 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4a806d390 msgr2=0x7fb4a8141120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4a806d390 0x7fb4a8141120 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7fb498034040 tx=0x7fb498004a40 comp rx=0 tx=0).stop 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 shutdown_connections 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4880744b0 0x7fb4880769c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4a8070e20 0x7fb4a81e5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4a806f180 0x7fb4a8154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 --2- 172.21.15.19:0/1598386207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4a806d390 0x7fb4a8141120 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.208+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 >> 172.21.15.19:0/1598386207 conn(0x7fb4a80d1720 msgr2=0x7fb4a80b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.209+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 shutdown_connections 2024-04-25T11:42:30.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.209+0000 7fb4adc00640 1 -- 172.21.15.19:0/1598386207 wait complete. 2024-04-25T11:42:30.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:30.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.512+0000 7ff1f5600640 1 Processor -- start 2024-04-25T11:42:30.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.513+0000 7ff1f5600640 1 -- start start 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.514+0000 7ff1f5600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1f006c5c0 0x7ff1f00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.514+0000 7ff1f5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f00b4ea0 0x7ff1f015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.514+0000 7ff1f5600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f00d1230 con 0x7ff1f0144370 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.514+0000 7ff1f5600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f00d06b0 con 0x7ff1f006c5c0 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.514+0000 7ff1f5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f00d0990 con 0x7ff1f00b4ea0 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.515+0000 7ff1efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1f006c5c0 0x7ff1f00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.515+0000 7ff1efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1f006c5c0 0x7ff1f00cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40672/0 (socket says 172.21.15.19:40672) 2024-04-25T11:42:30.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.515+0000 7ff1efe00640 1 -- 172.21.15.19:0/3708259100 learned_addr learned my addr 172.21.15.19:0/3708259100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:30.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.515+0000 7ff1efe00640 1 -- 172.21.15.19:0/3708259100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f00b4ea0 msgr2=0x7ff1f015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:30.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.515+0000 7ff1efe00640 1 --2- 172.21.15.19:0/3708259100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f00b4ea0 0x7ff1f015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.515+0000 7ff1efe00640 1 -- 172.21.15.19:0/3708259100 >> v1:172.21.15.169:6789/0 conn(0x7ff1f0144370 legacy=0x7ff1f00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:30.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.515+0000 7ff1efe00640 1 -- 172.21.15.19:0/3708259100 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00d0c70 con 0x7ff1f006c5c0 2024-04-25T11:42:30.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.516+0000 7ff1efe00640 1 --2- 172.21.15.19:0/3708259100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f006c5c0 0x7ff1f00cfa70 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc242e50e62ba497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:30.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.517+0000 7ff1ef400640 1 -- 172.21.15.19:0/3708259100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0037070 con 0x7ff1f006c5c0 2024-04-25T11:42:30.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.517+0000 7ff1ef400640 1 -- 172.21.15.19:0/3708259100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0032440 con 0x7ff1f006c5c0 2024-04-25T11:42:30.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.517+0000 7ff1ef400640 1 -- 172.21.15.19:0/3708259100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0032810 con 0x7ff1f006c5c0 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.517+0000 7ff1f5600640 1 -- 172.21.15.19:0/3708259100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f006c5c0 msgr2=0x7ff1f00cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.517+0000 7ff1f5600640 1 --2- 172.21.15.19:0/3708259100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f006c5c0 0x7ff1f00cfa70 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e002a210 comp rx=0 tx=0).stop 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.518+0000 7ff1f5600640 1 -- 172.21.15.19:0/3708259100 shutdown_connections 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.518+0000 7ff1f5600640 1 --2- 172.21.15.19:0/3708259100 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f00b4ea0 0x7ff1f015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.518+0000 7ff1f5600640 1 --2- 172.21.15.19:0/3708259100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f006c5c0 0x7ff1f00cfa70 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.518+0000 7ff1f5600640 1 -- 172.21.15.19:0/3708259100 >> 172.21.15.19:0/3708259100 conn(0x7ff1f00d1720 msgr2=0x7ff1f00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.518+0000 7ff1f5600640 1 -- 172.21.15.19:0/3708259100 shutdown_connections 2024-04-25T11:42:30.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.518+0000 7ff1f5600640 1 -- 172.21.15.19:0/3708259100 wait complete. 2024-04-25T11:42:30.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.520+0000 7ff1f5600640 1 Processor -- start 2024-04-25T11:42:30.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.520+0000 7ff1f5600640 1 -- start start 2024-04-25T11:42:30.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1f5600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1f006c5c0 0x7ff1f015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:30.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1f5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f0144370 0x7ff1f006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:30.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1f5600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f006cf10 0x7ff1f0161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:30.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1f5600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00d0450 con 0x7ff1f0144370 2024-04-25T11:42:30.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1f5600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00d02d0 con 0x7ff1f006c5c0 2024-04-25T11:42:30.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1f5600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f0147760 con 0x7ff1f006cf10 2024-04-25T11:42:30.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f0144370 0x7ff1f006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:30.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f0144370 0x7ff1f006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40678/0 (socket says 172.21.15.19:40678) 2024-04-25T11:42:30.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1e7400640 1 -- 172.21.15.19:0/4280625031 learned_addr learned my addr 172.21.15.19:0/4280625031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.522+0000 7ff1f4c00640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f006cf10 0x7ff1f0161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1efe00640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1f006c5c0 0x7ff1f015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1e7400640 1 -- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f006cf10 msgr2=0x7ff1f0161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:30.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1e7400640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f006cf10 0x7ff1f0161350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1e7400640 1 -- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1f006c5c0 msgr2=0x7ff1f015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1e7400640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1f006c5c0 0x7ff1f015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1e7400640 1 -- 172.21.15.19:0/4280625031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f006d320 con 0x7ff1f0144370 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1f4c00640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f006cf10 0x7ff1f0161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1efe00640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1f006c5c0 0x7ff1f015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1e7400640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f0144370 0x7ff1f006c9d0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800ba50 tx=0x7ff1d800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1ecc00640 1 -- 172.21.15.19:0/4280625031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8019070 con 0x7ff1f0144370 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1ecc00640 1 -- 172.21.15.19:0/4280625031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d800cc20 con 0x7ff1f0144370 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1ecc00640 1 -- 172.21.15.19:0/4280625031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8014680 con 0x7ff1f0144370 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f01fa7a0 con 0x7ff1f0144370 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.523+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f01f69f0 con 0x7ff1f0144370 2024-04-25T11:42:30.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.524+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f0050740 con 0x7ff1f0144370 2024-04-25T11:42:30.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.526+0000 7ff1ecc00640 1 -- 172.21.15.19:0/4280625031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff1d801c070 con 0x7ff1f0144370 2024-04-25T11:42:30.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.527+0000 7ff1ecc00640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1cc0744b0 0x7ff1cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:30.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.527+0000 7ff1ecc00640 1 -- 172.21.15.19:0/4280625031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5709+0+0 (secure 0 0 0) 0x7ff1d8095740 con 0x7ff1f0144370 2024-04-25T11:42:30.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.528+0000 7ff1efe00640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1cc0744b0 0x7ff1cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:30.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.529+0000 7ff1efe00640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1cc0744b0 0x7ff1cc0769c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0034000 tx=0x7ff1e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:30.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.531+0000 7ff1ecc00640 1 -- 172.21.15.19:0/4280625031 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1f006d320 con 0x7ff1f0144370 2024-04-25T11:42:30.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.737+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1f015f5f0 con 0x7ff1f0144370 2024-04-25T11:42:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.739+0000 7ff1ecc00640 1 -- 172.21.15.19:0/4280625031 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16619 (secure 0 0 0) 0x7ff1d805f8f0 con 0x7ff1f0144370 2024-04-25T11:42:30.727 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":143,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:30.672424+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1cc0744b0 msgr2=0x7ff1cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1cc0744b0 0x7ff1cc0769c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0034000 tx=0x7ff1e002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f0144370 msgr2=0x7ff1f006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f0144370 0x7ff1f006c9d0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800ba50 tx=0x7ff1d800bf10 comp rx=0 tx=0).stop 2024-04-25T11:42:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 shutdown_connections 2024-04-25T11:42:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff1cc0744b0 0x7ff1cc0769c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff1f006cf10 0x7ff1f0161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1f0144370 0x7ff1f006c9d0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 --2- 172.21.15.19:0/4280625031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1f006c5c0 0x7ff1f015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:30.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.742+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 >> 172.21.15.19:0/4280625031 conn(0x7ff1f00d1720 msgr2=0x7ff1f015be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:30.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.743+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 shutdown_connections 2024-04-25T11:42:30.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:30.743+0000 7ff1f5600640 1 -- 172.21.15.19:0/4280625031 wait complete. 2024-04-25T11:42:30.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:42:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.058+0000 7fbec5000640 1 Processor -- start 2024-04-25T11:42:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.059+0000 7fbec5000640 1 -- start start 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.061+0000 7fbec5000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbec00ce520 0x7fbec015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.061+0000 7fbec5000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbec00ce930 0x7fbec015d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.061+0000 7fbec5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 0x7fbec006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.061+0000 7fbec5000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbec006df20 con 0x7fbec00ce520 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.061+0000 7fbec5000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fbec00cda60 con 0x7fbec00ce930 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.061+0000 7fbec5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbec015dfe0 con 0x7fbec006d5d0 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.061+0000 7fbebf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbec00ce520 0x7fbec015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbebf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbec00ce520 0x7fbec015d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40684/0 (socket says 172.21.15.19:40684) 2024-04-25T11:42:31.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbebf400640 1 -- 172.21.15.19:0/1665000895 learned_addr learned my addr 172.21.15.19:0/1665000895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbebf400640 1 -- 172.21.15.19:0/1665000895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 msgr2=0x7fbec006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbebf400640 1 --2- 172.21.15.19:0/1665000895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 0x7fbec006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbeb7e00640 1 --2- 172.21.15.19:0/1665000895 >> v2:172.21.15.169:3300/0 conn(0x7fbec00ce930 0x7fbec015d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbebf400640 1 -- 172.21.15.19:0/1665000895 >> v2:172.21.15.169:3300/0 conn(0x7fbec00ce930 msgr2=0x7fbec015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbebf400640 1 --2- 172.21.15.19:0/1665000895 >> v2:172.21.15.169:3300/0 conn(0x7fbec00ce930 0x7fbec015d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.062+0000 7fbebf400640 1 -- 172.21.15.19:0/1665000895 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec015e2c0 con 0x7fbec00ce520 2024-04-25T11:42:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.063+0000 7fbebf400640 1 --2- 172.21.15.19:0/1665000895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec015d4b0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7fbec00b5150 tx=0x7fbeb002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d3aa9a9d80211e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbebea00640 1 -- 172.21.15.19:0/1665000895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0037070 con 0x7fbec00ce520 2024-04-25T11:42:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbebea00640 1 -- 172.21.15.19:0/1665000895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb0032440 con 0x7fbec00ce520 2024-04-25T11:42:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbebea00640 1 -- 172.21.15.19:0/1665000895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0032810 con 0x7fbec00ce520 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbec5000640 1 -- 172.21.15.19:0/1665000895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 msgr2=0x7fbec015d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbec5000640 1 --2- 172.21.15.19:0/1665000895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec015d4b0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7fbec00b5150 tx=0x7fbeb002a140 comp rx=0 tx=0).stop 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbec5000640 1 -- 172.21.15.19:0/1665000895 shutdown_connections 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbec5000640 1 --2- 172.21.15.19:0/1665000895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 0x7fbec006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbec5000640 1 --2- 172.21.15.19:0/1665000895 >> v2:172.21.15.169:3300/0 conn(0x7fbec00ce930 0x7fbec015d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbec5000640 1 --2- 172.21.15.19:0/1665000895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec015d4b0 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.064+0000 7fbec5000640 1 -- 172.21.15.19:0/1665000895 >> 172.21.15.19:0/1665000895 conn(0x7fbec0072f40 msgr2=0x7fbec0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.065+0000 7fbec5000640 1 -- 172.21.15.19:0/1665000895 shutdown_connections 2024-04-25T11:42:31.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.065+0000 7fbec5000640 1 -- 172.21.15.19:0/1665000895 wait complete. 2024-04-25T11:42:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.067+0000 7fbec5000640 1 Processor -- start 2024-04-25T11:42:31.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.067+0000 7fbec5000640 1 -- start start 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbec5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 0x7fbec01f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbec5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbec5000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbec00ce930 0x7fbec00cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbec5000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbec0160d30 con 0x7fbec00ce520 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbec5000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00ce000 con 0x7fbec00ce930 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbec5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00b73d0 con 0x7fbec006d5d0 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40700/0 (socket says 172.21.15.19:40700) 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbeb7e00640 1 -- 172.21.15.19:0/4125414427 learned_addr learned my addr 172.21.15.19:0/4125414427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbebfe00640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbec00ce930 0x7fbec00cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.069+0000 7fbebf400640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 0x7fbec01f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7e00640 1 -- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 msgr2=0x7fbec01f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7e00640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 0x7fbec01f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7e00640 1 -- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbec00ce930 msgr2=0x7fbec00cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7e00640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbec00ce930 0x7fbec00cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7e00640 1 -- 172.21.15.19:0/4125414427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec014d870 con 0x7fbec00ce520 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbebfe00640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbec00ce930 0x7fbec00cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7e00640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec00cb5e0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fbea800c7f0 tx=0x7fbea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7400640 1 -- 172.21.15.19:0/4125414427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8015070 con 0x7fbec00ce520 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7400640 1 -- 172.21.15.19:0/4125414427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea8004220 con 0x7fbec00ce520 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbeb7400640 1 -- 172.21.15.19:0/4125414427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8007620 con 0x7fbec00ce520 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec0156180 con 0x7fbec00ce520 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.070+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec0156740 con 0x7fbec00ce520 2024-04-25T11:42:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.072+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec0155ef0 con 0x7fbec00ce520 2024-04-25T11:42:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.073+0000 7fbeb7400640 1 -- 172.21.15.19:0/4125414427 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbea80079d0 con 0x7fbec00ce520 2024-04-25T11:42:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.074+0000 7fbeb7400640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbe9c0744b0 0x7fbe9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.074+0000 7fbeb7400640 1 -- 172.21.15.19:0/4125414427 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7fbea8095530 con 0x7fbec00ce520 2024-04-25T11:42:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.075+0000 7fbebf400640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbe9c0744b0 0x7fbe9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.076+0000 7fbebf400640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbe9c0744b0 0x7fbe9c0769c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0034000 tx=0x7fbeb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.079+0000 7fbeb7400640 1 -- 172.21.15.19:0/4125414427 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbec014d870 con 0x7fbec00ce520 2024-04-25T11:42:31.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.331+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbec0156910 con 0x7fbec00ce520 2024-04-25T11:42:31.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.717+0000 7fbeb7400640 1 -- 172.21.15.19:0/4125414427 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v363) ==== 118+0+0 (secure 0 0 0) 0x7fbec0156910 con 0x7fbec00ce520 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbe9c0744b0 msgr2=0x7fbe9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbe9c0744b0 0x7fbe9c0769c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0034000 tx=0x7fbeb002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 msgr2=0x7fbec00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec00cb5e0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fbea800c7f0 tx=0x7fbea800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 shutdown_connections 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbe9c0744b0 0x7fbe9c0769c0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbec00ce930 0x7fbec00cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbec00ce520 0x7fbec00cb5e0 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 --2- 172.21.15.19:0/4125414427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbec006d5d0 0x7fbec01f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 >> 172.21.15.19:0/4125414427 conn(0x7fbec0072f40 msgr2=0x7fbec015b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:31.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 shutdown_connections 2024-04-25T11:42:31.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:31.721+0000 7fbec5000640 1 -- 172.21.15.19:0/4125414427 wait complete. 2024-04-25T11:42:31.741 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:42:31.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:42:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.032+0000 7f238be00640 1 Processor -- start 2024-04-25T11:42:32.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.033+0000 7f238be00640 1 -- start start 2024-04-25T11:42:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.035+0000 7f238be00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f238c0ce930 0x7f238c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.035+0000 7f238be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 0x7f238c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.035+0000 7f238be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f238c06df20 con 0x7f238c0ce520 2024-04-25T11:42:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.035+0000 7f238be00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f238c0cda60 con 0x7f238c0ce930 2024-04-25T11:42:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.035+0000 7f238be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f238c15dfe0 con 0x7f238c06db10 2024-04-25T11:42:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.036+0000 7f238aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f238c0ce520 0x7f238c15d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51648/0 (socket says 172.21.15.19:51648) 2024-04-25T11:42:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.036+0000 7f238aa00640 1 -- 172.21.15.19:0/530889057 learned_addr learned my addr 172.21.15.19:0/530889057 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.036+0000 7f2383e00640 1 --2- 172.21.15.19:0/530889057 >> v2:172.21.15.169:3300/0 conn(0x7f238c0ce930 0x7f238c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.036+0000 7f238b400640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 0x7f238c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.036+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017449992 0 0) 0x7f237c001110 con 0x7f238c0ce520 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.036+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f236c003520 con 0x7f238c0ce520 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3373243225 0 0) 0x7f236c003520 con 0x7f238c0ce520 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 msgr2=0x7f238c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 0x7f238c0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 >> v2:172.21.15.169:3300/0 conn(0x7f238c0ce930 msgr2=0x7f238c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 --2- 172.21.15.19:0/530889057 >> v2:172.21.15.169:3300/0 conn(0x7f238c0ce930 0x7f238c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c15e2c0 con 0x7f238c0ce520 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f237c001860 con 0x7f238c0ce520 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 >> v1:172.21.15.19:6789/0 conn(0x7f238c0ce520 legacy=0x7f238c15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 0x7f236c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f236c003f80 0x7f236c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f236c004d40 0x7f236c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0cda60 con 0x7f236c003f80 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f238c15dfe0 con 0x7f236c004d40 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f236c00bb20 con 0x7f238c06db10 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.037+0000 7f2383e00640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f236c003f80 0x7f236c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.038+0000 7f238aa00640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 0x7f236c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.038+0000 7f238b400640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f236c004d40 0x7f236c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.038+0000 7f2383e00640 1 -- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 msgr2=0x7f236c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.038+0000 7f2383e00640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f238c06db10 0x7f236c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.038+0000 7f2383e00640 1 -- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f236c004d40 msgr2=0x7f236c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.038+0000 7f2383e00640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f236c004d40 0x7f236c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.038+0000 7f2383e00640 1 -- 172.21.15.19:0/530889057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c003520 con 0x7f236c003f80 2024-04-25T11:42:32.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.039+0000 7f2383e00640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f236c003f80 0x7f236c004800 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f2374009a90 tx=0x7f236c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba6881675c79a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.039+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374036070 con 0x7f236c003f80 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.039+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2374031400 con 0x7f236c003f80 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.039+0000 7f238a000640 1 -- 172.21.15.19:0/530889057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374009690 con 0x7f236c003f80 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.040+0000 7f238be00640 1 -- 172.21.15.19:0/530889057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f236c003f80 msgr2=0x7f236c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.040+0000 7f238be00640 1 --2- 172.21.15.19:0/530889057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f236c003f80 0x7f236c004800 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f2374009a90 tx=0x7f236c005830 comp rx=0 tx=0).stop 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.040+0000 7f238b400640 1 -- 172.21.15.19:0/530889057 reap_dead start 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.040+0000 7f238be00640 1 -- 172.21.15.19:0/530889057 shutdown_connections 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.040+0000 7f238be00640 1 -- 172.21.15.19:0/530889057 >> 172.21.15.19:0/530889057 conn(0x7f238c072f40 msgr2=0x7f238c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.040+0000 7f238be00640 1 -- 172.21.15.19:0/530889057 shutdown_connections 2024-04-25T11:42:32.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.040+0000 7f238be00640 1 -- 172.21.15.19:0/530889057 wait complete. 2024-04-25T11:42:32.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.042+0000 7f238be00640 1 Processor -- start 2024-04-25T11:42:32.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.043+0000 7f238be00640 1 -- start start 2024-04-25T11:42:32.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.044+0000 7f238be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236c003f80 0x7f238c152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.044+0000 7f238be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f238c14bd70 0x7f238c06faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.044+0000 7f238be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f238c14ce70 0x7f238c15f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.044+0000 7f238be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f238c160d30 con 0x7f238c14bd70 2024-04-25T11:42:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.044+0000 7f238be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0ce000 con 0x7f238c14ce70 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.044+0000 7f238be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f238c15dbc0 con 0x7f236c003f80 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.044+0000 7f2383e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f238c14bd70 0x7f238c06faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f238c14bd70 0x7f238c06faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40712/0 (socket says 172.21.15.19:40712) 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 -- 172.21.15.19:0/2451209085 learned_addr learned my addr 172.21.15.19:0/2451209085 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f238b400640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f238c14ce70 0x7f238c15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f238aa00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236c003f80 0x7f238c152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 -- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236c003f80 msgr2=0x7f238c152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236c003f80 0x7f238c152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 -- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f238c14ce70 msgr2=0x7f238c15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f238c14ce70 0x7f238c15f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 -- 172.21.15.19:0/2451209085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c1529a0 con 0x7f238c14bd70 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f238aa00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236c003f80 0x7f238c152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.045+0000 7f2383e00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f238c14bd70 0x7f238c06faf0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f2374034040 tx=0x7f2374004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.046+0000 7f2382a00640 1 -- 172.21.15.19:0/2451209085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374040020 con 0x7f238c14bd70 2024-04-25T11:42:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.046+0000 7f2382a00640 1 -- 172.21.15.19:0/2451209085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2374003c60 con 0x7f238c14bd70 2024-04-25T11:42:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.046+0000 7f2382a00640 1 -- 172.21.15.19:0/2451209085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374036040 con 0x7f238c14bd70 2024-04-25T11:42:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.046+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c070880 con 0x7f238c14bd70 2024-04-25T11:42:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.046+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c14c670 con 0x7f238c14bd70 2024-04-25T11:42:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.047+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238c076c60 con 0x7f238c14bd70 2024-04-25T11:42:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.049+0000 7f2382a00640 1 -- 172.21.15.19:0/2451209085 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2374043020 con 0x7f238c14bd70 2024-04-25T11:42:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.049+0000 7f2382a00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2368074450 0x7f2368076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.050+0000 7f2382a00640 1 -- 172.21.15.19:0/2451209085 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f23740b54b0 con 0x7f238c14bd70 2024-04-25T11:42:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.050+0000 7f238aa00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2368074450 0x7f2368076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.051+0000 7f238aa00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2368074450 0x7f2368076960 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f238c0b5b40 tx=0x7f237c001110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:32.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.054+0000 7f2382a00640 1 -- 172.21.15.19:0/2451209085 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f237404a070 con 0x7f238c14bd70 2024-04-25T11:42:32.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.304+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f238c0b5480 con 0x7f238c14bd70 2024-04-25T11:42:32.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.721+0000 7f2382a00640 1 -- 172.21.15.19:0/2451209085 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v364) ==== 122+0+0 (secure 0 0 0) 0x7f23740044e0 con 0x7f238c14bd70 2024-04-25T11:42:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2368074450 msgr2=0x7f2368076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2368074450 0x7f2368076960 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f238c0b5b40 tx=0x7f237c001110 comp rx=0 tx=0).stop 2024-04-25T11:42:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f238c14bd70 msgr2=0x7f238c06faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:32.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f238c14bd70 0x7f238c06faf0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f2374034040 tx=0x7f2374004730 comp rx=0 tx=0).stop 2024-04-25T11:42:32.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 shutdown_connections 2024-04-25T11:42:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2368074450 0x7f2368076960 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f238c14ce70 0x7f238c15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f238c14bd70 0x7f238c06faf0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 --2- 172.21.15.19:0/2451209085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f236c003f80 0x7f238c152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.725+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 >> 172.21.15.19:0/2451209085 conn(0x7f238c072f40 msgr2=0x7f238c1559e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.726+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 shutdown_connections 2024-04-25T11:42:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:32.726+0000 7f238be00640 1 -- 172.21.15.19:0/2451209085 wait complete. 2024-04-25T11:42:32.746 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:42:32.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.045+0000 7f7aace00640 1 Processor -- start 2024-04-25T11:42:33.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.045+0000 7f7aace00640 1 -- start start 2024-04-25T11:42:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aace00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7aa80b7c50 0x7f7aa80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aace00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80d44a0 0x7f7aa80d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aace00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa8161650 con 0x7f7aa8164860 2024-04-25T11:42:33.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aace00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa81643d0 con 0x7f7aa80b7c50 2024-04-25T11:42:33.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aace00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa80b5270 con 0x7f7aa80d44a0 2024-04-25T11:42:33.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aa7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7aa8164860 0x7f7aa80b7540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51664/0 (socket says 172.21.15.19:51664) 2024-04-25T11:42:33.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aa7400640 1 -- 172.21.15.19:0/2091989675 learned_addr learned my addr 172.21.15.19:0/2091989675 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:33.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.048+0000 7f7aa6a00640 1 --2- 172.21.15.19:0/2091989675 >> v2:172.21.15.19:3300/0 conn(0x7f7aa80b7c50 0x7f7aa80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.049+0000 7f7aa7e00640 1 --2- 172.21.15.19:0/2091989675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80d44a0 0x7f7aa80d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.049+0000 7f7aa6000640 1 -- 172.21.15.19:0/2091989675 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494858326 0 0) 0x7f7aa8161650 con 0x7f7aa8164860 2024-04-25T11:42:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.049+0000 7f7aa6a00640 1 -- 172.21.15.19:0/2091989675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80d44a0 msgr2=0x7f7aa80d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.049+0000 7f7aa6a00640 1 --2- 172.21.15.19:0/2091989675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80d44a0 0x7f7aa80d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.049+0000 7f7aa6a00640 1 -- 172.21.15.19:0/2091989675 >> v1:172.21.15.19:6789/0 conn(0x7f7aa8164860 legacy=0x7f7aa80b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.049+0000 7f7aa6a00640 1 -- 172.21.15.19:0/2091989675 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa8161370 con 0x7f7aa80b7c50 2024-04-25T11:42:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.050+0000 7f7aa6a00640 1 --2- 172.21.15.19:0/2091989675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa80b7c50 0x7f7aa80b4d30 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f7a900097a0 tx=0x7f7a9002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0184960d495a174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.050+0000 7f7aa6000640 1 -- 172.21.15.19:0/2091989675 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90037070 con 0x7f7aa80b7c50 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.050+0000 7f7aa6000640 1 -- 172.21.15.19:0/2091989675 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a90032440 con 0x7f7aa80b7c50 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.050+0000 7f7aa6000640 1 -- 172.21.15.19:0/2091989675 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90032810 con 0x7f7aa80b7c50 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 -- 172.21.15.19:0/2091989675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa80b7c50 msgr2=0x7f7aa80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 --2- 172.21.15.19:0/2091989675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa80b7c50 0x7f7aa80b4d30 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f7a900097a0 tx=0x7f7a9002a180 comp rx=0 tx=0).stop 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 -- 172.21.15.19:0/2091989675 shutdown_connections 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 --2- 172.21.15.19:0/2091989675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80d44a0 0x7f7aa80d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 --2- 172.21.15.19:0/2091989675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa80b7c50 0x7f7aa80b4d30 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 -- 172.21.15.19:0/2091989675 >> 172.21.15.19:0/2091989675 conn(0x7f7aa80d2720 msgr2=0x7f7aa80d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 -- 172.21.15.19:0/2091989675 shutdown_connections 2024-04-25T11:42:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.051+0000 7f7aace00640 1 -- 172.21.15.19:0/2091989675 wait complete. 2024-04-25T11:42:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.054+0000 7f7aace00640 1 Processor -- start 2024-04-25T11:42:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.054+0000 7f7aace00640 1 -- start start 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.055+0000 7f7aace00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa80d44a0 0x7f7aa814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.055+0000 7f7aace00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa8164860 0x7f7aa80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aace00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80ba9c0 0x7f7aa80badd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aace00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa80d4df0 con 0x7f7aa8164860 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aace00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8164060 con 0x7f7aa80d44a0 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aace00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa80d4f70 con 0x7f7aa80ba9c0 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa8164860 0x7f7aa80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa8164860 0x7f7aa80c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40734/0 (socket says 172.21.15.19:40734) 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 -- 172.21.15.19:0/1923817225 learned_addr learned my addr 172.21.15.19:0/1923817225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa7400640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa80d44a0 0x7f7aa814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa7e00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80ba9c0 0x7f7aa80badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 -- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80ba9c0 msgr2=0x7f7aa80badd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80ba9c0 0x7f7aa80badd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 -- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa80d44a0 msgr2=0x7f7aa814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa80d44a0 0x7f7aa814a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa6a00640 1 -- 172.21.15.19:0/1923817225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa80bb310 con 0x7f7aa8164860 2024-04-25T11:42:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.056+0000 7f7aa7e00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80ba9c0 0x7f7aa80badd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.057+0000 7f7aa6a00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa8164860 0x7f7aa80c8510 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f7a900098c0 tx=0x7f7a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.057+0000 7f7a9f400640 1 -- 172.21.15.19:0/1923817225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90041070 con 0x7f7aa8164860 2024-04-25T11:42:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.057+0000 7f7a9f400640 1 -- 172.21.15.19:0/1923817225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9003c930 con 0x7f7aa8164860 2024-04-25T11:42:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.057+0000 7f7a9f400640 1 -- 172.21.15.19:0/1923817225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90037040 con 0x7f7aa8164860 2024-04-25T11:42:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.057+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa814a5f0 con 0x7f7aa8164860 2024-04-25T11:42:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.057+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa8147c20 con 0x7f7aa8164860 2024-04-25T11:42:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.059+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa8050740 con 0x7f7aa8164860 2024-04-25T11:42:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.060+0000 7f7a9f400640 1 -- 172.21.15.19:0/1923817225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a90044070 con 0x7f7aa8164860 2024-04-25T11:42:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.061+0000 7f7a9f400640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a840744b0 0x7f7a840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.061+0000 7f7a9f400640 1 -- 172.21.15.19:0/1923817225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f7a900b5dd0 con 0x7f7aa8164860 2024-04-25T11:42:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.062+0000 7f7aa7400640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a840744b0 0x7f7a840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.063+0000 7f7aa7400640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a840744b0 0x7f7a840769c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0007ed0 tx=0x7f7aa0007e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.066+0000 7f7a9f400640 1 -- 172.21.15.19:0/1923817225 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7aa80bb310 con 0x7f7aa8164860 2024-04-25T11:42:33.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.310+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aa814a7c0 con 0x7f7aa8164860 2024-04-25T11:42:33.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.312+0000 7f7a9f400640 1 -- 172.21.15.19:0/1923817225 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 364 v364) ==== 95+0+5195 (secure 0 0 0) 0x7f7a900804c0 con 0x7f7aa8164860 2024-04-25T11:42:33.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:33.300 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":364,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":8020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":358}],"filesystems":[{"mdsmap":{"epoch":364,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:32.307292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8032},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8032":{"gid":8032,"name":"b","rank":0,"incarnation":360,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2610059988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2610059988},{"type":"v1","addr":"172.21.15.169:6835","nonce":2610059988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":7}]} 2024-04-25T11:42:33.300 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 364 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.314+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a840744b0 msgr2=0x7f7a840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.314+0000 7f7aace00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a840744b0 0x7f7a840769c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0007ed0 tx=0x7f7aa0007e60 comp rx=0 tx=0).stop 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.314+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa8164860 msgr2=0x7f7aa80c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.314+0000 7f7aace00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa8164860 0x7f7aa80c8510 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f7a900098c0 tx=0x7f7a900046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 shutdown_connections 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a840744b0 0x7f7a840769c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7aa80ba9c0 0x7f7aa80badd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7aa8164860 0x7f7aa80c8510 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 --2- 172.21.15.19:0/1923817225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7aa80d44a0 0x7f7aa814a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 >> 172.21.15.19:0/1923817225 conn(0x7f7aa80d2720 msgr2=0x7f7aa80b9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 shutdown_connections 2024-04-25T11:42:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.315+0000 7f7aace00640 1 -- 172.21.15.19:0/1923817225 wait complete. 2024-04-25T11:42:33.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.633+0000 7f3884800640 1 Processor -- start 2024-04-25T11:42:33.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.633+0000 7f3884800640 1 -- start start 2024-04-25T11:42:33.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.635+0000 7f3884800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f387c1443c0 0x7f387c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.635+0000 7f3884800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f387c0cfcf0 0x7f387c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.635+0000 7f3884800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387c15f5f0 con 0x7f387c070370 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.635+0000 7f3884800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f387c0708d0 con 0x7f387c1443c0 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.635+0000 7f3884800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f387c15f160 con 0x7f387c0cfcf0 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3881e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f387c1443c0 0x7f387c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3881e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f387c1443c0 0x7f387c06fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40738/0 (socket says 172.21.15.19:40738) 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3881e00640 1 -- 172.21.15.19:0/535870702 learned_addr learned my addr 172.21.15.19:0/535870702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3882800640 1 --2- 172.21.15.19:0/535870702 >> v2:172.21.15.169:3300/0 conn(0x7f387c0cfcf0 0x7f387c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3881e00640 1 -- 172.21.15.19:0/535870702 >> v2:172.21.15.169:3300/0 conn(0x7f387c0cfcf0 msgr2=0x7f387c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3881e00640 1 --2- 172.21.15.19:0/535870702 >> v2:172.21.15.169:3300/0 conn(0x7f387c0cfcf0 0x7f387c06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3881e00640 1 -- 172.21.15.19:0/535870702 >> v1:172.21.15.19:6789/0 conn(0x7f387c070370 legacy=0x7f387c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3881e00640 1 -- 172.21.15.19:0/535870702 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c15e830 con 0x7f387c1443c0 2024-04-25T11:42:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.636+0000 7f3880a00640 1 -- 172.21.15.19:0/535870702 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3976437911 0 0) 0x7f387c15f5f0 con 0x7f387c070370 2024-04-25T11:42:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.638+0000 7f3881e00640 1 --2- 172.21.15.19:0/535870702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c1443c0 0x7f387c06fe30 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f387c15c280 tx=0x7f387002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6e7be71f58afe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.638+0000 7f3880a00640 1 -- 172.21.15.19:0/535870702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870037070 con 0x7f387c1443c0 2024-04-25T11:42:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.638+0000 7f3880a00640 1 -- 172.21.15.19:0/535870702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3870032440 con 0x7f387c1443c0 2024-04-25T11:42:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.638+0000 7f3880a00640 1 -- 172.21.15.19:0/535870702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870032810 con 0x7f387c1443c0 2024-04-25T11:42:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 -- 172.21.15.19:0/535870702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c1443c0 msgr2=0x7f387c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 --2- 172.21.15.19:0/535870702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c1443c0 0x7f387c06fe30 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f387c15c280 tx=0x7f387002a270 comp rx=0 tx=0).stop 2024-04-25T11:42:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 -- 172.21.15.19:0/535870702 shutdown_connections 2024-04-25T11:42:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 --2- 172.21.15.19:0/535870702 >> v2:172.21.15.169:3300/0 conn(0x7f387c0cfcf0 0x7f387c06c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 --2- 172.21.15.19:0/535870702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c1443c0 0x7f387c06fe30 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 -- 172.21.15.19:0/535870702 >> 172.21.15.19:0/535870702 conn(0x7f387c0d1ae0 msgr2=0x7f387c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 -- 172.21.15.19:0/535870702 shutdown_connections 2024-04-25T11:42:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.639+0000 7f3884800640 1 -- 172.21.15.19:0/535870702 wait complete. 2024-04-25T11:42:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.641+0000 7f3884800640 1 Processor -- start 2024-04-25T11:42:33.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.641+0000 7f3884800640 1 -- start start 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3884800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c070370 0x7f387c0aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3884800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f387c1443c0 0x7f387c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3884800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387c1f8450 0x7f387c1f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3884800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f387c1447d0 con 0x7f387c070370 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3884800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f387c15edf0 con 0x7f387c1f8450 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3884800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f387c162b00 con 0x7f387c1443c0 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3881e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c070370 0x7f387c0aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3881e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c070370 0x7f387c0aade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40748/0 (socket says 172.21.15.19:40748) 2024-04-25T11:42:33.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.643+0000 7f3881e00640 1 -- 172.21.15.19:0/1420693641 learned_addr learned my addr 172.21.15.19:0/1420693641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3882800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387c1f8450 0x7f387c1f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3881400640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f387c1443c0 0x7f387c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3881e00640 1 -- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f387c1443c0 msgr2=0x7f387c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3881e00640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f387c1443c0 0x7f387c159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3881e00640 1 -- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387c1f8450 msgr2=0x7f387c1f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3881e00640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387c1f8450 0x7f387c1f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3881e00640 1 -- 172.21.15.19:0/1420693641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c1422b0 con 0x7f387c070370 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3882800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387c1f8450 0x7f387c1f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.644+0000 7f3881e00640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c070370 0x7f387c0aade0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f387002a2b0 tx=0x7f38700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.645+0000 7f3876000640 1 -- 172.21.15.19:0/1420693641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870041070 con 0x7f387c070370 2024-04-25T11:42:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.645+0000 7f3876000640 1 -- 172.21.15.19:0/1420693641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387003c870 con 0x7f387c070370 2024-04-25T11:42:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.645+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c142b30 con 0x7f387c070370 2024-04-25T11:42:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.645+0000 7f3876000640 1 -- 172.21.15.19:0/1420693641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870037040 con 0x7f387c070370 2024-04-25T11:42:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.645+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c142850 con 0x7f387c070370 2024-04-25T11:42:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.646+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387c050740 con 0x7f387c070370 2024-04-25T11:42:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.647+0000 7f3876000640 1 -- 172.21.15.19:0/1420693641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3870044070 con 0x7f387c070370 2024-04-25T11:42:33.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.648+0000 7f3876000640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38600744b0 0x7f38600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.649+0000 7f3876000640 1 -- 172.21.15.19:0/1420693641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f38700b5d30 con 0x7f387c070370 2024-04-25T11:42:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.649+0000 7f3881400640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38600744b0 0x7f38600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.650+0000 7f3881400640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38600744b0 0x7f38600769c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f387c0b2250 tx=0x7f386c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.654+0000 7f3876000640 1 -- 172.21.15.19:0/1420693641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3870080420 con 0x7f387c070370 2024-04-25T11:42:33.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.863+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f387c0ab550 con 0x7f387c070370 2024-04-25T11:42:33.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.865+0000 7f3876000640 1 -- 172.21.15.19:0/1420693641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16619 (secure 0 0 0) 0x7f387007fd20 con 0x7f387c070370 2024-04-25T11:42:33.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:33.853 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":143,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:30.672424+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:33.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38600744b0 msgr2=0x7f38600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38600744b0 0x7f38600769c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f387c0b2250 tx=0x7f386c009040 comp rx=0 tx=0).stop 2024-04-25T11:42:33.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c070370 msgr2=0x7f387c0aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:33.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c070370 0x7f387c0aade0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f387002a2b0 tx=0x7f38700046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 shutdown_connections 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38600744b0 0x7f38600769c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f387c1f8450 0x7f387c1f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f387c1443c0 0x7f387c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 --2- 172.21.15.19:0/1420693641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f387c070370 0x7f387c0aade0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 >> 172.21.15.19:0/1420693641 conn(0x7f387c0d1ae0 msgr2=0x7f387c0b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 shutdown_connections 2024-04-25T11:42:33.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:33.868+0000 7f3884800640 1 -- 172.21.15.19:0/1420693641 wait complete. 2024-04-25T11:42:33.881 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:42:33.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:42:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.165+0000 7f9762000640 1 Processor -- start 2024-04-25T11:42:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.165+0000 7f9762000640 1 -- start start 2024-04-25T11:42:34.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.167+0000 7f9762000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f975c0b6e10 0x7f975c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.167+0000 7f9762000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975c0b7220 con 0x7f975c1608d0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.167+0000 7f9762000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975c0b74d0 con 0x7f975c0cf2f0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.167+0000 7f9762000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f975c06f870 con 0x7f975c0b6e10 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.167+0000 7f9761600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f975c1608d0 0x7f975c160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50806/0 (socket says 172.21.15.19:50806) 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.167+0000 7f9761600640 1 -- 172.21.15.19:0/1710461915 learned_addr learned my addr 172.21.15.19:0/1710461915 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975be00640 1 --2- 172.21.15.19:0/1710461915 >> v2:172.21.15.169:3300/0 conn(0x7f975c0b6e10 0x7f975c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123150352 0 0) 0x7f975c0b7220 con 0x7f975c1608d0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9740003520 con 0x7f975c1608d0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 702080030 0 0) 0x7f9740003520 con 0x7f975c1608d0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 >> v2:172.21.15.169:3300/0 conn(0x7f975c0b6e10 msgr2=0x7f975c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 --2- 172.21.15.19:0/1710461915 >> v2:172.21.15.169:3300/0 conn(0x7f975c0b6e10 0x7f975c15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 >> v1:172.21.15.169:6789/0 conn(0x7f975c0cf2f0 legacy=0x7f975c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c0b77b0 con 0x7f975c1608d0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f9754001670 con 0x7f975c1608d0 2024-04-25T11:42:34.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 >> v1:172.21.15.19:6789/0 conn(0x7f975c1608d0 legacy=0x7f975c160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97400039b0 0x7f9740003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f9740004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97400051c0 0x7f974000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f975c06f870 con 0x7f9740004800 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f974000c160 con 0x7f97400039b0 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.168+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f974000c360 con 0x7f97400051c0 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f975be00640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f9740004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f975be00640 1 -- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97400051c0 msgr2=0x7f974000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f9760c00640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97400039b0 0x7f9740003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f975be00640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f97400051c0 0x7f974000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f975be00640 1 -- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97400039b0 msgr2=0x7f9740003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f975be00640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97400039b0 0x7f9740003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f975be00640 1 -- 172.21.15.19:0/1710461915 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9740003520 con 0x7f9740004800 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f9761600640 1 -- 172.21.15.19:0/1710461915 reap_dead start 2024-04-25T11:42:34.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.169+0000 7f9760c00640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97400039b0 0x7f9740003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:34.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.170+0000 7f975be00640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f9740004c10 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f9748009ac0 tx=0x7f9740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d35a153634d902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:34.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748036070 con 0x7f9740004800 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748031400 con 0x7f9740004800 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f975b400640 1 -- 172.21.15.19:0/1710461915 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748009690 con 0x7f9740004800 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f9762000640 1 -- 172.21.15.19:0/1710461915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 msgr2=0x7f9740004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f9762000640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f9740004c10 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f9748009ac0 tx=0x7f9740005830 comp rx=0 tx=0).stop 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f9762000640 1 -- 172.21.15.19:0/1710461915 shutdown_connections 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f9762000640 1 --2- 172.21.15.19:0/1710461915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f9740004c10 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f9762000640 1 -- 172.21.15.19:0/1710461915 >> 172.21.15.19:0/1710461915 conn(0x7f975c0d1720 msgr2=0x7f975c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f9762000640 1 -- 172.21.15.19:0/1710461915 shutdown_connections 2024-04-25T11:42:34.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.171+0000 7f9762000640 1 -- 172.21.15.19:0/1710461915 wait complete. 2024-04-25T11:42:34.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.174+0000 7f9762000640 1 Processor -- start 2024-04-25T11:42:34.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.174+0000 7f9762000640 1 -- start start 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.175+0000 7f9762000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f975c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.175+0000 7f9762000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975c15e360 0x7f975c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.175+0000 7f9762000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975c072ea0 0x7f975c0732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.175+0000 7f9762000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f975c15a810 con 0x7f9740004800 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.175+0000 7f9762000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f975c06f500 con 0x7f975c072ea0 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.175+0000 7f9762000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f975c15a990 con 0x7f975c15e360 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f975c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f975c1595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54902/0 (socket says 172.21.15.19:54902) 2024-04-25T11:42:34.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 -- 172.21.15.19:0/4025883080 learned_addr learned my addr 172.21.15.19:0/4025883080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f975be00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975c15e360 0x7f975c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9761600640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975c072ea0 0x7f975c0732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 -- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975c15e360 msgr2=0x7f975c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975c15e360 0x7f975c15e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 -- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975c072ea0 msgr2=0x7f975c0732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975c072ea0 0x7f975c0732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9760c00640 1 -- 172.21.15.19:0/4025883080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c0737f0 con 0x7f9740004800 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f9761600640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975c072ea0 0x7f975c0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.176+0000 7f975be00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975c15e360 0x7f975c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.177+0000 7f9760c00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f975c1595f0 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f974c00d830 tx=0x7f974c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.177+0000 7f9758c00640 1 -- 172.21.15.19:0/4025883080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c003da0 con 0x7f9740004800 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.177+0000 7f9758c00640 1 -- 172.21.15.19:0/4025883080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c017070 con 0x7f9740004800 2024-04-25T11:42:34.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.177+0000 7f9758c00640 1 -- 172.21.15.19:0/4025883080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c00b8f0 con 0x7f9740004800 2024-04-25T11:42:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.177+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c159b30 con 0x7f9740004800 2024-04-25T11:42:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.177+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c15ba10 con 0x7f9740004800 2024-04-25T11:42:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.178+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c050740 con 0x7f9740004800 2024-04-25T11:42:34.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.180+0000 7f9758c00640 1 -- 172.21.15.19:0/4025883080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f974c00ba90 con 0x7f9740004800 2024-04-25T11:42:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.180+0000 7f9758c00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f973c0744e0 0x7f973c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.181+0000 7f9758c00640 1 -- 172.21.15.19:0/4025883080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f974c0952d0 con 0x7f9740004800 2024-04-25T11:42:34.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.181+0000 7f975be00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f973c0744e0 0x7f973c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.183+0000 7f975be00640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f973c0744e0 0x7f973c0769f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9748033000 tx=0x7f9748008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:34.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.185+0000 7f9758c00640 1 -- 172.21.15.19:0/4025883080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f975c0737f0 con 0x7f9740004800 2024-04-25T11:42:34.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.383+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f975c06c770 con 0x7f973c0744e0 2024-04-25T11:42:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.405+0000 7f9758c00640 1 -- 172.21.15.19:0/4025883080 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252471 (secure 0 0 0) 0x7f975c06c770 con 0x7f973c0744e0 2024-04-25T11:42:34.396 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:42:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f973c0744e0 msgr2=0x7f973c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f973c0744e0 0x7f973c0769f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9748033000 tx=0x7f9748008030 comp rx=0 tx=0).stop 2024-04-25T11:42:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 msgr2=0x7f975c1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f975c1595f0 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f974c00d830 tx=0x7f974c00dcf0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 shutdown_connections 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f973c0744e0 0x7f973c0769f0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f975c072ea0 0x7f975c0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f975c15e360 0x7f975c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 --2- 172.21.15.19:0/4025883080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9740004800 0x7f975c1595f0 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.411+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 >> 172.21.15.19:0/4025883080 conn(0x7f975c0d1720 msgr2=0x7f975c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.412+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 shutdown_connections 2024-04-25T11:42:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.412+0000 7f9762000640 1 -- 172.21.15.19:0/4025883080 wait complete. 2024-04-25T11:42:34.427 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2024-04-25T11:42:32.831261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":298248,"kb_used_data":15496,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754676472,"statfs":{"total":773094113280,"available":772788707328,"internally_reserved":0,"allocated":15867904,"data_stored":8226732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220579"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759567+0000","last_change":"2024-04-25T11:42:29.696855+0000","last_active":"2024-04-25T11:42:30.759567+0000","last_peered":"2024-04-25T11:42:30.759567+0000","last_clean":"2024-04-25T11:42:30.759567+0000","last_became_active":"2024-04-25T11:42:29.696496+0000","last_became_peered":"2024-04-25T11:42:29.696496+0000","last_unstale":"2024-04-25T11:42:30.759567+0000","last_undegraded":"2024-04-25T11:42:30.759567+0000","last_fullsized":"2024-04-25T11:42:30.759567+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:45.369012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759395+0000","last_change":"2024-04-25T11:42:29.693167+0000","last_active":"2024-04-25T11:42:30.759395+0000","last_peered":"2024-04-25T11:42:30.759395+0000","last_clean":"2024-04-25T11:42:30.759395+0000","last_became_active":"2024-04-25T11:42:29.692804+0000","last_became_peered":"2024-04-25T11:42:29.692804+0000","last_unstale":"2024-04-25T11:42:30.759395+0000","last_undegraded":"2024-04-25T11:42:30.759395+0000","last_fullsized":"2024-04-25T11:42:30.759395+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:55.724030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686395+0000","last_change":"2024-04-25T11:42:29.692020+0000","last_active":"2024-04-25T11:42:30.686395+0000","last_peered":"2024-04-25T11:42:30.686395+0000","last_clean":"2024-04-25T11:42:30.686395+0000","last_became_active":"2024-04-25T11:42:29.691674+0000","last_became_peered":"2024-04-25T11:42:29.691674+0000","last_unstale":"2024-04-25T11:42:30.686395+0000","last_undegraded":"2024-04-25T11:42:30.686395+0000","last_fullsized":"2024-04-25T11:42:30.686395+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:50.265624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758557+0000","last_change":"2024-04-25T11:42:29.701146+0000","last_active":"2024-04-25T11:42:30.758557+0000","last_peered":"2024-04-25T11:42:30.758557+0000","last_clean":"2024-04-25T11:42:30.758557+0000","last_became_active":"2024-04-25T11:42:29.700667+0000","last_became_peered":"2024-04-25T11:42:29.700667+0000","last_unstale":"2024-04-25T11:42:30.758557+0000","last_undegraded":"2024-04-25T11:42:30.758557+0000","last_fullsized":"2024-04-25T11:42:30.758557+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:26.268376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683239+0000","last_change":"2024-04-25T11:42:29.674388+0000","last_active":"2024-04-25T11:42:30.683239+0000","last_peered":"2024-04-25T11:42:30.683239+0000","last_clean":"2024-04-25T11:42:30.683239+0000","last_became_active":"2024-04-25T11:42:29.673550+0000","last_became_peered":"2024-04-25T11:42:29.673550+0000","last_unstale":"2024-04-25T11:42:30.683239+0000","last_undegraded":"2024-04-25T11:42:30.683239+0000","last_fullsized":"2024-04-25T11:42:30.683239+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:31.102557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684308+0000","last_change":"2024-04-25T11:42:29.694222+0000","last_active":"2024-04-25T11:42:30.684308+0000","last_peered":"2024-04-25T11:42:30.684308+0000","last_clean":"2024-04-25T11:42:30.684308+0000","last_became_active":"2024-04-25T11:42:29.690316+0000","last_became_peered":"2024-04-25T11:42:29.690316+0000","last_unstale":"2024-04-25T11:42:30.684308+0000","last_undegraded":"2024-04-25T11:42:30.684308+0000","last_fullsized":"2024-04-25T11:42:30.684308+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.909590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683600+0000","last_change":"2024-04-25T11:42:29.698842+0000","last_active":"2024-04-25T11:42:30.683600+0000","last_peered":"2024-04-25T11:42:30.683600+0000","last_clean":"2024-04-25T11:42:30.683600+0000","last_became_active":"2024-04-25T11:42:29.697764+0000","last_became_peered":"2024-04-25T11:42:29.697764+0000","last_unstale":"2024-04-25T11:42:30.683600+0000","last_undegraded":"2024-04-25T11:42:30.683600+0000","last_fullsized":"2024-04-25T11:42:30.683600+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:02.461138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684097+0000","last_change":"2024-04-25T11:42:29.687970+0000","last_active":"2024-04-25T11:42:30.684097+0000","last_peered":"2024-04-25T11:42:30.684097+0000","last_clean":"2024-04-25T11:42:30.684097+0000","last_became_active":"2024-04-25T11:42:29.687251+0000","last_became_peered":"2024-04-25T11:42:29.687251+0000","last_unstale":"2024-04-25T11:42:30.684097+0000","last_undegraded":"2024-04-25T11:42:30.684097+0000","last_fullsized":"2024-04-25T11:42:30.684097+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.398770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683490+0000","last_change":"2024-04-25T11:42:29.682183+0000","last_active":"2024-04-25T11:42:30.683490+0000","last_peered":"2024-04-25T11:42:30.683490+0000","last_clean":"2024-04-25T11:42:30.683490+0000","last_became_active":"2024-04-25T11:42:29.681774+0000","last_became_peered":"2024-04-25T11:42:29.681774+0000","last_unstale":"2024-04-25T11:42:30.683490+0000","last_undegraded":"2024-04-25T11:42:30.683490+0000","last_fullsized":"2024-04-25T11:42:30.683490+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:40.612195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757989+0000","last_change":"2024-04-25T11:42:29.702312+0000","last_active":"2024-04-25T11:42:30.757989+0000","last_peered":"2024-04-25T11:42:30.757989+0000","last_clean":"2024-04-25T11:42:30.757989+0000","last_became_active":"2024-04-25T11:42:29.701810+0000","last_became_peered":"2024-04-25T11:42:29.701810+0000","last_unstale":"2024-04-25T11:42:30.757989+0000","last_undegraded":"2024-04-25T11:42:30.757989+0000","last_fullsized":"2024-04-25T11:42:30.757989+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:05.902696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683867+0000","last_change":"2024-04-25T11:42:29.690167+0000","last_active":"2024-04-25T11:42:30.683867+0000","last_peered":"2024-04-25T11:42:30.683867+0000","last_clean":"2024-04-25T11:42:30.683867+0000","last_became_active":"2024-04-25T11:42:29.689673+0000","last_became_peered":"2024-04-25T11:42:29.689673+0000","last_unstale":"2024-04-25T11:42:30.683867+0000","last_undegraded":"2024-04-25T11:42:30.683867+0000","last_fullsized":"2024-04-25T11:42:30.683867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:26.979184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758426+0000","last_change":"2024-04-25T11:42:29.701352+0000","last_active":"2024-04-25T11:42:30.758426+0000","last_peered":"2024-04-25T11:42:30.758426+0000","last_clean":"2024-04-25T11:42:30.758426+0000","last_became_active":"2024-04-25T11:42:29.701005+0000","last_became_peered":"2024-04-25T11:42:29.701005+0000","last_unstale":"2024-04-25T11:42:30.758426+0000","last_undegraded":"2024-04-25T11:42:30.758426+0000","last_fullsized":"2024-04-25T11:42:30.758426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.398770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688452+0000","last_change":"2024-04-25T11:42:29.693763+0000","last_active":"2024-04-25T11:42:30.688452+0000","last_peered":"2024-04-25T11:42:30.688452+0000","last_clean":"2024-04-25T11:42:30.688452+0000","last_became_active":"2024-04-25T11:42:29.691555+0000","last_became_peered":"2024-04-25T11:42:29.691555+0000","last_unstale":"2024-04-25T11:42:30.688452+0000","last_undegraded":"2024-04-25T11:42:30.688452+0000","last_fullsized":"2024-04-25T11:42:30.688452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:51.761630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758008+0000","last_change":"2024-04-25T11:42:29.706151+0000","last_active":"2024-04-25T11:42:30.758008+0000","last_peered":"2024-04-25T11:42:30.758008+0000","last_clean":"2024-04-25T11:42:30.758008+0000","last_became_active":"2024-04-25T11:42:29.705588+0000","last_became_peered":"2024-04-25T11:42:29.705588+0000","last_unstale":"2024-04-25T11:42:30.758008+0000","last_undegraded":"2024-04-25T11:42:30.758008+0000","last_fullsized":"2024-04-25T11:42:30.758008+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:48.013757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758331+0000","last_change":"2024-04-25T11:42:29.700899+0000","last_active":"2024-04-25T11:42:30.758331+0000","last_peered":"2024-04-25T11:42:30.758331+0000","last_clean":"2024-04-25T11:42:30.758331+0000","last_became_active":"2024-04-25T11:42:29.700467+0000","last_became_peered":"2024-04-25T11:42:29.700467+0000","last_unstale":"2024-04-25T11:42:30.758331+0000","last_undegraded":"2024-04-25T11:42:30.758331+0000","last_fullsized":"2024-04-25T11:42:30.758331+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.242694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.682939+0000","last_change":"2024-04-25T11:42:29.701331+0000","last_active":"2024-04-25T11:42:30.682939+0000","last_peered":"2024-04-25T11:42:30.682939+0000","last_clean":"2024-04-25T11:42:30.682939+0000","last_became_active":"2024-04-25T11:42:29.700988+0000","last_became_peered":"2024-04-25T11:42:29.700988+0000","last_unstale":"2024-04-25T11:42:30.682939+0000","last_undegraded":"2024-04-25T11:42:30.682939+0000","last_fullsized":"2024-04-25T11:42:30.682939+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:02.426781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703305+0000","last_change":"2024-04-25T11:42:29.709246+0000","last_active":"2024-04-25T11:42:30.703305+0000","last_peered":"2024-04-25T11:42:30.703305+0000","last_clean":"2024-04-25T11:42:30.703305+0000","last_became_active":"2024-04-25T11:42:29.707729+0000","last_became_peered":"2024-04-25T11:42:29.707729+0000","last_unstale":"2024-04-25T11:42:30.703305+0000","last_undegraded":"2024-04-25T11:42:30.703305+0000","last_fullsized":"2024-04-25T11:42:30.703305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:26.373977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759443+0000","last_change":"2024-04-25T11:42:29.689033+0000","last_active":"2024-04-25T11:42:30.759443+0000","last_peered":"2024-04-25T11:42:30.759443+0000","last_clean":"2024-04-25T11:42:30.759443+0000","last_became_active":"2024-04-25T11:42:29.687836+0000","last_became_peered":"2024-04-25T11:42:29.687836+0000","last_unstale":"2024-04-25T11:42:30.759443+0000","last_undegraded":"2024-04-25T11:42:30.759443+0000","last_fullsized":"2024-04-25T11:42:30.759443+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:50.265624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703731+0000","last_change":"2024-04-25T11:42:29.705202+0000","last_active":"2024-04-25T11:42:30.703731+0000","last_peered":"2024-04-25T11:42:30.703731+0000","last_clean":"2024-04-25T11:42:30.703731+0000","last_became_active":"2024-04-25T11:42:29.704800+0000","last_became_peered":"2024-04-25T11:42:29.704800+0000","last_unstale":"2024-04-25T11:42:30.703731+0000","last_undegraded":"2024-04-25T11:42:30.703731+0000","last_fullsized":"2024-04-25T11:42:30.703731+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:01.099157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758316+0000","last_change":"2024-04-25T11:42:29.689069+0000","last_active":"2024-04-25T11:42:30.758316+0000","last_peered":"2024-04-25T11:42:30.758316+0000","last_clean":"2024-04-25T11:42:30.758316+0000","last_became_active":"2024-04-25T11:42:29.688513+0000","last_became_peered":"2024-04-25T11:42:29.688513+0000","last_unstale":"2024-04-25T11:42:30.758316+0000","last_undegraded":"2024-04-25T11:42:30.758316+0000","last_fullsized":"2024-04-25T11:42:30.758316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:07.270671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685793+0000","last_change":"2024-04-25T11:42:29.705221+0000","last_active":"2024-04-25T11:42:30.685793+0000","last_peered":"2024-04-25T11:42:30.685793+0000","last_clean":"2024-04-25T11:42:30.685793+0000","last_became_active":"2024-04-25T11:42:29.702611+0000","last_became_peered":"2024-04-25T11:42:29.702611+0000","last_unstale":"2024-04-25T11:42:30.685793+0000","last_undegraded":"2024-04-25T11:42:30.685793+0000","last_fullsized":"2024-04-25T11:42:30.685793+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.013260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685213+0000","last_change":"2024-04-25T11:42:29.675532+0000","last_active":"2024-04-25T11:42:30.685213+0000","last_peered":"2024-04-25T11:42:30.685213+0000","last_clean":"2024-04-25T11:42:30.685213+0000","last_became_active":"2024-04-25T11:42:29.674709+0000","last_became_peered":"2024-04-25T11:42:29.674709+0000","last_unstale":"2024-04-25T11:42:30.685213+0000","last_undegraded":"2024-04-25T11:42:30.685213+0000","last_fullsized":"2024-04-25T11:42:30.685213+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:26.268376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758478+0000","last_change":"2024-04-25T11:42:29.689583+0000","last_active":"2024-04-25T11:42:30.758478+0000","last_peered":"2024-04-25T11:42:30.758478+0000","last_clean":"2024-04-25T11:42:30.758478+0000","last_became_active":"2024-04-25T11:42:29.689200+0000","last_became_peered":"2024-04-25T11:42:29.689200+0000","last_unstale":"2024-04-25T11:42:30.758478+0000","last_undegraded":"2024-04-25T11:42:30.758478+0000","last_fullsized":"2024-04-25T11:42:30.758478+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:00.346220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756666+0000","last_change":"2024-04-25T11:42:29.702678+0000","last_active":"2024-04-25T11:42:30.756666+0000","last_peered":"2024-04-25T11:42:30.756666+0000","last_clean":"2024-04-25T11:42:30.756666+0000","last_became_active":"2024-04-25T11:42:29.701064+0000","last_became_peered":"2024-04-25T11:42:29.701064+0000","last_unstale":"2024-04-25T11:42:30.756666+0000","last_undegraded":"2024-04-25T11:42:30.756666+0000","last_fullsized":"2024-04-25T11:42:30.756666+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:59.966683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758199+0000","last_change":"2024-04-25T11:42:29.713954+0000","last_active":"2024-04-25T11:42:30.758199+0000","last_peered":"2024-04-25T11:42:30.758199+0000","last_clean":"2024-04-25T11:42:30.758199+0000","last_became_active":"2024-04-25T11:42:29.713401+0000","last_became_peered":"2024-04-25T11:42:29.713401+0000","last_unstale":"2024-04-25T11:42:30.758199+0000","last_undegraded":"2024-04-25T11:42:30.758199+0000","last_fullsized":"2024-04-25T11:42:30.758199+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.763250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688693+0000","last_change":"2024-04-25T11:42:29.689859+0000","last_active":"2024-04-25T11:42:30.688693+0000","last_peered":"2024-04-25T11:42:30.688693+0000","last_clean":"2024-04-25T11:42:30.688693+0000","last_became_active":"2024-04-25T11:42:29.689517+0000","last_became_peered":"2024-04-25T11:42:29.689517+0000","last_unstale":"2024-04-25T11:42:30.688693+0000","last_undegraded":"2024-04-25T11:42:30.688693+0000","last_fullsized":"2024-04-25T11:42:30.688693+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:05.599441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757018+0000","last_change":"2024-04-25T11:42:29.694451+0000","last_active":"2024-04-25T11:42:30.757018+0000","last_peered":"2024-04-25T11:42:30.757018+0000","last_clean":"2024-04-25T11:42:30.757018+0000","last_became_active":"2024-04-25T11:42:29.693670+0000","last_became_peered":"2024-04-25T11:42:29.693670+0000","last_unstale":"2024-04-25T11:42:30.757018+0000","last_undegraded":"2024-04-25T11:42:30.757018+0000","last_fullsized":"2024-04-25T11:42:30.757018+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:14.975091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757385+0000","last_change":"2024-04-25T11:42:29.688082+0000","last_active":"2024-04-25T11:42:30.757385+0000","last_peered":"2024-04-25T11:42:30.757385+0000","last_clean":"2024-04-25T11:42:30.757385+0000","last_became_active":"2024-04-25T11:42:29.687705+0000","last_became_peered":"2024-04-25T11:42:29.687705+0000","last_unstale":"2024-04-25T11:42:30.757385+0000","last_undegraded":"2024-04-25T11:42:30.757385+0000","last_fullsized":"2024-04-25T11:42:30.757385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:43.651140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688493+0000","last_change":"2024-04-25T11:42:29.694587+0000","last_active":"2024-04-25T11:42:30.688493+0000","last_peered":"2024-04-25T11:42:30.688493+0000","last_clean":"2024-04-25T11:42:30.688493+0000","last_became_active":"2024-04-25T11:42:29.694204+0000","last_became_peered":"2024-04-25T11:42:29.694204+0000","last_unstale":"2024-04-25T11:42:30.688493+0000","last_undegraded":"2024-04-25T11:42:30.688493+0000","last_fullsized":"2024-04-25T11:42:30.688493+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:03.196909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688018+0000","last_change":"2024-04-25T11:42:29.706810+0000","last_active":"2024-04-25T11:42:30.688018+0000","last_peered":"2024-04-25T11:42:30.688018+0000","last_clean":"2024-04-25T11:42:30.688018+0000","last_became_active":"2024-04-25T11:42:29.706286+0000","last_became_peered":"2024-04-25T11:42:29.706286+0000","last_unstale":"2024-04-25T11:42:30.688018+0000","last_undegraded":"2024-04-25T11:42:30.688018+0000","last_fullsized":"2024-04-25T11:42:30.688018+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:59.970047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684821+0000","last_change":"2024-04-25T11:42:29.699173+0000","last_active":"2024-04-25T11:42:30.684821+0000","last_peered":"2024-04-25T11:42:30.684821+0000","last_clean":"2024-04-25T11:42:30.684821+0000","last_became_active":"2024-04-25T11:42:29.698859+0000","last_became_peered":"2024-04-25T11:42:29.698859+0000","last_unstale":"2024-04-25T11:42:30.684821+0000","last_undegraded":"2024-04-25T11:42:30.684821+0000","last_fullsized":"2024-04-25T11:42:30.684821+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:00.346220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758042+0000","last_change":"2024-04-25T11:42:29.704384+0000","last_active":"2024-04-25T11:42:30.758042+0000","last_peered":"2024-04-25T11:42:30.758042+0000","last_clean":"2024-04-25T11:42:30.758042+0000","last_became_active":"2024-04-25T11:42:29.703841+0000","last_became_peered":"2024-04-25T11:42:29.703841+0000","last_unstale":"2024-04-25T11:42:30.758042+0000","last_undegraded":"2024-04-25T11:42:30.758042+0000","last_fullsized":"2024-04-25T11:42:30.758042+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:07.270671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703461+0000","last_change":"2024-04-25T11:42:29.702175+0000","last_active":"2024-04-25T11:42:30.703461+0000","last_peered":"2024-04-25T11:42:30.703461+0000","last_clean":"2024-04-25T11:42:30.703461+0000","last_became_active":"2024-04-25T11:42:29.701784+0000","last_became_peered":"2024-04-25T11:42:29.701784+0000","last_unstale":"2024-04-25T11:42:30.703461+0000","last_undegraded":"2024-04-25T11:42:30.703461+0000","last_fullsized":"2024-04-25T11:42:30.703461+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:05.304391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"142'4","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.689286+0000","last_change":"2024-04-25T11:42:28.662201+0000","last_active":"2024-04-25T11:42:30.689286+0000","last_peered":"2024-04-25T11:42:30.689286+0000","last_clean":"2024-04-25T11:42:30.689286+0000","last_became_active":"2024-04-25T11:42:28.661792+0000","last_became_peered":"2024-04-25T11:42:28.661792+0000","last_unstale":"2024-04-25T11:42:30.689286+0000","last_undegraded":"2024-04-25T11:42:30.689286+0000","last_fullsized":"2024-04-25T11:42:30.689286+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:57.047396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.689453+0000","last_change":"2024-04-25T11:42:29.693546+0000","last_active":"2024-04-25T11:42:30.689453+0000","last_peered":"2024-04-25T11:42:30.689453+0000","last_clean":"2024-04-25T11:42:30.689453+0000","last_became_active":"2024-04-25T11:42:29.693163+0000","last_became_peered":"2024-04-25T11:42:29.693163+0000","last_unstale":"2024-04-25T11:42:30.689453+0000","last_undegraded":"2024-04-25T11:42:30.689453+0000","last_fullsized":"2024-04-25T11:42:30.689453+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:42.882880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683930+0000","last_change":"2024-04-25T11:42:28.660531+0000","last_active":"2024-04-25T11:42:30.683930+0000","last_peered":"2024-04-25T11:42:30.683930+0000","last_clean":"2024-04-25T11:42:30.683930+0000","last_became_active":"2024-04-25T11:42:28.660194+0000","last_became_peered":"2024-04-25T11:42:28.660194+0000","last_unstale":"2024-04-25T11:42:30.683930+0000","last_undegraded":"2024-04-25T11:42:30.683930+0000","last_fullsized":"2024-04-25T11:42:30.683930+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:39.335110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759087+0000","last_change":"2024-04-25T11:42:29.697958+0000","last_active":"2024-04-25T11:42:30.759087+0000","last_peered":"2024-04-25T11:42:30.759087+0000","last_clean":"2024-04-25T11:42:30.759087+0000","last_became_active":"2024-04-25T11:42:29.697444+0000","last_became_peered":"2024-04-25T11:42:29.697444+0000","last_unstale":"2024-04-25T11:42:30.759087+0000","last_undegraded":"2024-04-25T11:42:30.759087+0000","last_fullsized":"2024-04-25T11:42:30.759087+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.926190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686590+0000","last_change":"2024-04-25T11:42:28.663742+0000","last_active":"2024-04-25T11:42:30.686590+0000","last_peered":"2024-04-25T11:42:30.686590+0000","last_clean":"2024-04-25T11:42:30.686590+0000","last_became_active":"2024-04-25T11:42:28.662533+0000","last_became_peered":"2024-04-25T11:42:28.662533+0000","last_unstale":"2024-04-25T11:42:30.686590+0000","last_undegraded":"2024-04-25T11:42:30.686590+0000","last_fullsized":"2024-04-25T11:42:30.686590+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757734+0000","last_change":"2024-04-25T11:42:29.700925+0000","last_active":"2024-04-25T11:42:30.757734+0000","last_peered":"2024-04-25T11:42:30.757734+0000","last_clean":"2024-04-25T11:42:30.757734+0000","last_became_active":"2024-04-25T11:42:29.700354+0000","last_became_peered":"2024-04-25T11:42:29.700354+0000","last_unstale":"2024-04-25T11:42:30.757734+0000","last_undegraded":"2024-04-25T11:42:30.757734+0000","last_fullsized":"2024-04-25T11:42:30.757734+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:25.104518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759009+0000","last_change":"2024-04-25T11:42:28.657525+0000","last_active":"2024-04-25T11:42:30.759009+0000","last_peered":"2024-04-25T11:42:30.759009+0000","last_clean":"2024-04-25T11:42:30.759009+0000","last_became_active":"2024-04-25T11:42:28.656726+0000","last_became_peered":"2024-04-25T11:42:28.656726+0000","last_unstale":"2024-04-25T11:42:30.759009+0000","last_undegraded":"2024-04-25T11:42:30.759009+0000","last_fullsized":"2024-04-25T11:42:30.759009+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:50.745291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757970+0000","last_change":"2024-04-25T11:42:29.690685+0000","last_active":"2024-04-25T11:42:30.757970+0000","last_peered":"2024-04-25T11:42:30.757970+0000","last_clean":"2024-04-25T11:42:30.757970+0000","last_became_active":"2024-04-25T11:42:29.690312+0000","last_became_peered":"2024-04-25T11:42:29.690312+0000","last_unstale":"2024-04-25T11:42:30.757970+0000","last_undegraded":"2024-04-25T11:42:30.757970+0000","last_fullsized":"2024-04-25T11:42:30.757970+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:48.013757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758021+0000","last_change":"2024-04-25T11:42:28.655218+0000","last_active":"2024-04-25T11:42:30.758021+0000","last_peered":"2024-04-25T11:42:30.758021+0000","last_clean":"2024-04-25T11:42:30.758021+0000","last_became_active":"2024-04-25T11:42:28.649304+0000","last_became_peered":"2024-04-25T11:42:28.649304+0000","last_unstale":"2024-04-25T11:42:30.758021+0000","last_undegraded":"2024-04-25T11:42:30.758021+0000","last_fullsized":"2024-04-25T11:42:30.758021+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:12.300553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683660+0000","last_change":"2024-04-25T11:42:29.690861+0000","last_active":"2024-04-25T11:42:30.683660+0000","last_peered":"2024-04-25T11:42:30.683660+0000","last_clean":"2024-04-25T11:42:30.683660+0000","last_became_active":"2024-04-25T11:42:29.690468+0000","last_became_peered":"2024-04-25T11:42:29.690468+0000","last_unstale":"2024-04-25T11:42:30.683660+0000","last_undegraded":"2024-04-25T11:42:30.683660+0000","last_fullsized":"2024-04-25T11:42:30.683660+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.866512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685383+0000","last_change":"2024-04-25T11:42:28.654295+0000","last_active":"2024-04-25T11:42:30.685383+0000","last_peered":"2024-04-25T11:42:30.685383+0000","last_clean":"2024-04-25T11:42:30.685383+0000","last_became_active":"2024-04-25T11:42:28.653980+0000","last_became_peered":"2024-04-25T11:42:28.653980+0000","last_unstale":"2024-04-25T11:42:30.685383+0000","last_undegraded":"2024-04-25T11:42:30.685383+0000","last_fullsized":"2024-04-25T11:42:30.685383+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:41.866541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685474+0000","last_change":"2024-04-25T11:42:29.680408+0000","last_active":"2024-04-25T11:42:30.685474+0000","last_peered":"2024-04-25T11:42:30.685474+0000","last_clean":"2024-04-25T11:42:30.685474+0000","last_became_active":"2024-04-25T11:42:29.678815+0000","last_became_peered":"2024-04-25T11:42:29.678815+0000","last_unstale":"2024-04-25T11:42:30.685474+0000","last_undegraded":"2024-04-25T11:42:30.685474+0000","last_fullsized":"2024-04-25T11:42:30.685474+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.316892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.682968+0000","last_change":"2024-04-25T11:42:28.655419+0000","last_active":"2024-04-25T11:42:30.682968+0000","last_peered":"2024-04-25T11:42:30.682968+0000","last_clean":"2024-04-25T11:42:30.682968+0000","last_became_active":"2024-04-25T11:42:28.654795+0000","last_became_peered":"2024-04-25T11:42:28.654795+0000","last_unstale":"2024-04-25T11:42:30.682968+0000","last_undegraded":"2024-04-25T11:42:30.682968+0000","last_fullsized":"2024-04-25T11:42:30.682968+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:31.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756365+0000","last_change":"2024-04-25T11:42:29.688271+0000","last_active":"2024-04-25T11:42:30.756365+0000","last_peered":"2024-04-25T11:42:30.756365+0000","last_clean":"2024-04-25T11:42:30.756365+0000","last_became_active":"2024-04-25T11:42:29.687091+0000","last_became_peered":"2024-04-25T11:42:29.687091+0000","last_unstale":"2024-04-25T11:42:30.756365+0000","last_undegraded":"2024-04-25T11:42:30.756365+0000","last_fullsized":"2024-04-25T11:42:30.756365+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.763250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.702685+0000","last_change":"2024-04-25T11:42:28.647128+0000","last_active":"2024-04-25T11:42:30.702685+0000","last_peered":"2024-04-25T11:42:30.702685+0000","last_clean":"2024-04-25T11:42:30.702685+0000","last_became_active":"2024-04-25T11:42:28.645733+0000","last_became_peered":"2024-04-25T11:42:28.645733+0000","last_unstale":"2024-04-25T11:42:30.702685+0000","last_undegraded":"2024-04-25T11:42:30.702685+0000","last_fullsized":"2024-04-25T11:42:30.702685+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:42.492482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759651+0000","last_change":"2024-04-25T11:42:29.706581+0000","last_active":"2024-04-25T11:42:30.759651+0000","last_peered":"2024-04-25T11:42:30.759651+0000","last_clean":"2024-04-25T11:42:30.759651+0000","last_became_active":"2024-04-25T11:42:29.704945+0000","last_became_peered":"2024-04-25T11:42:29.704945+0000","last_unstale":"2024-04-25T11:42:30.759651+0000","last_undegraded":"2024-04-25T11:42:30.759651+0000","last_fullsized":"2024-04-25T11:42:30.759651+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:00.342533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686534+0000","last_change":"2024-04-25T11:42:28.654392+0000","last_active":"2024-04-25T11:42:30.686534+0000","last_peered":"2024-04-25T11:42:30.686534+0000","last_clean":"2024-04-25T11:42:30.686534+0000","last_became_active":"2024-04-25T11:42:28.654052+0000","last_became_peered":"2024-04-25T11:42:28.654052+0000","last_unstale":"2024-04-25T11:42:30.686534+0000","last_undegraded":"2024-04-25T11:42:30.686534+0000","last_fullsized":"2024-04-25T11:42:30.686534+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:57.662132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757933+0000","last_change":"2024-04-25T11:42:29.695665+0000","last_active":"2024-04-25T11:42:30.757933+0000","last_peered":"2024-04-25T11:42:30.757933+0000","last_clean":"2024-04-25T11:42:30.757933+0000","last_became_active":"2024-04-25T11:42:29.694403+0000","last_became_peered":"2024-04-25T11:42:29.694403+0000","last_unstale":"2024-04-25T11:42:30.757933+0000","last_undegraded":"2024-04-25T11:42:30.757933+0000","last_fullsized":"2024-04-25T11:42:30.757933+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:33.682475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759546+0000","last_change":"2024-04-25T11:42:28.657338+0000","last_active":"2024-04-25T11:42:30.759546+0000","last_peered":"2024-04-25T11:42:30.759546+0000","last_clean":"2024-04-25T11:42:30.759546+0000","last_became_active":"2024-04-25T11:42:28.656483+0000","last_became_peered":"2024-04-25T11:42:28.656483+0000","last_unstale":"2024-04-25T11:42:30.759546+0000","last_undegraded":"2024-04-25T11:42:30.759546+0000","last_fullsized":"2024-04-25T11:42:30.759546+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:17.656405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683016+0000","last_change":"2024-04-25T11:42:29.696298+0000","last_active":"2024-04-25T11:42:30.683016+0000","last_peered":"2024-04-25T11:42:30.683016+0000","last_clean":"2024-04-25T11:42:30.683016+0000","last_became_active":"2024-04-25T11:42:29.695706+0000","last_became_peered":"2024-04-25T11:42:29.695706+0000","last_unstale":"2024-04-25T11:42:30.683016+0000","last_undegraded":"2024-04-25T11:42:30.683016+0000","last_fullsized":"2024-04-25T11:42:30.683016+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:45.417509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"142'3","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684359+0000","last_change":"2024-04-25T11:42:28.651772+0000","last_active":"2024-04-25T11:42:30.684359+0000","last_peered":"2024-04-25T11:42:30.684359+0000","last_clean":"2024-04-25T11:42:30.684359+0000","last_became_active":"2024-04-25T11:42:28.651416+0000","last_became_peered":"2024-04-25T11:42:28.651416+0000","last_unstale":"2024-04-25T11:42:30.684359+0000","last_undegraded":"2024-04-25T11:42:30.684359+0000","last_fullsized":"2024-04-25T11:42:30.684359+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:45.898449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756441+0000","last_change":"2024-04-25T11:42:29.709598+0000","last_active":"2024-04-25T11:42:30.756441+0000","last_peered":"2024-04-25T11:42:30.756441+0000","last_clean":"2024-04-25T11:42:30.756441+0000","last_became_active":"2024-04-25T11:42:29.707835+0000","last_became_peered":"2024-04-25T11:42:29.707835+0000","last_unstale":"2024-04-25T11:42:30.756441+0000","last_undegraded":"2024-04-25T11:42:30.756441+0000","last_fullsized":"2024-04-25T11:42:30.756441+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:44.904701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"142'6","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758374+0000","last_change":"2024-04-25T11:42:28.653989+0000","last_active":"2024-04-25T11:42:30.758374+0000","last_peered":"2024-04-25T11:42:30.758374+0000","last_clean":"2024-04-25T11:42:30.758374+0000","last_became_active":"2024-04-25T11:42:28.653670+0000","last_became_peered":"2024-04-25T11:42:28.653670+0000","last_unstale":"2024-04-25T11:42:30.758374+0000","last_undegraded":"2024-04-25T11:42:30.758374+0000","last_fullsized":"2024-04-25T11:42:30.758374+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758247+0000","last_change":"2024-04-25T11:42:29.696427+0000","last_active":"2024-04-25T11:42:30.758247+0000","last_peered":"2024-04-25T11:42:30.758247+0000","last_clean":"2024-04-25T11:42:30.758247+0000","last_became_active":"2024-04-25T11:42:29.696069+0000","last_became_peered":"2024-04-25T11:42:29.696069+0000","last_unstale":"2024-04-25T11:42:30.758247+0000","last_undegraded":"2024-04-25T11:42:30.758247+0000","last_fullsized":"2024-04-25T11:42:30.758247+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.398770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703038+0000","last_change":"2024-04-25T11:42:28.650997+0000","last_active":"2024-04-25T11:42:30.703038+0000","last_peered":"2024-04-25T11:42:30.703038+0000","last_clean":"2024-04-25T11:42:30.703038+0000","last_became_active":"2024-04-25T11:42:28.650647+0000","last_became_peered":"2024-04-25T11:42:28.650647+0000","last_unstale":"2024-04-25T11:42:30.703038+0000","last_undegraded":"2024-04-25T11:42:30.703038+0000","last_fullsized":"2024-04-25T11:42:30.703038+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:46.078312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686348+0000","last_change":"2024-04-25T11:42:29.696398+0000","last_active":"2024-04-25T11:42:30.686348+0000","last_peered":"2024-04-25T11:42:30.686348+0000","last_clean":"2024-04-25T11:42:30.686348+0000","last_became_active":"2024-04-25T11:42:29.696046+0000","last_became_peered":"2024-04-25T11:42:29.696046+0000","last_unstale":"2024-04-25T11:42:30.686348+0000","last_undegraded":"2024-04-25T11:42:30.686348+0000","last_fullsized":"2024-04-25T11:42:30.686348+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:55.724030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758229+0000","last_change":"2024-04-25T11:42:28.660110+0000","last_active":"2024-04-25T11:42:30.758229+0000","last_peered":"2024-04-25T11:42:30.758229+0000","last_clean":"2024-04-25T11:42:30.758229+0000","last_became_active":"2024-04-25T11:42:28.659719+0000","last_became_peered":"2024-04-25T11:42:28.659719+0000","last_unstale":"2024-04-25T11:42:30.758229+0000","last_undegraded":"2024-04-25T11:42:30.758229+0000","last_fullsized":"2024-04-25T11:42:30.758229+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:02.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683236+0000","last_change":"2024-04-25T11:42:29.691401+0000","last_active":"2024-04-25T11:42:30.683236+0000","last_peered":"2024-04-25T11:42:30.683236+0000","last_clean":"2024-04-25T11:42:30.683236+0000","last_became_active":"2024-04-25T11:42:29.691038+0000","last_became_peered":"2024-04-25T11:42:29.691038+0000","last_unstale":"2024-04-25T11:42:30.683236+0000","last_undegraded":"2024-04-25T11:42:30.683236+0000","last_fullsized":"2024-04-25T11:42:30.683236+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:24.315576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757332+0000","last_change":"2024-04-25T11:42:28.643885+0000","last_active":"2024-04-25T11:42:30.757332+0000","last_peered":"2024-04-25T11:42:30.757332+0000","last_clean":"2024-04-25T11:42:30.757332+0000","last_became_active":"2024-04-25T11:42:28.641975+0000","last_became_peered":"2024-04-25T11:42:28.641975+0000","last_unstale":"2024-04-25T11:42:30.757332+0000","last_undegraded":"2024-04-25T11:42:30.757332+0000","last_fullsized":"2024-04-25T11:42:30.757332+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:02.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684521+0000","last_change":"2024-04-25T11:42:29.689006+0000","last_active":"2024-04-25T11:42:30.684521+0000","last_peered":"2024-04-25T11:42:30.684521+0000","last_clean":"2024-04-25T11:42:30.684521+0000","last_became_active":"2024-04-25T11:42:29.688662+0000","last_became_peered":"2024-04-25T11:42:29.688662+0000","last_unstale":"2024-04-25T11:42:30.684521+0000","last_undegraded":"2024-04-25T11:42:30.684521+0000","last_fullsized":"2024-04-25T11:42:30.684521+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.242694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"142'8","reported_seq":22,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758630+0000","last_change":"2024-04-25T11:42:28.650070+0000","last_active":"2024-04-25T11:42:30.758630+0000","last_peered":"2024-04-25T11:42:30.758630+0000","last_clean":"2024-04-25T11:42:30.758630+0000","last_became_active":"2024-04-25T11:42:28.649743+0000","last_became_peered":"2024-04-25T11:42:28.649743+0000","last_unstale":"2024-04-25T11:42:30.758630+0000","last_undegraded":"2024-04-25T11:42:30.758630+0000","last_fullsized":"2024-04-25T11:42:30.758630+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:58.953708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758260+0000","last_change":"2024-04-25T11:42:29.693003+0000","last_active":"2024-04-25T11:42:30.758260+0000","last_peered":"2024-04-25T11:42:30.758260+0000","last_clean":"2024-04-25T11:42:30.758260+0000","last_became_active":"2024-04-25T11:42:29.692646+0000","last_became_peered":"2024-04-25T11:42:29.692646+0000","last_unstale":"2024-04-25T11:42:30.758260+0000","last_undegraded":"2024-04-25T11:42:30.758260+0000","last_fullsized":"2024-04-25T11:42:30.758260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:56.224292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758938+0000","last_change":"2024-04-25T11:42:28.652398+0000","last_active":"2024-04-25T11:42:30.758938+0000","last_peered":"2024-04-25T11:42:30.758938+0000","last_clean":"2024-04-25T11:42:30.758938+0000","last_became_active":"2024-04-25T11:42:28.652057+0000","last_became_peered":"2024-04-25T11:42:28.652057+0000","last_unstale":"2024-04-25T11:42:30.758938+0000","last_undegraded":"2024-04-25T11:42:30.758938+0000","last_fullsized":"2024-04-25T11:42:30.758938+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:04.583102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757338+0000","last_change":"2024-04-25T11:42:29.691459+0000","last_active":"2024-04-25T11:42:30.757338+0000","last_peered":"2024-04-25T11:42:30.757338+0000","last_clean":"2024-04-25T11:42:30.757338+0000","last_became_active":"2024-04-25T11:42:29.691120+0000","last_became_peered":"2024-04-25T11:42:29.691120+0000","last_unstale":"2024-04-25T11:42:30.757338+0000","last_undegraded":"2024-04-25T11:42:30.757338+0000","last_fullsized":"2024-04-25T11:42:30.757338+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.727855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684383+0000","last_change":"2024-04-25T11:42:28.649062+0000","last_active":"2024-04-25T11:42:30.684383+0000","last_peered":"2024-04-25T11:42:30.684383+0000","last_clean":"2024-04-25T11:42:30.684383+0000","last_became_active":"2024-04-25T11:42:28.648536+0000","last_became_peered":"2024-04-25T11:42:28.648536+0000","last_unstale":"2024-04-25T11:42:30.684383+0000","last_undegraded":"2024-04-25T11:42:30.684383+0000","last_fullsized":"2024-04-25T11:42:30.684383+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:49.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758859+0000","last_change":"2024-04-25T11:42:29.685542+0000","last_active":"2024-04-25T11:42:30.758859+0000","last_peered":"2024-04-25T11:42:30.758859+0000","last_clean":"2024-04-25T11:42:30.758859+0000","last_became_active":"2024-04-25T11:42:29.685205+0000","last_became_peered":"2024-04-25T11:42:29.685205+0000","last_unstale":"2024-04-25T11:42:30.758859+0000","last_undegraded":"2024-04-25T11:42:30.758859+0000","last_fullsized":"2024-04-25T11:42:30.758859+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.242694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703199+0000","last_change":"2024-04-25T11:42:28.651314+0000","last_active":"2024-04-25T11:42:30.703199+0000","last_peered":"2024-04-25T11:42:30.703199+0000","last_clean":"2024-04-25T11:42:30.703199+0000","last_became_active":"2024-04-25T11:42:28.650945+0000","last_became_peered":"2024-04-25T11:42:28.650945+0000","last_unstale":"2024-04-25T11:42:30.703199+0000","last_undegraded":"2024-04-25T11:42:30.703199+0000","last_fullsized":"2024-04-25T11:42:30.703199+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:08.845636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703128+0000","last_change":"2024-04-25T11:42:29.698412+0000","last_active":"2024-04-25T11:42:30.703128+0000","last_peered":"2024-04-25T11:42:30.703128+0000","last_clean":"2024-04-25T11:42:30.703128+0000","last_became_active":"2024-04-25T11:42:29.697766+0000","last_became_peered":"2024-04-25T11:42:29.697766+0000","last_unstale":"2024-04-25T11:42:30.703128+0000","last_undegraded":"2024-04-25T11:42:30.703128+0000","last_fullsized":"2024-04-25T11:42:30.703128+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:58.467574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758157+0000","last_change":"2024-04-25T11:42:28.656051+0000","last_active":"2024-04-25T11:42:30.758157+0000","last_peered":"2024-04-25T11:42:30.758157+0000","last_clean":"2024-04-25T11:42:30.758157+0000","last_became_active":"2024-04-25T11:42:28.655708+0000","last_became_peered":"2024-04-25T11:42:28.655708+0000","last_unstale":"2024-04-25T11:42:30.758157+0000","last_undegraded":"2024-04-25T11:42:30.758157+0000","last_fullsized":"2024-04-25T11:42:30.758157+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:34.725563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.702813+0000","last_change":"2024-04-25T11:42:29.691606+0000","last_active":"2024-04-25T11:42:30.702813+0000","last_peered":"2024-04-25T11:42:30.702813+0000","last_clean":"2024-04-25T11:42:30.702813+0000","last_became_active":"2024-04-25T11:42:29.689254+0000","last_became_peered":"2024-04-25T11:42:29.689254+0000","last_unstale":"2024-04-25T11:42:30.702813+0000","last_undegraded":"2024-04-25T11:42:30.702813+0000","last_fullsized":"2024-04-25T11:42:30.702813+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:42.693174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756360+0000","last_change":"2024-04-25T11:42:28.652991+0000","last_active":"2024-04-25T11:42:30.756360+0000","last_peered":"2024-04-25T11:42:30.756360+0000","last_clean":"2024-04-25T11:42:30.756360+0000","last_became_active":"2024-04-25T11:42:28.652686+0000","last_became_peered":"2024-04-25T11:42:28.652686+0000","last_unstale":"2024-04-25T11:42:30.756360+0000","last_undegraded":"2024-04-25T11:42:30.756360+0000","last_fullsized":"2024-04-25T11:42:30.756360+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:49.249284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759116+0000","last_change":"2024-04-25T11:42:29.687841+0000","last_active":"2024-04-25T11:42:30.759116+0000","last_peered":"2024-04-25T11:42:30.759116+0000","last_clean":"2024-04-25T11:42:30.759116+0000","last_became_active":"2024-04-25T11:42:29.687453+0000","last_became_peered":"2024-04-25T11:42:29.687453+0000","last_unstale":"2024-04-25T11:42:30.759116+0000","last_undegraded":"2024-04-25T11:42:30.759116+0000","last_fullsized":"2024-04-25T11:42:30.759116+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.013260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757154+0000","last_change":"2024-04-25T11:42:28.658831+0000","last_active":"2024-04-25T11:42:30.757154+0000","last_peered":"2024-04-25T11:42:30.757154+0000","last_clean":"2024-04-25T11:42:30.757154+0000","last_became_active":"2024-04-25T11:42:28.658360+0000","last_became_peered":"2024-04-25T11:42:28.658360+0000","last_unstale":"2024-04-25T11:42:30.757154+0000","last_undegraded":"2024-04-25T11:42:30.757154+0000","last_fullsized":"2024-04-25T11:42:30.757154+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:57.662132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756887+0000","last_change":"2024-04-25T11:42:29.687804+0000","last_active":"2024-04-25T11:42:30.756887+0000","last_peered":"2024-04-25T11:42:30.756887+0000","last_clean":"2024-04-25T11:42:30.756887+0000","last_became_active":"2024-04-25T11:42:29.687410+0000","last_became_peered":"2024-04-25T11:42:29.687410+0000","last_unstale":"2024-04-25T11:42:30.756887+0000","last_undegraded":"2024-04-25T11:42:30.756887+0000","last_fullsized":"2024-04-25T11:42:30.756887+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:54.878227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684026+0000","last_change":"2024-04-25T11:42:28.660077+0000","last_active":"2024-04-25T11:42:30.684026+0000","last_peered":"2024-04-25T11:42:30.684026+0000","last_clean":"2024-04-25T11:42:30.684026+0000","last_became_active":"2024-04-25T11:42:28.659663+0000","last_became_peered":"2024-04-25T11:42:28.659663+0000","last_unstale":"2024-04-25T11:42:30.684026+0000","last_undegraded":"2024-04-25T11:42:30.684026+0000","last_fullsized":"2024-04-25T11:42:30.684026+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:04.583102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703680+0000","last_change":"2024-04-25T11:42:29.701838+0000","last_active":"2024-04-25T11:42:30.703680+0000","last_peered":"2024-04-25T11:42:30.703680+0000","last_clean":"2024-04-25T11:42:30.703680+0000","last_became_active":"2024-04-25T11:42:29.699830+0000","last_became_peered":"2024-04-25T11:42:29.699830+0000","last_unstale":"2024-04-25T11:42:30.703680+0000","last_undegraded":"2024-04-25T11:42:30.703680+0000","last_fullsized":"2024-04-25T11:42:30.703680+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:39.051407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759199+0000","last_change":"2024-04-25T11:42:28.647452+0000","last_active":"2024-04-25T11:42:30.759199+0000","last_peered":"2024-04-25T11:42:30.759199+0000","last_clean":"2024-04-25T11:42:30.759199+0000","last_became_active":"2024-04-25T11:42:28.647120+0000","last_became_peered":"2024-04-25T11:42:28.647120+0000","last_unstale":"2024-04-25T11:42:30.759199+0000","last_undegraded":"2024-04-25T11:42:30.759199+0000","last_fullsized":"2024-04-25T11:42:30.759199+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:30.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703484+0000","last_change":"2024-04-25T11:42:29.707285+0000","last_active":"2024-04-25T11:42:30.703484+0000","last_peered":"2024-04-25T11:42:30.703484+0000","last_clean":"2024-04-25T11:42:30.703484+0000","last_became_active":"2024-04-25T11:42:29.704368+0000","last_became_peered":"2024-04-25T11:42:29.704368+0000","last_unstale":"2024-04-25T11:42:30.703484+0000","last_undegraded":"2024-04-25T11:42:30.703484+0000","last_fullsized":"2024-04-25T11:42:30.703484+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:49.780342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"143'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.705855+0000","last_change":"2024-04-25T11:42:28.654964+0000","last_active":"2024-04-25T11:42:30.705855+0000","last_peered":"2024-04-25T11:42:30.705855+0000","last_clean":"2024-04-25T11:42:30.705855+0000","last_became_active":"2024-04-25T11:42:28.654473+0000","last_became_peered":"2024-04-25T11:42:28.654473+0000","last_unstale":"2024-04-25T11:42:30.705855+0000","last_undegraded":"2024-04-25T11:42:30.705855+0000","last_fullsized":"2024-04-25T11:42:30.705855+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:07.926295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686637+0000","last_change":"2024-04-25T11:42:29.693022+0000","last_active":"2024-04-25T11:42:30.686637+0000","last_peered":"2024-04-25T11:42:30.686637+0000","last_clean":"2024-04-25T11:42:30.686637+0000","last_became_active":"2024-04-25T11:42:29.692690+0000","last_became_peered":"2024-04-25T11:42:29.692690+0000","last_unstale":"2024-04-25T11:42:30.686637+0000","last_undegraded":"2024-04-25T11:42:30.686637+0000","last_fullsized":"2024-04-25T11:42:30.686637+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:45.369012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703656+0000","last_change":"2024-04-25T11:42:28.652813+0000","last_active":"2024-04-25T11:42:30.703656+0000","last_peered":"2024-04-25T11:42:30.703656+0000","last_clean":"2024-04-25T11:42:30.703656+0000","last_became_active":"2024-04-25T11:42:28.652487+0000","last_became_peered":"2024-04-25T11:42:28.652487+0000","last_unstale":"2024-04-25T11:42:30.703656+0000","last_undegraded":"2024-04-25T11:42:30.703656+0000","last_fullsized":"2024-04-25T11:42:30.703656+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:33.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684212+0000","last_change":"2024-04-25T11:42:29.696528+0000","last_active":"2024-04-25T11:42:30.684212+0000","last_peered":"2024-04-25T11:42:30.684212+0000","last_clean":"2024-04-25T11:42:30.684212+0000","last_became_active":"2024-04-25T11:42:29.696140+0000","last_became_peered":"2024-04-25T11:42:29.696140+0000","last_unstale":"2024-04-25T11:42:30.684212+0000","last_undegraded":"2024-04-25T11:42:30.684212+0000","last_fullsized":"2024-04-25T11:42:30.684212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.580891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757421+0000","last_change":"2024-04-25T11:42:28.649972+0000","last_active":"2024-04-25T11:42:30.757421+0000","last_peered":"2024-04-25T11:42:30.757421+0000","last_clean":"2024-04-25T11:42:30.757421+0000","last_became_active":"2024-04-25T11:42:28.649540+0000","last_became_peered":"2024-04-25T11:42:28.649540+0000","last_unstale":"2024-04-25T11:42:30.757421+0000","last_undegraded":"2024-04-25T11:42:30.757421+0000","last_fullsized":"2024-04-25T11:42:30.757421+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:59.326194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703355+0000","last_change":"2024-04-25T11:42:29.698119+0000","last_active":"2024-04-25T11:42:30.703355+0000","last_peered":"2024-04-25T11:42:30.703355+0000","last_clean":"2024-04-25T11:42:30.703355+0000","last_became_active":"2024-04-25T11:42:29.697363+0000","last_became_peered":"2024-04-25T11:42:29.697363+0000","last_unstale":"2024-04-25T11:42:30.703355+0000","last_undegraded":"2024-04-25T11:42:30.703355+0000","last_fullsized":"2024-04-25T11:42:30.703355+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:46.474813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686718+0000","last_change":"2024-04-25T11:42:28.663519+0000","last_active":"2024-04-25T11:42:30.686718+0000","last_peered":"2024-04-25T11:42:30.686718+0000","last_clean":"2024-04-25T11:42:30.686718+0000","last_became_active":"2024-04-25T11:42:28.663125+0000","last_became_peered":"2024-04-25T11:42:28.663125+0000","last_unstale":"2024-04-25T11:42:30.686718+0000","last_undegraded":"2024-04-25T11:42:30.686718+0000","last_fullsized":"2024-04-25T11:42:30.686718+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:30.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683443+0000","last_change":"2024-04-25T11:42:29.685183+0000","last_active":"2024-04-25T11:42:30.683443+0000","last_peered":"2024-04-25T11:42:30.683443+0000","last_clean":"2024-04-25T11:42:30.683443+0000","last_became_active":"2024-04-25T11:42:29.684823+0000","last_became_peered":"2024-04-25T11:42:29.684823+0000","last_unstale":"2024-04-25T11:42:30.683443+0000","last_undegraded":"2024-04-25T11:42:30.683443+0000","last_fullsized":"2024-04-25T11:42:30.683443+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:00.596387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756550+0000","last_change":"2024-04-25T11:42:28.653937+0000","last_active":"2024-04-25T11:42:30.756550+0000","last_peered":"2024-04-25T11:42:30.756550+0000","last_clean":"2024-04-25T11:42:30.756550+0000","last_became_active":"2024-04-25T11:42:28.653047+0000","last_became_peered":"2024-04-25T11:42:28.653047+0000","last_unstale":"2024-04-25T11:42:30.756550+0000","last_undegraded":"2024-04-25T11:42:30.756550+0000","last_fullsized":"2024-04-25T11:42:30.756550+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:44.352672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.702743+0000","last_change":"2024-04-25T11:42:29.697900+0000","last_active":"2024-04-25T11:42:30.702743+0000","last_peered":"2024-04-25T11:42:30.702743+0000","last_clean":"2024-04-25T11:42:30.702743+0000","last_became_active":"2024-04-25T11:42:29.695627+0000","last_became_peered":"2024-04-25T11:42:29.695627+0000","last_unstale":"2024-04-25T11:42:30.702743+0000","last_undegraded":"2024-04-25T11:42:30.702743+0000","last_fullsized":"2024-04-25T11:42:30.702743+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:22.977989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.687203+0000","last_change":"2024-04-25T11:42:28.666769+0000","last_active":"2024-04-25T11:42:30.687203+0000","last_peered":"2024-04-25T11:42:30.687203+0000","last_clean":"2024-04-25T11:42:30.687203+0000","last_became_active":"2024-04-25T11:42:28.666363+0000","last_became_peered":"2024-04-25T11:42:28.666363+0000","last_unstale":"2024-04-25T11:42:30.687203+0000","last_undegraded":"2024-04-25T11:42:30.687203+0000","last_fullsized":"2024-04-25T11:42:30.687203+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:35.258782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757211+0000","last_change":"2024-04-25T11:42:29.701372+0000","last_active":"2024-04-25T11:42:30.757211+0000","last_peered":"2024-04-25T11:42:30.757211+0000","last_clean":"2024-04-25T11:42:30.757211+0000","last_became_active":"2024-04-25T11:42:29.700993+0000","last_became_peered":"2024-04-25T11:42:29.700993+0000","last_unstale":"2024-04-25T11:42:30.757211+0000","last_undegraded":"2024-04-25T11:42:30.757211+0000","last_fullsized":"2024-04-25T11:42:30.757211+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:44.507891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684259+0000","last_change":"2024-04-25T11:42:28.659425+0000","last_active":"2024-04-25T11:42:30.684259+0000","last_peered":"2024-04-25T11:42:30.684259+0000","last_clean":"2024-04-25T11:42:30.684259+0000","last_became_active":"2024-04-25T11:42:28.658995+0000","last_became_peered":"2024-04-25T11:42:28.658995+0000","last_unstale":"2024-04-25T11:42:30.684259+0000","last_undegraded":"2024-04-25T11:42:30.684259+0000","last_fullsized":"2024-04-25T11:42:30.684259+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:02.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685458+0000","last_change":"2024-04-25T11:42:29.689800+0000","last_active":"2024-04-25T11:42:30.685458+0000","last_peered":"2024-04-25T11:42:30.685458+0000","last_clean":"2024-04-25T11:42:30.685458+0000","last_became_active":"2024-04-25T11:42:29.689290+0000","last_became_peered":"2024-04-25T11:42:29.689290+0000","last_unstale":"2024-04-25T11:42:30.685458+0000","last_undegraded":"2024-04-25T11:42:30.685458+0000","last_fullsized":"2024-04-25T11:42:30.685458+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.926190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759209+0000","last_change":"2024-04-25T11:42:28.651803+0000","last_active":"2024-04-25T11:42:30.759209+0000","last_peered":"2024-04-25T11:42:30.759209+0000","last_clean":"2024-04-25T11:42:30.759209+0000","last_became_active":"2024-04-25T11:42:28.650621+0000","last_became_peered":"2024-04-25T11:42:28.650621+0000","last_unstale":"2024-04-25T11:42:30.759209+0000","last_undegraded":"2024-04-25T11:42:30.759209+0000","last_fullsized":"2024-04-25T11:42:30.759209+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:41.866541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":350,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756217+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:42:30.756217+0000","last_peered":"2024-04-25T11:42:30.756217+0000","last_clean":"2024-04-25T11:42:30.756217+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:42:30.756217+0000","last_undegraded":"2024-04-25T11:42:30.756217+0000","last_fullsized":"2024-04-25T11:42:30.756217+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509626,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37132,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334708,"statfs":{"total":96636764160,"available":96598740992,"internally_reserved":0,"allocated":1830912,"data_stored":880729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509627,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41288,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330552,"statfs":{"total":96636764160,"available":96594485248,"internally_reserved":0,"allocated":1826816,"data_stored":880147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509627,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1830912,"data_stored":880729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509628,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41300,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330540,"statfs":{"total":96636764160,"available":96594472960,"internally_reserved":0,"allocated":1839104,"data_stored":880341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509628,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1830912,"data_stored":880217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509628,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37728,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334112,"statfs":{"total":96636764160,"available":96598130688,"internally_reserved":0,"allocated":2441216,"data_stored":1472239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509627,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1826816,"data_stored":880147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542330,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37728,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334112,"statfs":{"total":96636764160,"available":96598130688,"internally_reserved":0,"allocated":2441216,"data_stored":1472183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:42:34.431 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:42:34.569 INFO:tasks.ceph.mds.b.smithi169.stderr:2024-04-25T11:42:34.583+0000 7fb78a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:42:34.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.772+0000 7f714b200640 1 Processor -- start 2024-04-25T11:42:34.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.772+0000 7f714b200640 1 -- start start 2024-04-25T11:42:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.774+0000 7f714b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 0x7f714c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.774+0000 7f714b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f714c144370 0x7f714c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f714b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714c0b50e0 con 0x7f714c0b4cd0 2024-04-25T11:42:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f714b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f714c0d1230 con 0x7f714c144370 2024-04-25T11:42:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f714b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f714c0d06b0 con 0x7f714c06c5c0 2024-04-25T11:42:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f714a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f714c0b4cd0 0x7f714c161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50818/0 (socket says 172.21.15.19:50818) 2024-04-25T11:42:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f714a800640 1 -- 172.21.15.19:0/1328119910 learned_addr learned my addr 172.21.15.19:0/1328119910 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:34.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f7149400640 1 --2- 172.21.15.19:0/1328119910 >> v2:172.21.15.169:3300/0 conn(0x7f714c144370 0x7f714c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081951904 0 0) 0x7f7144001490 con 0x7f714c0b4cd0 2024-04-25T11:42:34.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7130003520 con 0x7f714c0b4cd0 2024-04-25T11:42:34.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.775+0000 7f7149e00640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 0x7f714c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 581656973 0 0) 0x7f7130003520 con 0x7f714c0b4cd0 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 msgr2=0x7f714c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 0x7f714c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 >> v2:172.21.15.169:3300/0 conn(0x7f714c144370 msgr2=0x7f714c0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 --2- 172.21.15.19:0/1328119910 >> v2:172.21.15.169:3300/0 conn(0x7f714c144370 0x7f714c0b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c0d0990 con 0x7f714c0b4cd0 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7144001bc0 con 0x7f714c0b4cd0 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 >> v1:172.21.15.19:6789/0 conn(0x7f714c0b4cd0 legacy=0x7f714c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 0x7f71300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f7130004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7130004d40 0x7f7130005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0d1230 con 0x7f7130004030 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0d06b0 con 0x7f7130004d40 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f713000bbd0 con 0x7f714c06c5c0 2024-04-25T11:42:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.776+0000 7f7149400640 1 --2- 172.21.15.19:0/1328119910 >> v2:172.21.15.169:3300/0 conn(0x7f714c144370 0x7f714c0b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f7149400640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f7130004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f714a800640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7130004d40 0x7f7130005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f7149e00640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 0x7f71300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f7149400640 1 -- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 msgr2=0x7f71300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f7149400640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c06c5c0 0x7f71300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f7149400640 1 -- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7130004d40 msgr2=0x7f7130005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f7149400640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7130004d40 0x7f7130005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.777+0000 7f7149400640 1 -- 172.21.15.19:0/1328119910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7130003520 con 0x7f7130004030 2024-04-25T11:42:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.778+0000 7f7149400640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f7130004800 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f714c147c60 tx=0x7f7130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4240b51ec36440df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.778+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138037070 con 0x7f7130004030 2024-04-25T11:42:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.778+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713802ad00 con 0x7f7130004030 2024-04-25T11:42:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.778+0000 7f7148a00640 1 -- 172.21.15.19:0/1328119910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713800f040 con 0x7f7130004030 2024-04-25T11:42:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.779+0000 7f714b200640 1 -- 172.21.15.19:0/1328119910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 msgr2=0x7f7130004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.779+0000 7f714b200640 1 --2- 172.21.15.19:0/1328119910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f7130004800 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f714c147c60 tx=0x7f7130005830 comp rx=0 tx=0).stop 2024-04-25T11:42:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.779+0000 7f714a800640 1 -- 172.21.15.19:0/1328119910 reap_dead start 2024-04-25T11:42:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.779+0000 7f714b200640 1 -- 172.21.15.19:0/1328119910 shutdown_connections 2024-04-25T11:42:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.779+0000 7f714b200640 1 -- 172.21.15.19:0/1328119910 >> 172.21.15.19:0/1328119910 conn(0x7f714c0d1720 msgr2=0x7f714c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.779+0000 7f714b200640 1 -- 172.21.15.19:0/1328119910 shutdown_connections 2024-04-25T11:42:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.779+0000 7f714b200640 1 -- 172.21.15.19:0/1328119910 wait complete. 2024-04-25T11:42:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.782+0000 7f714b200640 1 Processor -- start 2024-04-25T11:42:34.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.782+0000 7f714b200640 1 -- start start 2024-04-25T11:42:34.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.784+0000 7f714b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f714c1f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.784+0000 7f714b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c0c7510 0x7f714c160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.784+0000 7f714b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f714c160ea0 0x7f714c1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.784+0000 7f714b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0d02d0 con 0x7f7130004030 2024-04-25T11:42:34.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.784+0000 7f714b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0d0f50 con 0x7f714c160ea0 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.784+0000 7f714b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f714c0d0450 con 0x7f714c0c7510 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.784+0000 7f7149e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f714c1f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f714c1f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54930/0 (socket says 172.21.15.19:54930) 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 -- 172.21.15.19:0/520491127 learned_addr learned my addr 172.21.15.19:0/520491127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149400640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c0c7510 0x7f714c160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f714a800640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f714c160ea0 0x7f714c1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 -- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c0c7510 msgr2=0x7f714c160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c0c7510 0x7f714c160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 -- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f714c160ea0 msgr2=0x7f714c1f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f714c160ea0 0x7f714c1f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 -- 172.21.15.19:0/520491127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c1612b0 con 0x7f7130004030 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149400640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c0c7510 0x7f714c160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.785+0000 7f7149e00640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f714c1f3b30 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f713c00f5b0 tx=0x7f713c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.786+0000 7f7142000640 1 -- 172.21.15.19:0/520491127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c019020 con 0x7f7130004030 2024-04-25T11:42:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.786+0000 7f7142000640 1 -- 172.21.15.19:0/520491127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713c014870 con 0x7f7130004030 2024-04-25T11:42:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.786+0000 7f7142000640 1 -- 172.21.15.19:0/520491127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c014c60 con 0x7f7130004030 2024-04-25T11:42:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.786+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c1e8760 con 0x7f7130004030 2024-04-25T11:42:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.786+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1e81a0 con 0x7f7130004030 2024-04-25T11:42:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.787+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c15d0b0 con 0x7f7130004030 2024-04-25T11:42:34.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.789+0000 7f7142000640 1 -- 172.21.15.19:0/520491127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f713c01c020 con 0x7f7130004030 2024-04-25T11:42:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.791+0000 7f7142000640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f712c0744e0 0x7f712c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.791+0000 7f7142000640 1 -- 172.21.15.19:0/520491127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f713c095a10 con 0x7f7130004030 2024-04-25T11:42:34.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.791+0000 7f7149400640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f712c0744e0 0x7f712c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:34.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.793+0000 7f7149400640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f712c0744e0 0x7f712c0769f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f714c147c60 tx=0x7f713802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.796+0000 7f7142000640 1 -- 172.21.15.19:0/520491127 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f713c023020 con 0x7f7130004030 2024-04-25T11:42:34.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:34.996+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f714c15de90 con 0x7f712c0744e0 2024-04-25T11:42:35.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.017+0000 7f7142000640 1 -- 172.21.15.19:0/520491127 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252471 (secure 0 0 0) 0x7f714c15de90 con 0x7f712c0744e0 2024-04-25T11:42:35.008 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:35.009 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:42:35.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f712c0744e0 msgr2=0x7f712c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f712c0744e0 0x7f712c0769f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f714c147c60 tx=0x7f713802e000 comp rx=0 tx=0).stop 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 msgr2=0x7f714c1f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f714c1f3b30 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f713c00f5b0 tx=0x7f713c00fa70 comp rx=0 tx=0).stop 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 shutdown_connections 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f712c0744e0 0x7f712c0769f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f714c160ea0 0x7f714c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f714c0c7510 0x7f714c160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 --2- 172.21.15.19:0/520491127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7130004030 0x7f714c1f3b30 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 >> 172.21.15.19:0/520491127 conn(0x7f714c0d1720 msgr2=0x7f714c145e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 shutdown_connections 2024-04-25T11:42:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.022+0000 7f714b200640 1 -- 172.21.15.19:0/520491127 wait complete. 2024-04-25T11:42:35.036 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2024-04-25T11:42:34.832384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":298248,"kb_used_data":15496,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754676472,"statfs":{"total":773094113280,"available":772788707328,"internally_reserved":0,"allocated":15867904,"data_stored":8226732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":289394226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.212764"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759567+0000","last_change":"2024-04-25T11:42:29.696855+0000","last_active":"2024-04-25T11:42:30.759567+0000","last_peered":"2024-04-25T11:42:30.759567+0000","last_clean":"2024-04-25T11:42:30.759567+0000","last_became_active":"2024-04-25T11:42:29.696496+0000","last_became_peered":"2024-04-25T11:42:29.696496+0000","last_unstale":"2024-04-25T11:42:30.759567+0000","last_undegraded":"2024-04-25T11:42:30.759567+0000","last_fullsized":"2024-04-25T11:42:30.759567+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:45.369012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759395+0000","last_change":"2024-04-25T11:42:29.693167+0000","last_active":"2024-04-25T11:42:30.759395+0000","last_peered":"2024-04-25T11:42:30.759395+0000","last_clean":"2024-04-25T11:42:30.759395+0000","last_became_active":"2024-04-25T11:42:29.692804+0000","last_became_peered":"2024-04-25T11:42:29.692804+0000","last_unstale":"2024-04-25T11:42:30.759395+0000","last_undegraded":"2024-04-25T11:42:30.759395+0000","last_fullsized":"2024-04-25T11:42:30.759395+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:55.724030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686395+0000","last_change":"2024-04-25T11:42:29.692020+0000","last_active":"2024-04-25T11:42:30.686395+0000","last_peered":"2024-04-25T11:42:30.686395+0000","last_clean":"2024-04-25T11:42:30.686395+0000","last_became_active":"2024-04-25T11:42:29.691674+0000","last_became_peered":"2024-04-25T11:42:29.691674+0000","last_unstale":"2024-04-25T11:42:30.686395+0000","last_undegraded":"2024-04-25T11:42:30.686395+0000","last_fullsized":"2024-04-25T11:42:30.686395+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:50.265624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758557+0000","last_change":"2024-04-25T11:42:29.701146+0000","last_active":"2024-04-25T11:42:30.758557+0000","last_peered":"2024-04-25T11:42:30.758557+0000","last_clean":"2024-04-25T11:42:30.758557+0000","last_became_active":"2024-04-25T11:42:29.700667+0000","last_became_peered":"2024-04-25T11:42:29.700667+0000","last_unstale":"2024-04-25T11:42:30.758557+0000","last_undegraded":"2024-04-25T11:42:30.758557+0000","last_fullsized":"2024-04-25T11:42:30.758557+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:26.268376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683239+0000","last_change":"2024-04-25T11:42:29.674388+0000","last_active":"2024-04-25T11:42:30.683239+0000","last_peered":"2024-04-25T11:42:30.683239+0000","last_clean":"2024-04-25T11:42:30.683239+0000","last_became_active":"2024-04-25T11:42:29.673550+0000","last_became_peered":"2024-04-25T11:42:29.673550+0000","last_unstale":"2024-04-25T11:42:30.683239+0000","last_undegraded":"2024-04-25T11:42:30.683239+0000","last_fullsized":"2024-04-25T11:42:30.683239+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:31.102557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684308+0000","last_change":"2024-04-25T11:42:29.694222+0000","last_active":"2024-04-25T11:42:30.684308+0000","last_peered":"2024-04-25T11:42:30.684308+0000","last_clean":"2024-04-25T11:42:30.684308+0000","last_became_active":"2024-04-25T11:42:29.690316+0000","last_became_peered":"2024-04-25T11:42:29.690316+0000","last_unstale":"2024-04-25T11:42:30.684308+0000","last_undegraded":"2024-04-25T11:42:30.684308+0000","last_fullsized":"2024-04-25T11:42:30.684308+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.909590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683600+0000","last_change":"2024-04-25T11:42:29.698842+0000","last_active":"2024-04-25T11:42:30.683600+0000","last_peered":"2024-04-25T11:42:30.683600+0000","last_clean":"2024-04-25T11:42:30.683600+0000","last_became_active":"2024-04-25T11:42:29.697764+0000","last_became_peered":"2024-04-25T11:42:29.697764+0000","last_unstale":"2024-04-25T11:42:30.683600+0000","last_undegraded":"2024-04-25T11:42:30.683600+0000","last_fullsized":"2024-04-25T11:42:30.683600+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:02.461138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684097+0000","last_change":"2024-04-25T11:42:29.687970+0000","last_active":"2024-04-25T11:42:30.684097+0000","last_peered":"2024-04-25T11:42:30.684097+0000","last_clean":"2024-04-25T11:42:30.684097+0000","last_became_active":"2024-04-25T11:42:29.687251+0000","last_became_peered":"2024-04-25T11:42:29.687251+0000","last_unstale":"2024-04-25T11:42:30.684097+0000","last_undegraded":"2024-04-25T11:42:30.684097+0000","last_fullsized":"2024-04-25T11:42:30.684097+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.398770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683490+0000","last_change":"2024-04-25T11:42:29.682183+0000","last_active":"2024-04-25T11:42:30.683490+0000","last_peered":"2024-04-25T11:42:30.683490+0000","last_clean":"2024-04-25T11:42:30.683490+0000","last_became_active":"2024-04-25T11:42:29.681774+0000","last_became_peered":"2024-04-25T11:42:29.681774+0000","last_unstale":"2024-04-25T11:42:30.683490+0000","last_undegraded":"2024-04-25T11:42:30.683490+0000","last_fullsized":"2024-04-25T11:42:30.683490+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:40.612195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757989+0000","last_change":"2024-04-25T11:42:29.702312+0000","last_active":"2024-04-25T11:42:30.757989+0000","last_peered":"2024-04-25T11:42:30.757989+0000","last_clean":"2024-04-25T11:42:30.757989+0000","last_became_active":"2024-04-25T11:42:29.701810+0000","last_became_peered":"2024-04-25T11:42:29.701810+0000","last_unstale":"2024-04-25T11:42:30.757989+0000","last_undegraded":"2024-04-25T11:42:30.757989+0000","last_fullsized":"2024-04-25T11:42:30.757989+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:05.902696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683867+0000","last_change":"2024-04-25T11:42:29.690167+0000","last_active":"2024-04-25T11:42:30.683867+0000","last_peered":"2024-04-25T11:42:30.683867+0000","last_clean":"2024-04-25T11:42:30.683867+0000","last_became_active":"2024-04-25T11:42:29.689673+0000","last_became_peered":"2024-04-25T11:42:29.689673+0000","last_unstale":"2024-04-25T11:42:30.683867+0000","last_undegraded":"2024-04-25T11:42:30.683867+0000","last_fullsized":"2024-04-25T11:42:30.683867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:26.979184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758426+0000","last_change":"2024-04-25T11:42:29.701352+0000","last_active":"2024-04-25T11:42:30.758426+0000","last_peered":"2024-04-25T11:42:30.758426+0000","last_clean":"2024-04-25T11:42:30.758426+0000","last_became_active":"2024-04-25T11:42:29.701005+0000","last_became_peered":"2024-04-25T11:42:29.701005+0000","last_unstale":"2024-04-25T11:42:30.758426+0000","last_undegraded":"2024-04-25T11:42:30.758426+0000","last_fullsized":"2024-04-25T11:42:30.758426+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.398770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688452+0000","last_change":"2024-04-25T11:42:29.693763+0000","last_active":"2024-04-25T11:42:30.688452+0000","last_peered":"2024-04-25T11:42:30.688452+0000","last_clean":"2024-04-25T11:42:30.688452+0000","last_became_active":"2024-04-25T11:42:29.691555+0000","last_became_peered":"2024-04-25T11:42:29.691555+0000","last_unstale":"2024-04-25T11:42:30.688452+0000","last_undegraded":"2024-04-25T11:42:30.688452+0000","last_fullsized":"2024-04-25T11:42:30.688452+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:51.761630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758008+0000","last_change":"2024-04-25T11:42:29.706151+0000","last_active":"2024-04-25T11:42:30.758008+0000","last_peered":"2024-04-25T11:42:30.758008+0000","last_clean":"2024-04-25T11:42:30.758008+0000","last_became_active":"2024-04-25T11:42:29.705588+0000","last_became_peered":"2024-04-25T11:42:29.705588+0000","last_unstale":"2024-04-25T11:42:30.758008+0000","last_undegraded":"2024-04-25T11:42:30.758008+0000","last_fullsized":"2024-04-25T11:42:30.758008+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:48.013757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758331+0000","last_change":"2024-04-25T11:42:29.700899+0000","last_active":"2024-04-25T11:42:30.758331+0000","last_peered":"2024-04-25T11:42:30.758331+0000","last_clean":"2024-04-25T11:42:30.758331+0000","last_became_active":"2024-04-25T11:42:29.700467+0000","last_became_peered":"2024-04-25T11:42:29.700467+0000","last_unstale":"2024-04-25T11:42:30.758331+0000","last_undegraded":"2024-04-25T11:42:30.758331+0000","last_fullsized":"2024-04-25T11:42:30.758331+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.242694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.682939+0000","last_change":"2024-04-25T11:42:29.701331+0000","last_active":"2024-04-25T11:42:30.682939+0000","last_peered":"2024-04-25T11:42:30.682939+0000","last_clean":"2024-04-25T11:42:30.682939+0000","last_became_active":"2024-04-25T11:42:29.700988+0000","last_became_peered":"2024-04-25T11:42:29.700988+0000","last_unstale":"2024-04-25T11:42:30.682939+0000","last_undegraded":"2024-04-25T11:42:30.682939+0000","last_fullsized":"2024-04-25T11:42:30.682939+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:02.426781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703305+0000","last_change":"2024-04-25T11:42:29.709246+0000","last_active":"2024-04-25T11:42:30.703305+0000","last_peered":"2024-04-25T11:42:30.703305+0000","last_clean":"2024-04-25T11:42:30.703305+0000","last_became_active":"2024-04-25T11:42:29.707729+0000","last_became_peered":"2024-04-25T11:42:29.707729+0000","last_unstale":"2024-04-25T11:42:30.703305+0000","last_undegraded":"2024-04-25T11:42:30.703305+0000","last_fullsized":"2024-04-25T11:42:30.703305+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:26.373977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759443+0000","last_change":"2024-04-25T11:42:29.689033+0000","last_active":"2024-04-25T11:42:30.759443+0000","last_peered":"2024-04-25T11:42:30.759443+0000","last_clean":"2024-04-25T11:42:30.759443+0000","last_became_active":"2024-04-25T11:42:29.687836+0000","last_became_peered":"2024-04-25T11:42:29.687836+0000","last_unstale":"2024-04-25T11:42:30.759443+0000","last_undegraded":"2024-04-25T11:42:30.759443+0000","last_fullsized":"2024-04-25T11:42:30.759443+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:50.265624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703731+0000","last_change":"2024-04-25T11:42:29.705202+0000","last_active":"2024-04-25T11:42:30.703731+0000","last_peered":"2024-04-25T11:42:30.703731+0000","last_clean":"2024-04-25T11:42:30.703731+0000","last_became_active":"2024-04-25T11:42:29.704800+0000","last_became_peered":"2024-04-25T11:42:29.704800+0000","last_unstale":"2024-04-25T11:42:30.703731+0000","last_undegraded":"2024-04-25T11:42:30.703731+0000","last_fullsized":"2024-04-25T11:42:30.703731+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:01.099157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758316+0000","last_change":"2024-04-25T11:42:29.689069+0000","last_active":"2024-04-25T11:42:30.758316+0000","last_peered":"2024-04-25T11:42:30.758316+0000","last_clean":"2024-04-25T11:42:30.758316+0000","last_became_active":"2024-04-25T11:42:29.688513+0000","last_became_peered":"2024-04-25T11:42:29.688513+0000","last_unstale":"2024-04-25T11:42:30.758316+0000","last_undegraded":"2024-04-25T11:42:30.758316+0000","last_fullsized":"2024-04-25T11:42:30.758316+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:07.270671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685793+0000","last_change":"2024-04-25T11:42:29.705221+0000","last_active":"2024-04-25T11:42:30.685793+0000","last_peered":"2024-04-25T11:42:30.685793+0000","last_clean":"2024-04-25T11:42:30.685793+0000","last_became_active":"2024-04-25T11:42:29.702611+0000","last_became_peered":"2024-04-25T11:42:29.702611+0000","last_unstale":"2024-04-25T11:42:30.685793+0000","last_undegraded":"2024-04-25T11:42:30.685793+0000","last_fullsized":"2024-04-25T11:42:30.685793+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.013260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685213+0000","last_change":"2024-04-25T11:42:29.675532+0000","last_active":"2024-04-25T11:42:30.685213+0000","last_peered":"2024-04-25T11:42:30.685213+0000","last_clean":"2024-04-25T11:42:30.685213+0000","last_became_active":"2024-04-25T11:42:29.674709+0000","last_became_peered":"2024-04-25T11:42:29.674709+0000","last_unstale":"2024-04-25T11:42:30.685213+0000","last_undegraded":"2024-04-25T11:42:30.685213+0000","last_fullsized":"2024-04-25T11:42:30.685213+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:26.268376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758478+0000","last_change":"2024-04-25T11:42:29.689583+0000","last_active":"2024-04-25T11:42:30.758478+0000","last_peered":"2024-04-25T11:42:30.758478+0000","last_clean":"2024-04-25T11:42:30.758478+0000","last_became_active":"2024-04-25T11:42:29.689200+0000","last_became_peered":"2024-04-25T11:42:29.689200+0000","last_unstale":"2024-04-25T11:42:30.758478+0000","last_undegraded":"2024-04-25T11:42:30.758478+0000","last_fullsized":"2024-04-25T11:42:30.758478+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:00.346220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756666+0000","last_change":"2024-04-25T11:42:29.702678+0000","last_active":"2024-04-25T11:42:30.756666+0000","last_peered":"2024-04-25T11:42:30.756666+0000","last_clean":"2024-04-25T11:42:30.756666+0000","last_became_active":"2024-04-25T11:42:29.701064+0000","last_became_peered":"2024-04-25T11:42:29.701064+0000","last_unstale":"2024-04-25T11:42:30.756666+0000","last_undegraded":"2024-04-25T11:42:30.756666+0000","last_fullsized":"2024-04-25T11:42:30.756666+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:59.966683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758199+0000","last_change":"2024-04-25T11:42:29.713954+0000","last_active":"2024-04-25T11:42:30.758199+0000","last_peered":"2024-04-25T11:42:30.758199+0000","last_clean":"2024-04-25T11:42:30.758199+0000","last_became_active":"2024-04-25T11:42:29.713401+0000","last_became_peered":"2024-04-25T11:42:29.713401+0000","last_unstale":"2024-04-25T11:42:30.758199+0000","last_undegraded":"2024-04-25T11:42:30.758199+0000","last_fullsized":"2024-04-25T11:42:30.758199+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.763250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688693+0000","last_change":"2024-04-25T11:42:29.689859+0000","last_active":"2024-04-25T11:42:30.688693+0000","last_peered":"2024-04-25T11:42:30.688693+0000","last_clean":"2024-04-25T11:42:30.688693+0000","last_became_active":"2024-04-25T11:42:29.689517+0000","last_became_peered":"2024-04-25T11:42:29.689517+0000","last_unstale":"2024-04-25T11:42:30.688693+0000","last_undegraded":"2024-04-25T11:42:30.688693+0000","last_fullsized":"2024-04-25T11:42:30.688693+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:05.599441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757018+0000","last_change":"2024-04-25T11:42:29.694451+0000","last_active":"2024-04-25T11:42:30.757018+0000","last_peered":"2024-04-25T11:42:30.757018+0000","last_clean":"2024-04-25T11:42:30.757018+0000","last_became_active":"2024-04-25T11:42:29.693670+0000","last_became_peered":"2024-04-25T11:42:29.693670+0000","last_unstale":"2024-04-25T11:42:30.757018+0000","last_undegraded":"2024-04-25T11:42:30.757018+0000","last_fullsized":"2024-04-25T11:42:30.757018+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:14.975091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757385+0000","last_change":"2024-04-25T11:42:29.688082+0000","last_active":"2024-04-25T11:42:30.757385+0000","last_peered":"2024-04-25T11:42:30.757385+0000","last_clean":"2024-04-25T11:42:30.757385+0000","last_became_active":"2024-04-25T11:42:29.687705+0000","last_became_peered":"2024-04-25T11:42:29.687705+0000","last_unstale":"2024-04-25T11:42:30.757385+0000","last_undegraded":"2024-04-25T11:42:30.757385+0000","last_fullsized":"2024-04-25T11:42:30.757385+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:43.651140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688493+0000","last_change":"2024-04-25T11:42:29.694587+0000","last_active":"2024-04-25T11:42:30.688493+0000","last_peered":"2024-04-25T11:42:30.688493+0000","last_clean":"2024-04-25T11:42:30.688493+0000","last_became_active":"2024-04-25T11:42:29.694204+0000","last_became_peered":"2024-04-25T11:42:29.694204+0000","last_unstale":"2024-04-25T11:42:30.688493+0000","last_undegraded":"2024-04-25T11:42:30.688493+0000","last_fullsized":"2024-04-25T11:42:30.688493+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:03.196909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.688018+0000","last_change":"2024-04-25T11:42:29.706810+0000","last_active":"2024-04-25T11:42:30.688018+0000","last_peered":"2024-04-25T11:42:30.688018+0000","last_clean":"2024-04-25T11:42:30.688018+0000","last_became_active":"2024-04-25T11:42:29.706286+0000","last_became_peered":"2024-04-25T11:42:29.706286+0000","last_unstale":"2024-04-25T11:42:30.688018+0000","last_undegraded":"2024-04-25T11:42:30.688018+0000","last_fullsized":"2024-04-25T11:42:30.688018+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:59.970047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684821+0000","last_change":"2024-04-25T11:42:29.699173+0000","last_active":"2024-04-25T11:42:30.684821+0000","last_peered":"2024-04-25T11:42:30.684821+0000","last_clean":"2024-04-25T11:42:30.684821+0000","last_became_active":"2024-04-25T11:42:29.698859+0000","last_became_peered":"2024-04-25T11:42:29.698859+0000","last_unstale":"2024-04-25T11:42:30.684821+0000","last_undegraded":"2024-04-25T11:42:30.684821+0000","last_fullsized":"2024-04-25T11:42:30.684821+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:00.346220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758042+0000","last_change":"2024-04-25T11:42:29.704384+0000","last_active":"2024-04-25T11:42:30.758042+0000","last_peered":"2024-04-25T11:42:30.758042+0000","last_clean":"2024-04-25T11:42:30.758042+0000","last_became_active":"2024-04-25T11:42:29.703841+0000","last_became_peered":"2024-04-25T11:42:29.703841+0000","last_unstale":"2024-04-25T11:42:30.758042+0000","last_undegraded":"2024-04-25T11:42:30.758042+0000","last_fullsized":"2024-04-25T11:42:30.758042+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:07.270671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703461+0000","last_change":"2024-04-25T11:42:29.702175+0000","last_active":"2024-04-25T11:42:30.703461+0000","last_peered":"2024-04-25T11:42:30.703461+0000","last_clean":"2024-04-25T11:42:30.703461+0000","last_became_active":"2024-04-25T11:42:29.701784+0000","last_became_peered":"2024-04-25T11:42:29.701784+0000","last_unstale":"2024-04-25T11:42:30.703461+0000","last_undegraded":"2024-04-25T11:42:30.703461+0000","last_fullsized":"2024-04-25T11:42:30.703461+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:05.304391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"142'4","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.689286+0000","last_change":"2024-04-25T11:42:28.662201+0000","last_active":"2024-04-25T11:42:30.689286+0000","last_peered":"2024-04-25T11:42:30.689286+0000","last_clean":"2024-04-25T11:42:30.689286+0000","last_became_active":"2024-04-25T11:42:28.661792+0000","last_became_peered":"2024-04-25T11:42:28.661792+0000","last_unstale":"2024-04-25T11:42:30.689286+0000","last_undegraded":"2024-04-25T11:42:30.689286+0000","last_fullsized":"2024-04-25T11:42:30.689286+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:57.047396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.689453+0000","last_change":"2024-04-25T11:42:29.693546+0000","last_active":"2024-04-25T11:42:30.689453+0000","last_peered":"2024-04-25T11:42:30.689453+0000","last_clean":"2024-04-25T11:42:30.689453+0000","last_became_active":"2024-04-25T11:42:29.693163+0000","last_became_peered":"2024-04-25T11:42:29.693163+0000","last_unstale":"2024-04-25T11:42:30.689453+0000","last_undegraded":"2024-04-25T11:42:30.689453+0000","last_fullsized":"2024-04-25T11:42:30.689453+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:42.882880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683930+0000","last_change":"2024-04-25T11:42:28.660531+0000","last_active":"2024-04-25T11:42:30.683930+0000","last_peered":"2024-04-25T11:42:30.683930+0000","last_clean":"2024-04-25T11:42:30.683930+0000","last_became_active":"2024-04-25T11:42:28.660194+0000","last_became_peered":"2024-04-25T11:42:28.660194+0000","last_unstale":"2024-04-25T11:42:30.683930+0000","last_undegraded":"2024-04-25T11:42:30.683930+0000","last_fullsized":"2024-04-25T11:42:30.683930+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:39.335110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759087+0000","last_change":"2024-04-25T11:42:29.697958+0000","last_active":"2024-04-25T11:42:30.759087+0000","last_peered":"2024-04-25T11:42:30.759087+0000","last_clean":"2024-04-25T11:42:30.759087+0000","last_became_active":"2024-04-25T11:42:29.697444+0000","last_became_peered":"2024-04-25T11:42:29.697444+0000","last_unstale":"2024-04-25T11:42:30.759087+0000","last_undegraded":"2024-04-25T11:42:30.759087+0000","last_fullsized":"2024-04-25T11:42:30.759087+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.926190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686590+0000","last_change":"2024-04-25T11:42:28.663742+0000","last_active":"2024-04-25T11:42:30.686590+0000","last_peered":"2024-04-25T11:42:30.686590+0000","last_clean":"2024-04-25T11:42:30.686590+0000","last_became_active":"2024-04-25T11:42:28.662533+0000","last_became_peered":"2024-04-25T11:42:28.662533+0000","last_unstale":"2024-04-25T11:42:30.686590+0000","last_undegraded":"2024-04-25T11:42:30.686590+0000","last_fullsized":"2024-04-25T11:42:30.686590+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757734+0000","last_change":"2024-04-25T11:42:29.700925+0000","last_active":"2024-04-25T11:42:30.757734+0000","last_peered":"2024-04-25T11:42:30.757734+0000","last_clean":"2024-04-25T11:42:30.757734+0000","last_became_active":"2024-04-25T11:42:29.700354+0000","last_became_peered":"2024-04-25T11:42:29.700354+0000","last_unstale":"2024-04-25T11:42:30.757734+0000","last_undegraded":"2024-04-25T11:42:30.757734+0000","last_fullsized":"2024-04-25T11:42:30.757734+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:25.104518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759009+0000","last_change":"2024-04-25T11:42:28.657525+0000","last_active":"2024-04-25T11:42:30.759009+0000","last_peered":"2024-04-25T11:42:30.759009+0000","last_clean":"2024-04-25T11:42:30.759009+0000","last_became_active":"2024-04-25T11:42:28.656726+0000","last_became_peered":"2024-04-25T11:42:28.656726+0000","last_unstale":"2024-04-25T11:42:30.759009+0000","last_undegraded":"2024-04-25T11:42:30.759009+0000","last_fullsized":"2024-04-25T11:42:30.759009+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:50.745291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757970+0000","last_change":"2024-04-25T11:42:29.690685+0000","last_active":"2024-04-25T11:42:30.757970+0000","last_peered":"2024-04-25T11:42:30.757970+0000","last_clean":"2024-04-25T11:42:30.757970+0000","last_became_active":"2024-04-25T11:42:29.690312+0000","last_became_peered":"2024-04-25T11:42:29.690312+0000","last_unstale":"2024-04-25T11:42:30.757970+0000","last_undegraded":"2024-04-25T11:42:30.757970+0000","last_fullsized":"2024-04-25T11:42:30.757970+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:48.013757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758021+0000","last_change":"2024-04-25T11:42:28.655218+0000","last_active":"2024-04-25T11:42:30.758021+0000","last_peered":"2024-04-25T11:42:30.758021+0000","last_clean":"2024-04-25T11:42:30.758021+0000","last_became_active":"2024-04-25T11:42:28.649304+0000","last_became_peered":"2024-04-25T11:42:28.649304+0000","last_unstale":"2024-04-25T11:42:30.758021+0000","last_undegraded":"2024-04-25T11:42:30.758021+0000","last_fullsized":"2024-04-25T11:42:30.758021+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:12.300553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683660+0000","last_change":"2024-04-25T11:42:29.690861+0000","last_active":"2024-04-25T11:42:30.683660+0000","last_peered":"2024-04-25T11:42:30.683660+0000","last_clean":"2024-04-25T11:42:30.683660+0000","last_became_active":"2024-04-25T11:42:29.690468+0000","last_became_peered":"2024-04-25T11:42:29.690468+0000","last_unstale":"2024-04-25T11:42:30.683660+0000","last_undegraded":"2024-04-25T11:42:30.683660+0000","last_fullsized":"2024-04-25T11:42:30.683660+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.866512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685383+0000","last_change":"2024-04-25T11:42:28.654295+0000","last_active":"2024-04-25T11:42:30.685383+0000","last_peered":"2024-04-25T11:42:30.685383+0000","last_clean":"2024-04-25T11:42:30.685383+0000","last_became_active":"2024-04-25T11:42:28.653980+0000","last_became_peered":"2024-04-25T11:42:28.653980+0000","last_unstale":"2024-04-25T11:42:30.685383+0000","last_undegraded":"2024-04-25T11:42:30.685383+0000","last_fullsized":"2024-04-25T11:42:30.685383+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:41.866541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685474+0000","last_change":"2024-04-25T11:42:29.680408+0000","last_active":"2024-04-25T11:42:30.685474+0000","last_peered":"2024-04-25T11:42:30.685474+0000","last_clean":"2024-04-25T11:42:30.685474+0000","last_became_active":"2024-04-25T11:42:29.678815+0000","last_became_peered":"2024-04-25T11:42:29.678815+0000","last_unstale":"2024-04-25T11:42:30.685474+0000","last_undegraded":"2024-04-25T11:42:30.685474+0000","last_fullsized":"2024-04-25T11:42:30.685474+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:13.316892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.682968+0000","last_change":"2024-04-25T11:42:28.655419+0000","last_active":"2024-04-25T11:42:30.682968+0000","last_peered":"2024-04-25T11:42:30.682968+0000","last_clean":"2024-04-25T11:42:30.682968+0000","last_became_active":"2024-04-25T11:42:28.654795+0000","last_became_peered":"2024-04-25T11:42:28.654795+0000","last_unstale":"2024-04-25T11:42:30.682968+0000","last_undegraded":"2024-04-25T11:42:30.682968+0000","last_fullsized":"2024-04-25T11:42:30.682968+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:31.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756365+0000","last_change":"2024-04-25T11:42:29.688271+0000","last_active":"2024-04-25T11:42:30.756365+0000","last_peered":"2024-04-25T11:42:30.756365+0000","last_clean":"2024-04-25T11:42:30.756365+0000","last_became_active":"2024-04-25T11:42:29.687091+0000","last_became_peered":"2024-04-25T11:42:29.687091+0000","last_unstale":"2024-04-25T11:42:30.756365+0000","last_undegraded":"2024-04-25T11:42:30.756365+0000","last_fullsized":"2024-04-25T11:42:30.756365+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:48.763250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.702685+0000","last_change":"2024-04-25T11:42:28.647128+0000","last_active":"2024-04-25T11:42:30.702685+0000","last_peered":"2024-04-25T11:42:30.702685+0000","last_clean":"2024-04-25T11:42:30.702685+0000","last_became_active":"2024-04-25T11:42:28.645733+0000","last_became_peered":"2024-04-25T11:42:28.645733+0000","last_unstale":"2024-04-25T11:42:30.702685+0000","last_undegraded":"2024-04-25T11:42:30.702685+0000","last_fullsized":"2024-04-25T11:42:30.702685+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:42.492482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759651+0000","last_change":"2024-04-25T11:42:29.706581+0000","last_active":"2024-04-25T11:42:30.759651+0000","last_peered":"2024-04-25T11:42:30.759651+0000","last_clean":"2024-04-25T11:42:30.759651+0000","last_became_active":"2024-04-25T11:42:29.704945+0000","last_became_peered":"2024-04-25T11:42:29.704945+0000","last_unstale":"2024-04-25T11:42:30.759651+0000","last_undegraded":"2024-04-25T11:42:30.759651+0000","last_fullsized":"2024-04-25T11:42:30.759651+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:00.342533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686534+0000","last_change":"2024-04-25T11:42:28.654392+0000","last_active":"2024-04-25T11:42:30.686534+0000","last_peered":"2024-04-25T11:42:30.686534+0000","last_clean":"2024-04-25T11:42:30.686534+0000","last_became_active":"2024-04-25T11:42:28.654052+0000","last_became_peered":"2024-04-25T11:42:28.654052+0000","last_unstale":"2024-04-25T11:42:30.686534+0000","last_undegraded":"2024-04-25T11:42:30.686534+0000","last_fullsized":"2024-04-25T11:42:30.686534+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:57.662132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757933+0000","last_change":"2024-04-25T11:42:29.695665+0000","last_active":"2024-04-25T11:42:30.757933+0000","last_peered":"2024-04-25T11:42:30.757933+0000","last_clean":"2024-04-25T11:42:30.757933+0000","last_became_active":"2024-04-25T11:42:29.694403+0000","last_became_peered":"2024-04-25T11:42:29.694403+0000","last_unstale":"2024-04-25T11:42:30.757933+0000","last_undegraded":"2024-04-25T11:42:30.757933+0000","last_fullsized":"2024-04-25T11:42:30.757933+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:33.682475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759546+0000","last_change":"2024-04-25T11:42:28.657338+0000","last_active":"2024-04-25T11:42:30.759546+0000","last_peered":"2024-04-25T11:42:30.759546+0000","last_clean":"2024-04-25T11:42:30.759546+0000","last_became_active":"2024-04-25T11:42:28.656483+0000","last_became_peered":"2024-04-25T11:42:28.656483+0000","last_unstale":"2024-04-25T11:42:30.759546+0000","last_undegraded":"2024-04-25T11:42:30.759546+0000","last_fullsized":"2024-04-25T11:42:30.759546+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:17.656405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683016+0000","last_change":"2024-04-25T11:42:29.696298+0000","last_active":"2024-04-25T11:42:30.683016+0000","last_peered":"2024-04-25T11:42:30.683016+0000","last_clean":"2024-04-25T11:42:30.683016+0000","last_became_active":"2024-04-25T11:42:29.695706+0000","last_became_peered":"2024-04-25T11:42:29.695706+0000","last_unstale":"2024-04-25T11:42:30.683016+0000","last_undegraded":"2024-04-25T11:42:30.683016+0000","last_fullsized":"2024-04-25T11:42:30.683016+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:45.417509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"142'3","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684359+0000","last_change":"2024-04-25T11:42:28.651772+0000","last_active":"2024-04-25T11:42:30.684359+0000","last_peered":"2024-04-25T11:42:30.684359+0000","last_clean":"2024-04-25T11:42:30.684359+0000","last_became_active":"2024-04-25T11:42:28.651416+0000","last_became_peered":"2024-04-25T11:42:28.651416+0000","last_unstale":"2024-04-25T11:42:30.684359+0000","last_undegraded":"2024-04-25T11:42:30.684359+0000","last_fullsized":"2024-04-25T11:42:30.684359+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:45.898449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756441+0000","last_change":"2024-04-25T11:42:29.709598+0000","last_active":"2024-04-25T11:42:30.756441+0000","last_peered":"2024-04-25T11:42:30.756441+0000","last_clean":"2024-04-25T11:42:30.756441+0000","last_became_active":"2024-04-25T11:42:29.707835+0000","last_became_peered":"2024-04-25T11:42:29.707835+0000","last_unstale":"2024-04-25T11:42:30.756441+0000","last_undegraded":"2024-04-25T11:42:30.756441+0000","last_fullsized":"2024-04-25T11:42:30.756441+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:44.904701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"142'6","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758374+0000","last_change":"2024-04-25T11:42:28.653989+0000","last_active":"2024-04-25T11:42:30.758374+0000","last_peered":"2024-04-25T11:42:30.758374+0000","last_clean":"2024-04-25T11:42:30.758374+0000","last_became_active":"2024-04-25T11:42:28.653670+0000","last_became_peered":"2024-04-25T11:42:28.653670+0000","last_unstale":"2024-04-25T11:42:30.758374+0000","last_undegraded":"2024-04-25T11:42:30.758374+0000","last_fullsized":"2024-04-25T11:42:30.758374+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:38.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758247+0000","last_change":"2024-04-25T11:42:29.696427+0000","last_active":"2024-04-25T11:42:30.758247+0000","last_peered":"2024-04-25T11:42:30.758247+0000","last_clean":"2024-04-25T11:42:30.758247+0000","last_became_active":"2024-04-25T11:42:29.696069+0000","last_became_peered":"2024-04-25T11:42:29.696069+0000","last_unstale":"2024-04-25T11:42:30.758247+0000","last_undegraded":"2024-04-25T11:42:30.758247+0000","last_fullsized":"2024-04-25T11:42:30.758247+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.398770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703038+0000","last_change":"2024-04-25T11:42:28.650997+0000","last_active":"2024-04-25T11:42:30.703038+0000","last_peered":"2024-04-25T11:42:30.703038+0000","last_clean":"2024-04-25T11:42:30.703038+0000","last_became_active":"2024-04-25T11:42:28.650647+0000","last_became_peered":"2024-04-25T11:42:28.650647+0000","last_unstale":"2024-04-25T11:42:30.703038+0000","last_undegraded":"2024-04-25T11:42:30.703038+0000","last_fullsized":"2024-04-25T11:42:30.703038+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:46.078312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686348+0000","last_change":"2024-04-25T11:42:29.696398+0000","last_active":"2024-04-25T11:42:30.686348+0000","last_peered":"2024-04-25T11:42:30.686348+0000","last_clean":"2024-04-25T11:42:30.686348+0000","last_became_active":"2024-04-25T11:42:29.696046+0000","last_became_peered":"2024-04-25T11:42:29.696046+0000","last_unstale":"2024-04-25T11:42:30.686348+0000","last_undegraded":"2024-04-25T11:42:30.686348+0000","last_fullsized":"2024-04-25T11:42:30.686348+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:55.724030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758229+0000","last_change":"2024-04-25T11:42:28.660110+0000","last_active":"2024-04-25T11:42:30.758229+0000","last_peered":"2024-04-25T11:42:30.758229+0000","last_clean":"2024-04-25T11:42:30.758229+0000","last_became_active":"2024-04-25T11:42:28.659719+0000","last_became_peered":"2024-04-25T11:42:28.659719+0000","last_unstale":"2024-04-25T11:42:30.758229+0000","last_undegraded":"2024-04-25T11:42:30.758229+0000","last_fullsized":"2024-04-25T11:42:30.758229+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:02.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683236+0000","last_change":"2024-04-25T11:42:29.691401+0000","last_active":"2024-04-25T11:42:30.683236+0000","last_peered":"2024-04-25T11:42:30.683236+0000","last_clean":"2024-04-25T11:42:30.683236+0000","last_became_active":"2024-04-25T11:42:29.691038+0000","last_became_peered":"2024-04-25T11:42:29.691038+0000","last_unstale":"2024-04-25T11:42:30.683236+0000","last_undegraded":"2024-04-25T11:42:30.683236+0000","last_fullsized":"2024-04-25T11:42:30.683236+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:24.315576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757332+0000","last_change":"2024-04-25T11:42:28.643885+0000","last_active":"2024-04-25T11:42:30.757332+0000","last_peered":"2024-04-25T11:42:30.757332+0000","last_clean":"2024-04-25T11:42:30.757332+0000","last_became_active":"2024-04-25T11:42:28.641975+0000","last_became_peered":"2024-04-25T11:42:28.641975+0000","last_unstale":"2024-04-25T11:42:30.757332+0000","last_undegraded":"2024-04-25T11:42:30.757332+0000","last_fullsized":"2024-04-25T11:42:30.757332+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:02.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684521+0000","last_change":"2024-04-25T11:42:29.689006+0000","last_active":"2024-04-25T11:42:30.684521+0000","last_peered":"2024-04-25T11:42:30.684521+0000","last_clean":"2024-04-25T11:42:30.684521+0000","last_became_active":"2024-04-25T11:42:29.688662+0000","last_became_peered":"2024-04-25T11:42:29.688662+0000","last_unstale":"2024-04-25T11:42:30.684521+0000","last_undegraded":"2024-04-25T11:42:30.684521+0000","last_fullsized":"2024-04-25T11:42:30.684521+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.242694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"142'8","reported_seq":22,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758630+0000","last_change":"2024-04-25T11:42:28.650070+0000","last_active":"2024-04-25T11:42:30.758630+0000","last_peered":"2024-04-25T11:42:30.758630+0000","last_clean":"2024-04-25T11:42:30.758630+0000","last_became_active":"2024-04-25T11:42:28.649743+0000","last_became_peered":"2024-04-25T11:42:28.649743+0000","last_unstale":"2024-04-25T11:42:30.758630+0000","last_undegraded":"2024-04-25T11:42:30.758630+0000","last_fullsized":"2024-04-25T11:42:30.758630+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:58.953708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758260+0000","last_change":"2024-04-25T11:42:29.693003+0000","last_active":"2024-04-25T11:42:30.758260+0000","last_peered":"2024-04-25T11:42:30.758260+0000","last_clean":"2024-04-25T11:42:30.758260+0000","last_became_active":"2024-04-25T11:42:29.692646+0000","last_became_peered":"2024-04-25T11:42:29.692646+0000","last_unstale":"2024-04-25T11:42:30.758260+0000","last_undegraded":"2024-04-25T11:42:30.758260+0000","last_fullsized":"2024-04-25T11:42:30.758260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:56.224292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758938+0000","last_change":"2024-04-25T11:42:28.652398+0000","last_active":"2024-04-25T11:42:30.758938+0000","last_peered":"2024-04-25T11:42:30.758938+0000","last_clean":"2024-04-25T11:42:30.758938+0000","last_became_active":"2024-04-25T11:42:28.652057+0000","last_became_peered":"2024-04-25T11:42:28.652057+0000","last_unstale":"2024-04-25T11:42:30.758938+0000","last_undegraded":"2024-04-25T11:42:30.758938+0000","last_fullsized":"2024-04-25T11:42:30.758938+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:04.583102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757338+0000","last_change":"2024-04-25T11:42:29.691459+0000","last_active":"2024-04-25T11:42:30.757338+0000","last_peered":"2024-04-25T11:42:30.757338+0000","last_clean":"2024-04-25T11:42:30.757338+0000","last_became_active":"2024-04-25T11:42:29.691120+0000","last_became_peered":"2024-04-25T11:42:29.691120+0000","last_unstale":"2024-04-25T11:42:30.757338+0000","last_undegraded":"2024-04-25T11:42:30.757338+0000","last_fullsized":"2024-04-25T11:42:30.757338+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:59.727855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684383+0000","last_change":"2024-04-25T11:42:28.649062+0000","last_active":"2024-04-25T11:42:30.684383+0000","last_peered":"2024-04-25T11:42:30.684383+0000","last_clean":"2024-04-25T11:42:30.684383+0000","last_became_active":"2024-04-25T11:42:28.648536+0000","last_became_peered":"2024-04-25T11:42:28.648536+0000","last_unstale":"2024-04-25T11:42:30.684383+0000","last_undegraded":"2024-04-25T11:42:30.684383+0000","last_fullsized":"2024-04-25T11:42:30.684383+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:49.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758859+0000","last_change":"2024-04-25T11:42:29.685542+0000","last_active":"2024-04-25T11:42:30.758859+0000","last_peered":"2024-04-25T11:42:30.758859+0000","last_clean":"2024-04-25T11:42:30.758859+0000","last_became_active":"2024-04-25T11:42:29.685205+0000","last_became_peered":"2024-04-25T11:42:29.685205+0000","last_unstale":"2024-04-25T11:42:30.758859+0000","last_undegraded":"2024-04-25T11:42:30.758859+0000","last_fullsized":"2024-04-25T11:42:30.758859+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.242694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703199+0000","last_change":"2024-04-25T11:42:28.651314+0000","last_active":"2024-04-25T11:42:30.703199+0000","last_peered":"2024-04-25T11:42:30.703199+0000","last_clean":"2024-04-25T11:42:30.703199+0000","last_became_active":"2024-04-25T11:42:28.650945+0000","last_became_peered":"2024-04-25T11:42:28.650945+0000","last_unstale":"2024-04-25T11:42:30.703199+0000","last_undegraded":"2024-04-25T11:42:30.703199+0000","last_fullsized":"2024-04-25T11:42:30.703199+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:08.845636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703128+0000","last_change":"2024-04-25T11:42:29.698412+0000","last_active":"2024-04-25T11:42:30.703128+0000","last_peered":"2024-04-25T11:42:30.703128+0000","last_clean":"2024-04-25T11:42:30.703128+0000","last_became_active":"2024-04-25T11:42:29.697766+0000","last_became_peered":"2024-04-25T11:42:29.697766+0000","last_unstale":"2024-04-25T11:42:30.703128+0000","last_undegraded":"2024-04-25T11:42:30.703128+0000","last_fullsized":"2024-04-25T11:42:30.703128+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:58.467574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.758157+0000","last_change":"2024-04-25T11:42:28.656051+0000","last_active":"2024-04-25T11:42:30.758157+0000","last_peered":"2024-04-25T11:42:30.758157+0000","last_clean":"2024-04-25T11:42:30.758157+0000","last_became_active":"2024-04-25T11:42:28.655708+0000","last_became_peered":"2024-04-25T11:42:28.655708+0000","last_unstale":"2024-04-25T11:42:30.758157+0000","last_undegraded":"2024-04-25T11:42:30.758157+0000","last_fullsized":"2024-04-25T11:42:30.758157+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:34.725563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.702813+0000","last_change":"2024-04-25T11:42:29.691606+0000","last_active":"2024-04-25T11:42:30.702813+0000","last_peered":"2024-04-25T11:42:30.702813+0000","last_clean":"2024-04-25T11:42:30.702813+0000","last_became_active":"2024-04-25T11:42:29.689254+0000","last_became_peered":"2024-04-25T11:42:29.689254+0000","last_unstale":"2024-04-25T11:42:30.702813+0000","last_undegraded":"2024-04-25T11:42:30.702813+0000","last_fullsized":"2024-04-25T11:42:30.702813+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:42.693174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756360+0000","last_change":"2024-04-25T11:42:28.652991+0000","last_active":"2024-04-25T11:42:30.756360+0000","last_peered":"2024-04-25T11:42:30.756360+0000","last_clean":"2024-04-25T11:42:30.756360+0000","last_became_active":"2024-04-25T11:42:28.652686+0000","last_became_peered":"2024-04-25T11:42:28.652686+0000","last_unstale":"2024-04-25T11:42:30.756360+0000","last_undegraded":"2024-04-25T11:42:30.756360+0000","last_fullsized":"2024-04-25T11:42:30.756360+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:49.249284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759116+0000","last_change":"2024-04-25T11:42:29.687841+0000","last_active":"2024-04-25T11:42:30.759116+0000","last_peered":"2024-04-25T11:42:30.759116+0000","last_clean":"2024-04-25T11:42:30.759116+0000","last_became_active":"2024-04-25T11:42:29.687453+0000","last_became_peered":"2024-04-25T11:42:29.687453+0000","last_unstale":"2024-04-25T11:42:30.759116+0000","last_undegraded":"2024-04-25T11:42:30.759116+0000","last_fullsized":"2024-04-25T11:42:30.759116+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:08.013260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757154+0000","last_change":"2024-04-25T11:42:28.658831+0000","last_active":"2024-04-25T11:42:30.757154+0000","last_peered":"2024-04-25T11:42:30.757154+0000","last_clean":"2024-04-25T11:42:30.757154+0000","last_became_active":"2024-04-25T11:42:28.658360+0000","last_became_peered":"2024-04-25T11:42:28.658360+0000","last_unstale":"2024-04-25T11:42:30.757154+0000","last_undegraded":"2024-04-25T11:42:30.757154+0000","last_fullsized":"2024-04-25T11:42:30.757154+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:57.662132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756887+0000","last_change":"2024-04-25T11:42:29.687804+0000","last_active":"2024-04-25T11:42:30.756887+0000","last_peered":"2024-04-25T11:42:30.756887+0000","last_clean":"2024-04-25T11:42:30.756887+0000","last_became_active":"2024-04-25T11:42:29.687410+0000","last_became_peered":"2024-04-25T11:42:29.687410+0000","last_unstale":"2024-04-25T11:42:30.756887+0000","last_undegraded":"2024-04-25T11:42:30.756887+0000","last_fullsized":"2024-04-25T11:42:30.756887+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:54.878227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684026+0000","last_change":"2024-04-25T11:42:28.660077+0000","last_active":"2024-04-25T11:42:30.684026+0000","last_peered":"2024-04-25T11:42:30.684026+0000","last_clean":"2024-04-25T11:42:30.684026+0000","last_became_active":"2024-04-25T11:42:28.659663+0000","last_became_peered":"2024-04-25T11:42:28.659663+0000","last_unstale":"2024-04-25T11:42:30.684026+0000","last_undegraded":"2024-04-25T11:42:30.684026+0000","last_fullsized":"2024-04-25T11:42:30.684026+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:04.583102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703680+0000","last_change":"2024-04-25T11:42:29.701838+0000","last_active":"2024-04-25T11:42:30.703680+0000","last_peered":"2024-04-25T11:42:30.703680+0000","last_clean":"2024-04-25T11:42:30.703680+0000","last_became_active":"2024-04-25T11:42:29.699830+0000","last_became_peered":"2024-04-25T11:42:29.699830+0000","last_unstale":"2024-04-25T11:42:30.703680+0000","last_undegraded":"2024-04-25T11:42:30.703680+0000","last_fullsized":"2024-04-25T11:42:30.703680+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:39.051407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759199+0000","last_change":"2024-04-25T11:42:28.647452+0000","last_active":"2024-04-25T11:42:30.759199+0000","last_peered":"2024-04-25T11:42:30.759199+0000","last_clean":"2024-04-25T11:42:30.759199+0000","last_became_active":"2024-04-25T11:42:28.647120+0000","last_became_peered":"2024-04-25T11:42:28.647120+0000","last_unstale":"2024-04-25T11:42:30.759199+0000","last_undegraded":"2024-04-25T11:42:30.759199+0000","last_fullsized":"2024-04-25T11:42:30.759199+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:30.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703484+0000","last_change":"2024-04-25T11:42:29.707285+0000","last_active":"2024-04-25T11:42:30.703484+0000","last_peered":"2024-04-25T11:42:30.703484+0000","last_clean":"2024-04-25T11:42:30.703484+0000","last_became_active":"2024-04-25T11:42:29.704368+0000","last_became_peered":"2024-04-25T11:42:29.704368+0000","last_unstale":"2024-04-25T11:42:30.703484+0000","last_undegraded":"2024-04-25T11:42:30.703484+0000","last_fullsized":"2024-04-25T11:42:30.703484+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:49.780342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"143'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.705855+0000","last_change":"2024-04-25T11:42:28.654964+0000","last_active":"2024-04-25T11:42:30.705855+0000","last_peered":"2024-04-25T11:42:30.705855+0000","last_clean":"2024-04-25T11:42:30.705855+0000","last_became_active":"2024-04-25T11:42:28.654473+0000","last_became_peered":"2024-04-25T11:42:28.654473+0000","last_unstale":"2024-04-25T11:42:30.705855+0000","last_undegraded":"2024-04-25T11:42:30.705855+0000","last_fullsized":"2024-04-25T11:42:30.705855+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:07.926295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686637+0000","last_change":"2024-04-25T11:42:29.693022+0000","last_active":"2024-04-25T11:42:30.686637+0000","last_peered":"2024-04-25T11:42:30.686637+0000","last_clean":"2024-04-25T11:42:30.686637+0000","last_became_active":"2024-04-25T11:42:29.692690+0000","last_became_peered":"2024-04-25T11:42:29.692690+0000","last_unstale":"2024-04-25T11:42:30.686637+0000","last_undegraded":"2024-04-25T11:42:30.686637+0000","last_fullsized":"2024-04-25T11:42:30.686637+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:45.369012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703656+0000","last_change":"2024-04-25T11:42:28.652813+0000","last_active":"2024-04-25T11:42:30.703656+0000","last_peered":"2024-04-25T11:42:30.703656+0000","last_clean":"2024-04-25T11:42:30.703656+0000","last_became_active":"2024-04-25T11:42:28.652487+0000","last_became_peered":"2024-04-25T11:42:28.652487+0000","last_unstale":"2024-04-25T11:42:30.703656+0000","last_undegraded":"2024-04-25T11:42:30.703656+0000","last_fullsized":"2024-04-25T11:42:30.703656+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:33.863648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684212+0000","last_change":"2024-04-25T11:42:29.696528+0000","last_active":"2024-04-25T11:42:30.684212+0000","last_peered":"2024-04-25T11:42:30.684212+0000","last_clean":"2024-04-25T11:42:30.684212+0000","last_became_active":"2024-04-25T11:42:29.696140+0000","last_became_peered":"2024-04-25T11:42:29.696140+0000","last_unstale":"2024-04-25T11:42:30.684212+0000","last_undegraded":"2024-04-25T11:42:30.684212+0000","last_fullsized":"2024-04-25T11:42:30.684212+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.580891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757421+0000","last_change":"2024-04-25T11:42:28.649972+0000","last_active":"2024-04-25T11:42:30.757421+0000","last_peered":"2024-04-25T11:42:30.757421+0000","last_clean":"2024-04-25T11:42:30.757421+0000","last_became_active":"2024-04-25T11:42:28.649540+0000","last_became_peered":"2024-04-25T11:42:28.649540+0000","last_unstale":"2024-04-25T11:42:30.757421+0000","last_undegraded":"2024-04-25T11:42:30.757421+0000","last_fullsized":"2024-04-25T11:42:30.757421+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:59.326194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.703355+0000","last_change":"2024-04-25T11:42:29.698119+0000","last_active":"2024-04-25T11:42:30.703355+0000","last_peered":"2024-04-25T11:42:30.703355+0000","last_clean":"2024-04-25T11:42:30.703355+0000","last_became_active":"2024-04-25T11:42:29.697363+0000","last_became_peered":"2024-04-25T11:42:29.697363+0000","last_unstale":"2024-04-25T11:42:30.703355+0000","last_undegraded":"2024-04-25T11:42:30.703355+0000","last_fullsized":"2024-04-25T11:42:30.703355+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:46.474813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.686718+0000","last_change":"2024-04-25T11:42:28.663519+0000","last_active":"2024-04-25T11:42:30.686718+0000","last_peered":"2024-04-25T11:42:30.686718+0000","last_clean":"2024-04-25T11:42:30.686718+0000","last_became_active":"2024-04-25T11:42:28.663125+0000","last_became_peered":"2024-04-25T11:42:28.663125+0000","last_unstale":"2024-04-25T11:42:30.686718+0000","last_undegraded":"2024-04-25T11:42:30.686718+0000","last_fullsized":"2024-04-25T11:42:30.686718+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:30.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.683443+0000","last_change":"2024-04-25T11:42:29.685183+0000","last_active":"2024-04-25T11:42:30.683443+0000","last_peered":"2024-04-25T11:42:30.683443+0000","last_clean":"2024-04-25T11:42:30.683443+0000","last_became_active":"2024-04-25T11:42:29.684823+0000","last_became_peered":"2024-04-25T11:42:29.684823+0000","last_unstale":"2024-04-25T11:42:30.683443+0000","last_undegraded":"2024-04-25T11:42:30.683443+0000","last_fullsized":"2024-04-25T11:42:30.683443+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:00.596387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756550+0000","last_change":"2024-04-25T11:42:28.653937+0000","last_active":"2024-04-25T11:42:30.756550+0000","last_peered":"2024-04-25T11:42:30.756550+0000","last_clean":"2024-04-25T11:42:30.756550+0000","last_became_active":"2024-04-25T11:42:28.653047+0000","last_became_peered":"2024-04-25T11:42:28.653047+0000","last_unstale":"2024-04-25T11:42:30.756550+0000","last_undegraded":"2024-04-25T11:42:30.756550+0000","last_fullsized":"2024-04-25T11:42:30.756550+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:44.352672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.702743+0000","last_change":"2024-04-25T11:42:29.697900+0000","last_active":"2024-04-25T11:42:30.702743+0000","last_peered":"2024-04-25T11:42:30.702743+0000","last_clean":"2024-04-25T11:42:30.702743+0000","last_became_active":"2024-04-25T11:42:29.695627+0000","last_became_peered":"2024-04-25T11:42:29.695627+0000","last_unstale":"2024-04-25T11:42:30.702743+0000","last_undegraded":"2024-04-25T11:42:30.702743+0000","last_fullsized":"2024-04-25T11:42:30.702743+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:22.977989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.687203+0000","last_change":"2024-04-25T11:42:28.666769+0000","last_active":"2024-04-25T11:42:30.687203+0000","last_peered":"2024-04-25T11:42:30.687203+0000","last_clean":"2024-04-25T11:42:30.687203+0000","last_became_active":"2024-04-25T11:42:28.666363+0000","last_became_peered":"2024-04-25T11:42:28.666363+0000","last_unstale":"2024-04-25T11:42:30.687203+0000","last_undegraded":"2024-04-25T11:42:30.687203+0000","last_fullsized":"2024-04-25T11:42:30.687203+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:35.258782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.757211+0000","last_change":"2024-04-25T11:42:29.701372+0000","last_active":"2024-04-25T11:42:30.757211+0000","last_peered":"2024-04-25T11:42:30.757211+0000","last_clean":"2024-04-25T11:42:30.757211+0000","last_became_active":"2024-04-25T11:42:29.700993+0000","last_became_peered":"2024-04-25T11:42:29.700993+0000","last_unstale":"2024-04-25T11:42:30.757211+0000","last_undegraded":"2024-04-25T11:42:30.757211+0000","last_fullsized":"2024-04-25T11:42:30.757211+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:44.507891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.684259+0000","last_change":"2024-04-25T11:42:28.659425+0000","last_active":"2024-04-25T11:42:30.684259+0000","last_peered":"2024-04-25T11:42:30.684259+0000","last_clean":"2024-04-25T11:42:30.684259+0000","last_became_active":"2024-04-25T11:42:28.658995+0000","last_became_peered":"2024-04-25T11:42:28.658995+0000","last_unstale":"2024-04-25T11:42:30.684259+0000","last_undegraded":"2024-04-25T11:42:30.684259+0000","last_fullsized":"2024-04-25T11:42:30.684259+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:02.180569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.685458+0000","last_change":"2024-04-25T11:42:29.689800+0000","last_active":"2024-04-25T11:42:30.685458+0000","last_peered":"2024-04-25T11:42:30.685458+0000","last_clean":"2024-04-25T11:42:30.685458+0000","last_became_active":"2024-04-25T11:42:29.689290+0000","last_became_peered":"2024-04-25T11:42:29.689290+0000","last_unstale":"2024-04-25T11:42:30.685458+0000","last_undegraded":"2024-04-25T11:42:30.685458+0000","last_fullsized":"2024-04-25T11:42:30.685458+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:28.608082+0000","last_clean_scrub_stamp":"2024-04-25T11:42:28.608082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:14.926190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.759209+0000","last_change":"2024-04-25T11:42:28.651803+0000","last_active":"2024-04-25T11:42:30.759209+0000","last_peered":"2024-04-25T11:42:30.759209+0000","last_clean":"2024-04-25T11:42:30.759209+0000","last_became_active":"2024-04-25T11:42:28.650621+0000","last_became_peered":"2024-04-25T11:42:28.650621+0000","last_unstale":"2024-04-25T11:42:30.759209+0000","last_undegraded":"2024-04-25T11:42:30.759209+0000","last_fullsized":"2024-04-25T11:42:30.759209+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:42:27.591742+0000","last_clean_scrub_stamp":"2024-04-25T11:42:27.591742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:41.866541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":350,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T11:42:30.756217+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:42:30.756217+0000","last_peered":"2024-04-25T11:42:30.756217+0000","last_clean":"2024-04-25T11:42:30.756217+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:42:30.756217+0000","last_undegraded":"2024-04-25T11:42:30.756217+0000","last_fullsized":"2024-04-25T11:42:30.756217+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509626,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37132,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334708,"statfs":{"total":96636764160,"available":96598740992,"internally_reserved":0,"allocated":1830912,"data_stored":880729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509627,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41288,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330552,"statfs":{"total":96636764160,"available":96594485248,"internally_reserved":0,"allocated":1826816,"data_stored":880147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509627,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1830912,"data_stored":880729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509628,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41300,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330540,"statfs":{"total":96636764160,"available":96594472960,"internally_reserved":0,"allocated":1839104,"data_stored":880341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509628,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1830912,"data_stored":880217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509628,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37728,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334112,"statfs":{"total":96636764160,"available":96598130688,"internally_reserved":0,"allocated":2441216,"data_stored":1472239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509627,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1826816,"data_stored":880147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542330,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37728,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334112,"statfs":{"total":96636764160,"available":96598130688,"internally_reserved":0,"allocated":2441216,"data_stored":1472183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:42:35.040 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:42:35.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:42:35.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.382+0000 7f7bf2600640 1 Processor -- start 2024-04-25T11:42:35.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.382+0000 7f7bf2600640 1 -- start start 2024-04-25T11:42:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf2600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bec06db10 0x7f7bec0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf2600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bec06df20 con 0x7f7bec0ce520 2024-04-25T11:42:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf2600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7bec0cda60 con 0x7f7bec06db10 2024-04-25T11:42:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec15dfe0 con 0x7f7bec0ce930 2024-04-25T11:42:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf1c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bec06db10 0x7f7bec0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf1c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bec06db10 0x7f7bec0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54942/0 (socket says 172.21.15.19:54942) 2024-04-25T11:42:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bf1c00640 1 -- 172.21.15.19:0/3256350303 learned_addr learned my addr 172.21.15.19:0/3256350303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.385+0000 7f7bebe00640 1 --2- 172.21.15.19:0/3256350303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.386+0000 7f7bf1c00640 1 -- 172.21.15.19:0/3256350303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 msgr2=0x7f7bec06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.386+0000 7f7bf1c00640 1 --2- 172.21.15.19:0/3256350303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.386+0000 7f7bf1c00640 1 -- 172.21.15.19:0/3256350303 >> v1:172.21.15.169:6789/0 conn(0x7f7bec0ce520 legacy=0x7f7bec15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.386+0000 7f7bf1c00640 1 -- 172.21.15.19:0/3256350303 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec15e2c0 con 0x7f7bec06db10 2024-04-25T11:42:35.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.387+0000 7f7bf1c00640 1 --2- 172.21.15.19:0/3256350303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec06db10 0x7f7bec0d00e0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009e50 tx=0x7f7be402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf1f12c425a8f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.388+0000 7f7beb400640 1 -- 172.21.15.19:0/3256350303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4036070 con 0x7f7bec06db10 2024-04-25T11:42:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.388+0000 7f7beb400640 1 -- 172.21.15.19:0/3256350303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be4031600 con 0x7f7bec06db10 2024-04-25T11:42:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.388+0000 7f7beb400640 1 -- 172.21.15.19:0/3256350303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be40319d0 con 0x7f7bec06db10 2024-04-25T11:42:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.388+0000 7f7bf2600640 1 -- 172.21.15.19:0/3256350303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec06db10 msgr2=0x7f7bec0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.388+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3256350303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec06db10 0x7f7bec0d00e0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009e50 tx=0x7f7be402a680 comp rx=0 tx=0).stop 2024-04-25T11:42:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.389+0000 7f7bf2600640 1 -- 172.21.15.19:0/3256350303 shutdown_connections 2024-04-25T11:42:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.389+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3256350303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec06db10 0x7f7bec0d00e0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.389+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3256350303 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.389+0000 7f7bf2600640 1 -- 172.21.15.19:0/3256350303 >> 172.21.15.19:0/3256350303 conn(0x7f7bec072f40 msgr2=0x7f7bec073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.389+0000 7f7bf2600640 1 -- 172.21.15.19:0/3256350303 shutdown_connections 2024-04-25T11:42:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.389+0000 7f7bf2600640 1 -- 172.21.15.19:0/3256350303 wait complete. 2024-04-25T11:42:35.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.391+0000 7f7bf2600640 1 Processor -- start 2024-04-25T11:42:35.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.391+0000 7f7bf2600640 1 -- start start 2024-04-25T11:42:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bec06db10 0x7f7bec0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec0ce520 0x7f7bec13bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec13c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec160d30 con 0x7f7bec0ce520 2024-04-25T11:42:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec0ce000 con 0x7f7bec06db10 2024-04-25T11:42:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec15dbc0 con 0x7f7bec0ce930 2024-04-25T11:42:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec0ce520 0x7f7bec13bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec0ce520 0x7f7bec13bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54958/0 (socket says 172.21.15.19:54958) 2024-04-25T11:42:35.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bebe00640 1 -- 172.21.15.19:0/3888975232 learned_addr learned my addr 172.21.15.19:0/3888975232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf1c00640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec13c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.393+0000 7f7bf1200640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bec06db10 0x7f7bec0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bebe00640 1 -- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 msgr2=0x7f7bec13c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bebe00640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec13c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bebe00640 1 -- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bec06db10 msgr2=0x7f7bec0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bebe00640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bec06db10 0x7f7bec0b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bebe00640 1 -- 172.21.15.19:0/3888975232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec0b16f0 con 0x7f7bec0ce520 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bf1200640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bec06db10 0x7f7bec0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:35.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bf1c00640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec13c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:35.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bebe00640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec0ce520 0x7f7bec13bf50 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800d5d0 tx=0x7f7bd800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:35.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7be8c00640 1 -- 172.21.15.19:0/3888975232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80125f0 con 0x7f7bec0ce520 2024-04-25T11:42:35.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7be8c00640 1 -- 172.21.15.19:0/3888975232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd8019070 con 0x7f7bec0ce520 2024-04-25T11:42:35.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7be8c00640 1 -- 172.21.15.19:0/3888975232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd80129f0 con 0x7f7bec0ce520 2024-04-25T11:42:35.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.394+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bec0b19d0 con 0x7f7bec0ce520 2024-04-25T11:42:35.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.395+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bec0b1f90 con 0x7f7bec0ce520 2024-04-25T11:42:35.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.396+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bec13ca60 con 0x7f7bec0ce520 2024-04-25T11:42:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.397+0000 7f7be8c00640 1 -- 172.21.15.19:0/3888975232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7bd8012b90 con 0x7f7bec0ce520 2024-04-25T11:42:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.398+0000 7f7be8c00640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.398+0000 7f7be8c00640 1 -- 172.21.15.19:0/3888975232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f7bd80956d0 con 0x7f7bec0ce520 2024-04-25T11:42:35.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.399+0000 7f7bf1200640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.400+0000 7f7bf1200640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000f60 tx=0x7f7be0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:35.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.403+0000 7f7be8c00640 1 -- 172.21.15.19:0/3888975232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7bd805fe50 con 0x7f7bec0ce520 2024-04-25T11:42:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.678+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7bec076c60 con 0x7f7bec0ce520 2024-04-25T11:42:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.684+0000 7f7be8c00640 1 -- 172.21.15.19:0/3888975232 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f7bd805f750 con 0x7f7bec0ce520 2024-04-25T11:42:35.671 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T11:42:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDQPSpmfKtVARAALpGaYT6uUWkctQPGcMT/2w== 2024-04-25T11:42:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-04-25T11:42:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-04-25T11:42:35.671 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:42:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7bcc0744b0 msgr2=0x7f7bcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000f60 tx=0x7f7be0009040 comp rx=0 tx=0).stop 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec0ce520 msgr2=0x7f7bec13bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec0ce520 0x7f7bec13bf50 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800d5d0 tx=0x7f7bd800da90 comp rx=0 tx=0).stop 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 shutdown_connections 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7bcc0744b0 0x7f7bcc0769c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7bec0ce930 0x7f7bec13c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bec0ce520 0x7f7bec13bf50 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 --2- 172.21.15.19:0/3888975232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7bec06db10 0x7f7bec0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.686+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 >> 172.21.15.19:0/3888975232 conn(0x7f7bec072f40 msgr2=0x7f7bec15d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:35.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.687+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 shutdown_connections 2024-04-25T11:42:35.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.687+0000 7f7bf2600640 1 -- 172.21.15.19:0/3888975232 wait complete. 2024-04-25T11:42:35.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.988+0000 7f7a95a00640 1 Processor -- start 2024-04-25T11:42:35.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.988+0000 7f7a95a00640 1 -- start start 2024-04-25T11:42:35.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.990+0000 7f7a95a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7a901434d0 0x7f7a90070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.990+0000 7f7a95a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a901612f0 0x7f7a900d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.990+0000 7f7a95a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a900709c0 con 0x7f7a900d0ed0 2024-04-25T11:42:35.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.990+0000 7f7a95a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7a901438e0 con 0x7f7a901434d0 2024-04-25T11:42:35.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.990+0000 7f7a95a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9006f3e0 con 0x7f7a901612f0 2024-04-25T11:42:35.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.990+0000 7f7a8e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7a900d0ed0 0x7f7a90160be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50834/0 (socket says 172.21.15.19:50834) 2024-04-25T11:42:35.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.990+0000 7f7a8e000640 1 -- 172.21.15.19:0/2906538036 learned_addr learned my addr 172.21.15.19:0/2906538036 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:35.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8f400640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a901612f0 0x7f7a900d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/2906538036 >> v2:172.21.15.169:3300/0 conn(0x7f7a901434d0 0x7f7a90070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147504680 0 0) 0x7f7a900709c0 con 0x7f7a900d0ed0 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a74003520 con 0x7f7a900d0ed0 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3058902297 0 0) 0x7f7a74003520 con 0x7f7a900d0ed0 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a901612f0 msgr2=0x7f7a900d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a901612f0 0x7f7a900d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 >> v2:172.21.15.169:3300/0 conn(0x7f7a901434d0 msgr2=0x7f7a90070480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 --2- 172.21.15.19:0/2906538036 >> v2:172.21.15.169:3300/0 conn(0x7f7a901434d0 0x7f7a90070480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9006f6c0 con 0x7f7a900d0ed0 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f7a78001670 con 0x7f7a900d0ed0 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.991+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 >> v1:172.21.15.19:6789/0 conn(0x7f7a900d0ed0 legacy=0x7f7a90160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8d600640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a74003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8d600640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a74004800 0x7f7a74004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8d600640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a74005150 0x7f7a7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a901438e0 con 0x7f7a740039b0 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9006f3e0 con 0x7f7a74004800 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7400bf00 con 0x7f7a74005150 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a74003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8ea00640 1 -- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a74005150 msgr2=0x7f7a7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a74005150 0x7f7a7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8ea00640 1 -- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a74004800 msgr2=0x7f7a74004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a74004800 0x7f7a74004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.992+0000 7f7a8ea00640 1 -- 172.21.15.19:0/2906538036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a74003520 con 0x7f7a740039b0 2024-04-25T11:42:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.993+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a74003e20 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009fc0 tx=0x7f7a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ad19096fdad618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80009c10 con 0x7f7a740039b0 2024-04-25T11:42:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8002e640 con 0x7f7a740039b0 2024-04-25T11:42:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a8d600640 1 -- 172.21.15.19:0/2906538036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8002ea30 con 0x7f7a740039b0 2024-04-25T11:42:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a95a00640 1 -- 172.21.15.19:0/2906538036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 msgr2=0x7f7a74003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a95a00640 1 --2- 172.21.15.19:0/2906538036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a74003e20 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009fc0 tx=0x7f7a74005830 comp rx=0 tx=0).stop 2024-04-25T11:42:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a8f400640 1 -- 172.21.15.19:0/2906538036 reap_dead start 2024-04-25T11:42:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a95a00640 1 -- 172.21.15.19:0/2906538036 shutdown_connections 2024-04-25T11:42:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.994+0000 7f7a95a00640 1 -- 172.21.15.19:0/2906538036 >> 172.21.15.19:0/2906538036 conn(0x7f7a900d1ae0 msgr2=0x7f7a900d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.995+0000 7f7a95a00640 1 -- 172.21.15.19:0/2906538036 shutdown_connections 2024-04-25T11:42:35.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.995+0000 7f7a95a00640 1 -- 172.21.15.19:0/2906538036 wait complete. 2024-04-25T11:42:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.997+0000 7f7a95a00640 1 Processor -- start 2024-04-25T11:42:35.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.997+0000 7f7a95a00640 1 -- start start 2024-04-25T11:42:35.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a95a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a90143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a95a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a90134d30 0x7f7a900b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a95a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9015f2d0 0x7f7a90134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a95a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9006f090 con 0x7f7a740039b0 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a95a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9006fc80 con 0x7f7a9015f2d0 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a95a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7a901612f0 con 0x7f7a90134d30 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a90143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a90143a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54980/0 (socket says 172.21.15.19:54980) 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 -- 172.21.15.19:0/4073954977 learned_addr learned my addr 172.21.15.19:0/4073954977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 -- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a90134d30 msgr2=0x7f7a900b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8e000640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a90134d30 0x7f7a900b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8f400640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9015f2d0 0x7f7a90134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a90134d30 0x7f7a900b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 -- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9015f2d0 msgr2=0x7f7a90134050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:35.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9015f2d0 0x7f7a90134050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:35.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:35.999+0000 7f7a8ea00640 1 -- 172.21.15.19:0/4073954977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a901440d0 con 0x7f7a740039b0 2024-04-25T11:42:35.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.000+0000 7f7a8ea00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a90143a60 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002d120 tx=0x7f7a80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:35.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.000+0000 7f7a86a00640 1 -- 172.21.15.19:0/4073954977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8002e440 con 0x7f7a740039b0 2024-04-25T11:42:35.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.000+0000 7f7a86a00640 1 -- 172.21.15.19:0/4073954977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8002ecf0 con 0x7f7a740039b0 2024-04-25T11:42:35.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.000+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9015e850 con 0x7f7a740039b0 2024-04-25T11:42:35.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.000+0000 7f7a86a00640 1 -- 172.21.15.19:0/4073954977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80003bd0 con 0x7f7a740039b0 2024-04-25T11:42:35.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.000+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a90146a40 con 0x7f7a740039b0 2024-04-25T11:42:35.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.001+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a90146c40 con 0x7f7a740039b0 2024-04-25T11:42:35.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.003+0000 7f7a86a00640 1 -- 172.21.15.19:0/4073954977 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a8004a020 con 0x7f7a740039b0 2024-04-25T11:42:35.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.004+0000 7f7a86a00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a6c0744b0 0x7f7a6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:35.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.004+0000 7f7a86a00640 1 -- 172.21.15.19:0/4073954977 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f7a800b5b50 con 0x7f7a740039b0 2024-04-25T11:42:35.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.005+0000 7f7a8e000640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a6c0744b0 0x7f7a6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:35.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.007+0000 7f7a8e000640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a6c0744b0 0x7f7a6c0769c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7a901600e0 tx=0x7f7a78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:35.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.008+0000 7f7a86a00640 1 -- 172.21.15.19:0/4073954977 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a9015e850 con 0x7f7a740039b0 2024-04-25T11:42:36.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.241+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a90140540 con 0x7f7a740039b0 2024-04-25T11:42:36.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.242+0000 7f7a86a00640 1 -- 172.21.15.19:0/4073954977 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 364 v364) ==== 95+0+5195 (secure 0 0 0) 0x7f7a80010040 con 0x7f7a740039b0 2024-04-25T11:42:36.230 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:36.230 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":364,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":8020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":358}],"filesystems":[{"mdsmap":{"epoch":364,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:32.307292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8032},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8032":{"gid":8032,"name":"b","rank":0,"incarnation":360,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2610059988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2610059988},{"type":"v1","addr":"172.21.15.169:6835","nonce":2610059988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":7}]} 2024-04-25T11:42:36.230 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 364 2024-04-25T11:42:36.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a6c0744b0 msgr2=0x7f7a6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a6c0744b0 0x7f7a6c0769c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7a901600e0 tx=0x7f7a78009040 comp rx=0 tx=0).stop 2024-04-25T11:42:36.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 msgr2=0x7f7a90143a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a90143a60 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002d120 tx=0x7f7a80009750 comp rx=0 tx=0).stop 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 shutdown_connections 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7a6c0744b0 0x7f7a6c0769c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a9015f2d0 0x7f7a90134050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a90134d30 0x7f7a900b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 --2- 172.21.15.19:0/4073954977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a740039b0 0x7f7a90143a60 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 >> 172.21.15.19:0/4073954977 conn(0x7f7a900d1ae0 msgr2=0x7f7a9013fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 shutdown_connections 2024-04-25T11:42:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.245+0000 7f7a95a00640 1 -- 172.21.15.19:0/4073954977 wait complete. 2024-04-25T11:42:36.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 364, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:42:29.193619+0000', 'modified': '2024-04-25T11:42:32.307292+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8032}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8032': {'gid': 8032, 'name': 'b', 'rank': 0, 'incarnation': 360, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/2610059988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 2610059988}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2610059988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8032, 'qdb_cluster': [8032]} 2024-04-25T11:42:36.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:42:36.662 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:42:36.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.966+0000 7fb8cb200640 1 Processor -- start 2024-04-25T11:42:36.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.966+0000 7fb8cb200640 1 -- start start 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.968+0000 7fb8cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.968+0000 7fb8cb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb8c40b71d0 0x7fb8c40b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.968+0000 7fb8cb200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c40d28c0 con 0x7fb8c40b9050 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.968+0000 7fb8cb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c40b8da0 con 0x7fb8c40b71d0 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.968+0000 7fb8cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c40d2ba0 con 0x7fb8c40b8990 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8ca800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb8c40b71d0 0x7fb8c40b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8c9e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb8c40b9050 0x7fb8c40ce9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50840/0 (socket says 172.21.15.19:50840) 2024-04-25T11:42:36.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8c9e00640 1 -- 172.21.15.19:0/2103762567 learned_addr learned my addr 172.21.15.19:0/2103762567 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:36.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8c9400640 1 --2- 172.21.15.19:0/2103762567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8ca800640 1 -- 172.21.15.19:0/2103762567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 msgr2=0x7fb8c40cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:36.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8ca800640 1 --2- 172.21.15.19:0/2103762567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8ca800640 1 -- 172.21.15.19:0/2103762567 >> v1:172.21.15.19:6789/0 conn(0x7fb8c40b9050 legacy=0x7fb8c40ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8ca800640 1 -- 172.21.15.19:0/2103762567 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c409cc50 con 0x7fb8c40b71d0 2024-04-25T11:42:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.969+0000 7fb8c9400640 1 --2- 172.21.15.19:0/2103762567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:36.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.970+0000 7fb8ca800640 1 --2- 172.21.15.19:0/2103762567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c40b75e0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009d90 tx=0x7fb8c002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8692e63012cb18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.971+0000 7fb8c8a00640 1 -- 172.21.15.19:0/2103762567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007740 con 0x7fb8c40b71d0 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.971+0000 7fb8c8a00640 1 -- 172.21.15.19:0/2103762567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c00078e0 con 0x7fb8c40b71d0 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.971+0000 7fb8c8a00640 1 -- 172.21.15.19:0/2103762567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007cb0 con 0x7fb8c40b71d0 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.971+0000 7fb8cb200640 1 -- 172.21.15.19:0/2103762567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 msgr2=0x7fb8c40b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.971+0000 7fb8cb200640 1 --2- 172.21.15.19:0/2103762567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c40b75e0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009d90 tx=0x7fb8c002a660 comp rx=0 tx=0).stop 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.972+0000 7fb8cb200640 1 -- 172.21.15.19:0/2103762567 shutdown_connections 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.972+0000 7fb8cb200640 1 --2- 172.21.15.19:0/2103762567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c40b75e0 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.972+0000 7fb8cb200640 1 --2- 172.21.15.19:0/2103762567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.972+0000 7fb8cb200640 1 -- 172.21.15.19:0/2103762567 >> 172.21.15.19:0/2103762567 conn(0x7fb8c40d03f0 msgr2=0x7fb8c40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:36.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.972+0000 7fb8cb200640 1 -- 172.21.15.19:0/2103762567 shutdown_connections 2024-04-25T11:42:36.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.972+0000 7fb8cb200640 1 -- 172.21.15.19:0/2103762567 wait complete. 2024-04-25T11:42:36.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.974+0000 7fb8cb200640 1 Processor -- start 2024-04-25T11:42:36.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.974+0000 7fb8cb200640 1 -- start start 2024-04-25T11:42:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8cb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c41d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8cb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c40b9050 0x7fb8c41d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8cb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c40d2550 con 0x7fb8c40b71d0 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8cb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c40cf0b0 con 0x7fb8c40b9050 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c40b7b20 con 0x7fb8c40b8990 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c41d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c41d65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54994/0 (socket says 172.21.15.19:54994) 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3292054284 learned_addr learned my addr 172.21.15.19:0/3292054284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8ca800640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c40b9050 0x7fb8c41d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8c9400640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.976+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 msgr2=0x7fb8c40b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c40b9050 msgr2=0x7fb8c41d59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c40b9050 0x7fb8c41d59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3292054284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c40b19a0 con 0x7fb8c40b71d0 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8ca800640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c40b9050 0x7fb8c41d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c41d65d0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c3f0 tx=0x7fb8b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:36.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8be000640 1 -- 172.21.15.19:0/3292054284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8017070 con 0x7fb8c40b71d0 2024-04-25T11:42:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8be000640 1 -- 172.21.15.19:0/3292054284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b80074e0 con 0x7fb8c40b71d0 2024-04-25T11:42:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c40b1c80 con 0x7fb8c40b71d0 2024-04-25T11:42:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8be000640 1 -- 172.21.15.19:0/3292054284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80059a0 con 0x7fb8c40b71d0 2024-04-25T11:42:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.977+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c40b2240 con 0x7fb8c40b71d0 2024-04-25T11:42:36.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.980+0000 7fb8be000640 1 -- 172.21.15.19:0/3292054284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8b801a070 con 0x7fb8c40b71d0 2024-04-25T11:42:36.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.980+0000 7fb8be000640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8a40788c0 0x7fb8a407add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.981+0000 7fb8be000640 1 -- 172.21.15.19:0/3292054284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7fb8b80943c0 con 0x7fb8c40b71d0 2024-04-25T11:42:36.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.981+0000 7fb8c9400640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8a40788c0 0x7fb8a407add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.982+0000 7fb8d3820740 1 Processor -- start 2024-04-25T11:42:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.982+0000 7fb8d3820740 1 -- start start 2024-04-25T11:42:36.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.982+0000 7fb8c9400640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8a40788c0 0x7fb8a407add0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb8c41f7c80 tx=0x7fb8b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.983+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff167210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.983+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff181f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.983+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff182480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.983+0000 7fb8d3820740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e5ff0a0600 con 0x55e5fed9a270 2024-04-25T11:42:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.983+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e5ff03d100 con 0x55e5fed713b0 2024-04-25T11:42:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.983+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e5ff09c5c0 con 0x55e5fed79590 2024-04-25T11:42:36.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff167210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff167210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55010/0 (socket says 172.21.15.19:55010) 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3139171664 learned_addr learned my addr 172.21.15.19:0/3139171664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3139171664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 msgr2=0x55e5ff182480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff182480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9400640 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff181f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3139171664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 msgr2=0x55e5ff181f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff181f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 -- 172.21.15.19:0/3139171664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5fedb22d0 con 0x55e5fed9a270 2024-04-25T11:42:36.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff167210 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8007790 tx=0x7fb8b805e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1781f4924e17c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:36.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.984+0000 7fb8b6600640 1 -- 172.21.15.19:0/3139171664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8067210 con 0x55e5fed9a270 2024-04-25T11:42:36.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8b6600640 1 -- 172.21.15.19:0/3139171664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b805a780 con 0x55e5fed9a270 2024-04-25T11:42:36.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8b6600640 1 -- 172.21.15.19:0/3139171664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8005d40 con 0x55e5fed9a270 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 -- 172.21.15.19:0/3139171664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 msgr2=0x55e5ff167210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff167210 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8007790 tx=0x7fb8b805e0c0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 -- 172.21.15.19:0/3139171664 shutdown_connections 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff181f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff182480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 --2- 172.21.15.19:0/3139171664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff167210 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 -- 172.21.15.19:0/3139171664 >> 172.21.15.19:0/3139171664 conn(0x55e5fed8e3c0 msgr2=0x55e5ff13c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.985+0000 7fb8d3820740 1 -- 172.21.15.19:0/3139171664 shutdown_connections 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.986+0000 7fb8d3820740 1 -- 172.21.15.19:0/3139171664 wait complete. 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.987+0000 7fb8d3820740 1 -- start start 2024-04-25T11:42:36.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.987+0000 7fb8d3820740 1 Processor -- start 2024-04-25T11:42:36.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.988+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed713b0 0x55e5ff169f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.988+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed79590 0x55e5ff17fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.988+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff1801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.988+0000 7fb8d3820740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e5ff027f70 con 0x55e5fed79590 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.988+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e5ff17f280 con 0x55e5fed9a270 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.988+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e5ff16a9a0 con 0x55e5fed713b0 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8d3820740 10 client.0 _ll_register_callbacks cb 0x55e5ff096480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed79590 0x55e5ff17fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed79590 0x55e5ff17fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55016/0 (socket says 172.21.15.19:55016) 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 -- 172.21.15.19:0/993150304 learned_addr learned my addr 172.21.15.19:0/993150304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8ca800640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff1801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed713b0 0x55e5ff169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 -- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed713b0 msgr2=0x55e5ff169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed713b0 0x55e5ff169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 -- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 msgr2=0x55e5ff1801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff1801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9400640 1 -- 172.21.15.19:0/993150304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5fee7c6b0 con 0x55e5fed79590 2024-04-25T11:42:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed713b0 0x55e5ff169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.989+0000 7fb8ca800640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff1801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8c9400640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed79590 0x55e5ff17fc80 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0012720 tx=0x7fb8b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c4541375ca0802 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:36.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8aaa00640 1 -- 172.21.15.19:0/993150304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b001f610 con 0x55e5fed79590 2024-04-25T11:42:36.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b805a780 con 0x55e5fed79590 2024-04-25T11:42:36.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8aaa00640 1 -- 172.21.15.19:0/993150304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7fb8b001f7b0 con 0x55e5fed79590 2024-04-25T11:42:36.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8aaa00640 1 -- 172.21.15.19:0/993150304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) ==== 24+0+0 (secure 0 0 0) 0x7fb8b002c050 con 0x55e5fed79590 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8d3820740 10 client.11514 fetch_fsmap learned FSMap version 364 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.990+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) -- 0x55e5ff17f020 con 0x55e5fed79590 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8aaa00640 1 -- 172.21.15.19:0/993150304 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b805a780 con 0x55e5fed79590 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8aaa00640 1 -- 172.21.15.19:0/993150304 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 364) ==== 3494+0+0 (secure 0 0 0) 0x7fb8b002f020 con 0x55e5fed79590 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8d3820740 10 client.11514 fetch_fsmap finished waiting for FSMap version 364 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8d3820740 10 client.11514 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8d3820740 20 client.11514 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8d3820740 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5fed58e40 0x55e5ff17e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8d3820740 4 client.11514 mds_command: new command op to 8032 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 --> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ff0ae210 con 0x55e5fed58e40 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.991+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5fed58e40 0x55e5ff17e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.992+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5fed58e40 0x55e5ff17e5c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f68a5f6214f16a server_cookie=a4f401078b11d90f in_seq=0 out_seq=0 2024-04-25T11:42:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.992+0000 7fb8aaa00640 10 client.11514 ms_handle_connect on v2:172.21.15.169:6834/2610059988 2024-04-25T11:42:36.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8aaa00640 1 -- 172.21.15.19:0/993150304 <== mds.0 v2:172.21.15.169:6834/2610059988 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e5ff0ae210 con 0x55e5fed58e40 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8aaa00640 10 client.11514 handle_command_reply: tid=0 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 client.11514 shutdown 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed79590 msgr2=0x55e5ff17fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed79590 0x55e5ff17fc80 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0012720 tx=0x7fb8b0004730 comp rx=0 tx=0).stop 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 shutdown_connections 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5fed58e40 0x55e5ff17e5c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff1801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed79590 0x55e5ff17fc80 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 --2- 172.21.15.19:0/993150304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed713b0 0x55e5ff169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8aaa00640 0 client.11514 ms_handle_reset on v2:172.21.15.169:6834/2610059988 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 >> 172.21.15.19:0/993150304 conn(0x55e5fed8e3c0 msgr2=0x55e5ff13c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 shutdown_connections 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 1 -- 172.21.15.19:0/993150304 wait complete. 2024-04-25T11:42:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:36.995+0000 7fb8d3820740 20 client.11514 trim_cache size 0 max 16384 2024-04-25T11:42:37.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.016+0000 7fb8d3820740 1 Processor -- start 2024-04-25T11:42:37.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.016+0000 7fb8d3820740 1 -- start start 2024-04-25T11:42:37.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.017+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff167210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.017+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff167750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.017+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff13c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.017+0000 7fb8d3820740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c406f580 con 0x55e5fed9a270 2024-04-25T11:42:37.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.017+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c406e8e0 con 0x55e5fed713b0 2024-04-25T11:42:37.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.017+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c406ef30 con 0x55e5fed79590 2024-04-25T11:42:37.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff13c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff13c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55030/0 (socket says 172.21.15.19:55030) 2024-04-25T11:42:37.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 -- 172.21.15.19:0/4159023902 learned_addr learned my addr 172.21.15.19:0/4159023902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:37.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff167210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8c9400640 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff167750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 -- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 msgr2=0x55e5ff167750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff167750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 -- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 msgr2=0x55e5ff167210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff167210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8ca800640 1 -- 172.21.15.19:0/4159023902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ff1833a0 con 0x55e5fed9a270 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.018+0000 7fb8c9400640 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff167750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.019+0000 7fb8ca800640 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff13c850 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009eb0 tx=0x7fb8c002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0bea033434e1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.019+0000 7fb8b5c00640 1 -- 172.21.15.19:0/4159023902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007900 con 0x55e5fed9a270 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.019+0000 7fb8b5c00640 1 -- 172.21.15.19:0/4159023902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c0007aa0 con 0x55e5fed9a270 2024-04-25T11:42:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.019+0000 7fb8b5c00640 1 -- 172.21.15.19:0/4159023902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007e70 con 0x55e5fed9a270 2024-04-25T11:42:37.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 -- 172.21.15.19:0/4159023902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 msgr2=0x55e5ff13c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff13c850 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009eb0 tx=0x7fb8c002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:37.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 -- 172.21.15.19:0/4159023902 shutdown_connections 2024-04-25T11:42:37.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed9a270 0x55e5ff13c850 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff167750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 --2- 172.21.15.19:0/4159023902 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed713b0 0x55e5ff167210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 -- 172.21.15.19:0/4159023902 >> 172.21.15.19:0/4159023902 conn(0x55e5fed8e3c0 msgr2=0x55e5ff1838d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:37.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 -- 172.21.15.19:0/4159023902 shutdown_connections 2024-04-25T11:42:37.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.020+0000 7fb8d3820740 1 -- 172.21.15.19:0/4159023902 wait complete. 2024-04-25T11:42:37.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.021+0000 7fb8d3820740 1 -- start start 2024-04-25T11:42:37.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.021+0000 7fb8d3820740 1 Processor -- start 2024-04-25T11:42:37.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed713b0 0x55e5ff1e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff17f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8d3820740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff17fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8d3820740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e5ff16a690 con 0x55e5fed713b0 2024-04-25T11:42:37.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55e5ff155cc0 con 0x55e5fed9a270 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8d3820740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55e5ff1824b0 con 0x55e5fed79590 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8d3820740 10 client.0 _ll_register_callbacks cb 0x7fb8b000b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed713b0 0x55e5ff1e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed713b0 0x55e5ff1e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55036/0 (socket says 172.21.15.19:55036) 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9e00640 1 -- 172.21.15.19:0/1006421366 learned_addr learned my addr 172.21.15.19:0/1006421366 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8ca800640 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff17fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9400640 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff17f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9e00640 1 -- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 msgr2=0x55e5ff17f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff17f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9e00640 1 -- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 msgr2=0x55e5ff17fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.023+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff17fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8c9e00640 1 -- 172.21.15.19:0/1006421366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ff1378f0 con 0x55e5fed713b0 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8c9e00640 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed713b0 0x55e5ff1e9330 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fb8b805e100 tx=0x7fb8b801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1818b9b3a2287d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8aaa00640 1 -- 172.21.15.19:0/1006421366 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b805ab40 con 0x55e5fed713b0 2024-04-25T11:42:37.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8aaa00640 1 -- 172.21.15.19:0/1006421366 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7fb8b8092290 con 0x55e5fed713b0 2024-04-25T11:42:37.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c0007aa0 con 0x55e5fed713b0 2024-04-25T11:42:37.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8aaa00640 1 -- 172.21.15.19:0/1006421366 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=364) ==== 24+0+0 (secure 0 0 0) 0x7fb8b805a780 con 0x55e5fed713b0 2024-04-25T11:42:37.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8d3820740 10 client.11520 fetch_fsmap learned FSMap version 364 2024-04-25T11:42:37.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.024+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=364,monmap=2+}) -- 0x55e5ff167720 con 0x55e5fed713b0 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8aaa00640 1 -- 172.21.15.19:0/1006421366 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b807fe50 con 0x55e5fed713b0 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8aaa00640 1 -- 172.21.15.19:0/1006421366 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 364) ==== 3494+0+0 (secure 0 0 0) 0x7fb8b8005b40 con 0x55e5fed713b0 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8d3820740 10 client.11520 fetch_fsmap finished waiting for FSMap version 364 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8d3820740 10 client.11520 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8d3820740 20 client.11520 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8d3820740 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5ff136990 0x55e5ff200240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8d3820740 4 client.11520 mds_command: new command op to 8032 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:42:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 --> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] -- command(tid 0: {"prefix": "status"}) -- 0x55e5ff0a8080 con 0x55e5ff136990 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.025+0000 7fb8c9400640 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5ff136990 0x55e5ff200240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.026+0000 7fb8aaa00640 10 client.11520 ms_handle_connect on v2:172.21.15.169:6834/2610059988 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.026+0000 7fb8c9400640 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5ff136990 0x55e5ff200240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aeae7e95c95e009 server_cookie=89bcf2c600dbd810 in_seq=0 out_seq=0 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8aaa00640 1 -- 172.21.15.19:0/1006421366 <== mds.0 v2:172.21.15.169:6834/2610059988 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55e5ff0a8080 con 0x55e5ff136990 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8aaa00640 10 client.11520 handle_command_reply: tid=0 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 client.11520 shutdown 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed713b0 msgr2=0x55e5ff1e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed713b0 0x55e5ff1e9330 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7fb8b805e100 tx=0x7fb8b801e040 comp rx=0 tx=0).stop 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 shutdown_connections 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:6834/2610059988,v1:172.21.15.169:6835/2610059988] conn(0x55e5ff136990 0x55e5ff200240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e5fed9a270 0x55e5ff17fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e5fed79590 0x55e5ff17f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8aaa00640 0 client.11520 ms_handle_reset on v2:172.21.15.169:6834/2610059988 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 --2- 172.21.15.19:0/1006421366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e5fed713b0 0x55e5ff1e9330 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 >> 172.21.15.19:0/1006421366 conn(0x55e5fed8e3c0 msgr2=0x55e5ff1838d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 shutdown_connections 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.027+0000 7fb8d3820740 1 -- 172.21.15.19:0/1006421366 wait complete. 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8d3820740 20 client.11520 trim_cache size 0 max 16384 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8032, 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:42:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 7.3467811210000002, 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 364, 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 143, 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 143, 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.502620160999999 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8a40788c0 msgr2=0x7fb8a407add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8a40788c0 0x7fb8a407add0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb8c41f7c80 tx=0x7fb8b0008040 comp rx=0 tx=0).stop 2024-04-25T11:42:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 msgr2=0x7fb8c41d65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c41d65d0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c3f0 tx=0x7fb8b800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 shutdown_connections 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb8a40788c0 0x7fb8a407add0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8c40b9050 0x7fb8c41d59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8c40b8990 0x7fb8c40b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 --2- 172.21.15.19:0/3292054284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb8c40b71d0 0x7fb8c41d65d0 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 >> 172.21.15.19:0/3292054284 conn(0x7fb8c40d03f0 msgr2=0x7fb8c40b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 shutdown_connections 2024-04-25T11:42:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.028+0000 7fb8cb200640 1 -- 172.21.15.19:0/3292054284 wait complete. 2024-04-25T11:42:37.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:42:37.166 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:42:37.174 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:42:37.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.335+0000 7fbbcb200640 1 Processor -- start 2024-04-25T11:42:37.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.335+0000 7fbbcb200640 1 -- start start 2024-04-25T11:42:37.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.337+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc140e00 0x7fbbcc143550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.337+0000 7fbbcb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbcc06dce0 0x7fbbcc141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.337+0000 7fbbcb200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc141210 con 0x7fbbcc143a90 2024-04-25T11:42:37.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.337+0000 7fbbcb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbbcc141b40 con 0x7fbbcc06dce0 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.337+0000 7fbbcb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc143ea0 con 0x7fbbcc140e00 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbca800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbcc06dce0 0x7fbbcc141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbc3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbbcc143a90 0x7fbbcc06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50850/0 (socket says 172.21.15.19:50850) 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbc3e00640 1 -- 172.21.15.19:0/966402084 learned_addr learned my addr 172.21.15.19:0/966402084 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbc9e00640 1 --2- 172.21.15.19:0/966402084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc140e00 0x7fbbcc143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbc9400640 1 -- 172.21.15.19:0/966402084 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1247172357 0 0) 0x7fbbcc141210 con 0x7fbbcc143a90 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbc9400640 1 -- 172.21.15.19:0/966402084 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbb0003520 con 0x7fbbcc143a90 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbca800640 1 -- 172.21.15.19:0/966402084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc140e00 msgr2=0x7fbbcc143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbca800640 1 --2- 172.21.15.19:0/966402084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc140e00 0x7fbbcc143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbca800640 1 -- 172.21.15.19:0/966402084 >> v1:172.21.15.19:6789/0 conn(0x7fbbcc143a90 legacy=0x7fbbcc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.338+0000 7fbbca800640 1 -- 172.21.15.19:0/966402084 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc141510 con 0x7fbbcc06dce0 2024-04-25T11:42:37.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.339+0000 7fbbca800640 1 --2- 172.21.15.19:0/966402084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc06dce0 0x7fbbcc141fd0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009d90 tx=0x7fbbc402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45e0c402377004a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:37.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.340+0000 7fbbc9400640 1 -- 172.21.15.19:0/966402084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4007740 con 0x7fbbcc06dce0 2024-04-25T11:42:37.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.340+0000 7fbbc9400640 1 -- 172.21.15.19:0/966402084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc40078e0 con 0x7fbbcc06dce0 2024-04-25T11:42:37.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.340+0000 7fbbc9400640 1 -- 172.21.15.19:0/966402084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4007c90 con 0x7fbbcc06dce0 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.340+0000 7fbbcb200640 1 -- 172.21.15.19:0/966402084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc06dce0 msgr2=0x7fbbcc141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.340+0000 7fbbcb200640 1 --2- 172.21.15.19:0/966402084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc06dce0 0x7fbbcc141fd0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009d90 tx=0x7fbbc402a660 comp rx=0 tx=0).stop 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.341+0000 7fbbcb200640 1 -- 172.21.15.19:0/966402084 shutdown_connections 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.341+0000 7fbbcb200640 1 --2- 172.21.15.19:0/966402084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc06dce0 0x7fbbcc141fd0 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.341+0000 7fbbcb200640 1 --2- 172.21.15.19:0/966402084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc140e00 0x7fbbcc143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.341+0000 7fbbcb200640 1 -- 172.21.15.19:0/966402084 >> 172.21.15.19:0/966402084 conn(0x7fbbcc072ee0 msgr2=0x7fbbcc0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.341+0000 7fbbcb200640 1 -- 172.21.15.19:0/966402084 shutdown_connections 2024-04-25T11:42:37.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.341+0000 7fbbcb200640 1 -- 172.21.15.19:0/966402084 wait complete. 2024-04-25T11:42:37.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.343+0000 7fbbcb200640 1 Processor -- start 2024-04-25T11:42:37.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.343+0000 7fbbcb200640 1 -- start start 2024-04-25T11:42:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc140e00 0x7fbbcc15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc143a90 0x7fbbcc12ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbcb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc15d4d0 0x7fbbcc12efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbcb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1417d0 con 0x7fbbcc143a90 2024-04-25T11:42:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbcb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc1443d0 con 0x7fbbcc140e00 2024-04-25T11:42:37.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbcb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc0d1490 con 0x7fbbcc15d4d0 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc143a90 0x7fbbcc12ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc143a90 0x7fbbcc12ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55062/0 (socket says 172.21.15.19:55062) 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbc3e00640 1 -- 172.21.15.19:0/3127828469 learned_addr learned my addr 172.21.15.19:0/3127828469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.345+0000 7fbbc9e00640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc140e00 0x7fbbcc15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbca800640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc15d4d0 0x7fbbcc12efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc3e00640 1 -- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc15d4d0 msgr2=0x7fbbcc12efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc3e00640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc15d4d0 0x7fbbcc12efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc3e00640 1 -- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc140e00 msgr2=0x7fbbcc15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc3e00640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc140e00 0x7fbbcc15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc3e00640 1 -- 172.21.15.19:0/3127828469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc157e00 con 0x7fbbcc143a90 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbca800640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc15d4d0 0x7fbbcc12efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:37.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc3e00640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc143a90 0x7fbbcc12ea90 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40010e0 tx=0x7fbbb400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc2000640 1 -- 172.21.15.19:0/3127828469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4017070 con 0x7fbbcc143a90 2024-04-25T11:42:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbc2000640 1 -- 172.21.15.19:0/3127828469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb4005d00 con 0x7fbbcc143a90 2024-04-25T11:42:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.346+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc1580e0 con 0x7fbbcc143a90 2024-04-25T11:42:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.347+0000 7fbbc2000640 1 -- 172.21.15.19:0/3127828469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4007310 con 0x7fbbcc143a90 2024-04-25T11:42:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.347+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc1583c0 con 0x7fbbcc143a90 2024-04-25T11:42:37.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.348+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbcc050740 con 0x7fbbcc143a90 2024-04-25T11:42:37.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.349+0000 7fbbc2000640 1 -- 172.21.15.19:0/3127828469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbbb401a070 con 0x7fbbcc143a90 2024-04-25T11:42:37.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.350+0000 7fbbc2000640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbba80744b0 0x7fbba80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.351+0000 7fbbc2000640 1 -- 172.21.15.19:0/3127828469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7fbbb40949f0 con 0x7fbbcc143a90 2024-04-25T11:42:37.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.351+0000 7fbbc9e00640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbba80744b0 0x7fbba80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.353+0000 7fbbc9e00640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbba80744b0 0x7fbba80769c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006fd0 tx=0x7fbbbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:37.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.354+0000 7fbbc2000640 1 -- 172.21.15.19:0/3127828469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbbb4021020 con 0x7fbbcc143a90 2024-04-25T11:42:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.606+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbbcc0b7710 con 0x7fbbcc143a90 2024-04-25T11:42:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.607+0000 7fbbc2000640 1 -- 172.21.15.19:0/3127828469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v364) ==== 53+0+73 (secure 0 0 0) 0x7fbbb405f1f0 con 0x7fbbcc143a90 2024-04-25T11:42:37.594 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.609+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbba80744b0 msgr2=0x7fbba80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.609+0000 7fbbcb200640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbba80744b0 0x7fbba80769c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006fd0 tx=0x7fbbbc008040 comp rx=0 tx=0).stop 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc143a90 msgr2=0x7fbbcc12ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc143a90 0x7fbbcc12ea90 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40010e0 tx=0x7fbbb400c890 comp rx=0 tx=0).stop 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 shutdown_connections 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fbba80744b0 0x7fbba80769c0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbbcc15d4d0 0x7fbbcc12efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbcc143a90 0x7fbbcc12ea90 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 --2- 172.21.15.19:0/3127828469 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbbcc140e00 0x7fbbcc15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 >> 172.21.15.19:0/3127828469 conn(0x7fbbcc072ee0 msgr2=0x7fbbcc070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:37.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 shutdown_connections 2024-04-25T11:42:37.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37.610+0000 7fbbcb200640 1 -- 172.21.15.19:0/3127828469 wait complete. 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:42:37.624 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:42:37.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:37.625 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:42:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:42:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:42:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:42:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:42:37.677 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:37.677 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41716sec preferred_lft 41716sec 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:42:37.678 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:42:37.679 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:42:37.680 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:37.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:37.681 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:42:37.763 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:42:37.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:37.767 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:42:37.817 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:42:37.817 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:42:37.817 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:42:37.818 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:42:37.818 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:42:37.819 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:42:37.819 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:42:37.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:42:37.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:42:37.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:42:37.819 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:42:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:42:38.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:42:38.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:38.038 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:42:38.083 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:42:38.083 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:42:38.084 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:42:38.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:38.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:42:38.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:42:38.219 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:38.274 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:42:38.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:42:38.274 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:38.327 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:42:38.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:38.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:42:38.540 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:42:38.540 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:42:38.588 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:42:38.588 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:42:38.588 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:42:38.588 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:42:38.588 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:42:38.590 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:42:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:42:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:42:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:42:38.591 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:42:38.595 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:42:38.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:38.595 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:38.675 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:42:38.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:42:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:42:38.834 INFO:teuthology.orchestra.run.smithi019.stdout:client8101 2024-04-25T11:42:38.834 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:38.834 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:38.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:38.889 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2024-04-25T11:42:38.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:38.958 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2024-04-25T11:42:39.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:39.018 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2024-04-25T11:42:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:39.080 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2024-04-25T11:42:39.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:39.152 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T11:42:39.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:39.214 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2024-04-25T11:42:39.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:39.284 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2024-04-25T11:42:39.347 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:42:39.347 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:42:39.347 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:42:39.347 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:42:39.347 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2024-04-25T11:42:39.347 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:42:39.493 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-04-25T11:42:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:42:39.507 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:42:39.507 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:42:39.507 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:42:39.507 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:42:39.507 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2024-04-25T11:42:39.507 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:42:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628279 2024-04-25T11:42:39.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:42:39.713 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-04-25T11:42:39.713 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-04-25T11:42:39.714 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-04-25T11:42:39.714 DEBUG:teuthology.orchestra.run.smithi019:> 2024-04-25T11:42:39.714 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2024-04-25T11:42:39.714 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-04-25T11:42:39.897 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628278 2024-04-25T11:42:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:42:39.912 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:39.912 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:40.013 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:40.013 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:40.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:40.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.363+0000 7f2756800640 1 Processor -- start 2024-04-25T11:42:40.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.363+0000 7f2756800640 1 -- start start 2024-04-25T11:42:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.365+0000 7f2756800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 0x7f275006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.365+0000 7f2756800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f275015f5f0 con 0x7f27500cfcf0 2024-04-25T11:42:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.365+0000 7f2756800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f275015e830 con 0x7f2750070370 2024-04-25T11:42:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.365+0000 7f2756800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f27500708d0 con 0x7f27501443c0 2024-04-25T11:42:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.366+0000 7f274fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f27500cfcf0 0x7f275006c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50862/0 (socket says 172.21.15.19:50862) 2024-04-25T11:42:40.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.366+0000 7f274fe00640 1 -- 172.21.15.19:0/3623401419 learned_addr learned my addr 172.21.15.19:0/3623401419 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.366+0000 7f274f400640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 0x7f275006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748012952 0 0) 0x7f275015f5f0 con 0x7f27500cfcf0 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2734003520 con 0x7f27500cfcf0 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1534245143 0 0) 0x7f2734003520 con 0x7f27500cfcf0 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 msgr2=0x7f275006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 0x7f275006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 >> v1:172.21.15.169:6789/0 conn(0x7f2750070370 legacy=0x7f27500cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275015eb10 con 0x7f27500cfcf0 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f27480019f0 con 0x7f27500cfcf0 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 >> v1:172.21.15.19:6789/0 conn(0x7f27500cfcf0 legacy=0x7f275006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27340039e0 0x7f2734003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2734004800 0x7f2734004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 0x7f27340052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27500708d0 con 0x7f2734004800 2024-04-25T11:42:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f273400bb50 con 0x7f27340039e0 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.367+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f273400bd50 con 0x7f27501443c0 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274ea00640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2734004800 0x7f2734004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274f400640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27340039e0 0x7f2734003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274fe00640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 0x7f27340052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274ea00640 1 -- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 msgr2=0x7f27340052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274ea00640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27501443c0 0x7f27340052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274ea00640 1 -- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27340039e0 msgr2=0x7f2734003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274ea00640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27340039e0 0x7f2734003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.368+0000 7f274ea00640 1 -- 172.21.15.19:0/3623401419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2734003520 con 0x7f2734004800 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.369+0000 7f274ea00640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2734004800 0x7f2734004c10 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f2740001720 tx=0x7f2734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2429c161d57b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.369+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740001370 con 0x7f2734004800 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.369+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27400052d0 con 0x7f2734004800 2024-04-25T11:42:40.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.369+0000 7f274e000640 1 -- 172.21.15.19:0/3623401419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274002abc0 con 0x7f2734004800 2024-04-25T11:42:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.370+0000 7f2756800640 1 -- 172.21.15.19:0/3623401419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2734004800 msgr2=0x7f2734004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.370+0000 7f2756800640 1 --2- 172.21.15.19:0/3623401419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2734004800 0x7f2734004c10 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f2740001720 tx=0x7f2734005830 comp rx=0 tx=0).stop 2024-04-25T11:42:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.370+0000 7f274fe00640 1 -- 172.21.15.19:0/3623401419 reap_dead start 2024-04-25T11:42:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.370+0000 7f2756800640 1 -- 172.21.15.19:0/3623401419 shutdown_connections 2024-04-25T11:42:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.370+0000 7f2756800640 1 -- 172.21.15.19:0/3623401419 >> 172.21.15.19:0/3623401419 conn(0x7f27500d1ae0 msgr2=0x7f27500d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.370+0000 7f2756800640 1 -- 172.21.15.19:0/3623401419 shutdown_connections 2024-04-25T11:42:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.370+0000 7f2756800640 1 -- 172.21.15.19:0/3623401419 wait complete. 2024-04-25T11:42:40.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.373+0000 7f2756800640 1 Processor -- start 2024-04-25T11:42:40.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.373+0000 7f2756800640 1 -- start start 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.374+0000 7f2756800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2734004800 0x7f2750136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.374+0000 7f2756800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27500b1680 0x7f27500b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.374+0000 7f2756800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27501eaa90 0x7f27501eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.374+0000 7f2756800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27501447d0 con 0x7f27500b1680 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.374+0000 7f2756800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f275015f160 con 0x7f27501eaa90 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.374+0000 7f2756800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2750162b00 con 0x7f2734004800 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27500b1680 0x7f27500b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27500b1680 0x7f27500b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55070/0 (socket says 172.21.15.19:55070) 2024-04-25T11:42:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 -- 172.21.15.19:0/2805887503 learned_addr learned my addr 172.21.15.19:0/2805887503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274fe00640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27501eaa90 0x7f27501eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274f400640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2734004800 0x7f2750136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 -- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2734004800 msgr2=0x7f2750136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2734004800 0x7f2750136150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 -- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27501eaa90 msgr2=0x7f27501eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27501eaa90 0x7f27501eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.375+0000 7f274ea00640 1 -- 172.21.15.19:0/2805887503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27500b1fd0 con 0x7f27500b1680 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.376+0000 7f274ea00640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27500b1680 0x7f27500b1a90 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f2740001840 tx=0x7f2740001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.376+0000 7f2747400640 1 -- 172.21.15.19:0/2805887503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740037040 con 0x7f27500b1680 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.376+0000 7f2747400640 1 -- 172.21.15.19:0/2805887503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2740005480 con 0x7f27500b1680 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.376+0000 7f2747400640 1 -- 172.21.15.19:0/2805887503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740001370 con 0x7f27500b1680 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.376+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27501eb3e0 con 0x7f27500b1680 2024-04-25T11:42:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.376+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27501e8430 con 0x7f27500b1680 2024-04-25T11:42:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.377+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275006f950 con 0x7f27500b1680 2024-04-25T11:42:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.379+0000 7f2747400640 1 -- 172.21.15.19:0/2805887503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2740044020 con 0x7f27500b1680 2024-04-25T11:42:40.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.380+0000 7f2747400640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f272c0744b0 0x7f272c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.380+0000 7f2747400640 1 -- 172.21.15.19:0/2805887503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f2740007640 con 0x7f27500b1680 2024-04-25T11:42:40.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.380+0000 7f274f400640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f272c0744b0 0x7f272c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.382+0000 7f274f400640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f272c0744b0 0x7f272c0769c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f27500d1860 tx=0x7f2738008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:40.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.385+0000 7f2747400640 1 -- 172.21.15.19:0/2805887503 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27501eb3e0 con 0x7f27500b1680 2024-04-25T11:42:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.590+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27500b21b0 con 0x7f27500b1680 2024-04-25T11:42:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.592+0000 7f2747400640 1 -- 172.21.15.19:0/2805887503 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+16619 (secure 0 0 0) 0x7f27500b1fd0 con 0x7f27500b1680 2024-04-25T11:42:40.580 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:40.581 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":143,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:30.672424+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f272c0744b0 msgr2=0x7f272c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f272c0744b0 0x7f272c0769c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f27500d1860 tx=0x7f2738008040 comp rx=0 tx=0).stop 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27500b1680 msgr2=0x7f27500b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27500b1680 0x7f27500b1a90 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f2740001840 tx=0x7f2740001120 comp rx=0 tx=0).stop 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 shutdown_connections 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f272c0744b0 0x7f272c0769c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27501eaa90 0x7f27501eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27500b1680 0x7f27500b1a90 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 --2- 172.21.15.19:0/2805887503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2734004800 0x7f2750136150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.595+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 >> 172.21.15.19:0/2805887503 conn(0x7f27500d1ae0 msgr2=0x7f27500cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.596+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 shutdown_connections 2024-04-25T11:42:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.596+0000 7f2756800640 1 -- 172.21.15.19:0/2805887503 wait complete. 2024-04-25T11:42:40.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:42:27.399161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:42:28.185279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:42:40.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:42:40.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.915+0000 7f306ae00640 1 Processor -- start 2024-04-25T11:42:40.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.915+0000 7f306ae00640 1 -- start start 2024-04-25T11:42:40.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.917+0000 7f306ae00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30640cd9a0 0x7f306415f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.917+0000 7f306ae00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30640b0af0 0x7f306406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.917+0000 7f306ae00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f306406df20 con 0x7f306406db10 2024-04-25T11:42:40.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.917+0000 7f306ae00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f30640ceab0 con 0x7f30640cd9a0 2024-04-25T11:42:40.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.917+0000 7f306ae00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f30640cdea0 con 0x7f30640b0af0 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30640cd9a0 0x7f306415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30640cd9a0 0x7f306415f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55086/0 (socket says 172.21.15.19:55086) 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063e00640 1 -- 172.21.15.19:0/1493959865 learned_addr learned my addr 172.21.15.19:0/1493959865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063400640 1 --2- 172.21.15.19:0/1493959865 >> v2:172.21.15.169:3300/0 conn(0x7f30640b0af0 0x7f306406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063e00640 1 -- 172.21.15.19:0/1493959865 >> v2:172.21.15.169:3300/0 conn(0x7f30640b0af0 msgr2=0x7f306406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063e00640 1 --2- 172.21.15.19:0/1493959865 >> v2:172.21.15.169:3300/0 conn(0x7f30640b0af0 0x7f306406d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063e00640 1 -- 172.21.15.19:0/1493959865 >> v1:172.21.15.169:6789/0 conn(0x7f306406db10 legacy=0x7f3064161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:40.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.918+0000 7f3063e00640 1 -- 172.21.15.19:0/1493959865 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30640ce180 con 0x7f30640cd9a0 2024-04-25T11:42:40.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.920+0000 7f3063e00640 1 --2- 172.21.15.19:0/1493959865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30640cd9a0 0x7f306415f360 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f3064162f90 tx=0x7f305402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f437c86723be8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.920+0000 7f3062a00640 1 -- 172.21.15.19:0/1493959865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054036070 con 0x7f30640cd9a0 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.920+0000 7f3062a00640 1 -- 172.21.15.19:0/1493959865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3054031600 con 0x7f30640cd9a0 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.920+0000 7f3062a00640 1 -- 172.21.15.19:0/1493959865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540319d0 con 0x7f30640cd9a0 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 -- 172.21.15.19:0/1493959865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30640cd9a0 msgr2=0x7f306415f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 --2- 172.21.15.19:0/1493959865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30640cd9a0 0x7f306415f360 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f3064162f90 tx=0x7f305402a6f0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 -- 172.21.15.19:0/1493959865 shutdown_connections 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 --2- 172.21.15.19:0/1493959865 >> v2:172.21.15.169:3300/0 conn(0x7f30640b0af0 0x7f306406d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 --2- 172.21.15.19:0/1493959865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30640cd9a0 0x7f306415f360 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 -- 172.21.15.19:0/1493959865 >> 172.21.15.19:0/1493959865 conn(0x7f3064073090 msgr2=0x7f30640734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:40.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 -- 172.21.15.19:0/1493959865 shutdown_connections 2024-04-25T11:42:40.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.921+0000 7f306ae00640 1 -- 172.21.15.19:0/1493959865 wait complete. 2024-04-25T11:42:40.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.924+0000 7f306ae00640 1 Processor -- start 2024-04-25T11:42:40.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.924+0000 7f306ae00640 1 -- start start 2024-04-25T11:42:40.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.925+0000 7f306ae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f306406db10 0x7f306413b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.925+0000 7f306ae00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30640b0af0 0x7f3064157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.925+0000 7f306ae00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cd9a0 0x7f306413ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.925+0000 7f306ae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30640b0f00 con 0x7f306406db10 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.925+0000 7f306ae00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f30640ce740 con 0x7f30640cd9a0 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.925+0000 7f306ae00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30640cef40 con 0x7f30640b0af0 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f306406db10 0x7f306413b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f306406db10 0x7f306413b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55098/0 (socket says 172.21.15.19:55098) 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 -- 172.21.15.19:0/1617130147 learned_addr learned my addr 172.21.15.19:0/1617130147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3068e00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cd9a0 0x7f306413ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063400640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30640b0af0 0x7f3064157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 -- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30640b0af0 msgr2=0x7f3064157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30640b0af0 0x7f3064157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 -- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cd9a0 msgr2=0x7f306413ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cd9a0 0x7f306413ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.926+0000 7f3063e00640 1 -- 172.21.15.19:0/1617130147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3064140b50 con 0x7f306406db10 2024-04-25T11:42:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.927+0000 7f3063e00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f306406db10 0x7f306413b1c0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f3054009f40 tx=0x7f3054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:40.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.927+0000 7f305be00640 1 -- 172.21.15.19:0/1617130147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054041030 con 0x7f306406db10 2024-04-25T11:42:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.927+0000 7f305be00640 1 -- 172.21.15.19:0/1617130147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3054031400 con 0x7f306406db10 2024-04-25T11:42:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.927+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306413a6e0 con 0x7f306406db10 2024-04-25T11:42:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.927+0000 7f305be00640 1 -- 172.21.15.19:0/1617130147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054036040 con 0x7f306406db10 2024-04-25T11:42:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.927+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30641f4d70 con 0x7f306406db10 2024-04-25T11:42:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.928+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30641f4fd0 con 0x7f306406db10 2024-04-25T11:42:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.929+0000 7f305be00640 1 -- 172.21.15.19:0/1617130147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3054044070 con 0x7f306406db10 2024-04-25T11:42:40.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.930+0000 7f305be00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30440744b0 0x7f30440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:40.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.931+0000 7f305be00640 1 -- 172.21.15.19:0/1617130147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f30540b5c50 con 0x7f306406db10 2024-04-25T11:42:40.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.931+0000 7f3063400640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30440744b0 0x7f30440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.932+0000 7f3063400640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30440744b0 0x7f30440769c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3050006fd0 tx=0x7f3050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:40.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:40.936+0000 7f305be00640 1 -- 172.21.15.19:0/1617130147 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f305404b070 con 0x7f306406db10 2024-04-25T11:42:41.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.145+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3064050740 con 0x7f306406db10 2024-04-25T11:42:41.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.146+0000 7f305be00640 1 -- 172.21.15.19:0/1617130147 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7f306406df20 con 0x7f306406db10 2024-04-25T11:42:41.134 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:42:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.149+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30440744b0 msgr2=0x7f30440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30440744b0 0x7f30440769c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3050006fd0 tx=0x7f3050008040 comp rx=0 tx=0).stop 2024-04-25T11:42:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f306406db10 msgr2=0x7f306413b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f306406db10 0x7f306413b1c0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f3054009f40 tx=0x7f3054004a40 comp rx=0 tx=0).stop 2024-04-25T11:42:41.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 shutdown_connections 2024-04-25T11:42:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f30440744b0 0x7f30440769c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30640cd9a0 0x7f306413ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30640b0af0 0x7f3064157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 --2- 172.21.15.19:0/1617130147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f306406db10 0x7f306413b1c0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 >> 172.21.15.19:0/1617130147 conn(0x7f3064073090 msgr2=0x7f30640b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 shutdown_connections 2024-04-25T11:42:41.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.150+0000 7f306ae00640 1 -- 172.21.15.19:0/1617130147 wait complete. 2024-04-25T11:42:41.168 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:42:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.463+0000 7f1f17200640 1 Processor -- start 2024-04-25T11:42:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.463+0000 7f1f17200640 1 -- start start 2024-04-25T11:42:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.465+0000 7f1f17200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f181460a0 0x7f1f180ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.465+0000 7f1f17200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180ce610 0x7f1f180cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.465+0000 7f1f17200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f18070190 con 0x7f1f180cef60 2024-04-25T11:42:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.465+0000 7f1f17200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1f18070470 con 0x7f1f181460a0 2024-04-25T11:42:41.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.465+0000 7f1f17200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f18143c20 con 0x7f1f180ce610 2024-04-25T11:42:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.466+0000 7f1f15400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180ce610 0x7f1f180cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.466+0000 7f1f15e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f181460a0 0x7f1f180ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.466+0000 7f1f15400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180ce610 0x7f1f180cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:51826/0 (socket says 172.21.15.19:51826) 2024-04-25T11:42:41.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.466+0000 7f1f15400640 1 -- 172.21.15.19:0/2798235803 learned_addr learned my addr 172.21.15.19:0/2798235803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.467+0000 7f1f15e00640 1 -- 172.21.15.19:0/2798235803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180ce610 msgr2=0x7f1f180cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.467+0000 7f1f15e00640 1 --2- 172.21.15.19:0/2798235803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180ce610 0x7f1f180cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.467+0000 7f1f15e00640 1 -- 172.21.15.19:0/2798235803 >> v1:172.21.15.169:6789/0 conn(0x7f1f180cef60 legacy=0x7f1f18144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.467+0000 7f1f15e00640 1 -- 172.21.15.19:0/2798235803 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f18143f00 con 0x7f1f181460a0 2024-04-25T11:42:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.468+0000 7f1f15e00640 1 --2- 172.21.15.19:0/2798235803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f181460a0 0x7f1f180ce0d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097a0 tx=0x7f1f0002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852e325ea3233ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.468+0000 7f1f14a00640 1 -- 172.21.15.19:0/2798235803 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00037070 con 0x7f1f181460a0 2024-04-25T11:42:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.468+0000 7f1f14a00640 1 -- 172.21.15.19:0/2798235803 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f00032440 con 0x7f1f181460a0 2024-04-25T11:42:41.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.468+0000 7f1f14a00640 1 -- 172.21.15.19:0/2798235803 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00032810 con 0x7f1f181460a0 2024-04-25T11:42:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 -- 172.21.15.19:0/2798235803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f181460a0 msgr2=0x7f1f180ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 --2- 172.21.15.19:0/2798235803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f181460a0 0x7f1f180ce0d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097a0 tx=0x7f1f0002a180 comp rx=0 tx=0).stop 2024-04-25T11:42:41.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 -- 172.21.15.19:0/2798235803 shutdown_connections 2024-04-25T11:42:41.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 --2- 172.21.15.19:0/2798235803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180ce610 0x7f1f180cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 --2- 172.21.15.19:0/2798235803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f181460a0 0x7f1f180ce0d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 -- 172.21.15.19:0/2798235803 >> 172.21.15.19:0/2798235803 conn(0x7f1f18072f40 msgr2=0x7f1f18073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:41.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 -- 172.21.15.19:0/2798235803 shutdown_connections 2024-04-25T11:42:41.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.469+0000 7f1f17200640 1 -- 172.21.15.19:0/2798235803 wait complete. 2024-04-25T11:42:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.472+0000 7f1f17200640 1 Processor -- start 2024-04-25T11:42:41.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.472+0000 7f1f17200640 1 -- start start 2024-04-25T11:42:41.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.473+0000 7f1f17200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f180ce610 0x7f1f181e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.473+0000 7f1f17200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180cef60 0x7f1f181f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.473+0000 7f1f17200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f181460a0 0x7f1f181ebba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.473+0000 7f1f17200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18144f90 con 0x7f1f181460a0 2024-04-25T11:42:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.473+0000 7f1f17200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18144e10 con 0x7f1f180ce610 2024-04-25T11:42:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.473+0000 7f1f17200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f18145110 con 0x7f1f180cef60 2024-04-25T11:42:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.474+0000 7f1f16800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f181460a0 0x7f1f181ebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.474+0000 7f1f16800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f181460a0 0x7f1f181ebba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55112/0 (socket says 172.21.15.19:55112) 2024-04-25T11:42:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.474+0000 7f1f16800640 1 -- 172.21.15.19:0/2285911749 learned_addr learned my addr 172.21.15.19:0/2285911749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.474+0000 7f1f15400640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180cef60 0x7f1f181f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.474+0000 7f1f15e00640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f180ce610 0x7f1f181e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.474+0000 7f1f16800640 1 -- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180cef60 msgr2=0x7f1f181f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.474+0000 7f1f16800640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180cef60 0x7f1f181f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f16800640 1 -- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f180ce610 msgr2=0x7f1f181e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f16800640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f180ce610 0x7f1f181e9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f16800640 1 -- 172.21.15.19:0/2285911749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f181eab00 con 0x7f1f181460a0 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f16800640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f181460a0 0x7f1f181ebba0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c730 tx=0x7f1f1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f0e000640 1 -- 172.21.15.19:0/2285911749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003b70 con 0x7f1f181460a0 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f0e000640 1 -- 172.21.15.19:0/2285911749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f10003d10 con 0x7f1f181460a0 2024-04-25T11:42:41.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f0e000640 1 -- 172.21.15.19:0/2285911749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10005310 con 0x7f1f181460a0 2024-04-25T11:42:41.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f181eade0 con 0x7f1f181460a0 2024-04-25T11:42:41.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.475+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181eb3a0 con 0x7f1f181460a0 2024-04-25T11:42:41.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.477+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f18076c60 con 0x7f1f181460a0 2024-04-25T11:42:41.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.478+0000 7f1f0e000640 1 -- 172.21.15.19:0/2285911749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f100056c0 con 0x7f1f181460a0 2024-04-25T11:42:41.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.479+0000 7f1f0e000640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ef4074530 0x7f1ef4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.479+0000 7f1f0e000640 1 -- 172.21.15.19:0/2285911749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f1f10095230 con 0x7f1f181460a0 2024-04-25T11:42:41.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.480+0000 7f1f15e00640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ef4074530 0x7f1ef4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.481+0000 7f1f15e00640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ef4074530 0x7f1ef4076a40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1f00034000 tx=0x7f1f0002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:41.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.484+0000 7f1f0e000640 1 -- 172.21.15.19:0/2285911749 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f1005f9a0 con 0x7f1f181460a0 2024-04-25T11:42:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.687+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f1813d810 con 0x7f1f181460a0 2024-04-25T11:42:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.688+0000 7f1f0e000640 1 -- 172.21.15.19:0/2285911749 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) ==== 104+0+11 (secure 0 0 0) 0x7f1f1005f2a0 con 0x7f1f181460a0 2024-04-25T11:42:41.675 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:42:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ef4074530 msgr2=0x7f1ef4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ef4074530 0x7f1ef4076a40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f1f00034000 tx=0x7f1f0002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f181460a0 msgr2=0x7f1f181ebba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f181460a0 0x7f1f181ebba0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c730 tx=0x7f1f1000cbf0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 shutdown_connections 2024-04-25T11:42:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ef4074530 0x7f1ef4076a40 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f181460a0 0x7f1f181ebba0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f180cef60 0x7f1f181f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 --2- 172.21.15.19:0/2285911749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f180ce610 0x7f1f181e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 >> 172.21.15.19:0/2285911749 conn(0x7f1f18072f40 msgr2=0x7f1f180cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:41.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.691+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 shutdown_connections 2024-04-25T11:42:41.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:41.692+0000 7f1f17200640 1 -- 172.21.15.19:0/2285911749 wait complete. 2024-04-25T11:42:41.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:42:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.003+0000 7f55baa00640 1 Processor -- start 2024-04-25T11:42:41.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.003+0000 7f55baa00640 1 -- start start 2024-04-25T11:42:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55baa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b406dce0 0x7f55b40af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55baa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f55b415fff0 0x7f55b40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55baa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b40af9b0 con 0x7f55b40b0a50 2024-04-25T11:42:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55baa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f55b40b54c0 con 0x7f55b415fff0 2024-04-25T11:42:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55baa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55b40afc90 con 0x7f55b406dce0 2024-04-25T11:42:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55ba000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f55b415fff0 0x7f55b40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55ba000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f55b415fff0 0x7f55b40b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55118/0 (socket says 172.21.15.19:55118) 2024-04-25T11:42:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55ba000640 1 -- 172.21.15.19:0/1373245206 learned_addr learned my addr 172.21.15.19:0/1373245206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:41.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.006+0000 7f55b8c00640 1 --2- 172.21.15.19:0/1373245206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b406dce0 0x7f55b40af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.007+0000 7f55ba000640 1 -- 172.21.15.19:0/1373245206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b406dce0 msgr2=0x7f55b40af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.007+0000 7f55ba000640 1 --2- 172.21.15.19:0/1373245206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b406dce0 0x7f55b40af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.007+0000 7f55ba000640 1 -- 172.21.15.19:0/1373245206 >> v1:172.21.15.19:6789/0 conn(0x7f55b40b0a50 legacy=0x7f55b406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.007+0000 7f55ba000640 1 -- 172.21.15.19:0/1373245206 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b40aff70 con 0x7f55b415fff0 2024-04-25T11:42:41.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.008+0000 7f55ba000640 1 --2- 172.21.15.19:0/1373245206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b415fff0 0x7f55b40b4f80 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009e50 tx=0x7f55ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cfd2c390f0b5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:41.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.008+0000 7f55b3e00640 1 -- 172.21.15.19:0/1373245206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac036070 con 0x7f55b415fff0 2024-04-25T11:42:41.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.008+0000 7f55b3e00640 1 -- 172.21.15.19:0/1373245206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac031600 con 0x7f55b415fff0 2024-04-25T11:42:41.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.008+0000 7f55b3e00640 1 -- 172.21.15.19:0/1373245206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0319d0 con 0x7f55b415fff0 2024-04-25T11:42:41.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 -- 172.21.15.19:0/1373245206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b415fff0 msgr2=0x7f55b40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 --2- 172.21.15.19:0/1373245206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b415fff0 0x7f55b40b4f80 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009e50 tx=0x7f55ac02a680 comp rx=0 tx=0).stop 2024-04-25T11:42:41.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 -- 172.21.15.19:0/1373245206 shutdown_connections 2024-04-25T11:42:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 --2- 172.21.15.19:0/1373245206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b415fff0 0x7f55b40b4f80 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 --2- 172.21.15.19:0/1373245206 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b406dce0 0x7f55b40af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 -- 172.21.15.19:0/1373245206 >> 172.21.15.19:0/1373245206 conn(0x7f55b4073160 msgr2=0x7f55b4073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 -- 172.21.15.19:0/1373245206 shutdown_connections 2024-04-25T11:42:41.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.009+0000 7f55baa00640 1 -- 172.21.15.19:0/1373245206 wait complete. 2024-04-25T11:42:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.011+0000 7f55baa00640 1 Processor -- start 2024-04-25T11:42:41.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.012+0000 7f55baa00640 1 -- start start 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55baa00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b40b0a50 0x7f55b415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55baa00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b415fff0 0x7f55b414e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55baa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b414f140 0x7f55b40b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55baa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4160400 con 0x7f55b414f140 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55baa00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f55b40b0530 con 0x7f55b40b0a50 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55baa00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f55b406f090 con 0x7f55b415fff0 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55ba000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b414f140 0x7f55b40b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55ba000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b414f140 0x7f55b40b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55120/0 (socket says 172.21.15.19:55120) 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.013+0000 7f55ba000640 1 -- 172.21.15.19:0/3814707404 learned_addr learned my addr 172.21.15.19:0/3814707404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55b9600640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b40b0a50 0x7f55b415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55b8c00640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b415fff0 0x7f55b414e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55ba000640 1 -- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b415fff0 msgr2=0x7f55b414e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55ba000640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b415fff0 0x7f55b414e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55ba000640 1 -- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b40b0a50 msgr2=0x7f55b415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55ba000640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b40b0a50 0x7f55b415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55ba000640 1 -- 172.21.15.19:0/3814707404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b40cf850 con 0x7f55b414f140 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55b9600640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b40b0a50 0x7f55b415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:42.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55ba000640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b414f140 0x7f55b40b2220 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f55ac034040 tx=0x7f55ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55b1600640 1 -- 172.21.15.19:0/3814707404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac036050 con 0x7f55b414f140 2024-04-25T11:42:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55b1600640 1 -- 172.21.15.19:0/3814707404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac042070 con 0x7f55b414f140 2024-04-25T11:42:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55b1600640 1 -- 172.21.15.19:0/3814707404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac009510 con 0x7f55b414f140 2024-04-25T11:42:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.014+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b40cfaf0 con 0x7f55b414f140 2024-04-25T11:42:42.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.015+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b40d00b0 con 0x7f55b414f140 2024-04-25T11:42:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.016+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b415b6b0 con 0x7f55b414f140 2024-04-25T11:42:42.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.017+0000 7f55b1600640 1 -- 172.21.15.19:0/3814707404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f55ac007bd0 con 0x7f55b414f140 2024-04-25T11:42:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.018+0000 7f55b1600640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5594074530 0x7f5594076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.018+0000 7f55b1600640 1 -- 172.21.15.19:0/3814707404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f55ac0b5ff0 con 0x7f55b414f140 2024-04-25T11:42:42.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.019+0000 7f55b9600640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5594074530 0x7f5594076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.020+0000 7f55b9600640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5594074530 0x7f5594076a40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f55b40aedd0 tx=0x7f55a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.023+0000 7f55b1600640 1 -- 172.21.15.19:0/3814707404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55ac080850 con 0x7f55b414f140 2024-04-25T11:42:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.225+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f55b4073ec0 con 0x7f55b414f140 2024-04-25T11:42:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.226+0000 7f55b1600640 1 -- 172.21.15.19:0/3814707404 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) ==== 100+0+11 (secure 0 0 0) 0x7f55ac080150 con 0x7f55b414f140 2024-04-25T11:42:42.214 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.229+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5594074530 msgr2=0x7f5594076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.229+0000 7f55baa00640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5594074530 0x7f5594076a40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f55b40aedd0 tx=0x7f55a8009040 comp rx=0 tx=0).stop 2024-04-25T11:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.229+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b414f140 msgr2=0x7f55b40b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.229+0000 7f55baa00640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b414f140 0x7f55b40b2220 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f55ac034040 tx=0x7f55ac004a40 comp rx=0 tx=0).stop 2024-04-25T11:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 shutdown_connections 2024-04-25T11:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5594074530 0x7f5594076a40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55b414f140 0x7f55b40b2220 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55b415fff0 0x7f55b414e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 --2- 172.21.15.19:0/3814707404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55b40b0a50 0x7f55b415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 >> 172.21.15.19:0/3814707404 conn(0x7f55b4073160 msgr2=0x7f55b4158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 shutdown_connections 2024-04-25T11:42:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.230+0000 7f55baa00640 1 -- 172.21.15.19:0/3814707404 wait complete. 2024-04-25T11:42:42.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:42:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.533+0000 7ff06ca00640 1 Processor -- start 2024-04-25T11:42:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.534+0000 7ff06ca00640 1 -- start start 2024-04-25T11:42:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.536+0000 7ff06ca00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff068160360 0x7ff0681647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.536+0000 7ff06ca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06806c7d0 con 0x7ff068070680 2024-04-25T11:42:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.536+0000 7ff06ca00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06806cab0 con 0x7ff0680b5c90 2024-04-25T11:42:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.536+0000 7ff06ca00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff06806d100 con 0x7ff068160360 2024-04-25T11:42:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.537+0000 7ff065e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff068070680 0x7ff0680af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50878/0 (socket says 172.21.15.19:50878) 2024-04-25T11:42:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.537+0000 7ff065e00640 1 -- 172.21.15.19:0/2294306162 learned_addr learned my addr 172.21.15.19:0/2294306162 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.537+0000 7ff05fe00640 1 --2- 172.21.15.19:0/2294306162 >> v2:172.21.15.169:3300/0 conn(0x7ff068160360 0x7ff0681647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.537+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853190574 0 0) 0x7ff06806c7d0 con 0x7ff068070680 2024-04-25T11:42:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.537+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff04c003520 con 0x7ff068070680 2024-04-25T11:42:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3796499957 0 0) 0x7ff04c003520 con 0x7ff068070680 2024-04-25T11:42:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 >> v2:172.21.15.169:3300/0 conn(0x7ff068160360 msgr2=0x7ff0681647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 --2- 172.21.15.19:0/2294306162 >> v2:172.21.15.169:3300/0 conn(0x7ff068160360 0x7ff0681647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 >> v1:172.21.15.169:6789/0 conn(0x7ff0680b5c90 legacy=0x7ff0680b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06806cd90 con 0x7ff068070680 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ff058001670 con 0x7ff068070680 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 >> v1:172.21.15.19:6789/0 conn(0x7ff068070680 legacy=0x7ff0680af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff04c0039b0 0x7ff04c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c004800 0x7ff04c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff04c005270 0x7ff04c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff06806d100 con 0x7ff04c005270 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c00c200 con 0x7ff04c0039b0 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff04c00c400 con 0x7ff04c004800 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff066800640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff04c005270 0x7ff04c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.538+0000 7ff065e00640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff04c0039b0 0x7ff04c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.539+0000 7ff05fe00640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c004800 0x7ff04c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.539+0000 7ff066800640 1 -- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c004800 msgr2=0x7ff04c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.539+0000 7ff066800640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c004800 0x7ff04c004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.539+0000 7ff066800640 1 -- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff04c0039b0 msgr2=0x7ff04c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.539+0000 7ff066800640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff04c0039b0 0x7ff04c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.539+0000 7ff066800640 1 -- 172.21.15.19:0/2294306162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c003520 con 0x7ff04c005270 2024-04-25T11:42:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.540+0000 7ff066800640 1 -- 172.21.15.19:0/2294306162 reap_dead start 2024-04-25T11:42:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.540+0000 7ff066800640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff04c005270 0x7ff04c00b9c0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7ff060001e20 tx=0x7ff04c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edff9da219913d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.540+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060001a70 con 0x7ff04c005270 2024-04-25T11:42:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.540+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06002ae40 con 0x7ff04c005270 2024-04-25T11:42:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.540+0000 7ff065400640 1 -- 172.21.15.19:0/2294306162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060003bb0 con 0x7ff04c005270 2024-04-25T11:42:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.541+0000 7ff06ca00640 1 -- 172.21.15.19:0/2294306162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff04c005270 msgr2=0x7ff04c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.541+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff04c005270 0x7ff04c00b9c0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7ff060001e20 tx=0x7ff04c005830 comp rx=0 tx=0).stop 2024-04-25T11:42:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.541+0000 7ff06ca00640 1 -- 172.21.15.19:0/2294306162 shutdown_connections 2024-04-25T11:42:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.541+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2294306162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff04c005270 0x7ff04c00b9c0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.541+0000 7ff06ca00640 1 -- 172.21.15.19:0/2294306162 >> 172.21.15.19:0/2294306162 conn(0x7ff0680d1ae0 msgr2=0x7ff0680d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.541+0000 7ff06ca00640 1 -- 172.21.15.19:0/2294306162 shutdown_connections 2024-04-25T11:42:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.541+0000 7ff06ca00640 1 -- 172.21.15.19:0/2294306162 wait complete. 2024-04-25T11:42:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.543+0000 7ff06ca00640 1 Processor -- start 2024-04-25T11:42:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.544+0000 7ff06ca00640 1 -- start start 2024-04-25T11:42:42.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.545+0000 7ff06ca00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c005270 0x7ff06815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.545+0000 7ff06ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff06815ea20 0x7ff06814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.545+0000 7ff06ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff06814bdf0 0x7ff06814aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.545+0000 7ff06ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff068164970 con 0x7ff06815ea20 2024-04-25T11:42:42.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.545+0000 7ff06ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0681647f0 con 0x7ff06814bdf0 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.545+0000 7ff06ca00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff068164af0 con 0x7ff04c005270 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.545+0000 7ff05fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff06815ea20 0x7ff06814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff06815ea20 0x7ff06814b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55146/0 (socket says 172.21.15.19:55146) 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 -- 172.21.15.19:0/2664703918 learned_addr learned my addr 172.21.15.19:0/2664703918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff066800640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff06814bdf0 0x7ff06814aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff065e00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c005270 0x7ff06815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 -- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c005270 msgr2=0x7ff06815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c005270 0x7ff06815e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 -- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff06814bdf0 msgr2=0x7ff06814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff06814bdf0 0x7ff06814aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 -- 172.21.15.19:0/2664703918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06814c650 con 0x7ff06815ea20 2024-04-25T11:42:42.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.546+0000 7ff05fe00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff06815ea20 0x7ff06814b8b0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ff050010890 tx=0x7ff050010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.547+0000 7ff05e000640 1 -- 172.21.15.19:0/2664703918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05000f040 con 0x7ff06815ea20 2024-04-25T11:42:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.547+0000 7ff05e000640 1 -- 172.21.15.19:0/2664703918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff05000eb80 con 0x7ff06815ea20 2024-04-25T11:42:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.547+0000 7ff05e000640 1 -- 172.21.15.19:0/2664703918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05001c400 con 0x7ff06815ea20 2024-04-25T11:42:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.547+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06814c930 con 0x7ff06815ea20 2024-04-25T11:42:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.547+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06814cef0 con 0x7ff06815ea20 2024-04-25T11:42:42.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.548+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06814c3c0 con 0x7ff06815ea20 2024-04-25T11:42:42.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.549+0000 7ff05e000640 1 -- 172.21.15.19:0/2664703918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff0500253f0 con 0x7ff06815ea20 2024-04-25T11:42:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.550+0000 7ff05e000640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0440744e0 0x7ff0440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.550+0000 7ff05e000640 1 -- 172.21.15.19:0/2664703918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7ff0500959b0 con 0x7ff06815ea20 2024-04-25T11:42:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.551+0000 7ff065e00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0440744e0 0x7ff0440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.552+0000 7ff065e00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0440744e0 0x7ff0440769f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff068071250 tx=0x7ff058009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.555+0000 7ff05e000640 1 -- 172.21.15.19:0/2664703918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff050018030 con 0x7ff06815ea20 2024-04-25T11:42:42.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.774+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff068050740 con 0x7ff06815ea20 2024-04-25T11:42:42.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.776+0000 7ff05e000640 1 -- 172.21.15.19:0/2664703918 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v143) ==== 80+0+0 (secure 0 0 0) 0x7ff0500600a0 con 0x7ff06815ea20 2024-04-25T11:42:42.763 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-04-25T11:42:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.778+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0440744e0 msgr2=0x7ff0440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0440744e0 0x7ff0440769f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff068071250 tx=0x7ff058009040 comp rx=0 tx=0).stop 2024-04-25T11:42:42.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff06815ea20 msgr2=0x7ff06814b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff06815ea20 0x7ff06814b8b0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ff050010890 tx=0x7ff050010d50 comp rx=0 tx=0).stop 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 shutdown_connections 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff0440744e0 0x7ff0440769f0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff06814bdf0 0x7ff06814aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff06815ea20 0x7ff06814b8b0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 --2- 172.21.15.19:0/2664703918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff04c005270 0x7ff06815e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 >> 172.21.15.19:0/2664703918 conn(0x7ff0680d1ae0 msgr2=0x7ff0680b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 shutdown_connections 2024-04-25T11:42:42.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:42.779+0000 7ff06ca00640 1 -- 172.21.15.19:0/2664703918 wait complete. 2024-04-25T11:42:42.793 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:42:42.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:42.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:42.940 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:42:42.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:42.941 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:42:42.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.287+0000 7fcd97e00640 1 Processor -- start 2024-04-25T11:42:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.288+0000 7fcd97e00640 1 -- start start 2024-04-25T11:42:43.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.290+0000 7fcd97e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd980b6430 0x7fcd9806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:43.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.290+0000 7fcd97e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcd98160aa0 0x7fcd98160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:43.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.290+0000 7fcd97e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd981613f0 con 0x7fcd980d0bd0 2024-04-25T11:42:43.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.290+0000 7fcd97e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9806c7d0 con 0x7fcd980b6430 2024-04-25T11:42:43.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.290+0000 7fcd97e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9806cab0 con 0x7fcd98160aa0 2024-04-25T11:42:43.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd96a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd980b6430 0x7fcd9806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:43.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd96a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd980b6430 0x7fcd9806d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55156/0 (socket says 172.21.15.19:55156) 2024-04-25T11:42:43.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd96a00640 1 -- 172.21.15.19:0/3711342117 learned_addr learned my addr 172.21.15.19:0/3711342117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd8fe00640 1 --2- 172.21.15.19:0/3711342117 >> v2:172.21.15.169:3300/0 conn(0x7fcd98160aa0 0x7fcd98160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd96a00640 1 -- 172.21.15.19:0/3711342117 >> v2:172.21.15.169:3300/0 conn(0x7fcd98160aa0 msgr2=0x7fcd98160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd96a00640 1 --2- 172.21.15.19:0/3711342117 >> v2:172.21.15.169:3300/0 conn(0x7fcd98160aa0 0x7fcd98160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd96a00640 1 -- 172.21.15.19:0/3711342117 >> v1:172.21.15.169:6789/0 conn(0x7fcd980d0bd0 legacy=0x7fcd980d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:43.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.291+0000 7fcd96a00640 1 -- 172.21.15.19:0/3711342117 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9806cd90 con 0x7fcd980b6430 2024-04-25T11:42:43.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.292+0000 7fcd96a00640 1 --2- 172.21.15.19:0/3711342117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd980b6430 0x7fcd9806d870 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fcd98071020 tx=0x7fcd8802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50474f3cb22a35ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:43.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.293+0000 7fcd96000640 1 -- 172.21.15.19:0/3711342117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88037070 con 0x7fcd980b6430 2024-04-25T11:42:43.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.293+0000 7fcd96000640 1 -- 172.21.15.19:0/3711342117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8802ae20 con 0x7fcd980b6430 2024-04-25T11:42:43.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.293+0000 7fcd96000640 1 -- 172.21.15.19:0/3711342117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88032660 con 0x7fcd980b6430 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.293+0000 7fcd97e00640 1 -- 172.21.15.19:0/3711342117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd980b6430 msgr2=0x7fcd9806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.293+0000 7fcd97e00640 1 --2- 172.21.15.19:0/3711342117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd980b6430 0x7fcd9806d870 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fcd98071020 tx=0x7fcd8802a110 comp rx=0 tx=0).stop 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.293+0000 7fcd97e00640 1 -- 172.21.15.19:0/3711342117 shutdown_connections 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.293+0000 7fcd97e00640 1 --2- 172.21.15.19:0/3711342117 >> v2:172.21.15.169:3300/0 conn(0x7fcd98160aa0 0x7fcd98160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.294+0000 7fcd97e00640 1 --2- 172.21.15.19:0/3711342117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd980b6430 0x7fcd9806d870 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.294+0000 7fcd97e00640 1 -- 172.21.15.19:0/3711342117 >> 172.21.15.19:0/3711342117 conn(0x7fcd980d1720 msgr2=0x7fcd980d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.294+0000 7fcd97e00640 1 -- 172.21.15.19:0/3711342117 shutdown_connections 2024-04-25T11:42:43.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.294+0000 7fcd97e00640 1 -- 172.21.15.19:0/3711342117 wait complete. 2024-04-25T11:42:43.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.296+0000 7fcd97e00640 1 Processor -- start 2024-04-25T11:42:43.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.296+0000 7fcd97e00640 1 -- start start 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd980b6430 0x7fcd980cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd980d0bd0 0x7fcd98143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98160aa0 0x7fcd981440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9806ddb0 con 0x7fcd98160aa0 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcd980b6840 con 0x7fcd980d0bd0 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9806df30 con 0x7fcd980b6430 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98160aa0 0x7fcd981440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:43.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98160aa0 0x7fcd981440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55172/0 (socket says 172.21.15.19:55172) 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.298+0000 7fcd97400640 1 -- 172.21.15.19:0/2296434015 learned_addr learned my addr 172.21.15.19:0/2296434015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd96a00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd980b6430 0x7fcd980cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd8fe00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd980d0bd0 0x7fcd98143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd97400640 1 -- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd980b6430 msgr2=0x7fcd980cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd97400640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd980b6430 0x7fcd980cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd97400640 1 -- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd980d0bd0 msgr2=0x7fcd98143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd97400640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd980d0bd0 0x7fcd98143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd97400640 1 -- 172.21.15.19:0/2296434015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd980cdcf0 con 0x7fcd98160aa0 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd8fe00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd980d0bd0 0x7fcd98143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:43.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd97400640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98160aa0 0x7fcd981440c0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000b9e0 tx=0x7fcd9000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd8ea00640 1 -- 172.21.15.19:0/2296434015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90003b70 con 0x7fcd98160aa0 2024-04-25T11:42:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd8ea00640 1 -- 172.21.15.19:0/2296434015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd90003d10 con 0x7fcd98160aa0 2024-04-25T11:42:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.299+0000 7fcd8ea00640 1 -- 172.21.15.19:0/2296434015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000cb60 con 0x7fcd98160aa0 2024-04-25T11:42:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.300+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd981595d0 con 0x7fcd98160aa0 2024-04-25T11:42:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.300+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd98159b90 con 0x7fcd98160aa0 2024-04-25T11:42:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.301+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd98159340 con 0x7fcd98160aa0 2024-04-25T11:42:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.303+0000 7fcd8ea00640 1 -- 172.21.15.19:0/2296434015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd9000cd00 con 0x7fcd98160aa0 2024-04-25T11:42:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.304+0000 7fcd8ea00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd740744b0 0x7fcd740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.304+0000 7fcd8ea00640 1 -- 172.21.15.19:0/2296434015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7fcd90095120 con 0x7fcd98160aa0 2024-04-25T11:42:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.304+0000 7fcd96a00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd740744b0 0x7fcd740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.306+0000 7fcd96a00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd740744b0 0x7fcd740769c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcd88034000 tx=0x7fcd8802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:43.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.309+0000 7fcd8ea00640 1 -- 172.21.15.19:0/2296434015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd9005f810 con 0x7fcd98160aa0 2024-04-25T11:42:43.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.557+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd980b5f70 con 0x7fcd98160aa0 2024-04-25T11:42:43.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.558+0000 7fcd8ea00640 1 -- 172.21.15.19:0/2296434015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 364 v364) ==== 95+0+5195 (secure 0 0 0) 0x7fcd9005f110 con 0x7fcd98160aa0 2024-04-25T11:42:43.545 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:43.546 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":364,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":8020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":358}],"filesystems":[{"mdsmap":{"epoch":364,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:32.307292+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8032},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8032":{"gid":8032,"name":"b","rank":0,"incarnation":360,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2610059988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":2610059988},{"type":"v1","addr":"172.21.15.169:6835","nonce":2610059988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":7}]} 2024-04-25T11:42:43.546 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 364 2024-04-25T11:42:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.560+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd740744b0 msgr2=0x7fcd740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.560+0000 7fcd97e00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd740744b0 0x7fcd740769c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcd88034000 tx=0x7fcd8802e000 comp rx=0 tx=0).stop 2024-04-25T11:42:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.560+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98160aa0 msgr2=0x7fcd981440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.560+0000 7fcd97e00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98160aa0 0x7fcd981440c0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000b9e0 tx=0x7fcd9000bea0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 shutdown_connections 2024-04-25T11:42:43.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcd740744b0 0x7fcd740769c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd980d0bd0 0x7fcd98143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98160aa0 0x7fcd981440c0 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 --2- 172.21.15.19:0/2296434015 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcd980b6430 0x7fcd980cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 >> 172.21.15.19:0/2296434015 conn(0x7fcd980d1720 msgr2=0x7fcd980d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 shutdown_connections 2024-04-25T11:42:43.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:43.561+0000 7fcd97e00640 1 -- 172.21.15.19:0/2296434015 wait complete. 2024-04-25T11:42:43.576 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T11:42:43.907 INFO:teuthology.orchestra.run.smithi169.stdout:{"message":"","return_code":0} 2024-04-25T11:42:43.908 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T11:42:43.908 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:42:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.211+0000 7f1416800640 1 Processor -- start 2024-04-25T11:42:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.211+0000 7f1416800640 1 -- start start 2024-04-25T11:42:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.213+0000 7f1416800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14101443c0 0x7f141006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f1416800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14100cfcf0 0x7f141006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f1416800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f141015f5f0 con 0x7f1410070370 2024-04-25T11:42:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f1416800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f14100708d0 con 0x7f14101443c0 2024-04-25T11:42:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f1416800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f141015f160 con 0x7f14100cfcf0 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f140f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f14101443c0 0x7f141006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f140ea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1410070370 0x7f14100cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43022/0 (socket says 172.21.15.19:43022) 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f140ea00640 1 -- 172.21.15.19:0/86768663 learned_addr learned my addr 172.21.15.19:0/86768663 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.214+0000 7f140fe00640 1 --2- 172.21.15.19:0/86768663 >> v2:172.21.15.169:3300/0 conn(0x7f14100cfcf0 0x7f141006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.215+0000 7f140f400640 1 -- 172.21.15.19:0/86768663 >> v2:172.21.15.169:3300/0 conn(0x7f14100cfcf0 msgr2=0x7f141006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.215+0000 7f140f400640 1 --2- 172.21.15.19:0/86768663 >> v2:172.21.15.169:3300/0 conn(0x7f14100cfcf0 0x7f141006c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.215+0000 7f140f400640 1 -- 172.21.15.19:0/86768663 >> v1:172.21.15.19:6789/0 conn(0x7f1410070370 legacy=0x7f14100cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.215+0000 7f140f400640 1 -- 172.21.15.19:0/86768663 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141015e830 con 0x7f14101443c0 2024-04-25T11:42:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.215+0000 7f140e000640 1 -- 172.21.15.19:0/86768663 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701518415 0 0) 0x7f141015f5f0 con 0x7f1410070370 2024-04-25T11:42:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.215+0000 7f140fe00640 1 --2- 172.21.15.19:0/86768663 >> v2:172.21.15.169:3300/0 conn(0x7f14100cfcf0 0x7f141006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.216+0000 7f140f400640 1 --2- 172.21.15.19:0/86768663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f141006fe30 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f141015c280 tx=0x7f140002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bcb697bbff5478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.216+0000 7f140e000640 1 -- 172.21.15.19:0/86768663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400037070 con 0x7f14101443c0 2024-04-25T11:42:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.216+0000 7f140e000640 1 -- 172.21.15.19:0/86768663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1400032440 con 0x7f14101443c0 2024-04-25T11:42:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.216+0000 7f140e000640 1 -- 172.21.15.19:0/86768663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400032810 con 0x7f14101443c0 2024-04-25T11:42:44.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 -- 172.21.15.19:0/86768663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 msgr2=0x7f141006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 --2- 172.21.15.19:0/86768663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f141006fe30 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f141015c280 tx=0x7f140002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 -- 172.21.15.19:0/86768663 shutdown_connections 2024-04-25T11:42:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 --2- 172.21.15.19:0/86768663 >> v2:172.21.15.169:3300/0 conn(0x7f14100cfcf0 0x7f141006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 --2- 172.21.15.19:0/86768663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f141006fe30 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 -- 172.21.15.19:0/86768663 >> 172.21.15.19:0/86768663 conn(0x7f14100d1ae0 msgr2=0x7f14100d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 -- 172.21.15.19:0/86768663 shutdown_connections 2024-04-25T11:42:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.217+0000 7f1416800640 1 -- 172.21.15.19:0/86768663 wait complete. 2024-04-25T11:42:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.220+0000 7f1416800640 1 Processor -- start 2024-04-25T11:42:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.220+0000 7f1416800640 1 -- start start 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.221+0000 7f1416800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1410070370 0x7f14100c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.221+0000 7f1416800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f1410160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f1416800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100c7cb0 0x7f1410160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f1416800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14101447d0 con 0x7f14101443c0 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f1416800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f141015edf0 con 0x7f14100c7cb0 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f1416800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1410162b00 con 0x7f1410070370 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f1410160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f1410160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56014/0 (socket says 172.21.15.19:56014) 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 -- 172.21.15.19:0/2161429490 learned_addr learned my addr 172.21.15.19:0/2161429490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140f400640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1410070370 0x7f14100c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140fe00640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100c7cb0 0x7f1410160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 -- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1410070370 msgr2=0x7f14100c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1410070370 0x7f14100c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 -- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100c7cb0 msgr2=0x7f1410160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100c7cb0 0x7f1410160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140ea00640 1 -- 172.21.15.19:0/2161429490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141013fce0 con 0x7f14101443c0 2024-04-25T11:42:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.222+0000 7f140f400640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1410070370 0x7f14100c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.223+0000 7f140ea00640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f1410160670 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c3c0 tx=0x7f13fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.223+0000 7f1407400640 1 -- 172.21.15.19:0/2161429490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0074e0 con 0x7f14101443c0 2024-04-25T11:42:44.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.223+0000 7f1407400640 1 -- 172.21.15.19:0/2161429490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13fc0047e0 con 0x7f14101443c0 2024-04-25T11:42:44.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.223+0000 7f1407400640 1 -- 172.21.15.19:0/2161429490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc004bd0 con 0x7f14101443c0 2024-04-25T11:42:44.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.223+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141013ffc0 con 0x7f14101443c0 2024-04-25T11:42:44.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.223+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1410140580 con 0x7f14101443c0 2024-04-25T11:42:44.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.224+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1410050740 con 0x7f14101443c0 2024-04-25T11:42:44.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.226+0000 7f1407400640 1 -- 172.21.15.19:0/2161429490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f13fc011070 con 0x7f14101443c0 2024-04-25T11:42:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.227+0000 7f1407400640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13f00744b0 0x7f13f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.227+0000 7f1407400640 1 -- 172.21.15.19:0/2161429490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5709+0+0 (secure 0 0 0) 0x7f13fc094da0 con 0x7f14101443c0 2024-04-25T11:42:44.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.228+0000 7f140f400640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13f00744b0 0x7f13f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.229+0000 7f140f400640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13f00744b0 0x7f13f00769c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1400034000 tx=0x7f140002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.233+0000 7f1407400640 1 -- 172.21.15.19:0/2161429490 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13fc05f5b0 con 0x7f14101443c0 2024-04-25T11:42:44.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.469+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f141006f950 con 0x7f14101443c0 2024-04-25T11:42:44.458 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:42:44.470+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:42:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.475+0000 7f1407400640 1 -- 172.21.15.19:0/2161429490 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v365) ==== 161+0+0 (secure 0 0 0) 0x7f13fc05eeb0 con 0x7f14101443c0 2024-04-25T11:42:44.462 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13f00744b0 msgr2=0x7f13f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13f00744b0 0x7f13f00769c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1400034000 tx=0x7f140002e000 comp rx=0 tx=0).stop 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 msgr2=0x7f1410160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f1410160670 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c3c0 tx=0x7f13fc00c880 comp rx=0 tx=0).stop 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 shutdown_connections 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f13f00744b0 0x7f13f00769c0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14100c7cb0 0x7f1410160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14101443c0 0x7f1410160670 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 --2- 172.21.15.19:0/2161429490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1410070370 0x7f14100c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 >> 172.21.15.19:0/2161429490 conn(0x7f14100d1ae0 msgr2=0x7f14100b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 shutdown_connections 2024-04-25T11:42:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.478+0000 7f1416800640 1 -- 172.21.15.19:0/2161429490 wait complete. 2024-04-25T11:42:44.495 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T11:42:44.502 INFO:tasks.ceph.mds.b.smithi169.stderr: -6463> 2024-04-25T11:42:34.583+0000 7fb78a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:42:44.651 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2024-04-25T11:42:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.683+0000 7f69d4f29280 1 -- start start 2024-04-25T11:42:44.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.683+0000 7f69d4f29280 1 Processor -- start 2024-04-25T11:42:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0b680 0x558469d18580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0bb00 0x558469d18000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0c400 0x558469d18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d4f29280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4c780 con 0x558469d0bb00 2024-04-25T11:42:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4c600 con 0x558469d0c400 2024-04-25T11:42:44.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55846aa4c480 con 0x558469d0b680 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0bb00 0x558469d18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.684+0000 7f69d1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0bb00 0x558469d18000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56030/0 (socket says 172.21.15.19:56030) 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d1c00640 1 -- 172.21.15.19:0/1872099269 learned_addr learned my addr 172.21.15.19:0/1872099269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d2600640 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0b680 0x558469d18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d3000640 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0c400 0x558469d18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d1c00640 1 -- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0b680 msgr2=0x558469d18580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d1c00640 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0b680 0x558469d18580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d1c00640 1 -- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0c400 msgr2=0x558469d18b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d1c00640 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0c400 0x558469d18b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d1c00640 1 -- 172.21.15.19:0/1872099269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558465c5e780 con 0x558469d0bb00 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d2600640 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0b680 0x558469d18580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:44.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.685+0000 7f69d1c00640 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0bb00 0x558469d18000 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x55846aa878f0 tx=0x558469d5c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849cbf360b2a3b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69cf400640 1 -- 172.21.15.19:0/1872099269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558465c50680 con 0x558469d0bb00 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69cf400640 1 -- 172.21.15.19:0/1872099269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558465c56700 con 0x558469d0bb00 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69cf400640 1 -- 172.21.15.19:0/1872099269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558465c50b60 con 0x558469d0bb00 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69d4f29280 1 -- 172.21.15.19:0/1872099269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558465c5e3c0 con 0x558469d0bb00 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69d4f29280 1 -- 172.21.15.19:0/1872099269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x558465c5f680 con 0x558469d0bb00 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69d4f29280 4 waiting for MDS map... 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69cf400640 1 -- 172.21.15.19:0/1872099269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x55846ab36600 con 0x558469d0bb00 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69cf400640 1 -- 172.21.15.19:0/1872099269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 365) ==== 2885+0+0 (secure 0 0 0) 0x558469cb5180 con 0x558469d0bb00 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69d4f29280 4 Got MDS map 365 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69d4f29280 10 main: JournalTool::main 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.686+0000 7f69d4f29280 4 main: JournalTool: connecting to RADOS... 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.687+0000 7f69d4f29280 1 Processor -- start 2024-04-25T11:42:44.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.688+0000 7f69d4f29280 1 -- start start 2024-04-25T11:42:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0a480 0x558469d19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d4f29280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4c300 con 0x558469d0a900 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4c000 con 0x558469d0ad80 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x558469cbc900 con 0x558469d0a480 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56034/0 (socket says 172.21.15.19:56034) 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d2600640 1 -- 172.21.15.19:0/3292915534 learned_addr learned my addr 172.21.15.19:0/3292915534 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0a480 0x558469d19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.689+0000 7f69d3000640 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69d1c00640 1 -- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0ad80 msgr2=0x558469d19080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69d1c00640 1 -- 172.21.15.19:0/3292915534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a900 msgr2=0x558469d19600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69d1c00640 1 -- 172.21.15.19:0/3292915534 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558465c5f4a0 con 0x558469d0a480 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69d2600640 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0a480 0x558469d19b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55846aa87c20 tx=0x558469d206c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37aebfc284a4b223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69ccc00640 1 -- 172.21.15.19:0/3292915534 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558465c50820 con 0x558469d0a480 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69ccc00640 1 -- 172.21.15.19:0/3292915534 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558465c56fc0 con 0x558469d0a480 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.690+0000 7f69ccc00640 1 -- 172.21.15.19:0/3292915534 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558465c50680 con 0x558469d0a480 2024-04-25T11:42:44.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 -- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0a480 msgr2=0x558469d19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0a480 0x558469d19b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55846aa87c20 tx=0x558469d206c0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 -- 172.21.15.19:0/3292915534 shutdown_connections 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0a480 0x558469d19b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3292915534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 -- 172.21.15.19:0/3292915534 >> 172.21.15.19:0/3292915534 conn(0x558469d0b200 msgr2=0x558469d61800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 -- 172.21.15.19:0/3292915534 shutdown_connections 2024-04-25T11:42:44.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.691+0000 7f69d4f29280 1 -- 172.21.15.19:0/3292915534 wait complete. 2024-04-25T11:42:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.692+0000 7f69d4f29280 1 Processor -- start 2024-04-25T11:42:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.692+0000 7f69d4f29280 1 -- start start 2024-04-25T11:42:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a480 0x558469d19b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d4f29280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4d980 con 0x558469d0a480 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4d800 con 0x558469d0a900 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55846aa4c300 con 0x558469d0ad80 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a480 0x558469d19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a480 0x558469d19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56036/0 (socket says 172.21.15.19:56036) 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d3000640 1 -- 172.21.15.19:0/1492544224 learned_addr learned my addr 172.21.15.19:0/1492544224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d1c00640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.694+0000 7f69d2600640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d3000640 1 -- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0ad80 msgr2=0x558469d19080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d3000640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d3000640 1 -- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0a900 msgr2=0x558469d19600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d3000640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d3000640 1 -- 172.21.15.19:0/1492544224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558465c5e780 con 0x558469d0a480 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d1c00640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d3000640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a480 0x558469d19b80 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x55846ab4d950 tx=0x55846ab52e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69cae00640 1 -- 172.21.15.19:0/1492544224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558465c50820 con 0x558469d0a480 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69cae00640 1 -- 172.21.15.19:0/1492544224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558465c56e00 con 0x558469d0a480 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69cae00640 1 -- 172.21.15.19:0/1492544224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558465c50680 con 0x558469d0a480 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558465c5e3c0 con 0x558469d0a480 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558465c5f680 con 0x558469d0a480 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.695+0000 7f69d4f29280 4 main: JournalTool: resolving pool 14 2024-04-25T11:42:44.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.697+0000 7f69cae00640 1 -- 172.21.15.19:0/1492544224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x55846ab28c00 con 0x558469d0a480 2024-04-25T11:42:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.698+0000 7f69cae00640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x558469d0d180 0x558469d1a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.698+0000 7f69cae00640 1 -- 172.21.15.19:0/1492544224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x55846abb2800 con 0x558469d0a480 2024-04-25T11:42:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.698+0000 7f69d4f29280 4 main: JournalTool: creating IoCtx.. 2024-04-25T11:42:44.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.698+0000 7f69d4f29280 4 main: Executing for rank 0 2024-04-25T11:42:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.699+0000 7f69d2600640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x558469d0d180 0x558469d1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.699+0000 7f69d2600640 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x558469d0d180 0x558469d1a100 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x55846aa87080 tx=0x558469d5a660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.700+0000 7f69d4f29280 1 -- start start 2024-04-25T11:42:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.700+0000 7f69d4f29280 1 Processor -- start 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.701+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0c880 0x558469d1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.701+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0cd00 0x558469d1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.701+0000 7f69d4f29280 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0d600 0x558469d1b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.701+0000 7f69d4f29280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4d680 con 0x558469d0c880 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.701+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55846aa4d500 con 0x558469d0cd00 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.701+0000 7f69d4f29280 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55846aa4d380 con 0x558469d0d600 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0c880 0x558469d1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0c880 0x558469d1a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56042/0 (socket says 172.21.15.19:56042) 2024-04-25T11:42:44.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 -- 172.21.15.19:0/3623623829 learned_addr learned my addr 172.21.15.19:0/3623623829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:44.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0d600 0x558469d1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d2600640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0cd00 0x558469d1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0d600 msgr2=0x558469d1b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0d600 0x558469d1b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0cd00 msgr2=0x558469d1ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0cd00 0x558469d1ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d3000640 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558465c5e000 con 0x558469d0c880 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.702+0000 7f69d2600640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0cd00 0x558469d1ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69d3000640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0c880 0x558469d1a680 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x55846a0ae060 tx=0x55846ab537a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce05d3fa8941a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69c7200640 1 -- 172.21.15.19:0/3623623829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558469fda820 con 0x558469d0c880 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69c7200640 1 -- 172.21.15.19:0/3623623829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558465c56c40 con 0x558469d0c880 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69c7200640 1 -- 172.21.15.19:0/3623623829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558469fda9c0 con 0x558469d0c880 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558465c5e780 con 0x558469d0c880 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x558465c5e3c0 con 0x558469d0c880 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69d4f29280 4 waiting for MDS map... 2024-04-25T11:42:44.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69c7200640 1 -- 172.21.15.19:0/3623623829 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x55846a0b4200 con 0x558469d0c880 2024-04-25T11:42:44.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69c7200640 1 -- 172.21.15.19:0/3623623829 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 365) ==== 2885+0+0 (secure 0 0 0) 0x558469cb5500 con 0x558469d0c880 2024-04-25T11:42:44.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.703+0000 7f69d4f29280 4 Got MDS map 365 2024-04-25T11:42:44.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.704+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558469d0da80 0x558469d1b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.704+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e144) -- 0x55846a0c4400 con 0x558469d0da80 2024-04-25T11:42:44.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.704+0000 7f69d2600640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558469d0da80 0x558469d1b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.704+0000 7f69d2600640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558469d0da80 0x558469d1b700 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=d988551024f377e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.705+0000 7f69d2600640 1 -- 172.21.15.19:0/3623623829 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x558469d52000 con 0x558469d0da80 2024-04-25T11:42:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.705+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e144) -- 0x55846a0c4800 con 0x558469d0da80 2024-04-25T11:42:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.706+0000 7f69d2600640 1 -- 172.21.15.19:0/3623623829 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv22 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x558469d52000 con 0x558469d0da80 2024-04-25T11:42:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.706+0000 7f69c5400640 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e144) -- 0x55846a0c4c00 con 0x558469d0da80 2024-04-25T11:42:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.706+0000 7f69c5400640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55846aa66000 0x55846a0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.706+0000 7f69c5400640 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e144) -- 0x55846a0c5000 con 0x55846aa66000 2024-04-25T11:42:44.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.706+0000 7f69d2600640 1 -- 172.21.15.19:0/3623623829 <== osd.1 v2:172.21.15.19:6800/856233193 3 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x558469d52000 con 0x558469d0da80 2024-04-25T11:42:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.706+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55846aa66000 0x55846a0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.707+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55846aa66000 0x55846a0ce000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=156c7d14b55f870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.708+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558469d52500 con 0x55846aa66000 2024-04-25T11:42:44.695 INFO:teuthology.orchestra.run.smithi019.stdout:old journal was 4226679~2030 2024-04-25T11:42:44.695 INFO:teuthology.orchestra.run.smithi019.stdout:new journal start will be 8388608 (4159899 bytes past old end) 2024-04-25T11:42:44.695 INFO:teuthology.orchestra.run.smithi019.stdout:writing journal head 2024-04-25T11:42:44.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.708+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- osd_op(unknown.0.0:5 14.14 14:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846a0c5800 con 0x558469d0da80 2024-04-25T11:42:44.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.710+0000 7f69d2600640 1 -- 172.21.15.19:0/3623623829 <== osd.1 v2:172.21.15.19:6800/856233193 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v144'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x558469d52000 con 0x558469d0da80 2024-04-25T11:42:44.698 INFO:teuthology.orchestra.run.smithi019.stdout:writing EResetJournal entry 2024-04-25T11:42:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:6 14.1b 14:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846a0c5c00 con 0x55846aa66000 2024-04-25T11:42:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:7 14.b 14:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846aa6a000 con 0x55846aa66000 2024-04-25T11:42:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55846aa66480 0x55846a0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- osd_op(unknown.0.0:8 14.1f 14:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846aa6a400 con 0x55846aa66480 2024-04-25T11:42:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x55846aa66900 0x55846a0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:9 14.1a 14:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846aa6a800 con 0x55846aa66900 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55846aa66d80 0x55846a0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- osd_op(unknown.0.0:10 14.1e 14:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846aa6ac00 con 0x55846aa66d80 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- osd_op(unknown.0.0:11 14.1d 14:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846aa6b000 con 0x55846aa66900 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55846aa66d80 0x55846a0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d3000640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55846aa66480 0x55846a0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.711+0000 7f69d2600640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x55846aa66900 0x55846a0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.712+0000 7f69d1c00640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55846aa66d80 0x55846a0cf080 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=5895a4502e58123c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.712+0000 7f69d3000640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55846aa66480 0x55846a0ce580 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=a932f10efa171615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.714+0000 7f69d2600640 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x55846aa66900 0x55846a0ceb00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=47777b46cc9fc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.714+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== osd_op_reply(6 200.00000002 [delete] v144'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558469d52500 con 0x55846aa66000 2024-04-25T11:42:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.716+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 <== osd.6 v2:172.21.15.169:6824/2510135033 3 ==== osd_op_reply(7 200.00000003 [delete] v144'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558469d52500 con 0x55846aa66000 2024-04-25T11:42:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.716+0000 7f69c5400640 1 -- 172.21.15.19:0/3623623829 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- osd_op(unknown.0.0:12 14.1b 14:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55846a0c5400 con 0x55846aa66000 2024-04-25T11:42:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.716+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== osd_op_reply(10 200.00000006 [delete] v144'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558469d52500 con 0x55846aa66d80 2024-04-25T11:42:44.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.720+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 <== osd.6 v2:172.21.15.169:6824/2510135033 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v144'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x558469d52500 con 0x55846aa66000 2024-04-25T11:42:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.721+0000 7f69d2600640 1 -- 172.21.15.19:0/3623623829 <== osd.4 v2:172.21.15.169:6816/3656468664 1 ==== osd_op_reply(9 200.00000005 [delete] v144'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558469d52000 con 0x55846aa66900 2024-04-25T11:42:44.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.722+0000 7f69d3000640 1 -- 172.21.15.19:0/3623623829 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== osd_op_reply(8 200.00000004 [delete] v144'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558469d52780 con 0x55846aa66480 2024-04-25T11:42:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d2600640 1 -- 172.21.15.19:0/3623623829 <== osd.4 v2:172.21.15.169:6816/3656468664 2 ==== osd_op_reply(11 200.00000007 [delete] v144'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x558469d52000 con 0x55846aa66900 2024-04-25T11:42:44.710 INFO:teuthology.orchestra.run.smithi019.stdout:done 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558469d0da80 msgr2=0x558469d1b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x558469d0da80 0x558469d1b700 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55846aa66d80 msgr2=0x55846a0cf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x55846aa66d80 0x55846a0cf080 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x55846aa66900 msgr2=0x55846a0ceb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x55846aa66900 0x55846a0ceb00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55846aa66000 msgr2=0x55846a0ce000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x55846aa66000 0x55846a0ce000 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55846aa66480 msgr2=0x55846a0ce580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x55846aa66480 0x55846a0ce580 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0c880 msgr2=0x558469d1a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 --2- 172.21.15.19:0/3623623829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0c880 0x558469d1a680 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x55846a0ae060 tx=0x55846ab537a0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 reap_dead start 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 reap_dead start 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 shutdown_connections 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 >> 172.21.15.19:0/3623623829 conn(0x558469d0a000 msgr2=0x55846a058800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d1c00640 1 -- 172.21.15.19:0/3623623829 reap_dead start 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 shutdown_connections 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.723+0000 7f69d4f29280 1 -- 172.21.15.19:0/3623623829 wait complete. 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x558469d0d180 msgr2=0x558469d1a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x558469d0d180 0x558469d1a100 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x55846aa87080 tx=0x558469d5a660 comp rx=0 tx=0).stop 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a480 msgr2=0x558469d19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a480 0x558469d19b80 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x55846ab4d950 tx=0x55846ab52e40 comp rx=0 tx=0).stop 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 shutdown_connections 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x558469d0d180 0x558469d1a100 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0a900 0x558469d19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0ad80 0x558469d19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1492544224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0a480 0x558469d19b80 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 >> 172.21.15.19:0/1492544224 conn(0x558469d0b200 msgr2=0x558469d61800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 shutdown_connections 2024-04-25T11:42:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.724+0000 7f69d4f29280 1 -- 172.21.15.19:0/1492544224 wait complete. 2024-04-25T11:42:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 -- 172.21.15.19:0/1872099269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0bb00 msgr2=0x558469d18000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0bb00 0x558469d18000 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x55846aa878f0 tx=0x558469d5c120 comp rx=0 tx=0).stop 2024-04-25T11:42:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 -- 172.21.15.19:0/1872099269 shutdown_connections 2024-04-25T11:42:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558469d0c400 0x558469d18b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558469d0bb00 0x558469d18000 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 --2- 172.21.15.19:0/1872099269 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558469d0b680 0x558469d18580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 -- 172.21.15.19:0/1872099269 >> 172.21.15.19:0/1872099269 conn(0x558469d0bf80 msgr2=0x558469d61000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 -- 172.21.15.19:0/1872099269 shutdown_connections 2024-04-25T11:42:44.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:44.725+0000 7f69d4f29280 1 -- 172.21.15.19:0/1872099269 wait complete. 2024-04-25T11:42:44.726 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.231131, result: old journal was 4226679~2030 new journal start will be 8388608 (4159899 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T11:42:44.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T11:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.019+0000 7f308a200640 1 Processor -- start 2024-04-25T11:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.019+0000 7f308a200640 1 -- start start 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.021+0000 7f308a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f308406e760 0x7f30840d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.021+0000 7f308a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f30840d1270 0x7f30840d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.021+0000 7f308a200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3084161970 con 0x7f308415e0c0 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.021+0000 7f308a200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f30841614e0 con 0x7f308406e760 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.021+0000 7f308a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f30840d1a40 con 0x7f30840d1270 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3082a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f308406e760 0x7f30840d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3082a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f308406e760 0x7f30840d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56050/0 (socket says 172.21.15.19:56050) 2024-04-25T11:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3082a00640 1 -- 172.21.15.19:0/2498799179 learned_addr learned my addr 172.21.15.19:0/2498799179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3082a00640 1 -- 172.21.15.19:0/2498799179 >> v2:172.21.15.169:3300/0 conn(0x7f30840d1270 msgr2=0x7f30840d49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3083e00640 1 --2- 172.21.15.19:0/2498799179 >> v2:172.21.15.169:3300/0 conn(0x7f30840d1270 0x7f30840d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3082a00640 1 --2- 172.21.15.19:0/2498799179 >> v2:172.21.15.169:3300/0 conn(0x7f30840d1270 0x7f30840d49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3082a00640 1 -- 172.21.15.19:0/2498799179 >> v1:172.21.15.169:6789/0 conn(0x7f308415e0c0 legacy=0x7f308406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3082a00640 1 -- 172.21.15.19:0/2498799179 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3084160bb0 con 0x7f308406e760 2024-04-25T11:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.022+0000 7f3083e00640 1 --2- 172.21.15.19:0/2498799179 >> v2:172.21.15.169:3300/0 conn(0x7f30840d1270 0x7f30840d49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.023+0000 7f3082a00640 1 --2- 172.21.15.19:0/2498799179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308406e760 0x7f30840d0d30 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f30840b5790 tx=0x7f307002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997022c4bc3744e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.024+0000 7f3082000640 1 -- 172.21.15.19:0/2498799179 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070037070 con 0x7f308406e760 2024-04-25T11:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.024+0000 7f3082000640 1 -- 172.21.15.19:0/2498799179 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3070032440 con 0x7f308406e760 2024-04-25T11:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.024+0000 7f3082000640 1 -- 172.21.15.19:0/2498799179 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070032810 con 0x7f308406e760 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.024+0000 7f308a200640 1 -- 172.21.15.19:0/2498799179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308406e760 msgr2=0x7f30840d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.024+0000 7f308a200640 1 --2- 172.21.15.19:0/2498799179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308406e760 0x7f30840d0d30 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f30840b5790 tx=0x7f307002a230 comp rx=0 tx=0).stop 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.025+0000 7f308a200640 1 -- 172.21.15.19:0/2498799179 shutdown_connections 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.025+0000 7f308a200640 1 --2- 172.21.15.19:0/2498799179 >> v2:172.21.15.169:3300/0 conn(0x7f30840d1270 0x7f30840d49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.025+0000 7f308a200640 1 --2- 172.21.15.19:0/2498799179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308406e760 0x7f30840d0d30 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.025+0000 7f308a200640 1 -- 172.21.15.19:0/2498799179 >> 172.21.15.19:0/2498799179 conn(0x7f30840d2800 msgr2=0x7f30840d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.025+0000 7f308a200640 1 -- 172.21.15.19:0/2498799179 shutdown_connections 2024-04-25T11:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.025+0000 7f308a200640 1 -- 172.21.15.19:0/2498799179 wait complete. 2024-04-25T11:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.027+0000 7f308a200640 1 Processor -- start 2024-04-25T11:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.027+0000 7f308a200640 1 -- start start 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f308a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f308406e760 0x7f30840cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f308a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30840d1270 0x7f308414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f308a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308415e0c0 0x7f30840cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f308a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30840d1680 con 0x7f308415e0c0 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f308a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3084161170 con 0x7f308406e760 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f308a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f30840d1800 con 0x7f30840d1270 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f3083e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308415e0c0 0x7f30840cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f3083e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308415e0c0 0x7f30840cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56058/0 (socket says 172.21.15.19:56058) 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.029+0000 7f3083e00640 1 -- 172.21.15.19:0/3399868026 learned_addr learned my addr 172.21.15.19:0/3399868026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3083400640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f308406e760 0x7f30840cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3082a00640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30840d1270 0x7f308414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3083e00640 1 -- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30840d1270 msgr2=0x7f308414f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3083e00640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30840d1270 0x7f308414f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3083e00640 1 -- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f308406e760 msgr2=0x7f30840cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3083e00640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f308406e760 0x7f30840cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3083e00640 1 -- 172.21.15.19:0/3399868026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30840cfc30 con 0x7f308415e0c0 2024-04-25T11:42:45.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3082a00640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30840d1270 0x7f308414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f3083e00640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308415e0c0 0x7f30840cadf0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cd50 tx=0x7f307c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.030+0000 7f307b400640 1 -- 172.21.15.19:0/3399868026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00f680 con 0x7f308415e0c0 2024-04-25T11:42:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.031+0000 7f307b400640 1 -- 172.21.15.19:0/3399868026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c00f840 con 0x7f308415e0c0 2024-04-25T11:42:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.031+0000 7f307b400640 1 -- 172.21.15.19:0/3399868026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00fc30 con 0x7f308415e0c0 2024-04-25T11:42:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.031+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308415a490 con 0x7f308415e0c0 2024-04-25T11:42:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.031+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308415aa50 con 0x7f308415e0c0 2024-04-25T11:42:45.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.032+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30840cfeb0 con 0x7f308415e0c0 2024-04-25T11:42:45.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.033+0000 7f307b400640 1 -- 172.21.15.19:0/3399868026 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f307c0213f0 con 0x7f308415e0c0 2024-04-25T11:42:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.034+0000 7f307b400640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3064074500 0x7f3064076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.034+0000 7f307b400640 1 -- 172.21.15.19:0/3399868026 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f307c096080 con 0x7f308415e0c0 2024-04-25T11:42:45.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.035+0000 7f3083400640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3064074500 0x7f3064076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.036+0000 7f3083400640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3064074500 0x7f3064076a10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3074006fd0 tx=0x7f3074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:45.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.040+0000 7f307b400640 1 -- 172.21.15.19:0/3399868026 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f307c014030 con 0x7f308415e0c0 2024-04-25T11:42:45.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.286+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f30840d0140 con 0x7f308415e0c0 2024-04-25T11:42:45.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.489+0000 7f307b400640 1 -- 172.21.15.19:0/3399868026 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v366) ==== 162+0+0 (secure 0 0 0) 0x7f30840d0140 con 0x7f308415e0c0 2024-04-25T11:42:45.477 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T11:42:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3064074500 msgr2=0x7f3064076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3064074500 0x7f3064076a10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3074006fd0 tx=0x7f3074008040 comp rx=0 tx=0).stop 2024-04-25T11:42:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308415e0c0 msgr2=0x7f30840cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308415e0c0 0x7f30840cadf0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cd50 tx=0x7f307c00a650 comp rx=0 tx=0).stop 2024-04-25T11:42:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 shutdown_connections 2024-04-25T11:42:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3064074500 0x7f3064076a10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f308415e0c0 0x7f30840cadf0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30840d1270 0x7f308414f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 --2- 172.21.15.19:0/3399868026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f308406e760 0x7f30840cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.493+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 >> 172.21.15.19:0/3399868026 conn(0x7f30840d2800 msgr2=0x7f308415e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.494+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 shutdown_connections 2024-04-25T11:42:45.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.494+0000 7f308a200640 1 -- 172.21.15.19:0/3399868026 wait complete. 2024-04-25T11:42:45.513 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:45.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.804+0000 7ff8db200640 1 Processor -- start 2024-04-25T11:42:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.804+0000 7ff8db200640 1 -- start start 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8db200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc168720 0x7ff8dc0cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8db200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff8dc06c5c0 0x7ff8dc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8db200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8dc071180 con 0x7ff8dc0cf2f0 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8db200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff8dc070cf0 con 0x7ff8dc06c5c0 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8db200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc0700e0 con 0x7ff8dc168720 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8d9e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff8dc0cf2f0 0x7ff8dc168010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43024/0 (socket says 172.21.15.19:43024) 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8d9e00640 1 -- 172.21.15.19:0/2975907740 learned_addr learned my addr 172.21.15.19:0/2975907740 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:45.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8da800640 1 --2- 172.21.15.19:0/2975907740 >> v2:172.21.15.19:3300/0 conn(0x7ff8dc06c5c0 0x7ff8dc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.807+0000 7ff8d1400640 1 --2- 172.21.15.19:0/2975907740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc168720 0x7ff8dc0cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.808+0000 7ff8da800640 1 -- 172.21.15.19:0/2975907740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc168720 msgr2=0x7ff8dc0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.808+0000 7ff8da800640 1 --2- 172.21.15.19:0/2975907740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc168720 0x7ff8dc0cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.808+0000 7ff8da800640 1 -- 172.21.15.19:0/2975907740 >> v1:172.21.15.19:6789/0 conn(0x7ff8dc0cf2f0 legacy=0x7ff8dc168010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.808+0000 7ff8da800640 1 -- 172.21.15.19:0/2975907740 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc0703c0 con 0x7ff8dc06c5c0 2024-04-25T11:42:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.809+0000 7ff8da800640 1 --2- 172.21.15.19:0/2975907740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc06c5c0 0x7ff8dc06c9d0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40097a0 tx=0x7ff8d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6224ed624427713e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.809+0000 7ff8d9400640 1 -- 172.21.15.19:0/2975907740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4037070 con 0x7ff8dc06c5c0 2024-04-25T11:42:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.809+0000 7ff8d9400640 1 -- 172.21.15.19:0/2975907740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d4032440 con 0x7ff8dc06c5c0 2024-04-25T11:42:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.809+0000 7ff8d9400640 1 -- 172.21.15.19:0/2975907740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4032810 con 0x7ff8dc06c5c0 2024-04-25T11:42:45.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 -- 172.21.15.19:0/2975907740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc06c5c0 msgr2=0x7ff8dc06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 --2- 172.21.15.19:0/2975907740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc06c5c0 0x7ff8dc06c9d0 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40097a0 tx=0x7ff8d402a180 comp rx=0 tx=0).stop 2024-04-25T11:42:45.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 -- 172.21.15.19:0/2975907740 shutdown_connections 2024-04-25T11:42:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 --2- 172.21.15.19:0/2975907740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc06c5c0 0x7ff8dc06c9d0 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 --2- 172.21.15.19:0/2975907740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc168720 0x7ff8dc0cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 -- 172.21.15.19:0/2975907740 >> 172.21.15.19:0/2975907740 conn(0x7ff8dc0d1720 msgr2=0x7ff8dc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 -- 172.21.15.19:0/2975907740 shutdown_connections 2024-04-25T11:42:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.810+0000 7ff8db200640 1 -- 172.21.15.19:0/2975907740 wait complete. 2024-04-25T11:42:45.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.812+0000 7ff8db200640 1 Processor -- start 2024-04-25T11:42:45.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.813+0000 7ff8db200640 1 -- start start 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8db200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc06c5c0 0x7ff8dc1f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8db200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8dc0cf2f0 0x7ff8dc14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8db200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc1fe570 0x7ff8dc1fe980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8db200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc168d00 con 0x7ff8dc1fe570 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8db200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc070980 con 0x7ff8dc0cf2f0 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8db200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc168e80 con 0x7ff8dc06c5c0 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8da800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc1fe570 0x7ff8dc1fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8da800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc1fe570 0x7ff8dc1fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56070/0 (socket says 172.21.15.19:56070) 2024-04-25T11:42:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.814+0000 7ff8da800640 1 -- 172.21.15.19:0/2944872974 learned_addr learned my addr 172.21.15.19:0/2944872974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8d1400640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8dc0cf2f0 0x7ff8dc14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8d9e00640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc06c5c0 0x7ff8dc1f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8da800640 1 -- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc06c5c0 msgr2=0x7ff8dc1f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8da800640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc06c5c0 0x7ff8dc1f0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8da800640 1 -- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8dc0cf2f0 msgr2=0x7ff8dc14d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8da800640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8dc0cf2f0 0x7ff8dc14d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8da800640 1 -- 172.21.15.19:0/2944872974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc1feec0 con 0x7ff8dc1fe570 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8d1400640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8dc0cf2f0 0x7ff8dc14d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.815+0000 7ff8da800640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc1fe570 0x7ff8dc1fe980 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40098c0 tx=0x7ff8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.816+0000 7ff8d2a00640 1 -- 172.21.15.19:0/2944872974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4037040 con 0x7ff8dc1fe570 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.816+0000 7ff8d2a00640 1 -- 172.21.15.19:0/2944872974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d4042070 con 0x7ff8dc1fe570 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.816+0000 7ff8d2a00640 1 -- 172.21.15.19:0/2944872974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4030b50 con 0x7ff8dc1fe570 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.816+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc1f1970 con 0x7ff8dc1fe570 2024-04-25T11:42:45.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.816+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc15f1c0 con 0x7ff8dc1fe570 2024-04-25T11:42:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.817+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc1f1e90 con 0x7ff8dc1fe570 2024-04-25T11:42:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.818+0000 7ff8d2a00640 1 -- 172.21.15.19:0/2944872974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff8d4030cf0 con 0x7ff8dc1fe570 2024-04-25T11:42:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.819+0000 7ff8d2a00640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff8b80744b0 0x7ff8b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.819+0000 7ff8d2a00640 1 -- 172.21.15.19:0/2944872974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7ff8d40b5da0 con 0x7ff8dc1fe570 2024-04-25T11:42:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.820+0000 7ff8d9e00640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff8b80744b0 0x7ff8b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.821+0000 7ff8d9e00640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff8b80744b0 0x7ff8b80769c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0012e0 tx=0x7ff8cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:45.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:45.824+0000 7ff8d2a00640 1 -- 172.21.15.19:0/2944872974 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8dc1f1e90 con 0x7ff8dc1fe570 2024-04-25T11:42:46.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.071+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8dc050740 con 0x7ff8dc1fe570 2024-04-25T11:42:46.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.072+0000 7ff8d2a00640 1 -- 172.21.15.19:0/2944872974 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 367 v367) ==== 95+0+5187 (secure 0 0 0) 0x7ff8d404c020 con 0x7ff8dc1fe570 2024-04-25T11:42:46.060 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:46.060 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":367,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":367,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:45.497042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-25T11:42:46.060 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 367 2024-04-25T11:42:46.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff8b80744b0 msgr2=0x7ff8b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff8b80744b0 0x7ff8b80769c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0012e0 tx=0x7ff8cc009040 comp rx=0 tx=0).stop 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc1fe570 msgr2=0x7ff8dc1fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc1fe570 0x7ff8dc1fe980 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40098c0 tx=0x7ff8d40046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 shutdown_connections 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff8b80744b0 0x7ff8b80769c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8dc0cf2f0 0x7ff8dc14d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8dc1fe570 0x7ff8dc1fe980 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 --2- 172.21.15.19:0/2944872974 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff8dc06c5c0 0x7ff8dc1f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 >> 172.21.15.19:0/2944872974 conn(0x7ff8dc0d1720 msgr2=0x7ff8dc0b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:46.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 shutdown_connections 2024-04-25T11:42:46.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:46.075+0000 7ff8db200640 1 -- 172.21.15.19:0/2944872974 wait complete. 2024-04-25T11:42:46.492 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 367, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:42:29.193619+0000', 'modified': '2024-04-25T11:42:45.497042+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 144, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8020}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8020': {'gid': 8020, 'name': 'c', 'rank': 0, 'incarnation': 367, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6834/3539191189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 3539191189}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 3539191189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:42:46.492 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8020:up:replay 2024-04-25T11:42:47.494 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:47.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.801+0000 7f9d16200640 1 Processor -- start 2024-04-25T11:42:47.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.802+0000 7f9d16200640 1 -- start start 2024-04-25T11:42:47.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.803+0000 7f9d16200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d100b9050 0x7f9d100b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.803+0000 7f9d16200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d100b71d0 0x7f9d100b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.803+0000 7f9d16200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d100cfa70 con 0x7f9d100cf660 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.803+0000 7f9d16200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9d100d2ba0 con 0x7f9d100b71d0 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.803+0000 7f9d16200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f9d100d28c0 con 0x7f9d100b9050 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d15800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d100b71d0 0x7f9d100b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d15800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d100b71d0 0x7f9d100b75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56086/0 (socket says 172.21.15.19:56086) 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d15800640 1 -- 172.21.15.19:0/1430523584 learned_addr learned my addr 172.21.15.19:0/1430523584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d14e00640 1 --2- 172.21.15.19:0/1430523584 >> v2:172.21.15.169:3300/0 conn(0x7f9d100b9050 0x7f9d100b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d15800640 1 -- 172.21.15.19:0/1430523584 >> v2:172.21.15.169:3300/0 conn(0x7f9d100b9050 msgr2=0x7f9d100b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d15800640 1 --2- 172.21.15.19:0/1430523584 >> v2:172.21.15.169:3300/0 conn(0x7f9d100b9050 0x7f9d100b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d15800640 1 -- 172.21.15.19:0/1430523584 >> v1:172.21.15.19:6789/0 conn(0x7f9d100cf660 legacy=0x7f9d100ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d15800640 1 -- 172.21.15.19:0/1430523584 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1009cc50 con 0x7f9d100b71d0 2024-04-25T11:42:47.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.804+0000 7f9d0fe00640 1 -- 172.21.15.19:0/1430523584 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795486502 0 0) 0x7f9d100cfa70 con 0x7f9d100cf660 2024-04-25T11:42:47.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.805+0000 7f9d15800640 1 --2- 172.21.15.19:0/1430523584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d100b75e0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f9d10074710 tx=0x7f9d0802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc07ef2b09c1eec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:47.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.806+0000 7f9d0fe00640 1 -- 172.21.15.19:0/1430523584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08036070 con 0x7f9d100b71d0 2024-04-25T11:42:47.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.806+0000 7f9d0fe00640 1 -- 172.21.15.19:0/1430523584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d08031600 con 0x7f9d100b71d0 2024-04-25T11:42:47.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.806+0000 7f9d0fe00640 1 -- 172.21.15.19:0/1430523584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d080319d0 con 0x7f9d100b71d0 2024-04-25T11:42:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 -- 172.21.15.19:0/1430523584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 msgr2=0x7f9d100b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 --2- 172.21.15.19:0/1430523584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d100b75e0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f9d10074710 tx=0x7f9d0802a730 comp rx=0 tx=0).stop 2024-04-25T11:42:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 -- 172.21.15.19:0/1430523584 shutdown_connections 2024-04-25T11:42:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 --2- 172.21.15.19:0/1430523584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d100b75e0 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 --2- 172.21.15.19:0/1430523584 >> v2:172.21.15.169:3300/0 conn(0x7f9d100b9050 0x7f9d100b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 -- 172.21.15.19:0/1430523584 >> 172.21.15.19:0/1430523584 conn(0x7f9d100d03f0 msgr2=0x7f9d100d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:47.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 -- 172.21.15.19:0/1430523584 shutdown_connections 2024-04-25T11:42:47.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.807+0000 7f9d16200640 1 -- 172.21.15.19:0/1430523584 wait complete. 2024-04-25T11:42:47.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.809+0000 7f9d16200640 1 Processor -- start 2024-04-25T11:42:47.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.809+0000 7f9d16200640 1 -- start start 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d16200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d101e3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d16200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100b9050 0x7f9d1009fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d16200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d100cf660 0x7f9d101e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d16200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d100d2550 con 0x7f9d100b71d0 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d16200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9d100cf0b0 con 0x7f9d100b9050 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d16200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f9d100b7b20 con 0x7f9d100cf660 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d14e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d101e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d14e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d101e3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56088/0 (socket says 172.21.15.19:56088) 2024-04-25T11:42:47.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.811+0000 7f9d14e00640 1 -- 172.21.15.19:0/2184933978 learned_addr learned my addr 172.21.15.19:0/2184933978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.812+0000 7f9d07e00640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100b9050 0x7f9d1009fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.812+0000 7f9d14e00640 1 -- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d100cf660 msgr2=0x7f9d101e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.812+0000 7f9d14e00640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d100cf660 0x7f9d101e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.812+0000 7f9d14e00640 1 -- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100b9050 msgr2=0x7f9d1009fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.812+0000 7f9d14e00640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100b9050 0x7f9d1009fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.812+0000 7f9d14e00640 1 -- 172.21.15.19:0/2184933978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d101e38d0 con 0x7f9d100b71d0 2024-04-25T11:42:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.812+0000 7f9d14e00640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d101e3390 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800d800 tx=0x7f9cf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.813+0000 7f9d0d600640 1 -- 172.21.15.19:0/2184933978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8018020 con 0x7f9d100b71d0 2024-04-25T11:42:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.813+0000 7f9d0d600640 1 -- 172.21.15.19:0/2184933978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cf8009660 con 0x7f9d100b71d0 2024-04-25T11:42:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.813+0000 7f9d0d600640 1 -- 172.21.15.19:0/2184933978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8009a50 con 0x7f9d100b71d0 2024-04-25T11:42:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.813+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d100a09d0 con 0x7f9d100b71d0 2024-04-25T11:42:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.813+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d100a0f90 con 0x7f9d100b71d0 2024-04-25T11:42:47.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.814+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d10050740 con 0x7f9d100b71d0 2024-04-25T11:42:47.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.816+0000 7f9d0d600640 1 -- 172.21.15.19:0/2184933978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9cf801b020 con 0x7f9d100b71d0 2024-04-25T11:42:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.817+0000 7f9d0d600640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9cec0744b0 0x7f9cec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.817+0000 7f9d0d600640 1 -- 172.21.15.19:0/2184933978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f9cf80958c0 con 0x7f9d100b71d0 2024-04-25T11:42:47.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.817+0000 7f9d07e00640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9cec0744b0 0x7f9cec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.819+0000 7f9d07e00640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9cec0744b0 0x7f9cec0769c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9d100d1280 tx=0x7f9d00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:47.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:47.821+0000 7f9d0d600640 1 -- 172.21.15.19:0/2184933978 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9cf8022020 con 0x7f9d100b71d0 2024-04-25T11:42:48.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.070+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d100b8050 con 0x7f9d100b71d0 2024-04-25T11:42:48.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.072+0000 7f9d0d600640 1 -- 172.21.15.19:0/2184933978 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 369 v369) ==== 95+0+5187 (secure 0 0 0) 0x7f9cf8064e20 con 0x7f9d100b71d0 2024-04-25T11:42:48.059 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:48.059 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":369,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":369,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:46.518078+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-25T11:42:48.060 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 369 2024-04-25T11:42:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9cec0744b0 msgr2=0x7f9cec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9cec0744b0 0x7f9cec0769c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9d100d1280 tx=0x7f9d00009040 comp rx=0 tx=0).stop 2024-04-25T11:42:48.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 msgr2=0x7f9d101e3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d101e3390 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800d800 tx=0x7f9cf800dcc0 comp rx=0 tx=0).stop 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 shutdown_connections 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f9cec0744b0 0x7f9cec0769c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d100b9050 0x7f9d1009fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d100cf660 0x7f9d101e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 --2- 172.21.15.19:0/2184933978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d100b71d0 0x7f9d101e3390 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 >> 172.21.15.19:0/2184933978 conn(0x7f9d100d03f0 msgr2=0x7f9d10071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 shutdown_connections 2024-04-25T11:42:48.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:48.075+0000 7f9d16200640 1 -- 172.21.15.19:0/2184933978 wait complete. 2024-04-25T11:42:48.494 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 369, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:42:29.193619+0000', 'modified': '2024-04-25T11:42:46.518078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 144, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8020}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8020': {'gid': 8020, 'name': 'c', 'rank': 0, 'incarnation': 367, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.19:6834/3539191189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 3539191189}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 3539191189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T11:42:48.494 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8020:up:rejoin 2024-04-25T11:42:48.545 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:42:48.558+0000 7f90ae800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:42:49.494 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.790+0000 7f083d200640 1 Processor -- start 2024-04-25T11:42:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.790+0000 7f083d200640 1 -- start start 2024-04-25T11:42:49.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f083d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08380d1070 0x7f0838162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:49.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f083d200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0838144e60 0x7f0838145270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:49.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f083d200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083806fc70 con 0x7f0838144350 2024-04-25T11:42:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f083d200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f083806ff50 con 0x7f08380d1070 2024-04-25T11:42:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f083d200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f0838070230 con 0x7f0838144e60 2024-04-25T11:42:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f082fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08380d1070 0x7f0838162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f082fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08380d1070 0x7f0838162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56100/0 (socket says 172.21.15.19:56100) 2024-04-25T11:42:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.793+0000 7f082fe00640 1 -- 172.21.15.19:0/621430567 learned_addr learned my addr 172.21.15.19:0/621430567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:49.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.794+0000 7f0837e00640 1 --2- 172.21.15.19:0/621430567 >> v2:172.21.15.169:3300/0 conn(0x7f0838144e60 0x7f0838145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.794+0000 7f082fe00640 1 -- 172.21.15.19:0/621430567 >> v2:172.21.15.169:3300/0 conn(0x7f0838144e60 msgr2=0x7f0838145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.794+0000 7f082fe00640 1 --2- 172.21.15.19:0/621430567 >> v2:172.21.15.169:3300/0 conn(0x7f0838144e60 0x7f0838145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.794+0000 7f082fe00640 1 -- 172.21.15.19:0/621430567 >> v1:172.21.15.169:6789/0 conn(0x7f0838144350 legacy=0x7f0838161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.794+0000 7f082fe00640 1 -- 172.21.15.19:0/621430567 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0838070510 con 0x7f08380d1070 2024-04-25T11:42:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.795+0000 7f082fe00640 1 --2- 172.21.15.19:0/621430567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08380d1070 0x7f0838162200 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f08380b4820 tx=0x7f082002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680dbaae1a80b346 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.795+0000 7f0836a00640 1 -- 172.21.15.19:0/621430567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820037070 con 0x7f08380d1070 2024-04-25T11:42:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.795+0000 7f0836a00640 1 -- 172.21.15.19:0/621430567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0820032440 con 0x7f08380d1070 2024-04-25T11:42:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.795+0000 7f0836a00640 1 -- 172.21.15.19:0/621430567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820032810 con 0x7f08380d1070 2024-04-25T11:42:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 -- 172.21.15.19:0/621430567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08380d1070 msgr2=0x7f0838162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 --2- 172.21.15.19:0/621430567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08380d1070 0x7f0838162200 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f08380b4820 tx=0x7f082002a230 comp rx=0 tx=0).stop 2024-04-25T11:42:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 -- 172.21.15.19:0/621430567 shutdown_connections 2024-04-25T11:42:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 --2- 172.21.15.19:0/621430567 >> v2:172.21.15.169:3300/0 conn(0x7f0838144e60 0x7f0838145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 --2- 172.21.15.19:0/621430567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08380d1070 0x7f0838162200 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 -- 172.21.15.19:0/621430567 >> 172.21.15.19:0/621430567 conn(0x7f08380d1720 msgr2=0x7f08380d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 -- 172.21.15.19:0/621430567 shutdown_connections 2024-04-25T11:42:49.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.796+0000 7f083d200640 1 -- 172.21.15.19:0/621430567 wait complete. 2024-04-25T11:42:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.799+0000 7f083d200640 1 Processor -- start 2024-04-25T11:42:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.799+0000 7f083d200640 1 -- start start 2024-04-25T11:42:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.800+0000 7f083d200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08380d1070 0x7f0838160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.800+0000 7f083d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838144350 0x7f0838160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.800+0000 7f083d200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0838144e60 0x7f08381e7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.800+0000 7f083d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0838162740 con 0x7f0838144350 2024-04-25T11:42:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.800+0000 7f083d200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0838147910 con 0x7f08380d1070 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.800+0000 7f083d200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f08381628c0 con 0x7f0838144e60 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838144350 0x7f0838160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838144350 0x7f0838160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56114/0 (socket says 172.21.15.19:56114) 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 -- 172.21.15.19:0/2194407399 learned_addr learned my addr 172.21.15.19:0/2194407399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 -- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0838144e60 msgr2=0x7f08381e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0838144e60 0x7f08381e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 -- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08380d1070 msgr2=0x7f0838160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08380d1070 0x7f0838160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:49.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 -- 172.21.15.19:0/2194407399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083814e350 con 0x7f0838144350 2024-04-25T11:42:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.801+0000 7f082fe00640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838144350 0x7f0838160c40 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f0820000c00 tx=0x7f08200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.802+0000 7f082f400640 1 -- 172.21.15.19:0/2194407399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820041070 con 0x7f0838144350 2024-04-25T11:42:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.802+0000 7f082f400640 1 -- 172.21.15.19:0/2194407399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f082003c890 con 0x7f0838144350 2024-04-25T11:42:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.802+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083814e630 con 0x7f0838144350 2024-04-25T11:42:49.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.802+0000 7f082f400640 1 -- 172.21.15.19:0/2194407399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820037040 con 0x7f0838144350 2024-04-25T11:42:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.802+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083814ebf0 con 0x7f0838144350 2024-04-25T11:42:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.803+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083813f7a0 con 0x7f0838144350 2024-04-25T11:42:49.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.804+0000 7f082f400640 1 -- 172.21.15.19:0/2194407399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0820044070 con 0x7f0838144350 2024-04-25T11:42:49.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.805+0000 7f082f400640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08140744e0 0x7f08140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:49.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.805+0000 7f082f400640 1 -- 172.21.15.19:0/2194407399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f08200b5e50 con 0x7f0838144350 2024-04-25T11:42:49.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.806+0000 7f0837400640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08140744e0 0x7f08140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.807+0000 7f0837400640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08140744e0 0x7f08140769f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0828006fd0 tx=0x7f0828006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:49.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:49.810+0000 7f082f400640 1 -- 172.21.15.19:0/2194407399 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0820080570 con 0x7f0838144350 2024-04-25T11:42:50.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.054+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0838050740 con 0x7f0838144350 2024-04-25T11:42:50.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.056+0000 7f082f400640 1 -- 172.21.15.19:0/2194407399 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 370 v370) ==== 95+0+5194 (secure 0 0 0) 0x7f082007fe70 con 0x7f0838144350 2024-04-25T11:42:50.043 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:50.043 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":370,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":370,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:48.519583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":9,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:42:50.043 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 370 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.058+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08140744e0 msgr2=0x7f08140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.058+0000 7f083d200640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08140744e0 0x7f08140769f0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0828006fd0 tx=0x7f0828006bc0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.058+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838144350 msgr2=0x7f0838160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.058+0000 7f083d200640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838144350 0x7f0838160c40 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f0820000c00 tx=0x7f08200046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.058+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 shutdown_connections 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.058+0000 7f083d200640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f08140744e0 0x7f08140769f0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.058+0000 7f083d200640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0838144e60 0x7f08381e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.059+0000 7f083d200640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0838144350 0x7f0838160c40 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.059+0000 7f083d200640 1 --2- 172.21.15.19:0/2194407399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08380d1070 0x7f0838160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.059+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 >> 172.21.15.19:0/2194407399 conn(0x7f08380d1720 msgr2=0x7f0838072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.059+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 shutdown_connections 2024-04-25T11:42:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.059+0000 7f083d200640 1 -- 172.21.15.19:0/2194407399 wait complete. 2024-04-25T11:42:50.474 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 370, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:42:29.193619+0000', 'modified': '2024-04-25T11:42:48.519583+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 144, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8020}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8020': {'gid': 8020, 'name': 'c', 'rank': 0, 'incarnation': 367, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.19:6834/3539191189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 3539191189}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 3539191189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8020, 'qdb_cluster': [8020]} 2024-04-25T11:42:50.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:42:50.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:42:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.778+0000 7f82a1000640 1 Processor -- start 2024-04-25T11:42:50.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.778+0000 7f82a1000640 1 -- start start 2024-04-25T11:42:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.780+0000 7f82a1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c06c5c0 0x7f829c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.780+0000 7f82a1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f829c144370 0x7f829c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f82a1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f829c0b50e0 con 0x7f829c0b4cd0 2024-04-25T11:42:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f82a1000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f829c0d1230 con 0x7f829c144370 2024-04-25T11:42:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f82a1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f829c0d06b0 con 0x7f829c06c5c0 2024-04-25T11:42:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f8293e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f829c144370 0x7f829c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f8293e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f829c144370 0x7f829c0b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56118/0 (socket says 172.21.15.19:56118) 2024-04-25T11:42:50.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f8293e00640 1 -- 172.21.15.19:0/1760300811 learned_addr learned my addr 172.21.15.19:0/1760300811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:50.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f829b400640 1 --2- 172.21.15.19:0/1760300811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c06c5c0 0x7f829c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f8293e00640 1 -- 172.21.15.19:0/1760300811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c06c5c0 msgr2=0x7f829c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f8293e00640 1 --2- 172.21.15.19:0/1760300811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c06c5c0 0x7f829c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f8293e00640 1 -- 172.21.15.19:0/1760300811 >> v1:172.21.15.169:6789/0 conn(0x7f829c0b4cd0 legacy=0x7f829c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f8293e00640 1 -- 172.21.15.19:0/1760300811 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c0d0990 con 0x7f829c144370 2024-04-25T11:42:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.781+0000 7f829b400640 1 --2- 172.21.15.19:0/1760300811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c06c5c0 0x7f829c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.783+0000 7f8293e00640 1 --2- 172.21.15.19:0/1760300811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c0b4790 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f8284009870 tx=0x7f828402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc4e769d357eade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:50.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.783+0000 7f829aa00640 1 -- 172.21.15.19:0/1760300811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284037070 con 0x7f829c144370 2024-04-25T11:42:50.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.783+0000 7f829aa00640 1 -- 172.21.15.19:0/1760300811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8284032440 con 0x7f829c144370 2024-04-25T11:42:50.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.783+0000 7f829aa00640 1 -- 172.21.15.19:0/1760300811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284032810 con 0x7f829c144370 2024-04-25T11:42:50.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 -- 172.21.15.19:0/1760300811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 msgr2=0x7f829c0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 --2- 172.21.15.19:0/1760300811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c0b4790 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f8284009870 tx=0x7f828402a200 comp rx=0 tx=0).stop 2024-04-25T11:42:50.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 -- 172.21.15.19:0/1760300811 shutdown_connections 2024-04-25T11:42:50.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 --2- 172.21.15.19:0/1760300811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c0b4790 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 --2- 172.21.15.19:0/1760300811 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c06c5c0 0x7f829c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 -- 172.21.15.19:0/1760300811 >> 172.21.15.19:0/1760300811 conn(0x7f829c0d1720 msgr2=0x7f829c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:50.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 -- 172.21.15.19:0/1760300811 shutdown_connections 2024-04-25T11:42:50.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.784+0000 7f82a1000640 1 -- 172.21.15.19:0/1760300811 wait complete. 2024-04-25T11:42:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.786+0000 7f82a1000640 1 Processor -- start 2024-04-25T11:42:50.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.786+0000 7f82a1000640 1 -- start start 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f82a1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f829c06c5c0 0x7f829c161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f82a1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c0b4cd0 0x7f829c138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f82a1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f82a1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f829c0d02d0 con 0x7f829c144370 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f82a1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f829c0d0f50 con 0x7f829c06c5c0 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f82a1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f829c0d0450 con 0x7f829c0b4cd0 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f829be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f829be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56134/0 (socket says 172.21.15.19:56134) 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f829be00640 1 -- 172.21.15.19:0/213379258 learned_addr learned my addr 172.21.15.19:0/213379258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f8293e00640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c0b4cd0 0x7f829c138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f829be00640 1 -- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c0b4cd0 msgr2=0x7f829c138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f829be00640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c0b4cd0 0x7f829c138fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f829be00640 1 -- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f829c06c5c0 msgr2=0x7f829c161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.788+0000 7f829be00640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f829c06c5c0 0x7f829c161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.789+0000 7f829be00640 1 -- 172.21.15.19:0/213379258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c162410 con 0x7f829c144370 2024-04-25T11:42:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.789+0000 7f829be00640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c072de0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f829400ba10 tx=0x7f829400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.789+0000 7f8293400640 1 -- 172.21.15.19:0/213379258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294019070 con 0x7f829c144370 2024-04-25T11:42:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.789+0000 7f8293400640 1 -- 172.21.15.19:0/213379258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829400cb40 con 0x7f829c144370 2024-04-25T11:42:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.789+0000 7f8293400640 1 -- 172.21.15.19:0/213379258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294014440 con 0x7f829c144370 2024-04-25T11:42:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.789+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c139500 con 0x7f829c144370 2024-04-25T11:42:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.789+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c0cb480 con 0x7f829c144370 2024-04-25T11:42:50.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.792+0000 7f8293400640 1 -- 172.21.15.19:0/213379258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f829401c070 con 0x7f829c144370 2024-04-25T11:42:50.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.793+0000 7f8293400640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82780744e0 0x7f82780769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.793+0000 7f8293400640 1 -- 172.21.15.19:0/213379258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f8294094ef0 con 0x7f829c144370 2024-04-25T11:42:50.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.793+0000 7f829b400640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82780744e0 0x7f82780769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.794+0000 7f82a961d740 1 Processor -- start 2024-04-25T11:42:50.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.794+0000 7f82a961d740 1 -- start start 2024-04-25T11:42:50.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.795+0000 7f829b400640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82780744e0 0x7f82780769f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f828c006fd0 tx=0x7f828c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:50.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b36270 0x556c53f03210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f1df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f82a961d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f82a961d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c53e3c600 con 0x556c53b15590 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556c53dd9100 con 0x556c53b36270 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556c53e385c0 con 0x556c53b0d3b0 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56140/0 (socket says 172.21.15.19:56140) 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 -- 172.21.15.19:0/1615214658 learned_addr learned my addr 172.21.15.19:0/1615214658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 -- 172.21.15.19:0/1615214658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 msgr2=0x556c53f1df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 --2- 172.21.15.19:0/1615214658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f1df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 -- 172.21.15.19:0/1615214658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b36270 msgr2=0x556c53f03210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 --2- 172.21.15.19:0/1615214658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b36270 0x556c53f03210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.796+0000 7f829be00640 1 -- 172.21.15.19:0/1615214658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c53b4e2d0 con 0x556c53b15590 2024-04-25T11:42:50.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.797+0000 7f829be00640 1 --2- 172.21.15.19:0/1615214658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1e480 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f8294093290 tx=0x7f829405e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947ab9ef38a1b194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.797+0000 7f828be00640 1 -- 172.21.15.19:0/1615214658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294063230 con 0x556c53b15590 2024-04-25T11:42:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.797+0000 7f828be00640 1 -- 172.21.15.19:0/1615214658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294067ce0 con 0x556c53b15590 2024-04-25T11:42:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.797+0000 7f828be00640 1 -- 172.21.15.19:0/1615214658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294067a50 con 0x556c53b15590 2024-04-25T11:42:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 -- 172.21.15.19:0/1615214658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 msgr2=0x556c53f1e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 --2- 172.21.15.19:0/1615214658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1e480 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f8294093290 tx=0x7f829405e720 comp rx=0 tx=0).stop 2024-04-25T11:42:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 -- 172.21.15.19:0/1615214658 shutdown_connections 2024-04-25T11:42:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 --2- 172.21.15.19:0/1615214658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1e480 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 --2- 172.21.15.19:0/1615214658 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f1df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 --2- 172.21.15.19:0/1615214658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b36270 0x556c53f03210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 -- 172.21.15.19:0/1615214658 >> 172.21.15.19:0/1615214658 conn(0x556c53b2a3c0 msgr2=0x556c53ed8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 -- 172.21.15.19:0/1615214658 shutdown_connections 2024-04-25T11:42:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.798+0000 7f82a961d740 1 -- 172.21.15.19:0/1615214658 wait complete. 2024-04-25T11:42:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.799+0000 7f82a961d740 1 -- start start 2024-04-25T11:42:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.799+0000 7f82a961d740 1 Processor -- start 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b0d3b0 0x556c53f1cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b15590 0x556c53f1d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f82a961d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b36270 0x556c53f04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f82a961d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c53dc3f70 con 0x556c53b36270 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556c53f05eb0 con 0x556c53b0d3b0 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556c53f04120 con 0x556c53b15590 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f82a961d740 10 client.0 _ll_register_callbacks cb 0x556c53e32480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b36270 0x556c53f04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b36270 0x556c53f04b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56156/0 (socket says 172.21.15.19:56156) 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 -- 172.21.15.19:0/3352264412 learned_addr learned my addr 172.21.15.19:0/3352264412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 -- 172.21.15.19:0/3352264412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b15590 msgr2=0x556c53f1d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b15590 0x556c53f1d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 -- 172.21.15.19:0/3352264412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b0d3b0 msgr2=0x556c53f1cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b0d3b0 0x556c53f1cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.801+0000 7f829be00640 1 -- 172.21.15.19:0/3352264412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c53c186b0 con 0x556c53b36270 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f829be00640 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b36270 0x556c53f04b70 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f829405e760 tx=0x7f8294009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526f000ca11dc444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f8288c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f8288c00640 1 -- 172.21.15.19:0/3352264412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294063230 con 0x556c53b36270 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8294067ce0 con 0x556c53b36270 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f8288c00640 1 -- 172.21.15.19:0/3352264412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f829405a7f0 con 0x556c53b36270 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f8288c00640 1 -- 172.21.15.19:0/3352264412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=370) ==== 24+0+0 (secure 0 0 0) 0x7f829405a9f0 con 0x556c53b36270 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f82a961d740 10 client.11634 fetch_fsmap learned FSMap version 370 2024-04-25T11:42:50.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=370,monmap=2+}) -- 0x556c53f05bf0 con 0x556c53b36270 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f8288c00640 1 -- 172.21.15.19:0/3352264412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294067ce0 con 0x556c53b36270 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.802+0000 7f8288c00640 1 -- 172.21.15.19:0/3352264412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 370) ==== 3494+0+0 (secure 0 0 0) 0x7f829405db50 con 0x556c53b36270 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f82a961d740 10 client.11634 fetch_fsmap finished waiting for FSMap version 370 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f82a961d740 10 client.11634 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f82a961d740 20 client.11634 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f82a961d740 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53af4e40 0x556c53f3b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f82a961d740 4 client.11634 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c53e4a210 con 0x556c53af4e40 2024-04-25T11:42:50.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f829b400640 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53af4e40 0x556c53f3b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f829b400640 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53af4e40 0x556c53f3b2a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850325430efc6259 server_cookie=40a5ef3a19631dad in_seq=0 out_seq=0 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.803+0000 7f8288c00640 10 client.11634 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.804+0000 7f8288c00640 1 -- 172.21.15.19:0/3352264412 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c53e4a210 con 0x556c53af4e40 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.804+0000 7f8288c00640 10 client.11634 handle_command_reply: tid=0 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.804+0000 7f82a961d740 1 client.11634 shutdown 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b36270 msgr2=0x556c53f04b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b36270 0x556c53f04b70 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f829405e760 tx=0x7f8294009f60 comp rx=0 tx=0).stop 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 shutdown_connections 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53af4e40 0x556c53f3b2a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b36270 0x556c53f04b70 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b15590 0x556c53f1d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 --2- 172.21.15.19:0/3352264412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b0d3b0 0x556c53f1cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f8288c00640 0 client.11634 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:50.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 >> 172.21.15.19:0/3352264412 conn(0x556c53b2a3c0 msgr2=0x556c53f03210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 shutdown_connections 2024-04-25T11:42:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 1 -- 172.21.15.19:0/3352264412 wait complete. 2024-04-25T11:42:50.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.805+0000 7f82a961d740 20 client.11634 trim_cache size 0 max 16384 2024-04-25T11:42:50.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.826+0000 7f82a961d740 1 Processor -- start 2024-04-25T11:42:50.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.826+0000 7f82a961d740 1 -- start start 2024-04-25T11:42:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f82a961d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b2a3c0 0x556c53ed8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f17700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b15590 0x556c53efe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f82a961d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f829c201d20 con 0x556c53b2a3c0 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f829c15a540 con 0x556c53b15590 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f829c2017d0 con 0x556c53b0d3b0 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f829b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b2a3c0 0x556c53ed8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f829b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b2a3c0 0x556c53ed8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56162/0 (socket says 172.21.15.19:56162) 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.827+0000 7f829b400640 1 -- 172.21.15.19:0/1568756650 learned_addr learned my addr 172.21.15.19:0/1568756650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f829b400640 1 -- 172.21.15.19:0/1568756650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 msgr2=0x556c53f17700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f829b400640 1 --2- 172.21.15.19:0/1568756650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f17700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f829b400640 1 -- 172.21.15.19:0/1568756650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b15590 msgr2=0x556c53efe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f829b400640 1 --2- 172.21.15.19:0/1568756650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b15590 0x556c53efe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f829b400640 1 -- 172.21.15.19:0/1568756650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c53e38780 con 0x556c53b2a3c0 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f829b400640 1 --2- 172.21.15.19:0/1568756650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b2a3c0 0x556c53ed8560 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f828c00afd0 tx=0x7f828c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf9f7b4160616b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f828b400640 1 -- 172.21.15.19:0/1568756650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0158c0 con 0x556c53b2a3c0 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f828b400640 1 -- 172.21.15.19:0/1568756650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c015a60 con 0x556c53b2a3c0 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.828+0000 7f828b400640 1 -- 172.21.15.19:0/1568756650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c015e10 con 0x556c53b2a3c0 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 -- 172.21.15.19:0/1568756650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b2a3c0 msgr2=0x556c53ed8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 --2- 172.21.15.19:0/1568756650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b2a3c0 0x556c53ed8560 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f828c00afd0 tx=0x7f828c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 -- 172.21.15.19:0/1568756650 shutdown_connections 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 --2- 172.21.15.19:0/1568756650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b15590 0x556c53efe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 --2- 172.21.15.19:0/1568756650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f17700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 --2- 172.21.15.19:0/1568756650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b2a3c0 0x556c53ed8560 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 -- 172.21.15.19:0/1568756650 >> 172.21.15.19:0/1568756650 conn(0x556c53f04ca0 msgr2=0x556c53ed9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 -- 172.21.15.19:0/1568756650 shutdown_connections 2024-04-25T11:42:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.829+0000 7f82a961d740 1 -- 172.21.15.19:0/1568756650 wait complete. 2024-04-25T11:42:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.830+0000 7f82a961d740 1 -- start start 2024-04-25T11:42:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.830+0000 7f82a961d740 1 Processor -- start 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f91560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f82a961d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f82a961d740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b2a3c0 0x556c53f94770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f82a961d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c53efddd0 con 0x556c53b15590 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556c53efd9f0 con 0x556c53b2a3c0 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f82a961d740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556c53efc5b0 con 0x556c53b0d3b0 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f82a961d740 10 client.0 _ll_register_callbacks cb 0x7f829405e5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f8293e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f8293e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56174/0 (socket says 172.21.15.19:56174) 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.832+0000 7f8293e00640 1 -- 172.21.15.19:0/3090408731 learned_addr learned my addr 172.21.15.19:0/3090408731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f829b400640 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f91560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8293e00640 1 -- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 msgr2=0x556c53f91560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8293e00640 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f91560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8293e00640 1 -- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b2a3c0 msgr2=0x556c53f94770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f829be00640 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b2a3c0 0x556c53f94770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8293e00640 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b2a3c0 0x556c53f94770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8293e00640 1 -- 172.21.15.19:0/3090408731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c53ed4580 con 0x556c53b15590 2024-04-25T11:42:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8293e00640 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1a7c0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f8284000c00 tx=0x7f8284032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43427ba5767eb9ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8288c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8288c00640 1 -- 172.21.15.19:0/3090408731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284040020 con 0x556c53b15590 2024-04-25T11:42:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f8288c00640 1 -- 172.21.15.19:0/3090408731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f828403c810 con 0x556c53b15590 2024-04-25T11:42:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.833+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828c015a60 con 0x556c53b15590 2024-04-25T11:42:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f8288c00640 1 -- 172.21.15.19:0/3090408731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=370) ==== 24+0+0 (secure 0 0 0) 0x7f828404a050 con 0x556c53b15590 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 10 client.11640 fetch_fsmap learned FSMap version 370 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=370,monmap=2+}) -- 0x556c53f1b490 con 0x556c53b15590 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f8288c00640 1 -- 172.21.15.19:0/3090408731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c015a60 con 0x556c53b15590 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f8288c00640 1 -- 172.21.15.19:0/3090408731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 370) ==== 3494+0+0 (secure 0 0 0) 0x7f828404d020 con 0x556c53b15590 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 10 client.11640 fetch_fsmap finished waiting for FSMap version 370 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 10 client.11640 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 20 client.11640 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53aca280 0x556c53ed25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 4 client.11640 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "status"}) -- 0x556c53e44080 con 0x556c53aca280 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.834+0000 7f829b400640 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53aca280 0x556c53ed25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f829b400640 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53aca280 0x556c53ed25d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd91ad35d7c289d8 server_cookie=2debb875e5886683 in_seq=0 out_seq=0 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f8288c00640 10 client.11640 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f8288c00640 1 -- 172.21.15.19:0/3090408731 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x556c53e44080 con 0x556c53aca280 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f8288c00640 10 client.11640 handle_command_reply: tid=0 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 client.11640 shutdown 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 msgr2=0x556c53f1a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1a7c0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f8284000c00 tx=0x7f8284032b20 comp rx=0 tx=0).stop 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 shutdown_connections 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c53aca280 0x556c53ed25d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c53b2a3c0 0x556c53f94770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c53b15590 0x556c53f1a7c0 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f8288c00640 0 client.11640 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 --2- 172.21.15.19:0/3090408731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c53b0d3b0 0x556c53f91560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.835+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 >> 172.21.15.19:0/3090408731 conn(0x556c53f04ca0 msgr2=0x556c53eff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 shutdown_connections 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a961d740 1 -- 172.21.15.19:0/3090408731 wait complete. 2024-04-25T11:42:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a961d740 20 client.11640 trim_cache size 0 max 16384 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "23a56573-5965-4bbb-bd60-741a44babbed", 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8020, 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.3313530729999998, 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 370, 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 144, 2024-04-25T11:42:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 144, 2024-04-25T11:42:50.827 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 27.320441956 2024-04-25T11:42:50.827 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:42:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82780744e0 msgr2=0x7f82780769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82780744e0 0x7f82780769f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f828c006fd0 tx=0x7f828c008040 comp rx=0 tx=0).stop 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 msgr2=0x7f829c072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c072de0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f829400ba10 tx=0x7f829400bed0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 shutdown_connections 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f82780744e0 0x7f82780769f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f829c144370 0x7f829c072de0 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f829c0b4cd0 0x7f829c138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 --2- 172.21.15.19:0/213379258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f829c06c5c0 0x7f829c161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.836+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 >> 172.21.15.19:0/213379258 conn(0x7f829c0d1720 msgr2=0x7f829c146790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.837+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 shutdown_connections 2024-04-25T11:42:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:50.837+0000 7f82a1000640 1 -- 172.21.15.19:0/213379258 wait complete. 2024-04-25T11:42:50.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:42:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.131+0000 7f04f3200640 1 Processor -- start 2024-04-25T11:42:51.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.131+0000 7f04f3200640 1 -- start start 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec13fe10 0x7f04ec06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04ec06db10 0x7f04ec142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f3200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f04ec142c70 0x7f04ec070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f3200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f04ec06df20 con 0x7f04ec06db10 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f3200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f04ec15d5e0 con 0x7f04ec142c70 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04ec15d8c0 con 0x7f04ec13fe10 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04ec06db10 0x7f04ec142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04ec06db10 0x7f04ec142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56178/0 (socket says 172.21.15.19:56178) 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f1400640 1 -- 172.21.15.19:0/2188019168 learned_addr learned my addr 172.21.15.19:0/2188019168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f1e00640 1 --2- 172.21.15.19:0/2188019168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec13fe10 0x7f04ec06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:51.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.133+0000 7f04f1400640 1 -- 172.21.15.19:0/2188019168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec13fe10 msgr2=0x7f04ec06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.134+0000 7f04f2800640 1 --2- 172.21.15.19:0/2188019168 >> v2:172.21.15.169:3300/0 conn(0x7f04ec142c70 0x7f04ec070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.134+0000 7f04f1400640 1 --2- 172.21.15.19:0/2188019168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec13fe10 0x7f04ec06d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.134+0000 7f04f1400640 1 -- 172.21.15.19:0/2188019168 >> v2:172.21.15.169:3300/0 conn(0x7f04ec142c70 msgr2=0x7f04ec070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.134+0000 7f04f1400640 1 --2- 172.21.15.19:0/2188019168 >> v2:172.21.15.169:3300/0 conn(0x7f04ec142c70 0x7f04ec070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.134+0000 7f04f1400640 1 -- 172.21.15.19:0/2188019168 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec15dba0 con 0x7f04ec06db10 2024-04-25T11:42:51.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.134+0000 7f04f2800640 1 --2- 172.21.15.19:0/2188019168 >> v2:172.21.15.169:3300/0 conn(0x7f04ec142c70 0x7f04ec070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.135+0000 7f04f1400640 1 --2- 172.21.15.19:0/2188019168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec06db10 0x7f04ec142730 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0741e0 tx=0x7f04dc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fb4a9d87d856c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.135+0000 7f04f0a00640 1 -- 172.21.15.19:0/2188019168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc037070 con 0x7f04ec06db10 2024-04-25T11:42:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.135+0000 7f04f0a00640 1 -- 172.21.15.19:0/2188019168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc032440 con 0x7f04ec06db10 2024-04-25T11:42:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.135+0000 7f04f0a00640 1 -- 172.21.15.19:0/2188019168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc032810 con 0x7f04ec06db10 2024-04-25T11:42:51.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 -- 172.21.15.19:0/2188019168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec06db10 msgr2=0x7f04ec142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 --2- 172.21.15.19:0/2188019168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec06db10 0x7f04ec142730 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0741e0 tx=0x7f04dc02a140 comp rx=0 tx=0).stop 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 -- 172.21.15.19:0/2188019168 shutdown_connections 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 --2- 172.21.15.19:0/2188019168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec06db10 0x7f04ec142730 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 --2- 172.21.15.19:0/2188019168 >> v2:172.21.15.169:3300/0 conn(0x7f04ec142c70 0x7f04ec070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 --2- 172.21.15.19:0/2188019168 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec13fe10 0x7f04ec06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 -- 172.21.15.19:0/2188019168 >> 172.21.15.19:0/2188019168 conn(0x7f04ec072ee0 msgr2=0x7f04ec0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 -- 172.21.15.19:0/2188019168 shutdown_connections 2024-04-25T11:42:51.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.136+0000 7f04f3200640 1 -- 172.21.15.19:0/2188019168 wait complete. 2024-04-25T11:42:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.138+0000 7f04f3200640 1 Processor -- start 2024-04-25T11:42:51.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.139+0000 7f04f3200640 1 -- start start 2024-04-25T11:42:51.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f3200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec06db10 0x7f04ec1f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:51.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f3200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec13fe10 0x7f04ec143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:51.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec142c70 0x7f04ec145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec143080 con 0x7f04ec142c70 2024-04-25T11:42:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f3200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec15d2d0 con 0x7f04ec13fe10 2024-04-25T11:42:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f3200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04ec143200 con 0x7f04ec06db10 2024-04-25T11:42:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec142c70 0x7f04ec145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec142c70 0x7f04ec145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56184/0 (socket says 172.21.15.19:56184) 2024-04-25T11:42:51.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.140+0000 7f04f2800640 1 -- 172.21.15.19:0/1824277309 learned_addr learned my addr 172.21.15.19:0/1824277309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f1e00640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec06db10 0x7f04ec1f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:51.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f1400640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec13fe10 0x7f04ec143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f2800640 1 -- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec06db10 msgr2=0x7f04ec1f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f2800640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec06db10 0x7f04ec1f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f2800640 1 -- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec13fe10 msgr2=0x7f04ec143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f2800640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec13fe10 0x7f04ec143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f2800640 1 -- 172.21.15.19:0/1824277309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec13e4a0 con 0x7f04ec142c70 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f1400640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec13fe10 0x7f04ec143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.141+0000 7f04f2800640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec142c70 0x7f04ec145180 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f04e800cd80 tx=0x7f04e800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.142+0000 7f04e6000640 1 -- 172.21.15.19:0/1824277309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e800f680 con 0x7f04ec142c70 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.142+0000 7f04e6000640 1 -- 172.21.15.19:0/1824277309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e800f840 con 0x7f04ec142c70 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.142+0000 7f04e6000640 1 -- 172.21.15.19:0/1824277309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e800fc30 con 0x7f04ec142c70 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.142+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ec13e780 con 0x7f04ec142c70 2024-04-25T11:42:51.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.142+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ec1f9d80 con 0x7f04ec142c70 2024-04-25T11:42:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.143+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ec050740 con 0x7f04ec142c70 2024-04-25T11:42:51.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.144+0000 7f04e6000640 1 -- 172.21.15.19:0/1824277309 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f04e80213f0 con 0x7f04ec142c70 2024-04-25T11:42:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.145+0000 7f04e6000640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04d00744b0 0x7f04d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:51.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.146+0000 7f04e6000640 1 -- 172.21.15.19:0/1824277309 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f04e8095f30 con 0x7f04ec142c70 2024-04-25T11:42:51.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.146+0000 7f04f1e00640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04d00744b0 0x7f04d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.148+0000 7f04f1e00640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04d00744b0 0x7f04d00769c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f04e0006fd0 tx=0x7f04e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:51.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.151+0000 7f04e6000640 1 -- 172.21.15.19:0/1824277309 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04e8014030 con 0x7f04ec142c70 2024-04-25T11:42:51.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.396+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04ec143af0 con 0x7f04ec142c70 2024-04-25T11:42:51.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.397+0000 7f04e6000640 1 -- 172.21.15.19:0/1824277309 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v370) ==== 53+0+73 (secure 0 0 0) 0x7f04e80606f0 con 0x7f04ec142c70 2024-04-25T11:42:51.384 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:42:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.399+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04d00744b0 msgr2=0x7f04d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.399+0000 7f04f3200640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04d00744b0 0x7f04d00769c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f04e0006fd0 tx=0x7f04e0008040 comp rx=0 tx=0).stop 2024-04-25T11:42:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec142c70 msgr2=0x7f04ec145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec142c70 0x7f04ec145180 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f04e800cd80 tx=0x7f04e800a650 comp rx=0 tx=0).stop 2024-04-25T11:42:51.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 shutdown_connections 2024-04-25T11:42:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f04d00744b0 0x7f04d00769c0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f04ec13fe10 0x7f04ec143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04ec142c70 0x7f04ec145180 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 --2- 172.21.15.19:0/1824277309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f04ec06db10 0x7f04ec1f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 >> 172.21.15.19:0/1824277309 conn(0x7f04ec072ee0 msgr2=0x7f04ec13c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 shutdown_connections 2024-04-25T11:42:51.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51.400+0000 7f04f3200640 1 -- 172.21.15.19:0/1824277309 wait complete. 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:42:51.414 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:42:51.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:51.415 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T11:42:51.468 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41702sec preferred_lft 41702sec 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:42:51.469 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::a8c6:73ff:fe75:d813/64 scope link 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 3a:9c:f4:1f:62:01 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::389c:f4ff:fe1f:6201/64 scope link 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether c6:e7:28:14:84:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::c4e7:28ff:fe14:8478/64 scope link 2024-04-25T11:42:51.470 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:42:51.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:51.471 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:42:51.545 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T11:42:51.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:51.549 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T11:42:51.595 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T11:42:51.595 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:42:51.595 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:42:51.595 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:42:51.595 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:42:51.595 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:42:51.596 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T11:42:51.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:42:51.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:42:51.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:42:51.596 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T11:42:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:42:51.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:42:51.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:51.859 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:42:51.908 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:42:51.908 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:42:51.908 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:42:51.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:51.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T11:42:51.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:42:51.989 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:52.043 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:42:52.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:42:52.044 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:52.098 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:42:52.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:52.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:42:52.311 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:42:52.312 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:42:52.354 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:42:52.354 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:42:52.354 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.169:3300,172.21.15.169:3301" 2024-04-25T11:42:52.354 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with new device syntax: 0@23a56573-5965-4bbb-bd60-741a44babbed.cephfs=/ 2024-04-25T11:42:52.354 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.19:3300/172.21.15.169:3300/172.21.15.169:3301" will pass to kernel 2024-04-25T11:42:52.355 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:42:52.355 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:42:52.356 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:42:52.356 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:42:52.356 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:42:52.361 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:42:52.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:52.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:42:52.431 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:42:52.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:42:52.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:42:52.557 INFO:teuthology.orchestra.run.smithi019.stdout:client7136 2024-04-25T11:42:52.558 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:52.558 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:52.610 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T11:42:52.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.811+0000 7f647793cf40 1 Processor -- start 2024-04-25T11:42:52.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.811+0000 7f647793cf40 1 -- start start 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f647793cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 0x563537b4b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f647793cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537b57100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f647793cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b57640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f647793cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563537ac7e70 con 0x563537a7f5f0 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f647793cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563537a25d20 con 0x563537b58dd0 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f647793cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563537ac83f0 con 0x563537a7f1e0 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537b57100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537b57100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56208/0 (socket says 172.21.15.19:56208) 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 -- 172.21.15.19:0/3597322740 learned_addr learned my addr 172.21.15.19:0/3597322740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:52.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 -- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 msgr2=0x563537b4b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 0x563537b4b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6471400640 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b57640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 -- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 msgr2=0x563537b57640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b57640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.813+0000 7f6470000640 1 -- 172.21.15.19:0/3597322740 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563537b57f10 con 0x563537a7f5f0 2024-04-25T11:42:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.814+0000 7f6471400640 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b57640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:52.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.814+0000 7f6470000640 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537b57100 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f6458025c40 tx=0x7f6458051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90977b0e4fc45d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.814+0000 7f646ec00640 1 -- 172.21.15.19:0/3597322740 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645805a050 con 0x563537a7f5f0 2024-04-25T11:42:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.814+0000 7f646ec00640 1 -- 172.21.15.19:0/3597322740 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458066070 con 0x563537a7f5f0 2024-04-25T11:42:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.814+0000 7f646ec00640 1 -- 172.21.15.19:0/3597322740 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458055b30 con 0x563537a7f5f0 2024-04-25T11:42:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 -- 172.21.15.19:0/3597322740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 msgr2=0x563537b57100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537b57100 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f6458025c40 tx=0x7f6458051000 comp rx=0 tx=0).stop 2024-04-25T11:42:52.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 -- 172.21.15.19:0/3597322740 shutdown_connections 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b57640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537b57100 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 --2- 172.21.15.19:0/3597322740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 0x563537b4b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 -- 172.21.15.19:0/3597322740 >> 172.21.15.19:0/3597322740 conn(0x5635379a1ac0 msgr2=0x563537ab12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 -- 172.21.15.19:0/3597322740 shutdown_connections 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.815+0000 7f647793cf40 1 -- 172.21.15.19:0/3597322740 wait complete. 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.816+0000 7f647793cf40 1 Processor -- start 2024-04-25T11:42:52.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.816+0000 7f647793cf40 1 -- start start 2024-04-25T11:42:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f647793cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 0x563537be6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f647793cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537acb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f647793cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f647793cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563537b55ce0 con 0x563537a7f5f0 2024-04-25T11:42:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f647793cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563537b4bde0 con 0x563537b58dd0 2024-04-25T11:42:52.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f647793cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x563537b4dbf0 con 0x563537a7f1e0 2024-04-25T11:42:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537acb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537acb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56212/0 (socket says 172.21.15.19:56212) 2024-04-25T11:42:52.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 -- 172.21.15.19:0/4240847395 learned_addr learned my addr 172.21.15.19:0/4240847395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:52.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6471400640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470a00640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 0x563537be6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 -- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 msgr2=0x563537be6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x563537a7f1e0 0x563537be6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 -- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 msgr2=0x563537b447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563537b58dd0 0x563537b447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 -- 172.21.15.19:0/4240847395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563537be5260 con 0x563537a7f5f0 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f6470000640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537acb330 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f6458025c40 tx=0x7f6458003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f646d800640 1 -- 172.21.15.19:0/4240847395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458055e20 con 0x563537a7f5f0 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f646d800640 1 -- 172.21.15.19:0/4240847395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645805b040 con 0x563537a7f5f0 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f646d800640 1 -- 172.21.15.19:0/4240847395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458061670 con 0x563537a7f5f0 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.819+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563537b4ec50 con 0x563537a7f5f0 2024-04-25T11:42:52.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.820+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563537be4cf0 con 0x563537a7f5f0 2024-04-25T11:42:52.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.822+0000 7f646d800640 1 -- 172.21.15.19:0/4240847395 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6458073020 con 0x563537a7f5f0 2024-04-25T11:42:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.824+0000 7f646d800640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f644c072ec0 0x7f644c0753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.824+0000 7f646d800640 1 -- 172.21.15.19:0/4240847395 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f64580d9420 con 0x563537a7f5f0 2024-04-25T11:42:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.824+0000 7f647793cf40 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563537acb870 0x563537b4cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.824+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 15.7 15:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e144) -- 0x563537b48ec0 con 0x563537acb870 2024-04-25T11:42:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.824+0000 7f6470a00640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f644c072ec0 0x7f644c0753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.825+0000 7f6471400640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563537acb870 0x563537b4cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.825+0000 7f6470a00640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f644c072ec0 0x7f644c0753d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f646000c430 tx=0x7f6460007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:52.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.825+0000 7f6471400640 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563537acb870 0x563537b4cf70 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:52.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.826+0000 7f6471400640 1 -- 172.21.15.19:0/4240847395 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) ==== 164+0+91 (crc 0 0 0) 0x7f646800e4b0 con 0x563537acb870 2024-04-25T11:42:52.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.826+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563537acb870 msgr2=0x563537b4cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.826+0000 7f647793cf40 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x563537acb870 0x563537b4cf70 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f644c072ec0 msgr2=0x7f644c0753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f644c072ec0 0x7f644c0753d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f646000c430 tx=0x7f6460007040 comp rx=0 tx=0).stop 2024-04-25T11:42:52.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 msgr2=0x563537acb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 --2- 172.21.15.19:0/4240847395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563537a7f5f0 0x563537acb330 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f6458025c40 tx=0x7f6458003020 comp rx=0 tx=0).stop 2024-04-25T11:42:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f6471400640 1 -- 172.21.15.19:0/4240847395 reap_dead start 2024-04-25T11:42:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 shutdown_connections 2024-04-25T11:42:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 >> 172.21.15.19:0/4240847395 conn(0x5635379a1ac0 msgr2=0x563537b533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 shutdown_connections 2024-04-25T11:42:52.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:52.827+0000 7f647793cf40 1 -- 172.21.15.19:0/4240847395 wait complete. 2024-04-25T11:42:52.827 INFO:teuthology.orchestra.run.smithi019.stdout:Ufile1dir1 2024-04-25T11:42:52.828 DEBUG:teuthology.orchestra.run.smithi019:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T11:42:53.072 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "ancestors": [ 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1099511627776, 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "file1", 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 8 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1, 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "dir1", 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 29 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:53.073 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T11:42:53.074 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 15, 2024-04-25T11:42:53.074 INFO:teuthology.orchestra.run.smithi019.stdout: "old_pools": [] 2024-04-25T11:42:53.074 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:42:53.074 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:53.089 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent rhubarb 2024-04-25T11:42:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 Processor -- start 2024-04-25T11:42:53.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 -- start start 2024-04-25T11:42:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 0x55826bdec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 0x55826be6fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826be70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55826bde9ea0 con 0x55826be7ae00 2024-04-25T11:42:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55826bd47d20 con 0x55826bda1210 2024-04-25T11:42:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.296+0000 7f53fbf1bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55826bdea140 con 0x55826bda1620 2024-04-25T11:42:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826be70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826be70810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56224/0 (socket says 172.21.15.19:56224) 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 -- 172.21.15.19:0/2847866848 learned_addr learned my addr 172.21.15.19:0/2847866848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f4600640 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 0x55826bdec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 -- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 msgr2=0x55826be6fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f3c00640 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 0x55826be6fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 0x55826be6fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 -- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 msgr2=0x55826bdec8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 0x55826bdec8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.297+0000 7f53f5000640 1 -- 172.21.15.19:0/2847866848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826bda0dc0 con 0x55826be7ae00 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.298+0000 7f53f5000640 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826be70810 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009870 tx=0x7f53ec005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29315a4fa97b7708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.298+0000 7f53f2800640 1 -- 172.21.15.19:0/2847866848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0088d0 con 0x55826be7ae00 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.298+0000 7f53f2800640 1 -- 172.21.15.19:0/2847866848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec008a70 con 0x55826be7ae00 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.298+0000 7f53f2800640 1 -- 172.21.15.19:0/2847866848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec008e60 con 0x55826be7ae00 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.298+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2847866848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 msgr2=0x55826be70810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.298+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826be70810 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009870 tx=0x7f53ec005a00 comp rx=0 tx=0).stop 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.299+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2847866848 shutdown_connections 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.299+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826be70810 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.299+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 0x55826be6fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.299+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2847866848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 0x55826bdec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.299+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2847866848 >> 172.21.15.19:0/2847866848 conn(0x55826bcc3ac0 msgr2=0x55826be79df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.299+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2847866848 shutdown_connections 2024-04-25T11:42:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.299+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2847866848 wait complete. 2024-04-25T11:42:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.300+0000 7f53fbf1bf40 1 Processor -- start 2024-04-25T11:42:53.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.300+0000 7f53fbf1bf40 1 -- start start 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.301+0000 7f53fbf1bf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 0x55826bf09040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.301+0000 7f53fbf1bf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 0x55826be72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.301+0000 7f53fbf1bf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826bdd2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.301+0000 7f53fbf1bf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55826bded060 con 0x55826be7ae00 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.301+0000 7f53fbf1bf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55826be77000 con 0x55826bda1210 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.301+0000 7f53fbf1bf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55826be729a0 con 0x55826bda1620 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826bdd2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826bdd2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56228/0 (socket says 172.21.15.19:56228) 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 -- 172.21.15.19:0/2639627707 learned_addr learned my addr 172.21.15.19:0/2639627707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f3c00640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 0x55826be72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 -- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 msgr2=0x55826be72fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55826bda1620 0x55826be72fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f4600640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 0x55826bf09040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 -- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 msgr2=0x55826bf09040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55826bda1210 0x55826bf09040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.302+0000 7f53f5000640 1 -- 172.21.15.19:0/2639627707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826bf09580 con 0x55826be7ae00 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.303+0000 7f53f5000640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826bdd2650 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009df0 tx=0x7f53ec003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.303+0000 7f53f1400640 1 -- 172.21.15.19:0/2639627707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec022020 con 0x55826be7ae00 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.303+0000 7f53f1400640 1 -- 172.21.15.19:0/2639627707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec013830 con 0x55826be7ae00 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.303+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55826be73520 con 0x55826be7ae00 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.303+0000 7f53f1400640 1 -- 172.21.15.19:0/2639627707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec013c20 con 0x55826be7ae00 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.303+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826bde3630 con 0x55826be7ae00 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.305+0000 7f53f1400640 1 -- 172.21.15.19:0/2639627707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f53ec025020 con 0x55826be7ae00 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.307+0000 7f53f1400640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53d0072ec0 0x7f53d00753d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.307+0000 7f53f1400640 1 -- 172.21.15.19:0/2639627707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f53ec095990 con 0x55826be7ae00 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.307+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55826bf18ed0 0x55826bf1b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.307+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- osd_op(unknown.0.0:1 15.7 15:e2e07930:::10000000001.00000000:head [setxattr parent (7) in=13b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144) -- 0x55826bf1b830 con 0x55826bf18ed0 2024-04-25T11:42:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.307+0000 7f53f4600640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53d0072ec0 0x7f53d00753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.308+0000 7f53f3c00640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55826bf18ed0 0x55826bf1b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.309+0000 7f53f4600640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53d0072ec0 0x7f53d00753d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f53e40044f0 tx=0x7f53e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.309+0000 7f53f3c00640 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55826bf18ed0 0x55826bf1b2f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53f3c00640 1 -- 172.21.15.19:0/2639627707 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (7)] v144'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f53dc054870 con 0x55826bf18ed0 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55826bf18ed0 msgr2=0x55826bf1b2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x55826bf18ed0 0x55826bf1b2f0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53d0072ec0 msgr2=0x7f53d00753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f53d0072ec0 0x7f53d00753d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f53e40044f0 tx=0x7f53e4008040 comp rx=0 tx=0).stop 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 msgr2=0x55826bdd2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 --2- 172.21.15.19:0/2639627707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826be7ae00 0x55826bdd2650 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009df0 tx=0x7f53ec003020 comp rx=0 tx=0).stop 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53f5000640 1 -- 172.21.15.19:0/2639627707 reap_dead start 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 shutdown_connections 2024-04-25T11:42:53.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 >> 172.21.15.19:0/2639627707 conn(0x55826bcc3ac0 msgr2=0x55826be77820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:53.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 shutdown_connections 2024-04-25T11:42:53.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.314+0000 7f53fbf1bf40 1 -- 172.21.15.19:0/2639627707 wait complete. 2024-04-25T11:42:53.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:53.319 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink1) 2024-04-25T11:42:53.374 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:42:53.388+0000 7f90a9800640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input [buffer:3] 2024-04-25T11:42:53.374 INFO:teuthology.orchestra.run.smithi019.stderr:stat: cannot statx 'testdir/hardlink1': Input/output error 2024-04-25T11:42:53.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:42:53.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.673+0000 7fdb69200640 1 Processor -- start 2024-04-25T11:42:53.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.673+0000 7fdb69200640 1 -- start start 2024-04-25T11:42:53.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb69200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb6406d560 0x7fdb6415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb69200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb6415a2f0 0x7fdb640b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb69200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb641611e0 con 0x7fdb640b6a00 2024-04-25T11:42:53.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb69200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdb64160d50 con 0x7fdb6406d560 2024-04-25T11:42:53.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb69200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb6415a700 con 0x7fdb6415a2f0 2024-04-25T11:42:53.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb61e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdb640b6a00 0x7fdb6406ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43056/0 (socket says 172.21.15.19:43056) 2024-04-25T11:42:53.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb61e00640 1 -- 172.21.15.19:0/4063165918 learned_addr learned my addr 172.21.15.19:0/4063165918 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:53.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.676+0000 7fdb61400640 1 --2- 172.21.15.19:0/4063165918 >> v2:172.21.15.19:3300/0 conn(0x7fdb6406d560 0x7fdb6415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.677+0000 7fdb62800640 1 --2- 172.21.15.19:0/4063165918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb6415a2f0 0x7fdb640b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.677+0000 7fdb61400640 1 -- 172.21.15.19:0/4063165918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb6415a2f0 msgr2=0x7fdb640b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:53.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.677+0000 7fdb61400640 1 --2- 172.21.15.19:0/4063165918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb6415a2f0 0x7fdb640b4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.677+0000 7fdb61400640 1 -- 172.21.15.19:0/4063165918 >> v1:172.21.15.19:6789/0 conn(0x7fdb640b6a00 legacy=0x7fdb6406ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.677+0000 7fdb61400640 1 -- 172.21.15.19:0/4063165918 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb64160420 con 0x7fdb6406d560 2024-04-25T11:42:53.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.677+0000 7fdb60a00640 1 -- 172.21.15.19:0/4063165918 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343598231 0 0) 0x7fdb641611e0 con 0x7fdb640b6a00 2024-04-25T11:42:53.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.678+0000 7fdb61400640 1 --2- 172.21.15.19:0/4063165918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6406d560 0x7fdb6415d6c0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0097a0 tx=0x7fdb4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d2dccc6bf0cfa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.678+0000 7fdb60a00640 1 -- 172.21.15.19:0/4063165918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c037070 con 0x7fdb6406d560 2024-04-25T11:42:53.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.678+0000 7fdb60a00640 1 -- 172.21.15.19:0/4063165918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4c032440 con 0x7fdb6406d560 2024-04-25T11:42:53.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.678+0000 7fdb60a00640 1 -- 172.21.15.19:0/4063165918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c032810 con 0x7fdb6406d560 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.679+0000 7fdb69200640 1 -- 172.21.15.19:0/4063165918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6406d560 msgr2=0x7fdb6415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.679+0000 7fdb69200640 1 --2- 172.21.15.19:0/4063165918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6406d560 0x7fdb6415d6c0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0097a0 tx=0x7fdb4c02a180 comp rx=0 tx=0).stop 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.680+0000 7fdb69200640 1 -- 172.21.15.19:0/4063165918 shutdown_connections 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.680+0000 7fdb69200640 1 --2- 172.21.15.19:0/4063165918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb6415a2f0 0x7fdb640b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.680+0000 7fdb69200640 1 --2- 172.21.15.19:0/4063165918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6406d560 0x7fdb6415d6c0 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.680+0000 7fdb69200640 1 -- 172.21.15.19:0/4063165918 >> 172.21.15.19:0/4063165918 conn(0x7fdb640d1ae0 msgr2=0x7fdb640d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.680+0000 7fdb69200640 1 -- 172.21.15.19:0/4063165918 shutdown_connections 2024-04-25T11:42:53.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.680+0000 7fdb69200640 1 -- 172.21.15.19:0/4063165918 wait complete. 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.682+0000 7fdb69200640 1 Processor -- start 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.682+0000 7fdb69200640 1 -- start start 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb69200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6406d560 0x7fdb6415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb69200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb640b6a00 0x7fdb6415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb69200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6415a2f0 0x7fdb64153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb69200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb640ce580 con 0x7fdb6415a2f0 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb69200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641609e0 con 0x7fdb6406d560 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb69200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fdb640ce700 con 0x7fdb640b6a00 2024-04-25T11:42:53.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb62800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6415a2f0 0x7fdb64153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb62800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6415a2f0 0x7fdb64153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56244/0 (socket says 172.21.15.19:56244) 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.684+0000 7fdb62800640 1 -- 172.21.15.19:0/1245211250 learned_addr learned my addr 172.21.15.19:0/1245211250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb61e00640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6406d560 0x7fdb6415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb61400640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb640b6a00 0x7fdb6415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb62800640 1 -- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb640b6a00 msgr2=0x7fdb6415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb62800640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb640b6a00 0x7fdb6415dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb62800640 1 -- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6406d560 msgr2=0x7fdb6415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb62800640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6406d560 0x7fdb6415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb62800640 1 -- 172.21.15.19:0/1245211250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb64154840 con 0x7fdb6415a2f0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb62800640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6415a2f0 0x7fdb64153fd0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c007d70 tx=0x7fdb5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.685+0000 7fdb5a000640 1 -- 172.21.15.19:0/1245211250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c014420 con 0x7fdb6415a2f0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.686+0000 7fdb5a000640 1 -- 172.21.15.19:0/1245211250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb5c019070 con 0x7fdb6415a2f0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.686+0000 7fdb5a000640 1 -- 172.21.15.19:0/1245211250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c014820 con 0x7fdb6415a2f0 2024-04-25T11:42:53.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.686+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb64146f80 con 0x7fdb6415a2f0 2024-04-25T11:42:53.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.686+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb64147500 con 0x7fdb6415a2f0 2024-04-25T11:42:53.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.687+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6415c6d0 con 0x7fdb6415a2f0 2024-04-25T11:42:53.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.689+0000 7fdb5a000640 1 -- 172.21.15.19:0/1245211250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdb5c0149c0 con 0x7fdb6415a2f0 2024-04-25T11:42:53.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.690+0000 7fdb5a000640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb400744b0 0x7fdb400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.690+0000 7fdb5a000640 1 -- 172.21.15.19:0/1245211250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fdb5c095b00 con 0x7fdb6415a2f0 2024-04-25T11:42:53.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.691+0000 7fdb61e00640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb400744b0 0x7fdb400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.692+0000 7fdb61e00640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb400744b0 0x7fdb400769c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fdb54000f60 tx=0x7fdb54007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.696+0000 7fdb5a000640 1 -- 172.21.15.19:0/1245211250 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb64146f80 con 0x7fdb6415a2f0 2024-04-25T11:42:53.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.941+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb640b33e0 con 0x7fdb6415a2f0 2024-04-25T11:42:53.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.942+0000 7fdb5a000640 1 -- 172.21.15.19:0/1245211250 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 370 v370) ==== 95+0+5194 (secure 0 0 0) 0x7fdb5c060220 con 0x7fdb6415a2f0 2024-04-25T11:42:53.928 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:53.928 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":370,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":370,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:48.519583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":9,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:42:53.928 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 370 2024-04-25T11:42:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb400744b0 msgr2=0x7fdb400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb400744b0 0x7fdb400769c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fdb54000f60 tx=0x7fdb54007c00 comp rx=0 tx=0).stop 2024-04-25T11:42:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6415a2f0 msgr2=0x7fdb64153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6415a2f0 0x7fdb64153fd0 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c007d70 tx=0x7fdb5c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 shutdown_connections 2024-04-25T11:42:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fdb400744b0 0x7fdb400769c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb6415a2f0 0x7fdb64153fd0 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb640b6a00 0x7fdb6415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.944+0000 7fdb69200640 1 --2- 172.21.15.19:0/1245211250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb6406d560 0x7fdb6415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.945+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 >> 172.21.15.19:0/1245211250 conn(0x7fdb640d1ae0 msgr2=0x7fdb641516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:53.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.945+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 shutdown_connections 2024-04-25T11:42:53.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:53.945+0000 7fdb69200640 1 -- 172.21.15.19:0/1245211250 wait complete. 2024-04-25T11:42:53.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T11:42:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.267+0000 7f7eb2000640 1 Processor -- start 2024-04-25T11:42:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.267+0000 7f7eb2000640 1 -- start start 2024-04-25T11:42:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.269+0000 7f7eb2000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7eac144310 0x7f7eac06fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.269+0000 7f7eb2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac0b72b0 0x7f7eac161e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.269+0000 7f7eb2000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac144720 con 0x7f7eac161a80 2024-04-25T11:42:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.269+0000 7f7eb2000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f7eac06f640 con 0x7f7eac144310 2024-04-25T11:42:54.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.269+0000 7f7eb2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac1624d0 con 0x7f7eac0b72b0 2024-04-25T11:42:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.270+0000 7f7eb1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac0b72b0 0x7f7eac161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.270+0000 7f7eb0c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7eac144310 0x7f7eac06fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.270+0000 7f7eb1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac0b72b0 0x7f7eac161e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52176/0 (socket says 172.21.15.19:52176) 2024-04-25T11:42:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.270+0000 7f7eb1600640 1 -- 172.21.15.19:0/2324536268 learned_addr learned my addr 172.21.15.19:0/2324536268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:54.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.270+0000 7f7eabe00640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f7eac161a80 0x7f7eac0b6ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:42690/0 (socket says 172.21.15.19:42690) 2024-04-25T11:42:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.271+0000 7f7eb1600640 1 -- 172.21.15.19:0/2324536268 >> v2:172.21.15.169:3300/0 conn(0x7f7eac144310 msgr2=0x7f7eac06fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.271+0000 7f7eb1600640 1 --2- 172.21.15.19:0/2324536268 >> v2:172.21.15.169:3300/0 conn(0x7f7eac144310 0x7f7eac06fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.271+0000 7f7eb1600640 1 -- 172.21.15.19:0/2324536268 >> v1:172.21.15.169:6789/0 conn(0x7f7eac161a80 legacy=0x7f7eac0b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.271+0000 7f7eb1600640 1 -- 172.21.15.19:0/2324536268 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac1627b0 con 0x7f7eac0b72b0 2024-04-25T11:42:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.272+0000 7f7eb1600640 1 --2- 172.21.15.19:0/2324536268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac0b72b0 0x7f7eac161e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009870 tx=0x7f7ea402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c67455636555b3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.273+0000 7f7eab400640 1 -- 172.21.15.19:0/2324536268 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4037070 con 0x7f7eac0b72b0 2024-04-25T11:42:54.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.273+0000 7f7eab400640 1 -- 172.21.15.19:0/2324536268 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4032440 con 0x7f7eac0b72b0 2024-04-25T11:42:54.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.273+0000 7f7eab400640 1 -- 172.21.15.19:0/2324536268 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4032810 con 0x7f7eac0b72b0 2024-04-25T11:42:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 -- 172.21.15.19:0/2324536268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac0b72b0 msgr2=0x7f7eac161e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 --2- 172.21.15.19:0/2324536268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac0b72b0 0x7f7eac161e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009870 tx=0x7f7ea402a250 comp rx=0 tx=0).stop 2024-04-25T11:42:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 -- 172.21.15.19:0/2324536268 shutdown_connections 2024-04-25T11:42:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 --2- 172.21.15.19:0/2324536268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac0b72b0 0x7f7eac161e90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 --2- 172.21.15.19:0/2324536268 >> v2:172.21.15.169:3300/0 conn(0x7f7eac144310 0x7f7eac06fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 -- 172.21.15.19:0/2324536268 >> 172.21.15.19:0/2324536268 conn(0x7f7eac0b4260 msgr2=0x7f7eac072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 -- 172.21.15.19:0/2324536268 shutdown_connections 2024-04-25T11:42:54.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.274+0000 7f7eb2000640 1 -- 172.21.15.19:0/2324536268 wait complete. 2024-04-25T11:42:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.276+0000 7f7eb2000640 1 Processor -- start 2024-04-25T11:42:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.277+0000 7f7eb2000640 1 -- start start 2024-04-25T11:42:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac144310 0x7f7eac1f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb2000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac161a80 0x7f7eac1e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb2000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac1326e0 0x7f7eac15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb2000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1478d0 con 0x7f7eac144310 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb2000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac070010 con 0x7f7eac161a80 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb2000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac147a50 con 0x7f7eac1326e0 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac144310 0x7f7eac1f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac144310 0x7f7eac1f9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48650/0 (socket says 172.21.15.19:48650) 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.278+0000 7f7eb0c00640 1 -- 172.21.15.19:0/863503767 learned_addr learned my addr 172.21.15.19:0/863503767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eabe00640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac161a80 0x7f7eac1e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eb1600640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac1326e0 0x7f7eac15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eb0c00640 1 -- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac1326e0 msgr2=0x7f7eac15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac1326e0 0x7f7eac15d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eb0c00640 1 -- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac161a80 msgr2=0x7f7eac1e77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac161a80 0x7f7eac1e77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eb0c00640 1 -- 172.21.15.19:0/863503767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac0b0810 con 0x7f7eac144310 2024-04-25T11:42:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.279+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac144310 0x7f7eac1f9780 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00efc0 tx=0x7f7e9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.280+0000 7f7ea8c00640 1 -- 172.21.15.19:0/863503767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c018020 con 0x7f7eac144310 2024-04-25T11:42:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.280+0000 7f7ea8c00640 1 -- 172.21.15.19:0/863503767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c009e40 con 0x7f7eac144310 2024-04-25T11:42:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.280+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac1f9fd0 con 0x7f7eac144310 2024-04-25T11:42:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.280+0000 7f7ea8c00640 1 -- 172.21.15.19:0/863503767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0146a0 con 0x7f7eac144310 2024-04-25T11:42:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.280+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac1fa590 con 0x7f7eac144310 2024-04-25T11:42:54.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.283+0000 7f7ea8c00640 1 -- 172.21.15.19:0/863503767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e9c005d10 con 0x7f7eac144310 2024-04-25T11:42:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.284+0000 7f7ea8c00640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.284+0000 7f7ea8c00640 1 -- 172.21.15.19:0/863503767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f7e9c095fd0 con 0x7f7eac144310 2024-04-25T11:42:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.284+0000 7f7eba58c740 1 Processor -- start 2024-04-25T11:42:54.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.284+0000 7f7eba58c740 1 -- start start 2024-04-25T11:42:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.284+0000 7f7eabe00640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2874350 0x5644b2c41760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b284b490 0x5644b2c3e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eba58c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5644b2b7abc0 con 0x5644b2853670 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5644b2b16ed0 con 0x5644b2874350 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5644b2b76b80 con 0x5644b284b490 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eabe00640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7eac146e70 tx=0x7f7e98007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eb1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eb1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c173c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48654/0 (socket says 172.21.15.19:48654) 2024-04-25T11:42:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eb1600640 1 -- 172.21.15.19:0/2035685798 learned_addr learned my addr 172.21.15.19:0/2035685798 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eabe00640 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b284b490 0x5644b2c3e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2874350 0x5644b2c41760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eb1600640 1 -- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b284b490 msgr2=0x5644b2c3e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eb1600640 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b284b490 0x5644b2c3e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.286+0000 7f7eb1600640 1 -- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2874350 msgr2=0x5644b2c41760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7eb1600640 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2874350 0x5644b2c41760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7eb1600640 1 -- 172.21.15.19:0/2035685798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b29587c0 con 0x5644b2853670 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7eabe00640 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b284b490 0x5644b2c3e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2874350 0x5644b2c41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7eb1600640 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c173c0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009990 tx=0x7f7ea402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67815c941dc63132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7ea1200640 1 -- 172.21.15.19:0/2035685798 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4037040 con 0x5644b2853670 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7ea1200640 1 -- 172.21.15.19:0/2035685798 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4042070 con 0x5644b2853670 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.287+0000 7f7ea1200640 1 -- 172.21.15.19:0/2035685798 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4032920 con 0x5644b2853670 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 -- 172.21.15.19:0/2035685798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 msgr2=0x5644b2c173c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c173c0 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009990 tx=0x7f7ea402c000 comp rx=0 tx=0).stop 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 -- 172.21.15.19:0/2035685798 shutdown_connections 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c173c0 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b284b490 0x5644b2c3e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 --2- 172.21.15.19:0/2035685798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2874350 0x5644b2c41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 -- 172.21.15.19:0/2035685798 >> 172.21.15.19:0/2035685798 conn(0x5644b28684a0 msgr2=0x5644b2c16b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 -- 172.21.15.19:0/2035685798 shutdown_connections 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.288+0000 7f7eba58c740 1 -- 172.21.15.19:0/2035685798 wait complete. 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.289+0000 7f7eba58c740 1 -- start start 2024-04-25T11:42:54.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.290+0000 7f7eba58c740 1 Processor -- start 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.291+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c3cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.291+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.291+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c6c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.291+0000 7f7eba58c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5644b2b019f0 con 0x5644b2853670 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.291+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5644b2c5ba40 con 0x5644b284b490 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.291+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5644b2c6fe50 con 0x5644b2874350 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.291+0000 7f7eba58c740 10 client.0 _ll_register_callbacks cb 0x5644b2afff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48662/0 (socket says 172.21.15.19:48662) 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 -- 172.21.15.19:0/1614932185 learned_addr learned my addr 172.21.15.19:0/1614932185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c3cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eb1600640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c6c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 -- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 msgr2=0x5644b2c6c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c6c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 -- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 msgr2=0x5644b2c3cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c3cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eabe00640 1 -- 172.21.15.19:0/1614932185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b29b8f50 con 0x5644b2853670 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.292+0000 7f7eb1600640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c6c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59450 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f7e98001aa0 tx=0x7f7e98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0b435c34bb42a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7e95600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7e95600640 1 -- 172.21.15.19:0/1614932185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98005260 con 0x5644b2853670 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7e95600640 1 -- 172.21.15.19:0/1614932185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f7e98005400 con 0x5644b2853670 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea4042070 con 0x5644b2853670 2024-04-25T11:42:54.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7e95600640 1 -- 172.21.15.19:0/1614932185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=370) ==== 24+0+0 (secure 0 0 0) 0x7f7e98005600 con 0x5644b2853670 2024-04-25T11:42:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7eba58c740 10 client.11682 fetch_fsmap learned FSMap version 370 2024-04-25T11:42:54.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=370,monmap=2+}) -- 0x5644b284c0b0 con 0x5644b2853670 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.293+0000 7f7e95600640 1 -- 172.21.15.19:0/1614932185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4042070 con 0x5644b2853670 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7e95600640 1 -- 172.21.15.19:0/1614932185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 370) ==== 3494+0+0 (secure 0 0 0) 0x7f7e98022430 con 0x5644b2853670 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7eba58c740 10 client.11682 fetch_fsmap finished waiting for FSMap version 370 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7eba58c740 10 client.11682 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7eba58c740 20 client.11682 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2833130 0x5644b2c5a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7eba58c740 4 client.11682 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b2b887d0 con 0x5644b2833130 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.294+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2833130 0x5644b2c5a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.295+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2833130 0x5644b2c5a1e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e236f72989c39 server_cookie=1174e6695db2ea8e in_seq=0 out_seq=0 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.295+0000 7f7e95600640 10 client.11682 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:54.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7e95600640 1 -- 172.21.15.19:0/1614932185 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644b2b887d0 con 0x5644b2833130 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7e95600640 10 client.11682 handle_command_reply: tid=0 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 client.11682 shutdown 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 msgr2=0x5644b2c59450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59450 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f7e98001aa0 tx=0x7f7e98000bc0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 shutdown_connections 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2833130 0x5644b2c5a1e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c6c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59450 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1614932185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c3cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7e95600640 0 client.11682 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 >> 172.21.15.19:0/1614932185 conn(0x5644b28684a0 msgr2=0x5644b2c16b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 shutdown_connections 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.296+0000 7f7eba58c740 1 -- 172.21.15.19:0/1614932185 wait complete. 2024-04-25T11:42:54.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.297+0000 7f7eba58c740 20 client.11682 trim_cache size 0 max 16384 2024-04-25T11:42:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.312+0000 7f7eba58c740 1 Processor -- start 2024-04-25T11:42:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.312+0000 7f7eba58c740 1 -- start start 2024-04-25T11:42:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c58f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eba58c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac201f70 con 0x5644b2853670 2024-04-25T11:42:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac0b22c0 con 0x5644b284b490 2024-04-25T11:42:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac201b10 con 0x5644b2874350 2024-04-25T11:42:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48664/0 (socket says 172.21.15.19:48664) 2024-04-25T11:42:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eabe00640 1 -- 172.21.15.19:0/1365744139 learned_addr learned my addr 172.21.15.19:0/1365744139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c58f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.314+0000 7f7eb1600640 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7eabe00640 1 -- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 msgr2=0x5644b2c103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7eabe00640 1 -- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 msgr2=0x5644b2c58f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c58f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7eabe00640 1 -- 172.21.15.19:0/1365744139 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b2c117a0 con 0x5644b2853670 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59490 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006010 tx=0x7f7e98009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d24d08d438ab66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7e97e00640 1 -- 172.21.15.19:0/1365744139 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98030030 con 0x5644b2853670 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7e97e00640 1 -- 172.21.15.19:0/1365744139 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98022430 con 0x5644b2853670 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.315+0000 7f7e97e00640 1 -- 172.21.15.19:0/1365744139 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980225e0 con 0x5644b2853670 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 -- 172.21.15.19:0/1365744139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 msgr2=0x5644b2c59490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59490 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f7e98006010 tx=0x7f7e98009070 comp rx=0 tx=0).stop 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 -- 172.21.15.19:0/1365744139 shutdown_connections 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b2853670 0x5644b2c59490 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1365744139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b284b490 0x5644b2c58f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 -- 172.21.15.19:0/1365744139 >> 172.21.15.19:0/1365744139 conn(0x5644b28684a0 msgr2=0x5644b2c15330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 -- 172.21.15.19:0/1365744139 shutdown_connections 2024-04-25T11:42:54.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.316+0000 7f7eba58c740 1 -- 172.21.15.19:0/1365744139 wait complete. 2024-04-25T11:42:54.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.317+0000 7f7eba58c740 1 -- start start 2024-04-25T11:42:54.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.317+0000 7f7eba58c740 1 Processor -- start 2024-04-25T11:42:54.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b284b490 0x5644b2c5a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2853670 0x5644b2c6aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eba58c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eba58c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5644b2c5b7c0 con 0x5644b284b490 2024-04-25T11:42:54.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5644b2c6c940 con 0x5644b2853670 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eba58c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5644b2c3da40 con 0x5644b2874350 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eba58c740 10 client.0 _ll_register_callbacks cb 0x7f7e98007eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eb0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b284b490 0x5644b2c5a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eb0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b284b490 0x5644b2c5a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48680/0 (socket says 172.21.15.19:48680) 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.319+0000 7f7eb0c00640 1 -- 172.21.15.19:0/1971942503 learned_addr learned my addr 172.21.15.19:0/1971942503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2853670 0x5644b2c6aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eb0c00640 1 -- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 msgr2=0x5644b2c103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eb1600640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eb0c00640 1 -- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2853670 msgr2=0x5644b2c6aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2853670 0x5644b2c6aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eb0c00640 1 -- 172.21.15.19:0/1971942503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b2c3dd40 con 0x5644b284b490 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2853670 0x5644b2c6aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:54.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eb0c00640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b284b490 0x5644b2c5a260 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c990 tx=0x7f7e9c0600b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4d6037a69b8b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7e95600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7e95600640 1 -- 172.21.15.19:0/1971942503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c068b30 con 0x5644b284b490 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7e95600640 1 -- 172.21.15.19:0/1971942503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f7e9c05f250 con 0x5644b284b490 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.320+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e98022430 con 0x5644b284b490 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7e95600640 1 -- 172.21.15.19:0/1971942503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=370) ==== 24+0+0 (secure 0 0 0) 0x7f7e9c07f050 con 0x5644b284b490 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 10 client.11688 fetch_fsmap learned FSMap version 370 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=370,monmap=2+}) -- 0x5644b2c6b330 con 0x5644b284b490 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7e95600640 1 -- 172.21.15.19:0/1971942503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e98022430 con 0x5644b284b490 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7e95600640 1 -- 172.21.15.19:0/1971942503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 370) ==== 3494+0+0 (secure 0 0 0) 0x7f7e9c05b960 con 0x5644b284b490 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 10 client.11688 fetch_fsmap finished waiting for FSMap version 370 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 10 client.11688 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 20 client.11688 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2c17740 0x5644b2c17b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 4 client.11688 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T11:42:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5644b2b82640 con 0x5644b2c17740 2024-04-25T11:42:54.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.321+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2c17740 0x5644b2c17b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7e95600640 10 client.11688 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eabe00640 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2c17740 0x5644b2c17b50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbb6b62f4231298 server_cookie=d9dd4156b267d81d in_seq=0 out_seq=0 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7e95600640 1 -- 172.21.15.19:0/1971942503 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+145 (crc 0 0 0) 0x5644b2b82640 con 0x5644b2c17740 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7e95600640 10 client.11688 handle_command_reply: tid=0 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 client.11688 shutdown 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b284b490 msgr2=0x5644b2c5a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b284b490 0x5644b2c5a260 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c990 tx=0x7f7e9c0600b0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 shutdown_connections 2024-04-25T11:42:54.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5644b2c17740 0x5644b2c17b50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5644b2853670 0x5644b2c6aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5644b2874350 0x5644b2c103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 --2- 172.21.15.19:0/1971942503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644b284b490 0x5644b2c5a260 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7e95600640 0 client.11688 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.322+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 >> 172.21.15.19:0/1971942503 conn(0x5644b28684a0 msgr2=0x5644b2c591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.323+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 shutdown_connections 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.323+0000 7f7eba58c740 1 -- 172.21.15.19:0/1971942503 wait complete. 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.323+0000 7f7eba58c740 20 client.11688 trim_cache size 0 max 16384 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:54.312 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:42:54.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:54.313 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-04-25T11:42:54.313 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 2680762331, 2024-04-25T11:42:54.313 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-04-25T11:42:54.313 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/testdir/hardlink1" 2024-04-25T11:42:54.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:54.313 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.323+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e8c0744e0 msgr2=0x7f7e8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.323+0000 7f7eb2000640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7eac146e70 tx=0x7f7e98007ce0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.323+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac144310 msgr2=0x7f7eac1f9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.323+0000 7f7eb2000640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac144310 0x7f7eac1f9780 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00efc0 tx=0x7f7e9c00c490 comp rx=0 tx=0).stop 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 shutdown_connections 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f7e8c0744e0 0x7f7e8c0769f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac161a80 0x7f7eac1e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7eac1326e0 0x7f7eac15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 --2- 172.21.15.19:0/863503767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7eac144310 0x7f7eac1f9780 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 >> 172.21.15.19:0/863503767 conn(0x7f7eac0b72b0 msgr2=0x7f7eac070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 shutdown_connections 2024-04-25T11:42:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.324+0000 7f7eb2000640 1 -- 172.21.15.19:0/863503767 wait complete. 2024-04-25T11:42:54.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.634+0000 7fad74a00640 1 Processor -- start 2024-04-25T11:42:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.634+0000 7fad74a00640 1 -- start start 2024-04-25T11:42:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.636+0000 7fad74a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad70167000 0x7fad70169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.637+0000 7fad74a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad7015f5f0 con 0x7fad701443c0 2024-04-25T11:42:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.637+0000 7fad74a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad7015e830 con 0x7fad700cf5e0 2024-04-25T11:42:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.637+0000 7fad74a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad700708d0 con 0x7fad70167000 2024-04-25T11:42:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.637+0000 7fad6de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fad701443c0 0x7fad7006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55442/0 (socket says 172.21.15.19:55442) 2024-04-25T11:42:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.637+0000 7fad6de00640 1 -- 172.21.15.19:0/3900228724 learned_addr learned my addr 172.21.15.19:0/3900228724 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.637+0000 7fad6e800640 1 --2- 172.21.15.19:0/3900228724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad70167000 0x7fad70169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1342909187 0 0) 0x7fad7015f5f0 con 0x7fad701443c0 2024-04-25T11:42:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad54003520 con 0x7fad701443c0 2024-04-25T11:42:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2239548783 0 0) 0x7fad7015e830 con 0x7fad700cf5e0 2024-04-25T11:42:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad7015f5f0 con 0x7fad700cf5e0 2024-04-25T11:42:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2890923988 0 0) 0x7fad54003520 con 0x7fad701443c0 2024-04-25T11:42:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6e800640 1 -- 172.21.15.19:0/3900228724 >> v1:172.21.15.169:6789/0 conn(0x7fad700cf5e0 legacy=0x7fad700cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6e800640 1 -- 172.21.15.19:0/3900228724 >> v1:172.21.15.19:6789/0 conn(0x7fad701443c0 legacy=0x7fad7006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.638+0000 7fad6e800640 1 -- 172.21.15.19:0/3900228724 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7015eb10 con 0x7fad70167000 2024-04-25T11:42:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.640+0000 7fad6e800640 1 --2- 172.21.15.19:0/3900228724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad70167000 0x7fad70169420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad68009dc0 tx=0x7fad6802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776bfe76486ff47f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.640+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600015f0 con 0x7fad70167000 2024-04-25T11:42:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.640+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad68007540 con 0x7fad70167000 2024-04-25T11:42:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.640+0000 7fad6ca00640 1 -- 172.21.15.19:0/3900228724 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad680078f0 con 0x7fad70167000 2024-04-25T11:42:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.641+0000 7fad74a00640 1 -- 172.21.15.19:0/3900228724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad70167000 msgr2=0x7fad70169420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.641+0000 7fad74a00640 1 --2- 172.21.15.19:0/3900228724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad70167000 0x7fad70169420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad68009dc0 tx=0x7fad6802a660 comp rx=0 tx=0).stop 2024-04-25T11:42:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.641+0000 7fad74a00640 1 -- 172.21.15.19:0/3900228724 shutdown_connections 2024-04-25T11:42:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.641+0000 7fad74a00640 1 --2- 172.21.15.19:0/3900228724 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad70167000 0x7fad70169420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.641+0000 7fad74a00640 1 -- 172.21.15.19:0/3900228724 >> 172.21.15.19:0/3900228724 conn(0x7fad700d1ae0 msgr2=0x7fad700d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.642+0000 7fad74a00640 1 -- 172.21.15.19:0/3900228724 shutdown_connections 2024-04-25T11:42:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.642+0000 7fad74a00640 1 -- 172.21.15.19:0/3900228724 wait complete. 2024-04-25T11:42:54.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.645+0000 7fad74a00640 1 Processor -- start 2024-04-25T11:42:54.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.645+0000 7fad74a00640 1 -- start start 2024-04-25T11:42:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.647+0000 7fad74a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad700cf5e0 0x7fad70143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.647+0000 7fad74a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad701443c0 0x7fad700ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.647+0000 7fad74a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad70167000 0x7fad70165d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.647+0000 7fad74a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad7006d5d0 con 0x7fad70167000 2024-04-25T11:42:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.648+0000 7fad74a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fad7015f160 con 0x7fad701443c0 2024-04-25T11:42:54.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.648+0000 7fad74a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fad701447d0 con 0x7fad700cf5e0 2024-04-25T11:42:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.648+0000 7fad6e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad70167000 0x7fad70165d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.648+0000 7fad6e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad70167000 0x7fad70165d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48682/0 (socket says 172.21.15.19:48682) 2024-04-25T11:42:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.648+0000 7fad6e800640 1 -- 172.21.15.19:0/3409131901 learned_addr learned my addr 172.21.15.19:0/3409131901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.648+0000 7fad6de00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad700cf5e0 0x7fad70143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.648+0000 7fad6d400640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad701443c0 0x7fad700ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad6e800640 1 -- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad700cf5e0 msgr2=0x7fad70143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad6e800640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad700cf5e0 0x7fad70143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad6e800640 1 -- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad701443c0 msgr2=0x7fad700ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad6e800640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad701443c0 0x7fad700ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad6e800640 1 -- 172.21.15.19:0/3409131901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad700aa3d0 con 0x7fad70167000 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad6d400640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad701443c0 0x7fad700ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad6e800640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad70167000 0x7fad70165d70 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fad68009ee0 tx=0x7fad68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad66000640 1 -- 172.21.15.19:0/3409131901 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68009450 con 0x7fad70167000 2024-04-25T11:42:54.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad66000640 1 -- 172.21.15.19:0/3409131901 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad68004240 con 0x7fad70167000 2024-04-25T11:42:54.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad66000640 1 -- 172.21.15.19:0/3409131901 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad680045f0 con 0x7fad70167000 2024-04-25T11:42:54.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.649+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad700aa6b0 con 0x7fad70167000 2024-04-25T11:42:54.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.650+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad700aac70 con 0x7fad70167000 2024-04-25T11:42:54.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.651+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad70050740 con 0x7fad70167000 2024-04-25T11:42:54.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.653+0000 7fad66000640 1 -- 172.21.15.19:0/3409131901 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad68039040 con 0x7fad70167000 2024-04-25T11:42:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.654+0000 7fad66000640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad500744b0 0x7fad500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.654+0000 7fad66000640 1 -- 172.21.15.19:0/3409131901 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fad680b67e0 con 0x7fad70167000 2024-04-25T11:42:54.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.654+0000 7fad6de00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad500744b0 0x7fad500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.656+0000 7fad6de00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad500744b0 0x7fad500769c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fad60001400 tx=0x7fad60001390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.659+0000 7fad66000640 1 -- 172.21.15.19:0/3409131901 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad68040030 con 0x7fad70167000 2024-04-25T11:42:54.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.898+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad7006f950 con 0x7fad70167000 2024-04-25T11:42:54.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.899+0000 7fad66000640 1 -- 172.21.15.19:0/3409131901 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 370 v370) ==== 95+0+5194 (secure 0 0 0) 0x7fad68080f00 con 0x7fad70167000 2024-04-25T11:42:54.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:54.886 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":370,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":370,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:48.519583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":9,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:42:54.886 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 370 2024-04-25T11:42:54.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.901+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad500744b0 msgr2=0x7fad500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.901+0000 7fad74a00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad500744b0 0x7fad500769c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fad60001400 tx=0x7fad60001390 comp rx=0 tx=0).stop 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad70167000 msgr2=0x7fad70165d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad70167000 0x7fad70165d70 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fad68009ee0 tx=0x7fad68004a40 comp rx=0 tx=0).stop 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 shutdown_connections 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fad500744b0 0x7fad500769c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad701443c0 0x7fad700ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad70167000 0x7fad70165d70 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 --2- 172.21.15.19:0/3409131901 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad700cf5e0 0x7fad70143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 >> 172.21.15.19:0/3409131901 conn(0x7fad700d1ae0 msgr2=0x7fad70160b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 shutdown_connections 2024-04-25T11:42:54.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:54.902+0000 7fad74a00640 1 -- 172.21.15.19:0/3409131901 wait complete. 2024-04-25T11:42:54.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 2680762331 2024-04-25T11:42:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.210+0000 7f1c6be00640 1 Processor -- start 2024-04-25T11:42:55.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.210+0000 7f1c6be00640 1 -- start start 2024-04-25T11:42:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.212+0000 7f1c6be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c6c0cf870 0x7f1c6c0d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.212+0000 7f1c6be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c6c0cfc80 con 0x7f1c6c15d620 2024-04-25T11:42:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.212+0000 7f1c6be00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c6c0cff10 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.212+0000 7f1c6be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1c6c06dce0 con 0x7f1c6c0cf870 2024-04-25T11:42:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.213+0000 7f1c6aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c6c0cf870 0x7f1c6c0d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.213+0000 7f1c6aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1c6c0cf870 0x7f1c6c0d0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48698/0 (socket says 172.21.15.19:48698) 2024-04-25T11:42:55.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.213+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2722624502 learned_addr learned my addr 172.21.15.19:0/2722624502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.213+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2722624502 >> v1:172.21.15.169:6789/0 conn(0x7f1c6c0cdd60 legacy=0x7f1c6c141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.213+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2722624502 >> v1:172.21.15.19:6789/0 conn(0x7f1c6c15d620 legacy=0x7f1c6c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.213+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2722624502 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c0d01f0 con 0x7f1c6c0cf870 2024-04-25T11:42:55.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.213+0000 7f1c6a000640 1 -- 172.21.15.19:0/2722624502 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693422590 0 0) 0x7f1c6c0cfc80 con 0x7f1c6c15d620 2024-04-25T11:42:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.214+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/2722624502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cf870 0x7f1c6c0d0fb0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00bb10 tx=0x7f1c5c02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10167e59a928a486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.215+0000 7f1c6a000640 1 -- 172.21.15.19:0/2722624502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c032790 con 0x7f1c6c0cf870 2024-04-25T11:42:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.215+0000 7f1c6a000640 1 -- 172.21.15.19:0/2722624502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c5c038070 con 0x7f1c6c0cf870 2024-04-25T11:42:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.215+0000 7f1c6a000640 1 -- 172.21.15.19:0/2722624502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c032b50 con 0x7f1c6c0cf870 2024-04-25T11:42:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.215+0000 7f1c6be00640 1 -- 172.21.15.19:0/2722624502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cf870 msgr2=0x7f1c6c0d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.215+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2722624502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cf870 0x7f1c6c0d0fb0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00bb10 tx=0x7f1c5c02c8d0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.216+0000 7f1c6be00640 1 -- 172.21.15.19:0/2722624502 shutdown_connections 2024-04-25T11:42:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.216+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2722624502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cf870 0x7f1c6c0d0fb0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.216+0000 7f1c6be00640 1 -- 172.21.15.19:0/2722624502 >> 172.21.15.19:0/2722624502 conn(0x7f1c6c072f40 msgr2=0x7f1c6c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.216+0000 7f1c6be00640 1 -- 172.21.15.19:0/2722624502 shutdown_connections 2024-04-25T11:42:55.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.216+0000 7f1c6be00640 1 -- 172.21.15.19:0/2722624502 wait complete. 2024-04-25T11:42:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.218+0000 7f1c6be00640 1 Processor -- start 2024-04-25T11:42:55.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.218+0000 7f1c6be00640 1 -- start start 2024-04-25T11:42:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cdd60 0x7f1c6c13d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c6c0cf870 0x7f1c6c1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c6c15d620 0x7f1c6c1419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c0d0a90 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c0d07b0 con 0x7f1c6c15d620 2024-04-25T11:42:55.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6be00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c6c15da30 con 0x7f1c6c0cf870 2024-04-25T11:42:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cdd60 0x7f1c6c13d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cdd60 0x7f1c6c13d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48700/0 (socket says 172.21.15.19:48700) 2024-04-25T11:42:55.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2391331755 learned_addr learned my addr 172.21.15.19:0/2391331755 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c6c0cf870 0x7f1c6c1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.220+0000 7f1c6b400640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c6c15d620 0x7f1c6c1419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c6c0cf870 msgr2=0x7f1c6c1556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c6c0cf870 0x7f1c6c1556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c6c15d620 msgr2=0x7f1c6c1419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c6c15d620 0x7f1c6c1419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6aa00640 1 -- 172.21.15.19:0/2391331755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c157dc0 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6b400640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c6c15d620 0x7f1c6c1419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cdd60 0x7f1c6c13d620 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00b2c0 tx=0x7f1c5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c62a00640 1 -- 172.21.15.19:0/2391331755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c033830 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c62a00640 1 -- 172.21.15.19:0/2391331755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c5c033a10 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c62a00640 1 -- 172.21.15.19:0/2391331755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c033e00 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c13db60 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.221+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c156560 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.224+0000 7f1c62a00640 1 -- 172.21.15.19:0/2391331755 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1c5c04e440 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.225+0000 7f1c62a00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c480744b0 0x7f1c480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.225+0000 7f1c62a00640 1 -- 172.21.15.19:0/2391331755 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f1c5c0b6270 con 0x7f1c6c0cdd60 2024-04-25T11:42:55.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.225+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c480744b0 0x7f1c480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.225+0000 7f1c78d7c740 1 Processor -- start 2024-04-25T11:42:55.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.226+0000 7f1c78d7c740 1 -- start start 2024-04-25T11:42:55.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c480744b0 0x7f1c480769c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000f60 tx=0x7f1c54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574049092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x5574048dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557404843900 con 0x5574045143e0 2024-04-25T11:42:55.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5574047e01f0 con 0x55740451c5c0 2024-04-25T11:42:55.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55740483f8c0 con 0x55740453d2a0 2024-04-25T11:42:55.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x5574048dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x5574048dd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48706/0 (socket says 172.21.15.19:48706) 2024-04-25T11:42:55.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.227+0000 7f1c63e00640 1 -- 172.21.15.19:0/628077807 learned_addr learned my addr 172.21.15.19:0/628077807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574049092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c63e00640 1 -- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 msgr2=0x5574049092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c63e00640 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574049092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c63e00640 1 -- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 msgr2=0x5574048dd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c6b400640 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c63e00640 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c63e00640 1 -- 172.21.15.19:0/628077807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557404621710 con 0x5574045143e0 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.228+0000 7f1c63e00640 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x5574048dd3d0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f1c54001e90 tx=0x7f1c54011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42b2ac58b6129c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c5b400640 1 -- 172.21.15.19:0/628077807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c540035f0 con 0x5574045143e0 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c5b400640 1 -- 172.21.15.19:0/628077807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c54003790 con 0x5574045143e0 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c5b400640 1 -- 172.21.15.19:0/628077807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54003b60 con 0x5574045143e0 2024-04-25T11:42:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c78d7c740 1 -- 172.21.15.19:0/628077807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 msgr2=0x5574048dd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x5574048dd3d0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f1c54001e90 tx=0x7f1c54011000 comp rx=0 tx=0).stop 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c78d7c740 1 -- 172.21.15.19:0/628077807 shutdown_connections 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x5574048dd3d0 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/628077807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574049092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.229+0000 7f1c78d7c740 1 -- 172.21.15.19:0/628077807 >> 172.21.15.19:0/628077807 conn(0x5574045313f0 msgr2=0x557404924100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.230+0000 7f1c78d7c740 1 -- 172.21.15.19:0/628077807 shutdown_connections 2024-04-25T11:42:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.230+0000 7f1c78d7c740 1 -- 172.21.15.19:0/628077807 wait complete. 2024-04-25T11:42:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.231+0000 7f1c78d7c740 1 -- start start 2024-04-25T11:42:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.231+0000 7f1c78d7c740 1 Processor -- start 2024-04-25T11:42:55.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.232+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 0x557404904a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.232+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x557404904f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x55740490b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5574047cb040 con 0x55740451c5c0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55740490cae0 con 0x5574045143e0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5574049208c0 con 0x55740453d2a0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c78d7c740 10 client.0 _ll_register_callbacks cb 0x5574047fb220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x557404904f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x557404904f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48712/0 (socket says 172.21.15.19:48712) 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 -- 172.21.15.19:0/2503389716 learned_addr learned my addr 172.21.15.19:0/2503389716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 -- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 msgr2=0x55740490b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c6b400640 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x55740490b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 0x557404904a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x55740490b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 -- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 msgr2=0x557404904a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 0x557404904a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.233+0000 7f1c63e00640 1 -- 172.21.15.19:0/2503389716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557404681ea0 con 0x55740451c5c0 2024-04-25T11:42:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x557404904f40 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400eb80 tx=0x7f1c54000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae1f5f5862f6f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c4fe00640 1 -- 172.21.15.19:0/2503389716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54021450 con 0x55740451c5c0 2024-04-25T11:42:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c54003790 con 0x55740451c5c0 2024-04-25T11:42:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c4fe00640 1 -- 172.21.15.19:0/2503389716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f1c540215f0 con 0x55740451c5c0 2024-04-25T11:42:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c4fe00640 1 -- 172.21.15.19:0/2503389716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=370) ==== 24+0+0 (secure 0 0 0) 0x7f1c54032050 con 0x55740451c5c0 2024-04-25T11:42:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c78d7c740 10 client.11709 fetch_fsmap learned FSMap version 370 2024-04-25T11:42:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=370,monmap=2+}) -- 0x557404515000 con 0x55740451c5c0 2024-04-25T11:42:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.234+0000 7f1c4fe00640 1 -- 172.21.15.19:0/2503389716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c54003790 con 0x55740451c5c0 2024-04-25T11:42:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c4fe00640 1 -- 172.21.15.19:0/2503389716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 370) ==== 3494+0+0 (secure 0 0 0) 0x7f1c54035020 con 0x55740451c5c0 2024-04-25T11:42:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c78d7c740 10 client.11709 fetch_fsmap finished waiting for FSMap version 370 2024-04-25T11:42:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c78d7c740 10 client.11709 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c78d7c740 20 client.11709 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574044fc080 0x557404941560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c78d7c740 4 client.11709 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557404851510 con 0x5574044fc080 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574044fc080 0x557404941560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.236+0000 7f1c4fe00640 10 client.11709 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.235+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574044fc080 0x557404941560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e306dab6220d378 server_cookie=72c21e7901b677ef in_seq=0 out_seq=0 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.236+0000 7f1c4fe00640 1 -- 172.21.15.19:0/2503389716 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557404851510 con 0x5574044fc080 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.236+0000 7f1c4fe00640 10 client.11709 handle_command_reply: tid=0 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.236+0000 7f1c78d7c740 1 client.11709 shutdown 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 msgr2=0x557404904f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x557404904f40 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400eb80 tx=0x7f1c54000ed0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 shutdown_connections 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574044fc080 0x557404941560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x55740490b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x557404904f40 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c4fe00640 0 client.11709 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2503389716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 0x557404904a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 >> 172.21.15.19:0/2503389716 conn(0x5574045313f0 msgr2=0x557404909a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 shutdown_connections 2024-04-25T11:42:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2503389716 wait complete. 2024-04-25T11:42:55.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.237+0000 7f1c78d7c740 20 client.11709 trim_cache size 0 max 16384 2024-04-25T11:42:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.252+0000 7f1c78d7c740 1 Processor -- start 2024-04-25T11:42:55.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.253+0000 7f1c78d7c740 1 -- start start 2024-04-25T11:42:55.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 0x55740490b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x55740491dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1fd910 con 0x55740451c5c0 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1fcca0 con 0x5574045143e0 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1c6c1fd2c0 con 0x55740453d2a0 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x55740491dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c63e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x55740491dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48724/0 (socket says 172.21.15.19:48724) 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.254+0000 7f1c63e00640 1 -- 172.21.15.19:0/2967520694 learned_addr learned my addr 172.21.15.19:0/2967520694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c63e00640 1 -- 172.21.15.19:0/2967520694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 msgr2=0x5574048dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2967520694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c63e00640 1 -- 172.21.15.19:0/2967520694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 msgr2=0x55740490b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2967520694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 0x55740490b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c63e00640 1 -- 172.21.15.19:0/2967520694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55740490d4c0 con 0x55740451c5c0 2024-04-25T11:42:55.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c63e00640 1 --2- 172.21.15.19:0/2967520694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x55740491dab0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f1c54003c00 tx=0x7f1c54027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3ed5e61ba23b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c5aa00640 1 -- 172.21.15.19:0/2967520694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c540035f0 con 0x55740451c5c0 2024-04-25T11:42:55.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c5aa00640 1 -- 172.21.15.19:0/2967520694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c54003790 con 0x55740451c5c0 2024-04-25T11:42:55.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.255+0000 7f1c5aa00640 1 -- 172.21.15.19:0/2967520694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54021650 con 0x55740451c5c0 2024-04-25T11:42:55.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2967520694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 msgr2=0x55740491dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2967520694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x55740491dab0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f1c54003c00 tx=0x7f1c54027000 comp rx=0 tx=0).stop 2024-04-25T11:42:55.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2967520694 shutdown_connections 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2967520694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2967520694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55740451c5c0 0x55740491dab0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/2967520694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5574045143e0 0x55740490b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2967520694 >> 172.21.15.19:0/2967520694 conn(0x5574045313f0 msgr2=0x557404905500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2967520694 shutdown_connections 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.256+0000 7f1c78d7c740 1 -- 172.21.15.19:0/2967520694 wait complete. 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.257+0000 7f1c78d7c740 1 -- start start 2024-04-25T11:42:55.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.257+0000 7f1c78d7c740 1 Processor -- start 2024-04-25T11:42:55.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x557404903910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c78d7c740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55740490bce0 con 0x5574045143e0 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557404920450 con 0x55740451c5c0 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c78d7c740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55740490c330 con 0x55740453d2a0 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c78d7c740 10 client.0 _ll_register_callbacks cb 0x7f1c54001d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c6aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x557404903910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c6aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x557404903910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48728/0 (socket says 172.21.15.19:48728) 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.259+0000 7f1c6aa00640 1 -- 172.21.15.19:0/4240752406 learned_addr learned my addr 172.21.15.19:0/4240752406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6b400640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6aa00640 1 -- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 msgr2=0x5574048e04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c63e00640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048e04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6aa00640 1 -- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 msgr2=0x5574048dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6aa00640 1 -- 172.21.15.19:0/4240752406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557404921b80 con 0x5574045143e0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6b400640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c6aa00640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x557404903910 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c02cec0 tx=0x7f1c5c07fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6878ec0113ba6eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c4fe00640 1 -- 172.21.15.19:0/4240752406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c088dd0 con 0x5574045143e0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c4fe00640 1 -- 172.21.15.19:0/4240752406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f1c5c084150 con 0x5574045143e0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c54003790 con 0x5574045143e0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c4fe00640 1 -- 172.21.15.19:0/4240752406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=370) ==== 24+0+0 (secure 0 0 0) 0x7f1c5c080290 con 0x5574045143e0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c78d7c740 10 client.11715 fetch_fsmap learned FSMap version 370 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.260+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=370,monmap=2+}) -- 0x557404922410 con 0x5574045143e0 2024-04-25T11:42:55.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c4fe00640 1 -- 172.21.15.19:0/4240752406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c54003790 con 0x5574045143e0 2024-04-25T11:42:55.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c4fe00640 1 -- 172.21.15.19:0/4240752406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 370) ==== 3494+0+0 (secure 0 0 0) 0x7f1c5c043070 con 0x5574045143e0 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c78d7c740 10 client.11715 fetch_fsmap finished waiting for FSMap version 370 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c78d7c740 10 client.11715 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c78d7c740 20 client.11715 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574048d8df0 0x557404934cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c78d7c740 4 client.11715 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2680762331}] 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2680762331}) -- 0x5574049267f0 con 0x5574048d8df0 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.261+0000 7f1c63e00640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574048d8df0 0x557404934cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c63e00640 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574048d8df0 0x557404934cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e482a9a3ea97704a server_cookie=2405940ac45ed2c8 in_seq=0 out_seq=0 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c4fe00640 10 client.11715 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c4fe00640 1 -- 172.21.15.19:0/4240752406 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5574049267f0 con 0x5574048d8df0 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c4fe00640 10 client.11715 handle_command_reply: tid=0 2024-04-25T11:42:55.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 client.11715 shutdown 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 msgr2=0x557404903910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x557404903910 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c02cec0 tx=0x7f1c5c07fa00 comp rx=0 tx=0).stop 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 shutdown_connections 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5574048d8df0 0x557404934cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55740451c5c0 0x5574048dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55740453d2a0 0x5574048e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 --2- 172.21.15.19:0/4240752406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5574045143e0 0x557404903910 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c4fe00640 0 client.11715 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 >> 172.21.15.19:0/4240752406 conn(0x5574045313f0 msgr2=0x5574048dd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.262+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 shutdown_connections 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c78d7c740 1 -- 172.21.15.19:0/4240752406 wait complete. 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c78d7c740 20 client.11715 trim_cache size 0 max 16384 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c480744b0 msgr2=0x7f1c480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c480744b0 0x7f1c480769c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000f60 tx=0x7f1c54009040 comp rx=0 tx=0).stop 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cdd60 msgr2=0x7f1c6c13d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cdd60 0x7f1c6c13d620 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00b2c0 tx=0x7f1c5c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 shutdown_connections 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c480744b0 0x7f1c480769c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1c6c15d620 0x7f1c6c1419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1c6c0cf870 0x7f1c6c1556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 --2- 172.21.15.19:0/2391331755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6c0cdd60 0x7f1c6c13d620 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.263+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 >> 172.21.15.19:0/2391331755 conn(0x7f1c6c072f40 msgr2=0x7f1c6c0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.264+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 shutdown_connections 2024-04-25T11:42:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.264+0000 7f1c6be00640 1 -- 172.21.15.19:0/2391331755 wait complete. 2024-04-25T11:42:55.273 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100000001f6.00000000 2024-04-25T11:42:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.471+0000 7f43fc93cf40 1 Processor -- start 2024-04-25T11:42:55.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.471+0000 7f43fc93cf40 1 -- start start 2024-04-25T11:42:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.472+0000 7f43fc93cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb1e0 0x55a4a77ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43fc93cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb5f0 0x55a4a76ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43fc93cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 0x55a4a76eaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43fc93cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4a77343f0 con 0x55a4a76eb5f0 2024-04-25T11:42:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43fc93cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a4a7733e70 con 0x55a4a77c4dd0 2024-04-25T11:42:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43fc93cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a4a77346d0 con 0x55a4a76eb1e0 2024-04-25T11:42:55.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb5f0 0x55a4a76ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb5f0 0x55a4a76ea570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48740/0 (socket says 172.21.15.19:48740) 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 -- 172.21.15.19:0/3983366355 learned_addr learned my addr 172.21.15.19:0/3983366355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5a00640 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb1e0 0x55a4a77ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 -- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb1e0 msgr2=0x55a4a77ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb1e0 0x55a4a77ba0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f6400640 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 0x55a4a76eaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 -- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 msgr2=0x55a4a76eaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 0x55a4a76eaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.473+0000 7f43f5000640 1 -- 172.21.15.19:0/3983366355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a7614b30 con 0x55a4a76eb5f0 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.474+0000 7f43f5000640 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb5f0 0x55a4a76ea570 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f43e4025c20 tx=0x7f43e404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbb9cc6b9769dae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.474+0000 7f43ef400640 1 -- 172.21.15.19:0/3983366355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4065070 con 0x55a4a76eb5f0 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.474+0000 7f43ef400640 1 -- 172.21.15.19:0/3983366355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e405a040 con 0x55a4a76eb5f0 2024-04-25T11:42:55.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.474+0000 7f43ef400640 1 -- 172.21.15.19:0/3983366355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4059070 con 0x55a4a76eb5f0 2024-04-25T11:42:55.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 -- 172.21.15.19:0/3983366355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb5f0 msgr2=0x55a4a76ea570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb5f0 0x55a4a76ea570 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f43e4025c20 tx=0x7f43e404e000 comp rx=0 tx=0).stop 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 -- 172.21.15.19:0/3983366355 shutdown_connections 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 0x55a4a76eaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb5f0 0x55a4a76ea570 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/3983366355 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb1e0 0x55a4a77ba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 -- 172.21.15.19:0/3983366355 >> 172.21.15.19:0/3983366355 conn(0x55a4a760dac0 msgr2=0x55a4a771b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 -- 172.21.15.19:0/3983366355 shutdown_connections 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.475+0000 7f43fc93cf40 1 -- 172.21.15.19:0/3983366355 wait complete. 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.476+0000 7f43fc93cf40 1 Processor -- start 2024-04-25T11:42:55.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.476+0000 7f43fc93cf40 1 -- start start 2024-04-25T11:42:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43fc93cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb1e0 0x55a4a771d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43fc93cf40 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb5f0 0x55a4a77aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43fc93cf40 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 0x55a4a771b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43fc93cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4a77c11b0 con 0x55a4a76eb1e0 2024-04-25T11:42:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43fc93cf40 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55a4a7851990 con 0x55a4a77c4dd0 2024-04-25T11:42:55.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43fc93cf40 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55a4a77c01e0 con 0x55a4a76eb5f0 2024-04-25T11:42:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43f5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb1e0 0x55a4a771d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43f5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb1e0 0x55a4a771d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48746/0 (socket says 172.21.15.19:48746) 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43f5a00640 1 -- 172.21.15.19:0/2774107013 learned_addr learned my addr 172.21.15.19:0/2774107013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43f5000640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb5f0 0x55a4a77aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.478+0000 7f43f6400640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 0x55a4a771b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43f5a00640 1 -- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb5f0 msgr2=0x55a4a77aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43f5a00640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55a4a76eb5f0 0x55a4a77aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43f5a00640 1 -- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 msgr2=0x55a4a771b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43f5a00640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55a4a77c4dd0 0x55a4a771b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43f5a00640 1 -- 172.21.15.19:0/2774107013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a77b8fc0 con 0x55a4a76eb1e0 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43f5a00640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb1e0 0x55a4a771d5c0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f43e800bd90 tx=0x7f43e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43ee000640 1 -- 172.21.15.19:0/2774107013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8019070 con 0x55a4a76eb1e0 2024-04-25T11:42:55.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43ee000640 1 -- 172.21.15.19:0/2774107013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e8007af0 con 0x55a4a76eb1e0 2024-04-25T11:42:55.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43ee000640 1 -- 172.21.15.19:0/2774107013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8014410 con 0x55a4a76eb1e0 2024-04-25T11:42:55.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4a77b92a0 con 0x55a4a76eb1e0 2024-04-25T11:42:55.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.479+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a77c0430 con 0x55a4a76eb1e0 2024-04-25T11:42:55.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.482+0000 7f43ee000640 1 -- 172.21.15.19:0/2774107013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f43e801c070 con 0x55a4a76eb1e0 2024-04-25T11:42:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.483+0000 7f43ee000640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43d8072e90 0x7f43d80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.483+0000 7f43ee000640 1 -- 172.21.15.19:0/2774107013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f43e8092f20 con 0x55a4a76eb1e0 2024-04-25T11:42:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.483+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x55a4a7851550 0x55a4a77b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.483+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- osd_op(unknown.0.0:1 14.a 14:57fe7535:::100000001f6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e144) -- 0x55a4a77b7190 con 0x55a4a7851550 2024-04-25T11:42:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.483+0000 7f43f5000640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43d8072e90 0x7f43d80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.484+0000 7f43f6400640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x55a4a7851550 0x55a4a77b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.484+0000 7f43f6400640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x55a4a7851550 0x55a4a77b6c50 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.485+0000 7f43f5000640 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43d8072e90 0x7f43d80753a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f43e4045960 tx=0x7f43e4051000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43f6400640 1 -- 172.21.15.19:0/2774107013 <== osd.2 v2:172.21.15.19:6813/2392070031 1 ==== osd_op_reply(1 100000001f6.00000000 [delete] v144'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43f0011040 con 0x55a4a7851550 2024-04-25T11:42:55.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x55a4a7851550 msgr2=0x55a4a77b6c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x55a4a7851550 0x55a4a77b6c50 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43d8072e90 msgr2=0x7f43d80753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f43d8072e90 0x7f43d80753a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f43e4045960 tx=0x7f43e4051000 comp rx=0 tx=0).stop 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb1e0 msgr2=0x55a4a771d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 --2- 172.21.15.19:0/2774107013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4a76eb1e0 0x55a4a771d5c0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f43e800bd90 tx=0x7f43e800be80 comp rx=0 tx=0).stop 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43f6400640 1 -- 172.21.15.19:0/2774107013 reap_dead start 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 shutdown_connections 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 >> 172.21.15.19:0/2774107013 conn(0x55a4a760dac0 msgr2=0x55a4a77be100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 shutdown_connections 2024-04-25T11:42:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.488+0000 7f43fc93cf40 1 -- 172.21.15.19:0/2774107013 wait complete. 2024-04-25T11:42:55.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:42:55.489 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink2) 2024-04-25T11:42:55.535 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-04-25T11:42:55.549+0000 7f90a9800640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-04-25T11:42:55.536 INFO:teuthology.orchestra.run.smithi019.stderr:stat: cannot statx 'testdir/hardlink2': Input/output error 2024-04-25T11:42:55.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:42:55.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.829+0000 7f3ff1e00640 1 Processor -- start 2024-04-25T11:42:55.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.829+0000 7f3ff1e00640 1 -- start start 2024-04-25T11:42:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3ff1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fec0b4830 0x7f3fec06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3ff1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b2dc0 0x7f3fec0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3ff1e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fec06d870 con 0x7f3fec0d4570 2024-04-25T11:42:55.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3ff1e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3fec0b4c40 con 0x7f3fec0b4830 2024-04-25T11:42:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3ff1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3fec06e460 con 0x7f3fec0b2dc0 2024-04-25T11:42:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3fea000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fec0b4830 0x7f3fec06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3fea000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3fec0b4830 0x7f3fec06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48756/0 (socket says 172.21.15.19:48756) 2024-04-25T11:42:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3fea000640 1 -- 172.21.15.19:0/3275365481 learned_addr learned my addr 172.21.15.19:0/3275365481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.832+0000 7f3feb400640 1 --2- 172.21.15.19:0/3275365481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b2dc0 0x7f3fec0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.833+0000 7f3fea000640 1 -- 172.21.15.19:0/3275365481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b2dc0 msgr2=0x7f3fec0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.833+0000 7f3fea000640 1 --2- 172.21.15.19:0/3275365481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b2dc0 0x7f3fec0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.833+0000 7f3fea000640 1 -- 172.21.15.19:0/3275365481 >> v1:172.21.15.169:6789/0 conn(0x7f3fec0d4570 legacy=0x7f3fec0d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.833+0000 7f3fea000640 1 -- 172.21.15.19:0/3275365481 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec06db10 con 0x7f3fec0b4830 2024-04-25T11:42:55.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.834+0000 7f3fea000640 1 --2- 172.21.15.19:0/3275365481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0b4830 0x7f3fec06e8f0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f3fec06e5e0 tx=0x7f3fd802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a2d7a9ab706717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.834+0000 7f3fe9600640 1 -- 172.21.15.19:0/3275365481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8037070 con 0x7f3fec0b4830 2024-04-25T11:42:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.834+0000 7f3fe9600640 1 -- 172.21.15.19:0/3275365481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd8032440 con 0x7f3fec0b4830 2024-04-25T11:42:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.834+0000 7f3fe9600640 1 -- 172.21.15.19:0/3275365481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8032810 con 0x7f3fec0b4830 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.835+0000 7f3ff1e00640 1 -- 172.21.15.19:0/3275365481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0b4830 msgr2=0x7f3fec06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.835+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/3275365481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0b4830 0x7f3fec06e8f0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f3fec06e5e0 tx=0x7f3fd802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.835+0000 7f3ff1e00640 1 -- 172.21.15.19:0/3275365481 shutdown_connections 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.835+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/3275365481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b2dc0 0x7f3fec0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.835+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/3275365481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0b4830 0x7f3fec06e8f0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.835+0000 7f3ff1e00640 1 -- 172.21.15.19:0/3275365481 >> 172.21.15.19:0/3275365481 conn(0x7f3fec0d3a50 msgr2=0x7f3fec0d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.836+0000 7f3ff1e00640 1 -- 172.21.15.19:0/3275365481 shutdown_connections 2024-04-25T11:42:55.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.836+0000 7f3ff1e00640 1 -- 172.21.15.19:0/3275365481 wait complete. 2024-04-25T11:42:55.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.839+0000 7f3ff1e00640 1 Processor -- start 2024-04-25T11:42:55.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.839+0000 7f3ff1e00640 1 -- start start 2024-04-25T11:42:55.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.841+0000 7f3ff1e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fec0b2dc0 0x7f3fec0a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.841+0000 7f3ff1e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b4830 0x7f3fec157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.841+0000 7f3ff1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0d4570 0x7f3fec0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.841+0000 7f3ff1e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec0d0fb0 con 0x7f3fec0d4570 2024-04-25T11:42:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.841+0000 7f3ff1e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec0d0bc0 con 0x7f3fec0b2dc0 2024-04-25T11:42:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.841+0000 7f3ff1e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3fec0b3710 con 0x7f3fec0b4830 2024-04-25T11:42:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0d4570 0x7f3fec0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0d4570 0x7f3fec0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48758/0 (socket says 172.21.15.19:48758) 2024-04-25T11:42:55.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 -- 172.21.15.19:0/570879417 learned_addr learned my addr 172.21.15.19:0/570879417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:55.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3fea000640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b4830 0x7f3fec157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feaa00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fec0b2dc0 0x7f3fec0a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 -- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b4830 msgr2=0x7f3fec157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b4830 0x7f3fec157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 -- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fec0b2dc0 msgr2=0x7f3fec0a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fec0b2dc0 0x7f3fec0a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3feb400640 1 -- 172.21.15.19:0/570879417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec0c83d0 con 0x7f3fec0d4570 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.842+0000 7f3fea000640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b4830 0x7f3fec157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.843+0000 7f3feb400640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0d4570 0x7f3fec0c7e90 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4007e90 tx=0x7f3fe400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.843+0000 7f3fe3400640 1 -- 172.21.15.19:0/570879417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4014420 con 0x7f3fec0d4570 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.843+0000 7f3fe3400640 1 -- 172.21.15.19:0/570879417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe4019070 con 0x7f3fec0d4570 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.843+0000 7f3fe3400640 1 -- 172.21.15.19:0/570879417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4014800 con 0x7f3fec0d4570 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.843+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec0a01b0 con 0x7f3fec0d4570 2024-04-25T11:42:55.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.843+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec0a0490 con 0x7f3fec0d4570 2024-04-25T11:42:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.844+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fec050740 con 0x7f3fec0d4570 2024-04-25T11:42:55.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.846+0000 7f3fe3400640 1 -- 172.21.15.19:0/570879417 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fe40149a0 con 0x7f3fec0d4570 2024-04-25T11:42:55.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.846+0000 7f3fe3400640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fcc0744e0 0x7f3fcc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.847+0000 7f3fe3400640 1 -- 172.21.15.19:0/570879417 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f3fe4095a90 con 0x7f3fec0d4570 2024-04-25T11:42:55.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.847+0000 7f3feaa00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fcc0744e0 0x7f3fcc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:55.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.848+0000 7f3feaa00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fcc0744e0 0x7f3fcc0769f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc001630 tx=0x7f3fdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:55.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:55.851+0000 7f3fe3400640 1 -- 172.21.15.19:0/570879417 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fe405fad0 con 0x7f3fec0d4570 2024-04-25T11:42:55.971 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:42:55.985+0000 7fb164400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T11:42:56.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.100+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fec0d1220 con 0x7f3fec0d4570 2024-04-25T11:42:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.102+0000 7f3fe3400640 1 -- 172.21.15.19:0/570879417 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 371 v371) ==== 95+0+5195 (secure 0 0 0) 0x7f3fe4063dd0 con 0x7f3fec0d4570 2024-04-25T11:42:56.088 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:56.088 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":371,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":371,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:55.522938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:42:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 371 2024-04-25T11:42:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fcc0744e0 msgr2=0x7f3fcc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fcc0744e0 0x7f3fcc0769f0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc001630 tx=0x7f3fdc009040 comp rx=0 tx=0).stop 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0d4570 msgr2=0x7f3fec0c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0d4570 0x7f3fec0c7e90 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4007e90 tx=0x7f3fe400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 shutdown_connections 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3fcc0744e0 0x7f3fcc0769f0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fec0d4570 0x7f3fec0c7e90 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fec0b4830 0x7f3fec157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 --2- 172.21.15.19:0/570879417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fec0b2dc0 0x7f3fec0a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 >> 172.21.15.19:0/570879417 conn(0x7f3fec0d3a50 msgr2=0x7f3fec0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 shutdown_connections 2024-04-25T11:42:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.105+0000 7f3ff1e00640 1 -- 172.21.15.19:0/570879417 wait complete. 2024-04-25T11:42:56.117 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T11:42:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.424+0000 7f487ec00640 1 Processor -- start 2024-04-25T11:42:56.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.424+0000 7f487ec00640 1 -- start start 2024-04-25T11:42:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.426+0000 7f487ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.426+0000 7f487ec00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f48780b78e0 0x7f48780b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.426+0000 7f487ec00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48780b8230 con 0x7f48780d4190 2024-04-25T11:42:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.426+0000 7f487ec00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4878163c70 con 0x7f48780b78e0 2024-04-25T11:42:56.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.426+0000 7f487ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4878164230 con 0x7f48780b4f40 2024-04-25T11:42:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.427+0000 7f487d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.427+0000 7f487d800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:52324/0 (socket says 172.21.15.19:52324) 2024-04-25T11:42:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.427+0000 7f487e200640 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f48780d4190 0x7f48780d45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:42710/0 (socket says 172.21.15.19:42710) 2024-04-25T11:42:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.427+0000 7f487d800640 1 -- 172.21.15.19:0/1967877668 learned_addr learned my addr 172.21.15.19:0/1967877668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.427+0000 7f487ce00640 1 --2- 172.21.15.19:0/1967877668 >> v2:172.21.15.169:3300/0 conn(0x7f48780b78e0 0x7f48780b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.428+0000 7f487ce00640 1 -- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 msgr2=0x7f48781646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.428+0000 7f487ce00640 1 --2- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.428+0000 7f487ce00640 1 -- 172.21.15.19:0/1967877668 >> v1:172.21.15.169:6789/0 conn(0x7f48780d4190 legacy=0x7f48780d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.428+0000 7f487ce00640 1 -- 172.21.15.19:0/1967877668 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48780b5350 con 0x7f48780b78e0 2024-04-25T11:42:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.428+0000 7f487d800640 1 --2- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.429+0000 7f487ce00640 1 --2- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 0x7f48780b7cf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a2c90ec3f12228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.430+0000 7f4877e00640 1 -- 172.21.15.19:0/1967877668 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c007740 con 0x7f48780b78e0 2024-04-25T11:42:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.430+0000 7f4877e00640 1 -- 172.21.15.19:0/1967877668 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c0078e0 con 0x7f48780b78e0 2024-04-25T11:42:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.430+0000 7f4877e00640 1 -- 172.21.15.19:0/1967877668 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c007cb0 con 0x7f48780b78e0 2024-04-25T11:42:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 -- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 msgr2=0x7f48780b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 --2- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 0x7f48780b7cf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c02a660 comp rx=0 tx=0).stop 2024-04-25T11:42:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 -- 172.21.15.19:0/1967877668 shutdown_connections 2024-04-25T11:42:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 --2- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 0x7f48780b7cf0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 --2- 172.21.15.19:0/1967877668 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 -- 172.21.15.19:0/1967877668 >> 172.21.15.19:0/1967877668 conn(0x7f48780d2980 msgr2=0x7f48780d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 -- 172.21.15.19:0/1967877668 shutdown_connections 2024-04-25T11:42:56.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.431+0000 7f487ec00640 1 -- 172.21.15.19:0/1967877668 wait complete. 2024-04-25T11:42:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.434+0000 7f487ec00640 1 Processor -- start 2024-04-25T11:42:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.434+0000 7f487ec00640 1 -- start start 2024-04-25T11:42:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487ec00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 0x7f48781f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48780d4190 0x7f48780ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487ec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4878163ec0 con 0x7f48780d4190 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f48781613d0 con 0x7f48780b78e0 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487ec00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f48780d4cb0 con 0x7f48780b4f40 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48780d4190 0x7f48780ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48780d4190 0x7f48780ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48772/0 (socket says 172.21.15.19:48772) 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487e200640 1 -- 172.21.15.19:0/1178664659 learned_addr learned my addr 172.21.15.19:0/1178664659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487d800640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.436+0000 7f487ce00640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 0x7f48781f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487e200640 1 -- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 msgr2=0x7f48781e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487e200640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781e9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487e200640 1 -- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 msgr2=0x7f48781f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487e200640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 0x7f48781f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487e200640 1 -- 172.21.15.19:0/1178664659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48781ea600 con 0x7f48780d4190 2024-04-25T11:42:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487e200640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48780d4190 0x7f48780ce980 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f487000c3c0 tx=0x7f487000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f4875600640 1 -- 172.21.15.19:0/1178664659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48700074e0 con 0x7f48780d4190 2024-04-25T11:42:56.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f4875600640 1 -- 172.21.15.19:0/1178664659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4870007680 con 0x7f48780d4190 2024-04-25T11:42:56.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f4875600640 1 -- 172.21.15.19:0/1178664659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48700049e0 con 0x7f48780d4190 2024-04-25T11:42:56.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48781ea8e0 con 0x7f48780d4190 2024-04-25T11:42:56.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.437+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48781f6950 con 0x7f48780d4190 2024-04-25T11:42:56.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.440+0000 7f4875600640 1 -- 172.21.15.19:0/1178664659 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f487001a020 con 0x7f48780d4190 2024-04-25T11:42:56.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.442+0000 7f4875600640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f485c0744b0 0x7f485c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.442+0000 7f4875600640 1 -- 172.21.15.19:0/1178664659 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f4870094af0 con 0x7f48780d4190 2024-04-25T11:42:56.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.442+0000 7f48871a8740 1 Processor -- start 2024-04-25T11:42:56.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.442+0000 7f48871a8740 1 -- start start 2024-04-25T11:42:56.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.442+0000 7f487d800640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f485c0744b0 0x7f485c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.443+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f42ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.443+0000 7f48871a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f42e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.443+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f42c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f48871a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f4225bc0 con 0x55c9f3ef6490 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c9f41c1ed0 con 0x55c9f3f1f350 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c9f4221b80 con 0x55c9f3efe670 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487d800640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f485c0744b0 0x7f485c0769c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f48780d3f40 tx=0x7f4868008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f42e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f42e90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48786/0 (socket says 172.21.15.19:48786) 2024-04-25T11:42:56.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 -- 172.21.15.19:0/895154992 learned_addr learned my addr 172.21.15.19:0/895154992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487e200640 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f42c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487d800640 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f42ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 -- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 msgr2=0x55c9f42c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f42c23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 -- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 msgr2=0x55c9f42ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f42ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487ce00640 1 -- 172.21.15.19:0/895154992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f40037c0 con 0x55c9f3ef6490 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.444+0000 7f487e200640 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f42c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.445+0000 7f487ce00640 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f42e90f0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f486c02ac50 tx=0x7f486c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1074d4cf0aabc35e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.445+0000 7f4866000640 1 -- 172.21.15.19:0/895154992 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c007980 con 0x55c9f3ef6490 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.445+0000 7f4866000640 1 -- 172.21.15.19:0/895154992 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c007b20 con 0x55c9f3ef6490 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.445+0000 7f4866000640 1 -- 172.21.15.19:0/895154992 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c045480 con 0x55c9f3ef6490 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 -- 172.21.15.19:0/895154992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 msgr2=0x55c9f42e90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f42e90f0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f486c02ac50 tx=0x7f486c02f000 comp rx=0 tx=0).stop 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 -- 172.21.15.19:0/895154992 shutdown_connections 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f42c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f42e90f0 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 --2- 172.21.15.19:0/895154992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f42ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 -- 172.21.15.19:0/895154992 >> 172.21.15.19:0/895154992 conn(0x55c9f3f134a0 msgr2=0x55c9f42c1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 -- 172.21.15.19:0/895154992 shutdown_connections 2024-04-25T11:42:56.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.446+0000 7f48871a8740 1 -- 172.21.15.19:0/895154992 wait complete. 2024-04-25T11:42:56.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.447+0000 7f48871a8740 1 -- start start 2024-04-25T11:42:56.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.448+0000 7f48871a8740 1 Processor -- start 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.449+0000 7f48871a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.449+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3efe670 0x55c9f4316190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.449+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3f1f350 0x55c9f42ee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.449+0000 7f48871a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f41ac9f0 con 0x55c9f3ef6490 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.449+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c9f42dab90 con 0x55c9f3efe670 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.449+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c9f42fe5f0 con 0x55c9f3f1f350 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.449+0000 7f48871a8740 10 client.0 _ll_register_callbacks cb 0x55c9f41aaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48796/0 (socket says 172.21.15.19:48796) 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 -- 172.21.15.19:0/3984473674 learned_addr learned my addr 172.21.15.19:0/3984473674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487e200640 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3f1f350 0x55c9f42ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 -- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3f1f350 msgr2=0x55c9f42ee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487ce00640 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3efe670 0x55c9f4316190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3f1f350 0x55c9f42ee140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 -- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3efe670 msgr2=0x55c9f4316190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3efe670 0x55c9f4316190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.450+0000 7f487d800640 1 -- 172.21.15.19:0/3984473674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f4063f50 con 0x55c9f3ef6490 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f487d800640 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315c50 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f4868015750 tx=0x7f48680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8964b4e83fe5ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f485aa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f485aa00640 1 -- 172.21.15.19:0/3984473674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868026070 con 0x55c9f3ef6490 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f485aa00640 1 -- 172.21.15.19:0/3984473674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f48680147b0 con 0x55c9f3ef6490 2024-04-25T11:42:56.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486c007b20 con 0x55c9f3ef6490 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f485aa00640 1 -- 172.21.15.19:0/3984473674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=371) ==== 24+0+0 (secure 0 0 0) 0x7f4868031050 con 0x55c9f3ef6490 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f48871a8740 10 client.11739 fetch_fsmap learned FSMap version 371 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=371,monmap=2+}) -- 0x55c9f3ef70b0 con 0x55c9f3ef6490 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.451+0000 7f485aa00640 1 -- 172.21.15.19:0/3984473674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c007b20 con 0x55c9f3ef6490 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f485aa00640 1 -- 172.21.15.19:0/3984473674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 371) ==== 3494+0+0 (secure 0 0 0) 0x7f4868034020 con 0x55c9f3ef6490 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f48871a8740 10 client.11739 fetch_fsmap finished waiting for FSMap version 371 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f48871a8740 10 client.11739 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f48871a8740 20 client.11739 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f48871a8740 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f3ede130 0x55c9f42d9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f48871a8740 4 client.11739 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f42337d0 con 0x55c9f3ede130 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.452+0000 7f487ce00640 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f3ede130 0x55c9f42d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.453+0000 7f487ce00640 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f3ede130 0x55c9f42d9520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec78abd21ae8b82 server_cookie=7c1578d85b0d7f9b in_seq=0 out_seq=0 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.453+0000 7f485aa00640 10 client.11739 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.453+0000 7f485aa00640 1 -- 172.21.15.19:0/3984473674 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c9f42337d0 con 0x55c9f3ede130 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.453+0000 7f485aa00640 10 client.11739 handle_command_reply: tid=0 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 client.11739 shutdown 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 msgr2=0x55c9f4315c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315c50 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f4868015750 tx=0x7f48680046c0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 shutdown_connections 2024-04-25T11:42:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f3ede130 0x55c9f42d9520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3efe670 0x55c9f4316190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3f1f350 0x55c9f42ee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 --2- 172.21.15.19:0/3984473674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315c50 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f485aa00640 0 client.11739 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 >> 172.21.15.19:0/3984473674 conn(0x55c9f3f134a0 msgr2=0x55c9f42e7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 shutdown_connections 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 1 -- 172.21.15.19:0/3984473674 wait complete. 2024-04-25T11:42:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.454+0000 7f48871a8740 20 client.11739 trim_cache size 0 max 16384 2024-04-25T11:42:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.470+0000 7f48871a8740 1 Processor -- start 2024-04-25T11:42:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.470+0000 7f48871a8740 1 -- start start 2024-04-25T11:42:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.471+0000 7f48871a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.471+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f4316110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.471+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f4303e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f48871a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4878201d30 con 0x55c9f3ef6490 2024-04-25T11:42:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4878162a30 con 0x55c9f3f1f350 2024-04-25T11:42:56.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4878163080 con 0x55c9f3efe670 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48806/0 (socket says 172.21.15.19:48806) 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 -- 172.21.15.19:0/328852593 learned_addr learned my addr 172.21.15.19:0/328852593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487e200640 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f4303e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 -- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 msgr2=0x55c9f4316110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f4316110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 -- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 msgr2=0x55c9f4303e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f4303e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487d800640 1 -- 172.21.15.19:0/328852593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f42ede50 con 0x55c9f3ef6490 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.472+0000 7f487e200640 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f4303e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.473+0000 7f487d800640 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315bd0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f4868014490 tx=0x7f4868022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7eb4be111f3388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.473+0000 7f4865600640 1 -- 172.21.15.19:0/328852593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868026070 con 0x55c9f3ef6490 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.473+0000 7f4865600640 1 -- 172.21.15.19:0/328852593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486801a040 con 0x55c9f3ef6490 2024-04-25T11:42:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.473+0000 7f4865600640 1 -- 172.21.15.19:0/328852593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868014c20 con 0x55c9f3ef6490 2024-04-25T11:42:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 -- 172.21.15.19:0/328852593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 msgr2=0x55c9f4315bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315bd0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f4868014490 tx=0x7f4868022000 comp rx=0 tx=0).stop 2024-04-25T11:42:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 -- 172.21.15.19:0/328852593 shutdown_connections 2024-04-25T11:42:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3f1f350 0x55c9f4303e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f4316110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 --2- 172.21.15.19:0/328852593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3ef6490 0x55c9f4315bd0 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 -- 172.21.15.19:0/328852593 >> 172.21.15.19:0/328852593 conn(0x55c9f3f134a0 msgr2=0x55c9f42c0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 -- 172.21.15.19:0/328852593 shutdown_connections 2024-04-25T11:42:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.474+0000 7f48871a8740 1 -- 172.21.15.19:0/328852593 wait complete. 2024-04-25T11:42:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.475+0000 7f48871a8740 1 -- start start 2024-04-25T11:42:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.475+0000 7f48871a8740 1 Processor -- start 2024-04-25T11:42:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3ef6490 0x55c9f4303e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f48871a8740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f4304350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f48871a8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3f1f350 0x55c9f42bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f48871a8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f42d7e60 con 0x55c9f3f1f350 2024-04-25T11:42:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55c9f42d78a0 con 0x55c9f3ef6490 2024-04-25T11:42:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f48871a8740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x55c9f42fe3c0 con 0x55c9f3efe670 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f48871a8740 10 client.0 _ll_register_callbacks cb 0x7f4868011eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.477+0000 7f487e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3f1f350 0x55c9f42bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3f1f350 0x55c9f42bc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48822/0 (socket says 172.21.15.19:48822) 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 -- 172.21.15.19:0/908179397 learned_addr learned my addr 172.21.15.19:0/908179397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487d800640 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3ef6490 0x55c9f4303e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487ce00640 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f4304350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 -- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 msgr2=0x55c9f4304350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f4304350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 -- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3ef6490 msgr2=0x55c9f4303e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3ef6490 0x55c9f4303e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 -- 172.21.15.19:0/908179397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f42d6c50 con 0x55c9f3f1f350 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f487e200640 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3f1f350 0x55c9f42bc110 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f487005bcf0 tx=0x7f487005e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e88ddb15d3e0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.478+0000 7f485aa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f485aa00640 1 -- 172.21.15.19:0/908179397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487005eee0 con 0x55c9f3f1f350 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f485aa00640 1 -- 172.21.15.19:0/908179397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f4870067450 con 0x55c9f3f1f350 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486801a040 con 0x55c9f3f1f350 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f485aa00640 1 -- 172.21.15.19:0/908179397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=371) ==== 24+0+0 (secure 0 0 0) 0x7f4870010510 con 0x55c9f3f1f350 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 10 client.11745 fetch_fsmap learned FSMap version 371 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=371,monmap=2+}) -- 0x55c9f4320fc0 con 0x55c9f3f1f350 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f485aa00640 1 -- 172.21.15.19:0/908179397 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486801a040 con 0x55c9f3f1f350 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f485aa00640 1 -- 172.21.15.19:0/908179397 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 371) ==== 3494+0+0 (secure 0 0 0) 0x7f4870004b80 con 0x55c9f3f1f350 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 10 client.11745 fetch_fsmap finished waiting for FSMap version 371 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 10 client.11745 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 20 client.11745 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f42bc740 0x55c9f42d5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 4 client.11745 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.479+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55c9f422d640 con 0x55c9f42bc740 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.480+0000 7f487d800640 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f42bc740 0x55c9f42d5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.480+0000 7f487d800640 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f42bc740 0x55c9f42d5890 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db3126e38344cbf server_cookie=ec5286808ab26730 in_seq=0 out_seq=0 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.480+0000 7f485aa00640 10 client.11745 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f485aa00640 1 -- 172.21.15.19:0/908179397 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+294 (crc 0 0 0) 0x55c9f422d640 con 0x55c9f42bc740 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f485aa00640 10 client.11745 handle_command_reply: tid=0 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 client.11745 shutdown 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3f1f350 msgr2=0x55c9f42bc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3f1f350 0x55c9f42bc110 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f487005bcf0 tx=0x7f487005e590 comp rx=0 tx=0).stop 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 shutdown_connections 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x55c9f42bc740 0x55c9f42d5890 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f3f1f350 0x55c9f42bc110 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c9f3efe670 0x55c9f4304350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f485aa00640 0 client.11745 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 --2- 172.21.15.19:0/908179397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c9f3ef6490 0x55c9f4303e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 >> 172.21.15.19:0/908179397 conn(0x55c9f3f134a0 msgr2=0x55c9f4315fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 shutdown_connections 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 1 -- 172.21.15.19:0/908179397 wait complete. 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.481+0000 7f48871a8740 20 client.11745 trim_cache size 0 max 16384 2024-04-25T11:42:56.470 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3620364700, 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511628279, 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/testdir/hardlink2" 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "dir_frag", 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3909137653, 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511628278, 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "frag": "*", 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/dir2" 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T11:42:56.471 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T11:42:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f485c0744b0 msgr2=0x7f485c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f485c0744b0 0x7f485c0769c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f48780d3f40 tx=0x7f4868008040 comp rx=0 tx=0).stop 2024-04-25T11:42:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48780d4190 msgr2=0x7f48780ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48780d4190 0x7f48780ce980 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f487000c3c0 tx=0x7f487000c880 comp rx=0 tx=0).stop 2024-04-25T11:42:56.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 shutdown_connections 2024-04-25T11:42:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f485c0744b0 0x7f485c0769c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48780b78e0 0x7f48781f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48780d4190 0x7f48780ce980 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 --2- 172.21.15.19:0/1178664659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48780b4f40 0x7f48781e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 >> 172.21.15.19:0/1178664659 conn(0x7f48780d2980 msgr2=0x7f48780d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 shutdown_connections 2024-04-25T11:42:56.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.482+0000 7f487ec00640 1 -- 172.21.15.19:0/1178664659 wait complete. 2024-04-25T11:42:56.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.792+0000 7fc4ff200640 1 Processor -- start 2024-04-25T11:42:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.792+0000 7fc4ff200640 1 -- start start 2024-04-25T11:42:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc50006c5c0 0x7fc5000cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4ff200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc500144370 0x7fc5000b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4ff200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5000b50e0 con 0x7fc5000b4cd0 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4ff200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc5000d1230 con 0x7fc500144370 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4ff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000d06b0 con 0x7fc50006c5c0 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4fd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc500144370 0x7fc5000b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4fd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc500144370 0x7fc5000b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48834/0 (socket says 172.21.15.19:48834) 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.794+0000 7fc4fd400640 1 -- 172.21.15.19:0/3226151063 learned_addr learned my addr 172.21.15.19:0/3226151063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.795+0000 7fc4fde00640 1 --2- 172.21.15.19:0/3226151063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc50006c5c0 0x7fc5000cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.795+0000 7fc4fd400640 1 -- 172.21.15.19:0/3226151063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc50006c5c0 msgr2=0x7fc5000cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.795+0000 7fc4fd400640 1 --2- 172.21.15.19:0/3226151063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc50006c5c0 0x7fc5000cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.795+0000 7fc4fd400640 1 -- 172.21.15.19:0/3226151063 >> v1:172.21.15.169:6789/0 conn(0x7fc5000b4cd0 legacy=0x7fc500161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.795+0000 7fc4fd400640 1 -- 172.21.15.19:0/3226151063 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5000d0990 con 0x7fc500144370 2024-04-25T11:42:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.796+0000 7fc4fd400640 1 --2- 172.21.15.19:0/3226151063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc5000b4790 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009870 tx=0x7fc4ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86925c03cb9878f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4fca00640 1 -- 172.21.15.19:0/3226151063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec037070 con 0x7fc500144370 2024-04-25T11:42:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4fca00640 1 -- 172.21.15.19:0/3226151063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec032440 con 0x7fc500144370 2024-04-25T11:42:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4fca00640 1 -- 172.21.15.19:0/3226151063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec032810 con 0x7fc500144370 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4ff200640 1 -- 172.21.15.19:0/3226151063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 msgr2=0x7fc5000b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4ff200640 1 --2- 172.21.15.19:0/3226151063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc5000b4790 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009870 tx=0x7fc4ec02a210 comp rx=0 tx=0).stop 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4ff200640 1 -- 172.21.15.19:0/3226151063 shutdown_connections 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4ff200640 1 --2- 172.21.15.19:0/3226151063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc5000b4790 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4ff200640 1 --2- 172.21.15.19:0/3226151063 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc50006c5c0 0x7fc5000cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4ff200640 1 -- 172.21.15.19:0/3226151063 >> 172.21.15.19:0/3226151063 conn(0x7fc5000d1720 msgr2=0x7fc5000d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.797+0000 7fc4ff200640 1 -- 172.21.15.19:0/3226151063 shutdown_connections 2024-04-25T11:42:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.798+0000 7fc4ff200640 1 -- 172.21.15.19:0/3226151063 wait complete. 2024-04-25T11:42:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.800+0000 7fc4ff200640 1 Processor -- start 2024-04-25T11:42:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.800+0000 7fc4ff200640 1 -- start start 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.801+0000 7fc4ff200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc50006c5c0 0x7fc5000cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.801+0000 7fc4ff200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000b4cd0 0x7fc500158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.801+0000 7fc4ff200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc500070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4ff200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000d02d0 con 0x7fc500144370 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4ff200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc5000d0f50 con 0x7fc50006c5c0 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4ff200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc5000d0450 con 0x7fc5000b4cd0 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fe800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc500070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fe800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc500070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48846/0 (socket says 172.21.15.19:48846) 2024-04-25T11:42:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fe800640 1 -- 172.21.15.19:0/2450596621 learned_addr learned my addr 172.21.15.19:0/2450596621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fde00640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc50006c5c0 0x7fc5000cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fd400640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000b4cd0 0x7fc500158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fe800640 1 -- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000b4cd0 msgr2=0x7fc500158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fe800640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000b4cd0 0x7fc500158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fe800640 1 -- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc50006c5c0 msgr2=0x7fc5000cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.802+0000 7fc4fe800640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc50006c5c0 0x7fc5000cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.803+0000 7fc4fe800640 1 -- 172.21.15.19:0/2450596621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5000cb050 con 0x7fc500144370 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.803+0000 7fc4fe800640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc500070420 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b410 tx=0x7fc4f800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.803+0000 7fc4f6000640 1 -- 172.21.15.19:0/2450596621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8015030 con 0x7fc500144370 2024-04-25T11:42:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.803+0000 7fc4f6000640 1 -- 172.21.15.19:0/2450596621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f8004020 con 0x7fc500144370 2024-04-25T11:42:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.803+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5000cb560 con 0x7fc500144370 2024-04-25T11:42:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.803+0000 7fc4f6000640 1 -- 172.21.15.19:0/2450596621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8004410 con 0x7fc500144370 2024-04-25T11:42:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.803+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5000cbb20 con 0x7fc500144370 2024-04-25T11:42:56.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.804+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5000cb2d0 con 0x7fc500144370 2024-04-25T11:42:56.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.806+0000 7fc4f6000640 1 -- 172.21.15.19:0/2450596621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc4f8018070 con 0x7fc500144370 2024-04-25T11:42:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.807+0000 7fc4f6000640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4e00744e0 0x7fc4e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.807+0000 7fc4f6000640 1 -- 172.21.15.19:0/2450596621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fc4f8094e40 con 0x7fc500144370 2024-04-25T11:42:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.808+0000 7fc4fde00640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4e00744e0 0x7fc4e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.809+0000 7fc4fde00640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4e00744e0 0x7fc4e00769f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00099c0 tx=0x7fc4f0003990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:56.813+0000 7fc4f6000640 1 -- 172.21.15.19:0/2450596621 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4f805f4d0 con 0x7fc500144370 2024-04-25T11:42:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.056+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc50006cc20 con 0x7fc500144370 2024-04-25T11:42:57.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.057+0000 7fc4f6000640 1 -- 172.21.15.19:0/2450596621 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 371 v371) ==== 95+0+5195 (secure 0 0 0) 0x7fc4f805edd0 con 0x7fc500144370 2024-04-25T11:42:57.043 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:57.043 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":371,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":371,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:55.522938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:42:57.044 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 371 2024-04-25T11:42:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.059+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4e00744e0 msgr2=0x7fc4e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.059+0000 7fc4ff200640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4e00744e0 0x7fc4e00769f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00099c0 tx=0x7fc4f0003990 comp rx=0 tx=0).stop 2024-04-25T11:42:57.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.059+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 msgr2=0x7fc500070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.059+0000 7fc4ff200640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc500070420 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b410 tx=0x7fc4f800b8d0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 shutdown_connections 2024-04-25T11:42:57.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc4e00744e0 0x7fc4e00769f0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc500144370 0x7fc500070420 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5000b4cd0 0x7fc500158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 --2- 172.21.15.19:0/2450596621 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc50006c5c0 0x7fc5000cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 >> 172.21.15.19:0/2450596621 conn(0x7fc5000d1720 msgr2=0x7fc50006f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 shutdown_connections 2024-04-25T11:42:57.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.060+0000 7fc4ff200640 1 -- 172.21.15.19:0/2450596621 wait complete. 2024-04-25T11:42:57.074 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 3620364700 2024-04-25T11:42:57.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.373+0000 7f1cf5400640 1 Processor -- start 2024-04-25T11:42:57.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.373+0000 7f1cf5400640 1 -- start start 2024-04-25T11:42:57.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.375+0000 7f1cf5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf006dce0 0x7f1cf015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.375+0000 7f1cf5400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1cf00b0af0 0x7f1cf015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.375+0000 7f1cf5400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf00cdea0 con 0x7f1cf00cd9a0 2024-04-25T11:42:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.375+0000 7f1cf5400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf00ceab0 con 0x7f1cf00b0af0 2024-04-25T11:42:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.375+0000 7f1cf5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf00ce180 con 0x7f1cf006dce0 2024-04-25T11:42:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cee800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1cf00cd9a0 0x7f1cf006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55470/0 (socket says 172.21.15.19:55470) 2024-04-25T11:42:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cee800640 1 -- 172.21.15.19:0/306151309 learned_addr learned my addr 172.21.15.19:0/306151309 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:57.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cef200640 1 --2- 172.21.15.19:0/306151309 >> v2:172.21.15.19:3300/0 conn(0x7f1cf00b0af0 0x7f1cf015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cede00640 1 --2- 172.21.15.19:0/306151309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf006dce0 0x7f1cf015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1ced400640 1 -- 172.21.15.19:0/306151309 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 116399022 0 0) 0x7f1cf00cdea0 con 0x7f1cf00cd9a0 2024-04-25T11:42:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cef200640 1 -- 172.21.15.19:0/306151309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf006dce0 msgr2=0x7f1cf015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cef200640 1 --2- 172.21.15.19:0/306151309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf006dce0 0x7f1cf015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cef200640 1 -- 172.21.15.19:0/306151309 >> v1:172.21.15.19:6789/0 conn(0x7f1cf00cd9a0 legacy=0x7f1cf006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.376+0000 7f1cef200640 1 -- 172.21.15.19:0/306151309 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf00ce460 con 0x7f1cf00b0af0 2024-04-25T11:42:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.377+0000 7f1cef200640 1 --2- 172.21.15.19:0/306151309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf015faa0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009d90 tx=0x7f1ce802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55b03641f0e81d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.378+0000 7f1ced400640 1 -- 172.21.15.19:0/306151309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8007740 con 0x7f1cf00b0af0 2024-04-25T11:42:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.378+0000 7f1ced400640 1 -- 172.21.15.19:0/306151309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce80078e0 con 0x7f1cf00b0af0 2024-04-25T11:42:57.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.378+0000 7f1ced400640 1 -- 172.21.15.19:0/306151309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8007cb0 con 0x7f1cf00b0af0 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 -- 172.21.15.19:0/306151309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 msgr2=0x7f1cf015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 --2- 172.21.15.19:0/306151309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf015faa0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009d90 tx=0x7f1ce802a660 comp rx=0 tx=0).stop 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 -- 172.21.15.19:0/306151309 shutdown_connections 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 --2- 172.21.15.19:0/306151309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf015faa0 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 --2- 172.21.15.19:0/306151309 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf006dce0 0x7f1cf015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 -- 172.21.15.19:0/306151309 >> 172.21.15.19:0/306151309 conn(0x7f1cf0073090 msgr2=0x7f1cf00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 -- 172.21.15.19:0/306151309 shutdown_connections 2024-04-25T11:42:57.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.379+0000 7f1cf5400640 1 -- 172.21.15.19:0/306151309 wait complete. 2024-04-25T11:42:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.382+0000 7f1cf5400640 1 Processor -- start 2024-04-25T11:42:57.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.382+0000 7f1cf5400640 1 -- start start 2024-04-25T11:42:57.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.384+0000 7f1cf5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf0144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.384+0000 7f1cf5400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00cd9a0 0x7f1cf015d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.384+0000 7f1cf5400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf015d8f0 0x7f1cf0131390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.384+0000 7f1cf5400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00cef40 con 0x7f1cf00b0af0 2024-04-25T11:42:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.384+0000 7f1cf5400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00b0f00 con 0x7f1cf015d8f0 2024-04-25T11:42:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.384+0000 7f1cf5400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf00cf0c0 con 0x7f1cf00cd9a0 2024-04-25T11:42:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.384+0000 7f1cee800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf0144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf0144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48862/0 (socket says 172.21.15.19:48862) 2024-04-25T11:42:57.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 -- 172.21.15.19:0/3553026707 learned_addr learned my addr 172.21.15.19:0/3553026707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cef200640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf015d8f0 0x7f1cf0131390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cede00640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00cd9a0 0x7f1cf015d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 -- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00cd9a0 msgr2=0x7f1cf015d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00cd9a0 0x7f1cf015d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 -- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf015d8f0 msgr2=0x7f1cf0131390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf015d8f0 0x7f1cf0131390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 -- 172.21.15.19:0/3553026707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf01453e0 con 0x7f1cf00b0af0 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cede00640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00cd9a0 0x7f1cf015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.385+0000 7f1cee800640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf0144c80 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000c490 tx=0x7f1ce000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.386+0000 7f1ce6a00640 1 -- 172.21.15.19:0/3553026707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0018070 con 0x7f1cf00b0af0 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.386+0000 7f1ce6a00640 1 -- 172.21.15.19:0/3553026707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce0007500 con 0x7f1cf00b0af0 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.386+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf015dd00 con 0x7f1cf00b0af0 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.386+0000 7f1ce6a00640 1 -- 172.21.15.19:0/3553026707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0004a20 con 0x7f1cf00b0af0 2024-04-25T11:42:57.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.386+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf0146d40 con 0x7f1cf00b0af0 2024-04-25T11:42:57.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.389+0000 7f1ce6a00640 1 -- 172.21.15.19:0/3553026707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ce001b070 con 0x7f1cf00b0af0 2024-04-25T11:42:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.390+0000 7f1ce6a00640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ccc0744e0 0x7f1ccc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.390+0000 7f1cfc43a740 1 Processor -- start 2024-04-25T11:42:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.390+0000 7f1ce6a00640 1 -- 172.21.15.19:0/3553026707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f1ce0094580 con 0x7f1cf00b0af0 2024-04-25T11:42:57.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.390+0000 7f1cfc43a740 1 -- start start 2024-04-25T11:42:57.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.390+0000 7f1cede00640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ccc0744e0 0x7f1ccc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cede00640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ccc0744e0 0x7f1ccc0769f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8006fd0 tx=0x7f1cd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501a7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c5010d900 con 0x556c4fde65c0 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556c500aa1f0 con 0x556c4fdde3e0 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556c501098c0 con 0x556c4fe072a0 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cef200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cef200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48864/0 (socket says 172.21.15.19:48864) 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cef200640 1 -- 172.21.15.19:0/1544304896 learned_addr learned my addr 172.21.15.19:0/1544304896 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cede00640 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.392+0000 7f1cee800640 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cef200640 1 -- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 msgr2=0x556c501d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cef200640 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cef200640 1 -- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 msgr2=0x556c501a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cef200640 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cef200640 1 -- 172.21.15.19:0/1544304896 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4feeb710 con 0x556c4fde65c0 2024-04-25T11:42:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cef200640 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501a7720 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009eb0 tx=0x7f1ce802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c535db2d90d140b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cdf000640 1 -- 172.21.15.19:0/1544304896 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8007970 con 0x556c4fde65c0 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cdf000640 1 -- 172.21.15.19:0/1544304896 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce8007b10 con 0x556c4fde65c0 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.393+0000 7f1cdf000640 1 -- 172.21.15.19:0/1544304896 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8045430 con 0x556c4fde65c0 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 -- 172.21.15.19:0/1544304896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 msgr2=0x556c501a7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501a7720 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009eb0 tx=0x7f1ce802f000 comp rx=0 tx=0).stop 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 -- 172.21.15.19:0/1544304896 shutdown_connections 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501a7720 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/1544304896 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 -- 172.21.15.19:0/1544304896 >> 172.21.15.19:0/1544304896 conn(0x556c4fdfb3f0 msgr2=0x556c501edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 -- 172.21.15.19:0/1544304896 shutdown_connections 2024-04-25T11:42:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.394+0000 7f1cfc43a740 1 -- 172.21.15.19:0/1544304896 wait complete. 2024-04-25T11:42:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.395+0000 7f1cfc43a740 1 -- start start 2024-04-25T11:42:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.396+0000 7f1cfc43a740 1 Processor -- start 2024-04-25T11:42:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.397+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.397+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fde65c0 0x556c50200e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.397+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c50201360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.397+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c50095040 con 0x556c4fe072a0 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.397+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556c501cef60 con 0x556c4fdde3e0 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.397+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556c501e9a70 con 0x556c4fde65c0 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.397+0000 7f1cfc43a740 10 client.0 _ll_register_callbacks cb 0x556c500c5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c50201360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c50201360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48876/0 (socket says 172.21.15.19:48876) 2024-04-25T11:42:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 -- 172.21.15.19:0/424748085 learned_addr learned my addr 172.21.15.19:0/424748085 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cede00640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fde65c0 0x556c50200e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cee800640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 -- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fde65c0 msgr2=0x556c50200e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fde65c0 0x556c50200e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 -- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 msgr2=0x556c501ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 -- 172.21.15.19:0/424748085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4ff4bea0 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cee800640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cef200640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c50201360 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009eb0 tx=0x7f1ce8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4521aa8a39c25a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.398+0000 7f1cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cd7400640 1 -- 172.21.15.19:0/424748085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8007e40 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cd7400640 1 -- 172.21.15.19:0/424748085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f1ce8003990 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce8007b10 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cd7400640 1 -- 172.21.15.19:0/424748085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=371) ==== 24+0+0 (secure 0 0 0) 0x7f1ce8055050 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 10 client.11766 fetch_fsmap learned FSMap version 371 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=371,monmap=2+}) -- 0x556c4fddf000 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cd7400640 1 -- 172.21.15.19:0/424748085 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce8007b10 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cd7400640 1 -- 172.21.15.19:0/424748085 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 371) ==== 3494+0+0 (secure 0 0 0) 0x7f1ce8058020 con 0x556c4fe072a0 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 10 client.11766 fetch_fsmap finished waiting for FSMap version 371 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 10 client.11766 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 20 client.11766 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c4fdc6080 0x556c501ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 4 client.11766 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:42:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.399+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5011b510 con 0x556c4fdc6080 2024-04-25T11:42:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.400+0000 7f1cee800640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c4fdc6080 0x556c501ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.400+0000 7f1cee800640 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c4fdc6080 0x556c501ee3c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7ad948aeb649aa server_cookie=a0b111928f2fbdde in_seq=0 out_seq=0 2024-04-25T11:42:57.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.400+0000 7f1cd7400640 10 client.11766 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cd7400640 1 -- 172.21.15.19:0/424748085 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c5011b510 con 0x556c4fdc6080 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cd7400640 10 client.11766 handle_command_reply: tid=0 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 client.11766 shutdown 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 msgr2=0x556c50201360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c50201360 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009eb0 tx=0x7f1ce8004ab0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 shutdown_connections 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c4fdc6080 0x556c501ee3c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c50201360 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fde65c0 0x556c50200e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/424748085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 >> 172.21.15.19:0/424748085 conn(0x556c4fdfb3f0 msgr2=0x556c501a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.401+0000 7f1cd7400640 0 client.11766 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.402+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 shutdown_connections 2024-04-25T11:42:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.402+0000 7f1cfc43a740 1 -- 172.21.15.19:0/424748085 wait complete. 2024-04-25T11:42:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.402+0000 7f1cfc43a740 20 client.11766 trim_cache size 0 max 16384 2024-04-25T11:42:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.417+0000 7f1cfc43a740 1 Processor -- start 2024-04-25T11:42:57.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.417+0000 7f1cfc43a740 1 -- start start 2024-04-25T11:42:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0201e00 con 0x556c4fde65c0 2024-04-25T11:42:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0141270 con 0x556c4fdde3e0 2024-04-25T11:42:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf01418c0 con 0x556c4fe072a0 2024-04-25T11:42:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cede00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cede00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501ebb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48882/0 (socket says 172.21.15.19:48882) 2024-04-25T11:42:57.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.419+0000 7f1cede00640 1 -- 172.21.15.19:0/2280831943 learned_addr learned my addr 172.21.15.19:0/2280831943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cef200640 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cee800640 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cede00640 1 -- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 msgr2=0x556c501f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cede00640 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501f0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cede00640 1 -- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 msgr2=0x556c501aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cede00640 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cede00640 1 -- 172.21.15.19:0/2280831943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c501f5160 con 0x556c4fde65c0 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cef200640 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cee800640 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cede00640 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501ebb30 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00b3380 tx=0x7f1cd8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78113159813ad3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cde600640 1 -- 172.21.15.19:0/2280831943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd801e040 con 0x556c4fde65c0 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cde600640 1 -- 172.21.15.19:0/2280831943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd8027070 con 0x556c4fde65c0 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.420+0000 7f1cde600640 1 -- 172.21.15.19:0/2280831943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800f040 con 0x556c4fde65c0 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 -- 172.21.15.19:0/2280831943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 msgr2=0x556c501ebb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501ebb30 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00b3380 tx=0x7f1cd8014000 comp rx=0 tx=0).stop 2024-04-25T11:42:57.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 -- 172.21.15.19:0/2280831943 shutdown_connections 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fe072a0 0x556c501f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fde65c0 0x556c501ebb30 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/2280831943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 -- 172.21.15.19:0/2280831943 >> 172.21.15.19:0/2280831943 conn(0x556c4fdfb3f0 msgr2=0x556c501ef6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 -- 172.21.15.19:0/2280831943 shutdown_connections 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.421+0000 7f1cfc43a740 1 -- 172.21.15.19:0/2280831943 wait complete. 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.422+0000 7f1cfc43a740 1 -- start start 2024-04-25T11:42:57.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.423+0000 7f1cfc43a740 1 Processor -- start 2024-04-25T11:42:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fde65c0 0x556c501d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cfc43a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c501d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556c501ec540 con 0x556c4fe072a0 2024-04-25T11:42:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x556c501cfa50 con 0x556c4fde65c0 2024-04-25T11:42:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cfc43a740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x556c501ecb90 con 0x556c4fdde3e0 2024-04-25T11:42:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cfc43a740 10 client.0 _ll_register_callbacks cb 0x7f1ce8007540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.424+0000 7f1cef200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c501d6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c501d6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48898/0 (socket says 172.21.15.19:48898) 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 -- 172.21.15.19:0/4072536734 learned_addr learned my addr 172.21.15.19:0/4072536734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cede00640 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fde65c0 0x556c501d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cee800640 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 -- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fdde3e0 msgr2=0x556c501aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 -- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fde65c0 msgr2=0x556c501d6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fde65c0 0x556c501d6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 -- 172.21.15.19:0/4072536734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c501d0cb0 con 0x556c4fe072a0 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cef200640 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c501d6c80 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009e80 tx=0x7f1ce8033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b36210f7092d752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.425+0000 7f1cd7400640 1 -- 172.21.15.19:0/4072536734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce805d030 con 0x556c4fe072a0 2024-04-25T11:42:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cd7400640 1 -- 172.21.15.19:0/4072536734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f1ce804a070 con 0x556c4fe072a0 2024-04-25T11:42:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd8027070 con 0x556c4fe072a0 2024-04-25T11:42:57.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cd7400640 1 -- 172.21.15.19:0/4072536734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=371) ==== 24+0+0 (secure 0 0 0) 0x7f1ce8007d00 con 0x556c4fe072a0 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 10 client.11772 fetch_fsmap learned FSMap version 371 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=371,monmap=2+}) -- 0x556c502042e0 con 0x556c4fe072a0 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cd7400640 1 -- 172.21.15.19:0/4072536734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd8027070 con 0x556c4fe072a0 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cd7400640 1 -- 172.21.15.19:0/4072536734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 371) ==== 3494+0+0 (secure 0 0 0) 0x7f1ce8058040 con 0x556c4fe072a0 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 10 client.11772 fetch_fsmap finished waiting for FSMap version 371 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 10 client.11772 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 20 client.11772 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c501cda20 0x556c501fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 4 client.11772 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3620364700}] 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.426+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3620364700}) -- 0x556c50115380 con 0x556c501cda20 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.427+0000 7f1cee800640 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c501cda20 0x556c501fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.427+0000 7f1cee800640 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c501cda20 0x556c501fbf80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d466e70a0e1365a server_cookie=69e9e9e6d0504503 in_seq=0 out_seq=0 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.427+0000 7f1cd7400640 10 client.11772 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:57.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.427+0000 7f1cd7400640 1 -- 172.21.15.19:0/4072536734 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x556c50115380 con 0x556c501cda20 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.427+0000 7f1cd7400640 10 client.11772 handle_command_reply: tid=0 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.427+0000 7f1cfc43a740 1 client.11772 shutdown 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 msgr2=0x556c501d6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c501d6c80 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009e80 tx=0x7f1ce8033040 comp rx=0 tx=0).stop 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 shutdown_connections 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x556c501cda20 0x556c501fbf80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556c4fde65c0 0x556c501d6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556c4fe072a0 0x556c501d6c80 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 --2- 172.21.15.19:0/4072536734 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556c4fdde3e0 0x556c501aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cd7400640 0 client.11772 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:57.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 >> 172.21.15.19:0/4072536734 conn(0x556c4fdfb3f0 msgr2=0x556c501a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 shutdown_connections 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 1 -- 172.21.15.19:0/4072536734 wait complete. 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.428+0000 7f1cfc43a740 20 client.11772 trim_cache size 0 max 16384 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ccc0744e0 msgr2=0x7f1ccc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ccc0744e0 0x7f1ccc0769f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8006fd0 tx=0x7f1cd8008040 comp rx=0 tx=0).stop 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 msgr2=0x7f1cf0144c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf0144c80 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000c490 tx=0x7f1ce000c950 comp rx=0 tx=0).stop 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 shutdown_connections 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1ccc0744e0 0x7f1ccc0769f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cf015d8f0 0x7f1cf0131390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cf00cd9a0 0x7f1cf015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 --2- 172.21.15.19:0/3553026707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cf00b0af0 0x7f1cf0144c80 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 >> 172.21.15.19:0/3553026707 conn(0x7f1cf0073090 msgr2=0x7f1cf00b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 shutdown_connections 2024-04-25T11:42:57.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.429+0000 7f1cf5400640 1 -- 172.21.15.19:0/3553026707 wait complete. 2024-04-25T11:42:57.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:42:57.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.733+0000 7f17f1000640 1 Processor -- start 2024-04-25T11:42:57.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.733+0000 7f17f1000640 1 -- start start 2024-04-25T11:42:57.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.735+0000 7f17f1000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f17ec06c5c0 0x7f17ec0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.736+0000 7f17f1000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17ec0d1230 con 0x7f17ec0b4ea0 2024-04-25T11:42:57.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.736+0000 7f17f1000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17ec0d06b0 con 0x7f17ec144370 2024-04-25T11:42:57.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.736+0000 7f17f1000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f17ec0d0990 con 0x7f17ec06c5c0 2024-04-25T11:42:57.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.736+0000 7f17ebe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f17ec0b4ea0 0x7f17ec15e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55474/0 (socket says 172.21.15.19:55474) 2024-04-25T11:42:57.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.736+0000 7f17ebe00640 1 -- 172.21.15.19:0/3328742325 learned_addr learned my addr 172.21.15.19:0/3328742325 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:57.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.736+0000 7f17eb400640 1 --2- 172.21.15.19:0/3328742325 >> v2:172.21.15.169:3300/0 conn(0x7f17ec06c5c0 0x7f17ec0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3959906005 0 0) 0x7f17ec0d1230 con 0x7f17ec0b4ea0 2024-04-25T11:42:57.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17d0003520 con 0x7f17ec0b4ea0 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3709981428 0 0) 0x7f17d0003520 con 0x7f17ec0b4ea0 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 >> v2:172.21.15.169:3300/0 conn(0x7f17ec06c5c0 msgr2=0x7f17ec0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 --2- 172.21.15.19:0/3328742325 >> v2:172.21.15.169:3300/0 conn(0x7f17ec06c5c0 0x7f17ec0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 >> v1:172.21.15.169:6789/0 conn(0x7f17ec144370 legacy=0x7f17ec0b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec0d0c70 con 0x7f17ec0b4ea0 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f17e40019f0 con 0x7f17ec0b4ea0 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 >> v1:172.21.15.19:6789/0 conn(0x7f17ec0b4ea0 legacy=0x7f17ec15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00039b0 0x7f17d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d0004800 0x7f17d0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d00051c0 0x7f17d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec0d0990 con 0x7f17d00051c0 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17d000c160 con 0x7f17d0004800 2024-04-25T11:42:57.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.737+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f17d000c360 con 0x7f17d00039b0 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17ebe00640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d00051c0 0x7f17d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17eb400640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00039b0 0x7f17d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17eaa00640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d0004800 0x7f17d0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17ebe00640 1 -- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00039b0 msgr2=0x7f17d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17ebe00640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00039b0 0x7f17d0003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17ebe00640 1 -- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d0004800 msgr2=0x7f17d0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17ebe00640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d0004800 0x7f17d0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17ebe00640 1 -- 172.21.15.19:0/3328742325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d0003520 con 0x7f17d00051c0 2024-04-25T11:42:57.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.738+0000 7f17eaa00640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17d0004800 0x7f17d0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:57.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.739+0000 7f17ebe00640 1 -- 172.21.15.19:0/3328742325 reap_dead start 2024-04-25T11:42:57.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.739+0000 7f17ebe00640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d00051c0 0x7f17d000b9c0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f17e400a860 tx=0x7f17d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ca3411f31ab3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e400a480 con 0x7f17d00051c0 2024-04-25T11:42:57.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e402f070 con 0x7f17d00051c0 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17ea000640 1 -- 172.21.15.19:0/3328742325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e402bbe0 con 0x7f17d00051c0 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17f1000640 1 -- 172.21.15.19:0/3328742325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d00051c0 msgr2=0x7f17d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17f1000640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d00051c0 0x7f17d000b9c0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f17e400a860 tx=0x7f17d0005830 comp rx=0 tx=0).stop 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17f1000640 1 -- 172.21.15.19:0/3328742325 shutdown_connections 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17f1000640 1 --2- 172.21.15.19:0/3328742325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d00051c0 0x7f17d000b9c0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17f1000640 1 -- 172.21.15.19:0/3328742325 >> 172.21.15.19:0/3328742325 conn(0x7f17ec0d1720 msgr2=0x7f17ec0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17f1000640 1 -- 172.21.15.19:0/3328742325 shutdown_connections 2024-04-25T11:42:57.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.740+0000 7f17f1000640 1 -- 172.21.15.19:0/3328742325 wait complete. 2024-04-25T11:42:57.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.743+0000 7f17f1000640 1 Processor -- start 2024-04-25T11:42:57.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.743+0000 7f17f1000640 1 -- start start 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.744+0000 7f17f1000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00051c0 0x7f17ec159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.744+0000 7f17f1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17ec160960 0x7f17ec132bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17f1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17ec133100 0x7f17ec1336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17f1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec0d0450 con 0x7f17ec160960 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17f1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec0d02d0 con 0x7f17ec133100 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17f1000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f17ec147760 con 0x7f17d00051c0 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17ec160960 0x7f17ec132bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17ec160960 0x7f17ec132bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48926/0 (socket says 172.21.15.19:48926) 2024-04-25T11:42:57.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 -- 172.21.15.19:0/1776460610 learned_addr learned my addr 172.21.15.19:0/1776460610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eb400640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00051c0 0x7f17ec159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17ebe00640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17ec133100 0x7f17ec1336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 -- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00051c0 msgr2=0x7f17ec159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00051c0 0x7f17ec159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 -- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17ec133100 msgr2=0x7f17ec1336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17ec133100 0x7f17ec1336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eaa00640 1 -- 172.21.15.19:0/1776460610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec1610d0 con 0x7f17ec160960 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17eb400640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00051c0 0x7f17ec159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.745+0000 7f17ebe00640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17ec133100 0x7f17ec1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.746+0000 7f17eaa00640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17ec160960 0x7f17ec132bc0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f17d400d770 tx=0x7f17d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.746+0000 7f17e3400640 1 -- 172.21.15.19:0/1776460610 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4005270 con 0x7f17ec160960 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.746+0000 7f17e3400640 1 -- 172.21.15.19:0/1776460610 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d4017070 con 0x7f17ec160960 2024-04-25T11:42:57.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.746+0000 7f17e3400640 1 -- 172.21.15.19:0/1776460610 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d400b6b0 con 0x7f17ec160960 2024-04-25T11:42:57.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.746+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ec1613b0 con 0x7f17ec160960 2024-04-25T11:42:57.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.746+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ec161970 con 0x7f17ec160960 2024-04-25T11:42:57.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.747+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17ec160e40 con 0x7f17ec160960 2024-04-25T11:42:57.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.749+0000 7f17e3400640 1 -- 172.21.15.19:0/1776460610 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f17d400b850 con 0x7f17ec160960 2024-04-25T11:42:57.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.749+0000 7f17e3400640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17c80744b0 0x7f17c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:57.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.750+0000 7f17e3400640 1 -- 172.21.15.19:0/1776460610 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f17d4095030 con 0x7f17ec160960 2024-04-25T11:42:57.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.750+0000 7f17eb400640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17c80744b0 0x7f17c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:57.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.752+0000 7f17eb400640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17c80744b0 0x7f17c80769c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0099e0 tx=0x7f17dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:57.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:57.755+0000 7f17e3400640 1 -- 172.21.15.19:0/1776460610 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17d405f880 con 0x7f17ec160960 2024-04-25T11:42:57.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.000+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17ec06ccb0 con 0x7f17ec160960 2024-04-25T11:42:57.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.001+0000 7f17e3400640 1 -- 172.21.15.19:0/1776460610 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 371 v371) ==== 95+0+5195 (secure 0 0 0) 0x7f17d4001230 con 0x7f17ec160960 2024-04-25T11:42:57.988 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:57.988 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":371,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":371,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:55.522938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":11,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:42:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 371 2024-04-25T11:42:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17c80744b0 msgr2=0x7f17c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17c80744b0 0x7f17c80769c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0099e0 tx=0x7f17dc008040 comp rx=0 tx=0).stop 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17ec160960 msgr2=0x7f17ec132bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17ec160960 0x7f17ec132bc0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f17d400d770 tx=0x7f17d400dc30 comp rx=0 tx=0).stop 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 shutdown_connections 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f17c80744b0 0x7f17c80769c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17ec133100 0x7f17ec1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17ec160960 0x7f17ec132bc0 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 --2- 172.21.15.19:0/1776460610 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17d00051c0 0x7f17ec159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 >> 172.21.15.19:0/1776460610 conn(0x7f17ec0d1720 msgr2=0x7f17ec15e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 shutdown_connections 2024-04-25T11:42:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.004+0000 7f17f1000640 1 -- 172.21.15.19:0/1776460610 wait complete. 2024-04-25T11:42:58.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 3909137653 2024-04-25T11:42:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.310+0000 7fb423e00640 1 Processor -- start 2024-04-25T11:42:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.310+0000 7fb423e00640 1 -- start start 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb423e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb424144340 0x7fb42406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb423e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb42415ee60 0x7fb42415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb423e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb42406df20 con 0x7fb42406db10 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb423e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb42415f270 con 0x7fb424144340 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb423e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb424144750 con 0x7fb42415ee60 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb422a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb424144340 0x7fb42406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb422a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb424144340 0x7fb42406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48940/0 (socket says 172.21.15.19:48940) 2024-04-25T11:42:58.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.313+0000 7fb422a00640 1 -- 172.21.15.19:0/2282491932 learned_addr learned my addr 172.21.15.19:0/2282491932 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:58.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.314+0000 7fb422a00640 1 -- 172.21.15.19:0/2282491932 >> v2:172.21.15.169:3300/0 conn(0x7fb42415ee60 msgr2=0x7fb42415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.314+0000 7fb422a00640 1 --2- 172.21.15.19:0/2282491932 >> v2:172.21.15.169:3300/0 conn(0x7fb42415ee60 0x7fb42415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.314+0000 7fb422a00640 1 -- 172.21.15.19:0/2282491932 >> v1:172.21.15.169:6789/0 conn(0x7fb42406db10 legacy=0x7fb42415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.314+0000 7fb422a00640 1 -- 172.21.15.19:0/2282491932 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb424070000 con 0x7fb424144340 2024-04-25T11:42:58.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.315+0000 7fb422a00640 1 --2- 172.21.15.19:0/2282491932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424144340 0x7fb42406d5d0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7fb4240d1790 tx=0x7fb41402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc039b70d1bafd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.315+0000 7fb421600640 1 -- 172.21.15.19:0/2282491932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414037070 con 0x7fb424144340 2024-04-25T11:42:58.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.315+0000 7fb421600640 1 -- 172.21.15.19:0/2282491932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414032440 con 0x7fb424144340 2024-04-25T11:42:58.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.315+0000 7fb421600640 1 -- 172.21.15.19:0/2282491932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414032810 con 0x7fb424144340 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 -- 172.21.15.19:0/2282491932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424144340 msgr2=0x7fb42406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 --2- 172.21.15.19:0/2282491932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424144340 0x7fb42406d5d0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7fb4240d1790 tx=0x7fb41402a230 comp rx=0 tx=0).stop 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 -- 172.21.15.19:0/2282491932 shutdown_connections 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 --2- 172.21.15.19:0/2282491932 >> v2:172.21.15.169:3300/0 conn(0x7fb42415ee60 0x7fb42415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 --2- 172.21.15.19:0/2282491932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb424144340 0x7fb42406d5d0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 -- 172.21.15.19:0/2282491932 >> 172.21.15.19:0/2282491932 conn(0x7fb424072f40 msgr2=0x7fb424073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 -- 172.21.15.19:0/2282491932 shutdown_connections 2024-04-25T11:42:58.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.316+0000 7fb423e00640 1 -- 172.21.15.19:0/2282491932 wait complete. 2024-04-25T11:42:58.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.319+0000 7fb423e00640 1 Processor -- start 2024-04-25T11:42:58.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.320+0000 7fb423e00640 1 -- start start 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.321+0000 7fb423e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb42406db10 0x7fb4240afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.321+0000 7fb423e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb424144340 0x7fb4241f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb42415ee60 0x7fb424134d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb42406f800 con 0x7fb42415ee60 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb42406f520 con 0x7fb42406db10 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb42406fae0 con 0x7fb424144340 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb42415ee60 0x7fb424134d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb42415ee60 0x7fb424134d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48946/0 (socket says 172.21.15.19:48946) 2024-04-25T11:42:58.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 -- 172.21.15.19:0/1441677223 learned_addr learned my addr 172.21.15.19:0/1441677223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb422000640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb424144340 0x7fb4241f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb422a00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb42406db10 0x7fb4240afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 -- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb424144340 msgr2=0x7fb4241f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb424144340 0x7fb4241f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 -- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb42406db10 msgr2=0x7fb4240afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb42406db10 0x7fb4240afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.322+0000 7fb423400640 1 -- 172.21.15.19:0/1441677223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb424162040 con 0x7fb42415ee60 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.323+0000 7fb423400640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb42415ee60 0x7fb424134d40 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b9e0 tx=0x7fb41c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.323+0000 7fb41aa00640 1 -- 172.21.15.19:0/1441677223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c019070 con 0x7fb42415ee60 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.323+0000 7fb41aa00640 1 -- 172.21.15.19:0/1441677223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41c00cb40 con 0x7fb42415ee60 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.323+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4240af0f0 con 0x7fb42415ee60 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.323+0000 7fb41aa00640 1 -- 172.21.15.19:0/1441677223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c014440 con 0x7fb42415ee60 2024-04-25T11:42:58.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.323+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4240af6b0 con 0x7fb42415ee60 2024-04-25T11:42:58.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.326+0000 7fb41aa00640 1 -- 172.21.15.19:0/1441677223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb41c01c070 con 0x7fb42415ee60 2024-04-25T11:42:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.327+0000 7fb41aa00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4000744e0 0x7fb4000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.327+0000 7fb41aa00640 1 -- 172.21.15.19:0/1441677223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fb41c094fa0 con 0x7fb42415ee60 2024-04-25T11:42:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.327+0000 7fb430c22740 1 Processor -- start 2024-04-25T11:42:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.327+0000 7fb422a00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4000744e0 0x7fb4000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.327+0000 7fb430c22740 1 -- start start 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb422a00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4000744e0 0x7fb4000769f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb414034000 tx=0x7fb41402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b7b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb430c22740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7b4f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b4f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb430c22740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5642e7ab5900 con 0x5642e77863e0 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5642e7a521f0 con 0x5642e778e5c0 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5642e7ab18c0 con 0x5642e77af2a0 2024-04-25T11:42:58.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb422000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7b4f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb422000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7b4f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48962/0 (socket says 172.21.15.19:48962) 2024-04-25T11:42:58.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb422000640 1 -- 172.21.15.19:0/1547910500 learned_addr learned my addr 172.21.15.19:0/1547910500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.329+0000 7fb422a00640 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b7b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb422000640 1 -- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 msgr2=0x5642e7b7b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb423400640 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b4f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb422000640 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b7b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb422000640 1 -- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 msgr2=0x5642e7b4f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb422000640 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b4f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb422000640 1 -- 172.21.15.19:0/1547910500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e7893710 con 0x5642e77863e0 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb422a00640 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b7b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb423400640 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b4f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb422000640 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7b4f3d0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00b760 tx=0x7fb40c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b5e272f17711b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb413400640 1 -- 172.21.15.19:0/1547910500 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00d950 con 0x5642e77863e0 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb413400640 1 -- 172.21.15.19:0/1547910500 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c019070 con 0x5642e77863e0 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.330+0000 7fb413400640 1 -- 172.21.15.19:0/1547910500 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00dcf0 con 0x5642e77863e0 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 -- 172.21.15.19:0/1547910500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 msgr2=0x5642e7b4f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7b4f3d0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00b760 tx=0x7fb40c00bc20 comp rx=0 tx=0).stop 2024-04-25T11:42:58.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 -- 172.21.15.19:0/1547910500 shutdown_connections 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b4f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7b4f3d0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 --2- 172.21.15.19:0/1547910500 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b7b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 -- 172.21.15.19:0/1547910500 >> 172.21.15.19:0/1547910500 conn(0x5642e77a33f0 msgr2=0x5642e7b96100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 -- 172.21.15.19:0/1547910500 shutdown_connections 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.331+0000 7fb430c22740 1 -- 172.21.15.19:0/1547910500 wait complete. 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.332+0000 7fb430c22740 1 -- start start 2024-04-25T11:42:58.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.333+0000 7fb430c22740 1 Processor -- start 2024-04-25T11:42:58.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.334+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77863e0 0x5642e7b93100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.334+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b94aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.334+0000 7fb430c22740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77af2a0 0x5642e7b6d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.334+0000 7fb430c22740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5642e7a3d040 con 0x5642e77af2a0 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.334+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5642e7b6dc00 con 0x5642e778e5c0 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.334+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5642e7b937b0 con 0x5642e77863e0 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.334+0000 7fb430c22740 10 client.0 _ll_register_callbacks cb 0x5642e7a6d220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77af2a0 0x5642e7b6d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77af2a0 0x5642e7b6d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48970/0 (socket says 172.21.15.19:48970) 2024-04-25T11:42:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 -- 172.21.15.19:0/3276033031 learned_addr learned my addr 172.21.15.19:0/3276033031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 -- 172.21.15.19:0/3276033031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77863e0 msgr2=0x5642e7b93100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77863e0 0x5642e7b93100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 -- 172.21.15.19:0/3276033031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 msgr2=0x5642e7b94aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b94aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 -- 172.21.15.19:0/3276033031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e78f3ea0 con 0x5642e77af2a0 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb423400640 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77af2a0 0x5642e7b6d0a0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7fb41c05c230 tx=0x7fb41c05f590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb52a3956f9b970d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb407e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:58.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.335+0000 7fb407e00640 1 -- 172.21.15.19:0/3276033031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c05efc0 con 0x5642e77af2a0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb407e00640 1 -- 172.21.15.19:0/3276033031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fb41c08dc80 con 0x5642e77af2a0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40c019070 con 0x5642e77af2a0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb407e00640 1 -- 172.21.15.19:0/3276033031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=371) ==== 24+0+0 (secure 0 0 0) 0x7fb41c063370 con 0x5642e77af2a0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 10 client.11790 fetch_fsmap learned FSMap version 371 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=371,monmap=2+}) -- 0x5642e7787000 con 0x5642e77af2a0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb407e00640 1 -- 172.21.15.19:0/3276033031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c019070 con 0x5642e77af2a0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb407e00640 1 -- 172.21.15.19:0/3276033031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 371) ==== 3494+0+0 (secure 0 0 0) 0x7fb41c061070 con 0x5642e77af2a0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 10 client.11790 fetch_fsmap finished waiting for FSMap version 371 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 10 client.11790 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 20 client.11790 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e776e080 0x5642e7b90d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 4 client.11790 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.336+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e7ac3510 con 0x5642e776e080 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.337+0000 7fb422a00640 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e776e080 0x5642e7b90d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.337+0000 7fb422a00640 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e776e080 0x5642e7b90d20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1465e14607ba46cf server_cookie=96063d5e0f373c9a in_seq=0 out_seq=0 2024-04-25T11:42:58.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.337+0000 7fb407e00640 10 client.11790 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb407e00640 1 -- 172.21.15.19:0/3276033031 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642e7ac3510 con 0x5642e776e080 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb407e00640 10 client.11790 handle_command_reply: tid=0 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 client.11790 shutdown 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77af2a0 msgr2=0x5642e7b6d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77af2a0 0x5642e7b6d0a0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7fb41c05c230 tx=0x7fb41c05f590 comp rx=0 tx=0).stop 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 shutdown_connections 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e776e080 0x5642e7b90d20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e778e5c0 0x5642e7b94aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77af2a0 0x5642e7b6d0a0 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb430c22740 1 --2- 172.21.15.19:0/3276033031 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77863e0 0x5642e7b93100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.338+0000 7fb407e00640 0 client.11790 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:58.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.339+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 >> 172.21.15.19:0/3276033031 conn(0x5642e77a33f0 msgr2=0x5642e7b7ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.339+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 shutdown_connections 2024-04-25T11:42:58.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.339+0000 7fb430c22740 1 -- 172.21.15.19:0/3276033031 wait complete. 2024-04-25T11:42:58.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.339+0000 7fb430c22740 20 client.11790 trim_cache size 0 max 16384 2024-04-25T11:42:58.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.354+0000 7fb430c22740 1 Processor -- start 2024-04-25T11:42:58.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.354+0000 7fb430c22740 1 -- start start 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77863e0 0x5642e7b46ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb430c22740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e778e5c0 0x5642e7b6b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b6b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb430c22740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb424201eb0 con 0x5642e778e5c0 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb42415b7f0 con 0x5642e77863e0 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb4241fa7d0 con 0x5642e77af2a0 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb422000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e778e5c0 0x5642e7b6b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb422000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e778e5c0 0x5642e7b6b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48984/0 (socket says 172.21.15.19:48984) 2024-04-25T11:42:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.356+0000 7fb422000640 1 -- 172.21.15.19:0/857094300 learned_addr learned my addr 172.21.15.19:0/857094300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb423400640 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b6b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422000640 1 -- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 msgr2=0x5642e7b6b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422000640 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b6b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422a00640 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77863e0 0x5642e7b46ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422000640 1 -- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77863e0 msgr2=0x5642e7b46ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422000640 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77863e0 0x5642e7b46ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422000640 1 -- 172.21.15.19:0/857094300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e7b6bcf0 con 0x5642e778e5c0 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb423400640 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b6b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422a00640 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77863e0 0x5642e7b46ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb422000640 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e778e5c0 0x5642e7b6b270 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fb40c008010 tx=0x7fb40c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a590c4dcf265ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb412a00640 1 -- 172.21.15.19:0/857094300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c004220 con 0x5642e778e5c0 2024-04-25T11:42:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb412a00640 1 -- 172.21.15.19:0/857094300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c019040 con 0x5642e778e5c0 2024-04-25T11:42:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.357+0000 7fb412a00640 1 -- 172.21.15.19:0/857094300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00d950 con 0x5642e778e5c0 2024-04-25T11:42:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 -- 172.21.15.19:0/857094300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e778e5c0 msgr2=0x5642e7b6b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e778e5c0 0x5642e7b6b270 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fb40c008010 tx=0x7fb40c010000 comp rx=0 tx=0).stop 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 -- 172.21.15.19:0/857094300 shutdown_connections 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e77af2a0 0x5642e7b6b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e778e5c0 0x5642e7b6b270 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 --2- 172.21.15.19:0/857094300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77863e0 0x5642e7b46ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 -- 172.21.15.19:0/857094300 >> 172.21.15.19:0/857094300 conn(0x5642e77a33f0 msgr2=0x5642e7b94860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 -- 172.21.15.19:0/857094300 shutdown_connections 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.358+0000 7fb430c22740 1 -- 172.21.15.19:0/857094300 wait complete. 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.359+0000 7fb430c22740 1 -- start start 2024-04-25T11:42:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.360+0000 7fb430c22740 1 Processor -- start 2024-04-25T11:42:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.361+0000 7fb430c22740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7c16020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.361+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e778e5c0 0x5642e7b76c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.361+0000 7fb430c22740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77af2a0 0x5642e7b6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.361+0000 7fb430c22740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5642e7b6d680 con 0x5642e77863e0 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.361+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5642e7b94fe0 con 0x5642e77af2a0 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.361+0000 7fb430c22740 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x5642e7b7fdd0 con 0x5642e778e5c0 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.361+0000 7fb430c22740 10 client.0 _ll_register_callbacks cb 0x7fb41c05b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7c16020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7c16020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48994/0 (socket says 172.21.15.19:48994) 2024-04-25T11:42:58.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 -- 172.21.15.19:0/3036089224 learned_addr learned my addr 172.21.15.19:0/3036089224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 -- 172.21.15.19:0/3036089224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e778e5c0 msgr2=0x5642e7b76c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e778e5c0 0x5642e7b76c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 -- 172.21.15.19:0/3036089224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77af2a0 msgr2=0x5642e7b6ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77af2a0 0x5642e7b6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 -- 172.21.15.19:0/3036089224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e7b4d930 con 0x5642e77863e0 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb422a00640 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7c16020 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7fb414009960 tx=0x7fb41402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738c68fbc0510b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb407e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T11:42:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.362+0000 7fb407e00640 1 -- 172.21.15.19:0/3036089224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414053060 con 0x5642e77863e0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb407e00640 1 -- 172.21.15.19:0/3036089224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fb41403d700 con 0x5642e77863e0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40c019040 con 0x5642e77863e0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb407e00640 1 -- 172.21.15.19:0/3036089224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=371) ==== 24+0+0 (secure 0 0 0) 0x7fb414059050 con 0x5642e77863e0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 10 client.11796 fetch_fsmap learned FSMap version 371 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=371,monmap=2+}) -- 0x5642e7b766d0 con 0x5642e77863e0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb407e00640 1 -- 172.21.15.19:0/3036089224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c019040 con 0x5642e77863e0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb407e00640 1 -- 172.21.15.19:0/3036089224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 371) ==== 3494+0+0 (secure 0 0 0) 0x7fb41405c020 con 0x5642e77863e0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 10 client.11796 fetch_fsmap finished waiting for FSMap version 371 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 10 client.11796 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 20 client.11796 populate_metadata read hostname 'smithi019' 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e7b6b3d0 0x5642e7c165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 4 client.11796 mds_command: new command op to 8020 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3909137653}] 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.363+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 --> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3909137653}) -- 0x5642e7abd380 con 0x5642e7b6b3d0 2024-04-25T11:42:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.364+0000 7fb422000640 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e7b6b3d0 0x5642e7c165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.364+0000 7fb422000640 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e7b6b3d0 0x5642e7c165e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d61e75ad7c3aa5 server_cookie=be95ff2c58f2590c in_seq=0 out_seq=0 2024-04-25T11:42:58.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.364+0000 7fb407e00640 10 client.11796 ms_handle_connect on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.364+0000 7fb407e00640 1 -- 172.21.15.19:0/3036089224 <== mds.0 v2:172.21.15.19:6832/3539191189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x5642e7abd380 con 0x5642e7b6b3d0 2024-04-25T11:42:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.364+0000 7fb407e00640 10 client.11796 handle_command_reply: tid=0 2024-04-25T11:42:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.364+0000 7fb430c22740 1 client.11796 shutdown 2024-04-25T11:42:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 msgr2=0x5642e7c16020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7c16020 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7fb414009960 tx=0x7fb41402c070 comp rx=0 tx=0).stop 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 shutdown_connections 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:6832/3539191189,v1:172.21.15.19:6834/3539191189] conn(0x5642e7b6b3d0 0x5642e7c165e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642e77af2a0 0x5642e7b6ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642e778e5c0 0x5642e7b76c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 --2- 172.21.15.19:0/3036089224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642e77863e0 0x5642e7c16020 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb407e00640 0 client.11796 ms_handle_reset on v2:172.21.15.19:6832/3539191189 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 >> 172.21.15.19:0/3036089224 conn(0x5642e77a33f0 msgr2=0x5642e7b4b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 shutdown_connections 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 1 -- 172.21.15.19:0/3036089224 wait complete. 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb430c22740 20 client.11796 trim_cache size 0 max 16384 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4000744e0 msgr2=0x7fb4000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb423e00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4000744e0 0x7fb4000769f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb414034000 tx=0x7fb41402e000 comp rx=0 tx=0).stop 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb42415ee60 msgr2=0x7fb424134d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.365+0000 7fb423e00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb42415ee60 0x7fb424134d40 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b9e0 tx=0x7fb41c00bea0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 shutdown_connections 2024-04-25T11:42:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb4000744e0 0x7fb4000769f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb42415ee60 0x7fb424134d40 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb424144340 0x7fb4241f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 --2- 172.21.15.19:0/1441677223 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb42406db10 0x7fb4240afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 >> 172.21.15.19:0/1441677223 conn(0x7fb424072f40 msgr2=0x7fb4240b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 shutdown_connections 2024-04-25T11:42:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:58.366+0000 7fb423e00640 1 -- 172.21.15.19:0/1441677223 wait complete. 2024-04-25T11:42:58.376 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:42:58.377 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:42:58.477 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:42:58.592 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:42:58.592 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:42:58.592 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:58.592 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:58.653 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:42:58.653 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:42:58.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:42:58.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.005+0000 7f2ec2400640 1 Processor -- start 2024-04-25T11:42:58.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.005+0000 7f2ec2400640 1 -- start start 2024-04-25T11:42:58.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.007+0000 7f2ec2400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ebc06e680 0x7f2ebc0b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.007+0000 7f2ec2400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2ebc08cd60 0x7f2ebc08d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.007+0000 7f2ec2400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ebc0b5290 con 0x7f2ebc0b4e80 2024-04-25T11:42:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.007+0000 7f2ec2400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2ebc0d1d70 con 0x7f2ebc06e680 2024-04-25T11:42:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.007+0000 7f2ec2400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f2ebc0ce6d0 con 0x7f2ebc08cd60 2024-04-25T11:42:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ebc06e680 0x7f2ebc0b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ebc06e680 0x7f2ebc0b90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49006/0 (socket says 172.21.15.19:49006) 2024-04-25T11:42:58.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1000640 1 -- 172.21.15.19:0/775871904 learned_addr learned my addr 172.21.15.19:0/775871904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1a00640 1 --2- 172.21.15.19:0/775871904 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc08cd60 0x7f2ebc08d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1000640 1 -- 172.21.15.19:0/775871904 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc08cd60 msgr2=0x7f2ebc08d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1000640 1 --2- 172.21.15.19:0/775871904 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc08cd60 0x7f2ebc08d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1000640 1 -- 172.21.15.19:0/775871904 >> v1:172.21.15.169:6789/0 conn(0x7f2ebc0b4e80 legacy=0x7f2ebc0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1000640 1 -- 172.21.15.19:0/775871904 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc08d6b0 con 0x7f2ebc06e680 2024-04-25T11:42:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.008+0000 7f2ec1a00640 1 --2- 172.21.15.19:0/775871904 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc08cd60 0x7f2ebc08d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:58.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.009+0000 7f2ec1000640 1 --2- 172.21.15.19:0/775871904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc06e680 0x7f2ebc0b90e0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc074030 tx=0x7f2eac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f650596050fa0630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.010+0000 7f2ebb400640 1 -- 172.21.15.19:0/775871904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac004030 con 0x7f2ebc06e680 2024-04-25T11:42:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.010+0000 7f2ebb400640 1 -- 172.21.15.19:0/775871904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac0041d0 con 0x7f2ebc06e680 2024-04-25T11:42:58.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.010+0000 7f2ebb400640 1 -- 172.21.15.19:0/775871904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac003610 con 0x7f2ebc06e680 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.010+0000 7f2ec2400640 1 -- 172.21.15.19:0/775871904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc06e680 msgr2=0x7f2ebc0b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.011+0000 7f2ec2400640 1 --2- 172.21.15.19:0/775871904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc06e680 0x7f2ebc0b90e0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc074030 tx=0x7f2eac02a180 comp rx=0 tx=0).stop 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.011+0000 7f2ec2400640 1 -- 172.21.15.19:0/775871904 shutdown_connections 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.011+0000 7f2ec2400640 1 --2- 172.21.15.19:0/775871904 >> v2:172.21.15.169:3300/0 conn(0x7f2ebc08cd60 0x7f2ebc08d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.011+0000 7f2ec2400640 1 --2- 172.21.15.19:0/775871904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc06e680 0x7f2ebc0b90e0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.011+0000 7f2ec2400640 1 -- 172.21.15.19:0/775871904 >> 172.21.15.19:0/775871904 conn(0x7f2ebc0b5460 msgr2=0x7f2ebc0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.011+0000 7f2ec2400640 1 -- 172.21.15.19:0/775871904 shutdown_connections 2024-04-25T11:42:58.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.011+0000 7f2ec2400640 1 -- 172.21.15.19:0/775871904 wait complete. 2024-04-25T11:42:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.013+0000 7f2ec2400640 1 Processor -- start 2024-04-25T11:42:58.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.013+0000 7f2ec2400640 1 -- start start 2024-04-25T11:42:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ec2400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc06e680 0x7f2ebc161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ec2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc08cd60 0x7f2ebc0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ec2400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0b4e80 0x7f2ebc151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ec2400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc0d2560 con 0x7f2ebc08cd60 2024-04-25T11:42:59.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ec2400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc0d2280 con 0x7f2ebc0b4e80 2024-04-25T11:42:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ec2400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2ebc0d2840 con 0x7f2ebc06e680 2024-04-25T11:42:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ebbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc08cd60 0x7f2ebc0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ebbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc08cd60 0x7f2ebc0caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49014/0 (socket says 172.21.15.19:49014) 2024-04-25T11:42:59.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.015+0000 7f2ebbe00640 1 -- 172.21.15.19:0/3682411845 learned_addr learned my addr 172.21.15.19:0/3682411845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ec1a00640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0b4e80 0x7f2ebc151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ec1000640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc06e680 0x7f2ebc161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ebbe00640 1 -- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc06e680 msgr2=0x7f2ebc161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ebbe00640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc06e680 0x7f2ebc161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ebbe00640 1 -- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0b4e80 msgr2=0x7f2ebc151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ebbe00640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0b4e80 0x7f2ebc151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ebbe00640 1 -- 172.21.15.19:0/3682411845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc0caf60 con 0x7f2ebc08cd60 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ec1a00640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0b4e80 0x7f2ebc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2ebbe00640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc08cd60 0x7f2ebc0caa20 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009fd0 tx=0x7f2ea400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.016+0000 7f2eb8c00640 1 -- 172.21.15.19:0/3682411845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4015020 con 0x7f2ebc08cd60 2024-04-25T11:42:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.017+0000 7f2eb8c00640 1 -- 172.21.15.19:0/3682411845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea4011830 con 0x7f2ebc08cd60 2024-04-25T11:42:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.017+0000 7f2eb8c00640 1 -- 172.21.15.19:0/3682411845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4011c00 con 0x7f2ebc08cd60 2024-04-25T11:42:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.017+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc0c8b40 con 0x7f2ebc08cd60 2024-04-25T11:42:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.017+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc0c9100 con 0x7f2ebc08cd60 2024-04-25T11:42:59.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.018+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ebc050740 con 0x7f2ebc08cd60 2024-04-25T11:42:59.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.020+0000 7f2eb8c00640 1 -- 172.21.15.19:0/3682411845 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ea4018020 con 0x7f2ebc08cd60 2024-04-25T11:42:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.021+0000 7f2eb8c00640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e980744b0 0x7f2e980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.021+0000 7f2eb8c00640 1 -- 172.21.15.19:0/3682411845 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f2ea4095100 con 0x7f2ebc08cd60 2024-04-25T11:42:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.022+0000 7f2ec1000640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e980744b0 0x7f2e980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.023+0000 7f2ec1000640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e980744b0 0x7f2e980769c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2eac033000 tx=0x7f2eac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.027+0000 7f2eb8c00640 1 -- 172.21.15.19:0/3682411845 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ea401f020 con 0x7f2ebc08cd60 2024-04-25T11:42:59.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.237+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ebc0c88d0 con 0x7f2ebc08cd60 2024-04-25T11:42:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.239+0000 7f2eb8c00640 1 -- 172.21.15.19:0/3682411845 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16750 (secure 0 0 0) 0x7f2ea405f8c0 con 0x7f2ebc08cd60 2024-04-25T11:42:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:42:59.226 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":144,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:44.471720+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/2610059988":"2024-04-26T11:42:44.471606+0000","172.21.15.169:6834/2610059988":"2024-04-26T11:42:44.471606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e980744b0 msgr2=0x7f2e980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e980744b0 0x7f2e980769c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2eac033000 tx=0x7f2eac02e000 comp rx=0 tx=0).stop 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc08cd60 msgr2=0x7f2ebc0caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc08cd60 0x7f2ebc0caa20 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009fd0 tx=0x7f2ea400b870 comp rx=0 tx=0).stop 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 shutdown_connections 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2e980744b0 0x7f2e980769c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ebc0b4e80 0x7f2ebc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ebc08cd60 0x7f2ebc0caa20 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 --2- 172.21.15.19:0/3682411845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ebc06e680 0x7f2ebc161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.242+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 >> 172.21.15.19:0/3682411845 conn(0x7f2ebc0b5460 msgr2=0x7f2ebc15bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:59.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.243+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 shutdown_connections 2024-04-25T11:42:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.243+0000 7f2ec2400640 1 -- 172.21.15.19:0/3682411845 wait complete. 2024-04-25T11:42:59.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:42:27.399161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:42:28.185279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:42:59.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:42:59.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.559+0000 7fba1a000640 1 Processor -- start 2024-04-25T11:42:59.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.559+0000 7fba1a000640 1 -- start start 2024-04-25T11:42:59.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.561+0000 7fba1a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 0x7fba1406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.561+0000 7fba1a000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fba140cffc0 0x7fba140d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba1a000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba140d2730 con 0x7fba140b7800 2024-04-25T11:42:59.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba1a000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fba140b3900 con 0x7fba140cffc0 2024-04-25T11:42:59.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba1a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba140d29f0 con 0x7fba1415c8e0 2024-04-25T11:42:59.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba18c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fba140b7800 0x7fba1406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55486/0 (socket says 172.21.15.19:55486) 2024-04-25T11:42:59.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba18c00640 1 -- 172.21.15.19:0/3034728338 learned_addr learned my addr 172.21.15.19:0/3034728338 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:42:59.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba19600640 1 --2- 172.21.15.19:0/3034728338 >> v2:172.21.15.169:3300/0 conn(0x7fba140cffc0 0x7fba140d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba0be00640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 0x7fba1406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.562+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1569363887 0 0) 0x7fba140d2730 con 0x7fba140b7800 2024-04-25T11:42:59.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9f8003520 con 0x7fba140b7800 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3604733089 0 0) 0x7fb9f8003520 con 0x7fba140b7800 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 msgr2=0x7fba1406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 0x7fba1406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 >> v2:172.21.15.169:3300/0 conn(0x7fba140cffc0 msgr2=0x7fba140d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 --2- 172.21.15.19:0/3034728338 >> v2:172.21.15.169:3300/0 conn(0x7fba140cffc0 0x7fba140d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba140d2cd0 con 0x7fba140b7800 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fba04001670 con 0x7fba140b7800 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 >> v1:172.21.15.19:6789/0 conn(0x7fba140b7800 legacy=0x7fba1406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f80039b0 0x7fb9f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 0x7fb9f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9f8004d40 0x7fb9f8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba140b3900 con 0x7fb9f8004d40 2024-04-25T11:42:59.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba140d29f0 con 0x7fb9f80039b0 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f800b9f0 con 0x7fba1415c8e0 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.563+0000 7fba19600640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9f8004d40 0x7fb9f8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.564+0000 7fba18c00640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f80039b0 0x7fb9f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.564+0000 7fba0be00640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 0x7fb9f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.564+0000 7fba19600640 1 -- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 msgr2=0x7fb9f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.564+0000 7fba19600640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1415c8e0 0x7fb9f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.564+0000 7fba19600640 1 -- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f80039b0 msgr2=0x7fb9f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.564+0000 7fba19600640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f80039b0 0x7fb9f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.564+0000 7fba19600640 1 -- 172.21.15.19:0/3034728338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f8003520 con 0x7fb9f8004d40 2024-04-25T11:42:59.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.565+0000 7fba19600640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9f8004d40 0x7fb9f8005150 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fba140cfa70 tx=0x7fb9f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc8e358e86ab01e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:59.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.565+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c009c10 con 0x7fb9f8004d40 2024-04-25T11:42:59.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.565+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0c036070 con 0x7fb9f8004d40 2024-04-25T11:42:59.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.565+0000 7fba13e00640 1 -- 172.21.15.19:0/3034728338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c031670 con 0x7fb9f8004d40 2024-04-25T11:42:59.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.566+0000 7fba1a000640 1 -- 172.21.15.19:0/3034728338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9f8004d40 msgr2=0x7fb9f8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.566+0000 7fba1a000640 1 --2- 172.21.15.19:0/3034728338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb9f8004d40 0x7fb9f8005150 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fba140cfa70 tx=0x7fb9f8005830 comp rx=0 tx=0).stop 2024-04-25T11:42:59.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.566+0000 7fba19600640 1 -- 172.21.15.19:0/3034728338 reap_dead start 2024-04-25T11:42:59.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.566+0000 7fba1a000640 1 -- 172.21.15.19:0/3034728338 shutdown_connections 2024-04-25T11:42:59.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.566+0000 7fba1a000640 1 -- 172.21.15.19:0/3034728338 >> 172.21.15.19:0/3034728338 conn(0x7fba1409b7d0 msgr2=0x7fba140b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:59.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.566+0000 7fba1a000640 1 -- 172.21.15.19:0/3034728338 shutdown_connections 2024-04-25T11:42:59.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.567+0000 7fba1a000640 1 -- 172.21.15.19:0/3034728338 wait complete. 2024-04-25T11:42:59.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.569+0000 7fba1a000640 1 Processor -- start 2024-04-25T11:42:59.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.569+0000 7fba1a000640 1 -- start start 2024-04-25T11:42:59.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba1a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f8004d40 0x7fba1415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba1a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba140b18a0 0x7fba14149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba1a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1414fd60 0x7fba14150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba1a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba14075e80 con 0x7fba140b18a0 2024-04-25T11:42:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba1a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fba140b36c0 con 0x7fb9f8004d40 2024-04-25T11:42:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba1a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fba14076000 con 0x7fba1414fd60 2024-04-25T11:42:59.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba0be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba140b18a0 0x7fba14149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba0be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba140b18a0 0x7fba14149fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49026/0 (socket says 172.21.15.19:49026) 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.571+0000 7fba0be00640 1 -- 172.21.15.19:0/877049342 learned_addr learned my addr 172.21.15.19:0/877049342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba18c00640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f8004d40 0x7fba1415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba19600640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1414fd60 0x7fba14150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba0be00640 1 -- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1414fd60 msgr2=0x7fba14150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba0be00640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1414fd60 0x7fba14150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba0be00640 1 -- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f8004d40 msgr2=0x7fba1415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba0be00640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f8004d40 0x7fba1415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba0be00640 1 -- 172.21.15.19:0/877049342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba141506b0 con 0x7fba140b18a0 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba19600640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1414fd60 0x7fba14150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.572+0000 7fba0be00640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba140b18a0 0x7fba14149fc0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc011600 tx=0x7fb9fc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.573+0000 7fba11600640 1 -- 172.21.15.19:0/877049342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00f8c0 con 0x7fba140b18a0 2024-04-25T11:42:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.573+0000 7fba11600640 1 -- 172.21.15.19:0/877049342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00fa60 con 0x7fba140b18a0 2024-04-25T11:42:59.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.573+0000 7fba11600640 1 -- 172.21.15.19:0/877049342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00fe10 con 0x7fba140b18a0 2024-04-25T11:42:59.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.573+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba140afa70 con 0x7fba140b18a0 2024-04-25T11:42:59.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.573+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba140afd30 con 0x7fba140b18a0 2024-04-25T11:42:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.574+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba14076c60 con 0x7fba140b18a0 2024-04-25T11:42:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.575+0000 7fba11600640 1 -- 172.21.15.19:0/877049342 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb9fc015460 con 0x7fba140b18a0 2024-04-25T11:42:59.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.576+0000 7fba11600640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb9f00744b0 0x7fb9f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:59.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.576+0000 7fba11600640 1 -- 172.21.15.19:0/877049342 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fb9fc095c30 con 0x7fba140b18a0 2024-04-25T11:42:59.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.577+0000 7fba18c00640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb9f00744b0 0x7fb9f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:59.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.578+0000 7fba18c00640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb9f00744b0 0x7fb9f00769c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fba04001050 tx=0x7fba04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:59.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.582+0000 7fba11600640 1 -- 172.21.15.19:0/877049342 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba141506b0 con 0x7fba140b18a0 2024-04-25T11:42:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.786+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba140b1f10 con 0x7fba140b18a0 2024-04-25T11:42:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.786+0000 7fba11600640 1 -- 172.21.15.19:0/877049342 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7fb9fc019030 con 0x7fba140b18a0 2024-04-25T11:42:59.773 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:42:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb9f00744b0 msgr2=0x7fb9f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb9f00744b0 0x7fb9f00769c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fba04001050 tx=0x7fba04009040 comp rx=0 tx=0).stop 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba140b18a0 msgr2=0x7fba14149fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba140b18a0 0x7fba14149fc0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc011600 tx=0x7fb9fc011ac0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 shutdown_connections 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb9f00744b0 0x7fb9f00769c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fba1414fd60 0x7fba14150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba140b18a0 0x7fba14149fc0 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 --2- 172.21.15.19:0/877049342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb9f8004d40 0x7fba1415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 >> 172.21.15.19:0/877049342 conn(0x7fba1409b7d0 msgr2=0x7fba1415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.789+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 shutdown_connections 2024-04-25T11:42:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:42:59.790+0000 7fba1a000640 1 -- 172.21.15.19:0/877049342 wait complete. 2024-04-25T11:42:59.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:43:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.112+0000 7fe2f5c00640 1 Processor -- start 2024-04-25T11:43:00.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.112+0000 7fe2f5c00640 1 -- start start 2024-04-25T11:43:00.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2f5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2f0144340 0x7fe2f006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2f5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f006db10 0x7fe2f015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2f5c00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f006df20 con 0x7fe2f015ec90 2024-04-25T11:43:00.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2f5c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f0144750 con 0x7fe2f0144340 2024-04-25T11:43:00.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2f5c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f0070000 con 0x7fe2f006db10 2024-04-25T11:43:00.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2f0144340 0x7fe2f006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2f0144340 0x7fe2f006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49036/0 (socket says 172.21.15.19:49036) 2024-04-25T11:43:00.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.115+0000 7fe2efe00640 1 -- 172.21.15.19:0/808759515 learned_addr learned my addr 172.21.15.19:0/808759515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.116+0000 7fe2e7400640 1 --2- 172.21.15.19:0/808759515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f006db10 0x7fe2f015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.116+0000 7fe2efe00640 1 -- 172.21.15.19:0/808759515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f006db10 msgr2=0x7fe2f015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.116+0000 7fe2efe00640 1 --2- 172.21.15.19:0/808759515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f006db10 0x7fe2f015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.116+0000 7fe2efe00640 1 -- 172.21.15.19:0/808759515 >> v1:172.21.15.169:6789/0 conn(0x7fe2f015ec90 legacy=0x7fe2f0161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:00.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.116+0000 7fe2efe00640 1 -- 172.21.15.19:0/808759515 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f006f240 con 0x7fe2f0144340 2024-04-25T11:43:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.117+0000 7fe2efe00640 1 --2- 172.21.15.19:0/808759515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0144340 0x7fe2f006d5d0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009e50 tx=0x7fe2e002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b12f663279197b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.117+0000 7fe2ef400640 1 -- 172.21.15.19:0/808759515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0036070 con 0x7fe2f0144340 2024-04-25T11:43:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2ef400640 1 -- 172.21.15.19:0/808759515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0031600 con 0x7fe2f0144340 2024-04-25T11:43:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2ef400640 1 -- 172.21.15.19:0/808759515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e00319d0 con 0x7fe2f0144340 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2f5c00640 1 -- 172.21.15.19:0/808759515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0144340 msgr2=0x7fe2f006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/808759515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0144340 0x7fe2f006d5d0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009e50 tx=0x7fe2e002a6c0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2f5c00640 1 -- 172.21.15.19:0/808759515 shutdown_connections 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/808759515 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f006db10 0x7fe2f015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/808759515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f0144340 0x7fe2f006d5d0 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.118+0000 7fe2f5c00640 1 -- 172.21.15.19:0/808759515 >> 172.21.15.19:0/808759515 conn(0x7fe2f0072f40 msgr2=0x7fe2f0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.119+0000 7fe2f5c00640 1 -- 172.21.15.19:0/808759515 shutdown_connections 2024-04-25T11:43:00.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.119+0000 7fe2f5c00640 1 -- 172.21.15.19:0/808759515 wait complete. 2024-04-25T11:43:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.121+0000 7fe2f5c00640 1 Processor -- start 2024-04-25T11:43:00.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.121+0000 7fe2f5c00640 1 -- start start 2024-04-25T11:43:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.122+0000 7fe2f5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f006db10 0x7fe2f015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2f5c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0144340 0x7fe2f01f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2f5c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f015ec90 0x7fe2f00afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2f5c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f006fae0 con 0x7fe2f006db10 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2f5c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f006f800 con 0x7fe2f0144340 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2f5c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f015f0a0 con 0x7fe2f015ec90 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f006db10 0x7fe2f015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f006db10 0x7fe2f015aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49046/0 (socket says 172.21.15.19:49046) 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 -- 172.21.15.19:0/331289061 learned_addr learned my addr 172.21.15.19:0/331289061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2f5200640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f015ec90 0x7fe2f00afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2e7400640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0144340 0x7fe2f01f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 -- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f015ec90 msgr2=0x7fe2f00afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f015ec90 0x7fe2f00afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 -- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0144340 msgr2=0x7fe2f01f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0144340 0x7fe2f01f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.123+0000 7fe2efe00640 1 -- 172.21.15.19:0/331289061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f01fa8c0 con 0x7fe2f006db10 2024-04-25T11:43:00.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.124+0000 7fe2efe00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f006db10 0x7fe2f015aad0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0034040 tx=0x7fe2e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:00.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.124+0000 7fe2ecc00640 1 -- 172.21.15.19:0/331289061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0036050 con 0x7fe2f006db10 2024-04-25T11:43:00.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.124+0000 7fe2ecc00640 1 -- 172.21.15.19:0/331289061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0042070 con 0x7fe2f006db10 2024-04-25T11:43:00.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.124+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f015b320 con 0x7fe2f006db10 2024-04-25T11:43:00.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.124+0000 7fe2ecc00640 1 -- 172.21.15.19:0/331289061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0009510 con 0x7fe2f006db10 2024-04-25T11:43:00.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.124+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f015b8e0 con 0x7fe2f006db10 2024-04-25T11:43:00.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.125+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f015ea50 con 0x7fe2f006db10 2024-04-25T11:43:00.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.127+0000 7fe2ecc00640 1 -- 172.21.15.19:0/331289061 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe2e0007c50 con 0x7fe2f006db10 2024-04-25T11:43:00.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.128+0000 7fe2ecc00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2cc0744e0 0x7fe2cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.128+0000 7fe2ecc00640 1 -- 172.21.15.19:0/331289061 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fe2e00b6100 con 0x7fe2f006db10 2024-04-25T11:43:00.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.129+0000 7fe2e7400640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2cc0744e0 0x7fe2cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.130+0000 7fe2e7400640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2cc0744e0 0x7fe2cc0769f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8006fd0 tx=0x7fe2d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:00.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.133+0000 7fe2ecc00640 1 -- 172.21.15.19:0/331289061 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2f015ea50 con 0x7fe2f006db10 2024-04-25T11:43:00.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.339+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2f0076c60 con 0x7fe2f006db10 2024-04-25T11:43:00.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.340+0000 7fe2ecc00640 1 -- 172.21.15.19:0/331289061 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7fe2f006df20 con 0x7fe2f006db10 2024-04-25T11:43:00.326 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.342+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2cc0744e0 msgr2=0x7fe2cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.342+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2cc0744e0 0x7fe2cc0769f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8006fd0 tx=0x7fe2d8008040 comp rx=0 tx=0).stop 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.342+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f006db10 msgr2=0x7fe2f015aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.342+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f006db10 0x7fe2f015aad0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0034040 tx=0x7fe2e0004a40 comp rx=0 tx=0).stop 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 shutdown_connections 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe2cc0744e0 0x7fe2cc0769f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2f0144340 0x7fe2f01f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2f015ec90 0x7fe2f00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 --2- 172.21.15.19:0/331289061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2f006db10 0x7fe2f015aad0 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 >> 172.21.15.19:0/331289061 conn(0x7fe2f0072f40 msgr2=0x7fe2f00b5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:00.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 shutdown_connections 2024-04-25T11:43:00.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.343+0000 7fe2f5c00640 1 -- 172.21.15.19:0/331289061 wait complete. 2024-04-25T11:43:00.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:43:00.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.661+0000 7f35d1600640 1 Processor -- start 2024-04-25T11:43:00.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.661+0000 7f35d1600640 1 -- start start 2024-04-25T11:43:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.664+0000 7f35d1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 0x7f35cc0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.664+0000 7f35d1600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35cc071110 con 0x7f35cc070630 2024-04-25T11:43:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.664+0000 7f35d1600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35cc06fb50 con 0x7f35cc06dce0 2024-04-25T11:43:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.664+0000 7f35d1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc0713f0 con 0x7f35cc15adf0 2024-04-25T11:43:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.664+0000 7f35caa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f35cc070630 0x7f35cc06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55496/0 (socket says 172.21.15.19:55496) 2024-04-25T11:43:00.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.664+0000 7f35caa00640 1 -- 172.21.15.19:0/2752776329 learned_addr learned my addr 172.21.15.19:0/2752776329 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:00.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35cb400640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 0x7f35cc0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416417751 0 0) 0x7f35cc071110 con 0x7f35cc070630 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35b0003520 con 0x7f35cc070630 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 158593241 0 0) 0x7f35b0003520 con 0x7f35cc070630 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 msgr2=0x7f35cc0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 0x7f35cc0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 >> v1:172.21.15.169:6789/0 conn(0x7f35cc06dce0 legacy=0x7f35cc160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc06fe30 con 0x7f35cc070630 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f35bc001670 con 0x7f35cc070630 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.665+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 >> v1:172.21.15.19:6789/0 conn(0x7f35cc070630 legacy=0x7f35cc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35c9600640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35b00039e0 0x7f35b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35c9600640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35b0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35c9600640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 0x7f35b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc0713f0 con 0x7f35b0004800 2024-04-25T11:43:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35b000bb20 con 0x7f35b00039e0 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35b000df30 con 0x7f35cc15adf0 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35ca000640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35b0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35caa00640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35b00039e0 0x7f35b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35ca000640 1 -- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 msgr2=0x7f35b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35cb400640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 0x7f35b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35ca000640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc15adf0 0x7f35b0005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35ca000640 1 -- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35b00039e0 msgr2=0x7f35b0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35ca000640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35b00039e0 0x7f35b0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.666+0000 7f35ca000640 1 -- 172.21.15.19:0/2752776329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b0003520 con 0x7f35b0004800 2024-04-25T11:43:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.667+0000 7f35ca000640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35b0004c10 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f35b8001860 tx=0x7f35b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56387053ce053e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.668+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8037070 con 0x7f35b0004800 2024-04-25T11:43:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.668+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b802ac30 con 0x7f35b0004800 2024-04-25T11:43:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.668+0000 7f35c9600640 1 -- 172.21.15.19:0/2752776329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b8001430 con 0x7f35b0004800 2024-04-25T11:43:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.668+0000 7f35d1600640 1 -- 172.21.15.19:0/2752776329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 msgr2=0x7f35b0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.668+0000 7f35d1600640 1 --2- 172.21.15.19:0/2752776329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35b0004c10 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f35b8001860 tx=0x7f35b0005830 comp rx=0 tx=0).stop 2024-04-25T11:43:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.669+0000 7f35cb400640 1 -- 172.21.15.19:0/2752776329 reap_dead start 2024-04-25T11:43:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.669+0000 7f35d1600640 1 -- 172.21.15.19:0/2752776329 shutdown_connections 2024-04-25T11:43:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.669+0000 7f35d1600640 1 -- 172.21.15.19:0/2752776329 >> 172.21.15.19:0/2752776329 conn(0x7f35cc073090 msgr2=0x7f35cc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.669+0000 7f35d1600640 1 -- 172.21.15.19:0/2752776329 shutdown_connections 2024-04-25T11:43:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.669+0000 7f35d1600640 1 -- 172.21.15.19:0/2752776329 wait complete. 2024-04-25T11:43:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.671+0000 7f35d1600640 1 Processor -- start 2024-04-25T11:43:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.671+0000 7f35d1600640 1 -- start start 2024-04-25T11:43:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35d1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35cc149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35d1600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35cc159c20 0x7f35cc14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35d1600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc0af5f0 0x7f35cc0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35d1600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc070c80 con 0x7f35b0004800 2024-04-25T11:43:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35d1600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc0701a0 con 0x7f35cc159c20 2024-04-25T11:43:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35d1600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc070e00 con 0x7f35cc0af5f0 2024-04-25T11:43:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35caa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35cc149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35caa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35cc149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49078/0 (socket says 172.21.15.19:49078) 2024-04-25T11:43:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.673+0000 7f35caa00640 1 -- 172.21.15.19:0/3684370131 learned_addr learned my addr 172.21.15.19:0/3684370131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35ca000640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35cc159c20 0x7f35cc14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35cb400640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc0af5f0 0x7f35cc0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35caa00640 1 -- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc0af5f0 msgr2=0x7f35cc0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35caa00640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc0af5f0 0x7f35cc0afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35caa00640 1 -- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35cc159c20 msgr2=0x7f35cc14a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35caa00640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35cc159c20 0x7f35cc14a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35caa00640 1 -- 172.21.15.19:0/3684370131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc0aff40 con 0x7f35b0004800 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35ca000640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35cc159c20 0x7f35cc14a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.674+0000 7f35caa00640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35cc149c40 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00e730 tx=0x7f35bc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.675+0000 7f35c2a00640 1 -- 172.21.15.19:0/3684370131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00c660 con 0x7f35b0004800 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.675+0000 7f35c2a00640 1 -- 172.21.15.19:0/3684370131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35bc018070 con 0x7f35b0004800 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.675+0000 7f35c2a00640 1 -- 172.21.15.19:0/3684370131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00ca60 con 0x7f35b0004800 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.675+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc158b80 con 0x7f35b0004800 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.675+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc159140 con 0x7f35b0004800 2024-04-25T11:43:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.676+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc050740 con 0x7f35b0004800 2024-04-25T11:43:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.677+0000 7f35c2a00640 1 -- 172.21.15.19:0/3684370131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f35bc01d460 con 0x7f35b0004800 2024-04-25T11:43:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.679+0000 7f35c2a00640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35ac0744b0 0x7f35ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.679+0000 7f35c2a00640 1 -- 172.21.15.19:0/3684370131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f35bc0951a0 con 0x7f35b0004800 2024-04-25T11:43:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.680+0000 7f35ca000640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35ac0744b0 0x7f35ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:00.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.681+0000 7f35ca000640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35ac0744b0 0x7f35ac0769c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f35b8034000 tx=0x7f35b8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:00.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.685+0000 7f35c2a00640 1 -- 172.21.15.19:0/3684370131 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35cc0aff40 con 0x7f35b0004800 2024-04-25T11:43:00.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.888+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35cc06cf00 con 0x7f35b0004800 2024-04-25T11:43:00.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.889+0000 7f35c2a00640 1 -- 172.21.15.19:0/3684370131 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7f35bc05f950 con 0x7f35b0004800 2024-04-25T11:43:00.875 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:43:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.891+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35ac0744b0 msgr2=0x7f35ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.891+0000 7f35d1600640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35ac0744b0 0x7f35ac0769c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f35b8034000 tx=0x7f35b8009030 comp rx=0 tx=0).stop 2024-04-25T11:43:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.891+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 msgr2=0x7f35cc149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.891+0000 7f35d1600640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35cc149c40 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00e730 tx=0x7f35bc00ebf0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.891+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 shutdown_connections 2024-04-25T11:43:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.892+0000 7f35d1600640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f35ac0744b0 0x7f35ac0769c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.892+0000 7f35d1600640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35cc159c20 0x7f35cc14a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.892+0000 7f35d1600640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35cc0af5f0 0x7f35cc0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.892+0000 7f35d1600640 1 --2- 172.21.15.19:0/3684370131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0004800 0x7f35cc149c40 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.892+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 >> 172.21.15.19:0/3684370131 conn(0x7f35cc073090 msgr2=0x7f35cc15b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.892+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 shutdown_connections 2024-04-25T11:43:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:00.892+0000 7f35d1600640 1 -- 172.21.15.19:0/3684370131 wait complete. 2024-04-25T11:43:00.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:43:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.207+0000 7f509a200640 1 Processor -- start 2024-04-25T11:43:01.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.207+0000 7f509a200640 1 -- start start 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.209+0000 7f509a200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5094160360 0x7f50941647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.209+0000 7f509a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50940b5c90 0x7f50940b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.209+0000 7f509a200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50940b65e0 con 0x7f5094070680 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.209+0000 7f509a200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f509406d100 con 0x7f50940b5c90 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.209+0000 7f509a200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f509406c7d0 con 0x7f5094160360 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.210+0000 7f5093e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50940b5c90 0x7f50940b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.210+0000 7f5093e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50940b5c90 0x7f50940b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49090/0 (socket says 172.21.15.19:49090) 2024-04-25T11:43:01.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.210+0000 7f5093e00640 1 -- 172.21.15.19:0/727397007 learned_addr learned my addr 172.21.15.19:0/727397007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.210+0000 7f5093e00640 1 -- 172.21.15.19:0/727397007 >> v2:172.21.15.169:3300/0 conn(0x7f5094160360 msgr2=0x7f50941647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.210+0000 7f5093e00640 1 --2- 172.21.15.19:0/727397007 >> v2:172.21.15.169:3300/0 conn(0x7f5094160360 0x7f50941647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.210+0000 7f5093e00640 1 -- 172.21.15.19:0/727397007 >> v1:172.21.15.169:6789/0 conn(0x7f5094070680 legacy=0x7f50940af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:01.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.210+0000 7f5093e00640 1 -- 172.21.15.19:0/727397007 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509406cab0 con 0x7f50940b5c90 2024-04-25T11:43:01.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.212+0000 7f5093e00640 1 --2- 172.21.15.19:0/727397007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f50940b60a0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f50940b46e0 tx=0x7f508c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0583b862035074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:01.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.212+0000 7f5092000640 1 -- 172.21.15.19:0/727397007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c036070 con 0x7f50940b5c90 2024-04-25T11:43:01.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.212+0000 7f5092000640 1 -- 172.21.15.19:0/727397007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c031600 con 0x7f50940b5c90 2024-04-25T11:43:01.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.212+0000 7f5092000640 1 -- 172.21.15.19:0/727397007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c02ae60 con 0x7f50940b5c90 2024-04-25T11:43:01.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 -- 172.21.15.19:0/727397007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 msgr2=0x7f50940b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 --2- 172.21.15.19:0/727397007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f50940b60a0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f50940b46e0 tx=0x7f508c02a680 comp rx=0 tx=0).stop 2024-04-25T11:43:01.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 -- 172.21.15.19:0/727397007 shutdown_connections 2024-04-25T11:43:01.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 --2- 172.21.15.19:0/727397007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f50940b60a0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 --2- 172.21.15.19:0/727397007 >> v2:172.21.15.169:3300/0 conn(0x7f5094160360 0x7f50941647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 -- 172.21.15.19:0/727397007 >> 172.21.15.19:0/727397007 conn(0x7f50940d1ae0 msgr2=0x7f50940d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 -- 172.21.15.19:0/727397007 shutdown_connections 2024-04-25T11:43:01.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.213+0000 7f509a200640 1 -- 172.21.15.19:0/727397007 wait complete. 2024-04-25T11:43:01.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.215+0000 7f509a200640 1 Processor -- start 2024-04-25T11:43:01.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.216+0000 7f509a200640 1 -- start start 2024-04-25T11:43:01.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.217+0000 7f509a200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5094070680 0x7f509415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.217+0000 7f509a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f509415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.217+0000 7f509a200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5094160360 0x7f509415b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.217+0000 7f509a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50940afbb0 con 0x7f50940b5c90 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.217+0000 7f509a200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f509406fcc0 con 0x7f5094160360 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.217+0000 7f509a200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f50940afd30 con 0x7f5094070680 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.217+0000 7f5092a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f509415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5092a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f509415ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49098/0 (socket says 172.21.15.19:49098) 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5092a00640 1 -- 172.21.15.19:0/653491961 learned_addr learned my addr 172.21.15.19:0/653491961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5093e00640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5094160360 0x7f509415b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5093400640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5094070680 0x7f509415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5092a00640 1 -- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5094070680 msgr2=0x7f509415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5092a00640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5094070680 0x7f509415a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5092a00640 1 -- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5094160360 msgr2=0x7f509415b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5092a00640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5094160360 0x7f509415b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.218+0000 7f5092a00640 1 -- 172.21.15.19:0/653491961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50940ad9c0 con 0x7f50940b5c90 2024-04-25T11:43:01.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.219+0000 7f5092a00640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f509415ac20 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f508000b440 tx=0x7f5080009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.219+0000 7f508b400640 1 -- 172.21.15.19:0/653491961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080015020 con 0x7f50940b5c90 2024-04-25T11:43:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.219+0000 7f508b400640 1 -- 172.21.15.19:0/653491961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5080011830 con 0x7f50940b5c90 2024-04-25T11:43:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.219+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50940ac920 con 0x7f50940b5c90 2024-04-25T11:43:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.219+0000 7f508b400640 1 -- 172.21.15.19:0/653491961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080011c20 con 0x7f50940b5c90 2024-04-25T11:43:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.219+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50940acee0 con 0x7f50940b5c90 2024-04-25T11:43:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.220+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5094050740 con 0x7f50940b5c90 2024-04-25T11:43:01.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.222+0000 7f508b400640 1 -- 172.21.15.19:0/653491961 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5080018020 con 0x7f50940b5c90 2024-04-25T11:43:01.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.223+0000 7f508b400640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50740744e0 0x7f50740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.223+0000 7f508b400640 1 -- 172.21.15.19:0/653491961 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f5080095200 con 0x7f50940b5c90 2024-04-25T11:43:01.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.224+0000 7f5093400640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50740744e0 0x7f50740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.225+0000 7f5093400640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50740744e0 0x7f50740769f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5084006fd0 tx=0x7f5084008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:01.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.228+0000 7f508b400640 1 -- 172.21.15.19:0/653491961 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f508001f020 con 0x7f50940b5c90 2024-04-25T11:43:01.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.444+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f509406f090 con 0x7f50940b5c90 2024-04-25T11:43:01.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.445+0000 7f508b400640 1 -- 172.21.15.19:0/653491961 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v144) ==== 80+0+124 (secure 0 0 0) 0x7f508005f2d0 con 0x7f50940b5c90 2024-04-25T11:43:01.431 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.169:6835/2610059988 2024-04-26T11:42:44.471606+0000 2024-04-25T11:43:01.431 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.169:6834/2610059988 2024-04-26T11:42:44.471606+0000 2024-04-25T11:43:01.432 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-04-25T11:43:01.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50740744e0 msgr2=0x7f50740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50740744e0 0x7f50740769f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5084006fd0 tx=0x7f5084008040 comp rx=0 tx=0).stop 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 msgr2=0x7f509415ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f509415ac20 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f508000b440 tx=0x7f5080009f90 comp rx=0 tx=0).stop 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 shutdown_connections 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f50740744e0 0x7f50740769f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5094160360 0x7f509415b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50940b5c90 0x7f509415ac20 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 --2- 172.21.15.19:0/653491961 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5094070680 0x7f509415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 >> 172.21.15.19:0/653491961 conn(0x7f50940d1ae0 msgr2=0x7f50940b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 shutdown_connections 2024-04-25T11:43:01.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:01.448+0000 7f509a200640 1 -- 172.21.15.19:0/653491961 wait complete. 2024-04-25T11:43:01.461 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:43:01.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:01.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:43:01.559 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:43:01.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:01.559 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:43:01.614 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:43:01.614 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:43:01.614 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:01.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:43:01.674 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:43:01.674 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:43:01.674 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:01.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:43:01.735 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:43:01.735 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:43:01.735 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:01.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:43:01.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.099+0000 7f8abbe00640 1 Processor -- start 2024-04-25T11:43:02.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.100+0000 7f8abbe00640 1 -- start start 2024-04-25T11:43:02.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.102+0000 7f8abbe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8ab4070680 0x7f8ab41647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.102+0000 7f8abbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ab40b5c90 0x7f8ab40b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.102+0000 7f8abbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab40b65e0 con 0x7f8ab4160360 2024-04-25T11:43:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.102+0000 7f8abbe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab406d100 con 0x7f8ab40b5c90 2024-04-25T11:43:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.102+0000 7f8abbe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab406c7d0 con 0x7f8ab4070680 2024-04-25T11:43:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ab40b5c90 0x7f8ab40b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab8a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ab4160360 0x7f8ab40af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55512/0 (socket says 172.21.15.19:55512) 2024-04-25T11:43:02.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab8a00640 1 -- 172.21.15.19:0/612461561 learned_addr learned my addr 172.21.15.19:0/612461561 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab9400640 1 --2- 172.21.15.19:0/612461561 >> v2:172.21.15.169:3300/0 conn(0x7f8ab4070680 0x7f8ab41647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab9e00640 1 -- 172.21.15.19:0/612461561 >> v2:172.21.15.169:3300/0 conn(0x7f8ab4070680 msgr2=0x7f8ab41647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab9e00640 1 --2- 172.21.15.19:0/612461561 >> v2:172.21.15.169:3300/0 conn(0x7f8ab4070680 0x7f8ab41647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab9e00640 1 -- 172.21.15.19:0/612461561 >> v1:172.21.15.19:6789/0 conn(0x7f8ab4160360 legacy=0x7f8ab40af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab9e00640 1 -- 172.21.15.19:0/612461561 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab406cab0 con 0x7f8ab40b5c90 2024-04-25T11:43:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab9400640 1 --2- 172.21.15.19:0/612461561 >> v2:172.21.15.169:3300/0 conn(0x7f8ab4070680 0x7f8ab41647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:02.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.103+0000 7f8ab3e00640 1 -- 172.21.15.19:0/612461561 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193490619 0 0) 0x7f8aa4001490 con 0x7f8ab4160360 2024-04-25T11:43:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.105+0000 7f8ab9e00640 1 --2- 172.21.15.19:0/612461561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab40b5c90 0x7f8ab40b60a0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4072b00 tx=0x7f8aac02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfaf9b10e52c3a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.105+0000 7f8ab3e00640 1 -- 172.21.15.19:0/612461561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac037070 con 0x7f8ab40b5c90 2024-04-25T11:43:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.105+0000 7f8ab3e00640 1 -- 172.21.15.19:0/612461561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac02add0 con 0x7f8ab40b5c90 2024-04-25T11:43:02.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.105+0000 7f8ab3e00640 1 -- 172.21.15.19:0/612461561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac032660 con 0x7f8ab40b5c90 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 -- 172.21.15.19:0/612461561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab40b5c90 msgr2=0x7f8ab40b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 --2- 172.21.15.19:0/612461561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab40b5c90 0x7f8ab40b60a0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4072b00 tx=0x7f8aac02a080 comp rx=0 tx=0).stop 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 -- 172.21.15.19:0/612461561 shutdown_connections 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 --2- 172.21.15.19:0/612461561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab40b5c90 0x7f8ab40b60a0 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 --2- 172.21.15.19:0/612461561 >> v2:172.21.15.169:3300/0 conn(0x7f8ab4070680 0x7f8ab41647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 -- 172.21.15.19:0/612461561 >> 172.21.15.19:0/612461561 conn(0x7f8ab40d1ae0 msgr2=0x7f8ab40d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 -- 172.21.15.19:0/612461561 shutdown_connections 2024-04-25T11:43:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.106+0000 7f8abbe00640 1 -- 172.21.15.19:0/612461561 wait complete. 2024-04-25T11:43:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.108+0000 7f8abbe00640 1 Processor -- start 2024-04-25T11:43:02.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.108+0000 7f8abbe00640 1 -- start start 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4070680 0x7f8ab41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab40b5c90 0x7f8ab41eb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8abbe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4160360 0x7f8ab4144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8abbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab40afbb0 con 0x7f8ab4070680 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8abbe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab406fcc0 con 0x7f8ab4160360 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8abbe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab40afd30 con 0x7f8ab40b5c90 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8ab9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4070680 0x7f8ab41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8ab9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4070680 0x7f8ab41560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49124/0 (socket says 172.21.15.19:49124) 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.110+0000 7f8ab9400640 1 -- 172.21.15.19:0/1187565945 learned_addr learned my addr 172.21.15.19:0/1187565945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab8a00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab40b5c90 0x7f8ab41eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9e00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4160360 0x7f8ab4144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9400640 1 -- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab40b5c90 msgr2=0x7f8ab41eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9400640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab40b5c90 0x7f8ab41eb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9400640 1 -- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4160360 msgr2=0x7f8ab4144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9400640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4160360 0x7f8ab4144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9400640 1 -- 172.21.15.19:0/1187565945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab4142d80 con 0x7f8ab4070680 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9e00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4160360 0x7f8ab4144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab9400640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4070680 0x7f8ab41560d0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c820 tx=0x7f8aa800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:02.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab1600640 1 -- 172.21.15.19:0/1187565945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8015070 con 0x7f8ab4070680 2024-04-25T11:43:02.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab1600640 1 -- 172.21.15.19:0/1187565945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa8007a30 con 0x7f8ab4070680 2024-04-25T11:43:02.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab4143280 con 0x7f8ab4070680 2024-04-25T11:43:02.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.111+0000 7f8ab1600640 1 -- 172.21.15.19:0/1187565945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8007e20 con 0x7f8ab4070680 2024-04-25T11:43:02.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.112+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab4143840 con 0x7f8ab4070680 2024-04-25T11:43:02.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.113+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab4050740 con 0x7f8ab4070680 2024-04-25T11:43:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.114+0000 7f8ab1600640 1 -- 172.21.15.19:0/1187565945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8aa8018070 con 0x7f8ab4070680 2024-04-25T11:43:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.117+0000 7f8ab1600640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a980744b0 0x7f8a980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.117+0000 7f8ab1600640 1 -- 172.21.15.19:0/1187565945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f8aa8095240 con 0x7f8ab4070680 2024-04-25T11:43:02.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.117+0000 7f8ab8a00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a980744b0 0x7f8a980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.119+0000 7f8ab8a00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a980744b0 0x7f8a980769c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40b6350 tx=0x7f8aa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.121+0000 7f8ab1600640 1 -- 172.21.15.19:0/1187565945 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8aa805f8d0 con 0x7f8ab4070680 2024-04-25T11:43:02.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.368+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ab40b60a0 con 0x7f8ab4070680 2024-04-25T11:43:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.369+0000 7f8ab1600640 1 -- 172.21.15.19:0/1187565945 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 372 v372) ==== 95+0+5195 (secure 0 0 0) 0x7f8aa805f1d0 con 0x7f8ab4070680 2024-04-25T11:43:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:02.355 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":372,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":372,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:59.523278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":12,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:43:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 372 2024-04-25T11:43:02.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a980744b0 msgr2=0x7f8a980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a980744b0 0x7f8a980769c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40b6350 tx=0x7f8aa4009040 comp rx=0 tx=0).stop 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4070680 msgr2=0x7f8ab41560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4070680 0x7f8ab41560d0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c820 tx=0x7f8aa800cce0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 shutdown_connections 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8a980744b0 0x7f8a980769c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ab4160360 0x7f8ab4144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ab40b5c90 0x7f8ab41eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 --2- 172.21.15.19:0/1187565945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ab4070680 0x7f8ab41560d0 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 >> 172.21.15.19:0/1187565945 conn(0x7f8ab40d1ae0 msgr2=0x7f8ab40b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 shutdown_connections 2024-04-25T11:43:02.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.372+0000 7f8abbe00640 1 -- 172.21.15.19:0/1187565945 wait complete. 2024-04-25T11:43:02.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.675+0000 7f5ee4400640 1 Processor -- start 2024-04-25T11:43:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.675+0000 7f5ee4400640 1 -- start start 2024-04-25T11:43:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee4400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5edc160360 0x7f5edc1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee4400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5edc0b5c90 0x7f5edc0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee4400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc0b65e0 con 0x7f5edc070680 2024-04-25T11:43:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee4400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5edc06d100 con 0x7f5edc0b5c90 2024-04-25T11:43:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee4400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f5edc06c7d0 con 0x7f5edc160360 2024-04-25T11:43:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee2400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5edc0b5c90 0x7f5edc0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee2400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5edc0b5c90 0x7f5edc0b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49138/0 (socket says 172.21.15.19:49138) 2024-04-25T11:43:02.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.678+0000 7f5ee2400640 1 -- 172.21.15.19:0/75662984 learned_addr learned my addr 172.21.15.19:0/75662984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.679+0000 7f5ee1000640 1 --2- 172.21.15.19:0/75662984 >> v2:172.21.15.169:3300/0 conn(0x7f5edc160360 0x7f5edc1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.679+0000 7f5ee2400640 1 -- 172.21.15.19:0/75662984 >> v2:172.21.15.169:3300/0 conn(0x7f5edc160360 msgr2=0x7f5edc1647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.679+0000 7f5ee2400640 1 --2- 172.21.15.19:0/75662984 >> v2:172.21.15.169:3300/0 conn(0x7f5edc160360 0x7f5edc1647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.679+0000 7f5ee2400640 1 -- 172.21.15.19:0/75662984 >> v1:172.21.15.169:6789/0 conn(0x7f5edc070680 legacy=0x7f5edc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.679+0000 7f5ee2400640 1 -- 172.21.15.19:0/75662984 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc06cab0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.679+0000 7f5ee1000640 1 --2- 172.21.15.19:0/75662984 >> v2:172.21.15.169:3300/0 conn(0x7f5edc160360 0x7f5edc1647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.680+0000 7f5ee2400640 1 --2- 172.21.15.19:0/75662984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc0b60a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0b46e0 tx=0x7f5ed402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d800b7254df08f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.680+0000 7f5edbe00640 1 -- 172.21.15.19:0/75662984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4036070 con 0x7f5edc0b5c90 2024-04-25T11:43:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5edbe00640 1 -- 172.21.15.19:0/75662984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed4031600 con 0x7f5edc0b5c90 2024-04-25T11:43:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5edbe00640 1 -- 172.21.15.19:0/75662984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40319d0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5ee4400640 1 -- 172.21.15.19:0/75662984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 msgr2=0x7f5edc0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5ee4400640 1 --2- 172.21.15.19:0/75662984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc0b60a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0b46e0 tx=0x7f5ed402a660 comp rx=0 tx=0).stop 2024-04-25T11:43:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5ee4400640 1 -- 172.21.15.19:0/75662984 shutdown_connections 2024-04-25T11:43:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5ee4400640 1 --2- 172.21.15.19:0/75662984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc0b60a0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5ee4400640 1 --2- 172.21.15.19:0/75662984 >> v2:172.21.15.169:3300/0 conn(0x7f5edc160360 0x7f5edc1647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.681+0000 7f5ee4400640 1 -- 172.21.15.19:0/75662984 >> 172.21.15.19:0/75662984 conn(0x7f5edc0d1ae0 msgr2=0x7f5edc0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.682+0000 7f5ee4400640 1 -- 172.21.15.19:0/75662984 shutdown_connections 2024-04-25T11:43:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.682+0000 7f5ee4400640 1 -- 172.21.15.19:0/75662984 wait complete. 2024-04-25T11:43:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.684+0000 7f5ee4400640 1 Processor -- start 2024-04-25T11:43:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.684+0000 7f5ee4400640 1 -- start start 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee4400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5edc070680 0x7f5edc15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee4400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc14e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee4400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5edc160360 0x7f5edc1ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee4400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc0afbb0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee4400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc06fcc0 con 0x7f5edc160360 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee4400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc0afd30 con 0x7f5edc070680 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc14e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49146/0 (socket says 172.21.15.19:49146) 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee1000640 1 -- 172.21.15.19:0/1343317670 learned_addr learned my addr 172.21.15.19:0/1343317670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:02.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee1000640 1 -- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5edc070680 msgr2=0x7f5edc15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee1a00640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5edc070680 0x7f5edc15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee1000640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5edc070680 0x7f5edc15cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee2400640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5edc160360 0x7f5edc1ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.686+0000 7f5ee1000640 1 -- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5edc160360 msgr2=0x7f5edc1ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ee1000640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5edc160360 0x7f5edc1ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ee1000640 1 -- 172.21.15.19:0/1343317670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc14bcf0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ee1000640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc14e770 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c7f0 tx=0x7f5ec800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ed9600640 1 -- 172.21.15.19:0/1343317670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8015070 con 0x7f5edc0b5c90 2024-04-25T11:43:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ed9600640 1 -- 172.21.15.19:0/1343317670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec80077c0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ed9600640 1 -- 172.21.15.19:0/1343317670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8007bd0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc15dd80 con 0x7f5edc0b5c90 2024-04-25T11:43:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.687+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc15e020 con 0x7f5edc0b5c90 2024-04-25T11:43:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.689+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc050740 con 0x7f5edc0b5c90 2024-04-25T11:43:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.690+0000 7f5ed9600640 1 -- 172.21.15.19:0/1343317670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5ec8018070 con 0x7f5edc0b5c90 2024-04-25T11:43:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.691+0000 7f5ed9600640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5ebc0744e0 0x7f5ebc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.691+0000 7f5ed9600640 1 -- 172.21.15.19:0/1343317670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f5ec8095410 con 0x7f5edc0b5c90 2024-04-25T11:43:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.692+0000 7f5ee1a00640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5ebc0744e0 0x7f5ebc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.693+0000 7f5ee1a00640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5ebc0744e0 0x7f5ebc0769f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5edc071250 tx=0x7f5ed0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.696+0000 7f5ed9600640 1 -- 172.21.15.19:0/1343317670 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ec805faa0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.902+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5edc0c7630 con 0x7f5edc0b5c90 2024-04-25T11:43:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.905+0000 7f5ed9600640 1 -- 172.21.15.19:0/1343317670 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16750 (secure 0 0 0) 0x7f5ec805f3a0 con 0x7f5edc0b5c90 2024-04-25T11:43:02.892 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:02.892 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":144,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:44.471720+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/2610059988":"2024-04-26T11:42:44.471606+0000","172.21.15.169:6834/2610059988":"2024-04-26T11:42:44.471606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5ebc0744e0 msgr2=0x7f5ebc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5ebc0744e0 0x7f5ebc0769f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5edc071250 tx=0x7f5ed0008040 comp rx=0 tx=0).stop 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 msgr2=0x7f5edc14e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc14e770 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c7f0 tx=0x7f5ec800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 shutdown_connections 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f5ebc0744e0 0x7f5ebc0769f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5edc160360 0x7f5edc1ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5edc0b5c90 0x7f5edc14e770 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 --2- 172.21.15.19:0/1343317670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5edc070680 0x7f5edc15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 >> 172.21.15.19:0/1343317670 conn(0x7f5edc0d1ae0 msgr2=0x7f5edc0d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.908+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 shutdown_connections 2024-04-25T11:43:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:02.909+0000 7f5ee4400640 1 -- 172.21.15.19:0/1343317670 wait complete. 2024-04-25T11:43:02.923 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:27:12.475086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:42:27.399161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:42:28.185279+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:43:02.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:43:03.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.232+0000 7f4aad400640 1 Processor -- start 2024-04-25T11:43:03.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.232+0000 7f4aad400640 1 -- start start 2024-04-25T11:43:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aad400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aa8144340 0x7f4aa806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aad400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4aa806db10 0x7f4aa815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa815ec90 0x7f4aa8161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aad400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa8144750 con 0x7f4aa8144340 2024-04-25T11:43:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aad400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f4aa806df20 con 0x7f4aa806db10 2024-04-25T11:43:03.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aad400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa8070000 con 0x7f4aa815ec90 2024-04-25T11:43:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aa7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aa8144340 0x7f4aa806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aa7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aa8144340 0x7f4aa806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49148/0 (socket says 172.21.15.19:49148) 2024-04-25T11:43:03.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1846900290 learned_addr learned my addr 172.21.15.19:0/1846900290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.235+0000 7f4aaca00640 1 --2- 172.21.15.19:0/1846900290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa815ec90 0x7f4aa8161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.236+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1846900290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa815ec90 msgr2=0x7f4aa8161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.236+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1846900290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa815ec90 0x7f4aa8161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.236+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1846900290 >> v2:172.21.15.169:3300/0 conn(0x7f4aa806db10 msgr2=0x7f4aa815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.236+0000 7f4a9f400640 1 --2- 172.21.15.19:0/1846900290 >> v2:172.21.15.169:3300/0 conn(0x7f4aa806db10 0x7f4aa815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.236+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1846900290 >> v2:172.21.15.169:3300/0 conn(0x7f4aa806db10 0x7f4aa815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.236+0000 7f4aa7e00640 1 -- 172.21.15.19:0/1846900290 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa806f240 con 0x7f4aa8144340 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.236+0000 7f4a9f400640 1 --2- 172.21.15.19:0/1846900290 >> v2:172.21.15.169:3300/0 conn(0x7f4aa806db10 0x7f4aa815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:03.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.237+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1846900290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa806d5d0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8073ef0 tx=0x7f4a9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcaab9dd26ae9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:03.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.237+0000 7f4aa7400640 1 -- 172.21.15.19:0/1846900290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98037070 con 0x7f4aa8144340 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.237+0000 7f4aa7400640 1 -- 172.21.15.19:0/1846900290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a98032440 con 0x7f4aa8144340 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.237+0000 7f4aa7400640 1 -- 172.21.15.19:0/1846900290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98032810 con 0x7f4aa8144340 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 -- 172.21.15.19:0/1846900290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 msgr2=0x7f4aa806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 --2- 172.21.15.19:0/1846900290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa806d5d0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8073ef0 tx=0x7f4a9802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 -- 172.21.15.19:0/1846900290 shutdown_connections 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 --2- 172.21.15.19:0/1846900290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa815ec90 0x7f4aa8161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 --2- 172.21.15.19:0/1846900290 >> v2:172.21.15.169:3300/0 conn(0x7f4aa806db10 0x7f4aa815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 --2- 172.21.15.19:0/1846900290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa806d5d0 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 -- 172.21.15.19:0/1846900290 >> 172.21.15.19:0/1846900290 conn(0x7f4aa8072f40 msgr2=0x7f4aa8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 -- 172.21.15.19:0/1846900290 shutdown_connections 2024-04-25T11:43:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.238+0000 7f4aad400640 1 -- 172.21.15.19:0/1846900290 wait complete. 2024-04-25T11:43:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.241+0000 7f4aad400640 1 Processor -- start 2024-04-25T11:43:03.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.241+0000 7f4aad400640 1 -- start start 2024-04-25T11:43:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.242+0000 7f4aad400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa806db10 0x7f4aa80b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.242+0000 7f4aad400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa813ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.242+0000 7f4aad400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa815ec90 0x7f4aa8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.242+0000 7f4aad400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa806fae0 con 0x7f4aa8144340 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.242+0000 7f4aad400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa806f800 con 0x7f4aa815ec90 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.242+0000 7f4aad400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa815f0a0 con 0x7f4aa806db10 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa813ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa813ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49152/0 (socket says 172.21.15.19:49152) 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 -- 172.21.15.19:0/1905037913 learned_addr learned my addr 172.21.15.19:0/1905037913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa806db10 0x7f4aa80b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 -- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa806db10 msgr2=0x7f4aa80b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa806db10 0x7f4aa80b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 -- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa815ec90 msgr2=0x7f4aa8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4aaca00640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa815ec90 0x7f4aa8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa815ec90 0x7f4aa8159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.243+0000 7f4a9f400640 1 -- 172.21.15.19:0/1905037913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa80b6410 con 0x7f4aa8144340 2024-04-25T11:43:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.244+0000 7f4a9f400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa813ea70 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c7c0 tx=0x7f4a9000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:03.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.244+0000 7f4aa4c00640 1 -- 172.21.15.19:0/1905037913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90015070 con 0x7f4aa8144340 2024-04-25T11:43:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.244+0000 7f4aa4c00640 1 -- 172.21.15.19:0/1905037913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a900077c0 con 0x7f4aa8144340 2024-04-25T11:43:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.244+0000 7f4aa4c00640 1 -- 172.21.15.19:0/1905037913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90007b90 con 0x7f4aa8144340 2024-04-25T11:43:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.244+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa80b66f0 con 0x7f4aa8144340 2024-04-25T11:43:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.244+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa80b0180 con 0x7f4aa8144340 2024-04-25T11:43:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.245+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa8076c60 con 0x7f4aa8144340 2024-04-25T11:43:03.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.246+0000 7f4aa4c00640 1 -- 172.21.15.19:0/1905037913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a90018070 con 0x7f4aa8144340 2024-04-25T11:43:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.247+0000 7f4aa4c00640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a840744e0 0x7f4a840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.248+0000 7f4aa4c00640 1 -- 172.21.15.19:0/1905037913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f4a90094fc0 con 0x7f4aa8144340 2024-04-25T11:43:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.248+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a840744e0 0x7f4a840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.249+0000 7f4aa7e00640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a840744e0 0x7f4a840769f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4a98034000 tx=0x7f4a9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:03.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.252+0000 7f4aa4c00640 1 -- 172.21.15.19:0/1905037913 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a9005f770 con 0x7f4aa8144340 2024-04-25T11:43:03.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.456+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4aa80b7920 con 0x7f4aa8144340 2024-04-25T11:43:03.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.457+0000 7f4aa4c00640 1 -- 172.21.15.19:0/1905037913 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7f4a9005f070 con 0x7f4aa8144340 2024-04-25T11:43:03.443 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T11:43:03.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a840744e0 msgr2=0x7f4a840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a840744e0 0x7f4a840769f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4a98034000 tx=0x7f4a9802e000 comp rx=0 tx=0).stop 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 msgr2=0x7f4aa813ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa813ea70 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c7c0 tx=0x7f4a9000cc80 comp rx=0 tx=0).stop 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 shutdown_connections 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4a840744e0 0x7f4a840769f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa815ec90 0x7f4aa8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aa8144340 0x7f4aa813ea70 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 --2- 172.21.15.19:0/1905037913 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa806db10 0x7f4aa80b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 >> 172.21.15.19:0/1905037913 conn(0x7f4aa8072f40 msgr2=0x7f4aa8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 shutdown_connections 2024-04-25T11:43:03.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.460+0000 7f4aad400640 1 -- 172.21.15.19:0/1905037913 wait complete. 2024-04-25T11:43:03.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:43:03.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.759+0000 7f8d41a00640 1 Processor -- start 2024-04-25T11:43:03.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.759+0000 7f8d41a00640 1 -- start start 2024-04-25T11:43:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c06dce0 0x7f8d3c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d3c15ec90 0x7f8d3c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d3c070000 con 0x7f8d3c144340 2024-04-25T11:43:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8d3c144750 con 0x7f8d3c15ec90 2024-04-25T11:43:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3c06f240 con 0x7f8d3c06dce0 2024-04-25T11:43:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d3c15ec90 0x7f8d3c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8d3c15ec90 0x7f8d3c15f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49168/0 (socket says 172.21.15.19:49168) 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d41000640 1 -- 172.21.15.19:0/3873160505 learned_addr learned my addr 172.21.15.19:0/3873160505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.761+0000 7f8d3be00640 1 --1- 172.21.15.19:0/3873160505 >> v1:172.21.15.19:6789/0 conn(0x7f8d3c144340 0x7f8d3c06d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55516/0 (socket says 172.21.15.19:55516) 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.762+0000 7f8d3b400640 1 --2- 172.21.15.19:0/3873160505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c06dce0 0x7f8d3c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.762+0000 7f8d41000640 1 -- 172.21.15.19:0/3873160505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c06dce0 msgr2=0x7f8d3c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.762+0000 7f8d41000640 1 --2- 172.21.15.19:0/3873160505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c06dce0 0x7f8d3c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.762+0000 7f8d41000640 1 -- 172.21.15.19:0/3873160505 >> v1:172.21.15.19:6789/0 conn(0x7f8d3c144340 legacy=0x7f8d3c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.762+0000 7f8d41000640 1 -- 172.21.15.19:0/3873160505 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3c06f520 con 0x7f8d3c15ec90 2024-04-25T11:43:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.762+0000 7f8d3aa00640 1 -- 172.21.15.19:0/3873160505 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 126439018 0 0) 0x7f8d3c070000 con 0x7f8d3c144340 2024-04-25T11:43:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.763+0000 7f8d41000640 1 --2- 172.21.15.19:0/3873160505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c15ec90 0x7f8d3c15f4a0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009e50 tx=0x7f8d3402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7682b57224a38ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.763+0000 7f8d3aa00640 1 -- 172.21.15.19:0/3873160505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34036070 con 0x7f8d3c15ec90 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.763+0000 7f8d3aa00640 1 -- 172.21.15.19:0/3873160505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d34031600 con 0x7f8d3c15ec90 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.763+0000 7f8d3aa00640 1 -- 172.21.15.19:0/3873160505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340319d0 con 0x7f8d3c15ec90 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 -- 172.21.15.19:0/3873160505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c15ec90 msgr2=0x7f8d3c15f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 --2- 172.21.15.19:0/3873160505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c15ec90 0x7f8d3c15f4a0 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009e50 tx=0x7f8d3402a710 comp rx=0 tx=0).stop 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 -- 172.21.15.19:0/3873160505 shutdown_connections 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 --2- 172.21.15.19:0/3873160505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c15ec90 0x7f8d3c15f4a0 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 --2- 172.21.15.19:0/3873160505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c06dce0 0x7f8d3c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 -- 172.21.15.19:0/3873160505 >> 172.21.15.19:0/3873160505 conn(0x7f8d3c072f40 msgr2=0x7f8d3c073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 -- 172.21.15.19:0/3873160505 shutdown_connections 2024-04-25T11:43:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.764+0000 7f8d41a00640 1 -- 172.21.15.19:0/3873160505 wait complete. 2024-04-25T11:43:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.767+0000 7f8d41a00640 1 Processor -- start 2024-04-25T11:43:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.767+0000 7f8d41a00640 1 -- start start 2024-04-25T11:43:03.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.768+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c144340 0x7f8d3c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.768+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c15ec90 0x7f8d3c1f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.768+0000 7f8d41a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c158b70 0x7f8d3c0abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.768+0000 7f8d41a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c15f0a0 con 0x7f8d3c158b70 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.768+0000 7f8d41a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c06fae0 con 0x7f8d3c15ec90 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.768+0000 7f8d41a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3c15f220 con 0x7f8d3c144340 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c158b70 0x7f8d3c0abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c158b70 0x7f8d3c0abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49176/0 (socket says 172.21.15.19:49176) 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 -- 172.21.15.19:0/2205007516 learned_addr learned my addr 172.21.15.19:0/2205007516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d3be00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c144340 0x7f8d3c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d3b400640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c15ec90 0x7f8d3c1f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 -- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c144340 msgr2=0x7f8d3c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c144340 0x7f8d3c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 -- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c15ec90 msgr2=0x7f8d3c1f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c15ec90 0x7f8d3c1f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.769+0000 7f8d41000640 1 -- 172.21.15.19:0/2205007516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3c1f4750 con 0x7f8d3c158b70 2024-04-25T11:43:03.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.770+0000 7f8d41000640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c158b70 0x7f8d3c0abfa0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009e50 tx=0x7f8d34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.770+0000 7f8d33e00640 1 -- 172.21.15.19:0/2205007516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34036050 con 0x7f8d3c158b70 2024-04-25T11:43:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.770+0000 7f8d33e00640 1 -- 172.21.15.19:0/2205007516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d34042070 con 0x7f8d3c158b70 2024-04-25T11:43:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.770+0000 7f8d33e00640 1 -- 172.21.15.19:0/2205007516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34009510 con 0x7f8d3c158b70 2024-04-25T11:43:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.770+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3c1f4a30 con 0x7f8d3c158b70 2024-04-25T11:43:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.770+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3c0aa510 con 0x7f8d3c158b70 2024-04-25T11:43:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.771+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3c1f44c0 con 0x7f8d3c158b70 2024-04-25T11:43:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.773+0000 7f8d33e00640 1 -- 172.21.15.19:0/2205007516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8d34007bd0 con 0x7f8d3c158b70 2024-04-25T11:43:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.774+0000 7f8d33e00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d1c0744b0 0x7f8d1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.774+0000 7f8d33e00640 1 -- 172.21.15.19:0/2205007516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f8d340b6140 con 0x7f8d3c158b70 2024-04-25T11:43:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.774+0000 7f8d3be00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d1c0744b0 0x7f8d1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.776+0000 7f8d3be00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d1c0744b0 0x7f8d1c0769c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c000f60 tx=0x7f8d2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.778+0000 7f8d33e00640 1 -- 172.21.15.19:0/2205007516 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d34080870 con 0x7f8d3c158b70 2024-04-25T11:43:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.982+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d3c076c60 con 0x7f8d3c158b70 2024-04-25T11:43:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.983+0000 7f8d33e00640 1 -- 172.21.15.19:0/2205007516 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7f8d34080170 con 0x7f8d3c158b70 2024-04-25T11:43:03.969 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T11:43:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d1c0744b0 msgr2=0x7f8d1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d1c0744b0 0x7f8d1c0769c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c000f60 tx=0x7f8d2c009040 comp rx=0 tx=0).stop 2024-04-25T11:43:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c158b70 msgr2=0x7f8d3c0abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c158b70 0x7f8d3c0abfa0 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009e50 tx=0x7f8d34004a40 comp rx=0 tx=0).stop 2024-04-25T11:43:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 shutdown_connections 2024-04-25T11:43:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8d1c0744b0 0x7f8d1c0769c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d3c15ec90 0x7f8d3c1f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8d3c158b70 0x7f8d3c0abfa0 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 --2- 172.21.15.19:0/2205007516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d3c144340 0x7f8d3c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 >> 172.21.15.19:0/2205007516 conn(0x7f8d3c072f40 msgr2=0x7f8d3c15c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 shutdown_connections 2024-04-25T11:43:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:03.985+0000 7f8d41a00640 1 -- 172.21.15.19:0/2205007516 wait complete. 2024-04-25T11:43:04.001 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:43:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.287+0000 7f10c7e00640 1 Processor -- start 2024-04-25T11:43:04.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.287+0000 7f10c7e00640 1 -- start start 2024-04-25T11:43:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f10c80b0a50 0x7f10c80af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f10c806dce0 0x7f10c80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c80af9b0 con 0x7f10c815fff0 2024-04-25T11:43:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f10c80b54c0 con 0x7f10c80b0a50 2024-04-25T11:43:04.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f10c80afc90 con 0x7f10c806dce0 2024-04-25T11:43:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f10c80b0a50 0x7f10c80af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f10c80b0a50 0x7f10c80af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37528/0 (socket says 172.21.15.19:37528) 2024-04-25T11:43:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c6a00640 1 -- 172.21.15.19:0/3548046377 learned_addr learned my addr 172.21.15.19:0/3548046377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.290+0000 7f10c6000640 1 --1- 172.21.15.19:0/3548046377 >> v1:172.21.15.19:6789/0 conn(0x7f10c815fff0 0x7f10c806d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35026/0 (socket says 172.21.15.19:35026) 2024-04-25T11:43:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.291+0000 7f10c7400640 1 --2- 172.21.15.19:0/3548046377 >> v2:172.21.15.169:3300/0 conn(0x7f10c806dce0 0x7f10c80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.291+0000 7f10c6a00640 1 -- 172.21.15.19:0/3548046377 >> v2:172.21.15.169:3300/0 conn(0x7f10c806dce0 msgr2=0x7f10c80b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:04.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.291+0000 7f10c6a00640 1 --2- 172.21.15.19:0/3548046377 >> v2:172.21.15.169:3300/0 conn(0x7f10c806dce0 0x7f10c80b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.291+0000 7f10c6a00640 1 -- 172.21.15.19:0/3548046377 >> v1:172.21.15.19:6789/0 conn(0x7f10c815fff0 legacy=0x7f10c806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.291+0000 7f10c6a00640 1 -- 172.21.15.19:0/3548046377 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c80aff70 con 0x7f10c80b0a50 2024-04-25T11:43:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.292+0000 7f10c6a00640 1 --2- 172.21.15.19:0/3548046377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80af340 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f10c80b1570 tx=0x7f10b802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d3eec0786fdc29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.292+0000 7f10c5600640 1 -- 172.21.15.19:0/3548046377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8037070 con 0x7f10c80b0a50 2024-04-25T11:43:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.292+0000 7f10c5600640 1 -- 172.21.15.19:0/3548046377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b8032440 con 0x7f10c80b0a50 2024-04-25T11:43:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.292+0000 7f10c5600640 1 -- 172.21.15.19:0/3548046377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8032810 con 0x7f10c80b0a50 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 -- 172.21.15.19:0/3548046377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 msgr2=0x7f10c80af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 --2- 172.21.15.19:0/3548046377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80af340 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f10c80b1570 tx=0x7f10b802a160 comp rx=0 tx=0).stop 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 -- 172.21.15.19:0/3548046377 shutdown_connections 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 --2- 172.21.15.19:0/3548046377 >> v2:172.21.15.169:3300/0 conn(0x7f10c806dce0 0x7f10c80b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 --2- 172.21.15.19:0/3548046377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80af340 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 -- 172.21.15.19:0/3548046377 >> 172.21.15.19:0/3548046377 conn(0x7f10c8073160 msgr2=0x7f10c8073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 -- 172.21.15.19:0/3548046377 shutdown_connections 2024-04-25T11:43:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.293+0000 7f10c7e00640 1 -- 172.21.15.19:0/3548046377 wait complete. 2024-04-25T11:43:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.296+0000 7f10c7e00640 1 Processor -- start 2024-04-25T11:43:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.296+0000 7f10c7e00640 1 -- start start 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.297+0000 7f10c7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.297+0000 7f10c7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c815fff0 0x7f10c8161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.297+0000 7f10c7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c81615d0 0x7f10c815e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.297+0000 7f10c7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8160400 con 0x7f10c80b0a50 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10c80b0530 con 0x7f10c81615d0 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f10c806f090 con 0x7f10c815fff0 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37530/0 (socket says 172.21.15.19:37530) 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 -- 172.21.15.19:0/1168975826 learned_addr learned my addr 172.21.15.19:0/1168975826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6000640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c815fff0 0x7f10c8161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 -- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c815fff0 msgr2=0x7f10c8161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c815fff0 0x7f10c8161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 -- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c81615d0 msgr2=0x7f10c815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c7400640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c81615d0 0x7f10c815e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c81615d0 0x7f10c815e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.298+0000 7f10c6a00640 1 -- 172.21.15.19:0/1168975826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c8161e50 con 0x7f10c80b0a50 2024-04-25T11:43:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.299+0000 7f10c6a00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80d0d10 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f10b802a1a0 tx=0x7f10b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.299+0000 7f10bea00640 1 -- 172.21.15.19:0/1168975826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8041020 con 0x7f10c80b0a50 2024-04-25T11:43:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.299+0000 7f10bea00640 1 -- 172.21.15.19:0/1168975826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b80306c0 con 0x7f10c80b0a50 2024-04-25T11:43:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.299+0000 7f10bea00640 1 -- 172.21.15.19:0/1168975826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8037040 con 0x7f10c80b0a50 2024-04-25T11:43:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.299+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c8162130 con 0x7f10c80b0a50 2024-04-25T11:43:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.299+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c80ce910 con 0x7f10c80b0a50 2024-04-25T11:43:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.300+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c8076c60 con 0x7f10c80b0a50 2024-04-25T11:43:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.301+0000 7f10bea00640 1 -- 172.21.15.19:0/1168975826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10b8044020 con 0x7f10c80b0a50 2024-04-25T11:43:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.302+0000 7f10bea00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f10a80744e0 0x7f10a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.303+0000 7f10bea00640 1 -- 172.21.15.19:0/1168975826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f10b80b5a50 con 0x7f10c80b0a50 2024-04-25T11:43:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.303+0000 7f10c6000640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f10a80744e0 0x7f10a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.304+0000 7f10c6000640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f10a80744e0 0x7f10a80769f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f10c806d5f0 tx=0x7f10b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.308+0000 7f10bea00640 1 -- 172.21.15.19:0/1168975826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10b804b030 con 0x7f10c80b0a50 2024-04-25T11:43:04.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.500+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10c80b6630 con 0x7f10c80b0a50 2024-04-25T11:43:04.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.501+0000 7f10bea00640 1 -- 172.21.15.19:0/1168975826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7f10b8080170 con 0x7f10c80b0a50 2024-04-25T11:43:04.488 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T11:43:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f10a80744e0 msgr2=0x7f10a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f10a80744e0 0x7f10a80769f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f10c806d5f0 tx=0x7f10b4009040 comp rx=0 tx=0).stop 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 msgr2=0x7f10c80d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80d0d10 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f10b802a1a0 tx=0x7f10b80046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 shutdown_connections 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f10a80744e0 0x7f10a80769f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10c81615d0 0x7f10c815e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f10c815fff0 0x7f10c8161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 --2- 172.21.15.19:0/1168975826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c80b0a50 0x7f10c80d0d10 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 >> 172.21.15.19:0/1168975826 conn(0x7f10c8073160 msgr2=0x7f10c80684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 shutdown_connections 2024-04-25T11:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.504+0000 7f10c7e00640 1 -- 172.21.15.19:0/1168975826 wait complete. 2024-04-25T11:43:04.514 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.806+0000 7fb355000640 1 Processor -- start 2024-04-25T11:43:04.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.806+0000 7fb355000640 1 -- start start 2024-04-25T11:43:04.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.808+0000 7fb355000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3501443c0 0x7fb35006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.808+0000 7fb355000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3500cfcf0 0x7fb35006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.808+0000 7fb355000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35015f5f0 con 0x7fb350070370 2024-04-25T11:43:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.808+0000 7fb355000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb3500708d0 con 0x7fb3501443c0 2024-04-25T11:43:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.808+0000 7fb355000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb35015f160 con 0x7fb3500cfcf0 2024-04-25T11:43:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.809+0000 7fb345400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb350070370 0x7fb3500cf5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35038/0 (socket says 172.21.15.19:35038) 2024-04-25T11:43:04.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.809+0000 7fb345400640 1 -- 172.21.15.19:0/3967758954 learned_addr learned my addr 172.21.15.19:0/3967758954 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.809+0000 7fb34e800640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3500cfcf0 0x7fb35006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.809+0000 7fb34de00640 1 --2- 172.21.15.19:0/3967758954 >> v2:172.21.15.169:3300/0 conn(0x7fb3501443c0 0x7fb35006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.809+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1634910073 0 0) 0x7fb35015f5f0 con 0x7fb350070370 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.809+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb334003520 con 0x7fb350070370 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 580461131 0 0) 0x7fb334003520 con 0x7fb350070370 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3500cfcf0 msgr2=0x7fb35006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3500cfcf0 0x7fb35006c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 >> v2:172.21.15.169:3300/0 conn(0x7fb3501443c0 msgr2=0x7fb35006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 --2- 172.21.15.19:0/3967758954 >> v2:172.21.15.169:3300/0 conn(0x7fb3501443c0 0x7fb35006fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35015e830 con 0x7fb350070370 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fb338001670 con 0x7fb350070370 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 >> v1:172.21.15.19:6789/0 conn(0x7fb350070370 legacy=0x7fb3500cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3340039b0 0x7fb334003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb334004800 0x7fb334004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb334005150 0x7fb33400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3500708d0 con 0x7fb3340039b0 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb35015f160 con 0x7fb334005150 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb33400bf00 con 0x7fb334004800 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.810+0000 7fb34de00640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3340039b0 0x7fb334003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.811+0000 7fb345400640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb334004800 0x7fb334004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.811+0000 7fb34e800640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb334005150 0x7fb33400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.811+0000 7fb34de00640 1 -- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb334004800 msgr2=0x7fb334004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.811+0000 7fb34de00640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb334004800 0x7fb334004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.811+0000 7fb34de00640 1 -- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb334005150 msgr2=0x7fb33400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.811+0000 7fb34de00640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb334005150 0x7fb33400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.811+0000 7fb34de00640 1 -- 172.21.15.19:0/3967758954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb334003520 con 0x7fb3340039b0 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.812+0000 7fb34de00640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3340039b0 0x7fb334003e20 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fb340009a90 tx=0x7fb334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74f91d514a2df61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.812+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340036070 con 0x7fb3340039b0 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.812+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb340031400 con 0x7fb3340039b0 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.812+0000 7fb34d400640 1 -- 172.21.15.19:0/3967758954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb340009690 con 0x7fb3340039b0 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.813+0000 7fb355000640 1 -- 172.21.15.19:0/3967758954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3340039b0 msgr2=0x7fb334003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.813+0000 7fb355000640 1 --2- 172.21.15.19:0/3967758954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3340039b0 0x7fb334003e20 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fb340009a90 tx=0x7fb334005830 comp rx=0 tx=0).stop 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.813+0000 7fb34e800640 1 -- 172.21.15.19:0/3967758954 reap_dead start 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.813+0000 7fb355000640 1 -- 172.21.15.19:0/3967758954 shutdown_connections 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.813+0000 7fb355000640 1 -- 172.21.15.19:0/3967758954 >> 172.21.15.19:0/3967758954 conn(0x7fb3500d1ae0 msgr2=0x7fb3500d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.813+0000 7fb355000640 1 -- 172.21.15.19:0/3967758954 shutdown_connections 2024-04-25T11:43:04.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.813+0000 7fb355000640 1 -- 172.21.15.19:0/3967758954 wait complete. 2024-04-25T11:43:04.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.815+0000 7fb355000640 1 Processor -- start 2024-04-25T11:43:04.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.816+0000 7fb355000640 1 -- start start 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.817+0000 7fb355000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3340039b0 0x7fb35015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.817+0000 7fb355000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35013f2e0 0x7fb35013d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.817+0000 7fb355000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb35013f6f0 0x7fb35013d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.817+0000 7fb355000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3501447d0 con 0x7fb35013f2e0 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.817+0000 7fb355000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb35015edf0 con 0x7fb35013f6f0 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.817+0000 7fb355000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb350162b00 con 0x7fb3340039b0 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35013f2e0 0x7fb35013d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35013f2e0 0x7fb35013d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37548/0 (socket says 172.21.15.19:37548) 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 -- 172.21.15.19:0/2423288579 learned_addr learned my addr 172.21.15.19:0/2423288579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb34de00640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3340039b0 0x7fb35015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb34e800640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb35013f6f0 0x7fb35013d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 -- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3340039b0 msgr2=0x7fb35015b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3340039b0 0x7fb35015b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 -- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb35013f6f0 msgr2=0x7fb35013d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb35013f6f0 0x7fb35013d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb345400640 1 -- 172.21.15.19:0/2423288579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500cf000 con 0x7fb35013f2e0 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.818+0000 7fb34de00640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3340039b0 0x7fb35015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.819+0000 7fb345400640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35013f2e0 0x7fb35013d1d0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d7a0 tx=0x7fb33800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.819+0000 7fb346a00640 1 -- 172.21.15.19:0/2423288579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb338019070 con 0x7fb35013f2e0 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.819+0000 7fb346a00640 1 -- 172.21.15.19:0/2423288579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3380072d0 con 0x7fb35013f2e0 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.819+0000 7fb346a00640 1 -- 172.21.15.19:0/2423288579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3380047e0 con 0x7fb35013f2e0 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.819+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3500cf2e0 con 0x7fb35013f2e0 2024-04-25T11:43:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.819+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35013a820 con 0x7fb35013f2e0 2024-04-25T11:43:04.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.820+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35013a610 con 0x7fb35013f2e0 2024-04-25T11:43:04.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.822+0000 7fb346a00640 1 -- 172.21.15.19:0/2423288579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb338004980 con 0x7fb35013f2e0 2024-04-25T11:43:04.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.822+0000 7fb346a00640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb32c0744e0 0x7fb32c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.823+0000 7fb346a00640 1 -- 172.21.15.19:0/2423288579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fb338094b10 con 0x7fb35013f2e0 2024-04-25T11:43:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.823+0000 7fb34de00640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb32c0744e0 0x7fb32c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.825+0000 7fb34de00640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb32c0744e0 0x7fb32c0769f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb340033000 tx=0x7fb340008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:04.827+0000 7fb346a00640 1 -- 172.21.15.19:0/2423288579 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb33805f2d0 con 0x7fb35013f2e0 2024-04-25T11:43:05.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.072+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb350050740 con 0x7fb35013f2e0 2024-04-25T11:43:05.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.073+0000 7fb346a00640 1 -- 172.21.15.19:0/2423288579 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 372 v372) ==== 95+0+5195 (secure 0 0 0) 0x7fb3380672c0 con 0x7fb35013f2e0 2024-04-25T11:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":372,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":372,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:59.523278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":12,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:43:05.060 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 372 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb32c0744e0 msgr2=0x7fb32c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb32c0744e0 0x7fb32c0769f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb340033000 tx=0x7fb340008030 comp rx=0 tx=0).stop 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35013f2e0 msgr2=0x7fb35013d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35013f2e0 0x7fb35013d1d0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d7a0 tx=0x7fb33800dc60 comp rx=0 tx=0).stop 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 shutdown_connections 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb32c0744e0 0x7fb32c0769f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb35013f6f0 0x7fb35013d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb35013f2e0 0x7fb35013d1d0 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 --2- 172.21.15.19:0/2423288579 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3340039b0 0x7fb35015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 >> 172.21.15.19:0/2423288579 conn(0x7fb3500d1ae0 msgr2=0x7fb350160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 shutdown_connections 2024-04-25T11:43:05.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.076+0000 7fb355000640 1 -- 172.21.15.19:0/2423288579 wait complete. 2024-04-25T11:43:05.088 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.364+0000 7fa364c00640 1 Processor -- start 2024-04-25T11:43:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.364+0000 7fa364c00640 1 -- start start 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.366+0000 7fa364c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa3600cf2f0 0x7fa3600cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa364c00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa36006c5c0 0x7fa36006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa364c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa360071180 con 0x7fa360168010 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa364c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa360070cf0 con 0x7fa3600cf2f0 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa364c00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fa3600700e0 con 0x7fa36006c5c0 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa35f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa3600cf2f0 0x7fa3600cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa35f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa3600cf2f0 0x7fa3600cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37560/0 (socket says 172.21.15.19:37560) 2024-04-25T11:43:05.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa35f400640 1 -- 172.21.15.19:0/278174571 learned_addr learned my addr 172.21.15.19:0/278174571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa35f400640 1 -- 172.21.15.19:0/278174571 >> v2:172.21.15.169:3300/0 conn(0x7fa36006c5c0 msgr2=0x7fa36006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa35f400640 1 --2- 172.21.15.19:0/278174571 >> v2:172.21.15.169:3300/0 conn(0x7fa36006c5c0 0x7fa36006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.367+0000 7fa35f400640 1 -- 172.21.15.19:0/278174571 >> v1:172.21.15.19:6789/0 conn(0x7fa360168010 legacy=0x7fa360168420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.368+0000 7fa35f400640 1 -- 172.21.15.19:0/278174571 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3600703c0 con 0x7fa3600cf2f0 2024-04-25T11:43:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.368+0000 7fa35f400640 1 --2- 172.21.15.19:0/278174571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3600cf2f0 0x7fa3600cf700 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fa3600b1390 tx=0x7fa35002a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ccda3768e435e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.369+0000 7fa35e000640 1 -- 172.21.15.19:0/278174571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350037070 con 0x7fa3600cf2f0 2024-04-25T11:43:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.369+0000 7fa35e000640 1 -- 172.21.15.19:0/278174571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa350032440 con 0x7fa3600cf2f0 2024-04-25T11:43:05.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.369+0000 7fa35e000640 1 -- 172.21.15.19:0/278174571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350032810 con 0x7fa3600cf2f0 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.369+0000 7fa364c00640 1 -- 172.21.15.19:0/278174571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3600cf2f0 msgr2=0x7fa3600cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.369+0000 7fa364c00640 1 --2- 172.21.15.19:0/278174571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3600cf2f0 0x7fa3600cf700 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fa3600b1390 tx=0x7fa35002a280 comp rx=0 tx=0).stop 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.370+0000 7fa364c00640 1 -- 172.21.15.19:0/278174571 shutdown_connections 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.370+0000 7fa364c00640 1 --2- 172.21.15.19:0/278174571 >> v2:172.21.15.169:3300/0 conn(0x7fa36006c5c0 0x7fa36006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.370+0000 7fa364c00640 1 --2- 172.21.15.19:0/278174571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3600cf2f0 0x7fa3600cf700 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.370+0000 7fa364c00640 1 -- 172.21.15.19:0/278174571 >> 172.21.15.19:0/278174571 conn(0x7fa3600d1720 msgr2=0x7fa3600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.370+0000 7fa364c00640 1 -- 172.21.15.19:0/278174571 shutdown_connections 2024-04-25T11:43:05.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.370+0000 7fa364c00640 1 -- 172.21.15.19:0/278174571 wait complete. 2024-04-25T11:43:05.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.372+0000 7fa364c00640 1 Processor -- start 2024-04-25T11:43:05.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.372+0000 7fa364c00640 1 -- start start 2024-04-25T11:43:05.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa364c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36006c5c0 0x7fa3601447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa364c00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3600cf2f0 0x7fa360163690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa364c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa360168010 0x7fa3601ede50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa364c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa360168d00 con 0x7fa360168010 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa364c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa360070980 con 0x7fa36006c5c0 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa364c00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fa360168e80 con 0x7fa3600cf2f0 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa360168010 0x7fa3601ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa360168010 0x7fa3601ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37562/0 (socket says 172.21.15.19:37562) 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 -- 172.21.15.19:0/380572955 learned_addr learned my addr 172.21.15.19:0/380572955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35f400640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36006c5c0 0x7fa3601447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35ea00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3600cf2f0 0x7fa360163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 -- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3600cf2f0 msgr2=0x7fa360163690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3600cf2f0 0x7fa360163690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 -- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36006c5c0 msgr2=0x7fa3601447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36006c5c0 0x7fa3601447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.374+0000 7fa35fe00640 1 -- 172.21.15.19:0/380572955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3601642e0 con 0x7fa360168010 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.375+0000 7fa35fe00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa360168010 0x7fa3601ede50 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fa35800ba10 tx=0x7fa35800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.375+0000 7fa357400640 1 -- 172.21.15.19:0/380572955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358019070 con 0x7fa360168010 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.375+0000 7fa357400640 1 -- 172.21.15.19:0/380572955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa35800cb60 con 0x7fa360168010 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.375+0000 7fa357400640 1 -- 172.21.15.19:0/380572955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358014440 con 0x7fa360168010 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.375+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3601645c0 con 0x7fa360168010 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.375+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa360164b80 con 0x7fa360168010 2024-04-25T11:43:05.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.376+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa360050740 con 0x7fa360168010 2024-04-25T11:43:05.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.378+0000 7fa357400640 1 -- 172.21.15.19:0/380572955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa35801c070 con 0x7fa360168010 2024-04-25T11:43:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.379+0000 7fa357400640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3400744b0 0x7fa3400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.380+0000 7fa357400640 1 -- 172.21.15.19:0/380572955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fa358094fe0 con 0x7fa360168010 2024-04-25T11:43:05.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.380+0000 7fa35f400640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3400744b0 0x7fa3400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.381+0000 7fa35f400640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3400744b0 0x7fa3400769c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa350034000 tx=0x7fa35002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.385+0000 7fa357400640 1 -- 172.21.15.19:0/380572955 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa35805f690 con 0x7fa360168010 2024-04-25T11:43:05.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.583+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa360164070 con 0x7fa360168010 2024-04-25T11:43:05.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.586+0000 7fa357400640 1 -- 172.21.15.19:0/380572955 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16750 (secure 0 0 0) 0x7fa35805ef90 con 0x7fa360168010 2024-04-25T11:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":144,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:42:44.471720+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/2610059988":"2024-04-26T11:42:44.471606+0000","172.21.15.169:6834/2610059988":"2024-04-26T11:42:44.471606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3400744b0 msgr2=0x7fa3400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3400744b0 0x7fa3400769c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa350034000 tx=0x7fa35002e000 comp rx=0 tx=0).stop 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa360168010 msgr2=0x7fa3601ede50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa360168010 0x7fa3601ede50 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7fa35800ba10 tx=0x7fa35800bed0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 shutdown_connections 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fa3400744b0 0x7fa3400769c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa360168010 0x7fa3601ede50 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa3600cf2f0 0x7fa360163690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 --2- 172.21.15.19:0/380572955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa36006c5c0 0x7fa3601447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 >> 172.21.15.19:0/380572955 conn(0x7fa3600d1720 msgr2=0x7fa3601674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 shutdown_connections 2024-04-25T11:43:05.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.589+0000 7fa364c00640 1 -- 172.21.15.19:0/380572955 wait complete. 2024-04-25T11:43:05.599 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:43:05.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.891+0000 7f3d51a00640 1 Processor -- start 2024-04-25T11:43:05.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.891+0000 7f3d51a00640 1 -- start start 2024-04-25T11:43:05.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.893+0000 7f3d51a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d4c1616a0 0x7f3d4c15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.893+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c1643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.893+0000 7f3d51a00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d4c164c80 con 0x7f3d4c06e030 2024-04-25T11:43:05.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.893+0000 7f3d51a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3d4c160980 con 0x7f3d4c1616a0 2024-04-25T11:43:05.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.894+0000 7f3d51a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4c160bc0 con 0x7f3d4c163fc0 2024-04-25T11:43:05.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.894+0000 7f3d4aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d4c1616a0 0x7f3d4c15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.894+0000 7f3d4aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d4c1616a0 0x7f3d4c15ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37574/0 (socket says 172.21.15.19:37574) 2024-04-25T11:43:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.894+0000 7f3d4aa00640 1 -- 172.21.15.19:0/3887838931 learned_addr learned my addr 172.21.15.19:0/3887838931 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.894+0000 7f3d4b400640 1 --2- 172.21.15.19:0/3887838931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c1643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.895+0000 7f3d4aa00640 1 -- 172.21.15.19:0/3887838931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 msgr2=0x7f3d4c1643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.895+0000 7f3d4aa00640 1 --2- 172.21.15.19:0/3887838931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c1643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.895+0000 7f3d4aa00640 1 -- 172.21.15.19:0/3887838931 >> v1:172.21.15.169:6789/0 conn(0x7f3d4c06e030 legacy=0x7f3d4c06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.895+0000 7f3d4aa00640 1 -- 172.21.15.19:0/3887838931 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c160ea0 con 0x7f3d4c1616a0 2024-04-25T11:43:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.896+0000 7f3d4aa00640 1 --2- 172.21.15.19:0/3887838931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c15ded0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009870 tx=0x7f3d3c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5d18ea2caaa803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.896+0000 7f3d49600640 1 -- 172.21.15.19:0/3887838931 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c037070 con 0x7f3d4c1616a0 2024-04-25T11:43:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.896+0000 7f3d49600640 1 -- 172.21.15.19:0/3887838931 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3c032440 con 0x7f3d4c1616a0 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.896+0000 7f3d49600640 1 -- 172.21.15.19:0/3887838931 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c032810 con 0x7f3d4c1616a0 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 -- 172.21.15.19:0/3887838931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 msgr2=0x7f3d4c15ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3887838931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c15ded0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009870 tx=0x7f3d3c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 -- 172.21.15.19:0/3887838931 shutdown_connections 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3887838931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c1643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3887838931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c15ded0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 -- 172.21.15.19:0/3887838931 >> 172.21.15.19:0/3887838931 conn(0x7f3d4c0d27c0 msgr2=0x7f3d4c0d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 -- 172.21.15.19:0/3887838931 shutdown_connections 2024-04-25T11:43:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.897+0000 7f3d51a00640 1 -- 172.21.15.19:0/3887838931 wait complete. 2024-04-25T11:43:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.899+0000 7f3d51a00640 1 Processor -- start 2024-04-25T11:43:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.900+0000 7f3d51a00640 1 -- start start 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d4c06e030 0x7f3d4c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d51a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c0b77c0 con 0x7f3d4c1616a0 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d51a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c164910 con 0x7f3d4c06e030 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d51a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4c0b7940 con 0x7f3d4c163fc0 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d4a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d4a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c14a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37576/0 (socket says 172.21.15.19:37576) 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.901+0000 7f3d4a000640 1 -- 172.21.15.19:0/3889976156 learned_addr learned my addr 172.21.15.19:0/3889976156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4aa00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d4c06e030 0x7f3d4c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4b400640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4a000640 1 -- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 msgr2=0x7f3d4c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4a000640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c15ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4a000640 1 -- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d4c06e030 msgr2=0x7f3d4c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4a000640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d4c06e030 0x7f3d4c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4a000640 1 -- 172.21.15.19:0/3889976156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c0ccaa0 con 0x7f3d4c1616a0 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.902+0000 7f3d4a000640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c14a590 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3c0 tx=0x7f3d3400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.903+0000 7f3d42a00640 1 -- 172.21.15.19:0/3889976156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d340052d0 con 0x7f3d4c1616a0 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.903+0000 7f3d42a00640 1 -- 172.21.15.19:0/3889976156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d34005490 con 0x7f3d4c1616a0 2024-04-25T11:43:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.903+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4c1fa320 con 0x7f3d4c1616a0 2024-04-25T11:43:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.903+0000 7f3d42a00640 1 -- 172.21.15.19:0/3889976156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34007260 con 0x7f3d4c1616a0 2024-04-25T11:43:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.903+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4c1fa600 con 0x7f3d4c1616a0 2024-04-25T11:43:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.904+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4c050740 con 0x7f3d4c1616a0 2024-04-25T11:43:05.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.905+0000 7f3d42a00640 1 -- 172.21.15.19:0/3889976156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3d34007400 con 0x7f3d4c1616a0 2024-04-25T11:43:05.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.907+0000 7f3d42a00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d280744b0 0x7f3d280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.907+0000 7f3d42a00640 1 -- 172.21.15.19:0/3889976156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f3d34094740 con 0x7f3d4c1616a0 2024-04-25T11:43:05.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.908+0000 7f3d4aa00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d280744b0 0x7f3d280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.909+0000 7f3d4aa00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d280744b0 0x7f3d280769c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c034000 tx=0x7f3d3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:05.912+0000 7f3d42a00640 1 -- 172.21.15.19:0/3889976156 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d3405ef00 con 0x7f3d4c1616a0 2024-04-25T11:43:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.157+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d4c0d3980 con 0x7f3d4c1616a0 2024-04-25T11:43:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.158+0000 7f3d42a00640 1 -- 172.21.15.19:0/3889976156 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 372 v372) ==== 95+0+5195 (secure 0 0 0) 0x7f3d3405e800 con 0x7f3d4c1616a0 2024-04-25T11:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":372,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":372,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:59.523278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":12,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:43:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 372 2024-04-25T11:43:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.161+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d280744b0 msgr2=0x7f3d280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.161+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d280744b0 0x7f3d280769c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c034000 tx=0x7f3d3c02e000 comp rx=0 tx=0).stop 2024-04-25T11:43:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.161+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 msgr2=0x7f3d4c14a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.161+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c14a590 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c3c0 tx=0x7f3d3400c7c0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 shutdown_connections 2024-04-25T11:43:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f3d280744b0 0x7f3d280769c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3d4c163fc0 0x7f3d4c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1616a0 0x7f3d4c14a590 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 --2- 172.21.15.19:0/3889976156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d4c06e030 0x7f3d4c0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 >> 172.21.15.19:0/3889976156 conn(0x7f3d4c0d27c0 msgr2=0x7f3d4c0d5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 shutdown_connections 2024-04-25T11:43:06.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.162+0000 7f3d51a00640 1 -- 172.21.15.19:0/3889976156 wait complete. 2024-04-25T11:43:06.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:06.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.472+0000 7fda4ac00640 1 Processor -- start 2024-04-25T11:43:06.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.473+0000 7fda4ac00640 1 -- start start 2024-04-25T11:43:06.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4ac00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda440cf2f0 0x7fda440b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4415a810 0x7fda440b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4ac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda441608d0 0x7fda44160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4ac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fda4406f870 con 0x7fda441608d0 2024-04-25T11:43:06.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4ac00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fda44161220 con 0x7fda440cf2f0 2024-04-25T11:43:06.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda440b4e90 con 0x7fda4415a810 2024-04-25T11:43:06.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda441608d0 0x7fda44160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fda441608d0 0x7fda44160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37586/0 (socket says 172.21.15.19:37586) 2024-04-25T11:43:06.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.475+0000 7fda4a200640 1 -- 172.21.15.19:0/291606924 learned_addr learned my addr 172.21.15.19:0/291606924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.476+0000 7fda49800640 1 --2- 172.21.15.19:0/291606924 >> v2:172.21.15.169:3300/0 conn(0x7fda440cf2f0 0x7fda440b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.476+0000 7fda48e00640 1 --2- 172.21.15.19:0/291606924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4415a810 0x7fda440b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.476+0000 7fda4a200640 1 -- 172.21.15.19:0/291606924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4415a810 msgr2=0x7fda440b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.476+0000 7fda4a200640 1 --2- 172.21.15.19:0/291606924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4415a810 0x7fda440b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.476+0000 7fda4a200640 1 -- 172.21.15.19:0/291606924 >> v2:172.21.15.169:3300/0 conn(0x7fda440cf2f0 msgr2=0x7fda440b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.476+0000 7fda4a200640 1 --2- 172.21.15.19:0/291606924 >> v2:172.21.15.169:3300/0 conn(0x7fda440cf2f0 0x7fda440b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.476+0000 7fda4a200640 1 -- 172.21.15.19:0/291606924 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda440b74d0 con 0x7fda441608d0 2024-04-25T11:43:06.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.477+0000 7fda4a200640 1 --2- 172.21.15.19:0/291606924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda44160ce0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fda440d26d0 tx=0x7fda3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8197f9843e6ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.477+0000 7fda43e00640 1 -- 172.21.15.19:0/291606924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c036070 con 0x7fda441608d0 2024-04-25T11:43:06.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.477+0000 7fda43e00640 1 -- 172.21.15.19:0/291606924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3c031600 con 0x7fda441608d0 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.477+0000 7fda43e00640 1 -- 172.21.15.19:0/291606924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0319d0 con 0x7fda441608d0 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 -- 172.21.15.19:0/291606924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 msgr2=0x7fda44160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 --2- 172.21.15.19:0/291606924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda44160ce0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7fda440d26d0 tx=0x7fda3c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 -- 172.21.15.19:0/291606924 shutdown_connections 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 --2- 172.21.15.19:0/291606924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda44160ce0 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 --2- 172.21.15.19:0/291606924 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda4415a810 0x7fda440b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 --2- 172.21.15.19:0/291606924 >> v2:172.21.15.169:3300/0 conn(0x7fda440cf2f0 0x7fda440b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 -- 172.21.15.19:0/291606924 >> 172.21.15.19:0/291606924 conn(0x7fda440d1720 msgr2=0x7fda440d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.478+0000 7fda4ac00640 1 -- 172.21.15.19:0/291606924 shutdown_connections 2024-04-25T11:43:06.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.479+0000 7fda4ac00640 1 -- 172.21.15.19:0/291606924 wait complete. 2024-04-25T11:43:06.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.481+0000 7fda4ac00640 1 Processor -- start 2024-04-25T11:43:06.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.481+0000 7fda4ac00640 1 -- start start 2024-04-25T11:43:06.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.482+0000 7fda4ac00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440cf2f0 0x7fda44156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.482+0000 7fda4ac00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4415a810 0x7fda4415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.482+0000 7fda4ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda4415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda4406f500 con 0x7fda441608d0 2024-04-25T11:43:06.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4ac00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda4406f220 con 0x7fda4415a810 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4ac00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fda4415ac20 con 0x7fda440cf2f0 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda4415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda4415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37598/0 (socket says 172.21.15.19:37598) 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 -- 172.21.15.19:0/493256895 learned_addr learned my addr 172.21.15.19:0/493256895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda49800640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440cf2f0 0x7fda44156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda48e00640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4415a810 0x7fda4415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 -- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440cf2f0 msgr2=0x7fda44156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440cf2f0 0x7fda44156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 -- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4415a810 msgr2=0x7fda4415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4415a810 0x7fda4415eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda4a200640 1 -- 172.21.15.19:0/493256895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda441faa40 con 0x7fda441608d0 2024-04-25T11:43:06.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.483+0000 7fda49800640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440cf2f0 0x7fda44156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:06.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.484+0000 7fda4a200640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda4415f080 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02ac90 tx=0x7fda3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.484+0000 7fda41600640 1 -- 172.21.15.19:0/493256895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c03f020 con 0x7fda441608d0 2024-04-25T11:43:06.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.484+0000 7fda41600640 1 -- 172.21.15.19:0/493256895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3c037040 con 0x7fda441608d0 2024-04-25T11:43:06.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.484+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda441502e0 con 0x7fda441608d0 2024-04-25T11:43:06.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.484+0000 7fda41600640 1 -- 172.21.15.19:0/493256895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c036070 con 0x7fda441608d0 2024-04-25T11:43:06.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.484+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441f9f60 con 0x7fda441608d0 2024-04-25T11:43:06.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.485+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda44151b60 con 0x7fda441608d0 2024-04-25T11:43:06.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.487+0000 7fda41600640 1 -- 172.21.15.19:0/493256895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda3c042020 con 0x7fda441608d0 2024-04-25T11:43:06.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.488+0000 7fda41600640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda24074530 0x7fda24076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.488+0000 7fda41600640 1 -- 172.21.15.19:0/493256895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7fda3c0b59f0 con 0x7fda441608d0 2024-04-25T11:43:06.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.488+0000 7fda49800640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda24074530 0x7fda24076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.490+0000 7fda49800640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda24074530 0x7fda24076a40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fda38006fd0 tx=0x7fda38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.493+0000 7fda41600640 1 -- 172.21.15.19:0/493256895 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda3c049020 con 0x7fda441608d0 2024-04-25T11:43:06.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.738+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda44050740 con 0x7fda441608d0 2024-04-25T11:43:06.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.739+0000 7fda41600640 1 -- 172.21.15.19:0/493256895 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 372 v372) ==== 95+0+5195 (secure 0 0 0) 0x7fda441502e0 con 0x7fda441608d0 2024-04-25T11:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":372,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366}],"filesystems":[{"mdsmap":{"epoch":372,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:42:29.193619+0000","modified":"2024-04-25T11:42:59.523278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":144,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8020":{"gid":8020,"name":"c","rank":0,"incarnation":367,"state":"up:active","state_seq":12,"addr":"172.21.15.19:6834/3539191189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3539191189},{"type":"v1","addr":"172.21.15.19:6834","nonce":3539191189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8020,"qdb_cluster":[8020]},"id":7}]} 2024-04-25T11:43:06.726 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 372 2024-04-25T11:43:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda24074530 msgr2=0x7fda24076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda24074530 0x7fda24076a40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fda38006fd0 tx=0x7fda38008040 comp rx=0 tx=0).stop 2024-04-25T11:43:06.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 msgr2=0x7fda4415f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda4415f080 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02ac90 tx=0x7fda3c004a40 comp rx=0 tx=0).stop 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 shutdown_connections 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fda24074530 0x7fda24076a40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda4415a810 0x7fda4415eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda441608d0 0x7fda4415f080 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 --2- 172.21.15.19:0/493256895 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440cf2f0 0x7fda44156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.742+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 >> 172.21.15.19:0/493256895 conn(0x7fda440d1720 msgr2=0x7fda440703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.743+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 shutdown_connections 2024-04-25T11:43:06.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:06.743+0000 7fda4ac00640 1 -- 172.21.15.19:0/493256895 wait complete. 2024-04-25T11:43:06.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:07.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.045+0000 7f2af9000640 1 Processor -- start 2024-04-25T11:43:07.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.045+0000 7f2af9000640 1 -- start start 2024-04-25T11:43:07.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af40b4ea0 0x7f2af40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af4144370 0x7f2af415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af9000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af40d1230 con 0x7f2af406c5c0 2024-04-25T11:43:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af9000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2af40d06b0 con 0x7f2af40b4ea0 2024-04-25T11:43:07.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2af40d0990 con 0x7f2af4144370 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af40b4ea0 0x7f2af40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2af40b4ea0 0x7f2af40cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37608/0 (socket says 172.21.15.19:37608) 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af2a00640 1 -- 172.21.15.19:0/1239163454 learned_addr learned my addr 172.21.15.19:0/1239163454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af3400640 1 --1- 172.21.15.19:0/1239163454 >> v1:172.21.15.19:6789/0 conn(0x7f2af406c5c0 0x7f2af40b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35058/0 (socket says 172.21.15.19:35058) 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.047+0000 7f2af3e00640 1 --2- 172.21.15.19:0/1239163454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af4144370 0x7f2af415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.048+0000 7f2af2a00640 1 -- 172.21.15.19:0/1239163454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af4144370 msgr2=0x7f2af415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.048+0000 7f2af2a00640 1 --2- 172.21.15.19:0/1239163454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af4144370 0x7f2af415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.048+0000 7f2af2a00640 1 -- 172.21.15.19:0/1239163454 >> v1:172.21.15.19:6789/0 conn(0x7f2af406c5c0 legacy=0x7f2af40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.048+0000 7f2af2a00640 1 -- 172.21.15.19:0/1239163454 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af40d0c70 con 0x7f2af40b4ea0 2024-04-25T11:43:07.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.048+0000 7f2af2000640 1 -- 172.21.15.19:0/1239163454 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 64204365 0 0) 0x7f2af40d1230 con 0x7f2af406c5c0 2024-04-25T11:43:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.049+0000 7f2af2a00640 1 --2- 172.21.15.19:0/1239163454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af40b4ea0 0x7f2af40cfa70 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009e50 tx=0x7f2ae402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9d12746fdc66fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.049+0000 7f2af2000640 1 -- 172.21.15.19:0/1239163454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4036070 con 0x7f2af40b4ea0 2024-04-25T11:43:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.049+0000 7f2af2000640 1 -- 172.21.15.19:0/1239163454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae4031600 con 0x7f2af40b4ea0 2024-04-25T11:43:07.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.049+0000 7f2af2000640 1 -- 172.21.15.19:0/1239163454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40319d0 con 0x7f2af40b4ea0 2024-04-25T11:43:07.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 -- 172.21.15.19:0/1239163454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af40b4ea0 msgr2=0x7f2af40cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 --2- 172.21.15.19:0/1239163454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af40b4ea0 0x7f2af40cfa70 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009e50 tx=0x7f2ae402a710 comp rx=0 tx=0).stop 2024-04-25T11:43:07.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 -- 172.21.15.19:0/1239163454 shutdown_connections 2024-04-25T11:43:07.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 --2- 172.21.15.19:0/1239163454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af4144370 0x7f2af415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 --2- 172.21.15.19:0/1239163454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af40b4ea0 0x7f2af40cfa70 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 -- 172.21.15.19:0/1239163454 >> 172.21.15.19:0/1239163454 conn(0x7f2af40d1720 msgr2=0x7f2af40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 -- 172.21.15.19:0/1239163454 shutdown_connections 2024-04-25T11:43:07.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.050+0000 7f2af9000640 1 -- 172.21.15.19:0/1239163454 wait complete. 2024-04-25T11:43:07.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.052+0000 7f2af9000640 1 Processor -- start 2024-04-25T11:43:07.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.052+0000 7f2af9000640 1 -- start start 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af9000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2af406c5c0 0x7f2af413a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af4144370 0x7f2af415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af9000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af413faf0 0x7f2af413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af9000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2af40d0450 con 0x7f2af4144370 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af9000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2af40d02d0 con 0x7f2af406c5c0 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af9000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f2af4147760 con 0x7f2af413faf0 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af4144370 0x7f2af415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af4144370 0x7f2af415b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37610/0 (socket says 172.21.15.19:37610) 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af2a00640 1 -- 172.21.15.19:0/2338848173 learned_addr learned my addr 172.21.15.19:0/2338848173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af2a00640 1 -- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af413faf0 msgr2=0x7f2af413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af2a00640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af413faf0 0x7f2af413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af2a00640 1 -- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2af406c5c0 msgr2=0x7f2af413a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.054+0000 7f2af2a00640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2af406c5c0 0x7f2af413a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.055+0000 7f2af2a00640 1 -- 172.21.15.19:0/2338848173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af413dbd0 con 0x7f2af4144370 2024-04-25T11:43:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.055+0000 7f2af2a00640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af4144370 0x7f2af415b490 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4034040 tx=0x7f2ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.055+0000 7f2aeb400640 1 -- 172.21.15.19:0/2338848173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4036050 con 0x7f2af4144370 2024-04-25T11:43:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.055+0000 7f2aeb400640 1 -- 172.21.15.19:0/2338848173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae4042070 con 0x7f2af4144370 2024-04-25T11:43:07.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.055+0000 7f2aeb400640 1 -- 172.21.15.19:0/2338848173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4009510 con 0x7f2af4144370 2024-04-25T11:43:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.055+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af4140440 con 0x7f2af4144370 2024-04-25T11:43:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.055+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af413e120 con 0x7f2af4144370 2024-04-25T11:43:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.056+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af413afc0 con 0x7f2af4144370 2024-04-25T11:43:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.058+0000 7f2aeb400640 1 -- 172.21.15.19:0/2338848173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ae4007c50 con 0x7f2af4144370 2024-04-25T11:43:07.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.059+0000 7f2aeb400640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744e0 0x7f2ad00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.059+0000 7f2aeb400640 1 -- 172.21.15.19:0/2338848173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5795+0+0 (secure 0 0 0) 0x7f2ae40b6180 con 0x7f2af4144370 2024-04-25T11:43:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.059+0000 7f2af3400640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744e0 0x7f2ad00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.060+0000 7f2af3400640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744e0 0x7f2ad00769f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2adc007fd0 tx=0x7f2adc007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.063+0000 7f2aeb400640 1 -- 172.21.15.19:0/2338848173 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2af4140440 con 0x7f2af4144370 2024-04-25T11:43:07.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.265+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2af4050740 con 0x7f2af4144370 2024-04-25T11:43:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.276+0000 7f2aeb400640 1 -- 172.21.15.19:0/2338848173 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16894 (secure 0 0 0) 0x7f2ae40808a0 con 0x7f2af4144370 2024-04-25T11:43:07.263 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:07.263 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":145,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:43:07.252370+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:42:27.399161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T11:42:28.185279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.0","mappings":[{"from":5,"to":3}]},{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.18","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6835/2610059988":"2024-04-26T11:42:44.471606+0000","172.21.15.169:6834/2610059988":"2024-04-26T11:42:44.471606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:07.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744e0 msgr2=0x7f2ad00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744e0 0x7f2ad00769f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2adc007fd0 tx=0x7f2adc007960 comp rx=0 tx=0).stop 2024-04-25T11:43:07.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af4144370 msgr2=0x7f2af415b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af4144370 0x7f2af415b490 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4034040 tx=0x7f2ae4004a40 comp rx=0 tx=0).stop 2024-04-25T11:43:07.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 shutdown_connections 2024-04-25T11:43:07.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f2ad00744e0 0x7f2ad00769f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2af413faf0 0x7f2af413ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2af4144370 0x7f2af415b490 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 --2- 172.21.15.19:0/2338848173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2af406c5c0 0x7f2af413a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 >> 172.21.15.19:0/2338848173 conn(0x7f2af40d1720 msgr2=0x7f2af40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.279+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 shutdown_connections 2024-04-25T11:43:07.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.280+0000 7f2af9000640 1 -- 172.21.15.19:0/2338848173 wait complete. 2024-04-25T11:43:07.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:43:07.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:43:07.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:43:07.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.571+0000 7f15dd200640 1 Processor -- start 2024-04-25T11:43:07.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.571+0000 7f15dd200640 1 -- start start 2024-04-25T11:43:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.573+0000 7f15dd200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f15d80b0ae0 0x7f15d80b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.573+0000 7f15dd200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15d806fe40 0x7f15d8070250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.573+0000 7f15dd200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d8160aa0 con 0x7f15d81458f0 2024-04-25T11:43:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.573+0000 7f15dd200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f15d8160d80 con 0x7f15d806fe40 2024-04-25T11:43:07.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.573+0000 7f15dd200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f15d8161060 con 0x7f15d80b0ae0 2024-04-25T11:43:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.573+0000 7f15d7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15d806fe40 0x7f15d8070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.574+0000 7f15d7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15d806fe40 0x7f15d8070250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37626/0 (socket says 172.21.15.19:37626) 2024-04-25T11:43:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.574+0000 7f15d7e00640 1 -- 172.21.15.19:0/3239671974 learned_addr learned my addr 172.21.15.19:0/3239671974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.574+0000 7f15d7e00640 1 -- 172.21.15.19:0/3239671974 >> v2:172.21.15.169:3300/0 conn(0x7f15d80b0ae0 msgr2=0x7f15d80b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.574+0000 7f15d7e00640 1 --2- 172.21.15.19:0/3239671974 >> v2:172.21.15.169:3300/0 conn(0x7f15d80b0ae0 0x7f15d80b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.574+0000 7f15d7e00640 1 -- 172.21.15.19:0/3239671974 >> v1:172.21.15.169:6789/0 conn(0x7f15d81458f0 legacy=0x7f15d8145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.574+0000 7f15d7e00640 1 -- 172.21.15.19:0/3239671974 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d8161340 con 0x7f15d806fe40 2024-04-25T11:43:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.575+0000 7f15d7e00640 1 --2- 172.21.15.19:0/3239671974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d806fe40 0x7f15d8070250 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f15d80723b0 tx=0x7f15d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697a3725878dd5c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.575+0000 7f15d6000640 1 -- 172.21.15.19:0/3239671974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0037070 con 0x7f15d806fe40 2024-04-25T11:43:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.575+0000 7f15d6000640 1 -- 172.21.15.19:0/3239671974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d0032440 con 0x7f15d806fe40 2024-04-25T11:43:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.575+0000 7f15d6000640 1 -- 172.21.15.19:0/3239671974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0032810 con 0x7f15d806fe40 2024-04-25T11:43:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 -- 172.21.15.19:0/3239671974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d806fe40 msgr2=0x7f15d8070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 --2- 172.21.15.19:0/3239671974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d806fe40 0x7f15d8070250 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f15d80723b0 tx=0x7f15d002a230 comp rx=0 tx=0).stop 2024-04-25T11:43:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 -- 172.21.15.19:0/3239671974 shutdown_connections 2024-04-25T11:43:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 --2- 172.21.15.19:0/3239671974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d806fe40 0x7f15d8070250 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 --2- 172.21.15.19:0/3239671974 >> v2:172.21.15.169:3300/0 conn(0x7f15d80b0ae0 0x7f15d80b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 -- 172.21.15.19:0/3239671974 >> 172.21.15.19:0/3239671974 conn(0x7f15d80d1b40 msgr2=0x7f15d80d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 -- 172.21.15.19:0/3239671974 shutdown_connections 2024-04-25T11:43:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.576+0000 7f15dd200640 1 -- 172.21.15.19:0/3239671974 wait complete. 2024-04-25T11:43:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.578+0000 7f15dd200640 1 Processor -- start 2024-04-25T11:43:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.579+0000 7f15dd200640 1 -- start start 2024-04-25T11:43:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15dd200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d806fe40 0x7f15d8163450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15dd200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d80b0ae0 0x7f15d813f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15dd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81458f0 0x7f15d815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15dd200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80b0ef0 con 0x7f15d81458f0 2024-04-25T11:43:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15dd200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8146540 con 0x7f15d80b0ae0 2024-04-25T11:43:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15dd200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f15d8162a70 con 0x7f15d806fe40 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81458f0 0x7f15d815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81458f0 0x7f15d815f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37628/0 (socket says 172.21.15.19:37628) 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.580+0000 7f15d7e00640 1 -- 172.21.15.19:0/3598423963 learned_addr learned my addr 172.21.15.19:0/3598423963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7400640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d806fe40 0x7f15d8163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d6a00640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d80b0ae0 0x7f15d813f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7e00640 1 -- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d806fe40 msgr2=0x7f15d8163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7e00640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d806fe40 0x7f15d8163450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7e00640 1 -- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d80b0ae0 msgr2=0x7f15d813f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7e00640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d80b0ae0 0x7f15d813f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7e00640 1 -- 172.21.15.19:0/3598423963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d8140170 con 0x7f15d81458f0 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7400640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d806fe40 0x7f15d8163450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15d7e00640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81458f0 0x7f15d815f3c0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f15d002a270 tx=0x7f15d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.581+0000 7f15cf400640 1 -- 172.21.15.19:0/3598423963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0041070 con 0x7f15d81458f0 2024-04-25T11:43:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.582+0000 7f15cf400640 1 -- 172.21.15.19:0/3598423963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d003c950 con 0x7f15d81458f0 2024-04-25T11:43:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.582+0000 7f15cf400640 1 -- 172.21.15.19:0/3598423963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0037040 con 0x7f15d81458f0 2024-04-25T11:43:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.582+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d8140450 con 0x7f15d81458f0 2024-04-25T11:43:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.582+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d8140a10 con 0x7f15d81458f0 2024-04-25T11:43:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.583+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d8050740 con 0x7f15d81458f0 2024-04-25T11:43:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.585+0000 7f15cf400640 1 -- 172.21.15.19:0/3598423963 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15d0044070 con 0x7f15d81458f0 2024-04-25T11:43:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.585+0000 7f15cf400640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15b40744b0 0x7f15b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.586+0000 7f15cf400640 1 -- 172.21.15.19:0/3598423963 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5882+0+0 (secure 0 0 0) 0x7f15d00b5e60 con 0x7f15d81458f0 2024-04-25T11:43:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.586+0000 7f15d7400640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15b40744b0 0x7f15b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.588+0000 7f15d7400640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15b40744b0 0x7f15b40769c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f15d80b6e60 tx=0x7f15c80073c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.591+0000 7f15cf400640 1 -- 172.21.15.19:0/3598423963 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15d0080530 con 0x7f15d81458f0 2024-04-25T11:43:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.832+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f15d80b01c0 con 0x7f15d81458f0 2024-04-25T11:43:07.819 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-04-25T11:43:07.833+0000 7fb161200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:43:07.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.837+0000 7f15cf400640 1 -- 172.21.15.19:0/3598423963 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v373) ==== 161+0+0 (secure 0 0 0) 0x7f15d007fe30 con 0x7f15d81458f0 2024-04-25T11:43:07.823 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:43:07.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15b40744b0 msgr2=0x7f15b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15b40744b0 0x7f15b40769c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f15d80b6e60 tx=0x7f15c80073c0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81458f0 msgr2=0x7f15d815f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81458f0 0x7f15d815f3c0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f15d002a270 tx=0x7f15d00046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 shutdown_connections 2024-04-25T11:43:07.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f15b40744b0 0x7f15b40769c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f15d80b0ae0 0x7f15d813f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d81458f0 0x7f15d815f3c0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 --2- 172.21.15.19:0/3598423963 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f15d806fe40 0x7f15d8163450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.841+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 >> 172.21.15.19:0/3598423963 conn(0x7f15d80d1b40 msgr2=0x7f15d80af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.842+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 shutdown_connections 2024-04-25T11:43:07.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:07.842+0000 7f15dd200640 1 -- 172.21.15.19:0/3598423963 wait complete. 2024-04-25T11:43:07.847 INFO:tasks.ceph.mds.c.smithi019.stderr: -1587> 2024-04-25T11:42:48.558+0000 7f90ae800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:43:07.850 INFO:tasks.ceph.mds.c.smithi019.stderr: -1021> 2024-04-25T11:42:53.388+0000 7f90a9800640 -1 mds.0.cache corrupt backtrace on ino x00x10000000001: void inode_backtrace_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version 5 < 104: Malformed input [buffer:3] 2024-04-25T11:43:07.852 INFO:tasks.ceph.mds.c.smithi019.stderr: -720> 2024-04-25T11:42:55.549+0000 7f90a9800640 -1 log_channel(cluster) log [ERR] : dir 0x100000001f6 object missing on disk; some files may be lost (/dir2) 2024-04-25T11:43:07.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:43:07.985 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-04-25T11:43:08.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.139+0000 7f4275200640 1 Processor -- start 2024-04-25T11:43:08.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.139+0000 7f4275200640 1 -- start start 2024-04-25T11:43:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.141+0000 7f4275200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42700b4d80 0x7f42700b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.141+0000 7f4275200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42700b8270 0x7f42700b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.141+0000 7f4275200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42700d4420 con 0x7f42700b9130 2024-04-25T11:43:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.141+0000 7f4275200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f42700d0b10 con 0x7f42700b4d80 2024-04-25T11:43:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.141+0000 7f4275200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f42700d46c0 con 0x7f42700b8270 2024-04-25T11:43:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.141+0000 7f426ea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f42700b9130 0x7f42700b7b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35066/0 (socket says 172.21.15.19:35066) 2024-04-25T11:43:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.141+0000 7f426ea00640 1 -- 172.21.15.19:0/2834364193 learned_addr learned my addr 172.21.15.19:0/2834364193 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426f400640 1 --2- 172.21.15.19:0/2834364193 >> v2:172.21.15.19:3300/0 conn(0x7f42700b4d80 0x7f42700b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426fe00640 1 --2- 172.21.15.19:0/2834364193 >> v2:172.21.15.169:3300/0 conn(0x7f42700b8270 0x7f42700b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426e000640 1 -- 172.21.15.19:0/2834364193 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3410050028 0 0) 0x7f42700d4420 con 0x7f42700b9130 2024-04-25T11:43:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426e000640 1 -- 172.21.15.19:0/2834364193 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4250003520 con 0x7f42700b9130 2024-04-25T11:43:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426f400640 1 -- 172.21.15.19:0/2834364193 >> v2:172.21.15.169:3300/0 conn(0x7f42700b8270 msgr2=0x7f42700b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426f400640 1 --2- 172.21.15.19:0/2834364193 >> v2:172.21.15.169:3300/0 conn(0x7f42700b8270 0x7f42700b9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426f400640 1 -- 172.21.15.19:0/2834364193 >> v1:172.21.15.19:6789/0 conn(0x7f42700b9130 legacy=0x7f42700b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.142+0000 7f426f400640 1 -- 172.21.15.19:0/2834364193 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42700d49a0 con 0x7f42700b4d80 2024-04-25T11:43:08.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.143+0000 7f426f400640 1 --2- 172.21.15.19:0/2834364193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700b5260 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f427008f1f0 tx=0x7f426002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e78f952368eb0da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f426e000640 1 -- 172.21.15.19:0/2834364193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260037070 con 0x7f42700b4d80 2024-04-25T11:43:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f426e000640 1 -- 172.21.15.19:0/2834364193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4260032440 con 0x7f42700b4d80 2024-04-25T11:43:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f426e000640 1 -- 172.21.15.19:0/2834364193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42600327f0 con 0x7f42700b4d80 2024-04-25T11:43:08.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f4275200640 1 -- 172.21.15.19:0/2834364193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 msgr2=0x7f42700b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f4275200640 1 --2- 172.21.15.19:0/2834364193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700b5260 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f427008f1f0 tx=0x7f426002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f4275200640 1 -- 172.21.15.19:0/2834364193 shutdown_connections 2024-04-25T11:43:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f4275200640 1 --2- 172.21.15.19:0/2834364193 >> v2:172.21.15.169:3300/0 conn(0x7f42700b8270 0x7f42700b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f4275200640 1 --2- 172.21.15.19:0/2834364193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700b5260 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f4275200640 1 -- 172.21.15.19:0/2834364193 >> 172.21.15.19:0/2834364193 conn(0x7f42700d2bd0 msgr2=0x7f42700d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.144+0000 7f4275200640 1 -- 172.21.15.19:0/2834364193 shutdown_connections 2024-04-25T11:43:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.145+0000 7f4275200640 1 -- 172.21.15.19:0/2834364193 wait complete. 2024-04-25T11:43:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.147+0000 7f4275200640 1 Processor -- start 2024-04-25T11:43:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.147+0000 7f4275200640 1 -- start start 2024-04-25T11:43:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.148+0000 7f4275200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.148+0000 7f4275200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42700b9130 0x7f4270160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.148+0000 7f4275200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4270160d40 0x7f4270161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.148+0000 7f4275200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42700d08f0 con 0x7f42700b4d80 2024-04-25T11:43:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.148+0000 7f4275200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f42700d0770 con 0x7f42700b9130 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.148+0000 7f4275200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4270160430 con 0x7f4270160d40 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37676/0 (socket says 172.21.15.19:37676) 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 -- 172.21.15.19:0/3680308343 learned_addr learned my addr 172.21.15.19:0/3680308343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 -- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4270160d40 msgr2=0x7f4270161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426fe00640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4270160d40 0x7f4270161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426ea00640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42700b9130 0x7f4270160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4270160d40 0x7f4270161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 -- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42700b9130 msgr2=0x7f4270160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42700b9130 0x7f4270160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426f400640 1 -- 172.21.15.19:0/3680308343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4270161690 con 0x7f42700b4d80 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.149+0000 7f426ea00640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42700b9130 0x7f4270160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:08.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.150+0000 7f426f400640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700cf830 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f4260000c00 tx=0x7f42600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.150+0000 7f4267400640 1 -- 172.21.15.19:0/3680308343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260041030 con 0x7f42700b4d80 2024-04-25T11:43:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.150+0000 7f4267400640 1 -- 172.21.15.19:0/3680308343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426003d420 con 0x7f42700b4d80 2024-04-25T11:43:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.150+0000 7f4267400640 1 -- 172.21.15.19:0/3680308343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260037040 con 0x7f42700b4d80 2024-04-25T11:43:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.150+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42700cd430 con 0x7f42700b4d80 2024-04-25T11:43:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.150+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42700cd9f0 con 0x7f42700b4d80 2024-04-25T11:43:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.151+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270076c60 con 0x7f42700b4d80 2024-04-25T11:43:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.152+0000 7f4267400640 1 -- 172.21.15.19:0/3680308343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4260044070 con 0x7f42700b4d80 2024-04-25T11:43:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.153+0000 7f4267400640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f424c0744e0 0x7f424c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.154+0000 7f4267400640 1 -- 172.21.15.19:0/3680308343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5968+0+0 (secure 0 0 0) 0x7f42600b5ad0 con 0x7f42700b4d80 2024-04-25T11:43:08.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.154+0000 7f426ea00640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f424c0744e0 0x7f424c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.156+0000 7f426ea00640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f424c0744e0 0x7f424c0769f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f42700d5c60 tx=0x7f4258009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.158+0000 7f4267400640 1 -- 172.21.15.19:0/3680308343 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42600800f0 con 0x7f42700b4d80 2024-04-25T11:43:08.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.407+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f42700b52f0 con 0x7f42700b4d80 2024-04-25T11:43:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.842+0000 7f4267400640 1 -- 172.21.15.19:0/3680308343 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v374) ==== 104+0+0 (secure 0 0 0) 0x7f42600880e0 con 0x7f42700b4d80 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.845+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f424c0744e0 msgr2=0x7f424c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f424c0744e0 0x7f424c0769f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f42700d5c60 tx=0x7f4258009040 comp rx=0 tx=0).stop 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 msgr2=0x7f42700cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700cf830 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f4260000c00 tx=0x7f42600046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 shutdown_connections 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f424c0744e0 0x7f424c0769f0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42700b9130 0x7f4270160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4270160d40 0x7f4270161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 --2- 172.21.15.19:0/3680308343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42700b4d80 0x7f42700cf830 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 >> 172.21.15.19:0/3680308343 conn(0x7f42700d2bd0 msgr2=0x7f42700cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 shutdown_connections 2024-04-25T11:43:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:08.846+0000 7f4275200640 1 -- 172.21.15.19:0/3680308343 wait complete. 2024-04-25T11:43:08.862 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:43:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.164+0000 7f6ff7400640 1 Processor -- start 2024-04-25T11:43:09.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.164+0000 7f6ff7400640 1 -- start start 2024-04-25T11:43:09.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.166+0000 7f6ff7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ff00cd9a0 0x7f6ff015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.166+0000 7f6ff7400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ff006dce0 0x7f6ff015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.166+0000 7f6ff7400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ff00cdea0 con 0x7f6ff00b0af0 2024-04-25T11:43:09.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.166+0000 7f6ff7400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff00ceab0 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.166+0000 7f6ff7400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6ff00ce180 con 0x7f6ff006dce0 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.167+0000 7f6ff4a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ff00cd9a0 0x7f6ff015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.167+0000 7f6fefe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6ff00b0af0 0x7f6ff006d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35076/0 (socket says 172.21.15.19:35076) 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.167+0000 7f6fefe00640 1 -- 172.21.15.19:0/2207744230 learned_addr learned my addr 172.21.15.19:0/2207744230 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.167+0000 7f6ff5400640 1 --2- 172.21.15.19:0/2207744230 >> v2:172.21.15.169:3300/0 conn(0x7f6ff006dce0 0x7f6ff015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.167+0000 7f6ff4a00640 1 -- 172.21.15.19:0/2207744230 >> v2:172.21.15.169:3300/0 conn(0x7f6ff006dce0 msgr2=0x7f6ff015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.168+0000 7f6ff4a00640 1 --2- 172.21.15.19:0/2207744230 >> v2:172.21.15.169:3300/0 conn(0x7f6ff006dce0 0x7f6ff015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.168+0000 7f6ff4a00640 1 -- 172.21.15.19:0/2207744230 >> v1:172.21.15.19:6789/0 conn(0x7f6ff00b0af0 legacy=0x7f6ff006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.168+0000 7f6ff4a00640 1 -- 172.21.15.19:0/2207744230 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff00ce460 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.168+0000 7f6fef400640 1 -- 172.21.15.19:0/2207744230 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430948757 0 0) 0x7f6ff00cdea0 con 0x7f6ff00b0af0 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.169+0000 7f6ff4a00640 1 --2- 172.21.15.19:0/2207744230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff015f360 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0148020 tx=0x7f6fe002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f257d9af01d63e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.169+0000 7f6fef400640 1 -- 172.21.15.19:0/2207744230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007740 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.169+0000 7f6fef400640 1 -- 172.21.15.19:0/2207744230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe00078e0 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.169+0000 7f6fef400640 1 -- 172.21.15.19:0/2207744230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007cb0 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.170+0000 7f6ff7400640 1 -- 172.21.15.19:0/2207744230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 msgr2=0x7f6ff015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.170+0000 7f6ff7400640 1 --2- 172.21.15.19:0/2207744230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff015f360 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0148020 tx=0x7f6fe002a680 comp rx=0 tx=0).stop 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.170+0000 7f6ff7400640 1 -- 172.21.15.19:0/2207744230 shutdown_connections 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.170+0000 7f6ff7400640 1 --2- 172.21.15.19:0/2207744230 >> v2:172.21.15.169:3300/0 conn(0x7f6ff006dce0 0x7f6ff015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.170+0000 7f6ff7400640 1 --2- 172.21.15.19:0/2207744230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff015f360 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.170+0000 7f6ff7400640 1 -- 172.21.15.19:0/2207744230 >> 172.21.15.19:0/2207744230 conn(0x7f6ff0073090 msgr2=0x7f6ff00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.171+0000 7f6ff7400640 1 -- 172.21.15.19:0/2207744230 shutdown_connections 2024-04-25T11:43:09.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.171+0000 7f6ff7400640 1 -- 172.21.15.19:0/2207744230 wait complete. 2024-04-25T11:43:09.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.173+0000 7f6ff7400640 1 Processor -- start 2024-04-25T11:43:09.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.174+0000 7f6ff7400640 1 -- start start 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.175+0000 7f6ff7400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff00b0af0 0x7f6ff0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.175+0000 7f6ff7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.175+0000 7f6ff7400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff01fe9b0 0x7f6ff006d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.175+0000 7f6ff7400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff00cef40 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.175+0000 7f6ff7400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff00b0f00 con 0x7f6ff00b0af0 2024-04-25T11:43:09.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.175+0000 7f6ff7400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff00cf0c0 con 0x7f6ff01fe9b0 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.176+0000 7f6fefe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.176+0000 7f6fefe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff006d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37694/0 (socket says 172.21.15.19:37694) 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.176+0000 7f6fefe00640 1 -- 172.21.15.19:0/3650171619 learned_addr learned my addr 172.21.15.19:0/3650171619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.176+0000 7f6ff4a00640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff00b0af0 0x7f6ff0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.176+0000 7f6ff5400640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff01fe9b0 0x7f6ff006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fefe00640 1 -- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff01fe9b0 msgr2=0x7f6ff006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fefe00640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff01fe9b0 0x7f6ff006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fefe00640 1 -- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff00b0af0 msgr2=0x7f6ff0158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fefe00640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff00b0af0 0x7f6ff0158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fefe00640 1 -- 172.21.15.19:0/3650171619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff01fab50 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fefe00640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff006d1d0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00c460 tx=0x7f6fdc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fecc00640 1 -- 172.21.15.19:0/3650171619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc018070 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fecc00640 1 -- 172.21.15.19:0/3650171619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc007500 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6fecc00640 1 -- 172.21.15.19:0/3650171619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004a20 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.177+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff01fae30 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.178+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff01fb3f0 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.179+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff0050740 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.181+0000 7f6fecc00640 1 -- 172.21.15.19:0/3650171619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6fdc004bc0 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.182+0000 7f6fecc00640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6fd00744b0 0x7f6fd00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.183+0000 7f6fecc00640 1 -- 172.21.15.19:0/3650171619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5968+0+0 (secure 0 0 0) 0x7f6fdc094660 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.183+0000 7f6ff4a00640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6fd00744b0 0x7f6fd00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.184+0000 7f6ff4a00640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6fd00744b0 0x7f6fd00769c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0034000 tx=0x7f6fe0033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.190+0000 7f6fecc00640 1 -- 172.21.15.19:0/3650171619 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fdc05ed00 con 0x7f6ff00cd9a0 2024-04-25T11:43:09.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:09.402+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6ff015e290 con 0x7f6ff00cd9a0 2024-04-25T11:43:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.286+0000 7f6fecc00640 1 -- 172.21.15.19:0/3650171619 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7f6fdc05e600 con 0x7f6ff00cd9a0 2024-04-25T11:43:10.273 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:43:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6fd00744b0 msgr2=0x7f6fd00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6fd00744b0 0x7f6fd00769c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0034000 tx=0x7f6fe0033040 comp rx=0 tx=0).stop 2024-04-25T11:43:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 msgr2=0x7f6ff006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff006d1d0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00c460 tx=0x7f6fdc00c920 comp rx=0 tx=0).stop 2024-04-25T11:43:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 shutdown_connections 2024-04-25T11:43:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6fd00744b0 0x7f6fd00769c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ff01fe9b0 0x7f6ff006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ff00cd9a0 0x7f6ff006d1d0 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 --2- 172.21.15.19:0/3650171619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ff00b0af0 0x7f6ff0158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.290+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 >> 172.21.15.19:0/3650171619 conn(0x7f6ff0073090 msgr2=0x7f6ff015fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.291+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 shutdown_connections 2024-04-25T11:43:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.291+0000 7f6ff7400640 1 -- 172.21.15.19:0/3650171619 wait complete. 2024-04-25T11:43:10.316 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:43:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.593+0000 7f6ab9200640 1 Processor -- start 2024-04-25T11:43:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.593+0000 7f6ab9200640 1 -- start start 2024-04-25T11:43:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.595+0000 7f6ab9200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ab414aac0 0x7f6ab41655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.595+0000 7f6ab9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ab40b50b0 0x7f6ab40b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.595+0000 7f6ab9200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab4165ae0 con 0x7f6ab406d5d0 2024-04-25T11:43:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.595+0000 7f6ab9200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab414aed0 con 0x7f6ab40b50b0 2024-04-25T11:43:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.595+0000 7f6ab9200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6ab406f460 con 0x7f6ab414aac0 2024-04-25T11:43:10.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.596+0000 7f6ab3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ab40b50b0 0x7f6ab40b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.596+0000 7f6ab3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ab40b50b0 0x7f6ab40b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37706/0 (socket says 172.21.15.19:37706) 2024-04-25T11:43:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.596+0000 7f6ab3e00640 1 -- 172.21.15.19:0/559875395 learned_addr learned my addr 172.21.15.19:0/559875395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.596+0000 7f6ab3e00640 1 -- 172.21.15.19:0/559875395 >> v2:172.21.15.169:3300/0 conn(0x7f6ab414aac0 msgr2=0x7f6ab41655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.596+0000 7f6ab3e00640 1 --2- 172.21.15.19:0/559875395 >> v2:172.21.15.169:3300/0 conn(0x7f6ab414aac0 0x7f6ab41655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.596+0000 7f6ab3e00640 1 -- 172.21.15.19:0/559875395 >> v1:172.21.15.169:6789/0 conn(0x7f6ab406d5d0 legacy=0x7f6ab406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.596+0000 7f6ab3e00640 1 -- 172.21.15.19:0/559875395 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab406f740 con 0x7f6ab40b50b0 2024-04-25T11:43:10.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.597+0000 7f6ab3e00640 1 --2- 172.21.15.19:0/559875395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab40b50b0 0x7f6ab40b54c0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4073ef0 tx=0x7f6aac02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf35562390b987d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.598+0000 7f6ab2000640 1 -- 172.21.15.19:0/559875395 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac036070 con 0x7f6ab40b50b0 2024-04-25T11:43:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.598+0000 7f6ab2000640 1 -- 172.21.15.19:0/559875395 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aac031600 con 0x7f6ab40b50b0 2024-04-25T11:43:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.598+0000 7f6ab2000640 1 -- 172.21.15.19:0/559875395 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0319d0 con 0x7f6ab40b50b0 2024-04-25T11:43:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 -- 172.21.15.19:0/559875395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab40b50b0 msgr2=0x7f6ab40b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 --2- 172.21.15.19:0/559875395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab40b50b0 0x7f6ab40b54c0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4073ef0 tx=0x7f6aac02a5d0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 -- 172.21.15.19:0/559875395 shutdown_connections 2024-04-25T11:43:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 --2- 172.21.15.19:0/559875395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab40b50b0 0x7f6ab40b54c0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 --2- 172.21.15.19:0/559875395 >> v2:172.21.15.169:3300/0 conn(0x7f6ab414aac0 0x7f6ab41655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 -- 172.21.15.19:0/559875395 >> 172.21.15.19:0/559875395 conn(0x7f6ab4072f40 msgr2=0x7f6ab4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 -- 172.21.15.19:0/559875395 shutdown_connections 2024-04-25T11:43:10.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.599+0000 7f6ab9200640 1 -- 172.21.15.19:0/559875395 wait complete. 2024-04-25T11:43:10.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.602+0000 7f6ab9200640 1 Processor -- start 2024-04-25T11:43:10.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.602+0000 7f6ab9200640 1 -- start start 2024-04-25T11:43:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.603+0000 7f6ab9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab406d5d0 0x7f6ab4149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:10.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.603+0000 7f6ab9200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ab40b50b0 0x7f6ab4146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab9200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ab414aac0 0x7f6ab41faab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab9200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406ffe0 con 0x7f6ab406d5d0 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab9200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab406fd00 con 0x7f6ab40b50b0 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab9200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ab40b4a90 con 0x7f6ab414aac0 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab406d5d0 0x7f6ab4149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab406d5d0 0x7f6ab4149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37720/0 (socket says 172.21.15.19:37720) 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 -- 172.21.15.19:0/3542428872 learned_addr learned my addr 172.21.15.19:0/3542428872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3e00640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ab414aac0 0x7f6ab41faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab2a00640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ab40b50b0 0x7f6ab4146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 -- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ab414aac0 msgr2=0x7f6ab41faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ab414aac0 0x7f6ab41faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 -- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ab40b50b0 msgr2=0x7f6ab4146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ab40b50b0 0x7f6ab4146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3400640 1 -- 172.21.15.19:0/3542428872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab41484b0 con 0x7f6ab406d5d0 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab3e00640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ab414aac0 0x7f6ab41faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.604+0000 7f6ab2a00640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ab40b50b0 0x7f6ab4146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.605+0000 7f6ab3400640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab406d5d0 0x7f6ab4149550 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400b440 tx=0x7f6aa400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.605+0000 7f6aab400640 1 -- 172.21.15.19:0/3542428872 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4015020 con 0x7f6ab406d5d0 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.605+0000 7f6aab400640 1 -- 172.21.15.19:0/3542428872 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa40118b0 con 0x7f6ab406d5d0 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.605+0000 7f6aab400640 1 -- 172.21.15.19:0/3542428872 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4011ca0 con 0x7f6ab406d5d0 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.605+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab4148790 con 0x7f6ab406d5d0 2024-04-25T11:43:10.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.605+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab4148d50 con 0x7f6ab406d5d0 2024-04-25T11:43:10.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.607+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab4076c60 con 0x7f6ab406d5d0 2024-04-25T11:43:10.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.608+0000 7f6aab400640 1 -- 172.21.15.19:0/3542428872 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6aa4018020 con 0x7f6ab406d5d0 2024-04-25T11:43:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.609+0000 7f6aab400640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6a90074530 0x7f6a90076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:10.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.609+0000 7f6aab400640 1 -- 172.21.15.19:0/3542428872 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5443+0+0 (secure 0 0 0) 0x7f6aa4095240 con 0x7f6ab406d5d0 2024-04-25T11:43:10.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.610+0000 7f6ab2a00640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6a90074530 0x7f6a90076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:10.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.611+0000 7f6ab2a00640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6a90074530 0x7f6a90076a40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6ab41691d0 tx=0x7f6a9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:10.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.614+0000 7f6aab400640 1 -- 172.21.15.19:0/3542428872 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6aa401f020 con 0x7f6ab406d5d0 2024-04-25T11:43:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:10.822+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6ab4070e70 con 0x7f6ab406d5d0 2024-04-25T11:43:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.299+0000 7f6aab400640 1 -- 172.21.15.19:0/3542428872 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7f6aa400d040 con 0x7f6ab406d5d0 2024-04-25T11:43:11.286 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T11:43:11.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6a90074530 msgr2=0x7f6a90076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6a90074530 0x7f6a90076a40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6ab41691d0 tx=0x7f6a9c008040 comp rx=0 tx=0).stop 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab406d5d0 msgr2=0x7f6ab4149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab406d5d0 0x7f6ab4149550 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400b440 tx=0x7f6aa400b900 comp rx=0 tx=0).stop 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 shutdown_connections 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6a90074530 0x7f6a90076a40 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ab40b50b0 0x7f6ab4146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ab414aac0 0x7f6ab41faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 --2- 172.21.15.19:0/3542428872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ab406d5d0 0x7f6ab4149550 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.303+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 >> 172.21.15.19:0/3542428872 conn(0x7f6ab4072f40 msgr2=0x7f6ab40b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:11.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.304+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 shutdown_connections 2024-04-25T11:43:11.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.305+0000 7f6ab9200640 1 -- 172.21.15.19:0/3542428872 wait complete. 2024-04-25T11:43:11.328 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:43:11.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.623+0000 7fe1a6600640 1 Processor -- start 2024-04-25T11:43:11.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.623+0000 7fe1a6600640 1 -- start start 2024-04-25T11:43:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.625+0000 7fe1a6600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe1a00b4f40 0x7fe1a01646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.625+0000 7fe1a6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.625+0000 7fe1a6600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a00b8230 con 0x7fe1a00d4190 2024-04-25T11:43:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.625+0000 7fe1a6600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a0163c70 con 0x7fe1a00b4f40 2024-04-25T11:43:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.625+0000 7fe1a6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a0164230 con 0x7fe1a00b78e0 2024-04-25T11:43:11.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe197e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe1a5200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe1a00b4f40 0x7fe1a01646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe197e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:58240/0 (socket says 172.21.15.19:58240) 2024-04-25T11:43:11.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe197e00640 1 -- 172.21.15.19:0/3492804118 learned_addr learned my addr 172.21.15.19:0/3492804118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:11.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe197e00640 1 -- 172.21.15.19:0/3492804118 >> v2:172.21.15.169:3300/0 conn(0x7fe1a00b4f40 msgr2=0x7fe1a01646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:11.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe197e00640 1 --2- 172.21.15.19:0/3492804118 >> v2:172.21.15.169:3300/0 conn(0x7fe1a00b4f40 0x7fe1a01646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe197e00640 1 -- 172.21.15.19:0/3492804118 >> v1:172.21.15.169:6789/0 conn(0x7fe1a00d4190 legacy=0x7fe1a00d45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.626+0000 7fe197e00640 1 -- 172.21.15.19:0/3492804118 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a00b5350 con 0x7fe1a00b78e0 2024-04-25T11:43:11.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.628+0000 7fe197e00640 1 --2- 172.21.15.19:0/3492804118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00b7cf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe188009870 tx=0x7fe18802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907f5c2b70fd46d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.629+0000 7fe19fe00640 1 -- 172.21.15.19:0/3492804118 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188037070 con 0x7fe1a00b78e0 2024-04-25T11:43:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.629+0000 7fe19fe00640 1 -- 172.21.15.19:0/3492804118 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188032440 con 0x7fe1a00b78e0 2024-04-25T11:43:11.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.629+0000 7fe19fe00640 1 -- 172.21.15.19:0/3492804118 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188032810 con 0x7fe1a00b78e0 2024-04-25T11:43:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 -- 172.21.15.19:0/3492804118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 msgr2=0x7fe1a00b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 --2- 172.21.15.19:0/3492804118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00b7cf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe188009870 tx=0x7fe18802a270 comp rx=0 tx=0).stop 2024-04-25T11:43:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 -- 172.21.15.19:0/3492804118 shutdown_connections 2024-04-25T11:43:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 --2- 172.21.15.19:0/3492804118 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00b7cf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 --2- 172.21.15.19:0/3492804118 >> v2:172.21.15.169:3300/0 conn(0x7fe1a00b4f40 0x7fe1a01646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 -- 172.21.15.19:0/3492804118 >> 172.21.15.19:0/3492804118 conn(0x7fe1a00d2980 msgr2=0x7fe1a00d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:11.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 -- 172.21.15.19:0/3492804118 shutdown_connections 2024-04-25T11:43:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.630+0000 7fe1a6600640 1 -- 172.21.15.19:0/3492804118 wait complete. 2024-04-25T11:43:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.633+0000 7fe1a6600640 1 Processor -- start 2024-04-25T11:43:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.633+0000 7fe1a6600640 1 -- start start 2024-04-25T11:43:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.635+0000 7fe1a6600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00b4f40 0x7fe1a0154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.635+0000 7fe1a6600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.635+0000 7fe1a6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a00d4190 0x7fe1a00cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.635+0000 7fe1a6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0163ec0 con 0x7fe1a00d4190 2024-04-25T11:43:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.635+0000 7fe1a6600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01613d0 con 0x7fe1a00b4f40 2024-04-25T11:43:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.635+0000 7fe1a6600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a00d4cb0 con 0x7fe1a00b78e0 2024-04-25T11:43:11.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.636+0000 7fe1a5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a00d4190 0x7fe1a00cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.636+0000 7fe1a5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a00d4190 0x7fe1a00cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37726/0 (socket says 172.21.15.19:37726) 2024-04-25T11:43:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.636+0000 7fe1a5c00640 1 -- 172.21.15.19:0/365945406 learned_addr learned my addr 172.21.15.19:0/365945406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.636+0000 7fe1a5200640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00b4f40 0x7fe1a0154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.636+0000 7fe197e00640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a5c00640 1 -- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 msgr2=0x7fe1a00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a5c00640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a5c00640 1 -- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00b4f40 msgr2=0x7fe1a0154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a5c00640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00b4f40 0x7fe1a0154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a5c00640 1 -- 172.21.15.19:0/365945406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a0154da0 con 0x7fe1a00d4190 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe197e00640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a5c00640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a00d4190 0x7fe1a00cefd0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7fe19800c680 tx=0x7fe19800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:11.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe19d600640 1 -- 172.21.15.19:0/365945406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198003b70 con 0x7fe1a00d4190 2024-04-25T11:43:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a00cd5f0 con 0x7fe1a00d4190 2024-04-25T11:43:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe19d600640 1 -- 172.21.15.19:0/365945406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe198003d10 con 0x7fe1a00d4190 2024-04-25T11:43:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe19d600640 1 -- 172.21.15.19:0/365945406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198005310 con 0x7fe1a00d4190 2024-04-25T11:43:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.637+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a00cdbb0 con 0x7fe1a00d4190 2024-04-25T11:43:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.639+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a0076c60 con 0x7fe1a00d4190 2024-04-25T11:43:11.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.640+0000 7fe19d600640 1 -- 172.21.15.19:0/365945406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1980054b0 con 0x7fe1a00d4190 2024-04-25T11:43:11.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.642+0000 7fe19d600640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe17c0744b0 0x7fe17c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.642+0000 7fe19d600640 1 -- 172.21.15.19:0/365945406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fe198094e80 con 0x7fe1a00d4190 2024-04-25T11:43:11.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.642+0000 7fe1a5200640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe17c0744b0 0x7fe17c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.644+0000 7fe1a5200640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe17c0744b0 0x7fe17c0769c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00d3f40 tx=0x7fe190009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:11.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.647+0000 7fe19d600640 1 -- 172.21.15.19:0/365945406 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1a00b8230 con 0x7fe1a00d4190 2024-04-25T11:43:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.850+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe1a00d3f70 con 0x7fe1a00d4190 2024-04-25T11:43:11.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.859+0000 7fe19d600640 1 -- 172.21.15.19:0/365945406 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fe19805f820 con 0x7fe1a00d4190 2024-04-25T11:43:11.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe17c0744b0 msgr2=0x7fe17c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe17c0744b0 0x7fe17c0769c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00d3f40 tx=0x7fe190009190 comp rx=0 tx=0).stop 2024-04-25T11:43:11.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a00d4190 msgr2=0x7fe1a00cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a00d4190 0x7fe1a00cefd0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7fe19800c680 tx=0x7fe19800cb40 comp rx=0 tx=0).stop 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 shutdown_connections 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe17c0744b0 0x7fe17c0769c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1a00d4190 0x7fe1a00cefd0 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1a00b78e0 0x7fe1a00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 --2- 172.21.15.19:0/365945406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1a00b4f40 0x7fe1a0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 >> 172.21.15.19:0/365945406 conn(0x7fe1a00d2980 msgr2=0x7fe1a00b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 shutdown_connections 2024-04-25T11:43:11.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:11.863+0000 7fe1a6600640 1 -- 172.21.15.19:0/365945406 wait complete. 2024-04-25T11:43:11.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-04-25T11:43:12.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.176+0000 7f1f14200640 1 Processor -- start 2024-04-25T11:43:12.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.176+0000 7f1f14200640 1 -- start start 2024-04-25T11:43:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.179+0000 7f1f14200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1f0c0cf5e0 0x7f1f0c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.179+0000 7f1f14200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c0cff30 con 0x7f1f0c1443c0 2024-04-25T11:43:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.179+0000 7f1f14200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c15f5f0 con 0x7f1f0c06c5c0 2024-04-25T11:43:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.179+0000 7f1f14200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f1f0c0708d0 con 0x7f1f0c0cf5e0 2024-04-25T11:43:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.179+0000 7f1f11800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1f0c1443c0 0x7f1f0c06fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35084/0 (socket says 172.21.15.19:35084) 2024-04-25T11:43:12.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.179+0000 7f1f11800640 1 -- 172.21.15.19:0/2282485272 learned_addr learned my addr 172.21.15.19:0/2282485272 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:12.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f10e00640 1 --2- 172.21.15.19:0/2282485272 >> v2:172.21.15.169:3300/0 conn(0x7f1f0c0cf5e0 0x7f1f0c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956095101 0 0) 0x7f1f00001110 con 0x7f1f0c1443c0 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ef4003520 con 0x7f1f0c1443c0 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4234748921 0 0) 0x7f1ef4003520 con 0x7f1f0c1443c0 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 >> v2:172.21.15.169:3300/0 conn(0x7f1f0c0cf5e0 msgr2=0x7f1f0c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 --2- 172.21.15.19:0/2282485272 >> v2:172.21.15.169:3300/0 conn(0x7f1f0c0cf5e0 0x7f1f0c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 >> v1:172.21.15.169:6789/0 conn(0x7f1f0c06c5c0 legacy=0x7f1f0c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c15e830 con 0x7f1f0c1443c0 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f1f00001840 con 0x7f1f0c1443c0 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 >> v1:172.21.15.19:6789/0 conn(0x7f1f0c1443c0 legacy=0x7f1f0c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.180+0000 7f1f0be00640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef40039b0 0x7f1ef4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f0be00640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef4004800 0x7f1ef4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f0be00640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1ef400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c0708d0 con 0x7f1ef40051c0 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef400c160 con 0x7f1ef40039b0 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef400c360 con 0x7f1ef4004800 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f12200640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1ef400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f11800640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef40039b0 0x7f1ef4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f10e00640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef4004800 0x7f1ef4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f12200640 1 -- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef4004800 msgr2=0x7f1ef4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f12200640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1ef4004800 0x7f1ef4004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f12200640 1 -- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef40039b0 msgr2=0x7f1ef4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f12200640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1ef40039b0 0x7f1ef4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:12.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.181+0000 7f1f12200640 1 -- 172.21.15.19:0/2282485272 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef4003520 con 0x7f1ef40051c0 2024-04-25T11:43:12.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.182+0000 7f1f12200640 1 -- 172.21.15.19:0/2282485272 reap_dead start 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.182+0000 7f1f12200640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1ef400b9c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001e20 tx=0x7f1ef4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f6c24fbcc4df73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.183+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04001a70 con 0x7f1ef40051c0 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.183+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0402ad70 con 0x7f1ef40051c0 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.183+0000 7f1f0be00640 1 -- 172.21.15.19:0/2282485272 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04003bb0 con 0x7f1ef40051c0 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.183+0000 7f1f14200640 1 -- 172.21.15.19:0/2282485272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 msgr2=0x7f1ef400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.183+0000 7f1f14200640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1ef400b9c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001e20 tx=0x7f1ef4005830 comp rx=0 tx=0).stop 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.184+0000 7f1f14200640 1 -- 172.21.15.19:0/2282485272 shutdown_connections 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.184+0000 7f1f14200640 1 --2- 172.21.15.19:0/2282485272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1ef400b9c0 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.184+0000 7f1f14200640 1 -- 172.21.15.19:0/2282485272 >> 172.21.15.19:0/2282485272 conn(0x7f1f0c0d1ae0 msgr2=0x7f1f0c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.184+0000 7f1f14200640 1 -- 172.21.15.19:0/2282485272 shutdown_connections 2024-04-25T11:43:12.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.184+0000 7f1f14200640 1 -- 172.21.15.19:0/2282485272 wait complete. 2024-04-25T11:43:12.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.186+0000 7f1f14200640 1 Processor -- start 2024-04-25T11:43:12.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.186+0000 7f1f14200640 1 -- start start 2024-04-25T11:43:12.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f14200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1f0c0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f14200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c0ced70 0x7f1f0c141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f14200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c0cf180 0x7f1f0c1416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f14200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c15edf0 con 0x7f1ef40051c0 2024-04-25T11:43:12.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f14200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c15f160 con 0x7f1f0c0ced70 2024-04-25T11:43:12.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f14200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c1447d0 con 0x7f1f0c0cf180 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f11800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1f0c0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f11800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1f0c0b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37744/0 (socket says 172.21.15.19:37744) 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.188+0000 7f1f11800640 1 -- 172.21.15.19:0/2336810296 learned_addr learned my addr 172.21.15.19:0/2336810296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f10e00640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c0ced70 0x7f1f0c141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f12200640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c0cf180 0x7f1f0c1416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f11800640 1 -- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c0cf180 msgr2=0x7f1f0c1416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f11800640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c0cf180 0x7f1f0c1416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f11800640 1 -- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c0ced70 msgr2=0x7f1f0c141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f11800640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c0ced70 0x7f1f0c141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f11800640 1 -- 172.21.15.19:0/2336810296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c0ae640 con 0x7f1ef40051c0 2024-04-25T11:43:12.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f11800640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1f0c0b0d40 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e860 tx=0x7f1f0000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:12.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.189+0000 7f1f09600640 1 -- 172.21.15.19:0/2336810296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000ee50 con 0x7f1ef40051c0 2024-04-25T11:43:12.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.190+0000 7f1f09600640 1 -- 172.21.15.19:0/2336810296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f00018070 con 0x7f1ef40051c0 2024-04-25T11:43:12.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.190+0000 7f1f09600640 1 -- 172.21.15.19:0/2336810296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000cb10 con 0x7f1ef40051c0 2024-04-25T11:43:12.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.190+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c15b950 con 0x7f1ef40051c0 2024-04-25T11:43:12.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.190+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c14b490 con 0x7f1ef40051c0 2024-04-25T11:43:12.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.191+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c0ae810 con 0x7f1ef40051c0 2024-04-25T11:43:12.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.192+0000 7f1f09600640 1 -- 172.21.15.19:0/2336810296 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f0000ccb0 con 0x7f1ef40051c0 2024-04-25T11:43:12.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.193+0000 7f1f09600640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1eec0744b0 0x7f1eec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:12.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.193+0000 7f1f09600640 1 -- 172.21.15.19:0/2336810296 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f1f00094da0 con 0x7f1ef40051c0 2024-04-25T11:43:12.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.194+0000 7f1f10e00640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1eec0744b0 0x7f1eec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:12.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.195+0000 7f1f10e00640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1eec0744b0 0x7f1eec0769c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c15f8a0 tx=0x7f1ef8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:12.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.198+0000 7f1f09600640 1 -- 172.21.15.19:0/2336810296 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f0c0cff30 con 0x7f1ef40051c0 2024-04-25T11:43:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:12.480+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7f1f0c050740 con 0x7f1ef40051c0 2024-04-25T11:43:13.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.315+0000 7f1f09600640 1 -- 172.21.15.19:0/2336810296 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v835) ==== 135+0+0 (secure 0 0 0) 0x7f1f0005f7c0 con 0x7f1ef40051c0 2024-04-25T11:43:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1eec0744b0 msgr2=0x7f1eec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1eec0744b0 0x7f1eec0769c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c15f8a0 tx=0x7f1ef8008040 comp rx=0 tx=0).stop 2024-04-25T11:43:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 msgr2=0x7f1f0c0b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1f0c0b0d40 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e860 tx=0x7f1f0000ed20 comp rx=0 tx=0).stop 2024-04-25T11:43:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 shutdown_connections 2024-04-25T11:43:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1eec0744b0 0x7f1eec0769c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1f0c0ced70 0x7f1f0c141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1f0c0cf180 0x7f1f0c1416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 --2- 172.21.15.19:0/2336810296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ef40051c0 0x7f1f0c0b0d40 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.318+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 >> 172.21.15.19:0/2336810296 conn(0x7f1f0c0d1ae0 msgr2=0x7f1f0c143710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.319+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 shutdown_connections 2024-04-25T11:43:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:13.319+0000 7f1f14200640 1 -- 172.21.15.19:0/2336810296 wait complete. 2024-04-25T11:43:13.335 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ok 2024-04-25T11:43:13.336 INFO:tasks.cephfs_test_runner: 2024-04-25T11:43:13.336 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T11:43:13.336 INFO:tasks.cephfs_test_runner:Ran 6 tests in 903.125s 2024-04-25T11:43:13.336 INFO:tasks.cephfs_test_runner: 2024-04-25T11:43:13.336 INFO:tasks.cephfs_test_runner:OK 2024-04-25T11:43:13.336 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T11:43:13.346 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T11:43:13.362 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T11:43:13.362 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:43:13.362 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:13.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:43:13.421 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:43:13.421 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:13.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:43:13.482 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:43:13.482 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:13.540 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:43:13.540 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:43:13.541 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:13.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:43:13.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:13.592 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:43:13.651 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:43:13.651 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:43:13.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:13.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:43:13.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:13.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:43:13.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:13.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T11:43:13.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:13.945 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T11:43:13.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:13.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T11:43:14.121 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T11:43:14.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:43:14.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:14.126 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T11:43:14.185 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:43:14.185 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:43:14.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:14.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:43:14.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:14.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:43:14.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:14.347 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T11:43:14.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:14.471 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-04-25T11:43:14.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:14.513 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-04-25T11:43:14.634 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-04-25T11:43:14.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:43:14.638 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T11:43:14.647 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:43:14.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:43:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.953+0000 7f842e000640 1 Processor -- start 2024-04-25T11:43:14.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.953+0000 7f842e000640 1 -- start start 2024-04-25T11:43:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.955+0000 7f842e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280b6e10 0x7f842815a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.955+0000 7f842e000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84281608d0 0x7f8428160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.955+0000 7f842e000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8428161220 con 0x7f84280cf2f0 2024-04-25T11:43:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.955+0000 7f842e000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f842806f870 con 0x7f84281608d0 2024-04-25T11:43:14.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.955+0000 7f842e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f84280b7220 con 0x7f84280b6e10 2024-04-25T11:43:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.956+0000 7f842d600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84281608d0 0x7f8428160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.956+0000 7f8427e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280b6e10 0x7f842815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.956+0000 7f842d600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84281608d0 0x7f8428160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:47814/0 (socket says 172.21.15.19:47814) 2024-04-25T11:43:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.956+0000 7f842d600640 1 -- 172.21.15.19:0/332323351 learned_addr learned my addr 172.21.15.19:0/332323351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:14.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.956+0000 7f842cc00640 1 --1- 172.21.15.19:0/332323351 >> v1:172.21.15.169:6789/0 conn(0x7f84280cf2f0 0x7f84280b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.19:56796/0 (socket says 172.21.15.19:56796) 2024-04-25T11:43:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.957+0000 7f842d600640 1 -- 172.21.15.19:0/332323351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280b6e10 msgr2=0x7f842815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.957+0000 7f842d600640 1 --2- 172.21.15.19:0/332323351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280b6e10 0x7f842815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.957+0000 7f842d600640 1 -- 172.21.15.19:0/332323351 >> v1:172.21.15.169:6789/0 conn(0x7f84280cf2f0 legacy=0x7f84280b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:14.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.957+0000 7f842d600640 1 -- 172.21.15.19:0/332323351 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84280b74d0 con 0x7f84281608d0 2024-04-25T11:43:14.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.958+0000 7f842d600640 1 --2- 172.21.15.19:0/332323351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 0x7f8428160ce0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f842000bb10 tx=0x7f842002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a45bf776925344c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:14.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.959+0000 7f8427400640 1 -- 172.21.15.19:0/332323351 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420036020 con 0x7f84281608d0 2024-04-25T11:43:14.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.959+0000 7f8427400640 1 -- 172.21.15.19:0/332323351 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8420032820 con 0x7f84281608d0 2024-04-25T11:43:14.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.959+0000 7f8427400640 1 -- 172.21.15.19:0/332323351 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420032bf0 con 0x7f84281608d0 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.959+0000 7f842e000640 1 -- 172.21.15.19:0/332323351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 msgr2=0x7f8428160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.959+0000 7f842e000640 1 --2- 172.21.15.19:0/332323351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 0x7f8428160ce0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f842000bb10 tx=0x7f842002c920 comp rx=0 tx=0).stop 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.960+0000 7f842e000640 1 -- 172.21.15.19:0/332323351 shutdown_connections 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.960+0000 7f842e000640 1 --2- 172.21.15.19:0/332323351 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 0x7f8428160ce0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.960+0000 7f842e000640 1 --2- 172.21.15.19:0/332323351 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280b6e10 0x7f842815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.960+0000 7f842e000640 1 -- 172.21.15.19:0/332323351 >> 172.21.15.19:0/332323351 conn(0x7f84280d1720 msgr2=0x7f84280d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.960+0000 7f842e000640 1 -- 172.21.15.19:0/332323351 shutdown_connections 2024-04-25T11:43:14.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.960+0000 7f842e000640 1 -- 172.21.15.19:0/332323351 wait complete. 2024-04-25T11:43:14.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.962+0000 7f842e000640 1 Processor -- start 2024-04-25T11:43:14.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.962+0000 7f842e000640 1 -- start start 2024-04-25T11:43:14.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84280b6e10 0x7f8428146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:14.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842e000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280cf2f0 0x7f8428149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:14.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842e000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 0x7f842815f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:14.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842e000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f842806f500 con 0x7f84280b6e10 2024-04-25T11:43:14.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842e000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f842806f220 con 0x7f84281608d0 2024-04-25T11:43:14.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842e000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f842806fd00 con 0x7f84280cf2f0 2024-04-25T11:43:14.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84280b6e10 0x7f8428146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:14.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84280b6e10 0x7f8428146a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60510/0 (socket says 172.21.15.19:60510) 2024-04-25T11:43:14.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.964+0000 7f842cc00640 1 -- 172.21.15.19:0/2564261854 learned_addr learned my addr 172.21.15.19:0/2564261854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f842d600640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 0x7f842815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f8427e00640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280cf2f0 0x7f8428149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f842cc00640 1 -- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280cf2f0 msgr2=0x7f8428149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f842cc00640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280cf2f0 0x7f8428149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f842cc00640 1 -- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 msgr2=0x7f842815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:14.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f842cc00640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 0x7f842815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f842cc00640 1 -- 172.21.15.19:0/2564261854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8428147510 con 0x7f84280b6e10 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f842cc00640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84280b6e10 0x7f8428146a90 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f841800c730 tx=0x7f841800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.965+0000 7f8424c00640 1 -- 172.21.15.19:0/2564261854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418017070 con 0x7f84280b6e10 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.966+0000 7f8424c00640 1 -- 172.21.15.19:0/2564261854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841800ce40 con 0x7f84280b6e10 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.966+0000 7f8424c00640 1 -- 172.21.15.19:0/2564261854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841800a860 con 0x7f84280b6e10 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.966+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281477f0 con 0x7f84280b6e10 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.966+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8428147db0 con 0x7f84280b6e10 2024-04-25T11:43:14.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.967+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8428050740 con 0x7f84280b6e10 2024-04-25T11:43:14.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.968+0000 7f8424c00640 1 -- 172.21.15.19:0/2564261854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f841801a070 con 0x7f84280b6e10 2024-04-25T11:43:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.969+0000 7f8424c00640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84040744b0 0x7f84040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:14.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.969+0000 7f8424c00640 1 -- 172.21.15.19:0/2564261854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f8418094650 con 0x7f84280b6e10 2024-04-25T11:43:14.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.970+0000 7f8427e00640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84040744b0 0x7f84040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:14.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.971+0000 7f8427e00640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84040744b0 0x7f84040769c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f84281ea770 tx=0x7f8410008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:14.974+0000 7f8424c00640 1 -- 172.21.15.19:0/2564261854 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8428161220 con 0x7f84280b6e10 2024-04-25T11:43:15.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.173+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f842806c770 con 0x7f84040744b0 2024-04-25T11:43:15.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.176+0000 7f8424c00640 1 -- 172.21.15.19:0/2564261854 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f842806c770 con 0x7f84040744b0 2024-04-25T11:43:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:15.163 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:43:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84040744b0 msgr2=0x7f84040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84040744b0 0x7f84040769c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f84281ea770 tx=0x7f8410008040 comp rx=0 tx=0).stop 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84280b6e10 msgr2=0x7f8428146a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84280b6e10 0x7f8428146a90 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f841800c730 tx=0x7f841800cbf0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 shutdown_connections 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f84040744b0 0x7f84040769c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84281608d0 0x7f842815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84280cf2f0 0x7f8428149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 --2- 172.21.15.19:0/2564261854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84280b6e10 0x7f8428146a90 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 >> 172.21.15.19:0/2564261854 conn(0x7f84280d1720 msgr2=0x7f84280b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 shutdown_connections 2024-04-25T11:43:15.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.179+0000 7f842e000640 1 -- 172.21.15.19:0/2564261854 wait complete. 2024-04-25T11:43:15.195 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":623,"stamp":"2024-04-25T11:43:14.849870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":302840,"kb_used_data":15960,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671880,"statfs":{"total":773094113280,"available":772784005120,"internally_reserved":0,"allocated":16343040,"data_stored":8458749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36338,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-101,"num_write_kb":-71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.008535"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":360,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T11:43:11.306052+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:43:11.306052+0000","last_peered":"2024-04-25T11:43:11.306052+0000","last_clean":"2024-04-25T11:43:11.306052+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:43:11.306052+0000","last_undegraded":"2024-04-25T11:43:11.306052+0000","last_fullsized":"2024-04-25T11:43:11.306052+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509635,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509636,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33032,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37772,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334068,"statfs":{"total":96636764160,"available":96598085632,"internally_reserved":0,"allocated":2490368,"data_stored":1500826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509635,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542338,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2473984,"data_stored":1495175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:43:15.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:43:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.478+0000 7f5761400640 1 Processor -- start 2024-04-25T11:43:15.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.478+0000 7f5761400640 1 -- start start 2024-04-25T11:43:15.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.480+0000 7f5761400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c0b75f0 0x7f575c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.480+0000 7f5761400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f575c1605c0 0x7f575c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.480+0000 7f5761400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f575c0cf270 0x7f575c169000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.480+0000 7f5761400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f575c0cf680 con 0x7f575c1605c0 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.480+0000 7f5761400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f575c06c6f0 con 0x7f575c0cf270 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.480+0000 7f5761400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f575c0cfb40 con 0x7f575c0b75f0 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f575c1605c0 0x7f575c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f575c1605c0 0x7f575c0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60516/0 (socket says 172.21.15.19:60516) 2024-04-25T11:43:15.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 -- 172.21.15.19:0/3632288404 learned_addr learned my addr 172.21.15.19:0/3632288404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575be00640 1 --2- 172.21.15.19:0/3632288404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c0b75f0 0x7f575c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 -- 172.21.15.19:0/3632288404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c0b75f0 msgr2=0x7f575c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f5760a00640 1 --2- 172.21.15.19:0/3632288404 >> v2:172.21.15.169:3300/0 conn(0x7f575c0cf270 0x7f575c169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 --2- 172.21.15.19:0/3632288404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c0b75f0 0x7f575c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 -- 172.21.15.19:0/3632288404 >> v2:172.21.15.169:3300/0 conn(0x7f575c0cf270 msgr2=0x7f575c169000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 --2- 172.21.15.19:0/3632288404 >> v2:172.21.15.169:3300/0 conn(0x7f575c0cf270 0x7f575c169000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f575b400640 1 -- 172.21.15.19:0/3632288404 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c0764e0 con 0x7f575c1605c0 2024-04-25T11:43:15.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.481+0000 7f5760a00640 1 --2- 172.21.15.19:0/3632288404 >> v2:172.21.15.169:3300/0 conn(0x7f575c0cf270 0x7f575c169000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.482+0000 7f575b400640 1 --2- 172.21.15.19:0/3632288404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c1605c0 0x7f575c0ced30 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f575c0736e0 tx=0x7f574802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa8aa69f5f6e95c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.483+0000 7f575aa00640 1 -- 172.21.15.19:0/3632288404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748037070 con 0x7f575c1605c0 2024-04-25T11:43:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.483+0000 7f575aa00640 1 -- 172.21.15.19:0/3632288404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5748032440 con 0x7f575c1605c0 2024-04-25T11:43:15.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.483+0000 7f575aa00640 1 -- 172.21.15.19:0/3632288404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748032810 con 0x7f575c1605c0 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 -- 172.21.15.19:0/3632288404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c1605c0 msgr2=0x7f575c0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 --2- 172.21.15.19:0/3632288404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c1605c0 0x7f575c0ced30 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f575c0736e0 tx=0x7f574802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 -- 172.21.15.19:0/3632288404 shutdown_connections 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 --2- 172.21.15.19:0/3632288404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c1605c0 0x7f575c0ced30 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 --2- 172.21.15.19:0/3632288404 >> v2:172.21.15.169:3300/0 conn(0x7f575c0cf270 0x7f575c169000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 --2- 172.21.15.19:0/3632288404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c0b75f0 0x7f575c0ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 -- 172.21.15.19:0/3632288404 >> 172.21.15.19:0/3632288404 conn(0x7f575c09b7d0 msgr2=0x7f575c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:15.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 -- 172.21.15.19:0/3632288404 shutdown_connections 2024-04-25T11:43:15.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.484+0000 7f5761400640 1 -- 172.21.15.19:0/3632288404 wait complete. 2024-04-25T11:43:15.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.486+0000 7f5761400640 1 Processor -- start 2024-04-25T11:43:15.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.486+0000 7f5761400640 1 -- start start 2024-04-25T11:43:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f5761400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c0b75f0 0x7f575c14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f5761400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f575c0cf270 0x7f575c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f5761400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c1605c0 0x7f575c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f5761400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f575c076200 con 0x7f575c0b75f0 2024-04-25T11:43:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f5761400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f575c075f20 con 0x7f575c0cf270 2024-04-25T11:43:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f5761400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f575c06cb80 con 0x7f575c1605c0 2024-04-25T11:43:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f575be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c0b75f0 0x7f575c14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f575be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c0b75f0 0x7f575c14a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60522/0 (socket says 172.21.15.19:60522) 2024-04-25T11:43:15.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.488+0000 7f575be00640 1 -- 172.21.15.19:0/4207052388 learned_addr learned my addr 172.21.15.19:0/4207052388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575b400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f575c0cf270 0x7f575c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f5760a00640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c1605c0 0x7f575c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575be00640 1 -- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c1605c0 msgr2=0x7f575c0b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575be00640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c1605c0 0x7f575c0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575be00640 1 -- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f575c0cf270 msgr2=0x7f575c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575be00640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f575c0cf270 0x7f575c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575be00640 1 -- 172.21.15.19:0/4207052388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c15e670 con 0x7f575c0b75f0 2024-04-25T11:43:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575b400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f575c0cf270 0x7f575c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f575be00640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c0b75f0 0x7f575c14a120 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f574c00c570 tx=0x7f574c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.489+0000 7f5753e00640 1 -- 172.21.15.19:0/4207052388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c015070 con 0x7f575c0b75f0 2024-04-25T11:43:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.490+0000 7f5753e00640 1 -- 172.21.15.19:0/4207052388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f574c004220 con 0x7f575c0b75f0 2024-04-25T11:43:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.490+0000 7f5753e00640 1 -- 172.21.15.19:0/4207052388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c003610 con 0x7f575c0b75f0 2024-04-25T11:43:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.490+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c15e950 con 0x7f575c0b75f0 2024-04-25T11:43:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.490+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c0b4500 con 0x7f575c0b75f0 2024-04-25T11:43:15.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.491+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f575c050740 con 0x7f575c0b75f0 2024-04-25T11:43:15.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.492+0000 7f5753e00640 1 -- 172.21.15.19:0/4207052388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f574c0037b0 con 0x7f575c0b75f0 2024-04-25T11:43:15.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.493+0000 7f5753e00640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f573c0744b0 0x7f573c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.493+0000 7f5753e00640 1 -- 172.21.15.19:0/4207052388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f574c094d00 con 0x7f575c0b75f0 2024-04-25T11:43:15.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.494+0000 7f575b400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f573c0744b0 0x7f573c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.495+0000 7f575b400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f573c0744b0 0x7f573c0769c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5748034000 tx=0x7f574802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:15.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.498+0000 7f5753e00640 1 -- 172.21.15.19:0/4207052388 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f574c05f690 con 0x7f575c0b75f0 2024-04-25T11:43:15.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.700+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f575c0b14c0 con 0x7f573c0744b0 2024-04-25T11:43:15.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.702+0000 7f5753e00640 1 -- 172.21.15.19:0/4207052388 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f575c0b14c0 con 0x7f573c0744b0 2024-04-25T11:43:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:15.689 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:43:15.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f573c0744b0 msgr2=0x7f573c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f573c0744b0 0x7f573c0769c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5748034000 tx=0x7f574802e000 comp rx=0 tx=0).stop 2024-04-25T11:43:15.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c0b75f0 msgr2=0x7f575c14a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c0b75f0 0x7f575c14a120 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f574c00c570 tx=0x7f574c00ca30 comp rx=0 tx=0).stop 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 shutdown_connections 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f573c0744b0 0x7f573c0769c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f575c0cf270 0x7f575c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f575c1605c0 0x7f575c0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 --2- 172.21.15.19:0/4207052388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c0b75f0 0x7f575c14a120 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 >> 172.21.15.19:0/4207052388 conn(0x7f575c09b7d0 msgr2=0x7f575c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 shutdown_connections 2024-04-25T11:43:15.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.705+0000 7f5761400640 1 -- 172.21.15.19:0/4207052388 wait complete. 2024-04-25T11:43:15.718 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":623,"stamp":"2024-04-25T11:43:14.849870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":302840,"kb_used_data":15960,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671880,"statfs":{"total":773094113280,"available":772784005120,"internally_reserved":0,"allocated":16343040,"data_stored":8458749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36338,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-101,"num_write_kb":-71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.008535"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":360,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T11:43:11.306052+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:43:11.306052+0000","last_peered":"2024-04-25T11:43:11.306052+0000","last_clean":"2024-04-25T11:43:11.306052+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:43:11.306052+0000","last_undegraded":"2024-04-25T11:43:11.306052+0000","last_fullsized":"2024-04-25T11:43:11.306052+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509635,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509636,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33032,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37772,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334068,"statfs":{"total":96636764160,"available":96598085632,"internally_reserved":0,"allocated":2490368,"data_stored":1500826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509635,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542338,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2473984,"data_stored":1495175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:43:15.718 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:43:15.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:43:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.995+0000 7f21b8600640 1 Processor -- start 2024-04-25T11:43:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.996+0000 7f21b8600640 1 -- start start 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.997+0000 7f21b8600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21b01434d0 0x7f21b0070480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b8600640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f21b01612f0 0x7f21b00d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b8600640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b00709c0 con 0x7f21b00d0ed0 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b8600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f21b01438e0 con 0x7f21b01434d0 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b8600640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f21b006f3e0 con 0x7f21b01612f0 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21b01434d0 0x7f21b0070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21b01434d0 0x7f21b0070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60532/0 (socket says 172.21.15.19:60532) 2024-04-25T11:43:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b5c00640 1 -- 172.21.15.19:0/3621505034 learned_addr learned my addr 172.21.15.19:0/3621505034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.998+0000 7f21b6600640 1 --2- 172.21.15.19:0/3621505034 >> v2:172.21.15.169:3300/0 conn(0x7f21b01612f0 0x7f21b00d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.999+0000 7f21b5c00640 1 -- 172.21.15.19:0/3621505034 >> v2:172.21.15.169:3300/0 conn(0x7f21b01612f0 msgr2=0x7f21b00d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.999+0000 7f21b5c00640 1 --2- 172.21.15.19:0/3621505034 >> v2:172.21.15.169:3300/0 conn(0x7f21b01612f0 0x7f21b00d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.999+0000 7f21b5c00640 1 -- 172.21.15.19:0/3621505034 >> v1:172.21.15.169:6789/0 conn(0x7f21b00d0ed0 legacy=0x7f21b0160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:15.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:15.999+0000 7f21b5c00640 1 -- 172.21.15.19:0/3621505034 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b006f6c0 con 0x7f21b01434d0 2024-04-25T11:43:15.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.000+0000 7f21b5c00640 1 --2- 172.21.15.19:0/3621505034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b01434d0 0x7f21b0070480 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009870 tx=0x7f21a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60570c9a64d22c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:15.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.000+0000 7f21afe00640 1 -- 172.21.15.19:0/3621505034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4037070 con 0x7f21b01434d0 2024-04-25T11:43:15.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.000+0000 7f21afe00640 1 -- 172.21.15.19:0/3621505034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a4032440 con 0x7f21b01434d0 2024-04-25T11:43:15.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.000+0000 7f21afe00640 1 -- 172.21.15.19:0/3621505034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4032810 con 0x7f21b01434d0 2024-04-25T11:43:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 -- 172.21.15.19:0/3621505034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b01434d0 msgr2=0x7f21b0070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 --2- 172.21.15.19:0/3621505034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b01434d0 0x7f21b0070480 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009870 tx=0x7f21a402a270 comp rx=0 tx=0).stop 2024-04-25T11:43:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 -- 172.21.15.19:0/3621505034 shutdown_connections 2024-04-25T11:43:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 --2- 172.21.15.19:0/3621505034 >> v2:172.21.15.169:3300/0 conn(0x7f21b01612f0 0x7f21b00d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 --2- 172.21.15.19:0/3621505034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b01434d0 0x7f21b0070480 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 -- 172.21.15.19:0/3621505034 >> 172.21.15.19:0/3621505034 conn(0x7f21b00d1ae0 msgr2=0x7f21b00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:15.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 -- 172.21.15.19:0/3621505034 shutdown_connections 2024-04-25T11:43:15.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.001+0000 7f21b8600640 1 -- 172.21.15.19:0/3621505034 wait complete. 2024-04-25T11:43:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.004+0000 7f21b8600640 1 Processor -- start 2024-04-25T11:43:15.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.004+0000 7f21b8600640 1 -- start start 2024-04-25T11:43:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b8600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21b00d0ed0 0x7f21b015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b8600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21b01434d0 0x7f21b0143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b8600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b013f5c0 0x7f21b0143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b8600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21b006f090 con 0x7f21b013f5c0 2024-04-25T11:43:15.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b8600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f21b006fc80 con 0x7f21b00d0ed0 2024-04-25T11:43:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b8600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f21b00b51a0 con 0x7f21b01434d0 2024-04-25T11:43:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b013f5c0 0x7f21b0143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b013f5c0 0x7f21b0143fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60538/0 (socket says 172.21.15.19:60538) 2024-04-25T11:43:15.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.006+0000 7f21b6600640 1 -- 172.21.15.19:0/3677404539 learned_addr learned my addr 172.21.15.19:0/3677404539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b5200640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21b01434d0 0x7f21b0143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b5c00640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21b00d0ed0 0x7f21b015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b6600640 1 -- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21b01434d0 msgr2=0x7f21b0143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b6600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21b01434d0 0x7f21b0143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b6600640 1 -- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21b00d0ed0 msgr2=0x7f21b015aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:15.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b6600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21b00d0ed0 0x7f21b015aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b6600640 1 -- 172.21.15.19:0/3677404539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b0160880 con 0x7f21b013f5c0 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b5c00640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21b00d0ed0 0x7f21b015aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b5200640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21b01434d0 0x7f21b0143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21b6600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b013f5c0 0x7f21b0143fa0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f21a8007e90 tx=0x7f21a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.007+0000 7f21ad600640 1 -- 172.21.15.19:0/3677404539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8014420 con 0x7f21b013f5c0 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.008+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b0145160 con 0x7f21b013f5c0 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.008+0000 7f21ad600640 1 -- 172.21.15.19:0/3677404539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a8019070 con 0x7f21b013f5c0 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.008+0000 7f21ad600640 1 -- 172.21.15.19:0/3677404539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8014800 con 0x7f21b013f5c0 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.008+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b015f450 con 0x7f21b013f5c0 2024-04-25T11:43:15.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.009+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b015f6b0 con 0x7f21b013f5c0 2024-04-25T11:43:15.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.010+0000 7f21ad600640 1 -- 172.21.15.19:0/3677404539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f21a80149a0 con 0x7f21b013f5c0 2024-04-25T11:43:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.011+0000 7f21ad600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21900744b0 0x7f21900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:15.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.011+0000 7f21ad600640 1 -- 172.21.15.19:0/3677404539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f21a80958a0 con 0x7f21b013f5c0 2024-04-25T11:43:15.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.012+0000 7f21b5c00640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21900744b0 0x7f21900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.013+0000 7f21b5c00640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21900744b0 0x7f21900769c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f21a4034000 tx=0x7f21a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:16.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.016+0000 7f21ad600640 1 -- 172.21.15.19:0/3677404539 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21b0160880 con 0x7f21b013f5c0 2024-04-25T11:43:16.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.217+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21b0050740 con 0x7f21900744b0 2024-04-25T11:43:16.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.220+0000 7f21ad600640 1 -- 172.21.15.19:0/3677404539 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f21b0050740 con 0x7f21900744b0 2024-04-25T11:43:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:16.206 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:43:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.223+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21900744b0 msgr2=0x7f21900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.223+0000 7f21b8600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21900744b0 0x7f21900769c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f21a4034000 tx=0x7f21a402e000 comp rx=0 tx=0).stop 2024-04-25T11:43:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.223+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b013f5c0 msgr2=0x7f21b0143fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.223+0000 7f21b8600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b013f5c0 0x7f21b0143fa0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f21a8007e90 tx=0x7f21a800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 shutdown_connections 2024-04-25T11:43:16.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f21900744b0 0x7f21900769c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21b013f5c0 0x7f21b0143fa0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f21b01434d0 0x7f21b0143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 --2- 172.21.15.19:0/3677404539 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f21b00d0ed0 0x7f21b015aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 >> 172.21.15.19:0/3677404539 conn(0x7f21b00d1ae0 msgr2=0x7f21b00d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:16.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 shutdown_connections 2024-04-25T11:43:16.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.224+0000 7f21b8600640 1 -- 172.21.15.19:0/3677404539 wait complete. 2024-04-25T11:43:16.233 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":623,"stamp":"2024-04-25T11:43:14.849870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":302840,"kb_used_data":15960,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671880,"statfs":{"total":773094113280,"available":772784005120,"internally_reserved":0,"allocated":16343040,"data_stored":8458749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36338,"num_objects":-27,"num_object_clones":0,"num_object_copies":-54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-27,"num_whiteouts":0,"num_read":-90,"num_read_kb":-92,"num_write":-101,"num_write_kb":-71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.008535"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":360,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T11:43:11.306052+0000","last_change":"2024-04-25T11:27:14.421912+0000","last_active":"2024-04-25T11:43:11.306052+0000","last_peered":"2024-04-25T11:43:11.306052+0000","last_clean":"2024-04-25T11:43:11.306052+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:43:11.306052+0000","last_undegraded":"2024-04-25T11:43:11.306052+0000","last_fullsized":"2024-04-25T11:43:11.306052+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:27:13.389809+0000","last_clean_scrub_stamp":"2024-04-25T11:27:13.389809+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:24.975702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509635,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509636,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33032,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509636,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37772,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334068,"statfs":{"total":96636764160,"available":96598085632,"internally_reserved":0,"allocated":2490368,"data_stored":1500826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509635,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542338,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2473984,"data_stored":1495175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:43:16.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.503+0000 7ff78b200640 1 Processor -- start 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.504+0000 7ff78b200640 1 -- start start 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.506+0000 7ff78b200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff7840b6270 0x7ff7840b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.506+0000 7ff78b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 0x7ff784071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.506+0000 7ff78b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7840720d0 con 0x7ff784163db0 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.506+0000 7ff78b200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7ff7840b6ca0 con 0x7ff7840b6270 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.506+0000 7ff78b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff7841613b0 con 0x7ff784071780 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.507+0000 7ff78a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff784163db0 0x7ff7841641c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41596/0 (socket says 172.21.15.19:41596) 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.507+0000 7ff78a800640 1 -- 172.21.15.19:0/1569344064 learned_addr learned my addr 172.21.15.19:0/1569344064 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.507+0000 7ff789400640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 0x7ff784071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.507+0000 7ff789e00640 1 --2- 172.21.15.19:0/1569344064 >> v2:172.21.15.169:3300/0 conn(0x7ff7840b6270 0x7ff7840b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.507+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629748862 0 0) 0x7ff780001490 con 0x7ff784163db0 2024-04-25T11:43:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.507+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff76c003520 con 0x7ff784163db0 2024-04-25T11:43:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 749131707 0 0) 0x7ff76c003520 con 0x7ff784163db0 2024-04-25T11:43:16.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 msgr2=0x7ff784071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 0x7ff784071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 >> v2:172.21.15.169:3300/0 conn(0x7ff7840b6270 msgr2=0x7ff7840b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 --2- 172.21.15.19:0/1569344064 >> v2:172.21.15.169:3300/0 conn(0x7ff7840b6270 0x7ff7840b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7841607c0 con 0x7ff784163db0 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7ff780001bc0 con 0x7ff784163db0 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 >> v1:172.21.15.19:6789/0 conn(0x7ff784163db0 legacy=0x7ff7841641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0039b0 0x7ff76c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 0x7ff76c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c004d40 0x7ff76c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7840b6ca0 con 0x7ff76c004d40 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841613b0 con 0x7ff76c0039b0 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff76c00ba70 con 0x7ff784071780 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff78a800640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c004d40 0x7ff76c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff789e00640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0039b0 0x7ff76c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.508+0000 7ff789400640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 0x7ff76c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.509+0000 7ff78a800640 1 -- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 msgr2=0x7ff76c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.509+0000 7ff78a800640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff784071780 0x7ff76c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.509+0000 7ff78a800640 1 -- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0039b0 msgr2=0x7ff76c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.509+0000 7ff78a800640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff76c0039b0 0x7ff76c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.509+0000 7ff78a800640 1 -- 172.21.15.19:0/1569344064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c003520 con 0x7ff76c004d40 2024-04-25T11:43:16.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.510+0000 7ff78a800640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c004d40 0x7ff76c005150 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7ff7840517c0 tx=0x7ff76c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db20f68d35be186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:16.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.510+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78000a5d0 con 0x7ff76c004d40 2024-04-25T11:43:16.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.510+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff78002f070 con 0x7ff76c004d40 2024-04-25T11:43:16.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.510+0000 7ff788a00640 1 -- 172.21.15.19:0/1569344064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78002bdb0 con 0x7ff76c004d40 2024-04-25T11:43:16.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.511+0000 7ff78b200640 1 -- 172.21.15.19:0/1569344064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c004d40 msgr2=0x7ff76c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.511+0000 7ff78b200640 1 --2- 172.21.15.19:0/1569344064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c004d40 0x7ff76c005150 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7ff7840517c0 tx=0x7ff76c005830 comp rx=0 tx=0).stop 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.511+0000 7ff78a800640 1 -- 172.21.15.19:0/1569344064 reap_dead start 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.511+0000 7ff78b200640 1 -- 172.21.15.19:0/1569344064 shutdown_connections 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.511+0000 7ff78b200640 1 -- 172.21.15.19:0/1569344064 >> 172.21.15.19:0/1569344064 conn(0x7ff784070270 msgr2=0x7ff784070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.511+0000 7ff78b200640 1 -- 172.21.15.19:0/1569344064 shutdown_connections 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.511+0000 7ff78b200640 1 -- 172.21.15.19:0/1569344064 wait complete. 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.513+0000 7ff78b200640 1 Processor -- start 2024-04-25T11:43:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.513+0000 7ff78b200640 1 -- start start 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78b200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841f6900 0x7ff784149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78b200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff78415c420 0x7ff78415c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff78415e280 0x7ff78415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff784161040 con 0x7ff78415e280 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78b200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff784160d60 con 0x7ff7841f6900 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78b200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7ff7840b1c20 con 0x7ff78415c420 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff78415e280 0x7ff78415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff78415e280 0x7ff78415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60550/0 (socket says 172.21.15.19:60550) 2024-04-25T11:43:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.515+0000 7ff78a800640 1 -- 172.21.15.19:0/2855859191 learned_addr learned my addr 172.21.15.19:0/2855859191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff789400640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff78415c420 0x7ff78415c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff789e00640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841f6900 0x7ff784149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff78a800640 1 -- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff78415c420 msgr2=0x7ff78415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff78a800640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff78415c420 0x7ff78415c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff78a800640 1 -- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841f6900 msgr2=0x7ff784149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff78a800640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841f6900 0x7ff784149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff78a800640 1 -- 172.21.15.19:0/2855859191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78415cd70 con 0x7ff78415e280 2024-04-25T11:43:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff78a800640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff78415e280 0x7ff78415e690 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7ff78000aa70 tx=0x7ff780001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff77e000640 1 -- 172.21.15.19:0/2855859191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78000a5d0 con 0x7ff78415e280 2024-04-25T11:43:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff77e000640 1 -- 172.21.15.19:0/2855859191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff78002bd80 con 0x7ff78415e280 2024-04-25T11:43:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff77e000640 1 -- 172.21.15.19:0/2855859191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7800406c0 con 0x7ff78415e280 2024-04-25T11:43:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.516+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78414a780 con 0x7ff78415e280 2024-04-25T11:43:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.517+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff784147eb0 con 0x7ff78415e280 2024-04-25T11:43:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.518+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff784050740 con 0x7ff78415e280 2024-04-25T11:43:16.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.519+0000 7ff77e000640 1 -- 172.21.15.19:0/2855859191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff780040860 con 0x7ff78415e280 2024-04-25T11:43:16.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.520+0000 7ff77e000640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7680744b0 0x7ff7680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.520+0000 7ff77e000640 1 -- 172.21.15.19:0/2855859191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7ff7800b5240 con 0x7ff78415e280 2024-04-25T11:43:16.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.520+0000 7ff789e00640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7680744b0 0x7ff7680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.522+0000 7ff789e00640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7680744b0 0x7ff7680769c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff7841618e0 tx=0x7ff778008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.525+0000 7ff77e000640 1 -- 172.21.15.19:0/2855859191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff78415cd70 con 0x7ff78415e280 2024-04-25T11:43:16.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.773+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7840b1660 con 0x7ff78415e280 2024-04-25T11:43:16.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.775+0000 7ff77e000640 1 -- 172.21.15.19:0/2855859191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+13162 (secure 0 0 0) 0x7ff7840720d0 con 0x7ff78415e280 2024-04-25T11:43:16.762 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:16.762 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":148,"fsid":"23a56573-5965-4bbb-bd60-741a44babbed","created":"2024-04-25T11:26:59.218749+0000","modified":"2024-04-25T11:43:11.287963+0000","last_up_change":"2024-04-25T11:27:12.384716+0000","last_in_change":"2024-04-25T11:27:03.178286+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:27:12.475086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ccfa8d67-0c42-4220-8b9c-0c14439eb128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6809","nonce":1318554626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6811","nonce":1318554626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6818","nonce":1318554626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1318554626},{"type":"v1","addr":"172.21.15.19:6814","nonce":1318554626}]},"public_addr":"172.21.15.19:6809/1318554626","cluster_addr":"172.21.15.19:6811/1318554626","heartbeat_back_addr":"172.21.15.19:6818/1318554626","heartbeat_front_addr":"172.21.15.19:6814/1318554626","state":["exists","up"]},{"osd":1,"uuid":"dbc3e03d-1ba0-4402-aede-39b9bc0d5b69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6801","nonce":856233193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6803","nonce":856233193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6807","nonce":856233193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":856233193},{"type":"v1","addr":"172.21.15.19:6805","nonce":856233193}]},"public_addr":"172.21.15.19:6801/856233193","cluster_addr":"172.21.15.19:6803/856233193","heartbeat_back_addr":"172.21.15.19:6807/856233193","heartbeat_front_addr":"172.21.15.19:6805/856233193","state":["exists","up"]},{"osd":2,"uuid":"7b9ccdd0-9b59-4ebe-9159-2c1c2ae4008c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6815","nonce":2392070031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6817","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6819","nonce":2392070031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6823","nonce":2392070031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2392070031},{"type":"v1","addr":"172.21.15.19:6821","nonce":2392070031}]},"public_addr":"172.21.15.19:6815/2392070031","cluster_addr":"172.21.15.19:6819/2392070031","heartbeat_back_addr":"172.21.15.19:6823/2392070031","heartbeat_front_addr":"172.21.15.19:6821/2392070031","state":["exists","up"]},{"osd":3,"uuid":"73047778-1fdc-4988-90c3-b714ba889607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6825","nonce":2286957460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6827","nonce":2286957460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6831","nonce":2286957460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":2286957460},{"type":"v1","addr":"172.21.15.19:6829","nonce":2286957460}]},"public_addr":"172.21.15.19:6825/2286957460","cluster_addr":"172.21.15.19:6827/2286957460","heartbeat_back_addr":"172.21.15.19:6831/2286957460","heartbeat_front_addr":"172.21.15.19:6829/2286957460","state":["exists","up"]},{"osd":4,"uuid":"de754db2-fb58-454c-a7f6-8ede347e3b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6817","nonce":3656468664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6819","nonce":3656468664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6823","nonce":3656468664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3656468664},{"type":"v1","addr":"172.21.15.169:6821","nonce":3656468664}]},"public_addr":"172.21.15.169:6817/3656468664","cluster_addr":"172.21.15.169:6819/3656468664","heartbeat_back_addr":"172.21.15.169:6823/3656468664","heartbeat_front_addr":"172.21.15.169:6821/3656468664","state":["exists","up"]},{"osd":5,"uuid":"0f0ab79f-f6b2-4d41-95d8-0e6f58c6a517","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6809","nonce":2942667610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6811","nonce":2942667610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6815","nonce":2942667610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":2942667610},{"type":"v1","addr":"172.21.15.169:6813","nonce":2942667610}]},"public_addr":"172.21.15.169:6809/2942667610","cluster_addr":"172.21.15.169:6811/2942667610","heartbeat_back_addr":"172.21.15.169:6815/2942667610","heartbeat_front_addr":"172.21.15.169:6813/2942667610","state":["exists","up"]},{"osd":6,"uuid":"c44cb1d3-012a-4d38-beec-cd774e53493c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6825","nonce":2510135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6827","nonce":2510135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6831","nonce":2510135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2510135033},{"type":"v1","addr":"172.21.15.169:6829","nonce":2510135033}]},"public_addr":"172.21.15.169:6825/2510135033","cluster_addr":"172.21.15.169:6827/2510135033","heartbeat_back_addr":"172.21.15.169:6831/2510135033","heartbeat_front_addr":"172.21.15.169:6829/2510135033","state":["exists","up"]},{"osd":7,"uuid":"e35356bf-5d6d-4f56-96a7-208752898bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6801","nonce":2355270624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6803","nonce":2355270624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6807","nonce":2355270624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2355270624},{"type":"v1","addr":"172.21.15.169:6805","nonce":2355270624}]},"public_addr":"172.21.15.169:6801/2355270624","cluster_addr":"172.21.15.169:6803/2355270624","heartbeat_back_addr":"172.21.15.169:6807/2355270624","heartbeat_front_addr":"172.21.15.169:6805/2355270624","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.766371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.873113+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.756281+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.791755+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.702764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.348463+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.758950+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:27:08.553924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/3539191189":"2024-04-26T11:43:07.833797+0000","172.21.15.19:6832/3539191189":"2024-04-26T11:43:07.833797+0000","172.21.15.169:6835/2610059988":"2024-04-26T11:42:44.471606+0000","172.21.15.169:6834/2610059988":"2024-04-26T11:42:44.471606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7680744b0 msgr2=0x7ff7680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7680744b0 0x7ff7680769c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff7841618e0 tx=0x7ff778008040 comp rx=0 tx=0).stop 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff78415e280 msgr2=0x7ff78415e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff78415e280 0x7ff78415e690 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7ff78000aa70 tx=0x7ff780001360 comp rx=0 tx=0).stop 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 shutdown_connections 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7ff7680744b0 0x7ff7680769c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff78415e280 0x7ff78415e690 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff78415c420 0x7ff78415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 --2- 172.21.15.19:0/2855859191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7841f6900 0x7ff784149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.778+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 >> 172.21.15.19:0/2855859191 conn(0x7ff784070270 msgr2=0x7ff78415d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.779+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 shutdown_connections 2024-04-25T11:43:16.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:16.779+0000 7ff78b200640 1 -- 172.21.15.19:0/2855859191 wait complete. 2024-04-25T11:43:17.789 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T11:43:17.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.100+0000 7fd1bb200640 1 Processor -- start 2024-04-25T11:43:18.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.101+0000 7fd1bb200640 1 -- start start 2024-04-25T11:43:18.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1bb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1b4144340 0x7fd1b406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1bb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406db10 0x7fd1b415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1bb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd1b415ec90 0x7fd1b4161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1bb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b4144750 con 0x7fd1b4144340 2024-04-25T11:43:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1bb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b406df20 con 0x7fd1b415ec90 2024-04-25T11:43:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1bb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b4070000 con 0x7fd1b406db10 2024-04-25T11:43:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1b9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1b4144340 0x7fd1b406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1b9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1b4144340 0x7fd1b406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60556/0 (socket says 172.21.15.19:60556) 2024-04-25T11:43:18.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.103+0000 7fd1b9e00640 1 -- 172.21.15.19:0/712311308 learned_addr learned my addr 172.21.15.19:0/712311308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1ba800640 1 --2- 172.21.15.19:0/712311308 >> v2:172.21.15.169:3300/0 conn(0x7fd1b415ec90 0x7fd1b4161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1b9400640 1 --2- 172.21.15.19:0/712311308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406db10 0x7fd1b415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1b9e00640 1 -- 172.21.15.19:0/712311308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406db10 msgr2=0x7fd1b415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1b9e00640 1 --2- 172.21.15.19:0/712311308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406db10 0x7fd1b415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1b9e00640 1 -- 172.21.15.19:0/712311308 >> v2:172.21.15.169:3300/0 conn(0x7fd1b415ec90 msgr2=0x7fd1b4161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1b9e00640 1 --2- 172.21.15.19:0/712311308 >> v2:172.21.15.169:3300/0 conn(0x7fd1b415ec90 0x7fd1b4161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1b9e00640 1 -- 172.21.15.19:0/712311308 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b406f240 con 0x7fd1b4144340 2024-04-25T11:43:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1ba800640 1 --2- 172.21.15.19:0/712311308 >> v2:172.21.15.169:3300/0 conn(0x7fd1b415ec90 0x7fd1b4161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.104+0000 7fd1b9400640 1 --2- 172.21.15.19:0/712311308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406db10 0x7fd1b415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:18.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.105+0000 7fd1b9e00640 1 --2- 172.21.15.19:0/712311308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b4144340 0x7fd1b406d5d0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4147c30 tx=0x7fd1a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123e8d34c85744b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1b8a00640 1 -- 172.21.15.19:0/712311308 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8037070 con 0x7fd1b4144340 2024-04-25T11:43:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1b8a00640 1 -- 172.21.15.19:0/712311308 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8032440 con 0x7fd1b4144340 2024-04-25T11:43:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1b8a00640 1 -- 172.21.15.19:0/712311308 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8032810 con 0x7fd1b4144340 2024-04-25T11:43:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1bb200640 1 -- 172.21.15.19:0/712311308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b4144340 msgr2=0x7fd1b406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1bb200640 1 --2- 172.21.15.19:0/712311308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b4144340 0x7fd1b406d5d0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4147c30 tx=0x7fd1a802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1bb200640 1 -- 172.21.15.19:0/712311308 shutdown_connections 2024-04-25T11:43:18.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1bb200640 1 --2- 172.21.15.19:0/712311308 >> v2:172.21.15.169:3300/0 conn(0x7fd1b415ec90 0x7fd1b4161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1bb200640 1 --2- 172.21.15.19:0/712311308 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b406db10 0x7fd1b415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1bb200640 1 --2- 172.21.15.19:0/712311308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b4144340 0x7fd1b406d5d0 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.106+0000 7fd1bb200640 1 -- 172.21.15.19:0/712311308 >> 172.21.15.19:0/712311308 conn(0x7fd1b4072f40 msgr2=0x7fd1b4073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.107+0000 7fd1bb200640 1 -- 172.21.15.19:0/712311308 shutdown_connections 2024-04-25T11:43:18.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.107+0000 7fd1bb200640 1 -- 172.21.15.19:0/712311308 wait complete. 2024-04-25T11:43:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.109+0000 7fd1bb200640 1 Processor -- start 2024-04-25T11:43:18.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.109+0000 7fd1bb200640 1 -- start start 2024-04-25T11:43:18.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1bb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406db10 0x7fd1b40b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1bb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144340 0x7fd1b4141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1bb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b415ec90 0x7fd1b413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1bb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b406fae0 con 0x7fd1b406db10 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1bb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b406f800 con 0x7fd1b415ec90 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1bb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b415f0a0 con 0x7fd1b4144340 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406db10 0x7fd1b40b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406db10 0x7fd1b40b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60566/0 (socket says 172.21.15.19:60566) 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 -- 172.21.15.19:0/2181863549 learned_addr learned my addr 172.21.15.19:0/2181863549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1ba800640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b415ec90 0x7fd1b413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 -- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144340 msgr2=0x7fd1b4141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9400640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144340 0x7fd1b4141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1b4144340 0x7fd1b4141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 -- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b415ec90 msgr2=0x7fd1b413e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1b415ec90 0x7fd1b413e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.111+0000 7fd1b9e00640 1 -- 172.21.15.19:0/2181863549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b415ab90 con 0x7fd1b406db10 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.112+0000 7fd1b9e00640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406db10 0x7fd1b40b5fa0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.112+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8037040 con 0x7fd1b406db10 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.112+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8042070 con 0x7fd1b406db10 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.112+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b415ae70 con 0x7fd1b406db10 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.112+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8030bb0 con 0x7fd1b406db10 2024-04-25T11:43:18.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.112+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b415b430 con 0x7fd1b406db10 2024-04-25T11:43:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.113+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1b4076c60 con 0x7fd1b406db10 2024-04-25T11:43:18.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.115+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd1a8047460 con 0x7fd1b406db10 2024-04-25T11:43:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.116+0000 7fd1ae000640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1980744e0 0x7fd1980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.116+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fd1a80b6640 con 0x7fd1b406db10 2024-04-25T11:43:18.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.116+0000 7fd1ae000640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7fd19807a490 0x7fd19807c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.116+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1a80067f0 con 0x7fd19807a490 2024-04-25T11:43:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.116+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fd1a80b69f0 con 0x7fd1b406db10 2024-04-25T11:43:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.116+0000 7fd1ba800640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7fd19807a490 0x7fd19807c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.117+0000 7fd1b9400640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1980744e0 0x7fd1980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.117+0000 7fd1ba800640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7fd19807a490 0x7fd19807c8e0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.118+0000 7fd1b9400640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1980744e0 0x7fd1980769f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd1b415a320 tx=0x7fd1a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.118+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== osd.0 v2:172.21.15.19:6808/1318554626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1a80067f0 con 0x7fd19807a490 2024-04-25T11:43:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.132+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 --> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd1b40b7920 con 0x7fd19807a490 2024-04-25T11:43:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.133+0000 7fd1ae000640 1 -- 172.21.15.19:0/2181863549 <== osd.0 v2:172.21.15.19:6808/1318554626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd1b40b7920 con 0x7fd19807a490 2024-04-25T11:43:18.119 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:18.119 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:18.119 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.133+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7fd19807a490 msgr2=0x7fd19807c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.133+0000 7fd1bb200640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:6808/1318554626,v1:172.21.15.19:6809/1318554626] conn(0x7fd19807a490 0x7fd19807c8e0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1980744e0 msgr2=0x7fd1980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fd1980744e0 0x7fd1980769f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd1b415a320 tx=0x7fd1a4008040 comp rx=0 tx=0).stop 2024-04-25T11:43:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406db10 msgr2=0x7fd1b40b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 --2- 172.21.15.19:0/2181863549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1b406db10 0x7fd1b40b5fa0 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a80046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1ba800640 1 -- 172.21.15.19:0/2181863549 reap_dead start 2024-04-25T11:43:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 shutdown_connections 2024-04-25T11:43:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 >> 172.21.15.19:0/2181863549 conn(0x7fd1b4072f40 msgr2=0x7fd1b415c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 shutdown_connections 2024-04-25T11:43:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.134+0000 7fd1bb200640 1 -- 172.21.15.19:0/2181863549 wait complete. 2024-04-25T11:43:18.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T11:43:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.444+0000 7f2937000640 1 Processor -- start 2024-04-25T11:43:18.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.444+0000 7f2937000640 1 -- start start 2024-04-25T11:43:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.446+0000 7f2937000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 0x7f29300d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.446+0000 7f2937000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2930161650 con 0x7f2930164860 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.446+0000 7f2937000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2930161370 con 0x7f29300b7c50 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.446+0000 7f2937000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f29301643d0 con 0x7f29300b4d30 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.446+0000 7f2935c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2930164860 0x7f29300b7540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41612/0 (socket says 172.21.15.19:41612) 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.446+0000 7f2935c00640 1 -- 172.21.15.19:0/4081710866 learned_addr learned my addr 172.21.15.19:0/4081710866 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857145910 0 0) 0x7f2930161650 con 0x7f2930164860 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f2936600640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 0x7f29300d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2918003520 con 0x7f2930164860 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3735837921 0 0) 0x7f2918003520 con 0x7f2930164860 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 msgr2=0x7f29300d4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 0x7f29300d4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 >> v1:172.21.15.169:6789/0 conn(0x7f29300b7c50 legacy=0x7f29300d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2930163d80 con 0x7f2930164860 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7f2924001670 con 0x7f2930164860 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 >> v1:172.21.15.19:6789/0 conn(0x7f2930164860 legacy=0x7f29300b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 0x7f29180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2918004010 0x7f2918004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918004d40 0x7f2918005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29301643d0 con 0x7f2918004d40 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f291800bc50 con 0x7f2918004010 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.447+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f291800be50 con 0x7f29300b4d30 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2936600640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918004d40 0x7f2918005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2935c00640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 0x7f29180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2935200640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2918004010 0x7f2918004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2936600640 1 -- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 msgr2=0x7f29180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2936600640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29300b4d30 0x7f29180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2936600640 1 -- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2918004010 msgr2=0x7f2918004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2936600640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2918004010 0x7f2918004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2936600640 1 -- 172.21.15.19:0/4081710866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2918003520 con 0x7f2918004d40 2024-04-25T11:43:18.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.448+0000 7f2935200640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2918004010 0x7f2918004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.449+0000 7f2936600640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918004d40 0x7f2918005210 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f2928007330 tx=0x7f2918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876313fb55df8df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.450+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928033070 con 0x7f2918004d40 2024-04-25T11:43:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.450+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292802f660 con 0x7f2918004d40 2024-04-25T11:43:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.450+0000 7f292fe00640 1 -- 172.21.15.19:0/4081710866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928009be0 con 0x7f2918004d40 2024-04-25T11:43:18.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.451+0000 7f2937000640 1 -- 172.21.15.19:0/4081710866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918004d40 msgr2=0x7f2918005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.451+0000 7f2937000640 1 --2- 172.21.15.19:0/4081710866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2918004d40 0x7f2918005210 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f2928007330 tx=0x7f2918005830 comp rx=0 tx=0).stop 2024-04-25T11:43:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.451+0000 7f2936600640 1 -- 172.21.15.19:0/4081710866 reap_dead start 2024-04-25T11:43:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.451+0000 7f2937000640 1 -- 172.21.15.19:0/4081710866 shutdown_connections 2024-04-25T11:43:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.451+0000 7f2937000640 1 -- 172.21.15.19:0/4081710866 >> 172.21.15.19:0/4081710866 conn(0x7f29300d2720 msgr2=0x7f29300d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.451+0000 7f2937000640 1 -- 172.21.15.19:0/4081710866 shutdown_connections 2024-04-25T11:43:18.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.451+0000 7f2937000640 1 -- 172.21.15.19:0/4081710866 wait complete. 2024-04-25T11:43:18.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.454+0000 7f2937000640 1 Processor -- start 2024-04-25T11:43:18.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.455+0000 7f2937000640 1 -- start start 2024-04-25T11:43:18.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2937000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2918004d40 0x7f2930159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2937000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29300ca320 0x7f293015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2937000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29300ca730 0x7f293015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2937000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29300d4bb0 con 0x7f29300ca320 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2937000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f29300b5140 con 0x7f29300ca730 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2937000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f29300d4d30 con 0x7f2918004d40 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2935200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29300ca320 0x7f293015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2935200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29300ca320 0x7f293015ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60582/0 (socket says 172.21.15.19:60582) 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2935200640 1 -- 172.21.15.19:0/1549754861 learned_addr learned my addr 172.21.15.19:0/1549754861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2935c00640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2918004d40 0x7f2930159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.457+0000 7f2936600640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29300ca730 0x7f293015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2935200640 1 -- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2918004d40 msgr2=0x7f2930159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2935200640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2918004d40 0x7f2930159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2935200640 1 -- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29300ca730 msgr2=0x7f293015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2935200640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29300ca730 0x7f293015f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2935200640 1 -- 172.21.15.19:0/1549754861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29300cf080 con 0x7f29300ca320 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2935c00640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2918004d40 0x7f2930159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2935200640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29300ca320 0x7f293015ed20 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f291c00d7d0 tx=0x7f291c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f292d600640 1 -- 172.21.15.19:0/1549754861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c005070 con 0x7f29300ca320 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f292d600640 1 -- 172.21.15.19:0/1549754861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291c005230 con 0x7f29300ca320 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f292d600640 1 -- 172.21.15.19:0/1549754861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c007030 con 0x7f29300ca320 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.458+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29301f66e0 con 0x7f29300ca320 2024-04-25T11:43:18.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.459+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301f6ca0 con 0x7f29300ca320 2024-04-25T11:43:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.460+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2930050740 con 0x7f29300ca320 2024-04-25T11:43:18.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.461+0000 7f292d600640 1 -- 172.21.15.19:0/1549754861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f291c01a020 con 0x7f29300ca320 2024-04-25T11:43:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.462+0000 7f292d600640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29100744e0 0x7f29100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.462+0000 7f292d600640 1 -- 172.21.15.19:0/1549754861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f291c0945f0 con 0x7f29300ca320 2024-04-25T11:43:18.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.463+0000 7f2935c00640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29100744e0 0x7f29100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.464+0000 7f2935c00640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29100744e0 0x7f29100769f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f2924001cc0 tx=0x7f2924001db0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.469+0000 7f292d600640 1 -- 172.21.15.19:0/1549754861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f291c05f200 con 0x7f29300ca320 2024-04-25T11:43:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.677+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f29300b7cf0 con 0x7f29100744e0 2024-04-25T11:43:18.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.679+0000 7f292d600640 1 -- 172.21.15.19:0/1549754861 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f29300b7cf0 con 0x7f29100744e0 2024-04-25T11:43:18.665 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T11:43:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.681+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29100744e0 msgr2=0x7f29100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.681+0000 7f2937000640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29100744e0 0x7f29100769f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f2924001cc0 tx=0x7f2924001db0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.681+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29300ca320 msgr2=0x7f293015ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.681+0000 7f2937000640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29300ca320 0x7f293015ed20 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f291c00d7d0 tx=0x7f291c00dc90 comp rx=0 tx=0).stop 2024-04-25T11:43:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 shutdown_connections 2024-04-25T11:43:18.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f29100744e0 0x7f29100769f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29300ca730 0x7f293015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29300ca320 0x7f293015ed20 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 --2- 172.21.15.19:0/1549754861 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2918004d40 0x7f2930159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 >> 172.21.15.19:0/1549754861 conn(0x7f29300d2720 msgr2=0x7f29300b9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 shutdown_connections 2024-04-25T11:43:18.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.682+0000 7f2937000640 1 -- 172.21.15.19:0/1549754861 wait complete. 2024-04-25T11:43:18.697 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T11:43:18.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.997+0000 7fccee200640 1 Processor -- start 2024-04-25T11:43:18.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:18.998+0000 7fccee200640 1 -- start start 2024-04-25T11:43:18.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.000+0000 7fccee200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 0x7fcce806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.000+0000 7fccee200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcce80b50b0 0x7fcce80b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.000+0000 7fccee200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce806df20 con 0x7fcce814aac0 2024-04-25T11:43:18.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.000+0000 7fccee200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcce814aed0 con 0x7fcce80b50b0 2024-04-25T11:43:18.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.000+0000 7fccee200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcce806f460 con 0x7fcce806d5d0 2024-04-25T11:43:18.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fccece00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcce814aac0 0x7fcce81655a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41618/0 (socket says 172.21.15.19:41618) 2024-04-25T11:43:18.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fccece00640 1 -- 172.21.15.19:0/2406713256 learned_addr learned my addr 172.21.15.19:0/2406713256 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404760936 0 0) 0x7fccdc001110 con 0x7fcce814aac0 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7e00640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 0x7fcce806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccd0003520 con 0x7fcce814aac0 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcced800640 1 --2- 172.21.15.19:0/2406713256 >> v2:172.21.15.169:3300/0 conn(0x7fcce80b50b0 0x7fcce80b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1582380462 0 0) 0x7fccd0003520 con 0x7fcce814aac0 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 msgr2=0x7fcce806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 0x7fcce806d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 >> v2:172.21.15.169:3300/0 conn(0x7fcce80b50b0 msgr2=0x7fcce80b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 --2- 172.21.15.19:0/2406713256 >> v2:172.21.15.169:3300/0 conn(0x7fcce80b50b0 0x7fcce80b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce806f740 con 0x7fcce814aac0 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.001+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fccdc001840 con 0x7fcce814aac0 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 >> v1:172.21.15.19:6789/0 conn(0x7fcce814aac0 legacy=0x7fcce81655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7400640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00039b0 0x7fccd0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7400640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd0004800 0x7fccd0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7400640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 0x7fccd0005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcce814aed0 con 0x7fccd00039b0 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcce806f460 con 0x7fccd0004800 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fccd000bc30 con 0x7fcce806d5d0 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fccece00640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00039b0 0x7fccd0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcced800640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 0x7fccd0005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fcce7e00640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd0004800 0x7fccd0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fccece00640 1 -- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 msgr2=0x7fccd0005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fccece00640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcce806d5d0 0x7fccd0005280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fccece00640 1 -- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd0004800 msgr2=0x7fccd0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fccece00640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccd0004800 0x7fccd0004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.002+0000 7fccece00640 1 -- 172.21.15.19:0/2406713256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd0003520 con 0x7fccd00039b0 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.003+0000 7fccece00640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00039b0 0x7fccd0003e20 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7fccdc001eb0 tx=0x7fccd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9923d015981ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.004+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc036070 con 0x7fccd00039b0 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.004+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccdc005af0 con 0x7fccd00039b0 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.004+0000 7fcce7400640 1 -- 172.21.15.19:0/2406713256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc007e30 con 0x7fccd00039b0 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.004+0000 7fccee200640 1 -- 172.21.15.19:0/2406713256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00039b0 msgr2=0x7fccd0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.004+0000 7fccee200640 1 --2- 172.21.15.19:0/2406713256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccd00039b0 0x7fccd0003e20 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7fccdc001eb0 tx=0x7fccd0005830 comp rx=0 tx=0).stop 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.005+0000 7fcced800640 1 -- 172.21.15.19:0/2406713256 reap_dead start 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.005+0000 7fccee200640 1 -- 172.21.15.19:0/2406713256 shutdown_connections 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.005+0000 7fccee200640 1 -- 172.21.15.19:0/2406713256 >> 172.21.15.19:0/2406713256 conn(0x7fcce8072f40 msgr2=0x7fcce8073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.005+0000 7fccee200640 1 -- 172.21.15.19:0/2406713256 shutdown_connections 2024-04-25T11:43:18.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.005+0000 7fccee200640 1 -- 172.21.15.19:0/2406713256 wait complete. 2024-04-25T11:43:18.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.007+0000 7fccee200640 1 Processor -- start 2024-04-25T11:43:18.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.007+0000 7fccee200640 1 -- start start 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fccee200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd00039b0 0x7fcce814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fccee200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce81467c0 0x7fcce8161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fccee200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce8161f40 0x7fcce814b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fccee200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcce806ffe0 con 0x7fcce81467c0 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fccee200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcce806fd00 con 0x7fcce8161f40 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fccee200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcce80b4a90 con 0x7fccd00039b0 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fcce7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce81467c0 0x7fcce8161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fcce7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce81467c0 0x7fcce8161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60594/0 (socket says 172.21.15.19:60594) 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.009+0000 7fcce7e00640 1 -- 172.21.15.19:0/4073470805 learned_addr learned my addr 172.21.15.19:0/4073470805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fccece00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd00039b0 0x7fcce814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce7e00640 1 -- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd00039b0 msgr2=0x7fcce814c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce7e00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fccd00039b0 0x7fcce814c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce7e00640 1 -- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce8161f40 msgr2=0x7fcce814b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce7e00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcce8161f40 0x7fcce814b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:18.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce7e00640 1 -- 172.21.15.19:0/4073470805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce8162350 con 0x7fcce81467c0 2024-04-25T11:43:18.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce7e00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce81467c0 0x7fcce8161a00 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e5c0 tx=0x7fccd400ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:18.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4017070 con 0x7fcce81467c0 2024-04-25T11:43:18.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd4013610 con 0x7fcce81467c0 2024-04-25T11:43:18.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd400f040 con 0x7fcce81467c0 2024-04-25T11:43:18.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce814d110 con 0x7fcce81467c0 2024-04-25T11:43:18.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.010+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce814d6d0 con 0x7fcce81467c0 2024-04-25T11:43:18.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.011+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcce8076c60 con 0x7fcce81467c0 2024-04-25T11:43:18.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.013+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fccd401a070 con 0x7fcce81467c0 2024-04-25T11:43:19.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.014+0000 7fcce4c00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc80744e0 0x7fccc80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.014+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fccd4094ab0 con 0x7fcce81467c0 2024-04-25T11:43:19.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.014+0000 7fcce4c00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fccc807a490 0x7fccc807c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.014+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccc807cfb0 con 0x7fccc807a490 2024-04-25T11:43:19.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.014+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fccd40992a0 con 0x7fcce81467c0 2024-04-25T11:43:19.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.014+0000 7fcced800640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fccc807a490 0x7fccc807c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.014+0000 7fccece00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc80744e0 0x7fccc80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.015+0000 7fcced800640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fccc807a490 0x7fccc807c8e0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.016+0000 7fccece00640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc80744e0 0x7fccc80769f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fccdc033000 tx=0x7fccdc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.016+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== osd.1 v2:172.21.15.19:6800/856233193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fccc807cfb0 con 0x7fccc807a490 2024-04-25T11:43:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.031+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 --> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcce814cec0 con 0x7fccc807a490 2024-04-25T11:43:19.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.031+0000 7fcce4c00640 1 -- 172.21.15.19:0/4073470805 <== osd.1 v2:172.21.15.19:6800/856233193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcce814cec0 con 0x7fccc807a490 2024-04-25T11:43:19.018 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:19.018 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:19.018 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:19.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fccc807a490 msgr2=0x7fccc807c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:6800/856233193,v1:172.21.15.19:6801/856233193] conn(0x7fccc807a490 0x7fccc807c8e0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc80744e0 msgr2=0x7fccc80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fccc80744e0 0x7fccc80769f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fccdc033000 tx=0x7fccdc02e000 comp rx=0 tx=0).stop 2024-04-25T11:43:19.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce81467c0 msgr2=0x7fcce8161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 --2- 172.21.15.19:0/4073470805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcce81467c0 0x7fcce8161a00 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e5c0 tx=0x7fccd400ea80 comp rx=0 tx=0).stop 2024-04-25T11:43:19.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fcced800640 1 -- 172.21.15.19:0/4073470805 reap_dead start 2024-04-25T11:43:19.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 shutdown_connections 2024-04-25T11:43:19.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 >> 172.21.15.19:0/4073470805 conn(0x7fcce8072f40 msgr2=0x7fcce8147910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:19.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 shutdown_connections 2024-04-25T11:43:19.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.032+0000 7fccee200640 1 -- 172.21.15.19:0/4073470805 wait complete. 2024-04-25T11:43:19.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T11:43:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.349+0000 7fcb33200640 1 Processor -- start 2024-04-25T11:43:19.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.349+0000 7fcb33200640 1 -- start start 2024-04-25T11:43:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb33200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcb340b75f0 0x7fcb340ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb33200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb34168000 0x7fcb3416a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb33200640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb340cfb40 con 0x7fcb341605c0 2024-04-25T11:43:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb33200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3406c6f0 con 0x7fcb34168000 2024-04-25T11:43:19.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb33200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fcb340764e0 con 0x7fcb340b75f0 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb32800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb34168000 0x7fcb3416a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb32800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb34168000 0x7fcb3416a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60610/0 (socket says 172.21.15.19:60610) 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.352+0000 7fcb32800640 1 -- 172.21.15.19:0/1927389298 learned_addr learned my addr 172.21.15.19:0/1927389298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.353+0000 7fcb31e00640 1 --2- 172.21.15.19:0/1927389298 >> v2:172.21.15.169:3300/0 conn(0x7fcb340b75f0 0x7fcb340ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.353+0000 7fcb32800640 1 -- 172.21.15.19:0/1927389298 >> v2:172.21.15.169:3300/0 conn(0x7fcb340b75f0 msgr2=0x7fcb340ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.353+0000 7fcb32800640 1 --2- 172.21.15.19:0/1927389298 >> v2:172.21.15.169:3300/0 conn(0x7fcb340b75f0 0x7fcb340ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.353+0000 7fcb32800640 1 -- 172.21.15.19:0/1927389298 >> v1:172.21.15.169:6789/0 conn(0x7fcb341605c0 legacy=0x7fcb340ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.353+0000 7fcb32800640 1 -- 172.21.15.19:0/1927389298 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb340b7b60 con 0x7fcb34168000 2024-04-25T11:43:19.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.354+0000 7fcb32800640 1 --2- 172.21.15.19:0/1927389298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb34168000 0x7fcb3416a420 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fcb340cfa90 tx=0x7fcb2c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9d2b0b9e790634 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.354+0000 7fcb30a00640 1 -- 172.21.15.19:0/1927389298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c036070 con 0x7fcb34168000 2024-04-25T11:43:19.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.354+0000 7fcb30a00640 1 -- 172.21.15.19:0/1927389298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb2c031600 con 0x7fcb34168000 2024-04-25T11:43:19.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.354+0000 7fcb30a00640 1 -- 172.21.15.19:0/1927389298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c0319d0 con 0x7fcb34168000 2024-04-25T11:43:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 -- 172.21.15.19:0/1927389298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb34168000 msgr2=0x7fcb3416a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 --2- 172.21.15.19:0/1927389298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb34168000 0x7fcb3416a420 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7fcb340cfa90 tx=0x7fcb2c02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 -- 172.21.15.19:0/1927389298 shutdown_connections 2024-04-25T11:43:19.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 --2- 172.21.15.19:0/1927389298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb34168000 0x7fcb3416a420 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 --2- 172.21.15.19:0/1927389298 >> v2:172.21.15.169:3300/0 conn(0x7fcb340b75f0 0x7fcb340ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 -- 172.21.15.19:0/1927389298 >> 172.21.15.19:0/1927389298 conn(0x7fcb3409b7d0 msgr2=0x7fcb340b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 -- 172.21.15.19:0/1927389298 shutdown_connections 2024-04-25T11:43:19.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.355+0000 7fcb33200640 1 -- 172.21.15.19:0/1927389298 wait complete. 2024-04-25T11:43:19.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.358+0000 7fcb33200640 1 Processor -- start 2024-04-25T11:43:19.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.358+0000 7fcb33200640 1 -- start start 2024-04-25T11:43:19.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.359+0000 7fcb33200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb340b75f0 0x7fcb3414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.359+0000 7fcb33200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb341605c0 0x7fcb340d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb33200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb34168000 0x7fcb341fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb33200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3406cb80 con 0x7fcb341605c0 2024-04-25T11:43:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb33200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcb34076200 con 0x7fcb34168000 2024-04-25T11:43:19.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb33200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3406cd00 con 0x7fcb340b75f0 2024-04-25T11:43:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb31400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb341605c0 0x7fcb340d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb31400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb341605c0 0x7fcb340d1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60620/0 (socket says 172.21.15.19:60620) 2024-04-25T11:43:19.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb31400640 1 -- 172.21.15.19:0/2049763889 learned_addr learned my addr 172.21.15.19:0/2049763889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb32800640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb34168000 0x7fcb341fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.360+0000 7fcb31e00640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb340b75f0 0x7fcb3414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb31400640 1 -- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb340b75f0 msgr2=0x7fcb3414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb31400640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb340b75f0 0x7fcb3414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb31400640 1 -- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb34168000 msgr2=0x7fcb341fa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb31400640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb34168000 0x7fcb341fa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb31400640 1 -- 172.21.15.19:0/2049763889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb341542a0 con 0x7fcb341605c0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb31400640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb341605c0 0x7fcb340d1eb0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b420 tx=0x7fcb2400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb2a000640 1 -- 172.21.15.19:0/2049763889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24011830 con 0x7fcb341605c0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb2a000640 1 -- 172.21.15.19:0/2049763889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24016070 con 0x7fcb341605c0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb2a000640 1 -- 172.21.15.19:0/2049763889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb24011c10 con 0x7fcb341605c0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb340cfeb0 con 0x7fcb341605c0 2024-04-25T11:43:19.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.361+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3414ebd0 con 0x7fcb341605c0 2024-04-25T11:43:19.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.363+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb340cd7e0 con 0x7fcb341605c0 2024-04-25T11:43:19.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.364+0000 7fcb2a000640 1 -- 172.21.15.19:0/2049763889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb2401b460 con 0x7fcb341605c0 2024-04-25T11:43:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.365+0000 7fcb2a000640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb100744b0 0x7fcb100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.365+0000 7fcb2a000640 1 -- 172.21.15.19:0/2049763889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fcb24094b80 con 0x7fcb341605c0 2024-04-25T11:43:19.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.366+0000 7fcb31e00640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb100744b0 0x7fcb100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.367+0000 7fcb31e00640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb100744b0 0x7fcb100769c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c006fd0 tx=0x7fcb1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.370+0000 7fcb2a000640 1 -- 172.21.15.19:0/2049763889 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb340cd7e0 con 0x7fcb341605c0 2024-04-25T11:43:19.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.577+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fcb34050740 con 0x7fcb100744b0 2024-04-25T11:43:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.578+0000 7fcb2a000640 1 -- 172.21.15.19:0/2049763889 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcb34050740 con 0x7fcb100744b0 2024-04-25T11:43:19.564 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T11:43:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb100744b0 msgr2=0x7fcb100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb100744b0 0x7fcb100769c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c006fd0 tx=0x7fcb1c008040 comp rx=0 tx=0).stop 2024-04-25T11:43:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb341605c0 msgr2=0x7fcb340d1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb341605c0 0x7fcb340d1eb0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b420 tx=0x7fcb2400b8e0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 shutdown_connections 2024-04-25T11:43:19.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcb100744b0 0x7fcb100769c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcb34168000 0x7fcb341fa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb341605c0 0x7fcb340d1eb0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 --2- 172.21.15.19:0/2049763889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcb340b75f0 0x7fcb3414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 >> 172.21.15.19:0/2049763889 conn(0x7fcb3409b7d0 msgr2=0x7fcb34158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:19.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 shutdown_connections 2024-04-25T11:43:19.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.581+0000 7fcb33200640 1 -- 172.21.15.19:0/2049763889 wait complete. 2024-04-25T11:43:19.595 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T11:43:19.595 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.893+0000 7f6ea7200640 1 Processor -- start 2024-04-25T11:43:19.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.894+0000 7f6ea7200640 1 -- start start 2024-04-25T11:43:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea7200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ea806d940 0x7f6ea81645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea806dd50 0x7f6ea8164b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea7200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ea8165050 0x7f6ea80d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea7200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea8165460 con 0x7f6ea806d940 2024-04-25T11:43:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea7200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea80d1330 con 0x7f6ea8165050 2024-04-25T11:43:19.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea809d1d0 con 0x7f6ea806dd50 2024-04-25T11:43:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ea806d940 0x7f6ea81645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ea806d940 0x7f6ea81645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60630/0 (socket says 172.21.15.19:60630) 2024-04-25T11:43:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea5e00640 1 -- 172.21.15.19:0/1953020662 learned_addr learned my addr 172.21.15.19:0/1953020662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.896+0000 7f6ea5400640 1 --2- 172.21.15.19:0/1953020662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea806dd50 0x7f6ea8164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.897+0000 7f6ea6800640 1 --2- 172.21.15.19:0/1953020662 >> v2:172.21.15.169:3300/0 conn(0x7f6ea8165050 0x7f6ea80d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.897+0000 7f6ea5e00640 1 -- 172.21.15.19:0/1953020662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea806dd50 msgr2=0x7f6ea8164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:19.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.897+0000 7f6ea5e00640 1 --2- 172.21.15.19:0/1953020662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea806dd50 0x7f6ea8164b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.897+0000 7f6ea5e00640 1 -- 172.21.15.19:0/1953020662 >> v2:172.21.15.169:3300/0 conn(0x7f6ea8165050 msgr2=0x7f6ea80d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.897+0000 7f6ea5e00640 1 --2- 172.21.15.19:0/1953020662 >> v2:172.21.15.169:3300/0 conn(0x7f6ea8165050 0x7f6ea80d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.897+0000 7f6ea5e00640 1 -- 172.21.15.19:0/1953020662 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea80d1050 con 0x7f6ea806d940 2024-04-25T11:43:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.898+0000 7f6ea5e00640 1 --2- 172.21.15.19:0/1953020662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806d940 0x7f6ea81645d0 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8050e10 tx=0x7f6e9002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d32d4adf3adde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.898+0000 7f6ea4a00640 1 -- 172.21.15.19:0/1953020662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90037070 con 0x7f6ea806d940 2024-04-25T11:43:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.899+0000 7f6ea4a00640 1 -- 172.21.15.19:0/1953020662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e90032440 con 0x7f6ea806d940 2024-04-25T11:43:19.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.899+0000 7f6ea4a00640 1 -- 172.21.15.19:0/1953020662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90032810 con 0x7f6ea806d940 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.899+0000 7f6ea7200640 1 -- 172.21.15.19:0/1953020662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806d940 msgr2=0x7f6ea81645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.899+0000 7f6ea7200640 1 --2- 172.21.15.19:0/1953020662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806d940 0x7f6ea81645d0 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8050e10 tx=0x7f6e9002a140 comp rx=0 tx=0).stop 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.899+0000 7f6ea7200640 1 -- 172.21.15.19:0/1953020662 shutdown_connections 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.900+0000 7f6ea7200640 1 --2- 172.21.15.19:0/1953020662 >> v2:172.21.15.169:3300/0 conn(0x7f6ea8165050 0x7f6ea80d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.900+0000 7f6ea7200640 1 --2- 172.21.15.19:0/1953020662 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea806dd50 0x7f6ea8164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.900+0000 7f6ea7200640 1 --2- 172.21.15.19:0/1953020662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806d940 0x7f6ea81645d0 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.900+0000 7f6ea7200640 1 -- 172.21.15.19:0/1953020662 >> 172.21.15.19:0/1953020662 conn(0x7f6ea80d0390 msgr2=0x7f6ea80d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.900+0000 7f6ea7200640 1 -- 172.21.15.19:0/1953020662 shutdown_connections 2024-04-25T11:43:19.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.900+0000 7f6ea7200640 1 -- 172.21.15.19:0/1953020662 wait complete. 2024-04-25T11:43:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.903+0000 7f6ea7200640 1 Processor -- start 2024-04-25T11:43:19.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.903+0000 7f6ea7200640 1 -- start start 2024-04-25T11:43:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806dd50 0x7f6ea814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea7200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea8165050 0x7f6ea815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea7200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8072630 0x7f6ea8072a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80d3790 con 0x7f6ea806dd50 2024-04-25T11:43:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea7200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80d3610 con 0x7f6ea8072630 2024-04-25T11:43:19.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea7200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80b92e0 con 0x7f6ea8165050 2024-04-25T11:43:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806dd50 0x7f6ea814ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806dd50 0x7f6ea814ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60632/0 (socket says 172.21.15.19:60632) 2024-04-25T11:43:19.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea5e00640 1 -- 172.21.15.19:0/2116543954 learned_addr learned my addr 172.21.15.19:0/2116543954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea5400640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea8165050 0x7f6ea815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.905+0000 7f6ea6800640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8072630 0x7f6ea8072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea5e00640 1 -- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea8165050 msgr2=0x7f6ea815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea5e00640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ea8165050 0x7f6ea815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea5e00640 1 -- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8072630 msgr2=0x7f6ea8072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea5e00640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8072630 0x7f6ea8072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea5e00640 1 -- 172.21.15.19:0/2116543954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea8072f80 con 0x7f6ea806dd50 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea6800640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ea8072630 0x7f6ea8072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:19.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea5e00640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806dd50 0x7f6ea814ee60 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f6e9002a6b0 tx=0x7f6e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90041020 con 0x7f6ea806dd50 2024-04-25T11:43:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e90030760 con 0x7f6ea806dd50 2024-04-25T11:43:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90037040 con 0x7f6ea806dd50 2024-04-25T11:43:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.906+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea815e300 con 0x7f6ea806dd50 2024-04-25T11:43:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.907+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea815eaf0 con 0x7f6ea806dd50 2024-04-25T11:43:19.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.907+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ea8076c60 con 0x7f6ea806dd50 2024-04-25T11:43:19.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.909+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6e90044020 con 0x7f6ea806dd50 2024-04-25T11:43:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.910+0000 7f6e9e000640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6e840744b0 0x7f6e840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.910+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f6e900b5d50 con 0x7f6ea806dd50 2024-04-25T11:43:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.911+0000 7f6e9e000640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f6e8407a460 0x7f6e8407c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.911+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e90001d30 con 0x7f6e8407a460 2024-04-25T11:43:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.911+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f6e900b6100 con 0x7f6ea806dd50 2024-04-25T11:43:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.911+0000 7f6ea5400640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6e840744b0 0x7f6e840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.911+0000 7f6ea6800640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f6e8407a460 0x7f6e8407c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.912+0000 7f6ea6800640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f6e8407a460 0x7f6e8407c8b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.912+0000 7f6ea5400640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6e840744b0 0x7f6e840769c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8159560 tx=0x7f6e98003a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.913+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== osd.2 v2:172.21.15.19:6813/2392070031 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e90001d30 con 0x7f6e8407a460 2024-04-25T11:43:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.928+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 --> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6ea80b51e0 con 0x7f6e8407a460 2024-04-25T11:43:19.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.928+0000 7f6e9e000640 1 -- 172.21.15.19:0/2116543954 <== osd.2 v2:172.21.15.19:6813/2392070031 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6ea80b51e0 con 0x7f6e8407a460 2024-04-25T11:43:19.914 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:19.915 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:19.915 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.929+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f6e8407a460 msgr2=0x7f6e8407c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.929+0000 7f6ea7200640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:6813/2392070031,v1:172.21.15.19:6815/2392070031] conn(0x7f6e8407a460 0x7f6e8407c8b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.929+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6e840744b0 msgr2=0x7f6e840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.929+0000 7f6ea7200640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f6e840744b0 0x7f6e840769c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8159560 tx=0x7f6e98003a50 comp rx=0 tx=0).stop 2024-04-25T11:43:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.929+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806dd50 msgr2=0x7f6ea814ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.929+0000 7f6ea7200640 1 --2- 172.21.15.19:0/2116543954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ea806dd50 0x7f6ea814ee60 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f6e9002a6b0 tx=0x7f6e900046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.929+0000 7f6ea6800640 1 -- 172.21.15.19:0/2116543954 reap_dead start 2024-04-25T11:43:19.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.930+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 shutdown_connections 2024-04-25T11:43:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.930+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 >> 172.21.15.19:0/2116543954 conn(0x7f6ea80d0390 msgr2=0x7f6ea80b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.930+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 shutdown_connections 2024-04-25T11:43:19.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:19.930+0000 7f6ea7200640 1 -- 172.21.15.19:0/2116543954 wait complete. 2024-04-25T11:43:19.944 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T11:43:20.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.244+0000 7f83d7e00640 1 Processor -- start 2024-04-25T11:43:20.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.244+0000 7f83d7e00640 1 -- start start 2024-04-25T11:43:20.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.247+0000 7f83d7e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83d80b5420 0x7f83d80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.247+0000 7f83d7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d806f7f0 0x7f83d8163db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.247+0000 7f83d7e00640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d8160840 con 0x7f83d8070620 2024-04-25T11:43:20.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.247+0000 7f83d7e00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f83d8160ab0 con 0x7f83d80b5420 2024-04-25T11:43:20.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.247+0000 7f83d7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83d8160d90 con 0x7f83d806f7f0 2024-04-25T11:43:20.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.248+0000 7f83d6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83d80b5420 0x7f83d80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.248+0000 7f83d6a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f83d80b5420 0x7f83d80700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.19:47968/0 (socket says 172.21.15.19:47968) 2024-04-25T11:43:20.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.248+0000 7f83d6a00640 1 -- 172.21.15.19:0/1924249081 learned_addr learned my addr 172.21.15.19:0/1924249081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:20.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.248+0000 7f83d7400640 1 --2- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d806f7f0 0x7f83d8163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.249+0000 7f83d6a00640 1 -- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d806f7f0 msgr2=0x7f83d8163db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:20.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.249+0000 7f83d6a00640 1 --2- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d806f7f0 0x7f83d8163db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.249+0000 7f83d6a00640 1 -- 172.21.15.19:0/1924249081 >> v1:172.21.15.169:6789/0 conn(0x7f83d8070620 legacy=0x7f83d806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.249+0000 7f83d6a00640 1 -- 172.21.15.19:0/1924249081 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d8161070 con 0x7f83d80b5420 2024-04-25T11:43:20.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.249+0000 7f83d7400640 1 --2- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d806f7f0 0x7f83d8163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:20.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.250+0000 7f83d6a00640 1 --2- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 0x7f83d80700e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af80efd035ff8356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:20.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.251+0000 7f83d6000640 1 -- 172.21.15.19:0/1924249081 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8037070 con 0x7f83d80b5420 2024-04-25T11:43:20.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.251+0000 7f83d6000640 1 -- 172.21.15.19:0/1924249081 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c8032440 con 0x7f83d80b5420 2024-04-25T11:43:20.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.251+0000 7f83d6000640 1 -- 172.21.15.19:0/1924249081 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8032810 con 0x7f83d80b5420 2024-04-25T11:43:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.251+0000 7f83d7e00640 1 -- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 msgr2=0x7f83d80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.251+0000 7f83d7e00640 1 --2- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 0x7f83d80700e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009870 tx=0x7f83c802a250 comp rx=0 tx=0).stop 2024-04-25T11:43:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.252+0000 7f83d7e00640 1 -- 172.21.15.19:0/1924249081 shutdown_connections 2024-04-25T11:43:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.252+0000 7f83d7e00640 1 --2- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d806f7f0 0x7f83d8163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.252+0000 7f83d7e00640 1 --2- 172.21.15.19:0/1924249081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 0x7f83d80700e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.252+0000 7f83d7e00640 1 -- 172.21.15.19:0/1924249081 >> 172.21.15.19:0/1924249081 conn(0x7f83d80d1720 msgr2=0x7f83d80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:20.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.252+0000 7f83d7e00640 1 -- 172.21.15.19:0/1924249081 shutdown_connections 2024-04-25T11:43:20.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.252+0000 7f83d7e00640 1 -- 172.21.15.19:0/1924249081 wait complete. 2024-04-25T11:43:20.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.254+0000 7f83d7e00640 1 Processor -- start 2024-04-25T11:43:20.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.254+0000 7f83d7e00640 1 -- start start 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d8070620 0x7f83d814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d7e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 0x7f83d815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d7e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d815b730 0x7f83d815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83d80b0f60 con 0x7f83d8070620 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d7e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f83d80b0de0 con 0x7f83d80b5420 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d7e00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f83d80b4790 con 0x7f83d815b730 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d8070620 0x7f83d814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d8070620 0x7f83d814c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60648/0 (socket says 172.21.15.19:60648) 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.256+0000 7f83d6a00640 1 -- 172.21.15.19:0/4105432470 learned_addr learned my addr 172.21.15.19:0/4105432470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:20.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83cfe00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 0x7f83d815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83d7400640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d815b730 0x7f83d815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83d6a00640 1 -- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d815b730 msgr2=0x7f83d815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83d6a00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d815b730 0x7f83d815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83d6a00640 1 -- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 msgr2=0x7f83d815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83d6a00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 0x7f83d815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83d6a00640 1 -- 172.21.15.19:0/4105432470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d815d810 con 0x7f83d8070620 2024-04-25T11:43:20.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83d6a00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d8070620 0x7f83d814c0b0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f83c8000c00 tx=0x7f83c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.257+0000 7f83cea00640 1 -- 172.21.15.19:0/4105432470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8037040 con 0x7f83d8070620 2024-04-25T11:43:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.258+0000 7f83cea00640 1 -- 172.21.15.19:0/4105432470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c8042070 con 0x7f83d8070620 2024-04-25T11:43:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.258+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d815c080 con 0x7f83d8070620 2024-04-25T11:43:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.258+0000 7f83cea00640 1 -- 172.21.15.19:0/4105432470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8030880 con 0x7f83d8070620 2024-04-25T11:43:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.258+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d815dd60 con 0x7f83d8070620 2024-04-25T11:43:20.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.259+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d8050740 con 0x7f83d8070620 2024-04-25T11:43:20.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.260+0000 7f83cea00640 1 -- 172.21.15.19:0/4105432470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83c8030a20 con 0x7f83d8070620 2024-04-25T11:43:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.261+0000 7f83cea00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83b40744e0 0x7f83b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.261+0000 7f83cea00640 1 -- 172.21.15.19:0/4105432470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f83c80b5950 con 0x7f83d8070620 2024-04-25T11:43:20.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.262+0000 7f83cfe00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83b40744e0 0x7f83b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.263+0000 7f83cfe00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83b40744e0 0x7f83b40769f0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f83c0007fd0 tx=0x7f83c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.266+0000 7f83cea00640 1 -- 172.21.15.19:0/4105432470 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83c8080260 con 0x7f83d8070620 2024-04-25T11:43:20.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.469+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f83d815ad40 con 0x7f83b40744e0 2024-04-25T11:43:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.470+0000 7f83cea00640 1 -- 172.21.15.19:0/4105432470 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f83d815ad40 con 0x7f83b40744e0 2024-04-25T11:43:20.456 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T11:43:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83b40744e0 msgr2=0x7f83b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83b40744e0 0x7f83b40769f0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f83c0007fd0 tx=0x7f83c0009040 comp rx=0 tx=0).stop 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d8070620 msgr2=0x7f83d814c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d8070620 0x7f83d814c0b0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f83c8000c00 tx=0x7f83c80046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 shutdown_connections 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f83b40744e0 0x7f83b40769f0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f83d80b5420 0x7f83d815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f83d815b730 0x7f83d815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 --2- 172.21.15.19:0/4105432470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83d8070620 0x7f83d814c0b0 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 >> 172.21.15.19:0/4105432470 conn(0x7f83d80d1720 msgr2=0x7f83d80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 shutdown_connections 2024-04-25T11:43:20.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.473+0000 7f83d7e00640 1 -- 172.21.15.19:0/4105432470 wait complete. 2024-04-25T11:43:20.486 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T11:43:20.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:20.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.797+0000 7f5377000640 1 Processor -- start 2024-04-25T11:43:20.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.797+0000 7f5377000640 1 -- start start 2024-04-25T11:43:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f5377000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53700cd9a0 0x7f537015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f5377000640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f537006dce0 0x7f537015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f5377000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53700cdea0 con 0x7f53700b0af0 2024-04-25T11:43:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f5377000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f53700ceab0 con 0x7f53700cd9a0 2024-04-25T11:43:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f5377000640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f53700ce180 con 0x7f537006dce0 2024-04-25T11:43:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f536fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53700cd9a0 0x7f537015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f536fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53700cd9a0 0x7f537015f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60656/0 (socket says 172.21.15.19:60656) 2024-04-25T11:43:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.800+0000 7f536fe00640 1 -- 172.21.15.19:0/2934153156 learned_addr learned my addr 172.21.15.19:0/2934153156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:20.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.801+0000 7f5375000640 1 --2- 172.21.15.19:0/2934153156 >> v2:172.21.15.169:3300/0 conn(0x7f537006dce0 0x7f537015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.801+0000 7f536fe00640 1 -- 172.21.15.19:0/2934153156 >> v2:172.21.15.169:3300/0 conn(0x7f537006dce0 msgr2=0x7f537015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.801+0000 7f536fe00640 1 --2- 172.21.15.19:0/2934153156 >> v2:172.21.15.169:3300/0 conn(0x7f537006dce0 0x7f537015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.801+0000 7f536fe00640 1 -- 172.21.15.19:0/2934153156 >> v1:172.21.15.169:6789/0 conn(0x7f53700b0af0 legacy=0x7f537006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.801+0000 7f536fe00640 1 -- 172.21.15.19:0/2934153156 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53700ce460 con 0x7f53700cd9a0 2024-04-25T11:43:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.801+0000 7f5375000640 1 --2- 172.21.15.19:0/2934153156 >> v2:172.21.15.169:3300/0 conn(0x7f537006dce0 0x7f537015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:20.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.802+0000 7f536fe00640 1 --2- 172.21.15.19:0/2934153156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53700cd9a0 0x7f537015f360 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f5370148020 tx=0x7f536002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53097f83dd00c3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.803+0000 7f536f400640 1 -- 172.21.15.19:0/2934153156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360037070 con 0x7f53700cd9a0 2024-04-25T11:43:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.803+0000 7f536f400640 1 -- 172.21.15.19:0/2934153156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5360032440 con 0x7f53700cd9a0 2024-04-25T11:43:20.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.803+0000 7f536f400640 1 -- 172.21.15.19:0/2934153156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360032810 con 0x7f53700cd9a0 2024-04-25T11:43:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 -- 172.21.15.19:0/2934153156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53700cd9a0 msgr2=0x7f537015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 --2- 172.21.15.19:0/2934153156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53700cd9a0 0x7f537015f360 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f5370148020 tx=0x7f536002a230 comp rx=0 tx=0).stop 2024-04-25T11:43:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 -- 172.21.15.19:0/2934153156 shutdown_connections 2024-04-25T11:43:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 --2- 172.21.15.19:0/2934153156 >> v2:172.21.15.169:3300/0 conn(0x7f537006dce0 0x7f537015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 --2- 172.21.15.19:0/2934153156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53700cd9a0 0x7f537015f360 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 -- 172.21.15.19:0/2934153156 >> 172.21.15.19:0/2934153156 conn(0x7f5370073090 msgr2=0x7f53700734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 -- 172.21.15.19:0/2934153156 shutdown_connections 2024-04-25T11:43:20.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.804+0000 7f5377000640 1 -- 172.21.15.19:0/2934153156 wait complete. 2024-04-25T11:43:20.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.807+0000 7f5377000640 1 Processor -- start 2024-04-25T11:43:20.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.808+0000 7f5377000640 1 -- start start 2024-04-25T11:43:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.809+0000 7f5377000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53700b0af0 0x7f53701e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.809+0000 7f5377000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53700cd9a0 0x7f53701561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.809+0000 7f5377000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53701faa60 0x7f53700b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.809+0000 7f5377000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53700cef40 con 0x7f53701faa60 2024-04-25T11:43:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.809+0000 7f5377000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53700b0f00 con 0x7f53700b0af0 2024-04-25T11:43:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.809+0000 7f5377000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f53700cf0c0 con 0x7f53700cd9a0 2024-04-25T11:43:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.809+0000 7f5375000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53701faa60 0x7f53700b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53701faa60 0x7f53700b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60668/0 (socket says 172.21.15.19:60668) 2024-04-25T11:43:20.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 -- 172.21.15.19:0/2871071679 learned_addr learned my addr 172.21.15.19:0/2871071679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f536fe00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53700b0af0 0x7f53701e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 -- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53700cd9a0 msgr2=0x7f53701561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5367e00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53700cd9a0 0x7f53701561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f53700cd9a0 0x7f53701561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 -- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53700b0af0 msgr2=0x7f53701e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53700b0af0 0x7f53701e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 -- 172.21.15.19:0/2871071679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53700b1380 con 0x7f53701faa60 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.810+0000 7f5375000640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53701faa60 0x7f53700b2420 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f5368007e70 tx=0x7f536800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.811+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536801a070 con 0x7f53701faa60 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.811+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368003bb0 con 0x7f53701faa60 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.811+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368004030 con 0x7f53701faa60 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.811+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53700b1660 con 0x7f53701faa60 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.811+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53700b1c20 con 0x7f53701faa60 2024-04-25T11:43:20.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.812+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5370050740 con 0x7f53701faa60 2024-04-25T11:43:20.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.813+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f536801d070 con 0x7f53701faa60 2024-04-25T11:43:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.814+0000 7f536cc00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f534c0744b0 0x7f534c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.815+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f5368095280 con 0x7f53701faa60 2024-04-25T11:43:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.815+0000 7f536cc00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f534c07a460 0x7f534c07c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.815+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f534c07cf80 con 0x7f534c07a460 2024-04-25T11:43:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.815+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f536809c050 con 0x7f53701faa60 2024-04-25T11:43:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.815+0000 7f5367e00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f534c07a460 0x7f534c07c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.815+0000 7f536fe00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f534c0744b0 0x7f534c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:20.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.816+0000 7f5367e00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f534c07a460 0x7f534c07c8b0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:20.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.816+0000 7f536fe00640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f534c0744b0 0x7f534c0769c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5360034000 tx=0x7f536002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:20.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.817+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== osd.3 v2:172.21.15.19:6824/2286957460 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f534c07cf80 con 0x7f534c07a460 2024-04-25T11:43:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.832+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 --> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f53701fafd0 con 0x7f534c07a460 2024-04-25T11:43:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.832+0000 7f536cc00640 1 -- 172.21.15.19:0/2871071679 <== osd.3 v2:172.21.15.19:6824/2286957460 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f53701fafd0 con 0x7f534c07a460 2024-04-25T11:43:20.818 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:20.819 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:20.819 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f534c07a460 msgr2=0x7f534c07c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:6824/2286957460,v1:172.21.15.19:6825/2286957460] conn(0x7f534c07a460 0x7f534c07c8b0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f534c0744b0 msgr2=0x7f534c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f534c0744b0 0x7f534c0769c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5360034000 tx=0x7f536002e000 comp rx=0 tx=0).stop 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53701faa60 msgr2=0x7f53700b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 --2- 172.21.15.19:0/2871071679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53701faa60 0x7f53700b2420 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f5368007e70 tx=0x7f536800d4a0 comp rx=0 tx=0).stop 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5375000640 1 -- 172.21.15.19:0/2871071679 reap_dead start 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 shutdown_connections 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 >> 172.21.15.19:0/2871071679 conn(0x7f5370073090 msgr2=0x7f53701584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 shutdown_connections 2024-04-25T11:43:20.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:20.833+0000 7f5377000640 1 -- 172.21.15.19:0/2871071679 wait complete. 2024-04-25T11:43:20.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T11:43:21.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.133+0000 7f1cb5000640 1 Processor -- start 2024-04-25T11:43:21.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.133+0000 7f1cb5000640 1 -- start start 2024-04-25T11:43:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.135+0000 7f1cb5000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1cb0144370 0x7f1cb00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.135+0000 7f1cb5000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb00d1230 con 0x7f1cb006c5c0 2024-04-25T11:43:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.135+0000 7f1cb5000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb00d06b0 con 0x7f1cb00b4ea0 2024-04-25T11:43:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.135+0000 7f1cb5000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1cb00d0990 con 0x7f1cb0144370 2024-04-25T11:43:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1caf400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1cb006c5c0 0x7f1cb00b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41630/0 (socket says 172.21.15.19:41630) 2024-04-25T11:43:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1caf400640 1 -- 172.21.15.19:0/865346416 learned_addr learned my addr 172.21.15.19:0/865346416 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:21.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1cafe00640 1 --2- 172.21.15.19:0/865346416 >> v2:172.21.15.19:3300/0 conn(0x7f1cb0144370 0x7f1cb00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1caea00640 1 -- 172.21.15.19:0/865346416 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081924576 0 0) 0x7f1cb00d1230 con 0x7f1cb006c5c0 2024-04-25T11:43:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1caea00640 1 -- 172.21.15.19:0/865346416 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c90003520 con 0x7f1cb006c5c0 2024-04-25T11:43:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1cafe00640 1 -- 172.21.15.19:0/865346416 >> v1:172.21.15.169:6789/0 conn(0x7f1cb00b4ea0 legacy=0x7f1cb015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1cafe00640 1 -- 172.21.15.19:0/865346416 >> v1:172.21.15.19:6789/0 conn(0x7f1cb006c5c0 legacy=0x7f1cb00b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.136+0000 7f1cafe00640 1 -- 172.21.15.19:0/865346416 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb00d0c70 con 0x7f1cb0144370 2024-04-25T11:43:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.138+0000 7f1cafe00640 1 --2- 172.21.15.19:0/865346416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0144370 0x7f1cb00cfa70 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009d90 tx=0x7f1ca802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b283baee82bed8be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.138+0000 7f1caea00640 1 -- 172.21.15.19:0/865346416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8007740 con 0x7f1cb0144370 2024-04-25T11:43:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.138+0000 7f1caea00640 1 -- 172.21.15.19:0/865346416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca80078e0 con 0x7f1cb0144370 2024-04-25T11:43:21.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.138+0000 7f1caea00640 1 -- 172.21.15.19:0/865346416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8007c90 con 0x7f1cb0144370 2024-04-25T11:43:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.139+0000 7f1cb5000640 1 -- 172.21.15.19:0/865346416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0144370 msgr2=0x7f1cb00cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.139+0000 7f1cb5000640 1 --2- 172.21.15.19:0/865346416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0144370 0x7f1cb00cfa70 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009d90 tx=0x7f1ca802a6a0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.139+0000 7f1cb5000640 1 -- 172.21.15.19:0/865346416 shutdown_connections 2024-04-25T11:43:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.139+0000 7f1cb5000640 1 --2- 172.21.15.19:0/865346416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0144370 0x7f1cb00cfa70 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.139+0000 7f1cb5000640 1 -- 172.21.15.19:0/865346416 >> 172.21.15.19:0/865346416 conn(0x7f1cb00d1720 msgr2=0x7f1cb00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.139+0000 7f1cb5000640 1 -- 172.21.15.19:0/865346416 shutdown_connections 2024-04-25T11:43:21.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.139+0000 7f1cb5000640 1 -- 172.21.15.19:0/865346416 wait complete. 2024-04-25T11:43:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.141+0000 7f1cb5000640 1 Processor -- start 2024-04-25T11:43:21.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.141+0000 7f1cb5000640 1 -- start start 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cb5000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb006c5c0 0x7f1cb01f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cb5000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb0144370 0x7f1cb013d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cb5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0162320 0x7f1cb0146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cb5000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb00d0450 con 0x7f1cb0162320 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cb5000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb00d02d0 con 0x7f1cb006c5c0 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cb5000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb0147760 con 0x7f1cb0144370 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0162320 0x7f1cb0146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0162320 0x7f1cb0146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60684/0 (socket says 172.21.15.19:60684) 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.143+0000 7f1cafe00640 1 -- 172.21.15.19:0/4162607840 learned_addr learned my addr 172.21.15.19:0/4162607840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1ca7e00640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb0144370 0x7f1cb013d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1caf400640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb006c5c0 0x7f1cb01f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1cafe00640 1 -- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb0144370 msgr2=0x7f1cb013d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1cafe00640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb0144370 0x7f1cb013d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1cafe00640 1 -- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb006c5c0 msgr2=0x7f1cb01f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1cafe00640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb006c5c0 0x7f1cb01f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1cafe00640 1 -- 172.21.15.19:0/4162607840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb01453a0 con 0x7f1cb0162320 2024-04-25T11:43:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.144+0000 7f1cafe00640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0162320 0x7f1cb0146700 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f1ca802ac90 tx=0x7f1ca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.145+0000 7f1ca7400640 1 -- 172.21.15.19:0/4162607840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca803f070 con 0x7f1cb0162320 2024-04-25T11:43:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.145+0000 7f1ca7400640 1 -- 172.21.15.19:0/4162607840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca8031bc0 con 0x7f1cb0162320 2024-04-25T11:43:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.145+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb0145660 con 0x7f1cb0162320 2024-04-25T11:43:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.145+0000 7f1ca7400640 1 -- 172.21.15.19:0/4162607840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8009450 con 0x7f1cb0162320 2024-04-25T11:43:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.145+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb0145c20 con 0x7f1cb0162320 2024-04-25T11:43:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.146+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb0050740 con 0x7f1cb0162320 2024-04-25T11:43:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.147+0000 7f1ca7400640 1 -- 172.21.15.19:0/4162607840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ca8043410 con 0x7f1cb0162320 2024-04-25T11:43:21.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.148+0000 7f1ca7400640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c8c0744e0 0x7f1c8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.148+0000 7f1ca7400640 1 -- 172.21.15.19:0/4162607840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f1ca80b6460 con 0x7f1cb0162320 2024-04-25T11:43:21.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.149+0000 7f1caf400640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c8c0744e0 0x7f1c8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.150+0000 7f1caf400640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c8c0744e0 0x7f1c8c0769f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005880 tx=0x7f1ca0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.153+0000 7f1ca7400640 1 -- 172.21.15.19:0/4162607840 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ca80807a0 con 0x7f1cb0162320 2024-04-25T11:43:21.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.365+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f1cb015ef60 con 0x7f1c8c0744e0 2024-04-25T11:43:21.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.366+0000 7f1ca7400640 1 -- 172.21.15.19:0/4162607840 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1cb015ef60 con 0x7f1c8c0744e0 2024-04-25T11:43:21.352 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T11:43:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c8c0744e0 msgr2=0x7f1c8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c8c0744e0 0x7f1c8c0769f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005880 tx=0x7f1ca0009040 comp rx=0 tx=0).stop 2024-04-25T11:43:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0162320 msgr2=0x7f1cb0146700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0162320 0x7f1cb0146700 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f1ca802ac90 tx=0x7f1ca8004a40 comp rx=0 tx=0).stop 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 shutdown_connections 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f1c8c0744e0 0x7f1c8c0769f0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1cb0162320 0x7f1cb0146700 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1cb0144370 0x7f1cb013d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 --2- 172.21.15.19:0/4162607840 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1cb006c5c0 0x7f1cb01f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 >> 172.21.15.19:0/4162607840 conn(0x7f1cb00d1720 msgr2=0x7f1cb0072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 shutdown_connections 2024-04-25T11:43:21.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.369+0000 7f1cb5000640 1 -- 172.21.15.19:0/4162607840 wait complete. 2024-04-25T11:43:21.382 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T11:43:21.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.686+0000 7faf09400640 1 Processor -- start 2024-04-25T11:43:21.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.686+0000 7faf09400640 1 -- start start 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf09400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faf04144340 0x7faf0406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf09400640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf0415ee60 0x7faf0415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf09400640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf0406df20 con 0x7faf0406db10 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf09400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faf0415f270 con 0x7faf04144340 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf09400640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7faf04144750 con 0x7faf0415ee60 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf03e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faf04144340 0x7faf0406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf03e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faf04144340 0x7faf0406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60688/0 (socket says 172.21.15.19:60688) 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.689+0000 7faf03e00640 1 -- 172.21.15.19:0/1268709428 learned_addr learned my addr 172.21.15.19:0/1268709428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.690+0000 7faf03e00640 1 -- 172.21.15.19:0/1268709428 >> v2:172.21.15.169:3300/0 conn(0x7faf0415ee60 msgr2=0x7faf0415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.690+0000 7faf03e00640 1 --2- 172.21.15.19:0/1268709428 >> v2:172.21.15.169:3300/0 conn(0x7faf0415ee60 0x7faf0415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.690+0000 7faf03e00640 1 -- 172.21.15.19:0/1268709428 >> v1:172.21.15.169:6789/0 conn(0x7faf0406db10 legacy=0x7faf0415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.690+0000 7faf03e00640 1 -- 172.21.15.19:0/1268709428 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf04070000 con 0x7faf04144340 2024-04-25T11:43:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.691+0000 7faf03e00640 1 --2- 172.21.15.19:0/1268709428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf04144340 0x7faf0406d5d0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7faf040d1790 tx=0x7faef402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0315110caf2bab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.691+0000 7faf02a00640 1 -- 172.21.15.19:0/1268709428 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4037070 con 0x7faf04144340 2024-04-25T11:43:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.691+0000 7faf02a00640 1 -- 172.21.15.19:0/1268709428 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef4032440 con 0x7faf04144340 2024-04-25T11:43:21.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.691+0000 7faf02a00640 1 -- 172.21.15.19:0/1268709428 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4032810 con 0x7faf04144340 2024-04-25T11:43:21.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 -- 172.21.15.19:0/1268709428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf04144340 msgr2=0x7faf0406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 --2- 172.21.15.19:0/1268709428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf04144340 0x7faf0406d5d0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7faf040d1790 tx=0x7faef402a230 comp rx=0 tx=0).stop 2024-04-25T11:43:21.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 -- 172.21.15.19:0/1268709428 shutdown_connections 2024-04-25T11:43:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 --2- 172.21.15.19:0/1268709428 >> v2:172.21.15.169:3300/0 conn(0x7faf0415ee60 0x7faf0415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 --2- 172.21.15.19:0/1268709428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf04144340 0x7faf0406d5d0 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 -- 172.21.15.19:0/1268709428 >> 172.21.15.19:0/1268709428 conn(0x7faf04072f40 msgr2=0x7faf04073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 -- 172.21.15.19:0/1268709428 shutdown_connections 2024-04-25T11:43:21.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.692+0000 7faf09400640 1 -- 172.21.15.19:0/1268709428 wait complete. 2024-04-25T11:43:21.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.695+0000 7faf09400640 1 Processor -- start 2024-04-25T11:43:21.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.696+0000 7faf09400640 1 -- start start 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.697+0000 7faf09400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0406db10 0x7faf041f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.697+0000 7faf09400640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf04144340 0x7faf041f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf09400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf0415ee60 0x7faf040b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf09400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf0406f800 con 0x7faf0406db10 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf09400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faf0406f520 con 0x7faf0415ee60 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf09400640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7faf0406fae0 con 0x7faf04144340 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf03e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0406db10 0x7faf041f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf03e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0406db10 0x7faf041f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60694/0 (socket says 172.21.15.19:60694) 2024-04-25T11:43:21.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf03e00640 1 -- 172.21.15.19:0/1726915655 learned_addr learned my addr 172.21.15.19:0/1726915655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf08a00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf0415ee60 0x7faf040b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.698+0000 7faf03400640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf04144340 0x7faf041f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf03e00640 1 -- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf04144340 msgr2=0x7faf041f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf03e00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf04144340 0x7faf041f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf03e00640 1 -- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf0415ee60 msgr2=0x7faf040b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf03e00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf0415ee60 0x7faf040b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf03e00640 1 -- 172.21.15.19:0/1726915655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf041f8a90 con 0x7faf0406db10 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf08a00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf0415ee60 0x7faf040b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:21.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf03e00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0406db10 0x7faf041f3f20 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7faef4000c00 tx=0x7faef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4041070 con 0x7faf0406db10 2024-04-25T11:43:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef403c870 con 0x7faf0406db10 2024-04-25T11:43:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4037040 con 0x7faf0406db10 2024-04-25T11:43:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.699+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0415a950 con 0x7faf0406db10 2024-04-25T11:43:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.700+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0415af10 con 0x7faf0406db10 2024-04-25T11:43:21.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.701+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf0415b5d0 con 0x7faf0406db10 2024-04-25T11:43:21.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.702+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faef4044070 con 0x7faf0406db10 2024-04-25T11:43:21.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.703+0000 7faefbe00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faee40744e0 0x7faee40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.703+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7faef40b60f0 con 0x7faf0406db10 2024-04-25T11:43:21.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.703+0000 7faefbe00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7faee407a490 0x7faee407c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.703+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faef404cec0 con 0x7faee407a490 2024-04-25T11:43:21.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.703+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7faef40b64a0 con 0x7faf0406db10 2024-04-25T11:43:21.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.704+0000 7faf03400640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faee40744e0 0x7faee40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.704+0000 7faf08a00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7faee407a490 0x7faee407c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.706+0000 7faf08a00640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7faee407a490 0x7faee407c8e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.706+0000 7faf03400640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faee40744e0 0x7faee40769f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7faef0006fd0 tx=0x7faef0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.708+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== osd.4 v2:172.21.15.169:6816/3656468664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faef404cec0 con 0x7faee407a490 2024-04-25T11:43:21.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.723+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 --> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7faf04076c60 con 0x7faee407a490 2024-04-25T11:43:21.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.724+0000 7faefbe00640 1 -- 172.21.15.19:0/1726915655 <== osd.4 v2:172.21.15.169:6816/3656468664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7faf04076c60 con 0x7faee407a490 2024-04-25T11:43:21.711 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:21.711 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:21.711 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7faee407a490 msgr2=0x7faee407c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6816/3656468664,v1:172.21.15.169:6817/3656468664] conn(0x7faee407a490 0x7faee407c8e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faee40744e0 msgr2=0x7faee40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7faee40744e0 0x7faee40769f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7faef0006fd0 tx=0x7faef0008040 comp rx=0 tx=0).stop 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0406db10 msgr2=0x7faf041f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 --2- 172.21.15.19:0/1726915655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf0406db10 0x7faf041f3f20 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7faef4000c00 tx=0x7faef40046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf08a00640 1 -- 172.21.15.19:0/1726915655 reap_dead start 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 shutdown_connections 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 >> 172.21.15.19:0/1726915655 conn(0x7faf04072f40 msgr2=0x7faf0415c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 shutdown_connections 2024-04-25T11:43:21.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:21.725+0000 7faf09400640 1 -- 172.21.15.19:0/1726915655 wait complete. 2024-04-25T11:43:21.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T11:43:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.038+0000 7f8aece00640 1 Processor -- start 2024-04-25T11:43:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.038+0000 7f8aece00640 1 -- start start 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.040+0000 7f8aece00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae806c5c0 0x7f8ae80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.040+0000 7f8aece00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ae8144370 0x7f8ae80b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.040+0000 7f8aece00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae80b50e0 con 0x7f8ae80b4cd0 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.040+0000 7f8aece00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae80d1230 con 0x7f8ae8144370 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.040+0000 7f8aece00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae80d06b0 con 0x7f8ae806c5c0 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8ae7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ae80b4cd0 0x7f8ae8161a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41632/0 (socket says 172.21.15.19:41632) 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8ae7e00640 1 -- 172.21.15.19:0/3911574033 learned_addr learned my addr 172.21.15.19:0/3911574033 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8adfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8ae8144370 0x7f8ae80b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8ae7400640 1 --2- 172.21.15.19:0/3911574033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae806c5c0 0x7f8ae80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8adfe00640 1 -- 172.21.15.19:0/3911574033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae806c5c0 msgr2=0x7f8ae80cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8adfe00640 1 --2- 172.21.15.19:0/3911574033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae806c5c0 0x7f8ae80cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8adfe00640 1 -- 172.21.15.19:0/3911574033 >> v1:172.21.15.19:6789/0 conn(0x7f8ae80b4cd0 legacy=0x7f8ae8161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.041+0000 7f8adfe00640 1 -- 172.21.15.19:0/3911574033 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae80d0990 con 0x7f8ae8144370 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.042+0000 7f8adfe00640 1 --2- 172.21.15.19:0/3911574033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8144370 0x7f8ae80b4790 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00097a0 tx=0x7f8ad002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f8ce64038c3cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.043+0000 7f8ae6a00640 1 -- 172.21.15.19:0/3911574033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0037070 con 0x7f8ae8144370 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.043+0000 7f8ae6a00640 1 -- 172.21.15.19:0/3911574033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad0032440 con 0x7f8ae8144370 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.043+0000 7f8ae6a00640 1 -- 172.21.15.19:0/3911574033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0032810 con 0x7f8ae8144370 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 -- 172.21.15.19:0/3911574033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8144370 msgr2=0x7f8ae80b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 --2- 172.21.15.19:0/3911574033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8144370 0x7f8ae80b4790 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00097a0 tx=0x7f8ad002a180 comp rx=0 tx=0).stop 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 -- 172.21.15.19:0/3911574033 shutdown_connections 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 --2- 172.21.15.19:0/3911574033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae8144370 0x7f8ae80b4790 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 --2- 172.21.15.19:0/3911574033 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae806c5c0 0x7f8ae80cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 -- 172.21.15.19:0/3911574033 >> 172.21.15.19:0/3911574033 conn(0x7f8ae80d1720 msgr2=0x7f8ae80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 -- 172.21.15.19:0/3911574033 shutdown_connections 2024-04-25T11:43:22.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.044+0000 7f8aece00640 1 -- 172.21.15.19:0/3911574033 wait complete. 2024-04-25T11:43:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.046+0000 7f8aece00640 1 Processor -- start 2024-04-25T11:43:22.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.046+0000 7f8aece00640 1 -- start start 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8aece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae806c5c0 0x7f8ae813e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8aece00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae80b4cd0 0x7f8ae80cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8aece00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae8144370 0x7f8ae815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8aece00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae80d02d0 con 0x7f8ae806c5c0 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8aece00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae80d0f50 con 0x7f8ae8144370 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8aece00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae80d0450 con 0x7f8ae80b4cd0 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8ae7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae806c5c0 0x7f8ae813e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8ae7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae806c5c0 0x7f8ae813e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60708/0 (socket says 172.21.15.19:60708) 2024-04-25T11:43:22.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.048+0000 7f8ae7400640 1 -- 172.21.15.19:0/2588691204 learned_addr learned my addr 172.21.15.19:0/2588691204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8adfe00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae80b4cd0 0x7f8ae80cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8ae7e00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae8144370 0x7f8ae815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8ae7400640 1 -- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae80b4cd0 msgr2=0x7f8ae80cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8ae7400640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae80b4cd0 0x7f8ae80cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8ae7400640 1 -- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae8144370 msgr2=0x7f8ae815fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8ae7400640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae8144370 0x7f8ae815fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8ae7400640 1 -- 172.21.15.19:0/2588691204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae80cb5c0 con 0x7f8ae806c5c0 2024-04-25T11:43:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8ae7400640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae806c5c0 0x7f8ae813e250 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800d5f0 tx=0x7f8ad800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.049+0000 7f8adf400640 1 -- 172.21.15.19:0/2588691204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80125f0 con 0x7f8ae806c5c0 2024-04-25T11:43:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.050+0000 7f8adf400640 1 -- 172.21.15.19:0/2588691204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad8019070 con 0x7f8ae806c5c0 2024-04-25T11:43:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.050+0000 7f8adf400640 1 -- 172.21.15.19:0/2588691204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80129d0 con 0x7f8ae806c5c0 2024-04-25T11:43:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.050+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae80cb8a0 con 0x7f8ae806c5c0 2024-04-25T11:43:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.050+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae80cbe60 con 0x7f8ae806c5c0 2024-04-25T11:43:22.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.051+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae8050740 con 0x7f8ae806c5c0 2024-04-25T11:43:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.052+0000 7f8adf400640 1 -- 172.21.15.19:0/2588691204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8ad8012b70 con 0x7f8ae806c5c0 2024-04-25T11:43:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.053+0000 7f8adf400640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac40744b0 0x7f8ac40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.053+0000 7f8adf400640 1 -- 172.21.15.19:0/2588691204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f8ad80954c0 con 0x7f8ae806c5c0 2024-04-25T11:43:22.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.054+0000 7f8adfe00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac40744b0 0x7f8ac40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.055+0000 7f8adfe00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac40744b0 0x7f8ac40769c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0034000 tx=0x7f8ad002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.058+0000 7f8adf400640 1 -- 172.21.15.19:0/2588691204 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ad805fe50 con 0x7f8ae806c5c0 2024-04-25T11:43:22.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.265+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8ae80cfd70 con 0x7f8ac40744b0 2024-04-25T11:43:22.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.266+0000 7f8adf400640 1 -- 172.21.15.19:0/2588691204 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8ae80cfd70 con 0x7f8ac40744b0 2024-04-25T11:43:22.252 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T11:43:22.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.268+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac40744b0 msgr2=0x7f8ac40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac40744b0 0x7f8ac40769c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0034000 tx=0x7f8ad002e000 comp rx=0 tx=0).stop 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae806c5c0 msgr2=0x7f8ae813e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae806c5c0 0x7f8ae813e250 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800d5f0 tx=0x7f8ad800dab0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 shutdown_connections 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f8ac40744b0 0x7f8ac40769c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ae8144370 0x7f8ae815fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8ae80b4cd0 0x7f8ae80cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 --2- 172.21.15.19:0/2588691204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ae806c5c0 0x7f8ae813e250 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 >> 172.21.15.19:0/2588691204 conn(0x7f8ae80d1720 msgr2=0x7f8ae815c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 shutdown_connections 2024-04-25T11:43:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.269+0000 7f8aece00640 1 -- 172.21.15.19:0/2588691204 wait complete. 2024-04-25T11:43:22.280 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T11:43:22.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:22.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.584+0000 7fb7f9800640 1 Processor -- start 2024-04-25T11:43:22.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.585+0000 7fb7f9800640 1 -- start start 2024-04-25T11:43:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.587+0000 7fb7f9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406d940 0x7fb7f41645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.587+0000 7fb7f9800640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7f40cdf60 0x7fb7f40ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.587+0000 7fb7f9800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f40d1050 con 0x7fb7f406dd50 2024-04-25T11:43:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.587+0000 7fb7f9800640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f40d1330 con 0x7fb7f40cdf60 2024-04-25T11:43:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.587+0000 7fb7f9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f409d1d0 con 0x7fb7f406d940 2024-04-25T11:43:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.588+0000 7fb7f8e00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb7f40cdf60 0x7fb7f40ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.588+0000 7fb7f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406d940 0x7fb7f41645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.588+0000 7fb7f3e00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406d940 0x7fb7f41645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.19:50374/0 (socket says 172.21.15.19:50374) 2024-04-25T11:43:22.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.588+0000 7fb7f3e00640 1 -- 172.21.15.19:0/1260160955 learned_addr learned my addr 172.21.15.19:0/1260160955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:22.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.589+0000 7fb7f3e00640 1 -- 172.21.15.19:0/1260160955 >> v2:172.21.15.169:3300/0 conn(0x7fb7f40cdf60 msgr2=0x7fb7f40ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:22.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.589+0000 7fb7f3e00640 1 --2- 172.21.15.19:0/1260160955 >> v2:172.21.15.169:3300/0 conn(0x7fb7f40cdf60 0x7fb7f40ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.589+0000 7fb7f3e00640 1 -- 172.21.15.19:0/1260160955 >> v1:172.21.15.169:6789/0 conn(0x7fb7f406dd50 legacy=0x7fb7f4164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.589+0000 7fb7f3e00640 1 -- 172.21.15.19:0/1260160955 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f409ce60 con 0x7fb7f406d940 2024-04-25T11:43:22.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.590+0000 7fb7f3e00640 1 --2- 172.21.15.19:0/1260160955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406d940 0x7fb7f41645d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cbedd2d7a5b9d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.591+0000 7fb7f2a00640 1 -- 172.21.15.19:0/1260160955 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4037070 con 0x7fb7f406d940 2024-04-25T11:43:22.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.591+0000 7fb7f2a00640 1 -- 172.21.15.19:0/1260160955 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e4032440 con 0x7fb7f406d940 2024-04-25T11:43:22.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.591+0000 7fb7f2a00640 1 -- 172.21.15.19:0/1260160955 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4032810 con 0x7fb7f406d940 2024-04-25T11:43:22.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.592+0000 7fb7f9800640 1 -- 172.21.15.19:0/1260160955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406d940 msgr2=0x7fb7f41645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.592+0000 7fb7f9800640 1 --2- 172.21.15.19:0/1260160955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406d940 0x7fb7f41645d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.593+0000 7fb7f9800640 1 -- 172.21.15.19:0/1260160955 shutdown_connections 2024-04-25T11:43:22.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.593+0000 7fb7f9800640 1 --2- 172.21.15.19:0/1260160955 >> v2:172.21.15.169:3300/0 conn(0x7fb7f40cdf60 0x7fb7f40ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.593+0000 7fb7f9800640 1 --2- 172.21.15.19:0/1260160955 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406d940 0x7fb7f41645d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.593+0000 7fb7f9800640 1 -- 172.21.15.19:0/1260160955 >> 172.21.15.19:0/1260160955 conn(0x7fb7f40d0390 msgr2=0x7fb7f40d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:22.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.593+0000 7fb7f9800640 1 -- 172.21.15.19:0/1260160955 shutdown_connections 2024-04-25T11:43:22.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.593+0000 7fb7f9800640 1 -- 172.21.15.19:0/1260160955 wait complete. 2024-04-25T11:43:22.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.596+0000 7fb7f9800640 1 Processor -- start 2024-04-25T11:43:22.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.596+0000 7fb7f9800640 1 -- start start 2024-04-25T11:43:22.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.598+0000 7fb7f9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f406d940 0x7fb7f41e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f9800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406dd50 0x7fb7f40ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f9800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7f40cdf60 0x7fb7f40b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f9800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f40d3790 con 0x7fb7f406d940 2024-04-25T11:43:22.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f9800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f40d3610 con 0x7fb7f40cdf60 2024-04-25T11:43:22.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f9800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f40b92e0 con 0x7fb7f406dd50 2024-04-25T11:43:22.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f406d940 0x7fb7f41e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f406d940 0x7fb7f41e51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60720/0 (socket says 172.21.15.19:60720) 2024-04-25T11:43:22.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f3e00640 1 -- 172.21.15.19:0/3955452831 learned_addr learned my addr 172.21.15.19:0/3955452831 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:22.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f3400640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406dd50 0x7fb7f40ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f3e00640 1 -- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406dd50 msgr2=0x7fb7f40ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f3e00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406dd50 0x7fb7f40ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f3e00640 1 -- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7f40cdf60 msgr2=0x7fb7f40b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.599+0000 7fb7f8e00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7f40cdf60 0x7fb7f40b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.600+0000 7fb7f3e00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7f40cdf60 0x7fb7f40b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.600+0000 7fb7f3e00640 1 -- 172.21.15.19:0/3955452831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f41f9f50 con 0x7fb7f406d940 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.600+0000 7fb7f3400640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb7f406dd50 0x7fb7f40ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.600+0000 7fb7f8e00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb7f40cdf60 0x7fb7f40b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.600+0000 7fb7f3e00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f406d940 0x7fb7f41e51d0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009990 tx=0x7fb7e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.600+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4037040 con 0x7fb7f406d940 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.600+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e4042070 con 0x7fb7f406d940 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.601+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4030940 con 0x7fb7f406d940 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.601+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f41fa230 con 0x7fb7f406d940 2024-04-25T11:43:22.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.601+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f41fa7f0 con 0x7fb7f406d940 2024-04-25T11:43:22.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.602+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7f4076c60 con 0x7fb7f406d940 2024-04-25T11:43:22.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.603+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7e402ad10 con 0x7fb7f406d940 2024-04-25T11:43:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.605+0000 7fb7ebe00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7d00744e0 0x7fb7d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.605+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fb7e40b5f30 con 0x7fb7f406d940 2024-04-25T11:43:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.605+0000 7fb7ebe00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb7d007a490 0x7fb7d007c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.605+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7e4032440 con 0x7fb7d007a490 2024-04-25T11:43:22.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.605+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fb7e40b62e0 con 0x7fb7f406d940 2024-04-25T11:43:22.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.605+0000 7fb7f3400640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7d00744e0 0x7fb7d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.606+0000 7fb7f8e00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb7d007a490 0x7fb7d007c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.607+0000 7fb7f3400640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7d00744e0 0x7fb7d00769f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb7f41f9cd0 tx=0x7fb7dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.607+0000 7fb7f8e00640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb7d007a490 0x7fb7d007c8e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.610+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== osd.5 v2:172.21.15.169:6808/2942667610 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7e4032440 con 0x7fb7d007a490 2024-04-25T11:43:22.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.626+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 --> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb7f40b51e0 con 0x7fb7d007a490 2024-04-25T11:43:22.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.627+0000 7fb7ebe00640 1 -- 172.21.15.19:0/3955452831 <== osd.5 v2:172.21.15.169:6808/2942667610 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb7f40b51e0 con 0x7fb7d007a490 2024-04-25T11:43:22.614 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:22.614 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:22.614 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb7d007a490 msgr2=0x7fb7d007c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6808/2942667610,v1:172.21.15.169:6809/2942667610] conn(0x7fb7d007a490 0x7fb7d007c8e0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7d00744e0 msgr2=0x7fb7d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fb7d00744e0 0x7fb7d00769f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb7f41f9cd0 tx=0x7fb7dc009040 comp rx=0 tx=0).stop 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f406d940 msgr2=0x7fb7f41e51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 --2- 172.21.15.19:0/3955452831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f406d940 0x7fb7f41e51d0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009990 tx=0x7fb7e40046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f8e00640 1 -- 172.21.15.19:0/3955452831 reap_dead start 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 shutdown_connections 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 >> 172.21.15.19:0/3955452831 conn(0x7fb7f40d0390 msgr2=0x7fb7f40cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.628+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 shutdown_connections 2024-04-25T11:43:22.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.629+0000 7fb7f9800640 1 -- 172.21.15.19:0/3955452831 wait complete. 2024-04-25T11:43:22.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T11:43:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.943+0000 7f381fe00640 1 Processor -- start 2024-04-25T11:43:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.943+0000 7f381fe00640 1 -- start start 2024-04-25T11:43:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f382006c5c0 0x7f38200cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3820144370 0x7f38200b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381fe00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f38200b4cd0 0x7f3820161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f38200b50e0 con 0x7f382006c5c0 2024-04-25T11:43:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381fe00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7f38200d1230 con 0x7f38200b4cd0 2024-04-25T11:43:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38200d06b0 con 0x7f3820144370 2024-04-25T11:43:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f382006c5c0 0x7f38200cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f382006c5c0 0x7f38200cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60734/0 (socket says 172.21.15.19:60734) 2024-04-25T11:43:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.945+0000 7f381ea00640 1 -- 172.21.15.19:0/3459192721 learned_addr learned my addr 172.21.15.19:0/3459192721 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.946+0000 7f381e000640 1 --2- 172.21.15.19:0/3459192721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3820144370 0x7f38200b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.946+0000 7f381f400640 1 --2- 172.21.15.19:0/3459192721 >> v2:172.21.15.169:3300/0 conn(0x7f38200b4cd0 0x7f3820161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.946+0000 7f381ea00640 1 -- 172.21.15.19:0/3459192721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3820144370 msgr2=0x7f38200b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.946+0000 7f381ea00640 1 --2- 172.21.15.19:0/3459192721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3820144370 0x7f38200b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.946+0000 7f381ea00640 1 -- 172.21.15.19:0/3459192721 >> v2:172.21.15.169:3300/0 conn(0x7f38200b4cd0 msgr2=0x7f3820161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.946+0000 7f381ea00640 1 --2- 172.21.15.19:0/3459192721 >> v2:172.21.15.169:3300/0 conn(0x7f38200b4cd0 0x7f3820161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.946+0000 7f381ea00640 1 -- 172.21.15.19:0/3459192721 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38200d0990 con 0x7f382006c5c0 2024-04-25T11:43:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.947+0000 7f381ea00640 1 --2- 172.21.15.19:0/3459192721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382006c5c0 0x7f38200cfa70 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f38200b6cd0 tx=0x7f381002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57506986633fb48a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.948+0000 7f381d600640 1 -- 172.21.15.19:0/3459192721 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810037070 con 0x7f382006c5c0 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.948+0000 7f381d600640 1 -- 172.21.15.19:0/3459192721 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3810032440 con 0x7f382006c5c0 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.948+0000 7f381d600640 1 -- 172.21.15.19:0/3459192721 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810032810 con 0x7f382006c5c0 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.948+0000 7f381fe00640 1 -- 172.21.15.19:0/3459192721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382006c5c0 msgr2=0x7f38200cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.948+0000 7f381fe00640 1 --2- 172.21.15.19:0/3459192721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382006c5c0 0x7f38200cfa70 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f38200b6cd0 tx=0x7f381002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.949+0000 7f381fe00640 1 -- 172.21.15.19:0/3459192721 shutdown_connections 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.949+0000 7f381fe00640 1 --2- 172.21.15.19:0/3459192721 >> v2:172.21.15.169:3300/0 conn(0x7f38200b4cd0 0x7f3820161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.949+0000 7f381fe00640 1 --2- 172.21.15.19:0/3459192721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3820144370 0x7f38200b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.949+0000 7f381fe00640 1 --2- 172.21.15.19:0/3459192721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382006c5c0 0x7f38200cfa70 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.949+0000 7f381fe00640 1 -- 172.21.15.19:0/3459192721 >> 172.21.15.19:0/3459192721 conn(0x7f38200d1720 msgr2=0x7f38200d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.949+0000 7f381fe00640 1 -- 172.21.15.19:0/3459192721 shutdown_connections 2024-04-25T11:43:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.949+0000 7f381fe00640 1 -- 172.21.15.19:0/3459192721 wait complete. 2024-04-25T11:43:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.951+0000 7f381fe00640 1 Processor -- start 2024-04-25T11:43:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.951+0000 7f381fe00640 1 -- start start 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381fe00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f382006c5c0 0x7f3820070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381fe00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38200b4cd0 0x7f38201ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3820144370 0x7f3820144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38200d02d0 con 0x7f3820144370 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381fe00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38200d0f50 con 0x7f382006c5c0 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381fe00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f38200d0450 con 0x7f38200b4cd0 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3820144370 0x7f3820144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3820144370 0x7f3820144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60742/0 (socket says 172.21.15.19:60742) 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381f400640 1 -- 172.21.15.19:0/818871484 learned_addr learned my addr 172.21.15.19:0/818871484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381e000640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38200b4cd0 0x7f38201ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.953+0000 7f381ea00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f382006c5c0 0x7f3820070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381f400640 1 -- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38200b4cd0 msgr2=0x7f38201ea1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381f400640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38200b4cd0 0x7f38201ea1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381f400640 1 -- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f382006c5c0 msgr2=0x7f3820070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381f400640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f382006c5c0 0x7f3820070380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381f400640 1 -- 172.21.15.19:0/818871484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3820145c60 con 0x7f3820144370 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381e000640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38200b4cd0 0x7f38201ea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381ea00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f382006c5c0 0x7f3820070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381f400640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3820144370 0x7f3820144780 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f381800cc30 tx=0x7f381800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f3816a00640 1 -- 172.21.15.19:0/818871484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381800ce40 con 0x7f3820144370 2024-04-25T11:43:22.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f3816a00640 1 -- 172.21.15.19:0/818871484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3818003990 con 0x7f3820144370 2024-04-25T11:43:22.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f3816a00640 1 -- 172.21.15.19:0/818871484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818003d40 con 0x7f3820144370 2024-04-25T11:43:22.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38200ccf10 con 0x7f3820144370 2024-04-25T11:43:22.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.954+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382013f590 con 0x7f3820144370 2024-04-25T11:43:22.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.956+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382015d040 con 0x7f3820144370 2024-04-25T11:43:22.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.957+0000 7f3816a00640 1 -- 172.21.15.19:0/818871484 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3818003ee0 con 0x7f3820144370 2024-04-25T11:43:22.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.958+0000 7f3816a00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38000744b0 0x7f38000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:22.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.958+0000 7f3816a00640 1 -- 172.21.15.19:0/818871484 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f38180953f0 con 0x7f3820144370 2024-04-25T11:43:22.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.958+0000 7f381ea00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38000744b0 0x7f38000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:22.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.960+0000 7f381ea00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38000744b0 0x7f38000769c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3810034000 tx=0x7f381002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:22.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:22.963+0000 7f3816a00640 1 -- 172.21.15.19:0/818871484 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3818014030 con 0x7f3820144370 2024-04-25T11:43:23.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.169+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f38200d1e10 con 0x7f38000744b0 2024-04-25T11:43:23.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.171+0000 7f3816a00640 1 -- 172.21.15.19:0/818871484 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f38200d1e10 con 0x7f38000744b0 2024-04-25T11:43:23.157 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T11:43:23.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38000744b0 msgr2=0x7f38000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38000744b0 0x7f38000769c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3810034000 tx=0x7f381002e000 comp rx=0 tx=0).stop 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3820144370 msgr2=0x7f3820144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3820144370 0x7f3820144780 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f381800cc30 tx=0x7f381800a5c0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 shutdown_connections 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f38000744b0 0x7f38000769c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3820144370 0x7f3820144780 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f38200b4cd0 0x7f38201ea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 --2- 172.21.15.19:0/818871484 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f382006c5c0 0x7f3820070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.173+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 >> 172.21.15.19:0/818871484 conn(0x7f38200d1720 msgr2=0x7f382015c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.174+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 shutdown_connections 2024-04-25T11:43:23.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.174+0000 7f381fe00640 1 -- 172.21.15.19:0/818871484 wait complete. 2024-04-25T11:43:23.189 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T11:43:23.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:23.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.482+0000 7fe36a000640 1 Processor -- start 2024-04-25T11:43:23.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.482+0000 7fe36a000640 1 -- start start 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.484+0000 7fe36a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3640d5240 0x7fe3640d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.484+0000 7fe36a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640bae50 0x7fe36406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe36a000640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3640d1420 con 0x7fe36406dd60 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe36a000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe3640b9130 con 0x7fe3640d5240 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe36a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3640b99b0 con 0x7fe3640bae50 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe368c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3640d5240 0x7fe3640d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe368c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3640d5240 0x7fe3640d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60748/0 (socket says 172.21.15.19:60748) 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe368c00640 1 -- 172.21.15.19:0/467843186 learned_addr learned my addr 172.21.15.19:0/467843186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe369600640 1 --2- 172.21.15.19:0/467843186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640bae50 0x7fe36406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.485+0000 7fe368c00640 1 -- 172.21.15.19:0/467843186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640bae50 msgr2=0x7fe36406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:23.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.486+0000 7fe368c00640 1 --2- 172.21.15.19:0/467843186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640bae50 0x7fe36406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.486+0000 7fe368c00640 1 -- 172.21.15.19:0/467843186 >> v1:172.21.15.169:6789/0 conn(0x7fe36406dd60 legacy=0x7fe36406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:23.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.486+0000 7fe368c00640 1 -- 172.21.15.19:0/467843186 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3640b9360 con 0x7fe3640d5240 2024-04-25T11:43:23.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.486+0000 7fe369600640 1 --2- 172.21.15.19:0/467843186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640bae50 0x7fe36406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:23.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.487+0000 7fe368c00640 1 --2- 172.21.15.19:0/467843186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640d5240 0x7fe3640d0ab0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009870 tx=0x7fe34c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7015e30799bca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:23.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.487+0000 7fe363400640 1 -- 172.21.15.19:0/467843186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c037070 con 0x7fe3640d5240 2024-04-25T11:43:23.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.487+0000 7fe363400640 1 -- 172.21.15.19:0/467843186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe34c032440 con 0x7fe3640d5240 2024-04-25T11:43:23.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.487+0000 7fe363400640 1 -- 172.21.15.19:0/467843186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c032810 con 0x7fe3640d5240 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.488+0000 7fe36a000640 1 -- 172.21.15.19:0/467843186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640d5240 msgr2=0x7fe3640d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.488+0000 7fe36a000640 1 --2- 172.21.15.19:0/467843186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640d5240 0x7fe3640d0ab0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009870 tx=0x7fe34c02a200 comp rx=0 tx=0).stop 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.488+0000 7fe36a000640 1 -- 172.21.15.19:0/467843186 shutdown_connections 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.488+0000 7fe36a000640 1 --2- 172.21.15.19:0/467843186 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640bae50 0x7fe36406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.488+0000 7fe36a000640 1 --2- 172.21.15.19:0/467843186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640d5240 0x7fe3640d0ab0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.488+0000 7fe36a000640 1 -- 172.21.15.19:0/467843186 >> 172.21.15.19:0/467843186 conn(0x7fe3640d5d50 msgr2=0x7fe3640d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.489+0000 7fe36a000640 1 -- 172.21.15.19:0/467843186 shutdown_connections 2024-04-25T11:43:23.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.489+0000 7fe36a000640 1 -- 172.21.15.19:0/467843186 wait complete. 2024-04-25T11:43:23.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.491+0000 7fe36a000640 1 Processor -- start 2024-04-25T11:43:23.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.491+0000 7fe36a000640 1 -- start start 2024-04-25T11:43:23.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe36a000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36406dd60 0x7fe3640b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe36a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640bae50 0x7fe364158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe36a000640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640d5240 0x7fe3640b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe36a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640d0ff0 con 0x7fe3640bae50 2024-04-25T11:43:23.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe36a000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640d1700 con 0x7fe36406dd60 2024-04-25T11:43:23.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe36a000640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe3640d1170 con 0x7fe3640d5240 2024-04-25T11:43:23.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe363e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640bae50 0x7fe364158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe363e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640bae50 0x7fe364158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60764/0 (socket says 172.21.15.19:60764) 2024-04-25T11:43:23.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.493+0000 7fe363e00640 1 -- 172.21.15.19:0/1585331782 learned_addr learned my addr 172.21.15.19:0/1585331782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:23.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe369600640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640d5240 0x7fe3640b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe368c00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36406dd60 0x7fe3640b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe363e00640 1 -- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640d5240 msgr2=0x7fe3640b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe363e00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640d5240 0x7fe3640b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe363e00640 1 -- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36406dd60 msgr2=0x7fe3640b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe363e00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36406dd60 0x7fe3640b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe363e00640 1 -- 172.21.15.19:0/1585331782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36415e560 con 0x7fe3640bae50 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe369600640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3640d5240 0x7fe3640b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe368c00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe36406dd60 0x7fe3640b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe363e00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640bae50 0x7fe364158f40 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fe35400ba80 tx=0x7fe35400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35400cc00 con 0x7fe3640bae50 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35400cda0 con 0x7fe3640bae50 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354003bb0 con 0x7fe3640bae50 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.494+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3640b1a40 con 0x7fe3640bae50 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.495+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3640b2000 con 0x7fe3640bae50 2024-04-25T11:43:23.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.496+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe364076c60 con 0x7fe3640bae50 2024-04-25T11:43:23.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.497+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe354014440 con 0x7fe3640bae50 2024-04-25T11:43:23.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.498+0000 7fe360c00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3400744e0 0x7fe3400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.498+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fe354095ae0 con 0x7fe3640bae50 2024-04-25T11:43:23.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.499+0000 7fe360c00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7fe34007a490 0x7fe34007c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.499+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe354019e80 con 0x7fe34007a490 2024-04-25T11:43:23.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.499+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7fe354095e90 con 0x7fe3640bae50 2024-04-25T11:43:23.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.499+0000 7fe368c00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3400744e0 0x7fe3400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.499+0000 7fe369600640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7fe34007a490 0x7fe34007c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.500+0000 7fe369600640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7fe34007a490 0x7fe34007c8e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:23.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.501+0000 7fe368c00640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3400744e0 0x7fe3400769f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe34c034000 tx=0x7fe34c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:23.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.503+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== osd.6 v2:172.21.15.169:6824/2510135033 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe354019e80 con 0x7fe34007a490 2024-04-25T11:43:23.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.517+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 --> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe3640b17f0 con 0x7fe34007a490 2024-04-25T11:43:23.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.519+0000 7fe360c00640 1 -- 172.21.15.19:0/1585331782 <== osd.6 v2:172.21.15.169:6824/2510135033 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe3640b17f0 con 0x7fe34007a490 2024-04-25T11:43:23.505 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:23.505 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:23.505 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:23.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.519+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7fe34007a490 msgr2=0x7fe34007c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.519+0000 7fe36a000640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6824/2510135033,v1:172.21.15.169:6825/2510135033] conn(0x7fe34007a490 0x7fe34007c8e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.519+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3400744e0 msgr2=0x7fe3400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe36a000640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe3400744e0 0x7fe3400769f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe34c034000 tx=0x7fe34c02e000 comp rx=0 tx=0).stop 2024-04-25T11:43:23.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640bae50 msgr2=0x7fe364158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe36a000640 1 --2- 172.21.15.19:0/1585331782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3640bae50 0x7fe364158f40 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fe35400ba80 tx=0x7fe35400bf40 comp rx=0 tx=0).stop 2024-04-25T11:43:23.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe369600640 1 -- 172.21.15.19:0/1585331782 reap_dead start 2024-04-25T11:43:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 shutdown_connections 2024-04-25T11:43:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 >> 172.21.15.19:0/1585331782 conn(0x7fe3640d5d50 msgr2=0x7fe3640ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 shutdown_connections 2024-04-25T11:43:23.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.520+0000 7fe36a000640 1 -- 172.21.15.19:0/1585331782 wait complete. 2024-04-25T11:43:23.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T11:43:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.841+0000 7fc8bb200640 1 Processor -- start 2024-04-25T11:43:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.841+0000 7fc8bb200640 1 -- start start 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.843+0000 7fc8bb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.843+0000 7fc8bb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc8b415a810 0x7fc8b40b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.843+0000 7fc8bb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc8b41608d0 0x7fc8b4160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.843+0000 7fc8bb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b406f870 con 0x7fc8b41608d0 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.843+0000 7fc8bb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b4161220 con 0x7fc8b415a810 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.843+0000 7fc8bb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b40b4e90 con 0x7fc8b40cf2f0 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc8b41608d0 0x7fc8b4160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc8b41608d0 0x7fc8b4160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35248/0 (socket says 172.21.15.19:35248) 2024-04-25T11:43:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 -- 172.21.15.19:0/1783407356 learned_addr learned my addr 172.21.15.19:0/1783407356 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8b9e00640 1 --2- 172.21.15.19:0/1783407356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8b9400640 1 --2- 172.21.15.19:0/1783407356 >> v2:172.21.15.169:3300/0 conn(0x7fc8b415a810 0x7fc8b40b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 -- 172.21.15.19:0/1783407356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 msgr2=0x7fc8b40b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 --2- 172.21.15.19:0/1783407356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b40b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 -- 172.21.15.19:0/1783407356 >> v2:172.21.15.169:3300/0 conn(0x7fc8b415a810 msgr2=0x7fc8b40b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 --2- 172.21.15.19:0/1783407356 >> v2:172.21.15.169:3300/0 conn(0x7fc8b415a810 0x7fc8b40b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8ba800640 1 -- 172.21.15.19:0/1783407356 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b40b74d0 con 0x7fc8b41608d0 2024-04-25T11:43:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.844+0000 7fc8b9400640 1 --2- 172.21.15.19:0/1783407356 >> v2:172.21.15.169:3300/0 conn(0x7fc8b415a810 0x7fc8b40b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.845+0000 7fc8ba800640 1 --2- 172.21.15.19:0/1783407356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b41608d0 0x7fc8b4160ce0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406e010 tx=0x7fc8b002c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e9dab14a836b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.846+0000 7fc8b8a00640 1 -- 172.21.15.19:0/1783407356 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000b2e0 con 0x7fc8b41608d0 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.846+0000 7fc8b8a00640 1 -- 172.21.15.19:0/1783407356 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b0009600 con 0x7fc8b41608d0 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.846+0000 7fc8b8a00640 1 -- 172.21.15.19:0/1783407356 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000b480 con 0x7fc8b41608d0 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 -- 172.21.15.19:0/1783407356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b41608d0 msgr2=0x7fc8b4160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1783407356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b41608d0 0x7fc8b4160ce0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406e010 tx=0x7fc8b002c8a0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 -- 172.21.15.19:0/1783407356 shutdown_connections 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1783407356 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b41608d0 0x7fc8b4160ce0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1783407356 >> v2:172.21.15.169:3300/0 conn(0x7fc8b415a810 0x7fc8b40b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1783407356 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b40b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 -- 172.21.15.19:0/1783407356 >> 172.21.15.19:0/1783407356 conn(0x7fc8b40d1720 msgr2=0x7fc8b40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 -- 172.21.15.19:0/1783407356 shutdown_connections 2024-04-25T11:43:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.847+0000 7fc8bb200640 1 -- 172.21.15.19:0/1783407356 wait complete. 2024-04-25T11:43:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.849+0000 7fc8bb200640 1 Processor -- start 2024-04-25T11:43:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.850+0000 7fc8bb200640 1 -- start start 2024-04-25T11:43:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.851+0000 7fc8bb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b41f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.851+0000 7fc8bb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b415a810 0x7fc8b4152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.851+0000 7fc8bb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8b41608d0 0x7fc8b4157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.851+0000 7fc8bb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b406f500 con 0x7fc8b415a810 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.851+0000 7fc8bb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b406f220 con 0x7fc8b41608d0 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.851+0000 7fc8bb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b415ac20 con 0x7fc8b40cf2f0 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b415a810 0x7fc8b4152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b415a810 0x7fc8b4152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35250/0 (socket says 172.21.15.19:35250) 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 -- 172.21.15.19:0/1124666783 learned_addr learned my addr 172.21.15.19:0/1124666783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9e00640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b41f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8ba800640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8b41608d0 0x7fc8b4157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 -- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 msgr2=0x7fc8b41f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b41f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 -- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8b41608d0 msgr2=0x7fc8b4157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8b41608d0 0x7fc8b4157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.852+0000 7fc8b9400640 1 -- 172.21.15.19:0/1124666783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b40b0e20 con 0x7fc8b415a810 2024-04-25T11:43:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.853+0000 7fc8b9400640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b415a810 0x7fc8b4152350 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c820 tx=0x7fc8a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.853+0000 7fc8ae000640 1 -- 172.21.15.19:0/1124666783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4015070 con 0x7fc8b415a810 2024-04-25T11:43:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.853+0000 7fc8ae000640 1 -- 172.21.15.19:0/1124666783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a40079d0 con 0x7fc8b415a810 2024-04-25T11:43:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.853+0000 7fc8ae000640 1 -- 172.21.15.19:0/1124666783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4007d80 con 0x7fc8b415a810 2024-04-25T11:43:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.853+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b41536d0 con 0x7fc8b415a810 2024-04-25T11:43:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.853+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b4153c90 con 0x7fc8b415a810 2024-04-25T11:43:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.854+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b4050740 con 0x7fc8b415a810 2024-04-25T11:43:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.856+0000 7fc8ae000640 1 -- 172.21.15.19:0/1124666783 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc8a4018070 con 0x7fc8b415a810 2024-04-25T11:43:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.857+0000 7fc8ae000640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8980744b0 0x7fc8980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.857+0000 7fc8ae000640 1 -- 172.21.15.19:0/1124666783 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fc8a4094ea0 con 0x7fc8b415a810 2024-04-25T11:43:23.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.857+0000 7fc8b9e00640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8980744b0 0x7fc8980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:23.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.859+0000 7fc8b9e00640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8980744b0 0x7fc8980769c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006fd0 tx=0x7fc8a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:23.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:23.861+0000 7fc8ae000640 1 -- 172.21.15.19:0/1124666783 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8a405f830 con 0x7fc8b415a810 2024-04-25T11:43:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.066+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fc8b406c770 con 0x7fc8980744b0 2024-04-25T11:43:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.067+0000 7fc8ae000640 1 -- 172.21.15.19:0/1124666783 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc8b406c770 con 0x7fc8980744b0 2024-04-25T11:43:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T11:43:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.070+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8980744b0 msgr2=0x7fc8980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.070+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8980744b0 0x7fc8980769c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006fd0 tx=0x7fc8a8008040 comp rx=0 tx=0).stop 2024-04-25T11:43:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.070+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b415a810 msgr2=0x7fc8b4152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.070+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b415a810 0x7fc8b4152350 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c820 tx=0x7fc8a400cce0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 shutdown_connections 2024-04-25T11:43:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fc8980744b0 0x7fc8980769c0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8b41608d0 0x7fc8b4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8b415a810 0x7fc8b4152350 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 --2- 172.21.15.19:0/1124666783 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8b40cf2f0 0x7fc8b41f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 >> 172.21.15.19:0/1124666783 conn(0x7fc8b40d1720 msgr2=0x7fc8b4072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 shutdown_connections 2024-04-25T11:43:24.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.071+0000 7fc8bb200640 1 -- 172.21.15.19:0/1124666783 wait complete. 2024-04-25T11:43:24.082 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T11:43:24.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:43:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.379+0000 7f041e800640 1 Processor -- start 2024-04-25T11:43:24.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.379+0000 7f041e800640 1 -- start start 2024-04-25T11:43:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.382+0000 7f041e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0418161f20 0x7f041816ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.382+0000 7f041e800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041815e0b0 con 0x7f04180b4680 2024-04-25T11:43:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.382+0000 7f041e800640 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041806df10 con 0x7f041815dca0 2024-04-25T11:43:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.382+0000 7f041e800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f041806c760 con 0x7f0418161f20 2024-04-25T11:43:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.382+0000 7f041ca00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f04180b4680 0x7f0418161810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59554/0 (socket says 172.21.15.19:59554) 2024-04-25T11:43:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.382+0000 7f041ca00640 1 -- 172.21.15.19:0/1897309403 learned_addr learned my addr 172.21.15.19:0/1897309403 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:24.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.382+0000 7f041de00640 1 --2- 172.21.15.19:0/1897309403 >> v2:172.21.15.19:3300/0 conn(0x7f0418161f20 0x7f041816ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.383+0000 7f0417e00640 1 -- 172.21.15.19:0/1897309403 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1362768431 0 0) 0x7f0404001110 con 0x7f04180b4680 2024-04-25T11:43:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.383+0000 7f041de00640 1 -- 172.21.15.19:0/1897309403 >> v1:172.21.15.169:6789/0 conn(0x7f041815dca0 legacy=0x7f041806d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.383+0000 7f041de00640 1 -- 172.21.15.19:0/1897309403 >> v1:172.21.15.19:6789/0 conn(0x7f04180b4680 legacy=0x7f0418161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.383+0000 7f041de00640 1 -- 172.21.15.19:0/1897309403 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041806ca40 con 0x7f0418161f20 2024-04-25T11:43:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.384+0000 7f041de00640 1 --2- 172.21.15.19:0/1897309403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418161f20 0x7f041816ae50 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f0410009d90 tx=0x7f041002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37534277994a2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.384+0000 7f0417e00640 1 -- 172.21.15.19:0/1897309403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410007740 con 0x7f0418161f20 2024-04-25T11:43:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.384+0000 7f0417e00640 1 -- 172.21.15.19:0/1897309403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04100078e0 con 0x7f0418161f20 2024-04-25T11:43:24.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.384+0000 7f0417e00640 1 -- 172.21.15.19:0/1897309403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410007cb0 con 0x7f0418161f20 2024-04-25T11:43:24.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.385+0000 7f041e800640 1 -- 172.21.15.19:0/1897309403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418161f20 msgr2=0x7f041816ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.385+0000 7f041e800640 1 --2- 172.21.15.19:0/1897309403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418161f20 0x7f041816ae50 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f0410009d90 tx=0x7f041002a660 comp rx=0 tx=0).stop 2024-04-25T11:43:24.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.385+0000 7f041e800640 1 -- 172.21.15.19:0/1897309403 shutdown_connections 2024-04-25T11:43:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.385+0000 7f041e800640 1 --2- 172.21.15.19:0/1897309403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0418161f20 0x7f041816ae50 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.385+0000 7f041e800640 1 -- 172.21.15.19:0/1897309403 >> 172.21.15.19:0/1897309403 conn(0x7f04180739c0 msgr2=0x7f04180b7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.385+0000 7f041e800640 1 -- 172.21.15.19:0/1897309403 shutdown_connections 2024-04-25T11:43:24.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.385+0000 7f041e800640 1 -- 172.21.15.19:0/1897309403 wait complete. 2024-04-25T11:43:24.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.388+0000 7f041e800640 1 Processor -- start 2024-04-25T11:43:24.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.388+0000 7f041e800640 1 -- start start 2024-04-25T11:43:24.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04180b4680 0x7f041815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041e800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f041815dca0 0x7f0418144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041e800640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0418161f20 0x7f04180b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041e800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f041806d2e0 con 0x7f04180b4680 2024-04-25T11:43:24.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041e800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f041806d000 con 0x7f041815dca0 2024-04-25T11:43:24.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041e800640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f04180b4a90 con 0x7f0418161f20 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04180b4680 0x7f041815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04180b4680 0x7f041815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35278/0 (socket says 172.21.15.19:35278) 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 -- 172.21.15.19:0/3056022546 learned_addr learned my addr 172.21.15.19:0/3056022546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041ca00640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f041815dca0 0x7f0418144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041de00640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0418161f20 0x7f04180b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 -- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0418161f20 msgr2=0x7f04180b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0418161f20 0x7f04180b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 -- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f041815dca0 msgr2=0x7f0418144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f041815dca0 0x7f0418144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041d400640 1 -- 172.21.15.19:0/3056022546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181447d0 con 0x7f04180b4680 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.390+0000 7f041ca00640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f041815dca0 0x7f0418144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.391+0000 7f041d400640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04180b4680 0x7f041815e290 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c3c0 tx=0x7f040c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.391+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0052d0 con 0x7f04180b4680 2024-04-25T11:43:24.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.391+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c005490 con 0x7f04180b4680 2024-04-25T11:43:24.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.391+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c007260 con 0x7f04180b4680 2024-04-25T11:43:24.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.391+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181450c0 con 0x7f04180b4680 2024-04-25T11:43:24.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.391+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0418145680 con 0x7f04180b4680 2024-04-25T11:43:24.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.392+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04180b4230 con 0x7f04180b4680 2024-04-25T11:43:24.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.394+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f040c007400 con 0x7f04180b4680 2024-04-25T11:43:24.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.395+0000 7f0415600640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03f80744b0 0x7f03f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.395+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f040c094c10 con 0x7f04180b4680 2024-04-25T11:43:24.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.395+0000 7f0415600640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f03f807a460 0x7f03f807c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.395+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f040c010e50 con 0x7f03f807a460 2024-04-25T11:43:24.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.395+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) ==== 24+0+0 (secure 0 0 0) 0x7f040c094fc0 con 0x7f04180b4680 2024-04-25T11:43:24.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.396+0000 7f041ca00640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03f80744b0 0x7f03f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.396+0000 7f041de00640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f03f807a460 0x7f03f807c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.397+0000 7f041de00640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f03f807a460 0x7f03f807c8b0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.397+0000 7f041ca00640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03f80744b0 0x7f03f80769c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f041815ab40 tx=0x7f0404009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.400+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== osd.7 v2:172.21.15.169:6800/2355270624 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f040c010e50 con 0x7f03f807a460 2024-04-25T11:43:24.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.415+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 --> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0418076c60 con 0x7f03f807a460 2024-04-25T11:43:24.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f0415600640 1 -- 172.21.15.19:0/3056022546 <== osd.7 v2:172.21.15.169:6800/2355270624 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0418076c60 con 0x7f03f807a460 2024-04-25T11:43:24.403 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:43:24.403 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:43:24.403 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f03f807a460 msgr2=0x7f03f807c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6800/2355270624,v1:172.21.15.169:6801/2355270624] conn(0x7f03f807a460 0x7f03f807c8b0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03f80744b0 msgr2=0x7f03f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f03f80744b0 0x7f03f80769c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f041815ab40 tx=0x7f0404009040 comp rx=0 tx=0).stop 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04180b4680 msgr2=0x7f041815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 --2- 172.21.15.19:0/3056022546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04180b4680 0x7f041815e290 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c3c0 tx=0x7f040c00c880 comp rx=0 tx=0).stop 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041de00640 1 -- 172.21.15.19:0/3056022546 reap_dead start 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 shutdown_connections 2024-04-25T11:43:24.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.417+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 >> 172.21.15.19:0/3056022546 conn(0x7f04180739c0 msgr2=0x7f0418070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.418+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 shutdown_connections 2024-04-25T11:43:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.418+0000 7f041e800640 1 -- 172.21.15.19:0/3056022546 wait complete. 2024-04-25T11:43:24.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T11:43:24.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.731+0000 7fcf7ea00640 1 Processor -- start 2024-04-25T11:43:24.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.732+0000 7fcf7ea00640 1 -- start start 2024-04-25T11:43:24.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.734+0000 7fcf7ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf78144500 0x7fcf78070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.734+0000 7fcf7ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf780cdb90 0x7fcf78161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.734+0000 7fcf7ea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7806f5f0 con 0x7fcf780cdfa0 2024-04-25T11:43:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.734+0000 7fcf7ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcf78070200 con 0x7fcf780cdb90 2024-04-25T11:43:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.734+0000 7fcf7ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7806f8d0 con 0x7fcf78144500 2024-04-25T11:43:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf7ca00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcf780cdfa0 0x7fcf780ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59568/0 (socket says 172.21.15.19:59568) 2024-04-25T11:43:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf7ca00640 1 -- 172.21.15.19:0/618117250 learned_addr learned my addr 172.21.15.19:0/618117250 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:24.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf77400640 1 --2- 172.21.15.19:0/618117250 >> v2:172.21.15.19:3300/0 conn(0x7fcf780cdb90 0x7fcf78161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf77e00640 1 --2- 172.21.15.19:0/618117250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf78144500 0x7fcf78070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf77400640 1 -- 172.21.15.19:0/618117250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf78144500 msgr2=0x7fcf78070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf77400640 1 --2- 172.21.15.19:0/618117250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf78144500 0x7fcf78070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf77400640 1 -- 172.21.15.19:0/618117250 >> v1:172.21.15.19:6789/0 conn(0x7fcf780cdfa0 legacy=0x7fcf780ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf77400640 1 -- 172.21.15.19:0/618117250 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7806fbb0 con 0x7fcf780cdb90 2024-04-25T11:43:24.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.735+0000 7fcf76a00640 1 -- 172.21.15.19:0/618117250 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048344432 0 0) 0x7fcf7806f5f0 con 0x7fcf780cdfa0 2024-04-25T11:43:24.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.736+0000 7fcf77400640 1 --2- 172.21.15.19:0/618117250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780cdb90 0x7fcf78161c00 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fcf600097a0 tx=0x7fcf6002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f54c9429180ef2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf76a00640 1 -- 172.21.15.19:0/618117250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60037070 con 0x7fcf780cdb90 2024-04-25T11:43:24.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf76a00640 1 -- 172.21.15.19:0/618117250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf60032440 con 0x7fcf780cdb90 2024-04-25T11:43:24.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf76a00640 1 -- 172.21.15.19:0/618117250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60032810 con 0x7fcf780cdb90 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf7ea00640 1 -- 172.21.15.19:0/618117250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780cdb90 msgr2=0x7fcf78161c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/618117250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780cdb90 0x7fcf78161c00 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fcf600097a0 tx=0x7fcf6002a180 comp rx=0 tx=0).stop 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf7ea00640 1 -- 172.21.15.19:0/618117250 shutdown_connections 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/618117250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf780cdb90 0x7fcf78161c00 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/618117250 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf78144500 0x7fcf78070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.737+0000 7fcf7ea00640 1 -- 172.21.15.19:0/618117250 >> 172.21.15.19:0/618117250 conn(0x7fcf78073050 msgr2=0x7fcf78073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.738+0000 7fcf7ea00640 1 -- 172.21.15.19:0/618117250 shutdown_connections 2024-04-25T11:43:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.738+0000 7fcf7ea00640 1 -- 172.21.15.19:0/618117250 wait complete. 2024-04-25T11:43:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.740+0000 7fcf7ea00640 1 Processor -- start 2024-04-25T11:43:24.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.740+0000 7fcf7ea00640 1 -- start start 2024-04-25T11:43:24.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf780cdb90 0x7fcf780cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ea00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf780cdfa0 0x7fcf781572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf78144500 0x7fcf781e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7806f2e0 con 0x7fcf78144500 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7806f160 con 0x7fcf780cdb90 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ea00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fcf78162140 con 0x7fcf780cdfa0 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf78144500 0x7fcf781e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf78144500 0x7fcf781e8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35302/0 (socket says 172.21.15.19:35302) 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf7ca00640 1 -- 172.21.15.19:0/2621145165 learned_addr learned my addr 172.21.15.19:0/2621145165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf77400640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf780cdfa0 0x7fcf781572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.742+0000 7fcf77e00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf780cdb90 0x7fcf780cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ca00640 1 -- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf780cdfa0 msgr2=0x7fcf781572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ca00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf780cdfa0 0x7fcf781572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ca00640 1 -- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf780cdb90 msgr2=0x7fcf780cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ca00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf780cdb90 0x7fcf780cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ca00640 1 -- 172.21.15.19:0/2621145165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf780cb300 con 0x7fcf78144500 2024-04-25T11:43:24.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ca00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf78144500 0x7fcf781e8e80 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000c700 tx=0x7fcf7000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf6fe00640 1 -- 172.21.15.19:0/2621145165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70018070 con 0x7fcf78144500 2024-04-25T11:43:24.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf6fe00640 1 -- 172.21.15.19:0/2621145165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7000ce10 con 0x7fcf78144500 2024-04-25T11:43:24.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf6fe00640 1 -- 172.21.15.19:0/2621145165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000a880 con 0x7fcf78144500 2024-04-25T11:43:24.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf780cb5e0 con 0x7fcf78144500 2024-04-25T11:43:24.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.743+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf780cbba0 con 0x7fcf78144500 2024-04-25T11:43:24.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.745+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf78050740 con 0x7fcf78144500 2024-04-25T11:43:24.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.746+0000 7fcf6fe00640 1 -- 172.21.15.19:0/2621145165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf7001b070 con 0x7fcf78144500 2024-04-25T11:43:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.747+0000 7fcf6fe00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf54074500 0x7fcf54076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.747+0000 7fcf6fe00640 1 -- 172.21.15.19:0/2621145165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fcf70094620 con 0x7fcf78144500 2024-04-25T11:43:24.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.748+0000 7fcf77e00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf54074500 0x7fcf54076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.749+0000 7fcf77e00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf54074500 0x7fcf54076a10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.752+0000 7fcf6fe00640 1 -- 172.21.15.19:0/2621145165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf7005f0c0 con 0x7fcf78144500 2024-04-25T11:43:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.961+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fcf78162510 con 0x7fcf54074500 2024-04-25T11:43:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.962+0000 7fcf6fe00640 1 -- 172.21.15.19:0/2621145165 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcf78162510 con 0x7fcf54074500 2024-04-25T11:43:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.965+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf54074500 msgr2=0x7fcf54076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.965+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf54074500 0x7fcf54076a10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006fd0 tx=0x7fcf68008040 comp rx=0 tx=0).stop 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.965+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf78144500 msgr2=0x7fcf781e8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.965+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf78144500 0x7fcf781e8e80 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000c700 tx=0x7fcf7000cbc0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.965+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 shutdown_connections 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.965+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fcf54074500 0x7fcf54076a10 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.966+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf78144500 0x7fcf781e8e80 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.966+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcf780cdfa0 0x7fcf781572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.966+0000 7fcf7ea00640 1 --2- 172.21.15.19:0/2621145165 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcf780cdb90 0x7fcf780cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.966+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 >> 172.21.15.19:0/2621145165 conn(0x7fcf78073050 msgr2=0x7fcf78160fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.966+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 shutdown_connections 2024-04-25T11:43:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:24.966+0000 7fcf7ea00640 1 -- 172.21.15.19:0/2621145165 wait complete. 2024-04-25T11:43:24.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:43:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.271+0000 7f4af2600640 1 Processor -- start 2024-04-25T11:43:25.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.272+0000 7f4af2600640 1 -- start start 2024-04-25T11:43:25.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.274+0000 7f4af2600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aec070680 0x7f4aec1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.274+0000 7f4af2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec160360 0x7f4aec0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.274+0000 7f4af2600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aec06c7d0 con 0x7f4aec0af9e0 2024-04-25T11:43:25.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.274+0000 7f4af2600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4aec06d100 con 0x7f4aec070680 2024-04-25T11:43:25.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.274+0000 7f4af2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec06cab0 con 0x7f4aec160360 2024-04-25T11:43:25.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.274+0000 7f4aeb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4aec070680 0x7f4aec1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.274+0000 7f4aebe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4aec0af9e0 0x7f4aec0b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59578/0 (socket says 172.21.15.19:59578) 2024-04-25T11:43:25.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.275+0000 7f4aebe00640 1 -- 172.21.15.19:0/2741234822 learned_addr learned my addr 172.21.15.19:0/2741234822 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:25.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.275+0000 7f4aeaa00640 1 --2- 172.21.15.19:0/2741234822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec160360 0x7f4aec0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.275+0000 7f4aeb400640 1 -- 172.21.15.19:0/2741234822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec160360 msgr2=0x7f4aec0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.275+0000 7f4aeb400640 1 --2- 172.21.15.19:0/2741234822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec160360 0x7f4aec0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.275+0000 7f4aeb400640 1 -- 172.21.15.19:0/2741234822 >> v1:172.21.15.19:6789/0 conn(0x7f4aec0af9e0 legacy=0x7f4aec0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.275+0000 7f4aea000640 1 -- 172.21.15.19:0/2741234822 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097403643 0 0) 0x7f4aec06c7d0 con 0x7f4aec0af9e0 2024-04-25T11:43:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.275+0000 7f4aeb400640 1 -- 172.21.15.19:0/2741234822 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec06cd90 con 0x7f4aec070680 2024-04-25T11:43:25.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.277+0000 7f4aeb400640 1 --2- 172.21.15.19:0/2741234822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec070680 0x7f4aec1647f0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0097a0 tx=0x7f4adc02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ea05ea96b02132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.277+0000 7f4aea000640 1 -- 172.21.15.19:0/2741234822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc037070 con 0x7f4aec070680 2024-04-25T11:43:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.277+0000 7f4aea000640 1 -- 172.21.15.19:0/2741234822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc02adb0 con 0x7f4aec070680 2024-04-25T11:43:25.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.277+0000 7f4aea000640 1 -- 172.21.15.19:0/2741234822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc032660 con 0x7f4aec070680 2024-04-25T11:43:25.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 -- 172.21.15.19:0/2741234822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec070680 msgr2=0x7f4aec1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 --2- 172.21.15.19:0/2741234822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec070680 0x7f4aec1647f0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0097a0 tx=0x7f4adc02a060 comp rx=0 tx=0).stop 2024-04-25T11:43:25.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 -- 172.21.15.19:0/2741234822 shutdown_connections 2024-04-25T11:43:25.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 --2- 172.21.15.19:0/2741234822 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec160360 0x7f4aec0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 --2- 172.21.15.19:0/2741234822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec070680 0x7f4aec1647f0 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 -- 172.21.15.19:0/2741234822 >> 172.21.15.19:0/2741234822 conn(0x7f4aec0d1ae0 msgr2=0x7f4aec0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:25.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 -- 172.21.15.19:0/2741234822 shutdown_connections 2024-04-25T11:43:25.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.278+0000 7f4af2600640 1 -- 172.21.15.19:0/2741234822 wait complete. 2024-04-25T11:43:25.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.281+0000 7f4af2600640 1 Processor -- start 2024-04-25T11:43:25.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.282+0000 7f4af2600640 1 -- start start 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.283+0000 7f4af2600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aec070680 0x7f4aec15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.283+0000 7f4af2600640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec0af9e0 0x7f4aec14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.283+0000 7f4af2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec160360 0x7f4aec1eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4af2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0b6520 con 0x7f4aec160360 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4af2600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0b63a0 con 0x7f4aec070680 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4af2600640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec06d590 con 0x7f4aec0af9e0 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec160360 0x7f4aec1eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec160360 0x7f4aec1eb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35322/0 (socket says 172.21.15.19:35322) 2024-04-25T11:43:25.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 -- 172.21.15.19:0/3484804600 learned_addr learned my addr 172.21.15.19:0/3484804600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aeaa00640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec0af9e0 0x7f4aec14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aeb400640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aec070680 0x7f4aec15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 -- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec0af9e0 msgr2=0x7f4aec14e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec0af9e0 0x7f4aec14e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 -- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aec070680 msgr2=0x7f4aec15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aec070680 0x7f4aec15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aebe00640 1 -- 172.21.15.19:0/3484804600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec0c8320 con 0x7f4aec160360 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.284+0000 7f4aeb400640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aec070680 0x7f4aec15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.285+0000 7f4aebe00640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec160360 0x7f4aec1eb5f0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c700 tx=0x7f4ae400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.285+0000 7f4ae3400640 1 -- 172.21.15.19:0/3484804600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4018070 con 0x7f4aec160360 2024-04-25T11:43:25.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.285+0000 7f4ae3400640 1 -- 172.21.15.19:0/3484804600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae400ce10 con 0x7f4aec160360 2024-04-25T11:43:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.285+0000 7f4ae3400640 1 -- 172.21.15.19:0/3484804600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400a880 con 0x7f4aec160360 2024-04-25T11:43:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.285+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec1f6400 con 0x7f4aec160360 2024-04-25T11:43:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.285+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec15e4a0 con 0x7f4aec160360 2024-04-25T11:43:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.287+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aec15e670 con 0x7f4aec160360 2024-04-25T11:43:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.288+0000 7f4ae3400640 1 -- 172.21.15.19:0/3484804600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ae400aa20 con 0x7f4aec160360 2024-04-25T11:43:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.289+0000 7f4ae3400640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4acc0744b0 0x7f4acc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.289+0000 7f4ae3400640 1 -- 172.21.15.19:0/3484804600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7f4ae4094630 con 0x7f4aec160360 2024-04-25T11:43:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.290+0000 7f4aeb400640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4acc0744b0 0x7f4acc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.291+0000 7f4aeb400640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4acc0744b0 0x7f4acc0769c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4adc034000 tx=0x7f4adc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:25.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.294+0000 7f4ae3400640 1 -- 172.21.15.19:0/3484804600 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ae405efc0 con 0x7f4aec160360 2024-04-25T11:43:25.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.496+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 --> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4aec1f65d0 con 0x7f4acc0744b0 2024-04-25T11:43:25.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.499+0000 7f4ae3400640 1 -- 172.21.15.19:0/3484804600 <== mgr.4105 v2:172.21.15.169:6832/1049297059 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13400 (secure 0 0 0) 0x7f4aec1f65d0 con 0x7f4acc0744b0 2024-04-25T11:43:25.485 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:25.486 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4acc0744b0 msgr2=0x7f4acc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4acc0744b0 0x7f4acc0769c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4adc034000 tx=0x7f4adc02e000 comp rx=0 tx=0).stop 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec160360 msgr2=0x7f4aec1eb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec160360 0x7f4aec1eb5f0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c700 tx=0x7f4ae400cbc0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 shutdown_connections 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7f4acc0744b0 0x7f4acc0769c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4aec160360 0x7f4aec1eb5f0 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aec0af9e0 0x7f4aec14e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 --2- 172.21.15.19:0/3484804600 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aec070680 0x7f4aec15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 >> 172.21.15.19:0/3484804600 conn(0x7f4aec0d1ae0 msgr2=0x7f4aec164260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 shutdown_connections 2024-04-25T11:43:25.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.502+0000 7f4af2600640 1 -- 172.21.15.19:0/3484804600 wait complete. 2024-04-25T11:43:25.519 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2024-04-25T11:43:24.853052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":302848,"kb_used_data":15976,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671872,"statfs":{"total":773094113280,"available":772783996928,"internally_reserved":0,"allocated":16359424,"data_stored":8464400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":293654066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003999"},"pg_stats":[{"pgid":"1.0","version":"148'74","reported_seq":372,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T11:43:19.966700+0000","last_change":"2024-04-25T11:43:19.966783+0000","last_active":"2024-04-25T11:43:19.966700+0000","last_peered":"2024-04-25T11:43:19.966700+0000","last_clean":"2024-04-25T11:43:19.966700+0000","last_became_active":"2024-04-25T11:27:14.420987+0000","last_became_peered":"2024-04-25T11:27:14.420987+0000","last_unstale":"2024-04-25T11:43:19.966700+0000","last_undegraded":"2024-04-25T11:43:19.966700+0000","last_fullsized":"2024-04-25T11:43:19.966700+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'74","last_scrub_stamp":"2024-04-25T11:43:19.966549+0000","last_deep_scrub":"148'74","last_deep_scrub_stamp":"2024-04-25T11:43:19.966549+0000","last_clean_scrub_stamp":"2024-04-25T11:43:19.966549+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:35.145427+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509636,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509638,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41352,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509638,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33032,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509638,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37768,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2490368,"data_stored":1500826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509637,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37192,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334648,"statfs":{"total":96636764160,"available":96598679552,"internally_reserved":0,"allocated":1896448,"data_stored":910458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542340,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37768,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334072,"statfs":{"total":96636764160,"available":96598089728,"internally_reserved":0,"allocated":2490368,"data_stored":1500826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:43:25.522 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T11:43:25.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.798+0000 7fe4cb200640 1 Processor -- start 2024-04-25T11:43:25.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.798+0000 7fe4cb200640 1 -- start start 2024-04-25T11:43:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.800+0000 7fe4cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 0x7fe4cc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4cb200640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe4cc15ee60 0x7fe4cc15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4cb200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4cc06df20 con 0x7fe4cc06db10 2024-04-25T11:43:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4cb200640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fe4cc15f270 con 0x7fe4cc15ee60 2024-04-25T11:43:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc144750 con 0x7fe4cc144340 2024-04-25T11:43:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4c9400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe4cc06db10 0x7fe4cc15e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59590/0 (socket says 172.21.15.19:59590) 2024-04-25T11:43:25.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4c9400640 1 -- 172.21.15.19:0/2198230043 learned_addr learned my addr 172.21.15.19:0/2198230043 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-04-25T11:43:25.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4c9e00640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 0x7fe4cc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.801+0000 7fe4ca800640 1 --2- 172.21.15.19:0/2198230043 >> v2:172.21.15.169:3300/0 conn(0x7fe4cc15ee60 0x7fe4cc15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075534911 0 0) 0x7fe4b8001110 con 0x7fe4cc06db10 2024-04-25T11:43:25.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4b0003520 con 0x7fe4cc06db10 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3695044409 0 0) 0x7fe4b0003520 con 0x7fe4cc06db10 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 msgr2=0x7fe4cc06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 0x7fe4cc06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 >> v2:172.21.15.169:3300/0 conn(0x7fe4cc15ee60 msgr2=0x7fe4cc15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 --2- 172.21.15.19:0/2198230043 >> v2:172.21.15.169:3300/0 conn(0x7fe4cc15ee60 0x7fe4cc15f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc070000 con 0x7fe4cc06db10 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2771392351 0 0) 0x7fe4b8001840 con 0x7fe4cc06db10 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 >> v1:172.21.15.19:6789/0 conn(0x7fe4cc06db10 legacy=0x7fe4cc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 0x7fe4b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b0003f80 0x7fe4b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0004d40 0x7fe4b0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc144750 con 0x7fe4b0004d40 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b000ba70 con 0x7fe4b0003f80 2024-04-25T11:43:25.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.802+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b000bbf0 con 0x7fe4cc144340 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4ca800640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0004d40 0x7fe4b0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4c9400640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b0003f80 0x7fe4b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4c9e00640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 0x7fe4b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4ca800640 1 -- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 msgr2=0x7fe4b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4ca800640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 0x7fe4b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4ca800640 1 -- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b0003f80 msgr2=0x7fe4b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4ca800640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b0003f80 0x7fe4b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4ca800640 1 -- 172.21.15.19:0/2198230043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b0003520 con 0x7fe4b0004d40 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4c9400640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4b0003f80 0x7fe4b0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.803+0000 7fe4c9e00640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc144340 0x7fe4b00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.804+0000 7fe4ca800640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0004d40 0x7fe4b0005150 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009ac0 tx=0x7fe4b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f67148ad3622b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.804+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4036070 con 0x7fe4b0004d40 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.804+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c4031400 con 0x7fe4b0004d40 2024-04-25T11:43:25.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.804+0000 7fe4c8a00640 1 -- 172.21.15.19:0/2198230043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4009690 con 0x7fe4b0004d40 2024-04-25T11:43:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.805+0000 7fe4cb200640 1 -- 172.21.15.19:0/2198230043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0004d40 msgr2=0x7fe4b0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.805+0000 7fe4cb200640 1 --2- 172.21.15.19:0/2198230043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4b0004d40 0x7fe4b0005150 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009ac0 tx=0x7fe4b0005830 comp rx=0 tx=0).stop 2024-04-25T11:43:25.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.805+0000 7fe4ca800640 1 -- 172.21.15.19:0/2198230043 reap_dead start 2024-04-25T11:43:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.805+0000 7fe4cb200640 1 -- 172.21.15.19:0/2198230043 shutdown_connections 2024-04-25T11:43:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.805+0000 7fe4cb200640 1 -- 172.21.15.19:0/2198230043 >> 172.21.15.19:0/2198230043 conn(0x7fe4cc072f40 msgr2=0x7fe4cc073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.805+0000 7fe4cb200640 1 -- 172.21.15.19:0/2198230043 shutdown_connections 2024-04-25T11:43:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.805+0000 7fe4cb200640 1 -- 172.21.15.19:0/2198230043 wait complete. 2024-04-25T11:43:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.808+0000 7fe4cb200640 1 Processor -- start 2024-04-25T11:43:25.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.808+0000 7fe4cb200640 1 -- start start 2024-04-25T11:43:25.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4cb200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc139160 0x7fe4cc1e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4cb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4cc15aa50 0x7fe4cc13a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4cb200640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc15ae60 0x7fe4cc15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4cb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc06f800 con 0x7fe4cc15aa50 2024-04-25T11:43:25.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4cb200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc06f520 con 0x7fe4cc139160 2024-04-25T11:43:25.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4cb200640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fe4cc06fae0 con 0x7fe4cc15ae60 2024-04-25T11:43:25.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4cc15aa50 0x7fe4cc13a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4c9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4cc15aa50 0x7fe4cc13a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35336/0 (socket says 172.21.15.19:35336) 2024-04-25T11:43:25.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4c9400640 1 -- 172.21.15.19:0/3506842474 learned_addr learned my addr 172.21.15.19:0/3506842474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.810+0000 7fe4c9e00640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc139160 0x7fe4cc1e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4ca800640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc15ae60 0x7fe4cc15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c9400640 1 -- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc15ae60 msgr2=0x7fe4cc15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c9400640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc15ae60 0x7fe4cc15d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c9400640 1 -- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc139160 msgr2=0x7fe4cc1e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c9400640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc139160 0x7fe4cc1e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:25.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c9400640 1 -- 172.21.15.19:0/3506842474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc139570 con 0x7fe4cc15aa50 2024-04-25T11:43:25.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c9400640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4cc15aa50 0x7fe4cc13a8c0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80012b0 tx=0x7fe4b800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:25.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8019070 con 0x7fe4cc15aa50 2024-04-25T11:43:25.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b8004cb0 con 0x7fe4cc15aa50 2024-04-25T11:43:25.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8004e60 con 0x7fe4cc15aa50 2024-04-25T11:43:25.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.811+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc139820 con 0x7fe4cc15aa50 2024-04-25T11:43:25.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.812+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc139de0 con 0x7fe4cc15aa50 2024-04-25T11:43:25.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.813+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc076c60 con 0x7fe4cc15aa50 2024-04-25T11:43:25.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.814+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe4b801c070 con 0x7fe4cc15aa50 2024-04-25T11:43:25.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.816+0000 7fe4c2000640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe4ac0744b0 0x7fe4ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:25.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.816+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fe4b8094b60 con 0x7fe4cc15aa50 2024-04-25T11:43:25.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.816+0000 7fe4c9e00640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe4ac0744b0 0x7fe4ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:25.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.818+0000 7fe4c9e00640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe4ac0744b0 0x7fe4ac0769c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc1437a0 tx=0x7fe4bc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:25.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:25.820+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4cc06df20 con 0x7fe4cc15aa50 2024-04-25T11:43:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.022+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fe4cc0b7920 con 0x7fe4cc15aa50 2024-04-25T11:43:26.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.029+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7fe4b805f580 con 0x7fe4cc15aa50 2024-04-25T11:43:26.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.030+0000 7fe4c2000640 1 -- 172.21.15.19:0/3506842474 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe4b809a080 con 0x7fe4cc15aa50 2024-04-25T11:43:26.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.032+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe4ac0744b0 msgr2=0x7fe4ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:26.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.032+0000 7fe4cb200640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe4ac0744b0 0x7fe4ac0769c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc1437a0 tx=0x7fe4bc00b040 comp rx=0 tx=0).stop 2024-04-25T11:43:26.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.032+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4cc15aa50 msgr2=0x7fe4cc13a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:26.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.032+0000 7fe4cb200640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4cc15aa50 0x7fe4cc13a8c0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80012b0 tx=0x7fe4b800df70 comp rx=0 tx=0).stop 2024-04-25T11:43:26.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 shutdown_connections 2024-04-25T11:43:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fe4ac0744b0 0x7fe4ac0769c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4cc15ae60 0x7fe4cc15d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4cc15aa50 0x7fe4cc13a8c0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 --2- 172.21.15.19:0/3506842474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4cc139160 0x7fe4cc1e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 >> 172.21.15.19:0/3506842474 conn(0x7fe4cc072f40 msgr2=0x7fe4cc0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 shutdown_connections 2024-04-25T11:43:26.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.033+0000 7fe4cb200640 1 -- 172.21.15.19:0/3506842474 wait complete. 2024-04-25T11:43:26.043 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:43:26.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:26.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.335+0000 7fce54a00640 1 Processor -- start 2024-04-25T11:43:26.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.335+0000 7fce54a00640 1 -- start start 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce54a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce50070e20 0x7fce5015c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce54a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce5006f180 0x7fce5006ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce54a00640 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fce5006d390 0x7fce500cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce54a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fce5006d7a0 con 0x7fce5006f180 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce54a00640 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 -- 0x7fce50071230 con 0x7fce5006d390 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce54a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fce5006f8f0 con 0x7fce50070e20 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce45e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce5006f180 0x7fce5006ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce45e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce5006f180 0x7fce5006ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35344/0 (socket says 172.21.15.19:35344) 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.338+0000 7fce45e00640 1 -- 172.21.15.19:0/1868743050 learned_addr learned my addr 172.21.15.19:0/1868743050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:26.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.339+0000 7fce4e800640 1 --2- 172.21.15.19:0/1868743050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce50070e20 0x7fce5015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:26.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.339+0000 7fce4f200640 1 --2- 172.21.15.19:0/1868743050 >> v2:172.21.15.169:3300/0 conn(0x7fce5006d390 0x7fce500cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:26.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.339+0000 7fce45e00640 1 -- 172.21.15.19:0/1868743050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce50070e20 msgr2=0x7fce5015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:26.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.339+0000 7fce45e00640 1 --2- 172.21.15.19:0/1868743050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce50070e20 0x7fce5015c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.339+0000 7fce45e00640 1 -- 172.21.15.19:0/1868743050 >> v2:172.21.15.169:3300/0 conn(0x7fce5006d390 msgr2=0x7fce500cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:26.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.339+0000 7fce45e00640 1 --2- 172.21.15.19:0/1868743050 >> v2:172.21.15.169:3300/0 conn(0x7fce5006d390 0x7fce500cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.339+0000 7fce45e00640 1 -- 172.21.15.19:0/1868743050 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5006fbd0 con 0x7fce5006f180 2024-04-25T11:43:26.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.340+0000 7fce45e00640 1 --2- 172.21.15.19:0/1868743050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006f180 0x7fce5006ce50 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7fce50072cb0 tx=0x7fce3802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2504feac41f7117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:26.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.340+0000 7fce4de00640 1 -- 172.21.15.19:0/1868743050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38032790 con 0x7fce5006f180 2024-04-25T11:43:26.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.340+0000 7fce4de00640 1 -- 172.21.15.19:0/1868743050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fce38038070 con 0x7fce5006f180 2024-04-25T11:43:26.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.340+0000 7fce4de00640 1 -- 172.21.15.19:0/1868743050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38032b70 con 0x7fce5006f180 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.341+0000 7fce54a00640 1 -- 172.21.15.19:0/1868743050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006f180 msgr2=0x7fce5006ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.341+0000 7fce54a00640 1 --2- 172.21.15.19:0/1868743050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006f180 0x7fce5006ce50 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7fce50072cb0 tx=0x7fce3802c890 comp rx=0 tx=0).stop 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.341+0000 7fce54a00640 1 -- 172.21.15.19:0/1868743050 shutdown_connections 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.341+0000 7fce54a00640 1 --2- 172.21.15.19:0/1868743050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006f180 0x7fce5006ce50 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.341+0000 7fce54a00640 1 --2- 172.21.15.19:0/1868743050 >> v2:172.21.15.169:3300/0 conn(0x7fce5006d390 0x7fce500cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.341+0000 7fce54a00640 1 --2- 172.21.15.19:0/1868743050 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce50070e20 0x7fce5015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.341+0000 7fce54a00640 1 -- 172.21.15.19:0/1868743050 >> 172.21.15.19:0/1868743050 conn(0x7fce500d1720 msgr2=0x7fce500d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.342+0000 7fce54a00640 1 -- 172.21.15.19:0/1868743050 shutdown_connections 2024-04-25T11:43:26.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.342+0000 7fce54a00640 1 -- 172.21.15.19:0/1868743050 wait complete. 2024-04-25T11:43:26.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.345+0000 7fce54a00640 1 Processor -- start 2024-04-25T11:43:26.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.345+0000 7fce54a00640 1 -- start start 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.347+0000 7fce54a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006d390 0x7fce500aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.347+0000 7fce54a00640 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce5006f180 0x7fce5014de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.347+0000 7fce54a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce50070e20 0x7fce5014e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.347+0000 7fce54a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce5006f590 con 0x7fce5006d390 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.347+0000 7fce54a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fce50070190 con 0x7fce50070e20 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.347+0000 7fce54a00640 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 -- 0x7fce5006c5c0 con 0x7fce5006f180 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006d390 0x7fce500aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006d390 0x7fce500aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35348/0 (socket says 172.21.15.19:35348) 2024-04-25T11:43:26.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 -- 172.21.15.19:0/1276746011 learned_addr learned my addr 172.21.15.19:0/1276746011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:43:26.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4f200640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce50070e20 0x7fce5014e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce45e00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce5006f180 0x7fce5014de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 -- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce5006f180 msgr2=0x7fce5014de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce5006f180 0x7fce5014de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 -- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce50070e20 msgr2=0x7fce5014e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce50070e20 0x7fce5014e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 -- 172.21.15.19:0/1276746011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce50152ba0 con 0x7fce5006d390 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.348+0000 7fce4e800640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006d390 0x7fce500aef80 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c820 tx=0x7fce4000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.349+0000 7fce47400640 1 -- 172.21.15.19:0/1276746011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40015070 con 0x7fce5006d390 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.349+0000 7fce47400640 1 -- 172.21.15.19:0/1276746011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fce40004020 con 0x7fce5006d390 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.349+0000 7fce47400640 1 -- 172.21.15.19:0/1276746011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40004410 con 0x7fce5006d390 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.349+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce50152e80 con 0x7fce5006d390 2024-04-25T11:43:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.349+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce50153440 con 0x7fce5006d390 2024-04-25T11:43:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.350+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce50050740 con 0x7fce5006d390 2024-04-25T11:43:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.351+0000 7fce47400640 1 -- 172.21.15.19:0/1276746011 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce40007830 con 0x7fce5006d390 2024-04-25T11:43:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.352+0000 7fce47400640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce2c0744e0 0x7fce2c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.353+0000 7fce47400640 1 -- 172.21.15.19:0/1276746011 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5029+0+0 (secure 0 0 0) 0x7fce40094ea0 con 0x7fce5006d390 2024-04-25T11:43:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.353+0000 7fce45e00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce2c0744e0 0x7fce2c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.355+0000 7fce45e00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce2c0744e0 0x7fce2c0769f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fce38037000 tx=0x7fce38032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:26.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.357+0000 7fce47400640 1 -- 172.21.15.19:0/1276746011 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce4005f830 con 0x7fce5006d390 2024-04-25T11:43:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.602+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce500b2d20 con 0x7fce5006d390 2024-04-25T11:43:26.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.603+0000 7fce47400640 1 -- 172.21.15.19:0/1276746011 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 374 v374) ==== 95+0+3878 (secure 0 0 0) 0x7fce4005f130 con 0x7fce5006d390 2024-04-25T11:43:26.590 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:43:26.590 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":374,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3385178705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3385178705},{"type":"v1","addr":"172.21.15.19:6835","nonce":3385178705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1738598484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1738598484},{"type":"v1","addr":"172.21.15.169:6837","nonce":1738598484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":359},{"gid":7115,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/908724351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":908724351},{"type":"v1","addr":"172.21.15.169:6835","nonce":908724351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":366},{"gid":11913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2133549290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2133549290},{"type":"v1","addr":"172.21.15.19:6834","nonce":2133549290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":374}],"filesystems":[]} 2024-04-25T11:43:26.590 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 374 2024-04-25T11:43:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce2c0744e0 msgr2=0x7fce2c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce2c0744e0 0x7fce2c0769f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fce38037000 tx=0x7fce38032000 comp rx=0 tx=0).stop 2024-04-25T11:43:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006d390 msgr2=0x7fce500aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:26.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006d390 0x7fce500aef80 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c820 tx=0x7fce4000cce0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 shutdown_connections 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:6832/1049297059,v1:172.21.15.169:6833/1049297059] conn(0x7fce2c0744e0 0x7fce2c0769f0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce50070e20 0x7fce5014e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce5006f180 0x7fce5014de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 --2- 172.21.15.19:0/1276746011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce5006d390 0x7fce500aef80 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 >> 172.21.15.19:0/1276746011 conn(0x7fce500d1720 msgr2=0x7fce50072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 shutdown_connections 2024-04-25T11:43:26.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:43:26.606+0000 7fce54a00640 1 -- 172.21.15.19:0/1276746011 wait complete. 2024-04-25T11:43:26.617 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T11:43:26.618 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T11:43:26.618 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:43:26.618 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:26.649 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:43:26.649 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:43:26.649 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:26.748 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:43:26.748 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:43:26.748 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:26.796 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:43:26.797 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:43:26.797 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:26.840 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:43:26.840 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T11:43:26.840 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T11:43:26.841 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:26.938 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T11:43:26.938 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T11:43:26.938 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.018 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T11:43:27.018 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T11:43:27.018 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.096 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T11:43:27.096 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T11:43:27.096 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.160 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T11:43:27.161 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T11:43:27.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.231 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T11:43:27.231 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T11:43:27.231 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.305 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T11:43:27.306 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T11:43:27.306 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.386 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T11:43:27.387 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T11:43:27.387 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.460 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T11:43:27.460 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T11:43:27.461 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T11:43:27.461 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.547 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T11:43:27.547 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T11:43:27.547 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.639 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T11:43:27.639 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T11:43:27.639 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T11:43:27.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.716 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T11:43:27.716 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T11:43:27.716 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.763 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T11:43:27.763 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T11:43:27.763 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:43:27.810 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T11:43:27.811 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T11:43:27.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | egrep -v 'MDS abort because newly corrupt dentry' | head -n 1 2024-04-25T11:43:27.936 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:43:27.936 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T11:43:28.186 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:43:28.186 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T11:43:28.318 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:43:28.318 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T11:43:28.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:43:28.458 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T11:43:28.614 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:43:28.614 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T11:43:28.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:43:28.953 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T11:43:29.105 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:43:29.106 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T11:43:29.247 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:43:29.247 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T11:43:29.404 INFO:tasks.ceph:Archiving mon data... 2024-04-25T11:43:29.405 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/data/mon.a.tgz 2024-04-25T11:43:29.405 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-04-25T11:43:29.464 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.grYnufIT3G 2024-04-25T11:43:29.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.grYnufIT3G 2024-04-25T11:43:29.716 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.grYnufIT3G 2024-04-25T11:43:29.864 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.grYnufIT3G is 619KB 2024-04-25T11:43:29.978 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.grYnufIT3G 2024-04-25T11:43:30.038 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/data/mon.b.tgz 2024-04-25T11:43:30.038 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-25T11:43:30.097 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.jlersDV4El 2024-04-25T11:43:30.098 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jlersDV4El 2024-04-25T11:43:30.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.jlersDV4El 2024-04-25T11:43:30.480 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.jlersDV4El is 640KB 2024-04-25T11:43:30.591 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.jlersDV4El 2024-04-25T11:43:30.647 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/data/mon.c.tgz 2024-04-25T11:43:30.647 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-04-25T11:43:30.695 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.u5dh2KptBN 2024-04-25T11:43:30.696 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.u5dh2KptBN 2024-04-25T11:43:30.924 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.u5dh2KptBN 2024-04-25T11:43:31.064 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.u5dh2KptBN is 640KB 2024-04-25T11:43:31.176 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.u5dh2KptBN 2024-04-25T11:43:31.231 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T11:43:31.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:43:31.239 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:43:31.385 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:43:31.440 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:43:31.440 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T11:43:31.442 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/remote/smithi019/crash 2024-04-25T11:43:31.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:43:31.583 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/remote/smithi169/crash 2024-04-25T11:43:31.584 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:43:31.669 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T11:43:31.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:43:31.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:43:31.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:43:31.895 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:43:31.983 INFO:tasks.ceph:Compressing logs... 2024-04-25T11:43:31.983 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:43:31.991 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:43:32.069 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33496.log 2024-04-25T11:43:32.070 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:43:32.071 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.33496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38860.log 2024-04-25T11:43:32.071 INFO:teuthology.orchestra.run.smithi169.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33496.log.gz 2024-04-25T11:43:32.072 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:43:32.073 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.38860.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.1.38860.log.gz -5 2024-04-25T11:43:32.073 INFO:teuthology.orchestra.run.smithi169.stderr: --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T11:43:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91551.log 2024-04-25T11:43:32.075 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T11:43:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57272.log 2024-04-25T11:43:32.076 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T11:43:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72810.log 2024-04-25T11:43:32.077 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91551.log.gz 2024-04-25T11:43:32.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39134.log 2024-04-25T11:43:32.078 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57272.log.gz 2024-04-25T11:43:32.078 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33358.log 2024-04-25T11:43:32.079 INFO:teuthology.orchestra.run.smithi169.stderr: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:43:32.079 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33452.log 2024-04-25T11:43:32.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78666.log 2024-04-25T11:43:32.080 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72810.log.gz 2024-04-25T11:43:32.081 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60281.log 2024-04-25T11:43:32.081 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39134.log.gz 2024-04-25T11:43:32.082 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.33358.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T11:43:32.082 INFO:teuthology.orchestra.run.smithi169.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33358.log.gz 2024-04-25T11:43:32.082 INFO:teuthology.orchestra.run.smithi169.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:43:32.083 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.33452.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T11:43:32.083 INFO:teuthology.orchestra.run.smithi169.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.33452.log.gz 2024-04-25T11:43:32.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89037.log 2024-04-25T11:43:32.084 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78666.log.gz 2024-04-25T11:43:32.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60281.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60281.log.gz 2024-04-25T11:43:32.084 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59605.log 2024-04-25T11:43:32.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65289.log 2024-04-25T11:43:32.086 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T11:43:32.086 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33187.log 2024-04-25T11:43:32.086 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89037.log.gz 2024-04-25T11:43:32.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42050.log 2024-04-25T11:43:32.086 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59605.log.gz 2024-04-25T11:43:32.087 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44243.log 2024-04-25T11:43:32.087 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65289.log.gz 2024-04-25T11:43:32.087 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T11:43:32.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76415.log 2024-04-25T11:43:32.088 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42050.log.gz 2024-04-25T11:43:32.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52004.log 2024-04-25T11:43:32.089 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44243.log.gz 2024-04-25T11:43:32.089 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.33187.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33187.log.gz 2024-04-25T11:43:32.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76415.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76415.log.gz 2024-04-25T11:43:32.090 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59977.log 2024-04-25T11:43:32.091 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33231.log 2024-04-25T11:43:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52004.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52004.log.gz 2024-04-25T11:43:32.091 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84047.log 2024-04-25T11:43:32.092 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39275.log 2024-04-25T11:43:32.093 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85601.log 2024-04-25T11:43:32.093 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59977.log.gz 2024-04-25T11:43:32.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75041.log 2024-04-25T11:43:32.094 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84047.log.gz 2024-04-25T11:43:32.094 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.33231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33231.log.gz 2024-04-25T11:43:32.094 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33398.log 2024-04-25T11:43:32.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85601.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85601.log.gz 2024-04-25T11:43:32.095 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88475.log 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33278.log 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44573.log 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75041.log.gz 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60659.log 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88475.log.gz 2024-04-25T11:43:32.099 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.33398.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33398.log.gzgzip/var/log/ceph/ceph-client.3.39275.log: -5 --verbose -- 2024-04-25T11:43:32.099 INFO:teuthology.orchestra.run.smithi169.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39275.log.gz 2024-04-25T11:43:32.099 INFO:teuthology.orchestra.run.smithi169.stderr: /var/log/ceph/ceph-client.admin.33318.log 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44573.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44573.log.gz 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85994.log 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60659.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.60659.log.gz 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.70228.log 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85994.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85994.log.gz 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53281.log 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49877.log 2024-04-25T11:43:32.103 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70228.log.gz 2024-04-25T11:43:32.104 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49440.log 2024-04-25T11:43:32.105 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53281.log.gz 2024-04-25T11:43:32.105 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.admin.33278.log: /var/log/ceph/ceph-client.admin.33318.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T11:43:32.105 INFO:teuthology.orchestra.run.smithi169.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33278.log.gz 2024-04-25T11:43:32.105 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mds.d.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33318.log.gz 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50297.log 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49877.log.gz 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51794.log 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49440.log.gz 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91509.log 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50297.log.gz 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83611.log 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51794.log.gz 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61619.log 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91509.log.gz 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83611.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.83611.log.gz 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.88786.log 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61619.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61619.log.gz 2024-04-25T11:43:32.113 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87042.log 2024-04-25T11:43:32.114 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67204.log 2024-04-25T11:43:32.114 INFO:teuthology.orchestra.run.smithi019.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.88786.log.gz 2024-04-25T11:43:32.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87042.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87042.log.gz 2024-04-25T11:43:32.115 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75531.log 2024-04-25T11:43:32.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50129.log 2024-04-25T11:43:32.116 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67204.log.gz 2024-04-25T11:43:32.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58278.log 2024-04-25T11:43:32.117 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75531.log.gz 2024-04-25T11:43:32.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38472.log 2024-04-25T11:43:32.119 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50129.log.gz 2024-04-25T11:43:32.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40002.log 2024-04-25T11:43:32.120 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58278.log.gz 2024-04-25T11:43:32.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62741.log 2024-04-25T11:43:32.121 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38472.log.gz 2024-04-25T11:43:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49961.log 2024-04-25T11:43:32.122 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40002.log.gz 2024-04-25T11:43:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73929.log 2024-04-25T11:43:32.123 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62741.log.gz 2024-04-25T11:43:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66928.log 2024-04-25T11:43:32.124 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49961.log.gz 2024-04-25T11:43:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45102.log 2024-04-25T11:43:32.126 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.73929.log.gz 2024-04-25T11:43:32.126 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75311.log 2024-04-25T11:43:32.127 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66928.log.gz 2024-04-25T11:43:32.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54724.log 2024-04-25T11:43:32.128 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45102.log.gz 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91635.log 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75311.log.gz 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54724.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54724.log.gz 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54153.log 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63911.log 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91635.log.gz 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82511.log 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54153.log.gz 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66582.log 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.63911.log.gz 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90970.log 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82511.log.gz 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48043.log 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66582.log.gz 2024-04-25T11:43:32.137 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80575.log 2024-04-25T11:43:32.137 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90970.log.gz 2024-04-25T11:43:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48043.log.gz 2024-04-25T11:43:32.139 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61482.log 2024-04-25T11:43:32.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72361.log 2024-04-25T11:43:32.140 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80575.log.gz 2024-04-25T11:43:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61482.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61482.log.gz 2024-04-25T11:43:32.141 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53012.log 2024-04-25T11:43:32.142 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42302.log 2024-04-25T11:43:32.143 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72361.log.gz 2024-04-25T11:43:32.144 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49236.log 2024-04-25T11:43:32.144 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53012.log.gz 2024-04-25T11:43:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42302.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42302.log.gz 2024-04-25T11:43:32.145 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84442.log 2024-04-25T11:43:32.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51549.log 2024-04-25T11:43:32.147 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49236.log.gz 2024-04-25T11:43:32.148 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.84442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84442.log.gz 2024-04-25T11:43:32.148 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63449.log 2024-04-25T11:43:32.149 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51549.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51549.log.gz 2024-04-25T11:43:32.149 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57062.log 2024-04-25T11:43:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85741.log 2024-04-25T11:43:32.150 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63449.log.gz 2024-04-25T11:43:32.152 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87754.log 2024-04-25T11:43:32.152 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57062.log.gz 2024-04-25T11:43:32.153 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86149.log 2024-04-25T11:43:32.154 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87754.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.85741.log.gz 2024-04-25T11:43:32.155 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55754.log 2024-04-25T11:43:32.155 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87754.log.gz 2024-04-25T11:43:32.156 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70288.log 2024-04-25T11:43:32.156 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86149.log.gz 2024-04-25T11:43:32.157 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42134.log 2024-04-25T11:43:32.157 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55754.log.gz 2024-04-25T11:43:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49039.log 2024-04-25T11:43:32.158 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.70288.log.gz 2024-04-25T11:43:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83476.log 2024-04-25T11:43:32.160 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42134.log.gz 2024-04-25T11:43:32.160 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63708.log 2024-04-25T11:43:32.160 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49039.log.gz 2024-04-25T11:43:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44183.log 2024-04-25T11:43:32.162 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83476.log.gz 2024-04-25T11:43:32.162 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40254.log 2024-04-25T11:43:32.163 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.63708.log.gz 2024-04-25T11:43:32.164 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44183.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44183.log.gz 2024-04-25T11:43:32.164 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38682.log 2024-04-25T11:43:32.165 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40254.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40254.log.gz 2024-04-25T11:43:32.165 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64639.log 2024-04-25T11:43:32.166 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38682.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.38682.log.gz 2024-04-25T11:43:32.167 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37915.log 2024-04-25T11:43:32.167 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91971.log 2024-04-25T11:43:32.168 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64639.log.gz 2024-04-25T11:43:32.169 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85390.log 2024-04-25T11:43:32.169 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37915.log.gz 2024-04-25T11:43:32.169 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T11:43:32.170 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91971.log: gzip -5 89.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.91971.log.gz 2024-04-25T11:43:32.170 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.76475.log 2024-04-25T11:43:32.171 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61396.log 2024-04-25T11:43:32.172 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.85390.log.gz 2024-04-25T11:43:32.173 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76475.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76475.log.gz 2024-04-25T11:43:32.173 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44825.log 2024-04-25T11:43:32.174 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56632.log 88.7% 2024-04-25T11:43:32.174 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61396.log.gz 2024-04-25T11:43:32.175 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59194.log 2024-04-25T11:43:32.175 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44825.log.gz 2024-04-25T11:43:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56632.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56632.log.gz 2024-04-25T11:43:32.176 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52611.log 2024-04-25T11:43:32.178 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-04-25T11:43:32.178 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59194.log.gz 2024-04-25T11:43:32.179 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58572.log 2024-04-25T11:43:32.179 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52611.log.gz 2024-04-25T11:43:32.180 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70111.log 2024-04-25T11:43:32.180 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-04-25T11:43:32.181 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64481.log 2024-04-25T11:43:32.181 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58572.log.gz 2024-04-25T11:43:32.182 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63523.log 2024-04-25T11:43:32.183 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70111.log.gz 2024-04-25T11:43:32.183 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55816.log 2024-04-25T11:43:32.183 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64481.log.gz 2024-04-25T11:43:32.185 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91929.log 2024-04-25T11:43:32.185 INFO:teuthology.orchestra.run.smithi019.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.63523.log.gz 2024-04-25T11:43:32.186 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55816.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.55816.log.gz --verbose 2024-04-25T11:43:32.186 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.83814.log 2024-04-25T11:43:32.187 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76270.log 2024-04-25T11:43:32.187 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91929.log.gz 2024-04-25T11:43:32.188 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83814.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83814.log.gz 2024-04-25T11:43:32.188 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44488.log 2024-04-25T11:43:32.189 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79399.log 2024-04-25T11:43:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91012.log 2024-04-25T11:43:32.191 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.76270.log.gz 2024-04-25T11:43:32.191 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44488.log.gz 2024-04-25T11:43:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87880.log 2024-04-25T11:43:32.192 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79399.log.gz 2024-04-25T11:43:32.193 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91012.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.91719.log -- replaced with /var/log/ceph/ceph-client.admin.91012.log.gz 2024-04-25T11:43:32.193 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:32.194 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87880.log.gz 2024-04-25T11:43:32.195 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84756.log 2024-04-25T11:43:32.196 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69792.log 2024-04-25T11:43:32.196 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91719.log.gz 2024-04-25T11:43:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91845.log 2024-04-25T11:43:32.197 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84756.log.gz 2024-04-25T11:43:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76924.log 2024-04-25T11:43:32.198 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69792.log.gz 2024-04-25T11:43:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73588.log 2024-04-25T11:43:32.200 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91845.log.gz 2024-04-25T11:43:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74811.log 2024-04-25T11:43:32.200 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.76924.log.gz 2024-04-25T11:43:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73588.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73588.log.gz 2024-04-25T11:43:32.202 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59338.log 2024-04-25T11:43:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-04-25T11:43:32.203 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74811.log.gz 2024-04-25T11:43:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59338.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59338.log.gz 2024-04-25T11:43:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91382.log 2024-04-25T11:43:32.206 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43637.log 2024-04-25T11:43:32.206 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-04-25T11:43:32.207 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65409.log 2024-04-25T11:43:32.207 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91382.log.gz 2024-04-25T11:43:32.208 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86443.log 2024-04-25T11:43:32.208 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43637.log.gz 2024-04-25T11:43:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44998.log 2024-04-25T11:43:32.210 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65409.log.gz 2024-04-25T11:43:32.210 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86569.log 2024-04-25T11:43:32.211 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86443.log.gz 2024-04-25T11:43:32.212 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74538.log 2024-04-25T11:43:32.212 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44998.log.gz 2024-04-25T11:43:32.213 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79287.log 2024-04-25T11:43:32.213 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86569.log.gz 2024-04-25T11:43:32.215 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74538.log: gzip -5 --verbose 89.2% -- -- replaced with /var/log/ceph/ceph-client.admin.74538.log.gz 2024-04-25T11:43:32.215 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.47907.log 2024-04-25T11:43:32.216 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39114.log 2024-04-25T11:43:32.216 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79287.log.gz 2024-04-25T11:43:32.217 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91054.log 2024-04-25T11:43:32.217 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-04-25T11:43:32.218 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40128.log 2024-04-25T11:43:32.218 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39114.log.gz 2024-04-25T11:43:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65946.log 2024-04-25T11:43:32.219 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91054.log.gz 2024-04-25T11:43:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40128.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40128.log.gz 2024-04-25T11:43:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:43:32.222 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55336.log 2024-04-25T11:43:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- 92.4% /var/log/ceph/ceph-client.admin.76205.log -- replaced with /var/log/ceph/ceph-client.admin.65946.log.gz 2024-04-25T11:43:32.223 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82787.log 2024-04-25T11:43:32.224 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55336.log.gz 2024-04-25T11:43:32.225 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62559.log 2024-04-25T11:43:32.226 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76205.log.gz 2024-04-25T11:43:32.226 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40380.log 2024-04-25T11:43:32.227 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.82787.log.gz 2024-04-25T11:43:32.228 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70396.log 2024-04-25T11:43:32.228 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62559.log.gz 87.3% 2024-04-25T11:43:32.228 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:43:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40380.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40380.log.gz 2024-04-25T11:43:32.229 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39803.log 2024-04-25T11:43:32.230 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53449.log 2024-04-25T11:43:32.230 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70396.log.gz 2024-04-25T11:43:32.231 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86695.log 2024-04-25T11:43:32.231 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39803.log.gz 2024-04-25T11:43:32.232 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60785.log 2024-04-25T11:43:32.232 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53449.log.gz 2024-04-25T11:43:32.233 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83272.log 2024-04-25T11:43:32.234 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86695.log.gz 2024-04-25T11:43:32.234 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58488.log 2024-04-25T11:43:32.235 INFO:teuthology.orchestra.run.smithi169.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T11:43:32.236 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60785.log.gz 2024-04-25T11:43:32.236 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57524.log 2024-04-25T11:43:32.236 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83272.log.gz 2024-04-25T11:43:32.237 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64355.log 2024-04-25T11:43:32.237 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58488.log.gz 2024-04-25T11:43:32.238 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87246.log 2024-04-25T11:43:32.238 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57524.log.gz 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89103.log 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.64355.log.gz 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91593.log 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87246.log.gz 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77388.log 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.89103.log.gz 2024-04-25T11:43:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91593.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91593.log.gz 2024-04-25T11:43:32.243 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75587.log 2024-04-25T11:43:32.244 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77388.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.77388.log.gz 2024-04-25T11:43:32.244 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.90471.log 2024-04-25T11:43:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75587.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75587.log.gz 2024-04-25T11:43:32.245 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76593.log 2024-04-25T11:43:32.246 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51507.log 89.5% 2024-04-25T11:43:32.246 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90471.log.gz 2024-04-25T11:43:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76593.log.gz 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42512.log 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51507.log.gz 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50851.log 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42512.log.gz 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50851.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50851.log.gz 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74454.log 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48515.log 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-04-25T11:43:32.254 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50363.log 2024-04-25T11:43:32.254 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74454.log.gz 2024-04-25T11:43:32.255 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43805.log 2024-04-25T11:43:32.256 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48515.log.gz 2024-04-25T11:43:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62001.log 2024-04-25T11:43:32.256 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50363.log.gz 2024-04-25T11:43:32.257 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70732.log 2024-04-25T11:43:32.257 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43805.log.gz 2024-04-25T11:43:32.258 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68375.log 2024-04-25T11:43:32.259 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70732.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62001.log.gz 2024-04-25T11:43:32.259 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40590.log 2024-04-25T11:43:32.260 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70732.log.gz 2024-04-25T11:43:32.261 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55106.log 2024-04-25T11:43:32.261 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68375.log.gz 2024-04-25T11:43:32.262 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71315.log 2024-04-25T11:43:32.262 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40590.log.gz 2024-04-25T11:43:32.263 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52275.log 2024-04-25T11:43:32.263 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55106.log.gz 2024-04-25T11:43:32.264 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59380.log 2024-04-25T11:43:32.264 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71315.log.gz 2024-04-25T11:43:32.265 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52275.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T11:43:32.265 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52275.log.gz 2024-04-25T11:43:32.266 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81987.log 2024-04-25T11:43:32.266 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59380.log.gz 2024-04-25T11:43:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-04-25T11:43:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.81987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81987.log.gz 2024-04-25T11:43:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76769.log 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43721.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80883.log 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71748.log 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76769.log.gz 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83772.log 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80883.log.gz 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87544.log 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71748.log.gz 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57566.log 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83772.log.gz 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45442.log 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87544.log.gz 2024-04-25T11:43:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69114.log 2024-04-25T11:43:32.277 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57566.log.gz 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73428.log 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45442.log.gz 2024-04-25T11:43:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69114.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69114.log.gz 2024-04-25T11:43:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46376.log 2024-04-25T11:43:32.280 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53617.log 2024-04-25T11:43:32.281 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73428.log.gz 2024-04-25T11:43:32.281 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63671.log 2024-04-25T11:43:32.281 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46376.log.gz 2024-04-25T11:43:32.282 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91333.log 2024-04-25T11:43:32.283 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63671.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53617.log.gz 2024-04-25T11:43:32.283 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80118.log 2024-04-25T11:43:32.283 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63671.log.gz 2024-04-25T11:43:32.284 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73081.log 2024-04-25T11:43:32.284 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91333.log.gz 2024-04-25T11:43:32.285 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75919.log 2024-04-25T11:43:32.286 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80118.log.gz 2024-04-25T11:43:32.287 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73081.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73081.log.gz 2024-04-25T11:43:32.287 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83555.log 2024-04-25T11:43:32.287 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82590.log 2024-04-25T11:43:32.288 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.75919.log.gz 2024-04-25T11:43:32.289 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88601.log 2024-04-25T11:43:32.289 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83555.log.gz 2024-04-25T11:43:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42844.log 2024-04-25T11:43:32.290 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88601.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.82590.log.gz 2024-04-25T11:43:32.291 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.88601.log.gz -5 2024-04-25T11:43:32.291 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38850.log 2024-04-25T11:43:32.292 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68859.log 2024-04-25T11:43:32.292 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42844.log.gz 2024-04-25T11:43:32.293 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65247.log 2024-04-25T11:43:32.293 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38850.log.gz 2024-04-25T11:43:32.294 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82125.log 2024-04-25T11:43:32.294 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68859.log.gz 2024-04-25T11:43:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65247.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65247.log.gz 2024-04-25T11:43:32.295 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71968.log 2024-04-25T11:43:32.296 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67111.log 2024-04-25T11:43:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71968.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.82125.log.gz 2024-04-25T11:43:32.297 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44057.log 2024-04-25T11:43:32.297 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71968.log.gz 2024-04-25T11:43:32.298 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83229.log 2024-04-25T11:43:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44057.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44057.log.gz 2024-04-25T11:43:32.299 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68691.log 2024-04-25T11:43:32.300 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.67111.log.gz 2024-04-25T11:43:32.301 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47250.log 2024-04-25T11:43:32.301 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83229.log.gz 2024-04-25T11:43:32.301 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68033.log 2024-04-25T11:43:32.301 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68691.log.gz 2024-04-25T11:43:32.302 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47124.log 2024-04-25T11:43:32.303 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47250.log.gz 2024-04-25T11:43:32.303 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68075.log 2024-04-25T11:43:32.304 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68033.log.gz 2024-04-25T11:43:32.304 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47124.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47124.log.gz -5 2024-04-25T11:43:32.305 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67745.log 2024-04-25T11:43:32.305 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87712.log 2024-04-25T11:43:32.307 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82745.log 2024-04-25T11:43:32.307 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67745.log.gz 2024-04-25T11:43:32.307 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.68075.log.gz 2024-04-25T11:43:32.308 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90639.log 2024-04-25T11:43:32.308 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87712.log.gz 2024-04-25T11:43:32.309 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82745.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82745.log.gz 2024-04-25T11:43:32.309 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89478.log 2024-04-25T11:43:32.310 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38640.log 2024-04-25T11:43:32.310 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90639.log.gz 2024-04-25T11:43:32.311 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48997.log 2024-04-25T11:43:32.311 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89478.log.gz 2024-04-25T11:43:32.312 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65493.log 2024-04-25T11:43:32.312 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38640.log.gz 2024-04-25T11:43:32.313 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70574.log 2024-04-25T11:43:32.314 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65493.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48997.log.gz 2024-04-25T11:43:32.314 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82307.log 2024-04-25T11:43:32.315 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70574.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.65493.log.gz 2024-04-25T11:43:32.316 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70574.log.gz 2024-04-25T11:43:32.316 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39806.log 2024-04-25T11:43:32.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65983.log 2024-04-25T11:43:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39806.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82307.log.gz 2024-04-25T11:43:32.318 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39806.log.gz 2024-04-25T11:43:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87204.log 2024-04-25T11:43:32.319 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65983.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.65983.log.gz 2024-04-25T11:43:32.320 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39131.log 2024-04-25T11:43:32.321 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64207.log 2024-04-25T11:43:32.321 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87204.log.gz 2024-04-25T11:43:32.322 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39131.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.39131.log.gz 2024-04-25T11:43:32.322 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.44953.log 2024-04-25T11:43:32.323 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58947.log 2024-04-25T11:43:32.324 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44953.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.64207.log.gz 2024-04-25T11:43:32.324 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44953.log.gz 2024-04-25T11:43:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52706.log 2024-04-25T11:43:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72463.log 2024-04-25T11:43:32.326 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58947.log.gz 2024-04-25T11:43:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52706.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52706.log.gz 2024-04-25T11:43:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82864.log 2024-04-25T11:43:32.329 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49610.log 2024-04-25T11:43:32.329 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72463.log.gz 2024-04-25T11:43:32.330 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76027.log 2024-04-25T11:43:32.330 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82864.log.gz 2024-04-25T11:43:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49610.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49610.log.gz 2024-04-25T11:43:32.331 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75859.log 2024-04-25T11:43:32.332 INFO:teuthology.orchestra.run.smithi169.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T11:43:32.333 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91288.log 2024-04-25T11:43:32.333 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76027.log.gz 2024-04-25T11:43:32.334 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75859.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75859.log.gz 2024-04-25T11:43:32.334 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43973.log 2024-04-25T11:43:32.335 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72202.log 2024-04-25T11:43:32.335 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91288.log.gz 2024-04-25T11:43:32.336 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43973.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:43:32.336 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43973.log.gz 2024-04-25T11:43:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81046.log 2024-04-25T11:43:32.338 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72202.log.gz 2024-04-25T11:43:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77490.log 2024-04-25T11:43:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78455.log 2024-04-25T11:43:32.341 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77490.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.77490.log.gz 2024-04-25T11:43:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48473.log 2024-04-25T11:43:32.342 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.81046.log.gz 2024-04-25T11:43:32.343 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56810.log 2024-04-25T11:43:32.344 INFO:teuthology.orchestra.run.smithi019.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.78455.log.gz 2024-04-25T11:43:32.344 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48473.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:43:32.344 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48473.log.gz 2024-04-25T11:43:32.344 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60869.log 2024-04-25T11:43:32.345 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78238.log 2024-04-25T11:43:32.347 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56810.log.gz 2024-04-25T11:43:32.347 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90167.log 2024-04-25T11:43:32.347 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60869.log.gz 2024-04-25T11:43:32.348 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87796.log 2024-04-25T11:43:32.348 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90167.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78238.log.gz 2024-04-25T11:43:32.349 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90167.log.gz 2024-04-25T11:43:32.349 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50648.log 2024-04-25T11:43:32.351 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41966.log 2024-04-25T11:43:32.351 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87796.log.gz 2024-04-25T11:43:32.351 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69386.log 2024-04-25T11:43:32.352 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50648.log.gz 2024-04-25T11:43:32.353 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68112.log 2024-04-25T11:43:32.353 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41966.log.gz 2024-04-25T11:43:32.354 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89405.log 2024-04-25T11:43:32.354 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69386.log.gz 2024-04-25T11:43:32.355 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63985.log 2024-04-25T11:43:32.355 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.68112.log.gz 2024-04-25T11:43:32.356 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88949.log 2024-04-25T11:43:32.357 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89405.log.gz 2024-04-25T11:43:32.357 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89964.log 2024-04-25T11:43:32.357 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63985.log.gz 2024-04-25T11:43:32.359 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38934.log 2024-04-25T11:43:32.359 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88949.log.gz 2024-04-25T11:43:32.360 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86275.log 2024-04-25T11:43:32.360 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89964.log.gz 2024-04-25T11:43:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38934.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38934.log.gz 2024-04-25T11:43:32.361 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76518.log 2024-04-25T11:43:32.362 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70456.log 2024-04-25T11:43:32.363 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86275.log.gz 2024-04-25T11:43:32.364 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72978.log 2024-04-25T11:43:32.364 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76518.log.gz 2024-04-25T11:43:32.365 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70456.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70456.log.gz 2024-04-25T11:43:32.365 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57661.log 2024-04-25T11:43:32.366 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47478.log 2024-04-25T11:43:32.366 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72978.log.gz 2024-04-25T11:43:32.367 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57661.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57661.log.gz 2024-04-25T11:43:32.367 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61108.log 2024-04-25T11:43:32.369 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60533.log 2024-04-25T11:43:32.370 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47478.log.gz 2024-04-25T11:43:32.370 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88643.log 2024-04-25T11:43:32.370 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61108.log.gz 2024-04-25T11:43:32.371 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79619.log 2024-04-25T11:43:32.371 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60533.log.gz 2024-04-25T11:43:32.373 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46322.log 2024-04-25T11:43:32.373 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88643.log.gz 2024-04-25T11:43:32.374 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60155.log 2024-04-25T11:43:32.374 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.79619.log.gz 2024-04-25T11:43:32.375 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46322.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46322.log.gz 2024-04-25T11:43:32.375 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64244.log 2024-04-25T11:43:32.377 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38173.log 2024-04-25T11:43:32.377 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60155.log.gz 2024-04-25T11:43:32.378 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86779.log 2024-04-25T11:43:32.378 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64244.log.gz 2024-04-25T11:43:32.379 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39454.log 2024-04-25T11:43:32.379 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38173.log.gz 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55148.log 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi019.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86779.log.gz 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48305.log 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39454.log.gz 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78112.log 2024-04-25T11:43:32.383 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55148.log.gz 2024-04-25T11:43:32.383 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64546.log 2024-04-25T11:43:32.384 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48305.log.gz 2024-04-25T11:43:32.384 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85288.log 2024-04-25T11:43:32.385 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78112.log.gz 2024-04-25T11:43:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83187.log 2024-04-25T11:43:32.387 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77546.log 2024-04-25T11:43:32.387 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.64546.log.gz 2024-04-25T11:43:32.387 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85288.log.gz 2024-04-25T11:43:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60239.log 2024-04-25T11:43:32.388 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83187.log.gz 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54069.log 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77546.log.gz 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60239.log.gz 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47166.log 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70785.log 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54069.log.gz 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89856.log 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47166.log.gz 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58530.log 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70785.log.gz 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89856.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89856.log.gz 2024-04-25T11:43:32.396 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81733.log 2024-04-25T11:43:32.396 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69464.log 2024-04-25T11:43:32.397 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58530.log.gz 2024-04-25T11:43:32.397 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67363.log 2024-04-25T11:43:32.399 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.81733.log.gz 2024-04-25T11:43:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79129.log 2024-04-25T11:43:32.399 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69464.log.gz 2024-04-25T11:43:32.400 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61665.log 2024-04-25T11:43:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79129.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67363.log.gz 2024-04-25T11:43:32.401 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79129.log.gz 2024-04-25T11:43:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66479.log 2024-04-25T11:43:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61665.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61665.log.gz 2024-04-25T11:43:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62856.log 2024-04-25T11:43:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89675.log 2024-04-25T11:43:32.404 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66479.log.gz 2024-04-25T11:43:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73258.log 2024-04-25T11:43:32.404 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62856.log.gz 2024-04-25T11:43:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79517.log 2024-04-25T11:43:32.406 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.89675.log.gz 2024-04-25T11:43:32.407 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68168.log 2024-04-25T11:43:32.407 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.73258.log.gz 2024-04-25T11:43:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76307.log 2024-04-25T11:43:32.408 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79517.log.gz 2024-04-25T11:43:32.409 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80961.log 2024-04-25T11:43:32.409 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68168.log.gz 2024-04-25T11:43:32.410 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71561.log 2024-04-25T11:43:32.410 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76307.log.gz 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69572.log 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80961.log.gz 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75251.log 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71561.log.gz 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53533.log 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69572.log.gz 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39805.log 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75251.log.gz 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48599.log 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53533.log.gz 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47809.log 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39805.log.gz 2024-04-25T11:43:32.418 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83931.log 2024-04-25T11:43:32.418 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48599.log.gz 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79727.log 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47809.log.gz 2024-04-25T11:43:32.420 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39470.log 2024-04-25T11:43:32.420 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83931.log.gz 2024-04-25T11:43:32.421 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43553.log 2024-04-25T11:43:32.421 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79727.log.gz 2024-04-25T11:43:32.422 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60365.log 2024-04-25T11:43:32.423 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39470.log.gz 2024-04-25T11:43:32.423 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51878.log 2024-04-25T11:43:32.424 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43553.log.gz 2024-04-25T11:43:32.424 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90387.log 2024-04-25T11:43:32.425 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60365.log.gz 2024-04-25T11:43:32.426 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86947.log 2024-04-25T11:43:32.426 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51878.log.gz 2024-04-25T11:43:32.427 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39750.log 2024-04-25T11:43:32.427 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90387.log.gz 2024-04-25T11:43:32.428 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86947.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-osd.3.log -- replaced with /var/log/ceph/ceph-client.admin.86947.log.gz 2024-04-25T11:43:32.428 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:32.429 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44099.log 2024-04-25T11:43:32.429 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39750.log.gz 2024-04-25T11:43:32.430 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2024-04-25T11:43:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44099.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44099.log.gz 2024-04-25T11:43:32.431 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59007.log 2024-04-25T11:43:32.432 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87586.log 2024-04-25T11:43:32.433 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2024-04-25T11:43:32.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45400.log 2024-04-25T11:43:32.434 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59007.log.gz 2024-04-25T11:43:32.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-04-25T11:43:32.435 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87586.log.gz 2024-04-25T11:43:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45400.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45400.log.gz 2024-04-25T11:43:32.436 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56345.log 2024-04-25T11:43:32.438 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41822.log 2024-04-25T11:43:32.438 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-04-25T11:43:32.439 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75639.log 2024-04-25T11:43:32.439 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56345.log.gz 2024-04-25T11:43:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41822.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41822.log.gz 2024-04-25T11:43:32.440 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89898.log 2024-04-25T11:43:32.441 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75639.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75639.log.gz 2024-04-25T11:43:32.441 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83889.log 2024-04-25T11:43:32.442 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55331.log 2024-04-25T11:43:32.443 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89898.log.gz 2024-04-25T11:43:32.443 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83889.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83889.log.gz 2024-04-25T11:43:32.444 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74037.log 2024-04-25T11:43:32.445 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53575.log 2024-04-25T11:43:32.445 INFO:teuthology.orchestra.run.smithi019.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.55331.log.gz 2024-04-25T11:43:32.446 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74037.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74037.log.gz 2024-04-25T11:43:32.446 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66657.log 2024-04-25T11:43:32.447 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85498.log 2024-04-25T11:43:32.447 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53575.log.gz 2024-04-25T11:43:32.448 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66657.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66657.log.gz 2024-04-25T11:43:32.448 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66371.log 2024-04-25T11:43:32.449 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72244.log 2024-04-25T11:43:32.449 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85498.log.gz 2024-04-25T11:43:32.450 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63042.log 2024-04-25T11:43:32.451 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66371.log.gz 2024-04-25T11:43:32.451 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84100.log 2024-04-25T11:43:32.451 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72244.log.gz 2024-04-25T11:43:32.452 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49316.log 2024-04-25T11:43:32.453 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63042.log.gz 2024-04-25T11:43:32.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47732.log 2024-04-25T11:43:32.454 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84100.log.gz 2024-04-25T11:43:32.455 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77157.log 2024-04-25T11:43:32.455 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49316.log.gz 2024-04-25T11:43:32.455 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.47732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47732.log.gz 2024-04-25T11:43:32.456 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62819.log 2024-04-25T11:43:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77157.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77157.log.gz 2024-04-25T11:43:32.457 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63412.log 2024-04-25T11:43:32.458 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46436.log 2024-04-25T11:43:32.459 INFO:teuthology.orchestra.run.smithi019.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.62819.log.gz 2024-04-25T11:43:32.459 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39707.log 2024-04-25T11:43:32.459 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.63412.log.gz 2024-04-25T11:43:32.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90597.log 2024-04-25T11:43:32.460 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46436.log.gz 2024-04-25T11:43:32.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63079.log 2024-04-25T11:43:32.462 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39707.log.gz 2024-04-25T11:43:32.463 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85718.log 2024-04-25T11:43:32.463 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90597.log.gz 2024-04-25T11:43:32.464 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79970.log 2024-04-25T11:43:32.464 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63079.log.gz 2024-04-25T11:43:32.465 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86611.log 2024-04-25T11:43:32.465 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85718.log.gz 2024-04-25T11:43:32.466 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90867.log 2024-04-25T11:43:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86611.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.79970.log.gz 2024-04-25T11:43:32.467 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose 88.0% -- /var/log/ceph/ceph-client.admin.85558.log -- replaced with /var/log/ceph/ceph-client.admin.86611.log.gz 2024-04-25T11:43:32.467 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:32.468 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49654.log 2024-04-25T11:43:32.469 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90867.log.gz 2024-04-25T11:43:32.469 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56978.log 2024-04-25T11:43:32.469 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85558.log.gz 2024-04-25T11:43:32.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49654.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49654.log.gz 2024-04-25T11:43:32.471 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58404.log 2024-04-25T11:43:32.472 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55858.log 2024-04-25T11:43:32.472 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56978.log.gz 2024-04-25T11:43:32.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58404.log: gzip -5 88.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.58404.log.gz -- 2024-04-25T11:43:32.473 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.61270.log 2024-04-25T11:43:32.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81192.log 2024-04-25T11:43:32.474 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55858.log.gz 2024-04-25T11:43:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61270.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61270.log.gz 2024-04-25T11:43:32.475 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48347.log 2024-04-25T11:43:32.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41562.log 2024-04-25T11:43:32.476 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81192.log.gz 2024-04-25T11:43:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48347.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.48347.log.gz --verbose 2024-04-25T11:43:32.477 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.39461.log 2024-04-25T11:43:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.2.41562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.41562.log.gz 2024-04-25T11:43:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38384.log 2024-04-25T11:43:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39461.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39461.log.gz 2024-04-25T11:43:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49747.log 2024-04-25T11:43:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39132.log 2024-04-25T11:43:32.480 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38384.log.gz 2024-04-25T11:43:32.481 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49747.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49747.log.gz 2024-04-25T11:43:32.482 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42344.log 2024-04-25T11:43:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73757.log 2024-04-25T11:43:32.483 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39132.log.gz 2024-04-25T11:43:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42344.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42344.log.gz 2024-04-25T11:43:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58097.log 2024-04-25T11:43:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69675.log 2024-04-25T11:43:32.485 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73757.log.gz 2024-04-25T11:43:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58097.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58097.log.gz 2024-04-25T11:43:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71851.log 2024-04-25T11:43:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86317.log 2024-04-25T11:43:32.487 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69675.log.gz 2024-04-25T11:43:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71851.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71851.log.gz 2024-04-25T11:43:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53407.log 2024-04-25T11:43:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2024-04-25T11:43:32.489 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86317.log.gz 2024-04-25T11:43:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53407.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53407.log.gz 2024-04-25T11:43:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92137.log 2024-04-25T11:43:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92097.log 2024-04-25T11:43:32.491 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2024-04-25T11:43:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.92137.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92137.log.gz 2024-04-25T11:43:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52401.log 2024-04-25T11:43:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.92097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73198.log 2024-04-25T11:43:32.493 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92097.log.gz 2024-04-25T11:43:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52401.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52401.log.gz 2024-04-25T11:43:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84612.log 2024-04-25T11:43:32.495 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73198.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T11:43:32.495 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73198.log.gz 2024-04-25T11:43:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84612.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84612.log.gz 2024-04-25T11:43:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87122.log 2024-04-25T11:43:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51441.log 2024-04-25T11:43:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45738.log 2024-04-25T11:43:32.499 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87122.log.gz 2024-04-25T11:43:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33749.log 2024-04-25T11:43:32.500 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51441.log.gz 2024-04-25T11:43:32.501 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42554.log 2024-04-25T11:43:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.45738.log: /var/log/ceph/ceph-client.admin.33749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45738.log.gz 2024-04-25T11:43:32.502 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33749.log.gz 2024-04-25T11:43:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55592.log 2024-04-25T11:43:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42554.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42554.log.gz 2024-04-25T11:43:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87988.log 2024-04-25T11:43:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42596.log 2024-04-25T11:43:32.505 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87988.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55592.log.gz 2024-04-25T11:43:32.506 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87988.log.gz 2024-04-25T11:43:32.506 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73471.log 2024-04-25T11:43:32.507 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71519.log 2024-04-25T11:43:32.507 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42596.log.gz 2024-04-25T11:43:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73471.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73471.log.gz 2024-04-25T11:43:32.509 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49919.log 2024-04-25T11:43:32.510 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84570.log 2024-04-25T11:43:32.510 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49919.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71519.log.gz 2024-04-25T11:43:32.511 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86191.log 2024-04-25T11:43:32.511 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49919.log.gz 2024-04-25T11:43:32.512 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84570.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84570.log.gz 2024-04-25T11:43:32.512 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72870.log 2024-04-25T11:43:32.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57356.log 2024-04-25T11:43:32.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72870.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86191.log.gz 2024-04-25T11:43:32.515 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66699.log 2024-04-25T11:43:32.515 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.72870.log.gz 2024-04-25T11:43:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44327.log 2024-04-25T11:43:32.516 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57356.log.gz 2024-04-25T11:43:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67148.log 2024-04-25T11:43:32.518 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66699.log.gz 2024-04-25T11:43:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89628.log 2024-04-25T11:43:32.518 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44327.log.gz 2024-04-25T11:43:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69852.log 2024-04-25T11:43:32.520 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67148.log.gz 2024-04-25T11:43:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62255.log 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89628.log: /var/log/ceph/ceph-client.admin.69852.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69852.log.gz 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52569.log 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.89628.log.gz 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.62255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86401.log 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62255.log.gz 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52569.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T11:43:32.525 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52569.log.gz 2024-04-25T11:43:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86401.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86401.log.gz 2024-04-25T11:43:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T11:43:32.527 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74616.log 2024-04-25T11:43:32.528 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79947.log 2024-04-25T11:43:32.529 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74755.log 2024-04-25T11:43:32.530 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74616.log.gz 2024-04-25T11:43:32.531 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57865.log 2024-04-25T11:43:32.531 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79947.log.gz 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74755.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74755.log.gz 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57865.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57865.log.gz 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40338.log 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65881.log 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55900.log 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40338.log.gz 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65881.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65881.log.gz 2024-04-25T11:43:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55900.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.55900.log.gz 2024-04-25T11:43:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60197.log 2024-04-25T11:43:32.541 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T11:43:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69344.log 2024-04-25T11:43:32.543 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40086.log 2024-04-25T11:43:32.543 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60197.log: /var/log/ceph/ceph-client.admin.69344.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60197.log.gz 2024-04-25T11:43:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69344.log.gz -5 2024-04-25T11:43:32.544 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.91803.log 2024-04-25T11:43:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52868.log 2024-04-25T11:43:32.545 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40086.log.gz 2024-04-25T11:43:32.546 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43391.log 2024-04-25T11:43:32.546 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91803.log.gz 2024-04-25T11:43:32.548 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56894.log 2024-04-25T11:43:32.548 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52868.log.gz 2024-04-25T11:43:32.549 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43391.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43391.log.gz 2024-04-25T11:43:32.549 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64281.log 2024-04-25T11:43:32.550 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56894.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56894.log.gz 2024-04-25T11:43:32.550 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79194.log 2024-04-25T11:43:32.551 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91245.log 2024-04-25T11:43:32.552 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64281.log: /var/log/ceph/ceph-client.admin.79194.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.64281.log.gz 2024-04-25T11:43:32.552 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42971.log 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91245.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.79194.log.gz 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42886.log 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91245.log.gz 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74225.log 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42971.log.gz 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42886.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42886.log.gz 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54844.log 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.74225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63338.log 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74225.log.gz 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59422.log 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54844.log.gz 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63338.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63338.log.gz 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72426.log 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59422.log.gz 2024-04-25T11:43:32.562 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92013.log 2024-04-25T11:43:32.564 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2024-04-25T11:43:32.565 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.92013.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92013.log.gz 2024-04-25T11:43:32.565 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.72426.log.gz 2024-04-25T11:43:32.566 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74412.log 2024-04-25T11:43:32.566 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46225.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2024-04-25T11:43:32.567 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77041.log 2024-04-25T11:43:32.569 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44141.log 2024-04-25T11:43:32.569 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74412.log.gz 2024-04-25T11:43:32.570 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80290.log 2024-04-25T11:43:32.570 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77041.log.gz 2024-04-25T11:43:32.571 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68775.log 2024-04-25T11:43:32.571 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44141.log.gz 2024-04-25T11:43:32.573 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75083.log 88.4% 2024-04-25T11:43:32.573 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80290.log.gz 2024-04-25T11:43:32.574 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75742.log 2024-04-25T11:43:32.574 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68775.log.gz 2024-04-25T11:43:32.576 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75083.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75083.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66194.log 2024-04-25T11:43:32.576 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75742.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75742.log.gz 2024-04-25T11:43:32.578 INFO:teuthology.orchestra.run.smithi169.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T11:43:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66194.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66194.log.gz 2024-04-25T11:43:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40422.log 2024-04-25T11:43:32.580 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54111.log 2024-04-25T11:43:32.582 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39463.log 2024-04-25T11:43:32.582 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40422.log.gz 2024-04-25T11:43:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54111.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54111.log.gz 2024-04-25T11:43:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39463.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39463.log.gz 2024-04-25T11:43:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52046.log 2024-04-25T11:43:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90820.log 2024-04-25T11:43:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82391.log 2024-04-25T11:43:32.588 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52046.log.gz 2024-04-25T11:43:32.589 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90820.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90820.log.gz 2024-04-25T11:43:32.590 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82391.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82391.log.gz 2024-04-25T11:43:32.590 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56936.log 2024-04-25T11:43:32.592 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83664.log 2024-04-25T11:43:32.594 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81458.log 2024-04-25T11:43:32.594 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56936.log.gz 2024-04-25T11:43:32.595 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83664.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83664.log.gz 2024-04-25T11:43:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81458.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81458.log.gz 2024-04-25T11:43:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40632.log 2024-04-25T11:43:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74863.log 2024-04-25T11:43:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75143.log 2024-04-25T11:43:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74863.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40632.log.gz 2024-04-25T11:43:32.600 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74863.log.gz 2024-04-25T11:43:32.602 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75143.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.75143.log.gz 2024-04-25T11:43:32.602 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39130.log 2024-04-25T11:43:32.604 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27450.log 2024-04-25T11:43:32.605 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86863.log 2024-04-25T11:43:32.605 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39130.log.gz 2024-04-25T11:43:32.606 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.tmp-client.admin.27450.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27450.log.gz 2024-04-25T11:43:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86863.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86863.log.gz 2024-04-25T11:43:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85952.log 2024-04-25T11:43:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77270.log 2024-04-25T11:43:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57146.log 2024-04-25T11:43:32.611 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85952.log.gz 2024-04-25T11:43:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77270.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77270.log.gz 2024-04-25T11:43:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57146.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57146.log.gz 2024-04-25T11:43:32.614 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66427.log 2024-04-25T11:43:32.615 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38258.log 2024-04-25T11:43:32.617 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64318.log 2024-04-25T11:43:32.617 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66427.log.gz 2024-04-25T11:43:32.618 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38258.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38258.log.gz 2024-04-25T11:43:32.620 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64318.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.64318.log.gz 2024-04-25T11:43:32.620 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78154.log 2024-04-25T11:43:32.621 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45550.log 2024-04-25T11:43:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43847.log 2024-04-25T11:43:32.623 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78154.log.gz 2024-04-25T11:43:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45550.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T11:43:32.624 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45550.log.gz 2024-04-25T11:43:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60617.log 2024-04-25T11:43:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43847.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43847.log.gz 2024-04-25T11:43:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61917.log 2024-04-25T11:43:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81481.log 2024-04-25T11:43:32.627 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60617.log.gz 2024-04-25T11:43:32.629 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68563.log 2024-04-25T11:43:32.629 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61917.log.gz 2024-04-25T11:43:32.630 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85064.log 2024-04-25T11:43:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.68563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68563.log.gz 2024-04-25T11:43:32.631 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87416.log 2024-04-25T11:43:32.631 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.81481.log.gz 2024-04-25T11:43:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71926.log 2024-04-25T11:43:32.632 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85064.log.gz 2024-04-25T11:43:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42638.log 2024-04-25T11:43:32.634 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87416.log.gz 2024-04-25T11:43:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63153.log 2024-04-25T11:43:32.635 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71926.log.gz 2024-04-25T11:43:32.636 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46043.log 2024-04-25T11:43:32.637 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42638.log.gz 2024-04-25T11:43:32.638 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60575.log 2024-04-25T11:43:32.638 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63153.log.gz 2024-04-25T11:43:32.638 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52527.log 2024-04-25T11:43:32.639 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46043.log.gz 2024-04-25T11:43:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86485.log 2024-04-25T11:43:32.641 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60575.log.gz 2024-04-25T11:43:32.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56058.log 2024-04-25T11:43:32.642 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52527.log.gz 2024-04-25T11:43:32.643 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78783.log 2024-04-25T11:43:32.643 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86485.log.gz 2024-04-25T11:43:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77210.log 2024-04-25T11:43:32.645 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56058.log.gz 2024-04-25T11:43:32.645 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2024-04-25T11:43:32.645 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78783.log.gz 2024-04-25T11:43:32.646 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67088.log 2024-04-25T11:43:32.647 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77210.log.gz 2024-04-25T11:43:32.648 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44615.log 2024-04-25T11:43:32.648 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2024-04-25T11:43:32.650 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66091.log 2024-04-25T11:43:32.650 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67088.log.gz 2024-04-25T11:43:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64780.log 2024-04-25T11:43:32.651 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44615.log.gz 2024-04-25T11:43:32.652 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87628.log 2024-04-25T11:43:32.652 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66091.log.gz 2024-04-25T11:43:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46248.log 2024-04-25T11:43:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87628.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64780.log.gz 2024-04-25T11:43:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85213.log 2024-04-25T11:43:32.655 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87628.log.gz 2024-04-25T11:43:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48557.log 2024-04-25T11:43:32.656 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.46248.log.gz 2024-04-25T11:43:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72631.log 2024-04-25T11:43:32.657 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85213.log.gz 2024-04-25T11:43:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88906.log 2024-04-25T11:43:32.659 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48557.log.gz 2024-04-25T11:43:32.660 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79830.log 2024-04-25T11:43:32.660 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72631.log.gz 2024-04-25T11:43:32.661 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84876.log 2024-04-25T11:43:32.661 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88906.log.gz 2024-04-25T11:43:32.663 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2024-04-25T11:43:32.663 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79830.log.gz 2024-04-25T11:43:32.664 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52088.log 2024-04-25T11:43:32.664 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84876.log.gz 2024-04-25T11:43:32.665 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78843.log 2024-04-25T11:43:32.665 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2%/var/log/ceph/ceph-client.admin.52088.log: -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz 2024-04-25T11:43:32.666 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61833.log 2024-04-25T11:43:32.666 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52088.log.gz 2024-04-25T11:43:32.668 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84146.log 2024-04-25T11:43:32.668 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.78843.log.gz 2024-04-25T11:43:32.669 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44530.log 2024-04-25T11:43:32.669 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61833.log.gz 2024-04-25T11:43:32.670 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73368.log 2024-04-25T11:43:32.670 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84146.log.gz 2024-04-25T11:43:32.671 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57314.log 2024-04-25T11:43:32.672 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44530.log.gz 2024-04-25T11:43:32.672 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68979.log 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73368.log.gz 2024-04-25T11:43:32.674 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63190.log 2024-04-25T11:43:32.674 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57314.log.gz 2024-04-25T11:43:32.675 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72750.log 2024-04-25T11:43:32.675 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68979.log.gz 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40296.log 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63190.log.gz 2024-04-25T11:43:32.678 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45358.log 2024-04-25T11:43:32.678 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72750.log.gz 2024-04-25T11:43:32.679 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67257.log 2024-04-25T11:43:32.679 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40296.log.gz 2024-04-25T11:43:32.681 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45190.log 2024-04-25T11:43:32.681 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45358.log.gz 2024-04-25T11:43:32.682 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46482.log 2024-04-25T11:43:32.682 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67257.log.gz 2024-04-25T11:43:32.683 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79787.log 2024-04-25T11:43:32.683 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45190.log.gz 2024-04-25T11:43:32.684 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80182.log 2024-04-25T11:43:32.685 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46482.log.gz 2024-04-25T11:43:32.686 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73314.log 2024-04-25T11:43:32.686 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79787.log.gz 2024-04-25T11:43:32.687 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38598.log 2024-04-25T11:43:32.687 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80182.log.gz 2024-04-25T11:43:32.688 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71441.log 2024-04-25T11:43:32.688 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73314.log.gz 2024-04-25T11:43:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69269.log 2024-04-25T11:43:32.690 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38598.log.gz 2024-04-25T11:43:32.691 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80757.log 2024-04-25T11:43:32.691 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71441.log.gz 2024-04-25T11:43:32.692 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69632.log 2024-04-25T11:43:32.692 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69269.log.gz 2024-04-25T11:43:32.693 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87372.log 2024-04-25T11:43:32.693 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80757.log.gz 2024-04-25T11:43:32.695 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70499.log 2024-04-25T11:43:32.695 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69632.log.gz 2024-04-25T11:43:32.696 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90555.log 2024-04-25T11:43:32.696 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87372.log.gz 2024-04-25T11:43:32.697 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64059.log 2024-04-25T11:43:32.697 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70499.log.gz 2024-04-25T11:43:32.698 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70069.log 2024-04-25T11:43:32.699 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90555.log.gz 2024-04-25T11:43:32.700 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62931.log 2024-04-25T11:43:32.700 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64059.log.gz 2024-04-25T11:43:32.701 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52233.log 2024-04-25T11:43:32.701 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70069.log.gz 2024-04-25T11:43:32.702 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49835.log 2024-04-25T11:43:32.703 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.62931.log.gz 2024-04-25T11:43:32.704 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64096.log 2024-04-25T11:43:32.704 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52233.log.gz 2024-04-25T11:43:32.705 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49835.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49835.log.gz 2024-04-25T11:43:32.706 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78316.log 2024-04-25T11:43:32.707 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70251.log 2024-04-25T11:43:32.707 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64096.log.gz 2024-04-25T11:43:32.708 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42260.log 2024-04-25T11:43:32.708 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78316.log.gz 2024-04-25T11:43:32.710 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63301.log 2024-04-25T11:43:32.711 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50003.log 2024-04-25T11:43:32.711 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.70251.log.gz 2024-04-25T11:43:32.711 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42260.log.gz 2024-04-25T11:43:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2024-04-25T11:43:32.714 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63301.log.gz 2024-04-25T11:43:32.714 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62893.log 2024-04-25T11:43:32.714 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50003.log.gz 2024-04-25T11:43:32.715 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51710.log 2024-04-25T11:43:32.715 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2024-04-25T11:43:32.716 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54682.log 2024-04-25T11:43:32.716 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62893.log.gz 2024-04-25T11:43:32.717 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67787.log 2024-04-25T11:43:32.718 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51710.log.gz 2024-04-25T11:43:32.719 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70639.log 2024-04-25T11:43:32.719 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54682.log.gz 2024-04-25T11:43:32.720 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73995.log 2024-04-25T11:43:32.720 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67787.log.gz 2024-04-25T11:43:32.721 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46778.log 2024-04-25T11:43:32.723 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73995.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73995.log.gz 2024-04-25T11:43:32.723 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.70639.log.gz 2024-04-25T11:43:32.723 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79011.log 2024-04-25T11:43:32.723 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.46778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46778.log.gz 2024-04-25T11:43:32.724 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76821.log 2024-04-25T11:43:32.725 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79339.log 2024-04-25T11:43:32.726 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79011.log.gz 2024-04-25T11:43:32.727 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41258.log 2024-04-25T11:43:32.727 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76821.log.gz 2024-04-25T11:43:32.728 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44783.log 2024-04-25T11:43:32.728 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79339.log.gz 2024-04-25T11:43:32.730 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2024-04-25T11:43:32.730 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41258.log.gz 2024-04-25T11:43:32.731 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54886.log 2024-04-25T11:43:32.731 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44783.log.gz 2024-04-25T11:43:32.732 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2024-04-25T11:43:32.732 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2024-04-25T11:43:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90513.log 2024-04-25T11:43:32.734 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54886.log.gz 2024-04-25T11:43:32.735 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85011.log 2024-04-25T11:43:32.735 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2024-04-25T11:43:32.736 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51346.log 2024-04-25T11:43:32.736 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90513.log.gz 2024-04-25T11:43:32.737 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64022.log 2024-04-25T11:43:32.737 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85011.log.gz 2024-04-25T11:43:32.739 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42008.log 2024-04-25T11:43:32.739 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51346.log.gz 2024-04-25T11:43:32.740 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64022.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T11:43:32.740 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64022.log.gz 2024-04-25T11:43:32.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52952.log 2024-04-25T11:43:32.742 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42008.log.gz 2024-04-25T11:43:32.743 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65034.log 2024-04-25T11:43:32.744 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59689.log 2024-04-25T11:43:32.744 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52952.log.gz 2024-04-25T11:43:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.65034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65034.log.gz 2024-04-25T11:43:32.746 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90765.log 2024-04-25T11:43:32.747 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62433.log 2024-04-25T11:43:32.747 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90765.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59689.log.gz 2024-04-25T11:43:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60827.log 2024-04-25T11:43:32.748 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90765.log.gz 2024-04-25T11:43:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62433.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62433.log.gz 2024-04-25T11:43:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56726.log 2024-04-25T11:43:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60827.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60827.log.gz 2024-04-25T11:43:32.751 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-04-25T11:43:32.753 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66539.log 2024-04-25T11:43:32.753 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56726.log.gz 2024-04-25T11:43:32.754 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48828.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz 2024-04-25T11:43:32.754 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78511.log 2024-04-25T11:43:32.755 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66539.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38766.log 2024-04-25T11:43:32.756 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66539.log.gz 2024-04-25T11:43:32.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55550.log 2024-04-25T11:43:32.757 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78511.log.gz 2024-04-25T11:43:32.758 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38766.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38766.log.gz 2024-04-25T11:43:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64583.log 2024-04-25T11:43:32.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55550.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55550.log.gz 2024-04-25T11:43:32.760 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42470.log 2024-04-25T11:43:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78899.log 2024-04-25T11:43:32.761 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64583.log.gz 2024-04-25T11:43:32.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58826.log 2024-04-25T11:43:32.763 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42470.log.gz 2024-04-25T11:43:32.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78899.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78899.log.gz 2024-04-25T11:43:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58320.log 2024-04-25T11:43:32.765 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.58826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58826.log.gz 2024-04-25T11:43:32.766 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48683.log 2024-04-25T11:43:32.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83730.log 2024-04-25T11:43:32.767 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58320.log.gz 2024-04-25T11:43:32.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61791.log 2024-04-25T11:43:32.769 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48683.log.gz 2024-04-25T11:43:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83730.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83730.log.gz 2024-04-25T11:43:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75471.log 2024-04-25T11:43:32.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56548.log 2024-04-25T11:43:32.772 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61791.log.gz 2024-04-25T11:43:32.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78070.log 2024-04-25T11:43:32.773 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75471.log.gz 2024-04-25T11:43:32.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56548.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56548.log.gz 2024-04-25T11:43:32.775 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71603.log 2024-04-25T11:43:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78070.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78070.log.gz 2024-04-25T11:43:32.776 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86233.log 2024-04-25T11:43:32.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37996.log 2024-04-25T11:43:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86233.log: 89.1% 92.3% -- replaced with /var/log/ceph/ceph-client.admin.71603.log.gz 2024-04-25T11:43:32.779 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86233.log.gz 2024-04-25T11:43:32.779 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.37996.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60929.log 2024-04-25T11:43:32.780 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37996.log.gz 2024-04-25T11:43:32.781 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41738.log 2024-04-25T11:43:32.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56852.log 2024-04-25T11:43:32.783 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60929.log.gz 2024-04-25T11:43:32.784 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41738.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39804.log 2024-04-25T11:43:32.784 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41738.log.gz 2024-04-25T11:43:32.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61354.log 2024-04-25T11:43:32.785 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56852.log.gz 2024-04-25T11:43:32.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66311.log 2024-04-25T11:43:32.786 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39804.log.gz 2024-04-25T11:43:32.788 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58905.log 2024-04-25T11:43:32.788 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61354.log.gz 2024-04-25T11:43:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66311.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66311.log.gz 2024-04-25T11:43:32.789 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73704.log 2024-04-25T11:43:32.790 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58905.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58905.log.gz 2024-04-25T11:43:32.791 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80715.log 2024-04-25T11:43:32.792 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70849.log 2024-04-25T11:43:32.792 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73704.log.gz 2024-04-25T11:43:32.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80715.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80715.log.gz 2024-04-25T11:43:32.794 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73221.log 2024-04-25T11:43:32.796 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62601.log 2024-04-25T11:43:32.796 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70849.log.gz 2024-04-25T11:43:32.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72571.log 2024-04-25T11:43:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62601.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.73221.log.gz 87.9% 2024-04-25T11:43:32.799 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62601.log.gz 2024-04-25T11:43:32.799 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50977.log 2024-04-25T11:43:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72571.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72571.log.gz 2024-04-25T11:43:32.800 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71640.log 2024-04-25T11:43:32.801 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2024-04-25T11:43:32.802 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50977.log.gz 2024-04-25T11:43:32.803 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54640.log 2024-04-25T11:43:32.803 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.71640.log.gz 2024-04-25T11:43:32.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45917.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2024-04-25T11:43:32.805 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48169.log 2024-04-25T11:43:32.806 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82265.log 2024-04-25T11:43:32.806 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54640.log.gz 2024-04-25T11:43:32.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91467.log 2024-04-25T11:43:32.808 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48169.log.gz 2024-04-25T11:43:32.809 INFO:teuthology.orchestra.run.smithi169.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T11:43:32.809 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82265.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.82265.log.gz -- 2024-04-25T11:43:32.809 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.77430.log 2024-04-25T11:43:32.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68733.log 2024-04-25T11:43:32.810 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91467.log.gz 2024-04-25T11:43:32.811 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51201.log 2024-04-25T11:43:32.812 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77430.log.gz 2024-04-25T11:43:32.813 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68733.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68733.log.gz 2024-04-25T11:43:32.813 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59073.log 2024-04-25T11:43:32.814 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51201.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51201.log.gz 2024-04-25T11:43:32.815 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80224.log 2024-04-25T11:43:32.816 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85834.log 2024-04-25T11:43:32.817 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59073.log.gz 2024-04-25T11:43:32.818 INFO:teuthology.orchestra.run.smithi169.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T11:43:32.818 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80224.log: 91.3%/var/log/ceph/ceph-client.admin.85834.log: -- replaced with /var/log/ceph/ceph-client.admin.80224.log.gz 2024-04-25T11:43:32.819 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85834.log.gz 2024-04-25T11:43:32.819 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41864.log 2024-04-25T11:43:32.820 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50935.log 2024-04-25T11:43:32.822 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41864.log: gzip 89.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41864.log.gz -- 2024-04-25T11:43:32.822 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.90681.log 2024-04-25T11:43:32.823 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69520.log 2024-04-25T11:43:32.824 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50935.log.gz 2024-04-25T11:43:32.825 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90681.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90681.log.gz 2024-04-25T11:43:32.825 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43175.log 2024-04-25T11:43:32.826 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69520.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69520.log.gz 2024-04-25T11:43:32.826 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38215.log 2024-04-25T11:43:32.828 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54231.log 2024-04-25T11:43:32.828 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43175.log.gz 2024-04-25T11:43:32.830 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38215.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38215.log.gz 2024-04-25T11:43:32.830 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53871.log 2024-04-25T11:43:32.831 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77064.log 2024-04-25T11:43:32.832 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54231.log.gz 2024-04-25T11:43:32.832 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.53871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53871.log.gz 2024-04-25T11:43:32.833 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55634.log 2024-04-25T11:43:32.834 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88853.log 2024-04-25T11:43:32.835 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55634.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55634.log.gz 2024-04-25T11:43:32.835 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.77064.log.gz 2024-04-25T11:43:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43307.log 2024-04-25T11:43:32.836 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88853.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88853.log.gz 2024-04-25T11:43:32.837 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38092.log 2024-04-25T11:43:32.838 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88721.log 2024-04-25T11:43:32.839 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43307.log.gz 2024-04-25T11:43:32.840 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38092.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38092.log.gz 2024-04-25T11:43:32.841 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.88721.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66868.log 2024-04-25T11:43:32.841 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88721.log.gz 2024-04-25T11:43:32.842 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42929.log 2024-04-25T11:43:32.843 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64738.log 2024-04-25T11:43:32.844 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66868.log.gz 2024-04-25T11:43:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42929.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42929.log.gz 2024-04-25T11:43:32.845 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68817.log 2024-04-25T11:43:32.846 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64738.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64738.log.gz 2024-04-25T11:43:32.847 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86737.log 2024-04-25T11:43:32.848 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84834.log 2024-04-25T11:43:32.848 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68817.log.gz 2024-04-25T11:43:32.849 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87330.log 2024-04-25T11:43:32.849 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86737.log.gz 2024-04-25T11:43:32.851 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70344.log 2024-04-25T11:43:32.851 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84834.log.gz 2024-04-25T11:43:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47208.log 2024-04-25T11:43:32.852 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87330.log.gz 2024-04-25T11:43:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53142.log 2024-04-25T11:43:32.853 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70344.log.gz 2024-04-25T11:43:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47208.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47208.log.gz 2024-04-25T11:43:32.855 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60701.log 2024-04-25T11:43:32.856 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53142.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43133.log 2024-04-25T11:43:32.856 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53142.log.gz 2024-04-25T11:43:32.858 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86527.log 2024-04-25T11:43:32.858 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60701.log.gz 2024-04-25T11:43:32.859 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43133.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43133.log.gz 2024-04-25T11:43:32.860 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.86527.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56464.log 2024-04-25T11:43:32.860 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86527.log.gz 2024-04-25T11:43:32.861 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84955.log 2024-04-25T11:43:32.862 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59774.log 2024-04-25T11:43:32.862 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56464.log.gz 2024-04-25T11:43:32.864 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84955.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84955.log.gz 2024-04-25T11:43:32.865 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63634.log 2024-04-25T11:43:32.866 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53098.log 2024-04-25T11:43:32.866 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59774.log.gz 2024-04-25T11:43:32.868 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90429.log 2024-04-25T11:43:32.868 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63634.log.gz 2024-04-25T11:43:32.868 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53098.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53098.log.gz 2024-04-25T11:43:32.869 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57949.log 2024-04-25T11:43:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90429.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90429.log.gz 2024-04-25T11:43:32.870 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55000.log 2024-04-25T11:43:32.872 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57949.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57949.log.gz 2024-04-25T11:43:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.55000.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64523.log 2024-04-25T11:43:32.873 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55000.log.gz 2024-04-25T11:43:32.873 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71696.log 2024-04-25T11:43:32.875 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74370.log 2024-04-25T11:43:32.875 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64523.log.gz 2024-04-25T11:43:32.877 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71696.log: gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71696.log.gz 2024-04-25T11:43:32.877 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.85129.log 2024-04-25T11:43:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74370.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74370.log.gz 2024-04-25T11:43:32.878 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88517.log 2024-04-25T11:43:32.880 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47394.log 2024-04-25T11:43:32.880 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85129.log.gz 2024-04-25T11:43:32.881 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88517.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88517.log.gz 2024-04-25T11:43:32.883 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59935.log 2024-04-25T11:43:32.883 INFO:teuthology.orchestra.run.smithi019.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47394.log.gz 2024-04-25T11:43:32.884 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-04-25T11:43:32.885 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82223.log 2024-04-25T11:43:32.885 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59935.log.gz 2024-04-25T11:43:32.886 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82699.log 2024-04-25T11:43:32.887 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-04-25T11:43:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82223.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82223.log.gz 2024-04-25T11:43:32.888 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59153.log 2024-04-25T11:43:32.889 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82699.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38300.log 2024-04-25T11:43:32.889 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82699.log.gz 2024-04-25T11:43:32.891 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61312.log 2024-04-25T11:43:32.891 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59153.log.gz 2024-04-25T11:43:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38300.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38300.log.gz 2024-04-25T11:43:32.892 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-04-25T11:43:32.894 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61312.log: gzip 89.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.61312.log.gz -- 2024-04-25T11:43:32.894 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.76363.log 2024-04-25T11:43:32.895 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69058.log 2024-04-25T11:43:32.895 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-04-25T11:43:32.896 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76363.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76363.log.gz 2024-04-25T11:43:32.897 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77753.log 2024-04-25T11:43:32.897 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.69058.log.gz 2024-04-25T11:43:32.898 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83434.log 2024-04-25T11:43:32.900 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86653.log 2024-04-25T11:43:32.900 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77753.log.gz 2024-04-25T11:43:32.901 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83434.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83434.log.gz 2024-04-25T11:43:32.901 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41924.log 2024-04-25T11:43:32.902 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86653.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86653.log.gz 2024-04-25T11:43:32.903 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90125.log 2024-04-25T11:43:32.904 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65163.log 2024-04-25T11:43:32.904 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41924.log.gz 2024-04-25T11:43:32.905 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90125.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90125.log.gz 2024-04-25T11:43:32.906 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.65163.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69815.log 2024-04-25T11:43:32.906 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65163.log.gz 2024-04-25T11:43:32.907 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66334.log 2024-04-25T11:43:32.909 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89586.log 2024-04-25T11:43:32.910 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66334.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.69815.log.gz 2024-04-25T11:43:32.911 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89586.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89586.log.gz 2024-04-25T11:43:32.911 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.66334.log.gz 2024-04-25T11:43:32.912 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80799.log 2024-04-25T11:43:32.913 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71357.log 2024-04-25T11:43:32.914 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71273.log 2024-04-25T11:43:32.914 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80799.log.gz 2024-04-25T11:43:32.916 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45959.log 2024-04-25T11:43:32.916 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71357.log.gz 2024-04-25T11:43:32.917 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67829.log 2024-04-25T11:43:32.917 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71273.log.gz 2024-04-25T11:43:32.919 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45959.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45959.log.gz 2024-04-25T11:43:32.919 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57823.log 2024-04-25T11:43:32.919 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67829.log.gz 2024-04-25T11:43:32.920 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45484.log 2024-04-25T11:43:32.921 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81341.log 2024-04-25T11:43:32.921 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57823.log.gz 2024-04-25T11:43:32.923 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44285.log 2024-04-25T11:43:32.923 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81341.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45484.log.gz 2024-04-25T11:43:32.924 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81341.log.gz 2024-04-25T11:43:32.925 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53491.log 2024-04-25T11:43:32.926 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78400.log 2024-04-25T11:43:32.926 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44285.log.gz 2024-04-25T11:43:32.927 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60743.log 2024-04-25T11:43:32.927 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53491.log.gz 2024-04-25T11:43:32.930 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78400.log: /var/log/ceph/ceph-client.admin.60743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39745.log 2024-04-25T11:43:32.930 INFO:teuthology.orchestra.run.smithi019.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.78400.log.gz 2024-04-25T11:43:32.931 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60743.log.gz 2024-04-25T11:43:32.931 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79054.log 2024-04-25T11:43:32.932 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40770.log 2024-04-25T11:43:32.932 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39745.log.gz 2024-04-25T11:43:32.934 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75106.log 2024-04-25T11:43:32.934 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79054.log.gz 2024-04-25T11:43:32.935 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40770.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40770.log.gz 2024-04-25T11:43:32.935 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84254.log 2024-04-25T11:43:32.937 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52910.log 2024-04-25T11:43:32.938 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84254.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.75106.log.gz 2024-04-25T11:43:32.938 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84254.log.gz 2024-04-25T11:43:32.938 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74923.log 2024-04-25T11:43:32.940 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59236.log 2024-04-25T11:43:32.940 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52910.log.gz 2024-04-25T11:43:32.941 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77941.log 2024-04-25T11:43:32.941 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74923.log.gz 2024-04-25T11:43:32.942 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51019.log 2024-04-25T11:43:32.943 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59236.log.gz 2024-04-25T11:43:32.944 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.77941.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87502.log 2024-04-25T11:43:32.944 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.0.77941.log.gz 2024-04-25T11:43:32.945 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51019.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51019.log.gz 2024-04-25T11:43:32.945 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2024-04-25T11:43:32.946 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87502.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87502.log.gz 2024-04-25T11:43:32.947 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58236.log 2024-04-25T11:43:32.948 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38428.log 2024-04-25T11:43:32.948 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2024-04-25T11:43:32.950 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81645.log 2024-04-25T11:43:32.950 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58236.log.gz 2024-04-25T11:43:32.951 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38428.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38428.log.gz 2024-04-25T11:43:32.951 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80841.log 2024-04-25T11:43:32.952 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81645.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81645.log.gz 2024-04-25T11:43:32.953 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49705.log 2024-04-25T11:43:32.954 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62517.log 2024-04-25T11:43:32.954 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80841.log.gz 2024-04-25T11:43:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49705.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49705.log.gz 2024-04-25T11:43:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72674.log 2024-04-25T11:43:32.957 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62517.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50045.log 2024-04-25T11:43:32.957 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62517.log.gz 2024-04-25T11:43:32.958 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79442.log 2024-04-25T11:43:32.958 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72674.log.gz 2024-04-25T11:43:32.960 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50045.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50045.log.gz 2024-04-25T11:43:32.960 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92055.log 2024-04-25T11:43:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79442.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79442.log.gz 2024-04-25T11:43:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63264.log 2024-04-25T11:43:32.963 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.92055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55214.log 2024-04-25T11:43:32.963 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92055.log.gz 2024-04-25T11:43:32.964 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63264.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63264.log.gz 2024-04-25T11:43:32.964 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81299.log 2024-04-25T11:43:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55214.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55214.log.gz 2024-04-25T11:43:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76130.log 2024-04-25T11:43:32.967 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91761.log 2024-04-25T11:43:32.968 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81299.log.gz 2024-04-25T11:43:32.969 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50551.log 2024-04-25T11:43:32.969 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76130.log.gz 2024-04-25T11:43:32.970 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91761.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91761.log.gz 2024-04-25T11:43:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90006.log 2024-04-25T11:43:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.50551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50551.log.gz 2024-04-25T11:43:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85353.log 2024-04-25T11:43:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56590.log 2024-04-25T11:43:32.974 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0%/var/log/ceph/ceph-client.admin.85353.log: -- replaced with /var/log/ceph/ceph-client.admin.90006.log.gz 2024-04-25T11:43:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66151.log 2024-04-25T11:43:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56590.log: 87.5% 92.4% -- replaced with /var/log/ceph/ceph-client.admin.85353.log.gz 2024-04-25T11:43:32.977 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56590.log.gz 2024-04-25T11:43:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55256.log 2024-04-25T11:43:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66151.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66151.log.gz 2024-04-25T11:43:32.978 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41105.log 2024-04-25T11:43:32.979 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76247.log 2024-04-25T11:43:32.979 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55256.log.gz 2024-04-25T11:43:32.981 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.41105.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.0.41105.log.gz 2024-04-25T11:43:32.981 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72137.log 2024-04-25T11:43:32.983 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76247.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58148.log 2024-04-25T11:43:32.983 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76247.log.gz 2024-04-25T11:43:32.984 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51752.log 2024-04-25T11:43:32.984 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72137.log.gz 2024-04-25T11:43:32.985 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80478.log 2024-04-25T11:43:32.985 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58148.log.gz 2024-04-25T11:43:32.986 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81574.log 2024-04-25T11:43:32.987 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51752.log.gz 2024-04-25T11:43:32.987 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.80478.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.80478.log.gz -5 2024-04-25T11:43:32.988 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64692.log 2024-04-25T11:43:32.989 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51097.log 2024-04-25T11:43:32.989 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81574.log.gz 2024-04-25T11:43:32.992 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57230.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64692.log.gz 2024-04-25T11:43:32.992 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:32.992 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51097.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72084.log 2024-04-25T11:43:32.992 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz 2024-04-25T11:43:32.993 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69021.log 2024-04-25T11:43:32.993 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57230.log.gz 2024-04-25T11:43:32.996 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72084.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72084.log.gz 2024-04-25T11:43:32.996 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.69021.log: -5 --verbose -- /var/log/ceph/ceph-client.0.83059.log 2024-04-25T11:43:32.997 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78741.log 2024-04-25T11:43:32.997 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.69021.log.gz 2024-04-25T11:43:32.998 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.83059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.83059.log.gz 2024-04-25T11:43:32.998 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90723.log 2024-04-25T11:43:32.999 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67429.log 2024-04-25T11:43:33.000 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78741.log.gz 2024-04-25T11:43:33.001 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60407.log 2024-04-25T11:43:33.001 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90723.log.gz 2024-04-25T11:43:33.002 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67429.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75975.log 2024-04-25T11:43:33.002 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67429.log.gz 2024-04-25T11:43:33.003 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63560.log 2024-04-25T11:43:33.004 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60407.log.gz 2024-04-25T11:43:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75975.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75975.log.gz 2024-04-25T11:43:33.005 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60019.log 2024-04-25T11:43:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61959.log 2024-04-25T11:43:33.006 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63560.log.gz 2024-04-25T11:43:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62679.log 2024-04-25T11:43:33.008 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60019.log.gz 2024-04-25T11:43:33.009 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61959.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61959.log.gz 2024-04-25T11:43:33.009 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49524.log 2024-04-25T11:43:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62679.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62679.log.gz 2024-04-25T11:43:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84188.log 2024-04-25T11:43:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2024-04-25T11:43:33.012 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49524.log.gz 2024-04-25T11:43:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51920.log 2024-04-25T11:43:33.014 INFO:teuthology.orchestra.run.smithi169.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T11:43:33.016 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.84188.log.gz 2024-04-25T11:43:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.49566.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.67991.log 2024-04-25T11:43:33.016 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3%/var/log/ceph/ceph-client.admin.51920.log: -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2024-04-25T11:43:33.017 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51920.log.gz 2024-04-25T11:43:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47082.log 2024-04-25T11:43:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67991.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67991.log.gz 2024-04-25T11:43:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47082.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47082.log.gz 2024-04-25T11:43:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63227.log 2024-04-25T11:43:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82349.log 2024-04-25T11:43:33.022 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77599.log 2024-04-25T11:43:33.022 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63227.log.gz 2024-04-25T11:43:33.023 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82349.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82349.log.gz 2024-04-25T11:43:33.024 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.78806.log 2024-04-25T11:43:33.025 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77599.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59893.log 2024-04-25T11:43:33.025 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77599.log.gz 2024-04-25T11:43:33.026 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63782.log 2024-04-25T11:43:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59893.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59893.log.gz 2024-04-25T11:43:33.028 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.78806.log.gz 2024-04-25T11:43:33.028 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92179.log/var/log/ceph/ceph-client.admin.63782.log: 2024-04-25T11:43:33.029 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.63782.log.gz 2024-04-25T11:43:33.030 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75699.log 2024-04-25T11:43:33.031 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.92179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-04-25T11:43:33.031 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92179.log.gz 2024-04-25T11:43:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75699.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75699.log.gz 2024-04-25T11:43:33.033 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79582.log 2024-04-25T11:43:33.033 INFO:teuthology.orchestra.run.smithi169.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T11:43:33.033 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43349.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-04-25T11:43:33.034 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68221.log 2024-04-25T11:43:33.035 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65451.log 2024-04-25T11:43:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68221.log: 89.0% 92.3% -- replaced with /var/log/ceph/ceph-client.admin.79582.log.gz 2024-04-25T11:43:33.038 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68221.log.gz 2024-04-25T11:43:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.65451.log: -- /var/log/ceph/ceph-client.admin.49482.log 2024-04-25T11:43:33.038 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65451.log.gz 2024-04-25T11:43:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65331.log 2024-04-25T11:43:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68267.log 2024-04-25T11:43:33.040 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49482.log.gz 2024-04-25T11:43:33.042 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65331.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65331.log.gz 2024-04-25T11:43:33.042 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.68267.log: -- /var/log/ceph/ceph-client.admin.42176.log 2024-04-25T11:43:33.042 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68267.log.gz 2024-04-25T11:43:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88559.log 2024-04-25T11:43:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53235.log 2024-04-25T11:43:33.045 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42176.log.gz 2024-04-25T11:43:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88559.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88559.log.gz 2024-04-25T11:43:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.53235.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44407.log 2024-04-25T11:43:33.048 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53235.log.gz 2024-04-25T11:43:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48431.log 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60449.log 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44407.log.gz 2024-04-25T11:43:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42218.log 2024-04-25T11:43:33.051 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48431.log.gz 2024-04-25T11:43:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60449.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60449.log.gz 2024-04-25T11:43:33.053 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40548.log 2024-04-25T11:43:33.054 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58638.log 2024-04-25T11:43:33.054 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42218.log.gz 2024-04-25T11:43:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87670.log 2024-04-25T11:43:33.055 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40548.log.gz 2024-04-25T11:43:33.057 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58638.log: gzip -5 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58638.log.gz 2024-04-25T11:43:33.057 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.82469.log 2024-04-25T11:43:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87670.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87670.log.gz 2024-04-25T11:43:33.058 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89352.log 2024-04-25T11:43:33.060 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85330.log 2024-04-25T11:43:33.060 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82469.log.gz 2024-04-25T11:43:33.061 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89352.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.89352.log.gz 2024-04-25T11:43:33.062 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85446.log 2024-04-25T11:43:33.062 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85330.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85330.log.gz 2024-04-25T11:43:33.063 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61707.log 2024-04-25T11:43:33.064 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81799.log 2024-04-25T11:43:33.064 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85446.log.gz 2024-04-25T11:43:33.066 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57741.log 2024-04-25T11:43:33.066 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61707.log.gz 2024-04-25T11:43:33.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70186.log 2024-04-25T11:43:33.067 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81799.log.gz 2024-04-25T11:43:33.068 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44015.log 2024-04-25T11:43:33.068 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57741.log.gz 2024-04-25T11:43:33.070 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70186.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70186.log.gz 2024-04-25T11:43:33.070 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40170.log 2024-04-25T11:43:33.071 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76653.log 2024-04-25T11:43:33.071 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44015.log.gz 2024-04-25T11:43:33.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40170.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40170.log.gz 2024-04-25T11:43:33.073 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63856.log 2024-04-25T11:43:33.074 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84918.log 2024-04-25T11:43:33.075 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76653.log.gz 2024-04-25T11:43:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63856.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.63856.log.gz 2024-04-25T11:43:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63005.log 2024-04-25T11:43:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78358.log 2024-04-25T11:43:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63005.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.84918.log.gz 2024-04-25T11:43:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose 89.7% -- -- replaced with /var/log/ceph/ceph-client.admin.63005.log.gz 2024-04-25T11:43:33.079 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.61577.log 2024-04-25T11:43:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78358.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78358.log.gz 2024-04-25T11:43:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41780.log 2024-04-25T11:43:33.082 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85778.log 2024-04-25T11:43:33.082 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61577.log.gz 2024-04-25T11:43:33.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41780.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41780.log.gz 2024-04-25T11:43:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41696.log 2024-04-25T11:43:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85778.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.85778.log.gz 2024-04-25T11:43:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73156.log 2024-04-25T11:43:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41696.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41696.log.gz 2024-04-25T11:43:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88995.log 2024-04-25T11:43:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54297.log 2024-04-25T11:43:33.088 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73156.log.gz 2024-04-25T11:43:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88995.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88995.log.gz 2024-04-25T11:43:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62968.log 2024-04-25T11:43:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54297.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54297.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67871.log 2024-04-25T11:43:33.091 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53193.log 2024-04-25T11:43:33.093 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62968.log.gz 2024-04-25T11:43:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39746.log 2024-04-25T11:43:33.094 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67871.log.gz 2024-04-25T11:43:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58446.log 2024-04-25T11:43:33.095 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53193.log.gz 2024-04-25T11:43:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74676.log 2024-04-25T11:43:33.096 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39746.log.gz 2024-04-25T11:43:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62764.log 2024-04-25T11:43:33.098 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58446.log.gz 2024-04-25T11:43:33.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43679.log 2024-04-25T11:43:33.099 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74676.log.gz 2024-04-25T11:43:33.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70957.log 2024-04-25T11:43:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43679.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.62764.log.gz 2024-04-25T11:43:33.102 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43679.log.gz 2024-04-25T11:43:33.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88194.log 2024-04-25T11:43:33.103 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70957.log.gz 2024-04-25T11:43:33.104 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62475.log 2024-04-25T11:43:33.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.88194.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75817.log 2024-04-25T11:43:33.105 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.0.88194.log.gz 2024-04-25T11:43:33.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62475.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62475.log.gz 2024-04-25T11:43:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63597.log 2024-04-25T11:43:33.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75817.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75817.log.gz 2024-04-25T11:43:33.108 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78951.log 2024-04-25T11:43:33.110 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48127.log 2024-04-25T11:43:33.110 INFO:teuthology.orchestra.run.smithi019.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.63597.log.gz 2024-04-25T11:43:33.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78951.log: gzip -5 --verbose -- 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78951.log.gz 2024-04-25T11:43:33.111 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.38514.log 2024-04-25T11:43:33.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48127.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.48127.log.gz --verbose 2024-04-25T11:43:33.112 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.51243.log 2024-04-25T11:43:33.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65205.log 2024-04-25T11:43:33.113 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38514.log.gz 2024-04-25T11:43:33.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38892.log 2024-04-25T11:43:33.115 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51243.log.gz 2024-04-25T11:43:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65205.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65205.log.gz 2024-04-25T11:43:33.116 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53985.log 2024-04-25T11:43:33.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76087.log 2024-04-25T11:43:33.118 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38892.log.gz 2024-04-25T11:43:33.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65923.log 2024-04-25T11:43:33.121 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53985.log.gz 2024-04-25T11:43:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70027.log 2024-04-25T11:43:33.122 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76087.log.gz 2024-04-25T11:43:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83954.log 2024-04-25T11:43:33.122 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65923.log.gz 2024-04-25T11:43:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56246.log 2024-04-25T11:43:33.123 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70027.log.gz 2024-04-25T11:43:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46163.log 2024-04-25T11:43:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.56246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56246.log.gz 2024-04-25T11:43:33.126 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87460.log 2024-04-25T11:43:33.126 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83954.log.gz 2024-04-25T11:43:33.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40044.log 2024-04-25T11:43:33.127 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46163.log.gz 2024-04-25T11:43:33.128 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46590.log 2024-04-25T11:43:33.128 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87460.log.gz 2024-04-25T11:43:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40044.log: gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40044.log.gz 2024-04-25T11:43:33.130 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.65586.log 2024-04-25T11:43:33.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75882.log 2024-04-25T11:43:33.131 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46590.log.gz 2024-04-25T11:43:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80064.log 2024-04-25T11:43:33.132 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65586.log.gz 2024-04-25T11:43:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39466.log 2024-04-25T11:43:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80064.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.75882.log.gz 2024-04-25T11:43:33.135 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80064.log.gz 2024-04-25T11:43:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38724.log 2024-04-25T11:43:33.137 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39466.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.39466.log.gz 2024-04-25T11:43:33.137 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.79231.log 2024-04-25T11:43:33.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38724.log: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T11:43:33.138 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38724.log.gz 2024-04-25T11:43:33.138 INFO:teuthology.orchestra.run.smithi169.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T11:43:33.139 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51591.log 2024-04-25T11:43:33.140 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75199.log 2024-04-25T11:43:33.140 INFO:teuthology.orchestra.run.smithi019.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.79231.log.gz 2024-04-25T11:43:33.141 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72286.log 2024-04-25T11:43:33.141 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51591.log.gz 2024-04-25T11:43:33.142 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-25T11:43:33.142 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m1.102s 2024-04-25T11:43:33.142 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m4.899s 2024-04-25T11:43:33.142 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.503s 2024-04-25T11:43:33.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55466.log 2024-04-25T11:43:33.143 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75199.log.gz 2024-04-25T11:43:33.144 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57020.log 2024-04-25T11:43:33.144 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72286.log.gz 2024-04-25T11:43:33.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56506.log 2024-04-25T11:43:33.145 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55466.log.gz 2024-04-25T11:43:33.146 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85887.log 2024-04-25T11:43:33.146 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57020.log.gz 2024-04-25T11:43:33.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75429.log 2024-04-25T11:43:33.147 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56506.log.gz 2024-04-25T11:43:33.148 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82083.log 2024-04-25T11:43:33.149 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85887.log.gz 2024-04-25T11:43:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60061.log 2024-04-25T11:43:33.150 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75429.log.gz 2024-04-25T11:43:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43217.log 2024-04-25T11:43:33.151 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82083.log.gz 2024-04-25T11:43:33.152 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45148.log 2024-04-25T11:43:33.152 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60061.log.gz 2024-04-25T11:43:33.154 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43217.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43217.log.gz 2024-04-25T11:43:33.154 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.55060.log 2024-04-25T11:43:33.155 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42386.log 2024-04-25T11:43:33.155 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45148.log.gz 2024-04-25T11:43:33.156 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64133.log 2024-04-25T11:43:33.156 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55060.log.gz 2024-04-25T11:43:33.157 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57398.log 2024-04-25T11:43:33.158 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42386.log.gz 2024-04-25T11:43:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73887.log 2024-04-25T11:43:33.159 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.64133.log.gz 2024-04-25T11:43:33.159 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71808.log 2024-04-25T11:43:33.160 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57398.log.gz 2024-04-25T11:43:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77313.log 2024-04-25T11:43:33.162 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73887.log.gz 2024-04-25T11:43:33.162 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71808.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71808.log.gz 2024-04-25T11:43:33.163 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66759.log 2024-04-25T11:43:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77313.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77313.log.gz 2024-04-25T11:43:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57907.log 2024-04-25T11:43:33.166 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63819.log 2024-04-25T11:43:33.166 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.66759.log.gz 2024-04-25T11:43:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79559.log 2024-04-25T11:43:33.167 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57907.log.gz 2024-04-25T11:43:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90293.log 2024-04-25T11:43:33.168 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63819.log.gz 2024-04-25T11:43:33.169 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60491.log 2024-04-25T11:43:33.169 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79559.log.gz 2024-04-25T11:43:33.170 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65746.log 2024-04-25T11:43:33.170 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90293.log.gz 2024-04-25T11:43:33.172 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83518.log 2024-04-25T11:43:33.172 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60491.log.gz 2024-04-25T11:43:33.173 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90209.log 2024-04-25T11:43:33.173 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65746.log.gz 2024-04-25T11:43:33.174 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64170.log 2024-04-25T11:43:33.175 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90928.log 2024-04-25T11:43:33.175 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90209.log.gz 2024-04-25T11:43:33.175 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.83518.log.gz 2024-04-25T11:43:33.176 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67617.log 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.64170.log.gz 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90928.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.66039.log 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90928.log.gz 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.67617.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48767.log 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.0.67617.log.gz 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46001.log 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66039.log.gz 2024-04-25T11:43:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66722.log 2024-04-25T11:43:33.182 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48767.log.gz 2024-04-25T11:43:33.182 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42794.log 2024-04-25T11:43:33.183 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46001.log.gz 2024-04-25T11:43:33.184 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77645.log 2024-04-25T11:43:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72028.log 2024-04-25T11:43:33.185 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42794.log.gz 2024-04-25T11:43:33.186 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.66722.log.gz 2024-04-25T11:43:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52359.log 2024-04-25T11:43:33.186 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77645.log.gz 2024-04-25T11:43:33.187 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90251.log 2024-04-25T11:43:33.188 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.72028.log.gz 2024-04-25T11:43:33.188 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52786.log 2024-04-25T11:43:33.189 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52359.log.gz 2024-04-25T11:43:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.90251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2024-04-25T11:43:33.190 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90251.log.gz 2024-04-25T11:43:33.191 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39447.log 2024-04-25T11:43:33.191 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52786.log.gz 2024-04-25T11:43:33.192 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61526.log 2024-04-25T11:43:33.192 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz 2024-04-25T11:43:33.193 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74718.log 2024-04-25T11:43:33.193 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39447.log.gz 2024-04-25T11:43:33.195 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46085.log 2024-04-25T11:43:33.195 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61526.log.gz 2024-04-25T11:43:33.196 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58035.log 2024-04-25T11:43:33.197 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89790.log 2024-04-25T11:43:33.197 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46085.log.gz 2024-04-25T11:43:33.197 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.74718.log.gz 2024-04-25T11:43:33.198 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77453.log 2024-04-25T11:43:33.198 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58035.log.gz 2024-04-25T11:43:33.200 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51962.log 2024-04-25T11:43:33.200 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.89790.log.gz 2024-04-25T11:43:33.201 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79675.log 2024-04-25T11:43:33.202 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74496.log 2024-04-25T11:43:33.202 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51962.log.gz 2024-04-25T11:43:33.202 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.77453.log.gz 2024-04-25T11:43:33.203 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46524.log 2024-04-25T11:43:33.203 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79675.log.gz 2024-04-25T11:43:33.205 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58362.log 2024-04-25T11:43:33.205 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74496.log.gz 2024-04-25T11:43:33.206 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39469.log 2024-04-25T11:43:33.207 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46524.log.gz 2024-04-25T11:43:33.207 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59542.log 2024-04-25T11:43:33.207 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58362.log.gz 2024-04-25T11:43:33.209 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44657.log 2024-04-25T11:43:33.209 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39469.log.gz 2024-04-25T11:43:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59542.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59542.log.gz 2024-04-25T11:43:33.210 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84666.log 2024-04-25T11:43:33.211 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52485.log 2024-04-25T11:43:33.211 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44657.log.gz 2024-04-25T11:43:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72926.log 2024-04-25T11:43:33.213 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84666.log.gz 2024-04-25T11:43:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61749.log 2024-04-25T11:43:33.214 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52485.log.gz 2024-04-25T11:43:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77101.log 2024-04-25T11:43:33.215 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72926.log.gz 2024-04-25T11:43:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33850.log 2024-04-25T11:43:33.216 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61749.log.gz 2024-04-25T11:43:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69908.log 2024-04-25T11:43:33.218 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77101.log.gz 2024-04-25T11:43:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.33850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71145.log 2024-04-25T11:43:33.218 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33850.log.gz 2024-04-25T11:43:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69908.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69908.log.gz 2024-04-25T11:43:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40464.log 2024-04-25T11:43:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.71145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71145.log.gz 2024-04-25T11:43:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44741.log 2024-04-25T11:43:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52172.log 2024-04-25T11:43:33.223 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40464.log.gz 2024-04-25T11:43:33.224 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-04-25T11:43:33.224 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44741.log.gz 2024-04-25T11:43:33.225 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81003.log 2024-04-25T11:43:33.225 INFO:teuthology.orchestra.run.smithi019.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52172.log.gz 2024-04-25T11:43:33.226 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48913.log 2024-04-25T11:43:33.226 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-04-25T11:43:33.228 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33463.log 2024-04-25T11:43:33.228 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81003.log.gz 2024-04-25T11:43:33.229 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79171.log 2024-04-25T11:43:33.229 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48913.log.gz 2024-04-25T11:43:33.230 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.33463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69409.log 2024-04-25T11:43:33.230 INFO:teuthology.orchestra.run.smithi019.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.33463.log.gz 2024-04-25T11:43:33.231 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57188.log 2024-04-25T11:43:33.231 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79171.log.gz 2024-04-25T11:43:33.232 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69750.log 2024-04-25T11:43:33.234 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48641.log 2024-04-25T11:43:33.234 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57188.log.gz 2024-04-25T11:43:33.234 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.69409.log.gz 2024-04-25T11:43:33.235 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75354.log 2024-04-25T11:43:33.235 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69750.log.gz 2024-04-25T11:43:33.236 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57440.log 2024-04-25T11:43:33.236 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48641.log.gz 2024-04-25T11:43:33.238 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89291.log 2024-04-25T11:43:33.238 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75354.log.gz 2024-04-25T11:43:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83356.log 2024-04-25T11:43:33.239 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57440.log.gz 2024-04-25T11:43:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.89291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.89291.log.gz 2024-04-25T11:43:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70616.log 2024-04-25T11:43:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83356.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83356.log.gz 2024-04-25T11:43:33.242 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80007.log 2024-04-25T11:43:33.243 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81083.log 2024-04-25T11:43:33.243 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70616.log.gz 2024-04-25T11:43:33.244 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83991.log 2024-04-25T11:43:33.244 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.80007.log.gz 2024-04-25T11:43:33.245 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81518.log 2024-04-25T11:43:33.246 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81083.log.gz 2024-04-25T11:43:33.247 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65704.log 2024-04-25T11:43:33.247 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83991.log.gz 2024-04-25T11:43:33.248 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60323.log 2024-04-25T11:43:33.248 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.81518.log.gz 2024-04-25T11:43:33.249 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52317.log 2024-04-25T11:43:33.249 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65704.log.gz 2024-04-25T11:43:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83314.log 2024-04-25T11:43:33.251 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60323.log.gz 2024-04-25T11:43:33.251 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44699.log 2024-04-25T11:43:33.252 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52317.log.gz 2024-04-25T11:43:33.253 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.83314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61013.log 2024-04-25T11:43:33.253 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83314.log.gz 2024-04-25T11:43:33.254 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45316.log 2024-04-25T11:43:33.254 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44699.log.gz 2024-04-25T11:43:33.255 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39133.log 2024-04-25T11:43:33.255 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61013.log.gz 2024-04-25T11:43:33.257 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59296.log 2024-04-25T11:43:33.257 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45316.log.gz 2024-04-25T11:43:33.258 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39133.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T11:43:33.258 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39133.log.gz 2024-04-25T11:43:33.259 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63745.log 2024-04-25T11:43:33.259 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59296.log.gz 2024-04-25T11:43:33.260 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86359.log 2024-04-25T11:43:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50171.log 2024-04-25T11:43:33.262 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63745.log.gz 2024-04-25T11:43:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88763.log 2024-04-25T11:43:33.263 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86359.log.gz 2024-04-25T11:43:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50171.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50171.log.gz 2024-04-25T11:43:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61875.log 2024-04-25T11:43:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.88763.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88763.log.gz 2024-04-25T11:43:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81416.log 2024-04-25T11:43:33.267 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73611.log 2024-04-25T11:43:33.267 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61875.log.gz 2024-04-25T11:43:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39467.log 2024-04-25T11:43:33.269 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81416.log.gz 2024-04-25T11:43:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69166.log 2024-04-25T11:43:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39467.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.73611.log.gz 2024-04-25T11:43:33.271 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39467.log.gz 2024-04-25T11:43:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89748.log 2024-04-25T11:43:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69166.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69166.log.gz 2024-04-25T11:43:33.273 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-04-25T11:43:33.275 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76999.log 2024-04-25T11:43:33.275 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89748.log.gz 2024-04-25T11:43:33.276 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48955.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-04-25T11:43:33.277 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.76999.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54027.log 2024-04-25T11:43:33.277 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76999.log.gz 2024-04-25T11:43:33.278 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57991.log 2024-04-25T11:43:33.279 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76713.log 2024-04-25T11:43:33.279 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54027.log.gz 2024-04-25T11:43:33.281 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57991.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57991.log.gz 2024-04-25T11:43:33.281 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52130.log 2024-04-25T11:43:33.282 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76713.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76713.log.gz 2024-04-25T11:43:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54485.log 2024-04-25T11:43:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89520.log 2024-04-25T11:43:33.284 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52130.log.gz 2024-04-25T11:43:33.285 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.54485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54485.log.gz 2024-04-25T11:43:33.285 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51303.log 2024-04-25T11:43:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.89520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73038.log 2024-04-25T11:43:33.287 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89520.log.gz 2024-04-25T11:43:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76676.log 2024-04-25T11:43:33.288 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51303.log.gz 2024-04-25T11:43:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73038.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73038.log.gz 2024-04-25T11:43:33.290 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66815.log 2024-04-25T11:43:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91425.log 2024-04-25T11:43:33.292 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66815.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.76676.log.gz 2024-04-25T11:43:33.293 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51399.log 2024-04-25T11:43:33.293 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66815.log.gz 2024-04-25T11:43:33.294 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91425.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91425.log.gz 2024-04-25T11:43:33.294 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68937.log 2024-04-25T11:43:33.295 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65530.log 2024-04-25T11:43:33.296 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51399.log.gz 2024-04-25T11:43:33.297 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59647.log 2024-04-25T11:43:33.297 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68937.log.gz 2024-04-25T11:43:33.298 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65530.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65530.log.gz 2024-04-25T11:43:33.299 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46891.log 2024-04-25T11:43:33.300 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43091.log 2024-04-25T11:43:33.300 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59647.log.gz 2024-04-25T11:43:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43433.log 2024-04-25T11:43:33.301 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46891.log.gz 2024-04-25T11:43:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43091.log: gzip 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.39128.log 2024-04-25T11:43:33.303 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43091.log.gz 2024-04-25T11:43:33.303 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43433.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43433.log.gz 2024-04-25T11:43:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87288.log 2024-04-25T11:43:33.305 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43595.log 2024-04-25T11:43:33.305 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39128.log.gz 2024-04-25T11:43:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73648.log 2024-04-25T11:43:33.307 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87288.log.gz 2024-04-25T11:43:33.308 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73845.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43595.log.gz 2024-04-25T11:43:33.308 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73648.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73648.log.gz 2024-04-25T11:43:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55424.log 2024-04-25T11:43:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48211.log 2024-04-25T11:43:33.311 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73845.log.gz 2024-04-25T11:43:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68309.log 2024-04-25T11:43:33.312 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55424.log.gz 2024-04-25T11:43:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48211.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48211.log.gz 2024-04-25T11:43:33.314 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63948.log 2024-04-25T11:43:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43763.log 2024-04-25T11:43:33.315 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68309.log.gz 2024-04-25T11:43:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63116.log 2024-04-25T11:43:33.316 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.63948.log.gz 2024-04-25T11:43:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43763.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67046.log 2024-04-25T11:43:33.318 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43763.log.gz 2024-04-25T11:43:33.319 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62067.log 2024-04-25T11:43:33.320 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.63116.log.gz 2024-04-25T11:43:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67046.log: gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67046.log.gz 2024-04-25T11:43:33.321 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43931.log 2024-04-25T11:43:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62067.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62067.log.gz 2024-04-25T11:43:33.322 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57482.log 2024-04-25T11:43:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65806.log 2024-04-25T11:43:33.323 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43931.log.gz 2024-04-25T11:43:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57482.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57482.log.gz 2024-04-25T11:43:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38342.log 2024-04-25T11:43:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77687.log 2024-04-25T11:43:33.327 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65806.log.gz 2024-04-25T11:43:33.327 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53323.log 2024-04-25T11:43:33.328 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38342.log.gz 2024-04-25T11:43:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77687.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69226.log 2024-04-25T11:43:33.330 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77687.log.gz 2024-04-25T11:43:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43475.log 2024-04-25T11:43:33.331 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53323.log.gz 2024-04-25T11:43:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.69226.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.87922.log 2024-04-25T11:43:33.333 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4%/var/log/ceph/ceph-client.admin.43475.log: -- replaced with /var/log/ceph/ceph-client.admin.69226.log.gz 2024-04-25T11:43:33.333 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43475.log.gz 2024-04-25T11:43:33.334 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-04-25T11:43:33.335 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66971.log 2024-04-25T11:43:33.336 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.87922.log.gz 2024-04-25T11:43:33.337 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79905.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-04-25T11:43:33.337 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:33.337 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66971.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66971.log.gz 2024-04-25T11:43:33.338 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86821.log 2024-04-25T11:43:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79905.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79905.log.gz 2024-04-25T11:43:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70891.log 2024-04-25T11:43:33.340 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86821.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86821.log.gz 2024-04-25T11:43:33.341 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82646.log 2024-04-25T11:43:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55508.log 2024-04-25T11:43:33.343 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.70891.log.gz/var/log/ceph/ceph-client.admin.82646.log: 2024-04-25T11:43:33.343 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82646.log.gz 2024-04-25T11:43:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63375.log 2024-04-25T11:43:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55508.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55508.log.gz 2024-04-25T11:43:33.345 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40506.log 2024-04-25T11:43:33.347 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38556.log 2024-04-25T11:43:33.347 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63375.log.gz 2024-04-25T11:43:33.348 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40506.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40506.log.gz 2024-04-25T11:43:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60971.log 2024-04-25T11:43:33.349 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38556.log.gz 2024-04-25T11:43:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53683.log 2024-04-25T11:43:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73546.log 2024-04-25T11:43:33.352 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60971.log.gz 2024-04-25T11:43:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53683.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53683.log.gz 2024-04-25T11:43:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65639.log 2024-04-25T11:43:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73546.log.gz 2024-04-25T11:43:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91677.log 2024-04-25T11:43:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72519.log 2024-04-25T11:43:33.356 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65639.log.gz 2024-04-25T11:43:33.357 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91677.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91677.log.gz 2024-04-25T11:43:33.358 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5/var/log/ceph/ceph-client.admin.72519.log: --verbose -- /var/log/ceph/ceph-client.admin.61438.log 2024-04-25T11:43:33.358 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72519.log.gz 2024-04-25T11:43:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67913.log 2024-04-25T11:43:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48725.log 2024-04-25T11:43:33.360 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61438.log.gz 2024-04-25T11:43:33.362 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78196.log 2024-04-25T11:43:33.362 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67913.log.gz 2024-04-25T11:43:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54766.log 2024-04-25T11:43:33.363 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48725.log.gz 2024-04-25T11:43:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78196.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.50893.log 2024-04-25T11:43:33.364 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78196.log.gz 2024-04-25T11:43:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66269.log 2024-04-25T11:43:33.366 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54766.log.gz 2024-04-25T11:43:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54598.log 2024-04-25T11:43:33.367 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50893.log.gz 2024-04-25T11:43:33.368 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87838.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66269.log.gz 2024-04-25T11:43:33.368 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:33.370 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54598.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54598.log.gz 2024-04-25T11:43:33.370 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64392.log 2024-04-25T11:43:33.371 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.87838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38808.log 2024-04-25T11:43:33.371 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87838.log.gz 2024-04-25T11:43:33.373 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72403.log 2024-04-25T11:43:33.373 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64392.log.gz 2024-04-25T11:43:33.374 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81139.log 2024-04-25T11:43:33.374 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38808.log.gz 2024-04-25T11:43:33.375 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72403.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72403.log.gz 2024-04-25T11:43:33.375 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78563.log 2024-04-25T11:43:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81139.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81139.log.gz 2024-04-25T11:43:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84714.log 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75494.log 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78563.log.gz 2024-04-25T11:43:33.379 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.84714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76881.log 2024-04-25T11:43:33.379 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84714.log.gz 2024-04-25T11:43:33.381 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54947.log 2024-04-25T11:43:33.382 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76881.log: 92.3% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76881.log.gz 2024-04-25T11:43:33.382 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75494.log.gz 2024-04-25T11:43:33.382 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53365.log 2024-04-25T11:43:33.383 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54947.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54947.log.gz 2024-04-25T11:43:33.384 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70676.log 2024-04-25T11:43:33.385 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61189.log 2024-04-25T11:43:33.385 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53365.log.gz 2024-04-25T11:43:33.387 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70676.log: gzip 89.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.56768.log 2024-04-25T11:43:33.387 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70676.log.gz 2024-04-25T11:43:33.388 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61189.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61189.log.gz 2024-04-25T11:43:33.388 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85171.log 2024-04-25T11:43:33.389 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71991.log 2024-04-25T11:43:33.389 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56768.log.gz 2024-04-25T11:43:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50255.log 2024-04-25T11:43:33.391 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85171.log.gz 2024-04-25T11:43:33.393 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71991.log: /var/log/ceph/ceph-client.admin.50255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45274.log 2024-04-25T11:43:33.393 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.71991.log.gz 2024-04-25T11:43:33.393 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50255.log.gz 2024-04-25T11:43:33.394 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69961.log 2024-04-25T11:43:33.396 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45274.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91887.log 2024-04-25T11:43:33.396 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45274.log.gz 2024-04-25T11:43:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51836.log 2024-04-25T11:43:33.397 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69961.log.gz 2024-04-25T11:43:33.399 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.91887.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91887.log.gz 2024-04-25T11:43:33.399 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64846.log 2024-04-25T11:43:33.399 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51836.log.gz 2024-04-25T11:43:33.400 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50087.log 2024-04-25T11:43:33.401 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72833.log 2024-04-25T11:43:33.402 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64846.log.gz 2024-04-25T11:43:33.403 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81257.log 2024-04-25T11:43:33.403 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50087.log.gz 2024-04-25T11:43:33.404 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52443.log 2024-04-25T11:43:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81257.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81257.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42428.log 2024-04-25T11:43:33.406 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:33.406 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.72833.log.gz 2024-04-25T11:43:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52443.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52443.log.gz 2024-04-25T11:43:33.408 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.42428.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.80617.log 2024-04-25T11:43:33.408 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42428.log.gz 2024-04-25T11:43:33.409 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50756.log 2024-04-25T11:43:33.411 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.80617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85676.log 2024-04-25T11:43:33.411 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.80617.log.gz 2024-04-25T11:43:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50756.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50756.log.gz 2024-04-25T11:43:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74966.log 2024-04-25T11:43:33.413 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.85676.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85676.log.gz 2024-04-25T11:43:33.414 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57104.log 2024-04-25T11:43:33.415 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82553.log 2024-04-25T11:43:33.415 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74966.log.gz 2024-04-25T11:43:33.417 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57104.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57104.log.gz 2024-04-25T11:43:33.417 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.82553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47544.log 2024-04-25T11:43:33.419 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43049.log 2024-04-25T11:43:33.419 INFO:teuthology.orchestra.run.smithi019.stderr: 92.5%/var/log/ceph/ceph-client.admin.47544.log: -- replaced with /var/log/ceph/ceph-client.admin.82553.log.gz 2024-04-25T11:43:33.420 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47544.log.gz 2024-04-25T11:43:33.420 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42680.log 2024-04-25T11:43:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73799.log 2024-04-25T11:43:33.421 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43049.log.gz 2024-04-25T11:43:33.423 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46933.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42680.log.gz 2024-04-25T11:43:33.423 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:33.424 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73799.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73799.log.gz 2024-04-25T11:43:33.424 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63486.log 2024-04-25T11:43:33.426 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43889.log 2024-04-25T11:43:33.426 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63486.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46933.log.gz 2024-04-25T11:43:33.427 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63486.log.gz 2024-04-25T11:43:33.427 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51139.log 2024-04-25T11:43:33.428 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43889.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43889.log.gz 2024-04-25T11:43:33.428 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81691.log 2024-04-25T11:43:33.430 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86054.log 2024-04-25T11:43:33.430 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51139.log.gz 2024-04-25T11:43:33.431 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.81691.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81691.log.gz 2024-04-25T11:43:33.432 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58190.log 2024-04-25T11:43:33.433 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86054.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71399.log 2024-04-25T11:43:33.433 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86054.log.gz 2024-04-25T11:43:33.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78623.log 2024-04-25T11:43:33.434 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58190.log.gz 2024-04-25T11:43:33.436 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71399.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.71399.log.gz --verbose 2024-04-25T11:43:33.436 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.86905.log 2024-04-25T11:43:33.437 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78623.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67303.log 2024-04-25T11:43:33.437 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78623.log.gz 2024-04-25T11:43:33.438 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.86905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53054.log 2024-04-25T11:43:33.438 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86905.log.gz 2024-04-25T11:43:33.440 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67303.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67303.log.gz 2024-04-25T11:43:33.440 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53054.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53054.log.gz 2024-04-25T11:43:33.476 INFO:teuthology.orchestra.run.smithi019.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T11:43:33.542 INFO:teuthology.orchestra.run.smithi019.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T11:43:33.609 INFO:teuthology.orchestra.run.smithi019.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T11:43:33.651 INFO:teuthology.orchestra.run.smithi019.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T11:43:33.983 INFO:teuthology.orchestra.run.smithi019.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T11:43:33.986 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T11:43:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m1.946s 2024-04-25T11:43:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m5.821s 2024-04-25T11:43:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m3.173s 2024-04-25T11:43:33.988 INFO:tasks.ceph:Archiving logs... 2024-04-25T11:43:33.988 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/remote/smithi019/log 2024-04-25T11:43:33.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:43:35.848 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/remote/smithi169/log 2024-04-25T11:43:35.849 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:43:36.495 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:43:36.504 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:43:36.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:43:36.513 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:43:36.665 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:43:36.675 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:43:36.675 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:43:36.681 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:43:36.730 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:43:36.738 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:43:36.742 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:43:36.742 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T11:43:36.742 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 7 377 92 +675us[ +682us] +/- 42ms 2024-04-25T11:43:36.742 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 93 -506us[ -499us] +/- 81ms 2024-04-25T11:43:36.742 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -565us[ -565us] +/- 81ms 2024-04-25T11:43:36.743 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:43:36.749 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:43:36.749 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T11:43:36.749 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +613us[ +585us] +/- 43ms 2024-04-25T11:43:36.749 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -592us[ -620us] +/- 82ms 2024-04-25T11:43:36.749 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -606us[ -606us] +/- 81ms 2024-04-25T11:43:36.749 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:43:36.751 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:43:36.759 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:43:36.760 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:43:36.770 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:43:36.986 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:43:37.183 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:43:37.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:43:37.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:43:37.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:43:37.373 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:43:37.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:43:37.392 INFO:teuthology.task.internal:Duration was 1513.928571 seconds 2024-04-25T11:43:37.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:43:37.402 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:43:37.402 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:43:37.410 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:43:37.541 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:43:37.546 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:43:37.952 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:43:37.952 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:43:37.953 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:43:38.017 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:43:38.017 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:43:38.096 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:43:38.096 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:43:38.104 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:43:38.188 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:43:38.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:43:38.192 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:43:38.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:43:38.410 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:43:38.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:43:38.500 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:43:38.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:43:38.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:43:38.605 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:43:38.703 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-04-25T11:43:38.705 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-25T11:43:38.750 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:43:38.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:43:38.800 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:43:38.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:43:38.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:43:38.864 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:43:38.865 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/remote/smithi019 2024-04-25T11:43:38.865 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:43:38.968 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673135/remote/smithi169 2024-04-25T11:43:38.968 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:43:39.064 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:43:39.064 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:43:39.072 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:43:39.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:43:39.138 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:43:39.138 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:43:39.183 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:43:39.184 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:43:39.189 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:43:39.242 INFO:teuthology.orchestra.run.smithi019.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:43 /home/ubuntu/cephtest 2024-04-25T11:43:39.248 INFO:teuthology.orchestra.run.smithi169.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:43 /home/ubuntu/cephtest 2024-04-25T11:43:39.252 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:43:39.262 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:43:39.286 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/damage} duration: 1513.9285707473755 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T11:43:39.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:43:39.365 INFO:teuthology.run:pass